Bad HBase long for Occurrence modified

12672
Reporter: mdoering
Assignee: omeyn
Type: Bug
Summary: Bad HBase long for Occurrence modified
Priority: Blocker
Resolution: Fixed
Status: Closed
Created: 2013-01-28 18:47:54.884
Updated: 2013-12-17 15:16:52.078
Resolved: 2013-02-21 12:47:11.199
        
Description: modified 3616444592354833000l from here
http://staging.gbif.org:8080/occurrence-ws/occurrence/32473603

corresponds to
Fri Jan 04 17:47:13 CET 114602537

That cant be right]]>
    

Author: lfrancke@gbif.org
Comment: Problem is that there is a verbatim String in HBase saying "2012-11-09 10:06:11" which gets converted to said Long which translates to that bad year. There should only be proper longs in that column. So somewhere in the processing something goes wrong.
Created: 2013-01-28 23:46:19.386
Updated: 2013-01-28 23:46:19.386

Author: omeyn@gbif.org
Comment: I've fixed the symptom, so now all imod are longs that produce a reasonable year.  The bad entries were all created on Nov 9 of last year, which roughly coincides with the last rollover. I'll look into the possible causes of the problem in the next days.
Created: 2013-01-31 09:14:04.585
Updated: 2013-01-31 09:14:04.585

Author: omeyn@gbif.org
Comment: Symptom now fixed in uat_occurrence as well, but still not fixed at source.
Created: 2013-02-11 10:14:48.64
Updated: 2013-02-11 10:14:48.64

Author: omeyn@gbif.org
Comment: Source problem was in the modified.q hive script used for processing raw occurrences imported from mysql. Fixed now.
Created: 2013-02-21 12:47:05.964
Updated: 2013-02-21 12:47:05.964