11709
Reporter: mdoering
Assignee: omeyn
Type: Improvement
Summary: Show partial gathering date
Priority: Major
Resolution: Fixed
Status: Closed
Created: 2012-08-21 20:13:02.908
Updated: 2013-08-29 14:45:40.083
Resolved: 2013-01-25 14:49:52.402
Description: show only the right precision, e.g. only year or month and only time if it really existed
]]>
Author: kbraak@gbif.org
Comment: Please see the attached screenshots showing how we currently display occurrenceDate, and my suggestion for displaying a partial date from occurrenceYear, occurrenceMonth, occurrenceDay. Please note that where there is a year, an occurrenceDate will be produced nonetheless during processing having just year. Therefore, we only need to worry about displaying month and day on their own, in the absence of a year. Question, [~mdoering@gbif.org] can we show the month as a number (e.g. "1") or should we interpret it (e.g. "Jan")?
Created: 2013-01-10 12:33:55.609
Updated: 2013-01-10 12:33:55.609
Author: mdoering@gbif.org
Created: 2013-01-10 12:49:13.865
Updated: 2013-01-10 12:49:13.865
Ideally I would like to see a single way of showing the date and substitute parts from the atoms if available.
So if we want to use the current date format "Oct 7, 2006 12:00 AM" we should show the partial date exactly like that, simply omitting whats missing: "2006", "Oct, 2006" or "Oct 7, 2006"
Author: kbraak@gbif.org
Comment: These 3 screenshots demonstrate how the partial gathering date will be shown in keeping with the format "Oct 7, 2000". Work committed in: http://code.google.com/p/gbif-portal/source/detail?r=1297
Created: 2013-01-10 14:58:09.239
Updated: 2013-01-10 14:58:09.239
Author: kbraak@gbif.org
Created: 2013-01-14 16:29:24.287
Updated: 2013-01-14 16:29:24.287
[~mdoering@gbif.org][~omeyn@gbif.org] Just a quick question:
If an occurrence record only has a year, nothing else, the field 'occurrence date' still gets saved as a Java Date. For example, if the year was 1900, we apparently convert that into Date = "Jan 1, 1900 12:00:00 AM". This is consequently displayed in the portal as:
GATHERING DATE
Jan 1, 1900 12:00:00 AM
Is there any danger of claiming to know more about an occurrence record's date than initially supplied? If someone wanted to search for the occurrence of all bird species in a particular month, we could be misleading people. That is of course assuming our occurrence search uses the occurrence date field instead of year/month/day, which I'm not sure about, eh [~fmendez@gbif.org].
Author: mdoering@gbif.org
Created: 2013-01-14 16:38:54.116
Updated: 2013-01-14 16:38:54.116
We need to be careful with this, yes. We could hack around this in the portal and check for dates that are exactly midnight on new years eve and then show the pieces only. But we still have the full date in the json ws, in the downloads, all over.
If we use the date only for complete dates we still have similar issues with the time precision and we cant use the field for searches easily. [~omeyn@gbif.org] any ideas?
Author: fmendez@gbif.org
Comment: The occurrence search, in this moment,uses the fields year and month only. In this way the search supports cases like: search by month (even if the occurence doesn't have a year), search by year (even if the occurrence doesn't have a month) and search by year/month. The day field is not used since it can contains any string, seems that it is not being interpreted.
Created: 2013-01-15 08:55:17.916
Updated: 2013-01-15 08:55:17.916
Author: omeyn@gbif.org
Created: 2013-01-15 13:58:53.979
Updated: 2013-01-15 13:58:53.979
Right now all the data we're seeing is the old processing from the HIT - none of my new processing is visible - so [~kbraak@gbif.org] can hopefully comment better. In my new processing (which is still naive and therefore bad) I write year, month, and day into the corresponding fields if they're provided, and if a verbatim date string is provided I write that into the date field. If a date string is not provided I build it as year-month-day with this logic:
String date = "";
if (year) {
date = year;
if (month) {
date = date + "-" + month;
if (day) {
date = date + "-" + day;
}
}
}
As I said, naive and far from perfect. Doing it better is part of my processor improvements that I hope to start next week.
Author: mdoering@gbif.org
Comment: does year/month/day get populated if the date exists and can be understood? Otherwise our solr index might be missing data
Created: 2013-01-15 14:16:30.173
Updated: 2013-01-15 14:16:30.173
Author: kbraak@gbif.org
Created: 2013-01-17 12:13:50.408
Updated: 2013-01-17 12:13:50.408
The HIT is responsible for writing the raw_occurrence_record, which has year, month, day fields. Rollover processing writes occurrence_records which have the occurrence_date field. (occurrence_record also has year and month fields)
To clarify how the HIT handles the incoming occurrence date information: if the information arrives as year, month, day, the information is simply written as is to raw_occurrence_record. If the information arrives as a single date string, the string is 1st deconstructed assuming it is in standard format YYYY-MM-DD. If that didn't work, the string is tokenized, and its individual tokens evaluated to identify year, month, and day. It could be only year gets identified for example. The year, month, day are then written to raw_occurrence_record if they aren't null of course.
For actual code, please refer to: "parse(String date, Map map)" in http://code.google.com/p/gbif-indexingtoolkit/source/browse/branches/synchroniser-gbif-refactor/src/main/java/org/gbif/harvest/portal/synchronise/utils/text/DateParser.java
and "setDateCollected(String dateCollectedString, RawOccurrenceRecord ror, String dataFileDirectory)" in http://code.google.com/p/gbif-indexingtoolkit/source/browse/branches/synchroniser-gbif-refactor/src/main/java/org/gbif/harvest/portal/synchronise/GBIFPortalSynchroniser.java
Author: kbraak@gbif.org
Comment: [~omeyn@gbif.org], you offered to further investigate how the occurrence_record.occurrence_date is produced from raw_occurrence_record.year/month/day Thanks.
Created: 2013-01-18 16:37:49.306
Updated: 2013-01-18 16:37:49.306
Author: kbraak@gbif.org
Comment: There is another problem with partial dates during rollover processing reported in OCC-183
Created: 2013-01-24 17:14:20.994
Updated: 2013-01-24 17:14:20.994
Author: kbraak@gbif.org
Created: 2013-01-25 14:49:52.425
Updated: 2013-01-25 14:49:52.425
The partial occurrence date is showing fine on the occurrence page.
The question of how the occurrence_record.occurrence_date are actually written has been dealt with in a new occurrence store issue: OCC-187
Closing issue
