Author: mdoering@gbif.org
Comment: http 500 in API call already: http://api.gbif.org/v1/species/100155447/verbatim
Created: 2015-11-11 10:09:03.669
Updated: 2015-11-11 10:09:03.669
Author: mdoering@gbif.org
Created: 2015-11-11 11:56:38.864
Updated: 2015-11-11 11:56:38.864
This is a more serious problem than initially expected. The Kryo serialized verbatim data exists in different formats apparently as we did not pay attention to registered class order in the KryoFactory. The assigned integers need to be the same over time, either by keeping the registration order the same or by explicitly assigning a stable int.
As we have been indexing with different versions of the code the verbatim binary data exists in different formats and we need to **reindex all checklist data** in order to fix this properly!
Author: mdoering@gbif.org
Created: 2015-11-13 14:27:02.58
Updated: 2015-11-13 14:27:02.58
CLB modified to store JSON instead of kryo binary for the verbatim data. Kryo is limited to just intermediate processing now.
Sample JSON as stored:
{code:javascript}
{"extensions":{"http://rs.gbif.org/terms/1.0/VernacularName":[],"http://rs.tdwg.org/dwc/terms/ResourceRelationship":[],"http://rs.gbif.org/terms/1.0/Description":[{"http://purl.org/dc/terms/description":"herb,shrub,tree,vine","http://purl.org/dc/terms/type":"habit"}],"http://rs.gbif.org/terms/1.0/Distribution":[]},"http://rs.tdwg.org/dwc/terms/scientificName":"Equisetopsida C. Aghard","http://rs.tdwg.org/dwc/terms/taxonID":"73","http://purl.org/dc/terms/modified":"2011-02-21T12:19-0500","http://rs.tdwg.org/dwc/terms/taxonomicStatus":"accepted","http://rs.tdwg.org/dwc/terms/class":"Equisetopsida","http://rs.tdwg.org/dwc/terms/acceptedNameUsage":"Equisetopsida C. Aghard","http://purl.org/dc/terms/bibliographicCitation":"Brouillet et al. 2010+. Equisetopsida C. Aghard in VASCAN, the Database of Vascular Plants of Canada. http://data.canadensys.net/vascan/taxon/73 (consulted on 2015-09-24)","http://rs.tdwg.org/dwc/terms/taxonRank":"class","http://purl.org/dc/terms/references":"http://data.canadensys.net/vascan/taxon/73","http://rs.tdwg.org/dwc/terms/nameAccordingToID":"http://dx.doi.org/10.1111/j.1095-8339.2009.01002.x","http://rs.tdwg.org/dwc/terms/acceptedNameUsageID":"73","http://rs.tdwg.org/dwc/terms/nameAccordingTo":"Chase, M.W. & J.L. Reveal. 2009. A phylogenetic classification of land plants to accompany APG III. Botanical Journal of the Linnaen Society 161 (2): 122-127.","http://rs.tdwg.org/dwc/terms/scientificNameAuthorship":"C. Aghard"}
{code}
Author: mdoering@gbif.org
Created: 2015-11-13 14:28:42.913
Updated: 2015-11-13 14:28:42.913
A cli exposed tool created to migrate existing kryo data to json:
https://github.com/gbif/checklistbank/blob/master/checklistbank-cli/src/main/java/org/gbif/checklistbank/kryo/migrate/VerbatimUsageMigrator.java