clb importer fails for ITIS - pro parte synonyms supported?
16407
Reporter: mdoering
Assignee: mdoering
Type: Bug
Summary: clb importer fails for ITIS - pro parte synonyms supported?
Priority: Critical
Resolution: Fixed
Status: Closed
Created: 2014-09-16 12:05:31.218
Updated: 2014-10-02 17:31:56.547
Resolved: 2014-10-02 17:31:56.473
Description: There seems to be several synonym relations for a single node, needs to be verified. For pro parte synonyms several accepted taxa must be allowed. Could be that the code does not support pro parte synonyms yet?
Start importing checklist 2cffcaed-aa4f-4f02-8961-a14df9b3ac0a
org.neo4j.graphdb.NotFoundException: More than one relationship[SYNONYM_OF, OUTGOING] found for Node[358979]
at org.neo4j.kernel.impl.core.NodeProxy.getSingleRelationship(NodeProxy.java:232) ~[checklistbank-cli.jar:2.9-SNAPSHOT]
at org.gbif.checklistbank.neo.NeoMapper.getRelatedTaxonKey(NeoMapper.java:289) ~[checklistbank-cli.jar:2.9-SNAPSHOT]
at org.gbif.checklistbank.neo.NeoMapper.read(NeoMapper.java:270) ~[checklistbank-cli.jar:2.9-SNAPSHOT]
at org.gbif.checklistbank.cli.importer.Importer.buildClbNameUsage(Importer.java:167) ~[checklistbank-cli.jar:2.9-SNAPSHOT]
at org.gbif.checklistbank.cli.importer.Importer.syncDataset(Importer.java:103) ~[checklistbank-cli.jar:2.9-SNAPSHOT]
at org.gbif.checklistbank.cli.importer.Importer.run(Importer.java:82) ~[checklistbank-cli.jar:2.9-SNAPSHOT]
at org.gbif.checklistbank.cli.importer.ImporterService.handleMessage(ImporterService.java:71) ~[checklistbank-cli.jar:2.9-SNAPSHOT]
]]>
Author: mdoering@gbif.org
Created: 2014-09-18 16:47:26.725
Updated: 2014-09-18 16:47:26.725
Pro parte fixed and normalizerIT test added for pro parte synonyms:
https://github.com/gbif/checklistbank/commit/eefec5b7c8cbe1f6a7e17810b45fdea0df3192d6
Author: mdoering@gbif.org
Created: 2014-09-29 22:43:32.986
Updated: 2014-09-29 22:43:32.986
Pro parte is fixed but now we see foreign key constraint issues in postgres with extensions:
-----
ERROR [2014-09-29 18:05:24,284+0200] [pool-8-thread-3] org.gbif.checklistbank.cli.importer.Importer: Failed to sync usage 202423 from dataset 2cffcaed-aa4f-4f02-8961-a14df9b3ac0a
org.apache.ibatis.exceptions.PersistenceException:
### Error committing transaction. Cause: org.postgresql.util.PSQLException: ERROR: insert or update on table "vernacular_name" violates foreign key constraint "vernacular_name_usage_fk_fkey"
Detail: Key (usage_fk)=(113768) is not present in table "name_usage".
### Cause: org.postgresql.util.PSQLException: ERROR: insert or update on table "vernacular_name" violates foreign key constraint "vernacular_name_usage_fk_fkey"
Detail: Key (usage_fk)=(113768) is not present in table "name_usage".
at org.apache.ibatis.exceptions.ExceptionFactory.wrapException(ExceptionFactory.java:26) ~[checklistbank-cli.jar:2.9-SNAPSHOT]
-----
Looking at that taxonID (Animalia kingdom) we see it is persisted:
http://api.gbif-dev.org/v1/species/140576241
But all kingdoms usages have incertae sedis with taxonKey=0 (backbone???) as the kingdom given, sth wrong:
http://api.gbif-dev.org/v1/species/root/2cffcaed-aa4f-4f02-8961-a14df9b3ac0a
Author: mdoering@gbif.org
Created: 2014-09-30 13:52:43.093
Updated: 2014-10-01 15:35:52.517
trouble comes from usages that point to themselves in the higher classification keys. For example the plantae kingdom record has a kingdomKey pointing to itself. When replacing the neo ids with the newly generated clb ones that key is not yet mapped and the trouble starts. Somehow we do not seem to throw an exception if keys cannot be mapped
Adressed in https://github.com/gbif/checklistbank/commit/c6df2745ebbf308e29e7e81f6e65809e14ba0b63