clb-importer: investigate into why we see NodeId not in CLB yet
17715
Reporter: mdoering
Assignee: mdoering
Type: Bug
Summary: clb-importer: investigate into why we see NodeId not in CLB yet
Priority: Critical
Resolution: Fixed
Status: Resolved
Created: 2015-07-23 01:21:18.432
Updated: 2015-10-12 18:20:29.382
Resolved: 2015-10-12 18:20:29.344
Description: 11361:ERROR [2015-07-23 00:36:42,333+0200] [pool-10-thread-1] org.gbif.checklistbank.cli.common.RabbitBaseService: Failed to process dataset 234376ba-a94e-4ce2-9d48-c8214da43864
11362-java.lang.RuntimeException: java.lang.IllegalStateException: NodeId not in CLB yet: 8
11363- at org.gbif.checklistbank.index.NameUsageDocConverter.toObject(NameUsageDocConverter.java:137) ~[checklistbank-cli.jar:2.20]
11364- at org.gbif.checklistbank.index.NameUsageIndexServiceSolr.insertOrUpdate(NameUsageIndexServiceSolr.java:89) ~[checklistbank-cli.jar:2.20]
-----
Failed to process dataset 863f639b-8af9-412f-aab8-20904555af5a
11499-java.lang.RuntimeException: java.lang.IllegalStateException: NodeId not in CLB yet: 17
-----
Failed to process dataset ae4a076d-f805-4bd2-b26d-f2b66e1b75eb
10565-java.lang.RuntimeException: java.lang.IllegalStateException: NodeId not in CLB yet: 30
-----
Failed to process dataset 1e2f2229-045d-49c4-a9cd-48f5d66288de
10528-java.lang.RuntimeException: java.lang.IllegalStateException: NodeId not in CLB yet: 136
-----
Failed to process dataset fb42069b-ba92-4113-9a14-b92f0b70624b
4031-java.lang.RuntimeException: java.lang.IllegalStateException: NodeId not in CLB yet: 1
-----
Failed to process dataset 5abc9731-46cc-40a1-9c04-790cdc2a5ec9
1581-java.lang.RuntimeException: java.lang.IllegalStateException: NodeId not in CLB yet: 2
-----
Failed to process dataset 0f66de86-d95f-47d1-af8d-b126ac38857a
1276-java.lang.IllegalStateException: NodeId not in CLB yet: 1
]]>
Author: mdoering@gbif.org
Comment: Running the imports locally I cannot reproduce the issue
Created: 2015-10-12 14:25:31.333
Updated: 2015-10-12 14:25:31.333
Author: mdoering@gbif.org
Comment: imports run fine with new clb-cli 2.25 deployed
Created: 2015-10-12 16:24:34.14
Updated: 2015-10-12 16:24:34.14
Author: mdoering@gbif.org
Created: 2015-10-12 16:27:02.115
Updated: 2015-10-12 16:27:17.407
0f66de86-d95f-47d1-af8d-b126ac38857a still fails on bla6 and can be reproduced locally!
-----
9635 [main] INFO o.g.c.cli.importer.Importer - Start importing checklist 0f66de86-d95f-47d1-af8d-b126ac38857a
{noformat}
Metazoa [KINGDOM]
Arthropoda [PHYLUM]
Insecta [CLASS]
Hymenoptera [ORDER]
Formicidae [FAMILY]
Martialis [GENUS]
Martialis heureka [SPECIES]
Matialinae [SUBFAMILY]
*Martialis heureka [SPECIES]
Martialis [GENUS]
*Martialis heureka [SPECIES]
Matialinae [SUBFAMILY]
*Martialis heureka [SPECIES]
{noformat}
9791 [main] INFO o.g.c.cli.importer.Importer - First synced usage key for dataset 0f66de86-d95f-47d1-af8d-b126ac38857a is 100002086
9898 [main] ERROR o.g.c.cli.importer.Importer - Matialinae (nodeID=3) has unprocessed GENUS reference to nodeId 1
9899 [main] ERROR o.g.c.cli.importer.Importer - Failed to sync Matialinae taxonID '85F1D22363D0E75B8CB167F8FFB0539A.taxon' from dataset 0f66de86-d95f-47d1-af8d-b126ac38857a
Author: mdoering@gbif.org
Created: 2015-10-12 18:05:00.502
Updated: 2015-10-12 18:05:00.502
This is an error that shows up for a weird classification given.
There is an accepted subfamily with no classification which has a synonym which is a species and is classified to belong to a certain genus.
The clb normalizer then uses the genus as the parent for the subfamily - this needs fixed.
{noformat}
85F1D22363D0E75B8CB167F8FFB0539A.taxon subfamily Matialinae http://treatment.plazi.org/id/85F1D22363D0E75B8CB167F8FFB0539A
85F1D22363D0E75B8CB167F8FFB0539A.syn1 85F1D22363D0E75B8CB167F8FFB0539A.taxon Martialis species Martialis heureka synonym http://treatment.plazi.org/id/85F1D22363D0E75B8CB167F8FFB0539A
{noformat}
Author: mdoering@gbif.org
Comment: Fixed by making sure ranks line up: https://github.com/gbif/checklistbank/commit/ebec5e77c9a9a00a5a49764f60fcea9c16c95ccd
Created: 2015-10-12 18:20:29.38
Updated: 2015-10-12 18:20:29.38