Until recently, I could use the API call "http://a...
13668
Reporter: feedback bot
Assignee: mdoering
Type: Bug
Summary: Until recently, I could use the API call "http://a...
Resolution: Fixed
Status: Closed
Created: 2013-08-29 00:13:07.819
Updated: 2013-09-17 13:53:06.132
Resolved: 2013-09-17 13:53:06.109
Description: Until recently, I could use the API call "http://api.gbif.org/name_usage/Panthera%20tigris" to return all name usages with the exact canonical name of "Panthera tigris. This call no longer works (it returns a 404 error). Are there any plans to bring this call back, or to add an option to '/name_usage/search' to return exact canonical matches only?
Since I used that API call in the validator I described at http://gbif.blogspot.co.uk/2013/07/validating-scientific-names-with.html, it'll be offline until I can find an alternative. Sorry!
*Reporter*: Gaurav Vaidya
*E-mail*: [mailto:gaurav@ggvaidya.com]]]>
Author: mdoering@gbif.org
Created: 2013-08-29 11:37:10.735
Updated: 2013-08-29 11:37:10.735
We had removed that search cause it is basically covered by the name usage matching service. Shame on us we did not update our API wiki, I've removed it just now from the wiki.
The replacement call that should work for you goes like this. You can add &verbose=true to debug the results:
http://api.gbif.org/lookup/name_usage?strict=true&name=Panthera%20tigris
Well, thinking about it it actually will not work with that single call, cause it only matches against our backbone, not any other checklists. The solution that we usually recommend is to look at the related usages for the matched backbone usage, so it becomes a 2 step process unfortunately:
This gives you the list of all related usages for the above matched Panthera tigris:
http://api.gbif.org/name_usage/5219416/related
You can restrict that list to a single dataset, so its quicker and you dont need to filter results yourself. For example take the IUCN datasetKey:
http://api.gbif.org/name_usage/5219416/related?datasetKey=19491596-35ae-4a91-9a98-85cf505f1bd3
Will that work for you? We really like to keep the OpenRefine plugin working, its a super great tool!
Author: gaurav
Created: 2013-08-29 21:45:49.709
Updated: 2013-08-29 21:45:49.709
Markus: thanks for the speedy reply!
It looks like I can still use http://api.gbif.org/name_usage/search?q=%22Panthera+tigris%22 which does appear to search across all the checklists. Unfortunately, this API call doesn't have a "strict" mode. However, I could get all 183 names, then filter out all the names whose canonicalName (or scientificName) does not equal the original search string. That should give me the same results I got with the old API call.
Would that be okay? I worry that making two requests on the API call is going to slow everything down quite a bit, UNLESS you're planning to add more magic to the /related call -- matching synonyms across checklists, for instance (i.e. checklist A says "Felis tigris" is a junior synonym of "Panthera tigris", so add in all checklists which use "Felis tigris"). Which would be the better approach given your plans for the API?
Author: gaurav
Comment: Hi Markus! I tried using the API calls you suggested, since neither of them require paging. The first query (http://api.gbif.org/lookup/name_usage?strict=true&name=Canis+familiaris) works fine, but http://api.gbif.org/name_usage/5219200/related is currently down. I think that's because the rest of UAT is down? If so, TaxRefine should be back up when UAT is!
Created: 2013-08-30 08:02:41.578
Updated: 2013-08-30 08:02:41.578
Author: mdoering@gbif.org
Created: 2013-08-30 12:51:21.257
Updated: 2013-08-30 12:51:21.257
Probably, our tomcat died last night. The API call is working for me now: http://api.gbif.org/name_usage/5219200/related
And yes, I think this is much better than using our (also slower) solr search API which is more along the lines of a free text search and requires paging. And as you say the related method will potentially bring up also slightly misspelled names in other lists that we consider to actually be the same concept. And this can then be improved and extended over time without the need for a changing API call
Author: gaurav
Comment: Sounds good! TaxRefine looks like it's back up as well. Thanks for your help!
Created: 2013-08-30 22:10:27.081
Updated: 2013-08-30 22:10:27.081
Author: gaurav
Comment: I've just realized something: the old /name_uage/[binomial] query would search for the binomial across all checklists, but the new /lookup/name_usage only searches for names in GBIF Nub. This means that names like "Manumiella druggii" (listed in EOL at http://uat.gbif.org/species/123120900) would not be found by the new /lookup/name_usage at all (http://api.gbif.org/lookup/name_usage?strict=true&name=Manumiella%20druggii), although the full text search does find it: http://api.gbif.org/name_usage/search?q=Manumiella%20druggii -- hmm. I think I might have to use the full text search after all.
Created: 2013-08-30 22:20:49.746
Updated: 2013-08-30 22:20:49.746
Author: gaurav
Comment: I just wanted to check if there was a way to search for name usages across all checklists again without hitting the full-text search: we're working on a project which deals with old, fossil names, and PaleoDB is the only checklist with those kinds of names. With the old API call, we would find those records as well as records in more well-used checklists, but with the new system, if the name isn't in GBIF Nub, it won't show up at all. Any chance of getting the old API call back, or is it really dead for good? If so, I'll add a full text search in case I can't find records in GBIF Nub.
Created: 2013-09-04 21:10:08.249
Updated: 2013-09-04 21:10:08.249
Author: mdoering@gbif.org
Comment: we can consider adding it back, I will trigger a discussion tomorrow. But for many checklists all names exist also in the GBIF backbone as it was build using them, paleoDB for example should be entirely included in the nub. The second search call in case it is not in the nub sounds also good to me. Let me return tomorrow with a definite answer.
Created: 2013-09-04 22:40:54.016
Updated: 2013-09-04 22:40:54.016
Author: gaurav
Comment: Excellent -- thank you so much for your quick reply!
Created: 2013-09-05 02:04:39.923
Updated: 2013-09-05 02:04:39.923
Author: gaurav
Comment: One more thing: I've just realized that lookup doesn't work if you search for a name which exists in GBIF Nub multiple times, e.g. http://api.gbif.org/lookup/name_usage/?name=Ficus -- which is exactly the case that TaxRefine is best at dealing with! So I'll definitely need to fix this one way or another. On the plus side, this reinforces just how cool the old API call was, so maybe it's a good reason to bring it back?
Created: 2013-09-05 05:20:33.796
Updated: 2013-09-05 05:20:33.796
Author: mdoering@gbif.org
Created: 2013-09-05 09:48:01.477
Updated: 2013-09-05 09:48:01.477
If you want to allow more than a single match you need to set verbose to true:
http://api.gbif.org/lookup/name_usage/?name=Ficus&verbose=true
The idea of the lookup is that is unambigously matches to a single name usage in our nub. If several matches are equally likely you will get no immediate match, but a list of alternatives ordered by confidence instead. It would be good if users of the service could optionally limit the results be specifying a general taxonomic context, e.g. then only plant names should be matched set kingdom=plantae:
http://api.gbif.org/lookup/name_usage/?name=Ficus&kingdom=plantae&verbose=true
Author: mdoering@gbif.org
Comment: well, we could add it back in, but everyone is so busy that I would like to push that down the line a bit. Is it a total blocker for you?
Created: 2013-09-05 16:30:35.145
Updated: 2013-09-05 16:30:35.145
Author: gaurav
Comment: I definitely think that a canonical-name-across-all-checklists search is the best possible way to expose the checklists in the GBIF Portal. But thanks for listing some alternatives! I'll put both the &verbose option and full-text search as a fallback into TaxRefine and see how close that gets us.
Created: 2013-09-09 22:52:25.418
Updated: 2013-09-09 22:52:25.418
Author: gaurav
Comment: Okay, one downside with the full-text matter is that people try to resolve names like "moth" as often as they do "Ficus". In the latter case, the &verbose option gives us the right answers, but in the former, we go to a full text search and get ... a lot of matches. I'm limiting it to the first five hundred, assuming the server is sorting by relevance, but searching canonical names across all checklists would be the easiest way to sort this out. Tomorrow, I'll check to see if we're really missing out on many names because of this problem, or if GBIF Nub should have most/all of the names we need. Does your previous comment suggest that the canonical-name-across-all-checklists search might be back in at a later date?
Created: 2013-09-10 05:17:04.545
Updated: 2013-09-10 05:17:04.545
Author: mdoering@gbif.org
Created: 2013-09-15 17:36:07.889
Updated: 2013-09-17 09:33:35.74
I have added back an implementation to list all usages for a given canonical name:
https://code.google.com/p/gbif-common-resources/source/detail?r=1714
The new API call takes the name as a query parameter called "name" like this and is case insensitive:
http://apidev.gbif.org/species?name=PUMA%20conColor