Decide on the future of portal/mendeley integration

14705
Reporter: omeyn
Assignee: bko
Type: Task
Summary: Decide on the future of portal/mendeley integration
Priority: Major
Resolution: Fixed
Status: Closed
Created: 2014-01-15 10:38:12.89
Updated: 2015-03-05 16:49:26.254
Resolved: 2015-03-05 16:49:26.221
        
Description: (Oliver: Because our current system is limited - both in terms of maintenance and fixing the paging buttons.)

(Below provided by Burke)
The Mendeley module originally developed for NPT Startup was meant to be light-weight and to only pull citations about a country according to country name tags. It avoids manipulating citations as internal Drupal content type on purpose to reduce cost on both implementation and maintenance. It mainly does 3 things:

1. Retrieve and assemble document metadata from GBIF Library by using Mendeley API;
2. Analyse retrieved documents according to defined sets of tags; the results are stored in the file system as JSON files;
3. Expose analysed results as HTML, JSON or array via internal function calls.

As the requirement for the portal is much more than it was scoped in the beginning, the Mendeley module appears to have limits that need to be lifted:

1. Limitation on the Mendeley API: Mendeley API doesn't expose all required metadata in the initial document call, for example the journal name, so it requires another call to collect enough metadata. This second call is dependent on the availability of their UUID, which appears to be affected by the change on the GBIF Library. These calls are subject to the rate limit (150 calls per hour for the first one, 500 calls for the second one), which adds up the complexity of retrieving. As a result, we see incomplete metadata (for example I put [pending journal name]) and sometimes one or all documents in a category would disappear, depending on how we organise documents via Mendeley web/desktop client.

2. The way it stores analysed results in the file system may limit some aspects of content management: 1) integration with the reference management (biblio module), which could improve the linking of citation with news items; 2) managing tags in Mendeley together with tags of news items; 3) exposing the analysed result via our formal GBIF API.

The first limitation is mainly on Mendeley API team's side. I have filed requests from their support channels and also emails to the lead developer. No follow-up since last September. I believe higher level engagement can easily improve it if they can see the value of addressing the need of GBIF. We'll need 1) the journal name in the initial call or higher call rate from our IP address; 2) to understand how/how long the change in the GBIF library will reflect by the API, including assignment of UUID.

The second limitation is depending on what we want to do with Mendeley documents in the near future. Do we want to make the documents linkable to our news items? Do we want to allow visitors to 'query' documents by string or tag from our portal instead of from Mendeley Web? How do we eventually expose Mendeley documents from GBIF API. Answering these can help decide whether we want to go for a heavier way of managing Mendeley documents.]]>
    

Author: bko@gbif.org
Comment: [~cvizitiu@gbif.org], [~mdoering@gbif.org], [~trobertson@gbif.org] and [~thirsch@gbif.org] I am bringing this issue to your attention so we could find time to scope the short term requirement before I can make major refinement of this module. Thanks!
Created: 2014-01-15 14:34:25.106
Updated: 2014-01-15 14:34:25.106

Author: thirsch@gbif.org
Comment: I agree we should clarify requirements on the Mendeley papers before embarking on the next major refinement. I will try to write up what I see as the minimum functionality I would like to see, with "nice to have" options on top of that, and we can decide the way forward on that basis. I'll get to it as soon as I can but somewhat overwhelmed at present so may not be until [~trobertson@gbif.org] is back. 
Created: 2014-01-15 14:48:56.724
Updated: 2014-01-15 14:48:56.724

Author: bko@gbif.org
Comment: Sure - it makes good sense that we wait until Tim R is back. :)
Created: 2014-01-15 14:51:37.512
Updated: 2014-01-15 14:51:37.512

Author: bko@gbif.org
Created: 2015-03-05 16:49:26.251
Updated: 2015-03-05 16:49:26.251
        
Mendeley synchronisation is back to live by using their 2.0 API with OAuth2 authentication.

http://www.gbif-dev.org/mendeley