Create a metadata updater for standalone EML files as well as ones coming from DwC-A files

12213
Reporter: lfrancke
Assignee: mdoering
Type: NewFeature
Summary: Create a metadata updater for standalone EML files as well as ones coming from DwC-A files
Priority: Major
Resolution: Fixed
Status: Closed
Created: 2012-11-05 17:08:14.812
Updated: 2013-12-17 15:46:36.589
Resolved: 2013-12-17 15:31:11.246
        
Description: This tool needs to listen to messages sent by the downloader project (CRAWLER-38) and read the metadata from the filesystem and use it to update the registry. It should then send another message announcing that it has done so and probably which datasets have changed/been added/deleted.

After the archives have been downloaded and extracted, this service should take the (EML) metadata and simply push it to the registry via the webservices which in turn updates the stored mysql registry. In case the archive contains constituents metadata, this service must also make sure to synchronize all of the constituent datasets storing the datasetName/ID as a tag in the registry so our dataset uuid key can be linked to the ids used inside the archive.]]>
    

Author: mdoering@gbif.org
Comment: Only dwca is implemented. Do have the need to support EML only datasets? Do we need to update only metadata and not process the data? If so we need to modify our messaging and probably also the crawler coordinator
Created: 2012-11-21 09:35:57.765
Updated: 2012-11-21 09:35:57.765

Author: mdoering@gbif.org
Created: 2012-11-21 09:37:37.194
Updated: 2012-11-21 09:37:37.194
        
Closing, but only dwca based metadata updates are supported, see last comment. If EML only is indeed needed let's create another ticket:

http://code.google.com/p/gbif-crawler/source/detail?r=145
http://code.google.com/p/gbif-crawler/source/detail?r=148
http://code.google.com/p/gbif-crawler/source/detail?r=155
    

Author: lfrancke@gbif.org
Created: 2012-11-21 10:10:31.022
Updated: 2012-11-21 10:10:31.022
        
The EML part was not meant for EML only but if we want to do a metadata update for a DwC-A coming from an IPT we don't have to download the whole file again but can just use the EML Endpoint for that. I'm fine with implementing this in a different issue later.

Also this issue is not really done yet as not all review things have been addressed so I'd like to reopen it so it does not get forgotten.
    

Author: mdoering@gbif.org
Comment: I dont understand that. Downloading only the EML to kickoff the entire dwca processing does not seem to make sense. Either the dwca has changed or not, including the EML. Downloading only a standalone eml file seems to make only sense if we dont want to touch the archive and only deal with the metadata - which Im not hearing we want to do.
Created: 2012-11-21 11:53:16.612
Updated: 2012-11-21 11:53:16.612

Author: omeyn@gbif.org
Comment: Agreeing with Markus as saying this is done.
Created: 2013-12-17 15:31:11.278
Updated: 2013-12-17 15:31:11.278