Build script that generates the hive table for downloads

15048
Reporter: omeyn
Assignee: fmendez
Type: SubTask
Summary: Build script that generates the hive table for downloads
Priority: Major
Resolution: Fixed
Status: Resolved
Created: 2014-02-12 15:54:48.57
Updated: 2014-02-19 11:45:40.789
Resolved: 2014-02-19 11:45:40.743
        
Description: Done when:
- any environment's occurrence and occurrence_hdfs tables exist and are mapped properly to hbase
- the header file written to a "known" place in hdfs for use in creating the final download
- all terms in dc, dwc, and gbif (as .simpleName() ) are represented in v_, and interp columns match the json exposed fields
- key must come first. after that we loop through dc, then dwc, then gbif terms using the ordering provided by .values()
- build meta.xml for download as part of the process]]>
    

Author: mdoering@gbif.org
Comment: Please specify a ";" as the multiValueDelimiter in the meta.xml for the interpreted term GbifTerm.issue
Created: 2014-02-17 12:35:22.07
Updated: 2014-02-17 12:35:22.07

Author: mdoering@gbif.org
Comment: GbifTerm contains quite a few checklist only terms that are not worth adding to the downloads (i.e. all values below comment "species profile checklist extension")
Created: 2014-02-17 12:56:49.55
Updated: 2014-02-17 12:56:49.55

Author: fmendez@gbif.org
Created: 2014-02-19 11:45:40.787
Updated: 2014-02-19 11:45:40.787
        
meta.xml: will be generated on the fly by the download workflow
hdfs table creation will be performed on each environment