Add Metrics to the Crawler Coordinator

11963
Reporter: lfrancke
Assignee: lfrancke
Type: Improvement
Summary: Add Metrics to the Crawler Coordinator
Priority: Minor
Resolution: Fixed
Status: Closed
Created: 2012-10-01 15:42:49.382
Updated: 2013-12-17 15:46:38.56
Resolved: 2012-10-11 14:28:49.489
        
Description: Add some metrics to the part of the crawler coordinator as it is implemented right now. These are the metrics I think might make sense:

* Number of successful crawl "schedules"
* Number of unsuccessful crawl "schedules"
* Failure reasons: Number of already scheduled datasets, number of datasets without valid endpoints
* Number of crawl requests per protocol: DiGIR, BioCASe, TAPIR
* The last successful UUID that was enqueued
* A timer which counts the number of requests per hour and how long they usually take (I expect the latter to be very short and mostly uninteresting but if it goes up it is an indication of something bad going on)

These will be added using Yammer's Metrics library.]]>
    

Author: lfrancke@gbif.org
Comment: I'll commit this if there are no further comments.
Created: 2012-10-10 12:38:24.376
Updated: 2012-10-10 12:38:24.376

Author: trobertson@gbif.org
Comment: These look sensible +1
Created: 2012-10-10 15:46:05.376
Updated: 2012-10-10 15:46:05.376