Locking mechanism for key generation unusably slow

12727
Reporter: omeyn
Assignee: omeyn
Type: Bug
Summary: Locking mechanism for key generation unusably slow
Description: When the fragmenter is running with 500 threads the fragment processor can't keep up because all of its threads are waiting for the zookeeper lock in order to generate new occurrence keys. The worst case should be devolving to single threaded behaviour (e.g. if all incoming fragments are new and for the same dataset) but currently it appears to stop in deadlock.
Priority: Blocker
Resolution: Fixed
Status: Closed
Created: 2013-02-11 10:48:04.907
Updated: 2013-12-17 15:16:53.258
Resolved: 2013-02-21 12:49:41.138

Author: omeyn@gbif.org
Comment: There were multiple compounding issues here - the locking has been replaced with an HBase implementation, the zookeeper counts have been temporarily patched to relieve contention when multiple threads attempt to update the same counter, and GBIFCOM-125 is about adding concurrency to message sends to relieve that contention.
Created: 2013-02-21 12:49:41.194
Updated: 2013-02-21 12:49:41.194