Cluster slave cloudera-scm-agent doesn't come up properly on boot

15983
Reporter: omeyn
Type: Bug
Summary: Cluster slave cloudera-scm-agent doesn't come up properly on boot
Priority: Major
Resolution: Fixed
Status: Closed
Created: 2014-06-23 13:49:16.458
Updated: 2017-10-06 15:23:26.746
Resolved: 2017-10-06 15:23:26.724
        
Description: Appears to be some problem grabbing port 9000 on initial boot. Subsequent service restart works.

[23/Jun/2014 12:28:47 +0000] 6152 MainThread _cplogging   INFO     [23/Jun/2014:12:28:47] ENGINE Bus STARTING
[23/Jun/2014 12:28:47 +0000] 6152 MainThread _cplogging   INFO     [23/Jun/2014:12:28:47] ENGINE Started monitor thread '_TimeoutMonitor'.
[23/Jun/2014 12:28:47 +0000] 6152 HTTPServer Thread-2 _cplogging   ERROR    [23/Jun/2014:12:28:47] ENGINE Error in HTTP server: shutting down
Traceback (most recent call last):
  File "/usr/lib64/cmf/agent/build/env/lib/python2.6/site-packages/CherryPy-3.2.2-py2.6.egg/cherrypy/process/servers.py", line 187, in _start_http_thread
    self.httpserver.start()
  File "/usr/lib64/cmf/agent/build/env/lib/python2.6/site-packages/CherryPy-3.2.2-py2.6.egg/cherrypy/wsgiserver/wsgiserver2.py", line 1825, in start
    raise socket.error(msg)
error: No socket could be created on ('c4n4.gbif.org', 9000) -- [Errno 99] Cannot assign requested address

Initial boot:

[23/Jun/2014 12:28:47 +0000] 6152 HTTPServer Thread-2 _cplogging   INFO     [23/Jun/2014:12:28:47] ENGINE Bus STOPPING
[23/Jun/2014 12:28:47 +0000] 6152 HTTPServer Thread-2 _cplogging   INFO     [23/Jun/2014:12:28:47] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('c4n4.gbif.org', 9000)) already shut down
[23/Jun/2014 12:28:47 +0000] 6152 HTTPServer Thread-2 _cplogging   INFO     [23/Jun/2014:12:28:47] ENGINE Stopped thread '_TimeoutMonitor'.
[23/Jun/2014 12:28:47 +0000] 6152 HTTPServer Thread-2 _cplogging   INFO     [23/Jun/2014:12:28:47] ENGINE Bus STOPPED
[23/Jun/2014 12:28:47 +0000] 6152 HTTPServer Thread-2 _cplogging   INFO     [23/Jun/2014:12:28:47] ENGINE Bus EXITING
[23/Jun/2014 12:28:47 +0000] 6152 HTTPServer Thread-2 _cplogging   INFO     [23/Jun/2014:12:28:47] ENGINE Bus EXITED


subsequent restart:

[23/Jun/2014 12:36:18 +0000] 6871 MainThread _cplogging   INFO     [23/Jun/2014:12:36:18] ENGINE Bus STARTING
[23/Jun/2014 12:36:18 +0000] 6871 MainThread _cplogging   INFO     [23/Jun/2014:12:36:18] ENGINE Started monitor thread '_TimeoutMonitor'.
[23/Jun/2014 12:36:18 +0000] 6871 MainThread _cplogging   INFO     [23/Jun/2014:12:36:18] ENGINE Serving on c4n4.gbif.org:9000
[23/Jun/2014 12:36:18 +0000] 6871 MainThread _cplogging   INFO     [23/Jun/2014:12:36:18] ENGINE Bus STARTED
]]>
    

Author: mblissett
Comment: No longer relevant.
Created: 2017-10-06 15:23:26.743
Updated: 2017-10-06 15:23:26.743