Issue 14859

Operating system updates on GBIF servers

14859
Reporter: acenja
Assignee: acenja
Type: Task
Summary: Operating system updates on GBIF servers
Description: Installing operating system updates/bugfixes/security patches on the GBIF servers.
Priority: Major
Resolution: Done
Status: Closed
Created: 2014-01-21 12:03:05.147
Updated: 2016-02-15 14:21:25.585
Resolved: 2016-02-15 14:21:25.575
TimeOriginalEstimate: 144000
TimeEstimate: 34800
TimeSpent: 109200 />


Author: acenja
Created: 2014-01-21 12:16:38.411
Updated: 2014-01-21 12:31:38.925
        
Operating system updates are installed from the Centos and RHEL repositories, then the servers are rebooted. After rebooting, the applications running on each server are checked using the provided methods and/or URLs. Issues related to RPMs and/or particularities of the applications are fixed.

Servers running virtualized guests are updated at the same time as the guests.


    


Author: acenja
Comment: After the upgrade/reboot of b7g3; Sonar would not start. Debugging showed a connection to the DB was not established. Further debugging found that a local postgresql instance had been updated to 9.3 but not set to start at boot time (fixed). At the same time the postgres yum repository was left enabled by the person who installed it. Made no modification in the postgres93 repo configuration at this time.
Created: 2014-01-21 16:14:12.004
Updated: 2014-01-21 16:55:53.405


Author: acenja
Comment: On b14g3 (staging) the update process fails with RPM conflicts regarding postgres. Investigating. Fixed as separate task since it had nothing to do with the OS updates.
Created: 2014-01-22 11:16:20.889
Updated: 2014-01-22 12:07:18.876


Author: acenja
Comment: Will try to update b11g10 (cdhmanager) separately and then shut it down to allow faster updates on the rest of the Bla11 guests.
Created: 2014-01-22 13:32:12.008
Updated: 2014-01-22 13:32:12.008


Author: acenja
Created: 2014-01-23 11:49:44.117
Updated: 2014-01-23 15:37:31.843
        
Skipped servers:
- Jawa (updated 2 weeks ago)
- Pzb2 was skipped as it had too many NFS exports mounted for active crawling (b7g5, b14g2, b11g9, bla9)
- All Cxnx servers skipped, as usual
- Rancor (fresh install December 2013)
- Transit and the 2 iSCSI storage providers; too many exported mounts active (c4n12, c4n3, c4n11, jawa, c4n7, c4n9, c4n10, c4n2, c4n8, c4n1, c4n5, c4n4, c4n6)