Centos-Stream Broke

     Centos-Stream is broken at present.  I tried to install the current openssl on it and unfortunately sshd was built against 1.1.1b which has some different symbols than 1.1.1k so does not work.  Pulling it out also didn’t fix for reasons I do not understand so doing a fresh install.

 

OpenSSL and Kernel Upgrades mostly completed

     With the exception of Redhat based machines, all updates are completed.  I am going to have to build openssl for the Redhat machines because Redhat seems to be ignoring the openssl exploits so there may be some reboots later this evening of Centos7/8/stream, Fedora, and Scientific7.

Kernel, Openssl Upgrade / Reboots Tonight

     I do not normally do kernel updates mid-week, I prefer to wait until Friday on the off chance something goes horribly wrong, to provide the most time to recover before the business week.

     However, a serious vulnerability has been discovered in openssl and I’m going to have to reboot all the machines just to get any old copies of openssl out of memory so might as well do a kernel upgrade at the same time.

     Most machines will remain on openssl 1.1.1f but it will be a patched version that fixes the exploit.  The webserver with any luck will be on openssl 1.1.1k, this is just because it’s already on a self-compiled version of openssl to get the most current encryptions.

     Normally I would start this at 11pm but because of the seriousness of this exploit, I am going to proceed as soon as I have the current software in place on all the machines but some time after 5PM.  The downtime for the entire system should be less than 1/2 hour and any given machine not more than about ten minutes.

Maintenance This Weekend

     I may or may not do kernel upgrades Friday starting at 11PM, there are two new releases since the last but I’m waiting to see if this is stable or not.  If not will do otherwise I’m going to hold off on that until the following weekend.

     The other thing I will be doing over the weekend is upgrading openssl on various servers.  There have been security exploits found in the existing version and so far the vendors haven’t fix it but there will be a new release with it fixed and I already compile it myself on a number of servers because the vendors use old versions that don’t have all the encryption suites I want.

Mail / Upgrades

     After servers were rebooted, postfix did not start ont he mail server and I did not notice before I went to bed.  This affected the ability to send outgoing e-mail.

Kernel Upgrade Finished

     Kernel upgrades went smoothly.  My oldest son Carl, was visiting so I didn’t post this earlier, but only had two machines not boot up properly, mxlinux and the web server, and a reboot was all it too for them.  ALL NIS bindings and NFS mounts happened correctly the first time around so it feels like these aspects of Linux are finally maturing to the point where the are reliable.

Kernel Upgrade 3/23/21 11pm PST

     I am going to be upgrading kernels on most of our servers starting at 11PM.  If things go smoothly we should be done by 11:30PM however the last round we had a couple of machines that had a corrupted Grub configuration that had to be fixed so it took a bit longer.