Nobody at Isomedia has gotten back to me but the packet loss has resolved so I assume someone found what was wrong or if it was a DDoS attack, it’s ended.  At any rate it’s cleared up.

     We are seeing significant packet loss from various locations to our servers in the Bellevue CoLo facility.  I have determined the issue to be between Isomedia’s Seattle ring router and Bellevue core router and generated a trouble ticket.  We are not seeing heavy traffic to our facility but it may be another customer in the facility is under DoS attack or there maybe hardware or routing issues.  At any rate a ticket has been generated and we are waiting on a response.

     Centos-Stream is broken at present.  I tried to install the current openssl on it and unfortunately sshd was built against 1.1.1b which has some different symbols than 1.1.1k so does not work.  Pulling it out also didn’t fix for reasons I do not understand so doing a fresh install.

     With the exception of Redhat based machines, all updates are completed.  I am going to have to build openssl for the Redhat machines because Redhat seems to be ignoring the openssl exploits so there may be some reboots later this evening of Centos7/8/stream, Fedora, and Scientific7.

     I do not normally do kernel updates mid-week, I prefer to wait until Friday on the off chance something goes horribly wrong, to provide the most time to recover before the business week.

     However, a serious vulnerability has been discovered in openssl and I’m going to have to reboot all the machines just to get any old copies of openssl out of memory so might as well do a kernel upgrade at the same time.

     Most machines will remain on openssl 1.1.1f but it will be a patched version that fixes the exploit.  The webserver with any luck will be on openssl 1.1.1k, this is just because it’s already on a self-compiled version of openssl to get the most current encryptions.

     Normally I would start this at 11pm but because of the seriousness of this exploit, I am going to proceed as soon as I have the current software in place on all the machines but some time after 5PM.  The downtime for the entire system should be less than 1/2 hour and any given machine not more than about ten minutes.

Farcebook does not tell me WHICH of their policies I’ve violated, though they seem to think this is worth a 90-day ban on posting or commenting. All I was trying to do is provide some historical perspective to what is happening in Amerika today. This is an example of why I created https://friendica.eskimo.com and https://hubzilla.eskimo.com/ and why I do not censor on these.

     I may or may not do kernel upgrades Friday starting at 11PM, there are two new releases since the last but I’m waiting to see if this is stable or not.  If not will do otherwise I’m going to hold off on that until the following weekend.

     The other thing I will be doing over the weekend is upgrading openssl on various servers.  There have been security exploits found in the existing version and so far the vendors haven’t fix it but there will be a new release with it fixed and I already compile it myself on a number of servers because the vendors use old versions that don’t have all the encryption suites I want.

     The manjaro disk image resizing operation is completed.  It is back up and available.

     I am going to be taking Manjaro down for a while this afternoon.  I ran out of space during an upgrade and need to increase the size of disk image.

     After servers were rebooted, postfix did not start ont he mail server and I did not notice before I went to bed.  This affected the ability to send outgoing e-mail.