The server maintenance is completed for the night. The new drivers would not compile against 5.14.0 so we are back to 5.13.12 for now.
Monthly Archives: August 2021
Eskimo Maintenance Work 11PM – 1AM Aug 30-31 Pacific Daylight Time
I am going to be booting one of the physical servers (Iglulik) into a new 5.14 kernel tonight and this will take the web server, Mint, Debian, and Ubuntu shell servers down for a short interval, approximately ten minutes, EXCEPT the web server.
The web server will be down longer because the upgrade to 5.14 requires rebuilding the iomemory drivers for the flash drive used for our MariaDB database.
This will mean also that https://friendica.eskimo.com/, https://hubzilla.eskimo.com/, and https://nextcloud.eskimo.com/.
Other web based services such as Squirrelmail will also be unavailable during this time frame. Home directories for all shell servers will be temporarily unavailable during the reboot but should be available within about five minutes. You can access e-mail via the shell servers or via IMAP/SMTP while work on the web server is under way.
5.14 has a new copyless driver for the Intel E1000 Ethernet chipset used on this server and it is my hope that this driver will fix the hardware offloading issues with the existing driver that force us to disable hardware offloading.
Debian Upgrades – Connecting with Old Putty
I am receiving complaints from people unable to connect to servers that they used to be able to using putty on versions of Windows prior to Windows 10.
The reason you are unable to connect is that as exploits are found in various encryption schemes, those schemes are retired in favor of newer more secure encryption algorithms.
All versions of Windows prior to Windows 10, with the exception of Windows 8.1, are past end of life. If you are using one of these, you will not be receiving updates that update security issues like encryption vulnerabilities and thus you will not be able to connect with your old encryption methods.
Putty is an aftermarket ssh client for older version of Windows. If you have Windows 10, you do not need putty as it comes with openssh, you just need to install it as one of Windows features (it is not installed by default).
If you are using 8.1, you should upgrade your putty installation to the July 21, 2021 release. If you are using an older version of Windows you can try this as well but older versions may not have the necessary libraries for it to work properly.
Kernel Upgrades Completed
System kernel upgrades are completed. All services are back online.
Debian came up first time properly so I ended up not doing any systemd debugging today. Whether this was a fluke or they fixed something with upgrades since last kernel upgrade I do not know. If it messed up again I will debug at that time.
Kernel Upgrade and Debian Friday Aug 20th 11PM-Midnight
We will be upgrading all servers to 5.11.12 on Friday at 11pm. This will take about an hour.
I will also be adding some debugging code in Debian to try to gather more information with respect to problems with the systemd configuration so it may be down for a longer period while I go through a few tweak / reboot / examine debug log cycles.
HVAC failure at Co-Location Facility
I have received notice that there is a failure of some HVAC equipment at the Isomedia facility where our equipment is located.
Unlike Integra, they have people monitoring these things and have already dispatched personal to repair.
I have temperature monitors in the servers and so far they haven’t shown any increase in temperature. Not too worried about the servers since I have a very large amount of fan in each cabinet except for the old UltraSparc machines used for dial-up radius authentication still. They will tolerate 104F max, right now it is 82F so still a good margin of safety.
Debian Bullseye Up
Debian Bullseye is UP. The systemd configuration still is not correct so it is very slow to boot but it does eventually start everything just not in the correct order and only after various timeouts. I will figure out systemd eventually. I keep hoping Poettering will give up and go away but he seems determined to ruin Linux.
Debian Upgrade Not Going Well
After upgrade to bulleys, debian will not boot with NIS enabled. NIS (Network Information Services) is required in order for you to be able to login. This seems to be yet another Poettering systemd induced snafu. I am looking for a solution.
Debian Upgrade In Progress
I am in the process of upgrading debian.eskimo.com from Buster to Bullseye.
During this process ssh sessions may be terminated and new connections will not be accepted until completion. At least one reboot will be required.
System Maintenance Completed
System maintenance completed. There were minor issues but no major problems, everything is up and running.