The kernel upgrades were successful and relatively trouble free.  We are now operating on Linux 6.1.3, which was released yesterday.

     Planning to upgrade to a 6.1.2 kernel Friday 1/6 at 11pm Pacific Time.  The present kernel, 6.0.15 has a nasty bug where it locks hard, no kernel dump, no auto reboot, no magic sys request key, only power cycling the affected machine restores service.  The inability to get a kernel dump makes this bug particularly difficult to troubleshoot.  Since this bug has persisted from 6.0.12, I’m going to try a 6.1 kernel and hope for better.

     This will result in outages between 11pm-11:30pm of all services lasting about 5-10 minutes each EXCEPT for yacy which takes close to 45 minutes to rebuild it’s database after every reboot.

     This will affect all of Eskimo North’s paid services such as mail, web hosting, virtual private servers, shell accounts, etc, as well as our free services including https://nextcloud.eskimo.com/, https://friendica.eskimo.com/, https://hubzilla.eskimo.com/, and as I mentioned, https://yacy.eskimo.com/.

      The machine that hosts many of the virtual private servers, ice, is down.  I am headed down to the co-lo to find out why.

     My facsimile machine ran out of paper today and the spare paper I had all got wet from the tree limb through my roof, so it will be nonoperational until I can get more paper.  Please e-mail or call instead.

     We are going to attempt to change the default version of PHP to PHP 8.0.26 tonight.  The last time I attempted this there were too many applications that did not work but since that time several new releases have come out so we are going to try again.

     Just a reminder, if your own apps do not work well with PHP 8.0 you can override the PHP version: https://www.eskimo.com/support/override-php-version/

     Within half an hour of booting 6.0.15, the server which holds home directories and much shared disk space spontaneously booted.  While still not good, this is better than the behavior of 6.0.12 which locked up and did not reboot.

    Two customers reported mail failing to gmail, with the bounces indicating improper SPF records, however, I changed the SPF record along with the IP of the mail server at the same time AND pushed out the changes manually to all of our name servers.  Gmail was caching old data.  I have since tested and Google is again accepting our mail.

     I will be doing kernel upgrades tonight that were originally scheduled for last night.  I was able to recover my vehicle that my wife abandoned on the way home from work the night before last and most of the ice and snow has melted off the roads now.  I prefer doing them on Friday nights and especially don’t really want to be doing them Christmas evening but the existing kernel has a flaw that has so far resulted in the lock-up of four machines so really needs to be replaced as soon as possible.

     The photo gallery function of Friendica has been fixed.

     Talking to the provider of the infrastructure we use for dial-up, it is not yet set in stone, but we may be able to continue to offer this service into 2024, one of their larger customers is considering renewing their contract and if this happens they will continue into 2024, I will let you know as soon as I know.

     I still can’t get to my car and by the looks of our street it won’t happen tonight so the kernel upgrade is going to be put-off until at least Saturday night.  Hopefully the machines will make it another day without seizing because if they do I can’t get to the co-lo to fix at the moment.

     Kernel upgrades may or may not happen tonight.  I let my wife take my car to work last night because it is better in snow and ice than hers.  Coming home, she slipped a tiny bit, panic’d, parked it and walked the rest of the way.  Now, the road in front of my house is so icy I can not walk up it to get my car and so unless it melts enough that I can do that before 11pm, it will not happen tonight.