I found the Manjaro XFCE spin installer works, including with manual partitioning.  Sad Mate doesn’t but not surprised.  It will probably take 4-5 hours to get everything configured but should be working again later this evening.

     Manjaro may be down for a while, possibly until the next distro comes out.  The existing installation was hurt by installing gcc-13.1 which corrupted some of the library files.

     I restored from backups but the Manjaro update system won’t let you update a system that is too old, so that didn’t work.

     I tried to install from the old ISO, that didn’t work, for the same reason.

     I then downloaded the CURRENT Mate distro, and installed from that.  It would not work if I manually partitioned.  So I let Mate partition the way it wanted to.

     After installing I tried to login and it would not accept my password.  So I booted with the install disk again and tried to mount the root partition but it said file system was not recognized.

     I’m trying some different desktop spins now on the off chance that only the Mate spin is broken but if if it’s all of them then it will be down until a non-broken ISO becomes available.

     Manjaro failed to boot with the new kernel and I chased the issue to my recent installation of gcc-13.1 on this machine.  It changed some system libraries in a way that broke them.

     I am restoring from backup, then will bring the machine up to date, then will install the new kernel, all others are complete.

     We will be doing a kernel upgrade of all servers tonight starting at 11pm.  If all goes well it should conclude by 11:30, if not and we have to make a trip to the co-location facility, then 12:30.  No individual service with the exception of yacy should be down for more than about ten minutes.

     Igloo, one of the physical hosts that among other things hosts the mail spool, crashed earlier this afternoon.

     I was compiling a new version of GCC (13.1) using eight cores and it just got too hot.  I stress test these machines thoroughly when I initially turn them up but over time CPUs suffer something known as chip rot, essentially even though electrons are 1/2000th the mass of protons, when electrons move one way atoms occasionally move the other ever so slightly and over time this creep causes features to be less well defined and maximum stable CPU speeds become slightly less.  I have already down clocked slightly ice and iglulik, this machine is the only one still at 4.4 GHz, I will need to bump it down to 4.3 or 4.2 the next time I am at the co-lo.

     In the meantime I am completing the compilations on this machine using only four cores so it generates less heat.

     I am also working on a new 10 core / 20 thread server using an i9-10900x that will have 256GB of RAM to handle some of the more CPU and memory intensive workload.  Unlike the other machines which have two memory channels, this one has four.  It will also have everything, even the root partition, on RAID.

I thought this was pretty cool: Eskimo-Aleut languages

     Kernel upgrades are completed.  Sorry it took longer to get some of the virtual private servers up than normal.  One of the physical servers failed to come back up into the normal run level, it booted but systemd hung (which unfortunately it sometimes does though this is not frequent on the physical servers).

     I will be performing a kernel upgrade tonight starting at 11pm, expecting to finish by 11:30pm.  Expected downtime for most services less than ten minutes, yacy.eskimo.com may take as much as 45 minutes owing to the rebuilding of the database after reboot.

     This upgrade will affect all of Eskimo North’s services, shell accounts, UUCP, e-mail, web hosting, mail lists, and virtual private servers.

     It will also affect our free services, https://friendica.eskimo.com/, https://hubzilla.eskimo.com/, https://nextcloud.eskimo.com/, and https://yacy.eskimo.com/.

     The fax machine has been repaired so you can send me faxes now if that is your preferred method of communicating.

     Kernel upgrades are all completed except for Manjaro (did not have time to finish compiling), it will be done later this morning.