The old shell server is seriously broken.  It will not boot.  Gets data exception errors before you can even finish typing the boot command.  Brought it back home to try to determine what.  I’m thinking a bad DIMM or CPU is most likely.  I have no spares of either so if it is one of the two it is either going to return a single CPU box or with less memory.

     If anyone has any SS-10 hardware they want to get rid of or M-Bus CPUs or 64MB DIMMS these things used, I’d be happy to take it off your hand.

     This machine is the NIS master so I can not change passwords or enable or disable accounts until I get it operational.

     The mail server is up.  It was locked up hard, not even the Linux magic sys request key had any effect.  Power cycling it brought it back up.  Nothing in the logs useful to provide any post mortem information.

     This is the only Intel machine still running an I7-2600 rather than an i7-6700k.  I am planning  on upgrading it soon.  The i7-2600 only supports 32gb of RAM where the i7-6700k supports 64gb.  You can not give Linux too much RAM, it uses any excess for buffer and that improves performance.  There are some newer Intel CPUs available and I have not decided which I am going to go with yet.

     The old eskimo.com shell server is not responding.  I will need to make another trip down to the co-lo facility to fix that.  It is responding to pings but nothing else.  This is a wedged state the SunOS kernel occasionally gets into.

     The physical machine that hosts the mail spool has crashed.  I am on the way to the co-location facility to repair.  This is unrelated to the work I was doing.

     I will be rebooting the web server at approximately 2AM in order to make kernel and glibc updates effective.  This will take approximately five minutes.  Reboot on this machine is somewhat slow because of the use of caching of file systems in memory which provides faster web response but takes some time to flush upon a reboot.

     I replaced the 100 mbit switch by a new Linkeyes gigabit switch.  This is a made-in-china brand like TP-Link but unfortunately LinkSys no longer makes rack mount equipment, Cisco bought them out and discontinued those products and I’m not a big fan of Crisco equipment.  Hopefully this will last longer than the two TP-Link switches did.

     I am going to replace the 100mbit switch that I put in place temporarily when our last gigabit switch failed with a new gigabit switch made by a different manufacturer tonight.

     There will be very brief (under 2 seconds) interruptions in connectivity that may cause a momentary pause in videos being watched or keystroke echoes.  There is a remote possibility that I may have to reboot the mail server if it does not automatically recognize the speed change.  In the past it has always recognized increases but not decreases so this probably will not be necessary but it is a possibility.

     This is why I do not use the Yahoo group for status when we have an outage and instead ask people to check here if our own website is down.

Hello,

Your message to the EskimoNorthUsers group was not approved.
The owner of the group controls the content posted to it and has the
right to approve or reject messages accordingly.

In this case, your message was automatically rejected because the
moderator didn’t approve it within 14 days. We do this to provide a
high quality of service for our users.

A complete copy of your message has been attached for your
convenience.

Thank you for choosing Yahoo Groups

Regards,

Yahoo Groups Customer Care

     The new replacement Gigabit switch, a TP-Link, failed less than a month after being placed into service.  Suffice it to say that I’m going to try to find something other than another TP-Link to replace it with.

     Bought a different brand, it will be here in a few days.  With any luck I’ll be able to replace it Saturday night.

     No process has run up against the memory limits I put in place on Iglulik and memory usage has remained stable since the kernel upgrade to 4.8.

     This makes it likely that the issue was a bug in the 4.4 kernel on that machine.  It was the only machine I had running that particular kernel.

     It is possible that whatever went wrong before just hasn’t happened again yet, but if it does hopefully the limits in place will identify the process, prevent the virtual machines being killed by the Linux OOM killer, and provide some diagnostic information.

    Centos6 ships with php 5.3, a rather antique version and not the version we use on our web server.  I installed php70 from remi repositories but it did not work as advertised, upgrade php did not remove all pieces of 5.3, so we ended up with a mix of php 5.3 and php 7.0 that did not play together well and caused errors on every update.

     I deleted ALL php from this machine today and am in the process of re-installing PHP 7.0 so temporarily there will be many missing modules until this process is completed.