I need to reboot the host machines which will cause a brief interruption in all services lasting 5-10 minutes if all goes well.  If all does not go well and I need to drive to the co-location facility, it could be as long as an hour but I have not had any failure to boots since the upgrade to Ubuntu 17.10 so I do not expect difficulties.

     I am taking the Linux Mint shell server, mint.eskimo.com, down for an upgrade from Serena 18.1 to Sonya 18.2.  Serena has been particularly problematic with NIS always failing to bind because the ordering of applications starting is not right in systemd, in particular systemd does not wait for portmap to start before trying to start ypbind.  Early versions of Ubuntu with systemd had similar issues, I am hoping 18.2 will resolve this.

     Because Mint online upgrades generally do not work well, particularly when third party repositories are concerned, and because there has been a change of hardware architecture from an i7-2600 to an i7-6850k which supports some additional instructions, I am doing a fresh install which will require recompiling some third party apps.  So it will likely be down most of this evening.

     Please consider using debian or ubuntu as alternatives, like Mint they are debian or debian derived but generally work better.  Hopefully Sonya will function better.

    I have installed Libre Office Suite on all shell servers except “eskimo.com” which hasn’t had any new software available in two decades.  On those machines on which it is already present, I installed missing elements so that it is complete.

     I’ve also installed codeblocks, an Integrated Development Environment suitable for C development.  I’m a command line person myself but if an IDE is your preference, this is now available to you on all shell servers except “eskimo.com”.

     OpenSuse.eskimo.com has been upgraded to leap 42.3.  An attempt at an online upgrade failed, it did everything except apparently write the boot block correctly because upon rebooting it could not even start grub.

     So I did a fresh install of 42.3, which is probably just as well as the upgrade from 42.1 to 42.2, while successful, was less than 100% clean.

     I am still installing applications so if there is anything particularly important, please create a ticket using https://www.eskimo.com/ and then the Support drop down menu, select “Tickets”.

     The upgrade of OpenSuse went well right up to the point where a reboot was required and at that point locked up the whole qemu/kvm system, some issue with the address still in use like it did not properly free it up when the virtual host went down.

     Anyway, it required a reboot of iglulik which hosts a number of virtual machines and so took many things out of service for about two minutes around 9pm.

     At this point I am going to do a fresh install of opensuse so it will be out of service for a while.

     The shell server opensuse.eskimo.com is presently in the process of being upgraded from Leap 42.2 to 42.3.  Because of this it may not be fully functional, especially third party software like x2go which may need to be recompiled, or the nx-libs upon which it depends, afterward.

    Everything should be up and running now.  Our web server is now 22% faster making it now faster than 97% of the Internet according to Pingdom, before it was faster than 94%.

     Many other services have been sped up as well but to a lesser degree.  Most of the shell servers are also 22% faster now.

     I hope things will be stable but it will take a couple of weeks to be sure.  When CPU speed is increased, the voltage also needs to be increased.  No chart or other guide as to how much is provided but the rule of thumb is the absolute minimum necessary for stability at a given clock speed since heat is very directly related to voltage.  I ran stress tests for an hour, I’d really like to run them for weeks but can’t do so on in service machines.

I have finished work on the host hardware but have not brought up all the virtual machines yet.  I will post a completion notice here when done.

     I will be taking the server that hosts users home directories and a number of virtual machines down this evening in order to make some adjustments to the bios settings.  This will require multiple adjustments, boots, benchmarking and readjustments in order to achieve the maximum possible performance.  I will start this work around 11pm and it may take several hours to finish.