The upgrade appears mostly to have been a success.  However, on the web server it broke NFS version 4 so I’m currently running version 3.  I know this has to be something broken in my configuration because it is working properly on another 16.04.1 server here.

     I had hoped it would resolve the MySQL hanging and dying thing but it did not even though it replaced it with a newer version so I will continue to troubleshoot that.  I have some ideas on how to narrow it down but it will take some time as I have to wait for it to fail on it’s own as I do not yet know what induces a failure.

     The upgrade of the operating system on the web and ftp server has mostly been completed.  It us up and running on 16.04.1 LTS.

     However, it broke nfs version 4 in a way that cost all NFS mounted partitions to be not to allow any read or write which make it rather useless.  For now I’ve reverted back to NFSv3 until I can figure out what is wrong.

     The web server is in the process of being upgraded from Ubuntu 15.10, which is no longer receiving security updates, to Ubuntu 16.04.1 LTS, which will be supported for some time.

     I hope that it will be stable this time.  When 16.04 LTS came out I upgraded and found that it was not stable, but I’ve been running several other servers on 16.04.1 as well as a couple of workstations and thus far they have all been stable so I am optimistic that the stability issues have been fixed in the ‘.1’ release.

     If not, I have made an image of the machine prior to upgrading so we can revert if necessary.

     There may be an interval of several hours where MySQL and PHP are not working and our website will be down during this time.  This is because of the way that Ubuntu updates, first it downloads all the necessary package, then it unpacks them over the existing packages, THEN it configures the new packages.  Between the time of unpacking and configuring there may be some time where things do not function.

     If you have seen this article, Linux Bug leaves USA Today and other top sites vulnerable to serious TCP/IP hijacking attacks, just want you to know not our site, and if you have a site hosted here and provide an SSL certificate then not yours either.

     This attack only works if a site is not encrypted.  Protect your site from malware injection by encrypting it.  If you provide us with an SSL certificate your site will be encrypted, there is no charge for this by us, you probably will pay a fee to the certificate issuer although there are ways to get free certificates.

     We have also taken measures to protect you from another newly discovered attack that takes advantage of a flaw in Intel hardware having to do with the copy on write function for shared memory pages.

     I am going to take the web server down for about 1/2 hour tonight do back it up prior to starting an update to Ubuntu 16.04.1.  I’ve got 16.04.1 working on several other servers and so far now it has been stable.

     I am also having issues with mysql on this machine that I need to investigate further.  It occasionally is going off into lala land where it is still running but does not respond.

     Tekexplore has published an exploit that uses a hardware fault in Intel processors in combination with kvm, a daemon that finds memory pages with the same content on multiple virtual machines and merges them into one to save memory on the host machine marking that page copy on write so that in theory when it is written a copy is made and the original left alone so it doesn’t affect other virtual machines sharing that page.

     Apparently it is possible to alter a single bit in a page and not trip the copy on write and by altering just a single bit in ssh keys, it is possible to weaken ssh considerably allowing it to be compromised.

     I have turned off kvm on our machines.  It is of limited use in our environment because our servers are not overcommitted and thus the only thing freed up shared pages does is provide more memory for disk caching.  I monitored the logs for some time and found that this was infrequent and so we were getting little performance benefit.

     The mysql server on the web server died this morning with no apparent cause.  I restarted it, it ran for several hours, then died again.

     The second time it died, I ran myisamchk’s on all of the tables, there were quite a few with errors that it fixed.

     Hopefully that will take care of it, else I will reboot the server if it continues.

     Because neither owncloud, nor newly developed nextcloud, nor osTicket in spite of months of promises, will run under PHP 7, and I’ve also got a customers website consisting of 1910 PHP code that totally breaks, I am setting up a new server with two year old software specifically for these applications.

     The new server will be called “antique.eskimo.com”, it will not be something you can login to, it will just be a web server setup like the existing machine except that it will be running PHP 5.6 and Ubuntu 14.04 LTS so that it can run antique PHP code.  Being an LTS release it will still have security updates so will not represent a security threat.

     In addition to being able to run old code, it will also provide a back door to get to our ticket system or webmail when the main web server is inoperable as it will be on a totally different physical host machine.

     Previously, we had a cloud service called owncloud.

     When we upgraded to PHP7, it broke the version of owncloud we had because it was not PHP7 ready.

     I upgraded to version 9 but was unable to get authentication to work.

     Then, owncloud split, the original developer and a number of others left to start nextcloud.

     Nextcloud appears to be much better documented and the documentation includes imap authentication which is what we were using.

     So I am attempting to install nextcloud as a replacement for owncloud.  At this point all of the owncloud clients will also work with next cloud.  They have agreed to keep the api the same so the same clients should continue to work.

     The ubuntu.eskimo.com shell server has been upgraded to the latest release, 16.04.1 LTS.  This brings with it an upgrade of PHP to version 7.08, the same as on our web server.

     Therefore, if you are doing any PHP development and need to test scripts, this machine would be the one to use.

     This server is also equipped with a broad range of applications and has more memory than most of the others so it is a good machine to use.

     If there are some applications you need which are not present, please contact support@eskimo.com.