User Meeting

     About 20 people showed for the user meeting which was good.

     It was fun meeting a couple of my co-workers from PNB/USWest/Qwest days.  They had changed so much I did not recognize them at first but then it has been 21 years since I left.

     Regarding a few projects in the works, I am working on getting ownCloud 9.1 working.  I had setup a server with the previous long term version of Ubuntu in order to get the older ownCloud, osTicket, and one customers website working again since the upgrade of our server broke these things but then a new ownCloud came out that did seem to work with PHP7 so I decided instead to try to get it operational.

     PHP7 is desirable to PHP5.6 because it speeds up execution by about 30%, that makes php based sites like WordPress sites, and many of the applications we have run faster.

     I want Eskimo to succeed and I know for us to succeed we have to help you succeed.  Any business website is heavily dependent upon Google searches for new traffic.  Google has gotten very good at finding sites with relevant content.  When it has a bunch to offer, and it almost always does, several things are very important after relevancy, and that is mobile friendliness, SSL encryption, and speed.  Sites that have all of these things get bumped to the top of the search results and that’s where you want to me and we can help you get there.

     Another service I am working on is providing streaming audio (and in theory video) via Icecast so that people interested in creating an Internet radio station or offering some streaming service, will have a way to do so.

     This mostly is an old interest of mine, Radio, being re-kindled in the era of modern technologies, a desire to understand how to current way of doing thing works from a technical and operational perspective, and as always a desire to give the average person a venue for getting their information, ideas, views, art, etc, out there.

CenturyLink DSL

     Our customers using CenturyLink circuits, the backhaul provider, Mammoth Communications, experienced an equipment failure during maintenance that caused the  loss of both primary and secondary BGP-4 routers.  This basically resulted in a loss of routing.  All equipment is back in service and coming up now.  Service should be fully restored in about twenty minutes.

Client Mail Server “mail.eskimo.com” Up

     I was unable to determine the reason mail.eskimo.com was returning an error code when trying to send.  There was no resource exhaustion I could identify, plenty of RAM, Disk, load was low, memory not exhausted.  Restarting postfix didn’t fix it and there didn’t appear anything bad in the queue.

     I opted for the Windows solution at this point and rebooted the server.  After the reboot it accepted and sent mail again.

     A post mortem analysis of the logs showed that the smtp daemon was unable to open /dev/null.  Not sure why it needs that but apparently something did go wrong in the operating system kernel. This is the first time I’ve ever seen a kernel issue on that version of Linux.

Client Mail server mail.eskimo.com Sending

     We are experiencing difficulties with our client mail server, “mail.eskimo.com”, returning 451 service unavailable when sending.

     I do not yet know what is causing this.  I have been unable to identify any issue with the software configuration, mailq, or exhausted system resources.

     I am still troubleshooting.

Playing with caching

     I’m experimenting with caching out of memory by using /tmp as a tmpfs file system and putting the cache directory there.  This has the disadvantage of clearing the cache when the machine boots but it also serves pages faster and reduces disk I/O somewhat.

     The unfortunate side effect of this is that the picture at the top will no longer change.

MySQL Fixed

     Finally got our MySQL woes fixed.  The issue is that 5.7.14 which is distributed in the Ubuntu 16.04.1 LTS repositories, is broken, at least in our environment with our workload.

     I installed MySQL 5.7.15, one point issue newer from the MySQL Community Server repository and it has performed flawlessly thus far.

     You can now use MySQL tools on the shell server to connect to and manipulate your databases.  The server that has the same version of PHP as the main web server is ubuntu.eskimo.com, which like our web server, is running Ubuntu 16.04.1 LTS.

MySQL 5.7

     I’ve written a small script that pings mysql once a minute and if it does not respond immediately it restarts it.  This way at least we won’t have lengthy outages while I figure it out.

MySQL 5.7 Problems – MariaDB

     The version of mysql provided with Ubuntu 16.04 is version 5.7, an upgrade from the previous version 5.6.

     MySQL 5.7 has proven to be unstable and locks up periodically.  I have spent several days trying to troubleshoot this, fixed a number of issues, and optimized the configuration.  Still it locks up and dies from time to time.

     I tried to upgrade to MariaDB 10.  An in place upgrade failed, and dumping the old database and sourcing the resulting sql file in MariaDB also failed.  It completes but the grant tables aren’t recognized which basically makes it useless.  I have yet to find a fix for this.  I apologize for this instability.

     If anyone can find a clean path to migrate from MySQL 5.7 to MariaDB 10.0, please let me know (e-mail: nanook@eskimo.com) or call me at 206-812-0051.