Letter to my Customers

Dear Eskimo North Customer,

– User Meeting –

I am tentively considering a user meeting October 29th at Spiros on Aurora but would like at least 20 people to RSVP to indicate an interest in attending. So if you would like this to happen please e-mail me and let me know you are interested.

– OwnCloud –

OwnClouod cloud services are back. We previously lost them due to an upgrade in PHP version which was incompatible. The system requirements only listed a minimum PHP version so I did not anticipate this disruption. Because I was unable to get the old software working on the newer platform, it was necessary to start over with version 9.1 which is in place now with even more Apps. There are also more ways to connect with it now.

You can get to it on the web at:

https://www.eskimo.com/owncloud/

Or go to our main web site:

https://www.eskimo.com/

… And select Web Apps -> OwnCloud.

– Shell Server Updates –

Fedora has been upgraded to Fedora 24, a massive improvement over Fedora 23 which was seriously broken. An attempt to update in place  failed miserably and so it was necessary to install from scratch. Getting that to work was a bit of a challange owing to the fact that it would not work with the stock virtual machine configuration but I did finally figure out how to make it go.

Ubuntu has been upgraded to Xenial, 16.04.1 LTS. Upgrade in place went smooth as it usually does with Ubuntu. We did have some issues with x2go because the x2go server incorporated version 1.2 of xrandr and Ubuntu 16.04 requires version 1.3 or later. I worked with the developers to get a pre-release installed that fixes this problem. If anyone else has this issue with a Ubuntu machine, there are instructions on our web site for fixing this.

– WordPress Sites –

If you are using Comet Cache on your WordPress site do not install the current upgrade being offered. It is broken. I installed it on our main site and it resulted in requests just hanging indefinitely. I will let you know when a working upgrade is available.

– Personal Stuff –

Now that I’ve had my blood sugar reasonably under control for a number of months, I am starting to see some healing of my nerves. I no longer have permanent numbness anywhere, only occasional intermittent numbness below my knees. My feet are no longer numb ever. Pain level is down considerably from what it was a few months ago. Most of the time no pain while I’m up and only occasionally mild pain when I am laying down for many hours. So I’m back to having a more normal amount of functional hours in the day to get things done.

– Projects –

I am still trying to find a replacement for osTicket as it is broken under PHP7 and development seems to have stalled on that project.

I am still working on on getting real time streaming for audio and video working so people can do things like Internet radio and television stations. My own interest in this is in radio and technology has changed so much since I was last involved in radio that there is a major learning curve, but that’s something I enjoy. Johnny Five Need Input…

– Customers –

I appreciate your business and I very much appreciate it when you tell your friends about us. Since we’re really a niche provider word of mouth is particularly important.

If there is anything I can do to make this service more useful or aesthetically appealing to you, please do let me know.

Robert Dinse
Nanook@Eskimo.Com

Owncloud is back, Version 9.1

     When we upgraded our web server a while back to Ubuntu 16.04.1 with PHP7, it broke some applications, our ticket system, Owncloud, a few other minor odds and ends.

     I was not able to do an upgrade of ownCloud from version 8 because I did not know that it would not work with PHP7, it only specified minimum version numbers.

     Because 9.1 is a fresh install I had to basically wipe everything and start from scratch.

     Please do test ownCloud.  You can get there two ways:

        Directly https://www.eskimo.com/owncloud/

        or https://www.eskimo.com  Select Web Apps Pull Down, Click on owncloud.

     Please send bug reports to nanook@eskimo.com.

     Thank you!

Web Server Broke

     Web server stopped serving pages this morning.  Not sure what went wrong but I was able to login to it okay and Apache still showed running.of life, will be stable.

     An update, it is not a problem with the web server but a problem with WordPress plugins.  For some reason the particular collection I use which got along fine until an update to Comet Cache now hangs.

     As a temporary fix I am going to remove Comet Cache from our main site.  This will make it slightly slower until an update to fix becomes available.

Power Outage

     There is presently a power outage at the co-location facility where our servers are housed and it is running on emergency generator power.  There is enough fuel to last through the night during which time we hope that commercial AC power is restored.  If not there may be problems.  Just letting you all know there is the potential for service interruption due to power if fuel runs out before power is restored.

DNS Failure Resolved

     Around 7pm tonight I found one of my computers was unable to reach our website.  After some investigation I found that DNS had failed, none of our servers responded to queries with information about our own servers although they still would resolve outside addresses okay.

     I chased the problem down to apparmor on our master DNS server which is a stealth DNS server and not publicly accessible.

     In the upgrade from Ubuntu 15.10 to Ubuntu 16.04LTS, Ubuntu renamed all of the DNS directories from /var/named/* to /var/bind/* but did not physically move the directories from their existing location.

     This resulted in our master DNS server being unable to access it’s conf file, log file, or any of the zone records.  This did not immediately cause problems but once the records in the slave servers expired, then they were unable to answer queries regarding those records.

     I corrected the paths in apparmor and now it is able to access it’s conf file.  The master records have been propagated to the slave servers which are now correctly answering queries.

Fedora Upgraded

Fedora 24     Fedora 24 is now available at fedora.eskimo.com.  I was unable to successfully do an online upgrade and so had to re-install fresh.  That was fraught with some difficulties as well.  The virt-manager in Ubuntu 16.04 selects a hardware profile for fedora that doesn’t work.  I was able to manually change it and install it.

     I have not yet re-installed all the applications we had or configured mail server on it, so sending mail from fedora probably will not work at this time.

     I am impressed with the improvement from Fedora 23 which in my opinion should never have been released.  It was just super ugly broken.  The dnf command used to install software (does not function) now mostly does function, much better than it did in Fedora 23, and they have fixed groups in yum extender to where they mostly work again.

     One thing yum extender does not do well is resolve dependencies, still issues there.  There appears to be no tool that really works automatically for this so when there are dependency issues it is pretty much necessary to figure it out by hand.  My opinion is they should get over the “not invented here” issue and take synaptic and modify it to work with rpms.

     If you use x2go to connect, you may get a small 800×600 screen size.  To fix this bring up a terminal and type xrandr -s HORZxVERT where HORZ is your screens horizontal resolution and VERT your screens vertical resolution in pixels.  The cause of this problem is that x2go’s xserver has xrandr version 1.2 and with Fedora 24 and Ubuntu 16.04, at least version 1.3 is required.  The command line version however is 1.5, so it will work fine.

Services Restored

     Services are restored on mail and shell servers.  There wasn’t really a problem with the shell server, rather it was an issue with the server that held the mail spool files.

     For some reason, after an upgrade the network interfaces all went away.  After a reboot it mysteriously revived.   It would appear there are bugs in the systemd startup files in 16.04 yet.

     While everything is currently backup, I am going to upgrade the operating system on the machine that holds users home directories so there will be another short interruption but once it is complete all operating system software will be current.

Servers

     I am unable to login to either of the physical hosts hosting the mail spool and home directories.  I will need to go to the co-location facility to reboot.  This can be anywhere from a 30-minute to 2-hour drive depending upon traffic.  I am headed that way to restore services as soon as I can.

Mail / Shellx

     We are having difficulties with the server that houses the mail spool files.  It is in the middle of an upgrade and it has interrupted some services.  The mail server is affected and thus web mail and customer smtp / imap are presently dysfunctional.

     We are working on completing this software upgrade which will restore services.  No mail will be lost, any incoming mail will be spooled and delivered when the server is back to being fully functional.  The mail spool itself is on a RAID10 partition so even loss of hardware will not lose files and they are backed up weekly in addition to the RAID.

     Because this is a physical host that is involved, I will need to drive down to the co-lo to reboot when this is finished and so will be unavailable for telephone support for several hours.

     Please use an alternate shell server during this time frame.  See Services->Shells->Servers for alternate servers.