There was an issue with the pop-3 and imap mail server today. It stopped accepting connections for imap and ssh when I tested, some customers said it also stopped for smtp although it did not for me.
Since it was not accepting connections via ssh, I was unable to get into the server while it was in a failed state to see what was wrong. I had to force reset it and then nothing in the logs really offered a clue.
I upgraded the owncloud server to version 10.0.2, for some reason the backend authentication is not working so unless you specifically create an account in owncloud, it will not recognize your username and password. I am working on this. If you do create an account, use the same username and password so that when I get the backend authentication that calls the system authentication method working again, it will match.
Also many of the apps are gone or no longer supported in this version. I am researching where to find current versions to replace them.
Isofusion, previously Isomedia, where we have our equipment co-located, will be performing network maintenance on the mornings of June 15th and June 22nd between 12AM – 4AM. This will cause periods of high latency and packet loss during this maintenance window.
Mint is temporarily unavailable. I was trying to fix an issue with identd which was causing delays connecting to any site that uses tcp_wrappers, and ended up firewalling myself off from the machine so that I can’t login to it except as root non-graphically. It is quicker and easier for me to recover from a back up then to try to fix it with that interface.
I will be taking the web server down for approximately 1/2 hour to image the machine in case future restoration is necessary. It will be unavailable from 5:15AM until 5:45AM May 29th, 2017.
I will be moving vps1-vps5 private virtual servers to a different physical host for load balancing. You will experience a 20-30 minute outage of these servers during this operation.
The client mail server is back up. The copy took less time than I had anticipated owing to the faster disk subsystem. Mail is now on a RAID10 system with 4 Western Digital 7200RPM Black drives. These drives are each capable of sustained write speeds of around 120MB/s and the machine has adequate bandwidth to fully saturate all four of them continuously. This should make for a much nicer mail experience.
There will be some slowness for a couple of more hours as I move some other hosts. The network interface is 1GB and this tends to saturate it when I do these moves but it will help balance the load system wide to provide the best user experience possible.
The last hold out, ‘eskimo.com’, is back up and running!
I am going to be taking the client mail server down for approximately 1/2 hour to move it to different hardware both for load balancing purposes and to get it on the same physical machine as the mail spool files so that 1GB files don’t need to constantly be sent back and forth across the internal network. This will commence at 4:40AM on May 29th and should conclude by 5:10AM.
During this interval mail will still be received and spooled. It can also be accessed via shell mailers, but not webmail. Any outgoing mail sent from shell mailers should be spooled unless your client connects directly in which case you will get a rejection during this interval.
Everything is back up except for the old eskimo shell server. That will require another trip to the co-location facility and, after being there all night, and then working on it more from home, I’m too tired for another 44 mile round trip drive. So please use centos6 or some other server for today and I will get back down and restore it later tonight.
There is a possibility some e-mail may be rejected between about 3:30AM and around 6:30AM this morning when all of the mail servers were off line or inoperative because the spool was not mounted or NIS bound.
This evening I had to reboot a server that was the server that currently all of the files are served from because updates required it. It did not boot. It took me three hours to get it to boot. Something in new nvidia drivers conflicted with kernel-nfs-server used to serve those files and the system purged the latter including the /exports file that tells it what machines to make the files available to.
I have solved the video conflict by removing the nvidia drivers and using the Linux nouveau drivers instead. They are slower but it’s not like I am going to play video games on this machine. They are adequate for everything else.
I am now rebuilding the exports file by hand. Hope to have everything operational again in a couple of hours. I will focus on the most used services first.