Centos6 is down for maintenance. It is being moved to a newer faster physical host machine. This should be completed in about ten minutes. All of the other shell servers and services are available during this move. This is to clear the existing machine for an operating system and hardware upgrade that will require the existing physical host being removed from service for a while.
The mail server is now replaced and upgraded but things will be somewhat slow still for a couple of days.
Two things, one it is on a RAID but the RAID hasn’t had time to sync all the disks before I put it in service, it is still doing that.
Two is that I’m still moving virtual machines around and disk I/O is becoming fast enough that it can saturate a gigabit interface. So between these two things it will be somewhat slower than optimal for a few more days but should be better than what it was.
Since the mail server failed several weeks ago and I moved the services to another machine, I have had some occasional complaints of slow response but until yesterday I have not been able to reproduce them.
Yesterday it got slow on me and I found what was happening was people hitting the servers with brute force password guessing from botnets so heavily that it was overloading the machine.
We can reasonably defend against a small number of IP addresses but there are currently very large botnets. This is largely because of the newly discovered security flaws in the Internet of things, televisions with a default password, routers with the same, and other devices.
Several virtual machines on this box were hit with these attacks simultaneously and it just exhausted resources.
I have the new mail server hardware up and running and I’m working on getting all of the necessary software installed. It will have 4x as much memory and 50% more CPU than the old machine. I’ve been burning in the hardware (running CPU intensive test software while overclocked to find any weak hardware) and it’s run this way for nine days no problem. So now I am working on getting the RAID partitions configured and x2go so I can access remotely then I will move it down to the co-location facility and move the mail services to it.