Shellx, UUCP, Mail

     The shell server shellx, UUCP machine, and mail server are all going down to move to new hardware.  Shellx first, then UUCP, then mail.  An downtime of about 30 minutes is expected for each.

Mostly Working

     The web and ftp servers are now on the new machine.  I’ve mostly got it working.  The one thing that is still broken is the ability to use quota -v.  For some reason it core dumps on the host, and on the guests it says can’t connect with the quota daemon.  It may be a firewall issue.

     I’ve got virtual machines working with networking fully functional and was able to move a guest virtual machine from a CentOS 6.7 host machine to a Ubuntu 15.04 host machine.

     What I am going to do now is move the rest of virtual guess off of Iglulik and move the guts to a bigger case and replace the Intel stock useless cooler with an evo212 like I used on the new box and that will allow it to perform better.  Presently it gets too hot and throttles the CPU slowing things down.

     Then I will reload Iglulik with Ubuntu 15.04 like I have on the new machine.  I’ll add some disks and configure the important stuff raid10 as with the new box.  When it’s done, will do the same with virtual.  Then we’ll have a good modern OS on all the host machines and adequate cooling for them to run full tilt.

    At present everything is in place to provide Private Virtual Servers.  If you want one, talk to me.  I’ll have details on the website in a few days.

Another Operator Malfunction

     Trying to create a bridge on the new server for virtual machines, when I attempted to put it into service it failed because I had neglected to tie the Ethernet on the local machine to it, isolating it from the network.

     Another drive to the co-lo facility was necessary.  Sorry for the interruption.

Not Smooth

     Sorry for the downtime tonight.  To put it mildly things did not go as planned.  Needed to reboot the new server to remount some mount points and instead of booting it hung on a lightdm shutdown.  Before it was in actual duty I’d done this a few hundred times without incident.

     Also for some reason virtual machines aren’t working with static IP addresses on the new box.  But things should be up and running.  Sorry for this downtime.  It’s in the interest of making things faster, more redundant (all customer data is now on RAID except for MySQL and it will be once I get this virtual host routing issue figured out.

Web Outage

    I apologize for the web outage.  Files got copied but with the wrong ownership.  It’s a problem with old idmapd on CentOS 6.x not properly talking to new idmapd on Ubuntu.  I’ve temporarily worked around it by mounting files with nfsv3, but the ultimate fix will be getting rid of all the old kernels on the servers so that modern idmapd is present on all the nfs servers, then can go back to version 4.  Working on that.

Maintenance 10/3 22:00 – 10/4 02:00

     I am planning on taking the mail system down for approximately 15 minutes tomorrow evening and the web server about an hour in the time frame between 10pm-2am.

     I need to take the mail system down to move the spool to a new server and the web server down to move the virtual machine.

Major Construction Ahead

     We will be moving various machines over to a new Host machine as well as some file systems in order to reconfigure existing hosts to address both old operating systems and thermal issues.

     The operating system is Centos 6 presently, going to move to Ubuntu 15.04 and 15.10 when it gets out of beta and proves itself stable (I’ve learned that when Ubuntu is in Beta it is REALLY beta).  Ubuntu stays current where CentOS does not.  Even Centos7 is still on a 3.10 kernel when 4.2 is current.

     The other thing about Ubuntu is they provide a means to upgrade to the next major release in place rather than requiring a re-install as CentOS does so moving to this will allow us to stay current in the future with less disruption.

     Also moving major file systems like mail spool and home directories to RAID10 arrays for speed and redundancy.

     I will announce specific downtimes for specific servers and services as I prepare to move them.  I will do my best to minimize downtimes, particularly during high traffic times.

     The thermal issues are that the CPUs are getting too hot under load resulting in throttling of their speed.  The stock Intel coolers they came with are inadequate.

     The fix for this is to relocate the components into a larger case that will accommodate a more massive evo 212 cooler.  I’ve used one of these in the construction of our new i7-6700k platform that will be used for private virtual servers and stress tested it at 4.5Ghz with cpuburn and mprime (the Linux equivalent of prime95) and it is able to carry away the heat of the CPU running at that speed for hours.