One of our mail servers somehow corrupted it’s root file system in such a manner that it thinks it is out of free space even though it’s at 61%. The fsck program does not fix whatever is wrong so I am restoring from a virtual machine image backup.
There is quite a lot of mail stuck in spool which I hope to copy from the broken image to the backup image.
If all goes smoothly no mail will be lost and only that stuck in spool will be delayed. If that spool is not salvageable than it is possible for some incoming mail to be lost but I do not expect that to be the case.
The NetBeans 8.2 Integrated Development Environment has been installed on all shell servers except the old “eskimo.com” SunOS machine.
Most will have it as a graphical link on the Gnome or Mate desktop but it can also be found under Applications -> Programming on these machines.
Scientific Linux is back together better than ever.
Now it boasts a 4.8 kernel in spite of being generally an antique, it also boasts having Java 8 and NetBeans 8.2 installed with all the packages.
The Scientific Linux 6 shell server, “scientific“, currently does not have a functional desktop installed. This is because software sub-systems from two different sources created conflicts that made updates uninstallable.
To resolve this it was necessary to uninstall a great number of software packages, perform the upgrades, then re-install. This is still in progress.
I made a change to the web server today that causes PHP sessions to be stored in a tempfs file system in memory. This should speed up applications that use sessions like WordPress and PHPBB and owncloud.
It was more important that I get the guests and my workstation updated as they are open to public attack. The host machines really don’t provide any services so are less exposed. Upgrades were slow in part because the Dyn.com DoS attack impeded the ability to resolve ubuntu’s server names. It may happen tomorrow if things go well.
DynDNS is currently undergoing a massive DDOS attack. Sites that outsource their DNS to DynDNS are currently difficult to reach.
I have found that Google’s name servers tend to cache data forever and so even when the primary name server is down, Google’s name servers will usually return good data.
I had quite a bit of mail backing up in the mail queues of mx1 and mx2 owing to the inability to resolve DNS of destination sites. I have added a Google public name server to these machines and re-queued the mail and much of it is making it out now.
I had to reboot all the shell servers and the web server to install new kernels to fix a security problem. I will be rebooting the physical host machines around 9pm tomorrow. This will momentarily interrupt all services but is necessary to eliminate a security exploit.
The flakey Ethernet cable has been replaced with one that is solid.
There will be a period of roughly 3-10 seconds where you will not have access to your home directories. This will cause whatever is trying to access these files to hang for this period.
This is necessary to replace a defective Ethernet cable that ties the NFS server hosting your /home directories to the switch. The existing cable sometimes does not make connection if you wiggle it. This is not a good thing for such a critical server and so needs to be addressed sooner rather than later.
I will plug the new cable into another port in the Ethernet switch, route it to the server, then unplug the existing cable and plug the new one in. That will probably take a second but since it’s a different port the MAC address on the switch end changes and it will take the computer and the switch a few seconds to figure that out via ARP protocol.