I felt a bit more detail was necessary... We are trying to get the virtual domain server upgraded to glibc and also the 2.2.10 kernel. The primary reason for this is effeciency, the main web server went from almost CPU saturated to approximately 10% CPU occupancy. Some complaints we had about ftp stopped. The news server went from about 50% occupancy to about 10%, the peak rate of news processing (when Sprint is actually in the mood to feed us) just about doubled. The disk occupancy also appeared to drop significantly. Clearly the newer kernel and libraries are a massive improvement and we want to propogate them to the rest of the machines here. When upgrading www, we ran into a problem with init, so Aaron attempted to avoid this by building a static init and sh first on www2 (the virtual domain server). Something went wrong and the static versions didn't get put in place. During the copies of the libraries the machine crashed, and it apparently crashed while there was a lot of pending disk I/O because fsck on the partitions found massive problems. After the fsck finished, we moved the old libraries back but the machine would still not complete the boot-up into multi-user mode. I ended up having to restore most of the root and /usr partition from tape. We don't want to cause another outage like this, so I have obtained another machine and we are just going to basically load that machine from scratch with all the newest stuff, and then when we get it operational replace the existing box and re-load it and use it to replace another machine, and so on until they're all done.