Webmail, imap, and pop were unavailable from 7:15PM to approximately 8:30PM.
I restarted mail to test a new kernel before I restart the large servers tonight. For some reason dovecot automatic start failed and I did not realize it at the time.
I got a call just before 8:30, checked, and found it was not running and restarted and it started fine so no idea what went wrong earlier.
An update; I discovered what went wrong earlier is that at some point Ubuntu changed where dovecot / systemd expected the PID file to be and it required manually editing the unit file. I did that so it should be okay in future restarts.
Saturday morning just after midnight Pacific time, I will be rebooting servers into a new kernel. If this kernel behaves properly we will remain on 5.7.4, if not I will need to reboot again to 5.6.0. 5.7.0-5.7.2 has not functioned properly as NFS server.
The reboot did not in and of itself fix the problem but everything restarted did point me at the source so I could fix it. It is completed, everything should be functional again.
I have to reboot a machine because the NIS subsystem, which is how authentication information is pushed to all machines, is not working properly on this machine.
This will briefly make e-mail unavailable as well as the shell servers centos7, centos8, scientific7, and fedora.
I will be rebooting all servers shortly after 1AM to load a new 5.7.2 kernel, the 5.7.0 kernel was slightly unstable in nfs server applications.
The shell server, mxlinux.eskimo.com, is again available.
It has been upgraded from 18.3 to 19.2. So far 19.2 appears to be MUCH more solid and bug free relative to 18.3. I have yet to find any actual bugs in the implementation (but a hell of a lot of operator errors).
I have not installed many user applications yet, so many mailers, news readers, editors, etc, are not there yet but will be soon. I think you may want to consider this as a good alternative to debian and ubuntu now. It is based off of the Debian 10 code base.
The mail system is basically functional and alpine and mailx are installed but most other mailers are not yet installed. The pan news reader is installed, most others are not installed.
Iglulik spontaneously rebooted about 3AM, this is the second time it has done this and Ice, the server which has /mail spool, has also spontaneously booted once. It appears that 5.7 has some stability issues when NFS file systems are exported. The machines not exporting file systems have been completely stable.
I know that the Linux community is doing a lot of work on NFS right now, which is good because NFS in Linux has been buggy forever but not in ways that crash machines but in ways that when a server goes away and comes back the clients do not always recover properly. There was a lot of work in the 5.7 kernel and they just came out with a new nfs-kernel package. So hopefully they’ll get this resolved soon.
So I’ve compiled 5.7.1 now on Iglulik. If it does not spontaneously boot again before tonight I will be rebooting in the early AM to load the new kernel.
I am taking mxlinux down to upgrade from MxLinux 18.3 to MxLinux 19.2. Because of a change in the debian code base upon which MxLinux is based and because they’ve bastardized Debian beyond the point where normal upgrade procedures can be used, a complete re-install is necessary so it may be down for a few days.
Mail issue was a problem with Dovecot. It did not open the SASL socket for some reason. Restarted and now is okay.
We are having some problems with the mail sub-system that I am still struggling to understand. It is responding slowly and refusing service to some hosts while permitting it to others and I have not yet been able to determine why.