What I’d hoped to do with mail didn’t work. I’ve got to do a bit more research before making another attempt.
Monthly Archives: October 2020
Emergency Reboots Tonight
We have had some issues with three different customers accessing e-mail. I was unable to replicate this until tonight. When it did fail for me the failures indicate an NFS problem with the new kernels, consequently on the NFS servers and mail clients I am going to revert to a previously known working kernel shortly. This will, unfortunately, interrupt everyone’s session.
Drive Taking Errors
One of our machines has a drive that is taking some errors. It completely passes the SMART internal diagnostics but the errors indicate problems finding sector headers which can happen if a machine isn’t shutdown properly say during a power outage it can clobber some sector headers.
While this can be fixed by a format, this drive is older than dirt (about eight years) so I’ve ordered a replacement. I am expecting to take this machine down Halloween evening for drive replacement. This is the only non-RAID drive on the machine but it’s used for booting. If it fails no data will be lost but the machine will be unable to boot until we replace it. I have a smaller drive which I could replace it with if the replacement does not arrive on time but I would rather replace it with a fresh drive and one with a larger cache should provide faster boot times.
Digital Ocean Spam / Virus
We have received a large number of spams containing a virus from Digital Ocean address spaces. We are receiving these exclusively from digital address space. For every one of these I have sent e-mail to their published abuse address, firstname.lastname@example.org and to their NOC at email@example.com.
I have yet to receive a single reply and as a consequence I initially started blocking individual addresses these came from. But still they continue. Now I am blocking entire address blocks as we receive this virus / spam. I am also sending this to blacklist maintainers as well as using it as a source to train our baysian filters.
At present the following address space is blocked for incoming mail:
126.96.36.199 REJECT Spam Digital Ocean
188.8.131.52 REJECT Spam Digital Ocean
184.108.40.206 REJECT Digital Ocean Virus
220.127.116.11 REJECT Digital Ocean Virus
18.104.22.168 REJECT Digital Ocean Virus
22.214.171.124 REJECT Digital Ocean Virus
126.96.36.199 REJECT Digital Ocean Virus
188.8.131.52 REJECT Digital Ocean Virus
184.108.40.206 REJECT Digital Ocean Virus
220.127.116.11/16 REJECT Digital Ocean Virus
18.104.22.168/17 REJECT Digital Ocean Virus
22.214.171.124/16 REJECT Digital Ocean Virus
126.96.36.199/16 REJECT Digital Ocean Virus
188.8.131.52/16 REJECT Digital Ocean Virus
184.108.40.206/16 REJECT Digital Ocean Virus
220.127.116.11/17 REJECT Digital Ocean Virus
18.104.22.168/17 REJECT Digital Ocean Virus
22.214.171.124/17 REJECT Digital Ocean Virus
I don’t like to do this but when a company will not respond to complaints and the spams are viral in nature, I am left with little choice. I have also submitted a copy to clam-av folks to generate a signature for this.
Unannounced Kernel Upgrade
I apologize for the unannounced kernel upgrade this morning but it was done rapidly because a security flaw was discovered in 5.8 and earlier kernels that I wanted to eliminate as rapidly as possible. We are now running 5.9 kernels.
This took two rounds last night because my first build of 5.9 was not correctly configured for our servers so I had to rebuild, re-install, and reboot again.
5.9 has a minor bug in the NFS code that is printing some warnings. It involves a race condition when a client attempts to open a file it doesn’t have permissions to open. Since the open would have failed anyway on the basis of permissions I do not believe this bug has any significant operational consequences other than making noise in the kernel logs.
I checked bugzilla and there is already a bug report filed though given the relatively low severity I doubt it will get rapid attention, but I’m added to the notification list so we will update again once fixed.