I will be taking various shell servers down for imaging (making a backup of the entire machine in case it fails or gets corrupted) for about 1/2 hour each. I will only take one server down at a time so if one is unavailable the others will remain available during that time frame.
I plan on taking the web server down for about twenty minutes tonight to image the web server so that recent improvements are not lost in the event the machine is damaged.
We have a new alternative to ownCloud available called “NextCloud”. It is available via https://www.eskimo.com/nextcloud, or by using the Web-Apps drop down menu.
The development of ownCloud seems to have stopped at a point where it is slow and buggy, and limited in features. NextCloud has far more apps and capabilities. Not all apps are configured yet, some require fairly extensive configuration so not all apps are working yet, still the working set is far larger than owncloud.
One nice thing about the new postfix is that the error messages are much more explicit. The old version used to say “unknown host” if the inverse DNS was missing, if the forward DNS pointed to a different IP address, or if the EHLO command was wrong. The new one says things like, “EHLO must be a fully qualified domain name”, etc, spells it out much better. Hopefully this will translate into having to spend less time helping clueless system administrators configure their mail servers.
The gray listing has substantially reduced the amount of spam getting through. Not zero percent but about 5% of what made it through previously and most of that gets snagged by spamassassin.
The Bayesian filters still need your help to train them recognize spam. Spam should be “bounced” (not forwarded) to “email@example.com”. You can do this with pine or any other mailer that has a bounce function. The bounce keeps the original originating address where forwarding changes the originating address to your address in many mailers.
I logged in today at about 2PM to find my INBOX full of spam in spite of just having replaced the mail servers with the newest version of everything to address this.
Examining the spam I found it was all coming through mx2, and upon further examination, I found the reason for this was that I had forgot to allow smtp and smtps through the firewall on mx1. That is fixed so both servers are functioning now.
I examined the spam and found that it was all being properly scored by spam assassin but they have found ways to craft their message to not look like spam and be scored low.
And it’s not that the new filters are not working at all, there were about 20 spams in my INBOX, but 180 had been properly sent to spam. They just are sending such a huge volume that even 1-out-of-9 getting through is too much.
Frustrating this is, but being on the newest operating system with all the newest software provided some more options. I have now implemented gray listing. What this does is when a message for a person arrives from an unknown location, it sends a temporary failure result back to the sending site and refuses delivery.
RFC compliant mail systems will wait a brief period and then retry, but much spam software is not RFC compliant and just blasts it out once and moves on.
I started with a short interval because I do not want to delay legitimate e-mail unnecessarily but a longer one than initially set may be necessary.
I have installed the Flowblade video editor on Debian, Mint, and Ubuntu. I find this editor closer to Premier and more intuitive than most other Linux based video editors I have tried. The video will be somewhat jerky via x2go when you play it, but the finished file once downloaded or uploaded to youtube or whatever, should be fine.
I replaced the incoming mail servers yesterday and this resolved the issue of spam not being scored. For some individuals, even though spam is being scored correctly, it is not being placed in the spam folder. What those have in common so far is that they all use tcsh for their login shell.
I found that I had not installed tcsh on these servers so did that around noon today.
I found there were some other shells missing, basically everything except sh, bash, and dash, and installed those as well. If you get spam in your INBOX and it has a header that shows “X-Spam: Yes”, please do not delete it and contact support. Thank you.
The incoming mail servers mx1 and mx2 have been replaced with Ubuntu based servers. Still using postfix as the MTA but it’s a newer version as is procmail and smartlist. DKIM is now supported as well so that’s one more tool to stop bogus e-mail.
Please let me know if you see any more e-mail not scored by spamassassin.
Apache httpd has been upgraded to version 2.4.27, apr has been upgraded to 6.0.2, and apr-util to 6.0.0. Should not be much change in functionality, there is some improvement in memory footprint. The less memory code uses, the more is available for cache.
Our web server stopped talking to the network today. I have not yet had a chance to do any forensics but I was able to connect to the machine via the Virtual Machine Manager but not ssh or www. And from what I’ve been able to determine so far even internal connections between applications such as the web server to the mysql database failed.
I have rebooted it, which restored network connectivity, and will investigate further.