Virtual Machine Migration

There may be some points where things are a bit sluggish today as I migrate some virtual machines from one box to another.  This involves copying images around 100GB.  With the old 100mb/s switch this would pretty much stop things.  I’m hoping not with the 1GB switch, still it’s going to task disk I/O and other resources on the machines pretty heavily.

The purpose for migrating these is for load balancing and to provide better redundancy when physical hosts are down by spreading functionality across multiple physical boxes.

Big Increase in BotNet Activity

Over the weekend, the amount IP addresses that brute force password guessing attacks originate from as detected by fail2ban, log scanning and automatic action script, has more than doubled from about 300 IPs per day to about 750 and that seems to be growing.

This pattern is usually indicative of some new Windows malware out in the wild successfully propagating to a huge number of machines that can then be used for things like password guessing and distributed denial of service attacks.

Relating to DDOS attacks, a large Botnet is attempting to use our DNS servers as DDOS amplifiers.  This won’t work because we have rate limiting configured on the external views for all of our servers, but it generated so much crap in the syslog’s that it ran some servers out of disk space.

As a result of this I’ve added code that bans IPs for an extended period if they exceed rate limit thresholds which both quieted down the logs and reduced CPU load on the name servers servers substantially.

The relevance to you is, if you are running Windows, make sure your anti-viral and anti-malware software and it’s databases are up to date and run scans frequently.

I recommend running Malware Bytes, as 9 times out of 10, when I have a customers computer that is infected, it’s the application that finds the infection.

Second thing, if your password is easily guessable, for example, a dictionary word, or a dictionary word with a number after it, it should be changed to something more complex.

An ideal password will contain no dictionary words, no proper names or anything related to your account such as your login, a combination of UPPER CASE, lower case, punctuation characters such as ~!@#$%^&*()_+=-`{}[]|:;”‘<>,.?/ and numbers 0123456789.

Mail Server Maintenance 1/4/2015 00:01-00:25

I will be taking the client mail server down for maintenance just after midnight to image it for recovery in the event of file system corruption.  This should take less than about 25 minutes.

During this time, pop, imap, outbound smtp, and webmail will be unavailable.  Incoming mail will still be processed.  Mail can be viewed and/or deleted using shell mailers like pine, mail, mush, mh, but outgoing mail can not be sent during this time.  Some mailers that use the local mail smtp server will queue, but any that try to send via will fail.

Switch Upgrade Completed

The work on the switch upgrade has been complete.  Now even copying an 8gb file only increases ping times to about 65ms which isn’t noticeable where before it would lock them machines up for sometimes seconds because the file transfers sucked up all the bandwidth.

Switch Upgrade – Brief Interruptions

I will be upgrading the existing 100-base-T switch to a Gigabit switch to accommodate increased traffic between the file servers and other machines.

This increase traffic is largely the result of a move from NFSv3 to NFSv4.  NFSv3 wasn’t able to use the full bandwidth of a 100-base-T connection but NFSv4 will totally saturate the link during a large file move.  This is resulting in occasional latency spikes.

The change-out will create only brief interruptions of several seconds as machines are moved from one switch to the other (a physical movement of connectors over a distance of several inches).