Centos7 is paused temporarily for cloning.  I have been totally unable to get a fresh install of Centos7 based upon the most recent distribution to behave properly, in particular, idmapd will not start and even if I start it manually it does not work, but it works on the existing machine so I’m cloaning it as a work around.

    When the copy finishes, Centos7 will resume execution where it left off.  If I’ve stopped something of yours, please be patient.  I need to do this to get a database someone’s shop is dependent upon up and running again.

     I have removed status.net because people have found a way to cause it to lock tables in the database and then cause processes to pile up until it exhausts all database connections.

     The code is old and is no longer being maintained.  It has been replaced by pump.io but pump.io is incompatible with a web server running on the same machine as it has it’s own server engine rather than using Apache.

     I may place it on another host here but at present it is not installed.

     I am going to reboot mail.eskimo.com, shellx.eskimo.com, and ftp.eskimo.com to change memory allocation to hopefully address some response issues this morning.

     This will be a very brief outage, should be less than a minute.

     We have been occasionally running into the max_connections limit on mysqld.  I’ve bumped it up a few notches so that issue should be resolved.

     Mint is back up and running however not all of the applications are installed and configured.  In particular postfix isn’t setup yet so don’t try to send mail, it will just get stuck in queue or rejected outright.  And on the subject of mail, most mailers are not yet installed.

     Unfortunately, there was no clean way to upgrade from Mint 14 to 17.2 cleanly except to do a clean install.

     Our shell server mint.eskimo.com is presently down for an upgrade to Mint 17.2.  If you need a debian based server in the meantime please use the newly upgraded ubuntu.eskimo.com or debian.eskimo.com.

     We were hit with two denial of service attacks today.  The first was close to 9AM and only lasted a few seconds.  The second was between 3:00pm and 3:45pm lasting 45 minutes.

     After analysis of the traffic, I determined that it was not possible to block this type of attack without disrupting legitimate traffic.

     The real problem is that our router is not able to handle the volume of packets that can be generated easily these days.  Simply put, it doesn’t have enough CPU to examine 100,000 packets per second.

     I ordered a replacement router today, the existing unit is one that we have used since 1995.  The replacement should be here in 7-10 days and should handle more than 10x the traffic in terms of packets per second than our current router is completely swamped by.  In addition, it will change our edge interface from 100mb/s half duplex to 1G/s full duplex.

     It is possible for a denial of service attack to generate as much as 46G/s and that would overwhelm even our new router but the majority of attacks we’ve seen lately have not even been 100mb/s and instead of exhausting bandwidth are exhausting router CPU.

     Our shell server, ubuntu.eskimo.com, has been upgraded to version 15.04 Vivid Vervet.

     Unlike the first attempt when 15.04 first came out, this upgrade went almost flawlessly.  The only boot problem being some features missing in the kernel built that this version requires and the non-existence of postgres user and group (it is in NIS but the start-up sequence with systemd doesn’t have NIS enabled at the time it tries to create the necessary log files).

     These two things have been corrected and now the boot-up is 100% clean and mighty fast (takes about two seconds from launch to login).

     The denial of service attack lasted between about 4:00PM and 4:15PM Pacific Time.  It ended just as I was successful at getting an Isomedia tech on the phone so we weren’t able to identify the source.

     It took me 3 hours to make a trip 22 miles in each direction.  Mariners game screwing up traffic on the way there, concert at the Key Arena screwing up traffic on the way back, plus two accidents.

     Someone launched a denial of service attack against our router at the co-location facility.  Even though it is presently up and routing traffic, it has stopped allowing administrative connections so I can neither monitor traffic or adjust firewall rules.

     For this reason I am making a trip to the co-location facility to reboot the router, this is the only way to restore administrative functionality, and will be out of the office for a couple of hours.