First, I want to apologize for not getting back to you all regarding various ideas and suggestions.  Some of them are actionable and some not right away but I am working through them and will get back to you so please be patient.

     I had more than one person ask about docker so I will address this publicly here.  There are three different options.  On the Docker Store site there is the Community Edition which has a license that basically forbids it’s use in an environment like ours, and there is their commercial version which is $70/node per month which is insane and cost prohibitive so we can’t do that.  The remaining option is virtually every Linux distribution has some incarnation of it with varying degrees of being complete and up to date.  Since this is the only cost effective option it is the one I am going with.

Dear Eskimo North Customer,

This August, we have relatively few high dollar accounts up for
renewal, and this means I will likely encounter a revenue shortage unless
I can find ways to sell more service.

In the past I’ve come up with various ideas but they’ve largely
flopped. I had quite a few people ask about private virtual servers, so I
put in place the resources to provide them but so far have only sold six.

Our web server is one of relatively few capable of serving both
http1.1 and http2. Although modern debian claimed Linuxes included an
Apache module for this, the version of openssl included doesn’t have the
required capabilities. I’ve compiled a newer verison of openssl to make
this possible. This provides a speed advantage over other providers.

Google, when they index your pages, all other things being equal
pages that have both https and http2 are given higher priority, after that
speed counts.

Anyway, I am asking for ideas, either for new things you’d like that
we aren’t presently providing but could, or ways to sell more of our
existing services.

Please tell me what would incentivize you or people you know to
purchase additional services from us. Right now I have a surplus of host
CPU, memory, and disk that I’d like to find a use for.

Thank you.

Sincerely,

Robert John Dinse
Nanook@Eskimo.com

If you receive an e-mail like this:

Hi Customer,

Your Apple ID will be disable because of some violated policies.

You need to sign and verify it as soon as possible. (we have noticed that
your account information appears to be invalid and unverified).

Please open the attached file and verify your apple id before 24 hours, or
your apple id will be disable permanent.

Sincerely,

Apple Support

     Do NOT open the attachment and do NOT respond to it.  This is a third party forging an e-mail to look like Apple but it is NOT Apple and if you respond to this you will be giving a third party your Apple ID with which they can make purchases.

     Reboots completed, all machines tested for remounting of /home and /mail via NFS also all tested to be sure NIS bound successfully.  Only two machines did not mount properly, debian and uucp, and in the case of UUCP, it is not relying on either of those file systems so was not service impacting.  All are corrected now.

    I plan on rebooting all of the Linux based machines tonight just after midnight in order to load a newer kernel.  Since we’ve been on Ubuntu 18.04 these have gone mostly smooth with the exception of some shell servers not remounting file systems correctly.  I will check these immediately after reboot.  Services should be interrupted for about five minutes each with the exception of those shell servers that do not properly remount.  It takes about an hour to check them all.

     Whoever launched this attack was forging IP addresses inside our network.  I’ve added a firewall rule blocking inside addresses from originating on the external router interface.  This should prevent this type of attack in the future.

     We’ve got a denial of service attack aimed at our name servers right now.  It has actually been going on for more than 24 hours.

     The server being hit the hardest is 204.122.16.8, therefore you may wish to use one of the others, 204.122.16.9, 204.122.16.3, and 204.122.16.7 as your first choice for a while.

     I am endeavoring to determine all the sources the attack is originating from and block them however there are many so this may take a while.

     While gcc 7.3 is the default on Ubuntu, gcc 8.1 is available.  To use 8.1 put the following in your .bashrc file or otherwise set the variable CC = gcc-8: 

     export CC=”gcc-8″

     This will cause gcc-8 to automatically be used by most compile scripts.

     There are those who say 8.1 is still experimental, this may be but I compiled apache with it and it has been running stable for months.  Apache is a pretty good compiler stress test.

     The code that gcc-8 generates is significantly smaller and faster than gcc 7.3 which is the default.

     At present this is available on ubuntu.eskimo.com only.

     With the exception of eskimo.com which is an old Sparc box with 64MB running SunOS 4.1.4, I checked all shell servers to make sure x2go and the xfce Desktop were working properly.

     I found several machines where x2go was not going full screen because of the wrong version of xrandr and corrected those.  They were Fedora, Scientific7, and OpenSuse.

     I found several machines with broken installations of xfce and corrected those, these were centos7, debian, and mxlinux.

     I received an e-mail asking why Ubuntu was mounting NFS file systems with version 4.0 instead of 4.1, I responded I did not know and would need to research it.

     What I found was that actually version 4.2 is supported by most modern systems, and according to the manual the NFS client is supposed to negotiate the highest version mutually supported by the client and server but for some reason it will not do this.

     I found that if I added vers=4.2 to the mount options on machines which support it (the old centos6 machine does not with the stock kernel and is using 4.1 instead), then when the machine boots it will use that version.  This should not be necessary if things worked as documented but that is not the case.  Performance of version 4.2 is significantly better so changing this substantially improved the web server performance because it accesses it’s content via NFS.