Mail Server

     Our client mail server is having some issues with NFS connectivity to Ubuntu.  I am going to reboot it in hopes of clearing this problem however, NFS reconnecting is not 100% after a reboot in Linux so I will have to manually check all the shell servers which may take about twenty minutes after the reboot.

Red Hat Based Servers

     A while back vnc and rdp no longer functioned on the various Redhat based servers, centos7, centos8, centos-stream, fedora, and scientific7.

     This was caused by an update the xrdp that required a newer version of openssl but they did not provider a corresponding upgraded openssl.

     That has since been resolved and these remote access protocols are now available on all of these machines.

     In addition, I have installed mosh on all of these machines.  Mosh is a remote access protocol designed to work around intermittent connectivity such as that provided by a cell phone.

Web Server Tonight

     Our web server was unavailable or intermittently available this evening for about 1/2 hour owing to some hackers in France first trying to exploit a buffer overflow that existed in Apache 2.4.50, and when that failed just a high traffic denial of service attack that ran the server load over 250 briefly at which point it responded too slowly and most browsers timed out.  I had to set the timeout in Firefox to 60 seconds so I could connect and see what was going on with the server.

     I have since blocked all of the address space this attack was coming from, an address block in Paris France used for overflow belonging to OVH hosting, a hacker and spammer haven, 54.36.148.0/22 and compiled and installed Apache 2.4.51 which fixes this exploit (also required recompiling the apr and apr-util libraries as those installed by Ubuntu 20.04 were too old and lacked a function needed by Apache httpd 2.4.51).

Debian Restored

     Debian is now, to the best of my knowledge, fully restored to health.  Most of the software that was previously online is there, there are a few packages I removed because I felt they hindered security without providing much use for remote users.  If there is something missing you need still let me know.

     What was formerly called pine, a mail program, is now alpine.  It is the same program but the University of Washington changed the name and license.

     What was formerly called pico, a small text editor used by default with pine, is now called nano.  It is the same program but the University of Washington changed the name and license.

     X-forwarding now works again.

     VNC and RDP now work again BUT I was only able to get them to work with the slick greeter and that greeter puts the Desktop selection on a pulldown menu that doesn’t have room for all of the Desktops installed, so if you use VNC or RDP you will not be able to choose any desktop.  X2go does not use the greeter so  you can choose any desktop with X2go.

     I discovered in the process that the original problem was caused by a conflict between ufw and firewalld.  Both were installed prior to the upgrade to Bullseye and both used to be mutually compatible, both entered rules into iptables and you just got the sum total of those rules.  Now however if you try to enable both it locks up the machine and it will not boot properly.  Firewalld no longer works by itself either, it does not enter the rules you specify into iptables, in short it is just plain broken so I am now using ufw exclusively for firewall control.

    I’ve made a backup in it’s fixed state so we should be able to recover from future maladies.

Debian is Up

I found what was causing the boot problems, firewalld and ufw were simultaneously installed.  In earlier revisions these played good together but now they lock up when both installed.  And firewalld by itself does not seem to function anymore, does not add anything into iptables.

VNC is just giving a black screen.  Installed same as I had it before so having a hard time trying to determine why.

Debian

     I was trying to get vnc working and for some reason the firewall was blocking it.  I had ufw allow vnc but still couldn’t connect to it off of the machine.  So I turned off firewalld and the machine locked up.  Obviously this is not correct behavior and I am still troubleshooting.  Availability of debian is likely to be intermittent tonight and random crashes are quite probable during the troubleshooting phase.

Debian

     Debian is back up except I do not have all the graphical access working again and much software is not installed.

     I would appreciate your help.  Installing all the software previously installed is not workable as it causes systemd to get stuck in a loop and not boot properly.  I ended up re-installing a second time after working all night to recover it.  So I only want to install things people actually use.

     To that end, I’m asking if you are a regular Debian user, please login to debian.eskimo.com and check if software is present that you need, ESPECIALLY if you had cron jobs there (which I have not restored yet).  If software is missing, please use the ticket system (website https://www.eskimo.com/ under Support pulldown, Tickets) to initiate a ticket so I know what needs to be re-installed and/or fixed.

     There will be periodic downtime during this process in order to make frequent backups to essentially checkpoint the system so I don’t end up having to do yet another full re-install.

 

Debian

In the process of installing software it exploded again, issue with nvidia drivers.  I’m going to have to start-over again.  It’s 5:30AM so need to get some sleep.

 

 

Debian Up

     Debian is back up but software is still re-installing and I am not re-installing your cron jobs until all the software is re-installed as some of them depend upon packages not yet installed.

     Also, rdp and vnc and web desktop access are working again and ssh -X redirection is also working again.  None of these have worked since the upgrade to bullseye.  This is a fresh install of bullseye but I dumped all the installed packages before re-loading so I would know what to re-install.  Also all the /usr/local stuff but it is not restored yet.  It may take a day or two for all the software to install at the rate it’s going.