Yesterday evening and part of today, authentication was broken for Qwest lines and part of MegaPop. This was the result of fried fiber in the tunnel in Boston where the train fire was which isolated Qwests authentication server from Star2's, a port aggregator we use, and cutting MegaPOP's authentication servers off from a couple of their POPs. Last night we were also hit with a SYN flood, a type of packet flood, the IRC server was the target but it was very intense and caused about 90% packet loss which pretty much isolated us from the net. Sprint was very busy owing to the Boston problems and a recent security problem in Cisco routers IOS and so they couldn't get a NULL route in very quickly, took about 45 minutes which is not usual for Sprint. I left the NULL route for the IRC server in place today and then other hosts were were attacked, including Eskimo, and because the shell server has to be fairly exposed to the net for all of the service it provides to work, it wasn't well protected and they were able to exhaust kernel memory repeatedly and crash it several times. They also targeted other hosts here and crippled them enough that I eventually had to reboot them. Because the IRC related servers is usually the draw for these kinds of attacks, I've made some changes this evening that hopefully will somewhat limit damage to the rest of the network. One T1 now routes one class C incoming which will be reserved for IRC related servers. The other two route the remaining network here. This way when the IRC servers are flooded it won't trash the rest of the network although it will not prevent attacks like the one today where they targeted other machines. I've got the client IRC server, the one accessible by the public, moved to this seperately routed class C and will move chat shortly. All three T1's are still used for outgoing traffic of all types so this will not affect outbound capacity.