I apologize that the last two months have had some difficulties and interruptions but there was good reason for it.
As you can see, web response latency spikes are GONE. This means people viewing your web page hosted here will never see a delay.
You can verify this for yourself by signing up forGoogle Webmaster tools, on your site hosted here, and following the Crawl Stats (under the Crawl pull-down).
In addition to the improvement of web response time, all user data is now on RAID10 disk arrays. RAID10 means both mirrored (for redundancy) and stripped (for speed).
Drive failures will not result in lost customer data and the read speed is doubled. RAID10 keeps a fully redundant copy of your data, not just hash sums, so there is no rebuild time in the event of a failed disk.
I still don’t know how people are stopping the name servers but I put a script in place on all the servers that checks them once a minute and restarts them if they stopped so at least there won’t be prolonged outages while I research this issue.
Our public name servers were struck with some sort of denial of service attack today that stopped them all from functioning. A restart brought them back online. I am still researching this and trying to gain a better understanding of who is doing what and how to prevent a recurrence.
If you find your website not working or if you are using our name servers and find that you can not resolve pages, please contact us right away.
I have filed a bug report with Internet Systems Consortium.
If anyone has any knowledge of this exploit and how to prevent it, please e-mail firstname.lastname@example.org or call me at (206) 812-0051.
traceroute to 220.127.116.11 (18.104.22.168), 30 hops max, 60 byte packets 1 50-251-249-54-static.hfc.comcastbusiness.net (22.214.171.124) 0.441 ms 0.414 ms 0.406 ms 2 126.96.36.199 (188.8.131.52) 14.070 ms 14.064 ms 14.054 ms 3 te-0-2-0-2-sur03.seattle.wa.seattle.comcast.net (184.108.40.206) 15.024 ms 168.675 ms 267.358 ms
^^^^^^^^^^^^^^^^^^^^^^^^^^^ Starts to get bad here… 4 be-20-ar01.seattle.wa.seattle.comcast.net (220.127.116.11) 1190.337 ms 1280.773 ms 1280.772 ms
^^^^^^^^^^^^^^^^^^^^^^^^^^^ Really gets bad here. 5 be-33650-cr02.seattle.wa.ibone.comcast.net (18.104.22.168) 1784.715 ms 2266.942 ms 2349.390 ms 6 he-0-11-0-0-pe05.seattle.wa.ibone.comcast.net (22.214.171.124) 2346.864 ms 2814.403 ms 2814.620 ms 7 as11404-1-c.seattle.wa.ibone.comcast.net (126.96.36.199) 15.518 ms 14.991 ms 15.245 ms 8 agg2-sea-a-t8-3.bb.spectrumnet.us (188.8.131.52) 14.710 ms 14.713 ms agg2-sea-a-t8-2.bb.spectrumnet.us (184.108.40.206) 23.925 ms 9 isomedia-SEA-10000M.demarc.spectrumnet.us (220.127.116.11) 23.919 ms 23.909 ms 24.883 ms 10 sea-ring.isomedia.com (18.104.22.168) 29.859 ms 30.093 ms 44.073 ms 11 east-core.isomedia.com (22.214.171.124) 44.307 ms 140.992 ms 794.775 ms 12 bllvwaelrt0.eskimo.net (126.96.36.199) 871.223 ms 856.486 ms 856.458 ms
The latency has moved farther up into Comcast’s network. This is an issue that appears to be squarely rooted in Comcast, and not real surprising since it was down most of the night last night entirely, including telephone service.
If anyone NOT on Comcast is experiencing latency, if you would e-mail a traceroute so I could look for common elements it would be appreciated.
If you ARE on Comcast, please consider calling 1-800-391-3000 and generating a trouble ticket.