The new server software seems to have stopped the crashes but I still wasn’t satisfied with the performance of ns1 and ns2, especially ns1. I discovered that the main load was from the mail servers accessing various real time black hole lists as part of Spamassasins’ spam scoring.
To alleviate this I set-up two new name servers specifically for mx1 and mx2 to use, taking the load off of ns1 and ns2. These new servers are not directly available to the public but have been created to reduce the load and improve the response time to those that are. I now consistently get responses <100ms on all four public name servers.