The few times I did get connected, I also had a lot of lag. My client was clearly running far ahead of the server, as it would stop routinely to catch up. I am on a different internet connection than usual, but it should actually be much better than the one I usually play on. I could also connect to benjad's server perfectly fine.
I'm wondering if the Australia - USA fiber pipes were loaded last night and they were kicking strange port numbers? Additionally, since you're running a non-headless server, you've shrunk the window down so there is very little visible display, right? If you have the full display, it can consume a lot more system resources, leading it to lag.
Also: it doesn't seem to be restricted to the 64 game; I also think I crashed the 128 game at least once last night.