Ongoing outages at random over three days is bad enough for any business site, but when it happens to New Zealand's Stock Exchange, some hard questions need to be asked for rather obvious reasons.
So far there's been little in the way of detail from NZX and its multiple technology providers as to what's causing the outages. The sharemarket has only issued terse comments and stayed quiet on the technical issues.
Meanwhile, the outages are continuing. As of writing, the entire NZX.com disappeared off the internet for around 20 minutes before midday.
The site came back up for a while, and then didn't load again after that. There will be some unhappy traders out there, and eyes overseas wondering what's going on, which is not what you want at a time when markets are already spooked by the pandemic ravaging economies.
The outages this week are blamed on a distributed denial of service (DDoS) attack, which overwhelmed NZX's provider Spark's network.
If so, it would've been one hell of a large amount of traffic to do that. Content delivery network Akamai that also offers DDoS protection - it has the network capacity to sink heaps of traffic - said it has seen attacks peaking at almost 200 gigabits per second the last few days, in the Asia Pacific region, targeting financial institutions.
The attackers, who some security researchers reckon are copycats using names like Fancy Bear, a group connected to Russian government hackers, and the Armada Collective, are asking for around half a million dollars in Bitcoin to stop the traffic floods. The price is going up every day if payment's not made.
DDoS attacks are extremely common and there's a cottage industry of criminals that advertise access to their massive worldwide botnets of compromised computers and insecure broadband modems. Pay a few bucks and you too can take out a website, to disrupt their business and in some cases to extort them (don't even think about it: DDoS "booters" are often caught and get long jail sentences).
It is possible to mitigate and defend against DDoS attacks however, with several big companies providing that service. If there was a DDoS, why wasn't NZX protected against such attacks?
NZX now appears to be shoring up what was a vulnerable online presence. For example, the servers that direct computers looking for nzx.com sat on the same network on Spark, and there were only two of them.
If an attacker wants to make the whole of nzx.com disappear off the internet, swamping the network with those two servers on it would be an easy way to do it.
That weakness is not only against requirements by the Internet Assigned Numbers Authority (IANA), it's very simple to find with only a small amount of technical knowledge.
This is a basic, well-known requirement that shouldn't have been missed. It was a rookie mistake, akin to painting a bull's eye on your back.
Now it appears NZX is fixing that mistake, and six nameservers for nzx.com now show up on Akamai's network. To take out Akamai, the DDoS attackers would need a lot of firepower, in the terabits per second range that they threaten with in their extortion letters. It could also explain why there we're seeing the site appear and disappear off the internet intermittently.
Technical issues apart, NZX hasn't done itself any favours by keeping quiet on this. When content delivery network Cloudflare went down recently, its chief executive Matthew Prince fronted on social media straight away and apologised for the embarrassing cable unplugging incident in a data centre that sent millions of customers offline.
There needs to be a warts and all post-mortem on this incident from NZX, so that others who operate critical infrastructure can learn from what went wrong.