IBM throws ISP under a bus for Australia's #Censusfail
Big Blue claims ISP allowed DDoS. ISP says IBM rejected DDoS advice and services
IBM has blamed a supplier for causing the failure of Australia's online census, which went offline on the very night millions of households were required to describe their disposition.
Big Blue's submission (PDF) to Australia's Standing Committees on Economics, which is conducting an Inquiry into the Preparation, Administration and Management of the 2016 Census by The Australian Bureau of Statistics puts the blame for the failure at the feet of a company called NextGen Networks.
IBM does so because it says it devised a distributed denial of service (DDoS) prevention plan called “Island Australia” that involved “blocking or diverting international traffic intended for the eCensus site before it reaches the site, while leaving the system free to continue to process domestic traffic.”
“This method was chosen because the primary risk of DDoS attacks of sufficient size to disrupt site availability was considered to be from foreign sources.”
IBM's submission says two carriers were chosen to bring traffic to the Census site, Telstra and NextGen. Both were informed about “Island Australia” and how to implement it. But come Census day, IBM says “a Singapore link operated by one of NextGen’s upstream suppliers (Vocus Communications or Vocus) had not been closed off and this was the route through which the attack traffic had entered the NextGen link to the eCensus site.”
Big Blue's document says Vocus 'fessed up to the error on Census night.
The submissions says the DDoS that made it past the Island Australia defences then “... caused one of the mechanisms used by IBM to monitor the performance of the eCensus site to miscarry.”
“As a result, some IBM employees who were observing the monitor mistakenly formed the view that there was a risk that data was being exfiltrated from the website and that the risk needed to be further investigated. Out of an abundance of caution, IBM shut down access to the site and assessed the situation.”
NextGen has also made a submission (PDF) to the inquiry and it disputes IBM's version of events.
The carrier says IBM rejected its offer of dedicated DDoS protection and also ignored advice that Island Australia was not a decent DDoS shield. The submission also says that IBM tested NextGen's Island Australia rig on August 5th, and passed it as ready to roll.
Here's an excerpt from the submission:
“IBM had earlier in June 2016 requested Nextgen to provide a specific IP address range (due to a requirement to advertise the IP addresses to both Nextgen and Telstra), without providing any context to Nextgen regarding IBM’s “Island Australia”. After becoming aware of “Island Australia”, Nextgen advised IBM that the IP address range requested by IBM was part of a larger aggregate network, and therefore it was not possible to provide specific international routing restrictions for this range. Nextgen recommended using an alternative IP address range, which would give IBM better control, but this was rejected by IBM.”
Vocus has also made a submission (PDF) in which it blames both IBM and NextGen for the DDoS.
Big Blue is criticised because in Vocus' opinion “the Island Australia approach does not consider the reality of overseas network operators connecting to Australian service providers inside Australian borders.”
NextGen cops some pain because Vocus says it “was not informed of IBM’s DDoS mitigation strategy, Island Australia or its specific requirements, until after the fourth [DDoS] attack. As a result, any assumption that Vocus was required to, or had implemented Island Australia or geo-blocking … are inaccurate.”
In a delicious irony, NextGen's submission also notes its recent acquisition by none other than Vocus. Which will make life interesting at the first all-hands meeting once the acquisition closes.
The Inquiry will issue a report on November 24th. The Register's Australian outpost has laid in copious stocks of popcorn ahead of the report's release. ®