This year's Census was nothing short of a spectacular debacle after the website where Australians were to fill out the survey went down for nearly two days. Last night, the Australian Bureau of Statistics (ABS) chief David Kalisch fronted the Senate Estimates in parliament to answer questions about the incident. We found out that the ABS will have to spend around $30 million to fix the damage. He also admitted that the ABS made a number of poor judgement calls for Census 2016. Here's what he had to say along with a recap of what has happened since the Census outage occurred two months ago.
Prior to this year's Census, the ABS pushed for the population survey to be moved predominantly online, which would save the agency over $100 million. The five-year budget for the 2016 Census was around $470 million. Despite the ABS continuously reassuring the public that the online Census form can handle an immense amount of traffic, the website was hit by multiple DDoS attacks that crashed it for about 40 hours.
This left a lot of Australians panicking because they were under the impression that if they didn't fill out the survey on Census Night, then they would be fined; which wasn't the case.
Kalisch told the Senate Estimates committee that the ABS has already spent $20 million in remediation costs in the aftermath of #censusfail and is expected to spend another $10 million. He admitted that the ABS made a series of bad judgment calls when preparing for Census 2016, which contributed to the failure, including not running a longer consultation for changes that were proposed for the Census.
One of the changes was for the ABS to keep names and addresses collected in Census 2016 for four years. Previously, names and addresses were only kept for 18 months. This decision caused an uproar and led to recommendations by privacy pundits to boycott the Census.
Here are some key stats and facts that have come out of the Senate Estimates hearing and in the weeks since #censusfail:
- The ABS has called out IBM, the company that hosted the Census website, for failing to prevent the DDoS attacks. In its submission to the Senate committee investigating #censusfail:
"The online Census system was hosted by IBM under contract to the ABS and the DDoS attack should not have been able to disrupt the system. Despite extensive planning and preparation by the ABS for the 2016 Census this risk was not adequately addressed by IBM and the ABS will be more comprehensive in its management of risk in the future. "… During 2016 the ABS had sought and received various assurances from IBM about operational preparedness and resilience to DDoS attacks."
- IBM has subsequently blamed Netgen Networks, the upstream provider, for not adequately geo-blocking traffic from overseas. The DDoS attacks reportedly came from abroad. Considering the Census only concerns people in Australia, it would have made sense to block traffic from overseas.
- Nextgen said IBM refused DDoS protection when it was offered.
- Even after all the troubles this year, Kalisch still believes the future of the Census is online.
- The ABS has stressed that no data was compromised during the DDoS attacks.
- Overall, 8.4 million Census forms, online and in paper format, have been received by the ABS
- There have been 10,000 cases of people refusing to complete the Census. Around 1800 refusal letters have sent out by the ABS. The agency has also issued 239 notices of direction but so far nobody has been fined.
Senate Standing Committees on Economics is currently holding an inquiry on Census 2016 and we'll hear more about what lessons could be learned from the incident in the coming weeks.