Australia's going to need a standalone metadata retention bureau

Australia's proposed mandatory metadata retention laws are being made to look dangerous, but that won't stop them passing the nation's Parliament. Recent voting patterns suggest that opposition parties will more or less wave them through. Community activists, so insular and febrile in support of national broadband network, probably won't make a dent.

Figuring out how to do data retention right might therefore be more important than opposition to the proposal, because it's scary for law enforcement agencies to have it but scarier to think criminals will be able to access it by hacking metadata repositories.

And that's just what will happen if small internet service providers and carriers have to do their own retention.

I expect that even big carriers and ISPs won't do a great job of it: retention's not core business, will be done grudgingly. Regulators will doubtless be under-resourced and (initially at least) inexperienced, so carriers will be able to make a best effort, not the best possible effort.

That's not good enough.

Hence my belief that Australia will need a central metadata retention bureau to make storing the stuff possible under decent security.

As it happens, the nations businesses have form running this kind of thing. The financial services industry runs clearinghouses linking different participants. These organisations take data flows from many participants, route them between one another, settle things up and make sure that billions of dollars end up in the right place.

Ever heard of ePal, the Consumer Electronic Clearing System (CECS) or the Australian Payments Clearing Association (APCA)?

The first entity makes EFTPOS happen, the second handles all debit and credit card transactions and the third is an industry body that regulates payments. That you probably haven't heard of the three bodies mentioned probably they are doing their jobs without incident.

Could a carrier match that performance? Not if complaints to the Telecommunications Industry Ombudsman are a guide: Australia's telcos are forever messing things up. Yet these are the entities we're going to trust to collect, protect and share metadata?

I think not. Let's instead get ourselves a metadata retention bureau.

A bureau would do two things: perform collection of metadata from carriers and ISPs, then store it for the required period. By doing so, the bureau would free telcos and ISPs from the need to devise their own security rigs and operational processes. And it would have the chance to throw money and expertise at security of a central physical repository of retained data.

Australia's government is currently running a cloud first strategy, so my hypothetical bureau should store data in the cloud. Cloud operators must have extraordinarily high regard for every conceivable nuance of security. A bureau with the security of retained data at the very core of its being is a good match for suppliers with security at the core of their very being.

I don't want to pick winners here, but both Microsoft's Azure and Amazon Web Services operate facilities in Australia that are certified to store non-classified Australian government data. The former will store on disk from $0.24/gigabyte/month. Amazon's Glacier archival storage service is $0.012/gigabyte/month. At that price, and assuming each subscriber to a telco or ISP generates a gigabyte of data a month, we're looking at $0.29 a year for storage of 24 months worth of retained metadata. ^*

I also like the fact that Glacier data isn't online, for two reasons. Firstly, near-line storage is far harder for a criminal to mine. Secondly, the recovery time of a few hours reduces the likelihood of frivolous lookups of metadata because resources needed to recover are finite. Someone abusing the system with silly lookups will be put in their place by investigators doing important work.

Another virtue of a bureau is that its existence makes it easier for government to throw in some cash, as it has indicated it is willing to do because data retention is not core carrier business. Pity the bureaucrat who has to figure out what to pay to Optus, what to pay to Telstra and what to pay small, gaming-centric ISP Zeno.

Far better to guarantee a few years of funding to a bureau, stitch up an industry body to oversee it and get that organisation to figure out graded membership fees for ISPs and carriers.

Now for the tough question: who has the guts to run the bureau? The likes of IBM and HP have form doing big, complex, sensitive government work. Both have substantial Australian bit barns of substance. Big consultants like Unisys and CSC could do the job, as could the likes of Accenture.

I've mentioned AWS and Microsoft above. Both have many capable channel partners and also quietly operate consulting arms.

To summarise: various Australian entities have the brains and resources to do a better job of metadata retention than telcos and ISPs. Cutting the carriers out of the loop looks sensible, because metadata retention is not core business and will therefore be done grudgingly and ultimately badly. An independent bureau offers the chance for better, easier-to-fund, metadata retention done once and done properly.

Our elected leaders couldn't object to that, could they? ®

Bootnote ^* If iiNet's Steve Dalby can do better than 1.2c a gigabyte a month in China, good luck to him. But Dalby also knows full well that data sovereignty will nobble him: there's no way do iiNet customers want their metadata in China. And there's also a snowball's chance in hell Dolby wants to pay for data carriage to China. As for Malcolm Turnbull's suggestion he wouldn't mind Vodafone storing metadata with one of its offshore siblings, perhaps it's best to write that one off as a former lawyer recalling harmonised privacy laws among many nations that mean data taken offshore would not put Australians in a worse position than data stored beneath the Southern Cross. That some laws, on some readings, make it possible to do so doesn't change the fact doing so is political poison. And just not advisable.