New Snowden leak: How NSA shared 850-billion-plus metadata records

'Federated search' spaffed info all over Five Eyes chums

Top 5 reasons to deploy VMware with Tegile

Documents leaked by Edward Snowden suggest during the noughties, the NSA massively expanded the information it shared with its Five Eyes allies and other agencies.

In slides given to The Intercept, the NSA boasts that its ICREACH program “increases NSA communications metadata sharing from 50 billion records to 850+ billion records”, with a footnote adding that 126 billion of these new records came from “2nd party SIGINT partners”.

At the same time, there was also a massive increase in the metadata the NSA was storing. The date, time, duration, called number and calling party – drawn from old PSTN systems – plus more than 20 new fields including latitude and longitude became part of the data-sharing.

To cope with all of this the NSA also needed to create a federated database search engine, in essence its own middleware sitting between the databases and the users.

Fast growth of NSA datasets

Massive increase in NSA's metadata volumes in 2006/2007. Image: The Intecept

The documents, dated 2006 and 2007, describe ICREACH as a “federated query” engine that would search “across all data sets for information relating to a target identifier.” The leaked documents say an agent could search from a single login to retrieve “phone number, global mobile satellite and cellular events and selectors, e-mail address, etc”.

Data brokers were also crafted to give the NSA's partners – the Australian Signals Directorate (then DSD), Britain's GCHQ, New Zealand's GCSB, Canada's CSE and the US FBI – access to the same search capabilities.

New fields added to what the NSA shared

The NSA shared more metadata fields post-ICREACH. Image: The Intercept

While maintaining the position that metadata is “information about content (but not the content itself)”, the NSA also notes that the data collected includes “formats and protocols used to render the information for people and systems”.

The middleware also meant that analysts were able to work on intelligence leads “without requiring access to raw intelligence”, the leaks claim. ®

Beginner's guide to SSL certificates

More from The Register

next story
Bladerunner sequel might actually be good. Harrison Ford is in it
Go ahead, you're all clear, kid... Sorry, wrong film
Musicians sue UK.gov over 'zero pay' copyright fix
Everyone else in Europe compensates us - why can't you?
I'll be back (and forward): Hollywood's time travel tribulations
Quick, call the Time Cops to sort out this paradox!
Megaupload overlord Kim Dotcom: The US HAS RADICALISED ME!
Now my lawyers have bailed 'cos I'm 'OFFICIALLY' BROKE
Euro Parliament VOTES to BREAK UP GOOGLE. Er, OK then
It CANNA do it, captain.They DON'T have the POWER!
Forget Hillary, HP's ex CARLY FIORINA 'wants to be next US Prez'
Former CEO has political ambitions again, according to Washington DC sources
prev story


Seattle children’s accelerates Citrix login times by 500% with cross-tier insight
Seattle Children’s is a leading research hospital with a large and growing Citrix XenDesktop deployment. See how they used ExtraHop to accelerate launch times.
Forging a new future with identity relationship management
Learn about ForgeRock's next generation IRM platform and how it is designed to empower CEOS's and enterprises to engage with consumers.
How to determine if cloud backup is right for your servers
Two key factors, technical feasibility and TCO economics, that backup and IT operations managers should consider when assessing cloud backup.
High Performance for All
While HPC is not new, it has traditionally been seen as a specialist area – is it now geared up to meet more mainstream requirements?
Business security measures using SSL
Examines the major types of threats to information security that businesses face today and the techniques for mitigating those threats.