New Snowden leak: How NSA shared 850-billion-plus metadata records

'Federated search' spaffed info all over Five Eyes chums

Beginner's guide to SSL certificates

Documents leaked by Edward Snowden suggest during the noughties, the NSA massively expanded the information it shared with its Five Eyes allies and other agencies.

In slides given to The Intercept, the NSA boasts that its ICREACH program “increases NSA communications metadata sharing from 50 billion records to 850+ billion records”, with a footnote adding that 126 billion of these new records came from “2nd party SIGINT partners”.

At the same time, there was also a massive increase in the metadata the NSA was storing. The date, time, duration, called number and calling party – drawn from old PSTN systems – plus more than 20 new fields including latitude and longitude became part of the data-sharing.

To cope with all of this the NSA also needed to create a federated database search engine, in essence its own middleware sitting between the databases and the users.

Fast growth of NSA datasets

Massive increase in NSA's metadata volumes in 2006/2007. Image: The Intecept

The documents, dated 2006 and 2007, describe ICREACH as a “federated query” engine that would search “across all data sets for information relating to a target identifier.” The leaked documents say an agent could search from a single login to retrieve “phone number, global mobile satellite and cellular events and selectors, e-mail address, etc”.

Data brokers were also crafted to give the NSA's partners – the Australian Signals Directorate (then DSD), Britain's GCHQ, New Zealand's GCSB, Canada's CSE and the US FBI – access to the same search capabilities.

New fields added to what the NSA shared

The NSA shared more metadata fields post-ICREACH. Image: The Intercept

While maintaining the position that metadata is “information about content (but not the content itself)”, the NSA also notes that the data collected includes “formats and protocols used to render the information for people and systems”.

The middleware also meant that analysts were able to work on intelligence leads “without requiring access to raw intelligence”, the leaks claim. ®

Protecting against web application threats using SSL

More from The Register

next story
Phones 4u slips into administration after EE cuts ties with Brit mobe retailer
More than 5,500 jobs could be axed if rescue mission fails
Israeli spies rebel over mass-snooping on innocent Palestinians
'Disciplinary treatment will be sharp and clear' vow spy-chiefs
Apple CEO Tim Cook: TV is TERRIBLE and stuck in the 1970s
The iKing thinks telly is far too fiddly and ugly – basically, iTunes
Huawei ditches new Windows Phone mobe plans, blames poor sales
Giganto mobe firm slams door shut on Microsoft. OH DEAR
Phones 4u website DIES as wounded mobe retailer struggles to stay above water
Founder blames 'ruthless network partners' for implosion
Found inside ISIS terror chap's laptop: CELINE DION tunes
REPORT: Stash of terrorist material found in Syria Dell box
OECD lashes out at tax avoiding globocorps' location-flipping antics
You hear that, Amazon, Google, Microsoft et al?
Show us your Five-Eyes SECRETS says Privacy International
Refusal to disclose GCHQ canteen menus and prices triggers Euro Human Rights Court action
prev story


Providing a secure and efficient Helpdesk
A single remote control platform for user support is be key to providing an efficient helpdesk. Retain full control over the way in which screen and keystroke data is transmitted.
Saudi Petroleum chooses Tegile storage solution
A storage solution that addresses company growth and performance for business-critical applications of caseware archive and search along with other key operational systems.
Security and trust: The backbone of doing business over the internet
Explores the current state of website security and the contributions Symantec is making to help organizations protect critical data and build trust with customers.
Reg Reader Research: SaaS based Email and Office Productivity Tools
Read this Reg reader report which provides advice and guidance for SMBs towards the use of SaaS based email and Office productivity tools.
Security for virtualized datacentres
Legacy security solutions are inefficient due to the architectural differences between physical and virtual environments.