Original URL: https://www.theregister.com/2011/05/04/nsa_fort_mead_data_center/
US spooks to build 60 megawatt data center
What Federal budget crisis?
Posted in Channel, 4th May 2011 14:33 GMT
The Obama administration might have been wrestling mightily with a recalcitrant Republican Congress high on strong tea over the US government's fiscal 2012 budget, which had tens of billions of dollars in spending slashed, but the one thing that both Democrats and Republicans can agree on is that the National Security Agency needs a new data center weighing in at 60 megawatts.
According to declassified documents made available by the comptroller's office for the Department of Defense, the US government's fiscal 2012 budget includes $860.6m to build a high performance computing center at the NSA's Fort Meade, Maryland, headquarters facility. That is the cost for the facility alone, not the cost of the servers, storage, and networking gear that will inhabit the data center.
The Fort Meade data center is in the planning and design phase right now, which costs $35m of the total budget, and will be paid for in two phases during construction in fiscal 2013 and 2014 when the construction is actually done in the facility, which is in the suburbs northeast of Washington DC.
The budget includes $567.8m for the primary facility, including modular shell buildings and raised floor for the data center) and then anti-terrorism hardening, fire protection, and force protection enhancements. The mechanical systems in the facility account for $118.4m of the budget, and the electrical systems for the 60 megawatt facility will cost $225m. Supporting facilities (meaning, for the humans) cost another $180.6m.
The NSA's Fort Meade headquarters
The NSA budget does not specify what kind of computing gear will be housed at the Fort Meade facility, other than it is "designed to support state-of-the-art high-performance computing devices and associated hardware architecture" and that the computing center areas have "depressed slab construction" with a floor load rating of around 600 pounds per square foot. The data center will use air-cooled and water-cooled equipment and will have computer room air handlers (CRAHs) and computer room air conditions (CRACs); all of the mechanical systems and air distribution systems will have backup.
The Department of Defense expects to award a construction contract for the facility by September 2012, with construction expected to start three months later and for the data center module to be completed by June 2014.
Two years ago, the NSA got a $1.53bn allocation to build an even larger 65 megawatt data center on a 240-acre site at the National Guard facility in Camp Williams, Utah. Construction began on this data center in September 2009 and is expected to be completed in May 2014. This facility, which is located south of Salt Lake City, has about 100,000 feet of data center floor space and comprises over 1 million feet of total space.
The US Army Corps of Engineers oversees these construction jobs, but subcontracts the work out to commercial companies. The biggest beneficiaries of these NSA data center construction jobs, aside from the workers who get paid to do the work, are the US senators that get to brag about bringing jobs back to their states.
While the data centers in Fort Meade and Camp Williams are interesting, what inquiring minds want to know is what iron the NSA is using and what it is using it for. In theory, these two facilities are used to snoop on foreign communications as they come into the United States, not to spy on citizens located inside the country.
The NSA has been an enthusiastic supporter of supercomputer Cray over the decades, and is one of the main funders of Cray's "ThreadStorm" XMT multithreaded supercomputers. These machines pair the ThreadStorm processors, designed by Tera Computer founder Burton Smith, who now is a technical fellow at Microsoft Research, and James Rottsolk, who used to run Cray after Tera bought the name-brand super maker in 2000.
The MTA, and then follow-on XMT designs, are packed with lots of threads, deep instruction pipelines, and fast switching between threads to keep the machine humming. The current "ThreadStorm" processor used in the XMT machines has 128 threads and the XMT system can scale up to a mind-bobbling 8,192 threads with 128TB of shared memory across all those threads. This is a perfect machine for hyperscale pattern matching.
Cray's Custom Engineering division is expected to roll out an XMT-2 processor later this year, and it will presumably plug into the current XE6 blade servers that also support Opteron 6100 chips. Cray is a licensee of the Opteron socket design as well as HyperTransport interconnects, and the XMT chips plugged into Rev F Opteron sockets.
If this is indeed one of the key systems going into the two NSA data centers, the US spook agency will not be the first organization to get an MTA-2 system. The Swiss National Supercomputing Centre (CSCS) in Manno, Switzerland, inked a deal to get the first MTA-2 machine back in February. ®