Feeds

Google Spanner — instamatic redundancy for 10 million servers?

Mountain View wants your exabyte

Internet Security Threat Report 2014

Google’s massively global infrastructure now employs a proprietary system that automatically moves and replicates loads between its mega data centers when traffic and hardware issues arise.

The distributed technology was first hinted at — in classically coy Google fashion — during a conference this summer, and Google fellow Jeff Dean has now confirmed its existence in a presentation (PDF) delivered at a symposium earlier this month.

The platform is known as Spanner. Dean’s presentation calls it a “storage and computation system that spans all our data centers [and that] automatically moves and adds replicas of data and computation based on constraints and usage patterns.” This includes constraints related to bandwidth, packet loss, power, resources, and “failure modes”.

Dean speaks of an “automated allocation of resources across [Google’s] entire fleet of machines” — and that's quite a fleet. Google now has at least 36 data centers across the globe — though a handful may still be under construction. And as Data Center Knowledge recently noticed, the goal is to span a far larger fleet.

According to Dean’s presentation, Google is intent on scaling Spanner to between one million and 10 million servers, encompassing 10 trillion (1013) directories and a quintillion (1018) bytes of storage. And all this would be spread across “100s to 1000s” of locations around the world.

Imagine that. A single corporation housing an exabyte of the world's data across thousands of custom-built data centers.

Google Spanner

Google’s 10-million-server vision

Dean declined to discuss the presentation with The Reg. And Google’s PR arm has yet to respond to specific questions about the Spanner setup. But Google senior manager of engineering and architecture Vijay Gill alluded to the technology during an appearance at the cloud-happy Structure 09 mini-conference in San Francisco earlier this year.

Top 5 reasons to deploy VMware with Tegile

More from The Register

next story
Azure TITSUP caused by INFINITE LOOP
Fat fingered geo-block kept Aussies in the dark
NASA launches new climate model at SC14
75 days of supercomputing later ...
Yahoo! blames! MONSTER! email! OUTAGE! on! CUT! CABLE! bungle!
Weekend woe for BT as telco struggles to restore service
You think the CLOUD's insecure? It's BETTER than UK.GOV's DATA CENTRES
We don't even know where some of them ARE – Maude
Cloud unicorns are extinct so DiData cloud mess was YOUR fault
Applications need to be built to handle TITSUP incidents
BOFH: WHERE did this 'fax-enabled' printer UPGRADE come from?
Don't worry about that cable, it's part of the config
Stop the IoT revolution! We need to figure out packet sizes first
Researchers test 802.15.4 and find we know nuh-think! about large scale sensor network ops
DEATH by COMMENTS: WordPress XSS vuln is BIGGEST for YEARS
Trio of XSS turns attackers into admins
prev story

Whitepapers

Choosing cloud Backup services
Demystify how you can address your data protection needs in your small- to medium-sized business and select the best online backup service to meet your needs.
Forging a new future with identity relationship management
Learn about ForgeRock's next generation IRM platform and how it is designed to empower CEOS's and enterprises to engage with consumers.
Seattle children’s accelerates Citrix login times by 500% with cross-tier insight
Seattle Children’s is a leading research hospital with a large and growing Citrix XenDesktop deployment. See how they used ExtraHop to accelerate launch times.
5 critical considerations for enterprise cloud backup
Key considerations when evaluating cloud backup solutions to ensure adequate protection security and availability of enterprise data.
High Performance for All
While HPC is not new, it has traditionally been seen as a specialist area – is it now geared up to meet more mainstream requirements?