Feeds

NoSQL's CAP theorem busters: We don't drop ACID

'We are not relational fans here'

The Power of One Brief: Top reasons to choose HP BladeSystem

Different classes

Rosenthal describes FoundationDB as more like a back-end storage engine than the full SQL database. FoundationDB is a key-value store – the simplest form of NoSQL database. A key-value store is a simple API that has put, get and delete.

Having stripped back to basics, FoundationDB adds the ACID transactional capabilities using a network topology that breaks up and assigns workloads to different machines in a cluster, with machines using a new language – called Flow.

One class of machines in the database cluster are used for transition processing and conflict resolution, to ensure the integrity of data and transitions. Dumb notes are decoupled from smart transactional nodes, which know about ACID properties and transactions. Writes are filtered back to conflict-resolution machines, to enforce ACID.

“That’s very difficult but it enables a couple of really, really cool things,” Rosenthal tells us. “Once you have database with transactional integrity you can build up rich data models on top of simpler ones. That means for every other NoSQL database in market your app has to fit the model of the database.

'We think one of the reasons there are so many NoSQL databases on the market is because unless your application perfectly matches their data model, it becomes difficult to build data abstractions' - Rosenthal

“We think one of the reasons there are so many NoSQL databases on the market is because unless your application perfectly matches their data model, it becomes difficult to build data abstractions,” he says.

Take MongoDB: if you’re building an application for mobile phones and want to add location, you’d need a spatial index and any changes to documents would need to be made to both the documents and index. Without transactional capabilities, you cannot update the index data reliably and you have to go through the extra step of writing business and application logic. A transactional database would let you update the data and index and have confidence in the result.

Helping glue FoundationDB together is Flow, a new language that extends C++ with the addition of 10 Erlang keyword-types and an actor model concurrency for concurrent calculation. It took Rosenthal’s team three years to design and build Flow’s tools and the mathematical algorithms but just two weeks to actually build Flow, Rosenthal claimed. “Erlang has some really great programming tools for distributed systems but it’s sort of slow; C++ is fast but doesn’t have great tool. Flow ads tools and lets us write in a concise way,” Rosenthal said.

“The first year-and-a-half of development was spend exclusively in simulation – there was no real code to talk to a real network or hard disk, everything was simulated. Once we got programmatically solid in the simulation we started replacing components with their real world counterparts, and then we started working on the performance, which is crucial; performance can always be improved.”

The program was tested during hundreds of thousands of simulations run at night, with failures such as lost packets and rebooted machines simulated. Tests were run on a pair of 96 core clusters using a micro SSD and Intel-based servers.

Looking ahead, Rosenthal doesn’t see FoundationDB as replacing NoSQL or relational approaches, nor does he believe Flow will replace SQL. But he does see FoundationDB getting picked up by web start-ups and already established companies who want to grow rather than those trying to do something like replace an Oracle database.

The Stonebraker factor

You might not be surprised to learn, having read this, that Rosenthal agrees with relational database pioneer Michael Stonebraker – one of the main architects of the first relational database, Ingres, and the object-relational DBMS PostgreSQL. Stonebraker has been flamed by the pups of NoSQL for saying they are wrong to dump ACID, while he’s now behind VoltDB – a post-relational database. “A distributed data store without concurrency control is a toy and makes building things on it a lot harder,” he says.

FoundationDB is emerging as a new breed of NoSQL provider that, from the start, realises the rules of decades of computing have survived for a reason. As far as Rosenthal is concerned, FoundationDB will help bring those familiar with relational into the new world of web-scale NoSQL without making trade-offs.

“Relational is not going away - there’s always going to be apps for it,” Rosenthal says. “What we hear from lot of the people we talk to is they are looking at possibly using NoSQL and they know NOSQL will become more solid, but they know SQL will scale eventually - so they are hedging.

“We think our early adopters are going to be developers thinking about using other NoSQL systems and they’ll look at this and say the performance is great and it has full transaction. That’s worth switching to,” he said. ®

* The CAP theorem states that any networked shared-data system can have at most two of three desirable properties:

  • Consistency - equivalent to having a single up-to-date copy of the data;

  • high Availability of that data (for updates); and
  • tolerance to network Partitions.

** The ACID principles:

  • Atomicity - a transaction is all or nothing
  • Consistency - only valid data is written to the database
  • Isolation - pretend all transactions are happening serially and the data is correct
  • Durability - what you write is what you get

Securing Web Applications Made Simple and Scalable

More from The Register

next story
Apple fanbois SCREAM as update BRICKS their Macbook Airs
Ragegasm spills over as firmware upgrade kills machines
HIDDEN packet sniffer spy tech in MILLIONS of iPhones, iPads – expert
Don't panic though – Apple's backdoor is not wide open to all, guru tells us
NO MORE ALL CAPS and other pleasures of Visual Studio 14
Unpicking a packed preview that breaks down ASP.NET
Captain Kirk sets phaser to SLAUGHTER after trying new Facebook app
William Shatner less-than-impressed by Zuck's celebrity-only app
Do YOU work at Microsoft? Um. Are you SURE about that?
Nokia and marketing types first to get the bullet, says report
Microsoft takes on Chromebook with low-cost Windows laptops
Redmond's chief salesman: We're taking 'hard' decisions
Cheer up, Nokia fans. It can start making mobes again in 18 months
The real winner of the Nokia sale is *drumroll* ... Nokia
Mozilla fixes CRITICAL security holes in Firefox, urges v31 upgrade
Misc memory hazards 'could be exploited' - and guess what, one's a Javascript vuln
prev story

Whitepapers

Designing a Defense for Mobile Applications
Learn about the various considerations for defending mobile applications - from the application architecture itself to the myriad testing technologies.
How modern custom applications can spur business growth
Learn how to create, deploy and manage custom applications without consuming or expanding the need for scarce, expensive IT resources.
Reducing security risks from open source software
Follow a few strategies and your organization can gain the full benefits of open source and the cloud without compromising the security of your applications.
Boost IT visibility and business value
How building a great service catalog relieves pressure points and demonstrates the value of IT service management.
Consolidation: the foundation for IT and business transformation
In this whitepaper learn how effective consolidation of IT and business resources can enable multiple, meaningful business benefits.