Feeds

NoSQL's CAP theorem busters: We don't drop ACID

'We are not relational fans here'

High performance access to file storage

Different classes

Rosenthal describes FoundationDB as more like a back-end storage engine than the full SQL database. FoundationDB is a key-value store – the simplest form of NoSQL database. A key-value store is a simple API that has put, get and delete.

Having stripped back to basics, FoundationDB adds the ACID transactional capabilities using a network topology that breaks up and assigns workloads to different machines in a cluster, with machines using a new language – called Flow.

One class of machines in the database cluster are used for transition processing and conflict resolution, to ensure the integrity of data and transitions. Dumb notes are decoupled from smart transactional nodes, which know about ACID properties and transactions. Writes are filtered back to conflict-resolution machines, to enforce ACID.

“That’s very difficult but it enables a couple of really, really cool things,” Rosenthal tells us. “Once you have database with transactional integrity you can build up rich data models on top of simpler ones. That means for every other NoSQL database in market your app has to fit the model of the database.

'We think one of the reasons there are so many NoSQL databases on the market is because unless your application perfectly matches their data model, it becomes difficult to build data abstractions' - Rosenthal

“We think one of the reasons there are so many NoSQL databases on the market is because unless your application perfectly matches their data model, it becomes difficult to build data abstractions,” he says.

Take MongoDB: if you’re building an application for mobile phones and want to add location, you’d need a spatial index and any changes to documents would need to be made to both the documents and index. Without transactional capabilities, you cannot update the index data reliably and you have to go through the extra step of writing business and application logic. A transactional database would let you update the data and index and have confidence in the result.

Helping glue FoundationDB together is Flow, a new language that extends C++ with the addition of 10 Erlang keyword-types and an actor model concurrency for concurrent calculation. It took Rosenthal’s team three years to design and build Flow’s tools and the mathematical algorithms but just two weeks to actually build Flow, Rosenthal claimed. “Erlang has some really great programming tools for distributed systems but it’s sort of slow; C++ is fast but doesn’t have great tool. Flow ads tools and lets us write in a concise way,” Rosenthal said.

“The first year-and-a-half of development was spend exclusively in simulation – there was no real code to talk to a real network or hard disk, everything was simulated. Once we got programmatically solid in the simulation we started replacing components with their real world counterparts, and then we started working on the performance, which is crucial; performance can always be improved.”

The program was tested during hundreds of thousands of simulations run at night, with failures such as lost packets and rebooted machines simulated. Tests were run on a pair of 96 core clusters using a micro SSD and Intel-based servers.

Looking ahead, Rosenthal doesn’t see FoundationDB as replacing NoSQL or relational approaches, nor does he believe Flow will replace SQL. But he does see FoundationDB getting picked up by web start-ups and already established companies who want to grow rather than those trying to do something like replace an Oracle database.

The Stonebraker factor

You might not be surprised to learn, having read this, that Rosenthal agrees with relational database pioneer Michael Stonebraker – one of the main architects of the first relational database, Ingres, and the object-relational DBMS PostgreSQL. Stonebraker has been flamed by the pups of NoSQL for saying they are wrong to dump ACID, while he’s now behind VoltDB – a post-relational database. “A distributed data store without concurrency control is a toy and makes building things on it a lot harder,” he says.

FoundationDB is emerging as a new breed of NoSQL provider that, from the start, realises the rules of decades of computing have survived for a reason. As far as Rosenthal is concerned, FoundationDB will help bring those familiar with relational into the new world of web-scale NoSQL without making trade-offs.

“Relational is not going away - there’s always going to be apps for it,” Rosenthal says. “What we hear from lot of the people we talk to is they are looking at possibly using NoSQL and they know NOSQL will become more solid, but they know SQL will scale eventually - so they are hedging.

“We think our early adopters are going to be developers thinking about using other NoSQL systems and they’ll look at this and say the performance is great and it has full transaction. That’s worth switching to,” he said. ®

* The CAP theorem states that any networked shared-data system can have at most two of three desirable properties:

  • Consistency - equivalent to having a single up-to-date copy of the data;

  • high Availability of that data (for updates); and
  • tolerance to network Partitions.

** The ACID principles:

  • Atomicity - a transaction is all or nothing
  • Consistency - only valid data is written to the database
  • Isolation - pretend all transactions are happening serially and the data is correct
  • Durability - what you write is what you get

High performance access to file storage

More from The Register

next story
Android engineer: We DIDN'T copy Apple OR follow Samsung's orders
Veep testifies for Samsung during Apple patent trial
This time it's 'Personal': new Office 365 sub covers just two devices
Redmond also brings Office into Google's back yard
Windows 8.1, which you probably haven't upgraded to yet, ALREADY OBSOLETE
Pre-Update versions of new Windows version will no longer support patches
Batten down the hatches, Ubuntu 14.04 LTS due in TWO DAYS
Admins dab straining server brows in advance of Trusty Tahr's long-term support landing
Microsoft lobs pre-release Windows Phone 8.1 at devs who dare
App makers can load it before anyone else, but if they do they're stuck with it
Half of Twitter's 'active users' are SILENT STALKERS
Nearly 50% have NEVER tweeted a word
Windows XP still has 27 per cent market share on its deathbed
Windows 7 making some gains on XP Death Day
Internet-of-stuff startup dumps NoSQL for ... SQL?
NoSQL taste great at first but lacks proper nutrients, says startup cloud whiz
Microsoft TIER SMEAR changes app prices whether devs ask or not
Some go up, some go down, Redmond goes silent
Red Hat to ship RHEL 7 release candidate with a taste of container tech
Grab 'near-final' version of next Enterprise Linux next week
prev story

Whitepapers

Securing web applications made simple and scalable
In this whitepaper learn how automated security testing can provide a simple and scalable way to protect your web applications.
Five 3D headsets to be won!
We were so impressed by the Durovis Dive headset we’ve asked the company to give some away to Reg readers.
HP ArcSight ESM solution helps Finansbank
Based on their experience using HP ArcSight Enterprise Security Manager for IT security operations, Finansbank moved to HP ArcSight ESM for fraud management.
The benefits of software based PBX
Why you should break free from your proprietary PBX and how to leverage your existing server hardware.
Mobile application security study
Download this report to see the alarming realities regarding the sheer number of applications vulnerable to attack, as well as the most common and easily addressable vulnerability errors.