Feeds

NoSQL's CAP theorem busters: We don't drop ACID

'We are not relational fans here'

Secure remote control for conventional and virtual desktops

Different classes

Rosenthal describes FoundationDB as more like a back-end storage engine than the full SQL database. FoundationDB is a key-value store – the simplest form of NoSQL database. A key-value store is a simple API that has put, get and delete.

Having stripped back to basics, FoundationDB adds the ACID transactional capabilities using a network topology that breaks up and assigns workloads to different machines in a cluster, with machines using a new language – called Flow.

One class of machines in the database cluster are used for transition processing and conflict resolution, to ensure the integrity of data and transitions. Dumb notes are decoupled from smart transactional nodes, which know about ACID properties and transactions. Writes are filtered back to conflict-resolution machines, to enforce ACID.

“That’s very difficult but it enables a couple of really, really cool things,” Rosenthal tells us. “Once you have database with transactional integrity you can build up rich data models on top of simpler ones. That means for every other NoSQL database in market your app has to fit the model of the database.

'We think one of the reasons there are so many NoSQL databases on the market is because unless your application perfectly matches their data model, it becomes difficult to build data abstractions' - Rosenthal

“We think one of the reasons there are so many NoSQL databases on the market is because unless your application perfectly matches their data model, it becomes difficult to build data abstractions,” he says.

Take MongoDB: if you’re building an application for mobile phones and want to add location, you’d need a spatial index and any changes to documents would need to be made to both the documents and index. Without transactional capabilities, you cannot update the index data reliably and you have to go through the extra step of writing business and application logic. A transactional database would let you update the data and index and have confidence in the result.

Helping glue FoundationDB together is Flow, a new language that extends C++ with the addition of 10 Erlang keyword-types and an actor model concurrency for concurrent calculation. It took Rosenthal’s team three years to design and build Flow’s tools and the mathematical algorithms but just two weeks to actually build Flow, Rosenthal claimed. “Erlang has some really great programming tools for distributed systems but it’s sort of slow; C++ is fast but doesn’t have great tool. Flow ads tools and lets us write in a concise way,” Rosenthal said.

“The first year-and-a-half of development was spend exclusively in simulation – there was no real code to talk to a real network or hard disk, everything was simulated. Once we got programmatically solid in the simulation we started replacing components with their real world counterparts, and then we started working on the performance, which is crucial; performance can always be improved.”

The program was tested during hundreds of thousands of simulations run at night, with failures such as lost packets and rebooted machines simulated. Tests were run on a pair of 96 core clusters using a micro SSD and Intel-based servers.

Looking ahead, Rosenthal doesn’t see FoundationDB as replacing NoSQL or relational approaches, nor does he believe Flow will replace SQL. But he does see FoundationDB getting picked up by web start-ups and already established companies who want to grow rather than those trying to do something like replace an Oracle database.

The Stonebraker factor

You might not be surprised to learn, having read this, that Rosenthal agrees with relational database pioneer Michael Stonebraker – one of the main architects of the first relational database, Ingres, and the object-relational DBMS PostgreSQL. Stonebraker has been flamed by the pups of NoSQL for saying they are wrong to dump ACID, while he’s now behind VoltDB – a post-relational database. “A distributed data store without concurrency control is a toy and makes building things on it a lot harder,” he says.

FoundationDB is emerging as a new breed of NoSQL provider that, from the start, realises the rules of decades of computing have survived for a reason. As far as Rosenthal is concerned, FoundationDB will help bring those familiar with relational into the new world of web-scale NoSQL without making trade-offs.

“Relational is not going away - there’s always going to be apps for it,” Rosenthal says. “What we hear from lot of the people we talk to is they are looking at possibly using NoSQL and they know NOSQL will become more solid, but they know SQL will scale eventually - so they are hedging.

“We think our early adopters are going to be developers thinking about using other NoSQL systems and they’ll look at this and say the performance is great and it has full transaction. That’s worth switching to,” he said. ®

* The CAP theorem states that any networked shared-data system can have at most two of three desirable properties:

  • Consistency - equivalent to having a single up-to-date copy of the data;

  • high Availability of that data (for updates); and
  • tolerance to network Partitions.

** The ACID principles:

  • Atomicity - a transaction is all or nothing
  • Consistency - only valid data is written to the database
  • Isolation - pretend all transactions are happening serially and the data is correct
  • Durability - what you write is what you get

Remote control for virtualized desktops

More from The Register

next story
Euro Parliament VOTES to BREAK UP GOOGLE. Er, OK then
It CANNA do it, captain.They DON'T have the POWER!
Download alert: Nearly ALL top 100 Android, iOS paid apps hacked
Attack of the Clones? Yeah, but much, much scarier – report
NSA SOURCE CODE LEAK: Information slurp tools to appear online
Now you can run your own intelligence agency
Post-Microsoft, post-PC programming: The portable REVOLUTION
Code jockeys: count up and grab your fabulous tablets
Twitter App Graph exposes smartphone spyware feature
You don't want everyone to compile app lists from your fondleware? BAD LUCK
Microsoft adds video offering to Office 365. Oh NOES, you'll need Adobe Flash
Lovely presentations... but not on your Flash-hating mobe
prev story

Whitepapers

10 ways wire data helps conquer IT complexity
IT teams can automatically detect problems across the IT environment, spot data theft, select unique pieces of transaction payloads to send to a data source, and more.
Getting started with customer-focused identity management
Learn why identity is a fundamental requirement to digital growth, and how without it there is no way to identify and engage customers in a meaningful way.
How to determine if cloud backup is right for your servers
Two key factors, technical feasibility and TCO economics, that backup and IT operations managers should consider when assessing cloud backup.
Reg Reader Research: SaaS based Email and Office Productivity Tools
Read this Reg reader report which provides advice and guidance for SMBs towards the use of SaaS based email and Office productivity tools.
Beginner's guide to SSL certificates
De-mystify the technology involved and give you the information you need to make the best decision when considering your online security options.