Time to reject traditional database techniques?
'Big' data and the BI challenge
Specialized databases such as those built by Google and Yahoo, data warehouse software such as Vertica and Monet and innovative DBMS such as H-Store were all claimed to outperform relational DBMS products. Even in Online Transaction Processing (OLTP) - the traditional strong point of classic relational databases - H-Store was claimed to perform better.
At the heart of the argument against relational DBMS are, firstly, what are seen as the limitations of the old relational model and SQL and, secondly, how they may either be upgraded or replaced. Some argue in favor of new approaches such as the MapReduce technology used by Google to power its massive search engine operations. Others hold true to the transactional integrity and ACID properties built into traditional DBMS such as IBMS DB2, Oracle, and Microsoft's SQL Server.
Even the DBMS gurus can be confusing. While advocating new approaches to DBMS in the 2007 VLDB paper, Stonebraker provoked a storm in January when he co-authored a critique of Google's MapReduce that was widely acknowledged as a "new approach".
One of Stonebraker's criticisms of MapReduce was the lack of SQL-like tools. The omission was remedied in August by newcomers Aster Data and Greenplum - so it appears that there's still a need for at least some bits of relational DBMS technology.
But even MapReduce has limitations. Recent analysis carried out by eBay revealed some resource usage problems.
The future of DBMS technology rests on a combination of tried-and-tested techniques from the past and innovations to cope with large data volumes and more demanding users.
The recent announcements from Oracle and Microsoft embody some of the changes that point towards some sort of consensus on future development of DBMS. Oracle's Exadata and Microsoft's Kilimanjaro take on ideas from more modern approaches to DBMS and fold them into the tradition.
Oracle and Microsoft's new plans also include in-memory processing, massively-parallel processing, and the column-storage approach used in data warehouse products such as Sybase IQ and, more recently, Vertica, and Google's BigTable
SQL and the relational model appear, it seems, are positioned to survive intact. ®