Is the relational database now a commodity?

Your thoughts on the past and present

By Dale Vile, Freeform Dynamics, 16th April 2007

Reg Reader Workshop For those of us who were around in the industry during the mid to late 80s, it is interesting to think back to a time when vendors of relational database management systems (RDBMS) were struggling to be taken seriously.

The line up then was DB2, Ingres, Informix, Oracle and Sybase, and back in those days RDBMS specialists took pride in being able to tune engines and database designs to get the most out of these very special beasts. In expert circles, debates would rage over the relative merits of multi-threaded versus multi-engine servers, row versus page level locking, the efficiency and performance of stored procedures versus the standard nature of raw SQL, and so on.

These were the days when Sybase SQL Server would come pre-configured out of the box with 4MB of cache and many users actually ran with that default. And the largest database size? Well, there was a race in the industry between vendors who could come up with the first reference site with a 1GB database, then Sybase started its "1GB Club", which was made up of some of the largest and most adventurous users.

It is a little sad when us old pioneers hear people talking about the RDBMS now being a commodity, but who are we to argue when you can download and run MySQL - an engine that is much more powerful and versatile than anything we were working with the early days - for zero cost? You can even get a community version of the previously "big iron" DB2 from IBM for free.

There is a counter-argument, however, that while basic RDBMS server functionality to handle structured relational data probably has become commoditised, the world has moved on. From a physical perspective, for example, platform architectures are a lot more sophisticated and varied, and today we have to think about everything from stand alone small footprint servers, through clustering, blade servers, and virtualisation, to full blown grid computing. The type and range of information that needs to be managed is also quite different – XML documents, binary documents, graphics, video, voice and geo-spatial data.

We have then seen the RDBMS assume new roles. While specific analytical servers to support business intelligence are still in demand, for example, more and more organisations are using the analytical features that are now embedded natively in mainstream RDBMSs to build very functional data warehouses (see chart).

Finally, there are the modern-day requirements for openness and interoperability, and the sheer performance, scalability, and resilience characteristics necessary to handle high volume transaction systems, busy 24/7 public facing websites, and highly dynamic data warehouses to quench the ever growing thirst for the latest business performance information among business users.

Against this background, is it really just a case of picking any old RDBMS and running with it for the job at hand? If we had true commoditisation, this would be the case.

So what do you think?

We know you have something to say on how the requirements for database management systems have evolved and what will be important in the future, so give us your thoughts in the discussion areas below. ®

Requirements Checklist for Choosing a Cloud Backup and Recovery Service Provider

COMMENTS

House Rules Send Corrections

All Reader Comments (20)

Latest Comments

Posted Sunday 22nd April 2007 22:35 GMT

Anonymous Coward

DBMS Commoditization - Not So Fast, As One Size Won't Fit All

MIT professor Michael Stonebraker (who architected the INGRES relational DBMS and the object-relational DBMS POSTGRES) believes that DBMS users with modest requirements are likely well served by using one of the popular open source DBMSs. Hence, at the low end, he expects open-source DBMSs to take over.

But he notes that there are several markets where DBMS requirements are going up faster than hardware is getting cheap. Hence, the DBMS problem in these markets is getting harder, not easier, over time.

For more on this topic, see his paper “One Size Fits All: Part 2 – Benchmarking Results." You can find this paper here: http://www.cidrdb.org/cidr2007/papers/cidr07p20.pdf

One example discussed at length in the paper is data warehousing. He notes that business analysts have a seemingly inexhaustable thirst for making business decisions based on more and more data. So warehouse size is going up much faster than disks are getting cheaper.

Also, query complexity appears to go up as the square of warehouse size. Hence, warehouse administrators are in more pain over time, not less.

There are a variety of warehouse-specific code lines that offer a factor of 1-2 orders of magnitude better performance than the "one size fits all" systems from the open source vendors.

Other markets where a specialized architecture will yield comparable advantage include streaming data, scientific and intelligence applications, and text.

Professor Stonebraker will present a paper at the VLDB 2007 conference in September that will argue the same point for the OLTP market.

Hence, whenever a user cares about 1-2 orders of magnitude in performance, he will not be a candidate for an open source product.

Posted Wednesday 18th April 2007 09:38 GMT

Robin

Another word from a developer

I entirely agree with Micha.

But this all depends on the application being developed. I get the impression that nearly all of the DBAs discussing the merits of one DB or another are missing the point in terms of most database driven applications developed today. That is I would estimate that at least 90% of such applications will never have a need for any specialist feature of all the big DB manufacturers because they will never experience such huge demand. I'm talking about small agencies developing small, low volume, websites for many different clients. Such clients couldn't care less about the DB. For such developers systems such as Hibernate, Ruby on Rails or Subsonic are a godsend exactly because they cut out the entire DB layer. Sure Yahoo or Amazon will never be able to run their systems using off the shelf ORM tools, but then most applications are not Yahoo or Amazon.

Where I work I am the only developer which means I have to do everything from front end HTML and JavaScript to DB administration. I am not an expert in any of these but am sufficiently capable to work well in all of these areas. We use MySql and it is perfect for our needs because a) it is cheap and b) it is quick (or at least quick enough). We may soon be moving to MS Sql Server because this integrates better with .Net framework we use. 3 years ago we simply honestly didn't have the cash to spend 5k on a DB. Now we do. What I am trying to illustrate here is the needs and possibilities of the developer and his client are the biggest factor in making these decisions.

If you have the cash to employ a full time DBA then obviously you'd better make sure you are using the most appropriate DB. Until such a time the question is not relevant in my view. And if you run into DB optimisation issues before you can employ a full time DBA then there is something wrong with your business plan!

Robin

Posted Tuesday 17th April 2007 19:52 GMT

paddy carroll