Gurus answer MapReduce young turks

Database blues and grays

By Phil Manchester • Get more from this author

Posted in Software, 1st February 2008 21:15 GMT

Free whitepaper – Fundamental Principles of Generators for Information Technology

The two database gurus whose blog produced a storm of protest over their criticism of Google's MapReduce technology last month have hit back with a robust defense.

In their latest joint posting Michael Stonebraker and David DeWitt have respond to four specific issues raised by their critics: that MapReduce is not a database and, therefore, should not be judged as such, that MapReduce is scalable, that MapReduce is cheap, and - finally - that the two are merely "the old guard" defending traditional database management (DBMS) technology.

They claim that the original critique never said that MapReduce is a database - only that it is capable of performing some of the same tasks - specifically the analysis of large datasets. It is, therefore, legitimate to make the comparison between the two and conclude that "most things that are possible in MapReduce are also possible in a SQL engine".

On the scalability of MapReduce, Stonebraker and DeWitt note that the jury is still out on this and "as far as we are aware there are no published papers that study the scalability of MapReduce in a controlled scientific fashion". They also dismiss the claim that DBMS is expensive compared to MapReduce - pointing out open source alternatives to expensive commercial products.

Finally, they admit to being "gray beards" who "have seen a lot of ideas come and go" but add that gray beards and young turks should not be adversaries.

The initial response to Stonebraker and DeWitt's defense indicates that the argument looks set to continue. While a few commenters have reacted positively, the majority remain unconvinced.®

Free whitepaper – Systems management simplified