Feeds

Google blesses Hadoop with MapReduce patent license

Safety for stuffed elephants

5 things you didn’t know about cloud backup

Three months after securing a patent for MapReduce - the distributed number-crunching platform that underpins its world-spanning infrastructure - Google has granted a license to Apache Hadoop, easing infringement concerns hovering over the MapReduce-mimicking open source project.

Apache legal counsel Lawrence Rosen announced the news with an email to the Apache board late last week, and his note was soon posted to a public mailing list.

Rosen did not immediately respond to a request for comment. Nor did Google.

In mid-January, Google won a US patent for "a system and method for efficient large-scale data processing." The patent - which you can see here - describes a means of splitting data-crunching tasks into tiny sub-tasks and mapping them across distributed machines, before reducing the results into one master calculation.

Google uses this MapReduce setup to crunch data across the massive distributed infrastructure buttressing its online services, and though the platform is proprietary, the company published a research paper describing its basic setup in December 2004.

This paper - along with a sister paper describing the company's GFS distributed file system - became the basis for Hadoop. The platform was originally developed by Doug Cutting to back his Nutch open source web crawler, and it was eventually open sourced at Apache. Famously, it's named for a yellow stuffed elephant that belongs to Cutting's son.

When Google won its patent, the general assumption among the Hadoop community was that it posed no threat to the open source project. "Google has lots of patents, and it basically has no track record of using those patents offensively, either involving licensing or pursuing people for infringement," said Mike Olson, chief executive of Cloudera, a company that has commercialized Hadoop in Red Hat-like fashion.

Then he pointed out that Google is a member of the Open Invention Network patent pool, which grants licenses for patented technology in an effort to promote Linux. "All of this convinces us that this is a strategic move from Google and not something that is aimed at the head of any Hadoop adopter or satellite company - Cloudera included."

What's more, Google has long used Hadoop as a way of exposing potential hires to its "Big Data" ways. And even if the company did take legal action, you have to wonder how well its patent would hold up. The map and reduce functions described by Google have been a part of parallel programming for decades.

But now, Mountain View has officially eased fears of legal action. Rosen writes in the email: "Several weeks ago I sought clarification from Google about its recent patent 7,650,331 ["System and method for efficient large-scale data processing"] that may be infringed by implementation of the Apache Hadoop and Apache MapReduce projects. I just received word from Google's general counsel that 'we have granted a license for Hadoop, terms of which are specified in the CLA [contributor licensing agreement].'"

It's unclear what the terms are. But they seem to have passed muster. "I am very pleased to reassure the Apache community about Google's continued generosity and commitment to ASF and open source", Rosen continued.

This is good news not only the likes of Cloudera, but for many of the industry's biggest names as well. The open source Hadoop now underpins everything from Facebook and Yahoo! to, believe it or not, portions of Microsoft Bing. ®

Gartner critical capabilities for enterprise endpoint backup

More from The Register

next story
Why has the web gone to hell? Market chaos and HUMAN NATURE
Tim Berners-Lee isn't happy, but we should be
Apple promises to lift Curse of the Drained iPhone 5 Battery
Have you tried turning it off and...? Never mind, here's a replacement
Microsoft boots 1,500 dodgy apps from the Windows Store
DEVELOPERS! DEVELOPERS! DEVELOPERS! Naughty, misleading developers!
Eat up Martha! Microsoft slings handwriting recog into OneNote on Android
Freehand input on non-Windows kit for the first time
Linux turns 23 and Linus Torvalds celebrates as only he can
No, not with swearing, but by controlling the release cycle
Scratched PC-dispatch patch patched, hatched in batch rematch
Windows security update fixed after triggering blue screens (and screams) of death
This is how I set about making a fortune with my own startup
Would you leave your well-paid job to chase your dream?
prev story

Whitepapers

Best practices for enterprise data
Discussing how technology providers have innovated in order to solve new challenges, creating a new framework for enterprise data.
Implementing global e-invoicing with guaranteed legal certainty
Explaining the role local tax compliance plays in successful supply chain management and e-business and how leading global brands are addressing this.
Advanced data protection for your virtualized environments
Find a natural fit for optimizing protection for the often resource-constrained data protection process found in virtual environments.
How modern custom applications can spur business growth
Learn how to create, deploy and manage custom applications without consuming or expanding the need for scarce, expensive IT resources.
High Performance for All
While HPC is not new, it has traditionally been seen as a specialist area – is it now geared up to meet more mainstream requirements?