Feeds

Intel just installed itself in Hadoop guru Cloudera – and why their rivals should be worried

Chipzilla's distro shelved, cash injected, exec in place

Internet Security Threat Report 2014

Analysis Intel is canning its own Hadoop distribution in favor of using Cloudera's flavor of the analytics software. And the chip giant is making a "significant equity investment" in the upstart that's higher than had been reported elsewhere.

This strategic move was announced by Intel and Cloudera on Thursday following years of heavy investment by Chipzilla into its own Intel Distribution for Apache Hadoop, which will now be put to sleep. Some of its code will be merged into Cloudera's tech.

The partnership is a big bet by Intel that the future of data storage and computation lies in the Hadoop data processing framework – an open-source technology originally developed by engineers at Yahoo! in 2005 in response to two Google papers describing software it had built to deal with a deluge of online data.

One of the creators of Hadoop, Doug Cutting, works for Cloudera, which formed in 2009 to commercialize the technology. These days, Cloudera can be found helping companies such as AMD shift workloads out of traditional systems like Oracle Database and into Hadoop.

As a result of the partnership, Intel will keep its engineers assigned to Hadoop development, and put an exec on Cloudera's board of directors, as well as buying a stake in the upstart.

"Hadoop will be the application that is deployed across more servers than any other enterprise application," said Intel's data center group chief Diane Bryant on a webcast discussing the coupling.

Cloudera announced last week that it raised $160m in a round of funding from companies including T Rowe Price and Google Ventures.

Next week, Cloudera will announce the completion of an additional round of fundraising that will include an investment from Intel. The chip maker has already put out a statement about that, noting: "This is Intel's single largest data center technology investment in its history."

A Venturebeat scoop on Wednesday indicated Intel will pour as much as $90m into the company, but we've heard it's far more.

"I believe those people are underestimating the amount of investment," Jason Waxman, chief of Intel's cloud infrastructure group, told El Reg on Thursday. "The rumor's floating are inaccurate," Cloudera chief Tom Riley told us when we asked about the figure.

There, there, Oracle. Dry your tears on these dollar bills

Placing an as-yet-unnamed Intel executive on the board of Cloudera will have huge ramifications for other IT giants keen to exercise influence over the software distribution.

That board-level move will give x86 overlord Intel valuable intelligence and tighten its grip on the steering wheel of Hadoop, perhaps guiding the software away from other processor architectures, such as those designed by Oracle and IBM (SPARC/Power) and ARM.

When we asked Cloudera what would happen, hypothetically, if a non-x86 chip company came up with a briefcase of cash wanting a board seat.

"That hypothetical would be awkward," Riley said. "We're partnering with the industry leader. The industry leader has given us enough money."

Almost all of Cloudera's customers are running on Intel hardware, Riley said.

ClouderaIntel

Hadoop troupe ... Intel's Diane Bryant and Cloudera's Tom Riley

The partnership means Intel and Cloudera will work together to add some of the software the chipmaker wrote for its own distribution to Cloudera's source code base, as well as the main Apache Hadoop project.

"We've taken two great distributions and are combining the best of them into one," said Riley on the webcast. "Intel is bringing engineers, Intel is bringing field resources to this relationship."

There will be "lots and lots of engineers on both sides," Riley added to El Reg afterwards. "We're building a joint strategic partnership office team members assigned to it."

Intel will hook the Hadoop code up to architecture extensions in its chips, such as Trusted Execution Technology, tightening Cloudera's hardware integration. It also plans to do a lot of work tying Hadoop into its "Internet of Things" low-power silicon as well, Waxman told us.

Cloudera has steadily increased its share of the Hadoop market through a combination of expertise and marketing. It competes with rivals like Hortonworks, MapR and VMware's spinoff Pivotal.

Intel's move is significant as it's trying to exercise control over an open-source technology that has the potential to boost data-processing capabilities within data centers and subtly shape the hardware market. Just as Linux shook up how servers and server clusters are built, Hadoop may do the same for processors and Intel wants to be there. ®

Internet Security Threat Report 2014

More from The Register

next story
Docker's app containers are coming to Windows Server, says Microsoft
MS chases app deployment speeds already enjoyed by Linux devs
IBM storage revenues sink: 'We are disappointed,' says CEO
Time to put the storage biz up for sale?
'Hmm, why CAN'T I run a water pipe through that rack of media servers?'
Leaving Las Vegas for Armenia kludging and Dubai dune bashing
'Urika': Cray unveils new 1,500-core big data crunching monster
6TB of DRAM, 38TB of SSD flash and 120TB of disk storage
Facebook slurps 'paste sites' for STOLEN passwords, sprinkles on hash and salt
Zuck's ad empire DOESN'T see details in plain text. Phew!
SDI wars: WTF is software defined infrastructure?
This time we play for ALL the marbles
Windows 10: Forget Cloudobile, put Security and Privacy First
But - dammit - It would be insane to say 'don't collect, because NSA'
prev story

Whitepapers

Forging a new future with identity relationship management
Learn about ForgeRock's next generation IRM platform and how it is designed to empower CEOS's and enterprises to engage with consumers.
Cloud and hybrid-cloud data protection for VMware
Learn how quick and easy it is to configure backups and perform restores for VMware environments.
Three 1TB solid state scorchers up for grabs
Big SSDs can be expensive but think big and think free because you could be the lucky winner of one of three 1TB Samsung SSD 840 EVO drives that we’re giving away worth over £300 apiece.
Reg Reader Research: SaaS based Email and Office Productivity Tools
Read this Reg reader report which provides advice and guidance for SMBs towards the use of SaaS based email and Office productivity tools.
Security for virtualized datacentres
Legacy security solutions are inefficient due to the architectural differences between physical and virtual environments.