Feeds

Microsoft brings Hadoop option to SQL Server

Open-source crunchware SQOOPs to conquer

Securing Web Applications Made Simple and Scalable

Microsoft customers running SQL Server are getting a taste of really big data processing through an injection of Hadoop.

The company has released early code that will let Microsoft customers plug the open-source Java architecture from Doug Cutting into SQL Server 2008 R2, SQL Server Parallel Data Warehouse for huge data warehouses, as well as the next version of Microsoft's database, which is codenamed Denali.

Hadoop was built by Cutting, who was inspired by Google's MapReduce. It is becoming something of an industry standard for processing huge amounts of data on clustered servers thanks to the fact that its code is open. Hadoop has also been adopted by top-tier web properties including Amazon, Facebook and Twitter.

The industry thinking is that Hadoop can trickle down to customers outside the rarefied circles of serious number-crunchers, where it is used to understand the changing minutiae of millions of users' likes and status updates in order to change services in response. The aim is for Hadoop to find its feet in more mainstream IT.

Microsoft's Research unit has been working on something that sounds remarkably similar to Hadoop, called Dryad, since about 2006. Earlier this year the plan was to "productise" Dryad through integration with SQL Server and its Windows Azure cloud. There have been no updates from Microsoft, but it seems Dryad must now compete for the affections of big-data lovers on SQL Server.

The Microsoft connectors are called Hadoop Connector for SQL Server Parallel Data Warehouse and Hadoop Connector for SQL Server and are available as Community Technology Previews (CTPs).

The connectors are two-way, letting you move data backwards and forwards between Hadoop and Microsoft's database servers

Microsoft said the connectors would let its customers analyse unstructured data in Hadoop and then pull that back into the SQL Server environments for analysis.

Both connectors use SQL to Hadoop (SQOOP) to transfer the data "efficiently" between the Hadoop File System (HDFS) and Microsoft's relational databases. The Parallel Data Warehouse uses PDW Bulk Load/Extract tool for fast import and export of data.

SQL Server PDW customers can get the Hadoop connector from Microsoft while users of the regular SQL Server 2008 R2 can get the code for Hadoop Connector for SQL Server here. ®

The Essential Guide to IT Transformation

More from The Register

next story
EU's top data cops to meet Google, Microsoft et al over 'right to be forgotten'
Plan to hammer out 'coherent' guidelines. Good luck chaps!
Manic malware Mayhem spreads through Linux, FreeBSD web servers
And how Google could cripple infection rate in a second
FLAPE – the next BIG THING in storage
Find cold data with flash, transmit it from tape
Seagate chances ARM with NAS boxes for the SOHO crowd
There's an Atom-powered offering, too
Intel teaches Oracle how to become the latest and greatest Xeon Whisperer
E7-8895 v2 chips are best of the bunch, and with firmware-unlocked speed control
Gartner: To the right, to the right – biz sync firms who've won in a box to the right...
Magic quadrant: Top marks for, er, completeness of vision, EMC
prev story

Whitepapers

Top three mobile application threats
Prevent sensitive data leakage over insecure channels or stolen mobile devices.
The Essential Guide to IT Transformation
ServiceNow discusses three IT transformations that can help CIO's automate IT services to transform IT and the enterprise.
Mobile application security vulnerability report
The alarming realities regarding the sheer number of applications vulnerable to attack, and the most common and easily addressable vulnerability errors.
How modern custom applications can spur business growth
Learn how to create, deploy and manage custom applications without consuming or expanding the need for scarce, expensive IT resources.
Consolidation: the foundation for IT and business transformation
In this whitepaper learn how effective consolidation of IT and business resources can enable multiple, meaningful business benefits.