Feeds

Teradata update chews up, spits out columns

Plus: SQL-MapReduce alliance appliance

Designing a Defense for Mobile Applications

Data warehousing pioneer Teradata is turning up the dial on its eponymous parallel database to 14, adding in slew of new features that include the ability to process columnar data as well as the more standard-row-based chewing in relational databases. The company has also released an appliance running the hybrid row/column nCluster database that it got through its acquisition of Aster Data in March for $263m.

Scott Gnau, president of Teradata Labs, tells The Reg that the columnar support in Teradata 14 is not based on Aster Data nCluster, but is rather a tweak of the Teradata database and its underlying file system. The work on this feature predates the Aster deal, and has been in development for the past 18 months.

"We figured out a better mousetrap to provide columnar structures in the database," says Gnau. "This is a really big deal."

Columnar databases have been around for three decades, says Gnau, and can provide a significant speed-up for accessing and – sometimes more importantly – compressing certain kinds of data stored in a database. For example, if your iTunes library could be organized into columns, you could search the Beatles column for and get to a tune very quickly in the database. Otherwise, you'd have to search each song in the database for an artist named Beatles.

However, adding or updating data in columnar databases can be hard, Gnau says, which is why they are not prevalent in commercial computing. Yet.

Instead of building a separate columnar database and bolting it onto the side of the data warehouse, Teradata' feature makes use of changes to the Teradata file system that allow it to store a column-based table, just like it currently stores row-based tables.

The optimizers in the Teradata system are aware of columnar and row-based data, and you can mix and match columnar and row tables within the same database and even create a hybrid table that can be searched by row or column. This hybrid table is created by taking a row table and designating a couple of key columns for searches within it. The columnar tables in Teradata 14 can have normal database functions such as joins performed on them, and they can be compressed using run length, dictionary, trim, delta on mean, null, and UTF8 methods.

Teradata 14 will also get a bunch of new SQL commands that help with application portability from other database-management systems, and will also support a number of new ARRAY and NUMBER data types not previously supported without having to convert that data into a native format.

Teradata has also added row-level access controls to the database to enhance its security, and temporal data-analysis tools can now present data across multiple time zones as if they were in one time zone – a multinational company can, for example, look at all of its global operations on a single day or week during normal business hours across each zone.

The database also now has block-level data compression, and based on the hotness or coldness of the data it will compress or unzip data. Cold data can be migrated to slower storage in the cluster, should you have any.

The columnar support will be available in December when Teradata 14 ships. The new database is in beta testing now at Teradata customers.

Aster Data appliance for MapReduce

Unstructured data is building up all over the data centers of the world, particularly from web-page clicks, and every company believes that they have to keep every bit of data – structured or unstructured – because it may someday be useful.

Aster Data's SQL-MapReduce and the way it handles these steaming heaps of structured and unstructured data made a it particularly interesting acquisition for Teradata.

Well, there's also the fact that Teradata had to keep it from falling into the enemy hands of Oracle, IBM, or EMC; and that the nCluster hybrid row/column database runs on parallel x86 clusters, just like the Teradata software – in fact, both companies use Dell PowerEdge hardware. But we digress...

The news today is that Teradata is putting out Aster Database 5.0, an update to that SQL-MapReduce hybrid database, as well as plunking it and MapReduce onto a preconfigured appliance called – you guessed it – the Aster MapReduce Appliance.

Teradata Aster Appliance

Teradata's Aster MapReduce Appliance

If you don't want to buy the appliance, Aster Database 5.0 release sans appliance is certified to run on Red Hat and SUSE Linuxes on x86 servers. (Aster did RHEL, Teradata does SLES.)

The SQL-MapReduce framework that is part of the Aster Database allows for SQL programs or other business-analytics tools to make MapReduce calls to chew on data and use the results of that processing in their own SQL transactions. With the 5.0 release of the database, Teradata has tweaked the system-management software to do a better job of allocating memory in the cluster for SQL and MapReduce processes. It also includes pre-built MapReduce modules for analyzing clickstream behavior or doing marketing analysis, among other things.

The Aster MapReduce Appliance is based on two-socket Xeon 5600 server nodes with six-core Xeon X5675 processors running at 3.06GHz. The nodes are linked together using 10 Gigabit Ethernet switches, and the software stack – which includes the Aster Database at the 4.6.2 release or higher running on SUSE Linux Enterprise Server 11 – has the Aster-Teradata adapter that allows for data to be sucked out of a Teradata data warehouse to be chewed by the Aster database. The Aster MapReduce appliance scales up to six racks of servers, yielding more than 200TB of user space when compression is turned on.

"The appliance is all about the TCO, not the TCA," says Gnau, who adds that customers were asking Teradata to build a pre-configured system. "Because we are bundling and optimizing it, you can buy less hardware than you would in a cobbled-together solution."

The Aster MapReduce Appliance will ship in the first quarter of next year. Prices for the getup have not been set yet. ®

The Power of One eBook: Top reasons to choose HP BladeSystem

More from The Register

next story
Apple fanbois SCREAM as update BRICKS their Macbook Airs
Ragegasm spills over as firmware upgrade kills machines
Attack of the clones: Oracle's latest Red Hat Linux lookalike arrives
Oracle's Linux boss says Larry's Linux isn't just for Oracle apps anymore
THUD! WD plonks down SIX TERABYTE 'consumer NAS' fatboy
Now that's a LOT of porn or pirated movies. Or, you know, other consumer stuff
EU's top data cops to meet Google, Microsoft et al over 'right to be forgotten'
Plan to hammer out 'coherent' guidelines. Good luck chaps!
US judge: YES, cops or feds so can slurp an ENTIRE Gmail account
Crooks don't have folders labelled 'drug records', opines NY beak
Manic malware Mayhem spreads through Linux, FreeBSD web servers
And how Google could cripple infection rate in a second
FLAPE – the next BIG THING in storage
Find cold data with flash, transmit it from tape
prev story

Whitepapers

Designing a Defense for Mobile Applications
Learn about the various considerations for defending mobile applications - from the application architecture itself to the myriad testing technologies.
How modern custom applications can spur business growth
Learn how to create, deploy and manage custom applications without consuming or expanding the need for scarce, expensive IT resources.
Reducing security risks from open source software
Follow a few strategies and your organization can gain the full benefits of open source and the cloud without compromising the security of your applications.
Boost IT visibility and business value
How building a great service catalog relieves pressure points and demonstrates the value of IT service management.
Consolidation: the foundation for IT and business transformation
In this whitepaper learn how effective consolidation of IT and business resources can enable multiple, meaningful business benefits.