Feeds

In praise of FPGAs

Building an active data warehouse appliance

7 Elements of Radically Simple OS Migration

All data warehouse appliances have a massively parallel architecture in which there are multiple nodes that put processing as close as possible to the disk drives.

In Netezza's Performance Server these nodes are known as Snippet Processing Units (SPUs: pronounced, incidentally, to rhyme with Gnu rather than being a homonym for throwing up - though Teradata might think otherwise) where a snippet is executable machine code (compiled SQL).

In hardware terms an SPU consists of an FPGA (field programmable gate array) and a PowerPC processor. Now, when vendors talk about using commodity hardware neither of these are the sort of things that spring to mind.

Nevertheless, FPGAs are widely used for graphics processing, video and audio streaming and other applications, while PowerPC chips are commonly employed in small profile devices. In particular, PowerPC chips have very low power requirements compared to PC chips: for example, a Netezza rack holding 5.5Tb of user data (16.5Tb actual capacity) has a power rating of just 4,180 watts and, of course, there is commensurately low heat output. Given the current pressures on data centres this is a significant advantage that Netezza has not made as much of as it might.

The way the FPGA works in Netezza's architecture is that it effectively works like an event processing engine: data is streamed from disk and then passes through the query that is represented by the snippet (and is loaded into the FPGA). I had not previously appreciated this until a recent visit to Netezza's user conference but you can imagine (if you are a regular reader) my interest.

While I was in Boston for Netezza's conference, I also had a meeting with Kaskad Technologies, which is a new entrant into the event processing space, helping to build the new fully electronic Boston Stock Exchange, which goes live on 27 October, with its surveillance application.

Now, Kaskad has done some clever things with its event processing product but it is more or less comparable to the conventional vendors (Progress, StreamBase, Coral8 et al) in architectural terms. However, what is much more interesting is that the company is already working on its second generation of event processing architecture. This will involve the use of multiple finite state machines running in parallel, hanging off a single bus.

In other words, instead of having a single large server you have multiple small servers running in parallel. And how is Kaskad planning to implement these small servers? Using FPGAs. And bearing in mind that FPGAs are the technology of choice for streaming applications this obviously makes sense.

Now, readers may recall that I recently suggested that a better way to do what Teradata calls active data warehousing (whereby you use change data capture to trickle feed the warehouse and then run actionable queries and processes) would be to front-end an event processing engine on to a data warehouse appliance. In fact, Sybase (albeit that Sybase IQ is not an appliance) is doing exactly this with its Risk Analytics Platform, for which Sybase is talking to a number of event processing vendors about front-ending their products.

However, the use of FPGAs opens up another possibility. Why not have different types of FPGA within the same architecture? You could have some FPGAs processing data that was streamed from disk and other FPGAs that processed data that was streamed from external environments off a bus.

In other words, you could build an active data warehouse appliance. Or, looking at it another way: an integrated event processing/warehouse environment. Given that you often want to access historic information for event processing purposes this could be an architecture that makes sense.

Copyright © 2006, IT-Analysis.com

Best practices for enterprise data

More from The Register

next story
Microsoft's Euro cloud darkens: US FEDS can dig into foreign servers
They're not emails, they're business records, says court
Sysadmin Day 2014: Quick, there's still time to get the beers in
He walked over the broken glass, killed the thugs... and er... reconnected the cables*
VMware builds product executables on 50 Mac Minis
And goes to the Genius Bar for support
Multipath TCP speeds up the internet so much that security breaks
Black Hat research says proposed protocol will bork network probes, flummox firewalls
Auntie remains MYSTIFIED by that weekend BBC iPlayer and website outage
Still doing 'forensics' on the caching layer – Beeb digi wonk
Microsoft says 'weird things' can happen during Windows Server 2003 migrations
Fix coming for bug that makes Kerberos croak when you run two domain controllers
Cisco says network virtualisation won't pay off everywhere
Another sign of strain in the Borg/VMware relationship?
prev story

Whitepapers

7 Elements of Radically Simple OS Migration
Avoid the typical headaches of OS migration during your next project by learning about 7 elements of radically simple OS migration.
Implementing global e-invoicing with guaranteed legal certainty
Explaining the role local tax compliance plays in successful supply chain management and e-business and how leading global brands are addressing this.
Consolidation: The Foundation for IT Business Transformation
In this whitepaper learn how effective consolidation of IT and business resources can enable multiple, meaningful business benefits.
Solving today's distributed Big Data backup challenges
Enable IT efficiency and allow a firm to access and reuse corporate information for competitive advantage, ultimately changing business outcomes.
A new approach to endpoint data protection
What is the best way to ensure comprehensive visibility, management, and control of information on both company-owned and employee-owned devices?