Original URL: http://www.theregister.co.uk/2009/09/15/oracle_sun_exadata_v2/

Oracle, Sun speed-launch Exadata V2

So much for HP iron, Sparcs MIA

By Timothy Prickett Morgan

Posted in HPC, 15th September 2009 23:57 GMT

If Oracle is trying to convince Sun customers that it is committed to the Sparc platform, perhaps it is not trying hard enough.

As it turns out, "the world's first OLTP database machine with FlashFire technology" that El Reg told you last weekend was coming and that Oracle and Sun trotted out today is based on Sun's "Galaxy" family of x64 blade servers, not Sun's Sparc T series processors and their related "Niagara" family of boxes.

There's nothing wrong with creating tightly coupled, application-specific x64 blade server stacks, complete with Oracle database software, fast interconnects for linking server nodes, and fast storage like the flash-based arrays that Oracle and Sun announced today as the next Exadata database appliance. What is wrong, however, is that there is not a Sparc companion product.

Sun has said in advertisements that started running at the beginning of September that it would pit a Sun Sparc setup against IBM's Power 595 and best it on the TPC-C online transaction processing test. Last week, Oracle reaffirmed its commitment to the Sparc processor and its servers in another ad, so presumably there is a Sparc T5440 with FlashFire technology (or some variant thereof) that we get to see at Oracle's OpenWorld customer event on October 14.

Today's Exadata V2 launch event started 15 minutes late, if you missed the Webcast launch (you can see the replay here when it becomes available), and started with an "extreme performance" theme that showed Captain Larry Ellison and Oracle's World Cup catamaran striking impressive poses, and then cut to a live Ellison in the studio, who doesn't do his own clicking during presentations and started out by admonishing some worker with "Next slide, please, I already know who I am."

Just in case you forgot who Larry Ellison, co-founder and chief executive officer of Oracle, is. That minute or so kinda captures it perfectly, and so does the impatient pace of the whole event, come to think of it.

It seems like Oracle has forgotten what its own product names were. Ellison kept referring to Exadata V1, a data warehousing configuration that Oracle cooked up this time last last as the HP Oracle Database Machine, a set of HP x64 servers running Linux, the Oracle 11g database, and the Real Application Server clustering extensions to the database; the Exadata part of the solution was the clustered file system running on storage servers that fed these machines.

But now that Oracle is in the process of acquiring Sun Microsystems - which was snubbed as the data warehousing appliance partner last fall when the Database Machine was announced - Oracle doesn't seem capable of saying Database Machine V2.

A place in the Sun

Interestingly, as Ellison started off his presentation, I could have sworn I heard him refer to "Sun Computer," not Sun Microsystems, in saying that this was the next product to come from a long partnership with Sun. When you buy so many companies, perhaps it is hard to keep it all straight. And you can probably afford to edit that mistake out of the replay before putting it up on the Web, too.

No matter what you call it, what is now being called Exadata V2 is better, stronger, and faster, and it is based on Sun hardware running Oracle software that is tuned to yield very impressive performance and scalability on database workloads.

While Exadata V1 (as El Reg will call the original HP clusters from this point on, as Oracle seems to be doing) was designed and tuned for data warehousing jobs and the massive amounts of sequential reads off of massive amounts of disk drives that data warehousing requires, Exadata V2 embeds flash drives into the storage systems and has optimizations for handling the random I/O reads and writes that online transaction processing (OLTP) systems embody.

Thanks to lots of software tweaks in the Oracle software stack that automatically move data from the hybrid flash-SATA disk storage arrays and into main memory in the server nodes (across a 40 Gb/sec InfiniBand fabric that links all of the server nodes to each other and to the storage), the new system can process over 1 million random I/O reads per second.

So, in sum: Exadata V1 is for data warehousing, Exadata V2 is for data warehousing or OLTP. Unlike all of those specialized data warehousing products sold by Teradata and Netezza or the in-memory OLTP database appliances that venture capitalists, according to Ellison, may be sorry they invested in.

While this may be technically true, there is no reason why the new storage servers embedded in the Exadata V2 product can't be hooked up to other x64 or RISC/Unix servers running the Oracle database and RAC extensions. And if Oracle wanted to, it could use other storage products with flash and do all kinds of optimizations to get data moving back and forth fast enough to support OLTP as well as data warehousing workloads.

If Oracle does indeed succeed in buying Sun - and that is almost a certainty - it seems highly unlikely that such optimizations will be done on non-Oracle iron. Right up until the minute that HP starts partnering with IBM and Microsoft to do similar optimizations to take advantage of flash with their databases.

Exadata V2

The Exadata V2 setup has three parts. For the server nodes that process queries and transactions, Oracle and Sun are using a two-socket Sun Fire X4170 server based on Intel's quad-core Xeon 5500 processors, that has 72 GB of memory. The X4170 is a 1U rack server, one of the Xeon 5500 boxes that Sun announced in April.

The eight Nehalem cores in the X4170 database server offer about twice the performance of the prior HP iron, according to John Fowler (who used to be senior vice president and general manager of Sun's Systems group but who Ellison referred to as the person who "heads up engineering at Sun"). And that's 2.5 times more memory (and a lot more memory bandwidth thanks to QuickPath Interconnect) compared to the earlier database appliance. The X4170 server has redundant paths to the InfiniBand switches that link it all together. Up to eight of these database servers can be put into an Exadata V2 rack.

The storage server portion of this setup does not include the F5100 flash array that Sun has been working on and that El Reg told you about last week. The storage array used in Exadata V2 is based on an X4275 server. The X4275 is a 2U rack server with room for a dozen 2.5-inch, 600 GB SAS drives, and it offers twice the capacity of the storage node used in the prior database machine. (Ellison said he was pushing Sun's engineers to use 2 GB SATA disks because the flash more than made up for the slowness of the SATA drives.) The secret sauce in this storage array, the FlashFire technology, seems to be a PCI-Express controller with two or perhaps four flash modules on it that offer up to 96 GB of flash capacity in total.

Each storage node in the Exadata 2 device can have four of these, and there are a total of fourteen in a rack, for a total of 56 flash units and over 5 TB of aggregate memory capacity. According to Fowler, each one of these flash drives yields the same I/O performance as somewhere between 200 and 300 disks.

And because of the use of flash drives, the Exadata 2 machine yields roughly twice the database performance, but consumes 14 per cent less juice. And it apparently costs less, too. The storage nodes can support up to 336 TB of data using 2 TB disks, and Ellison recommends putting the fattest disks possible in the X4275 because the flash drives have so much I/O that, given the tweaks Oracle has done in the storage areas of the database, the slowness of these fat disks is irrelevant.

With the compression algorithms Oracle has created for the Exadata box, a 4TB database can be compressed into main memory and a 15TB database can be compressed onto flash memory in a single-rack configuration.

The third element of the Exadata 2 setup is Sun's InfiniBand Switch 36, the low-end, fixed-port 40 Gb/sec switch that Sun debuted back in June aimed initially at supercomputer shops.

It is interesting that Sun and Oracle didn't choose the top-end 648-port switch. The important thing is that customers can add up to eight racks of the Exadata 2 machines together, clustering them to share workloads and do so "without adding any wires," as Ellison put it. The Exadata V1 machine used 20 Gb/sec InfiniBand switches from Voltaire.

The Exadata 2 database machine comes in four configurations. The base setup includes a rack, one database server, one storage server, and one switch. The database server has Oracle 11g Release 2 and Linux on it and the storage server has Exadata Storage Server Release 11.2 managing the files on the disks and flash. This base system costs $110,000. A quarter-rack system, which has two database servers and three storage servers, plus the rack and switch, costs $350,000. A half-rack, which has four database servers and seven storage servers plus a bunch of InfiniBand switches runs to $650,000, and a full rack costs $1.15m.

Stay tuned for Oracle's competitive analysis about how this stacks up to other data warehousing and OLTP boxes. ®