Oracle cranks Exalogic software stack up to 2
More oomph without bending new iron
Oracle could have launched its new Exalogic hardware in conjunction with the new rev of its middleware software stack, as El Reg speculated earlier this week , but the software giant decided it didn't need the latest Xeon E5 processors from Intel to boost the performance of its Exalogic "engineered systems." Instead, it rolled out a virtualized version of the Exalogic stack that delivers substantially better performance than the prior bare-metal version, as well as increased manageability.
That probably means that new Exadata and Exalogic hardware is coming at Oracle OpenWorld, which will be held in San Francisco from September 30 through October 4.
The big change with the Exalogic 2.0 software announced today is that it has been propped up on top of Oracle VM, which is Larry Ellison's riff on the open source Xen server virtualization hypervisor.
The prior Exalogic 1.0 software stack ran on bare-metal x86 servers with Solaris 10, Solaris 11, or Oracle Enterprise Linux, which is Ellison's RHEL-ish variant of Red Hat's commercial grade Linux that can sport an Oracle Unbreakable Enterprise Kernel if you want.
Block diagram of the Exalogic 2.0 software stack
Mike Palmeter, senior director of product management for the Exalogic line at Oracle, tells El Reg that the virtualized Exalogic stack is not yet available on the Solaris variant of Unix, but that Oracle will eventually deliver a variant of the stack supporting Solaris containers for customers who want a cloudy Solaris Exalogic engineered system.
Palmeter also says that it was always Oracle's intent to virtualize the underlying Exalogic servers, and that was one of the reasons why it was called the Exalogic Elastic Cloud, to use the product's full name.
The problem was that network cards and drivers supporting Single Root I/O Virtualization (SR-IOV), which allows for multiple logical domains or virtual machines to share a single physical I/O device on the system bus of a physical server, were not widely available two years ago, when the Exalogic 1.0 machines were announced. In fact, Oracle just announced SR-IOV support  for its Logical Domain hypervisor, sometimes called LDom and other times called Oracle VM for Sparc, in early June.
SR-IOV support is baked into the Oracle VM 3.1 hypervisor that Oracle announced in May , which puts the Xen 4.1.2 hypervisor atop Oracle's Unbreakable Enterprise Kernel R2 . It is this implementation of the Xen hypervisor that underpins the new Exalogic 2.0 software.
Oracle's Exalogic X2-2 hardware
SR-IOV support is important for another reason, in that it allows a virtual machine guest to bypass the hypervisor and talk directly to the network device. On virtualized machines running middleware and databases, the memory and CPU bandwidth issues have largely been solved, but I/O latency issues have been a big problem.
SR-IOV goes a long way toward dealing with I/O latency in a virtualized environment (by basically ignoring the hypervisor). In this case, explains Palmeter, Oracle is doing what it calls PVHVM, or "paravirtualized drivers on a hardware virtual machine."
Basically, PVHVM lets the guest OS talk directly to the host adapter in the physical server through user-space memory. Oracle has worked with its networking partner, Mellanox Technologies (in which it has a 10.2 per cent stake, you'll remember), to get these PVHVM drivers running in the guest OS, which in this case is Oracle Linux. Over time, these drivers will be ported to Solaris, and that's when you will be able to use Solaris containers on Exalogic machines.
Palmeter says that with this new approach, the Oracle VM hypervisor has an overhead of about 1 to 2 per cent of CPU and uses a few gigabytes of main memory, but that "application throughput and latency are essentially unaffected by the hypervisor."
You can live-migrate an image across the InfiniBand fabric inside of a single Exalogic instance, according to Palmeter, but you cannot live-migrate across different fabrics – at least not until a future release of the software. "We know how to do it, and we are working on it," he says.
Oracle is claiming some pretty big performance gains on this Exalogic 2.0 software:
Oracle Exalogic performance over time
Oracle just tossed that slide up during its launch webcast without explaining it, but presumably all of the hardware has been normalized between the 1.0 and 2.0 releases. The increased performace results from virtualization driving up utilization on each Exalogic server, plus other tunings in the software done in the past two years.
The new Exalogic 2.0 software stack also includes something called Oracle Traffic Director, which is an application load balancing and traffic shaping appliance that Oracle has been quietly shipping to customers for a while, because they couldn't wait for it.
Palmeter says that Traffic Director gets the traffic shaping and load balancing back where it belongs, in the application layer, and that it is based on the iPlanet code base that Oracle got through its acquisition of Sun (which acquired it from Netscape). This iPlanet code has been extended and tweaked to be a fully functional, optimized reverse proxy and load balancer, says Palmeter.
It is also aware of the AES encryption functions inside of Intel's Xeon 5600 processors, which are used in the latest Exalogic X2-2 iron, and you can dedicate a few nodes inside of an Exalogic setup to do the load balancing internally instead of relying on an outboard load balancer, like an F5 Networks BigIP box. Such applicances cannot be as application-aware, at least as far as Oracle is concerned.
Exalogic 2.0 software also includes Oracle Virtual Assembly Builder, which is a tool for creating XML metadata descriptions that encapsulate all of the ways different parts of an application stack are related to each other. It does prompt administrators for some setting information when necessary, but it largely automates the installation of applications and middleware on the Exalogic platform. It basically replaces a person reading a best-practices reference architecture and walking through an installation manually.
Virtual Assembly Builder includes a studio component for hacking together these XML stack installation instructions. You can take an existing VAB template (like the ones Oracle has created for 60 of its own applications) and hack them for your own apps, or you can start by having the studio sniff your existing application setup and cook up its own XML description of how it should be set up on an Exalogic stack.
The Virtual Assembly Builder also has a deployer component, which prompts admins when they are doing installations for the "settables and tunables" that can't be fully automated – or that won't be, because then admins would have nothing to do.
The server administration portal that is part of the Exalogic 2.0 stack has a command line interface, of course, and also a Java API, which is how Virtual Application Builder tells the portal and its underlying orchestration engine what to do. Other tools can link into this Java API.
Finally, because the Exalogic 2.0 stack is now virtualized (at least on Linux, anyway), you can use Oracle Enterprise Manager, the tool of choice for managing bare-metal and cloudy apps and infrastructure at Big Larry, to manage the provisioning and operations of the Exalogic virtual machines.
Exalogic 2.0 software is available today, and is compatible with the prior releases of the Exalogic hardware. It is also available for free to customers currently using Exalogic iron so long as they are under a maintenance contract.
The Exalogic X2-2 comes in full, half, quarter, and eighth rack variants, and at the moment scales across as many as four racks in a single cluster. But Palmeter says that no customer to date has installed more than two racks, and that a total of 720 cores running the application layer as efficiently as Exalogic does, is "a really big platform."
A full Exalogic rack has 30 two-socket x86 server nodes with a total of 360 Xeon 5600 cores, 2.9TB of memory, and 60TB of disk capacity. It uses QDR InfiniBand switching to link the nodes together and to provide remote direct memory access (RDMA) to goose the performance of the middleware.
The machine has gateways to link the InfiniBand network out to the Ethernet network that users need to get access to the machine. The full Exalogic rack has 32 10GE ports in its multiple gateways and has four 32-port InfiniBand switches. The full rack costs $1.075m, with $129,000 per year in systems support and $86,000 per year in operating systems support. ®