Oracle tweaks LDom hypervisor homegrown Sparcs
Gets agnostic about Sparc T generations – and maybe M4s, too
The Logical Domain hypervisor for Sparc T series processors, known formally as Oracle VM Server for Sparc, is probably one of the best technologies created by the former Sun Microsystems for its homegrown servers.
(Solaris and ZFS are two other key technologies.) Oracle has just kicked out a 2.2 release that makes LDoms more useful, and that is important in an increasingly virtual world.
Oracle VM for Sparc only runs on the company's own many-cored Sparc T series server chips and is tied to special features on those processors to do their magic. This is in much the same way as hypervisors for other chip architectures make use of electronics etched on x86, Power, or Itanium processors to do some of their virtualization magic.
But it is a shame that Fujitsu and Sun Microsystems could not have agreed on a virtualization strategy that spanned both the Sparc T chips made by Sun and now Oracle and the Sparc64-VII family of processors made by Fujitsu for the Sparc Enterprise M series of midrange and high-end SMP boxes.
Solaris containers work on both platforms, of course, as they do on the x86 implementation of Solaris 10 and 11, but LDoms are arguably better than the dynamic domain hardware partitions that Fujitsu supports and that Sun baked into its earlier UltraSparc-based systems.
Of course, with Oracle doing the Sparc M4 chip itself, it seems unlikely that LDoms will ever make it to the Sparc64 chips made by Fujitsu.
But there is every reason to expect that LDoms will be a key feature of the Sparc M4 chips, which were tested in Oracle's labs last September running Solaris 11, and which are expected to appear before the end of the year – alongside the Sparc T5 processors for entry and midrange machines. Punters expect the M4s to have similar cores to the T series machines, and this is important for reasons that will be made obvious now.
With Oracle VM Server for Sparc 2.1, announced last June, a server could be carved up into as many as 128 logical domain partitions, with a granularity for that LDom being as small as a single processor thread if you want to make very skinny server slices.
The 2.1 release also added live migration between physical systems, which was a big improvement over the "warm migration" that the earlier LDom 1.3 hypervisor supported (the LDom 1.3 came out in February 2010). This live migration in the 2.1 hypervisor had its limits, however. You had to have servers on both sides of the migration with the same processors and running at the same clock speeds – which is a bit limiting.
With the LDom 2.2 hypervisor, as you can see in the release notes, Oracle has tweaked the live migration feature so it can take a running domain and teleport it over the network to another machine – regardless of Sparc T series generation or clock speed. However, there are two limitations.
First, both LDom 2.1 and 2.2 are only supported on Sparc T2, T2+, T3, or T4 processors, not the original Sparc T1. So if you have those older Sparc T1 machines, you are out of luck. You also need to have the 2.2 hypervisor running on both servers and the domains have to be running Solaris 11 to do migrations across different processor generations.
It is possible that Oracle may backport features from Solaris 11 into a Solaris 10 update so live migration across CPU generations is possible. (See the comments in this blog post by Jeff Savit, principal engineer at Oracle responsible for the Sparc hypervisor, who adds a few hints but no confirmation of future product plans.)
Let's assume cross-CPU live migration becomes possible with both Solaris 10 and Solaris 11, and LDoms are supported on the future Sparc M4 systems. Oracle will then be able to peddle machines with from 1 to 64 sockets with the same virtualization and allowing for migration of Solaris workloads across the entire line of machines.
This would, for instance, allow Oracle to pitch the Sparc M4 machines as a consolidation box for multiple Sparc T series machines. Create a tool to convert dynamic domains to LDoms, and Oracle could chase the Sparc Enterprise M base (both those sold by Oracle and those sold by Fujitsu) with the M4s as well. And if I were Larry Ellison, that's what I would do.
The updated 2.2 hypervisor for the Sparc T line of servers also now integrates with Oracle Enterprise Manager Ops Center, which brings a GUI to the hypervisor so you don't have to use a command line interface. The hypervisor has also been tweaked to allow for partitions to be allocated at the "whole core" level instead of by thread in a much simpler manner than it has been done since LDom 2.0.
This whole core allocation method allows customers to can carve up Sparc T machines in such a way as to be compliant with Oracle's "hard partition" licensing practices for its database, middleware, and application software and it used to impose caps on these whole core allocations as well as other restrictions.
These have now been eased to allow for reconfiguration so customers can allocate at the core and change allocations quickly because they don't license Oracle software that way or are not concerned with software licensing issues.
Oracle has also added support for Single Root I/O virtualization (SR-IOV), a PCI-SIG standard for I/O virtualization that is embedded in the Sparc T3 and T4 processors and that is also being added to other server processors. SR-IOV allows for multiple logical domains to share a physical device on the I/O bus and to do so in an efficient manner. At the moment, Oracle is supporting Gigabit Ethernet devices on the system motherboards and 10 Gigabit Ethernet cards from Intel with the LDom 2.2 hypervisor.
Sometimes you don't want to virtualize an I/O device but rather let applications running inside of an LDom to talk directly to the device to avoid any performance overhead from virtualization. In the LDom world, this is called Direct I/O, and Oracle has boosted the number of PCI-Express adapter cards that can be used in this manner.
It bears pointing out that Oracle does not charge for licenses or support for the VM Server for Sparc hypervisor and that it bundles it into the base Sparc T series machines.
This is a key differentiator between x86 hypervisors from VMware and Citrix Systems on the x86 platform as well as the PowerVM hypervisor from IBM for its Power-based machinery. Oracle similarly does not charge for its Oracle VM variant of the Xen hypervisor for x86-based machines. ®