Original URL: https://www.theregister.com/2009/03/25/live_migration_issues/

AMD migrates live VMs from Shanghai to Istanbul

Flight of the fake servers

By Timothy Prickett Morgan

Posted in Channel, 25th March 2009 22:21 GMT

Virtual machines are not as virtual as you might think. It's a bit tricky to move a running virtual machine across different processor generations, and you can forget jumping from a Xeon to an Opteron server - or visa versa.

Considering that Intel's Xeon and Advanced Micro Devices' Opteron chips have their respective VT and AMD-V hardware-assisted virtualization electronics - and that the server virtualization hypervisor is supposed to be abstracting the underlying hardware - you might conclude that making a VM written for any x64 server hop to any other kind of x64 server would be pretty easy. Apparently not.

The next best thing is to be able to move VMs across generations of the same processor type. And AMD wants people to know that it has worked with server hypervisor juggernaut VMware to make sure various generations of its Opteron processors can allow VMs to bop around a network of Opteron servers using the live migration feature of ESX Server.

To that end, and as part of AMD's self-declared Virtualization Ecosystem Month [Gag. - Ed.], the company has released some nerd porn a YouTube video showing a running VM moving from older generations of Opterons to the forthcoming "Istanbul" six-core Opterons, due later this year. The newsy bit of the demo is that an Istanbul chip is running in the lab.

To prove that this VM hopping works across chip generations, AMD's labs set up three identical server platforms, but put different processor generations - quad-core "Barcelona," quad-core "Shanghai," and six-core Istanbul - in the machines. The demo shows a video stream running inside a VM running atop VMware's ESX Server 3.5 hypervisor hopping from the Barcelona to the Shanghai to the Istanbul boxes.

The ability to live migrate VMs between machines of different processor generations is part of the enhancements that AMD made with its AMD-V hardware-assisted virtualization electronics that is part of the most recent generations of Opterons processors. It is enabled, in part, by a feature called Rapid Virtualization Indexing, a subset of the AMD-V feature set that virtualizes memory management units (MMUs) on the chip to boost performance, and Extended Migration, another feature that has been part of the Opteron lineup since the single-core Rev E chips were announced in 2005.

The Extended Migration feature matches up with a VMware feature called Enhanced VMotion, which was added to ESX Server 3.5 with Update 2 back in April 2008, and the hardware and software features together allow the hypervisor on which a VM is migrating to only expose the AMD-V features that are supported by the underlying chip to be exposed to the incoming VM.

As Richard Brunner, chief platform architect at VMware, pointed out in a hardware-assisted virtualization webcast discussion hosted by AMD today, companies that are virtualizing are nonetheless setting up "fragmented pools of servers" even after they have been virtualized.

While AMD's Extended Migration feature and the companion Enhanced VMotion feature in ESX Server 3.5 help, as does Intel's similar VT FlexMigration feature for its own Xeon family of server chips, it is still not possible to migrate a VM from an Opteron server to a Xeon server, or visa versa. Intel's FlexMigration feature works on "Woodcrest" Xeon 5100, "Clovertown" Xeon 5300, and "Harpertown" Xeon 5400 Xeons in the two-socket space as well as the Xeon MP counterparts. It will presumably be a part of the upcoming "Nehalem EP" Xeon 5500 processors, due to launch on March 30.

Moreover, not all hypervisors support the Extended Migration and FlexMigration features, which adds to the complexity. AMD and Microsoft announced yesterday that the beta of Hyper-V for Windows Server 2008 R2 has support for RVI, which seems to imply that Hyper-V will eventually be able to support VM live migration across multiple chip generations, as ESX Server 3.5 from VMware does. But neither Microsoft nor AMD came out and said that. Other features may be needed before this is possible.

What seems clear is that IT shops are going to want a standardized disk format for virtual machines, one that works with any hypervisor on x64 iron, and they are similarly going to want to have the ability to move not only from Intel to Opteron servers and back, but across different hypervisors as they see fit. This may take a while to happen, but common formats and live migration across all platforms is something customers want. There may be very sound marketing reasons not to do this, but it will be hard to argue that it is not technically possible. ®