Virtualisation for Beginners
It's not just for server jockeys, you know
Hands On VMWare engineer Shawn Morel opens an entertaining and highly recommended seminar on the internal workings of his company's Fusion product for the Mac by dividing up an area of space with a couple of vertical lines. He points in turn to the three segments this creates: "You've got Userland, the Kernel space and Hyperspace."
He adds a few more scribbles.
"And then you've got all these little things running around here, and you can have one or more VMXs, and you've got the kernel module and you've got this thing called the VMM, and it's all very complicated and it all sort of works like magic."
And then he puts up the Q&A slide.
...it's all very complicated and it all sort of works like magic
The joke gets a huge laugh from the audience of developers. But, of course, he's right. Virtualisation is very complicated. And it does work like magic. But knowing a little bit about how the magic works can throw useful light on how you can make it work for you.
As Morel says later, in what turns out to be a pretty full discussion of the technicalities: "A lot of people on OS X don't really realise the benefits of virtualisation. They'll go, 'Oh, sweet, I can run that Windows app that I couldn't run before.' The rest of the industry is like, 'Oh great, I've got isolation, I've got disaster recovery and fault tolerance, and I can consolidate my servers."
His point is that all those enterprise-class benefits of virtualisation are there for the individual desktop user too. Nothing that happens inside a virtual machine can damage or change anything in the host operating system, or in any other virtual machine that might also be running.
A single-core Parallels Win XP VM under Mac OS X. The decoder is running at around 40f/s, pulling in frames from the Avisynth frameserver fed from a demuxed MPEG2 file
His remarks don't just apply to Mac users. Any operating system disaster happening inside a virtual machine running under any other operating system is transient, because you can simply close down the virtual machine and start it up again, without affecting the rest of the system. If - and Windows users will be familiar with this - the causes of the disaster survive a reboot, perhaps because the recent installation of the new app has messed everything up, there's no need for an intricate repair process. Just throw the whole VM away and go back to the last working snapshot.
This is something that Windows itself has always purported to be able to do since the arrival of XP, but - in my experience at least - the success rate has been less than sterling. A VM snapshot, by contrast, is equivalent to a complete new installation on brand new hardware with a slipstream update of all your apps up to the last point of failure.
Technically, if perhaps not legally, you can run Mac OS X in a VMWare virtual machine under Windows
The Technology Behind the Magic
Virtualisation originated in IBM's big mainframes of the 1960s and 1970s when the company hit on the idea of installing a thin software layer, the 'hypervisor' immediately above the hardware that would allow multiple operating systems to run simultaneously side by side without impacting on one another or even necessarily being aware of each others presence.
'Simultaneously' needs some qualification here, because the various operating systems in fact time-shared a single CPU. But the switching would take place fast enough - several thousand times a second - to create the illusion of simultaneity.
Multi-tasking operating systems use a similar technique to run multiple apps at the same time. In this scenario, called 'context switching', only the metrics associated with each running app - the 'state' of the app - need be stored. When switching between complete guest operating systems much more data has to be saved between each switch: the total state of the guest operating system itself, as well as the states of all the apps it happens to be running at the time. Engineers call this a 'world switch'.
A Solid Proposition
The earlier Pentium processors weren't designed with virtualisation in mind. But using a software technique called 'segment faulting', developers managed to introduce world switching, and hence virtualisation to Pentium-class processors in the late 1990s. The first VMware product allowed Windows to run in a virtual machine on a Linux host, and this technology was later extended to use Windows as a host for Linux and other operating systems.
Parallels 5 for the Mac allows you to set an 'Active Corner' when in Full Screen mode
By 2006, even modest desktop processors from Intel and AMD were becoming powerful enough for virtualisation to be a solid proposition. And that, literally, is what it became. The two x86 companies each introduced hardware assistance for virtualisation into its processors: AMD-v - codenamed 'Pacifica' - and Intel's own VTx - 'Vanderpool' - were similar but different hardware virtualisation support systems.
Today, products like VirtualBox, Parallels Desktop and VMWare will make use of these hardware features if they're available, but still perform well on processors without them.
Parallels, for example, uses these new Intel and AMD hardware virtualisation extensions because they are almost always available on modern hardware, especially on the Mac. But Parallels can also deliver full functionality without Pacifica or Vanderpool by running in 'software mode', using techniques similar to VMWare's segment faulting
A VM is a useful testing ground for a new operating system -- in this case Google's ChromeOS under VMWare
It's tempting to think that on a multicore processor, virtualisation might allocate different cores to each of the running operating systems, but this isn't how it works. Cores are shared in much the same way as in a uniprocessor system, although the virtualiser will typically allow the user to set up 'core affinity', which associates a particular virtual machine with one or more specific cores.
But What Can I Do With It?
The most obvious use for virtualisation is running legacy apps which are perhaps no longer supported on current versions of the operating system. A tried and trusted old DOS accounting system, for example, might be carried over to Linux inside a VirtualBox virtual machine running a readily available open source version of DOS like FreeDOS.
A valuable side effect is that the application will be running in an environment isolated from other applications on the same machine, and so will probably behave more reliably than in its original habitat. And whereas in the past a failure of the application and/or the DOS environment might entail a complete reinstallation of one or both of these, now DOS and the app it's hosting can simply be reinstated as a whole, should the need arise, by restoring a snapshot of the virtual machine.
When you 'revert to snapshot' like this you'll discover immediately - or preferably have had the nous to realise well in advance - that any data used or created in connection with the application should be stored outside the virtual machine. It's usually easy enough to set up the virtual machine so that drive D: (for instance) is actually mapped to a data directory owned by the host operating system, or off on a network drive somewhere. This ensures that your set of up-to-date data hasn't been discarded with the now overwritten old version of the virtual machine.
Your first ventures into virtualisation will probably be some variation of this, as a prop to run an old app on a new operating system. Running Windows apps on your Mac is an obvious example, and this opens up another useful possibility. The Mac understands Windows filesystems pretty well, but it can't, for example, write to NTFS. A Windows virtual machine running inside Mac OS X would get round this problem.
The cute 'Active Corner' in action. It curls down when approached by the mouse cursor, and a click on the exposed background switches into Window mode.
A practical example of this came up recently when a Nas device in an all-Windows shop failed. Users were able to recover the hard drive, but it was formatted for the Linux Ext 3 filesystem, which Windows can't read. A simple solution was to create a Linux virtual machine running under Windows, and recover the files from there.
Managing Virtual Machines
Virtual machines can grow very large. Typically, on creation, the machine will be allocated some maximum disk space, say 50GB, but will initially occupy only a tiny proportion of that within the host operating system's file space. As new files are added the space occupied grows dynamically, but this space isn't released when files are deleted within the guest - one more reason for ensuring data is stored outside your virtual machine.
Parallels offers a utility, Parallels Compressor, that can be applied to an offline virtual machine to seek out deleted space and remove it, shrinking down the virtual machine. Unfortunately, this valuable facility is close to useless in practice, because the utility refuses to function with virtual machines that have been snapshotted, and snapshotting is pretty well a fundamental requirement.
The problem is that snapshots store more than just the files within the VM - they also store the state of the processor and the entire contents of memory. "And the file system is not in a synchronised state," says Stas Protassov, Senior Vice President of Virtualisation and Storage at Parallels, "and you could corrupt the file system."
An update sometime next year is expected to fix this problem and make snapshots and the compressor compatible. Once this is in place Parallels foresees offering the option to run the compressor permanently in the background.
Apple users are familiar with the frustration of finding that some new appendage they've acquired is only supplied with software to communicate with Windows machines. Windows connection software for phones, cameras, or non-iPod MP3 devices should run well inside a Windows virtual machine, although the ultimate solution is to insist on hardware that embraces USB mass storage, and so can transfer data across all operating systems with no special software.
Parallels Desktop's Crystal Mode allows windows from the guest OS to float freely among the host OS' windows. Windows apps - marked with the orange Parallels symbol - are available from the Mac's Dock.
Do you need to power up the whole virtual machine just to run one small guest application? Yes, you do, but you can do this under the covers. Parallels Desktop's Crystal Mode allows windows from the guest operating system to float freely among those of the host operating system, and attaches the guest applications to the host's menuing system. VMWare Fusion takes the same approach with a similar facility called Unity.
In effect, this allows you to encapsulate the guest application and run it as if it were just another host app. If you can spare the memory, you can speed up the process having the guest operating system under the hypervisor ready-loaded at boot time.
The Downside of Hardware Virtualisation
“There is no software-visible bit whose setting indicates whether a logical processor is in VMX non-root operation. This fact may allow a VMM to prevent guest software from determining that it is running in a virtual machine” - Intel VTx specification.
Intel's VM hardware design means that it may be impossible for an operating system to know it's actually running as a guest, not as a host. This is useful, eg. for driver developers, but also allows for the mother and father of all rootkits.
It's been suggested that an operating system simply has to try to load a VM of its own, ie. launch its own hypervisor. If it can't then it's probably running as a VM itself. However, Joanna Rutkowska, the Polish security specialist who conceived the Blue Pill hypervisor rootkit, has shown that nested virtualisation is possible, and an OS capable of launching a hypervisor might still be running under an even higher privileged hypervisor.
In essence, this is the so-called God Problem. Key to atheist Richard Dawkins' argument for the non-existence of God, is the premise that "a world without God would be very different from a world with God". But need there be any detectable difference?
Both VMware and Parallels also allow you run the whole guest operating system either as a single entity in its own window, or full screen, which makes it look as if it owns the entire machine. In conjunction with the Mac OS X Spaces feature - a way of flicking immediately between virtual desktops with the touch of a key combination or a mouse gesture - this is a great way to switch almost instantly between multiple operating systems.
YouTube HD running smoothly on Windows 7 in a VirtualBox VM under Ubuntu Linux
One other side benefit of virtualisation should appeal to anyone who runs lengthy processes on their machine. It seems to happen all too often that you're re-encoding a movie as a background process, when you need to reboot your machine, to install a system update, say, or fix a hardware problem. If the conversion is running on the host operating system you'll either have to wait until it's finished, or stop it and start all over again from scratch. But if it's running in a virtual machine, you just suspend the virtual machine and then reboot the host or whatever else you have to do. When you restart the virtual machine, the compilation will pick up from exactly where you left it.
With a sufficiently powerful processor, running multiple guest operating systems becomes feasible. Virtualisation has a performance impact, of course - if you're thinking of using it for gaming, for instance, dream on - but for office and straight graphics work my Intel Core 2 Quad easily runs Ubuntu Linux, Windows 7 and Windows XP in separate VMs under Snow Leopard. ®
Sun's VirtualBox is a free virtualisation utility available for Windows, Mac OS X and Linux. It lacks some of the finer features of VMWare and Parallels, but otherwise does an excellent job.
VirtualBox on Linux depends on modules that need to be compiled especially for the particular kernel in use. But if you're using one of the current common Linux distributions you should be able to find your distribution-specific package here.