Virtualization and HPC - Will they ever marry?

No time for the virual

HPC workloads are more driven by memory bandwidth, I/O bandwidth, and clock cycles than the typical infrastructure workloads out there in the data center and are therefore not as readily virtualizable. To put it bluntly, HPC labs have enough worries about wringing performance out of their machines and about getting more parallelism into their codes to better take advantage of the increasing number of cores they have in a cluster. They can't deal with virtualization, too.

Virtualization has been a boon to infrastructure servers that were underutilized. A typical x64 server running Web, print, file, and other workloads might run with maybe 5, 10, or 15 per cent of their CPU cycles being used utilized on average. (There are always peaks that spike above that). Hypervisors allow four or five server instances to be crammed onto one single physical server, with the added bonus these days of faster server provisioning and disaster recovery to boot. But in HPC clusters, CPUs are running at near their peaks all the time they are doing work.

But, having said that, system administration is an issue for clusters, just like it is for other servers, and people cost more money than software and iron. Setting up and configuring nodes in the cluster is a pain, and virtualization can help. Think about Amazon's Elastic Compute Cloud utility computing setup, which runs atop a tweaked version of the open source Xen hypervisor.

While this EC2 capacity is available on the cheap, it runs in a virtualized mode, and you could argue that one of the reasons it is so cheap is because it is virtualized and hence flexible. It is possible that HPC shops wanting to run distinct applications on different flavors of Linux or a mix of Linux and Windows will use hypervisors allow for this configuring and reconfiguring more easily. But plenty of people are skeptical of the idea.

"The biggest use of virtualization is to allow multiple applications to run protected," explains David Scott, petascale product line architect in Intel's HPC platform unit. "This is potentially an area. Customers are thinking about it, but no one has done it yet."

And the reason why virtualization has not been used in HPC shops is the same one that made server virtualization in data centers take off slowly: server hugging. "The idea of giving a piece of a processor to someone else is completely alien to HPC people," says Scott.

And that is why, for now, server virtualization and HPC will probably remain oil and water - at least as long as there are graduate students and scientists to man the clusters for free or nearly so. Then again, if you shake up things enough, you can get oil and water to make a suspension. Maybe HPC's salad dressing days are ahead. ®