Feeds

Virtualization and HPC - Will they ever marry?

Imaginary-server overhead

Mobile application security vulnerability report

No time for the virual

HPC workloads are more driven by memory bandwidth, I/O bandwidth, and clock cycles than the typical infrastructure workloads out there in the data center and are therefore not as readily virtualizable. To put it bluntly, HPC labs have enough worries about wringing performance out of their machines and about getting more parallelism into their codes to better take advantage of the increasing number of cores they have in a cluster. They can't deal with virtualization, too.

Virtualization has been a boon to infrastructure servers that were underutilized. A typical x64 server running Web, print, file, and other workloads might run with maybe 5, 10, or 15 per cent of their CPU cycles being used utilized on average. (There are always peaks that spike above that). Hypervisors allow four or five server instances to be crammed onto one single physical server, with the added bonus these days of faster server provisioning and disaster recovery to boot. But in HPC clusters, CPUs are running at near their peaks all the time they are doing work.

But, having said that, system administration is an issue for clusters, just like it is for other servers, and people cost more money than software and iron. Setting up and configuring nodes in the cluster is a pain, and virtualization can help. Think about Amazon's Elastic Compute Cloud utility computing setup, which runs atop a tweaked version of the open source Xen hypervisor.

While this EC2 capacity is available on the cheap, it runs in a virtualized mode, and you could argue that one of the reasons it is so cheap is because it is virtualized and hence flexible. It is possible that HPC shops wanting to run distinct applications on different flavors of Linux or a mix of Linux and Windows will use hypervisors allow for this configuring and reconfiguring more easily. But plenty of people are skeptical of the idea.

"The biggest use of virtualization is to allow multiple applications to run protected," explains David Scott, petascale product line architect in Intel's HPC platform unit. "This is potentially an area. Customers are thinking about it, but no one has done it yet."

And the reason why virtualization has not been used in HPC shops is the same one that made server virtualization in data centers take off slowly: server hugging. "The idea of giving a piece of a processor to someone else is completely alien to HPC people," says Scott.

And that is why, for now, server virtualization and HPC will probably remain oil and water - at least as long as there are graduate students and scientists to man the clusters for free or nearly so. Then again, if you shake up things enough, you can get oil and water to make a suspension. Maybe HPC's salad dressing days are ahead. ®

Bridging the IT gap between rising business demands and ageing tools

More from The Register

next story
Manic malware Mayhem spreads through Linux, FreeBSD web servers
And how Google could cripple infection rate in a second
EU's top data cops to meet Google, Microsoft et al over 'right to be forgotten'
Plan to hammer out 'coherent' guidelines. Good luck chaps!
US judge: YES, cops or feds so can slurp an ENTIRE Gmail account
Crooks don't have folders labelled 'drug records', opines NY beak
FLAPE – the next BIG THING in storage
Find cold data with flash, transmit it from tape
Seagate chances ARM with NAS boxes for the SOHO crowd
There's an Atom-powered offering, too
Gartner: To the right, to the right – biz sync firms who've won in a box to the right...
Magic quadrant: Top marks for, er, completeness of vision, EMC
prev story

Whitepapers

Reducing security risks from open source software
Follow a few strategies and your organization can gain the full benefits of open source and the cloud without compromising the security of your applications.
Consolidation: The Foundation for IT Business Transformation
In this whitepaper learn how effective consolidation of IT and business resources can enable multiple, meaningful business benefits.
Application security programs and practises
Follow a few strategies and your organization can gain the full benefits of open source and the cloud without compromising the security of your applications.
Boost IT visibility and business value
How building a great service catalog relieves pressure points and demonstrates the value of IT service management.
Consolidation: the foundation for IT and business transformation
In this whitepaper learn how effective consolidation of IT and business resources can enable multiple, meaningful business benefits.