Feeds

VMware learns to measure itself

Births benchmark for AMD, Intel and the server crowd

HP ProLiant Gen8: Integrated lifecycle automation

Not afraid to help itself while it helps others, VMware today rolled out a public version of its homegrown virtual server benchmarking tool.

VMware has been shopping VMmark in beta form since last year and now feels confident enough to set the test suite free. Server makers and end users are expected to use VMmark as a way of gaging the performance of certain workloads - namely file serving, mail serving, databases, java applications and transaction processing - running inside of virtual machines. With any luck, VMmark will add some customer-friendly metrics to a market woefully lacking in vendor vs. vendor and application comparisons.

Of course, just about every move VMware makes is meant to further its cause as the center of the x86 virtualization universe. So, it's not surprising that we find VMware working with benchmarking experts at the Standard Performance Evaluation Corporation to make a very VMmark-like suite an industry-standard.

Rivals Microsoft and SWsoft - along with the likes of Sun, AMD, Intel, HP, Dell and Red Hat - have joined VMware and SPEC to form a subcommittee meant to develop this new benchmark.

XenSource is notably absent from the group and has long argued that it has a major performance lead over VMware due to the lightweight nature of the Xen hypervisor.

With VMmark, however, VMware has tried to shift the discussion away from the raw performance of a hypervisor layer to how well actual applications run on a fully-stacked, physical server.

"Customers will typically measure the overhead of virtualization software running on a particular box," Andrea Eubanks, a senior director at VMware, told us. "From the perspective of understanding how their workloads will run on different hardware, this is a very primitive way to measure performance."

The VMmark test suite includes load generators for the various workloads described earlier and includes SPEC's own SPECjbb and SPECweb code for testing Java and order entry. Half of the benchmarks run on Linux, while the other half run on Windows. Customers will find the Linux-friendly software wrapped up in handy virtual appliances. Microsoft's licensing restrictions blocked a similar deal for the Windows code, so some self-assembly is required.

Servers makers such as Sun and Dell have already started issuing VMmark scores.

Eubanks is looking for the SPEC work around a standard virtualization benchmark to be completed by the second half of 2008.

It's hard to get excited about benchmarks. While helpful, they often end up making matters more confusing for customers, as vendors tweak, prod and poke their boxes to attain the best possible - often unrealistic - scores.

That said, customers have largely been left to fend for themselves in the virtualization game. As the market leader, it's natural for VMware to take charge and gives OEMs a helping hand with the measurement process. The end result should be a bunch of server and chip makers doing their best to craft kit that runs virtual servers really well.

An industry standard benchmark remains the more ideal end goal, but you'll have to live with VMware and VMware alone until at least 2008.

There's more information on VMmark due to post here

Reducing security risks from open source software

More from The Register

next story
Sysadmin Day 2014: Quick, there's still time to get the beers in
He walked over the broken glass, killed the thugs... and er... reconnected the cables*
SHOCK and AWS: The fall of Amazon's deflationary cloud
Just as Jeff Bezos did to books and CDs, Amazon's rivals are now doing to it
Amazon Reveals One Weird Trick: A Loss On Almost $20bn In Sales
Investors really hate it: Share price plunge as growth SLOWS in key AWS division
US judge: YES, cops or feds so can slurp an ENTIRE Gmail account
Crooks don't have folders labelled 'drug records', opines NY beak
Auntie remains MYSTIFIED by that weekend BBC iPlayer and website outage
Still doing 'forensics' on the caching layer – Beeb digi wonk
BlackBerry: Toss the server, mate... BES is in the CLOUD now
BlackBerry Enterprise Services takes aim at SMEs - but there's a catch
The triumph of VVOL: Everyone's jumping into bed with VMware
'Bandwagon'? Yes, we're on it and so what, say big dogs
Carbon tax repeal won't see data centre operators cut prices
Rackspace says electricity isn't a major cost, Equinix promises 'no levy'
prev story

Whitepapers

Designing a Defense for Mobile Applications
Learn about the various considerations for defending mobile applications - from the application architecture itself to the myriad testing technologies.
Implementing global e-invoicing with guaranteed legal certainty
Explaining the role local tax compliance plays in successful supply chain management and e-business and how leading global brands are addressing this.
Top 8 considerations to enable and simplify mobility
In this whitepaper learn how to successfully add mobile capabilities simply and cost effectively.
Seven Steps to Software Security
Seven practical steps you can begin to take today to secure your applications and prevent the damages a successful cyber-attack can cause.
Boost IT visibility and business value
How building a great service catalog relieves pressure points and demonstrates the value of IT service management.