IETF doc seeks reliable vSwitch benchmark

Once switches become just another function to spawn, you'll need to know how they'll fare

By Richard Chirgwin

Posted in Virtualization, 25th September 2017 07:39 GMT

If you fancy wrapping your mind around the complexities that make virtual switches (vSwitches) hard to benchmark, an IETF informational RFC is worth a read.

Put together by Maryam Tahhan and Billy O'Mahony of Intel (note the usual disclaimer that RFCs are the work of individuals not employers) and Al Morton of AT&T, RFC 8204 is designed to help test labs get repeatable, reliable vSwitch benchmarks.

It's part of the ongoing work of the OpenNFV VSperf (vSwitch Performance) project group, which wants to avoid the kind of misunderstanding that happens when people set up tests that suit themselves.

The world of the vSwitch needs benchmarking, if for no other reason than for data centre designers to work out how much punch servers will need to match a physical switch, which is going to be important if switches become just another virtual service to be spun up as needed.

As the RFC notes: “A key configuration aspect for vSwitches is the number of parallel CPU cores required to achieve comparable performance with a given physical device or whether some limit of scale will be reached before the vSwitch can achieve the comparable performance level.”

And that's a knotty problem, because a vSwitch has available to it all of the knobs, buttons, levers and tweaks a sysadmin can apply to the server that hosts it.

Moreover, as they note, benchmarks have to be repeatable. Since the switches will likely run as VMs on commodity servers, there's a bunch of configuration parameters tests should capture that nobody bothers with when they're testing how fast a 40 Gbps Ethernet can pass packets port-to-port.

The current kitchen-sink list includes (for hardware) BIOS data, power management, CPU microcode level, the number of cores enabled and how many of those were used in a test, memory type and size, DIMM configurations, various network interface card details, and PCI configuration.

There's an even-longer list of software details: think “everything from the bootloader up”.

Not to mention that in the virtualized world, we're supposed to assume that a host runs lots of processes – even if that would take a benchmark standard down one rabbit hole too many.

“It's unlikely that the virtual switch will be the only application running on the system under test (SUT)”, the RFC notes, “so CPU utilization, cache utilization, and memory footprint should also be recorded for the virtual implementations of internetworking functions.

“However, internally measured metrics such as these are not benchmarks; they may be useful for the audience … to know and may also be useful if there is a problem encountered during testing.”

VSperf is documented here. ®

Sign up to our NewsletterGet IT in your inbox daily

Post a comment

More from The Register

Colt, Verizon show off inter-carrier SDN

Care for some extra bandwidth? Just turn the knob

Desktop virtualisation kit-calculator goes open source

Andre Leibovici's out of time, so it's your turn to step up

I got 257 problems, and they're all open source: Report shines light on Wild West of software

It's like a jungle sometimes, it makes me wonder how I keep from going under

Finally: Historic Eudora email code goes open source

'Member that innocent, pre-Zuckerberg time?

Open source community crams itself into big tent

Can't we just get along? At a sunny California inn with hors d'oeuvres, most definitely

Chinese giants give world another SDN and NFV platform

'Open-O' is backed by Huawei, China Mobile, Ericsson and Intel, among others

Apple unleashes FoundationDB as an open source project

Secretive company talks up the need for open community

Half of AT&T's networks are controlled by open-source SDN code

Fun fact of the day, we think you'll find

Rant launches Eric Raymond's next project: open-source the UPS

Batteries suck, chargers suck, monitoring sucks: surely we can do better than this

Transaction Processing Performance Council to deliver AI benchmark

GPU-makers aren't members of the TPC. Will they join?