Docker kicks KVM's butt in IBM tests
Big Blue finds containers are speedy, but may not have much room to improve
IBM Research has run done a side-by-side comparison of the KVM hypervisor and containerisation enfant terrible Docker and found the latter “ equals or exceeds KVM performance in every case we tested.”
Big Blue tested the two using the linear-equation solving package Linpack, the STREAM benchmark of memory bandwidth, network bandwidth using nuttcp, latency using netperf, Block I/O speeds with fio and Redis. The SysBench oltp benchmark gave MySQL a workout.
The resulting paper An Updated Performance Comparison of Virtual Machines and Linux Containers (PDF), finds that Docker allows the containerised computers it curates to hum along at impressive speeds.
As the graph below and others in the paper show, Docker kicks KVM's butt in some tasks, and holds its own comfortably in most of the tests conducted.
IBM's results for Docker vs. KVM results running Linpack on two sockets with 16 cores.
Each data point is the arithmetic mean obtained from ten runs.
Error bars indicate the standard deviation obtained overall runs.
With Docker only just having reached v 1.0 status, you might think that's goodnight for virtualisation, as if the first commercial version of the technology is already beating and established tool surely there's no future for the latter.
The IBMers involved beg to differ, writing that they have seen continuous improvement in KVM performance over the years, while things “can only get worse for containers because they started with near-zero overhead and VMs have gotten faster over time.”
Nor is Docker perfect, with the authors finding it network address translation makes extra traffic for networks.
The paper concludes with some interesting observations about how the differing performance of the two contenders will impact future cloud infrastructure designs. Here's the best bits from those considerations:
“Conventional wisdom (to the extent such a thing exists in the young cloud ecosystem) says that IaaS is implemented using VMs and PaaS is implemented using containers. We see no technical reason why this must be the case, especially in cases where container-based IaaS can offer better performance or easier deployment.
Containers can also eliminate the distinction between IaaS and 'bare metal' non-virtualized servers since they offer the control and isolation of VMs with the performance of bare metal. Rather than maintaining different images for virtualized and non-virtualized servers, the same Docker image could be efficiently deployed on anything from a fraction of a core to an entire machine.
We also question the practice of deploying containers inside VMs, since this imposes the performance overheads of VMs while giving no benefit compared to deploying containers directly on non-virtualized Linux. If one must use a VM, running it inside a container can create an extra layer of security since an attacker who can exploit QEMU would still be inside the container.”
Scripts used for the study can be found here, if you'd like to run the tests yourself. ®