Google offers universal cloud benchmarking tool
Because nothing could go wrong with Google's metrics about AWS and Azure
Google is trying to establish an independent cloud benchmark, driven by a new tool called PerfKit Benchmarker.
The tool's open source, but Google's not being entirely charitable: it says it created the tool because customers struggled to compare apples to apples when assessing the performance of Google's cloud and rival offerings.”
Google says the tool will measure “the end to end time to provision resources in the cloud, in addition to reporting on the most standard metrics of peak performance.” The upshot will be “... a way to easily benchmark across cloud platforms, while getting a transparent view of application throughput, latency, variance, and overhead.”
All of which sounds lovely, save for the fact that Google has an obvious interest in such a tool pointing out just how awesome its cloud is. The Chocolate Factory has at least collaborated on the effort, and sought “input” from “other cloud providers, analysts, and experts from academia” to create the tool. There's also a thirty-strong list of companies that helped with the tool - ARM, Broadcom, Canonical, Cisco, Intel, Mellanox, Microsoft, Qualcomm, Rackspace and Red Hat among them – plus quarterly oversight from MIT and Stanford.
All this fun is available under version 2 of the Apache Software Licence, which Google thinks is “making it easy for contributors to collaborate and maintain a balanced set of benchmarks.”
Developers seem to be taking that invitation, as there's already some discussion on GitHub about adding support for new features, operating systems and clouds. ®