What do microservices do to data stores? Netflix is built on them and had no idea!
So it built a now-open-source benchmark tool for Cassandra and Elasticsearch
Microservices are the new black for developers, but even one of their world's biggest and most prominent users – Netflix – has said they're a bugger to manage.
“As Netflix runs thousands of microservices, we are not always aware of the traffic that bundled microservices may generate on our backend systems,” write the company's Vinay Chella, Ioannis Papapanagiotou and Kunal Kundaje.
“Understanding the performance implications of new microservices on our backend systems was also a difficult task.”
Which rather re-enforces the notion that perhaps the container ecosystem is a mite immature.
But we digress.
To get on top of its microservices, Netflix decided it needed “a framework that could assist us in determining the behavior of our data store systems under various workloads, maintenance operations and instance types.
“We wanted to be mindful of provisioning our clusters, scaling them either horizontally (by adding nodes) or vertically (by upgrading the instance types), and operating under different workloads and conditions, such as node failures, network partitions, etc.”
So Netflix built it, called it NdBench and open-sourced it. Which is why that link back there goes to GitHub, where you can get the code for your own footling pleasure.
What does it do? For now it can test Cassandra (Thrift and CQL), Dynomite (Redis), and Elasticsearch, the three data stores Netflix employs. It can simulate workloads, measure performance under different operating systems, identify memory leaks, thrash CPUs to see how they react and much, much more.
Netflix says it's used the tool for “the last few years,” so if you decide to download you're getting a reasonably well-tested tool. The streamo-supremo nonetheless says it is “looking forward to receiving feedback, ideas, and contributions from the open source community.” ®