Cloud company avoids the cloud
Firebase goes for heavy metal performance over easy cloud VMs
An infrastructure startup has had to shift off of cloud servers to bare metal gear after it ran into performance issues, showing that virtualization is not always the optimal tech for cloudy apps.
Firebase, a backend-as-a-service company that provides a technology which lets developers sync JSON data in real time across applications, had to move its infrastructure to bare-metal servers after it ran into performance problems with its cloud provider, the company's cofounder and CTO Andrew Lee told The Register on Monday.
"Earlier this year it was a packet problem," Lee said. "Customers that came online and had unusually bursty patterns. A hundred thousand packets all at once. What would happen on the cloud servers was the machines would become unresponsive while they worked through that backlog of packets."
Firebase provides a platform for syncing data across multiple clients with latency on the order of a hundred milliseconds, allowing its tech to be used to build things such as collaborative document editing or drawing apps. Having unpredictable latency spikes in data replication is exactly the sort of problem it has been designed to solve, so the company had to figure out a solution – and that meant leaving the unnamed public cloud provider.
"Cloud servers have a virtual network stack inside the hypervisor," Lee said. "I was surprised by how early [the problem] came – we had to move off much sooner than we would have liked".
Moving away from the cloud provider and onto bare-metal servers from Softlayer had performance benefits, as the company was able to go from having around 250 IOPS per second on MongoDB to 40,000 for an SSD-backed server with SoftLayer. But this did change the cost profile a lot – "the servers in general are more expensive. The cheapest [dedicated server] I can get is $250 a month," Lee said. "On Amazon it's like $16."
Other tradeoffs include the time it takes to spin up a server, and being deprived of the constellation of services that come with typical cloud providers, along with guaranteed hardware maintenance and migration. "For us, if something goes wrong with the box it's up to us to reboot," he said.
Firebase's situation highlights some of the ways that large public clouds are technically inferior to owned hardware for certain classes of problem. "In the future I think everything will be in a cloud except for weird infrastructure [providers]," Lee said.
The company's experience matches up with other companies that have hit limits within their provider and been forced to move out of the virtualized abstraction layer that cloud enforces. Netflix, for example, turned to custom hardware for its own content distribution network (CDN) after it found the offerings from AWS lacking.
Firebase recently took in $5.6 million in funding to help the company expand its core engineering team and hire
marketers developer evangelists to grow the use of its product, which is available for free and becomes paid-for as customer's want greater performance. ®