Original URL: http://www.theregister.co.uk/2013/04/08/azul_systems_zing_lmax_exchange/

Upstart $3bn forex trader dumps Oracle JVM for Azul's Zing

A bad case of the jitters

By Timothy Prickett Morgan

Posted in Developer, 8th April 2013 10:03 GMT

Financial services companies were among the early and enthusiastic adopters of Java - but the jittery Java virtual machines from Oracle and IBM (and others before Oracle ate them) have been a pain for some.

Azul Systems is hoping to cash in on that grief among big banks, brokerages, high frequency traders, and others in similar businesses, where microseconds drive millions of dollars in profits or losses and you can't wait for the Java Virtual Machines to do garbage collection. This is why Azul ripped its JVM off its special Vega JVM processors back in October and created the Zing JVM for Linux.

Most financial services companies are pretty tight-lipped about anything that gives them a competitive advantage, but Azul founders Scott Sellers, CEO, and Gil Tene, CTO, have convinced a relative newcomer to foreign exchange trading, LMAX Exchange, to not only move off Oracle's HotSpot JVM but also to talk a little bit about its decision to make the move.

As Sellers explained it to El Reg back in the summer of 2010, when Zing was first being previewed, Azul never wanted to get into the business of making its special Vega chips for running JVMs or the Vega Compute Appliances that were sold as Java offload engines for traditional Unix, Windows, and proprietary servers. However, back in 2002, when Azul was founded, x86 processors didn't have enough virtualization support. This prompted Sellers, Tene, and Shyam Pillalamarri - the firm's vice president of engineering - to fix the garbage collection problem in hardware and then wrap a JVM around it with the Vega appliances, which started shipping in October 2004.

Many generations of x86 processors have come and gone since then, and the virtualization features in x86 processors from Intel and Advanced Micro Devices are now sufficient to the point that Azul decided to rip its JVM off its own Vega chips and port it to Linux running on x86 iron. That is, in essence, what Zing is.

To be more precise, the initial Zing 4 JVM from the summer of 2010 had a proxy architecture, which offloaded Java workloads onto a virtualized back-end appliance that functioned more or less like the Vega physical appliances. With Zing 5, which debuted in the fall of 2011, Azul got rid of the proxy architecture and tuned up its variant of the HotSpot JVM (which was licensed from Sun Microsystems in 2002) specifically for Linux operating systems running on modern Xeon and Opteron processors.

The JVM is designed to span multiple cores and threads and to have dynamic, rather than static, heap sizes that can grow to 512GB in size and substantially eliminate the garbage collection problems that other JVMs have. This garbage collection issue is a big deal in financial services apps, where service level agreements are getting rougher and tougher. This means that having consistent performance – not just for 99 per cent of transactions, but for all of them – is as important as having fast performance.

The secret sauce is something that Azul calls C4, which is not an explosive but rather short for Continuously Concurrent Compacting Collector. This is what allows the Zing JVM to span dozens of cores in a system and hundreds of gigabytes of main memory and deliver pauseless garbage collection.

Zing installs on Linux just like any other JVM – Oracle's HotSpot, IBM's J9 for WebSphere, or OpenJDK – and looks like any other JVM as far as applications are concerned. The initial Zing releases ran on Red hat Enterprise Linux and its CentOS clone, and support for SUSE Linux Enterprise Server was added in February 2012 with support for Canonical's Ubuntu Server following right behind it last April. In October last year, with the current Zing 5.5 release, Azul ported its JVM to VMware's ESXi hypervisors running Linux and also supported it running on Linux atop the Amazon Web Services EC2 compute cloud.

Windows support? Meh... maybe later

Azul has not ported Zing to run natively on Windows, and figures that it doesn't have to with more than 60 per cent of the 8 million servers that support Java workloads using Linux as their operating system, compared to around 30 per cent for Windows, based on estimates that Sellers provided to El Reg.

Block diagram of the Zing JVM

Block diagram of the Zing JVM

And the Linux server base supporting Java apps is growing very fast, so the market is moving away from Windows and towards Linux as best as Azul can see. That said, the company has conceded that it will probably have to support its JVM running on Microsoft's Hyper-V hypervisor and Windows natively because Windows is still a big part of the Java server base.

Which brings us to LMAX Exchange, an upstart foreign exchange and metals trading platform that was established in the City of London in October 2010. The company did a management buyout late last year – eat your heart out, Michael Dell – and has grown its business fast, where it is often matching buyers and sellers of currency to the tune of $3bn a day.

According to Edward McDaid, CIO at LMAX Exchange, the company has around 30 racks of x86 servers running SUSE Linux Enterprise Server, equipped with flash storage to speed them up, running in an Equinix data center west of London. The FX trader originally used Hewlett-Packard servers when it started, but switched to Dell boxes during last year's transition to Xeon E5 boxes. The FX trading platform is able to match 20,000 orders per second between currency buyers and sellers, and has peaked at around 40,000 trades per second under heavy load.

This Equinix facility is conveniently where a lot of other financial services firms park their servers and applications, so the latency hop between systems is fairly small. And it has a fat and fast pipe into one of the seven Equinix data centers in the New York region (this one is in lower Manhattan).

LMAX Exchange is currently using Oracle's HotSpot JVM on its servers to host its Java applications, and jitter is becoming an issue. The average latency for executing an FX trade on the LMAX system is around 3.5 milliseconds, which sounds good until you see that some trades take a lot longer than this because of the pause for garbage collection.

"Currently, we get over 97 per cent of trades with a pretty good latency, but there are some that go higher because of JVM jitter," says McDaid.

The company is running tests on its applications right now using the Zing JVM, and McDaid says that LMAX Exchange expects the latency to come down and for the jitter issue to be substantially done away with. And, as is typical for super-secretive financial services firms, he doesn't want to talk about what kind of improvement he expects to see.

McDaid did say that the software engineers at LMAX Exchange are going to spend less of their time trying to tweak their Java applications to deal with jitter and will be spending more of their time collaborating with software engineers at Azul to goose the performance of Zing and optimize it for low-latency trading environments.

Azul is similarly cagey about what a supported version of the Zing JVM costs, and only says that Zing costs around what a supported version of an Oracle, IBM, or Red Hat JVM will run enterprises and that it has an annual subscription model for Zing pricing. You can't easily get pricing for Oracle, IBM, or Red Hat JVMs, of course, so the comparison is accurate but perfectly useless.

Zing is distributed for free to open-source developers (but is not open source) so they can play around with it as they are developing code. But the minute you put it in production, you have to get a support contract.

By going with the Zing JVM and away from the Vega hardware appliances, Azul is not only trying to broaden its market, but also to beef up its business to make it an attractive takeover target from one of the established players in the Java racket. (Oracle, IBM, and Red Hat are potential suitors, obviously.)

Between 2003 and 2008, Azul raised $206.9m in 10 – yes, you heard me right – rounds of funding, with Accel Partners, Austin Ventures, ComVentures, Credit Suisse, JVax Investment Group, Meritech Capital Partners, Redpoint Ventures, Velocity Interactive Group, and Worldview Technology Partners all kicking in dough. At some point, these companies are going to want to cash out. ®