NY Stock Exchange cures jitters with Juniper

NYSE Euronext, which operates stock and options exchanges from ten data centers worldwide, has picked Juniper Networks as a primary backbone infrastructure supplier for a pair of redundant data centers that'll consolidate operations in New Jersey and outside London next year.

Steve Rubinow, NYSE Euronext's chief information officer, said his organization will deploy 10 Gigabit Ethernet switches and routers from Juniper to build a new stock trading system that'll reduce the average transit time for network traffic from around 150 to 50 microseconds.

Rubinow said NYSE Euronext's customers, who do the buying and selling and then do their trades on the exchange's systems, increasingly co-locate their own machinery within NYSE Euronext's data centers because latency is such an issue. It takes an electric signal a nanosecond to travel a foot - more or less.

With the NewYork Stock Exchange down on Wall Street being about ten miles away from the data center in New Jersey, the delay between Wall Street and the systems behind the NYSE is about 105 microseconds. This is not a big deal for some trading companies, but means millions of dollars for others.

The need for network speed and real-time computing to calculate options and do trades means NYSE Euronext and its switching suppliers are pushing up against the physics barriers of light speed. Customers, though, are accustomed to getting an order of magnitude reduction in latency for transactions over the exchange every two to three years.

The average transaction moves over the NYSE Euronext network on the order of hundreds of microseconds today, but only a few years ago it was tens of milliseconds, according to Rubinow.

NYSE Euronext has been working on a multi-decade data center plan for the past two years. It's been making its best guesses about the server, storage, networking, and data center power and cooling technologies that would be available in the coming years. After examining the roadmaps of various - and unnamed - switch and router providers, the organization decided a year ago to align itself with Juniper's roadmap.

This roadmap includes a set of 10 Gigabit Ethernet switches and routers available today and evolving to future products, developed under the code-name of Project Stratus at Juniper with Sun Microsystems' former chip guru David Yen steering development. The devices are expected to include 40 Gigabit Ethernet and 100 Gigabit Ethernet switches that will allow NYSE Euronext to push latencies on its trading network down to tens of microseconds.

The wide area network that NYSE Euronext has to link its trading and market data systems out to the world are already running on 100 Gigabit Ethernet fiber optic backbones using equipment from Ciena.

Bandwidth and latency are not the only issues that NYSE Euronext is coping with. According to Rubinow, trading partners watch the average transaction latency very carefully as their transactions run, but they watch the variance around that mean transaction time - known as the jitter in the financial systems lingo - even more carefully.

For every batch of tens of millions of trades that NYSE Euronext processors, Rubinow says there can be two or three that fall far outside of the mean latency. You can bet you last $5m that those trading partners will call up cranky and explain that they lost a lot of money because of the delays in processing those transactions.

Jitter is particularly high at the market open and close, when automated trading systems rev up to make their biggest trades.

So in addition to raw speed improvements, NYSE Euronext is interested in reducing this jitter factor by as much as possible. To that end, NYSE Euronext will be deploying Juniper switches and routers in a two-tier network rather than the typical three-tier setup.

The exact configuration has not been set yet, but Rubinow said NYSE Euronext will buy somewhere between 20 and 40 of Juniper's EX8216 10 Gigabit Ethernet modular switches, which have up to 768 ports per chassis and are based on Juniper's own chips and Junos network operating system.

The NYSE Euronext network will also include a few hundred of Juniper's EX2500 top-of-rack data center switches, which Juniper OEMs from an undisclosed vendor and that pack 24 ports a pop. The plan under Project Stratus is for Juniper to have a full set of switches that all use Juniper's ASICs and Junos operating system.

The new trading network will also make use of MX series of Ethernet services routers from Juniper. Which particular switches and routers, NYSE Euronext did not say. The MX boxes are based on Juniper's own chips and software.

On the market data side, NYSE Euronext has been partnering with InfiniBand switch maker, and now 10 Gigabit Ethernet player, Voltaire to beef up its market data networks using its Vantage 8500 40 Gb/sec InfiniBand switches, which just started shipping this week.

NYSE Technologies, which is the part of the company that actually hooks people into the NYSE and Euronext exchanges, has rolled out a market data system based on the Vantage 8500 switches. The system offers latencies per core switch in the range of 25 microseconds for one million messages per second on messages that are 200 bytes in size. ®