Bechtolsheim: The server is not the network

10GE goes mainstream

What has Bechtolsheim fired up about 10GE is that it is starting to go mainstream. Even with the generic networking business expected to see an annual revenue decline on the order of 20 per cent in 2009, according to Bechtolsheim, the number of 10GE ports attached to servers are expected to grow from about 400,000 in 2008 (against new server shipments of around 8 million globally). Bechtolsheim predicts that the number will more than double each year over the three years so that by 2011, over 4 million 10GE server ports will ship in that year and, if you do the math, with about 7.5 million 10GE server ports installed by 2011. About 2 per cent of all servers in the installed base had 10GE ports in 2008, which will rise to about 5 per cent this year, hit maybe 10 per cent in 2010 and to about 25 per cent or more in 2011.

"It is easier to forecast 10GE ports than revenues," says Bechtolsheim. "This is one of the few areas in IT with predictable growth."

Part of that reason is that starting this year, some server motherboards will come with integrated 10GE network interface cards, and that will change the economics of 10GE networking, much as motherboard integration did for 10 Mbit, 100 Mbit, and Gigabit Ethernet networking over the past decade. And this has to do with money as much as it does with integration. Right now, the server side of a Gigabit Ethernet switch is essentially free, and it costs maybe $150 per port, on average, for a Gigabit Ethernet switch. With the low prices that Bechtolsheim is bragging that he can deliver with Arista's switches, he can get a 10GE switch into the field for around $500 per port.

But the 10GE adapter on the server side also costs around $500 right now, which makes a 10GE port cost $1,000 a pop when you look at both the switch and the server side. This is a premium some customers - like supercomputing labs, government agencies, hyperscale Web sites, and financial services firms - will pay because of the low latency. But Bechtolsheim says that 10GE will not become a no-brainer for the entire IT industry until 10GE NICs are integrated on servers and the per-port costs drop to around $250 including both the server and the switch.

"At that point, the gap is so small that companies will switch to 10 Gigabit Ethernet even if the extra performance and low latency is not needed," says Bechtolsheim.

And Arista has every intention of getting its share of that 10GE market. The company started shipping its first switch - a 1U, rack-mounted 10GE switch called the 7100 Series - almost a year ago. It offers 24 or 48 ports in a variety of configurations that offer anywhere from 600 nanoseconds to 2.9 microseconds of latency and from 480 to 800 Gb/sec of throughput at layers 2 and 3 of the network. (The higher bandwidth boxes have more latency, since there is no such thing as a free lunch).

Bechtolsheim says the 7100 Series is really a 1U, dual-core x64 server with a 10GE networking ASIC and ports installed. Just like Bechtolsheim caught the x64 bug when he started Kealia, Arista is not interested in spending tens of millions of dollars developing its own silicon for handling 10GE networking functions. Rather, Arista has tapped Fulkcrum Microsystems for its 10GE chip, which is put inside of Arista's 1U box, which has a direct link to the x64 cores.

This, says Bechtolsheim, is a key differentiator for Arista's switches. The 7100 Series switch has an operating system called Extensible Operating System (EOS) and a virtualized implementation called vEOS that runs as a virtual appliance and that, like Cisco's Nexus 1000v virtual switch, integrates with VMware's vSphere hypervisor and virtual switch architecture. In this case, vEOS can run on the 7100 Series' x64 server and its hardened Linux operating system, not on an external blade.

Bechtolsheim also says that EOS and vEOS implement a stateless architecture that allows the switch to restart itself when it crashes exactly at the point where it crashes and that also has the added bonus of allowing EOS to be patched while the switch is running and without resorting to needing redundant controllers that are patched in succession, as Cisco's high-end switches require.

By adding an x64 processor to the switch, Arista expects to be able to quickly add new software functions to the switch without requiring changes to the hardware, something that HPC customers are looking for. And Arista fully expects that third parties will create add-on software for its switches and is encouraging this.

Looking ahead, Bechtolsheim says that Arista will roll out a full line of modular switches and will put machines in the field that scale to higher port counts and lower latencies than any other modular switch on the market. "We will be competitive on all fronts," Bechtolsheim boasts.

Arista has over 100 employees today, including contract engineers, and it has over 150 customers worldwide, with about 80 per cent of them in the United States. About a third of its customers come from the financial services industry, and the strong uptake of its 7100 Series switches among banks and brokerages is one of the main reasons why Arista is opening an office in London to develop products and sell and support them in the City. (Arista has partnered with systems integrator CTC of Japan to take on the financial and other HPC sectors in the Asia/Pacific market).