Feeds

Mellanox: Just jump straight to 40GE networks

Unless you really want only 10GE

Protecting against web application threats using SSL

Network adapter and switch maker Mellanox Technologies is riding the wave of upgrades to 10 Gigabit Ethernet switches and is excited about boosting sales of 10GE adapter cards when Intel launches its "Sandy Bridge-EP" Xeon E5 processors sometime this quarter. But Mellanox CEO and chairman Eyal Waldman has his eye on a bigger prize: peddling 40GE switches and network interface cards.

"If you look at the Romley systems, 10GE is not enough," Waldman tells El Reg, referring to the code-name of the server platform designs coming out of Intel that use the impending Xeon E5 processors and their "Patsburg" C600 chipset. "For many customers, going from Gigabit Ethernet straight to 40GE is what makes the most sense."

This may sound a bit contrarian, given that the ramp to 10GE networking is just building some momentum in the data center after being available for a decade. Server makers are just now starting to plunk 10GE ports down on their motherboards, and the cost of 10GE switches has come down considerably in the past two years.

Mellanox was just recently bragging about how it slashed prices for 10GE adapters and switches because it wanted to accelerate the adoption of 10GE networking in the data center as well as its burgeoning share of the 10GE NIC market, jumped from 4 per cent of ports in the first quarter of 2011 to 24.6 per cent in the fourth quarter - the classic hockey stick. But Waldman wants more, and wants to get customers thinking about the jump to 40GE – particularly after having just slashed the prices on 10GE adapters and switches by roughly in half last month.

To that end, Mellanox is kicking out a new ConnectX-3 network card for servers that runs at 40GE speeds, matching the existing SX1036 switch launched last May.

Mellanox ConnectX-3 40GE adapter card

The Mellanox ConnectX-3
40GE adapter card (click to enlarge)

The new ConnectX-3 EN card, code-named "Osprey", is a single-port 40GE adapter card that plugs into PCI-Express 3.0 peripherals slots on servers and has been certified to run with the forthcoming Xeon E5 servers. These are, at the moment, the only servers that have PCI-Express 3.0 support, and probably will be for some time.

Interestingly, the PCI-Express controller resides on the Xeon E5 chip itself, which is the first such integration for PCI-Express 3.0 but certainly not the first time a PCI controller has been on chip. Oracle's T Series CPUs, Tilera's Tile processors, and even Intel's old i960 RISC processor had integrated PCI on the die, just to name a few. But the jump to PCI-Express 3.0 is significant because the old PCI-Express 2.0 bus does not have enough bandwidth in an x8 slot to drive a 40GE Ethernet card or a 56Gb/sec InfiniBand card, and the PCI-Express 3.0 bus can drive it.

On the switch front, Mellanox already sells the SX1036 top of racker, which can be used as a 64-port 10GE switch or a 36-port 40GE switch. So you can now go 40GE end-to-end if you want, and the Xeon E5 server can drive the NICs. What you can't get, however, are list prices for the 40GE adapters and switches out of Mellanox.

But Waldman is a sporting type and tells El Reg that the company plans to charge around 2.5 times the cost of 10GE adapters and switch ports for the 40GE products. That gives you better bang for the bandwidth – if you do the math, 37.5 per cent better. Perhaps equally significantly for heat-sensitive data centers, the bandwidth per watt is much better for the 40GE switches, too.

Waldman knows that a 36-port 40GE switch is not going to cover all the needs of the data center market, and says that the company is also launching a hybrid 10GE/40GE switch, the SX1024, and will eventually get modular switches in the field that sport 108, 216, 324, and 648 ports in a 21U chassis.

Mellanox SX1024 10GE/40GE switch

The Mellanox SX1024 10GE/40GE switch (click to enlarge)

The SX1024 switch pairs 48 ports of 10GE (with SFP+ interfaces) to talk down into the rack to servers, with a dozen ports of 40GE uplinks (with QSFP interfaces) to talk up to the aggregation layer of the Ethernet network. The SX1024 comes in a 1U chassis, and supports special cables that can convert four of those 40GE ports into a 16 10GE ports. Port-to-port latency is 250 nanoseconds, and this particular switch has 1.92Tb/sec of non-blocking throughput. In typical operation, it burns 154 watts.

The SX1024 switch is, of course, based on Mellanox's SwitchX InfiniBand-Ethernet ASIC, which debuted last April and was rolled into the SX1036 back in May. But there's another benefit of its 40GE switches that the SwitchX ASIC gives Mellanox: Afterburners.

The SwitchX ASIC has 2.88Tb/sec of throughput and 4Tb/sec of switching and routing bandwidth and is also the backbone of the Fourteen Data Rate (FDR) InfiniBand switches sold by Mellanox, which includes the 36-port unmanaged SX6035 and SX6036 managed top of rackers and the 640-port SX6536 director switch.

Because SwitchX can already rev to 56Gb/sec on InfiniBand, with the turn of the golden screwdriver and some money for software licensing, the 40GE switches above can be upgraded to run at 56GE speeds. No one else in the networking racket is offering this capability. Mellanox has to do a deep sort on the SwitchX chips to make sure they can run at the higher 56GE speeds.

And thus far, no one is offering a single ASIC that does either Ethernet or InfiniBand and promises to be able to change protocols on the fly in the future as Mellanox is planning to do.

"Our goal, a few years down the road, will be that if you ask what protocols customers are using, we won't know," Waldman said.

Mellanox is also looking for network bandwidths needs to rise a lot faster than they have in the past few years. "I think the transition to 40GE will be very fast," he explained.

Mellanox: Bandwidth and low latency hungers are never sated

Mellanox: Bandwidth and low latency hungers are never sated.

To make the point, Waldman trotted out statistics for InfiniBand adoption. Double Data Rate (20Gb/sec) InfiniBand took a long time to take off, but QDR InfiniBand (running at 40Gb/sec) accounted for 4 per cent of revenues in its first quarter of availability, then hit 12 percent in the second quarter it was available, followed by 32 per cent in its third quarter and 52 per cent by the fourth quarter.

FDR InfiniBand (at 56Gb/sec) is on a similar track, representing 4 per cent of sales in the third calendar quarter last year (when products were first available), and accounted for 13 per cent of revenues in the InfiniBand line in the fourth quarter. ®

Choosing a cloud hosting partner with confidence

More from The Register

next story
Wanna keep your data for 1,000 YEARS? No? Hard luck, HDS wants you to anyway
Combine Blu-ray and M-DISC and you get this monster
Google+ GOING, GOING ... ? Newbie Gmailers no longer forced into mandatory ID slurp
Mountain View distances itself from lame 'network thingy'
US boffins demo 'twisted radio' mux
OAM takes wireless signals to 32 Gbps
Apple flops out 2FA for iCloud in bid to stop future nude selfie leaks
Millions of 4chan users howl with laughter as Cupertino slams stable door
Students playing with impressive racks? Yes, it's cluster comp time
The most comprehensive coverage the world has ever seen. Ever
Run little spreadsheet, run! IBM's Watson is coming to gobble you up
Big Blue's big super's big appetite for big data in big clouds for big analytics
Seagate's triple-headed Cerberus could SAVE the DISK WORLD
... and possibly bring us even more HAMR time. Yay!
prev story

Whitepapers

Secure remote control for conventional and virtual desktops
Balancing user privacy and privileged access, in accordance with compliance frameworks and legislation. Evaluating any potential remote control choice.
WIN a very cool portable ZX Spectrum
Win a one-off portable Spectrum built by legendary hardware hacker Ben Heck
Storage capacity and performance optimization at Mizuno USA
Mizuno USA turn to Tegile storage technology to solve both their SAN and backup issues.
High Performance for All
While HPC is not new, it has traditionally been seen as a specialist area – is it now geared up to meet more mainstream requirements?
The next step in data security
With recent increased privacy concerns and computers becoming more powerful, the chance of hackers being able to crack smaller-sized RSA keys increases.