InfiniBand: Caught in the Ethernet meatgrinder
Voltaire heads for the protocol hills
SaaS data loss: The problem you didn’t know you had
Comment InfiniBand supplier Voltaire may be moving towards a twin InfiniBand/Ethernet strategy as InfiniBand looks to get boxed in by 10gigE and the coming 100gigE.
InfiniBand is a low-latency and high bandwidth interconnect that currently runs at 20Gbit/s and is transitioning to 40Gbit/s. It is used as a cluster interconnect in High Performance Computing (HPC) data centres, for things such as Genome sequencing and other bio-sciences applications, and with supercomputers. InfiniBand is also used inside multi-processor/controller server and storage products, such as Isilon's IQ Series of clustered file storage products, Sun's Constellation switch and the Oracle-HP Database Machine.
InfiniBand is a fabric with host communications adapters for servers linking across cables to a switch. Voltaire makes switches, multi-service switches with InfiniBand, Gigabit Ethernet and Fibre Channel integrated together. It recently announced a deal with IBM to make switches for its Bladecenter.
With Ethernet rated at 10Gbit/s for now InfiniBand should be walking all over it, but it isn't, despite Voltaire and competitor Mellanox positioning InfiniBand as the data centre networking convergence fabric, capable of carrying Ethernet and Fibre Channel as protocols layered onto it. In fact an Ethernet blitzkrieg is poised to trample across data centres with Cisco, Force 10, Juniper, Extreme and Brocade/Foundry all pushing the Ethernet economics story as well as the new, shining, never lose a packet, low latency data centre Ethernet (DCE) pitch.
You would expect Voltaire to be worried but Asaf Somekh, Voltaire's marketing VP, says: "In general, we really like what we see in terms of convergence."
He takes it for granted that convergence is gradually taking place: "Fibre Channel (FCoE) and and regular network traffic will converge over some time, not overnight.
"Ethernet is gradually loaning or stealing capabilities from InfiniBand. DCE is changing Ethernet from the ground up into something that's much closer to InfiniBand. From Voltaire's perspective the process is converging InfiniBand and Ethernet, getting them much closer in the protocol sense."
Voltaire founded the Open Fabrics Alliance in 2004, when it was called the OpenIB Alliance, to accelerate the development of open source InfiniBand drivers and, a couple of years later, added in Ethernet, expanding the charter, so, he said: "InfiniBand-developed protocols can move to Ethernet."
Here's a possible key statement, for an InfiniBand supplier: "When you run these protocols you're agnostic of the underlying fabric." He expanded a bit on this, saying: "We have worked on protocols that we knew would be relevant to Ethernet as well. Ethernet is becoming so close to InfiniBand and we're leveraging our technology."
This gives rise to a question; is Voltaire's identity changing from an InfiniBand fabric-based company to a high-speed, low-latency, lossless protocol networking company? Somekh said the question was "interesting".
A conclusion could be that we might see Voltaire introducing DCE capabilities into its switches. Think Infininet or EtherBand as a potential future for Voltaire, and a migration path for InfiniBand suppliers whose mantra may become "Adapt Ethernet or die". ®
COMMENTS
Infinband already at 40Gbps / FC at 16Gbps
With Infinband already at 40G and FC at 16G, the Ethernet meat grinder needs to quickly increase its grinding wheel to adopt 40G Ethernet and not just sit back with the promise of 100G sometime in the future. We've seen this over the last decade where Fiber Channel has embraced higher speeds in a more incremental fashion (1, 2, 4, 8and now 16Gbps) at a much lower cost point than the "less flexible" Ethernet, which just offered 1Gbps and 10Gbps (take it our leave it).
The Ethernet community need to be more "flexible" to the faster changing needs of Data Center SAN's, they've made a good start agreeing to standardize 40G Ethernet (though with a few cries of heresy) and adopting the same physical layer as Infiniband (i.e ability to use the QSFP copper/optic modules). But we have to wait and see if Ethernet vendors offer this flexibility to customers or if they insist on sticking to the 10x dogma.
Infinband already at 40G
With Infinband already at 40G and FC at 16G, the Ethernet meat grinder needs to quickly increase its grinding wheel to adopt 40G Ethernet and not just sit back with the promise of 100G sometime in the future. We've seen this over the last decade where Fiber Channel has embraced higher speeds in a more incremental fashion (1, 2, 4, 8and now 16Gbps) at a much lower cost point than the "less flexible" Ethernet, which just offered 1Gbps and 10Gbps (take it our leave it).
The Ethernet community need to be more "flexible" to the faster changing needs of Data Center SAN's, they've made a good start agreeing to standardize 40G Ethernet (though with a few cries of heresy) but we have to wait and see if Ethernet vendors offer this flexibility to customers or if they insist on sticking to the 10x dogma.
Infiniband is still lower latency.
And from a pure (predictable) latency perspective native Infiniband (RDMA) will beat 10GigE everytime. Leaving aside the cheaper cost per port/higher throughput.
I'm sure ethernet will catch up - but we're a few years away from anything production ready.
IT infrastructure monitoring strategies
Agentless Backup is Not a Myth
Top 10 SIEM implementer’s checklist
Steps to Take Before Choosing a Business Continuity Partner
Enabling efficient data center monitoring
