Original URL: http://www.theregister.co.uk/2011/08/15/xsigo_private_virtual_interconnect/

Xsigo automagically floats cloud east and west

Not just north and south

By Timothy Prickett Morgan

Posted in Cloud, 15th August 2011 19:58 GMT

Xsigo Systems, which for years has been trying to carve a niche for itself in the virtualized network and server rackets, is taking another crack at it with a version of its VP family of I/O Directors, tweaked to provide direct and movable links between virtual machines on clustered servers.

Such traffic is commonly called East-West traffic to distinguish it from North-South traffic running up from servers through switches and out onto the network – and its an increasing part of the load on cloudy infrastructures. Virtual machines flit around the server pools like lightning bugs, and yet they need to remain connected to other VMs to run their workloads.

The problem with the current way of handling East-West traffic is that it relies on virtual LANs –VLANs – in the Layer 2 network. Each time you move a virtual machine, you have to reconfigure the VLANs.

If you are doing this by hand, VLAN reconfiguration can take hours and people can easily introduce errors, explains Jon Toor, vice president of marketing at Xsigo. And if you use a virtual switch, then there is another layer of complexity added to the stack that is not, says Toor, particularly agile because you still have to configure switch ports, virtual servers, and virtual switches by hand.

Xsigo East-West traffic

Xsigo's I/O Directors can now better handle server-to-server traffic – often called East-West traffic

To help make this easier, Xsigo has cooked up the Xsigo Server Fabric – a layer of abstraction software for its VP780q and VP560q I/O Director switches that creates and manages end-to-end connections between VMs on servers in the pool. This connection does away with the whole concept of VLANs and makes a private virtual interconnect between those VMs through the I/O Director switches. Xsigo has cooked up its own virtual network interface card that looks like a Broadcom NIC, and the VMs see this point-to-point link as simply two Broadcom NICs talking to each other.

"Today, you need to get the networking guy involved in every process," says Toor. "But with what we are doing, the virtualization layer runs on the private virtual interconnect and the network and server admins do not have to do their choreographed ballet." When a VM moves from machine to machine, the I/O Director just keeps track of what it was linked to, and moves the links.

If you want to create VLANs atop the server fabric – creating what is in essence a virtual private data center with private connections between VMs, their storage, and external networks – you can do that as well. You can also tier storage and network connectivity through the I/O Director, offering different service-level agreements for bandwidth and latency. The back-end of the I/O Director hooks out to routers and external networks, and up to eight I/O Directors can be linked together into a flat InfiniBand fabric.

Xsigo Systems VP780q and VP560q I/O Directors

The VP780q (left) and VP560q (right) I/O Directors using QDR InfiniBand.

Xsigo has two different flavors of I/O Director boxes, one that supports InfiniBand interconnects between server nodes, and another that supports Ethernet. Both machines are based on underlying InfiniBand technology, with the I/O Directors being upgraded to 40Gb/sec InfiniBand last October, and versions that plug into Ethernet ports on servers coming out a year ago.

The VP780 I/O Director has 1.5Tb/sec of aggregate bandwidth and it is based on chips from Mellanox. The 4U rack unit has 24 ports hooking back into the servers, and up to 15 I/O modules for linking out to networks and storage. Each module has four Gigabit Ethernet ports, one 10 Gigabit Ethernet port, two 4Gb Fibre Channel ports, and an SSL encryption offload module. The Ethernet modules can support iSCSI disk arrays and the Fibre Channel links go out to SANs. The VP780 can plug into an expansion switch that allows it to be hooked up to hundreds of servers. The VP560 has the same 24 ports back to the servers, but only has four I/O modules in a 2U chassis.

The Xsigo Server Fabric is only available on the InfiniBand versions of the I/O Directors for now and will run on the original 20Gb/sec as well as the new 40Gb/sec InfiniBand-based directors. Toor says that the server fabric and its private virtual interconnect will eventually be available on the Ethernet models.

The Xsigo Server Fabric consists of the XMS Management Software, a GUI and command-line tool that was previously used to create virtual NICs and virtual host-bus adapters, and now manages VM-to-VM links on a server cluster.

The I/O Director needs to be upgraded with Server Fabric Suite 1.0 software that implements these VM-to-VM links over the switch; this software will be available in December and will cost $1,000 per physical host server. The VP780q director costs $45,000, while the VP560q director costs $35,000, and the XMS management console costs $10,000 per director switch. ®