Oil the wheels of virtualisation with 802.1Qbg
The magic of protocols
Virtualisation enables dynamic workloads within a data centre by easing and automating virtual machine movement.
While the ability to move any virtual machine from any system located on any rack to any other system located on any other rack has become commonplace, elasticity of network configuration has lagged behind. A new protocol, 802.1Qbg, aims to change this.
Renato Recio, IBM Fellow and systems networking CTO, has taken the time to bring me up to speed on the problems of networking in large data centres and explain how 802.1Qbg helps to solve them.
“Large data centres are increasingly multi-tenant affairs. All layer 2 configuration that I need to do in the network gets automated,” he says.
“If I have a VLAN that needs to get created, rate limit controls that need to be applied, all of that can be automated. When that virtual machine moves around, it will move with that machine automatically. You don't have to go to the target server that's running the virtual machine and configure anything.
“802.1Qbg automates that layer 2 space in the physical network. If I want to do switching in the server – or external to the server – I can choose to have it done in the server or in the access switch.
“I may choose that mode because maybe I am running some intrusion detection in that physical switch. I can send all that traffic to a mirrored port. That's one example; there might be a lot of other examples.”
In today's world, packets probably do not flow from one server directly to their intended destination. There are often intermediate systems along the way, such as a firewall, an intrusion detection system, a deep inspection or debugging application. Packet flow is managed at the switch: VLANs, quaity of service, access control lists and other network configuration items are all defined one network device at a time.
In the old days, this worked fine. Servers take time to build, configure and move into production. Meanwhile, the necessary network specifications can be provided to the network team, a port provisioned and all additional network configurations made ready ahead of deployment.
As Recio explains, virtual machines disrupt this cosy little arrangement.
“The creation of 802.1Qbg for us was so that clients don’t have to manually configure stuff on the network. Prior to virtualisation, network admins configured things once for a server, and then they were done,” he says.
“After virtualisation, server admins would bring up a virtual machine right now. The physical network is cumbersome, it just doesn't work well in the virtual world. Layer 2 had to be automated.
“Before that virtual machine even starts to talk, you want [all your network rules and configuration] applied. What Qbg does is apply the configurations to the network switch and associate it with a virtual machine. I call this ‘network configuration state.’ The IEEE term is Virtual Station Interface [VSI] type.”
Divide by four
The IEEE standard called 802.1Qbg solves network configuration automation by defining a standard for replacing a dumb hypervisor’s switch with something a little more cooperative. 802.1Qbg-compliant hypervisors and access switches can exchange information about which virtual machine packets belong to, where they should be allowed to go, and how fast.
Recio explains that 802.1Qbg basically defines four protocols.
"The first is EVB [Edge Virtual Bridging]. What this does is very basic. It says ‘I support Qbg, do you? What kind of modes do you support? What are you capable of, here is what I'm capable of,’” he says.
“The second is the edge TLV [Type Link Value] protocol. This lets you transfer that VLSI [very large scale integration] configuration information in a reliable fashion. VSI Discovery and Configuration Protocol [VDP] basically does the association with the network state. There is a fourth protocol called multi-channel that lets you run both on both ports.
“When someone says they support Qbg, at a minimum they must support EVB. When we at IBM say we support Qbg, we mean we support EVB, Edge TLV Transport Protocol and VDP. We support all of Qbg.”
The virtual machine might end up on a different server
Let's take a look at what this means in practice. To set up a new virtual machine on a network without 802.1Qbg, I would have to know either its MAC address or IP address beforehand. I would then set up a bunch of rules, and hope neither the MAC nor the IP changes.
Alternatively, I could manage the individual port the server was attached to, but that would then apply to every virtual machine attached to that network port. If I want to move the virtual machine, per-port management can be burdensome. The virtual machine might end up on a different server and then I am back to square one.
Assuming that the various possible servers the virtual machine can move to are either located on the same physical switch, or one that the original switch is happily communicating with, per IP or per MAC address might work. The configuration should follow the virtual machine around regardless of the port it is accessing the network from.
United we stand
Templates, however, present a genuine problem. What if I want to clone that virtual machine? What if that virtual machine is stateless? In both situations I need a way to tie the network configuration to a class of virtual machine, not to a specific MAC or IP.
In an 802.1Qbg world the access switches, the hypervisor and the management tools all cooperate. Instead of configuration being done in the switch, the network configuration lives with the virtual machine.
When you create a new virtual machine, you define the network configuration. When that virtual machine is moved, the configuration software contacts both the destination hypervisor and relevant attached physical switch.
It sets up the appropriate network configuration, enables it, and then moves the virtual machine. The network configuration moves ahead of the virtual machine.
We have come to think of multiple physical servers with installed hypervisors as a pool of compute resources that thanks to modern management tools behaves like a single entity. 802.1Qbg is an important step along the same path for networking.
The days of manually configuring network hardware for every virtual machine move are gone. 802.1Qbg allows for dynamic virtual machine migration, while still preserving security and visibility of packet flow right down to the individual virtual machine.
With 802.1Qbg, we can finally start treating our switches, both physical and virtual, as one single virtual switching pool. It’s about time. ®