The DCB world is not enough
TRILL needed too
As Fibre Channel (FC) SAN storage gets emulated on Ethernet as FCoE, it's becoming clear that enhanced Ethernet is not enough; we need TRILL as well before full FCoE becomes a reality.
To make FCoE (Fibre Channel over Ethernet) practicable we need FC-using applications on servers and FC-accessed storage arrays to believe they are still using FC and not FC emulated on Ethernet. FCoE in other words has to behave like physical FC.
It's well understood that emulated FC must not drop frames and that frames should arrive quickly and not be delayed as they cross the network. Either condition would break FC, which was designed to have sole occupancy of the network it ran on.
Ethernet is a multi-user network with, for example, LAN and iSCSI traffic running on it. Adding FCoE to the mix means more data types crossing the network. The network's traffic load is generally increasing anyway because of the onrush of server visualisation, the rise of multi-core servers, and the steady increase in the number and capacity of the storage arrays the apps running in the virtual machines on multi-core servers access.
This data centre Ethernet is going to be congested and also going to have many, many switches sitting there between the servers, client PCs on the LAN, storage arrays and wide area network ports. The wish is to virtualise this network so that its resources can be more effectively used and its internal configuration altered without reconfiguring applications in servers and the other end points and vice vera.
Against this background how is end-to-end FCoE going to happen. There are two overview aspects to this we need to consider. The first is to look at how Ethernet can be made to perform like Fibre Channel. The second is to see how this Ethernet can provide high availability, load-balancing and high bandwidth as a basic feature so resource-hungry higher-level networking features don't have to be used.
Data Centre Bridging
The IEEE is developing Date Centre Ethernet (DCE) to provide a lossless and low-latency transmission of data, such as FCOE frames, across the network, traffic flow control in other words. There is a Data Centre Bridging task group working on this and looking at four aspects of the problem.
- Congestion Notification for congestion management - the 802.1Qau project.
- A priority-based flow control mechanism to ensure no data loss in congested networks, referenced by 802.1Qbb.
- Enhanced Transmission Selection (ETS) to assign network bandwidth to classes of traffic. The 802.1Qaz project refers to this.
- A discovery and capability exchange protocol to tell neighbours in a DCE network their capabilities and configuration to enable consistent configuration across the network. This is IEEE 802.1AB.
An IEEE 802.3x flow control standard uses a Pause frame to tell a sender to not send any more messages for a specific time. A Pause frame time value of 0 tells the sender it to resume transmitting at once. Ethernet provides an ability to set priority bits in an Ethernet message header. It is anticipated that the 802.1Qbb workgroup will combine 802.1Q prioritisation with the 802.3x Pause mechanism to provide priority-based flow control.
The idea would be to give FCoE traffic a high enough priority so it would sail through a congested Ethernet network while other traffic was delayed. If all this works well then this enhanced Ethernet could transmit FCoE traffic as well as physical Fibre Channel could transmit FC frames.
But DCE would still be limited in that it could not easily scale up as much as, or as easily as, the onrushing stampede of virtualised and multi-core servers demands. That's where TRILL comes in.