Mixing network traffic types on Ethernet

A recipe for success - or disaster?

Secure remote control for conventional and virtual desktops

Expert Clinic Is Fibre Channel over Ethernet (FCoE) a panacea for the difficulties inherent in running separate storage and general networks?

FCoE is not taking off and, indeed, there have been articles published that suggest it may die before it arrives. Let's look again at the idea of mixing general networking traffic and storage networking traffic. Is it, or can it be, a recipe for success?

Are we trying to mix different kinds of road traffic or converging road and rail when combining general and storage traffic on one network?

Tony Lock of Freeform Dynamics thinks this mixing can be done but needs a lot of careful preparation and monitoring. It won't work "out of the box."

Tony Lock - Programme Director, Freeform Dynamics

Tony Lock

For much of the past decade the “networking” that connects users to their applications and services has often been taken for granted. But if general networking has been less than widely appreciated, the networks that tie servers to storage have been almost invisible to everyone bar the all-too-few skilled storage administrators who delve into to the art. Many organisations are contemplating significant changes to their IT architectures and IT vendors are promoting a raft of new technologies in the storage arena. This brings up the issue of whether it is it feasible to bring specialised storage networks and general Ethernet data networks together?

General networking is now very firmly based around the TCP/IP protocol and Ethernet while storage networking is still grappling with many protocols and network technologies. Among these, perhaps the most firmly entrenched is Fibre Channel, a lossless, deterministic protocol designed to ensure that any data sent to the storage disks gets there with minimum latency and very little chance of data corruption. These characteristics were not originally enshrined in the standard Ethernet protocols employed for general networking. While work to incorporate these has continued apace, support is still far from universal and the need for new approaches, equipment and tools has a resulting uptick in the equipment cost.

The networks that tie servers to storage have been almost invisible to everyone bar the all-too-few skilled storage administrators who delve into to the art.

So is it possible to get storage traffic and general network traffic to cost-effectively share a common cabling system, namely Ethernet? Technically the answer is yes, as protocols such as FCoE (Fibre Channel over Ethernet) and iSCSI have now matured sufficiently for mainstream adoption. It is apparent that organisations are beginning to contemplate converging their networking stacks as well as management – and with it the cabling infrastructures they employ.

The cost savings of cabling only one storage and data network are attractive, as is the flexibility potentially available for dynamic reconfiguration. But there are significant challenges between contemplation of such a change and things happening on the ground. For one, network cable infrastructures have very long lifetimes and replacing them is by no means simple to achieve.

FreeForm Dynamics Chart

Impactful initiatives for the data centre.

Despite this, as the chart indicates, organisations are becoming aware of the impact that converged networks could have, even if their implementations have yet to ramp up. That said, there are questions concerning the feasibility of running storage and application traffic on the same physical networks without either users or applications, being affected by service degradation. Can you do it?

The answer is ‘yes’, but expecting it to just work out-of-the-box would be asking a bit much. Managing the complexity of convergence will require sophisticated traffic monitoring and management tools to ensure that service quality is maintained at desired levels. An understanding of the base line for the current usage and service quality combined with projections of future growth requirements of each of the services to be delivered over the network is essential – yet few organisations undertake such management processes routinely today.

To use a network to support all forms of traffic will require new processes to be put in place as well as the use of new tools. The lack of much “established best practice” – to date coupled with significant costs and operational challenges – make it highly likely that the adoption of combined networks will take place over years, not months.

Tony Lock is a As Programme Director at Freeform Dynamics, responsible for driving coverage in the areas of Systems Infrastructure and Management, IT Service Management, Outsourcing, and emerging hosting models such as Software as a Service and Cloud Computing.

Greg Ferro, our next expert, looks at the 'how' of the issue.

Greg Ferro - Network Architect and Senior Engineer/Designer.

Gregg Ferro

A key element of the decision to move to a Network Fabric is understanding that a Storage Fabric performs like a SCSI cable, or a channel: it can be emulated and adapted for a dynamic Ethernet network. This is true for both FCoE and iSCSI. FCoE uses the encapsulation of Fibre Channel into an Ethernet frame as a simple wrapper. No modification of the FC frame is performed, except to ensure that the FC MTU is 1440 bytes. Both FibreChannel and iSCSI are simple mechanisms for transporting SCSI commands from the host to the disk drive, where the disk drive is emulated as a LUN.

In data networks, emulating a “channel” is more difficult. Ethernet is designed as lossy protocol and, because of this, applications that use Ethernet are designed to tolerate delay or even loss. For example, the TCP/IP protocol will recover any lost packets, and re-order packets to recover from this network loss. An Ethernet switch is able to buffer data to some extent, since Ethernet is a lossy protocol and must decide which frame is to be transmitted in the event that a temporary overload occurs. Thus a short burst of network traffic of 100 milliseconds can be buffered for another 50 ms before being forwarded.

Data traffic is able to use available bandwidth that is not used by storage because storage data is bursty and relatively small in volume.

Data Centre Switches are a new category of Ethernet switches that provide much greater bandwidth, performance, low latency and improved resilience.

When an Ethernet frame is received on a Switch, it is placed into a memory location and waits for forwarding. For storage traffic this isn’t acceptable since the end-to-end delay should be as low as possible to ensure that storage throughput is fast. The Ethernet Switch is configured to immediately service the buffer that contains storage data as well as to forward the frame at the next interval.

In the event that a buffer overflows, Fibre Channel protocol requires any switch or host in the path to actively signal back to the source that an overflow has occurred – so that the source can cease sending until a signal is received stating that sending can resume.

Enhanced Transmission Selection (ETS IEEE802.1Qaz) provides a way to identify and group traffic so that Priority-based Flow Control (PFC IEEE802.1Qbb) can provide link-signalling for each hop to pause the sender group if congestion occurs. ETS is also the basis for a dynamic provisioning feature for a switch, to signal configuration data between switches to ensure that the configuration is correct between devices.

The combination of PFC and ETS mean that storage data can be selected and handled in the same way that a (Fibre Channel) SAN performs. Data traffic is able to use available bandwidth that is not used by storage because storage data is bursty and relatively small in volume. And a DCB (Data Centre Bridging) network works equally well for iSCSI, NFS and FCoE – giving Storage Administrators more choices.

Greg Ferro describes himself as Human Infrastructure for Cisco and Data Networking. He works freelance and has spent time at financial institutions, service providers, working for resellers and dot coms, in both the largish and smallish companies


Combining the two types of traffic – general LAN stuff with storage traffic – is technically feasible. This is only if, and it's a big if, you use the right kind of Ethernet gear and standards – such as DCB, PFC and ETS. It can be feasible if you carefully plan it, provision your upgraded Ethernet infrastructure appropriately and monitor it closely. There are no "Best Practices" set down, and Lock suggests that your existing network management processes will need extending and modifying to work well.

It is obvious FCoE is no plug 'n play, drop-in replacement or upgrade for ordinary Ethernet. So far there is little indication that converged storage and general networking are taking off.

The biggest reason for this, of course, is the cost of upgrading Ethernet combined with the investigative and implementation difficulties of managing the switch to converged networks. FCoE is no panacea. Whether it is going to be a generally effective route to convergence, we just cannot say. ®

Internet Security Threat Report 2014

More from The Register

next story
The cloud that goes puff: Seagate Central home NAS woes
4TB of home storage is great, until you wake up to a dead device
Fat fingered geo-block kept Aussies in the dark
You think the CLOUD's insecure? It's BETTER than UK.GOV's DATA CENTRES
We don't even know where some of them ARE – Maude
Intel offers ingenious piece of 10TB 3D NAND chippery
The race for next generation flash capacity now on
Want to STUFF Facebook with blatant ADVERTISING? Fine! But you must PAY
Pony up or push off, Zuck tells social marketeers
Oi, Europe! Tell US feds to GTFO of our servers, say Microsoft and pals
By writing a really angry letter about how it's harming our cloud business, ta
SAVE ME, NASA system builder, from my DEAD WORKSTATION
Anal-retentive hardware nerd in paws-on workstation crisis
prev story


Choosing cloud Backup services
Demystify how you can address your data protection needs in your small- to medium-sized business and select the best online backup service to meet your needs.
Getting started with customer-focused identity management
Learn why identity is a fundamental requirement to digital growth, and how without it there is no way to identify and engage customers in a meaningful way.
Reg Reader Research: SaaS based Email and Office Productivity Tools
Read this Reg reader report which provides advice and guidance for SMBs towards the use of SaaS based email and Office productivity tools.
Choosing a cloud hosting partner with confidence
Download Choosing a Cloud Hosting Provider with Confidence to learn more about cloud computing - the new opportunities and new security challenges.
Intelligent flash storage arrays
Tegile Intelligent Storage Arrays with IntelliFlash helps IT boost storage utilization and effciency while delivering unmatched storage savings and performance.