Cumulo Ethernet: Building Ethernet cloud fabrics

Expert Clinic Scaling up Ethernet for the cloud means longer wires and many more of them, plus many more switches. What are the effects of linking all this gear together? Do new risks emerge? Can we get economies of scale? What about security with multiple users of network links? Is ordinary Ethernet good enough?

Three experts put forward their views and they are different. One highlights the overhead and risks, a second tells us about the type of Ethernet needed, and our third guru talks about networking inefficiencies and problems and how these can be mitigated. Three experts; three different views.

Greg Ferro lays out the ground first, and his view is one that is initially antagonistic to realisable cloud efficiencies of scale. There are ways around the problems he outlines but they have risks themselves. It is definitely not plain sailing.

Greg Ferro - Network Architect and Senior Engineer/Designer

Scaling up Ethernet networks for a large scale Cloud appears, to the uninitiated, to be straightforward. Add some more switches, connect them together, and you will have more ports. But, you need to build a backbone that connects all of the Ethernet switches.

For switching, it’s similar. A few switches for tens of servers has no overhead, but a few hundred switches for several hundred servers' needs, has about 20-30 per cent overhead. Scaling further for cloud-style networks means even greater overheads. The costs incurred are not linear because the backbone switches require higher performance and better reliability. That is, hundreds of servers could be affected in a core switch failure, therefore redundant units are needed for SLAs (Service Level Agreements) that add nothing to the system performance.

To further compound the problem, the switch redundancy technology of Spanning Tree is an active/standby method. Thus, for every active connection there must be an unused standby connection to act as a redundant path in the event of switch failure. As a result, a significant percentage of the backbone is unused. All that power and space is, by modern standards, wasted.

Cloud Computing LAN networks are currently founded on deeply flawed technology with only limited controls.

As the number of logical servers increases, so does the physical server count, which creates a requirement for more Ethernet ports, which means more switches, and more switches means more switches to connect the switches. For each switch that connects to another switch, there are standby redundant units that are also necessary, that do nothing.

It’s like having managers of an engineering team — the managers don’t produce anything, but help with co-ordination of the team itself and with other teams. As you add more engineers you need more managers that are overheads to the actual purpose of the business. For each additional manager, you need more managers to manage the managers.

This is directly opposed to the logic of cloud efficiency. Cloud scaling is dependent on the network effect of non-linear cost increases as compute and storage volume grows. In short, the network effect means that as systems units (such as compute, storage or bandwidth) get larger, the costs get lower per unit. But for Ethernet networks of today, this isn’t true. As Ethernet networks gets larger, their complexity increases and their costs increase non-linearly.

This represents a significant business threat to cloud operating models. Unsurprisingly, a range of new technologies have sailed into view to help address these problems. Firstly, Layer 2 Multipath technologies, such as TRILL and SPBB, allow for Ethernet networks to move to Active/Active in full mesh connectivity. Second, the use of new silicon architectures such a Fat Clos Tree Fabrics, have delivered exponential increases in switch performance and reduced the number of “overhead” switch in the network core.

But the greatest Ethernet weakness is still not solved and is fraught with risks. Cloud Computing requires the dynamic reallocation of server resources without changing the IP address of the server ad therefore relies on coherent Layer 2 Ethernet segments. Ethernet is vulnerable to loops and is insecure against a range of unicast, broadcast or MAC attacks. This means that Cloud Computing LAN networks are currently founded on deeply flawed technology with only limited controls.

So far, there are few answers to this challenge. Unless of course, you completely adapt your software architecture to address it. But that’s not what Clouds are for, is it?

Greg Ferro describes himself as Human Infrastructure for Cisco and Data Networking. He works freelance as a Network Architect and Senior Engineer/Designer, mostly in the United Kingdom and previously in Asia Pacific region. He is currently focussing on Data Centre, Security and Application Networking technologies and spending a lot of time pondering design models, building operational excellence and creating business outcomes.

---

Our second expert is Brocade's Simon Pamplin, and he looks at metro-scale and WAN-scale Ethernet links, asserting that economies of scale can be realised.

Simon Pamplin - director pre-sales, UK and Ireland

The question for Cloud Services is what happens if the data has to traverse a metro or wide area network (WAN)? Ethernet has again evolved and has been upgraded to offer 100 Mbit/s, 1 Gbit/s, and now 10 Gbit/s speeds, into the Metro Area Network (MAN) and out to the WAN through ‘Carrier Grade Ethernet’.

Carrier Ethernet can be delivered over traditional (native) Ethernet-based networks and also over other transport technologies. Examples of underlying transport mechanisms are:

- Native Ethernet
- MPLS-based Layer 2 Virtual Private Networks (VPNs)
- IEEE 802.1ad Provider Bridges
- Ethernet over SONET

This unified approach cost effectively extends the compelling benefits of Ethernet to help both service providers and end users achieve substantial savings. Unparalleled economies of scale result from business, residential, and wireless networks sharing the same infrastructure.

[Cloud-scale Ethernet helps] service providers and end users achieve substantial savings.

Carrier Grade Ethernet is a ubiquitous, standardized service defined by five attributes:

Standardized Services- The Metro Ethernet Forum (MEF) has currently defined two standardized service types for the delivery of Carrier Ethernet: Ethernet Private Line (EPL) and Ethernet Virtual Private Line (EVPL).

Scalability - With many businesses seeing capacity growth rates of 20 to 30 per cent per annum, Ethernet scalability is seen as vital to meeting this capacity demand. Ethernet is already available in a wide range of interface speeds from 10 Megabits per second to 10 Gigabits per second. Intermediate speeds between standard Ethernet physical rates can be achieved using Link Aggregation.

Readability - Traditional LAN-based Ethernet was often perceived as a best-effort connectivity mechanism. Carrier Ethernet in contrast offers the capability to rapidly detect and recover from node, link, or service failures to offer a very high availability service to the end user. Recovery from failures, when they do happen, occurs in less than 50 milliseconds.

Quality of Service - Carrier Ethernet offers a wide range of granular bandwidth and QoS options. By defining attributes that are associated with the service, advanced SLAs can be offered to deliver the performance required for a target application. Carrier Ethernet allows the assurance of advanced SLAs using QoS guarantees that are made possible by service attributes associated with the service type:

Service Management - Through the ability to monitor, diagnose, and centrally manage the network using standards-based vendor independent tools

Carrier Ethernet and Ethernet-based services are seeing rapid adoption globally, driven by demand for cloud services provision. The compelling attributes of Carrier Ethernet, that is, standardised service definitions, scalability, reliability, Quality of Service, and service management, help providers realise significant savings in operational expenses annually and offer providers a strong competitive advantage.

Similarly, end users can realise substantial savings and higher performance by switching to a Carrier Ethernet service from their network service provider.

Simon Pamplin has over 18 years experience in the IT industry, both as customer and vendor. He has specialised in technical pre-sales and spent the last eight years at Brocade

---

Our third expert is Freeform Dynamics' Tony Lock, who is confident that WAN optimisation techniques will need to be used in cloud-scale Ethernets. He also thinks unanticipated problems may occur as users push the cloud Ethernet envelope.

Tony Lock - Programme Director, Freeform Dynamics

One subject that is getting plenty of attention in IT circles is “Cloud” although, truth be told, “Cloud” can be any of a number of different solution types. But with organisations looking at how they might utilise internal, external or hybrid clouds, one matter that cannot be ignored is the networking that will be used to interconnect everything. Are the networks of today ready for the requirements of tomorrow in terms of capacity, performance or security?

As every IT professional is aware, Ethernet was originally designed to handle traffic in a local area network rather than managing data flows across multiple sites or great distances. But over the course of the last two decades Ethernet has been deployed to support an increasingly broad array of protocols and its logical usage has been adapted to support operations across long distances.

So as “Cloud” moves slowly from the realm of the theoretical to the practical, questions are being asked of the network, sometimes for the first time in years. Whatever type of Cloud usage is envisaged, the network will be the glue that holds everything together and allows services to function and users to work. So just what implications will there be on networks as Cloud usage grows?

Perhaps the most obvious area that will need to be looked at concerns how data will be transported across wide area networks (WANs). Most of the protocols that are commonly used in networking, most notably TCP/IP, were, like Ethernet, not designed for use in WANs, and as a consequence can be inefficient when utilised in this context.

... every step change in performance will give rise to new and inventive ways to use and abuse the additional capacity.

To mitigate this, it is highly likely that the use of WAN optimisation and acceleration appliances will grow rapidly, not just as a result of “Cloud” but simply because even in relation to traditional distributed applications, organisations are looking to push more data across long distance communications links. Such devices will combine a number of techniques, including data compression and de-duplication, caching and packet data sizing, amongst others, to mitigate the challenges of moving data across wide area links thereby improving both data capacities and potentially reducing the latency in systems.

Moving from the question of service levels to risk, it will also be interesting to note how quickly organisations investigate whether they or their suppliers of “Cloud services” need to handle Ethernet traffic differently in public service environments or mixed private / public hybrid Clouds.

Many organisations already perceive there to be challenges around the “security” of public Clouds - partly as they will have data that resides beyond their own immediate abilities to manage it directly, and partly as a result of the potential for their systems to be running on systems and networks used by many organisations at once, i.e. in a multi-tenancy setup. For some organisations the answer may be to encrypt all traffic and data moving across networks outside of the business, while others may be comfortable that existing protocols give them all the security they require.

And finally, to make life more interesting, there is every likelihood that the adoption of WAN optimisation across external links along with 10 Gbit/s, 40 Gbit/s and even 100 Gbit/s Ethernet will ramp up in the coming year or two, slowly becoming more ubiquitous. The unwritten rule of networking may then kick in, which says that every step change in performance will give rise to new and inventive ways to use and abuse the additional capacity. High def video embedded in Cloud business applications anyone? Oh, they are there already, but this begs the question of do you actually know what is running on your networks today and what your business users’ plans for tomorrow will bring?

Tony Lock is a Programme Director at Freeform Dynamics, responsible for driving coverage in the areas of Systems Infrastructure and Management, IT Service Management, Outsourcing, and emerging hosting models such as Software as a Service and Cloud Computing. He also considers the role of financial models in relation to IT investment.

---

Points of view

Pamplin and Lock paint a promising picture. You can use Carrier Grade Ethernet to build metro and WAN Ethernet fabrics, and you can gain cloud efficiencies of scale. Futhermore, by using WAN optimisation techniques you increase the efficiency of cloud-scale Ethernet. E

Encryption can be offered to solve security problems but, be warned, users will probably push the could Ethernet envelope and introduce a whole new class of problems. That point seems highly likely to come true.

Ferro comes from a darker place, and he is crystal clear on his main point: Cloud Computing LAN networks are currently founded on deeply flawed technology with only limited controls. What to make of this? Tread carefully, slowly and treat out for problems in the areas he mentions. If he is right then Ethernet cloud fabrics could promise a world of networking hurt. ®