Off-the-shelf servers spar with million-dollar storage arrays

Cheap-as-chips Nexenta squares up to EMC and NetApp over performance

Intelligent flash storage arrays

VMworld 2011 Cheap and cheerful storage from Nexenta matched EMC and NetApp's multi-million-dollar systems in VMworld 2011's Hands On Lab, and took up some of the slack from its rivals when difficulties arose on the first day. EMC and NetApp have cooly brushed off Nexenta's claims it outperformed them.

The storage infrastructure for the Hands On Lab (HOL) at the show in Las Vegas was provided by the trio of vendors after they were approached by VMware, with Nexenta getting the call in February.

VMware wanted Nexenta to provide the storage component for a SuperRack concept, a commodity white box-based alternative to VCE's Vblock and the NetApp/Cisco Flexpod. Arista provided switching technology for the SuperRack.

Nexenta said it was responsible for up to 60 per cent of the Hands On Lab storage load at one time, despite only having a fifth of the work allocated to it, because of problems, it is speculated, with one of the other systems.

The lab ran on vSphere v5 in the cloud, a geo-distributed public cloud with data centres in Amsterdam (Colt), Miami (Terremark) and Las Vegas (Switch SuperNap). Each component lab was deployed using VMware's Cloud Director v1.5 (not the release generally available), and on NFS. We understand there were 480 lab seats, with dual-monitor workstations using VMware View desktops, and 24,000 lab seat hours of operation.

There were some 27 individual labs in the overall HOL, and around 5,000 people used the HOL in total during its 50 hours of operation.

Storage speeds and feeds

EMC provided two VNX 7500 arrays configured with three file blades plus a standby blade. The arrays came "with SSD, FAST Cache, FAST VP, and loads of 10GbE". The bulk of the EMC load ran on NFS, and VNX VSAs ran in many of the labs.

We understand one VNX 7500 was an all-flash configuration with 96 SSDs. The other had more than 140 15K RPM hard drives and an SSD storage tier. The estimated street list price for these two VNX configurations was $2.6 million.

NetApp supplied, we understand, two FAS3270 arrays, each with four Flash Cache cards and 144 15K RPM hard drives. The estimated list price for the pair was $1.4 million.

Nexenta provided four instances of its ZFS-based NexentaStor, known as SuperRack, running on SuperMicro Westmere hardware: essentially, generic commercial off-the-shelf (COTS) X86 machines. A pair of high-availability servers were used per instance. Data was stored on around 160 7,200 RPM 2TB Seagate SAS hard drives, with STEC 8GB Zeus RAM and Zeus IOPS (200GB) SSDs, which were used for read and write caching. It says its systems for the show cost $325,000 at list prices.

Storage performance

EMC said that:

  • there were 131.115 terabytes of NFS traffic in the labs;
  • there were 9.73728 billion NFS operations;
  • the average I/O size was 14KB;
  • internal average VNX NFS read latency was 1.484ms; and
  • internal average VNX NFS write latency was 2.867ms.

A total of 13,415 lab sessions were run, creating and destroying 148,138 virtual machines over four days. A VM was created every 1.215 seconds and there were 10 to 25 VMs per lab seat session, with each VM up to 26GB in size.

Nexenta claims 7.9 billion storage operations were performed by the four NexentaStor systems. They ran four out of eight HOL vertical app areas for the duration of the show. The peak load was 154,000 4KB NFS operations at sub-1ms latency and the peak bandwidth on a single Nexenta controller was 1,300MB/sec. The average latencies were:

  • Read: 2.194ms
  • Write: 2.133ms

Load sharing

Nexenta's CEO, Evan Powell, blogged:

There were instances during the show when NexentaStor took on a far greater workload than had originally been scheduled – for reasons we can only speculate. This meant that we were at times running over 50 per cent of the verticals whereas we originally were expected to carry approximately 20 per cent of the load. NexentaStor handled this without breaking a sweat.

I don’t want to publicly speculate about why we took on the additional workload – it’s something the organisers and other vendors may be able to share, although they may be reticent to do so.

EMC's Chad Sakac, a vExpert VP in the VMware Technology Alliance, said:

These HOL situations are always frantic, always push everyone (both the humans and the technology) to the limit, including VMware and the partners who support them. Every year, most of the folks - VMware and supporting partners - have a 'bad moment'. Frankly, it’s more about how one copes, in my opinion.

This year, it was relatively uneventful compared with 2009 and 2010. Yes, there were some periods where storage latency was bad. Yes, it placed a lot of stress on all the storage platforms. In particular, read latency in one case was pretty hardcore.

A NetApp source said: "I checked with our guys who were at the VM HOL. Zero failures. In fact, feedback was that we blew the rest away."

There was a DRAM failure on a Nexenta server but the high-availability failover moved the processing load to the paired server and users didn't notice the blip.

Outside the Nexenta environment, a misconfigured VM caused some performance problems as well, with that being fixed by upgrading its memory from 4GB to 8. On top of that a network latency issue, perhaps the same read latency issue Sakac referred to, affected the first day of the lab sessions.

The HOL will be a feature of VMworld in Denmark, to be held between 18 and 20 October, and the EMC, NetApp and Nexenta storage systems will be present there.

COTS work is one moral of the story

Flash played a huge role in all the HOL storage systems. In Sakac's view:

From a storage reporting standpoint ... SSD is the biggest game changer across the industry – and its commoditisation (in full swing) will change a lot. This will impact everything: server-side storage models (SSDs, PCIe-based flash, and virtual storage models), all SSD arrays, and the critical role of both caching and auto-tiering in shared storage models.

If you’re a storage vendor whose primary value is in technologies that have less relevance in that transition, or you are unable or unwilling to adapt, you’re in for a rough ride.

Another story here is that the cheap and cheerful off-the-shelf hardware in the Nexenta ZFS storage stood toe to toe with EMC and NetApp storage costing four to eight times as much.

Powell said: "We stood tall alongside the largest, most established storage brands with a comparable solution that comfortably held its own, despite it costing just a fraction of the EMC and NetApp offerings."

Nexenta kicked serious arse at VMworld - and EMC, NetApp and VMware each know it. ®

Intelligent flash storage arrays

More from The Register

next story
NSA SOURCE CODE LEAK: Information slurp tools to appear online
Now you can run your own intelligence agency
Fat fingered geo-block kept Aussies in the dark
NASA launches new climate model at SC14
75 days of supercomputing later ...
Yahoo! blames! MONSTER! email! OUTAGE! on! CUT! CABLE! bungle!
Weekend woe for BT as telco struggles to restore service
Cloud unicorns are extinct so DiData cloud mess was YOUR fault
Applications need to be built to handle TITSUP incidents
BOFH: WHERE did this 'fax-enabled' printer UPGRADE come from?
Don't worry about that cable, it's part of the config
Stop the IoT revolution! We need to figure out packet sizes first
Researchers test 802.15.4 and find we know nuh-think! about large scale sensor network ops
SanDisk vows: We'll have a 16TB SSD WHOPPER by 2016
Flash WORM has a serious use for archived photos and videos
Astro-boffins start opening universe simulation data
Got a supercomputer? Want to simulate a universe? Here you go
prev story


Go beyond APM with real-time IT operations analytics
How IT operations teams can harness the wealth of wire data already flowing through their environment for real-time operational intelligence.
Why CIOs should rethink endpoint data protection in the age of mobility
Assessing trends in data protection, specifically with respect to mobile devices, BYOD, and remote employees.
A strategic approach to identity relationship management
ForgeRock commissioned Forrester to evaluate companies’ IAM practices and requirements when it comes to customer-facing scenarios versus employee-facing ones.
Reg Reader Research: SaaS based Email and Office Productivity Tools
Read this Reg reader report which provides advice and guidance for SMBs towards the use of SaaS based email and Office productivity tools.
Protecting against web application threats using SSL
SSL encryption can protect server‐to‐server communications, client devices, cloud resources, and other endpoints in order to help prevent the risk of data loss and losing customer trust.