Symmetrix and the death of monolithic arrays
V-Max is modular through and through
EMC's Symmetrix V-Max, announced today, represents the death of monolithic storage arrays and the rise of combined scale-up/scale-out modular arrays built from commodity components tied together with ASICs, very clever software and geographical distance cluster interconnects. It's also a tribute to 3PAR's T-Class InServ design and solidifies a trend to modularity seen already with the DMX-4.
The basic V-Max building block is a pair of quad-core Xeon 2.3GHx (5400) processors, 16 host and 16 disk enclosure ports (8 each per Xeon quad-core), 128GB of global memory, an EMC ASIC to handle the global memory access, and the RapidIO interconnect endpoints.
EMC can now ride the Intel CPU evolution curve and we can look forward to Nehalem-based V-Max engines, roughly twice as powerful as the existing ones at the quad-core level. (V-Max Plus anybody?) These should plug in and play with the existing 5400 Xeon-based ones providing both a scale-up upgrade path and investment protection.
We know that V-Max can scale to 8 V-Max engines with the current release and that FAST software is coming to provide automated data movement between the V-Max storage tiers (flash disk - Fibre Channel disk - SATA disk).
Barry Burke, EMC's senior director and chief strategy officer for the Symmetrix Product Group, says that EMC is developing software to leverage flash drives as something more than just another tier of storage, but doesn't explain what that means. Certainly, in the future, we might see 2-tier V-Max arrays; ones with just a fast flash tier and a bulk storage SATA tier.
V-Max memory considerations
The 8-engine V-Max has eight sets of storage enclosures behind the engines combined into a single logical pool of storage capacity, with the storage processors enjoying the use of global memory made up from each engine's own 128GB local memory, and presumably combined and kept coherent by the EMC ASICs. Any access to memory by a V-Max processor is treated as a local access, and remote accesses are virtualised to seem local.
As well as enabling two kinds of memory access, one local and the other remote to a peer V-Max engine, the architectural design allows for a third kind of memory access and linkage. Burke describes it: "the Architecture allows for a third dimension of interconnect – a connection between different V-Max systems. This interconnect would not necessarily expand to share memory across all the nodes in two (or more) separate V-Max arrays, but it would allow multiple V-Max arrays to perform high-speed data transfers and even redirected I/O requests between different Symmetrix V-Max 'virtual partitions.'"
"This capability of the Architecture will be leveraged in the future to 'federate' different generations of V-Max arrays in order to scale to even greater capacities and performance, and will also be used to simplify technology refreshes. In the future, you’ll be able to “federate” a new V-Max with the one on your floor and non-disruptively relocate workloads, data and host I/O ports."
It seems to me that V-Max virtual partitions could have different characteristics. Data in a partition might be single instanced and compressed Celerra-style, for example.
V-Max uses an interconnect new to most of us - RapidIO. Burke states: "the first generation of the Symmetrix V-Max uses two active-active, non-blocking, serial RapidIO v1.3-compliant private networks as the inter-node Virtual Matrix Interconnect, which supports up to 2.5GB/sec full-duplex data transfer per connection – each 'director' has 2, and thus each 'engine' has 4 connections in the first-gen V-Max."
Why was RapidIO used and not InfiniBand? After all, RapidIO has evolved from bus technology whereas InfiniBand is already used in processor clusters and storage clusters (Isilon). Sun has just chosen Mellanox' 40Gbit/s InfiniBand ConnectX adapters and InfiniScale IV switch silicon for its new line of Sun Blade modular systems and Sun Datacenter Switches.