EMC crying two SAN breakup tears

Analysis Dell EMC is working on fixing the increasing split between primary storage data on flash and capacity data storage on object arrays by logically combining them underneath a 2 TIERS software abstraction layer.

The starting point is that SAN disk or hybrid flash/disk arrays are diverging into separate arrays under the pressures for faster access to primary data and more space being needed for secondary data.

One array is for primary data on flash and is called a hot edge or fast tier in Dell EMC's scheme of things.

The other is for secondary (nearline) data on disk accessed through an object storage system, which could be on- or off-premises and has slower access to data, but is significantly lower cost per GB than the hot edge store.

Dell EMC calls this a cold core or capacity tier, and the company is working on a 2 TIERS abstraction layer [PDF] to do this. A slide from that deck shows its starting premises:

We would disagree with the third sub-bullet on this slide, as on-premises file or block-accessed capacity disks are not being replaced by the cloud per se, but by object-accessed capacity disks that could be either on- or off-premises (in the cloud.)

EMC thinks that the hot edge could be in the hundreds of TBs area, while the capacity tier is much larger; think hundreds of petabytes.

A unifying abstraction layer would have metadata indicating on which tier a data item is located and where it is within that tier. There would then be a single global namespace for data items, one capable of encompassing trillions of objects. The layer code could also move data between the tiers as necessary, using a policy-driven approach for automated data placement (tiering.)

It believes that other approaches to logically unifying the two tiers can run out of metadata space in the hot edge, leading to slower capacity tier access for additional metadata. The way to fix this is not to massively increase metadata storage space in the fast tier, but to cache metadata in it instead.

The way this would work is by imagining that a set of client servers accessing the fast tier would send requests to a shared DSSD array using RDMA access, or they access a virtual flash SAN using ScaleIO to aggregate local direct-attached flash drives.

Behind this is an object-storage-based capacity tier, which could be an Isilon array or an ECS scale-out commodity appliance cluster. These two tiers can grow or shrink independently.

Note that this overall scheme, minus the flash-based fast tier, is somewhat similar to Quantum's StorNext product, which is sold into the entertainment and media workflow market.

Each server would access 2 TIERS software, which presents a SAN via a POSIX API and single namespace to the server's apps, and has policy-driven tiering to send old or unwanted data to the capacity tier. It maps the apps' access to that data into objects on the capacity tier.

The EMC software has the fast tier using a distributed Orange File system with a read-only, read-through translation service on a local FUSE file system. This service uses dynamically loaded namespaces (DLNs) for metadata tiering. A DLN points to a part of the global namespace, like a file system's sub-tree. Within that part are pointers to objects, like inodes in a file system directory.

We have no information on how DLNs are loaded or possibly pre-fetched.

With this general 2 TIERS scheme in mind, EMC suggests two ways to instantiate the idea, using a DSSD fast tier with Isilon or ECS capacity tiers:

An alternative is to host the whole thing in AWS using Omnibond's CloudyCluster, which deploys OrangeFS in AWS.

What we have here, in general, is an approach to a post-SAN/post-NAS array on-premises or public cloud world. The SAN/filer is broken into two separate pieces and logically re-combined using the 2 TIERS software with the broken SAN.

Comment

This kind of imaginative storage thinking is what we have come to expect of EMC, and there is no equivalent of it we know of in the storage development shops of its mainstream competitors. Indeed the only equivalent of such inventive creativity we can think of is in HPE's server division, where we see developments such as Synergy.

If Dell, infused with such EMC inventiveness, can apply this to its servers, then HPE would have cause to watch out. And if HPE could apply its Synergy creativity to its storage products, then Dell EMC would have stronger competition.

By the way, Dell EMC has registered the 2 TIERS trademark. ®