Cloud storage: Lower cost and increase uptime

Blocks and Files Listen and prepare to behold this vision. Storage arrays will become nearline vaults because storage memory will steal their primary data storage role.

There is a thundering great divide, a Grand Canyon, between server memory in the single digit terabyte area and "storage": double digit terabytes and up to multi-petabytes of data stored in disk drive arrays. Primary data is the data needed by server applications on demand, now, as close to this instant as possible.

Traditionally this data has been stored in disk drive arrays, with tiers of expensive fast or cheaper slow disk to store the most needed and less-needed data. Although such arrays have latterly taken to using SSDs, these are treated as very fast disks and nothing fundamental changes. A newer development is the appearance of all-flash arrays which don't treat the flash modules as disks and deliver data faster than a disk drive array.

We are at the beginning of an era in which primary data for access-latency sensitive applications, like VDI, is migrating from disk drive arrays to these all-flash arrays.

But, seen from a ten to twenty year viewpoint, these are primary data storage bus stops on the journey to another form of computing that takes the primary data storage responsibility away from arrays completely - from "storage" completely. This is the computing vision that is being pursued by Fusion-io, HP, SAP with HANA, Violin Memory and Seagate/Virident.

Here's an excerpt from a January 10, 2013 HP press release, "HP Opens New Centre of Excellence for In-memory Computing":

As new technologies and novel data sets are developed, all data-intensive, transactional and analytic applications will migrate to in-memory computing environments.

Since the launch of SAP HANA, HP has been at the forefront of innovating leading benchmarks and solutions. The first HP roadmap milestone of the new CoE is an 8-terabyte, single image development platform tuned for the SAP ERP and SAP Customer Relationship Management (CRM) applications on SAP HANA. Codenamed “Project Kraken,” this platform is the forerunner to the large memory systems scheduled for future availability.

An application executes with its entire data set in memory, which today is a single very large address space made up from DRAM and NAND and in which data is accessed as a memory-resident byte, block or page and not as a block or file stored on directly-attached or network-attached disks or SSDs. No host operating system's I/O subsystem is involved in such data access; it's a much faster memory-level access and applications will be accelerated two, three, four, five, ten, perhaps twenty or more times, depending upon how I/O-bound they are.

Storage memory, the NAND part of this, is non-volatile. Switch the system off and the data stays in place. Switch it on and the data is there, instantly. By the time storage memory becomes mainstream it may not be NAND. It could be a follow-on technology, such as a Resistive RAM implementation like HP's memristor or Phase-Change Memory. That does not change the essentials though, it merely makes the storage memory faster, denser and hopefully cheaper.

Primary data, an application's working set data, has begun a migration; trekking off networked disk, staying briefly on networked flash, but destined to arrive at storage memory inside servers. Copies of it, all of it or bits of it, will be held in storage memory too, where hyper-fast recovery from failure is needed.

Secondary data, the reference stuff, the colder information, the slower-to-restore protection copies, will be left behind in the disk drive arrays, serried shelves of twirling disk, gradually becoming as strange as tape libraries today to newly-qualified IT workers who take flash storage for granted.

These nearline vaults may be located in public clouds as well as in enterprise data centres and they will be fed by the storage memory server apps. Some kind of data transfer scheme will be needed but the complexity of today's backup, snapshot, RAID and replication schemes could be much simplified. Storage will become basically two-tier; nearline vaults fed by the storage memory servers, feeding tape archives in turn with the coldest, least-accessed data that must still be kept; nearline and offline.

Storage will lose its primacy in the data centre concerns of IT staff. Converged system stacks will consist of servers with storage memory and networking; just two basic components. These will be the data centre powerhouses with storage having a secondary role, like the fuel tank in a car. The interest is in the drive train, the engine and transmission. The car needs fuel but fuel tank technology is behind the scenes, boring even.

Storage has an unsolvable problem; it's slow. The only way to deal with slow access to stored data is to take it out of storage altogether and dump it it in memory instead. Today's hot technology is going to become tomorrow's archaic technology leftover. Love the times we're in; these are storage's glory days and they're coming to a close. ®

Steps to Take Before Choosing a Business Continuity Partner