Enterprise storage: A history of paper, rust and flash silicon

Has anything really changed since the punch card era?

Designing a Defense for Mobile Applications

The others became less profitable, fell into debt and collapsed or were taken over. At one time there were more than 200 disk drive makers. Now there are just three: Seagate, Toshiba and Western Digital/HGST.

Seagate ST-412 disk drive

Seagate's ST-412 disk driveE

The rise of disk drive arrays and their falling cost/GB meant that they could take over backup data storage duties, especially if repetitive data in backup data sets could be removed by deduplication technology. This was a devastating blow to the tape backup industry, leading to rapid format consolidation. Now there are effectively only two mainframe tape suppliers with proprietary data formats left, IBM and Oracle, and effectively just one server computer format, LTO. This is the Linear Tape Open format, now in its sixth LTO-6 generation. The LTO format is owned and developed by a three-member consortium; HP, IBM and Quantum.

While disks were loaded with more and more data and technology enabled them to be read and written to faster and faster, one huge problem remained. Disks could, and did, fail, causing all the stored data to be lost. The problem was avoided by making extra copies of data. The data was mathematically processed so that reduced amounts were needed, lowering data protection costs, with RAID technology - Redundant Arrays of Independent Drives. Different RAID schemes include RAID 0, 1, 2, 4, 5, 10, with each individual scheme optimised for data access speed, data protection surety, and capacity needs or combinations of these.

With RAID schemes the contents of a failed drive are rebuilt on another drive using the RAID data held on the surviving drives. But such RAID rebuilds took longer and longer as drive capacities grew.

hitahi t7k500 hard disk drive

HGST 7K500 3.5-in hard disk drive

As the number of drives in arrays grew to the hundreds and then past the thousand mark, it became necessary to protect against a second drive failure when recovery from a first drive failure was under way, and so the RAID 6 scheme was devised.

Disk drive growth rapidly increased the total amount of stored digital information and researchers reckoned digital storage capacity passed analogue capacity in 2002 and the rate of increase continues to accelerate.

IDC Digital Universe growth

IDC Digital Universe growth in Exabytes (IDC/EMC June 2011)

As the speed at which the disk ran was critical to how quickly data could be read or accessed, so the manufacturers pushed their technology to deliver ever faster rpms. Disk speeds continued to increase until they reached an effective limit of 15,000rpm. Any faster, and the disk platters risk flying apart under the centrifugal force and from vibration. Lower speed drives could store more data and may rotate at 5,400rpm. That meant that the time taken to read and write all the data to and from a drive went up as drive capacity increased, and this became more and more a limiting factor.

The problem is that computer processor speeds, impelled by Moore's Law, rose and rose and rose, leaving disk I/O speed far behind and utterly unable to catch up. A move to 2.5-inch drives meant that more could be put in a disk drive enclosure to increase the overall I/O rate, but not dramatically so.

This inability to keep up with host computer speeds provided the backdrop for what we are seeing now - the rise of solid-state NAND flash storage, with no moving heads, in fact no read/write heads at all.

NAND flash

Invented by Dr Fujio Masuoka at Toshiba around 1980, NAND flash is organised as cells in blocks which must be written to simultaneously. So it is not, strictly speaking, a random access device.

Flash Memory cell schematic

Schematic diagram of flash memory cell

One type of flash, NOR (named after the Not Or logic concept) is used in mobile phones and other devices as a form of read-only memory. NAND (Not And logic) is used for data storage in cameras, USB sticks, tablet computers, notebooks and a vast range of other devices needing tiny, low-power and persistent data storage with faster access than disk.

A semiconductor technology, with no mechanical parts, it is considerably smaller than disk at equivalent storage amounts. A 1TB USB stick fits in the palm of your hand and is much lighter than the equivalent 4-CD case stack sized 3.5-in hard disk drive.

A quick look at optical storage

CDs and DVDs have also been used for archiving data storage but their widespread use has been held back by their inherent disadvantages. They enjoy relatively low capacity and slow write-speeds compared with hard disk drives and tape is cheaper than both disk and CD/DVD optical for storing bulk data. Disk is also faster for writing and reading backup data. All of which means that optical disks are only found today in niche markets.

Meanwhile, there have been persistent attempts to develop holographic storage technology as a way of providing archival duration and high-capacity optical technology. They have all failed. So much so that it's now feasible to consider archival storage using very cheap flash memory, TLC flash.

The Power of One eBook: Top reasons to choose HP BladeSystem

Next page: All types of flash

More from The Register

next story
Apple fanbois SCREAM as update BRICKS their Macbook Airs
Ragegasm spills over as firmware upgrade kills machines
Auntie remains MYSTIFIED by that weekend BBC iPlayer and website outage
Still doing 'forensics' on the caching layer – Beeb digi wonk
Attack of the clones: Oracle's latest Red Hat Linux lookalike arrives
Oracle's Linux boss says Larry's Linux isn't just for Oracle apps anymore
THUD! WD plonks down SIX TERABYTE 'consumer NAS' fatboy
Now that's a LOT of porn or pirated movies. Or, you know, other consumer stuff
EU's top data cops to meet Google, Microsoft et al over 'right to be forgotten'
Plan to hammer out 'coherent' guidelines. Good luck chaps!
US judge: YES, cops or feds so can slurp an ENTIRE Gmail account
Crooks don't have folders labelled 'drug records', opines NY beak
Manic malware Mayhem spreads through Linux, FreeBSD web servers
And how Google could cripple infection rate in a second
prev story


Designing a Defense for Mobile Applications
Learn about the various considerations for defending mobile applications - from the application architecture itself to the myriad testing technologies.
How modern custom applications can spur business growth
Learn how to create, deploy and manage custom applications without consuming or expanding the need for scarce, expensive IT resources.
Reducing security risks from open source software
Follow a few strategies and your organization can gain the full benefits of open source and the cloud without compromising the security of your applications.
Boost IT visibility and business value
How building a great service catalog relieves pressure points and demonstrates the value of IT service management.
Consolidation: the foundation for IT and business transformation
In this whitepaper learn how effective consolidation of IT and business resources can enable multiple, meaningful business benefits.