Flash and the five-minute rule
NAND then there was disruption
SaaS data loss: The problem you didn’t know you had
Comment "Flash is a better disk ... and disk is a better tape." So said distinguished Microsoft engineer Jim Gray in 2006.
A few years before then he had formulated his famous five-minute rule of thumb, which said that it is better to cache disk-held data in DRAM if it was going to be re-used in five minutes or less. The idea was that, given the relative costs of DRAM, disk capacity, I/Os and data page sizes, it would cost less to cache such data in DRAM than keep it on disk.
This proved to be an enduring rule and the coming of affordable flash now means that it can be used to say when data should be stored in NAND or disk or DRAM. Given the steep price declines of flash, the predictions from it are that flash will become the place to store active data, not disk.
This picture of flash's prospects was presented by David Dale of NetApp at an SNIA data centre event in London on Tuesday. His session was called The Benefits of Solid State in Enterprise Storage Systems, and he set aside NetApp partiality to present an industry picture. That's the value of SNIA sessions like this - vendor reps don't sell or market their products, they educate.
Dale's pitch was fascinating and gave a glimpse of the turmoil that lies in wait for storage and server vendors that don't embrace flash, turmoil that can be predicted by Gray's five-minute rule.
Five-minute rule formula
The rule is worked out like this: you calculate the break-even relative interval (RI), and if it's less than five minutes you cache the data. The formula is:
RI = (Data pages per MB / IOPS per disk) x (price of disk / price per MB of RAM)
According to this article 1MB of RAM cost $5,000 in 1987, a 15 IOPS disk cost $15,000 and the data page size was 1KB. The RI formula, using these figures, works out at 205 secs. Data used more often than that should be kept in RAM with the rest on disk.
By 1997 prices and IOPS had changed. The MB of RAM cost $15, a 64 IOPS disk cost $2,000 and the page size was 8KB. The RI works out to pretty much the same: 267secs.
Fast forward to 2008, and the MB of RAM now costs 10 cents, the disk plus controller costs $650 and spurts out 183 IOPS - this is a 15K, 2.5-inch enterprise drive - and the RI is 2,273 secs, or 38 minutes. Oops. You can't feasibly hold all the data that will be re-used in 38 minutes in RAM or you can increase the page size. A 64KB one gives an RI of 568 seconds (nine minutes) which is better but still too high.
Next page: Enter Flash
COMMENTS
Flash
Yeah, but Apple will never allow it on the iPad...
...oh, THAT Flash
sorry!
the last minute rule
With all the due respect to (presumably) late Mr. Gray, that rule only worked in a rather narrow set of configurations. Realistically, servers, especially high end ones, are stuffed with as much memory as fits in them.
Flash life varies greatly
Some is good for well over 10^6 cycles.
Some is only good for 10^4 or less cycles.
There are a lot of other factors. High and low temperatures cause problems too.
IT infrastructure monitoring strategies
Agentless Backup is Not a Myth
Top 10 SIEM implementer’s checklist
Steps to Take Before Choosing a Business Continuity Partner
Enabling efficient data center monitoring
