Compute-storage mashup: Great for Big Data, but what of my mobile?
Latency HELL as 'tops and 'slabs probe cloud
Ensure Ease of Recovery with Asigra’s Agentless Software
Blocks and Files Should compute and storage dance closer together or farther apart?
In the enterprise there is a strengthening under-current of thought that says data access latency is bad. You shouldn't have to wait for data. In the networked storage array field that is causing the import of flash storage into arrays to kick disk seeks into the long grass, and the provision of higher-speed networking, such as InfiniBand and 1Gbit/s Fibre Channel. But that's just tinkering with network latency.
We're seeing flattening SANs and virtualised server area networkers using server IO virtualisation, like the Oracle Xsigo technology, to provide SAN data access facilities at direct-attached storage (DAS) speeds. The purest exhibition of this is to re-invent DAS as flash storage memory, treated as an adjunct to a server's RAM accessed through the memory IO subsystem and not the disk IO subsystem.
These are examples of bringing storage to compute. A reverse trend is also visible though not as readily apparent, and that is bringing compute to storage by running application software inside storage arrays, as EMC keeps on saying its going to do with VMAX, VNX and Isilon storage arrays. DataDirect Networks is actually doing it by running file systems such as GPFS and Lustre inside its storage arrays via the ExaScaler and Gridscaler products. These are system-level applications though and not end-user apps. Running actual end-user apps in storage arrays is still a distant prospect.
In client, end-point computing-land the reverse is happening; to a large extent compute and storage are getting further apart. The smart end-points, like mobile phones, tablets, Google-ised Chromebooks and virtualised desktops have very little local storage and the cloud or enterprise data centre is where the data is stored. Like Big Data, client data is centripetal. Why is network latency acceptable for end-points but not in servers?
Frankly, sitting at the end of what my ISP or mobile phone service provider laughably calls broadband, having the remote servers access their data at microsecond speeds instead of milliseconds while I'm waiting multiple seconds leaves me cold. I could care less. I want my data - right now. If low latency is good for data centre servers then it's good for client devices too. But no shared network, no affordable shared network, is ever going to be as fast as local storage, or as dependable, as those who suffer ISP link outages know only too well.
Put another way: Big Data is a data centre thing. Here out in the far-flung reaches, the outer galactic spiral arms of end-point computing, Big Data doesn't exist. It's a case of little or no data. Curious isn't it? Latency sauce for the data centre server goose is not latency sauce for the end-point gander. ®
COMMENTS
"Why is network latency acceptable for end-points but not in servers?"
I think it comes down to basic expectations. Network latency is not acceptable for end points in many cases, which is why we have many CDNs out there to specifically reduce that. Also fancy DNS systems that route traffic based on region or IP that use things like BGP anycast. One place I was at changed DNS solutions because it shaved something like 15ms off their DNS queries.
Other cases it's not a big deal, people have always been used to the latency and it's difficult to break the speed of light, so there really isn't a lot you can do, other than aforementioned options. Building multiple data centers and geo-diversifying your data is a really complicated topic to get right, which is why of course most folks don't do it.
Even going back a decade+ in Google's early days they were addressing latency issues by storing everything in main memory.
There's solutions to address latency on the server end, just a matter of how much latency is acceptable. The extreme cases of "must have everything local on SSD" i think are very rare. My own workload on an e-commerce site that pulls in a bunch of $$ runs mostly out of memory as well (storage workload is 90%+ write), it wasn't intended to be that way(was a surprise to all of us when we got the real stats), but with heavy caching it just worked out that way.
A better, longer term solution is of course to have smarter applications. Things that use I/O better, rather than brute force it by adding more software, as some of the early DBAs told me years ago - you can make a query 10% faster by adding hardware, or 100-1000% faster by fixing the query/data.
IT infrastructure monitoring strategies
Agentless Backup is Not a Myth
Top 10 SIEM implementer’s checklist
Steps to Take Before Choosing a Business Continuity Partner
Enabling efficient data center monitoring
