We Kidd you not: Ex-NetApp CTO Jay speaks his brains on storage tech
Shared storage, converged systems, flashy all-flash: it's all on the table
NetApp’s ex-CTO Jay Kidd has joined object storage supplier SwiftStack’s board and we had the opportunity to ask him some questions about his views on various technologies. We wondered how much his views now might differ from NetApp norms.
So we went ahead, and here are the technology topics and Jay Kidd's views:
Mainstream SAN and filer arrays
- The trend toward consolidation of shared storage for both SAN and NAS over the past 10+ years has shifted. No longer afraid of proliferation of storage-per-application silos, enterprises are comfortable with deploying different products for a few different classes of service.
All-flash systems will be the norm for anything performance-critical. Hybrid flash/disk arrays will be the mainstream for a while for capacity centric Tier 1 apps and many Tier 2 apps. A trend is starting in enterprises to consolidate their tier 3/4 app storage, backup storage, test/dev and use software-defined storage architectures running on commodity hardware to lower [the] costs of their least critical storage.
HCIAs (hyper-converged infrastructure appliances) and converged systems
- IT departments have been reducing skills in the area of systems integration and management, so products which take less time to deploy and manage are seeing success.
HCIAs like Nutanix are seeing success, and converged systems like Flexpod or Exadata also see success. HCIAs aren’t yet perceived as capable of scaling up to support traditional enterprise workloads yet, which leaves space for the Flexpod and VCE products. It will take a few years, but HCIA systems will harden and will ultimately gain credibility as platforms for Tier 1 apps.
Newer all-flash and hybrid arrays
- All-flash arrays and hybrid arrays loaded with all-flash will become the primary platform for performance centric apps – Tier 0. Both architectures can deliver comparable performance, efficiency, and price points since they are all working with the same raw components and software concepts. Density and power differences will matter to some customers and not to others.
HDD makers and their move into arrays
- These companies will be starting from scratch to build a channel to reach enterprise and mid-size customers. They may have success selling to the hyper-scale companies with whom they already have component purchasing relationships, but mainstream customers require a more instructive and collaborative sell, more help architecting and integrating, and first class enterprise post-sale support. It is difficult to build those systems capabilities within a fundamentally lower margin commodity business model and company culture.
3D XPoint memory and similar technologies
- These technologies will drive the next big architecture transformation in servers and storage. A tier of memory that is 100-1000x faster than NAND flash and cheaper than DRAM will become a natural solution for scaling out and up the growing class of in-memory applications. In-memory databases and analytics tools will be able to work on bigger problems. Storage systems can hold more metadata in memory, reducing latency to other storage. Containers can have access to persistent state without the latency of disk or networked disk access. As costs fall, it will become a standard part of every enterprise server.
NVMe fabrics and similar server-storage array linkage
- These projects are the precursors to fabrics that interconnect 3D XPoint memory. As such, they will be short-lived, but they will enable exploration and validation of architectures that will then see widespread adoption with the new memories. I am sure that the smart players in these fabrics are considering their role once XPoint is available.
- At certain scale, cloud storage is more expensive than owning your own, provided you have the datacenter space and your capacity requirement is fairly steady. So there is a growing interest in on-premises storage that is very low cost and operates much the same way that AWS S3 does. Object storage companies like SwiftStack are in this space with offerings that are pretty simple to deploy and use. There is also a trend toward object stores supporting file protocols and file storage supporting object protocols, so a tier of storage that supports unstructured data and can span premise and cloud in a consistent way is the desired end state.
Software-only (no hardware lock-in) storage
- Many of the new storage companies are building products that are solely software and run on commodity hardware. They will still deliver them as appliances, but customers will have the confidence they can change server vendors if they need to. This trend is accelerating fast in storage and will slowly gain traction in networking products as well.
- The question on whether public cloud is here to stay is answered by AWS financial results. Public cloud will be part of every single company’s IT strategy, from 100 per cent public or SaaS for newer companies down to selective use cases for the most conservative enterprises. AWS is clearly the leader here, Azure and Google are challengers.
- Data protection has been redefined by the VM and HCI trends. DR: Any application that must be brought up in a disaster requires replication accessible by copies of the key production applications. Freed of the need to rapidly backup an entire dataset for DR, Backup tools now focus on recovery of single files or portions of data in a much easier and faster way. Backup to the cloud is gaining traction and as hybrid clouds become easier to manage, and applications can be written once and deployed on-premises or in cloud, replication to the cloud will also pick up steam. Data security is a continuing concern, and encryption is a basic capability of every storage offering, but with most attacks tending to come via compromised credentials, the focus is more on detection of abnormal data access.
There are interesting takeaways here
- A trend is starting in enterprises to consolidate their tier 3/4 app storage, backup storage, test/dev and use software-defined storage architectures running on commodity hardware to lower [the] costs of their least critical storage.
- HCIA systems will harden and will ultimately gain credibility as platforms for Tier 1 apps.
- HDD suppliers will find it difficult to sell arrays because of their fundamentally lower margin commodity business model and company culture.
- XPoint - it will become a natural way to scale-out and scale-up the growing class of in-memory applications.
- XPoint - Containers can have access to persistent state without the latency of disk or networked disk access.
- XPoint - As costs fall, it will become a standard part of every enterprise server.
- NVMe fabrics will be short-lived precursors to fabrics that interconnect 3D XPoint memory [systems].
- Object storage - a tier of storage that supports unstructured data and can span on-premises and cloud in a consistent way is the desired end state.
- The software-only trend is accelerating fast in storage and will slowly gain traction in networking products as well.
Kidd's view on the short-life ahead of NVMe is unexpected. But the top take-away here for us is that Intel and Micron's XPoint is set to play a huge role in servers with persistent memory becoming a major tech topic both in servers and storage. ®