In a word, yes. Find out why I think so in this blog...
There are clearly workloads that really do need NVMe performance today, but customers whose general-purpose enterprise storage platforms are coming up for technology refresh are also asking this. We've just completed a major primary research project into NVMe usage and released three documents based around it (a write-up on a series on in-depth interviews with customers already using NVMe, a worldwide NVMe survey (again, focused around those already using it), and a five-year revenue forecast for the emerging NVMe-based All Flash Array (NAFA) market. IDC believes that, by 2021, NAFAs will be driving over 50% of all the external primary enterprise storage revenues. This penetration seems very rapid, given how we saw the switch from HDDs to NAND-flash based primary storage occur over the last seven years, but here's why IDC views it that way:
- In addition to the tier 0 workloads (e.g. transactional databases, etc.) that will move to the fastest storage almost as soon as it becomes available, there is a whole new class of quickly growing, real-time big data analytics workloads that will require NVMe performance, so there are even more workloads pushing the performance envelope than there were in the early stages of persistent flash storage
- NVMe significantly improves infrastructure density, meets the needs of customers looking for a higher IOPS to TB ratio, and enables denser mixed workload consolidation on smaller, less complex storage systems; for customers looking to use the performance and capacity of solid state media more efficiently, NVMe is the answer
- One of the attractions to all-flash storage was the TCO advantage when it was deployed at scale (relative to HDDs) – far fewer devices needed to meet performance requirements, reduced energy and floor space consumption, improved CPU utilization that led to needing fewer servers and potentially reducing software licensing costs – and many of these same advantages will accrue for companies moving from SCSI to NVMe, helping to streamline IT infrastructure and its associated costs
- NVMe sets customers up nicely to take advantage of future generations of even higher performance storage that will become available in the next one to two years like persistent memory (PMEM) and storage class memory (SCM) – these are technologies that just cannot be used with SCSI-based systems
- Five of the seven established enterprise storage vendors – Dell EMC, Hitachi Vantara, IBM, NetApp, and Pure Storage – are already shipping NAFAs, and three of those are also already shipping at least one NVMe over Fabric option (see the IDC NVMe over Fabric TechBrief (IDC , June 2018)) for more information about what NVMe over Fabric is and why it's important
- Some vendors (like Pure Storage) are providing NVMe-based versions of their enterprise storage arrays AT NO PRICE PREMIUM relative to the SCSI-based versions of the same platform, encouraging their installed bases to move more rapidly to this newer technology
- The switch from SCSI-based to NVMe-based solid state storage is conceptually easier to understand than the switch from HDDs to persistent flash and will have less of a disruptive impact; the HDD to flash switch completely changed how media was accessed, invalidating performance tuning skills and application enhancements that were very tied to how spinning disk operated, while the SCSI to NVMe switch basically just means everything runs faster and more efficiently (the change from block-addressable to byte-addressable storage which is effected by NVMe doesn't have these kinds of external impacts)
In networked enterprise storage, NVMe leads to 30-50% lower latencies, and 10x to 100x higher throughput and bandwidth, not to mention two orders of magnitude higher parallelism, than comparably configured SCSI-based systems, so for most workloads there will be a large performance improvement. Particularly in the big data analytics arena, there are workloads that could not even have been deployed on SCSI, and NVMe performance made them possible. There will be some attendant management simplifications as well – one that I've heard from those deploying NAFAs for relational database workloads is that managing database objects becomes much easier because you don’t need to create as many objects (and keep them appropriately synchronized) to get the performance you want.
As a result of this, many customers will at least be thinking about whether, during their next storage technology refresh, their next general-purpose platform needs to be built around NVMe or SCSI. Meanwhile, customers with silo'd tier 0 workloads are already deploying these systems and have been for at least the last two years. Shared storage arrays that meet IDC's definition of NAFAs will generate over $500M in revenue by the end of 2018. Vendors that offer both SCSI and NVMe-based enterprise class arrays will be best-positioned to serve the needs of these customers because they can offer both options and let customers decide which system best meets their requirements.
Although Pure Storage is not charging a premium for NVMe in its enterprise-class arrays, most vendors are. When considering new storage system purchases, customers will need to evaluate NVMe, determine whether it meets their needs over the anticipated life of the solution better than SCSI despite any price premiums (if they exist), and then move forward from there.