Starting with the June 2016 Tracker release, we will be using an updated All Flash Array (AFA) taxonomy that is more inclusive. In a nutshell any arrays that ship from the factory in all-flash configurations and do NOT optionally support hard disk drives (HDDs) will be considered AFAs. There will be three classes (or types) of AFAs, defined based on pedigree, to help customers understand key differences between them.
Starting with the June 2016 Tracker release, we will be using an updated All Flash Array (AFA) taxonomy that is more inclusive. In a nutshell, any arrays that ship from the factory in all-flash configurations and do NOT optionally support hard disk drives (HDDs) will be considered AFAs. There will be three classes (or types) of AFAs, defined based on pedigree, to help customers understand key differences between them, but our revenue and capacity forecasts will not include that level of granularity. There is an updated Worldwide Flash in the Datacenter Taxonomy, 2016 (IDC, May 2016) document that is already posted on idc.com that discusses the taxonomy update in detail.
Note that this updated taxonomy will cause us to re-state the AFA market size (in terms of revenue and capacity), making it larger than it was under the old taxonomy. We will be adjusting both the historical numbers as well as the forecasts. This change will also shake up the vendor shares a bit, since a number of vendors that weren't considered AFA players are now in the market, and one large player (NetApp) will have a much more significant share than it had in the past, particularly going forward. Platforms from vendors like EMC and HDS and eventually IBM that did not used to be classified as AFAs but now are will also affect their vendor shares.
In crafting the update, we had a lot of internal discussion about the category that was classified as "all flash configurations of hybrid flash arrays (HFAs)" under the old taxonomy. Many vendors have extensively flash optimized their storage operating environments and use the same software across both their HFA and AFA platforms. While they will ship all-flash configurations from the factory under a unique SKU, there are vendors that also optionally support HDDs in those platforms, claiming that if, after buying an AFA, the customer wants to move workloads to that system that do not require flash performance, this gives them the flexibility to do that more cost-effectively. A good example of this is HPE who ships all-flash configurations of their 3PAR StoreServ 8440 and 20840s but will allow customers to insert HDDs into these systems if they want to.
With the updated AFA taxonomy, IDC decided not to categorize systems that support HDDs as AFAs. Those continue to be categorized as HFAs, and we will retain the "all-flash configurations of HFAs" segment underneath the HFA moniker. Several factors influenced the decision to move to an expanded definition of AFAs:
- Regardless of their pedigree (purpose-built AFA or hybrid pedigree) the flash optimizations introduced into array operating environments have significantly narrowed the performance differential between all-flash configurations of both types
- Vendors are selling their all-flash configurations, regardless of pedigree, directly against other AFAs
- Defining an AFA as a system that only supports flash whereas an HFA is a system that can support both is intuitive and simplifies revenue and capacity tracking considerations
While vendors tout the flexibility of an AFA that can also support spinning disk, they can point to very few customers today that have taken advantage of this flexibility, and expect as flash costs continue to drop that there will be fewer of those going forward. We continue to hear from end users that they prefer flash over spinning disk for a variety of reasons that go well beyond just performance, and that if flash cost the same as spinning disk, it would be a no-brainer to buy all-flash.
Defining what "when flash costs the same as spinning disk" means is critical to this whole discussion. The cost of raw capacity is not the relevant metric, the entire system-level total cost of ownership (TCO) must be taken into account. Flash's ability to take advantage of a number of storage efficiency technologies, including in-line data reduction, thin provisioning, and space-efficient snapshots and clones, while still being able to consistently deliver latencies that are 10x better than HDDs means that the operative comparison is based around effective, not raw, capacity. HDDs, with variable latencies for enterprise workloads in the 5 millisecond to 20 millisecond range, do not support the use of these types of storage efficiency technologies on latency-sensitive primary storage applications. Flash, on the other hand, can cut its raw $/GB costs by ratios that vary from 2:1 (on relational database environments) to 10:1 or more in VDI, test/dev, and other clone-intensive environments.
Then, there are the secondary economic benefits of flash deployment at scale, which for primary storage environments include the need for far fewer devices (to meet IOPS requirements), significantly reduced energy and floor space consumption, the need for fewer servers to drive needed application performance, and lower software costs (since you'll need to license expensive applications like Oracle, SQL, etc. on far fewer servers). And then there are the reliability advantages – with flash devices typically being at least 3x more reliable than spinning disk, and most datacenters needing to plan for managing at least hundreds of terabytes of storage capacity in the near future, there are OPEX savings associated with not having to replace failing HDDs on a regular basis. The advantages that all of these factors confer only increase as datacenters consolidate more workloads onto their flash platforms – platforms which can support significantly higher application densities than HDD-based systems on a per terabyte basis.
Yes, there are application environments that are not very reducible, and there are also cold storage requirements where flash may not play well due to cost, but the fact is that with every drop in the effective $/GB cost of flash, it can be cost-justified for more workloads. All of these factors indicate that flash will come to dominate primary storage, and take over some percentage of secondary storage at a rapid rate over the next three to five years. Already, most data centers feel that all-flash strategies in the primary storage area are a matter of "when", not "if". HDDs will not go away completely – tape still hasn't gone away completely – but IDC does not see growth in the HDD markets. Conversely, we expect high growth in flash and other emerging memory technologies.
Based on this, we think that the need for all-flash configurations that also support HDDs will be a relatively short term phenomenon. Going forward, we think there will be AFAs, and there will be HFAs, with each platform highly tuned to support its designated media. HFA designs will evolve in the direction of "flash first" architectures that consistently deliver flash write performance at scale, and will have operating environments much more efficient at taking advantage of the mix of flash and spinning disk. That all said, we decided to keep things simple and straightforward with our updated taxonomy - AFAs support only flash (regardless of type) and HFAs support either (regardless of how they happen to be configured at any given point in time).