This is a series of blogs that explore the use of NPS and address some of the criticisms that have been leveled against it over the years. NPS can be an effective tool in an enterprise storage vendor's arsenal, but how effectively a vendor designs and implements their "NPS program" will determine their success with it
More reaction to criticisms leveled against the use of NPS to evaluate customer experience…
Experience and NPS score rarely match. For an NPS score (or any tool, for that matter) to have predictive value it should consistently forecast some outcome. With NPS, the assumption is that a high number indicates lots of happy customers, and prospective customers looking at that might expect that if they purchase the product or service, they will also become happy customers. I don't know that I would agree with Jared's claim that experience and NPS rarely match, but it is clear that sometimes they do not. If the one question concerns a willingness to recommend, then it is incumbent upon vendors to understand what goes into that for their customers. This may not be as simple as it sounds, as an example from the airline industry shows.
Most airlines have a pretty low customer satisfaction number for most of their customers, yet customers still consistently fly the same airlines they love to hate. In this case, low customer satisfaction does not appear to be correlated with low sales. In my opinion, it's because what most customers are buying is transportation between two locations that is most convenient for their schedule. Who delivers that is often secondary, since airline service is widely viewed as consistently mediocre across most providers. Research cited by Jared in his article as debunking NPS (cited in the bibliography rather than directly) indicate that Frequent Flyer programs for the most part aren't generating customer "stickiness" because they're all pretty much the same, so there's some other reason people continue to frequent certain airlines. In their book on Loyalty Myths, Timothy L. Keiningham et al voice the same suspicion I have about airlines: the primary purchase criteria is based on transportation convenience, so customers complain but continue to fly the same airlines because of convenience. If another airline offers the same or better transportation convenience, many of these customers have no loyalty to their "old" airline. Does your product or service fall into this category? If so, you want to be sure to understand your business and why customers really buy your product or service. There are clearly cases where customers don't recommend you, but may continue to purchase from you.
While the simplicity of NPS is one of its most attractive traits, there is potentially some complexity underneath that which is specific to a certain product, service or company that must be understood, and that understanding must inform programs, processes and platforms the provider puts in place to drive high customer satisfaction. Vendors crafting programs to drive high NPS (and presumably strong future sales) need to be aware of the corner cases Jared Spool mentions and take care that they don't infect the program. NPS puts undue importance on customers that show a VERY high willingness to recommend a product or service. If a vendor doesn't have a lot of these, they won't have a very high number. One of Jared's examples includes a business that has a lot of indifferent or mildly happy customers (based on their individual NPS scores) but this shows up as an aggregate NPS for the company of 0. While I agree that this is one of those examples where NPS seems not to work, I also note that to get a positive score a business needs a LOT of very happy customers willing to recommend their product or service. Businesses should strive for that, and if NPS may sometimes produce a low number if they don't have that quorum of delighted customers, to me it may not be such a bad thing.
One final point on this topic. Just because a business calculates an NPS number and uses it to make business decisions, that does not mean that it should publicly release the number. By and large, the public does not understand the number. An uninformed observer might assume that the scores between -100 and +100 can be compared based on a geometric scale, but it's my view that (because of the very heavy dependence on having lots of satisfied customers to generate positive scores) it might be appropriate to think of the scale as a more logarithmic one. Not exactly, but the difference between a score of 15 and 63 is much bigger than that same difference on a classroom test grading scale. So there is a difference for a business of using such a system internally and publicizing the number. If a vendor has not made the appropriate investments in programs, processes and culture, they might want to look more deeply into a number they generate.
NPS is a tool which has value when used intelligently. Like many tools, NPS can be used effectively when it is used appropriately. Using a tool like NPS does not substitute for business managers truly understanding what their product or service value proposition is, which customers they should be targeting, and what is important to those customers. What is clear is that there are vendors who have taken this measure (NPS) and built programs around it that have transcended customer expectations in enterprise storage in a way that increasingly matters to buyers. Those vendors have changed the industry for the better, in my opinion, and other enterprise storage providers are starting to introduce some of the same types of programs, processes and (even) culture. For those vendors that do this well, that can only benefit customers.
Might Nimble, Nutanix and Pure have been able to generate and grow their businesses without NPS? Most probably, but NPS provided a focus for them to build the right processes, programs and culture to delight their customers. The right happy customers are a good thing – they often (but not always) buy additional products faster than unhappy customers, they have a lower cost of sales, and when they recommend a product or service to their friends it has the ring of authenticity. They become effective "influencers" that can have a positive impact on future sales. None of this is guaranteed, and it works a little differently for different businesses, but in general knowing that a customer would unabashedly recommend your product or service is a good thing for a business.
The Bottom Line
Should every enterprise storage vendor be moving to implement NPS? As much as I like and appreciate what NPS can offer, I can't blindly make that recommendation. I can, however, suggest the following:
Vendors should be measuring the customer experience they generate and understand the reasons behind it and how much it costs to deliver that level of satisfaction. If they can work to improve that level over time without running afoul of profitability goals and stay focused on getting more of the "right" (i.e. profitable) type of customers, they should do so.
If they can use a standardized method to do this, that provides a better basis of comparison for prospective customers to use when evaluating product and service options in your market. A vendor who generates high levels of satisfaction among the right customers without gaming the system should welcome these comparisons, and customers should appreciate and use them. Vendors that are generating valid high numbers (like Nimble, Nutanix and Pure have) should be proud to publicize those numbers and help educate prospective customers about what they truly mean.
If you do choose to implement NPS, do so while realizing the limitations of such a tool. Apply it intelligently and use it to help evolve your business in a desired direction. Don't apply it lightly or just blindly accept all scores.
If the use of NPS helps to focus your company on doing a better job of serving the right customers, then it has been used effectively. But vendors and customers should both make sure they look behind the scores. A vendor who is using it to drive value for their customers should be happy to explain to sales prospects (and other interested parties like potential investors) what they've done to achieve a high score and how those programs, processes and workflows they've put in place as part of it will drive value for future customers.