Published in Uncategorized

Has NPS Won the Ultimate Battle of the Metrics?

“How likely is it that you would recommend this company to a friend or colleague?”

In the last decade, this one question has provoked a long-standing debate between business executives, looking for the ultimate method to measure customer satisfaction, and researchers, who believe that that there is no substance to these methods.

In the Red Corner: The Researchers

When business professionals attempt to do researchy-type things, the research community gets all worked up about methods. And the debate over the validity of the Net Promoter Score (NPS®) methodology is no exception. Researchers are mostly concerned with true causality or, at a minimum, true correlation. They dedicate their entire careers to proving relationships between variables — and the key word here is “proving.” In order for something to be proven as fact, it usually needs to be performed as an experiment in a controlled environment. Many, if not most, life scientists don’t even consider social science to be real science due to the lack of controlled experimental studies.

So when some business professional with an MBA starts making claims about correlation, the research community naturally starts to question.

Was the NPS study done in a lab? No.

Was it done with proper statistical models, which attempt to control noisy data? No.

Wait — if NPS doesn’t pass even the most fundamental research study litmus tests, why should we even pay attention to it?

In the Blue Corner: Fred Reichheld

Fred Reichheld is the father of NPS. He has written reams touting its business benefits, including the bestselling book The Ultimate Question, concluding that NPS is the single most reliable indicator of a company’s ability to grow.

And we have to hand to him; he is not someone who gives up easily. Faced with a stream of questions about NPS methodology, Reichheld addresses the rumblings in his blog:

“All we did was quantify this common sense in a way that made sense to business leaders … The practical leaders have little interest in advance statistical methods. Frankly, we see little value in the continued debate about cause versus correlation, timeframes, or statistical methods.”

In other words, at the end of the day, what matters is results. But does Riechheld’s information demonstrate enough correlation to “quantify common sense?”

Back in the Red Corner…

Some would argue that it does not. In a Journal of Marketing article titled “A Longitudinal Examination of Net Promoter and Firm Revenue Growth,” former VP of Loyalty at Ipsos, Timothy Keiningham, supported by a team of experts, claims that Reichheld does not have enough evidence to quantify this common sense. In fact, they go so far as to say that the data backing up Reichheld’s claim is flawed.

The authors demonstrate that the more mature concept of the American Customer Satisfaction Index (ACSI) is as strong and, in some cases, a stronger predictor of growth than NPS. The diagram below compares NPS® to ACSI in the same time-frames for a sample of industries. Notice how the R-squared, or correlation, is stronger for computer and airline industries.

Even with the strength of the ACSI aside, the thing still causing the most concern about NPS® is the lack of statistical evidence, as Forrester’s Richard Evensen explains:

“The biggest concern with the formula is that there is not sufficient quantitative evidence to support a correlation between NPS and customer actions (renewals or even actual recommendations). Given its time in the field, SatMetrix should be able to show 100s if not 1000s of cases with high correlation coefficients (Pearson’s r=70%+). To date, they have only provided a totally unsupported position that ‘findings support the link between Net Promoter and financials.’ Please provide us the proofs as I have several clients who disagree with your findings!”

NPS: Winning where it counts

Given such strong evidence against the metric, why does NPS continue to be so popular among businesses? Simplicity and culture. NPS is a metric that makes sense. If more people are willing to recommend your business than not to recommend, then your business will grow. The statistics don’t necessarily back this up, as many professionals have taken pains to demonstrate, but it nevertheless makes logical sense. And that’s just the kind of sense that businesses like.

However, a stronger argument, and one that I personally subscribe to, is that NPS programs drive a culture of customer centricity. I’ve worked with large airlines, retailers, and software companies that live and die by NPS. The common thread is that these companies have a strong customer-centric culture. When an employee’s bonus depends on whether the customers they interact with will recommend them, they will naturally focus on pleasing that customer. I don’t believe that NPS is the only metric that can do this, or necessarily the best metric, but it is a metric that can influence culture.

The Bottom Line

Is there enough evidence that NPS really is the “only number you should care about?” Absolutely not. Then again, neither are any of the other measures. In some industries, NPS can be a highly effective predictor of growth. The bottom line is that companies should not to get too caught up in finding the ultimate winner in this satisfaction battle, and should instead find the measure or combination of measures that work best for you.

This article was originally published on linkedin.

NPS®, Net Promoter® Net Promoter® Score are registered trademarks of Satmetrix Systems, Inc., Bain & Company and Fred Reichheld