The #1 question I got after publishing my Hierarchy of Marketplaces series is how to know if you are on the right track in Level 1 as you wait for your cohorts to mature, and whether you can use Net Promoter Score ("NPS") to measure happiness. Honestly, I’m skeptical of NPS’s utility. Instead I recommend measuring your funnel to what I’ll call “Happy GMV”. Below I describe what that is, and then elaborate on why I’m over NPS.
Happy GMV
As you wait for your cohorts to mature, start by asking yourself “what is my best guess at the buyer and seller experience that will lead to retention?” and then measure the percentage of your potential buyers and sellers that get that experience. The GMV that qualifies is your Happy GMV.
Tracking the percentage of GMV that is Happy GMV is a useful day-to-day proxy to tell you whether you’re on the right path to Minimum Viable Happiness. The higher your % of GMV that is Happy GMV, the better your “liquidity quality”, as my partner Bill Gurley calls it.
In the beginning, the experience that qualifies as Happy GMV will be a hunch, driven by first principles and user research. For example, you could imagine for a ride sharing company starting with the hunch that on the rider side it’s an experience where a rider gets picked up within a certain number of minutes and rates the driver 4 or 5 stars. So you would measure what percentage of riders open the app and get a car within a set number of minutes, and then the percentage of those that rate the driver 4 or 5 stars. That would be your initial hunch of what qualifies as Happy GMV.
Tracking this funnel would help you see the day-to-day operational levers you need to focus on in order to increase your conversion of potential GMV to Happy GMV. Over time, as your understanding of your customers improves and you start to have statistical significance in your cohorts, you’ll be able to tune your “Happy GMV” thresholds to more accurately reflect the experience that leads to retention and keep you focused on the most important levers. I’ve heard of companies having 5+ attributes that they count that they know creates a “happy” transaction.
If you’re interested in a few real-world examples, Lenny Ratchitsky, as usual, has a great post on these happy moments.
Net Promoter Score
Net Promoter Score (“NPS”), which asks how likely a customer is to refer a company or product to a friend or colleague and then subtracts the “detractors” from the “promoters”, was coined back in 2003. It was pointing at the right idea —if a customer enthusiastically tells a friend about a product, you clearly have product/market fit with that customer, and benefit from invaluable word of mouth (and if they won’t, you’re unlikely to get them back). But how NPS is calculated makes it an easily gamed, statistically noisy measure of intent, not of behavior.
Just do a Google search on Net Promoter Score, and you’ll come across plenty of thoughtful analyses on why it’s imperfect at best and certainly not worthy of being the north star metric many believe it to be. That’s not to say customer surveys have no value, or even that the NPS survey itself doesn’t have value. Surveys are critical for understanding your customers. But be wary of the utility of Net Promoter as a score, particularly at the early stages of a company.
Personally, I’m intrigued by Sean Ellis’s Product/Market Fit survey, and am working to get more data. It seems to get more to the core to me, is better designed from a survey design perspective, and still is useful with small sample sizes.
As I talk about in the first level of the Hierarchy of Marketplaces, what’s critical for any company to keep in their purview is not just whether someone likes your product, but how much happier a person is when they use your product versus any substitute.
Ellis’s question gets to the relative happiness your product creates versus any substitute. If you make your customers happy enough, you become indispensable. That’s the key to achieving net revenue retention (and organic growth). A noisy algorithm based on how likely someone says they are to refer a friend or colleague is not.
If you’re curious, Ellis writes more about it here, and Rahul Vohra wrote a great post on how he used it to find product/market fit for Superhuman here. And if you’ve used it, I’d love to hear what you think.