Scroll to top

MONITOR Metrics: Mixpanel Case Study

By Isabella Reynolds
How we monitored our metrics at Mixpanel

How We Monitored Our Metrics at Mixpanel and Made Adjustments to Triple Our NPS

Written by Kurby Chua, Principal Solutions Architect, Mixpanel

Most of you would agree that launching a product is only the first step in the journey. The most successful product companies know that they also need to continuously monitor the right product metrics after launch. They want to know if they’ve gone off course and take corrective action to get back on track.

Let me illustrate this using Mixpanel’s own experience with our product.

We were founded in 2009, so by 2019 the product was well established. Our product team was churning out new features steadily to help us close gaps so we could win more deals. So everything should have been smooth sailing, right?

Unfortunately, it wasn’t. As a B2B SaaS company, Net Promoter Score (NPS) is a metric we monitor closely. Despite our best efforts, in 2019 we found that our NPS was just 15 (the average for B2B SaaS was a score of 30).

It was a sign that something was wrong, and we had to find out why and fix it quickly. When we dug into the detailed NPS comments, we found that 50 percent of it was related to UX.

The NPS comments that break a UX designer’s heart.

Getting the Metrics Right and Digging Deeper into User Data

The question was: What do we do to change this? The first thing we realised was that our core metric wasn’t the right one. We wanted to address churn and had NPS as a core metric, but NPS is an output metric. It measures the output but does not inform us about what user behaviours we need to drive in order to help users experience product value, which then leads to user satisfaction and retention.

 

We used Mixpanel to analyze our user behaviour data and noticed that they loved being able to apply a filter or break down their queries (we call this segmentation), which leads to getting an answer to specific questions in Mixpanel. With that in mind, we defined a new focus metric called Learning User Verified — users who did segmentation at least three times in the last week. Turns out that this group of users retains at three times the rate of baseline user retention.

The product team then continued to dig into the data to identify user paths that support this new metric and a new 6-step onboarding success funnel that gave us clarity on what to fix within the product. This included improvements to our new user onboarding and product UX (a total of 92 product upgrades in 2020 alone).

Our 6-step onboarding success funnel focuses on driving new users to experience value in the product, which supports our focus metric.
Our NPS scores progressed a long way in 18 months

Getting Back on the Right Track

 

That, of course, can’t be the end of the story. We continued to monitor our metrics to make sure that we were on track. By January 2021:

  • Our NPS score had improved to 50 (three times higher after 18 months)
  • The activation funnel saw a triple-digit increase
  • Positive customer sentiment increased noticeably on social media and review platforms like G2

That was our story on how monitoring the right metrics and course-correcting had impacted our business positively. The journey continues, of course, and we’re always tracking our metrics and using data to help us monitor our progress.

 

Understanding how to set the right metrics and leverage user behaviour data and tools to measure these metrics is key. We’ll be running workshops (2 sessions on 18 May) where we discuss the importance of an analytics strategy and the key metrics required to track your product journey. So don’t miss it when you attend Leading the Product’s LTP DIGITAL 2022 | APAC!

This blog was contributed by LTP DIGITAL 2022 | APAC Gold Sponsor, Mixpanel. Check out how Mixpanel helps companies measure what matters, make decisions fast, and build better products through data.

And don’t forget to join the Mixpanel team at LTP DIGITAL 2022 | APAC on May 18, 2022. Get your team tickets today to access our corporate discounts.