Mastering the 'Product Metrics That Matter'

Imagine that you joined a new company. You are supposed to look at the Key Results that were set for your team and provide inputs. If done well, you have an opportunity to impress the leadership. Otherwise, it becomes a missed opportunity and can be a signal that you don’t understand what you are supposed to lead.

This is one of the key problems that product leads and executives face when they join a company. At junior levels, the mandate for metrics/ key results usually comes from the leadership. But when you move into the senior roles, you are supposed to provide inputs in key focus areas.

Now if you are good at your job, you mostly understand the laundry list of metrics. The deeper problem that you face is how to pick and choose the metrics that matter for the particular product in question. These are the Product Metrics That Matter.

I would use the abbreviation PMTM for such metrics. Note that it’s not a single metric, but a collection of metrics that provide the holistic picture of the product.

Let’s be honest — PMTMs are difficult to master, and here is why!

A great metric for one product can be a decent or even bad metric for another product. Take time spent as an example. While time spend can be a great metric for an attention product like Facebook, it could be a terrible metric for a productivity product like Superhuman (an email productivity tool). And that’s because the goal of Facebook is to provide the user entertainment value and meaningful interactions, and a direct measure of that entertainment value is total time spent. The goal of Superhuman is to make the end user more productive by helping them manage their inbox. So a person spending time beyond a threshold in Superhuman can be a bad thing.

A great metric in a particular year for a product can be a terrible metric for another year. Growth rates were the norm in ZIRP era, i.e. 2022. They are replaced by profitability and cost savings in 2024 for many products.

We can add more reasons, but it’s clear why the PMTM are difficult to master. And that lends to the next question — How does one go out about mastering the PMTM?

We have to start by thinking about the variables that determine the usefulness and relevance of a metric.

So what variables determine if a metric is useful and relevant?

There are four variables in particular:

1. product

2. company

3. industry

4. end users

Let’s talk about these variables one-by-one.

Product: When it comes to product, we need to look at the

• stage of the product

• it’s goal, and

• it’s business model.

An early stage product has most useful metrics around PMF, whereas a late stage product would be more growth or profitability oriented.

The specific goal of the product can also determine what metrics are most meaningful. We saw this in the example of Facebook vs Superhuman.

And finally, the business model, aka the way a product generates revenue (e.g., subscription-based, freemium, ad-supported) can also affect which metrics are prioritised. For example, a subscription model will focus more on retention rates, whereas an ad-model will prioritise user engagement.

Company: We need to next understand the strategic priorities for the company. Usually, these strategic priorities come from a large opportunities or threats. For example, OpenAI and advent of LLMs changed the strategic priorities of all large tech companies, especially Google. Upstarts like Perplexity and OpenAI have been threat to the Google’s core search business, and nothing changes the strategic priorities faster than that.

It’s worth mentioning that understanding the complete picture of a strategic priority is difficult. You require a deeper understanding of the industry for it. For example, if you are running a marketplace product like Airbnb, understanding how marketplaces evolve in general is important. Having an understanding of how the marketplaces in your domain or geography have evolved is even better. For Airbnb, that would be understanding the travel industry.

External factors in the industry such as market trends, economic conditions, and regulatory changes can impact which metrics are important at any given time. For example, during an economic downturn, cost-related metrics will gain more focus.

Industry: The most prevalent reasons to look at the industry are two-fold:

1. It helps you identify the industry-relevant metrics. That can cover your blindspots, but if not, it can provide confidence around the metrics you have chosen.

2. It helps you get the benchmarks. The benchmarks help you dig deeper into what sort of goal setting can be done for a specific area.

End Users: No discussion around metrics is complete without taking the ‘value created for user’ into the account. The PMTM should definitely have metrics that reflect value created for the users. CSAT, NPS, etc. are some example of user-focussed metrics. We should also look at defining different user focussed metrics for different personas since the value creation has a specific meaning for a specific persona.

Once we have built an understanding of product, company, industry, and end users, we are ready to look at key metrics and provide inputs. Note that while it’s harder to build an understanding of the company and industry as compared to that of the product and end users. That’s why the recommended approach would be to

1. start with the product and end users, and define key metrics

2. use company strategic priorities and industry to narrow down to a handful of PMTM (product metrics that matter)

Applying It All In a Particular Product

Imagine that you started managing a fitness subscription product where you provide workout sessions at offline fitness centres. It is a series B funded startup with 500 active members. The annual membership costs $1,000.

The user has to download the app and register to get a free trial of first 3 sessions. After the first 3 sessions, they have to start their monthly paid subscription.

You can start with the product and end users to arrive at metrics such as

Product Funnel

1. Acquisition

   1. New installs / month

   2. Cost per installs

2. Activation

   1. Install-to- at least 1 free session conversion

   2. Free trials-to-paid conversion

3. Retention

   1. Monthly subscription churn

4. Revenue

   1. Revenue per month

5. Referral

   1. % of installs from referrals

   2. % of paid users from referrals

Demand Side (User Experience) Metrics

1. % of times users are finding the workout sessions they are looking for

2. % of times they are booking the slot and making it to the centre

3. Average ratings post session

Supply Side Metrics

1. Types of sessions and their utilisation capacity %

2. Trainer utilisation capacity (% of spots filled for trainer sessions)

3. Trainer ratings

4. # of workout centres

5. Workout centre ratings

6. Workout centres and their utilisation capacity %

Now let’s add the nuance from the company and industry perspective.

1. Company’s strategic focus — is it on getting to profitability, or reducing churn, or both at this point of time?

2. Industry — where do we stand in the industry? what metrics do other industry players track? what’s the benchmark on key metrics?

Suppose you start by discussing company’s focus. You get to know that the churn from membership is high, and that’s the biggest problem for product’s growth. You also realise that subscriber churn is a function of user experience.

Next you benchmark this churn with industry and realise that it’s 2x the churn of the leading player.

Looking at both company’s focus and industry benchmarks, it looks like churn is the right area to focus!

The subscriber churn is affected by

1. User experience metrics

   1. % of times users are finding the workout sessions they are looking for

      1. Types of sessions and their booked utilisation capacity %

      2. Offline centres and their booked utilisation capacity %

      3. …

   2. % of times they are booking the slot and making it to the centre

      1. Types of sessions and their actual utilisation capacity %

      2. Offline centres and their actual utilisation capacity %

      3. …

   3. Average ratings post session

      1. Trainer ratings

      2. Workout centre ratings

      3. ..

2. Quality of subscribers you are attracting

   1. Activation %

   2. Conversion %

Rather than looking at AARRR funnel, you now have set of user experience metrics that reinforce each other and affect the one metric that matters most — churn. These are the PMTMs.

Finally, if you are in a real job setting, you also have the data to look at and see if churn has higher correlations with some of the input metrics we mentioned. That can narrow down your areas of focus further.

You can still track other important metrics like acquisition, activation, and referrals, but when it comes to goal setting, it needs to focus on the collection of metrics for churn.

Summary

The questions around PMTM can present themselves in various ways:

1. In a real job, they can appear when evaluating OKRs, or launching a new product

2. In the interview setup, they can present themselves as a north star metric question, or defining success metrics for a product or feature

All of them can be derived from the simple framework that has the acronym PICE.

Product — stage, goal, model

Industry — industry-relevant metrics, benchmarks

Company — strategic priorities and how they align with PMTM

End Users — value created for the end user

---

This would be all for this post. If you are interested in reading about metrics more, here are some previous posts you will find useful:

1. Product Metrics: To measure, or not to measure?

2. Decoding Growth: User Segmentation and Metrics

3. Product Metrics: North Star Metrics, Counter-metrics and Sensitivity

4. Fighting the Evils of Data

5. Growth Analytics 101

---

We have an upcoming Mastering the Product Interviews cohort where there are ~10% spots left. Check and enrol here — https://www.pmcurve.com/product-management-interviews

See you again next week,

Deepak

---

Thanks to Piyush Mayank, Utsav Raj, Pushkar Ravi, and Sorabh Vij for reading drafts of this.