3,000+ people have subscribed to the growth catalyst newsletter so far. To receive the newsletter weekly in your email, consider subscribing 👇
If you aren’t familiar with the newsletter and online sessions, you can read about it here
Off to the post,
Albert Einstein reportedly said, "Compound interest is the eighth wonder of the world. He who understands it earns it; he who doesn't, pays it"
And a century later, Warren Buffett became one of the greatest investors of all-time understanding the power of compound interest. Buffett understands compounding in a way most people don't. But why?
Because compounding is not intuitive to our brains, so we ignore its potential and focus on growing money in another way. Look at this chart. >99% of Warren Buffet's wealth came after his 50th birthday, and this guy is investing since he was 11. Notice $67 million at the age of 47?
Does the graph look intuitive to you?
While we are talking about wonders, is there one when it comes to product growth? Yes, there is, and you may have guessed already — retention. Retention is one thing that, when done well, will forgive all of our sins in product development.
Like compound interest, retention isn't intuitive to founders, PMs, marketers. We go after performance marketing to get more signups to grow products because our brains can easily understand n million signups.
We also try to make buttons more appealing or redesign the entire app/website hoping that things will improve. But to our surprise, it doesn't help most of the time.
Retention is the open secret of product development. Those who understand it build products the right way, those who don't keep chasing vanity metrics. And it's not our fault entirely; we keep hearing about MAU, registrations, signups, etc everywhere. Retention isn't usually discussed in their quarterly reports.
And that's the reason for writing the three posts around retention. In this post, we will understand retention and its nuances. In post #2, we will cover the ways to diagnose the problems with retention. Then we will talk about ways to improve retention in the last one (#3).
Let's start with defining retention.
Defining Retention
The fundamental definition of retention is retaining the new and existing users of your product.
Different teams measure retention in different ways. The key reason for the difference in measurement is that in order to make retention metrics useful for your product, you have to define it according to your product usage and business model.
The top 3 ways to define retention are
N-day retention: % of new users who come back on a specific day (N). For example, 7-Day retention would be % of new users coming back on the 7th day.
Rolling retention: % of new users who return on/after a specific day.
Range retention: It is a variation of N-Day retention where we can pick a range to calculate retention, says Day1-3 retention.
Let’s try to check our understanding through the table below. What does Day-7 retention look like? What about D3 rolling retention? Day 1-3 range retention?
Hint : We have limited data set so can calculate the numbers for a 7 day range only. D7 = 1%, D3 rolling retention = (D3+D4+D5+D6+D7) = 13%+6%+3%+2%+1% = 25%, Day 1-3 range retention = 88%
N-day retention is the most widely used method and hence called classic retention. Classic retention can also be measured at a week, month, or year level.
N-week classic retention: W1, W2, W3, and so on. On a weekly level, we look at % of new users in Week 0 (1-7 days) who came back in Week 1 (8-14 days), Week 2 (15-21 days), etc.
N-month classic retention: M1, M2, M3, and so on. On a monthly level, we look at % of new users in Month 0 (0-30 days) who came back in Month 1 (31-60 days), Month 2 (61-90 days), etc.
N-year classic retention: Y1, Y2, Y3, and so on. On an annual level, we look at % of new users in Year 0 who came back in Year 1, Year 2, etc.
We mentioned at the beginning that different teams measure retention in different ways. How should you measure your app retention?
How should I measure my app retention?
You need to define 3 things before finalizing the retention method.
Natural usage interval of your product
The key event of your product — this will map to the Aha! moment
Business model (CAC-LTV)
Let’s take two different products and understand how these 3 things matter — Facebook.com and Booking.com
Natural usage interval is once a month or quarter as we don’t travel every day
The key event is doing a transaction, i.e. booking a flight or hotel
The business model is commission per transaction
For facebook.com, it makes sense to look at daily classic retention (D1, D7, D30, D90, D180) as it will expect users to come back again and again.
For booking.com, a monthly or quarterly classic retention makes sense. You can also look at classic monthly or quarterly retention (M1, M2, or Q1, Q2) as most people travel once every 3-6 months.
It is very important to note that you should merge data across platforms while looking at retention (website, app, m-site).If you don't merge data, the calculation will go off and the insights will probably be wrong.
Oh, but why do we need a key event? It may happen that a lot of users downloaded your app, but didn’t signup and saw the feed/ made a transaction. These users didn’t reach their aha! moment and hence should be treated separately in retention analysis. This cohort has an activation problem, and not a retention problem.
The retention should be measured for users who performed a key event. They are your activated users. You can keep a tab for all new users’ retention as well, but need to worry if activated users’ retention is bad. This means that even after users experienced your product’s value, they aren’t returning. That’s bad.
What about the business model? For Facebook and booking, the business model aligns with the usage intervals of the product. High usage of Facebook along with a high # of users leads to a high # of ads and significant revenue. Low usage but high ticket size and commission of booking is good enough to recover CAC and other costs.
Why you can still fail with high retention? Usage <> Business Model Fit
Sometimes there is a mismatch between usage and business model, which raises questions around whether the product will be successful or not in the long run. An example includes ad-based OTT platforms. OTT platforms invest significant money in creating content. For example, Netflix invests > $10 B dollars in creating content. Creating content like movies and TV shows requires significant investment. Further, storing and delivering video content is also costly compared to text/images. So despite heavy usage, an OTT platform will fail if it relies on ads for revenue only (AVOD - advertising video on demand). This is the reason why Netflix has a subscription model (SVOD). Disney Hotstar and Hulu offer both AVOD and SVOD but are increasingly focussing on subscription.
Youtube still works fine with the AVOD model as it doesn’t invest anything in creating content and relies on individual creators to make videos. All it needs is to recover storage, delivery, and platform development costs.
Why you can still succeed with low retention? Usage <> Business Model Fit
Platforms with power-law distribution can succeed despite low retention as only a few users can lead to high returns.
- Low or zero marginal costs in serving to these users
- Exponential returns to scale for few customers that do retain
An example of this is Shopify. With zero or low acquisition and operational cost, these platforms like Shopify can attract and serve a high # of users at a low spend.
Even if few merchants on Shopify succeed and make big business, it will create a high return for Shopify as for every transaction, Shopify takes a small percentage from the overall cost of the sales. Here again, the usage <> business model fit is there. Ben Thompson spoke around this on @patrick_oshag's podcast: “The more companies that use Shopify and fail is a positive indicator for Shopify because that means they are getting more rolls at the dice at that one thing that hits it big. In a world of abundance, it’s all about increasing the rolls of the dice. It’s not about your failure rate, because failure is cheap - it’s often zero."
So now that we have understood how to define and measure retention, we can move on to why it matters. This will be a recap of what I have already written in earlier posts.
Why Retention Matters
Retention matters because of two reasons. First, it is a true indicator of product-market fit. As I wrote back in the piece Measuring Product-Market Fit
Here is a comparison of two products acquiring 3 million users/month for 12 months. For the sake of simplicity, we have taken monthly retention rate as constant. In reality, it keeps reducing over months.
Two products with same # of registered numbers with different retention rates. As you can see, retention rate can create huge gap between companies’ monthly active user (MAU) base. This gap will only become bigger over time, making the first one a huge success while the other one a cautionary tale. This is why, retention along with healthy acquisition is the leading indicator of product-market fit and one of the most important metric to track for product success.
The second reason why it matters is that it is not a vanity metric. Getting users by spending money through marketing is easy. Retaining them requires a good value in the product. So good retention is also a good signal of value creation. In absence of value creation, you can’t charge money and build a sustainable business over time.
But what do we call good retention?
What’s a Good Retention?
Retention varies across categories of the apps because of multiple reasons.
We have already discussed that the natural usage interval of different product categories is different. So products with different usage intervals should not be compared.
Further, the business models of products in the same category can be different. For example, AVOD vs SVOD in video streaming will work with different retention measures. Whereas daily retention is important in AVOD, monthly subscriber retention is important in SVOD.
Another factor is geography. Users across different geographies (India vs the USA) behave differently. We should keep this in mind while benchmarking.
I will emphasize that don’t compare retention rates unless the products fall in the same usage pattern, category, and business model. What I would emphasize is looking at whether your retention curve flattens over time. As I wrote in Frameworks for Building a Successful Product,
For a product to grow, the retention curve should flatten at some point just like the blue line curve in the picture below. In absence of curve flattening, all the user you gained will get out of your system at some point. As you can see in the red line, you would lose all users you acquired today in next 10 days. The product displaying the red line behavior doesn’t have product-market fit.
We will discuss more about this in the next post. Let’s now look at how to benchmark retention for your app.
How to Benchmark Your Retention
To benchmark your app/website, you should start with local competitors if possible. In order to get the competition numbers, you can use App Annie or Similarweb. App Annie is usually more reliable about app numbers, whereas Similarweb is more reliable for web numbers. The sample graph below is from App Annie.
If you are looking for global benchmarks, Lenny Rachisky along with Casey Winters wrote an excellent post about it taking inputs from growth experts across companies. This is a snapshot from his post. Retention at 6 months means classic Month-6 (M6) retention, i.e. % of new users in M0 who came back in M6.
Try to benchmark both locally and globally. Usually, in order to succeed, you need to beat local benchmarks and aspire for global benchmarks. But don’t let yourself down if you can’t beat, say Facebook.com benchmark and you are a social app.
Reasons for Low Retention
If you have low retention, it’s hard to tell some silver bullets which will work in this case. It is usually hard to figure out the reasons for your low retention.
You should remember that retention is one of the hardest metrics to move and a true test of product management and growth skills. This post would help you define how to look for retention for your product.
In the next post, we will discuss how to figure out the reasons for low retention. DM me on Twitter if you have any question.
Data from App Annie is extremely expensive - are there any low cost alternatives?
Really helpful, as always! :)