Tackling Some Pressing PM Dilemmas Using Marginal Returns
Tackling Some Pressing PM Dilemmas Using Marginal Returns
For the new ones here — 8,000+ smart, curious folks have subscribed to the growth catalyst newsletter so far. If you are new here, receive the newsletter in your email by subscribing 👇
You can read 50+ posts from the past here - https://www.growth-catalyst.in/archive
Product Management (PM) dilemmas are exhausting. Most of the times, they are subjective and have to operate in an irrational world of consumers and peers. So when a concept or framework helps in solving some of the dilemmas, I would expect PMs to pay attention.
In this post, we are going to answer three dilemmas:
How much time should Founders/PMs spend in user research?
Is adding more engineers to a project a good idea to hit timelines?
Should PMs learn to code?
Let’s start with a concept that can help us answer these questions.
Marginal Returns
The first piece of cake feels really good. The second piece might be good. But we may have to force ourselves to eat the third. Instead of enjoying it, we may start hating it if we keep eating. Why doesn’t the last piece taste as good as the first?
We’ve all experienced the moment when our enjoyment of something declines as we have more and more of it. With each additional unit, , less and less value gets added to our lives. Why doesn’t adding more of something continue to add up to something better?
Diminishing marginal returns is quite applicable in our day-to-day lives and can also help answer the questions listed above.
Marginal Returns is a concept borrowed from economics. It helps you understand how much extra output you will get if you increase your input (money, effort, time, etc.) by a little bit. In simple terms, it is defined as
Marginal ROI = (Change in Output / Change in Input)
Let’s take an example to understand this. In business, the output is often revenue, and the input is cost. So imagine a business that is generating $1000 every month today at the cost of $500. If we add another $100 as an investment (cost), we end up generating an additional $150. The marginal ROI in this case would be
Marginal ROI = Change in Revenue/ Change in Cost = $150/$100 = 1.5
Increasing, Constant, Diminishing, or Negative?
Marginal returns can be increasing, constant, diminishing, or negative.
We can understand it through a simple example (you can look at the graph below). Let’s say you started running Facebook Ads, spending $100 daily, for a product you just launched. Initially, you are getting a revenue of $500 for the $100 you are spending.
Since the return is good, you naturally want to increase the spend on Facebook Ads. You increase the daily spend from $100 to $200. Your revenue increase to $1000. So the ratio of change in output to change in input is 5. This would be increasing returns.
The numbers look good, so you increase the budget to $300. The revenue increase to $2100. The the ratio of change in revenue($100) to change in spend($100) is 1. This would be constant returns.
If we now increase the spend to $400, the revenue increases to $2150. The ratio of change in output ($50) to change in input ($100) is 0.5. This would be diminishing returns.
What about negative returns? Is that possible in such a marketing campaign? To have negative returns, the increase in spend should lead to loss of revenue. We usually don’t see negative returns in marketing campaigns. But you can observe it in other places.
Imagine studying for a test that is scheduled tomorrow at 9 am. The first few hours would be have increasing returns because you go from knowing nothing to something. The next few hours can have constant or diminishing returns. But if you keep studying till the test without proper rest, that might lead to negative returns since you performance in test might suffer due to mental exhaustion.
In the case of a test, the marginal return isn’t quite quantifiable. We have a good sense of marginal return, but can’t mathematically put it. Another example for that is — suppose you are planning to learn about chatGPT which is a hot topic these days. You have already read 10 articles around it, and now contemplating whether you should read 11th article based on returns you may get from it. It would be hard to quantify changes in your knowledge from the next article, but you very well know based on the outline of an article what kind of marginal returns it will provide. If it is a research paper, the marginal return may be increasing. If it’s another superficial article, it would have diminishing returns.
Now that we understand this, let’s jump to the questions.
Time Spent in User Research
Qualitative user research is one of the best ways to gain insights about the end users.
If your goal is to understand human behaviour to make design decisions, qualitative methods are much more effective in providing you with the information you need.
— Kim Goodwin in “Designing for the Digital Age”
Usability testing and 1:1 interviews are primary ways to gain these insights. Usability testing typically involves observing how users use the product and identifying issues they face in using the product. 1:1 interviews are where you talk to end users with a list of exploratory questions to validate few hypotheses.
Even if user research is useful, testing with users or talking to users takes time. So most PMs don’t do it because they feel they have to spend a lot of time doing it.
The reality is far from the perception because of high diminishing returns in research after first few users. Let’s talk about usability tests first.
Diminishing Returns in Usability Testing
The number of new issues identified with the first few participants in usability tests is high, and decreases as we add more participants. Eventually, saturation happens when when no new usability issues are identified.
In 2000, Jacob Nielsen published a statistic around usability testing that shows the diminishing returns in usability testing clearly.
75-80% of insights can be discovered by interacting with just 5-6 users, and the saturation occurs around 10 participants. So if you are planning to do usability testing, you just have to find 5-6 users and no more.
What about user interviews? How many participants do you need to interview to achieve saturation?
Diminishing Returns in 1:1 User Interviews
It turns out that getting a single number of users beyond which the diminishing returns kicks in for user interviews is a little more complex than usability tests because the segments of the population can be many or few depending on the project. As we increase segments, we need to interview each segment and total interviews increases. For example, a study looking at determining how people feel about a country’s healthcare might require a sample of 20–30 interviews to see saturation. The segments here are many - young, old, healthy, sick, men, women, etc. The scope of the research (accessing healthcare) is quite broad. On the other hand, a study looking to determine healthcare experience for patients with Type II Diabetes could require as few as 5-10 interviews. The segment as well as scope here is narrow.
In short, the marginal returns curve in case of interviews is similar to the one for usability tests and has diminishing returns but the saturation point varies depending on the # of segments.
A general rule of thumb is 5-10 interviews per segment.
Adding More Engineers to a Project
We have all been in situations where a software project gets delayed. One of the most common suggestions I have heard from business folks, and sometimes PMs, is to add more engineers to hit the promised timeline.
The origins of this suggestion comes from industrial era where adding more people for manual labour in a factory increases output. The concept also applies to other domains, where the task can be broken into smaller units, and can be assigned to different people without taking a hit in output per person.
To answer whether adding more engineers to a delayed project, we have to look at whether software projects can be broken into smaller units and can be assigned to different people without affecting output. Since the code written by one engineer has to work perfectly with the code written by another engineer, software projects don’t follow the same laws that hold true in the Industrial era.
You can also refer an an excellent book around this — the mythical man-month. It is a book on software engineering and project management by Fred Brooks first published in 1975.
The title of the book is not random, rather suggestive. The man-month (or woman-month) represents the effort required to complete a software project. So if a project required 10 man-months, it means that we would need 1 engineer to work 10 months to complete the project. One may suggest that if we have to complete it in 1 month, we can always add 10 engineers to it, thereby fulfilling 1x10 man-months criteria. And therein lies the myth because
1 engineer x 10 months != 10 engineers x 1 month
So adding more engineers to a project does not necessarily lead to increased output. In fact, adding more engineers can often lead to diminishing or negative returns. There are various reasons for this and as a product manager, it's important to understand why this is the case.
First of all, when you add more engineers to a project, you also add more communication overhead. Each engineer has to communicate with every other engineer on the team, and this can become a bottleneck as the team grows larger. Additionally, more engineers means more time spent on coordination and management, which can take away from actual development work. In the image below, you can see that increasing # of people increases communication lines geometrically.
Another reason for mythical man-month is that software projects are inherently complexity. For complex projects, new engineers have to spend time getting up to speed on the project and understanding its intricacies, which can slow down progress.
In summary, adding more engineers can be beneficial in some cases, but it's important to understand where before making that decision.
This brings us to our third question.
Should PMs Learn to Code?
I have already answered this question in my book — ‘Tech Simplified for PMs and Entrepreneurs’. So if you have read the book, you can skip this section.
For PMs who come from a non-software engineering background, this is an existential question. Learning to code can be beneficial for a PM in several ways. It can help PMs understand the technical aspects of the product, communicate effectively with engineers, and make informed decisions about the product roadmap.
Apart from learning to code, PMs also have to understand tech concepts, processes (agile, versioning, etc.), and high level system design related to the product they are managing.
So let’s look at the dilemma from the lens of marginal returns.
Increasing/constant returns: PMs should definitely understand tech terms and processes. This is where you have to start. If a PM doesn’t understand what an API is, they won’t be able to understand what developers are talking about. They can also explain technical issues to other stakeholders (CEO, Business, Design) around projects/features. This is where they have increasing returns in terms of peer trust and effective communication.
Once a PM has understood tech terms and processes, they can focus on understanding high level system design. This way, they can understand the technical complexities of product, and also contribute in high level technical discussions which can quicken decision making.
Diminishing/Negative returns: Learning to code has diminishing returns for PMs, and sometime negative depending on the product areas they manage.
As a PM, you aren’t supposed to write code. Add to that, learning to code can be time-consuming, and the time spent learning to code could be spent on other important PM responsibilities, such as conducting market research, developing a product roadmap, or working with cross-functional teams to bring the product to market. So spending time here can lead to distraction from core responsibilities.
Another tendency of such PMs is to get too much involved in technical aspects of the product which can lead to micromanagement and hinder the development process.
Overall, while learning to code can be beneficial for a PM, it is important to consider the potential disadvantages and opportunity costs. A PM should focus on developing the skills that are most relevant to their role and the product they are working on.
Where Marginal Returns Don’t Apply
Diminishing marginal returns apply in manufacturing or agriculture when a single input variable (say, # of employees) increases while keeping other input variables constant.
But it doesn’t apply if every input variable of the system (# of employees, machines, raw material, etc.) scales in the same proportion. That is where economies of scale kicks in, but that is a topic for some other day :)
To summarise, marginal returns is a beautiful concept that can be applied from software engineering projects to user research to even ever-debatable questions like ‘Should PMs code?’. I would encourage you to think of more applications of the concept.
QUICK UPDATE!
The first cohort of product-led growth course has been fantastic. I had kept an interest list for various topics I was planning to build content around, at https://www.pmcurve.com/.
The interest for product-led growth is highest among all, and we already have ~200 people waiting for it. So there is going to be a second cohort, starting April’23. You can apply for the course here, and checkout more about the course here.
See you next week! Do hit the like button for the post if it was useful/actionable :)