Baseline and Target
Two numbers to measure success
Measuring product success is tricky. Chasing profit, user satisfaction, or efficiency are likely insufficient success indicators because a myopic focus on metrics can lead us astray. Yet, not tracking performance metrics is also inadequate. A good product manager will track several performance indicators for their company, projects, teams, and themselves, so defining meaningful metrics is critical. Like swans, meaningful metrics come in pairs: A baseline and a target.
A baseline is a starting point—where you’re at today.
A target is a future point—where you’d like to be.
You cannot have a meaningful metric without these two numbers.
“Never believe that one number on its own can be meaningful. If offered one number, always ask for at least one more. Something to compare it with.”
– Hans Rosling, Factfulness (2018)
To define key performance indicators for a product, measure the difference between the baseline and the target so you’re not chasing a solo swan. Establishing a baseline is a prerequisite to setting a target.
Baseline
The baseline is essential because it necessitates two things:
Knowing what to measure
Having a firm grasp of reality
Knowing what to measure is no small task because pursuing the wrong metric can create perverse incentives. Venomous cobras were a public health crisis in Colonial India, so the British government offered a bounty for every dead cobra. Rather than decrease, the cobra population grew because people bred them for bounty money. The Cobra Effect occurs when the wrong metric—or even the right metric in the wrong context—is applied.
When measuring product performance, your first job is to select the right metric. In general, establishing metrics should follow the desired business outcome, but here are a handful of useful metrics to consider:
For an app in the consumer space: daily active users (DAU), cost per install (CPI), and lifetime value (LTV).
For a B2B SaaS product: customer acquisition cost (CAC), retention rate, and monthly recurring revenue (MRR).
For an internal product: error rate, user cycle time, and net promoter score (NPS).
Once you’ve selected the metric(s) you wish to track, establish a firm grasp of reality. This grounded perspective is the foundation for data-driven decisions, such as setting your target metric. To grasp reality, you must benchmark.
Benchmark
Benchmarking is the act of establishing your baseline. First, understand what data will help you create the baseline metric and how to capture said data. If you’re looking at DAU, you’ll need an analytics solution to show how often unique users return to the product. To understand NPS, you’ll need a user survey to source initial results.
When defining your benchmarking goals, consider the collection mechanism and the reporting frequency. Figure out how often you’ll need to receive and review this data. DAU would require daily updates, while MRR or CAC could occur monthly.
After collecting for a reasonable time (30 days is a good rule of thumb, but this varies by metric) and the collection mechanism has proved reliable, assess the mean and median of the benchmarked data to set your baseline.
After defining your baseline, you can set the target.
Target
Ironically, setting the target metric becomes more philosophical than data-driven. While the essence of the baseline metric is to form a firm grasp of reality, the target metric should inspire action. With that goal in mind, two schools of thought arise: The Realist and The Mystic.
The Realist sets attainable targets that are feasible to attain.
The Mystic sets idealistic targets that are almost impossible to attain.
The Realist believes that feasible targets inspire action because one can visualize success and define a rational path to get there. Imagine a business with $2M MRR (baseline) that wants to achieve $2.5M MRR (target) by year-end. This goal may be feasible if the company has a product-market fit, a decent sales funnel, and stable operations.
The Mystic believes that a high target metric can manifest the necessary behavior—inspiring teams to rise to the occasion and get creative. Imagine a business with $2M MRR (baseline) that wants to achieve $10M MRR (target) by year’s end. Even with product-market fit, a decent sales funnel, and stable operations, this goal could be impossible since it hinges on 5x growth in a year. Setting a target like this is aspirational, and that comes with risk. People might view it as absurd, ignore it as a vanity metric, and not invest the effort to make it happen. Or people could over-index on it at the cost of recreating the Cobra Effect.
As a rule of thumb, product managers are pragmatists, so we should bias toward The Realist mentality: set feasible target metrics informed by well-established baselines.
After accruing measurable wins and earning more ownership, we could unlock the gumption to set more Mystical targets.
Takeaways
To set up your success metrics for success:
You need a baseline (where you are) and a target (where you want to be).
If you don’t know your baseline, set up a collection mechanism to benchmark.
After establishing a baseline, set a feasible target metric informed by reality.
Action Items
This week, evaluate your product’s performance metrics. For each metric, ask yourself:
Is my baseline accurate?
Is my target feasible?
If you don’t have a baseline metric, what do you need to start benchmarking?
When it comes to product metrics, there is something between realistic (25% growth) and mystic (5X growth) - maybe it is aggressive or visionary - that sets a tough goal of 50% growth. I had a manager that believed you needed aggressive goals to be sure to hit the reasonable goals.
Excellent article covering product metrics! First time to hear the Cobra Theory!