Outcomes Are Hard

By Marty Cagan and Felipe Castro

Marty’s Note:

My co-author for this article is the product coach Felipe Castro.  I have known Felipe for many years, and he has collaborated with me on the outcome and OKR-related content in each of my books. Felipe specializes in helping companies move from output to outcomes.

At the highest level, moving to the product model is about moving from outputs to outcomes.

Conceptually, this is very straightforward and the value is intuitively clear to most people, but in practice, this is hard.

We’ve been talking and writing about the move to outcomes in all of our books, for many years.

The good news is that many other people have realized that just shipping features that don’t actually help the customers, or help the business, is not very meaningful.  And today, many strong teams and companies have successfully moved their focus to outcomes.

But for many companies, the move to outcomes goes nowhere fast.

In this article, we would like to try to address some of the most common mistakes and confusions that come up when organizations attempt to move to outcomes. 

We’ll also discuss some of the important nuances for those that have already thought about these concepts for a while.

The Concept of Outcomes

The basic idea behind outcomes is easy to explain:

We all know that some of the things we build make a difference, while others do not. That difference might be improving the lives of our customers, or improving the health of our business, or even helping our teams work more effectively.

Moving from outputs to outcomes means shifting attention from what we build (the outputs) to the difference we want to make (the outcomes).

But as anyone that has tried moving to outcomes can tell you, this is much easier said than done.

Framing Outcomes

When we talk about an outcome, we frame it around a problem to solve and the measures of success.

Specifically, this requires:

• Giving teams clear and meaningful problems to solve.

• Helping teams define how they’ll measure if they are truly solving each problem.

• Instrumenting the product so teams can immediately understand if they are achieving the intended outcome (i.e. solving the problem), and make adjustments based on the data.

While this framing may sound straightforward, many companies struggle as it can represent a significant behavioral and cultural change.

Common Mistakes When Moving to Outcomes

– Thinking You Can Move To Outcomes Without Moving To The Product Model

Far and away the single most common problem is thinking that you can move your organization to focus on outcomes, without actually doing the work of moving your organization to the product model.

This is understandable as outcomes are intuitive and conceptually simple, and it’s not immediately obvious to people why they would need to make a set of additional foundational changes in order to enable outcomes.

This is what happened to the countless thousands of companies that thought that they could layer in the OKR technique on top of their existing, output-based product roadmap processes, and expect any meaningful change.  This mistake is what fueled the backlash to OKR’s in so many companies.

But the OKR technique originated from, and is predicated upon, the product model.  

The OKR technique is quite straightforward if you already have established the product model as your way of working.

Moving to the product model involves significant changes to how you decide which problems you need to solve, how you solve those problems, and how you build, test and deploy your solutions.  But realize that every aspect of the move to the product model exists to enable your organization to deliver outcomes.

So the most important thing to understand is that moving to outcomes is the consequence of moving to the product model.

– Letting Existing Metrics Dictate Problems to Solve

We often see companies limiting themselves to the outcomes that can be measured by their current indicators. For everything else, they use outputs.

However, quite often we’ll find that some of the most pressing customer or business problems are not yet properly covered by established KPI’s.

Strong product leaders don’t let existing metrics dictate which problems to solve or which measures of success to use. They know they have to focus on the most critical problems, even if they have to define new indicators or do some additional instrumentation work to do it.

A simple technique is ensuring we define a clear and meaningful problem before committing to specific metrics.

– Vague or Ambiguous Problems to Solve

Feature-team companies are often filled with vague goals that no executive can properly define or measure (e.g., “provide integrated platforms”). When all you measure is project completion, it doesn’t really matter if your goals are vague.

In contrast, strong product model companies invest time upfront to define clear problems so they can ensure they’re working on the right problems, and save substantial time down the road. 

Here are several examples of what clear problems look like:

• Help customers solve issues without having to contact us
• Increase the percentage of shipments delivered with next-day delivery
• Make it easier for patients to reliably share data with their doctors
• Reduce the cost to acquire a new customer
• Reduce the operational costs of fulfillment
• Reduce the subscriber churn rate
• Reduce the time required to deploy a new or updated service to production
• Connect job seekers with more suitable jobs
• Reduce the average time spent handling a customer service call
• Reduce the time for a user to produce their first monthly report

Note that problems exist at different levels of granularity or “altitude;” some problems are more specific and can be tackled by a single product team, while others are broader and need to be tackled by multiple product teams collaborating closely; and some involve close collaboration with others in the company beyond the product teams.

Problems around optimizing existing metrics are usually the easiest to articulate (e.g., increasing conversion on the checkout page). In other scenarios, getting to clarity can be quite challenging, but often that’s where the major opportunities are.

Most measurement and alignment challenges are actually clarity challenges. When you have a clear definition of what you are trying to achieve, it makes it easier to measure success, create alignment between stakeholders, and discover effective solutions.  Achieving outcomes requires clarity.

– Not Having an Intentional Product Strategy

The examples above are all potentially important problems to solve, depending on the company’s product strategy.

But as we’ve discussed many times before, many companies do not have an intentional or explicit product strategy. They have business goals, and they have roadmaps of product ideas they hope will generate positive results.

The product strategy represents the data, the logic and the decisions that lead from our top-level business goals to the specific problems to solve, and outcomes to achieve, from the various product teams.

When some people hear the word “outcomes” they immediately jump to top-line and bottom-line definitions of business results – revenue and profit being the main examples.

These top-level goals are certainly examples of outcomes, but they are more typically the end-result of many different outcomes, at many different levels of granularity, all contributing to the ultimate result.

For example, are our customers getting their needs met with our products? How long does it take to sell our products? How long does it take for a customer to start receiving the value? How long do our customers continue to use our products? These are all outcomes that can contribute real impact to end-results such as revenue and profitability.

Instead of seeing outcomes in isolation, sometimes it’s easier to think of them as dominoes, with each piece helping to topple the next.

This is where strategy becomes essential: a proper product strategy helps identify the most critical problems that need to be solved to deliver on the business objectives.

Another important dimension when choosing outcomes is whether or not it’s within a given product team’s ability to directly impact the desired outcome.

A team won’t feel empowered to come up with an effective solution and achieve the necessary outcome, if they lack the access or ability to make the changes necessary.

They also will be unlikely to discover a solution that will work for the customer and the business if they can’t measure if they are making the necessary impact.

– Measuring the Wrong Things

When we look at the measures of success, we’re trying to define how we will know if we have truly solved this problem? Certainly just launching a feature is not enough.

Some people confuse outcomes with the much more general concept of KPIs (Key Performance Indicators). KPIs are simply analytics that measure some aspect of a product or business.

Most businesses have hundreds or even thousands of relevant KPI’s, which help the leaders evaluate the health of a product or a business. While outcomes are measured with KPI’s, the vast majority of KPIs are not helpful measures of successful outcomes.

Every KPI is a data point, and that data point may provide clues or supporting information, but we’re looking for the KPI (or KPI’s) that are true measures of our desired outcomes.

As a simple example, the gas gauge and the tachometer on the dashboard of a (gas powered) car are KPI’s, but if the outcome you care about is improving fuel efficiency, these KPI’s are not really measuring if you are solving the problem (e.g. increasing miles per gallon).

That’s why we want to define a clear problem to solve before discussing the metrics. The goal is to measure if we are solving the selected problem, even if that means defining new KPI’s that we don’t yet track today.

One useful technique for finding good KPI’s to measure outcome is thinking about the desired changes in user behavior (for example, the percentage of our customers making in-app purchases).  

But keep in mind that many important problems do not necessarily impact user behavior.  Also remember that problems-to-solve can refer to customer problems, and/or our company problems.

– Neglecting Data Collection

Another item that becomes non-negotiable if you want to move to outcomes is the need to instrument whatever we release (referred to as telemetry) so teams can immediately understand if they are solving the problem, and make adjustments based on the data.

Yet many companies are still not gathering and analyzing their data as they need to—focusing on outputs allows them to treat analytics and reporting as a “nice to have.”

Additionally, as discussed above, many teams shy away from adopting the proper measures of success simply because they don’t already have the necessary instrumentation in place.

If you want to focus on outcomes, you will need to commit to continuously improving your product’s telemetry, creating new KPI’s and collecting additional data where necessary to measure if you are solving the necessary problems.

Common Confusions Regarding Outcomes

– Alternative Ways To Frame Outcomes

Several people have been exploring the mechanisms and the consequences of this move to outcomes, and there has been some truly useful content written on the subject.

Some of our favorites are Jeff Patton’s outputs vs outcomes vs impact, and Teresa Torres’ distinction between product outcomes and business outcomes.

We often refer to the outcomes tackled by product teams as product outcomes, and they roll up to contribute to business outcomes (aka “business results”).

Another way to frame this is to distinguish between outcomes (the difference we make by solving important problems) and impact (the top-level business results these outcomes help achieve).

These conceptual distinctions and simplifications can help people feel more comfortable with their role in the move to outcomes. 

But it’s also important to realize that we want to encourage product teams to embrace work outside of their direct purview, in order to understand the issue, and what they can and should do to solve the problem.

For example, to understand what will be necessary to solve a problem that is showing up during the sales process, it is not unusual for a product manager to spend time with sales, going on sales calls, observing the interactions, to see where the issue might be.

– Inputs vs Outputs

Amazon is one of the best examples of the product model, but internally they use some language around the term “output” that can cause confusion when Amazonians use that language externally.

Amazon emphasizes the importance of focusing on the inputs, not the outputs.

This means focusing on the “input metrics” (e.g. selection, price, and convenience) that will drive the “output metrics” (e.g. orders, revenue, and profit).

When Amazon emphasizes the need to “manage the inputs, not the outputs,” this means focusing on managing the causes, not the effects.

This is also an important point, and relates directly to product strategy, but is not to be confused with the definition of outputs which are the things we build.

– Keep The Lights On

It’s critical to always remember that not everything needs to be a problem to solve with an outcome.

There is always a set of “Keep The Lights On” items that are like a punch list of typically minor work that our business partners need to be able to keep things running.

Summary

There’s no question that delivering outcomes is much more difficult than delivering output. It requires new competencies, new skills, and a different approach to defining and assigning work.

There are a number of other items relating to outcomes beyond what we’re highlighting here. The book EMPOWERED: Ordinary People; Extraordinary Products provides a much more comprehensive explanation with detailed examples and case studies.

But for those product teams that are serious about delivering real value, solving the underlying problem for our customers or our business, and delivering the necessary business results is the only real measure of success.