What is an outcome?
“So let’s start by defining the word in our context: an outcome is a change in human behavior that drives business results. Outcomes have nothing to do with making stuff—though they sometimes are created by making the right stuff. Instead, outcomes are the changes in customer, user, employee behavior that lead to good things for your company, your organization, or whomever is the focus of your work.”
Seiden, Joshua. Outcomes Over Output: Why customer behavior is the key metric for business success
For my entire career in software product development (software engineering and product management), I’ve been frustrated at product managers with their Excel spread sheets of 100s to 1000s of new features to be prioritized at least once a month. At my worst, I would yell and scream and throw things. I even fired a few product managers when they wouldn’t change their behavior.
Beyond the frustration with the feature list, I was bewildered as to why it would take us several version releases to come close to what users needed versus what they say they wanted. My lowest point came when one of our DEC corporate seagulls darkened my door during the development of ALL-IN-1.
Corporate Seagull
This corporate seagull was from the user experience usability group (today we call them UX). He exclaimed “I’m sorry but we can’t let you ship ALL-IN-1 V3 because it doesn’t meet our usability standards?” “Who are you again?” I asked as politely as I could muster. He explained who he was and the problems they saw with our product.
“I tell you what. We’ve just spent $10 million dollars and 18 months developing this version. We are going to ship it next week as planned. However, I will be happy to have you start working with us after we ship, so that we can fix these problems in the next release.”
With a snide smile, he responded “Oh no, we don’t work that way. You have to build it first and then we test it and then we tell you that it sucks.”
I threw my coffee cup at him and bellowed for him never to darken my door again. Fortunately, I missed.
I knew there had to be a better way for all of these problems, but I didn’t have time to look for them. I had software to build and ship.
When I was first exposed to the Institute of Design (ID) at Illinois Tech through Patrick Whitney and Larry Keeley in 1992, I found the better way I didn’t know I was searching for – human centered design (HCD).
I immersed myself in HCD by studying and teaching at ID and found the combination of user research (particularly through video ethnography) combined with the synthesis aspect of strategic design planning to be the holy grail I was looking for. But I couldn’t explain it to my product managers and development managers. Nor could I hire ID grads to help with the problem because I couldn’t teach them the business, operations, and the nature of the medium of software development fast enough. I was frustrated beyond belief.
Then as Chief Product Officer at FTI Consulting for the eDiscovery products, I discovered the power of outcomes versus outputs (feature lists). The two books that catalyzed the focus on Outcomes were:
Badass focuses on how to DEVELOP the users, not just providing a tool.
“I’m awesome!“
“It’s not about our product, our company, our brand.
“It’s not about how the user feels about us.
“It’s about how the user feels about himself, in the context of whatever it is our product, service, cause helps him do and be.
“But people don’t actually talk like that. Nobody says “I’m awesome” because of a product. They say, “I love this” or “This app is amazing.“
“It’s not about the actual words they say, but about the feelings that inspired them to say it. “I’m awesome because of this” is the feeling behind their actual words, “This thing is awesome.”
“Instead of looking for common attribute across successful products, we must look for common attributes across successful users of those products.”
Creating better users
What a concept. Don’t just make your product better. Figure out how to make your users better. With a little editing I shared the following slide with my product managers. I urged them to go beyond making a better Ringtail eDiscovery product and figure out ways we could make a better “resolver of disputes.”
Creating better users in their context
However, there is an immediate brick wall in the way of making this mental switch. In order to make a better user, I have to understand their context and business AND their customers. What do my customer’s customers need?
As I was pondering how to teach my product managers and development managers the importance of outcomes, I found Product Roadmaps ReLaunched. This book summarized the key to a good product roadmap – switch from talking about features to committing to outcomes.
“When a customer (or a CEO, or really any stakeholder) asks about whether a particular feature or design or other detail will be part of the solution, rather than answer the question, experienced product people have learned to turn it around. They ask “Why?” Why is that feature important to you? What is it about that date? The smartest product people are trying to understand what problem that stakeholder is trying to solve. This helps them understand their customers’ needs better, of course, but it also raises the level of discussion.
“With an understanding of the real goal, a product person can then ask the customer, “If I commit to solving this problem for you the best way I can, then do we have a deal?” Or, as Drift’s David Cancel suggests, “Rather than try to predict the future, why don’t I invite you into our process? If you are a key strategic customer, then when we get close to a possible solution for the thing that is of concern to you, we’ll bring you into a design review and let you give us feedback about whether it meets your needs.”
“When customers and other stakeholders make these demands, it’s because they don’t know how else to influence product direction.”
Lombardo, C. Todd; McCarthy, Bruce; Ryan, Evan; Connors, Michael. Product Roadmaps Relaunched: How to Set Direction while Embracing Uncertainty. O’Reilly Media.
Commit to Outcomes
Yet I still had the problem of training product managers to shift their thinking to outcomes versus features.
“Why not just make an endless list of features and ask our teams to work on that list—forever? In fact, a lot of contemporary project management turns out to work exactly this way. The problem with this approach is that features can be finished and delivered and “work perfectly” but still not deliver any value. Think about all those website pop ups that try to get you to subscribe to a company’s mailing list. Do they work? Technically, they function as specified. But do they deliver value? Turns out that on the whole, they don’t—people simply get annoyed and just abandon the web site instead.
“Our world is full of “features” like this that work as specified and yet deliver no value—or worse, create problems we never intended. If you’ve ever used a microwave oven you’ve experienced this problem: how many of those buttons do you use in real life?
“So if features don’t automatically create value, then it follows that we shouldn’t use them as the center of our planning process. In fact, we want to use a planning process that makes it possible to make as little stuff as possible and still achieve the outcome we seek. How do we do that? That is the question this book answers: we can instead use the idea of outcomes. Outcomes, or the human behaviors that drive business results, are what happen when you deliver the right features. Ideally, they happen when you’ve delivered as few features as possible.“
Seiden, Joshua. Outcomes Over Output: Why customer behavior is the key metric for business success . Sense & Respond Press.
An example is in order from the social impact sector with their Program Logic Model:
“Imagine that you work for a charitable organization and you’ve been asked to build a well in a small village that lacks modern plumbing. You’ve been given funding by a foundation that wants to increase the standard of living in this village. They have observed that villagers spend a large amount of time every day walking to the river to carry water. The foundation believes that if the villagers had a well in the center of the village, they wouldn’t have to carry water such long distances anymore, and they could use their time for other activities—ones that would allow them to improve their standard of living.
“In the social impact sector it’s common to use a model called the Program Logic Model to plan work like this and evaluate the results. In the diagram below you can see the building blocks of that model:
“For our well project, the model might be something like this: we plan our resources (the people, materials, money, and other things we need), we undertake a set of activities (traveling to the village, acquiring and transporting our materials, building a well). If all of this goes according to plan, we create the output—the well. If the well works as planned, we achieve our outcome—people in the village spend less time carrying water. That in turn, becomes an important contributor to the impact we seek: a higher standard of living in the village.
“Notice that the outcome—people spend less time carrying water—is a change in behavior that creates positive results.
“Why do we need all these levels in our model? Although our ultimate target is to improve the standard of living in the village, that target is actually a result of many factors. To see if our work is actually making a difference, we need checkpoints that are smaller, measurable, and more closely connected to the work that we’re doing. That’s where outcomes are important. By setting our outcome as “villagers spend less time carrying water” we have an easier time assessing the quality of our work.
Outcomes for Managers and Executives
“Setting goals as outcomes sounds simple, but it can be hard to do in practice. One thing that makes it hard is that we often set goals that are too high level—we tell a team to make our business more profitable, or to reduce risk, or something else that’s really a factor of many variables. These impact-level targets are too complex to be useful to our teams. Instead, we need to ask our teams to work on outcomes—the smaller, more manageable targets that, taken together, will create the impact we want. We do this by asking them to focus on changing customer behavior in a way that drives business results.
“We want our customers to log onto our site more often, or put an extra item in their shopping cart, or share an interesting article with a friend, or upload a picture, or complete a task in less time. What do all of these things have in common? They’re all measures of customer behavior. They might be small changes in a big system, but they are specific, and they allow our teams the flexibility to figure out the most efficient way to solve the problem, to deliver the behavior change that we seek, and to make a meaningful contribution to the impacts (revenue, profitability) that our executive leaders care about.
“So let’s review: you can manage a team by telling them what to make: that’s called managing outputs. It’s a problem because features don’t always deliver value. You can manage a team by asking them to target some high-level value, like growing revenue. That’s called managing impact. It’s a problem because it’s not specific enough.
“What you want is to manage with outcomes: ask teams to create a specific customer behavior that drives business results. That allows them to find the right solution, and keeps them focused on delivering value.
Seiden, Joshua. Outcomes Over Output: Why customer behavior is the key metric for business success . Sense & Respond Press. Kindle Edition.
When I am at my innovator best, I remember to pay attention to outcomes. When I get stuck, I go find a representative user to observe for a couple of hours. Where possible I bring a camera to augment my observation with video ethnography. In a blog post on Transactive Content, I shared a story of observing for outcomes:
Marty Smith was a senior partner and transactional attorney, formerly at K&L Gates. He took on the most important and complex contracting tasks for companies like Microsoft. As he negotiates clause by clause in these complex contracts he often has to go find similar clauses in contracts that he has constructed and then modified over the past 25 years. During a user research session on a “live” contract negotiation, we watched him spend over 30 minutes trying to find examples of ways in which he had modified a particular clause. He knew that he had done it about 30 times in the past, but couldn’t remember for which clients and which contracts. He finally gave up and had to craft his changes from scratch without the benefit of his previous work. With the Quicksilver Attenuated Search capability he would have found the documents which contain the clause within 30 seconds. The cost to the client from lost productivity >$500. The cost from not doing his best work – unknown. This happens several times a week for each transactional attorney we observed.
The Quicksilver prototype was patented and became a part of the Ringtail eDiscovery (now Nuix Discover) product and changed the searching behaviors of hundreds of attorneys.
Armed with these wonderful resources, particularly Outcomes Over Outputs, I required my product managers to start doing their roadmaps and engineering requirements in terms of outputs. They all nodded and went off to work on outcomes and nothing useful came back. What was so obvious to me, wasn’t so obvious to them. I even tried using an outcomes orientation as the backbone of a product planning session for a contract lifecycle management software company that included most of the senior executives. I knew I was in trouble when even the executives, including the Chief Product Officer, could not develop even a single outcome. Everything that came back in the working session was a feature. There was no linkage between any feature and an outcome or a business impact result.
Why is this so hard?
Often, when I am in doubt I go back and re-read something from Russ Ackoff. When I am really confused I re-read his Bell Labs story of how he “discovered” his Idealized Design technique.
“Doesn’t it strike you as odd,” the Bell Labs director said, “that the three most important contributions this laboratory has ever made to telephonic communications were made before any of you were born? What have you been doing?” he asked. “I’ll tell you,” he said. “You have been improving the parts of the system taken separately, but you have not significantly improved the system as a whole. The deficiency,” he said, “is not yours but mine. We’ve had the wrong research-and-development strategy. We have been focusing on improving parts of the system rather than focusing on the system as a whole. As a result, we have been improving the parts but not the whole. We have got to restart by focusing on designing the whole and then designing parts that fit it rather than vice versa. Therefore, gentlemen, we are going to begin by designing the system with which we would replace the existing system right now if we were free to replace it with whatever system we wanted, subject to only two not-very-restrictive constraints.”
Ackoff, Russell L.; Magidson, Jason; Addison, Herbert J.. Idealized Design: How to Dissolve Tomorrow’s Crisis…Today . Pearson Education (US).
Ackoff notes that most product developers focus on the deficiencies of an existing system. My product owners were focusing on the deficiencies of the existing system (at that point on its tenth major version). They were not looking at our customer’s system and the behaviors of the users. They were not looking at the business models of our customers. They could not “see” the behaviors that needed changing and the business results for our customers that could be enhanced.
They did not know how to see. They did not know how to observe. They did not know how to correlate what they were observing to meaningful business results and value.
I couldn’t just proclaim that we were going to move to an outcomes orientation, I had to provide relevant education in human centered design techniques AND business modelling in the Alexander Osterwalder and Ash Maurya and Steve Blank sense. Unfortunately, this knowledge would require several years and several masters degrees in a formal academic setting like the University of Washington or the Institute of Design at Illinois Tech.
My first attempt at teaching a subset of these techniques was in a graduate school class on “Designing a Human Centered Venture.” I was delighted that the ten week class produced a working Air Quality Monitor. Within the ten week class they were able to produce a “product” that they could use to test whether users would change their behavior. However, the business model and product marketing efforts were relatively weak in comparison.
Personal Air Quality Monitor
Now that I have the time while self-quarantined due to Covid-19, it is time to take Jeanine Blackwell up on her offer to help me build some online courses. One of the first courses will be about outcomes versus features.
What a great outcome if I could help product managers shift from infinite feature lists to outcome commitments in order to increase the business value to their customers.
A product produces outcomes.
