The Problem with 0-1 Metrics

READ ON

PUBLISHED

Aug 16, 2024

READING TIME

4 min & 18 sec

Dear Reader,

Welcome back! I hope you had (or still have) a relaxed or eventful (whatever your preference is) summer. I enjoyed the time off and being in places that filled me up. Among others, the baltic see. Which is where I captured one of my favorite summer shots:

Before we jump into today's issue, I want to share three things from friends that I believe you should be aware of:

Tanja Lau has launched a survey about PM burnout, and you can help her support more people in need.
With one month to go, there are still a few tickets and sponsorship opportunities available for Product at Heart, my favorite product conference. Will I see you there?
I was a guest on The ProductLive Podcast and enjoyed the amazing conversation with Omar, and I'm sure you will, too.

Let's get into this week's issue.

Outcomes over Outputs, but not at all cost: It’s a powerful shift for product teams to use measurable changes in customer behavior (aka Outcomes) to prioritize work and measure progress. However, if that progress takes weeks or months to detect, these metrics are not as helpful as they should be.

As Ravi Mehta once said, “The right goal depends on the team's understanding of what it is trying to accomplish.“ Whether a product team creates a new solution within a company or a newly founded startup, Measuring progress when you don’t have customer signals out of the gate can feel foggy.

While you’re on your way toward improving customer behaviors, the feedback loops for your actions are simply not there. There’s not much to check in on a metric that hasn’t moved and, you know, won’t move for months. 0-1 efforts require more pragmatic Output or even Input metrics to measure your progress.

Let’s look at an example: An analytics software wants to offer split testing functionality to increase the share of wallet and time spent with existing customers. The outcome to aim for would be the number of started or completed split tests by existing customers. But that metric would only begin to be measurable once the functionality is rolled out to a significant number of customers—it’s lagging to help the team working on this initiative see if they’re on track.

The team has to derive influenceable goals throughout the different stages of getting there.

HOW TO PUT THIS THEORY INTO PRACTICE

Clarify your Lagging Outcomes. Everyone has a customer. You should be able to describe and measure how you want your work to improve their experiences.
Embrace Outputs. An ambitious Output goal can be more helpful than a lofty Outcome goal that doesn't help the team get started. Your metrics have to help you measure progress, not meet someone else's standards.
Combine short-term and mid-term check-ins. Use a weekly cadence for your leading metrics, and review your progress monthly or quarterly to ensure you don't lose sight of the lagging Outcomes.

Did you enjoy this one or have feedback? Do reply. It's motivating. I'm not a robot; I read and respond to every subscriber email I get (ask around). If this newsletter isn't for you anymore, you can unsubscribe here.

Thank you for Practicing Product,

Tim

Join my In-Person Workshops in Berlin

I'm excited to bring my beloved in-person workshops back to Berlin in January 2025. You can choose between 1-day workshops on Product Strategy, Product OKRs, or Product Discovery OR get the full 3-day experience for you or your team.

LEARN MORE

(early bird pricing is available until Sep 21)

Content I found Practical This Week

Begin with the end in mind

When we see a problem, we naturally start thinking about what we can do, who can do it, and what's the due date. We often operate on autopilot, doing what we’ve always done. When we do that, we go through the motions without questioning what we want to achieve or how we work. It’s also extremely easy to fall in love with our ideas and believe they’ll all make a difference. We all tend to fall into the know-it-all mindset, assuming that we have everything figured out and that all our projects will generate results.

Read it here

Pre-Work for Setting OKRs

Examine previous OKRs or strategic plans to see what worked and what didn’t. Were objectives too ambitious or not challenging enough? Did key results deliver tangible outcomes? Leverage these insights to refine your OKRs, focusing on clear, measurable goals that build on past learnings. If people didn’t even get close to the desired effect, choose gentle “yoga stretch” goals instead of moonshot goals. If teams are achieving OKRs easily, then nudge them to aim 10% higher.

Read it here

OKR Myths

Myth #1: planning is a distraction from "real work": Potentially - when planning is done just for the sake of planning, yes, but, if you step back and look at it as a planning -> execution -> learning loop, you'll recognize the need for a shared method across your org to align and scale. Operationalizing how you work and how you work across organizational boundaries can be your secret sauce and allow you to do more "real work"

Read it here