🛠️ The Problem with 0-1 Metrics


The Problem with 0-1 Metrics

READ ON

​HERBIG.CO​

PUBLISHED

Aug 16, 2024

READING TIME

4 min & 18 sec

​Dear Reader,​

Welcome back! I hope you had (or still have) a relaxed or eventful (whatever your preference is) summer. I enjoyed the time off and being in places that filled me up. Among others, the baltic see. Which is where I captured one of my favorite summer shots:

Before we jump into today's issue, I want to share three things from friends that I believe you should be aware of:

  1. Tanja Lau has launched a survey about PM burnout, and you can help her support more people in need.
  2. With one month to go, there are still a few tickets and sponsorship opportunities available for Product at Heart, my favorite product conference. Will I see you there?
  3. ​I was a guest on The ProductLive Podcast and enjoyed the amazing conversation with Omar, and I'm sure you will, too.

Let's get into this week's issue.

​​Outcomes over Outputs, but not at all cost​: It’s a powerful shift for product teams to use measurable changes in customer behavior (aka Outcomes) to prioritize work and measure progress. However, if that progress takes weeks or months to detect, these metrics are not as helpful as they should be.

​​As Ravi Mehta once said​, “The right goal depends on the team's understanding of what it is trying to accomplish.“ Whether a product team creates a new solution within a company or a newly founded startup, Measuring progress when you don’t have customer signals out of the gate can feel foggy.

While you’re on your way toward improving customer behaviors, the feedback loops for your actions are simply not there. There’s not much to check in on a metric that hasn’t moved and, you know, won’t move for months. 0-1 efforts require more pragmatic Output or even Input metrics to measure your progress.

Let’s look at an example: An analytics software wants to offer split testing functionality to increase the share of wallet and time spent with existing customers. The outcome to aim for would be the number of started or completed split tests by existing customers. But that metric would only begin to be measurable once the functionality is rolled out to a significant number of customers​—it’s lagging​ to help the team working on this initiative see if they’re on track.

The team has to derive influenceable goals throughout the different stages of getting there.

HOW TO PUT THIS THEORY INTO PRACTICE

  • Clarify your Lagging Outcomes. Everyone has a customer. You should be able to describe and measure how you want your work to improve their experiences.
  • Embrace Outputs. An ambitious Output goal can be more helpful than a lofty Outcome goal that doesn't help the team get started. Your metrics have to help you measure progress, not meet someone else's standards.
  • Combine short-term and mid-term check-ins. Use a weekly cadence for your leading metrics, and review your progress monthly or quarterly to ensure you don't lose sight of the lagging Outcomes.

Did you enjoy this one or have feedback? Do reply. It's motivating. I'm not a robot; I read and respond to every subscriber email I get (ask around). If this newsletter isn't for you anymore, you can unsubscribe here.

Thank you for Practicing Product,

​Tim​

Join my In-Person Workshops in Berlin

I'm excited to bring my beloved in-person workshops back to Berlin in January 2025. You can choose between 1-day workshops on Product Strategy, Product OKRs, or Product Discovery OR get the full 3-day experience for you or your team.

(early bird pricing is available until Sep 21)

Content I found Practical This Week

Begin with the end in mind

When we see a problem, we naturally start thinking about what we can do, who can do it, and what's the due date. We often operate on autopilot, doing what we’ve always done. When we do that, we go through the motions without questioning what we want to achieve or how we work. It’s also extremely easy to fall in love with our ideas and believe they’ll all make a difference. We all tend to fall into the know-it-all mindset, assuming that we have everything figured out and that all our projects will generate results.

Pre-Work for Setting OKRs

Examine previous OKRs or strategic plans to see what worked and what didn’t. Were objectives too ambitious or not challenging enough? Did key results deliver tangible outcomes? Leverage these insights to refine your OKRs, focusing on clear, measurable goals that build on past learnings. If people didn’t even get close to the desired effect, choose gentle “yoga stretch” goals instead of moonshot goals. If teams are achieving OKRs easily, then nudge them to aim 10% higher.

OKR Myths

Myth #1: planning is a distraction from "real work": Potentially - when planning is done just for the sake of planning, yes, but, if you step back and look at it as a planning -> execution -> learning loop, you'll recognize the need for a shared method across your org to align and scale. Operationalizing how you work and how you work across organizational boundaries can be your secret sauce and allow you to do more "real work"

What did you think of this week's newsletter?

​👎​

​Bad​

​🤷‍♂️​

​Meh​

​👍​

​Great​

Who is Tim Herbig?

As a Product Management Coach, I guide Product Teams to measure the real progress of their evidence-informed decisions.

I focus on better practices to connect the dots of Product Strategy, Product OKRs, and Product Discovery.

Enjoy the newsletter? Please forward it. It only takes 2 clicks. Coming up with this one took 2 hours.

Product Practice Newsletter

1 tip & 3 resources per week to improve your Strategy, OKRs, and Discovery practices in less than 5 minutes. Explore my new book on realprogressbook.com

Read more from Product Practice Newsletter

Product Practice #407 Your Goal Depends on Another Team — Now What? PUBLISHED May 7, 2026 READ ON HERBIG.CO Dear Reader, Your Key Result says to "Improve Conversion Rate by 7%," but you only control on-site search. You want to drive customer retention, but the marketing team is focused on new acquisition. Most teams respond in one of two ways: they water down the goal until it fits their scope (and lose the ambition), or they keep the big goal and quietly accept they can't move it. Both lead...

Product Practice #406 Why your Company's POV on OKRs matters more than Processes PUBLISHED Apr 30, 2026 READ ON HERBIG.CO Dear Reader, Before I talk with companies about OKR cadences, templates, or tools, I ask them: "What do you expect to change by using OKRs?" The answers take a bit of time. Not because they don't exist, but because OKRs have been treated as a solution without a problem to solve. OKRs haven't been treated as a product, but as a process without purpose. They've chosen a...

Product Practice #405 How to treat Prototypes as Decision-Making Tools PUBLISHED Apr 23, 2026 READ ON HERBIG.CO From Strategy Choice to Planned Experiments in One Workshop In my upcoming live remote workshop From Strategy to Discovery, I take you from sharpening your Product Strategy, to defining leading indicators for measuring progress, all the way to prioritized assumptions you need to derisk end-to-end. Three 4h Live Sessions - Lifetime Material and Recording Access - Real Results Join...