Why Your 'Correct' Discovery Method Might Be Wrong


Why Your 'Correct' Discovery Method Might Be Wrong

READ ON

HERBIG.CO

PUBLISHED

Feb 7, 2025

READING TIME

3 min & 34 sec

​Dear Reader,​

"Which experiment should we run next?"

This question comes up in almost every Discovery coaching session I facilitate. Teams often focus on finding the methodologically perfect way to test their assumptions. But here's the thing: the most technically correct experiment isn't always the right one to run.

When choosing methods for Product Discovery, we often focus on what fits our research question or assumption best. Say you want to understand how users perceive features in your product.

A diary study might be the perfect method—capturing real usage patterns over time. But what if it takes three months to get reliable insights? A series of well-structured interviews or shadowing sessions might get you 80% of the way there in just two weeks.

Several factors determine your lead time to insight:

  • Your team's skills and capabilities
  • Industry context and constraints
  • Available infrastructure
  • Customer interaction patterns
  • Product characteristics

Here's another way to think about it: An A/B test might seem quick to start—taking just hours or days to implement. But depending on your traffic and conversion rates, it could take weeks or months to reach statistical significance. In contrast, while recruiting participants for qualitative interviews might take two weeks, you could have reliable insights within days of completing them.

Neither method is inherently better. What matters is the total time to reliable insight, not just how quickly you can get started.

So, when picking your next Discovery priority, ask yourself:

  1. How well does the method answer our research question?
  2. How technically correct is the method for our needs?
  3. What's our lead time to the next viable insight?

Here's a practical tip: When evaluating lead time, avoid abstract scoring systems. Instead, estimate the actual duration by adding:

  • Setup time (recruiting, preparation)
  • Execution time (running the experiment)
  • Synthesis time (analyzing and learning)

This concrete timeline helps you make practical trade-offs between methods.

Remember: The goal isn't to achieve perfect certainty. It's to reduce uncertainty enough to make confident decisions about what to build next.

1 Question For You To Put This Into Practice

Look at your current Discovery priorities: Which assumption could you test differently if you optimized for lead time to insight instead of methodological correctness?

Reply and let me know your answer.

Did you enjoy the newsletter? Please forward it. It only takes two clicks. Creating this one took two hours.

Thank you for Practicing Product,

Tim

Good News!

Some last tickets are available for my in-person Product Discovery workshop on March 10 in London (as part of Mind the Product conference).

Content I found Practical This Week

Product Discovery Should Speed Up Delivery, NOT Slow it Down

Now imagine a team that is responsible for both discovery and delivery. They have learned the hard way how critical it is to have those developing the product involved in discovery. This is easy for them to do too because they’re part of the same team! This immediately does two things: Kills off handoffs. Empowers those delivering the work with key context. This significantly reduces the time the team has to stop during delivery to clarify things.

Not everyone needs to be talking to customers

The idea that we should involve everyone in everything is appealing when we don't know how to collaborate. It's like pulling everyone into a meeting. Nobody knew how to break the problem apart, so now twenty people are sitting around the table. Putting everybody in one room is the lowest denominator of collaboration. Same with asking everyone to take part in the same work.

Guide to determine the first version of a new product or feature at Eventbrite

Experiment: An Experiment is an investigation into the casual relationship between two things. In some cases, you can ship an experiment without actually delivering value to customers.

MVP: An MVP is all about delivering value to users by building the smallest product you can to test a hypothesis. You ship to learn, which influences future product development.

MVF (Minimum Viable Feature): Similar to an MVP but instead of testing strategy-related assumptions on a product level, an MVF tests solution-specific ideas on top of an existing product.

Phased Development : You know exactly what you need to build, and you break that down into smaller phases over longer sprints.

Who is Tim Herbig?

As a Product Management Coach, I guide Product Teams to measure the real progress of their evidence-informed decisions.

I focus on better practices to connect the dots of Product Strategy, Product OKRs, and Product Discovery.

Product Practice Newsletter

1 tip & 3 resources per week to improve your Strategy, OKRs, and Discovery practices in less than 5 minutes.

Read more from Product Practice Newsletter

Product Practice #354 Stop Looking at Flat Dashboards READ ON HERBIG.CO PUBLISHED Mar 29, 2025 READING TIME 4 min & 06 sec Dear Reader, Jeff Patton explained why flat backlogs don’t work for prioritizing user stories more than 16 years ago: Arranging user stories in the order you’ll build them doesn’t help me explain to others what the system does. Try handing your user story backlog to your stakeholders or users when they ask you the question “what does the system you’re building do?” I...

Product Practice #353 How Duolingo Approaches Strategy, OKRs, and Discovery READ ON HERBIG.CO PUBLISHED Mar 7, 2025 READING TIME 4 min & 24 sec Dear Reader, Many Product Managers were in awe of the ways of working shared in The Duolingo Handbook a few weeks ago. While it’s an inspiring read, I used this as a reason to revisit some of my all-time favorite reads about how this company operates (or at least used to operate) and extract my takeaways with you. Duolingo focuses on “Movable” Metrics...

Product Practice #352 My Digitale Leute Summit 2024 Keynote Recording and Slides READ ON HERBIG.CO PUBLISHED Feb 28, 2025 READING TIME 1 min & 40 sec Dear Reader, I'm excited to share the full recording of my talk on How Product Teams Can Connect the Dots of Strategy, OKRs, and Discovery from last year's Digitale Leute Summit. You can think of it as the naturally progressing chapter (hint hint) that would follow my talk from Product at Heart 2024. As always, I won't require you, as an...