The Side of Evidence-Based Working Nobody Trains

​Dear Reader,​

Whenever I have the privilege to work with a company, the shift to more evidence-based decision-making is often at the core of the Outcome they want to achieve.

And while helping teams change the way they work is crucial for that, the biggest shift is not just on the side of individual contributors.

You see, Charlie Munger once said this very smart thing:

So, if you want to achieve an outcome of evidence-based ways of working, which incentive would have to change? Financial incentives aside, individuals are incentivized by the approval and direction of leadership roles. So, depending on what leadership asks the teams about, that's what gets optimized.

Asking about the certainty and completion of an idea will lead to more time spent on upfront details.

Asking about release dates will lead to more focus on fast shipping.

None of these defaults are bad per se. They just come with different consequences. Of course, if all of these consequences don't align with the outcome of evidence-based ways of working....you have a problem.

Teams trained in evidence-based methods returned to systems where the incentives have not shifted towards using this evidence. They learn that evidence is what you produce after a decision has been made. You can call it evidence-laundering. The team did internalize the methods, but the org just taught them what evidence is actually for here.

I think the solution is simple (but not necessarily easy): You have to change the defaults:

Instead of asking "How long will it take to build"? change to "What evidence suggests to you that this is a good idea?"

→ The incentive becomes to provide evidence and a narrative, not just to execute as fast as possible.

Instead of asking "Which metrics should we put into our OKRs?" change to "What do we want to do differently and how will we measure it worked?"

→ The incentive becomes to make a strategic choice and think about measuring it, not just to fill in values into a template.

Instead of asking "Do we have all the details specified?" change to "Which leading indicators will we look to for early feedback on direction and correction?"

→ The incentive becomes to course correct based on real-life feedback, not to achieve academic correction before getting started.

There is a reasonable objection to the question shifts above: they look obvious. Why don't more exec teams already ask them?

Because the question is often asked rhetorically, reversing a decision because the answer points away from where you wanted to go is the harder part. This discipline carries a cost that most companies punish, even when nobody admits to it. Which is why the question often gets asked, but the answer rarely changes the decision.

And changing defaults applies equally to teams and leadership:

• Leadership has to shift the questions they use to start conversations during roadmap reviews or 1:1s.
• Teams have to shift the dogmatic defaults they use for approaching problem-solving and execution.

If you want a quick test for which kind of org you are in, ask two questions.

1. When did an exec here last reverse a decision they had publicly championed because a team brought conflicting evidence?
2. What happened to that exec's standing in the room afterward?

If you cannot recall an example, the org is laundering evidence. If you can recall one, and the exec's standing dropped, the system actively punishes evidence-based work. If you can recall one and their standing held or rose, the org has it.

Until the room rewards people for changing their minds with a firm narrative, simply training the skills will continue to produce evidence-shaped artifacts.

Thank you for Practicing Product,

​Tim​

If you consume one thing this week, make it this...

The Art of Product Management in the Age of AI (Compared to 10 Years ago) Read it here

Who is Tim Herbig?

As a Product Management Coach, I guide Product Teams to measure the real progress of their evidence-informed decisions.

I focus on better practices to connect the dots of Product Strategy, Product OKRs, and Product Discovery.