Product Practice #308

Only these OKRs work for Migrations

READ ON

HERBIG.CO

PUBLISHED

Feb 8, 2024

READING TIME

4 min & 36 sec

Dear Reader,

I once worked with a product team whom I should help "improve their OKRs" during a technical migration. They were 12+ months from shipping the first iteration to a pilot user segment. And yet, their OKRs featured a multi-year time horizon, including metrics like NPS, Conversion Rate, Support Requests, and Revenue. Nobody seemed to care that these metrics had been unchanged for months and would continue to do so. All that mattered was having a "customer-centered" set of OKRs in place for this project.

This was more about OKR theater than utilizing OKRs to measure the real progress of a team. And yet, there were other options.

There's a valid case to be made that a solid Gantt chart and disciplined sprint reviews are practical tools for measuring the progress of a migration. There is no NEED for using OKRs in a scenario like this - And definitely not by using made-up metrics that try to capture the metrics that happen as a result of the migration weeks or months after it's done – these are simply too lagging to be used as decision-making tools.

Revisiting last week’s idea that useful team-level OKRs sit within a product team’s Sphere of Influence or Zone of Control, metrics about the results created by the migration do not help measure progress during the migration.

So, what’s left for product teams to influence or control during such a project? Typically, you look at metrics like velocity changes, met sprint commitments, requirements completions, user acceptance tests, staging performance, dummy data processing, etc.

For example, idealo published a blog post about their recent 2-year cloud migration project — Including the OKRs they had set for the effort. Let’s review their KRs:

KR1: We migrate 100% of all applications into AWS until the 1.10.23.

This works as a high-level KR for the entire project. It’s essentially the trackable progress of the project. Maybe smaller chunks or contributing metrics could be more beneficial for quarterly-ish durations.

KR2: The average time to recover (TTR) for AWS-related incidents levels 1 & 2 stays below 4h until the 1.12.23.

From what I can tell, this metric can only be measured after the migration is completed, making it less useful for measuring the progress of the migration itself.

KR3: idealo is staying within the allocated yearly AWS budget.

This reads more like an Objective than a KR since it lacks a specific measurement. I could see a rephrased KR read something like this: “x% of AWS operating weeks are costing below x€/team.” However, the nature of the KR looks semi-trackable throughout the migration. You have to complete a partial AWS migration before you can start to measure the costs. But since I imagine the transition will happen gradually, you can probably begin to measure it before the project's completion. But it reads more effect-like, not progress-like.

Note: I have no inside information about this project beyond the blog post, and this is a pure armchair critique with no intention at all to downplay the accomplishments of Idealo’s product teams. Kudos to them for nailing this migration.

There might always be voices in your company that become scared when they see non-Outcome OKRs being used. But guess what? OKRs are, first and foremost, about measuring progress, however that looks like. And, if pragmatic OKRs for a migration feel like such a bad idea to your company, then maybe month/year-long migrations or the forcing of OKRs onto these projects aren’t a good idea in the first place.

HOW TO PUT THIS THEORY INTO PRACTICE

Clarify the expected results of migration projects These are good indicators of how well these efforts are strategically aligned and can help you set post-launch OKRs.
Ditch lagging indicators for migration OKRs Focus on metrics about your team's way of working, business constraints, stakeholder interactions, or product quality: Anything that responds to your actions.
Connect OKRs and Task Management Link where the work happens (i.e., JIRA) to what you're trying to achieve to guide your decision-making (i.e., during Sprint Plannings).

Thank you for Practicing Product,

Tim

How to Dive Deeper into Product OKRs

Learn the strategies and tactics you need to use OKRs in a way that helps product teams prioritize work based on user problems and business goals—instead of replicating existing feature backlogs.

My Outcome OKRs for Product Teams Course enables product teams to use OKRs as a tool for decision-making in the context of Product Strategy, Product Discovery, and Scrum. Without the fluff, but with a focus on practicality in everyday work.

Learn more about my OKRs Course

Content I found Practical This Week

Metrics that cannot even be measured in retrospect

As covered in another article, customer churn severely limits growth and, more importantly, is a lagging indicator that customers don’t want what you’re selling. Therefore, it’s almost always a good use of time to uncover the main drivers of churn, and address them with product, process, people, price, and other words that start with “p.” So the common and very reasonable question is: If we do X, how much will churn go down? And while we’re actually doing X, how much is churn going down? And this leads to the crux: A year from now, will you then know the impact X had on churn? Unless X has an enormous and immediate impact, the answer is no.

Read it here

I thought OKRs were overkill for small scale teams. I was Wrong.

Defining our goals for the next quarter is still challenging, not as much as in a 100-person company, but it still demands discipline and thoughtful decision-making within the leadership team. The effort is undoubtedly worthwhile. We are now in the process of defining our quarterly OKRs for the third time. Focusing our energy on alignment over a few weeks makes execution much smoother during the quarter. We’re delivering more with less effort, and it is much easier to decide if a new initiative is worth pursuing through the lens of our quarterly objectives.

Read it here

Measuring What Doesn't Matter

In all but the smallest startups, these are team efforts, not the work of one person. Product managers, designers and other roles collaborate with software engineering to define, design and deploy solutions to these problems. The ideas may be the initial focus but it’s the effect they have on our users that tells us if we’ve built something good. These collaborations, debates and decisions rarely come from one person. Instead these type of OKRs should encompass the entire team working on this problem and ensure success is measured in terms of human behavior.

Read it here