How it feels being Interviewed by AI

​Dear Reader,​

​Last week, I invited you to join an AI-led concept test of a side project idea. Here's what participants share with me about their experience:

"It was like getting interviewed by someone who is doing this one of the first times."

"I felt free to say what I wanted without 'hurting' the interviewers. I felt listened (as the AI was repeating my points). But sometimes the conversation was cut off weirdly."

"The AI often interrupted me when I was thinking. In addition, it often started to respond, realized I wasn't finished, and then took a completely different approach, forgetting what it had said before. After two or three interruptions, I naturally lost the desire to speak naturally and fell into a telephone agent's way of speaking."

So, what can you make of this? Obviously, this is an n=3 sample, so it's not a definitive judgment on AI interviewers' capabilities. So, with a grain of salt, here's what I'm observing:

1. The AI ignores basic principles, such as rotating the starting variant, which impacts the feedback you'll get. Not helpful (even borderline harmful for your insights).
2. It's seemingly easier for people to drop out of AI-led interviews because there's no friction. Closing a browser tab is less uncomfortable than leaving a room.
3. Getting summaries with specific playback markers from feedback shared brings insights alive - but you don't need AI-led interviews for that.

The main caveat I shared last week about being intentional, where you insert AI into your discovery practices for what purposes, remains true:

As a result, here's how I would summarize the current state of AI tooling in the context of Product Discovery:

Oh, and in case you're curious: Here's the live link to the (messy and work-in-progress) prototype I used for concept testing: Pour Over Diary - A Platform for Specialty Coffee Enthusiasts.

Thank you for Practicing Product,

​Tim​

Get my Book

⭐️⭐️⭐️⭐️⭐️ "What makes Real Progress particularly special to me is that it’s not just a source of inspiration or a collection of ideas, but a practical guide you can return to again and again – no matter what area you’re currently focusing on." ​ Get your Copy Today

If you consume one thing this week, make it this...

Who is Tim Herbig?

As a Product Management Coach, I guide Product Teams to measure the real progress of their evidence-informed decisions.

I focus on better practices to connect the dots of Product Strategy, Product OKRs, and Product Discovery.