The Lost Art of Sitting in a Room with Your Users
Why the oldest tool in product research matters more in the age of AI agents
This is the first installment of The AI PM Playbook, a series on building products in the age of agents. I've been a product manager for over a decade. These are the lessons that took me years to learn and that I wish someone had written down for me.
Here is a confession that will make me sound like a Luddite: the most valuable product insight I ever received came from watching a 58-year-old woman hold a phone.
Not from a dashboard. Not from an A/B test. Not from a Mixpanel funnel. From sitting in a windowless room in Shenzhen, behind a one-way mirror, watching her try to find the camera app.
She tapped the screen with her index finger, just one, like she was pressing a doorbell. She squinted. She rotated the phone 90 degrees, then back. When the moderator asked her what she thought of the 200-megapixel camera, she said: “I don’t know what that means. I just want the photos to look like what I see.”
That single sentence killed a feature that three teams had spent months building. And it saved us from shipping a product that solved a problem nobody had.
This is a story about focus groups, the oldest, least glamorous, most unfashionable tool in the product manager’s toolkit. And about why, paradoxically, they matter more now than they have in twenty years.
The Paradox of the Agent Era
We are living through the most capability-rich moment in the history of software. AI agents can now browse the web, write code, query databases, draft reports, and chain these actions together in multi-step workflows. The gap between “what is technically possible” and “what we can ship” has never been wider.
And that gap is precisely the problem.
When capability is cheap, taste becomes expensive. When you can build almost anything, the hardest question is no longer how but what, and more importantly, for whom. I wrote previously about how great products don’t change user behavior; they embed into existing muscle memory. WhatsApp didn’t teach people a new way to communicate. It digitized what they were already doing. The products that fail are the ones that ask users to develop new habits for the privilege of using them.
In the agent era, this principle is even more critical. Every founder I talk to right now is building some variation of “AI that does X for you.” The technical capability is real. But the user understanding is often paper-thin. They are building agents that automate workflows they’ve never actually watched a human perform.
This is where focus groups come in. Not as a relic from the Mad Men era, but as the specific antidote to the specific disease of 2026: building powerful tools for imaginary users.
Why Online Surveys Are Lying to You
I can already hear the objection. “We do user research. We run surveys. We have data.”
Let me tell you what your online survey data actually contains: noise. A tremendous, statistically impressive quantity of noise.
Here is what happens when you send a survey to 500 people. Some of them are multitasking. Some are answering on the toilet. A meaningful percentage are satisficing, clicking whatever answer gets them to the end fastest. The ones who do engage thoughtfully are self-selecting in ways that systematically bias your sample.
But the deeper problem isn’t data quality. It’s data type. A survey can tell you what people say they want. It cannot tell you why they want it. It cannot capture the moment of hesitation, the furrowed brow, the half-sentence they start and then abandon. It cannot show you the 58-year-old woman pressing the screen like a doorbell.
Surveys are closed-loop instruments. You define the questions, you constrain the answers, and you get back a reflection of your own assumptions. If you’re a founder trying to find your blind spots, a survey is like looking for your car keys under the streetlight because that’s where the light is.
A focus group is the opposite. It is an open-loop instrument. You put four or five real humans in a room, you ask them a question, and then you shut up. You let the conversation go somewhere you didn’t expect. You watch their faces. You hear the specific language they use to describe their problems, language that is almost never the language you use internally.
This is the part that cannot be automated and cannot be surveyed. And it is the part that separates products that ship features from products that solve problems.
How to Assemble the Room: Selecting Your Focus Groups
The rest of this article, including my exact framework for selecting focus group participants, the questioning technique I developed, and the feature prioritization method that consistently surfaces your product’s “wow factor”, is available to paid subscribers.
This is the first piece in The AI PM Playbook series. Upcoming installments will cover pricing research, competitive teardowns, and how to write product specs in the age of agents.