AI and Product Management - Get Stuck In
Tips for getting to grips with how AI will change the product manager role
Note: I use the term “AI” throughout this article, but I'm mainly referring to tools powered by Large Language Models (LLMs), often combined with other models and technologies.
AI is not just another productivity tool — it's fundamentally changing how we work. Across industries it is automating some tasks and enhancing human capabilities in others.
Product management is no exception. While the product role involves diverse, hard-to-codify activities, AI is already impacting them: from enabling us to deploy working prototypes in minutes to speeding up tasks like writing product requirement documents.
To understand how our jobs will change, we can’t just read and theorise – we need to learn by doing. By experimenting with using AI for different tasks in our daily lives, we can learn where it excels and where it falls short.
Professor Ethan Mollick invented some useful concepts in his academic research and book Co-Intelligence, which help to frame our exploration. Here I share three of Mollick’s key ideas and how we can use them to explore AI’s impact on the product manager role.
The Jagged Frontier: Understanding AI’s Capabilities and Limitations
In a 2023 paper, Mollick and his co-authors introduced the concept of the jagged frontier to describe AI’s uneven, unpredictable and ever-changing capabilities.
“Imagine a fortress wall, with some towers and battlements jutting out into the countryside, while others fold back towards the center of the castle. That wall is the capability of AI, and the further from the center, the harder the task. Everything inside the wall can be done by the AI, everything outside is hard for the AI to do. The problem is that the wall is invisible, so some tasks that might logically seem to be the same distance away from the center, and therefore equally difficult – say, writing a sonnet and an exactly 50 word poem – are actually on different sides of the wall. The AI is great at the sonnet, but, because of how it conceptualizes the world in tokens, rather than words, it consistently produces poems of more or less than 50 words.”
Ethan Mollick, Centaurs and Cyborgs on the Jagged Frontier
There are many examples of how the jagged frontier manifests when it comes to product tasks. Take market research, the act of understanding the market your product will be sold in to help refine your value proposition. Models that can search the web can be super helpful in quickly compiling a list of companies that offer products or services in a particular market. The list won’t be perfect, but it’s often a good start and will contain a few names that you wouldn’t have come across otherwise. Contrast this with asking a model for statistics on a market, like total annual revenue or market size, and it struggles. When you dig into its sources, you can find that it has fabricated the numbers altogether.
The more we use AI, the more we learn about where the jagged frontier is for our discipline. We need to be able to judge if a model is right or wrong, either by checking a model’s reasoning and sources, or applying our expertise. And we should also remember that the jagged frontier is always moving. What a model is bad at today, it may be brilliant at next week. The only way to keep up is through hands-on experimentation.
Centaurs and Cyborgs: Two Approaches to AI-Augmented Work
A concept that might help us in exploring the bounds of the jagged frontier is that of Cyborg and Centaur work. First introduced in the 2023 paper and expanded on in Mollick’s book Co-Intelligence, this is a way of describing different approaches to using AI in work.
Centaur work separates human and AI tasks, in the way that horse and human are clearly delineated in a centaur’s body. To do Centaur work, we first need to identify tasks within the jagged frontier that AI can do reliably well. We can then delegate those tasks to AI, while we humans complete the tasks that AI can’t do so well.
To take a product example, giving AI well-prompted summarisation tasks can save time when synthesising user research. While the model groups insights and articulates themes, you can focus on the key messages you need to deliver when presenting the insights back to stakeholders.
But a word of caution against delegating completely. Even though AI can group and summarise insights much faster than you can, it doesn’t mean you should neglect to read the insights yourself. It’s a key part of understanding your users’ needs and the insights on which you’ll base your strategy. It also helps you check that the model has got its summary right – this need doesn’t go away, even in Centaur work.
Cyborg work, on the other hand, sees a much closer mixing of human and AI tasks. In Mollick’s words, Cyborg work sees us “intertwine [our] efforts with AI, moving back and forth over the jagged frontier”.
This is where I believe there’s most value in using AI in our product roles now. For example, asking a model to come up with some ideas to inspire you if you’re feeling stuck, or using it as an editor when you need to improve the readability of a piece of text you’re writing (!)
Facilitating effective collaboration between different disciplines is a key part of the product role. Cyborg work raises an interesting question: what does the future of collaboration look like when we’re all collaborating individually with our own AI assistants? We’ll save that for a future PB article.
Thinking of different tasks as part of Centaur and Cyborg work helps us to track the changing strengths of AI and its role in our daily work. It’s a safe bet that tasks that are firmly in the Cyborg space today will graduate to Centaur territory – with AI taking more and more responsibility – in the coming months.
Treating AI as a Person Shapeshifter
In Co-Intelligence, Mollick advises us to treat AI as a person—not because he believes sentient AI beings are close, but because assuming this mindset helps us get better results.
Due to the language-based nature of large language models (LLMs), we get better outputs when we prompt them like they’re people, albeit shape-shifting people. The shape-shifting part of this is important: don’t be fooled into thinking AI has a consistent, reliable identity. Instead, you need to tell it who you need them to be and how you want it to act.
Here are a few ways to manage AI like a person through prompting:
Define its role: tell the model what kind of person you need it to be for a given task, whether that’s a data analyst, tech lead or your CEO
Give clear, structured instructions: break up your prompt into step-by-step instructions and use reasoning mode to help models take a metaphorical breath when parsing and responding to your prompt
Go big on feedback: refine the model’s output with clear, actionable feedback, as though feeding back to a colleague
When experimenting with AI to see how it can help with product tasks, treat it like a keen, inexperienced, shape-shifting person, who needs close guidance and management.
Change is the Only Constant
How the role of product manager will change due to the proliferation of AI is still emerging, but we should learn through active exploration.
Think about these concepts while you explore:
Make an effort to use AI in your daily work, to continually learn where the jagged frontier is and how its changing;
Reflect on the results you get, share what you learn with your wider team and community, and learn about new approaches from others;
Be cognisant of when you’re using AI for Centaur work vs. Cyborg work, and how that changes over time;
Prompt AI like a person to get the most useful outputs – but remember that its identity is not stable, and it needs to be managed closely.
And most importantly, repeat your exploration on the same tasks again and again. The capabilities of AI are changing every day; the frontier is constantly shifting. As Mollick says, “today's AI is the worst AI you will ever use”. Our knowledge and opinions will need to constantly evolve too.
Read more from Ethan Mollick on his blog, One Useful Thing.
Finding Product Breaks valuable? Please consider sharing it with friends - or subscribe if you haven’t already.
 |
|