New to Agentic AI? Start here.

Learnings from a first AI build

Agentic AI is everywhere at the moment - it’s predicted that 50% of all service requests will be powered by agentic AI systems by 2030 (Gartner).

Wondering what it actually is, and how it compares to generative AI? Me too. I spent my first week on an AI project Googling orchestration and deterministic workflows, trying to wrap my head around all the new terminology. Throw in ever-changing technology and rapid turnaround times, there’s a lot to contend with for a product manager.

In short, while generative AI is AI that says something, agentic AI is AI that does something. Typically, it performs a task based on an instruction (e.g. ‘Write my Product Breaks article’). This process might be broken down into a series of tasks, bringing in multiple agents. Agent 1: ‘Research agentic AI for my Product Breaks article.’ Agent 2: ‘Write a first draft.’ Agent 3: ‘Make updates to the draft based on my feedback’.

Here’s what I wish I knew before I started with agentic AI.

1. It’s all about the workflow

Agentic AI can be a complex puzzle to communicate, especially when it’s still a work in progress. Mapping out an agentic workflow is a way to provide a central and up to date view of how the product is being built. This is key to the team and wider stakeholders’ understanding, particularly around where and how agents and humans will interact.

To get started, Andrew Ng’s AI course has a really helpful section on creating a workflow, which is very similar to a user journey but with AI-specific inputs. Ask yourself:

• What are the steps a human would take in the process, and how you would make updates if things weren’t right first time? How many steps would you need?

• What are the key pieces of information you need the LLM to extract – and where should they get them from?

• What agents are involved?

• What platforms are involved, and at which stage in the process?

2. Keep it simple

Once the process has been broken down and there’s a shared understanding of where agents can be used, roles need to be assigned to them.

To make agents the most effective, they need a singular, simple job to focus on. This might mean that one step in the process requires several of its own agents, but that way they can become better at their jobs and are easier to refine.

Tip: Giving agents unique names (e.g. Email Writing Agent) helps to keep track of and reference them.

3. You’ll need to manage expectations

A huge benefit of AI is that development is quicker, and prompts can be updated as and when to rapidly improve the output. The downside of this is it can result in high expectations around what’s possible. This is understandable given the narrative about AI, but not necessarily realistic.

Bringing in agentic AI likely means automating a process with multiple people, teams, platforms and information sources. As PM, a useful way to communicate this may be that it offers a chance to test out what can be automated, which keeps expectations in check.

4. AI is only as good as the data it can work with

This is another reason that expectations must be managed; AI needs high-quality data to produce high-quality output. In fact, this is one of the main barriers to AI adoption in general. There are two main elements to this:

Accessibility

A clear set of rules for storage and naming convention, so the LLM knows exactly where to look for information.

Mapping

A good understanding of where the information needs to go, how it will get there, and what requirements it should adhere to if managing an integration. This would also influence what data is collected in the first place, and in what format.

While it’s still possible to prove out the potential of AI without this, proving out the value is likely to be much more achievable with high quality data in place.

5. What is true today could change next week

AI capabilities and the technology powering them are rapidly changing – which is both a blessing and a curse. There are constantly new features available, but often these are still in beta or soon to be updated again. This might mean that workarounds are required to get a product built quicker, even if not fit for the long term.

In summary, working with agentic AI was a big learning curve as a PM, but a lot of the experience itself wasn’t new. Being comfortable with constant change and the ability to explain complexity are fundamentals from product management which will serve us well.