AI Product manager SERIES

Key Concepts of AI Agents for Product Managers

Here, we will break down the key concepts of modern AI: LLMs/models, tools/MCP, harnesses, and agents.

LLM Model

An LLM (Large Language Model) is an ML model that predicts the next word in a text. The most obvious example is the next-word predictor on your smartphone. You type in a text: "I'm going to..." and the model offers three options: "the store", "school", "the shower" — the three most probable continuations of that phrase. The model doesn't know for sure; it's just statistics. If you keep clicking the suggested options, it quickly gets hilarious when complete nonsense comes out :)

Modern models are trained in such a way that if you give them a question, they will start predicting the answer. The main thing to remember is that an LLM isn't a universal mind, but simply a next-word predictor.

LLM models on their own aren't overly useful because they can only operate with text: you can ask for a pancake recipe or write a fairly predictable (pun intended) blog post.

But everything changed when we learned to give the model tools to interact with the world around it: tools.

Tools

Tools allow the model to call an external service that enables it to retrieve information or perform an action. Typical examples of tools are:

search Jira for tickets by text
read a Jira ticket
edit a Jira ticket

When we call an LLM, in addition to the task formulated by the user, we pass it a list of available tools along with descriptions of how to invoke them. The LLM can answer the user directly (if it already knows the pancake recipe), or it can choose to call one of the tools (for instance, web search) instead.

Let's look at an example:

A product manager enters the prompt:
"Add a condition to the new discounts launch task stating that discounts cannot be combined with promo codes."

Tools available to the LLM:
jira-search-issues, jira-read-issue, jira-update-issu
*an issue is what tickets are called in Jira terminology

What happens under the hood:
The LLM decides: call the tool "jira-search-issues" with the search parameter "discounts"

Note that several events occurred here: 3 magical ones and one ordinary.

The first magical part is that the LLM, on its own, guessed that "add to the task" means it needs to jump into the task management system (Jira).
Furthermore, to add something somewhere, it first needs to find the relevant ticket, so it once again showed its genius and picked the right tool (jira-search-issues).
Finally, it figured out that if a product manager asks for something related to discounts, it makes sense to search for a ticket using the keyword "discounts."
All of this culminates in an absolutely non-magical tool call, similar to a standard API call.

MCP Server

So, where exactly do tools come from? How do they work? This is where the MCP (Model Context Protocol) comes in handy.

An MCP server implements a set of tools that allows an LLM to interact with a specific service. For example, an MCP for Gmail will most likely have tools to retrieve a list of messages, search messages by text, read the full body of an email, and perform certain actions on it (such as marking it as read).

MCP is very similar to an API, but there are two key differences:

MCP allows the model to discover which tools it has, complete with detailed text descriptions of all possible actions and their parameters, so the model knows how to use them.
A good MCP is structured in a human-readable format: it returns information in roughly the same way it would be organized in a UI, and enables the kind of actions a person would typically perform when working with that service.

Below is an example description of a Jira MCP with a single search tool (a real-world Jira MCP would have dozens of them).

{
"name": "jira_search_issues",
"description": "Search Jira issues using JQL.",
"inputSchema": {
     "type": "object",
     "properties": {
         "jql": {
             "type": "string", 
             "description": "JQL query, e.g. 'project = SEARCH AND status = \"In Progress\" ORDER BY   updated DESC'" 
        },
        "max_results": {
            "type": "integer",
            "description": "Maximum number of issues to return (default 25, max 100).",
            "default": 25 
        }
    }
}

It looks intimidating, but it's actually quite logical: it describes the jira_search_issues tool, which takes two input parameters: what to search for (either in JQL format or as a simple search string like "discounts") and how many results to return (defaulting to 25). Everything is described in text, and this is precisely where part of the magic happens: the LLM can read the description and understand whether this tool will be useful and how to call it.

What does the tool's actual code look like under the hood? It's just ordinary code that parses the parameters and calls the Jira API, which has already existed for 20 years. The LLM doesn't see any of this—it simply waits patiently for a response.

So, who writes MCP servers? The creators of the services who want people to use their products (and get paid). In this case, Jira's developers wrote their own MCPs so that product managers could access their service not only through the classic interface, but also via Claude Code (you can check out examples of product managers working with Jira through an agent here).

The Harness

If the model is just an LLM that predicts the next word, and MCP is a toolbox, then what actually ties all of this together?

The third important element of the magic is the harness.

The harness executes the tool call exactly as requested by the model, gets the result, and calls the model once again—this time with an updated dialogue history that now includes both the tool call and its output.

In essence, the result of the tool call becomes just another message in the chat. We call the LLM again and ask it to think once more and predict the next action (remember, an LLM simply predicts the next word based on everything that came before it). It's a rather fragile construction, isn't it? That is precisely why LLMs sometimes hit dead ends or make up complete nonsense.

Below is the same example, but with a deeper understanding of what is going on:

Harness: passes the user's prompt to the LLM and reminds it of the available tools
LLM: decides to call the tool "jira-search-issues" with the search parameter "discounts"
Harness: calls the tool.
Tool: found one task: TSK-2133 "Add discounts for products".
Harness: passed this back to the LLM.
LLM: great, now add the required text from the prompt to this ticket. To do this, call another tool (jira-update-issue) for ticket 'TSK-2133' and add the text: "Discounts cannot be combined with promo codes."
Harness: calls the "jira-update-issue" tool and adds the required text
Tool: successfully updated TSK-2133
Harness: everything is ready.
LLM: now I can let the user know that it's all done.

Here are the most famous examples of harnesses that we use:

Claude Code – originally created for coding, but turned out to be so powerful that it is often applied to other tasks as well (here is how product managers use it)
Claude Cowork – a harness created by Anthropic specifically for personal and managerial tasks, similar to Claude Code in many ways. Under the hood, it calls the exact same LLM as Claude Code.
Open Claw – this harness is geared more towards autonomous operation and comes out of the box with connectors to email, instant messengers, and other personal user services.
LangChain – a framework for building your own agents from scratch. It allows you to manually assemble the exact harness you need for a specific task.

It is the harness that allows us to easily connect new MCPs through a UI or by adding a few lines to a config file. The harness implements a multitude of under-the-hood features that make solving tasks with the model much more seamless and effective. For example, harnesses quietly "compress" the oldest messages in a conversation, enabling us to have very long sessions with the LLM. Or they provide the ability to store session histories, allowing us to return to and resume any of them at any time.

Harnesses also implement a form of long-term memory by extracting key facts about us through summarization, or by letting the LLM search through past conversation logs to retrieve and use things the user has previously mentioned on the topic.

AI Agent

An AI agent is an LLM call in a loop that allows the model to work on a task autonomously. The LLM generates tool calls, receives results, and, based on what it gets in response, can make further calls or take the next logical steps.

For example, if the ticket search returns an empty result or an error, the LLM will likely try searching differently or look for synonyms for the word "discount." If it finds the relevant ticket, it will first read its text and then call the editing tool to make the required change.

Harness: passes the user's prompt to the LLM and reminds it of the available tools
LLM: decides to call the tool "jira-search-issues" with the search parameter "discounts"
Harness: calls the tool.
Tool: found one task: TSK-2133 "Add discounts for products".
Harness: passed this back to the LLM.
LLM: great, now add the required text from the prompt to this ticket. To do this, call another tool (jira-update-issue) for ticket 'TSK-2133' and add the text: "Discounts cannot be combined with promo codes."
Harness: calls the "jira-update-issue" tool and adds the required text
Tool: successfully updated TSK-2133
Harness: everything is ready.
LLM (called again in the loop): Would you like me to check what other price modifiers might conflict with discounts?

And so, the model will keep being called in a loop until it decides on its own to stop and return the result to the user, or ask them a follow-up question. It is precisely this loop that turns an LLM into an agent, making it autonomous from the user while working on a task. Yes, that's right—all that hype that's been around for the past year is simply calling an LLM in a loop. Apologies if we just ruined all the magic for you!

In reality, the "try in a loop until it works" idea is remarkably powerful. For instance, when a model is developing a new feature in code, it can run tests, see an error, attempt to fix it, run the tests again, and so on. It can even test a new feature through a browser by clicking through the new functionality and triggering a refinement process if it determines that something was implemented incorrectly. And all of this happens in a loop: code, test, code, test, code, test, done. That is precisely why LLMs are so good at coding straightforward, easily verifiable features.

To bring it to the full circle: what actually calls the model in a loop? The answer is the harness—so the "ask the model what to do next" loop is baked right into it.

Summary

So, we've learned that a few important things exist in relation to the agentic AI magic:

LLM is capable of predicting the next word, making it seem smart (like the brain)
Tools are tools available to the model to interact with other services. Tools are usually connected via an MCP server (like the toolbox)
Harness is the wiring that connects the brain to the real world (like the body and hands)
Agent is calling the model in a loop until a satisfactory solution is reached (like a sequence of thoughts and actions to reach a goal)

If you want to power up your use of AI agents to automate product routine, we’ve created a fully simulated startup for you: a real product with a codebase, real Jira, a database, and a messy document dump filled with PRDs and strategies (just like in real life). So you can practice in an environment that truly resembles your actual work.

You can check real reviews, the program, and start learning here.

Authors:
Vladimir Kalmykov, Group PM Booking.com
Andrew Mende, Sr. PM AI Booking.com