14000+
38%
★4.8/5
alumni since 2020
come from colleagues’ recommendations
average review score
19000+
PMs in our community
trusted by:
Why product managers need to understand LLMs?
- LLMs are already inside the products you compete with. The question isn't whether to use them, but who does it better
- Shipping an AI feature is easy now. Proving it works is not, and that's what separates a PM who owns quality from one who hopes for the best
- Speaking the same language as your ML team saves weeks: you know what's genuinely hard and what takes a day
This course is for you if...
You want to stop guessing and start measuring
Learn to measure what you're currently eyeballing: build a dataset, define criteria, compare prompt and model versions.
You want to move into AI products
Get the experience interviewers actually ask about: not "I've used ChatGPT" but "I shipped an LLM feature and owned its quality."
You're the one who has to say no to bad AI ideas
Get the arguments and the numbers to back that call, instead of relying on gut feeling in front of stakeholders.
Before and after
PM without LLM knowledge
PM with LLM foundations
You can tell where an LLM helps and where it's a solution looking for a problem
You build evals: a dataset, clear criteria, a number you can track
You write prompts for a product feature and improve them from real results
You're not sure which tasks an LLM is actually good for
You have no way to check if the output is any good
An LLM sits apart from the rest of your product
You measure how often they happen and catch them before users do
You write prompts the waynyou'd chat with ChatGPT for personal use
Hallucinations are something you hope won't happen
You pick a model by reading the vendor's page
You compare models on accuracy, speed and what an error costs you
You get output in a format other services can read and use
Learn By Doing
Hands-on Approach
No boring videos or magical frameworks — learn through immersive, practical tasks.
Real Use Cases
Roll up your sleeves and solve product puzzles from top tech companies.
PM Job Simulator
Interact with colleagues and stakeholders, just like in a real job!
Self-Paced Learning
No deadlines, no course agenda — study whenever you want.
Student reviews
"Yes, I have understood a lot more: what tasks can be solved using ML, what parameters need to be considered when building a model, and other aspects. Thank you!"
"The course provides a basic understanding of how ML works in products, for which I am grateful. The main takeaway for me is that it helps to understand when and for which tasks it can be effectively applied. Thank you!"
"Excellent examples and, as always, there are tricky pitfalls! I would love to have more assignments where we need to dig into the data."
"Great course, it provides a very good basic understanding of ML, its evaluation, and usage! A wholehearted 5!"
"Thank you, a very useful course. I liked reinforcing theory with mandatory practice and the real-life examples. It makes you start looking at the products you use every day in a different way."
"It was interesting and informative. I enjoyed the course. I decided to take this course to be better prepared for the 'ML for Products' course."
What does the simulator look like?
Meet the Authors
- Vladimir KalmykovLead Product Manager Platform
Booking.com Netherlands- PhD in Computer Science
- 8 years in Product Management
- One of the first Tech PMs at Booking.com
- Leads a department as a manager of PMs
- Trained 2,000+ Tech PMs on various edu-platforms
- Andrew MendeCurrent Senior Machine Learning PM
Booking.com NetherlandsUses ML to create better recommendation systems at Booking.com.- 13+ years in Product Management
- Launched products in Yandex (search verticals)
- Trained 1,000+ PMs on various edu-platforms
- Master of Business Administration (MBA)
Course program
Across 8 lessons you'll go through every stage of building an LLM product, measuring quality at each one: from the accuracy of your first prompt to dialog evals and safety checks. Real cases throughout: support ticket classification, catalogue search, a pipeline with routing, and an AI stylist.
Everyone talks about LLM magic, but it all starts with a prompt. This lesson is about talking to a model so it solves a business problem instead of just making conversation.
In this lesson:
→ What makes a prompt good, and why it's not "just text"
→ What temperature is and how to control it
→ Why the same prompt gives different answers, and what to do about it
→ How to check whether the model handles the task: your first eval
What it looks like:
→ Write, test and improve a prompt for a specific task
→ Case: replacing manual support ticket tagging with AI
In this lesson:
→ What makes a prompt good, and why it's not "just text"
→ What temperature is and how to control it
→ Why the same prompt gives different answers, and what to do about it
→ How to check whether the model handles the task: your first eval
What it looks like:
→ Write, test and improve a prompt for a specific task
→ Case: replacing manual support ticket tagging with AI
A prompt that works is only the start. Next comes the actual work: finding where the model fails and pushing accuracy to a level you'd ship.
In this lesson:
→ Few-shot: how examples in the prompt lift accuracy
→ How to read eval results and find the categories the model gets wrong
→ Why narrowing the task often beats writing a longer prompt
→ How to decide what to measure when one category matters most to the business
What it looks like:
→ Improve the prompt step by step and watch what happens to accuracy
→ Case: getting ticket classification to a level you could launch
In this lesson:
→ Few-shot: how examples in the prompt lift accuracy
→ How to read eval results and find the categories the model gets wrong
→ Why narrowing the task often beats writing a longer prompt
→ How to decide what to measure when one category matters most to the business
What it looks like:
→ Improve the prompt step by step and watch what happens to accuracy
→ Case: getting ticket classification to a level you could launch
LLMs are good at open conversation, but they can also answer in a strict, predictable format. That's what turns a model from a chatbot into a piece of your product other services can work with.
In this lesson:
→ Why products need structured output and where it matters
→ How to write a JSON schema the model will follow
→ Why good field descriptions and examples beat a longer prompt
→ How domain expertise turns into instructions for an LLM
What it looks like:
→ Write a schema, run it on real queries, fix what doesn't line up
→ Case: search that turns a user's question into catalogue filters
In this lesson:
→ Why products need structured output and where it matters
→ How to write a JSON schema the model will follow
→ Why good field descriptions and examples beat a longer prompt
→ How domain expertise turns into instructions for an LLM
What it looks like:
→ Write a schema, run it on real queries, fix what doesn't line up
→ Case: search that turns a user's question into catalogue filters
There are a lot of models and they're all different. This lesson is about choosing between them with numbers instead of marketing pages, and knowing upfront what a feature will cost.
In this lesson:
→ What you're actually paying for: tokens, input and output
→ How instruct models differ from reasoning models, and when you need the latter
→ Why a model's size and its type are two separate things
→ When self-hosting an open-source model makes sense, and when it costs more than you think
What it looks like:
→ Work out what a feature costs and compare models on the same dataset
→ Case: picking a model for ticket classification, weighing accuracy against the cost of an error
In this lesson:
→ What you're actually paying for: tokens, input and output
→ How instruct models differ from reasoning models, and when you need the latter
→ Why a model's size and its type are two separate things
→ When self-hosting an open-source model makes sense, and when it costs more than you think
What it looks like:
→ Work out what a feature costs and compare models on the same dataset
→ Case: picking a model for ticket classification, weighing accuracy against the cost of an error
One big prompt or a chain of small ones? This lesson is about sorting out a messy stream of incoming messages, and why splitting a task into steps usually beats writing longer instructions.
In this lesson:
→ When a single prompt stops being enough
→ What routing is and how it works in a product
→ Why every step in a pipeline needs its own eval
→ How an explicit pipeline differs from a reasoning model
What it looks like:
→ Build a pipeline from several prompts and check each step separately
→ Case: one assistant handling both support questions and product search
In this lesson:
→ When a single prompt stops being enough
→ What routing is and how it works in a product
→ Why every step in a pipeline needs its own eval
→ How an explicit pipeline differs from a reasoning model
What it looks like:
→ Build a pipeline from several prompts and check each step separately
→ Case: one assistant handling both support questions and product search
Until now, evals compared the model's answer to a label. Here there's nothing to compare against: one request can have many correct answers. This lesson is about measuring quality when there's no single right answer.
In this lesson:
→ How to write quality criteria people won't read differently
→ What LLM-as-judge is and when you can let a model do the scoring
→ How to check whether the judge is right often enough to rely on
→ What data drift is, and why a good eval stops being good over time
What it looks like:
→ Write criteria, run the eval, and check it against human labels
→ Case: scoring the AI stylist's dialogs and taking the check into production
In this lesson:
→ How to write quality criteria people won't read differently
→ What LLM-as-judge is and when you can let a model do the scoring
→ How to check whether the judge is right often enough to rely on
→ What data drift is, and why a good eval stops being good over time
What it looks like:
→ Write criteria, run the eval, and check it against human labels
→ Case: scoring the AI stylist's dialogs and taking the check into production
An LLM product talks to users freely, and that's its strength. But the same freedom means it can respond to a provocation, leak its instructions, or wander into topics where your company has no business giving advice.
In this lesson:
→ Where the lines are: off-limits topics, grey areas, and requests that look risky but aren't
→ How teams actually defend: from keyword checks to a dedicated pipeline step
→ Why safety testing needs two different datasets
→ Why the bar here is higher than for quality
What it looks like:
→ Tune the assistant to decline gracefully without losing accuracy on normal requests
→ Case: locking down the AI stylist against provocations and prompt extraction
In this lesson:
→ Where the lines are: off-limits topics, grey areas, and requests that look risky but aren't
→ How teams actually defend: from keyword checks to a dedicated pipeline step
→ Why safety testing needs two different datasets
→ Why the bar here is higher than for quality
What it looks like:
→ Tune the assistant to decline gracefully without losing accuracy on normal requests
→ Case: locking down the AI stylist against provocations and prompt extraction
The course ends here, but a few things stay with you. We pull it all together and hand over what you'll actually use afterwards.
In this lesson:
→ A PM's checklist: situations, a glossary, and the checks to run before launching an LLM feature. Lives in Notion, duplicates to your workspace in one click
→ A LinkedIn-ready certificate
→ A discount on other ProductDo simulators
→ What to learn next so AI isn't your only strong skill
What it looks like:
→ Look back at the whole path: from a first prompt to evals and safety
→ Grab the checklist and decide where to go next
In this lesson:
→ A PM's checklist: situations, a glossary, and the checks to run before launching an LLM feature. Lives in Notion, duplicates to your workspace in one click
→ A LinkedIn-ready certificate
→ A discount on other ProductDo simulators
→ What to learn next so AI isn't your only strong skill
What it looks like:
→ Look back at the whole path: from a first prompt to evals and safety
→ Grab the checklist and decide where to go next
Be the first to know
when enrollment opens
when enrollment opens
Leave your email and we'll notify you before public launch — plus early bird pricing.
Our Students Work at
Still not sure?
Frequently Asked Questions
Highlight business impact: Explain how AI-prototyping knowledge will help you build smarter products that solve real customer problems and drive growth.
Emphasize upskilling ROI: Show that investing in your skills now means faster delivery, better decisions, and fewer costly mistakes down the road.
Connect to company goals: Align the course benefits with your company’s strategy, whether it’s innovation, efficiency, or entering new markets.
Stress team advantage: Point out how your new AI-prototyping skills will uplift your whole team’s expertise and collaboration.
Assure minimal downtime: Reassure your employer the course is hands-on, flexible, and won’t disrupt your current workload.
Simplify payment: Let them know we can issue an invoice if needed, making the process easy for company reimbursement.
Contact Sales
Emphasize upskilling ROI: Show that investing in your skills now means faster delivery, better decisions, and fewer costly mistakes down the road.
Connect to company goals: Align the course benefits with your company’s strategy, whether it’s innovation, efficiency, or entering new markets.
Stress team advantage: Point out how your new AI-prototyping skills will uplift your whole team’s expertise and collaboration.
Assure minimal downtime: Reassure your employer the course is hands-on, flexible, and won’t disrupt your current workload.
Simplify payment: Let them know we can issue an invoice if needed, making the process easy for company reimbursement.
Contact Sales
The cases are really real and reflect the diversity of products in the industry: not everyone has a "website" or a "store". In the "Fundamentals" course, we cover the clothes rental start-up; in the "Tech for PM" simulator, we will look at a taxi ordering app (like Uber), a music service (like Spotify), and a food delivery app. In the simulator "A/B testing" we will dive into experiments on the Meditation service, and in "Product Discovery" we will analyze the business of renting clothes.
The more cases you solve (and not just read/watch), the more experienced a product manager you become. There is no other way.
The more cases you solve (and not just read/watch), the more experienced a product manager you become. There is no other way.
One of the distinctive features of ProductDo is that all our courses are packaged in a simulator format. We can say that this is an important element of our product vision. The simulator as a learning tool has many wonderful properties:
- it requires constant active participation of the student. In order to move forward, you need to choose some action or answer a question every time.
- it allows you to give detailed feedback on any action: You can justify the preferred option with links and examples
- it allows you to move at your own pace and does not burden those who do not really like to work in a group with unnecessary communication.
Usually, they teach like this: theory → practice. If you have mastered the theory well, then you do the practice 100% or so. We have a completely different methodology. We start with a practical task that needs to be solved (and if the course suits you, then you don't know how to do it yet). Only when you have tried to solve the problem, we bring up some theory behind it. We believe that the cycle: trying to think independently → making a decision → feedback on a decision is a very effective way to master new material, especially for those who have already reached some professional level.
If you want to try the simulator format, take a free lesson. It contains questions from different areas of product management. At the same time, you can evaluate which skills you already have enough pumped, and which can be further enhanced.
- it requires constant active participation of the student. In order to move forward, you need to choose some action or answer a question every time.
- it allows you to give detailed feedback on any action: You can justify the preferred option with links and examples
- it allows you to move at your own pace and does not burden those who do not really like to work in a group with unnecessary communication.
Usually, they teach like this: theory → practice. If you have mastered the theory well, then you do the practice 100% or so. We have a completely different methodology. We start with a practical task that needs to be solved (and if the course suits you, then you don't know how to do it yet). Only when you have tried to solve the problem, we bring up some theory behind it. We believe that the cycle: trying to think independently → making a decision → feedback on a decision is a very effective way to master new material, especially for those who have already reached some professional level.
If you want to try the simulator format, take a free lesson. It contains questions from different areas of product management. At the same time, you can evaluate which skills you already have enough pumped, and which can be further enhanced.
We help by giving you a bunch of practical tasks and sharing our mistakes and experiences. We have already built so many products that with our examples we can help you formulate your area (more technical or more payment, related to ML / AI, or sales, etc.). We believe that a good PM chooses their favorite product themselves and does not need to be led by the hand.
The entire history of the simulator with comments and answers is saved. Access to the materials remains forever.
Yes, students who have successfully passed the course may generate a certificate and proudly add it to their LinkedIn profile to Licenses & Certifications, or upload it to Facebook in the Schools section.
Yet do keep in mind that upon applying to a product company, certificates do not really increase the chances of being hired, at least in our practice. If a resume mentions a certain course, then this may raise questions about what the candidate really knows right now. Therefore, you need to solve problems and develop skills, and not chase certificates. Fortunately, our courses have enough practice.
Yet do keep in mind that upon applying to a product company, certificates do not really increase the chances of being hired, at least in our practice. If a resume mentions a certain course, then this may raise questions about what the candidate really knows right now. Therefore, you need to solve problems and develop skills, and not chase certificates. Fortunately, our courses have enough practice.
You can be sure of the materials' quality and our mentors' experience. We've been teaching for many years, and we have thousands of happy PMs. Feel free to take a look at the student reviews for verification.
If you passed less than 30% of tasks of the simulator you could refund your money in full.
If you passed less than 30% of tasks of the simulator you could refund your money in full.
- Alex from ProductDoWe hope we answered all of your questions.
Still have more? Contact us via any messenger plug-in at the bottom right, or drop us a line at support@productdo.io, and we'll be happy to assist you.