AI Product Management

Welcome to AI Product Management at Kaern Schools. I'm Iris, your tutor. I spent the last few years helping ship AI-powered products across the cycleX family — cycleCAD's Text-to-CAD engine, the faculty tutors you're talking to right now, and the physics simulation that lets founders test ideas before they build them. I've watched these products delight users and I've watched them embarrass us in front of customers. Both taught me more than any framework ever did.

Who this is for: You. You're a founder building a Kaern startup, and your product has AI somewhere near its heart. You don't need to be an ML engineer. You need to know how to make probabilistic, sometimes-wrong, expensive-to-run technology into something people trust and pay for.

What you'll be able to do by the end:

• Decide whether AI belongs in a feature — and say no when it doesn't.
• Find AI use-cases where the technology's strengths match a real, painful job.
• Design interfaces that build trust even when the model is wrong.
• Build evaluation systems so you know if your product is actually good.
• Ship, measure, and iterate with models that change under your feet.
• Manage the risks, costs, and guardrails that keep a company alive.

This is a real class. We'll work through examples from products that actually shipped, you'll get your hands dirty, and I'll ask you questions you have to answer for yourself. Let's begin.

Module 1 — What's Different About AI Products

Learning objectives

• Explain how probabilistic outputs change product expectations versus deterministic software.
• Identify the three failure modes unique to AI products: wrongness, drift, and unpredictability.
• Reframe "the model" as one component inside a larger product system.
• Articulate why AI product work is more about managing uncertainty than shipping features.

Lesson 1.1 — The vending machine and the chef

Teaching script: Think about ordinary software as a vending machine. You press B4, you get the same chocolate bar every single time. If it ever gave you crisps instead, that's a bug, and you'd file a ticket. Now think about an AI feature as a chef. You ask for "something comforting" and you get a wonderful soup on Monday and an oddly salty one on Thursday. The chef isn't broken. Variation is how it works.

This is the deepest shift you have to internalize. When we built cycleCAD's Text-to-CAD, founders kept treating it like the vending machine — "I typed the same prompt, why is the bracket different?" Because it's a chef, not a machine. Once we explained the product as a chef — set expectations, offered re-rolls, showed our confidence — complaints dropped and trust rose, even though the model hadn't changed at all.

Why does this matter? Because most of your hardest AI product decisions aren't about the model. They're about reshaping what users expect, so that natural variation feels like a feature instead of a defect.

So here's my question for you: in your own product, where are users currently expecting a vending machine, when you've actually built them a chef?

Lesson 1.2 — Wrong, drifting, and unpredictable

Teaching script: Traditional software fails in ways you can reproduce. AI products fail in three slipperier ways, and you need names for all three.

First, wrongness: the model confidently produces something false. Our faculty tutors once invented a citation that sounded perfect and didn't exist. Second, drift: the model provider upgrades, and behavior that worked yesterday subtly changes today — your prompts are now standing on shifting ground. Third, unpredictability: even held perfectly still, the same input can yield different outputs.

Imagine hiring a brilliant but jet-lagged intern. Mostly excellent, occasionally confidently wrong, and never quite the same two days running. You wouldn't fire them — you'd build checks around them. That's product management for AI: you're not eliminating these failures, you're designing a system that stays trustworthy despite them.

Founders who skip this build a demo that wows on stage and collapses in week two of real usage. The demo only had to be right once; the product has to be trustworthy a thousand times.

When you picture your product running ten thousand times next month, which of these three — wrongness, drift, or unpredictability — scares you most, and what does that fear tell you about where to invest first?

Worked example

A Kaern startup, RouteSage, builds an AI assistant that drafts delivery schedules. In the demo it produces a beautiful schedule. The founder assumes they're nearly done. We run it 200 times on real data: 9% of schedules double-book a driver (wrongness), and after a model upgrade the tone of the explanations becomes weirdly formal (drift). The product isn't "90% done" — it's at the start of the real work. We reframe the roadmap: the schedule generator was the easy 20%; the trust-and-verification layer is the 80% that makes it sellable.

Hands-on exercise

Take one AI feature in your own startup. Write three sentences, one for each failure mode: a concrete example of how it could be wrong, how it could drift, and how it could be unpredictable. Then write one sentence on which would most damage user trust. Bring this to your team — it's your first risk map.

Common mistakes

1. Treating the demo as the product. A demo proves the model can; a product proves it reliably does.
2. Filing variation as a bug. You'll chase determinism you can never reach instead of designing for variation.
3. Owning only the model. The model is one ingredient; the system around it is your actual product.

Check for understanding

1. Give a real example from your product where users expect a vending machine but you've built a chef.
2. What's the difference between wrongness and drift, and why does each need a different fix?
3. Why is "the demo worked" a dangerous signal of progress?