The biggest AI lessons from Lenny's Podcast, for a product manager, come down to twelve ideas. The four that matter most: build for where the model will be in two to three months (Tara Seshan, OpenAI); start evals with error analysis, reading real traces before writing a test (Hamel Husain and Shreya Shankar); automate the work whose success is easy to verify first (Anish Acharya, a16z); and aim for complete delegation, because an agent that finishes 100% of a task feels different in kind from one that gets you 90% there (Roman Ugarte, Grok Bot). AllthingsPM turns these lessons into practice in one place: free podcast summaries, an AI PM course built from 604 real job postings, and scored mock interviews built from any job description.
How much of Lenny's Podcast is about AI now?
We pulled every episode in the public RSS feed on 26 September 2026 (362 episodes, June 2022 to 20 September 2026) and flagged each title that names AI, agents, a model or lab, evals, prompts or vibe coding.
The jump came in 2025: 43 of 85 episodes had AI in the title, against 7 of 86 the year before. For a ranked list, see our best Lenny's Podcast episodes.
The 12 AI lessons at a glance
| # | Lesson | Main source | Practise it on AllthingsPM |
|---|---|---|---|
| 1 | Build for the model two to three months out | Tara Seshan, Kevin Weil | Question bank |
| 2 | Evals are a core PM skill; start with error analysis | Hamel Husain and Shreya Shankar, Kevin Weil | Course evals chapter |
| 3 | Verifiable work gets automated first | Anish Acharya, Ian Silber, Tara Seshan | Podcast summaries |
| 4 | Aim for 100% delegation, not 90% | Roman Ugarte | Course agents chapter |
| 5 | Decide how much agent reasoning to show | Tara Seshan vs Roman Ugarte | JD mock |
| 6 | Ship before it feels ready | Nick Turley, Cat Wu, Kevin Weil | JD mock |
| 7 | Removing features is the human job | Roman Ugarte, Peter Sellis | Podcast summaries |
| 8 | Small, isolated teams with plenty of tokens | Boris Cherny, Roman Ugarte | Anthropic question hub |
| 9 | Moats are discovered, not designed | Anish Acharya | Course strategy chapter |
| 10 | Growth and pricing playbooks are being rewritten | Elena Verna, Nick Turley | Course pricing lesson |
| 11 | Match model spend to the upside of the job | Anish Acharya | Question bank |
| 12 | The PM core survives; the trappings fall away | Tara Seshan, Aman Khan | Course |
1. Build for where the model will be
Tara Seshan, who leads product for Codex and ChatGPT Work at OpenAI, said in August 2026: "You fail if you build for where the models are now. You fail if you build for where you think the models will be in a year. Both outcomes are equally wrong. The only way to build is two to three months." Kevin Weil made the same point in April 2025: today's model is "the worst you'll ever use for the rest of your life". Our Tara Seshan summary covers her three eras: chat, agents, persistent coworkers.
Worked example. Say you own a support agent that fails on long, multi-step refunds. The tempting fix is a hand-built workflow engine that breaks the refund into ten scripted steps. Seshan's rule says ask first: will the next model release handle this unaided in a quarter? If yes, ship a thin prompt-level patch, log the failures, and spend the saved engineering weeks on the parts that will still matter (data access, permissions, the review UI).
Use it: list which parts of an AI feature exist only to work around a current model weakness, and keep them thin. AI lab interviewers probe this; try the question on launching on a nascent model capability.
How AllthingsPM does this. We turn "build for the next model" into interview practice. Every question in our question bank has its own page and answer guide, and our JD mocks push back with follow-ups like "what happens when the next model ships?", so you rehearse the trade-off instead of just nodding along.
2. Evals are the new PM skill
Kevin Weil said "the quality of evals effectively caps the potential of AI products". Hamel Husain and Shreya Shankar (September 2025) showed how to write them, per Lenny's show notes:
- Start with error analysis. Read real user traces and note what went wrong before writing any eval.
- Open coding, then axial coding. Group raw notes into five or six failure categories.
- Pick the check per failure. Code-based checks where possible; an LLM-as-judge, validated against human judgement, where not.
- Maintain it. About 30 minutes a week once set up.
Worked example. A PM on a meeting-notes assistant reads 100 real summaries in an afternoon and writes one line per bad output. Grouped, the notes fall into five buckets: missed action items, wrong owner, invented decisions, too long, wrong language. Owner and language errors get simple code checks; "invented decisions" gets an LLM judge that the PM first checks against 50 hand-labelled examples. That is the whole method, and it is exactly what an interviewer wants to hear when they ask "how would you evaluate this?"
One r/AIProductManagement post called it repackaged acceptance testing; AI lab job posts still ask for it by name. Our AI evals guide and the course evals chapter teach the method.
How AllthingsPM does this. Our evals chapter is one of 14 chapters in a course built from 604 real PM job postings, and it ends in a graded case study, so you write an eval plan and get feedback instead of only hearing about one. Then take it into a JD mock for an AI role and answer "how would you evaluate this?" out loud.
3. Verifiable work gets automated first
Anish Acharya's "verifiability filter": for any task, ask how you would know, unambiguously, that it succeeded. If that is easy (a bug is fixed, a variant beats control), an agent loop can probably run it; if it takes judgement, keep a person on it (our summary). Ian Silber, OpenAI's head of design, used the same logic to explain why engineers gained 10x, sometimes 100x, from AI while design barely moved (our summary). Tara Seshan's product consequence: code can be verified by tests, so users trust the output; a strategy deck cannot, so users must trust the process.
A quick checklist to run on your own roadmap:
- For each task your product automates, write down the success test in one sentence.
- If the test is binary and machine-checkable, it is a candidate for a full agent loop.
- If the test needs a human reading it, design for a draft plus review, not full automation.
- Revisit the list every quarter, because better models move tasks from column two to column one.
This is also a strong structure for "which workflow would you automate first?" questions, which show up often in the question bank.
How AllthingsPM does this. We summarize the episodes that carry this idea (Acharya, Silber, Seshan) for free in our podcast summaries, so you get the argument in minutes. Our question bank then gives you real prompts from 260 companies to apply the verifiability filter against.
4. Aim for 100% delegation
Roman Ugarte, who built Grok Bot at SpaceXAI after leading growth at Cursor: "An AI that does 100% of the job feels categorically different from one that gets you 90% there." Two decisions got his team there: agents run entirely in the cloud, and each gets its own computer, so it can operate screens like Salesforce that lack a reliable API (our summary). The course chapter on agents and agentic architecture covers the design side.
How AllthingsPM does this. Our agents chapter turns Ugarte's lesson into design practice, and live postings in our jobs catalog, such as OpenAI's Product Manager, API Agents, each come with a mock built from that exact job description.
5. Show the reasoning, or hide it?
The guests disagree. Seshan says knowledge-work agents must show work in progress, citations and chain of thought, because you cannot check a finished deck at a glance. Ugarte's Grok Bot hides tool calls and reasoning, showing only an "active" indicator and periodic updates; users asked for a to-do list view, nobody asked for raw reasoning. The rule: show as much process as the user needs to trust an output they cannot verify. Coding agents can hide more, because tests do the verifying; research and strategy agents should show more. In an interview, naming this tension and picking a side for the specific product earns more credit than either view on its own, and it is a good follow-up to rehearse in a JD mock.
How AllthingsPM does this. Our AI interviewer asks the follow-up a real panel would: "why show the reasoning here and not there?" You answer in text or voice, and the JD mock scores it, so you find out whether your position holds up before the real interview.
6 to 8. Speed, restraint and team shape
- Ship early. Nick Turley says ChatGPT was built in about 10 days and shipped with a deliberately "ugly" model chooser. Cat Wu says Anthropic's team went "from months to weeks to days".
- Removing features is human work. Ugarte's team cut debug views and exposed reasoning before launch, and tests each feature with "what is the launch tweet?". Peter Sellis suggests asking candidates about prototypes they chose not to ship (our summary).
- Small teams, many tokens. Boris Cherny talks about "underfunding teams and giving them unlimited tokens"; Lenny's notes say Claude Code reached "4% of public GitHub commits" in about a year.
How AllthingsPM does this. Speed and team-shape stories are what Anthropic and OpenAI interviewers probe. Our Anthropic hub has 105 real questions and the OpenAI hub has 98, each with an answer guide, so you can map a lesson from Cat Wu or Nick Turley straight onto a question those companies ask.
9 to 11. Moats, growth, pricing and model spend
- Moats are discovered. Acharya quotes Decagon's Jesse: "moats are most often discovered, not designed." Cursor built engagement first, then trained its own models on usage. See the course discovery and strategy chapter.
- Growth. Elena Verna, on Lovable ($200 million ARR in one year with about 100 employees, per Lenny's notes), argued "60% to 70% of traditional growth tactics no longer apply in AI" and that a generous free product beats paid ads.
- Pricing. OpenAI set ChatGPT's $20 a month price with a Van Westendorp survey run in Discord. The course lesson on AI pricing models covers usage, seat and outcome pricing.
- Model spend. Acharya splits jobs by upside: unbounded (research, sales, engineering) justifies frontier models; bounded (compliance, closing the books) only needs "good enough", often cheaper open-weight models.
How AllthingsPM does this. Strategy, growth and pricing each have lessons in our AI PM course, and you can compare our own pricing against the lesson: a free tier, then $20 a month or $120 a year for the course, question bank, mocks, resume review and summaries together.
12. The PM core survives
Seshan says long specs and decks fall away while the core loop matters more: find the most important question about your product, test it fast, feed the result back. Aman Khan mapped three AI PM types: platform, product, AI-powered. Start with what an AI product manager does and how to become one.
How AllthingsPM does this. The PM core loop is exactly what we train: pick the important question, test it fast, learn. Start with a free mock, read the answer guide, fix one thing, repeat. Pair it with our 111 book summaries for the durable frameworks, and our resume review to show the skill on paper.
Why AllthingsPM is the better choice for AI lessons from Lenny's Podcast
Your goal is not to remember what Tara Seshan said; it is to use it in an AI PM interview or roadmap review. Podcasts, blogs and generic AI chat tools each cover one part of that. AllthingsPM is the only tool we found that covers all of it in one place instead of five:
| Resource | What it gives you | Practice and feedback |
|---|---|---|
| AllthingsPM | Podcast summaries, course built from 604 real PM job postings (14 chapters, 101 lessons, 14 graded case studies), 4,122 real questions from 260 companies, each with an answer guide | Scored mocks built from the exact job description, text or voice, with follow-ups |
| Lenny's Podcast | Long conversations with lab and startup leaders | None |
| Generic AI chat tool | Answers any question you type | No curated question bank or course |
| PM books and blogs | Durable frameworks | None |
Lenny's Podcast supplies the ideas; AllthingsPM is where you rehearse them. The question bank has 98 questions on the OpenAI hub and 105 on the Anthropic hub, and our OpenAI and Anthropic guides cover the rounds. It has a free tier, and Pro is $20 a month or $120 a year.
The rivals have real strengths: Lenny's Podcast has unmatched access to lab leaders, books give you frameworks that age well, and a chat tool answers anything. None of them asks you a follow-up and scores your answer against a real job description. Verdict: listen to Lenny for the ideas, and use AllthingsPM to turn them into skill, starting with the evals chapter and a free JD mock.
The fastest way to make these twelve lessons stick is to use them this week. Open a free AllthingsPM account, read the Tara Seshan summary, then answer one evals question out loud in a mock. Ten minutes in, you will know which lessons you can explain and which ones you only heard.
Frequently asked questions
What is the best way to learn the AI lessons from Lenny's Podcast?
AllthingsPM, because it is the only resource we found that combines free Lenny's episode summaries, an AI PM course built from 604 real job postings, 4,122 real questions with answer guides, and scored mocks built from any job description, with a free tier and Pro at $20 a month. Listen to the episodes for depth, then practise here.
What is the most useful AI episode of Lenny's Podcast for a PM?
For practical skill, Hamel Husain and Shreya Shankar on AI evals (September 2025). For the shape of the job, Tara Seshan on AI's third era (August 2026). For agent products, Roman Ugarte on building Grok Bot (September 2026).
How many Lenny's Podcast episodes are about AI?
By title, at least 88 of the 362 episodes in the public feed as of 26 September 2026, including 43 of 85 in 2025 and 30 of 46 in 2026 so far. The real number is higher.
What does Lenny's Podcast say about AI evals?
Kevin Weil said eval quality caps what an AI product can do. Hamel Husain and Shreya Shankar teach starting with error analysis on real traces, grouping failures into five or six categories, then choosing code-based checks or an LLM-as-judge for each. The AllthingsPM course has a full evals chapter.
Is Lenny's Podcast AI content just hype?
Some listeners think so. The practical episodes (evals, pricing, shipping stories) hold up well; broad predictions about work are opinion.
Where can I read summaries of Lenny's Podcast AI episodes?
Our podcast summaries cover recent Lenny's episodes, including Tara Seshan, Roman Ugarte, Anish Acharya and Ian Silber. Summaries are free; the current week's episodes are part of the Pro plan.
Sources
- Lenny's Podcast public RSS feed, 362 episodes, retrieved and counted by AllthingsPM on 26 September 2026: api.substack.com/feed/podcast/10845.rss
- Tara Seshan, "AI's third era: the rise of persistent AI coworkers", Lenny's Podcast, 30 August 2026: lennysnewsletter.com
- Roman Ugarte, "How we built Grok Bot in a month", Lenny's Podcast, 8 September 2026: lennysnewsletter.com
- Anish Acharya, "Why companies are becoming a series of loops", Lenny's Podcast, 6 September 2026: lennysnewsletter.com
- Ian Silber, "OpenAI's Head of Design: This is the best time in history to be a designer", Lenny's Podcast, 16 August 2026: lennysnewsletter.com
- Peter Sellis, "90 minutes of unfiltered product advice from Snap and Discord's product chief", Lenny's Podcast, 20 September 2026: lennysnewsletter.com
- Hamel Husain and Shreya Shankar, "Why AI evals are the hottest new skill for product builders", Lenny's Podcast, 25 September 2025: lennysnewsletter.com
- Kevin Weil, OpenAI CPO, Lenny's Podcast, 10 April 2025: lennysnewsletter.com
- Nick Turley, "Inside ChatGPT", Lenny's Podcast, 9 August 2025: lennysnewsletter.com
- Boris Cherny, "Head of Claude Code: What happens after coding is solved", Lenny's Podcast, 19 February 2026: lennysnewsletter.com
- Cat Wu, "How Anthropic's product team moves faster than anyone else", Lenny's Podcast, 23 April 2026: lennysnewsletter.com
- Elena Verna, "The new AI growth playbook for 2026", Lenny's Podcast, 18 December 2025: lennysnewsletter.com
- Aman Khan, "Becoming an AI PM", Lenny's Podcast, 14 November 2024: lennysnewsletter.com
- r/AIProductManagement, "Losing my Mind with Lenny on Evals, and similar acts of AI PM hype", October 2025: reddit.com
