Context

Most PMs report a 10% bump from AI tools, not a 2x to 3x one. This episode is a screen-share tour of what the larger jump actually looks like in practice. Mikhail Shcheglov, CPO at OLX Classifieds (a global classifieds marketplace) and formerly a group PM at Bolt and an IC PM for over a decade at Yandex, has spent five months building a single agentic "operating system" that his entire product org now runs on. It handles his email and calendar, screens stakeholder feature requests before they reach a PM, manages the recruiting funnel, and auto-maintains the design system. The value for a PM is less the specific tools he names and more the operating model underneath: how to store organizational knowledge, how to measure what the agent actually knows, and how the PM role changes when half of the busywork disappears.

The Big Idea

The real asset an AI-native org builds is a single, growing store of business, customer, and product knowledge that an agent can act on. The better the agent knows that context, the higher-level the work you can safely delegate to it, which frees PMs from process and points them almost entirely at discovery.

Shcheglov tracks how much of the company's context his agent holds as a personal KPI. At 54% coverage it already makes backlog-level calls like a junior-to-mid PM. He expects strategy-level help as that number climbs toward 70 to 90%.

Key Insights

Knowledge leakage is the hidden cost

• The problem: a domain expert in search, recommendations, or ML builds up years of context, then leaves and takes all of it with them. Every company has this hole and almost none of them measure it.
• Why it matters: if that business, customer, product, and technical knowledge lives in one shared store instead of one person's head, the org stops losing it when people churn, and the agent that reads it can take on more.
• The whole operating system is built to close this gap, not just to speed up individual tasks.

Output is cheap, quality is scarce

The cost of producing output (documents, code, mockups, prototypes) is now close to zero, so producing more of it is no longer the point. Shcheglov measures two things instead: the share of AI outputs that actually moved a business metric, and token consumption. A CPO's job shifts from building process scaffolding (planning cadences, review rituals) to owning the agentic system inside which the agent can act as a real collaborator, not just a suggestion engine.

Do not summarize your transcripts

He assumed meeting transcripts should be summarized before storage and tested it. Summarizing made retrieval 20 to 25% worse, for two reasons: you lose the granular detail where the answer usually lives, and you force every conversation into a template it was never shaped like. His decision was to store every transcript and every agent conversation raw, with no summarization.

Auto-generated skills lifted recall 31%

Hermes, one of the two frameworks he uses, watches which tasks get requested repeatedly and decides on its own to write a reusable skill for them (hiring evaluations, relocation case building). He tested this properly: five core topics, about ten questions each, a control group with no auto-generated skills against a treatment group with them. Recall, meaning how accurately the agent answered, rose 31%. That result is why he runs the skill-generation layer on all the time.

Make the agent a gatekeeper

Stakeholders are trained to pitch the agent before they reach a PM. A raw request ("I want video submissions for Autos listings because a competitor has it") has no problem framing, no impact estimate, and ignores set priorities. The agent asks clarifying questions, checks the request against existing priorities, declines politely if it does not clear the bar, and, if it does, adds it to the backlog and routes it to the domain owner. The full org chart is mapped internally so it knows who owns what.

One PM can cover four domains

Shcheglov estimates about half of a typical PM's time goes to process and rituals: weekly reports, stakeholder updates, demos, repetitive work needing little real thinking. Delegate that layer to the agent and one PM does the work of two, pointed at discovery. In practice he now runs one PM across three or four customer-facing domains, keeping dedicated owners only where a single change carries high risk.

Own your agent, do not outsource it

Handing the agent to an AI-ops hire or an engineering team is a mistake for two reasons. First, speed: he gets feedback daily and pushes changes straight into the GitHub repo from his phone via the Claude app, so feedback-to-deployment is near-instant. Second, impact: the agent saves time and steers decisions across the business, so the person with the most skin in the game should own it. Nobody without that stake iterates fast enough.

Mental Models & Frameworks

Context coverage as a KPI

• What it is: a single percentage for how much of the company's industry, business model, and customer knowledge the agent actually holds. His sits at 54%.
• How it is measured: a specific prompt asks the agent what share of knowledge it has across named dimensions (for OLX: five verticals like real estate and jobs, the P&L and its drivers, and customer segments down to cohort level), using everything currently in its memory. He found the agent fairly accurate at turning that abstract request into a number.
• When to use it: as a leadership metric to decide how much autonomy to grant. Below ~54% it makes backlog calls; he expects strategy-level work at 70 to 90%. Each org should define its own context dimensions rather than copy his.

Three-layer agent memory

• Knowledge graph: the structured web of interconnected nodes (contacts, projects, customers, funnel metrics) that shows how the org connects. He also reads it to spot silos, since teams with few overlapping nodes are a signal of weak cross-team discovery.
• Vector database: every piece of knowledge is converted to a vector, because most real queries are fuzzy and exact keyword matching fails roughly 75% of the time. The vector layer carries most retrieval.
• Raw transcripts: every meeting (via Granola) and every agent conversation is stored raw in Markdown files each day. This layer is what keeps the agent from losing context over time.

Imperatives over prompts

His rules file runs about 700 lines of imperatives, built up over time, because LLMs are trained to produce output that looks good rather than output that is correct. Key ones: no fabrications; think before you act (to stop plausible-looking answers with no real reasoning behind them); facts over guesswork; and a ban on "fake helpful," where the agent cannot do a task so it wastes tokens explaining how you could do it yourself.

CLAUDE.md short, SOUL.md long

Two config files load into every prompt. CLAUDE.md holds the highest priority and stays under 100 lines (the advice he attributes to Claude Code's creator). SOUL.md is second priority and runs about 800 lines. He accepts that some of those 800 lines even contradict each other, because in testing that longer, denser file still produced his most accurate output. Every imperative gets tested against real queries rather than trusted on faith.

Model routing by blast radius

An orchestrator picks the model per task by complexity and risk:

• Fable for complex requests or sensitive domains where an error has a large blast radius.
• Opus 4.8 for execution work and status reporting.
• Sonnet for low-level work that does not need high accuracy.

PM-side usage runs on a subscription with low token spend, so cost is not a tracked metric there. Engineering, which spends far more, runs a different scaffolding with its own model mix.

Two-group team staffing

When staffing a new area he splits domains into two groups:

• High complexity, high blast radius (monetization, search and ranking ML) where one tweak can swing conversion. These still need a dedicated human owner, and little has changed.
• Everything else (mostly customer-facing surfaces), where teams scale without adding headcount and one PM can own three or four domains across platforms.

Decision Principles

Principle: Delegate by error blast radius

• When: deciding how much autonomy to hand the agent, or whether a domain still needs a dedicated owner.
• Why: the cost of a wrong AI decision is not uniform. In areas where a single change can move revenue or conversion sharply, keep a human owner; in lower-risk, customer-facing areas, let the agent and a stretched PM cover more ground.

Principle: Keep humans as final decider

• When: the agent produces a judgment call, such as whether a candidate met the bar or which feature clears the priority threshold.
• Why: the agent is fast and gives a useful, often sobering second opinion built from transcripts and context, but Shcheglov is explicit that the human makes the final call. The value is an alternative point of view, not abdication.

Trade-offs & Nuance

Raw storage versus summarized memory

Storing every transcript raw preserves the detail that summaries strip out and lifts recall by 20 to 25% over summarizing. The trade-off is that raw storage only works if the retrieval layer is strong. Without a good vector search on top, dumping everything raw would bury the answer instead of surfacing it, which is why his three-layer memory exists.

Contradictory imperatives can still work

His 800-line SOUL.md file includes lines that contradict each other, which looks like context overload. He keeps it because the longer file measurably produced better recall in his testing. The nuance: this is not a reason to write sloppy rules, it is a reason to test every imperative against real queries rather than assume a shorter, cleaner file is automatically better.

Privacy versus transcription coverage

The system gets stronger the more meetings it ingests, but not every meeting should be recorded. His rule is opt-in: personal or one-on-one meetings are excluded by default, and transcripts are only stored and turned into skills when the employee willingly provides them. Access is also scoped, so a sensitive skill (like his "board skill") is available only to him and the executive committee.

Common Mistakes

Mistake: Outsourcing agent ownership

Delegating the agent to an AI-ops role or engineering team feels efficient but kills the two things that make it valuable: fast feedback-to-deployment (he ships changes daily from his phone) and direct ownership of a system that steers real business decisions. The person with the most stake should hold it.

Mistake: Chasing competitor features blindly

The classic stakeholder move is to demand a feature because a competitor shipped it, with no problem framing, no impact estimate, and no regard for set priorities, then to pressure a PM directly. Routing that request through a gatekeeper agent that forces clarifying questions and a priority check filters out the ones that should never reach the backlog.

Mistake: Letting agents be fake helpful

When an agent cannot complete a task (an expired token, a missing permission), the default LLM behavior is to spend tokens explaining how you could do it yourself. Shcheglov calls this "fake helpful" and bans it explicitly in his imperatives, because obvious do-it-yourself advice wastes tokens and the reader's time.

Practical Application

Define a context coverage metric

Write a prompt that asks your agent what share of knowledge it holds across the dimensions that matter for your business (your industry, your business model and its drivers, your customer segments), using everything in its memory. Track that percentage over time and watch it rise as PMs load in transcripts, research, and requirement docs. Treat it as directional, not a single source of truth.

Build a stakeholder gatekeeper flow

Stand up an agent that stakeholders pitch before they reach a PM. Give it your current priorities and org chart, and have it ask clarifying questions, decline requests that fail a problem-and-impact check, and route the survivors to the right domain owner's backlog. This protects PM focus and forces rigor onto every incoming request.

Run a control-versus-treatment recall test

To prove whether a change (like auto-generated skills) actually helps, ask your agent which topics get asked about most, generate about ten questions per topic, then compare answer accuracy with the change off (control) against on (treatment). Eyeball the first pass yourself, then, once the method is set, let the system run the evaluation. This is how he measured the 31% recall gain.

Automate your PM process layer

Identify the repetitive, low-cognition work that eats roughly half your week: weekly reports, stakeholder updates, demos, email triage, calendar scheduling across time zones. Delegate that layer to the agent (he no longer reads email or manages his own calendar, and gets a daily digest of what matters) so your remaining time points at discovery.

Route models by task risk

Do not run every task on your most expensive model. Use a top model for complex or high-risk work, a mid model for execution and status reporting, and a small model for low-accuracy tasks. Match model cost to the blast radius of getting the task wrong.

Questions to Consider

• What percentage of our company's core knowledge (industry, business model, customer segments) actually lives in one shared, queryable store versus in individual employees' heads?
• Which repetitive parts of our PMs' weeks (reports, updates, demos, scheduling) could be delegated so their time shifts toward talking to customers?
• Are we summarizing or templating our meeting notes and research before storing them, and is that quietly costing us retrieval accuracy later?
• Where in our org would a single wrong decision carry a large blast radius, and are those the only areas we still staff with a dedicated human owner?
• When we interview PMs, do we actually test how deeply they have automated their own work with AI, or only their traditional problem-solving fundamentals?

Bottom Line

An AI-native org is not a pile of AI tools, it is one growing, queryable store of company knowledge that an agent can act on, plus the discipline to measure how much of that knowledge the agent really holds. Get that right and the PM role does not shrink, it concentrates on the discovery work AI cannot do, while one PM covers the ground that used to take several.

Tools & Products

People to Follow

Mikhail Shcheglov

CPO at OLX Classifieds, previously a group PM at Bolt and an IC PM for over a decade at Yandex. He writes the Corporate Waters newsletter and open-sourced the setup behind this operating system. Worth following for a concrete, hands-on view of running a product org in an AI-native way.

Aakash Gupta

Host of The Growth Podcast and writer of a large product-and-AI newsletter (news.aakashg.com). He focuses on finding PMs and product leaders getting outsized (2x to 3x) gains from AI and reverse-engineering how they work.

Resources Mentioned