Metrics question
Before launch, what north-star, platform-health, and developer-experience metrics would you define for managed North, and how would you instrument onboarding, activation, and production usage so you can tell within 90 days whether you have product-market fit?
- Cohere
- Metrics
- Hard
Practice this question out loud. An AI interviewer asks it, follows up like a real interviewer would, and scores your answer. Type or speak.
Start a mock interview on this question · Mock interview from a job description
What this question tests
Whether you can define a launch metrics framework for a new managed product that actually answers the product-market fit question within a defined window, not just track generic usage.
How to approach it
- North star: percentage of new managed North customers reaching a defined production milestone, like a live agent handling real traffic, within their first 30 days.
- Platform-health metrics: deployment success rate, uptime, and mean time to resolve platform incidents, since managed infrastructure trust is foundational to any adoption story.
- Developer-experience metrics: time from signup to first successful agent deployment, and self-serve documentation or support ticket volume per active account, since friction here predicts churn before it shows up in usage data.
- Instrument onboarding as a funnel with named steps, account created, first agent configured, first agent deployed to production, so drop-off is visible stage by stage, not just as one aggregate activation number.
- Define product-market fit at day 90 concretely: a target percentage of accounts reaching production plus a retention or expansion signal, like accounts adding a second agent, not just initial signups.
What a strong answer includes
- Defines a concrete, time-boxed product-market fit bar at 90 days instead of leaving fit as a vague qualitative judgment.
- Separates platform health from developer experience, recognizing infrastructure reliability and ease of use are different failure modes with different fixes.
- Breaks onboarding into named funnel steps, which makes the framework immediately actionable for diagnosing where new customers get stuck.
Common mistakes
- Tracks only signups or trial starts, without any production-usage or retention signal.
- Leaves product-market fit as a vague feeling rather than a defined, measurable bar.
Likely follow-up questions
- What would you do if deployment success is high but few accounts reach production traffic.
- How would you separate onboarding friction from a genuine lack of product fit.
More metrics questions
- What metric stack would you use to determine whether North’s integrations portfolio is creating real customer value? Define the north-star metric, leading indicators, and guardrails across activation, time-to-stand-up, workflow usage, expansion, and retention, and explain how you would instrument the product so you can distinguish connector value from overall North usage.Cohere · Metrics · Hard
- How would you measure the performance and health of a Netflix Original?Microsoft · Metrics · Hard
- Imagine you are the PM in charge of Reactions on Facebook - the new way to interact with posts by using “love”, “haha”, “wow”, “sad”, and “angry” reactions. What would success look like in terms of number of non-like reactions per post at launch and how do you come up with this? Would this number differ by reaction? Why or why not?Meta · Metrics · Hard
- You launched a new signup flow to encourage new users to add more profile information. A/B test results indicate that the % of people that added more information increased by 8%. However, 7 day retention decreased by 2%. What do you do?Google · Metrics · Hard
- Facebook Ads dropped by 20%. Mark Zuckerberg calls you and asks you to fix it. What do you do?Meta · Metrics · Hard
- Imagine you're the product manager for Facebook Marketplace. Since many sellers don't mark items as sold, what existing functionality and metrics could you use to determine whether an item has likely sold?Meta · Metrics · Hard
More questions from Cohere
Learn the skill behind it
Chapters of the AI PM course that teach what this question tests.
- Chapter 9: Prove it paid off: outcomes, economics, and pricing
- Chapter 2: Data fluency: SQL, logs, and reading the truth yourself
- Chapter 14: Get the job: the AI PM interview loop