Metrics question
What metric stack would you use to determine whether North’s integrations portfolio is creating real customer value? Define the north-star metric, leading indicators, and guardrails across activation, time-to-stand-up, workflow usage, expansion, and retention, and explain how you would instrument the product so you can distinguish connector value from overall North usage.
- Cohere
- Metrics
- Hard
Practice this question out loud. An AI interviewer asks it, follows up like a real interviewer would, and scores your answer. Type or speak.
Start a mock interview on this question · Mock interview from a job description
What this question tests
Tests designing a metric stack that isolates the specific value of an integrations portfolio from overall product usage.
How to approach it
- Define the north-star metric, such as workflows completed that require at least one third-party connector, not total North usage.
- Add leading indicators: activation via connector install, time-to-stand-up from install to first successful call, and connector-specific workflow usage.
- Add expansion and retention metrics segmented by whether an account uses connectors, to isolate their incremental effect.
- Instrument connector-level usage separately from core North usage, tagging every workflow execution with which connectors it touched.
- Add guardrails, such as connector error rate and time-to-resolution, since a flaky connector could hurt overall retention.
What a strong answer includes
- Names a north-star tied specifically to connector-driven workflows, avoiding conflating it with all North usage.
- Proposes a concrete isolation method, such as comparing retention of connector-using accounts against a matched cohort without connectors.
- Includes time-to-stand-up as a leading indicator, since a slow setup kills activation before value is ever seen.
- Adds connector reliability as a guardrail, since integrations are a common source of silent product-wide churn.
Common mistakes
- Using total North usage as a stand-in for integration value, which can't isolate the connector's actual contribution.
- Skipping the instrumentation work needed to tag which workflows actually used a connector.
Likely follow-up questions
- How would you isolate connector value from selection bias, since power users may just be more likely to install connectors?
- Which single leading indicator would you watch weekly?
More metrics questions
- Before launch, what north-star, platform-health, and developer-experience metrics would you define for managed North, and how would you instrument onboarding, activation, and production usage so you can tell within 90 days whether you have product-market fit?Cohere · Metrics · Hard
- How would you measure the performance and health of a Netflix Original?Microsoft · Metrics · Hard
- Imagine you are the PM in charge of Reactions on Facebook - the new way to interact with posts by using “love”, “haha”, “wow”, “sad”, and “angry” reactions. What would success look like in terms of number of non-like reactions per post at launch and how do you come up with this? Would this number differ by reaction? Why or why not?Meta · Metrics · Hard
- You launched a new signup flow to encourage new users to add more profile information. A/B test results indicate that the % of people that added more information increased by 8%. However, 7 day retention decreased by 2%. What do you do?Google · Metrics · Hard
- Facebook Ads dropped by 20%. Mark Zuckerberg calls you and asks you to fix it. What do you do?Meta · Metrics · Hard
- Imagine you're the product manager for Facebook Marketplace. Since many sellers don't mark items as sold, what existing functionality and metrics could you use to determine whether an item has likely sold?Meta · Metrics · Hard
More questions from Cohere
Learn the skill behind it
Chapters of the AI PM course that teach what this question tests.
- Chapter 9: Prove it paid off: outcomes, economics, and pricing
- Chapter 2: Data fluency: SQL, logs, and reading the truth yourself
- Chapter 14: Get the job: the AI PM interview loop