Metrics question
You launch a new enterprise integration for legal teams. What metric stack would you use from launch through scale, covering adoption, workflow completion, trust or quality, retention, expansion, and commercial impact, and how would each metric influence what you iterate on next?
- Harvey
- Metrics
- Hard
Practice this question out loud. An AI interviewer asks it, follows up like a real interviewer would, and scores your answer. Type or speak.
Start a mock interview on this question · Mock interview from a job description
What this question tests
Tests the ability to design a full lifecycle metric stack for an enterprise legal integration and connect each metric to a decision.
How to approach it
- At launch, track activation: percent of licensed seats who complete a first workflow, like drafting a clause or querying a matter.
- Track workflow completion: percent of started tasks finished without abandoning to manual work, split by task type.
- Track trust and quality: citation accuracy, hallucination rate on legal claims, sampled by attorney review.
- Track retention: weekly active users among licensed seats, and time to second and third use after first activation.
- Track expansion and commercial impact: seat growth within the account, upsell to adjacent workflows, and renewal rate.
- Use activation gaps to fix onboarding, quality gaps to fix the model or retrieval, and retention gaps to fix workflow fit.
What a strong answer includes
- Sequences the metrics by lifecycle stage instead of listing them flat, showing which ones matter in month one versus month six.
- Ties trust and quality metrics specifically to legal risk, citation accuracy and hallucination rate, not generic satisfaction scores.
- Explains a concrete next action for at least two metrics, for example low activation triggers in product onboarding nudges.
Common mistakes
- Listing generic SaaS metrics without adapting them to legal specific trust requirements like citation accuracy.
- Tracking retention and expansion without first confirming quality is trusted, since low trust caps both.
Likely follow-up questions
- Which one metric would you cut if you could only track three?
- How would you measure trust separately from raw usage volume?
More metrics questions
- What metrics prove Harvey's value to a firm like PwC or Paul Weiss?Harvey · Metrics · Hard
- What KPI hierarchy would you use for Vault, from account-level adoption and active matters to search success, document coverage, and workflow outcomes, to measure value for law firms and enterprises? How would those metrics change your roadmap if usage is high but repeat usage in critical workflows is low?Harvey · Metrics · Hard
- What metrics would you define for Command Center across adoption, admin efficiency, and governance coverage, and how would you use those metrics to decide whether the product is actually improving Harvey’s enterprise land-and-expand motion?Harvey · Metrics · Hard
- Harvey is considering expanding its platform to additional regions. How would you decide whether multi-region expansion is the right product investment now versus later? Explain the customer signals, business considerations, and success metrics you would use, and how you would compare this investment against competing platform priorities.Harvey · Metrics · Hard
- For Vault’s search and Q&A experience, what metrics would you define for adoption, engagement, trust, and business value at the workspace and firm level? Which leading indicators would you watch in the first 90 days, and how would changes in those metrics alter your product decisions?Harvey · Metrics · Hard
- A pilot agent deployment has strong qualitative feedback from senior stakeholders, but end-user adoption is inconsistent and the customer’s data environment is messy. How would you structure the deployment plan, define the success metrics, diagnose whether the issue is workflow fit vs data readiness vs change management, and decide if this should graduate into a repeatable product for the broader vertical?Harvey · Metrics · Hard
More questions from Harvey
Learn the skill behind it
Chapters of the AI PM course that teach what this question tests.
- Chapter 9: Prove it paid off: outcomes, economics, and pricing
- Chapter 2: Data fluency: SQL, logs, and reading the truth yourself
- Chapter 14: Get the job: the AI PM interview loop