Strategy question
You inherit a platform where teams lack visibility into request failures, latency regressions, customer-impacting incidents, and model-quality degradation. How would you define the product scope for observability, decide what to ship first, and ensure the work improves customer outcomes rather than just adding internal tooling?
- Harvey
- Strategy
- Hard
Practice this question out loud. An AI interviewer asks it, follows up like a real interviewer would, and scores your answer. Type or speak.
Start a mock interview on this question · Mock interview from a job description
What this question tests
Tests scoping an observability product from scratch, prioritizing what ships first, and ensuring the investment improves customer outcomes, not just internal tooling.
How to approach it
- Start by identifying who is actually blocked today: which teams cannot diagnose incidents, and what decisions they cannot make without this visibility.
- Scope v1 around the highest-frequency pain: likely request failure and latency visibility first, since those directly affect customer-facing reliability.
- Sequence model-quality degradation detection second, since it typically requires more instrumentation maturity than basic failure and latency tracking.
- Tie the observability investment to a customer outcome explicitly, for example faster incident resolution reducing customer-visible downtime, not just 'better internal visibility'.
- Avoid building a broad internal analytics platform; scope tightly to the failure modes that map to real incidents teams have already had.
- Set an adoption and outcome metric, like mean time to detect and resolve, to prove the investment mattered beyond internal convenience.
What a strong answer includes
- Scopes v1 around the highest-frequency, most customer-visible pain, not a broad internal analytics wishlist.
- Explicitly ties the observability investment to a customer-facing outcome, like reduced downtime, rather than internal tooling for its own sake.
- Sets a concrete outcome metric, mean time to detect and resolve, as proof of value.
Common mistakes
- Building a broad internal analytics platform disconnected from real incident patterns.
- No metric connecting the observability work to a customer-facing outcome.
Likely follow-up questions
- How would you prioritize between failure visibility and model-quality degradation detection?
- How would you know if teams are actually using the new observability tools?
More strategy questions
- How should Harvey price for law firms vs. in-house legal teams?Harvey · Strategy · Hard
- How would you grow Harvey adoption among conservative, risk-averse law firms?Harvey · Strategy · Hard
- You inherit Vault with strong customer interest but limited engineering capacity. How would you set Vault’s product vision and a 12-month roadmap across file management, AI search/Q&A, document extraction, secure sharing, and platform investments like permissions, indexing, and integrations? Walk through your prioritization framework, major bets, and what you would explicitly defer.Harvey · Strategy · Hard
- Harvey can fund only one major integration track next: deeper iManage support, broader SharePoint coverage, or cross-platform capabilities like sync, permissions mapping, and admin controls. How would you decide what to build first, and what tradeoffs would you make across customer demand, implementation effort, security risk, and strategic leverage?Harvey · Strategy · Hard
- Harvey’s Command Center needs to serve three jobs for enterprise admins: understanding adoption, managing users and access, and enforcing governance. How would you define the product strategy and first 12-month roadmap for Command Center, and what framework would you use to make trade-offs across those three areas for Harvey’s largest legal customers?Harvey · Strategy · Hard
- Harvey is being pushed to ship new platform capabilities faster, while enterprise customers are escalating concerns about uptime, latency, and trust. As the PM for Core Platform, how would you build a 12-month infrastructure roadmap that balances reliability, performance, security, and feature velocity? Walk through your prioritization framework, the inputs you would use, and how you would handle tradeoffs when these goals conflict.Harvey · Strategy · Hard
More questions from Harvey
Learn the skill behind it
Chapters of the AI PM course that teach what this question tests.
- Chapter 4: Discovering and Sizing AI Opportunities
- Chapter 9: Measuring AI Product Economics
- Chapter 14: Interviewing for AI PM Roles