Metrics question

A new Harvey integration is enabled by enterprise admins, but weekly usage among lawyers is low and answers are rarely reused in active matters. How would you diagnose where the breakdown is, from setup and permissions to relevance, workflow fit, and trust, and what product changes would you test first?

Practice this question out loud. An AI interviewer asks it, follows up like a real interviewer would, and scores your answer. Type or speak.

What this question tests

Whether you can diagnose a low-adoption enterprise product problem by isolating the specific stage where it breaks down, from setup through trust.

Instrument the funnel: admin enablement, individual lawyer awareness, first use, relevance of results, and reuse in active matters, so you can see where usage drops.
Check setup and permissions first, since admin-level enablement does not guarantee individual lawyers have working access or even know it exists.
If access is fine, check relevance: sample actual queries and results to see if the integration surfaces genuinely useful matter data or noisy results.
If relevance is fine, check workflow fit: whether lawyers can act on results within their existing tools, or need an extra, disruptive step.
If workflow fit is fine, check trust: whether lawyers who tried it once did not return, pointing to a first-impression confidence problem rather than a structural one.
Test the top hypothesis with a small, targeted change, for example a setup nudge or a relevance fix, and measure whether weekly usage moves before rolling out broader changes.

Structures the diagnosis as a funnel with a clear stage-by-stage elimination process, rather than guessing at one cause.
Distinguishes setup and permissions issues, invisible to the end user, from relevance and trust issues, visible but still low-adoption.
Names reuse in active matters, not just first use, as the real bar for legal professionals.
Proposes testing the top hypothesis with a small change before a broad rollout, a disciplined, low-risk approach.

Jumping to a fix, like more training, without isolating which funnel stage is actually broken.
Treating low weekly usage and low reuse as the same problem, when they may have different causes.
No plan to validate a hypothesis with a small test before a larger rollout.

How would you get visibility into individual lawyer usage without violating firm confidentiality expectations?
What would you do if the drop-off is concentrated among one practice group?
How would you decide the diagnosis is right before committing engineering time to a fix?