Product design question

A global law firm wants Harvey for M&A diligence, but each partner runs diligence differently. How would you identify one pilot workflow to target, decide which parts Harvey should handle versus leave manual, and define the criteria for expanding from pilot to broader rollout?

Practice this question out loud. An AI interviewer asks it, follows up like a real interviewer would, and scores your answer. Type or speak.

What this question tests

Tests product design judgment for scoping an AI assisted legal workflow pilot amid inconsistent human processes, with criteria for expanding rollout.

Interview a sample of partners to find common steps across their varied diligence processes, since a pilot needs to generalize beyond one style.
Pick the pilot workflow around the step with the most consistent process across partners, for example document review and issue flagging over final judgment.
Decide the human versus Harvey split explicitly, for example Harvey surfaces and categorizes issues while the partner rules on materiality.
Design the pilot to collect evidence on accuracy and time saved against that specific split, not the full diligence process end to end.
Define expansion criteria up front, such as a minimum accuracy rate on flagged issues, before broadening to other diligence steps or teams.

Picks the pilot workflow based on where partner processes actually converge, rather than assuming diligence is one uniform process to automate.
Draws a clear, testable line between Harvey's role and the partner's judgment, for example flagging issues versus deciding materiality.
Defines expansion criteria before the pilot starts, such as an accuracy threshold, rather than deciding on anecdotal partner enthusiasm.
Treats the pilot as evidence for the human AI split, so expansion criteria are about proven division of labor, not just usage.

Trying to automate the full diligence process at once instead of finding the one workflow with the most process overlap.
Leaving the human versus AI division of labor vague, which makes it impossible to evaluate the pilot objectively.

How would you handle a partner whose process does not fit the chosen pilot workflow?
What would make you decide the pilot failed even with decent usage numbers?