AI & Technical question

How would you prevent and handle hallucinated case law in Harvey's answers?

Practice this question out loud. An AI interviewer asks it, follows up like a real interviewer would, and scores your answer. Type or speak.

What this question tests

Tests AI product judgment on preventing and catching hallucinations in a domain where a fabricated citation has real legal consequences.

Separate prevention from detection, since no model fully prevents hallucination and the product needs both a reduction strategy and a safety net.
On prevention, propose grounding every answer in retrieval from verified legal databases rather than relying on the model's parametric memory of case law.
On detection, propose an automated verification step that checks every generated citation actually exists and says what the answer claims it says.
Design the user facing safeguard: flag any claim that fails automated verification before the lawyer ever sees it, rather than showing an unverified answer.
Add a reporting channel so lawyers who catch a hallucination in practice can flag it, feeding back into both product fixes and model evaluation.
Confirm with the interviewer whether the concern is fabricated cases entirely, or real cases cited for the wrong proposition, since the detection method differs.

Distinguishes two failure modes clearly: a citation to a case that does not exist, and a citation to a real case that does not actually support the claim, since verification methods differ.
Proposes retrieval grounded generation over relying on model memory, directly addressing the root cause of fabricated case law.
Adds an automated post generation verification pass that checks citations against a real legal database before the answer reaches the user.
Treats a caught hallucination as a data point for retraining and evaluation, not just a one off bug fix, closing the loop over time.

Proposing only a disclaimer telling lawyers to double check, which does not meaningfully reduce the real risk of a court sanction.
Not distinguishing fully fabricated cases from real cases misapplied, which need different detection approaches.
Ignoring that this already happened in real courts, which the interviewer likely expects awareness of as context, without needing invented details.

How would you measure the hallucination rate on legal citations specifically?
What would you do if verification itself has false negatives and misses a bad citation?
How would you rebuild firm trust after a hallucinated citation reaches a filed document?