AI & Technical question

A major customer reports that Sierra's agent resolves English cases well but underperforms in Spanish on complex support flows. How would you determine whether the root cause is knowledge gaps, retrieval quality, prompt or tool-use failures, language understanding, policy handling, or escalation logic, and how would you prioritize the first fixes with engineering?

Practice this question out loud. An AI interviewer asks it, follows up like a real interviewer would, and scores your answer. Type or speak.

What this question tests

Tests structured root cause diagnosis across the AI stack for a language specific performance gap in complex support flows.

Segment the underperformance by specific complex flow type to see if it clusters around certain intents rather than Spanish broadly.
Check knowledge base coverage and accuracy in Spanish first, since knowledge gaps are a common and cheap to fix root cause.
Review retrieval quality specifically in Spanish, checking if relevant documents are being found and ranked correctly for Spanish queries.
Audit prompt and tool use behavior in Spanish conversations, since tool call formatting or intent extraction can behave differently across languages.
Test raw language understanding by isolating the model's Spanish comprehension from retrieval and workflow, to rule in or out a base model limitation.
Review escalation logic separately, since a fix elsewhere is wasted if escalation triggers are miscalibrated for Spanish and let bad answers through.

Segments by specific complex flow type rather than treating all Spanish conversations as one problem, narrowing the investigation efficiently.
Checks knowledge base and retrieval quality before assuming a model level language limitation, since those are typically faster and cheaper to fix.
Explicitly checks escalation logic as a separate layer, since it is often the safety net that should catch what the earlier layers miss.

Assuming the gap is a fundamental model limitation without first checking knowledge base coverage and retrieval quality in Spanish.
Fixing individual failure cases without segmenting by flow type to find the actual pattern behind the underperformance.

How would you prioritize fixes if the root cause is a mix of knowledge gaps and retrieval quality?
What would you do if the root cause turns out to be the underlying model's Spanish comprehension?