AI & Technical question

Researchers improve model steerability in a domain, but design partners give vague and conflicting feedback like 'more controllable,' 'too rigid,' and 'not trustworthy.' How would you translate that feedback into concrete product requirements and model evaluation criteria for research and engineering?

Practice this question out loud. An AI interviewer asks it, follows up like a real interviewer would, and scores your answer. Type or speak.

What this question tests

Ability to turn vague, even contradictory qualitative feedback into specific, testable requirements for both product and research.

Follow up directly with each design partner to get a concrete example behind each vague term, asking what specific interaction made them say too rigid or not trustworthy rather than accepting the label at face value.
Look for whether the apparently contradictory feedback actually maps to different use cases or user types, for example more controllable from technical users and too rigid from less technical users pointing to a need for adjustable defaults rather than one fixed behavior.
Translate the concrete examples into specific, testable product requirements, such as a configurable parameter for how strictly the model follows instructions, rather than a vague goal of better steerability.
Translate the trust related feedback into a model evaluation criterion, for example measuring how often the model's stated confidence or reasoning matches its actual behavior across a test set, since not trustworthy often points to a specific consistency gap.
Share the synthesized, concrete requirements back with design partners to validate you correctly interpreted their vague feedback before committing research and engineering resources.
Prioritize the requirements that came from multiple independent partners over a single partner's idiosyncratic complaint.

Digs past the vague labels to concrete examples through direct follow up, rather than guessing at what too rigid means.
Finds the pattern that reconciles apparently contradictory feedback, such as different needs by user type, instead of averaging it into mush.
Converts trust related complaints into a specific, measurable eval criterion, not just a general research direction.
Validates the synthesized requirements with partners before committing resources, closing the loop.

Averaging contradictory feedback into a compromise that satisfies no one instead of finding the underlying pattern.
Passing vague feedback straight to research and engineering without translating it into something testable.

What would you do if a design partner disagreed with your interpretation of their own feedback?
How would you prioritize when two partners want genuinely opposite behavior?