Metrics question

How would you measure user trust in an autonomous agent over time?

Practice this question out loud. An AI interviewer asks it, follows up like a real interviewer would, and scores your answer. Type or speak.

What this question tests

Metrics design for a hard-to-quantify construct, trust, tracked over time rather than as a single snapshot.

Define trust behaviorally: users delegate progressively more consequential or higher-stakes tasks to the agent over time without needing to double-check its work.
Track a delegation-depth metric: the average risk or complexity level of tasks a given user assigns to the agent, watched for growth over their tenure as a user.
Track a verification-behavior metric: how often users manually double-check or redo the agent's output, expecting this to decline over time as trust builds.
Track retention and expansion as lagging trust indicators, since users who do not trust an agent will simply stop using it or limit it to trivial tasks.
Add a direct survey signal periodically, asking users to rate their confidence in the agent for specific task types, to validate the behavioral proxies.
Define success as an upward trend in delegation depth combined with a downward trend in manual verification, tracked per user cohort over their first several months.

Defines trust through observable behavior (delegation depth, verification rate) rather than only asking users to self-report.
Tracks the metric as a trend over a user's tenure, matching the 'over time' framing in the question rather than a single snapshot.
Uses retention and task-scope expansion as a natural lagging indicator that reinforces the primary behavioral metrics.
Pairs behavioral proxies with periodic direct surveys to validate they are measuring what they claim to measure.
Segments by cohort to see how trust curves differ for new versus experienced users.

Measuring trust only through occasional user surveys without any behavioral, revealed-preference signal.
Treating trust as a static score rather than a trend that should be tracked over a user's lifecycle.

How would you define 'task risk level' consistently for the delegation-depth metric?
What would a declining trust trend look like, and how would you respond?
How would you validate the survey against the behavioral metrics?