Metrics question

What metrics define a successful Manus task run?

Practice this question out loud. An AI interviewer asks it, follows up like a real interviewer would, and scores your answer. Type or speak.

What this question tests

Metrics design for defining success on open-ended, variable autonomous tasks, harder than a simple pass/fail feature.

Define completion precisely: the task's stated goal was achieved and verifiable, not just that the agent stopped running or produced some output.
Add a quality dimension: whether the output required significant user rework versus was usable as-is, since technically 'complete' is not the same as 'good'.
Add an efficiency dimension: time and compute cost to reach completion, since a slow or expensive success is a weaker outcome than a fast one.
Add a trust dimension: whether the user accepted the result without needing to double-check or redo the task manually.
Combine these into a tiered outcome classification, such as fully successful, partially successful with rework needed, or failed, rather than a binary pass/fail.
Segment by task category, since research tasks, coding tasks, and web-based tasks will have very different natural success rates.

Moves beyond binary completion to a quality-weighted, tiered outcome classification, which better reflects real user value.
Adds efficiency (time, cost) as a dimension, since Manus tasks can be expensive to run and a slow success matters less.
Includes a trust dimension (did the user accept the result without redoing it) as the ultimate practical measure of success.
Segments by task type, avoiding a misleading blended metric across very different task categories.
Shows awareness that open-ended agent tasks need a richer definition of success than a simple feature completion check.

Treating 'the agent finished running' as equivalent to a successful task run.
Ignoring efficiency and cost, which matter a great deal for an expensive autonomous agent.

Chapters of the AI PM course that teach what this question tests.