Strategy question

Pick one emerging model capability you think Anthropic Labs could turn into a new product category in the next 12-18 months. How would you work with researchers to separate an interesting demo from real user leverage, and what explicit criteria would you use to decide whether it deserves an internal prototype?

Practice this question out loud. An AI interviewer asks it, follows up like a real interviewer would, and scores your answer. Type or speak.

What this question tests

Tests separating a compelling model demo from a durable product opportunity, and defining concrete criteria for internal prototype investment.

Pick one specific emerging capability, such as long-horizon autonomous coding or computer-use agents, and state it explicitly.
Define what real user leverage means for this capability: meaningful time saved or a new unlocked task, versus mere novelty.
Work with researchers to build a small set of real-world tasks, not benchmark scores, to test reliability.
Set explicit criteria for prototype investment, such as a minimum task success rate and early testers returning without prompting.
Decide based on that evidence, distinguishing genuine pull from researcher excitement alone.

Names one specific capability rather than speaking generically about AI agents.
Defines a concrete, falsifiable bar for real user leverage, such as task completion without human correction on a defined task set.
Distinguishes benchmark performance from field reliability, since demos often work on curated tasks but fail on messy real ones.
Uses return usage as the pull signal, not stated enthusiasm, as an illustrative criterion.

Chasing an impressive demo without defining what real user leverage would concretely look like.
Relying on researcher enthusiasm as the sole signal instead of testing with real users.

What would make you kill this idea after the prototype stage?
How would you avoid the sunk-cost trap if researchers are excited but users aren't?