Metrics question

What metrics would you track for Lindy's agent activation and retention?

Practice this question out loud. An AI interviewer asks it, follows up like a real interviewer would, and scores your answer. Type or speak.

What this question tests

Tests metrics design for a no code agent platform, distinguishing account signup from a user actually getting value.

Define activation precisely: not account creation, but the moment a user's first agent completes a real task successfully, for example an email triaged or a meeting booked.
Build the activation metric: percent of new signups reaching one successful agent run within a defined window, like 24 hours.
Build the retention metric: percent of activated users with at least one agent run in a rolling 7 and 30 day window, since a set and forget agent still counts as retained if it keeps running.
Add a depth metric: number of active agents per retained user, since growth often comes from existing users adding a second or third agent.
Add a guardrail: agent run failure rate, since a user whose agent silently fails will look retained in login data but is not getting real value.
Confirm with the interviewer whether retention should be measured by login activity or by agent runs, since a good agent needs no login at all.

Defines activation as a successful agent run, not signup or agent creation, which is the real moment of value for this product.
Uses agent run frequency rather than login frequency for retention, correctly recognizing Lindy's value is autonomous and does not require the user to return.
Adds agents per user as a growth wedge metric, since expansion within an account is often cheaper than new acquisition.
Includes run failure rate as a guardrail so retention numbers are not inflated by agents that are technically still installed but quietly broken.

Using login frequency as the retention metric for a product designed to work without the user checking in.
Treating account signup as activation instead of a real successful agent run.
No guardrail for silent agent failure, which would make retention look healthy while users are actually not getting value.

How would you detect an agent that is running but producing bad output?
What would you do if activation was high but 30 day retention was low?
How would you segment these metrics by use case, like sales versus personal productivity?