Metrics question
You are the single DRI for paid search, paid social, ASO, and lifecycle. Design the operating loop you would build in your first 90 days: what inputs you’d ingest, how you’d segment users and geos, how creative insights feed into new variants, how budget allocation decisions get made, and which parts must be automated vs human-reviewed. Be explicit about the metrics and decision rules that make the system improve over time.
- Replit
- Metrics
- Hard
Practice this question out loud. An AI interviewer asks it, follows up like a real interviewer would, and scores your answer. Type or speak.
Start a mock interview on this question · Mock interview from a job description
What this question tests
Tests the ability to design a full growth operating system as a single owner, covering inputs, segmentation, creative iteration, and automation boundaries.
How to approach it
- In the first 90 days, ingest platform spend and conversion data, product usage data, and creative performance data into one unified view.
- Segment users and geos by value, not just volume, so budget allocation reflects downstream activation and retention, not just cheap signups.
- Build a feedback loop where creative performance data, which messages and formats convert best per segment, directly informs the next round of creative briefs.
- Automate budget allocation for well understood, stable channels and segments, but keep new channel tests and high spend decisions under human review.
- Define explicit decision rules, for example automatically pause any campaign whose cost per activated user exceeds a set threshold for three consecutive days.
- Review the system's own decision rules monthly, adjusting thresholds as data volume and confidence grow, so the system improves rather than stays static.
What a strong answer includes
- Segments by downstream value rather than raw volume, which is the correct standard for a DRI optimizing lifecycle, not just top of funnel signups.
- Draws a clear line between what gets automated, stable channel budget shifts, and what stays human reviewed, new channels and large spend changes.
- Includes a concrete automatic decision rule with a specific threshold, showing the system actually has teeth rather than being purely advisory.
Common mistakes
- Automating every budget decision from day one without a human review boundary for new or high risk spend changes.
- Segmenting and optimizing purely on top of funnel signup volume instead of downstream activation and retention value.
Likely follow-up questions
- How would you decide when a channel has enough data history to move from human reviewed to automated?
- What would you do if the automated system starts making a decision you disagree with?
More metrics questions
- What metrics would you track for Replit Agent's success?Replit · Metrics · Medium
- How would you measure whether AI is actually helping users ship apps faster on Replit?Replit · Metrics · Medium
- Replit launches a new natural-language app builder for both existing users and new prospects. How would you decide which growth motions should come from product-led loops vs paid acquisition, which segments or geos you’d target first, and what first 3-5 experiments you’d run to test whether paid is incrementally driving durable activation rather than just more signups?Replit · Metrics · Hard
- How would you design a product-led land-and-expand motion for Replit enterprise so that a small group of internal builders grows into a company-wide rollout? What product milestones, leading indicators, and account-level metrics would you use to decide whether an account is ready to expand?Replit · Metrics · Hard
- A week after launch, CAC in one region spikes 35% while blended global CAC is flat. Walk me through how you’d detect it early, decompose the change by funnel stage and channel, separate mix-shift vs signal-quality vs creative-fatigue causes, and decide whether to cut spend, change targeting, refresh creative, or fix measurement.Replit · Metrics · Hard
- How would you measure the performance and health of a Netflix Original?Microsoft · Metrics · Hard
More questions from Replit
Learn the skill behind it
Chapters of the AI PM course that teach what this question tests.
- Chapter 9: Prove it paid off: outcomes, economics, and pricing
- Chapter 2: Data fluency: SQL, logs, and reading the truth yourself
- Chapter 14: Get the job: the AI PM interview loop