Facebook · Metrics · Senior · 45 to 60 minutes

How should Facebook test ads revenue growth?

Facebook is a social network where people share content and businesses reach those people through ads. Answer this product management case and get it graded line by line against a weighted rubric.

Metric selection
Experiment design
Prioritization
Monetization judgment
Stakeholder tradeoffs

The situation

The VP leading Facebook's ads monetization team asks you, a senior PM, for a recommendation on how to raise ads revenue through experimentation. Revenue growth matters this quarter, but leadership is wary of short term gains that come from making the product feel worse for people or reducing value for advertisers. Several teams have ideas for changes to the ads system, but there is not enough traffic and engineering capacity to run every large test at once. Assume you can sponsor only a few major A/B tests this quarter, and each one will face scrutiny from product, ads, and integrity leaders. Your job is to define how success should be measured, decide which experiments deserve priority, and show how you would run the first test with enough rigor that leadership can trust the result. The hard part is balancing a direct business goal against second order effects on user experience, advertiser outcomes, and the health of the marketplace.

What to produce

Write a 1 to 2 page experiment strategy memo for the ads leadership team.

Your answer must

Name one primary metric for the goal and explain why it is the best single success metric.
List 3 to 5 guardrail metrics that cover user experience, advertiser outcomes, and marketplace health.
Propose exactly three experiment ideas, each with a hypothesis, expected revenue mechanism, and an impact versus feasibility assessment.
Choose one of the three as the first test to run and outline the A/B test design, including randomization unit, decision window, sample size or power approach, and success criteria.
Discuss key risks and second order effects, including how a revenue gain could mask harm to users or advertisers, and how you would detect that.
Recommend a sequencing plan for the three experiments and explain why that order is best under the stated constraints.

Constraints

You cannot rely on launching a brand new ad product or a new partnership, focus on changes to the existing ads business.
You have limited experiment capacity this quarter, so overlapping large tests on the same traffic are hard to interpret.
Leadership will not accept a plan that optimizes revenue while ignoring meaningful harm to user experience or advertiser value.
Your recommendation must be supportable with online experimentation in a reasonable time frame, not only with long term effects that are impossible to measure now.

Format

Use sections for goal, metrics, experiment options, first test design, risks, and recommendation.

How Facebook actually played it

A worked answer on designing experiments to raise Facebook ads revenue. It proposes three high impact tests, uses eCPM as the primary metric and discusses feasibility, sample sizes and tradeoffs against user experience.

Pick one primary metric tied to the goal and keep others as guardrails
Rank experiments by impact and feasibility
Consider advertiser and user effects together

Built from A/B Testing PM Interview: Facebook Ads Revenue Question by Ajitesh, Tough Tongue AI (PM Interview Prep Weekly).