Intercom · Metrics · Mid · 45 to 60 minutes

How should Intercom prove support drives revenue?

Intercom makes customer messaging, support, and engagement software for businesses. Answer this product management case and get it graded line by line against a weighted rubric.

Experiment design
Metrics
Causal reasoning
Trade off judgment
Communication

The situation

At Intercom, the self serve onboarding team and the support education team are debating the value of in product help for new workspaces. These links appear throughout the product and point users to articles, tours, videos, and academy content that explain what to do next. Internal analysis shows that customers who use this content are about 2.5 times more likely to activate on features such as Series, but leaders are not convinced this means the content caused the improvement. It may simply be that more motivated customers both seek help and activate anyway. The VP of Product wants a recommendation before the next planning cycle, because expanding proactive support would require more product and content investment, while simplifying the UI could mean reducing these links. You are the PM for self serve onboarding. Recommend how Intercom should evaluate whether in product help truly improves activation and downstream business outcomes, and what decision criteria leadership should use.

What to produce

Write a 1 to 2 page experiment and metrics brief.

Your answer must

State the decision to be made and the main hypotheses, including at least one alternative explanation for the observed correlation.
Define a metric framework that includes primary success metrics, secondary metrics, and guardrails, and show how activation would connect to paid conversion or revenue.
Recommend a causal evaluation plan, such as an experiment or another rigorous design, with treatment and control logic, unit of analysis, timeframe, and target population.
Explain how you would interpret at least two different outcome patterns, including one where support contacts or searches decrease.
Identify the biggest risks, biases, or ethical concerns in your plan and how you would reduce them.

Constraints

Scope is limited to new self serve workspaces, not sales led customers.
You may change exposure to existing in product help links, but creating net new help content or changing pricing is out of scope.
Your plan should rely on product and support data Intercom already captures and should be practical to run during the normal trial and onboarding period.

Format

Use short sections with bullets under, decision, hypotheses, experiment design, metrics, interpretation, risks, recommendation.

How Intercom actually played it

Intercom ran an experiment where half of new self-serve workspaces had no links to help content for 28 days, to test whether proactive support affects activation and revenue. The post starts from a correlation (users of help content were 2.5 times likelier to activate) and designs a test to check causation. A clear lesson in moving from correlation to a controlled test.

Correlation between help usage and activation is not proof of impact.
Remove the feature for a control group to test its value.
Link support content to activation and revenue, not just ticket volume.

Built from The experiment that reveals how proactive support directly affects your bottom line by Jack Jenkins.