OpenAI · Metrics · Mid · 45 to 60 minutes

How should ChatGPT launch success be measured?

OpenAI is an AI research and product company that builds models and products such as ChatGPT. Answer this product management case and get it graded line by line against a weighted rubric.

Metrics design
Product judgment
Analytical thinking
Communication
Prioritization

The situation

You are a PM at OpenAI preparing for the public launch of ChatGPT, a conversational AI product that lets people interact with a large language model. The head of product asks you for a launch metrics plan for the first few weeks after release. There is no revenue model attached to this launch, and leadership sees the release partly as a chance to learn whether people discover durable use cases, not just try it once because it is new. The team can instrument basic product analytics, but there is little historical data for a product like this at broad consumer scale. Press attention may create a surge of signups, and infrastructure and safety teams want clarity on which numbers matter if usage spikes. You need to recommend how OpenAI should define success for this launch, what the primary metric should be, which supporting metrics and guardrails to watch, and how to avoid overreading early excitement.

What to produce

A 1 to 2 page launch metrics memo.

Your answer must

State the goal hierarchy for the launch and explain why the top goal fits a release with no revenue model.
Define one primary success metric with the exact user unit, formula, cohort, and time window.
Recommend 3 to 5 secondary metrics that help interpret adoption and depth of use, with a brief reason for each.
Recommend 2 to 4 guardrail metrics, including at least one user trust, quality, or safety consideration.
Explain how you would account for novelty effects in the first month and what review cadence or decision forum you would use.

Constraints

Do not use revenue, profit, or pricing uptake as the primary success metric.
Assume only basic launch instrumentation is available, not a mature analytics stack built from months of prior usage.
There is little historical benchmark data for a product like this at broad scale.
Early traffic may be heavily distorted by curiosity and media attention.

Format

Use headings and a compact metric table. Keep the memo practical and specific.

How OpenAI actually played it

A metrics case about ChatGPT's launch, a release with no revenue model that was really a learning experiment. The answer defines goals for that situation, then builds a primary retention metric, secondary engagement metrics and guardrails.

Match metrics to the launch's real goal, which here was learning, not revenue
Retention shows whether people found repeatable use cases
Account for novelty effects when judging early numbers

Built from OpenAI PM Interview: ChatGPT Launch Metrics by Ajitesh, Tough Tongue AI (PM Interview Prep Weekly).