Duolingo · Metrics · Mid · 45 to 60 minutes

How should Duolingo test copy by language?

Duolingo is a language learning app that teaches through short lessons, reminders, and in-app prompts in the learner's primary language. Answer this product management case and get it graded line by line against a weighted rubric.

Metric design
Experimentation
Prioritization
Cross functional thinking
Localization judgment

The situation

You are the PM for a product area shared by Growth and Localization at Duolingo. The localization team says that many in-app messages were originally written in English, then translated, but the tone and clarity may not land the same way across learner languages. Product teams care because these messages affect important behaviors such as notification opt-in, whether learners continue a lesson when they feel stuck, and whether they understand paid features. The problem is that testing small copy changes still takes engineering work each time, so only a limited number of experiments happen. Assume you have signals that some high-traffic messages perform unevenly by language, including reminders, lesson exit prompts, and feature explanation screens, but you do not yet know whether the issue is tone, clarity, cultural fit, or something else. Your director wants a recommendation for how Duolingo should evaluate and prioritize localized copy changes over the next quarter. The tension is that better local copy could improve learning and conversion, but running many market-specific tests could add complexity, slow teams down, and create inconsistent messaging.

What to produce

Write a 1 to 2 page product memo, about 700 to 900 words.

Your answer must

Define the product goal and the main decision the team should make this quarter.
Identify 1 north star metric and 2 to 4 supporting or guardrail metrics for localized copy experiments, and explain why each metric matters.
Propose an experiment strategy that distinguishes when to run one global test versus language-specific tests.
Prioritize 2 or 3 message types or surfaces to test first, with clear reasoning tied to user impact and traffic.
Describe what instrumentation or operational setup is needed to measure results reliably, including how teams would know whether to roll out, iterate, or stop a test.

Constraints

You cannot assume a full redesign of the app, only changes to existing in-app copy and message flows.
Engineering support is limited, so your plan should minimize custom code work per experiment.
The approach must work across many learner languages and should not assume that a winner in one language will generalize to others.
Protect learner trust, so any metric plan should include at least one guardrail against spammy or demotivating messaging.

Format

Use five headings: goal, metrics, experiment strategy, prioritization, risks and decision rules.

How Duolingo actually played it

Duolingo tests how in-app messages land across languages, since its playful tone may not travel. One winning German copy test that explained notifications foster learning success raised notification opt-in 8 percent. A neat case of experimenting on copy and localization.

Tone does not translate, test it per market.
Small copy changes can move opt-in rates.
Treat localization as part of experimentation.

Built from How we adjust in-app messaging for learners in different languages by Eva Raimondi.