Metrics question
Define the metrics for YouTube search.
Practice this question out loud. An AI interviewer asks it, follows up like a real interviewer would, and scores your answer. Type or speak.
Start a mock interview on this question · Mock interview from a job description
What this question tests
Tests building a metrics framework for search specifically, separating relevance quality from engagement and distinguishing search from the core watch-feed metrics.
How to approach it
- Clarify the goal: YouTube search should help users find the specific video they're looking for quickly and accurately, distinct from the recommendation feed's discovery goal.
- Define relevance metrics: click-through rate on the top 3 results, and zero-result or reformulated-query rate (users re-searching because the first results didn't satisfy them).
- Define satisfaction metrics: watch time or completion rate of the clicked video relative to its average (did the search result actually satisfy the intent, not just get clicked), and return-to-search-results rate (a sign the video wasn't what they wanted).
- Define efficiency metrics: time from query to a satisfying click (search-to-watch latency), since fast, accurate search is the core value proposition.
- Segment by query type: navigational queries (looking for a specific known video/channel) versus exploratory queries (broad topic search), since success looks different for each.
- Add a guardrail: search result diversity/freshness, to ensure ranking doesn't over-favor old high-view videos at the expense of new relevant content.
What a strong answer includes
- Separates relevance (did we find the right thing) from satisfaction (did watching it actually meet the need), which is a more rigorous framing than click-through rate alone.
- Explicitly distinguishes search from the recommendation feed, recognizing they serve different user intents (finding vs discovering) and need different metrics.
- Segments by query type (navigational vs exploratory), showing awareness that one metric doesn't fit all search use cases.
- Adds reformulated-query rate as a strong dissatisfaction signal, a specific, well-chosen metric beyond basic click-through rate.
Common mistakes
- Using only click-through rate as the metric, which rewards clickbait thumbnails rather than true relevance.
- Conflating search metrics with general watch-time/engagement metrics used for the recommendation feed.
- No segmentation by query intent, treating all searches as the same.
Likely follow-up questions
- How would you measure search quality for queries with no clear 'right answer' (broad topics)?
- How would you detect when ranking is being gamed by clickbait titles?
- How would voice search change these metrics?
More metrics questions
- How do you define success for Yelp reviews?Google · Metrics · Medium
- Utilization went down by 45% on app XYZ in Italy for the month of August. Give a reason why and draft a plan to fix it.Spotify · Metrics · Medium
- You launched a new signup flow to encourage new users to add more profile information. A/B test results indicate that the % of people that added more information increased by 8%. However, 7 day retention decreased by 2%. What do you do?Google · Metrics · Hard
- You walk in to your office and find that Google cloud subscription has gone down by 20%. What do you do as a product manager?Google · Metrics · Medium
- What would be the top six metrics for WhatsApp? Now pick one from the six. Tell me three things you would do to improve that metric.Google · Metrics · Medium
- How do you measure the success of Google Search?Google · Metrics · Medium
More questions from these companies
Learn the skill behind it
Chapters of the AI PM course that teach what this question tests.
- Chapter 9: Prove it paid off: outcomes, economics, and pricing
- Chapter 2: Data fluency: SQL, logs, and reading the truth yourself
- Chapter 14: Get the job: the AI PM interview loop