Labelbox · AI product · Senior · 45 to 60 minutes

How should Labelbox test the labeling bet?

Labelbox makes software for teams to label and organize data used to train machine learning models. Answer this product management case and get it graded line by line against a weighted rubric.

Product strategy
Discovery design
AI product judgment
Prioritization
Executive communication

The situation

It is mid 2017. You are the first product leader working with two technical founders who want to start a company around data labeling for machine learning teams. One founder has seen the pain up close at Planet, where teams move large image datasets around through desktop tools and manual file sharing, and there is no easy way for multiple people to collaborate with roles, permissions, and consistent workflows. The founders believe AI adoption will grow quickly, but the market is early. Some people in the AI community say a general labeling platform is impossible because every use case is too custom. Others are intrigued, but mostly for their own niche workflow. Assume the team has 6 months of runway and can either build a lightweight prototype or run a focused discovery effort with mockups and customer interviews, but not both deeply at the same time. The founders ask you for a recommendation on how to test whether this is a real, scalable company bet, not just a painful problem at one company. You need to decide what evidence matters, who to talk to, and what would justify moving from discovery into build mode.

What to produce

A 1 to 2 page decision memo.

Your answer must

State the core hypothesis for the business and at least two alternative explanations or failure modes.
Recommend how the team should spend the next 6 to 8 weeks across customer discovery, mockups, and any product building.
Define the target interview segments and explain why each segment matters for validation.
Provide 5 sample open-ended interview questions and note what each question is meant to learn.
Specify clear go, no-go, or narrow-the-scope criteria for moving from discovery into build mode.
Call out the biggest risks of false positives, including overfitting to one company, one warm network, or one niche use case.

Constraints

There is no working product yet, so the team can only credibly show sketches, mockups, or workflow concepts in the near term.
Most current evidence comes from one founder's experience and a small network of AI practitioners, which can bias the team.
Potential customers span different industries and data types, so bespoke requests could hide whether a general product exists.
The team has limited runway and cannot run a deep research effort and a deep build effort in parallel.

Format

Use sections titled Bet, Validation Plan, Interview Design, Decision Gates, and Risks. Bullets and a small table are fine.

How Labelbox actually played it

Co-founder explains validating the data-labeling platform idea through customer discovery and the two signals used to confirm scalability. Gives a template for testing a contrarian bet.

Validate both the problem and the scale of the problem
A contrarian view needs early customer evidence

Built from Labelbox's Path to PMF: Founders Need to be Contrarian and Right by First Round Review.