Product design question
How would you prevent 'bad' content from being uploaded to a social media platform?
- Medium
- Product design
- Medium
Practice this question out loud. An AI interviewer asks it, follows up like a real interviewer would, and scores your answer. Type or speak.
Start a mock interview on this question · Mock interview from a job description
What this question tests
Tests trust and safety product thinking: can you design a layered moderation system (prevention, detection, enforcement) rather than a single filter.
How to approach it
- Define 'bad content' concretely: split into categories like graphic violence, hate speech, spam/scams, and misinformation, since each needs different detection methods and thresholds.
- Design prevention at upload: automated classifiers (image and text) that scan content before it is fully public, catching clear violations like known CSAM hashes or explicit nudity instantly.
- Design detection after upload: a mix of ML classifiers for borderline content plus user reporting, since some harmful content (context-dependent hate speech, misinformation) is hard to catch pre-publish with certainty.
- Design enforcement: a tiered response from warning and reduced distribution for borderline content, to removal and account strikes for clear violations, avoiding a single blunt ban-or-allow decision.
- Add human review for edge cases and appeals, since automated systems will have false positives that harm legitimate users if there's no recourse.
- Define success metrics: prevalence of violating content (percent of views that are policy-violating), time-to-removal, and false-positive rate on removed content.
What a strong answer includes
- Separates content categories explicitly, since spam detection and hate-speech detection use fundamentally different signals and thresholds.
- Distinguishes pre-publish automated blocking (for clear-cut violations) from post-publish review (for context-dependent content), which mirrors how real platforms like Meta and YouTube operate.
- Includes an appeals process, showing awareness that automated moderation has real false-positive costs to legitimate creators.
- Uses prevalence (percent of views on violating content) as the core metric, which is the industry-standard way trust and safety teams measure this rather than raw takedown counts.
Common mistakes
- Proposing a single AI filter that blocks everything automatically, ignoring context-dependent content and appeals.
- No metric beyond 'remove bad content', missing how the team would actually measure success.
- Ignoring the false-positive cost to legitimate users from over-aggressive moderation.
Likely follow-up questions
- How would you handle content that is borderline but not clearly violating?
- How would you balance moderation speed against accuracy for live content?
- What would you do if moderators disagreed with the AI classifier's decision?
More product design questions
- What is that one product feature that would change the landscape of Devops/GitOps?Google · Product design · Hard
- How would you improve Outlook for the use case when people get overwhelmed by number of emails received after returning from a vacation?Microsoft · Product design · Medium
- You’ve just joined Meta and have been tasked with launching a new product to help people at home during emergencies. What would you build?Meta · Product design · Medium
- How do you design a camera for the elderly?Google · Product design · Medium
- Design a new feature for Twitter that improves the new user’s experience.Twitter · Product design · Medium
- How would you go about designing a new location sharing app for Google?Google · Product design · Medium
More questions from Medium
Learn the skill behind it
Chapters of the AI PM course that teach what this question tests.
- Chapter 4: Discovery and strategy for AI products
- Chapter 7: AI UX and human oversight: design for a system that is wrong sometimes
- Chapter 14: Get the job: the AI PM interview loop