Intercom · AI product · Senior · 45 to 60 minutes

How should Intercom launch an ML support bot?

Intercom makes customer messaging and support software that businesses use to talk with and support their customers. Answer this product management case and get it graded line by line against a weighted rubric.

AI product strategy
Risk prioritization
Experiment design
UX judgment
Cross functional sequencing

The situation

You are the PM leading a new AI support initiative at Intercom. Support teams using Intercom handle many repeated inbound questions, and leadership believes a bot that answers common questions could improve speed and efficiency. The VP of Product and head of machine learning want a recommendation before they commit a larger team. The problem is not just model accuracy. The bot would jump into live support conversations, so irrelevant answers could annoy end users and damage trust in both the business using Intercom and in Intercom itself. Even if the concept works for one company, Intercom serves tens of thousands of businesses, so each customer would need to set up, train, and maintain it without hands on help from Intercom. The team also does not know whether off the shelf language tools are already good enough for an initial launch, or whether a heavier custom ML effort will be needed. You need to propose the first version to pursue, how to test feasibility early, and when to scale investment versus stop.

What to produce

A 2 page product strategy memo.

Your answer must

Define the first shippable product envelope, including target user, core workflow, and what the bot will and will not do at launch.
Lay out a sequenced plan for reducing product, design, and ML risk over the first two quarters, including what to prototype before deeper engineering investment.
Explain how the experience should handle low confidence predictions and false positives so that imperfect model output is still acceptable to end users.
Propose how customers will set up and curate the bot at scale without manual services from Intercom.
Specify success metrics, feedback loops, and clear decision gates for staying with simpler technology, investing in custom ML, or stopping the effort.

Constraints

The model will make mistakes, and the plan cannot assume near perfect accuracy before launch.
The feature must work as a product for a large customer base, not as a concierge service with manual setup by Intercom.
The bot operates inside live support conversations, so trust and interruption cost are critical constraints.
The team should learn from real usage early rather than spending months in isolated research or design.

Format

Use sections titled product envelope, risk reduction plan, UX for imperfect predictions, customer setup, and metrics and decision gates.

How Intercom actually played it

Intercom's head of machine learning explains why building ML into a viable product differs from normal product development. The post shares lessons from Resolution Bot on scoping, managing uncertainty about what the model can do, and shipping something customers can trust. One of the better pieces on running an ML product process.

ML product work needs earlier proof of feasibility than usual features.
Design the product so imperfect model output is still useful.
Use customer feedback loops to improve accuracy over time.

Built from Building Resolution Bot: How to apply machine learning in product development by Fergal Reid.