Zendesk · AI product · Mid · 60 to 90 minutes

Zendesk AI Agent Trust and Handoff Design

Zendesk makes customer service software that helps businesses manage support across tickets, messaging, help centers, and automation. Answer this product management case and get it graded line by line against a weighted rubric.

User segmentation
AI quality design
Metric design
Prioritisation
Risk management

The situation

Your director of product for Zendesk AI has asked you to propose how the AI support agent should decide when to answer a customer directly, when to ask clarifying questions, and when to hand the conversation to a human agent. The ask comes after several large customers said they like the lower volume from automation, but they are worried that occasional wrong answers are harder to spot than no answer at all. Sales is hearing that trust, not feature breadth, is becoming the main blocker in some enterprise deals. Support leadership also says current reporting overstates success because a contained conversation is not always a correctly resolved one. Assume the following numbers for a mixed set of Zendesk customers: the AI agent attempts 40% of inbound support conversations, fully contains 24%, and about 7% of contained conversations later reopen, escalate, or receive clear negative feedback tied to a bad answer. You own this product area for digital support channels. The tension is that more aggressive automation improves efficiency and helps smaller teams, but a single confident wrong answer can damage customer trust, brand safety, and willingness to expand AI use.

What to produce

A 1 to 2 page product proposal.

Your answer must

Define the product scope and the primary user segments, including which customer types, ticket types, and channels you would include or exclude in the first release.
Propose a decision framework for when the AI agent should answer, ask a clarifying question, or hand off to a human, using at least three concrete signals or rules.
Define success metrics and guardrails, including how you will measure answer quality beyond simple containment or deflection.
Prioritise the MVP and the next phase, with clear trade-offs and a rationale for what you are not doing yet.
Describe a rollout and evaluation plan that includes risk controls, feedback loops, and how Zendesk would learn and improve after launch.

Constraints

You have one PM, five engineers, one data scientist, and shared design support for the next two quarters.
The first release must work in web messaging and email only, not voice or social channels.
You cannot change the underlying foundation model provider in this cycle, so your proposal should focus on product logic, orchestration, controls, and measurement.
Any use of conversation data must stay within existing customer privacy settings and regional data handling rules.

Format

Structure your answer with sections for scope, decision logic, metrics, roadmap, and rollout. Be explicit about trade-offs, thresholds, and what would cause the AI to step back from answering.