AI & Technical question
An agentic workflow has inconsistent model behavior: for the same user intent, sometimes the agent takes efficient steps and sometimes it opens irrelevant pages or asks unnecessary follow-ups. User feedback is mixed because some sessions feel magical and others fail badly. How would you work with research, engineering, and data science to define quality, instrument the workflow, and decide what to fix first?
- Perplexity
- AI & Technical
- Hard
Practice this question out loud. An AI interviewer asks it, follows up like a real interviewer would, and scores your answer. Type or speak.
Start a mock interview on this question · Mock interview from a job description
The answer guide for this question is on its way. You can still practice it now.
More ai & technical questions
- You launch a new artifact-generation capability, and users say the outputs are occasionally excellent but too inconsistent to rely on. How would you define quality, build an evaluation framework across model and product layers, and decide whether to improve, constrain, or roll back the feature?Perplexity · AI & Technical · Hard
- A Computer workflow saves users time, but in 5-10% of sessions the agent takes the wrong action or needs correction. How would you define product quality for this workflow, set launch thresholds, and design an eval and feedback loop to improve reliability after launch?Perplexity · AI & Technical · Hard
- Design a simple load balancer for Google.com. What data structures would you use?Google · AI & Technical · Hard
- You’re part of the Google Search web spam team. How would you detect duplicate websites?Google · AI & Technical · Hard
- Build a system to predict if something can go wrong in a cab ride.Lyft · AI & Technical · Hard
- Explain the data pipeline for the last AI project you worked on. What were the top challenges in getting data, and how did you resolve them?Google · AI & Technical · Hard
More questions from Perplexity
Learn the skill behind it
Chapters of the AI PM course that teach what this question tests.
- Chapter 1: Foundations: the model and the decisions it forces on you
- Chapter 8: Evals: define good and make the number defensible
- Chapter 6: Agents and agentic architecture