AI & Technical question
Design an evaluation framework for Figma’s AI-powered Design↔Code roundtripping workflows. How would you define the golden datasets and annotation pipeline, separate model-quality issues from product-UX issues, choose offline and online metrics, and set ship/no-ship thresholds for accuracy, editability, and user trust?
- Figma
- AI & Technical
- Hard
Practice this question out loud. An AI interviewer asks it, follows up like a real interviewer would, and scores your answer. Type or speak.
Start a mock interview on this question · Mock interview from a job description
What this question tests
Whether you can design a rigorous eval framework for a generative feature, separating model quality from product UX and choosing metrics that gate a real ship decision.
How to approach it
- Define the golden dataset: real design files and real codebases spanning common component patterns, edge cases like nested auto layout, and known failure cases from support tickets.
- Build an annotation pipeline where designers and engineers independently label correctness, since design fidelity and code correctness need different expertise.
- Separate model-quality issues, like wrong component mapping, from UX issues, like a correct conversion the user cannot easily review or edit.
- Pick offline metrics like structural accuracy against the golden set and editability (can a human fix a wrong section without starting over), plus online metrics like accepted-diff rate.
- Set ship thresholds per metric, for example a minimum structural accuracy plus a maximum rate of edits that silently break the design, and require both before shipping.
What a strong answer includes
- Distinguishes model failures from UX failures explicitly, since a correct model output with unreviewable UI is still a shipped failure.
- Defines editability as its own metric, not just accuracy, since roundtripping tools live or die on how easy corrections are.
- Sets numeric thresholds tied to specific metrics rather than a vague quality bar.
Common mistakes
- Treats accuracy as the only metric and ignores editability or trust.
- No plan for how annotators are trained or how disagreement between them is resolved.
Likely follow-up questions
- How would you handle disagreement between designer and engineer annotators.
- What would you do if accuracy is high but adoption stays low.
More ai & technical questions
- Design to Code quality is strong on Figma-owned demos but inconsistent across real customer repositories. How would you design the evaluation system for this product: define the task taxonomy, build the annotation pipeline, choose offline and human-in-the-loop quality measures, and set clear ship/no-ship criteria before expanding availability?Figma · AI & Technical · Hard
- Design the core of Figma’s external platform for agents and developer tools. How should the MCP Server and REST APIs expose file context, permissions, freshness, rate limits, versioning, and observability so partners like Claude or Cursor can build reliably on top of Figma without creating fragility for internal product teams using the same platform?Figma · AI & Technical · Hard
- Teams using Figma’s design-to-code workflow say the output is inconsistent, so they do not trust it for production work. How would you break the problem down across SceneGraph fidelity, component and token mapping, and code generation; what quality metrics or evals would you put in place; and how would you decide which issue to fix first?Figma · AI & Technical · Hard
- Design a simple load balancer for Google.com. What data structures would you use?Google · AI & Technical · Hard
- You’re part of the Google Search web spam team. How would you detect duplicate websites?Google · AI & Technical · Hard
- Build a system to predict if something can go wrong in a cab ride.Lyft · AI & Technical · Hard
More questions from Figma
Learn the skill behind it
Chapters of the AI PM course that teach what this question tests.
- Chapter 1: Foundations: the model and the decisions it forces on you
- Chapter 8: Evals: define good and make the number defensible
- Chapter 6: Agents and agentic architecture