Metrics question

Figma Weave can help teams generate many variations within defined brand or creative systems. What north-star and supporting metrics would you use to measure success across workflow adoption, output quality, scalability, and user impact, and how would you operationalize 'quality' when the output is creative and partly subjective?

Practice this question out loud. An AI interviewer asks it, follows up like a real interviewer would, and scores your answer. Type or speak.

What this question tests

Tests defining a north star and supporting metrics for a generative creative tool, and operationalizing subjective quality into something measurable.

Propose a north star candidate: the number of generated variations that get used in a final, shipped design asset, since it ties generation directly to real creative output rather than raw generation volume.
Support with workflow adoption metrics: percent of eligible teams using Weave within a brand system, and generations per active project, to show breadth and depth of adoption.
Support with a quality proxy: acceptance rate, the percent of generated variations kept or lightly edited versus fully discarded, as a measurable stand-in for a partly subjective quality judgment.
Support with scalability metrics: average time to produce a defined number of on-brand variations compared to a manual baseline, showing the tool's core promise of scale is real.
Operationalize quality further with periodic structured human review, a small panel rating a sample of outputs against defined brand and craft criteria, to validate that the acceptance-rate proxy still reflects real quality over time.
Watch for a vanity-metric trap: raw generation count can rise while acceptance rate falls, so always report them together, not generation volume alone.

Chooses a north star tied to real usage in shipped work, not raw generation count, avoiding an easily inflated vanity metric.
Operationalizes subjective quality with a concrete proxy, acceptance or light-edit rate, paired with periodic structured human review to keep the proxy honest.
Explicitly pairs generation volume with acceptance rate, guarding against a scenario where output quantity rises while real usefulness falls.
Includes a scalability metric that directly measures the tool's core value proposition, speed at scale, against a manual baseline.

Using raw generation count as the north star, which can rise even as quality or usefulness declines.
Treating quality as unmeasurable and skipping any proxy or structured review process for it.
Reporting adoption metrics without any quality or acceptance signal, missing whether output is actually useful.

How often would you run the structured human quality review, and who would do it?
What would you do if acceptance rate is high but few generations are used in truly final shipped work?