Metrics question

What metrics matter most for an inference-first cloud like Together?

Practice this question out loud. An AI interviewer asks it, follows up like a real interviewer would, and scores your answer. Type or speak.

What this question tests

Whether the candidate can identify the metrics that actually drive purchase decisions in an infrastructure business, not generic product metrics.

Clarify what inference-first means: customers care primarily about serving models fast, cheaply, and reliably in production, not training.
Propose the core metric triad: latency (time to first token and tokens per second), cost per million tokens, and uptime or error rate.
Propose a business-facing metric: gross margin per workload type, since serverless and dedicated capacity have very different cost structures.
Propose a growth metric: net revenue retention among existing customers, since infrastructure businesses grow mainly through expanding usage of live customers.
Tie it together: explain that customers switch providers primarily over price-performance and reliability, so those must be the dashboard's top-line numbers.

Names the specific technical metrics that matter for inference, latency, throughput, and cost per token, rather than generic engagement metrics.
Connects gross margin per workload type to the business model directly, since serverless and dedicated clusters have very different economics.
Proposes net revenue retention as the key growth signal, since infrastructure businesses expand mostly through existing customer usage growth, not just new logos.
Flags a guardrail metric, like uptime or error rate, since a fast but unreliable service loses enterprise trust quickly.

Proposing generic SaaS metrics like DAU or session length that do not fit an infrastructure and API business.
Ignoring cost per token and margin, which are central to whether the business is actually healthy.

Which metric would you prioritize if latency and cost improvements traded off against each other?
How would you measure reliability in a way that reflects real customer pain, not just server uptime?