Estimation question

Estimate the cost advantage of Groq's LPU vs. NVIDIA H100 for a high-volume workload.

Practice this question out loud. An AI interviewer asks it, follows up like a real interviewer would, and scores your answer. Type or speak.

What this question tests

Structured cost-comparison estimation across two different hardware architectures for a defined workload.

Define the workload: assume a high-volume inference task, say serving 1 billion tokens per day for a mid-size language model.
Estimate H100 throughput and cost: assume an H100 processes roughly 2,000 to 4,000 tokens per second for this model size, at a cloud rental cost of roughly 2 to 4 dollars per GPU-hour.
Estimate LPU throughput and cost: assume Groq's LPU delivers a meaningfully higher tokens-per-second rate for the same model, say 3 to 5 times higher, due to its architecture, at a comparable or somewhat higher per-chip hourly cost.
Compute cost per million tokens for each: divide hourly cost by tokens processed per hour, showing the LPU's higher throughput can offset its potentially higher hourly rate, yielding a lower cost per token overall.
State the conclusion as a range: assume the LPU comes out roughly 30 to 50 percent cheaper per token at high volume, clearly marked as an illustrative estimate, not a verified figure.

Builds the comparison on cost per token rather than cost per hour, since hourly chip rental price alone is not what a high-volume customer actually pays for.
States both throughput and hourly cost assumptions explicitly for each chip, showing the tradeoff rather than asserting Groq wins without arithmetic.
Arrives at a clear range, not a false-precision single number, and labels it explicitly as an estimate based on assumed figures.
Notes the advantage is workload-dependent, since batch training workloads may favor H100s while low-latency serving favors the LPU.

Comparing hourly rental price alone without normalizing for throughput differences between the two architectures.
Presenting a specific number as if it were a verified fact rather than an assumption-based estimate.

How would this comparison change for a training workload instead of inference?
What would happen to this cost advantage if NVIDIA cut H100 rental prices by half?