Mock system design interview at NVIDIA
A live 45-minute round on one of the questions reported in NVIDIA interviews, run by an AI interviewer that asks one thing at a time, probes what you say, and weighs what that round is reported to weigh. It ends with a debrief: a score, what was strong, what was missing.
Design an LLM Inference Service
Serve a large language model behind an API: queue the requests, batch them onto scarce GPUs, stream tokens back, and keep every accelerator busy without blowing the latency budget.
You will not see the requirements, the numbers or the diagram before you start — that is the point.
What the NVIDIA round is reported to look like
45–60 minutes, often close to accelerators: serving, scheduling or data movement for large models.
What it weighs
- Where the bottleneck actually is: memory bandwidth, interconnect or compute
- Batching and scheduling on scarce hardware
- Throughput and utilisation with numbers attached
- Failure handling in a fleet of expensive, long-running workers
The common mistake
Quoting hardware names without the arithmetic. The numbers are the answer here.
From public interview guides and candidate reports, not from NVIDIA. The interviewer is an AI running in that style; it is practice, not a prediction.
Start the round
Other companies: Anthropic · Google · Meta · Amazon · OpenAI · Microsoft · Uber · Stripe