Mock system design interview at NVIDIA

A live 45-minute round on a question reported in NVIDIA interviews, run by an AI interviewer that asks one thing at a time, probes what you say, looks at your whiteboard, and weighs what that round is reported to weigh. It ends with a debrief: a score, what was strong, what was missing.

your question

Design an LLM Inference Service

Serve a large language model behind an API: queue the requests, batch them onto scarce GPUs, stream tokens back, and keep every accelerator busy without blowing the latency budget.

You will not see the requirements, the numbers or the design before you start. That is the point.

Walk through it first

What the NVIDIA round is reported to look like

45–60 minutes, often close to accelerators: serving, scheduling or data movement for large models.

What it weighs

Where the bottleneck actually is: memory bandwidth, interconnect or compute
Batching and scheduling on scarce hardware
Throughput and utilisation with numbers attached
Failure handling in a fleet of expensive, long-running workers

The common mistake

Quoting hardware names without the arithmetic. The numbers are the answer here.

From public interview guides and candidate reports, and NVIDIA’s own interview guidance where it publishes any. Not endorsed by NVIDIA. The interviewer is an AI running in that style; it is practice, not a prediction.

Start the round

Timed · 45 minutesInterview conditions. The AI interviewer owns the clock; the AI Mentor is off.With the AI Mentor · untimedNo clock, and an AI Mentor beside you to ask when you are stuck.

The interviewer’s opening question is free. Answering it is part of Pro. See an example round and its debrief →

The 3 questions this round draws from

Each is reported in NVIDIA interviews. Pick one to sit, or read its reference design and walk through a strong answer first.

Design an LLM Inference ServiceServe a large language model behind an API: queue the requests, batch them onto scarce GPUs, stream tokens back, and keep every accelerator busy without blowing the latency budget.
drawn
Design a Large File Distribution SystemGet a 200 GB checkpoint from one origin onto 10,000 machines in five data centres within minutes, under capped links, verifying every byte.
Design a Container OrchestratorPlace 300,000 containers on 10,000 machines from declared desired state, keep them running through failures, and give every service a stable address.

Other companies: Anthropic · Google · Meta · Amazon · OpenAI · Microsoft · Uber · DoorDash