System Design Prep
← All companies

NVIDIA system design interview questions

One question on this site is reported in NVIDIA software engineer interviews. Each is a guided run through requirements, back-of-envelope numbers, an interactive design, user flows and deep dives with interviewer follow-ups. Patterns covered: gpu-batching, streaming, scheduling, kv-cache.

Run an AI mock NVIDIA interview →One of these questions, run as a live round in that style, with a debrief at the end.

Attribution comes from public interview guides and candidate reports, not from NVIDIA. It indicates where a question has been asked, nothing more.

🔒 Pro
hardgpu-batchingstreamingschedulingkv-cache

Design an LLM Inference Service

Serve a large language model behind an API: queue the requests, batch them onto scarce GPUs, stream tokens back, and keep every accelerator busy without blowing the latency budget.

asked at
15 components5 flows6 deep dives
AI interviewStudyPractice
not started

What the NVIDIA round looks like

45–60 minutes, often close to accelerators: serving, scheduling or data movement for large models.

What it is reported to weigh

  • Where the bottleneck actually is: memory bandwidth, interconnect or compute
  • Batching and scheduling on scarce hardware
  • Throughput and utilisation with numbers attached
  • Failure handling in a fleet of expensive, long-running workers

The common mistake

Quoting hardware names without the arithmetic. The numbers are the answer here.

From public interview guides and candidate reports, not from NVIDIA. Formats change; treat it as a prior, not a promise.

How to prepare for a NVIDIA system design interview

Large-company loops run 45 to 60 minutes, expect numbers, and spend most of the time on deep dives. Work each question above in practice mode until you can produce the requirements, the estimates and the flows without looking, then read the interview framework guide for how to structure the hour. The other questions on the site teach patterns that show up in NVIDIA interviews under different names.