Design YouTube

• Upload a video — Files up to several GB, resumable from a browser or phone, with a title, description and visibility. The video is watchable within minutes of the upload finishing.
• Watch a video — Playback starts in under two seconds anywhere in the world and adapts to the viewer's bandwidth without stalling. Seeking is instant.
• Video metadata and listing — Title, description, channel, duration, thumbnails, view count. Channel pages and a home page list videos; search is a separate system we only integrate with.
• View counting — Approximate counts on the page, accurate counts for creator analytics and payouts. Counts must resist simple replay inflation.
• Comments and likes — Modelled as a simple append and counter problem; we design the data path, not moderation.
• Visibility and takedown — Private, unlisted, public. A takedown must stop playback everywhere within minutes.
• Out of scope — Live streaming (mention how the pipeline changes), recommendations and ranking, search relevance, DRM licensing, monetisation and ads.

• Scale (1 B DAU · 5 B views/day · 500 h uploaded/min) — Reads outnumber writes by orders of magnitude and each read is megabytes per second. Bandwidth, not requests, is the cost.
• Start-up latency (< 2 s to first frame) — Requires the first segments to be at the edge and the manifest tiny. This drives the CDN and segment-size design.
• Smooth playback (rebuffer ratio < 0.5 %) — Adaptive bitrate must step down before the buffer empties. The hardest tradeoff: quality against stalls on variable networks.
• Upload to available (< 10 min for a 10-minute video) — Transcoding is parallelised by chunk so wall-clock time does not scale with duration.
• Durability (no lost uploads) — The original is the only copy the creator may have. It is written to replicated object storage before any acknowledgement.
• Availability (99.99 % for playback) — Playback degrades gracefully: if metadata services are down, cached manifests still play.
• Cost (egress and storage dominate) — Every design decision is checked against bytes stored per video and bytes shipped per view.

• Views per second: ~60 k avg · ~200 k peak — 5 B views/day ÷ 86 400 s ≈ ~58 k/s. Peak 3× in evening hours per region. Each view is a manifest fetch plus tens to hundreds of segment fetches from the CDN.
• Egress bandwidth: ~150 Tbit/s — Assume an average bitrate of 3 Mbit/s across quality levels and 50 M concurrent viewers at peak: 50 M × 3 Mbit/s = 150 Tbit/s. This is why the CDN is the design, not a box on it.
• Upload volume per day: ~720 k hours · ~2 PB raw — 500 h/min × 1 440 min = 720 000 h/day. Raw uploads average ~3 GB per hour of video (phone 1080p), so ~2 PB/day of originals arrive.
• Transcoded storage per day: ~1.5 PB — Each video becomes ~6 renditions (144p to 4K). Total output is roughly 0.7× the original size after modern codecs: 2 PB × 0.7 ≈ 1.5 PB/day, ~550 PB/year plus originals kept cold. Storage tiering is mandatory.
• Transcoding compute: ~60 k cores continuously — Transcoding 1 h of video into all renditions takes roughly 2 core-hours of a modern CPU (hardware encoders are faster). 720 k h/day × 2 = 1.44 M core-hours/day ÷ 24 ≈ 60 k cores busy all day, more at peak upload hours.
• Segments per view: ~150 — A 10-minute video in 4-second segments is 150 segments per rendition. A viewer fetches ~150 segment files plus a manifest and, on quality switches, a few extra. At 60 k views/s that is ~9 M segment requests/s at the edge.
• View events per day: 5 B · ~1 TB — One view event of ~200 B per view: 5 B × 200 B = 1 TB/day of events, plus heartbeat events for watch time (every 30 s while playing), roughly 10× that. Fine for a streaming pipeline; not fine for a row-per-event database.

• Player / uploader — The web or mobile app. Uploads in resumable chunks directly to object storage using pre-signed URLs. Plays video with an adaptive-bitrate player that reads a manifest and fetches segments from the CDN, switching quality based on measured throughput and buffer level.
• CDN (multi-tier edge cache) — Serves manifests, segments and thumbnails from PoPs near viewers. Multi-tier: edge → regional shield → origin, so a popular video is fetched from origin once per region. Signed URLs enforce visibility; short TTLs on manifests make takedowns fast.
• API gateway — Authenticates, rate limits, and routes upload, metadata and engagement calls. Playback bytes never pass through it.
• Upload service — Creates the video record, issues pre-signed multipart upload URLs, tracks chunk completion for resumability, and on completion validates the file and enqueues transcoding. Never touches the bytes itself.
• Raw store (S3 / GCS · originals) — Originals as uploaded, replicated across zones. Written before any acknowledgement, retained in a cold tier after transcoding so re-encoding with a better codec is possible later.
• Transcode queue (Kafka / SQS · per chunk) — One job per (video, chunk, rendition). A 10-minute video becomes hundreds of small independent jobs, which is what makes wall-clock transcoding time independent of duration. Priority lanes for popular channels.
• Transcoder fleet (ffmpeg / hardware encoders) — Stateless workers that pull a job, fetch the source chunk, encode one rendition, and write the segment. Also produce thumbnails, audio tracks and subtitles. Autoscaled on queue depth; spot instances are fine because jobs are small and idempotent.
• Pipeline orchestrator (DAG per video) — Splits the original into chunks at keyframes, fans out jobs, tracks completion per rendition, then stitches segment lists into manifests and marks the video ready. Handles retries and partial failure; the state machine is durable.
• Segment store (object storage · CDN origin) — Transcoded segments (4 s each) per rendition, plus manifests and thumbnails, laid out as immutable objects under a content-addressed path. Origin for the CDN. Tiered: hot for recent and popular, cold for the long tail.
• Metadata service — Video and channel records: title, description, status, visibility, renditions available, duration, thumbnail ids. Serves the watch page and listing pages. Emits change events on publish and takedown.
• Metadata DB (sharded MySQL / Spanner · by video id) — Source of truth for videos, channels, and visibility. Sharded by video id; channel → videos is a secondary index table. Vitess-style sharded MySQL is what YouTube actually runs.
• Metadata cache (Redis · watch page objects) — Watch-page metadata for the hot set of videos. A video page is read millions of times per write, so the cache hit rate is near 100 %. Invalidated on publish, edit and takedown.
• View event stream (Kafka · key=video id) — View starts, heartbeats and engagement events from players, batched by the client and acknowledged by an ingest endpoint. Feeds counting, analytics and recommendations.
• View counter (stream aggregation + dedupe) — Deduplicates view events per (viewer, video, window), filters obvious bots, and maintains approximate public counts (updated every few seconds) and exact daily counts for analytics. Public counts are eventually consistent by design.
• Analytics store (ClickHouse / BigQuery) — Watch time, retention curves, traffic sources per video per day. Written in batches from the stream; read by creator dashboards, never by the watch page.

1. Upload a video — Bytes go straight to object storage; the services only coordinate. Resumable, durable before acknowledgement, and the pipeline starts the moment the last chunk lands.
   1. Client creates the video and asks for upload URLs.
   2. Client uploads chunks in parallel directly to the raw store, retrying any that fail.
   3. Client signals completion; the upload service validates the assembled object.
   4. Upload service marks the video "processing" and starts the pipeline.
   5. Client polls or subscribes for status while the pipeline runs.
2. Transcode into renditions — Split the original at keyframes, encode every (chunk, rendition) pair independently on a large stateless fleet, and stitch the results into manifests. Parallelism makes a two-hour film finish in minutes.
   1. Orchestrator inspects the original and splits it into chunks at keyframe boundaries.
   2. Orchestrator fans out one job per chunk and rendition onto the queue.
   3. Transcoder workers pull jobs, read the chunk, encode, and write the segment to the segment store.
   4. Workers report completion; the orchestrator tracks progress per rendition.
   5. Orchestrator writes the manifests and marks renditions ready in metadata.
   6. Popular-channel videos are pre-pushed to regional CDN shields before publish.
3. Watch a video — Metadata from a cache, bytes from the CDN, quality chosen by the player. The origin is touched only on cache misses; the watch page is the same for everyone so it caches perfectly.
   1. Client requests the watch page metadata.
   2. Player fetches the master manifest and the first rendition playlist from the CDN.
   3. Player fetches segments sequentially, keeping a buffer of ~30 s ahead of the playhead.
   4. On a CDN miss, the edge fetches from the regional shield, which fetches from the segment store once.
   5. Player measures throughput and buffer, and switches rendition up or down between segments.
   6. Player sends a view-start event and periodic heartbeats to the event stream.
4. A video goes viral: 5 M concurrent viewers — One video, one manifest, 150 segments per rendition, five million players. The CDN absorbs it by design; the only origin-side work is keeping metadata and counts from becoming hot spots.
   1. Millions of players request the same manifest and segments across every PoP.
   2. Regional shields coalesce misses so origin sees at most one request per object per region.
   3. The watch-page metadata becomes a hot cache key.
   4. View events for one video hit one Kafka partition.
   5. The approximate count on the page updates every few seconds; exact counts settle later.
5. Takedown and visibility change — Bytes are cached in a thousand places with year-long TTLs. The design makes the manifest, not the segments, the enforcement point, and signs URLs where it matters.
   1. A takedown or visibility change is written to the metadata DB.
   2. Metadata service invalidates the watch-page cache and purges the manifest from the CDN.
   3. Segment URLs for non-public videos carry short-lived signatures, so already-issued links stop working.
   4. Sessions already playing receive a stop signal on their next heartbeat response.
   5. Objects are moved to a legal-hold tier or deleted after the retention period.

Ask: Why not transcode each video as one job? How do you make a two-hour upload available in minutes?

Good answers name: Split at keyframes, one job per (chunk, rendition), orchestrator stitches, One job per (video, rendition), One job per video, all renditions in one process, Managed transcoding service (AWS MediaConvert, GCP Transcoder).

Our pick: Chunk at keyframes into ~10-second pieces, aligned across renditions so segments line up for adaptive switching. One job per (chunk, rendition), hundreds per video, on a stateless autoscaled fleet pulling from a priority queue; workers write CMAF fragments to deterministic paths, so any job can be re-run safely. An orchestrator persists the plan and progress per video (a small durable state table or a workflow engine) and writes each rendition's playlist when it completes, low renditions first, publishing the video as soon as one rendition exists. A global analysis pass (complexity per scene) feeds per-chunk bitrate targets to keep quality consistent. This is the shape of YouTube's and Netflix's pipelines.

1. A worker dies halfway through a chunk. What happens?
   The job's lease expires (visibility timeout or Kafka rebalance) and another worker picks it up. Output paths are deterministic so the partial object, if any, is overwritten. The orchestrator only marks a chunk done on an explicit completion message that names the object, and verifies the object exists before writing the playlist. No coordination beyond that.
2. Uploads spike 10× for an event. What breaks?
   Queue depth grows and time-to-available stretches; the fleet autoscales on depth, and spot capacity is elastic but not infinite. Priority lanes ensure short videos and large channels stay fast. Beyond that, degrade: publish at 360p first and queue high renditions for later, and defer 4K entirely during the spike. Nothing is lost; the queue is durable.
3. Why fragmented MP4 (CMAF) rather than separate HLS TS and DASH outputs?
   One encode serves both HLS and DASH players, halving storage and encode cost. CMAF segments with byte-range addressing also let you store one file per rendition and serve segments as ranges, cutting object count by 150×. The cost is dropping very old players that only speak MPEG-TS.
4. How do you keep quality consistent across independently encoded chunks?
   Constant-quality encoding (CRF) rather than constant bitrate, so each chunk gets the bits it needs. A cheap first pass over the whole video produces a per-scene complexity map that feeds a per-chunk bitrate cap, keeping the rendition within its declared bandwidth. Encode chunk boundaries with a small overlap and trim, so the joins are invisible.
5. How would live streaming change this?
   The source arrives continuously, so the pipeline runs on a sliding window: encode each 2 to 4-second segment as it arrives, in parallel across renditions but sequentially in time, and append to a growing playlist. Latency is segment length times buffer depth (6 to 12 s typical; LL-HLS gets to 2 to 3 s with partial segments). Chunk-level parallelism is replaced by rendition-level parallelism, and the fleet is sized for concurrent streams rather than backlog.

Ask: How do you start playback in under two seconds and avoid rebuffering on a network that varies second to second?

Good answers name: HLS/DASH with 4-second aligned segments, player-side ABR, Server-side ABR (server picks the quality), Progressive download of one file per rendition, WebRTC.

Our pick: 4-second segments (2 s for live), keyframe-aligned across a rendition ladder of roughly 144p/200 kbit/s to 4K/15 Mbit/s in six or seven steps, exposed via a master manifest. The player starts at a low-middle rendition for a fast first frame, keeps a 30-second forward buffer, estimates throughput from recent segment downloads (harmonic mean to resist spikes), and switches up only with headroom (measured ≥ 1.5× the target bitrate) and down as soon as the buffer falls under ~10 s. Per-title encoding (a ladder tuned to each video's complexity, as Netflix does) saves 20 to 30 % of egress at the same quality. Start-up is under 2 s because the manifest and first segment total under 1 MB and are at the edge.

1. Why not 1-second segments for faster switching?
   Each segment needs a keyframe, and keyframes cost bits: 1-second GOPs raise bitrate 10 to 20 % at the same quality. Requests per view also rise fourfold and the CDN's per-request overhead becomes visible. Four seconds is a common compromise for VOD; live uses 2 s, and LL-HLS uses partial segments to get lower latency without shrinking the GOP.
2. Viewers on a 2 Mbit/s link keep rebuffering. Where do you look?
   First the heartbeat data: which rendition they were on when they rebuffered and what the estimate said. Common causes: the ABR is too aggressive switching up (raise the headroom factor), the ladder has a gap (add a 600 kbit/s rung), or the buffer target is too small for their jitter. Then the CDN: cache hit rate for their region and PoP latency; a miss to a far origin at the start of a segment can stall a thin buffer.
3. How do you measure playback quality overall?
   Rebuffer ratio (stalled time / watch time), time to first frame, average bitrate delivered, and switch frequency, all from player heartbeats, aggregated by region, ISP, device and CDN PoP. Alert on rebuffer ratio per region. A/B test ABR changes with these metrics; they are the product's quality SLOs.
4. Does the CDN change behaviour for 4K or HDR?
   No, just bigger objects: a 4K segment is ~8 MB. What changes is the share of egress: a few percent of viewers on 4K can be a third of the bytes. Per-title ladders and newer codecs (AV1 at roughly 30 % fewer bits than H.264 for the same quality) are how the cost is contained, and they are worth the extra encode CPU only for videos with enough views to amortise it.

Ask: Half a petabyte a day forever. How do you store it, and how do you keep the bill sane?

Good answers name: Immutable objects per (video, rendition, segment), tiered by view age, One file per rendition served via byte ranges, Keep only the original, transcode on demand, Custom distributed file system.

Our pick: Object storage with a path per video: seg/{video_id}/{rendition}/{n}.m4s plus manifests and thumbnails, all immutable; a re-encode writes under a new version prefix and the manifest is repointed. Originals go to the archive tier once transcoding succeeds. Renditions are tiered by a daily job on last-view time: hot for 30 days or any video with recent views, infrequent-access after that, and for videos with no views in a year, drop the 1080p and 4K renditions (re-encode from the original on demand) and archive the rest. The CDN's regional shields hold the working set, so the origin tier mostly serves first-in-region views. Erasure coding within the object store keeps durability at 11 nines at ~1.5× overhead instead of 3× replication. Expect storage to be the second-largest cost after egress and to be dominated by the tail.

1. A cold, archived video gets shared and starts trending. What does the viewer see?
   The manifest lists only renditions that are immediately servable; archived ones are hidden. The first view triggers a restore of the hot renditions (minutes for archive tiers), or, if only the original remains, a priority re-encode of a low rendition, which finishes in under a minute for most videos. Meanwhile the viewer gets the best rendition available. The tiering job treats the spike as "recent views" and keeps it hot again.
2. How do you delete a video with 900 objects atomically?
   You do not need atomicity; you need the manifest gone first. Remove the manifest and the metadata row (playback stops), then an idempotent async job lists the prefix and deletes in batches, retrying until the listing is empty. Legal deletion is confirmed when the listing is empty and the CDN purge has completed.
3. Should thumbnails and previews live with the segments?
   Same store, separate prefix, always hot: thumbnails are viewed far more than videos (every listing page shows dozens) and are tiny. Generate several sizes at transcode time, serve through the CDN with long TTLs, and version the path when the creator changes the thumbnail.
4. How much does a new codec save and when is it worth it?
   AV1 or VP9 versus H.264: 30 to 50 % fewer bits at equal quality, so proportionally less egress and storage, at 5 to 10× the encode CPU. Encode the new codec only for videos above a view threshold (the head), where the egress savings dwarf the encode cost, and keep H.264 as the universal fallback rendition. Decide per video from its view rate; that is what the large services do.

Ask: Why is the view count "approximate", and how do you count accurately for creator payouts without inflating on replay?

Good answers name: Stream aggregation: dedupe per (viewer, video, window), approximate public count, exact batch count, Increment a counter in the metadata DB per view, Redis INCR per view, periodic flush, Count from logs in a nightly batch only.

Our pick: Players send view_start and 30-second heartbeats to an ingest endpoint that batches into Kafka keyed by video (salted for hot videos). A stream job counts a view when a session reaches 30 s of watch time, de-duplicates on (viewer id or device hash, video, 24 h window) using state in RocksDB, and drops sessions failing cheap bot heuristics (no heartbeats, impossible rates per IP). It emits approximate totals per video every 5 s to the metadata cache (the number on the page, labelled as approximate) and exact daily aggregates to the analytics store, where a slower fraud model can retroactively adjust before payouts are calculated. YouTube shows counts frozen at 301 for a while on new videos for exactly this reason: verification lags display.

1. The dedupe state for a viral video is millions of entries. Does the stream job cope?
   Per-key state is a (viewer, video) set with a 24-hour TTL; 5 M viewers × ~40 bytes is 200 MB for one video, fine on one task, and the salted partitioning spreads it across 16 tasks. Use a Bloom filter per (video, hour) if memory becomes an issue, accepting a small false-positive rate that under-counts slightly.
2. Someone writes a script that opens the video 10 000 times. What stops the count?
   Dedupe by viewer or device within 24 h removes repeats from one identity; per-IP rate limits and the heartbeat requirement remove bots that do not actually play; and the batch fraud model catches distributed attacks by pattern (many new devices from one ASN, uniform watch times). Views are tentatively counted and later removed, so the public number can go down.
3. Why not count the view in the CDN logs instead of client events?
   CDN logs tell you a segment was fetched, not that it was watched, and prefetching and bots make that noisy. Client heartbeats give watch time, position, rebuffering and rendition, which you need for analytics anyway. Use CDN logs as a cross-check for gross discrepancies.
4. How do you show "watching now" for a live stream?
   A per-video HyperLogLog or a simple counter in Redis updated by heartbeats with a 60-second expiry window; read every few seconds by the page. Approximate, lossy, and separate from the durable view count.

Ask: What holds video and channel metadata, how is it sharded, and why does the watch page not hit it?

Good answers name: Sharded relational (Vitess/MySQL or Spanner) by video id, Redis cache, CDN for public JSON, Wide-column store (Cassandra) keyed by video id, Document store, Single Postgres with replicas.

Our pick: A videos table sharded by video id (random 64-bit id, so shards are uniform), a channel_videos index table sharded by channel id and written in the same transaction via the outbox relay, and a status index for moderation queues. Redis holds the rendered watch-page object per video with a 1-hour TTL and explicit invalidation on edit, publish and takedown; hot keys are replicated across shards. Public watch-page JSON is additionally cacheable at the CDN for 10 s, which absorbs viral spikes before they reach Redis. Reads of a creator's own video go to the primary for 5 s after an edit so they see their change (read-your-writes). Counts are not stored in this table; they come from the counter pipeline into the cache.

1. A creator edits the title and still sees the old one. Why, and what is the fix?
   They hit the cache or a replica before invalidation propagated. Fix: after a write, invalidate the cache synchronously before returning, and route the writer's reads to the primary for a few seconds via a short-lived cookie or header. Other viewers may see the old title for up to the CDN TTL, which is acceptable.
2. How do you list a channel's 20 000 videos newest-first?
   From the channel_videos index table keyed by channel id and ordered by a time-ordered video id (or published_at), with cursor pagination. It is written through the outbox when a video publishes, so it can lag the primary by a second. Never scan the videos table across shards.
3. Where do comments and likes live?
   Comments in a separate store keyed by video id, ordered by time, with a per-video partition (Cassandra or a sharded MySQL table by video id): append-heavy, read as a page per video. Likes are a counter problem like views: events into the stream, aggregated counts into the cache, and a per-user like set in Redis or a keyed table for the "you liked this" state.
4. How would you make the watch page survive a metadata outage?
   The CDN serves stale cached JSON (stale-if-error), the player has the manifest URL and plays from the CDN without any service, and view events buffer client-side. Only new uploads, edits and cold videos fail. Playback availability is decoupled from the control plane by design.