Design Twitch

Ingest 100 k live broadcasts, transcode them in real time, and deliver each about three seconds behind live, to three viewers or 500 k.

Last updated 2026-10-26. Difficulty: hard. Patterns: live-video, low-latency, transcoding, cdn, request-coalescing. Reported at Amazon, Google, Meta, TikTok, Netflix, Microsoft, Discord.

Functional requirements

• Go live from broadcasting software. OBS, a console or a phone app pushes one high-quality stream authenticated by the channel’s secret stream key: RTMP for most encoders, SRT or WebRTC (WHIP) where supported. The channel is watchable within a few seconds of the first frame.
• Watch live, a few seconds behind. About 3 seconds glass to glass by default, so chat and the stream feel like one conversation. A normal-latency mode (around 10 s) for players that cannot do low latency and viewers on poor networks.
• Quality that suits the connection. A ladder from 160p up to the source, chosen automatically by the player part by part, plus a manual quality menu and audio-only.
• Any audience size. Most channels have a handful of viewers; a few have hundreds of thousands, and a raid can move 50 k viewers onto a small channel in seconds. The same system serves all of them.
• Survive broadcaster hiccups. A dropped connection that comes back within 30 seconds continues the same broadcast: viewers see a short “reconnecting” slate rather than an offline page, and the broadcast stays one recording.
• Record every broadcast. Each broadcast becomes a VOD the moment it ends, and is rewindable while live. It is kept for a retention period set by the account’s tier.
• Live directory and viewer counts. Which channels are live, at which qualities, and roughly how many people are watching, refreshed every 10 to 30 seconds.
• Out of scope. Chat (the separate Live Comments question), monetisation and ad insertion, recommendations and directory ranking, clip editing, moderation, and DRM.

Non-functional requirements

• Latency (~3 s glass to glass · p95 < 5 s). This is the requirement that drives the hardest trade-off. The player buffer is the largest share of the delay, so every second removed is paid for in rebuffering risk, request rate and what the CDN must do.
• Scale (100 k live channels · 5 M viewers). At peak, with one channel at 500 k. The audience is heavily skewed: the top 10 k channels hold about 90 % of viewers, and the median channel has two or three.
• Egress (~15 Tbit/s at peak). Every viewer pulls their own copy, 25 times the ingest volume. Nothing can be pre-positioned, because each byte exists only a second before it is watched.
• Smooth playback (rebuffer < 1 % · join < 2 s). With only about 1.5 s of buffer, the player must step down quality early; joining must not wait for a whole segment.
• Broadcast reliability (no broadcast ended by our failure). An ingest PoP or transcoder failure may cost viewers a few seconds but must never end the broadcast. Streamers lose part of their audience every time they go offline.
• Recording durability (no lost broadcast). Segments reach object storage within seconds of being produced, so a VOD survives the failure of any live component.
• Cost (egress + transcoding dominate). Egress scales with viewers, transcoding with channels. Transcoding every channel the same way spends most of the compute on channels almost nobody is watching.

Back-of-envelope estimates

• Ingest bandwidth: ~600 Gbit/s. 100 k live channels × ~6 Mbit/s source (1080p60 is the common upload) = 600 Gbit/s into the ingest PoPs. Large, but small next to what goes out.
• Peak egress: ~15 Tbit/s. 5 M concurrent viewers × ~3 Mbit/s average delivered bitrate (a mix of source, 720p60 and mobile renditions) = 15 Tbit/s, 25× the ingest. One viewer-hour is ~1.35 GB. Egress is the largest bill in the system.
• The biggest channel alone: ~1.5 Tbit/s. 500 k viewers × 3 Mbit/s = 1.5 Tbit/s, a tenth of the platform’s egress on one stream. Over ~100 edge PoPs that is ~5 k viewers and ~15 Gbit/s per PoP for a single set of objects, more than one cache server should carry.
• Edge requests per second: ~20 M/s. A low-latency viewer makes two requests per 0.5 s part, a blocking playlist reload and the part itself, so 4 req/s each. 5 M × 4 = 20 M req/s, half of them for playlists: the playlist is requested as often as the video and is the only object that changes.
• Origin fetches for the 500 k channel: ~120 part fetches/s. Without coalescing: 500 k viewers × 2 parts/s = 1 M req/s at the origin. With edges and ~10 regional shields each collapsing to one upstream fetch per object: 10 shields × 6 renditions × 2 parts/s = 120/s, and about the same again for playlists. Audience size no longer reaches the origin.
• Transcode fleet: ~70 k vCPU, not 600 k. A full ladder (720p60, 480p, 360p, 160p, audio) from a 1080p60 source costs ~6 vCPU in software. For every channel: 100 k × 6 = 600 k vCPU. Ladder only for the top 10 k channels and repackage the source for the rest at ~0.1 vCPU: 60 k + 9 k ≈ 70 k vCPU, about 9× less, while still covering ~90 % of viewers.
• Recording volume: ~4 PB/day. About 60 k channels live on average across the day × 24 h = 1.44 M broadcast-hours. Source at 6 Mbit/s is 2.7 GB an hour: 1.44 M × 2.7 GB ≈ 3.9 PB/day, plus ~10 % for the ladders of transcoded channels. A 7-day default retention holds ~30 PB, which is why retention is a product decision.

Components

• Broadcaster (OBS · RTMP / SRT / WHIP): The streamer’s encoder (OBS, a console, a phone app) sends one stream, typically 1080p60 H.264 at ~6 Mbit/s with a keyframe every 2 seconds, authenticated by the channel’s stream key. It reconnects by itself after a drop, which is what makes a reconnect grace period worth having.
• Ingest PoP (nearest of many · thin): Terminates RTMP, SRT or WebRTC close to the broadcaster, so the lossy, uncontrolled part of the path is short, then forwards the stream over our backbone to the origin site chosen for it. Deliberately thin: check the key, buffer a few seconds, forward. Broadcasters reach the nearest healthy PoP through DNS steering.
• Stream control (keys · sessions · epochs): Owns each broadcast session: validates the stream key, creates the broadcast id, tracks live, reconnecting and ended, and runs the reconnect grace timer. Every accepted connection raises the session’s epoch, so a stale connection or transcoder can be fenced out.
• Transcode scheduler (ladder per channel): Places each broadcast on a transcoder slot in an origin site with spare capacity and decides its ladder: full for partners and channels with an audience, passthrough for the rest. Upgrades a channel mid-stream when its viewer count crosses a threshold, and gives the biggest channels a second, aligned transcoder.
• Live transcoder (real-time ABR ladder): Decodes the source once and encodes every rendition in one process, so keyframes and timestamps line up across the ladder and a player can switch at any segment boundary. Emits CMAF chunks as frames are encoded. When its input stops, it encodes a slate for the grace period so playlists keep advancing.
• Packager / origin (CMAF parts · LL-HLS): Groups chunks into 0.5 s parts and 2 s segments, keeps each rendition’s playlist current, and answers blocking playlist requests the instant the requested part exists. Holds the last few minutes in memory, writes each finished segment to the recording store, and never serves a viewer directly.
• Origin shield (regional mid-tier): About ten regional caches between the edges and the origin. Collapses all edge misses for an object into one upstream fetch, holds blocking playlist requests, and for hot channels receives each new part by push. It is why origin load depends on renditions, not viewers.
• CDN edge (coalescing · streams in-flight): PoPs near viewers. The first request for a part goes upstream and every later one waits on it, receiving bytes as they arrive rather than after the object completes. Blocking playlist URLs are cache keys that become valid exactly once, so even the playlist caches.
• Viewer player (LL-HLS · ABR · catch-up): Web, mobile and TV players. Keeps about 1.5 s of buffer, chooses a rendition per part from buffer level and measured throughput, and plays slightly fast to win back latency after a stall. Sends a heartbeat every 30 seconds.
• Playback API (join · token · CDN choice): Answers “play this channel”: is it live, which ladder, which CDN hostname, plus a short-lived signed token. The answer is per channel, not per viewer, so it is cached for a second or two and a 100 k-viewer join storm costs a handful of backend reads.
• Live state (Redis · per channel): The hot, rebuildable view of what is live right now: broadcast id, origin site, ladder, latency mode, hot flag and viewer count per channel. Written by stream control, the scheduler and the viewer counter; read on every join.
• Recording store (S3 · written as produced): Object storage holding every finished segment as {broadcast_id}/{rendition}/{seq}.m4s plus a VOD playlist that grows with them. Because it fills while the stream runs, the VOD exists the moment the broadcast ends. Lifecycle rules apply retention.
• Channel DB (channels · broadcasts · VODs): Durable records: channels and hashed stream keys, broadcast sessions with their epochs, and VOD entries pointing at recordings. A row per go-live is a tiny write rate, so a relational store sharded by channel id is ample; the epoch update is a conditional write.
• Stream events (Kafka · key = channel_id): Player heartbeats (channel, rendition, latency, rebuffering) and broadcast lifecycle events. Feeds viewer counts and quality monitoring, and the out-of-scope consumers such as go-live notifications and the directory.
• Viewer counter (per channel · every 10 s): Counts distinct viewers per channel from heartbeats over a sliding window and writes the number to live state every 10 seconds. That one number drives the directory, ladder upgrades and the hot-channel flag, so it is an input to cost as well as to the UI.

User flows

1. A broadcaster goes live. From the first RTMP packet to a playable low-latency stream in a few seconds. Authentication, placement and the ladder decision happen once, at connect time; after that the media path never waits on a database.
   1. The broadcaster’s encoder connects to the nearest ingest PoP and publishes with its stream key. DNS steering returns the closest healthy PoP, typically 10 to 40 ms away, so the lossy and uncontrolled part of the path is as short as possible. RTMP runs over TCP, so one lost packet stalls everything behind it; SRT retransmits over UDP within a fixed latency window and copes far better with a bad uplink.
   2. The ingest PoP asks stream control to validate the key and open a broadcast session. Control looks up the key by its hash and refuses a wrong or banned key before any media leaves the PoP. It either creates a new broadcast or, if the channel has one in its reconnect grace period, resumes it. Each accepted connection gets a higher epoch than the last, which later lets the transcoder ignore a stale connection that is still sending.
   3. Control asks the transcode scheduler for a slot and a ladder. The scheduler picks an origin site near the ingest PoP with spare transcode capacity and decides the ladder from what is known before anyone watches: partner status and the channel’s recent average audience, kept on the channel row and refreshed from analytics. A channel that averaged 4 viewers last week starts as passthrough; one that averaged 4 000 gets the full ladder from the first frame.
   4. The scheduler reserves the slot and the ingest PoP forwards the stream to it over the backbone. The forwarding hop runs on our own network, so it is low-loss and adds tens of milliseconds. The PoP keeps a few seconds of the stream in memory, so if the broadcast has to move to another transcoder it can replay from the last keyframe instead of losing frames.
   5. The transcoder encodes the ladder, or only repackages the source, and emits CMAF chunks. One process decodes once and encodes every rendition, forcing a keyframe at the same timestamp in all of them every 2 seconds, so segment N of 480p and segment N of 720p cover the same instant. Passthrough encodes nothing: it cuts the source at the broadcaster’s own keyframes, which is why platforms ask broadcasters for a 2-second keyframe interval.
   6. The origin assembles parts and segments and updates each rendition’s playlist. A part is announced as soon as it is complete, every 0.5 s, and a segment every 2 s. The playlist also carries a preload hint naming the next part, so players can request it before it exists. The origin keeps the last few minutes in memory and serves only shields, never viewers.
   7. Control marks the channel live and publishes a stream-started event. The channel shows as live only once its first segment is on the origin, so a viewer who clicks at once gets a playable stream rather than an empty playlist. Go-live notifications, the directory and search consume the event; none of them is on the media path.
2. A viewer joins and watches. The read path. One cached API call, then plain HTTP to the nearest edge, where blocking playlist reloads and coalesced part requests keep each viewer about three seconds behind live without the origin ever seeing them.
   1. The player asks the playback API for the channel. One read of live state answers everything: live or not, broadcast id, ladder, latency mode and which CDN hostname to use. The answer is the same for every viewer of the channel, so it is cached for a second or two; the only per-viewer part, the token, is signed rather than looked up.
   2. The player loads the master playlist and picks a starting rendition. The master playlist lists the ladder with bandwidth, resolution and frame rate. The player starts a rung or two below what it guesses the connection can sustain, because with 1.5 s of buffer a wrong first guess stalls almost immediately.
   3. The player asks for the playlist that will contain the next part, and the request waits until that part exists. This is LL-HLS blocking playlist reload: instead of polling, the player names the part it wants and the request is held. Every viewer of the rendition asks with the same URL, so each edge sends one request upstream and each shield one to the origin, and all of them are answered together the instant the part is published.
   4. The player fetches each part from the edge, which fills it once from the shield. With a preload hint the player requests the part before it is finished, and the edge and shield pass bytes on as the origin produces them, so the part arrives as it is encoded rather than half a second later. Parts and segments are immutable, so they are cached for a few minutes for late joiners and rewind.
   5. The player keeps about 1.5 s of buffer, adapts the rendition part by part, and catches up after a stall. With so little buffer, it steps down as soon as the buffer falls below about 1 s and steps up only after several parts arrive well ahead of need. Because parts stream at the speed they are produced, download time no longer measures bandwidth, so throughput is measured only over bursts where bytes were already waiting. After a stall it plays at 1.05 to 1.1× until it is back at its latency target.
   6. Every 30 seconds the player sends a heartbeat, which the viewer counter aggregates per channel. The heartbeat carries rendition, measured latency and rebuffering, so one stream feeds both viewer counts and quality dashboards. The counter writes a count per channel every 10 seconds; the directory, the ladder decisions and the hot-channel flag all read that one number.
3. One channel climbs to 500 k viewers. The scale-breaking case. Live content cannot be pre-positioned, so the defence is to stop audience size from reaching anything that does not scale with it: coalescing in two cache tiers, push to the shields, redundant transcoders and a cached join path.
   1. The viewer counter sees the channel pass 20 k viewers and the scheduler marks it hot. Scheduled events such as an esports final are marked hot hours ahead by an operator; organic growth is caught by the count. Hot is sticky for the rest of the broadcast, so a dip in viewers does not undo the preparation.
   2. The scheduler starts a second, aligned transcoder for the channel in another failure domain. Both encoders cut segments at the same source timestamps and number them the same way, so the origin can take either one’s output for any segment and losing a transcoder costs nothing visible. This 1+1 redundancy is too expensive for 100 k channels and cheap for the few hundred that carry a large share of the audience.
   3. The origin pushes each new part to every regional shield as it is produced. For a channel watched in every region, waiting for each shield to ask adds a round trip to every part, and push removes it. For small channels pull is cheaper, because most of them are watched from one or two regions. Either way the origin sees about 120 part requests a second for this channel, the same as at 5 k viewers.
   4. Edge servers collapse their viewers’ requests into one upstream fetch per part per rendition. At ~5 k viewers per PoP, each part is wanted by thousands of clients within about 100 ms; the edge answers them all from one fetch, streaming bytes as they arrive. At ~15 Gbit/s per PoP for one channel, the PoP replicates the channel’s objects across several cache servers instead of hashing each URL to just one.
   5. The playback API absorbs the join storm and moves a share of viewers to a second CDN if a region runs hot. A hundred thousand viewers joining in a minute are ~1 700 requests a second for one identical answer, served from the API’s one-second cache. For the largest events the playback API hands another CDN’s hostname to a percentage of viewers in regions near capacity; the objects are identical, only the hostname differs.
4. The broadcaster drops and reconnects. The failure viewers notice most. A broken uplink must look like a short pause, not an ended stream, and a connection that comes back, even through another PoP, must continue the same broadcast without two inputs fighting over it.
   1. The broadcaster’s connection drops and the ingest PoP reports it to stream control. Control moves the session to “reconnecting” and starts a 30-second grace timer instead of ending the broadcast. A TCP connection can die silently, so the PoP treats 5 seconds without media as a drop rather than waiting for the socket to close.
   2. The transcoder fills the gap with a slate so the playlists keep advancing. After a second without input, the transcoder encodes a “reconnecting” image and silence into every rendition at continuous timestamps, so segment numbers carry on. Players keep playing the slate instead of stalling, retrying and showing “offline”, and their buffers and positions survive the gap.
   3. The encoder reconnects, possibly through a different PoP, and control resumes the same broadcast with a new epoch. Control finds an open session for the channel inside its grace period and resumes it rather than creating a new broadcast, so viewers, the VOD and the viewer count all continue. The epoch moves from 1 to 2 with a conditional update, so two racing connections cannot both win.
   4. The new PoP forwards to the same transcoder, which accepts only the new epoch and switches back from the slate. If the old connection is half-open and still trickling packets, they carry epoch 1 and are dropped, so two inputs can never interleave. The switch happens at the new input’s first keyframe and is marked as a discontinuity, because timestamps, and even resolution, may have changed.
   5. Players cross the discontinuity without reloading. A player reaching the discontinuity tag resets its decoder timeline and carries on; the viewer sees the slate end and the stream resume. Without the slate, many players would have given up after a few empty playlist reloads and shown an error.
   6. If the transcoder itself dies, the scheduler restarts the broadcast on a new slot that continues the segment numbers. The scheduler notices the missed heartbeat within 2 to 3 seconds, starts a new slot with the last sequence number the origin published, and the ingest PoP replays its buffered seconds from the last keyframe. Viewers of a single-transcoder channel see a short stall or a slate; hot channels already run a second transcoder and see nothing.
5. The stream ends and becomes a VOD. The background path. The recording is written segment by segment while the stream is live, so ending a broadcast means closing playlists, not processing hours of video.
   1. While live, the origin writes every finished segment of every rendition to the recording store. Each 2 s segment is uploaded as it completes, under a key built from the broadcast, rendition and sequence number, so a retried upload overwrites the same object. A VOD playlist listing them is rewritten every few segments, which also lets viewers rewind further than the live window.
   2. The broadcaster stops; once the grace period expires, control ends the session and the scheduler releases the slot. An explicit stop from the encoder ends the broadcast at once; a drop ends it after the 30-second grace period. The transcoder flushes its last partial segment before it exits, so the final seconds are not lost.
   3. The origin closes every live playlist and finalises the VOD playlist. Appending EXT-X-ENDLIST tells live players the stream is over, so they stop reloading. The VOD playlist is the same list of segments with an end tag and the slate gaps marked; nothing is re-encoded, so it is watchable straight away.
   4. Control records the VOD, clears live state and publishes a stream-ended event. The VOD row points at the recording prefix and carries the expiry date from the account’s retention tier. Clearing live state takes the channel out of the directory; the event lets the viewer counter close its window and analytics total up the broadcast.
   5. VOD viewers play through the same CDN, with shields filling from the recording store. A VOD is ordinary on-demand video: immutable segments, long cache lifetimes, and a 30-second player buffer instead of 1.5. Most VOD views come in the first day, from viewers who missed the stream, so recent recordings are hot and the rest is a long tail.
   6. Lifecycle rules expire recordings at the end of their retention period. At about 4 PB a day, retention is the storage bill: 7 days by default, longer for partners, and no expiry for highlights the streamer keeps. A highlight is a playlist over a subset of segments, copied to a prefix that does not expire when it is created.

Deep dives

The theory behind it

• Live video streaming: How live streams reach millions of viewers: ingest with RTMP, SRT or WebRTC, real-time transcoding, segmenting for HLS and DASH, low-latency HLS, CDN fan-out, WebRTC for interactive streams, latency trade-offs, live chat at scale, recording and reliability.
• CDNs and edge computing: How a content delivery network works, push vs pull, cache keys and TTLs, what to put at the edge, edge functions and KV, and the pitfalls of caching dynamic content.
• Video streaming: transcoding, HLS, DASH and adaptive bitrate: How online video works end to end: uploads, transcoding into a bitrate ladder, segmenting for HLS and DASH, adaptive bitrate in the player, CDN delivery, live versus on-demand, and the numbers behind it.

Related

• Design a URL Shortener
• Design YouTube
• Design Google Maps
• Design a Stock Exchange
• All 83 system design interview questions
• Run this question for a friend: the interviewer kit