Design Twitch

• Go live from broadcasting software: OBS, a console or a phone app pushes one high-quality stream authenticated by the channel’s secret stream key: RTMP for most encoders, SRT or WebRTC (WHIP) where supported. The channel is watchable within a few seconds of the first frame.
• Watch live, a few seconds behind: About 3 seconds glass to glass by default, so chat and the stream feel like one conversation. A normal-latency mode (around 10 s) for players that cannot do low latency and viewers on poor networks.
• Quality that suits the connection: A ladder from 160p up to the source, chosen automatically by the player part by part, plus a manual quality menu and audio-only.
• Any audience size: Most channels have a handful of viewers; a few have hundreds of thousands, and a raid can move 50 k viewers onto a small channel in seconds. The same system serves all of them.
• Survive broadcaster hiccups: A dropped connection that comes back within 30 seconds continues the same broadcast: viewers see a short “reconnecting” slate rather than an offline page, and the broadcast stays one recording.
• Record every broadcast: Each broadcast becomes a VOD the moment it ends, and is rewindable while live. It is kept for a retention period set by the account’s tier.
• Live directory and viewer counts: Which channels are live, at which qualities, and roughly how many people are watching, refreshed every 10 to 30 seconds.
• Out of scope: Chat (the separate Live Comments question), monetisation and ad insertion, recommendations and directory ranking, clip editing, moderation, and DRM.

• Latency (~3 s glass to glass · p95 < 5 s): This is the requirement that drives the hardest trade-off. The player buffer is the largest share of the delay, so every second removed is paid for in rebuffering risk, request rate and what the CDN must do.
• Scale (100 k live channels · 5 M viewers): At peak, with one channel at 500 k. The audience is heavily skewed: the top 10 k channels hold about 90 % of viewers, and the median channel has two or three.
• Egress (~15 Tbit/s at peak): Every viewer pulls their own copy, 25 times the ingest volume. Nothing can be pre-positioned, because each byte exists only a second before it is watched.
• Smooth playback (rebuffer < 1 % · join < 2 s): With only about 1.5 s of buffer, the player must step down quality early; joining must not wait for a whole segment.
• Broadcast reliability (no broadcast ended by our failure): An ingest PoP or transcoder failure may cost viewers a few seconds but must never end the broadcast. Streamers lose part of their audience every time they go offline.
• Recording durability (no lost broadcast): Segments reach object storage within seconds of being produced, so a VOD survives the failure of any live component.
• Cost (egress + transcoding dominate): Egress scales with viewers, transcoding with channels. Transcoding every channel the same way spends most of the compute on channels almost nobody is watching.

• Ingest bandwidth: ~600 Gbit/s. 100 k live channels × ~6 Mbit/s source (1080p60 is the common upload) = 600 Gbit/s into the ingest PoPs. Large, but small next to what goes out.
• Peak egress: ~15 Tbit/s. 5 M concurrent viewers × ~3 Mbit/s average delivered bitrate (a mix of source, 720p60 and mobile renditions) = 15 Tbit/s, 25× the ingest. One viewer-hour is ~1.35 GB. Egress is the largest bill in the system.
• The biggest channel alone: ~1.5 Tbit/s. 500 k viewers × 3 Mbit/s = 1.5 Tbit/s, a tenth of the platform’s egress on one stream. Over ~100 edge PoPs that is ~5 k viewers and ~15 Gbit/s per PoP for a single set of objects, more than one cache server should carry.
• Edge requests per second: ~20 M/s. A low-latency viewer makes two requests per 0.5 s part, a blocking playlist reload and the part itself, so 4 req/s each. 5 M × 4 = 20 M req/s, half of them for playlists: the playlist is requested as often as the video and is the only object that changes.
• Origin fetches for the 500 k channel: ~120 part fetches/s. Without coalescing: 500 k viewers × 2 parts/s = 1 M req/s at the origin. With edges and ~10 regional shields each collapsing to one upstream fetch per object: 10 shields × 6 renditions × 2 parts/s = 120/s, and about the same again for playlists. Audience size no longer reaches the origin.
• Transcode fleet: ~70 k vCPU, not 600 k. A full ladder (720p60, 480p, 360p, 160p, audio) from a 1080p60 source costs ~6 vCPU in software. For every channel: 100 k × 6 = 600 k vCPU. Ladder only for the top 10 k channels and repackage the source for the rest at ~0.1 vCPU: 60 k + 9 k ≈ 70 k vCPU, about 9× less, while still covering ~90 % of viewers.
• Recording volume: ~4 PB/day. About 60 k channels live on average across the day × 24 h = 1.44 M broadcast-hours. Source at 6 Mbit/s is 2.7 GB an hour: 1.44 M × 2.7 GB ≈ 3.9 PB/day, plus ~10 % for the ladders of transcoded channels. A 7-day default retention holds ~30 PB, which is why retention is a product decision.

• Broadcaster (OBS · RTMP / SRT / WHIP): The streamer’s encoder (OBS, a console, a phone app) sends one stream, typically 1080p60 H.264 at ~6 Mbit/s with a keyframe every 2 seconds, authenticated by the channel’s stream key. It reconnects by itself after a drop, which is what makes a reconnect grace period worth having.
• Ingest PoP (nearest of many · thin): Terminates RTMP, SRT or WebRTC close to the broadcaster, so the lossy, uncontrolled part of the path is short, then forwards the stream over our backbone to the origin site chosen for it. Deliberately thin: check the key, buffer a few seconds, forward. Broadcasters reach the nearest healthy PoP through DNS steering.
• Stream control (keys · sessions · epochs): Owns each broadcast session: validates the stream key, creates the broadcast id, tracks live, reconnecting and ended, and runs the reconnect grace timer. Every accepted connection raises the session’s epoch, so a stale connection or transcoder can be fenced out.
• Transcode scheduler (ladder per channel): Places each broadcast on a transcoder slot in an origin site with spare capacity and decides its ladder: full for partners and channels with an audience, passthrough for the rest. Upgrades a channel mid-stream when its viewer count crosses a threshold, and gives the biggest channels a second, aligned transcoder.
• Live transcoder (real-time ABR ladder): Decodes the source once and encodes every rendition in one process, so keyframes and timestamps line up across the ladder and a player can switch at any segment boundary. Emits CMAF chunks as frames are encoded. When its input stops, it encodes a slate for the grace period so playlists keep advancing.
• Packager / origin (CMAF parts · LL-HLS): Groups chunks into 0.5 s parts and 2 s segments, keeps each rendition’s playlist current, and answers blocking playlist requests the instant the requested part exists. Holds the last few minutes in memory, writes each finished segment to the recording store, and never serves a viewer directly.
• Origin shield (regional mid-tier): About ten regional caches between the edges and the origin. Collapses all edge misses for an object into one upstream fetch, holds blocking playlist requests, and for hot channels receives each new part by push. It is why origin load depends on renditions, not viewers.
• CDN edge (coalescing · streams in-flight): PoPs near viewers. The first request for a part goes upstream and every later one waits on it, receiving bytes as they arrive rather than after the object completes. Blocking playlist URLs are cache keys that become valid exactly once, so even the playlist caches.
• Viewer player (LL-HLS · ABR · catch-up): Web, mobile and TV players. Keeps about 1.5 s of buffer, chooses a rendition per part from buffer level and measured throughput, and plays slightly fast to win back latency after a stall. Sends a heartbeat every 30 seconds.
• Playback API (join · token · CDN choice): Answers “play this channel”: is it live, which ladder, which CDN hostname, plus a short-lived signed token. The answer is per channel, not per viewer, so it is cached for a second or two and a 100 k-viewer join storm costs a handful of backend reads.
• Live state (Redis · per channel): The hot, rebuildable view of what is live right now: broadcast id, origin site, ladder, latency mode, hot flag and viewer count per channel. Written by stream control, the scheduler and the viewer counter; read on every join.
• Recording store (S3 · written as produced): Object storage holding every finished segment as {broadcast_id}/{rendition}/{seq}.m4s plus a VOD playlist that grows with them. Because it fills while the stream runs, the VOD exists the moment the broadcast ends. Lifecycle rules apply retention.
• Channel DB (channels · broadcasts · VODs): Durable records: channels and hashed stream keys, broadcast sessions with their epochs, and VOD entries pointing at recordings. A row per go-live is a tiny write rate, so a relational store sharded by channel id is ample; the epoch update is a conditional write.
• Stream events (Kafka · key = channel_id): Player heartbeats (channel, rendition, latency, rebuffering) and broadcast lifecycle events. Feeds viewer counts and quality monitoring, and the out-of-scope consumers such as go-live notifications and the directory.
• Viewer counter (per channel · every 10 s): Counts distinct viewers per channel from heartbeats over a sliding window and writes the number to live state every 10 seconds. That one number drives the directory, ladder upgrades and the hot-channel flag, so it is an input to cost as well as to the UI.

1. A broadcaster goes live: From the first RTMP packet to a playable low-latency stream in a few seconds. Authentication, placement and the ladder decision happen once, at connect time; after that the media path never waits on a database.
   1. The broadcaster’s encoder connects to the nearest ingest PoP and publishes with its stream key.
   2. The ingest PoP asks stream control to validate the key and open a broadcast session.
   3. Control asks the transcode scheduler for a slot and a ladder.
   4. The scheduler reserves the slot and the ingest PoP forwards the stream to it over the backbone.
   5. The transcoder encodes the ladder, or only repackages the source, and emits CMAF chunks.
   6. The origin assembles parts and segments and updates each rendition’s playlist.
   7. Control marks the channel live and publishes a stream-started event.
2. A viewer joins and watches: The read path. One cached API call, then plain HTTP to the nearest edge, where blocking playlist reloads and coalesced part requests keep each viewer about three seconds behind live without the origin ever seeing them.
   1. The player asks the playback API for the channel.
   2. The player loads the master playlist and picks a starting rendition.
   3. The player asks for the playlist that will contain the next part, and the request waits until that part exists.
   4. The player fetches each part from the edge, which fills it once from the shield.
   5. The player keeps about 1.5 s of buffer, adapts the rendition part by part, and catches up after a stall.
   6. Every 30 seconds the player sends a heartbeat, which the viewer counter aggregates per channel.
3. One channel climbs to 500 k viewers: The scale-breaking case. Live content cannot be pre-positioned, so the defence is to stop audience size from reaching anything that does not scale with it: coalescing in two cache tiers, push to the shields, redundant transcoders and a cached join path.
   1. The viewer counter sees the channel pass 20 k viewers and the scheduler marks it hot.
   2. The scheduler starts a second, aligned transcoder for the channel in another failure domain.
   3. The origin pushes each new part to every regional shield as it is produced.
   4. Edge servers collapse their viewers’ requests into one upstream fetch per part per rendition.
   5. The playback API absorbs the join storm and moves a share of viewers to a second CDN if a region runs hot.
4. The broadcaster drops and reconnects: The failure viewers notice most. A broken uplink must look like a short pause, not an ended stream, and a connection that comes back, even through another PoP, must continue the same broadcast without two inputs fighting over it.
   1. The broadcaster’s connection drops and the ingest PoP reports it to stream control.
   2. The transcoder fills the gap with a slate so the playlists keep advancing.
   3. The encoder reconnects, possibly through a different PoP, and control resumes the same broadcast with a new epoch.
   4. The new PoP forwards to the same transcoder, which accepts only the new epoch and switches back from the slate.
   5. Players cross the discontinuity without reloading.
   6. If the transcoder itself dies, the scheduler restarts the broadcast on a new slot that continues the segment numbers.
5. The stream ends and becomes a VOD: The background path. The recording is written segment by segment while the stream is live, so ending a broadcast means closing playlists, not processing hours of video.
   1. While live, the origin writes every finished segment of every rendition to the recording store.
   2. The broadcaster stops; once the grace period expires, control ends the session and the scheduler releases the slot.
   3. The origin closes every live playlist and finalises the VOD playlist.
   4. Control records the VOD, clears live state and publishes a stream-ended event.
   5. VOD viewers play through the same CDN, with shields filling from the recording store.
   6. Lifecycle rules expire recordings at the end of their retention period.

Ask: Standard HLS runs 20 seconds behind live. How do you get to 3 without viewers stalling?

Good answers name: Standard HLS or DASH with 6 s segments, Shorter plain segments (1 s), LL-HLS: 2 s segments, 0.5 s parts, blocking reload and preload hints, WebRTC to every viewer.

Our pick: LL-HLS over CMAF: a keyframe every 2 s, 2 s segments, each published as four 0.5 s parts. The playlist announces parts as they complete, plus a preload hint for the next one, and players use blocking playlist reload (_HLS_msn and _HLS_part) so they learn of a part the moment it exists instead of polling. The budget at the default target is about 0.3 s for the broadcaster’s encoder, 0.15 s of uplink, 0.05 s of backbone, 0.3 s of real-time transcoding, 0.5 s to complete a part, 0.1 s through shield and edge, and 1.5 s of player buffer (three parts): roughly 2.9 s. The buffer is the only large term and the only one worth trading, so it is the knob: a viewer on a jittery network gets a larger hold-back automatically rather than stalls. A normal-latency mode with plain 2 s segments and about 6 s of buffer remains as the fallback.

1. Why keep 2-second segments at all if parts do the delivery?
   Because the segment is where the keyframe is, and keyframes are expensive: halving the GOP to 1 s costs roughly 10 to 20 % more bits for the same picture. Segments are also what the recording, the VOD and normal-latency players use. Parts make delivery fine-grained while compression and storage stay coarse.
2. Your target is 3 s, but a viewer on hotel Wi-Fi keeps stalling. What do you do?
   Let that player grow its own hold-back: after a stall it aims for 4 or 5 seconds instead of 3, then creeps back while its network behaves. Latency is a per-viewer setting, not a global one. If it still stalls, ABR drops a rung, and the viewer can choose normal latency, which most never need.
3. How does a viewer joining mid-segment start quickly?
   Segments begin with keyframes and parts that start with one are flagged INDEPENDENT in the playlist, so the player starts from the latest independent part rather than waiting for the next segment. It downloads from there at full speed, then plays slightly fast until it reaches its target latency. Join time is the playback API call, the master playlist and the first media, comfortably under 2 s.
4. Why not chunked transfer of whole segments instead of parts?
   It works and needs fewer requests: one per 2 s segment, streamed as it is produced, which is the approach Periscope published as LHLS and low-latency DASH uses. The drawbacks are that native Apple players require LL-HLS, and an open-ended download makes bandwidth estimation harder still. Both use the same CMAF chunks, so one packager can serve LL-HLS to Apple devices and chunked DASH to players that prefer it.
5. How would you measure latency in production?
   Stamp wall-clock time into the stream at ingest (the program date-time tag in the playlist, or a timed metadata box per segment) and have the player report now minus that stamp in every heartbeat. That gives ingest-to-glass latency per viewer, region, CDN and rendition, and its p95 is the SLO. The broadcaster’s own encoding delay, about 0.3 s, is outside what we can measure and is estimated separately.

Ask: A new part appears every half second and every viewer wants it at once. What reaches the origin, and how do you cache a playlist that changes twice a second?

Good answers name: Pull-through CDN with short TTLs, Two coalescing cache tiers that stream in-flight objects and use blocking playlist URLs as cache keys, Origin pushes every part to every edge, Peer-to-peer delivery between viewers.

Our pick: Edges plus about ten regional shields, all coalescing. The first request for an object goes upstream; later ones attach to it and receive bytes as they arrive, so 5 000 viewers in a PoP cost one fetch and nobody waits for a part to finish before getting its first bytes. Parts and segments are immutable and cached for five minutes, which covers late joiners and rewind. Blocking playlist requests carry _HLS_msn and _HLS_part in the URL and the cache key includes them, so “the playlist that contains part 1043.1” is an object that becomes valid once and never changes: it is coalesced and cached like media, and the origin holds one waiting request per shield per rendition. The only mutable object left is the plain playlist for normal-latency players, cached for half a segment. For hot channels the origin pushes parts to the shields and edges spread the channel across several servers per PoP. Origin load for the 500 k channel is about 120 part fetches a second, the same as at 5 k viewers.

1. The first request for a part reaches the edge 50 ms before the part exists. What happens?
   With a preload hint that is the normal case: the edge forwards it, the shield forwards it, and the origin holds it open until the part starts, then streams it. Every request arriving meanwhile attaches to the same pending fetch. A request for something the playlist has not hinted, such as a part two ahead, gets a 404 that must not be cached for more than a moment, or it poisons the edge for everyone.
2. One edge server holds the hot part and 5 000 viewers in its PoP want it. Is that a problem?
   At 3 Mbit/s each that is 15 Gbit/s of one object from one server, near or beyond what one machine’s network card should carry. CDNs normally hash each URL to one server per PoP for cache efficiency, so a hot channel needs the opposite: copy its objects to several servers and spread viewers across them. The trigger is the same hot flag that turns on shield push.
3. Why not let the edge cache the plain playlist for one second and skip blocking reload?
   For normal-latency players we do exactly that. For low latency, a one-second-old playlist is two parts stale, so players would sit at least a second further from live and could not know about the part being produced. Blocking reload removes both the polling and the staleness: the response arrives the instant the part does, and it is the same response for everyone.
4. A shield fails during the 500 k-viewer final. What do viewers see?
   Its edges retry against the next-nearest shield, which is already fetching the same parts for its own region, so the origin sees nothing new and that shield gains one more set of edge fetches. Viewers in the region may get one part late; with 1.5 s of buffer that is usually absorbed, occasionally a short stall. Shield health is checked in seconds, because live has no room for long timeouts.
5. How do you know coalescing is working?
   Track origin requests per channel per rendition per second against the number of shields; the line should stay flat as the audience grows. At the edge, track the collapsed-request ratio and time to first byte for parts, which should be close to the part’s publish time plus one network hop. A rising origin request rate for one channel usually means a cache key is varying, often a per-viewer token that leaked into the URL.

Ask: Why not transcode every channel into every rendition?

Good answers name: Full ladder for every channel, Source passthrough for everyone, Ladder decided per channel from expected audience, upgraded live, The broadcaster encodes the ladder and uploads every rendition.

Our pick: Three tiers chosen by the scheduler at go-live from partner status and the channel’s recent average audience: full (six renditions including the source), lite (source plus 360p, about 1 vCPU) when spare capacity allows, and passthrough (the source repackaged, about 0.1 vCPU). A passthrough channel is upgraded mid-stream when its count stays above about 30 viewers for a minute, roughly where the ladder pays for itself in egress, or immediately when a raid toward it starts. Ladders are not removed mid-broadcast, because removing a rendition interrupts everyone watching it. Partners have reserved capacity, and everyone else gets ladders in order of audience until the pool is full, which matches how Twitch has long described its policy: quality options guaranteed for Partners and offered to other channels as capacity allows. Hardware encoders (GPUs or dedicated video chips) cut the cost of a ladder several times and lower the threshold, but do not remove it.

1. A streamer with 40 viewers is raided by a channel with 50 k. What happens in the next ten seconds?
   The raid starts in our own product, so the scheduler hears about it before the viewers move and starts a full ladder for the target at once. Until the new renditions have their first segment, the master playlist offers the source only, so the first arrivals start on source and switch down as soon as lower renditions appear. The CDN needs nothing special: coalescing absorbs 50 k viewers on this channel the same way it does on any other.
2. How does a player that is already watching discover renditions added mid-stream?
   The master playlist is normally read once, so the media playlist carries a small signal, a date-range tag, telling our player to reload the master. Third-party players that ignore it keep watching the source, which still works. New renditions start at a segment boundary with the same sequence numbers as the source, so moving to one is an ordinary rendition switch.
3. Egress is the bigger bill. What are its levers?
   Send less: cap the rendition at the player’s size, since a 400-pixel embed never needs 1080p, and drop background tabs and unwatched players to a low rendition or audio only. Pay less per byte: committed rates with several CDNs and our own caches in the largest regions and ISPs. Use fewer bits for the same picture: HEVC or AV1 for the biggest channels where viewers’ devices decode it, because the encode cost is per channel and the saving is per viewer.
4. What happens at peak when the transcode pool is full?
   Lite ladders are the first thing to go: running ones stay, but no new ones start. A partner or raid target that needs a slot is placed in another origin site with spare capacity, at a few tens of milliseconds more latency. Capacity is planned with about 20 % headroom over each site’s daily peak, so the last resort, removing a running ladder, should almost never happen.
5. Why decide at go-live rather than wait to see the audience?
   Because a big channel’s viewers arrive within seconds of the go-live notification, and starting them on a 6 Mbit/s source loses mobile viewers in the first minute. A channel’s audience is stable from one broadcast to the next, so history is a good predictor. Waiting for the count is only for channels with no history, and the upgrade costs one segment boundary.

Ask: Why forward the stream to a central transcode cluster instead of transcoding at the PoP that received it?

Good answers name: Thin ingest PoPs forwarding over the backbone to a few origin sites with pooled transcoders, Transcode at the ingest PoP, Broadcasters connect straight to the origin sites, A managed cloud live-encoding service per channel.

Our pick: Many thin ingest PoPs, reached through DNS steering to the nearest healthy one, that terminate RTMP, SRT and WHIP, check the key with stream control, keep a few seconds of buffer and forward over the backbone to an origin site chosen by the scheduler. Sites are picked by proximity first and spare transcode capacity second, and a full site sheds new broadcasts to the next, so the fleet behaves like one pool. The PoP’s buffer lets a broadcast move to a new transcoder by replaying from the last keyframe. Twitch’s engineering blog has described this shape: ingest at edge PoPs, then each stream routed over its backbone to an origin data centre chosen by a service that tracks transcode capacity and network health.

1. An ingest PoP loses power. What happens to the broadcasters on it?
   Their connections drop, their encoders reconnect after a few seconds, and DNS, which has already pulled the failed PoP, sends them to the next nearest one. Stream control resumes each broadcast within its grace period, so viewers see a few seconds of slate. Nothing about a broadcast lived only in the PoP, which is the point of keeping it thin.
2. Why offer SRT or WebRTC if RTMP works?
   RTMP runs over TCP, so on a lossy uplink one lost packet stalls everything behind it and the encoder starts dropping frames. SRT retransmits over UDP within a fixed latency window, which survives far worse networks, and WHIP lets a browser go live with no software installed. All three become the same internal stream at the PoP, so nothing behind it cares.
3. How do you stop someone streaming with a stolen key?
   Keys are stored hashed, can be revoked instantly, and resetting one ends the current broadcast. Only one connection per channel is accepted and the newest wins through the epoch, so the owner reconnecting cuts off a thief, and repeated takeovers from different networks are an alert. Limits on bitrate and resolution per channel stop a key from being used as free video hosting.
4. The backbone link from Europe to an origin site degrades. What do you do?
   Stop placing new broadcasts from the affected PoPs on that site first. For existing ones, prefer waiting, because moving a live broadcast costs a discontinuity. If loss starts hurting viewers, move broadcasts one at a time to another site at segment boundaries, with the PoP’s buffer covering the switch.

Ask: The streamer’s Wi-Fi drops for eight seconds. How does that not end the broadcast, and how do you stop two connections fighting over it?

Good answers name: A session with a grace period, epoch fencing and a slate from the transcoder, End the broadcast on disconnect and start a new one on reconnect, Keep the session but let the playlist stall, Two simultaneous uplinks from the broadcaster.

Our pick: Stream control holds each broadcast as a session in live, reconnecting or ended. A drop (socket closed, or 5 seconds without media) moves it to reconnecting for 30 seconds, and the transcoder, after a second without input, encodes a slate into every rendition at continuous timestamps, so playlists keep advancing and players keep playing. Any new connection with the channel’s key resumes the session and raises its epoch with a conditional write in the channel DB; the transcoder accepts media only from the current epoch, so a half-open old connection is ignored. The switch from slate back to live is a discontinuity at the new input’s first keyframe. If the grace period expires, the broadcast ends normally and the VOD closes with the slate trimmed. Transcoder failure is handled from the other side: the scheduler restarts the broadcast on a new slot that continues the sequence numbers, and hot channels already run a second transcoder.

1. Why 30 seconds of grace? Why not five minutes?
   Encoders reconnect within a few seconds and most real drops are short, so 30 seconds covers them. Longer means viewers stare at a slate while the streamer may have gone for good, and the transcode slot sits idle. It could be a channel setting, longer for streamers broadcasting from a phone on mobile data, whose connections drop more often.
2. Two encoders with the same key connect at the same moment. Who wins?
   Both reach stream control, both attempt the conditional update from epoch 1 to 2, and exactly one succeeds; the other is refused at its PoP. If they connect one after the other, the later one wins with epoch 3 and the earlier is cut off. That is right when the owner reconnects after a half-open connection, and a visible alert when a key is being shared.
3. The stream comes back at 720p instead of 1080p. Does anything break?
   The transcoder rebuilds its decoder at the discontinuity and stops producing renditions above the new source, removing them at the next master playlist reload rather than upscaling. Players treat a discontinuity as a reset point, so a change of resolution or even codec is fine. The VOD playlist keeps the discontinuity tag and plays through the change too.
4. How would you know reconnects are hurting viewers?
   Count reconnects per broadcast-hour by ingest PoP and broadcaster ISP: a PoP whose rate rises is a network problem before streamers report it. On the viewer side, measure audience lost across a reconnect, the viewers watching just before the drop against those watching a minute after it resumes. That number is why the grace period and the slate exist.

Ask: Where does the recording come from, and why is the VOD available the moment the stream ends?

Good answers name: Write segments to object storage as they are produced, with a growing VOD playlist, Record the ingest stream to one file and transcode after the broadcast, Keep the source only and transcode VOD renditions on demand, Re-encode popular VODs offline at higher efficiency.

Our pick: The origin uploads each finished segment to object storage as {broadcast_id}/{rendition}/{seq}.m4s, keyed so a retry overwrites rather than duplicates, and rewrites a VOD playlist every few segments. Slate segments are recorded but marked, so the finished VOD can skip them. At the end the origin appends an end tag and stream control writes the VOD row, so viewers can start watching seconds after the stream ends, through the same CDN with long cache lifetimes. Objects are tagged at upload with the account’s retention class and lifecycle rules expire them: at ~4 PB a day, a 7-day default holds about 30 PB, so retention is the storage lever. Highlights are new playlists over a subset of segments, copied to a prefix that does not expire.

1. Uploads to object storage fail for two minutes. Is the recording lost?
   No. The origin already keeps several minutes of segments in memory for live; it queues uploads and retries in order, and because keys are deterministic a retry that lands twice is harmless. Live playback is unaffected because it never reads from the recording store. If the outage outlasts the memory window, the origin spills to local disk, and the VOD playlist only ever lists segments known to be stored.
2. Millions of 2-second objects a day: is object count a problem?
   It is billions of objects a year, which object stores handle, but per-request costs and listings become noticeable. Two remedies: after the first few days, when VOD views have tailed off, concatenate each rendition into one file and serve segments as byte ranges; and never list prefixes on the read path, since the playlist already names every object.
3. A streamer played copyrighted music. How do you handle it in the VOD without re-encoding hours of video?
   Audio fingerprinting runs on the recorded audio after the stream, and matched spans are muted by rewriting only the affected segments with a silent audio track. Because segments are independent files, that is a few dozen new objects and a new playlist, not a new encode of the broadcast. The live stream is untouched.
4. How does rewinding ten minutes during a live stream work?
   The live playlist covers only the last minute or two, so the player switches to the VOD playlist, which already lists everything up to a few segments ago. Those segments come through the CDN, filled from the recording store, with a normal on-demand buffer. Returning to live switches back to the low-latency playlist.