Design Live Comments

• Post a comment on a live video: Short text and emoji, sent as a plain POST. The author sees their own comment instantly, before the server confirms it.
• See other viewers’ comments as they are posted: Within about 2 seconds of posting, on every stream from 3 viewers to 5 million.
• Late joiners see recent comments: Opening a stream shows the last 50 comments at once; a dropped connection resumes without gaps or duplicates.
• Readable comments on huge streams: Above about 50 comments a second, each viewer sees around 10 a second: a shared, quality-ranked sample plus the creator, moderators, their own friends and themselves.
• Reactions shown as bursts: Hearts and other reactions are counted per stream per second and drawn as floating bursts. Individual taps are never delivered to other viewers.
• Moderation: Obvious abuse is rejected before it is shown; comments flagged later by ML, reports or the creator vanish from every screen within seconds. Creators can ban users, block words and turn on slow mode.
• Live viewer count: Approximate, refreshed every few seconds.
• Scroll back and replay: Older comments page in from storage. After the stream ends, comments replay in step with the video, using the video timestamp stored with each one.
• Out of scope: The video pipeline itself (ingest, transcoding, HLS and the CDN), "went live" notifications, threaded replies, comment search and ads.

• Scale (50 M concurrent · 5 M on one stream): Heavy tailed: most of the ~500 k concurrent streams have fewer than 10 viewers and a few have millions. Concurrency sizes the gateway fleet; the biggest stream sizes the fan-out tree.
• Read/write asymmetry (1 write → up to 5 M reads): The requirement that drives the hardest tradeoff. Every choice keeps the cost of one comment proportional to the number of gateway servers its viewers sit on, never to the number of viewers.
• Delivery latency (p99 < 2 s post to screen): Budget: API and filter ~50 ms, log ~20 ms, router batch ≤ 100 ms, relay tree ~50 ms, gateway flush ≤ 250 ms, last mile 100–300 ms. On sampled streams the ML hold (up to 1 s) spends most of the slack.
• Ordering (per stream, no global order): Any two comments that two viewers both see appear in the same order for both. Nothing is ordered across streams, which is what lets every stream be handled independently.
• Delivery guarantee (durable once acked · live best effort): A 201 means the comment is in the log and will be stored. The live push may legitimately skip it (sampling, overload); no viewer is promised every comment.
• Availability (99.95 % for viewing): Comments must never affect video playback. Under overload, shed in this order: reaction animations, the sample size, then comments pause while the video keeps playing.
• Moderation latency (takedown p99 < 5 s): What matters is exposure: impressions of a bad comment before it is removed. One second on a 5 M stream is millions of impressions; on a 20-viewer stream it is 20.

• Gateway servers: ~1,000. 50 M concurrent viewers ÷ 50 k SSE connections per box = 1,000 gateways. A tuned box holds more idle connections than that; 50 k leaves CPU for up to 4 flushes a second per viewer and for absorbing a failed neighbour’s viewers.
• Comments posted per second: ~42 k/s. Assume each viewer comments about once every 20 minutes: 50 M ÷ 1,200 s ≈ 42 k/s at peak. A Kafka cluster of a dozen brokers absorbs that. Writes are not the hard part.
• Naive deliveries on the hottest stream: 25 billion/s. 5 k comments/s × 5 M viewers = 25 billion deliveries a second if every comment goes to every viewer. Impossible, and pointless: nobody can read 5,000 comments a second. This is the number that forces sampling.
• Hottest stream after sampling: ~50 M deliveries/s in ~20 M frames/s. Each viewer is shown about 10 comments/s: 5 M × 10 = 50 M comment deliveries a second. Gateways flush each viewer at most 4 times a second, so 5 M × 4 = 20 M SSE writes/s, about 100 k per gateway across the stream’s 200 gateways.
• Backbone copies per comment, random placement: ~630 for a 1,000-viewer stream. With viewers spread at random over 1,000 gateways, a 1,000-viewer stream touches 1,000 × (1 − (1 − 1/1,000)^1,000) ≈ 1,000 × (1 − 1/e) ≈ 632 gateways, so each comment is copied ~632 times. With stream affinity it touches 1. Random placement makes medium streams cost almost one copy per viewer.
• Hottest stream on the backbone: 200 gateways · 2 k msgs/s. Affinity puts at most 25 k of one stream’s viewers on a gateway: 5 M ÷ 25 k = 200 gateways. The router publishes 10 batches/s, so 200 × 10 = 2,000 deliveries a second. Every comment rides along for friend matching, ~300 B each: 1.5 MB/s per gateway, ~300 MB/s across the tree.
• Reaction ingest: ~3 M taps/s → ~500 k req/s. Assume 2 % of 50 M viewers are tapping at any moment, at 3 taps/s: 3 M taps/s. Clients batch and send once every 2 s while tapping: 1 M ÷ 2 = 500 k requests/s, more than 10× the comment rate. Reactions, not comments, are the biggest write stream.
• Comment storage per year: ~160 TB. Average concurrency over a day is ~40 % of peak, so ~17 k/s ≈ 1.5 B comments a day. At ~300 B each with metadata that is 450 GB/day ≈ 160 TB/yr before replication. Big, but ordinary for a wide-column store.

• Viewer app (web · iOS · Android): Plays the video and holds one SSE stream for the comments of the stream in focus, over the same HTTP/2 connection it uses for every other API call. Posts comments and batched reaction taps as ordinary POSTs and renders its own comment immediately. Remembers the last sequence number it rendered, which the browser sends back as Last-Event-ID on reconnect.
• Edge LB (L7 · HTTP/2 · stream affinity): Terminates TLS and HTTP/2 at the edge and routes each request on its own: POSTs to the Comment API, live streams to a gateway chosen by stream id. Placement is consistent hashing of stream_id with bounded load: a stream gets the next k gateways on the ring, with k sized from its viewer count in the Stream directory, so viewers of one stream share gateways instead of scattering across the fleet.
• Live gateways (SSE · 50 k conns/box): About 50 k SSE connections per box, 1,000 boxes at peak. A gateway subscribes to a stream’s topic when its first local viewer arrives and unsubscribes 30 s after the last leaves, keeps the last 500 broadcast comments per stream in memory for joins and reconnects, and flushes each viewer at most every 250 ms. On sampled streams it builds each viewer’s frame: the shared sample, pinned comments, friends’ comments from a local friend index, and the viewer’s own.
• Comment API (POST · history · retract): Stateless write path. Authenticates, applies the sync filter, and appends comments, reaction batches and retractions to the event log; the produce ack returns the partition offset, which becomes the comment’s sequence number. Also serves older comments from the store for scroll-back and replay, and is the single entry point for takedowns from ML, reports and creators.
• Sync filter (rules · bans · < 5 ms): The cheap moderation check every comment passes before it is accepted: global and per-creator blocked terms and links, bans at user, creator and stream level, per-user rate limits (one comment every 2 s per stream by default, tighter in slow mode) and a duplicate-text check. Runs as a sidecar next to each API instance with rules and bans pushed to it, so the check is a local call. It can reject, or shadow-accept spam so only its author sees it.
• Event log (Kafka · keyed by stream): Kafka with two topics. Each live stream is pinned to one comments partition for its whole life, so the partition offset is a per-stream sequence that only grows and replays to the same values. Reactions are keyed by (stream_id, viewer_id mod 16) so a burst of taps on one stream spreads over 16 partitions. Consumers: stream routers, the archiver, ML moderation and the reaction aggregator.
• Stream router (partition owner · sampler): Exactly one router owns each partition (a consumer group), so it sees every stream’s comments in order and can keep per-stream state: the comment rate, the sampled-mode flag and the scores. It appends broadcast comments to the recent cache, batches each stream for up to 100 ms and publishes one message per batch. Above 50 comments/s it switches a stream to sampled mode and tags each comment pinned, sample or rest.
• Relay tree (per-stream topics · 2 tiers): Per-stream topics sharded by stream_id across relay nodes. Gateways subscribe to the streams their viewers watch, so each batch crosses the backbone once per subscribed gateway, never once per viewer. A topic with more than ~50 subscribers gets a second tier of relays, each feeding ~15 gateways, so no one node pushes the hottest stream’s 300 MB/s alone. Fire and forget: a gateway that misses a batch sees the sequence gap and refills from the recent cache.
• Recent comments (Redis · last 500 per stream): A Redis sorted set per stream, scored by sequence number, holding the last 500 broadcast comments: everything on a normal stream, only the pinned and sample tiers on a sampled one. Read only when a gateway first subscribes to a stream or a reconnect needs more than the gateway’s own buffer holds, so a million joins cost a few hundred reads, not a million. Expires a day after the stream ends.
• Comments store (Cassandra · (stream, 10 min)): Every comment, for scroll-back, replay, appeals and audit. Partition key (stream_id, 10-minute bucket), clustered by sequence; streams in sampled mode add a 16-way shard to the key, because 5 k comments/s for 10 minutes is ~900 MB, far past a healthy partition. Each row keeps the author’s video timestamp so a replay shows comments in step with the video. Takedowns are soft deletes with a reason.
• Reaction aggregator (Flink · 1 s windows): A streaming job in two stages: the first sums taps per (stream, shard, second), the second merges the 16 shards and publishes one counts event per stream per second to the stream’s topic. Counts are approximate by design: a lost batch shifts a burst by a few hearts, which nobody can see. Batches carry a client batch id so a retried batch is not counted twice.
• ML moderation (GPU classifier · ~1 s): Consumes every comment from the log and scores it with a batched GPU classifier (hate, harassment, spam) within about 1 s at p99. Above the takedown threshold it asks the Comment API to retract the comment, the same path user reports and creator deletes take. On sampled streams the router holds sample candidates for up to 1 s, so a comment retracted in that window never reaches the shared sample.
• Social graph (top friends · cached): The existing social graph service. A gateway calls it once when a viewer joins a sampled stream, fetches that viewer’s top ~200 friends, and inverts the lists into a local map from author to the local viewers who should see that author’s comments. Results are cached; a failure only means no friend boost for that viewer.
• Stream directory (viewer counts · placement): Small replicated state per live stream: total viewers, summed from per-gateway counts reported every 5 s with a 15 s TTL so a dead gateway’s viewers drop out by themselves, and the placement width k = ceil(viewers ÷ 25 k). The load balancer keeps placements in memory with changes pushed within seconds; gateways get the viewer totals back in the response to their report.

1. Post a comment and see it everywhere: The core path: a comment is accepted, sequenced by the log, and copied once per gateway rather than once per viewer. Non-trivial because the same path has to work for a stream with 3 viewers and one with 5 million.
   1. Viewer posts a comment, and the client renders it at once.
   2. API runs the sync filter: rate limit, bans, blocked terms and duplicate text.
   3. API appends the comment to the stream’s partition of the event log; the offset becomes its sequence number.
   4. The stream router that owns the partition consumes it and appends the comment to the recent-comments cache.
   5. Router batches the stream’s comments for up to 100 ms and publishes one message per batch to the stream’s topic.
   6. The relay tier delivers the batch once to every gateway subscribed to the stream.
   7. Each gateway writes the batch into the SSE streams of its viewers of that stream.
   8. Separately, an archiver consumer writes every comment to the comments store.
2. Late joiner opens a live stream: A viewer arrives mid-stream and must see recent comments at once, then the live flow, without hammering a shared store when a million others arrive in the same minute.
   1. Viewer opens the live comments stream; the edge LB looks up the stream’s placement.
   2. LB hashes the stream id onto the gateway ring and picks the least loaded of its k gateways.
   3. If no local viewer was watching yet, the gateway subscribes to the stream’s topic and then loads the last 50 comments from the recent cache.
   4. If the gateway already holds the stream, it builds the snapshot from its own ring buffer.
   5. Gateway sends the snapshot, then streams live batches as they arrive.
   6. Gateway reports its per-stream viewer counts to the directory every 5 s.
   7. Viewer scrolls up for older comments; the API pages them from the store.
3. A stream hits 5 M viewers and 5 k comments/s: The scale-breaking case. Placement widens, the router switches to sampled mode, the relay tree grows a tier, and reactions become per-second counts, so each viewer gets about 10 comments a second instead of 5,000.
   1. Viewer counts climb, and the directory widens the stream’s gateway set as they grow.
   2. The router sees the comment rate pass 50 a second and switches the stream to sampled mode.
   3. Router scores each comment and tags it pinned, sample or rest.
   4. Router publishes 10 batches a second; the relay tree copies each one to the 200 gateways.
   5. Each gateway builds every viewer’s frame: pinned, sample, friends’ comments and their own.
   6. Viewers’ heart taps are batched on the client, posted every 2 s, and logged across 16 shards per stream.
   7. The aggregator sums taps per stream per second and publishes one counts event, which gateways merge into their next flush.
4. A gateway dies and its viewers reconnect: The failure path. Fifty thousand connections drop at once; they must land elsewhere, resume exactly where they were, and not stampede any shared store.
   1. A gateway crashes, dropping its 50 k SSE streams.
   2. Clients reconnect with jittered backoff, sending the last seq they rendered.
   3. The LB routes each viewer to the remaining gateways in its stream’s set.
   4. The new gateway subscribes if it needs to and fills the gap from its ring buffer or the recent cache.
   5. Client inserts by seq and drops anything it already rendered.
   6. If a relay dies instead, its gateways resubscribe through a sibling and fill any gap the same way.
   7. The directory drops the dead gateway’s counts when their 15 s TTL runs out.
5. ML takes down a comment already on screen: The asynchronous path. A comment passed the cheap filter and was shown; a second later the classifier flags it, and it must vanish from every screen, the cache and the replay.
   1. ML moderation consumes the comment from the log and scores it.
   2. Above the threshold, ML asks the Comment API to retract it.
   3. API appends a retraction event to the same stream partition.
   4. Router drops the comment from the recent cache and publishes the retraction.
   5. Gateways forward the retraction to viewers who were sent the comment, and clients remove it.
   6. The archiver soft-deletes the comment in the store, with its reason.

Ask: One comment has to reach 5 million screens within two seconds. How many copies get made, and where?

Good answers name: Per-stream topics, gateways subscribe, two-tier relay tree, Per-viewer push through a connection registry, Every gateway receives every comment, Clients poll a CDN-cached comments endpoint.

Our pick: Per-stream topics with gateways as the subscribers. The stream router batches each stream for up to 100 ms and publishes one message per batch to the stream’s topic on a relay tier sharded by stream_id. A gateway subscribes when its first viewer of a stream arrives and unsubscribes 30 s after the last one leaves. When a topic has more than about 50 subscribing gateways, its owner forwards to a second tier of relays that each feed about 15 gateways, so the hottest stream’s 2,000 deliveries and ~300 MB a second are spread over a dozen nodes. The gateway then writes to its own 25 k viewers from memory, which is where the factor of millions is paid. Delivery is fire and forget with per-stream sequence numbers: a gateway that misses a batch sees the gap and refills from the recent-comments cache. Facebook has described the same tree shape for live video itself, with edge caches coalescing requests so a surge of players becomes one fetch per edge.

1. A second-tier relay dies. What do the 15 gateways behind it see?
   Their subscriptions die with the connection, so within a second or two they resubscribe through the topic owner, which places them on a sibling relay. Batches published in the gap are lost on the live path; each gateway sees a jump in sequence numbers on its next batch and fetches the missing range from the recent-comments cache. Viewers see a short pause and then the missing comments, in order. Nothing durable ever lived on the relay, so there is nothing to recover.
2. Why batch for 100 ms instead of forwarding each comment as it arrives?
   Because pub/sub cost is per message, not per byte. On the hottest stream, per-comment forwarding is 5 k × 200 = 1 M messages a second through the tree; batching makes it 2,000 with the same bytes. On a quiet stream a batch usually holds one comment and the router publishes immediately after an idle period, so a lone comment is never held. 100 ms is 5 % of the 2 s budget.
3. How would this change at 10x, with 50 M viewers on one stream?
   The stream spans 2,000 gateways, so the tree gets a third tier; depth grows with the logarithm of the audience, which is why a tree scales and a star does not. The bigger change is the rest tier: 50 k comments a second to 2,000 gateways is 15 MB/s each, so gateways would register a Bloom filter of the authors their viewers care about, and relays would forward a rest-tier comment only when its author matches. The pinned and sample tiers still go everywhere.
4. How do you know end-to-end latency is under 2 seconds?
   Stamp each comment with the time the API accepted it and have gateways record now minus that stamp when they flush, bucketed by stream size so the hottest stream cannot hide inside a fleet average. Add canary viewers in every region that post a marked comment every few seconds and time how long it takes to appear on their own stream, which measures the whole path including the last mile. Alert per hop: router batching, relay delivery and gateway flush each get their own budget.
5. How does a gateway decide when to unsubscribe, and what happens in a reconnect storm?
   It keeps a count of local viewers per stream and unsubscribes 30 s after the count reaches zero, so a viewer flipping between apps does not churn subscriptions. In a reconnect storm, say a regional network blip, viewers come back to the same gateway sets because placement is by stream, and those gateways often still hold the subscription and the ring buffer. Subscriptions and cache reads scale with gateways per stream, so even a full storm costs a few hundred subscriptions for the hottest stream.

Ask: Slack holds WebSockets. Why are you using Server-Sent Events for live comments?

Good answers name: SSE over HTTP/2, posting with plain POST, WebSockets, Long polling, Comments as timed metadata inside the video segments.

Our pick: SSE over HTTP/2 for everything the server pushes, and ordinary POSTs for comments and reaction batches. The live endpoint is GET /v1/streams/{id}/live with Accept: text/event-stream; each event carries id: <seq> so a reconnect resumes with Last-Event-ID, and the gateway writes a comment line every 15 s so idle timeouts on NATs and load balancers never fire. Responses set Cache-Control: no-cache and X-Accel-Buffering: no so well-behaved proxies stream instead of buffering. A post is one request on a connection that is already open, so it costs one round trip, the same as a WebSocket frame and its ack, and it goes through the same auth, idempotency and rate limiting as the rest of the API. Memory per connection is similar to a WebSocket; the win is that every piece of HTTP infrastructure works unchanged. If the product grows two-way features, WebSockets are an upgrade to the last hop, not a different architecture: the gateways and the fan-out stay the same.

1. A corporate proxy buffers your SSE response. What does the viewer see, and what do you do?
   Nothing arrives until the proxy’s buffer fills or the response ends, so comments show up in big late lumps or not at all. The client detects it: if no event, not even the 15 s heartbeat, arrives within 20 s of opening the stream, it falls back to polling the history endpoint every 3 s. On our side we send the headers that tell well-behaved proxies not to buffer, but the client-side fallback is what actually handles hostile networks.
2. Doesn’t a POST per comment add latency compared with a frame on an open socket?
   Not on HTTP/2. The connection is already open and authenticated, so a POST is a new stream on it, with compressed headers, one round trip and no handshake. A WebSocket frame and its ack is also one round trip. And the author does not wait for it anyway, because the client renders their comment the moment they press send.
3. What happens on mobile when the app goes to the background?
   We close the stream. Live comments are only worth anything while the person is watching, and the OS would suspend the connection anyway. When the app comes back to the foreground it reopens with Last-Event-ID: a gap of a few seconds is filled exactly, and anything longer gets a fresh snapshot of the last 50, which is what the viewer actually wants.
4. When would you switch to WebSockets?
   When clients start sending often enough that per-request overhead matters, such as live polls with high participation, quiz shows or typing indicators, or when binary frames would help. Even then I would keep the gateways, the relay tree and the sequence numbers exactly as they are; only the last hop changes. Twitch is the example of an audience that sends a lot, and its chat speaks IRC, including over WebSockets.
5. How does SSE interact with stream-affinity routing?
   Well, because the stream id is in the URL of an ordinary HTTP request. The L7 load balancer reads /v1/streams/{id}/live, looks up that stream’s gateway set and proxies the HTTP/2 stream there, while POSTs on the same client connection go to the API pool. With WebSockets you can route on the upgrade URL too, but everything afterwards is stuck on that connection; with SSE every request is routed on its own merits.

Ask: The stream is getting 5,000 comments and over a million heart taps a second. What does each viewer actually receive?

Good answers name: Shared ranked sample, personal additions at the gateway, reactions as per-second counts, Deliver everything to everyone, Random per-viewer sampling at each gateway, Throttle the posters instead (slow mode, followers-only).

Our pick: The router measures each stream’s rate and switches it to sampled mode above 50 comments a second for 10 s, and back when it stays below 30 a second for 30 s, so it does not flap. In sampled mode every comment gets a score from cheap features: author tier (creator, moderator, verified guest), account age and history, prior removals, language match, length and a duplicate-text penalty. Comments are tagged pinned (creator and moderators, always shown, capped at 2 a second), sample (the best scored, about 10 a second, at most one per author per minute, and only after the ML hold of up to 1 s) or rest. Each gateway sends every viewer pinned plus sample, plus rest-tier comments from that viewer’s top friends via its local friend index, plus their own comment, which the client already rendered. Reactions never travel one by one: clients batch taps every 2 s, the aggregator sums them per stream per second in two stages (16 shards, then a merge), and one counts event a second goes to the stream’s topic; the client draws a burst on a log scale capped at about 20 hearts a second and shows its own taps immediately. YouTube’s live chat offers a filtered Top chat view next to the full Live chat, which is the same idea made visible to the viewer.

1. My comment was not picked for the sample. Do I know?
   No, and you do not need to: your client rendered it the moment you sent it, so from your seat it is in the chat. Friends watching see it through the friend index, and the creator’s moderation view shows every comment. What we must never do is tell someone their comment was seen by everyone when it was not, so any "seen by" or like counts are computed from real impressions.
2. How do you stop bots from flooding the sample?
   In layers. The sync filter rate-limits per user and rejects duplicate text, the scorer penalises new and low-reputation accounts and near-duplicates, the sample takes at most one comment per author per minute, and on sampled streams nothing enters the sample until the ML hold has passed. A coordinated farm of aged accounts posting distinct text can still slip through occasionally, which is why exposure of later-removed comments is a tracked metric.
3. Why select centrally at the router rather than at each gateway?
   Three reasons. Selecting once is 200 times less work than selecting at 200 gateways. A shared sample means viewers of a stream see the same conversation, so a reply appears alongside what it replies to. And the signals are central: ML retractions arrive on the stream’s partition, and author reputation is cheaper to look up once per comment than once per gateway. Gateways only add what is genuinely personal.
4. Reaction counts are approximate. Is that acceptable, and where do they go wrong?
   Yes: the display is a burst, and nobody can tell 41,200 hearts from 41,350. Counts go wrong when a batch is lost, say a client goes offline or a partition rebalances, which undercounts a little, and when a client retries a batch, which could double count, so batches carry a batch id and the first stage dedupes within the window. If an exact total is ever needed for a creator dashboard, it comes from a batch job over the reactions topic, not the live path.
5. At 10x, reaction ingest is 5 M requests a second. What changes?
   Sample the reporters. The stream’s config, sent down the SSE stream, gives clients a report probability p, say 0.1 on a 50 M-viewer stream; each client sends its batch with probability p and the aggregator multiplies by 1/p. The burst looks the same, counts in the tens of thousands are off by a few percent at most, and ingest drops tenfold. The viewer’s own taps still animate locally, so nobody notices that theirs were not sent.

Ask: Why not spread viewers across gateways at random, like any other stateless service?

Good answers name: Consistent hashing on stream_id with bounded load and a per-stream width, Random or least-connections placement, Plain consistent hashing on stream_id, Dedicated gateway pools for scheduled mega-events.

Our pick: The edge load balancer hashes stream_id onto a ring of gateways and gives the stream the next k gateways clockwise, with k = max(1, ceil(viewers ÷ 25 k)) from the Stream directory; within that set it picks the least loaded. Consistent hashing with bounded loads (Mirrokni, Thorup and Zadimoghaddam; HAProxy exposes it as hash-balance-factor) caps every gateway at 1.25× the fleet average, so when a stream’s set is full the next viewer spills to the next gateway on the ring instead of overloading one. k grows as soon as the directory sees the audience grow and shrinks only after 10 minutes, and changing k never moves existing connections, only where new ones go. Gateways report per-stream counts every 5 s with a 15 s TTL, so the directory heals itself when a gateway dies. If the directory is unreachable, the load balancer keeps its cached placements and treats unknown streams as k = 1, which bounded load turns into a sensible spread.

1. A celebrity goes live and gets a million viewers in two minutes. Walk me through placement.
   At first k is 1, so the first gateway fills to its bound and bounded load spills new viewers onto the next gateways on the ring, which is already the right behaviour. Within one 5 s report cycle the directory sees the count, k jumps, and the load balancer spreads new arrivals over the wider set. For creators with large followings we pre-set k from their follower count when the stream starts, so the burst never leans on spill-over at all.
2. A gateway holding 25 k viewers of the big stream dies. Where do they go?
   They reconnect with jittered backoff over a few seconds, and the load balancer hashes them to the same stream set minus the dead box, so they spread over the other 199 gateways, about 125 each, which already hold the subscription and the ring buffer. Each viewer resumes with Last-Event-ID and gets what it missed from memory. The directory drops the dead gateway’s counts after 15 s, so the viewer count does not double count.
3. A viewer’s feed autoplays three live videos at once. Three gateways?
   Only the video in focus opens a comments stream; the others show just their viewer counts, which the feed fetches with a cheap periodic call. When focus moves, the client closes one stream and opens the other, which with HTTP/2 is just a new request on the same connection to the edge. The edge routes that request to a different gateway, but the client still holds one TCP connection.
4. What does affinity cost you, and how do you know it is working?
   Less even load, a dependency on the directory, and correlated failures: losing a box hurts a few streams a lot rather than every stream a little. The metric that shows it is working is backbone copies per comment, weighted by comment volume, against the ideal of viewers ÷ 25 k + 1; random placement would put it in the hundreds. I would also watch the spread of connections per gateway, which bounded load is supposed to keep tight.
5. Why at most 25 k of one stream per gateway rather than filling a box with 50 k?
   To bound blast radius and hot spots. A box that is half one huge stream and half many small ones loses less of the big stream when it fails, keeps room for its small streams to grow, and spreads the hot stream’s per-frame work over twice as many CPUs. It doubles backbone copies for the biggest stream, from 100 to 200, which is a small price for that.

Ask: Do all viewers see the comments in the same order, and what does someone who joins late or drops for ten seconds see?

Good answers name: Per-stream sequence = log partition offset, Timestamps from the API servers, Per-stream counter (Redis INCR), One global sequencer.

Our pick: Pin each stream to a partition when it goes live, recorded with the stream so that adding partitions later never moves a live stream, and use the partition offset as its sequence number. The API returns it to the author in the 201, the router publishes batches with seq ranges, gateways deliver in seq order and drop anything at or below the last seq they sent a viewer, and clients insert by seq. Joining is a snapshot of the last 50 broadcast comments from the gateway’s in-memory ring, or from the recent-comments cache when the gateway is the first to subscribe, subscribing before reading so nothing falls in between. A reconnect sends Last-Event-ID: within the last 200 broadcast comments it gets exactly what it missed, and beyond that a fresh snapshot with a gap marker, because nobody wants five minutes of stale live chat. Retractions travel on the same partition, so a retraction can never overtake the comment it removes.

1. The router that owns a partition crashes mid-batch. What happens?
   The consumer group hands the partition to another router within seconds, and it resumes from the last committed offset, so it republishes some comments the old router already sent. Because the sequence is the offset, the republished comments carry the same numbers, gateways drop what they already delivered, and the cache writes are idempotent. Sampling state is rebuilt from the last 10 s of the partition. Viewers see a pause of a few seconds and no duplicates.
2. A viewer’s video is 15 seconds behind the author’s, and the comments spoil the goal. What can you do?
   Every comment carries the author’s video timestamp, the stream time they were watching when they typed. The client knows its own playhead, so it can hold each comment until its playhead reaches that time, capped at about 20 s. That trades wall-clock latency for being in step with the video, which is what the viewer actually experiences, so I would turn it on for sports and other spoiler-sensitive streams. Replays always work this way, which is how comments stay in sync with a recording.
3. The author sees their comment at the bottom right away, but in the real stream it lands 30 comments higher. Does it jump?
   No. The client keeps its optimistic copy where it drew it and, when the broadcast copy arrives, matches it by the comment id from the 201 and drops the duplicate. For the author the order is "mine first", as in every chat product, and for everyone else it is seq order. Nobody can see both views at once, so the difference is invisible.
4. A million people join the big stream in the first minute. What does the recent-comments cache see?
   Almost nothing. A join is served from the gateway’s own ring buffer whenever that gateway already holds the stream, which after the first few seconds is all 200 of them. The cache sees one read per gateway subscription, a few hundred for the whole surge, instead of 16 k reads a second on one hot key.
5. Why not let a reconnecting viewer replay everything they missed?
   They can scroll back from the store if they want to. But live chat is about now: a returning viewer wants context, roughly the last minute, and then the live flow. Replaying long gaps on reconnect would also make reconnect storms expensive, because every returning viewer would pull minutes of comments at the same moment.

Ask: An ML classifier needs about a second. How do you keep slurs and spam off a stream with 5 million viewers?

Good answers name: Sync cheap filter, async ML with retraction events, ML hold only on sampled streams, Synchronous ML on every comment, Reports and human review only, Client-side word filters.

Our pick: Every comment passes the sync filter in under 5 ms: blocked terms and links (global and the creator’s own), bans at user, creator and stream level, per-user rate limits, account-age rules and duplicate text. It rejects with a reason, or for spam it shadow-accepts, so only the author sees the comment and the spammer learns nothing. The ML job consumes every comment from the log and, above its threshold, asks the API to retract; the API appends a retraction to the stream’s partition, so it follows the comment through the router (which drops it from the recent cache), to every gateway (which forwards it to viewers who were sent the comment) and to the store as a soft delete with a reason. On sampled streams the router holds sample candidates for up to 1 s, and a comment retracted in that window never enters the sample; if the classifier falls behind, the router samples only from trusted authors until it catches up, so the biggest stream degrades to safer, not to unfiltered. Repeat offenders are banned automatically, and new bans reach every filter instance within seconds.

1. The ML fleet is down for ten minutes. What happens?
   Posting carries on, because the sync filter is local to each API instance. Small streams run on the filter alone, and sampled streams draw their sample only from the creator, moderators and established accounts. When the classifier recovers it works through the backlog from its committed offset and retracts anything it should have caught, so comments from the outage are still judged, just late. The alert is on the classifier’s lag, not its error rate.
2. Could the retraction reach a viewer before the comment does?
   Not on the normal path: they share a partition, the router publishes in partition order, and gateways forward in seq order. The edge case is a viewer who reconnects after the retraction was applied, and there the recent cache has already dropped the comment, so it is simply never sent. Clients also ignore retractions for ids they never received.
3. Why shadow-accept spam instead of rejecting it?
   A rejected spammer learns immediately that the filter caught them and edits the text until it passes; a shadow-accepted one sees their comment in the chat and keeps wasting effort on something nobody else sees. I would use it for spam and abuse only. For an ordinary policy problem, like a word the creator blocked, the honest answer is a rejection with a reason so a real person can fix their comment.
4. How fast does a ban take effect?
   The ban is written once and pushed to every sync-filter sidecar over a config channel, so it applies to the user’s next comment within a couple of seconds. At the same moment the creator’s "ban and remove messages" action retracts that user’s recent comments on the stream, one retraction event per comment, through the same path as an ML takedown.
5. How do you measure whether moderation is good enough?
   By exposure, not counts: impressions of comments that were later removed, per million impressions, broken down by stream size. Also time from post to takedown at p50 and p99, false positives from appeals and from a sample of decisions reviewed by people, and how often big streams run in trusted-only mode, which says how often the classifier is late.