Design Live Comments

Comments and reactions on live video that reach viewers within two seconds, at 50 M concurrent viewers and 5 M on one stream.

Last updated 2026-10-26. Difficulty: hard. Patterns: broadcast, pub-sub, sse, sampling, moderation. Reported at Meta, TikTok, Amazon, Google, Microsoft, LinkedIn.

Functional requirements

• Post a comment on a live video. Short text and emoji, sent as a plain POST. The author sees their own comment instantly, before the server confirms it.
• See other viewers’ comments as they are posted. Within about 2 seconds of posting, on every stream from 3 viewers to 5 million.
• Late joiners see recent comments. Opening a stream shows the last 50 comments at once; a dropped connection resumes without gaps or duplicates.
• Readable comments on huge streams. Above about 50 comments a second, each viewer sees around 10 a second: a shared, quality-ranked sample plus the creator, moderators, their own friends and themselves.
• Reactions shown as bursts. Hearts and other reactions are counted per stream per second and drawn as floating bursts. Individual taps are never delivered to other viewers.
• Moderation. Obvious abuse is rejected before it is shown; comments flagged later by ML, reports or the creator vanish from every screen within seconds. Creators can ban users, block words and turn on slow mode.
• Live viewer count. Approximate, refreshed every few seconds.
• Scroll back and replay. Older comments page in from storage. After the stream ends, comments replay in step with the video, using the video timestamp stored with each one.
• Out of scope. The video pipeline itself (ingest, transcoding, HLS and the CDN), "went live" notifications, threaded replies, comment search and ads.

Non-functional requirements

• Scale (50 M concurrent · 5 M on one stream). Heavy tailed: most of the ~500 k concurrent streams have fewer than 10 viewers and a few have millions. Concurrency sizes the gateway fleet; the biggest stream sizes the fan-out tree.
• Read/write asymmetry (1 write → up to 5 M reads). The requirement that drives the hardest tradeoff. Every choice keeps the cost of one comment proportional to the number of gateway servers its viewers sit on, never to the number of viewers.
• Delivery latency (p99 < 2 s post to screen). Budget: API and filter ~50 ms, log ~20 ms, router batch ≤ 100 ms, relay tree ~50 ms, gateway flush ≤ 250 ms, last mile 100–300 ms. On sampled streams the ML hold (up to 1 s) spends most of the slack.
• Ordering (per stream, no global order). Any two comments that two viewers both see appear in the same order for both. Nothing is ordered across streams, which is what lets every stream be handled independently.
• Delivery guarantee (durable once acked · live best effort). A 201 means the comment is in the log and will be stored. The live push may legitimately skip it (sampling, overload); no viewer is promised every comment.
• Availability (99.95 % for viewing). Comments must never affect video playback. Under overload, shed in this order: reaction animations, the sample size, then comments pause while the video keeps playing.
• Moderation latency (takedown p99 < 5 s). What matters is exposure: impressions of a bad comment before it is removed. One second on a 5 M stream is millions of impressions; on a 20-viewer stream it is 20.

Back-of-envelope estimates

• Gateway servers: ~1,000. 50 M concurrent viewers ÷ 50 k SSE connections per box = 1,000 gateways. A tuned box holds more idle connections than that; 50 k leaves CPU for up to 4 flushes a second per viewer and for absorbing a failed neighbour’s viewers.
• Comments posted per second: ~42 k/s. Assume each viewer comments about once every 20 minutes: 50 M ÷ 1,200 s ≈ 42 k/s at peak. A Kafka cluster of a dozen brokers absorbs that. Writes are not the hard part.
• Naive deliveries on the hottest stream: 25 billion/s. 5 k comments/s × 5 M viewers = 25 billion deliveries a second if every comment goes to every viewer. Impossible, and pointless: nobody can read 5,000 comments a second. This is the number that forces sampling.
• Hottest stream after sampling: ~50 M deliveries/s in ~20 M frames/s. Each viewer is shown about 10 comments/s: 5 M × 10 = 50 M comment deliveries a second. Gateways flush each viewer at most 4 times a second, so 5 M × 4 = 20 M SSE writes/s, about 100 k per gateway across the stream’s 200 gateways.
• Backbone copies per comment, random placement: ~630 for a 1,000-viewer stream. With viewers spread at random over 1,000 gateways, a 1,000-viewer stream touches 1,000 × (1 − (1 − 1/1,000)^1,000) ≈ 1,000 × (1 − 1/e) ≈ 632 gateways, so each comment is copied ~632 times. With stream affinity it touches 1. Random placement makes medium streams cost almost one copy per viewer.
• Hottest stream on the backbone: 200 gateways · 2 k msgs/s. Affinity puts at most 25 k of one stream’s viewers on a gateway: 5 M ÷ 25 k = 200 gateways. The router publishes 10 batches/s, so 200 × 10 = 2,000 deliveries a second. Every comment rides along for friend matching, ~300 B each: 1.5 MB/s per gateway, ~300 MB/s across the tree.
• Reaction ingest: ~3 M taps/s → ~500 k req/s. Assume 2 % of 50 M viewers are tapping at any moment, at 3 taps/s: 3 M taps/s. Clients batch and send once every 2 s while tapping: 1 M ÷ 2 = 500 k requests/s, more than 10× the comment rate. Reactions, not comments, are the biggest write stream.
• Comment storage per year: ~160 TB. Average concurrency over a day is ~40 % of peak, so ~17 k/s ≈ 1.5 B comments a day. At ~300 B each with metadata that is 450 GB/day ≈ 160 TB/yr before replication. Big, but ordinary for a wide-column store.

Components

• Viewer app (web · iOS · Android): Plays the video and holds one SSE stream for the comments of the stream in focus, over the same HTTP/2 connection it uses for every other API call. Posts comments and batched reaction taps as ordinary POSTs and renders its own comment immediately. Remembers the last sequence number it rendered, which the browser sends back as Last-Event-ID on reconnect.
• Edge LB (L7 · HTTP/2 · stream affinity): Terminates TLS and HTTP/2 at the edge and routes each request on its own: POSTs to the Comment API, live streams to a gateway chosen by stream id. Placement is consistent hashing of stream_id with bounded load: a stream gets the next k gateways on the ring, with k sized from its viewer count in the Stream directory, so viewers of one stream share gateways instead of scattering across the fleet.
• Live gateways (SSE · 50 k conns/box): About 50 k SSE connections per box, 1,000 boxes at peak. A gateway subscribes to a stream’s topic when its first local viewer arrives and unsubscribes 30 s after the last leaves, keeps the last 500 broadcast comments per stream in memory for joins and reconnects, and flushes each viewer at most every 250 ms. On sampled streams it builds each viewer’s frame: the shared sample, pinned comments, friends’ comments from a local friend index, and the viewer’s own.
• Comment API (POST · history · retract): Stateless write path. Authenticates, applies the sync filter, and appends comments, reaction batches and retractions to the event log; the produce ack returns the partition offset, which becomes the comment’s sequence number. Also serves older comments from the store for scroll-back and replay, and is the single entry point for takedowns from ML, reports and creators.
• Sync filter (rules · bans · < 5 ms): The cheap moderation check every comment passes before it is accepted: global and per-creator blocked terms and links, bans at user, creator and stream level, per-user rate limits (one comment every 2 s per stream by default, tighter in slow mode) and a duplicate-text check. Runs as a sidecar next to each API instance with rules and bans pushed to it, so the check is a local call. It can reject, or shadow-accept spam so only its author sees it.
• Event log (Kafka · keyed by stream): Kafka with two topics. Each live stream is pinned to one comments partition for its whole life, so the partition offset is a per-stream sequence that only grows and replays to the same values. Reactions are keyed by (stream_id, viewer_id mod 16) so a burst of taps on one stream spreads over 16 partitions. Consumers: stream routers, the archiver, ML moderation and the reaction aggregator.
• Stream router (partition owner · sampler): Exactly one router owns each partition (a consumer group), so it sees every stream’s comments in order and can keep per-stream state: the comment rate, the sampled-mode flag and the scores. It appends broadcast comments to the recent cache, batches each stream for up to 100 ms and publishes one message per batch. Above 50 comments/s it switches a stream to sampled mode and tags each comment pinned, sample or rest.
• Relay tree (per-stream topics · 2 tiers): Per-stream topics sharded by stream_id across relay nodes. Gateways subscribe to the streams their viewers watch, so each batch crosses the backbone once per subscribed gateway, never once per viewer. A topic with more than ~50 subscribers gets a second tier of relays, each feeding ~15 gateways, so no one node pushes the hottest stream’s 300 MB/s alone. Fire and forget: a gateway that misses a batch sees the sequence gap and refills from the recent cache.
• Recent comments (Redis · last 500 per stream): A Redis sorted set per stream, scored by sequence number, holding the last 500 broadcast comments: everything on a normal stream, only the pinned and sample tiers on a sampled one. Read only when a gateway first subscribes to a stream or a reconnect needs more than the gateway’s own buffer holds, so a million joins cost a few hundred reads, not a million. Expires a day after the stream ends.
• Comments store (Cassandra · (stream, 10 min)): Every comment, for scroll-back, replay, appeals and audit. Partition key (stream_id, 10-minute bucket), clustered by sequence; streams in sampled mode add a 16-way shard to the key, because 5 k comments/s for 10 minutes is ~900 MB, far past a healthy partition. Each row keeps the author’s video timestamp so a replay shows comments in step with the video. Takedowns are soft deletes with a reason.
• Reaction aggregator (Flink · 1 s windows): A streaming job in two stages: the first sums taps per (stream, shard, second), the second merges the 16 shards and publishes one counts event per stream per second to the stream’s topic. Counts are approximate by design: a lost batch shifts a burst by a few hearts, which nobody can see. Batches carry a client batch id so a retried batch is not counted twice.
• ML moderation (GPU classifier · ~1 s): Consumes every comment from the log and scores it with a batched GPU classifier (hate, harassment, spam) within about 1 s at p99. Above the takedown threshold it asks the Comment API to retract the comment, the same path user reports and creator deletes take. On sampled streams the router holds sample candidates for up to 1 s, so a comment retracted in that window never reaches the shared sample.
• Social graph (top friends · cached): The existing social graph service. A gateway calls it once when a viewer joins a sampled stream, fetches that viewer’s top ~200 friends, and inverts the lists into a local map from author to the local viewers who should see that author’s comments. Results are cached; a failure only means no friend boost for that viewer.
• Stream directory (viewer counts · placement): Small replicated state per live stream: total viewers, summed from per-gateway counts reported every 5 s with a 15 s TTL so a dead gateway’s viewers drop out by themselves, and the placement width k = ceil(viewers ÷ 25 k). The load balancer keeps placements in memory with changes pushed within seconds; gateways get the viewer totals back in the response to their report.

User flows

1. Post a comment and see it everywhere. The core path: a comment is accepted, sequenced by the log, and copied once per gateway rather than once per viewer. Non-trivial because the same path has to work for a stream with 3 viewers and one with 5 million.
   1. Viewer posts a comment, and the client renders it at once. The POST travels on the HTTP/2 connection the app already has open, so there is no handshake. The client draws the comment immediately; the author never waits on the network, which is why a plain POST is fast enough. client_comment_id makes a retry after a timeout harmless.
   2. API runs the sync filter: rate limit, bans, blocked terms and duplicate text. The filter is a sidecar with rules pushed to it, so this is a local call under 5 ms. Outcomes: allow, reject with a reason (422, or 429 for rate limits), or shadow-accept spam so only the author sees it. If the sidecar fails, established accounts are let through and new accounts are held back.
   3. API appends the comment to the stream’s partition of the event log; the offset becomes its sequence number. The stream was pinned to a partition when it went live, so all its comments sit in one ordered log. The produce ack carries the offset, which the API returns as seq. Once the ack is in, the comment is durable: everything downstream can be replayed from here.
   4. The stream router that owns the partition consumes it and appends the comment to the recent-comments cache. One router per partition means each stream’s comments are processed in order by one process, which is what makes the sequence meaningful. The cache keeps the last 500 broadcast comments per stream; writing the same member at the same score is idempotent, so a replay after a crash does no harm.
   5. Router batches the stream’s comments for up to 100 ms and publishes one message per batch to the stream’s topic. Pub/sub cost is per message, so batching turns the hottest stream’s 5 k comments a second into 10 messages a second. A stream that has been quiet is published immediately, so a lone comment is never held.
   6. The relay tier delivers the batch once to every gateway subscribed to the stream. For a typical stream that is one gateway, because stream affinity put all its viewers there. For the 5 M stream it is 200 gateways, reached through a second tier of relays. The multiplication by viewers has still not happened.
   7. Each gateway writes the batch into the SSE streams of its viewers of that stream. This is where the factor of millions is paid, in memory on 200 boxes, about 25 k writes each. Gateways flush each viewer at most every 250 ms, merging comments, reaction counts and the viewer count into one event. The event id is the last seq, which the browser sends back on reconnect.
   8. Separately, an archiver consumer writes every comment to the comments store. A separate consumer group, so a slow store can never delay the live path. Writes are idempotent on (stream_id, bucket, seq), so redelivery after a rebalance is harmless.
2. Late joiner opens a live stream. A viewer arrives mid-stream and must see recent comments at once, then the live flow, without hammering a shared store when a million others arrive in the same minute.
   1. Viewer opens the live comments stream; the edge LB looks up the stream’s placement. The LB holds placements in memory and has changes pushed to it, so the lookup is almost always local. Placement is the stream’s width k: how many gateways its viewers are spread over.
   2. LB hashes the stream id onto the gateway ring and picks the least loaded of its k gateways. For a small stream k is 1, so every viewer lands on the same box and the stream costs one subscription. Bounded load caps any gateway at 1.25× the fleet average and spills to the next gateway on the ring when a set is full.
   3. If no local viewer was watching yet, the gateway subscribes to the stream’s topic and then loads the last 50 comments from the recent cache. Subscribe first, then read: a comment published in between is in the subscription or the cache, possibly both, and the seq dedupes it. From now on the gateway keeps a ring buffer of the last 500 broadcast comments for this stream.
   4. If the gateway already holds the stream, it builds the snapshot from its own ring buffer. This is what absorbs a join storm: once a stream is on a gateway, every further join is served from memory. A million joins in a minute on the big stream become a few hundred cache reads in total.
   5. Gateway sends the snapshot, then streams live batches as they arrive. The snapshot carries the viewer count and the current reaction rate so the screen looks alive immediately. From here the viewer is on the live path of the post flow.
   6. Gateway reports its per-stream viewer counts to the directory every 5 s. Each gateway’s entry has a 15 s TTL, so a dead gateway’s viewers drop out by themselves. The directory sums the entries into the viewer count and the placement width, and returns the totals so gateways can show the count.
   7. Viewer scrolls up for older comments; the API pages them from the store. Scroll-back is an ordinary paginated read by seq, walking 10-minute buckets backwards. It is rare and not latency critical, so it stays off the live path entirely.
3. A stream hits 5 M viewers and 5 k comments/s. The scale-breaking case. Placement widens, the router switches to sampled mode, the relay tree grows a tier, and reactions become per-second counts, so each viewer gets about 10 comments a second instead of 5,000.
   1. Viewer counts climb, and the directory widens the stream’s gateway set as they grow. k = ceil(viewers ÷ 25 k), so 5 M viewers span 200 gateways. k grows within one report cycle and shrinks only after 10 minutes, and existing connections never move: only new joins go to the new gateways.
   2. The router sees the comment rate pass 50 a second and switches the stream to sampled mode. The threshold is roughly what a person can read. Hysteresis (on above 50/s for 10 s, off below 30/s for 30 s) stops it flapping. The mode is announced in the stream so clients can label the view "top comments".
   3. Router scores each comment and tags it pinned, sample or rest. Pinned: the creator and moderators, always shown, capped at 2/s. Sample: the best scored, about 10/s, at most one per author per minute, admitted only after the ML hold of up to 1 s. Rest: everything else, still stored, still shown to the author and their friends.
   4. Router publishes 10 batches a second; the relay tree copies each one to the 200 gateways. 2,000 backbone deliveries a second. Because every comment rides along for friend matching, each gateway takes ~1.5 MB/s and the tree ~300 MB/s, spread over about 14 second-tier relays so no single node carries the topic.
   5. Each gateway builds every viewer’s frame: pinned, sample, friends’ comments and their own. When a viewer joins a sampled stream the gateway fetches their top 200 friends once and inverts the lists into author → local viewers, about 5 M entries for 25 k viewers. Each rest-tier comment then costs one hash lookup to find who should see it. Frames are capped at about 10 comments a second.
   6. Viewers’ heart taps are batched on the client, posted every 2 s, and logged across 16 shards per stream. A goal on this stream can mean 1.5 M taps a second. Client batching cuts that to one request per tapping viewer every 2 s, and keying by (stream_id, viewer_id mod 16) spreads the burst over 16 partitions instead of one.
   7. The aggregator sums taps per stream per second and publishes one counts event, which gateways merge into their next flush. Stage one sums each shard per second; stage two merges the 16 shards. The client turns the count into a burst on a log scale, capped at about 20 floating hearts a second, and animates its own taps instantly.
4. A gateway dies and its viewers reconnect. The failure path. Fifty thousand connections drop at once; they must land elsewhere, resume exactly where they were, and not stampede any shared store.
   1. A gateway crashes, dropping its 50 k SSE streams. Its subscriptions die with its relay connections, so the relay tier stops sending to it at once. Nothing durable lived on the gateway: its ring buffers were copies of what the cache and the log already hold.
   2. Clients reconnect with jittered backoff, sending the last seq they rendered. Jitter spreads 50 k reconnects over a few seconds instead of one instant. Browsers send Last-Event-ID automatically; native clients store it themselves.
   3. The LB routes each viewer to the remaining gateways in its stream’s set. The dead box leaves the ring after missed health checks. Viewers of the big stream spread over its other 199 gateways, about 125 each; small streams move to the next gateway on the ring. Bounded load keeps any one survivor from taking too many.
   4. The new gateway subscribes if it needs to and fills the gap from its ring buffer or the recent cache. A gap within the last 200 broadcast comments is replayed exactly. Beyond that the viewer gets a fresh snapshot with gap: true, because minutes-old live chat is not worth reading. On the big stream the survivors already hold the subscription and the buffer, so most viewers are served from memory.
   5. Client inserts by seq and drops anything it already rendered. Overlap between the gap fill and the first live batch is normal; deduping on seq makes it invisible. The viewer sees at most a brief pause.
   6. If a relay dies instead, its gateways resubscribe through a sibling and fill any gap the same way. The gateway notices either the dropped relay connection or a jump in seq on the next batch. Viewers are untouched: the gateway repairs the gap before their next flush.
   7. The directory drops the dead gateway’s counts when their 15 s TTL runs out. The surviving gateways report the moved viewers in their next 5 s cycle, so the viewer count dips at most briefly and never double counts.
5. ML takes down a comment already on screen. The asynchronous path. A comment passed the cheap filter and was shown; a second later the classifier flags it, and it must vanish from every screen, the cache and the replay.
   1. ML moderation consumes the comment from the log and scores it. Its own consumer group, batching comments onto GPUs, p99 about 1 s. User reports and creator deletes skip this step and start at the next one.
   2. Above the threshold, ML asks the Comment API to retract it. One entry point for every takedown source, so auditing, appeals and creator notifications are written once. The request carries the model version so decisions can be traced and reversed if a model misbehaves.
   3. API appends a retraction event to the same stream partition. Same partition as the comment, so the retraction is ordered after it on every downstream path and can never overtake it. On a sampled stream, a retraction that lands within the 1 s hold keeps the comment out of the sample entirely.
   4. Router drops the comment from the recent cache and publishes the retraction. Removing it from the cache means no late joiner or reconnect can receive it again. The retraction rides in the next batch like any other event.
   5. Gateways forward the retraction to viewers who were sent the comment, and clients remove it. Gateways keep the ids they sent in the last few minutes per stream, so on a sampled stream the retraction reaches only the friends who saw a rest-tier comment. Clients ignore retractions for ids they never had.
   6. The archiver soft-deletes the comment in the store, with its reason. Kept for appeals, audit and legal holds, but never served to scroll-back or replay. If an appeal reverses the decision, a restore event travels the same path.

Deep dives

The theory behind it

• Live video streaming: How live streams reach millions of viewers: ingest with RTMP, SRT or WebRTC, real-time transcoding, segmenting for HLS and DASH, low-latency HLS, CDN fan-out, WebRTC for interactive streams, latency trade-offs, live chat at scale, recording and reliability.
• WebSockets vs Server-Sent Events vs long polling: Choosing how a server pushes updates to clients: short polling, long polling, Server-Sent Events and WebSockets compared on direction, latency, overhead, proxies, scaling and mobile behaviour, plus WebTransport and push notifications, with a decision guide for interviews.
• Real-time systems: WebSockets, SSE and push: Long polling vs Server-Sent Events vs WebSockets, connection servers and registries, pub/sub fan-out, presence, reconnection and resync, and scaling to millions of connections.

Related

• Design Nearby Friends
• Design a Content Moderation System
• All 83 system design interview questions
• Run this question for a friend: the interviewer kit