Updated 24 October 2026

Real-time systems: WebSockets, SSE and push

Long polling vs Server-Sent Events vs WebSockets, connection servers and registries, pub/sub fan-out, presence, reconnection and resync, and scaling to millions of connections.

Try it yourself, in a playground:Polling, long polling and WebSocketsOnline presenceWebSocket routing

Chat, notifications, live dashboards, collaborative editing, multiplayer games and ride tracking all need the server to push data to the client the moment it changes. HTTP was built the other way round. The design questions are how the push channel works, how a message finds the right connection among millions, and what happens when the connection drops.

Transport options

Short polling. The client asks every N seconds. Trivial, wasteful, latency up to N. Acceptable for a status page that changes rarely.

Long polling. The client asks and the server holds the request open until there is data or a timeout (say 30 s), then the client immediately asks again. Works through every proxy and firewall, near-real-time, but each message costs a full HTTP round trip and servers hold many open requests.

Server-Sent Events (SSE). One long-lived HTTP response that streams text/event-stream messages from server to client. Simple, auto-reconnects with Last-Event-ID, works over HTTP/2 multiplexing, but one-directional: the client sends via ordinary requests. Ideal for notifications, feeds, and streaming AI responses.

WebSockets. A persistent bidirectional TCP connection upgraded from HTTP. Lowest latency and overhead per message, the right choice for chat, games, collaborative editing, and anything with frequent client-to-server messages. Costs: stateful servers, load balancers must support the upgrade and long connections, and you own the reconnection logic.

Default: WebSockets for bidirectional, SSE for server-to-client only, long polling as a fallback. Mobile background delivery is a separate channel through APNs and FCM because the OS kills sockets.

The connection tier

Long-lived connections need their own tier of connection servers (gateways) whose only job is holding sockets, authenticating them, and forwarding frames. They are stateful in exactly one way: they know which users are connected to them. A server holds 100 k to 1 M idle connections depending on memory and heartbeat load; 10 M concurrent users is 20 to 100 servers.

Behind them, business services stay stateless. A message sent by user A goes to a message service over an ordinary RPC, is persisted, and then must be delivered to every connected member of the channel, wherever their sockets are. That is the fan-out problem.

Finding the connection: the registry

A connection registry maps user_id → connection server(s). Redis is the usual store: on connect, SADD conns:{user_id} server-17 with a TTL refreshed by heartbeats; on disconnect or server death, remove. To deliver to a user, look up their servers and forward. For large groups, a lookup per member is expensive; the alternatives:

• Pub/sub per channel. Each connection server subscribes (Redis pub/sub, Kafka, or a purpose-built broker) to the topics its connected users belong to. A message is published once per channel; every server with a subscriber receives it and forwards locally. No per-member lookup, and the subscription count is bounded by (servers × channels with a member on that server).
• Per-user delivery queues for reliable inbox semantics: each user has a queue, workers push into it, the connection server drains it. Heavier but gives at-least-once delivery.

Slack, Discord and Facebook Messenger all use a variant of the pub/sub approach with a registry for direct lookups.

Fan-out at scale

A channel with 50 k online members turns one send into 50 k deliveries. At 10 messages per second in that channel, that is 500 k frames per second from one source. Bound it: publish once per channel and let servers fan out locally (they already know their local subscribers); batch frames per server; and for very large channels, degrade features (no typing indicators, coalesced presence) so the per-message cost stays small. State the number: "fan-out is the largest write stream in the system, larger than the messages themselves".

Presence and typing

Presence (online/away) is the highest-volume, lowest-value signal. Every connect, disconnect and idle change is an event fanned out to everyone who can see that user; with large teams it exceeds message traffic by orders of magnitude. Design it lossy: heartbeats every 30 s to Redis with a 60-s TTL; presence changes batched per few seconds and only delivered for users currently visible on screen (the client subscribes to the presence of the 50 users in view, not all 10 000 in the workspace). Typing indicators are the same, with a 3-second TTL and no persistence.

Ordering, acknowledgement and resync

Sockets drop: mobile networks, laptop lids, deploys. Design for it as the normal case.

• Ids are the cursor. Every message has a time-ordered id per channel. The client tracks the last id it has seen per channel.
• Acknowledge. The server acks a client send with the assigned id; the client marks its optimistic message as sent. Unacked sends are retried with the same client_msg_id so the server deduplicates.
• Resync on reconnect. The client sends its last-seen ids; the server returns everything after them from the store (the gap), then resumes streaming. If the gap is too large (offline a week), the client refetches recent history instead.
• At-least-once plus dedupe is the delivery guarantee. Exactly-once over a socket is not achievable; idempotent clients make it unnecessary.

Heartbeats and half-open connections

TCP does not notice a dead peer for a long time. Both sides send pings (every 20 to 30 s) and close the connection after two missed pongs. The server uses the same interval to refresh the registry TTL, so a crashed server's entries expire on their own. Keep the ping interval short enough for NAT and load balancer idle timeouts (often 60 s).

Load balancing and deploys

WebSockets need an L4 or upgrade-aware L7 balancer with long idle timeouts, and a routing decision that is sticky by nature (the connection stays where it landed). Deploying connection servers means draining: stop accepting new connections, tell clients to reconnect (a GOAWAY-style frame) over a few minutes so they do not all reconnect at once, then stop. A restart without draining reconnects all users in one second, which is a thundering herd on auth and the registry; stagger with client-side jitter.

In the interview

"Clients hold a WebSocket to a connection-server tier, 500 k connections per server. A registry in Redis maps user to server with a heartbeat-refreshed TTL. Sends go over RPC to the message service, which persists, then publishes once per channel to a pub/sub layer that connection servers subscribe to for their local users, so fan-out cost is per-server, not per-member. Each message has a time-ordered id; clients resync from their last seen id on reconnect, and retries carry a client message id for dedupe. Presence is lossy: 30-second heartbeats, delivered only for users on screen."

Sizing a connection tier

The numbers that make a real-time answer concrete:

• A tuned server holds 50–100 k idle WebSocket connections; budget 25 k per box to leave CPU for fan-out. Memory is roughly 10–50 KB per connection with modest buffers.
• 1 M concurrent connections therefore needs ~40 boxes at the budgeted density, spread over at least three availability zones.
• A connection that sends a heartbeat every 30 s produces 1 M / 30 = 33 k messages/s of pure overhead at 1 M connections: heartbeats are a real load, which is why the interval is a design decision.
• Fan-out is the number that hurts: 5 k messages/s into channels averaging 100 online members is 500 k pushes/s, and that is what sizes the tier, not the connection count.

Reconnect storms

The failure mode unique to persistent connections: a deploy, an LB change or a network blip disconnects a large share of clients at once, and they all reconnect within the same second: a synchronised thundering herd against the auth service, the session store and the subscription registry.

The defences, and you should name at least three: exponential backoff with jitter on the client; a server-provided reconnect delay in the close frame so you can spread clients deliberately; connection draining a few percent at a time during deploys; caching the auth and subscription lookups that a reconnect performs; and a resume token so a reconnect is a cheap "catch me up from cursor X" rather than a full state rebuild.

Mobile realities

Phones suspend sockets in the background, switch between wifi and cellular constantly, and are killed by the OS. A design that assumes a stable socket is wrong.

• Use push notifications (APNs, FCM) for background delivery, and treat them as best-effort hints that wake the app; the app then resyncs from its cursor over the normal path.
• On foreground, resume from a cursor rather than replaying everything.
• Keep the heartbeat interval in the 30–60 s range as a compromise between battery and dead-connection detection, and back off when the app is idle.
• Assume duplicates, assume reordering, and dedupe by message id on the client.

Checklist

• Transport, and why (WebSocket, SSE, long polling, push).
• Connection count, per-box density and the resulting fleet size.
• Where the connection registry lives and how fan-out finds a user.
• Ordering and delivery guarantees, plus the resync-from-cursor path.
• Heartbeat interval and half-open detection.
• Deploy and reconnect-storm strategy.

Questions people ask

What is a real-time system in system design?

A real-time system pushes data from the server to the client the moment it changes, instead of waiting for the client to ask. Chat, notifications, live dashboards, collaborative editing, multiplayer games and ride tracking all need it. HTTP was built the other way round, so the design questions are how the push channel works, how a message finds the right connection among millions, and what happens when the connection drops.

WebSockets vs Server-Sent Events: which should you use?

Use WebSockets when the client also sends frequent messages, and Server-Sent Events (SSE) when data only flows from server to client. A WebSocket is a persistent bidirectional connection upgraded from HTTP, with the lowest latency and overhead per message, which suits chat, games and collaborative editing. SSE is one long-lived HTTP response streaming text/event-stream messages; it auto-reconnects with Last-Event-ID and suits notifications, feeds and streaming AI responses, while the client sends via ordinary requests.

What is the difference between long polling and WebSockets?

Long polling repeats ordinary HTTP requests that the server holds open, while a WebSocket is one persistent bidirectional connection. With long polling the server answers when there is data or a timeout (say 30 seconds) and the client immediately asks again; it works through every proxy and firewall, but each message costs a full HTTP round trip. WebSockets have the lowest per-message overhead but need stateful servers, upgrade-aware load balancers and client reconnection logic, so long polling is the fallback.

How do you scale WebSockets to millions of connections?

Put the sockets on a dedicated tier of connection servers that only hold, authenticate and forward them, and keep business services stateless behind it. A registry in Redis maps each user to their server, and each message is published once per channel to a pub/sub layer that servers subscribe to for their local users, so fan-out cost is per server, not per member. At a budget of 25 k connections per box, 1 M concurrent connections needs about 40 boxes across at least three availability zones.

How do clients avoid losing messages when a WebSocket reconnects?

They resync from a cursor: every message has a time-ordered id per channel, the client sends its last-seen ids on reconnect, and the server returns everything after them from the store before resuming the stream. Unacknowledged sends are retried with the same client_msg_id so the server deduplicates, which gives at-least-once delivery plus dedupe; exactly-once over a socket is not achievable. If the gap is too large, such as a week offline, the client refetches recent history instead.

What is a reconnect storm and how do you prevent it?

A reconnect storm happens when a deploy, load balancer change or network blip disconnects a large share of clients and they all reconnect in the same second, a thundering herd against the auth service, the session store and the subscription registry. The defences are exponential backoff with jitter on the client, a server-provided reconnect delay in the close frame, draining connections a few percent at a time during deploys, caching the lookups a reconnect performs, and a resume token so reconnecting is a cheap catch-up from a cursor.