Design Figma

500 k design files open at once, some with 200 people: objects converge live, cursors stream, and 300 MB files open in seconds.

Last updated 2026-10-26. Difficulty: hard. Patterns: multiplayer, last-writer-wins, websockets, fractional-indexing, checkpoints. Reported at Figma, Adobe, Atlassian, Notion, Microsoft, Google.

Functional requirements

• Many people edit one file at once. Shapes, frames, text, components and their instances. A change shows up for everyone in the file within a couple of hundred milliseconds, and every client ends up showing the same document.
• Local edits never wait. Creating, moving, resizing and restyling apply on the next frame. Even creating an object needs no round trip, which is why object ids are generated on the client.
• Structure edits. Group, ungroup, move a layer into another frame, reorder layers. Concurrent moves must never lose an object or make a frame its own ancestor.
• Live cursors and selections. Who is in the file, where their pointer is, what they have selected, and follow mode for a presenter. High frequency, lossy, and never saved.
• Brief offline, then reconnect. A dropped connection or twenty minutes on a train. Edits made meanwhile are kept and applied when the connection returns, without a conflict dialog.
• Version history. Autosaved and named versions; view any of them and restore one. A restore is a new edit that everyone in the file sees, not a rewrite of the past.
• Comments. Threads pinned to an object or a point on the canvas, with mentions. A pin follows its object when someone moves it.
• Open very large files quickly. Design systems and big product files reach hundreds of MB of document data plus images. The first page must be usable in seconds, not after the whole file downloads.
• Sharing and permissions. View, comment and edit roles per file, team and link. A viewer’s socket cannot send changes, and a revoke applies to a session that is already open.
• Out of scope. The rendering engine itself, prototyping and Dev Mode, plugins, publishing component libraries across files (instances inside one file are in scope), and billing.

Non-functional requirements

• Local responsiveness (< 16 ms, next frame). The server is never on the path of showing your own change. Everything about the conflict model follows from applying edits locally first.
• Propagation to others (p95 < 200 ms same region). Edits and cursors from collaborators. Cursors are sent at 10 Hz and interpolated on the receiver, so they look smooth without 60 Hz traffic.
• Convergence (every client = server state). Once changes stop flowing, every client shows exactly the server’s document. This drives the hardest trade-off: one server ordering each file makes property-level last-writer-wins enough and avoids a general CRDT, but that server is then a single owner whose failure must be survivable.
• Durability (no acknowledged change lost). An acknowledgement means the change is in the journal. Periodic checkpoints alone would lose everything since the last one when a server dies.
• Recovery from a server crash (file live again < 10 s). Clients keep editing locally meanwhile and resend what was not acknowledged. Bounded by the lease expiry plus loading the checkpoint and the journal after it.
• Open time (first page usable < 3 s). For any file size; typical files open in under 1.5 s. A 300 MB file cannot be downloaded whole in that time, which forces page-at-a-time loading.
• Scale (500 k open files · 1 M sockets). 10 M weekly users. Most files have 1 to 5 people in them; a design review can have 200, nearly all of them watching.
• Access control (server-side · revoke < 5 s). Design files are unreleased products. Roles are checked when the socket opens and enforced on every change, and a revoke ejects a live session.

Back-of-envelope estimates

• Open WebSocket connections: ~1 M. 500 k files with someone in them × an average of 2 clients each = 1 M sockets. At ~50 k per gateway host that is about 20 gateways. Connections are not the hard part.
• Change batches per second: ~500 k/s in · ~1 M/s out. About 20 % of open files have someone mid-edit at any instant: 100 k files. Each sends ~5 coalesced batches a second (a drag flushes at up to 20 Hz, then goes quiet) = 500 k batches/s, ~300 B each. With ~3 people in an active file, each batch goes to 2 others: ~1 M pushes/s.
• Cursor updates per second: ~1.8 M/s. About 600 k clients share a file with someone else (a solo file sends no presence at all). If 30 % are moving their pointer at once at 10 updates a second: 180 k × 10 = 1.8 M/s, 3.6 times the edit rate. This is why presence gets its own lossy channel.
• The 200-person design review: ~400 k msgs/s → ~4 k/s. 200 people each sending 10 cursor updates a second, each delivered to the 199 others as its own message: 200 × 10 × 199 ≈ 400 k messages/s from one file. Batching per recipient every 50 ms caps it at 200 × 20 = 4 k/s, each listing the cursors that moved (up to ~2 KB, ~40 KB/s per viewer). Batching, not more servers, is the fix.
• Multiplayer hosts (memory): ~100–150 hosts · 3–5 k files each. 500 k open files × 20 MB average in memory (median ~4 MB; the largest decode to 1–2 GB) = 10 TB. At ~100 GB of document memory per host: 100 hosts, about 5 k files each. Run ~150 so a host loss or a burst of big files has room, ~3 k files each. Memory, not CPU, sizes this tier.
• Journal writes: ~150 MB/s · ~13 TB/day. 500 k batches/s × 300 B = 150 MB/s, about 13 TB a day. Entries expire an hour after a checkpoint covers them, so the live journal is roughly the last hour of writes: ~550 GB across all files. A write-heavy, short-retention log, nothing like a document database.
• Checkpoint writes: ~170 MB/s. 100 k actively edited files, one checkpoint every 10 minutes each, writing only the pages that changed (~1 MB on average): 100 k × 1 MB ÷ 600 s ≈ 170 MB/s, ~15 TB a day. Rewriting whole files (5 MB mean) would be 5× that. Unpinned checkpoints expire after 7 days (~100 TB rolling); pinned versions stay.
• Downloading the largest file: ~48 s whole · ~1.6 s first page. A 300 MB checkpoint over a 50 Mbps connection: 300 × 8 ÷ 50 = 48 s. The first page’s chunk, capped near 10 MB, takes ~1.6 s. Downloading the whole file before showing anything is not an option at the bandwidth users have.

Components

• Comments service (pinned to object ids): Comment threads stored as rows next to the file metadata, each pinned to an object id plus an offset, or to a canvas point. The client places the pin from the object’s current position, so a pin follows its frame when someone moves it. Deliberately outside the multiplayer document: comments need permissions, mentions, notifications and queries across files, none of which the document model is good at.
• Metadata DB (sharded Postgres): Files, projects, teams, roles, share links and comments, sharded by file or team id. Relational and strongly consistent because a revoke must be exact and comment threads are ordinary rows. Nothing in it is on the per-change path: it is read when a file is opened and written when a comment is posted or a share changes.
• File API (open · auth · session token): The HTTP entry point for opening a file: checks the caller’s role in the metadata DB and returns a signed session token (file, user, role, 10-minute expiry) and the multiplayer address. The gateway validates that token without a database call. Also serves the file browser, sharing changes and other ordinary CRUD.
• Change journal (append-only · (file, seq)): Every accepted change batch, keyed by file id and sequence number, written before the change is acknowledged or broadcast. Appends are conditional writes on (file, seq), so a new owner writes a fence entry and any late append from an old owner collides and fails. Entries expire an hour after a checkpoint covers them: a high-write, short-retention log, a DynamoDB-style table or a BookKeeper-style log.
• File router (host leases · file → owner): Maps each open file to the host and worker that own it, with an epoch that increases on every move. Leases are held per host and renewed every second, not per file, so 500 k open files cost about 150 heartbeats a second. Places a new file by memory headroom using the size of its last checkpoint. etcd or ZooKeeper class: small, strongly consistent, and off the edit path.
• Editor client (local copy · pending queue): Holds the document (or the pages loaded so far) in memory and applies every edit locally on the next frame. Creates object ids as its session number plus a counter, so making a layer needs no round trip. Keeps unacknowledged changes in a pending queue and ignores server values for any property it has a pending change on, which is what stops remote edits from flickering over your own.
• Multiplayer gateway (WebSocket · fan-out): Terminates about 50 k WebSockets per host, validates the signed session token once at connect, and routes each file’s traffic to the worker that owns it. It also fans out: an owner sends each outbound batch once per gateway and the gateway copies it to its own sockets. Because browsers stay connected to the gateway, a multiplayer host failover is a re-route behind it, not a reconnect.
• Multiplayer server (one owner per file · in memory): The authority for each open file: holds the whole document in memory as a map from object id to properties, applies changes in arrival order with last-writer-wins per (object, property), and rejects changes that would break the tree. Journals each batch, then acknowledges and broadcasts it. Each file lives on exactly one single-threaded worker, about a hundred files per worker, so ordering is local and needs no locks; it also writes the file’s checkpoints.
• Checkpoint store (object storage · page chunks): The document at a sequence number: a small manifest plus one compressed binary chunk per page, and a shared chunk for components and styles used across pages, each named by its content hash. Only changed pages are rewritten, so a checkpoint of a 300 MB file usually writes a few MB, and versions share unchanged chunks for free. Immutable objects, so the CDN can cache them forever.
• CDN (immutable chunks · images): Serves checkpoint chunks and images from the edge with year-long cache headers, because both are named by content hash and never change. Access is through short-lived signed URLs, so a cached chunk is still unreadable without access to the file. Takes the bulk of every file open off the multiplayer servers.
• Presence relay (in memory · lossy · 50 ms): Cursors, selections, viewports and who is in the file, held in memory per file and never written anywhere. Batches each file’s updates every 50 ms into one message per gateway, drops stale updates instead of queueing them, and in files over 50 people sends full-rate cursors only for people near the viewer’s viewport. Placed by consistent hashing with no lease: if a relay dies, the next updates rebuild its state within a second.
• Event bus (Kafka · keyed by file id): Low-rate file events from the multiplayer servers: checkpoint written, editing went idle, file renamed. Never individual changes, which would be 500 k messages a second that no consumer needs. Keyed by file id so each file’s events stay in order for version history, thumbnails, search and notifications.
• Version history (pinned checkpoints · restore): Turns checkpoints into versions: pins the one taken when editing goes quiet for 30 minutes, plus every named version, and lets unpinned ones expire after 7 days. Viewing a version opens its manifest read-only from the CDN. Restoring asks the file’s owner to apply the difference as ordinary changes, so the restore is itself undoable and everyone in the file sees it arrive.
• Asset store (images by content hash): Image and video fills, uploaded by the client and named by the hash of their bytes, so the same screenshot pasted into a thousand files is stored once and a repeat upload is skipped. The document holds only the hash in a fill property. Several resolutions are stored per image, so a zoomed-out canvas fetches thumbnails, not originals.

User flows

1. Open a large file. The main read path. Most of the bytes come from the CDN rather than the multiplayer server, and the first page is usable before the rest of the file has arrived.
   1. Client asks the File API to open the file; it checks the caller’s role and returns a session token. The role lookup happens once per open, not per change. The token carries file, user and role and expires in 10 minutes; the client refreshes it over the socket, which bounds how long a missed revoke could last.
   2. Client opens a WebSocket; the gateway looks up which worker owns the file. The answer is cached at the gateway and only fetched again when the epoch changes, so the router sees opens and moves, never changes. If nobody owns the file, the router places it on a host with memory to spare for the file’s size.
   3. If the file is not in memory, the owner loads its latest checkpoint and the journal entries after it. A checkpoint at, say, seq 41 200 plus at most about 10 minutes of journal. Inside the region the server pulls chunks in parallel at around 1 GB/s, so even a 300 MB file is in memory in a few seconds; most files take tens of milliseconds.
   4. The owner tells the client which checkpoint it is based on and gives this tab a session number. The client downloads that exact checkpoint and the server then sends everything after it, so nothing is missed or applied twice. The session number is unique within the file and prefixes every object id this tab creates (1874:1, 1874:2, …).
   5. Client downloads the manifest, the shared chunk and the current page’s chunk from the CDN, and renders. Chunks are named by content hash, so the CDN caches them forever and a second person opening the same file usually hits the edge. The shared chunk (components and styles used across pages) comes first, so instances on the first page render correctly. Other pages download in the background or when the user switches to them.
   6. Images are fetched lazily as they come into view, at the resolution the zoom level needs. Image bytes are usually most of a large file’s weight and none of its structure. Fetching only what is on screen, at thumbnail size when zoomed out, keeps the first render light; a placeholder with the image’s size and average colour shows until it lands.
   7. The owner streams the changes made since the checkpoint, and the client joins presence. The catch-up is at most about 10 minutes of edits, since checkpoints are taken every 10 minutes during activity. Changes to pages that are not loaded yet are buffered on the client and applied when that page arrives. From here on the client is live.
2. Two people edit the same object. The core loop. Both edits apply locally first; the server orders them, the last one it receives wins for each property, and both clients end up showing the same thing without flicker.
   1. A designer drags a frame and changes its fill; the client applies both on the next frame. The change is a set of (object id, property, value) triples: x, y and fills on object 1874:12. Nothing waits for the network, and the triples go into the pending queue in the same instant.
   2. The client coalesces changes per frame and sends at most 20 batches a second. A drag changes x and y 60 times a second, but only the latest value per property matters, so a batch carries one value per (object, property). Each batch has the session number and a batch number, which is how the server recognises a resend after a reconnect.
   3. The owner checks the role and the tree rules, then applies each property in arrival order. Last-writer-wins per (object, property), where last means last to reach this single-threaded worker, so client clocks never matter. Meanwhile a colleague set the same frame’s fill to red: whichever batch arrived second wins the fill, and x and y are untouched by that conflict. Reparents that would create a cycle and changes to deleted objects are rejected here.
   4. The batch is appended to the change journal before anyone hears about it. Group commit: all batches for a file within a 10 ms window go in one conditional write, so a busy file costs tens of writes a second, not hundreds. The entry records (session, batch) so a resend can be recognised after a failover. Adds roughly 10–20 ms, well inside the 200 ms budget.
   5. The owner acknowledges the author and broadcasts the batch to everyone else through their gateways. The ack carries the sequence number, so the author drops those changes from its pending queue. Broadcast waits for the journal, so no one ever sees a change a crash could take back. One copy goes to each gateway with subscribers to the file, and the gateway copies it to its sockets.
   6. A client with a pending change to a property ignores the server’s value for it until its own is acknowledged. Without this rule, your frame would flash to your colleague’s colour and back while your own change is in flight. Once yours is acknowledged, whatever the server sends afterwards for that property is genuinely newer and is applied, so both clients still converge. Figma has described this rule in its public writing on multiplayer.
   7. A pasted image is uploaded through the CDN to the asset store, and only its hash enters the document. The bytes never pass through the multiplayer server or the journal: a 5 MB screenshot becomes one fill property holding a 32-byte hash. The image shows locally at once, and the client sends the change after the upload completes so no one fetches a hash that does not exist yet.
3. A 200-person design review. Where per-file scale, not file count, breaks the naive design: one presenter, 199 watchers, and 400 k cursor messages a second if nothing is batched.
   1. Two hundred people open the file; their sockets are spread over about 20 gateways. All of them route to the one worker that owns the file. Most are viewers, whose tokens let them receive changes but not send any, which the owner enforces on every batch.
   2. The owner sends each outbound batch once per gateway, and each gateway copies it to its own sockets. One worker writing to 200 sockets would spend its time on syscalls and encryption. About 20 internal sends per batch, with the copying spread over the gateways, keeps the worker free to order changes for the other files it holds.
   3. Outbound changes are batched per 33 ms tick for files with many subscribers. When several people drag at once, a tick merges their batches into one message per gateway, keeping only the latest value per property. Files with a handful of people skip the tick and send immediately, so small files pay no latency for it.
   4. Every client throttles its cursor to 10 updates a second and sends it on the presence channel. Presence frames share the socket but not the path: the gateway hands them to the presence relay, never to the multiplayer server. Receivers interpolate between updates, so 10 Hz looks smooth.
   5. The presence relay merges the file’s cursors every 50 ms into one message per gateway. This turns 400 k messages a second into about 4 k: each viewer gets at most 20 presence messages a second listing the cursors that moved. Above 50 people the relay sends full-rate cursors only for people on the viewer’s page and near its viewport, and everyone else as a once-a-second avatar list.
   6. The presenter’s viewport is part of their presence, so followers’ screens track it. Follow mode needs the presenter’s page and viewport rectangle at the same 10 Hz. It is presence, so a dropped update is corrected by the next one; nothing about following touches the document.
   7. A viewer on a slow connection loses presence first, then gets a resync if changes back up. Each socket has two queues: presence keeps only the newest message, and changes queue up to a limit (2 MB or 5 seconds). Past that the gateway closes the socket and the client reloads from a checkpoint, instead of holding unbounded memory for one slow reader.
4. A multiplayer server dies mid-session. What the single owner costs: when its host dies, every file on it must come back elsewhere from the last checkpoint plus the journal, with no acknowledged change lost and no batch applied twice.
   1. A host dies; gateways lose their internal connections, but browsers stay connected and keep editing locally. Users see a reconnecting badge, not an error. Their changes keep applying locally and pile up in the pending queue; nothing they have not seen acknowledged is treated as saved.
   2. The host’s lease expires and the router reassigns its files to other hosts with a higher epoch. Leases last 5 seconds and are renewed every second, so detection takes about 5 s. About 3 000 files spread over 149 hosts is roughly 20 each, so the reload is parallel rather than a queue. Placement accounts for size, so one host does not inherit three 2 GB files.
   3. The new owner fences the journal, then loads the latest checkpoint plus every entry after it. The fence is a conditional write of a marker at the next sequence number. The old owner appended strictly in order, so if it was only paused and wakes up, its next append lands on a slot that is already taken and fails. The new owner then holds every change the old one ever acknowledged.
   4. Gateways re-route the sockets; each client reports its last seen sequence and resends its unacknowledged batches. The client cannot know whether its last batches were journaled before the crash, so it resends all of them. The owner replies with the changes after the client’s last seen sequence, and the client re-applies its pending changes on top.
   5. The owner skips any resent batch the journal already holds, by session and batch number. Re-applying is not harmless under last-writer-wins: an old batch applied again after a colleague’s newer change would undo their work. The last applied batch per session is rebuilt from the journal during the load, so this check is a memory lookup.
   6. The same path serves a laptop that was offline for twenty minutes. If the gap is within the hour of journal kept after each checkpoint, the owner sends the missed changes. Otherwise the client downloads a fresh checkpoint, re-applies its pending changes on top and sends them. Figma has described reconnecting this way: fetch the latest state, reapply offline edits, resume syncing.
5. Versions, restore and comments. Everything outside the live loop. Versions are pinned checkpoints, a restore is an ordinary edit, and comments are rows in a database; none of it can slow down a drag.
   1. Every 10 minutes of activity, the owner checkpoints the pages that changed and announces it on the bus. Encoding runs from a copy of the changed pages on a background thread, so the worker keeps applying edits. Unchanged pages keep their old chunk hash, so a checkpoint of a 300 MB file where one page changed writes a few MB. Also taken when the last editor leaves, so an idle file opens without a journal tail.
   2. Version history pins a checkpoint when editing goes quiet for 30 minutes, or when someone names a version. Pinning marks the manifest and its chunks to be kept; unpinned checkpoints expire after 7 days. Naming a version asks the owner for a checkpoint now, so the version is exactly what the user saw. Because chunks are shared by hash, a thousand versions of a mostly unchanged file cost little more than one.
   3. Opening an old version loads its manifest read-only from the CDN, with no multiplayer session. Old versions are immutable, so they need no owner, no journal and no ordering. The same chunk loader that opens a live file renders them, page by page.
   4. Restoring a version asks the owner to apply the difference as ordinary changes. The owner compares the old version with the live document per (object, property) and applies the difference as one batch: re-created objects keep their old ids, removed ones are deleted. It is journaled and broadcast like any edit, so everyone in the file sees it arrive, and restoring again undoes it. The past is never rewritten.
   5. A comment is posted to the comments service, pinned to an object id and an offset. An ordinary transactional insert into sharded Postgres, with the role checked in the same database. It never touches the multiplayer server, so a burst of comments in a review costs the live session nothing. Other people in the file get it through a subscription on the comments service.
   6. The client draws each pin from its object’s current position, so pins follow moved frames. The comment stores the object id and an offset, plus the absolute canvas point at the time of posting. If the object is later deleted, the pin falls back to that point instead of disappearing.

Deep dives

The theory behind it

• CRDTs vs operational transformation: How collaborative editors merge concurrent edits: the problem of concurrent operations, operational transformation with a central server, CRDTs for text and data, convergence and intent preservation, metadata and performance costs, presence and cursors, and which to choose.
• Stateful vs stateless services: Why stateless services scale and recover easily, where state has to live instead (databases, caches, sessions, object storage), and how to run genuinely stateful services such as WebSocket gateways, game servers, caches and stream processors with sticky routing, partitioning, replication and draining.
• Presence and managing millions of connections: Holding millions of persistent WebSocket connections and showing who is online: connection gateways, routing messages to the right server, heartbeats, presence with TTLs, fan-out of status changes, typing indicators, reconnect storms and deploys.

Related

• Design Slack
• Design Google Docs
• Design Robinhood
• All 83 system design interview questions
• Run this question for a friend: the interviewer kit