Design Dropbox

Keep a folder identical on every device: upload huge files in resumable chunks, push only what changed, and reconcile two machines that both edited the same file offline.

Last updated 2026-09-22. Difficulty: hard. Patterns: chunking, dedupe, sync, object-storage, conflict-resolution. Reported at Dropbox and 5 more with Pro.

Functional requirements

• Upload and download files of any size. A 20 GB video and a 2 KB note go through the same path. Uploads resume after a dropped connection rather than starting again.
• Sync every change to every device. A file saved on a laptop appears on the phone without the user asking. The unit of sync is a change, not a folder scan.
• Only transfer what changed. Appending a row to a 2 GB CSV must not upload 2 GB. The client sends the chunks that differ and nothing else.
• Share files and folders. With another account, or as a public link. Permissions are viewer or editor, and revoking one takes effect on the next request.
• Version history and restore. Previous versions of a file for 30 days, restorable to any device, including after a delete.
• Work offline. Edits made with no network are queued locally and reconciled on reconnect, including when the same file changed on the server.
• Out of scope. Real-time collaborative editing of a document (that is a different design: see Google Docs), full-text search of file contents, and desktop-class file locking.

Non-functional requirements

• Durability (11 nines). Losing a user's only copy of a file is the one unrecoverable failure. Everything else is an inconvenience.
• Sync latency (< 5 s for a small change). From save on one online device to visible on another. Perceived as "instant"; anything over about ten seconds is perceived as broken.
• Upload throughput (saturate the client link). Parallel chunk uploads, so a fast connection is the limit rather than our ingest path.
• Availability (99.9 % writes, 99.99 % reads). A failed sync is retried later and is survivable. A download that fails during a presentation is not.
• Metadata consistency (read-your-writes per namespace). A device that just committed a change must see it in its own next listing, or the client will loop uploading it again.
• Storage efficiency (> 2× from dedupe). The same installer, deck or photo exists in thousands of accounts. Storing it once is the difference between a viable and an unviable business.
• Scale (100 M accounts, 10 M daily active). Metadata operations dominate request count; bytes dominate cost.

Back-of-envelope estimates

• Stored bytes: ~1 EB raw, ~400 PB after dedupe. 100 M accounts × 10 GB average stored = 1 EB logical. Global chunk dedupe plus compression typically removes 55–65 % on a consumer corpus, so ~400 PB physical.
• New bytes per day: ~300 TB. 10 M daily active × 30 MB of genuinely new content each = 300 TB/day ≈ 3.5 GB/s average, and peak is 3–4× that in the evening. Sized against that, not the average.
• Chunk size and count: 4 MB · ~100 B of index per chunk. 4 MB balances two costs: small enough that a one-line edit re-uploads little, large enough that a 10 GB file is 2 500 chunks rather than 10 million. 400 PB ÷ 4 MB = ~10^11 chunks × 100 B of index = ~10 TB of chunk metadata.
• Metadata operations per second: ~200 k/s. 10 M daily active devices, each polling or holding a connection and committing a handful of changes: assume 20 metadata calls per device per active hour over an 8-hour spread → ~200 k/s at peak. This is the number the metadata store must serve, and it is why metadata and bytes are separate systems.
• Long-lived connections: ~3 M concurrent. Peak-concurrent devices ≈ 30 % of daily active = 3 M. At ~30 k sockets per notification server that is ~100 servers; each connection is idle almost all the time, so memory per connection is what matters, not CPU.
• Egress at peak: ~40 GB/s. Downloads outweigh uploads roughly 4:1 on a sync product (every change is uploaded once and downloaded once per other device). Peak upload ~10 GB/s → ~40 GB/s egress, which is why chunk downloads go through a CDN and never through the API tier.
• Version history cost: +15 %. Keeping 30 days of versions stores only the chunks that changed, not whole files. On a corpus where a few percent of bytes change each month, history costs ~10–20 % extra rather than 30×.

Components

• Sync client (desktop · mobile agent): Watches the folder with an OS file-system event API, chunks changed files, keeps a local SQLite database of (path, chunk hashes, server version), and reconciles against the server. Most of this design's cleverness lives here: the server is mostly a metadata store and a bucket.
• Load balancer: Terminates TLS and spreads API calls. Separate pools for metadata calls (small, chatty) and for notification connections (long-lived, idle), because they scale on different resources.
• Metadata service: The authority on what exists: namespaces, paths, file versions, the chunk list of each version, and permissions. Every mutation is a transaction that bumps a monotonic cursor for the namespace. Never touches file bytes.
• Metadata DB (sharded SQL, by namespace): Files, versions, chunk lists and the per-namespace change journal. Sharded by namespace id so one user's (or one shared folder's) changes are a single-shard transaction and the journal is naturally ordered.
• Chunk service: Answers "which of these hashes do you already have?" and hands out short-lived presigned URLs for the ones we do not. Records chunk → storage location and a reference count. Bytes flow past it, not through it.
• Chunk index (key-value, hash-sharded): content_hash → (location, size, refcount). Roughly 10^11 rows, uniformly distributed by hash, read on every upload and written once per new chunk. The single biggest table in the system.
• Block storage (S3-class object store): Encrypted, erasure-coded chunks addressed by content hash. Immutable: a chunk is written once and never edited, which is what makes dedupe, history and caching all safe.
• CDN: Serves chunk downloads from the edge against signed URLs. Chunks are immutable and content-addressed, so they are infinitely cacheable: the ideal CDN workload.
• Notification service (long-poll / WebSocket): Holds a connection per online device and tells it "namespace 42 has changed past cursor 9971". It sends no content, only a nudge, so a missed message costs one extra poll and nothing else.
• Change bus (Kafka, partitioned by namespace): Every committed metadata change is published here. The notification service, the sharing service and the background jobs all read it, so adding a consumer never touches the write path.
• Sharing service: Owns membership and roles for shared namespaces, and public links with their expiry and password. A shared folder is a namespace mounted into several users' trees rather than a copy per user.
• GC & lifecycle: Expires versions past the retention window, decrements chunk reference counts, and deletes chunks whose count has been zero for a grace period. Runs deliberately late: deleting a chunk that is still referenced is data loss.
• Search & previews: Consumes the change bus to build filename search and to render thumbnails and document previews asynchronously. Off the critical path; a missing preview is cosmetic.

User flows

1. A new file is saved on a laptop. The write path: chunk locally, ask what is missing, upload only that to storage directly, then commit metadata last so a file never half-exists.
   1. The client sees a file-system event, waits for the file to settle, and chunks it. Content-defined chunking with a rolling hash, target 4 MB. Each chunk is hashed (SHA-256) and the list of hashes is the file's identity. A short debounce avoids uploading the fifteen intermediate saves an editor makes.
   2. Client asks the chunk service which of those hashes the server already has. For most files most chunks are already stored: by this user, or by someone else entirely. This one call is where the 2× storage saving and most of the bandwidth saving are earned.
   3. Client uploads the missing chunks straight to block storage, several at a time. Bytes never pass through our API tier: that is what lets a handful of metadata servers support tens of GB/s. Each chunk upload is independently retryable, which is what makes the whole upload resumable.
   4. Client commits the new file version: path, size, ordered chunk list, and the version it believes it is replacing. Commit is the only moment the file becomes real. It is a single-shard transaction: insert the version, update the path to point at it, append to the namespace journal, increment chunk refcounts. Because the parent version is supplied, a concurrent change is detected here rather than silently lost.
   5. The commit is published to the change bus; devices are notified and the previews job picks it up. Published inside the same transaction via an outbox row, drained by a relay: publishing separately would allow a committed file that no device is ever told about. Independent consumers hang off the bus: the notification service, filename search and thumbnail rendering, none of which the commit waits for.
2. The change reaches the user's phone. The read path: a nudge, a delta by cursor, and chunk downloads from the edge. The server never pushes content.
   1. The notification service reads the change from the bus and finds the connected devices for that namespace. It holds a map of namespace → connections on the node that owns them. A shared namespace fans out to every member's devices from one event.
   2. Each online device is nudged over its held connection with nothing but the new cursor. Deliberately contentless. A nudge that is lost, duplicated or out of order is harmless, because the client's next call asks the authority what actually changed.
   3. The device asks for everything after the cursor it last processed. A cursor-based delta, not a folder listing. It is idempotent and resumable: a device that has been offline for a week makes the same call and gets the accumulated changes in pages.
   4. The device works out which chunks it is missing and fetches them from the CDN. It already holds "b1c3…" from the previous version of the file, so only the changed chunk is downloaded. Content-addressed chunks are immutable, so the CDN can cache them forever and a popular shared file is served almost entirely from the edge.
   5. The device assembles the file, writes it atomically, and records the new cursor. Write to a temporary file, verify the whole-file hash, then rename into place: an interrupted sync never leaves a half-written file in the user's folder. The cursor is only advanced after the file is on disk.
3. One row is appended to a 2 GB CSV. The case that decides whether the product is usable on a real connection. Nothing but the tail should move.
   1. The client re-chunks the file with a content-defined boundary, not a fixed offset. A rolling hash over a sliding window picks boundaries from content. Appending shifts nothing, so every existing chunk keeps its hash; even an insert in the middle only changes the chunks around it, instead of re-cutting the whole file as fixed-size chunking would.
   2. The probe returns exactly one missing chunk out of ~500. 499 hashes match what is already stored, so the answer is a short list. The probe payload itself is 500 × 32 B = 16 KB, which is why hashes are sent in batches rather than one call per chunk.
   3. Four megabytes are uploaded instead of two gigabytes. A 500× reduction, on the single most common heavy-file pattern there is: logs, exports, datasets and virtual machine images all grow by appending.
   4. Commit records a new version whose chunk list is 499 old hashes and one new one. Versions are cheap because they share chunks. This is also why 30 days of history costs a few percent rather than a multiple.
   5. Other devices download the one new chunk and splice it into their local copy. The receiving device reconstructs from its own chunk store plus the one download, so the saving is symmetric: it applies to every device, not only the one that made the edit.
4. Two laptops edit the same file offline. The failure case a sync product is judged on. The rule: never silently lose a byte a user typed.
   1. Both devices are offline and each saves a different edit to /notes/plan.md at version 7. Each queues its change locally with parent_version 7. Neither knows about the other, and no amount of cleverness on the client can change that.
   2. The first device reconnects and commits; the file becomes version 8. An ordinary commit: parent_version 7 matches the current version, so it is accepted.
   3. The second device commits with parent_version 7 and is rejected. Compare-and-set on the version is the whole concurrency control. Without it the second write would overwrite the first and the user would never know.
   4. The client resolves by creating a conflicted copy, and commits that as a new path. It first pulls version 8, then writes its own content to "plan (Ana's conflicted copy 2026-09-22).md". Both edits survive and the user decides. Automatic three-way merge is attempted only for formats where it is safe, and never for binary files.
   5. Both files sync to every device and the conflict is surfaced in the UI. A conflicted copy is a visible, annoying artefact, and that is correct: the alternative is an invisible, permanent loss. The count of conflicted copies is a product metric worth watching.
5. A folder is shared with a colleague. A shared folder is one namespace mounted in two trees, not a copy. That choice decides the whole permission model.
   1. The owner shares /Work/Q4 with another account as editor. The subtree is split into its own namespace if it is not one already: a one-off operation that rewrites the parent pointers of that subtree, not the files or chunks.
   2. The namespace is mounted into the invitee's tree. A mount row: (user 88, namespace 907, path "/Q4", role editor). The files themselves are untouched, so sharing a 500 GB folder is a metadata write measured in milliseconds rather than a copy measured in hours.
   3. The invitee's devices are nudged and pull the namespace from cursor zero. Same delta mechanism as every other change; a new mount is just a namespace the device has not read yet.
   4. Either member's edits now fan out to both members' devices from the one namespace journal. One journal per namespace, not per user: fan-out cost is the number of connected devices, and the ordering question ("whose version is newer") has a single answer.
   5. Access is revoked; the mount is removed, and any chunks left unreferenced are collected later. Deleting the mount removes the folder from that user's tree immediately: no bytes move. Presigned URLs already issued cannot be recalled, so they are minutes long and per-chunk, which bounds the residual exposure by design rather than by hope. If the namespace itself is deleted, the GC dereferences its chunks and deletes the ones nothing else holds.

Deep dives

Related

• Design Google Docs
• Design a Web Crawler
• Run this question for a friend: the interviewer kit