System design concepts you can play with
Each concept comes with a playground: change the numbers, send the traffic, break it, and see why it works the way it does. Then read the short explanation, and what to say about it in an interview.
Reliability and failure
What happens when calls fail, retry, time out or overlap, and how to keep it safe.
- Token bucket and other rate limitersSend bursts through a token bucket, a leaky bucket, a fixed window and a sliding window side by side, and see which requests pass, wait or get a 429, and why.
- Distributed rate limitingEnforce one tenant’s rate limit across several gateways with local limits, a central Redis counter and periodic sync, and compare overshoot, wrong rejections and Redis load.
- The circuit breaker patternMake the same calls to a failing service with and without a circuit breaker, and watch it trip, fail fast, probe half-open and close again.
- Health checksBreak one of five servers so it fails fast, and see least connections funnel most traffic to it, shallow health checks miss it, and deep checks or outlier detection take it out.
- Bulkhead patternSlow down one of three dependencies and watch it take every thread in a shared pool, then add bulkheads and short timeouts to keep the others working.
- Shuffle shardingLet one poison customer crash its workers on a shared fleet, in cells and with shuffle sharding, and see how many other customers go down, degrade or stay untouched.
- Fair queuingShare eight workers between three tenants while one floods the queue, and compare a FIFO queue, per-tenant caps and fair round-robin queuing on waits and finish times.
- Priority queuesRun password resets, notifications and a digest batch through one worker pool under FIFO, strict priority, weighted shares and aging, and see who waits and who starves during a surge.
- Exponential backoff and jitterSend the same burst of clients at a busy server with five retry strategies and watch synchronized retry waves, retry storms and how jitter spreads them out.
- Retry stormsCut a server’s capacity for five seconds and watch eager retries keep it down for good, then compare backoff, a retry budget and dropping requests whose clients have given up.
- Deadline propagationSend requests through a chain of services with stalling queries, and compare fixed timeouts, retries at every layer, retries at one layer and a propagated deadline.
- Idempotency keysCharge a card through an API while responses get lost and retries overlap, with and without an idempotency key, and watch the card statement.
- Double-entry ledgerRecord four money transfers as balance updates and as double-entry journal entries, add a retried request and a crash mid-transfer, and see which approach loses or duplicates money.
- Reservation holdsSell 500 seats to 3,000 buyers with no holds, long holds, short holds and short holds renewed while active, and compare failed payments, seats blocked by leavers and unsold seats.
- Delivery semanticsRun 200 messages through a consumer that crashes, committing before or after processing, and see which messages are lost, repeated or handled exactly once.
- Message orderingSend order status events through a shared queue, partitions by order id and versioned writes, add a retried failure, and see which orders end in the wrong state.
- Dead-letter queuesRun a consumer against a queue with poison messages, and compare retrying in place, requeueing forever and moving failures to a dead-letter queue on throughput and waits.
- Webhook deliverySend webhooks to an endpoint that goes down for 30 minutes, 12 hours or 4 days, and compare dropping, per-event backoff and a per-endpoint queue with a breaker on lost events, wasted calls and ordering.
- Transactional outboxSave orders and publish events four ways through crashes and a broker outage, and see which lose events, invent them, or only send them twice.
- Saga patternBook a trip across four services with a saga, then fail a step, crash the orchestrator or break a compensation, and see whether the services end up consistent.
- Event sourcing and CQRSKeep a bank account as an append-only event log: replay it to any moment, take snapshots to load faster, and watch a read model lag behind.
- Load sheddingHit a server with a spike past capacity and watch an unbounded queue serve nothing, then shed load with a bounded queue or deadlines and keep serving.
- Adaptive concurrency limitsSend 400 requests a second to a service whose database degrades for a minute, and compare no limit, a fixed in-flight limit and an AIMD adaptive limit on goodput, wasted work and recovery.
- Canary deploymentsRoll out a buggy release all at once, rolling, blue-green and as a canary, and compare how fast each catches the bug, how many requests fail and how long a healthy rollout takes.
- Feature flag rolloutsRoll a feature out to a percentage of 400 users by random choice, by a hash of the user id and by a hash of flag and user, and see flicker, lost access and flags that overlap.
- Graceful shutdownRoll a deploy across four servers by killing them, by SIGTERM alone, and by deregistering then draining, and count the requests each way cuts off or sends to a stopped server.
- Burn rate alertingRun three days of errors with an outage, a slow leak and a blip past fixed thresholds and multi-window burn-rate alerts, and see which page, when, and for what.
- Trace samplingPlot thousands of requests with failures and slow outliers, and compare which traces head sampling and tail sampling keep, at what storage and memory cost.
- Cache stampedeLet a hot cache key expire under heavy traffic and watch the database flood, then stop it with request coalescing, stale-while-revalidate or early refresh.
- Fencing tokensPause a lock holder past its lease and watch two clients both believe they hold the lock, then see how fencing tokens stop the stale write.
Distributed systems
How replicas agree, disagree and catch up, and how nodes coordinate without a centre.
- CAP theoremCut the network between two regions selling the last items, and compare a CP design that refuses orders with an AP design that oversells, plus PACELC latency.
- Raft leader election and log replicationWatch a five-server Raft cluster elect a leader, stop the leader and see a new term begin, send writes that commit on a majority, and break the quorum.
- Quorum reads and writesPick N, W and R for a replicated key, make a replica lag, slow down or fail, and see when R + W > N stops stale reads and what it costs.
- Hinted handoffPartition a five-node ring and watch a strict quorum reject a write while a sloppy quorum accepts it on stand-in nodes with hints, serves a stale read, then hands the data back.
- Read-your-writes consistencyWrite to a leader, read from lagging followers, and compare five ways of routing reads by how many come back stale or go backwards in time.
- Database failover, RPO and RTOCrash a primary database under async, semi-sync or sync replication and see how many acknowledged writes are lost, what each write costs, and how long failover takes.
- Multi-region databasesPlace data across the US, Europe and Asia five ways and compare read and write latency per region, stale reads, conflicts and what losing a whole region costs.
- Backups and recoveryHit a database with a region outage or an accidental delete at any hour, and compare how much data nightly backups, log archiving and replicas lose and how long each takes to recover.
- Vector clocksWrite a shopping cart on three replicas, sync them in any order, and watch vector clocks keep concurrent writes as siblings while last write wins drops them.
- Clock skew and logical clocksReplay a rename across three replicas with a slow clock, and see last-writer-wins keep the wrong value under wall clocks while Lamport and hybrid logical clocks keep the right one.
- CRDTsEdit a like count and a shopping cart on two replicas during a network partition, heal it, and see what last-writer-wins loses and how CRDTs merge without conflicts.
- Two-phase commitRun one transaction across three databases with two-phase commit, then make a vote fail or the coordinator crash, and see who waits in doubt holding locks.
- Gossip protocolSpread an update through a cluster by gossip, round by round, and see how fanout, lost messages, crashed nodes and push-pull change how fast every node hears.
- Heartbeats and failure detectionWatch a day of heartbeats from a healthy node through GC pauses and a flaky network, and pick a timeout between false alarms and slow detection.
- Snowflake IDsGenerate 64-bit Snowflake IDs on three machines, see the timestamp, machine and sequence bits, and watch what bursts and clocks going backwards do.
- Short code collisionsDraw random short codes into a grid of every possible code, watch collisions arrive far sooner than expected, and see why generate-check-retry still works at billions of links.
Data and storage
How data is split, indexed, cached and stored on disk.
- Choosing a shard keyShard a posts table by user, post id or time, add a celebrity, and see which keys leave one hot shard and which make every read ask every shard.
- Consistent hashing on a hash ringAdd and remove servers on a consistent hashing ring with 300 keys, change the virtual nodes, and compare how many keys move with hash mod N.
- Rendezvous hashingPlace 1,200 keys on servers with hash mod N, a consistent hashing ring and rendezvous hashing, remove or add a server, and compare keys moved and load balance.
- Composite index column orderRun five queries on a million-row table against ten indexes, and see how column order decides rows read, rows fetched and whether a sort is needed.
- Local vs global indexesLook up orders by customer on 4 to 64 shards through a local index that scatters every query and a global index that writes twice, and compare shard requests, slow queries and index lag.
- B-tree indexGrow a B+ tree one key at a time, watch nodes split and the tree get taller, and compare ascending keys with random ones such as UUIDs.
- UUID primary keysInsert 200,000 rows keyed by random UUID v4 or time-ordered UUID v7 and watch page splits, half-empty pages and buffer pool misses grow once the index outgrows memory.
- Skip listBuild a skip list from coin flips, search it level by level, and compare its comparisons and pointers with a plain linked list, the structure behind Redis sorted sets.
- Timing wheelAdd timers to an eight-slot hashed timing wheel, tick the hand forward, and see why scheduling is constant time, what long delays cost, and how it compares with a heap.
- Write-ahead log and fsyncCommit to a write-ahead log under four fsync policies, then cut the power: compare commit latency, throughput and how many acknowledged writes are lost.
- Replication vs erasure codingStore 10 PB with copies or Reed-Solomon codes, change disk failure and repair rates, and compare raw storage, nines of durability and repair traffic.
- Storage tieringStore a petabyte of images for three years under Standard, Infrequent Access, Glacier Instant Retrieval and Deep Archive lifecycles, and see when tiering saves money and when retrieval fees eat it.
- Inverted index and search rankingSearch sixteen reviews through an inverted index: see posting lists, AND and OR queries, TF-IDF ranking and why common words cost the most.
- Vector search (IVF)Cluster vectors with k-means, search the nearest lists for a query, and see how the number of lists and probes trades recall against the vectors compared.
- Trie and typeaheadType into a search box backed by a trie, watch the walk down one character at a time, and see why each node caches its top suggestions.
- Typeahead debounceType a query into a search box that requests suggestions on every keystroke, drops stale responses or debounces, and see stale flashes, requests per query and how fast final results appear.
- Ranking algorithmsOrder the same posts by net score, upvote ratio, Wilson score and Reddit’s hot formula, vote on them, and see why each formula exists and what it gets wrong.
- Cursor vs offset paginationPage through a feed by offset and by cursor while posts arrive and are deleted, and see the duplicates, skipped posts and the cost of deep pages.
- Optimistic vs pessimistic lockingSell 50 units to a rush of buyers with no locking, row locks, version checks or one atomic update, and compare oversold units, waits and retries.
- Isolation levelsRun two transactions at read uncommitted, read committed, repeatable read and serializable, and see which anomalies each level allows: dirty reads, non-repeatable reads, phantoms, lost updates and write skew.
- MVCC bloatRun six hours of 2,000 updates a second on a Postgres table with only short transactions, a two-hour report or a forgotten idle session, and watch dead versions and table size grow, with and without timeouts.
- DeadlocksRun transfers that lock rows in opposite orders, watch the wait-for graph close into a cycle and the database abort one, then lock in id order and see deadlocks disappear.
- Online schema migrationRename a column while four app servers roll through deploys: in one step, with expand and contract done in the wrong order, and in the right order, and count failures, stale reads and lost changes.
- LSM trees and compactionWrite 7,700 entries into an LSM tree and watch flushes and compaction, leveled or size-tiered, trade write, read and space amplification.
- Log compactionRun a topic of user profiles for 60 days under 7-day retention, infinite retention and compaction, and see what each keeps and what a new consumer can rebuild from it.
- Row vs column storageStore one table by rows and by columns, run analytic and transactional queries, and see which cells each layout reads and how many bytes it costs at scale.
- Compression trade-offsMove 1 GB of logs over links from 10 Mbps to 10 Gbps with five codecs and see when compression speeds a transfer up, when the CPU becomes the bottleneck, and what it saves in storage.
- Time-series rollupsRoll two hours of per-second latency up to coarser resolutions, keep averages or maxima, and see how an incident disappears in averages and what each resolution costs to store.
- Bloom filters and false positivesAdd words to a Bloom filter bit by bit, check words it never saw, and watch false positives appear as it fills, measured against the formula.
- HyperLogLogCount a million distinct visitors in 12 KB: watch HyperLogLog registers fill, compare the estimate with the truth, and merge sketches without double counting.
- Count-min sketchCount a million hashtag mentions in a few kilobytes with a count-min sketch, change its width and rows, and see whether it still finds the true top 10.
- Reservoir samplingKeep 20 items from streams of 200 to 5,000 by keeping the first, a fixed probability, coin-flip replacement and Algorithm R, and see which samples are fair and stay the right size.
- Merkle treesChange keys on one replica and watch two Merkle trees find exactly what differs, comparing a handful of hashes instead of every key.
- Content-defined chunkingEdit a file and compare fixed-size chunks with content-defined chunks cut by a rolling hash: see which chunks change and how many bytes a sync client must upload.
- Multipart uploadsUpload 100 MB to 5 GB over connections that drop, with a single PUT, a resumable upload and parallel multipart parts, and compare time to finish and bytes wasted.
- LRU, LFU and FIFO cache evictionPlay the same requests through an LRU, an LFU and a FIFO cache side by side, and see which keys each one evicts, and why their hit rates differ.
- Cache sizingDrag a cache from 1 MB to 100 GB over 100 million items with long-tail, typical and head-heavy popularity, and see the hit ratio an LRU and a perfect cache reach and the memory each target needs.
- Cache write policiesRun the same reads and writes through cache-aside, write-through, write-back and write-around, and compare latency, hit ratio, database load, staleness and data loss.
Scale and traffic
Spreading load, queueing it and serving it fast.
- Load balancing algorithms comparedRoute the same traffic with round robin, random, least connections and power of two choices, then add a slow server and watch the queues and p99 latency.
- Session managementRun one fleet through a scale-out, a restart, a session store outage and a stolen session with sticky sessions, Redis sessions and signed tokens, and see who gets logged out.
- Queueing and Little’s lawPush a server from half busy to nearly full and watch latency climb the hockey stick, then check Little’s law, L = λW, against the simulation.
- BatchingSend items one by one or in batches, with and without a linger, at light and heavy load, and see how batch size sets capacity and how lingering trades latency for fewer requests.
- N+1 queriesLoad a feed page of 10 to 200 posts with authors and comment counts one query at a time, in parallel, batched or as one JOIN, and compare query waterfalls, page time and database load.
- Tail latency and hedged requestsFan one request out to 100 backends and watch a 1% stall become the typical case, then hedge slow calls and see what it costs in load.
- Coordinated omissionLoad test a server that stalls twice with a closed-loop tester, a corrected one and an open-loop generator, and see how the closed loop reports a 1.4 ms p99 for a service users saw take 1.5 s.
- AutoscalingRun a day of traffic against an autoscaled fleet, change the target utilization, boot time and schedule, and see when capacity falls behind demand.
- Serverless cold startsSend quiet, steady and spiky traffic to a serverless function on demand, with a keep-warm ping or with provisioned concurrency, and compare cold starts, peak instances and monthly cost.
- Connection poolingPut app servers with connection pools in front of a database, send a traffic spike, and see why a bigger pool can turn the spike into an outage, and how PgBouncer helps.
- Kafka partitions and consumer groupsSplit a topic into partitions, size a consumer group, add a hot key or crash a consumer, and watch each partition’s lag build or drain.
- Consumer rebalancingRestart six Kafka consumers one by one and compare eager rebalancing, cooperative sticky assignment and static membership on paused partitions, peak lag and catch-up time, with and without state.
- Watermarks and late dataPlot events by when they happened against when they arrived, move the watermark, and see how windowed results trade latency for completeness, and how allowed lateness handles stragglers.
- Fan-out on write vs readSize a news feed built by push, pull or a hybrid, add a celebrity with millions of followers, and compare inserts, lookups and delivery time.
- Hot keysPoint a share of traffic at one key on a sharded store and compare caching, replication, key splitting and both together, for read-heavy and write-heavy hot keys.
- Skewed joinsJoin 100 million clicks with ads when one viral ad has 1 to 30% of them, and compare a hash join, more tasks, salted hot keys and a broadcast join on job time and stragglers.
- Polling, long polling and WebSocketsDeliver messages to 100,000 users by short polling, long polling, server-sent events or WebSockets, and compare delay, wasted requests, bandwidth and open connections.
- Online presenceFollow one phone through ten connection drops and a departure, and tune the heartbeat, grace period, goodbye message and audience to trade false offlines, delay and messages per second.
- WebSocket routingDeliver 20,000 chat messages a second to a million users on 10 to 200 gateways by broadcasting, a connection registry or pub/sub channels, and compare per-gateway load, wasted messages and lookups.
- CDN caching and hit ratioSend 10,000 requests a second through a CDN and see how the TTL, the number of edge locations, popularity and an origin shield change hit ratio and origin load.
- HTTP cachingReplay a week of visits with a deploy in the middle under no caching, a one-day max-age, ETag revalidation and fingerprinted immutable assets, and compare bytes, requests and broken pages.
- DNS failoverFail one of two regions and compare DNS failover with long and short TTLs, multiple addresses with client retries and anycast, with and without resolvers that ignore the TTL.
- Adaptive bitrate streamingStream a video over a steady line, a tunnel and mobile data with fixed, throughput-based and buffer-based players, and compare quality, stalls and bitrate switches.
- Parallel transcodingTranscode a two-hour film on 10 to 200 workers with chunks from the whole film down to 2 seconds, add slow machines and backup tasks, and compare time to publish and wasted work.
- A/B test peekingRun hundreds of simulated A/B tests, check them every day, and see how peeking turns a 5% false positive rate into 24%, and how a fixed horizon or a sequential boundary fixes it.
- Multi-armed banditsServe three headlines with an even A/B split, epsilon-greedy and Thompson sampling, and compare clicks earned, clicks lost while learning and how often each picks the real winner.
- Ad auctionsBid for an ad slot against three rivals ranked by bid times click-through rate, switch between first-price and second-price rules, and see your profit at every bid.
- GeohashTap a map to run a geohash nearby search: see the cell and its eight neighbours, what each precision reads and misses, and why close places can have different prefixes.
- QuadtreeIndex a city’s places as a quadtree and as a fixed grid, click to find the 10 nearest, and compare the cells opened and places checked downtown and in the suburbs.
- Shortest path routingPlan the same trip with Dijkstra, A* and bidirectional search on a city grid with a river and traffic, and compare how many intersections each explores to find the route.
- MatchmakingRun a one-versus-one queue with Elo ratings using a strict, a wide and an expanding rating window, and compare waits, abandoned players and how often the underdog wins.
- Dispatch matchingMatch riders to drivers greedily and in batches that minimise total pickup distance, then run an hour of a city at rising demand and compare trips, empty driving and waits.
- Surge pricingLet a concert out into one city cell and compare a fixed price, a multiplier recomputed every minute and a smoothed, capped one: queues, priced-out riders, price swings and drivers arriving too late.