SysDesignPrep.com
System design interview question

Design Spotify

Stream 100 M tracks to 600 M listeners with sound in 200 ms, offline downloads, regional licensing, and every play counted for royalties.

Last updated 2026-10-26. Difficulty: hard. Patterns: prefetching, licensing, offline, counting, playlists. Reported at Spotify, Apple, Amazon, Google, Meta, TikTok.

Walk through a strong candidate's answer, turn by turn.

The interviewer asks, the candidate answers and draws, and you press Next. Skip to any stage or key moment, and ask the AI Mentor anything along the way.

Or sit it in the style of Spotify · Apple · Amazon · Google, run as that company’s round is reported to go.

The short answer

How to design Spotify in a system design interview, in five steps:

  1. Scope it. Search the catalogue, play a track, free and Premium tiers, regional licensing, playlists synced across devices, offline downloads for Premium, count plays for royalties, personalised weekly playlist.
  2. Set the targets. Scale: 600 M MAU · 250 M DAU · 10 M concurrent streams; Time to first audio: p90 < 200 ms; Gapless playback: 0 ms between tracks; Licensing correctness: takedowns < 15 min.
  3. Do the numbers. Play starts at peak: ~48 k/s (~100 k/s with skips); Catalogue storage, every rung: ~1.6 PB; Peak egress: ~1.6 Tbps.
  4. Draw the design. Audio CDN, Audio origin, Search service, Offline licenses, API gateway, Playback service, Catalogue & rights, Rights holders, Playlist service.
  5. Go deep where it is hard.
    • Audio in 200 ms, with no gaps → Fixed ladder with the rung chosen per track at play time, next track prefetched, first seconds of likely tracks cached on device.
    • A hot head and a long tail → Two commercial CDNs with popularity tiers: head pre-warmed at every edge, middle of the tail in regional shields, deep tail from origin.
    • Licensing checks on every play → Check in the playback service against compiled in-memory rights, then sign a 15-minute URL.
    • Offline downloads that expire → Encrypted files, keys wrapped with a hardware-bound device key, 30-day license renewed online, cap of 5 devices.
    • Counting plays that pay royalties → Idempotent play ids into a durable log, stream validation for dashboards, daily batch recount from the raw log with reconciliation and fraud quarantine.
    • Playlists edited from many devices → Per-playlist op log with a version, stable item ids, fractional position keys, server-ordered ops and snapshots.

Functional requirements

  • Search the catalogue. Tracks, artists and albums, as you type and tolerant of typos, ranked by relevance and popularity, and showing only what is playable in the listener’s country.
  • Play a track. Sound within ~200 ms of pressing play or of the previous track ending, no silence between tracks of a live album, and no stalls mid-track.
  • Free and Premium tiers. Free plays ads between tracks and caps quality at 160 kbps; Premium has no ads, up to 320 kbps and downloads. The tier is checked on every play, not only at login.
  • Regional licensing. A track can be playable in Sweden and not in Japan, Premium-only for two weeks after release, or taken down tomorrow. Every play obeys the rules for that country and tier at that moment.
  • Playlists synced across devices. Ordered lists of up to 10,000 tracks edited from phone, desktop and web, including collaborative playlists that several people edit at once. No edit is ever lost.
  • Offline downloads for Premium. Download playlists and play them with no network. Downloads stop playing if the subscription lapses or the device stays offline for more than 30 days.
  • Count plays for royalties. A play counts once it passes 30 seconds. Counts per track, country and tier become monthly statements for labels and publishers, so they must be exact, deduplicated and defended against fraud.
  • Personalised weekly playlist. A fresh 30-track playlist for every user every Monday, generated in batch.
  • Out of scope. Encoding the catalogue (a simpler version of the pipeline in Design YouTube), podcasts and video, lyrics, social features, ad targeting and the ad auction, and billing.

Non-functional requirements

  • Scale (600 M MAU · 250 M DAU · 10 M concurrent streams). A catalogue of 100 M tracks. Sessions are long and continuous, so most plays are the next track in a queue rather than a fresh tap.
  • Time to first audio (p90 < 200 ms). Measured from the tap, or from the end of the previous track, to the first sample out of the speaker. A cold tap only meets it on an edge cache hit over warm connections; prefetching makes queued tracks near zero.
  • Gapless playback (0 ms between tracks). Live albums and DJ mixes must flow sample to sample, so the next file must be on the device and decoding before the current one ends.
  • Licensing correctness (takedowns < 15 min). Never start a play the rights do not allow in that country and tier. New plays stop within seconds of a change; already issued URLs expire within 15 minutes.
  • Royalty accuracy (exactly once, reconciled). The requirement that drives the hardest trade-off. Plays come from clients we do not control, arrive weeks late from offline phones, get retried and get faked, yet each month’s statement must count every real play once and reconcile with zero differences.
  • Availability (99.99 % playback). Playback depends only on the resolve call and the CDN. Downloads must play with no network at all for up to 30 days.
  • Playlist sync (other devices < 5 s). Concurrent edits from several devices and collaborators converge to the same order everywhere, and none is lost.

Back-of-envelope estimates

  • Play starts at peak: ~48 k/s (~100 k/s with skips). 10 M concurrent streams ÷ 210 s per track ≈ 48 k/s if nobody skips; skips roughly double it to ~100 k/s. Each one is a rights check and a signed URL, so the playback service is sized for this, not for bytes.
  • Catalogue storage, every rung: ~1.6 PB. One 210 s track at 24, 96, 160 and 320 kbps is 0.63 + 2.5 + 4.2 + 8.4 ≈ 15.75 MB. 100 M tracks × 15.75 MB ≈ 1.6 PB, comparable to a video service’s whole catalogue, but spread over 400 M small files instead of a few hundred thousand large ones.
  • Peak egress: ~1.6 Tbps. 10 M streams × 160 kbps average = 1.6 Tbps, about 1 % of a large video service at peak. Two commercial CDNs carry this comfortably: bandwidth is not what makes this design hard.
  • Hot head at the edge: ~16 TB. Assume the 1 M most-played tracks (1 % of the catalogue) carry most plays. 1 M × 15.75 MB ≈ 16 TB at every rung, small enough to keep in every edge PoP. The other 99 M tracks cannot be everywhere, so a cold tap on one of them pays a trip to a shield or to origin.
  • Play events per second: ~87 k/s avg · ~150 k/s peak. 6 M average concurrent streams × 86 400 s ÷ 210 s ≈ 2.5 B plays/day, 10 per daily user. Three events per play (start, 30-second mark, end) = 7.5 B/day ≈ 87 k/s, ~150 k/s at peak. This is the stream the money is computed from.
  • Raw play log per year: ~1.4 PB. 7.5 B events × ~500 B = 3.75 TB/day ≈ 1.4 PB/year. Kept immutable for years in cheap object storage, because rights holders audit statements and every count must be recomputable from the raw events.
  • Rights data for every play check: ~10 GB. Three territory bitmaps per track (ad-supported, subscription, offline) over ~250 territories is 750 bits; with the next scheduled change that is ~100 B per track, so 100 M tracks ≈ 10 GB. It fits in memory on every playback instance, which makes the check a bit lookup rather than a database call.
  • Playlist storage: ~10 TB. Assume 5 B playlists × 50 items × ~40 B (item id, track id, position key, added by, added at) ≈ 10 TB, plus op logs. Small for a sharded store: the hard part is concurrent edits, not size.

Components

  • Audio CDN (two CDNs · head at every edge): Serves encrypted, immutable audio files by signed URL. Popularity-tiered: the ~1 M most-played tracks (~16 TB at every rung) are pre-warmed in every edge, the middle of the tail sits in regional shields, and the deep tail is fetched from origin on a miss. It checks only the URL’s signature and expiry and knows nothing about licensing; its cache key ignores the signature so all listeners share one copy.
  • Audio origin (object storage · 4 rungs): Every track at 24, 96, 160 and 320 kbps, ~1.6 PB, as immutable files named by content hash and encrypted with a per-file key. Only CDN shields read from it. Because names never change, files are cached for a year and nothing is ever invalidated.
  • Search service (inverted index · market filter): An inverted index over ~150 M tracks, artists and albums, fed from the catalogue by change data capture. Each document carries its territory bitmap, so the listener’s market is a filter applied before ranking, and ranking combines text match with popularity in that market and a personal boost. Serves typeahead at ~75 k queries/s at peak from replicas in each region.
  • Offline licenses (device keys · 30-day expiry): Registers devices (at most 5 per account) and issues offline licenses: each downloaded file’s key wrapped with a key that lives in the device’s secure hardware, valid for 30 days. Renewal re-checks tier and rights and returns tracks to delete. It never serves bytes.
  • Listener app (phone · desktop · web): Plays audio, keeps the queue, prefetches the first 30 seconds of the next track and decodes it early for gapless transitions, and caches the first seconds of tracks the listener is likely to play. Picks a quality rung per track at play time, stores encrypted downloads, and keeps play events on disk until the server acknowledges them.
  • API gateway (auth · routing · push): Terminates TLS for every API call, validates the session token (which carries user id, tier and home country) and routes to services. Holds a persistent connection per active device for small push messages such as "playlist changed". Audio bytes never pass through it.
  • Playback service (entitlement · signed URLs): Turns "play track X" into "fetch file F from these CDN hosts". Checks rights for the listener’s country and tier against an in-memory copy of the compiled rights (~10 GB), picks the file for the rung, and signs URLs that expire in 15 minutes. Stateless and scaled for ~100 k resolves/s; for Free listeners it also schedules ad breaks, which are just more audio files.
  • Catalogue & rights (deals by territory · tier · window): Source of truth for tracks, their files and the deals that govern them: per recording, the territories, commercial models (ad-supported, subscription, download) and start and end dates delivered by rights holders. A compiler turns deals into per-track bitmaps plus the next scheduled change and streams them to every playback instance within seconds.
  • Rights holders (labels · publishers): Deliver releases and deal terms as structured feeds (DDEX-style) and receive monthly usage statements and payments. They audit statements, so every number must be recomputable from raw data.
  • Playlist service (per-playlist op ordering): Accepts edits as operations (insert after item X, move, remove) with the version the client saw, orders them per playlist with a conditional append, assigns fractional position keys, and pushes the new version to other devices, which pull the ops they are missing. The weekly personalised playlist is written through the same API.
  • Playlist store (op log + snapshots): Per playlist: a header (owner, collaborators, version), items with stable ids and position keys, an append-only op log keyed by (playlist id, version), and a snapshot every 100 ops. Needs conditional writes on the version, so Spanner, or Cassandra with lightweight transactions. ~10 TB of items.
  • Event log (Kafka / Pub/Sub · key = play id): Durable log of play, ad and app events from every client, ~150 k/s at peak, acknowledged only once persisted. Keyed by play id, so a hit single spreads across partitions. Each hour is landed as immutable Parquet with control totals, the raw evidence every count is recomputed from.
  • Play validation (dedupe · 30 s rule · fraud): Dedupes events by play id, applies the 30-second rule, verifies that each online play carries a play token the playback service signed, and scores plays for fraud. Labels every play as counted, short, duplicate, fraud-held or invalid, and feeds live dashboards. The only source of plays for royalties and recommendations.
  • Royalty run (daily batch · reconciled): Each day recomputes exact qualified plays per (track, country, tier) from the raw log, deduplicated by play id over all history and attributed to the day each play happened. Refuses to publish unless every raw play id lands in exactly one bucket. Monthly, splits each market’s royalty pool by play share into statements; closed months are never edited, and late plays become adjustments.
  • Weekly recs (weekly batch): Reads validated listening history, finds candidates from listeners with similar taste and from track embeddings, filters out tracks unavailable in the user’s market or already heard, and writes each user’s 30-track weekly playlist through the playlist service. The output is just another playlist, so sync, downloads and playback need nothing new.

User flows

  1. Search, tap, hear it in 200 ms. The cold path is a search, one resolve call that checks rights and signs a URL, and one range request to the nearest edge. The warm path, which most plays take, has the next track on the device before the current one ends.
    1. The listener types a query; search returns tracks, artists and albums playable in their market. Typeahead sends a request per keystroke after a short debounce, ~75 k queries/s at peak. The market comes from the session token and filters on each document’s territory bitmap before ranking, so a track that cannot play never appears. Popularity in that market is part of the score, which is why typing "hello" shows the hit before 400 covers.
    2. The listener taps a track and the app asks the playback service to resolve it. The connection is already warm (HTTP/2 or QUIC), so this costs one round trip, ~40 ms on a phone. The app asks for a rung from tier, setting and network: 160 kbps on Free, 320 on Premium over Wi-Fi, 96 on a weak cellular link. It keeps that rung for the whole track.
    3. Playback checks the rights for this track, country and tier, and signs a short-lived URL. The check is a bit test in an in-memory copy of the compiled rights, kept fresh from the catalogue’s change feed, so it costs microseconds, not a database round trip. If this release is not licensed here but another release of the same recording is, playback swaps to it (relinking). The URL is an HMAC over the file path and an expiry 15 minutes out, so the CDN can verify it alone.
    4. The app requests the first bytes of the file from the nearest edge. Only ~32 KB, about a second of 320 kbps audio, is needed to start decoding: ~25 ms on a 10 Mbps link. The edge verifies signature and expiry, then serves from cache, where popular tracks are pre-warmed. On a hit, start-up is ~40 ms resolve + ~35 ms to the edge + ~25 ms transfer + ~40 ms decoder and audio output, about 140 ms.
    5. On a miss, the edge fetches from its regional shield, and the shield from origin. A deep-tail track adds 50 to 150 ms, the one case that misses the 200 ms target. It is rare by definition, and the next listener in that region hits the shield. Files never change, so there is nothing to invalidate.
    6. Thirty seconds before the track ends, the app resolves the next track in the queue and fetches its first 30 seconds. Most plays are the next track of an album, playlist or radio queue, so this is the path that matters most. When the current track ends the next one is already decoding, and trimming the encoder delay and padding from resolve makes the join gapless. The rest of the file is fetched as soon as it starts (~3 s at 10 Mbps); prefetching only ~600 KB caps the waste when the listener stops or skips.
    7. The app reports the play: started, passed 30 seconds, ended. Events are batched every few seconds and kept on disk until acknowledged, so a crash or a tunnel loses nothing. Each carries the play token from resolve as its play id, which makes a retried batch harmless. Playback never waits on this call.
  2. Two people edit a collaborative playlist. Edits are operations on stable item ids, ordered once per playlist and synced as a log, so concurrent edits from phones, laptops and friends converge without losing anyone’s change.
    1. Alice’s phone inserts a track after a given item, based on version 41. The client sends an intention, not a new list: insert this track after item it_77. Naming a neighbour’s stable id rather than an index keeps the meaning intact while others edit. The base version tells the server what the client had seen.
    2. The playlist service appends the op with the next version and a position key between the neighbours. Position keys are strings that sort: between "a4" and "a5" it picks "a4V", so an insert or a move writes one row instead of renumbering up to 10,000. The append is conditional on the current version, which orders ops for one playlist even when two service instances race.
    3. Bob’s laptop moves an item to the top, also based on version 41; the service rebases it onto 42. The op names item ids, so it means the same thing after Alice’s insert and applies as version 43 without bothering Bob. The conflict rules are fixed: two moves of one item, the later version wins; an edit to a removed item is a no-op; equal keys tie-break on item id.
    4. Other devices and collaborators learn the new version and pull the ops they are missing. The push carries only the playlist id and version; each device asks for ops since its own version. A device too far behind gets the latest snapshot plus the short tail instead. Offline devices pull when they next start.
    5. Every 100 ops the service writes a snapshot of the ordered items. Snapshots bound how many ops anyone replays and let old ops be compacted. If keys have grown long from many inserts in one spot, the snapshot writer rebalances them in a single op that every device applies like any other.
    6. Each weekend, the recommendation job writes every user’s weekly playlist through the same API. It reads only validated plays, so bot traffic cannot train it. Each run replaces the items of a system-owned playlist as a new version: 600 M playlists over a weekend is ~3.5 k writes/s, throttled and staggered by time zone so each is ready on Monday morning local time. Devices sync, download and play it like any other playlist.
  3. A superstar album drops at midnight. A scheduled spike: millions press play on the same 13 tracks as midnight rolls across time zones. The bytes are easy if they are placed in advance; the rights flip, the control plane and one very hot track in the event pipeline are where it can break.
    1. Days ahead, the label delivers the album with a release window per territory, and search indexes it hidden. The deal says available from 00:00 local time on Friday in every territory, for both tiers. The compiler records a scheduled change on each track, and the search documents carry an "available from" time, so the album is in the index but filtered out until it opens.
    2. The encrypted files are pushed to every edge before the window opens. 13 tracks × 15.75 MB ≈ 200 MB at every rung, trivial to place in every PoP. It is safe because no URL can be signed before the window and the files are useless without keys, which only resolve hands out.
    3. At local midnight in each country, playback instances start allowing the tracks. Each instance already holds the next scheduled change per track, so it flips at the exact second instead of waiting for a feed. Midnight rolling from New Zealand to Hawaii spreads the release into waves over a day rather than one global spike.
    4. Millions resolve the same tracks and search for the artist within minutes. Playback is stateless with rights in memory, so it scales by adding instances; it is pre-scaled for the known time, because autoscaling reacts in minutes and the spike arrives in seconds. The artist query is cached per market, and new-release and artist popularity stand in for the album’s own popularity, which is still zero.
    5. Every listener streams the same few files from edge caches. 5 M concurrent listeners on one track at 160 kbps is 800 Gbps, half our normal peak, all from edge memory. The cache key ignores the per-user signature, and shields coalesce misses, so origin sees about one request per file per region.
    6. Play events for one track flood the event log, spread by play id. Keyed by track id, one partition would take millions of plays an hour and fall behind. Keyed by play id the load spreads evenly, and dedupe still works because it only compares events of the same play. Per-track totals are summed in two stages: partial counts per partition, then a merge.
  4. Download, listen on a plane, report later. Downloads are encrypted files plus a device-bound license that expires. Plays made offline are buffered with their own ids and counted when the phone returns, days later, exactly once.
    1. A Premium listener downloads a 200-track playlist; playback checks tier, offline rights and the device count. Offline is its own right: some deals allow streaming but not downloads in some territories. Tracks that fail come back with a reason, so the app greys them out instead of failing silently.
    2. The license service registers the device and returns the file keys wrapped for it, valid for 30 days. The device key pair is generated in the phone’s secure hardware (Android Keystore, Secure Enclave) and the private key never leaves it, so the wrapped keys are useless on any other device. A sixth device is refused until the user removes one; removing a device revokes its license at its next check-in.
    3. The app downloads the encrypted files from the CDN with ordinary signed URLs. 196 tracks × 4.2 MB ≈ 820 MB at 160 kbps, over Wi-Fi by default and resumable with range requests. They are the same files the CDN serves for streaming; it cannot tell a download from a stream and does not need to.
    4. In flight, the app plays from disk and records each play with an id it mints itself. With no network there is no play token, so the client creates a time-ordered UUID per play and queues the events on disk with the device time and a monotonic counter. A clock set backwards is caught by comparing with the latest server time the app has seen.
    5. Days later the phone reconnects and uploads the buffered events; a dropped connection makes it send one batch twice. The client deletes events only after a batch is acknowledged, so a retry resends the same play ids, which dedupe makes harmless. The upload also reports the device clock offset, so event times can be corrected.
    6. Validation dedupes the plays and assigns each to the day it was played, not the day it arrived. An offline play must be for a file in a license that device held at the time, which stands in for the play token as proof the play was possible. If that month has closed, the plays become an adjustment on the next statement, never an edit to a closed one.
    7. Before 30 days pass, the app renews its license, and renewal re-checks tier and rights. A user who dropped to Free has the license revoked and downloads stop. Taken-down tracks are listed for deletion. A phone that never comes back online stops playing downloads when the license expires: 30 days is the most anyone keeps music they no longer pay for.
  5. The daily royalty run reconciles to zero. The background path where money is decided. Every play id lands in exactly one bucket, the totals reconcile against the raw log, and only then do counts become statements.
    1. Ingest lands each hour of raw events in an immutable store with control totals. When an hour closes, the ingest tier writes the event count and a checksum per partition. The batch later proves it read every event by matching them. Raw events are never edited; a correction is a new record.
    2. Validation labels every play: counted, under 30 seconds, duplicate, held for suspected fraud, or invalid. Fraud signals combine: listening hours per account beyond what a person can do, one track on repeat, accounts created in bulk, many accounts per device fingerprint or data-centre IP range, no searching or skipping. An online play without a valid play token is invalid outright. Held plays are quarantined, not deleted, so a false positive can be released before the month closes.
    3. The daily batch recomputes qualified plays per track, country and tier from the raw log. It dedupes by play id across all history, not a stream window, so an offline upload weeks late is still caught if it repeats. Plays are attributed to their event date. Recomputing from raw data every day, instead of trusting the stream’s running totals, is what makes the number auditable.
    4. Reconciliation checks that every raw play id is in exactly one bucket before anything is published. Distinct play ids in the raw log must equal counted + short + duplicate + fraud-held + invalid, per day and per partition, and raw event counts must match the ingest control totals. A difference of one blocks the run and pages someone: a late statement is a nuisance, a wrong one is a dispute.
    5. At month end, each market’s royalty pool is split by play share into statements for rights holders. The pool is a share of that market’s revenue per tier, divided pro rata by each track’s share of counted plays, then among recording and publishing owners. This is why fraud matters: fake plays do not create money, they take it from every other artist in the pool.
    6. Late offline plays and fraud reversals go into the next statement as adjustments. A closed month is never edited, as in a ledger: a correction is a new line that names the period it belongs to. Rights holders can audit any line back to raw events, because the raw log (~1.4 PB a year) is kept for years.

Deep dives

Audio in 200 ms, with no gaps

Why not stream audio with adaptive bitrate like video, and how do you get sound out within 200 ms of a tap?

A 3.5-minute track at 160 kbps is 4.2 MB. On a 10 Mbps phone connection the whole file arrives in about 3.4 seconds, roughly 60 times faster than it plays, so a few seconds after starting the entire track is buffered. Video is the opposite: gigabytes per title, a buffer that only ever holds a minute, and a player that must keep adapting. The start-up budget is also tight: a resolve round trip (~40 ms), first byte from the edge (~35 ms), the first ~32 KB (~25 ms) and the decoder and audio output (~40 ms) leave little room for a miss to origin, which alone adds 100 ms or more.

The observation that shapes the answer is that listening is continuous. Most plays are the next track of an album, playlist or radio queue, which the app knows about minutes ahead. If the start of the next track is on the device when the current one ends, its start-up time is zero, and the 200 ms budget only has to be met for cold taps, where warm connections and the edge cache carry it.

  • Fixed ladder with the rung chosen per track at play time, next track prefetched, first seconds of likely tracks cached on device chosen
  • Segmented HLS or DASH with mid-track bitrate switching situational: live audio (radio, concerts), where the stream is produced in real time and cannot be fetched ahead
  • Fetch on tap, no prefetch rejected
  • Download whole queues ahead situational: explicit downloads, where the user asked for the bytes and a license covers them

The answer: Encode every track once at a fixed ladder (24, 96, 160, 320 kbps) and let the app pick a rung at play time from the tier cap (Free up to 160, Premium up to 320), the user’s setting and measured throughput, keeping it for the whole track; the next track can choose differently. About 30 seconds before the current track ends, the app resolves the next one, fetches its first 30 seconds (~600 KB at 160 kbps) and starts decoding, then trims the encoder delay and padding returned by resolve so the last sample of one track is followed directly by the first of the next. The rest of the file is fetched as soon as it starts. For cold taps, connections to the gateway and both CDNs stay warm, an album or playlist page speculatively resolves its first tracks when it opens, and the app keeps the first 10 seconds of the ~500 tracks the listener is most likely to play on disk (200 KB each, ~100 MB). Spotify’s engineers published a paper in 2010 describing how their early desktop client combined local caching, prefetching and peer-to-peer delivery to keep start-up fast; this design keeps the caching and prefetching and serves bytes from CDNs.

The listener is on a train and throughput drops to 50 kbps a minute into a 320 kbps track. What happens?

Usually nothing audible: on a decent link the file finished downloading seconds after it started, so it plays from memory. If only part arrived, the app plays what it has and, if it would run dry, fetches the remainder at a lower rung from the same time offset, a rare deliberate switch rather than continuous adaptation. The next track is resolved at 24 or 96 kbps from the measured throughput.

How many bytes does prefetching waste?

Assume a quarter of prefetches are never played because the listener stops or picks something else. At ~48 k play starts a second that is 12 k wasted prefetches of ~600 KB, about 7 GB/s or 58 Gbps, under 4 % of the 1.6 Tbps peak. Prefetching the first 30 seconds rather than the whole file is what keeps it that low, and data-saver mode on cellular cuts it further.

Why cache only the first 10 seconds of likely tracks, not whole files?

Storage. The first 10 seconds of 500 tracks is 100 MB; the whole tracks at 160 kbps are 2.1 GB, which many phones cannot spare and the user never asked for. Ten seconds covers resolve and the first range request even on a slow network, after which the track streams normally. Whole copies are what downloads are for, under a license.

How do you measure time to first audio, and what do you alert on?

The app measures from the tap, or the end of the previous track, to the first sample handed to the audio device, and tags it with where the bytes came from: device cache, prefetch, edge, shield or origin. Dashboards split p50 and p90 by that source, platform and country, because a blended number hides a cold-start regression behind millions of prefetched plays. Alert on cold-start p90 and on any non-zero gap between tracks.

Why name files by content hash?

So they are immutable: a re-encode or corrected master gets a new file id and the old one never changes. The CDN caches for a year and nothing is ever invalidated, and identical audio on two releases, say an album and a compilation, is stored and cached once. The catalogue maps each track to its current file ids per rung.

A hot head and a long tail

Netflix places its popular catalogue inside ISPs before anyone asks. Why not do that here, and what do you cache where?

The catalogue is 100 M tracks, 1.6 PB at every rung, and plays are steeply skewed: assume the top 1 % of tracks (1 M, ~16 TB) carries most plays while tens of millions of tracks are played a handful of times a month. Peak egress is only ~1.6 Tbps, about 1 % of a large video service, spread across 10 M small, long-lived streams.

So the cost argument that justifies building appliances inside ISPs does not apply: commercial CDNs carry 1.6 Tbps comfortably. What matters is latency on a cold tap, which depends on where that particular track sits. The head can be everywhere; the tail cannot, and caching the tail on demand at every edge would evict the head.

  • Two commercial CDNs with popularity tiers: head pre-warmed at every edge, middle of the tail in regional shields, deep tail from origin chosen
  • Own appliances inside ISPs, filled overnight rejected
  • Pull-through LRU at every edge, no tiers situational: inside regional shields for the middle of the tail, where it is exactly right
  • Peer-to-peer between clients rejected

The answer: Serve from two commercial CDNs behind one URL scheme, with every resolve naming a host on each so the app fails over without another call. A daily job ranks tracks by plays per region over the last 7 days and pre-warms the top ~1 M per region at every rung in the edge PoPs (~16 TB), plus any scheduled releases; everything else is pull-through, with regional shields holding the middle of the tail and origin the rest. Files are immutable and named by content hash, cached for a year, and the cache key is the path alone so per-user signatures do not fragment the cache. The app reports first-byte time per CDN, and playback lists the faster CDN first per country. Spotify’s desktop client relied on peer-to-peer in its early years and later dropped it in favour of servers and CDNs; the tiering here is my design.

A 1990s track trends after appearing in a TV show. What happens?

For the first hour it behaves like a tail track: the first listener in each region pays a shield or origin fetch, and shields coalesce the rush so origin sees about one request per file per region. Within minutes it is in edge caches by pull-through, and the next daily run pins it. An hourly trending detector over the play stream can pre-warm tracks whose plays jump by orders of magnitude.

Why not cache everything at every edge? Storage is cheap.

Because 1.6 PB in each of hundreds of PoPs is hundreds of petabytes of fast storage for tracks most PoPs never serve. CDNs effectively sell what you can keep warm, and tail objects evict head objects. Pinning the head and leaving the tail to shields buys nearly the same hit rate for a tiny fraction of the storage.

One CDN degrades in Brazil. How do listeners cope?

Each resolve already returned a host on both CDNs, so the app retries the other on a timeout or a slow first byte, with no control-plane call. Apps report first-byte times per CDN, and playback reorders the hosts it returns per country from those reports, moving most traffic within minutes.

How do you know the CDN refuses expired URLs?

The edge validates the HMAC and expiry on every request, including cache hits, so an expired URL gets a 403 even for a hot file. Sample CDN logs for successful responses on expired signatures as a canary, and rotate the signing key daily with an overlap window. Even if a CDN got it wrong, the file is encrypted and useless without the key from resolve.

Licensing checks on every play

A track is licensed in Sweden but not Japan, and an album is Premium-only for its first two weeks. Where do you enforce that, and why not in the CDN?

Music rights are granted per recording, per territory, per commercial model (ad-supported streaming, subscription streaming, downloads) and per time window, and they change daily: releases open at local midnight, deals expire, takedowns arrive. A play the rights do not allow is a breach of contract, and with ~100 k resolves a second at peak the check cannot be a database query.

The CDN is the wrong place: it would have to know the user, country, tier and the rights of 100 M tracks, and call back on cache hits. The insight is that "may this listener play this file now" can be decided once, at resolve, and expressed as a URL that expires, so the CDN only has to check a signature.

  • Check in the playback service against compiled in-memory rights, then sign a 15-minute URL chosen
  • Entitlement lookup at the CDN edge with edge functions rejected
  • Separate CDN paths or buckets per country rejected
  • Hide unavailable tracks in the app only situational: as the display layer on top of server-side enforcement, never instead of it

The answer: The catalogue and rights store holds deals as rights holders deliver them: recording, territories, commercial models, start and end. A compiler materialises for every track three territory bitmaps (ad-supported, subscription, offline) plus the next scheduled change, ~100 B per track and ~10 GB in total, and streams changes to every playback instance, which holds the whole table in memory. At resolve, country and tier come from the session token (the country is the account’s home market), the check is a bit test, and if this release is unavailable but another release of the same recording is, playback relinks to it, which Spotify’s public Web API documents as track relinking. Then it signs a URL over the file path with a 15-minute expiry. Takedowns stop new resolves within seconds and old URLs within 15 minutes, and since files are encrypted with keys only resolve hands out, a leaked URL plays nothing.

A takedown arrives for a track 50,000 people are listening to right now. What happens to them?

New resolves fail within seconds of the change reaching playback instances. Listeners mid-track usually finish it, since the file is already on their devices, and their next resolve for it fails. For a legally urgent removal, playback can also tell clients over the push channel to stop the track; for ordinary takedowns, finishing the current play is acceptable and agreed with rights holders.

How does search respect the same rights?

Search documents carry the same territory bitmaps and next scheduled change, fed from the catalogue by change data capture. The market is a filter applied before ranking, so unplayable tracks never appear. An album with a future window is indexed early and filtered out until its time, so it shows up in the same second it becomes playable.

A user moves from Sweden to Japan. Which catalogue do they get?

Whatever the business policy says, enforced from one place: the country in the session token. A common policy keeps the home market while travelling for a limited time and then asks the user to update their country, which needs a payment method or proof from the new one. Tokens are reissued hourly, so a change applies within an hour.

How do you know the compiled bitmaps are right?

Two independent checks. A shadow evaluator recomputes a sample of resolves directly from the raw deals and compares answers continuously, alerting on any disagreement. And the compiler refuses to publish a version whose diff is suspiciously large, such as a quarter of a market disappearing, until a person approves it, because the most damaging bug is a bad feed taking down millions of tracks.

Why a 15-minute expiry and not 24 hours or 30 seconds?

It bounds how long a takedown or downgrade can be dodged with an old URL, while being longer than any track, so a slow download does not fail halfway. Thirty seconds would force re-resolves mid-track on slow links; 24 hours would make a leaked URL useful for a day. The app re-resolves transparently when a URL has expired, for example after a long pause.

Offline downloads that expire

A Premium user downloads 2,000 songs, cancels the subscription and keeps the phone in airplane mode. How do you stop them keeping the music?

Offline is a contract with rights holders: downloads only for paying subscribers, on a limited number of devices, and only while the subscription lasts. But an offline device cannot be asked anything, and a file on a phone can be copied. The design has to make copies useless and make every download expire unless the device regularly proves it is still entitled.

Two separate mechanisms do that. Encryption with keys bound to the device makes the files useless anywhere else. A license with an expiry that only the server can extend makes them useless on the same device once the subscription ends, with a known worst case: the license period.

  • Encrypted files, keys wrapped with a hardware-bound device key, 30-day license renewed online, cap of 5 devices chosen
  • Platform DRM (Widevine, FairPlay) for downloads situational: where the platform offers hardware-backed DRM; the license and renewal rules stay the same
  • Online check before every offline play rejected
  • Plain files with an app-level flag rejected

The answer: On download, the playback service checks Premium, the offline bit for each track in the listener’s market, and the device count. The license service registers the device’s public key, whose private half is generated and kept in the phone’s secure hardware, and returns each file’s content key wrapped with it, in a license that expires 30 days after issue; the files are the same encrypted files the CDN serves to everyone. The app refuses to play downloads after expiry and detects clock tampering by comparing with the latest server time it has seen and a monotonic counter. Any moment online renews the license, and renewal re-checks tier and rights, listing tracks to delete; a downgrade revokes it. Caps of 5 devices and 10,000 tracks per device bound the exposure (10,000 tracks at 160 kbps is 42 GB). Spotify publishes similar rules for its own downloads: go online at least once every 30 days, with limits on devices and downloaded songs.

The phone’s clock is set back two months to keep the license valid. Does it work?

No. The license stores its issue time from the server and the app stores the latest server time it has seen; a wall clock earlier than that is tampering, and downloads stop until the next check-in. A monotonic counter of playback time since the last check-in gives a second bound that does not depend on the wall clock at all.

A user hits the 5-device limit while setting up a new phone. What do they see?

A list of their devices with last-seen dates and a button to remove one. Removing frees the slot at once and revokes that device’s license at its next check-in; if it never comes back, its license still expires within 30 days. Support can reset the list, but it is audited, because it is also how one account would be shared by many people.

How are offline plays counted when there is no play token?

The client mints a time-ordered UUID per play and buffers the events. On upload, validation checks that the device held a license for that file at that time, the offline equivalent of the play token, and dedupes by play id. Plays are attributed to the day they happened, using the clock offset measured at upload.

A track is removed from the catalogue while it sits in 3 million downloads.

Every renewal response lists files to delete, so devices drop it at their next check-in, mostly within a day. Devices that stay offline keep it until their license expires, at most 30 days, which is the bound agreed with rights holders. For urgent legal removals, a push asks online devices to check in immediately.

Counting plays that pay royalties

Rights holders are paid by play. How do you count 2.5 billion plays a day exactly once, including plays from phones that were offline for weeks, and make it hard to fake?

Plays are reported by clients we do not control, over networks that drop requests, so events arrive duplicated, weeks late from offline phones, out of order, and sometimes fabricated. Yet a play is money: each market’s royalty pool is split by share of plays, so a counting error moves money between artists, and statements are audited. At ~150 k events/s at peak this cannot be a database increment per play.

Two observations shape the answer. Every play has an identity, its play id, so duplicates are a set problem rather than a guess. And the number that pays people does not need to be real time: it can be recomputed daily from an immutable raw log, deduplicated over all history and checked for exactness before use, while a separate streaming path serves dashboards that may be approximate.

  • Idempotent play ids into a durable log, stream validation for dashboards, daily batch recount from the raw log with reconciliation and fraud quarantine chosen
  • Increment a counter per track in a database rejected
  • Count from CDN logs or issued URLs situational: as a cross-check: a play event with no matching resolve or license is suspect
  • Exactly-once stream processing as the system of record situational: artist dashboards and trending, where minutes matter and small errors do not

The answer: An online play’s id is the play token signed at resolve; an offline play gets a client-minted, time-ordered UUID. Clients batch start, 30-second and end events, keep them until acknowledged and retry freely. Ingest writes to a log keyed by play id and lands each hour as immutable Parquet with control totals. A streaming job dedupes within a day, applies the 30-second rule and feeds dashboards. The daily batch is the system of record: it reads the raw log, dedupes by play id across all history, assigns event dates corrected by the device clock offset, labels every play as counted, short, duplicate, fraud-held or invalid, and refuses to publish unless every distinct play id lands in exactly one bucket and event counts match the ingest totals. Closed months are immutable; late plays and fraud reversals become adjustments on the next statement. Spotify has written publicly about its event delivery pipeline, including moving it to Google Cloud Pub/Sub; the counting rules here are my design.

Ten thousand free accounts on emulators stream one track on loop. What stops them getting paid?

Several signals together: listening hours per account beyond what a person does, one track on repeat, accounts created in bulk, many accounts per device fingerprint or data-centre IP range, and no human behaviour such as searching or skipping. Play tokens tie every online play to a resolve we issued, so fabricated events without a valid token are invalid outright and each token counts once. Suspect plays are held, not counted, and released or confirmed before the month closes.

Why not deduplicate in the stream with exactly-once semantics and skip the batch?

Late uploads arrive weeks later, so stream dedupe state would have to cover weeks of play ids, billions of keys, and a bug would silently corrupt totals with nothing to rebuild from. The batch is a plain distinct over immutable data that is easy to reason about and can be rerun after a fix. The stream stays for freshness, where small errors are fine.

The batch finds 412 more play ids in the raw log than in its buckets. What now?

The run stops, nothing is published for that day, and someone is paged. Typical causes are a partition read twice or skipped, a new event type the classifier does not know, or a schema change. Fix it, rerun the day from the raw log, and publish only when the reconciliation is exact.

Why partition the event log by play id rather than by track?

Dedupe compares only events of the same play, so play id keeps a play’s events together and spreads load evenly. Keyed by track, a superstar release would put millions of plays an hour on one partition. Per-track totals are a downstream aggregation: partial counts per partition, then a merge.

How long do you keep raw events, and what about deletion requests?

As long as statements can be audited, typically years by contract: ~1.4 PB a year, compressed columnar, in a cold tier after a few months. Personal fields are stored apart from the play facts, so a deletion request removes the identity while the counted play, which is the rights holder’s data too, remains as an anonymous record.

Playlists edited from many devices

A collaborative playlist is edited by three people on five devices, some of them offline. Why not store the list and overwrite it on save?

A playlist is an ordered list of up to 10,000 items, edited from phones, desktops and the web, and collaborative ones by several people at once. Saving the whole list with last-writer-wins loses whoever saved first. Storing an integer position per item means one move near the top renumbers every row after it, and two devices inserting at "position 3" at once collide.

The insight is that edits are small intentions about specific items (add this after that, move this before that, remove this), and that a server is always available to order them, even if a device catches up a week later. So store operations, give each item a stable identity and a position that can always be split, and let one place per playlist decide the order of operations.

  • Per-playlist op log with a version, stable item ids, fractional position keys, server-ordered ops and snapshots chosen
  • Whole-list overwrite, last writer wins rejected
  • Whole-list write with an optimistic version check and client-side merge situational: small personal playlists edited on one device at a time
  • Sequence CRDT replicated between devices situational: peer-to-peer collaboration with no central server

The answer: Each item has a client-minted item id, a track id and a position key: a base-62 string such as "a4V" that sorts between its neighbours, so there is always room between any two. Clients send operations that name item ids, with a base version; the playlist service orders them per playlist with a conditional append on the version, assigns keys and returns the new version. Fixed rules settle conflicts: concurrent inserts both apply (ties broken by item id), two moves of one item resolve to the later version, edits to a removed item are no-ops, and only the owner can rename or delete the playlist. Other devices are pushed the new version and pull missing ops, falling back to a snapshot written every 100 ops. Storage is a table keyed by playlist id with ops clustered by version, in a store with conditional writes such as Spanner or Cassandra with lightweight transactions; at ~10 TB, size is not the problem.

A phone was offline for a week and made 20 edits. It reconnects. What happens?

It sends its ops with base version 41 while the server is at 97. Each op names item ids, so the server rebases and applies them in order: inserts land next to the items they named, moves apply unless a later move of the same item already happened, and edits to removed items are dropped. The phone then pulls everything since 41 and converges.

Two people insert between the same two items at the same moment. What do they see?

The service orders the two ops, so the second gets a key between the first one’s new key and the right-hand neighbour; if keys were ever equal, item ids break the tie. Both items appear, in the same order on every device. That is what users care about: nothing disappears and everyone sees one order.

Position keys grow when people keep inserting at the top. Is that a problem?

Repeated inserts before the first item make keys grow by about a character each in the worst case. When any key passes ~32 characters, the snapshot writer issues one rebalance op that reassigns evenly spaced keys, and devices apply it like any other op. It is rare and cheap for 10,000 items.

How does the personalised weekly playlist fit into this?

A weekly batch reads validated plays, finds candidates from listeners with similar taste and from track embeddings, removes tracks unavailable in the user’s market or already heard, ranks them and keeps 30. It writes them through the playlist service as a new version of a system-owned playlist. Nothing downstream is special: sync, downloads and playback treat it like any other playlist.

A playlist has 2 million followers. Is that a fan-out problem?

Not for writes: following stores a reference, not a copy, so an edit is one op on one playlist. Reads are hot, so the latest snapshot and recent ops for popular playlists are cached by version, and followers pull "since my version" when they open it instead of each receiving a push.

The theory behind it

  • CDNs and edge computing: How a content delivery network works, push vs pull, cache keys and TTLs, what to put at the edge, edge functions and KV, and the pitfalls of caching dynamic content.
  • Counting at scale: How to count likes, views, clicks and unique visitors at high volume: exact versus approximate counts, sharded counters, batching through streams, idempotent counting, HyperLogLog for uniques, windowed counts and reconciliation.
  • Offline-first apps and sync: Designing apps that work without a connection and sync later: local-first storage, change logs and sync cursors, conflict resolution with last-writer-wins, merges and CRDTs, operational transformation, and keeping clients and server consistent.

Related

Report an issue

Something wrong with this question? A number that does not add up, a step whose answer key looks off, a typo. It goes straight to the author with the page you are on.

Watch a strong candidate answer thisTurn by turn, with the design drawn as they talkWalk through →Mock interview