CDN caching, TTLs and hit ratio: an interactive simulator
A CDN caches content in edge locations close to users, so most requests are answered nearby instead of by your origin servers. How well it works is measured by the hit ratio, the share of requests answered from cache, and it depends on three things: how long objects may be cached (the TTL), how concentrated requests are on popular objects, and how many separate edge caches the traffic is split across.
The playground sends 10,000 requests a second for 100,000 objects through a CDN. Change the TTL, the number of edge locations and how concentrated popularity is, add an origin shield, and watch the hit ratio and the load left for the origin.
10,000 requests a second for 100,000 objects, spread over 50 edge locations, each keeping an object for 1 min. 62% of requests are answered at the edge, against 91% if all the traffic went through one cache: each location sees only 1/50 of the requests, so less popular objects expire before they are asked for again. The origin sees 3,823 requests a second. The most popular objects are almost always cached; the object ranked 10,000th is a hit 9.0% of the time. In exchange, a change at the origin can take up to 1 min to reach every user unless the CDN is told to purge it.
Why the TTL matters most
An edge fetches an object on the first request, then answers every request for it until the TTL runs out. For an object requested r times a second, each cycle is one miss followed by about r × TTL hits, so the hit ratio is rT / (1 + rT). Popular objects are hits almost always at any TTL; the long tail of rarely requested objects needs a long TTL to be hit at all. Going from a 1 minute TTL to an hour takes 50 edge locations from 62% to 92% hits in the playground, and the origin from 3,823 requests a second to 815.
More edge locations, lower hit ratio
A CDN’s strength, being close to every user, splits the traffic: each location sees only its share of the requests, and caches fill independently. With one cache and a 1 minute TTL, 91% of requests are hits; spread over 50 locations, 62%. The object ranked 10,000th is a hit 83% of the time in the single cache and 9% at each of the 50 edges, because it is requested at each one too rarely to still be cached.
Origin shields and tiered caching
An origin shield is one extra cache layer between the edges and the origin. All the edges’ misses go to it, so it sees the combined demand for each object and caches it once for everyone. In the playground, with 50 edges and a 10 minute TTL, the edges still miss 20% of the time, but the origin load falls from 1,989 requests a second to 149. Cloudflare calls this tiered cache; CloudFront, Origin Shield; Fastly, shielding.
Long TTLs without stale content
- Version the URLs of assets that change:
app.3f9c1a.jscan be cached for a year, because a new version gets a new name. - Purge on change: tell the CDN to drop a URL, or every URL with a tag, when the content behind it changes, and keep the TTL long.
- Serve stale while revalidating: let edges keep serving a just-expired copy while they fetch a fresh one, so expiry costs no user a slow request.
- Split public from private: cache the shared page and fetch the per-user parts separately, instead of not caching the page at all.
One assumption to keep in mind: the playground’s caches never run out of space. Real edges evict the least recently used objects when they fill, which hurts the long tail further.
Cache headers in practice
# static assets with a content hash in the name: cache for a year, never revalidate
Cache-Control: public, max-age=31536000, immutable # /app.3f9c1a.js
# a product page: edges keep it 10 minutes, browsers 1, and serve stale while refreshing
Cache-Control: public, max-age=60, s-maxage=600, stale-while-revalidate=300
# per-user data: never stored by shared caches
Cache-Control: private, no-store
# after a change, purge instead of waiting out the TTL (Cloudflare API, by URL or tag)
POST /zones/:zone/purge_cache {"files": ["https://shop.example.com/p/8812"]}In a system design interview
Say what is cached where and for how long: “Images and video segments are immutable, with content hashes in their URLs, so the CDN caches them for a year. Product pages are cached at the edge for 10 minutes with stale-while-revalidate and purged by tag when a product changes. Behind the edges, an origin shield collapses misses, so the origin sees a few hundred requests a second rather than thousands.” Expect the follow-ups: personalised content, how a purge reaches every edge, and what happens to the origin when the CDN’s cache is cold.
Read the guide to CDNs and the edge or HTTP caching, see cache stampede for what happens when a hot object expires, or watch a strong candidate design YouTube.
Questions people ask
What is a CDN cache hit ratio?
The share of requests a CDN answers from its edge caches instead of forwarding to the origin. A high hit ratio means faster responses for users and less load and bandwidth at the origin. It depends mainly on cache TTLs, how concentrated requests are on popular content, and how traffic is split across edge locations.
How does TTL affect the cache hit ratio?
An edge caches an object for the TTL after fetching it, so for an object requested r times a second the hit ratio is about rT / (1 + rT). Popular objects are hits at almost any TTL, while rarely requested ones need long TTLs to be hit at all. Longer TTLs raise the hit ratio at the cost of serving stale content for longer.
Why do more edge locations lower the hit ratio?
Each location has its own cache and sees only its share of the traffic, so a moderately popular object is requested at each location less often and expires before the next request. Spreading the same traffic over 50 locations instead of one can cut the hit ratio substantially, especially for the long tail.
What is an origin shield?
An extra caching layer between a CDN’s edge locations and the origin. Edge misses go to the shield, which sees the combined demand for each object, caches it once, and forwards only its own misses to the origin. It greatly reduces origin load when there are many edges; providers call it Origin Shield, tiered cache or shielding.
How do you cache content for a long time without serving stale data?
Give changing assets versioned URLs with a content hash so they can be cached immutably, purge URLs or tags from the CDN when the underlying content changes, use stale-while-revalidate so expiry does not slow requests, and separate shared content from per-user content so the shared part can be cached.
What should not be cached on a CDN?
Responses that differ per user, such as account pages, carts or anything depending on session cookies, unless the cache key includes what makes them differ; responses with sensitive data, which should be marked private or no-store; and requests that change data. A mistake here can serve one user’s data to another.
What happens when a CDN cache is cold?
After a new deployment, a purge or a new edge coming online, caches are empty and nearly every request goes to the origin until popular objects are fetched again. Origins should be sized or protected for that burst, with an origin shield, request collapsing at the edge and warming the most popular content in advance.