Design Post Search

• Search posts by keywords: Words, hashtags and quoted phrases, all terms required. The same analyzer runs on posts and on queries, so "Earthquakes" finds "earthquake" and a hashtag stays one term.
• Sort by Recent or by Top: Recent is newest first; Top weighs likes, comments, recency and closeness to the searcher. Two orderings with different access patterns, and one index has to serve both.
• New posts are searchable within seconds: People search what is happening right now: a goal, an outage, an earthquake. An index that is minutes behind misses the reason the search was typed.
• Likes and comments move posts in Top: A post that takes off rises within about a minute, and results show current counts. Likes arrive at a million a second, which is the constraint.
• Results only include posts the searcher may see: Public posts, friends-only posts by the searcher’s friends, custom audiences that include them, and their own posts. Nothing by anyone they blocked or who blocked them.
• Deletes, edits and audience changes apply quickly: A deleted post stops appearing at once. An edit must stop matching the words it removed, or search reveals what was taken out.
• Page through results: Infinite scroll with a cursor. Posts arriving in the meantime must not duplicate or skip results on the next page.
• Out of scope: People, page and group search; typeahead (its own question); spelling correction and semantic retrieval; ads in results; how posts are stored beyond what search needs.

• Never show a post the viewer cannot see (0 leaks): The requirement behind the hardest tradeoff. A post missing for a few seconds is a shrug; a friends-only post shown to a stranger is a privacy incident. So results are personal, every one is re-checked against the source of truth, and caching is limited to what is the same for everyone.
• Freshness (searchable p99 < 10 s): Typically 1–2 s from commit. Rules out batch builds and periodic bulk refreshes: a post has to flow from the write path into an in-memory segment.
• Query latency (p99 < 500 ms): About 150 ms for the index fan-out, the rest for viewer context, counts and hydration. Latency follows the slowest partition, so partitions are replicated and requests hedged.
• Scale (50 k searches/s · 1 M likes/s): Peak rates, against 15 k posts/s. Likes outnumber post writes by about 65 to 1, so the engagement path, not the post path, is the write-heavy one.
• Corpus (~180 B posts/yr, kept for life): Tens of billions in any recent quarter, hundreds of billions overall. Most queries only need the last week, which is what makes tiering by time pay.
• Engagement in ranking (within ~1 min): Top reflects a surge in likes within a minute. Counts shown on results are read live at query time, so they are seconds fresh even where the index is not.
• Deletion (hidden at once · purged in days): Hidden from the moment the delete commits, out of the index within seconds, physically gone from every segment and snapshot within the purge deadline.
• Availability over completeness (99.95 %): Partial results beat an error: if a partition is slow, return what the others found and say so. This never extends to visibility, which is not negotiable.

• Post writes: ~6 k/s avg · ~15 k/s peak. 500 M posts/day ÷ 86 400 ≈ 5.8 k/s; evening peaks run ~2.5× that, ~15 k/s. Spread over 12 hot partitions that is ~1.25 k appends a second each. Writing new posts into the index is not the hard part.
• Like events: ~290 k/s avg · ~1 M/s peak. 1 B daily users × ~25 likes each = 25 B/day ÷ 86 400 ≈ 290 k/s, peaking near 1 M/s during live events. That is ~65× the peak post rate. This is the stream that must never rewrite an index document.
• Like-bucket changes: ~15 k/s avg · ~50 k/s peak. With power-of-two buckets, a post with n likes changes bucket only ~log2(n) times: 20 times on the way to a million. Most posts get a handful of likes and most likes land on a few popular posts, so assume ~2.5 changes per post over its life: 500 M × 2.5 ÷ 86 400 ≈ 15 k/s, about 20× fewer writes than likes.
• Index growth: ~100 GB/day · ~36 TB/yr. ~25 distinct terms per post with positions, compressed, ≈ 150 B, plus ~50 B of per-post fields (post id, author, time, audience, buckets) ≈ 200 B/post. 500 M × 200 B = 100 GB/day, ~36 TB/yr per copy before replicas.
• Hot tier size: ~700 GB · 12 partitions. Last 7 days = 3.5 B posts × 200 B ≈ 700 GB per copy. At ~60 GB of index per server that is 12 partitions. Small enough to hold entirely in RAM, which is what makes seconds-fresh appends and fast posting walks possible.
• Hot tier servers: ~240. Every query touches every hot partition. At ~2.5 k queries/s per replica (32 cores, ~10 ms of CPU per query), 50 k/s needs 20 replicas × 12 partitions = ~240 servers. Data size decides the partition count; query rate decides the replica count, separately for each tier.
• Viewer context per query: ~2.4 KB typical · 40 KB max. ~300 friends × 8 B ids = 2.4 KB; at the 5 000-friend cap, 40 KB, sent to each of 12 or more partitions. A Bloom filter at 1 % false positives needs ~10 bits per id, ~6 KB for 5 000 friends, and is safe only because the final check is exact.
• Exact-count reads: ~5 M keys/s peak. 50 k searches/s × ~100 finalists each = 5 M counter reads/s, batched into one multi-get per counter shard. Fetching counts for every match instead (a common term matches millions of posts) is what makes ranking on exact likes impossible at query time.

• App / web: The search box and result list. Sends the query with a sort (Recent or Top) and an opaque cursor for the next page. The counts it shows come from hydration at query time, never from the index.
• API gateway: Authenticates the viewer and takes the viewer id from the session, never from the request, because every visibility decision downstream trusts it. Rate-limits searches per user and routes posts, likes and searches to their services.
• Post service: Creates, edits and deletes posts. Assigns a time-ordered post id (Snowflake-style, so the id carries the creation time), validates the audience, and commits the row. Search is not on this path, so a search outage cannot stop anyone posting.
• Post store (sharded by post_id): Source of truth for posts: text, author, audience, created_at, version, deleted flag, and a coarse like bucket. The index is derived from it and can always be rebuilt from it. The privacy check reads it for every result, so the index’s copy of visibility never has the final word.
• Change stream (Kafka · CDC, key = post_id): Committed post changes from CDC on the post store, keyed by post id so a post’s create, edits, bucket changes and delete stay in order. Indexers write analysed documents to one log per index partition, and every replica of that partition tails it: the log is the replication mechanism and the recovery path.
• Indexer (stateless consumers): Analyses text (tokenise, lowercase, hashtags and mentions as whole terms, positions for phrases) and builds the index document. Routes every event by the creation time in the post id and hash(post_id): a new post to the hot tier, an edit or delete to whichever tier owns the original. Idempotent on (post_id, version).
• Hot index tier (7 days in RAM · 12 × 20): The last 7 days, 12 partitions by hash(post_id), 20 replicas each. A single writer appends to a live in-memory segment while readers walk it lock-free up to a published watermark, so a post is searchable the moment it is appended, and postings are walked newest first. Like buckets, audience and deleted bits sit in per-document arrays that change in place.
• Archive tiers (weekly / monthly slices, SSD): Everything older than a week, in time slices (weekly for the last quarter, monthly before), each partitioned by hash(post_id). Posting lists on SSD, per-document arrays in RAM. Recent slices get more replicas because they are queried more, and each slice records its time range and per-term maximum engagement so queries can skip it.
• Segment builder (batch merges): Merges hot segments into the weekly slice once they age out of the hot window, and compacts weekly slices into monthly ones. Every merge writes only live documents and folds in edits, which is where deletion becomes physical. Writes slices to object storage for archive servers to load, and can rebuild any slice from a post store snapshot when the analyzer changes.
• Search service (plan · fan out · merge): Analyses the query with the indexer’s analyzer, loads the viewer context, and plans by sort: Recent walks tiers newest first and stops when the page is full; Top prunes slices by their engagement upper bounds. Merges partition results, fetches exact counts for finalists, re-ranks, and sends the page to the privacy check. Returns partial results with a flag rather than an error.
• Result cache (public slice · ~10 s): For hot queries only: (normalised query, sort, language) to the top ~500 public candidates with scores, kept about 10 s. It never holds friends-only posts or anything per viewer, so sharing it cannot leak; blocks and the friends branch are applied per request after the read.
• Social graph (friends · blocks, cached): Friend lists, blocks in both directions (who I blocked, who blocked me) and custom audience lists, in a cached graph store. Search reads the viewer’s context once per query, so an unfriend or a block applies to the very next search with nothing reindexed.
• Engagement counters (by post_id · in memory + log): Like and comment edges plus an exact counter per post, sharded by post id and held in memory with a durable log. The counter moves only when an edge is new, so a retried like counts once. A viral post’s counter is split into sub-counters summed on read; search reads ~5 M counts a second at peak for finalists.
• Like service (likes · comments): Takes likes and comments, writes the edge and increments the counter in the engagement store. Only when the new count crosses a power of two does it write the post’s like bucket to the post store, which turns ~1 M likes a second into ~50 k small row updates.
• Privacy check (canonical can-see rules): The one implementation of "can this viewer see this post", shared with feed and profile. Reads each result’s current row (deleted, audience, version, text) and checks it against the viewer context, blocks in both directions, and whether the current text still matches the query. The index may be stale in either direction; this check makes the dangerous direction harmless.

1. A post becomes searchable in seconds: The core write. The post commits to its own store first; search hears about it from the change stream and makes it visible in an in-memory segment without ever blocking the writer.
   1. The client publishes a post; the post service assigns a time-ordered id and commits the row.
   2. Change data capture publishes the committed row to the change stream, keyed by post id.
   3. An indexer analyses the text and builds the index document.
   4. The indexer writes the document to the log of the hot partition that owns it.
   5. Each replica appends it to the live in-memory segment and advances the searchable watermark.
   6. A Recent search a moment later meets it first, because postings are walked newest first.
   7. When the live segment fills, it is sealed into a compact read-only segment.
2. Search "earthquake", sorted by Recent: The core read. Visibility is applied inside the index walk using the viewer’s friend set, the walk stops as soon as a page is full, and every result is re-checked against the source of truth before it is shown.
   1. The client sends the query; the search service analyses it with the indexer’s analyzer.
   2. It loads the viewer context: friend ids, whom the viewer blocked, and who blocked the viewer.
   3. The search service fans the query out to all 12 hot partitions with the visibility filter.
   4. If the hot tier cannot fill the page, the search continues into archive slices, newest first.
   5. The privacy check hydrates the page from the post store and re-checks every result.
   6. Exact counts for the page come from the engagement counters, and the page returns with a cursor.
3. A goal in the final: likes and searches spike: The scale-breaking case. A million likes a second must reach ranking without rewriting index documents, and thirty thousand people a second search the same word while each must see only what they may see.
   1. Likes pour in at a million a second; one post takes 50 k a second.
   2. The like service writes the post’s bucket only when its count crosses a power of two.
   3. The change reaches the post’s partition, which overwrites one byte in its bucket array.
   4. Thirty thousand people a second search "goal"; the public candidates are computed once and cached.
   5. For each viewer, blocked authors are removed and the viewer’s friends-only matches are added.
   6. Finalists get exact counts, are re-ranked, and pass the privacy check.
4. The indexer falls behind, then a replica dies: Failures should cost freshness or completeness, never correctness. Index lag means a new post appears late or a deleted one lingers in the index; the final check makes the second harmless.
   1. A bad indexer deploy leaves the hot partitions 90 seconds behind the change stream.
   2. A post deleted 30 seconds ago is still in the index and matches a query.
   3. A hot replica of partition 7 crashes; the search service routes around it.
   4. A replacement copies sealed segments from a healthy peer and replays the partition log from their offset.
   5. If a whole partition is slow, the search returns what the others found, flagged as partial.
5. Delete, edit and block: The privacy path. Removals are hidden at once by the final check, applied to the index as tombstones within seconds, and physically purged only when segments are merged.
   1. An author deletes a post from last year.
   2. The indexer routes the delete to the archive slice that owns the post and sets its deleted bit.
   3. Another author edits a phone number out of a post; the old version is tombstoned and the new one indexed.
   4. Someone blocks a viewer; that viewer’s next search excludes all of the blocker’s posts.
   5. Later, the builder merges the segments and drops tombstoned documents for good.

Ask: A post was published two seconds ago. How is it already in the results?

Good answers name: Change stream into an append-only in-memory segment, sealed and merged later, Index synchronously in the post write path, Off-the-shelf near-real-time engine (Elasticsearch / OpenSearch), 1 s refresh, Rebuild the index in batches every few minutes.

Our pick: The post store is the source of truth, and CDC puts every committed change on a Kafka topic keyed by post id. Stateless indexers analyse the text and write documents to one log per index partition, and every replica of that partition tails it. Each replica appends to a live in-memory segment with a single writer thread; readers see documents up to a published count, so there is no refresh step and no locking. At a fixed size, roughly an hour or two of posts, the segment is sealed into a compact read-only layout and a new one opens; after 7 days sealed segments are merged into archive slices by a batch builder. Likes, audience and deleted bits live in per-document arrays beside the postings so they can change in place. Twitter’s Earlybird paper (ICDE 2012) describes this design for tweets: in memory, one writer with lock-free readers, postings traversed newest first.

1. An indexer crashes after writing half a batch. What happens?
   It restarts from its last committed Kafka offset and writes those documents again. Every write carries (post_id, version) and a partition ignores a version it already holds, so the replay is harmless. Nothing is lost, because the offset is committed only after the partition log has the documents.
2. How do you know the freshness target is being met?
   Measure it end to end, not per component. A prober publishes a synthetic only-me post routed to each partition every few seconds and searches for it until it appears; that gap is the freshness metric, per partition, with an alert above 10 s. Consumer lag per partition then explains where the time went.
3. Posts get ten times more frequent. What breaks first?
   Not the append path: 150 k posts a second is ~12 k appends a second per partition, and adding partitions only affects the hot tier, which repartitions itself as it ages. RAM breaks first: the 7-day window becomes 7 TB per copy. So shrink the hot window to two or three days and give the first archive slice more replicas. Query cost per partition stays roughly flat, because early termination does not care how long a posting list is.
4. Why not let the post service write to the index directly and skip Kafka?
   Because that is a dual write: the post commits and the index write times out, or the index has a post whose transaction rolled back, and nothing reconciles them. CDC from the commit log gives "the event exists if and only if the row committed" for free. Kafka also gives replay: a new replica, a new analyzer or a bug fix can read the stream again.
5. Events for one post arrive out of order: an edit before the create.
   Keying by post id keeps one post’s events in order within a partition, so this mostly happens during replays and re-keys. Versions handle it anyway: a partition applies an event only if its version is newer than what it holds, and a tombstone is never overwritten by an older insert. An edit for an unknown post is parked briefly and retried, because the create is almost certainly right behind it.

Ask: Hundreds of billions of posts. How do you split the index so "newest first" stays cheap?

Good answers name: Time tiers (hot week in RAM, weekly and monthly slices on SSD), each hash-partitioned by post id, One pool, hash-partitioned by post id, Partition by author, Term partitioning.

Our pick: Three kinds of tier by creation time: the hot tier holds the last 7 days in RAM (12 partitions, 20 replicas); weekly slices cover the rest of the last quarter on SSD with four replicas; monthly slices cover everything older with two. Within every tier, documents are partitioned by hash(post_id), so a viral hour or a celebrity author spreads over all partitions. The time bits of the post id route any event to its owning slice without a lookup. The search service keeps a tier map (time range, partitions, replicas, per-term engagement maxima) and plans each query against it: Recent walks tiers newest to oldest and stops when the page is full; Top fans out wider but skips slices whose best possible score cannot make the page. It is the idea behind daily indices for logs, applied to a corpus that is almost always read newest first.

1. How many partitions does a Recent query actually touch?
   For a common term, the 12 hot partitions and nothing else: each finds 25 visible matches in its first few thousand postings. A rare term walks back a slice at a time, the next slice requested only when the previous one comes back short, or the next two in parallel when latency matters more than load. Track fan-out per query: it tells you whether the hot window is the right length.
2. Why seven days for the hot tier?
   It is a cost curve, not a rule. The window should be long enough that most queries finish inside it and short enough to fit in RAM at 20 replicas: 700 GB per copy, 14 TB across replicas. If 30 % of queries spill into the first weekly slice, either lengthen the window or give that slice more replicas, and pick whichever is cheaper.
3. A celebrity with 100 M followers posts. Does any partition get hot?
   Not from the write: it is one document in one hot partition. The reads are a burst of searches for their name, and the matching posts are spread over all 12 partitions by hash(post_id), with the public result cache absorbing the repeats. Partitioning by author would have put the post and every one of those reads on one partition.
4. How do you change the number of partitions in a tier?
   The hot tier repartitions by aging: write new posts with the new count, and within 7 days every hot partition is on the new layout. Archive slices are rebuilt by the builder into the new count one slice at a time, served from the old layout until the new one is loaded and checked. No live data moves between servers.
5. Old slices are rarely queried. Why keep them on servers at all?
   Because people do search for old posts, and a slower query on SSD is acceptable where minutes from object storage are not. Old slices get two replicas on dense SSD servers, with posting lists memory-mapped and only the per-document arrays in RAM. If queries for a slice fall to almost nothing, it can be served from object storage behind a local cache, trading latency for cost.

Ask: Top needs like counts. Why not just update the count on the index document?

Good answers name: Coarse log2 bucket in the index, updated on crossings; exact counts for finalists from a counter store, Update the index document on every like, Fetch exact counts at query time for every match, Refresh counts into the index in periodic batches.

Our pick: The engagement store keeps the (user, post) like edges and an exact counter per post, sharded by post id, in memory with a durable log; the counter moves only when the edge is new, and a viral post’s counter is split into sub-counters summed on read. When a like moves a post’s count across a power of two, the like service writes bucket = floor(log2(likes + 1)) to the post row, CDC carries it to the owning partition, and the partition overwrites one byte in a per-document array; comments get their own bucket the same way. Each segment also keeps, per term and per block of 128 postings, the largest bucket it contains, raised on every increase and lowered only by a rebuild, so it is always a safe upper bound. Recent never needs any of this, because time order lets it stop early; Top cannot stop early, so it uses those upper bounds to skip blocks and slices that cannot reach the current top 100. The best ~100 across partitions get exact counts in one batched multi-get and are re-ranked.

1. The bucket can be off by 2×. Doesn’t that make Top wrong?
   It makes the first stage approximate, and the first stage only has to get the right posts into the top 100, not order them. A post just under a boundary competes as if it had half its likes, which matters only at the edge of the 100; widen the finalist set if offline evaluation shows those misses. The final order uses exact counts.
2. A post goes from 10 likes to 100 k in a minute. How fast does it rise in Top?
   Its bucket climbs from 3 to 16 within that minute, one step each time the count doubles, and each step reaches the index in a couple of seconds. So it sits near its correct rank within seconds of each doubling, with nothing waiting for a batch. The count shown on the result is exact, because it comes from the counter store at query time.
3. Someone likes and unlikes in a loop right at a boundary.
   Increases apply at once; decreases apply lazily, only once the count falls below three quarters of the boundary, so flapping produces no writes. The per-term maxima are never lowered by an unlike anyway. The like service also rate-limits toggles per user, which is an abuse problem before it is an indexing one.
4. The counter store loses a shard.
   The counts are derived: the like edges are the truth, and a counter is rebuilt by counting the edges for that shard’s posts. While it rebuilds, Top ranks those posts on buckets alone and the result shows an approximate count from the bucket. Ranking is a little worse for a few minutes; visibility is unaffected.
5. Why put the bucket on the post row instead of publishing it straight to the index?
   So it travels on the same per-post ordered stream as edits and deletes, and the post store snapshot used to rebuild a slice already contains it. A separate topic would need its own ordering against tombstones, so a late bucket update cannot resurrect a deleted post, and its own rebuild path. The cost is ~50 k small row updates a second at peak, which a sharded post store absorbs.

Ask: Every searcher sees a different subset of posts. How do you never show one they cannot see, without a per-viewer index?

Good answers name: Filter in the index with audience fields and the viewer’s friend set, then an exact check on every result, Retrieve by relevance, then filter through a privacy service, Per-viewer index, or audience lists copied into each document, Bloom filter of the friend set sent with the query.

Our pick: Every document carries author_id and an audience class: public, friends, custom or only_me. The search service reads the viewer context from the graph on each query: friend ids, plus blocked and blocked-by ids merged into one exclusion set. Partitions admit a candidate if its author is not excluded and it is the viewer’s own, public, friends-only by a friend, or custom by a friend, flagged because only the post knows its audience list. The friend set is sent exactly when it is under ~1 000 ids and as a Bloom filter above that, with the page margin covering the ~1 % the final check will drop. The privacy check then reads each result’s current row and runs the canonical can-see rules on current graph data; anything that fails is dropped and counted. Facebook’s Unicorn paper (VLDB 2013) describes the same division of labour for social-graph search: the in-memory index narrows candidates with graph terms, and the final privacy check runs after retrieval, in the front end.

1. I unfriended someone a second ago. Can I still find their friends-only posts?
   No. The friend set is read from the graph on every query, so the next search no longer counts them as a friend and the partitions reject their friends-only posts. Even if a stale cache served the old set, the privacy check uses the current graph and drops them. Nothing about the post or the index has to change.
2. A query where almost every match is invisible: "happy birthday" from a viewer with 40 friends.
   Most matches are friends-only posts by strangers, so the walk admits maybe one posting in fifty and scans more to fill the page. Each partition has a scan budget; when it runs out it returns what it has with a resume point, and the search service asks for more or returns a shorter page. For queries like this it is cheaper to flip the plan: take the viewer’s friends’ recent posts, a few thousand documents, and match the terms against those.
3. How would you know if the index ever let through something it should not?
   Every drop at the final check is a near miss: the index admitted something the source of truth rejected. Track the drop rate by reason (deleted, audience, block, no longer matches); it should be small and explained by index lag. Separately, sample served results and re-verify them offline with an independent implementation of the rules; a disagreement there is a real leak and pages someone.
4. What about friends-of-friends audiences?
   That set is the union of the viewer’s friends’ friend lists, often hundreds of thousands of ids, too big to send exactly. Either send a Bloom filter of it, about 1.2 MB for a million ids at 1 %, or skip the in-index check for that audience class and flag those candidates for the exact check, which asks whether viewer and author share a friend. Flagging is simpler and correct either way; the Bloom filter only pays off if the audience class is common.
5. Why not cache each viewer’s friend set on the partitions instead of sending it?
   You can, keyed by (viewer, graph version), so a user paging through results sends it once. The risk is staleness: an unfriend must bump the version, or a partition keeps admitting posts it should not. The final check still catches that, so it is a performance optimisation with a correctness backstop, which is the only kind worth adding here.

Ask: Segments are immutable. A post is deleted, or edited to remove a phone number. How fast does it stop matching, and how?

Good answers name: Exact check at hydration, deleted-bit tombstones within seconds, delta segments for edits, purge on merge, Rely on the hydration check alone, Rewrite the affected segment on each delete, Global exclusion list at the search service.

Our pick: The privacy check reads the post store for every result, so a deleted or narrowed post is hidden from the moment its row commits. The delete then flows through CDC to the owning partition, chosen by the creation time in the post id and by hash(post_id), which sets the document’s bit in an in-memory deleted-documents bitmap: seconds, and nothing per query. An edit tombstones the old version and writes the new one into a small in-memory delta segment in the same partition, which every query scans in full and merges by created_at; versions make replays and reordering harmless. Account deactivation puts the author on an exclusion list at the search service until per-document tombstones land. The builder’s merges write only live documents and fold delta segments in, and a slice holding a deletion older than the purge deadline is rewritten regardless of schedule. Blocks never touch the index: they are per viewer, so they are applied from the graph at query time.

1. How long until a deleted post is gone from disk?
   Hot segments are gone within 7 days, when they merge into the weekly slice without the deleted documents. Archive slices are compacted on a schedule, and any slice holding a deletion older than the purge deadline, say 30 days, is rebuilt regardless. Snapshots and old slice versions in object storage expire on the same deadline, which is the part people forget.
2. A user deactivates and their 40 000 posts must vanish from search.
   Put their author id on the exclusion list the search service applies to every query, which hides everything in seconds with one write. Then emit per-post tombstones in the background and remove the author from the list once they have all landed. Reactivation is the reverse, so the list is also the fast path for undo.
3. How many deleted documents can a slice carry before it hurts?
   Deleted documents are still in the posting lists, so they cost scan time. Track the deleted fraction per slice and compact when it passes ~10 %, sooner for slices that are queried a lot. Recent weeks have the most deletes, because people delete soon after posting, and are the most queried, so they are compacted most often.
4. An edit and a delete of the same post arrive out of order.
   Each event carries the post’s version and a delete is a terminal version. A partition applies an event only if it is newer than what it holds, and a tombstone is never overwritten by an older insert. Because the stream is keyed by post id, this mostly matters during replays, which is exactly when it has to be right.
5. Why are blocks applied at query time rather than as tombstones?
   A block is per viewer: the blocker’s posts are hidden from one person and visible to everyone else, so there is no document to tombstone. Read as an exclusion set from the graph on each query, it applies on the next search and costs one membership check per candidate. It is the same reason friendships are not in the index.

Ask: Thirty thousand people a second search "goal". Can you cache the results?

Good answers name: Cache only the public candidates per query; apply blocks and merge the viewer’s friends branch per request, Cache full result pages per (query, sort), Cache per (viewer, query), No result cache, more replicas.

Our pick: Key (normalised query, sort, language), value the top ~500 public candidates with their scores, kept ~10 s and refreshed by one request under a single-flight lock while the others read the stale copy. Only queries above a rate threshold are cached, which is what the trending detector is for. On each request the search service drops candidates whose authors are in the viewer’s exclusion set, runs the friends branch (the same query restricted to authors in the viewer’s friend set), merges the two by score, fetches exact counts for the finalists and sends the page through the privacy check. The cache can make public results 10 s old; it cannot make anything visible to the wrong person, because it holds only public posts and everything still passes the final check.

1. A public post in the cached list is switched to friends-only.
   The privacy check reads the current audience at hydration and drops it for anyone who is not a friend, so the cache’s staleness costs a result slot for at most 10 s. You could also evict entries containing the post when its audience narrows, but with a 10 s TTL the bookkeeping is not worth it.
2. The key expires and 30 000 requests miss at once.
   Single flight: the first miss takes a short lock and recomputes, and the rest read the stale value, which is kept a little past its TTL for exactly this. Without it, every expiry is a burst of identical 12-partition fan-outs at the worst possible moment.
3. Why is the friends branch cheap enough to run on every request?
   For one viewer it covers about 300 authors and their recent posts, a few thousand documents, and the partition checks the author before scoring, so it skips most postings. The expensive part is the public scan, millions of postings for "goal", and that is what the cache saves. With 5 000 friends the branch is bigger, and it is still per viewer and never shared.
4. What hit rate do you expect, and how would you see the effect?
   During a spike the top few hundred queries can be a third of traffic or more, so 30 % or better on the public list; on an ordinary day much less. Compare partition load and scanned postings for cached keys before and after, and watch the age of served cached results. If cache age approaches the freshness target, the TTL is too long for that query.
5. Could you cache at the CDN?
   Not the response: it is personal and authenticated, so no shared proxy may cache it. The shareable artefact is the public candidate list, an internal object that never leaves the data centre. What a client can cache is its own last results, for back-navigation.