Design an Online Auction

• List an item for auction: Start price, an optional hidden reserve, a duration (1 to 10 days) or an explicit end time, and the close policy. The terms freeze at the first bid, because bidders relied on them.
• Bid with a maximum (proxy bid): The bidder states the most they will pay and the system bids for them one increment at a time, keeping the maximum secret. The visible price is the second-highest maximum plus one increment, capped at the highest.
• A definite answer to every bid: Leading, outbid by an existing maximum, or rejected (too low, auction ended). Each answer reflects every bid accepted before it, so two bidders are never both told they lead.
• Watch the price live: Anyone with the item open sees the price, bid count and time left update within a second without refreshing. The live view is a hint; the bid response is the truth.
• Close on time, exactly once: At end_at the auction stops taking bids and closes once, even if the close runs twice. With soft close, a bid in the final 2 minutes moves the end to 2 minutes after that bid.
• Charge the winner and settle: If the reserve is met, the winner is charged the final price on the saved payment method, with retries on failure and a second-chance offer to the runner-up as the fallback.
• Search and browse listings: Keyword, category, filters and sorts such as "ending soonest". A few seconds of staleness is fine, because nothing is decided from a search result.
• Notifications: Outbid, ending soon on watched items, won, lost and sold. At-least-once and deduplicated, and never on the bid path.
• Out of scope: Fixed-price Buy It Now checkout, shipping, ratings and feedback, seller fees and payouts beyond handing off to the payment system, photo hosting, and shill-bidding detection beyond keeping the complete bid log it needs.

• One order of bids per auction (linearisable per auction): Every bid on an auction is applied in a single sequence and its answer reflects all earlier ones. This drives the hardest tradeoff in the design: strict serialisation per auction, at exactly the moment contention peaks.
• Bid latency (p99 < 200 ms): In the final seconds a slow answer is a lost auction, and a bidder who waits retries, doubling the load on the hottest row. Measured separately for bids in the last minute.
• Live price freshness (p95 < 1 s): Watchers of a hot auction may skip intermediate prices, but never see an older price after a newer one.
• Close correctness (once, within ~1 s of end_at): A double close can create two orders or two charges; a late close delays the winner and the seller. Bids after end_at are refused whatever the timers do.
• Availability of bidding (99.99 %): An outage in the last minutes of an auction cannot be fixed afterwards without someone feeling cheated. Shed search and live updates first, never bid acceptance.
• Scale and skew (50 M live · 2.3 k bids/s avg): Modest in aggregate. The problem is skew: a hot auction takes 200 bids a second in its last 10 seconds while millions of others take none.
• Fair and auditable deadlines (server clock decides): Whether a bid beat the end is decided by the shard primary at commit, never by the client. Every bid, accepted or rejected, is kept with its server time for disputes.
• Search freshness (p95 < 5 s): Listings and prices reach search through the change stream. Eventual consistency is fine there because no bid is decided by the index.

• Bids per second: ~2.3 k/s avg · ~12 k/s peak. 50 M live auctions ÷ a 5-day average life = 10 M auctions ending a day; × ~20 bids each = 200 M bids/day ÷ 86 400 ≈ 2.3 k/s. Evening peaks run ~5× that: ~12 k/s. Spread over millions of auctions, this is a modest write load.
• Auctions closing per second: ~116/s avg · ~600/s peak. 10 M closes a day ÷ 86 400 ≈ 116/s, peaking ~5× in the evening: ~600/s. Each close is one guarded transaction and one payment, so the close path is small; the hard part is that 50 M pending deadlines must each fire once and on time.
• Hot auction, final seconds: ~200 bids/s on one row. A hot auction takes 5 000 bids in its final minute, about 40 % of them in the last 10 seconds: 2 000 ÷ 10 = 200 bids/s. A serialised commit with a synchronous replica takes ~2–3 ms, so one row tops out near 300–500 commits a second. Close enough that optimistic retries from many writers would collapse it.
• Fan-out for one hot auction: ~10 M msg/s naive · ~200 k/s coalesced. 50 k live watchers × 200 price changes a second = 10 M messages/s if every change is pushed. Coalescing to the latest state every 250 ms (4/s) gives 50 k × 4 = 200 k/s, spread over the push servers. Coalescing, not bigger servers, is what makes the hot auction affordable.
• Live connections: ~5 M · ~100 push servers. Assume 5 M item pages open at peak, each holding one Server-Sent Events connection. At ~50 k mostly idle connections per server that is 100 push servers, plus headroom for a reconnect storm during a deploy.
• Bid storage: ~30 GB/day · ~11 TB/yr. 200 M bids a day × ~150 B (auction id, bidder, maximum, price after, server time, outcome, client bid id) = 30 GB/day ≈ 11 TB/year. Kept for audit and disputes, partitioned by auction id so every question about one auction stays on one shard.
• Auction store shards: ~6 needed · run 8. Peak 12 k bid transactions/s, each an insert plus one conditional update. At a conservative ~2 k such transactions a second per shard primary that is 6; run 8 for headroom. Live auction rows are 50 M × ~1 KB = 50 GB, so the working set fits in memory: storage is not what shards this store, isolating write load is.
• Search index: ~100 GB · ~2.4 k changes/s. 50 M live listings × ~2 KB of indexed fields (title, category, attributes, price, bid count, end time) = 100 GB before replicas. Changes arrive at the bid rate plus ~116/s new listings, ~2.4 k/s; coalescing price updates per listing to one every few seconds cuts index writes well below that.

• Buyer / seller app: Web and mobile clients for listing, searching, watching and bidding. Generates a client bid id per bid attempt and reuses it on retry, keeps the highest auction version it has shown and ignores anything older, and draws the countdown from server time sent with each snapshot rather than the device clock.
• API gateway (auth · rate limits · routing): Authenticates, rate-limits bids per bidder per auction, and routes each bid by consistent hashing on auction id to the bid service instance that owns that auction. Serves item-page snapshots from the live state cache, so page views never reach the auction store.
• Bid service (owner per auction · proxy rules): Accepts or rejects bids. Each auction has one owning instance that queues its bids, applies the proxy rules in arrival order against the last committed state, and writes with a version check. Routing makes conflicts rare and allows group commit for hot auctions; the version check on the row is what makes a lost bid impossible.
• Auction store (sharded SQL · by auction_id): The source of truth for every auction and bid: one row per auction (state, price, leader, top two maximums, end_at, version) and an append-only bids table, all on the auction's shard. The bid, the deadline check, any soft-close extension and the outbox event commit in one transaction, with the shard primary's clock deciding the deadline. A synchronous replica means an acknowledged bid survives failover.
• Auction events (Kafka · key = auction_id): Every committed change (bid accepted, extended, opened, closed) from the outbox or CDC on the store's log, partitioned by auction id so each auction's events stay in version order. Feeds live fan-out, notifications, settlement and the search indexer, each at its own pace. Carries prices, never maximums or reserves.
• Live fan-out (coalesce to ≤ 4/s per auction): Turns auction events into live updates: the first change after a quiet spell goes out at once, later ones within 250 ms are folded into one carrying the latest state, and closes and extensions flush immediately. Writes the snapshot with a version guard and publishes it on the auction's channel. Coalescing here, before the fan-out, is what makes a hot auction cost 4 messages a second instead of 200.
• Live state + pub/sub (Redis · snapshot + channel): Holds the latest public snapshot of each active auction (price, bid count, end_at, state, version) and a pub/sub channel per auction. Item pages and reconnecting clients read the snapshot; push gateways subscribe to the channels. It may drop messages, because every message is the whole latest state and the next one replaces it.
• Push gateway (SSE · ~50 k conns/server): Holds the long-lived Server-Sent Events connections for open item pages, about 5 M at peak across 100 servers. Subscribes to an auction's channel only while it has a local watcher and writes each update once per connection. Read-only by design: bids never travel over it.
• Notifications (push · email, deduped): Sends outbid, ending-soon, won, lost and sold messages from the event stream. Deduplicates on (user, auction, kind, version) because the stream is at-least-once, and collapses a burst of outbid notices during a bidding war into the latest one.
• Listing service (create · edit · search): Creates and edits listings, validating the start price, reserve, duration and close policy, and pins the increment table and soft-close window onto the auction row. Freezes the terms once the first bid lands. Serves search and browse queries from the search index, never from the auction store.
• Close sweeper (leased per shard): One per shard, holding a lease: every second it finds auctions past end_at (and scheduled ones past start_at) through an index and applies a conditional close or open. The auction row is the timer, so soft-close extensions need no rescheduling, and a duplicate or late sweep matches zero rows.
• Settlement service (order per auction · retries): Turns auction.closed into an order keyed by auction id, so a redelivered event cannot create a second one, and charges the winner's saved payment method with an idempotency key per attempt. Timeouts retry with the same key; declines go back to the winner and retry on a schedule, ending in a second-chance offer to the runner-up.
• Search indexer (CDC consumer): Consumes the change stream, builds a search document per listing without the reserve or any maximum, and bulk-upserts with the auction version as an external version, so a late change can never overwrite a newer one. Coalesces price updates per listing to one write every few seconds.
• Search index (OpenSearch / Elasticsearch): Full-text search, facets and sorts (ending soonest, price, newly listed) over ~50 M live listings, about 100 GB plus replicas. Seconds behind the store by design: results show prices as hints, and nothing anyone bids on is read from here.
• Payment provider (Stripe / Adyen): Charges the winner's saved card. Slow and occasionally ambiguous: a timeout says nothing about whether money moved, which is why every charge carries an idempotency key and an unknown outcome is retried with the same key, never re-issued with a new one.

1. A bidder places a maximum bid: The core write. One transaction on the auction row decides the outcome, the new price and the deadline together; everything after it is derived from the committed event.
   1. The bidder submits a maximum bid with a client bid id.
   2. The gateway routes the bid to the bid service instance that owns this auction.
   3. The owner computes the outcome from the leader, the top two maximums and the new bid.
   4. One transaction inserts the bid and updates the auction row only if the version and the deadline still hold.
   5. The bidder gets a definite answer with the new price, version and end time.
   6. The committed change is published to the auction event stream, keyed by auction id.
   7. The previous leader is told they have been outbid.
2. A watcher sees the price move live: The main read. A snapshot first, then a stream of versioned latest-state updates: the live view may skip prices but never goes backwards, and none of it touches the auction row.
   1. The item page loads the current snapshot of the auction.
   2. The page opens an event stream for this auction on a push gateway.
   3. The push gateway subscribes to the auction's channel if this is its first local watcher.
   4. A bid commits; the fan-out service reads the event and decides whether to send now or coalesce.
   5. The fan-out service writes the new snapshot and publishes it on the auction's channel.
   6. Each subscribed gateway forwards the update to its local watchers, who drop anything older than what they show.
   7. After a dropped connection, the client reconnects and reloads the snapshot if it missed versions.
3. A hot auction's final ten seconds: The scale-breaking case: 200 bids a second on one row and 50 000 watchers. Serialise once and cheaply, and coalesce everything a human does not need.
   1. Bids arrive at 200 a second from thousands of bidders, all routed to the auction's owner.
   2. The owner rejects bids that are already too low without touching the database.
   3. The rest are applied in arrival order in memory and committed as one batch every few milliseconds.
   4. Under soft close, every batch in the final two minutes pushes the end time out.
   5. The fan-out service folds hundreds of changes a second into four updates a second.
   6. Every push gateway with watchers forwards each update to its share of the 50 000.
   7. New viewers arriving from a shared link load the snapshot from the cache, not the auction store.
4. The auction closes and the winner pays: The failure path. The sweeper may run twice or race a last bid, the event may arrive twice, and the payment may time out; the auction still closes once and the winner pays once.
   1. The shard's close sweeper finds auctions whose end time has passed.
   2. Each candidate is closed with a conditional update, together with an outbox event.
   3. A sweeper that lost its lease during a pause runs the same close, and matches zero rows.
   4. Watchers see the auction end at once.
   5. Settlement consumes the closed event and creates the order, keyed by auction id.
   6. Settlement charges the winner's saved payment method; the call times out.
   7. The retry returns the original successful charge, and the order is marked paid.
   8. The winner, the seller and the losing bidders are notified.
5. A listing goes live and becomes searchable: The asynchronous path. The seller writes once to the auction store; search learns about it, and about every later price, extension and close, from the change stream, seconds behind and without secret fields.
   1. The seller creates a listing with a start price, a reserve and a duration.
   2. The listing service writes the listing and its auction row on the auction's shard.
   3. A listing scheduled to start later is opened by the shard sweeper at its start time.
   4. The change is captured from the database log and published.
   5. The indexer builds the search document and upserts it, skipping anything older than what the index holds.
   6. A buyer searches a category sorted by ending soonest and sees the listing within seconds.

Ask: Two bids hit the same auction in the same millisecond. How do you apply both, in one order, without losing either?

Good answers name: Conditional write on the auction row (compare-and-set on a version), with bids routed to one owner per auction, Single writer per auction through a partitioned log (Kafka keyed by auction id), Pessimistic lock: SELECT … FOR UPDATE on the auction row, Distributed lock (Redis or ZooKeeper) around read-compute-write.

Our pick: Every auction has one row holding state, end_at, price, high_bidder, high_max, second_max and a version. A bid is one transaction on the auction's shard: insert the bid row, update the auction row WHERE id = ? AND version = ? AND state = 'open' AND end_at > now(), and insert an outbox event. Zero rows updated means someone else moved first or the auction closed, so re-read and recompute, which takes microseconds. The gateway routes bids by consistent hashing on auction id to one bid service instance, which keeps a queue per auction and the last committed state, so in steady state the version check never fails; during a burst it drains the queue in batches, computing each bid's outcome in arrival order and committing the batch as one transaction. Routing is a performance optimisation and the version check is the correctness guarantee, so a wrong route costs a retry, never a lost bid.

1. The bid service owner crashes with 30 bids queued. What happens to them?
   They were never committed, so they never happened, and none of those bidders was told they succeeded. Their clients time out and retry with the same client bid id, the gateway routes them to the new owner, and each bid is applied fresh against the current row. If one of them did commit just before the crash, the idempotency key returns its original answer instead of applying it twice. Nothing durable lives in the owner: it is a queue and a cache.
2. Ten times hotter: one auction takes 2 000 bids a second.
   Group commit carries it: at a 3 to 5 ms commit, batches of ten bids keep up, and every bid's outcome is still computed in order. Answers get a few milliseconds slower because a bid waits for its batch, so I would cap batches by time, around 5 ms, rather than by count. Before anything else I would check how many of those bids are already below the price plus one increment: the price never falls, so the owner can reject them from its cached state without a commit. Beyond that, a per-auction single writer on a log is the only way to go much faster than a commit loop.
3. Why not just run the transaction at SERIALIZABLE isolation and let the database sort it out?
   It would be correct, and under contention it does much the same thing as the version check: it detects the conflict and aborts one transaction. The cost is that every bid transaction pays for predicate tracking, and aborts surface as serialisation errors the application must retry anyway. An explicit version on one row makes the conflict cheap, visible and local. I would keep the default isolation level and make the one row that matters the conflict point.
4. A leading bidder raises their own maximum. Does that go through the same path?
   Yes, it is a bid like any other and takes the next version. It changes high_max but not the price, since nobody else's maximum moved, so watchers see nothing. The exception is a reserve: if the new maximum meets a reserve the price has not reached, the price jumps to the reserve. Serialising it matters because a raise racing a rival's bid decides who leads.
5. How do you know the bid path is healthy in the last minute of big auctions?
   Measure version-conflict retries per auction, owner queue depth and wait time, batch size, and bid latency split by time to end, because a p99 that looks fine overall can hide a bad p99 in the final ten seconds, which is the only window bidders care about. Alert on bids rejected as ended whose gateway receipt time was before end_at. That count is the unfairness the system itself causes.

Ask: A bidder says "up to $500". What price does everyone see, and how do you keep it right on every bid without replaying the whole history?

Good answers name: Top two maximums on the auction row, new state computed per bid, Recompute from all bids on every bid, Client-side auto-bidding: the bidder's agent places each increment, Sealed bids, price computed only at the close (second-price).

Our pick: The auction row stores high_bidder, high_max, second_max and the visible price, and each bid computes the next state from those and the bid alone. With inc(p) from the pinned price-band table: a bid m below price + inc(price) is rejected as too low; a bid above high_max makes the bidder the leader at price = min(m, high_max + inc(high_max)), with the old high_max becoming second_max; a bid at or below high_max leaves the leader alone, sets second_max = max(second_max, m) and price = min(high_max, m + inc(m)), and tells the bidder they were outbid by an existing maximum. A tie with high_max goes to the earlier bid, so the price becomes high_max. A leader raising their own maximum changes high_max only, unless it meets an unmet reserve, in which case the price rises to the reserve. Maximums never leave the auction store: events, the live cache and the index carry the price, the bid count and a masked leader name only.

1. Walk me through three bids: start $20, A bids up to $50, B up to $30, C up to $60.
   A is the only bidder, so A leads at the $20 start price with a secret maximum of $50. B's $30 is below A's maximum, so A stays ahead and the price becomes $31: B's maximum plus a $1 increment. C's $60 beats $50, so C leads at $51, A's maximum plus one increment. If the auction ends there, C pays $51, not $60, which is the whole point of proxy bidding.
2. A bidder retracts a bid. Now what?
   A retraction breaks the "price never goes down" property, so it is an admin-path operation, not a normal bid. It marks the bid retracted, recomputes the top two maximums from the remaining bid history for that auction (a few thousand rows at most, on one shard), and writes the new state with a version bump, which invalidates every owner's cached state through the version check. Watchers may see the price fall, which is correct and rare. Marketplaces limit retractions to narrow cases such as a typo, because they are also a shill-bidding tool.
3. How do you stop a seller seeing the maximums and shilling up to them?
   The maximums never leave the auction store: not in events, the live cache, the search index or the seller's dashboard. The API returns a bidder their own maximum and nothing else. A seller with a second account could still probe by bidding, which is why shill detection looks for accounts that bid on one seller's items and never win, linked devices and shared payment instruments. That is a trust-and-safety job reading the bid log, outside the bid path.
4. Should the increment table live in code or in data?
   In data, versioned, and pinned on the auction when it is listed, so a change during the auction cannot alter the rules for bids already placed. The proxy computation reads the pinned version. It also means a replay of the bid log months later reproduces exactly the same prices, which is what you need when a dispute arrives.
5. Two bidders submit the same maximum a millisecond apart. Who wins?
   Whichever the auction row sequenced first: the version order is the tie-break, and the earlier bid leads with the price at that maximum. Both get a definite answer, and the second is told they were outbid by an existing maximum. That is only fair if the ordering is explainable, so each bid row records its sequence and server commit time, and that record is what support shows in a dispute.

Ask: Half the bids on a hot auction land in the last ten seconds. Do you extend the auction, and what decides whether a bid at 20:00:00.002 counts?

Good answers name: Soft close: a bid in the final 2 minutes moves end_at to 2 minutes after that bid, Hard close at a fixed end time, with proxy bidding as the defence, Soft close with a cap on total extension, Random or hidden end within a window (candle auction).

Our pick: Default to soft close with a 2-minute window, configurable per category with an optional cap, and support hard close as a listing policy with a window of zero. The bid transaction does it atomically, SET end_at = GREATEST(end_at, now() + soft_close_window) WHERE … AND end_at > now(), so the deadline check and the extension use the same clock reading. That clock is the shard primary's, at commit: not the client's, which can be wrong or forged, and not the gateway's, because ordering bids by timestamps from many machines means waiting out the skew between them on every bid. A bid that reaches the primary after end_at is late, and the bidder is told so with the server time. The client countdown is drawn from end_at and a server-time offset sent with every snapshot, never the device clock, so nobody is misled by a countdown that runs fast.

1. A bidder in Sydney says the auction closed before their bid, and their screen showed one second left.
   Their bid reached the primary after end_at, which the bid row's server time shows, so it was late. Their countdown was right about end_at but could not include the 200 ms or so their request needed to cross the world. The honest fixes are to say on the page that bids must arrive before the end, and to keep both the gateway receipt time and the server time on the bid row so support can show exactly what happened. In a soft-close auction the argument mostly disappears, because that bid would have extended it.
2. The shard primary fails over 20 seconds before a hot auction ends. What happens?
   Bids on that shard fail for the failover window, perhaps 10 to 30 seconds, so bidders are locked out of the final moments through no fault of their own. The policy I would ship: before the shard's close sweeper resumes, extend every auction whose last 2 minutes overlapped the write outage by the outage plus the soft-close window, as one batch of conditional updates. The new primary's clock may differ from the old one by a few milliseconds, which falls inside the outage gap and reorders nothing that committed.
3. Why not timestamp bids at the gateway, closest to the user, to be fairer?
   Then the order of bids is decided by many clocks. Two gateways a few milliseconds apart in skew can order two bids differently from how they reach the row, so to be correct you would hold every bid until the maximum skew had passed, adding latency to all of them, and you would be trusting clocks that are harder to audit than one primary. The fairness gained is a fraction of the network latency difference anyway. One clock, at the point of serialisation, is explainable.
4. A soft-close bidding war runs for three hours. Is that a problem?
   For the system, no: it is just more bids at a human pace, each an ordinary write. For the product it can be, which is why the category config allows a cap. The things that must follow a moving end_at are the countdowns, which come from every snapshot; the "ending soonest" sort, which gets end_at through the change stream; and the close sweeper, which reads end_at at close time, so an old candidate list never closes an extended auction.
5. Does soft close change your capacity planning?
   It lowers the peak: with no reason to wait, the same bids arrive over minutes instead of seconds. I would still size the bid path for the hard-close shape, because hard close remains a supported policy and a soft-close auction still gets a burst every time a rival answers. Sizing for the worst supported case is cheaper than discovering it during a popular sale.

Ask: Fifty million auctions each have a deadline. How do you close every one on time, exactly once, and charge the winner exactly once?

Good answers name: The row is the timer: a leased sweeper per shard on an index of (state, end_at), with a conditional close, A delayed-message or timer service (timing wheel, delay queue), In-memory timers in the bid service owner, Close lazily on the next read or bid.

Our pick: Each shard has a close sweeper that holds a lease for that shard and every second runs SELECT id FROM auctions WHERE state = 'open' AND end_at <= now() ORDER BY end_at LIMIT 500. For each candidate, one transaction: UPDATE auctions SET state = 'closed', version = version + 1 WHERE id = ? AND state = 'open' AND end_at <= now(), plus an outbox event auction.closed with the winner, the final price and whether the reserve was met. The WHERE clause is the fence: a second sweeper, a stale one after a GC pause, or one racing a soft-close extension matches zero rows. Settlement consumes auction.closed and creates an order with the auction id as its primary key, so a redelivered event cannot create a second order, then charges the winner's saved payment method with an idempotency key of order id plus attempt number. A timeout retries with the same key; a decline notifies the winner and retries daily, and after three days the seller can make a second-chance offer to the runner-up at their own maximum (eBay offers sellers something similar).

1. A bid starts just before the deadline and is still committing when the sweeper finds the auction. Who wins?
   The bid, if it passed the deadline check. The sweeper's SELECT is only a candidate list; its UPDATE waits for the bid's row lock, then re-checks the WHERE clause against the committed row. Under soft close it sees the extended end_at and matches zero rows; under hard close it closes the auction with that bid included, which is correct. Nothing about the timer is trusted except as a hint to go and look.
2. The sweeper closes the auction, and the outbox relay dies before publishing. Does settlement ever happen?
   Yes, late. The outbox row committed with the close, so the relay publishes it when it recovers, and CDC would read the same committed change from the log. What I would watch is the age of the oldest unpublished outbox row and the count of auctions closed for more than a minute without an order. That second number catches every gap in the chain, whatever its cause.
3. The winner's payment is declined. Then what?
   A decline is a business outcome, not an error: the order moves to payment_failed, the winner is asked to update their card, and settlement retries daily for three days. After that the order is cancelled, the seller is told, and they may offer the item to the runner-up at the runner-up's own maximum, the price that bidder already agreed to. A timeout is different: the outcome is unknown, so we retry with the same idempotency key and never a new one, which is the payment system's rule applied here.
4. How would you know closes are happening on time?
   Close lag per shard: the time from end_at to the close commit, as a histogram, alerting when the p99 passes a few seconds. Open auctions whose end_at is more than a minute old, which should always be zero; anything else means a sweeper is stuck or lost its lease. And closed auctions with no order after a minute, which covers the outbox and settlement. Three numbers cover the whole chain.
5. Ten times the auctions: 500 M live, 6 000 closes a second at peak.
   The sweeper is per shard, so it scales with the shards ten times the bids would need anyway: at 60 shards it is 100 closes a second each, one indexed query and a hundred small transactions. The index on (state, end_at) grows with live auctions, but the sweeper only reads its head, which stays in memory. Settlement scales by consumer partitions. The real limit is the payment provider's rate limit at the evening peak, so charges go through a paced queue rather than straight off the close event.

Ask: 50 000 people are watching one auction that takes 200 bids a second. How do they all see the price, and what stops that melting the push tier?

Good answers name: SSE push gateways subscribed per auction, fed by a fan-out service that coalesces to the latest state every 250 ms, Poll a cached snapshot every 2 seconds, Push every bid event to every watcher, WebSockets carrying both bids and updates.

Our pick: The fan-out service consumes auction events from Kafka partitioned by auction id, so each auction's events reach one consumer in order. Per auction it sends the first change immediately and then at most one update per 250 ms carrying the latest state, except that closes and end-time extensions flush at once. Each update writes the snapshot to Redis with a version guard (only if newer) and publishes it on the auction's channel in the same script. Push gateways hold the SSE connections, subscribe to an auction's channel only while they have a local watcher, and forward each update; clients drop any version older than the one they show. A new or reconnecting watcher loads the snapshot first and then streams, so it never waits for the next bid to see a price.

1. A push gateway restarts and 50 000 clients reconnect at once.
   They reconnect with a randomised retry delay to other gateways, and each loads the snapshot from Redis or the CDN-cached snapshot endpoint, not the auction store. Deploys drain a gateway by closing its connections over a minute with jitter, never all at once. The bid path is not involved at all, which is the payoff of keeping the live view separate from the truth.
2. Can a watcher ever see the price go down?
   Only after a retraction. Otherwise every update carries the auction version, and the client ignores anything below the version it has shown, so an old message arriving late after a reconnect, or from a slower gateway, is discarded. The bidder's own bid response carries the version too, so their page jumps to the authoritative state immediately and late pushes cannot drag it back.
3. Why Redis pub/sub when it can drop messages?
   Because every message is the complete latest state, a dropped one is replaced by the next, and a client that suspects a gap re-reads the snapshot. Durable delivery would cost more and buy nothing here. Chat is the opposite case, where every message matters: the choice depends on whether the stream is a log or a value.
4. A viral auction has 2 M watchers.
   Then every push gateway has watchers for it, so each receives the auction's updates and fans them out locally. At 4 updates a second that is 8 M deliveries a second over 100 gateways, 80 k each, which is too hot, so I would coalesce that auction harder, to one update a second (20 k per gateway), serve its snapshot from the CDN, and add gateways. The publish side stays tiny either way, because coalescing happened before the fan-out.
5. How do you measure freshness?
   Stamp each update with the bid commit time, and measure the gap to delivery at the push gateway and in a sample of clients. Track p95 per auction tier, since a hot auction is up to 250 ms behind by design. Also count the stale versions clients drop: a rise means something upstream is reordering.

Ask: Buyers search 50 million listings by keyword, category and "ending soonest". Why not query the auction store, and how fresh does the index need to be?

Good answers name: CDC from the auction store into Kafka, and an indexer that upserts into OpenSearch, Dual write: the listing service writes the database and the index, Secondary indexes in the auction store, with scatter-gather across shards, Nightly rebuild from a snapshot.

Our pick: Listings and auction rows change in the store; Debezium-style CDC (or the same outbox) publishes those changes to Kafka keyed by auction id, and an indexer builds the search document (title, category, attributes, seller rating, current price, bid count, end_at, state) and bulk-upserts it into OpenSearch with the auction version as an external version, so a late or replayed change can never overwrite a newer one. The reserve and the maximums are never in the document. Price changes are coalesced per listing to at most one write every few seconds, which keeps index writes near the listing rate rather than the bid rate, while end_at changes in an auction's last minutes go through immediately. Closed auctions leave the live index on auction.closed. The target is p95 under 5 seconds from commit to searchable, measured by the indexer from the commit time in each change.

1. A results page shows $40 and the item page shows $55. Is that a bug?
   No, that is the design working: the results page is a few seconds behind, and the item page reads the live snapshot. It would be a bug if the item page or the bid response were behind. I would label list prices as the current bid with the bid count, and the click-through always reconciles. Nobody bids from a search result.
2. Ending soonest with soft close: an auction extends, but search still shows it ending in 10 seconds.
   end_at is just another field in the change stream, so the extension reaches the index within the lag target, and I would not coalesce end_at changes in an auction's last minutes. The results page also loads live snapshots for the rows on screen, so the countdown a buyer actually sees comes from the live state, not the index.
3. You change the title analyser. How do you reindex 50 M listings without downtime?
   Build a new index from a snapshot of the store while the CDC consumer writes changes to both the old and the new index from the snapshot's log position, then swap an alias once the new one has caught up. External versioning makes applying a change twice harmless. The bid path never notices, because nothing on it reads the index.
4. The indexer falls an hour behind. What breaks?
   Only search freshness: new listings are invisible and prices are stale, which hurts sellers but corrupts nothing. Kafka retains the changes, so the indexer catches up with larger bulk writes. The alert should fire long before an hour, on consumer lag measured as seconds since commit rather than as a message count.