Design Amazon
Run this for someone else. You hold the answers; they do not. Read the prompt, keep the clock, and use the probes below when an answer is thin. Do not show them this page.
Open with this
A 500 M-product marketplace: product pages at 100 k views/s, carts that never lose an item, and Prime Day checkout without overselling. Take a couple of minutes on requirements, then we will do some numbers, then the design. I will interrupt to keep us moving.
The clock
- 4 min: functional requirements and scope
- 4 min: non-functional requirements, with numbers
- 5 min: back-of-envelope estimates
- 16 min: high-level design and one or two flows
- 16 min: deep dives and the close
Move them on out loud when a section overruns. The commonest failure is spending twenty minutes on requirements and never reaching a deep dive, and preventing that is your job as much as theirs.
Requirements · 8 min
Listen for: a scoped set of capabilities, an explicit out-of-scope list, and numeric targets rather than adjectives. Prompt with “what are you not building?” if they never scope, and “what number would make that requirement real?” if they say “fast” or “highly available”.
Functional (9)
- Browse and search the catalogue: Hundreds of millions of products whose attributes differ completely by category, with faceted filters. Results come from a search index built from catalogue changes; ranking is out of scope.
- Product page with price, availability and a delivery date: Assembled from several teams’ services. It must stay fast when one of them is slow, and show a price and stock state that checkout will rarely contradict.
- Products with variants: A parent product (a shirt) with child SKUs (size × colour), each its own unit of price and stock, with the variation dimensions defined by the category.
- A cart that follows the buyer: Works signed out (a guest cart) and signed in, across devices, for weeks. The guest cart merges into the account cart at sign-in. Adding to the cart never reserves stock.
- Check out: Address, a shipping option with a promised date, and payment, at an authoritative price. One click on Place order produces exactly one order, however often it is retried.
- Never sell stock that is not there: Stock sits in about 100 warehouses. Reserve it when the order is placed, release it if the order fails, and do it without one hot row per product.
- Fulfil each order from the right warehouses: Chosen after the order is placed: the cheapest plan that still meets the promised date, which may split one order into several shipments.
- Order status and cancellation: Status per shipment. Cancelling before a shipment leaves releases its stock, and nothing that never shipped is charged.
- Out of scope: Search and recommendation ranking, seller tools and choosing between several sellers’ offers (one offer per SKU here), returns, warehouse operations, and payment internals: payment is a call to the payment service.
Non-functional (7)
- Product page latency (p99 < 250 ms server-side): The page waits on about six reads owned by different teams, so without deadlines its latency is the slowest one’s. A widget that misses its budget is replaced or dropped, never waited for.
- Read scale (100 k pages/s · 600 k reads/s): About 350 backend reads per order at the sale peak. No product page read may land on a system that checkout depends on, apart from a catalogue lookup on a cache miss.
- Freshness of price and stock (page < 5 s · checkout exact): The page may be a few seconds behind; the quote may not. Cart and checkout re-read the authority, so a stale page costs the buyer a message, never a wrong charge.
- Cart availability (99.99 %, no lost items): Add to cart is the step closest to revenue that needs nothing scarce, so it must keep working when other parts of the site are degraded.
- No oversell from concurrency (0 races · residual < 0.1 %): Physical counts are sometimes wrong (damaged or mislaid units), so a small residual is handled by policy: re-source, backorder or cancel. This drives the hardest tradeoff: an exact count per SKU against one deal SKU taking thousands of reservations a second.
- Place order (exactly once · p99 < 2 s): Most of the budget is the payment authorization. A retry, a double click or a crashed worker returns the same order, never a second one.
- Sale-day surge (10× orders at a known time): The spike arrives in seconds and autoscaling reacts in minutes, so capacity is raised beforehand, checkout is admitted at the rate payments can take, and recommendations are shed before anything on the path to an order.
Estimates · 5 min
Ask for two or three numbers, not all of them. What matters is whether they state assumptions, round sensibly, and say what the number implies. Push once with “where did that come from?”
The numbers (8)
- Orders per second: ~170/s · ~1.7 k/s on a sale day. 10 k orders/min ÷ 60 ≈ 170/s; 10× at a sale-day peak is ~1.7 k/s. At ~2.5 lines per order that is ~4 k reservations a second at the peak. Spread over hundreds of millions of stock rows this is easy; the hard part is the few SKUs everyone wants at once.
- Backend reads for product pages: ~600 k/s. 100 k page views/s × ~6 reads each (product document, price, stock, reviews, two recommendation widgets) = ~600 k reads/s, against ~1.7 k orders/s: about 350 reads per order at the peak. This is the number the caches and views exist for.
- Catalogue size: ~10 TB. 200 M parent products (500 M SKUs) × ~50 KB of attributes, text and variant data, images excluded = ~10 TB, ~30 TB with three replicas. Too big for one machine and only ever read by key, so a document store sharded by product id.
- Hot product cache: ~100 GB for ~90 % of reads. Views follow a steep power law: the top ~1 % of products (2 M) take about 90 % of page views. 2 M × 50 KB = ~100 GB, a small cache cluster. The long tail misses and costs one key lookup in the store.
- Price and stock views: ~45 GB, complete. 500 M SKUs × ~50 B of price data = 25 GB; 200 M stocked SKUs × ~100 B of per-region stock flags = 20 GB. Small enough to hold every SKU, so these are complete views kept current by events, not caches that miss to a system of record.
- Price changes: ~600/s. ~50 M price changes a day across the catalogue ÷ 86 400 ≈ 600/s. A 60 s TTL instead would refetch every SKU viewed in the last minute, say 5 M, every minute: ~80 k/s to the price service, and still up to a minute stale. Pushing changes is both cheaper and fresher.
- Carts: ~400 GB · ~5 k adds/s. 200 M live carts, signed-in and guest, × ~2 KB = ~400 GB. About 5 % of product views add to cart: 100 k × 0.05 = ~5 k adds/s at peak. A small, write-heavy key-value problem where availability matters far more than capacity.
- Busiest stock row on a deal: ~1 k reservations/s. A popular deal draws ~3 k reservations/s in its first minute. It is stocked in 10 warehouses, but the one nearest the largest metro takes about a third: ~1 k/s on one row, twice what a contended row serves at low latency (~500/s). Split into 10 sub-counters, each takes ~100/s.
High-level design · 16 min
Let them draw. Interrupt only to ask what backs a component or what a box actually does. Then pick one flow below and ask them to walk it end to end.
Components (15)
- Web / app: Loads the product page shell from the CDN and the dynamic buy box from the API, so the expensive part of the page is cached and the fresh part is small. Holds the guest cart id in a long-lived cookie until sign-in. Sends the checkout id as the idempotency key on Place order and polls the order rather than resubmitting.
- CDN (shell 10 min · images 1 yr): Caches the product page shell (title, images, bullets, variant picker) per product and locale, and every image under a content-hashed URL. Price, stock and anything personal are deliberately left out of what it caches, so a cached page can be minutes old without ever showing a wrong price. Listing changes purge by product id.
- API gateway (auth · routing · admission): Authenticates the session, routes page, cart and checkout calls, and applies per-route rate limits. On a sale day it is where load is shaped: optional traffic is shed first, and Place order is admitted at the rate checkout and the payment service can take, with a short queue instead of errors.
- Page aggregator (parallel reads · budgets): Builds the dynamic part of a product page from about six parallel reads, each with its own deadline and fallback, inside a 250 ms server budget. Required parts (document, price, stock) fall back to stale or coarse values; optional parts (reviews, recommendations) are dropped. It never calls a system of record on the page path except the catalogue on a cache miss.
- Catalogue store (document per product): One document per parent product, keyed by product id, with its variants (SKUs) inside and its attributes validated on write against the category’s schema in a schema registry: a TV must have a screen size, a shirt has size and colour. About 10 TB, sharded by product id, read only by key. Search and browse come from an index built from its change stream.
- Reviews & recs (other teams · optional): Review summaries and recommendation widgets, owned by other teams and called with tight budgets (60 ms and 80 ms). They make a page better, not possible, so they are the first thing dropped on a slow call or a sale day. Recommendations fall back to a precomputed best-seller list for the category.
- Price service (authoritative prices · quotes): The source of truth for each SKU’s price, scheduled deal prices and promotions. Publishes every change with a version and an effective time to the event bus, which keeps the price view current. Computes the authoritative quote for the cart page and checkout, the only price a buyer is ever charged.
- Read caches (docs · price + stock views): Two kinds of data with different rules. Product documents are a true cache: ~100 GB of the hottest 2 M products, keyed by id and version, filled on a miss. Prices and per-region stock flags are complete views of every SKU (~45 GB) written from change events, so they never miss and are seconds fresh.
- Cart store (key-value doc per cart): One durable document per cart, keyed by user id or by a random guest id from a cookie: lines with SKU, quantity and the price seen when added. Changed by operations (add, set quantity, remove) applied with a version check, so two devices’ edits both land. Never holds stock and never treats its stored price as truth. Guest carts expire after 30 days idle.
- Checkout & orders (quotes · saga · order store): Prices the cart and the checkout with authoritative calls, then runs Place order as a durable saga: record the order, reserve stock, authorize payment, commit, mark placed. Owns the order store (relational, sharded by customer id) with orders, shipments, the saga’s step log and an outbox table, so each state change and its event commit together. Captures payment per shipment.
- Event bus (Kafka · keyed by entity): Order events from the order store’s outbox (keyed by order id), price changes and stock flips (keyed by SKU). Consumers are the view updaters, the fulfilment planner, search indexing, emails and analytics. Keying by entity keeps each SKU’s and each order’s events in order, which lets a view apply versions monotonically.
- Payment service (another team’s boundary): Authorizes at Place order, captures per shipment and voids on cancellation, keyed by order id or shipment id so retries are safe. Its internals (providers, idempotency store, ledger, unknown outcomes) are the Payment System design; here it is a boundary with an agreed sale-day quota. A timeout from it is an unknown, resolved by asking again with the same key.
- Inventory service (reserve · commit · release): Answers which warehouses can ship a line by when, and owns reservations: soft with a 10-minute expiry at Place order, then committed to the order or released. Every change is a conditional update on one stock row, so correctness never depends on callers behaving. Publishes stock flips (in stock, low, out) per warehouse, not every unit.
- Fulfilment planner (warehouse choice · splits): Consumes order.placed and chooses where each line really ships from: the cheapest plan that meets the promised date, one box when one warehouse has everything and a split when not. Moves reservations when its plan differs from checkout’s, releases shipments to warehouses, and reports each one back to the order service. Re-sources or backorders a line a warehouse cannot find.
- Inventory DB (row per SKU × warehouse): One row per SKU per warehouse (on hand, reserved, available) plus reservation records: about 500 M rows and 50 GB, partitioned by SKU so a SKU’s warehouses and reservations share a shard and every transaction stays on one. Rows of SKUs flagged hot for a sale are split into sub-counters. Relational, because the design leans on conditional updates and small transactions.
Flows to ask them to walk (5)
- Sign in, check out, place the order: The main write path. The guest cart merges into the account, checkout prices everything from the authority, and Place order runs as a saga: record, reserve, authorize, commit. Every step is idempotent, so a retry or a crash repeats nothing.
- The buyer signs in with items in a guest cart; the guest cart merges into the account cart.
- The buyer opens checkout; the checkout service reads the cart and asks the price service for an authoritative quote.
- Shipping options and delivery dates come from current stock near the address, without reserving anything.
- The buyer clicks Place order; the order is recorded as pending under the checkout id before anything else happens.
- The saga reserves each line at a warehouse that can meet the chosen date, with a 10-minute expiry.
- The saga asks the payment service to authorize the quoted total, keyed by the order id.
- The saga commits the reservations, then marks the order placed and writes order.placed to the outbox in one transaction.
- The buyer sees the confirmation; the cart is cleared and the outbox relay publishes order.placed.
- Open a product page and add to cart: The main read path. One page needs a product document, a price, stock near the buyer, reviews and recommendations; the aggregator reads them in parallel from caches and views and drops whatever misses its budget.
- The buyer opens a product page; the CDN serves the page shell and the images.
- The client asks for the dynamic part of the page: buy box, review summary and recommendations.
- The aggregator reads the product document from the cache, falling back to the catalogue store on a miss.
- In the same fan-out it reads the SKU’s price and its stock flags for the buyer’s region from the views.
- Reviews and recommendations are fetched with their own short deadlines; recommendations miss theirs and are dropped.
- The buyer, not signed in, adds a size to the cart; it is written under a guest cart id from a cookie.
- Prime Day: a deal opens at 10:00: The scale-breaking case: ten times the orders, and one SKU that thousands of buyers want in the same second. The price flips everywhere on the clock, stock is split so no row serialises the sale, optional widgets are shed, and checkout is admitted at the rate payments can take.
- Hours before, capacity is raised and the deal SKUs are flagged hot, splitting their stock rows into sub-counters.
- The deal price was published ahead with a start time, so at 10:00:00 every view switches together.
- Page views surge; the aggregator sheds recommendations and other optional widgets first.
- Place order requests are admitted at the rate checkout and payments can take; the excess waits briefly.
- Reservations for the deal SKU land on random sub-counters at the nearest warehouse with stock.
- The last units go; the stock flip is published and pages show the deal as sold out within a second or two.
- Abandoned and declined checkouts return their units, and the deal briefly reopens.
- A declined card and a crashed worker: The failure path. Every step that succeeded is either completed or undone, driven by durable state, so a double click never makes two orders and a dead worker never strands stock or a hold on a card.
- The buyer double-clicks Place order; the second request finds the pending order under the same checkout id.
- Both lines are reserved; then the payment service declines the card.
- The buyer is asked for another card while the reservations stay held, and leaves instead.
- The timer fires; the saga releases each reservation and marks the order cancelled.
- Another order: the worker running its saga dies right after the authorization succeeds.
- The resumed saga commits the reservations and places the order; the buyer’s polling sees it confirmed.
- A sweeper releases any soft reservation past its expiry that no saga committed.
- After the order: warehouses, shipments, capture: The asynchronous path. Checkout reserved at good-enough warehouses; now a planner chooses where the order really ships from, may split it, and each shipment is captured as it leaves.
- The fulfilment planner consumes order.placed and loads the order’s lines and reservations.
- It chooses the cheapest plan that still meets the promised date: one warehouse with every line, or a split.
- Where the plan differs from checkout’s reservation, it moves it: reserve at the new warehouse, then release the old.
- The shipment is released to its warehouse for picking and packing.
- When the box leaves, on-hand stock is decremented and the reservations are consumed.
- The planner reports the shipment; the order service records it and captures that shipment’s amount.
- The buyer checks the order and sees each shipment’s status and date.
Deep dives · 16 min
Pick two. Ask the headline question, let them answer, then use the follow-ups. The follow-ups are where the level gets decided, so leave time for at least three of them.
A catalogue of products that share nothing
Ask: A TV has forty attributes and a T-shirt has six. How do you store hundreds of millions of products?
Good answers name: Document per parent product, per-category schemas validated on write, Relational table per category, Entity-attribute-value rows in one relational table, Relational core columns plus a JSON column for attributes.
Our pick: A document per parent product, keyed by product id, in a sharded document store: common fields (title, brand, category, images), an attributes map, and a variants array in which each variant is a SKU with its own id, its variation values (size M, colour blue) and images. Attributes are validated on write against the category’s schema in a schema registry: required attributes, types, units and allowed values, versioned so older documents still read. Price and stock stay out of the document, because they change at very different rates and are owned by other services. Every write bumps a version and emits a change event that feeds the search index (facets and search), the cache and a CDN purge. Amazon’s public seller documentation describes listings as parent and child ASINs with variation themes that depend on the category, which is this shape; how Amazon stores them internally is not public.
- A seller lists a product in the wrong category with half the required attributes. What happens?
The write is rejected against the category’s schema with the missing fields named, so bad data never reaches the store. For bulk feeds the valid rows go through and the invalid ones come back in an error report, because failing a 10,000-row feed for one bad row is worse. A change of category re-validates against the new schema. A classifier that suggests the category from the title catches most of these before they are submitted. - How does "TVs over 65 inches under $800" work if the store only does key lookups?
It does not run against the store. A search index (Elasticsearch or OpenSearch class) is built from the catalogue’s change stream, with each category’s filterable attributes indexed as typed fields, so facets and range filters come from the index. Price is joined in from the price events, a few seconds behind. The store answers "give me product X"; the index answers "which products match". - A shoe comes in 3,000 variants. Is it still one document?
No. Past a few hundred variants the document is too big to read on every page view, and some stores cap an item at 400 KB. Split it: the parent keeps the variation dimensions and a compact matrix of variant ids, and each variant is its own document keyed by SKU, fetched only for the one selected. The page needs the matrix to draw the picker and one variant to fill the buy box. - Why keep price and stock out of the product document?
Rate of change and ownership. A description changes a few times a year; a price can change many times a day and stock changes with every order. Putting them in the document would mean rewriting a 50 KB document and purging every cache that holds it on each change, and would put three teams’ writes on one record. Separate services with their own views let the document be cached for minutes while price and stock stay seconds fresh. - How do you make an attribute required without breaking millions of existing products?
Schemas are versioned and changes are additive by default, so a new optional attribute is free. Making one required takes two steps: add it as optional, backfill it or chase sellers for it, then require it for new writes only. Each document records the schema version it was validated against, and readers treat a missing optional attribute as unknown rather than as an error.
One page, six services, 250 ms
Ask: The product page needs the catalogue, price, stock, reviews and recommendations. What happens when one of them is slow?
Good answers name: Server-side aggregator with per-widget budgets and fallbacks, Client-side composition: the browser calls each service, Sequential calls in one page handler, Whole page cached at the CDN.
Our pick: The CDN serves a shell that is the same for everyone (title, images, bullets, variant picker), cached for 10 minutes and purged on listing changes. The client then makes one call to a page aggregator, which issues every read in parallel with its own deadline inside a 250 ms server budget: product document 50 ms (cache, then store), price and stock views 20 ms, reviews 60 ms, recommendations 80 ms. Each has a fallback: a stale document from a local cache; "See price in cart" if the price view is unavailable; "In stock" without a date if the stock view is; the cached review summary, then nothing; a precomputed best-seller list, then nothing. The response lists what was degraded, so degradation is measured rather than silent. A sale-day degrade level, set as a runtime flag, turns optional widgets down in a rehearsed order. Hedged requests (a second read after the p95 latency) are worth it for the document read, which is cheap and idempotent.
- It is Prime Day and recommendations are at 200 ms p99. What do you do?
Turn them down before they hurt anything. The degrade level is a runtime flag, and level 1 replaces live recommendations with precomputed lists per category that cost one cache read. If the page budget is still under pressure, level 2 serves review summaries stale and level 3 drops exact delivery dates for a coarse promise. The buy box (price, stock, Add to cart) is never degraded. The levels are rehearsed in game days before the sale, not invented during it. - Why not give every widget a generous timeout, say a second, so they rarely miss?
Because then one slow widget makes the whole page slow for every buyer, which costs more than the widget earns. The budget comes from what the page can afford end to end, and each timeout from that budget minus the time already spent. A widget that keeps missing its budget is a reason to fix or precompute that service, not to raise the budget. - How do you know the page is degraded if it still returns 200?
Every response carries the list of widgets that fell back or were dropped, and the aggregator emits a metric per widget: served, stale, dropped. Dashboards show the degraded rate per widget and per page type, with an alert when a required part falls back above a small threshold. Without that, fallbacks turn outages into silent revenue loss that nobody sees until the weekly numbers. - A product goes viral and its document is not cached. 50 k requests arrive in a second.
The CDN shell absorbs most page loads before they reach the aggregator. Within each aggregator instance, concurrent misses for the same key wait on one in-flight read, and the cache fill is guarded so only one request repopulates the key. With a few hundred instances that is a few hundred store reads, not 50 k, and the next second is served from the cache. - Why does the aggregator not call the price and inventory services directly?
Because then 100 k pages a second would land on the systems that checkout depends on, and a traffic spike on pages would become a checkout outage. The page reads views those services publish into the read caches, so the page path and the order path share no system of record. Cart and checkout read the services themselves, at about a hundredth of the rate, which is where exact values matter.
Fresh prices on a cached page
Ask: If product pages are cached, how do price and availability stay right, and what stops a buyer being charged a stale price?
Good answers name: Complete views updated by change events; cart and checkout read the authority, Short-TTL caches in front of the price and inventory services, Read price and stock from the source on every page view, Price inside the CDN-cached page, purged on change.
Our pick: Keep price and stock out of everything cached for minutes. The price service publishes each change with the SKU, the new price, an effective time and a version; the inventory service publishes a stock flip when a SKU at a warehouse crosses a threshold (in stock, low, out). Consumers write complete views into the read caches, price by SKU and stock flags by SKU and region, applying an event only if its version is newer, with a 10-minute TTL as a backstop and a sampled hourly comparison against the sources. Scheduled prices are published ahead with their start time, so the view switches on the clock instead of on a burst of events. The cart page and checkout never trust the views: they quote from the price service and check or reserve against the inventory service, and a difference from what the buyer saw is shown before they pay. A stale page then costs a message, never money.
- The buyer saw $19.99, the deal ended, and checkout says $29.99. What happens?
The quote shows the new price on the review page with a clear note that it changed since the item was added, before the buyer can place the order. The buyer is only ever charged the quote they confirmed, which has an id and a 15-minute expiry. If it expires before they click, it is recomputed and shown again. - The consumer that updates the price view falls an hour behind. How do you notice?
Consumer lag per partition is the first alarm: the bus offset says exactly how far behind the view is, in time. The second is the quote mismatch rate, the share of checkouts where the authoritative price differs from what the view showed. Outside a few seconds after changes it should be near zero; a jump means the view is stale even if lag looks fine, for example because a consumer is silently dropping events. - Why publish only threshold crossings for stock instead of the count?
Because the page shows only a few states (in stock, only a few left, out of stock) while the count changes with every order. Publishing every change would make the event rate follow the order rate times the number of warehouses; thresholds make it follow the sell-out rate. "Only 3 left" can come from the low-stock band with a slightly stale count, because checkout reserves the real one. - Members see a different price from everyone else. Does that break the views?
Not if the view holds the few price variants that exist per SKU (public, member, business) rather than a price per buyer, and the aggregator picks one by the buyer’s segment. Truly personal prices, such as a coupon only you hold, are applied at quote time and never on the page. What the page must never do is cache a personalised price under a shared key. - A price change and its reversal arrive out of order. What does the view show?
The newer one. Every event carries the price service’s version for that SKU, and the view writes conditionally, only when the incoming version is greater than the stored one. Events for a SKU are also keyed to one partition, so they normally arrive in order, but the version check is what makes a replay or a reordering harmless.
A cart that never loses an item
Ask: Where does the cart live, what happens when a guest signs in, and why does the cart not hold stock?
Good answers name: Durable key-value document per cart, operations applied with a version check, Dynamo-style always-writable cart with merge on read, Cart only in the browser (cookie or local storage), Cart rows in the relational order database.
Our pick: A document per cart in a replicated key-value store keyed by cart id: the user id for signed-in buyers, a random UUID in a long-lived cookie for guests. The document holds lines (SKU, quantity, price seen when added, time added) and saved-for-later items, with a version. Clients send operations (add, set quantity, remove), and the service applies each with a conditional write on the version, re-reading and re-applying on a conflict, so devices do not overwrite each other. On sign-in the guest cart merges into the account cart: union of lines, quantities summed for the same SKU and capped at the item’s limit, a notice to the buyer, and the guest cart marked merged_into so a repeated merge is a no-op. The cart never reserves stock and never treats its stored price as truth: the cart page is priced by the checkout service’s quote code, and checkout reserves. Amazon’s 2007 Dynamo paper used the shopping cart as its motivating example for an always-writable store, and its merge anomaly is why I would go that far only for multi-region writes.
- Why not reserve stock when the item goes into the cart?
Because most carts never become orders, and a reservation would hide stock from buyers who will buy. With around 5 k adds a second and carts that sit for weeks, cart reservations would lock up a large share of popular stock for nobody. Stock is checked when the cart is shown and reserved at Place order. The exception worth discussing is a limited-quantity deal, where a short claim at add-to-cart can be a product decision; even then it is a reservation in the inventory service with its own expiry, and the cart stores only its id. - The account cart has one of an item, the buyer adds one more as a guest, then signs in. How many?
Two, under the rule I chose: sum the quantities, cap at the item’s purchase limit, and tell the buyer the carts were merged. Taking the maximum is also defensible and avoids surprising doubles. What matters is that the rule is consistent, visible and idempotent; the merge runs once per guest cart because the guest cart records what it was merged into. - The cart store’s home region goes down. Can people still add to cart?
With one home region per cart, writes fail over to the replica region after detection, and a few seconds of the latest adds can be lost if replication was asynchronous. If that is not acceptable, run the cart store active-active across regions and accept per-line last-writer-wins or a union merge, with tombstones for removals so a removed item does not come back. The cart is one of the few places where that tradeoff is worth taking. - How long do you keep a cart, and is there a privacy angle?
Guest carts expire after 30 days idle through the store’s TTL; signed-in carts persist much longer and are deleted with the account. A cart says what someone was thinking of buying, which can be sensitive, so it is keyed by an opaque id, never shared across accounts, and leaves the cart service only in aggregate. - A buyer has 400 items in the cart. Any problem?
Cap it, at roughly 100 to 200 distinct lines, because documents have size limits and pricing 400 lines on every cart view is slow. Lines over the cap go to saved for later with a message. Large business orders belong in a bulk-order path rather than a stretched consumer cart.
Reserving stock without a hot row
Ask: How do you guarantee you never sell a unit you do not have, without serialising every order through one row?
Good answers name: Conditional decrement per (SKU, warehouse), soft reservations with expiry, sub-counters for hot SKUs, Read stock, check in code, then write, Stock counters in Redis, written back to the database asynchronously, One global count per SKU, warehouse chosen later.
Our pick: Each (SKU, warehouse) row holds on hand, reserved and available. At Place order the inventory service picks, per line, the nearest warehouse with stock that can meet the chosen date, and runs one transaction: a conditional decrement of available (WHERE available >= qty) and an insert of a soft reservation keyed by (order id, line) with a 10-minute expiry (5 minutes for deal SKUs). Commit clears the expiry when the order is placed; release returns the units; both are idempotent, and a sweeper releases expired soft reservations only if they are still soft. SKUs flagged hot for a sale have each warehouse row split into 10 sub-counters; a request tries a random bucket, then the others, then the next warehouse. Rows are partitioned by SKU, so a SKU’s warehouses and reservations share a shard and every transaction stays on one. For the physical residual, a small safety stock is held back on low-count SKUs, and when a warehouse cannot find a unit the planner re-sources it, backorders it against inbound stock with a new date, or cancels the line, by category policy.
- A bucket says 0 but its neighbour has 40. Does the buyer see "out of stock"?
No: an empty bucket is not an empty warehouse. The request tries the remaining buckets for that warehouse, a few cheap conditional updates, then the next warehouse that still meets the promise. The line is out of stock only when every eligible bucket is empty. A background job also rebalances buckets that drift far apart, which keeps the probing short until the last few units. - The inventory database primary fails during the deal. What is lost?
Nothing committed, with a synchronous replica, and failover takes seconds. During those seconds reservations fail fast, sagas retry with the same reservation keys, and Place order requests wait in the gateway’s queue. Because each reservation is keyed by (order id, line), a retry after failover either finds the reservation it made before or makes it now, never both. - Why reserve at Place order rather than when checkout opens?
Most buyers who open checkout do not place the order, and a reservation at that point hides stock from those who will, which matters most for exactly the scarce SKUs. Reserving at Place order means the checkout page’s delivery date is computed from current stock but not guaranteed, and occasionally a buyer is told the item just sold out. For a long checkout, such as a business buyer collecting approvals, a short hold at checkout start can be worth it; it is the same reservation call made earlier. - How would you know you are overselling?
Count the outcomes only an oversell produces: shipments a warehouse could not fill, lines backordered or cancelled after placement, and reservations that exceed on hand when a warehouse reports a cycle count. Those are dashboards per warehouse and per SKU. A nightly reconciliation also checks that reserved equals the sum of live reservations on every row, which catches leaks in either direction. - 10× the sale: 30 k reservations a second on one deal. Now what?
Sub-counters scale with the number of buckets, so first raise them, say to 50 per warehouse, and stock the deal in more warehouses. Past that, a relational row is the wrong tool for that one SKU: move its count into an in-memory counter with an atomic decrement for the length of the deal, hold back a buffer of units to absorb a failover, and write reservations to the database asynchronously with reconciliation. At that point it is the Ticketmaster problem, and a waiting room in front of the deal beats a faster counter.
Place order as a saga
Ask: Place order reserves stock, authorizes a card and creates an order in three systems. What happens when the second step fails, or the server dies after it?
Good answers name: Orchestrated saga with a durable step log in the order store, Choreography: each service reacts to the previous one’s event, Two-phase commit across inventory, payments and orders, Charge first, then reserve and create the order.
Our pick: Place order is a durable workflow keyed by the checkout id. Step zero inserts the order as pending under a unique constraint on checkout_id, so a double click finds the existing order. Then: reserve every line (soft, 10-minute expiry); authorize the quoted total with the order id as the payment idempotency key; commit the reservations; mark the order placed and write order.placed to the outbox in the same transaction; clear the cart. On a decline the order waits for another payment method until shortly before the reservations would expire, then compensates by releasing every reservation and marking the order cancelled. On a worker crash the 30-second lease lapses and another worker resumes from the last recorded step. A payment timeout is an unknown, retried with the same key, never treated as a decline. After placement the order moves placed → allocated → partially shipped → shipped → delivered, with cancelled reachable until a shipment leaves; each shipment is captured for its own amount as it ships, so a cancelled line is never charged.
- The authorization succeeds, then committing the reservations fails because the inventory database is failing over.
The saga retries the commit with backoff; it is idempotent on (order id, line), and the soft reservations have up to 10 minutes before they expire, far longer than a failover. If the inventory service stays down past a safety margin before expiry, the saga compensates: void the authorization, mark the order failed and tell the buyer. It never marks an order placed without committed stock. - A three-item order was reserved at three warehouses. Does it ship as three boxes?
Not necessarily. Checkout reserved each line at the nearest warehouse that could meet the promise, which is fast but not optimal. After the order is placed, the fulfilment planner looks at every warehouse with stock for the whole order and picks the cheapest plan that still meets the date, with a penalty per extra box: if one warehouse has all three, it moves the other two reservations there, reserving the new one before releasing the old. Doing this asynchronously keeps a slow optimisation off the buyer’s click. - Why capture at shipment and not when the order is placed?
Because the order may not ship in full: a line can be cancelled, backordered or found damaged, and charging for it means refunding later, which costs fees and trust. Capturing per shipment charges exactly what left the building. The cost is that authorizations expire, often after about 7 days, so a long backorder needs a fresh authorization, which the payment service handles with the same idempotency keys. - Checkout is at its limit on Prime Day. What happens to the next Place order?
The gateway admits Place order at the rate checkout and the payment service’s agreed quota can sustain, about 2.5 k a second, and queues the rest for a few seconds behind a "Placing your order" screen, keyed by checkout id so retries do not duplicate. If the queue grows past what can drain within the quote’s expiry, new checkouts see a short wait before they start, rather than failing at the last click. A buyer who has already clicked Place order is the last one to turn away. - How do you find orders that are stuck?
Every pending order has a deadline, and a sweep looks for pending orders past it; that count should be near zero, and a rise pages someone. Reconciliation also joins orders, reservations and authorizations nightly: an authorization without a placed order is voided, a committed reservation without an order is released, and a placed order missing either is investigated. Time in pending at p99 is the headline metric for checkout health.
Close · 5 min
Ask what breaks first at ten times the load, and what they would build next. Then give them your read: one thing that was strong, one thing that was missing, one thing to practise. Be specific; “good job” helps nobody.