Design Robinhood

A retail brokerage: live prices to 3 M phones, 20 k orders a second at the open, every share and cent exact.

Last updated 2026-10-27. Difficulty: hard. Patterns: market-data, order-management, ledger, websockets, fintech. Reported at Robinhood, Coinbase, Bloomberg, Stripe, Meta, Amazon.

Functional requirements

• Live prices. Watchlists and the detail page show bid, ask and last for 10,000 symbols. A person reads a price a few times a second at most, and only the newest value matters.
• Place market and limit orders. Idempotent on a client order id, and checked against market hours, halts, buying power and shares owned before anything leaves for a venue.
• Cancel an open order. A cancel is a request, not a guarantee: the order may fill first, and the app must say so.
• Order status and fills in real time. New, sent, partially filled, filled, cancelled, rejected. Pushed over the app’s socket, and as a notification when the app is closed.
• A portfolio value that moves live. Cash plus each position times its latest price, moving with every tick, with the day’s change against yesterday’s close.
• Stop and good-til-cancelled orders. Held by the broker and triggered on the price stream, because venues do not hold a stop, and many accept only day orders.
• Price alerts. Notify once when a symbol crosses a threshold. ~10 M active alerts, heavily skewed to a few popular symbols.
• Books, records and reconciliation. An immutable audit trail of every order event for regulators, and a nightly match of trades, positions and cash against the clearing firm.
• Out of scope. Options, crypto, margin and short selling, fractional shares, bank transfers (see Design Venmo), the matching engine (see Design a Stock Exchange), and clearing and settlement themselves.

Non-functional requirements

• No duplicate or lost orders (one order per tap, every fill booked). A duplicated buy spends the customer’s cash twice and cannot be taken back; a lost fill leaves positions wrong. Both are worse than a slow order.
• Order entry latency (p99 < 50 ms in our systems). From the gateway to the order leaving the FIX gateway: ~5 ms admission, ~15 ms checks and one shard transaction, ~5 ms to store and send. The phone’s network adds 50 to 200 ms we do not control.
• Quote freshness (p99 < 500 ms feed to screen). Up to 50 ms of batching, up to 250 ms of tick and the mobile network. Prices never move backwards on screen, even after a reconnect.
• Scale at the open (3 M sockets · 20 k orders/s). About 100× the day’s average order rate, arriving within seconds of 9:30. This is the requirement that drives the hardest tradeoff: everything is sized for the open, and correctness cannot be traded for it.
• Exact books (to the cent and the share). Every fill moves cash and shares in one transaction, and every night our books are matched against the clearing firm’s.
• Available while the market is open (99.99 % from 9:30 to 16:00 ET). Planned work happens outside market hours. When something must give, order entry and cancels are the last to go.
• Auditability (every order event, write-once). Regulators ask years later what happened to an order and what the market was when it was routed, and expect an exact answer.

Back-of-envelope estimates

• Orders per second: ~210/s avg · 20 k/s at the open. 5 M orders over a 6.5-hour session (23,400 s) ≈ 214/s. The first minutes after 9:30 run at 20 k/s, about 100× the average. Size for the open, not the day.
• Quote messages without conflation: ~22 B/s. 3 M phones × 15 watched symbols = 45 M subscriptions. People watch the busiest symbols, ~500 updates/s each at the open: 45 M × 500 ≈ 22 B messages/s. No fleet can write that to sockets and no person can read it.
• Frames with a 250 ms tick: ~12 M frames/s. One frame per phone per tick: 3 M × 4/s = 12 M frames/s, carrying at most 45 M × 4 = 180 M symbol updates, ~125× fewer than forwarding everything. At ~20 B per symbol that is ~3.6 GB/s in all.
• Quote edge servers: 60 at peak · run 100. 3 M sockets ÷ 50 k per server = 60 servers, each pushing 50 k × 4 = 200 k frames/s, ~60 MB/s. Run 100 (30 k sockets each) so a zone loss or a reconnect storm fits.
• Latest-quote table: ~1 MB. 10,000 symbols × ~100 B (bid, ask, sizes, last, previous close, high, low, status, sequence) = 1 MB. At the open about half change in each 50 ms batch: ~500 KB × 20/s = 10 MB/s per subscriber. The whole market fits on every server.
• Ledger writes at the open: ~300 k rows/s. 20 k fills/s × ~10 rows (execution, order, entry, 4 postings, cash and position rows, outbox) = 200 k, plus 20 k orders/s × ~5 rows = 100 k. Across 32 shards: ~9 k rows and ~1.25 k transactions per shard per second, comfortable for Postgres.
• Portfolio reads at the open: ~50 k/s. 3 M people load their portfolio in about a minute: 3 M ÷ 60 s = 50 k reads/s of cash, positions and open orders. On the shards they would compete with the writes that move money, so they are served from a cache.
• Audit trail growth: ~7.5 TB/yr. 5 M orders × ~6 events (received, sent, acknowledged, fills, done) × ~1 KB = 30 GB/day; × 252 trading days ≈ 7.5 TB/yr, kept for years in write-once storage. Small: the hard part is that it is complete.

Components

• Trading app (iOS · Android · web): Holds one WebSocket for prices and its own order updates, and calls REST for orders and the portfolio. It computes the portfolio value itself, cash plus each position times the latest streamed price. It creates a client order id when an order ticket opens and reuses it on every retry of that order.
• API gateway (auth · admission control): Terminates REST, authenticates the session and runs admission control: 5 orders a second per account and an order-entry token bucket sized to what the order path sustains, 25 k/s. Excess orders wait in a queue for up to 2 s, then get 503 with a jittered Retry-After. Cancels bypass the bucket, because they reduce risk and load.
• Order service (checks · state machine): Owns orders. Runs the pre-trade checks (market session, halts, price collar, buying power, shares owned) and persists the order with its hold in one transaction on the account’s shard before anything is sent. Applies every execution report in one transaction, idempotent on the execution id. Also owns alerts and broker-held stops, so everything that can create an order goes through it.
• Account DB (Postgres · 32 shards by account): Everything about one account on one shard: orders and their events, executions, a double-entry journal in which cash and each stock are instruments, cached cash and position rows with their holds, alerts and held stops. Sharding by account makes every order and every fill a single-shard transaction. House accounts are append-only with no cached row, so nothing contends on them.
• FIX gateway (sessions · sequence numbers): The only component that speaks FIX, holding several sessions to each venue. Stores every outbound message with its sequence number before writing it to the socket, and every inbound one before passing it on, in a log replicated to a standby, so a reconnect is repaired with resend requests. Refuses to send a ClOrdID twice, and never turns an unanswered order into a new one.
• Feed handler (A/B lines · 1 MB table): Consumes the consolidated feed once, on two redundant lines, using whichever copy of each sequence number arrives first. Keeps the latest quote, last sale and trading status for 10,000 symbols in about 1 MB. Every 50 ms it publishes one batch of the symbols that changed, with each interval’s high and low, to the quote edges, the order service and the trigger engine.
• Quote edge (WebSocket · 100 servers): WebSocket servers, sized for 50 k sockets and run at ~30 k. Each holds the whole latest-quote table from the batches, and per connection a subscription set and a dirty set; every 250 ms it sends one frame with the newest values of the subscribed symbols that changed, and skips the tick for a socket still draining its last frame. It also delivers the account’s own order updates.
• Portfolio cache (Redis · per account): Cash, buying power, positions and open orders per account, kept current from order and fill events on the bus and versioned by the account’s event sequence. Serves the ~50 k portfolio reads a second at the open, so the shards see only orders and fills. It is also what the app shows, read-only, when the order path has to be protected.
• Trigger engine (sorted thresholds by symbol): Watches the price for what the venues do not hold: stops, price alerts and good-til-cancelled orders overnight. Partitioned by symbol; per symbol, below thresholds sorted high to low and above thresholds sorted low to high, compared with each batch’s low and high, so an update costs two comparisons unless something fires. Rebuilt from a compacted topic on the bus.
• Event bus (Kafka · from the outbox): Fed by the transactional outbox on every shard and keyed by account, so one account’s events stay in order. Consumers: the portfolio cache, the quote edges (each event to the edge holding that account’s socket), notifications, the trigger engine and the audit archive. At least once, so every consumer dedupes on the event id.
• Venues (exchanges · market makers): Exchanges and wholesale market makers that execute our orders and return execution reports. Each FIX session has its own sequence numbers and a message-rate throttle. Each venue also sends a drop copy, a second session carrying a copy of every execution, which is compared with our books every few minutes.
• Consolidated feed (SIP or vendor · 1 M updates/s): The US consolidated quote and trade feeds (the SIPs) or a vendor’s normalised version: the national best bid and offer, last sales and halts for every listed stock, around 1 M updates a second at the open. Delivered on two lines, so a packet lost on one is usually present on the other.
• Audit archive (write-once object storage): Every order event (received, rejected, sent, acknowledged, filled, cancel requested, cancelled) with the time, the device, the route and the quote when it was routed. Written from the bus into object storage under a compliance lock, partitioned by day and shard. The source of the next morning’s Consolidated Audit Trail report and of any best-execution review.
• EOD reconciliation (nightly, on replicas): After the close, joins the clearing firm’s trades, positions and cash with our ledger: trades by execution id, positions by account and symbol, cash by account. Also checks our own invariants: every hold has an open order, every cached row equals its postings. Breaks become cases with both sides of the evidence; it never corrects money by itself.
• Clearing firm (settles T+1): The clearing firm (or our own clearing entity) that settles trades through the clearing house on T+1 and holds the firm’s positions at the depository. Its end-of-day files are the outside view of what every account owns and owes, so any disagreement with them is a break to explain.

User flows

1. Market buy, from tap to fill. The core write path. The order is checked and recorded with its cash set aside before it leaves, and the fill moves cash and shares in one transaction, so a crash at any step leaves a state that can be resumed.
   1. The app posts a market buy for 10 AAPL with the client order id it created when the order ticket opened. The id is created once per ticket and reused for every retry, so a double tap or a retry after a timeout names the same order. The gateway checks the session and the account’s 5 orders a second, and takes a token from the order-entry bucket; at the open an order may wait here for up to 2 s.
   2. The order service checks the market session, the halt status and the price collar against its in-memory copy of the latest quotes. AAPL is trading and not halted, and the ask is 189.50, so the market buy will be sent as a limit at the ask plus 5 %, 198.98. The collar bounds the worst price, which is what gives the hold an upper bound. No database is read: the status table is 1 MB and refreshed every 50 ms.
   3. In one transaction on the account’s shard, it inserts the order as new, places the hold and writes the order event to the outbox. A unique key on (account_id, client_order_id) makes a retry find the existing order instead of creating a second. The hold is a conditional update that fails if cash minus existing holds is under $1,989.80, so two orders racing for the same cash cannot both pass. Nothing has left for the market yet, and that is the point: a crash from here on leaves a recorded order to resolve.
   4. It hands the order to the FIX gateway, which numbers it, stores it and sends a NewOrderSingle to the chosen venue. The ClOrdID is derived from our order id, and the gateway refuses to send a ClOrdID it has already sent, so a recovery sweeper can safely call send again. The message is stored with its sequence number before it goes on the wire, and the order moves to sent. Tap to wire, our share of the latency is under 50 ms at p99.
   5. The venue acknowledges and fills it; the gateway journals each execution report and passes it to the order service. A retail market order usually fills within milliseconds, often in one execution, sometimes in several. Each report carries an ExecID that is unique at that venue. Journaling before forwarding means a crash between receiving a fill and booking it loses nothing: the journal is replayed.
   6. The order service books the fill in one transaction: the execution, the order’s state, the ledger entry, the cash and position rows, and the hold. The entry moves $1,895.20 and 10 shares of AAPL between the customer and the house street-side accounts, each instrument summing to zero; the $1,989.80 hold is released and the order becomes filled. About ten rows in one commit, so cash and shares can never disagree. A replayed report hits the unique key on the execution and changes nothing.
   7. The outbox event reaches the bus; the edge holding the user’s socket pushes the fill, the portfolio cache updates and the audit archive appends it. A small router reads the account-keyed topic and publishes each event on the account’s pub/sub channel, which the edge holding that socket subscribed to when the app connected. If the app is closed, the notification service sends a push instead. Every consumer dedupes on the event id, because the relay is at least once.
   8. The app adds 10 AAPL to its positions and values them from the next price tick. The update carries the account’s event sequence, so the app applies it once and in order, and fetches the portfolio again if it sees a gap. Buying did not change the portfolio’s value, only what it is made of; from now on AAPL’s ticks move it.
2. Live prices to 3 M phones. The main read path and the biggest stream in the system. One feed is consumed once and conflated twice, so the cost grows with phones times ticks, not with how often prices change.
   1. The feed handler takes the consolidated feed on two lines and keeps the newest quote for each symbol. Lines A and B carry the same packets; whichever copy of a sequence number arrives first is used, so a drop on one line is usually invisible. At the open that is ~1 M updates a second into a table of 10,000 rows, about 1 MB: the whole market fits in any server’s memory.
   2. Every 50 ms it publishes one batch of the symbols that changed to every quote edge. At the open about half the symbols change in any 50 ms, so a batch is ~500 KB and each edge takes ~10 MB/s. Broadcasting the whole market to 100 edges is cheaper than routing subscriptions symbol by symbol, and a subscriber that falls behind skips to the newest batch rather than replaying old ones.
   3. The app connects, subscribes to its watchlist and positions, and gets a snapshot from the edge’s own table. No database is involved: the edge already holds every symbol’s latest value. The snapshot also carries the previous close and the day’s range, so the app shows the day’s change at once. Subscriptions are capped at 50 symbols per socket, which also bounds what a misbehaving client can cost.
   4. The app loads cash, positions and open orders from the portfolio cache. The response carries the account’s event sequence, and order updates on the socket continue from it, so the app never misses or double-counts a fill. The read never touches the account shards, which matters most at 9:30, when 3 M people do it in the same minute.
   5. Every 250 ms the edge sends one frame with only the subscribed symbols that changed since the last frame. Each socket has a dirty set: symbols are marked as batches arrive and cleared when sent. Forwarding every update would be ~500 a second per watched symbol, ~22 B messages a second over 45 M subscriptions; the tick caps it at 4 frames a second per phone, 12 M frames a second in all.
   6. A phone on a weak connection falls behind, and the edge skips its ticks instead of queueing them. If the last frame is still in the socket’s send buffer at the next tick, the edge builds nothing; the dirty set keeps growing, and the frame that finally goes out carries the newest value of everything that changed. A queue would deliver old prices late, which is worse than delivering new ones less often.
   7. On every frame, the app recomputes the portfolio value as cash plus each position times its latest price. The server never computes a per-user value: that would turn one AAPL tick into a recomputation for every holder, millions of them four times a second, plus a per-user message to route. On the phone it is a dozen multiplications, and the total always matches the prices on the same screen.
3. 9:30: the open, and a gap that fires stops. The scale-breaking case: order flow jumps ~100× in seconds, 3 M apps connect, and a stock that opens 8 % lower crosses 30 k stops at once. Capacity is in place before the surge, and admission decides what waits.
   1. At 9:00 a scheduled scale-up brings quote edges, gateways and order services to the forecast peak, and every FIX session is already logged on. Reactive autoscaling takes minutes and the surge takes seconds, so capacity follows a calendar: the busiest of the last 20 opens times 1.5, more on earnings and index rebalance days. The account shards are not scaled at all; they are sized for the open all day, because a database cannot be split during a surge.
   2. 3 M apps connect within minutes; each edge admits new sockets at a capped rate and the apps back off with jitter. TLS handshakes are the expensive part, so each edge accepts at most 2,000 new connections a second and answers the rest with a retry hint; 100 edges can take all 3 M in about 15 s. Edges are never deployed during market hours, because draining 30 k sockets at 9:31 is a herd we would have made ourselves.
   3. Portfolio screens load from the portfolio cache, so the shards see only orders and fills. 3 M opens in a minute is ~50 k reads a second, which on the shards would compete with the writes that move money. The cache is updated from the bus all day, so it is warm at 9:30 and lags the ledger by well under a second.
   4. 20 k orders a second arrive; the gateway admits them through the order-entry bucket and holds the excess in a short queue. The bucket is set to 25 k/s, the rate the shards and FIX sessions sustain in load tests, and each account is held to 5 orders a second so a script cannot take a crowd’s capacity. An order waits at most 2 s and then gets 503; the app retries with the same client order id, so a retry can never become a second order. Cancels skip the bucket.
   5. TSLA opens 8 % lower; the first batch’s low crosses 30 k stop prices and 120 k price alerts at once. For each symbol the trigger engine keeps its below thresholds sorted high to low, so the batch low of 230.10 pops every stop and alert above it in one walk, and nothing is examined that does not fire. Stops are compared with last sales rather than quotes, because a stop means "if it trades there".
   6. Each triggered stop enters the order service as an order whose client order id is derived from the stop, in its own lane. A stop that fires twice, after a trigger engine restart for example, produces the same client order id and so the same order. Triggered stops get a reserved fifth of the order-entry capacity, so new orders cannot starve them, and go out highest stop price first, the order in which the price crossed them, as collared sells like any market order.
   7. Each fired alert is marked fired exactly once and handed to notifications in a paced lane. Marking an alert fired is a conditional update from active, so a replayed trigger changes nothing and sends nothing. Alert pushes have their own lane in the notification service: a fill notification must not wait behind 120 k "TSLA is below 235" messages.
   8. If shard commit latency passes its threshold, the gateway sheds charts and news, then serves portfolios read-only while order entry keeps its capacity. The ladder is ordered by what customers need at a bad open: first history, news and search go; then the portfolio is served from the cache with an "as of" time and nothing else writes to accounts; order entry and cancels are touched last. Each rung is a flag the gateway reads, flipped automatically on shard p99 commit latency over 50 ms or by the on-call engineer.
4. The FIX session drops with orders in flight. The failure that separates a broker from a toy. An order whose acknowledgement never arrived has an unknown outcome, and it is resolved by repairing the session and asking, never by sending it again.
   1. Orders 7781 and 7790 go out as sequence numbers 50,112 and 50,113, then the connection to the venue drops before any reply. Both orders are sent, with their holds in place. The venue may have received one, both or neither, and nothing on our side can tell which. Sending them again as new orders risks buying twice, and nobody would find out until both fills arrived.
   2. The gateway reconnects and logs on; the venue’s sequence number is ahead of what we received, so the gateway asks for the gap. Sequence numbers run for the whole trading day in both directions and are stored on both sides, so a reconnect continues the session rather than starting a new one. Anything the venue sent while we were gone is requested with a ResendRequest from the first number we did not receive.
   3. The venue resends what we missed, including the acknowledgement and fill for 7781, which are booked exactly once. Resent messages carry PossDupFlag, and a fill we had in fact booked before the drop hits the unique key on (venue, ExecID) and changes nothing. If the venue instead never received 50,113, it asks us for it, and we resend it from our store with PossDupFlag and the same ClOrdID, which it recognises as the same order.
   4. Order 7790 still has no reply once the session is level, so the gateway asks the venue for its status. Five seconds after the session is level is long enough for any reply in flight. An OrderStatusRequest names the ClOrdID, and asking is always safe to repeat, which is exactly what sending is not.
   5. The venue reports the order as unknown, so 7790 is marked rejected, its hold released and the customer asked to place it again. We do not resend it on the customer’s behalf: the market has moved since the tap, and if the original surfaced later, a resent copy would be a second fill. A late fill for an order we marked rejected is still booked, because an execution report is evidence of a trade, and it opens a case.
   6. If the gateway process itself dies, a standby takes over the session from the replicated message store with the same sequence numbers. A venue accepts one logon per session identity, so the standby takes a lease first and the old process is fenced out before the standby logs on. Because the store holds every message and number, the venue sees one session that briefly disconnected, not a new one that has lost its history.
5. The close and the nightly reconciliation. The background path that proves the day. Day orders expire, the audit trail becomes the regulator’s report, and every trade, position and dollar is matched against the clearing firm’s records before the next open.
   1. At 16:00 the venues expire unfilled day orders, and each expiry releases its hold. Each expiry arrives as an execution report, which the order service applies like a cancel. An order still open in our books after the venues have reported is a discrepancy to chase, never something to expire quietly on our own.
   2. The day’s order events, already in the audit archive, become the regulator’s report before the next morning. Each event was copied to write-once storage within seconds of happening, with the quote at routing time, so the Consolidated Audit Trail report is a read of the archive rather than a reconstruction from tables that change. About 30 M events and ~30 GB a day.
   3. The reconciliation job pulls the clearing firm’s trade, position and cash files for the day. Trades settle through the clearing firm on T+1, so its records are the outside view of what each account owns and owes. The files arrive after the close; the job runs once they are complete, hours before the next open.
   4. It joins trades by execution id, positions by account and symbol, and cash by account against the ledger. Out come three lists, matched, ours only and theirs only, plus quantity and amount mismatches. Customer positions per symbol plus the house’s must equal the firm’s position at the depository, share for share. The job reads replicas, so it never loads the primaries.
   5. It checks our own invariants: every hold belongs to an open order, every cached row equals its postings, every order’s filled quantity equals its executions. These catch our bugs rather than the street’s. A hold on a filled order is buying power taken from the customer for nothing; a cached balance that disagrees with its postings means some code path wrote a row without an entry, and that path must be found.
   6. A break: the clearing firm has a 50-share fill we never booked, so a case opens and the trade is booked as of its trade date. The job never moves money by itself. An operator books the missing execution through the order service as a new, back-dated entry, which publishes the usual events, so the customer is told and the audit trail shows the correction. The intraday drop-copy comparison should have caught it within minutes, so the break also opens a bug.
   7. Before the next session, GTC orders are queued to be sent again as day orders and the trigger engine reloads its thresholds. Many venues accept only day orders, so a good-til-cancelled order lives with us and is re-sent each morning. The trigger engine rebuilds from a compacted topic holding the current state of every alert and held stop, keyed by its id, so it never has to scan the shards.

Deep dives

The theory behind it

• Real-time market data distribution: How price feeds reach traders and apps: exchange feeds and multicast, sequence numbers and gap recovery, order book snapshots plus incremental updates, conflation for slow consumers, fan-out to web and mobile clients, entitlements, historical tick storage, and latency tiers.
• Idempotency keys in practice: How to make retried API requests safe: what idempotency means, client-generated keys, storing keys with request fingerprints and responses, handling concurrent duplicates, expiry, keys across service boundaries, idempotent consumers, and the mistakes that still cause double charges.
• Money in system design: ledgers, idempotency and reconciliation: How to handle money correctly: integer minor units, double-entry ledgers, balances derived from entries, idempotent payment calls, the unknown-outcome state, authorisation and capture, and daily reconciliation with providers.

Related

• Design Slack
• Design a Payment System
• Design Google Docs
• Design Figma
• All 83 system design interview questions
• Run this question for a friend: the interviewer kit