Design GitHub

• Clone, fetch and push over HTTPS and SSH: Standard git clients speaking the smart protocol, version 2 where the client supports it. Authentication by token or SSH key, branch protection enforced on push. Anything that works against git itself must work against us.
• Browse code and history in the web UI: Files at any commit, history, blame and compares, read straight from the repository. A popular README is viewed thousands of times a minute, so rendered output is cached by object id.
• Pull requests: Propose merging a branch, often from a fork, into another: a diff, line comments, reviews, required checks, and merge, squash or rebase. Whether it merges cleanly is shown before anyone presses the button.
• Forks: One click and instant, even for a 5 GB repository, and a popular repository has tens of thousands of them. A fork must not copy its parent’s history.
• Webhooks and CI triggers: Every push, pull request and review becomes an event delivered to subscribed endpoints and CI systems within seconds, signed, at least once.
• Code search: Substring and regex search across every repository the user can read: all public ones plus the private ones they have access to, and nothing else.
• Large files: Models, videos and game assets through Git LFS: git stores a small pointer and the bytes go to object storage. Plain git objects over 100 MB are refused.
• Out of scope: Issues and project boards (plain rows in the metadata database), the CI runners and deployments themselves (see the Code Deployment System), packages, Pages and Copilot. Webhook delivery uses the standard machinery from the webhooks guide; here we cover what is specific to git.

• No acknowledged push is ever lost (2 of 3 replicas before ack): A push that printed “main -> main” and then vanished in a failover breaks CI, deploys and trust. This is the requirement that drives the hardest trade-off: synchronous writes to a quorum cost push latency, in exchange for never losing or reordering a ref.
• Refs never go backwards (linearizable per ref): Once anyone has seen main at commit X, no replica may serve an older main. CI, deploys and every cache key on the commit a ref points to.
• Git availability (99.95 % reads and writes): Reads survive losing two of a repository’s three replicas and writes survive losing one. A dead file server must not take a single repository offline.
• Latency (no-op fetch < 100 ms · push ack p99 < 1 s): No-op fetches are what CI pollers and IDEs do all day, so they must be cheap for us as well as fast for them: one round trip and a short ref list. Push latency is dominated by the replica vote, about four round trips.
• Scale (400 M repos · ~100 k git ops/s peak): Clones and fetches outnumber pushes about 100 to 1, and CI and automation make most of them. Repositories range from a few KB to monorepos of 100 GB.
• Hot repositories stay contained (one repo cannot starve its neighbours): A repository cloned a million times a day, or fetched by 2,000 CI jobs in one minute, shares its three file servers with millions of other replicas. Its load must land on caches, not on its replicas.
• Private means private (zero cross-repository reads): Private contents never leak through forks, caches, code search or webhooks. Every git request, cached pack and search hit is checked against the caller’s permission on that repository.
• Event freshness (webhook p95 < 10 s after push): CI that starts a minute late is a minute added to every developer’s loop. At least once and signed; order is not guaranteed.

• Git operations per second: ~35 k/s avg · ~100 k/s peak. Assume 3 B clones and fetches a day, about 60 % from CI and automation: 3 × 10⁹ ÷ 86 400 ≈ 35 k/s. US and European working hours overlap into a peak of about 3× ≈ 100 k/s.
• Pushes per second: ~350/s avg · ~1 k/s peak. 30 M pushes a day ÷ 86 400 ≈ 350/s, about 1 k/s at peak, so reads outnumber writes ~100 : 1. Each push is a few round trips to three replicas: ~3 k replica writes a second over 1,500 servers. Write throughput is not the problem; write consistency is.
• Repository sizes and storage: ~10 PB unique · ~30 PB on disk. Heavy-tailed: median under 1 MB, p99 around 1 GB, a few hundred monorepos of 10–100 GB. Assume a mean of 25 MB once forks share their parent’s objects: 400 M × 25 MB = 10 PB, × 3 replicas = 30 PB, growing roughly 30 % a year.
• File servers: ~1,500 servers · ~800 k replicas each. NVMe servers with ~30 TB each, kept ~70 % full to leave room for repacks and moves: ~20 TB of repositories per server. 30 PB ÷ 20 TB = 1,500, and 400 M × 3 ÷ 1,500 = 800 k replicas per server, almost all tiny and idle. A server’s load comes from its few busy ones.
• CPU to serve git at peak: ~8 k cores. Assume 97 % of operations are incremental fetches at ~20 ms of CPU (ref list, negotiation, a small pack) and 3 % are clones at ~2 CPU-s with bitmaps: 97 k × 0.02 + 3 k × 2 ≈ 8 k cores, about 17 % of 1,500 × 32. Without reachability bitmaps the server side of a kernel clone took 44 s instead of 1.6 s, so clones alone would need ~130 k cores: three times the fleet.
• One hot repository: ~70 Gbit/s at peak. 1 M full clones a day of a 250 MB pack, peaking at 3×: 1 M × 250 MB × 3 ÷ 86 400 ≈ 8.7 GB/s ≈ 70 Gbit/s, plus ~70 cores of pack CPU. Its three file servers have 75 Gbit/s and 96 cores between them, shared with ~2.4 M other replicas. One repository can saturate its hosts, so its clones must be served from somewhere else.
• CI herd after one push: ~130 cores · ~13 GB/s for a minute. A push to a busy monorepo starts 2,000 jobs that each shallow-clone the new commit within 60 s, a 400 MB pack costing ~4 CPU-s to build: 2,000 × 4 ÷ 60 ≈ 133 cores, and 2,000 × 400 MB ÷ 60 ≈ 13 GB/s. Every one of those requests is identical, so building the pack once turns 8,000 CPU-seconds into 4.
• Webhook deliveries: ~6 k/s avg · ~20 k/s peak. 30 M pushes plus ~170 M pull request, review, comment and check events = 200 M events a day, each delivered to ~2.5 endpoints (repository hooks, organisation hooks, installed apps): 500 M a day ≈ 5.8 k/s, bursting to ~20 k/s. Each push delivery to a CI system turns into fetches, which is how webhooks become the CI herd.

• LFS & bundle store (S3-class · behind a CDN): Holds Git LFS objects keyed by their SHA-256, and pre-built clone bundles of hot repositories. Clients fetch both from the CDN with signed URLs, so gigabyte binaries and a million full clones a day never touch a file server. These are large, immutable, read whole and never deltified: exactly what object storage is good at, and exactly what git objects are not.
• Developer (git CLI · browser · IDE): Runs standard git over SSH or HTTPS and uses the web UI and API for pull requests and browsing. The client does real work: it negotiates which objects it already has, verifies every object it receives against its hash, and with shallow or partial clone decides how much history it wants.
• CI runners (Actions · Jenkins · Buildkite): Receive push and pull request webhooks and fetch the new commit, thousands of jobs at once for a busy repository. Ephemeral runners make shallow clones of exactly that commit, so their requests are byte-identical, which is what makes them cacheable. Most git reads come from machines like these, not from people.
• Web & API (Rails · REST and GraphQL): The UI and API for everything that is not the git protocol: pull requests, reviews, forks, settings, search. Repository contents come from the file servers over an internal RPC (tree at a commit, blob, diff, merge), never by cloning. Rendered blobs and diffs are cached by object id, which can never go stale because an object never changes.
• Git frontends (SSH · smart HTTP · auth): Terminate SSH and HTTPS, authenticate the key or token, check the caller’s permission on the repository and its branch protection rules, then hand the git stream to the router. Stateless. They also cap concurrent pack builds per repository and per user, the last line of defence when one repository is hammered.
• Webhook delivery (per-endpoint queues · HMAC): Turns each event into one delivery per subscribed endpoint (repository hooks, organisation hooks, installed apps), signs it with that endpoint’s secret and POSTs it with a 10 s timeout. Retries back off per endpoint, so one dead CI server never delays anyone else. About 6 k deliveries a second on average.
• Metadata DB (MySQL · partitioned by domain): Users, permissions, repositories and their fork networks, pull requests, reviews, comments, check results and webhook subscriptions. Nothing from inside git lives here except the commit ids that rows point at. GitHub has written that it runs on 1,200+ MySQL hosts in 50+ clusters, split by feature domain (2023).
• Repo router (routes · votes · repair): Knows where every repository lives (repository → fork network → three file servers) and which replicas are in sync. Reads go to the least-loaded in-sync replica; pushes are streamed to all three and committed when two agree on the ref update. It also owns replica health: it marks replicas out of sync, re-replicates networks that fell below three, and moves networks to rebalance disk and CPU.
• Pack cache (keyed by request · single-flight): Generated packs keyed by a hash of (repository, wants, haves, depth or filter, capabilities), on a fleet of cache servers with NVMe and fast NICs. Concurrent identical requests wait for the one being built and stream it as it is written. It takes both the CPU and the bytes of a CI herd off a repository’s three replicas; an entry is only ever served for the repository that produced it.
• File servers (git on local NVMe · ×3): About 1,500 servers, each holding ~800 k repository replicas as ordinary bare git repositories on local disk, grouped by fork network. They run git itself: receive-pack for pushes, upload-pack and pack-objects for fetches, merge-tree and diff for the web. Each also repacks its busy repositories continuously (geometric repacks, multi-pack indexes, bitmaps), which is what keeps fetches cheap.
• Event bus (Kafka · keyed by repository): Ref-update events (repository, ref, old, new, pusher, push id) relayed from the replicas’ committed ref journals, plus pull request and review events from the web tier. Keyed by repository so one repository’s events stay in order. Webhooks, PR workers, code search and notifications each consume it at their own pace.
• PR workers (background merges · diffs): When a pull request’s head or base moves, ask a file server for the test merge (git merge-tree with merge-ort, no working copy), store it as refs/pull/N/merge and record mergeable or conflicting in MySQL. Debounced and prioritised by who is looking, because one push to a busy main invalidates thousands of open pull requests at once.
• Code search (ngram index · sharded by blob id): A separate index of default-branch contents, fed by push events and reading only the changed blobs. Documents are keyed by blob id, so a file shared by 10,000 forks is indexed once with a list of repositories. Each query carries the set of repositories the caller may read and is filtered inside the index. GitHub’s Blackbird shards by blob id for exactly this reason.

1. A developer pushes to main: The core write. Objects are the easy part because they are content-addressed and can be written anywhere; the only thing that needs agreement is the ref moving from old to new, and two of three replicas must agree before the client hears “ok”.
   1. If the commits include LFS-tracked files, the client uploads those bytes to object storage before git sends anything.
   2. The client opens receive-pack over SSH; the git frontend authenticates the key and checks write permission.
   3. The router looks up the repository’s replica set and opens a receive session on each of its three file servers.
   4. The client sends its ref update commands and a packfile; the router streams the same bytes to all three replicas, which verify the objects into quarantine.
   5. Prepare: each replica locks the ref, checks it still points at the old value, and votes with a hash of the exact update.
   6. With two matching votes the router tells the replicas to commit; a replica that disagreed or timed out is marked out of sync for this repository.
   7. The client sees “main -> main”, and the committed update is relayed to the event bus.
2. An everyday fetch, and a file in the browser: The core read, and almost all of the load. A fetch is a negotiation followed by pack generation; protocol v2 keeps the negotiation small, and bitmaps plus reuse of stored deltas keep the pack cheap.
   1. The client connects with protocol v2 and asks only for the refs it cares about.
   2. The router sends the request to the least-loaded in-sync replica.
   3. Client and server negotiate: the client names the commits it wants and the commits it already has.
   4. The file server works out which objects to send from reachability bitmaps, and reuses deltas already stored in its packs.
   5. The pack streams back; the client verifies every object against its hash and updates its remote-tracking refs.
   6. In the browser, viewing a file at a commit is an RPC to the same replicas, cached by blob id.
3. A push triggers 2,000 CI jobs: The scale-breaking case. Within a minute of one push to a busy monorepo, 2,000 runners ask for the same new commit; served naively that needs more CPU and bandwidth than the repository’s three file servers have. The requests are identical, which is what makes them cheap.
   1. Webhook delivery sends the signed push event to the CI system’s endpoint.
   2. The CI system starts 2,000 jobs, and each fresh runner asks for a shallow clone of exactly that commit.
   3. The router turns each request into a pack cache key and looks it up.
   4. One in-sync replica builds the pack once, and the router tees it into the cache as it streams.
   5. Cache servers stream the pack to the remaining runners, spreading the bytes over the cache fleet instead of the three replicas.
   6. Runners that kept a workspace fetch only the new commits, and those requests collapse into a few dozen cache keys.
4. A file server dies mid-push: The failure path. A server crashes after the pack is streamed but before the vote; the push still commits on the other two replicas, reads move away from the dead server, and its 800 k replicas are rebuilt elsewhere within the hour.
   1. Replica fs-0207 stops answering while a push to acme/api is between streaming and prepare.
   2. The router commits on the two healthy replicas, marks fs-0207 out of sync for this repository, and acknowledges the push.
   3. Missed heartbeats declare the whole server dead, and every network with a replica on it drops to two.
   4. The router queues the affected networks for repair, most urgent first, and picks a new home for each.
   5. Each new host copies its networks from a healthy replica with git, hundreds of copies in parallel across the fleet.
   6. The router compares the new replica’s checksum with the healthy ones and admits it as a voting, readable replica.
5. A pull request from a fork, merged and indexed: The background path. Pull request metadata is rows in MySQL, but every diff and merge is computed by git on the file servers, mergeability is recomputed in the background whenever either side moves, and code search catches up from the push stream.
   1. A contributor opens a pull request from a branch in their fork into the parent’s main.
   2. A PR event goes onto the bus, and a PR worker asks a replica to build the test merge.
   3. The worker records the result, keyed by the base and head commits it was computed for.
   4. A reviewer opens the diff, which the web tier computes from the merge base on the file servers.
   5. The reviewer approves and presses merge; main moves to the prepared merge commit through the same replicated ref transaction as a push.
   6. The merge’s ref update goes onto the bus like any push: webhooks fire, and every other open PR against main becomes stale.
   7. The code search indexer reads the push event, finds the changed blobs and indexes only the ones it has never seen.
   8. A developer searches code; the query carries the repositories they can read and is filtered inside the index.

Ask: Why store repositories as real git repositories on file servers? Why not put the objects in a database or an object store and keep the servers stateless?

Good answers name: Bare git repositories on local NVMe, placed by fork network, three replicas, Objects in an object store or distributed KV, refs in a consistent database, A shared network filesystem (NFS) behind stateless git servers, Git objects as rows in a relational database.

Our pick: Store every repository as a bare git repository on local NVMe and let git do what git does. Place by fork network (a parent and all its forks share one object store, so they live together), keep three replicas of each network on servers in different racks, and put a router in front that maps repository → network → hosts and tracks which replicas are in sync. With ~30 TB servers kept ~70 % full, that is about 1,500 servers for 30 PB, each holding ~800 k replicas. Accept the cost of placement: the router rebalances networks by disk and by CPU, copying with git itself and switching the route only when the new copy’s checksum matches. This is the shape GitHub has described for Spokes (first called DGit): three replicas of each repository on ordinary file servers, replicated at the git level rather than the disk level.

1. One network grows to 100 GB and starts eating a third of its servers’ CPU. What do you do?
   Move it, then make it cheaper. The router places big or hot networks on servers with headroom, dedicated ones for the very largest, so the neighbours stop paying for them. Then cut the cost per operation: frequent geometric repacks with multi-pack bitmaps, a pinned pack cache, a CDN bundle, and steering its CI toward blobless clones with sparse checkout. Splitting one repository across servers is the one thing we refuse to do, because git operations need the whole object graph in one place.
2. Why place by fork network rather than by repository?
   Because forks share objects with their parent through alternates, which only works on the same filesystem. Placing by repository would mean copying the parent’s objects into every fork, the storage cost sharing exists to avoid, or reading objects remotely, the latency problem local disks exist to avoid. The price is that a network with 50,000 forks is one placement unit, which is the forks deep dive.
3. How do you move a network between servers without downtime?
   Add the destination as a fourth, non-voting replica: it copies everything from a healthy replica with git and receives every new push while it catches up. When its checksum matches the voting replicas, the router swaps it in and drops the old one in a single routing change. Readers and pushers never notice. A 20 GB network at a throttled 100 MB/s moves in a few minutes.
4. Do the file servers use RAID?
   No. The three replicas on three servers already are the redundancy, so RAID would double disk cost for protection we have, and a failed disk is handled like a failed server: its replicas are rebuilt elsewhere by many servers in parallel. GitHub’s earlier design used RAID plus a DRBD-mirrored hot spare for each file server, four copies of every disk; moving to application-level replicas let any server help repair any other and put the spares’ CPU to work.
5. How would you know placement is healthy?
   Per server: disk fill, CPU spent in pack-objects, and the number of networks below three in-sync replicas. Per repository: CPU seconds and bytes served per minute, so the top hundred consumers are visible and can be moved before they hurt neighbours. A background audit that compares replica checksums daily tells us whether replication is actually keeping replicas identical, rather than just believing it.

Ask: A push has to land on three servers. Why not write it to a primary and replicate asynchronously, like a database? And what exactly do the replicas agree on?

Good answers name: Stream to all three replicas, vote on the ref transaction, commit at two of three, A primary with asynchronous replicas, Disk-level mirroring (DRBD) to a standby server, A consensus log (Raft) of ref updates per repository.

Our pick: The router coordinates each push in three phases. Stream: the client’s pack goes to all three replicas, each indexing and verifying it into a quarantine directory. Prepare: each replica takes the ref locks, checks that every ref still has the old value the client expected, and votes with a hash of the exact update; this is git’s reference transaction (update-ref --stdin), with the reference-transaction hook reporting the prepared state. Commit: with two identical votes the router tells the replicas to commit, which publishes the objects and moves the refs, and any replica that voted differently, timed out or failed to commit is marked out of sync for that repository before the client is acknowledged. Each replica keeps a checksum over all its (ref, value) pairs, updated by XOR on every change, so “in sync” is one comparison. GitHub has described Spokes as using a three-phase commit and this kind of checksum, costing four round trips to the replicas per push, and GitLab’s Gitaly Cluster votes on a hash from the reference-transaction hook in a similar way.

1. The router crashes after two replicas voted yes but before it sent commit. What happens?
   The replicas hold prepared transactions with ref locks, and they abort them after a few seconds without a decision. No client was acknowledged, so aborting breaks no promise; the client’s connection dropped and it retries the push, which finds the same old value and succeeds. If commit had reached one replica before the crash, that replica now disagrees with the other two; its checksum gives it away, and repair resets its refs to the majority’s.
2. Two people push to main at the same moment.
   Both read main at the same old commit and both send “old → new”. The router serialises ref transactions per repository, which costs nothing because pushes to one repository are rare, so the second prepare happens after the first commits and fails its old-value check identically on every replica. The second pusher gets git’s usual “rejected, fetch first” and rebases. Without that serialisation the two could each win a different replica and leave no majority.
3. Why not keep one replica in another region for disaster recovery?
   You can, and GitHub described doing exactly that in “Stretching Spokes”. The cost is that a push waits for round trips to the distant replica whenever it is part of the quorum. With two replicas in one region and one far away, the near pair usually forms the quorum, pushes stay fast, and the far one lags by milliseconds; losing the whole region then means the far replica may lack the last pushes, a trade to choose explicitly.
4. How do you know replicas are really identical, not just their refs?
   Refs plus content addressing are the whole logical state: if two replicas have the same refs, they hold the same reachable objects, or one of them is missing objects, which a connectivity check finds. So the checksum over refs proves logical equality, and a background job verifies object hashes and connectivity to catch disk corruption. Packs can differ freely between replicas; their layout is not part of the state.
5. What does the vote cost a push, and is that acceptable?
   Within a region, four round trips of well under a millisecond each, plus the slower of two replicas’ index-pack. For a typical push that is tens of milliseconds next to the client’s upload. For a 1 GB push it is dominated by index-pack, which runs on the replicas in parallel. Pushes are 1 % of operations, so spending latency on them to make every read trustworthy is the right trade.

Ask: Every fetch makes the server compute a custom pack. How do you serve 100,000 of them a second without a fleet ten times the size?

Good answers name: Reachability bitmaps, verbatim delta reuse, protocol v2, continuous repacks, Shallow or partial clones for every client by default, One precomputed full pack per repository, sent to everyone, Generate every pack from scratch.

Our pick: Keep every active repository well packed and bitmapped, and let git’s reuse do the work. File servers run maintenance continuously, triggered by how many new packs a repository has accumulated rather than by the calendar: geometric repacks that fold small packs into larger ones so the pack count stays logarithmic, multi-pack indexes, and bitmaps for the tips of important refs. Serve protocol v2 so clients request only the refs they need with ls-refs, which keeps the 500,000 refs/pull/* entries out of every ordinary fetch. Reuse stored deltas verbatim whenever the base is also being sent, and use delta islands so a fork is never served a delta against an object only another fork has. A typical fetch then costs ~20 ms of CPU and a typical clone ~2 CPU-seconds: ~8 k cores at peak. Bitmaps came from JGit at Google and were brought to git by GitHub engineers, who reported the kernel clone dropping from 44 s to 1.6 s.

1. A repository has had 10,000 small pushes since its last repack. What does a fetch cost now?
   More, in two ways: thousands of small packs mean every object lookup checks many indexes, and recent objects are not in any bitmap, so counting walks them the slow way. That is why repacks are triggered by accumulated packs: a busy repository is consolidated every few hours by a geometric repack that only rewrites the small packs. A multi-pack index makes lookups one index again without rewriting anything big.
2. Why is a shallow fetch expensive for the server when it sends less data?
   Because the server cannot use history to decide what to send. A depth-1 clone has no haves, so the server walks the whole tree of that commit, without a bitmap for that exact set and often without delta reuse, since bases may sit outside the cut. Fetching onto or deepening a shallow clone is worse, because the server must reason about a boundary it cannot see. In 2016 GitHub rate-limited the CocoaPods Specs repository, which was fetched about 1.1 million times a week and kept more than five server CPUs pegged; besides the volume, GitHub pointed at shallow clones and one enormous flat directory.
3. How does protocol v2 save work if the client still downloads a pack?
   It removes the fixed cost every request used to pay. A CI poller or an IDE fetching every few minutes usually gets no new objects at all, and in the original protocol each of those requests still received the full ref advertisement, which was the whole response. With ls-refs and a prefix, a no-op fetch is one small request and a few hundred bytes back, and the server reads a handful of refs instead of half a million.
4. Why not cache every generated pack?
   Because most fetches are unique: each client has its own haves, so the hit rate on arbitrary fetches is low and the space would be wasted. Caching pays where requests are identical, which is the CI herd and shallow clones of a just-pushed commit; those get the pack cache. Everything else relies on bitmaps and reuse making computation cheap enough to do every time.
5. How do you measure whether this is working?
   CPU seconds per operation per repository, split into counting and compressing, tracked at p99 by repository size. The reuse ratio, bytes sent verbatim over bytes sent. Bitmap coverage, the share of sent objects that came from bitmapped packs. When counting time grows for a repository, its maintenance is falling behind, and that is an alert long before it is an outage.

Ask: One push starts 2,000 CI jobs that all fetch in the same minute, and one repository is cloned a million times a day. Its three file servers cannot carry either. What do you do?

Good answers name: A single-flight pack cache for identical requests, plus bundles on a CDN for hot repositories, More replicas for hot repositories, Rate-limit the repository, Ask the heavy users to change.

Our pick: Put a pack cache between the router and the file servers. The key hashes (repository id, wants, haves, depth or filter, and the capabilities that change the bytes); the first request takes a short build lease and runs pack-objects on one in-sync replica, teeing the output into the cache, and concurrent requests for the same key stream the file as it grows. Hot entries are copied to several cache servers so the bytes spread across NICs; entries are evicted by space and age and never need invalidating, because commits are immutable. For repositories with sustained full clones, publish a bundle (a pack of the default branch plus its refs) to object storage behind a CDN every few hours and advertise it with git’s bundle URI feature: the client downloads the bundle from the CDN, then fetches only what changed since, a few MB from us. Behind both, the frontends cap concurrent pack builds per repository, so the worst case is a queue rather than an outage. GitLab documents the same idea for its own CI: Gitaly’s pack-objects cache deduplicates concurrent identical fetches and serves them from cache.

1. Could an attacker use the pack cache to read a private repository?
   Only if an entry could be served for a different repository, and it cannot: the repository id is part of the key, and permission is checked on every request before the cache is consulted, exactly as for an uncached fetch. Two forks in one network asking for the same commit get separate entries. That costs some hit rate, and it is the right trade.
2. The CI system retries every failed fetch immediately. What happens when the cache tier is down?
   Without limits, a retry storm: 2,000 requests fail, retry at once, and keep three servers pegged indefinitely. So the frontends cap concurrent pack builds per repository per replica, say 16, and reject the rest with a retryable error and a jittered Retry-After. The herd becomes a queue that drains in a few minutes, and the repository’s neighbours keep their latency.
3. How fresh is a bundle, and what if the client’s git is too old for bundle URIs?
   It is rebuilt every few hours, or after enough new objects, so the follow-up fetch stays small; staleness only makes that fetch a bit bigger, never wrong, because the fetch brings the client to the real refs. Older clients do a normal clone, served by bitmaps and the pack cache. As clients upgrade, the share of full clones we build falls.
4. Ten times the CI volume. What breaks next?
   Bytes before CPU: the cache already collapses the CPU, but 20,000 runners downloading 400 MB each is 8 TB per push. The real fix is fewer bytes per job: persistent runners with warm workspaces fetching a few MB, blobless clones with sparse checkout for monorepos, or the CI system caching the checkout itself. We make those the documented default for big repositories and size the cache fleet for the rest.
5. How do you find hot repositories before they hurt anyone?
   Every file server reports CPU seconds and bytes served per repository each minute, and the router keeps a fleet-wide top list. A repository over a threshold, say 5 % of its servers’ CPU for ten minutes, gets a published bundle, cache priority and, if needed, a non-voting read replica. The same data shows which projects have pathological fetch patterns, which is sometimes a conversation with their maintainers.

Ask: A fork must be instant even for a 5 GB repository. How do forks share objects, and what does that sharing cost you later?

Good answers name: One shared object store per fork network; members read it through alternates, A full copy per fork, Copy-on-write filesystem clones (reflinks, snapshots), One global content-addressed store deduplicated across all repositories.

Our pick: Store each fork network as one shared object store, with the parent and every fork as small repositories that hold only refs and read objects through alternates. Pushes to any member write their objects into the shared store, and the network’s repacks consider the refs of every member, so an object lives as long as any member can reach it. Use delta islands, which GitHub contributed to git, so objects reachable from one fork are not stored as deltas against objects only another fork has; otherwise serving that fork would force the server to recompute deltas. Check read permission per repository, and decide explicitly whether a commit id from one member may be fetched through another: GitHub documents that commits in a fork network can be reached through any repository in it, which is convenient and, as researchers showed in 2024, a leak for anything pushed by mistake. When a repository must leave its network (it goes private, or is detached), copy the objects it can reach into a new network; that is slow but rare, and it is the price of instant forks.

1. Someone pushes an API key to their fork, then deletes the fork. Is the key gone?
   Not necessarily. The objects live in the network’s shared store and, in GitHub’s documented design, stay reachable by commit id through other members until garbage collection removes them, which only happens once nothing references them. Truffle Security showed in 2024 that commits from deleted forks could still be fetched through the parent this way, and GitHub said it works as intended. The honest answer is that a pushed secret must be rotated; the platform can also scan pushes for secrets and offer a purge that removes the objects from the whole network.
2. The parent repository is deleted. What happens to its 20,000 forks?
   They keep working, because the objects belong to the network’s store, not to the parent. The network picks a new root, usually the oldest or most active fork, and the parent’s refs are removed. Objects that only the parent’s refs reached become unreachable and are collected later.
3. A network has 50,000 forks and full garbage collection takes hours. How do you keep it manageable?
   Never run a full GC in the hot path. Geometric repacks fold new objects in cheaply, and full reachability over every member’s refs runs rarely as a throttled background job. Unreachable objects are kept in cruft packs for a grace period rather than deleted at once, because a concurrent push may be about to reference them. And the biggest networks get dedicated servers, because they are a placement unit we cannot split.
4. A public repository with public forks is made private. Now what?
   It cannot stay in a network whose objects public forks can fetch, so the network is split. The public forks stay in the existing network with a new root, and the newly private repository gets a fresh network holding a copy of the objects its refs reach. Copying 5 GB takes minutes, during which the repository is read-only; the split is rare enough that being slow is fine.
5. Why does a pull request from a fork not need any objects copied?
   Because the fork’s commits are already in the network’s shared store, which the parent reads through its alternates. Opening the pull request writes refs/pull/42/head in the parent pointing at the fork’s commit: one ref update and nothing else. Without the shared network, every pull request from a fork would begin with a fetch from one repository into another.

Ask: Where do a pull request’s diff and merge come from, and how do you show “no conflicts” on 5,000 open pull requests when main moves every two minutes?

Good answers name: Metadata in MySQL; diffs and merges by git on the file servers; mergeability recomputed lazily in the background, Recompute every open pull request on every push to the base, Store review data inside git, as Gerrit’s NoteDb does, Precompute and store every diff in the database.

Our pick: Keep pull request rows in MySQL, sharded by repository id so a repository’s pull requests, reviews and comments live together, recording the base and head commit at every update. Compute everything else with git on the network’s file servers: the diff from the merge base, and the test merge built with git merge-tree (merge-ort, no working copy) and stored as refs/pull/N/merge. Store the mergeability result with the (base, head) pair it was computed for. A push to main updates one row with main’s new commit, which makes every pull request against it stale without touching them; PR workers then recompute in priority order (being viewed, auto-merge enabled, recently updated, then the rest when idle), debounced so a burst of pushes costs one merge per pull request. Pressing merge moves main to the prepared merge commit through the same replicated ref transaction as a push, with main’s tested commit as the expected old value, so a merge can never land on top of a base it was not tested against. GitHub has written that it builds merges in the background ahead of the button, and that switching to merge-ort cut complex merges from 5 s or more to under 200 ms.

1. Someone presses merge just as main moves. What happens?
   The merge is a ref transaction whose expected old value is the main the test merge was built on. Main has moved, so the old-value check fails on every replica, nothing is written, and the pull request is re-checked against the new main, which takes a fraction of a second with merge-ort. The user sees a brief “checking”, or a conflict if there now is one. A merge never tested against the current main cannot land.
2. A pull request changes 40,000 files. How do you render the diff?
   Compute the list of changed paths first, which is cheap from two trees, then render each file’s diff on demand as it scrolls into view, with limits on lines per file and on total size before a “load diff” button. File diffs are cached by (base blob, head blob, options), which never goes stale. The page asks git for what is on screen, not for everything.
3. How do CI results attach to a pull request?
   Statuses and check runs are rows keyed by commit id, not by pull request, because one commit can be in several branches and pull requests. The page looks up the checks for its head commit, and branch protection evaluates them when the merge transaction is prepared. A new push changes the head commit, so old results simply stop applying.
4. Why shard pull request tables by repository rather than by user?
   Because nearly every query is within one repository: list open pull requests, load one with its reviews and comments, count them. Cross-repository views such as “awaiting my review” are served from a search index fed by pull request events, not by scattering a query over every shard. GitHub has written about splitting its MySQL into clusters by feature domain as it grew, and about the work of moving tables between them.
5. How stale can a mergeability badge be, and how do you know?
   Measure the time from a base or head change to a fresh result, per priority class: a few seconds for pull requests being viewed or set to auto-merge, minutes for idle ones. A stale badge shows “checking” rather than a wrong answer, so staleness costs a short wait, never a wrong merge. Queue depth and the age of the oldest stale viewed pull request are the alerts.