The bulkhead pattern: an interactive thread pool isolation simulator
The bulkhead pattern gives each dependency its own limited share of a service’s resources, such as threads, connections or memory, so that one slow or failing dependency can only exhaust its own share. Like the watertight compartments of a ship’s hull it is named after, it does not stop the failure; it stops the failure from flooding everything else.
The playground runs a checkout service with 40 worker threads calling payments, recommendations and search. Halfway through, recommendations slow down to several seconds per call. Compare one shared pool with bulkheads, and long timeouts with short ones.
From 20 s, recommendations take 3 to 10 seconds to answer. Each of those calls holds one of the 40 shared threads for up to 5 s, and at 120 calls a second they take every thread within moments. Now payments and search, which are perfectly healthy, find no free thread and are rejected: only 11% of payments and 10% of searches succeed while recommendations are slow. One slow dependency has taken down the whole service.
How one slow dependency takes everything down
A slow call is worse than a failed one. A failed call returns at once; a slow one holds a thread, a connection and memory for as long as it waits. In the playground, recommendations at 120 calls a second, each holding a thread for up to 5 seconds, need hundreds of threads; they take all 40 within a second. Payments and search are healthy, but their requests find no free thread and are rejected: only 11% of payments succeed while recommendations are slow. The least important feature on the page has taken checkout down with it.
How bulkheads contain it
With separate pools, 16 threads for payments, 8 for recommendations and 16 for search, recommendations fill their 8 and further recommendation calls are rejected immediately, while payments and search still have all their threads and succeed 100% of the time. The page renders without recommendations instead of failing. The same idea applies to any shared resource: separate connection pools per downstream service, separate queues or worker pools per tenant or priority, separate clusters for critical and best-effort traffic.
Timeouts are the other half
How many threads a slow dependency can hold is its call rate times its timeout. Cut recommendations’ timeout from 5 seconds to 200 ms and they hold about 24 threads instead of all 40, so even the shared pool survives: 99% of payments succeed. But that depends on there being room to spare; double the traffic and the shared pool fills again. Set timeouts from each dependency’s real latency, a little above its p99, and use bulkheads so that a timeout set too long cannot hurt anything else.
Sizing a bulkhead
- From Little’s law: a pool needs about rate × latency threads in normal operation; give it some headroom over that, and no more than you can afford to lose to that dependency.
- Reject, don’t queue: when a pool is full, fail at once and fall back, rather than queue requests behind calls that are stuck.
- Have a fallback: cached results, a default, or leaving the feature out.
- Combine with a circuit breaker: the bulkhead limits how much a failing dependency can hold; a circuit breaker stops calling it altogether once it is clearly down.
A bulkhead in code
// Resilience4j: at most 8 concurrent calls to recommendations, and don't wait for a slot
BulkheadConfig config = BulkheadConfig.custom()
.maxConcurrentCalls(8)
.maxWaitDuration(Duration.ZERO) // reject at once when full
.build();
Bulkhead recs = Bulkhead.of("recommendations", config);
TimeLimiter limit = TimeLimiter.of(Duration.ofMillis(200));
Supplier<List<Item>> call = Bulkhead.decorateSupplier(recs, () -> client.recommend(user));
List<Item> items = Try.ofSupplier(call)
.recover(BulkheadFullException.class, e -> List.of()) // degrade: no recommendations
.get();In a system design interview
When a service calls several others, say how a slow one is contained: “Each downstream call has its own connection pool and concurrency limit, sized from its rate and p99 latency, and a timeout just above that p99. If recommendations slow down, at most 8 requests wait on them and the rest get an empty list at once, so checkout keeps its own capacity.” Expect the follow-ups: how pools are sized, what the fallback is, and how this works with async I/O, where the scarce resource is connections or memory rather than threads.
Read the guide to rate limiting and resilience, see load shedding for protecting a service from its own callers, or watch a strong candidate design a payment system.
Questions people ask
What is the bulkhead pattern?
A resilience pattern that gives each dependency, or each class of work, its own limited pool of resources such as threads, connections or memory. If one dependency becomes slow or fails, it can only exhaust its own pool, and calls to other dependencies keep working. It is named after the watertight compartments in a ship’s hull.
Why can one slow dependency bring down a whole service?
Because each call to a slow dependency holds a thread, connection and memory while it waits. If those come from a pool shared with other calls, the slow calls accumulate until they hold all of it, and requests that would have gone to healthy dependencies find nothing free and fail. A slow dependency is often more dangerous than one that fails fast.
How do you size a bulkhead?
Use Little’s law: a dependency needs roughly its call rate times its normal latency in concurrent calls. Give its pool some headroom above that, but no more than you can afford to lose if that dependency hangs. When the pool is full, reject new calls immediately and fall back rather than queueing them.
What is the difference between a bulkhead and a circuit breaker?
A bulkhead limits how many resources calls to one dependency can hold at once, so a slow dependency cannot starve others. A circuit breaker stops calling a dependency entirely once it is failing, and probes it before resuming. They are complementary: the bulkhead contains the damage while the breaker is deciding, and the breaker stops wasted calls afterwards.
How do timeouts relate to bulkheads?
The resources a slow dependency holds equal its call rate times how long each call waits, so a shorter timeout means fewer threads held. Timeouts set just above the dependency’s p99 latency limit the damage; bulkheads guarantee the limit even if a timeout is set too long or traffic rises.
Where are bulkheads used besides thread pools?
Separate connection pools per downstream service, separate worker pools or queues per tenant or priority so one customer cannot exhaust shared capacity, separate clusters or cells for critical and best-effort traffic, and Kubernetes resource limits that stop one container from consuming a whole node.
What should happen when a bulkhead is full?
The call should fail immediately and the caller should degrade gracefully: return cached or default data, omit a non-essential feature such as recommendations, or tell the user to try again. Waiting for a slot just moves the pile-up into a queue and ties up the calling thread instead.