Circuit breaker pattern: an interactive simulator

A circuit breaker wraps calls to another service and stops making them when that service is failing. While calls succeed it is closed and lets them through. When too many recent calls fail, it opensand fails every call at once, without waiting on a service that will not answer. After a cool-down it goes half-open and lets one trial call through: if it succeeds the breaker closes, if not it opens again.

The playground makes the same minute of calls twice, without and with a breaker, while the service has an outage. Compare how many calls hit the dead service and how much time the client spends waiting on it.

How a circuit breaker works

1. Closed: calls go through, and the breaker records whether each succeeded. A call that times out counts as a failure.
2. Trip: when the failure rate over a window of recent calls passes a threshold (say 50% of the last 20, once there are at least 10), it opens.
3. Open: every call fails immediately with an error or a fallback. Nothing is sent to the failing service.
4. Half-open: after the open period, one trial call (or a few) is let through.
5. Reset: if the trial succeeds the breaker closes and normal traffic resumes; if it fails, the breaker opens for another period.

Why a circuit breaker helps

Without one, a dead dependency fails slowly: every call waits for the full timeout. In the playground’s outage, 400 calls each wait a second, which in a real server is 400 seconds of threads or connections held for nothing, and enough of that takes the caller down too: a cascading failure. With a breaker, the waiting stops after the first second or two, the caller answers at once with a fallback, and the struggling service gets fewer requests while it recovers instead of a pile-up of retries.

The cost is the open period: after the service recovers, the breaker keeps failing calls until its next trial succeeds. A shorter open period recovers sooner but probes a sick service more often.

Circuit breaker vs retry

Retries handle a blip: one call fails, the next probably succeeds. A circuit breaker handles an outage, where the next call will fail too and retrying only adds load. They work together: retry a failed call once or twice with backoff, and put the breaker outside the retries so that a dependency that keeps failing stops being called at all. Retries without a breaker multiply traffic to a service that is already down.

Choosing the settings

Keep one breaker per dependency (and per host when hosts fail independently), so one bad payment provider does not trip calls to the search service. Libraries do this for you: Resilience4j and Polly implement this state machine with a sliding window and a failure rate threshold; Netflix’s Hystrix popularised the pattern and is now in maintenance. In a service mesh, Envoy’s outlier detection ejects failing hosts, and what it calls circuit breakers are limits on connections and pending requests.

A circuit breaker in code

import time

class CircuitBreaker:
    def __init__(self, threshold=0.5, window=20, min_calls=10, open_for=5.0):
        self.threshold, self.window, self.min_calls, self.open_for = threshold, window, min_calls, open_for
        self.results, self.state, self.opened_at = [], "closed", 0.0

    def call(self, fn, fallback):
        if self.state == "open":
            if time.monotonic() - self.opened_at < self.open_for:
                return fallback()                     # fail fast, without calling
            self.state = "half-open"                  # time's up: let a trial through
        try:
            result = fn()                             # fn must have its own timeout
        except Exception:
            self._record(False)
            return fallback()
        self._record(True)
        return result

    def _record(self, ok):
        if self.state == "half-open":                 # the trial decides
            self.state = "closed" if ok else "open"
            self.opened_at, self.results = time.monotonic(), []
            return
        self.results = (self.results + [ok])[-self.window:]
        failed = self.results.count(False)
        if len(self.results) >= self.min_calls and failed / len(self.results) >= self.threshold:
            self.state, self.opened_at = "open", time.monotonic()

This sketch is single-threaded. A real one also lets only one trial through while half-open, counts slow calls as failures, and exposes its state as a metric, because a breaker opening is something to alert on.

In a system design interview

Say what it wraps, the numbers and the fallback: “Calls to the payment provider go through a circuit breaker with a 2 second timeout that trips at 50% failures over the last 50 calls and stays open 30 seconds; while it is open we accept the order, queue the charge and tell the customer it is processing.” Expect the follow-ups: the fallback (cached data, a default, a degraded page, or a queue), how retries and the breaker interact, bulkheads that cap how many threads one dependency can hold, and how you know it opened.

Read the resilience guide for retries, timeouts and bulkheads, or watch a strong candidate design a payment system.

Questions people ask