Feature flag percentage rollouts: stable, independent bucketing, interactive

A percentage rollout turns a feature on for a share of users, say 5%, then 20%, then everyone, so problems surface while few people are affected. How users are put into that share matters more than it looks. The rule has to be stable, so a user sees the same version on every request; monotonic, so growing the rollout only adds users; and independent across flags, so the same users are not first in line for everything.

The playground has 400 users and two flags. Switch how users are bucketed, grow the rollout, and watch who flickers, who loses the feature and who ends up with both.

Three ways to pick 10% of users

• Random on every request gives the right share overall, but each user flips between versions: over five page loads, about 4 in 10 users see both. The interface flickers, carts built in one version break in the other, and growing the rollout takes the feature away from users who had it.
• A hash of the user id gives each user a fixed bucket from 0 to 99, so answers are stable and raising the percentage only adds users. But every flag reuses the same buckets, so the users at the front of the line get every new feature at once: all of dark mode’s 10% also have the new checkout, and if conversion moves you cannot tell which flag moved it.
• A hash of the flag name and the user id keeps stability and monotonic growth and gives each flag its own ordering. About 1% of users, 10% of 10%, have both flags, as independent experiments require.

Real systems add a few details: bucket by account or company rather than user when a feature must be the same for a whole team, evaluate flags locally from a cached rule set instead of calling a service per request, and log which variant each user saw so analysis matches what really happened. Changing the salt reshuffles every user, which is occasionally what you want, for instance to re-run an experiment on fresh users.

Stable, independent bucketing in code

import hashlib

def bucket(flag: str, user_id: str) -> float:
    """A stable number in [0, 100) per user and flag."""
    h = hashlib.sha256(f"{flag}:{user_id}".encode()).digest()
    return int.from_bytes(h[:8], "big") % 10_000 / 100

def enabled(flag: str, user_id: str, percent: float) -> bool:
    # stable: same answer every request; monotonic: raising percent only adds users;
    # independent: a different user order for every flag, thanks to the flag in the hash
    return bucket(flag, user_id) < percent

enabled("new-checkout", "user-42", 10)

In a system design interview

When a design ships risky changes, describe the rollout mechanics: “Flags are rules distributed to every server and evaluated in-process. A user’s bucket is a hash of the flag key and the account id, so whole accounts move together, each flag is independent, and going from 5% to 25% only adds accounts. Exposure events go to the analytics pipeline so experiments compare like with like, and every flag has a kill switch that propagates within seconds.” Expect the follow-ups: how flag changes reach servers, cleaning up old flags, and targeting rules beyond percentages.

Read the guide to experiments and feature flags or deployment strategies, see canary deployments and A/B test peeking, or watch a strong candidate design Netflix.

Questions people ask