Job priorities, starvation, weighted shares and aging in a shared queue: an interactive simulator
When one pool of workers serves jobs of different urgency, the order it picks them in decides who waits. A single FIFO queue makes urgent jobs wait behind bulk work. Strict priority fixes that, but under sustained high-priority load the lowest class never runs: starvation. Weighted shares and aging guarantee progress for everyone, each with its own cost.
The playground runs an hour of jobs through workers that finish 12 a second: steady password resets and notifications, a batch of 6,000 weekly digests at the start, and a 15 minute surge of password resets.
Strict priority serves password resets instantly, even through the surge. But while they surge, everything below waits: notifications up to 3.8 min, and the digests get nothing at all for the whole surge, 38 min for the unluckiest. Under sustained high-priority load the lowest class starves completely.
What each policy costs
- One FIFO queue makes the urgent wait behind the bulk: password resets queued behind the digest batch, and then behind the surge, wait up to 8.3 minutes.
- Strict priority serves resets instantly, but sends no digests at all while resets surge; the unluckiest waits 38 minutes, and notifications wait up to 3.8 minutes. With a long enough surge, low priority never runs.
- Weighted shares of 6 : 3 : 1 keep everything moving, but cap resets at 60% of capacity, 7.2 jobs a second, below the surge’s 9: resets back up and wait up to 3.8 minutes. Weights must be set for the peak of the class that matters, not the average.
- Aging lets waiting jobs climb, capped below the top class. Resets are never delayed, digests keep trickling out during the surge, and notifications pay for it, up to 7.5 minutes.
A practical setup
# separate queues per class, workers pull by policy
queues:
critical: password resets, 2FA codes # strict first, but rate-limited per tenant
default: notifications, webhooks
bulk: digests, exports, re-indexing
worker loop (weighted, with a floor):
every 10 jobs: 7 from critical, 2 from default, 1 from bulk
if a queue is empty, its slots go to the next one
if the oldest bulk job has waited > 30 min, take one bulk job now # aging, bounded
# or: separate worker pools per class, so no policy can starve oneMany systems avoid the trade-off by giving each class its own workers, sized for its own load, so bulk work can never delay urgent work and urgent work can never starve bulk. The cost is idle capacity: workers for one class cannot help another.
In a system design interview
When a notification or job system has mixed urgency, say how it is isolated: “Transactional messages like password resets go on their own queue with dedicated workers, so a marketing blast or a digest batch can never delay them. Within the shared pool we use weighted pulling with a guaranteed minimum for bulk jobs, so they cannot starve. Per-tenant rate limits stop one customer from flooding the high-priority queue.” Expect the follow-ups: who decides a job’s priority, what stops everyone from marking their jobs urgent, and how to monitor the age of the oldest job in each queue.
Read the guide to background jobs or push notifications, see fair queuing and load shedding, or watch a strong candidate design a notification system.
Questions people ask
What is starvation in a priority queue?
When higher-priority work keeps arriving, a strict priority scheduler never gets to lower-priority jobs, so they wait indefinitely. It happens whenever the high-priority load alone can keep the workers busy.
What is aging in scheduling?
A technique that raises a job’s effective priority the longer it waits, so old low-priority jobs eventually get served. Capping the boost below the top class keeps critical work first while guaranteeing that everything else makes progress.
What are weighted priority queues?
Queues served in proportion to weights, for example 6, 3 and 1 jobs per round, with unused slots passed to the others. Every class makes progress, but a class can only use more than its weight when the others are idle, so weights must cover its peak.
Should urgent jobs share workers with bulk jobs?
Often not. Dedicated workers or queues for transactional work, such as password resets, guarantee their latency regardless of bulk load, at the cost of capacity that cannot be shared. Shared pools need weighting or limits to protect both sides.
How do you prevent everyone from marking jobs as high priority?
Assign priority by job type in the platform rather than letting callers choose, rate-limit or quota the high-priority queue per tenant, and alert when its volume grows, so the urgent lane stays reserved for genuinely urgent work.
How do message brokers support priorities?
Some, such as RabbitMQ, support per-message priorities within a queue; others, such as Kafka or SQS, do not, and systems use separate topics or queues per priority with consumers that pull from them in a chosen ratio.
What should you monitor in a priority queue system?
The age of the oldest waiting job in each class, not just queue length: a growing age in a low class is the earliest sign of starvation, and a non-zero age in the urgent class means it is under-provisioned.