Most production outages do not happen because a system was unable to do the work it was asked to do. They happen because the system could not refuse work. A queue grew until memory ran out. A connection pool exhausted itself trying to satisfy demand. A retry storm propagated downstream until every component in the chain was simultaneously overloaded. The original triggering event was almost beside the point — the failure mode was the inability to say no.
Backpressure is the discipline of making refusal a first-class behavior in your system. It is the set of patterns that let an overloaded component signal upstream callers to slow down, the architectural choices that prefer fast rejection to slow death, and the operational mindset that treats sustained queue growth as a bug rather than an inconvenience. This piece walks through what backpressure actually looks like in practice and why most systems do it badly by default.
The unbounded queue is a deferred crash
The most common backpressure failure is the unbounded queue. Code that looks like queue.put(work) with no bound, no timeout, and no overflow behavior is implicitly betting that consumers will always keep up with producers. They will not. The moment a downstream dependency slows down, work piles up in the queue. Memory grows. The OOM killer arrives. The service restarts. The queue is empty again, briefly. The cycle repeats.
The fix is to bound every queue and decide explicitly what happens when the bound is hit. The choices are: block the producer (which propagates pressure upstream), drop the work (which sheds load locally), or return an error to the caller (which lets the caller decide). Each is correct in different contexts; none is wrong. What is wrong is having no policy at all and discovering at 3 AM that the policy is to crash.
Block, drop, or fail: the three honest choices
Blocking is the right behavior for in-process producer-consumer pairs where the producer is itself a request handler that can hold the request open until the consumer catches up. It propagates pressure naturally: the request takes longer, the load balancer's timeouts fire, the upstream client sees a clear signal. The trap is when blocking happens in the request thread itself, holding worker capacity hostage to whatever is slow downstream. Block in dedicated worker threads, not in request paths.
Dropping is the right behavior for telemetry, analytics, log shipping, and other workloads where missing some work is acceptable but blocking the system is not. The discipline is to drop visibly: increment a counter, sample the dropped items, alert when drop rate exceeds a threshold. Silent drops are worse than no system at all because they hide the loss while you build dashboards from incomplete data.
Failing fast — returning an error to the caller — is the right behavior for synchronous APIs where the caller is itself a system that can react. The HTTP analog is 429 Too Many Requests with a Retry-After header. The caller can back off, retry, fall back to cached data, or surface the error to its own caller. The honest part is the speed: a 429 in 5ms is infinitely better than a 200 in 30 seconds, because the caller's resources are not held hostage waiting for an answer that may never come.
The retry storm is backpressure failure at the protocol level
Backpressure problems compound. A downstream service slows down. Upstream callers retry. Each retry consumes the slow service's capacity, which makes it slower, which causes more retries, which causes more slowness. The system enters a regime where most of its work is failed retries, and adding capacity does not help because the new capacity is immediately consumed by the storm.
The pattern that prevents retry storms is exponential backoff with full jitter, capped at a sensible maximum. The pattern that helps the system recover is the circuit breaker, which short-circuits calls to a known-failing dependency and gives it room to recover. The pattern that makes both of these work is the explicit retry budget: a per-tenant or per-endpoint cap on how many retries can be in flight at once, so retries cannot consume more than a fraction of total capacity. Without the budget, even well-designed retries become a denial-of-service attack against the system's own dependencies.
Token buckets and admission control
Rate limiting is backpressure at the API gateway. The token bucket is the canonical algorithm: requests consume tokens, tokens refill at a configured rate, and requests that find an empty bucket are rejected with 429. The bucket size determines how much burst is allowed. The refill rate determines the sustained throughput.
The subtle part is where to apply rate limits. Per-IP rate limits are easy to implement and easy to evade. Per-API-key rate limits are the right primitive for API products and are what we use across DocuMint, CronPing, FlagBit, and WebhookVault. Per-tenant limits at higher layers (background jobs, async workers) protect shared infrastructure from a single noisy tenant. Per-endpoint limits protect specific expensive operations from being called in tight loops by clients who have not read the documentation.
The deeper pattern beyond rate limiting is admission control: deciding at the system entry whether to accept a request at all, based on current load. Modern admission control measures the queue depth or response time at the entry and rejects new work when those metrics exceed thresholds, on the principle that accepting work the system cannot finish is worse than rejecting it. The rejection signal propagates pressure up to the load balancer and beyond, which is exactly what you want.
The producer-consumer contract
Inside a service, backpressure between producers and consumers requires an explicit contract. The asyncio queue with maxsize is one such contract: producers either await on a full queue (blocking) or call put_nowait and catch QueueFull (failing fast). The reactive-streams Publisher-Subscriber model is another: subscribers signal demand to publishers, and publishers are forbidden from sending more than was requested. Go's bounded channels are a third.
What these have in common is that the queue size is a knob the operator can turn. The queue size determines the latency-throughput trade-off: larger queues smooth bursts but add latency; smaller queues reject bursts but reduce tail latency. There is no universally right value. There is a value that matches your specific workload, and the right way to find it is to instrument queue depth as a metric and tune until the depth distribution matches your latency target.
Backpressure across the network
Within a process, backpressure is straightforward: the language and runtime usually give you the primitives. Across the network, it is harder, because TCP's flow control is operating at a layer below your application logic and may not match your application's notion of overload.
HTTP/2 has stream-level flow control that lets a server signal a client to slow down at the protocol level, but most application code never reaches into it. gRPC exposes flow-control signals through its streaming APIs, but the patterns are not widely used. The practical answer for HTTP APIs is the 429 response combined with the Retry-After header, which delegates the backoff decision to the client. The practical answer for asynchronous messaging is the bounded subscription combined with explicit ack-based pacing.
The metrics that matter
Backpressure is observable through a small set of metrics. Queue depth, sampled at high frequency, tells you whether the system is keeping up. Time spent waiting for work versus doing work, measured per worker, tells you whether the bottleneck is upstream or downstream of the worker. Drop rate and rejection rate tell you how often the system is exercising its overload behavior. Tail latency (p99, p99.9) tells you whether the system is degrading gracefully or hitting a cliff.
Healthy backpressure looks like: queue depth oscillates around a low steady-state value, drops happen during demonstrable spikes, p99 latency stays bounded even as throughput grows, and the system can absorb a 2x burst without falling over. Unhealthy backpressure looks like: queue depth grows monotonically, drops are zero until they suddenly are not, p99 latency degrades smoothly until it does not, and a 2x burst takes the system down.
The cultural part
Backpressure is one of those engineering disciplines where the technical primitives are well-understood but the organizational discipline is rare. Teams know about bounded queues and 429 responses; they often do not have the cultural muscle to insist on them everywhere, to treat every unbounded queue as a bug, to budget and instrument retries, to wire up admission control before the first incident demands it.
The argument for the discipline is the same argument as for any other reliability practice: the cost of doing it during normal operation is small, the cost of not doing it during an incident is enormous, and the difference between systems that survive 10x load spikes and systems that do not is almost always the systems that took backpressure seriously before they had to. The systems that refuse work gracefully are the ones that stay up. The systems that try to do all the work eventually fail to do any of it.