Backpressure

Bulkheads left you at a wall. A bounded pool fills, and the next call is turned away immediately rather than allowed to widen the breach. Rejecting is the right move locally, but it leaves a question sitting one step upstream: the caller just heard no, I'm full — so what should the caller do? Send the same request again and you have a retry storm beating on a closed door. Sit and wait and you need someone to tell you how long and how much. Every tool so far in this section acts at a single point: a timeout bounds one wait, a breaker cuts one dead dependency, a bulkhead bounds one pool. Backpressure is the first tool that is a conversation between stages — a signal that travels from the overwhelmed part of the system back toward whatever is feeding it, and asks it to slow down.

The idea is almost embarrassingly simple, and it is the one most systems get wrong: the fast can only go as fast as the slow will allow. A producer that outruns its consumer does not actually accomplish more work; it just piles the difference somewhere, and where it piles up decides whether your system degrades gracefully or falls over. Backpressure is how the slow part says so — how a consumer's real, current capacity is fed back up the line so the producer matches it, instead of being discovered the hard way when memory runs out.

The first thing everyone reaches for when a consumer can't keep up is a queue. Put a buffer between the fast producer and the slow consumer, and the spikes smooth out. This works, and it is genuinely one of the most useful moves in systems design, which is exactly why it is dangerous: it works right up until it doesn't, and then it fails in two nasty ways at once.

A queue can absorb a burst — a brief surge where arrivals temporarily outrun service, covered in Message Queues: Absorbing the Burst. What a queue cannot absorb is a trend. If the producer is sustainably faster than the consumer, no buffer size saves you; a bigger buffer only delays the reckoning. An unbounded queue is the purest form of this mistake. It is a silent promise to process everything eventually, and under sustained overload that is a promise the system cannot keep. It grows until the process runs out of memory and dies, taking the whole queue with it.

But the crash is not even the worst part, because there is a quieter failure that hits long before the memory does. A full buffer is pure latency. By Little's Law, the time an item spends waiting in a queue equals the queue's depth divided by the rate it drains. A queue that sits ten thousand items deep, drained at a thousand a second, is adding ten seconds of pure waiting to every single item that enters it, whether or not memory is anywhere near full. This is the same disease the networking world named bufferbloat: oversized buffers in routers that don't add throughput at all, they just add delay, because a packet's job now includes sitting in a long line. The buffer is where latency hides. Making it bigger doesn't make the system faster; it makes the system slower to notice it is behind.

So the buffer must be bounded — the same lesson as the wall in Bulkheads, where the bound was the whole point. And the instant you bound it, you are forced to answer the question the unbounded queue was letting you dodge: when the buffer is full, what happens to the next thing that arrives?

A full bounded queue leaves you exactly three options, and every backpressure design in existence is some combination of them.

Block the producer. Make it wait until there is room. This is the bluntest form of true backpressure: the consumer's full queue physically stops the producer, because the producer is stuck trying to enqueue. Nothing is lost, which is why it is the right default for work that must never be dropped. The cost is that the stall travels backward up the line, and if your stages form a cycle — A waiting on B while B waits on A — blocking can deadlock the whole thing. Blocking backpressure has to be bounded in time, never infinite.

Drop the work. Throw something away so the queue stays healthy. The system keeps moving and stays responsive, at the cost of losing data. Which thing you drop is a real decision: for a queue of fresh user requests you usually drop the newest (the old ones are already half-served), while for a stream of sensor readings or metrics you drop the oldest, because the latest value is the one that matters. Dropping is really load shedding, and it gets its own lesson next.

Signal upstream. Send the producer an explicit message — slow down, I'm at capacity — and let it lower its own rate at the source. This is backpressure in its fullest sense: no data lost, no thread stalled, the production rate simply drops to match what the consumer can handle. The catch is that it only works if the producer can be slowed, and if the signal actually reaches it.

A clean way to hold the three in your head: block is backpressure inside a process (a full in-memory queue stops the thread filling it), signal is backpressure across a network (where you can't hold a socket open forever, so you send word instead), and drop is what you do when you have run out of room to push back at all. Most real systems use all three at different boundaries.

Before you can signal a producer to slow down, one architectural choice decides how hard that will be, and it is the choice most people never make on purpose: does the producer push, or does the consumer pull?

In a push system, the producer sends whenever it has something, on its own schedule. It is the natural, obvious design, and it has a built-in flaw: the producer sets the pace, so a fast producer can bury a slow consumer with no natural brake. The classic footgun lives in message brokers — the two shapes you met in Queue Models: Broker (RabbitMQ) vs Log (Kafka). A broker that pushes with no prefetch limit will keep shoving messages at a consumer as fast as the network allows; the consumer, unable to process them that fast, buffers them in its own memory, and you get the unbounded-queue death from the last section relocated into the consumer, sometimes millions of unacknowledged messages deep. The fix is to cap the prefetch: never hand a consumer more unacknowledged work than a small fixed count, which is a credit limit by another name.

In a pull system, the consumer asks for work only when it is ready for more. This flips the flaw into a feature: the pace is set by whoever is slower, automatically, because the fast side simply waits to be asked. A log-style consumer polls for a batch, processes it, and polls again — and if it falls behind, it can pause and stop asking until it catches up, its growing lag the plainest backpressure signal there is. Reactive Streams turned this into a standard: the consumer calls request(n) to announce how many items it is ready for, and the producer is forbidden from sending even one more than that. Demand flows up, data flows down, and unbounded buffers become impossible by construction. Pull gives you backpressure for free; push makes you bolt it on.

There is one mechanism underneath request(n), the prefetch cap, and half the protocols you use every day, and it is worth seeing plainly because once you have it, backpressure across a network stops being mysterious. It is credit.

The accounting is tiny. The receiver grants the sender a number of credits equal to how much free buffer room it has. The sender spends one credit for each unit it sends and stops dead when its credits reach zero. As the receiver drains its buffer, it mails credits back, and the sender can send again. That's the whole protocol, and it has a guarantee blocking and dropping can't offer: the sender never transmits something the receiver has no room for, so the receive buffer can't overflow and the network in between isn't holding a backlog. It is backpressure expressed as a budget you are not allowed to overspend.

Once you see it, you see it everywhere, and you realize it is old. TCP has done this since 1981. Every TCP receiver advertises a receive window — the number of bytes it currently has room for — and the sender may not have more than that in flight. A receiver whose buffer is full advertises a zero window, which means stop, and sends a window update when it drains, which means go again. Crucially this is a separate control from congestion control: the receive window protects the receiver's buffer, the congestion window protects the network from overload, and a sender is bound by the smaller of the two. Confusing the two is one of the most common networking mistakes; flow control is about not drowning your peer, congestion control is about not drowning the road between you.

HTTP/2 and gRPC do the identical thing one layer up, and even use the word: credit-based WINDOW_UPDATE frames, with a starting window of 65,535 bytes for each stream and for the whole connection, topped back up when it drops below half. Reactive Streams' request(n) is the same idea with a different name. Different letters, one accounting rule: never send more than the receiver has told you it can hold.

The whole argument is easiest to believe when you can watch a queue misbehave and then fix it with a policy. Below is one production line: a producer filling a bounded queue that a slower consumer drains, with every rate in your hands.

Start by leaving the queue effectively unbounded and setting the producer faster than the consumer. Watch the queue swell and, more tellingly, watch the latency gauge climb right along with it — the buffer converting straight into waiting — until the whole thing tips over into an out-of-memory stop. Now switch the policy to drop and run it again: the queue stays shallow, the latency stays flat, and a loss counter ticks up every time work is thrown on the floor. Finally switch to signal and watch a slow-down arrow flow back up to the producer and clamp its effective rate to the consumer's; the queue settles low, latency stays flat, and nothing is lost. The line now runs at the speed of its slowest stage, which is the fastest it could ever safely run. Push the rates around and try to make it fall over. Under signal, you can't.

Backpressure is not a single valve; it is a chain. A slow stage signals the stage feeding it, which slows and signals the stage feeding it, and the calm propagates back toward the source one link at a time. That chain has two hard truths that decide whether it saves you.

The first: the signal has to reach the source. If any stage in the middle happily accepts everything and buffers it — the one component nobody bothered to bound — the pressure stops propagating and simply piles up there, and that naive middle stage becomes the thing that falls over. Backpressure only works if every link honors it; one unbounded queue anywhere breaks the whole chain.

The second is the honest limit of the whole idea: the ultimate source is often someone who will not slow down because you asked. You can push back on your own services all the way to the edge, but the edge is a phone in a stranger's hand, and it has no request(n) and honors no window. The most you can send a stranger is a polite signal — an HTTP 429 Too Many Requests with a Retry-After — and hope they respect it. Many clients don't, and a badly built one will answer your slow down by retrying harder, which is the retry storm from Retries, Budgets & Retry Storms aimed straight at the wall you just built. When the source cannot or will not be slowed, you have run out of room to push back, and exactly one move remains: stop trying to serve everything, and deliberately drop the least important work so the rest survives.

That move — choosing what to sacrifice when you can't make the sender wait — is the last tool in the overload kit, and the subject of the next lesson: Load Shedding & Graceful Degradation.

• Backpressure is a feedback signal, not a buffer. It matches the producer's rate to the consumer's real, current capacity by telling the producer to slow down. Rate limiting, from Rate Limiting: Token Bucket, Leaky Bucket, Windows, imposes a fixed ceiling you guessed in advance; backpressure reports the capacity you actually have right now.
• Unbounded queues are a lie. They absorb a burst but never a trend, and a big buffer doesn't add throughput, it adds latency (Little's Law) until it finally runs the process out of memory. Bound every queue.
• A full bounded queue gives three choices: block the producer (no loss, can deadlock), drop the work (load shedding, loses data), or signal upstream (true backpressure, only if the producer can be slowed).
• Pull beats push for flow control. A consumer that asks for work with request(n) or a polled batch sets the pace automatically; a pushing producer needs a prefetch cap or it buries the consumer in memory.
• Credit is the universal mechanism. The receiver grants credits equal to its free room; the sender spends and stops at zero. TCP's receive window (since 1981), HTTP/2's 65,535-byte WINDOW_UPDATE, and Reactive Streams' request(n) are one idea in three costumes. Keep flow control (protect the receiver) separate from congestion control (protect the network).
• Backpressure runs out at the edge. It must reach the source to work, and the ultimate source — a user on the open internet — can't be forced to slow. When pushing back is no longer possible, the only move left is to shed load on purpose: next.