L4 vs L7 Load Balancing (+ Sticky Sessions)

Last lesson we spent the whole time on how a load balancer picks a server — round-robin, least-connections, power-of-two, hashing. But every one of those algorithms quietly assumed something we never questioned: that the balancer is even allowed to look at whatever it needs to make the call.

A plain round-robin dispatcher can work with its eyes closed — request one to A, request two to B, it never needs to know what's in the request. But the moment you want a rule like "send anything under /api to the API servers and anything under /images to the static servers," the balancer has to actually read the request to obey it. And whether it can read the request — or is stuck staring at a sealed envelope — depends entirely on which layer of the network it operates at.

That's the whole lesson. There are two answers, and they have names borrowed from the OSI model: Layer 4 and Layer 7. L4 is fast and blind. L7 is smart and slower. Everything else — content routing, TLS termination, sticky sessions, the gRPC gotcha that trips up half of all first microservice deployments — falls out of that one difference: what can this thing see?

Start at the bottom. A Layer 4 load balancer works down at the transport layer — the world of TCP and UDP. At that layer, a request is just a connection with an envelope on it: a source IP and port, a destination IP and port. That's what L4 can read. Nothing else.

So an L4 balancer does the only thing it can: it picks a backend and forwards the whole connection to it, packets and all, without ever opening them. It's a postal sorting machine reading the ZIP code on the outside of the envelope and shoving it toward the right city — it never reads the letter, and it doesn't need to.

Because it never parses anything, it's fast. Per-connection CPU is almost nothing, so throughput runs close to the raw network line rate and latency sits down in the tens of microseconds — think 50–100 µs, not milliseconds. This is why AWS's Network Load Balancer (NLB) is an L4 device that advertises millions of requests per second at sub-millisecond latency, and why nginx does L4 through its dedicated stream module rather than its normal HTTP path.

The catch is the same thing that makes it fast: it's blind. It cannot route by URL, because it never sees the URL. It can't split /api from /images, can't read a header, can't honor a cookie, can't tell a login from a file upload. To an L4 balancer every request on a connection is the same opaque stream of bytes headed to the same backend. Fast, cheap, protocol-agnostic — and completely unaware of what it's carrying.

Now climb to the top of the stack. A Layer 7 load balancer works at the application layer — it speaks HTTP, HTTP/2, gRPC, WebSocket. And to speak HTTP, it does something L4 never does: it terminates the connection. The client's connection ends at the balancer. The balancer reads the full request — the method, the path, every header, every cookie — decides what to do, and then opens a fresh connection to whichever backend it chose. It's a reverse proxy that actually understands your protocol.

Once it can read the request, it can route on content, and that changes everything:

• GET /api/users → the API pool; GET /images/logo.png → the static/image pool. One hostname, many services, split by path. This is the front door of every microservice architecture.
• Host: tenant-a.app.com vs Host: tenant-b.app.com → different backends. Multi-tenant routing by the Host header.
• Cookie: session=u42 → the specific box holding that user's session (sticky sessions — we'll get there).
• Any header, any method, canary a release to X-Beta: true, block a bad request, rewrite, redirect.

AWS's Application Load Balancer (ALB) is exactly this: it routes on URL, host, and headers, and terminates TLS by default. In nginx, L7 is the normal http block with its location rules. The L7 balancer isn't picking a server despite the request — it's picking a server because of the request. It reads the letter, and routes on what the letter says.

Once a balancer can read the request, a whole toolbox opens up that's simply impossible at L4:

• Content-based routing — path, host, header, method. One entry point fanning out to dozens of services. This alone is why most HTTP systems live behind L7.
• Central TLS termination — decrypt once, at the edge, with certificates managed in one place; backends can then speak plain HTTP internally (or re-encrypt for zero-trust). L4 can pass TLS straight through, but it can't use what's inside.
• A security seam — a Web Application Firewall can only block a malicious request if something reads it. That reader is L7.
• Compression, redirects, retries, header rewrites, canary and blue-green by header, and real per-request observability — status codes, paths, and latencies, instead of L4's raw byte counts.

And now the trap that catches nearly everyone's first gRPC deployment — worth burning into memory:

gRPC runs over HTTP/2, and a single HTTP/2 connection multiplexes many requests (streams) over one long-lived pipe. An L4 balancer sees one connection and pins it — and therefore all of its streams — to a single backend. Your ten backend replicas sit idle while one of them takes every gRPC call on that connection. The load "balancer" balanced nothing.

The fix is L7, which understands HTTP/2 and can balance per stream instead of per connection. The same trap springs on any long-lived multiplexed connection. It's the cleanest example of the whole lesson: L4 isn't "worse," it's blind to a structure that matters here, and blindness has consequences.

This is the part you have to watch, because "what the balancer can see" is invisible by definition — and the gap between a sealed envelope and an open one is the entire lesson. So below is one client, one load balancer, and two backend pools, and you drive it.

Leave the balancer on L4 and send a few requests. Notice the balancer only ever shows you the envelope — TCP 10.2.3.4:51997 → backend — and every request rides the same connection to the same box, whether you asked for /api/users or /images/logo.png. It can't split them; it never saw the path. Now flip it to L7. The same envelope opens: the request line, the Host header, and the cookie appear, and watch the routing change in real time — /api/* peels off to the API pool, /images/* to the static pool, exactly because the balancer can finally read them.

Then push on it. Edit the path or add a cookie yourself. Turn sticky sessions on and watch a cookie'd user get pinned to one backend on every request — then kill that backend and watch the session evaporate even though the balancer is perfectly healthy. That last click is the sticky-session trade-off made physical, and it's exactly where the next two sections go.

If L7 can do everything L4 can plus route on content, why would anyone still choose L4? Because reading isn't free, and the bill comes due on the hot path.

• Latency. L4 forwards in microseconds. L7 has to parse the HTTP, apply its rules, and usually terminate TLS — that's real work on every single request, adding on the order of 1–5 ms. For most web apps that's noise. For a high-frequency trading feed or a firehose of tiny packets, it's a tax you refuse to pay.
• Throughput and CPU. Parsing and crypto burn cycles, so an L7 box handles far fewer requests per core than an L4 box moving packets at line rate. You pay for L7 in machines.
• It's a heavier chokepoint. Because L7 terminates connections, it holds real per-connection state, which makes its own high availability more delicate than a stateless packet forwarder's. Remember the very first lesson: the front door is a single point of failure, and a stateful front door is a stickier one to make redundant.
• It's a plaintext boundary. Terminating TLS means your requests are decrypted at the balancer. That's a genuine security surface — anyone who owns the L7 box sees cleartext — which is why zero-trust setups re-encrypt (mTLS) on the hop from balancer to backend.

None of this makes L7 "bad." It makes L7 a deliberate choice: you're spending latency, CPU, and a security boundary to buy intelligence. When you don't need the intelligence, L4 hands you speed for free.

We keep mentioning cookies and pinning, so let's meet the feature head-on. A sticky session (a.k.a. session affinity) is a rule that says: every request from this one user goes to the same backend for the life of their session, instead of being freely balanced across the pool.

Why would you ever want to defeat your own load balancer like that? Because of where the session lives. If a backend keeps a user's state — their login, their shopping cart, a half-finished form — in its own memory, then bouncing that user to a different box on the next request means the new box has never heard of them. The cart is empty. The login is gone. Pinning the user to the box that remembers them is the quick fix.

There are two ways to do the pinning, and they map cleanly onto the two layers:

• Cookie-based (L7). The balancer sets its own cookie (or reads one your app already sets) that names the chosen backend, and every future request carrying that cookie is routed straight back to it. It's precise — one user, one pin — and it's the standard for browser apps, because browsers store and resend cookies automatically.
• Source-IP hash (L4). No cookie needed — just hash(client_IP) → backend, so the same IP keeps landing on the same box. Cheap, but crude: an entire office or a mobile carrier can sit behind one shared NAT'd IP, so thousands of users collapse onto a single backend and turn it into a hotspot — and the mapping reshuffles whenever the pool changes.

Sticky sessions genuinely solve the immediate problem. The trouble is what they quietly cost you — which is the next section, and it's the part that separates people who reach for stickiness from people who know when to walk away from it.

Turn stickiness on and you've made your fleet a little bit brittle in four specific ways. Know them cold, because "just enable sticky sessions" is one of the most common and most quietly damaging shortcuts in system design.

• The load goes lopsided again. A pinned user can't be moved, so all your clever balancing from last lesson stops applying to them. One heavy user — or one giant NAT'd office IP — nails a single box, and least-connections can't rescue a flow it's not allowed to re-route. You defeated the load balancer on purpose; don't be surprised when things stop balancing.
• A dead box takes live sessions with it. If the pinned backend crashes and the session lived only in its RAM, that user is logged out and their cart is gone — even though the load balancer is perfectly healthy. You reintroduced exactly the single-point-of-failure fragility the pool existed to remove.
• Autoscaling stops working. Spin up three fresh servers under a load spike and… almost nothing moves to them, because existing users are pinned elsewhere and only brand-new sessions get to try the new boxes. You added capacity the traffic can't reach.
• And the one that matters most: sticky sessions don't make your app stateless — they only hide the statefulness behind routing. The state is still trapped in one server's memory; you've just papered a routing rule over it.

So here's the honest framing. The real cure is the one from the Stateful vs. Stateless lesson: externalize the session into a shared store — Redis, a database — so any box can serve any request and the load balancer is free to balance again. Sticky sessions are a bridge to buy time when you can't do that yet — a legacy app, a fast deadline. Use them knowingly, and plan the exit. A bridge you never leave is just a longer fall.

Strip the names away and the choice is one question: do you need to route on what's inside the request, or just move bytes fast? Here's the whole decision:

Your situation	Reach for
Plain HTTP/HTTPS app, microservice routing, TLS/WAF/canary, cookies	L7 — the smarts almost always beat the +ms
Non-HTTP protocol, raw throughput, ultra-low latency, pass-through TLS	L4 — speed and simplicity, no parsing
gRPC / HTTP-2 / any long-lived multiplexed connection	L7 — L4 pins all streams to one backend
Need static IPs + DDoS scrubbing and rich content routing	Both, stacked

The default for a normal web service is simple: start at L7. You almost certainly want path routing, one place for TLS, and per-request visibility, and a couple of milliseconds is a rounding error against a database call. Drop to L4 deliberately — when the protocol isn't HTTP, when you're chasing microseconds, or when you truly just need to forward packets and let the backend own TLS.

And the two aren't rivals — the grown-up answer is often both, in series. A common production pattern puts an L4 NLB out front (static IPs, brutal packet-moving speed, a first line against volumetric DDoS) feeding an L7 ALB behind it (all the content-aware routing). AWS explicitly supports this NLB-in-front-of-ALB shape. You get L4's speed at the edge and L7's brains where the routing decisions live. Layers stack; that's the whole point of layers.

Four beliefs that sound right and quietly cost people:

• "L7 is just the newer, better one — use it for everything." No. L4 wins outright on raw speed, non-HTTP protocols, and pass-through, and it stays a lean stateless forwarder. L7 buys intelligence with latency, CPU, a stateful chokepoint, and a plaintext boundary. Match the layer to the job, don't cargo-cult the higher number.
• "L4 can't do TLS." It can pass TLS through untouched, and it can even terminate it — but terminating TLS still doesn't let it read HTTP, so it still can't route by path, header, or cookie. Seeing the encrypted bytes and understanding the request are two different things.
• "Sticky sessions mean stateful backends are fine now." They aren't fine — they're hidden. Stickiness pins the problem in place: lopsided load, sessions that die with a box, autoscaling that can't reach users. It's a bridge to externalizing state, not a substitute for it.
• "Just put everything on L7 with stickiness on — one box, all features." That quietly rebuilds the exact single-point-of-failure and imbalance the whole load-balancing chapter exists to kill. Every feature you switch on at L7 has a price on the hot path; turn on what you need, and know what each one costs.

Fifteen minutes and you'll feel the difference between an envelope and a letter, on real processes.

1. Stand up two tiny backends that announce who they are — one that replies "API: <hostname>" and one that replies "STATIC: <hostname>". Two python -m http.server shims or two one-line Flask/Express apps are plenty.
2. Do L7 first, in nginx. In an http block, put both backends behind one server and add two location rules — location /api { proxy_pass http://api_backend; } and location /images { proxy_pass http://static_backend; }. Hit /api/users and /images/logo.png and watch them land on different backends. That split is L7 reading the path.
3. Now go L4 and watch the smarts vanish. Move the same backends into nginx's stream block (proxy_pass on a TCP port, no location, no paths — stream can't parse HTTP). Point your requests at it and notice you cannot express "send /api there and /images here" at all; every request on a connection just rides to whichever backend the L4 rule picked. Same backends, but the routing tool is blind.
4. Feel stickiness. Back in the http block, add ip_hash; (or an ALB target group with stickiness on) and watch one client stick to one backend across many requests. Then stop that backend and hit it again — observe the session/pin break even though nginx itself is fine.

The managed-cloud version is the same story in one console: an ALB with path-based rules routes /api and /images to different target groups; an NLB simply can't express that. The lesson isn't the config syntax — it's the muscle memory of watching L7 read what L4 can't even see.

A load balancer's power is bounded by what it can see, and what it can see is set by the layer it works at:

• L4 (transport) reads only the envelope — IP and port — and forwards the whole connection blind. Microsecond-fast, near line-rate, protocol-agnostic; the right call for non-HTTP, raw throughput, and pass-through TLS. It's also why plain L4 quietly breaks gRPC.
• L7 (application) terminates the connection and reads the letter — path, host, headers, cookies — to route on content, terminate TLS, run a WAF, and give you real observability. You pay for it in a few milliseconds, more CPU, a stateful chokepoint, and a plaintext boundary. It's the default for HTTP.
• Sticky sessions pin a user to one box (an L7 cookie or an L4 IP-hash) so an in-memory session stays reachable — but they unbalance the fleet, die with the box, and break autoscaling. They hide statefulness rather than removing it; the real fix is to externalize state, and stickiness is a bridge you should plan to leave.
• They stack. L4 out front for speed and DDoS, L7 behind it for brains, is a standard production shape. Layers aren't rivals.

We now have a front door that can be fast or smart, and that can spread load by count or by content. But we've been assuming it always knows which backends are actually alive to receive that traffic. It doesn't, not for free — a balancer routing to a dead box is worse than no balancer at all. How the load balancer detects failure and pulls a sick server out of rotation — health checks and failover — is the next lesson.