Polling & Long-Polling

You've just paid. The screen says "Preparing your order…" with a little spinner, and somewhere in a warehouse a machine finishes your print job. At that exact moment, the server knows. Your browser doesn't. And here's the uncomfortable fact this whole section of the course grows out of: HTTP gives the server no way to tell you.

Every conversation you've built across the last two sections — every REST call, every GraphQL query, every gateway hop — started the same way: the client asks, the server answers. That's not a convention, it's the protocol's shape. A server holding fresh news cannot dial your browser. It has no address for you, no open line, no mouth. The request-and-response world we just finished perfecting in the APIs section has a built-in silence, and real products live inside it: chat messages, order statuses, live scores, "your driver is arriving" — all news that is born on the server's side of the line.

So §7 opens with the only move HTTP actually permits, the move every engineer reaches for first and every real system still uses somewhere: if the server can't speak, ask it again. This lesson is about the two honest ways to do that — asking repeatedly, and asking patiently — and the dial between staleness and waste that you cannot escape, only choose a point on. Everything else in this section (sockets, push, queues, events) is an increasingly clever escape from that dial. You have to feel the dial first.

The obvious version first. The client sets a timer and asks on schedule:

setInterval(async () => {
  const res = await fetch('/orders/42/status');
  render(await res.json());
}, 5000);   // ask every 5 seconds

Every 5 seconds, a full HTTP request: headers out (a few hundred bytes — cookies, auth token, the works), the server looks up the status, headers back, a body that — let's be honest — almost always says nothing changed. This is short polling, and before we price it, give it its due: it is gloriously, productively dumb. No connection to babysit, no server-side state, works through every proxy and firewall on Earth, trivially load-balanced (each poll is an independent stateless request — §5's favorite kind), and you can build it in one line. There are places where this is the right answer, and we'll name them.

But now watch it under load, because the costs compound in two independent directions:

The staleness cost. An event that happens right after a poll waits the full interval to be discovered. On average, news is interval ÷ 2 old when you learn it; worst case, a full interval. Poll every 30 seconds and your "live" order page runs up to 30 seconds behind reality.

The waste cost. 10,000 users on that order page, each polling every 5 seconds, is a standing load of 2,000 requests per second — for a status that might change ten times an hour. Do the arithmetic on that: ~99.99% of those requests return empty. Nothing. You're paying full HTTP round trips — parse the headers, check the auth, hit the cache, serialize the nothing — millions of times a day, to deliver news that almost never exists. One client polling every second is 86,400 requests a day, per client.

Here's what makes short polling a genuinely deep topic rather than a toy: those two costs pull in exactly opposite directions, and the polling interval is the dial between them.

Halve the interval → average staleness halves → the request rate doubles. Double the interval → the bill halves → your users see stale data twice as long. There is no clever value. There is no formula that spits out "the right interval." At 1 second you're fresh and on fire; at 60 seconds you're cheap and lying to your users. Every choice is a compromise, and — this is the part that bites in production — the compromise that felt fine at 100 users becomes the outage at 100,000, because the waste scales with the fleet while the freshness doesn't improve at all.

Hold that feeling. The entire rest of this section — long polling in a moment, then WebSockets, server-sent events, webhooks, queues — is engineering trying to get off this dial. When someone asks you "why do we need WebSockets when polling works?", this dial is the answer, and you're about to watch it move.

Below is the dial, live. You control a fleet of clients (type your own fleet size — go on, put in a real number), the server's event rate, and the asking strategy; the sim shows every request flying, every empty response wasted, every event sitting on the server accruing staleness until a poll finally discovers it.

Do these four things, in order. First, on short polling, fire one event by hand right after a poll leaves — watch it sit there, orphaned, while the staleness clock runs, until the next scheduled ask finally picks it up. That wait is interval ÷ 2 on average, and now you've seen it. Second, drag the interval down to fix the staleness — freshness improves, and the request counter and the wasted-% meter detonate. That's the dial. Third, flip to long polling with the same settings — the same hand-fired event now comes back the instant it happens, the request rate collapses to roughly one per event, and the held-connections gauge quietly fills with one open socket per waiting client: the new bill. Fourth — the trap — crank the event rate up and watch long polling degenerate in front of you: every hold returns immediately, the request rate climbs right back to short polling's, and the sim names what you're seeing. Then hit restart the server and meet the thundering herd; the jitter toggle is the fix.

Now the trick that carried the internet's real-time features for a decade. The client asks the same question — but the server, instead of answering "nothing yet" immediately, holds the question open:

1. Client: GET /orders/42/status?since=v7 — "tell me when something's newer than v7."
2. Server: …no answer. The request parks. One second, five, fifteen — the connection just stays open, waiting.
3. The print job finishes. The server grabs every parked request for that order and answers them at that instant.
4. The client processes the news and immediately asks again, opening the next hold. If nothing happens for ~20–30 seconds, the server answers "nothing yet" anyway (the timeout), and the client re-asks. The loop never leaves the server without a parked question to answer.

Look at what this buys. Latency collapses to ~zero: news is delivered when it happens, not when a timer fires — no interval ÷ 2 tax, no dial position to agonize over. Waste collapses too: empty responses only happen at the timeout boundary, so when events are rare, the request rate drops from clients ÷ interval to roughly one request per event. Same HTTP, same proxies, same load balancers, no new protocol — and the graveyard-quiet feed that was costing you 2,000 req/s now costs you almost nothing.

It's still pull — the client still initiates everything, and each delivery still spends a full round trip re-asking. But it's pull wearing push's clothes, and it's good enough that (as you're about to see) some of the biggest systems on Earth still run on it by choice.

Nothing in this course is free, so here's long polling's bill — four line items, each one a real outage somebody had:

• One open connection per waiting client. 50,000 users on the page = 50,000 parked requests, held simultaneously. Remember the server anatomy lesson: if your stack burns a thread per in-flight request, each parked poll holds a thread hostage for 20 seconds doing nothing — long polling is how thread-per-request servers discover their ceiling. Event-loop servers park a held request for the cost of a few kilobytes, which is why this pattern and Node grew up together.
• Every box between you and the client has an idle-timeout opinion. Load balancers, reverse proxies, corporate middleboxes, mobile NATs — each will kill a connection that looks "idle," and a held poll looks exactly like idle. Every hop's timeout must exceed your hold time, which is the reason holds are 20–30 seconds and not five minutes. (This is also your first taste of a §7 theme: the infrastructure was built for request-response, and everything real-time fights it.)
• The thundering herd. Deploy, restart, or blip the server, and every client's held request dies at the same moment — and every client re-asks at the same moment. 50,000 simultaneous reconnects is a self-inflicted load spike precisely when the server is coldest. The fix is boring and mandatory: jitter — each client waits a random few hundred milliseconds before re-asking, smearing the stampede.
• The gap between polls. Between receiving an answer and opening the next hold there's a few-millisecond window where an event can land — on nobody's held request. Lose it and the client just… never hears. So real long-poll APIs are cursored: the client asks "everything since v7," and the server keeps a short buffer. Recognize the move? It's the pagination lesson's cursor, back for a second job.

If you think long polling is a legacy browser trick, here's the modern counter-example hiding in plain sight: Amazon SQS, the queue service half the internet's backends sit on, and its single most consequential setting — WaitTimeSeconds.

An SQS consumer is just a poller: it calls ReceiveMessage in a loop asking "anything for me?" With WaitTimeSeconds=0 — short polling — every call returns immediately, and AWS's docs admit a sharper wrinkle: each short poll samples only a subset of the queue's servers, so it can return empty even when messages exist. And AWS bills per request — including the empty ones. A quiet queue polled in a tight loop is a machine for converting silence into invoices.

Set WaitTimeSeconds=20 and the call becomes a long poll: SQS holds your question for up to 20 seconds, checks all its servers, and answers the moment a message lands. Empty responses only at the timeout. That one-line change typically cuts empty receives by 50–90% — and when Netflix combined 20-second long polling with maximum batch sizes across a workload of roughly a billion messages a day, their SQS bill dropped 73%. One parameter.

That's the honest status of this "old" pattern in 2026: it's not how your browser gets chat messages anymore — but it is exactly how servers consume queues at planetary scale, how Kubernetes controllers watch for changes, and how every polling SDK you've ever used quietly behaves under the hood. Asking patiently is a production pattern, not a fossil.

One short story, because it explains why this pattern refuses to die. In the mid-2000s, Gmail's chat and (later) Facebook's messenger had a problem: browsers had nothing for server push — WebSockets wouldn't be standardized until 2011. The workaround was long polling dressed up under the name Comet: the browser held an open request, the server answered when a message arrived, repeat forever. Billions of chat messages moved this way for years, over plain HTTP, through every corporate proxy and hotel firewall on Earth — precisely because it looked like ordinary web traffic.

That superpower never went away. Long polling still works anywhere HTTP works, no protocol upgrade, no middlebox negotiations — which is why real-time libraries like Socket.IO keep it as the automatic fallback when a fancy socket can't get through. The next two lessons give the server a genuine mouth; this pattern is what everything falls back to when the mouth is taped shut.

The decision, honestly framed:

• Short poll when updates are infrequent and nobody needs to-the-second freshness: a background job's status, an internal dashboard, a nightly-changing config. Its dumbness is the feature — stateless, cacheable, unbreakable, done in an afternoon. If a 30-second lie hurts nobody, don't build more than this.
• Long poll when you need near-real-time now, on plain HTTP, without new infrastructure: notification feeds at modest scale, the universal fallback path — and above all, server-to-server queue consumption, where it is not the compromise but the standard (SQS's 20 seconds).
• Neither when the conversation is genuinely two-way and busy (chat with typing indicators, multiplayer state, live collaboration). That's the socket's home turf — next lesson.

And now the trap, because it's the sharpest thing in this lesson and most explanations miss it entirely. Long polling's advantage evaporates at high event rates. Its request rate is roughly one per event — brilliant when events are rare. But push the event rate up until events arrive faster than your old polling interval, and every hold returns instantly: the client asks, gets an immediate answer, asks again, gets an immediate answer… you are now short polling at maximum speed, with none of the savings and all of the re-ask overhead — a full HTTP round trip per message. You watched this crossover happen in the sim. Say it the staff-engineer way: long polling buys you latency, not throughput. For rare events it's nearly perfect; for a firehose it degenerates into the thing it replaced — which is precisely the argument the next lesson stands on.

• "Polling is legacy — real systems push." SQS consumers, Kubernetes controllers (the reconcile loop is a poll!), CI status pages, and every Stripe SDK's retry-and-check flow poll today, on purpose. Boring-reliable is a feature. Polling is only wrong when events are frequent or seconds matter.
• "Long polling gives you real-time for free." You pay in a different currency: one held connection per waiting client, idle-timeout landmines at every hop, and a reconnect stampede on every deploy. Cheaper than short polling for rare events — never free.
• "Just poll faster." Halving the interval halves staleness and doubles the standing request rate, forever. The dial has two ends and your finger is on it; at fleet scale, "faster" is an outage with good intentions.
• "Long polling always means fewer requests." Only while events are rare. At high event rates every hold returns immediately and the request rate converges right back to short polling's — the degeneration trap.
• "A held request means the server is working the whole time." A healthy hold is parked, not busy — an event-loop server holds 50,000 of them in a few hundred megabytes. If each hold burns a thread, the problem isn't long polling; it's your server model, and long polling just found it first.

Twenty lines of dependency-free Node give you a real long-poll server and the most satisfying curl you'll run this week. Save as longpoll.js:

// longpoll.js — a complete long-polling server. No dependencies.
const http = require('http');
let waiting = [];                    // parked responses — the "held questions"

http.createServer((req, res) => {
  if (req.method === 'GET' && req.url === '/wait') {
    waiting.push(res);               // park it — nothing burns, the response just… waits
    setTimeout(() => {               // 20s timeout: answer "nothing yet", client re-asks
      if (waiting.includes(res)) {
        waiting = waiting.filter(r => r !== res);
        res.end(JSON.stringify({ event: null, note: 'timeout — ask again' }) + '\n');
      }
    }, 20000);
  } else if (req.method === 'POST' && req.url === '/event') {
    let body = '';
    req.on('data', c => body += c);
    req.on('end', () => {
      waiting.forEach(r => r.end(JSON.stringify({ event: body || 'ping' }) + '\n'));
      res.end(`delivered to ${waiting.length} waiting client(s)\n`);
      waiting = [];
    });
  } else res.end('try GET /wait or POST /event\n');
}).listen(9310, () => console.log('long-poll server on :9310'));

1. Terminal 1: node longpoll.js → you should see long-poll server on :9310.
2. Terminal 2: time curl http://localhost:9310/wait — and watch it hang. That's not broken; that's a held question. Your terminal is now a parked connection on the server.
3. Terminal 3, a few seconds later: curl -X POST -d '"print job 42 finished"' http://localhost:9310/event → you should see delivered to 1 waiting client(s).
4. Look back at Terminal 2 — it returned the instant you fired step 3, with {"event":"\"print job 42 finished\""}, and time reports a total almost exactly equal to however long you made it wait (ours: 3.031 total for a 3-second wait — delivery latency ≈ zero).
5. Run step 2 again and fire nothing. After 20 seconds: {"event":null,"note":"timeout — ask again"} — the empty answer at the timeout boundary, the only waste this pattern pays.
6. Feel the herd: open three terminals on /wait, then fire one event — delivered to 3 waiting client(s), all three return in the same instant. Now imagine 50,000 of them re-asking simultaneously after a restart, and you understand jitter in your fingers, not just your head.

• HTTP can't push. The server may hold the news, but the client owns the conversation — so the first honest real-time tool is asking again.
• Short polling is a dial, not a design. Average staleness = interval ÷ 2; standing cost = fleet ÷ interval. Fresher and cheaper pull in opposite directions, the right value doesn't exist, and the comfortable compromise at 100 users is the outage at 100,000.
• Long polling holds the question open — delivery at event time, empties only at the ~20-second timeout, request rate ≈ one per event. Pull, wearing push's clothes.
• The bill moves; it never vanishes: a held connection per waiting client, idle timeouts to out-wait at every hop, jitter against the thundering herd, and a cursor so the between-polls gap can't eat events.
• It's a production pattern, not a fossil: SQS's WaitTimeSeconds=20 cuts empty receives 50–90% (Netflix: −73% at a billion messages a day), and the Comet era ran the world's chat on it before sockets existed.
• The trap: at high event rates long polling degenerates into short polling — it buys latency, not throughput.

That last sentence is a loaded gun pointed at the next lesson. If a busy feed forces one full HTTP round trip per message even under long polling, the obvious question is: why does the client keep re-asking at all? What if — one handshake, and then the line just… stays open, both directions, forever? That's WebSockets: The Two-Way Street, and it's where the server finally gets a mouth.