Codelab: Load-Balance Two Servers with nginx

Six lessons of theory end here. In the next fifteen minutes you will stand up two real web servers, put real nginx in front of them, and watch — with your own eyes, in your own terminal — every idea this section taught: round-robin rotation, an algorithm swap, passive health checks quietly routing around a dead server, a rate limiter turning a flood into fast 429s, and the one thing a load balancer can never do.

Nothing here is simulated and nothing is skipped: every command below was run exactly as written, and every "you should see" block is the real output. When you're done you'll have a complete, disposable load-balancing lab — and the muscle memory that turns §5 from something you read into something you've operated.

The shape of what you're about to build:

you (curl) ──► nginx :8080 ──► server A :9001  ("hello from A")
                    └────────► server B :9002  ("hello from B")

One door, two backends, and you holding the kill switch.

You need exactly three things, and you almost certainly have two of them:

• python3 — for the two toy backends (any 3.x; check with python3 --version).
• curl — to fire requests (preinstalled on macOS and nearly every Linux).
• nginx — the star. One line:

# macOS
brew install nginx

# Debian / Ubuntu
sudo apt install nginx

One note for Linux folks: your package manager may auto-start a system nginx on port 80. We won't touch it — this lab runs its own nginx, from its own folder, on port 8080, with its own config and logs. Nothing you do here can hurt a real setup, and the last step deletes all of it.

Make the lab folder and step inside — every command in this codelab runs from here:

mkdir lb-lab && cd lb-lab && mkdir logs

A load-balancing lab is useless if you can't tell which server answered. So our backends have exactly one feature: they say their own name. Save this as server.py:

import http.server, sys
PORT, NAME = int(sys.argv[1]), sys.argv[2]
class H(http.server.BaseHTTPRequestHandler):
    def do_GET(self):
        self.send_response(200); self.end_headers()
        self.wfile.write(f"hello from {NAME}\n".encode())
    def log_message(self, *args): pass
http.server.HTTPServer(("127.0.0.1", PORT), H).serve_forever()

Start two of them in the background — same code, different port and name:

python3 server.py 9001 A &
python3 server.py 9002 B &

Now talk to each one directly, no load balancer yet:

curl localhost:9001
curl localhost:9002

You should see:

hello from A
hello from B

Two servers, two identities. Right now a client would have to choose which port to call — which is exactly the problem a load balancer exists to take off the client's hands.

Time for the front door. Save this as lb.conf — it is the whole load balancer, twenty lines:

error_log logs/error.log;
pid       nginx.pid;

events {}
http {
    access_log off;

    upstream backends {
        server 127.0.0.1:9001;
        server 127.0.0.1:9002;
    }
    server {
        listen 8080;
        location / {
            proxy_pass http://backends;
        }
    }
}

Read it once, because you already know every piece from the lessons. The upstream block is the pool — the roster of backends from the health-checks lesson. proxy_pass makes nginx a reverse proxy: it terminates your connection, picks a backend, and opens its own connection — that's an L7 balancer (it's reading HTTP), exactly the "reader" from the L4-vs-L7 lesson. And because we named no algorithm, we get the default the algorithms lesson promised: round-robin.

Check the syntax, then start it (the -p "$PWD" keeps every path — logs, pid — inside this folder):

nginx -t -c "$PWD/lb.conf" -p "$PWD"

You should see:

nginx: the configuration file .../lb-lab/lb.conf syntax is ok
nginx: configuration file .../lb-lab/lb.conf test is successful

nginx -c "$PWD/lb.conf" -p "$PWD"

No output means it's running. One door, port 8080, two backends behind it.

The moment of truth. Ask the door six times — not the servers:

for i in 1 2 3 4 5 6; do curl -s localhost:8080; done

You should see:

hello from A
hello from B
hello from A
hello from B
hello from A
hello from B

There it is — round-robin, live: strict rotation, request 1 to A, request 2 to B, around and around, exactly the "honest default" from the algorithms lesson. The client called one address six times and never knew there were two servers; nginx made the choice on every single request.

It's worth pausing on how boring this looks — perfectly alternating names — because that boring alternation is the entire §5 story working: single entry point, pool behind it, per-request choice. Everything else in this lab is us breaking that calm on purpose.

Now the health-checks lesson, for real. Kill server B mid-flight:

pkill -f "server.py 9002"

B is gone — its port refuses connections. If the client had been calling B directly, every request would now fail. Instead, ask the door again:

for i in 1 2 3 4 5 6; do curl -s localhost:8080; done

You should see:

hello from A
hello from A
hello from A
hello from A
hello from A
hello from A

Six for six, zero errors — and you did nothing. This is the passive health check from the health-checks lesson: nginx tried B, got connection refused, silently retried the request on A (so the client never saw the failure), and marked B as down. Proof it noticed — look at the log:

tail -1 logs/error.log

[error] ... connect() failed (61: Connection refused) while connecting to upstream ...

By default nginx gives a failed server a 10-second timeout before re-trying it (fail_timeout=10s, max_fails=1 — the thresholds-and-hysteresis dial, with real names). So bring B back and wait out the penalty box:

python3 server.py 9002 B &
sleep 11
for i in 1 2 3 4; do curl -s localhost:8080; done

You should see B earn its way back into rotation:

hello from A
hello from A
hello from B
hello from A

Kill a server: traffic reroutes, users never notice. Revive it: it rejoins. You just operated the exact loop the health-checks lesson drew — eject, reroute, re-admit.

Two upgrades in one edit. Replace lb.conf with:

error_log logs/error.log;
pid       nginx.pid;

events {}
http {
    access_log off;

    limit_req_zone $binary_remote_addr zone=api:10m rate=5r/s;

    upstream backends {
        least_conn;
        server 127.0.0.1:9001;
        server 127.0.0.1:9002;
    }
    server {
        listen 8080;
        location / {
            limit_req zone=api burst=5;
            limit_req_status 429;
            proxy_pass http://backends;
        }
    }
}

Two changes, both straight out of the lessons. least_conn; — one word — swaps round-robin for least-connections: nginx now routes each request to whichever backend has the fewest in flight, the "stop guessing, start looking" algorithm (with two instant backends you won't see a difference, but under real, uneven load this is the line that stops one server drowning while its neighbor idles). And the limit_req trio is the rate-limiting lesson in three directives: a shared counting zone keyed on client IP at 5 requests/second, a burst of 5 forgiven (that's the bucket), and rejections sent as a proper 429 instead of nginx's default 503.

Reload without dropping a single connection — the graceful config swap:

nginx -s reload -c "$PWD/lb.conf" -p "$PWD"

Here's a trap worth learning on purpose. Try the obvious flood first:

for i in $(seq 1 20); do curl -s -o /dev/null -w "%{http_code} " localhost:8080; done; echo

You should see:

200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200 200

All accepted?! The limiter isn't broken — your flood is too polite. A shell loop runs curls one at a time, and each process takes long enough to spawn that you never actually exceed 5 requests per second. Lesson inside the lesson: sequential clients can't generate concurrent load. To actually flood, fire in parallel:

seq 1 30 | xargs -P 10 -I{} curl -s -o /dev/null -w "%{http_code}\n" localhost:8080 | sort | uniq -c

You should see (numbers may wobble by one or two):

   6 200
  24 429

Now that's a rate limiter. Thirty requests arrived nearly at once; roughly the burst allowance passed (the bucket), and the other twenty-four got an instant, cheap 429 instead of being allowed to pile onto your backends. Feel how fast the 429s come back — that speed is the whole point: rejecting cheaply at the door beats failing expensively in the fleet.

One deliberate disaster, as tradition demands. Kill both backends:

pkill -f "server.py 900"
sleep 2
curl -s -o /dev/null -w "status: %{http_code}\n" localhost:8080

You should see:

status: 502

502 Bad Gateway — the load balancer's way of saying "I'm fine, but there is nobody behind me." nginx is still running, still listening, still perfectly healthy — and completely useless, because a load balancer routes capacity; it cannot create it. When the roster hits zero, the healthiest door on earth is a door to an empty room.

Hold onto this one, because it's the boundary of everything §5 can do for you: balancing, health checks, failover, and rate limiting all arrange your servers' capacity — none of them add any. When the whole pool is gone (or the whole pool is overloaded), the answer is more servers — which is why the next chapters of this course leave the front door and go build what's behind it.

Everything this lab created lives in one folder and two background jobs. Tear it all down:

nginx -s stop -c "$PWD/lb.conf" -p "$PWD"
pkill -f "server.py 900"
cd .. && rm -rf lb-lab

Verify the door is really gone:

curl -s -o /dev/null -w "%{http_code}" --max-time 2 localhost:8080 || echo "port closed ✓"

port closed ✓

No system files touched, no services left running, nothing on your machine remembers this lab existed — except you.

You didn't just run commands — you re-enacted the whole section. The map, step by step:

You did	You watched	The lesson it came from
upstream + proxy_pass	one door, pool behind it, client can't tell	Load Balancers: The Traffic Directors
six curls → A B A B A B	strict rotation by count	LB Algorithms — round-robin, the honest default
nginx terminating + re-proxying HTTP	an L7 balancer reading requests	L4 vs L7 — this was the "reader"
pkill B → six curls, all A, 0 errors	passive check + silent retry + fail_timeout penalty box	Health Checks & Failover — eject, reroute, re-admit
least_conn;	route by live in-flight count	LB Algorithms — stop guessing, start looking
limit_req + parallel flood → 6×200 24×429	burst forgiven, excess rejected fast	Rate Limiting — the bucket and the polite 429
kill both → 502	a healthy door to an empty room	the limit of the whole section — routing ≠ capacity

And one bonus lesson no slide could teach: your first flood failed to flood, because sequential curls can't exceed 5 req/s. Generating concurrent load is a skill of its own — remember xargs -P (or hey/wrk when you want real numbers).

The only §5 lesson you couldn't run on a laptop is Global Load Balancing — GeoDNS and anycast need the actual planet. Everything else: you just did it.

• The gap between theory and operation is about fifteen minutes. Two toy servers, twenty lines of nginx, and every §5 concept becomes something you've watched happen — that's a permanently different kind of knowing.
• The calm is the achievement. A B A B A B looks boring; that boredom is a single entry point, a pool, and a per-request choice all working. Production is this, times ten thousand.
• Failover done right is invisible. B died and no request failed — the error only exists in error.log. If you didn't look, you'd never know. (In production, "looking" is called monitoring, and it's a later chapter for a reason.)
• A rate limiter you haven't flooded is a rate limiter you haven't tested — and a sequential loop is not a flood. xargs -P is the difference between believing your config and having checked it.
• 502 is the section's boundary marker. Load balancing arranges capacity; it never creates it. When the pool is empty, no algorithm, health check, or limiter can save you — only more pool.

That closes the front door — §5 complete. You now command everything between the user and your servers: spreading, reading, healing, planet-scale routing, and polite refusal. The section checkpoint ahead will hand you three scenarios and make you pick the right front-door setup for each. After that, the course walks through the door — into APIs: the language your servers actually speak to the world.