API Gateways: The Front Door

Look back at what this section has quietly been stacking up. Contracts need enforcing. Requests need authenticating (two lessons from now, but you knew it was coming). Rate limits need applying — and the rate-limiting lesson explicitly deferred a question: where do the limits live? Versions need shimming (lesson 5's transform layers). Formats need translating — because lesson 7 just left you with REST at the edge and gRPC inside, and something has to convert between them.

Now imagine implementing all of that in a system with forty microservices. Every service grows its own TLS handling, its own token validation, its own rate limiter, its own version shims — forty slightly different implementations of the same policies, forty places for the security patch to miss one. The pattern practically names itself: these are cross-cutting concerns — things every request needs regardless of which service it's for — and copying them into every service is the distributed version of copy-pasting code.

The API gateway is where cross-cutting concerns go to live once. It's the single entrance for API traffic: every request, from every client, walks in through the same door and passes through the same ordered pipeline of checks — TLS, authentication, rate limits, validation — before being routed to whichever internal service it actually wants. Policy stops being forty implementations and becomes infrastructure: configured in one place, enforced on everything, updated in one deploy.

That's the easy half of the lesson. The interesting half is a question that sounds like trivia and turns out to be the whole design: in what order should the pipeline run its checks? Get it wrong and your rate limiter can't count, or attackers farm your CPU for free. Get it right and every hostile request dies at the cheapest possible moment. We'll build that intuition by hand.

First, the confusion that trips up every interview and half of all architecture meetings — because all three of these sit "in front" and forward traffic, and often ship in the same binary.

• A reverse proxy is the generic ancestor: anything that sits in front of servers and forwards requests on their behalf, maybe caching or terminating TLS along the way. The other two are specialized reverse proxies.
• A load balancer (§5, in full) answers one question: "which copy?" It spreads traffic across identical replicas of one service. It doesn't care what the request means — just which of N interchangeable boxes should serve it.
• An API gateway answers a different question: "may you enter — and which service do you need?" It governs traffic across many different services: authenticates it, rate-limits it, transforms it, then routes /orders/* to the orders service and /users/* to the users service. Routing to different destinations by meaning, with policy enforcement on the way — that's the gateway's job description.

And here's the part that resolves the confusion instead of adding to it: they stack. A real request path is: client → gateway (may you enter? which service?) → that service's load balancer (which replica?) → a box. The gateway makes the governance decision; the LB makes the copy decision. The fact that nginx or Envoy can play both roles doesn't merge the roles — it just means one bouncer can also valet-park.

(Where does the gateway itself run? Behind the §5 machinery, of course — DNS, anycast, an L4 balancer spreading across gateway instances. It's servers all the way down; each layer answers its own question.)

Open the gateway up and it's a sequence of chambers — each one a check that either rejects the request (cheaply, immediately) or passes it deeper. A production pipeline looks like this, in order:

1. TLS termination — decrypt once, at the door (§5's L7 lesson). Everything after works with plaintext HTTP.
2. Coarse rate limit (by IP) — before any real work: is this address flooding us? A §5-L6 token bucket keyed on IP. Rejections here cost almost nothing — which, as you're about to see, is the entire point of putting it first.
3. Authentication — who are you? Validate the API key or verify the JWT's signature (how tokens actually work is the next lesson; for today: it's a CPU-costing cryptographic check that yields an identity). No identity → 401, goodbye.
4. Fine rate limit (per user/key) — now that we know who you are, apply your quota: free tier gets 100/min, enterprise gets 10,000. Over → 429 with Retry-After (§5-L6's polite no, finally installed at its enforcement point).
5. Validation & transformation — malformed body? Reject with 400 before a backend wastes time. Rewrite headers, strip internal fields, apply lesson 5's version shims (Stripe's date-pinned transforms live exactly here).
6. Route (+ translate) — path/host → the right internal service, translating protocol if needed: REST/JSON from the browser becomes gRPC to the backend (lesson 7's hybrid, mechanized — this is also where gRPC-Web gets unwrapped).

Responses walk back through the pipeline in reverse (Envoy's filter chains are explicit about this: request filters run A→B→C, response filters C→B→A) — transforming outbound, compressing, logging.

Two properties make this shape powerful. Every rejection is cheap by construction — a denied request never touches a backend, never opens a DB connection, never costs more than the chambers it passed. And every policy exists exactly once — when the JWT library has a CVE, you patch one fleet, not forty services.

Now the question that separates gateway users from gateway designers: why that order? Shuffle the chambers and the gateway silently changes behavior — sometimes catastrophically. Two concrete failures make the whole principle:

Failure #1: per-user rate limit before authentication. The per-user limiter's job is "count requests per identity." Place it before the auth chamber and it has no identity to count — the JWT hasn't been verified, the user is ???. It either counts everything as one anonymous bucket (wrong) or passes everything through (worse). A user hammering you with 10,000 valid-token requests sails past a limiter that literally cannot see them. The fine limiter has a data dependency on auth. Order isn't style; it's dataflow.

Failure #2: authentication before any rate limit. Now flip it — auth first, limits after. Signature verification costs real CPU (that's next lesson's crypto). An attacker who sends garbage tokens at 50,000 requests/second isn't trying to get in — they're making your gateway do expensive cryptography on junk, and every request costs you a verification before the limiter ever sees it. You've built a CPU-burning machine with a polite queue. This is why the coarse IP limit goes before auth: it's nearly free, and it stops floods before they reach anything expensive.

Hence the production shape — cheap and identity-free checks first, expensive checks behind them, identity-dependent checks after identity exists: IP-limit → auth → user-limit → route. This isn't just theory: Kong (one of the most-deployed gateways) hard-codes plugin priorities for exactly this reason — its JWT plugin runs at priority 1005, its rate limiter at 901, so auth wins by default — and Kong Enterprise eventually shipped dynamic plugin ordering because real teams kept needing to re-arrange chambers for cases exactly like the two above. The pipeline's order is configuration in name, architecture in fact.

Order-as-design is a lesson your hands learn faster than your eyes, so here's the architect's chair: four chambers you can arrange in any order, and four kinds of traffic waiting to test your arrangement.

Start with the deliberately broken build: put the per-user limit before auth and fire the user flood — twenty valid-token requests from one over-eager user. Watch the limiter chamber shrug (?? who?) as all twenty sail through to your backends: it can't count what it can't identify. Now move auth ahead of it and re-fire — the flood dies at the user limit, exactly as designed.

Then try auth first, IP limit after and fire the unauthenticated flood: twenty garbage tokens from one IP. Every single one reaches the auth chamber and costs you a signature verification before dying — watch the work meter climb red with CPU you paid for junk. Re-arrange with the IP limit first and re-fire: the flood dies at chamber one, cost ≈ nothing.

When you think you've found the production order, the grader will score it attack-by-attack — where each request type died, and what it cost you before it did. The winning insight isn't the specific sequence; it's the reasoning: cheap before expensive, identity-free before identity-dependent, and every check placed where rejection costs the least.

Beyond checking and routing, gateways can compose. A mobile home screen needing user + orders + recommendations doesn't have to make three calls: the gateway (or a thin service behind it) fans out to all three internally — where latency is sub-millisecond — and returns one stitched response. That's the aggregation pattern, and it's the round-trips fix from the GraphQL lesson without adopting GraphQL: the chattiness moves from the phone's radio to the datacenter's LAN, where it's cheap.

Push that idea further and you meet the pattern SoundCloud and Netflix popularized: Backend for Frontend (BFF). Instead of one gateway serving every client, you run one gateway per client type — an iOS BFF, a web BFF, a partner-API BFF — each shaping responses, aggregations, and policies for exactly its audience. The iOS team owns the iOS BFF and ships freely; the partner BFF stays rigidly versioned per lesson 5. (And if the BFF idea sounds like "a server that lets each client ask for the shape it wants" — yes: GraphQL is often deployed precisely as the BFF, lesson 6 and this lesson shaking hands.)

The BFF also quietly solves a governance problem you'd otherwise discover at 2am: with one shared gateway, every team's routes, plugins, and transforms pile into a single config — one deploy, everyone's blast radius. Splitting by audience (or by domain) restores ownership boundaries. Which brings us to the ways this pattern goes wrong.

• A hop on everything. The gateway adds a millisecond-ish to every request and becomes the throughput ceiling for your whole API surface. Fine — until someone puts something slow in the pipeline. Budget it like the hot path it is.
• The most important SPOF you own. One door for everything means the door is the availability story — §5 L1's lesson, now with maximum stakes. Gateways run as redundant fleets behind L4 balancers, health-checked (§5 L4) like anything else. "Single entrance" describes the logical architecture, never the physical instance count.
• THE GOD-BOX (the anti-pattern that eats companies). The gateway is programmable — plugins, filters, Lua, WASM — and that power attracts logic like a porch light attracts moths. "Just add a plugin that checks inventory before routing." "Just have the gateway compute the discount." Eighteen months later, business rules from six teams live in the gateway config, every product change is a gateway deploy, and a typo in one team's plugin takes down everyone's traffic. You've rebuilt the monolith and given it a hat. The discipline is one sentence: the gateway may care how requests look — never what they mean. Auth, limits, shapes, versions, routing: yes. Pricing, inventory, eligibility: that's a service, and the gateway routes to it.
• Config is a contract. Forty services' routes and policies in one place needs code review, ownership, and tests like any other L1 contract artifact — because a routing typo is an outage with a very short stack trace.
• The door doesn't replace locks inside. The gateway is the front door; internal (east-west) traffic never passes through it. Services still authenticate what matters — defense in depth (the zero-trust conversation arrives with the auth lessons, next).

• "An API gateway is basically a load balancer." Different question entirely: the LB picks among identical copies ("which replica?"); the gateway governs and routes among different services ("may you enter, and which service?"). They stack in one request path — gateway first, then each service's LB.
• "It's just a proxy — the order of checks is an implementation detail." Order is the behavior. A per-user limiter before auth cannot count (no identity yet); auth before any limiter donates signature-verification CPU to every attacker; expensive checks early = maximum cost per rejected request. Kong literally encodes this in plugin priorities.
• "It's programmable, so putting logic there is convenient." That's the god-box trap. The test: does this code decide how a request looks (cross-cutting) or what it means (business)? The second kind belongs in a service — or the gateway becomes a monolith wearing a hat, and every team's bug becomes everyone's outage.
• "With a gateway, internal services don't need security." The gateway guards the front door; east-west traffic never sees it. Defense in depth stands — services still verify identity and authorization for anything that matters.
• "One gateway should serve everything." At scale the shared gateway becomes a coordination bottleneck — every team's routes in one deploy. BFFs (per client type) and per-domain gateways restore ownership. 'Single entrance' is per audience, not per company.

The fastest path to gateway intuition is assembling one from parts you already know:

1. Stand up two "services" — the §5-codelab trick: two Python one-liners on :9001 and :9002, one answering users, one orders.
2. Make nginx a gateway, not an LB. Where the codelab's upstream spread traffic across copies, now write two location blocks routing by path: /users/ → :9001, /orders/ → :9002. Curl both paths through :8080 — one door, two different services. That single config difference is the LB-vs-gateway distinction, in your terminal.
3. Add pipeline chambers. Bolt on §5-L6's limit_req (the IP limiter — chamber 2). Then fake cheap auth: if ($http_x_api_key != "secret123") { return 401; } in the location block. Curl with and without the header. You now have TLS-less-but-real chambers: limit → auth → route.
4. Feel the order. Put the auth if inside a location that the rate limit doesn't cover, flood it with bad keys, and watch nginx do per-request work the limiter never sees. Re-order. (In nginx the phases are partly fixed — which is itself the lesson: real gateways like Kong exist partly because plugin ordering needs to be a first-class feature.)
5. Ascend to a real gateway (optional, 20 min): docker run kong (or Tyk/KrakenD), declare the same two routes declaratively, and enable the key-auth + rate-limiting plugins. Read the plugin priority docs and find jwt (1005) vs rate-limiting (901) — the order lesson, encoded by professionals in production software you now know how to read.

The API gateway is where this whole section's policies stop being per-service copy-paste and become infrastructure:

• One entrance, one pipeline: TLS → coarse IP limit → authN → per-user limit → validate/transform → route. Every rejection cheap by construction; every policy defined once, enforced on everything.
• Order is the design: cheap before expensive, identity-free before identity-dependent. A user-limiter before auth can't count; auth before any limiter donates CPU to attackers. Kong's plugin priorities encode exactly this.
• Gateway ≠ LB: the gateway answers "may you enter + which service?", the LB answers "which copy?" — and they stack in every real request path.
• It can compose: aggregation collapses client round-trips; BFFs (one gateway per client type — SoundCloud/Netflix style) restore team ownership; GraphQL often is the BFF.
• The price: a hop, a maximally-important SPOF (run it as a fleet), config-as-contract — and the god-box discipline: the gateway may care how requests look, never what they mean.

One chamber in that pipeline got hand-waved today: "validate the JWT's signature and out pops an identity." How? What's actually inside a token? Why can the gateway trust it without calling anyone? And what's the difference between knowing who you are and knowing what you may do? That's authentication vs authorization — sessions and JWTs — the next lesson, where we crack the token open.