Serverless

In the first week of June 2024, an artists' social app called Cara went from 40,000 users to 650,000. Its API ran on serverless functions, and the platform did exactly what the brochure promises: absorbed a 16× explosion with no capacity meetings, no 3 a.m. fleet-resizing, no outage — peaking at 56 million function invocations a day.

Then the invoice arrived: about $96,000. For the week.

Here is the thing this lesson wants you to see before anything else: those are not two stories. The rescue and the bill are the same property, seen from its two sides. Serverless made Cara's compute perfectly proportional to demand — and proportional means the bill scales with your success as smoothly as the capacity does.

The last two lessons split the deployable and priced the seams. This lesson shrinks the deployable one more step — down to a single function — and hands the servers, the scaling, and the idle to someone else's platform. What you get back is a completely different billing model, and learning when that inversion wins is the whole game.

In this lesson: what serverless actually is, the anatomy of a cold start (it's a boot sequence, not mystery latency), the billing inversion, the crossover — computed from real public prices, in both directions — and a simulator where you race the meter against the fleet under three shapes of load.

Scope: choosing between specific cloud compute products gets its own decision lesson later in this course (EC2 vs EKS vs Lambda: The Decision); containers and Kubernetes have their own section. And the style serverless functions most often plug into — events as the system's spine — is the next lesson (Event-Driven Architecture & P2P).

Watch what this section has been doing to the deployment unit. The monolith shipped everything as one artifact. Microservices cut it into capability-sized pieces. Serverless takes one more step: the unit becomes a single function — one handler, deployed alone, with no process of yours running anywhere until someone calls it.

That last clause is the radical part. In every architecture so far, you kept servers alive — provisioned, patched, scaled, paid for — whether traffic came or not. In serverless (Function-as-a-Service, to give it its precise name), the platform holds a pool of machines, and when a request arrives it finds — or creates — an execution environment for your function, runs it, and eventually reclaims it. Three consequences follow directly:

• Scale-to-zero. No traffic, no environments, no cost. Your function is a file in storage, not a process on a box.
• Scale-per-request. Each environment handles one request at a time. A hundred concurrent requests means the platform stands up a hundred environments — you wrote no autoscaling policy.
• The platform owns the fleet. Patching, capacity, placement, the 3 a.m. page for a dying host — all of it moved across the counter.

What did you trade away? Control over the machine, the ability to keep anything in memory between requests, and — the subject of the next section — the guarantee that your process is already warm when a request shows up.

One vocabulary note before moving on: people also say "serverless" for managed services generally — queues, object stores, databases you don't operate. That wider spectrum is real (much of this course's data machinery lives on it), but this lesson means the compute slice: functions.

"Cold start" gets said like a curse and understood like weather. It's neither — it's a boot sequence with four parts you can name, time, and mostly control.

When a request arrives and no warm environment exists, the platform builds one. On AWS Lambda — worth studying because its internals are public — that means booting a Firecracker microVM: a real, tiny virtual machine, started in under 125 milliseconds. Then the language runtime loads. Then your initialization code runs — imports, SDK clients, database connections — everything outside the handler. Then, finally, the handler serves the request.

Look at where the time actually goes. The microVM is fast and not yours to change. The runtime varies by language: Python and Node cold-start at roughly 200-400 ms at the median; Java historically took seconds to warm its VM — until snapshot-restore (SnapStart) cut it to around 180 ms by booting once and restoring from the snapshot thereafter. But the part with the widest variance is the third: your init code. A function that imports half an SDK and opens four connections at startup digs its own cold-start grave — which is why the platform caps the init phase at ten seconds. Cold starts are mostly won and lost in code you own.

Two more facts complete the model. Warm starts skip everything but the handler — the platform reuses idle environments for subsequent requests, until it reaps them (minutes of idleness, at its discretion; the reaping is why cold starts come back after quiet periods). And the cold start is a tax on the unlucky request: the first one after idle, or the leading edge of a burst that outruns the warm pool. Averages barely notice it. Your p99 finds it immediately — which percentile lens you learned in How Engineers Measure Systems.

Is the cold start a law of nature? No — it's the price of full-VM isolation. Cloudflare Workers makes the opposite trade: it runs code in V8 isolates — the browser engine's sandboxes — which start in under 5 milliseconds, effectively zero because the isolate loads while the TLS handshake is still in flight. The cost of that trade is a narrower world: JavaScript and WASM, tighter limits, no arbitrary binaries. Shrink the isolation and the start shrinks with it. There's no free lunch here, only a menu.

Now the part that changes architecture decisions: what you pay for, and — more precisely — what you stop paying for.

Every fleet you've designed so far bills by capacity: you rent boxes sized for peak, and the meter runs whether requests come or not. Draw your daily load curve inside the rectangle of what you provisioned, and the white space between the curve and the rectangle's ceiling is money spent on silence. The previous lessons treated that as a fact of life.

Serverless bills by work. The unit is exquisitely small: on Lambda's published prices, $0.20 per million requests, plus $0.0000166667 per GB-second — memory times duration, metered per millisecond, for exactly the milliseconds your handler ran. (ARM shaves ~20% off; there's a permanent free tier of a million requests and 400,000 GB-seconds a month that real workloads blow through quickly.)

In the picture, the inversion is one sentence: the fleet pays for the rectangle; serverless pays for the area under the curve.

Say your handler runs 100 ms at 512 MB. One invocation costs $0.0000002 in requests plus 0.05 GB-seconds of compute — about a millionth of a dollar, all in. Nothing runs between requests, so between requests you pay nothing at all. That number feels unanswerable. Whether it actually is depends entirely on how many of those millionths you buy — which is the next section, and the reason this lesson exists.

Run the same API through both billing models, twice, with nothing changed but the load's shape. Handler: 100 ms at 512 MB. Fleet side priced with this course's standing teaching constants — 1,000 requests/second per $70/month box, minimum two for availability (Single Point of Failure explained why two).

Scenario one: a quiet internal API. Averages 2 requests/second — busy at 10 a.m., near-silent overnight and on weekends. That's 5.2 million requests a month:

• Requests: 5.2M × $0.20/M = $1.04
• Compute: 5.2M × 0.1 s × 0.5 GB = 259,200 GB-s → $4.32
• Serverless total: ≈$5.36/month. The two-box HA floor idles at $140/month. The meter is ~26× cheaper — because the fleet's rectangle was almost entirely empty space.

Scenario two: a saturated API. The same handler at a steady 1,000 requests/second, around the clock. Now it's 2.59 billion requests a month:

• Requests: 2,592M × $0.20/M = $518.40
• Compute: 2.59B × 0.1 s × 0.5 GB = 129.6M GB-s → $2,160.00
• Serverless total: ≈$2,678.40/month. The equivalent fleet — two or three boxes running hot — costs $140-210/month. The meter is ~13-19× more expensive — because there is no idle left to avoid paying for.

Same prices. Same handler. Same arithmetic — you can redo every line of it from the public price sheet. The only variable that moved was the shape, and the verdict flipped by a factor of hundreds.

So retire "is serverless cheaper?" as a question. The real question is: what fraction of your provisioned time would a fleet spend idle? High idle share — spiky, bursty, business-hours, event-driven — and the meter wins going away. Saturation — steady, high, 24/7 — and the fleet wins by an order of magnitude. Somewhere between them sits a crossover point, and finding it for your own workload is a five-minute calculation you now know how to do.

The crossover is a number you should find, not memorize. Below, the same API is priced both ways, live, while you control the load.

Try this first: pick the spiky shape at a modest peak and watch the meter demolish the fleet — the fleet must size for a peak that almost never happens. Then switch to steady and drag the peak up to 1,000 req/s: watch the two cost bars cross, and keep going until the fleet is winning by 15×. Now stretch the handler's duration and memory and watch serverless get expensive faster — the meter charges for milliseconds and megabytes; the fleet doesn't care. The readout under the race tracks the cold-start picture for each shape: steady traffic stays warm, while the spiky shape pays init on every burst's leading edge — the latency half of the same trade you're pricing.

Back to Cara, because both halves of that story are now legible.

The half that worked: a 16× user explosion in seven days, absorbed with zero capacity planning. No fleet resize, no load test, no war room. Scale-per-request means the platform stood up as many environments as the traffic demanded, minute by minute. A fleet sized for 40,000 users would have fallen over on day two; the meter didn't blink at 56 million invocations a day.

The half that hurt: nothing in the billing model saturates. A fleet has a natural cost ceiling — when the boxes are full, they're full; users see errors, but the bill stops climbing. The meter's whole virtue is that it never fills up — which means the invoice tracks your success with perfect fidelity, to roughly $96,000 for the week. Nobody had decided, in advance, what success was allowed to cost.

Two sober footnotes turn that story into engineering practice:

• "Infinite scale" has posted quotas. A typical account defaults to 1,000 concurrent executions per region, growing at about a thousand new environments per ten seconds. Those are support tickets, not laws — but a viral moment can hit the throttle (429s) before it hits the headline number. Know your limits before your traffic finds them.
• Budget alarms are architecture. With pay-per-use, a spending alert is the equivalent of a circuit breaker for your wallet — configured before launch, with a human decision attached: at what bill do we degrade, cache harder, or shed? The cost lessons at the end of this course (Cloud Cost Engineering) build this discipline out; here it's one sentence: proportional cost needs a proportional-cost plan.

Serverless buys its properties with real constraints. None is disqualifying; all are load-bearing in the decision.

Nothing survives between invocations. The platform may reuse your environment or silently discard it — so in-memory sessions, caches, and connection state are not yours to keep. Everything stateful externalizes to stores and caches, exactly the move Stateful vs Stateless: Where Does State Live? taught. Serverless is that lesson upgraded from advice to contract.

Executions are capped. Fifteen minutes, then the platform cuts you off. Long-running work restructures into queued steps — machinery this course covers in the messaging and pipeline lessons.

The p99 pays the cold-start tax — and the cure costs the disease. You can buy provisioned concurrency: environments kept permanently warm. Read that twice — you're paying for idle again. It's the fleet's billing model leaking back in, purchased per function. Sometimes worth it; never free.

Observing it is harder. A request now hops through managed services you can't SSH into; debugging is logs-and-traces or nothing — the observability section of this course stops being optional.

The architecture locks in more than the code. Your function body is portable; the event formats, triggers, permission wiring, and managed services around it are one provider's dialect. Migrations rewrite the wiring, not the handlers.

Weigh that list against what came off your plate: the fleet, the patching, the autoscaling policy, the idle bill, and the 3 a.m. host page. For the right shape of workload, it's the best trade in this section. For the wrong one, you just rented the world's most expensive steady-state computer.

The rubric compresses to one primary test and two guards.

The shape test. Sketch a week of this workload's load curve. Estimate the idle share — the fraction of provisioned time a fleet would spend waiting. Above roughly three-quarters idle, serverless is almost certainly the answer; the meter charges nothing for the silence that dominates. Below roughly a quarter idle, the fleet is almost certainly the answer; do the two-line arithmetic anyway because now you can. In between, price both — it's five minutes.

The latency guard. If the workload is user-facing with a tight p99, price the cold-start tax: what fraction of requests land cold, and is ~200-400 ms acceptable on that edge? Async and event-driven work usually shrugs; interactive checkout paths usually don't — or they buy provisioned warmth and accept the partial return to capacity billing.

The blast-radius-of-success guard. Before launch, name the bill at which you'd want to know within the hour. Set the alarm. Decide the response. Cara's lesson in one line: the platform will happily scale to a success you haven't budgeted.

For the marketplace, the rubric lands cleanly: notifications — the module the previous lesson extracted across a clean async seam, spiky ×10 on game nights, idle most of the day — is born serverless. The steady checkout core, running hot around the clock against a tight p99, stays on the fleet. One system, both answers, each defensible with arithmetic — which is exactly what "trade-offs over templates" has meant all course.

"Serverless means no servers." There are servers — real microVMs booting in under 125 ms. What changed is the rental agreement: by the millisecond of work instead of by the month of capacity, with the fleet on someone else's inventory.

"Serverless is always cheaper." It's cheaper where a fleet would idle. The same handler that costs $5 a month at 2 req/s costs $2,678 at a steady 1,000 req/s — against a $140-210 fleet. The meter beats the rectangle only when the rectangle is mostly empty.

"It scales infinitely." Two ceilings, not zero: quotas (a default of ~1,000 concurrent executions, ramping ~1,000 per 10 s) and the invoice, which scales with success without asking. Cara found the second one: ~$96k for a viral week.

"Cold starts make it unusable." Median cold starts for scripting runtimes are 200-400 ms, snapshot-restore tamed Java's seconds, and event-driven work barely notices. It's a p99 tax on latency-critical edges — a line item in the decision, not a veto over it.

"Serverless means our whole system goes serverless." It's a per-workload billing choice, not an identity. The marketplace runs its checkout on a fleet and its notifications on functions, and both choices came from the same arithmetic. Systems that pick per-shape beat systems that pick per-fashion — same verdict as the last lesson, one level down.

• Serverless shrinks the deployable to a function and inverts the bill: from paying for capacity (the rectangle) to paying for work (the area under the curve), with scale-to-zero at one end and scale-per-request at the other.
• A cold start is a boot sequence, not weather: microVM (~125 ms) → runtime → your init (the part you control) → handler. Warm reuse hides it; idleness and burst edges bring it back; your p99 is where it lives.
• The shape of the load decides the economics. Same handler, same prices: ~$5 vs $140 for an idle-heavy API; ~$2,678 vs $140-210 at steady saturation. Idle share is the whole question, and the crossover is a five-minute calculation.
• The meter never saturates. That's the rescue (Cara absorbed 16× growth unplanned) and the risk (~$96k/week, uncapped). Budget alarms are architecture; quotas are real; success needs a price you chose in advance.
• The constraints are the contract: no memory between invocations, capped execution, cold-start p99 tax (with warmth purchasable at the cost of un-inverting the bill), harder observability, deeper lock-in.

The style palette now has three entries: everything integrated, capabilities split, and compute borrowed by the millisecond. The next lesson adds the connective tissue that serverless functions were practically invented to handle — Event-Driven Architecture & P2P: events as the system's spine, and what happens when nobody calls anybody directly at all.