gRPC & Streaming

Two lessons ago, we left a thread hanging: protobuf's natural habitat, we said, is service-to-service RPC — where you control both ends and every millisecond counts. Time to pull that thread. And the cleanest way in is a single line of code:

user, err := client.GetUser(ctx, &GetUserRequest{Id: 42})

That's a network call — a request crossing machines, hitting a server, coming back. But look at it: it reads like calling a local function. Typed arguments, a typed return value, autocomplete in your editor. No URL to design, no verb to choose, no status code to map, no JSON to parse. This is RPC — Remote Procedure Call — the older-than-REST idea that the network should disappear behind a function call. gRPC is Google's modern take on it (open-sourced 2015, now the de facto standard for internal service traffic), and it's built from two ingredients you already know and one you've been waiting for:

• Protobuf (lesson 4) provides the contract and the bytes — but the .proto file grows a new block: service Users { rpc GetUser(GetUserRequest) returns (User); }. Not just message shapes — callable functions, declared in the schema.
• Code generation turns that service block into client stubs and server skeletons in a dozen languages. Your Go service, Python ML job, and Java billing system all get the same typed contract, enforced by their compilers. API design review becomes .proto review — lesson one's contract, with teeth.
• And the new ingredient — HTTP/2 — is what turns this from "faster REST" into something genuinely different. Because a persistent, multiplexed connection doesn't just carry request-response faster; it makes entirely new shapes of conversation possible. That's the real headline of this lesson.

One honest caveat before the tour, because RPC's history demands it: the function-call illusion is ergonomic, not physical. A "function" that can time out, half-complete, or need a retry is not a local function — lesson 3's ambiguous timeout applies in full, idempotency keys and all. gRPC's answer to that honesty is deadlines, and we'll get there. First: the four shapes.

Everything distinctive about gRPC rides on its transport, so spend two minutes on what HTTP/2 actually changed.

HTTP/1.1 — the transport under classic REST — is one-request-at-a-time per connection. Want concurrency? Open more connections; manage a pool; pay setup cost per call. HTTP/2 replaces that with binary framing: the connection carries a sequence of small frames, each stamped with a stream ID. Frames from different streams interleave freely — so one TCP connection carries many concurrent requests and responses, each stream independent. A slow response doesn't block a fast one behind it (goodbye, HTTP-level head-of-line blocking). Connections are persistent and long-lived: no handshake tax per call. And because each stream has its own lifecycle, either side can cancel one stream without disturbing its neighbors.

For gRPC this buys three concrete things. First, cheap calls: your service makes thousands of RPCs per second over a handful of warm connections. Second, cancellation that works: a user closes the tab, and the cancellation can ripple down the entire call tree, stream by stream. Third — and here a seed planted in the load-balancing chapter finally blooms — an operational trap: remember §5's warning that an L4 balancer sees one connection and pins everything on it to one backend? Now you can see exactly why: all those multiplexed streams look like a single flow to anything that can't read HTTP/2. Balance gRPC at L7, per-stream, or ten replicas will idle while one drowns.

(One footnote for precision: HTTP/2 fixed HTTP-level head-of-line blocking, but one lost TCP packet still stalls every stream on the connection — TCP-level HoL. That's the itch HTTP/3/QUIC exists to scratch, as the networking chapter mentioned.)

Here's the heart of the lesson. REST has exactly one conversation shape: request → response. Everything else — live feeds, uploads-in-progress, chat — gets faked on top of it (you've polled an API in a loop; that's the fake). gRPC makes four shapes native, each a one-line declaration in the .proto:

① Unary — rpc GetUser(Req) returns (User). One request, one response. A function call. This is 90% of RPCs, and everything from lessons 1–5 (contracts, idempotency, evolution) applies verbatim.

② Server streaming — rpc Watch(Req) returns (stream Update). The client asks once; the server responds with a stream of messages over time — for as long as the call lives. This is the subscription shape: a price ticker, a log tail, Kubernetes watch APIs, and the one you use daily — an LLM streaming tokens into your chat window. One stream keyword; no polling, no hacks.

③ Client streaming — rpc Upload(stream Chunk) returns (Receipt). Mirror image: the client sends a stream — file chunks, sensor readings, metrics batches — and the server answers once, at the end, with a summary. The upload shape.

④ Bidirectional streaming — rpc Chat(stream Msg) returns (stream Msg). Both sides stream, independently, on the same call — the server doesn't wait for the client to finish, and neither blocks the other. Two live lanes. Chat, collaborative editing, speech-to-text (audio flowing up while captions flow down), multiplayer state sync.

Notice what makes this more than a feature list: all four shapes are typed by the schema, multiplexed on one connection, and cancellable per-stream. The conversation shape becomes part of the contract — a reviewer sees stream in the .proto and knows exactly what kind of relationship these services have. REST could never say that in its grammar.

Timelines are for watching, so here are all four conversations, live, with you on the client side.

Run unary first — one dot across, one dot back, done; the familiar shape. Then open server streaming and feel the difference: one request, and the server just… keeps talking — token after token, tick after tick, until you close the stream (or a deadline reaps it). Try client streaming: feed chunks upstream at your own pace, then end the stream and collect the single receipt. And then the good one — bidirectional: type your own messages into the chat and watch the server's replies interleave with them, two independent lanes on one call, neither waiting for the other.

Two instruments make the systems lessons visible. The deadline slider arms every call with a time budget — drag it down and watch a long-running stream die mid-flight with DEADLINE_EXCEEDED, then check the three-hop chain view: the deadline propagates, each downstream service inheriting whatever budget remains. And the HTTP/2 lens shows the same four conversations the way the network sees them: stream IDs interleaved on one persistent connection — the exact picture that explains why an L4 balancer pins all of it to a single backend.

Now the production discipline that separates gRPC-in-a-demo from gRPC-at-Google — and it's the antidote to the function-call illusion.

Every gRPC call can carry a deadline: "I'm willing to wait 200ms." Not a client-side shrug-and-give-up — the deadline travels with the request. The server sees it, and when it expires, the call fails with DEADLINE_EXCEEDED and the server stops working on it. Compare that to the REST default, where a client timeout just means the client stopped listening while the server soldiers on, burning CPU on an answer nobody will read. (Recognize this? It's §5's "a timeout is the caller's own health check," promoted from advice to protocol feature. And yes — a deadline-exceeded call is exactly lesson 3's ambiguous timeout: did it commit before dying? Idempotency still applies.)

The staff-level part is propagation. Service A calls B with 200ms; B spends 80ms, then calls C — and gRPC (in most implementations, automatically) hands C the remaining 120ms, not a fresh budget. The whole call tree shares one clock. When the user's patience is spent, every service in the chain finds out — C included, three hops from the user — and stops. Without this, dead requests zombie-march through your microservices, consuming real capacity for answers that will be thrown away; with it, the system sheds abandoned work automatically. Set deadlines on every call (the gRPC team's own guidance) — an RPC without a deadline is a promise to wait forever.

Cancellation is the same muscle, user-triggered: because HTTP/2 can kill one stream cleanly, a client canceling a call — user closed the tab, typed a new search — can ripple the cancellation down the whole tree. Streams make this matter more: a server-streaming call with no deadline and no cancellation handling is an immortal faucet.

Time for the honest limitation — the one that decides where gRPC actually lives in your architecture.

Browsers cannot speak native gRPC. Not "it's awkward" — impossible: browser APIs (fetch/XHR) give you no control over HTTP/2 framing, no access to trailers, no way to even force HTTP/2. gRPC's wire protocol needs all three. The workaround is gRPC-Web: an envelope the browser can send, plus a translating proxy (classically Envoy with its grpc_web filter) that unwraps browser requests into real gRPC for your backends and re-wraps the responses.

It works — with a hard asterisk. Browser HTTP semantics require the request body to finish before the response starts, which kills half the lesson: unary ✓, server-streaming ✓, client-streaming ✗, bidirectional ✗. From a browser, through gRPC-Web, two of the four shapes simply don't exist. (Real-time browser communication — chat in an actual web app — runs on WebSockets and server-sent events, which is literally the next section of this course.)

So the architecture writes itself, and it's the same hybrid you saw with GraphQL: gRPC on the inside — service-to-service, where you control both ends, the compilers enforce the contract, and the streams flow free — and REST or GraphQL at the edge, where browsers, strangers, and curl live. This isn't a compromise borne of weakness; it's each tool in the habitat lesson 4 predicted for it.

The benchmarks first, conditions attached (internal service traffic, protobuf vs JSON): roughly 2× the throughput (~107% higher on small payloads, ~88% on large), ~45–48% lower latency, and meaningfully cheaper machines (~19% less CPU, ~34% less memory, ~41% less bandwidth) — with message sizes up to 10× smaller, which is lesson 4's protobuf doing the lifting. No magic: binary encoding (no text parsing) + multiplexed persistent connections (no per-call setup). The two ingredients this section already taught you, compounding.

The honest ledger, then:

The decision rule is the one lesson 4 planted: who's on the other end? You control both ends + volume is real + you want streaming or whole-chain deadlines → gRPC. Strangers, browsers, humans debugging, HTTP caching → REST. Nearly every serious shop runs the hybrid: REST/GraphQL at the edge, gRPC inside. The mistake worth naming is ideological uniformity — forcing gRPC to the browser through proxies and losing half its features, or running chatty JSON between your own backends at 50K req/s and paying the parse tax forever.

• "gRPC is just faster REST." It's a different model — functions-you-call versus resources-you-transfer — plus four native conversation shapes REST's grammar can't express. The speed is a consequence of protobuf + HTTP/2, not the point.
• "The function call means I can forget the network." The illusion is ergonomic, not physical. Every RPC can time out, half-complete, or need a retry — set a deadline on every call, handle DEADLINE_EXCEEDED and UNAVAILABLE, and remember lesson 3: a deadline-exceeded call is an ambiguous timeout, so idempotency still earns its keep.
• "We'll standardize on gRPC everywhere, frontend included." Browsers physically can't speak it; gRPC-Web plus a proxy recovers unary and server-streaming only. The edge belongs to REST/GraphQL; gRPC's habitat is east-west. That's architecture, not preference.
• "Streaming is an exotic feature we won't need." One stream keyword — and it's how every LLM token feed, price ticker, and watch API you use already works. The exotic thing is faking streams over request-response with polling loops (the next section prices that hack precisely).
• "HTTP/2 solved head-of-line blocking." At the HTTP level, yes. One lost TCP packet still stalls every stream on the connection — which is why HTTP/3 moved the whole game to QUIC/UDP. Know which layer your bottleneck lives at.

Forty-five minutes, any two languages, and RPC stops being a diagram:

1. Write the contract. A hello.proto with one service, two RPCs: rpc Greet(Name) returns (Greeting) (unary) and rpc CountTo(Target) returns (stream Number) (server-streaming). Note that the conversation shapes are visible in the schema.
2. Generate both ends in different languages. protoc (or buf generate) → a Python server, a Go or Node client. Marvel briefly at the typed stubs you didn't write. Implement Greet trivially; implement CountTo with a sleep(0.5) between numbers.
3. Feel the stream. Call CountTo(100) from the client and print numbers as they arrive — they flow, half-second by half-second. You did not poll. Now kill the client mid-stream (Ctrl-C) and watch the server's logs: the stream cancels — the server knows. Try faking that with REST and appreciate the difference.
4. Arm a deadline. Set a 2-second deadline on CountTo(100). Watch it die at 4 with DEADLINE_EXCEEDED — and confirm the server stopped counting (log it). That's abandoned work being shed automatically.
5. Poke the wire. grpcurl -plaintext localhost:50051 list then call a method from the terminal. Notice you needed a tool where REST needed eyes — the readability trade from lesson 4, felt in your fingers.
6. Bonus: run two clients concurrently and ss -t / netstat the server — count the TCP connections. Multiplexing means fewer sockets than you'd guess.

gRPC is what this section's threads braid into: lesson 1's contract, written as a .proto service; lesson 4's protobuf, carrying the bytes; HTTP/2 underneath, making the connection persistent, multiplexed, and cancellable:

• RPC model: remote calls that read like typed local functions, stubs generated per language, the schema as the single source of truth — with the honest caveat that the network never actually disappears (deadlines + lesson 3's idempotency are the adult supervision).
• Four conversation shapes, native and typed: unary (call), server-streaming (subscribe — tickers, LLM tokens), client-streaming (upload), bidirectional (chat). REST has one shape; gRPC's grammar says what kind of relationship two services have.
• Deadlines propagate — the whole call tree shares the caller's clock, and abandoned work gets shed automatically, three hops deep. Cancellation rides the same per-stream machinery.
• Browsers can't speak it — gRPC-Web recovers unary and server-streaming only. Hence the standard hybrid: REST/GraphQL at the edge, gRPC inside — and roughly 2× throughput at half the latency for the internal traffic that moved.
• One connection, many streams — which is why gRPC demands L7 balancing (§5's trap, finally fully explained).

And that server-streaming shape — the server pushing messages to a subscribed client — opens the door this course walks through next. Because browsers need push too (chat, notifications, live dashboards), and they can't use gRPC to get it. How the web does real-time — WebSockets, server-sent events, long-polling, and then the whole world of asynchronous communication: queues, pub/sub, and events — that's Section 7, and it starts now.