Checkpoint: Caching

One Chapter, One Reflex

Nine lessons ago, a cache was 'remember the answer.' Since then you've walked the whole stack: why hit rate is the entire game, who fills the cache and who owns the truth, what to evict, how copies lie and how to catch them, how one expiry becomes a stampede, how a fleet shares a keyspace, and how the same ideas run a planet-sized edge — plus a terminal session where every one of those numbers showed up for real.

Time to compress it into what it actually becomes on the job: a reflex. Not definitions — decisions. Below are ten situations pulled straight from production life. Each has one move that this chapter argues for and three that sound right (they're the exact traps the lessons warned about). The cheat-sheet is your map of the terrain; the checkpoint is the test drive.

A caching decision-map cheat sheet summarizing the whole caching chapter as five quick-reference bands. Band one, should I cache at all: yes when reads dominate, a few keys are hot, and a little staleness is survivable; no when access is uniform, writes dominate, or the value must be exact to the instant. Band two, the pattern: reads are filled by the app with cache-aside as the default or by the cache with read-through; writes go through for freshness, behind for speed with a loss window, or around to avoid pollution - and on every write, database first, then delete the key, never update it. Band three, when the cache fills: LRU as the default, LFU when popularity is stable or scans pollute recency, TTL with jitter as shelf life. Band four, when it breaks: a stampede is cured by single-flight locks, stale-while-revalidate, or warming; the stale-set race by a version or lease guard; a dead cache must be cheap to fall past with a time budget on every call. Band five, at scale: many nodes route by a consistent-hash ring or fixed slots so only what must move moves; at planet scale the CDN pulls by default and pushes what is predictable, and you purge the edge but version URLs for the browsers you cannot purge. A footer strip carries the three-question reflex: is it hot, how stale may it be, and who un-lies it - plus the reminder that hit rate is the number that matters. The palette is dark slate with emerald, sky, amber, violet, and red bands.

Make the Call: Ten Caches, Ten Decisions

No trick questions and nothing you haven't seen — every scenario is one of this chapter's core decisions wearing work clothes. Read each situation, commit to a move, and check the reasoning. Seven or more and the chapter is locked in; anything you miss points at the exact lesson to skim again.

The caching chapter as a gauntlet of ten real decisions: whether to cache at all, where to spend a performance budget, which write pattern absorbs a like-storm, what saves a cache from a nightly scan, the one correct write-path order, how to kill a dogpile, how to add a node without a miss storm, how a 40-gigabyte game patch reaches the edge, how to fix a bad bundle that browsers have already cached, and what to do when a dead Redis holds every request hostage. Instant right-or-wrong with the reasoning and the exact lesson to revisit, and a score that tells you whether the chapter is locked in.

What You Can Now Do

Walking out of this chapter, three questions should fire automatically whenever data crosses a slow boundary:

Is it hot? Caching only wins where reads repeat — a few hot keys, reads over writes. No locality, no cache (and no amount of memory changes that).
How stale may it be? That number is your design: it picks the TTL, decides delete-on-write versus CDC, sets max-age for the browsers you can never purge, and tells you whether write-behind's loss window is acceptable.
Who un-lies it? Every copy needs a named owner of its freshness — a TTL, a write path that deletes, an event stream, or a versioned URL that makes the old copy unreachable. 'Nobody' is how caches end up serving $220 for a$ 149 product.

And underneath all ten decisions, the chapter's five moves kept repeating: bound the lie (TTL as the floor), collapse the herd (single-flight, shields), version instead of fixing (keys, epochs, URLs), keep the cache losable (cache-aside + a time budget), and measure hit rate above everything — it's the one number that predicts the rest.

Wrapping Up §8

That's caching, end to end: from one RAM lookup to four hundred cities, with the same handful of ideas scaling all the way up. It's also the chapter you'll reuse most — nearly every system you design from here on will have at least one cache in it, and now you know exactly where its bodies get buried.

Next section — Estimation & Numbers: the back-of-the-envelope skill that turns 'should we cache this?' from a vibe into arithmetic. How many requests is that really? How much RAM does the hot set need? Does it fit on one box? The numbers you've been absorbing all course — latencies, throughputs, capacities — become the tools you reason with, fast, in interviews and design reviews alike.