Pattern: Scaling Reads

Reads outnumber writes by a hundred to one or more, and the fix is never one fix — each read earns its own rung.

You already know the ladder. The Read-Scaling Playbook built it and measured every rung on a real database: an index took one query from 48.3 ms to 0.17 ms and one box from 28.8 to 16,772 transactions a second, and the ladder is ordered by how many copies of the truth each rung creates — index zero, replica N, denormalise copies you maintain, cache a copy nobody maintains.

This lesson is not that lesson again. This is the interview version, and it exists because of one hard constraint that changes everything:

In production you diagnose with EXPLAIN. In an interview you cannot measure anything.

No query plan. No pgbench. No production traffic. The playbook's central discipline — measure before you climb — is simply unavailable to you in the room. So the whole skill becomes: get the diagnosis out of a question instead of a profiler.

That is the first thing to internalise, because it is why the pattern gets missed. Nobody will ever hand you "design a read-heavy system." What you get is:

• "Design Twitter."
• "Design Yelp."
• "Design Google Maps."
• "Design a news feed."
• "Design a product catalogue."

Every one of those is a read-scaling problem wearing a product name. The tell is not in the words — it is in the ratio and the shape, and you find both by asking. Read-heavy is the default rather than the exception: a hundred reads per write and up is ordinary for anything content-shaped, and "read-heavy" is usually defined as eighty percent or more of operations being reads.

So the recognition move is one question in the requirements phase, and you already know to ask it from Requirements & NFRs That Matter: "what's the read-to-write ratio?" If the answer starts with a number bigger than about ten, you are in this pattern and everything below applies.

And there is a second tell that is easier to miss and worth more. Ask what a stale answer would cost. "If someone saw this data a few seconds late, would they notice — and would they mind?" That single question decides which rungs are even available to you, before you have drawn anything.

Here is the whole pattern as a decision rubric rather than a recital. Each rung has a trigger, and because you cannot measure, every trigger is phrased as a question you can ask out loud.

Rung 1 · Index — "what do people look this up by?" If the answer is anything other than the primary key, there is an index in your future. This is the rung that costs nothing in consistency and it is the one candidates skip, because CREATE INDEX does not look like architecture. Say the field: "reviews get read by restaurant, so there's an index on restaurant_id." Four seconds, and it reads as someone who has run a database.

Rung 2 · Replica — "is this someone reading their own data, or other people's?" This is the sharpest question in the lesson and almost nobody asks it. A replica buys throughput and charges staleness — so it is fine for other people's data and actively wrong for the reader's own recent write. More on that in a moment; it is the trap.

Rung 3 · Denormalise — "how often does this change, versus how often is it read?" When the ratio is lopsided, move the work to the write. A restaurant's average rating changes when a review lands and is read on every single page view — so it is a column on the restaurant row, not an aggregate over a million reviews. You have moved work from read time to write time and taken on the job of keeping the two agreeing.

Rung 4 · Cache — "is there a hot minority?" A cache is worth exactly its hit rate, and its hit rate is decided by whether a small subset of the data carries most of the reads. A hot minority makes it superb. A long tail of distinct queries makes it a bill with nothing attached — for calibration, static content sits around 80–95% hit rate while dynamic or personalised endpoints sit nearer 20–60%, and a genuine long tail is worse than either.

⭐ Notice that not one of those four questions is "is it slow?" You cannot find out. Every one of them is a question about the shape of the access pattern — which is something the interviewer can simply tell you, if you ask.

Now the thing that actually distinguishes a senior answer, and it is not knowing more rungs.

Ask any candidate how they would scale reads on a review site and you will hear a version of this: "it's read-heavy, so I'd add read replicas and put a cache in front." It is not wrong, exactly. It is answered at the wrong level.

One prompt is never one read. Take "design a local-business review site" and look at what the home screen and a business page actually ask for:

Four reads, four different answers, and the fourth one's answer is the absence of a rung.

There is no read-scaling strategy for a system. There is a rung per read.

This is why the systemwide answer reads as junior. It is the audible signature of someone who stopped at the prompt and never looked at the queries — and it is a five-second fix: instead of "I'd add replicas and a cache", say "there are three different reads here and they don't want the same thing." Then take them one at a time.

Two traps live in this pattern. The first is the one interviewers deliberately set. The second is one you build for yourself, and it is the more interesting of the two.

The set trap: routing someone's own write to a replica

It arrives as a friendly-sounding question: "so where does this read go?" — about a read that happens to be the user's own data, seconds after they changed it. Their profile. Their order. Their review. Their playback position.

Answer "a read replica" and you have not made a performance mistake. You have designed a correctness bug, with a user's name on it: they save something, the read lands on a replica that has not heard yet, and their change appears to have been lost. Read Replicas & Replication Lag has the mechanism; what matters here is the interview reflex — the question "whose data is this?" comes before the question "where does it read from?"

And be honest about the numbers rather than hand-waving them, because the honesty is the signal. Same-region lag is typically under a second — Postgres 14's parallel apply took one high-write workload from four seconds to under one — but the tail is what bites: a documented incident had a MySQL replica four minutes behind because the apply thread is single-threaded and the binlog was arriving at 80 MB a second. "Usually milliseconds, occasionally minutes" is the sentence, and the design has to survive the second half of it.

The self-set trap: the better your cache, the harder your fall

This one nobody warns you about, and it is worth more than any other sentence in this lesson.

Start with the arithmetic, because it is arithmetic and you can do it out loud. Origin load is the miss rate, so:

• 90% hit → the database sees 10% of traffic
• 95% hit → 5% — the same cache, and you just halved the database's load
• 99% hit → 1% — a "four-point" improvement that cut it fivefold
• 99.9% hit → 0.1%

It is nonlinear at the top, which is why the last couple of points are where all the value is. Content-delivery people live by this: 95% offload leaves 5 GB of every 100 GB at the origin and 90% leaves 10 GB, so a five-point improvement in offload halves the origin fleet.

Now read the same number in the other direction, which almost nobody does:

A 99% hit rate means your database is sized for 1% of traffic.
So when the cache dies, it takes 100× its provisioned load — instantly, with no warning.

Nobody buys a datastore for traffic it never sees. You size it from what it takes on a normal day, and on a normal day the cache is standing in front of it lying about how much traffic exists. Amazon's own engineering guidance puts it bluntly: past a certain hit rate a cache stops being a performance layer and becomes a correctness dependency, and an extended cache outage causes an atypical spike in traffic to the service behind it — throttling, brownout, or worse, because if the cache died from load then failing open sends all of that load straight downstream.

⭐ The hit rate is not only how good your cache is. It is also how far your database has to fall.

Which gives you two honest answers, and saying either one unprompted is a strong signal:

1. Provision for the miss storm — size the datastore for a cold cache, and accept that you are paying for capacity you use on one bad day a year.
2. Shed load on the way down — accept that a cache failure degrades the product, and say exactly how: serve stale, serve a smaller page, or reject a share of traffic deliberately rather than falling over.

And two smaller ones worth having ready, because interviewers ask them in this order: invalidation ("how does the cache learn the data changed?") and stampede — when many keys expire at the same instant, every one of those requests misses together and hits the database as one wave. The fix is a single clause: jitter the TTL. Five minutes plus a random zero-to-thirty seconds, so expiries spread out instead of clustering.

Reading "a rung per read" is easy. Doing it under a clock, on a prompt you have not seen, is the part that has to become automatic.

Below is a cold prompt with three reads on one screen. They want three different rungs, and one of them wants no rung at all. Choose per read — then dial the hit rate and pull the plug on the cache.

Two things to do deliberately. Put a cache on all three and watch the database load fall to almost nothing — then kill the cache and watch a 100× spike arrive at a datastore you sized for 1% of traffic. And then drag the hit rate down and watch the multiplier shrink: a worse cache is a safer one, because you provisioned for what you could actually see. That tension is real, it is the whole design decision, and no amount of reading makes it land.

The pattern is worth nothing if it takes ninety seconds to say. Here is the whole thing, compressed to what you would actually speak in the room — about twenty-five seconds:

"Before I pick anything: what's the read-to-write ratio, and would anyone mind seeing this a few seconds late?

Right — a hundred to one, and staleness is fine except for a user's own data. Then these aren't one read. Nearby search filters on location, so that's a geo index. The rating is an aggregate read on every view and written rarely, so I'll denormalise it onto the business row. Popular business pages are a hot minority, so those get cached — and I'd want the hit rate, because that number is also how far the database falls if the cache dies. Their own reviews go to the primary; that's read-your-writes and a replica would show them their review disappearing."

Count what that did. It asked before designing, it named four rungs with a reason each, it priced the cache in both directions, and it caught the read-your-writes trap before anyone set it. That is the pattern performed rather than recited — and it fits inside the high-level design block with room to spare.

What it stands on. This pattern is the interview face of a section you have already done, and pointing back at the specific lesson is what turns a claim into a defence:

• The Read-Scaling Playbook — the ladder, measured, and the copies-of-the-truth ordering
• Indexes Deep-Dive and When Ordinary Indexes Fail — rung one, including why a geo or text query needs a different index rather than a bigger machine
• Read Replicas & Replication Lag and Caches & Databases: Read-Your-Writes Coherence — rung two and the trap that lives in it
• Denormalization and Materialized Views — rung three, and when the write path should own the work
• Caching Patterns, Cache Invalidation, Stampedes, Thundering Herds & Cache Warming and CDNs: Caching at the Edge — rung four, its invalidation story and its failure modes

Where you'll use it. Three of the practice problems are read-scaling problems at heart, and each one stresses a different rung:

• Twitter — fan-out is read scaling turned inside out. Pre-computing timelines is denormalisation at extreme scale, and it breaks on one input: a single account with fifty million followers costs fifty million writes for one post, so the standard answer is a hybrid — push below roughly ten thousand followers, pull for the celebrities and merge at read time. (That is the Fan-out: Feeds & Timelines pattern, later in this part of the course.)
• Yelp — the worked example above. Geo index, denormalised rating, cached hot pages, and the reviewer's own write going to the primary.
• Google Maps — the extreme case of rung four. Map tiles are immutable, and immutability deletes the entire cost of the top rung: there is no invalidation problem, so the cache stops being the most expensive decision on the ladder and becomes the obvious first move, at the edge.

⭐ That last point is the honest limit on the ladder's ordering. It is ordered by consistency cost — so for immutable data the top rung costs nothing and you should reach for it immediately. Saying "normally I'd earn a cache last, but this data never changes, so the invalidation problem doesn't exist" is a genuinely senior sentence.

• ⭐⭐⭐ The rung is a property of the READ, not of the system. "It's read-heavy so I'd add replicas and a cache" is answered at the wrong level. One prompt, several reads, several answers.
• ⭐⭐⭐ You cannot measure in an interview, so every rung is earned by a question: what do people look this up by (index) · is this their own data (replica) · how often does it change versus get read (denormalise) · is there a hot minority (cache).
• ⭐⭐ A 99% hit rate is a 100× load multiplier waiting to happen. The number that makes the cache impressive is the number that makes its failure fatal — so name the hit rate, and then say whether you provisioned for the miss storm or you shed load on the way down.
• ⭐ "Whose data is this?" comes before "where does it read from?" A replica for someone's own recent write is not a slow design, it is a wrong one. Lag is usually milliseconds and occasionally four minutes.
• ⭐ The ladder is ordered by consistency cost — so immutability reorders it. Map tiles and short-link mappings never change, so caching them costs nothing and goes first.
• A cache with no stated hit rate is not a design. And the two follow-ups always come in the same order: how does it learn the data changed, and what happens when every key expires at once (jitter the TTL).
• Say the read-to-write ratio and the staleness tolerance out loud in the requirements phase. Those two answers decide which rungs are even available before you draw a box.

Next: Pattern: Scaling Writes. Reads let you make copies of the truth and choose how stale each one may be. Writes offer no such comfort — there is only one place the truth can be changed, and everything interesting comes from that.