Checkpoint: Scaling Writes & Partitioning

Eight lessons ago, Partitioning vs Sharding opened on a wall the read-scaling section had walked you straight into: every lever in §4 scaled reads, but the writes still funneled into one leader, one table, one box — and no replica, cache, or index can absorb a write. A write has exactly one owner. Standing at the top of §5 now, the whole section collapses into one law that mirrors the last:

Reads scale by copies. Writes scale by splits.

You can't copy your way out of a write — you split it, and there are exactly three ways:

Split one table's rows across boxes — SHARD — for throughput. The shard key is the whole decision: high cardinality, even distribution, and in your top query's WHERE (or every read scatter-gathers). Consistent Hashing balances the keyspace, not per-key load — a viral key still lands on one node — so when a shard melts you diagnose the cause: you cannot add-shards your way out of one hot key (measured: 85% / 82% / 81% on a single shard at 4 / 8 / 16 shards, the peak climbing to 12.96×). Salt the key, isolate the whale.

Split different tables by function — FEDERATE — for isolation. The bill is sharp: the JOIN is gone — a SQL join lives in one database, so a cross-boundary query becomes an app-side IN-batch (2.2 ms done right) or an N+1 (66 ms, 29× slower). Federate first, shard second.

Split the cost per write — BATCH — amortize the fixed toll (an fsync, a round trip) on a 1/N curve with an early knee (109× at the peak, regressing to 95× when the batch is too big). The price is latency and a buffer you can lose (fsync-per-commit is 17.9× slower — that gap is the loss window you accept). And every split needs ids that don't collide with no coordinator: Snowflake — 1 sign | 41 timestamp | 10 worker | 12 sequence — sortable because time leads.

This checkpoint tests the one skill that ties it together: choosing where the data splits — a decision you can't cheaply take back.

Here's the decision made concrete. You've got a social app to shard across six boxes — users, posts, and the follows graph — and for each table you pick the shard key. Watch what your choice costs: which shards the table's top query has to touch (one, if the key is in the WHERE; all six if it isn't — a scatter-gather that waits for the slowest), and whether one shard catches fire when a celebrity shows up. Fair warning: one of the three tables has no perfect key — finding the least-wrong one, and naming what you'd do about the caveat, is the whole skill.

Eight scenarios, one per lesson. Every wrong option is a trap this section explicitly corrected — the reflex that sounds right and melts a shard, strands a batch, or deletes a JOIN. If one tempts you, the explanation names the lesson to re-read. Six of eight passes.

If the shard-key choices felt like recognition rather than guessing — status has three buckets, that's a hotspot; author_id co-locates a timeline but a celebrity melts it; who-follows-me is the wrong direction — then §5 did its job. You don't reach for a replica when the writes are the wall anymore; you split the data, on purpose, at a seam you chose, and you know which split buys throughput, which buys isolation, and which just buys cheaper writes.

That completes the scaling arc. §4 scaled reads with copies; §5 scaled writes with splits — and every split, every copy, every batch quietly assumed one thing we've been leaning on since Container 1 without ever pinning it down: that a group of changes either all happen or none do, that the database won't hand you a half-finished write. That assumption has a name and a precise set of guarantees, and the next section makes you earn every letter of it. Next — Transactions & Concurrency, starting with ACID, Precisely.