Multi-Region Patterns

Introduction

Disaster Recovery: RTO & RPO ended on a promotion. The top recovery tier — a full copy of your system already running in a second region — was so far past "recovery" that it wasn't really a backup at all; it was an architecture, live and serving. This lesson is that architecture. What does it actually mean to run in several regions at once, and what new problems does it hand you in exchange for the ones it solves?

First, the honest part, because it saves most teams a world of pain: most systems should not be multi-region. One region spread across a few availability zones already buys you roughly four nines of availability and none of the headaches you're about to meet. You go multi-region for exactly three reasons, and if none of them is true for you, don't. Latency: your users are spread across the planet and a single region leaves the far ones waiting a quarter-second for every click. Survival: you must outlive the loss of an entire region, which needs more nines than one region can give. Law: a regulation says certain data must physically stay inside certain borders, and no amount of engineering can argue with a government. Everything in this lesson is the price of those three benefits — and the bill comes due almost entirely on one line item: writes.

Why run in more than one region, drawn as a simple world with three data-center regions — one in the Americas, one in Europe, one in Asia — each serving the users nearest to it with a short green arrow, instead of every user in the world reaching across the ocean to a single box. Three labels name the only three good reasons to do it: LATENCY (be close to users), SURVIVAL (outlive the loss of a whole region), and LAW (keep data where it must legally stay). A quiet footnote admits the fourth truth: most apps don't need this — one region across a few availability zones is already about four nines.

Two Shapes: Active-Passive and Active-Active

A multi-region system comes in two shapes, and choosing between them is the first and biggest decision.

Active-passive keeps one region in charge. A single region — the leader — takes every write, while one or more other regions sit warm, holding an up-to-date copy through the replication you met in Sync vs Async Replication & Failover, serving reads at most, and waiting. If the leader dies, you promote a standby and carry on. Its great virtue is simplicity: because exactly one region ever accepts a write, two writes can never disagree, and a whole category of nightmares simply doesn't exist. Its costs are the two you already know from disaster recovery — users far from the leader pay the distance on every write, and surviving a failure means an actual failover, with the seconds of downtime that implies.

Active-active runs the whole system in every region at once, and every region takes live traffic, reads and writes both. Now every user is close to a region that can serve them fully, so writes are fast everywhere, and losing a region needs no failover at all — the survivors were already carrying their share, and traffic just stops flowing to the dead one. It is the better experience in every way but one, and that one is enormous: if two regions can both accept a write, then two regions can change the same thing at the same moment, and reconciling that is the hard problem the rest of this lesson circles. Active-passive trades latency for simplicity; active-active trades simplicity for latency, and hands you a conflict problem in the bargain. Most teams that think they need active-active actually need active-passive with good local read replicas, so be sure the latency is worth the complexity before you reach for it.

The two shapes of a multi-region system, side by side. On the left, ACTIVE-PASSIVE: one region is the leader and takes every write while a second region sits warm, only reading, ready to be promoted if the leader dies. One writer means no conflicts ever, but users far from the leader wait, and a failover is needed to survive a loss. On the right, ACTIVE-ACTIVE: both regions serve live traffic and both take writes, so every user is close to a writer and a lost region needs no failover at all — but now two regions can change the same thing at once, which is the whole hard problem of this lesson. Law band: active-passive trades latency for simplicity; active-active trades simplicity for latency.

Getting Users to the Right Region

Before any of this matters, a user in Tokyo has to actually land on the Tokyo region, and that steering happens above your application, in the layer you built in Global Load Balancing: GeoDNS & Anycast. There are two common ways to do it. Location-aware DNS answers the same hostname with a different region's address depending on where the question comes from, so the lookup itself hands each user their nearest region. Anycast, from Anycast: One IP, Many Places, goes further and advertises one address from every region at once, letting the network's own routing deliver each user to the closest entry point without any per-user decision at all.

The quiet beauty is that the machinery that puts users close is the same machinery that routes them away from trouble. Both approaches lean on health checks: the routing layer constantly asks each region "are you alive?", and the moment a region stops answering, it simply stops handing out that region's address. New users flow to healthy regions automatically, and in active-active that is the entire failover story — no promotion, no ceremony, just traffic declining to visit a corpse. This is also the layer where you deliberately shift traffic during a drill or a bad deploy, which is how a region evacuation like the one in Disaster Recovery: RTO & RPO is actually carried out.

How a user reaches the right region, drawn as a router in front of three regions. A user asks for the address, and a location-aware layer — geographic DNS, or an anycast address that the network itself routes to the nearest entry — sends them to the closest region, here the European one, over a short green path. When that layer's health check finds a region has gone dark, it simply stops handing out that region's address, and the next user is steered to a healthy one instead, over an amber reroute path. Law band: the same trick that puts users close is the trick that routes them away from a dead region.

Reads Are Easy; Writes Are the Whole Game

Here is the sentence to remember: across regions, reads are easy and writes are everything. A read can always be served from the nearest copy, and if that copy is a second or two stale — the replication lag from Read Replicas & Replication Lag — most of the time nobody notices or cares. Writes are where regions fight, because a write has to become the truth everywhere, and there are only three honest ways to arrange that.

Write-global: one region owns every write. You pick a single home region, and every write on Earth travels there to be applied, then flows back out to the others as a copy. It is beautifully simple and can never produce a conflict, because there is still only one writer — this is active-passive's discipline kept even while other regions serve reads locally. The price is blunt: a user on the far side of the world pays a full cross-region round trip on every write, and if the home region is unreachable, nobody can write at all.
Write-local: every region writes for itself. Each user writes to their own nearest region and gets an instant answer, and the regions copy each other's changes in the background. This is multi-leader replication from Replication Topologies, and it buys fast writes everywhere at the cost of the whole of The Conflict Problem: when two regions change the same record before they've heard about each other, you have two disagreeing truths and must reconcile them with the tools from Conflict Resolution Strategies — last-writer-wins, which is simple and silently throws one write away, or a CRDTs-style merge that keeps both when the data allows it. A global store like DynamoDB's global tables runs exactly this way, replicating in a second or two and resolving clashes by last-writer-wins.
Write-partitioned: every record has a home. You give each record a home region decided by a key — very often the user's own home region — and all writes to that record always go to that one region. Now writes to your own data are local and fast and can never conflict, because each record still has exactly one writer; only the rarer write to someone else's record pays a cross-region trip. It is the sweet spot for data that naturally belongs to one place, which is most user data, and as you'll see, sometimes the law makes this choice for you.

The console below makes the three real. Run the same traffic under each strategy and watch what you're actually buying: slow writes, conflicts, or a fixed home for every record.

Reads are easy; writes are the whole game — the three ways to handle a write across regions, in three rows. Reads (top, calm green): always served from the nearest copy, fast everywhere. Then the three write strategies. WRITE-GLOBAL: every write, wherever it starts, travels to one chosen home region — simple and conflict-free, but a user on the far side of the world waits a whole cross-ocean round trip for every write. WRITE-LOCAL: each user writes to their own nearest region for speed, but when two regions change the same record at once, a red spark — a conflict — must be resolved afterward, and last-writer-wins quietly drops one. WRITE-PARTITIONED: each record has a home region decided by a key (often the user's own region), so writes for local records are fast and can never conflict, while the rare foreign-record write pays the cross-region trip. Law band: pick your poison — slow writes, conflicts, or a fixed home.

Run a system on three continents. Pick how it handles writes — one global home, write-local for speed, write-partitioned by key, or a strongly-consistent global write — and watch reads and writes fly between the regions in real time, with the latency each one pays. Then break it: kill a region and see whether traffic just shifts or a failover stalls, or cut the link between regions and watch the choice land in front of you — let both sides keep writing and watch them split into two disagreeing brains, or keep them consistent and watch the cut-off region go dark. The write path and the partition, the two hardest things about running everywhere at once, on one map you can drive.

The Tax You Cannot Escape

Underneath every one of those choices sits a wall that no architecture can climb: the speed of light. Light in fiber crawls across the planet at a fixed pace, and the round trips are simply large — New York to London and back is about 75 milliseconds, New York to Singapore and back about 230. That is not a number you can optimize; it is physics, and it sets the price of agreement between regions.

So any write that insists on being confirmed by another region before it counts must pay that round trip, every time, and you are forced to choose. Synchronous replication waits for the far region to acknowledge before telling the user "saved" — the data is strongly consistent and never behind, but every write wears the full cross-region latency. This is what a globally-consistent database like Google's Spanner does, leaning on atomic clocks to order writes across the planet, and it is correct and it is slow. Asynchronous replication tells the user "saved" the instant the local region has it and ships the change onward in the background — writes are fast everywhere, but the other regions are always a beat behind, and a crash before the copy arrives loses that beat (exactly the data-loss window, the recovery point, from Disaster Recovery: RTO & RPO).

This is the real lesson of PACELC: CAP's Grown-Up Sibling, and multi-region is where it stops being theory. PACELC says that during a partition you trade availability against consistency — but else, in normal running with no failure at all, you still trade latency against consistency. Across regions there is no setting that gives you both low latency and strong consistency, because the planet is too big and light is too slow. You choose, per write, which one you can live without.

The tax you can't escape, drawn as a globe with light racing between regions. Light in fiber crosses an ocean in tens of milliseconds, and that is a hard floor set by physics: New York to London is about a 75-millisecond round trip, New York to Singapore about 230. So a write that insists on being confirmed by another region before it succeeds must pay that round trip every single time — that is the fork. SYNCHRONOUS (strong, slow): wait for the far region, always correct, but every write wears the latency, the choice a globally-consistent database like Spanner makes with atomic clocks. ASYNCHRONOUS (fast, stale): confirm locally and copy over later, quick everywhere, but the far region is always a second or two behind and a crash loses that tail, the choice a store like DynamoDB's global tables makes. Law band: across regions you may have low latency or strong consistency, never both — this is PACELC's else.

When Regions Cannot Talk

Everything so far assumed the regions can at least reach each other. The day they can't is the day multi-region earns its reputation. A partition — the subject of Network Partitions & Split Brain — is when the network between two regions fails while both regions are otherwise perfectly alive, each still holding a crowd of users demanding writes. Now the abstract warning of CAP, Properly arrives as a concrete, unavoidable decision, and an active-active system must pick one of two painful answers.

It can stay available: let both sides keep accepting writes as if nothing were wrong. Users never see an error, but the two regions are now editing the same records blind to each other, and they drift apart — a counter that read 4 becomes 5 in one region and 7 in the other. The system has split into two brains that each believe they are right, and when the link finally heals, someone has to merge two divergent histories and, with last-writer-wins, throw real work away. Or it can stay consistent: allow writes only on the side that can still reach a majority of the system, and have the isolated side refuse writes and return an error. Nothing ever diverges and no data is lost, but the cut-off region is down for writes until the network returns.

Active-passive doesn't escape this so much as pre-answer it: with a single leader, only the leader's side can write, so it stays consistent by construction — unless a botched failover promotes a second leader while the first is only unreachable, not dead, which hands you two leaders and the very split brain you were avoiding. The defense is fencing: a promotion must be blessed by a majority agreeing the old leader is gone (the consensus you'll recognize from leader election), so a lone region can never crown itself. Whichever shape you choose, a partition is where you find out what you truly decided, and a real region really does vanish sometimes, as us-east-1: The Region the Internet Leans On recounts.

When regions can't talk, drawn as a partition. The link between two active-active regions is cut by a lightning strike, and each side still has users demanding writes. Two futures fork from that moment. STAY AVAILABLE (top): both sides keep accepting writes, so the same counter becomes 5 in one region and 7 in the other — the system has split into two brains that disagree, and when the link heals someone has to reconcile them and lose data. STAY CONSISTENT (bottom): only the side that can still reach a majority keeps writing; the isolated side refuses writes and shows an error, so nothing diverges, but those users are down until the link returns. Law band: a partition forces the choice CAP always warned about — one region either serves wrong data or serves no data.

The Data That Cannot Leave

One driver of multi-region isn't about latency or uptime at all, and it doesn't care about your architecture diagram: the law. Data-residency rules like the European Union's GDPR require that certain data about people in a place physically stay in that place. If you hold data about EU residents, you generally cannot copy it to a region in the United States, not for performance, not for a read replica, not even for a backup — the bytes are not allowed to leave.

That single constraint quietly makes one of your write strategies mandatory. You cannot run write-local replicating everything everywhere, because that sprays regulated data across borders. You are pushed to write-partitioned by geography: EU users' records live and stay in an EU region, US users' in a US region, and the partition key is a person's legal home rather than anything about latency. It also collides head-on with the disaster-recovery instinct from the previous lesson — the safe reflex is to keep a copy far away, but compliance may forbid exactly that, so your backups and failover targets for regulated data have to stay in-region too. When you see a system split cleanly along national lines with no cross-border replication, that is usually not an engineer's preference. It is a lawyer's requirement, and it is one of the most common reasons a perfectly regional product suddenly has to think about regions at all.

Key Takeaways

Go multi-region for latency, survival, or law — and usually for none of them. One region across several availability zones is enough for most systems; reach across regions only when close users, more-than-four-nines uptime, or a data-residency rule demands it.
Active-passive is simple; active-active is fast. One writing region can never conflict but makes far users wait and needs a real failover. Every region writing is fast and failover-free but opens the conflict problem.
Reads are easy; writes are the whole game. Serve reads from the nearest copy, and pick a write strategy: write-global (one home, no conflicts, slow far writes), write-local (fast, but conflicts to resolve), or write-partitioned (a home region per record — fast and conflict-free for local data).
Across regions you get low latency or strong consistency, not both. The speed of light taxes every cross-region agreement, so synchronous is correct-but-slow and asynchronous is fast-but-stale. This is PACELC's else, and you choose per write.
A partition forces the CAP choice for real. Keep writing on both sides and split into two disagreeing brains, or freeze the minority side and go down there. Fence your failovers so a lost leader can't become two.

Multi-region shrinks your worst-case blast radius from everything to one region — a real and large win. But a region is still enormous, and the failures that hurt most now aren't lost data centers; they're a bad deploy or a single poison request that takes down the whole region at once, everywhere inside it, because everything in that region shares fate. The next idea shrinks the blast radius far below a region, on purpose, by slicing each region into many small, isolated copies that fail one at a time. That is Cell-Based Architecture & Deployment Stamps.