Event Sourcing

CQRS ended on a loose thread, pulled on purpose: the command model kept only its current state, but the event log kept the whole story, and the log could rebuild any read model you wanted, whenever you wanted. So why is the state the source of truth and not the log?

This lesson takes the question seriously and gives the honest answer: sometimes it should be. Event sourcing stores every change to your data as an immutable, past-tense event in an append-only log, and makes that log the system of record. Current state stops being the thing you protect and becomes something you derive: replay the events from the beginning and you get today. Stop replaying halfway and you get last Tuesday.

That one inversion buys you things no state-first system can offer: an audit trail that cannot disagree with the data because it is the data, the ability to reconstruct any moment in history, and the power to answer questions nobody thought to ask when the events were written. It also sends bills that state-first systems never see, and by the end you will know both columns well enough to say which contexts have earned the pattern. Almost none have. The ones that have tend to involve money.

Event sourcing gets introduced as an exotic pattern, so start with the disarming truth: it is the oldest data model in this course, and you have trusted it your whole life.

Your bank account is not a number that gets overwritten. It is a ledger: a list of transactions, each one dated and permanent, and the balance is whatever the transactions add up to. When the bank makes a mistake, no clerk erases a row. They append a correcting entry, and the ledger keeps both the error and the fix in plain view. Accountants have worked this way for five centuries, for the same reason event sourcing exists: a record you can edit is a record you cannot trust. Double-entry bookkeeping survived every technology change since the Medici because auditors will accept nothing less.

Software already agrees, in two places you have visited. WAL & Durability: Why Committed Data Survives showed that inside every serious database, the write-ahead log is written first and the tables are updated after; when the machine dies mid-write, recovery replays the log, because the log is what the engine actually trusts. And Queue Models: Broker (RabbitMQ) vs Log (Kafka) promised you would need the log model again: consumers reading an append-only sequence at their own pace, replaying from any offset. Both of those systems treat the log as the truth internally and hide it from you.

Event sourcing is one decision: stop hiding it. Promote the log from an internal recovery trick to the public data model of your service.

Three rules make the pattern, and each earns a close look.

Events are past-tense facts. Not SetBalance(30) but FundsWithdrawn(20). The grammar is doing real work: a command is a request that can still be refused, while an event is history, something that already happened, named so nobody downstream mistakes it for an instruction. An event carries what happened and the context that made it true, and once appended it never changes. That is why validation moves to the front door: the command side checks every rule before the event is written, because after the write, the fact is permanent. A state-first system can quietly fix bad data with an UPDATE. An event-sourced system cannot, so the bouncer at the door matters more here than anywhere else in this course.

State is a fold. To know the current balance, start from nothing and apply every event in order: opened, plus fifty, minus twenty, plus ten, minus five. Functional programmers call this a fold; everyone else calls it replay. The number you end with, thirty-five, is not stored as truth anywhere. It is computed, cached wherever convenient, and disposable, because the log can produce it again on demand. Each entity keeps its own small history, the wallet's events, the order's events, called that entity's stream, and the entity a stream belongs to is what this pattern's dialect calls an aggregate. When a service needs to decide whether a withdrawal is allowed, it loads that wallet's stream, folds it into a working state in memory, checks the rule, and appends the new event. The fold is the read; the append is the write. And because two commands can race on the same stream, the append carries the version the fold last saw: if history has grown in the meantime, the store refuses, and the command re-folds and retries, which is Version Conflicts' optimistic concurrency doing its exact job in a new house.

The past does not change. When event #3 turns out to be wrong, recorded as a twenty when the receipt says twenty-five, you do not edit it. You append a correction: AdjustmentMade(-5, corrects: #3). The accountant's move, exactly. The error stays visible, the fix stays visible, and anyone replaying the log passes through both, which means the log never lies about what the system believed at any moment, including the moments it believed something wrong. This is the rule that feels most alien coming from CRUD, and it is the entire source of the pattern's power: every guarantee in the next two sections exists because nobody, ever, gets to rewrite history.

The obvious objection arrives fast: a busy account accumulates a hundred thousand events, and folding all of them on every load is absurd. It is, and the fix is boring on purpose.

Every N events, persist a snapshot: the folded state as of event N. Loading the account becomes latest snapshot + the events since it, which turns a hundred thousand replays into a handful. The discipline is in what a snapshot is allowed to be: a cache, nothing more. Delete every snapshot in the system and you have lost performance and zero truth, because the log can rebuild them all. The moment a team treats snapshots as the record and the log as backup, they have quietly reinvented the state-first database with extra steps, and every guarantee this lesson promises evaporates.

You have seen this exact relationship before: it is Materialized Views one more time. A precomputed answer, refreshed from the source of record, disposable by design. The pattern keeps reusing one idea and changing what counts as the source.

Four capabilities fall out of the inversion, and none of them can be bolted onto a state-first system without lying a little.

• An audit trail that cannot disagree with the data. Most systems that need audit keep two records: the real tables, and an audit log written beside them. Two records can drift, and drifted audit is worse than none, because it testifies with confidence to things that did not happen. Event sourcing collapses the pair: the audit log and the database are the same object, so disagreement is not a bug you must prevent but a state that cannot be expressed. Regulators and auditors notice the difference, which is why the pattern clusters around money.

• Time travel. Replay the log to any point and you are holding the system's state at that moment, not an approximation of it. What did this customer's cart contain the second before the checkout bug fired? Fold events zero through eighty-one and look. Support tickets, incident forensics, and what did we know and when compliance questions all reduce to choosing where to stop replaying.

• Answers to questions nobody asked yet. CQRS made this promise and event sourcing keeps it: when product asks for a view nobody imagined, average time between first visit and first purchase, say, a state-first system shrugs, because it overwrote the evidence. An event-sourced system replays three years of history through a new projector and hands back the answer as if the feature had existed from day one.

• Debugging by replay. A corrupted state in a CRUD system is a crime scene with no witnesses. Here, you replay the account's events one by one and watch the state go wrong in front of you, then fix the code and replay again to prove the fix.

One caveat guards all four: projectors must be pure. Replay means every event will be processed again, possibly years later, possibly thousands of times. Code that folds events into state must do only that. The projector that also sends a confirmation email will send it again on every rebuild, to the delight of no one; side effects belong behind gateways that remember what has already been done, the discipline Idempotency: Safe to Retry built and this pattern quietly depends on.

Theory holds still; a log you are appending to does not. The wallet below is a real event-sourced aggregate, and you are its only user. Type real amounts and watch commands become past-tense facts, or get refused at the door before history is written. Then use the two controls that make this pattern unlike anything else you have driven: try to edit an event that is already in the log and see what the store says, and drag the time-travel slider to fold the same permanent history to any moment you like. Before you leave, ask the log a question it was never designed for and watch a brand-new projection get built from old facts.

The moment worth carrying out of the drill is the refusal. Every other store in this course would have let you fix event #3 in place, and the whole promise of this pattern lives in the fact that this one would not.

Now the column that decides real adoption. Event sourcing's costs are structural, not incidental: each one follows directly from the immutability that makes the pattern valuable.

Your events outlive your code, and you pay for that forever. The event you write tonight will be replayed by code you have not written yet, in five years, by an engineer who has not been hired yet. Rename a field, split an event in two, change a currency convention, and every historical event still carries the old shape, because immutable means immutable. Teams survive this with upcasters: translators that lift 2019-shaped events into 2026 shapes at read time, stacked version on version, tested against real history. It is Schema Evolution & API Versioning again, with the compatibility window stretched from months to the lifetime of the system. Veterans of the pattern name this cost first, and unprompted.

Immutability meets the right to be forgotten, and loses, unless you plan. GDPR says users can demand deletion; the log says nothing is ever deleted. The working escape is crypto-shredding: encrypt each user's personal fields with a key belonging to that user alone, and when the deletion request arrives, destroy the key. The events remain, structurally intact and replayable, but the personal data inside them is permanently unreadable, which regulators have accepted as erasure. The key-management machinery that makes this trustworthy is its own discipline, and Encryption at Rest & Secrets Management takes it up properly.

The log only grows. Snapshots fix replay time; nothing fixes storage, because deleting old events is deleting the truth. Ledger-shaped domains mostly shrug, since storage is cheap and history is the product. But compacting away the past quietly cancels time travel and late-arriving projections, so it is a decision about what the system promises, not a cleanup task.

Bugs write history too. A bad UPDATE in a CRUD system is repaired and forgotten. A bad event is a permanent fact that every future replay must survive, correctable but never removable. Validation at the door, replayed-against-history tests, and correction events are the whole defense.

And the familiar bill carries over unchanged: current state served from projections lags the log by the same window CQRS taught you to manage, author pinning, version waits, honest spinners and all.

CQRS was the minority case; event sourcing is the minority of the minority, and the test is one question: is your domain shaped like a ledger?

Some domains are. Payments and wallets, where the transaction history is the product and auditors are a design constraint. Inventory movements, where every discrepancy investigation is a replay. Compliance-heavy workflows, insurance claims, medical orders, trading, where what did we know and when is asked under oath. Collaborative editing, where the operation history is what makes undo and merge possible at all. In these contexts the log is not an architectural indulgence; it is the requirement, and event sourcing is simply the design that stops pretending otherwise.

Most domains are not. Profiles, settings, catalogs, session state: nobody audits them, nobody time-travels them, and their current value is the only value anyone will ever ask for. Event-sourcing those is paying the schema-evolution tax and the projection lag forever, for superpowers no one will use, and the wreckage of teams that event-sourced everything is a large part of the pattern's scary reputation.

Two boundary clarifications keep the judgment sharp. First, the pairing: event sourcing and CQRS travel together because a log answers current-state questions badly, so projections become the read side. Paired is not welded: CQRS runs happily over an ordinary database, and the two remain separate decisions. Second, the infrastructure: Kafka gives you a superb append-only log, but an event store also wants per-aggregate streams, optimistic append checks, and retention measured in the lifetime of the business, which is why dedicated stores exist. The distinction gets its full treatment when this course turns Kafka inside out in the interview-preparation lessons.

So the pattern's placement in your head should be: a specialist tool, adopted per bounded context, exactly where history is the product. Database per Service gave each service its own store; this lesson says that for one or two of those services, the right store is a log.

• The inversion is the whole pattern. The append-only event log is the system of record; every state, current included, is a rebuildable cache of it. You trusted this model before you could code, because your bank account is one.

• Three rules carry everything. Events are immutable past-tense facts, validated hard at the door because they cannot be unwritten. State is a fold over the log. The past is never edited, only corrected by appending, the accountant's move.

• Snapshots are caches of the fold. They buy load time, hold no truth, and the day a team treats them as the record, the pattern has silently died.

• The superpowers are exclusive. Audit that cannot disagree with the data, state at any moment in history, answers to questions asked years after the evidence, and debugging by replay. None survive impure projectors: fold in the projector, side effects behind idempotent gateways.

• The bills are structural. Events outlive code, so upcasting is forever. Deletion law meets immutability through crypto-shredding. The log only grows, bugs become permanent facts, and projection lag carries over from CQRS unchanged.

• Adopt it where the domain is a ledger. Money, inventory, compliance, collaboration: history is the product, so store the history. Everywhere else, current state is the only question, and a table answers it better than a log.

Every pattern in this course so far has organized one service's world: its gateway, its config, its data, now its history. But the events in tonight's log do not stop at the service boundary. An OrderPlaced event is also a message to inventory, to payment, to shipping, and suddenly one business action is a story told across four services, each with its own database and no shared transaction to keep them honest. Saga: The Data-Consistency Mechanics built the machinery for that problem; the next lesson, Saga: Orchestration vs Choreography, settles the question the machinery left open: who tells the story, a conductor, or the band listening to itself?