Sequential vs Random Access: Why Order Matters

Two programs read the same 4 GB file from the same hard disk.

The first finishes in about 20 seconds. The second takes almost four hours.

Same machine. Same file. Same bytes. The only difference is the order in which they asked for them: the first read the file front to back; the second read the same blocks in random order. That one variable — order — moved performance by a factor of several hundred.

This lesson is about why. Not the folklore version ("disks like sequential access") but the actual physics: what a single out-of-order read costs, why the bill exists on SSDs that have no moving parts, and why even RAM — random access memory, it's in the name — quietly rewards reading in order.

It's worth getting this deep in your bones, because some of the most important machinery in modern infrastructure is a direct monetization of this one fact: the write-ahead log that makes your database durable, the LSM trees inside Cassandra and RocksDB, and Kafka's entire storage design are all, at heart, tricks for converting random work into sequential work. We'll meet each of them properly later in the course — today you learn the physics they're built on.

Scope: this lesson explains the access-pattern law on all three tiers (disk, SSD, RAM). How databases and queues exploit it — WAL mechanics, SSTables, compaction — is deliberately deferred to WAL & Durability and LSM Trees & SSTables in the data-systems course.

A spinning disk is a record player that can write. The platter spins at a fixed speed (7,200 RPM on a typical server or desktop drive), and a single actuator arm swings a read/write head across it. To read a block, the head must be physically above it. That sentence is the entire lesson; everything else is arithmetic.

Ask that disk for one 4 KB block somewhere far from where the head currently sits, and the hardware runs a three-step errand:

1. Seek — the arm swings to the right track. Average on a desktop drive: ~8 ms.
2. Rotate — the head waits for the block to spin underneath it. On average that's half a revolution; at 7,200 RPM a full spin takes 8.3 ms, so: ~4.2 ms.
3. Transfer — the block streams under the head: ~0.02 ms for 4 KB.

Total: ~12.5 milliseconds, of which reading the data is 0.02. In other words, 99.8% of the cost of a random read is getting into position, and 0.2% is the read. You hired a courier, and the delivery is free — but you pay for every mile of the drive.

Now read the next block instead — the one physically adjacent. No seek. No rotational wait. The platter is already spinning the next bytes under a head that hasn't moved, at 180–200+ MB/s. Sequential access isn't "a bit faster." It's a different transaction entirely: you pay the positioning bill once, then the disk pours data at full speed for as long as you stay in order.

Scale the bill up and it becomes a ceiling nothing can raise.

At ~12.5 ms per random operation, one disk arm can complete about 80 random reads per second — that's the disk's random IOPS, and it doesn't matter how clever your code is. An enterprise 15,000-RPM drive with faster seeks gets to maybe 200 IOPS. That's the ceiling. When a storage engineer says "we needed 80,000 IOPS so we bought 400 spindles," this arithmetic is the whole story.

Throughput follows directly:

Throughput = IOPS × I/O size

• Random 4 KB reads: 80 IOPS × 4 KB ≈ 0.3 MB/s. (Measured on real 7,200-RPM drives: 0.5–1.5 MB/s — same order.)
• Random 1 MB reads: ~75 IOPS × 1 MB ≈ 75 MB/s — 200× better, from nothing but bigger requests.
• Fully sequential: ~200 MB/s — the positioning bill amortized to zero.

That middle line is one of the most-pulled levers in systems engineering: if you can't be perfectly sequential, be bigger. Every random operation pays the same ~12.5 ms toll whether it carries 4 KB or 1 MB, so carrying more per trip dilutes the toll. Databases read whole pages instead of rows, Kafka moves batches instead of messages, and operating systems read ahead in large block multiples — all of them are amortizing the same bill.

Run the numbers on our opening mystery and it stops being mysterious: 4 GB sequentially at 200 MB/s ≈ 20 seconds. The same 4 GB as a million random 4 KB reads at 12.5 ms each ≈ 13,000 seconds — about 3.6 hours. Nothing was broken. Nobody wrote bad code. The second program just asked in the wrong order.

In 2009, Adam Jacobs published a measurement in Communications of the ACM ("The Pathologies of Big Data") that most engineers still find hard to believe. Reading simple 4-byte values on then-modern hardware:

Where	In order	Random	Gap
Memory	~358M values/s	~36.7M values/s	~10×
Disk	~53.2M values/s	~316 values/s	~150,000×

Two things in that table should stop you cold.

First, the disk gap isn't 100× at this grain — it's five orders of magnitude. Second, look across the rows: sequential disk (53.2M/s) beats random memory (36.7M/s). A spinning platter, streaming in order, outran RAM being accessed out of order.

Sit with that, because it breaks the mental model most people carry — the tidy ladder from the last lesson where memory is simply "fast" and disk is simply "slow." The truth is two-dimensional: performance = where your data lives × the order you ask for it. A slow tier accessed in order can beat a fast tier accessed at random. The memory hierarchy tells you the price of each floor; access pattern decides whether you take the elevator or the stairs.

This is why the systems you'll meet later go to extraordinary lengths — extra copies of data, background compaction, whole storage engines — just to turn random work into sequential work. The prize isn't a percentage. It's orders of magnitude.

Numbers in a table are one thing; watching the collapse is another. Below, two readers race through the same file on the same device — one in order, one jumping randomly. Before you start: predict the gap. Then pick a device, run the race, and drag the block size mid-race to feel the amortization lever move.

Things worth trying: race the HDD at 4 KB (the brutal case), then slide block size up and watch random claw back throughput; switch to SSD and notice the gap shrinks but doesn't vanish; switch to RAM and see the same shape one more time, smaller.

The obvious objection: "That's spinning-rust physics. SSDs have no arm and no platter — surely order stops mattering?" It's the single most common misconception about storage, so let's kill it with numbers.

A modern NVMe drive advertises ~7,000 MB/s sequential (Gen4; Gen5 hits ~13,000). Ask the same drive for 4 KB blocks in random order, one dependent read at a time — read a block, look at it, decide what to read next, the way real request paths behave — and you get roughly 70–140 MB/s. The gap is 50–100×. No arm. No rotation. Still two orders of magnitude.

Why? On an SSD the bill changes currency but not existence:

• Reads pay per-operation latency. Every read is a full command round-trip (~80–100 µs). A sequential stream lets the drive and OS predict what's next — read-ahead fetches big contiguous runs before you ask. Random, dependent reads can't be predicted, so every one pays full fare, one at a time.
• Writes pay the erase-block tax. Flash reads and writes in pages (4–16 KB) but can only erase in whole erase blocks (megabytes). Random writes sprinkle live data across many blocks; to reclaim space, the drive's garbage collector must copy surviving pages elsewhere and erase whole blocks — so one logical write becomes several physical ones (write amplification). Sequential writes fill blocks end-to-end and die together, so cleanup is nearly free. On flash, sequential writes run about two orders of magnitude faster than random.

The honest fine print: give an NVMe drive hundreds of independent requests at once (queue depth 128+) and its controller stripes them across many flash dies in parallel — random bandwidth can then approach sequential. That's real, and it matters for batch analytics and benchmark charts. But notice the condition: independent and massively parallel. A request path that must read a value to decide its next read is neither — it's queue depth 1, and it pays the full gap. Order sets you free; dependence chains you.

And the trend is the opposite of what people assume: Gen5 doubled sequential throughput over Gen4 while random QD1 improved only ~20–30%. The gap is widening. Betting your design on "random will catch up" has lost for forty straight years.

One tier left, and it's the one with "random access" in its name. Surely RAM is flat?

Measure it. Walk a large array in order: each access costs ~12.8 ns (~37 CPU cycles). Chase pointers through the same data in random order: ~167 ns (~481 cycles) per access — ~13× slower, on identical hardware, over identical bytes. While pointer-chasing, the CPU completes roughly one instruction every 8 cycles; it spends the rest waiting for memory.

You already know the mechanism — it's the locality machinery from The Memory Hierarchy, now seen from the access-pattern side:

• Cache lines. Memory moves in 64-byte lines. Read element 0 of an array of 4-byte ints and elements 1–15 arrive free. Sequential access uses all 16; random access uses 1 and wastes 15.
• The prefetcher. The CPU watches your address stream, and if it spots a stride — every 64 bytes, every 128 — it silently fetches ahead, hiding memory latency entirely. But a pointer chase gives it nothing to spot: the next address is unknowable until the current load returns. Prediction is impossible when each step depends on the last.

That last sentence should sound familiar — it's the same dependence trap as SSD queue depth 1, wearing a different costume. Across all three tiers, the deep rule is the same: hardware is spectacular at predictable, independent work and helpless against dependent, unpredictable work. Order is how you make yourself predictable.

This is why "array vs. linked list" — the most boring question in a data-structures course — is worth 1–2 orders of magnitude in the real world, and why performance engineers say things like "the fastest data structure is a flat array, sorted by what you'll read next."

Once you know sequential access is 2–5 orders of magnitude cheaper, a design strategy falls out by itself: never do random work on the critical path — do sequential work now, and reshape it into random work later, in the background, in bulk. Three of the most important systems in modern infrastructure are exactly this move:

The write-ahead log (WAL). When you commit a transaction, Postgres does not run around updating table pages scattered across the disk. It appends a description of the change to one sequential log file and flushes that — a single cheap, in-order write buys full durability. The scattered page updates happen later, lazily, batched. Every serious database works this way; the full mechanics get their own lesson (WAL & Durability).

LSM trees. Cassandra, RocksDB, and LevelDB refuse to do random writes at all: writes land in memory and flush as immutable, sorted, sequential files (SSTables). Random-write workload in, sequential-write workload out. The catch — background compaction, and reads that must consult several files — is a deliberate purchase we'll price in LSM Trees & SSTables.

Kafka. The famous line from its design docs: don't fear the filesystem. Each partition is an append-only log; reads are in-order scans; segment files are pre-allocated so data stays contiguous; the OS page cache read-ahead and write-behind do the heavy lifting. On six commodity 7,200-RPM disks: linear writes at ~600 MB/s vs random writes at ~100 KB/s — a 6,000× difference the architecture harvests by never asking the disk to jump. Kafka isn't fast despite using disks. It's fast because it uses them in the only order they love.

Spot the shared skeleton: append now, reorganize later. When you meet these systems in the data-systems course, you won't be memorizing architectures — you'll be recognizing this lesson, applied.

A law this strong invites cargo-culting, so here's the honest boundary. Random access is the right choice surprisingly often:

• When the working set fits in RAM. If your hot data lives in cache or memory, "random" reads never touch the slow tier — the 13× RAM penalty is real but rarely your bottleneck. This is most OLTP systems on a good day, and it's why just add RAM genuinely works until it doesn't.
• When requests are independent and massively parallel. A search index fanning out thousands of unrelated lookups can keep an NVMe drive's queue full and let internal parallelism eat the latency. The law punishes dependent randomness, not randomness itself.
• When you're buying something with it. B-tree databases (Postgres, MySQL) accept scattered page writes precisely because a B-tree keeps data sorted, making point lookups and range scans cheap. LSM engines make the opposite trade — sequential writes, messier reads. Neither is wrong: order is a currency, and every storage engine chooses where to spend it — on the write path or the read path.

The staff-engineer takeaway isn't "sequential good, random bad." It's a question you now ask reflexively about any data-heavy design: what's the access pattern, and who's paying the positioning bill — the client at request time, or a background process in bulk? Systems feel fast when the answer is "the background process."

"SSDs have no moving parts, so access order doesn't matter." The bill changes currency, not existence: per-operation latency on dependent reads (50–100× at queue depth 1) and the erase-block/garbage-collection tax on random writes.

"RAM is random-access memory — it's literally the name." The name means addressable in any order, not equally fast in any order. Cache lines and the prefetcher make in-order access ~13× faster than pointer-chasing.

"When I commit, the database writes my row where it lives." It appends your change to a sequential log first and defers the scattered page writes. Your commit latency is a sequential write on purpose.

"Kafka is fast because it avoids the disk." Backwards — Kafka embraces the disk and refuses to use it randomly. Append-only logs + page cache = ~600 MB/s from commodity spindles.

"The box says 7 GB/s, so that's what my app gets." That number is sequential, huge requests, deep queues. Your app's dependent 4 KB reads at queue depth 1 get ~1–2% of it.

"New hardware will erase the gap." Gen5 doubled sequential and nudged random QD1 by ~20–30%. The gap has widened every generation for forty years — it's physics plus economics, not a missing feature.

Ten minutes with fio (the standard storage benchmark, available on every platform) turns this lesson from something you read into something you measured. Write your predictions down first — sequential MB/s, random MB/s, the ratio — then run:

# Sequential read, 1 MB blocks (the streaming case)
fio --name=seq --rw=read --bs=1M --size=1g --filename=testfile --direct=1

# Random read, 4 KB blocks, queue depth 1 (the dependent-read case)
fio --name=rand --rw=randread --bs=4k --size=1g --filename=testfile --direct=1 --iodepth=1

# Random read again, but with parallelism (the honest counter-case)
fio --name=randqd --rw=randread --bs=4k --size=1g --filename=testfile --direct=1 --iodepth=128
rm testfile

Three predictions to test: (1) the seq-vs-rand-QD1 ratio on your SSD lands in the 20–100× range; (2) iodepth=128 claws back a large chunk of the gap — that's die-level parallelism, not magic; (3) if you can find a machine with a spinning disk, the QD1 ratio jumps into the hundreds. The gap between your prediction and the measurement is where the learning is.

The whole lesson in five lines:

• A random access pays a positioning bill; sequential access pays it once and streams. On a hard disk the bill is ~12.5 ms of seek + rotation — 99.8% of the cost of a 4 KB read.
• Throughput = IOPS × I/O size. Can't be sequential? Be bigger — amortization is the second-best lever.
• The law survives the loss of moving parts: SSDs pay per-operation latency (50–100× at queue depth 1) and the erase-block tax; even RAM pays ~13× when you defeat its cache lines and prefetcher. Sequential disk can beat random memory (Jacobs: 53.2M vs 36.7M values/s).
• The deep rule: hardware rewards predictable, independent access and punishes dependent, unpredictable access — on every tier.
• WAL, LSM trees, and Kafka are this physics turned into architecture: append now, reorganize later.

You now have the two physical dimensions of data performance: where data lives (the memory hierarchy) and the order you touch it (this lesson). Next we climb from hardware to the operating system's units of work — Processes vs Threads — where the question stops being "how fast can I read?" and becomes "how many things can I do at once, and what do they share?"