Tiered & Cold Storage

You run a photo service with a billion photos in it. The ones uploaded this morning are getting viewed thousands of times — friends tagging, feeds refreshing, thumbnails rendering. A photo from 2019 might get opened once this whole year, if ever. And yet, on S3 Standard, you pay the exact same $0.023 per gigabyte per month for both. Multiply that across a billion mostly-forgotten photos and you're burning money to keep cold data on hot, expensive storage.

This is the central fact of storage at scale: data has a temperature, and it cools with age. It's hot for its first few days after creation, warm for a while, then goes cold and mostly stays there. The pattern is astonishingly consistent — across almost every real system, 70 to 90 percent of stored data is cold, rarely or never read again. Backups, logs, old media, compliance archives, last year's photos: written once, kept for years, read almost never.

So the move is obvious once you see it: don't pay hot prices for cold data. Keep fresh, frequently-read data on fast, expensive storage; as it cools, move it down a ladder of cheaper, slower tiers. Match the storage tier to the data's temperature. Here's the whole idea in two curves — access falling with age, and the price you pay falling right behind it.

The object storage you met in Block vs File vs Object Storage isn't sold at one price — it's a ladder of storage classes, each trading access speed for cost. On S3, top to bottom (per GB-month, and these are real 2026 US-East prices):

• S3 Standard — $0.023. Hot data. Millisecond reads, no retrieval fee. The default.
• S3 Standard-IA (Infrequent Access) — $0.0125. Still instant to read, but ~half the storage cost; you pay a small per-GB fee when you read. For data you keep but touch occasionally.
• S3 Glacier Instant Retrieval — $0.004. Archive prices with millisecond access — for data read maybe once a quarter but needed fast when it is.
• S3 Glacier Flexible Retrieval — $0.0036. Cheap, but reads take minutes to hours (you request a restore, then wait).
• S3 Glacier Deep Archive — $0.00099. The floor. Tape-class economics for write-once-keep-forever data, but reads take up to 12 hours. (These cheap tiers keep your bytes durable on spun-down disks with erasure coding — the cheap-durability trick from Erasure Coding vs Replication — which is part of how the price gets this low.)

Look at the spread: Standard to Deep Archive is $0.023 down to$0.00099 — about 23× cheaper. In money that's roughly $24 per terabyte-month versus$1.01 — the difference between a $24,000 monthly bill and a$1,000 one for the same petabyte. That gap is the entire reason cold storage exists. But notice what's quietly changing as you go down that ladder, beyond just the price.

Going down the ladder, two other things move — in the opposite direction from the price. Retrieval gets expensive, and it gets slow. That's the trap, and it's where naive cost-cutting goes wrong.

Retrieval cost. The cheap tiers charge you per gigabyte read back. Deep Archive is ~$0.02/GB to retrieve, Glacier Instant$0.03/GB. Storage is nearly free; reading is not. Retrieval latency. The truly cheap tiers aren't normal storage you can GET — you issue a restore job and wait: Glacier Flexible takes minutes to hours, Deep Archive takes up to 12 hours before the bytes are even available. And minimum storage duration — IA bills you for at least 30 days, Glacier 90, Deep Archive 180 — so deleting or moving data early still costs you the remainder. Cold storage punishes anything that churns.

Here's the counterintuitive part, and it's worth getting right because it's exactly what a good follow-up question probes. Storage in Deep Archive is so cheap (23× less) that the per-GB retrieval fee only overtakes the storage savings if you read the data more than about 1.1 times per gigabyte per month — which is to say, only if the data is genuinely hot, in which case it shouldn't be in cold storage at all. So the steady-state bill almost always favors tiering. The real dangers are the two things a spreadsheet misses:

1. Latency is a correctness problem, not a cost line. If a user — or a lawsuit — needs a Deep-Archive object now, "12 hours" is an outage. You tiered data that had a latency requirement it can no longer meet.
2. The one-time retrieval storm. A re-processing job, an audit, or a disaster recovery that reads the whole dataset back detonates the bill: pulling 1 PB back from Deep Archive is 1,048,576 GB × $0.02 = ~$20,972 in retrieval alone — a single event that can erase months of storage savings. Tier the data that will stay cold; the moment cold data gets read en masse, you pay for the discount.

Storage down, retrieval and latency up — the whole trade on one ladder:

You don't move billions of objects by hand. You write a lifecycle policy — a set of age-based rules on a bucket or prefix — and S3 transitions objects for you as they cross each age threshold. A typical one reads like the cooling curve itself:

lifecycle policy  (photos/*)
  age  0d  → S3 Standard          (hot: viewed constantly)
  age 30d  → Standard-IA          (cooling: occasional views)
  age 90d  → Glacier Flexible     (cold: rarely opened)
  age 365d → Glacier Deep Archive (frozen: keep for compliance)
  age  7y  → expire (delete)      (optional last rule)

You're encoding when data cools as age thresholds, and S3 does the moving. How much does it save? Real math, on a modeled 1 PB of photos with a realistic access-decay curve and the actual AWS prices:

1 PB · monthly cost           storage   retrieval    TOTAL
  all-hot (S3 Standard)        $24,117        $0     $24,117
  sane lifecycle (above)        $2,236      $335      $2,571     ← 89% cheaper

An 89% cut — about $258,000 a year — for one policy and zero code, because most of that petabyte is old and cold and doesn't need to sit in Standard. But the thresholds matter: set them too aggressively and you push still-warm data into Deep Archive, trading a tiny extra storage saving for 12-hour reads and a fatter retrieval bill on the accesses that are still happening. Finding the thresholds that match your actual access curve is the real skill — so drive it: pick a dataset, drag the transitions, and watch the storage bill, the retrieval bill, and the worst-case latency fight each other.

Lifecycle policies assume you know the cooling curve — that a photo is cold by 90 days, a log by 30. But plenty of data doesn't cool on a schedule: an old document nobody's touched in a year suddenly trends; a dormant dataset gets rediscovered. Age is only a proxy for the thing you actually care about, which is how recently something was accessed.

S3 Intelligent-Tiering tiers by that directly. Instead of age rules, AWS monitors each object and moves it automatically: not accessed for 30 days → Infrequent tier; not accessed for 90 → Archive Instant; and it springs back to the frequent tier the instant anyone reads it. The pitch is that you never guess and never get burned by a retrieval fee — Intelligent-Tiering has no retrieval charges and no minimum-duration penalty for its automatic moves. The cost is a small monitoring fee of $0.0025 per 1,000 objects per month for watching access patterns.

So the choice is really time-based vs access-based. If you know the curve — logs, backups, regulatory archives with a predictable life — hard-coded lifecycle policies are cheaper and explicit. If access is unpredictable, Intelligent-Tiering earns its monitoring fee by never stranding hot data in a cold tier. The one place it stings is millions of tiny objects, where the per-object monitoring fee can dwarf the storage it's optimizing — there, tier big, or don't tier at all.

Tiering is one of the highest-leverage cost moves in a system, and one of the easiest to botch. The ways it goes wrong all share a root cause: tiering data whose access pattern doesn't match the tier.

• The retrieval-bill surprise. Someone kicks off a backfill or an ML re-training run that reads a petabyte back from Glacier, and the retrieval line on the invoice dwarfs a year of the storage savings that justified the move. Tier data that will stay cold.
• The latency landmine. A "restore" you assumed was instant is a 12-hour job, and now an incident, an audit, or a customer is waiting half a day for data you archived too deep. Tier to the worst-case latency you must support, not just the price.
• The churn penalty. You transition or delete data before its minimum duration and pay the pro-rated remainder anyway — cold tiers punish anything short-lived or mutable.
• The tiny-object tax. IA and Glacier bill a minimum object size and per-object overhead; Intelligent-Tiering's per-object fee adds up. Millions of 4 KB objects cost more to tier than to leave hot. Aggregate small objects, or leave them alone.

The through-line: don't tier by wishful thinking, tier by measured access. Cheap storage is a reward for data that truly went cold — and a trap for data that didn't.

• ⭐ Data has a temperature that cools with age. Most stored data (70–90%) is cold. Don't pay hot prices for it — move it down a ladder of tiers as it cools.
• ⭐ The price ladder spans ~23×. S3 Standard $0.023 → Deep Archive$0.00099 per GB-month ($24 vs$1.01 per TB-month). That spread is why cold storage exists.
• ⭐⭐ Cheaper to store = costlier and slower to read. Cold tiers charge per-GB retrieval and make you wait (Deep Archive: up to 12 h) with minimum durations (30/90/180 days). Steady-state, tiering almost always wins; the real dangers are latency SLAs and one-time retrieval storms (1 PB back from Deep Archive ≈ $20,972).
• Automate it two ways. Age-based lifecycle policies when you know the cooling curve (a modeled 1 PB dropped 89%); access-based Intelligent-Tiering when you don't (monitoring fee, no retrieval fee, springs back on access).
• Match the tier to the access pattern, not the age or the sticker price. Tier what stays cold; keep what might be read fast — or read a lot — hot.

That closes the storage tour: block, file, and object; a blob store's internals; distributed file systems; erasure coding for cheap durability; chunking big media; transcoding pipelines; and now the lifecycle that ages data from hot to frozen. Next is the §7 Checkpoint — you'll be handed a few raw data shapes and asked to pick the right storage and tier for each, and defend it.