How Training Actually Works (Backprop Recap)

Introduction

Section 1 was about whether to fine-tune. Section 2 — Fine-Tuning Mechanics — is about how it actually works under the hood, so that when you read about memory bottlenecks, quantization, LoRA, and learning rates, none of it is magic. And it all rests on one thing: the training loop.

Here's the liberating truth this lesson delivers: fine-tuning is not a special technique — it's the exact same loop that pre-trained the model, continued on your data with a small learning rate. Pre-training, fine-tuning, even LoRA — same four steps, repeated. Understand this loop once and the entire rest of the container becomes obvious: memory is just "how much of this loop do I have to hold in the GPU," and LoRA is just "do the loop on a tiny slice of the weights." Let's recap it from first principles.

An infographic titled 'How Training Actually Works', a backpropagation recap and the opener of section two on fine-tuning mechanics, showing that all training is one loop repeated. The loop has four steps connected by arrows: forward pass, where input flows through the network to predict a probability distribution over the next token; loss, where cross-entropy measures how wrong that prediction was as the negative log-likelihood of the true next token; backward pass, where backpropagation applies the chain rule to compute the gradient of the loss with respect to every weight, the direction and magnitude to nudge each weight to reduce loss; and update, where the optimizer takes a gradient-descent step, new weight equals old weight minus the learning rate times the gradient. Then it repeats for every batch and a few epochs. The optimizer is AdamW, the default for virtually every large language model, which keeps a running momentum and variance per weight and therefore needs about twice the model size in optimizer state, a fact that sets up the memory bottleneck. The learning rate is the single most important knob: typical fine-tuning values are one to five times ten to the minus five, because too high a rate makes the loss overshoot and diverge so the model blows up, while too low makes it crawl. Batch sizes run from four to sixty-four, and one to three epochs is usually enough for fine-tuning because the model already learned from pre-training, with more epochs risking overfitting. The crucial point is that fine-tuning is the exact same loop, continued on your data with a small learning rate, which is why understanding it explains the memory math, why LoRA updates only a tiny adapter to need fewer gradients and optimizer states, and how quantization and hyperparameters work.

All Training Is One Loop, Repeated

Strip away the jargon and every model — GPT, Claude, Llama — learns with the same four-step loop, run over batches of data for a few passes (epochs):

① FORWARD → ② LOSS → ③ BACKWARD → ④ UPDATE → ↻ repeat

Forward pass — push an input through the network; out comes a prediction.
Loss — measure how wrong the prediction was, as a single number.
Backward pass (backpropagation) — work out which way to nudge every weight to make the loss smaller.
Update — actually nudge the weights a little in that direction (the optimizer step).

Repeat a few million times and the loss drops — the model gets less wrong. That's it. That's learning. Now let's make each step concrete.

Forward Pass & the Loss (Cross-Entropy)

Forward pass. For a language model, the input is a sequence of tokens and the output is a probability distribution over the entire vocabulary for the next token — the model's guess at what comes next (exactly the next-token prediction you met in the foundations).

The loss. We need one number for how wrong that guess was. For language models that number is cross-entropy loss: it compares the model's predicted distribution to the truth (a one-hot vector — the real next token has probability 1) and computes the negative log-likelihood of the true token. In plain terms: if the model put high probability on the right next token, the loss is small; if it put high probability on the wrong one, the loss is large. This is the same objective as pre-training — fine-tuning just runs it on your examples (your format, your style, your domain), which is why it shifts behavior.

Backpropagation: Getting the Gradient

Now the clever part — the one Geoffrey Hinton & co. made famous. We have a loss number; we have millions (billions) of weights. Which way should each one move to reduce the loss? The answer is the gradient: the derivative of the loss with respect to each weight, ∂loss/∂w. It points in the direction of steepest increase in loss — so we'll step the opposite way.

Backpropagation is the efficient algorithm that computes this gradient for every weight in one backward sweep, by applying the chain rule from the loss back through each layer. The output is a gradient for every single weight — the same size as the model itself. (Hold that thought: that's the first half of why training is so memory-hungry — L190 (The Memory Bottleneck & Memory Math) is the whole story.)

The Optimizer Step: Gradient Descent & AdamW

With a gradient in hand, the update is one line of arithmetic — gradient descent:

w ← w − η · gradient

Move each weight a small step down the loss hill. The step size is η, the learning rate — the single most important hyperparameter in all of training.

In practice we don't use plain gradient descent; we use AdamW, the default optimizer for virtually every LLM (GPT, BERT, Llama, T5…). AdamW improves on raw descent by keeping, per weight, a running average of recent gradients (momentum, β₁≈0.9) and of their variance (β₂≈0.999) to scale the step adaptively. The catch — and it matters enormously for the next lesson — is that those two running averages are two extra numbers per weight, so AdamW's optimizer state is ~2× the size of the model. Weights + gradients + optimizer state is why full fine-tuning needs ~4× the model's size just to hold the training — the bottleneck L190 is all about, and the reason LoRA exists.

Learning Rate, Batches & Epochs

Three knobs control how the loop runs (you'll tune them properly in L194 (Fine-Tuning Hyperparameters & Tactics) — here's the intuition):

Learning rate (η) — the step size, and the knob that makes or breaks a run. Too high → the steps overshoot the minimum and the loss diverges (explodes — the model "blows up"); too low → it crawls and wastes compute. Typical fine-tuning values are 1e‑5 to 5e‑5 for full fine-tuning (a bit higher, ~1e‑4–3e‑4, for LoRA). You'll feel this in the lab below.
Batch size — how many examples you average the gradient over per step (commonly 4–64, limited by GPU memory). Bigger = smoother gradient, more memory.
Epochs — how many passes over your dataset. For fine-tuning, 1–3 is usually enough — the model already learned the world in pre-training; you're just nudging it. More epochs → overfitting (the L186 (Bad Reasons to Fine-Tune) trap), where it memorizes your data and forgets how to generalize.

See It: The Training Loop Lab

Drive the loop yourself. Drag the learning rate, then click Step (or Step ×10) and watch the ball roll down the loss landscape — and the loss curve respond. Three things to try: a good rate (~5e‑5) for a smooth descent; a tiny rate (1e‑5) to watch it crawl; and crank it up toward 1e‑3 to watch the loss diverge — the single most common fine-tuning failure, now something you can see.

Interactive: the Training Loop Lab. The user drives the four-step training loop on an arrow-connected pipeline, forward then loss then backward then update then repeat, by dragging the learning rate on a realistic fine-tuning scale from one times ten to the minus five to one times ten to the minus three, and clicking Step or Step times ten to run gradient-descent iterations. A loss-landscape panel shows a ball stepping downhill as the weight updates by new weight equals old weight minus learning rate times gradient on a convex bowl, the loop nodes display the live numbers each step (the loss percentage, the gradient, and the update with the current learning rate), a loss-over-steps sparkline records the trajectory, and a verdict reacts to the learning rate: too low crawls, a good rate descends smoothly, a bit too high oscillates back and forth across the minimum, and too high diverges so the loss explodes and the model blows up. It makes the universal dynamics of training, and why the learning rate is the most important knob, something the user feels rather than just reads.

Everything in code is just this loop. Here it is in full — and notice it's the same four lines whether you're pre-training, full-fine-tuning, or training a LoRA:

import torch
from torch.nn.functional import cross_entropy

# The ENTIRE training loop. This is ALL fine-tuning is, repeated over your data.
optimizer = torch.optim.AdamW(model.parameters(), lr=5e-5)   # the optimizer + learning rate

for epoch in range(3):                       # 1-3 epochs is usually enough for fine-tuning
    for batch in dataloader:                 # a batch of YOUR examples
        logits = model(batch.input_ids)              # 1) FORWARD: predict next-token distribution
        loss = cross_entropy(logits, batch.labels)   # 2) LOSS: how wrong (negative log-likelihood)
        loss.backward()                              # 3) BACKWARD: backprop -> a gradient per weight
        optimizer.step()                             # 4) UPDATE: w <- w - lr * grad  (AdamW)
        optimizer.zero_grad()                        # clear gradients for the next step

# That's it. What changes between pre-training, full FT, and LoRA is only:
#   - WHICH weights get gradients (all of them vs a tiny adapter -> L192/L193),
#   - the DATA you loop over, and the LEARNING RATE. The loop is identical.

Why This Sets Up the Whole Section

Hold onto this loop, because the next five lessons are all just consequences of it:

Memory (L190) — the loop must hold the weights + a gradient per weight + AdamW's ~2× optimizer state + activations in GPU memory. That's the bottleneck.
Quantization (L191 — Numerical Precision & Quantization) — store those numbers in fewer bits (e.g. 4-bit) to shrink the memory.
Full FT vs PEFT (L192) and LoRA & QLoRA (L193) — run the update step on only a tiny adapter instead of every weight, so you need gradients/optimizer state for a fraction of the parameters — the 90%+ savings from Section 1's cost lesson.
Hyperparameters (L194) — choosing the learning rate, batch size, and epochs you just met.

None of it is new machinery. It's all "how do I run this loop cheaper?"

🧪 Try It Yourself

Predict, then check. Before touching the lab: what will happen to the loss if the learning rate is 10× too high? Write your prediction (diverge? oscillate? crawl?). Then open the Training Loop Lab, set η to a good value and Step ×10 to confirm a smooth descent, then multiply η by ~10 and Step again — did it do what you predicted? Finally, explain in one sentence why a too-high learning rate makes the loss go up instead of down (hint: think about overshooting the bottom of the bowl).

Mental-Model Corrections

"Fine-tuning is a different algorithm from pre-training." No — it's the same loop (forward → loss → backward → update), just continued on your data with a smaller learning rate.
"Backprop changes the weights." Backprop only computes the gradients; the optimizer changes the weights (w ← w − η·g). Two separate steps.
"A bigger learning rate trains faster." Up to a point — then it overshoots and diverges. There's a sweet spot; past it, the loss explodes.
"The model is the only thing in GPU memory during training." Training also holds a gradient per weight and AdamW's ~2× optimizer state — that's why training needs far more memory than inference (L190).
"More epochs = better." For fine-tuning, 1–3 is usually enough; more overfits (memorizes your data, forgets how to generalize).

Key Takeaways

All training is one loop: forward (predict) → loss (how wrong — cross-entropy for LMs) → backward (backprop → a gradient per weight via the chain rule) → update (w ← w − η·g) → repeat.
Fine-tuning is the same loop, continued on your data with a small learning rate — what changes is which weights get updated, the data, and the LR.
The optimizer is AdamW (the LLM default); its momentum + variance per weight make optimizer state ~2× the model — a key driver of training memory.
Learning rate is the most important knob: too high → diverges (loss explodes), too low → crawls. Fine-tuning LRs are ~1e‑5–5e‑5. Batch 4–64; 1–3 epochs (more → overfit).
This loop sets up all of Section 2: memory (hold the loop), quantization (fewer bits), LoRA/QLoRA (update a tiny slice), and hyperparameters (tune the loop).

Next: L190 — The Memory Bottleneck & Memory Math — exactly how much GPU memory this loop needs, and why a 7B model needs far more than 7B-worth of RAM to train.