Fine-Tuning Hyperparameters & Tactics

Introduction

You now have the machinery: L193 (LoRA & QLoRA Explained) gave you the adapter, L191 (Numerical Precision & Quantization) the 4-bit base, L190 (The Memory Bottleneck & Memory Math) the budget. This lesson closes the section by turning all of it into a runnable recipe — and, more importantly, teaching the craft that separates a fine-tune that works from one that quietly fails: reading a training run.

Here's the honest truth most tutorials skip: the hyperparameters themselves are nearly a solved problem — there's a strong default recipe you can almost always start from. The skill isn't guessing the numbers; it's watching two loss curves and knowing which knob to turn when they misbehave. By the end you'll read learning_rate, num_train_epochs, and lora_alpha not as mystery dials but as levers with predictable failure modes — and you'll have felt all four in the lab.

An infographic titled 'Fine-Tuning Hyperparameters & Tactics', the closing lesson of section two on fine-tuning mechanics, which turns the LoRA and QLoRA machinery into a runnable recipe and teaches the craft of reading a training run. The central idea is that the entire job of tuning is reading two curves over training time: the training loss, computed on the data the model is learning from, and the validation loss, computed on a held-out split it never trains on. Four shapes tell you everything. When training loss and validation loss both descend and flatten together with only a small gap, the run is converging and you should stop near there. When training loss keeps falling but validation loss bottoms out and turns back up, the model is overfitting, memorizing the training examples, and the fix is to stop early at the validation minimum, use fewer epochs, add more data, lower the rank, or add dropout and weight decay. When both losses stay stuck high, the model is underfitting because the learning rate is too low or there were too few epochs, so raise the learning rate toward two times ten to the minus four and add epochs. When the loss spikes, oscillates, or goes to not-a-number, the learning rate is too high and training is diverging, so lower it and clip the gradients. The learning rate is the single most important and most dangerous knob: LoRA tolerates a learning rate roughly ten times higher than full fine-tuning, around two times ten to the minus four versus one to five times ten to the minus five. Epochs should be one to three for instruction tuning, because more epochs overfit fast on small datasets. A strong default recipe is rank sixteen, alpha thirty-two, dropout zero point zero five, learning rate two times ten to the minus four with a cosine schedule and three percent warmup, three epochs, and an effective batch of sixteen built with gradient accumulation, targeting all linear layers. The supporting tactics are gradient accumulation to simulate a big batch on a small GPU, gradient clipping to tame spikes, learning-rate warmup then cosine decay, and early stopping that keeps the checkpoint at the validation-loss minimum rather than the last, often overfit, step. The takeaway is that you start from the recipe and then let the two loss curves tell you exactly which knob to turn.

The Whole Job: Reading Two Curves

Strip away everything else and fine-tuning monitoring is two numbers over time:

Training loss — the L189 (How Training Actually Works) loss, measured on the data the model is learning from. It should fall steadily.
Validation loss — the same loss measured on a held-out split the model never trains on. This is the one that tells the truth about whether the model is actually getting better vs. just memorizing.

A held-out validation set is non-negotiable — without it you are flying blind. Four shapes tell you everything you need to know:

What you see	Diagnosis	The move
Both curves ↓ together, small gap	✅ Converging	Healthy — stop near the flattening
Train ↓, val ↑ (turns up)	❌ Overfitting	Early-stop, fewer epochs, more data, dropout
Both stuck high	⚠️ Underfitting	Raise LR, add epochs, raise rank
Loss spikes / NaN	❌ Diverging	LR too high — lower it, clip gradients

The gap between val and train loss is the overfitting signal. Every knob below is just a way to move these two curves — and the lab at the end lets you bend them with your own hands.

The #1 Knob: Learning Rate

If you tune one thing, tune this. The learning rate is the step size for the gradient update from L189 (How Training Actually Works) — and it's both the most powerful and the most dangerous knob.

Too high → the loss spikes, oscillates, or NaNs. Training diverges; the weights overshoot the minimum every step. (Also a frequent cause of fast overfitting on short runs.)
Too low → the loss crawls. You waste your epochs barely learning — the classic underfit.
Just right → a smooth, steady descent.

The key fact for our setting: LoRA tolerates a much higher LR than full fine-tuning. Because you're only training a tiny low-rank adapter (L193), a bigger step is safe — and necessary to learn in a few epochs. Typical values:

LoRA / QLoRA: ~2e-4 (often 1e-4 to 3e-4).
Full fine-tuning: ~1e-5 to 5e-5 — ~10× lower, because every weight is moving.

Pair it with a schedule (next section), and when in doubt, lower it — a slightly-too-low LR wastes time; a too-high one wastes the whole run.

Epochs, Dataset Size & Overfitting

An epoch is one full pass over your training data. The instinct "more training = better" is exactly wrong here, and it's the single most common way fine-tunes fail.

Epochs: 1–3 for instruction tuning. Past that, on a typical dataset, the model stops learning the task and starts memorizing the examples — train loss keeps dropping while val loss climbs. That's the overfitting curve from the table.
Dataset size is the other half of the equation. A small dataset (a few hundred to a couple thousand rows) overfits fast — sometimes within the first epoch. A larger, more varied set (L195 (Data Quality, Coverage & Quantity) covers this) tolerates more epochs before it turns.
Rank interacts too: a higher LoRA r (more capacity, L193) can fit — and overfit — faster.

This is why you watch the validation curve instead of picking a fixed number of epochs. The right stopping point is where val loss bottoms out, not a round number you chose in advance. In the lab, shrink the dataset and crank the epochs — you'll watch the val curve turn up and find that bottom yourself.

The Supporting Knobs: Batch, Accumulation, Schedule, Regularization

Beyond LR and epochs, a handful of settings shape how the descent happens:

Batch size — how many examples per gradient step. Bigger = smoother, more stable gradients, but more VRAM (the L190 budget). On a small GPU you often can't fit the batch you want — which is where the next knob saves you.
Gradient accumulation — the practical trick: run several small micro-batches, sum their gradients, and only then take one optimizer step. batch=4 × accumulation=4 gives an effective batch of 16 with the memory of 4. This is how you simulate a big batch on a small card.
Warmup + LR schedule — start the LR near zero and ramp up over the first ~3% of steps (avoids a destructive jolt while the optimizer settles), then decay it — a cosine schedule is the common default — so the model takes fine steps as it converges.
Regularization — weight_decay (~0.01) and lora_dropout (~0.05) gently fight overfitting; raise them (e.g. dropout 0.1) if the val curve turns up early.

lora_alpha (α ≈ 2r) from L193 rounds out the set. None of these are as decisive as LR or epochs — but they're the difference between a fine run and a clean one.

The Recipe: A Strong Default to Start From

Here's the honest shortcut: for instruction SFT on ~5–50k examples of a Llama-3.x or Qwen2.5 model, start from this exact config and only change what the curves tell you to. These numbers are the well-worn community default in 2026 — they'll get you a good run on the first try far more often than hand-picked values.

from trl import SFTConfig, SFTTrainer
from peft import LoraConfig

# A STRONG default recipe. Start here; let the loss curves tell you what to change.
peft_config = LoraConfig(
    r=16,                       # adapter capacity (4-64); 16 is the sweet spot (L193)
    lora_alpha=32,              # update scaling, ~2*r
    lora_dropout=0.05,          # light regularization
    target_modules='all-linear',  # attn + MLP > attention-only
    task_type='CAUSAL_LM',
)

args = SFTConfig(
    learning_rate=2e-4,         # LoRA likes ~10x higher LR than full FT (1e-5..5e-5)
    lr_scheduler_type='cosine', # ramp up, then decay
    warmup_ratio=0.03,          # ease in over the first ~3% of steps
    num_train_epochs=3,         # 1-3; more overfits fast on small data
    per_device_train_batch_size=4,
    gradient_accumulation_steps=4,  # -> EFFECTIVE batch 16 on a small GPU
    bf16=True,
    max_grad_norm=1.0,          # gradient clipping - tames loss spikes
    eval_strategy='steps', eval_steps=50,   # WATCH the validation loss
    logging_steps=10,
)

Tactics: Early Stopping & Reading It Live

The recipe gets you a good start; the tactics keep the run from going off the rails. The most important one is early stopping — let the trainer halt automatically when validation loss stops improving, and keep the checkpoint at the val-loss minimum, not the last (often overfit) step. That one setting saves more fine-tunes than any clever hyperparameter.

from transformers import EarlyStoppingCallback

# Keep the BEST checkpoint (val-loss minimum), not the last step.
args.load_best_model_at_end = True
args.metric_for_best_model = 'eval_loss'
args.greater_is_better = False

trainer = SFTTrainer(
    model=base, args=args, peft_config=peft_config,
    train_dataset=train_ds, eval_dataset=val_ds,   # a held-out split is non-negotiable
    callbacks=[EarlyStoppingCallback(early_stopping_patience=3)],  # stop if val stalls
)
trainer.train()

# Reading the two curves WHILE it runs - the whole skill in 4 lines:
#   train DOWN, val DOWN  (together)   -> healthy, keep going
#   train DOWN, val UP                 -> OVERFITTING  (early-stop / fewer epochs / more data)
#   train FLAT, val FLAT  (both high)  -> UNDERFITTING (raise LR, add epochs, raise r)
#   loss SPIKES / NaN                  -> LR TOO HIGH  (lower LR; max_grad_norm clips it)

Other tactics worth knowing: gradient clipping (max_grad_norm=1.0, already in the recipe) caps the gradient size to tame spikes; gradient accumulation (also in the recipe) buys you a bigger effective batch; and don't trust loss alone at the end — run the actual task eval (the Container-4 evals skillset) on the best checkpoint, because a lower val loss doesn't always mean a better model for your task.

See It: The Hyperparameter Tuning Lab

Now bend the curves yourself. Start from the recipe (the ↺ Load the recipe button) and watch a clean convergence. Then break it on purpose: crank the learning rate until the loss explodes (diverging); drop it until the curves crawl (underfitting); shrink the dataset and add epochs until the validation curve turns up and the early-stop marker appears (overfitting). The verdict names what you're seeing and the fix — train your eye until you can diagnose a run at a glance.

Interactive: the Hyperparameter Tuning Lab. The user tunes a fine-tuning run with five knobs and watches the training and validation loss curves respond live. They drag the learning rate across a logarithmic range, the number of epochs, the LoRA rank, and the dataset size, and toggle warmup with a cosine schedule. A chart plots the training loss and the validation loss over training progress, and a verdict names which of four outcomes the current settings produce and how to fix it: converging when both curves descend together, overfitting when training falls but validation turns up, with a dashed marker showing where to early-stop at the validation minimum, underfitting when both stay high, and diverging when the learning rate is too high and the loss explodes. A live readout shows the gap between the final validation and training loss, the overfitting tell. A load-the-recipe button snaps every knob to the 2026 gold default, rank sixteen, alpha thirty-two, learning rate two times ten to the minus four, cosine schedule with warmup, and three epochs, producing a clean convergence the user can then perturb to see each failure mode.

Notice the pattern: there's a basin of good settings around the recipe, and you fall out of it in recognizable ways. You're not memorizing numbers — you're learning to read the descent.

Why This Matters

Hyperparameter intuition is what turns fine-tuning from a dice roll into an engineering discipline. The engineer who can glance at a loss curve and say "that's overfitting — early-stop and add data" ships a working adapter in one afternoon; the one who can't burns a week and a GPU budget on runs that memorize noise. And because the default recipe is so good, the leverage is almost entirely in the diagnosis — a skill you can only build by watching curves move, which is exactly what the lab is for. This closes Section 2: Fine-Tuning Mechanics — you can now go from "should I fine-tune?" all the way to a tuned, monitored, early-stopped run. Next comes the thing that matters even more than any hyperparameter: the data.

🧪 Try It Yourself

Diagnose three runs. Open the Tuning Lab and reproduce each failure mode, then write the one-line fix for each:

Make it diverge. Push the learning rate up until the loss explodes. What's the fix, and what does max_grad_norm do here?
Make it overfit. Set the dataset small (~500) and epochs high (8–10). Where does the early-stop marker land, and name three different ways to fix overfitting.
Make it underfit. Drop the learning rate to ~1e-5. Why do both curves stay high, and which two knobs pull it back?

Then hit ↺ Load the recipe and confirm it converges cleanly. Bonus: from the recipe, batch=4 × grad_accum=4 — what's the effective batch, and why would you raise grad_accum instead of batch on a small GPU?

Mental-Model Corrections

"More epochs = a better model." Past 1–3, you usually get a worse one — train loss falls while val loss climbs. Watch the val curve, not the epoch counter.
"Use the same learning rate as full fine-tuning." No — LoRA wants ~10× higher (~2e-4 vs ~2e-5), because only a tiny adapter is moving. Copy a full-FT LR into a LoRA run and it barely learns.
"Lowest training loss = best model." A near-zero training loss with a rising validation loss is the signature of overfitting — the model memorized your data. The val curve is the honest one.
"Bigger batch size is always better, and I can't afford it." Bigger is smoother but you don't need the VRAM for it — gradient accumulation gives you the effective batch cheaply.
"I'll find magic hyperparameters." The recipe is already strong. The real skill is diagnosis — reading the curves and turning the one knob that's wrong.

Key Takeaways

The job is reading two curves: train loss (data it learns from) and validation loss (held-out). Their gap is the overfitting signal.
Four shapes, four moves: both ↓ = converging (stop near flat); train ↓ / val ↑ = overfitting (early-stop, fewer epochs, more data); both high = underfitting (raise LR, add epochs); spikes/NaN = LR too high.
LR is the #1 knob — LoRA wants ~2e-4, ~10× higher than full FT's ~2e-5; when unsure, lower it.
Epochs 1–3; small datasets overfit fast — let the val minimum pick the stopping point, not a round number.
Start from the recipe — r=16, α=32, lr=2e-4, cosine+warmup, 3 epochs, effective batch 16 via grad accumulation, all-linear — and add early stopping to keep the best checkpoint.

That closes Section 2: Fine-Tuning Mechanics. Next, Section 3 — L195 (Data Quality, Coverage & Quantity): the dataset that decides whether any of these knobs matter.