Hands-On: Fine-Tune a Small Model with LoRA

Introduction

This is it — the capstone. Across this container you learned when to fine-tune (Section 1), the mechanics (LoRA, QLoRA, memory, hyperparameters — Section 2), how to build the dataset (Section 3), and the post-training methods and evaluation (Section 4). Now you put it all together into one runnable pipeline — and fine-tune a real model, end to end, on a free GPU.

Here's the headline that would've been science fiction a few years ago: thanks to QLoRA and Unsloth, you can fine-tune a capable small model (Llama-3.2-3B, Qwen) on a free Google Colab T4 (16 GB) in a single afternoon — and walk away with a ~100 MB adapter (or a q4_k_m GGUF) you run on your laptop in Ollama. This lesson is the complete recipe: load → adapt → format → train → evaluate → ship, plus the debugging craft (OOM, loss, template bugs) that turns a notebook that errors into one that works.

An infographic titled 'Hands-On: Fine-Tune a Small Model with LoRA', the capstone of the fine-tuning and model customization course, which assembles the whole container into one runnable pipeline that fits on a free Google Colab T4 GPU. The pipeline has seven steps. First, set up the environment with Unsloth, which trains about two times faster and uses roughly seventy percent less memory, on a pinned stack of Python three eleven, PyTorch two point five, and CUDA twelve. Second, load a four-bit base model with FastLanguageModel from-pretrained, for example Llama three point two three billion in four-bit, which is the memory bottleneck and quantization lever from section two. Third, attach LoRA adapters with get-peft-model, rank sixteen, alpha sixteen, dropout zero, targeting all seven attention and MLP projections, with gradient checkpointing on. Fourth, format the dataset by applying the model's own chat template into a text field, the formatting step from section three. Fifth, train with the TRL supervised fine-tuning trainer using the section-two recipe: learning rate two times ten to the minus four, cosine schedule with warmup, per-device batch two times gradient accumulation four for an effective batch of eight, while watching the train and validation loss. Sixth, evaluate the candidate against the base on the same held-out test, checking the task delta and general-capability regression. Seventh, save and deploy: a roughly one-hundred-megabyte LoRA adapter, or a merged model, or a q4_k_m GGUF exported for Ollama, llama.cpp, and LM Studio. The hands-on craft is the debugging: out-of-memory is fixed by keeping four-bit, dropping the batch toward one, lowering the sequence length, and keeping gradient checkpointing on; a non-decreasing loss means the learning rate is wrong or it is overfitting; and garbage inference almost always means the wrong chat template or a missing end-of-sequence token. The takeaway is that fine-tuning a small model end to end, from decision to a deployable artifact, is now a single afternoon on a free GPU.

The Pipeline: Seven Steps, One Afternoon

The whole hands-on is seven steps, each one a lesson you've already met — now wired into a single runnable flow:

Environment — Unsloth (≈2× faster, ~70% less memory) on a pinned stack (Python 3.11+, PyTorch 2.5+, CUDA 12.x). Pin versions for reproducibility.
Load a 4-bit base — QLoRA's frozen NF4 base (L191/L193): Llama-3.2-3B-Instruct-bnb-4bit.
Attach LoRA adapters — the L193 config (r=16, α=16, all 7 target modules), with gradient checkpointing on.
Format the dataset — apply the model's own chat template into a text field (the L199 formatting step — get this exactly right).
Train — TRL's SFTTrainer with the L194 recipe (lr 2e-4, cosine + warmup, effective batch 8), watching train/val loss.
Evaluate — the delta vs base on a held-out test + a regression check (L205).
Save & deploy — a tiny LoRA adapter, a merged model, or a GGUF for Ollama/llama.cpp.

You'll run this exact flow — and feel the VRAM and loss gates — in the Fine-Tune Run lab. Let's see the code.

Steps 1–3: Load a 4-Bit Base & Attach LoRA

Unsloth wraps the QLoRA setup into a few lines. Load the 4-bit base (this is the L190 memory win — a 3B fits in ~2 GB), then attach LoRA adapters with the L193 config:

from unsloth import FastLanguageModel

# STEP 2 - load a 4-bit (QLoRA) base. ~2 GB for a 3B model -> fits a free T4. (L190/L191/L193)
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name   = 'unsloth/Llama-3.2-3B-Instruct-bnb-4bit',
    max_seq_length = 2048,        # lower this first if you hit OOM
    load_in_4bit = True,          # the QLoRA frozen NF4 base
)

# STEP 3 - attach LoRA adapters (the trainable part). (L193)
model = FastLanguageModel.get_peft_model(
    model, r = 16, lora_alpha = 16, lora_dropout = 0,
    target_modules = ['q_proj','k_proj','v_proj','o_proj',     # all-linear:
                      'gate_proj','up_proj','down_proj'],      # attention + MLP
    use_gradient_checkpointing = 'unsloth',   # big VRAM saver - keep this ON
)
# Only the LoRA adapters train: ~1-2% of the parameters.

Steps 4–5: Format the Data & Train

Format every row with the model's own chat template (the L199 step that prevents the #1 silent bug), then train with TRL's SFTTrainer using the L194 recipe — and mask the loss to the responses:

from trl import SFTTrainer, SFTConfig

# STEP 4 - format with the MODEL'S OWN chat template (+ EOS). (L199)
def fmt(ex): return {'text': tokenizer.apply_chat_template(ex['messages'], tokenize=False)}
ds = dataset.map(fmt)            # train.jsonl from your L200 dataset build

# STEP 5 - train with the L194 recipe; watch train + val loss.
trainer = SFTTrainer(
    model = model, tokenizer = tokenizer, train_dataset = ds, eval_dataset = val_ds,
    args = SFTConfig(
        learning_rate = 2e-4, lr_scheduler_type = 'cosine', warmup_ratio = 0.03,  # L194
        per_device_train_batch_size = 2, gradient_accumulation_steps = 4,  # -> eff. batch 8
        num_train_epochs = 3, bf16 = True,
        completion_only_loss = True,            # loss-mask the prompt (L199)
        eval_strategy = 'steps', eval_steps = 50,
    ),
)
trainer.train()    # if val loss climbs -> overfitting; if loss spikes -> LR too high (L194)

Steps 6–7: Evaluate, Save & Run in Ollama

Before you trust it, evaluate vs the base (L205). Then save — just the tiny adapter, a merged model, or a GGUF you can run locally. Use the same chat template at inference — the #1 cause of garbage output:

# STEP 6 - EVALUATE vs the base on a held-out test: ship on the DELTA, not the absolute. (L205)
#   check task lift AND a general-capability slice (catastrophic forgetting).

# STEP 7 - SAVE & DEPLOY.
model.save_pretrained('lora_adapter')          # ~100 MB - just the adapter
model.save_pretrained_merged('merged', tokenizer)   # base + adapter, full model
model.save_pretrained_gguf('gguf', tokenizer,  # for Ollama / llama.cpp / LM Studio
                           quantization_method = 'q4_k_m')   # small + good quality
# model.push_to_hub_gguf('me/my-model', tokenizer, quantization_method='q4_k_m')

#   $ ollama create my-model -f Modelfile   &&   ollama run my-model
# At inference, use the SAME chat template you trained with - or you get gibberish.

The Hands-On Craft: Debugging a Run

Three problems hit everyone the first time — and each maps to an earlier lesson:

CUDA out of memory (OOM). The most common wall. Fixes, in order: keep load_in_4bit=True (QLoRA), drop per_device_train_batch_size toward 1 (use grad-accum to keep the effective batch), lower max_seq_length, and keep use_gradient_checkpointing='unsloth' ON. This is the L190 memory math — the lab shows you exactly which lever frees the most VRAM.
Loss not decreasing (or climbing). Watch train and val loss (L194): a flat/high loss means the LR is too low (raise toward 2e-4); val loss climbing means overfitting (fewer epochs / early-stop); loss spikes/NaN means the LR is too high.
Garbage at inference (gibberish, won't stop). Almost always the wrong chat template or a missing EOS (L199) — you must use the exact template you trained with, in Ollama/llama.cpp too. The single most common cause of "it trained fine but the output is nonsense."

Knowing these three turns a frustrating afternoon into a shipped model.

See It: The Fine-Tune Run

Configure the whole run and watch it fit, train, and ship. Pick a model and a GPU, choose 4-bit vs 16-bit, and set the LoRA rank, batch, max_seq, LR, epochs, and dataset size. First the VRAM bar tells you if it fits or OOMs (try an 8B in 16-bit on the 12 GB card — boom — then fix it with 4-bit + smaller batch + gradient checkpointing). Then watch the training outcome and the eval delta vs base, and collect your artifact — a tiny adapter + a q4_k_m GGUF for Ollama. It's the entire container in one screen.

Interactive: the Fine-Tune Run, the course capstone. The user configures an end-to-end QLoRA run and the lab plays out the two decisive gates and the payoff. Controls set the base model, the GPU, four-bit QLoRA versus sixteen-bit LoRA, the LoRA rank, the per-device batch, the maximum sequence length, gradient checkpointing, the learning rate, the epochs, and the dataset size. First the lab computes a VRAM check and draws a breakdown bar of weights, LoRA plus optimizer, activations, and overhead against the GPU's capacity, so an eight-billion model in sixteen-bit overflows a twelve-gigabyte card while a three-billion model in four-bit fits a free T4. Then it resolves the training outcome as converged, overfit, underfit, or diverged from the learning rate, epochs, and dataset size, and reports the evaluation delta over the base for both task and general capability, flagging catastrophic forgetting. Finally it shows the artifact produced, a roughly one-hundred-megabyte adapter plus a q4_k_m GGUF for Ollama. A ship-or-fix verdict integrates the whole container: out-of-memory with the ordered fixes, diverged from too high a learning rate, overfit with a general-capability drop, underfit, or shipped. An arrow pipeline runs four-bit base to LoRA to format to train to eval to GGUF to Ollama.

Every knob in that lab is a lesson you learned — memory (L190), quantization (L191), LoRA (L193), the recipe (L194), the data (Section 3), and the eval (L205) — now working together.

Why This Matters: The Whole Container, Realized

You came into this container able to use models; you leave able to customize them. That's a genuine step-change in capability — the difference between a prompt engineer and an AI engineer who can reshape a model's behavior. And the barrier has collapsed: what once required a research lab and a GPU cluster is now a free notebook and an afternoon. You can take an open base model, teach it your task, your tone, or your reasoning, evaluate that it genuinely improved, and ship a laptop-sized artifact — all with the decisions and discipline this container gave you: should I even fine-tune? (S1), LoRA or full, what config? (S2), is my data clean, covered, decontaminated? (S3), did it actually get better without forgetting? (S4). That's the complete loop of fine-tuning, and you now own it.

🧪 Try It Yourself

Run the full pipeline in the Fine-Tune Run lab:

Cause an OOM, then fix it. Put an 8B model in 16-bit on the 12 GB card. Now get it to fit by changing the fewest knobs — which lever frees the most VRAM, and why (L190)?
Get a clean ship. Configure a 3B QLoRA run on the T4 that converges, beats the base, and doesn't overfit. What LR, epochs, and dataset size did you use (L194/L195)?
Break it two ways. Trigger diverge (how?) and overfit (how?). For overfit, what happens to general capability, and what's that called (L205)?

Bonus: your fine-tune trains perfectly but produces gibberish in Ollama. What's the single most likely cause, and the fix (L199)?

Mental-Model Corrections

"Fine-tuning needs a GPU cluster." Not anymore — QLoRA + Unsloth fine-tune a 3B–8B model on a free Colab T4.
"OOM means the model's too big." Usually it's the activations: drop the batch and max_seq_length, keep 4-bit + gradient checkpointing (L190) — the model itself fits fine.
"If training loss drops, I'm done." Check val loss (overfitting), then evaluate vs base (L205). Training loss alone proves nothing.
"It trained, so the output will be good." Not if you use the wrong chat template at inference — the #1 cause of gibberish (L199). Use the exact template you trained with.
"I should merge and ship the full model." Often the ~100 MB adapter (or a q4_k_m GGUF) is all you need — smaller, swappable, and laptop-runnable.

Key Takeaways

The full pipeline: load a 4-bit base → attach LoRA → format with the chat template → train (the L194 recipe) → evaluate vs base → save a GGUF → run in Ollama — on a free T4, in an afternoon.
The stack: Unsloth (≈2× faster, ~70% less memory) + TRL SFTTrainer + QLoRA; pin your versions.
Debug by lesson: OOM → 4-bit + smaller batch/seq + grad-checkpointing (L190); loss → LR / overfit (L194); gibberish → wrong chat template / EOS (L199).
Ship the delta, not the demo: evaluate vs the base and watch for forgetting (L205) before you deploy.
The artifact: a ~100 MB LoRA adapter or a q4_k_m GGUF you run locally — your own specialized model.

That completes Fine-Tuning & Model Customization. You can now decide on, build the data for, train, evaluate, and ship a custom model end to end — the full craft of reshaping a model to your will.