Model Merging (SLERP, TIES, DARE)

Introduction

You've now learned to train specialist models — SFT for instruction-following, DPO (L202) for alignment, GRPO (L203) for reasoning. This lesson is the surprising one: how to combine several fine-tuned models into a single, more capable model — with no training, no GPU, and no data. Just arithmetic on their weights.

It sounds like it shouldn't work, yet model merging is one of the most practical tricks in open-model AI: it's how the community produces competitive models on a laptop by blending the best public fine-tunes. Take a math model and a code model (both fine-tuned from the same base), average their weights the right way, and get one model good at both. By the end you'll understand why it works (task vectors), the enemy it fights (interference), and the ladder of methods — Linear → SLERP → TIES → DARE — that each handle interference more cleverly.

An infographic titled 'Model Merging, SLERP, TIES, DARE', the fourth lesson of section four on advanced post-training, on combining several fine-tuned models into one with no extra training, no GPU, and no data. The key concept is the task vector: a fine-tuned model equals the base model plus a delta, where the delta, theta fine-tuned minus theta base, is a direction in weight space that encodes the new skill. Task arithmetic adds these deltas back to the base, so a math fine-tune's task vector plus a code fine-tune's task vector, added to the shared base, yields one model good at both, and subtracting a task vector can even remove a behavior. The enemy is interference: where two task vectors disagree, having opposite signs on the same parameter, naive averaging makes them cancel and waters down both skills. The methods form a ladder that handles interference ever more cleverly. Linear, or model soup, simply averages the weights, which is fine for similar models but weak under conflict. SLERP, spherical linear interpolation, blends along a spherical path rather than a straight line, preserving magnitude so conflicts cancel less, but it merges only two models at a time. TIES, trim elect-sign and merge, first trims the tiny redundant deltas to zero, then elects the dominant sign for each parameter, then averages only the models that agree with that sign, resolving conflicts by a vote and scaling to many models. DARE, drop and rescale, randomly drops ninety to ninety-nine percent of the deltas and rescales the survivors to preserve magnitude, since most updates are redundant; combined with TIES as dare-ties it gives the least interference and often the strongest merge. Passthrough, or frankenmerge, instead concatenates layers from different models to make a bigger model with an exotic parameter count, like goliath made from two seventy-billion models. The tooling is mergekit, a YAML config that streams the weights. The hard rules are that all parents must share the same base model, a merge cannot exceed what the parents collectively know, and you must always evaluate the result. The takeaway is that merging is free arithmetic on task vectors that combines specialist models, with interference as the thing the methods exist to manage.

The Big Idea: Task Vectors

Merging makes sense once you see a fine-tuned model the right way. A fine-tune is just the base model plus a change:

Δθ = θ_fine-tuned − θ_base (the "task vector")

That delta — the task vector — is a direction in weight space that encodes the new skill. Adding it back to the base (θ_base + Δθ) reconstructs the fine-tune; and that's the doorway to task arithmetic:

θ_merged = θ_base + α·Δθ_A + α·Δθ_B + …

Add a math task vector and a code task vector to the base, and you get a model that's good at both — no training required. You can even subtract a task vector to remove a behavior ("forget" a style or a capability). Why does it work? Under simplifying assumptions the task vector points along the negative gradient of the task loss — a genuinely meaningful direction — and fine-tunes of the same base sit in a connected low-loss basin, so combining their directions lands you somewhere useful. Which is also the one hard requirement: every model you merge must share the same base.

The Enemy: Interference

If task vectors just added up cleanly, merging would be trivial. The problem is interference — and it's what every merge method exists to manage.

Two task vectors disagree when they push the same parameter in opposite directions (one says +0.7, the other −0.7). Naive averaging of those gives ~0 — the two updates cancel, and you lose what both models had learned for that parameter. Across thousands of parameters, this waters down every skill: the merged model is mediocre at math and code, instead of good at both.

There's also redundancy: most of a task vector's entries are tiny — fine-tuning changes a few parameters a lot and most barely at all. Those tiny deltas mostly add noise to a merge.

So the smarter methods do two things: ignore the redundant tiny deltas, and resolve the sign conflicts instead of letting them cancel. That's exactly the ladder below — you'll watch the conflicts (⚡) cancel under Linear and get resolved under TIES in the lab.

The Methods Ladder: Linear → SLERP → TIES → DARE

Each method handles interference more cleverly than the last:

Linear / Model Soup — just average the weights (optionally weighted). Dead simple, and great when the models are similar. But where they conflict, the updates cancel — weak under disagreement.
SLERP (Spherical Linear Interpolation) — interpolate along the sphere rather than a straight line. There's often a lower-loss spherical path, and it preserves the vectors' magnitude so conflicts cancel less — the smoothest blend. Limitation: only two models at a time.
TIES (Trim, Elect Sign, Merge) — the answer for many models. Three steps: (1) Trim — keep only the top-magnitude deltas, zero the rest (kill the redundant noise); (2) Elect Sign — for each parameter, pick the dominant direction across models (a vote); (3) Merge — average only the models that agree with the elected sign. Conflicts are resolved, not cancelled.
DARE (Drop And REscale) — randomly drop 90–99% of each task vector's deltas (most are redundant!) and rescale the survivors to keep the overall magnitude. Stacked with TIES's sign step (dare_ties), it gives the least interference and is often the strongest merge.
Passthrough / Frankenmerge — a different trick: concatenate layers from different models to build a bigger model (two 7B → a 9B; goliath-120b from two Llama-2-70B). Very experimental, occasionally magical.

In Practice: mergekit & Choosing a Method

Merging has been democratized by mergekit — you don't need training infrastructure; you write a YAML config naming the base, the models, the method, and per-layer coefficients, and it streams the weights (it never loads every parameter at once). Practitioners with no GPU cluster routinely top leaderboards with merges of public fine-tunes.

Which method?

Two models, smooth blend → SLERP (the most common).
Many models, avoid interference → TIES, or DARE-TIES for maximum sparsification.
Quick average of same-base checkpoints → Linear / Model Soup.
Want a bigger model → Passthrough (experimental).

Here's a mergekit config that DARE-TIES-merges a math and a code fine-tune of the same base:

# mergekit config.yaml - merge a math + a code fine-tune (SAME base) with DARE-TIES.
models:
  - model: org/Llama-3.1-8B-math      # a fine-tune of the base
    parameters: { weight: 0.5, density: 0.6 }   # density = TIES trim (keep top 60%)
  - model: org/Llama-3.1-8B-code      # another fine-tune of the SAME base
    parameters: { weight: 0.5, density: 0.6 }
merge_method: dare_ties               # drop+rescale (DARE) + sign election (TIES)
base_model: meta-llama/Llama-3.1-8B   # the REQUIRED shared base
dtype: bfloat16
# $ mergekit-yaml config.yaml ./merged-model   # no GPU training - just weight arithmetic
# Then EVALUATE: a merge can be worse than either parent on some tasks.

The Limits: Same Base, No Free Lunch, Always Evaluate

Merging feels like magic, so it's worth being clear about what it can't do:

All parents must share the same base. Task vectors are deltas from a specific base; merge a Llama fine-tune with a Qwen fine-tune and the weights correspond to nothing. Same architecture, same pretraining — non-negotiable (you'll see this break in the lab).
A merge can't exceed what the parents collectively know. It recombines existing skills; it doesn't create new capability (the same ceiling as distillation's "student can't beat the teacher"). If neither parent can do X, the merge can't either.
It's unpredictable — always evaluate. Merging is cheap to try but the result can be worse than either parent on some tasks. Benchmark the merged model on what you care about. (The 2026 frontier — evolutionary merging — literally searches merge configs against an eval to find a good one.)

Used within those limits, merging is one of the highest-leverage, lowest-cost tools you have: a way to combine the specialists from this entire section into one deployable model, for free.

See It: The Merge Lab

Merge two fine-tunes by hand. You've got a math model and a code model as task vectors — the ⚡ marks where they conflict (opposite signs). Start with Linear and watch those conflicts cancel (both skill bars drop). Now switch to TIES and see the conflicts get resolved (trim the small deltas, elect the dominant sign) — both skills recover. Push DARE-TIES and crank the drop rate to see most deltas vanish while the merge holds. Finally, flip to different base and watch merging become undefined.

Interactive: the Merge Lab. Two parent fine-tunes, a math model and a code model, are shown as task vectors, the delta from the shared base, over a row of ten parameters, with sign-conflicts flagged by a lightning bolt where the two disagree. The user picks a merge method of Linear and model-soup, SLERP, TIES, or DARE-TIES, and tunes the weight balance between the two parents, the TIES trim density that keeps only the top fraction of parameters, and the DARE drop rate. The lab then computes the merged vector per parameter and shows it as a third row, with trimmed parameters shown as zero and sign-elected ones resolved, plus two skill-retention bars indicating how much of each parent's direction survived the merge. A verdict explains each method: Linear cancels conflicting updates and waters down both skills; SLERP preserves magnitude along a spherical path for two models; TIES trims, elects the dominant sign, and merges only the agreeing parameters for many models; and DARE-TIES drops most deltas, rescales, and applies sign election for the least interference. A same-versus-different-base toggle demonstrates that merging across different base models is undefined because the task vectors no longer line up.

The whole field is in that grid: add the task vectors, but resolve the conflicts — and never forget the parents must share a base.

Why This Matters

Model merging is the cheapest capability multiplier in the stack. Instead of serving five specialist models (or juggling five LoRA adapters at inference), you merge them into one set of weights — simpler to deploy, no adapter-switching, often better than any single parent. It's why a huge fraction of the top open models on the leaderboards are merges, produced by people with no training budget at all. For an AI engineer it's the move when you have several good same-base fine-tunes and want one model that does it all — applied with the discipline of share a base and always evaluate. It also closes the training half of Section 4; next you'll learn to measure whether any of this actually worked — L205 (Evaluating a Fine-Tuned Model).

🧪 Try It Yourself

Watch interference get resolved in the Merge Lab:

See the cancellation. With Linear, find the ⚡ conflict parameters. What value does the merge give them, and what does that do to the skill-retention bars for both parents?
Resolve it. Switch to TIES. What do the trim and elect-sign steps do to those conflict parameters, and why do both skills recover compared to Linear?
Drop almost everything. Use DARE-TIES and raise the drop rate to ~95%. Why can you throw away most of the deltas and still keep the skills (what does rescaling do)?

Bonus: flip to different base. In one sentence, why is the merge undefined — what is a task vector measured relative to?

Mental-Model Corrections

"Merging needs training/GPU." No — it's pure weight arithmetic (mergekit streams the files). No training, no data, minutes on a CPU.
"You can merge any two models." Only models that share the same base. Different bases → task vectors don't line up → undefined.
"Just average the weights." That's Linear, and it cancels conflicts. TIES/DARE trim, sign-elect, and drop-rescale to resolve interference — much stronger for conflicting models.
"A merge can be better than its parents at new things." It recombines existing skills; it can't create capability neither parent had.
"If the config is valid, the merge is good." Merging is unpredictable — a merge can be worse than either parent. Always evaluate.

Key Takeaways

A fine-tune = base + a task vector (Δθ = θ_ft − θ_base). Task arithmetic adds task vectors to the base to combine skills — no training, no GPU.
Interference is the enemy: conflicting-sign deltas cancel under naive averaging, watering down every skill.
The ladder: Linear/Soup (average) → SLERP (spherical path, 2 models) → TIES (trim + elect sign + merge agreeing, many models) → DARE (drop 90–99% + rescale; dare_ties is often strongest). Passthrough stacks layers for a bigger model.
Tooling: mergekit + a YAML config; the open community tops leaderboards with merges.
Hard rules: all parents share a base, a merge can't exceed the parents, and you must always evaluate the result.

Next: L205 — Evaluating a Fine-Tuned Model — how to actually measure whether your SFT, DPO, GRPO, or merge made the model better.