Title: Lecture 11 — Why Models Forget, and What LoRA Actually Protects
Course: AIINFRA 201: Model Adaptation — Fine-Tuning & Quantization > Section: Week 11 — Catastrophic Forgetting, Adapter Merging, and Cloud Training

📖 Lecture — Why Models Forget, and What LoRA Actually Protects

By now you've fine-tuned models and quantized them for deployment. This week we tackle a problem that quietly undermines both efforts: when you teach a model something new, it can forget what it already knew. This is catastrophic forgetting, and it is well-documented across the LLM parameter range you're likely to work with — roughly 1B to 7B parameters. Research shows forgetting severity actually increases as model scale increases within this range, which runs counter to the intuition that "bigger models are more robust." The mechanism is straightforward: fine-tuning updates weights via gradient descent, and nothing in that process protects the weight patterns that encoded prior capabilities. If the new task's gradients push in a different direction than the old task needed, the old capability gets overwritten — not maliciously, just as a side effect of optimization. Three mechanistic causes. Recent mechanistic analysis breaks catastrophic forgetting down into three contributing causes, and understanding them helps you predict when forgetting will be severe:

Cause | What's happening | Practical signal
Gradient interference in attention weights | Gradients from the new task point in directions that conflict with the attention patterns the old task relied on | Sharp accuracy drop on prior benchmarks after only a few hundred steps
Representational drift in intermediate layers | Hidden-state representations shift away from the geometry earlier layers had learned, even when final-layer outputs still look reasonable early in training | Old task loss degrades gradually across training, not just at the end
Flattening of the loss landscape around the prior task's minimum | Fine-tuning can flatten the loss basin that represented good performance on the old task, making that region less "sticky" and easier to wander away from | Old task performance doesn't recover even with small learning rates

Four families of mitigation. No single fix eliminates forgetting; instead there are four complementary strategies:

1. Replay-based — mix a sample of old-task data back into the fine-tuning set so gradients continue to reinforce prior behavior.
2. Regularization-based — add a penalty term that discourages the fine-tuned weights from drifting too far from the original weights (conceptually similar to elastic weight consolidation).
3. Gradient-based — techniques like sharpness-aware minimization (SAM) that seek flatter, more robust minima less prone to being disrupted by later updates.
4. Architecture-based — isolate the new task's changes into a separate structure rather than overwriting shared weights. LoRA is the flagship example here: it freezes the base model and learns small low-rank adapter matrices, so the original weights are never directly modified.

Now the crucial correction. It is tempting to conclude that because LoRA freezes the base model, it fully prevents catastrophic forgetting. This is a misconception. Freezing the base weights means they cannot be overwritten, but the composed behavior of base-plus-adapter can still drift substantially from the base model's original behavior, especially when the domain shift between old and new tasks is large. LoRA often forgets less than full fine-tuning — the search space is constrained to low-rank matrices, so there's less capacity to overwrite broad prior knowledge — but "less" is not "none." Fine-tune a LoRA adapter on a domain far from the base model's pretraining distribution (a highly specialized legal or medical corpus, say), and you can still see meaningful degradation on general benchmarks. Treat LoRA as a mitigation, not a guarantee, and always measure. Adapter merging: powerful, but not a free lunch. Once you have multiple LoRA adapters — one tuned for customer support, one for code generation — a second misconception appears: that you can simply merge them into the base model simultaneously and get a model good at both. This is also false. Merging multiple task-specific adapters is not straightforward; their low-rank update directions can conflict the same way full fine-tuning gradients do, and naive averaging often degrades both tasks rather than combining them cleanly. Keeping adapters separate, and swapping them in at inference time, is frequently the safer choice for preserving task isolation. When you do need one merged model, mergekit (from Arcee AI) offers two more principled workflows: extracting a LoRA adapter from a fully fine-tuned model by diffing base and tuned weights, and merging multiple LoRA adapters using methods like TIES (which trims small, conflicting updates before merging) and DARE (which randomly drops and rescales redundant updates). These beat naive averaging, but always evaluate each task's performance after merging — never assume it worked.