Title: Discussion 6 — Does LoRA Actually Protect You From Forgetting?
Course: AIINFRA 201: Model Adaptation — Fine-Tuning & Quantization > Section: Week 06 — Parameter-Efficient Fine-Tuning: LoRA and PEFT

💬 Discussion 6 — Does LoRA Actually Protect You From Forgetting? (20 pts)

Imagine you've LoRA-fine-tuned a model and it now performs great on your training domain, say, customer-support tickets for a software product. But when you test it on general tasks (unrelated Q&A, basic reasoning, casual conversation), you notice its answers have drifted or degraded compared to the original base model. This looks like catastrophic forgetting, the phenomenon where a model loses previously learned capabilities while acquiring new ones. Post a response addressing:

1. Do you think LoRA inherently protects a model from catastrophic forgetting better than full fine-tuning would, worse, or does it depend on configuration? Justify your reasoning using what you know about how LoRA freezes the base weights versus how full fine-tuning updates every parameter.
2. How would you actually measure the difference in forgetting between a LoRA-adapted model and a fully fine-tuned version of the same base model? Propose a concrete evaluation approach (what you'd test, what you'd compare against, what would count as evidence of forgetting).
3. Does rank or target-module choice (attention-only vs. MLP/all-linear) plausibly affect how much forgetting occurs? Give your best reasoning even if you're not 100% certain.

Respond to at least two classmates, engaging with their reasoning or evaluation approach rather than just agreeing.