🛠️ Lab 11 — Measuring Forgetting and Merging Adapters (50 pts)

Goal: Empirically demonstrate catastrophic forgetting by comparing full fine-tuning against LoRA fine-tuning on pre/post benchmark scores, then train two separate LoRA adapters and experiment with merging them using TIES and DARE. Steps:

  1. Using the same small base model and local/free tooling from Weeks 9–10 (Hugging Face transformers, peft, and trl, running locally or in a free-tier Colab/Kaggle notebook), record baseline scores on the MMLU and ARC subsets you used in Week 10's evaluation harness. Save these as your "pre-fine-tune" scores.
  2. Fine-tune a copy of the base model on a narrow task/dataset (e.g., a small instruction-style dataset or a domain-specific text corpus) using full-parameter fine-tuning. Re-run the Week 10 MMLU/ARC evaluation on this fine-tuned model and record the "post-fine-tune" scores.
  3. Starting fresh from the base model, fine-tune a LoRA adapter on the same narrow task/dataset (same data, same number of steps/epochs, so the comparison is fair). Re-run the same MMLU/ARC evaluation on the base model + LoRA adapter combination.
  4. Build a comparison table (pre-fine-tune vs. full-fine-tune vs. LoRA) showing the score drop on MMLU/ARC for each approach, and write 3–4 sentences interpreting whether your results support "LoRA forgets less" — and whether that protection looks complete or partial.
  5. Call merge_and_unload() on your LoRA adapter to produce a standalone merged model, and confirm it loads and runs like a normal model (no PEFT wrapper needed).
  6. Train a second LoRA adapter on a different small task/dataset than the one in Step 3 (choose something meaningfully different, e.g., a different domain or a different task type).
  7. Using the PEFT model-merging guide and the PEFT merging blog post (see Resources), experiment with merging your two adapters using TIES and DARE. Evaluate the merged model on both original tasks/domains and record whether performance held up, degraded, or improved for each.
  8. Write a short reflection (1 paragraph) comparing what you observed with naive merging expectations versus what TIES/DARE actually produced.

Deliverables: Submit your notebook/script, the pre/post/LoRA benchmark comparison table, your merged-model evaluation results (TIES and DARE, per task), and your two written reflections (Step 4 and Step 8) as a PDF or text document, plus the standalone merged model file (or a link to where it's hosted, e.g., a local path or your Hugging Face account) if the file is too large to upload directly.