🛠️ Lab 6 — LoRA Rank Sweep and Merge Benchmark (50 pts)

Goal: Build a LoRA adapter with Hugging Face PEFT, empirically compare trainable-parameter counts against full fine-tuning, sweep rank and target-module choices to observe their tradeoffs, then merge your adapter into a standalone model and benchmark it against the adapter+base setup — all using free, local/Colab tools. Steps:

  1. Open a free Google Colab notebook (GPU runtime) and install transformers, peft, datasets, and accelerate. Work through the official Hugging Face LLM Course notebook on LoRA fine-tuning via PEFT, adapting it to a small base model (for example, a model in the 100M–1B parameter range) so it fits comfortably on a free-tier GPU.
  2. Load your base model and tokenizer, then print and record the total parameter count of the unmodified base model.
  3. Build a LoraConfig with r=8 and alpha=16 (following the alpha = 2r guideline), targeting the attention projection modules. Wrap the base model with get_peft_model() and call .print_trainable_parameters(). Record the trainable-parameter count and the percentage of the total this represents.
  4. Fine-tune this first configuration briefly on a small task dataset (a subset of a Hugging Face Hub dataset is fine — a few hundred to a couple thousand examples is enough for this exercise) and record training loss and a qualitative sample of output quality.
  5. Run a second configuration: change the rank to r=4 and then r=16, adjusting alpha each time to keep alpha = 2r. Keep everything else the same. Record trainable-parameter counts and training loss for each rank.
  6. Run a third configuration: keep r=8/alpha=16, but change target_modules from attention-only to all-linear (or explicitly include the MLP/feed-forward projection layers). Record the new trainable-parameter count and training loss, and compare qualitatively against the attention-only run from Step 3.
  7. Pick your best-performing configuration and call merge_and_unload() to produce a standalone merged model. Time inference (e.g., average time per generation over 5–10 prompts) for the merged model versus the unmerged adapter+base setup, and compare output quality between the two — they should be functionally identical.
  8. Write up your results in a short report: a table of rank/target-module configurations with trainable-parameter counts and percentages, your observations on training stability across ranks, which target-module choice performed better and why you think that matches the lecture's discussion of intrinsic dimensionality, and your merged-vs-unmerged latency/quality comparison.

Deliverables: Submit your Colab notebook (.ipynb, shared link or file upload) with all cells executed and outputs visible, plus your written report (as a text entry or attached document) containing the configuration comparison table and your analysis.