Title: Quiz 6 — Parameter-Efficient Fine-Tuning: LoRA and PEFT
Course: AIINFRA 201: Model Adaptation — Fine-Tuning & Quantization > Section: Week 06 — Parameter-Efficient Fine-Tuning: LoRA and PEFT

Q: What does LoRA inject into a pretrained model to enable efficient fine-tuning?
- A second full copy of every weight matrix
- Trainable low-rank decomposition matrices (A and B) into targeted layers, while freezing the original weights
- A larger attention head count
- A quantized version of the entire model
A: Trainable low-rank decomposition matrices (A and B) into targeted layers, while freezing the original weights

Q: With LoRA, the original pretrained weights are updated directly during training, just at a slower rate than full fine-tuning.
- True
- False
A: False

Q: What is PEFT?
- A quantization format for compressing models
- Hugging Face's unified library for LoRA and related parameter-efficient tuning methods
- A dataset format for instruction tuning
- A GPU scheduling tool
A: Hugging Face's unified library for LoRA and related parameter-efficient tuning methods

Q: What method do you call on a trained PEFT model to fold the LoRA adapter weights into the base model, producing a standalone model with zero additional inference latency?
Feedback: Sample/expected answer: merge_and_unload()

Q: Rank (r) and alpha can be freely chosen independently of one another without affecting training stability.
- True
- False
A: False

Q: Following the common guideline discussed in the lecture, if you set LoRA rank r = 8, what alpha value would you choose?
- 4
- 8
- 16
- 64
A: 16

Q: According to recent research referenced in the lecture, which target-module strategy often outperforms adapting attention modules alone?
- Adapting only the embedding layer
- Adapting only the final output layer
- Targeting MLP/feed-forward layers (or all linear modules)
- Reducing the number of attention heads
A: Targeting MLP/feed-forward layers (or all linear modules)

Q: Name one advantage of keeping a LoRA adapter separate from the base model (rather than merging it) when deploying to production.
Feedback: Sample/expected answer: You can swap or stack multiple task-specific adapters onto the same frozen base model without duplicating the full base model for each task.