Title: Quiz 5 — Supervised Fine-Tuning Fundamentals with Hugging Face TRL
Course: AIINFRA 201: Model Adaptation — Fine-Tuning & Quantization > Section: Week 05 — Supervised Fine-Tuning Fundamentals with Hugging Face TRL

Q: Supervised Fine-Tuning trains a model using the same next-token prediction objective as pretraining, just applied to target completions instead of general text.
- True
- False
A: True

Q: What is the minimum you need to start a basic TRL SFTTrainer run?
- A base model name and a reward model
- A base model name and a Hugging Face dataset
- A hand-written prompt template and a tokenizer configuration file
- A quantized checkpoint and a LoRA adapter
A: A base model name and a Hugging Face dataset

Q: By default, SFTTrainer computes loss over the entire input sequence, including the prompt tokens.
- True
- False
A: False

Q: What setting would you have to change, and to what value, to make SFTTrainer compute loss over the full sequence instead of just the completion?
Feedback: Sample/expected answer: Set ****`completion_only_loss=False`**** (not recommended for instruction tuning).

Q: A teammate says, "You have to manually build a single prompt string by concatenating system, user, and assistant turns before TRL can train on it." What's the issue with this claim?
- It's correct — TRL always requires pre-built prompt strings
- It's incorrect — TRL natively accepts conversational datasets of role/content dicts and applies the chat template automatically
- It's correct only for base models, never for instruct models
- It's incorrect — TRL never applies chat templates under any circumstance
A: It's incorrect — TRL natively accepts conversational datasets of role/content dicts and applies the chat template automatically

Q: Which TRL feature lets you fine-tune on a dataset too large to fully load into RAM?
- `completion_only_loss`
- Sequence packing
- `IterableDataset` support, tokenizing and packing on the fly
- `chunked_nll`
A: `IterableDataset` support, tokenizing and packing on the fly

Q: Name the loss mode that TRL v1.7 made the default, and state its approximate effect on peak VRAM usage.
Feedback: Sample/expected answer: `chunked_nll`****; roughly a 30% average reduction in peak VRAM usage.

Q: What problem does sequence packing primarily address on datasets with many short examples?
- It prevents overfitting on repeated examples
- It reduces wasted computation from padding tokens, by roughly 3–5x on short datasets
- It replaces the need for a chat template
- It increases the effective vocabulary size of the tokenizer
A: It reduces wasted computation from padding tokens, by roughly 3–5x on short datasets