Title: Discussion 4 — Debugging Erratic Loss: Is It the Template?
Course: AIINFRA 201: Model Adaptation — Fine-Tuning & Quantization > Section: Week 04 — Instruction and Chat Formatting, Tokenization, and Templates

💬 Discussion 4 — Debugging Erratic Loss: Is It the Template? (20 pts)

Imagine you're fine-tuning a chat model and the training loss is erratic — it spikes, plateaus oddly, or just doesn't behave the way you'd expect from a healthy run. You suspect the chat template might be the culprit rather than your learning rate, data quality, or batch size. In your post, address:

- What specific checks would you run first to confirm (or rule out) a chat-template mismatch? Be concrete — what would you print, compare, or inspect?
- What would the actual fix look like once you found the problem (for example: reapplying the correct template, adding and training new special tokens, or realigning your dataset preprocessing)?
- Why is this kind of failure so easy to miss compared to, say, an out-of-memory error or an obvious data-loading bug?

Post your initial response, then respond to at least two classmates — ideally to someone who proposed a different diagnostic approach than you did, and compare notes on what you'd check first.