Title: Discussion 14 — Is the Tradeoff Worth It?
Course: AIINFRA 201: Model Adaptation — Fine-Tuning & Quantization > Section: Week 14 — Knowledge Distillation and Quality-Size Tradeoff Analysis

💬 Discussion 14 — Is the Tradeoff Worth It? (20 pts)

Prompt: Imagine you've distilled a 70B-parameter model down to a 7B student. Benchmarking shows the distilled 7B student is 15% slower than a non-distilled 7B baseline, but 8% more accurate on your target task. Is this tradeoff worth it for a real application you can imagine deploying (name the application and its constraints — e.g., a customer support chatbot, a coding assistant, a real-time voice agent)? Consider:

- Does your application care more about latency or accuracy at the margin, and why?
- How would you explain this tradeoff to a non-technical stakeholder who just wants "the model that works best"?
- What additional evidence (beyond this single 15%/8% pair of numbers) would you want before recommending a decision?

Post your initial response (150+ words), then respond to at least two classmates — ideally to someone who reached a different conclusion than you, or who picked a different application domain.