Title: Quiz 13 — Advanced Quantization: AWQ, GPTQ, FP8, and KV-Cache Quantization
Course: AIINFRA 201: Model Adaptation — Fine-Tuning & Quantization > Section: Week 13 — Advanced Quantization: AWQ, GPTQ, FP8, and KV-Cache Quantization

Q: AWQ computes Hessian information to decide which weights to protect during quantization.
- True
- False
A: False

Q: Approximately what fraction of weights does AWQ consider "salient" and worth protecting?
- 25%–50%
- 10%–20%
- 0.1%–1%
- Nearly all weights
A: 0.1%–1%

Q: What technique does GPTQ use to compensate for quantization error as it processes each layer?
- Random rounding
- Second-order (Hessian-weighted) error compensation
- Per-channel activation scaling
- Simple rounding-to-nearest with no compensation
A: Second-order (Hessian-weighted) error compensation

Q: Name one concrete risk of GPTQ's calibration approach, and briefly explain why it happens.
Feedback: Sample/expected answer: Overfitting to the calibration dataset, which can cause out-of-distribution accuracy distortion — because the Hessian-weighted updates are optimized specifically for the statistics of the calibration data, they may not generalize to production traffic that looks different.

Q: FP8 KV-cache quantization should always give a large throughput improvement, regardless of serving engine or workload shape.
- True
- False
A: False

Q: Which combination does research suggest often works best for KV-cache quantization?
- Q4 keys and Q4 values (symmetric)
- Q8 keys and Q4 values
- Q4 keys and Q8 values
- Keys and values must always match precision
A: Q4 keys and Q8 values

Q: As of 2026, which library is the maintained integration point for GPTQ quantization in Hugging Face Transformers, Optimum, and PEFT?
- AutoGPTQ
- GPTQModel
- bitsandbytes
- AutoAWQ
A: GPTQModel

Q: Name the two FP8 formats supported on NVIDIA Hopper/Blackwell hardware and briefly describe the trade-off between them.
Feedback: Sample/expected answer: E4M3 and E5M2. E4M3 has more mantissa bits (more precision, less dynamic range); E5M2 has more exponent bits (more dynamic range, less precision).