Title: Quiz 14 — Fractional GPUs, MIG, Quantization, and KV-Cache Savings
Course: AIINFRA 302: AI Security, Guardrails & Governance > Section: Week 14 — Fractional GPUs, MIG, Quantization, and KV-Cache Savings

Q: MIG and time-slicing both provide hardware-enforced memory isolation between jobs sharing a GPU.
- True
- False
A: False

Q: Which GPU generations support NVIDIA MIG?
- Any CUDA-capable GPU
- Pascal and newer
- Ampere and newer (e.g., A100, L40, H100)
- Only H100 and H200
A: Ampere and newer (e.g., A100, L40, H100)

Q: Name one risk of using time-slicing instead of MIG for a multi-tenant workload where one tenant is untrusted or unpredictable.
Feedback: Sample/expected answer: A "noisy neighbor" can starve or cause an out-of-memory (OOM) failure in other tenants' jobs, since there is no memory isolation between them.

Q: Moving KV cache and/or weights from FP16 to FP8 on FP8-native hardware (like H100) can roughly:
- Triple memory usage with no throughput change
- Halve memory and roughly double throughput
- Have no effect on memory or throughput
- Only reduce throughput, never memory
A: Halve memory and roughly double throughput

Q: INT8 KV-cache quantization is typically near-lossless, but INT4 and lower precisions measurably degrade generation quality.
- True
- False
A: True

Q: Between the key cache and the value cache, which is generally more sensitive to quantization-induced quality loss?
Feedback: Sample/expected answer: The key cache is more sensitive than the value cache.

Q: What does vLLM's PagedAttention primarily solve?
- It compresses model weights permanently to reduce file size on disk
- It manages the KV cache in fixed-size pages (like OS virtual memory) to reduce fragmentation and enable larger batches
- It replaces the need for any GPU quantization
- It only works on CPU-based inference
A: It manages the KV cache in fixed-size pages (like OS virtual memory) to reduce fragmentation and enable larger batches

Q: Why can't you reliably size a GPU deployment by looking at model weight memory alone?
Feedback: Sample/expected answer: Because the KV cache scales with batch size × sequence length and often rivals or exceeds weight memory, directly limiting how many requests can be served concurrently.