Title: Quiz 15 — Observability, Cost Analysis, and Right-Sizing
Course: AIINFRA 200: Production Inference Serving & GPU Orchestration > Section: Week 15 — Observability, Cost Analysis, and Right-Sizing

Q: Which tool is most appropriate for a single developer doing a quick sanity check on GPU memory and utilization while iterating locally?
- DCGM Exporter
- Prometheus
- `nvidia-smi`
- Grafana
A: `nvidia-smi`

Q: A GPU reporting 100% utilization in nvidia-smi or DCGM guarantees that the workload is compute-efficient and well-parallelized.
- True
- False
A: False

Q: What Prometheus endpoint does the DCGM exporter expose GPU telemetry on, and what monitoring/visualization tools typically consume that endpoint in a production stack?
Feedback: Sample/expected answer: It exposes telemetry at a ****`/metrics`**** endpoint, which Prometheus scrapes and stores as time series; Grafana then queries Prometheus to render dashboards.

Q: Which vLLM-specific metric would you check first to determine whether incoming requests are being queued because the engine cannot schedule them yet?
- `vllm:gpu_cache_usage_perc`
- `vllm:num_requests_waiting`
- GPU-Util from DCGM
- Memory-copy utilization
A: `vllm:num_requests_waiting`

Q: Industry surveys indicate that most organizations run their GPUs above 85% utilization even at peak load.
- True
- False
A: False

Q: A GPU shows high utilization but low actual throughput, and DCGM shows high memory-copy utilization alongside plenty of free VRAM. What is the most likely bottleneck?
- Insufficient VRAM capacity
- Memory bandwidth — compute units are stalled waiting on data transfer
- The GPU driver is misconfigured
- The model is too small for the GPU
A: Memory bandwidth — compute units are stalled waiting on data transfer

Q: Explain, in your own words, why switching a Llama-3-70B deployment from BF16 to a 4-bit format like AWQ is described as the single biggest cost optimization available, referencing the GPU hardware difference it enables.
Feedback: Sample/expected answer: BF16 weights require at least two H100 80GB GPUs (\~\$2.69/hr each) to fit the model, while 4-bit AWQ quantization shrinks memory needs enough to run on dual RTX A6000 GPUs (\~\$0.49/hr each) — a much cheaper hardware class — while serving the same model.

Q: A 100-GPU cluster billed at \$2/hr per GPU improves fleet utilization by 10 percentage points through better batching and right-sizing. Approximately how much does this save per year (per the figures discussed this week)?
- About \$17,500/year
- About \$175,000/year
- About \$1,750,000/year
- About \$75,000/year
A: About \$175,000/year