Title: Quiz 13 — GPU FinOps Foundations: Cost per 1,000 Requests and Right-Sizing
Course: AIINFRA 302: AI Security, Guardrails & Governance > Section: Week 13 — GPU FinOps Foundations: Cost per 1,000 Requests and Right-Sizing

Q: Which formula correctly expresses cost per million tokens?
- (requests/hour) / (GPU \$/hour) × 1,000,000
- (cluster \$/hour) / (tokens/sec × 3600) × 1,000,000
- (GPU \$/hour) × (tokens/sec) × 3600
- (tokens/sec) / (GPU \$/hour)
A: (cluster \$/hour) / (tokens/sec × 3600) × 1,000,000

Q: Cost per 1,000 requests is a fixed, directly comparable number across workloads, regardless of prompt or completion length.
- True
- False
A: False

Q: On the same H100 running Mixtral-8x7B, what is the approximate difference in cost per million tokens between 1 request/sec and 25 requests/sec?
- No real difference — GPU cost is fixed
- About 2x
- About 15x or more
- About 100x
A: About 15x or more

Q: Name the single dominant lever that explains why the same GPU can have wildly different cost-per-token outcomes.
Feedback: Sample/expected answer: Utilization (achieved through batching/request rate against the GPU)

Q: Right-sizing means always choosing the largest available GPU so that the workload never bottlenecks.
- True
- False
A: False

Q: A workload shows SM utilization consistently below 50% and VRAM usage below 40%. What does this suggest?
- The GPU is correctly sized
- The workload is over-tiered and could run on a smaller/cheaper GPU
- The model is too large to fit
- Batching is already maximized
A: The workload is over-tiered and could run on a smaller/cheaper GPU

Q: Why must a latency SLA be fixed before optimizing for cost, rather than after?
Feedback: Sample/expected answer: Because the SLA caps how aggressively requests can be batched; batching raises throughput and lowers cost per token but also increases per-request latency, so optimizing cost first risks violating the latency promise.

Q: In the lecture's build-vs-buy example, self-hosting only beats hosted API pricing (like GPT-4o-mini) above approximately what volume?
- 1 million tokens/month
- 500 million tokens/month
- 11 billion tokens/month
- 1 trillion tokens/month
A: 11 billion tokens/month