Title: Quiz 15 — Autoscaling Economics, Spot Instances, and Build-vs-Buy
Course: AIINFRA 302: AI Security, Guardrails & Governance > Section: Week 15 — Autoscaling Economics, Spot Instances, and Build-vs-Buy

Q: In the KEDA + Karpenter two-layer autoscaling pattern, which statement correctly describes the division of labor?
- KEDA provisions GPU nodes; Karpenter scales pods based on metrics
- KEDA scales pods based on custom metrics (including scale-to-zero); Karpenter provisions/deprovisions GPU nodes
- KEDA and Karpenter both operate only at the node level
- Karpenter handles scale-to-zero; KEDA handles Spot instance selection
A: KEDA scales pods based on custom metrics (including scale-to-zero); Karpenter provisions/deprovisions GPU nodes

Q: AWS recommends consolidationPolicy: WhenEmpty with a consolidateAfter of around one hour for spiky inference workloads, to avoid repeatedly tearing down and rebuilding nodes.
- True
- False
A: True

Q: Why does naive scale-to-zero for GPU inference pods often fail to deliver the expected savings?
- Kubernetes does not support scaling GPU pods to zero
- GPU cold starts (image pull, runtime startup, multi-GB model load into VRAM) are slow, so aggressive scale-to-zero can burn money and drop requests during cold-start windows
- Spot instances cannot be scaled to zero
- KEDA only supports scale-to-zero for CPU workloads
A: GPU cold starts (image pull, runtime startup, multi-GB model load into VRAM) are slow, so aggressive scale-to-zero can burn money and drop requests during cold-start windows

Q: Name one workload characteristic that makes Spot/preemptible GPUs a good fit, and one that makes them a poor fit.
Feedback: Sample/expected answer: Good fit: batch, interruptible, or checkpointed/retryable workloads (e.g., offline batch inference, fine-tuning). Poor fit: latency-critical, synchronous, user-facing APIs with tight P99 SLAs.

Q: Self-hosting an LLM is always cheaper than paying per-token API prices, regardless of traffic volume.
- True
- False
A: False

Q: Roughly what sustained utilization threshold is commonly cited as the point where self-hosting starts to beat managed API pricing?
- Any utilization above 0%
- Roughly \>50% utilization, or on the order of 10M+ tokens/day per GPU
- Exactly 100% utilization only
- Utilization is irrelevant to the comparison
A: Roughly \>50% utilization, or on the order of 10M+ tokens/day per GPU

Q: A team's GPU rental bill for self-hosting is \$4,000/month, but the lecture noted self-hosting "commonly runs 3–5x the raw GPU price" once a certain cost category is included. Name that cost category and roughly what it costs per year.
Feedback: Sample/expected answer: DevOps/operations maintenance (on-call, patching, scaling), commonly estimated around \$145,000/year.

Q: As of December 2025, what happened to Hugging Face's Text Generation Inference (TGI), and what does this illustrate about build-vs-buy decisions?
- TGI was acquired by AWS; it illustrates that self-hosting tools only get more support over time
- TGI entered maintenance mode with Hugging Face recommending migration to vLLM or SGLang; it illustrates that vendor/tool longevity risk is a real, often-invisible cost in build-vs-buy decisions
- TGI became the default inference engine for all cloud providers; it illustrates that self-hosting has no hidden risk
- TGI was merged into Karpenter; it illustrates that autoscaling and inference serving tools are converging
A: TGI entered maintenance mode with Hugging Face recommending migration to vLLM or SGLang; it illustrates that vendor/tool longevity risk is a real, often-invisible cost in build-vs-buy decisions