Title: Quiz 13 — Autoscaling, Load Balancing, and the KAI Scheduler
Course: AIINFRA 200: Production Inference Serving & GPU Orchestration > Section: Week 13 — Autoscaling, Load Balancing, and the KAI Scheduler

Q: Standard Kubernetes HPA using CPU metrics reliably detects when a GPU inference pod is overloaded.
- True
- False
A: False

Q: Which two signals does this week's lecture recommend for autoscaling GPU inference workloads instead of CPU?
- Disk I/O and network latency
- GPU utilization and request queue depth
- Pod restart count and node count
- Memory limits and memory requests
A: GPU utilization and request queue depth

Q: What is the KAI Scheduler, and who open-sourced it?
Feedback: Sample/expected answer: A high-performance Kubernetes scheduler for orchestrating GPU clusters across the AI lifecycle (interactive dev through distributed training and inference); open-sourced by NVIDIA (via its Run:ai engine) in April 2025, and now a CNCF Sandbox project.

Q: What is "gang scheduling"?
- Scheduling pods in the order they were submitted
- All-or-nothing coordinated placement of every pod in a distributed job
- A method for compressing container images
- A load-balancing strategy for round-robin traffic
A: All-or-nothing coordinated placement of every pod in a distributed job

Q: The Kubernetes Cluster Autoscaler automatically reads fractional GPU requests placed in pod annotations by GPU-sharing tools.
- True
- False
A: False

Q: Which tool did this week's lab use to scale inference pod replicas based on an external queue-depth metric from Prometheus?
- KAI Scheduler
- KEDA
- Karpenter
- kube-scheduler
A: KEDA

Q: Name one capability the KAI Scheduler provides that lets high-priority inference jobs take precedence over lower-priority training jobs on the same cluster.
Feedback: Sample/expected answer: Priority-based queues with preemption — KAI lets high-priority inference workloads preempt lower-priority training workloads sharing the same cluster.

Q: Which load-balancing strategy routes each new request to the pod with the fewest active connections, rather than distributing requests in strict sequence?
- Round-robin
- Least-connections
- Random
- Sticky sessions
A: Least-connections