Title: Discussion 13 — CPU-Bound vs. GPU-Bound Observability
Course: AIINFRA 200: Production Inference Serving & GPU Orchestration > Section: Week 13 — Autoscaling, Load Balancing, and the KAI Scheduler

💬 Discussion 13 — CPU-Bound vs. GPU-Bound Observability (20 pts)

How should observability and autoscaling strategies differ between CPU-bound inference workloads and GPU-bound workloads, and what metrics matter most for each? In your initial post, address: what signal(s) you would dashboard and alert on for each workload type, why CPU/memory-based HPA is (or isn't) sufficient in each case, and one real or plausible scenario where trusting the wrong metric would have caused an outage or a wasted-spend incident. Then respond to at least two classmates, comparing your metric choices to theirs and noting any tradeoffs they may have missed.