Title: Discussion 9 — Beyond Average CPU: What Scaling Signals Are We Missing?
Course: AIINFRA 101: Applied Infrastructure & Containerization > Section: Week 09 — Scaling & GPU-Aware Scheduling: HPA and the NVIDIA GPU Operator

💬 Discussion 9 — Beyond Average CPU: What Scaling Signals Are We Missing? (20 pts)

HPA's default behavior scales on the average resource utilization across all Pods in a Deployment. Imagine a service running 10 replicas where 9 are healthy and fast but 1 is struggling — average CPU looks perfectly normal, so HPA sees no reason to scale, yet real users are hitting serious tail latency.

In a post of at least 150 words, answer:

- Why is average utilization a poor proxy for tail-latency problems like P99 degradation?
- What additional signal(s) — custom metrics, external metrics, or a different scaling strategy — would you layer on top of CPU/memory-based HPA to catch this?
- If this same service were GPU-backed and shared via time-slicing instead of CPU-bound, how would the "average looks fine, one workload is starving" failure mode change — better, worse, or just different?

Then respond to at least two classmates — push back on their proposed metric if it has a blind spot, or build on it with a complementary signal.