Through Weeks 6–8 you told Kubernetes exactly how many replicas of a Deployment you wanted, and it kept that number running. This week you hand two harder decisions over to the cluster: how many Pods should be running right now, and which node has the specialized hardware a Pod actually needs.
Part 1 — The Horizontal Pod Autoscaler. HPA is a controller that watches a resource metric — usually CPU or memory utilization — and automatically adjusts a Deployment's replicas field to keep that metric near a target you set. This is "horizontal" scaling because it changes the number of Pods, not the resources given to each one. By default, every 15 seconds the HPA controller queries a metrics source, compares usage against your target, and computes:
desiredReplicas = ceil[ currentReplicas * ( currentMetricValue / desiredMetricValue ) ]With 2 replicas at 80% CPU against a 50% target: ceil[2 * (80/50)] = 4. HPA is a proportional controller — it scales in direct proportion to how far off-target usage is, not by guessing a number.
HPA doesn't measure anything itself — it queries the Metrics Server, a cluster add-on that scrapes kubelet resource data and exposes it through the metrics.k8s.io API. No Metrics Server means kubectl top errors and any HPA sits reporting <unknown>. Always write HPA manifests against autoscaling/v2 (not v1) — it supports custom/external metrics and richer status conditions for debugging.
The detail that trips up almost everyone: HPA's utilization formula is a percentage of the requested amount. No resources.requests.cpu on the container means no denominator, so HPA reports unknown and never scales — full stop. Because metrics are noisy, HPA's behavior block also lets you set separate stabilization windows: scale-up reacts immediately (0s) so spikes are met quickly, while scale-down defaults to a 300s window so a single dip doesn't trigger thrashing.
Three corrections worth internalizing: (1) HPA cannot compute anything without requests defined — it doesn't degrade gracefully, it just stalls. (2) HPA typically takes 1–2 minutes to observe a sustained change and act, so it can't out-run a sharp burst, and it cannot heal an unhealthy Pod — scaling a crash-looping container just produces more copies of the same broken thing. (3) CPU and memory limits fail differently under pressure: exceeding a CPU limit only throttles the container, but exceeding a memory limit gets it OOMKilled and restarted — so tight memory limits are far riskier on an autoscaled workload than tight CPU limits.
Part 2 — GPU-aware scheduling. Kubernetes has no built-in concept of "this node has a GPU." Historically that meant hand-installing the NVIDIA driver, the NVIDIA Container Toolkit, and a device plugin on every GPU node, and keeping them version-aligned forever. The NVIDIA GPU Operator (standardizing on v26.3.3 this term) packages the driver, Container Toolkit, Kubernetes device plugin, GPU Feature Discovery (GFD), and DCGM Exporter into one Helm-installable operator that rolls them out to every GPU node as a DaemonSet — including nodes you add later:
helm install --wait --generate-name \
-n gpu-operator --create-namespace \
nvidia/gpu-operator --version=v26.3.3Once it completes, every GPU node exposes nvidia.com/gpu as a schedulable resource, exactly the way nodes natively expose cpu and memory. The device plugin registers the GPU with the kubelet; GFD labels nodes with GPU model/capability metadata for placement decisions; DCGM Exporter exposes utilization/memory/temperature/power as Prometheus metrics with no separate agent to deploy.
A Pod requests a GPU the same way it requests CPU or memory — resources.limits: {nvidia.com/gpu: 1} — and by default gets exclusive, whole-device access: no other Pod can touch that card, even at 10% utilization. GPU time-slicing (a ConfigMap telling the operator to advertise, say, 4 slices per physical GPU) lets multiple Pods share one card for small, latency-tolerant workloads — but it provides no memory isolation: all sliced workloads share the same VRAM pool, so one workload can starve its neighbors. Never time-slice a training job or anything with high/unpredictable memory needs.
Two corrections: with the Operator installed, running nvidia-smi directly on the host OS won't work. The driver lives inside a driver pod, so you kubectl exec into it. And a manual, node-by-node device-plugin setup is not "basically the same thing" as the Operator — the Operator automates the entire stack (driver, toolkit, plugin, GFD, monitoring) through one Helm install, kept consistent automatically across the whole cluster.
The throughline for this week: both HPA and the GPU Operator turn a manual, per-node/per-decision chore into an automated control loop — but neither one can reason about workload health, only about resource quantity.