Title: Week 13 Resources
Course: AIINFRA 200: Production Inference Serving & GPU Orchestration > Section: Week 13 — Autoscaling, Load Balancing, and the KAI Scheduler

📚 Resources

- Title: KAI Scheduler (open-sourced Run:ai engine) for Kubernetes GPU scheduling: https://github.com/NVIDIA/KAI-Scheduler
- Title: KEDA-based autoscaling on GPU/queue-depth metrics: https://www.cncf.io/blog/2026/05/27/gpu-autoscaling-on-kubernetes-with-keda-building-an-external-scaler/
- Title: KAI Scheduler Documentation (NVIDIA Cloud Functions): https://docs.nvidia.com/nvcf/kai-scheduler
- Title: Horizontal Pod Autoscaling — Kubernetes Documentation: https://kubernetes.io/docs/concepts/workloads/autoscaling/horizontal-pod-autoscale/
- Title: HorizontalPodAutoscaler Walkthrough: https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale-walkthrough/

Week 14 · Disaggregated Prefill/Decode with NVIDIA Dynamo

Notion ID: 392c08fd-0278-816e-8830-e27f150ad3f8 Notion URL: https://app.notion.com/p/392c08fd0278816e8830e27f150ad3f8