📚 Resources

• Title: KServe and Triton Inference Server for model serving on Kubernetes — URL: https://kserve.github.io/website/docs/model-serving/predictive-inference/frameworks/overview
• Title: vLLM vs Triton vs KServe — where each fits — URL: https://www.kubenatives.com/p/vllm-vs-triton-vs-kserve-kubernetes
• Title: Deploy a FastAPI ML inference service to a local kind cluster end-to-end — URL: https://medium.com/@austin.adeyanju/deploying-machine-learning-models-on-kubernetes-a-practical-guide-with-fastapi-docker-and-kind-048fdf1483f4
• Title: Run a Stateless Application Using a Deployment — URL: https://kubernetes.io/docs/tasks/run-application/run-stateless-application-deployment/
• Title: Using a Service to Expose Your App — URL: https://kubernetes.io/docs/tutorials/kubernetes-basics/expose/expose-intro/

Week 09 · Scaling & GPU-Aware Scheduling: HPA and the NVIDIA GPU Operator