🛠️ Lab 8 — Containerize and Deploy a FastAPI Inference Service to kind (50 pts)

Goal: Take the small FastAPI inference app you built in Week 3 (serving a pre-trained scikit-learn or PyTorch model on Iris or MNIST data), package it as a Docker image, load that image into a local kind cluster, and deploy it with a Kubernetes Deployment and Service — then scale it and watch requests load-balance across replicas. Steps:

  1. Revisit your Week 3 app. Confirm your FastAPI app exposes a /predict endpoint that accepts JSON (e.g. {"features": [...]}) and returns a prediction from your Iris or MNIST model. Make sure it runs locally with uvicorn before containerizing it.
  2. Write a Dockerfile (free, local docker — no cloud registry needed) that installs your dependencies, copies your app and model file, and starts the server with uvicorn app:app --host 0.0.0.0 --port 8000.
  3. Build the image with docker build -t inference-app:week9 . and confirm it runs correctly with docker run -p 8000:8000 inference-app:week9 plus a local curl test.
  4. Create a local kind cluster (if you don't already have one) with kind create cluster --name aiinfra102.
  5. Load your image into the cluster with kind load docker-image inference-app:week9 --name aiinfra102. (If you're using minikube instead, use minikube image load inference-app:week9.) This step matters because kind/minikube clusters can't pull from your local Docker daemon automatically — you have to push the image in explicitly.
  6. Write a Deployment manifest (deployment.yaml) requesting 2 replicas, referencing your inference-app:week9 image with imagePullPolicy: Never (so Kubernetes uses the locally loaded image instead of trying to pull from a registry), and exposing container port 8000.
  7. Write a Service manifest (service.yaml) of type ClusterIP that selects your Deployment's Pods and forwards port 8000.
  8. Apply both manifests with kubectl apply -f deployment.yaml -f service.yaml and confirm with kubectl get pods and kubectl get svc that everything is running.
  9. Port-forward to the Service with kubectl port-forward svc/inference-service 8000:8000, then test inference from another terminal with curl -X POST localhost:8000/predict -d '{"features": [...]}' (use real feature values matching your model's expected input).
  10. Scale the Deployment with kubectl scale deployment inference-app --replicas=3, confirm three Pods are running with kubectl get pods, and send several more curl requests. Use kubectl logs on each Pod (or add a Pod-name field to your API response) to confirm requests are load-balancing across replicas rather than all hitting one Pod.

Deliverables: Submit (a) your Dockerfile, deployment.yaml, and service.yaml; (b) terminal output/screenshots showing the kind load docker-image step, kubectl get pods before and after scaling, and at least three successful curl responses; and (c) a short written reflection (150–250 words) on what you observed when scaling from 2 to 3 replicas and how you confirmed load-balancing was actually happening.