🛠️ Lab 9 — Deploying and Scaling a Model with Ray Serve and KubeRay (50 pts)

Goal: Deploy the same small model two ways — as a standalone Ray Serve application and as a RayService on a local Kubernetes cluster via the KubeRay operator — then load-test both to observe autoscaling behavior and compare the developer experience. All tools used are free and run locally; no GPU or cloud account is required. Steps:

1. Set up your environment. On your local machine (or a free Colab CPU runtime), create a Python virtual environment and install Ray: pip install "ray[serve]". Verify with ray --version.
2. Deploy standalone Ray Serve. Follow the official Ray Serve example (from the Resources list below) to serve a small text or image classification model (for example, a MobileNet-style example) as a Ray Serve deployment. Confirm you can send it a single request and get a prediction back.
3. Load-test the standalone deployment. Write a short concurrent request script (Python with concurrent.futures or a simple loop using requests/httpx) that fires 20–50 concurrent requests at your endpoint. Record response times and observe request routing across replicas in the Ray dashboard.
4. Scale replicas manually. Change num_replicas in your Ray Serve deployment config (try 1, then 2, then 4) and re-run your load test after each change. Record how latency and throughput shift as you add replicas.
5. Stand up a local Kubernetes cluster. Use kind or minikube to create a free local cluster (kind create cluster or minikube start).
6. Install KubeRay. Install the open-source KubeRay operator via Helm following the official KubeRay GitHub examples in Resources.
7. Deploy a RayService. Write and apply a RayService custom resource manifest that serves the same (or an equivalently small) model, following the official Ray-on-Kubernetes and KubeRay examples.
8. Repeat your load test against RayService. Run your same concurrent request script against the RayService endpoint (via kubectl port-forward or the exposed service) and record the same latency/throughput metrics.
9. Compare and reflect. In a short written report, compare setup complexity, time-to-first-request, scaling behavior, and observability between the standalone Ray Serve deployment and the KubeRay/RayService deployment.

Deliverables: Submit (a) your Ray Serve deployment code and RayService YAML manifest, (b) your load-test script, (c) a results table or chart showing latency/throughput at each replica count for both deployment methods, and (d) a 300–500 word written comparison of the two developer experiences, submitted as a file upload or text entry.