Title: Discussion 9 — Ray Serve or KServe: What Would You Recommend?
Course: AIINFRA 200: Production Inference Serving & GPU Orchestration > Section: Week 09 — Ray Serve and KServe for Scalable Model Serving

💬 Discussion 9 — Ray Serve or KServe: What Would You Recommend? (20 pts)

Imagine you're advising a mid-sized organization that already runs its applications on Kubernetes and now wants to add inference workloads for both a couple of stable production models and a rapidly-changing set of experimental pipelines. Should they choose Ray Serve, KServe, or some combination of both? In your post, address:

- What organizational factors (team experience with Python vs. Kubernetes operators, existing infrastructure investment, how frequently models change) should drive the decision?
- Where in this scenario would Ray Serve's flexibility be worth the extra operational surface area, and where would KServe's Kubernetes-native standardization be the safer choice?
- Would you ever recommend running both in the same organization? Why or why not?

Post your initial response (150+ words), then respond to at least two classmates, engaging directly with a factor or tradeoff they raised that you didn't cover yourself.