📚 Resources

• Title: NVIDIA GenAI-Perf benchmarking tool — URL: https://developer.nvidia.com/blog/llm-performance-benchmarking-measuring-nvidia-nim-performance-with-genai-perf/
• Title: vLLM built-in bench CLI (serve/latency/throughput) — URL: https://docs.vllm.ai/en/latest/benchmarking/cli/
• Title: Core LLM serving benchmark metrics — URL: https://developer.nvidia.com/blog/llm-benchmarking-fundamental-concepts/
• Title: Representative cross-engine benchmark results — URL: https://www.marktechpost.com/2025/11/19/vllm-vs-tensorrt-llm-vs-hf-tgi-vs-lmdeploy-a-deep-technical-comparison-for-production-llm-inference/
• Title: vLLM Production Metrics Documentation — URL: https://docs.vllm.ai/en/stable/usage/metrics/

Week 09 · Ray Serve and KServe for Scalable Model Serving

Notion ID: 392c08fd-0278-8186-850e-d653c4209f00 Notion URL: https://app.notion.com/p/392c08fd02788186850ed653c4209f00