Title: Week 08 Resources
Course: AIINFRA 200: Production Inference Serving & GPU Orchestration > Section: Week 08 — Serving Engine Benchmarking Lab

📚 Resources

- Title: NVIDIA GenAI-Perf benchmarking tool — URL: https://developer.nvidia.com/blog/llm-performance-benchmarking-measuring-nvidia-nim-performance-with-genai-perf/
- Title: vLLM built-in bench CLI (serve/latency/throughput) — URL: https://docs.vllm.ai/en/latest/benchmarking/cli/
- Title: Core LLM serving benchmark metrics — URL: https://developer.nvidia.com/blog/llm-benchmarking-fundamental-concepts/
- Title: Representative cross-engine benchmark results — URL: https://www.marktechpost.com/2025/11/19/vllm-vs-tensorrt-llm-vs-hf-tgi-vs-lmdeploy-a-deep-technical-comparison-for-production-llm-inference/
- Title: vLLM Production Metrics Documentation — URL: https://docs.vllm.ai/en/stable/usage/metrics/

Week 09 · Ray Serve and KServe for Scalable Model Serving

Notion ID: 392c08fd-0278-8186-850e-d653c4209f00 Notion URL: https://app.notion.com/p/392c08fd02788186850ed653c4209f00