📚 Resources

• Title: vLLM tensor-parallel and pipeline-parallel serving flags: https://docs.vllm.ai/en/stable/serving/parallelism_scaling/
• Title: Parallelism and Scaling — vLLM Documentation: https://docs.vllm.ai/en/stable/serving/parallelism_scaling/
• Title: Tensor Parallelism vs Pipeline Parallelism for Multi-GPU: https://gigagpu.com/tensor-vs-pipeline-parallelism/
• Title: Parallelism in TensorRT-LLM: https://nvidia.github.io/TensorRT-LLM/features/parallel-strategy.html
• Title: Parallelisms Guide — Megatron Bridge (NVIDIA NeMo): https://docs.nvidia.com/nemo/megatron-bridge/latest/parallelisms.html

Week 13 · Autoscaling, Load Balancing, and the KAI Scheduler

Notion ID: 392c08fd-0278-818e-a7e2-e9d13f8fa4dd Notion URL: https://app.notion.com/p/392c08fd0278818ea7e2e9d13f8fa4dd