📚 Resources

• NVIDIA MIG (Multi-Instance GPU): https://docs.nvidia.com/datacenter/cloud-native/gpu-operator/latest/gpu-sharing.html
• vLLM llm-compressor — quantization and KV-cache compression: https://github.com/vllm-project/llm-compressor
• vLLM Quantized KV Cache documentation: https://docs.vllm.ai/en/latest/features/quantization/quantized_kvcache/
• Comparing MIG and Time-Slicing for GPU Resource Sharing — OpenMetal: https://openmetal.io/resources/blog/mig-vs-time-slicing-gpu-sharing/
• Unlocking Longer Generation with KV Cache Quantization — Hugging Face: https://huggingface.co/blog/kv-cache-quantization
• vLLM: Easy, Fast, and Cheap LLM Serving with PagedAttention: https://blog.vllm.ai/2023/06/20/vllm.html

Week 15 · Autoscaling Economics, Spot Instances, and Build-vs-Buy

Notion ID: 392c08fd-0278-8181-abd8-c730d32d9d8e Notion URL: https://app.notion.com/p/392c08fd02788181abd8c730d32d9d8e