Title: Week 14 Resources
Course: AIINFRA 302: AI Security, Guardrails & Governance > Section: Week 14 — Fractional GPUs, MIG, Quantization, and KV-Cache Savings

📚 Resources

- NVIDIA MIG (Multi-Instance GPU): https://docs.nvidia.com/datacenter/cloud-native/gpu-operator/latest/gpu-sharing.html
- vLLM llm-compressor — quantization and KV-cache compression: https://github.com/vllm-project/llm-compressor
- vLLM Quantized KV Cache documentation: https://docs.vllm.ai/en/latest/features/quantization/quantized_kvcache/
- Comparing MIG and Time-Slicing for GPU Resource Sharing — OpenMetal: https://openmetal.io/resources/blog/mig-vs-time-slicing-gpu-sharing/
- Unlocking Longer Generation with KV Cache Quantization — Hugging Face: https://huggingface.co/blog/kv-cache-quantization
- vLLM: Easy, Fast, and Cheap LLM Serving with PagedAttention: https://blog.vllm.ai/2023/06/20/vllm.html

Week 15 · Autoscaling Economics, Spot Instances, and Build-vs-Buy

Notion ID: 392c08fd-0278-8181-abd8-c730d32d9d8e Notion URL: https://app.notion.com/p/392c08fd02788181abd8c730d32d9d8e