📚 Resources

• NVIDIA Container Toolkit setup for Docker GPU runtime: https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html
• Docker Compose GPU reservations and CDI: https://docs.docker.com/compose/how-tos/gpu-support/
• Ollama install and systemd service (dev-only serving): https://docs.ollama.com/linux
• LM Studio headless mode and lms CLI: https://lmstudio.ai/docs/developer/core/headless
• NVIDIA/nvidia-container-toolkit GitHub Repository: https://github.com/NVIDIA/nvidia-container-toolkit

Week 03 · Production Inference Fundamentals and the vLLM Engine

Notion ID: 392c08fd-0278-818b-9a36-efd09d788f43 Notion URL: https://app.notion.com/p/392c08fd0278818b9a36efd09d788f43