If you've worked with AI models before, you already know the pain of "it works on my machine." A teammate installs a different version of CUDA, or a library, or a Python patch release, and suddenly your training script crashes on their laptop. Containers exist to solve exactly this problem, and understanding how they solve it — at the operating-system level — is the foundation for everything else in this course, including the Kubernetes orchestration we'll cover in later weeks.
For decades, the standard way to isolate a workload was the virtual machine (VM). A hypervisor (like VMware, KVM, or Hyper-V) virtualizes physical hardware — CPU, memory, disk, network — and each VM boots its own complete guest operating system on top of that virtual hardware. That guest OS has its own kernel, its own drivers, its own init process. It's a fully separate computer that happens to be running inside another computer. Containers take a different approach. Instead of virtualizing hardware and booting a new OS, a container shares the host machine's kernel and uses kernel features to create the illusion of an isolated environment for a single process (or small group of processes). There's no second kernel to boot, no virtual BIOS, no guest OS startup sequence. This is why a container can start in milliseconds to a few seconds, while a VM typically takes 30–90 seconds or more to boot.
| Aspect | Virtual Machine | Container |
|---|---|---|
| What's virtualized | Physical hardware | Operating system processes |
| Includes a guest OS/kernel? | Yes, full OS | No — shares host kernel |
| Typical startup time | Tens of seconds to minutes | Milliseconds to a few seconds |
| Disk footprint | Gigabytes (full OS + app) | Megabytes to low gigabytes (app + deps only) |
| Isolation strength | Strong (separate kernel) | Weaker (shared kernel) |
| Density per host | Lower (fewer VMs fit) | Higher (many more containers fit) |
| Good fit for AI workloads | Full-stack test environments, untrusted multi-tenant kernels | Fast-scaling inference, reproducible training environments, CI/CD |
For AI infrastructure specifically, this efficiency difference is enormous. Imagine you need to autoscale an inference service from 2 replicas to 50 replicas in response to traffic. If each replica is a VM, you're waiting minutes and paying for a full guest OS's memory overhead per replica. If each replica is a container, you can scale in seconds and pack far more replicas onto the same GPU host, because containers aren't paying the "tax" of a redundant kernel per instance.
Containers aren't magic — they rely on two specific Linux kernel primitives:
Together, namespaces (isolation of view) and cgroups (limits on usage) are the entire secret behind container isolation. When you later request a GPU limit in a Kubernetes pod spec, you are ultimately configuring cgroup-style resource controls under the hood.
A Docker image is a read-only, layered template. Each instruction in a Dockerfile (e.g., installing a package, copying in code) creates a new layer stacked on the ones before it. Layers are cached and reused, so pulling an updated image often only requires downloading the one or two layers that changed — not the whole thing. Critically, an image does not include a full guest OS or kernel; it typically includes just enough of a minimal filesystem (a base image like ubuntu or alpine) plus your application and its dependencies. A VM image, by contrast, must contain an entire installable operating system, because the VM will boot its own kernel from scratch. An image is also not the same thing as a container. The image is the immutable template sitting in a registry or on disk. A container is what you get when you run that image: a live (or stopped) instance with its own thin writable layer on top of the image's read-only layers. You can start ten containers from one image; each gets its own writable layer and process namespace, but they all share the same underlying read-only image layers on disk.
Early in Docker's history, the image and runtime formats were Docker-specific. The Open Container Initiative (OCI) was created to standardize the container image format and runtime specification so that no single vendor controls the ecosystem. Because of the OCI standard, an image built with Docker can be run by containerd, CRI-O, or other OCI-compliant runtimes — this is exactly why Kubernetes can swap out its underlying container runtime without breaking the images your team already built.
Sharing a kernel is what makes containers fast and dense, but it's also a security tradeoff. Because every container on a host ultimately makes system calls into the same kernel, a kernel-level vulnerability or a container escape bug can potentially affect every container on that host. A VM's guest kernel is fully separate from the host kernel, so a compromised VM is far less likely to affect its neighbors. Neither model is "better" in isolation — the right choice depends on your trust boundary, which is exactly what this week's discussion asks you to reason through.
"Containers are lightweight VMs." This is a popular but misleading shorthand. Containers do not include a full guest OS or kernel the way VMs do. They share the host's kernel and isolate processes using namespaces and cgroups. Calling them "lightweight VMs" obscures why they're lightweight — it isn't that they're a smaller VM, it's that they aren't virtualizing hardware or booting an OS at all. "An image and a container are the same thing." They are not. The image is an immutable, read-only template. A container is a running (or stopped) instance created from that image, with its own writable layer. Understanding this distinction now will save you confusion later when we discuss image registries, tagging, and multi-stage builds.