Title: Lecture 11 — From Device Plugins to Dynamic Resource Allocation and MIG
Course: AIINFRA 200: Production Inference Serving & GPU Orchestration > Section: Week 11 — Dynamic Resource Allocation and MIG Partitioning

📖 Lecture — From Device Plugins to Dynamic Resource Allocation and MIG

For most of Kubernetes' history, requesting a GPU meant one thing: asking the scheduler for an integer count of an opaque resource, like nvidia.com/gpu: 1. The device-plugin model that powered this was simple but blunt. It could hand a pod "a GPU," but it had no vocabulary for which kind of GPU, how much of it, or what shape of GPU instance a workload actually needed. That gap is exactly what this week's two technologies close: NVIDIA Multi-Instance GPU (MIG) partitioning, which physically carves up a GPU, and Kubernetes Dynamic Resource Allocation (DRA), which gives the scheduler a rich, structured language for requesting those carved-up pieces. Dynamic Resource Allocation reached general availability in Kubernetes v1.35 and has been enabled by default since v1.34. DRA replaces simple integer resource counts with three structured objects:

DRA object | Role
DeviceClass | A cluster-scoped definition of a category of allocatable device (e.g., "A100 MIG 3g.20gb instances") that vendors or admins publish.
ResourceClaim | A namespaced request for a specific device matching a DeviceClass, created by or on behalf of a pod.
ResourceSlice | The inventory: what devices actually exist on which nodes, published by a driver so the scheduler knows what's available to claim.

This matters because it lets a pod ask for attributes — "a device with at least 20GB of dedicated memory and MIG support" — instead of just a bare count. It's the same evolutionary leap that PersistentVolumeClaims made for storage, now applied to accelerators. The hardware side of this story is NVIDIA Multi-Instance GPU (MIG), supported on Ampere-generation-and-newer datacenter GPUs (A100, H100, and successors). MIG partitions a single physical GPU into as many as seven fully isolated GPU instances, each with its own dedicated compute cores, memory controller, and memory address space. Because the isolation happens in hardware, one MIG instance cannot see or touch another instance's memory or steal its compute cycles — a fundamentally different guarantee than software-level sharing schemes. MIG configuration happens at the driver level using nvidia-smi mig subcommands (e.g., nvidia-smi mig -cgi <profile> to create a GPU instance, -cci to create a compute instance within it), and it supports two operating strategies:

Strategy | Behavior | Best for
single | Every GPU in the cluster is partitioned identically, into uniform instance sizes. | Homogeneous workload fleets where every job needs roughly the same slice.
mixed | Each GPU can carry distinct instance profiles (e.g., one 3g.20gb and two 2g.10gb instances on the same card). | Heterogeneous fleets — some teams need more memory, others need less — matching allocation to actual demand distribution.

At KubeCon Europe 2026, NVIDIA donated its Dynamic Resource Allocation Driver for GPUs to the Kubernetes community. This driver publishes MIG instances (and full GPUs) as ResourceSlices and lets pods write attribute-based ResourceClaims against them — for example, requesting "any device with MIG profile 3g.20gb" rather than hardcoding a device-plugin resource name per profile. This is the mechanism that finally connects DRA's structured request model to MIG's structured hardware partitions, and it's why organizations combining the two report up to 7x better GPU utilization: instead of handing a full A100 to a workload that only needs a seventh of it, the scheduler right-sizes the allocation to the request. Now, let's correct three misconceptions that trip up almost everyone new to this material: Misconception 1 — "More time-sliced replicas means more compute time." Time-slicing lets multiple pods share one physical GPU by taking turns, and it's tempting to assume that requesting two replicas of a time-sliced resource gets you double the compute time of a pod that requested one. It does not. Time-slicing shares GPU execution time equally across all processes scheduled onto the device, regardless of how many replicas were requested. Replica count affects how many pods can be scheduled onto the GPU, not how large each pod's slice of time is. Misconception 2 — "Time-slicing isolates memory the way MIG does." It does not. Time-slicing provides no hardware or software memory isolation at all — every process sharing a time-sliced GPU sits in the same global VRAM address space. A pod with a memory leak or an oversized batch can exhaust VRAM and starve every other pod on that GPU. MIG is the only one of the two that gives each instance its own dedicated, protected memory region. Misconception 3 — "DRA partitions a GPU once, at cluster setup, like static MIG." Static MIG partitioning is indeed fixed at setup time via nvidia-smi mig until an administrator reconfigures it. DRA is different: it creates and claims partitions on-demand, only when a pod with matching resource requirements actually arrives and its ResourceClaim is satisfied. This dynamic behavior is powerful, but it introduces real operational tradeoffs — scheduling delay while a claim is being satisfied, and potential fragmentation if oddly-shaped claims are allocated in a suboptimal order across the fleet. Taken together, MIG gives you hardware-guaranteed slices of a GPU, and DRA gives the scheduler a structured, attribute-aware way to request and hand out those slices on demand. That combination is quickly becoming the standard pattern for running mixed inference workloads efficiently on shared GPU fleets.