Title: Lecture 4 — Frameworks, Drivers, Containers, and Model Formats
Course: AIINFRA 100: AI Systems & Development Environments > Section: Week 04 — The AI Software Stack: Frameworks, CUDA, Drivers, Model Formats

📖 Lecture — Frameworks, Drivers, Containers, and Model Formats

So far in this course you have looked at AI mostly from the outside: what models can do, what hardware they need, and how data flows through a system. This week we open the toolbox that AI engineers actually reach for every single day. If you have ever tried to install AI software and hit a wall of cryptic errors, this lecture explains why that wall exists — and how professionals get around it.

PyTorch and TensorFlow: the two big frameworks

A deep learning "framework" is a software library that gives you the building blocks for creating and training neural networks — the math, the GPU acceleration, the tools to save and load models — so you don't have to write all of that from scratch. Two frameworks dominate the field: PyTorch and TensorFlow. PyTorch is written in a very "Pythonic" style and uses eager execution, meaning code runs line-by-line the way you'd expect from ordinary Python, which makes it easier to read, write, and debug. It is now the dominant framework in AI research and it powers the Hugging Face ecosystem, which is the largest hub for pretrained models. TensorFlow (often used through its friendlier Keras interface) is still common in some production and enterprise deployments, especially in organizations that adopted it years ago. For these reasons, this course — and most beginner-focused resources in 2025–2026 — recommends PyTorch as your first framework. You will get more tutorials, more community support, and a more direct path into the current AI ecosystem.

Framework | Style | Where it shines today
PyTorch | Eager execution, Pythonic | Research, Hugging Face ecosystem, beginner-friendly
TensorFlow / Keras | Graph-based (with eager mode available) | Some enterprise & production pipelines

CUDA, drivers, and the version-matching puzzle

A GPU is only fast for AI because of software that lets frameworks talk to it. CUDA is NVIDIA's parallel computing platform and API — it's what allows PyTorch to hand off matrix math to an NVIDIA GPU instead of the CPU. But there are actually two CUDA-related things on your machine, and they are not the same:

1. The GPU driver — installed at the operating-system level, it lets your computer talk to the GPU hardware at all.
2. The CUDA Toolkit — a separate set of libraries and compilers that frameworks like PyTorch are built against.

Here's the part that trips up almost every beginner: running nvidia-smi in a terminal shows you the maximum CUDA version your driver can support — not necessarily the CUDA Toolkit version actually installed on your system, and not necessarily the version PyTorch was built for. If these don't line up correctly, you can get confusing install errors, or worse, a silent fallback to CPU-only mode with no clear error at all. The fix isn't to memorize version numbers — it's to always use the official installer selector (see Resources) and let it tell you the right combination for your OS and hardware.

Containers: solving "it works on my machine"

A container (most commonly built with Docker) packages your code together with its exact dependencies — Python version, framework version, system libraries — into one portable unit. That unit runs identically on your laptop, a classmate's machine, or a cloud server, because it's not relying on whatever happens to already be installed on the host. This matters enormously in AI work because ML dependency chains are unusually deep and fragile: a specific Python version, a specific framework version, and (as you just learned) a specific CUDA version all have to agree with each other. Docker sidesteps this by shipping the whole environment as one artifact instead of a fragile list of setup instructions. One important limit to know: a plain Docker container does not get GPU access automatically. Using a GPU from inside a container requires extra setup (such as the NVIDIA Container Toolkit) on top of a standard Docker install. Many beginners assume GPU access "just works" in a container the way it does on the host machine — it doesn't, unless you've configured it.

Environment management: keeping projects from fighting each other

Before you even get to containers, you need a way to keep one project's Python packages from colliding with another's. Tools like venv (built into Python), conda, and newer, faster tools like uv create an isolated "environment" per project, each with its own package versions. Conda is especially popular in ML specifically because — unlike plain venv/pip — it can also manage non-Python dependencies, including certain CUDA libraries, all from the same tool.

Model file formats: not just a wrapper detail

When you download a pretrained model, it comes as a file (or set of files) in a particular format, and the format is not a cosmetic choice — it changes what you can safely and efficiently do with the model.

Format | Best for | Key thing to know
safetensors | Sharing/loading Hugging Face model weights | Safe by design — cannot execute arbitrary code when loaded, unlike older pickle-based .pt/.bin files; fast and memory-mappable
GGUF | Efficient local/CPU inference via llama.cpp | Self-contained and quantization-friendly, but not meant for fine-tuning
ONNX | Moving a model between frameworks or onto specialized hardware | Cross-framework, vendor-neutral; stores the computation graph, not just the weights

The key beginner misconception to unlearn: these formats are not interchangeable stand-ins for each other. Older pickle-based formats (.pt, .bin) can run arbitrary code the moment you load them, which is a real security risk if you download weights from an untrusted source — safetensors was created specifically to close that hole. GGUF trades away fine-tuning flexibility for efficient local inference. ONNX exists specifically to let a model trained in PyTorch run somewhere that doesn't natively support PyTorch at all. Choosing a format means choosing a purpose.

Bringing it together

A realistic AI software stack looks like: a Python environment (venv/conda) that isolates your dependencies → a framework (PyTorch) installed with the correct CUDA build for your GPU and driver → a model file in the right format for your use case (safetensors for a Hugging Face model you'll keep tinkering with; GGUF if you just want fast local inference; ONNX if you need to hand it to a different framework) → optionally, all of it wrapped in a Docker container so it can move anywhere without breaking. Every one of these layers exists to solve a real, specific pain point that engineers hit constantly — none of it is complexity for its own sake.