Title: Quiz 3 — Computational Resources for AI
Course: AIINFRA 100: AI Systems & Development Environments > Section: Week 03 — Computational Resources: CPUs, GPUs, TPUs, and VRAM

Q: Which statement best describes why GPUs are faster than CPUs for deep learning?
- GPUs have higher clock speeds than CPUs
- GPUs have thousands of simpler cores optimized for parallel matrix/tensor math
- GPUs can run more different types of software at once
- GPUs use less electricity than CPUs
A: GPUs have thousands of simpler cores optimized for parallel matrix/tensor math

Q: TPUs are general-purpose chips that can be used for any kind of software, just like GPUs.
- True
- False
A: False

Q: What is VRAM, and why is it often described as "the real bottleneck" in AI work rather than raw compute speed?
Feedback: Sample/expected answer: VRAM is the GPU's own onboard memory. A model must fit entirely in VRAM to run on that GPU — if there isn't enough VRAM, the model can't be loaded or trained at all, regardless of how fast the chip's compute is.

Q: Roughly how much VRAM does inference typically require per billion model parameters at fp16 precision?
- About 0.5 GB per billion parameters
- About 2 GB per billion parameters
- About 12 GB per billion parameters
- About 50 GB per billion parameters
A: About 2 GB per billion parameters

Q: Why does training a model require roughly 2–4x more memory than simply running (inference on) that same model?
Feedback: Sample/expected answer: Training must store not just the model's weights but also the gradients and the optimizer state (e.g., Adam's per-parameter bookkeeping), which together add up to roughly 12 bytes per parameter, versus about 2 bytes per parameter for fp16 inference.

Q: What is NVLink?
- A cloud service for renting GPUs by the hour
- NVIDIA's proprietary high-speed GPU-to-GPU interconnect, much faster than PCIe
- A type of VRAM found only in TPUs
- The software library used to train neural networks
A: NVIDIA's proprietary high-speed GPU-to-GPU interconnect, much faster than PCIe

Q: Doubling the number of GPUs used in training will typically double the training speed.
- True
- False
A: False

Q: A student wants to practice classical machine learning with scikit-learn on a small dataset. What hardware do they most likely need?
- A TPU rented from Google Cloud
- Multiple GPUs connected via NVLink
- A CPU is sufficient
- A GPU with at least 80 GB of VRAM
A: A CPU is sufficient