Title: Quiz 2 — AI Workloads & Infrastructure Requirements
Course: AIINFRA 100: AI Systems & Development Environments > Section: Week 02 — AI Workloads: Training vs. Inference, Latency vs. Throughput

Q: Which best describes the primary goal of a training workload?
- Responding to a single user request as fast as possible
- Maximizing total throughput across a large dataset over an extended period
- Minimizing the number of GPUs used
- Running only on CPUs
A: Maximizing total throughput across a large dataset over an extended period

Q: Inference workloads are simply smaller, cheaper versions of training workloads, optimizing for the same goals.
- True
- False
A: False

Q: A company wants to build a customer-facing chatbot that must reply within a second or two. Which is most important for this workload?
- Low latency
- Maximum total throughput regardless of response time
- Using spot (interruptible) compute
- Running on a fixed nightly schedule
A: Low latency

Q: Define "throughput" in the context of AI infrastructure.
Feedback: Sample/expected answer: Throughput is the number of requests, tokens, or records a system can process per unit of time (e.g., requests per second) — a measure of total processing capacity rather than the speed of any single request.

Q: Achieving both very low latency and very high throughput at the same time is typically free — there is no meaningful tradeoff.
- True
- False
A: False

Q: A nightly job that re-scores an entire customer database for churn risk, with no user waiting on the result, is best described as:
- Real-time/online inference
- Training
- Batch inference
- None of the above
A: Batch inference

Q: Name one infrastructure feature that is typically more important for real-time inference serving than for a training cluster.
Feedback: Sample/expected answer: Any reasonable answer such as: autoscaling to handle bursty traffic, redundancy/failover, or geographic distribution to keep latency low for users in different regions. (Training clusters instead prioritize high-bandwidth interconnects like InfiniBand across large GPU pools.)

Q: What is a common, costly mistake organizations make when matching infrastructure to workload type?
- Using a training-optimized cluster to serve real-time inference requests, or using real-time-serving infrastructure for a large batch job
- Using Google Colab for a class lab exercise
- Measuring latency with time.perf_counter()
- Running batch jobs overnight instead of during the day
A: Using a training-optimized cluster to serve real-time inference requests, or using real-time-serving infrastructure for a large batch job