Title: Lab 2 — Measuring Latency vs. Throughput
Course: AIINFRA 100: AI Systems & Development Environments > Section: Week 02 — AI Workloads: Training vs. Inference, Latency vs. Throughput

🛠️ Lab 2 — Measuring Latency vs. Throughput (50 pts)

Goal: Get hands-on evidence of the latency vs. throughput tradeoff by calling a free public API two different ways — once like a real-time chatbot would, and once like an overnight batch job would — and comparing the results. All tools used in this lab are free. You'll use Google Colab (a free, browser-based Python notebook — no installation required) and a free public API. Steps:

1. Go to Google Colab and sign in with a free Google account. Create a new notebook.
2. In the first cell, import the tools you'll need:

import requests
import time

1. Pick a free public API that needs no API key, such as https://api.agify.io/?name=michael (it guesses an age from a name and responds quickly). Test it once with requests.get(...) and print the result to confirm it works.
2. Simulate real-time/chatbot use (latency test): Write a loop that sends one request at a time, 10 times in a row, using different names. Before each request, record the time with time.perf_counter(), then record it again right after the response comes back. Store each individual response time in a list.
3. Print out all 10 individual response times and calculate the average. This average is your latency — how long a user would wait for one answer, similar to waiting for a chatbot reply.
4. Simulate a nightly batch job (throughput test): Write a loop that sends 30–50 requests back-to-back as fast as possible (still one at a time is fine, since this is a beginner lab — the point is volume, not true parallelism). Record the total start time before the loop and the total end time after it finishes.
5. Calculate throughput: total number of requests divided by total elapsed time, giving you requests-per-second.
6. Make a simple table (you can just print it, or use a markdown cell) comparing: average single-request latency (step 5) vs. requests-per-second throughput (step 7). Optionally, plot the 10 individual latency values from step 4 using a basic matplotlib bar chart (matplotlib is pre-installed in Colab).
7. In a markdown cell at the bottom of your notebook, write 2–3 sentences answering: Which scenario (step 4 or step 6) resembles a real-time chatbot, and which resembles a nightly data-processing job? Why would a company make different infrastructure choices for each?
8. Download your notebook (File \> Download \> Download .ipynb) or export it as a PDF, and prepare it for submission.

Deliverables: Submit your Colab notebook (.ipynb or PDF export) showing your code, your latency and throughput results, your comparison table, and your 2–3 sentence reflection, via the online upload/text entry submission box for this assignment.