🛠️ Lab 3 — Standing Up a Production-Style vLLM Server (50 pts)

Goal: Launch a real OpenAI-API-compatible vLLM inference server on a free GPU runtime, call it with the standard openai Python client, and expose it to the public internet — experiencing firsthand the client/server split that separates production inference serving from the single-user, single-machine Ollama workflow you used in Week 2. Steps:

1. Open a GPU runtime. Go to Google Colab (colab.research.google.com), create a new notebook, and under Runtime → Change runtime type, select the T4 GPU (free tier). Confirm the GPU is attached by running !nvidia-smi in a cell.
2. Install vLLM. In a new cell, run !pip install vllm. This will take several minutes — use the time to re-read the "vLLM Quickstart" resource linked below.
3. Launch the server. Because vllm serve is a long-running foreground process, start it in the background from a cell using !nohup vllm serve facebook/opt-125m --port 8000 > vllm.log 2>&1 &. We're using facebook/opt-125m because it's small enough to load quickly on a T4 and is ideal for learning the mechanics rather than waiting on a large download.
4. Confirm it's up. Poll the log (!tail -n 30 vllm.log) until you see the server report it is running and accepting connections on port 8000. This may take a few minutes as the model loads and the KV-cache is allocated.
5. Call it with the OpenAI client. In a new cell, !pip install openai, then write a short Python script that creates an openai.OpenAI(base_url="http://localhost:8000/v1", api_key="not-needed") client and sends a chat.completions.create(...) request. Print the response.
6. Compare to Week 2. In a markdown cell, write 3–5 sentences contrasting this workflow with Ollama: what has to be running, what protocol you're speaking, and who could call this server (one machine vs. any client on a network).
7. Tunnel it to the public internet with ngrok. Sign up for a free ngrok account, install the ngrok Python package or CLI in the notebook (!pip install pyngrok), authenticate with your free ngrok authtoken, and run ngrok.connect(8000) to get a public HTTPS URL for your vLLM server.
8. Call your server from "outside." In a separate cell (simulating an external client — pretend this code lives on a different machine), point the openai client's base_url at your public ngrok URL instead of localhost, and successfully get a completion back. This step is the payoff: it proves your inference server is now a real network service, not a notebook-local function call.
9. Capture your evidence. Take a screenshot (or copy the printed output) showing: (a) the server startup log confirming it loaded the model, (b) a successful local request/response, and (c) a successful request/response through the public ngrok URL.

Deliverables: Submit your completed Colab notebook (.ipynb, shared as a downloadable file or exported link) containing all cells and outputs from Steps 1–9, plus a short (150–250 word) written reflection comparing the vLLM client/server architecture to the Week 2 Ollama setup and noting one thing that surprised you about running a "real" inference server.