🛠️ Lab 6 — Quantize and Compile TinyLlama with TensorRT-LLM (50 pts)

Goal: Use NVIDIA's published, Colab-compatible TensorRT-LLM workflow to quantize a small model with INT4 AWQ, compile it into an optimized engine with trtllm-build, and produce a before/after benchmark report that demonstrates the "compile once, serve fast" theme of the week. All tools used are free and run on a free-tier GPU (e.g., Google Colab's free T4 runtime). Steps:

1. Set up your environment. In a fresh Colab notebook (or any free-tier GPU environment you have access to), confirm a GPU runtime is attached, then install TensorRT-LLM via pip following the install instructions in the TensorRT-LLM Quick Start Guide.
2. Download the baseline model. Pull TinyLlama/TinyLlama-1.1B-Chat-v1.0 from Hugging Face using the huggingface_hub or transformers library.
3. Benchmark the "before" state. Using the unmodified Hugging Face checkpoint (running with standard transformers generation, FP16), measure and record: (a) time-to-first-token (TTFT) and (b) tokens/sec throughput, on a fixed prompt of your choice, averaged over at least 3 runs.
4. Quantize the checkpoint. Run the examples/quantization/quantize.py script from the TensorRT-LLM repository against TinyLlama with --qformat int4_awq, producing a converted, quantized checkpoint directory.
5. Compile the engine. Run trtllm-build --checkpoint_dir <your_quantized_checkpoint_dir> to compile the quantized checkpoint into an optimized TensorRT engine. If the build fails, do not simply re-run the same command — first delete the build tree (rm -r build cpp/build, or the Colab-equivalent path) and rebuild.
6. Benchmark the "after" state. Load the compiled engine and repeat the same TTFT and tokens/sec measurement from Step 3, using the same prompt and the same number of runs.
7. Write your before/after optimization report. Include: a table comparing before (FP16, uncompiled) vs. after (INT4 AWQ, compiled engine) TTFT and tokens/sec; the wall-clock time your quantize + build steps took; and 3–5 sentences reflecting on what you'd need to change if you had to redeploy this engine on a different GPU model.

Deliverables: Submit your notebook (.ipynb or exported .pdf) showing all commands and output, plus your before/after optimization report (table + reflection), via online upload or text entry.