Title: Quiz 6 — TensorRT-LLM Compilation and Optimization
Course: AIINFRA 200: Production Inference Serving & GPU Orchestration > Section: Week 06 — TensorRT-LLM Compilation and Optimization

Q: TensorRT-LLM loads a Hugging Face checkpoint and serves it directly, the same way vLLM does.
- True
- False
A: False

Q: Which command performs the second step of the TensorRT-LLM compile pipeline, building the optimized engine from a converted checkpoint?
- `python quantize.py --dtype float16`
- `trtllm-build --checkpoint_dir ./tllm_checkpoint_1gpu_fp8`
- `pip install tensorrt-llm`
- `python serve.py --engine ./engine`
A: `trtllm-build --checkpoint_dir ./tllm_checkpoint_1gpu_fp8`

Q: Name one quantization format TensorRT-LLM supports that NVIDIA specifically recommends for Ada-generation GPUs.
Feedback: Sample/expected answer: INT8 SmoothQuant

Q: What is kernel fusion, as used in TensorRT-LLM engine compilation?
- Running multiple models on one GPU simultaneously
- Merging multiple ops (e.g., LayerNorm, matmul, bias-add, activation) into a single optimized CUDA kernel
- Combining multiple GPUs into a single virtual device
- Splitting one large kernel into many small ones for parallelism
A: Merging multiple ops (e.g., LayerNorm, matmul, bias-add, activation) into a single optimized CUDA kernel

Q: A TensorRT-LLM engine compiled for an A100 GPU will run without modification on an H100 GPU.
- True
- False
A: False

Q: A student's trtllm-build command fails, and they re-run the exact same command, which fails again the same way. What should they do differently before rebuilding?
Feedback: Sample/expected answer: Delete the build tree first (e.g., ****`rm -r build cpp/build`****) since stale intermediate artifacts are often the real cause of the failure, then rebuild.

Q: Roughly how long can a multi-GPU tensor-parallel TensorRT-LLM engine build take per (model, GPU, dtype) combination?
- A few seconds
- 1–2 minutes
- 10–90 minutes
- It is always instant regardless of configuration
A: 10–90 minutes

Q: Approximately how much can FP8 precision improve throughput compared to FP16/BF16 on supported hardware, according to NVIDIA?
- No meaningful difference
- Roughly 10% faster
- Roughly double the throughput
- Roughly ten times the throughput
A: Roughly double the throughput