Title: Quiz 5 — Hugging Face TGI and SGLang Serving Engines
Course: AIINFRA 200: Production Inference Serving & GPU Orchestration > Section: Week 05 — Hugging Face TGI and SGLang Serving Engines

Q: As of late 2025, Hugging Face TGI is the recommended choice for all new production LLM deployments.
- True
- False
A: False

Q: What is the current status of Hugging Face TGI as of late 2025?
- Actively expanding with new features
- Deprecated and deleted from GitHub
- In maintenance mode / read-only
- Merged into vLLM
A: In maintenance mode / read-only

Q: What is the name of the core architectural feature that distinguishes SGLang from other serving engines?
Feedback: Sample/expected answer: RadixAttention

Q: RadixAttention improves performance primarily by:
- Compressing model weights to FP8
- Indexing token prefixes in a radix tree to find and reuse shared KV-cache across requests
- Running each request on a dedicated GPU
- Skipping the attention mechanism entirely for short prompts
A: Indexing token prefixes in a radix tree to find and reuse shared KV-cache across requests

Q: SGLang is essentially "vLLM with a different name" because both use paged KV-cache management.
- True
- False
A: False

Q: On which type of workload would RadixAttention's prefix-caching advantage be LEAST beneficial?
- A RAG pipeline where many queries share the same retrieved context
- A multi-turn chatbot resending full conversation history each turn
- A batch of completely unrelated, unique one-off prompts with no shared text
- A customer support bot reusing the same system prompt across sessions
A: A batch of completely unrelated, unique one-off prompts with no shared text

Q: Name one technique that originated in TGI's design that is now considered a standard feature across modern serving engines.
Feedback: Sample/expected answer: Continuous batching (also acceptable: tensor parallelism, or FlashAttention)

Q: Why does bitsandbytes quantization sometimes underperform GPT-Q or AWQ on inference throughput, despite similar memory savings?
- It only works on CPUs
- It quantizes weights automatically at load time but its dequantization at inference time can be slower than calibrated methods like GPT-Q/AWQ
- It does not support any transformer models
- It requires twice the GPU memory
A: It quantizes weights automatically at load time but its dequantization at inference time can be slower than calibrated methods like GPT-Q/AWQ