Title: Quiz 9 — The Model Layer: Serving, Adaptation, or RAG Integration
Course: AIINFRA 303: Capstone Project > Section: Week 09 — The Model Layer: Serving, Adaptation, or RAG Integration

Q: What is the primary innovation that lets vLLM serve open models with high throughput and low memory fragmentation?
- Larger batch sizes alone
- PagedAttention with a block-based KV cache
- Running the model in 8-bit precision only
- Caching entire responses in Redis
A: PagedAttention with a block-based KV cache

Q: vllm serve <model> exposes an OpenAI-compatible API endpoint.
- True
- False
A: True

Q: What does LoRA freeze during training?
- The learning rate schedule
- The tokenizer vocabulary
- The base model's original weights
- The training dataset
A: The base model's original weights

Q: Roughly how much does LoRA reduce the number of trainable parameters compared to full fine-tuning?
Feedback: Sample/expected answer: Roughly two orders of magnitude (about 100x fewer trainable parameters)

Q: In a RAG pipeline, what happens immediately before chunks are retrieved to answer a query?
- The base model is re-trained on the new documents
- The incoming query is embedded so it can be similarity-matched against stored vectors
- The vector database is deleted and rebuilt
- The documents are re-chunked from scratch
A: The incoming query is embedded so it can be similarity-matched against stored vectors

Q: One of RAG's key benefits is that the knowledge base can be updated without retraining the model.
- True
- False
A: True

Q: A team's RAG pipeline is returning wrong answers. According to this week's lecture, where should they look first?
- Immediately swap in a larger, more expensive LLM
- Rewrite the prompt template
- Inspect the ingestion, chunking, indexing, and retrieval steps for what was actually retrieved
- Add more few-shot examples to the prompt
A: Inspect the ingestion, chunking, indexing, and retrieval steps for what was actually retrieved

Q: Why is chunking purely by fixed byte/character size often a worse strategy than chunking on semantic boundaries with slight overlap?
Feedback: Sample/expected answer: Fixed-size chunks can cut across meaning, stripping needed context or fragmenting related ideas, while tiny chunks in particular can inflate cost without improving retrieval quality; semantic-boundary chunking (e.g., by paragraph or section) with slight overlap preserves context better.