Title: Quiz 7 — NVIDIA Triton Inference Server and Model Repositories
Course: AIINFRA 200: Production Inference Serving & GPU Orchestration > Section: Week 07 — NVIDIA Triton Inference Server and Model Repositories

Q: Which of the following best describes NVIDIA Triton Inference Server?
- A single-framework serving engine optimized only for PyTorch
- An open-source inference server that can serve models from multiple framework backends simultaneously via HTTP/gRPC
- A container orchestration tool that replaces Kubernetes
- A GPU driver management utility
A: An open-source inference server that can serve models from multiple framework backends simultaneously via HTTP/gRPC

Q: A Triton model repository can auto-generate config.pbtxt for every supported backend, so hand-editing it is never necessary.
- True
- False
A: False

Q: What specific directory-structure element do beginners most often forget when setting up a model in a Triton model repository, causing Triton to refuse to load an otherwise-correct model?
Feedback: Sample/expected answer: A numeric version sub-directory (e.g., a folder named "1") inside the model's directory, holding the actual model file(s).

Q: In a Triton config.pbtxt, which of the following is NOT something the file is used to declare?
- Model name
- Backend type
- Max batch size
- The Kubernetes namespace the pod runs in
A: The Kubernetes namespace the pod runs in

Q: When Triton's vLLM backend receives a request, it hands the request to vLLM's AsyncEngine, which manages inflight batching and PagedAttention.
- True
- False
A: True

Q: Which Triton mechanism is best suited for a pipeline that needs conditional branching or loops between model calls, rather than a simple linear tensor-chain?
- Ensemble models
- Business Logic Scripting (BLS)
- Dynamic batching
- Perf Analyzer
A: Business Logic Scripting (BLS)

Q: A student runs Perf Analyzer against their Triton model and gets an error. What should they check first, before assuming the model itself is broken?
Feedback: Sample/expected answer: That the config.pbtxt input/output shapes match what the model actually expects.

Q: How do Triton ensemble models pass data between chained models (e.g., preprocessing -\> model -\> postprocessing)?
- The client must manually call each model in sequence and pass outputs to the next request
- Each model's output tensors are mapped to the next model's input tensors entirely within Triton configuration, with no client-side coordination
- Ensembles require a separate message queue like Kafka between each step
- Ensembles only work if all models share the exact same backend
A: Each model's output tensors are mapped to the next model's input tensors entirely within Triton configuration, with no client-side coordination