Title: Discussion 4 — Rethinking the Throughput/Latency Tradeoff
Course: AIINFRA 200: Production Inference Serving & GPU Orchestration > Section: Week 04 — PagedAttention and Continuous Batching Deep Dive

💬 Discussion 4 — Rethinking the Throughput/Latency Tradeoff (20 pts)

Prompt: How does the interaction between PagedAttention's block-based memory allocation and continuous batching's iteration-level eviction/insertion fundamentally change the throughput/latency tradeoff compared to static batching? In your post, be specific: reference what happens to a finished sequence's memory under each approach, and what happens to a newly-arrived request that wants to join. If you completed Lab 4, bring in at least one number from your own results to support your point. Aim for 150-200 words. Respond to at least two classmates — look for posts that emphasize a different angle (memory management vs. scheduling vs. your own lab data) than yours, and push on whether their explanation would still hold if request lengths were much more uniform across the workload.