Title: Discussion 12 — Choosing TP vs. PP for Your Deployment
Course: AIINFRA 200: Production Inference Serving & GPU Orchestration > Section: Week 12 — Multi-GPU Tensor and Pipeline Parallelism

💬 Discussion 12 — Choosing TP vs. PP for Your Deployment (20 pts)

Given the communication and synchronization differences between tensor parallelism and pipeline parallelism, when would you choose each strategy — and how does that choice interact with model size, batch size, and hardware interconnect bandwidth? Ground your answer in a concrete (real or hypothetical) scenario: describe the model size, GPU count and interconnect (NVLink vs. PCIe), and expected request pattern (low-latency single requests vs. high-throughput batch traffic), then justify whether you'd pick TP, PP, or a combination of both. Post your initial response, then respond to at least two classmates — ideally engaging with scenarios different from your own (for example, if you picked a latency-sensitive NVLink scenario, respond to someone who picked a throughput-focused multi-node scenario).