Title: Lecture 1 — Why Scoping Is the Hardest Part of Production AI
Course: AIINFRA 303: Capstone Project > Section: Week 01 — Capstone Kickoff: Scoping a Production AI System

📖 Lecture — Why Scoping Is the Hardest Part of Production AI

If you've built anything with an LLM API or a Jupyter notebook, you already know the seductive part of AI development: you can get something working in an afternoon. A chatbot that answers questions. A classifier that sorts images. A RAG pipeline that summarizes documents. It demos beautifully. Then someone asks, "Great — can we put this in front of real users?" and the real project begins. Here is the uncomfortable truth this course is built around: a production ML/AI system spans project scoping, data needs, modeling strategy, and deployment/serving requirements — and only a small fraction of it is model code. The model is often the smallest, most replaceable piece. The infrastructure around it — how data flows in, how predictions are served reliably, how you know when something breaks, how much it costs to keep running — is what actually determines whether a project survives contact with production. This week, before you write a line of model code for your capstone, you are going to scope the system, not just the model.

MLOps: the discipline of getting to production reliably

The name for this discipline is MLOps — the practice of reliably and repeatedly moving machine learning models into production. It borrows heavily from DevOps but adds the extra complexity of data and models as first-class, versioned artifacts. Four pillars hold it up:

Pillar | What it means in practice | Why your capstone needs it from day one
Reproducibility & traceability | Every model, dataset, and result can be recreated and traced back to the exact code/data/config that produced it | You'll be iterating for weeks — you need to know why last week's version behaved differently
Collaborative, full-lifecycle workflow | Data scientists, engineers, and stakeholders share one workflow from data to deployment, not separate silos | Even solo, you are wearing all these hats — the workflow discipline still applies
Continuous monitoring | Once deployed, the system is watched for data drift, performance decay, and failures | A capstone that's demoed once and never checked again isn't production-grade
Code-defined pipelines (environment parity) | Environments, dependencies, and infrastructure are defined as code so "works on my machine" doesn't happen | This is exactly what uv gives you starting today

Notice that none of these four pillars is "pick a better model." That's the point.

Narrowing to a vertical slice

The single highest-leverage move you can make this week is to shrink your capstone idea until it is almost embarrassingly small. Pick one user, one input, one output, one deployment target. Not "a platform that helps small businesses with marketing." Instead: "a Slack-deployed tool that takes a product description (text input) from one small-business owner and returns three ad headline variants (text output)." That's a vertical slice — thin, but it runs end-to-end. Once you have that slice, write a one-paragraph problem statement and an explicit in/out-of-scope list. The out-of-scope list is arguably more valuable than the in-scope one, because it's the thing that stops scope creep three weeks from now when you're tempted to "just add multi-language support." To sketch how your slice's pieces fit together — the person, the system, the external services it talks to — use a lightweight C4 Model diagram. At the scoping stage you only need the top two levels: System Context (your system and the people/systems around it) and Container (the major deployable pieces — API, database, model server). Mermaid's experimental C4 syntax lets you write these as text and version them alongside your code, which keeps the diagram honest as the project evolves instead of going stale in a slide deck.

Environment and data, made concrete

Two more decisions belong in Week 01, not Week 03. First, set up your Python environment with uv (uv init, uv add, uv run) so that reproducibility is baked in from commit one — this is the code-defined-pipelines pillar in action, on day one, at zero cost. Second, anchor your problem statement to a real, free dataset — from the UCI Machine Learning Repository or the Hugging Face Datasets Hub — rather than a hypothetical one. A real dataset forces concrete answers to questions you cannot scope around: How many rows do you actually have? Are the labels clean or noisy? Is the volume enough to support the modeling strategy you're imagining? Hypothetical data lets you dodge these questions; real data does not.

Correcting three beliefs that sink capstones

"If the AI works in my demo, the hard part is done." Demos are controlled environments: curated inputs, no concurrent users, no cost pressure, no monitoring. Production is the opposite on all four counts. AI MVPs routinely work beautifully in a demo and then fail in production — not because the model got worse, but because of architecture that can't handle real traffic, data quality issues that curated demo inputs never exposed, and costs nobody planned for. The demo tells you the idea is plausible, not that it's done. Overengineering before validating the need. It is tempting to build the multi-region, autoscaling, fully observable version of your capstone in Week 1. Resist it. Building complex integrations and infrastructure before confirming anyone needs the thing is one of the top reasons AI projects blow past budget and timeline. Build the thinnest slice that proves the idea, then add rigor as you validate. Assuming the problem needs ML at all. The most expensive mistake in this list is choosing the wrong problem — building an ML solution before validating that the underlying business goal even requires ML. Sometimes a lookup table or a regular-expression rule does the job better, cheaper, and more explainably. Part of scoping well is being willing to discover your capstone doesn't need a model — and that's a legitimate, valuable finding, not a failure. Scoping isn't the boring prerequisite to the "real" capstone work. It is the real work. Get it right this week and every subsequent week gets easier.