Title: Lecture 2 — Deciding on Purpose: ADRs and the Metrics That Prove You're Right
Course: AIINFRA 303: Capstone Project > Section: Week 02 — Requirements, Success Metrics, and Architecture Decision Records

📖 Lecture — Deciding on Purpose: ADRs and the Metrics That Prove You're Right

By now your capstone has a toolchain: containers, a cloud target, CI/CD, an inference-serving stack, maybe a fine-tuned model, agents, RAG, security controls, and a FinOps budget you're watching. What it doesn't have yet — and what separates a class project from a production-grade one — is a paper trail of judgment. Six weeks from now, when you or a teammate asks "why did we pick Postgres over a vector-only store?" or "why does the API return in under 2 seconds and not 200ms?", you need an answer that isn't "I think we talked about it." This week is about writing that answer down before you need it, and about defining numbers that tell you honestly whether your system works.

Architecture Decision Records

An Architecture Decision Record (ADR) is a short document capturing one architecturally significant decision, the context that led to it, and its consequences. It is not a design document, a spec, or a diary entry — it's a single decision, isolated, dated, and versioned. The standard sections, popularized by Michael Nygard and refined by the MADR (Markdown Architectural Decision Records) template, are:

Section | Purpose | Typical length
Title | Short noun phrase naming the decision | 1 line
Status | Proposed, Accepted, Deprecated, or Superseded | 1 word
Context | The forces at play: constraints, requirements, options considered | 1–2 paragraphs
Decision | What you chose and why, in plain language | 1 paragraph
Consequences | What becomes easier, what becomes harder, what risk you accepted | 1 paragraph, bulleted is fine

ADRs live in your repository — commonly in a doc/adr/ directory — as numbered markdown files (0001-use-postgres-for-metadata.md, 0002-serve-model-with-a-rest-api.md). Keeping them in the repo instead of a wiki or slide deck matters for three reasons: they're versioned alongside the code they describe, they're diffable so you can see exactly how a decision's wording evolved through pull-request review, and they're discoverable by anyone who clones the repo, with no separate login or tool required. This week, assign yourself 3–5 ADRs covering the early, foundational decisions of your capstone — things like your model-serving approach, cloud vs. local deployment, or database choice. These are the decisions that are expensive to reverse later, which is exactly why they deserve a record now. One rule makes ADRs durable over time: superseded ADRs are never edited, only linked to their replacement. If you decide in Week 8 that your Week 2 database choice was wrong, you don't rewrite ADR-0003 — you write a new ADR-0012 that says "Supersedes ADR-0003," and you go back and mark ADR-0003's status as "Superseded by ADR-0012." The old record stays exactly as it was written, preserving the historical truth of what you believed and why, at the time you believed it. That history is often more valuable than the current decision, because it tells the next person (including future-you) what was already tried and rejected.

Success metrics: SLI, SLO, SLA, and error budget

Google's Site Reliability Engineering (SRE) practice gives us a precise vocabulary for talking about "how good is good enough," and it's worth internalizing the hierarchy exactly:

Term | Definition | Example for a capstone inference API
SLI (Service Level Indicator) | The thing you actually measure | p95 latency of /predict requests
SLO (Service Level Objective) | The internal target you set for that SLI | p95 latency \< 800ms over a rolling 28 days
SLA (Service Level Agreement) | A contractual promise to an external party, usually with a penalty if missed | "99.5% uptime or service credit" (most capstones won't need a real SLA, but you should understand the distinction)
Error budget | The allowed room for failure, 1 − SLO | If SLO = 99.5% availability, your error budget is 0.5% of requests/time

The error budget reframes reliability as something you spend, not something you maximize infinitely. If you have budget left, you can ship faster and take more risks; if you've burned through it, you slow down and stabilize. For this week's lab and discussion, pick at least one SLI that matters for your specific capstone (latency, availability, retrieval relevance, whatever is core to your product) and set a concrete SLO — a number, with a time window, that you could actually check a dashboard against.

Correcting three common mistakes

"Accuracy is a fine metric to report." Not on imbalanced data. If 95% of your data belongs to one class, a model that blindly predicts that class every time scores 95% accuracy while catching zero instances of the minority class — the class you likely care about most (fraud, defect, disease-positive). Choose metrics that match your actual goal: precision when false positives are costly, recall when false negatives are costly, F1 when you need a balance, ROC-AUC when you need a threshold-independent view. Report the confusion matrix, not just one number. "I'll write the ADR after we ship, once things are stable." Writing ADRs after the fact strips them of their value. A retroactive ADR reconstructs context from memory and quietly forgets the alternatives you actually considered — it becomes a justification, not a record. Write the ADR while the decision is being made or debated, even if it's rough. The value is in capturing live uncertainty, not polished hindsight. "I just need to record what we chose." An ADR that lists only the chosen option, with no rejected alternatives or trade-offs, invites the same argument to happen again in three months, because nobody documented why the alternatives lost. A good ADR always names at least one option that was seriously considered and rejected, and says why — that's what stops the re-litigation. Finally, for lightweight requirements tracking alongside your ADRs, GitHub Projects lets you track requirements as Issues in table, board, or roadmap views, with custom fields for success metrics, status, and target dates — a free, low-ceremony way to keep "what we're building" and "why we decided to build it that way" linked together.