🎓 Final Capstone Project (100 pts)
The task: Design — and partially build — a real AI system that a business or institution could plausibly deploy, and prove that you made deliberate, defensible security and governance decisions at every layer. You are not building a toy chatbot. You are building the security and governance layer around an LLM-powered system, and showing your reasoning at each decision point covered across Weeks 1–15.
Choose a scenario
Pick one (or propose your own and get instructor sign-off):
- Scenario A — Internal HR assistant answering employee policy questions over a RAG knowledge base containing sensitive HR documents.
- Scenario B — Customer support agent with tool-calling access to order/account data for an e-commerce business.
- Scenario C — Healthcare intake assistant that triages patient questions using a retrieval-augmented knowledge base (HIPAA-adjacent, non-production).
- Scenario D — Internal code/dev assistant with access to a private code repository and internal documentation.
Requirements
Your submission must include all three components: 1. Written Design Document (3–6 pages) Map each decision explicitly back to a skill from this course. At minimum, address:
- System overview — what the system does, who uses it, and what data it touches.
- Threat model (Weeks 1–3) — identify at least 4 relevant risks from the OWASP LLM Top 10 for your scenario, and your specific mitigation for each (including prompt-injection defenses, direct and indirect).
- Guardrail architecture (Weeks 4–6) — describe your input/output filtering and content-safety layer, including what triggers a block vs. a flag-for-review.
- Data privacy & secure RAG design (Weeks 7–9) — how sensitive data is identified, redacted, access-controlled, or scoped within your retrieval pipeline.
- Governance & evaluation plan (Weeks 10–12) — your eval harness approach, a short model/system card for the assistant, and your red-team test plan (at least 5 adversarial test cases with expected safe behavior).
- FinOps & scaling plan (Weeks 13–15) — estimated cost per 1,000 requests, your model/instance choice and why, and what you'd monitor in production (cost, latency, drift) with alert thresholds.
- Known limitations — what you did not solve, and what you'd do with more time/budget.
2. Architecture Diagram One diagram (hand-drawn and photographed, or built in a free tool such as draw.io, Excalidraw, or Mermaid) showing: user → application layer → guardrail layer(s) → retrieval/data layer → LLM → guardrail layer(s) → user, plus where logging/monitoring hooks into the flow. Label each guardrail and control point. 3. Working Prototype Artifact A partial, runnable implementation — it does not need to be production-complete, but it must demonstrably run and show at least:
- One working guardrail check (input or output) actually executing in code, not just described
- One working retrieval call against a small sample dataset you create (can be a handful of documents)
- One logged/printed trace of a red-team test case from your design document being caught or handled by your system
Use the free/low-cost toolchain from this course: your existing LLM API key (free-tier or trial credits are fine), an open-source guardrail library or simple rule-based filter you code yourself, your RAG stack from AIINFRA 301, and a free-tier notebook (Google Colab) or local environment.
Deliverables
- [ ] Design document (PDF or shared doc link), 3–6 pages, addressing all 7 sections above
- [ ] One architecture diagram (image or PDF), clearly labeled
- [ ] Prototype code (notebook, repo zip, or shared repo link) that runs and produces visible output for the three required behaviors above
- [ ] A short README (half a page is fine) explaining how to run the prototype and which design-document section each code section maps to
Grading
Graded using the Capstone / Project Rubric (100 pts total). This assignment maps to all five course learning outcomes (CLO1–CLO5) — it is the single assignment that demonstrates you can do the full job, not just one piece of it.