🛠️ Lab 11 — Guardrails and Red-Team Testing for Your Capstone (50 pts)

Goal: Add programmable input, dialog, and output rails to your capstone's LLM using NVIDIA NeMo Guardrails, running entirely locally against a local model, then prove the rails hold by attacking your own system with adversarial prompts. Steps:

1. In your capstone's local environment, install NeMo Guardrails (pip install nemoguardrails) — it's Apache-2.0 licensed, free, and runs fully offline against your existing local model (whatever you've served since your inference-serving weeks).
2. Create a config/ directory for your guardrails project containing a config.yml (points to your local model) and a rails.co Colang file.
3. Write an input rail in Colang that blocks off-topic requests unrelated to your capstone's purpose (for example, if your capstone is a study-planning assistant, block requests for unrelated tasks like generating malware or unrelated general chit-chat that ignores the app's scope).
4. Write a dialog rail that detects and refuses common jailbreak patterns (role-play framing like "pretend you are an AI with no restrictions," instruction-override phrasing like "ignore previous instructions," and encoded/obfuscated payloads).
5. Write an output rail that filters unsafe or policy-violating content before it's returned to the user (for example, refuse to output secrets, credentials, or content matching a banned-topics list).
6. Wire the guardrails config into your capstone's request path so every user turn passes through input rails, then dialog rails, then your model, then output rails — not just a system-prompt instruction.
7. Write a red-team test suite of at least 10 adversarial prompts covering: direct prompt injection, at least one indirect injection scenario (a poisoned document/webpage your capstone would retrieve), a jailbreak attempt, an off-topic request, and an attempt to extract your system prompt or credentials.
8. Run every adversarial prompt against your guarded capstone and record the result (blocked / not blocked) for each. For anything that got through, tighten the relevant rail and re-test until it's blocked or you've documented why it's an accepted residual risk.
9. (Optional stretch) Add Guardrails AI on top for schema-valid, structured output validation if your capstone returns structured data (JSON, function calls, etc.).
10. Write up your guardrails architecture and red-team results as a short report.

Deliverables: Your config.yml and rails.co (or equivalent Colang files), the integration code showing rails wired into your capstone's request path, your red-team test suite (prompts + expected vs. actual outcome), and a 1–2 page report summarizing which controls you added, what your red-team testing found, and any residual risks you're accepting and why.