🛠️ Lab 6 — Build and Compare: NeMo Guardrails vs. LLM Guard (50 pts)

Goal: Implement the same three guardrail policies — (1) stay on-topic, (2) block a banned subject, (3) self-check output — first in NeMo Guardrails, then in LLM Guard, using only free, local, CPU-runnable tools. Compare the two frameworks on developer ergonomics, latency, and coverage. Steps:

  1. Set up your local model backend. Install Ollama and pull a small free local model (e.g., ollama pull llama3.2 or another compact model your machine can run on CPU). Confirm you can get a response from it locally with no API key.
  2. Install NeMo Guardrails. pip install nemo-guardrails (Python 3.10–3.13). Point its config at your local Ollama model instead of a hosted provider.
  3. Author a topical rail (Colang). Write a Colang flow that keeps the bot restricted to one topic of your choice (e.g., "only answer questions about houseplant care"). Test with a straightforward off-topic question and confirm the bot redirects or refuses.
  4. Author a banned-topic input rail. Add a second rail that explicitly blocks one named subject (e.g., no medical dosage advice). Test with a direct question on that subject.
  5. Author an output self-check rail. Add an output rail that re-checks the bot's own draft response against a policy (e.g., "the response must not contain the banned topic") before it's returned to the user.
  6. Adversarial testing (NeMo). Attempt to bypass your three rails with at least 3 adversarial prompts, including at least one multi-turn attempt (spread the banned request across two or more messages) and one indirect phrasing attempt. Record what got through and what didn't.
  7. Install LLM Guard. pip install llm-guard. Using the same local Ollama model, implement the equivalent three policies: an input scanner to block prompt injection related to your banned topic (or a BanTopics scanner), a TokenLimit scanner to cap tokens (standing in for "keep it on-topic" by bounding response scope), and an output scanner (e.g., BanTopics or NoRefusal/toxicity on output) to self-check the response.
  8. Adversarial testing (LLM Guard). Run the same 3+ adversarial prompts from Step 6 against your LLM Guard pipeline and record results.
  9. Write your comparison. In 400–600 words, compare the two frameworks on: (a) developer ergonomics (how much code/config did each require?), (b) latency (rough wall-clock time per request, each framework vs. no guardrails), and (c) coverage (which adversarial prompts got through which framework, and why).

Deliverables: Your NeMo Colang config files and Python driver script, your LLM Guard Python script, a log/transcript of all adversarial test attempts and results for both frameworks, and your 400–600 word written comparison — submitted as a single upload (zip or combined document) plus the comparison pasted into the text-entry box.