🛠️ Lab 4 — Building an Input Scanning Pipeline with LLM Guard and Rebuff (50 pts)

Goal: Stand up a free, CPU-friendly, local input-scanning pipeline using LLM Guard and Rebuff, test it against your own Week 3 Gandalf-winning prompts plus a batch of benign prompts, and build a confusion matrix that shows you exactly how much protection — and how many gaps — these tools actually provide. Steps:

1. Set up your environment: create a fresh Python virtual environment and run pip install llm-guard rebuff. No GPU, no paid API keys, and no cloud services are required for this lab — everything runs locally and for free.
2. Retrieve the prompts you used in Week 3 to beat the Gandalf challenges (save them as a plain-text list, one prompt per line, in a file called injection_prompts.txt).
3. Write a short Python script that loads LLM Guard's input_scanners.PromptInjection scanner (backed by ProtectAI/deberta-v3-base-prompt-injection-v2) plus the BanTopics and TokenLimit scanners, and runs every prompt in injection_prompts.txt through the scan pipeline. Record, for each prompt, whether it was flagged (blocked) or passed.
4. Assemble a second file, benign_prompts.txt, with at least 15 ordinary, harmless prompts you'd expect a well-behaved user to send (e.g., "Summarize this article," "Help me write a cover letter"). Run these through the same LLM Guard pipeline and record the results.
5. Repeat the process using Rebuff's free heuristic and canary-token layers only — explicitly skip Rebuff's paid LLM-based detection layer so the whole lab stays zero-cost. Run both your malicious (Week 3) and benign prompt sets through Rebuff and record the flagged/passed outcome for each.
6. Build a confusion matrix (true positive, false positive, true negative, false negative) for each tool, treating your Week 3 prompts as "known malicious" and your new benign set as "known safe." Calculate precision, recall, and F1 by hand or with a simple script.
7. Optional stretch goal: download the free deepset/prompt-injections test split from Hugging Face and re-run your LLM Guard and/or Rebuff pipeline against it to compute precision/recall/F1 on a larger, independently-labeled dataset.
8. Write a short reflection (250–400 words) answering: Which tool caught more of your Week 3 prompts? Did either tool produce false positives on your benign prompts? After reading Rebuff's own documentation admitting it "cannot provide 100% protection," what would you tell a product manager who wants to ship with only one of these scanners as their sole defense?

Deliverables: Submit your Python script(s), your injection_prompts.txt and benign_prompts.txt files, your confusion matrix and precision/recall/F1 calculations (table or spreadsheet), and your written reflection — either as an uploaded file bundle or pasted directly into the text entry box.