🛠️ Lab 5 — Scanning and Manually Red-Teaming a Local Model (50 pts)

Goal: Use the free, open-source garak scanner to automatically probe a local model for jailbreak, injection, toxicity, and data-leakage vulnerabilities, then bridge into manual red-teaming by testing hand-picked jailbreak prompts from a public dataset against the same model — all fully free and local, no paid API keys required. Steps:

  1. Set up a local target model. Install Ollama (or use a small Hugging Face model you already have from AIINFRA 300/301) and pull a small open model (e.g., llama3.2 or similar) so you have a local inference endpoint to attack.
  2. Install garak. Run python -m pip install -U garak (v0.14.0 or later) in a virtual environment.
  3. Run garak's baseline probes against your local model. Point garak at your Ollama/Hugging Face endpoint and run at least three probe families: one jailbreak probe, one prompt injection probe, and one toxicity or data-leakage probe. Save the generated report.
  4. Read the report. Identify which probe families your model failed (i.e., where a harmful/unintended completion got through) versus which it resisted. Note the specific prompts that succeeded.
  5. Sample manual jailbreak prompts. Pull a sample of 10–15 prompts from the free Necent/llm-jailbreak-prompt-injection-dataset on Hugging Face. Read through them and cluster your sample into at least three strategy types (e.g., DAN-style persona, hypothetical/fictional framing, token smuggling, refusal suppression).
  6. Manually test your sample. Run your clustered prompts by hand against the same local model you scanned with garak. Record which prompts succeeded (model complied with something it should have refused) versus which were blocked.
  7. Compare and reflect. Write a short comparison: did garak's automated probes catch the same weaknesses your manual cluster testing found, or did one method surface issues the other missed? What does that tell you about relying on only one testing method?

Deliverables: Submit your garak report (or a summary export of it), your labeled prompt clusters with pass/fail results, and a 300–500 word comparison write-up, either as an upload or pasted into the text entry box.