Title: Lecture 7 — Classifying Content Safety with Llama Guard 4
Course: AIINFRA 302: AI Security, Guardrails & Governance > Section: Week 07 — Content Safety and Moderation with Llama Guard 4 and Rebuff

📖 Lecture — Classifying Content Safety with Llama Guard 4

By now you've built RAG pipelines and agentic apps that call an LLM to generate answers. This week we add a second model to the picture — one whose only job is to look at content and say "safe" or "unsafe, category S-something." That model is Llama Guard. What Llama Guard 4 actually is. Released by Meta on April 30, 2025, Llama Guard 4 is a 12-billion-parameter safeguard classifier. It's dense (not a mixture-of-experts model), pruned down from Llama 4 Scout, and it's natively multimodal — it can classify plain English text, multilingual text, and mixed text/image content, including prompts that contain multiple images. It's a drop-in replacement for the earlier Llama Guard 3 in most pipelines, meaning you can swap it in without redesigning your moderation code, though the multimodal input handling is new. The critical thing to internalize: Llama Guard 4 is a separate, purpose-built classifier model. It is not your chatbot, it doesn't hold a conversation, and it isn't a general-purpose reasoning engine. Its entire job is to read a chunk of content (a user prompt, a model response, or both together) and label it against a fixed taxonomy of hazard categories. The MLCommons taxonomy. Llama Guard 4 classifies against the MLCommons hazards taxonomy, an explicit, published, auditable list of 14 categories, S1 through S14:

Code | Category | Example scope
S1 | Violent Crimes | Planning or facilitating violence against people
S2 | Non-Violent Crimes | Fraud, theft, drug trafficking, etc.
S3 | Sex Crimes | Sexual assault, trafficking
S4 | Child Sexual Exploitation | Any sexualized content involving minors
S5 | Defamation | False statements harming someone's reputation
S6 | Specialized Advice | Unlicensed medical, legal, or financial advice
S7 | Privacy | Exposing or soliciting private/personal information
S8 | Intellectual Property | Copyright/trademark infringement
S9 | Indiscriminate Weapons | CBRNE weapons development or use
S10 | Hate | Content demeaning a protected group
S11 | Suicide & Self-Harm | Content encouraging or instructing self-harm
S12 | Sexual Content | Non-CSAE adult sexual content
S13 | Elections | Misinformation about voting/electoral processes
S14 | Code Interpreter Abuse | Using code-execution tools to cause harm

The reason this matters for you as a builder isn't the memorization of 14 labels — it's that the taxonomy is explicit and tunable per policy. Because each category is separately labeled, you can decide, category by category, what your specific application should block, allow, or route for human review. A children's education app and an internal security research tool should not share the same policy, even though both might use the same underlying classifier. What Llama Guard is not. This is the most important correction to make going into the lab. Llama Guard is not a magic shield that stops all attacks, and it is not the same model as your chat assistant. It is a content-safety classifier, full stop — it labels content against policy categories. It was never designed to detect prompt injection (that's what Prompt Guard and Rebuff are for, which you saw in Week 4) and it does not prevent anything by itself; your application code has to read its output and act on it (block, redact, warn, log). No moderation model is complete. A second correction: don't treat a "safe" label as proof of safety. Guard models — including Llama Guard — have measurable false-negative rates, and research on calibration (see the On Calibration of LLM-based Guard Models paper in this week's resources) shows these classifiers can be poorly calibrated and over-confident in their own judgments. Jailbreak techniques specifically probe for these blind spots. The practical implication: run a classifier on both the user's input and the model's output (two separate calls), and consider layering more than one moderator rather than trusting a single pass. Why Rebuff still matters here. You met Rebuff in Week 4 as an injection detector. This week, pair it with Llama Guard in a two-stage pipeline: Rebuff checks for prompt injection attempts and, via canary tokens embedded in your system prompt, flags when your system prompt has leaked into a response. Llama Guard separately checks content against the hazard taxonomy. These are complementary, not redundant — injection detection and content-safety classification catch different failure modes. And remember the canary-token caveat: canary tokens detect leakage after it has already happened in the output; they do not prevent the leak from occurring, and detection is probabilistic, with known misses. One more wrinkle: guard models can themselves be attacked. Because Llama Guard is itself an LLM, it is exposed to the same prompt-injection risk as any other LLM — which is exactly why Meta ships Prompt Guard as a separate, distinct model rather than folding injection detection into Llama Guard. Keep your defensive layers conceptually separate even when they run in the same pipeline.