// AIINFRA 302 · Semester 3

AI Security, Guardrails & Governance

Securing, Governing, and Cost-Optimizing Production LLM Systems

This hands-on course prepares learners to secure, govern, and financially optimize generative AI and LLM systems in production. Students master the OWASP LLM Top 10 (2025), build layered defenses against prompt injection and jailbreaks, and deploy modern guardrail frameworks such as NVIDIA NeMo Guardrails, Meta Llama Guard 4, and Microsoft Presidio for PII protection. The course also covers AI governance frameworks including the NIST AI RMF, ISO/IEC 42001, and the EU AI Act, along with responsible AI, bias, and fairness practices. Learners finish with GPU FinOps skills to measure and reduce inference cost per 1,000 requests using spot instances, fractional GPUs, quantization, and autoscaling economics.

Contact hours54 hrs

Credit equivalent3-unit

PrerequisiteAIINFRA 301

Length16 weeks

01 / outcomes

Outcomes

Course objectives

Assess LLM applications against the OWASP LLM Top 10 (2025) and construct layered defenses against direct and indirect prompt injection, jailbreaks, and data leakage.
Deploy and configure production guardrail frameworks (NeMo Guardrails, Llama Guard 4, LLM Guard, Rebuff, and Presidio) to enforce input/output validation and PII redaction.
Implement AI security controls for data privacy, secrets and access management, model provenance, and supply-chain integrity across the model lifecycle.
Evaluate AI systems against the NIST AI RMF, ISO/IEC 42001, and EU AI Act, producing model cards, risk documentation, and bias and fairness assessments.
Calculate and optimize GPU inference economics using cost per 1,000 requests, spot and fractional GPUs, quantization, KV-cache tuning, and autoscaling to make build-vs-buy decisions.

Student learning outcomes

Assess LLM applications against the OWASP LLM Top 10 and build layered defenses against prompt injection, jailbreaks, and data leakage.
Deploy and configure production guardrail frameworks such as NeMo Guardrails, Llama Guard 4, LLM Guard, Rebuff, and Presidio for input/output validation and PII redaction.
Implement AI security controls for data privacy, secrets and access management, model provenance, and supply-chain integrity.
Evaluate AI systems against the NIST AI RMF, ISO/IEC 42001, and EU AI Act, producing model cards and risk and fairness documentation.
Calculate and optimize GPU inference economics using cost per 1,000 requests, spot/fractional GPUs, quantization, and autoscaling.

02 / schedule

16-week schedule

Wk 01

AI Security Landscape and the LLM Threat Model

Introduces the AI security landscape and establishes a threat model for LLM applications.

Wk 02

OWASP LLM Top 10 (2025) Deep Dive

Provides a deep dive into the OWASP LLM Top 10 (2025) risk categories for LLM applications.

Wk 03

Direct and Indirect Prompt Injection Attacks

Covers direct and indirect prompt injection attack techniques against LLM applications.

Wk 04

Prompt Injection Defenses and Input Validation

Covers layered defenses and input validation strategies against prompt injection attacks.

Wk 05

Jailbreaks, Red-Teaming, and Adversarial Testing

Covers jailbreak techniques and red-teaming/adversarial testing methods for LLM systems.

Wk 06

Guardrails Frameworks: NeMo Guardrails and LLM Guard

Covers deploying and configuring the NeMo Guardrails and LLM Guard guardrail frameworks.

Wk 07

Content Safety and Moderation with Llama Guard 4 and Rebuff

Covers content safety and moderation using Llama Guard 4 and Rebuff.

Wk 08

PII Protection with Presidio and Output Validation

Midterm week: covers PII protection with Microsoft Presidio and output validation, alongside the course midterm exam covering weeks 1-7.

Midterm · covers Wks 1–7

Wk 09

Data Privacy, Secrets Management, and Access Control

Covers data privacy practices, secrets management, and access control for AI systems.

Wk 10

Model Provenance and AI Supply-Chain Security

Covers model provenance tracking and AI supply-chain security controls.

Wk 11

AI Governance Frameworks: NIST AI RMF, ISO/IEC 42001, and the EU AI Act

Covers evaluating AI systems against the NIST AI RMF, ISO/IEC 42001, and the EU AI Act.

Wk 12

Responsible AI, Bias, Fairness, and Model Cards

Covers responsible AI practices, bias and fairness assessment, and producing model cards.

Wk 13

GPU FinOps Foundations: Cost per 1,000 Requests and Right-Sizing

Introduces GPU FinOps foundations, including cost per 1,000 requests and right-sizing inference deployments.

Wk 14

Fractional GPUs, MIG, Quantization, and KV-Cache Savings

Covers cost savings from fractional GPUs, NVIDIA MIG, quantization, and KV-cache tuning.

Wk 15

Autoscaling Economics, Spot Instances, and Build-vs-Buy

Covers autoscaling economics, spot instance usage, and build-vs-buy decisions for inference.

Wk 16

Capstone Project & Course Review

Final capstone week: students deliver a project tying together security, governance, and cost optimization, and review the course.

Capstone

03 / tools

Tools & frameworks

Security & Red-Teaming

OWASP LLM Top 10 (2025)GarakPyRITPromptfoo

Guardrails

NVIDIA NeMo GuardrailsLLM GuardRebuffGuardrails AI

Content Safety

Meta Llama Guard 4Llama Prompt Guard 2OpenAI Moderation APIAzure AI Content Safety

PII & Privacy

Microsoft PresidiospaCyFakerNightfall AI

Structured Output & Validation

PydanticInstructorOutlinesJSON Schema

Governance Frameworks

NIST AI RMF 1.0ISO/IEC 42001EU AI ActMITRE ATLAS

Documentation & Fairness

Hugging Face Model CardsFairlearnAI Fairness 360Croissant metadata

Supply-Chain & Provenance

Sigstore/CosignCycloneDX AI-BOMHugging Face HubModelScan

Serving & GPU Optimization

vLLMNVIDIA MIGTensorRT-LLMGPTQ/AWQ quantization

FinOps & Observability

Kubecost/OpenCostPrometheus + GrafanaNVIDIA DCGMKarpenter autoscaling

← AIINFRA 301: Production RAG & LLMOps — Observability and Evaluation AIINFRA 303: Capstone Project →