Title: Quiz 3 — Direct and Indirect Prompt Injection Attacks
Course: AIINFRA 302: AI Security, Guardrails & Governance > Section: Week 03 — Direct and Indirect Prompt Injection Attacks

Q: Prompt injection only happens when a malicious user types an attack directly into the chat box.
- True
- False
A: False

Q: Who coined the term "prompt injection," and by analogy to what earlier security concept?
- Greshake et al.; buffer overflow B) Simon Willison; SQL injection C) Microsoft Research; cross-site scripting D) OWASP; phishing
A: Greshake et al.; buffer overflow B) Simon Willison; SQL injection C) Microsoft Research; cross-site scripting D) OWASP; phishing

Q: Name the three legs of Simon Willison's "lethal trifecta."
Feedback: Sample/expected answer: (1) access to private data, (2) exposure to untrusted content, (3) ability to communicate externally

Q: According to the lethal trifecta framework, what happens if you remove just one of the three legs from an agent's design?
- Nothing changes; all three must be removed B) The exfiltration attack breaks C) The model becomes slower D) The agent can no longer use tools at all
A: Nothing changes; all three must be removed B) The exfiltration attack breaks C) The model becomes slower D) The agent can no longer use tools at all

Q: Content retrieved by a RAG pipeline or returned by a tool call should be treated as trusted, since it came from a system the developer controls.
- True
- False
A: False

Q: What is Crescendo, as described in the 2025 USENIX Security paper?
- A single-turn jailbreak using encoded text B) A multi-turn jailbreak that starts benign and progressively escalates toward a prohibited objective C) A defense technique that blocks all multi-turn conversations D) A dataset of labeled injection strings
A: A single-turn jailbreak using encoded text B) A multi-turn jailbreak that starts benign and progressively escalates toward a prohibited objective C) A defense technique that blocks all multi-turn conversations D) A dataset of labeled injection strings

Q: In one sentence, explain why the Greshake et al. (2023) Bing Chat / plugin-assistant demonstration matters for RAG system design today.
Feedback: Sample/expected answer: Sample answer: It showed that indirect injection via retrieved web content can hijack an LLM assistant's behavior, proving that any text a model reads — not just user-typed input — is a potential instruction and must be treated as untrusted.

Q: A 2025 multi-model study found that the large majority (94.1%) of the 17 state-of-the-art models tested were vulnerable to at least one injection vector, showing this risk is not theoretical.
- True
- False
A: True