By now you've built LLM apps and RAG pipelines (AIINFRA 300, 301) that call external APIs, connect to vector stores, and pull from internal document sets. Every one of those connections needs a credential, and every one of those documents has an owner who may not want it visible to everyone. This week is about closing the gap between "it works in my demo" and "it's safe to run against real production data." That gap is made of three things: how you store secrets, how you scope access, and what you do the moment something leaks. Myth #1: "I moved my API key from a hardcoded string into an environment variable, so I'm secure now." This is real progress — you're no longer shipping the key inside source control — but env vars are still not a secrets store. On Linux, any process running as the same user can read another process's environment through /proc/[pid]/environ. Env vars aren't encrypted at rest, they don't have fine-grained access control (any code path in your app can read os.environ), and there's no audit log telling you which service read which key and when. A real secrets store — HashiCorp Vault, AWS Secrets Manager, GCP Secret Manager — gives you all three. Treat env vars as a transport mechanism for a secret that was fetched at runtime from a real vault, not as the vault itself. Myth #2: "I accidentally committed a key, but I deleted the file and pushed a fix, so it's gone." Git history is append-only. The commit that introduced the secret still exists in the repository's object database, reachable by anyone who clones the repo or browses its history — deleting the file in a later commit only removes it from the current working tree, not from history. Scrubbing history (git filter-repo, BFG) is possible but slow, disruptive to collaborators, and — critically — does nothing about the fact that the secret was already exposed to anyone with repo access before you noticed. The only response that actually closes the exposure is to rotate or revoke the credential immediately. Treat "remove from git" as cleanup, not remediation. Myth #3: "One long-lived API key per agent is fine as long as it's stored somewhere safe." Storage location is only half the risk equation; the other half is the credential's lifetime and scope. A long-lived key with no expiration and broad scope means that if it ever does leak — through a misconfigured log, a debugging session, a compromised dependency — the blast radius is unbounded until a human notices and rotates it. This is where dynamic secrets change the game. HashiCorp's 2025 reference architecture for AI workloads uses Vault's OpenAI secrets engine plugin to generate session-scoped API keys that auto-expire after a short window (as little as 5 minutes) for a LangChain application. The agent never touches a static key at all — it asks Vault for a lease, uses it for the duration of a task, and the credential expires whether or not anyone remembers to revoke it. This is a "zero hardcoded secrets" pattern: nothing long-lived to steal. The same short-lived, scoped-credential philosophy shows up at the protocol level. The November 2025 revision of the Model Context Protocol spec (2025-11-25) formalized OAuth 2.1 for remote MCP servers, including token-audience binding. In plain terms: a token issued for use against MCP Server A is cryptographically bound to that server as its intended audience, so if Server A is compromised, an attacker can't take the stolen token and replay it against Server B to pull data from a completely different resource. This matters as you wire agents into more and more tools — access control isn't just "does this agent have a key," it's "is this specific token valid for this specific resource, right now." Credential hygiene solves who can authenticate, but not who should see what. This is where over-privileged service accounts become a serious AI-specific risk: a RAG agent typically authenticates once with a broad service account, then answers questions for many different end users — which means it can expose far more data than any single human could reach by clicking around manually. The fix is retrieval-time authorization: enforce document-level RBAC/ABAC at the moment of retrieval, so each chunk returned to the LLM is filtered to what that specific user is authorized to see, not what the service account can technically reach. Finally, don't forget the vector store itself. Embeddings are not anonymous — OWASP's LLM08 (Vector and Embedding Weaknesses) recognizes that embeddings can leak the sensitive text they were derived from. Your vector database is a sensitive datastore in its own right and deserves the same encryption, key management, and audit logging you'd apply to any PII-holding system — not an exemption because "it's just vectors."
| Anti-pattern | Why it fails | Better pattern |
|---|---|---|
| Hardcoded API key in source | Visible to anyone with repo access, forever in history | Fetch from a secrets manager at runtime |
| API key in an env var only | No encryption at rest, no ACLs, readable via /proc/[pid]/environ |
Env var as transport for a short-lived secret pulled from Vault/KMS |
| One long-lived key per agent | Unbounded blast radius if leaked | Short-lived, scoped, auto-expiring dynamic credentials |
| Delete-and-push after a leak | Secret still in git history, already exposed | Rotate/revoke immediately; treat history scrub as secondary cleanup |
| Single shared service account for RAG retrieval | Agent can expose the union of everyone's permissions | Document-level RBAC/ABAC enforced at retrieval time |
| Vector store treated as "just embeddings" | Embeddings can leak source text (OWASP LLM08) | Encrypt, control keys, and audit the vector store like any PII datastore |
The throughline for this week: identity and secrets are not a one-time setup task, they're an ongoing discipline of minimizing lifetime, minimizing scope, and minimizing blast radius — for every credential and every user your AI system touches.