Title: Lecture 8 — From Fixed Pipelines to Graphs and Agents
Course: AIINFRA 301: Production RAG & LLMOps — Observability and Evaluation > Section: Week 08 — Advanced Retrieval Patterns: GraphRAG and Agentic RAG

📖 Lecture — From Fixed Pipelines to Graphs and Agents

Every RAG system you've built so far follows the same shape: embed a query, do a similarity search against a vector store, stuff the top-k chunks into a prompt, generate an answer. That pipeline is reliable and cheap, but it has a blind spot — it treats retrieval as a single, static lookup. Some questions need more than a lookup. "What did the Q3 report say about churn?" is a lookup question. "What are the major recurring themes across all of our customer feedback this year?" is not — no single chunk contains that answer, because the answer is a property of the whole corpus. This week we look at two different ways production systems handle that gap: GraphRAG, which restructures the corpus itself, and Agentic RAG, which restructures the retrieval process. GraphRAG, developed by Microsoft Research, starts with an offline indexing phase. An LLM reads through your source documents and extracts entities and the relationships between them, building a knowledge graph. GraphRAG then runs the Leiden algorithm — a community-detection algorithm — over that graph to group related entities into hierarchical "communities" (think: clusters of clusters, from tight-knit topic groups up to broad themes). For each community, an LLM generates a natural-language summary. The result is a two-level index: an entity level (the raw graph of entities and relationships) and a community level (the hierarchy of summaries). That two-level index is what enables two distinct query mechanisms:

Query mode | Best for | How it works | Example question
Local Search | Specific, entity-centered questions | Walks the entity graph outward from matched entities, pulling in related entities, relationships, and source text | "Who reported to whom in the reorg described in these memos?"
Global Search | Broad, thematic, corpus-wide questions | Reasons over the community summary hierarchy, map-reducing partial answers from many communities into one | "What are the main themes across all board meeting minutes this year?"

This Local vs. Global split is the core mental model for GraphRAG: if you can point to roughly where the answer lives, use Local; if the answer requires synthesizing across the entire corpus, use Global. Agentic RAG takes a completely different approach to the same problem: instead of restructuring the corpus, it restructures the retrieval loop. Rather than "retrieve once, generate once," an agent — commonly built today with LangGraph, the most mature open framework for this pattern — decides at runtime which tool to call: vector search, a graph query, a SQL query against structured data, or even a web search. Because LangGraph models the workflow as a graph with support for loops, persistent state, and human-in-the-loop checkpoints, the agent can retrieve, look at what it got back, decide it's insufficient, reformulate, and retrieve again — a plan → retrieve → critique → re-retrieve cycle instead of one shot. If you want hands-on practice, GraphRAG's setup is refreshingly concrete: pip install graphrag, then graphrag init, which scaffolds a .env and a settings.yaml for you to configure. The important operational flag here: indexing is an expensive, LLM-heavy offline batch job — every entity and relationship extraction, every community summary, is an LLM call. Start with a small corpus before you point it at anything production-sized. Correcting two common misconceptions. First: it is tempting to treat GraphRAG and Agentic RAG as strictly-better upgrades and reach for them on every project. They're not free. Agents inherit "agent messiness" — looping, dead ends, extra hallucination surface, and higher cost and latency from multiple LLM calls per query. GraphRAG adds an expensive offline graph-construction step before you've answered a single question. Recent benchmarks show that a well-built agentic search loop over plain dense retrieval can close much of the quality gap to GraphRAG without that upfront indexing cost — so the decision is a real tradeoff, not an obvious upgrade path. Second: building a knowledge graph does not make GraphRAG's answers automatically correct. Many basic implementations have no built-in mechanism to validate that a generated graph query makes sense, or to check whether what was retrieved is actually sufficient to answer the question. Query validation and context-sufficiency checks are things you have to design in — they don't come for free just because there's a graph underneath.