📖 Lecture — What Makes an AI System "Agentic"?

Welcome to AIINFRA 300. You've spent the last several courses getting comfortable with Python, REST APIs, containers, and cloud deployment. This week we start stacking a new layer on top of that foundation: agentic AI. Before you write a line of agent code, you need a clear, defensible answer to a deceptively simple question — what actually makes a system "agentic," and what doesn't? Workflows vs. agents. Anthropic's engineering team, in their widely-cited piece "Building Effective AI Agents," draws a sharp and useful line. A workflow is a system where LLMs and tools are orchestrated through predefined code paths — you, the developer, decide the sequence of steps, and the LLM fills in specific tasks within that fixed structure. An agent, by contrast, is a system where the LLM dynamically directs its own process — it decides which tools to call, in what order, and when it's done, based on what it observes as it goes. Anthropic's guidance is blunt: start with the simplest solution possible, usually a single well-optimized LLM call, and only add agentic complexity — loops, planning, multi-step tool orchestration — when that simpler solution demonstrably falls short. This isn't a hedge; it's a cost-and-reliability argument. Agentic loops multiply the number of model calls, multiply the number of places where errors can compound, and multiply latency. You should treat "let's make it an agent" as a decision that needs to be earned, not a default. Defining agentic AI. So when do you cross that line? Think of "agenticness" as a dial with three settings you can turn independently:

Agentic AI systems are autonomous systems pursuing complex goals with minimal ongoing human oversight. Critically, the division of labor is: the user defines the "what" (the goal, the constraints, the definition of done) and the agent determines the "how" (the specific sequence of actions taken to get there). A system that just executes a hardcoded sequence you specified step-by-step, even if an LLM is involved at each step, is a workflow — you defined both the what and the how. Correcting three common misconceptions. Now let's address head-on the ways people misuse this term, because you will hear all three in industry and need to push back correctly.

1. "Anything built on an LLM is an agent." False. An LLM is a component — a reasoning engine you call. A single API call, a chatbot that answers one turn at a time, or a wrapper script around a prompt is not agentic just because a model is involved. Agency is a property of the system's control flow, not of which component sits inside it.
2. "More agentic is always better." Also false, and this is the one experienced engineers get wrong most often. Anthropic explicitly recommends the simplest solution that meets the bar. A single, well-optimized LLM call with good retrieval frequently outperforms an agent on cost, latency, and reliability, because every additional autonomous step is a chance for the system to go off the rails. Agentic systems trade latency and cost for task performance on genuinely complex, open-ended problems — that trade is only worth making when the problem actually demands it.
3. "Adding tools to an LLM makes it an agent." Tool use is necessary but not sufficient. A model that can call a calculator or search the web is still just executing a workflow if the calling pattern is fixed by your code. True agency requires a closed feedback loop: the system plans, acts, observes the result, and adapts its next action based on that observation — and it needs state management (tracking what's been tried) and error handling (recovering when a tool call fails or returns something unexpected) to sustain that loop reliably.

The provider landscape you're building into. This judgment call doesn't happen in a vacuum — it happens against a fast-consolidating market. Google, OpenAI, and Anthropic together control roughly 90% of the \~\$37B enterprise LLM market as of the end of 2025. Within that trio, the competitive order recently shifted: in early 2026, Anthropic captured 34.4% of enterprise AI tool spend versus OpenAI's 32.3% — the first time Anthropic has surpassed OpenAI in business adoption. Yet no single vendor dominates: 81% of CIOs report using three or more model families in production, which is exactly why a vendor-neutral integration layer matters. That's where the Model Context Protocol (MCP) comes in — one year post-launch, the MCP ecosystem has grown to 97M+ monthly SDK downloads, SDKs in five languages, and 500+ public servers, with backing from Anthropic, OpenAI, Google, and Microsoft. MCP is quickly becoming the standard way agents connect to tools and data regardless of which model is doing the reasoning — the subject of the rest of this course. Why this matters for the rest of the course. Every subsequent week — MCP servers, tool design, multi-agent orchestration, deployment — assumes you can look at a system and correctly say "this is a workflow" or "this is an agent," and justify why, and that you understand the multi-provider reality you'll be building against. That judgment call, more than any specific framework syntax, is the skill employers are hiring for right now.