📖 Lecture — From "Did It Work?" to "How Did It Get There?"

Up to now in this course, you have mostly asked one question about your agents: did the final answer come out right? That question — an outcome-level metric — is necessary, but for agentic systems it is nowhere near sufficient. An agent that books the wrong flight after three wasted tool calls and a hallucinated confirmation number, then gets lucky and stumbles onto the correct one, "passes" an outcome-only check. An agent that reasons cleanly, picks the right tool on the first try, and fails only because an external API returned a stale price "fails" the same check. Neither verdict tells you what to fix. This week is about evaluating the whole path an agent takes, not just where it ends up. Outcome vs. trajectory metrics. Outcome-level metrics measure task completion: did the agent produce the correct final answer, in the correct format, within budget? Trajectory-level metrics score the entire path to get there — step efficiency (did it take a reasonable number of actions?), argument correctness (were tool calls parameterized correctly?), tool correctness (did it choose the right tool for the job?), plan adherence (did it stick to a sensible plan, or thrash?), plan quality (was the plan itself sound?), and reasoning quality (were intermediate justifications coherent?). For agents, how the answer was reached matters as much as the answer itself, because a fragile path that happened to succeed once will not generalize to the next user request.

Agent-as-a-Judge. This idea extends the familiar LLM-as-judge pattern (one LLM call grading another LLM's output) into something built for agents: a full agentic system — with its own tools, memory, and multi-step reasoning — critiques another agent's entire trajectory, not just its final output. Because the judge is itself agentic, it can inspect decision points, re-run intermediate tool calls, and check whether an earlier wrong turn was actually recovered from later. This captures much richer signal than a single-shot judge reading a final transcript. Benchmarks built for tool-use. MCP-Bench evaluates tool-using LLM agents on complex, realistic tasks executed against live MCP servers — not toy sandboxes — and scores them with an LLM-as-a-judge across three dimensions: task completion quality, tool-selection rationale, and planning effectiveness. This mirrors exactly the trajectory-level thinking above: it does not just check whether the task got done, it checks whether the agent picked defensible tools for defensible reasons along the way. Tooling you can use today. You do not need to build judge infrastructure from scratch. DeepEval is an open-source, Pytest-style framework with 50+ modular metrics spanning RAG, agents, multi-turn conversations, and safety, and it supports custom LLM-as-judge metrics, pass@k sampling, trajectory grading, and world-state grading. OpenAI Evals and LangSmith/LangChain Evals (paired with the AgentEvals library of readymade trajectory evaluators) offer overlapping capabilities with different ecosystem integrations. All three let you write evaluation as code and run it in CI, the same way you already run unit tests. Designing a judge you can trust. A judge prompt is a spec, not a suggestion — write it like one. Give the judge an explicit rubric (what separates a 1 from a 5), few-shot examples of graded outputs at different quality levels, and require a structured JSON response that includes cited evidence before a numeric score, so the model has to justify its verdict rather than pattern-match to a number. Even then, validate: run the same trajectory through the judge multiple times and check for score stability, and combine offline benchmark scores (e.g., AgentBench) with online scoring of real production traces, since production inputs are messier than curated benchmark tasks. Judges are not infallible. In expert domains — law, medicine, mental health — LLM-judge agreement with human experts can drop to roughly 64–68%, below the agreement rate between human experts themselves. Judges also tend to apply hard pass/fail thresholds that miss partial correctness (an agent that is 80% right on a nuanced legal question is not "wrong," but a binary judge may score it that way). The fix is not to abandon automated judges — it's to calibrate them against human-labeled samples regularly and use layered evaluation, where automated judges triage at scale and humans review disagreements or high-stakes domains. Correcting three common misconceptions. First, a high score on a public benchmark like MMLU does not mean your agent will do well on your task — popular benchmarks are largely saturated and test factual recall in a vacuum, divorced from your specific retrieval pipeline, prompts, and tools; build task-specific evaluations on your own data instead. Second, passing unit tests or schema checks is not the same as evaluating the agent — automated structural tests verify that a response is well-formed, not that it is true; a response can pass every JSON-schema check and still hallucinate a false, legally binding commitment. Third, LLM-as-judge is not automatically objective or reliable enough to use alone — reliability varies significantly by domain, judges apply hard thresholds that miss partial credit, and the only defensible practice is to calibrate against human labels and layer automated judging with human review rather than trusting a single judge's verdict.