Title: Lecture 12 — From Log Lines to Traces: How Langfuse and Phoenix See Your LLM App
Course: AIINFRA 301: Production RAG & LLMOps — Observability and Evaluation > Section: Week 12 — Tracing and Observability: Langfuse and Arize Phoenix

📖 Lecture — From Log Lines to Traces: How Langfuse and Phoenix See Your LLM App

By now you've built RAG pipelines and probably debugged one the hard way: staring at print() statements, unsure whether the retriever pulled the wrong chunks, the prompt mangled context, or the LLM hallucinated. This week fixes that with a proper observability data model — the one used by two free, self-hostable tools: Langfuse and Arize Phoenix.

The trace/observation model

First misconception to unlearn: a trace is not one flat log line per request. If it were, you'd know a request was slow or wrong, but not where. Instead, Langfuse and Phoenix model an LLM request as a hierarchical tree:

Concept | What it represents | Example in a RAG app
Trace | The top-level container for one end-to-end request. Carries metadata like userId, sessionId, and tags. | One user question hitting your /ask endpoint
Span | A generic unit of work nested inside a trace. | "retrieve_documents", "rerank", "build_prompt"
Generation | A specialized span specifically for an LLM call. Captures model name, input/output tokens, cost, and latency. | The actual call to your chat completion model
Event | A discrete, instantaneous occurrence — not a duration. | "cache_hit", "guardrail_triggered", "retry_attempted"

Because these nest (a Trace contains Spans, a Span can contain child Spans or Generations, Generations can have Events attached), you can answer questions a flat log never could: Which retrieval call returned irrelevant chunks? Did the slowdown happen in embedding, reranking, or generation? Localizing degradation to a specific node is the entire point of structured tracing. This matters especially for the Generation type. If every LLM call were logged as a plain Span, you'd have no structured place for model name, token counts, or cost — you'd be parsing it out of free text after the fact. Modeling it as a Generation lets your backend natively aggregate "total spend on gpt-4o-mini this week" or "p95 generation latency" with no custom parsing. Keep this in mind for Discussion 12.

OpenTelemetry: the shared substrate

The good news: you are not learning two incompatible proprietary systems. Both Langfuse and Arize Phoenix build on OpenTelemetry (OTEL), the industry-standard tracing protocol. Langfuse can act as a full OTEL backend via its /api/public/otel endpoint, so any OTEL-instrumented app can ship traces to it without a Langfuse-specific SDK. Phoenix achieves the same portability through OpenInference, an OTEL-compatible semantic convention for LLM spans, auto-instrumenting libraries like the OpenAI SDK once you call register() and .instrument(). Instrumentation you write once is portable across backends — comparing both, as Lab 12 asks, means pointing the same spans at a different collector, not rewriting code.

Setting up Langfuse (self-hosted, Docker Compose)

Langfuse ships as a Docker Compose stack you run entirely on your laptop: clone the repo, set three required environment variables (NEXTAUTH_SECRET, SALT, ENCRYPTION_KEY), run docker-compose up -d, then visit the local UI and create the first account — it automatically becomes admin. Instrumenting Python code is then simple with SDK v4.5.1:

from langfuse import observe, get_client

@observe
def answer_question(query: str) -> str:
    docs = retrieve(query)       # nested calls are auto-traced too
    return generate(query, docs)

The @observe decorator wraps any function and automatically creates a nested Span (or Generation, if it detects an LLM call) inside the current trace. Version note: older tutorials referencing from langfuse.decorators import observe are outdated — that module was removed. The current import is from langfuse import observe, get_client; use get_client() to manually update trace-level metadata like tags, user ID, or session ID.

Setting up Arize Phoenix (self-hosted)

Phoenix installs as a pip package, no Docker required:

pip install arize-phoenix openinference-instrumentation-openai

from phoenix.otel import register

tracer_provider = register()
# then instrument your LLM client library, e.g.:
from openinference.instrumentation.openai import OpenAIInstrumentor
OpenAIInstrumentor().instrument(tracer_provider=tracer_provider)

register() starts the Phoenix session; the UI is available at localhost:6006, and it also exposes an OTel gRPC collector on port 4317 for spans from other OTEL-instrumented services, not just Python. Every OpenAI call your app makes after .instrument() is automatically captured as a Generation-equivalent span — no manual decoration needed.

The propagation gotcha: root spans don't share attributes with children

The second misconception this week corrects, and it trips up almost everyone the first time: setting an attribute like user_id** on the root span does not automatically make it available on child spans. OpenTelemetry spans are independent objects; a child does not inherit arbitrary attributes from its parent just because it's nested underneath it. Set user_id only once at the top, and every other span in the trace simply won't have that field when you try to filter or aggregate by it. The fix is OpenTelemetry Baggage combined with a BaggageSpanProcessor**. Baggage propagates key-value context through the whole call chain (not just one level down); the BaggageSpanProcessor copies baggage entries onto every span it processes, filtered to the keys you specify:

from opentelemetry.baggage import set_baggage, attach
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BaggageSpanProcessor

provider = TracerProvider()
provider.add_span_processor(BaggageSpanProcessor(lambda key: key in {"user_id", "session_id"}))

ctx = set_baggage("user_id", "u_123")
attach(ctx)

With this in place, every span created afterward — root, child, grandchild — carries user_id, and your dashboards can filter and aggregate correctly across the whole trace, not just the top node. Once retrieval, reranking, and generation are each their own Span or Generation inside one Trace, you can pinpoint exactly which stage degraded, instead of guessing from a pile of print statements.