Observability & Debugging

Why chains are harder to debug than a single API call#

A single LLM API call is easy to debug: one prompt goes in, one response comes out, and if something looks wrong there's exactly one place to look. A multi-step chain — especially one involving retrieval, memory, and possibly an agent deciding its own sequence of tool calls — has many places a problem could originate: a bad retrieval, a memory step accidentally dropping context, a prompt template rendering with the wrong variable, or the model itself reasoning incorrectly given otherwise-correct input. Debugging a chain means being able to see what actually happened at each step, not just the final output.

Tracing chain execution#

Tracing captures the full execution of a chain — every step's input and output, in order, with timing — as a structured record you can inspect after the fact, rather than trying to infer what happened purely from the final answer. LangChain's tracing integration (LangSmith) captures this automatically once configured:

export LANGCHAIN_TRACING_V2=true
export LANGCHAIN_API_KEY=your-api-key-here

# No code change needed in the chain itself -- tracing is enabled via
# environment configuration, and every .invoke() call is captured automatically
response = rag_chain.invoke("What's the refund policy for enterprise customers?")

With tracing enabled, each step of a run (the retriever call, the exact rendered prompt, the raw model response, the parser's output) is recorded individually and viewable as a timeline. This turns "the answer was wrong, somewhere" into "the retriever returned zero relevant chunks for this query, so the model correctly said it didn't know" — a concrete, actionable finding instead of a guess.

For situations without a hosted tracing tool, even a basic verbose/debug mode gives some of this visibility:

import langchain
langchain.debug = True  # prints every step's input/output to stdout

response = rag_chain.invoke("What's the refund policy for enterprise customers?")

Common failure modes to look for#

• Silent retrieval failures. The retriever runs without error but returns irrelevant or empty results — often invisible unless you specifically inspect what was retrieved, since the chain doesn't "fail" in any exception-raising sense; it just answers badly or says it doesn't know.
• Prompt template variable mismatches. A typo in a variable name ({qeustion} instead of {question}) either raises a clear error or, worse, silently renders as a literal, unfilled placeholder in the prompt — tracing the exact rendered prompt (not just the template) catches this immediately.
• Memory dropping needed context. A window or token-limited memory strategy (see this track's memory page) can silently drop the exact earlier turn a later question depends on — the chain runs fine, and the answer is just wrong or confused, with no error to point at the cause.
• An agent looping or picking the wrong tool. An agent (see this track's agents-and-tools page) can get stuck calling the same tool repeatedly without making progress, or pick a plausible-but-wrong tool for a request — visible in a trace as a sequence of think/act/observe steps that isn't converging, but invisible if you only look at whether a final answer eventually appeared.
• Output parser mismatches. The model's raw response doesn't match what the output parser expects (for example, it added a conversational preamble before a JSON block), causing a parsing error that, without a trace, looks like "the chain crashed" rather than clearly showing the exact raw text that failed to parse.

Common mistakes#

• Debugging only by staring at the chain's final output — for anything beyond a single-step chain, the final output alone rarely tells you which step actually went wrong.
• Enabling tracing only in production incidents, after the fact — tracing needs to be on (or easy to turn on) before a problem happens, since you generally can't retroactively trace a request that already completed without it.
• Assuming an agent that "eventually answered" behaved efficiently — a trace can reveal it took many more tool calls (and much more cost/latency) than necessary to get there, which is invisible if you only check whether a final answer showed up.