How to Trace and Debug an AI Agent

The first agent bug you hit will not look like a bug. It will look like the agent being stupid. It called the wrong tool, or called the right tool with nonsense arguments, or went round the loop nine times doing almost the same thing.

Your instinct is to fix the prompt. Resist it for ten minutes, because you cannot debug what you cannot see, and print statements do not survive contact with a system whose control flow is decided at runtime by a language model.

Why ordinary logging fails here

In a normal program you wrote the path, so a log line tells you where you are. In an agent you did not write the path — the model chose it. That breaks logging in three specific ways.

There is no fixed sequence to log against. One run takes four steps, the next takes nineteen. Grepping a flat log for what happened means reconstructing a tree from a stream.

The interesting state is the context window, and it is enormous. The question you actually need answered is what did the model see immediately before it made that decision, and that is thousands of tokens of accumulated history, tool results and instructions. No log line holds it.

Failures compound silently. A tool returns something slightly wrong at step three. The model incorporates it, reasons on, and fails visibly at step eleven. The log line you go looking at is eight steps downstream of the cause.

What you need is a trace: a tree of spans, where each model call, each tool call, and each nested agent invocation is a node with its inputs, outputs, timings and token counts attached.

The three things every span must carry

Whatever tooling you pick, an agent trace is only useful if each span records:

1. The exact input. For a model call, the full rendered messages — not your template, the thing that was actually sent. For a tool call, the exact arguments the model produced.
2. The exact output. The completion, including tool-call requests. For tools, the exact result string that went back into context.
3. Cost and latency. Input tokens, output tokens, wall time.

Miss the first and you will spend an afternoon proving the model saw what you assumed it saw. It usually didn't — a template variable came through empty, or a tool result got truncated somewhere.

There is a standard now, and it is worth following

OpenTelemetry maintains GenAI semantic conventions, now in their own repository. They define agent spans specifically — not just model calls:

• create_agent {gen_ai.agent.name} — operation create_agent
• invoke_agent {gen_ai.agent.name} — operation invoke_agent, emitted as a CLIENT span for a remote service and INTERNAL for a local framework call
• invoke_workflow {gen_ai.workflow.name} — operation invoke_workflow
• plan {gen_ai.agent.name} — operation plan

With attributes including gen_ai.operation.name and gen_ai.provider.name (both required), gen_ai.agent.id, gen_ai.request.model, gen_ai.request.temperature, gen_ai.conversation.id, and gen_ai.usage.input_tokens / gen_ai.usage.output_tokens.

Note that invoke_workflow is a distinct operation from invoke_agent. That is the same distinction as agent versus workflow — the model chose the path, or you did — carried down into the telemetry, which is a good sign the conventions were designed by people who had actually operated one of these.

One honest caveat: these attributes are marked Development, not Stable. error.type is the only stable one in the set. Expect renames. Follow them anyway — a rename is a migration, whereas a bespoke schema is a permanent tax, and OpenTelemetry means you can point at Grafana, Honeycomb, Datadog, Phoenix or Langfuse without re-instrumenting.

Reading a trace: the four failure shapes

Once you can see the tree, agent failures sort into four recognisable shapes. Knowing them turns debugging from archaeology into pattern-matching.

The tool was never called. The model had a tool that would have worked and didn't reach for it. This is almost always a description problem, not a reasoning problem — see how to design tools your agent can actually use. Check whether the description says when to use it, or only what it does.

The tool was called with garbage. Look at the arguments in the span, then at the inputSchema you published. Missing description fields on parameters, an enum you documented in prose instead of in the schema, a date format you assumed. The model wrote what your schema told it to write.

The loop spins. The same tool, similar arguments, three or four times. This means the result is not answering the question and the model has no other move. Look at the tool's output: it is usually an unhelpful error string, an empty list that should have said no matches for X, or a wall of JSON the model can't extract from. Fix the result, not the prompt.

The context poisoned itself. Something wrong entered at step three and everything downstream is confidently built on it. Trace back to the first span where the content diverges from reality. This one is why you keep full inputs.

The general lesson: most agent failures are tool failures wearing a reasoning costume. Traces are what let you tell the difference.

Traces are not evals

A trace tells you what happened in one run. It cannot tell you whether the change you just made improved anything, because agents are non-deterministic and your sample size is one.

Keep both. Traces diagnose a specific failure. Evals tell you whether the fix worked across a set of cases without breaking two others. Fixing a prompt on the strength of one trace and shipping it is how agents get quietly worse over time — everyone remembers the trace they fixed, nobody measures the ones they broke.

The useful workflow is a cycle: a trace surfaces a failure, the failure becomes an eval case, the eval tells you whether the fix generalises.

Practical instrumentation notes

Log to stderr, not stdout, if you are inside an MCP server. stdout carries JSON-RPC on the stdio transport and anything else corrupts the stream. This is also the MCP specification's own guidance now that its logging primitive is deprecated — see what changed in MCP 2026-07-28.

Capture prompts and completions, and think about what that means. Traces containing full inputs will contain whatever your users typed and whatever your tools returned. That is a data-retention decision, not a technical one. Redact before storage, not after.

Give every run a conversation ID and propagate it. gen_ai.conversation.id exists for this. Without it, a support ticket saying it did something weird this morning is unanswerable.

Record token counts per span, not per run. The run total tells you the bill. The per-span breakdown tells you which tool's verbose output is causing it, which is the only version you can act on — see how to optimise agent cost and latency.

Trace in development too. The instinct is to add observability when something breaks in production. For agents, the trace is how you understand the thing at all. Turn it on at the first tool call.

Where to start

If you have an agent running with no instrumentation, the smallest useful step is not adopting a platform. It is writing every model call and tool call — full input, full output, token counts — to a structured log with a shared run ID, and rendering it as a tree.

That gets you most of the debugging value in an afternoon. Adopt the OpenTelemetry conventions when you want the dashboards, the alerting and the ability to change vendors without re-instrumenting.

What you should not do is keep guessing at prompts. Every hour spent tweaking wording without a trace is an hour spent debugging a system you cannot see.