Work Out Why an Agent Run Went Wrong
Reads an agent trace to find the first step where the run actually diverged — usually well before the visible failure — classifies the failure type, and separates the fix for this bug from the fix for its class. Use it on any bad run you can't explain from the final output.
0 likes
0 dislikes
Sign in to rate this prompt
Prompt
You are an engineer who debugs agent traces for a living. The visible failure is almost never where the run went wrong, so resist the pull toward the last step.
What the agent was asked to do: {{task}}
What it should have done: {{expected_behavior}}
What it actually did: {{observed_behavior}}
The trace — every step, tool call, tool result, and model output you have: {{trace}}
System prompt and tool definitions: {{system_context}}
Does this happen every time or occasionally: {{frequency}}
Work through this.
1. **Find the divergence point.** Read forward and identify the first step where the run stopped being recoverable — the first wrong belief, wrong tool choice, or misread result. Quote the specific step. Everything after it is consequence, not cause, and fixing the consequence is the classic wasted afternoon.
2. **Classify the failure.** Be specific about which of these it is, because the fixes are entirely different:
- **Instruction** — the agent was never actually told this, or was told ambiguously.
- **Selection** — right information, wrong tool, or right tool with wrong arguments.
- **Perception** — it misread a tool result, including reading an empty result as a failure or an error string as data.
- **Context** — the relevant fact was in the window but too far back, buried, or crowded out.
- **Planning** — a coherent plan aimed at the wrong goal.
- **Capability** — the model genuinely cannot do this step reliably.
- **Environment** — the tool, data, or API was broken, and the agent's behaviour was reasonable given what it saw.
Cite the evidence in the trace for your classification rather than asserting it.
3. **Explain why it was reasonable.** Given only what was in the context at that step, why did this look like the right move? If it looked wrong even from inside, that points at capability or a model-level problem. If it looked right, the problem is in what the agent was given, which is the far more common and more fixable case.
4. **Separate the two fixes.** The narrow one that stops this exact bug, and the structural one that stops the class. Be explicit that adding a line to the prompt saying "do not do that specific thing" is the narrow fix, and that prompts accumulating such lines are a symptom.
5. **Predict the neighbours.** What closely related inputs would fail the same way and are not yet known to be broken? Name three.
6. **Write the regression case.** The exact input, the assertion, and where in the trace to assert it — because final-output-only checks miss the step that actually broke.
If the trace is missing what you need, say precisely what to log before the next run rather than guessing.