The Agentic Loop, Explained

How the model-decides, harness-executes, context-accumulates cycle actually works, where it breaks, and how to bound it in production.

An AI agent isn't a smarter prompt. It's a loop — the same small cycle running over and over until the model decides it's done or something forces it to stop. Understanding that cycle is the difference between debugging an agent and just staring at transcripts hoping for insight.

The Core Cycle

Strip away the branding and every agent framework runs the same loop: the model receives context and a goal, decides on an action, something executes that action, the result gets fed back into context, and the cycle repeats. Anthropic's own description of agent architecture puts it plainly — the core setup is an environment, a set of tools, and a system prompt, with the model called in a loop until it decides to stop or hits a checkpoint.

That's different from what most people picture when they hear AI call: a single request, a single response, done. The loop differs because the model's output on iteration one becomes part of the input on iteration two. Nothing about it is exotic — it's a while statement — but that repetition is what turns a language model into something that can complete multi-step, open-ended work.

If you haven't built one yet, how to build your first AI agent walks through assembling this loop end to end. This article is about what's actually happening inside it.

The Four Roles

Every agentic loop divides work across four things, and conflating them is a common source of confusion when debugging.

  • The model is the decision-maker. Given the current context, it decides what to do next — call a tool, ask a clarifying question, or declare the task finished. It has no memory beyond what's in its context window, and it can't act on the world directly.
  • The harness (runtime) is the code wrapping the model. It executes whatever the model decides to do, manages the context window, appends results, and calls the model again. This is Claude Code, an agent SDK's tool-runner, a LangGraph node, or hand-rolled orchestration code — the plumbing, not the intelligence.
  • The tools are the agent's only way to affect or observe anything outside its own context: reading a file, running a shell command, hitting an API, querying a database. The model can only pick from tools the harness has exposed to it.
  • The stopping condition decides when the loop ends: the model emits a final answer with no further tool calls, a success check passes, a max-iteration or budget limit is hit, or a human interrupts.

Mixing these up is why the agent is stuck reports are often vague — stuck could mean the model keeps choosing bad actions, the harness is mismanaging context, a tool is silently failing, or there's no real stopping condition at all. Diagnosing a stalled agent starts with figuring out which of the four roles is misbehaving.

One Iteration, Walked Through

Take a concrete task: find out why the test suite is failing and fix it. Here's a single turn of the loop:

  1. Context in. The model receives the system prompt, the user's goal, and everything accumulated so far — prior tool calls, their results, any earlier reasoning.
  2. Decision. The model decides its next action. Not I should run the tests as a diary entry — a structured tool call: run_command(cmd: "bin/rails test").
  3. Execution. The harness — not the model — actually runs that command. The model has no shell access; it only requested one.
  4. Result appended. The harness captures stdout, stderr, and exit code, and appends that result back into the context as a tool result message.
  5. Loop again. The model now sees the failing test's stack trace in context and decides its next action — maybe read_file(path: "app/models/prompt.rb") — and the cycle repeats.

This keeps going: read a file, propose an edit, apply it, rerun the tests, read the new output. Eventually the model either sees green tests and reports success, or the harness cuts it off.

A minimal version of the loop looks like this:

context = [system_prompt, user_goal]

while not stopped:
    action = model.decide(context)

    if action.is_final_answer:
        stopped = True
        break

    if iterations >= MAX_ITERATIONS:
        stopped = True
        report_incomplete()
        break

    result = harness.execute_tool(action)
    context.append(action)
    context.append(result)
    iterations += 1

Every production agent framework is a more careful version of this — with retries, timeouts, summarization, and permission checks layered in — but the skeleton doesn't change.

Why Accumulating Context Matters

Every iteration grows the context: the tool call, the tool result, sometimes the model's own reasoning. That accumulation has two direct costs.

Token cost compounds. Because the full context is resent on every model call, a loop with a large history pays for that history again and again — an agentic loop can rack up far more model calls than a single-shot prompt, and each call gets pricier as context grows. Prompt caching and routing cheaper models to simple steps blunt this, but they don't eliminate it.

Drift sets in. As context fills with tool output, error messages, and half-finished reasoning, the model's attention gets diluted. It can lose track of the original goal, repeat work it already did, or anchor on a stale plan from ten turns ago instead of what the latest tool result actually says. This is why long-running harnesses invest in context management — compacting old turns into summaries, offloading notes to a scratch file outside the context window, or delegating side quests to a sub-agent that returns a short summary instead of its full working history.

Tool design bears on this directly: a tool that returns a bloated, unstructured dump eats context budget and increases drift on every call it's used in. That's a large part of what designing tools your AI agent can actually use is about — the loop only works as well as what gets written back into it.

Common Failure Modes

The loop's simplicity is also its risk surface. Three failure modes show up constantly in practice:

  • Infinite or near-infinite loops. The model keeps retrying a failing action, oscillates between two states, or never produces a final answer because its stopping criteria are ambiguous. Left unbounded, this burns tokens and time with nothing to show for it.
  • Error cascades. A tool call returns a bad or malformed result — a truncated file read, an API error the model misreads as success — and the model builds several subsequent decisions on that faulty premise. Each turn compounds the mistake instead of correcting it.
  • Losing the thread. On long tasks, the model drifts from the original goal, starts optimizing for a sub-problem it invented, or forgets constraints stated early in the conversation before context was compacted or trimmed.

The mitigations are unglamorous but effective, and any harness running loops in production needs all three:

  • Max iterations or a token/cost budget, so a stuck agent fails loudly instead of running indefinitely.
  • Explicit error handling on tool results — surfacing failures clearly to the model instead of letting ambiguous output get treated as success, and giving the model a real recovery path (retry, ask for help, abandon the sub-task).
  • Clear, checkable termination conditions defined up front — a test suite passing, a specific file existing, an explicit success schema — rather than relying on the model to know when it's done.

Loop vs. Single Call vs. Fixed Workflow

A single LLM call has no loop at all: one prompt in, one completion out, no tool execution, no reacting to intermediate results. A fixed workflow — a predetermined sequence of steps, possibly each involving an LLM call — has structure but no real decision-making; the order is set by the code, not chosen by the model at runtime. The agentic loop sits apart from both because the model decides the next action dynamically, based on what actually happened last iteration, and that sequence isn't known in advance.

That flexibility makes loops powerful for open-ended problems and risky overkill for well-defined ones — a fixed workflow is often more reliable, cheaper, and easier to debug when the steps are already known. Choosing correctly between the three is its own decision, covered in AI agent vs. workflow vs. single call: how to choose.

Where This Fits

The loop is the engine; everything else — tool design, memory strategy, guardrails, evaluation — is built around keeping that engine from running off the rails or burning through budget for no reason. If you're assembling one for the first time, how to build your first AI agent is the practical next step from here.

← Back to Articles