The Agent Loop

A minimal loop fits in twelve lines. Its structure determines where you enforce budgets, recovery, approvals and observability. Explicit controls make crashes recoverable, spending bounded, incidents diagnosable and human interruption possible. Keep those boundaries visible so the loop stays readable.

4.1The loop, minimally#

Strip away everything and an agent is this:

def agent(goal: str, model, tools) -> str:
    messages = [system_prompt(), user(goal)]
    while True:
        response = model.complete(messages, tools=tool_schemas(tools))
        messages.append(response.message)
        if not response.message.tool_calls:
            return response.message.content          # a final answer
        for call in response.message.tool_calls:
            result = tools[call.name](**call.arguments)
            messages.append(tool_message(call.id, result))

This loop can produce useful work, but it cannot be stopped, bounded, resumed or audited safely.

The cycle has four parts: receive history and available actions, propose an action, observe its result, then decide again. A final answer ends the cycle. ReAct introduced interleaved reasoning and acting in 2022.¹ Many modern harnesses use this pattern, with additional execution, recovery and safety controls.

What has changed since is where the intelligence sits. The 2022 framing needed elaborate prompt scaffolding to coax reasoning-then-acting out of a model that had never been trained for it. Models are now trained on tool use directly: the action-request format is native, the decision to stop is native, and much of the prompt engineering that early frameworks encoded has become dead weight. The practical consequence is that a modern loop should be smaller than a 2023 loop, not larger. If your orchestration layer is fighting the model’s trained behaviour, the layer is the problem.

4.2Anatomy of a turn#

A turn is one iteration: one model call plus the actions it requested.

Step 0, budget. Check ceilings before doing anything that costs money. This ordering has a real consequence: a run that has hit its ceiling should not pay to assemble a 90,000-token prompt it will never send. It also means budgets are enforced at turn boundaries, so a single turn can overshoot — a nuance worth internalising, because it is why a max_cost_usd of one dollar does not by itself guarantee a bill under one dollar. If you need a hard per-call cap, it belongs in the gateway, not the loop.

Step 1, assemble. Build the prompt. Not “append to a list” — build, from sections, under a budget, in an order chosen for cache stability. This is Chapter 8’s subject, and the reason it is a distinct step rather than an accumulating array is that an accumulating array has no policy, and a system without a context policy degrades in a way nobody can explain.

Step 2, decide. One model call. The response is one of three things: a final answer, a set of action requests, or a refusal. Modern APIs also return thinking or reasoning content, which is discussed below.

Step 3, vet. Every requested action passes guardrails, then policy, then — if policy says ASK — a human. The critical property is that vetting is sequential and complete before any execution. A run-level HALT on the third call must prevent the batch from starting. An ordinary per-call DENY blocks that call and may leave other approved calls eligible; this is the reference policy, not an atomic batch transaction. This is cheap to get right in the loop and extremely expensive to discover in production, because the failure is silent: the model asked for something it should not have been allowed, was correctly denied, and two side effects happened anyway.

Step 4, act. Execute what survived. Independent reads can run concurrently. This reference serialises mutations; production may parallelise disjoint mutations only with an explicit isolation and ordering contract. Which is which is a property declared on the tool, not guessed from its name.

Step 5, observe. Append results and loop. Return well-formed errors to the model as observations so it can choose a corrective action.

Step 3 in the reference implementation:

def _execute_calls(self, calls, inp, action_gr):
    approved: list[tuple[ToolCall, Tool]] = []

    for call in calls:                       # vet the WHOLE batch first
        tool = self.tools.get(call.name)
        if tool is None:
            self._observe(unknown_tool_error(call), executed=False)
            continue

        if v := action_gr.worst(call):        # guardrails: content-shaped
            if v.severity is Severity.HALT:
                return RunOutcome.LOOP_DETECTED, "", v.detail
            if v.severity is Severity.BLOCK:
                self._observe(blocked(call, v), executed=False)
                continue

        verdict = self.policy.evaluate(call, tool, self.policy_ctx)
        # a tool's risk metadata may only ever TIGHTEN the verdict
        if (verdict.decision is Decision.ALLOW and tool.mutates
                and not tool.reversible and verdict.rule != "session.grant"):
            verdict = Verdict(Decision.ASK, "irreversible action", "tool.irreversible")

        if verdict.decision is Decision.DENY:
            self._observe(denied(call, verdict), executed=False)
            continue
        if verdict.decision is Decision.ASK and not self._ask_human(call, verdict):
            self._observe(declined(call), executed=False)
            continue

        approved.append((call, tool))

    for call, tool, result in self._invoke_all(approved):   # only now, act
        self._observe(result)

A denial is an observation: the model receives its reason and can choose another route. executed=False excludes denied calls from the executed trajectory. A guardrail HALT returns a terminal triple through helpers typed Optional[terminal], preserving the distinction between a deliberate stop and a crash.

4.3Parallel execution without a nondeterministic transcript#

Concurrency is valuable only when it shortens the run without changing what the run means. The reference loop therefore introduces parallelism after every call has been vetted, and only inside _invoke_all. Here is the relevant code:

reads = [(c, t) for c, t in approved if not t.mutates]
writes = [(c, t) for c, t in approved if t.mutates]
out: list[tuple[ToolCall, Tool, ToolResult]] = []

if reads and self.config.parallel_reads and len(reads) > 1:
    with self.tracer.span("tools.parallel", "internal",
                          **{"reagent.parallel.n": len(reads)}):
        with ThreadPoolExecutor(max_workers=min(8, len(reads))) as pool:
            pending = []
            for c, t in reads:
                snapshot = contextvars.copy_context()
                pending.append(
                    (c, t, pool.submit(snapshot.run, self._invoke_one, c, t)))
            for c, t, fut in pending:
                out.append((c, t, fut.result()))
else:
    out += [(c, t, self._invoke_one(c, t)) for c, t in reads]

out += [(c, t, self._invoke_one(c, t)) for c, t in writes]

The split depends on harness-owned metadata. mutates is declared on the registered Tool; the model does not get to make a call parallel by naming it read_status, and the loop does not infer safety from an HTTP verb. A read-only call may still be slow, expensive or privacy-sensitive, and it can observe concurrent changes by other actors. Read-only does not mean snapshot-consistent. Mutating calls remain serial because their order can change the world they encounter.

The grouping carries a precise semantic contract. Reference Agent treats calls emitted in one model response as independent proposals: all reads run before all writes, even if the model listed a write between two reads. A read that is supposed to verify a write must therefore be proposed on the next turn, after the write’s observation is in history. That is the correct ReAct shape — act, observe, then decide what verifies the effect. If your API promises source ordering across a mixed batch, this implementation is wrong for your contract; preserve order or reject mixed batches instead of hoping the model guessed your scheduler.

Parallel execution must not make the trace lie. Python worker threads do not automatically inherit the caller’s contextvars, so each submission carries a copied context. Without those two lines, the tool spans still exist but appear as unrelated roots. The tools worked; the trace claims nobody asked them to. Chapter 23 explains why a structurally false trace is worse than a missing duration: it breaks causal diagnosis while looking complete.

The second loop over pending also preserves proposal order in the returned observations. Futures finish in whatever order the dependencies permit, but Reference Agent collects their results in the order they were submitted. That choice trades some immediacy for a repeatable transcript: replaying the same scripted model does not shuffle observations because one endpoint was faster on Tuesday. It also creates a batch barrier. The fastest result is not appended for the model until the slowest approved read finishes, because _invoke_all returns the whole list before _execute_calls observes it. For three short independent reads, that is a clean trade. For a search fan-out containing one 30-second straggler, it is visible latency and a reason to use per-tool deadlines or split the work across turns.

Do not confuse parallelism with a larger allowance. Six reads dispatched together still consume six tool calls, six dependency quotas and the sum of their monetary cost; only their wall-clock intervals overlap. Reserve budget for the whole approved batch before submitting any member. Otherwise a loop with three calls left in its tool budget can launch eight at once and discover the breach only on the next turn.

4.4Termination is the hardest part#

Starting an agent is easy. Stopping one correctly is where the engineering is, and it is systematically under-designed because the happy path — the model decides it is done — is the only one most teams implement.

A production loop distinguishes eight terminal outcomes, each requiring a different operational response:

The distinction between BUDGET_EXHAUSTED and LOOP_DETECTED illustrates why granularity pays. Both describe a run that did not finish. But a budget exhaustion says the task was bigger than the allowance, and the fix is arithmetic. A loop detection says the agent was stuck, and the fix is almost always a tool returning an error the model cannot act on — a stack trace instead of a sentence, a 403 with no explanation, a schema error that does not name the missing field. One is a capacity problem and one is a design problem, and collapsing them into “failed” means you will tune the wrong thing.

Ordering the stop conditions#

When several stop conditions could fire, which one gets to name the outcome matters. Consider an agent making the same tool call over and over with a turn ceiling of twenty and a repeat threshold of three. Both will stop it. But loop detected names the actual problem and budget exhausted buries it, so the loop detector must be checked first — when the repetition threshold is reached within an admitted turn. Figure 4.1 does not guarantee this precedence if the turn ceiling fires before the next proposal is generated. Specify tie-breaking and test both orderings.

def test_loop_detection_beats_the_turn_ceiling(toolset, policy, tmp_path):
    """A stuck agent should be reported as stuck, not as out-of-budget.
    Both stop it; only one names the actual problem."""
    path = str(tmp_path / "example.txt")
    (tmp_path / "example.txt").write_text("fixture")
    same = [Script(calls=[ToolCall("read_file", {"path": path})])] * 10
    loop = make_loop(toolset, policy, same, budget=Budget(max_turns=9))
    assert loop.run("read it forever").outcome is RunOutcome.LOOP_DETECTED

The ceilings you actually need#

Turns catch structural non-convergence — the agent that will never finish regardless of budget.

Tokens catch context blowup, usually a tool returning something enormous.

Wall-clock catches hangs in dependencies, which token limits cannot see because a stalled HTTP call spends no tokens.

Cost catches the expensive-model-plus-long-context combination that looks fine in token count and is ruinous in dollars.

Tool calls catch fan-out — the agent that spawns forty parallel searches per turn and stays under every other limit.

Hermes documents a default main budget of 500 iterations and a delegated cap of 50. Child budgets are independent, so their combined work can exceed the parent’s cap.² These are product defaults, not workload recommendations. Set child limits and a shared budget from task risk, cost and expected duration. Unattended work also needs recovery rules. Checked 9 November 2026.

4.5A loop failure, turn by turn#

At 10:07, Reference Agent is asked to explain a failed checkout deployment. It reads the incident record, then calls fetch_dashboard with the deployment id. The monitoring service returns HTTP 403 — but the adapter hands the model a 1,900-character HTML login page, complete with scripts and styling, rather than a typed authentication error.

The user sees the event stream say reading deployment telemetry for nearly a minute. The first diagnosis is that the model is weak: it keeps choosing the same tool despite evidence that the tool does not work. The trace shows a more useful mechanism. The model did not receive evidence it could act on. It received a web page containing the words sign in, generated request ids and changing timestamps. Each response looked superficially new, while the only reasonable route to the requested telemetry still appeared to be fetch_dashboard.

The following figures are illustrative, but the arithmetic is exact:

The correct signature for repetition is the tool name plus canonical arguments, not the model-generated call id and not the returned text. Call ids are fresh on every proposal. Error pages often contain fresh timestamps. Include either in the comparison and identical intent looks different forever. With a three-repeat threshold over the canonical call signature, turn four becomes LOOP_DETECTED. With only a twenty-turn ceiling, the same run reaches at least $1.059 if each of the remaining sixteen turns costs only $0.054, and then ends as BUDGET_EXHAUSTED — an outcome that blames capacity for a broken observation.

The immediate fix is not a sterner prompt. The tool adapter should translate the response into something like: Authentication expired. Re-authenticate or escalate; retrying unchanged will return the same result. That observation names the condition, names the available choices and declares whether an unchanged retry can make progress. On the next turn, Reference Agent can ask for re-authentication, use an approved alternative source, or escalate with the incident facts it already has.

The repeat detector remains necessary. Adapters regress, dependencies invent new error shapes, and models sometimes ignore perfectly good instructions. But the detector limits the blast radius; it does not repair the dependency contract. Raising the turn ceiling would make the incident longer and more expensive. Lowering it would stop legitimate investigations early. Changing the model might hide the fault temporarily. The durable fix is to make tool failures actionable and make repeated intent visible to the loop.

This generalises beyond exact repetition. An oscillation between search_logs and fetch_dashboard is the same failure with period two. A sequence that changes an irrelevant argument on every retry is the same failure disguised as exploration. Detect signatures over a short window, record the period and show the repeated observations together. The operator needs to see the cycle, not twenty isolated red spans.

4.6Streaming, and the perception of progress#

A loop that returns only when finished is a loop whose user has no idea whether it is working. For an agent that runs for minutes, this is not a polish issue; it is the difference between a tool people trust and one they kill and retry, doubling your cost and teaching them the agent is unreliable.

Three things can be streamed, and they are independent decisions.

Token streaming emits model output during generation. Many APIs support it. It improves perceived latency and is particularly useful for speech output (Chapter 18).

Event streaming reports harness activity: reading three files, tool returned, waiting for approval. It makes progress visible during long tasks. Publish these events from the state log to the user interface.

Intermediate results surface partial products — a draft, a table, a file diff — before the run completes. The highest-value and most expensive to design, because it requires the harness to know which artefacts are presentable.

The design constraint that catches people is that streaming and durability interact. If you stream a partial response and the run then crashes, what did the user see, and what does the log say? The resolution is that streamed output is a view of state, never a substitute for it: append the event first, then emit. Systems that emit first and persist later produce the worst failure in this class — a user who saw an answer the system has no record of.

4.7Reasoning and progress are different signals#

Use the provider’s supported reasoning controls and preserve required continuation items (Chapter 2). OpenAI may return opaque reasoning items and an optional summary. These items are not a raw reasoning transcript. Preserve them according to the API contract. Reasoning guide, checked 9 November 2026.

Stream useful progress from actual harness events: a tool started, an artifact changed, a verification passed, or an external job is still running. A fluent status message, reasoning summary or repeated plan is not evidence of forward progress. For long work, require a concrete change in authoritative state or a verified wait on a live job. If neither occurs, inspect the cause, change the approach within scope, or escalate a demonstrated blocker. Preserve the original goal across context boundaries and audit every completion requirement before marking the task done.

4.8Loop variants by workload#

Four workloads require different loop configurations:

Conversational loops run one to fifteen turns with a human attending. Latency dominates; state is the conversation; interruption must be excellent, while authority and data controls still apply.

Task loops run tens to hundreds of turns toward a defined goal — analyse this data, fix this test, produce this report. The human sets the goal and reads the output. This is where budgets, planning (Chapter 6) and trajectory evaluation (Chapter 24) start earning their keep.

Long-horizon loops run for hours or days, unattended. Two concerns become central: context management, because accumulated evidence may outgrow the active window; and durability, because a six-hour run has meaningful exposure to deployments, rate limits and process failure. A long-horizon loop without checkpointing is a lottery ticket.

Real-time voice loops invert the usual priorities. One to three turns, a sub-second latency budget, and a human audibly waiting. A standard loop needs adaptation: you cannot afford a full context assembly per turn. Voice loops pre-assemble, cache aggressively, and may perform authorized, independent reads while speaking. Do not speculate on mutations or bypass an approval while speech is unfinished.

The variant that surprises teams is that these coexist in one system. Reference Agent is reachable by voice and by chat, and takes long-running background work. It does not have three loops; it has one loop with configuration — LoopConfig in the reference implementation — because three loops means the guardrail you add in one is missing from the other two.

4.9Reserve the next turn, then measure convergence#

A ceiling checked at the top of the loop limits what has already happened. It does not, by itself, cap the turn you are about to admit. At $0.82 of a $1.00 cost ceiling, tracker.exceeded() correctly says the run is still inside budget. If the next model call and tool batch can cost $0.28, admitting them can still produce a $1.10 run. The check is not faulty; it answers the weaker question.

For a hard bound, turn the preflight check into an admission calculation:

admit = consumed + reserved_next_step <= ceiling

The reservation must cover exposure, not an optimistic average. Reserve the estimated input at the selected model’s rate, the configured maximum output, and the maximum permitted cost of the proposed tool batch. Use the gateway’s authoritative token count and pricing when available; a character-count estimate is useful for planning but too loose for a financial guarantee. Time needs the same treatment: current elapsed time plus the model deadline plus the longest concurrent read path, then the serial write deadlines.

Tool calls do not exist at the first preflight, so admission happens in two stages. Before the model call, reserve its input and maximum output plus any configured per-turn tool allowance. After the response has been vetted, replace that allowance with the approved batch’s declared maximum before dispatch. If tools have no declared cost bound, you can enforce a model-only cap or a soft total-cost target, but you cannot honestly promise a hard combined cap.

That reservation path is a required production gateway or shared-ledger extension. The reference Reference Agent does not implement it: BudgetTracker.exceeded() checks only accumulated values at turn boundaries, while record_call() increments tool-call usage when _invoke_one dispatches a call. Treat the hard-cap arithmetic here as the production contract to add, not as a guarantee already provided by the reference loop.

Suppose an illustrative Reference Agent task is at turn eleven. The model rate configured for this example is $3 per million input tokens and $15 per million output tokens. The next request contains 60,000 input tokens, permits at most 4,096 output tokens, and may invoke tools costing up to $0.04. Its reserved cost is:

• input: 60,000 / 1,000,000 × $3 = $0.18
• output: 4,096 / 1,000,000 × $15 = $0.06144
• tools: $0.04
• total next-step exposure: $0.28144

The projected run cost is $0.82 + $0.28144 = $1.10144, so a one-dollar hard cap rejects the turn before prompt submission. It may compact and recalculate, route to a cheaper permitted model, return BUDGET_EXHAUSTED with the projection, or ask an external controller for a new run with a different budget. It may not silently negotiate with itself.

Three details prevent false confidence. First, reservation can be conservative without charging the reservation: after the call, replace it with actual usage. Second, concurrent tools reserve the sum of monetary costs and call counts even though wall-clock reserves only the longest concurrent path. Third, an in-process timeout cannot guarantee termination if a dependency ignores cancellation. The gateway or worker boundary must enforce the deadline from outside the blocked call (Chapter 22).

Reservations across concurrent runs#

A reservation enforced by one process is sufficient for one run and insufficient for a shared tenant budget. Suppose Reference Agent has consumed $72 of a tenant’s illustrative $100 daily allowance. Two workers begin at the same time, each reads $28 remaining, and each reserves a next step of $20. Both local checks pass. If both steps spend their reservation, the tenant reaches $112. No individual loop violated the value it read; the system violated the budget because reading and reserving were separate operations.

Shared ceilings need an atomic ledger. Admission must change committed + reserved in the same transaction that checks the ceiling. The first worker moves the exposure from $72 to $92 and receives a reservation id. The second now sees only $8 available and is rejected or queued. When the first step finishes at an actual cost of $14, the ledger converts $14 from reserved to committed and releases the unused $6. If the worker dies, the reservation expires only after the controller has established that no external spend can still arrive against it.

That last clause matters. A five-minute lease is not evidence that a model provider or paid tool abandoned the request. Releasing it while the request remains live allows a replacement run to spend the same balance. Tie reservation expiry to the external request deadline, and reconcile late usage records before treating the released amount as final. For a provider whose usage arrives asynchronously, keep a settlement margin based on the maximum unreported exposure rather than pretending the dashboard is a bank balance.

Budgets also form a vector, not one interchangeable pool. Spare turn capacity cannot pay for exhausted wall-clock time; unused token allowance cannot authorise a sixth tool call after a five-call fan-out limit. Reserve and settle each resource independently, then admit only if every hard dimension fits. The first rejecting dimension becomes the reason attached to BUDGET_EXHAUSTED, which is more useful than “budget exceeded” and lets the operator change the correct ceiling.

Not every threshold deserves this machinery. A per-run turn limit is local and exact. A weekly cost target used for capacity planning can be eventually consistent and alert at 80%. A contractual tenant cap or an emergency kill switch needs central, atomic admission and a fail-closed path when the ledger is unavailable. Label each limit as hard or soft in configuration. Otherwise an engineer will eventually treat a monitoring threshold as a guarantee, or make a harmless planning target take production offline.

Budgets tell you when to stop. They do not tell you whether the loop was making progress before it stopped. Add convergence measures beside the outcome histogram:

These measures answer different questions. Turns to completion tells you how much deliberation successful work requires. Repeated-intent depth tells you whether an apparently active loop is revisiting the same decision. Headroom tells you whether success is robust to ordinary variance: a run completing at 99.7% of its time allowance is a success in the product metric and a near miss in the operating metric. Attribution agreement tests whether the outcome taxonomy names causes rather than whichever limit happened to fire last.

Do not set one universal turn target across voice, chat and background work. Segment by workload, tenant policy, tool family and model configuration, then compare a release with its own baseline. A drop from twelve turns to seven is not automatically an improvement if the agent stopped verifying results. Pair efficiency with outcome and trajectory quality (Chapter 24), and inspect examples whenever the aggregate moves.

The operational sequence is simple. Page on a breached hard invariant: admitted exposure above a hard cap, a run with no terminal outcome, or a repeated-intent depth above the configured detector threshold. Trend p50 and p95 convergence measures. Sample traces behind the change. Then fix the earliest boundary that produced misleading evidence. The loop is small enough that every one of these numbers should map back to one explicit branch.

4.10What the loop must never do#

Never execute an action before vetting the whole batch. Covered above; the reason it recurs is that the natural loop shape is for call in calls: vet(call); execute(call), which is wrong in a way that testing rarely catches.

Never turn a denial into an exception. A policy denial is expected operation. Raising on it conflates “the system correctly refused” with “the system broke,” which pollutes your error rate, pages the wrong person, and — worst — denies the model the chance to route around a blocked path.

Never let the model choose its own budget. Any mechanism where the model requests more turns, a bigger allowance, or an extension is a mechanism where a confused or manipulated model removes its own limits. Budgets are set outside the loop and are not negotiable from inside it.

Never lose the reason. Every terminal outcome, every denial, every guardrail trip carries a human-readable reason. Debugging a run that ended with outcome=guardrail_stop and no detail is archaeology.

Never make resumption a separate code path. If starting and resuming are different functions, they will diverge, and the divergence will be discovered during an incident. Chapter 5 is the whole argument.