Compaction & the Long Horizon

The reconciliation task from Chapter 5 ran for three hours and forty minutes and produced 340 turns. At turn 40 it was working well. At turn 120 it had started re-reading files it had already read. At turn 200 it asked a question the user had already answered. At turn 290 it produced a summary that contradicted a finding from turn 60.

The controls missed a quality failure. No tool failed or guardrail fired. The run stayed within budget while its answers became less reliable. This long-horizon failure can occur before a crash or capacity error makes the problem visible.

9.1The horizon problem#

An agent’s horizon is the length of task it can complete before context becomes the binding constraint. It depends on the workload as well as the window.

Consider the arithmetic. A 200,000-token window sounds generous until you account for what a working agent actually accumulates. Reference Agent’s stable prefix is 19,000 tokens. Its plan artefact is 800. Retrieved documents add up to 8,000 on a turn that needs them. That leaves roughly 165,000 tokens for the transcript — which sounds like a lot until you notice that a single turn with a file read and a model response can be 4,000 tokens. Forty turns of real work and the window is full.

Except it is worse than full, because usable capacity may be lower than nominal capacity. Two things degrade before you hit the ceiling.

Attention decay. Chapter 8’s lost-in-the-middle finding applies with compounding force here: the longer the transcript, the more of it sits in the low-attention middle, and the more likely a relevant earlier observation is effectively invisible.² The agent has the information and cannot reliably use it, which is operationally identical to not having it and much harder to diagnose.

Distractor accumulation. A 340-turn transcript contains dozens of superseded facts: the file as it was before the edit, the hypothesis that turned out wrong, the error that was subsequently fixed. Each is a plausible thing for the model to latch onto. The transcript is not a neutral record; it is an increasingly noisy one, and the noise grows faster than the signal because most turns produce observations that matter for two turns and then never again.

The practical consequence is a design rule that surprises people: compact well before the window is full. Waiting until 95% can leave insufficient room for a large tool result or compaction request. It does not prove that the model was already degraded or that a fixed number of earlier turns failed. A 60–75% trigger is an illustrative starting range; select it from measured continuation quality, growth bursts and output headroom. Hermes Agent documents a preflight compression check when context exceeds roughly half the window, with gateway-side automatic compression at 85% as a backstop.³ Two thresholds, one proactive and one defensive, is a good pattern.

9.2Budgeting the sawtooth#

Figure 9.1 is a schematic. The worked budget below uses the same mechanism with a trigger-to-floor drop of 101,600 tokens. A trigger ratio by itself is not a capacity plan. You need to know what the ratio applies to, what survives the drop, how quickly the transcript grows again, and how much emergency room remains when one tool returns far more than expected.

Start with the 200,000-token Reference Agent window above. Reserve 19,000 tokens for the stable operating prefix, 800 for the plan and 8,000 for retrieved material that may be needed on the next turn. Hold back another 7,200 as an output and estimation reserve. Token counters are estimates until the provider accepts the request, and a large tool result can arrive between one estimate and the next. The resulting transcript budget is 165,000 tokens:

200,000  model window
−19,000  stable prefix
−   800  plan
− 8,000  retrieval allowance
− 7,200  output and estimation reserve
────────
165,000  usable transcript budget

The proactive trigger is 70% of that transcript budget, not 70% of the whole model window. It fires at 115,500 transcript tokens. Suppose the protected head is 1,000 tokens, the six recent messages occupy 12,000, and the target summary is 900. Immediately after compaction, the transcript is about 13,900 tokens. The operation has reclaimed 101,600 tokens:

115,500  transcript at trigger
− 1,000  protected goal and contract
−12,000  recent working tail
−   900  replacement summary
────────
101,600  tokens reclaimed

At an illustrative average growth of 2,400 tokens per turn, that buys roughly forty-two more turns before the next proactive trigger: 101,600 ÷ 2,400 = 42.3. That interval belongs to this worked scenario; it is not a numeric reading of Figure 9.1. Some turns only decide or acknowledge a result; others ingest a file or a long query response. Plan with a distribution, not the largest turn you remember.

The backstop is not a second normal operating point. Here it sits at 85% of the transcript budget, or 140,250 tokens. The 24,750-token gap between the proactive trigger and the backstop absorbs a tool response that was larger than forecast, a delayed compaction job, or an inaccurate token estimate. If ordinary runs regularly cross the first threshold and rely on the second, the schedule is broken even when no request exceeds the model window.

Set that gap from observed turn sizes. If the 95th-percentile increase between two preflight checks is 18,000 tokens, a 24,750-token gap leaves 6,750 tokens for estimation error and response headroom. If the 99th percentile is 31,000, the backstop cannot catch your tail risk; compact earlier, truncate or externalise oversized tool results, or reserve more space. Percent-of-window defaults are starting points. The production threshold is a statement about the largest ordinary burst your loop must survive without asking a degraded context to rescue itself.

The floor matters as much as the ceiling. If the protected tail grows from 12,000 to 32,000 tokens because six recent messages now contain two database dumps, the post-compaction transcript becomes 33,900 tokens. With all other assumptions unchanged, reclaimed capacity falls from 101,600 to 81,600, and the tooth narrows from about forty-two turns to thirty-four. Nothing about trigger_ratio=0.70 changed, yet the practical horizon shrank by roughly a fifth. keep_recent is therefore a token budget disguised as a message count. Production code should enforce both: keep up to six messages, subject to a maximum retained-token allowance, while preserving enough working state to continue.

There is a second floor that arithmetic alone cannot show: the information floor. A fixed 900-token summary can remain exactly 900 tokens after every round while representing more history each time. The compression ratio looks stable, but each token carries a growing number of facts, decisions and exceptions. After eight rounds, the summary is not merely compressing the last forty-two turns; it is also carrying whatever survived the preceding seven summaries. A flat token count can conceal rising semantic pressure.

You can see that pressure by tracking summary lineage. Record the ids of facts and decisions presented to the summariser and the ids it emits. At round one, perhaps 31 durable items compete for 900 tokens. At round five, 96 do. If the summary is still the same size, one of three things happened: the entries became more concise, older entries were externalised, or information disappeared. Only the first two are acceptable. A summary token count without item survival tells you which size you bought, not what you lost.

Finally, budget the compaction call itself. Eight teeth mean eight additional model calls, each reading the middle it is about to replace. Compaction saves future context; it does not make the past free. That cost is justified when it prevents dozens of subsequent calls from carrying a bloated transcript, but the break-even point depends on how many turns remain. Compacting immediately before the task ends adds cost and risk for no benefit. The trigger should check both context pressure and the expectation of continued work — for example, an unfinished plan with several runnable steps — rather than firing mechanically after the terminal answer has already been produced.

9.3Compaction: the mechanism#

Compaction replaces a span of the transcript with a summary of it. The mechanism is three decisions, and all three are load-bearing.

# Excerpt from Compactor; _text and domain types are defined in memory.py.
def compact(self, messages: Sequence[Message]) -> tuple[list[Message], CompactionResult]:
    before = sum(estimate_tokens(_text(m)) for m in messages)
    head = list(messages[: self.keep_head])
    tail = list(messages[-self.keep_recent:]) if self.keep_recent else []
    middle = list(messages[self.keep_head: len(messages) - len(tail)])
    if not middle:
        return list(messages), CompactionResult("", len(tail), 0, before, before)

    summary = self.summarizer.summarize(middle, self.target_summary_tokens)
    # Summarization cannot elevate authority: source text may be hostile.
    new_msgs = head + [Message("user", summary, trust="untrusted")] + tail
    after = sum(estimate_tokens(_text(m)) for m in new_msgs)
    return new_msgs, CompactionResult(summary, len(tail), len(middle), before, after)

keep_head protects the goal and the operating contract. These are the things the agent is for, and summarising them is how goal erosion (Chapter 6) becomes structural rather than accidental. In practice this should be enforced, not configured: a compactor that can eat the goal eventually will.

keep_recent protects the live working state. The last few turns carry what the agent is doing right now — the file it just opened, the error it is diagnosing, the half-finished edit. Summarising them is the single most damaging compaction mistake, and its symptom is unmistakable: the agent loses its place and starts over.

trigger_ratio is the schedule. Discussed above; the number matters less than the fact that it is a fraction rather than a ceiling.

The summary remains trust="untrusted": transformation does not promote external text into system authority. Protected instructions come from the canonical contract; summary claims carry evidence and remain subject to re-verification. The reference retains message slices, so production adapters must also preserve complete tool-call/result groups and provider-required continuation items when choosing boundaries.

What a good summary contains#

The summariser is where compaction quality actually lives, and a generic “summarise the above” prompt produces a summary optimised for readability rather than for continuation. What the agent needs is not a précis; it is a handoff document. Six things belong in it:

1. The goal, verbatim. Not paraphrased. Copied.
2. Established facts with provenance. “The connector has three callers: reports.py:88, etl/nightly.py:214, dash/api.py:41 (found by grep at turn 34).” Facts without provenance cannot be re-verified and will be re-derived.
3. Decisions and their reasons. “Chose to patch rather than replace, because the replacement needs a schema migration that is out of scope.” Without the reason, the next window relitigates the decision.
4. What has been tried and failed. The highest-value and most-omitted category. Without it the agent will retry the failed approach, which is a large fraction of observed long-run waste.
5. Open questions and blockers. Including anything awaiting a human.
6. The immediate next action. One sentence. This is what lets the post-compaction turn continue rather than reorient.

What should not be in it: full file contents, complete tool outputs, the reasoning that led to a discarded hypothesis, and pleasantries. A summary that is 40% quoted code has not compacted anything.

Produce a handoff record for a fresh agent continuing this task.
It will see ONLY your record and the last few turns.

GOAL (copy verbatim, do not paraphrase):
ESTABLISHED FACTS (each with how it was determined):
DECISIONS MADE (each with why):
TRIED AND FAILED (each with the error or reason):
OPEN QUESTIONS / BLOCKED ON:
IMMEDIATE NEXT ACTION (one sentence):

Omit anything not needed to continue. Do not include file contents.

9.4Provider compaction and application state#

As checked on 9 November 2026, OpenAI documents both server-side compaction through context_management and an explicit /responses/compact endpoint. Their compacted continuation includes opaque encrypted state, not necessarily an editable prose summary. Keep returned items intact and follow the provider’s chaining rules. OpenAI compaction guide.

Anthropic’s context-engineering report describes compaction, structured notes and delegated contexts as complementary techniques. None is a substitute for a durable task record: keep requirements, action IDs, evidence, unresolved effects, budget use and the next verified step outside a lossy summary. Effective context engineering, published 29 September 2025; checked 8 September 2026.

A continuation should first re-read authoritative state, validate any live job handle, and choose the next requirement still lacking proof. Keep working while evidence changes or a live dependency is progressing. Repeated commentary or plan rewriting alone is not progress. A blocker needs a concrete failed dependency or missing decision; a timer expiring is not proof that an external job stopped. Before completion, compare the finished artifacts and runtime behavior with every acceptance requirement, including any authorized amendments.

A preprint submitted on 3 November 2026, You don’t need long context, reports little or no accuracy loss with frequent compaction.¹ Its findings apply to the tested long-input and long-horizon tasks. They support testing shorter active contexts, while leaving workload-specific reliability unproven. Retain paired continuation tests for critical qualifiers and unresolved effects.

9.5The three strategies#

Compacting in place is one of three answers to the horizon problem, and they are genuinely different architectures with different costs.

Compact in place is the default. One run, one trace, one run id, periodic replacement of the middle. Its weakness is that it is cumulative: after the third compaction you are summarising a summary of a summary, and detail loss compounds. The mitigation is not a better summariser; it is to keep the head immutable and to write durable facts out to memory (Chapter 10) rather than trusting them to survive successive summarisation.

Handoff ends the run and starts a fresh one from a written record. A reset can remove distractors but can also lose useful state. Whether it improves quality is empirical; the handoff must carry enough evidence to continue, and the interface should make continuity clear. It is the right shape when work naturally segments: a day boundary, a phase transition, a shift change. It also has a property the others lack: the handoff document is an artefact a human can read and edit before the next run picks it up, which makes it the best available intervention point on a long autonomous task.

Delegate keeps detailed work out of the parent context. A sub-agent reads ninety files and reports three findings; the parent’s context grows by three findings. This provides context isolation independently of any benefit from specialisation.

The products separate parent and child conversations. Goose subrecipes share no conversation history, memory or state.⁴ Hermes children start with fresh conversations, so the parent must pass sufficient goal and context.⁵ Cline gives each child a separate context window and adds its token use to the session total.⁶ Separate contexts reduce parent growth. They do not establish security isolation.

9.6The qualifier that vanished#

Consider an illustrative Reference Agent reconciliation run across two enterprise tenants. The goal is to migrate a reporting connector without changing report semantics. At turn 54, a profile of the combined export establishes a crucial fact: case_id is unique within a tenant, not across tenants. Seventeen ids occur in both tenant A and tenant B. The durable row identity is therefore (tenant_id, case_id).

At turn 68, Reference Agent tries a global deduplication on case_id. The row count drops by 47 because several duplicated ids have multiple events. The reconciliation fails. Reference Agent records the dead end, changes the transform to use the composite key and obtains matching row counts. It then moves on to timezone normalisation. By turn 92, the context crosses the proactive threshold and is compacted.

The summary contains this sentence:

Duplicate cases were removed before reconciliation; row counts then matched. Continue with timezone normalisation.

Every word is defensible, and the sentence is still operationally false. It preserved the action — deduplicate — while dropping the condition that made the action correct — by (tenant_id, case_id). It also omitted the failed global-key attempt. The recent tail contains only the timezone work, so neither surviving part of context carries the qualifier.

At turn 117, a schema adjustment requires the transform to be regenerated from the raw export. Reference Agent reads the summary, sees “duplicate cases were removed,” and writes the most obvious implementation: deduplicate on case_id. The single-tenant fixture passes because ids are unique inside that fixture. The combined job silently discards 47 valid rows. No instruction changed, the goal survived byte-identically, and the summary did not contain a contradiction. The run failed through omission.

The first debugging assumption is usually that the model forgot the goal. The second is that compaction occurred too aggressively. Neither explains this case. Raising the target from 900 to 1,200 tokens might retain the composite key by chance, but it supplies no guarantee and delays the same failure until more qualifiers compete for space. Keeping ten recent messages also misses, because the decisive evidence is older than the tail. The actual mechanism is that the handoff format treats facts as prose when the continuation requires an executable invariant.

The fix has three parts. First, represent the fact at the granularity at which it can be checked:

INVARIANT R-17
Claim: export row identity is (tenant_id, case_id), never case_id alone.
Evidence: duplicate profile at artifacts/id-profile.csv; 17 ids cross tenant boundaries.
Regression: combined-tenant fixture must retain all 47 rows lost by global deduplication.
Status: active until the source schema changes.

Second, put that record somewhere addressable and protect its identifier in the handoff. The compact summary need not reproduce the full evidence; it must say “apply invariant R-17” and identify where R-17 can be read. Third, turn the discovered failure into a regression case. A fact that matters to correctness but exists only as prose will eventually be paraphrased. A fact encoded in a test can still be ignored, but it cannot disappear silently.

This incident exposes four distinct losses that a summary evaluation must separate. Entity loss drops tenant_id entirely. Qualifier loss remembers the key but forgets that uniqueness is tenant-scoped. Provenance loss retains the claim but removes the profile that can verify it. Negative-knowledge loss retains the successful approach but forgets that global deduplication already failed. A generic semantic-similarity score can rate the compact sentence highly while missing all four, because most of its words overlap with the source history.

The failure also shows why “keep the goal verbatim” is necessary but insufficient. The goal constrains what Reference Agent must achieve. It cannot carry every fact about how this particular dataset behaves. Long-horizon reliability needs a hierarchy: immutable goal, structured handoff, addressable evidence, and executable checks. Asking one summary to serve all four roles creates a document that is simultaneously too long for context and too vague for continuation.

The general test is simple. For each statement in a handoff, ask what mistake a fresh agent could make while still agreeing with the statement. “Deduplicate cases” permits a global key. “Deduplicate on (tenant_id, case_id); global case_id lost 47 valid rows” does not. Good compaction does not merely preserve topics. It narrows the space of actions back to the decisions the earlier context already earned.

9.7Externalising state to the filesystem#

A fourth strategy is externalising state outside the context window.

The insight is that a context window is a bad place to store anything you might need later, because it is expensive, volatile, and lossy under compaction. A file is a good place: cheap, durable, and re-readable on demand. So instead of carrying findings in the transcript, the agent writes them down.

work/
  GOAL.md            the original request, verbatim, never edited
  FINDINGS.md        established facts, appended with provenance
  DECISIONS.md       choices and their reasons
  DEAD_ENDS.md       what was tried and why it failed
  NEXT.md            the immediate next action, overwritten each turn
  artifacts/         generated outputs, referenced not inlined

This turns the horizon problem into a retrieval problem, which is a much better problem to have. The source detail can remain lossless on disk, but continuation still depends on correct pointers, access, freshness and retrieval. A summary that only says “read FINDINGS.md” may omit the reason a specific invariant matters. And it produces something valuable as a side effect: a human-auditable record of decisions and evidence that outlives the run.

Aider’s repo map is a variant of the same idea from the other direction: rather than carrying file contents in context, carry a ranked structural map — “the most important classes and functions along with their types and call signatures” — built with tree-sitter and ranked by a graph algorithm over file dependencies, under an explicit token budget.⁷ The context holds a map; the detail stays on disk and is fetched when needed.

Externalised state must be re-read, which costs turns; a badly organised work directory is as hard to navigate as a long transcript; and the agent must be instructed and trained to use it, which means the operating contract grows and the discipline has to be reinforced. In practice the pattern works well when the file structure is small, fixed, and named in the contract, and works badly when the agent is left to invent its own filing system.

9.8What compaction cannot fix#

Compaction cannot recover a lost decision. If the reason for a choice was summarised away, the agent will relitigate it, and may decide differently. This is why decisions with reasons are a required field in the handoff record and why, for consequential decisions, the right home is a file rather than a summary.

Compaction cannot fix a bad trajectory. Three hours of going the wrong way, compacted, is a compact record of going the wrong way. Chapter 6’s replan triggers and Chapter 20’s budgets are the countermeasures; compaction is orthogonal to correctness.

Compaction has a floor. Below some size, a summary cannot carry enough to continue. If your handoff record needs to be 6,000 tokens to be sufficient, that is the real number, and forcing it to 900 produces an agent that starts over. The correct response to “the summary is not enough” is externalisation or delegation, not a more aggressive summariser.

9.9Measuring whether compaction works#

Post-compaction repetition rate. Count raw repeats in the twenty turns after compaction, then classify them by resource version and purpose. A high raw rate is a cheap, objective investigation signal, not proof that the summary lost facts: re-reading a changed resource or deliberately verifying an edit is legitimate. Unexplained repeats against the same version for the same purpose are wasted spend and a strong reason to inspect the handoff and paired trace.

Goal fidelity. Assert that the goal string in the state after compaction is byte-identical to the original. A test, not a metric — and one that should fail the build.

Compression ratio and its trend. Tokens after divided by tokens before. Watch the trend across successive compactions in one run: a ratio that stays constant establishes only stable size, not fact retention, and a ratio that climbs means the summary is growing to hold accumulated content, which predicts the floor problem above.

Task outcome by compaction count. Group completed runs by how many compactions they underwent and compare success rates. More compactions also select longer, harder tasks. Treat a lower success rate as a diagnostic, then use paired checkpoint experiments to distinguish compaction damage from task difficulty.

Compaction can invalidate a cache if transcript-derived content is included in the stable prefix. Keep the transcript volatile and record cache misses in the context ledger. Subsequent identical requests may warm the changed prefix again.

9.10Evaluate continuation, not summarisation#

A compactor should be evaluated on whether another agent can continue, not on whether a reader likes the summary. Fluency, semantic similarity and compression ratio are properties of the text. The product requirement is preservation of the decisions and constraints needed for the next action. A summary can score well on all three textual measures and still produce the 47-row loss above.

Build the evaluation set from real pre-compaction checkpoints. For each checkpoint, save the protected head, the middle that will be summarised, the recent tail, the external state visible at that moment and the next several ground-truth events. Mark the continuation-critical items in the middle: constraints, established facts, decision reasons, failed approaches, blockers and the intended next action. Do this before generating candidate summaries, so the labels do not bend around what a particular summariser happened to keep.

Then run two continuations from the same checkpoint. The full-history arm receives the original transcript. The compacted arm receives the protected head, candidate handoff and identical recent tail. Freeze the model version, tools, permissions and external artefacts. Differences between the two arms are then attributable mainly to the context transformation rather than to a different environment. This is a paired evaluation, not a comparison between naturally short and naturally long tasks.

Score the handoff at three levels.

1. Record fidelity asks whether each labelled item survived with its operative qualifiers and provenance. A mention of “deduplication” does not satisfy R-17. The composite key and tenant scope must be recoverable.
2. Next-action fidelity asks whether the compacted arm chooses an action compatible with the recorded decisions. Exact wording need not match; the permitted action space must.
3. Trajectory fidelity asks whether the next several steps avoid dead ends already encountered, duplicate tool calls without reason, and constraint violations. This catches summaries that look complete but cannot guide behaviour.

The labels need their own contract. For every critical item, write a positive criterion and a near-miss. R-17’s positive criterion is that both fields and the tenant-scoped uniqueness claim are recoverable. “Use the case key,” “deduplicate cases,” and “case ids are usually unique” are near-misses even though each is semantically related. Have two reviewers independently label a sample, then resolve disagreements by tightening the criterion. If reviewers cannot agree on whether a fact survived, a fresh agent has no chance of treating it consistently.

Natural checkpoints are necessary but not sufficient, because rare omissions may be the costly ones. Add a mutation suite derived from real failures. Delete one qualifier, reverse one decision reason, remove the evidence pointer, turn a known dead end into an open option, or replace an exact threshold with “small.” The evaluator should reject each mutated handoff even when its embedding remains close to the accepted one. This is not an attempt to simulate every possible bad summary. It is a regression suite for the semantic operations that have already caused harm.

Keep development and holdout checkpoints separate. If you tune the prompt against the same fifty traces every week, it will learn their vocabulary and field shapes without becoming a better continuation record. A holdout set from different task families exposes that overfitting. Refresh it when Reference Agent gains a tool or enters a workflow with new state semantics, just as you would refresh an evaluation set after a product boundary changes (Chapter 24).

Suppose an illustrative evaluation samples fifty traces at four checkpoints each, producing 200 paired continuations. Reviewers label six continuation-critical items per checkpoint, or 1,200 item opportunities. The candidate handoff preserves 1,166 and loses 34, giving item recall of 1,166 ÷ 1,200 = 97.17%. That number sounds strong until you split the items by severity.

The three missing critical constraints outweigh the other 31 omissions. Use a release gate that rejects any known critical-constraint loss; an average quality score cannot compensate for it.

The six-outcome gap also needs diagnosis rather than immediate attribution. With 200 continuations, it is a useful signal, not proof that the true success penalty is exactly three percentage points. Read the paired traces. If four failures retry recorded dead ends and two violate omitted qualifiers, you have direct evidence about the handoff. If the arms diverged because a nondeterministic search returned different results, fix the fixture before tuning the compactor.

Raw duplicate-call rate is a high-value investigation signal, not proof of information loss. Define a semantic duplicate as a repeated call against the same resource version for the same fact or purpose, with no intervening action that makes verification necessary. Reading the same path twice is waste under that definition; reading it after an edit is legitimate verification. Hash the relevant external version into the tool-call signature and classify the purpose before interpreting a rise as possible compaction loss. Otherwise an agent that correctly re-reads its modified output looks worse than one that blindly trusts stale content.

Compression ratio is similarly easy to game. A summariser can achieve a spectacular ratio by deleting the hard parts, or a weak ratio by quoting every tool result. Treat the ratio as a resource constraint after fidelity passes, never as the objective. Compare candidates on a frontier: for each token target, plot critical-item recall and continuation success. Choose the smallest record that meets the behavioural gate, not the smallest record your prompt can produce.

Production monitoring completes the loop. Offline paired tests tell you whether a compactor version is safe to ship. Live traces tell you when the task distribution has moved: a new tool emits larger results, summaries approach their token ceiling, provenance resolution falls, or post-compaction repetition rises for one workflow. Slice every metric by task type and compaction count. An overall 4% repetition rate can conceal 1% on coding tasks and 18% on multi-tenant reconciliation.

When a live metric moves, replay the exact checkpoint through the paired harness before changing the trigger. A higher trigger changes when information is lost; a larger summary changes how much context you pay to retain. Neither repairs a missing field in the handoff schema. Measurement earns its cost when it identifies which lever to pull.