Long-running agents rarely fail because they run out of context window — they fail because nobody designed what happens to attention quality once the transcript outgrows what the model can usefully weigh. Here is how tiered compaction, tool-output pruning, and sub-agent isolation actually work.
Most descriptions of agent context management start with "the context window is too small." That's rarely the actual failure. Modern models ship with context windows from 128K up past a million tokens — plenty of room, numerically, for a long agent run. The failure that actually takes down long-running agents is that nobody designed what happens after the transcript outgrows the budget for good attention, not the budget for raw token count. An agent that's technically still "within context" at 180K tokens can already be producing worse decisions than it did at 20K, because the signal it needs is buried under nine tool calls' worth of noise it doesn't need anymore.
This post is about the mechanics of managing that: what compaction actually does, where naive versions of it break, and the handful of patterns — hierarchical summarization, tool-output pruning, and sub-agent context isolation — that hold up in practice.
Two different problems get conflated under "context management":
Capacity is a hard wall you hit eventually. Attention quality degrades continuously, long before you hit that wall. Transformers don't have uniform recall across a long context — empirical work on long-context retrieval (the "needle in a haystack" style evaluations that became standard after early long-context models shipped) consistently shows recall sagging in the middle of very long inputs, even when the needle is technically present. An agent transcript is a worse case than a haystack eval, because it isn't a pile of unrelated facts — it's mostly redundant or superseded information: three earlier attempts at a grep that didn't match, a file read whose contents were already summarized, a tool error that got fixed two turns later. None of it is wrong to keep, exactly, but all of it competes for attention with the one constraint from turn 4 that actually determines whether turn 40 succeeds.
So the real design question for a long-running agent isn't "when do we run out of room" — it's "what's the smallest transcript that still contains every fact the agent needs to act correctly right now."
Hard truncation — drop the oldest N messages when you're over budget — is the most common first implementation, and it fails the most predictably. Early tool calls often establish constraints that stay load-bearing for the whole session: a file path the user corrected, a library version pin, a "don't touch the auth module" instruction. Truncate the front of the transcript and the agent cheerfully violates a constraint it was explicitly given, with no memory that it was ever given.
Naive summarization — pass the whole transcript to the model and ask for a summary when you cross a threshold — is better but has its own failure mode: summarization is lossy in a way that's invisible until it matters. A summarizer optimizing for brevity will compress "the migration must run idempotently because a prior attempt partially applied and corrupted three rows in staging" down to "handle migration idempotency," which sounds equivalent and isn't. The reason is often more load-bearing than the rule, because the reason is what lets the agent recognize edge cases the rule didn't anticipate. Summarize away the reason and the agent will satisfy the letter of a compressed instruction while missing what it actually protected against.
Sliding windows — keep only the last N turns — solve recency but actively discard anything established early, which is exactly backwards for the kind of session-level constraints (coding conventions, safety rules, user preferences stated once at the start) that need to survive the entire run.
Not all context is equally compressible. A workable compaction scheme separates the transcript into tiers with different rules:
A cheap win that's easy to miss: most token bloat in agent loops isn't the conversation, it's raw tool output — a full file dump, a complete npm install log, a 3,000-row query result the agent only needed the row count from. The fix isn't summarizing this after the fact; it's not putting it in the transcript in the first place. Structured tool responses that return exactly what the caller asked for — a count, a diff, a status code — instead of everything the underlying operation happened to produce, avoid the compaction problem for a large fraction of tokens entirely. This is one of the practical arguments for narrow, single-purpose tools over one that dumps its full internal state back to the model: an MCP tool that returns a computed result (say, a validated JSON diff or a formatted timestamp conversion) hands the agent exactly the fact it needs, with nothing to prune later. Contrast that with a tool that pastes back a whole file or log and then relies on summarization to clean up after itself — by the time compaction runs, the damage to attention quality already happened for however many turns the bloat sat in context.
The other lever, alongside compacting a single transcript, is not putting things in it at all. When a parent agent delegates a bounded subtask — "read these forty files and tell me which one defines the auth middleware" — to a subagent, the subagent's entire exploration (every file it opened, every grep that missed) stays in its context and never touches the parent's. The parent gets back one message: the answer. This is context management by isolation rather than by compression, and it composes with tiered retention rather than replacing it — a long-running orchestrator can run for hours without ever holding more than a thin log of "delegated X, got back Y" in its own transcript, no matter how much exploration happened underneath.
The actual trigger logic is less interesting than people expect — it's a threshold check plus a re-summarization call, not a novel algorithm:
def maybe_compact(transcript, token_budget, threshold=0.75):
used = count_tokens(transcript)
if used < token_budget * threshold:
return transcript # plenty of headroom, do nothing
pinned = [m for m in transcript if m.pinned]
working = transcript[-WORKING_TURNS:]
compactable = [
m for m in transcript
if m not in pinned and m not in working
]
if not compactable:
return transcript # nothing left to compact; let it ride to the hard limit
summary = summarize(
compactable,
instruction=(
"Preserve constraints, decisions, and *why* they were made. "
"Drop resolved errors, redundant tool output, and anything "
"superseded by a later turn."
),
)
return pinned + [summary_message(summary)] + working
The details that matter aren't in this function — they're in summarize's instruction, and in how aggressively threshold is set. Compact too early and you pay the summarization cost (and its lossiness risk) on sessions that would have finished fine without it. Compact too late and you're asking a model to summarize a transcript that's already past the point where its own attention to that transcript was reliable — a compounding failure, since a degraded read of the history produces a degraded summary of it.
Compaction and isolation manage what's in the context; they don't fix a model being asked to hold too many simultaneous constraints regardless of length. A session with twelve interacting requirements is hard for a model to track well whether it's expressed in 2,000 tokens or 20,000 — no amount of pruning turns that into an easy problem, because the difficulty is combinatorial, not spatial. The corollary for anyone designing an agent harness: treat context management as necessary hygiene that keeps a long session from actively hurting itself, not as a substitute for keeping the task itself decomposed into pieces a model can actually reason about at once.
Takeaway: if your agent gets worse the longer it runs, check what's actually in its context before you check the model. Pin the handful of facts that must never be forgotten, summarize the rest with an explicit instruction to preserve why not just what, keep raw tool output out of the transcript when a narrower response would do, and push bounded subtasks into sub-agents whose exploration never has to enter the parent's history at all. None of this requires a bigger context window — it requires deciding, on purpose, what's still worth the model's attention at each point in a long run.