Skip to content

Context Compression Strategies: Offloading and Summarization

Tiered compression (offloading large payloads and summarizing history) keeps long-running agents within the context window without losing task continuity.

Learn it hands-on: Offload vs Summarize, a guided lesson with quizzes.

The problem

Long-horizon tasks accumulate context from conversation turns, tool inputs, and tool outputs. Without compression, the agent truncates arbitrarily or the session fails. Compression keeps the task intent and critical state while discarding low-value content.

Tiered compression

LangChain's Deep Agents framework implements three compression tiers, applied in order as context pressure increases (Context Management for Deep Agents):

graph TD
    A[Context fills] --> B{Large tool response?}
    B -->|Yes| C[Offload to filesystem]
    C --> D[Replace with reference + summary]
    B -->|No| E{85% threshold reached?}
    D --> E
    E -->|Yes| F[Summarise conversation history]
    F --> G[Restart with compressed context]
    E -->|No| H[Continue normally]

Tier 1: offload large tool responses

Replace large tool payloads (full files, API responses, search results) with a filesystem reference and brief summary. Full content goes to disk, and the agent re-reads it when needed. This keeps content recoverable without holding payloads in active context. Addressable recall compaction generalizes the same move to every compacted observation, keyed by a stable identifier. You can configure the thresholds. Frameworks typically set them in the tens of thousands of tokens.

Tier 2: summarize conversation history

When context fills further, summarize prior turns. Keep the current objective, key artifacts, decisions and rationale, and next steps. Discard exploratory turns, superseded instructions, resolved errors, and intermediate reasoning that did not affect outcomes. Reasoning that is still load-bearing is a different case: dropping it between tool calls makes the agent re-derive its working hypothesis every turn, which retaining reasoning across tool calls avoids. The agent restarts with the summary as prior context. Anthropic's context engineering guide calls this "compaction" and names it a core strategy for long-horizon tasks.

Cache preservation during compaction

Compaction reuses the parent session's cached prefix, so a cache_control breakpoint at the end of the system prompt keeps that cache valid across the cycle. Only the new summary lands as a fresh entry, which keeps post-compaction turns cheap (Anthropic's compaction guide).

Progressive five-stage compaction

OPENDEV extends the two-tier approach with Adaptive Context Compaction (ACC), a five-stage pipeline triggered at specific context budget thresholds (Bui, 2026 §2.3.6):

Stage Trigger Action
1 — Warning 70% budget Log context pressure for monitoring; no data reduction
2 — Observation Masking 80% budget Replace older tool results with compact reference pointers
2.5 — Fast Pruning 85% budget Prune older tool outputs beyond recency window (see Usage-Reinforced Memory Decay for a retention-scored alternative to the fixed window)
3 — Aggressive Masking 90% budget Shrink preservation window to only most recent outputs
4 — Full Compaction 99% budget Serialize history to scratch file; LLM-summarize middle portion

Recent tool outputs stay at full fidelity. An Artifact Index serialized into compaction summaries tracks every file touched, and the summary carries the history archive path. That makes compaction effectively non-lossy (Bui, 2026 §2.3.6).

Graduated stages let the agent degrade step by step rather than hitting a single compression cliff where the full history collapses at once.

What to preserve in summaries

Summaries that only capture "what happened" without "what matters next" cause objective drift. An effective summary structure:

Section Content
Objective The original task and any scope changes
State What has been built, changed, or decided
Constraints Any constraints surfaced during the session
Next steps The immediate next action

Why it works

Transformer attention runs over all tokens in the window. As context grows, relevant signal competes with accumulated noise (redundant tool outputs, superseded reasoning, resolved errors) and retrieval precision degrades. Compression reduces this noise floor. Offloading removes content that is addressable on demand but rarely needed. Summarization distills decision rationale and state into a compact form the model can condition on. The mechanism is selective discarding, not lossy encoding. Artifacts remain on disk, so compaction is non-destructive for recoverable content.

The effect is measurable. One empirical study reports that pruning context to the last five tool call/response pairs, plus summarization, reached 91.6% task completion versus 71% for full-context agents, at a fraction of the tokens and runtime. This supports combining the offload and summarize tiers rather than carrying full history (Pruning and summarizing context for tool-using agents).

When this backfires

Compression degrades task continuity when applied incorrectly:

  • Silent context loss: aggressive summarization drops subtle constraints whose importance only emerges later. Anthropic's context engineering guide recommends starting with maximum recall and iterating toward precision, not the reverse.
  • Premature compaction: a too-low threshold forces lossy summarization when context is still navigable. If the summary omits scope constraints, it causes objective drift.
  • Broken recoverability: offloaded payloads deleted or moved after compaction cannot be re-read, which makes the approach worse than in-context storage. The observation store must persist for the full session lifetime.
  • Compounding errors across cycles: each cycle introduces summarization error. Long sessions accumulate drift a single summary cannot undo.

Testing compression

  • Threshold stress-testing: lower the threshold, then verify task continuity across cycles
  • Recoverability: after offloading, verify the agent retrieves content on demand
  • Objective drift check: after summarization, verify the next action matches the original task

Key Takeaways

  • Tiered compression is ordered on purpose: offload before summarizing, so recoverable content moves to disk before anything is discarded for good.
  • Five-stage compaction degrades gradually. The Artifact Index serialized into each summary keeps every touched file traceable, so even the 99%-budget full-compaction stage stays effectively non-lossy.
  • Summaries must preserve task objective, current state, and next steps rather than just action history.
  • Offloading preserves recoverability. Summarization is lossy, so retain decision rationale, not just outcomes.
  • Compaction reuses the cached prefix, so a cache_control breakpoint at the end of the system prompt keeps post-compaction turns cheap.

Example

Pseudocode showing how tiered compression maps to agent configuration:

# Pseudocode — illustrates the tiered compression pattern,
# not a specific framework's API.

agent = Agent(
    tools=[...],
    # Tier 1: offload tool responses above 20k tokens to disk
    max_observation_length=20_000,
    observation_store="./agent_observations/",
    # Tier 2: summarise at 85% context budget
    compaction_threshold=0.85,
    compaction_summary_prompt=(
        "Summarise: (1) current objective, (2) key artifacts created, "
        "(3) decisions made and rationale, (4) immediate next step."
    ),
)

The summarizer prompt structure maps to the preservation table above: objective, state, constraints, next steps.

FAQ

Does pruning context improve results, or only cut cost?

Both, on the available evidence. Pruning context to the last five tool call/response pairs, plus summarization, reached 91.6% task completion against 71% for full-context agents, at a fraction of the tokens and runtime (Pruning and summarizing context for tool-using agents). That supports combining the offload and summarize tiers rather than carrying full history.

What happens if the compaction threshold is set too low?

Premature compaction forces lossy summarization while the context is still navigable, and it causes objective drift when the summary omits scope constraints. Anthropic's context engineering guide recommends starting with maximum recall and iterating toward precision, not the reverse, because aggressive summarization drops subtle constraints whose importance only emerges later.

How do I test a compression setup?

Three checks. Lower the threshold deliberately, then verify task continuity across cycles. After an offload, verify the agent retrieves the stored content on demand. Payloads deleted or moved after compaction cannot be re-read, so the observation store must persist for the session lifetime. After summarization, verify the next action still matches the original task.

Established — multiple independent sources Last reviewed: 2026-06-13