Skip to content

Choosing a Compression Budget for Agent Control Context

Set the compression budget for an agent's always-loaded instructions from environment-verified task success, because token-reduction ratios do not predict where control breaks.

An agent control context is the static system-side layer the model sees on every turn: instructions plus tool documentation, covering tool preconditions, argument constraints, policies, planning rules, output requirements, and recovery rules (Hou and Yang, arXiv:2608.01056v1). Compressing it is a budget decision, and the only budget you can defend is one you set by running the agent in its environment and counting completed tasks.

Check these conditions before compressing

The published frontier transfers only under conditions your setup may not meet.

  • Caching. The control context is the most cacheable region of a request, and caching alone cuts API cost by 41 to 80 percent and time to first token by 13 to 31 percent on multi-turn agentic work with 10,000-token system prompts (Lumer et al., arXiv:2601.06007v2). Anthropic's cache is ordered tools, then system, then messages; a change at one level invalidates that level and everything after it, and the rewrite bills at 1.25 times base input (Anthropic — Prompt caching). Compressing a stable control context can cost more than leaving it alone.
  • Method. Token-importance compressors fail on this surface specifically, collapsing to mean reward at or below 0.05 across all 17 tested environment and backbone combinations (Zhang and Sun, arXiv:2605.26596v1).
  • Measurement. Input-token reduction is not a cost metric, because compression can expand output enough to raise total spend. At a 0.3 retention ratio one provider showed 56x output expansion on MBPP and 5x on HumanEval, with prompt structure rather than provider identity as the moderator (Johnson, arXiv:2603.23527v1).

Read the measured frontier as regions

CompressAgent measured 15,525 runs across nine control contexts, three task families, and three fixed Qwen endpoints against a 93.8 percent full-context baseline (arXiv:2608.01056v1):

Retained context Generic rewriting Section-based Obligation-aware Mechanical truncation
75% 92.7% 92.4% 80.4% 55.4%
50% 37.8% 67.9% 49.5% 22.1%
35% 19.9% 47.0% 39.0% 16.6%
25% 16.7% 20.4% 21.2% 16.1%
10% 9.8% 11.7% 2.4% 0.3%

Method choice dominates below 50 percent retention: section-based compression holds 47.0 percent at a 35 percent budget where generic rewriting has fallen to 19.9 percent. The 75 percent figure marks a region boundary rather than a measured threshold, because the authors fit no change points and name the 75 to 50 percent gap as their coarsest interval.

Treat 75 percent as a starting hypothesis, then walk the budget down on a held-out task set until environment-verified success drops below what you will accept. CompressAgent used 45 development and 225 held-out test tasks in deterministic sandboxes. Watch the error types as you go, not just the totals.

Why it works

Compression breaks agents through the machinery of acting rather than through reasoning, which is why environment outcomes detect it and text-similarity scores do not. The tokens carrying action semantics are the identifiers, brackets, argument names, and action verbs. Those are exactly the tokens that rank lowest under self-information, so an importance-ranking compressor strips them first and the environment rejects what remains. AGORA names this action-grammar destruction and reports all 17 tested combinations of environment, backbone, and method collapsing to mean reward at or below 0.05 despite 1.3x to 13.3x compression (Zhang and Sun, arXiv:2605.26596v1). CompressAgent corroborates from the failure side: 79.7 percent of its 9,992 failures were tool-execution errors and 20.0 percent were output or action-parsing errors, so "compression changes an agent's operational failure mode, not merely answer accuracy" (arXiv:2608.01056v1).

Keeping semantic sections whole is the practical consequence, on weaker evidence. CompressAgent reads its section-based result as stable boundaries and source order preserving interactions among tool definitions, policies, protocols, and recovery clauses, but runs no ablation on section types, so that half is inferred. AGORA's four-way component ablation supplies the independent support: it isolates the structural keep-floor as the dominant quality lever (arXiv:2605.26596v1).

When this backfires

  • Prompt caching is already active and the control context is stable, per the caching condition above.
  • You port a budget rather than measure one. The safe ratio moves with prompt structure, by a factor of 11 in measured output expansion between two benchmarks under identical compression (arXiv:2603.23527v1).
  • You reach for a general token-importance compressor. LLMLingua scored 10.0 percent at a 50 percent budget and 0.0 percent at 25 percent on this benchmark (arXiv:2608.01056v1).
  • The agent takes irreversible or safety-critical actions. The authors state control context compression should not be used in safety-critical settings without environment-level validation and continuous monitoring (arXiv:2608.01056v1).
  • The control context is already short. Deleting obligations you do not need beats compressing the ones you do, and it needs no evaluation harness to verify. See system-prompt bloat reduction.
  • Your results come from a different setting. The benchmark used nine English control contexts, three fixed Qwen endpoints, temperature 0, no native function calling, and deterministic sandboxes; the authors call the model spread a robustness check rather than scale-law evidence (arXiv:2608.01056v1).

FAQ

Is 75 percent retained context a safe default?

Only for methods that keep semantic units intact, and only as a hypothesis to test. At a 75 percent budget, generic rewriting reached 92.7 percent and section-based 92.4 percent against a 93.8 percent baseline, but obligation-aware compression had already fallen to 80.4 percent and mechanical truncation to 55.4 percent (arXiv:2608.01056v1). The grid fits no change point, so measure your own.

Which failures signal that I have compressed too far?

Tool-execution errors first, then action-parsing errors. Across 9,992 measured failures, 79.7 percent were tool-execution and 20.0 percent were output or action parsing, with execution errors dominating at moderate budgets and parsing failures taking over below 25 percent retention (arXiv:2608.01056v1). Rising malformed tool calls mean the control layer has lost its action envelope.

Does compressing the system prompt reliably cut cost?

No. Input-token reduction can be offset or reversed by output expansion, measured at 56x on MBPP at a 0.3 retention ratio (arXiv:2603.23527v1). Prompt caching also bills the repeated prefix at 0.1 times base input on a read and charges 1.25 times base input to rewrite it (Anthropic). Measure total spend per completed task.

Key Takeaways

  • Set the budget by running held-out tasks in the environment and counting completions; token-reduction ratios and text-similarity scores miss the failure mode entirely.
  • Compress by whole semantic units and keep source order. Section-based compression held 47.0 percent success at a 35 percent budget where generic rewriting reached 19.9 percent.
  • Instrument tool-execution and action-parsing error rates as the early warning; they move before aggregate success does.
  • Check prompt caching before compressing. A stable control context in a cached prefix is already cheap, and editing it invalidates the cache.
  • Treat 75 percent as a hypothesis carried by one coarse-gridded benchmark on nine English control contexts and three Qwen endpoints.
Emerging — single recent source Last reviewed: 2026-08-04