Skip to content

Constraint Encoding Does Not Fix Constraint Compliance

Reformatting constraints — more structured, more compact, more formal — does not improve how reliably models follow them. Compliance is determined by what constraints say, not how they are laid out.

Related lesson: The Ceiling — this concept features in a hands-on lesson with quizzes.

The experiment

A study across 11 models, 16 benchmark tasks, and 830+ LLM invocations tested three encoding forms — including a compact header form — and four propagation modes against a constraint satisfaction rate (CSR) metric (Fang et al., 2025).

Compact headers cut constraint-portion tokens by about 71% and full-prompt tokens by 25 to 30%. This result replicated across three independent rounds.

Compliance did not move. There was no statistically significant difference in CSR across encoding forms or propagation modes. Effect sizes were negligible (Cliff's δ < 0.01) (Fang et al., 2025).

Compact headers are a free token saving. They are not a compliance fix.

What actually drives compliance

The study found that constraint type — not encoding — produced the largest compliance variation: a 9 percentage point gap between conventional and counter-intuitive constraints. Counter-intuitive constraints failed at 10 to 100% rates regardless of encoding. Conventional constraints achieved 99%+ compliance regardless of encoding (Fang et al., 2025).

The gap between understanding and execution also depends on how you measure it. Model self-assessment systematically overestimates compliance relative to rule-based scoring (Fang et al., 2025). A model that reports following a constraint may not be.

Why this matters for practitioners

Engineers debugging compliance failures often reach for formatting as the fix — restructuring bullets, adding headers, switching to YAML-style constraint blocks. The evidence does not support this approach.

When a model misses a constraint:

Actual cause Correct response
Constraint count too high Decompose into sequential turns; see Constraint Degradation
Constraint is counter-intuitive or conflicts with training priors Simplify, split, or enforce mechanically via linter/test
Constraint position buried in a long prompt Front-load or repeat; see Critical Instruction Repetition
Constraint competes with too many others Move to schema validation, type checker, or pre-commit hook
Reformatted constraint (encoding change) Has no measurable effect — do not invest here

Using compact encoding for its actual benefit

Compact headers are worth adopting — for token budget reasons. A 71% reduction in constraint token consumption is meaningful across long sessions or busy agent loops. This adds to the saving from prompt compression applied to the rest of the system prompt.

# Compact header form — token-efficient, compliance-neutral
[CONSTRAINTS]
lang: TypeScript
no-imports: os, fs
return: Promise<Result>
max-lines: 50

# Natural language form — same compliance, higher token cost
The function must be written in TypeScript. You should not import
the os or fs modules. It must return a Promise<Result>. Keep the
implementation under 50 lines.

Both forms produce the same constraint satisfaction rate. Use the compact form to reduce token consumption, not to improve compliance.

When this backfires

Compact headers are token-efficient and compliance-neutral, but the trade-off is not zero:

  • Human readability drops: the [CONSTRAINTS] block shown above is harder to audit than the prose form when a constraint silently fails and you need to trace why.
  • Token savings do not matter at low volume — a single-use prompt has no busy agent loop to amortize the saved tokens across.
  • Ambiguity grows at the edges: compressing a rule like max-lines: 50 into a bare key-value pair forces the model to infer intent on unusual inputs, while the prose form ("Keep the implementation under 50 lines") leaves less room for misreading.
  • Counter-intuitive constraints stay unsolved either way — see Constraint Degradation for the lever that actually moves them.
  • The neutrality result is scoped to constraint blocks inside a coding prompt. Broader prompt-format work found that format can move task performance by up to 40% on smaller models, while larger models hold steadier (He et al., 2024). Do not extend encoding neutrality beyond the constraint-satisfaction setting.

Key Takeaways

  • Encoding form (natural language vs. structured headers vs. formal spec) has no measurable effect on constraint compliance — Cliff's δ < 0.01 across 830+ invocations
  • Constraint type dominates compliance: counter-intuitive constraints fail at high rates regardless of how they are formatted
  • Compact headers yield a real 25–30% full-prompt token reduction — worth applying for cost and context budget reasons
  • When compliance fails, invest in constraint design: simplify, reduce count, remove counter-intuitive requirements, or enforce mechanically
  • Model self-assessment of compliance overestimates actual satisfaction rate — measure with rule-based scoring, not model self-report
Emerging — single recent source Last reviewed: 2026-05-27
Feedback