Agent Design

Architecture, delegation, memory, control, reliability, and harness patterns for building effective agents.

Core Design

Foundational architecture decisions — how to structure agents, delegate work, and separate concerns.

• Agent-First Software Design — Architect systems where AI agents are the primary consumers, using machine-readable APIs and structured outputs instead of visual UIs
• Agent-Operable Interface Design (Affora) — Keep a screen's controls, choices, state, and outcomes recoverable from its machine-readable structure so one interface serves both a person and a computer-use agent, and fix only the deficits that measurably move agent success
• The Agent Stack Bet: Architectural Decisions for Production Agents — Four architectural bets — identity, universal context, durability, and platform — that move production-agent concerns from application code to the platform layer, with the conditions under which each bet pays off
• Emergent Architecture in AI-Driven Codebases — AI coding agents produce codebases with measurable architectural biases — pattern replication, abstraction bloat, and stack convergence — that compound across agent runs
• Agentic AI Architecture: From Prompt-Response to Goal-Directed Systems — Reference architecture separating cognitive reasoning from execution, a topology taxonomy for multi-agent coordination, and an enterprise hardening checklist
• Agent Development Lifecycle for Agent Products — A four-phase meta-lifecycle (build, test, deploy, monitor) where verdict-labeled production traces feed the next evaluation and build cycle
• Agentic Flywheel: Self-Improving Agent Systems — A closed loop where agents analyze their own traces and metrics to generate harness improvements that make all future agent work better
• Agent-Discoverable Slash Commands — Slash commands become model-callable primitives when the agent's planner can read their descriptions and invoke them mid-loop, collapsing the boundary between human-invoked shortcuts and agent-invoked tools
• Agents vs Commands: Separation of Role and Workflow — Commands define what to do; agents define who does it — separating orchestration from expertise lets you change either without touching the other
• Agentless vs Autonomous: When Simple Beats Complex — Simple two-phase workflows often outperform complex autonomous agents — empirical evidence for starting with constrained approaches rather than maximizing AI autonomy
• Agent Terminology Disambiguation for AI Coding Systems — Eight overlapping terms — workflow, autonomous agent, AI assistant, RAG pipeline, workflow engine — describe distinct systems with distinct failure modes; matching the term to what you are building changes which patterns apply
• Classical SE Patterns as Agent Design Analogues — Classical GoF patterns and SOLID principles have direct structural analogues in agent systems, with the concern shift moving from reuse to control and safety
• Coding Agent Scope Expansion: When to Extend Beyond the Codebase — Extending a coding agent into browser, ops, and knowledge-work only works when the coding scaffold — loop, verification, eval, credential boundary — transfers into the new domain
• Applying Coding Agents to Non-Code Tasks — A coding agent pays off on prose, config, or data work only when you can name a command that fails whenever the artifact is wrong
• Minimum-Sufficient Execution: Estimate Scope Before Spending Budget — Estimate a task's scope, run the smallest path that passes verification, and expand only on failure — with the ACRR metric and E3 loop, under the conditions where it pays off
• Cognitive Reasoning vs Execution: A Two-Layer Agent Architecture — Separate the agent layer that decides what to do from the layer that acts — typed tool interfaces enforce the boundary and make each independently testable
• CoALA Decision-Making Loop as an Orchestration Lens — The propose -> evaluate -> select -> act loop from CoALA, used as a vocabulary for locating where each orchestration tactic improves an agent — not a prescription for runtime structure
• CoALA Structured Action Space: Internal vs External Actions — Split the agent's actions into internal (reason, retrieve, learn) and external (ground) — the boundary surfaces cost, reversibility, and permission profiles that monolithic tool lists hide
• Discrete Phase Separation — Prevent context contamination by running research, planning, and execution in separate conversations — only distilled artifacts cross phase boundaries
• Domain-Scoped Parallel Exploration for Multi-File Change Localization — Partition a localization agent's exploration along domain seams when a change spans subsystems — context isolation, not parallelism, is the active ingredient
• The Delegation Decision: When to Use an Agent vs Do It Yourself — Agent delegation has overhead; match task characteristics to agent strengths rather than delegating everything or nothing
• Delegation Threshold Calibration for Orchestrator Agents — Calibrate the orchestrator's delegate-or-handle-inline threshold against handoff cost, review tax, and the 15× token multiplier of true multi-agent
• Selective Dynamic Concurrency for Long-Horizon Coding Agents — Enable runtime sub-agent concurrency per task, not by default: it cost 1.41x to 3.31x the tokens in a three-agent study and paid off only with ownable components, comparable attempts, or independent validation
• Empowerment Over Automation — AI tools should skip tedious work while preserving your autonomy over architectural decisions, domain logic, and creative choices
• Eval Strategy by Agent Generation: A Structure-to-Eval Locator — Six structural levels of agent architecture — prompt, chain, ReAct loop, workflow graph, modern loop, harness — each open an eval surface the prior level cannot see
• Execution-First Delegation — Instead of scripting steps, specify the outcome and the boundaries; the agent determines how
• Execution Lineage: DAG of Artifacts vs Agent Loops — Represent revisable AI-native work as a DAG of artifact-producing computations with explicit dependencies, stable boundaries, and identity-based replay — so unrelated edits don't perturb the final output
• Externalization in LLM Agents: Memory, Skills, Protocols, and Harness — Reliable agents externalize cognitive burdens into persistent infrastructure — four components that transform hard internal problems into tractable retrieval and composition tasks
• Agent-Ready Data Architecture for Analytics Agents — Publish grain, join semantics, authoritative markers, and completeness windows in the warehouse itself; a schema-only change moved text-to-SQL from 64.5% to 90%
• Governed Sources of Truth for Analytics Agents (Structure Over Access) — Layer semantic models, lineage, and skill routers between an analytics agent and the warehouse; raw corpus access alone moved Anthropic's accuracy by less than a point
• Inversion Analysis: Surface Capabilities Competitors Cannot Replicate — Inversion asks what your architecture enables that others cannot replicate, producing novel integrations rather than feature parity
• Open Agent School Pattern Mapping — Map the Open Agent School academic pattern taxonomy to practical coding-agent primitives like maxTurns, PreToolUse hooks, and CLAUDE.md memory
• Persona-as-Code: Defining Agent Roles as Structured Documents — Encode each agent's domain, responsibilities, constraints, output artifacts, and scope exclusions as a Markdown file so roles are explicit, auditable, and composable
• ReAct (Reason + Act): Interleaved Reasoning-Action Loops — Interleave a thought, a tool call, and an observation each step, re-conditioning on real evidence — pays back on sparse-feedback tool-grounded tasks, wastes inference on predictable workflows
• Role Orchestration on a Single Model — Invoke the same frozen small model in three distinct roles — summariser, agent, corrector — to roughly double task-goal completion on constrained hardware without additional training
• Graph of Thoughts: Directed Graph Reasoning for Multi-Path Problems — Model reasoning as a directed graph to aggregate insights across independent paths — the Aggregate operation that neither CoT nor Tree of Thoughts can express
• Petri Net of Thoughts: Formal Process Models as Prompting Scaffolds — Use Petri net formalism to derive reasoning structure from process evidence, giving each LLM call a state-aware prompt constrained by formally defined transitions
• Self-Discover Reasoning: LLM-Composed Reasoning Structures — Enable the model to compose a task-specific reasoning plan from a library of atomic modules before solving, outperforming fixed strategies like Chain-of-Thought on complex analytical tasks
• Runtime Scaffold Evolution — Agents synthesize, modify, and deploy custom tools during active problem-solving rather than relying on a fixed toolkit
• Layered Domain Architecture — Pin one intra-domain layer order (Types → Config → Repo → Service → Runtime → UI) with downward-only dependencies enforced by a linter so agents place new code in the same slot every session
• Model-Neutral Agent Architecture: Model Portability Over Cloud Portability — A conditional architectural bet — model portability pays back faster than cloud portability when cross-vendor capability churns quarterly and the team has portable evals; inverts on single-CLI platforms and workloads bound to vendor-native features
• Separation of Knowledge and Execution — Structure agent systems in three layers — skills (knowledge), agents (execution), and commands (orchestration) — so each layer changes independently
• Solver-Externalized Constraint Reasoning (MaxSAT/SMT Encoding) — Have the agent emit solver code for z3, python-sat, or OR-Tools instead of reasoning through constraints in prose — then verify the solver's output against the original intent
• Structured Agentic Software Engineering (SASE) — A framework for transitioning from AI-augmented to goal-directed agentic SE, with structured artifacts (MRPs, CRPs, BriefingScript) that close the speed-vs-trust gap
• Task-Specific Agents vs Role-Based Agents — Build agents for specific tasks rather than generic roles — narrow scope produces more precise output and reduces context confusion
• Three Reasoning Spaces: Plan, Bead, and Code — Treat plan space, bead space, and code space as explicit gates — transitioning between them deliberately prevents architecture drift during implementation

Memory & State

How agents persist, retrieve, and synthesize information across turns and sessions.

• Memory Retrieval as a Control Decision — Treat memory injection as a control decision — abstain, gate before action, or utility-rank retrieved memory rather than always injecting top-k
• Agent Memory Patterns: Learning Across Conversations — Persist knowledge across conversations using scoped memory systems so agents accumulate institutional knowledge rather than starting fresh every session
• Code-Native Memory Substrates — Root agent memory in codebase artifacts — typed VCS units, AST diffs, and git-backed task graphs — so structure replaces lossy natural-language summaries
• Git-Bound Memory for the Agentic Development Lifecycle — Bind agent memory to version control so commits, merges, and review supply the ground truth, freshness, and verification bespoke memory stores have to rebuild
• Executable Memory: User-State as Code for Personalized Agents — Compile user memory into typed code with rule functions — beats retrieval 99% vs 6-43% on aggregate queries, loses to text retrieval on plain recall
• Compiled Specialist Agents: Muscle Memory for Recurring Intent — Compile a recurring request pattern into a purpose-built specialist agent rather than retrieving memories into a general one — pays off only where the pattern repeats and routing accuracy is measured
• AX/UX/DX Triad: Three Experience Layers in Agent Systems — Treat Agent Experience, User Experience, and Developer Experience as separate design surfaces — conflating them degrades all three
• CoALA Memory Taxonomy as a Classifier for Harness Artifacts — Apply CoALA's four memory types as a classifier over the artifacts an agent harness already has; the value is diagnostic, not prescriptive
• Durable Interactive Artifacts: Agent Output Outside the Transcript — Treat agent outputs as persistent, re-openable, re-runnable workspace objects separate from the chat transcript — the transcript holds reasoning, the artifact holds state
• Interactive Canvases: Agent-Generated Visual Artifacts as Outputs — Canvases are an output-shape choice, not a new pattern: they earn their overhead only on irreducibly multi-dimensional output, and they change what review means
• Canvas as Durable Workflow State: The Four-Step Blueprint and What It Costs — Build a canvas as the persistent state surface for a repeated workflow only when repetition amortizes the build, you engineer the persistence, and the approval points survive a distracted reviewer
• Episodic Memory Retrieval — Retrieve relevant past interaction episodes — not isolated facts — so agents recall what was tried, what failed, and what worked when facing similar problems
• Generative Agents Memory Stream — Three-layer architecture (observation stream, scored retrieval, reflection synthesis) for maintaining coherent behavior across long-running, high-observation-density agent sessions
• Handoff Skill: Structured Context Transfer Between Agent Sessions — A model-invocable skill that compacts the current session into a temp-file handoff document a fresh agent can pick up — invoked at an explicit transfer point, distinct from harness-detected recap and from raw-transcript forwarding
• Layered Mutability: Governing Persistent Self-Modifying Agents — A five-layer lens (pretraining, alignment, self-narrative, memory, weight-level adaptation) for deciding where governance attaches in persistent agents and when compositional drift will ratchet past a visible-layer rollback
• Structured Task-State Ledger for Tool-Calling Agents (LedgerAgent) — Maintain task state as a typed dictionary outside the prompt and gate write tools against executable policy predicates — pays back on multi-turn, policy-bound, write-heavy workflows where pass^k reliability matters
• Execution-State Ledger for Long-Horizon Coding Agents — Track which observations still describe the repository in a deterministic ledger, then apply it at two boundaries: inform the model before it acts, and screen its command before it runs
• Epistemic Working Memory for Multi-Hop Reasoning (SLEUTH) — An explicit ledger of confirmed facts, ranked hypotheses, and open questions that drives the next action, so multi-hop reasoning stops degrading as the context window fills
• Knowledge Graphs as Provenance-Carrying Agent Memory — Store memory as typed relations that each record their source document, so an evaluator can check a claim rather than judge it — worth the construction cost only on cross-document joins and citable feedback
• Memory Synthesis from Execution Logs — Extract causal lessons from agent execution traces — what worked, what failed, which approaches were abandoned and why — so every run makes future runs more effective
• RAG over Thinking Traces — For reasoning-intensive tasks, swap the document corpus for prior thinking trajectories — the same retrieve-then-generate pipeline beats both no-RAG and document-RAG with flat or lower inference cost
• Session Initialization Ritual: How Agents Orient Themselves — A mandatory startup sequence that every agent session executes before touching code — verify state, orient to progress, confirm baseline health, then act
• Session Recap: Goal-Shaped Handoff at Context Boundaries — A structured, agent-authored artifact at compaction, resume, or fork boundaries that preserves goal-state rather than text-density — restoring why and what-next across discontinuities
• Memory Transfer Learning: Cross-Domain Memory Reuse — How coding agents transfer learned memories across different task domains, why abstraction level determines transferability, and when cross-domain memory causes negative transfer
• Scope-Matched Retrieval for Persisted Agent Skills — Retrieve a persisted skill only for the task family whose evidence certified it; holding the gate decisions fixed and changing only retrieval scope took harmful deployments from six of eight to none
• Subtask-Level Memory for Software Engineering Agents — Store and retrieve memory at individual reasoning stages, not whole sessions, to prevent misguided retrieval when tasks share surface similarity
• Tiered Memory Architecture: Episodic-to-Semantic Consolidation — Separate raw episode storage from a curated semantic tier and promote facts between them only on observed re-use — pays off for long operation windows, adds cost without benefit elsewhere
• Cue-Anchored Working Memory (Delivery, Not Storage) — A second memory tier the harness delivers by injecting operational facts deterministically when a context cue fires, so the agent never has to remember to look them up
• Wiki Memory: Agent-Maintained Compressed Knowledge Base — Run an agent over raw sources to build and maintain a compact, file-based knowledge layer future agents read instead of re-deriving structure via query-time RAG — for durable domain knowledge with enough query volume to amortize
• Self-Correcting Memory: Evidence-Backed Claim Repair — Bind each stored claim to a versioned line range so a deterministic diff marks it stale before any model judges it — the repair half of wiki memory, and only where the source of truth is versioned
• Auto-Merging a Wiki Agent's Documentation Pull Requests — One named deployment merges its doc agent's nightly pull requests without review, which keeps the wiki current and removes the only step that compared the generated text against the code
• Typed Memory Provenance and Assertion Release Gating — Record each stored claim's role, validity window, and disclosure scope, then check candidate output against it before release — tagging origin alone stopped 1 unsafe release in 19
• Organizing Filesystem Agent Memory for Retrieval Cost — Organizing a markdown memory tree roughly halves per-query retrieval cost on large stores while leaving answer quality unchanged — a cost optimization with a size threshold
• Persistent Teammate Workspace: Durable State for Agent Teams — A per-teammate directory on disk that survives compaction and process exit — worth its token and security cost only when the team's work genuinely outlives one session
• Project-Scoped Agent Workspace: Durable Context for Clean-Context Subagents — A synced project file set is the only durable channel between subagents that start with clean context, so its write policy decides the quality of every future dispatch
• Weakest Consistent Learning: What Agent Loops Should Persist — Persist the learning that permits the most future cases while staying consistent with what was observed, under three stated conditions
• Source-Linked Tool Observations: Repairing Stale Context — Store each tool result with the source and span it came from so a later write can repair the context block, because a warning parked beside stale content does not displace it

Control & Orchestration

Patterns for steering agent behavior, detecting convergence, and managing execution flow.

• Agent as Tool vs Handoff: Who Keeps the Conversation — Registering a specialist as a callable tool keeps the parent in control with a clean context window, while a handoff transfers the conversation to the specialist
• Subagent vs In-Context Skill Execution — Invoke a skill package as a subagent when it declares an input-output contract, and load it in-context when it does not; the measured result reverses on contract-less packages
• Background Todo Agent — Route the agent's todo-list maintenance loop to a small background model so the frontier model spends its attention budget on the active sub-task instead of bookkeeping
• In-Agent Task Prioritization: Ranking the Next Action — Rank pending work by a composite score (urgency, value, dependency, blast radius, staleness) — distinct from routing and scheduling — so the agent's scarce attention lands on the item that pays back most per turn
• Controlling Agent Output: Concise Answers, Not Essays — Matching the agent's response format to what you actually need reduces noise and preserves context budget
• Critic Agent Pattern — A second model reviews the primary agent's plan before execution begins, catching structural errors early when recovery is cheap
• Cross-Session Peer Messaging with a Posture-Keyed Inbox Gate — Two independently steered sessions exchange plain-text messages through per-session inboxes, with the receiving side gating delivery on permission posture rather than sender identity
• Delegation Scoping: Session Relationship and Workspace — Scoping delegated work takes two independent choices: whether the task belongs to the current session's deliverable, and whether it needs repository files
• Deterministic Orchestration for Structured Modernization — When the workflow shape is stable, encoding orchestration in code reserves the LLM for translation choices — comparable accuracy at up to 3.5x lower token cost
• Decision Unbundling: What Moves Into the Orchestrator — Moving a routing or classification call onto a decision model shrinks the model call and grows the harness, which now owns the candidate list, the threshold, the escalation route, and the durability
• LLM-as-Code Agentic Programming for Agent Harnesses — When the workflow shape is enumerable, the program holds control flow and the LLM is a callable component; context scales with call-tree depth instead of accumulated steps
• Bounded Agent Steps Inside a Deterministic Workflow — Keep the agentic loop but run it as one fenced workflow stage — pre-set goal, tool allowlist, turn cap, typed output — and only where the stage boundary is a durable domain seam
• Canvas as Control Surface: Steering a Long-Running Agent Mid-Run — A canvas the agent writes to while it works re-encodes the run as inspectable state, but only pays off on recurring workflows with few, well-placed approval points
• Artifact-Driven Workflow Compilation for Agent Execution — Make every workflow step declare the artifacts it reads and writes, so the executor looks up its inputs instead of inferring them — worth it for fixed procedures you run many times
• AST-Grounded Critic Loop for Documentation Maintenance — Combine AST-anchored retrieval with a critic-guided Reflexion loop, treating doc-vs-code consistency as a structural verification problem rather than a text-generation problem
• DSPy: Programmatic Prompt Optimization — Replace hand-written prompts with optimizable modules: define a metric, supply training examples, and an optimizer searches prompt and few-shot space automatically for compound pipelines
• GEPA: Reflective Prompt Evolution with Pareto Selection — Evolve prompts by reflecting on execution traces in natural language and keeping the Pareto frontier of candidates per-instance — tuned for rich textual feedback and sample-efficient gains over MIPROv2 and RL
• Evaluator-Optimizer Pattern — Two distinct LLM roles in a loop: a generator produces output and an evaluator critiques it, feeding structured feedback back until a quality threshold is met
• Event-Driven Agent Routing — Route work between agents and human teams by reacting to status-change events rather than maintaining a central coordinator that owns the full workflow
• Goal Monitoring and Progress Tracking — Planning tells the agent what to do; monitoring tells you whether it actually did it and whether it wandered off
• Grill Me: Developer-Initiated Plan Interrogation — Direct the agent to challenge your plan rather than execute it, surfacing hidden assumptions and decision gaps before implementation begins
• Classifier-Subagent Run Mode: Per-Call Permission Routing — A three-tier router — allowlist, sandbox, classifier subagent — for shell, MCP, and fetch calls, with project-specific permissions.json custom instructions steering the classifier's allow/refactor/escalate verdict across Cursor, Claude Code, and Codex
• Inference-Time Tool-Call Reviewer — A reviewer agent inspects each provisional tool call before dispatch, gated by Helpfulness-Harmfulness metrics that quantify when feedback adds net value
• Interactive Clarification for Underspecified Tasks — Agents that explore the codebase first and ask targeted clarification questions recover up to 74% of the performance lost to underspecified inputs
• Issue Requirements Preprocessing — Transforming raw issue descriptions into structured requirements before code generation improves patch resolution rates by 17% on average
• Splitting the Drift Judge from the Advisor (LivePlan) — Separate the component that decides whether an agent has drifted from the component that decides what to do about it, so the always-on half runs without an LLM call
• Watcher Side Agents: Give the Second Agent a Trigger It Did Not Pick — A concurrent, read-only agent that writes notes to the operator earns its cost on long sessions when a rule fires it, because a watcher asked continuously whether anything is wrong finds something
• Minimum-Sufficient Control Ladder: Escalate by Failure Mode — An ordered algorithm for adding agent control mechanisms — Tool Use, Reflection, Evaluator-Optimizer, Human-in-the-Loop, Parallelization — only when a named failure mode justifies the next rung
• The Orchestrator's Attention Budget: Delegating to Protect Context — Justify a sub-agent by what it keeps out of the orchestrator's context rather than by time saved, since imported worker output charges attention on every remaining turn
• Plan Compliance in Agents: Measure What They Execute, Not What You Wrote — Agents silently deviate from instructed plans; plan quality, phase alignment, and periodic reminders determine whether the plan you wrote actually runs
• Plan as Control Flow: Enforce the Plan in the Harness — For long, structured tasks, make the harness dispatch the plan one unit at a time; skip it for short or exploratory work
• PR-Subscribed Agent Ownership — An agent subscribes to the pull request it opened and wakes on CI and review events until the checks pass, viable only with a real oracle, data-plane-only authority, a retry budget, and per-agent isolation
• Proactive Idle-Time Anticipation (ProAct) — Predict likely next user needs from dialogue history plus persistent memory during the idle window between turns, then prefetch evidence under a value gate — pays back only when need chains are predictable and push fatigue is tolerable
• Progressive Disclosure for Agent Definitions — Keep agent definitions minimal — identity and scope only — and load detailed task knowledge on demand through skills rather than front-loading everything
• Three Skill Disclosure Decisions to Move Off the Model — Progressive disclosure leaves skill selection and tool gating to the model and fixes skill freshness at thread start; a skill file or the app holds each answer
• Prompted Uncertainty Decomposition for Clarification Routing — Elicit action confidence and request uncertainty as two separate prompted scalars so a black-box agent asks the user only when ambiguity lives in the spec
• Self-Reporting Loops: Autonomous Routines That File Their Own Backlog — Scheduled and autonomous runs file out-of-scope observations to the tracker so signal survives the session boundary, contingent on a trusted substrate, deduplication, and severity routing
• Specialized Agent Roles — Assign distinct specializations to parallel agents so they complement rather than compete on the same problems
• Sprint Contracts — A pre-coding agreement between planner, generator, and evaluator agents that converts vague goals into graded scoring dimensions before implementation begins — preventing evaluator rationalization
• Steering Running Agents: Mid-Run Redirection and Follow-Ups — Send a mid-execution message that redirects tool calls without discarding the context already built
• Tool Preamble: User-Visible Status Updates Before Tool Calls — A short visible message before tool execution in multi-step agent runs reduces perceived latency without altering behavior; apply at phase boundaries, not per call
• Agent Status Signals as Inference — A status signal is generated off the run it describes, so it may claim only what its own inputs settle: the inbound message, or the clock
• Verification-Gated Agent Autonomy via Automated Review — Pair broader agent autonomy with an automated review gate that screens output — trust scales through verification rather than per-action human approval, under specific conditions
• Capability Declarations for Agents That Act on Data — Declare permissions, an owner, live-state preconditions, and a reversibility class per action, so retrieved text can inform what an agent proposes without ever authorizing it
• Deterministic Precondition Gates for Tool-Using Agents — A read-only deterministic predicate checks a proposed tool call against current state and blocks a forbidden write before it silently corrupts state — the deterministic counterpart to a probabilistic reviewer
• Enforcement Modes in Spec-First Agent Frameworks — Sort a framework's controls by whether the agent can edit them: persuasion, front-loaded structure, or code the agent runs inside — and check what that ordering has actually measured
• Schema-Checked Rule Extraction for Tool-Call Gates — Check each LLM-extracted tool-call rule against your tool schemas before the gate enforces it, because a rule that blocks its own precondition tool looks like a working control
• Specification Authority Boundary: Agents Propose, the Runtime Commits — Sort each requirement into blocking or advisory before you enforce anything, then let only validated evidence write the state that says a requirement is met
• Learning Execution Guardrails from Agent Failure Traces — Mine reviewed anomalous runs into conditional When/DoNot/Unless/Instead rules and route them per instruction; abnormal execution fell 69.0% to 26.7% while over-refusal rose from zero to 19.3%
• Protecting the Test Oracle From the Agent — The agent changing an implementation must not own the artifacts that define correctness, and that set covers snapshots, compatibility baselines, and CI waiver labels as well as test files
• Informed Abstention as a Tool-Boundary Runtime Gate — Three pre-execution checks catch a missing required field, an unconfirmable state, and an absent approval, then route to clarification, a bounded re-read, or human handoff
• Delegated-Autonomy Boundary Artifacts (AJR and ADP) — Write an agent's authority boundary down as two reviewable requirements artifacts — an Agency Justification Record and an Agentic Delegation Policy — instead of leaving it implicit in prompts and tool schemas
• Trigger-Level Gating for Autonomous Agent Intake — Three gates stand between an incoming issue and a changed one — trigger, tool, and approval — and only the first two carry authority; confidence-based approval queues govern attention, not permission
• Approval Gate Granularity in Agent Pipelines — Bundling agent approvals cuts reviewer load only on the axis that removes decisions; batching the approval queue moves wait time and leaves the count unchanged
• One-Shot Permission Answers for Out-of-Directory Reads — Claude Code's allow-and-ask-again answer in auto mode clears one read outside the working directories, while the saved alternative on the same prompt clears every later one too
• Binding an Agent's Effect to the Approval It Claims — Attach the task, revision, and reviewed object set to each agent-initiated effect, because a per-call permission check passes on the send to a recipient the user just removed
• Fleet-Level Irreversibility Budgets for Agent Effects — One shared account of residual risk, charged before each effect commits, catches the overdraw that individually compliant agents create together — and the fleet size below which it buys nothing
• Agent-Native Filesystems: Gating Effects, Not Commands — Attach permission rules to filesystem paths instead of command strings, stage every mutation until the user commits, and keep the undo inside the filesystem
• Model-Directed Subagent Tiering: Lead Model Picks the Tier — Let the lead agent pick subagent kind, tier, effort and reuse at each dispatch instead of a router or one fixed sidekick; the evidence is vendor-run and depends on cache lifetime
• Provider-Hosted Subagent Delegation: One Model Price for the Whole Tree — Hosted delegation lets the model spawn subagents inside one API request, but every agent inherits the request's model and price, so size that model to the hardest subtask in the tree
• Verdict-Driven Inspection Cadence for Agent Monitors (Cadence) — Set a runtime monitor's next inspection interval from its own last verdict: widen after advisory guidance, reset to the base after a discarded step

Reliability

Making agents robust — backpressure, idempotency, cost awareness, error recovery, and self-correction.

• Administrative Effort Ceilings for Reasoning Budget — An admin-set maxEffortLevel bounds every operator effort control; caps compose by minimum across settings scopes, and the clamp is silent in headless and background runs
• Agent Circuit Breaker — Wrap external tools with per-tool failure-tracking state machines that block calls during degraded states, preventing token waste on retry loops
• The Agent Escalation Ladder Priced in Energy and Latency — On self-hosted open-weight models one agent costs 1.20x a direct call's energy, two cost 3.06x and four cost 6.36x, while accuracy stays flat on four of five tasks
• Agent-Client Admission Control for Agentic Traffic — Retry budgets, honored 429s, and token-denominated buckets let an agent client throttle itself inside the gap between a burst and the autoscaler's response, for traffic you own and with enforcement still on the server
• Caller-Actionable Error Steps in Tool Responses — A next step the caller cannot run cost 37 points of recovery on expired credentials; write the repair as a call to one of your own tools, and only where the failure is invisible in the agent's own call
• Retrieval Tools That Refuse: Making a Search Miss Visible — A search tool that returns top passages hides a miss; a one-line refusal raised abstention from 24% to 84% for Qwen3 and Haiku 4.5 with an oracle trigger, and almost nothing for Sonnet 5.5 and Opus 5.5
• Retry, Switch, or Abstain: Supplying a Tool-Recovery Policy at Runtime — Hand the agent a fallback map and recovery constraints in context so it retries, switches tools, or stops — worth up to 16.8 points of robustness, and only where failures are observable
• Self-Healing Production Agent — A closed-loop pipeline that detects post-deploy regressions, triages causality, and dispatches a sub-agent to open a fix PR — with human review at the merge gate
• The Advisor Strategy: Frontier Model as Strategic Advisor — Pair a cost-effective executor model with a frontier advisor that provides strategic guidance on hard decisions — within a single API call, no orchestration required
• Agent Backpressure: Automated Feedback for Self-Correction — Automated tooling — type systems, test suites, linters, CI pipelines — creates feedback loops that agents use to self-correct without human intervention
• Auto Model Selection: Harness-Driven Routing per Task — Vendor-side Auto modes let the harness pick the model per request based on availability, policy, and plan — useful for executor-class work, costly for long sessions and eval-gated CI
• Calibrated Deciders for In-Loop Agent Decisions — Moving a routing or risk-gating decision onto a smaller model pays only when its confidence ranks its own correct answers above its wrong ones, and that ranking has to be measured on your traffic
• Copilot Auto Tiers: Weighting Cost Against Quality — The efficiency, balance, and intelligence tiers reweight which model auto prefers per prompt; they set no price, no quality floor, and no org policy
• Behavioral Drivers of Coding Agent Success and Failure — Why agents fail the tasks humans call easy, and the trajectory structure that predicts success — derived from trajectory analysis of 19 agents across 8 frameworks and 14 LLMs
• Comparison-Only Advisor: Steering a Large Actor With a Tiny Comparator — A 0.5B comparator ranks the actor's proposed action against sampled alternatives and hands back the winner as non-binding advice, conditional on an environment you can rewind
• Cross-Vendor Competitive Routing — Assign competing vendor agents to the same task, collect independent results, and let a human or automated gate select the winner
• Decision-Fork Replay: Grading an Agent's Mid-Run Choices — Replay a branch point in a recorded run and let the outcomes label the better direction; the best of 14 models scored 59.7%, and on the two models ablated, extra reasoning effort did not help
• Dispatch-Time Reasoning Level for Delegated Agents — Choosing a delegated agent's reasoning level at hand-off is a one-shot commitment, so select on blast radius, reversibility, and detection cost rather than estimated difficulty
• Difficulty-Aware Topology Selection for Coding Agents — The gain from agent collaboration runs from 2.4 to 21.1 pass@1 points across difficulty tertiles at a flat tenfold price, so the topology decision belongs to the task rather than to the deployment
• Evidence-Path Divergence for Multi-Agent Fault Tolerance — A redundant delivery route buys fault tolerance only downstream of where the evidence it carries is produced, and only while the fault misses the other route
• Decoupled Search Grounding: A Vendor-Agnostic Grounding Boundary — Lift retrieval out of the reasoning model and into an MCP-compatible gateway so provider, caching, and evidence rendering become independently tunable controls — pays off only when strict output contracts, cacheable query mix, and real multi-vendor routing all hold
• Deterministic Fast Paths: Answer Without a Model Call — Answer the easy majority of retrieval questions from a score margin, a cache hit, or a dictionary lookup, once the threshold is calibrated against your own labeled data and a guard catches silent false hits
• Field-Change Intent Instead of Model-Written Diffs — Have the agent emit which resource, field, and value to change, then let a deterministic span editor apply it, because tolerant patchers misapply about 1 in 7
• Dual-Budget Control for Search Agents — Under hard limits on both tool calls and generated tokens, score each candidate action by Value-of-Information per unit budget and spend the next unit on the highest-ranking action
• Effective Feedback Compute (EFC) for Harness Comparison — A trace-level scaling coordinate that credits feedback only when it is informative, valid, non-redundant, and retained — replacing raw tokens or tool calls when comparing two harnesses on the same multi-turn task
• Effort as a Verification Dial: What a Higher Level Buys — Effort buys self-verification and edge-case testing, so raise it where hidden cases decide the result and keep it low where the risk is a misread requirement
• Harness-Enforced Stopping on Judged-Useless Tool Results — Agents call a failing source's results useless 97-100% of the time and keep querying it; count the run in harness code and remove every action but finish
• Escalation Channels: A Reporting Tool Instead of a Reward Hack — Give the agent a structured tool for reporting broken test infrastructure at the point of conflict, paired with a written prohibition on shortcuts — either half alone raised hacking for one model
• Evidence-First Reports From Failure-Diagnosis Agents — Engineers named the Evidence section one of the most valuable parts of an agent's failure report whether or not they accepted its root cause; keep the verdict, but put the retrieved logs above it
• Gateway Model Routing — Treat an Anthropic-compatible gateway as both inference target and model catalog, so a single config knob controls what the harness can call and what it shows in the picker
• Gateway Hint Headers for Routing and Budgeting Agent Calls — Have the client declare request class, subagent type, tool durations, and compaction state in headers, so a gateway routes and attributes without reading the prompt
• Exactly-Once Enforcement Layer: Model, Harness or Contract — Which layer must enforce exactly-once depends on whether a read-back can reveal what happened; frontier models handle a lost acknowledgement, and an in-flight request needs the write contract unless the service documents a short in-flight bound to wait out
• Exception Handling and Recovery Patterns — Agents fail; the question is whether they fail forward (recover and continue) or fail catastrophically (corrupt state, lose progress, repeat work)
• Agent Failure Trajectories and the Recovery Window — Failure has an onset, an evolution, and a narrow recovery window — a decisive error at median step 7 and a one-step window before lock-in mean validation must move earlier than final-outcome checks
• Feedback as Capability Equalizer — Weaker models with high-quality iterative feedback outperform stronger models without it — invest in feedback loop quality before upgrading the model
• Five-Failure-Layers Diagnostic — Before swapping models, force every observed agent failure through a fixed harness-layer attribution — task spec, context, execution environment, verification, state — so "the model is dumb" resolves to a specific gap
• Heuristic-Based Effort Scaling in Agent Prompts — Encode resource allocation rules in system prompts so agents spend proportional effort — few tool calls for simple lookups, many subagents for complex research
• Idempotent Agent Operations: Safe to Retry — Design agent operations so that running the same task twice produces the same end state — not duplicate artifacts, conflicting state, or compounded errors
• Tool Operability: Interfaces That Survive a Lost Response — A lost response leaves committed and uncommitted state indistinguishable to an agent; lifecycle identity, durable state, and declared effect semantics are what let it continue
• Interactive Effort Sliders: Per-Turn Reasoning-Budget Controls — Expose reasoning budget as an interactive, per-turn operator control — the third option alongside static effort config and heuristic effort scaling
• Long-Running Agents: Durability, Checkpoints, and Resumability — The operational shape of agents that work for hours, days, or weeks — three walls (finite context, no persistent state, unreliable self-grading) and the five primitives that recur across Anthropic, Cursor, and Google designs
• Remote Agent Host Sessions over SSH and Dev Tunnels — Run the agent loop on a remote host whose lifecycle is decoupled from the client editor — SSH attach, dev tunnel reverse, or cloud worker — so sessions survive laptop sleep, network drops, and editor restarts
• Multi-Client Session Attachment: One Session, Many Clients — Several front ends attach to one live agent session at once; the host holds the only authoritative state, and client-contributed tools stay behind
• Per-User Supervisor Process for Background Agent Sessions — A managed-daemon model where a per-user supervisor spawns each background agent session as a detached process, reconnects via on-disk roster, evicts idle non-pinned sessions, and restarts in place onto an updated binary
• Outcome Monitors: Recovery Affordances for Tool Failures — A silent tool failure raises no error; an advisory receipt naming the broken property and the substitute tools the agent can call is what recovers the run
• Observation Contract Preservation — Tool outputs like presigned URLs and session tokens are contract-bound — preserve their bytes and respect their expiry on the second call, or the chain fails silently
• Per-Call Budget Hints on Tool Invocations — Lift the reasoning or returned-token cap on individual tool calls — narrowly, when the call is infrequent and information-dense — instead of re-tuning the global default
• Per-Tool Extended Reasoning Opt-In: Tool-Call-Scoped Budgets — A single tool call opts itself into deeper reasoning via a per-call parameter, leaving the turn's global reasoning effort unchanged for every other step
• Per-Run Budget Reservation for Coding Agent Model Calls — Reserve the worst-case cost of a model call before making it and refund the unused hold after; the run halts before a request it cannot afford, not after
• Planning Stage Preconditions: Budget Headroom and Task Text — A planning stage draws from the same token ledger as execution, so it earns its place only when the run has headroom above execution's own use and the planner can read the task issue
• Progressive Spend Threshold Alerting for Agent Cost Governance — Pair a soft cost cap with progressive alerts at fixed budget percentages so operators get graceful intervention windows before agent work hits a hard cutoff
• Reasoning Budget Allocation: The Reasoning Sandwich — Allocate maximum reasoning compute to planning and verification phases, reduced compute to execution — rather than using a fixed level throughout
• Reasoning Effort Over Tool Scaffolding for First-Try Reliability — On a fixed budget for a single-shot agentic build, raising reasoning effort buys first-try functional reliability more cost-effectively than adding testing tools or design prompts — under three explicit conditions
• Residual Completion for Stateful Agent Handoffs (CFRC) — Freeze a contract from the progress a predecessor model already committed and let the successor finish only the remainder, for handoffs where a restart would repeat irreversible effects
• Recoverability-Gated Self-Evolution for Agent Harnesses — Admit a self-modification to an agent harness only after its undo has been verified against states other than the one the mutation was written in
• Refilling Tool Budgets: Rationing by Rate, Not Session Cap — Ration an expensive tool by a configurable refill rate instead of a hard per-session cap, and surface the balance to the agent so arriving capacity actually gets spent
• Requirement-Keyed Checkpoints for Shifting Requirements — Tag each checkpoint with the requirement it satisfied and resume from the last compatible one only when a change withdraws an earlier requirement; otherwise continue from head or restart
• Rollback-First Design: Every Agent Action Should Be Reversible — Before choosing how an agent will perform an action, choose how you will undo it — if recovery costs more than one command, reconsider the approach
• Router-Imposed Quality Ceiling: Committing Before Output — A router that picks a specialist model before any candidate output exists caps quality at that model's best answer; the bound holds for every single-commit router, and tuning the specialists never lifts it
• Routing Break-Even: When a Cheaper Model Actually Pays — Divide the router's judging cost by the price gap between the two models to get the offload fraction routing must clear; when the models sit close in price, that figure passes 100% and routing cannot pay
• Running an Agent's Context Step on a Smaller Model — Settle the compression policy on the frontier model first, then move the summarize, compress and extract calls to a small model; price the step you are replacing rather than the vendor's cross-task average
• Per-Surface Routing Visibility in Copilot (HydraFusion) — HydraFusion runs in Copilot CLI, Visual Studio Code, and the Copilot app; the docs say you can follow each step while it works, and only the CLI keeps a record you can export and grade
• Cache-Safe Routing Boundaries: Where a Router May Act — A model switch only avoids paying for the conversation twice at a session's first prompt, in a fresh side lane, or at a subagent launch; anywhere else the cheaper stretch must last tens of requests to repay two cold cache writes
• Task-Distribution Router Fit: Where the Decision Belongs — A router's accuracy tracks how well its tier criteria match your real task mix, so measure the mix before picking tiers; LangChain's 64% median cost cut came from a frontier-only control, which tests routing rather than placement
• Per-Stage Model Assignment in a Generate-Repair-Refine Pipeline — Profile each candidate model separately at generation, repair, and refinement, because the best generator is often not the best repairer; the correctness margin narrows to six ties once the pool is current, and survives on hard tasks, low-resource languages, and performance refinement
• RubricRefine: Pre-Execution Rubric Refinement for Code-Mode Tool Use — Generate a task- and registry-specific rubric, score candidate tool-use code against explicit contract checks, and repair failures before any execution — for multi-step tool sequences where contract violations run silently to completion
• Scout-Then-Route: Verify the Handoff Before Routing — Send a cheap scout into the repository, sandbox-verify its reproduction claims and strip what fails, then pick the fixer on price rather than skill
• Selective Checkpoint Restore Across Code and Conversation State — Restore the agent's mental model and re-edit, restore the edits and re-plan, or reset both — when code and conversation are stored separately, the choice is three actions, not one
• Selective Revalidation for Pending Agent Actions — Record the executable conditions that justified a pending action and recheck only the ones a state change can reach, so a version conflict stops being treated as a decision conflict
• Specialized Small Language Models as Agent Sub-Tools — Hide a small fine-tuned model behind a tool-call interface so a large orchestrator can offload high-volume narrow operations — search, exploration, terminal output filtering — without spending its own context budget
• Tail Control for Agent Workflows: Engineering for the Failure Tail, Not the Average — Engineer non-deterministic agent workflows for worst-case usability — per-step p95 timeouts, hedged re-draws, graceful degradation — because the failure tail sets the reliability behind an API, not the median run
• Task Feasibility Awareness: Stop Before You Start — An up-front check that the current tools can satisfy a task at all, halting the infeasible before the agent burns a long chain — the inverse of premature completion
• Tenant Model Policy: Organization-Scoped Rules for AI Model Selection — An admin-tier policy plane that decides which AI models an org or tenant can invoke — only safe when policy is strict-priority, denials are explicit, and rules age with model deprecations
• Trajectory-Conditioned Model Escalation (SWE-Router) — Run a cheap model for a few exploratory turns, then read its partial trajectory to decide whether to escalate to a stronger model — the routing signal for agentic tasks that lack binary test feedback
• Wink: Classifying and Auto-Correcting Coding Agent Misbehaviors — An async trajectory-observer system that classifies misbehaviors into three categories and injects targeted course-corrections
• WIP=1 and Little's Law: Kanban Throughput Theory for Agent Task Design — Cap an agent's active task count at one until verification passes; Little's Law makes the cycle-time consequence quantitative

Harness & Tools

The runtime infrastructure that hosts and constrains agent execution.

• Agent Composition Patterns: Chains, Fan-Out, Pipelines, Supervisors — Multi-agent workflows follow four structural patterns — sequential chains, parallel fan-out, staged pipelines, and supervisor-coordinator — each suited to different task structures
• Choosing an Integration Layer for an Embedded Agent Harness — Adopting an open harness means picking a one-shot process, an SDK, or an app-server protocol, and that rung fixes which side of the split holds conversation state, tools, and approvals
• Running Several Coding Agents Behind One Harness Interface — An adapter layer turns an agent swap into a dependency change, and the published per-adapter capability table is what tells you which capabilities do not survive the common interface
• Production Hosting Topology for Self-Hosted Agent SDK Runtimes — Pick a container-lifecycle pattern, autoscale on token rate, mediate credentials through a sidecar proxy, and route long-running sessions by consistent hashing so a self-hosted Agent SDK survives real concurrency, multi-tenancy, and prompt injection
• Cloud-Agent Three-Layer State Decoupling — Split a cloud agent's state across agent loop, machine state, and conversation state so pods, sessions, and threads each migrate, hibernate, and recover independently
• Continuation Authority in Agent Migration — The right to act as a migrated agent is held by the deployment boundary, not by the checkpoint, so copying an agent's memory produces a replica rather than a successor
• Dual-Write Append-Mirror for Agent Transcript Externalization — Write the agent transcript to local disk first and forward each batch to a remote store as a best-effort mirror — so a store outage degrades the externalization, not the agent
• Agent Harness: Initializer and Coding Agent — Structure long-running agent work as two distinct phases — an initializer that prepares the environment, and a coding agent that picks up reliably from wherever any prior session left off
• Agent Runtime Middleware: Per-Call Interception Pipeline — Compose cross-cutting concerns as ordered pre/post handlers around every model and tool call, with a placement matrix for middleware vs. hooks vs. tool wrappers vs. prompt rules
• Agent Pushback Protocol — Agents evaluate requests at both implementation and requirements level, surface concerns, and wait for explicit confirmation before executing
• Model a Single Agent Turn as Many Inference and Tool-Call Iterations — A single user-facing turn is an iterative sequence of model inference and tool execution steps, not a single round-trip inference call
• Delta Channels: Bounded Checkpoint Storage for Append-Only Agent State — Store only the per-step diff and write a full snapshot every K steps so long-session checkpoint storage stays O(N) instead of O(N²) and resume latency stays bounded
• Sandbox Forking: Branch Agent Runs From a Warm Snapshot — Prepare an environment once, snapshot it, then fork a copy per run — worth doing only when setup is expensive and the children need no unique identity or live connections
• Full Sandbox Snapshots Beat Retry Only for Untracked State — Restoring a whole sandbox and branching from it lost to plain retry 17/25 vs 6/25 on one Terminal-Bench subset, and won 4/5 vs 0/5 only on a task gated by a database outside version control
• Continuously Built Agent Environments — Rebuild the agent environment in the background on a schedule and activate only builds that succeed, so each delegated task starts prepared instead of paying cold start
• Renting a Cloud Agent's Execution Sandbox — Pointing a cloud agent at a third-party sandbox moves tool execution only, so decide it on a named requirement and expect inference and tool output to stay with the vendor
• Deferred Permission Pattern — Use PreToolUse hook defer decisions to pause headless Claude Code sessions at tool calls and resume them after out-of-band human approval
• Deny-Fallback Permissions for Unattended Agent Runs — Resolve the unanswerable permission prompt to deny on a host with no human, keeping the permission mode and rules deciding every other call
• Action-Class Permission Gates Leave Task Fit and Reversibility to the Human — A permission prompt names an action and a target, so whether it fits the task and whether it can be undone are supplied by the person answering it
• Most-Restrictive-Wins Fusion for Parallel Agent Control Returns — The deny > defer > ask > allow merge function that fuses parallel hook decisions, classifier verdicts, and permission rules into a single agent-control answer
• Designing Agent Plugins to Survive Co-Installation — Ship the root manifest, keep client data in extensions, and prefix exported capability names, because conformance is checked on one plugin and the failure needs two
• Product-Operation Tool Surfaces for In-Product Agents — An agent living inside a product enumerates each action as its own shallow tool and withholds the SDK, then scopes the live catalog per thread so tool selection holds up
• Typed Tool Surfaces and Out-of-Loop Correctness Gates — Name the domain's operations as tools, cap the loop in code, and check the answer downstream — a trade that won multi-hop questions by 9 points and lost single-hop ones by 25
• Tool Confirmation Carousel: Batched UI for Per-Call Approvals — A carousel control reviews multiple pending tool calls in one navigable surface instead of scattered modals — useful only for residual approvals that allowlists and sandboxes cannot absorb
• Six-Shape Approval Response Taxonomy — The Claude Agent SDK exposes six distinct responses to a tool-approval prompt (approve, approve with changes, approve and remember, reject, suggest alternative, redirect entirely) composed from three callback knobs over a binary protocol
• Harness Design Dimensions and Archetypes — Five dimensions and five archetypes from a 70-project empirical study — a population-level lens for reading harness choices and predicting where effort is missing
• Computer Use as the Last Integration Tier — Driving a GUI is the bottom rung of the integration ladder; pick it only when no API, MCP server, terminal command, filesystem tool, or browser tool reaches the task, and only for short reversible workflows
• Harness Engineering — The discipline of designing agent environments — layered architecture, mechanical enforcement, legibility — so agents reliably produce correct results
• DSLs as a Constraining Harness for LLM Code Generation — Co-build a DSL or domain model with the LLM, constrain generation by it, and keep the DSL as the durable source of truth — narrowing the solution space with a validator-backed grammar
• Recurring Control Belongs in Harness Code, Not Context — Move an agent's repeated control decisions out of per-task model context into persistent harness code, once task volume, a regression gate, and a sandbox justify the build
• Harness Impermanence — Author agent scaffolding as depreciating capital — design for low cost of removal so native model capability can replace it cleanly
• Situated Harness Layers: Fix at the Layer That Owns It — An eight-layer model of what a harness manages, ordered by change cadence, so a fix lands on the layer that owns the problem rather than a faster one that cannot hold it
• Fleet Harness Attribution — Pin model and task, swap whole harnesses, and measure pass rate alongside input-token consumption across a model fleet to attribute outcomes to the harness layer rather than the model
• Isometric Harness Ablation — Pin the model, remove one harness subsystem at a time, measure the score drop — the resulting per-subsystem table ranks investment priorities
• Task Shape Decides What a Heavier Agent Harness Buys — Harness gain shrinks with model strength on fixed-workflow issue repair and grows with it on open-ended repository work, with a measured per-component ledger
• Prompt-Only Baseline Before a Specialized Agent Subsystem — Measure a prompt-only agent loop before adopting a memory or self-improvement subsystem; it beat both on two benchmarks, but only where the loop exposes its prompt and history as code variables
• Model-Set Parity: Reading Harness Efficiency Claims — A published cross-harness efficiency figure measures the harness only when both arms run the same models; change the model set and the number prices a configuration
• Model Rankings Reverse Across Agent Harnesses — Across 66 model-harness configurations the Claude-minus-GPT gap on Terminal-Bench 4 moved 38.09 points on a scaffold change alone, and the vendor's own harness is not reliably its best
• Reliability of an Automatically Selected Agent Harness — When a search procedure picks the winning harness, report the worst run and the lower-quantile selected lift alongside the mean
• Deletion and Cost Rules for an Evolving Agent Harness — Score-only acceptance never removes anything, so add a deletion rule for components that stopped earning and a price for the ones that did — unregularized evolution spent 2.4 times the tokens for 0.6 points of transfer
• Agent Determinism Moves to the Last Unconstrained Axis — Constraining tool sequence while leaving the plan as free text made reproducibility worse in two of four cells; schema-validate the axis that is not already at ceiling
• Suspect the Harness, Not the Model — When a released coding agent regresses after an update at a fixed model, suspect the harness before the model — a longitudinal study found flat resolve rate but 70%+ token inflation across 35 releases
• Lane-Based Execution Queueing — Isolate concurrent agent tasks into named queues with per-lane concurrency limits to prevent output interleaving, race conditions, and deadlocks
• Managed vs Self-Hosted Agent Harness — Decision framework for choosing between managed agent services and self-hosted harnesses based on compliance, memory ownership, model routing, and ops capacity
• Compose vs Fork vs Build: Adopting an Agent Harness — How you obtain a self-hosted harness: composing on a maintained open framework buys an upgrade path worth exactly as much as the upstream's compatibility commitment
• The Harness as Product: What Listed-Rate Pricing Buys — When a coding assistant bills at the model's listed API rates, the money buys the harness, governance, and workflow layer — the build-vs-buy question over raw tokens, and when each layer is worth zero
• Outcome Pricing as a Scope Signal — A per-outcome price reports bounded cost variance only when the buyer records the outcome, which is why a merged-changeset price says something about the harness and an inferred resolution does not
• Local Model Viability Factors for Coding — The RAM, model-size, context, tool-calling, and privacy factors that gate whether a self-hosted local model is good enough for a coding task, versus staying hosted
• Multi-Shape BYOK Provider — One BYOK provider that natively speaks Chat Completions, Responses, and Messages — with the API family declared per endpoint — replaces single-shape compatibility adapters that silently down-translate provider-specific capability
• Per-Model Harness Tuning — Treat the backing model as a first-class harness variable — express prompt, tool, and middleware deltas as declarative model-keyed overrides instead of forcing one configuration to work everywhere
• Natural Language Tool Selection (NLT) — Describe tools in prose and parse the model's plain-language choice instead of forcing JSON tool calls — a large accuracy win on weak or non-native-tool-calling models, near-zero on structured-optimized frontier ones
• Tools as Typed Code Stubs (Programmatic Tool Calling) — Expose tools as typed Python stubs the model calls from a script — it matched or beat JSON tool calling on 11 of 14 models, but only past a code-emission gate and a fan-out-shaped workload
• Recursive Agent Harnesses (RAH) — A parent agent generates and runs a script that spawns subagent harnesses in parallel — each with its own tools and context — making the recursive unit a full harness rather than a bare model call
• Browser as Agent Action Space — Design an agent-driven browser tool loop with element-addressed actions, signal-triggered observation, and per-step assertions — but only when control flow genuinely depends on what the page returns
• Generated Programs as Web Agent Action Space — Let the web agent write and run a program against a disposable browser while the workspace holds the durable code and logs — worth it past three gates, and weaker than plan-then-execute on injection
• Scoped Browser DevTools Access for Runtime Diagnosis — Give the coding agent a read-oriented Chrome DevTools Protocol attachment for diagnosing runtime, network, and console errors — but only when the agent's other tools cannot close the lethal trifecta against the imported DOM
• Per-Task Agent Routing Across Coding Harnesses — Route each task to a harness by its measured token cost, latency, and known failure mode instead of standardizing a whole team on one house agent
• Temporary Compensatory Mechanisms — Design scaffolding that compensates for current model limitations as removable layers, not load-bearing architecture
• The Think Tool — A mid-stream reasoning checkpoint that fires between tool calls, giving agents an explicit space to reflect on tool output before deciding the next action
• VS Code Agents App: Agent-Native Parallel Task Execution — Run multiple agent sessions simultaneously across projects — each session inherits workspace custom instructions and MCP servers, enabling practical fan-out task execution

Last reviewed: 2026-05-27