Skip to content

Terminal-First Agent Interfaces with Browser Escalation

Where platform API coverage is good, default an agent to terminal and API access and escalate to a browser only for session-bound or UI-bound tasks.

Give the agent a shell and a filesystem, point it at the platform's REST API, and add browser access only for the task classes with no programmatic path. Across 729 tasks on ServiceNow, GitLab and ERPNext, terminal agents running Claude Opus 4.6 reached 78.7% success at $1.22 per task against web agents' 79.9% at $3.97 (Bechard et al., arXiv:2604.00073v3). Success is level, cost is not.

The conditions this default needs

The result is conditional. Check all three before adopting terminal-first as the standing choice.

API coverage has to be real. An API agent's ceiling is the endpoint surface, not the interface style. On WebArena, an API-only agent scored 43.9% on GitLab, which publishes 988 documented endpoints, and 18.9% on Reddit, which exposes 31 (Song et al., arXiv:2410.16464v3).

The agent needs the API documentation on disk. In the enterprise study the terminal agent worked from official platform documentation scraped to markdown and stored locally (arXiv:2604.00073v3). Without it the agent guesses endpoint shapes.

Escalation needs a model strong enough to route. Below that bar the router loses to the terminal specialist it was meant to improve on, at several times the price.

Escalate to the browser for three task classes

The study names the cases where the UI is the shorter path (arXiv:2604.00073v3):

  • Session-dependent operations. ServiceNow impersonation carries state in browser cookies rather than in the API response. Terminal agents scored 0% on it, against two clicks in the browser.
  • Values that exist only as rendered output. Reading a dashboard chart forces the terminal agent to recompute the aggregation over the underlying tables.
  • Artifacts authored in a UI builder. Workflows built through Flow Designer have no reliable public API equivalent.

Everything else routes to the terminal.

Why it works

Two different mechanisms produce the two gaps, and conflating them leads to the wrong redesign.

Against tool-augmented agents, the cause is tool granularity rather than the terminal. The ServiceNow tool-use agent held 83 narrow per-endpoint tools and could not set fields beyond what each tool defined. Swapping that catalog for a single generic HTTP call tool reached 79.2% success at $0.25 per task, and the authors conclude that "tool granularity, rather than terminal access, explains most of the gap between tool-use and terminal agents" (arXiv:2604.00073v3). A coarse tool recovers the flexibility; the shell is one way to obtain it, not the only one.

Against web agents, the cause is observation cost. The filesystem lets the agent offload large intermediate results outside the model's context window and filter them with standard utilities, while DOM snapshots and screenshots enter context whole (arXiv:2604.00073v3).

When this backfires

  • Thin API coverage inverts the default. On WebArena's Reddit a hybrid browser-and-API agent scored 51.9% against API-only's 18.9% (arXiv:2410.16464v3). With a small endpoint surface the browser is the primary path.
  • Sufficiency does not hold on every benchmark. Hybrid agents beat API-only agents on WebArena overall, 38.9% against 29.2% (arXiv:2410.16464v3), and web agents beat terminal agents on GitLab for all four models in the enterprise study (arXiv:2604.00073v3).
  • A weak router makes escalation cost more than it returns. With Claude Sonnet 4.6 the hybrid agent scored 72.1% against the terminal specialist's 73.6% at four times the cost. Opus 4.6 improved on the specialist, reaching 83.0% (arXiv:2604.00073v3).
  • Terminal-only leaves measurable headroom. A per-task oracle picking the better agent on ServiceNow reached 89.1% against terminal's 73.6% (arXiv:2604.00073v3).
  • The action space is wider than the task. The study's own harness handed the agent authentication headers as readable environment variables, which its authors flag as poor practice and offset with role-scoped service accounts and read-only credentials (arXiv:2604.00073v3). That is the credential boundary an MCP server would hold, and terminal-first spends it.

Example

A ServiceNow automation covering incident triage, catalog requests and user administration. Incident and catalog work has full REST coverage, so the terminal path is a curl against the Table API, a jq filter over the response, and intermediate result sets written to files rather than held in context. Terminal agents scored 79.1% at $1.94 per task on the study's 330 ServiceNow tasks with Claude Opus 4.6, against a web agent's 77.6% at $4.21 (arXiv:2604.00073v3).

One task in the set asks the agent to act as another user to verify a permission. Session state for impersonation lives in the browser cookie, not the API response, and terminal agents complete it 0% of the time (arXiv:2604.00073v3). That single task class gets browser access, and the rest of the suite does not. Routing the whole suite through the browser instead would roughly double the bill for slightly lower success.

Key Takeaways

  • Terminal-first is a default with an escalation list attached, not a claim that browsers are unnecessary. Write the list before you deploy.
  • The three escalation triggers are session state held outside API responses, values that exist only as rendered output, and artifacts authored in UI builders.
  • The gap over MCP-style agents measures tool granularity, so a single coarse API tool captures most of it without granting shell access.
  • Verify endpoint coverage per platform before committing. The same agent design swings 25 points between a well-documented API and a thin one.
  • Cost the router, not just the agents. Escalation on a weak model can score below the terminal specialist while spending four times as much.
Emerging — single recent source Last reviewed: 2026-08-08