Applying Coding Agents to Non-Code Tasks¶

A coding agent earns its scaffold on non-code work only when you can name a command that fails whenever the artifact is wrong.

Decide the check before you decide the task. A coding agent's loop is edit a file, run something over it, read the result, edit again. Inside a codebase, incremental verification supplies the signal for free. Outside code that loop has no ground truth until you build one, so the first design step on a budget, a report, or a data cleanup is naming the command whose output disagrees with the agent when the deliverable is wrong.

Start from the check, not the artifact¶

The move that makes a non-code task tractable is converting it into something a script produces, so re-running the script is the check. Scope expansion treats a verification signal in the new domain as a precondition; this is where that signal comes from.

Anthropic's analysis of roughly 400,000 Claude Code sessions between October 2025 and April 2026 found that "every one of the ten largest occupations in our dataset lands within seven points of software engineers" on success (Anthropic). Its worked legal example is a lawyer building a script that flags missing clauses across a folder of contracts. That session produced code, where success rates ran 34% for software occupations and 29% for everyone else.

Checks that hold up outside a test suite:

A generator script, so the deliverable is reproducible and a diff against the last run is the regression test.
A format or schema validator over the output: required columns, parseable dates, character limits, resolvable links.
An arithmetic constraint the agent cannot argue with, such as a reconciliation that must sum to a known total.
A golden sample from earlier work that the new output is compared against.

Turning a judgement call into a structural one is the same move frontend agents make with component libraries. Anthropic's Growth Marketing team runs the first two checks together: their workflow "processes CSV files with hundreds of ads, identifies underperformers, and generates new variations within strict character limits" (Anthropic, 2025-07-24). The character limit is a rule a script applies to every row.

Why it works¶

The gain comes from the scaffold rather than the model, and the causal evidence holds the model constant. In OpenAI's GDPval evaluation of real knowledge-work deliverables, adding prompt scaffolding to GPT-5 raised its use of multimodal self-inspection from 15% to 97%, cut egregious formatting errors in PowerPoint files from 86% to 64%, removed black-square artifacts that had affected over half of generated PDFs, and improved human preference win rates by 5 percentage points (arXiv:2510.04374v1).

That also sets the ceiling. Across 44 occupations, Claude Opus 4.1 was the strongest model tested and reached parity or better on 47.6% of deliverables against industry experts (arXiv:2510.04374v1). A manufactured check moves work toward that line without clearing it.

When this backfires¶

One-off deliverables. A repo, generator, and review loop never amortize when nothing is regenerated; a chat turn produces the same document for less.
Acceptance criteria made of taste. Substituting a model judge for a test leaves real scoring noise: on RuVerBench even the strongest models "exhibit substantial noise" verifying rubrics in agentic scenarios, weaker models are prompt-sensitive, and majority voting shows diminishing returns (arXiv:2606.29920v1). Self-grading is worse, because models "struggle to self-correct their responses without external feedback, and at times, their performance even degrades after self-correction" (arXiv:2310.01798v2).
Office tasks that live in a browser or a conversation. On TheAgentCompany the strongest agent finished 30.3% of tasks autonomously, and "DS, Admin, and Finance tasks are the lowest, with many LLMs completing none of the tasks successfully" (arXiv:2412.14161v3). The blockers were web UI navigation, ignoring a colleague's answer after asking the right question, and abandoning long cross-referencing work.
Credentialed personal data plus a live browser. One agent holding a logged-in profile, financial exports, and untrusted pages assembles the lethal trifecta with no verification signal.
Thin prompts. GDPval's under-contextualized condition, at 42% of the original token length, degraded results because models "struggled to figure out context" (arXiv:2510.04374v1).

Silent failure is the common thread. GDPval's expert graders found models that hallucinated data, miscalculated, ignored reference files, and promised deliverables they never produced, with roughly 29% of GPT-5's failures rated bad or catastrophic (arXiv:2510.04374v1). A runnable check surfaces those before you ship.

Example¶

Two versions of one marketing task, differing only in where the check lives.

Anthropic's Growth Marketing team asks for a script instead of copy. The agent reads a CSV of hundreds of live ads, flags the underperformers, writes replacements, and applies the character limits every ad platform enforces (Anthropic, 2025-07-24). Next month's export reruns the same limits over new rows. An over-length headline fails there rather than at upload.

The counterfactual is the same request without a script. The agent returns text that looks correct, nothing counts the characters, nothing compares this month's output to last month's, and the first defect signal is a rejected upload or a live ad reading badly.

Key Takeaways¶

Write the acceptance command first. If you cannot name one, the work belongs in a chat window rather than an agent loop.
Ask for the generator rather than the artifact; the rerun is what makes the output checkable next month.
Expect to build the check yourself, because no domain outside code hands you one already running.
Do not promote a model judge to ground truth. Use it to triage candidates, and keep a deterministic gate behind it.
Budget for a review pass on whatever the check cannot see, because the documented failures are quiet ones.