Skip to content

Bug-Class Hints as Exploit Input for Coding Agents

A hint naming a component and a bug class can give a coding agent the exploit search bound a CVE description supplies.

A bug-class hint is any leak that localizes a vulnerability without describing it. Madhavapeddy lists "a mailing list question, an odd commit in an orphan branch, or a context leak" as enough to alert someone else's agent [Source: Madhavapeddy, 2026]. Conventional process embargoes the bug and "assumes that secrecy of the details protects users" [Source: Madhavapeddy, 2026]. A hint naming a component and a defect class can do what that secrecy was preventing.

When the hint is enough

Three conditions have to hold together.

  • The hint localizes. It names a component and a defect class, not just a project. Fang et al. measured the gap: given a CVE description, their GPT-4 agent exploited 87% of a 15-vulnerability set, and "without the description, GPT-4 can exploit only 7%" [Source: Fang et al., arXiv:2404.08144v2].
  • The target reproduces locally. The agent needs a running oracle to iterate against. Fang's authors dropped vulnerabilities they could not rebuild, citing "unspecified dependencies, broken docker containers, or underspecified descriptions in the CVEs" [Source: Fang et al., arXiv:2404.08144v2].
  • The named class is reachable in the code. A path-traversal hint pays off where a filesystem path is derived from request input, and returns nothing useful where no such path exists.

What this changes for you

Treat a specific hint about a dependency you run in production as an open exposure window, not as advance notice. The window opens when the hint circulates.

Ship the mitigation on the report instead of on the release. Madhavapeddy notes the cohttp bug had one available immediately, "just normalise percent-encoded path separators in the request URL", a rule that "was implementable the minute the report arrived, and also deployable while the full fix went through review, testing and packaging" [Source: Madhavapeddy, 2026].

Run the same localized search over your own code before release, and size that work by how many candidates you can validate rather than by how much you can scan.

Why it works

The bottleneck an embargo protects is localization, not exploit authorship. In Fang's experiment the writing capability is identical in both arms and only the search bound changes, moving success from 7% to 87% [Source: Fang et al., arXiv:2404.08144v2]. The description is exactly what that 80-point gap measures, and it is exactly what an embargo withholds. A hint naming the component and the defect class reconstitutes that bound.

No benchmark has measured hints directly. Fang's two arms are the CVE description and nothing at all, so the hint case is an inference from field reports.

The mechanism is contested. Chris Rohlf audited that dataset and reports finding "public exploits for 11 out of the vulnerabilities", each linked from the National Vulnerability Database entry, and that "in many cases this NVD link is the first Google search result" [Source: Rohlf, 2024]. Fang's agent had web search. On that reading the description names which public proof-of-concept to fetch, so the 87% measures retrieval rather than derivation.

The cohttp case is the harder one for the objection. That report "arrived privately on a Slack channel via Jane Street", and the agent produced its exploit before the fix pull request was public [Source: Madhavapeddy, 2026]. A retrieval account needs a public artifact.

When this backfires

  • The rumour does not localize. "There is a bug somewhere in X" sits in the 7% arm, and chasing it spends scarce triage capacity. Pesoli et al. put it as "the resulting bottleneck is not only finding more bugs; it is absorbing, validating, triaging, patching, and shipping a larger stream of reports" [Source: Pesoli et al., arXiv:2605.24632v1].
  • You point an agent at your own dependencies with no triage capacity behind it. The curl project's confirmed-vulnerability rate ran "somewhere north of 15%" of submissions in earlier years and "plummeted to below 5%" from 2025 as AI-assisted reports arrived [Source: Stenberg, 2026]. At one in twenty, a small team buys a candidate backlog instead of a shorter exposure window. That is verification capacity saturation.
  • You read the ten-minute probe as proof of the thesis. Those probes followed a public pull request carrying the fix, and Madhavapeddy reads them as "indicating that automated watchers are keeping an eye on public repositories" [Source: Madhavapeddy, 2026]. Watching public diffs is the ordinary patch gap, not the hint effect.
  • You credit every acceleration statistic to agents. M-Trends 2026 reports that "the mean time to exploit vulnerabilities dropped to an estimated -7 days, meaning exploitation is routinely occurring before a patch is even released", and names rising zero-day use against edge and core network devices as the trend that figure underscores [Source: Google Cloud, M-Trends 2026]. It establishes the timeline and not the cause.

Example

Anil Madhavapeddy "released a security fix for OCaml's cohttp 6.3.0" [Source: Madhavapeddy, 2026]. The advisory is OSEC-2026-16, summarized as "Path traversal in Cohttp.Path.resolve_local_file" and scored 8.7 High [Source: OSV]. While the report was still private he pointed an agent at the affected code and asked it about path normalization. He reports that it "trivially created an exploit to probe a local live server in under a minute", and concludes "I found I could use my own agents to find the exploit just by knowing roughly what it was about" [Source: Madhavapeddy, 2026].

About ten minutes after he opened the fix pull request, "this website was fielding probes for percent-encoded traversal sequences" [Source: Madhavapeddy, 2026]. That second window is the ordinary patch gap; the first one ran while the report was still private.

The load lands downstream on maintainers. rclone maintainer Nick Craig-Wood reports "about 20 security disclosures through GitHub" across the project's first ten years and "over 40 in the last month", with GitHub CVE assignment slipping from two or three days to three or four weeks [Source: Willison, 2026].

Key Takeaways

  • Score a leak by whether it names a component and a defect class. That line is what separates Fang's 7% arm from its 87% arm.
  • An embargo now buys time in proportion to how vague the leak is, not to how long the details stay unpublished.
  • Where a request-level or configuration-level mitigation exists, deploy it on the report and let the packaged fix follow on its own schedule.
  • Carry the caveat when you argue this internally. Rohlf's retrieval objection applies to the benchmark, and the private-report cases are what carry the claim.
Emerging — single recent source Last reviewed: 2026-08-30