Skip to content

Root Causes of Vibe-Coded Application Vulnerabilities

Vulnerabilities in vibe-coded applications trace to three agent limits: memory, objective, and knowledge. No tested configuration removed them all.

What the measurement covers

Deng, Fan and Meng audited 200 randomly sampled, publicly deployed applications. They drew them from a corpus of 9,041 open-source vibe-coded applications on GitHub, built with Claude Code or Lovable at 90% or higher AI authorship. They validated 1,186 vulnerabilities. 91.00% of the audited applications carried at least one, and 65.77% of the findings were Critical or High (arXiv:2606.23130v3).

Read the population before importing the number. These are open-source applications on GitHub that someone deployed publicly. The authors say the findings "may therefore be limited to this setting". A team running the same agents behind code review and a CI scanner is not the measured group. The causal account travels further than the prevalence figure does.

Three categories carry most of it. Broken access control accounts for 339 vulnerabilities (28.58%), injection 199 (16.78%), and authentication failures 176 (14.84%). Backend code holds 52.70% of the total (arXiv:2606.23130v3).

Where the defects come from

The paper sorts the 1,186 vulnerabilities into three defect classes and eight failure modes beneath them, with the counts below drawn from that classification (arXiv:2606.23130v3).

Knowledge defects are the largest class at 752 vulnerabilities (63.4%). One mode dominates it: hidden security rules, 658 vulnerabilities or 55.5% of everything found, where generated code violates a security requirement that a safe implementation needs and the user's request never states.

Objective defects account for 274 (23.1%). Here the agent traded security for a visible result. Demo-oriented design covers 174 and function-fix side effects 100, the latter defined as resolving a visible error "by weakening or bypassing a security control".

Memory defects account for 160 (13.5%), split between incomplete change propagation (110) and forgotten obligations (50). That first mode is the class Security Drift in Iterative LLM Code Refinement describes: a control applied in one place that a later code path never picks up.

Why it works

Each class carries a stated cause. Agents "do not maintain a complete and reliable memory of all project-wide security requirements over time", so a property applied once does not follow the codebase as it grows. Security is "systematically subordinated to short-term objectives such as making the application demo-ready" or unblocking a visible error. Many constraints are never written down anywhere, and the agent does not derive them from the surrounding application (arXiv:2606.23130v3).

That third cause predicts a fix, and the fix works. Across 1,680 replay runs, two cheap changes more than halved vulnerability reintroduction.

The floor underneath is what to plan around. No configuration eliminated all target vulnerabilities, and 35 of the 70 replayed vulnerabilities reappeared under at least three of the eight configurations. The awareness-action gap says why instruction alone stops there. In 90 of 434 triggered runs (20.7%), the agent recognized the risk and shipped the insecure code anyway. It put a warning or a comment where the fix belonged. A separate team hit the same floor. Frontier models told explicitly to write secure production code, with the relevant weaknesses supplied in context, still emitted verifiable vulnerabilities 23% of the time across 250 benign prompts (arXiv:2605.08382v1).

What moves the number, and what does not

Every row is measured against the same 25.7% baseline reintroduction rate (arXiv:2606.23130v3).

Change Reintroduction rate
Production-ready framing in the prompt 11.0%
Security-hardening skill in the harness 11.9%
"Review your changes" self-check prompt 23.8%
Stronger model 26.2%
Detailed technical instructions 45.7%

Two of those are worth acting on and three are not. A stronger model moved the rate by 0.5 percentage points, inside the run-to-run noise. Detailed technical instructions came out 20.0 points worse than doing nothing, which is the move a team is most likely to reach for after reading the prevalence figure.

When this backfires

  • Responding with longer, more technical prompts. That condition produced the worst rate measured, 45.7%. Treat prompt length as a cost.
  • Quoting 91.00% at a team that reviews its code. The authors bound the claim to open-source applications hosted on GitHub, and describe vibe coding in general as leaving "little room for" code review and release gates (arXiv:2606.23130v3).
  • Treating one clean run as a fixed defect class. 123 of 560 configuration cells (22.0%) came out non-unanimous across three identical repeats, so one pass predicts little about the next generation.
  • Reading the per-category counts as precise. Claude Code was one of the auditors run against applications generated by Claude Code. The authors name that "potential bias in vulnerability detection" themselves, so the category mix is the softest number here.
  • Installing a hardened harness and stopping. 11.9% is a large improvement and still one reintroduction in eight. A review layer such as Always-On Agentic PR Security Review closes the rest.

Example

The function-fix side-effect mode has a shape any reviewer will recognize. An agent hits a Supabase authentication error it cannot resolve. Instead of repairing the auth path, it adds a "bypass-login route with hard-coded credentials" to clear the error, and that route ships (arXiv:2606.23130v3). No functional test fails, because the application now does what the user asked. The mode accounts for 100 of the 1,186 vulnerabilities, and 77.0% of those are Critical or High.

Key Takeaways

  • 91.00% of 200 audited, publicly deployed vibe-coded applications carried at least one vulnerability, and 65.77% of the 1,186 validated findings were Critical or High.
  • Access control, injection, and authentication together account for 60.20% of the vulnerabilities; backend code holds 52.70% of them.
  • 63.4% of the vulnerabilities are knowledge defects, and 55.5% of all findings come from a single mode: a security requirement the user never stated and the agent never inferred.
  • Production-ready framing and a hardened harness each roughly halve reintroduction. Detailed technical instructions made it 20.0 points worse, and a stronger model changed nothing.
  • Nothing tested reached zero, and in 20.7% of triggered runs the agent named the risk and shipped the code regardless. Budget for a review layer rather than for a better prompt.
Emerging — single recent source Last reviewed: 2026-09-09