main logo icon

Published on

October 13, 2026

|

17 min read

Vibe Coding Security Statistics 2026: AI-Generated Code Vulnerability Rates

Primary-source vibe coding security statistics for 2026: how often AI-generated code fails security tests, what coding agents get wrong, leaked secrets, hallucinated packages, AI-linked CVEs, incidents and how often developers check AI output.

Arafat Afzalzada

Arafat Afzalzada

Founder

Web App SecurityLLM Security

Summarize with AI

ChatGPTPerplexityGeminiGrokClaude

TL;DR

Across more than 100 AI models in four testing snapshots, Veracode's average security pass rate for generated code is 56%, and the best of 11 models tested in summer 2026, GPT-5.5, passed 68% of its security tasks. In Carnegie Mellon's SusVibes benchmark (ICML 2026), SWE-agent with Claude 4 Sonnet wrote working code for 57% of 186 real-world feature requests, but only 11.8% were both working and secure; 79.3% of its working solutions were vulnerable. GitGuardian measured a 3.2% secret-leak rate in Claude Code-assisted public commits in 2025, against a 1.5% baseline for all public GitHub commits, and leaked AI-service credentials rose 81% to 1,275,105. Georgia Tech's Vibe Security Radar had confirmed 74 CVEs traced to AI coding tools by April 2026, 14 of them critical, with 35 in March 2026 alone. Developers use these tools daily but check them unevenly: in Stack Overflow's 2026 survey, 53.0% of the 13,162 developers asked inspect tests or security implications before using AI output, and only 48% of developers in Sonar's survey always check AI-assisted code before committing.

On 28 July 2026, Veracode reported that code written by AI models still fails its security tests nearly 44% of the time: across more than 100 models and four testing snapshots, the average security pass rate sits at 56%, although the code compiles almost every time (Veracode, 2026 GenAI Code Security Report). The best of the 11 models it tested in summer 2026, OpenAI's GPT-5.5, passed 68%. Coding agents show the same gap: in Carnegie Mellon's SusVibes benchmark, published at ICML 2026, SWE-agent with Claude 4 Sonnet built working features for 57% of 186 real-world tasks, but only 11.8% were both working and secure (Zhao et al.). Vibe coding, Andrej Karpathy's term for accepting generated code without reading it, ships that gap to production.

The gap shows up outside benchmarks too. Claude Code-assisted public commits leaked secrets at 3.2% in 2025, against a 1.5% baseline for all public GitHub commits (GitGuardian). Georgia Tech's Vibe Security Radar had confirmed 74 CVEs traced to AI coding tools by April 2026, 35 of them in March (Georgia Tech). Meanwhile 73% of AI coding assistant users use them daily (Stack Overflow 2026), and only 48% of developers in Sonar's survey always check AI-assisted code before committing it.

This post is the Stingrai research team's 2026 reference for vibe coding security statistics. It brings together more than 100 figures from 26 primary sources, including Veracode, Carnegie Mellon, ETH Zurich, GitGuardian, Georgia Tech, GitHub, Stack Overflow, Google, Sonar and peer-reviewed studies. Lead data is 2025 and 2026 benchmark runs, telemetry and surveys, the freshest available as of 13 October 2026; studies from 2021 to 2023 are dated where they appear. Every figure carries its source and edition, plus its data window where the publisher gives one, so any claim can be audited inline.

Quick answer. AI-generated code fails security tests often, and newer models have not fixed it. Veracode's average security pass rate across more than 100 models is 56%, and the best of 11 models tested in summer 2026 passed 68%. In Carnegie Mellon's SusVibes paper, the best of 12 agent settings wrote working code for 57% of real-world tasks, yet 79.3% of those solutions were vulnerable. Failures cluster in authorization, secrets, injection and dependencies, so vibe-coded applications need authenticated, role-based testing and code review, not only a scanner: Stingrai's web application penetration testing tests these classes signed in as every user role.

TL;DR: key vibe coding security statistics

  • Security pass rate of AI-generated code (Veracode, 100+ models, four snapshots): 56% on average; the best of 11 new models, GPT-5.5, passed 68% (Veracode, July 2026).

  • Working agent solutions that were insecure (SusVibes, ICML 2026): 79.3% for SWE-agent with Claude 4 Sonnet, which solved 57% of 186 tasks but only 11.8% securely (Zhao et al.).

  • Correct and secure backends, top model (BaxBench leaderboard, data to May 2026, no security prompt): 56.1% for Claude Opus 4.5 Thinking; 34.9% of its working backends were successfully exploited (BaxBench).

  • Secret-leak rate in Claude Code-assisted public commits (GitGuardian, 2025): 3.2%, against a 1.5% baseline (GitGuardian).

  • CVEs traced to AI coding tools (Georgia Tech, to April 2026): 74 confirmed, 14 critical and 25 high; 35 in March 2026 (Georgia Tech).

  • Hallucinated package names in generated code (USENIX Security 2025, 2023-era models): at least 5.2% on average for commercial models and 21.7% for open-source models; 205,474 unique fake names (Spracklen et al.).

  • Issues per pull request, AI-co-authored versus human-only (CodeRabbit, 470 PRs): 10.83 versus 6.45; security findings about 1.6 times as frequent (94 against 60 per 100 PRs) (CodeRabbit).

  • Most common CodeQL alert on GitHub (Octoverse, year to August 2025): broken access control, in 151,000+ repositories, up 172% (GitHub).

  • Developers who inspect tests or security implications before using AI output (Stack Overflow 2026): 53.0%; 9.6% use answers as-is (Stack Overflow).

  • Developers who always check AI-assisted code before committing (Sonar, January 2026): 48%, while 96% do not fully trust it (Sonar).

Key takeaways

  • Working code is not secure code. Veracode's models compile close to 100% of the time but pass 56% of security tasks on average, and 79.3% of the working solutions from the best of 12 settings in the SusVibes paper were vulnerable.

  • Newer models do better on some benchmarks but have not closed the gap. Veracode's average is virtually unchanged at 56% and its best 2026 score (68%) is below GPT-5 Mini's 72% in October 2025; on the SusVibes leaderboard the best working-and-secure rate reached 43.5% by August 2026.

  • Failures cluster in authorization, secrets, injection and dependencies. Broken access control is GitHub's top CodeQL alert, models failed to prevent cross-site scripting in 86% of Veracode's 2025 cases, and 43% of hallucinated packages reappeared on every re-run in a Python test.

  • Daily use has outrun verification. In Stack Overflow's 2026 survey, 73% of coding-assistant users use them daily, yet 53.0% of developers asked inspect tests or security implications before using AI output; in Sonar's survey, 48% always check AI-assisted code before committing.

  • The incidents so far are configuration and authorization failures. Missing row-level security exposed 170 of 1,645 scanned Lovable projects in 2025, a Replit agent deleted production data, and a permission regression left public Lovable projects readable from 3 February to 20 April 2026.

Methodology

Sources. Benchmarks, peer-reviewed studies, vendor research, surveys, Georgia Tech's radar, the NVD and statements from the parties to each incident, each dated in the References and read on 13 October 2026.

Rules applied. Each figure is reported as its publisher states it (average, share or count, with its population and edition) and compared only within its own source. Figures a source contradicts were dropped: a CCS 2023 SQL injection rate (text and table disagree), Stack Overflow's 2025 trust figures and one of Escape's two app counts. SusVibes figures come from the final ICML 2026 version (186 tasks), not the December 2025 preprint (200 tasks). Veracode's full 2026 report is gated, so its figures come from the 28 July press release; its 56% headline does not reconcile with its 51% to 53% subgroup averages (the 11 summer 2026 models average 56.3% by our arithmetic) and is reported as stated. Figures that could not be confirmed against a primary source were dropped rather than estimated.

Limits. Georgia Tech's radar, GitGuardian's Claude Code rate and CodeRabbit's study rely on commit markers such as co-author tags, so they miss AI-written code that carries none.

How often AI-generated code fails security tests

Veracode gives each model 80 code-completion tasks in Java, Python, C# and JavaScript, each with a secure and an insecure way to finish, and scans the output with its static analyzer. Prompts carry no security instructions, and the tests run on raw models, not agents.

  • July 2025 report: in 45% of test cases the models introduced an OWASP Top 10 vulnerability. Java failed 72% of the time, and models failed to defend against cross-site scripting (CWE-80) in 86% of cases and log injection (CWE-117) in 88%.

  • October 2025 snapshot (published 18 November 2025): GPT-5 Mini set Veracode's highest pass rate to date, 72%, against 52% for the non-reasoning GPT-5-chat; Claude Sonnet 4.5 scored 50%.

  • Summer 2026 (published 28 July 2026): GPT-5.5 led 11 new models at 68%; six sat between 50% and 53%, and Qwen3.7-max came last at 50%.

Across all the models in Veracode's dataset, released from 2023 to 2026, coding-specialized models averaged 51% against 52% for general-purpose ones, and large, medium and small models 53%, 51% and 51%. The release also gives reasoning models 56% against 51% and Python 63% against Java's 30%, without stating the population.

GPT-5.5 leads Veracode's summer 2026 leaderboard at 68%

Figure 1: Security pass rate by model, summer 2026, with Veracode's 56% headline average. Source: Veracode, 2026 GenAI Code Security Report press release, Fig. 1.

BaxBench, from ETH Zurich's SRI Lab and partners, asks models for whole backend applications: 392 tasks across 14 frameworks and 6 languages, checked with functional tests and expert-written exploits. In its 2025 paper, exploits succeeded on average against about half of each model's correct programs. On the leaderboard (data last updated May 2026), Claude Opus 4.5 Thinking leads at 56.1% correct and secure without a security reminder, against 28.8% for OpenAI o1, yet 34.9% of its working backends were successfully exploited; a generic reminder lifted it to 66.1%.

Benchmark

What it tests

Headline figure

Publisher

GenAI Code Security, summer 2026

80 completion tasks, 4 languages, static analysis

56% average security pass rate across 100+ models; best new model 68%

Veracode

BaxBench leaderboard

392 backend apps, tests plus exploits

Top model 56.1% correct and secure; 34.9% of its correct apps exploited

ETH Zurich SRI Lab and partners

SusVibes, ICML 2026

186 feature requests in real Python repositories, 12 agent setups

Paper's best of 12 setups: 57% working, 11.8% working and secure

Carnegie Mellon and partners

Rows use different tasks and scoring; compare within a row, not across rows.

Coding agents: working is not the same as secure

Each of SusVibes' 186 tasks is a feature request from a Python repository where human programmers once committed a vulnerable implementation, across 79 CWE types. Agents work in the repository from a feature description with no security guidance, and their patches must pass the project's functional and security tests. Three agent frameworks (SWE-agent, OpenHands, Claude Code) ran four models.

Agents write working code far more often than secure code

Figure 2: Share of the 186 SusVibes tasks each agent setting solved with working code (FuncPass) and with working and secure code (SecPass). Source: Zhao et al., "Is Vibe Coding Safe?", ICML 2026, Table 3.

The paper's best setting, SWE-agent with Claude 4 Sonnet, solved 57% of tasks, yet 79.3% of those working solutions were insecure. In the Claude Code framework, Claude 4 Sonnet solved 41.4% of tasks but 5.9% securely. On SWE-agent with Claude 4 Sonnet, asking the agent to name the weakness first raised SecPass to 14.5% but cut functional correctness by 7 points, and telling it the weakness raised SecPass to 15.1% at a 1.6-point cost.

Newer agents do better on the SusVibes leaderboard (data updated 21 August 2026): Claude Opus 4.8 with Pi CLI passed functional tests on 88.7% of the same 186 tasks, but the best working-and-secure rate was 43.5%, for GPT-5.5 with mini-SWE-agent.

Prompting can help on a small scale: in a study accepted at CECIIS 2026, six web applications generated with security requirements had 24 confirmed findings against 51 without, none Critical or High (Andročec, 2026). The author calls the results descriptive, and the most severe finding surfaced only in manual testing.

Do AI assistants make developers write less secure code?

User and field studies are mixed.

  • Stanford, ACM CCS 2023. Of 47 participants, 3% of those using a Codex-based assistant wrote a secure solution to a message-signing task, against 21% of the control group, and assisted participants were likelier to believe their code was secure (Perry et al.).

  • NYU, USENIX Security 2023. Among 58 students writing C, AI help raised critical security bugs by no more than 10% (Sandoval et al.).

  • Public GitHub projects, ACM TOSEM 2025. Of 733 snippets generated by Copilot, CodeWhisperer and Codeium, 29.5% of Python and 24.2% of JavaScript snippets had security weaknesses (Fu et al.).

  • Pull requests, December 2025. In 470 open-source pull requests, AI-co-authored changes averaged 10.83 issues against 6.45 for human-only ones, and security findings were 1.57 times as frequent (94 against 60 per 100 PRs): improper password handling rose 1.88 times, insecure object references 1.91 times and cross-site scripting, from a small base, 2.74 times. CodeRabbit sells AI code review and inferred authorship from co-authorship signals (CodeRabbit report).

The earliest benchmark set the baseline: in 2021, NYU researchers found about 40% of 1,689 programs GitHub Copilot wrote for 89 security-relevant scenarios vulnerable (Pearce et al., IEEE S&P 2022).

Leaked secrets and hallucinated packages

GitGuardian counted 28.65 million new hardcoded secrets in public GitHub commits in 2025, up 34%. AI-service secrets rose 81% to 1,275,105, and Claude Code-assisted commits leaked secrets at 3.2% across 2025, about twice the 1.5% baseline; the rate peaked in August at about 2.4 times the human baseline and by December had effectively matched it (report). GitGuardian cautions that developers still decide what gets committed.

AI-assisted development is adding to secrets sprawl

Figure 3: AI-related findings from GitGuardian's State of Secrets Sprawl 2026, covering public GitHub in 2025. Source: GitGuardian, 17 March 2026.

Generated apps reuse secrets. Invicti generated 20,656 web applications with models such as gpt-5 and found 1,182 using the string "supersecretkey"; SQL injection and cross-site scripting were rarer than expected (Invicti, November 2025).

Spracklen and colleagues generated 576,000 Python and JavaScript samples with 16 models from 2023: of 2.23 million package references, 440,445 (19.7%) named packages that do not exist, 205,474 of them unique. Commercial models averaged at least 5.2% hallucinated packages and open-source models 21.7%. When 500 prompts that produced a hallucination were re-run 10 times (Python, four models), 43% of the hallucinated packages reappeared every time, which makes slopsquatting practical (USENIX Security 2025).

Package hallucinations are common and repeatable

Figure 4: Average share of hallucinated package names by model type, and how often a hallucinated package reappeared when its prompt was re-run 10 times. Source: Spracklen et al., USENIX Security 2025.

Vibe-coded apps in production

  • Escape (published October 2025). Escape analyzed more than 5,600 public apps built on vibe coding platforms, most on Lovable, and found more than 2,000 vulnerabilities, 400+ exposed secrets and 175 exposures of personal data; its passive scans give lower bounds (Escape).

  • Lovable row-level security (2025). A March 2025 scan by a researcher at Replit, a Lovable competitor, found 303 endpoints across 170 of 1,645 Lovable projects, about 10.3%, with inadequate row-level security, exposing data to unauthenticated users (Palmer). CVE-2025-48757 is rated CVSS 9.3 Critical by MITRE; Lovable disputes it, saying each customer is responsible for its application's data (NVD).

  • GitHub code scanning (year to August 2025). Broken access control overtook injection as the most common CodeQL alert, in more than 151,000 repositories, up 172%. GitHub notes that new CodeQL coverage of GitHub Actions revealed misconfigured permissions, and says much of the rise stems from misconfigured CI/CD permissions and AI-generated scaffolds that skip critical auth checks (Octoverse 2025).

  • AI-linked CVEs (to April 2026). Georgia Tech's Vibe Security Radar flags CVEs whose introducing commit carries an AI tool's signature. Across more than 43,000 advisories it had confirmed 74 cases, 14 critical and 25 high: about 18 in the second half of 2025 and 56 in the first quarter of 2026, 35 of them in March. Claude Code and Copilot lead partly because they leave the clearest signatures (Georgia Tech, 9 April 2026).

Flaws in the tools themselves are covered in AI coding assistant CVEs 2026 and the MCP server security assessment.

Vibe coding security incidents, 2025 to 2026

Date

What happened

Primary source

15 to 17 March 2025

A founder whose SaaS was "built with Cursor, zero hand written code" reports being under attack: maxed-out API keys, bypassed subscriptions and junk database records

Founder's posts on X, 15 March and 17 March

20 March to 29 May 2025

Missing row-level security exposes data in 170 of 1,645 scanned Lovable projects; CVE-2025-48757 published, disputed by Lovable

Palmer (Replit), NVD

18 to 20 July 2025

A Replit agent deletes data from SaaStr founder Jason Lemkin's production database during a code freeze; Replit's CEO calls it "unacceptable" and rolls out dev/prod database separation

Lemkin, Masad

3 February to 20 April 2026

A backend regression lets any Lovable user with a project link read public projects' chats and code; reports from 22 February went unescalated, and a fix shipped within two hours of public disclosure

Lovable, 22 April 2026

Supabase, the database behind many vibe-coded apps, now enables row-level security by default for tables created in its dashboard and emails owners when a table is created without it (Supabase, 7 January 2026).

Adoption and trust: who checks AI-written code

Stack Overflow's 2026 survey (30,903 responses from 169 countries, published 6 October 2026) found that 83% of the 17,464 who answered use at least one AI tool at work and 65.9% use AI coding assistants or agents, 73.0% of them daily. 48.0% trust AI output when they can easily verify it; only 6.6% trust it with important work decisions (Stack Overflow). In 2025, 72% said vibe coding, generating software from LLM prompts, was not part of their professional work, and 5% more said so emphatically (Stack Overflow 2025).

Only half of developers inspect tests or security implications before using AI output

Figure 5: What developers do before using an AI-generated answer for a development task (select all that apply, n = 13,162). Source: Stack Overflow 2026 Developer Survey.

  • Google DORA 2025. Of nearly 5,000 technology professionals, 90% use AI, while 30% trust it only "a little" (23%) or "not at all" (7%) (Google, September 2025).

  • Sonar, January 2026. More than 1,100 developers report that 42% of the code they commit is AI-generated or assisted; 96% do not fully trust it to be functionally correct, only 48% always check it before committing, and 35% use AI coding tools through personal accounts (Sonar).

  • Google, April 2026. 75% of all new code at Google is AI-generated and approved by engineers, up from 50% the previous fall (Pichai, 22 April 2026).

See also AI cyber attack statistics 2026 and prompt injection statistics 2026.

What this means for defenders

  • Treat generated code as unreviewed code. Scan every pull request for flaws and secrets, and rotate any credential that reached a commit: GitGuardian found over 64% of secrets valid in 2022 still valid in January 2026.

  • Test authorization signed in, as every role. Scanners rarely model who may do what, which is why scanners miss BOLA and IDOR. In Stingrai's data (October 2024 to August 2026), 70% of the 40 web application tests that produced findings surfaced at least one High or Critical authentication or authorization finding (State of Penetration Testing 2026).

  • Verify every dependency before installing it. Hallucinated names repeat, so confirm a suggested package exists and has a real maintainer history; pin versions.

  • Keep agents away from production. Give coding agents development credentials only, separate development and production databases, and gate destructive commands.

  • Prompt for security, then test anyway. Security prompts help in some studies, not others, and the twin-prompt study's worst finding surfaced only in manual testing.

Where Stingrai fits. Stingrai is a global CREST-accredited penetration testing services company founded in Toronto, Canada in 2021, trusted by companies from startups to enterprises to meet audit requirements for SOC 2, ISO 27001, CMMC, PCI DSS and HIPAA. OSCE³, OSWE, OSEP, CREST CRT certified pentesters, who are also world-class security researchers and bug bounty hunters. Choose from fully human-led or hybrid (AI agents plus human penetration testers) engagements across web, API, mobile, AI and LLM, cloud, network, Active Directory and social engineering penetration tests and red team engagements. Its web application penetration testing is human-led: two named penetration testers from a team holding OSCE³, OSWE, OSEP, CREST CRT and CISSP certifications, with 18 published CVEs across the team, test black, grey or white box, signed in as every user role, with role-based authorization testing for IDOR and broken access control alongside business logic, injection, authentication and session handling, source-assisted where you give repository access. For one web application and associated APIs there are also two fixed packages: the Autonomous Pentest, in which Snipe, Stingrai's autonomous testing agent, performs black-box testing and white-box code review, hunts complex vulnerabilities such as IDOR, business logic and authorization flaws, and opens AutoFix pull requests with PR gating, at US$3,000 per assessment; and the Hybrid Pentest, in which certified penetration testers work alongside Snipe throughout the engagement, which is US$6,800 (pricing). Mobile, cloud, AI and LLM and every other scope get human-led testing, quoted per engagement. Every confirmed finding is posted to the PTaaS platform the same day it is confirmed, retesting is included, and every human-led and hybrid report ships with an attestation letter, delivered as a one-time annual test or a continuous program.

How to cite this page

Stingrai Research Team. "Vibe Coding Security Statistics 2026: AI-Generated Code Vulnerability Rates." Stingrai, 13 October 2026. https://www.stingrai.io/blog/vibe-coding-security-statistics-2026

Cite single figures to their primary source.

Frequently Asked Questions

What are the most important vibe coding security statistics in 2026?

Veracode's average security pass rate for AI-generated code is 56% across more than 100 models, and the best model it tested in summer 2026, GPT-5.5, passed 68%. In the SusVibes paper (ICML 2026), 79.3% of the working solutions from the best of 12 agent settings were vulnerable. Claude Code-assisted public commits leaked secrets at 3.2% in 2025 against a 1.5% baseline, and Georgia Tech had traced 74 CVEs to AI coding tools by April 2026.

What percentage of AI-generated code contains vulnerabilities?

It depends on the test. Veracode's 2026 report says models fail its security tests nearly 44% of the time across more than 100 models (45% of test cases in its July 2025 edition); on the BaxBench leaderboard, 34.9% of the top model's working backends were successfully exploited. The benchmarks use different tasks, so the rates are not directly comparable.

Are newer AI models better at writing secure code?

It depends on the benchmark. Veracode's average across four snapshots is 56%, which it says is virtually unchanged since its 2025 report, and its best 2026 score (68%) is below GPT-5 Mini's 72% from October 2025. Other leaderboards show gains: on BaxBench, Claude Opus 4.5 Thinking writes correct and secure backends 56.1% of the time against 28.8% for OpenAI o1, and on SusVibes the best working-and-secure rate rose from 12.9% in the ICML paper to 43.5% by August 2026. Even the leaders fail often: 34.9% of the top BaxBench model's working backends were exploited.

What vulnerabilities are most common in AI-generated code?

Authorization, secrets, injection and dependencies. Models failed to prevent cross-site scripting in 86% of Veracode's 2025 cases, GitHub attributes much of broken access control's rise to misconfigured CI/CD permissions and AI-generated scaffolds that skip auth checks, Invicti found 1,182 of 20,656 generated apps using "supersecretkey", and models hallucinate package names.

What is slopsquatting?

Slopsquatting is registering a package name that AI models hallucinate, so developers who install the suggestion get the attacker's code. In a USENIX Security 2025 study of 2023-era models, 19.7% of package references in generated code named packages that do not exist.

How many CVEs have been traced to AI coding tools?

Georgia Tech's Vibe Security Radar had confirmed 74 CVEs whose vulnerable code came from AI coding tools as of April 2026, 14 critical and 25 high, with 35 in March 2026 alone. It relies on commit signatures, so the real number is higher.

What vibe coding security incidents have happened?

Missing row-level security exposed data in 170 of 1,645 scanned Lovable projects in 2025 (CVE-2025-48757, disputed by Lovable), a Replit agent deleted production data in July 2025, and from February to April 2026 any Lovable user with a link could read public projects' code and chat history.

How do you test a vibe-coded application?

Test the running application signed in as every role, focused on authorization, IDOR, business logic and secrets, and review the source. Stingrai's web application penetration testing does both, as a one-time annual test or a continuous program.

References

  1. Veracode. _2026 GenAI Code Security Report_, press release. 28 July 2026. https://www.veracode.com/news/llms-are-getting-smarter-but-not-safer-veracode-2026-genai-code-security-report-finds-ai-generated-code-security-has-stalled-at-56%25-pass-rate/

  2. Veracode. _GenAI Code Security Report, October 2025 update._ 18 November 2025. https://www.veracode.com/press-release/veracode-research-reveals-openais-gpt-5-models-lead-the-way-in-secure-code-while-wider-industry-progress-stalls/

  3. Veracode. _2025 GenAI Code Security Report_ and blog summary. 30 July 2025. https://www.veracode.com/wp-content/uploads/2025_GenAI_Code_Security_Report_Final.pdf and https://www.veracode.com/blog/genai-code-security-report/

  4. Zhao et al. (Carnegie Mellon University and others). _Is Vibe Coding Safe? Benchmarking Vulnerability of Agent-Generated Code in Real-World Tasks._ ICML 2026. https://proceedings.mlr.press/v306/zhao26ax.html

  5. SusVibes team. _SusVibes leaderboard_, data updated 21 August 2026. https://leililab.github.io/susvibes-leaderboard/

  6. Vero et al. (ETH Zurich and others). _BaxBench: Can LLMs Generate Correct and Secure Backends?_ 2025; leaderboard data May 2026. https://baxbench.com/

  7. Pearce et al. (NYU). _Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions._ IEEE S&P 2022. https://arxiv.org/abs/2108.09293

  8. Andročec. _Vibe Coding and Web Application Security: A Twin-Prompt Study._ CECIIS 2026; preprint 21 August 2026. https://arxiv.org/abs/2608.20963

  9. Perry et al. (Stanford). _Do Users Write More Insecure Code with AI Assistants?_ ACM CCS 2023. https://arxiv.org/abs/2211.03622

  10. Sandoval et al. (NYU). _Lost at C: A User Study on the Security Implications of Large Language Model Code Assistants._ USENIX Security 2023. https://arxiv.org/abs/2208.09727

  11. Fu et al. _Security Weaknesses of Copilot-Generated Code in GitHub Projects: An Empirical Study._ ACM TOSEM 2025. https://arxiv.org/abs/2310.02059

  12. CodeRabbit. _State of AI vs Human Code Generation Report._ 17 December 2025. https://www.coderabbit.ai/content/assets/code-rabbit-state-of-ai-vs-human-code-generation-report-lite.pdf

  13. GitGuardian. _The State of Secrets Sprawl 2026._ 17 March 2026. https://blog.gitguardian.com/the-state-of-secrets-sprawl-2026/ and https://cdn.prod.website-files.com/5ee9da909a44e856ddcbaa4f/69c424234dad7521ead5ed42_The%20State%20of%20Secrets%20Sprawl%202026_v1.1.pdf

  14. Invicti. _Security issues in vibe-coded web applications: 20,000 apps built and analyzed._ 5 November 2025. https://www.invicti.com/blog/security-labs/security-issues-in-vibe-coded-web-apps-analyzed

  15. Spracklen et al. _We Have a Package for You! A Comprehensive Analysis of Package Hallucinations by Code Generating LLMs._ USENIX Security 2025. https://www.usenix.org/conference/usenixsecurity25/presentation/spracklen

  16. Escape. _How we discovered over 2k high-impact vulnerabilities in apps built with vibe coding platforms._ 29 October 2025. https://escape.tech/blog/methodology-how-we-discovered-vulnerabilities-apps-built-with-vibe-coding/

  17. Matt Palmer (Replit). _Statement on CVE-2025-48757_ and advisory. 29 May 2025. https://mattpalmer.io/posts/statement-on-CVE-2025-48757/ and https://mattpalmer.io/posts/CVE-2025-48757/

  18. NIST. _National Vulnerability Database: CVE-2025-48757._ https://nvd.nist.gov/vuln/detail/CVE-2025-48757

  19. GitHub. _Octoverse 2025._ 28 October 2025. https://github.blog/news-insights/octoverse/octoverse-a-new-developer-joins-github-every-second-as-ai-leads-typescript-to-1/

  20. Georgia Tech. _Bad Vibes: AI-Generated Code is Vulnerable, Researchers Warn._ 9 April 2026. https://www.cc.gatech.edu/news/bad-vibes-ai-generated-code-vulnerable-researchers-warn

  21. Lovable. _Our response to the April 2026 incident._ 22 April 2026. https://lovable.dev/blog/our-response-to-the-april-2026-incident

  22. Posts on X. Andrej Karpathy, 2 February 2025; a founder, 15 and 17 March 2025; Jason Lemkin, 18 July 2025; Amjad Masad (Replit), 20 July 2025. https://x.com/karpathy/status/1886192184808149383, https://x.com/leojrr/status/1900767509621674109, https://x.com/leojrr/status/1901560276488511759, https://x.com/jasonlk/status/1946069562723897802, https://x.com/amasad/status/1946986468586721478

  23. Supabase. _Supabase Security Retro: 2025._ 7 January 2026. https://supabase.com/blog/supabase-security-2025-retro

  24. Stack Overflow. _2026 Developer Survey_ (6 October 2026) and _2025 Developer Survey_ (29 July 2025). https://survey.stackoverflow.co/2026/ai/data and https://survey.stackoverflow.co/2025/ai

  25. Google. _2025 DORA Report_ (23 September 2025) and Sundar Pichai's Cloud Next 2026 remarks (22 April 2026). https://blog.google/technology/developers/dora-report-2025/ and https://blog.google/innovation-and-ai/infrastructure-and-cloud/google-cloud/cloud-next-2026-sundar-pichai/

  26. Sonar. _State of Code Developer Survey._ 8 January 2026. https://www.sonarsource.com/company/press-releases/sonar-data-reveals-critical-verification-gap-in-ai-coding/

Talk to Stingrai

AI-written code passes functional tests and still ships authorization gaps and hardcoded secrets. Stingrai's penetration testers can scope a one-time or continuous test of your web application and associated APIs, signed in as every role. Request a quote, see pricing or estimate a budget with the pentest cost calculator.

0 views

0

X

Related reading

Best Healthtech Penetration Testing Companies (2026): TGV, Ontario Health and HIPAA
Web App SecurityLLM Security

Best Healthtech Penetration Testing Companies (2026): TGV, Ontario Health and HIPAA

Healthtech penetration testing companies ranked for 2026, for vendors selling to health systems, with what TGV, Ontario Health and HIPAA require.

24 min read

Cloud Security Statistics 2026: Breaches, Misconfigurations and Identity Attacks
Network SecurityWeb App Security

Cloud Security Statistics 2026: Breaches, Misconfigurations and Identity Attacks

Cloud security statistics 2026: identity drove 83% of cloud and SaaS intrusions, and public cloud breaches cost US$5.38M against a US$4.99M global average.

27 min read

TGV Certification for AI Health Tools in Quebec (2026): 69 AI Criteria and the Pentest
LLM SecurityWeb App Security

TGV Certification for AI Health Tools in Quebec (2026): 69 AI Criteria and the Pentest

Quebec's April 2026 TGV list adds 69 AI criteria and a prompt injection test report for generative AI. What AI health tools must document, log and test.

20 min read

Contents

X