Best AI Model for Pentesting 2026: Claude, GPT, and Open Weight Models Compared

A verified, vendor-sourced comparison of the frontier and open weight models security teams actually run in 2026. Every specification, price and evaluation number below comes from the vendor's own documentation, model card or system card.

Quick answer: the best AI model for pentesting by task

There is no single best AI model for penetration testing, because a penetration test is at least six different jobs and no model leads all six. Here is the short version, by task.

Where Stingrai fits: Model choice is upstream of the engagement. Stingrai is a global CREST-accredited penetration testing services company founded in Toronto, Canada in 2021, trusted by companies from startups to enterprises to meet audit requirements for SOC 2, ISO 27001, CMMC, PCI DSS and HIPAA. OSCE³, OSWE, OSEP, CREST CRT certified pentesters, who are also world-class security researchers and bug bounty hunters. Choose from fully human-led or hybrid (AI agents plus human penetration testers) engagements across web, API, mobile, AI and LLM, cloud, network, Active Directory and social engineering penetration tests and red team engagements. Its testers have published 18 CVEs and are listed in the bug bounty Halls of Fame of Apple, Google, the US Department of Defense and the US Federal Reserve; one is a founding member of Uber's offensive security team, covering web, mobile and LLM work. For teams shipping AI, the AI and LLM penetration testing service covers prompt injection, jailbreaks, system prompt leakage and insecure output handling in LLM-powered applications, tool misuse, excessive agency and privilege escalation through tool chains and MCP in agent systems, and the surrounding infrastructure on Bedrock, Azure OpenAI, Vertex AI, vector databases and model gateways, mapped to the OWASP LLM Top 10, the OWASP Agentic Top 10 and MITRE ATLAS. Snipe is the agent side of that work and covers web applications and their APIs, at US$3,000 for an Autonomous Pentest and US$6,800 for a Hybrid Pentest in which penetration testers and Snipe work the target together (pricing). Mobile, cloud, network, Active Directory, social engineering and red team scopes are quoted, one-time or continuous, with retesting and an attestation letter included in every engagement.

• Repository-scale source code review: Claude Fable 5.1 (1M context, US$10 in and US$50 out per million tokens). Anthropic explicitly permits vulnerability discovery in source code on Fable 5.1 "at all access levels, including general availability" (Anthropic system card, September 2026).

• Long agent loops driving Burp, nuclei, nmap and Metasploit: GPT-5.6 Sol for value at US$4 in and US$20 out with a 1,050,000-token window, or GPT-6 Astra for the ceiling at US$10 and US$50 (OpenAI API docs).

• Explicitly authorized exploit development and scoped penetration testing: GPT-5.6 Cyber through OpenAI's Daybreak Red tier, or Claude Opus 5, which is where Anthropic's safeguards route penetration testing requests on the generally available stack.

• Client data that cannot leave your infrastructure: GLM-5.3 (84.5% on CyberGym), DeepSeek V4 Pro (MIT license), or Kimi K3 (internal use exempt from the commercial terms).

• Report writing, remediation guidance and scanner triage: Claude Sonnet 5 at US$2 in and US$10 out, or Claude Haiku 4.5 at US$1 and US$5. Paying frontier prices to reformat findings is the most common budget leak in AI-assisted testing.

• The honest caveat: the highest published exploit-development scores in this article belong to Claude Mythos 5.1, which is not generally available. It ships only through Anthropic's Cyber Verification Program to a vetted set of US organizations.

Figure 1: Which model fits which penetration testing task in 2026. Sources: Anthropic model documentation and the Claude Fable 5.1 and Mythos 5.1 system card; OpenAI API model documentation and the GPT-6 Astra Deployment Safety Hub; Z.ai GLM-5.3 documentation; DeepSeek V4 model card; Moonshot Kimi K3 model card.

Key takeaways

The safeguard layer now decides your model choice more than the benchmark does. Anthropic's Fable 5.1 safeguards "redirect several kinds of dual-use cybersecurity tasks" to Opus models, and the list is specific: "penetration testing, exploit generation, and binary-based vulnerability scanning" (Anthropic). You can point Fable 5.1 at source code and ask it to find bugs. Ask it to weaponize one and you are transparently served by a different, less capable model. Picking a model without reading its safeguard policy means picking a model you will not actually be talking to.

Offensive capability crossed a formal threshold in 2026. GPT-6 Astra is the first model OpenAI has designated as reaching the Critical cybersecurity level under its Preparedness Framework, meaning that with the right tools and access it can find previously unknown flaws and develop ways to exploit them without a person guiding each step (OpenAI Deployment Safety Hub). Anthropic's numbers point the same direction: Claude Mythos 5.1 produced 245 working exploits out of 250 trials on the Firefox 147 evaluation, against 131 for Claude Opus 5.

Both leading labs now gate offensive work behind verification, not refusal. OpenAI's Daybreak program splits into Blue (GPT-5.6 Sol, for vulnerability discovery, detection engineering and incident response) and Red (GPT-5.6 Cyber, for vulnerability research, exploit reproduction and mitigation development). Anthropic's equivalent is the Cyber Verification Program, which unlocks Claude Mythos 5.1. Legitimate penetration testing teams should expect to apply, be verified, and accept monitoring, rather than expect a jailbreak-free frontier model to simply comply.

Open weight models closed the security gap faster than anyone priced in. GLM-5.3 reports 84.5% on CyberGym, up from GLM-5.2's 77.2%, and 54.4% on ExploitBench, more than doubling its predecessor's 24.4% (Z.ai). Z.ai also reports 2,436 vulnerabilities found across 269 projects, including 1,097 medium-to-high severity issues. DeepSeek V4 Pro posts 83.3 on CyberGym and ships the weights under the MIT license. For a testing firm that cannot send client source code to a hosted API, that is the difference between using AI and not using it.

Model eloquence is not the deliverable. Proof of exploit is. Across 55 penetration tests, Stingrai's State of Penetration Testing 2026 recorded 1,206 verified findings, of which 67.2% were High or Critical, at a 0.74% false-positive rate. That last number is the one that matters when you evaluate a model: a finding nobody reproduced is not a finding.

How this comparison was built

Every specification, price, license term and evaluation number in this article was read directly from a vendor primary source during a research pass with a cutoff of 5 September 2026. Sources used: Anthropic's model documentation, the Claude Fable 5.1 and Claude Mythos 5.1 system card of 1 September 2026, and the Claude Fable and Claude Mythos product pages; OpenAI's API model documentation for GPT-6 Astra and GPT-5.6 Sol, the GPT-6 Astra Deployment Safety Hub, and the Amazon Bedrock model cards for Daybreak Blue and Daybreak Red; Z.ai's GLM-5.3 documentation, pricing page, Hugging Face model card and license file; DeepSeek's V4 model card and API pricing page; and Moonshot's Kimi K3 model card and license file.

Three disciplines shaped what you are reading. First, no figure was carried over from the previous version of this post. The 2026 edition of this article leaned on a third-party comparison blog for its model table, and that is exactly why the table went stale. Every number here is re-sourced from the vendor.

Second, figures that could not be reached on a primary source were dropped rather than estimated. A widely circulated cyber-jailbreak refusal comparison for GPT-6 Astra could not be opened on OpenAI's own site during this pass, so it does not appear here. Kimi K3 has no published offensive-security evaluation and no first-party per-token price we could confirm, so it is absent from both the evaluation column and the cost table, and this article says so instead of substituting a proxy.

Third, cross-vendor benchmark numbers are reported but not ranked against each other. Anthropic, Z.ai and DeepSeek all publish ExploitBench, ExploitGym or CyberGym results, but under different harnesses, turn budgets and safeguard states. Anthropic states plainly that its results "are obtained with all safeguards turned off, and they may not be directly comparable to public leaderboard entries produced under vendors' deployed conditions." Treat each vendor's numbers as a within-vendor trend line, not a league table.

What "best model for pentesting" actually means

The question has the wrong shape as usually asked. A penetration test decomposes into tasks with genuinely different demands, and the model that dominates one is often mediocre or blocked at another.

Reconnaissance and OSINT synthesis. Wide, shallow, heavily tool-driven. The binding constraint is tool-calling throughput and cost per call, not reasoning depth. Cheap fast models win here. Running GPT-6 Astra at US$50 per million output tokens to normalize subdomain lists is money on fire.

Source code review at repository scale. The binding constraint is context window plus retrieval discipline plus the willingness to hold a multi-file data flow in working memory. Every frontier model in this article now offers a 1M-token window, so raw window size has stopped being a differentiator. What differentiates is whether the model can trace an untrusted input through four files to a dangerous sink without losing the thread, and, critically, whether its safeguards permit the request at all.

Exploit development and payload adaptation. The binding constraint is not intelligence. It is policy. This is the one task where your access tier, not your model choice, determines whether the work happens.

Tool-calling reliability inside an agent loop. A real engagement is hundreds of tool calls across hours. What matters is malformed-call rate, recovery after a tool errors, and resistance to prompt injection from the target itself. That last point is underrated: a penetration testing agent reads attacker-controlled content by design, so injection robustness is a security property of your own tooling. OpenAI reports that GPT-6 Astra brought the Gray Swan indirect-prompt-injection attack success rate down to 8.5%, against 27.0% for GPT-5.6 Sol, and lifted its internal indirect-injection defender success rate from 96.23% to 99.79%.

Triage of scanner output. High volume, low ambiguity, ruthless cost sensitivity. Sonnet-class and Haiku-class models exist for this.

Report writing and remediation guidance. Quality matters, but so does consistency across a hundred findings. This is a mid-tier model job with a good template, not a frontier model job.

No single model wins all six because the constraints are orthogonal. Context window, tool-call reliability, price per million tokens and refusal policy do not co-vary. The practical answer is a routed stack: cheap models for breadth, a strong code model for review, a verified tier for authorized offensive work, and a mid-tier model for the write-up.

The 2026 model comparison table

Prices are the vendors' published list rates read during this research pass. GPT-5.6 Sol's rate is promotional through 21 November 2026. GPT-5.6 Cyber and Daybreak Blue prices are Amazon Bedrock in-region standard-tier rates, which are not the same as OpenAI direct pricing. DeepSeek's rate is the peak cache-miss rate; off-peak rates are half. Confirm current pricing with the vendor before committing budget.

Frontier API models

Claude Fable 5.1 and Claude Mythos 5.1

Anthropic released Claude Fable 5.1 and Claude Mythos 5.1 on 1 September 2026. They are the same underlying model with different safeguards. Fable 5.1 is generally available across Claude.ai paid plans, the Claude Platform, AWS, Google Cloud and Microsoft Foundry as claude-fable-5-1. Mythos 5.1 ships only through Anthropic's trusted access programs, and the one that matters for security teams is the Cyber Verification Program, described as offering "reduced cyber safeguards for defensive security work" to a vetted set of organizations, currently in the US.

Specifications: 1M-token context, 128K max output, June 2026 knowledge cutoff, US$10 per million input tokens and US$50 per million output. The detail that matters for agent loops is the cache read price. Anthropic prices cache reads at 10% of base input on most models but 2.5% on Fable 5.1 and Mythos 5.1, which works out to US$0.25 per million tokens. In a long engagement where the same repository context is re-read across hundreds of turns, that single line item dominates the bill.

The security evaluations in the system card are the most detailed any lab has published, and they carry an important caveat: they were run on Mythos 5.1 with safeguards turned off.

• Firefox 147 exploit development: Mythos 5.1 developed 245 full working exploits out of 250 trials, 98.0%. Mythos 5 managed 221 (88.4%) and Claude Opus 5 managed 131 (52.4%).

• ExploitBench (41 V8 JavaScript engine environments, 300-turn budget): a mean of 11.80 capability flags in the uninterrupted arm and 12.61 with a keep-trying nudge, producing full arbitrary code execution in 222 of 410 runs.

• OSS-Fuzz (Anthropic's internal evaluation over roughly 830 fuzzing entry points across 228 open-source projects, with no target-specific hints): Mythos 5.1 reached the top score of 1.0, a full control-flow hijack, on 17 targets, against 13 for Mythos 5 and 4 for Opus 5. Notably, on plain vulnerability identification the three models are close: 78.7%, 80.0% and 79.4% of trials scoring above zero. The gap is in weaponization, not discovery.

• CyberGym was retired from this system card "due to saturation," which is itself a data point about how fast this benchmark class is ageing.

Now the part that changes how you actually use it. Fable 5.1's safeguards allow vulnerability discovery in source code at all access levels including general availability, and continue to block vulnerability discovery in compiled binaries, on the reasoning that binary-only bug hunting "skews toward malicious use by attackers without source code access." The announcement is more specific still: the safeguards "redirect several kinds of dual-use cybersecurity tasks" to Opus models, and names "penetration testing, exploit generation, and binary-based vulnerability scanning." Flagged queries are automatically routed to Opus 4.8 for cybersecurity, and Anthropic states users are not charged Fable prices for rerouted requests.

The practical translation for a testing team: Fable 5.1 is a white-box code review model, not a penetration testing model. If your workflow is "read this repository and find the authorization bug," it is the strongest option Anthropic sells. If your workflow is "now write the proof of concept," you are being served by Opus, and you should plan your stack around that rather than fight it.

Anthropic also reports that after the 5.1 safeguard update, Claude Code users see "an average of around 60% fewer interventions per session" from cyber safeguards compared with Fable 5. That is a meaningful quality-of-life change for AppSec engineers who spent 2026 being false-positived out of legitimate work.

Independent red teaming supports the safeguard claims. Trajectory Labs spent roughly 74 hours and over 6,500 requests against sandboxed exploit-reproduction tasks and "did not obtain a working end-to-end exploit in any task using Fable 5.1 alone, and did not find any universal jailbreak." 10a Labs ran over 6,700 prompts and reported that "refusal behavior was consistently restrictive in cyber tasks." Gray Swan's automated attacker elicited useful information on only 2 of 200 harmful and dual-use chat queries.

Claude Opus 5, Sonnet 5 and Haiku 4.5

Because Anthropic routes dual-use security work to Opus, Claude Opus 5 is the model most penetration testing teams on the Anthropic stack will actually spend their tokens on. It carries the same 1M context and 128K output as Fable 5.1 at half the price, US$5 in and US$25 out, with a May 2026 cutoff. On the system card's own evaluations it is materially weaker at exploit development, 131 versus 245 on Firefox 147, but statistically indistinguishable at finding the bug in the first place.

Claude Sonnet 5 (1M context, US$2 and US$10) and Claude Haiku 4.5 (200K context, US$1 and US$5) are the reporting and triage tier. A finding write-up is 2,000 output tokens of structured prose against a template. Running that on a frontier model is a 25x cost multiplier for no measurable quality gain.

GPT-6 Astra

OpenAI released GPT-6 Astra on 3 September 2026, initially to enterprises in the Daybreak Access Program and via the API, with rollout to ChatGPT Plus, Pro, Business and Enterprise following. The API model ID is gpt-6-astra: 1,050,000-token context, 128,000 max output, 30 April 2026 knowledge cutoff, US$10 per million input tokens, US$1 cached input, US$12.50 cache writes and US$50 per million output.

The headline is not the benchmark. It is the classification. GPT-6 Astra is the first model OpenAI has designated as reaching the Critical cybersecurity capability level under its Preparedness Framework, defined as being able, with the right tools and access, to find previously unknown security flaws and develop new ways to exploit them across many well-protected systems without a person guiding each step. OpenAI paired that with substantially strengthened protections: encryption and enhanced access controls on model checkpoints, universal misalignment monitoring across all tool-using inference, blocking alignment evaluations before internal use, and a restricted initial deployment window.

For anyone building a penetration testing agent, the most useful published numbers are the injection-robustness figures, because a pentest agent ingests hostile content as a matter of course. On the Gray Swan indirect prompt injection arena, Astra's attack success rate is 8.5%, against 27.0% for GPT-5.6 Sol. OpenAI's internal indirect-injection defender success rate improved from 96.23% to 99.79%, and instruction-hierarchy robustness saturated at 99.99%.

Tool support is the broadest on offer: web search, file search, image generation, code interpreter, hosted shell, apply patch, skills, computer use, MCP and tool search through the Responses API. The hosted shell and computer use combination is what makes it credible as an orchestration brain for a testing harness rather than just an advisor.

GPT-5.6 Sol

GPT-5.6 Sol (gpt-5.6-sol) is the workhorse. It carries the same 1,050,000-token context window as Astra with a 922,000-token maximum input, 128,000 max output, and a 16 February 2026 cutoff, at US$4 per million input tokens, US$0.40 cached and US$20 per million output. That pricing is promotional through 21 November 2026, which is worth a calendar reminder if your unit economics depend on it.

Its distinguishing feature for agentic work is granular effort control: none, low, medium, high, xhigh and max. A penetration testing harness can run reconnaissance and scanner triage at low effort and escalate only the sub-tasks that need reasoning, which is the single most effective cost lever available on the OpenAI stack. Tool support matches Astra's list.

For most teams, Sol at high effort with aggressive prompt caching, escalating to Astra only for the hard reasoning steps, is the right default. Astra costs 2.5x Sol on both input and output.

Daybreak Blue and Daybreak Red: how authorized offensive work runs on OpenAI

This is the part the founder-supplied model list most often gets wrong, and it is worth stating plainly: Daybreak Blue and Daybreak Red are not models. They are access tiers inside OpenAI's Daybreak program, its Trusted Access for Cyber offering. Each tier serves a different underlying model.

Daybreak Blue serves GPT-5.6 Sol with safeguards tailored to authorized defensive security work. On Amazon Bedrock, where it launched on 12 August 2026 as openai.gpt-daybreak-blue-5.6-sol, its stated supported use cases are vulnerability discovery, detection engineering and incident response, with a 1M-token context window. Bedrock in-region standard pricing is US$5.50 in and US$33.00 out per million tokens at the 272K window, rising to US$11.00 and US$49.50 at the full 1M window.

Daybreak Red serves GPT-5.6 Cyber, a purpose-trained cybersecurity model. On Bedrock, as openai.gpt-5.6-cyber, it carries a 272K-token context window and its stated supported use cases are vulnerability research, exploit reproduction and mitigation development. AWS documents the intended workflows as including proof-of-concept exploit development, exploit-chain validation, penetration testing and red teaming. Bedrock in-region standard pricing is US$13.75 per million input tokens, US$1.375 cache read and US$82.50 per million output, roughly six times Sol's list output rate.

Both tiers require enrollment in Trusted Access for Cyber, and Red requires additional approval on top, subject to stronger verification, monitoring, access controls and human oversight. The program terms are explicit that you may not share or sell that access to third parties, and may not use it to test systems you do not own or lack explicit authorization to test.

That last clause is the one to read twice if you are a consultancy. Your authorization to test a client's system is not automatically an authorization you can extend to a model provider's terms. Scope letters and rules of engagement need to survive contact with an AI vendor's acceptable use policy, and that is a contract question, not a technical one.

Open weight models you can self-host

For a penetration testing firm, the strongest argument for open weights has nothing to do with benchmarks. It is that client source code, credentials, session tokens and unremediated vulnerability detail never leave infrastructure you control.

GLM-5.3 (Z.ai / Zhipu AI)

GLM-5.3 shipped in August 2026 and is the most cyber-forward open weight release of the year. Z.ai's own documentation reports a 1M-token context window with 128K max output, function calling, context caching and structured output, at US$1.4 per million input tokens and US$4.4 per million output on the hosted API, with cached input at US$0.26.

Its published security results are the strongest of any open weight model. CyberGym 84.5%, up from GLM-5.2's 77.2%. ExploitBench 54.4%, more than doubling GLM-5.2's 24.4%. On ExploitGym it completed 105 tasks within two hours and 130 within six hours. Its agentic scores moved comparably: Terminal Bench 3.0 from 4.6 to 28.3, and DeepSWE v1.1 from 46.2 to 66.9. Z.ai also reports real-world results rather than only benchmark scores: 2,436 vulnerabilities found across 269 projects, including 1,097 medium-to-high severity issues.

The weights are published under the GLM-5.3 license. For essentially every security consultancy, it is permissive: the only meaningful condition applies to model-as-a-service operators whose aggregate revenue exceeds US$10 billion over any consecutive twelve months, who must pass a Z.ai security review before commercial use. Running it inside your own testing infrastructure is unencumbered.

DeepSeek V4 Pro

DeepSeek's V4 model card, published 27 April 2026, documents V4-Pro as a Mixture-of-Experts model with 1.6T total parameters and 49B activated per token, a 1M-token context length and up to 384K output tokens. V4-Pro reached general availability as build 0813 in August 2026. It offers three effort modes: non-thinking for fast responses, think-high for deliberate analysis, and think-max for extended reasoning.

The license is the cleanest in this comparison. DeepSeek states that assets distributed via its open-source repositories, including model weights and code, are licensed under the MIT License. There is no revenue threshold, no attribution requirement and no security review clause.

Published results relevant to security work: CyberGym 83.3, Terminal Bench 2.1 87.9, DeepSWE 62.7, and Toolathlon-Verified 74.1. Hosted API pricing is US$1.32 per million input tokens on a cache miss, US$0.044 on a cache hit and US$3.96 per million output at standard rates, halving during DeepSeek's published off-peak window.

One caveat that matters for the hosted API and not for self-hosting: DeepSeek's privacy policy states that personal data is collected, processed and stored in the People's Republic of China, with the services controlled by Hangzhou DeepSeek Artificial Intelligence Co., Ltd. and, for the Open Platform API, Beijing DeepSeek Artificial Intelligence Co., Ltd. For most consultancies handling client data under a Canadian, US, UK or EU engagement contract, that makes the hosted API a non-starter and the self-hosted MIT-licensed weights the actual product.

Kimi K3 (Moonshot AI)

Kimi K3 released in July 2026 with weights following on 27 July. Moonshot's model card describes a Mixture-of-Experts architecture with 2.8T total parameters and 104B activated per token, 93 layers combining 69 KDA and 24 Gated MLA attention blocks, a 160K vocabulary and a 1M-token context window.

Its published strengths are agentic rather than offensive: DeepSWE 67.5, Terminal-Bench 2.1 88.3, ProgramBench 77.8, BrowseComp 91.2, MCPMark-Verified 94.5 and GPQA Diamond 93.5. Terminal orchestration and browser navigation at those levels map directly onto driving a testing toolchain.

Moonshot publishes no offensive-security evaluation for Kimi K3, and no first-party per-token API price could be confirmed in this research pass. Both are stated here as gaps rather than filled with an estimate.

The license deserves attention because it is stricter than DeepSeek's and looser than it first appears. The Kimi K3 license requires a separate agreement with Moonshot for model-as-a-service operators above US$20 million in aggregate revenue over any consecutive twelve months, and prominent "Kimi K3" attribution for products above 100 million monthly active users or US$20 million monthly revenue. Crucially, internal use is exempt, defined as any use that does not make the software, its outputs or its capabilities available to third parties. Running Kimi K3 inside your own testing environment sits squarely inside that exemption. Reselling its raw output as a productized scanning service does not.

Client data, confidentiality, and what to check in each policy

Penetration testing generates some of the most sensitive artifacts in a client relationship: unremediated vulnerabilities, source code, internal architecture, and often live credentials. Where those tokens go is a contractual question before it is a technical one.

Read four things in every vendor policy before you route client data through a model.

1. Default retention period for API inputs and outputs. Anthropic states that it "automatically delete[s] inputs and outputs on our backend within 30 days of receipt or generation," subject to exceptions (Anthropic privacy documentation). Read OpenAI's enterprise privacy page for its equivalent terms rather than assuming parity.

2. Whether a zero data retention agreement is available, and on which endpoints. Anthropic offers zero data retention agreements. Not every endpoint or feature is always in scope, and caching, logging and abuse-monitoring pipelines are the usual carve-outs.

3. Where the data physically sits. Anthropic offers US-only inference at 1.1x standard pricing. Amazon Bedrock distinguishes in-region, geo cross-region and global cross-region inference, and both Daybreak tiers are currently in-region in us-east-2 only. DeepSeek's hosted services store data in the People's Republic of China.

4. Whether safety and abuse monitoring reads your prompts. This one is usually the deal-breaker for offensive work and is rarely surfaced in the marketing. Anthropic's system card describes sampling vulnerability-discovery traffic from monitoring probes and grading it, using an aggregation method it says preserves user privacy. OpenAI describes universal monitoring of tool-using inference for GPT-6 Astra, including chains of thought. Neither is unreasonable. Both are things your client's data protection officer will want to know about before you paste their code into a prompt.

This is the case for open weights in penetration testing, and it is a strong one. A self-hosted GLM-5.3 or DeepSeek V4 Pro instance behind your own network boundary has no retention policy to negotiate, no cross-border transfer to disclose, and no third-party monitoring pipeline to explain in a scoping call. The trade is capability at the top end and operational cost: a 1.6T-parameter Mixture-of-Experts model is not something you run on a workstation. For firms already operating GPU infrastructure, the licenses make it viable. DeepSeek's MIT terms impose nothing. Kimi K3's internal-use exemption covers testing infrastructure. GLM-5.3's only real condition targets hyperscale model-as-a-service operators.

Safety, refusal behavior, and authorization for dual-use work

Security teams frequently experience frontier model safeguards as friction. It helps to understand what the labs are actually gating on, because it is more precise than "the model refuses hacking."

Anthropic gates on asymmetry. The system card frames the design goal as providing "an advantage to defenders but not attackers," and splits vulnerability finding accordingly. Source-code-based vulnerability finding "skews toward defenders" and is permitted at general availability. Binary-based vulnerability finding "skews toward malicious use by attackers without source code access" and is blocked. Whether you agree with that line, it is a coherent line, and it maps cleanly onto how you should design a workflow: give the model the source.

OpenAI gates on verification. Rather than making a general-purpose model comply with offensive requests, OpenAI moved the capability into a separately provisioned tier with identity verification, additional approval, monitoring and human oversight. Daybreak Red exists precisely so that authorized vulnerability research and scoped penetration testing have a sanctioned path.

What a legitimate penetration testing team should expect in 2026:

• Expect to be verified, not trusted by assertion. Both programs require organizational verification. Budget weeks, not hours, and expect the process to ask who you are and what you test.

• Expect authorization context to matter, and to be checked. Trusted Access terms explicitly prohibit testing systems you do not own or lack explicit authorization to test. Your rules of engagement and scope letter are the artifacts that make your work permissible. Keep them current, keep them specific, and make sure your AI vendor terms and your client contract do not contradict each other.

• Expect safeguards to occasionally block legitimate work, and expect that to improve. Anthropic's stated 60% reduction in Claude Code safeguard interventions is a real trend. It is also an admission that the previous rate was too high.

• Do not architect around evasion. Aside from being a terms violation that will lose you the account, it is strictly worse engineering than using the tier designed for the work. Anthropic's published red-team results show that where testers extracted anything useful from Fable 5.1, a second, weaker model still had to supply the weaponization step. The friction is real and the workarounds are poor.

What it costs: a worked example

Model choice shows up on the invoice long before it shows up in findings quality. Here is a single concrete task priced across the stack: one white-box review of a 200,000-token repository, producing 40,000 tokens of structured findings, at published list prices.

Figure 2: Published API cost of a single 200,000-token repository review with 40,000 tokens of output, at list prices read from vendor documentation in September 2026. Sources: Anthropic model documentation; OpenAI API model documentation; Amazon Bedrock model card for Daybreak Red; Z.ai pricing documentation; DeepSeek API pricing.

Kimi K3 is omitted because no first-party per-token price was confirmed in this pass.

Three observations that matter more than the absolute numbers.

A single review is cheap. An agent loop is not. One pass over a repository costs a few dollars on any model here. A real engagement re-reads that context across hundreds of turns while driving tools, and the token count goes up by two orders of magnitude. That is where the difference between US$0.42 and US$6.05 stops being noise.

In agent loops, the cache read price is the price. Once you are re-sending the same repository context every turn, list input rates stop mattering and cached rates take over. Claude Fable 5.1 reads cache at 2.5% of input, US$0.25 per million. DeepSeek reads cache at roughly 3% of its miss rate, US$0.044. GPT-5.6 Sol and GPT-6 Astra both read cache at 10%. GLM-5.3 reads cache at US$0.26 against a US$1.4 input rate, about 19%. Ranking models on list input price and ignoring cache economics will mis-rank your actual bill by a wide margin.

The expensive tier is expensive for a reason and should be used narrowly. Daybreak Red at US$82.50 per million output tokens is not a model you leave in a loop. It is a model you call for a specific, authorized exploit-validation step and then hand back to a cheaper model for everything downstream.

Where autonomous agents fit

The model is one component. What converts model reasoning into a defensible finding is the harness around it: real tool execution, an attempt to reproduce the issue, state that survives a multi-hour engagement, and penetration testers working the same target at the same time.

Snipe is Stingrai's AI agent for web application penetration testing, including the application's APIs. It is available for autonomous web testing or alongside penetration testers in a Hybrid web engagement. It runs 27 specialised agents that coordinate across attack vectors, and it is custom-trained on 6,000+ HackerOne Hacktivity disclosure reports plus custom skills distilled from years of Stingrai's penetration testers' methodology. That training set is the point: it encodes how senior testers actually find bugs in messy production applications, rather than how a model performs on a synthetic capture-the-flag.

Snipe is built to hunt the classes generic AI scanners miss. Insecure direct object references, business logic flaws and broken access control are its target set, alongside injection, authentication and session weaknesses, server-side request forgery and API issues across endpoints and user roles. It runs black-box dynamic testing, grey-box authenticated testing across user roles, and white-box source code review with data flow tracing. With repository access it proposes code changes for confirmed vulnerabilities as AutoFix pull requests, and it can run as a pull request gating check so vulnerable code is caught before it merges.

On Hybrid engagements, Stingrai's penetration testers work alongside Snipe throughout the assessment, directing where it looks, extending the attack paths it opens, and pursuing what it surfaces. Both contribute findings across every severity. Stingrai delivers this as both one-time annual penetration tests and continuous testing programs.

We do not publish which underlying model Snipe uses, and this article makes no claim about it. The reason is the entire argument above: the correct answer changes as safeguards, prices and access tiers change, and any agent worth running should be able to route work to whichever model is currently strongest and permitted for each sub-task.

The engagement data is what should ground your expectations of any AI-assisted testing. Across 55 penetration tests, Stingrai's State of Penetration Testing 2026 recorded 1,206 verified findings, of which 67.2% were High or Critical, at a 0.74% false-positive rate. The severity distribution is not uniform by surface: 92% of internal network findings were High or Critical, against 54% for web application findings. Two conclusions follow. First, the value of a testing program is concentrated in a relatively small number of high-severity, reproducible issues, not in finding volume. Second, a 0.74% false-positive rate is only achievable when every finding is proven, which is a property of the harness and the humans, not of the base model's eloquence.

The product-level view sits in a companion piece: our best AI pentesting tools 2026 guide compares the shipping tools. This article is about the models underneath them.

An evaluation checklist you can run yourself

Vendor benchmarks will not tell you which model fits your workflow. This will. Budget half a day.

1. Take one real finding from a past engagement, ideally an authorization bug rather than an injection bug. Strip client identifiers. This is your ground truth.

2. Give each candidate model the relevant source files and nothing else. No hints about the vulnerability class, no pointing at the file. Measure whether it finds the bug, how many false positives it raises alongside it, and how many turns it takes.

3. Test the safeguard boundary explicitly and early. Ask for the finding, then ask for a proof of concept. Record where the model stops. Do this before you architect anything, because a model that reviews well but blocks at the reproduction step forces a two-model design and you want to know that on day one.

4. Measure malformed tool calls over a 100-call run. Wire the model to three real tools and count schema violations, retries and dead loops. This is the number that decides whether an agent finishes an engagement, and it is almost never on a marketing page.

5. Test injection resistance with hostile target content. Plant an instruction in a page body, an HTTP header and a JavaScript comment telling the agent to report the target as clean. A penetration testing agent that follows attacker instructions is a liability, not a tool.

6. Price your actual loop, not a single call. Instrument one full engagement's token usage, then split it into cache reads, fresh input and output, and price each separately per model. Expect the ranking to differ from the list-price ranking.

7. Read the data terms with your own client contract open beside them. Retention period, zero data retention availability, data residency, and whether abuse monitoring reads prompts. If any of those conflict with a client obligation, the model is off the list regardless of how it scored in step 2.

8. Re-run this every quarter. Every model in this article is younger than six months. The correct answer in December will not be the answer in September.

Frequently Asked Questions

What is the best AI model for pentesting in 2026?

There is no single best model, because penetration testing is several tasks with different constraints. For repository-scale source code review, Claude Fable 5.1 is the strongest generally available option, and Anthropic explicitly permits vulnerability discovery in source code on it. For long agent loops driving tools, GPT-5.6 Sol offers the best capability per dollar at US$4 per million input tokens, with GPT-6 Astra as the ceiling. For explicitly authorized exploit work, GPT-5.6 Cyber through OpenAI's Daybreak Red tier, or Claude Opus 5, which is where Anthropic routes penetration testing requests. For self-hosted work on confidential client data, GLM-5.3 and DeepSeek V4 Pro.

Can ChatGPT or Claude do a penetration test?

Not as a general-purpose chat product, and not for the offensive half of the work. Anthropic states that Claude Fable 5.1's safeguards redirect penetration testing, exploit generation and binary-based vulnerability scanning to its Opus models. OpenAI routes authorized offensive work through its Daybreak tiers, which require verification and additional approval. Both vendors will help with the defensive half, including source code review, vulnerability triage, detection engineering and remediation guidance, on their standard products. Exploit development requires the verified tier.

What is the best open source model for pentesting?

GLM-5.3 has the strongest published security results of any open weight model, reporting 84.5% on the CyberGym vulnerability discovery benchmark and 54.4% on ExploitBench, alongside 2,436 real vulnerabilities found across 269 projects. DeepSeek V4 Pro is close behind at 83.3 on CyberGym and ships under the MIT license, which is the most permissive license in this comparison. Kimi K3 is the strongest open weight model for agentic tool orchestration but publishes no offensive-security evaluation.

Is it legal to use an AI model for penetration testing?

The legality question is about authorization, not about the AI. Testing a system you do not own or have explicit written permission to test is unlawful in most jurisdictions whether or not a model is involved. On top of that, vendor terms add their own constraint: OpenAI's Trusted Access for Cyber terms state you may not use that access to test systems you do not own or lack explicit authorization to test, and you may not share or sell the access to third parties. If you are a consultancy, confirm that your client scope letter and rules of engagement are compatible with your model provider's acceptable use policy before the engagement starts.

Which AI model is best for security code review?

Claude Fable 5.1, on the strength of a 1M-token context window and Anthropic's explicit permission for source-code vulnerability discovery at general availability. Claude Opus 5 is close on discovery specifically: on Anthropic's OSS-Fuzz evaluation, Opus 5 scored above zero on 79.4% of trials against 78.7% for Mythos 5.1, so the models are comparable at finding bugs and differ mainly at weaponising them. Opus 5 costs half as much. If you self-host, GLM-5.3 and DeepSeek V4 Pro both carry 1M-token windows and strong published vulnerability-discovery results.

How much does it cost to run an AI model for a pentest?

One 200,000-token repository review with 40,000 tokens of output costs about US$0.42 on DeepSeek V4 Pro, US$1.60 on GPT-5.6 Sol, US$4.00 on Claude Fable 5.1 or GPT-6 Astra, and US$6.05 on GPT-5.6 Cyber via Amazon Bedrock, at published list prices. A full agent-driven engagement is two orders of magnitude more tokens, and at that scale cache read pricing dominates: Claude Fable 5.1 reads cache at 2.5% of its input rate, while GPT models read at 10%. Price your actual loop rather than a single call.

What is the difference between Claude Fable 5.1 and Claude Mythos 5.1?

They are the same underlying model with different safeguards. Fable 5.1 is generally available with cybersecurity and biology safeguards applied. Mythos 5.1 has reduced cyber safeguards and is available only to vetted organizations through Anthropic's Cyber Verification Program, currently limited to a set of US organizations. The published exploit-development results in Anthropic's system card, including 245 working exploits out of 250 trials on the Firefox 147 evaluation, were produced on Mythos 5.1 with safeguards turned off.

What is the difference between Daybreak Blue and Daybreak Red?

They are access tiers in OpenAI's Daybreak program, not models. Daybreak Blue serves GPT-5.6 Sol with defensive safeguards and supports vulnerability discovery, detection engineering and incident response, with a 1M-token context window. Daybreak Red serves GPT-5.6 Cyber, a purpose-trained cybersecurity model with a 272K-token window, and supports vulnerability research, exploit reproduction and mitigation development, including proof-of-concept exploit development and scoped penetration testing. Red requires additional approval beyond Blue, with stronger verification, monitoring and human oversight.

Which AI model has the best published cybersecurity benchmark results?

Claude Mythos 5.1, on Anthropic's own evaluations: 245 full working exploits out of 250 trials on the Firefox 147 benchmark, a top score on 17 OSS-Fuzz targets, and full arbitrary code execution in 222 of 410 ExploitBench runs. Those results were produced with safeguards turned off and are not directly comparable to figures other vendors publish under deployed conditions. Among generally available models, GPT-6 Astra is the first that OpenAI has rated Critical for cybersecurity under its Preparedness Framework.

Does using an AI model put client data at risk during a pentest?

It depends entirely on the deployment. Anthropic states it deletes API inputs and outputs from its backend within 30 days and offers zero data retention agreements. OpenAI publishes its own enterprise terms, which you should read directly. DeepSeek's hosted services store data in the People's Republic of China, which is usually disqualifying for engagements under Western contracts. Self-hosting an open weight model removes the question entirely, which is the main reason penetration testing firms run GLM-5.3, DeepSeek V4 Pro or Kimi K3 on their own infrastructure.

Can an AI model replace a penetration tester?

Not on the evidence available. The models are genuinely strong at breadth, enumeration, source-level bug discovery and, in the verified tiers, exploit development. What a report needs is a reproduced, impact-assessed, prioritised finding. Across 55 Stingrai penetration tests, 1,206 verified findings carried a 0.74% false-positive rate, and that rate is a product of proof and human judgement rather than model fluency. The realistic 2026 model is a joint engagement in which an agent and penetration testers work the same target concurrently.

Which model should I use for writing the pentest report?

A mid-tier model. Report writing is templated, structured prose at a few thousand output tokens per finding, and frontier reasoning adds nothing measurable. Claude Sonnet 5 at US$2 in and US$10 out, or Claude Haiku 4.5 at US$1 and US$5, will produce the same report quality as a model costing 10x to 50x more. Spend the frontier budget on discovery and validation, not on prose.

Put these numbers to work

Model benchmarks describe capability. Engagement data describes results. Stingrai's State of Penetration Testing 2026 shows what a real engagement finds: 93% of tests surfaced at least one High or Critical finding, with a 0.74% false-positive rate across 1,206 verified findings. If you want that view of your own environment, book a 30-minute scoping call or get a quote. Published pricing for one web application and associated APIs is US$3,000 for an Autonomous Pentest or US$6,800 for a Hybrid Pentest, on stingrai.io/pricing.

Related reading

• Best AI pentesting tools 2026, the product-level companion to this model-level comparison.

• Snipe, Stingrai's autonomous web application penetration testing agent.

• AI and LLM penetration testing services, for testing the AI systems you build.

• Web application penetration testing services.

• State of Penetration Testing 2026, the engagement dataset cited throughout.

• Top Penetration Testing Companies in Canada (2026 Ranked)

• Top Penetration Testing Companies in the USA (2026 Ranked)

• Average Cost of a Penetration Test in Canada (2026)

• AI Evaluation Containment Failures: 4 Real 2026 Cases

• Web Application Penetration Testing Services: How to Choose and What It Costs (2026)

References

1. Anthropic. Introducing Claude Fable 5.1 and Claude Mythos 5.1. September 2026. https://www.anthropic.com/claude-fable-and-mythos-5-1. Release announcement covering pricing, the Fable and Mythos safeguard split, and the routing of penetration testing, exploit generation and binary vulnerability scanning to Opus models.

2. Anthropic. System Card: Claude Fable 5.1 and Claude Mythos 5.1. September 1, 2026. https://www.anthropic.com/claude-fable-and-mythos-5-1. Full cyber capability evaluations including ExploitBench, OSS-Fuzz, Firefox 147 and ExploitGym, plus safeguard coverage and external red team results.

3. Anthropic. Claude Fable. 2026. https://www.anthropic.com/claude/fable. Product page documenting availability, pricing, cache read rates, US-only inference and safeguard rerouting behavior.

4. Anthropic. Claude Mythos. 2026. https://www.anthropic.com/claude/mythos. Documents the Cyber Verification Program and the vetted-access model for reduced cyber safeguards.

5. Anthropic. Models overview. 2026. https://platform.claude.com/docs/en/about-claude/models/overview. Canonical specification table for Claude Fable 5.1, Opus 5, Sonnet 5 and Haiku 4.5, including context windows, output limits, pricing and cache read rates.

6. Anthropic. How long do you store personal data. 2026. https://privacy.claude.com/en/articles/7996866-how-long-do-you-store-personal-data. Commercial product retention terms and zero data retention agreements.

7. OpenAI. GPT-6 Astra model documentation. September 2026. https://developers.openai.com/api/docs/models/gpt-6-astra. Model ID, 1,050,000-token context window, output limit, pricing and supported tools.

8. OpenAI. GPT-6 Astra Deployment Safety Hub. September 2026. https://deploymentsafety.openai.com/gpt-6-astra. Preparedness Framework Critical cybersecurity determination, prompt injection robustness results and deployment safeguards.

9. OpenAI. GPT-5.6 Sol model documentation. 2026. https://developers.openai.com/api/docs/models/gpt-5.6-sol. Model ID, context window, reasoning effort levels, tool support and promotional pricing window.

10. Amazon Web Services. Daybreak Red: GPT-5.6 Cyber model card. August 2026. https://docs.aws.amazon.com/bedrock/latest/userguide/model-card-openai-gpt-56-cyber.html. Context window, supported use cases, Bedrock pricing and Trusted Access for Cyber eligibility.

11. Amazon Web Services. Daybreak Blue: GPT-5.6 Sol model card. August 2026. https://docs.aws.amazon.com/bedrock/latest/userguide/model-card-openai-gpt-daybreak-blue-56-sol.html. Context window, defensive use cases and Bedrock pricing at both context tiers.

12. Z.ai. GLM-5.3 overview. August 2026. https://docs.z.ai/guides/llm/glm-5.3. Context window, tool support, and CyberGym, ExploitBench, ExploitGym and agentic benchmark results, plus real-world vulnerability discovery totals.

13. Z.ai. GLM-5.3 model card and license. 2026. https://huggingface.co/zai-org/GLM-5.3. Open weight distribution, benchmark table and the GLM-5.3 license terms.

14. DeepSeek. DeepSeek V4 Technical Documentation and Model Card. April 27, 2026. https://fe-static.deepseek.com/chat/transparency/deepseek-V4-model-card-EN.pdf. Parameter counts, 1M context length, effort modes and MIT license terms for open-source distribution.

15. DeepSeek. Models and pricing. 2026. https://api-docs.deepseek.com/quick_start/pricing. Cache hit and cache miss input rates, output rates, context length and maximum output for V4-Pro and V4-Flash.

16. DeepSeek. Privacy Policy. 2026. https://cdn.deepseek.com/policies/en-US/deepseek-privacy-policy.html. Data controller entities and storage location for hosted services.

17. Moonshot AI. Kimi K3 model card and license. July 2026. https://huggingface.co/moonshotai/Kimi-K3. Architecture, parameter counts, 1M context window, agentic benchmark results and the Kimi K3 license, including the internal-use exemption.

18. Stingrai. State of Penetration Testing 2026. 2026. https://www.stingrai.io/blog/state-of-penetration-testing-2026. Engagement dataset of 1,206 verified findings across 55 tests, severity distribution by surface, and false-positive rate.

19. Stingrai. Snipe. 2026. https://www.stingrai.io/snipe. Autonomous web application penetration testing agent: specialised agent fleet, training corpus, black-box, grey-box and white-box coverage, AutoFix pull requests and pull request gating.