PentAGI, Shannon, Strix and Cairn: Open-Source AI Pentest Agents Compared (2026)

Open-source AI penetration testing agents are now part of real intrusions as well as real security programs. On 30 October 2026, Google Threat Intelligence Group (GTIG) and Mandiant reported that since late August 2026 they have observed threat actors operating open-source multi-agent pentesting platforms, among them ARTEX, PentAGI and CAIRN, and possibly custom harnesses, within Mandiant incident response engagements. Agents of this kind also draw heavy attention on GitHub. As of 13 November 2026, Strix has 75,209 stars, Shannon 51,799, PentAGI 34,048 and Cairn, the project we take Google's CAIRN to be, 5,100.

Quick answer: PentAGI, Shannon, Strix and Cairn are four open-source agents that plan and run penetration testing steps with a large language model, and they are built for different jobs. PentAGI is a self-hosted, general-purpose platform with a web interface that runs Kali Linux tools inside Docker sandboxes. Shannon tests web applications and APIs with the source code in hand and reports only what it managed to exploit. Strix is a command-line team of agents for applications, source code and APIs that fits into CI/CD. Cairn is a general search engine for goal-directed problems, first validated in a penetration testing competition. Every result these projects publish is self-reported. An agent run is not a scoped penetration test, and running one safely needs written authorization, an isolated host, test credentials and logging your SOC knows about.

This comparison was built from each project's own repository and documentation, the GitHub API and Google's report, all read on 13 November 2026. The four agents appear in alphabetical order in the tables and sections below and are not ranked. Numbers published by the project that benefits from them are labeled self-reported, and every figure links to its publisher so any claim can be checked.

The four agents at a glance

Figure 1: The four agents in alphabetical order. A star count measures attention, not test quality. Sources: each project's repository and documentation and the GitHub REST API, read 13 November 2026.

Key takeaways

These are four different products, not four versions of one. Shannon needs your source code and covers five vulnerability classes in web applications and APIs. Strix accepts code, URLs, domains, IP addresses and API specifications. PentAGI takes any target you describe and runs general Kali Linux tooling. Cairn is a search engine that happens to have been proven on penetration testing first.

Isolation is the operator's job, and the defaults matter. PentAGI's README says each flow sandbox publishes two TCP ports for out-of-band callbacks on every interface of the Docker host by default. Cairn's example configuration runs its worker container on the host network. Strix's documentation says a local source directory is mounted into the sandbox live and writable. None of these is a flaw in the tool. Each is a decision the person running it has to make on purpose.

Google's report changes the operating picture for both sides. A team that runs an agent of this kind is running the class of software Mandiant has now met in incident response, PentAGI by name, so the run has to be distinguishable from an attack. A defender now has to assume that the scanning and exploitation tempo these tools produce can arrive without a human at the keyboard.

How this comparison was built

The comparison covers four projects that we selected. It is not a complete list of open-source agents, or of the frameworks in Google's report. PentAGI is described in that report. Google also names a CAIRN without describing it; we match that name to oritera's Cairn, as explained below. Shannon and Strix are not in the report, and are two widely starred projects that security teams ask about alongside the others. For each one we read the README, the licence file, the safety and configuration documents where they exist, any published benchmark material, and the repository metadata returned by the GitHub REST API.

Three limits apply. First, this is a documentation comparison. We did not run the four tools against a common target, so nothing here says which one finds more. Second, safety controls are described as each project documents them, and a documented control is not proof that it holds under pressure. Third, Google's report gives no description of CAIRN and links no repository. oritera's Cairn is the only widely used open-source project of that name we found on GitHub that is tied to penetration testing, so we treat it as the one Google means. That identification is ours, not Google's.

ARTEX is covered in its own section below and is not in the comparison, because its original repository is no longer available to read.

The agents one by one

Cairn

Cairn describes itself as a general-purpose problem-solving engine that defines no roles and no workflows. Given an origin and a goal, it searches for a path through an unknown state space, and its README presents penetration testing as the first domain where that approach was validated.

Architecture. The engine is a blackboard with an explicit graph of facts and intents. A fact is a confirmed finding, an intent is a direction not yet explored, and a hint is human judgment that an operator can inject at any time. Worker agents have no fixed roles and coordinate only through that shared graph. A server keeps it consistent, and a dispatcher schedules tasks and starts one worker container per project.

Models. Cairn does not call a model API itself. Its workers are existing coding agents, and the README lists Claude Code, Codex and Pi as supported backends, each pointed at whichever model endpoint the operator configures.

What it tests. The README frames goals such as getting a shell or capturing a flag and does not describe a vulnerability report as a deliverable, so a team that expects a findings document should plan the reporting separately.

Licence and maturity. AGPL-3.0, which the README describes as for personal and educational use, with a commercial licence on request for commercial or proprietary use. The repository was created on 19 April 2026, the latest tagged release is v0.2.1 of 10 May 2026 and the last push was on 7 September 2026.

Safety defaults to check. The recommended setup runs one worker container per project, with task timeouts and caps on concurrent workers. Two defaults need a decision: the example configuration puts the worker container on the host network, and the dispatcher connects to Docker through the host socket. An optional local mode runs agents with the operator's own permissions and no sandbox, as the README says plainly.

Published claim. The README reports that Cairn solved 54 of 54 problems and finished third in the second edition of the Tencent Cloud Hackathon AI penetration testing challenge, which it says drew 610 teams and 1,345 participants. That is the project's own account. The README adds that the system had never been tested before the competition.

PentAGI

PentAGI (Penetration testing Artificial General Intelligence) is an open-source, self-hosted multi-agent system developed by VXControl. The operator describes a target and a goal in plain language. A team of agents plans the work, runs security tools inside a Docker sandbox, keeps what it learned and produces a report.

Architecture. A Go backend with a web interface, REST and GraphQL APIs and a Docker Compose deployment. Specialized agents handle planning, testing, coding, search, memory and reporting. Results and successful approaches are stored in PostgreSQL with pgvector, and an optional Neo4j knowledge graph tracks hosts, services and findings. A flow can run in an automation mode, end to end, or in an assistant mode where a person steers each step.

Models. The README lists 12 providers, from OpenAI, Anthropic and Google Gemini to DeepSeek, Qwen and Mistral, plus any OpenAI-compatible endpoint and local models through Ollama or vLLM, and VXControl's site counts 12 built-in providers as of PentAGI 2.2. For open models under 32 billion parameters the README calls its execution monitoring and task planning features essential, and says they cost two to three times the execution time and token use.

What it tests. Whatever the operator describes. The sandbox runs more than 20 standard tools, with nmap, Metasploit and sqlmap given as examples, from a Kali Linux image.

Licence and maturity. MIT for the source code, with an end user licence agreement that restricts use to authorized testing and makes the operator solely responsible for what the agents do. The repository was created on 6 January 2025 and the latest release is v2.2.0 of 5 October 2026.

Safety defaults to check. Commands run in sandbox containers, hard limits on tool calls are always active, and the web interface binds to localhost by default. The README is frank about the sharp edges, and three need action before a first run:

• Each flow sandbox publishes two TCP ports for out-of-band callbacks, from the range 28000 to 29999, on every interface of the Docker host by default, and Docker routes published ports around common host firewall rules.

• A fresh installation creates a default administrator account with a documented password that must be changed at first login.

• Mounting the host's Docker socket into sandboxes gives an autonomous agent the host's Docker API, which the README says is enough to compromise the entire node. It recommends a separate, hardened Docker daemon on a dedicated worker server.

Scope is written into the task as rules of engagement. VXControl's site says network egress is filterable per run. The README documents no setting for it and tells operators to restrict sandbox traffic at the network level.

Published claim. The README publishes no penetration testing benchmark score.

In Google's report. PentAGI is the second framework Google describes after ARTEX. Most observed PentAGI activity stayed at WHOIS asset mapping or active vulnerability scanning. Some operations did progress to active intrusions and persistence across multiple organizations. Google also identified a customized attack orchestration platform built on the open-source edition, likely used between at least late August and early October 2026 for systematic reconnaissance and vulnerability scanning against organizations in Asia, including system integrators and major telecommunications firms, and one instance of PentAGI chained with proxy services to target the aerospace sector.

Shannon

Shannon is an autonomous AI penetration testing agent for web applications and APIs, developed by Keygraph. It reads the application's source code to find candidate attack paths, then uses browser automation and command-line tools against the running application, and it puts a vulnerability in the report only when it produced a working proof of concept.

Architecture. A TypeScript command-line tool that starts an ephemeral Docker worker and mounts the target repository read-only. The pipeline has six stages: reconnaissance and vulnerability analysis by specialized agents, an optional agentic static analysis of the code, reconciliation of the two candidate lists, exploitation, validation that discards anything not demonstrated, and reporting. Output is a PDF and Markdown report plus JSON and SARIF 2.1.0. Official integrations exist for GitHub Actions and GitLab CI/CD, and a severity gate counts only exploited findings.

Models. Anthropic, OpenAI, xAI and AWS Bedrock are built in. Other providers, gateways and local models served through Ollama, vLLM or LM Studio are supported, and one model is used for the whole run. The README warns that Anthropic and OpenAI apply real-time safeguards to security workloads that can interrupt a scan, and tells users to complete those providers' guidance for legitimate testers first.

What it tests. The open-source edition needs source code. Its safety and limitations guide lists five classes: broken authentication, broken authorization, injection, cross-site scripting and server-side request forgery. Because it reports only what it can exploit, it does not report many vulnerable dependency, insecure configuration or policy findings. Black-box testing without source code and business logic testing are listed as features of Keygraph's commercial editions.

Licence and maturity. AGPL-3.0, with commercial licensing for organizations that need other terms. The repository was created on 27 September 2025 and the latest release is v3.5.1 of 11 November 2026. Keygraph is not accepting external code contributions at present.

Safety defaults to check. The documentation requires explicit written authorization and says not to run Shannon against production systems, because exploitation can create users, modify or delete data and trigger outbound traffic. Rules of engagement and avoid and focus rules go in a configuration file, and the exploitation phase can be switched off. The guide warns against pointing Shannon at untrusted codebases, because malicious content in a repository can steer a tool that reads it, and says final reports can contain weakly supported or incorrect details, so human review is essential.

Published claim. Keygraph published a follow-up to a study by a security consultancy of two commercial platforms, which Keygraph itself describes as sponsored by one of them. It ran Shannon v3 against the same open-source application, Photoview 2.4.0, with source code and administrator credentials. Across three models, ordered from the most capable to the cheapest, it reports 24, 10 and 18 findings, of which it counted 23, 10 and 18 as true positives, at model costs of US$115, US$35.07 and US$6.10. Against seven later-patched vulnerabilities that Keygraph identified from the project's commit history, the three runs caught 6, 3 and 3. Keygraph scored its own runs. The single false positive is instructive: the agent read an intended administrator feature as a flaw, and Keygraph names better understanding of business logic as work in progress.

Strix

Strix is an open-source AI penetration testing tool from Strix, whose site names the company as OmniSecure, Inc. It runs autonomous agents that execute the code or application dynamically, look for vulnerabilities and try to confirm each one with a proof of concept.

Architecture. A Python command-line tool with an interactive terminal interface and a headless mode for servers and pipelines. Agents are organized as a graph, with specialized agents for different attacks and assets that share discoveries. They work inside a Kali Linux based Docker container equipped with an HTTP interception proxy, an automated browser, a terminal, a Python runtime and a long list of preinstalled open-source security tools.

Models. The documentation covers OpenAI, Anthropic, Google Vertex AI, AWS Bedrock, Azure OpenAI, OpenRouter, Vercel AI Gateway, Novita AI and local models. The README marks three of its recommended frontier models as needing access through the providers' cyber programs. The local models page warns that most local models, especially those under 70 billion parameters, struggle with the agentic tasks Strix depends on, and recommends them only when privacy is the overriding concern.

What it tests. Targets can be a local directory, a GitHub repository, a web application URL, a domain, an IP address or an API specification. The README claims coverage across access control, injection, server-side and client-side flaws, business logic, authentication, infrastructure and cloud misconfiguration, and API security. Three scan modes trade depth for time: quick in minutes, standard in 30 minutes to an hour, and deep, the default, in one to four hours.

Licence and maturity. Apache-2.0. The repository was created on 5 August 2025 and the latest release is v1.7.1 of 10 November 2026. It is the most starred of the four, and GitHub counted 603 open issues and pull requests on the day we read it.

Safety defaults to check. A budget cap and a per-agent turn limit, 500 by default, bound a run, and base URLs declared in an API specification are authorized as in-scope hosts. Three behaviors need a decision before the first run. A local directory is mounted into the sandbox live and writable, so the agent edits the real files. Credentials for authenticated testing are passed as free-text instructions, which makes them part of the prompt. Telemetry is on by default and can be switched off with a documented setting. The documentation index lists no safety or limitations page comparable to Shannon's.

Published claim. Strix's benchmarks repository reports that Strix v0.4.0 solved 100 of 104 challenges, 96 percent, on the XBOW validation benchmarks in black-box mode with a Gemini 3 Pro preview model. The harness is public, which is to Strix's credit. The figure is still the project's own and dates from a much earlier version.

ARTEX: why it is not in the comparison

ARTEX is the framework Google's report is named after, and people searching for these tools will meet the name. It is not compared here because there is no original repository left to read. Google reports that the creator took the repository down in October 2026 because of widespread threat actor abuse, and that the framework continues to spread through community forks, including localized variations and a commercial version. The original repository address returned a not-found page when we checked on 13 November 2026.

What is known comes from the incident reports. Google describes ARTEX as an autonomous, multi-agent, LLM-driven pentesting and offensive command-and-control platform, written in Go with an embedded Next.js web console and built around six specialized sub-agents. Microsoft adds that it can use any AI model. CrowdStrike found an instance using a DeepSeek model as its primary backend.

A tool whose author withdrew it, and which now circulates as forks of unknown provenance, has no accountable maintainer we could identify and no trustworthy source to install from. Our analysis of what Google observed with ARTEX and what to test covers the intrusions themselves.

Benchmark claims: who made them and who checked

Figure 2: Published results for the four agents. Every figure was published by the project it describes. Sources: each project's repository, the XBOW validation benchmarks repository and Google's report of 30 October 2026.

None of these results was independently reproduced in any source we read. Three things change how they should be read.

The setting decides the number. A capture-the-flag benchmark has one hidden flag per challenge and a clear success condition. A real application has no flag, and the hard part is deciding what counts as a vulnerability at all. On CVE-Bench, built from critical real-world web application CVEs, the best agent framework its authors evaluated resolved up to 13 percent of vulnerabilities in a March 2025 paper (arXiv 2503.17332).

A benchmark can run out. The XBOW set was released in 2024. Its publisher now states in its own repository that the set is outdated and saturated, that typical performance is approximately 100 percent and that it no longer separates one framework from another (XBOW validation benchmarks). A high score on it shows that a framework works, not that it is better than the next one.

The model moves the result as much as the agent. In Keygraph's runs the same Shannon version caught 6 of 7 known flaws with the most capable model and 3 of 7 with each cheaper one. PentAGI, Shannon and Strix each warn in their documentation that small or local models give weaker results.

Our AI pentesting benchmark scoreboard applies the same questions to the wider field: what was measured, what the agent was handed, and who confirmed the finding.

What Google's report means if you want to run one

Google places these frameworks in a long line of tools built for security professionals and adopted by attackers, and names Cobalt Strike and Metasploit as the earlier examples. Nobody concluded that defenders should stop using Metasploit. They concluded that its use has to be authorized, contained and visible. The same applies here, with less margin, because an agent acts faster than the person supervising it. Google observed individual ARTEX execution sequences running in bursts of roughly 2 to 25 seconds.

Authorization

All four projects say it in their own documentation: test only what you own or have explicit permission to test, and Shannon and Strix both specify written permission. For an organization, written permission means more than a manager's approval.

• A named scope and a time window, signed by someone with authority over every system in it, including systems run by a hosting provider or a SaaS vendor. Cloud platforms publish their own testing policies, covered in our guide to cloud penetration testing rules of engagement.

• A decision about production. Shannon's documentation says not to run it against production systems at all. Our note on running an autonomous pentest against production sets out the conditions under which that risk is acceptable.

• The model provider's terms. Strix and Shannon both tell users that frontier providers gate or interrupt security workloads unless the account has been cleared for them. Routing requests through an LLM-proxy service to hide what they are for is a behavior Google attributes to threat actors, and CrowdStrike found an ARTEX instance reaching its model through what was likely an API proxy or reseller. It is not a shortcut for a legitimate team. Our comparison of AI models for penetration testing covers what each provider allows.

• The licence. MIT and Apache-2.0 place few conditions on internal use. AGPL-3.0, used by Shannon and Cairn, carries source-sharing obligations when modified software is offered over a network, and both projects offer commercial terms.

Isolation

Treat the agent's host as hostile from the first minute. It executes commands chosen by a language model, it reads content written by whoever controls the target, and a prompt injection in a page or a repository can redirect it.

• Give it its own machine. A disposable virtual machine or a dedicated worker node, not an engineer's laptop and not a shared build server. PentAGI's and Shannon's documentation both recommend this.

• Keep the Docker socket away from the sandbox. PentAGI's README explains that mounting the host socket into a sandbox gives an autonomous agent control of the whole node. In PentAGI's default Compose file and in Cairn's dispatcher the orchestrator itself holds the host socket, which is one more reason to give either tool a machine of its own.

• Control the network both ways. Restrict what the sandbox can reach to the agreed targets and the model endpoint, and restrict what can reach the sandbox. PentAGI's default callback ports stay reachable until a firewall in front of the host says otherwise.

• Keep management interfaces private. Change default credentials before anything else, leave the web interface bound to localhost or behind a VPN, and do not publish the local viewers or APIs. An exposed agent console is an attack tool waiting for an operator.

• Avoid unsandboxed modes and protect the working copy. Cairn's optional local mode removes the only boundary between the model's decisions and your workstation, and Strix edits a mounted local directory in place, so point it at a clean clone.

Credentials

In several of the intrusions Google describes, the adversary supplied ARTEX with compromised credentials and instructed it to carry out the following stages. Whatever you give an agent, assume it will use every permission attached to it, and assume the credential itself may leak.

• Test accounts only, created for the run, one per role, with no access to real customer data, and deleted or rotated afterwards.

• No long-lived cloud keys and no administrator sessions. If the test needs cloud access, issue a short-lived role scoped to the test resources. Mandiant's guidance in the same report is to replace long-lived secrets with short-lived workload credentials.

• Remember where the prompt goes. With a hosted model, source code, target details, findings and any credentials written into the instructions are sent to the model provider. A locally hosted model keeps that traffic inside your environment at a cost in capability that the PentAGI, Shannon and Strix documentation all acknowledge. Our AI pentest data handling checklist lists the questions to settle first.

• Guard the agent's own secrets. The host holds model API keys, target credentials and everything the agent found, so treat it as a high-value system and wipe it afterwards.

Logging

A run that your SOC cannot tell apart from an intrusion is a failed run, whatever it finds.

• Announce it. Give the SOC the source addresses, the time window and the target list in advance, and agree how to stop the run.

• Keep the agent's own record. All four keep one: PentAGI in PostgreSQL, Shannon as pipeline artifacts, Strix on disk and Cairn in its fact and intent graph. Copy it somewhere the agent cannot write to, which is what OWASP's Autonomous Penetration Testing Standard asks for.

• Use the run as a detection test. Mandiant recommends watching identity velocity, API bursts and new outbound destinations. An authorized agent run produces all three on demand, so record which alerts fired and how long a person took to respond. Our detection and response benchmarks explain how to score that.

• Review before you act on findings. An unreviewed agent report sent to engineering spends their time on the false positives.

What it means for defenders

Google's report is about attackers, and our ARTEX analysis covers it in full, with Mandiant's five defensive layers. Three of its points bear on these tools. The techniques are not new: Google says the mechanics of these intrusions remain the same, such as compromised credentials and unpatched systems, and PentAGI's sandbox runs the standard tools that ship with Kali Linux, so existing detection content still applies and what changes is the tempo. The current tooling is loud: one partial command execution by ARTEX generated 240 log events, and Google writes that such frameworks "currently trade stealth for velocity", which makes response time the harder problem. And names will not hold: Mandiant notes that tool names, hashes and IP addresses will not cover much of this activity, and Google saw a customized platform built on open-source PentAGI.

In practice, treat unannounced agent-style scanning as an incident unless it matches a registered test, watch production servers for new outbound destinations such as external AI platforms and webhook services, and find any instance of these tools inside your own organization before someone else does. Google expects attacks with little to no human interaction in the near term, which it gives as 1 to 3 months, and stealthier, fully autonomous attacks in the medium term, 3 to 6 months. Those are forecasts dated 30 October 2026, not observations.

Safety controls measured against OWASP APTS

The OWASP Autonomous Penetration Testing Standard (APTS) is a governance standard for autonomous testing platforms, at version 0.1.0, with 173 tier-required requirements across 8 domains in three cumulative tiers (OWASP APTS). Its first tier, 72 requirements, comes down to three promises: the platform will not test outside scope, it can be stopped immediately, and it provides an audit trail. The standard has no certification body and can be self-assessed, and none of the four READMEs mentions it. It is still a useful yardstick.

Its starting assumption, set out in the standard's introduction, is that the agent runtime may attempt actions outside its mandate for any reason, so the platform must make such attempts fail by construction. In concrete terms APTS asks for a kernel-enforced sandbox whose boundary the agent has no credentials to move, a tool and action allow-list enforced by a component outside the model and not by the system prompt, and an audit trail the agent cannot reach. The table below sets what each project documents beside those expectations.

The table reads the documentation only, and works best as a list of questions to put to any agent before it touches a real system. Our explainer on what OWASP APTS requires goes through the tiers, and the AI pentesting tools guide applies the same questions to commercial products.

Where these agents stop and where human penetration testers are needed

The honest answer comes from the projects' own documentation and from independent research, and the two agree.

The same research shows what agents are good at. In the study behind the false-positive figure, the authors' own multi-agent framework placed second overall against ten cybersecurity professionals, with 9 valid vulnerabilities at an 82 percent valid submission rate. Breadth, systematic enumeration and repetition on every build are real strengths, and they are why Mandiant's own guidance recommends that defenders use LLM-enabled tooling, including agentic source-code review and continuous AI-assisted vulnerability testing, to find attack paths before an attacker does.

What remains with people is judgment and accountability.

• Deciding what the application is for. Whether a user should be able to see another tenant's invoice or approve their own refund is a question about the business. Authorization and business logic flaws are where an agent most needs direction.

• Stopping, confirming and ranking. A human tester decides when a chain has proven its point and when the next step would do damage, whether a finding is real and what to fix first. An agent optimizes for the goal it was given.

• Scope beyond the web application. None of the four READMEs names Active Directory, mobile applications, Wi-Fi, phishing or physical security as a coverage area. Those are separate disciplines with their own methods.

• Standing behind the result. A customer, an auditor or a regulator who asks for a penetration test report is asking for a named, qualified party to take responsibility for the scope, the method and the conclusions. A self-run tool produces evidence, not that assurance.

The practical division is the one Shannon's authors describe themselves: run an agent often, on staging, across the long tail of applications and releases, and bring in people for depth and for the report that other parties rely on. Our guides on splitting an attack surface between autonomous and human testing and on running a 30-day AI pentest bake-off on your own application turn that into a plan.

Where Stingrai fits

Stingrai is a CREST-accredited penetration testing company, accredited for both Penetration Testing and AI-Enabled Penetration Testing, offering fully human-led and hybrid (human penetration testers plus AI agents) penetration tests, delivered by experienced penetration testers who are also world-class security researchers and bug bounty hunters, with OSCE³, OSWE, OSEP and CREST CRT certifications across the team. Founded in Toronto, Canada in 2021, it works with companies from startups to enterprises across web, API, mobile, AI and LLM, cloud, network, Active Directory and social engineering penetration tests and red team engagements.

For teams weighing an open-source agent, three parts of that are relevant.

• Snipe. Snipe is Stingrai's autonomous AI agent for web applications and associated APIs, with a specialist sub-agent for AI and LLM applications. It coordinates specialized agents across authentication, access control, business logic and injection testing, works black-box, grey-box with test accounts for each role, and white-box with repository access, and was trained on more than 6,000 HackerOne Hacktivity disclosure reports and on the methodology of Stingrai's penetration testers. It is delivered as part of a Stingrai engagement that starts with a confirmed scope and a signed statement of work, not as a tool to download. Findings come with proof-of-concept evidence, and with repository access Snipe proposes AutoFix pull requests and runs automated retests.

• Hybrid and human-led testing. In a Hybrid Pentest, Stingrai's penetration testers work alongside Snipe throughout the engagement, directing where it hunts and pursuing what it surfaces. Human-led engagements are staffed by two named penetration testers from a team with 18 published CVEs and reviewed by the team lead. Human-led and hybrid reports come with an attestation letter, and retesting of fixed findings is included. This is the option when a customer or auditor will rely on the report.

• Testing how you would cope with an agent-driven attacker. AI and LLM penetration testing covers AI agents and their tool integrations, including tool misuse, excessive agency, privilege escalation through tool chains and MCP. Red team and purple team engagements measure whether detection and containment keep up with activity at automated tempo.

Engagements run as one-time annual tests or as continuous programs, with findings posted to the PTaaS platform as they are confirmed. Stingrai's published fixed prices cover one web application and associated APIs: the Autonomous Pentest, which is Snipe on its own and delivers a pentest report, is US$3,000 one-time or US$650 a month on a 12-month plan, and the Hybrid Pentest is US$6,800 one-time or US$1,275 a month (pricing). Network, Active Directory, cloud, mobile and red team work is scoped per engagement.

Frequently Asked Questions

What is PentAGI?

PentAGI is an open-source, self-hosted multi-agent system for autonomous penetration testing, developed by VXControl and released under the MIT licence. The operator describes a target and a goal in plain language, and a team of AI agents plans the work and runs standard security tools such as nmap, Metasploit and sqlmap inside Docker sandboxes. It supports 12 built-in model providers plus local models, and Google reported on 30 October 2026 that threat actors have also used it in real intrusions.

What is the best open-source AI pentesting tool?

There is no single best one, because the four projects compared here do different jobs. Shannon suits teams that have the source code of a web application or API and want exploit-confirmed findings in CI/CD. Strix accepts code, URLs, domains and API specifications from the command line. PentAGI is a general-purpose platform with a web interface, and Cairn is a goal-directed search engine first validated in a penetration testing competition. No independent test has compared them on a common target, and every published result is self-reported.

What is Shannon AI?

Shannon is an open-source autonomous AI penetration testing agent for web applications and APIs, developed by Keygraph under the AGPL-3.0 licence. It analyzes the application's source code, attacks the running application, and reports a vulnerability only when it has produced a working proof of concept. The open-source edition covers broken authentication, broken authorization, injection, cross-site scripting and server-side request forgery, and its documentation says not to run it against production systems.

What is Strix AI?

Strix is an open-source AI penetration testing tool released under the Apache-2.0 licence. It runs a team of autonomous agents inside a Kali Linux based Docker sandbox against local code, repositories, web applications, domains, IP addresses or API specifications, from an interactive command line or headless in CI/CD. Strix reports that version 0.4.0 solved 100 of 104 challenges on the XBOW validation benchmarks, a figure that is self-reported on a benchmark its publisher now calls saturated.

Have open-source AI pentest agents been used in real attacks?

Yes. Google Threat Intelligence Group and Mandiant reported on 30 October 2026 that since late August 2026 they have observed threat actors operating open-source multi-agent pentesting platforms, among them ARTEX, PentAGI and CAIRN, and possibly custom harnesses, within Mandiant incident response engagements. Most observed PentAGI activity stayed at WHOIS asset mapping or active vulnerability scanning, but some operations progressed to active intrusions and persistence across multiple organizations.

Is it safe to run PentAGI, Shannon or Strix on my own systems?

It can be, with four conditions in place: written authorization for a named scope and time window, an isolated host with controlled network access, test credentials only, and logging that your SOC knows about in advance. The defaults need attention. PentAGI publishes sandbox callback ports on every interface of the Docker host by default, Strix mounts a local source directory writable, and Shannon's documentation says to use staging or disposable environments and not production.

Which AI models do these agents run on?

All four are bring-your-own-model tools. PentAGI lists 12 built-in providers plus any OpenAI-compatible endpoint and local models through Ollama or vLLM. Shannon has Anthropic, OpenAI, xAI and AWS Bedrock built in. Strix documents OpenAI, Anthropic, Vertex AI, Bedrock, Azure and several gateways. Cairn uses Claude Code, Codex or Pi as its workers. The projects' own documentation warns that small or local models give weaker results, and frontier providers gate offensive security work behind verification programs.

Are the benchmark results for these tools independently verified?

Not in any source we read. Strix published its own result on the XBOW validation benchmarks, Keygraph scored Shannon's runs on one open-source application itself, and Cairn's README reports a hackathon result. PentAGI's README publishes no test score. The independent research that exists, such as CVE-Bench and a live-network study by Stanford, Carnegie Mellon and Gray Swan AI, evaluates other agent frameworks, including the study authors' own, and shows both real capability and higher false-positive rates than human professionals.

Can an open-source AI pentest agent replace a penetration test?

No. Shannon's README says it is built to work alongside expert testers, not to replace them, and that its findings still require human review. The agents cover a narrower set of vulnerability classes than a penetration test scope, need direction on business logic and authorization, and produce evidence without a named, qualified party standing behind the result. They are useful for frequent testing of staging environments between human-led engagements.

How can defenders tell an AI pentest agent from a human attacker?

By tempo and noise more than by tool names. Google observed ARTEX execution sequences in bursts of roughly 2 to 25 seconds and found the activity far noisier than a competent human operator, with 240 log events from one partial command execution. Mandiant advises that tool names, hashes and IP addresses will not cover much of this activity, and recommends detecting changes in identity velocity, API call bursts and new outbound destinations.

Related reading

• ARTEX and Agentic Offensive Frameworks: What Google Observed and What to Test (2026)

• AI Pentesting Benchmark Results 2026: A Tracked, Sourced Scoreboard

• Best AI Pentesting Tools 2026: Hybrid vs Autonomous for AppSec Teams

• Best AI Model for Pentesting 2026: Claude, GPT, and Open Weight Models Compared

• What Is OWASP APTS? The Autonomous Pentest Governance Standard, Explained

• Autonomous or Human Pentesting: How to Split Your Attack Surface by Performer

• The 30 Day AI Pentest Bake-Off: A Pilot Playbook and Scorecard for Your Own App

References

1. Google Threat Intelligence Group and Mandiant. ARTEX and Friends: Enter Agentic Offensive Frameworks. 30 October 2026. https://cloud.google.com/blog/topics/threat-intelligence/artex-and-friends-enter-agentic-offensive-frameworks. Observations of ARTEX and PentAGI in intrusions, and Mandiant's five layers of defense.

2. VXControl. PentAGI repository, README and end user licence agreement. Read 13 November 2026. https://github.com/vxcontrol/pentagi. Architecture, model providers, sandbox and deployment guidance, licence.

3. VXControl. Company site. Read 13 November 2026. https://vxcontrol.com/. Provider count and release timeline for PentAGI.

4. Keygraph. Shannon repository, README, safety and limitations guide, configuration guide and benchmark write-up. Read 13 November 2026. https://github.com/KeygraphHQ/shannon. Pipeline, coverage, safety guidance and Keygraph's own Photoview results.

5. Strix. Strix repository and documentation. Read 13 November 2026. https://github.com/usestrix/strix and https://docs.strix.ai/llms.txt. Targets, scan modes, sandbox, options and local model guidance.

6. Strix. Benchmarks repository, XBEN results. Read 13 November 2026. https://github.com/usestrix/benchmarks/tree/main/XBEN. Strix v0.4.0 result on the XBOW validation benchmarks.

7. XBOW. Validation benchmarks repository. Read 13 November 2026. https://github.com/xbow-engineering/validation-benchmarks. The 104-challenge set and its publisher's notice that it is outdated.

8. oritera. Cairn repository, README and example dispatcher configuration. Read 13 November 2026. https://github.com/oritera/Cairn. Architecture, worker backends, licence and reported competition result.

9. OWASP. Autonomous Penetration Testing Standard (APTS), version 0.1.0, README and Introduction. Read 13 November 2026. https://github.com/OWASP/APTS. Domains, tiers, conformance model and containment assumptions.

10. Stanford, Carnegie Mellon and Gray Swan AI. Comparing AI Agents to Cybersecurity Professionals in Real-World Penetration Testing. arXiv 2512.09882, submitted 10 December 2025. https://arxiv.org/abs/2512.09882. Ten professionals against AI agents on a live university network.

11. Zhu, Kang and co-authors. CVE-Bench: A Benchmark for AI Agents' Ability to Exploit Real-World Web Application Vulnerabilities. arXiv 2503.17332, submitted 21 March 2025. https://arxiv.org/abs/2503.17332. Exploitation of critical-severity web application CVEs by agent frameworks.

12. Microsoft Security Research and Microsoft Threat Intelligence. AI-driven, large-scale nation-state campaign using ARTEX enables supply chain compromise. 21 October 2026. https://www.microsoft.com/en-us/security/blog/2026/10/21/storm-0940-uses-artex-agentic-ai-connectwise-supply-chain-compromise/. Description of ARTEX.

13. CrowdStrike Counter Adversary Operations. Unknown Threat Actor Uses AI-Driven ARTEX to Target South Korean Finance. 7 October 2026. https://www.crowdstrike.com/en-us/blog/unknown-threat-actor-uses-artex-to-target-south-korean-finance/. Model backend of an observed ARTEX instance.

14. GitHub. REST API repository metadata. Read 13 November 2026. https://docs.github.com/en/rest/repos/repos. Dates, releases, licences and star counts for the four repositories.

Talk to Stingrai

Tried an open-source agent and want to know what it missed, or need a penetration test report that a customer or auditor will accept? Stingrai scopes autonomous, hybrid and human-led tests of web applications and APIs, with retesting, as a one-time annual engagement or a continuous program. Get a quote.