A vulnerability assessment finds and ranks known weaknesses across your systems. A penetration test proves which of those weaknesses an attacker can actually exploit, and what they reach when they do.
Quick answer: A vulnerability assessment finds and ranks known weaknesses; a penetration test proves which of them an attacker can exploit, and PCI DSS v4.0.1 keeps the two as separate requirements (11.3 scanning, 11.4 testing). The guide covers definitions, a side-by-side comparison, which one you need, what SOC 2, ISO 27001, HIPAA and CMMC require, and how both combine. Stingrai delivers the penetration-test half as a one-time annual test or continuous programme, with Snipe hunting IDOR, business-logic and authorisation flaws scanners miss.
The one-line difference: a vulnerability assessment tells you what might be exploitable, a penetration test tells you what is.
Both aim to uncover security weaknesses, but they differ in approach and depth, and that difference decides which one satisfies an auditor, which one belongs in your release pipeline, and which one you should be buying this quarter. This guide gives the definitions, a full side-by-side comparison, a decision framework by scenario, and a clause-level account of what SOC 2, ISO/IEC 27001:2022, PCI DSS v4.0.1, HIPAA and CMMC actually require.
Definitions
Vulnerability assessment (VA): A broad, largely automated review that discovers, catalogues and prioritises known security weaknesses across a defined set of systems, without confirming that any of them can be exploited. Output is a ranked inventory of potential issues.
Penetration test (PT): An authorised, objective-driven simulated attack in which skilled testers actively attempt to exploit weaknesses, chain them together, and demonstrate real business impact. Output is a set of verified findings with proven attack paths.
Vulnerability scan: The automated tooling step inside a vulnerability assessment. A scan is one input to an assessment. Neither is a penetration test.
Penetration testing vs vulnerability assessment: side-by-side comparison
Dimension | Vulnerability assessment | Penetration test |
|---|---|---|
Goal | Identify and rank potential weaknesses | Confirm which weaknesses are exploitable and what they expose |
Core question answered | What could be wrong here? | What can an attacker actually do here? |
Method | Automated scanning, configuration review, version and patch checks | Manual tradecraft plus tooling: reconnaissance, exploitation, privilege escalation, chaining, post-exploitation |
Depth | Surface level, breadth over depth | Deep, targeted, follows attack paths across systems |
Automation vs human | Predominantly automated; human effort is mostly triage | Human-led by definition; automation accelerates discovery but does not replace judgement |
Coverage | Broad sweep of all in-scope assets | Focused on realistic attack scenarios and high-value targets |
Output | Ranked list of potential issues with severity scores | Verified findings with reproduction steps, evidence, business impact and attack narrative |
False positives | Common, an expected cost of automation | Rare, because findings are proven by exploitation |
Typical duration | Hours to days, repeatable on demand | One to four weeks depending on scope, plus reporting and retest |
Typical frequency | Continuous, monthly or quarterly | Annually, after significant change, or continuously through a testing programme |
Who performs it | Internal security or IT staff running a scanning platform | Experienced offensive security testers, internal or external, with organisational independence where a framework requires it |
Cost model | Recurring tool subscription, usually per asset or per IP, often bundled into an existing cloud or vulnerability management platform | Priced per engagement by scope. See our penetration testing cost guide for current ranges by test type |
Compliance evidence value | Satisfies scanning obligations such as PCI DSS 11.3 and evidences vulnerability management controls | Satisfies explicit testing mandates such as PCI DSS 11.4 and CMMC Level 3, and evidences control effectiveness under SOC 2 and ISO 27001 |
What is a vulnerability assessment?
A vulnerability assessment answers one question at scale: where are the known weaknesses? Scanning platforms compare what they find against vulnerability databases and configuration baselines, then rank results by severity so remediation can be prioritised.
Assessments run in two modes, and the distinction matters more than most buyers realise:
Authenticated scanning uses credentials to inspect the system from the inside. It sees installed package versions, patch levels and configuration state, which makes it far more accurate and much less prone to false positives.
Unauthenticated scanning looks at the system the way an anonymous outsider would. It is useful for perimeter awareness but produces noisier results, because it infers versions rather than reading them.
The strength of a vulnerability assessment is coverage and cadence. It is cheap enough to run continuously across everything you own, which is exactly what you want for known, patchable, catalogued issues.
Its limit is structural. A scanner reports that a component is affected by a known vulnerability. It does not establish that the vulnerable code path is reachable in your deployment, that exploitation succeeds against your configuration, or that the result is worth anything to an attacker. It also cannot find flaws that exist only in your own logic: broken authorisation between tenants, an insecure direct object reference, a workflow that can be replayed to issue a refund twice. Those bugs have no CVE to match against, because they are unique to your application.
What is a penetration test?
A penetration test starts where the scanner stops. Testers work against defined objectives, attempt real exploitation, and chain individually minor findings into a path that reaches something that matters.
That chaining is the part automation misses. A verbose error message is noise. A predictable identifier is noise. An overly permissive role is noise. Combined in the right order they become a full account takeover, and only a tester following the path end to end will report it as one finding with critical severity rather than three low-severity notes nobody prioritises.
A credible test produces more than a findings list. It produces reproduction steps an engineer can follow, evidence of what was reached, an assessment of business impact, and a retest confirming the fix actually worked. That last artefact is the one auditors ask for most often and the one weakest reports omit.
Is a vulnerability scan a penetration test?
No. They are different activities with different outputs, and no framework treats them as interchangeable.
PCI DSS is the clearest on this point because it separates them into two distinct requirements and says so explicitly. Guidance under Requirement 11.4.1 states that a vulnerability scan is not a penetration test, that a penetration test is an active process that usually involves exploitation, and that penetration testing is a highly manual process.
This is the single most common error in vendor marketing. A product that runs automated scans and calls the output a penetration test will not satisfy PCI DSS Requirement 11.4, and it will not satisfy a CMMC Level 3 assessor, whose evidence objectives require both automated scanning tools and expert-driven ad hoc testing.
Which one do you need?
Most organisations eventually need both. The sequencing question is what actually gets asked, so here it is by scenario.
Your situation | Start with | Why |
|---|---|---|
No security testing at all today | Vulnerability assessment | Establish asset inventory and clear the known, patchable backlog before paying testers to rediscover it |
Enterprise prospect is blocking a deal on a security review | Penetration test | Buyers ask for a test report and a remediation letter, not scan output |
Preparing a first SOC 2 Type 2 | Both, test scheduled inside the observation period | Scanning evidences your vulnerability management control; the test evidences control effectiveness under CC4.1 |
In scope for PCI DSS | Both, they are separate requirements | 11.3 scanning and 11.4 testing are independent obligations; neither substitutes for the other |
Shipping application changes weekly | Continuous scanning plus application testing tied to release | An annual point-in-time test cannot cover 50 releases a year on its own |
Custom application handling money or sensitive records | Penetration test | Business logic and authorisation flaws are the dominant risk and scanners do not find them |
Large estate, unknown exposure | Vulnerability assessment first | You cannot scope a meaningful test until you know what exists |
Post-incident | Penetration test | You need to know what an attacker can reach now, not what is theoretically unpatched |
The general rule: use vulnerability assessment for coverage and penetration testing for certainty. Scanning keeps the known-issue backlog under control across everything. Testing tells you whether the things you cannot scan for will actually hurt you.
What compliance frameworks actually require
This is where most published comparisons go wrong, usually by repeating clause numbers that were retired years ago. Here is the current position, framework by framework.
Framework | Penetration testing mandated? | Scanning obligation | Frequency set by the framework |
|---|---|---|---|
PCI DSS v4.0.1 | Yes, Requirement 11.4 | Yes, Requirement 11.3, including quarterly ASV scans under 11.3.2 | Testing every 12 months and after significant change; scanning quarterly |
SOC 2 | No. Named once in the whole Trust Services Criteria, in a non-binding point of focus under CC4.1 | Not mandated, but CC7.1's point of focus is labelled "Conducts Vulnerability Scans" | None. Your own control description sets it |
ISO/IEC 27001:2022 | No. Never named in any clause or Annex A control title | Not named. A.8.8 covers management of technical vulnerabilities | None set anywhere in the standard |
HIPAA (in force today) | No. The word "penetration" does not appear in the Security Rule | Not named. Risk analysis and periodic evaluation are the obligations | None. The rule says "periodic" |
CMMC Level 1 and 2 | No | Yes, RA.L2-3.11.2 at Level 2 | Organisation-defined |
CMMC Level 3 | Yes, CA.L3-3.12.1e | Yes, plus expert-driven ad hoc testing | At least annually or on significant security change |
PCI DSS v4.0.1: the 11.3 versus 11.4 distinction competitors get wrong
Under PCI DSS v4.0.1, Requirement 11.3 is vulnerability scanning and Requirement 11.4 is penetration testing. They are separate requirements with separate clocks.
Requirement 11.3 covers internal scanning at 11.3.1 and external scanning at 11.3.2, where external scans must be performed quarterly by an Approved Scanning Vendor. Requirement 11.4 covers penetration testing: 11.4.1 requires a documented nine-element methodology, 11.4.2 is internal testing and 11.4.3 is external testing, both at least once every 12 months and after any significant infrastructure or application upgrade or change. Requirement 11.4.4 requires exploitable findings to be corrected and the testing repeated to verify the correction. Segmentation testing sits at 11.4.5 annually for all entities and 11.4.6 every six months for service providers.
Penetration testing did not always live at 11.4. Under PCI DSS v3.x it sat at 11.3, with segmentation testing at 11.3.4. Any page telling you in 2026 that "PCI DSS Requirement 11.3 requires annual penetration testing" is quoting a structure that has not been current since v4.0. The confusion is partly forgivable: PCI SSC's own Penetration Testing Guidance information supplement is version 1.1, dated 2017, and still uses the old numbering. The methodology advice in it remains useful; the clause references do not.
The tester must be a qualified internal resource or qualified external third party with organisational independence, and is explicitly not required to be a QSA or ASV. Our PCI DSS Requirement 11.4 guide covers every sub-requirement, the nine methodology elements and the evidence a QSA expects.
SOC 2
SOC 2 does not require a penetration test. The phrase "penetration testing" appears exactly once in the entire 2017 Trust Services Criteria, inside a point of focus under CC4.1 that lists evaluation types management "may include". Points of focus are illustrative rather than binding, and management may customise or substitute them.
Auditors still expect testing evidence, for a practical reason: CC4.1 requires you to perform ongoing or separate evaluations of whether controls are present and functioning, and a penetration test is the separate evaluation most service organisations choose. Your binding obligation is therefore the sentence you wrote into your own control description. Write "annual penetration test" and the auditor will test you against annual.
One trap worth naming: CC7.1 is not the pentest criterion, though it is routinely cited as one. Its point of focus is labelled "Conducts Vulnerability Scans". Hand an auditor a test report against a CC7.1 request and expect them to come back asking for scan output. This is a case where the vulnerability assessment, not the test, is the right artefact. Our SOC 2 penetration testing guide covers scoping and timing inside a Type 2 observation period.
ISO/IEC 27001:2022
ISO/IEC 27001:2022 never names penetration testing in any clause or Annex A control title, and sets no testing frequency anywhere in the standard. Controls flow from your own risk treatment under clause 6.1.3, and Annex A functions as a non-exhaustive cross-check rather than a mandatory list.
Note also that the old A.12.6.1 reference still circulating belongs to the 2013 edition. In the 2022 edition the relevant control is A.8.8 Management of technical vulnerabilities, and its title carries no frequency. A.8.29 covers security testing in development and acceptance, which is event-driven rather than calendar-based. A.5.35 is an independent review of your approach to managing information security, a governance activity that is frequently and incorrectly sold as proof that ISO demands a third-party technical test.
Where does the "annual ISO pentest" idea come from? Not from ISO 27001. It comes from ISO/IEC 17021-1:2015, which governs certification bodies and their surveillance audit cycle. That is when the auditor visits, not how often you test.
HIPAA
The HIPAA Security Rule in force today does not use the word "penetration" anywhere. The obligations are a risk analysis under 45 CFR 164.308(a)(1)(ii)(A), which requires "an accurate and thorough assessment of the potential risks and vulnerabilities to the confidentiality, integrity, and availability of electronic protected health information", and a periodic evaluation under 164.308(a)(8), which requires a "periodic technical and nontechnical evaluation". Neither names a method and neither sets a frequency.
This is changing, but it has not changed yet. The Notice of Proposed Rulemaking published in the Federal Register on 6 January 2025 (RIN 0945-AA22) would require regulated entities to conduct penetration testing at least once every 12 months and automated vulnerability scans at least every six months. The comment period closed in March 2025, and no final rule has been issued. The OMB Unified Agenda currently targets July 2027 for final action, pushed back from an earlier date. Treat annual testing as a strong risk-based practice and a likely future obligation, not as a current legal requirement.
CMMC
CMMC Levels 1 and 2 impose no penetration testing obligation on the contractor. The word "penetration" does not appear in NIST SP 800-171 Rev 2 at all. The Level 2 controls mistaken for a mandate are RA.L2-3.11.2 for scanning, CA.L2-3.12.1 for periodic control assessment and CA.L2-3.12.3 for continuous monitoring. None of them names a method.
CMMC Level 3 is the exception and contains CMMC's only explicit penetration testing mandate, CA.L3-3.12.1e, at least annually or on significant security change. Its evidence objectives require both automated scanning tools and expert-driven ad hoc tests, which is a direct statement that scanning alone is insufficient. See our CMMC penetration testing guide for scoping and scoring detail, and our cross-framework evidence guide for what assessors accept under each.
How the two combine in a modern programme
Treating this as an either/or is the actual mistake. The two techniques cover different failure modes, and a serious programme layers them.
Layer one: continuous vulnerability scanning. Authenticated scanning across the whole estate, running continuously, feeding a remediation queue with an agreed service level by severity. This handles the known, catalogued, patchable population. It is cheap, it is fast, and it should never be the thing you are proud of.
Layer two: testing in the pipeline. Known-class weaknesses are cheapest to fix before they merge. Snipe, Stingrai's autonomous web application penetration testing agent, is purpose-built for the classes generic scanners never reach: IDOR, business logic flaws and broken authorisation. It is custom-trained on more than 6,000 HackerOne Hacktivity disclosure reports and on skills distilled from our own pentesters' methodology, performs both black-box dynamic testing and white-box review against application source, generates AutoFix pull requests, and can gate pull requests so vulnerable code never merges.
Layer three: human plus AI penetration testing. Senior testers validate and extend what Snipe finds, pursue the attack chains that require judgement, and produce the report, the evidence and the retest that an auditor or an enterprise buyer will accept. Stingrai delivers this as a one-time annual penetration test or as a continuous testing programme, depending on what your release cadence, your customers and your auditor require. Both produce the same class of evidence; they differ in how often it lands. Engagement options and current packages are on our pricing page.
The division of labour is simple. Scanning keeps the known population under control. Pipeline testing stops new application flaws from shipping. The penetration test proves what an attacker can actually reach, and gives you the artefact everyone else asks for.
Frequently Asked Questions
What is the difference between a vulnerability assessment and a penetration test?
A vulnerability assessment identifies and ranks known weaknesses across your systems using largely automated scanning, without confirming that any of them can be exploited. A penetration test uses skilled human testers to actively exploit weaknesses, chain them together and demonstrate real business impact. In one line: a vulnerability assessment tells you what might be exploitable, a penetration test tells you what is. Assessments are broad, fast and repeatable; tests are deep, targeted and produce verified findings with proven attack paths.
Is a vulnerability scan a penetration test?
No. A vulnerability scan is the automated tooling step inside a vulnerability assessment, and no compliance framework treats the two as interchangeable. PCI DSS separates them into Requirement 11.3 for scanning and Requirement 11.4 for penetration testing, and guidance under 11.4.1 states directly that a vulnerability scan is not a penetration test, that a penetration test is an active process usually involving exploitation, and that penetration testing is a highly manual process.
Which is required for SOC 2?
Neither is mandated by name. "Penetration testing" appears once in the entire Trust Services Criteria, inside a non-binding point of focus under CC4.1, and there is no pentest criterion, control number or required frequency. In practice auditors expect testing evidence because CC4.1 requires separate evaluations of whether controls are functioning, and a test is the evaluation most organisations select. Scanning matters separately: the point of focus under CC7.1 is labelled "Conducts Vulnerability Scans", so a test report is the wrong artefact to offer against a CC7.1 request.
Which is required for PCI DSS?
Both, as separate requirements. Requirement 11.3 mandates vulnerability scanning, including quarterly external scans by an Approved Scanning Vendor under 11.3.2. Requirement 11.4 mandates penetration testing: internal testing under 11.4.2 and external testing under 11.4.3, both at least once every 12 months and after any significant infrastructure or application upgrade or change, with exploitable findings corrected and retested under 11.4.4. Under PCI DSS v3.x penetration testing sat at 11.3, which is why outdated pages still cite that number.
How much does each cost?
They use different pricing models. A vulnerability assessment is normally a recurring tool subscription priced per asset or per IP, and is often already bundled into a cloud platform or vulnerability management suite you own. A penetration test is priced per engagement by scope, driven by application count, user roles, network size and the depth of business logic testing required. Current ranges by test type are in our penetration testing cost guide.
How often should you run each one?
Run vulnerability scanning continuously, or at minimum monthly, because its value is coverage and freshness. Penetration testing frequency depends on your obligations and your release cadence: PCI DSS sets 12 months and after significant change, CMMC Level 3 sets at least annually or on significant security change, and SOC 2 and ISO 27001 set nothing, meaning your own policy or control description is the binding cadence. Teams shipping application changes weekly generally need testing tied to releases rather than a single annual engagement.
Can a vulnerability assessment replace a penetration test?
No, and attempting the substitution fails on both technical and compliance grounds. Technically, scanners match against known vulnerability databases, so they cannot find flaws unique to your own logic such as broken authorisation between tenants, insecure direct object references or replayable business workflows, and they cannot chain low-severity findings into a critical attack path. For compliance, PCI DSS Requirement 11.4 and CMMC Level 3 require testing explicitly, and CMMC Level 3 evidence objectives require both automated scanning tools and expert-driven ad hoc tests.
Do you need both a vulnerability assessment and a penetration test?
For most organisations, yes. They cover different failure modes. Scanning handles the known, catalogued, patchable population continuously and cheaply across your whole estate. Penetration testing proves what an attacker can actually reach and produces the report, evidence and retest that auditors and enterprise buyers ask for. If you have no testing at all today, start with a vulnerability assessment to clear the known backlog, then run a penetration test so you are not paying testers to rediscover unpatched software.
Related reading
SOC 2 Penetration Testing: What Auditors Expect and How to Scope It
Pentest Evidence Auditors Accept: SOC 2, ISO 27001, PCI DSS and CMMC
Testing with Stingrai
Stingrai is a CREST-accredited penetration testing service provider founded in 2021, with teams in Toronto and London, 18 published CVEs and a 5.0 out of 5.0 rating across 19 Clutch reviews. Our team holds OSCE3, OSCP, OSWE, OSEP, CREST CRT and CISSP certifications, and presents research at DEFCON and BSides.
Our penetration testing supports your SOC 2, ISO 27001, PCI DSS and CMMC compliance programme by producing what the auditor asks for: a documented methodology, findings mapped to the clauses you are evidencing, and remediation retesting written into the engagement rather than sold as an afterthought. We deliver this as a one-time annual penetration test or as a continuous testing programme, and both include the report, evidence and retest artefacts.
Talk to our team about scoping, or see our PTaaS platform and pricing.
