CISA Secure by Design Pledge: Evidence Matrix for Buyers

CISA's Secure by Design pledge is a voluntary, nonbinding commitment for enterprise software products and services. It contains seven goals. Signatories who make measurable progress should publicly document it, while those who cannot are encouraged to share their work and challenges. CISA pledge, page 1 The pledge is useful to software buyers when each goal can be connected to product evidence. This original matrix shows what to ask for and where a penetration test can contribute.

Quick answer. Ask a supplier for product-specific evidence under each relevant pledge goal: authentication adoption, default credential removal, weakness-class reduction, patch adoption, disclosure policy, CVE transparency and intrusion evidence. A pentest can test selected product controls and supply findings, but cannot prove the supplier's organization-wide progress on its own. Stingrai can scope human-led testing around the product behaviors a buyer needs to verify.

The seven-goal evidence matrix

The goal names and core intent come from CISA's pledge. The evidence examples are editorial buyer questions, not CISA-required documentation. A supplier may demonstrate progress in other valid ways. This table is a procurement aid, not a pledge compliance determination.

Figure 1: A buyer traces a pledge goal to the product control, evidence owner, question and testing limit. The path is an original editorial model. Source: CISA, Secure by Design Pledge.

Read the pledge as a progress commitment

The pledge asks a software manufacturer to make a good-faith effort toward its goals after signing. It gives the manufacturer discretion over how to demonstrate the core criteria. That is different from a product certification, a test report or a universal assurance that every product already satisfies every goal. CISA pledge, page 1 A buyer can still use the pledge as a strong opening question: what changed for the specific product being procured, how was that change measured, and what remains outside the current rollout?

The first procurement step is to identify the unit of evidence. A supplier may have a company-wide policy, a product-line roadmap, a release-specific change or a customer-facing control. These are different levels of proof. A public statement that MFA adoption increased may be meaningful for the supplier's program but say little about whether the purchased edition offers a secure administrator recovery flow. Conversely, one successful sign-in test can prove a selected control behavior but cannot establish an adoption trend across customers.

Ask the supplier to mark each artifact with its product, edition, deployment model, release, evidence period and owner. That simple envelope makes subsequent questions fairer. It lets the buyer distinguish evidence from the same product from a corporate statement about a different line. It also prevents a report about a staging environment from silently becoming a claim about production. This is a buyer's evidence convention, not a format required by CISA.

Why the pledge needs product-level evidence

CISA says participants can choose how to make progress and may start with a set of products while publishing a roadmap for others. It asks companies that make measurable progress to document how, and encourages public explanation of challenges when progress is not yet measurable. CISA pledge, page 1 A buyer should therefore ask which product, edition and release the supplier's public statement covers. A company-level statement may be true yet leave the purchased product outside the current rollout.

Evidence should be dated and attributable. An MFA screenshot from a marketing demo does not tell a buyer whether the feature is enabled by default for administrators in the deployed edition. A promise to reduce SQL injection does not tell the buyer what baseline and time window the supplier used. A patch portal does not show adoption. These are not reasons to dismiss a pledge; they are reasons to ask for the measurement behind a claim.

Use testing where a control has observable behavior

A scoped assessment can verify that an administrator sign-in path enforces the stated MFA behavior, that a fresh test installation does not accept a shared default password, that a selected high-risk workflow rejects a vulnerability-class test, or that a customer can retrieve an audit event after a safe action. These are concrete observations. They do not establish trends across all products or customers.

For authentication, test the named product edition with synthetic user and admin accounts. Include recovery, alternate sign-in and service identities in the scope if they matter. CISA's MFA goal speaks to increasing use across products, with emphasis on stronger methods; a single pentest can assess a feature's behavior but cannot measure adoption. CISA pledge, page 2 Ask the supplier for adoption figures and methods separately.

For vulnerability classes, ask which class the supplier chose and how it measures prevalence. CISA gives examples including SQL injection, cross-site scripting and memory safety. It even notes that better disclosure can increase CVE counts in the short term while the longer-term class reduction improves. CISA pledge, page 3 A buyer should not judge the program from raw CVE count alone. A test can confirm individual flaws or controls; engineering trend evidence has to come from the supplier's own program.

For intrusion evidence, request a log sample from the edition being purchased. Agree a low-impact test event, then ask whether the customer can retrieve actor, time, action and relevant context. CISA's goal is about increasing customers' ability to gather evidence of intrusions, not merely the existence of an internal provider log. CISA pledge, pages 6-7 The report should state what was visible to the customer and what was only visible to the supplier.

Make procurement questions specific

Ask the supplier to map each public pledge claim to a product, release and evidence owner. Ask which goal is already met, which has a dated roadmap, and which is not yet measured. Request the methodology behind any percentage, including numerator, denominator, period and excluded product lines. The pledge does not prescribe a single proof format, so the buyer's job is to decide what evidence is enough for its own risk decision.

Where an external pentest is part of the packet, write its role precisely. It can supply observed test results, findings, remediation proof and retest status for a bounded product surface. The supplier should provide aggregate adoption, patch delivery, VDP and CVE process evidence. A buyer may also ask for a redacted sample report and a security architecture review. These materials answer different questions; a single logo or certificate should not be used as a substitute for them.

The strongest evidence matrix has a final column for limits. If the supplier's MFA metric excludes legacy tenants, say so. If the test used a staging build without production logging, say so. If a vulnerability class was assessed only in one service, say so. Clear limits make the supplier's progress easier to trust and help buyers decide what follow-up they need.

Ask for the proof that fits each goal

Multi-factor authentication

CISA's MFA goal concerns measurable increase in use across the manufacturer's products. The pledge encourages stronger methods and administrator enrollment; its examples include default enrollment and reporting adoption by user type and method. A buyer should therefore request both a product-control answer and a measurement answer. For the product, ask which users and administrators are required or prompted to enroll, whether recovery and alternate sign-in paths preserve the intended protection, and whether a baseline edition supports the customer's identity provider. For progress, ask how the supplier counts enrolled versus eligible accounts and which products or customer cohorts were excluded.

A pentest can test a named sign-in flow and recovery path with synthetic accounts. It can record whether the stated behavior held in that build. It cannot infer the supplier's adoption denominator from those test accounts. If the supplier provides a percentage without defining eligible users, methods and observation period, the buyer cannot compare it with another supplier's percentage. The next question is methodological, not whether the number looks impressive.

Default passwords

The default-password goal defines default passwords as universally shared passwords present by default across a product. CISA gives examples of instance-unique initial secrets, customer-created passwords and transitions for deployed products. A buyer should ask whether the purchased version can be installed without a shared credential, what happens at first setup, and what support exists for already deployed versions. Request the product inventory behind the supplier's progress statement, not merely a screenshot of a new installation wizard.

A controlled test installation can verify its own setup behavior. It cannot prove that every legacy deployment has migrated. If the supplier reports that the current release no longer uses a shared default, ask how older supported releases are identified and how customers are guided to replace the credential. The answer affects the buyer's deployment plan and upgrade timetable. A test result for a fresh instance and a migration metric for existing installations answer different questions.

Reducing entire vulnerability classes

CISA allows the manufacturer to choose a class it will work to reduce, such as SQL injection, cross-site scripting or memory safety weaknesses. The pledge text asks for actions toward a significant measurable reduction across products. A buyer should request the named class, the component population, the starting measure, the current measure and the engineering control expected to prevent recurrence. A broad claim that "security improved" does not identify which class was addressed.

Testing can find a particular weakness or examine a prevention control in a selected product. It cannot establish a manufacturer-wide trend from one engagement. To compare a supplier's progress over time, the buyer needs a consistent counting method. Ask whether the denominator changed because products were added or retired and whether the reported class includes findings from customer reports, internal testing and public disclosures. The pledge's own discussion warns that better discovery or reporting can temporarily increase CVE counts, so a raw count alone can mislead.

Security patches

The patch goal focuses on measurably increasing the installation of security patches by customers. CISA allows progress to be shown through customer behavior or product changes that ease patching. A buyer should ask which supported versions are in the metric, how updates reach customers, whether automatic update capability is available, and how end-of-life transitions are communicated. For an installed product, evidence might include an adoption trend and a tested update path. For a SaaS service, the supplier may apply patches itself, so a customer-installation percentage may not be the right measure.

A pentest can inspect a selected update or deployment path if it is in scope. It cannot count installations across the supplier's customer base. If the supplier cites a patch rate, ask what counts as patched, when the count was taken and whether disconnected or unsupported instances were excluded. A policy that promises rapid patches and evidence that customers actually received them are separate artifacts.

Vulnerability disclosure policy

CISA's VDP goal calls for a public policy that authorizes good-faith testing, provides a report channel, commits against recommending or pursuing legal action against compliant researchers, and allows public disclosure under coordinated practices. A buyer can verify that the policy exists, is discoverable and covers the purchased product. Ask who receives reports, how a reporter learns that the submission arrived, and where the policy draws scope boundaries. These are process questions. A private pentest under a separate contract does not establish that members of the public have an authorized route to report flaws.

The useful artifact is the policy itself, with its scope and contact path, plus a description of how reports are handled. A test team can check whether the published path is reachable and whether its own authorized communications worked, but should not submit an unsolicited vulnerability report as a substitute for the supplier's VDP evidence. The buyer should keep the VDP distinct from its own contract's rules of engagement.

CVE transparency

The CVE goal is more specific than a generic promise to publish advisories. CISA calls for accurate CWE and CPE fields in every CVE record for the manufacturer's products. It also calls for timely CVEs for, at minimum, critical or high-impact vulnerabilities that require customer patch action or have evidence of active exploitation. A buyer can sample public records and ask for the supplier's policy for deciding when to issue a CVE. A record with an identifier but no useful affected-product or weakness information may fail the buyer's practical need even when a press release exists.

External testing can validate the technical substance of a sampled finding and fix. The supplier or its disclosure partners own the publication process and product identifiers. A buyer should not ask a pentest vendor to guarantee that all future CVEs will be timely. It should ask the supplier how it identifies affected versions, links fixes to disclosures and corrects incomplete records. Avoid using fewer published CVEs as a shortcut for better security: CISA cautions that more complete reporting may increase counts in the short term.

Evidence of intrusions

The intrusion-evidence goal is explicitly about increasing customers' ability to gather evidence of incidents affecting the product. CISA's examples include logs for configuration changes, identity activity and business-relevant data access, plus retention and customer access for cloud services. Ask what a customer can export in the purchased edition, how long it remains available, which roles can retrieve it and what context accompanies an event. A provider-only internal log does not answer the same customer question.

A safe, agreed test event can check whether the customer can see an actor, action, target and time in its own interface or export. This establishes a bounded observation. It does not prove that every historical intrusion would be detected or reconstructed. The report should record any logging delay, missing field or environment difference. If an event was visible only to the supplier, the buyer may need a different support and incident-response arrangement.

Adjust the evidence request to the deployment model

The pledge covers on-premises software, cloud services and SaaS, but the proof for a goal can look different by model. CISA's pledge explicitly discusses provider-applied patches for SaaS and customer-accessible logs for cloud services. A buyer should not force every supplier into the same metric. The common requirement is that the metric or artifact answer the relevant risk question for the purchased product.

This table is an editorial comparison, not additional pledge language. Hybrid deployments may need both columns. If a supplier operates the control plane while customers install agents, ask for evidence at each side of that boundary. If an acquired product uses a different identity system or release process, request its own scope statement. A company-level pledge update can be accurate while a particular product remains on a published roadmap.

Build an internal decision record

A procurement team should keep the supplier's claim, the exact artifact supporting it, its product scope, date, limitations and the buyer's conclusion in the same record. The conclusion can be "sufficient for this purchase," "needs a contractual commitment," or "requires a follow-up test." These are buyer decisions, not CISA grades. Recording the reason prevents a future reviewer from treating the presence of a pledge logo as equivalent to product evidence.

For controls that can be tested, put a specific test acceptance criterion in the statement of work. For example: the assessment will use agreed synthetic administrator and standard-user accounts on the named edition and will report the observed MFA enrollment, recovery and alternate sign-in behavior. For customer-visible logging: the tester will perform an agreed safe action and report what the customer could retrieve, including the event fields and timing. These statements are narrow enough to be checked without making claims about every tenant or customer.

For goals that depend on organizational process, request a dated policy, trend or public artifact from the supplier itself. A pentest report should not be stretched into proof of customer patch adoption, disclosure transparency or company-wide weakness reduction. If an artifact is missing, record the gap and decide whether a roadmap, contract term or other assurance is acceptable. That is more useful than marking a goal green or red on an undefined basis.

The same record can support renewal. Keep the original product scope and ask what changed at the next review: new releases, newly included product lines, altered metrics, closed findings and remaining exclusions. Avoid comparing percentages across periods unless the supplier's method and denominator stayed comparable. A trend with a clear method is more informative than a large figure with no defined population.

What this means for software teams

A software maker can use the matrix to prepare a public progress report that readers can actually assess. For each goal, name the control, product coverage, owner, evidence, date and next measurement. Use test findings to improve the product, then link them to prevention work. For example, if repeated authorization flaws appear, the improvement may be a shared authorization component plus regression tests, not merely a string of individual fixes. This is an editorial recommendation, not a CISA-specified implementation.

Stingrai's web application testing, cloud testing and AI and LLM testing can examine selected product behaviors and provide confirmed findings with retesting. The customer controls its pledge reporting and product-wide metrics. A one-time test can answer a defined release question; a continuous program can revisit controls as the product changes.

Method and citation

This article uses the CISA Secure by Design pledge PDF, read on 15 October 2026. Goal names and the voluntary status are sourced to CISA. The evidence matrix and buyer questions are original editorial analysis. Cite CISA for the pledge's official text and this page for the procurement mapping.

Frequently Asked Questions

Is the CISA Secure by Design pledge mandatory?

No. CISA calls it voluntary and not legally binding. Buyers can still request product-specific evidence when a supplier cites the pledge.

Can a penetration test prove all seven pledge goals?

No. A pentest can test selected observable controls, such as authentication or customer-visible logs. Product-wide adoption, patch installation trends, disclosure processes and CVE reporting require additional supplier evidence.

What should a buyer ask a pledge signatory for?

Ask which products and releases each claim covers, the dated metric or artifact behind it, the evidence owner, known exclusions and the next planned measurement. Ask for bounded test evidence where product behavior can be observed.

Does fewer CVEs always mean a stronger Secure by Design program?

No. CISA notes that more complete discovery and reporting can raise CVE counts in the short term. Ask about weakness-class trends, remediation and transparent reporting rather than a raw count alone.

Related reading

• NIST SSDF Penetration Test Evidence Matrix for Buyers

• 2025 CWE Top 25: What a Pentest Can Actually Cover

• Supply Chain Attack Statistics 2026: Open Source, Third Party, and SaaS Risk

Talk to Stingrai

Bring the product, release and control behaviors you want observed. Get a Quote or book a free discovery call.