NIST SP 800-204D describes CI/CD pipelines that move source through build, test, package and deploy stages. Each stage can carry a different identity and trust decision. A red-team exercise can follow one authorized path through those decisions to learn whether a synthetic release could be influenced and whether defenders would detect and contain the attempt.
Quick answer. A CI/CD build-chain red-team exercise begins with a defined developer or repository identity and aims for a harmless, nonproduction artifact or release marker. It tests the approval, build, provenance, deployment and SOC decisions encountered on that path. Stingrai's human-led red teaming can exercise the agreed objective and compare pipeline records with defender case evidence. The statement of work should exclude live production release changes, real secrets and customer data unless a separate authorization explicitly covers them.
The buyer's decision is not whether the pipeline has a long checklist of security settings. It is whether a person or workload with a plausible starting position could cause a release decision that the organization would not expect, and whether anyone would recognize that change before it matters. A pipeline configuration penetration test is valuable for broad coverage. This guide addresses a narrower adversarial question: from approved developer access to a synthetic build and release outcome, what is prevented, what is observed and who acts?
Choose a release objective the business can recognize
The end state should be a safe proxy for a real business risk. Examples include an approved marker appearing in a test artifact, a test package attempting to cross a protected promotion gate, or a nonproduction deployment request reaching an environment where a separate reviewer is expected. The marker must contain no working secret or harmful code. It must have a unique digest or identifier so the white team can verify which source revision, build run and release action produced it. The objective should be written as a decision, such as "can this account cause a test artifact to be promoted beyond its authorized boundary?"
That phrasing makes the result measurable. If source review blocks the change, record the reviewer decision. If a build runs but the artifact is quarantined, record the provenance and registry decision. If promotion is requested but a protected environment refuses it, record the gate. If a test release is allowed, record whether the SOC saw and correctly understood the unusual identity or sequence. "Pipeline compromised" is too broad for an evidence-based report; each trust boundary needs its own result.
The starting identity can be a developer with ordinary repository rights, a maintainer of a nonproduction dependency, an approved service account or an assumed-breach workstation with a test session. These starting points are not interchangeable. An administrator who can edit every repository and environment proves little about a normal developer boundary. A public pull request scenario has a different authorization and risk profile from a private repository maintainer scenario. The buyer should select the position most relevant to its threat model and give the provider the exact permitted access before requesting a price.
MITRE ATT&CK T1677 names poisoned pipeline execution as manipulation of CI/CD processes by injecting code into the build process. That technique describes a behavior family, not a license to modify production builds. A buyer can exercise the detection question with an inert marker in an isolated repository or branch. The white team can verify how the pipeline responds without introducing malicious software or exposing credentials.
Keep the target nonproduction but the trust decision real
A disposable test pipeline with no relationship to enterprise identity can be too easy. It may omit the approval, artifact registry, cloud trust and logging conditions that make the real release process risky. A safer design uses a nonproduction repository or branch with the same relevant access-control model, or a protected test environment that follows the same class of review and promotion rules. The sponsor should decide what fidelity is necessary for the decision and document where the test differs from production.
Do not use a live customer package merely to claim realism. A synthetic artifact can carry a marker and be verified by hash, build record and promotion state. The provider can then show whether an unapproved change would have crossed the intended boundary. If the real production gate cannot be exercised without disruptive risk, test its policy in a dry run, a staging counterpart or a controlled approval decision, and state the limitation in the conclusion. An honest bounded result is more useful than a dramatic but unsafe demonstration.
Map the trust chain before the exercise
A red-team exercise needs a minimal trust map, not a complete inventory of every YAML file. List the source host, repository, branch protections, human and bot identities, runner type, artifact store, signing or provenance service, deployment environment and cloud role that matter to the selected objective. Name the owner of each boundary. This map tells the red team what it may touch and tells the white team where to collect evidence.
The important question at each transition is "who or what may cause the next state?" A developer may propose a source change but not approve it. A workflow may build a branch but not publish an artifact. A runner may write to an artifact store but not deploy. An environment reviewer may approve a promotion, while the cloud identity grants only scoped deployment rights. If the proposal treats all these as one "CI/CD access" permission, it cannot test the real chain.
For GitHub Actions, deployment protection rules can require manual approval, delay a job or restrict the branches that use an environment. GitHub's secure-use reference recommends minimum required GITHUB_TOKEN permissions. These are examples of decisions to verify, not assurances that a specific environment is secure. The actual exercise must inspect what policy was effective for the test run and what action occurred.
Cloud authentication needs its own boundary. GitHub documents OIDC as a way for workflows to authenticate to cloud providers. OIDC avoids storing some long-lived cloud credentials in the workflow, but the trust policy and assigned cloud permissions still determine what the job can do. A buyer should ask whether the synthetic build could obtain a cloud role it was not supposed to have, and whether the cloud and SOC logs would show that decision. The existing GitHub Actions OIDC trust matrix covers configuration-specific OIDC testing; this red-team guide follows the selected release objective and defender response.
Trace the artifact, not just the workflow run
A workflow result alone cannot prove which source produced a package. SLSA's provenance specification defines verifiable information about where, when and how an artifact was produced. SLSA build provenance gives consumers a basis to check whether the artifact came from the expected build. The exercise should preserve the source revision, build invocation, builder identity, artifact digest, provenance statement if present, registry record and release decision. A missing provenance record is an observation; a present record must still be verified against the expected source and builder.
This is a buyer-facing distinction. A report that says "the workflow ran" may miss an artifact substituted later or a release that consumed an unexpected package. Conversely, a suspicious source change that never produced an artifact is not a proven release compromise. The evidence chain should let an engineer answer which exact object was built, which exact object was promoted and whether those objects match the approved release policy.
The red team does not need to publish a live package or use a real distribution channel. A private test registry and a synthetic package can prove the decision. The statement of work should name the allowed registry, organization, repository and environment. If the pipeline has multiple promotion routes, select one or quote them separately. An exercise that follows one path should not imply coverage of every route or every repository in the estate.
Include the defender and the release owner
The SOC may monitor identity events, repository changes, workflow executions, runner behavior, artifact publication, cloud role use and deployment approvals through different products. A security leader needs to know which of those events reached an analyst and whether the analyst understood the release context. An unusual workflow run might be expected during a maintenance window; an approved reviewer change might be suspicious if performed by an identity outside the release team. The case requires business context, not only a raw event.
GitHub's organization audit log records who performed an organization action, what it was and when. That is one source, not the full telemetry chain. The exercise may also require repository events, workflow run metadata, runner logs, artifact-store records, cloud audit events and the SOC case. The white team should gather these independently and compare them with the red-team timeline after the test. If a relevant event was never collected, the resulting gap is collection. If collected but never flagged, it is detection. If flagged but dismissed without source-to-artifact context, it is triage. If escalated to the wrong release owner, it is handoff.

Figure 1: Evidence points from an approved developer identity to a synthetic release and defender response. Source: NIST SP 800-204D.
The release owner must participate. The SOC may identify a suspicious change but lack authority to freeze a build, revoke a token or block promotion. The rules of engagement should state who may take those actions, what safe action is permitted against the synthetic target and how the owner is reached. A case that accurately warns of risk but reaches no one with release authority is an operational failure. A release owner who blocks the promotion but leaves no case record may have prevented this test while leaving the next one harder to investigate.
Decide what the analysts know
A blind exercise can test normal monitoring, but the white team must still be able to stop the scenario and distinguish it from a real incident. A purple-team replay can test a specific detection hypothesis and improve runbooks with analysts watching. Both are useful, but they are different tests. The report should state which stage was blind, which was collaborative and whether analysts knew the marker or exact timing. A detection built during a replay should not be presented as proof that the original blind exercise was caught.
Capture decision times in a consistent timezone, along with the relevant run IDs and case IDs. Do not reduce the response to one "time to detect" number without noting what started the clock and which event counted as detection. The first workflow event, first artifact change and first SOC case may occur at different points. A timeline that preserves them helps the buyer identify where the delay occurred and what it means for a real release window.
Set rules of engagement for a safe build-chain test
The written authorization should name the sponsor, repository and organization owner, permitted developer identity, source branches, workflow or pipeline, runner boundary, artifact registry, deployment environment, cloud accounts, exercise window and prohibited actions. It should include the synthetic marker and the exact proof of success. A test that unexpectedly reaches a production deployment environment should stop for white-team review before any change is made there. The provider should not assume that access to a repository authorizes it to use every connected cloud role or external package registry.
Protect real credentials and customer data. The red team should use test identities and inert artifacts. It should not copy secrets into workflow output, send them to an external endpoint, publish a package under a customer-facing name or alter a production deployment. A synthetic access decision can establish whether a trust boundary would have allowed an action. If a real secret is accidentally exposed, the white team should treat that as an incident and follow the agreed response route rather than extending the test for more dramatic proof.
The rules should also specify whether source or workflow files may be changed, who reviews those changes, how the changes are rolled back and what logs may be retained. An organization may want to test a developer-initiated source change but not an edit to central workflow configuration. That is a valid boundary. The result then applies to the allowed path, not to every possible pipeline manipulation. MITRE's poisoned pipeline execution technique includes several behavior patterns; the exercise needs only the one that answers the buyer's question safely.
Set practical stop conditions: a real security incident, unexpected release activity, a failure in the test isolation, a third-party alert that cannot be attributed to the exercise, or any evidence of production impact. Name a reachable white-team lead and an alternate. If the SOC treats the activity as a real compromise, the white team should be able to disclose the test identity and halt activity without forcing analysts to choose between exercise secrecy and containment. The purpose is to learn whether the organization would respond, not to win a contest against its own responders.
Document the build and release evidence in advance
Before starting, the parties should agree which records will prove each stage. The source host may hold commit, review and branch-protection records. The CI platform may hold workflow definitions, run identifiers, job permissions and runner events. The artifact store may hold package digest and publication metadata. The release system may hold approval and deployment records. Cloud logs may show role assumption or resource actions. The SOC may have alerts, investigations and incident tickets. The white team should know who can export each record before the test window opens.
This evidence plan prevents a common weak conclusion: "we could not tell whether the gate was bypassed because the logs expired or the owner was unavailable." A red-team provider cannot create telemetry the environment never collected. It can, however, identify the missing records and limit the conclusion accordingly. If the buyer needs a baseline before attempting the objective, put that baseline in the proposal. If the environment has no production-like nonproduction counterpart, agree how much of the chain can be tested safely and label the untested edge.
Compare quotes by the decision they can support
Red-team proposals for CI/CD can vary widely. One bidder may review a handful of workflow files and run a test build. Another may include developer identity, repository controls, runner trust, artifact provenance, promotion approval, cloud permissions and SOC response. These are different products. Use the decision matrix below to force an apples-to-apples scope discussion before discussing price.
Scope item | Question the buyer must answer | Deliverable to request |
|---|---|---|
Starting identity | Which developer or bot access is plausible? | Permission snapshot and authorization |
Source boundary | Which repository, branch and review rule matter? | Source and review decision trace |
Build boundary | Which workflow, runner and token permissions apply? | Run ID, effective permissions and safe action result |
Artifact boundary | Which package or image represents the objective? | Digest, provenance and registry decision |
Release boundary | Which environment approval is expected? | Promotion attempt and gate outcome |
Defender boundary | Which events should reach the SOC and release owner? | Alert, case, escalation and action timeline |
Retest | Which exact path will be repeated after changes? | Same-marker verification and differences |
Ask the bidder to name the test platform and its limitations. A GitHub Actions exercise is not automatically transferable to Azure DevOps, GitLab or a custom release system. The underlying trust questions are similar, but evidence sources, approval mechanisms and cloud integrations differ. A proposal should say whether the team has current access to the particular repository, runner, registry and deployment environment. If any owner has not authorized the work, that boundary should be listed as excluded, not silently assumed.
Ask whether the price includes a defender debrief and a retest. A technical narrative alone may show the path but leave the SOC unable to improve a detection. A SOC workshop without an independent red-team timeline may miss what the analysts never saw. The most useful package brings the two together: an objective outcome, a verified artifact lineage, a decision timeline and named remediation owners. One-time exercises can answer a release-transformation or board question. A continuous program may revisit the same path as identities and pipelines change. Stingrai offers both one-time assessments and continuous programs, and the proposal should state which cadence is being quoted.
Separate a full pipeline pentest from this scenario
The existing CI/CD pipeline penetration-testing scope guide covers broad technical surfaces such as secrets, runners, tokens, OIDC and dependencies. A buyer may need that assessment to find weaknesses across many pipelines. The red-team exercise here deliberately follows one developer-to-release objective and adds defender decision-making. It may discover a technical weakness, but it is not a substitute for inventorying all workflows, dependencies or connected accounts.
A strong proposal can combine the services explicitly. For example, broad pipeline penetration testing can identify candidate trust paths, then a red-team exercise can use an approved path to test whether a protected promotion and the SOC response hold. The services should have separate scopes and conclusions. If the same provider performs both, ask it to state whether the red team knows the pentest findings in advance. That knowledge affects what the exercise demonstrates about discovery and detection.
Turn the exercise into a release control improvement
The debrief should involve development, platform engineering, release management, cloud identity and the SOC. Walk the evidence chain from starting identity to final marker. The source owner explains the review and branch decision. The platform owner explains workflow and runner permissions. The artifact owner explains provenance and registry handling. The release owner explains approval. The SOC explains what arrived, what it inferred and how it escalated. The red team supplies the independent test timeline and the exact authorized path.
Prioritize the fix that breaks or exposes the tested chain. Tighten a developer permission if the account could change protected release inputs. Restrict a workflow token if it had unnecessary write access. Adjust a cloud trust policy if a nonproduction workflow could obtain a production role. Require provenance verification if promotion accepted an artifact without checking origin. Improve an alert if the relevant event reached the SOC but its context was not recognized. Update the release on-call route if the SOC case reached no decision maker. Each fix should have an owner and a retest condition.
Avoid overstating one result. A blocked synthetic release does not prove every pipeline is protected. A successful test artifact does not prove a production package could have been altered if the production environment has different controls. A detection in a warned replay does not prove blind detection. The report should explicitly name what was simulated, what was actually exercised and which production control could only be inspected. The buyer can then decide whether to run a second scenario against another identity, release path or platform.
Where Stingrai fits
Stingrai is a global CREST-accredited penetration testing services company founded in Toronto, Canada in 2021, trusted by companies from startups to enterprises to meet audit requirements for SOC 2, ISO 27001, CMMC, PCI DSS and HIPAA. OSCE³, OSWE, OSEP, CREST CRT certified penetration testers, who are also world-class security researchers and bug bounty hunters. Choose from fully human-led or hybrid (AI agents plus human penetration testers) engagements across web, API, mobile, AI and LLM, cloud, network, Active Directory and social engineering penetration tests and red team engagements.
For this buyer decision, Stingrai's human-led red team can scope an approved developer-to-release objective and test the sequence of trust decisions, detection and response. A separate cloud penetration test can examine broader cloud identity, control-plane and CI-related credential paths when the buyer needs component coverage as well. The red-team quote should identify the repositories, pipeline platform, synthetic target, defender participants and exclusions.
The report can support a security or compliance program with a documented path, defensible evidence, remediation ownership and retest result. It should not imply that one exercise validates every release route. Bring a repository and release inventory, a candidate starting identity, the nonproduction environment, the SOC logging map and any production restrictions to Get a Quote. A discovery conversation can help choose the smallest safe objective that still answers the business question.
Frequently Asked Questions
What should a CI/CD build-chain red-team exercise include?
A CI/CD build-chain red-team exercise should include an approved starting identity, a named repository and workflow, a synthetic nonproduction artifact or release objective, written stop rules, source-to-artifact evidence, release-gate decision, SOC case timeline and a defender debrief. The report should show what was blocked, what was detected and what was allowed.
How is this different from CI/CD pipeline penetration testing?
Pipeline penetration testing broadly validates technical weaknesses in the authorized build environment. A build-chain red-team exercise follows one bounded developer-to-release objective and measures the defender's interpretation and response as well as the technical outcome. A buyer needing inventory-wide coverage should scope that separately.
Can the exercise be run without touching production releases?
Yes. Use an inert marker in a nonproduction artifact, a protected test environment and a private registry or safe promotion gate. The white team should document how closely those controls match production and state any boundary that was inspected rather than exercised. Real customer packages and secrets are unnecessary for a useful result.
What does SLSA provenance add to the evidence?
SLSA provenance records verifiable information about where, when and how an artifact was produced. In an exercise, the buyer can compare the expected source revision, builder and output digest with the actual artifact and release decision. Provenance must be verified against expectations; its mere presence is not proof of a safe release.
Is OIDC enough to protect cloud deployment permissions?
No. OIDC is an authentication mechanism for a workflow to obtain cloud access. The trust policy and cloud role still determine which workflow and identity can act, and the release gate determines whether promotion is approved. The exercise should test the effective decision for the synthetic run.
Which defender records matter most?
Preserve source and review history, workflow runs, runner and token context, artifact digest and provenance, registry publication, environment approval, cloud role use and the SOC case. Compare timestamps and IDs across the records. A missing event, missed analytic, mistaken triage and failed handoff are distinct findings.
What should a buyer ask in a red-team proposal?
Ask for the starting identity, precise repository and environment boundary, synthetic objective, excluded production actions, telemetry plan, analyst-awareness mode, stop authority, deliverables and retest condition. Request a sample redacted response timeline that shows the provider can distinguish technical access from detection and containment.
Does Stingrai use Snipe to red-team CI/CD pipelines?
No. Stingrai's CI/CD and cloud red-team engagements are human-led. Snipe is the autonomous agent for web applications and associated APIs; its specialist sub-agent tests AI and LLM applications, including MCP servers, alongside named penetration testers. The build-chain objective is planned and operated by human testers.
Related reading
Talk to Stingrai
Share the approved developer identity, repository and release boundary, synthetic objective and SOC participation plan through Get a Quote, or book a free discovery call to shape a safe human-led build-chain red-team exercise.


