A GitHub App's registration permissions determine the GitHub APIs and webhook events it can use, and an installation access token can be narrowed to particular repositories and permissions. Those documented controls define a practical test target: the SaaS application that obtains the grants, receives events and decides what to do next. GitHub, Choosing permissions for a GitHub App; GitHub, Generating an installation access token.
Quick answer. A GitHub App penetration test should prove that an authorized installation can reach only its granted repositories and actions, that the app protects credentials and verifies incoming webhooks, and that an event cannot make the app act for the wrong customer. Scope the buyer's app, backend, stored data and controlled test installations. Stingrai's human-led web application and API penetration testing can test those application and integration boundaries against an agreed statement of work.
The purchase trigger is usually concrete. A SaaS vendor is about to ask enterprise customers for repository access. A security team is reviewing a new integration that can read code or change pull requests. A product team has expanded its GitHub App permissions, added a webhook-driven workflow, or moved from a pilot organization to several customer installations. At that point, a generic web pentest report may leave the buyer's most important question unanswered: can one installation, user or event make the service cross a customer boundary?
This guide turns that question into a scope and an evidence request. It is written for the organization that owns the app or is commissioning an assessment with the app vendor's cooperation. It does not assume that an enterprise customer may test a vendor's SaaS or GitHub's own service without authorization.
The unit of testing is an installation, not a login page
A GitHub App has more than one identity path. It may act as its installation, or it may act on behalf of an authorized user. GitHub says an installation token's effective access depends on the app's grants, while a user access token is further limited by the user's rights and the account where the app is installed. Using the installation identity for an action that is supposed to be limited by the current user's rights can therefore make the app do more than the user should be allowed to request. GitHub, Best practices for creating a GitHub App; GitHub, Authenticating on behalf of a user.
That distinction changes procurement language. A request for a "GitHub integration pentest" may get you an OAuth callback review and a scan of a public API. The useful test follows the installation through your system: who installed it, which organization and repositories it covers, what the app stores for that customer, which token it mints for a job, and which event or human request caused that job. The assessor must be able to observe the app's own decision at each boundary.
Boundary | Buyer-owned asset to put in scope | Question the test must answer |
|---|---|---|
Installation grant | App registration, installation records and customer-to-installation mapping | Does the service associate each installation with the correct customer and current repository grant? |
Token issuance | Token minting service, cache, worker and credential store | Does each job use the intended installation and the narrowest feasible repository and permission set? |
User authorization | Login, session and user-triggered app actions | Does a user-requested action still respect that user's current rights and organization context? |
Webhook reception | Owned HTTPS endpoint, queue, parser and job dispatcher | Is a delivery authenticated, handled once for the intended action, and bound to the correct installation? |
Data retention | Customer records, caches, exports and logs | Can one tenant see another tenant's fetched code, metadata, findings or job results after access changes? |
GitHub's own installation management page distinguishes an app installation from a separate user authorization. An owner can change repository access, suspend or uninstall an app; removal from one account does not itself describe the state of every other installation or user authorization. The test should use those state changes as normal cases, not only the initial install. GitHub, Reviewing and modifying installed GitHub Apps.
Draw the evidence path before testing
The diagram below is a buyer's evidence map. It is not an exploit chain or a measured risk score. It shows where an assessor should collect a decision record: the grant, the token's actual scope, the event validation result, the application authorization decision and the data that remains available. GitHub documents those boundaries separately, so the report must reconnect them for one business workflow. GitHub, Choosing permissions; GitHub, Installation access tokens; GitHub, Validating webhook deliveries.
Figure 1: The evidence required across an installation grant, token, webhook, app action and retained data. Sources: GitHub, App permissions, GitHub, Installation access tokens, and GitHub, Webhook validation.
The outcome is stronger than a checklist marked "signature enabled" or "least privilege configured." It ties a synthetic event or user request to the customer, installation, allowed repository, action and result. A buyer can then ask whether the same trace changes correctly when a repository is removed, an employee loses access, or the installation is suspended.
Give the testers an inventory that can become a statement of work
The initial scoping call should produce an inventory, not a promise to "test GitHub." The app owner should supply its registration's current permission list, the event subscriptions, the token types in use, the backend components that issue or receive credentials, and the product actions those components can trigger. GitHub recommends minimum app permissions, minimum webhook subscriptions and appropriate token types. These are assessable design choices, not a substitute for testing observed behavior. GitHub, Best practices for creating a GitHub App.
Registration and grants. Record whether the app is installed on organizations, user accounts or enterprise accounts, and whether installations can see all or only selected repositories. List repository, organization, enterprise and account permissions separately. GitHub's permissions page assigns different resources to those categories and notes that some API access needs a high privilege such as Administration. The SOW should say which requested grants will be reviewed and which are outside the current release. GitHub, Choosing permissions for a GitHub App.
Identity paths. Mark each product action as app-initiated or user-initiated. For user actions, describe how the product checks both the GitHub user's authority and its own local customer role. GitHub says a user must authorize an app before it can act on that user's behalf, and a user token is constrained by the user's access, the app's access and the installation account. The assessor should ask where those constraints are enforced in the product, not infer them from the presence of a login button. GitHub, Authenticating on behalf of a user.
Event routes. List each subscribed webhook event, the action variants the app processes, the receiving endpoint, queue and worker, and the business operation that follows. GitHub recommends checking both event type and action, because event types can have multiple actions and may gain new ones. It also recommends using the delivery identifier to handle repeats, including legitimate redelivery that carries the same identifier. GitHub, Best practices for using webhooks.
Credentials and data. Locate the private key, webhook secret, client secret, installation tokens, user tokens and any cached repository content. The inventory should name systems and custodians, never paste live values into the SOW. GitHub says the app private key can authenticate across every account where the app is installed, and advises secure storage and limited access. If the app stores fetched customer data, GitHub also advises retaining the organization or enterprise context that gave the user access. GitHub, Best practices for creating a GitHub App.
Test fixtures. Provide approved test organizations, repositories, users and synthetic content that reproduce the important cases: an installed organization, a different installation, an ungranted repository, a user who loses membership, and an installation whose repository selection changes. GitHub documents the relevant installation and repository-change events. The application owner should define which state changes can safely be made during the engagement. GitHub, Webhook events and payloads.
A useful SOW sentence is: "Assess the customer-owned GitHub App registration and backend against the approved test installations, repositories and synthetic records. Validate current grants, token issuance, webhook authenticity and replay handling, user and installation authorization, cross-customer data separation, revocation behavior and the evidence produced for each decision. Do not test GitHub infrastructure or unrelated customer accounts." The parties can then attach exact asset names, contacts, test windows and exclusions.
Acceptance matrix for the engagement
The following matrix is a starting point for a buyer and assessor to edit together. "Evidence" means the observation that should appear in the report, with test identities and repository names redacted for distribution. Passing a configuration review alone is insufficient where the issue concerns what the running service actually does.
Area | Controlled question | Evidence to request | Acceptance condition |
|---|---|---|---|
App permission inventory | Are requested grants tied to named product functions? | Registration permission list mapped to API calls and subscribed events | Every grant has an owner and use case; surplus access is documented for removal or explicitly accepted. |
Installation selection | Can a request from one tenant choose another tenant's installation? | Synthetic cross-tenant request and server-side decision trace | The request is denied before a token is minted or customer data is read. |
Repository selection | Does a job stay within the installation's selected repositories? | Allowed and ungranted repository cases using test installations | The service refuses the ungranted or out-of-scope repository and records why. |
Token narrowing | Does a job request only the repositories and permissions it needs? | Token issuance parameters and resulting scope for a sample job | The token cannot exercise a broader action than the job requires where GitHub supports narrowing. |
Token type | Is a user-requested action carried by an appropriately user-bound identity? | Product action, GitHub token type and resulting attribution | A user's loss of access cannot be bypassed by silently substituting the installation token. |
Private-key custody | Can a normal app user, support role or job log read the private key? | Storage and access review with safe negative checks | Only approved service roles can use the key; no live value appears in evidence. |
Webhook authenticity | Does the endpoint reject an invalid or missing signature before dispatch? | Handler decision, sanitized request metadata and queue outcome | No business job is created from an unauthenticated delivery. |
Event authorization | Does a valid delivery authorize only its intended event and action? | Event-to-handler map and benign unsupported-action case | The handler rejects unrecognized actions and binds the accepted event to the correct installation. |
Duplicate handling | Can the same delivery cause an unsafe repeated business action? | Redelivery case and idempotency record | A repeat is recognized without an unintended duplicate effect, while legitimate retry remains supportable. |
Grant changes | Does the app update local scope after repository access changes? | Before-and-after grant snapshot and service behavior | Removed repository access no longer yields stored or newly fetched protected data. |
Uninstall or suspension | Does the app stop new work for an inactive installation? | Controlled state-change case and job/queue disposition | New privileged work is blocked and retained data follows the agreed lifecycle. |
Customer data isolation | Can two installations retrieve each other's cached results? | Synthetic records and cross-tenant read attempt | The service authorizes each read against the correct customer and installation, including exports and background jobs. |
The matrix is intentionally about observable decisions. GitHub's token endpoint allows an installation token to be restricted to particular repositories and permissions. Without explicit restrictions, the token receives the permissions granted to the app and access to the repositories granted to that installation. The assessor should compare the app's intent, the minting request and the actual API operation, rather than mark a short-lived token as safe by itself. GitHub says installation access tokens expire after one hour. GitHub, Generating an installation access token.
A report should label a control "not applicable" when the product genuinely has no corresponding path. An app that never acts for users does not need a fabricated user-token test. An app that never stores repository content still needs a review of any metadata, logs or task results it retains. The acceptance condition should match the buyer's data and business action, not a generic vendor template.
What a webhook signature proves, and what it does not
GitHub recommends verifying each webhook against the secret and the original payload before processing it. The X-Hub-Signature-256 header carries the signature, and GitHub's guidance calls for a timing-safe comparison. A test should show that a missing or invalid signature is rejected before the event reaches the queue or a worker that can act on a repository. GitHub, Validating webhook deliveries.
A correct signature is one gate, not a complete authorization decision. The application must still decide whether the event type and action are supported, whether the installation belongs to the tenant record it is about to update, and whether the referenced repository is still within current scope. GitHub recommends checking event and action before processing. Its installation and installation-repository events give apps a way to observe grant changes, but the app remains responsible for the behavior of its own cache and jobs. GitHub, Best practices for using webhooks; GitHub, Webhook events and payloads.
Duplicate handling deserves an explicit acceptance case. GitHub recommends using X-GitHub-Delivery to recognize a repeat, while noting that a requested redelivery uses the original identifier. A buyer should ask the assessor to test the business effect, not merely confirm that a delivery ID is stored. A repeated event may be harmless for a read-only notification yet costly for a deployment, report publication or another write. The acceptable behavior should be agreed in the SOW for each sensitive action. GitHub, Best practices for using webhooks.
Operational timing belongs in the scope only where it affects safety and reliability. GitHub expects a 2XX response within 10 seconds of webhook receipt; the product may acknowledge the delivery and process it through a queue. The assessment should still trace the event across that queue and show that authentication and tenant binding cannot be skipped during asynchronous work. GitHub, Best practices for using webhooks.
Permission changes are product changes
GitHub allows an app owner to request new permissions, but an existing installation retains its previous rights until the account owner approves the change. GitHub also distinguishes account permissions, which may prompt a user to reauthorize. A test of a newly added product feature must include both states: installations that accepted the expanded grant and installations still operating on the earlier grant. The app should neither mislead users about feature availability nor silently route the task through a different, broader credential. GitHub, Approving updated permissions for a GitHub App; GitHub, Choosing permissions.
Repository selection can change independently of a product release. GitHub lets organization owners review an installed app and move between all repositories and selected repositories. It also emits an installation-repositories event when the set changes. The app's queue, cache and user interface need a defined response to a removed repository. The assessor should distinguish a GitHub API denial from a local data leak: losing the ability to make a fresh API call does not automatically remove the data already copied into the SaaS product. GitHub, Reviewing installed GitHub Apps; GitHub, Webhook events and payloads; GitHub, Best practices for creating a GitHub App.
This matters most when a user moves between organizations. GitHub tells app developers to store durable IDs and the organization or enterprise context of retained data, and to recheck that a user's current membership and SSO state still permit access. An authorized read yesterday is not an authorization rule for an export today. The penetration test should use synthetic records to demonstrate how the product behaves after membership loss or organization switching. GitHub, Best practices for creating a GitHub App.
Avoid four common scope failures
A registration screenshot instead of an effective-access test. The screenshot shows requested permissions. It does not show the subset the account owner granted, the repositories selected for that installation, the rights on the token minted for a job or the local role that started it. GitHub documents each as a separate part of the access decision. The report should connect them in one trace. GitHub, Choosing permissions; GitHub, Installation access tokens.
One friendly organization with no negative case. A successful workflow in a single test org cannot establish tenant separation. Use a second authorized installation and synthetic records to prove that the app rejects an installation ID, repository reference or cached result from the wrong customer. The test is of the SaaS product's mapping and authorization, not an attempt to access unrelated GitHub accounts.
Signature verification with no downstream test. An endpoint can verify a delivery correctly and still dispatch the wrong job or repeat a write. GitHub's own guidance treats signature verification, event/action filtering and delivery-ID handling as separate practices. The assessment should observe all three and then inspect the business result. GitHub, Webhook validation; GitHub, Webhook best practices.
A report that ignores what the app kept. GitHub recommends preserving the organization context for data stored by an app and reassessing membership and SSO status. If a repository is removed or a user leaves an organization, the buyer needs to know which copied data, search index, export and log remains visible through the SaaS product. The result may be a product policy decision, but the report should make it explicit. GitHub, Best practices for creating a GitHub App.
Rules of engagement that protect the buyer and other customers
An app vendor can authorize testing of its own app and provide installations in controlled GitHub organizations. The SOW should identify those organizations, repositories, local tenant records, users, webhook endpoints and environments by durable inventory identifiers. Do not place production private keys, webhook secrets or customer repository content in the brief or final report. Use synthetic data, and make any destructive or production-affecting workflow opt in with a named owner and test window.
The assessor should state which GitHub-hosted actions are needed to observe the app's behavior and obtain the app owner's authorization for those test accounts. A customer buying an assessment of its own installation should coordinate with the SaaS vendor before probing the vendor's backend. GitHub App documentation describes how an installation can be suspended, uninstalled or narrowed, but that does not grant anyone permission to test another party's service. GitHub, Reviewing installed GitHub Apps.
Define a stop condition for unexpected data access and a direct contact for the test team. Limit replay, delivery and token cases to benign synthetic events. The point is to show whether the application blocks an unauthorized transition and records enough evidence to remediate it, not to demonstrate a damaging action. Retests should repeat the failed boundary with the same fixture and then verify the normal authorized workflow still works.
What the report should give procurement and engineering
An enterprise buyer needs a report that can answer both "what was tested?" and "what changed after remediation?" Ask for an executive summary, a signed asset and identity inventory, a matrix of attempted and observed outcomes, a finding record with safe reproduction evidence, and a retest status tied to the original case. The technical appendix should identify which permissions and webhook events were in scope on the test date. These change independently of the SaaS vendor's marketing description.
The most useful evidence is a decision trace. It begins with a test user or event, identifies the customer and installation, names the allowed repository and operation, records the token class and effective grant without exposing the token, and ends with the observed product and GitHub API outcome. For a rejected case, the trace should show where the service blocked it. For a retained-data case, the trace should show what the user could still retrieve from the product after a grant changed.
A report that only says "webhooks secure" or "least privilege implemented" is difficult to use in a customer security review. The buyer can request a compact, redacted evidence table that answers the acceptance matrix. It can be shared with security reviewers without handing them secrets or another customer's data. If the service supports both one-time enterprise reviews and ongoing releases, keep the same matrix so the next test shows whether new permissions or events changed the boundary.
Quote comparison and vendor questions
The cost driver is the number of distinct decision paths, not merely the count of GitHub API endpoints. An app that reads one selected repository and posts a notification needs a different effort from an app with many permissions, user-delegated actions, several event handlers, background jobs and retained repository content. Ask each bidder to state assumptions about installations, roles, event variants, owned environments, source access and retest depth. A quote that excludes the queue or data store may miss the exact point where a verified event becomes a cross-tenant action.
Put these questions to the vendor before signing:
Which test installations and repository-selection states are included?
Will the team inspect the difference between installation and user token paths in running behavior?
How will the team test a removed repository, a suspended installation and a user who loses access without touching another customer's data?
Does the scope include the webhook handler, queue, worker and any downstream job that writes to GitHub or to the SaaS product?
What evidence will show both a rejected cross-customer action and a successful authorized action?
Will the retest cover the original denied path, the fix and the normal workflow after the fix?
Which configuration snapshots, logs and redacted traces can be shared with an enterprise customer or auditor?
The quote should list these items as testable deliverables. It should also say what the buyer must provide, including test accounts, safe content, app registration access, a temporary test environment if needed and an escalation contact. That is more useful than asking for a broad promise of "GitHub security testing" with no acceptance criteria.
Where Stingrai fits
Stingrai is a proud CREST-accredited penetration testing firm. Its human-led web application testing covers authentication, access control and business logic; its API testing covers role and tenant boundaries. For a GitHub App, the engagement can be scoped around the customer's app backend, webhook handler and authorized test installations, with findings and retesting delivered through Stingrai's platform. Stingrai, About Us; Stingrai, API Penetration Testing.
An app with many customer installations, separate environments or a broader cloud control plane needs a custom scope and quote. Stingrai offers one-time engagements and continuous programs; the right option depends on release cadence and the evidence the buyer needs. The SOW should identify the exact product and test identities before work starts. Stingrai, Get a Quote.
Frequently Asked Questions
What should a GitHub App penetration test include?
A GitHub App penetration test should cover the app's installation-to-customer mapping, granted repositories and permissions, token custody and selection, user authorization, webhook validation, downstream actions and retained data. Stingrai can scope these as a human-led web and API assessment using authorized test installations and synthetic records. The report should show an allowed and a denied outcome at each relevant boundary.
Is a GitHub App token enough to prove a user may perform an action?
No. An installation token represents the app installation, while a user access token also reflects the user's current access. The application must authorize the business action against the correct customer, installation and user context. The assessment should show which identity was used and what happened when the requesting user lacked access.
Should the test include GitHub's own infrastructure?
The ordinary buyer scope is the customer-owned app, backend, data and approved test installations. Any testing of GitHub's own service would require its own authorization and rules. The statement of work should name the permitted organizations, repositories and actions so that the boundary is clear.
Is a webhook signature enough to pass the assessment?
No. Signature verification is an entry gate. The app must also filter the event and action, bind the delivery to the correct installation, handle duplicates safely and authorize each downstream job. The report should show those decisions and their business result, including a benign rejected case.
What changes when an enterprise customer removes a repository?
The assessment should observe whether the app stops new work for that repository and whether previously copied content or results remain accessible through the SaaS product. The buyer and vendor should agree on the intended retention and deletion behavior. The test should prove the implemented behavior with synthetic content.
Do we need both installation and user-token cases?
Include both only if the product uses both. A background job that acts as the app and a user-requested action have different authorization questions. The SOW should enumerate the actual token paths, then require evidence for each path that the product uses.
What should an enterprise security reviewer receive?
The reviewer should receive a redacted scope, the grants and events tested, the accepted and rejected cases, findings, remediation status and retest evidence. The report should not contain live private keys, tokens, secrets or another customer's data. A concise decision trace is more useful than a configuration screenshot alone.
How often should a GitHub App be retested?
Choose a cadence tied to material changes in grants, webhook subscriptions, user authorization paths, token handling, tenant mapping or stored data. An annual test can support a recurring review, while a significant new permission or workflow can justify a focused retest before rollout. The SOW should define the trigger and the exact boundary to repeat.
Related reading
How to Scope a SaaS OAuth and Connected App Penetration Test
Third-Party API Integration Penetration Testing: Buyer Scope
API Authorization Test Matrix: What a Buyer Should Ask to See
References
GitHub. Choosing permissions for a GitHub App. Product documentation, accessed 1 November 2026. Defines app permission categories and grant behavior.
GitHub. Generating an installation access token for a GitHub App. Product documentation, accessed 1 November 2026. Describes token scope and expiration.
GitHub. Authenticating with a GitHub App on behalf of a user. Product documentation, accessed 1 November 2026. Describes user access constraints.
GitHub. Best practices for creating a GitHub App. Product documentation, accessed 1 November 2026. Covers credentials, token type, authorization and data context.
GitHub. Validating webhook deliveries. Product documentation, accessed 1 November 2026. Describes signature validation.
GitHub. Best practices for using webhooks. Product documentation, accessed 1 November 2026. Covers event filtering, delivery identity and response timing.
GitHub. Webhook events and payloads. Product documentation, accessed 1 November 2026. Defines installation and repository-change event payloads.
GitHub. Reviewing and modifying installed GitHub Apps. Product documentation, accessed 1 November 2026. Describes repository selection and installation state.
GitHub. Approving updated permissions for a GitHub App. Product documentation, accessed 1 November 2026. Describes approval of expanded grants.
Talk to Stingrai
Bring your app's permission inventory, event list and intended customer workflow to Get a Quote, or book a free discovery call to agree on a test boundary and evidence package.
