Your firewalls won’t help. Neither will your intrusion detection or endpoint protection. Supply chain attacks bypass your perimeter defences completely—they compromise trusted dependencies before they ever reach your systems. When malicious code arrives through your package manager, all your security theatre is just that. Theatre.
In September 2025, the Shai-Hulud worm infected over 500 npm packages with combined weekly downloads exceeding 2.6 billion. It harvested credentials from GitHub, AWS, GCP, and Azure, then self-replicated through maintainer accounts. The blast radius was exponential.
Here’s the problem. Supply chain compromises take 267 days to identify and contain on average. That’s nine months of attacker access to your systems. But regulatory disclosure requirements give you 72 hours.
This playbook gives you an hour-by-hour timeline across five phases: triage and detection (0-2h), containment and assessment (2-8h), credential rotation (8-24h), validation and auditing (24-48h), disclosure and reporting (48-72h). It includes communication templates and procedural guidance for multi-cloud credential rotation.
Sure, comprehensive supply chain security measures reduce your risk. But even organisations with strong preventive controls need breach response plans. This assumes the compromise has already been detected. Here’s your response plan.
What Happens in the First 2 Hours After Detecting a Supply Chain Breach?
Get your incident command structure activated within minutes. Designate an incident commander who owns the timeline, makes containment decisions, coordinates teams, and interfaces with executive leadership. This is their only job until the incident closes.
Set up a war room, whether physical or virtual. It needs to be separate from potentially compromised communication channels. If credentials were stolen, assume your Slack and email are monitored.
Isolate affected systems as your initial containment step. Disable CI/CD pipelines. Pause deployments. Disconnect compromised build servers from production networks. Don’t wait for complete understanding. Containment first, forensics second.
Start documenting everything now. Capture package names and versions, identify suspicious postinstall scripts, record the timeline of first detection signals. You’ll need this for forensic analysis and legal compliance.
Preserve your forensic evidence immediately. Snapshot package-lock.json files, export GitHub Actions workflow history, backup CloudTrail and GCP logs. Attackers commonly clean up once they know they’ve been detected.
Start your preliminary blast radius assessment. Which packages are affected? What credentials might be exposed? Which production systems consumed compromised dependencies? You won’t have complete answers in two hours, but you need to start mapping scope.
Notify key stakeholders, but delay customer and public disclosure until you understand the blast radius. Inform executive leadership, legal counsel, and your compliance team. Disclosure before completing your assessment forces you to issue corrections later, which damages trust more than a delayed but accurate notification.
Your first response time target for P0 issues should be under 15 minutes. That’s war room activation time, not resolution time.
How Does Blast Radius Assessment Work for Supply Chain Attacks?
Map your dependency graph comprehensively. Use npm ls --depth=Infinity package-name or yarn why package-name to identify all direct and transitive dependencies consuming compromised packages. Transitive dependencies are where the real damage hides.
Calculate your download impact using npm’s weekly download statistics. Weekly downloads of 2.6 billion translate to potentially hundreds of thousands of affected systems. This gives you an order-of-magnitude estimate. It’s imprecise, but it tells you whether this is a targeted incident or an everyone-panic situation.
Identify credential exposure scope by auditing all systems where compromised packages ran. CI/CD servers, developer workstations, and production deployments all have different secrets accessible. Create a matrix correlating systems with the credentials available: GitHub PATs, AWS keys, GCP credentials, Azure credentials, npm tokens.
Trace lateral movement by examining GitHub Actions workflows for secrets context usage. Check CloudTrail and GCP logs for API calls from potentially compromised credentials. Review firewall logs for webhook exfiltration traffic to domains like webhook.site.
Reachability analysis identifies which vulnerable functions are actually called in your code, showing that only approximately 2% of dependency vulnerabilities are truly exploitable. Focus on what’s actually invoked, not what’s merely present.
Assess production impact by determining if compromised code reached customer-facing systems. Check deployment logs for affected package versions. Evaluate what data could be accessed from potentially stolen credentials. This drives your disclosure obligations.
How Do I Rotate AWS Credentials After a Supply Chain Breach?
With your blast radius mapped, begin systematic credential rotation starting with the most commonly compromised platforms.
Use the AWS CLI or Console to deactivate keys first. This prevents a race condition where attackers use old keys while you’re generating new ones.
Audit AWS usage by querying CloudTrail logs for all API calls made with potentially compromised credentials. Look for unauthorised resource creation, data access, and backdoor IAM users, roles, or policies. This forensic step happens before rotation. Once you delete the old keys, you lose the audit trail.
Environment variables represent a security anti-pattern—they’re difficult to rotate and leak into logs. Use dedicated secrets management instead: AWS Secrets Manager, Azure Key Vault, or HashiCorp Vault.
Generate new credentials using aws iam create-access-key. Prefer temporary STS credentials with time-limited validity over long-lived keys.
Update application secrets by deploying new credentials to Kubernetes secrets, AWS Secrets Manager, environment variables, and CI/CD pipeline configurations. Automate this. Manual rotation across dozens of services introduces errors.
Your automation should list and deactivate existing keys, create new keys, wait for credential propagation (typically 5 minutes), update all applications, verify functionality, then delete old keys only after confirming success.
Verify rotation completeness by monitoring CloudTrail for authentication errors. Common missed locations include Lambda environment variables, ECS task definitions, CloudFormation parameters, and CodeBuild environment variables.
Credential-rotation schedules keep service accounts fresh, with changes propagating automatically to every system. Set up automated rotation for ongoing security, not just incident response.
How Do I Rotate GitHub, GCP, and Azure Credentials During Incident Response?
AWS credentials represent only one attack vector. GitHub, GCP, and Azure credentials require parallel rotation efforts.
CISA recommends rotating ALL credentials: npm tokens, GitHub PATs, AWS keys, GCP and Azure credentials, and SSH keys. Assume anything accessible from compromised systems is stolen.
For GitHub PAT rotation, list all personal access tokens in Settings → Developer settings → Personal access tokens. Revoke compromised tokens, generate new tokens with minimum required scopes, and update CI/CD systems. Enable token expiration for future tokens to limit blast radius.
GCP credential rotation requires identifying compromised service account keys via GCP audit logs. Create new keys using gcloud iam service-accounts keys create, update Kubernetes secrets and GCP Secret Manager, then disable old keys with gcloud iam service-accounts keys disable. Verify workloads authenticate successfully before deletion.
Azure credential rotation involves rotating service principal secrets using az ad sp credential reset, updating Azure Key Vault references, and regenerating managed identity credentials where applicable.
Workload identities eliminate the bootstrap secret problem entirely by providing applications with secure identities from cloud platforms. This is the long-term fix. For incident response, you’re stuck with key rotation.
Multi-cloud rotation requires orchestrating GitHub PAT creation, GCP service account key rotation, and Azure service principal credential reset. Document all current credentials before starting, create new credentials for each platform, update all consuming services, wait for propagation and verify functionality, then revoke old credentials.
Handle API rate limits with retry logic. Validate credential propagation before revoking—premature revocation creates outages.
Use centralised secrets platforms for single control plane orchestration.
Implement phishing-resistant MFA using FIDO2 or WebAuthn hardware security keys on all GitHub and cloud accounts. CISA defines phishing-resistant MFA as authentication immune to phishing attacks.
How Do I Audit package-lock.json for Compromised Dependencies?
While rotating credentials addresses immediate access concerns, you need to verify the integrity of your dependency chain to prevent reinfection.
Compare lock file snapshots from your last known-good commit versus the current compromised state. Use git diff to identify injected malicious package versions.
Verify integrity hashes by validating SHA-512 hashes in the lock file against the npm registry. The npm ci command fails if hashes don’t match, indicating tampering. This is why you use npm ci for security validation, not npm install.
Use npm ci for its precision. It deletes node_modules completely, installs exact versions from package-lock.json, and validates integrity hashes. npm install may resolve different versions, obscuring compromise evidence.
Identify unexpected version changes—packages that jumped versions without corresponding package.json changes suggest malicious updates.
Audit postinstall scripts by extracting and reviewing all lifecycle hooks from package.json files in node_modules. Supply chain attacks commonly operate through malicious lifecycle hooks executed during package installation.
Flag suspicious commands: curl, wget, eval, chmod, and TruffleHog execution. Attackers have weaponised security tools like TruffleHog to scan filesystems and enumerate cloud secret managers.
Check for typosquatting by reviewing your dependency list for packages with names similar to popular libraries but with subtle character substitutions.
Validate dependency provenance by confirming packages originate from the expected npm registry. CISA recommends auditing all npm dependencies using package-lock.json files, pinning packages to versions from before September 16, 2025.
What Should I Include in My SBOM Validation During Breach Response?
Generate your current SBOM using Syft, CycloneDX CLI, or SPDX generators. Run syft packages dir:. -o cyclonedx-json > sbom-cyclonedx.json to create a comprehensive inventory.
Compare against your baseline SBOM from the last clean deployment. Diff pre-incident versus current to identify newly introduced or version-changed components.
Validate component hashes by verifying cryptographic hashes match expected values from trusted sources.
Check for known vulnerabilities by cross-referencing SBOM components against CVE databases and GitHub Security Advisories. Feed generated SBOMs into vulnerability scanners using tools like Grype: grype sbom:sbom-cyclonedx.json. Prioritise exploitable vulnerabilities during incident response.
SBOM validation during incident response follows the same principles as ongoing SBOM management with tighter timelines. You’re looking for deviations from known-good state, not conducting a comprehensive security audit.
Verify transitive dependency chains by ensuring your SBOM captures the full dependency tree including nested transitive dependencies.
CycloneDX is security-oriented and lightweight, optimised for CI/CD integration. SPDX is comprehensive, excelling in licensing, compliance, and provenance tracking.
Choose CycloneDX for security-focused incident response when vulnerability detection is the priority. Choose SPDX if you need compliance documentation alongside security validation.
Automate SBOM generation into CI/CD pipelines for ongoing validation. Store SBOMs as deployment artifacts. Version SBOMs alongside code. When the next incident happens, you’ll have baseline SBOMs ready for comparison.
How Do I Remove Malicious GitHub Actions Workflows After a Breach?
Audit all workflow files by reviewing the .github/workflows directory for every repository in your organisation.
Identify workflows created or modified during the compromise window using Git history. Flag suspicious characteristics: triggers on pull_request_target (allows code execution from forks), workflows with secrets context usage (potential exfiltration), workflows pushing to protected branches without review, or workflows containing curl commands to webhook domains like ${{ toJSON(secrets) }}.
Review workflow run history by examining GitHub Actions logs for unauthorised executions. Identify workflows that accessed secrets. Check for data exfiltration via curl or webhook commands. This tells you what was stolen, not just what could have been stolen.
Disable suspicious workflows instead of deleting them. This preserves forensic evidence while preventing further execution.
Delete malicious workflows after documenting for forensics. Do this via pull request to maintain an audit trail. Force-push to remove from history only if workflow files contain embedded credentials requiring immediate removal.
CISA recommends strengthening GitHub security with branch protection rules and Secret Scanning. Require pull request reviews for .github/workflows changes. Enable CODEOWNERS for the workflow directory. Prevent force-pushes to protected branches.
Rotate repository secrets by assuming all secrets accessible to compromised workflows are stolen. Rotate repository, environment, and organisation secrets.
Securing developer environments reduces risk of credential exposure through compromised IDEs or AI assistant integrations.
When Should I Disclose the Breach to Customers and Regulators?
With containment complete and credentials rotated, you face disclosure obligations with tight regulatory deadlines.
GDPR requires notification within 72 hours of becoming aware of breach. CCPA requires notifying affected consumers as soon as possible. US state laws vary. Sector-specific regulations like HIPAA and PCI-DSS have additional requirements.
Regulatory disclosure deadlines typically begin when your organisation becomes aware of the breach, not when it occurred. Extended dwell time increases required disclosure scope and regulatory scrutiny.
Complete your blast radius assessment before disclosure. Confirm customer data impact scope and validate that containment measures deployed successfully.
Disclose if customer credentials were compromised, production data was accessed, services were disrupted, or regulatory obligations mandate disclosure regardless of impact.
Both GDPR and CCPA demand detailed breach notifications including the nature of the breach, affected data categories, and steps taken to mitigate impact.
Here’s a customer notification template:
Subject: Security Incident Notification – Action Required
We are writing to inform you of a security incident affecting [Company Name].
What happened: On [date], we detected that malicious code in a third-party software dependency compromised our [affected systems]. The incident was contained on [date].
What information was affected: The incident may have exposed [specific data types: credentials, customer data, etc.]. We have no evidence that your data was accessed, but we cannot rule it out completely.
What we’re doing: We have rotated all potentially compromised credentials, removed malicious code from our systems, validated our software supply chain, and implemented additional security monitoring.
What you should do: As a precaution, we recommend you [specific actions: rotate passwords, enable MFA, monitor accounts for suspicious activity].
Support: If you have questions, contact us at [email/phone]. We have established a dedicated support channel for this incident.
We take security seriously and apologise for this incident.
GDPR fines have two levels: up to €10 million or 2% of annual global turnover for less severe violations; up to €20 million or 4% of annual global turnover for severe violations. CCPA enforcement involves civil penalties up to $2,500 per unintentional violation or $7,500 per intentional violation.
Prepare communication templates: customer email, FAQ document, regulatory report, and public blog post. Sequence notifications: regulators first, then customers, then public. Establish a customer support channel and monitor social media.
FAQ
What is the average cost of a supply chain security breach?
Average global breach cost reaches $4.44 million, with U.S. averages at $10.22 million. Projected global annual cost of software supply chain attacks hits $60 billion in 2025. Healthcare faces highest costs at $7.42 million average, financial sector follows at $5.56 million.
How long does it take to detect a supply chain attack on average?
Supply chain compromises take 267 days to identify and contain on average, substantially longer than traditional breaches at 60-90 days.
What makes supply chain attacks different from other security breaches?
Supply chain attacks compromise trusted dependencies before they reach your organisation, bypassing perimeter defences completely. Traditional security controls like firewalls, EDR, and network monitoring are ineffective against malicious code introduced through legitimate package managers.
What credentials do supply chain attackers typically target?
Shai-Hulud harvested GitHub personal access tokens, AWS access keys and Secrets Manager contents, GCP service credentials, Azure credentials, npm authentication tokens, and cloud metadata endpoints. These enable persistent repository access, lateral movement, data exfiltration, and deployment pipeline access.
Should I use npm ci or npm install for security validation?
Use npm ci for security validation during incident response. It deletes node_modules completely, installs exact versions from package-lock.json, and validates integrity hashes, failing if any package was tampered with. npm install may resolve different versions or update the lock file, obscuring compromise evidence.
How do I know if my SBOM is comprehensive enough?
Comprehensive SBOMs include all direct and transitive dependencies with exact versions, cryptographic hashes, supplier information, and licence data. Compare component count in SBOM against node_modules contents ensuring no unlisted dependencies. Run SBOM generation tools multiple times verifying consistent output.
What are phishing-resistant MFA methods?
CISA recommends phishing-resistant multifactor authentication immune to phishing attacks, typically FIDO2 or WebAuthn hardware security keys like YubiKey or Google Titan Security Key. SMS codes and authenticator app OTPs remain vulnerable to sophisticated phishing attacks.
How quickly do I need to rotate credentials after discovering a breach?
Begin credential rotation within 8 hours of confirming credential exposure to minimise attacker access window. Complete rotation of GitHub, AWS, GCP, and Azure credentials within 24 hours. Balance speed with thoroughness to avoid incomplete rotation leaving backdoor access.
What is a self-replicating npm worm?
Shai-Hulud represents the first successful self-replicating worm in the npm ecosystem. Malware queries the npm registry for maintainer packages and force-publishes patches recursively for self-propagation. Shai Hulud 2.0 compromised approximately 492 npm packages in November 2025 affecting Zapier, ENS Domains, and AsyncAPI.
Can I delete GitHub Actions workflows to remove malware?
Disable workflows instead of deleting them to preserve forensic evidence. Delete workflows via pull request maintaining Git history after documenting for your incident report. Force-push deletion only if workflow files contain embedded credentials requiring immediate removal.
What tools should I use for automated credential scanning?
TruffleHog, GitGuardian, and GitHub Secret Scanning detect committed credentials in code repositories. Ironically, attackers can repurpose credential scanning tools to harvest secrets. Enable GitHub Secret Scanning push protection to prevent credential commits.
How do I handle disclosure if breach occurred months ago but just discovered?
Regulatory disclosure deadlines typically begin when your organisation becomes aware of the breach, not when it occurred. State discovery date, estimated breach date range if determinable from forensics, and explain the detection delay.
After the 72 Hours
You’ve contained the breach, rotated credentials, validated your dependencies, and disclosed to regulators and customers. Now what?
After completing 72-hour response, transition from crisis mode to strengthening long-term security posture. Conduct a post-incident review to identify how the compromise occurred and what preventive measures would have detected it earlier.
Post-incident hardening checklist:
- Implement automated SBOM generation in your CI/CD pipeline
- Set up dependency scanning with tools like Grype or Trivy
- Enable GitHub Advanced Security including secret scanning and dependency review
- Configure branch protection requiring reviews for workflow changes
- Establish automated credential rotation schedules for all service accounts
- Deploy runtime monitoring for suspicious package installation behaviour
- Create playbooks for specific attack scenarios discovered during this incident
Review your incident response plan and update it based on what you learned. Document what worked, what didn’t, and what you wish you’d had ready. The next incident will be different, but preparation still matters.
Supply chain breaches are inevitable. This playbook ensures you’re prepared when detection happens. For comprehensive prevention-focused supply chain security, implement proactive measures alongside your incident response capabilities.
