How to rotate leaked environment variables and API keys after a platform breach: a step-by-step guide

In short

If a hosting or workflow platform tells you that an attacker may have read your environment variables, treat every secret stored there as public: make a list of them, create a replacement for each, switch your apps to the replacement, revoke the old one, and then look in the provider's logs for use you did not make. Do it in that order, because revoking first takes your own service down, and replacing without revoking leaves the stolen key working. The rest of this guide explains each step and why the order matters.

This week's reason to write it: on 1 November 2026 the AI workflow platform Trigger.dev said an attacker "gained unauthorized access to environment variables" for a subset of its customers' organisations, and in recent weeks Railway, Northflank and OpenCode have disclosed similar exposures. Each of those was a different incident with its own details; the clean-up is the same.

Why environment variables are a target

The Twelve-Factor App method, which many platforms follow, says an app's config "is everything that is likely to vary between deploys", including "credentials to external services", and recommends storing it in environment variables. They are easy to change between deploys and unlikely to be checked into a repository by accident. That convenience is why platforms that run your code keep them in one place: the platform's database or control plane holds the plain values so it can inject them into every build and run.

It also means a breach of that platform exposes everything in one go. The OWASP Secrets Management Cheat Sheet is blunt about the trade-off: environment variables "are generally accessible to all processes and may be included in logs or system dumps", and it does not recommend them unless other methods are not possible. If you have no choice, the rest of this guide is what you do when the platform is the weak point.

Step 1: Work out what was exposed

Read the provider's notice first. Look for three things: which environments or projects were affected, a time window, and whether the provider says secrets were read or only could have been. Trigger.dev, for example, said exposure covered "a subset of organizations" and that it had emailed all impacted account owners with steps to mitigate; its status page marked the incident resolved at 22:50 UTC on 1 November, and the company has said a fuller report will follow. The Register, which reported the incident on 2 November, said the company had not named affected customers and declined to give more detail. If the notice is vague, assume the worst case your plan can bear: everything stored in that account is exposed.

Then make an inventory. For each project and environment (production, staging, preview), list every variable that is a credential, not just the obvious ones:

• API keys for AI providers, payment processors, email and SMS services
• Database connection strings, which contain passwords
• Signing secrets: JWT secrets, session secrets, webhook signing secrets
• OAuth client secrets and personal or service access tokens (for GitHub, cloud accounts, package registries)
• Cloud access keys and service-account credentials
• Anything that unlocks other systems, such as a deploy hook URL with a token in it

Config that is not secret (a public hostname, a feature flag, a region) needs no rotation. If you cannot tell whether a value is a secret, treat it as one.

Step 2: Rotate in the right order

The sequence that avoids downtime and leaves nothing exposed is create, deploy, revoke.

1. Create a new credential at the service that issues it. Where the service allows several keys at once, make a new one and leave the old one in place. Give the new one the least privilege it needs; this is a good moment to split one all-powerful key into narrower ones. OWASP's lifecycle guidance says new secrets must be generated with sufficient strength and with the minimum privileges for their role.
2. Put the new value in your secret store (not in the code or in chat), then redeploy or restart so running processes pick it up. Environment variables are read at startup in most runtimes, so a changed value does nothing until the process restarts.
3. Verify the new credential works, for example with a health check or a test call.
4. Revoke the old credential at the issuer. OWASP's wording is that when secrets are potentially compromised you "must securely revoke them to restrict access". Rotation is not complete until the old value is dead, because an attacker who copied it can keep using it until then.

Some secrets need more than a swap. A database password change breaks every client at once unless you create a second user first. A JWT signing key may have to be accepted in both old and new form for a short time so existing sessions do not all fail; once that window closes, remove the old key. Webhook signing secrets usually need a coordinated change at both ends. Do the hard ones deliberately, but do the easy ones (AI provider keys, which can be created and deleted at will) first, because those are the easiest for thieves to turn into money; The Register's headline this week was that token thieves are costing companies $100K+ per day in illicit LLM charges.

Step 3: Look for use you did not make

Rotation stops future use. It does not tell you whether the secret was already used. For each provider, look at:

• Usage and billing graphs. A sudden spike in tokens, requests or compute is the first sign of a stolen AI or cloud key. Set a spending cap or alert if the provider offers one.
• Audit and access logs. Cloud providers, GitHub, payment processors and most SaaS tools show where a key was used from. Look for IP addresses, regions or user agents you do not recognise in the window between the exposure and your rotation.
• New resources. Check for servers, buckets, users, tokens, webhooks or deploy keys you did not create. Attackers who find a powerful key often use it to make a second way back in.
• Your own data. If a database URL was exposed and the database is reachable from the internet, review what was read or changed.

Keep notes with times. If personal data was involved, your legal duties (for example a notification deadline under privacy law) may start from when you became aware of the exposure, and you will want a record of what you did.

Step 4: Close the gap that let it matter

After the urgent work, reduce what a future leak is worth.

• Short-lived credentials. Where the provider supports it, use tokens that expire (OWASP recommends creating secrets to expire after a defined time where possible). A stolen key that dies in an hour is worth far less than one that lives for years.
• Scoped keys per environment and service. Separate keys for production and preview, and for each service, so one exposure does not open everything. Trigger.dev, for example, lists multiple API keys per environment, each with its own scope and expiration, among its recent features.
• A secrets manager for the most valuable values. Fetch them at runtime from a manager with its own audit log instead of pasting them into a platform's settings page. The tool is not the point; having a second place with separate access control and logs is.
• Secret scanning on your repositories. GitHub describes secret scanning as detecting exposed credentials automatically, before they can be exploited; run it, and treat any alert as a rotation.
• Spend limits and alerts on every paid API, so theft shows up as a notification instead of a bill.
• A rotation runbook. Write down, for each secret, who can rotate it, where it is stored, what depends on it and how to roll it back. The first rotation is slow because nobody knows where things are; the second should take minutes.

Common mistakes

• Revoking first. The outage is self-inflicted and usually happens in the middle of the incident when you need your systems most.
• Rotating only production. Preview and staging often hold real keys, sometimes the same ones.
• Forgetting copies. The same key may also sit in a CI system, a local .env file, a teammate's laptop or a second hosting account. Rotating the key fixes all of them, but only if you revoke the old one.
• Trusting "no evidence of misuse". A provider that cannot see use of a secret has not shown it was not used. Rotation is cheap compared with the alternative.
• Pasting the new secret into chat or a ticket. That makes a new exposure. Use the secret store or a one-time sharing method.

Quick checklist

1. Read the provider's notice; note the affected projects and time window.
2. List every credential in those projects and environments.
3. For each: create a new one, deploy, test, then revoke the old one.
4. Check usage, audit logs and new resources for the exposure window.
5. Add expiry, narrower scopes, spending caps and scanning.
6. Write down what you did and when.

Sources

• Trigger.dev: Security Disclosure, Sunday November 1st (status page incident) and its incident report
• The Register: Thieves pilfer environment variables from Trigger AI workflow tool
• OWASP Secrets Management Cheat Sheet
• The Twelve-Factor App: III. Config
• GitHub Docs: About secret scanning