Rotating Secrets for January 4th Incident

Steps for Secret Rotation (Updated March 13th)

Note: Please refer to this blog post for up-to-date information and Incident Report on the January 4th Security Incident.

Based on the January 4th security notice, we recommend taking the following steps to rotate your tokens: 

There are multiple ways to do this, and we encourage you and your teams to use your preferred methods. Here is an approach you may choose to follow:

 

Note: It is recommended that you log out of CircleCI prior to revoking access to GitHub

  1. OAuth tokens:
    1. For GitHub: As of 07:30 UTC on January 7, all GitHub OAuth tokens have been rotated on behalf of CircleCI customers. Customers who wish to do so may rotate their own OAuth tokens by logging out of the CircleCI application, going to https://github.com/settings/applications, selecting “Authorized OAuth Apps”, and then revoking the CircleCI entry. Once that’s done, log back into CircleCI to trigger reauthorization.
    2. For Bitbucket: As of 10:00 UTC on January 6, 2023 our partners at Atlassian expired all OAuth tokens for Bitbucket users. Bitbucket tokens will refresh for users upon login, and no additional action is needed here. Bitbucket users will still need to replace SSH tokens.
    3. For GitLab: GitLab users do not need to reauthorize their application access. As a precautionary measure, we would still recommend GitLab users rotate their environment variables, Personal and Project API tokens, and all SSH keys.
  2. Project API tokens: To rotate them, go to Project Settings > API Permissions > Add API Token. Update: CircleCI has revoked all tokens created before 00:00 UTC on January 5, 2023. 
  3. Project environment variables: Go to Project Settings > Environment Variables and then create an environment variable with the same name to replace the existing value.
  4. Context variables: Go to Organization Settings > Contexts and do the same thing as for project environment variables for each context. Update: As of 23:00 UTC on January 9, 2023 we have updated the Contexts API to include the last "updated_at" date and time stamp. This gives the necessary information to determine if secret rotation was successfully completed. We will be rolling out additional changes to ensure the updated_at date is include in the UI, in addition to the API. You can read more in the API documentation on contexts and environment variables.
  5. User API tokens: Go to User Settings > Personal API Tokens and then delete and recreate any tokens you might be using. Update: CircleCI has revoked all tokens created before 00:00 UTC on January 5, 2023.
  6. Project SSH keys:
    1. Go to Project Settings > SSH Keys. 
    2. Delete the Deploy Key and add it again. 
    3. If you were using any additional keys, then those need to be deleted and recreated. 
    4. Note: SSH keys will also need to be rotated from the target environment. 
  7. Runner Tokens: using the CircleCI CLI, run the following commands:
    1. circleci runner token list <resource-class name> 
    2. circleci runner token delete "<token identifier>"
    3. circleci runner token create <resource-class-name> "<nickname>"
    4. Following these commands, you will need to add the created token to your launch-agent-config.yml and restart your runner service

Note: there is also a tool for discovering all your secrets on CircleCI that can be used to find an actionable list of items for rotation.


Update - 9 January 2023:
We have added the functionality to return SHA256 signature for checkout keys using our get-checkout-key API V1.1 endpoint.

Please see below for a example API Call :

  • curl -H "Circle-Token: <circle-token>" https://circleci.com/api/v1.1/project/:vcs-type/:username/:project/checkout-key?digest=sha256
  • Please note the sha256 query parameter here

 

Update - 13 March 2023:
We have updated the tool to further aid in discovery of secrets that are stored in CircleCI but may not be visible in the UI, e.g. variables for renamed projects or projects deleted from GitHub but not from CircleCI.

For more information, please visit https://circleci.com/blog/january-4-2023-security-alert/

Was this article helpful?
24 out of 32 found this helpful

Comments

0 comments

Article is closed for comments.