Docker Hub rate limiting FAQ

Beginning November 1, 2020, Docker Hub will enable rate limits based on the originating IP address. However, CircleCI has partnered with Docker to ensure that our users can continue to access Docker Hub without rate limits. On November 1st, with few exceptions* (listed below), you should not be impacted by any rate limits when pulling images from Docker Hub through CircleCI.

However, these rate limits may go into effect for CircleCI users in the future. That’s why we’re encouraging you and your team to add Docker Hub authentication to your CircleCI configuration and consider upgrading your Docker Hub plan, as appropriate, to prevent any impact from rate limits in the future.

We have additional information and up to date details on the "Authenticate with Docker to avoid impact of Nov. 1st rate limits" Discuss post.

*Exceptions: Remote Docker and Machine Executors will be impacted by the rate limiting unless pulling CircleCI-published images.

Below we will describe frequently asked questions on how this affects your CircleCI jobs and how to handle each situation.

How do I authenticate with Docker Hub?

You will need to pass an auth key with your Docker image and utilize a username and password. Specific steps can be found here: https://circleci.com/docs/2.0/private-images/ 

How can I confirm my authentication was successful?

If you have passed the auth key on your Docker pulls and the pulls continue to be successful, such as on the "Spin up environment" step, your credentials are being successfully used. At this time we don't have a way to see in the UI the authenticated credentials.

However, if you are passing the key and the wrong credentials are set, you will receive an error and the build will fail. 

Note: CircleCi shows the warning message in the spin up environment step if you have not added auth key yet.

>Warning: No authentication provided, this pull may be subject to rate limits.

mceclip0.png

My builds use the Docker executor, does this affect me?

No, as mentioned previously CircleCI has partnered with Docker to ensure that our users can continue to access Docker Hub without rate limits.

However, these rate limits may go into effect for CircleCI users in the future. That’s why we’re encouraging you and your team to add Docker Hub authentication to your CircleCI configuration and consider upgrading your Docker Hub plan, as appropriate, to prevent any impact from rate limits in the future.

As mentioned in the exceptions section though, the use of Remote Docker with a non-CircleCI convenience image is affected by these rate limits and you will want to ensure you authenticate.

My builds use the Machine executor, does this affect me?

Maybe, if you utilize Docker (i.e. pull images) within your Machine executor jobs this could affect you. As mentioned in the above exceptions, if you are using a non-CircleCI image you will be impacted.

You will need to use the docker login -u username -p password command with your credentials to log in to Docker Hub, before running the docker commands that potentially pull images; such as, docker pull, docker run, or docker build.

I use CircleCI convenience images, does this affect me?

No, as mentioned previously CircleCI has partnered with Docker to ensure that our users can continue to access Docker Hub without rate limits.

However, these rate limits may go into effect for CircleCI users in the future. That’s why we’re encouraging you and your team to add Docker Hub authentication to your CircleCI configuration and consider upgrading your Docker Hub plan, as appropriate, to prevent any impact from rate limits in the future.

I use custom Docker images, does this affect me?

If the custom Docker images are being used in a Remote Docker or Machine executor, yes, they are affected.

However, custom docker images used with the Docker executor won't be affected because of the previously mentioned partnership.

Although, these rate limits may go into effect for CircleCI users in the future and we are still encouraging users to authenticate. 

What actions can I take ahead of November 1st, 2020?

Depending on your CircleCI setup, you may not need to take action. Please reference the other FAQ sections to see if rate limiting will be applied to your builds.

If you do need to take action, it would be to set up Docker authentication in your projects, we have more information on those actions in the "My build fails to pull Docker image with "Too Many Requests" - Rate Limiting by Docker Hub" article

Can I update all of my projects that require Docker authentication en masse?

At the moment each project will need to be updated individually.

If I am blocked after the change happens, what actions can I take?

If you haven't set up the authentication described above, that would be the first step.

If you have set up authentication and hit the limits, you will either want to change how often you are building or explore Docker Hub's plans that allow for unlimited pulls.

 

Was this article helpful?
0 out of 0 found this helpful

Comments

0 comments

Article is closed for comments.