Introduction
Docker in Docker (DiD) is a technique that allows running Docker commands within a Docker container. In this article, we will explore the concept of Docker in Docker in the context of CircleCI jobs. We'll discuss the limitations, possible workarounds, and optimizations to ensure smooth integration of Docker in Docker in your CircleCI workflows.
Running Docker Commands in CircleCI Jobs
CircleCI jobs running on a Docker executor inherently have limitations regarding privileged mode and direct access to the Docker daemon. Consequently, executing Docker commands within a Docker container in CircleCI jobs is not feasible. However, CircleCI offers a feature called Remote Docker that enables running Docker commands by utilizing a VM within the job.
Utilizing Remote Docker for Docker in Docker
To run Docker in Docker-like functionality in CircleCI jobs, you can leverage the Remote Docker feature. This involves creating a custom Docker image that includes the necessary security components and using it within your CircleCI job to pull and execute the required Docker image.
By employing the Remote Docker feature, you can effectively run Docker commands within your CircleCI job while leveraging a VM. Each Docker command execution will occur within a Docker container running as a separate VM, enabling parallelization and saving resources compared to running multiple separate jobs.
Optimizing CircleCI Jobs with Docker in Docker
Using Docker in Docker within CircleCI jobs allows for greater flexibility and control. However, it's important to consider factors like resource utilization and potential bottlenecks. While utilizing VMs alongside Docker containers can optimize resource usage, VM overhead can still impact performance.
To optimize CircleCI jobs utilizing Docker in Docker, consider:
**Efficient VM Usage**: Be mindful of VM overhead and ensure your jobs are properly scaled to balance resource utilization. Adjust the number of parallel scans or operations based on available resources and the performance requirements of your specific use case.
By implementing these optimizations, you can ensure efficient resource usage and streamline your CircleCI jobs utilizing Docker in Docker.
Non-docker commands in Docker
Sometimes, you may need to run non-docker commands in your remote docker environment.
Although only Docker commands are automatically executed in the remote Docker environment, you can programmatically execute any set of commands in the remote Docker environment by SSHing into it and chaining any subsequent commands to your shell command.
To SSH into the remote Docker environment, run ssh remote-docker while SSH'd into any CircleCI job that uses setup_remote_docker or execute it programmatically in a run step. (Make sure the setup_remote_docker step runs first, otherwise, your SSH command will fail!)
Using `ssh remote-docker` in a run step:
run: |
ssh remote-docker \ echo "this echo command will be executed in the remote Docker environment \
rather than the initial job container itself"
The challenge presented here is that only a singular command can be passed to the SSH shell—for example, if you run ssh remote-docker touch file1 && touch file2, file1 will be created in the remote Docker environment, but file2will be created in your job's local container.
So, if you want to run a more complex set of commands in the remote Docker environment, programmatically, then you need to find a way to have them executed as a single command.
Some good approaches are outlined in this stackoverflow.
These suggestions include using a "here document" or saving your commands to a shell script which you then pipe into your ssh command.
If you are using remote-docker commands to update your Docker storage driver, please checkout this helpful thread in our forum.
Improved Performance and Pricing for Jobs using `setup_remote_docker` in CircleCI
At CircleCI, we are constantly working on improving our platform to provide faster and more reliable job execution for our users. One area where we have made significant improvements is in jobs that utilize the `setup_remote_docker` feature. This support article aims to explain the changes we have made, their benefits, and how they may impact pricing.
The Previous Setup
Previously, when jobs used `setup_remote_docker`, they utilized a primary container that had a size determined by the resource class specified in the `config.yml` file. Additionally, a remote Linux virtual machine of medium size was provisioned to execute Docker commands. The compute credits per minute for the job were determined by the size of the primary container, while the remote virtual machine was not charged separately.
However, this setup had some drawbacks. The communication between the primary container and the remote virtual machine required network traffic to flow over the internet. This introduced the possibility of transient network failures and adversely impacted the performance of jobs, leading to longer execution times.
The New and Improved Architecture
To address the limitations of the previous setup, we have implemented a new architecture that provides better flexibility, reliability, and performance for jobs using `setup_remote_docker`.
With our redesigned architecture, all remote Docker jobs now run exclusively in a single virtual machine. This means that you have the ability to rightsize the remote Docker engine according to your specific compute needs. Whether you require minimal compute resources or need to scale up to utilize more power, you have full control over the compute capacity used.
The elimination of cross-internet network transfers between the primary container and the virtual machine significantly improves job performance and reliability. By keeping all operations within a single virtual machine, you'll experience faster execution times and reduced occurrences of network-related failures.
Pricing Update
Starting from June 15, 2023, jobs that use `setup_remote_docker` will be identified as "Remote Docker" executors in the CircleCI user interface. This labeling will make it easier to differentiate them from non-remote Docker jobs and compare their performance.
With the introduction of the new executor type, there are some changes to the pricing of jobs using `setup_remote_docker`. The pricing is now aligned with the resource class specified in the `config.yml` file. It's essential to understand how these pricing updates may affect your usage.
Here's a breakdown of the pricing changes:
1. If a job is defined in your `config.yml` file with a Small Docker resource class and uses `setup_remote_docker`, starting from June 15, 2023, the job will execute as an Executor = Remote Docker with a Resource Class = Medium.
2. If a job is defined with a Medium+ Docker resource class and uses `setup_remote_docker`, starting from June 15, 2023, the job will execute as an Executor = Remote Docker with a Resource Class = Large.
It's important to note that no action is required on your part regarding these changes unless you wish to adjust the resource class sizes to better suit your team's needs.
FAQ
In the forum post associated with this support article, we have addressed specific questions and concerns raised by our users. We aim to provide clarity and transparency regarding the improvements and pricing changes. Here are some key points:
1. Users inquiring about the previous pricing model for different resource classes and the impact of the changes.
- We clarified that the previous pricing model charged based on the primary container's resource class, and a remote medium-sized Linux VM was used regardless of the specified resource class.
- We acknowledged that there may have been confusion about the pricing changes and stated that future messaging would be clearer.
2. Users questioning the removal of the Small resource class.
- We reassured users that the Small resource class is still available for jobs not utilizing `setup_remote_docker`. However, if a job uses `setup_remote_docker`, it will now utilize the Medium resource class instead of Small.
3. Users expressing concerns about the price increase for the XLarge resource class.
- We addressed the discrepancy in pricing, stating that the new pricing is aligned with the Linux VM resource class prices. The Linux VM resource class prices include additional factors beyond just CPU/RAM specifications.
- We acknowledged that the price increase may seem significant but emphasized that it aligns with our pricing model for similarly-sized resource classes.
4. Users looking for alternatives to mitigate the pricing changes and improve cost-efficiency.
- We mentioned the option of exploring self-hosted runners, which can be valuable depending on specific requirements and can potentially eliminate per-minute fees if resources are already available. However, we acknowledged that implementing self-hosted runners may require refactoring processes and ensuring full utilization of the resources.
Understanding Docker in Docker in CircleCI Jobs
SSH into the Remote Docker Environment
Sometimes, you may need to run non-Docker commands in your remote Docker environment.
To SSH into the remote Docker environment, run `ssh remote-docker` while SSH'd into any CircleCI job that uses `setup_remote_docker` or execute it programmatically in a `run` step. (Make sure the `setup_remote_docker` step runs first, otherwise, your SSH command will fail!)
**Using `ssh remote-docker` in a `run` step:**
```
run: |
ssh remote-docker echo "this echo command will be executed in the remote Docker environment
rather than the initial job container itself"
```
The challenge here is that only a singular command can be passed to the SSH shell. If you want to run a more complex set of commands in the remote Docker environment programmatically, you need to find a way to have them executed as a single command. Some suggested approaches include using a "here document" or saving your commands to a shell script which you then pipe into your `ssh` command.
If you are using `remote-docker` commands to update your Docker storage driver, please check out this helpful thread in our forum.
Conclusion
By understanding and leveraging the Remote Docker feature in CircleCI, you can effectively utilize Docker in Docker functionality within your CircleCI jobs. Optimizing resource utilization and parallelization can further streamline your workflows. Additionally, knowing how to SSH into the remote Docker environment allows you to execute non-Docker commands programmatically.
Additional Resources
Here are some additional resources related to Remote Docker in CircleCI:
1. CircleCI Documentation - Remote Docker
- This is the official CircleCI documentation that provides detailed information about using Remote Docker in CircleCI.
2. CircleCI Forum Thread - Running Non-Docker Commands in Remote Docker Environment
- This forum thread discusses and provides solutions for running non-Docker commands in the remote Docker environment.
3. CircleCI Forum Thread - setup_remote_docker SSH Key
- This forum thread addresses the issue of the SSH key when using setup_remote_docker.
4. CircleCI Blog - Running Docker Commands on a Remote Server
- This blog post covers the basics of running Docker commands on a remote server.
5. Stack Overflow Thread - Running Commands Inside Docker Container via SSH
- This Stack Overflow thread discusses various approaches to running commands inside a Docker container via SSH.
These resources should provide additional information and guidance on working with Remote Docker in CircleCI.
Comments
Article is closed for comments.