Overview
Jobs that use Remote Docker (setup_remote_docker) or the machine executor with Docker Layer Caching (DLC) enabled may fail during Set up container environment or Spin up container environment — before your build steps run — with an Infrastructure Failure badge and an error like:
error communicating with docker daemon: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
This often appears intermittently (re-running the job may succeed) and is commonly tied to multi-architecture image builds that register QEMU/binfmt handlers on the Remote Docker host.
The failure is usually caused by a corrupted DLC snapshot, not by your Dockerfile or application code.
Affected Projects
Your project may be affected if all of the following are true:
-
docker_layer_caching: trueonsetup_remote_dockeror themachineexecutor - Multi-arch builds using
docker buildxwith--platform linux/amd64,linux/arm64(or similar) - A QEMU/binfmt setup step (
tonistiigi/binfmt,multiarch/qemu-user-static, or equivalent)
The same error can also appear from other DLC corruption (for example after a Docker engine version change), but the cleanup below is specifically required for multi-arch + DLC.
Solution
These steps can be followed to solve this issue
1. Add a binfmt cleanup step (required)
Add this as the last step in any job that installs QEMU/binfmt and uses DLC. Use when: always so cleanup runs even if an earlier step fails - otherwise a failed build can still save a corrupt cache.
Recommended (if you use tonistiigi/binfmt):
- run:
name: Clean up multi-architecture build artifacts
when: always
command: docker run --privileged --rm tonistiigi/binfmt --uninstall qemu-*Alternative (manual binfmt_misc cleanup):
- run:
name: Clean up multi-architecture build artifacts
when: always
command: |
docker run --rm --privileged busybox sh -c '
for f in /proc/sys/fs/binfmt_misc/qemu-*; do
echo -1 > "$f" 2>/dev/null
done
'2. Purge the project’s DLC cache
Clear the existing corrupt snapshot:
- Web app: Project Settings → Docker Layer Caching → Delete Cache Contents
-
CLI:
circleci project dlc purge <vcs-type> <org> <project>
See Purge DLC.
3. Roll out cleanup on every active branch
DLC is project-scoped. Any branch that runs multi-arch setup without the cleanup step can write a corrupt cache that affects later jobs on other branches. Merge or cherry-pick the cleanup to all active feature branches, not only main.
4. Pin your binfmt image (recommended)
Avoid :latest on QEMU setup images so silent image updates do not change behaviour unexpectedly: docker run --privileged --rm tonistiigi/binfmt:v7.0.0 --install all. Match the tag to whatever your build already uses; the important part is pinning a specific version.
5. Name your Buildx builder (recommended with DLC)
When using buildx, unnamed builders get random names and their volumes may not be preserved correctly across runs. See Buildx builder instances:
- run:
name: Create Buildx builder
command: |
docker buildx create --name my-dlc-builder --use
docker buildx inspect --bootstrapRelated resources
Comments
Article is closed for comments.