Resolving failed to start cmd: fork/exec /bin/bash: bad file descriptor Failures on Self-Hosted Container Runners

Intermittent failures with error failed to start cmd in pty and non pty: failed to start cmd: fork/exec /bin/bash: bad file descriptor on container runners in can disrupt deployment pipelines. This issue typically arises because the image lacks the bash shell, which is required for executing commands. This also happens when using arm base node trying to run old container images as well such as cimg/base:stable. 

Solution

Install Bash

1. Modify Your Dockerfile: Add instructions to install bash. For example, if you're using an Ubuntu-based image. 

   FROM your-base-image
   RUN apt-get update && apt-get install -y bash

2. Rebuild and apply the Image: After updating the Dockerfile, rebuild your custom image to include bash. Ensure that your self-hosted runner uses the updated image

Override the Default Shell

Specify the Shell: In your .circleci/config.yml, set the shell attribute for steps or executors. 

jobs:
  build:
    docker:
      - image: your-custom-image
    steps:
      - run:
          name: Your Step Name
          shell: /bin/sh -eo pipefail
          command: echo "Hello"

Update the cimg/base Image

1. Check Current Image Version: Ensure you are not using an outdated version of the cimg/base image.
2. Update to Latest Version (multi-arch support): Use the latest version, such as cimg/base:current, to benefit from recent updates and fixes.

Additional Resources

• Docker Hub Documentation for cimg/base
• CircleCI Documentation on Self-Hosted Runners
• CircleCI Documentation on Shell Configuration
• CircleCI Documentation on Creating Custom Image