"fork: retry: Resource temporarily unavailable" When Using Command Substitution in $BASH_ENV

This article explains how single-quoted command substitutions in $BASH_ENV can exhaust a container's process limit, causing jobs to fail with fork: retry: Resource temporarily unavailable.

Symptoms

A job fails with an infrastructure error and the following message:

The task agent failed to run exited with code: run failed: exit status 1.

The step log contains thousands of repeated lines:

/tmp/.bash_env-...-build: fork: retry: Resource temporarily unavailable
/home/circleci/.pyenv/libexec/pyenv: fork: retry: Resource temporarily unavailable

Other characteristics:

No actual commands execute in the failing step.
The job may run for many minutes before timing out.
The error is intermittent - the same pipeline sometimes succeeds and sometimes fails.
The Resources tab may not show signs of resource exhaustion (CPU or RAM).

Cause

CircleCI sources $BASH_ENV at the start of every run step. If $BASH_ENV contains command substitutions wrapped in single quotes, those commands re-execute every time a new step or subshell starts.

The problematic pattern:

echo 'export MY_VAR="$(some-slow-command --with-args)"'  $BASH_ENV

Single quotes ('...') prevent evaluation at write time. The literal command text is stored in $BASH_ENV rather than the result. Every subsequent step re-runs the command.

When the command takes time to complete, processes from previous sources can still be running when the next source begins:

Step 1 sources $BASH_ENV - spawns the command (still running).
Step 2 sources $BASH_ENV - spawns the command again.
Orb sub-steps, subshells, and piped commands multiply the effect.
The container's process (PID) limit is exhausted.
The OS can no longer create new processes ("Resource temporarily unavailable").
The job hangs until the task agent times out.

Note: This issue can be intermittent because it depends on how quickly the command completes. The Resources tab may not indicate this problem as it only shows CPU and RAM, not process/PID count.

Solution

Run the command once, capture the output in a variable, then write the static result into $BASH_ENV.

Before (causes fork exhaustion):

- run:
    name: Set environment variables
    command: |
      echo 'export MY_VAR="$(some-slow-command --flag)"'  $BASH_ENV
      source $BASH_ENV

After (safe):

- run:
    name: Set environment variables
    command: |
      source $BASH_ENV
      MY_VAR="$(some-slow-command --flag)"
      echo "export MY_VAR=\"${MY_VAR}\""  $BASH_ENV

What changed:

source $BASH_ENV is moved before the command so previously set variables are available.
The command runs once and the output is captured in a local variable.
The double-quoted echo writes the resolved value into $BASH_ENV, not the command itself.

Extended example

Before (problematic):

- run:
    name: Init environment
    command: |
      echo 'export RESOURCE_IDS="$(my-cli describe-resources --filter "Name=env,Values=${ENV}" --output text | sed -E "s/[[:space:]]+/,/g")"'  $BASH_ENV
      echo 'export APP_VERSION="$(curl -s https://api.example.com/version | jq -r .version)"'  $BASH_ENV
      source $BASH_ENV

After (fixed):

- run:
    name: Init environment
    command: |
      source $BASH_ENV
      RESOURCE_IDS="$(my-cli describe-resources --filter "Name=env,Values=${ENV}" --output text | sed -E 's/[[:space:]]+/,/g')"
      APP_VERSION="$(curl -s https://api.example.com/version | jq -r .version)"
      echo "export RESOURCE_IDS=\"${RESOURCE_IDS}\""  $BASH_ENV
      echo "export APP_VERSION=\"${APP_VERSION}\""  $BASH_ENV

How to Identify This Issue

Search the config for $BASH_ENV writes that use single quotes:

echo 'export ...'  $BASH_ENV

If the text inside the single quotes contains $(...) or backticks (`...`), the command will re-execute on every step. Apply the double-quote pattern shown above.

To confirm, add a temporary debug step to inspect the contents of $BASH_ENV:

- run:
    name: Debug BASH_ENV contents
    command: cat $BASH_ENV

If the output shows raw commands instead of resolved values, the single-quote pattern is the cause.

Verification

After applying the fix:

Rerun the pipeline.
Confirm the previously failing step completes without fork: retry messages in the log.
Optionally verify the environment variables contain expected values:

- run:
    name: Verify environment variables
    command: |
      echo "MY_VAR=$MY_VAR"

Additional Resources