Max Memory Consumption and Memory Usage Over Time
The method of acquiring memory usage data varies between execution environments and is useful in the optimization and debugging of jobs. Below, we have outlined a few methods that will provide this data.
Docker Executor Max Memory Usage
Docker executor users can output the max memory consumed by a job by adding the following as the last step in the job:
- run:
command: cat /sys/fs/cgroup/memory/memory.max_usage_in_bytes
when: always
Note: This will be accurate if the job has one Docker image. Multiple Docker image builds will only report the memory usage of the primary image.
Memory usage will be reported in bytes. You can convert to GiB using your favorite search engine to check it against the amount of RAM available to your job, according to its assigned resource class.
This can help troubleshoot out-of-memory (OOM) errors.
Docker/Machine Executors Memory Usage Over Time
To log memory usage over time for both Docker and machine executors, you can also add this step as the first step in your job:
- run:
command: |
while true; do
sleep 5
# NOTE: on MacOS, the f argument is not supported.
# In this case, you can drop the f argument instead.
ps auxwwf
echo "======"
done
background: true
Alternatively, we can also take advantage of the top command (available on Docker, Machine (Linux) or MacOS executors). This can help show both memory and CPU utilization by individual processes.
- run:
name: Profile CPU and memory every 5s (background)
command: |
while true; do
sleep 5
printf "\n\n$(date)\n"
top -bcn1 -w512
echo "======"
done
background: true
Note that you have may have to install these tools, or use an alternative command if these tools are not available in the specific Executor.
Remote Docker
To get the memory usage of the Remote Docker environment, you can pass the ps command through SSH with
ssh remote-docker ps auxwwf
Comments
Article is closed for comments.