How to Handle Task Agent Termination in CircleCI

In certain scenarios, you may find that your task agent isn't exiting as expected after a TERM signal. This can happen when the service is terminated, and there is no handler to manage the task agent's exit. In such cases, the machine runner will manually send a KILL signal to the task agent.

Identifying the Issue

The first step to troubleshoot this issue is to check the logs from the runner. You should look for a message similar to "Runner received kill signal". Here's how to get the logs:

1. Open your terminal.
2. Run the following command: journalctl -u circleci.

For more information on verifying if the service is running, you can refer to the CircleCI documentation here.

Solution

1. Open your Systemd service file.
2. Add KillMode=process under the [Service] section. Your file should look something like this:

[Unit]
...
[Service]
...
KillMode=process
[Install]
...

After making this change, try running your service again and check if the task agent exits as expected after the TERM signal.

Additional Resources 

• https://circleci.com/docs/machine-runner-3-configuration-reference/#customizing-job-timeouts-and-drain-timeouts