Handling Test File Outputs for CircleCI Test Reruns

Overview

When rerunning tests in CircleCI, users may see an output that lists individual test suites instead of individual spec files. This article provides a solution to ensure that file names are outputted, making it easier to rerun tests in case of failure.

Prerequisites

• A CircleCI account and a project set up for continuous integration.
• Familiarity with CircleCI configuration and the circleci tests command.

Instructions

To capture the test file names during a rerun, follow these steps:

1. Use the circleci tests glob command to list your test files.
2. Pipe the output to circleci tests run with the --command flag to save the file names to a temporary file.
3. Optionally, use the --split-by=timings flag to split tests based on their previous run times.
4. Check if the temporary file is non-empty before proceeding to halt execution if necessary.
5. Create a directory for test results.
6. Pass the list of test file names from the temporary file to your test runner command.
7. Store the test results.

Here's an example configuration:

-run:
  command: |
    circleci tests glob "src/**/*js" | circleci tests run --command=">files.txt xargs echo"
--verbose --split-by=timings #split-by=timings is optional
      [ -s tmp/files.txt ] || circleci-agent step half #if a re-run and there are no tests to
rerun for this parallel run, stop execution

- run:
    name: run tests
    command: |
       mkdir test-results
       ... #pas files.txt into your test command
       
- store_test_results
    path: rest-results

This configuration writes the list of test file names to files.txt. On a non-rerun, this list will include all test file names. On a rerun, it will contain only the names of files with at least one failed test from the previous run.

Solution

By following the steps above, you can ensure that the output of a rerun will be the file names of the tests, rather than the names of individual test suites. This allows for a more straightforward mapping of tests to files and simplifies the rerun process.

Additional Resources

For more detailed information on test splitting and other advanced features, refer to the CircleCI Documentation.