Overview

This article is meant to help provide troubleshooting steps to address issues the "Rerun failed tests" feature in CircleCI rerunning all tests, and not just failed tests.

Issue: All tests are still being rerun

If you find that all tests are being rerun after clicking "Rerun failed tests," please follow the steps below to resolve the issue.

1. Enable verbose mode: Make sure that the --verbose setting is enabled when invoking circleci tests run. This will display the tests that circleci tests run is receiving during a rerun.
   
    
2. Upload JUnit XML: Use the store_artifacts command to upload the JUnit XML file containing the test results to CircleCI. This is the same file(s) that is being uploaded with store_test_results.
   
    
3. Inspect JUnit XML: Manually inspect the newly uploaded JUnit XML via the Artifacts tab in CircleCI. Ensure that the XML file contains either a file attribute or a classname attribute. If neither attribute is present, unexpected behavior may occur when trying to rerun tests.
   
    
4. Check xargs is present: Ensure that xargs is present in the --command= argument.

If you are using a common language or framework, you may be able to find an example in our documentation that outlines a standard implementation of this.

This can be useful in debugging errors seen in your pipelines, and finding niche settings that are required for certain frameworks, for example the junit_family=legacy settings required in pytest.

Additional Resources

• Rerun Failed Tests
• [Product Launch] Rerun Failed Tests (circleci tests run)