Test Summary

The test summary in the UI is a place where we display some basic information about tests such as which tests failed, or what the slowest tests were. For basic configuration to set up the test summary you can view our docs. This document will outline some caveats, limitations, and common issues people run into when trying to get the test summary working.

Ensure Test Reports are in Supported Format

Currently, we support JUnit XML. 

To note, there is not one official schema or standard for JUnit. Each test reporting tool may output its JUnit XML reports differently. This can cause some issues, in particular with our test-splitting feature. 

To split tests by timing, each <testcase> element in the JUnit XML report should include the following attributes:

• file : file path of the test file which the test case can be found in
• name : name of the test case
• time : time taken for the test case to complete
• classname : class name of the test case

Report File Locations

Test data can be located in the following ways:

•  Directly in the path directory: For example, if your path is set to test-results, files like test-results/report.xml will be found.
• In subdirectories of the path directory: For example: test-results/somefolder/report.xml
• As a single file path: The path can point directly to a single test results file

The store_test_results path can point directly to a directory containing JUnit XML files, or to subdirectories containing those files.

Subfolders Cannot be Hidden Folders

Test data subfolders cannot be hidden folders. test-results/.somefolder/report.xml would be ignored by our system.

Compressed Test Results Should not be Larger than 100 MB