The test summary in the UI is a place where we display some basic information about tests such as which tests failed, or what the slowest tests were. For basic configuration to set up the test summary you can view our docs. This document will outline some caveats, limitations, and common issues people run into when trying to get the test summary working.
Ensure Test Reports are in Supported Format
Currently, we support JUnit XML and Cucumber JSON formats.
To note, there is not one official schema or standard for JUnit.
Each test reporting tool may output their JUnit XML reports differently. This can cause some issues, in particular to our test-splitting feature.
To split tests by timing, each <testcase> element in the JUnit XML report should include the following attributes:
- file : file path of the test file which the test case can be found in
- name : name of the test case
- time : time taken for the test case to complete
- classname : class name of the test case
Report Files Must be in a Subfolder
Test data must be inside a subfolder of whatever path is provided to
If their path is set to
test-results then the test report file or files must be in a subfolder of
test-results, and not directly within it. For example:
Subfolders Cannot be Hidden Folders
Test data subfolders cannot be hidden folders.
test-results/.somefolder/report.xml would be ignored by our system.
Test Files Cannot be Larger than 15MB
Test files larger than 15MB will fail to upload.
Note that this upload failure will not show up in the build Ui.
Instead, it fails silently in the background.