Generating Testwise Coverage

In order to use the Test Impact Analysis, you'll need to provide per-test coverage in the Testwise Coverage report format. Unfortunately, most coverage tools do not provide coverage information on a per-test level, but rather for the full test run.

We offer plugins for certain languages and test runners, such as Java+JUnit 5 with Maven/Gradle, which can collect Testwise Coverage. It is recommended that you use these plugins if they are available for your language and test runner. We are also working on extending the support for other languages and test runners.

If you use technologies for which we do not have plugins available, but are already able to gather coverage data at a per-test(-suite) level, you can use our teamscale-build command-line tool.

This tutorial will teach you how to use teamscale-build to convert these reports into a single Testwise Coverage report. This will allow you to use the Test Impact Analysis.

1. Download

Download the teamscale-build tool from here and install it.

2. Place All Coverage Files Into a Directory

For this tutorial, we have prepared two sample coverage reports. You can download them from here:

Sample Reports

The ZIP file contains the following files:

sample_reports/
├── expected_output.json
└── input_files/
    └── com/
        └── company/
            ├── AnotherTest/
            │   └── testSomethingElse.failure.10s.simple
            └── SomeTest/
                └── testSomething.passed.5s.simple

The expected_output.json is a copy of the Testwise Coverage report that we are about to generate. The input_files contain the coverage reports. The reports are given in the Teamscale Simple Coverage format, but most of the Teamscale-supported coverage formats are also supported by teamscale-build.

3. Convert the Reports

Execute the following command to convert all Simple coverage reports from the directory input_files to a single Testwise Coverage report and save this report to the file testwise-report.json.

teamscale-build coverage testwise -i ./input_files -f SIMPLE -o testwise-report.json

4. Inspecting the Generated Testwise Coverage Report

Let's open the generated JSON file in a text editor.

4.1 Test names

The uniformPath key shows the name of the detected test cases. These are generated from the relative file path of your input coverage files (excluding the file extension). The exact usage on how the uniformPath is generated is described in the reference documentation for this tool.

4.2 Test execution result and test duration

The test execution result and test duration are parsed from the file name of the test reports. You can also find how you can declare test results and durations in the reference documentation for this tool.

Creating the folder structure

You are responsible for creating the correct folder structure and file names yourself.