Generating Testwise Coverage

In order to use the Test Impact Analysis, you'll need to provide per-test coverage in the Testwise Coverage report format. Unfortunately, most coverage tools do not provide coverage information on a per-test level, but rather for the full test run.

We offer plugins for certain languages and test runners, such as Java+JUnit 5 with Maven/Gradle, which can collect Testwise Coverage. It is recommended that you use these plugins if they are available for your language and test runner. We are also working on extending the support for other languages and test runners.

If you use technologies for which we do not have plugins available, but are already able to gather coverage data at a per-test(-suite) level, you can use our teamscale-build command-line tool.

This tutorial will teach you how to use teamscale-build to convert these reports into a single Testwise Coverage report. This will allow you to use the Test Impact Analysis.

1. Download

Download the teamscale-build tool from here and install it.

2. Place All Coverage Files Into a Directory

For this tutorial, we have prepared two sample coverage reports. You can download them from here:

Sample Reports

The ZIP file contains the following files:

sample_reports/
├── expected_output.json
└── input_files/
    ├── AnotherTest/
    │   ├── metadata.json
    │   └── SIMPLE/
    │       └── report1.simple
    └── SomeTest/
        ├── metadata.json
        └── SIMPLE/
            └── report1.simple

The expected_output.json is a copy of the Testwise Coverage report that we are about to generate. The input_files contain the coverage reports. The reports are given in the Teamscale Simple Coverage format, but most of the Teamscale-supported coverage formats are also supported by teamscale-build. The format of the reports is encoded in the name of the folder containing the reports.

The metadata.json files contain metadata about each test such as its path, duration and result. See the teamscale-build documentation for all available attributes.

Creating the folder structure

For your own tests, you are responsible for creating the correct folder structure and metadata.json files yourself.

3. Convert the Reports

Execute the following command to convert all Simple coverage reports from the directory input_files to a single Testwise Coverage report and save this report to the file testwise-report.json.

teamscale-build coverage testwise -i ./input_files -o testwise-report.json

4. Uploading the Generated Testwise Coverage Report to Teamscale

Now, repeat the same process for your own tests to generate a testwise coverage report for your software system.

To view the result in Teamscale, use our upload tool teamscale-upload to upload the generated testwise-report.json to your Teamscale instance. Specify the format TESTWISE_COVERAGE.

You can view the results under Metrics > Tests.