Teamscale Documentation

Appearance

Command-Line Client for Build (teamscale-build) ​

teamscale-build is a native command-line tool that supports typical use cases as faced on continuous integration servers and in CI/CD pipelines. The tool currently supports:

  • compressing coverage reports from common formats to a single compact Teamscale-specific report. This reduces the database size of Teamscale, the size of Teamscale backups and improves analysis performance.
  • converting multiple test-specific coverage reports to a single Testwise Coverage report. This is useful if your tooling can produce one coverage report per test case but not our custom Testwise Coverage JSON format.

How-To Guide

Our Tutorial describes how to use this tool in practice with a simple example.

Download ​

You can download the latest version here (Changelog):

Then extract the zip file and place it in a folder where your shell can pick it up (somewhere on your PATH).

On macOS you will additionally need to run sudo xattr -d com.apple.quarantine teamscale-build to tell Gatekeeper that it may allow to execute the binary.

The command-line client exposes its functionality as a set of commands. You can use the built-in help option, --help, to list all available commands.

shell
teamscale-build --help

The various commands can be further configured by command-line options. In particular, you can use the help option, --help, built into all commands to list their respective options. For example:

shell
teamscale-build coverage testwise --help

Many options are common, while others are specific to a command and explained in the following alongside their respective command.

The coverage compact command ​

The coverage compact command converts multiple coverage reports of one format to a single Teamscale Compact coverage report. This format is a Teamscale-specific coverage format that allows to upload coverage information in a compressed format to reduce the amount of data processed by the Teamscale instance. If you are unfamiliar with the Compact coverage format, you might want to read the specification first. An example of the command would be:

shell
teamscale-build coverage compact -i /tmp/project -o /tmp/compact-coverage.json -f JACOCO

Command-specific options ​

There exists the following command-specific option:

  • -f or --format The coverage format, must be one of the supported formats.
  • -u/--include-uncovered-lines (optional): Flag if uncovered lines should be included in the output. This is only relevant information for preprocessor languages like C/C++/C#. Default is false. You will have to enable the feature toggle -Dcom.teamscale.feature-toggle.use-coverable-lines-from-coverage-reports=true on your Teamscale instance to let Teamscale make use of this data. By default, Teamscale will ignore that information and determine the coverable lines heuristically.

The coverage testwise command ​

The coverage testwise command converts multiple coverage reports of one format to a single Teamscale Testwise Coverage report. If you are unfamiliar with the Testwise Coverage format, you might want to read the specification first. An example of the command would be:

shell
teamscale-build coverage testwise -i /tmp/project -o /tmp/testwise-coverage.json

This scans the given directory recursively for files, parses all of them as JaCoCo reports and outputs a merged Testwise Coverage report to the given json file.

Input path structure ​

The provided input paths (given via -i) will be traversed recursively for files named metadata.json. This metadata.json is expected to contain a testInfo object with the test's path, duration and result. A metadata.json might look like this:

{ "testInfo": 
  {
    "uniformPath": "my/cool/Test1", 
    "result": "PASSED",
    "duration": 1 
  }
 }

The given fields are:

  • uniformPath(String): Represents the test name that will be shown in Teamscale
  • result(String, default PASSED): The test execution result. Possible values for result include PASSED, IGNORED, SKIPPED, FAILURE, ERROR, and INCONCLUSIVE. See the Testwise Coverage specification for details on the semantics of those values.
  • duration(double, default 1): Duration of the test execution interpreted as seconds For a full reference of supported fields see TestInfo.

teamscale-build will look for test coverage to associate with the test next to the metadata.json. The coverage must be placed in directories named after the report formats (see supported formats) contained in them. If the folders contain multiple reports they will be merged.

An example directory schema be:

root
  arbitarily/many/subfolders
    testcase-folder1
      metadata.json
      JACOCO
        report1.xml
        report2.xml
        ...
    testcase-folder2
      metadata.json
      SIMPLE
        simple-coverage.txt
    ...

Where root is provided via the -i option. In this example, the coverage information from report1.xml and report2.xml would be mapped to the information provided in the metadata.json of testcase-folder1 (JaCoCo coverage) and simple-coverage.txt will be linked to the metadata.json of testcase-folder2 (SIMPLE coverage).

Command-specific options ​

The following command-specific options are available:

  • -c/--complete (optional): By default, the generated Testwise Coverage report is flagged as "partial", meaning Testwise Coverage in Teamscale for tests that are not included in this report is kept unchanged. If instead you’d like all tests that are not part of the generated report to be removed from Teamscale (e.g. because the tests were deleted meanwhile), you can mark the generated report as complete with this flag.
  • -u/--include-coverable-lines: (optional): Determines whether coverable lines should be included in the output. This is only relevant information for languages with a pre-processor like C/C++/C#. Default is false. You will have to enable the feature toggle -Dcom.teamscale.feature-toggle.use-coverable-lines-from-coverage-reports=true on your Teamscale instance to let Teamscale make use of this data. By default, Teamscale will ignore that information and determine the coverable lines heuristically.

The coverage raw-testwise command ​

The coverage raw-testwise command converts multiple Bullseye coverage reports to a single Raw Testwise Coverage report. The Raw Testwise Coverage report is very similar to the output of the coverage testwise command. Bullseye reports however cannot be converted via the coverage testwise command, since the original source code is needed for the conversion. Thus, this command packages the original reports into a single file that can be uploaded to Teamscale where the conversion happens. An example of the command would be:

shell
teamscale-build coverage raw-testwise -i /tmp/project -o /tmp/raw-testwise-coverage.json

For the upload you need to specify the RAW_TESTWISE format, which is available since Teamscale 9.2.

Input path structure ​

The input path and directory schema is the same as in the testwise command. The raw-testwise command supports any of Teamscale's supported coverage formats, but its main use-case remains the BULLSEYE format and the non-raw testwise command should be preferred if possible.

Command-specific options ​

The following command-specific options are available:

  • -c/--complete (optional): By default, the generated Raw Testwise Coverage report is flagged as "partial", meaning Testwise Coverage in Teamscale for tests that are not included in this report is kept unchanged. If instead you’d like all tests that are not part of the generated report to be removed from Teamscale (e.g. because the tests were deleted meanwhile), you can mark the generated report as complete with this flag.

Common Options ​

Many command-line options are common to all commands offered by teamscale-build. These are described in the following.

Input path ​

Option name: -i or --input

Path of a report file or a directory containing report files that should be parsed. Can be used multiple times within the same command. All reports must have the same coverage format.

Input coverage format ​

The report format of input coverage reports (case-insensitive). Can only be used once within the same command. Supported formats are

Teamscale-supported formats

The supported formats are a subset of all Teamscale-supported formats.

Output path ​

Option name: -o or --output

Path of the to be generated coverage report output file. In case a file exists at this path, the file will not be overwritten, but instead, the command fails.

Logging ​

Option name: --stacktrace

In case any problems arise, or you need to report an issue to our support, you can log trace information by using the --stacktrace option.