Teamscale Documentation

Appearance

Command-Line Client for Build (teamscale-build) ​

teamscale-build is a native command-line tool that supports typical use cases as faced on continuous integration servers and in CI/CD pipelines. The tool currently supports:

  • compressing coverage reports from common formats to a single compact Teamscale-specific report. This reduces the database size of Teamscale, the size of Teamscale backups and improves analysis performance.
  • converting multiple test-specific coverage reports to a single Testwise Coverage report. This is useful if your tooling can produce one coverage report per test case but not our custom Testwise Coverage JSON format.

How-To Guide

Our Tutorial describes how to use this tool in practice with a simple example.

Download ​

You can download the latest version here (Changelog):

Then extract the zip file and place it in a folder where your shell can pick it up (somewhere on your PATH).

On macOS you will additionally need to run sudo xattr -d com.apple.quarantine teamscale-build to tell Gatekeeper that it may allow to execute the binary.

The command-line client exposes its functionality as a set of commands. You can use the built-in help option, --help, to list all available commands.

shell
teamscale-build --help
teamscale-build --help

The various commands can be further configured by command-line options. In particular, you can use the help option, --help, built into all commands to list their respective options. For example:

shell
teamscale-build coverage testwise --help
teamscale-build coverage testwise --help

Many options are common, while others are specific to a command and explained in the following alongside their respective command.

The coverage compact command ​

The coverage compact command converts multiple coverage reports of one format to a single Teamscale Compact coverage report. This format is a Teamscale-specific coverage format that allows to upload coverage information in a compressed format to reduce the amount of data processed by the Teamscale instance. If you are unfamiliar with the Compact coverage format, you might want to read the specification first. An example of the command would be:

shell
teamscale-build coverage compact -i /tmp/project -o /tmp/compact-coverage.json -f JACOCO
teamscale-build coverage compact -i /tmp/project -o /tmp/compact-coverage.json -f JACOCO

Command-specific options ​

There exists the following command-specific option:

  • -u/--include-uncovered-lines (optional): Flag if uncovered lines should be included in the output. This is only relevant information for preprocessor languages like C/C++/C#. Default is false. You will have to enable the feature toggle -Dcom.teamscale.feature-toggle.use-coverable-lines-from-coverage-reports=true on your Teamscale instance to let Teamscale make use of this data. By default, Teamscale will ignore that information and determine the coverable lines heuristically.

The coverage testwise command ​

The coverage testwise command converts multiple coverage reports of one format to a single Teamscale Testwise Coverage report. If you are unfamiliar with the Testwise Coverage format, you might want to read the specification first. An example of the command would be:

shell
teamscale-build coverage testwise -i /tmp/project -o /tmp/testwise-coverage.json -f JACOCO
teamscale-build coverage testwise -i /tmp/project -o /tmp/testwise-coverage.json -f JACOCO

This scans the given directory recursively for files, parses all of them as JaCoCo reports and outputs a merged Testwise Coverage report to the given json file.

Input path structure ​

Each coverage report file holds the coverage information for the execution of one test case. The file name and directory structure of the provided input paths (given via -i) determine the test path, result, and duration:

  • The test path is based on the file's location relative to the directory specified as an input path (excluding the file extension). This is the hierarchically structured name of the test that will be shown in Teamscale. If the file is directly specified as an input path, the uniformPath will only be the file name (excluding the file extension).
  • The duration of a test case, given in seconds, can be specified by adding a floating point number with a s suffix to the file name before the extension (e.g., SampleTest.12.3s.xml for 12.3s of test runtime).
  • The result of a test case can also be specified by modifying the file name before the extension. Possible values include PASSED, IGNORED, SKIPPED, FAILURE, ERROR, and INCONCLUSIVE. The values are case-insensitive (e.g., SampleTest.passed.xml). See the Testwise Coverage specification for details on the semantics of those values.
  • Both duration and result are optional, and the order between both is not relevant.

Example: If the directory /tmp/project is passed as an input option and the file /tmp/project/module1/SampleTest.12.3s.PASSED.xml is found, it will be interpreted as a test case with the following specifications:

  • test path: module1/SampleTest
  • duration: 12.3 seconds
  • result: PASSED

Report Per Test-Suite

The coverage reports do not necessarily have to map 1:1 to a single test case, but might also be a testsuite that Teamscale will treat as one test.

Command-specific options ​

The following command-specific options are available:

  • -c/--complete (optional): By default, the generated Testwise Coverage report is flagged as "partial", meaning Testwise Coverage in Teamscale for tests that are not included in this report is kept unchanged. If instead you’d like all tests that are not part of the generated report to be removed from Teamscale (e.g. because the tests were deleted meanwhile), you can mark the generated report as complete with this flag.
  • -u/--include-coverable-lines: (optional): Determines whether coverable lines should be included in the output. This is only relevant information for languages with a pre-processor like C/C++/C#. Default is false. You will have to enable the feature toggle -Dcom.teamscale.feature-toggle.use-coverable-lines-from-coverage-reports=true on your Teamscale instance to let Teamscale make use of this data. By default, Teamscale will ignore that information and determine the coverable lines heuristically.

The coverage raw-testwise command ​

The coverage raw-testwise command converts multiple Bullseye coverage reports to a single Raw Testwise Coverage report. The Raw Testwise Coverage report is very similar to the output of the coverage testwise command. Bullseye reports however cannot be converted via the coverage testwise command, since the original source code is needed for the conversion. Thus, this command packages the original reports into a single file that can be uploaded to Teamscale where the conversion happens. An example of the command would be:

shell
teamscale-build coverage raw-testwise -i /tmp/project -o /tmp/raw-testwise-coverage.json
teamscale-build coverage raw-testwise -i /tmp/project -o /tmp/raw-testwise-coverage.json

For the upload you need to specify the RAW_TESTWISE format, which is available since Teamscale 9.2.

Command-specific options ​

The following command-specific options are available:

  • -c/--complete (optional): By default, the generated Raw Testwise Coverage report is flagged as "partial", meaning Testwise Coverage in Teamscale for tests that are not included in this report is kept unchanged. If instead you’d like all tests that are not part of the generated report to be removed from Teamscale (e.g. because the tests were deleted meanwhile), you can mark the generated report as complete with this flag.

Common Options ​

Many command-line options are common to all commands offered by teamscale-build. These are described in the following.

Input path ​

Option name: -i or --input

Path of a report file or a directory containing report files that should be parsed. Can be used multiple times within the same command. All reports must have the same coverage format.

Input coverage format ​

Option name: -f or --format

The report format of input coverage reports (case-insensitive). Can only be used once within the same command. Supported formats are

Teamscale-supported formats

The supported formats are a subset of all Teamscale-supported formats.

Output path ​

Option name: -o or --output

Path of the to be generated coverage report output file. In case a file exists at this path, the file will not be overwritten, but instead, the command fails.

Logging ​

Option name: --stacktrace

In case any problems arise, or you need to report an issue to our support, you can log trace information by using the --stacktrace option.