# How to Set Up External Tools & Formats

# Clang Static Analyzer

The Open Source LLVM clang compiler includes a static analyzer that performs dataflow analysis on C/C++ code. Analysis is started using the scan-build script, which wraps the make process. To enable output in plist format (which we require), the option -plist must be passed to the scan-build command. Additionally, you might want to enable additional checkers using the -enable-checker command. To obtain a list of all available checkers, run the following command:

clang -cc1 -analyzer-checker-help

Note that it is sufficient to provide a package/category to the analyzer to enable more checks. A typical invocation of the static analyzer might look like this:

scan-build -plist
-enable-checker alpha
-enable-checker core
-enable-checker cplusplus
-enable-checker deadcode
-enable-checker security
-enable-checker unix
make

To upload analysis results, you should use the external-report service as described in our article on external uploads.

# Clang-Tidy

While Clang-Tidy is integrated as an internal analysis tool, the Clang-Tidy binaries are generally not shipped with Teamscale. Our docker-image distributions contain the correct clang-tidy version, but distributions based on zip-files do not contain clang-tidy. Instead, you will need to install clang-tidy 10.0.0 on your machine. More details are described here.

# FindBugs and SpotBugs

FindBugs is an Open Source bug pattern search tool working on the byte code of Java programs. This means that in order to use FindBugs, you have to first compile the code.

SpotBugs is the successor of FindBugs, which is no longer maintained. Since SpotBugs reports are compatible with the FindBugs report format, they can be uploaded with the same analysis-tool name (FINDBUGS). To upload analysis results, you should use the external-report service as described here.

# JaCoCo

JaCoCo is an Open Source code coverage tool for Java. Teamscale allows to upload XML coverage reports produced by JaCoCo. Please refer to our Java test coverage guide for instructions on how to set up JaCoCo for your tests.

To upload JaCoCo coverage data, you should use the external-report service as described here.

# Cobertura

Cobertura is an Open Source code coverage tool for Java. Teamscale allows to upload XML coverage reports produced by Cobertura. You can generate an XML report with the Cobertura Ant task cobertura-report by setting the parameter format to xml as follows:

<cobertura-report format="xml"
	destdir="${coveragereport.dir}"
	srcdir="${src.dir}" />

For further details on the usage of the Cobertura Ant tasks, please refer to the Cobertura user manual.

To upload Cobertura coverage data, you should use the external-report service as described here.

# gcov

The tool gcov measures line coverage for C/C++ code compiled with the GCC compiler. To determine the code coverage, the code must be compiled using the switch --coverage. This creates files with the extension gcno during compilation. During execution of the code, the coverage data is stored into files with ending gcda. The program gcov is then used to convert the coverage data into readable files with the extension gcov as follows:

gcov -o <OBJ_DIR> -l -p <SOURCE_FILES>

Here the <OBJ_DIR> is the directory containing the object files. For projects with many subdirectories it should be noted that the parameter must point directly to the directory containing the matching object. The relative path of the source files is not used. The paramters -l and -p are used to ensure that files with the same name but different directories do not collide.

Alternatively, the python script gcovr can be used to execute gcov for complex project layouts. To preserve the gcov files, the parameter --keep must be used. Please note that gcovr does not support merging of coverage files (see gcovr FAQ and this StackOverflow question, which is needed when code is executed in more than one test run. If you have to merge coverage files consider using lcov (also see this StackOverflow question) .

To upload coverage results to Teamscale, you should use the external-report service as described here.

# ESLint

While ESLint is an internal analysis tool, the ESLint binaries are not shipped with Teamscale. Instead, you will need to install NodeJS on your machine. Once NodeJS is installed, you can run the following command to install the correct ESLint version:

npm i -g eslint@0.24.1

You will need to have both the NodeJS and ESLint binaries on the PATH and restart Teamscale before you can enable ESLint for your project.

# TSLint

While TSLint is an internal analysis tool, the TSLint binaries are not shipped with Teamscale. Instead, you will need to install NodeJS on your machine. Once NodeJS is installed, you can run the following command to install the correct TSLint version:

npm i -g tslint@5.9.1 typescript@2.7.1

You will need to have both the NodeJS and TSLint binaries on the PATH and restart Teamscale before you can enable TSLint for your project.

# abaplint

While abaplint is an internal analysis tool, the abaplint binaries are not shipped with Teamscale. Instead, you will need to install NodeJS on your machine. Once NodeJS is installed, you can run the following command to install the correct TSLint version:

npm i -g @abaplint/cli@2.32.1

You will need to have both the NodeJS and abaplint binaries on the PATH and restart Teamscale before you can enable abaplint for your project.

# PC-lint

PC-lint is a static analysis tool for C/C++ that covers a wide range of checks. While the tool is delivered as a Windows executable, it can also be used with Linux using wine and a wrapper script. PClint is fully integrated into Teamscale as an incremental analysis tool, but the tool has to be on the path or the explicit path to the tool must be configured in the analysis profile.

Note for running PC-lint with Teamscale on Windows:

If PC-lint crashes during the execution, the Windows crash dialog will open and block Teamscale until the dialog is closed manually. To prevent this behavior, log in as the user running Teamscale and set the following key in the registry to 1:

HKEY_CURRENT_USER\Software\Microsoft\Windows\Windows Error Reporting\DontShowUI

The check descriptions in Teamscale are up-to-date with PC-lint version 9.00L (i.e., Teamscale can import and display findings from a report generated by this PC-lint version).

# Pylint

Pylint is an Open Source code analysis tool for Python working directly on the source code.

To import Pylint analysis results into Teamscale, you need to create the generated report using a special format. This can be achieved by using the following command line call:

pylint --score=no --reports=no --msg-template='{{line: {line}, messageId: "{msg_id}", message: "{msg}", absolutePath: "{abspath}", messageName: "{symbol}"}}' /path/to/python/files > report.json

This command generates a quasi JSON file which can be uploaded to Teamscale, for example with the Teamscale Python client’s upload_report method, manually via the external uploads view for your project or simply using curl via the REST interface.
The above command is necessary because the JSON report which can be generated directly with Pylint (i.e. pylint -f json ...) does not contain all information necessary to seamlessly integrate the report into Teamscale.

# StyleCop

StyleCop is an Open Source style violation checker for C# working directly on the source code. StyleCop is full integrated into Teamscale as an incremental analysis step, so to run StyleCop, you only have to enable it when configuring the analysis profile.

TIP

When running on Windows, the .NET framework v4.5 or higher must be installed. When running on Linux, Mono 4.0 or higher has to be installed to execute StyleCop.

# Parasoft C/C++test

The reports from Parasoft C/C++test are fully self-contained. The content and structure, however, depend completely on the configuration of the tool. Hence, before the first import of such a report, the findings need to be made known to Teamscale using the following steps:

  1. Create a new analysis profile (e.g., called CppWithParasoft) with Parasoft C/C++test enabled in the tools section
  2. Call the following service to upload the report and update the analysis profile:
curl -X POST -F report=@parasoft.xml 'http://<teamscale-url>/custom-parasoft-findings?analysis-profile=CppWithParasoft'
  1. The analysis profile should now be updated with rules and can be adjusted if needed
  2. Create a project with the updated analysis profile

# Xcode Test Coverage

Teamscale supports importing line-based test coverage produced by Xcode and the xccov command line tool. As the Xcode tooling cannot produce a comprehensive report with line coverage information for all files in a project, the following shell script must be used to obtain a coverage report that can be imported to Teamscale:

#!/bin/bash
if [ $# -eq 0 ]
  then
  echo "Please provide path to xccovarchive directory"
  exit 1
fi
coverage=""
archive=$1
files=$(xcrun xccov view "$archive" --file-list)
while read -r file; do
  coverage+="$file"
  coverage+="\n"
  coverage+="$(xcrun xccov view "$archive" --file "$file")"
done <<< "$files"
echo "${coverage}"

This command will output the coverage report to standard out. You can pipe the output into a file which can then be uploaded via the REST API as described here.

./generate-coverage /path/to/xccovarchive > coverage.txt

# BullseyeCoverage

BullseyeCoverage can generate C and C++ code coverage for a wide variety of Linux, Windows and embedded C and C++ compilers. It works by instrumenting the source code and can thus also be used for software that must be tested with optimizations enabled, as well as for manual tests.

To instrument your source code with Bullseye, first install Bullseye on the computer that builds your software (e.g., your build agents).

  • For a Windows installation: Select Install for all users, build servers, services, kernel mode debugging during the installation.
  • Select all relevant compilers during the installation. For Windows please make sure to select at least Microsoft Visual C++ Build Tools and Microsoft Visual Studio XXXX (where XXXX is the Visual Studio version you use).
  • Select and remember the path where the .cov file will be stored. This location must be writable by your build process.

In order to produce an instrumented binary for your tests, please modify your build scripts to:

  1. Delete the old .cov file if present
  2. Run cov01 -1 to enable instrumentation
  3. Build your software as usual. e.g., run msbuild or make etc. Bullseye will automatically instrument all code that is built after enabling instrumentation. I.e. if your build consists of several subprojects, they will all be instrumented. To configure which code should be instrumented, please refer to the Bullseye documentation on excludes.

Separate `.cov` Files for Parallel Builds

To change the path where the .cov file will be generated by the build process, you can set the environment variable COVFILE. This allows e.g., always generating it in the local build directory instead of in a fixed global system directory. This also prevents problems when multiple builds are running in parallel, as these would otherwise write to the same .cov file.

After this, a .cov file will be generated that matches the code you just compiled in the location you specified (either via the COVFILE environment variable or in the location you selected during the installation of Bullseye). Make sure to save it together with your build output so you always use the correct .cov file that corresponds to the instrumented binary.

In order to record test coverage with the instrumented binary

  1. Deploy the instrumented binary and its corresponding .cov file to your test environment (this works both for automated tests and for manual tests).
  2. Set the environment variable COVFILE to the absolute path of the .cov file.
  3. Start your application as you normally would (make sure that the application's process sees the COVFILE environment variable).
  4. After your application has terminated, all coverage data will have been written to the .cov file.

In order to upload the recorded coverage to Teamscale

  1. Convert the .cov file to XML: Run covxml -f /path/to/test.cov -o /path/to/output.xml
  2. Upload the generated XML file to Teamscale as described here.

For futher configuration options, please refer to the Bullseye documentation.