How to Upload External Analysis Results to Teamscale

It is possible to upload external analysis results to Teamscale, either via the REST API or via repository connectors. External analysis results include test coverage reports, test execution results, custom findings, custom metrics and more.

• Upload via Web UI
• Upload via Command Line
  • Uploading Multiple Report Files of the Same Format
  • Uploading Multiple Files with Different Formats
  • Specifying the Teamscale access key
• Upload via GitHub Action
• Upload via REST API
  • Prerequisites
  • Uploading a single report
  • Uploading data for historic dates or revisions
  • Uploading Multiple Reports At Once (Sessions)
• Upload via Gradle Plugin
• Import via Artifactory / S3
  • Layout Prerequisites
  • Configuration Steps
  • Configuration Example

Upload via Web UI

For testing and debugging purposes it can sometimes be helpful to manually upload reports via the Web UI.

Upload via Command Line

With our command line utility teamscale-upload (opens new window) for Windows and Linux you can easily upload external analysis reports to Teamscale.

If you want to upload a report file from your CI pipeline, you can run:

teamscale-upload --server http://<TEAMSCALE_URL> --project <PROJECT> --user <USER> --accesskey <ACCESSKEY> --partition <PARTITION> --format <FORMAT> <PATH/TO/REPORT/FILE>

This command will automatically detect the version control commit to which the report belongs by querying your CI system (if it is supported, e.g. Jenkins, Azure DevOps, ...) or any Git or SVN checkout in the current working directory.

GitHub Action

If you are using GitHub as your CI system we recommend using teamscale-upload via GitHub Action which is even easier to integrate into your CI workflow.

Necessary Permissions in Teamscale

The user you supply on the command line must have the Perform External Uploads permission for the project to which you are uploading the reports. For a production setup we recommend that you create a technical user in Teamscale and assign it the Build role for that project.

Self-Signed Certificates

If you're accessing Teamscale via HTTPS and are signing the certificates yourself or via an internal CA, you can simplify the initial setup of this tool by using --insecure. This will disable certificate validation.

Once you're successfully uploading your analysis results, you can re-enable validation and provide the necessary certificates as a Java key store via --trusted-keystore.

Valid Formats

Choosing the correct format is crucial. Otherwise, Teamscale cannot interpret the report file and will log errors to the Worker Log. It must be one of the formats supported by Teamscale. Casing does not matter.

Check the Exit Code and Command Line Output

teamscale-upload will exit with code 0 if the upload was fully successful. Any other exit code indicates a problem that you must address. The problem is logged to the console.

Further Options

Run teamscale-upload --help to see all available options.

Uploading Multiple Report Files of the Same Format

If you need to upload more than one file of the same format, you can either specify more than one path on the command line or use wildcards:

• * matches any number of characters in the current directory or file name
• ** matches any number of nested directories
• ? matches any single character

Always Use Single Quotes Around Wildcard Strings

Since * and ? are characters that are also interpreted by your shell, you must always use single quotes around paths with wildcards.

E.g. 'path/**/report*.xml' matches all of the following:

path/to/nested/report.xml
path/report.xml
path/report123.xml

Uploading Multiple Files with Different Formats

If you need to upload more than one type of report, e.g. test coverage results and findings, you can use an input file. In this file, you can define multiple formats and one or more patterns per format, e.g:

[jacoco]
jacoco1/**.xml
jacoco2/**.xml
[findbugs]
pattern1/**.findbugs.xml
pattern2/**.findbugs.xml

Use this file with the --input command line option:

teamscale-upload --server http://<TEAMSCALE_URL> --project <PROJECT> --user <USER> --accesskey <ACCESSKEY> --partition <PARTITION> --input <PATH/TO/INPUT/FILE>

Specifying the Teamscale access key

teamscale-upload offers two ways to securely specify the Teamscale access key:

• Via specifying the environment variable $TEAMSCALE_ACCESS_KEY
  • Example:

$ export TEAMSCALE_ACCESS_KEY=<key>
$ teamscale-upload --server http://<TEAMSCALE_URL> --project <PROJECT> --user <USER> --partition <PARTITION> --format <FORMAT> <PATH/TO/REPORT/FILE>

• Via the standard input by specifying accesskey - and then entering the key via stdin.
  • Example:

$ cat ~/accesskey.txt | teamscale-upload --server http://<TEAMSCALE_URL> --project <PROJECT> --user <USER> --accesskey - --partition <PARTITION> --format <FORMAT> <PATH/TO/REPORT/FILE>

Additional way to specify the Teamscale access key (not recommended)

There is one additional unrecommended way to specify the Teamscale access key: Entering it directly via the command-line option --accesskey <key>. As it leaks the provided key to the standard output in an uncensored fashion, it should only be used when the above options are not applicable.

$ teamscale-upload --server http://<TEAMSCALE_URL> --project <PROJECT> --user <USER> --accesskey <key> --partition <PARTITION> --format <FORMAT> <PATH/TO/REPORT/FILE>

Upload via GitHub Action

If you are using GitHub as your CI system we recommend using teamscale-upload via GitHub Action which is available on the GitHub Marketplace (opens new window). It provides the same options as teamscale-upload and can be used as follows:

uses: 'cqse/teamscale-upload-action@v2.4.0'
    with:
      server: 'https://demo.teamscale.com'
      project: 'teamscale-upload'
      user: 'build'
      partition: 'Github Action > Linux Branch And Timestamp'
      accesskey: ${{ secrets.ACCESS_KEY }}
      format: 'SIMPLE'
      branch-and-timestamp: 'master:HEAD'
      message: 'This is a test message.'
      files: 'test_resources/coverage.simple test_resources/coverage2.simple'

A more detailed description of the parameters can be found here.

Upload via REST API

If you cannot use the teamscale-upload utility, you can also manually upload files via our REST API. We do recommend that you use teamscale-upload if possible since it includes helpful handling for many common errors, typos, etc.

The REST service external-analysis/session/<SESSION_ID>/report allows uploading results from analysis tools like FindBugs/SpotBugs (opens new window) or different coverage tools.

Prerequisites

In order to upload a report to Teamscale, the server which is running the Teamscale instance must be accessible via HTTP(S).

As a first step, create a new user which will be able to upload external analysis results to Teamscale. You can either use your administration account for this, or better use a special user with the Build role.

Uploading a single report

To upload a single external analysis report to Teamscale for the current HEAD commit (see below for uploads for historic dates), issue the following HTTP request:

• HTTP method: POST

• Request URL:

  http://<YOUR_TEAMSCALE_SERVER>:<PORT>/api/projects/<TEAMSCALE_PROJECT_ID>/external-analysis/session/auto-create/report?format=<FORMAT>&partition=<PARTITION>&message=<MESSAGE>
  

  • YOUR_TEAMSCALE_SERVER:PORT is the address and port of the Teamscale instance.
  • TEAMSCALE_PROJECT_ID is the ID of the Teamscale project into which to store the coverage.
  • FORMAT is the report format of the uploaded file. It must be one of the formats supported by Teamscale.
  • PARTITION is a logical name for the type of coverage. All coverage with the same partition will be grouped together. This, e.g., allows you to upload both coverage for your Java and your JavaScript tests without the results of one overriding those of the other. The name is descriptive and can be any arbitrary string.
  • MESSAGE is the message shown in the Activity perspective to the user. It should make clear that coverage was uploaded.

• Body: Form data. May contain any number of elements with name "report" and the content of a single result XML file.

• Authentication: HTTP Basic Authentication with the build or admin user and their Access Token (not their password!)

Sample Upload with Curl

curl --request POST --user bob:e434mikm35d --form "report=@coverage.xml" "http://teamscale-server:8080/api/projects/myProject/external-analysis/session/auto-create/report?format=JACOCO&partition=Unit%20Tests&message=Unit%20Test%20Coverage"

Always Quote the URL

Please note that the URL must be enclosed in quotes. This is required as most shells treat the & character specially and would thus truncate the URL. If you are not sure you quoted the URL correctly, you can use curl -v to get verbose output from curl, including the actual URL it tries to contact.

File encoding

If not explicitly specified, Teamscale will interpret the encoding of the provided file based on it's BOM, or default to UTF-8. Change the charset of the Content-Type header when uploading a Teamscale report via curl to prevent parsing errors because of misinterpreted encodings.

An example how to upload a report with a specified UTF-16 encoding:

curl --request POST --user bob:e434mikm35d --form "report=@coverage.xml;type=text/xml;charset=UTF-16" "http://teamscale-server:8080/api/projects/myProject/external-analysis/session/auto-create/report?format=JACOCO&partition=Unit%20Tests&message=Unit%20Test%20Coverage"

Note the type definition type=text/xml;charset=UTF-16 after the file name.

Request Parameters

Details about the service's parameters can be best retrieved via the online service documentation as described here.

In addition to the common pitfalls when dealing with the REST API, you need to check if the external tool is activated in the project's analysis profile.

Furthermore, please refer to the details for supported upload formats.

Uploading data for historic dates or revisions

Note that the uploaded results will be used for the current HEAD commit in Teamscale on the main branch (e.g., latest commit on master for Git, latest changeset on the TFS main branch or latest commit on trunk for SVN). Should this not be the behaviour you want, you can override this by adding the following to the request's URL:

&revision=<REVISION>

where REVISION is, depending on your version control system, a:

• full Git SHA hash, or
• SVN revision number, or
• TFS changeset ID

Regardless of whether the revision has already been analyzed or not, Teamscale will store the external analysis results for the code with the revision. When later it is analyzed, the upload is integrated with the revision into Teamscale.

If concerned project has a multi-connector setup, the VCS repository that contains the revision for which an upload is meant, can be specified in the request's URL using the repository identifier (see connector options) for the project connector:

&repository=<REPOSITORY_IDENTIFIER>

Teamscale also supports uploading external analysis results to timestamps. This can be achieved by adding the following to the request's URL:

&t=<TIMESTAMP>

If you additionally need to specify the branch for the upload, you can use the t parameter as well:

&t=<BRANCH>:<TIMESTAMP>

Uploading Multiple Reports At Once (Sessions)

If you want to upload multiple reports in a single logical commit, you should use Teamscale's session concept. For session handling, the service external-analysis/session is used. The following example uploads two different reports as a single commit using curl. For more details, see the service documentation.

curl --request POST --user bob:e434mikm35d http://localhost:8080/api/projects/my-project-id/external-analysis/session?message=myMessage&partition=myPartition

This command will return the session ID, in this case Sdfb38cb0-c348-40fb-82f0-144bf0e41beb. Using this session, we can upload multiple reports.

curl --request POST --user bob:e434mikm35d --form "report=@./MyReport.xml" http://localhost:8080/api/projects/myProject/external-analysis/session/Sdfb38cb0-c348-40fb-82f0-144bf0e41beb/report?format=FINDBUGS

curl --request POST --user bob:e434mikm35d --form "report=@./OtherReport.xml" http://localhost:8080/api/projects/myProject/external-analysis/session/Sdfb38cb0-c348-40fb-82f0-144bf0e41beb/report?format=FXCOP

Finally, we commit the session using a POST request:

curl --request POST --user bob:e434mikm35d http://localhost:8080/api/projects/my-project-id/external-analysis/session/Sdfb38cb0-c348-40fb-82f0-144bf0e41beb

Upload via Gradle Plugin

With our Teamscale Gradle Plugin (opens new window) you can easily upload artifacts directly from Gradle without any additional tooling involved. It can automate uploading JUnit, JaCoCo and Testwise Coverage reports. This guide expects that you have already configured your test tasks and the jacoco plugin (opens new window).

Test Impact Analysis

For more in depth instructions on how to set up Testwise Coverage collection and Impacted Test execution please refer to our tutorial.

You can apply the plugin by adding it to your plugins block of the top level project.

plugins {
	id "com.teamscale" version "23.1.1"
}

The teamscale extension allows you to configure where reports should be uploaded to:

teamscale {
	server {
		url = 'https://teamscale.mycompany.com:8080'
		project = 'test'
		userName = 'build'
		userAccessToken = property('teamscale.access-token')
	}
}

The example assumes that you pass the Teamscale Access Token to Gradle via

gradle -Pteamscale.access-token=82l1jtkIx6xG7DDG34FLsKhejcHz1cMu ...`

By default the plugin will try to detect the correct commit it needs to upload to from the Git repository. In case the Git repository is not present when Gradle is executed, which is the case in some CI setups, you need to manually specify the commit hash:

teamscale {
	commit {
		revision = System.getenv('CI_COMMIT_SHA')
	}
}

You can specify which artifacts you want to upload and to which partition via the reports block:

teamscale {
	report {
		jacoco()
		junit()
		testwiseCoverage()
	}
}

These calls will configure all existing JacocoReport, Test and TestImpacted tasks respectively to produce the necessary reports and pick them up after the tasks were executed.

All aspects of the upload can be configured individually:

teamscale {
	report {
		jacoco {
			// The partition all JaCoCo reports should be uploaded to
			// This is the only required setting
			partition = "Unit Tests"
		}
		junit {
			partition = "Automated Tests"
			// This is the message that will be shown in Teamscale for the artificial coverage commit (optional)
			message = "JUNIT gradle upload" // This is the default value
			// Whether reports of this type should be uploaded (Optional, default is true)
			upload = true
		}
	}
}

tasks.named("test") {
	// ...
	teamscale {
		report {
			// partition, message and upload can also be overwritten for individual tasks
			upload = false
		}
	}
}

The actual upload is performed with the teamscaleReportUpload task of the top level project. It is not executed automatically, so you need to explicitly call it either as part of your CI Gradle command

gradle --continue test jacocoTestReport teamscaleReportUpload

or model the desired behavior in Gradle e.g.

tasks.named("jacocoTestReport") {
	// Report should always be uploaded after it was generated
	finalizedBy teamscaleReportUpload
}
tasks.named("teamscaleReportUpload") {
	// Report generation is required to run before generating the report
	dependsOn jacocoTestReport
}

Import via Artifactory / S3

External analysis results can be directly imported to Teamscale via the Artifactory or S3 repository connector. The connector will extract the results from the repository, process the reports and store them in the configured partition.

Layout Prerequisites

Before importing external analysis results to Teamscale via Artifactory / S3, please ensure the following:

• All relevant external results reports are available in Artifactory / S3.
• The external results reports are packed in archives (i.e. ZIP files).
• The folder structure used follows this pattern:

  / arbitrarily / many / directories / before / the / upload / root
     / 'uploads'
     / <branch name> 
     / <commit timestamp in ms (Java long)>[-<commit hash>]?
     / <upload partition> 
     / <artifact type (EReportFormat, case insensitive), e.g. 'jacoco'> 
     / none / or / some / intermediate / directories /
     / <[uncompressed file | Zip archive file ]>
  

  All profilers / upload tools provided with Teamscale follow this format automatically, making it easy to configure. If you use your own schema, make sure to adapt the extraction patterns accordingly.

Configuration Steps

To import external results reports to a project:

1. Add an Artifactory / S3 connector to the project.
2. Configure the Artifactory connector options so that Teamscale can extract the reports from Artifactory.
3. If you used the standard path format as specified in Prerequisites, you can skip this step. Otherwise, configure the following options so that Teamscale can process the reports:
   • Analysis report mapping: Ant pattern used to map the reports inside the ZIP archives to their corresponding report format. Example: **/junit.xml -> JUNIT, **/jacoco.xml -> JACOCO
   • Partition Pattern: Regular expression used to extract the partition from the path of the artifacts. The first non-null group is used as partition, e.g. /coverage/([^/]+)/ would use the sub-folders of coverage as the partition names.
4. Save the project.

Configuration Example

Below is the structure of the Artifactory containing the external results reports.

• lcov-reports.zip contains the LCOV coverage report (lcov.info).
• findings-reports.zip contains the external findings report for findings (warnings.json).

- MyRepository
|
| -> MyProjectName
  |
  | -> uploads
    |
    | -> master
      |
      | -> 1578650400000-4e8afe52737255ba40875a1af1fced1e08fc5316
        |
        | -> coverage
          |
          | -> LCOV
            |
            | -> lcov-reports.zip
        |
        | -> findings
          |
          | -> GENERIC_FINDINGS
            |
            | -> findings-reports.zip
      |
      | -> 1578651300000-70eb476d01abd0d877d064f57a6da5e14c677687
        |
        | -> coverage
          |
          | -> LCOV
            |
            | -> lcov-reports.zip

Accordingly, the Artifactory connector's configuration would look as follows:

And this would be the end result and how the uploaded results via Artifactory would appear in Teamscale: