How to Upload External Analysis Results to Teamscale ​

It is possible to upload external analysis results to Teamscale, either via the REST API or via repository connectors. External analysis results include test coverage reports, test execution results, custom findings, custom metrics and more.

Upload via Web UI ​

For testing and debugging purposes it can sometimes be helpful to manually upload reports via the Web UI.

Upload via Command Linerecommended ​

With our command line utility teamscale-upload for Windows and Linux you can easily upload external analysis reports to Teamscale.

If you want to upload a report file from your CI pipeline, you can run:

teamscale-upload --server http://<TEAMSCALE_URL> --project <PROJECT> --user <USER> --accesskey <ACCESSKEY> --partition <PARTITION> --format <FORMAT> <PATH/TO/REPORT/FILE>

teamscale-upload --server http://<TEAMSCALE_URL> --project <PROJECT> --user <USER> --accesskey <ACCESSKEY> --partition <PARTITION> --format <FORMAT> <PATH/TO/REPORT/FILE>

This command will automatically detect the version control commit to which the report belongs by querying your CI system (if it is supported, e.g. Jenkins, Azure DevOps, ...) or any Git or SVN checkout in the current working directory.

Uploading Multiple Report Files of the Same Format ​

If you need to upload more than one file of the same format, you can either specify more than one path on the command line or use wildcards:

• * matches any number of characters in the current directory or file name
• ** matches any number of nested directories
• ? matches any single character

E.g. 'path/**/report*.xml' matches all of the following:

path/to/nested/report.xml
path/report.xml
path/report123.xml

path/to/nested/report.xml
path/report.xml
path/report123.xml

Uploading Multiple Files with Different Formats ​

If you need to upload more than one type of report, e.g. test coverage results and findings, you can use an input file. In this file, you can define multiple formats and one or more patterns per format, e.g:

[jacoco]
jacoco1/**.xml
jacoco2/**.xml
[findbugs]
pattern1/**.findbugs.xml
pattern2/**.findbugs.xml

[jacoco]
jacoco1/**.xml
jacoco2/**.xml
[findbugs]
pattern1/**.findbugs.xml
pattern2/**.findbugs.xml

Use this file with the --input command line option:

teamscale-upload --server http://<TEAMSCALE_URL> --project <PROJECT> --user <USER> --accesskey <ACCESSKEY> --partition <PARTITION> --input <PATH/TO/INPUT/FILE>

teamscale-upload --server http://<TEAMSCALE_URL> --project <PROJECT> --user <USER> --accesskey <ACCESSKEY> --partition <PARTITION> --input <PATH/TO/INPUT/FILE>

Specifying the Teamscale access key ​

teamscale-upload offers two ways to securely specify the Teamscale access key:

• Via specifying the environment variable $TEAMSCALE_ACCESS_KEY
  • Example:

$ export TEAMSCALE_ACCESS_KEY=<key>
$ teamscale-upload --server http://<TEAMSCALE_URL> --project <PROJECT> --user <USER> --partition <PARTITION> --format <FORMAT> <PATH/TO/REPORT/FILE>

$ export TEAMSCALE_ACCESS_KEY=<key>
$ teamscale-upload --server http://<TEAMSCALE_URL> --project <PROJECT> --user <USER> --partition <PARTITION> --format <FORMAT> <PATH/TO/REPORT/FILE>

• Via the standard input by specifying accesskey - and then entering the key via stdin.
  • Example:

$ cat ~/accesskey.txt | teamscale-upload --server http://<TEAMSCALE_URL> --project <PROJECT> --user <USER> --accesskey - --partition <PARTITION> --format <FORMAT> <PATH/TO/REPORT/FILE>

$ cat ~/accesskey.txt | teamscale-upload --server http://<TEAMSCALE_URL> --project <PROJECT> --user <USER> --accesskey - --partition <PARTITION> --format <FORMAT> <PATH/TO/REPORT/FILE>

Additional way to specify the Teamscale access key (not recommended) ​

There is one additional unrecommended way to specify the Teamscale access key: Entering it directly via the command-line option --accesskey <key>. As it leaks the provided key to the standard output in an uncensored fashion, it should only be used when the above options are not applicable.

$ teamscale-upload --server http://<TEAMSCALE_URL> --project <PROJECT> --user <USER> --accesskey <key> --partition <PARTITION> --format <FORMAT> <PATH/TO/REPORT/FILE>

$ teamscale-upload --server http://<TEAMSCALE_URL> --project <PROJECT> --user <USER> --accesskey <key> --partition <PARTITION> --format <FORMAT> <PATH/TO/REPORT/FILE>

Upload via GitHub Action ​

If you are using GitHub as your CI system we recommend using teamscale-upload via GitHub Action which is available on the GitHub Marketplace. It provides the same options as teamscale-upload and can be used as follows:

uses: 'cqse/teamscale-upload-action@v2.4.0'
    with:
      server: 'https://demo.teamscale.com'
      project: 'teamscale-upload'
      user: 'build'
      partition: 'Github Action > Linux Branch And Timestamp'
      accesskey: ${{ secrets.ACCESS_KEY }}
      format: 'SIMPLE'
      message: 'This is a test message.'
      files: 'test_resources/coverage.simple test_resources/coverage2.simple'

uses: 'cqse/teamscale-upload-action@v2.4.0'
    with:
      server: 'https://demo.teamscale.com'
      project: 'teamscale-upload'
      user: 'build'
      partition: 'Github Action > Linux Branch And Timestamp'
      accesskey: ${{ secrets.ACCESS_KEY }}
      format: 'SIMPLE'
      message: 'This is a test message.'
      files: 'test_resources/coverage.simple test_resources/coverage2.simple'

A more detailed description of the parameters can be found here.

Upload via REST API ​

If you cannot use the teamscale-upload utility, you can also manually upload files via our REST API. We do recommend that you use teamscale-upload if possible since it includes helpful handling for many common errors, typos, etc.

The REST service external-analysis/session/<SESSION_ID>/report allows uploading results from analysis tools like FindBugs/SpotBugs or different coverage tools.

Prerequisites ​

In order to upload a report to Teamscale, the server which is running the Teamscale instance must be accessible via HTTP(S).

As a first step, create a new user which will be able to upload external analysis results to Teamscale. You can either use your administration account for this, or better use a special user with the Build role.

Uploading a single report ​

To upload a single external analysis report to Teamscale for the current HEAD commit (see below for uploads for historic dates), issue the following HTTP request:

• HTTP method: POST

• Request URL:

  bash

  http://<YOUR_TEAMSCALE_SERVER>:<PORT>/api/projects/<TEAMSCALE_PROJECT_ID>/external-analysis/session/auto-create/report?format=<FORMAT>&partition=<PARTITION>&message=<MESSAGE>

  http://<YOUR_TEAMSCALE_SERVER>:<PORT>/api/projects/<TEAMSCALE_PROJECT_ID>/external-analysis/session/auto-create/report?format=<FORMAT>&partition=<PARTITION>&message=<MESSAGE>

  • YOUR_TEAMSCALE_SERVER:PORT is the address and port of the Teamscale instance.
  • TEAMSCALE_PROJECT_ID is the ID of the Teamscale project into which to store the coverage.
  • FORMAT is the report format of the uploaded file. It must be one of the formats supported by Teamscale.
  • PARTITION is a logical name for the type of coverage. All coverage with the same partition will be grouped together. This, e.g., allows you to upload both coverage for your Java and your JavaScript tests without the results of one overriding those of the other. The name is descriptive and can be any arbitrary string.
  • MESSAGE is the message shown in the Activity perspective to the user. It should make clear that coverage was uploaded.

• Body: Form data. May contain any number of elements with name "report" and the content of a single result XML file.

• Authentication: HTTP Basic Authentication with the build or admin user and their Access Token (not their password!)

Sample Upload with Curl ​

curl --request POST --user bob:e434mikm35d --form "report=@coverage.xml" "http://teamscale-server:8080/api/projects/myProject/external-analysis/session/auto-create/report?format=JACOCO&partition=Unit%20Tests&message=Unit%20Test%20Coverage"

curl --request POST --user bob:e434mikm35d --form "report=@coverage.xml" "http://teamscale-server:8080/api/projects/myProject/external-analysis/session/auto-create/report?format=JACOCO&partition=Unit%20Tests&message=Unit%20Test%20Coverage"

An example how to upload a report with a specified UTF-16 encoding:

curl --request POST --user bob:e434mikm35d --form "report=@coverage.xml;type=text/xml;charset=UTF-16" "http://teamscale-server:8080/api/projects/myProject/external-analysis/session/auto-create/report?format=JACOCO&partition=Unit%20Tests&message=Unit%20Test%20Coverage"

curl --request POST --user bob:e434mikm35d --form "report=@coverage.xml;type=text/xml;charset=UTF-16" "http://teamscale-server:8080/api/projects/myProject/external-analysis/session/auto-create/report?format=JACOCO&partition=Unit%20Tests&message=Unit%20Test%20Coverage"

Note the type definition type=text/xml;charset=UTF-16 after the file name.

In addition to the common pitfalls when dealing with the REST API, you need to check if the external tool is activated in the project's analysis profile.

Furthermore, please refer to the details for supported upload formats.

Uploading data for historic dates or revisions ​

Note that the uploaded results will be used for the current HEAD commit in Teamscale on the main branch (e.g., latest commit on master for Git, latest changeset on the TFS main branch or latest commit on trunk for SVN). Should this not be the behaviour you want, you can override this by adding the following to the request's URL:

&revision=<REVISION>

where REVISION is, depending on your version control system, a:

• full Git SHA hash, or
• SVN revision number, or
• TFS changeset ID

Regardless of whether the revision has already been analyzed or not, Teamscale will store the external analysis results for the code with the revision. When later it is analyzed, the upload is integrated with the revision into Teamscale.

If concerned project has a multi-connector setup, the VCS repository that contains the revision for which an upload is meant, can be specified in the request's URL using the repository identifier (see connector options) for the project connector:

&repository=<REPOSITORY_IDENTIFIER>

Teamscale also supports uploading external analysis results to timestamps. This can be achieved by adding the following to the request's URL:

&t=<TIMESTAMP>

If you additionally need to specify the branch for the upload, you can use the t parameter as well:

&t=<BRANCH>:<TIMESTAMP>

Uploading Multiple Reports At Once (Sessions) ​

If you want to upload multiple reports in a single logical commit, you should use Teamscale's session concept. For session handling, the service external-analysis/session is used. The following example uploads two different reports as a single commit using curl. For more details, see the service documentation.

curl --request POST --user bob:e434mikm35d http://localhost:8080/api/projects/my-project-id/external-analysis/session?message=myMessage&partition=myPartition

curl --request POST --user bob:e434mikm35d http://localhost:8080/api/projects/my-project-id/external-analysis/session?message=myMessage&partition=myPartition

This command will return the session ID, in this case Sdfb38cb0-c348-40fb-82f0-144bf0e41beb. Using this session, we can upload multiple reports.

curl --request POST --user bob:e434mikm35d --form "report=@./MyReport.xml" http://localhost:8080/api/projects/myProject/external-analysis/session/Sdfb38cb0-c348-40fb-82f0-144bf0e41beb/report?format=FINDBUGS

curl --request POST --user bob:e434mikm35d --form "report=@./MyReport.xml" http://localhost:8080/api/projects/myProject/external-analysis/session/Sdfb38cb0-c348-40fb-82f0-144bf0e41beb/report?format=FINDBUGS

curl --request POST --user bob:e434mikm35d --form "report=@./OtherReport.xml" http://localhost:8080/api/projects/myProject/external-analysis/session/Sdfb38cb0-c348-40fb-82f0-144bf0e41beb/report?format=FXCOP

curl --request POST --user bob:e434mikm35d --form "report=@./OtherReport.xml" http://localhost:8080/api/projects/myProject/external-analysis/session/Sdfb38cb0-c348-40fb-82f0-144bf0e41beb/report?format=FXCOP

Finally, we commit the session using a POST request:

curl --request POST --user bob:e434mikm35d http://localhost:8080/api/projects/my-project-id/external-analysis/session/Sdfb38cb0-c348-40fb-82f0-144bf0e41beb

curl --request POST --user bob:e434mikm35d http://localhost:8080/api/projects/my-project-id/external-analysis/session/Sdfb38cb0-c348-40fb-82f0-144bf0e41beb

Upload via Gradle Plugin ​

With our Teamscale Gradle Plugin you can easily upload artifacts directly from Gradle without any additional tooling involved. It can automate uploading JUnit, JaCoCo and Testwise Coverage reports. This guide expects that you have already configured your test tasks and the jacoco plugin.

You can apply the plugin by adding it to your plugins block of the top level project.

plugins {
	id "com.teamscale" version "27.0.1"
}

plugins {
	id "com.teamscale" version "27.0.1"
}

The teamscale extension allows you to configure where reports should be uploaded to:

teamscale {
	server {
		url = 'https://teamscale.mycompany.com:8080'
		project = 'test'
		userName = 'build'
		userAccessToken = property('teamscale.access-token')
	}
}

teamscale {
	server {
		url = 'https://teamscale.mycompany.com:8080'
		project = 'test'
		userName = 'build'
		userAccessToken = property('teamscale.access-token')
	}
}

The example assumes that you pass the Teamscale Access Token to Gradle via

gradle -Pteamscale.access-token=82l1jtkIx6xG7DDG34FLsKhejcHz1cMu ...`

gradle -Pteamscale.access-token=82l1jtkIx6xG7DDG34FLsKhejcHz1cMu ...`

By default the plugin will try to detect the correct commit it needs to upload to from the Git repository. In case the Git repository is not present when Gradle is executed, which is the case in some CI setups, you need to manually specify the commit hash:

teamscale {
	commit {
		revision = System.getenv('CI_COMMIT_SHA')
	}
}

teamscale {
	commit {
		revision = System.getenv('CI_COMMIT_SHA')
	}
}

You can specify which artifacts you want to upload and to which partition via the reports block:

teamscale {
	report {
		jacoco()
		junit()
		testwiseCoverage()
	}
}

teamscale {
	report {
		jacoco()
		junit()
		testwiseCoverage()
	}
}

These calls will configure all existing JacocoReport, Test and TestImpacted tasks respectively to produce the necessary reports and pick them up after the tasks were executed.

All aspects of the upload can be configured individually:

teamscale {
	report {
		jacoco {
			// The partition all JaCoCo reports should be uploaded to
			// This is the only required setting
			partition = "Unit Tests"
		}
		junit {
			partition = "Automated Tests"
			// This is the message that will be shown in Teamscale for the artificial coverage commit (optional)
			message = "JUNIT gradle upload" // This is the default value
			// Whether reports of this type should be uploaded (Optional, default is true)
			upload = true
		}
	}
}

tasks.named("test") {
	// ...
	teamscale {
		report {
			// partition, message and upload can also be overwritten for individual tasks
			upload = false
		}
	}
}

teamscale {
	report {
		jacoco {
			// The partition all JaCoCo reports should be uploaded to
			// This is the only required setting
			partition = "Unit Tests"
		}
		junit {
			partition = "Automated Tests"
			// This is the message that will be shown in Teamscale for the artificial coverage commit (optional)
			message = "JUNIT gradle upload" // This is the default value
			// Whether reports of this type should be uploaded (Optional, default is true)
			upload = true
		}
	}
}

tasks.named("test") {
	// ...
	teamscale {
		report {
			// partition, message and upload can also be overwritten for individual tasks
			upload = false
		}
	}
}

The actual upload is performed with the teamscaleReportUpload task of the top level project. It is not executed automatically, so you need to explicitly call it either as part of your CI Gradle command

gradle --continue test jacocoTestReport teamscaleReportUpload

gradle --continue test jacocoTestReport teamscaleReportUpload

or model the desired behavior in Gradle e.g.

tasks.named("jacocoTestReport") {
	// Report should always be uploaded after it was generated
	finalizedBy teamscaleReportUpload
}
tasks.named("teamscaleReportUpload") {
	// Report generation is required to run before generating the report
	dependsOn jacocoTestReport
}

tasks.named("jacocoTestReport") {
	// Report should always be uploaded after it was generated
	finalizedBy teamscaleReportUpload
}
tasks.named("teamscaleReportUpload") {
	// Report generation is required to run before generating the report
	dependsOn jacocoTestReport
}

Import via Artifactory / S3 ​

External analysis results can be directly imported to Teamscale via the Artifactory or S3 repository connector. The connector will extract the results from the repository, process the reports and store them in the configured partition.

Layout Prerequisites ​

Before importing external analysis results to Teamscale via Artifactory / S3, please ensure the following:

• All relevant external results reports are available in Artifactory / S3.
• The external results reports are packed in archives (i.e. ZIP files).
• The folder structure used follows this pattern:

  / arbitrarily / many / directories / before / the / upload / root
     / 'uploads'
     / <branch name> 
     / <commit timestamp in ms (Java long)>[-<commit hash>]?
     / <upload partition> 
     / <artifact type (EReportFormat, case insensitive), e.g. 'jacoco'> 
     / none / or / some / intermediate / directories /
     / <Zip archive file>

  / arbitrarily / many / directories / before / the / upload / root
     / 'uploads'
     / <branch name> 
     / <commit timestamp in ms (Java long)>[-<commit hash>]?
     / <upload partition> 
     / <artifact type (EReportFormat, case insensitive), e.g. 'jacoco'> 
     / none / or / some / intermediate / directories /
     / <Zip archive file>

  All profilers / upload tools provided with Teamscale follow this format automatically, making it easy to configure. If you use your own schema, make sure to adapt the extraction patterns accordingly.

Configuration Steps ​

To import external results reports to a project:

1. Add an Artifactory / S3 connector to the project.
2. Configure the Artifactory connector options so that Teamscale can extract the reports from Artifactory.
3. If you used the standard path format as specified in Prerequisites, you can skip this step. Otherwise, configure the following options so that Teamscale can process the reports:
   • Analysis report mapping: Ant pattern used to map the reports inside the ZIP archives to their corresponding report format. Example: **/junit.xml -> JUNIT, **/jacoco.xml -> JACOCO
   • Partition Pattern: Regular expression used to extract the partition from the path of the artifacts. The first non-null group is used as partition, e.g. /coverage/([^/]+)/ would use the sub-folders of coverage as the partition names.
4. Save the project.

Configuration Example ​

Below is the structure of the Artifactory containing the external results reports.

• lcov-reports.zip contains the LCOV coverage report (lcov.info).
• findings-reports.zip contains the external findings report for findings (warnings.json).

- MyRepository
|
| -> MyProjectName
  |
  | -> uploads
    |
    | -> master
      |
      | -> 1578650400000-4e8afe52737255ba40875a1af1fced1e08fc5316
        |
        | -> coverage
          |
          | -> LCOV
            |
            | -> lcov-reports.zip
        |
        | -> findings
          |
          | -> GENERIC_FINDINGS
            |
            | -> findings-reports.zip
      |
      | -> 1578651300000-70eb476d01abd0d877d064f57a6da5e14c677687
        |
        | -> coverage
          |
          | -> LCOV
            |
            | -> lcov-reports.zip

- MyRepository
|
| -> MyProjectName
  |
  | -> uploads
    |
    | -> master
      |
      | -> 1578650400000-4e8afe52737255ba40875a1af1fced1e08fc5316
        |
        | -> coverage
          |
          | -> LCOV
            |
            | -> lcov-reports.zip
        |
        | -> findings
          |
          | -> GENERIC_FINDINGS
            |
            | -> findings-reports.zip
      |
      | -> 1578651300000-70eb476d01abd0d877d064f57a6da5e14c677687
        |
        | -> coverage
          |
          | -> LCOV
            |
            | -> lcov-reports.zip

Accordingly, the Artifactory connector's configuration would look as follows:

And this would be the end result and how the uploaded results via Artifactory would appear in Teamscale:

Storing analysis results in an external storage backend ​

By default, uploads of external analysis results over the REST API interface offered by Teamscale will be stored in Teamscale's internal storage. However, Teamscale also provides the option to use an external storage backend. Some advantages of using an external storage backend are:

• Data can be shared over multiple projects without duplicating it, which is especially useful for shadow instances and project copies.
• Since external storage backends such as S3 or Artifactory are typically built with additional data-safety measures, they can often offer better data persistence guarantees.
• The size of Teamscale backups is not impacted by external analysis results, as they are pulled from an external source.
• Dedicated storage services can provide much more elaborate data management than Teamscale.
• External data might need to be persisted anyway. In this case, duplication is avoided.
• Teamscale can always access the full report data, leading to better analysis results in certain rollback scenarios.

Configuring Teamscale to Store Analysis Results In S3 ​

1. Create an external account, containing the URL, user and API key. The account has to specify "multiple systems" (default), "S3", or "External Storage Backend (S3)" to show up in the following configuration.
2. Navigate to Admin -> Storage Backends. This page contains a list of all configured external storage backends (internal storage is not displayed, since it is the default).
3. Click New external storage backend to open the configuration dialog.

4. Once the external storage backend is configured, it still needs to be enabled to have any effect. There are two ways to enable an external storage backend: on a global basis, or on a project-by-project basis.

   1. By default, the Force selected default for all projects toggle on Admin -> Storage Backends is enabled. This means that the external storage backend of all projects is configured globally by the instance administrator, and a project administrator cannot overwrite this for individual projects. When no specific external storage backend is selected, the default fallback is Teamscale's internal storage. This is the case for the initial configuration state. As a result, all projects will store external upload data in Teamscale's internal storage. To change the default, select a storage backend from the list as the new default. See below for a screenshot of the default selection button. The currently selected default is highlighted in green and shows a checkmark icon. This will cause all new external analysis results on all projects to be stored in the selected storage backend instead of in Teamscale's internal storage.

   2. Sometimes, you may want to migrate projects on a server one-by-one, or even use multiple external storage backends at the same time. To do this, disable the Force selected default for all projects toggle. Project administrators can then select an external storage backend for each project. To change the external storage backend of a project, navigate to the Edit Project page and select the storage backend from the dropdown. Note that even when the Force selected default for all projects toggle is deactivated, the instance administrator can still select a global default external storage backend. Project administrators are then free to either configure a specific, custom storage backend for their projects, or use the global default instead.

Here's a screenshot of the global default button.

Configuring Uploads ​

If you are already uploading external analysis results in Teamscale via any of the described upload methods, no changes are required on this end. Once the external storage backend is configured in Teamscale, it will be recognized by Teamscale and uploads will be redirected automatically.

Disabling External Storage Backends ​

Similar to enabling an external storage backend, disabling can take two forms:

1. If the instance has a globally enforce external storage backend (Force selected default for all projects is enabled), simply deselect the current default.
2. Otherwise, i.e., if Force selected default for all projects is disabled, one option would be to enable it and ensure no default is selected, which forces all projects to use Teamscale's internal storage again. Alternatively, if only one project should go back to internal storage, select Internal Storage on the project edit page.

Lastly, if new analysis results were indeed uploaded while the external storage backend was enabled, Teamscale has generated a new External Storage Backend connector in the project. This connector will continue reading data from the external storage backend, hence keeping all existing external analysis results intact. If you want to remove all existing external data imported through this storage backend from a project, delete this connector as well. It will not be generated again as long as internal storage is used by the project.

Migrating External Analysis Results Between Storage Backends ​

Automated migration between different external storage backends is not supported. However, data can be moved between external storage backends manually, as long as the directory structure stays intact.