# How to Upload External Analysis Results to Teamscale
It is possible to upload external analysis results to Teamscale, either via the REST API or via repository connectors. External analysis results include test coverage reports, test execution results, custom findings, custom metrics and more.
# Upload via Web UI
For testing and debugging purposes it can sometimes be helpful to manually upload reports via the Web UI.
# Upload via Command Line
With our command line utility teamscale-upload
(opens new window) for Windows and Linux you can easily upload external analysis reports to Teamscale.
If you want to upload a report file from your CI pipeline, you can run:
teamscale-upload --server http://<TEAMSCALE_URL> --project <PROJECT> --user <USER> --accesskey <ACCESSKEY> --partition <PARTITION> --format <FORMAT> <PATH/TO/REPORT/FILE>
This command will automatically detect the version control commit to which the report belongs by querying your CI system (if it is supported, e.g. Jenkins, Azure DevOps, ...) or any Git or SVN checkout in the current working directory.
GitHub Action
If you are using GitHub as your CI system we recommend using teamscale-upload via GitHub Action which is even easier to integrate into your CI workflow.
Necessary Permissions in Teamscale
The user you supply on the command line must have the Perform External Uploads permission for the project to which you are uploading the reports. For a production setup we recommend that you create a technical user in Teamscale and assign it the Build role for that project.
Self-Signed Certificates
If you're accessing Teamscale via HTTPS and are signing the certificates yourself or via an internal CA, you can simplify the initial setup of this tool by using --insecure
.
This will disable certificate validation.
Once you're successfully uploading your analysis results, you can re-enable validation and provide the necessary certificates as a Java key store via --trusted-keystore
.
Valid Formats
Choosing the correct format is crucial. Otherwise, Teamscale cannot interpret the report file and will log errors to the Worker Log. It must be one of the formats supported by Teamscale. Casing does not matter.
Check the Exit Code and Command Line Output
teamscale-upload
will exit with code 0 if the upload was fully successful.
Any other exit code indicates a problem that you must address.
The problem is logged to the console.
Further Options
Run teamscale-upload --help
to see all available options.
# Uploading Multiple Report Files of the Same Format
If you need to upload more than one file of the same format, you can either specify more than one path on the command line or use wildcards:
*
matches any number of characters in the current directory or file name**
matches any number of nested directories?
matches any single character
Always Use Single Quotes Around Wildcard Strings
Since *
and ?
are characters that are also interpreted by your shell, you must always use single quotes around paths with wildcards.
E.g. 'path/**/report*.xml'
matches all of the following:
path/to/nested/report.xml
path/report.xml
path/report123.xml
# Uploading Multiple Files with Different Formats
If you need to upload more than one type of report, e.g. test coverage results and findings, you can use an input file. In this file, you can define multiple formats and one or more patterns per format, e.g:
[jacoco]
jacoco1/**.xml
jacoco2/**.xml
[findbugs]
pattern1/**.findbugs.xml
pattern2/**.findbugs.xml
Use this file with the --input
command line option:
teamscale-upload --server http://<TEAMSCALE_URL> --project <PROJECT> --user <USER> --accesskey <ACCESSKEY> --partition <PARTITION> --input <PATH/TO/INPUT/FILE>
# Specifying the Teamscale access key
teamscale-upload
offers two ways to securely specify the Teamscale access key:
- Via specifying the environment variable
$TEAMSCALE_ACCESS_KEY
- Example:
$ export TEAMSCALE_ACCESS_KEY=<key>
$ teamscale-upload --server http://<TEAMSCALE_URL> --project <PROJECT> --user <USER> --partition <PARTITION> --format <FORMAT> <PATH/TO/REPORT/FILE>
- Via the standard input by specifying
accesskey -
and then entering the key via stdin.- Example:
$ cat ~/accesskey.txt | teamscale-upload --server http://<TEAMSCALE_URL> --project <PROJECT> --user <USER> --accesskey - --partition <PARTITION> --format <FORMAT> <PATH/TO/REPORT/FILE>
# Additional way to specify the Teamscale access key (not recommended)
There is one additional unrecommended way to specify the Teamscale access key: Entering it directly via the command-line option --accesskey <key>
.
As it leaks the provided key to the standard output in an uncensored fashion, it should only be used when the above options are not applicable.
$ teamscale-upload --server http://<TEAMSCALE_URL> --project <PROJECT> --user <USER> --accesskey <key> --partition <PARTITION> --format <FORMAT> <PATH/TO/REPORT/FILE>
# Upload via GitHub Action
If you are using GitHub as your CI system we recommend using teamscale-upload via GitHub Action which is available on the GitHub Marketplace (opens new window). It provides the same options as teamscale-upload and can be used as follows:
uses: 'cqse/teamscale-upload-action@v2.4.0'
with:
server: 'https://demo.teamscale.com'
project: 'teamscale-upload'
user: 'build'
partition: 'Github Action > Linux Branch And Timestamp'
accesskey: ${{ secrets.ACCESS_KEY }}
format: 'SIMPLE'
branch-and-timestamp: 'master:HEAD'
message: 'This is a test message.'
files: 'test_resources/coverage.simple test_resources/coverage2.simple'
A more detailed description of the parameters can be found here.
# Upload via REST API
If you cannot use the teamscale-upload
utility, you can also manually upload files via our REST API.
We do recommend that you use teamscale-upload
if possible since it includes helpful handling for many common errors, typos, etc.
The REST service external-analysis/session/<SESSION_ID>/report
allows uploading results from analysis tools like FindBugs/SpotBugs (opens new window) or different coverage tools.
# Prerequisites
In order to upload a report to Teamscale, the server which is running the Teamscale instance must be accessible via HTTP(S).
As a first step, create a new user which will be able to upload external
analysis results to Teamscale. You can either use your administration
account for this, or better use a special user with the Build
role.
# Uploading a single report
To upload a single external analysis report to Teamscale for the current HEAD commit (see below for uploads for historic dates), issue the following HTTP request:
HTTP method:
POST
Request URL:
http://<YOUR_TEAMSCALE_SERVER>:<PORT>/api/projects/<TEAMSCALE_PROJECT_ID>/external-analysis/session/auto-create/report?format=<FORMAT>&partition=<PARTITION>&message=<MESSAGE>
YOUR_TEAMSCALE_SERVER:PORT
is the address and port of the Teamscale instance.TEAMSCALE_PROJECT_ID
is the ID of the Teamscale project into which to store the coverage.FORMAT
is the report format of the uploaded file. It must be one of the formats supported by Teamscale.PARTITION
is a logical name for the type of coverage. All coverage with the same partition will be grouped together. This, e.g., allows you to upload both coverage for your Java and your JavaScript tests without the results of one overriding those of the other. The name is descriptive and can be any arbitrary string.MESSAGE
is the message shown in the Activity perspective to the user. It should make clear that coverage was uploaded.
Body: Form data. May contain any number of elements with name "report" and the content of a single result XML file.
Authentication: HTTP Basic Authentication with the build or admin user and their Access Token (not their password!)
# Sample Upload with Curl
curl --request POST --user bob:e434mikm35d --form "report=@coverage.xml" "http://teamscale-server:8080/api/projects/myProject/external-analysis/session/auto-create/report?format=JACOCO&partition=Unit%20Tests&message=Unit%20Test%20Coverage"
Always Quote the URL
Please note that the URL must be enclosed in quotes.
This is required as most shells treat the &
character specially and would thus truncate the URL.
If you are not sure you quoted the URL correctly, you can use curl -v
to get verbose output from curl, including the actual URL it tries to contact.
File encoding
If not explicitly specified, Teamscale will interpret the encoding of the provided file based on it's BOM, or default to UTF-8. Change the charset
of the Content-Type
header when uploading a Teamscale report via curl
to prevent parsing errors because of misinterpreted encodings.
An example how to upload a report with a specified UTF-16 encoding:
curl --request POST --user bob:e434mikm35d --form "report=@coverage.xml;type=text/xml;charset=UTF-16" "http://teamscale-server:8080/api/projects/myProject/external-analysis/session/auto-create/report?format=JACOCO&partition=Unit%20Tests&message=Unit%20Test%20Coverage"
Note the type definition type=text/xml;charset=UTF-16
after the file name.
Request Parameters
Details about the service's parameters can be best retrieved via the online service documentation as described here.
In addition to the common pitfalls when dealing with the REST API, you need to check if the external tool is activated in the project's analysis profile.
Furthermore, please refer to the details for supported upload formats.
# Uploading data for historic dates or revisions
Note that the uploaded results will be used for the current HEAD commit in Teamscale on the main branch (e.g., latest commit on master for Git, latest changeset on the TFS main branch or latest commit on trunk for SVN). Should this not be the behaviour you want, you can override this by adding the following to the request's URL:
&revision=<REVISION>
where REVISION is, depending on your version control system, a:
- full Git SHA hash, or
- SVN revision number, or
- TFS changeset ID
Regardless of whether the revision has already been analyzed or not, Teamscale will store the external analysis results for the code with the revision. When later it is analyzed, the upload is integrated with the revision into Teamscale.
If concerned project has a multi-connector setup, the VCS repository that contains the revision for which an upload is meant, can be specified in the request's URL using the repository identifier (see connector options) for the project connector:
&repository=<REPOSITORY_IDENTIFIER>
Teamscale also supports uploading external analysis results to timestamps. This can be achieved by adding the following to the request's URL:
&t=<TIMESTAMP>
If you additionally need to specify the branch for the upload, you can use the t
parameter as well:
&t=<BRANCH>:<TIMESTAMP>
# Uploading Multiple Reports At Once (Sessions)
If you want to upload multiple reports in a single logical commit, you should use Teamscale's session concept. For session handling, the service external-analysis/session is used. The following example uploads two different reports as a single commit using curl. For more details, see the service documentation.
curl --request POST --user bob:e434mikm35d http://localhost:8080/api/projects/my-project-id/external-analysis/session?message=myMessage&partition=myPartition
This command will return the session ID, in this case Sdfb38cb0-c348-40fb-82f0-144bf0e41beb
.
Using this session, we can upload multiple reports.
curl --request POST --user bob:e434mikm35d --form "report=@./MyReport.xml" http://localhost:8080/api/projects/myProject/external-analysis/session/Sdfb38cb0-c348-40fb-82f0-144bf0e41beb/report?format=FINDBUGS
curl --request POST --user bob:e434mikm35d --form "report=@./OtherReport.xml" http://localhost:8080/api/projects/myProject/external-analysis/session/Sdfb38cb0-c348-40fb-82f0-144bf0e41beb/report?format=FXCOP
Finally, we commit the session using a POST request:
curl --request POST --user bob:e434mikm35d http://localhost:8080/api/projects/my-project-id/external-analysis/session/Sdfb38cb0-c348-40fb-82f0-144bf0e41beb
# Upload via Gradle Plugin
With our Teamscale Gradle Plugin (opens new window) you can easily upload artifacts directly from Gradle without any additional tooling involved. It can automate uploading JUnit, JaCoCo and Testwise Coverage reports. This guide expects that you have already configured your test tasks and the jacoco plugin (opens new window).
Test Impact Analysis
For more in depth instructions on how to set up Testwise Coverage collection and Impacted Test execution please refer to our tutorial.
You can apply the plugin by adding it to your plugins block of the top level project.
plugins {
id "com.teamscale" version "23.1.1"
}
The teamscale
extension allows you to configure where reports should be uploaded to:
teamscale {
server {
url = 'https://teamscale.mycompany.com:8080'
project = 'test'
userName = 'build'
userAccessToken = project.property('teamscale.access-token')
}
}
The example assumes that you pass the Teamscale Access Token to Gradle via
gradle -Pteamscale.access-token=82l1jtkIx6xG7DDG34FLsKhejcHz1cMu ...`
By default the plugin will try to detect the correct commit it needs to upload to from the Git repository. In case the Git repository is not present when Gradle is executed, which is the case in some CI setups, you need to manually specify the commit hash:
teamscale {
commit {
revision = System.getenv('CI_COMMIT_SHA')
}
}
You can specify which artifacts you want to upload and to which partition via the reports
block:
teamscale {
report {
jacoco()
junit()
testwiseCoverage()
}
}
These calls will configure all existing JacocoReport
, Test
and TestImpacted
tasks respectively to produce the necessary reports and pick them up after the tasks were executed.
All aspects of the upload can be configured individually:
teamscale {
report {
jacoco {
// The partition all JaCoCo reports should be uploaded to
// This is the only required setting
partition = "Unit Tests"
}
junit {
partition = "Automated Tests"
// This is the message that will be shown in Teamscale for the artificial coverage commit (optional)
message = "JUNIT gradle upload" // This is the default value
// Whether reports of this type should be uploaded (Optional, default is true)
upload = true
}
}
}
tasks.named("test") {
// ...
teamscale {
report {
// partition, message and upload can also be overwritten for individual tasks
upload = false
}
}
}
The actual upload is performed with the teamscaleReportUpload
task of the top level project.
It is not executed automatically, so you need to explicitly call it either as part of your CI Gradle command
gradle --continue test jacocoTestReport teamscaleReportUpload
or model the desired behavior in Gradle e.g.
tasks.named("jacocoTestReport") {
// Report should always be uploaded after it was generated
finalizedBy teamscaleReportUpload
}
tasks.named("teamscaleReportUpload") {
// Report generation is required to run before generating the report
dependsOn jacocoTestReport
}
# Import via Artifactory
External analysis results can be directly imported to Teamscale via the Artifactory repository connector. The Artifactory connector will extract the results from the repository, process the reports and store them in the configured partition.
# Prerequisites
Before importing external analysis results to Teamscale via Artifactory, please ensure the following:
- All relevant external results reports are available in Artifactory.
- The external results reports are packed in archives (i.e. ZIP files).
- The path of the external results reports in Artifactory includes the timestamps at which they should be uploaded.
# Configuration Steps
To import external results reports to a project:
- Add an Artifactory connector to the project.
- Configure the Artifactory connector options so that Teamscale can extract the reports from Artifactory.
- Configure the following options so that Teamscale can process the reports:
Analysis report mapping
: Ant pattern used to map the reports inside the ZIP archives to their corresponding report format. Example:**/junit.xml -> JUNIT, **/jacoco.xml -> JACOCO
Partition Pattern
: Regular expression used to extract the partition from the path of the artifacts. The first non-null group is used as partition, e.g./coverage/([^/]+)/
would use the sub-folders of coverage as the partition names.
- Save the project.
# Configuration Example
Below is the structure of the Artifactory containing the external results reports.
lcov.zip
contains the LCOV coverage report (lcov.info
).misra.zip
contains the external findings report for MISRA warnings (warnings.json
).
- MyRepository
|
| -> ExternalReports
|
| -> Branches
|
| -> master
|
| -> coverage
|
| -> 20200110-110037
|
| -> lcov.zip
|
| -> 20200110-111535
|
| -> lcov.zip
|
| -> findings
|
| -> 20200110-110037
|
| -> misra.zip
Accordingly, the Artifactory connector's configuration would look as follows:
And this would be the end result and how the uploaded results via Artifactory would appear in Teamscale: