Teamscale Java Profiler

The Teamscale Java Profiler records coverage in a JVM process and uploads it to Teamscale at regular intervals. This simplifies profiling of tests in long-running environments, such as deployments running in an application server, Kubernetes cluster, as well as manual tests. In addition, it offers the following advantages:

• configuration managed in Teamscale
• ability to record testwise coverage
• directly uploads coverage to Teamscale
• centralized monitoring of running profilers in Teamscale

This page serves to give a broad overview of the Teamscale Java Profiler and its options. If you want to set up Test Gap Analysis (TGA) or Test Impact Analysis (TIA), please have a look at our tutorials (TGA/TIA) and how-to pages pages.

---

• Prerequisites
  • Memory
  • Commit Info
• Download & Installation
  • Installation
• Options
  • Configuration Options
  • Advanced Options
  • Path format
  • Teamscale-Specific Proxy Options
  • File Storage Upload
• Testwise Coverage Recording
• Docker & Kubernetes
• Troubleshooting

Prerequisites

Memory

The profiler needs a small amount of additional memory to store coverage data. If your JVM is configured to only use very little memory, you might have to increase this limit in order for the profiler to function properly. How much memory is needed depends on how much code is being analyzed and thus on how you configure the profiler.

Commit Info

Teamscale always associates coverage reports to a specific code commit as otherwise the reported covered/uncovered lines don't make sense. So, if you want to set up an automated coverage upload to Teamscale, it is necessary to provide the currently checked out git revision to the profiler.

To upload the coverage to the correct commit in Teamscale, add a git.properties file to your jar/war. Depending on your environment there are multiple options:

Just add the following plugin to your build:

<build>
    <plugins>
		<plugin>
			<groupId>io.github.git-commit-id</groupId>
			<artifactId>git-commit-id-maven-plugin</artifactId>
			<version>9.0.1</version>
			<executions>
				<execution>
					<id>get-the-git-infos</id>
					<goals>
						<goal>revision</goal>
					</goals>
					<phase>initialize</phase>
				</execution>
			</executions>
			<configuration>
				<generateGitPropertiesFile>true</generateGitPropertiesFile>
			</configuration>
		</plugin>
    </plugins>
</build>

You can also upload coverage to multiple Teamscale projects.

Download & Installation

You can download the latest version of the Teamscale Java Profiler directly from Github or use our Docker Distribution.

Installation

This section describes the basic steps to install the profiler successfully in most environments.

Installation - Special Cases

For additional setup steps (e.g. in case you are using an application server) please have a look at our advanced setup page.

Via Installer recommended

Unzip the downloaded zip file into any folder. All files contained in the zip file are required for the profiler to work.

Run the following command:

./installer install https://your.teamscale.url your-build-user-name your-access-token

This will install the profiler to /opt/teamscale-profiler/java and will attach it to all Java applications.

Then for each application where you currently use the Teamscale Java Profiler:

1. Create a profiler configuration within Teamscale
2. Set the TEAMSCALE_JAVA_PROFILER_CONFIG_ID=my-configuration-id environment variable for the Java process that runs your system under test (e.g., the application server or systemd service).
3. Restart the application and verify that the profiler is shown in Teamscale under Running Profilers
4. Perform some actions in the system under test and shut it down
5. Verify that new coverage was uploaded to Teamscale, which can be checked in the External Uploads View

Manual installation

If using our recommended installer is not possible, e.g. because you don't have the permission to run it on the target system, you can configure the profiler on your application's JVM directly by adding the following JVM parameter:

    -javaagent:path/to/teamscale-jacoco-agent.jar=OPTIONS

Where

• path/to/teamscale-jacoco-agent.jar is the path to the Jar file of the Teamscale Java Profiler (inside the lib folder of the downloaded zip)
• OPTIONS are one or more comma-separated options for the profiler in the format key1=value1,key2=value2 and so on.

Example:

-javaagent:path/to/teamscale-jacoco-agent.jar=config-id=Config_Name,teamscale-server-url=https://localhost:8080,teamscale-user=user,teamscale-access-token=123456token

The Order of the Command-Line Arguments Matters

The -javaagent option must be specified before the -jar option!

Options

Configuration Options

The following options are available for the Teamscale Java Profiler:

Option	Description
config-id	A profiler configuration ID as defined in Teamscale. This allows to centrally manage the profiler configuration in Teamscale's UI. Alternatively you can also set the TEAMSCALE_JAVA_PROFILER_CONFIG_ID environment variable to that value.
config-file	A file which contains one or more of the previously named options as key=value entries, separated by line breaks. The file may also contain comments starting with #. Alternatively, you can set the TEAMSCALE_JAVA_PROFILER_CONFIG_FILE environment variable to that value.
includes	Include patterns for classes. Separate multiple patterns with a semicolon. This may speed up the profiled application and reduce the size of the output XML. These patterns are matched against the Java class names. E.g., to match all classes in package com.yourcompany and com.yourotherpackage and all their subpackages you can use *com.yourcompany.*;*com.yourotherpackage.*. (the initial star before each package name is a precaution in case your classes are nested inside e.g. a src folder, which might be interpreted as a part of the package name. We recommend always using this form).Make sure to include all relevant application code but no external libraries.
excludes	Exclude patterns for classes. Same syntax as the includes parameter.
mode	Which coverage collection mode to use. Can be either normal or testwise. Default is normal. When using the testwise mode, please have a look at our testwise coverage recording page.
debug	true, false or a path to which the logs should be written to. Default is false. This option turns on debug mode. The logs will be shown in Teamscale or in the file provided by the path.
interval	The interval in minutes between uploads of the current coverage to Teamscale (Default is 480, i.e. 8 hours). If set to 0 coverage is only dumped at JVM shutdown.
teamscale-server-url	The HTTP(S) URL of the Teamscale instance to which coverage should be uploaded.
teamscale-project	The project alias or ID within Teamscale to which the coverage belongs. If not specified, the teamscale.project property must be specified via the git.properties file in at least one of the profiled JARs/WARs/EARs.
teamscale-user	The username used to authenticate against Teamscale. The user account must have the "Perform External Uploads" permission on the given project.
teamscale-access-token	The access key of the user.
teamscale-partition	The partition within Teamscale to upload coverage to.
teamscale-revision	The source control revision (e.g. SVN revision or Git hash) that has been used to build the system under test. Teamscale uses this to map the coverage to the corresponding source code. For an alternative see git-properties-jar.
teamscale-repository	The repository id in your Teamscale project which Teamscale should use to look up the revision, if given. Not setting this option will lead to a lookup in all repositories in the Teamscale project.
teamscale-commit	The commit (Format: branch:timestamp) which has been used to build the system under test. Teamscale uses this to map the coverage to the corresponding source code. Thus, this must be the exact code commit from the VCS that was deployed. For an alternative see git-properties-jar.
teamscale-message	The commit message shown within Teamscale for the coverage upload. Default is "Agent coverage upload"
validate-ssl	Can be used to disable SSL validation (not recommended). Defaults to true.
http-server-port	The port at which the profiler should start an HTTP server that listens for commands. Have a look at the profilers REST API.

Advanced Options

In most cases, the above list of options is sufficient, but there are some more options to support niche use-cases:

Option	Description
out	The path to a writable directory where the generated coverage XML files will be stored (for details, please see the path format section). Defaults to the subdirectory coverage inside the profilers installation directory.
class-dir	The path under which all class files of the profiled application are stored. Normally, this is inferred by the profiler automatically. For some application, profiling performance may improve if you specify it explicitly. May be a directory or a Jar/War/Ear/... file. Separate multiple paths with a semicolon. You may also supply one or more .txt files with classpath entries separated by newlines (for details, please see the path format section).
dump-on-exit	Whether a coverage report should be written on JVM shutdown. Default is true.
duplicates	Defines how JaCoCo handles duplicate class files. This is by default set to WARN to make the initial setup of the tool as easy as possible. However, this should be set to FAIL for productive use if possible. In special cases you can also set it to IGNORE to print no warnings. See the special section on duplicates.
ignore-uncovered-classes	Whether classes without any recorded coverage should be ignored when generating the XML coverage report. Since Teamscale assumes classes not contained in the report to have no coverage at all, this can reduce report sizes for large systems. Default is false.
upload-metadata	Paths to files that should also be included in uploaded zips. Separate multiple paths with a semicolon. You can use this to include useful metadata about the deployed application with the coverage, e.g. its version number.
obfuscate-security-related-outputs	Boolean value determining if security critical information such as access keys are obfuscated when printing them to the console or into the log. Default is true.
teamscale-revision-manifest-jar	As an alternative to git-properties-jar the profiler accepts the repository revision provided in the given jar/war's META-INF/MANIFEST.MF file. The revision must be supplied as a main attribute called Revision (preferred) or as an attribute called Git_Commit, which belongs to an entry called Git.
teamscale-commit-manifest-jar	As an alternative to git-properties-jar the profiler accepts values supplied via Branch and Timestamp entries in the given jar/war's META-INF/MANIFEST.MF file. Expects a path as argument.
git-properties-jar	Path to the jar containing a git.properties file with Commit Infos. If nothing is configured, the profiler automatically searches all loaded Jar/War/Ear/... files for a git.properties file.
search-git-properties-recursively	Specifies whether to search for git.properties files recursively in folders or archive (jar, war, ear, aar) files. Default is true.
git-properties-commit-date-format	The Java data pattern git.commit.time is encoded with in git.properties. Defaults to yyyy-MM-dd'T'HH:mm:ssZ
sap-nwdi-applications	Needed when profiling in a SAP NetWeaver Development Infrastructure. It must be a semicolon separated list of applications. Each application is specified as a fully qualified classname (referred to as marker class) and a Teamscale project alias or ID separated by a colon. The marker class must be guaranteed to be executed when the application is running and is unique amongst the other deployed applications. E.g. com.company.app1.Main:app1alias;com.company.app2.Starter:ts-app2-id. The coverage is uploaded to master at the timestamp of the last modification date of the given marker class.
logging-config	Path to a logback configuration XML file. Use this to change the logging behaviour of the profiler. Some sample configurations are provided with the profiler in the logging folder, e.g. to enable debug logging or log directly to the console (for details, please see the logging config section).

Path format

All paths supplied to the profiler can be absolute or relative to the working directory. Furthermore, paths may contain glob patterns with *, ** and ?.

Teamscale-Specific Proxy Options

The profiler respects the standard JVM system properties http(s).proxyHost/Port if you set those for your application. If the profiler should use different proxy settings than your application, you can also configure a proxy server via the profiler's options. These options will be prioritized over the JVM proxy settings.

If no or not all profiler proxy options are provided the profiler will fall back to the system properties of the JVM.

The following options can be set in the profiler configuration:

Option	Description
proxy-https-host/proxy-http-host	The host name of the proxy server.
proxy-https-port/proxy-http-port	The port of the proxy server.
proxy-https-user/proxy-http-user	The username for the proxy server.
proxy-https-password/proxy-http-password	The password for the proxy user.
proxy-password-file	Path to a file that contains the password for proxy server authentication. This file may only contain the password and nothing else.

If you don't want to set these settings inside the configuration of the profiler, you can also set special proxy JVM properties when starting your profiled application. These properties are identical to the JVM proxy flags with the addition of the teamscale. prefix. For example -Dteamscale.https.proxyHost. The result is the same as using the proxy-options in the configuration.

File Storage Upload

In case the Java Profiler is unable to directly upload coverage to Teamscale, you can also upload it to your file storage solution.

In order to upload your coverage to Artifactory, the following options are available:

Option	Description
artifactory-url	the HTTP(S) url of the artifactory server to upload the reports to. The URL may include a subpath on the artifactory server, e.g. https://artifactory.acme.com/my-repo/my/subpath.
artifactory-partition	The partition name that is parsed later on via Teamscale Artifactory connector options.
artifactory-user	The name of an artifactory user with write access.
artifactory-password	The password of the artifactory user.
artifactory-api-key	(alternative to artifactory-user and artifactory-password) The API key for artifactory from a user with write access.
artifactory-path-suffix	The path within the storage location between the default path and the uploaded artifact.

The standard upload schema

The artifactory uploads will be organized in a very specific schema, so Teamscale can later find them and process the reports.

/ arbitraryly / many / directories / before / the / upload / root
    / 'uploads'
    / <branch name> 
    / <commit timestamp (Java long)>[-<commit hash>]?
    / <upload partition> 
    / <artifact type> 
    / none / or / some / intermediate / directories /
    / <Zip archive file>

The following options must be set in the Teamscale Artifactory Connector to import reports in Artifactory:

• Path Search Pattern: uploads/** or **/uploads/** when directories before the upload root are present
• Branch Extraction Pattern: uploads\/([^/]+)\/\d+(?:-[abcdef0-9]+)?\/[^/]+\/[^/]+\/.*
• Timestamp Extraction Pattern: uploads\/[^/]+\/(\d+)(?:-[abcdef0-9]+)?\/[^/]+\/[^/]+\/.*
• Timestamp Interpretation: timestamp:millis
• Prefix Extraction Pattern: uploads\/[^/]+\/\d+(?:-[abcdef0-9]+)?\/([^/]+\/[^/]+\/.*)
• Partition Pattern (expert option): ([^/]+)\/[^/]+\/.*
• Analysis report mapping (expert option): **/jacoco/**->JACOCO

Testwise Coverage Recording

You can use the Teamscale Java Profiler to record testwise coverage. For more information on that, please have a look at our dedicated subpage.

Docker & Kubernetes

Have a look at our how-to guide if you want to set up the profiler for Docker or Kubernetes

Troubleshooting

In case you encounter any problems, please have a look at our Troubleshooting page.