Teamscale Documentation

Appearance

Setting Up Test Coverage Profiling for Java ​

In order to use the Test Gap analysis, you need to record test coverage and upload it to Teamscale. To record test coverage for Java (or other JVM languages, such as Kotlin), we recommend you use either JaCoCo or the Teamscale JaCoCo Agent. Our guidelines for this decision are:

  • If you want to profile your unit tests or other automated tests that are run in your CI environment, use JaCoCo.
  • If you want to profile manual tests, use the Teamscale JaCoCo Agent.
  • If you want to profile a long-running process and want to upload test coverage to Teamscale in regular intervals (e.g., when profiling a web application), use the Teamscale JaCoCo Agent.

TIP

In our general Java TGA tutorial we outline the steps needed to set up TGA for a concrete sample project.

Using JaCoCo ​

In order to set up JaCoCo for automated tests running in your CI environment, install and configure one of its build system plugins:

These plugins will write XML files containing the coverage information to disk after your tests have finished. You can upload these files to Teamscale as described in the instructions for uploading external analysis results.

Using the Teamscale JaCoCo Agent ​

The Teamscale JaCoCo Agent is a JVM coverage profiler that builds on JaCoCo. It records coverage from a JVM process and uploads it to Teamscale at regular intervals. This simplifies profiling of tests in long running environments, such as deployments running in an application server.

The following instructions are for setting up the agent on a server or local machine. If your application is instead running in Docker, please read our Docker guide instead.

Including build information (git.properties) ​

To upload the coverage to the correct commit in Teamscale, add a git.properties file to your jar/war.

Just add the following plugin to your build:

xml
<build>
    <plugins>
        <plugin>
            <groupId>pl.project13.maven</groupId>
            <artifactId>git-commit-id-plugin</artifactId>
            <version>4.0.0</version>
        </plugin>
    </plugins>
</build>
<build>
    <plugins>
        <plugin>
            <groupId>pl.project13.maven</groupId>
            <artifactId>git-commit-id-plugin</artifactId>
            <version>4.0.0</version>
        </plugin>
    </plugins>
</build>

Now build and deploy your application.

Attaching the Teamscale JaCoCo Agent to Your System Under Test ​

Download the latest release of the Teamscale JaCoCo Agent and extract it into some directory next to your deployed application. We will refer to this directory as TEAMSCALE_JACOCO_AGENT_DIR in the following.

Usage with Docker

There is also a docker image of the Teamscale JaCoCo Agent. For instructions on how to set up test-coverage profiling for a dockerized setup, have a look at these instructions.

As a next step, add the Teamscale JaCoCo Agent as an agent to your system under test, by adding the following JVM argument to the configuration of your deployment.

shell
-javaagent:TEAMSCALE_JACOCO_AGENT_DIR/lib/teamscale-jacoco-agent.jar
-javaagent:TEAMSCALE_JACOCO_AGENT_DIR/lib/teamscale-jacoco-agent.jar

Make sure to replace TEAMSCALE_JACOCO_AGENT_DIR with the actual agent file path.

DANGER

The -javaagent option must be specified before the -jar option!

If you are using an application server like WebSphere, JBoss, Wildfly etc. please have look at the specific instructions in the agent's documentation.

Validating Coverage Recording ​

Now start your system under test, as you normally do, and perform some actions in the application, such that some part of the functionality is executed. Then shut down the application. The directory TEAMSCALE_JACOCO_AGENT_DIR/coverage should contain an XML file with coverage data.

To verify that the results are as expected you can manually upload the JaCoCo report in Teamscale. Afterwards, open the Test Gaps perspective and select the project and branch that you uploaded the report to. Also make sure the new coverage partition is selected in the coverage partition chooser. Next, click on the Execution cell of the Summary row. The execution summary A dialog opens and should show some green rectangles, which represent the covered methods.

TIP

In case you do not see test coverage in the treemap, e.g., no green rectangles, please check whether the agent's log files contain any relevant warnings or errors. Please also check the points mentioned in the "Test coverage does not show up in Teamscale" Troubleshooting section. The Teamscale JaCoCo Verifier might help you to find out whether the XML file actually contains relevant coverage. In case it does not, you might want to turn on debug logging in the Teamscale JaCoCo Agent and have a look at the log files. Please refer to the agent's documentation for more details.

Generating a Configuration File ​

The profiler is configured with a config file, which can be generated from within Teamscale. To configure the agent for your Java or Kotlin application, go to the Projects perspective and click on the button. To configure the agent for any other JVM language you need to manually prepare a properties file in the lines of the example shown below. On the next screen, select Set Up Coverage Profiler. You will be presented with a dialog that lets you configure the agent:

The JaCoCo Agent Configuration Wizard

  • Please select the user whose credentials should be used for the coverage upload.
  • Please select a partition, which is a logical name that groups related coverage, e.g., the type of test that will be profiled (Manual Test vs. Unit Test vs. Regression Test etc).
  • Please select all packages that should be profiled. Subpackages are included as well.

Selecting Appropriate Packages

Teamscale automatically suggests packages to profile. Please review them carefully.

Please ensure the packages you select are "future-proof", i.e. they also match packages you might add in the future. This saves you time later, because you won't have to adjust the profiler configuration whenever you add new packages.

Finally, please click Download Profiler Configuration.

The generated configuration file looks similar to the following:

properties
teamscale-server-url=https://your.teamscale.url
teamscale-project=your-project
teamscale-user=your-build-user-name
teamscale-access-token=your-access-token
teamscale-partition=Manual Tests
teamscale-server-url=https://your.teamscale.url
teamscale-project=your-project
teamscale-user=your-build-user-name
teamscale-access-token=your-access-token
teamscale-partition=Manual Tests

Uploading via Teamscale JaCoCo Agent ​

Now that we have the necessary configuration file, copy it next to your system under test and reference the file as config-file in the JVM arguments:

shell
-javaagent:TEAMSCALE_JACOCO_AGENT_DIR/lib/teamscale-jacoco-agent.jar=config-file=javaagent.properties
-javaagent:TEAMSCALE_JACOCO_AGENT_DIR/lib/teamscale-jacoco-agent.jar=config-file=javaagent.properties

Now repeat the steps from before:

  • Start the system under test
  • Perform some actions
  • Shut down the system

This time, the agent automatically uploads the coverage to Teamscale. By default, the upload happens at an interval of 8 hours and on JVM shutdown. The interval can freely be adjusted to fit your needs, by adding, an e.g., an interval=60 entry to the properties file, to set the upload interval to 60 minutes.

For a detailed reference on all the configuration options of the Teamscale JaCoCo Agent, please refer to the agent's documentation.