Recording Test Coverage for Java with Docker ​

This how-to will show you how to set up test coverage recording for a dockerized Java application or any other language running on the Java Virtual Machine (JVM), so you can use the Test Gap analysis. This setup works for both automated and manual tests. The following graphic gives a quick overview over the final setup:

Coverage is recorded using the Teamscale JaCoCo Agent, which is a Java agent that can attach to any JVM to record the lines of code executed during any test (manual or automated). The agent also handles the automatic upload of the recorded test coverage to Teamscale on shutdown of the JVM or in configurable time intervals.

Prerequisite: Generate git.properties ​

The agent needs to be able to find a git.properties file, which contains the Git SHA1 of the commit that was used to build your Docker image. You can easily generate such a file and include it in your Jar/War/Ear/... files by using the corresponding Maven or Gradle build plugin.

Adjust Your Dockerfile ​

By default, the agent sends the coverage it collected when the JVM inside your Docker container is shut down. This requires that the JVM receives the SIGTERM signal sent by your orchestration tooling. Thus, you'll need to ensure that the JVM is the main process inside the docker image. This can be achieved by using exec to replace the shell process with your JVM process:

# this will not work:
CMD java -jar /your.jar

# use this instead:
CMD exec java -jar /your.jar

# this will not work:
CMD java -jar /your.jar

# use this instead:
CMD exec java -jar /your.jar

The same applies if you're using ENTRYPOINT instead of CMD:

# this will not work:
ENTRYPOINT java -jar /your.jar

# use this instead:
ENTRYPOINT exec java -jar /your.jar

# this will not work:
ENTRYPOINT java -jar /your.jar

# use this instead:
ENTRYPOINT exec java -jar /your.jar

If your application is wrapped inside a start script, you'll likewise need to ensure that you use exec to replace the shell process with the JVM process.

docker container top determined_dijkstra

docker container top determined_dijkstra

docker inspect -f '{{.State.Pid}}' determined_dijkstra

docker inspect -f '{{.State.Pid}}' determined_dijkstra

Deploy the Agent ​

You'll need to deploy the cqse/teamscale-jacoco-agent image next to your own Docker images. This image will do nothing by itself, it simply provides the agent's Jar file to your containers so you don't have to include the agent in your images.

To make the agent available to your JVM, you'll need to mount the /agent volume of the cqse/teamscale-jacoco-agent container into every Docker container for which you'd like to record coverage. This volume contains the agent's jar file.

Configure the Agent ​

The agent expects a configuration file as its input. You'll have to include it in your Docker image. Teamscale has a wizard to generate this config file for you. Include this file in your Docker images in a well-known path, e.g. /jacocoagent.properties.

Activate the Agent ​

In order to put all parts together and activate the agent for your application, you'll need to set a JVM argument. This is most often done via the environment variable JAVA_TOOL_OPTIONS, which is automatically picked up by the JVM itself. However, if your setup overrides JAVA_TOOL_OPTIONS, e.g. in a start script, you may need to use a different environment variable that is picked up by your start scripts (e.g. JAVA_OPTS is used by many frameworks). The agent's documentation contains detailed instructions for many web application servers.

JAVA_TOOL_OPTIONS="-javaagent:/agent/teamscale-jacoco-agent.jar=config-file=/jacocoagent.properties"

JAVA_TOOL_OPTIONS="-javaagent:/agent/teamscale-jacoco-agent.jar=config-file=/jacocoagent.properties"

You'll need to set this environment variable in your Docker orchestration tooling (e.g. Docker Compose, Helm, ECS, Kubernetes).

Now you can restart your container and the agent will record coverage and upload it to Teamscale in regular intervals.

docker exec -it YOUR_CONTAINER_ID cat /logs/teamscale-jacoco-agent.log | less

docker exec -it YOUR_CONTAINER_ID cat /logs/teamscale-jacoco-agent.log | less

Kubernetes: Set an Appropriate Grace Period ​

When the pod running your application and our agent is stopped, the agent will upload all collected coverage to Teamscale. This adds some seconds to your application's shutdown time. By default, Kubernetes will kill any pod that takes longer than 30s to shut down. This will interrupt the agent and the coverage data will not be uploaded and is lost.

Thus, you must set an appropriate terminationGracePeriodSeconds value that is large enough so your application can shut down cleanly and the agent has a chance to send its coverage data to Teamscale. What "appropriate" means depends on your application. We recommend you run a few experiments to determine how long your application takes to shut down cleanly with and without our agent enabled and then add some buffer on top of the largest measured value.

Example: Docker Compose 3 ​

{
  version: '3',
  services: {
    yourapp: {
      image: your/image,
      # activate the agent by setting an appropriate environment variable
      environment: {
        JAVA_TOOL_OPTIONS: "-javaagent:/agent/teamscale-jacoco-agent.jar=config-file=/jacocoagent.properties"
      },
      # use the agent's volume
      volumes: [
        agent-jar:/agent:ro
      ]
    },
    agent: {
      image: cqse/teamscale-jacoco-agent:v23.1.1,
      # make the agent's volume available
      volumes: [
        agent-jar:/agent:ro
      ]
    }
  },
  # declares the agent's volume
  volumes: [
    agent-jar:
  ]
}

{
  version: '3',
  services: {
    yourapp: {
      image: your/image,
      # activate the agent by setting an appropriate environment variable
      environment: {
        JAVA_TOOL_OPTIONS: "-javaagent:/agent/teamscale-jacoco-agent.jar=config-file=/jacocoagent.properties"
      },
      # use the agent's volume
      volumes: [
        agent-jar:/agent:ro
      ]
    },
    agent: {
      image: cqse/teamscale-jacoco-agent:v23.1.1,
      # make the agent's volume available
      volumes: [
        agent-jar:/agent:ro
      ]
    }
  },
  # declares the agent's volume
  volumes: [
    agent-jar:
  ]
}