Analyzing Issue Test Gaps
As a developer or tester, you often want to ensure that no changes to your software system slip through your testing efforts. Teamscale's Test Gap analysis (TGA) enables you to continuously monitor this during development. It automatically identifies Test Gaps on the changes made for each issue from your issue tracker, once you connect your issue tracker to your Teamscale project. This guides explains how to access and work with this information:
You may also watch our webinar on Test Gap analysis to see parts of this guide in action.
Viewing Issue Test Gaps
Teamscale displays Issue Test Gaps as a Test Gap treemap on the Issue Details view. The simplest way to open this view for a particular issue, is to press »I« anywhere in Teamscale and enter the Issue ID in the dialog. To browse and search through all issues, go to the Test Gaps » Issues page. You will find the Test Gap treemap in a foldable section below the general issue details.
The treemap shows exactly the methods that were changed in the context of the current issue. Therefore, it never shows any gray, i.e., unchanged, parts of your software system.
Teamscale identifies the issue-related changes by searching commit messages for mentions of issue IDs. In this way, it can identify and filter for the relevant changes, regardless of whether you work with feature branches or in a trunk-based development scenario, where all work is committed to a single branch. If the issue-related changes are on a feature branch, Teamscale automatically detects and selects this branch for you, as displayed above the treemap.
As the baseline for the TGA, Teamscale automatically uses the version of your system immediately before the first change in the context of the issue. The end date of the analysis is always Now, to ensure all changes in the context of the issue are considered.
For convenience, you may quickly choose any (combination of) Coverage sources for the TGA through the selector at the upper right of the treemap. Coverage sources are Partitions, which allow you, e.g., to distinguish coverage from different test stages.
Otherwise, this treemap works like any other Test Gap treemap in Teamscale.
Reduce Risk vs. All Green
Test Gap analysis enables conscious decisions about where to direct your limited testing resources, in order to minimize the risk that comes with changes to your software system. The goal is to prevent changes from slipping through testing without you knowing, not to try to close all Test Gaps. An all-green Test Gap treemap does not give you any guarantees, since it cannot prove the absence of bugs.
Open Issue Details from Jira
If your issue tracker is Jira and you enable Jira Server Issue Updates, Teamscale adds links to the Issue Details view to your Jira tickets.
Getting an Overview
Teamscale provides an overview over Issue Test Gaps for multiple issues on the Test Gaps » Issues page. This allows you, for example, to revisit all issues related to a release or assigned to you.
At the top of this page, you may choose the Issue Metric that identifies the issues you want to look at.
The list below shows each issue with the number of methods that were added or changed in the context of that issue and the respective Issue Test Gaps aggregated in a bar chart. The percentage above the chart specifies which fraction of all the added or changed methods remains untested.
At the top-right of the list you find the Test Gap statistics aggregated over all the issues and the Coverage sources selector, to choose the (combination of) coverage sources for the TGA.
Webinar:
Webinar on Test Gap Analysis
- 0:00: Motivation and Basics
- 33:30: Practical Examples and Technical Details
- 52:44: Live Demo
- 1:10:30: Cost-Benefit Analysis