# Custom Check Framework

Custom checks can be used to easily define rules checked by Teamscale. The Teamscale distribution already comes with about 100 build-in checks for different languages, which are build upon the same framework.

The »Teamscale Custom Check API« allows users to extend Teamscale by writing custom analyses that create findings. These custom checks are executed within Teamscale's incremental analysis engine and, thus, provide real-time feedback to developers on every commit. Consequently, Teamscale treats the findings created by custom checks equal to all other findings, such that all finding-related features, like tolerating, flagging as false positive, visualizations, filtering etc., can be used just as usual.

A custom check is a local analysis in Teamscale, i.e. its implementation is given a single source file as an input and the check can create findings for code locations within this file. A check can use the source code of the file as plain text, the sequence of tokens or the abstract syntax tree (AST) of the file or a combination of these for its analysis. Each Teamscale distribution already contains many built-in checks, e.g., for detecting bad practices or uncovering bugs.

# Implementing a Custom Check

# Setting up the Development Environment

It is recommended to use the Eclipse development environment to implement Custom Checks. To set up a development workspace, please perform the following steps:

  1. Download an up-to-date version of the Eclipse IDE and unzip it.

  2. Launch Eclipse and create a new workspace.

  3. Download the Custom Check Example project from the following URL, create an Eclipse project as described there and import it into your Eclipse workspace.
    https://github.com/cqse/teamscale-custom-check-sample

# Writing a custom check

To implement a custom check, create a new Java class deriving from CheckImplementationBase that is annotated with @Check . The class CheckImplementationBase contains the analysis context (ICheckContext) which you can use to get information about the analyzed source code file. This includes the textual representation of the code, the stream of tokens in the file (the output of the scanner/lexer), as well as the abstract syntax tree (AST, output of the parser). The only method your Custom Check needs to implement (overwrite) is named execute(). The baseclass offers you helper methods to select certain elements in the AST using XPath expressions and to create findings easily. However, in many cases your Custom Check class does not need to use this rather general base class but a more specific subclass of CheckImplementationBase, e.g., one of the following:

  • EntityCheckBase for checks that select a set of entities via XPath and inspect each of these.

  • EntityFindingCheckBase for checks that select a set of entities via XPath and create a finding for each selected entity.

  • EntityTokenCheckBase for checks that select entities via XPath and process the tokens of each selected entity.

In the @Check annotation you must specify meta-data about your check. This meta-data is used by Teamscale mainly for configuration of the analyses (quality profile) and the presentation of the findings your check will produce. The meta-data includes the following information:

  • name: Name of the check

  • description: A description of the check that explains developers why the findings of the check are a problem and how they should fix them.

  • groupName: The analysis group the check should be contained in (e.g. General Checks)

  • languages: The programming language(s) for which the check can be used.

Furthermore, a Custom Check must specify on which representation of the code it performs the analyses (using the parameters-argument of Check). E.g., the check may only access the AST if the option ECheckParameter.ABSTRACT_SYNTAX_TREE is specified or type information will only be present if type resolution is explicitly switched on by using ECheckParameter.TYPE_RESOLUTION. Teamscale needs this information to cluster the checks, so that representations that are not needed are not calculated at all.

# Sample Custom Check

The following code listing shows a simple Custom Check:

@Check(name = "Abstract types should not have constructors (CA1012)", 
    description = "Constructors on abstract types can be called only by derived types. Because public constructors create instances of a type, and you cannot create instances of an abstract type, an abstract type that has a public constructor is incorrectly designed.", 
    groupName = "Language misuse", 
    languages = { ELanguage.CS }, 
    parameters = { ECheckParameter.ABSTRACT_SYNTAX_TREE })

public class AbstractTypesShouldNotHaveConstructorsCheck extends EntityFindingCheckBase {

  /** 
   * Returns an XPath selection string that
   * selects public abstract types with public
   * constructors.
   */
  @Override
  protected String getXPathSelectionString() {
    return "//TYPE[modifiers('public', 'abstract')]/METHOD[subtype('constructor') and modifiers('public')]";
  }

  /** 
   * Returns the message for the created finding.
   */
  @Override
  protected String getFindingMessage(ShallowEntity entity) {
    return "An abstract type must not have a public constructor.";
  }
}

# Structure of the AST and selection via XPath

The AST consists of so called shallow entities. These entities are the nodes of the AST (e.g., types or methods). They are called shallow because the AST is only shallow in the sense that it is constructed down to the statement level but does not have detailed information on expressions within statements.

The enum EShallowEntityType documents the list of possible shallow entity types. These are also used by the XPath notation to make a selection on AST nodes. For example, the XPath expression //METHOD would select all shallow entities that denote a method. Shallow entities also have a subtype, which further specify the entity. For instance a method entity can have the subtype 'constructor' denoting the special case of a constructor. Subtypes may be specific for a programming language. For a list of possible values, refer to the class SubTypeNames. Selection functions can be used to further filter shallow entities. As an example, the XPath expression //METHOD[modifiers('public')] selects all public methods. For a list of possible selection functions, please refer to the package eu.cqse.check.framework.core.xpath.functions.

# Testing Your Custom Check

The Teamscale Custom Check Framework provides a Unit-Testing framework you should use to write tests for your checks. The easiest way to test your check is using data-driven tests. This way you do not have to write any testcode but only provide test data in terms of code listings that contain the findings your check should identify and a file describing the expected results. To define a data-driven test you need to perform the following steps:

  1. Create a new subfolder within the test-data folder using the class name of your custom check class as the folder name (e.g., MyCheck)

  2. Place a test code file (e.g., MyTestClass.java) to test your check against in the newly created folder.

  3. Create a second file in the same folder and use the same name followed by .expected, e.g., MyTestClass.java.expected. This file will be used to compare the findings detected by your check with the expected findings. For every finding your check should emit, put a line into this file using the following format: Offsets <fromOffset>-<toOffset> (lines <fromLine>-<toLine>): <findingMessage> The offsets here are character offsets in the source code. Please specify the following information corresponding to your test-data file: fromOffset/toOffset: Offsets to the tokens the finding should be annotated. fromLine/toLine: The line number that should be marked with a finding of your check. findingMessage: The message that should be displayed for your finding.

  4. To be able to execute the tests, you need to create a single test class deriving from CheckTestBase that will automatically create a test suite from the test data (this class is already available in the test-src folder of Custom Check Example project):

    /** Main class for testing checks */
    @RunWith(Parameterized.class)
    public class CheckTest extends CheckTestBase {
    
      /** Constructor. */
      public CheckTest(File referenceFile, Map<String, CheckInfo> checkInfoBySimpleClassName) {
        super(referenceFile, checkInfoBySimpleClassName);
      }
    
      /** Generate Test Parameters. */
      @Parameters(name = "{0}")
      public static Collection<Object[]> generateParameters() throws IOException {
        return CheckTestBase.generateParameters(new CheckTest(null, null));
      }
    }
    
  5. To run the tests, launch the test class as a JUnit-Test in Eclipse and inspect the results (information about the actual results of your check are written to the directory test-tmp).

# Deploying your Custom Check

  1. Package your custom check into a JAR file using the JAR Export Wizard of Eclipse. You do not need to re-package any of the included libraries. The check classes are sufficient.
  2. Place the JAR in the custom-checks subfolder of your Teamscale installation. The location of this folder can be customized in teamscale.properties.
  3. Restart Teamscale. The checks become available when configuring analysis profiles.

# Custom Check API Evolution

The custom check API and thus any custom check binaries are compatible between patch releases (e.g., 5.6.0 and 5.6.1). However, when updating to a new feature release (e.g., 5.6.x to 5.7.x) a rebuild of your custom checks against the latest customer check API is recommended.