Setting Up Test Coverage Profiling for Python

In order to use the Test Gap analysis, you need to record test coverage and upload it to Teamscale.

Coverage.py

To install coverage.py run

pip install coverage

Check that your installed version has the C extension enabled

You can do this by running coverage --version and checking that the output mentions that the extension is enabled, e.g. "Coverage.py, version 6.3.2 with C extension" The extension improves performance and allows for more features to be used.

In order to collect coverage information for a test run you need to invoke your test runner framework using coverage.py. An example for executing a test suite may look like this:

coverage -m <MODULE_NAME> arg1 arg2 arg3

Where <MODULE_NAME> can be e.g. pytest or unittest. The resulting coverage information will be stored in a binary file named .coverage.

Sub-Processes are Not Automatically Instrumented

By default, only the main process is instrumented. If you use, e.g., an application server like gunicorn or your application spawns sub-processes in other ways, you'll need to take additional steps to instrument those.

You can combine different test runs

If you need to run different test suites for the same project, you can combine these runs to get one single .coverage file.

In order to have a file in a format that is readable by Teamscale, you must create a .lcov file by running:

coverage lcov

Invoking this command inside a directory with a .coverage file will take the coverage information and convert it into the lcov format, producing a coverage.lcov file. Upload the coverage.lcov file to Teamscale and specify the report format LCOV.

You may want to specify files or folders that should not be instrumented.

Specifying source files will speed up the coverage collection process.