Teamscale Docs

Appearance

Best Practices for Coverage Uploads ​

Test coverage is usually the largest and most frequently uploaded kind of external analysis data. Every upload has to be transferred, parsed, stored and read again during history analysis, so coverage uploads from a busy CI pipeline can noticeably slow down a Teamscale instance and inflate its backups.

We recommend you do the following:

Merge and Compact Coverage Before Uploading ​

A test stage rarely produces a single coverage report. A build with 30 modules produces 30 coverage files, and uploading each of them with its own teamscale-upload call makes Teamscale open and process 30 separate upload sessions for what is, from your perspective, one test run.

Merge them instead. The coverage compact command of our teamscale-build CLI reads all coverage reports below a directory and writes a single Teamscale Compact Coverage report. That report is also much smaller than the reports it replaces, because Compact Coverage stores covered lines as line ranges instead of repeating structural information that Teamscale already derives from your source code. You therefore get both effects at once: one upload session per test stage, and less data to transfer, store and back up.

Merge the coverage of each test stage into one Compact Coverage report and upload that report in a single teamscale-upload call.

Use one partition per test stage, for example Unit Tests, Integration Tests and System Tests, so you can tell in Teamscale which stage covered which code.

bash
teamscale-build coverage compact -i ./build/reports -o ./compact-coverage.json -f JACOCO

TEAMSCALE_ACCESS_KEY=$MY_ACCESS_KEY teamscale-upload --server "$TEAMSCALE_URL" --project my-project --user ci --partition "Unit Tests" --format TEAMSCALE_COMPACT_COVERAGE ./compact-coverage.json

Replace JACOCO with the format your test tooling produces. teamscale-build converts from all common coverage formats.

Store Uploaded Reports in an External Storage Backend ​

By default, Teamscale stores uploaded reports in its internal storage, where they take up space in every backup. Teamscale can instead store them in an S3 bucket and read them back from there when it needs them.

Configure an S3 external storage backend for instances with high coverage upload volume.

This keeps Teamscale's backups small, allows several projects to share the same uploaded data without duplicating it, and allows you to prune old uploads with the data management features of your S3 service.

Configuring the backend does not change anything about your CI pipeline. Once an external storage backend is configured and enabled, Teamscale redirects uploads to it automatically, so the commands above stay as they are.

Limit Coverage Uploads for Old Releases/Commits ​

Every upload belongs to a commit. If that commit is older than the newest commit that Teamscale has already analyzed, Teamscale rolls back to the commit the upload belongs to and reanalyzes everything that comes after to update historic metric trends. The further back the commit lies, the longer this reanalysis takes, and uploads for more recent commits have to wait until it is finished.

Process uploads for commits older than three days at night.

In Admin > Settings > Late External Upload Processing, set the schedule to a cron expression in the evening, for example 0 21 * * * (every day at 21:00), and the age threshold to 4320 minutes (three days). Starting in the evening leaves the whole night to integrate these uploads while nobody is working. Coverage for commits this old usually does not need to be processed right away, and postponing it gives priority to recent data during the day.

Reject uploads for commits older than 30 days.

In the project options (), section External Uploads, set Max upload age (days) to 30. Coverage for code this old rarely informs current decisions, but integrating it can cause very large rollbacks.

Teamscale rejects such uploads with an error, so the upload step of your pipeline fails. If you intentionally upload historic coverage, for example to backfill an old release, clear the option for the duration of the upload.

The ages above refer to the code under test, not to the time of the test run.

Teamscale determines the age of an upload from the timestamp of the commit you upload the coverage for. This timestamp determines for how many newer commits Teamscale has to recalculate data, and thus how long the integration takes. For example, if you run today's tests against a release that was built from a commit made six weeks ago, the upload is six weeks old, even though the coverage report was generated today.