How to Handle Backups (Import, Export, Automatic Backups)

Backups in Teamscale are very helpful when upgrading to a new Teamscale version, or to mitigate the risk of potential data loss.

• Exporting Backups
• Importing backups
• Storage snapshot backups
  • Configuring Automatic Snapshots
  • Recovering using a Storage Snapshot

Exporting Backups

Depending on your needs (e.g., during an update), you can export only global data (like users, groups, settings, permissions, dashboards), project data (e.g., tolerations, false positives, baselines, and external findings), or both. Furthermore, you can choose to only export project data for some rather than all projects.

Once you have chosen what to export, you have two options to initiate the backup:

1. When you click Download Backup, your browser will download a single backup ZIP and you can store it locally.

2. Alternatively, you can provide an URI (see table below) and click Store Backup to let Teamscale store your backup remotely. This way is often perferrable if the backup ZIP would be too large to comforably download.

Rescue backup on startup

Under certain error conditions (e.g. database corruption) the above mechanism may not work anymore for exporting backups. In this case the command line argument -b <PATH-TO-BACKUP-FILE> passed to the teamscale.sh or .bat file can be used to dump a rescue backup during the startup of Teamscale.

Importing backups

As for exporting backups, you can two options to import a backup:

1. When you click Select a Backup File, your browser will let you select a single backup ZIP which you have downloaded earlier. By clicking Upload Backup you initiate the import.

2. Alternatively, you can provide an URI (see table below) you earlier exported a backup to. Then click Import Backup to let Teamscale fetch the backup and import it.

URI Schemes for Backup

file

A path to a backup ZIP on the Teamscale instance's local disk, e.g., /var/backups/teamscale/1.zip or file:///var/backups/teamscale/1.zip

s3+http,s3+https

A location in an S3 bucket, using either HTTP or HTTPS are transport, e.g., s3+http://01234567:abcdefgh@s3. The format of the URI is as follows: (s3+http|s3+https)://[<access key>:<secret key>@]<hostname>[:<port>]/<bucket>/<path> To avoid ambiguity, the S3 bucket must be part of the URI's path, not the hostname. If access key or secret key contain a slash (/), it must be percent-escaped (%2F). If desired, access key and secret key can be kept in the Account Credentials of external credentials rather then encoded in the URI. To do so, create Account Credentials with an ID and URL of http(s)://<hostname>[:<port>] Username and password are your access key and secret key, respectively.

Storage snapshot backups

Teamscale can be configured to automatically create snapthots of its database.

Configuring Automatic Snapshots

The backups created by Teamscale contain all user generated data (e.g. configurations, architectures, dashboards, and tolerated findings), and uploads (such as coverage data). These backups are useful when migrating to a new version or recovering data that was mistakenly deleted. However, restarting an instance from such a backup requires a reanalysis, which can take some time and might be unsuitable for crash recovery, when fast recovery time is more important.

This gap is filled by storage snapshot backups, which store a snapshot of the entire storage system in one backup file, which has a size that is almost as big as the storage directory. These storage snapshot backups can be created manually in the Backup view of the Admin perspective. It is recommended to configure a regular schedule for creating storage snapshots in the Teamscale Settings . The backup location supports the same placeholders and URI schemes as the normal backup, so storing snapshots in S3 is possible.

Snapshot Support

Note that storage snapshot backups are only supported by these database types: RocksDB, In-Memory (see also Configuring Teamscale on how to configure the database type)

Recovering using a Storage Snapshot

To recover from a crash and start a Teamscale instance from the storage snapshot, you have to start the instance with a specific property. For this, add the following parameter to the jvm.properties config file.

-Dcom.teamscale.snapshot-backup.load-location=LOCATION

Please note that the import will only be performed, if the storage system is completely empty, to prevent you from accidentally destroying a live instance. In this case, a warning will be logged and the existing instance will be started without any import. Additionally, the version of Teamscale must be the same as the version that was used to create the snapshot. You can not use snapshots during a migration to a newer version of Teamscale.

When working with storage snapshot backups, please consider these general guidelines:

• Backups should be stored on a separate machine (or even data center) to prevent data loss in case of disk failures or environmental hazards.

• The restore procedure should be tested regularly, both to ensure that all data is available and the steps are known and also to get an idea of how long the restore procedure takes.

• Also think about backup procedures for the operating system and hardware for a fallback machine, as in case of a hardware failure, you will also need a new machine to start from.

• Think about procedures to switch URLs to a new machine in case of hardware failures, such as load balancers or suitable DNS update strategies.