How to Review Merge Requests with AI Agents
Teamscale can review merge requests using review agents: AI agents backed by large language models that inspect the changes, use tools to explore the code base, and produce review comments. Several agents with different focus areas can review the same merge request, their comments are verified and consolidated, and the result is shown in Teamscale and can be posted back to the code collaboration platform.
Prerequisites
- An AI model provider must be configured, and a chat model routing rule must select a model for the
MERGE_REQUEST_REVIEWtask. Additionally, routing rules forMERGE_REQUEST_REVIEW_VERIFICATIONandMERGE_REQUEST_REVIEW_CONSOLIDATIONare highly recommended. See AI Model Providers for details. - The project must be connected to a code collaboration platform with merge request analysis enabled, for example, GitLab.
- At least one review agent must be linked to the project.
Review Agents
A review agent is a reusable review configuration and provides the context and tools to an LLM. It consists of:
- Main prompt: The overall instructions to the agent, for example, its persona and focus.
- Review rules: Specific instructions on top of the main prompt, where each rule is one focused check.
- Tools: Functions the agent can choose to call while reviewing. The agent decides when to use them, and only the tools you enable are available to it. Enabling too many tools can cause some LLMs to deviate from the initial goal.
- Attachments: Named context added to the prompt. This is added to the initial agent prompt, and you can refer to an attachment by its name in the main prompt.
The available tools are:
| Tool | Gives the agent access to |
|---|---|
| Code search | Searching and reading files in the code base beyond the diff. |
| Dependency lookup | Incoming and outgoing dependencies of the changed files. |
| Commit history | The version history of files, including the commit message and their content at older revisions. |
| Finding descriptions | The descriptions of Teamscale's finding categories. |
| Existing findings | The findings present in the changed version of the code. |
| Added findings | The findings introduced by the merge request. |
| Work items | Searching and reading work items: issues, spec items, and test items. |
The available attachments are:
| Attachment | Content |
|---|---|
| Issue Info | The issue linked to the merge request, if one exists. Optionally includes the issue's metadata. |
| Merge Request Info | The merge request's title and description. |
| Agent Instruction Files | Instruction files for AI coding agents contained in the repository (CLAUDE.md, AGENTS.md, Copilot instructions) that apply to the changed files. |
Teamscale ships with a set of preconfigured review agents. You can use them as they are, adjust them to your needs, or use them as templates for your own agents. Your edits to the shipped agents are never overwritten by Teamscale updates.
Review Agents and Model Routing
A review agent does not specify which model it uses. Instead, the model is selected by the AI model routing rules, where the agent's name is available as the task details attribute. This allows routing different agents to different models, for example, a stronger model for the Security Review agent. A routing rule can even select multiple models for the same agent.
Managing Review Agents
Review agents are managed in the Project Configuration perspective in the Review Agents view.
Creating a review agent requires the global Create Review Agents permission. The creator becomes the owner of the agent and can grant other users access via role assignments, analogous to other configuration objects such as dashboards.
Every saved change creates a new version of the agent, and you can add a comment describing the change. The version history and a comparison of any two versions are available from the Version column of the Review Agents view, so you can track how an agent's instructions evolved. Review agents can also be exported to and imported from .tsreviewagent files, which allows sharing them between Teamscale instances.
Linking Review Agents to Projects
Review agents are global objects, but a review can only run agents that are explicitly linked to the project. This keeps control over which prompts, tools, and model costs apply to each project with the project administrators.
To link review agents, edit the project in the Project Configuration perspective and select the agents in the Review Agents field of the Agentic Review Configuration section.
A review agent cannot be deleted while it is still linked to a project.
Running a Review from the Web UI
Open the merge request in the Activity perspective and switch to the Agentic Review tab. Select the review agents to run (all linked agents are preselected) and click Generate new Review.
The review runs as a background job on the server, so you can leave the page and come back later. While it is running, a progress panel shows one step per review agent, including the agent's current activity, followed by the Comment Verification and Review Consolidation steps. A running review can be canceled with the Cancel Review button.
Behind the scenes, each generated review passes through two additional phases that improve its quality:
- Verification: Each review comment is checked by a separate model conversation that tries to refute it against the actual code. Refuted comments are dropped, which reduces false positives. If no AI model is configured for the task
MERGE_REQUEST_REVIEW_VERIFICATION, this step is skipped and all comments are preserved. - Consolidation: Comments from different agents that describe the same problem are merged into a single comment, attributed to all reporting agents. When several agents agree on a comment, this is a strong signal for its relevance. If no AI model is configured for the task
MERGE_REQUEST_REVIEW_CONSOLIDATION, this step is skipped and duplicates are preserved.
The result consists of a summary and a list of review comments. Each comment references the affected file and lines, is classified as Major or Minor, can show the affected code inline, and names the review agents that found it. If the source branch changes after the review was generated, Teamscale marks the review as possibly outdated.
The metadata line below the summary shows the estimated cost of the review, broken down by review, verification, and consolidation step. Cost estimation requires AI pricing to be configured.
Past reviews of the same merge request remain accessible via Show past reviews. Reviews are deleted automatically after 90 days.
Rating Review Comments
Each review comment can be rated by every user with one of three ratings: Very helpful, Helpful, or Not helpful / false positive.
Rating comments serves two purposes: you can filter the comment list by rating, for example, to hide comments already dismissed as not helpful, and the ratings are aggregated in the Agentic Review Statistics chart of the AI Usage Information view, which shows how helpful the generated reviews are over time and thus helps tuning your review agents.
Ratings are personal: each user sees and edits only their own rating.
Advanced Configuration: Reviewing with Multiple Models
When the routing rule for the MERGE_REQUEST_REVIEW task selects multiple models with the Route to all models behavior, each selected review agent runs once per model. This can be used to benchmark models against each other with the same agent configuration, or to obtain more intensive review feedback by combining the perspectives of several models in a single review.
The runs of an agent are shown as <agent name> [<model name>] in the review progress and in the comment attribution, so you can tell the models' results apart. The cost breakdown of the review records each run separately as well, including the model that served it. The consolidation step merges the comments of all runs just like comments from different agents, so a comment reported by several models is a strong signal, and duplicates do not clutter the review.
The verification and consolidation steps always use a single model. If their routing rules select multiple models, only the first selected model is used.
Triggering a Review from GitLab
For GitLab, merge request authors can trigger an agentic review directly from the merge request, without opening Teamscale.
To enable this, activate the option Enable Agentic Reviews for merge requests on the project's GitLab connector. The trigger is delivered through GitLab's webhook, so the webhook configured for the repository must include comment events (the Note Hook).
The author then posts a comment containing /teamscale review on the merge request. The command must be the first text of a top-level comment and is only accepted on open merge requests. Only the merge request's author can trigger the review, which prevents other users from starting potentially costly AI reviews on someone else's merge request. Since the comment carries no agent selection, a review triggered from GitLab always runs all review agents linked to the project.
Teamscale acknowledges the command with an 👀 reaction and a progress comment linking to the merge request view in Teamscale. When the review finishes, the reaction changes to ✅, each review comment is posted as an inline discussion on the affected line, and the summary is posted as a reply.
When a review is re-run, the previous review's inline discussions and summary are removed to avoid clutter. Discussions where a user has replied are kept, because they may contain an ongoing conversation.
Posting a Review to the Code Collaboration Platform
A review generated in the web UI can also be posted to the merge request manually using the Send review to GitLab button in the Agentic Review tab. This requires the project permission Send AI Reviews to Code Collaboration Platform and the connector option Enable Agentic Reviews for merge requests. Comments on files or lines that are not part of the merge request's diff cannot be placed as inline comments; the summary then states how many comments could be posted.
Monitoring Costs and Debugging
Every model interaction of an agentic review is recorded in the AI Log, including the full prompts and responses. This is the first place to look when a review behaves unexpectedly, for example, to inspect which tools an agent called and what instructions it received.