Pull request review
Halon reviews pull requests in the repositories your GitHub, GitLab and Bitbucket connections hold. It reads each change against what Firefight knows about your systems, past incidents and their postmortems, what depends on the code on the map, live metrics and the code itself, and comments only on problems it has checked.
Asking for a review
Section titled “Asking for a review”By default Halon reviews a pull request when someone asks for one.
| Code host | How to ask |
|---|---|
| GitHub | Comment on the pull request mentioning Firefight’s app with the word review, such as @firefight review. GitHub does not let you pick an app as a reviewer, so asking is a comment |
| GitLab | Add the user your connection acts as to the merge request’s reviewers, or comment mentioning it with the word review |
| Bitbucket | Add the user your connection acts as to the pull request’s reviewers, or comment mentioning it with the word review |
A comment works when it mentions Firefight’s handle on that host, @firefight or @halon, followed by the word review on the same line, such as @halon review this please. Halon → Pull requests in the dashboard lists each connected code host with the exact handle to use there.
Halon also reviews every pull request it opens itself, as soon as it opens.
Reviewing every pull request
Section titled “Reviewing every pull request”An admin can have Halon review pull requests as soon as they open, without anyone asking. Open Halon → Pull requests in the dashboard’s sidebar and choose under Reviewing without being asked.
| Choice | What Halon reviews |
|---|---|
| Only when someone asks | Pull requests someone asks about, and its own. The default |
| Every pull request when it opens, in every repository | Every pull request in every repository your code hosts reach |
| Every pull request when it opens, in the repositories you choose | Every pull request in the repositories you pick from the map. Choose at least one |
Click Save, and a toast says what Halon now reviews. A draft is reviewed once it is marked ready. Each review spends from your AI budget like any other run. Changing this needs the permission to manage the workspace, and agents can change it with update_workspace_settings over MCP, as pull_request_reviews_on_open (off, all or chosen) and pull_request_review_repositories.
What a review posts
Section titled “What a review posts”As soon as a review starts, Halon posts one summary comment on the pull request and keeps it up to date. It lists the findings as each one is checked, then says what the change does, its risk (low, medium or high) and why, and how many files it reviewed, with any file it left out and why, such as a binary or a file that may hold secrets. Findings Halon withdrew are listed at the end.
When the review finishes, each finding is a comment on its line. A finding on a line the diff does not show, or one the code host would not place a comment on, goes in the review’s own body under Not on a line. A serious finding also gets a comment of its own the moment it is found, so everyone on the pull request is notified.
Every finding reads the same way:
| Part | What it says |
|---|---|
| Header | The category, the severity (Critical, Major, Minor or Trivial) and the effort to fix it (Quick win, Heavy lift or Low value) |
| Claim | One line, in bold, saying what goes wrong |
| Why | Why it matters here |
| How it fails | The chain from the change to the failure, step by step |
| Evidence | What shows it, such as an incident it was seen in before, a postmortem, a service downstream on the map, a metric in production or the code at the pull request’s head, each linked |
| Suggested fix | What to change |
| How Halon checked | What Halon ran to make sure the finding holds |
| Instructions for a coding agent | The fix written for a coding agent, which it is told to check against the code before acting on |
A finding that came from a lesson names it, such as Lesson from INC-052 or Lesson from PR #412, linked to where it came from.
Halon reads the code and your systems but does not run the pull request’s tests. On Bitbucket, which shows folded sections as plain text, How Halon checked and the coding agent’s instructions are written as plain sections.
On a public repository
Section titled “On a public repository”Anyone can read a public repository, so Halon keeps your internal detail off it. A finding there keeps its claim, its suggested fix, and its why and failure chain written in terms of the code, and leaves out the evidence, How Halon checked and the coding agent’s instructions. Each finding links to See the evidence in Firefight instead. The summary comment leaves out what the change does, why the risk is what it is and why a finding was withdrawn, and links to the same place. Halon’s replies on the threads are written in terms of the code too. A repository whose visibility the code host does not say is treated as public.
See the evidence in Firefight opens the review in the dashboard, with every finding in full, for anyone who may read investigations. A resource on the map you cannot see is left out of the evidence for you.
After a push
Section titled “After a push”Each new push to a pull request Halon reviewed is reviewed again. Halon also checks every open thread whose file the push changed and replies to say whether the fix holds. A fixed thread is resolved, and its comment is edited to lead with Fixed in and the commit. The summary comment counts what is still open and what is fixed.
When you reply on one of Halon’s threads, Halon answers. If your reply shows the finding was wrong, Halon says so, withdraws it and resolves the thread.
Lessons from incidents and from Halon’s pull requests
Section titled “Lessons from incidents and from Halon’s pull requests”Reviews check every change against the lessons your team approved. A lesson comes from an incident Halon learned from, or from one of Halon’s own pull requests that a person changed before merging or closed, when they said why or the same cause came up before. A merged pull request nobody changed teaches nothing.
Lessons live on the Lessons tab of the Memory page, under Halon in the dashboard’s sidebar. The tab shows how many wait for approval, and the filters split them into Waiting for approval, Approved and Turned down. Each lesson shows what it is about, where it came from and what reviews do with it.
| Action | What happens |
|---|---|
| Approve | Reviews check every pull request against it from then on |
| Edit | Your wording replaces it as a lesson approved by you. The old wording is kept as turned down, so Halon does not learn it again |
| Turn down | Reviews never check it, Halon stops using it, and it is not learned again. You can say why |
Deciding on a lesson, and marking it strict, takes the Decide on lessons permission. Admins hold it, and so do an incident’s lead for that incident’s lessons and whoever asked Halon for a pull request for that pull request’s lessons, without a grant. Grant it under Permissions to let anyone else decide on every lesson, and set someone to No access to take the default away. A button you cannot use says who can in its tooltip, and every decision shows a toast.
Strict lessons
Section titled “Strict lessons”When you approve or edit a lesson, switch on Strict to have it hold pull requests. A pull request that goes against a strict lesson gets a failing Halon review check until the finding is fixed or someone overrides it, and the check passes once every such finding is. A pull request no strict lesson touches gets no check at all. To make an approved lesson strict, or not, choose Edit and change the switch.
Pull requests held by strict lessons sits at the top of the Lessons tab while any pull request is held. It lists each one with the finding, the lesson behind it and whether its check is failing or was overridden, by whom and why. When Firefight could not set the check on the code host, the row says so and why, such as “Couldn’t set the check on GitHub” with the host’s reason, and it tries again at the next change. The check’s link on the code host opens this list with that pull request marked.
The check names the lessons behind it, such as “1 finding from a strict lesson is open: Lesson from INC-052.” On a public repository it names none and says only “1 finding from a strict lesson is open.”
To let a pull request through, choose Override and say why. The finding stays on the pull request, and the check passes once nothing else holds it. Overriding takes the Override strict lessons permission. Admins hold it, and so do the owners of the services the pull request belongs to, meaning the members and manager of each service in the catalog and of its owner teams, for that pull request without a grant. Grant it under Permissions to let anyone else override on any pull request, and set someone to No access to take the default away. When you cannot override, the button’s tooltip says who can.
Making the check required
Section titled “Making the check required”Halon sets its check on the pull request’s latest commit, as a commit status on GitHub and GitLab and a build status on Bitbucket. Setting it needs:
| Code host | What it needs |
|---|---|
| GitHub | Commit statuses read and write on Firefight’s GitHub App. When the app asks for a new permission, an owner of your GitHub organization accepts it on GitHub |
| GitLab | A token with the api scope and at least the Developer role in the project |
To stop a held pull request from merging, make Halon review a required check in your branch protection rules on the code host. This is optional. Without it, the check still shows on the pull request but does not block the merge.
High risk pull requests in Slack
Section titled “High risk pull requests in Slack”When Halon reviews a pull request as high risk, it also posts it once in the Slack channel of the service that owns the repository, or its owner team’s channel when the service has none. The service is the catalog entry whose Repository attribute names the repository. The message links the pull request, quotes why it is risky, lists the open findings most severe first with the lesson behind each, and has an Open pull request button.
To stop these for a service, open its entry in the catalog and switch off Post high risk pull requests. The switch shows on entries that have a repository, and is on by default.
Problems prevented
Section titled “Problems prevented”Halon’s Performance page opens with the problems its reviews prevented, meaning serious, major or critical findings that were fixed before their pull request merged.
Setting up a code host
Section titled “Setting up a code host”Halon reviews through your existing GitHub, GitLab or Bitbucket connection, see Integrations. On GitLab, commenting needs a token with the api scope and at least the Reporter role in the project, and resolving a thread needs the Developer role. On Bitbucket, commenting needs a token with the write:pullrequest:bitbucket scope.
Halon posts and resolves through the connection’s own pull request tools, as Firefight’s Halon pull request review agent. It is granted only those tools, and every comment, edit and resolve is in Activity under its name. A tool switched off under Integrations stops reviews posting on that connection. When an approval rule holds one of these posts, nothing else of that review is posted until it is decided. Replies, resolves and edits made meanwhile wait, and go up in order once the approval is given or declined.
A review reads the pull request through the connection as Halon, so it needs Halon’s read access to that connection. If an admin takes it away under Permissions, no new review starts there, and the refused read is in Activity. A review already under way stops, and its summary comment says it stopped because “Halon has no access to read the code host connection it reads through.”