Halon on call
Halon can take the first look at an alert when nobody is awake to. When an alert opens an incident, Halon reads the logs, deploys and metrics of what broke, ties it to a cause, pages whoever is on call with its answer, and proposes a fix.
Everything here is set under Halon → On-call, which admins see in the sidebar. Changing the switches and the spending ceiling needs the permission to manage the workspace, and changing unattended rules needs the permission to manage permissions. Both switches are off until you turn them on, and saving shows a toast.
Start Halon when an alert opens an incident
Section titled “Start Halon when an alert opens an incident”Switch on Start Halon when an alert opens an incident and save. From then on, every incident an alert opens gets a Halon investigation as soon as its channel is ready. The investigation posts in the incident channel the same way it does when someone presses Investigate, and the first message says the alert started it.
Alerts that group into one incident start Halon once. Later alerts that join the same incident do not start another investigation. See Grouping and deduplication for how alerts group.
Spending ceiling
Section titled “Spending ceiling”Spend at most sets how much the investigations alerts start may spend together in any hour. It covers a storm of alerts that do not group into one incident, such as many services failing at once. It starts at $20, and you can set anything from $0.50 to $1,000.
Each investigation gets what is left of the hour’s amount, up to its usual budget. Once less than $0.50 is left, the incident still opens, and its channel says Halon did not start on this alert, with an Investigate button to start it by hand. The page shows how much of the ceiling investigations started in the last hour are using right now.
Agents can change both switches and the ceiling over MCP with update_workspace_settings.
Page whoever is on call
Section titled “Page whoever is on call”Switch on Page whoever is on call with what Halon found to have Halon page someone at the end of an investigation an alert started. Halon starts every such investigation knowing who your on-call tool has on call right now, and picks the person on call for what broke.
With PagerDuty or Opsgenie connected, Halon raises a real page there:
| Tool | What Halon raises |
|---|---|
| PagerDuty | An incident on the service whose escalation policy has the person on call. PagerDuty’s escalation wakes them. |
| Opsgenie | An alert for the team, escalation, schedule or person that has them on call. Opsgenie’s routing wakes them. |
The page carries Halon’s answer and the fix it proposes, so the person knows what is broken before they open the channel. The incident’s timeline shows the page with a link to it in the tool, and the Pages panel on the incident page lists each page and where it stands. The person does not need to be a member of your workspace.
If the tool refuses the page, or an approval rule holds it, Halon escalates the incident in Firefight to the person instead when they are a member, with the reason the tool gave. They get the usual escalation message and reminder. A page an approval rule held is raised once an admin approves it.
Without PagerDuty or Opsgenie, or when Firefight cannot page through them, Halon escalates the incident to the person, who must be a member, and only when something it read, such as Jira Service Management Operations’ schedule, shows they are on call. With no way to tell who is on call, it says so in its answer and pages nobody.
Paging from a chat, an agent or a workflow
Section titled “Paging from a chat, an agent or a workflow”Ask Halon to page whoever is on call during an incident and it raises the page the same way. Agents connected over MCP can page with the page_on_call tool, and a workflow can use the Page on call step (see Workflows). Without an on-call tool they escalate the incident instead. An incident that is over cannot be paged for.
On-call tools
Section titled “On-call tools”The On-call tools card under Halon → On-call lists each PagerDuty and Opsgenie connection, what a page goes to there, and whether Firefight can page through it.
Firefight raises pages and keeps them in step as Firefight paging, which is granted only the tools that needs on each connection when you connect it. Approval rules on those tools still apply, and Gateway → Activity names whose change each call carried. If that grant is revoked or one of those tools is switched off, the card says why and Halon escalates instead. Grant again gives Firefight paging back what it lost. It needs the permission to manage permissions, and a tool switched off has to be switched on again under Integrations first. With no on-call tool connected, the card says Halon pages by escalating the incident and links you to connect one.
Pages stay in step with the incident
Section titled “Pages stay in step with the incident”A page Firefight raised, and a page from an alert your PagerDuty or Opsgenie alert source sent, stay in step with the incident both ways.
| You do this | This happens |
|---|---|
| Resolve the incident in Firefight | Each open page is resolved in PagerDuty, or its alert closed in Opsgenie |
| Acknowledge an escalation in Firefight | Each page still paging is acknowledged in the tool |
| Acknowledge the page in the tool | The timeline shows who acknowledged it. In Opsgenie, a member’s open escalations are acknowledged too |
| Escalate the page or add a note in the tool | The timeline shows it, with the note or who it went to. Escalations come back from PagerDuty only |
| Resolve the page in the tool | The incident is resolved once that was its last open page and a person resolved it |
An alert that resolves itself, such as a monitor clearing, never resolves the incident. Resolving an incident is a person’s call.
When the tool does not take a change from Firefight, for example because of a permissions error, the timeline says so and the Pages panel shows the reason. Make the change in the tool by hand.
Bringing changes back from the tool
Section titled “Bringing changes back from the tool”What people do in PagerDuty or Opsgenie reaches Firefight through the tool’s alert source, so add one under Settings → Alert Sources if you have not. The On-call tools card says which alert source brings changes back, or links you to add one. See Connect PagerDuty and Connect Opsgenie for the setup.
- PagerDuty. In the generic webhook (v3) subscription, send
incident.triggered,incident.reopened,incident.resolved,incident.acknowledged,incident.escalatedandincident.annotated. - Opsgenie. In the Webhook integration, post when an alert is created, acknowledged or closed, and when a note is added.
A page Firefight raised comes back through the alert source as an alert too. It joins the incident it was raised for rather than opening a second one.
Opsgenie
Section titled “Opsgenie”Atlassian ends support for Opsgenie on April 5, 2027, and moves its customers to Jira Service Management Operations. The Opsgenie connection says so on its card.
Halon can read who is on call now and next, list escalations, alerts and incidents, and, when you allow each one, raise, acknowledge, close, snooze, assign and escalate alerts, add a responder or a note, resolve and close incidents, put someone on call on a schedule for a while, and start a maintenance window that quiets an integration or an alert policy.
Jira Service Management Operations
Section titled “Jira Service Management Operations”Through a Jira connection, Halon reads who is on call in Jira Service Management Operations and its open alerts, and acknowledges, escalates or closes an alert when you allow it. Atlassian offers these only to a Jira connection made with an API token, once an organization admin allowed API tokens. Atlassian’s server cannot raise a new alert, so to page someone Halon uses PagerDuty or Opsgenie when connected, or escalates the incident.
Unattended rules
Section titled “Unattended rules”An unattended rule is a change your team lets Halon make on its own, decided ahead. For example, “Restart checkout when the average of its 5xx responses over the last 10 minutes is above 50.”
Unattended rules are for when nobody is around. What Halon may change in someone’s chat without asking is set under Settings → Workspace, see Changes without asking.
Rules apply only to an investigation an alert started. When such an investigation proposes a fix, Halon applies it on its own only when all of these hold:
- Every change in the fix is one a rule allows, the same change to the same thing on the map.
- Halon holds the permission for that change. Grant it to the Halon agent under Gateway → Permissions. A rule never grants anything by itself.
- A fresh reading of the rule’s metric, taken right then, is above its threshold.
Otherwise the fix waits for someone to apply it, as any fix does.
Halon also leaves a change for a person when it would stop something someone else started, change what customers get for a while (such as a block or a feature flag), or write data. In a chat those changes ask the owner, a chosen time, or a count of the rows first, and nobody is there to answer.
Adding a rule
Section titled “Adding a rule”Under Unattended rules, click Add rule and answer each question.
| Question | Choices |
|---|---|
| Halon may | Roll back or Restart |
| What on the map | A resource that a connected platform can roll back or restart |
| When the average of its | The metric to read, such as 5xx responses, p95 latency, CPU or memory, and how many minutes to average over (10 to start with, at most a day) |
| Is above | The threshold, in the units your provider reports |
A rule that cannot act yet says why on its row, for example when Halon holds no permission for the change or the resource left the map. Switch a rule off to keep it without using it.
What you see when Halon acts
Section titled “What you see when Halon acts”When Halon applies a fix under a rule, it posts in the incident channel. The message names the fix, each rule it acted under and who set it, what Halon read just before acting, and how to undo each step. Undo fix on the fix’s progress message has Halon write the steps that put it back.
When a rule covered the fix but Halon did not act, for example because the reading was under the threshold, it says why under its answer, and the fix waits for someone to apply it.
Every call Halon makes under a rule is in Gateway → Activity, named with the rule. If an approval rule would have held the change, the request is approved by the unattended rule, and Gateway → Approvals shows the rule as the one that decided.
Deleting a rule
Section titled “Deleting a rule”A rule Halon has acted under stays for the record, and its row says how many times. Turn it off instead. A rule Halon never acted under can be deleted.
Over the API and MCP
Section titled “Over the API and MCP”With the permission to manage permissions, you can read the same rules over the API with GET /unattended_rules and over MCP with list_unattended_rules. What Halon may change on its own is set only by a person, so a change sent with a token, over the API or MCP, is refused with a message saying so. Make the change on the page, or ask Halon in a chat and confirm it there.
Approvals for whoever is on call
Section titled “Approvals for whoever is on call”On any approval rule, switch on Whoever is on call may approve too. The people an incident was escalated to, and the members Halon paged through an on-call tool, can then approve the requests that rule holds for that incident. When nobody working the incident can approve a request, Firefight sends it to them in a direct message. Someone escalated to after a request was made gets it then, once. See Approvals.