Workflows

A workflow is a short list of steps that Firefight runs for you, such as “every weekday at 9:00, look through the logs for probing and raise a notice if there is any”. Each workflow acts as its own identity, never as the person who started it, and can do only what an admin granted it.

Where workflows live

Open Workflows under Halon in the dashboard. The page has four tabs:

Tab	Shows
Workflows	Every workflow, whether it is a draft or live, when it starts, its next run and how its last real run went. Test runs never count as the last run
Runs	Every run, newest first
Done	Workflows that run once, after their run, with what they did
Library	Firefight’s own workflows to start from, and your workflows with where each came from, with Export as a file and Import a workflow. See Workflow library

Click a workflow to open its canvas, where you see and change its steps, try it, publish it and read its runs. Click a run to open it on the same canvas, with each step in the order it ran, what it took in and gave back, and every call it made. See Building a workflow on the canvas.

Making a workflow

There are four ways to make one.

• From a starter. Press New from the library on the Workflows tab and pick one. Check something later asks what to check, when, in which time zone and where to post the verdict. Look for probing in the logs asks which days, at what time and in which time zone. Press Make draft to make it.
• From Firefight’s library. On the Library tab, press Use on one of Firefight’s checks, such as Disk space check or Certificate check, and use it as it is or make it your own copy. See Workflow library.
• From a file. On the Library tab, press Import a workflow and pick a file exported from another workspace.
• By asking Halon. In a chat, say what you want and when, for example “in 30 minutes check checkout and tell me how it is” or “every weekday at 9:00 check the logs for probing”. Halon shows a card first with when it runs, each step, what it will be able to do and any questions it asks, and makes it once you confirm. A time you give without a time zone is taken in yours, and everyone who reads the card sees that same moment in their own time zone, with the zone named. Anything you ask Halon to do later or on a repeat becomes a workflow.

A workflow made from a chat also shows a card in that chat, after the answer that made it, with Run now and either Cancel for one that runs once or Turn off for one that repeats. When the chat was in a Slack thread, the card is posted in the thread with the same buttons.

A new workflow starts as a draft. It does nothing until it is published. Change it on its canvas, step by step, or ask Halon beside the canvas to change it for you.

When a workflow runs

A workflow has one trigger. Every workflow can also be run by hand, whatever its trigger.

Trigger	Runs
Run by hand	When someone presses Run on the dashboard, presses a runbook’s Run it, or asks Halon. It can ask questions first, each with a default, which the run then uses
On a schedule	On a repeat, in its own time zone. It repeats every 5 to 720 minutes, every 1 to 24 hours at a minute you pick, on days of the week at a time, or once a month, such as the first Monday at 09:00
Once at a time	Once, at a date and time up to a year ahead, in its own time zone. Afterwards it moves to the Done tab
Incident declared	When an incident is declared
Incident changed	Once each time an incident changes in a way you pick: its status, severity, type, lead, a role or a custom field, or it being accepted, resolved, reopened, cancelled or escalated
Alert received	Once for each alert routing lets through, after grouping, so a storm of the same alert starts one run. An alert routing drops, or one that joins an open group, starts nothing
Halon raised a notice	When Halon raises a problem, from a check, a security event or a workflow, or when one gets worse
Investigation finished	When a Halon investigation concludes or gives up, with what it found and how sure it is
Follow up or postmortem changed	When a follow up is created or completed, or a postmortem is started or completed
Another workflow’s run failed	When a run of another workflow fails, to tell someone or put it right. Test runs do not count
Inbound webhook	When a signed POST reaches the workflow’s own address. See Starting a workflow from outside
Deploy or CI finished	When a connected code host, CI service or deploy platform says a deploy or a CI run ended, with how it ended
Resource changed	When the resource map records a change on a resource: it appeared, went away, deployed, changed status, was renamed, scaled or a setting moved
Service scaled	When the resource map sees a service run more or fewer instances than before, with how many it ran and how many it runs now. See When a service scales
Security event	When a connected integration reports a security event, such as a secret that leaked into a repository
Issue assigned to Firefight	When someone assigns or hands an issue to Firefight in a connected tracker, or adds the label the trigger names to a GitHub, GitLab or Linear issue. See Starting on a label
Called by a workflow	When another workflow runs it from a Run workflow step, with its inputs, and takes what it hands back

Change the trigger by clicking it on the canvas, or by picking another one at the top of the canvas’s list of steps.

Only when

A trigger that starts when something happens has Only when conditions, so it runs for some of those things and not all of them, such as only critical incidents or only alerts for the checkout service. Each condition picks what it reads, how it compares (such as is one of, contains, starts with, matches the pattern or is empty) and what with. A count, such as the instances a service runs, also compares as a number, with is more than or is less than and a number such as 20. Where the field has a fixed set of values, such as your severities, you pick them from a list. A run starts only when every condition holds, and a trigger holds up to 20.

Steps read what started the run with the picker in each field, such as {{trigger.incident.identifier}} or {{trigger.alert.service}}.

What a workflow changes itself never starts that same workflow again. Each workflow starts at most its Most runs per hour, 100 unless you set another, so a storm of alerts or changes cannot start more. A run that something happening would start past it is skipped, and the workflow’s page says how many runs were skipped in each hour of the last week. Any other start that is refused, such as a second run for the same thing while One live run per holds, is shown on the workflow’s page with why.

When a service scales

A Service scaled workflow runs each time the resource map sees a service’s instance count change, between one sync and the next or when the integration says something changed. The count is the one shown as Instances on the service’s panel on the map. See Instances and scaling for which integrations report it.

Its Only when conditions read:

Condition	Reads
Instances after	How many instances the service runs now, as a number
Instances before	How many it ran before, as a number
Resource name	The service’s name on the map
Where it runs	The integration that reports it
Kind of resource	Service, database and the other kinds on the map

For example, Instances after is more than 20 runs it only when a service scales past 20. Steps read the counts as {{trigger.change.from}} and {{trigger.change.to}}, and the service as {{trigger.resource.name}}. A Resource changed workflow that waits for every change hears a scale too.

Starting on a label

An Issue assigned to Firefight trigger runs when someone assigns an issue to Firefight in Linear. Give it a label as well, and it also runs when someone adds that label to an issue in GitHub, GitLab or Linear, or opens one with it, whatever its case. Leave the label empty and it runs only on assignment. Adding a label starts the workflow and nothing else, so Halon does not reply in the issue the way it does when the issue is assigned.

GitHub and GitLab issues cannot be assigned to Firefight, so there a label is the only way to start it. Firefight hears a GitLab label once it follows changes in your group or project, the same way it hears a pipeline finish. A confidential GitLab issue never starts it.

Steps read {{trigger.issue.how}}, which is assigned or labelled, and {{trigger.issue.labels}}, the labels that were added, beside the issue’s key, title and link and who assigned or labelled it. A GitHub or GitLab issue’s key reads like acme/web#12. GitHub names who labelled an issue by username only, so a Who assigned it, by email condition never matches a GitHub label.

Starting a workflow from outside

A workflow whose trigger is Inbound webhook has its own address. Click the trigger on the canvas to see it, under Its address, with Copy address. An admin also sees Copy secret and Replace address. Replacing asks first, and the old address and secret stop working at once.

Send a POST to the address with a JSON object as the body, signed the same way Firefight signs its outbound webhooks:

Header	Holds
X-Firefight-Timestamp	The time you signed it, in ISO 8601
X-Firefight-Signature	v1= followed by the HMAC-SHA256, in hex, of v1:<timestamp>:<body>, keyed by the workflow’s secret
X-Firefight-Delivery	Optional. An id of your own, so the same delivery sent again starts nothing new

A signature more than five minutes old is refused. Steps read the body as {{trigger.body}}, and Only when conditions read its fields as body.<field>, such as body.service.

Answer	Means
202	A run started, with its run_id
200	Nothing started, because the body did not meet the trigger’s conditions
400 or 422	The body is not a JSON object
401	The signature is missing, wrong or more than five minutes old
404	No workflow starts at this address, or it is switched off
409	The workflow could not start a run, with why, such as having already started its most runs per hour
413	The body is larger than 100 KB
429	More than 120 deliveries to this address in a minute

Starting from a runbook

A runbook can link a workflow, and its Run it button starts that workflow with the questions it asks. See Runbooks.

Making a one off repeat

A workflow that ran once at a time moves to the Done tab with what it did. To run it again on a repeat, press Repeat it there and choose how often. That makes a draft you publish as usual.

What a step can do

Step	Does
Call a tool	Calls any tool Firefight or a connected integration offers, such as reading a service’s status or restarting it
Ask Halon	Asks Halon a question about what earlier steps found and gets a verdict and a summary back
Run a skill	Has Halon follow one of its skills, such as the probing scan. It only reads
Investigate	Starts a Halon investigation of an incident or a problem and waits for what it finds and how sure it is
Classify	Has Halon sort what it is given into one of the routes you name, each a path out of the step
Open incident	Declares an incident through your declare form, as /ff declare does
Update incident	Changes one thing about an incident: its status, severity, type, a field, its lead or a role, who is invited, or a runbook
Post incident update	Posts an update on an incident, once or again every few minutes while it is live
Page on call	Pages whoever is on call through your on-call tool, reading who that is first, or escalates to a member when you have none
Create follow up	Adds a follow up to an incident, and opens its issue in your tracker when you ask
Post to channel	Posts a message to a Slack channel, or back where the workflow was asked for
Message a person	Sends a direct message to a member, an incident’s lead, whoever holds one of its roles, or whoever is on call for it
Raise a notice	Raises a monitoring notice with Halon, the same as one Halon raises on its own. A notice about a resource that is not on the map is still raised, and names the resource in words
Status page update	Posts or updates your status page through one of your connected status pages, with any of its tools that change something
Run command	Runs one shell command in the run’s sandbox, with ff for your connected tools and the step’s data in the file named by $STEP_DATA. It only reads unless you say it changes something
Run script	Runs a Python, Ruby or JavaScript script in the run’s sandbox. It reads the step’s data as JSON on its input and prints JSON last, which later steps read
Check from outside	Checks an address as a user meets it: whether it answers, its status, its TLS certificate and its DNS
Browser	Opens a page in Halon’s browser on desktop and phone, follows a path you set and keeps a screenshot of each, shown on the run. One step of the path may send what the page sends, which is a change and never happens in a run nobody pressed
HTTP request	Sends one HTTP request. Addresses on private networks are refused. GET and HEAD read, any other method changes something. Name a connection to send its credentials, which go only to that connection’s own hosts. An answer of 400 or more fails the step
Send webhook	Sends a signed POST with the run’s data through one of your outbound webhooks, as a delivery you see in its list
Ask a person	Asks someone a question in a direct message and on the run, with choices to pick from, a short form to fill, or both. See Asking people
Approval	Asks named people or a workspace role to approve before the run goes on. See Asking people
Manual step	Pauses the run and gives a person a task, such as opening a ticket, with Done in Slack and on the run. See Asking people
Branch	Takes a different route depending on what an earlier step found
Set data	Names values for later steps, such as a service from an alert. A value that is only an earlier value keeps its shape, and a JSON object or list you write becomes one, which is how you reshape data
Stop	Ends the run, as succeeded, leaving the steps that have not run yet, or as failed with the reason you give
Delay	Waits for some seconds, minutes, hours or days, or until a time of day
Watch	Watches something for a while, such as an error rate after a change, the way Halon watches in a chat. It only reads, and goes on by the succeeded, failed, timed out or stopped route. See Watching
Wait for event, For each, Join, Run workflow	Wait for something to happen, run steps once for each item of a list, wait for every path into them, or run another workflow

On the canvas each step is marked Reads or Changes. A step that changes something, such as a restart, says how to undo it and is followed by a step that checks it worked.

Checking a change is still needed

In a run nobody pressed, one started on a schedule or by something happening, Halon first checks that the problem is still there before a step changes anything, such as whether the alert is still firing or the service still down. The check is a call to Halon, so it counts against the run’s spending ceiling.

• When the problem is still there, the step makes its change.
• When the problem has cleared, the step is skipped and the run says why, with what Halon read. The steps after it carry on.
• When Halon cannot tell, for example because the run has reached its spending ceiling, the run stops before the change.

The check is on for every step that changes something. To make a change without checking, turn off Check it is still needed first in the step’s settings. Steps that tell people or send something on, such as Post to channel, Message a person, Post incident update, Raise a notice, Send webhook and the sending step of Browser, never check first. A run you start by hand or a test run never checks.

A step that calls something outside Firefight needs it granted to the workflow, which publishing does. HTTP request has its own permission, HTTP Requests, read for GET and HEAD and create for the rest.

Asking people

Ask a person, Approval and Manual step each pause the run until someone answers. Each has Wait for in its settings, a day unless you change it, from 5 minutes to 7 days. When nobody answers in that time, the step takes its route for that: unanswered, withdrawn or not done.

Ask a person asks a member by email, an incident’s lead, whoever holds one of its roles, or whoever is on call for it. Each person gets a direct message in Slack with a button for each choice, or Answer when the question has a short form, which opens a form in Slack. The question is on the run’s page too, with the same choices or Answer. The first answer counts, from anyone asked, and in a test run you can answer it yourself.

Under Choices, write each choice as the words you want on its button, such as “Roll back now”, and the button shows exactly that. Each choice is a route out of the step, named the same way on the canvas, so the run goes the way the person picked. Changing a choice’s words later keeps its connection. Later steps read each answer of the form by its name, such as {{steps.ask.reason}}.

Approval asks named people, by email, or anyone holding the admin or owner role. The request is posted in the incident’s channel when the step names an incident, otherwise in the channel of the team that owns the workflow, or your incidents channel when it has none. People you name also get it by direct message. A role is asked in the channel only, so nobody messages every admin. The request is listed under Gateway → Approvals and shown on the run’s page with Approve and Deny. The run takes the approved or denied route.

A person you name on an Approval step can approve or deny that request from Slack, the Approvals page or the run’s page, even without the approvals permission. It lets them decide only that request. A request that a role decides needs the approvals permission, as every approval does. See Approvals.

Name an incident on an Approval step to add conditions it must still meet, under Still true, such as its stage being active. They are read again every few minutes while the request waits, and once more when someone approves. When the incident no longer meets them, or nobody decides in time, the request is withdrawn and the run takes the withdrawn route, so an approval nobody needs any more is never acted on. Its message in Slack says it was withdrawn and why.

Manual step gives a person a task, such as “Open a ticket with the payment provider”, and waits until it is done. Give it to a member by email or to an incident’s lead, who gets it by direct message, or to anyone, which posts it in the incident’s channel, or the owning team’s channel when you name no incident. The task is on the run’s page too. Whoever it is given to presses Done, and anyone in the workspace can when it is given to anyone. In a test run you can mark it done yourself.

Add a note to the step, such as “Ticket number”, to have Done ask for it. Later steps read it as {{steps.ticket.note}}, and who did it as {{steps.ticket.done_by}}. The run takes the done route, or not done when nobody marks it done in time.

A question, approval or task still waiting when its run stops is withdrawn, and its messages say so.

Publishing

Publishing makes the draft live. The Publish dialog on the canvas lists what the workflow will be able to do, such as reading the resource map or posting to a channel, and publishing grants exactly that to the workflow’s identity in the same step. Approval rules still apply to its changes, the same as for anyone else. A step that changes who can reach something or sets a secret, such as setting a GitHub Actions secret, always waits for an admin to approve it, whatever the workflow was granted. See Approvals.

When an approval rule holds publishing a workflow Halon or an agent made, it waits as a draft and is published once an admin approves, exactly as it was when it was asked about. If the draft changed since, approving is refused and says so, so nobody publishes what they did not look at.

Only an admin can publish. A member presses Ask an admin to publish instead, and the admins get a direct message in Slack with Publish and a link to the workflow, at most once a day for the same member and workflow. Admins also see waiting requests on the Workflows tab and above the canvas. The member is told the answer either way.

Once live, the switch in the canvas’s header and in the list turns it on or off.

Test runs

Press Test run on the canvas to try a draft or the live version as yourself, with what you may read. Every step that reads runs for real. Every step that would change something only shows what it would do, unless you turn it on for real in the dialog. A test run is marked as one on the Runs tab. It never counts toward Most runs per hour and never reports a failure to anyone. See Trying a workflow.

Settings

Each workflow has its own settings, under Settings on its canvas, beside What it can do, which lists what its steps need granted. Press Save settings to keep a change. Changing them is an admin’s.

Setting	What it does
Name and Description	What the workflow is called and what it does
Owning team	Where a failed run is reported. Without one, the person who published it is told in Slack
Spending ceiling per run, in dollars	The most one run may spend on AI and on its sandbox time together. Once a run reaches it, a Halon step, a sandbox step or the check before a change stops the run instead of spending more. Empty means no ceiling, so only your workspace’s AI budget limits it
Most runs per hour	100 unless you set another, from 1 to 1,000. A run a schedule, something happening or the workflow’s own address would start past it is skipped, and the workflow’s page says how many and when. A run you start by hand past it is refused with why
One live run per	Stops two runs for the same thing overlapping, such as {{inputs.service}} for one run per service at a time. Empty lets runs overlap
When a change lands in a freeze window	Stop the run, or Wait for the freeze to end. Freeze windows come from your handbook
Post a one line all clear when a run finds nothing	Off by default, so a run that finds nothing stays quiet

You can also ask Halon to change a setting, such as “raise the most runs per hour of Restart web to 250”. Halon shows you the change and makes it only once you confirm, as you, so it needs the same permission the page does and any approval rule still applies.

A workflow that has never run can be deleted there too. One that ran is turned off instead, so its runs stay readable.

How long finished runs keep their details is a workspace setting. Set it under Keep run details for on the Workspace settings page, from 1 to 400 days, 90 unless you change it. A run you pin keeps its details however old it gets.

When a run fails

A failed run is reported in the owning team’s Slack channel, or to whoever published the workflow, with Open the run. When the failure came from the step’s own settings, Halon suggests a fix once for each way it fails on the published version. Publishing a new version lets Halon look at each failure again. The fix waits on the canvas for you to accept or dismiss. Accept adds it to the draft, which you publish like any other change.

When a scheduled run cannot start because of an error in Firefight, it is tried again for 15 minutes. After that, that time is given up and shows as a failed run, reported the same way, and the next scheduled time runs as usual.

You can stop a run that is still going with Stop run in its panel on the canvas.

Who can make workflows

Every member can make workflows, test them and run published ones without a grant, so every member sees New from the library, and Use and Import a workflow on the Library tab. An admin can take that away from one member with No access on the Permissions screen, the same as asking Halon, and that member no longer sees those buttons. Restore gives it back. See Permissions.

Publishing, switching a workflow on or off, changing its settings, deleting it and taking a newer library version are an admin’s. A member asks an admin to publish instead.

Over MCP

Agents connected over MCP can use list_workflows, create_workflow and run_workflow, and update_workflow_settings sets how long runs keep their details. create_workflow publishes the workflow when the agent may publish it and grant what it needs, and otherwise leaves it as a draft and asks the admins. It can set a workflow’s most runs per hour, and list_workflows says each workflow’s and how many runs it skipped in the last week. update_workflow changes any of a workflow’s settings, the same ones its Settings page holds, as does PATCH /api/v1/workflows/:id over the API. A workflow made this way follows the same rules as one made in the dashboard. See Connect AI agents (MCP).