Connect Grafana
Firefight has a dedicated alert source for Grafana Alerting. Grafana sends a group of alerts in each request, and Firefight turns each one into its own alert that resolves when Grafana resolves it.
Set up the Firefight side
Section titled “Set up the Firefight side”Go to Settings → Alert Sources, click Add source, give it a name like “Grafana production”, and pick the Grafana provider. When you create it, Firefight opens the source’s setup. It shows the ingest URL and a Copy token button for the source’s secret token, and the same steps as this page.
Set up the Grafana side
Section titled “Set up the Grafana side”- In Grafana, open Alerting → Contact points and add a contact point with the Webhook integration.
- Paste the ingest URL as its URL.
- Under Optional Webhook settings, paste the token into Authorization Header Credentials and leave the scheme as
Bearer. - Leave Disable resolved message off, save the contact point, and use it in a notification policy.
Requests that fail the check are rejected with 401, and the source’s row in Settings → Alert Sources shows the rejection.
To have Grafana sign each request instead, paste the token as the HMAC Signature secret, keep the header X-Grafana-Alerting-Signature, and set the timestamp header to X-Grafana-Alerting-Timestamp. Firefight refuses a signed request more than five minutes old.
What Firefight reads
Section titled “What Firefight reads”Each alert gets these fields, which you can match in your routing rules.
| Field | Value |
|---|---|
title | The alertname label |
description | The summary and description annotations |
service | The service label, else job |
severity_raw | The severity label |
team | The team label |
environment | The environment label, else env |
url | The panel or dashboard the rule watches, else the alert rule in Grafana |
The alert’s title in Slack and on the dashboard links to its page in Grafana.
Firing and resolving
Section titled “Firing and resolving”Each alert in Grafana’s group is its own alert in Firefight, identified by Grafana’s fingerprint for its labels. A firing alert counts on the same Firefight alert each time Grafana repeats it, and Grafana’s resolved message resolves it.
See Grouping & deduplication for how repeat firings collapse into one alert.