Grafana and Alertmanager

Freesbe takes alerts from Grafana (Grafana alerting, in Grafana Cloud or on your own servers) and from Prometheus Alertmanager. Once an admin connects one:

• Each alert posts into the channel you pick as one card, with its severity, summary, where it’s firing (the instance, host or node label), its description and its other labels.
• The same card stays current. When the alert resolves, the card turns green and says how long it lasted, and a reply under it lets everyone following the thread know. Grafana and Alertmanager repeat an alert while it’s firing; Freesbe posts it once.
• Silence for 1 hour on the card silences the alert in Grafana or Alertmanager, when you give Freesbe an API token. The silence says who pressed it.
• A live problems board pinned in the channel lists every alert firing right now, worst first, and reports post on their own every morning or every Monday.

Grafana and Alertmanager post as their own members, Grafana and Alertmanager, so their cards are easy to tell apart from people.

Connect Grafana

Only owners and admins can connect it. You need someone who can edit contact points in Grafana.

1. Open Workspace settings → Integrations & apps → Integrations → Monitoring and click Connect under Grafana.
2. Give the connection a name, like Grafana Cloud, and pick the channel it posts to.
3. Pick which alerts to post under Alerts to post (see Severities).
4. For the Silence for 1 hour button, fill in Grafana address and API token (see Silence from the card). You can add them later.
5. Click Connect Grafana. Freesbe shows the Freesbe URL that Grafana sends alerts to. Copy it now: it’s only shown once.

Then, in Grafana:

1. Open Alerting → Contact points and click Add contact point. Name it Freesbe, pick the Webhook integration, and paste the Freesbe URL in URL. Leave the rest as it is.
2. Click Test to see a card in the channel, then Save contact point.
3. In Alerting → Notification policies, send the alerts you want to the Freesbe contact point (or pick it as the contact point on each alert rule). Keep Disable resolved message off, so cards turn green.

Connect Alertmanager

1. In Freesbe, click Connect under Alertmanager, name it (Production Prometheus), pick the channel, and click Connect Alertmanager. Copy the Freesbe URL.

2. In alertmanager.yml, add a receiver with the URL, and route alerts to it:

   receivers:
     - name: freesbe
       webhook_configs:
         - url: "https://api.freesbe.net/hooks/monitoring/alertmanager/…"
           send_resolved: true
   
   route:
     receiver: freesbe

   To send only some alerts, add the receiver under a child route with matchers, such as team = "storefront", instead of making it the default.

3. Reload Alertmanager (curl -X POST http://localhost:9093/-/reload, or restart it). The next alert that fires posts a card.

Cards from Alertmanager link to the alert’s source (Prometheus’s graph of its expression) with Open source. Add a runbook_url annotation to your rules and cards without a source link use it instead.

Severities

Freesbe reads each alert’s severity label (or priority), with the values people usually use:

Label value	On the card	Icon
critical, page, emergency, fatal, p1	Critical	🚨
high, error, major, p2	High	🔴
minor, medium, average, p3	Minor	🟠
warning, warn, p4	Warning	🟡
info, low, none, p5	Info	🔵
no severity label	High	🔴

Alerts to post leaves out alerts below the severity you pick. Their recoveries always follow the alert, so a card never stays red after the alert resolves.

The cards

Each card shows:

• The severity, and whether the alert is firing, silenced or resolved.
• The alert’s summary annotation (or its alertname), linked to the Grafana panel or the alert’s source, and where it’s firing.
• Its description annotation, or for Grafana its values, like B: 97.3.
• Its other labels, like team: storefront, and the Silence for 1 hour and Open in Grafana (or Open source) buttons.

Every alert in a notification is its own card, so a group of five alerts posts five cards. An alert that fires again after it resolved is a new card.

Silence from the card

With an API token, everyone in the channel can press Silence for 1 hour on a firing alert. Freesbe adds a silence matching all of the alert’s labels, for an hour, with a comment saying who pressed it (Silenced from Freesbe by Maya Chen). The card shows who silenced it, and the board lists it as silenced. To silence for longer or end it early, use Grafana or Alertmanager.

• Grafana: open Administration → Users and access → Service accounts, add one with the Editor role (or a role with alerting silence permissions), and click Add service account token. Fill in Grafana address (where you open Grafana, like https://northwind.grafana.net) and paste the token in API token.
• Alertmanager: Alertmanager has no sign-in of its own, so it’s usually behind a proxy that asks for one. Fill in Alertmanager address and, in API token, what the proxy accepts: a bearer token, or a user name and password as user:password.

Freesbe checks the token before saving it, and keeps it encrypted. It calls the API from the internet, so the address must be reachable over HTTPS from outside your network. If it isn’t, you still get every card; the button just isn’t shown.

The board and reports

Like Zabbix, each connection pins a live problems board in its channel, listing the alerts firing right now, worst first, with how long and who silenced them. Turn it off with Pin a live problems board in the connection’s settings.

Reports count the alerts Freesbe got over the last day or week: by severity, still firing, total firing time, the typical time to resolve, the noisiest alerts and the busiest hosts. Pick Every morning at 9:00 or Monday mornings at 9:00 under Reports. See The live problems board and Reports for more.

Questions

Does Freesbe need to reach my Grafana or Alertmanager? Only for Silence for 1 hour. They send alerts to Freesbe, so cards work with a server that can reach the internet but isn’t reachable from it.

Who can see the cards? Everyone in the channel. Use a private channel for alerts only some people should see.

Is the URL a secret? Yes: anyone with it can post alerts into the channel. If it leaks, delete the connection and connect again for a new URL.

Can I use both? Yes. Connect each tool, or the same tool several times for different channels, each with its own URL.