From 75900139ae6c81adf604412d7699fdc4c39c5f63 Mon Sep 17 00:00:00 2001 From: Pepe Cano <825430+ppcano@users.noreply.github.com> Date: Wed, 25 Sep 2024 12:44:17 +0200 Subject: [PATCH] Alerting docs: update `Configure Alertmanagers` (#93712) * Alerting docs: update `Configure Alertmanagers` - clarify alertmanager types - specify that each alertmanager manages its own alerting resources * Update docs/sources/alerting/set-up/configure-alertmanager/index.md Co-authored-by: brendamuir <100768211+brendamuir@users.noreply.github.com> * Update docs/sources/alerting/set-up/configure-alertmanager/index.md Co-authored-by: brendamuir <100768211+brendamuir@users.noreply.github.com> * Update docs/sources/alerting/set-up/configure-alertmanager/index.md Co-authored-by: brendamuir <100768211+brendamuir@users.noreply.github.com> * use `Alertmanager` - capitalize the first letter * Small copy changes * Minor cosmetic updates to `Add an Alertmanager` section * Update docs/sources/alerting/set-up/configure-alertmanager/index.md Co-authored-by: brendamuir <100768211+brendamuir@users.noreply.github.com> --------- Co-authored-by: brendamuir <100768211+brendamuir@users.noreply.github.com> --- .../set-up/configure-alertmanager/index.md | 55 +++++++++++-------- 1 file changed, 33 insertions(+), 22 deletions(-) diff --git a/docs/sources/alerting/set-up/configure-alertmanager/index.md b/docs/sources/alerting/set-up/configure-alertmanager/index.md index 86956ad6837..52ef493b7c4 100644 --- a/docs/sources/alerting/set-up/configure-alertmanager/index.md +++ b/docs/sources/alerting/set-up/configure-alertmanager/index.md @@ -34,17 +34,32 @@ Grafana Alerting is based on the architecture of the Prometheus alerting system. {{< figure src="/media/docs/alerting/alerting-alertmanager-architecture.png" max-width="750px" alt="A diagram with the alert generator and alert manager architecture" >}} -**Grafana Alertmanager** +This architecture decouples alert rule evaluation from notification handling, allowing alerts to be forwarded to other Alertmanagers. -Grafana has its own built-in Alertmanager, referred to as "Grafana" in the user interface. It is the default Alertmanager and can only handle Grafana-managed alerts. +Grafana can use different Alertmanagers. It’s important to note that each Alertmanager manages its own independent alerting resources, such as: -**Cloud Alertmanager** +- Contact points and notification templates +- Notification policies and mute timings +- Silences +- Active notifications -Each Grafana Cloud instance comes preconfigured with an additional Alertmanager (`grafanacloud-STACK_NAME-ngalertmanager`) from the Mimir (Prometheus) instance running in the Grafana Cloud Stack. The Cloud Alertmanager can handle both Grafana-managed and data source-managed alerts. +Use the `Choose Alertmanager` on these pages to switch between Alertmanagers. -**Other Alertmanagers** +{{< figure src="/media/docs/alerting/alerting-choose-alertmanager.png" max-width="750px" alt="A screenshot choosing an Alertmanager in the notification policies UI" >}} -Grafana Alerting also supports sending alerts to other alertmanagers, such as the [Prometheus Alertmanager](https://prometheus.io/docs/alerting/latest/alertmanager/), which can handle Grafana-managed alerts and data sources-managed alerts such as alerts from Loki, Mimir, and Prometheus. +## Types of Alertmanagers in Grafana + +Grafana can be configured to manage both Grafana-managed and data source-managed alerts using various Alertmanagers, depending on your infrastructure and alerting requirements. + +- **Grafana Alertmanager**: Grafana has its own built-in Alertmanager, referred to as "Grafana" in the user interface. It is the default Alertmanager and can only handle Grafana-managed alerts. + +- **Cloud Alertmanager**: Each Grafana Cloud instance comes preconfigured with an additional Alertmanager (`grafanacloud-STACK_NAME-ngalertmanager`) from the Mimir (Prometheus) instance running in the Grafana Cloud Stack. + + The Cloud Alertmanager is available exclusively in Grafana Cloud and can handle both Grafana-managed and data source-managed alerts. + + Some Grafana Cloud services, such as **Kubernetes Monitoring** and **Synthetic Monitoring** use the Cloud Alertmanager to create and manage alerts. + +- **Other Alertmanagers**: Grafana Alerting also supports sending alerts to other Alertmanagers, such as the [Prometheus Alertmanager](https://prometheus.io/docs/alerting/latest/alertmanager/), which can handle Grafana-managed alerts and data sources-managed alerts such as alerts from Loki, Mimir, and Prometheus. You can use a combination of Alertmanagers. The decision often depends on your alerting setup and where your alerts are being generated. Here are two examples of when you may want to add an Alertmanager and send your alerts there instead of using the built-in Grafana Alertmanager. @@ -56,36 +71,32 @@ You can use a combination of Alertmanagers. The decision often depends on your a From Grafana, you can configure and administer your own Alertmanager to receive Grafana alerts. -{{% admonition type="note" %}} -Grafana Alerting does not support sending alerts to the AWS Managed Service for Prometheus due to the lack of sigv4 support in Prometheus. -{{% /admonition %}} +After adding an Alertmanager, you can use the Grafana Alerting UI to manage notification policies, contact points, and other alerting resources from within Grafana, with support for HTTP basic authentication credentials. -After you have added the Alertmanager, you can use the Grafana Alerting UI to manage silences, contact points, and notification policies. A drop-down option in these pages allows you to switch between alertmanagers. - -{{< figure src="/media/docs/alerting/alerting-choose-alertmanager.png" max-width="750px" alt="A screenshot choosing an Alertmanager in the notification policies UI" >}} - -Alertmanagers should now be configured as data sources using Grafana Configuration from the main Grafana navigation menu. This enables you to manage the contact points and notification policies of external alertmanagers from within Grafana and also encrypts HTTP basic authentication credentials. - -To add an Alertmanager, complete the following steps. +Alertmanagers should be configured as data sources using Grafana Configuration from the main Grafana navigation menu. To add an Alertmanager, complete the following steps. 1. Click **Connections** in the left-side menu. -2. On the Connections page, search for `Alertmanager`. -3. Click the **Create a new data source** button. +1. On the Connections page, search for `Alertmanager`. +1. Click the **Create a new data source** button. If you don't see this button, you may need to install the plugin, relaunch your Cloud instance, and then repeat steps 1 and 2. -4. Fill out the fields on the page, as required. +1. Fill out the fields on the page, as required. If you are provisioning your data source, set the flag `handleGrafanaManagedAlerts` in the `jsonData` field to `true` to send Grafana-managed alerts to this Alertmanager. **Note:** Prometheus, Grafana Mimir, and Cortex implementations of Alertmanager are supported. For Prometheus, contact points and notification policies are read-only in the Grafana Alerting UI. -5. Click **Save & test**. +1. Click **Save & test**. + +{{% admonition type="note" %}} +Grafana Alerting does not support sending alerts to the AWS Managed Service for Prometheus due to the lack of sigv4 support in Prometheus. +{{% /admonition %}} + +## Manage Alertmanager configurations -{{< admonition type="note" >}} On the Settings page, you can manage your Alertmanager configurations and configure where Grafana-managed alert instances are forwarded. - Manage which Alertmanagers receive alert instances from Grafana-managed rules without navigating and editing data sources. - Manage version snapshots for the built-in Alertmanager, which allows administrators to roll back unintentional changes or mistakes in the Alertmanager configuration. - Compare the historical snapshot with the latest configuration to see which changes were made. - {{< /admonition >}}