mirror of
https://github.com/grafana/grafana.git
synced 2025-02-25 18:55:37 -06:00
Alerting docs: Provision alerting resources
updates (#82221)
* Alert provisioning: initial restructuring * Fix products labels * Restructure `Import and export Grafana Alerting resources` * Change URL to `export-alerting-resources` * Complete `Export alerting resources` * Export alerting resources * Update `configuration files` provisioning * Terraform Provisioning * Change to `Provision/Import/Export` terms and some notes * Replace `config` to `configuration` * Set (menu)Title `Export alerting resources` * Minor change on note about `Export Alerting endpoints` * Fix `doc-validator` issues * Fix grammar * Update docs/sources/alerting/set-up/provision-alerting-resources/terraform-provisioning/index.md Co-authored-by: brendamuir <100768211+brendamuir@users.noreply.github.com> * Update docs/sources/alerting/set-up/provision-alerting-resources/terraform-provisioning/index.md Co-authored-by: brendamuir <100768211+brendamuir@users.noreply.github.com> * Fix numbered lists and `Note:` without admonition * Convert text-based notes (`Note:`) to `admonition` blocks * Replace text-based `Note:` with adminitions * Remove `file-provisioning` grafana-cloud links * Update `Export alerting resources` intro * nitpicky format order --------- Co-authored-by: brendamuir <100768211+brendamuir@users.noreply.github.com>
This commit is contained in:
parent
815e61258c
commit
9c92329bee
@ -54,7 +54,7 @@ Grafana Alerting supports many additional configuration options, from configurin
|
||||
|
||||
The following topics provide you with advanced configuration options for Grafana Alerting.
|
||||
|
||||
- [Provision alert rules using file provisioning][file-provisioning]
|
||||
- [Provision alert rules using file provisioning](/docs/grafana/<GRAFANA_VERSION>/alerting/set-up/provision-alerting-resources/file-provisioning)
|
||||
- [Provision alert rules using Terraform][terraform-provisioning]
|
||||
- [Add an external Alertmanager][configure-alertmanager]
|
||||
- [Configure high availability][configure-high-availability]
|
||||
@ -72,9 +72,6 @@ The following topics provide you with advanced configuration options for Grafana
|
||||
[data-source-management]: "/docs/grafana/ -> /docs/grafana/<GRAFANA VERSION>/administration/data-source-management"
|
||||
[data-source-management]: "/docs/grafana-cloud/ -> /docs/grafana/<GRAFANA VERSION>/administration/data-source-management"
|
||||
|
||||
[file-provisioning]: "/docs/grafana/ -> /docs/grafana/<GRAFANA VERSION>/alerting/set-up/provision-alerting-resources/file-provisioning"
|
||||
[file-provisioning]: "/docs/grafana-cloud/ -> /docs/grafana-cloud/alerting-and-irm/alerting/set-up/provision-alerting-resources/file-provisioning"
|
||||
|
||||
[terraform-provisioning]: "/docs/grafana/ -> /docs/grafana/<GRAFANA VERSION>/alerting/set-up/provision-alerting-resources/terraform-provisioning"
|
||||
[terraform-provisioning]: "/docs/grafana-cloud/ -> /docs/grafana-cloud/alerting-and-irm/alerting/set-up/provision-alerting-resources/terraform-provisioning"
|
||||
{{% /docs/reference %}}
|
||||
|
@ -2,7 +2,7 @@
|
||||
aliases:
|
||||
- ../provision-alerting-resources/
|
||||
canonical: https://grafana.com/docs/grafana/latest/alerting/set-up/provision-alerting-resources/
|
||||
description: Import and export alerting resources
|
||||
description: Provision alerting resources
|
||||
keywords:
|
||||
- grafana
|
||||
- alerting
|
||||
@ -14,11 +14,11 @@ labels:
|
||||
- cloud
|
||||
- enterprise
|
||||
- oss
|
||||
title: Import and export Grafana Alerting resources
|
||||
title: Provision Alerting resources
|
||||
weight: 300
|
||||
---
|
||||
|
||||
# Import and export Grafana Alerting resources
|
||||
# Provision Alerting resources
|
||||
|
||||
Alerting infrastructure is often complex, with many pieces of the pipeline that often live in different places. Scaling this across multiple teams and organizations is an especially challenging task. Importing and exporting (or provisioning) your alerting resources in Grafana Alerting makes this process easier by enabling you to create, manage, and maintain your alerting data in a way that best suits your organization.
|
||||
|
||||
@ -26,116 +26,55 @@ You can import alert rules, contact points, notification policies, mute timings,
|
||||
|
||||
You cannot edit imported alerting resources in the Grafana UI in the same way as alerting resources that were not imported. You can only edit imported contact points, notification policies, templates, and mute timings in the source where they were created. For example, if you manage your alerting resources using files from disk, you cannot edit the data in Terraform or from within Grafana.
|
||||
|
||||
## Import alerting resources
|
||||
|
||||
Choose from the options below to import (or provision) your Grafana Alerting resources.
|
||||
|
||||
1. [Use configuration files to provision your alerting resources](/docs/grafana/<GRAFANA_VERSION>/alerting/set-up/provision-alerting-resources/file-provisioning), such as alert rules and contact points, through files on disk.
|
||||
|
||||
{{< admonition type="note" >}}
|
||||
File provisioning is not available in Grafana Cloud instances.
|
||||
{{< /admonition >}}
|
||||
|
||||
1. Use [Terraform to provision alerting resources][alerting_tf_provisioning].
|
||||
|
||||
1. Use the [Alerting provisioning HTTP API][alerting_http_provisioning] to manage alerting resources.
|
||||
|
||||
{{< admonition type="note" >}}
|
||||
The JSON output from the majority of Alerting HTTP endpoints isn't compatible for provisioning via configuration files.
|
||||
Instead, use the [Export Alerting endpoints](/docs/grafana/<GRAFANA_VERSION>/alerting/set-up/provision-alerting-resources/export-alerting-resources#export-api-endpoints) to return or download the alerting resources in provisioning format.
|
||||
{{< /admonition >}}
|
||||
|
||||
## Export alerting resources
|
||||
|
||||
You can export both manually created and provisioned alerting resources. For more information, refer to [Export alerting resources][alerting_export].
|
||||
|
||||
To modify imported alert rules, you can use the **Modify export** feature to edit and then export.
|
||||
|
||||
Choose from the options below to import your Grafana Alerting resources.
|
||||
## View provisioned alerting resources
|
||||
|
||||
1. Use file provisioning to manage your Grafana Alerting resources, such as alert rules and contact points, through files on disk.
|
||||
To view your provisioned resources in Grafana, complete the following steps.
|
||||
|
||||
{{% admonition type="note" %}}
|
||||
File provisioning is not available in Grafana Cloud instances.
|
||||
{{% /admonition %}}
|
||||
1. Open your Grafana instance.
|
||||
1. Navigate to Alerting.
|
||||
1. Click an alerting resource folder, for example, Alert rules.
|
||||
|
||||
2. Use the Alerting Provisioning HTTP API.
|
||||
|
||||
For more information on the Alerting Provisioning HTTP API, refer to [Alerting provisioning HTTP API][alerting_provisioning].
|
||||
|
||||
Here is a ready-to-use template for alert rules:
|
||||
|
||||
#### Alert rules template
|
||||
|
||||
```
|
||||
{
|
||||
"title": "TEST-API_1",
|
||||
"ruleGroup": "API",
|
||||
"folderUID": "FOLDER",
|
||||
"noDataState": "OK",
|
||||
"execErrState": "OK",
|
||||
"for": "5m",
|
||||
"orgId": 1,
|
||||
"uid": "",
|
||||
"condition": "B",
|
||||
"annotations": {
|
||||
"summary": "test_api_1"
|
||||
},
|
||||
"labels": {
|
||||
"API": "test1"
|
||||
},
|
||||
"data": [
|
||||
{
|
||||
"refId": "A",
|
||||
"queryType": "",
|
||||
"relativeTimeRange": {
|
||||
"from": 600,
|
||||
"to": 0
|
||||
},
|
||||
"datasourceUid": " XXXXXXXXX-XXXXXXXXX-XXXXXXXXXX",
|
||||
"model": {
|
||||
"expr": "up",
|
||||
"hide": false,
|
||||
"intervalMs": 1000,
|
||||
"maxDataPoints": 43200,
|
||||
"refId": "A"
|
||||
}
|
||||
},
|
||||
{
|
||||
"refId": "B",
|
||||
"queryType": "",
|
||||
"relativeTimeRange": {
|
||||
"from": 0,
|
||||
"to": 0
|
||||
},
|
||||
"datasourceUid": "-100",
|
||||
"model": {
|
||||
"conditions": [
|
||||
{
|
||||
"evaluator": {
|
||||
"params": [
|
||||
6
|
||||
],
|
||||
"type": "gt"
|
||||
},
|
||||
"operator": {
|
||||
"type": "and"
|
||||
},
|
||||
"query": {
|
||||
"params": [
|
||||
"A"
|
||||
]
|
||||
},
|
||||
"reducer": {
|
||||
"params": [],
|
||||
"type": "last"
|
||||
},
|
||||
"type": "query"
|
||||
}
|
||||
],
|
||||
"datasource": {
|
||||
"type": "__expr__",
|
||||
"uid": "-100"
|
||||
},
|
||||
"hide": false,
|
||||
"intervalMs": 1000,
|
||||
"maxDataPoints": 43200,
|
||||
"refId": "B",
|
||||
"type": "classic_conditions"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
3. Use [Terraform](https://www.terraform.io/).
|
||||
Provisioned resources are labeled **Provisioned**, so that it is clear that they were not created manually.
|
||||
|
||||
**Useful Links:**
|
||||
|
||||
[Grafana provisioning][provisioning]
|
||||
|
||||
[Grafana Alerting provisioning API][alerting_provisioning]
|
||||
|
||||
{{% docs/reference %}}
|
||||
[alerting_provisioning]: "/docs/grafana/ -> /docs/grafana/<GRAFANA VERSION>/developers/http_api/alerting_provisioning"
|
||||
[alerting_provisioning]: "/docs/grafana-cloud/ -> /docs/grafana/<GRAFANA VERSION>/developers/http_api/alerting_provisioning"
|
||||
[alerting_tf_provisioning]: "/docs/grafana/ -> /docs/grafana/<GRAFANA VERSION>/alerting/set-up/provision-alerting-resources/terraform-provisioning"
|
||||
[alerting_tf_provisioning]: "/docs/grafana-cloud/ -> /docs/grafana/<GRAFANA VERSION>/alerting/set-up/provision-alerting-resources/terraform-provisioning"
|
||||
[alerting_http_provisioning]: "/docs/grafana/ -> /docs/grafana/<GRAFANA VERSION>/alerting/set-up/provision-alerting-resources/http-api-provisioning"
|
||||
[alerting_http_provisioning]: "/docs/grafana-cloud/ -> /docs/grafana/<GRAFANA VERSION>/alerting/set-up/provision-alerting-resources/http-api-provisioning"
|
||||
[alerting_export]: "/docs/grafana/ -> /docs/grafana/<GRAFANA VERSION>/alerting/set-up/provision-alerting-resources/export-alerting-resources"
|
||||
[alerting_export]: "/docs/grafana-cloud/ -> /docs/grafana/<GRAFANA VERSION>/alerting/set-up/provision-alerting-resources/export-alerting-resources"
|
||||
|
||||
[alerting_export_http]: "/docs/grafana/ -> /docs/grafana/<GRAFANA VERSION>/alerting/set-up/provision-alerting-resources/export-alerting-resources#export-api-endpoints"
|
||||
[alerting_export_http]: "/docs/grafana-cloud/ -> /docs/grafana/<GRAFANA VERSION>/alerting/set-up/provision-alerting-resources/export-alerting-resources#export-api-endpoints"
|
||||
|
||||
[provisioning]: "/docs/grafana/ -> /docs/grafana/<GRAFANA VERSION>/administration/provisioning"
|
||||
[provisioning]: "/docs/grafana-cloud/ -> /docs/grafana/<GRAFANA VERSION>/administration/provisioning"
|
||||
|
@ -0,0 +1,106 @@
|
||||
---
|
||||
aliases:
|
||||
- ../../provision-alerting-resources/view-provisioned-resources/
|
||||
- ./view-provisioned-resources/
|
||||
canonical: https://grafana.com/docs/grafana/latest/alerting/set-up/provision-alerting-resources/export-alerting-resources/
|
||||
description: Export alerting resources in Grafana
|
||||
keywords:
|
||||
- grafana
|
||||
- alerting
|
||||
- alerting resources
|
||||
- provisioning
|
||||
labels:
|
||||
products:
|
||||
- cloud
|
||||
- enterprise
|
||||
- oss
|
||||
title: Export alerting resources
|
||||
weight: 300
|
||||
---
|
||||
|
||||
# Export alerting resources
|
||||
|
||||
Export your alerting resources, such as alert rules, contact points, and notification policies for provisioning, automatically importing single folders and single groups.
|
||||
|
||||
The export options listed below enable you to download resources in YAML, JSON, or Terraform format, facilitating their provisioning through [configuration files](/docs/grafana/<GRAFANA_VERSION>/alerting/set-up/provision-alerting-resources/file-provisioning) or [Terraform][alerting_tf_provisioning].
|
||||
|
||||
## Export alert rules
|
||||
|
||||
To export alert rules from the Grafana UI, complete the following steps.
|
||||
|
||||
1. Click **Alerts & IRM** -> **Alert rules**.
|
||||
1. To export all Grafana-managed rules, click **Export rules**.
|
||||
1. To export a folder, change the **View as** to **List**.
|
||||
1. Select the folder you want to export and click the **Export rules folder** icon.
|
||||
1. To export a group, change the **View as** to **Grouped**.
|
||||
1. Find the group you want to export and click the **Export rule group** icon.
|
||||
1. Choose the format to export in.
|
||||
|
||||
The exported rule data appears in different formats - YAML, JSON, Terraform.
|
||||
|
||||
1. Click **Copy Code** or **Download**.
|
||||
|
||||
a. Choose **Copy Code** to go to an existing file and paste in the code.
|
||||
|
||||
b. Choose **Download** to download a file with the exported data.
|
||||
|
||||
## Modify and export alert rules without saving changes
|
||||
|
||||
Use the **Modify export** mode to edit and export an alert rule without updating it.
|
||||
|
||||
{{% admonition type="note" %}} This feature is for Grafana-managed alert rules only. It is available to Admin, Viewer, and Editor roles. {{% /admonition %}}
|
||||
|
||||
To export a modified alert rule without saving the modifications, complete the following steps from the Grafana UI.
|
||||
|
||||
1. Click **Alerts & IRM** -> **Alert rules**.
|
||||
1. Locate the alert rule you want to edit and click **More** -> **Modify Export** to open the Alert Rule form.
|
||||
1. From the Alert Rule form, edit the fields you want to change. Changes made are not applied to the alert rule.
|
||||
1. Click **Export**.
|
||||
1. Choose the format to export in.
|
||||
|
||||
The exported rule data appears in different formats - YAML, JSON, Terraform.
|
||||
|
||||
1. Click **Copy Code** or **Download**.
|
||||
|
||||
a. Choose **Copy Code** to go to an existing file and paste in the code.
|
||||
|
||||
b. Choose **Download** to download a file with the exported data.
|
||||
|
||||
## Export API endpoints
|
||||
|
||||
You can also use the **Alerting provisioning HTTP API** to export alerting resources in YAML or JSON formats for provisioning.
|
||||
|
||||
Note that most Alerting endpoints return a JSON format that is not compatible for provisioning via configuration files, except the ones listed below.
|
||||
|
||||
| Method | URI | Summary |
|
||||
| ------ | ---------------------------------------------------------------- | ---------------------------------------------------------------------------------------- |
|
||||
| GET | /api/v1/provisioning/alert-rules/:uid/export | [Export an alert rule in provisioning file format.][export_rule] |
|
||||
| GET | /api/v1/provisioning/folder/:folderUid/rule-groups/:group/export | [Export an alert rule group in provisioning file format.][export_rule_group] |
|
||||
| GET | /api/v1/provisioning/alert-rules/export | [Export all alert rules in provisioning file format.][export_rules] |
|
||||
| GET | /api/v1/provisioning/contact-points/export | [Export all contact points in provisioning file format.][export_contacts] |
|
||||
| GET | /api/v1/provisioning/policies/export | [Export the notification policy tree in provisioning file format.][export_notifications] |
|
||||
|
||||
These endpoints accept a `download` parameter to download a file containing the exported resources.
|
||||
|
||||
{{% docs/reference %}}
|
||||
[alerting_tf_provisioning]: "/docs/grafana/ -> /docs/grafana/<GRAFANA VERSION>/alerting/set-up/provision-alerting-resources/terraform-provisioning"
|
||||
[alerting_tf_provisioning]: "/docs/grafana-cloud/ -> /docs/grafana/<GRAFANA VERSION>/alerting/set-up/provision-alerting-resources/terraform-provisioning"
|
||||
|
||||
[alerting_http_provisioning]: "/docs/grafana/ -> /docs/grafana/<GRAFANA VERSION>/alerting/set-up/provision-alerting-resources/http-api-provisioning"
|
||||
[alerting_http_provisioning]: "/docs/grafana-cloud/ -> /docs/grafana/<GRAFANA VERSION>/alerting/set-up/provision-alerting-resources/http-api-provisioning"
|
||||
|
||||
[export_rule]: "/docs/grafana/ -> /docs/grafana/<GRAFANA VERSION>/alerting/set-up/provision-alerting-resources/http-api-provisioning/#span-idroute-get-alert-rule-exportspan-export-an-alert-rule-in-provisioning-file-format-_routegetalertruleexport_"
|
||||
[export_rule]: "/docs/grafana-cloud/ -> /docs/grafana/<GRAFANA VERSION>/alerting/set-up/provision-alerting-resources/http-api-provisioning/#span-idroute-get-alert-rule-exportspan-export-an-alert-rule-in-provisioning-file-format-_routegetalertruleexport_"
|
||||
|
||||
[export_rule_group]: "/docs/grafana/ -> /docs/grafana/<GRAFANA VERSION>/alerting/set-up/provision-alerting-resources/http-api-provisioning/#span-idroute-get-alert-rule-group-exportspan-export-an-alert-rule-group-in-provisioning-file-format-_routegetalertrulegroupexport_"
|
||||
[export_rule_group]: "/docs/grafana-cloud/ -> /docs/grafana/<GRAFANA VERSION>/alerting/set-up/provision-alerting-resources/http-api-provisioning/#span-idroute-get-alert-rule-group-exportspan-export-an-alert-rule-group-in-provisioning-file-format-_routegetalertrulegroupexport_"
|
||||
|
||||
[export_rules]: "/docs/grafana/ -> /docs/grafana/<GRAFANA VERSION>/alerting/set-up/provision-alerting-resources/http-api-provisioning/#span-idroute-get-alert-rules-exportspan-export-all-alert-rules-in-provisioning-file-format-_routegetalertrulesexport_"
|
||||
[export_rules]: "/docs/grafana-cloud/ -> /docs/grafana/<GRAFANA VERSION>/alerting/set-up/provision-alerting-resources/http-api-provisioning/#span-idroute-get-alert-rules-exportspan-export-all-alert-rules-in-provisioning-file-format-_routegetalertrulesexport_"
|
||||
|
||||
[export_contacts]: "/docs/grafana/ -> /docs/grafana/<GRAFANA VERSION>/alerting/set-up/provision-alerting-resources/http-api-provisioning/#span-idroute-get-contactpoints-exportspan-export-all-contact-points-in-provisioning-file-format-_routegetcontactpointsexport_"
|
||||
[export_contacts]: "/docs/grafana-cloud/ -> /docs/grafana/<GRAFANA VERSION>/alerting/set-up/provision-alerting-resources/http-api-provisioning/#span-idroute-get-contactpoints-exportspan-export-all-contact-points-in-provisioning-file-format-_routegetcontactpointsexport_"
|
||||
|
||||
[export_notifications]: "/docs/grafana/ -> /docs/grafana/<GRAFANA VERSION>/alerting/set-up/provision-alerting-resources/http-api-provisioning/#span-idroute-get-policy-tree-exportspan-export-the-notification-policy-tree-in-provisioning-file-format-_routegetpolicytreeexport_"
|
||||
[export_notifications]: "/docs/grafana-cloud/ -> /docs/grafana/<GRAFANA VERSION>/alerting/set-up/provision-alerting-resources/http-api-provisioning/#span-idroute-get-policy-tree-exportspan-export-the-notification-policy-tree-in-provisioning-file-format-_routegetpolicytreeexport_"
|
||||
{{% /docs/reference %}}
|
@ -11,14 +11,14 @@ keywords:
|
||||
- provisioning
|
||||
labels:
|
||||
products:
|
||||
- cloud
|
||||
- enterprise
|
||||
- oss
|
||||
title: Use file provisioning to manage alerting resources
|
||||
menuTitle: Use configuration files to provision
|
||||
title: Use configuration files to provision alerting resources
|
||||
weight: 100
|
||||
---
|
||||
|
||||
## Use file provisioning to manage alerting resources
|
||||
# Use configuration files to provision alerting resources
|
||||
|
||||
Manage your alerting resources using files from disk. When you start Grafana, the data from these files is created in your Grafana system. Grafana adds any new resources you created, updates any that you changed, and deletes old ones.
|
||||
|
||||
@ -26,26 +26,28 @@ Arrange your files in a directory in a way that best suits your use case. For ex
|
||||
|
||||
Details on how to set up the files and which fields are required for each object are listed below depending on which resource you are provisioning.
|
||||
|
||||
**Note:**
|
||||
For a complete guide about how Grafana provisions resources, refer to the [Provision Grafana][provisioning] documentation.
|
||||
|
||||
Importing takes place during the initial set up of your Grafana system, but you can re-run it at any time using the [Grafana Admin API][reload-provisioning-configurations].
|
||||
{{< admonition type="note" >}}
|
||||
|
||||
### Import alert rules
|
||||
- You cannot edit provisioned resources from files in Grafana. You can only change the resource properties by changing the provisioning file and restarting Grafana or carrying out a hot reload. This prevents changes being made to the resource that would be overwritten if a file is provisioned again or a hot reload is carried out.
|
||||
|
||||
- Importing takes place during the initial set up of your Grafana system, but you can re-run it at any time using the [Grafana Admin API](/docs/grafana/<GRAFANA_VERSION>/developers/http_api/admin#reload-provisioning-configurations).
|
||||
|
||||
- Importing an existing alerting resource results in a conflict. First, when present, remove the resources you plan to import.
|
||||
{{< /admonition >}}
|
||||
|
||||
## Import alert rules
|
||||
|
||||
Create or delete alert rules in your Grafana instance(s).
|
||||
|
||||
1. Create alert rules in Grafana.
|
||||
1. Use the [Alerting provisioning API][alerting_provisioning] export endpoints to download a provisioning file for your alert rules.
|
||||
1. Copy the contents into a YAML or JSON configuration file in the default provisioning directory or in your configured directory.
|
||||
1. [Export][alerting_export] and download a provisioning file for your alert rules.
|
||||
1. Copy the contents into a YAML or JSON configuration file in the `provisioning/alerting` directory.
|
||||
|
||||
Example configuration files can be found below.
|
||||
|
||||
1. Ensure that your files are in the right directory on the node running the Grafana server, so that they deploy alongside your Grafana instance(s).
|
||||
1. Delete the alert rules in Grafana that are going to be imported.
|
||||
|
||||
**Note:**
|
||||
|
||||
If you do not delete the alert rule, it will clash with the imported alert rule once uploaded.
|
||||
1. Add the file(s) to your GitOps workflow, so that they deploy alongside your Grafana instance(s).
|
||||
|
||||
Here is an example of a configuration file for creating alert rules.
|
||||
|
||||
@ -134,17 +136,17 @@ deleteRules:
|
||||
uid: my_id_1
|
||||
```
|
||||
|
||||
### Import contact points
|
||||
## Import contact points
|
||||
|
||||
Create or delete contact points in your Grafana instance(s).
|
||||
|
||||
1. Create a contact point in Grafana.
|
||||
1. Use the [Alerting provisioning API][alerting_provisioning] export endpoints to download a provisioning file for your contact point.
|
||||
1. Copy the contents into a YAML or JSON configuration file in the default provisioning directory or in your configured directory.
|
||||
1. [Export][alerting_export] and download a provisioning file for your contact point.
|
||||
1. Copy the contents into a YAML or JSON configuration file in the `provisioning/alerting` directory.
|
||||
|
||||
Example configuration files can be found below.
|
||||
|
||||
1. Ensure that your files are in the right directory on the node running the Grafana server, so that they deploy alongside your Grafana instance(s).
|
||||
1. Add the file(s) to your GitOps workflow, so that they deploy alongside your Grafana instance(s).
|
||||
|
||||
Here is an example of a configuration file for creating contact points.
|
||||
|
||||
@ -184,12 +186,14 @@ deleteContactPoints:
|
||||
uid: first_uid
|
||||
```
|
||||
|
||||
#### Settings
|
||||
### Settings
|
||||
|
||||
Here are some examples of settings you can use for the different
|
||||
contact point integrations.
|
||||
|
||||
##### Alertmanager
|
||||
{{< collapse title="Alertmanager" >}}
|
||||
|
||||
#### Alertmanager
|
||||
|
||||
```yaml
|
||||
type: prometheus-alertmanager
|
||||
@ -202,7 +206,11 @@ settings:
|
||||
basicAuthPassword: abc123
|
||||
```
|
||||
|
||||
##### DingDing
|
||||
{{< /collapse >}}
|
||||
|
||||
{{< collapse title="DingDing" >}}
|
||||
|
||||
#### DingDing
|
||||
|
||||
```yaml
|
||||
type: dingding
|
||||
@ -216,7 +224,11 @@ settings:
|
||||
{{ template "default.message" . }}
|
||||
```
|
||||
|
||||
##### Discord
|
||||
{{< /collapse >}}
|
||||
|
||||
{{< collapse title="Discord" >}}
|
||||
|
||||
#### Discord
|
||||
|
||||
```yaml
|
||||
type: discord
|
||||
@ -232,7 +244,11 @@ settings:
|
||||
{{ template "default.message" . }}
|
||||
```
|
||||
|
||||
##### E-Mail
|
||||
{{< /collapse >}}
|
||||
|
||||
{{< collapse title="E-Mail" >}}
|
||||
|
||||
#### E-Mail
|
||||
|
||||
```yaml
|
||||
type: email
|
||||
@ -248,7 +264,11 @@ settings:
|
||||
{{ template "default.title" . }}
|
||||
```
|
||||
|
||||
##### Google Chat
|
||||
{{< /collapse >}}
|
||||
|
||||
{{< collapse title="Google Chat" >}}
|
||||
|
||||
#### Google Chat
|
||||
|
||||
```yaml
|
||||
type: googlechat
|
||||
@ -260,7 +280,11 @@ settings:
|
||||
{{ template "default.message" . }}
|
||||
```
|
||||
|
||||
##### Kafka
|
||||
{{< /collapse >}}
|
||||
|
||||
{{< collapse title="Kafka" >}}
|
||||
|
||||
#### Kafka
|
||||
|
||||
```yaml
|
||||
type: kafka
|
||||
@ -271,7 +295,11 @@ settings:
|
||||
kafkaTopic: topic1
|
||||
```
|
||||
|
||||
##### LINE
|
||||
{{< /collapse >}}
|
||||
|
||||
{{< collapse title="LINE" >}}
|
||||
|
||||
#### LINE
|
||||
|
||||
```yaml
|
||||
type: line
|
||||
@ -280,7 +308,11 @@ settings:
|
||||
token: xxx
|
||||
```
|
||||
|
||||
##### Microsoft Teams
|
||||
{{< /collapse >}}
|
||||
|
||||
{{< collapse title="Microsoft Teams" >}}
|
||||
|
||||
#### Microsoft Teams
|
||||
|
||||
```yaml
|
||||
type: teams
|
||||
@ -297,7 +329,11 @@ settings:
|
||||
{{ template "default.message" . }}
|
||||
```
|
||||
|
||||
##### OpsGenie
|
||||
{{< /collapse >}}
|
||||
|
||||
{{< collapse title="OpsGenie" >}}
|
||||
|
||||
#### OpsGenie
|
||||
|
||||
```yaml
|
||||
type: opsgenie
|
||||
@ -319,7 +355,11 @@ settings:
|
||||
sendTagsAs: both
|
||||
```
|
||||
|
||||
##### PagerDuty
|
||||
{{< /collapse >}}
|
||||
|
||||
{{< collapse title="PagerDuty" >}}
|
||||
|
||||
#### PagerDuty
|
||||
|
||||
```yaml
|
||||
type: pagerduty
|
||||
@ -339,7 +379,11 @@ settings:
|
||||
{{ template "default.message" . }}
|
||||
```
|
||||
|
||||
##### Pushover
|
||||
{{< /collapse >}}
|
||||
|
||||
{{< collapse title="Pushover" >}}
|
||||
|
||||
#### Pushover
|
||||
|
||||
```yaml
|
||||
type: pushover
|
||||
@ -367,7 +411,11 @@ settings:
|
||||
{{ template "default.message" . }}
|
||||
```
|
||||
|
||||
##### Slack
|
||||
{{< /collapse >}}
|
||||
|
||||
{{< collapse title="Slack" >}}
|
||||
|
||||
#### Slack
|
||||
|
||||
```yaml
|
||||
type: slack
|
||||
@ -399,7 +447,11 @@ settings:
|
||||
{{ template "slack.default.text" . }}
|
||||
```
|
||||
|
||||
##### Sensu Go
|
||||
{{< /collapse >}}
|
||||
|
||||
{{< collapse title="Sensu Go" >}}
|
||||
|
||||
#### Sensu Go
|
||||
|
||||
```yaml
|
||||
type: sensugo
|
||||
@ -421,7 +473,11 @@ settings:
|
||||
{{ template "default.message" . }}
|
||||
```
|
||||
|
||||
##### Telegram
|
||||
{{< /collapse >}}
|
||||
|
||||
{{< collapse title="Telegram" >}}
|
||||
|
||||
#### Telegram
|
||||
|
||||
```yaml
|
||||
type: telegram
|
||||
@ -435,7 +491,11 @@ settings:
|
||||
{{ template "default.message" . }}
|
||||
```
|
||||
|
||||
##### Threema Gateway
|
||||
{{< /collapse >}}
|
||||
|
||||
{{< collapse title="Threema Gateway" >}}
|
||||
|
||||
#### Threema Gateway
|
||||
|
||||
```yaml
|
||||
type: threema
|
||||
@ -448,7 +508,11 @@ settings:
|
||||
recipient_id: A9R4KL4S
|
||||
```
|
||||
|
||||
##### VictorOps
|
||||
{{< /collapse >}}
|
||||
|
||||
{{< collapse title="VictorOps" >}}
|
||||
|
||||
#### VictorOps
|
||||
|
||||
```yaml
|
||||
type: victorops
|
||||
@ -459,7 +523,11 @@ settings:
|
||||
messageType: CRITICAL
|
||||
```
|
||||
|
||||
##### Webhook
|
||||
{{< /collapse >}}
|
||||
|
||||
{{< collapse title="Webhook" >}}
|
||||
|
||||
#### Webhook
|
||||
|
||||
```yaml
|
||||
type: webhook
|
||||
@ -480,7 +548,11 @@ settings:
|
||||
maxAlerts: '10'
|
||||
```
|
||||
|
||||
##### WeCom
|
||||
{{< /collapse >}}
|
||||
|
||||
{{< collapse title="WeCom" >}}
|
||||
|
||||
#### WeCom
|
||||
|
||||
```yaml
|
||||
type: wecom
|
||||
@ -495,17 +567,27 @@ settings:
|
||||
{{ template "default.title" . }}
|
||||
```
|
||||
|
||||
### Import notification policies
|
||||
{{< /collapse >}}
|
||||
|
||||
## Import notification policies
|
||||
|
||||
Create or reset the notification policy tree in your Grafana instance(s).
|
||||
|
||||
In Grafana, the entire notification policy tree is considered a single, large resource. Add new specific policies as sub-policies under the root policy. Since specific policies may depend on each other, you cannot provision subsets of the policy tree; the entire tree must be defined in a single place.
|
||||
|
||||
{{% admonition type="warning" %}}
|
||||
|
||||
Since the policy tree is a single resource, provisioning it will overwrite a policy tree created through any other means.
|
||||
|
||||
{{< /admonition >}}
|
||||
|
||||
1. Create a notification policy in Grafana.
|
||||
1. Use the [Alerting provisioning API][alerting_provisioning] export endpoints to download a provisioning file for your notification policy.
|
||||
1. Copy the contents into a YAML or JSON configuration file in the default provisioning directory or in your configured directory.
|
||||
1. [Export][alerting_export] and download a provisioning file for your notification policy.
|
||||
1. Copy the contents into a YAML or JSON configuration file in the `provisioning/alerting` directory.
|
||||
|
||||
Example configuration files can be found below.
|
||||
|
||||
1. Ensure that your files are in the right directory on the node running the Grafana server, so that they deploy alongside your Grafana instance(s).
|
||||
1. Add the file(s) to your GitOps workflow, so that they deploy alongside your Grafana instance(s).
|
||||
|
||||
Here is an example of a configuration file for creating notification policies.
|
||||
|
||||
@ -581,13 +663,7 @@ resetPolicies:
|
||||
- 1
|
||||
```
|
||||
|
||||
**Note:**
|
||||
|
||||
In Grafana, the entire notification policy tree is considered a single, large resource. Add new specific policies as sub-policies under the root policy. Since specific policies may depend on each other, you cannot provision subsets of the policy tree; the entire tree must be defined in a single place.
|
||||
|
||||
Since the policy tree is a single resource, applying it will overwrite a policy tree created through any other means.
|
||||
|
||||
### Import templates
|
||||
## Import templates
|
||||
|
||||
Create or delete templates in your Grafana instance(s).
|
||||
|
||||
@ -595,7 +671,7 @@ Create or delete templates in your Grafana instance(s).
|
||||
|
||||
Example configuration files can be found below.
|
||||
|
||||
2. Add the file(s) to your GitOps workflow, so that they deploy alongside your Grafana instance(s).
|
||||
1. Add the file(s) to your GitOps workflow, so that they deploy alongside your Grafana instance(s).
|
||||
|
||||
Here is an example of a configuration file for creating templates.
|
||||
|
||||
@ -627,7 +703,7 @@ deleteTemplates:
|
||||
name: my_first_template
|
||||
```
|
||||
|
||||
### Import mute timings
|
||||
## Import mute timings
|
||||
|
||||
Create or delete mute timings in your Grafana instance(s).
|
||||
|
||||
@ -676,65 +752,68 @@ deleteMuteTimes:
|
||||
name: mti_1
|
||||
```
|
||||
|
||||
### File provisioning using Kubernetes
|
||||
## File provisioning using Kubernetes
|
||||
|
||||
If you are a Kubernetes user, you can leverage file provisioning using Kubernetes configuration maps.
|
||||
|
||||
1. Create one or more configuration maps as follows.
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: grafana-alerting
|
||||
data:
|
||||
provisioning.yaml: |
|
||||
templates:
|
||||
- name: my_first_template
|
||||
template: the content for my template
|
||||
```
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: grafana-alerting
|
||||
data:
|
||||
provisioning.yaml: |
|
||||
templates:
|
||||
- name: my_first_template
|
||||
template: the content for my template
|
||||
```
|
||||
|
||||
2. Add the file(s) to your GitOps workflow, so that they deploy alongside your Grafana instance(s).
|
||||
1. Add the file(s) to your GitOps workflow, so that they deploy alongside your Grafana instance(s).
|
||||
|
||||
```yaml
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: grafana
|
||||
spec:
|
||||
replicas: 1
|
||||
selector:
|
||||
matchLabels:
|
||||
app: grafana
|
||||
template:
|
||||
metadata:
|
||||
name: grafana
|
||||
labels:
|
||||
app: grafana
|
||||
spec:
|
||||
containers:
|
||||
- name: grafana
|
||||
image: grafana/grafana:latest
|
||||
ports:
|
||||
- name: grafana
|
||||
containerPort: 3000
|
||||
volumeMounts:
|
||||
- mountPath: /etc/grafana/provisioning/alerting
|
||||
name: grafana-alerting
|
||||
readOnly: false
|
||||
volumes:
|
||||
- name: grafana-alerting
|
||||
configMap:
|
||||
defaultMode: 420
|
||||
name: grafana-alerting
|
||||
```
|
||||
```yaml
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: grafana
|
||||
spec:
|
||||
replicas: 1
|
||||
selector:
|
||||
matchLabels:
|
||||
app: grafana
|
||||
template:
|
||||
metadata:
|
||||
name: grafana
|
||||
labels:
|
||||
app: grafana
|
||||
spec:
|
||||
containers:
|
||||
- name: grafana
|
||||
image: grafana/grafana:latest
|
||||
ports:
|
||||
- name: grafana
|
||||
containerPort: 3000
|
||||
volumeMounts:
|
||||
- mountPath: /etc/grafana/provisioning/alerting
|
||||
name: grafana-alerting
|
||||
readOnly: false
|
||||
volumes:
|
||||
- name: grafana-alerting
|
||||
configMap:
|
||||
defaultMode: 420
|
||||
name: grafana-alerting
|
||||
```
|
||||
|
||||
This eliminates the need for a persistent database to use Grafana Alerting in Kubernetes; all your provisioned resources appear after each restart or re-deployment. Grafana still requires a database for normal operation, you do not need to persist the contents of the database between restarts if all objects are provisioned using files.
|
||||
|
||||
{{% docs/reference %}}
|
||||
[alerting_provisioning]: "/docs/grafana/ -> /docs/grafana/<GRAFANA VERSION>/developers/http_api/alerting_provisioning"
|
||||
[alerting_provisioning]: "/docs/grafana-cloud/ -> /docs/grafana/<GRAFANA VERSION>/developers/http_api/alerting_provisioning"
|
||||
**Useful Links:**
|
||||
|
||||
[reload-provisioning-configurations]: "/docs/grafana/ -> /docs/grafana/<GRAFANA VERSION>/developers/http_api/admin#reload-provisioning-configurations"
|
||||
[reload-provisioning-configurations]: "/docs/grafana-cloud/ -> /docs/grafana/<GRAFANA VERSION>/developers/http_api/admin#reload-provisioning-configurations"
|
||||
[Grafana provisioning][provisioning]
|
||||
|
||||
{{% docs/reference %}}
|
||||
[alerting_export]: "/docs/grafana/ -> /docs/grafana/<GRAFANA VERSION>/alerting/set-up/provision-alerting-resources/export-alerting-resources"
|
||||
[alerting_export]: "/docs/grafana-cloud/ -> /docs/grafana/<GRAFANA VERSION>/alerting/set-up/provision-alerting-resources/export-alerting-resources"
|
||||
[provisioning]: "/docs/grafana/ -> /docs/grafana/<GRAFANA VERSION>/administration/provisioning"
|
||||
[provisioning]: "/docs/grafana-cloud/ -> /docs/grafana/<GRAFANA VERSION>/administration/provisioning"
|
||||
{{% /docs/reference %}}
|
||||
|
@ -0,0 +1,20 @@
|
||||
---
|
||||
canonical: https://grafana.com/docs/grafana/latest/developers/http_api/alerting_provisioning/
|
||||
description: Create and manage alerting resources using the HTTP API
|
||||
keywords:
|
||||
- grafana
|
||||
- alerting
|
||||
- alerting resources
|
||||
- provisioning
|
||||
labels:
|
||||
products:
|
||||
- cloud
|
||||
- enterprise
|
||||
- oss
|
||||
title: Use the HTTP API to manage alerting resources
|
||||
weight: 400
|
||||
---
|
||||
|
||||
# Use the HTTP API to manage alerting resources
|
||||
|
||||
{{< docs/shared lookup="alerts/alerting_provisioning.md" source="grafana" version="latest" >}}
|
@ -11,41 +11,46 @@ keywords:
|
||||
- Terraform
|
||||
labels:
|
||||
products:
|
||||
- cloud
|
||||
- enterprise
|
||||
- oss
|
||||
title: Use Terraform to manage alerting resources
|
||||
menuTitle: Use Terraform to provision
|
||||
title: Use Terraform to provision alerting resources
|
||||
weight: 200
|
||||
---
|
||||
|
||||
# Use Terraform to manage alerting resources
|
||||
# Use Terraform to provision alerting resources
|
||||
|
||||
Use Terraform’s Grafana Provider to manage your alerting resources and provision them into your Grafana system. Terraform provider support for Grafana Alerting makes it easy to create, manage, and maintain your entire Grafana Alerting stack as code.
|
||||
|
||||
For more information on managing your alerting resources using Terraform, refer to the [Grafana Provider](https://registry.terraform.io/providers/grafana/grafana/latest/docs) documentation.
|
||||
Refer to [Grafana Provider](https://registry.terraform.io/providers/grafana/grafana/latest/docs) documentation for more examples and information on Terraform Alerting schemas.
|
||||
|
||||
Complete the following tasks to create and manage your alerting resources using Terraform.
|
||||
|
||||
1. Create an API key for provisioning.
|
||||
1. Configure the Terraform provider.
|
||||
1. Define your alerting resources in Terraform.
|
||||
1. Define your alerting resources in Terraform. [Export alerting resources][alerting_export] in Terraform format, or implement the [Terraform Alerting schemas](https://registry.terraform.io/providers/grafana/grafana/latest/docs).
|
||||
|
||||
1. Run `terraform apply` to provision your alerting resources.
|
||||
|
||||
## Before you begin
|
||||
{{< admonition type="note" >}}
|
||||
|
||||
- Ensure you have the grafana/grafana [Terraform provider](https://registry.terraform.io/providers/grafana/grafana/1.28.0) 1.27.0 or higher.
|
||||
- By default, you cannot edit resources provisioned from Terraform from the UI. This ensures that your alerting stack always stays in sync with your code. To change the default behaviour, refer to [Edit provisioned resources in the Grafana UI](#edit-provisioned-resources-in-the-grafana-ui).
|
||||
|
||||
- Ensure you are using Grafana 9.1 or higher.
|
||||
- Before you begin, ensure you have the [Grafana Terraform Provider](https://registry.terraform.io/providers/grafana/grafana/) 1.27.0 or higher, and are using Grafana 9.1 or higher.
|
||||
|
||||
{{< /admonition >}}
|
||||
|
||||
## Create an API key for provisioning
|
||||
|
||||
You can [create a normal Grafana API key][api-keys] to authenticate Terraform with Grafana. Most existing tooling using API keys should automatically work with the new Grafana Alerting support.
|
||||
You can create a [service account token][service-accounts] to authenticate Terraform with Grafana. Most existing tooling using API keys should automatically work with the new Grafana Alerting support.
|
||||
|
||||
There are also dedicated RBAC roles for alerting provisioning. This lets you easily authenticate as a [service account][service-accounts] with the minimum permissions needed to provision your Alerting infrastructure.
|
||||
There are also dedicated RBAC roles for alerting provisioning. This lets you easily authenticate as a service account with the minimum permissions needed to provision your Alerting infrastructure.
|
||||
|
||||
To create an API key for provisioning, complete the following steps.
|
||||
|
||||
1. Create a new service account for your CI pipeline.
|
||||
1. Assign the role “Access the alert rules Provisioning API.”
|
||||
1. Create a new service account.
|
||||
1. Assign the role or permission to access the [Alerting provisioning API][alerting_http_provisioning].
|
||||
1. Create a new service account token.
|
||||
1. Name and save the token for use in Terraform.
|
||||
|
||||
@ -73,70 +78,68 @@ provider "grafana" {
|
||||
}
|
||||
```
|
||||
|
||||
## Provision contact points and templates
|
||||
## Import contact points and templates
|
||||
|
||||
Contact points connect an alerting stack to the outside world. They tell Grafana how to connect to your external systems and where to deliver notifications. There are over fifteen different [integrations](https://registry.terraform.io/providers/grafana/grafana/latest/docs/resources/contact_point#optional) to choose from.
|
||||
Contact points connect an alerting stack to the outside world. They tell Grafana how to connect to your external systems and where to deliver notifications.
|
||||
|
||||
To provision contact points and templates, complete the following steps.
|
||||
To provision contact points and templates, refer to the [grafana_contact_point schema](https://registry.terraform.io/providers/grafana/grafana/latest/docs/resources/contact_point) and [grafana_message_template schema](https://registry.terraform.io/providers/grafana/grafana/latest/docs/resources/message_template), and complete the following steps.
|
||||
|
||||
1. Copy this code block into a .tf file on your local machine.
|
||||
1. Copy this code block into a `.tf` file on your local machine.
|
||||
|
||||
This example creates a contact point that sends alert notifications to Slack.
|
||||
This example creates a contact point that sends alert notifications to Slack.
|
||||
|
||||
```HCL
|
||||
resource "grafana_contact_point" "my_slack_contact_point" {
|
||||
name = "Send to My Slack Channel"
|
||||
```HCL
|
||||
resource "grafana_contact_point" "my_slack_contact_point" {
|
||||
name = "Send to My Slack Channel"
|
||||
|
||||
slack {
|
||||
url = <YOUR_SLACK_WEBHOOK_URL>
|
||||
text = <<EOT
|
||||
{{ len .Alerts.Firing }} alerts are firing!
|
||||
slack {
|
||||
url = <YOUR_SLACK_WEBHOOK_URL>
|
||||
text = <<EOT
|
||||
{{ len .Alerts.Firing }} alerts are firing!
|
||||
|
||||
Alert summaries:
|
||||
{{ range .Alerts.Firing }}
|
||||
{{ template "Alert Instance Template" . }}
|
||||
{{ end }}
|
||||
EOT
|
||||
}
|
||||
}
|
||||
```
|
||||
Alert summaries:
|
||||
{{ range .Alerts.Firing }}
|
||||
{{ template "Alert Instance Template" . }}
|
||||
{{ end }}
|
||||
EOT
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
You can create multiple external integrations in a single contact point. Notifications routed to this contact point will be sent to all integrations. This example shows multiple integrations in the same Terraform resource.
|
||||
You can create multiple external integrations in a single contact point. Notifications routed to this contact point will be sent to all integrations. This example shows multiple integrations in the same Terraform resource.
|
||||
|
||||
```
|
||||
resource "grafana_contact_point" "my_multi_contact_point" {
|
||||
name = "Send to Many Places"
|
||||
```
|
||||
resource "grafana_contact_point" "my_multi_contact_point" {
|
||||
name = "Send to Many Places"
|
||||
|
||||
slack {
|
||||
url = "webhook1"
|
||||
...
|
||||
}
|
||||
slack {
|
||||
url = "webhook2"
|
||||
...
|
||||
}
|
||||
teams {
|
||||
...
|
||||
}
|
||||
email {
|
||||
...
|
||||
}
|
||||
}
|
||||
```
|
||||
slack {
|
||||
url = "webhook1"
|
||||
...
|
||||
}
|
||||
slack {
|
||||
url = "webhook2"
|
||||
...
|
||||
}
|
||||
teams {
|
||||
...
|
||||
}
|
||||
email {
|
||||
...
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
2. Enter text for your notification in the text field.
|
||||
1. Enter text for your notification in the text field.
|
||||
|
||||
The `text` field supports [Go-style templating](https://pkg.go.dev/text/template). This enables you to manage your Grafana Alerting notification templates directly in Terraform.
|
||||
The `text` field supports [Go-style templating](https://pkg.go.dev/text/template). This enables you to manage your Grafana Alerting notification templates directly in Terraform.
|
||||
|
||||
3. Run the command ‘terraform apply’.
|
||||
1. Run the command `terraform apply`.
|
||||
|
||||
4. Go to the Grafana UI and check the details of your contact point.
|
||||
1. Go to the Grafana UI and check the details of your contact point.
|
||||
|
||||
By default, you cannot edit resources provisioned via Terraform from the UI. This ensures that your alerting stack always stays in sync with your code.
|
||||
1. Click **Test** to verify that the contact point works correctly.
|
||||
|
||||
5. Click **Test** to verify that the contact point works correctly.
|
||||
|
||||
**Note:**
|
||||
### Reuse templates
|
||||
|
||||
You can reuse the same templates across many contact points. In the example above, a shared template ie embedded using the statement `{{ template “Alert Instance Template” . }}`
|
||||
|
||||
@ -155,196 +158,193 @@ EOT
|
||||
}
|
||||
```
|
||||
|
||||
## Provision notification policies and routing
|
||||
## Import notification policies and routing
|
||||
|
||||
Notification policies tell Grafana how to route alert instances to your contact points. They connect firing alerts to your previously defined contact points using a system of labels and matchers.
|
||||
|
||||
To provision notification policies and routing, complete the following steps.
|
||||
To provision notification policies and routing, refer to the [grafana_notification_policy schema](https://registry.terraform.io/providers/grafana/grafana/latest/docs/resources/notification_policy), and complete the following steps.
|
||||
|
||||
1. Copy this code block into a .tf file on your local machine.
|
||||
{{% admonition type="warning" %}}
|
||||
|
||||
In this example, the alerts are grouped by `alertname`, which means that any notifications coming from alerts which share the same name, are grouped into the same Slack message. You can provide any set of label keys here, or you can use the special label `"..."` to route by all label keys, sending each alert in a separate notification.
|
||||
Since the policy tree is a single resource, provisioning the `grafana_notification_policy` resource will overwrite a policy tree created through any other means.
|
||||
|
||||
If you want to route specific notifications differently, you can add sub-policies. Sub-policies allow you to apply routing to different alerts based on label matching. In this example, we apply a mute timing to all alerts with the label a=b.
|
||||
{{< /admonition >}}
|
||||
|
||||
```HCL
|
||||
resource "grafana_notification_policy" "my_policy" {
|
||||
group_by = ["alertname"]
|
||||
contact_point = grafana_contact_point.my_slack_contact_point.name
|
||||
1. Copy this code block into a `.tf` file on your local machine.
|
||||
|
||||
group_wait = "45s"
|
||||
group_interval = "6m"
|
||||
repeat_interval = "3h"
|
||||
In this example, the alerts are grouped by `alertname`, which means that any notifications coming from alerts which share the same name, are grouped into the same Slack message. You can provide any set of label keys here, or you can use the special label `"..."` to route by all label keys, sending each alert in a separate notification.
|
||||
|
||||
policy {
|
||||
matcher {
|
||||
label = "a"
|
||||
match = "="
|
||||
value = "b"
|
||||
}
|
||||
group_by = ["..."]
|
||||
contact_point = grafana_contact_point.a_different_contact_point.name
|
||||
mute_timings = [grafana_mute_timing.my_mute_timing.name]
|
||||
If you want to route specific notifications differently, you can add sub-policies. Sub-policies allow you to apply routing to different alerts based on label matching. In this example, we apply a mute timing to all alerts with the label a=b.
|
||||
|
||||
policy {
|
||||
matcher {
|
||||
label = "sublabel"
|
||||
match = "="
|
||||
value = "subvalue"
|
||||
}
|
||||
contact_point = grafana_contact_point.a_third_contact_point.name
|
||||
group_by = ["..."]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
```HCL
|
||||
resource "grafana_notification_policy" "my_policy" {
|
||||
group_by = ["alertname"]
|
||||
contact_point = grafana_contact_point.my_slack_contact_point.name
|
||||
|
||||
2. In the mute_timings field, link a mute timing to your notification policy.
|
||||
group_wait = "45s"
|
||||
group_interval = "6m"
|
||||
repeat_interval = "3h"
|
||||
|
||||
3. Run the command ‘terraform apply’.
|
||||
policy {
|
||||
matcher {
|
||||
label = "a"
|
||||
match = "="
|
||||
value = "b"
|
||||
}
|
||||
group_by = ["..."]
|
||||
contact_point = grafana_contact_point.a_different_contact_point.name
|
||||
mute_timings = [grafana_mute_timing.my_mute_timing.name]
|
||||
|
||||
4. Go to the Grafana UI and check the details of your notification policy.
|
||||
policy {
|
||||
matcher {
|
||||
label = "sublabel"
|
||||
match = "="
|
||||
value = "subvalue"
|
||||
}
|
||||
contact_point = grafana_contact_point.a_third_contact_point.name
|
||||
group_by = ["..."]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Note:**
|
||||
1. In the mute_timings field, link a mute timing to your notification policy.
|
||||
|
||||
Since the policy tree is a single resource, applying it will overwrite a policy tree created through any other means.
|
||||
1. Run the command `terraform apply`.
|
||||
|
||||
By default, you cannot edit resources provisioned from Terraform from the UI. This ensures that your alerting stack always stays in sync with your code.
|
||||
1. Go to the Grafana UI and check the details of your notification policy.
|
||||
|
||||
5. Click **Test** to verify that the notification point is working correctly.
|
||||
1. Click **Test** to verify that the notification point is working correctly.
|
||||
|
||||
## Provision mute timings
|
||||
## Import mute timings
|
||||
|
||||
Mute timings provide the ability to mute alert notifications for defined time periods.
|
||||
|
||||
To provision mute timings, complete the following steps.
|
||||
To provision mute timings, refer to the [grafana_mute_timing schema](https://registry.terraform.io/providers/grafana/grafana/latest/docs/resources/mute_timing), and complete the following steps.
|
||||
|
||||
1. Copy this code block into a .tf file on your local machine.
|
||||
1. Copy this code block into a `.tf` file on your local machine.
|
||||
|
||||
In this example, alert notifications are muted on weekends.
|
||||
In this example, alert notifications are muted on weekends.
|
||||
|
||||
```HCL
|
||||
resource "grafana_mute_timing" "my_mute_timing" {
|
||||
name = "My Mute Timing"
|
||||
```HCL
|
||||
resource "grafana_mute_timing" "my_mute_timing" {
|
||||
name = "My Mute Timing"
|
||||
|
||||
intervals {
|
||||
times {
|
||||
start = "04:56"
|
||||
end = "14:17"
|
||||
}
|
||||
weekdays = ["saturday", "sunday", "tuesday:thursday"]
|
||||
months = ["january:march", "12"]
|
||||
years = ["2025:2027"]
|
||||
}
|
||||
}
|
||||
```
|
||||
intervals {
|
||||
times {
|
||||
start = "04:56"
|
||||
end = "14:17"
|
||||
}
|
||||
weekdays = ["saturday", "sunday", "tuesday:thursday"]
|
||||
months = ["january:march", "12"]
|
||||
years = ["2025:2027"]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
2. Run the command ‘terraform apply’.
|
||||
3. Go to the Grafana UI and check the details of your mute timing.
|
||||
4. Reference your newly created mute timing in a notification policy using the `mute_timings` field.
|
||||
1. Run the command `terraform apply`.
|
||||
1. Go to the Grafana UI and check the details of your mute timing.
|
||||
1. Reference your newly created mute timing in a notification policy using the `mute_timings` field.
|
||||
This will apply your mute timing to some or all of your notifications.
|
||||
|
||||
**Note:**
|
||||
1. Click **Test** to verify that the mute timing is working correctly.
|
||||
|
||||
By default, you cannot edit resources provisioned from Terraform from the UI. This ensures that your alerting stack always stays in sync with your code.
|
||||
|
||||
5. Click **Test** to verify that the mute timing is working correctly.
|
||||
|
||||
## Provision alert rules
|
||||
## Import alert rules
|
||||
|
||||
[Alert rules][alerting-rules] enable you to alert against any Grafana data source. This can be a data source that you already have configured, or you can [define your data sources in Terraform](https://registry.terraform.io/providers/grafana/grafana/latest/docs/resources/data_source) alongside your alert rules.
|
||||
|
||||
To provision alert rules, complete the following steps.
|
||||
To provision alert rules, refer to the [grafana_rule_group schema](https://registry.terraform.io/providers/grafana/grafana/latest/docs/resources/rule_group), and complete the following steps.
|
||||
|
||||
1. Create a data source to query and a folder to store your rules in.
|
||||
|
||||
In this example, the [TestData][testdata] data source is used.
|
||||
In this example, the [TestData][testdata] data source is used.
|
||||
|
||||
Alerts can be defined against any backend datasource in Grafana.
|
||||
Alerts can be defined against any backend datasource in Grafana.
|
||||
|
||||
```HCL
|
||||
resource "grafana_data_source" "testdata_datasource" {
|
||||
name = "TestData"
|
||||
type = "testdata"
|
||||
}
|
||||
```HCL
|
||||
resource "grafana_data_source" "testdata_datasource" {
|
||||
name = "TestData"
|
||||
type = "testdata"
|
||||
}
|
||||
|
||||
resource "grafana_folder" "rule_folder" {
|
||||
title = "My Rule Folder"
|
||||
}
|
||||
```
|
||||
resource "grafana_folder" "rule_folder" {
|
||||
title = "My Rule Folder"
|
||||
}
|
||||
```
|
||||
|
||||
2. Define an alert rule.
|
||||
1. Define an alert rule.
|
||||
|
||||
For more information on alert rules, refer to [how to create Grafana-managed alerts](/blog/2022/08/01/grafana-alerting-video-how-to-create-alerts-in-grafana-9/).
|
||||
For more information on alert rules, refer to [how to create Grafana-managed alerts](/blog/2022/08/01/grafana-alerting-video-how-to-create-alerts-in-grafana-9/).
|
||||
|
||||
3. Create a rule group containing one or more rules.
|
||||
1. Create a rule group containing one or more rules.
|
||||
|
||||
In this example, the `grafana_rule_group` resource group is used.
|
||||
In this example, the `grafana_rule_group` resource group is used.
|
||||
|
||||
```HCL
|
||||
resource "grafana_rule_group" "my_rule_group" {
|
||||
name = "My Alert Rules"
|
||||
folder_uid = grafana_folder.rule_folder.uid
|
||||
interval_seconds = 60
|
||||
org_id = 1
|
||||
```HCL
|
||||
resource "grafana_rule_group" "my_rule_group" {
|
||||
name = "My Alert Rules"
|
||||
folder_uid = grafana_folder.rule_folder.uid
|
||||
interval_seconds = 60
|
||||
org_id = 1
|
||||
|
||||
rule {
|
||||
name = "My Random Walk Alert"
|
||||
condition = "C"
|
||||
for = "0s"
|
||||
rule {
|
||||
name = "My Random Walk Alert"
|
||||
condition = "C"
|
||||
for = "0s"
|
||||
|
||||
// Query the datasource.
|
||||
data {
|
||||
ref_id = "A"
|
||||
relative_time_range {
|
||||
from = 600
|
||||
to = 0
|
||||
}
|
||||
datasource_uid = grafana_data_source.testdata_datasource.uid
|
||||
// `model` is a JSON blob that sends datasource-specific data.
|
||||
// It's different for every datasource. The alert's query is defined here.
|
||||
model = jsonencode({
|
||||
intervalMs = 1000
|
||||
maxDataPoints = 43200
|
||||
refId = "A"
|
||||
})
|
||||
}
|
||||
// Query the datasource.
|
||||
data {
|
||||
ref_id = "A"
|
||||
relative_time_range {
|
||||
from = 600
|
||||
to = 0
|
||||
}
|
||||
datasource_uid = grafana_data_source.testdata_datasource.uid
|
||||
// `model` is a JSON blob that sends datasource-specific data.
|
||||
// It's different for every datasource. The alert's query is defined here.
|
||||
model = jsonencode({
|
||||
intervalMs = 1000
|
||||
maxDataPoints = 43200
|
||||
refId = "A"
|
||||
})
|
||||
}
|
||||
|
||||
// The query was configured to obtain data from the last 60 seconds. Let's alert on the average value of that series using a Reduce stage.
|
||||
data {
|
||||
datasource_uid = "__expr__"
|
||||
// You can also create a rule in the UI, then GET that rule to obtain the JSON.
|
||||
// This can be helpful when using more complex reduce expressions.
|
||||
model = <<EOT
|
||||
{"conditions":[{"evaluator":{"params":[0,0],"type":"gt"},"operator":{"type":"and"},"query":{"params":["A"]},"reducer":{"params":[],"type":"last"},"type":"avg"}],"datasource":{"name":"Expression","type":"__expr__","uid":"__expr__"},"expression":"A","hide":false,"intervalMs":1000,"maxDataPoints":43200,"reducer":"last","refId":"B","type":"reduce"}
|
||||
EOT
|
||||
ref_id = "B"
|
||||
relative_time_range {
|
||||
from = 0
|
||||
to = 0
|
||||
}
|
||||
}
|
||||
// The query was configured to obtain data from the last 60 seconds. Let's alert on the average value of that series using a Reduce stage.
|
||||
data {
|
||||
datasource_uid = "__expr__"
|
||||
// You can also create a rule in the UI, then GET that rule to obtain the JSON.
|
||||
// This can be helpful when using more complex reduce expressions.
|
||||
model = <<EOT
|
||||
{"conditions":[{"evaluator":{"params":[0,0],"type":"gt"},"operator":{"type":"and"},"query":{"params":["A"]},"reducer":{"params":[],"type":"last"},"type":"avg"}],"datasource":{"name":"Expression","type":"__expr__","uid":"__expr__"},"expression":"A","hide":false,"intervalMs":1000,"maxDataPoints":43200,"reducer":"last","refId":"B","type":"reduce"}
|
||||
EOT
|
||||
ref_id = "B"
|
||||
relative_time_range {
|
||||
from = 0
|
||||
to = 0
|
||||
}
|
||||
}
|
||||
|
||||
// Now, let's use a math expression as our threshold.
|
||||
// We want to alert when the value of stage "B" above exceeds 70.
|
||||
data {
|
||||
datasource_uid = "__expr__"
|
||||
ref_id = "C"
|
||||
relative_time_range {
|
||||
from = 0
|
||||
to = 0
|
||||
}
|
||||
model = jsonencode({
|
||||
expression = "$B > 70"
|
||||
type = "math"
|
||||
refId = "C"
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
// Now, let's use a math expression as our threshold.
|
||||
// We want to alert when the value of stage "B" above exceeds 70.
|
||||
data {
|
||||
datasource_uid = "__expr__"
|
||||
ref_id = "C"
|
||||
relative_time_range {
|
||||
from = 0
|
||||
to = 0
|
||||
}
|
||||
model = jsonencode({
|
||||
expression = "$B > 70"
|
||||
type = "math"
|
||||
refId = "C"
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
4. Go to the Grafana UI and check your alert rule.
|
||||
1. Run the command `terraform apply`.
|
||||
1. Go to the Grafana UI and check your alert rule.
|
||||
|
||||
You can see whether or not the alert rule is firing. You can also see a visualization of each of the alert rule’s query stages
|
||||
|
||||
@ -352,12 +352,36 @@ When the alert fires, Grafana routes a notification through the policy you defin
|
||||
|
||||
For example, if you chose Slack as a contact point, Grafana’s embedded [Alertmanager](https://github.com/prometheus/alertmanager) automatically posts a message to Slack.
|
||||
|
||||
## Edit provisioned resources in the Grafana UI
|
||||
|
||||
By default, you cannot edit resources provisioned via Terraform in Grafana. To enable editing these resources in the Grafana UI, use the `disable_provenance` attribute on alerting resources:
|
||||
|
||||
```HCL
|
||||
provider "grafana" {
|
||||
url = "http://grafana.example.com/"
|
||||
auth = var.grafana_auth
|
||||
}
|
||||
|
||||
resource "grafana_mute_timing" "mute_all" {
|
||||
name = "mute all"
|
||||
disable_provenance = true
|
||||
intervals {}
|
||||
}
|
||||
```
|
||||
|
||||
**Useful Links:**
|
||||
|
||||
[Grafana Terraform Provider documentation](https://registry.terraform.io/providers/grafana/grafana/latest/docs)
|
||||
|
||||
{{% docs/reference %}}
|
||||
[alerting-rules]: "/docs/grafana/ -> /docs/grafana/<GRAFANA VERSION>/alerting/alerting-rules"
|
||||
[alerting-rules]: "/docs/grafana-cloud/ -> /docs/grafana-cloud/alerting-and-irm/alerting/alerting-rules"
|
||||
|
||||
[api-keys]: "/docs/grafana/ -> /docs/grafana/<GRAFANA VERSION>/administration/api-keys"
|
||||
[api-keys]: "/docs/grafana-cloud/ -> /docs/grafana/<GRAFANA VERSION>/administration/api-keys"
|
||||
[alerting_export]: "/docs/grafana/ -> /docs/grafana/<GRAFANA VERSION>/alerting/set-up/provision-alerting-resources/export-alerting-resources"
|
||||
[alerting_export]: "/docs/grafana-cloud/ -> /docs/grafana/<GRAFANA VERSION>/alerting/set-up/provision-alerting-resources/export-alerting-resources"
|
||||
|
||||
[alerting_http_provisioning]: "/docs/grafana/ -> /docs/grafana/<GRAFANA VERSION>/alerting/set-up/provision-alerting-resources/http-api-provisioning"
|
||||
[alerting_http_provisioning]: "/docs/grafana-cloud/ -> /docs/grafana/<GRAFANA VERSION>/alerting/set-up/provision-alerting-resources/http-api-provisioning"
|
||||
|
||||
[service-accounts]: "/docs/grafana/ -> /docs/grafana/<GRAFANA VERSION>/administration/service-accounts"
|
||||
[service-accounts]: "/docs/grafana-cloud/ -> /docs/grafana/<GRAFANA VERSION>/administration/service-accounts"
|
||||
|
@ -1,108 +0,0 @@
|
||||
---
|
||||
aliases:
|
||||
- ../../provision-alerting-resources/view-provisioned-resources/
|
||||
canonical: https://grafana.com/docs/grafana/latest/alerting/set-up/provision-alerting-resources/view-provisioned-resources/
|
||||
description: Manage provisioned alerting resources in Grafana
|
||||
keywords:
|
||||
- grafana
|
||||
- alerting
|
||||
- alerting resources
|
||||
- provisioning
|
||||
labels:
|
||||
products:
|
||||
- cloud
|
||||
- enterprise
|
||||
- oss
|
||||
menuTitle: Manage provisioned alerting resources
|
||||
title: Manage provisioned alerting resources
|
||||
weight: 300
|
||||
---
|
||||
|
||||
# Manage provisioned alerting resources
|
||||
|
||||
Verify that your alerting resources were created in Grafana, as well as edit or export your provisioned alerting resources.
|
||||
|
||||
## View provisioned alerting resoureces
|
||||
|
||||
To view your provisioned resources in Grafana, complete the following steps.
|
||||
|
||||
1. Open your Grafana instance.
|
||||
1. Navigate to Alerting.
|
||||
1. Click an alerting resource folder, for example, Alert rules.
|
||||
|
||||
Provisioned resources are labeled **Provisioned**, so that it is clear that they were not created manually.
|
||||
|
||||
## Export provisioned alerting resources
|
||||
|
||||
Export your alerting resources, such as alert rules, contact points, and notification policies in JSON, YAML, or Terraform format. You can export all Grafana-managed alert rules, single folders, and single groups.
|
||||
|
||||
To export provisioned alerting resources from the Grafana UI, complete the following steps.
|
||||
|
||||
1. Click **Alerts & IRM** -> **Alert rules**.
|
||||
1. To export all Grafana-managed rules, click **Export rules**.
|
||||
1. To export a folder, change the **View as** to **List**.
|
||||
1. Select the folder you want to export and click the **Export rules folder** icon.
|
||||
1. To export a group, change the **View as** to **Grouped**.
|
||||
1. Find the group you want to export and click the **Export rule group** icon.
|
||||
1. Choose the format to export in.
|
||||
|
||||
Note that formats JSON and YAML are suitable only for file provisioning. To get rule definition in provisioning API format, use the provisioning GET API.
|
||||
|
||||
1. Click **Copy Code** or **Download**.
|
||||
1. Choose **Copy Code** to go to an existing file and paste in the code.
|
||||
1. Choose **Download** to download a file with the exported data.
|
||||
|
||||
## Edit provisioned alert rules
|
||||
|
||||
Use the **Modify export** mode for alert rules to edit provisioned alert rules and export a modified version.
|
||||
|
||||
{{% admonition type="note" %}} This feature is for Grafana-managed alert rules only. It is available to Admin, Viewer, and Editor roles. {{% /admonition %}}
|
||||
|
||||
To edit provisioned alerting alert rules from the Grafana UI, complete the following steps.
|
||||
|
||||
1. Click **Alerts & IRM** -> **Alert rules**.
|
||||
1. Locate the alert rule you want to edit and click **More** -> **Modify Export** to open the Alert Rule form.
|
||||
1. From the Alert Rule form, edit the fields you want to change.
|
||||
1. Click **Export** to export all alert rules within the group.
|
||||
|
||||
You can only export groups of rules; not single rules.
|
||||
The exported rule data appears in different formats - HTML, JSON, Terraform.
|
||||
|
||||
1. Choose the format to export in.
|
||||
1. Click **Copy Code** or **Download**.
|
||||
|
||||
a. Choose **Copy Code** to go to an existing file and paste in the code.
|
||||
|
||||
b. Choose **Download** to download a file with the exported data.
|
||||
|
||||
## Edit API-provisioned alerting resources
|
||||
|
||||
To enable editing of API-provisioned resources in the Grafana UI, add the `X-Disable-Provenance` header to the following requests in the API:
|
||||
|
||||
- `POST /api/v1/provisioning/alert-rules`
|
||||
- `PUT /api/v1/provisioning/folder/{FolderUID}/rule-groups/{Group}` (calling this endpoint will change provenance for all alert rules within the alert group)
|
||||
- `POST /api/v1/provisioning/contact-points`
|
||||
- `POST /api/v1/provisioning/mute-timings`
|
||||
- `PUT /api/v1/provisioning/policies`
|
||||
- `PUT /api/v1/provisioning/templates/{name}`
|
||||
|
||||
To reset the notification policy tree to the default and unlock it for editing in the Grafana UI, use the `DELETE /api/v1/provisioning/policies` endpoint.
|
||||
|
||||
In Terraform, you can use the `disable_provenance` attribute on alerting resources:
|
||||
|
||||
```
|
||||
provider "grafana" {
|
||||
url = "http://grafana.example.com/"
|
||||
auth = var.grafana_auth
|
||||
}
|
||||
|
||||
resource "grafana_mute_timing" "mute_all" {
|
||||
name = "mute all"
|
||||
disable_provenance = true
|
||||
intervals {}
|
||||
}
|
||||
```
|
||||
|
||||
**Note:**
|
||||
|
||||
You cannot edit provisioned resources from files in Grafana. You can only change the resource properties by changing the provisioning file and restarting Grafana or carrying out a hot reload. This prevents changes being made to the resource that would be overwritten if a file is provisioned again or a hot reload is carried out.
|
File diff suppressed because it is too large
Load Diff
1622
docs/sources/shared/alerts/alerting_provisioning.md
Normal file
1622
docs/sources/shared/alerts/alerting_provisioning.md
Normal file
File diff suppressed because it is too large
Load Diff
Loading…
Reference in New Issue
Block a user