mirror of
https://github.com/grafana/grafana.git
synced 2025-02-25 18:55:37 -06:00
Docs: Consolidation and add call-outs for semantics in Alert Provisioning documentation (#68848)
* Provisioning landing page * Move editing bit to view page * Specify API vs File, and provide terraform example * Improvements around notif policies * Docs around contact points * Use h2 instead of h1 * Address feedback * Move text about policy granularity
This commit is contained in:
Alexander Weaver
GitHub
parent
87ff5d1212
commit
128990dbb7
@ -24,26 +24,17 @@ There are three options to choose from:
|
||||
|
||||
For more information on the Alerting Provisioning HTTP API, refer to [Alerting provisioning API]({{< relref "../../../developers/http_api/alerting_provisioning" >}}).
|
||||
|
||||
1. Provision your alerting resources using Terraform.
|
||||
1. Provision your alerting resources using [Terraform](https://www.terraform.io/).
|
||||
|
||||
**Note:**
|
||||
|
||||
Currently, provisioning for Grafana Alerting supports alert rules, contact points, mute timings, and templates. Provisioned alerting resources using file provisioning or Terraform can only be edited in the source that created them and not from within Grafana or any other source. For example, if you provision your alerting resources using files from disk, you cannot edit the data in Terraform or from within Grafana.
|
||||
|
||||
To allow editing of provisioned resources in the Grafana UI, add the `X-Disable-Provenance` header to the following requests in the API:
|
||||
|
||||
- `POST /api/v1/provisioning/alert-rules`
|
||||
- `PUT /api/v1/provisioning/folder/{FolderUID}/rule-groups/{Group}` (calling this endpoint will change provenance for all alert rules within the alert group)
|
||||
- `POST /api/v1/provisioning/contact-points`
|
||||
- `POST /api/v1/provisioning/mute-timings`
|
||||
- `PUT /api/v1/provisioning/policies`
|
||||
- `PUT /api/v1/provisioning/templates/{name}`
|
||||
Currently, provisioning for Grafana Alerting supports alert rules, contact points, notification policies, mute timings, and templates. Provisioned alerting resources using file provisioning or Terraform can only be edited in the source that created them and not from within Grafana or any other source. For example, if you provision your alerting resources using files from disk, you cannot edit the data in Terraform or from within Grafana.
|
||||
|
||||
**Useful Links:**
|
||||
|
||||
[Grafana provisioning][provisioning]
|
||||
|
||||
[Grafana Cloud provisioning](/docs/grafana-cloud/infrastructure-as-code/terraform/)
|
||||
[Terraform provisioning](/docs/grafana-cloud/infrastructure-as-code/terraform/)
|
||||
|
||||
[Grafana Alerting provisioning API][alerting_provisioning]
|
||||
|
||||
|
||||
@ -486,7 +486,7 @@ settings:
|
||||
|
||||
### Provision notification policies
|
||||
|
||||
Create or reset notification policies in your Grafana instance(s).
|
||||
Create or reset the notification policy tree in your Grafana instance(s).
|
||||
|
||||
1. Create a YAML or JSON configuration file.
|
||||
|
||||
@ -494,7 +494,7 @@ Create or reset notification policies in your Grafana instance(s).
|
||||
|
||||
2. Add the file(s) to your GitOps workflow, so that they deploy alongside your Grafana instance(s).
|
||||
|
||||
Here is an example of a configuration file for creating notification policiies.
|
||||
Here is an example of a configuration file for creating notification policies.
|
||||
|
||||
```yaml
|
||||
# config file version
|
||||
@ -548,13 +548,16 @@ policies:
|
||||
# <duration> How long to wait before sending a notification again if it has already
|
||||
# been sent successfully for an alert. (Usually ~3h or more), default = 4h
|
||||
repeat_interval: 4h
|
||||
# <list> Zero or more child routes
|
||||
# <list> Zero or more child policies. The schema is the same as the root policy.
|
||||
# routes:
|
||||
# - Another recursively nested policy...
|
||||
# # Another recursively nested policy...
|
||||
# - receiver: another-receiver
|
||||
# matchers:
|
||||
# - ...
|
||||
# ...
|
||||
```
|
||||
|
||||
Here is an example of a configuration file for resetting notification policies.
|
||||
Here is an example of a configuration file for resetting the policy tree back to its default value:
|
||||
|
||||
```yaml
|
||||
# config file version
|
||||
@ -565,6 +568,12 @@ resetPolicies:
|
||||
- 1
|
||||
```
|
||||
|
||||
**Note:**
|
||||
|
||||
In Grafana, the entire notification policy tree is considered a single, large resource. Add new specific policies as sub-policies under the root policy. Since specific policies may depend on each other, you cannot provision subsets of the policy tree; the entire tree must be defined in a single place.
|
||||
|
||||
Since the policy tree is a single resource, applying it will overwrite a policy tree created through any other means.
|
||||
|
||||
### Provision templates
|
||||
|
||||
Create or delete templates in your Grafana instance(s).
|
||||
@ -707,7 +716,7 @@ spec:
|
||||
name: grafana-alerting
|
||||
```
|
||||
|
||||
This eliminates the need for a persistent database to use Grafana Alerting in Kubernetes; all your provisioned resources appear after each restart or re-deployment.
|
||||
This eliminates the need for a persistent database to use Grafana Alerting in Kubernetes; all your provisioned resources appear after each restart or re-deployment. Grafana still requires a database for normal operation, you do not need to persist the contents of the database between restarts if all objects are provisioned using files.
|
||||
|
||||
{{% docs/reference %}}
|
||||
[alerting_provisioning]: "/docs/grafana/ -> /docs/grafana/<GRAFANA VERSION>/developers/http_api/alerting_provisioning"
|
||||
|
||||
@ -96,6 +96,29 @@ EOT
|
||||
}
|
||||
```
|
||||
|
||||
You can create multiple external integrations in a single contact point. Notifications routed to this contact point will be sent to all integrations. This example shows multiple integrations in the same Terraform resource.
|
||||
|
||||
```
|
||||
resource "grafana_contact_point" "my_multi_contact_point" {
|
||||
name = "Send to Many Places"
|
||||
|
||||
slack {
|
||||
url = "webhook1"
|
||||
...
|
||||
}
|
||||
slack {
|
||||
url = "webhook2"
|
||||
...
|
||||
}
|
||||
teams {
|
||||
...
|
||||
}
|
||||
email {
|
||||
...
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
2. Enter text for your notification in the text field.
|
||||
|
||||
The `text` field supports [Go-style templating](https://pkg.go.dev/text/template). This enables you to manage your Grafana Alerting notification templates directly in Terraform.
|
||||
@ -104,7 +127,7 @@ The `text` field supports [Go-style templating](https://pkg.go.dev/text/template
|
||||
|
||||
4. Go to the Grafana UI and check the details of your contact point.
|
||||
|
||||
You cannot edit resources provisioned via Terraform from the UI. This ensures that your alerting stack always stays in sync with your code.
|
||||
By default, you cannot edit resources provisioned via Terraform from the UI. This ensures that your alerting stack always stays in sync with your code.
|
||||
|
||||
5. Click **Test** to verify that the contact point works correctly.
|
||||
|
||||
@ -129,13 +152,13 @@ EOT
|
||||
|
||||
## Provision notification policies and routing
|
||||
|
||||
Notification policies tell Grafana how to route alert instances, as opposed to where. They connect firing alerts to your previously defined contact points using a system of labels and matchers.
|
||||
Notification policies tell Grafana how to route alert instances to your contact points. They connect firing alerts to your previously defined contact points using a system of labels and matchers.
|
||||
|
||||
To provision notification policies and routing, complete the following steps.
|
||||
|
||||
1. Copy this code block into a .tf file on your local machine.
|
||||
|
||||
In this example, the alerts are grouped by `alertname`, which means that any notifications coming from alerts which share the same name, are grouped into the same Slack message.
|
||||
In this example, the alerts are grouped by `alertname`, which means that any notifications coming from alerts which share the same name, are grouped into the same Slack message. You can provide any set of label keys here, or you can use the special label `"..."` to route by all label keys, sending each alert in a separate notification.
|
||||
|
||||
If you want to route specific notifications differently, you can add sub-policies. Sub-policies allow you to apply routing to different alerts based on label matching. In this example, we apply a mute timing to all alerts with the label a=b.
|
||||
|
||||
@ -179,7 +202,9 @@ resource "grafana_notification_policy" "my_policy" {
|
||||
|
||||
**Note:**
|
||||
|
||||
You cannot edit resources provisioned from Terraform from the UI. This ensures that your alerting stack always stays in sync with your code.
|
||||
Since the policy tree is a single resource, applying it will overwrite a policy tree created through any other means.
|
||||
|
||||
By default, you cannot edit resources provisioned from Terraform from the UI. This ensures that your alerting stack always stays in sync with your code.
|
||||
|
||||
5. Click **Test** to verify that the notification point is working correctly.
|
||||
|
||||
@ -216,7 +241,7 @@ resource "grafana_mute_timing" "my_mute_timing" {
|
||||
|
||||
**Note:**
|
||||
|
||||
You cannot edit resources provisioned from Terraform from the UI. This ensures that your alerting stack always stays in sync with your code.
|
||||
By default, you cannot edit resources provisioned from Terraform from the UI. This ensures that your alerting stack always stays in sync with your code.
|
||||
|
||||
5. Click **Test** to verify that the mute timing is working correctly.
|
||||
|
||||
|
||||
@ -24,6 +24,31 @@ To view your provisioned resources in Grafana, complete the following steps.
|
||||
|
||||
Provisioned resources are labeled **Provisioned**, so that it is clear that they were not created manually.
|
||||
|
||||
## Edit API-provisioned alerting resources
|
||||
|
||||
To enable editing of API-provisioned resources in the Grafana UI, add the `X-Disable-Provenance` header to the following requests in the API:
|
||||
|
||||
- `POST /api/v1/provisioning/alert-rules`
|
||||
- `PUT /api/v1/provisioning/folder/{FolderUID}/rule-groups/{Group}` (calling this endpoint will change provenance for all alert rules within the alert group)
|
||||
- `POST /api/v1/provisioning/contact-points`
|
||||
- `POST /api/v1/provisioning/mute-timings`
|
||||
- `PUT /api/v1/provisioning/policies`
|
||||
- `PUT /api/v1/provisioning/templates/{name}`
|
||||
|
||||
To reset the notification policy tree to the default and unlock it for editing in the Grafana UI, use the `DELETE /api/v1/provisioning/policies` endpoint.
|
||||
|
||||
To pass the `X-Disable-Provenance` header from Terraform, add it to the `http_headers` field on the provider object:
|
||||
|
||||
```
|
||||
provider "grafana" {
|
||||
url = "http://grafana.example.com/"
|
||||
auth = var.grafana_auth
|
||||
http_headers = {
|
||||
"X-Disable-Provenance" = "true"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Note:**
|
||||
|
||||
You cannot edit provisioned resources from Grafana. You can only change the resource properties by changing the provisioning file and restarting Grafana or carrying out a hot reload. This prevents changes being made to the resource that would be overwritten if a file is provisioned again or a hot reload is carried out.
|
||||
You cannot edit provisioned resources from files in Grafana. You can only change the resource properties by changing the provisioning file and restarting Grafana or carrying out a hot reload. This prevents changes being made to the resource that would be overwritten if a file is provisioned again or a hot reload is carried out.
|
||||
|
||||
Loading…
Reference in New Issue
Block a user