Docs: Move labels and annotations topic to alerting fundamentals section (#49283)

* Moved annotation and labels topic to fundamentals and added label matching to the same section

* Cosmetic updates to "How label matching works topic"

* Few more changes to annotations and label section. Also added stem sentence to silences topics.

* Fixed one more broken relref.

* Update docs/sources/alerting/fundamentals/_index.md

Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com>

Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com>
This commit is contained in:
JitaC 2022-05-19 19:17:57 -04:00 committed by GitHub
parent 64aedb78ba
commit 6397b34f8b
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
16 changed files with 97 additions and 138 deletions

View File

@ -29,7 +29,7 @@ Before you begin, we recommend that you familiarize yourself with some of the [f
- [Create Grafana Mimir or Loki managed alerting rules]({{< relref "alerting-rules/create-mimir-loki-managed-rule.md" >}})
- [View existing alerting rules and manage their current state]({{< relref "alerting-rules/rule-list.md" >}})
- [View the state and health of alerting rules]({{< relref "./fundamentals/state-and-health.md" >}})
- [View alert groupings]({{< relref "./alert-groups.md" >}})
- [View alert groupings]({{< relref "./alert-groups/_index.md" >}})
- [Add or edit an alert contact point]({{< relref "./contact-points/_index.md" >}})
- [Add or edit notification policies]({{< relref "./notifications/_index.md" >}})
- [Add or edit silences]({{< relref "./silences/_index.md" >}})

View File

@ -2,7 +2,7 @@
aliases = ["/docs/grafana/latest/alerting/alert-groups/", "/docs/grafana/latest/alerting/unified-alerting/alert-groups/"]
description = "Alert groups"
keywords = ["grafana", "alerting", "alerts", "groups"]
title = "Alert groups"
title = "View alert groupings"
weight = 445
+++

View File

@ -1,39 +0,0 @@
+++
aliases = ["/docs/grafana/latest/alerting/alerting-rules/alert-annotation-label/", "/docs/grafana/latest/alerting/unified-alerting/alerting-rules/alert-annotation-label/"]
description = "Annotations and labels for alerting"
keywords = ["grafana", "alerting", "guide", "rules", "create"]
title = "Annotations and labels for alerting rules"
weight = 401
+++
# Annotations and labels for alerting rules
Annotations and labels are key value pairs associated with alerts originating from the alerting rule, datasource response, and as a result of alerting rule evaluation. They can be used in alert notifications directly or in [templates]({{< relref "../contact-points/message-templating/_index.md" >}}) and [template functions]({{< relref "../contact-points/message-templating/template-functions" >}}) to create notification contact dynamically.
## Annotations
Annotations are key-value pairs that provide additional meta-information about an alert. You can use the following annotations: `description`, `summary`, `runbook_url`, `alertId`, `dashboardUid`, and `panelId`. For example, a description, a summary, and a runbook URL. These are displayed in rule and alert details in the UI and can be used in contact point message templates.
## Labels
Labels are key-value pairs that contain information about, and are used to uniquely identify an alert. The label set for an alert is generated and added to throughout the alerting evaluation and notification process.
### How are labels used?
- The complete set of labels for an alert is what uniquely identifies an alert within Grafana Alerts.
- The Alertmanager uses labels to match alerts for [silences]({{< relref "../silences/" >}}) and [alert groups]({{< relref "../alert-groups/" >}}) in [notification policies]({{< relref "../notifications/_index.md" >}}).
- The alerting UI displays labels for every alert instance generated by the evaluation of that rule.
- Contact points can access labels to dynamically generate notifications that contain information specific to the alert that is resulting in a notification.
- Labels can be added to an [alerting rule]({{< relref "../alerting-rules/" >}}). These manually configured labels are able to use template functions and reference other labels. Labels added to an alerting rule here take precedence in the event of a collision between labels.
{{< figure src="/static/img/docs/alerting/unified/rule-edit-details-8-0.png" max-width="550px" caption="Alert details" >}}
#### Variables available to alerting rule labels and annotations
The following template variables are available when expanding annotations and labels.
| Name | Description |
| ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| $labels | The labels from the query or condition. For example, `{{ $labels.instance }}` and `{{ $labels.job }}`. This is unavailable when the rule uses a [classic condition]({{< relref "./create-grafana-managed-rule/#single-and-multi-dimensional-rule" >}}). |
| $values | The values of all reduce and math expressions that were evaluated for this alert rule. For example, `{{ $values.A }}`, `{{ $values.A.Labels }}` and `{{ $values.A.Value }}` where `A` is the `refID` of the expression. If the rule uses classic conditions, then a combination of the `refID` and position of the condition is used. For example, `{{ $values.A0.Value }}` or `{{ $values.A1.Value }}` |
| $value | The value string of the alert instance. For example, `[ var='A' labels={instance=foo} value=10 ]`. |

View File

@ -32,7 +32,7 @@ Grafana allows you to create alerting rules that query one or more data sources,
- In **Configure no data and error handling**, configure alerting behavior in the absence of data. Use the guidelines in [No data and error handling](#no-data-and-error-handling).
- Click **Preview alerts** to check the result of running the query at this moment. Preview excludes no data and error handling.
1. In Step 4, add additional metadata associated with the rule.
- Add a description and summary to customize alert messages. Use the guidelines in [Annotations and labels for alerting]({{< relref "./alert-annotation-label.md" >}}).
- Add a description and summary to customize alert messages. Use the guidelines in [Annotations and labels for alerting]({{< relref "../fundamentals/annotation-label/_index.md" >}}).
- Add Runbook URL, panel, dashboard, and alert IDs.
- Add custom labels.
1. Click **Save** to save the rule or **Save and exit** to save the rule and go back to the Alerting page.

View File

@ -35,7 +35,7 @@ For Grafana Mimir and Loki data sources to work with Grafana 8.0 alerting, enabl
- Enter a PromQL or LogQL expression. The rule fires if the evaluation result has at least one series with a value that is greater than 0. An alert is created for each series.
{{< figure src="/static/img/docs/alerting/unified/rule-edit-mimir-query-8-0.png" max-width="550px" caption="Alert details" >}}
1. In Step 3, add additional metadata associated with the rule.
- Add a description and summary to customize alert messages. Use the guidelines in [Annotations and labels for alerting]({{< relref "./alert-annotation-label.md" >}}).
- Add a description and summary to customize alert messages. Use the guidelines in [Annotations and labels for alerting]({{< relref "../fundamentals/annotation-label/_index.md" >}}).
- Add Runbook URL, panel, dashboard, and alert IDs.
- Add custom labels.
1. Click **Save** to save the rule or **Save and exit** to save the rule and go back to the Alerting page.

View File

@ -40,7 +40,7 @@ Grafana allows you to create alerting rules for an external Grafana Mimir or Lok
- In the **For** text box, specify the duration for which the condition must be true before an alert fires. If you specify `5m`, the condition must be true for 5 minutes before the alert fires.
> **Note:** Once a condition is met, the alert goes into the `Pending` state. If the condition remains active for the duration specified, the alert transitions to the `Firing` state, else it reverts to the `Normal` state.
1. In Step 4, add additional metadata associated with the rule.
- Add a description and summary to customize alert messages. Use the guidelines in [Annotations and labels for alerting]({{< relref "./alert-annotation-label.md" >}}).
- Add a description and summary to customize alert messages. Use the guidelines in [Annotations and labels for alerting]({{< relref "../fundamentals/annotation-label/_index.md" >}}).
- Add Runbook URL, panel, dashboard, and alert IDs.
- Add custom labels.
1. To evaluate the rule and see what alerts it would produce, click **Preview alerts**. It will display a list of alerts with state and value of for each one.

View File

@ -6,8 +6,9 @@ weight = 110
# Alerting fundamentals
This section covers the fundamental concepts of Grafana 8 alerting.
This section includes the following fundamental concepts of Grafana alerting:
- [Annotations and labels for alerting rules]({{< relref "./annotation-label/_index.md" >}})
- [Alertmanager]({{< relref "./alertmanager.md" >}})
- [State and health of alerting rules]({{< relref "./state-and-health.md" >}})
- [Evaluating Grafana managed alerts]({{< relref "./evaluate-grafana-alerts.md" >}})

View File

@ -0,0 +1,25 @@
+++
aliases = ["/docs/grafana/latest/alerting/alerting-rules/alert-annotation-label/", "/docs/grafana/latest/alerting/unified-alerting/alerting-rules/alert-annotation-label/"]
description = "Annotations and labels for alerting"
keywords = ["grafana", "alerting", "guide", "rules", "create"]
title = "Annotations and labels for alerting rules"
weight = 401
+++
# Annotations and labels for alerting rules
Annotations and labels are key value pairs associated with alerts originating from the alerting rule, datasource response, and as a result of alerting rule evaluation. They can be used in alert notifications directly or in [templates]({{< relref "../../contact-points/message-templating/_index.md" >}}) and [template functions]({{< relref "../../contact-points/message-templating/template-functions" >}}) to create notification contact dynamically.
## Annotations
Annotations are key-value pairs that provide additional meta-information about an alert. You can use the following annotations: `description`, `summary`, `runbook_url`, `alertId`, `dashboardUid`, and `panelId`. For example, a description, a summary, and a runbook URL. These are displayed in rule and alert details in the UI and can be used in contact point message templates.
## Labels
Labels are key-value pairs that contain information about, and are used to uniquely identify an alert. The label set for an alert is generated and added to throughout the alerting evaluation and notification process.
Before you begin using annotations and labels, familiarize yourself with:
- [Labels in Grafana alerting]({{< relref "./how-to-use-labels.md" >}})
- [How label matching works]({{< relref "./how-to-use-labels.md" >}})
- [Template variables for alerting rule labels and annotations]({{< relref "./variables-label-annotation.md" >}})

View File

@ -0,0 +1,18 @@
+++
title = "Labels in Grafana alerting"
description = "Learn about labels and label matchers in alerting"
keywords = ["grafana", "alerting", "guide", "fundamentals"]
weight = 117
+++
# Labels in Grafana alerting
This topic explains why labels are a fundamental component of alerting.
- The complete set of labels for an alert is what uniquely identifies an alert within Grafana alerts.
- The Alertmanager uses labels to match alerts for [silences]({{< relref "../../silences/_index.md" >}}) and [alert groups]({{< relref "../../alert-groups/_index.md" >}}) in [notification policies]({{< relref "../../notifications/_index.md" >}}).
- The alerting UI shows labels for every alert instance generated during evaluation of that rule.
- Contact points can access labels to dynamically generate notifications that contain information specific to the alert that is resulting in a notification.
- Labels can be added to an [alerting rule]({{< relref "../../alerting-rules/_index.md" >}}). These manually configured labels are able to use template functions and reference other labels. Labels added to an alerting rule take precedence in the event of a collision between labels.
{{< figure src="/static/img/docs/alerting/unified/rule-edit-details-8-0.png" max-width="550px" caption="Alert details" >}}

View File

@ -1,17 +1,13 @@
+++
title = "Labels and label matchers"
title = "Label matchers"
description = "Learn about labels and label matchers in alerting"
keywords = ["grafana", "alerting", "guide", "fundamentals"]
weight = 117
+++
# Labels and label matchers
# How label matching works
To link alert rules to various other alerting concepts like [notification policies]() and [silences]() you make use of labels and label matchers.
This allows for a very flexible way to manage your alert instances, what policy should handle them and which alerts to silence.
## How label matching works
Use labels and label matchers to link alert rules to [notification policies]({{< relref "../../notifications/_index.md" >}}) and [silences]({{< relref "../../silences/_index.md" >}}). This allows for a very flexible way to manage your alert instances, specify which policy should handle them, and which alerts to silence.
A label matchers consists of 3 distinct parts, the **label**, the **value** and the **operator**.
@ -28,16 +24,15 @@ A label matchers consists of 3 distinct parts, the **label**, the **value** and
| `=~` | Select labels that regex-match the value. |
| `!~` | Select labels that do not regex-match the value. |
## Example
## Example of a label matcher
Imagine we've defined the following set of labels for our alert.
`{ foo=bar, baz=qux, id=12 }`
A label matcher defined as `foo=bar` will match this alert rule.
In this situation,
A label matcher defined as `foo!=bar` will _not_ match this alert rule.
A label matcher defined as `id=~[0-9]+` will match this alert rule.
A label matcher defined as `baz!~[0-9]+` will match this alert rule.
- A label matcher defined as `foo=bar` will match this alert rule.
- A label matcher defined as `foo!=bar` will _not_ match this alert rule.
- A label matcher defined as `id=~[0-9]+` will match this alert rule.
- A label matcher defined as `baz!~[0-9]+` will match this alert rule.

View File

@ -0,0 +1,16 @@
+++
title = "Template variables for alerting rule labels and annotations"
description = "Learn about labels and label matchers in alerting"
keywords = ["grafana", "alerting", "guide", "fundamentals"]
weight = 117
+++
# Template variables for alerting rule labels and annotations
The following template variables are available when expanding annotations and labels.
| Name | Description |
| ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| $labels | The labels from the query or condition. For example, `{{ $labels.instance }}` and `{{ $labels.job }}`. This is unavailable when the rule uses a [classic condition]({{< relref "../../alerting-rules/create-grafana-managed-rule/#single-and-multi-dimensional-rule" >}}). |
| $values | The values of all reduce and math expressions that were evaluated for this alert rule. For example, `{{ $values.A }}`, `{{ $values.A.Labels }}` and `{{ $values.A.Value }}` where `A` is the `refID` of the expression. If the rule uses classic conditions, then a combination of the `refID` and position of the condition is used. For example, `{{ $values.A0.Value }}` or `{{ $values.A1.Value }}` |
| $value | The value string of the alert instance. For example, `[ var='A' labels={instance=foo} value=10 ]`. |

View File

@ -48,7 +48,7 @@ You can configure grouping to be `group_by: [alertname]` (take note that the `en
1. Click **Notification policies**.
1. From the **Alertmanager** dropdown, select an Alertmanager. By default, the Grafana Alertmanager is selected.
1. To add a top level specific policy, go to the **Specific routing** section and click **New specific policy**.
1. In **Matching labels** section, add one or more rules for matching alert labels. For more information, see ["Labels and label matchers"]({{< relref "../fundamentals/labels-and-label-matchers.md" >}}).
1. In **Matching labels** section, add one or more rules for matching alert labels. For more information, see ["Labels and label matchers"]({{< relref "../fundamentals/annotation-label/labels-and-label-matchers.md" >}}).
1. In **Contact point**, add the [contact point]({{< relref "../contact-points/_index.md" >}}) to send notification to if alert matches only this specific policy and not any of the nested policies.
1. Optionally, enable **Continue matching subsequent sibling nodes** to continue matching nested policies even after the alert matched the parent policy. When this option is enabled, you can get more than one notification. Use it to send notification to a catch-all contact point as well as to one of more specific contact points handled by nested policies.
1. Optionally, enable **Override grouping** to specify the same grouping as the root policy. If this option is not enabled, the root policy grouping is used.

View File

@ -1,65 +0,0 @@
+++
aliases = ["/docs/grafana/latest/alerting/silences/", "/docs/grafana/latest/alerting/unified-alerting/silences/"]
description = "Silences alert notifications"
keywords = ["grafana", "alerting", "silence", "mute"]
title = "Silences"
weight = 450
+++
# Silences
Use silences to stop notifications from one or more alerting rules. Silences do not prevent alert rules from being evaluated. Nor do they not stop alerting instances from being shown in the user interface. Silences only stop notifications from getting created. A silence lasts for only a specified window of time.
Silences do not prevent alert rules from being evaluated. They also do not stop alert instances being shown in the user interface. Silences only prevent notifications from being created.
You can configure Grafana managed silences as well as silences for an [external Alertmanager data source]({{< relref "../datasources/alertmanager.md" >}}). For more information, see [Alertmanager]({{< relref "./fundamentals/alertmanager.md" >}}).
> **Note:** Before Grafana v8.2, the configuration of the embedded Alertmanager was shared across organisations. Users of Grafana 8.0 and 8.1 are advised to use the new Grafana 8 Alerts only if they have one organisation. Otherwise, silences for the Grafana managed alerts will be visible by all organizations.
## Add a silence
To add a silence:
1. In the Grafana menu, click the **Alerting** (bell) icon to open the Alerting page listing existing alerts.
1. In the Alerting page, click **Silences** to open the page listing existing contact points.
1. From Alertmanager drop-down, select an external Alertmanager to create and manage silences for the external data source. Otherwise, keep the default option of Grafana.
1. Click **New Silence** to open the Create silence page.
1. In **Silence start and end**, select the start and end date to indicate when the silence should go into effect and expire.
1. Optionally, in **Duration**, specify how long the silence is enforced. This automatically updates the end time in the **Silence start and end** field.
1. In the **Name** and **Value** fields, enter one or more _Matching Labels_. Matchers determine which rules the silence will apply to. For more information, see [Label matching for alert suppression](#label-matching-for-alert-suppression).
1. In **Comment**, add details about the silence.
1. In **Creator**, enter the name of the silence owner or keep the default owner.
1. Click **Create**.
### Label matching for alert suppression
Grafana suppresses notifications only for alerts with labels that match all the "Matching Labels" specified in the silence.
- The **Label** field is the name of the label to match. It must exactly match the label name.
- The **Operator** field is the operator to match against the label value. The available operators are:
- `=`: Select labels that are exactly equal to the provided string.
- `!=`: Select labels that are not equal to the provided string.
- `=~`: Select labels that regex-match the provided string.
- `!~`: Select labels that do not regex-match the provided string.
- The **Value** field matches against the corresponding value for the specified **Label** name. How it matches depends on the **Operator** value.
1. In the Alerting page, click **Silences** to view the list of existing silences.
1. Find the silence you want to edit, then click **Edit** (pen icon).
1. Make changes, then click **Submit** to save your changes.
## Remove silences
1. In the Alerting page, click **Silences** to view the list of existing silences.
1. Find the silence you want to end, then click **Unsilence**.
> **Note:** Silences that have ended are retained and listed for five days. You cannot remove a silence manually.
## Create a URL to silence form with defaults filled in
When linking to a silence form, provide the default matching labels and comment via `matchers` and `comment` query parameters. The `matchers` parameter requires one more matching labels of the type `[label][operator][value]` joined by a comma while the `operator` parameter can be one of the following: `=` (equals, not regex), `!=` (not equals, not regex), `=~` (equals, regex), `!~` (not equals, regex).
For example, to link to silence form with matching labels `severity=critical` & `cluster!~europe-.*` and comment `Silence critical EU alerts`, create a URL `https://mygrafana/alerting/silence/new?matchers=severity%3Dcritical%2Ccluster!~europe-*&comment=Silence%20critical%20EU%20alert`.
To link to a new silence page for an [external Alertmanager]({{< relref "../datasources/alertmanager.md" >}}), add a `alertmanager` query parameter with the Alertmanager data source name.

View File

@ -8,15 +8,17 @@ aliases = ["/docs/grafana/latest/alerting/unified-alerting/silences/"]
# Create a silence
Silences stop notifications from getting created and last for only a specified window of time.
To add a silence:
1. In the Grafana menu, click the **Alerting** (bell) icon to open the Alerting page listing existing alerts.
1. In the Alerting page, click **Silences** to open the page listing existing contact points.
1. From Alertmanager drop-down, select an external Alertmanager to create and manage silences for the external data source. Otherwise, keep the default option of Grafana.
1. Click **New Silence** to open the Create silence page.
1. In **Silence start and end**, select the start and end date to indicate when the silence should go into effect and expire.
1. Optionally, in **Duration**, specify how long the silence is enforced. This automatically updates the end time in the **Silence start and end** field.
1. In the **Name** and **Value** fields, enter one or more _Matching Labels_. Matchers determine which rules the silence will apply to. For more information, see [Label matching for alert suppression]({{< relref "./label-matching-alert-suppression.md" >}}).
1. In **Comment**, add details about the silence.
1. In **Creator**, enter the name of the silence owner or keep the default owner.
1. Click **Create**.
2. In the Alerting page, click **Silences** to open the page listing existing contact points.
3. From Alertmanager drop-down, select an external Alertmanager to create and manage silences for the external data source. Otherwise, keep the default option of Grafana.
4. Click **New Silence** to open the Create silence page.
5. In **Silence start and end**, select the start and end date to indicate when the silence should go into effect and expire.
6. Optionally, in **Duration**, specify how long the silence is enforced. This automatically updates the end time in the **Silence start and end** field.
7. In the **Name** and **Value** fields, enter one or more _Matching Labels_. Matchers determine which rules the silence will apply to. For more information, see [Label matching for alert suppression]({{< relref "./label-matching-alert-suppression.md" >}}).
8. In **Comment**, add details about the silence.
9. In **Creator**, enter the name of the silence owner or keep the default owner.
10. Click **Create**.

View File

@ -8,7 +8,11 @@ aliases = ["/docs/grafana/latest/alerting/unified-alerting/silences/"]
# Edit silences
1. I In the Grafana menu, click the **Alerting** (bell) icon to open the Alerting page listing existing alerts.
1. Click **Silences** to view the list of existing silences.
1. Find the silence you want to edit, then click **Edit** (pen icon).
1. Make the desired changes, then click **Submit** to save your changes.
Silences stop notifications from getting created.
To edit a silence:
1. In the Grafana menu, click the **Alerting** (bell) icon to open the Alerting page listing existing alerts.
2. Click **Silences** to view the list of existing silences.
3. Find the silence you want to edit, then click **Edit** (pen icon).
4. Make the desired changes, then click **Submit** to save your changes.

View File

@ -8,6 +8,8 @@ aliases = ["/docs/grafana/latest/alerting/unified-alerting/silences/"]
# Remove silences
Silences stop notifications from getting created.
1. In the Grafana menu, click the **Alerting** (bell) icon to open the Alerting page listing existing alerts.
1. Click **Silences** to view the list of existing silences.
1. Select the silence you want to end, then click **Unsilence**.