IntenseWebs/grafana
3
0
Fork 0
mirror of https://github.com/grafana/grafana.git synced 2025-02-25 18:55:37 -06:00
Code Issues Packages Projects Releases Wiki Activity

Docs: Improved docs for templating labels and annotations (#63539)

Browse Source
This commit is contained in:
George Robinson 2023-02-23 08:51:46 +00:00 committed by GitHub
parent 057d9c45fa
commit c9497dc533
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 221 additions and 135 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

356
docs/sources/alerting/fundamentals/annotation-label/variables-label-annotation.md
View File

@ -3,249 +3,335 @@ description: Learn about templating of labels and annotations
keywords:
- grafana
- alerting
- guide
- fundamentals
- templating
- labels
- annotations
title: Templating labels and annotations
weight: 117
---
# Templating labels and annotations
In Grafana it is possible to template labels and annotations just like in Prometheus. Those who have used Prometheus before should be familiar with `$labels` and `$value` as these variables contain the labels and value of the alert. You can use the same variables in Grafana to template labels and annotations, even if the alert does not use a Prometheus datasource.
In Grafana you template labels and annotations just like you would in Prometheus. If you have used Prometheus before then you should be familiar with the `$labels` and `$value` variables which contain the labels and value of the alert. You can use the same variables in Grafana, even if the alert does not use a Prometheus datasource. If you haven't used Prometheus before then don't worry as each of these variables, and how to template them, will be explained as you follow the rest of this page.
For example, suppose you want to create an alert rule in Grafana that fires when one of your instances is down for more than 5 minutes, and that each alert fired should have a summary annotation to tell you which instance is down:
## Go's templating language
Templates for labels and annotations are written in Go's templating language, [text/template](https://pkg.go.dev/text/template).
### Opening and closing tags
In text/template, templates start with `{{` and end with `}}` irrespective of whether the template prints a variable or executes control structures such as if statements. This is different from other templating languages such as Jinja where printing a variable uses `{{` and `}}` and control structures use `{%` and `%}`.
### Print
To print the value of something use `{{` and `}}`. You can print the the result of a function or the value of a variable. For example, to print the `$labels` variable you would write the following:
```
{{ $labels }}
```
### Iterate over labels
To iterate over each label in `$labels` you can use a `range`. Here `$k` refers to the name and `$v` refers to the value of the current label. For example, if your query returned a label `instance=test` then `$k` would be `instance` and `$v` would be `test`.
```
{{ range $k, $v := $labels }}
{{ $k }}={{ $v }}
{{ end }}
```
## The labels, value and values variables
### The labels variable
The `$labels` variable contains the labels from the query. For example, a query that checks if an instance is down might return an instance label with the name of the instance that is down. For example, suppose you have an alert rule that fires when one of your instances has been down for more than 5 minutes. You want to add a summary to the alert that tells you which instance is down. With the `$labels` variable, you can create a summary that prints the instance label in the summary:
```
Instance {{ $labels.instance }} has been down for more than 5 minutes
```
## Labels with dots
### Labels with dots
If the label contains a dot (full stop or period) in its name then the following will not work:
If the label you want to print contains a dot (full stop or period) in its name using the same dot in the template will not work:
```
Instance {{ $labels.instance.name }} has been down for more than 5 minutes
```
This is because it is attempting to use a non-existing field `name` in `$labels.instance` rather than `instance.name` in `$labels`. Instead use the `index` function to print `instance.name`:
This is because the template is attempting to use a non-existing field called `name` in `$labels.instance`. You should instead use the `index` function, which prints the label `instance.name` in the `$labels` variable:
```
Instance {{ index $labels "instance.name" }} has been down for more than 5 minutes
```
## Use values in labels and annotations
### The value variable
Grafana supports `$value` when templating labels and annotations. However, while `$value` in Prometheus is a floating point number contains the value of the expression, `$value` in Grafana is a string containing the labels and values of all Threshold, Reduce and Maths expressions. It does not contain the value of queries as a single query can return anywhere from 1 to 10,000s of rows or metrics.
The `$value` variable works different from Prometheus. In Prometheus `$value` is a floating point number containing the value of the expression, but in Grafana it is a string containing the labels and values of all Threshold, Reduce and Math expressions, and Classic Conditions for this alert rule. It does not contain the results of queries, as these can return anywhere from 10s to 10,000s of rows or metrics.
This `$value` variable is called the Value String. If you were to use it in the template of a summary annotation:
If you were to use the `$value` variable in the summary of an alert:
```
{{ $labels.instance }} has an average 95th percentile request latency above 1s: {{ $value }})
{{ $labels.service }} has over 5% of responses with 5xx errors: {{ $value }})
```
you will get the following summary:
The summary might look something like the following:
```
http_server has an average 95th percentile request latency above 1s: [ var='B' labels={instance=http_server} value=10 ]
api has an over 5% of responses with 5xx errors: [ var='B' labels={service=api} value=6.789 ]
```
To get just the value of `B` you can instead use `$values`:
Here `var='B'` refers to the expression with the RefID B. In Grafana, all queries and expressions are identified by a RefID that identifies each query and expression in an alert rule. Similarly `labels={service=api}` refers to the labels, and `value=6.789` refers to the value.
You might have observed that there is no RefID A. That is because in most alert rules the RefID A refers to a query, and since queries can return many rows or time series they are not included in `$value`.
### The values variable
If the `$value` variable contains more information than you need, you can instead print the labels and value of individual expressions using `$values`. Unlike `$value`, the `$values` variable is a table of objects containing the labels and floating point values of each expression, indexed by their RefID.
If you were to print the value of the expression with RefID `B` in the summary of the alert:
```
{{ $labels.instance }} has an average 95th percentile request latency above 1s: {{ $values.B }}
{{ $labels.service }} has over 5% of responses with 5xx errors: {{ $values.B }}%
```
and then you will get this summary:
The summary will contain just the value:
```
http_server has an average 95th percentile request latency above 1s: 11
api has an over 5% of responses with 5xx errors: 6.789%
```
## Alert rules with multiple queries, or expressions
If you have an alert rule with multiple queries and expressions, or a single query with a Reduce and Math expression, like in the following example:
{{< figure src="/static/img/docs/alerting/unified/grafana-alerting-histogram-quantile.png" class="docs-image--no-shadow" caption="An alert rule that uses histogram_quantile to compute 95th percentile" >}}
Then the Value String will not just include the value of the alert condition, but the labels and values of all Threshold, Reduce and Maths expressions.
**Example 1**: The Value String of an alert rule with a single query and Reduce expression `B`:
However, while `{{ $values.B }}` prints the number 6.789, it is actually a string as you are printing the object that contains both the labels and value for RefID B, not the floating point value of B. To use the floating point value of RefID B you must use the `Value` field from `$values.B`. If you were to humanize the floating point value in the summary of an alert:
```
[ var='B' labels={instance=http_server} value=11 ]
{{ $labels.service }} has over 5% of responses with 5xx errors: {{ humanize $values.B.Value }}%
```
**Example 2**: The Value String of an alert rule with a single query, Reduce expression `B` and a Math expression `C`:
### No data, execution errors and timeouts
If the query in your alert rule returns no data, or fails because of a datasource error or timeout, then any Threshold, Reduce or Math expressions that use that query will also return no data or an error. When this happens these expression will be absent from `$values`. It is good practice to check that a RefID is present before using it as otherwise your template will break should your query return no data or an error. You can do this using an if statement:
```
[ var='B' labels={instance=http_server} value=11, var='C' labels={instance=http_server} value=1 ]
```
If you were to write a summary annotation such as:
```
{{ $labels.instance }} has an average 95th percentile request latency above 1s: {{ $values.C }})
```
You would find that because the condition of the alert `C` is a Math expression with a boolean comparison, it must return either a `0` or a `1`. What you want instead is the average of the 95th percentile, and you can get this from the reduce expression `B`:
```
{{ $labels.instance }} has an average 95th percentile request latency above 1s: {{ $values.B }})
```
## No data and execution errors or timeouts
Should query `A` return no data then the reduce expression `B` will also return no data. This means that
`{{ $values.B }}` will be nil. To ensure that labels and annotations can still be templated even when a query returns no data, we can use an if statement to check for this condition:
```
{{ if $values.B }}{{ $labels.instance }} has a 95th percentile request latency above 1s: {{ $values.B }}){{ end }}
{{ if $values.B }}{{ $labels.service }} has over 5% of responses with 5xx errors: {{ humanizePercentage $values.B.Value }}{{ end }}
```
## Classic Conditions
If the rule uses Classic Conditions instead of Reduce and Math expressions, then `$values` contains the combination of the Ref ID and position of the condition. For example, `{{ $values.A0 }}` and `{{ $values.A1 }}`.
If the rule uses Classic Conditions instead of Threshold, Reduce and Math expressions, then the `$values` variable is indexed by both the Ref ID and position of the condition in the Classic Condition. For example, if you have a Classic Condition with RefID B containing two conditions, then `$values` will contain two conditions `B0` and `B1`.
## Reference
```
The first condition is {{ $values.B0 }}, and the second condition is {{ $values.B1 }}
```
### Variables
The following template variables are available when expanding labels and annotations:
| Name | Description |
| ------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| $labels | The labels from the query or condition. For example, `{{ $labels.instance }}` and `{{ $labels.job }}`. This is unavailable when the rule uses a [classic condition]({{< relref "../../alerting-rules/create-grafana-managed-rule/#single-and-multi-dimensional-rule" >}}). |
| $values | The values of all reduce and math expressions that were evaluated for this alert rule. For example, `{{ $values.A }}`, `{{ $values.A.Labels }}` and `{{ $values.A.Value }}` where `A` is the `refID` of the reduce or math expression. If the rule uses a classic condition instead of a reduce and math expression, then `$values` contains the combination of the `refID` and position of the condition. |
| $value | The value string of the alert instance. For example, `[ var='A' labels={instance=foo} value=10 ]`. |
### Functions
## Functions
The following functions are also available when expanding labels and annotations:
| Name | Argument type | Return type | Description |
| ----------------------------------------- | ------------------------------------------------------------ | ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- |
| [humanize](#humanize) | number or string | string | Converts a number to a more readable format, using metric prefixes. |
| [humanize1024](#humanize1024) | number or string | string | Like humanize, but uses 1024 as the base rather than 1000. |
| [humanizeDuration](#humanizeduration) | number or string | string | Converts a duration in seconds to a more readable format. |
| [humanizePercentage](#humanizepercentage) | number or string | string | Converts a ratio value to a fraction of 100. |
| [humanizeTimestamp](#humanizetimestamp) | number or string | string | Converts a Unix timestamp in seconds to a more readable format. |
| [title](#title) | string | string | strings.Title, capitalises first character of each word. |
| [toUpper](#toupper) | string | string | strings.ToUpper, converts all characters to upper case. |
| [toLower](#tolower) | string | string | strings.ToLower, converts all characters to lower case. |
| [match](#match) | pattern, text | boolean | regexp.MatchString Tests for a unanchored regexp match. |
| [reReplaceAll](#rereplaceall) | pattern, replacement, text | string | Regexp.ReplaceAllString Regexp substitution, unanchored. |
| [graphLink](#graphlink) | string - JSON Object with `"expr"` and `"datasource"` fields | string | Returns the path to graphical view in [Explore](https://grafana.com/docs/grafana/latest/explore/) for the given expression and data source. |
| [tableLink](#tablelink) | string- JSON Object with `"expr"` and `"datasource"` fields | string | Returns the path to tabular view in [Explore](https://grafana.com/docs/grafana/latest/explore/) for the given expression and data source. |
| [args](#args) | []interface{} | map[string]interface{} | Converts a list of objects to a map with keys, for example, arg0, arg1. Use this function to pass multiple arguments to templates. |
| [externalURL](#externalurl) | nothing | string | Returns a string representing the external URL. |
| [pathPrefix](#pathprefix) | nothing | string | Returns the path of the external URL. |
### args
#### humanize
The `args` function translates a list of objects to a map with keys arg0, arg1 etc. This is intended to allow multiple arguments to be passed to templates.
**Template string** `{ humanize $value }`
#### Example
**Input** `1234567.0`
```
{{define "x"}}{{.arg0}} {{.arg1}}{{end}}{{template "x" (args 1 "2")}}
```
**Expected** `1.235M`
```
1 2
```
#### humanize1024
### externalURL
**TemplateString** `{ humanize1024 $value } `
The `externalURL` function returns the external URL of the Grafana server as configured in the ini file(s).
**Input** `1048576.0`
#### Example
**Expected** `1Mi`
```
{{ externalURL }}
```
#### humanizeDuration
```
https://example.com/grafana
```
**TemplateString** `{ humanizeDuration $value }`
### graphLink
**Input** `899.99`
The `graphLink` function returns the path to the graphical view in [Explore](https://grafana.com/docs/grafana/latest/explore/) for the given expression and data source.
**Expected** `14m 59s`
#### Example
#### humanizePercentage
```
{{ graphLink "{\"expr\": \"up\", \"datasource\": \"gdev-prometheus\"}" }}
```
**TemplateString** `{ humanizePercentage $value }`
```
/explore?left=["now-1h","now","gdev-prometheus",{"datasource":"gdev-prometheus","expr":"up","instant":false,"range":true}]
```
**Input** `0.1234567`
### humanize
**Expected** `12.35%`
The `humanize` function humanizes decimal numbers.
#### humanizeTimestamp
#### Example
**TemplateString** `{ $value | humanizeTimestamp }`
```
{{ humanize 1000.0 }}
```
**Input** `1435065584.128`
```
1k
```
**Expected** `2015-06-23 13:19:44.128 +0000 UTC`
### humanize1024
#### title
The `humanize1024` works similar to `humanize` but but uses 1024 as the base rather than 1000.
**TemplateString** `{ $value | title }`
#### Example
**Input** `aa bb CC`
```
{{ humanize1024 1024.0 }}
```
**Expected** `Aa Bb Cc`
```
1ki
```
#### toUpper
### humanizeDuration
**TemplateString** `{ $value | toUpper }`
The `humanizeDuration` function humanizes a duration in seconds.
**Input** `aa bb CC`
#### Example
**Expected** `AA BB CC`
```
{{ humanizeDuration 60.0 }}
```
#### toLower
```
1m 0s
```
**TemplateString** `{ $value | toLower }`
### humanizePercentage
**Input** `aA bB CC`
The `humanizePercentage` function humanizes a ratio value to a percentage.
**Expected** `aa bb cc`
#### Example
#### match
```
{{ humanizePercentage 0.2 }}
```
**TemplateString** `{ match "a+" $labels.instance }`
```
20%
```
**Input** `aa`
### humanizeTimestamp
**Expected** `true`
The `humanizeTimestamp` function humanizes a Unix timestamp.
#### reReplaceAll
#### Example
**TemplateString** `{{ reReplaceAll "localhost:(.*)" "my.domain:$1" $labels.instance }}`
```
{{ humanizeTimestamp 1577836800.0 }}
```
**Input** `localhost:3000`
```
2020-01-01 00:00:00 +0000 UTC
```
**Expected** `my.domain:3000`
### match
#### graphLink
The `match` function matches the text against a regular expression pattern.
**TemplateString** `{{ graphLink "{\"expr\": \"up\", \"datasource\": \"gdev-prometheus\"}" }}`
#### Example
**Expected** `/explore?left=["now-1h","now","gdev-prometheus",{"datasource":"gdev-prometheus","expr":"up","instant":false,"range":true}]`
```
{{ match "a.*" "abc" }}
```
#### tableLink
```
true
```
**TemplateString** `{{ tableLink "{\"expr\": \"up\", \"datasource\": \"gdev-prometheus\"}" }}`
### pathPrefix
**Expected** `/explore?left=["now-1h","now","gdev-prometheus",{"datasource":"gdev-prometheus","expr":"up","instant":true,"range":false}]`
The `pathPrefix` function returns the path of the Grafana server as configured in the ini file(s).
#### args
#### Example
**TemplateString** `{{define "x"}}{{.arg0}} {{.arg1}}{{end}}{{template "x" (args 1 "2")}}`
```
{{ pathPrefix }}
```
**Expected** `1 2`
```
/grafana
```
#### externalURL
### tableLink
**TemplateString** `{ externalURL }`
The `tableLink` function returns the path to the tabular view in [Explore](https://grafana.com/docs/grafana/latest/explore/) for the given expression and data source.
**Expected** `http://localhost/path/prefix`
#### Example
#### pathPrefix
```
{{ tableLink "{\"expr\": \"up\", \"datasource\": \"gdev-prometheus\"}" }}
```
**TemplateString** `{ pathPrefix }`
```
/explore?left=["now-1h","now","gdev-prometheus",{"datasource":"gdev-prometheus","expr":"up","instant":true,"range":false}]
```
**Expected** `/path/prefix`
### title
The `title` function capitalizes the first character of each word.
#### Example
```
{{ title "hello, world!" }}
```
```
Hello, World!
```
### toLower
The `toLower` function returns all text in lowercase.
#### Example
```
{{ toLower "Hello, world!" }}
```
```
hello, world!
```
### toUpper
The `toUpper` function returns all text in uppercase.
#### Example
```
{{ toLower "Hello, world!" }}
```
```
HELLO, WORLD!
```
### reReplaceAll
The `reReplaceAll` function replaces text matching the regular expression.
#### Example
```
{{ reReplaceAll "localhost:(.*)" "example.com:$1" "localhost:8080" }}
```
```
example.com:8080
```