From 406f0458b21b31751ee03252491a1c6ddcf72942 Mon Sep 17 00:00:00 2001 From: Kyle Brandt Date: Thu, 3 Jun 2021 12:15:24 -0400 Subject: [PATCH] Alerting/Docs: Non-timeseries / numeric alerting (#35190) Co-authored-by: Sofia Papagiannaki Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com> --- .../create-grafana-managed-rule.md | 2 +- .../grafana-managed-numeric-rule.md | 67 +++++++++++++++++++ 2 files changed, 68 insertions(+), 1 deletion(-) create mode 100644 docs/sources/alerting/unified-alerting/grafana-managed-numeric-rule.md diff --git a/docs/sources/alerting/unified-alerting/create-grafana-managed-rule.md b/docs/sources/alerting/unified-alerting/create-grafana-managed-rule.md index b5b634cc6e7..a1f81e2b116 100644 --- a/docs/sources/alerting/unified-alerting/create-grafana-managed-rule.md +++ b/docs/sources/alerting/unified-alerting/create-grafana-managed-rule.md @@ -58,7 +58,7 @@ So, as you can see from the above scenario Grafana will not send out notificatio You can use reduce and math expressions to create a rule that will create an alert per series returned by the query. 1. Add one or more queries -2. Add a `reduce` expression for each query to aggregate values in the selected time range into a single value. Not needed in case a query returns a single value per series. +2. Add a `reduce` expression for each query to aggregate values in the selected time range into a single value. With some data sources this is not needed for [rules using numeric data]({{< relref "./grafana-managed-numeric-rule.md" >}}). 3. Add a `math` expressions with the condition for the rule. Not needed in case a query or a reduce expression already returns 0 if rule should not be firing, or > 0 if it should be firing. Some examples: `$B > 70` if it should fire in case value of B query/expression is more than 70. `$B < $C * 100` in case it should fire if value of B is less than value of C multiplied by 100. If queries being compared have multiple series in their results, series from different queries are matched if they have the same labels or one is a subset of the other. See or [expressions documentation]({{< relref "../../panels/expressions.md" >}}) for in depth explanation of `math` and `reduce` expressions. diff --git a/docs/sources/alerting/unified-alerting/grafana-managed-numeric-rule.md b/docs/sources/alerting/unified-alerting/grafana-managed-numeric-rule.md new file mode 100644 index 00000000000..d1389a545e3 --- /dev/null +++ b/docs/sources/alerting/unified-alerting/grafana-managed-numeric-rule.md @@ -0,0 +1,67 @@ ++++ +title = "Grafana managed alert rules for numeric data" +description = "Grafana managed alert rules for numeric data" +keywords = ["grafana", "alerting", "guide", "rules", "create"] +weight = 400 ++++ + +# Alerting on numeric data + +Among certain data sources numeric data that is not time series can be directly alerted on, or passed into Server Side Expressions (SSE). This allows for more processing and resulting efficiency within the data source, and it can also simplify alert rules. +When alerting on numeric data instead of time series data, there is no need to reduce each labeled time series into a single number. Instead labeled numbers are returned to Grafana instead. + +## Tabular Data + +This feature is supported with backend data sources that query tabular data: + +- SQL data sources such as MySQL, Postgres, MSSQL, and Oracle. +- The Azure Kusto based services: Azure Monitor (Logs), Azure Monitor (Azure Resource Graph), and Azure Data Explorer. + +A query with Grafana managed alerts or SSE is considered numeric with these data sources, if: + +- The "Format AS" option is set to "Table" in the data source query. +- The table response returned to Grafana from the query includes only one numeric (e.g. int, double, float) column, and optionally additional string columns. + +If there are string columns then those columns become labels. The name of column becomes the label name, and the value for each row becomes the value of the corresponding label. If multiple rows are returned, then each row should be uniquely identified their labels. + +## Example + +For a MySQL table called "DiskSpace": + +| Time | Host | Disk | PercentFree +| ----------- | --- | -----| -------- +| 2021-June-7 | web1 | /etc | 3 +| 2021-June-7 | web2 | /var | 4 +| 2021-June-7 | web3 | /var | 8 +| ... | ... | ... | ... + +You can query the data filtering on time, but without returning the time series to Grafana. For example, an alert that would trigger per Host, Disk when there is less than 5% free space: + +```sql +SELECT Host, Disk, CASE WHEN PercentFree < 5.0 THEN PercentFree ELSE 0 END FROM ( + SELECT + Host, + Disk, + Avg(PercentFree) + FROM DiskSpace + Group By + Host, + Disk + Where __timeFilter(Time) +``` + +This query returns the following Table response to Grafana: + +| Host | Disk | PercentFree +| --- | -----| -------- +| web1 | /etc | 3 +| web2 | /var | 4 +| web3 | /var | 0 + +When this query is used as the **condition** in an alert rule, then the non-zero will be alerting. As a result, three alert instances are produced: + +| Labels | Status +| ----------------------| ------ +| {Host=web1,disk=/etc} | Alerting +| {Host=web2,disk=/var} | Alerting +| {Host=web3,disk=/var} | Normal