Commit Graph
100 Commits
Author SHA1 Message Date
George Robinson b6db1ed524 Revert "Alerting: Add is_paused attr to the POST alert rule group endpoint" (#62310)
Revert "Alerting: Add is_paused attr to the POST alert rule group endpoint (#62253)"

This reverts commit 3ccafe3a5a.
2023-01-27 13:41:36 +01:00
George Robinson a7eab8e46e Alerting: Support context.Context in Loki interface (#61979)
This commit adds support for canceleable contexts in the Loki
interface.
2023-01-26 09:31:20 +00:00
George Robinson 239d94205a Alerting: Return chan <-error for #61811 (#61858) 2023-01-24 15:41:38 +00:00
George Robinson d4256b352d Docs: Rename Message templates to Notification templates (#59477)
This commit renames "Message templates" to "Notification templates"
in the user interface as it suggests that these templates cannot
be used to template anything other than the message. However, message
templates are much more general and can be used to template other fields
too such as the subject of an email, or the title of a Slack message.
2023-01-18 17:26:34 +00:00
George Robinson fa9f1c3a52 Docs: Re-order nav for Manage your alert notifications (#60897) 2023-01-11 08:44:01 +00:00
George Robinson 2a291afbae Alerting: Use consts from alerting package (#61241) 2023-01-10 19:59:13 +00:00
George Robinson d19d8c6625 Alerting: Update Alerting and Alertmanager to v0.25.1 (#61233)
Update Alerting and Alertmanager to v0.25.1
2023-01-10 16:17:07 +00:00
George Robinson 35ad9e23ce Docs: Update weights for Customize notifications without changing the order of pages (#60898) 2023-01-10 11:07:00 +00:00
George Robinson 402dc5e4d6 Alerting: Redo refactoring from reverted fix in #56812 (#61051)
This pull request re-applies the refactoring of ConditionsCmd from a
reverted fix #56812 for mathexp.noData. It does not add the fix, or
tests for the fix, because those were added in #56816. We use the
additional test coverage added in #56816 and #58650 to avoid the
reoccurrence of regressions that caused us to revert #56812 the
first time.
2023-01-09 17:01:19 +00:00
George Robinson 9af7adef76 Alerting: Support customizable timeout for screenshots (#60981)
This commit adds a customizable timeout for screenshots called
capture_timeout. The default value is 10 seconds, and the maximum
value is 30 seconds. This timeout should be less than the minimum
Interval of all Evaluation Groups to avoid back pressure on alert
rule evaluation.
2023-01-05 16:07:46 +00:00
George Robinson 1a442de094 Docs: Rephrase opening sentence for Customize notifications (#60560) 2022-12-21 10:51:02 +00:00
George Robinson 5d4e35c3d5 Docs: Fix links in Using Go's templating language (#60564) 2022-12-20 12:30:43 +00:00
George Robinson d7b555c405 Docs: Remove old message templates documentation (#60548) 2022-12-20 08:23:42 +00:00
f6aea16894 Docs: Templating notifications (#60109)
* Docs: Templating notifications

* Feedback

* More feedback

* Edit template notifications

* More edits

* Fix typo

Co-authored-by: brendamuir <100768211+brendamuir@users.noreply.github.com>
Co-authored-by: Armand Grillet <2117580+armandgrillet@users.noreply.github.com>
2022-12-19 15:01:18 +00:00
George Robinson 342b0f3668 Alerting: Rename contact point type to receiver in the user interface (#59589)
This commit renames contact point type to integration in the user interface
as "New contact point type" suggests that you are creating a new type,
when instead you are just creating a new integration of a pre-determined type,
such as email or Slack. You can also have two or more integrations of the same
type, something which is not apparent from the current text.

This change will also make it easier to explain the difference between
a contact point integration, and how to configure an integration of a certain type,
such as email.
2022-12-14 20:50:36 +00:00
George Robinson 76601f3ae7 Alerting: Better define how we set states (#59977)
This commit better defines how we set states in resultNormal,
resultAlerting, resultError and resultNoData. It changes the existing
code to call methods such as SetAlerting, SetPending, SetNormal,
SetError and NoData instead of assigning values to each individual field
whenever the state is changed. This should make it easier to understand
what fields should be set for which states and avoid cases where states are
missing, or have additional unexpected fields.
2022-12-08 20:12:13 +00:00
George Robinson 6359dab040 Alerting: Change resultError in preparation for supporting ForError duration (#59894) 2022-12-07 10:45:56 +00:00
George Robinson 3c249e1b99 Fix incorrect start time for DatasourceError alerts (#59903) 2022-12-06 18:44:06 +00:00
George Robinson ec1d93c8ab Alerting: Upload images to Slack via files.upload (#59163)
This commit makes a number of changes to how images work in Slack
notifications.

It adds support for uploading images to Slack via the files.upload
API when the contact point has a token. Images are no longer linked
via a URL if a token is present.

Each image uploaded to Slack is posted as a reply to the original
notification. Up to maxImagesPerThreadTs images can be posted as
replies before a final message is sent with:

  There are no images than can be shown here. To see the panels for
  all firing and resolved alerts please check Grafana

Incoming Webhooks cannot upload files via files.upload and so webhooks
require the image to be uploaded to cloud storage and linked via URL.
2022-12-02 09:41:24 +00:00
George Robinson 311e1e56f2 Docs: Reference Alertmanagers in Notifications docs and move Alertmanager docs (#59673) 2022-12-02 09:41:01 +00:00
George Robinson 4073d044c1 Docs: Consolidate Templating labels and annotations docs into a single page (#59685) 2022-12-02 09:40:48 +00:00
George Robinson 8ed2426e8e Docs: Improve documentation for notifications (#59504) 2022-12-01 09:57:45 +00:00
George Robinson c8c1499cd0 Docs: Add docs for labels with dots (#59352) 2022-11-29 13:15:48 +00:00
George Robinson 110fdf4da9 Alerting: Fix ConditionsCmd behavior when last is No Data (#58953) 2022-11-28 21:56:22 +00:00
George Robinson 0af3515e95 Docs: Improve docs for images in notifications (#59033) 2022-11-28 14:11:00 +00:00
George Robinson e7316ff13f Docs: Improve Labels and annotations docs (#59325)
This commit makes a number of changes to the docs for Labels and
annotations.

 1. It changes the order in which Labels and annotations are mentioned
    from Annotations and labels to Labels and annotations as this is
    the order shown in the UI when creating and editing alert rules.
    It is also the order in the Prometheus documentation.

 2. It changes most of the documentation explaining what labels and
    annotations are and the differences between them. It also adds
    some paragraphs on Custom Labels.
2022-11-28 14:09:02 +00:00
George Robinson c9286ff794 Alerting: Remove comments for fixed behavior (#58951) 2022-11-23 15:37:15 +01:00
George Robinson 7376eee6ff Alerting: Support From and To when taking screenshots (#59166)
This commit adds support for the From and To parameters that set the
time range for the graph. The default is 6 hours, which can sometimes
be too large of a time range.
2022-11-23 09:28:24 +00:00
George Robinson 2f878acd9d Alerting: Add Troubleshooting to Images in notifications docs (#58955) 2022-11-21 11:21:59 +00:00
George Robinson 4da97aef7d Alerting: Fix metric in Images in Notifications docs (#58954) 2022-11-18 16:28:00 +00:00
George Robinson b77c3946a5 Alerting: Fix ConditionsCmd No Data for "has no value" (#58634)
This commit fixes a bug where ConditionsCmd returns No Data even when
the condition checks for "has no value". It should return 1 with a nil
match.
2022-11-18 09:28:21 +00:00
George Robinson 48c34d310c Alerting: Add tests that check current No Data behaviour with two conditions (#58650) 2022-11-18 09:04:43 +00:00
George Robinson bd87b46b15 Alerting: Improve test coverage for ConditionsCmd (#58603) 2022-11-11 09:27:35 +00:00
George Robinson c5ae1bcfe0 Alerting: Fix logging pointer address of DashboardUID and PanelID variables (#58539) 2022-11-10 09:58:38 +00:00
George Robinson 68600c224b Alerting: Log when alert rule cannot be screenshot to help debugging (#58537) 2022-11-10 09:41:31 +00:00
George Robinson 7e852720e3 Alerting: Fix images cached on rule instead of dashboard panel signature (#58510) 2022-11-09 17:01:48 +00:00
George Robinson b92a0223e3 Alerting: Improve debug logs in image service (#58507) 2022-11-09 16:32:58 +00:00
George Robinson acfa066ba2 Alerting: Change TestConditionsCmd to assert on mathexp.Results (#58434) 2022-11-09 16:07:43 +00:00
George Robinson 1290951b65 Alerting: Small improvements to staleResultsHandler (#58007) 2022-11-09 11:08:32 +00:00
George Robinson fdeefaee42 Alerting: Reorder tests in classic_test.go (#58425) 2022-11-09 09:13:47 +00:00
George Robinson c646ff0ce3 Alerting: Fix screenshots were not cached (#58493) 2022-11-09 01:52:16 +00:00
George Robinson ad9ac85ee0 Alerting: Use hash of opts in singleflight (#58474) 2022-11-08 22:37:49 +00:00
George Robinson 72275e97d2 Use fnv64 for InmemCacheService (#58468) 2022-11-08 22:05:15 +00:00
George Robinson aa69a8463f Revert "Alerting: Fix mathexp.NoData in ConditionsCmd (#56812)" (#58423)
This reverts commit 5fa0936b7e.
2022-11-08 13:35:58 +00:00
George Robinson 5fa0936b7e Alerting: Fix mathexp.NoData in ConditionsCmd (#56812)
This commit fixes an issue where mathexp.NoData would return an error
in ConditionsCmd (Classic Condition) instead of no data. It further
refactors the Execute method to make it easier to understand.
2022-11-08 11:41:57 +00:00
George Robinson 8353f307aa Alerting: Fix test fails in some environments (#58251) 2022-11-07 16:34:37 +00:00
George Robinson f2e4cb7c4e Alerting: Fix feedback (#57922) 2022-11-02 22:36:14 +00:00
George Robinson 215ffee437 Alerting: Fix screenshot is not taken for stale series (#57982) 2022-11-02 22:14:22 +00:00
George Robinson 4c581b5f85 Alerting: Fix response is not returned for invalid Duration in Provisioning API (#58046) 2022-11-02 08:21:23 -04:00
George Robinson b0a927b138 Alerting: Add debug logs in validateAndGetPrefix (#57002) 2022-10-31 16:40:28 +00:00
George Robinson 2f85172718 Alerting: Remove blank comment (#56889) 2022-10-14 13:28:41 +01:00
George Robinson 004bb7689d Alerting: Fix mathexp.NoData for ConditionsCmd (#56816) 2022-10-12 17:34:28 +01:00
George Robinson 67d93ceea0 Alerting: Add doc comments to classic.go (#56724) 2022-10-12 10:55:48 +01:00
George Robinson 52965de369 Alerting: Add doc comments to state struct and normalize fields (#56647) 2022-10-11 09:30:33 +01:00
George Robinson 802d67eeca Alerting: Support values in notification templates (#56457)
We have received a lot of feedback regarding the ValueString in alert notifications. Perhaps one of the most frequent complaints about ValueString is that it is difficult to read because it contains a lot of information, and the information is shown as a JSON-like string. Users have often asked how it can be templated and the answer is that it can't.

Until now users have been able to add custom annotations to their alert rules which contains values via the $values variable added in previous versions of Grafana. However, these custom annotations must be added for each of the user's alert rule, instead of once in a template that all of their alerts can be notified via.

This commit adds then the much requested feature to support values in notification templates. Users can then create a single template that prints the annotations, labels and values of their alerts in a format of their choice!
2022-10-10 13:40:21 +01:00
George Robinson 762688d67f Alerting: Fix pq: missing FROM-clause for table "a" (#56453)
This commit fixes a bug where changing the Folder or Rule Group of an existing rule returns the following error in PostgreSQL "pq: missing FROM-clause for table a"
2022-10-07 10:18:49 +01:00
George Robinson a49fcbdbbc Alerting: Add frames for all queries and expressions (#55609)
This commit is one of two commits to make the data frames for all queries and expressions in an alert rule available to the state package for rendering a graph. It renames Result to Condition, and creates an additional field called
Results that is a map of Ref ID to data.Frames.
2022-09-27 10:05:29 +01:00
George Robinson bad4f7fec5 Alerting: Change screenshots to use components (#55156)
* Alerting: Change screenshots to use components

This commit changes screenshots to use a number of components instead of a set of functional wrappers.

It moves the uploading of screenshots from the screenshot package to the image package so we can re-use the same code for both uploading screenshots and server-side images; SingleFlight from the screenshot package to the image package so we can use it for both taking and uploading the screenshot, where as before it was used just for taking the screenshot; and it also removes the use of a cache because we know that screenshots can be taken at most once per tick of the scheduler.
2022-09-21 10:25:07 +01:00
George Robinson 7d20766ae9 Alerting: Fix mathexp.NoData cannot be reduced (#55347)
This commit fixes a bug where queries from datasources such as InfluxDB that returned no data would not create a DatasourceNoData alert, but instead an error "can only reduce type series, got type noData".
2022-09-21 10:01:51 +01:00
George Robinson 4a1ca7e32f Alerting: Change default alert state to Error on execution error or timeout (#55345) 2022-09-19 10:42:04 +01:00
George Robinson 5561f935e6 Alerting: Fix send resolved notifications (#54793)
This commit fixes a bug where we did not send resolved alerts to Alertmanager for resolved alert instances. This meant that resolved notifications did not have the annotations from the resolved state, and a result did not also have the resolved screenshot.
2022-09-15 17:25:05 +01:00
George Robinson 77e53f9986 Alerting: Fix boolean comparison on PostgreSQL (#54730) 2022-09-06 08:28:42 +01:00
George Robinson c4d32dd687 Fix nil logger in SqlStore (#54726) 2022-09-05 18:24:19 +01:00
George Robinson e163a976fa Alerting: Fix width of Adapative Cards in Teams notifications (#53996) 2022-08-22 16:04:22 +01:00
George Robinson c960301aa8 Alerting: Fix links in Microsoft Teams notifications (#54003) 2022-08-22 12:08:38 +02:00
George Robinson 6d69d7acb8 Alerting: Fix notifications for Microsoft Teams (#53810)
This commit fixes notifications for Microsoft Teams where notifications
would just contain "Card" rather than the title.
2022-08-17 16:46:46 +01:00
George Robinson 5e1d628f21 Alerting: Use Adaptive Cards in Teams notifications (#53532)
This commit changes the cards in Teams notifications from Office 365
Connector cards to Adaptive Cards to fix an issue where images were not
shown in Teams for desktop and web. Since Office 365 Connector cards
are deprecated, it made sense to move to Adapative Cards and fix this
bug at the same time.

The Adaptive Card messages maintain the design of the Office 365
Connector Card with a number of minor differences:

- In Adaptive Card messages the color of the title is red or green
  depending on the status of the alerts, where as with Office 365
  connector cards there was a colored border at the top of the title

- In Adaptive Card messages the title is bold to make it easier to read
  when the color is red or green

- In Adaptive Card messages the thumbnails for images are medium size
  if there are more than two images, otherwise large size
2022-08-10 19:51:20 +01:00
George Robinson 196b781c70 Alerting: Delete expired images from the database (#53236)
This commit adds a DeleteExpiredService that deletes expired images from the database. It is run in the periodic collector service.
2022-08-09 15:28:36 +01:00
George Robinson 23fc5383d9 Docs: Update docs for images in notifications (#52721) 2022-07-26 10:09:48 +01:00
George Robinson 0af09b8c57 Alerting: Add support for images in Line alerts (#50703) 2022-07-21 11:43:06 +02:00
George Robinson 094dfd0d4d Alerting: Fix Slack push notifications (#52391)
This commit fixes push notifications for Slack which used to show "This content cannot be displayed". The text field is shown in both the message and the push notification.
2022-07-19 17:42:24 +01:00
George Robinson e7feff6d99 Alerting: Move debug log line to where alert rules are updated (#52318) 2022-07-18 11:27:06 -04:00
George Robinson 91fd0223a4 Datasources: Allow configuration of the TTL (#52161) 2022-07-15 10:48:52 +01:00
George Robinson 34d45977ca Alerting: Fix bug where state did not change between Alerting and Error (#52204)
This commit fixes a bug where the state did not change from Alerting to Error if the evaluation result returned an error, or from Error to Alerting if evaluations stopped returning errors.
2022-07-14 10:53:39 +01:00
George Robinson 2b2c09b8d5 Docs: Improve docs for differences and limitations (#51905) 2022-07-07 14:49:03 +01:00
George Robinson ef1fb9d3a3 Docs: Improve docs for images in notifications (#51900) 2022-07-07 14:38:02 +01:00
George Robinson 6844ac9879 Alerting: Change __alertScreenshotToken__ to __alertImageToken__ (#50771) 2022-07-04 06:05:36 -04:00
George Robinson d59db0b8e6 Alerting: Better metrics and renamed BrowserScreenshotService (#51564) 2022-06-30 17:47:23 +01:00
George Robinson 7fea330dc1 Alerting: Remove withStoredImage and change forEachFunc (#51384) 2022-06-30 15:27:57 +01:00
George Robinson c8fa4a9397 Alerting: Fix documentation for Grafana 9.0 (#51614) 2022-06-30 10:34:08 +01:00
George Robinson fe797dcfdc Alerting: Improvements to image package (#51576)
This commit makes a number of improvements to the image package:

- Improved comments
- Return the correct error when a dashboard does not exist
- Set a timeout in context.Context
2022-06-29 20:30:13 +01:00
George Robinson ed56755dd7 Alerting: Remove duplicate error message (#51515) 2022-06-29 09:39:24 +01:00
George Robinson f04dfc589c Alerting: Add support for images in Pushover alerts (#51372) 2022-06-28 09:40:01 +01:00
George Robinson 5053468c65 Alerting: Add support for images in Telegram (#51433) 2022-06-28 09:39:40 +01:00
George Robinson dc68213114 Alerting: Remove fmt.Println from Threema (#51380) 2022-06-24 14:50:53 +01:00
George Robinson 6e44b36a30 Alerting: Add support for images in Kafka alerts (#50758) 2022-06-22 11:03:08 +01:00
George Robinson 99516360c9 Alerting: Add support for images in VictorOps alerts (#50759) 2022-06-22 10:00:50 +01:00
George Robinson 624a3240f0 Alerting: Update docs for images in notifications (#50760) 2022-06-21 13:24:48 +02:00
George Robinson c8466d285c Alerting: Add support for image annotation in Alertmanager alerts (#50686) 2022-06-21 09:06:00 +01:00
George Robinson 67046c5e79 Alerting: Add support for images in Threema alerts (#50734) 2022-06-20 15:45:35 +01:00
George Robinson 7235480be5 Alerting: Use ErrImagesDone in Discord and SensuGo (#51106) 2022-06-20 14:39:27 +01:00
George Robinson 18c3456d13 Alerting: Support up to N fake images (#51111) 2022-06-20 14:34:53 +01:00
George Robinson 62c2b1ec78 Alerting: Add ErrImagesDone to return from withStoredImages (#51098) 2022-06-20 10:56:28 +01:00
George Robinson 2dbaf259a7 Alerting: Update test funcs for notifications (#51013) 2022-06-20 09:05:21 +01:00
George Robinson 5736a3c59e Alerting: Docs on how to template annotations and labels (#51024) 2022-06-17 18:30:33 +01:00
George Robinson 8c01f28285 Docs: How to template annotations and labels (#50972) 2022-06-17 10:37:43 +01:00
George Robinson 87f3bb3156 Alerting: Add support for images in SensuGo alerts (#50718) 2022-06-15 10:15:16 +01:00
George Robinson 06da129210 Docs: Add paragraph about rolling back to opt-out (#50583) 2022-06-10 11:19:20 +01:00
George Robinson bda47df4ad Alerting: Invalid setting of enabled for unified alerting should return error (#49876) 2022-06-10 08:59:58 +01:00
George Robinson 9d537efb48 Docs: Images in Notifications (#50425) 2022-06-09 12:38:20 +02:00