Commit Graph

226 Commits

Author SHA1 Message Date
Domas
cb9912ec0a
Alerting: button to test contact point (#37475) 2021-08-18 10:16:35 +03:00
George Robinson
3ca00f90b5
Contact point testing (#37308)
This commit adds contact point testing to ngalerts via a new API
endpoint. This endpoint accepts JSON containing a list of
receiver configurations which are validated and then tested
with a notification for a test alert. The endpoint returns JSON
for each receiver with a status and error message. It accepts
a configurable timeout via the Request-Timeout header (in seconds)
up to a maximum of 30 seconds.
2021-08-17 13:49:05 +01:00
Sofia Papagiannaki
7a01fb369d
Alerting: Fix API spec generation (#37852)
* Alerting: Fix API spec generation

* Apply suggestion from code review

Co-authored-by: gotjosh <josue@grafana.com>
2021-08-13 16:15:53 +03:00
gotjosh
f3f3fcc727
Alerting: Introduces /api/v1/ngalert/alertmanagers to expose discovered and dropped Alertmanager(s) (#37632)
* Alerting: Expose discovered and dropped Alertmanagers

Exposes the API for discovered and dropped Alertmanagers.

* make admin config poll interval configurable

* update after rebase

* wordsmith

* More wordsmithing

* change name of the config

* settings package too
2021-08-13 13:14:36 +01:00
Kyle Brandt
aef67994a1
Annotations: Fix alerting annotation coloring (#37412)
Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
2021-08-12 09:37:54 -07:00
Sofia Papagiannaki
04d5dcb7c8
Alerting: modify DB table, accessors and migration to restrict org access (#37414)
* Alerting: modify table and accessors to limit org access appropriately

* Update migration to create multiple Alertmanager configs

* Apply suggestions from code review

Co-authored-by: gotjosh <josue@grafana.com>

* replace mg.ClearMigrationEntry()

mg.ClearMigrationEntry() would create a new session.
This commit introduces a new migration for clearing an entry from migration log for replacing  mg.ClearMigrationEntry() so that all dashboard alert migration operations will run inside the same transaction.
It adds also `SkipMigrationLog()` in Migrator interface for skipping adding an entry in the migration_log.

Co-authored-by: gotjosh <josue@grafana.com>
2021-08-12 16:04:09 +03:00
SLAMA
5986d99f51
Alerting frontend : fix line notifier (#37744)
- Fixes #37425
- change `line` type string to uppercase
2021-08-10 18:59:53 +01:00
gotjosh
f83cd401e5
Alerting: Send alerts to external Alertmanager(s) (#37298)
* Alerting: Send alerts to external Alertmanager(s)

Within this PR we're adding support for registering or unregistering
sending to a set of external alertmanagers. A few of the things that are
going are:

- Introduce a new table to hold "admin" (either org or global)
  configuration we can change at runtime.
- A new periodic check that polls for this configuration and adjusts the
  "senders" accordingly.
- Introduces a new concept of "senders" that are responsible for
  shipping the alerts to the external Alertmanager(s). In a nutshell,
this is the Prometheus notifier (the one in charge of sending the alert)
mapped to a multi-tenant map.

There are a few code movements here and there but those are minor, I
tried to keep things intact as much as possible so that we could have an
easier diff.
2021-08-06 13:06:56 +01:00
Kyle Brandt
aa904a5a04
NGAlert: Send resolve signal to alertmanager on alerting -> Normal (#37363) 2021-07-29 20:29:17 +02:00
gotjosh
442a6677fc
Alerting: Refactor Run of the scheduler (#37157)
* Alerting: Refactor `Run` of the scheduler

A bit of a refactor to make the diff easier to read for supporting
external Alertmanagers.

We'll introduce another routine that checks the database for
configuration and spawns other routines accordingly.

* Block the wait.

* Fix test
2021-07-27 11:52:59 +01:00
Ganesh Vernekar
e8ac802e4f
Alerting: Remove unused fields in Pagerduty struct (#37198)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-07-27 10:41:48 +05:30
David Parrott
b5f464412d
Alerting: automatically remove stale alerting states (#36767)
* initial attempt at automatic removal of stale states

* test case, need espected states

* finish unit test

* PR feedback

* still multiply by time.second

* pr feedback
2021-07-26 18:12:04 +02:00
Ganesh Vernekar
a65975cca0
Alerting: Remove the fixed wait for notification delivery (#37203)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-07-26 15:15:09 +02:00
George Robinson
2f4c893cf3
Expand the value string in annotations and labels of alerts (#37051)
This commit makes it possible to use the value string in
annotations and labels for alerts with "{{ $value }}"
2021-07-22 15:20:44 +01:00
Sofia Papagiannaki
7815ed511f
Alerting: Refactor API endpoints for fetching alert rules (#37055)
* Refactor ruler API endpoint for listing rules

* Refactor prometheus API endpoint for listing rules

* Update HTTP API docs
2021-07-22 09:53:14 +03:00
David Parrott
fa0bed7118
do not over write alerting rule duration (#36930) 2021-07-20 11:49:35 +05:30
Djairho Geuens
4cadbba686
Email: Allow configuration of content types for email notifications (#34530)
* Alerting: Allow configuration of content types for email notifications

* Fix lint error

* Improves email templates

* Improve configuration documentation

Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com>

* Improve code comments

Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com>

* Improve configuration documentation

Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com>

* Improve email template

* Remove unnecessary predeclaration

Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>

* Adds handling for unrecognized content type

Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>

* Move utility function outside of util package

* Fixes syntax

* Remove unused package

* Fix lint error

* improve email templates

* Fix test

* Alerting: Allow configuration of content types for email notifications

* Fix lint error

* Improves email templates

* Improve configuration documentation

Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com>

* Improve code comments

Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com>

* Improve configuration documentation

Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com>

* Improve email template

* Remove unnecessary predeclaration

Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>

* Adds handling for unrecognized content type

Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>

* Move utility function outside of util package

* Fixes syntax

* Remove unused package

* Fix lint error

* improve email templates

* Fix test

* Fix comment style

Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com>

* Fix template formatting

* Add test and improve error handling

* Fix test

* Fix formatting

* Fix formatting

* Improve documentation and regenerates txt template

* Update docs/sources/administration/configuration.md

Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>

Co-authored-by: Djairho Geuens <djairho.geuens@ae.be>
Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com>
Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>
Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com>
Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>
2021-07-19 13:31:51 +03:00
Sofia Papagiannaki
f308ba91e3
Alerting: Improve receiver initialisation errors (#36814)
* Alerting: Improve receiver initialisation errors
2021-07-19 11:58:35 +03:00
Sofia Papagiannaki
afe6e793ff
Alerting: deactivate an Alertmanager configuration (#36794)
* Alerting: deactivate an Alertmanager configuration

Implement DELETE /api/alertmanager/grafana/config/api/v1/alerts
by storing the default configuration which stops existing cnfiguration
from being in use.

* Apply suggestions from code review
2021-07-16 20:07:31 +03:00
George Robinson
456dac1303
Expand the value of math and reduce expressions in annotations and labels (#36611)
* Expand the value of math and reduce expressions in annotations and labels

This commit makes it possible to use the values of reduce and math
expressions in annotations and labels via their RefIDs. It uses the
Stringer interface to ensure that "{{ $values.A }}" still prints the
value in decimal format while also making the labels for each RefID
available with "{{ $values.A.Labels }}" and the float64 value with
"{{ $values.A.Value }}"
2021-07-15 13:10:56 +01:00
David Parrott
19f18bcecc
Alerting: annotation on state change (#36535)
* WIP

* Add annotation on alert state change

* move annotation creation to manager

* praise the linter!

* add debug msg when creating annotation
2021-07-13 09:50:10 -07:00
David Parrott
310d3ebe3d
change template expansion missing value handling (#36679) 2021-07-13 06:57:18 -07:00
Ganesh Vernekar
8efe1856e2
Alerting: A better and cleaner way to know if Alertmanager is initialised (#36659)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-07-12 18:53:01 +05:30
Ganesh Vernekar
e19c690426
Alerting: Fix potential panic in Alertmanager when starting up (#36562)
* Alerting: Fix potential panic in Alertmanager when starting up

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Fix reviews

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-07-12 11:15:16 +05:30
Andrej Ocenas
ea2ba06b93
CloudWatch/Logs: Fix log alerts in new unified alerting (#36558)
* Pass FromAlert header from new alerting

* Add better error messages
2021-07-09 13:43:22 +02:00
Ganesh Vernekar
94d2520a84
Alerting: Allow space in label and annotation names (#36549)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-07-08 18:26:09 +05:30
gotjosh
a86ad1190c
Alerting: Refactor state manager as a dependency (#36513)
* Alerting: Refactor state manager as a dependency

Within the scheduler, the state manager was being passed around a
certain number of functions. I've introduced it as a dependency to keep
the "service" interfaces as clean and homogeneous as possible.

This is relevant, because I'm going to introduce live reload of these
components as part of my next PR and it is better if dependencies are
self-contained.

* remove unused functions

* Fix a few more tests

* Make sure the `stateManager` is declared before the schedule
2021-07-07 17:18:31 +01:00
Sofia Papagiannaki
fc90d47863
Alerting API: Restrict access to Alertmanager configuration (#36507)
* Alerting API: Restrict access to Alertmanager configuration to viewers
2021-07-07 16:29:18 +03:00
Sofia Papagiannaki
8a3edf280e
Alerting: Fix prometheus API to check folder permissions (#36301) 2021-07-05 10:49:14 +03:00
Ganesh Vernekar
8fe58fc2e3
Alerting: Add additional newlines to Microsoft Teams notification message where necessary (#36126)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-06-24 20:52:00 +05:30
Kyle Brandt
19f764739b
Alerting: Change __value__ label to __value_string__ annotation and add ValueString variable in notifications (#36032)
* Alerting: Allow __value__ label in notifications
was being removed by removePrivateItems
discoverd in #36020, but issue is not about that specifically

* __value__ label to __value_string__ annotation
and .ValueString extended property for notifications
2021-06-24 12:45:49 +05:30
Ganesh Vernekar
9a5c1f06df
Alerting: Template all possible variables in notification channels (#35749)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-06-22 15:12:54 +05:30
Ganesh Vernekar
33d6e11175
Alerting: Decouple default template from channel tests (#35239)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-06-21 07:59:09 +05:30
David Parrott
4732f832f7
Alerting: recalculate EndsAt (#35830)
* setEndsAt

* one more test case

* add should clause to tests
2021-06-17 10:01:46 -07:00
Ganesh Vernekar
dcd4bf1615
Alerting: Fill the empty GeneratorURL (#35740)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-06-16 15:34:12 +05:30
Sofia Papagiannaki
e5a5b8e3fe
Alerting: Fix updating alert rule properties with missing/zero values (#35512)
* Fix deleting labels and annotations

* Add test

* Keep no data and error start if not provided

* Allow setting interval and for to zero during rule updates
2021-06-15 20:55:25 +03:00
Sofia Papagiannaki
abe35c8c01
Alerting: Add error recovery during rule evaluations (#35450)
* Alerting: Eval recovery after query failure

* Apply suggestions from code review
2021-06-15 19:30:21 +03:00
gotjosh
f7ed35336d
Alerting: Implement /status for the notification system (#33227)
* Alerting: Implement /status for the notification system

Implements the necessary plumbing to have a /status endpoint on the
notification system.

* Add API examples

* Update API specs

* Update prometheus/common dependency

Co-authored-by: Sofia Papagiannaki <sofia@grafana.com>
2021-06-15 19:14:02 +03:00
Sofia Papagiannaki
fba90b8f9b
Alerting: Recact html responses (#35277) 2021-06-04 20:57:24 +03:00
Sofia Papagiannaki
8cda1f5153
Alerting: Allow rules with same title across folders (#35270)
* Alerting: Allow rules with same title across folders

* Add test
2021-06-04 20:45:26 +03:00
Sofia Papagiannaki
15c55b0115
Alerting: Fix notification channel migration and handle case when Alertmanager default configuration is absent (#35086)
* Fix dashboard alert and nootifier migration for MySQL

* Fix POSTing Alertmanager configuration if no current configuration exists

in case the default configuration has not be stored yet
or has failed to get stored

* Change CreatedAt field type
2021-06-04 15:52:41 +03:00
Ganesh Vernekar
8417088969
Alerting: Expand {{$labels.xyz}} template in labels and annotations (#35159)
* Alerting: Expand `{{$labels.xyz}}` template in labels and annotations

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Fix annotation not updating for same alert

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-06-03 19:24:36 +02:00
Ganesh Vernekar
a30e60a0b8
Alerting: Do not hard fail on templating errors in channels (#35165)
* Alerting: Do not hard fail on templating errors in channels

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>

* Fix review

Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-06-03 19:39:32 +05:30
Ganesh Vernekar
a23674ef99
Alerting: Migrate tags as labels and not annotations (#34990)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
2021-05-31 19:47:17 +05:30
Sofia Papagiannaki
355be158b7
[Alerting]: fix/cleanup API examples (#34588) 2021-05-31 11:18:29 +03:00
Chip Wolf ‮
badec6c6ad
Alerting: Add support for configuring avatar URL for the Discord notifier (#33355)
Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>
2021-05-28 23:00:21 +02:00
Owen Diehl
cc38613ba4
alerting: fixes per-receiver metric cardinality (#34915) 2021-05-28 12:31:23 -04:00
Owen Diehl
9aca032d10
Alerting/consistent api errors (#34858)
* consolidates alertmanager api errors

* util & testing consistent errors

* consistent errors for rest of ngalert apis

* updates expected errors in testware

* bump ci

* linting

* unrelated: dashboard.go lint
2021-05-28 11:55:03 -04:00
Kyle Brandt
b47e7d12e6
Alerting: Extract values from MD expr alerts (#34757)
When using mulit-dimensional Grafana managed alerts (e.g. SSE math) extract refIds values and labels so they can be shown in the notification and dashboards.
2021-05-28 11:04:20 -04:00
Danilo Bargen
83a83de10a
Clarify that Threema Gateway Alerts support only Basic IDs (#34828)
Threema Gateway supports two types of IDs: Basic IDs (where the
encryption is managed by the API server) and End-to-End IDs (where the
keys are managed by the user).

This plugin currently does not support End-to-End IDs (since it's much
more complex to implement, because the encryption needs to happen
locally). Add a few clarifications to the UI.

Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>
2021-05-28 08:54:55 +02:00