* Alerting: Send alerts to external Alertmanager(s)
Within this PR we're adding support for registering or unregistering
sending to a set of external alertmanagers. A few of the things that are
going are:
- Introduce a new table to hold "admin" (either org or global)
configuration we can change at runtime.
- A new periodic check that polls for this configuration and adjusts the
"senders" accordingly.
- Introduces a new concept of "senders" that are responsible for
shipping the alerts to the external Alertmanager(s). In a nutshell,
this is the Prometheus notifier (the one in charge of sending the alert)
mapped to a multi-tenant map.
There are a few code movements here and there but those are minor, I
tried to keep things intact as much as possible so that we could have an
easier diff.
* Alerting: deactivate an Alertmanager configuration
Implement DELETE /api/alertmanager/grafana/config/api/v1/alerts
by storing the default configuration which stops existing cnfiguration
from being in use.
* Apply suggestions from code review
* Alerting: Implement /status for the notification system
Implements the necessary plumbing to have a /status endpoint on the
notification system.
* Add API examples
* Update API specs
* Update prometheus/common dependency
Co-authored-by: Sofia Papagiannaki <sofia@grafana.com>
* Fix dashboard alert and nootifier migration for MySQL
* Fix POSTing Alertmanager configuration if no current configuration exists
in case the default configuration has not be stored yet
or has failed to get stored
* Change CreatedAt field type
When, and currently only when using a classic condition, evaluation information is added (which is like the EvalMatches from dashboard alerting).
This is returned via the API and can be included in notifications by reading the `__value__` label attached `.Alerts` in the template. It is a string.
* nest cache by orgID, ruleUID, stateID
* update accessors to use new cache structure
* test and linter fixup
* fix panic
Co-authored-by: Kyle Brandt <kyle@grafana.com>
* add comment to identify what's going on with nested maps in cache
Co-authored-by: Kyle Brandt <kyle@grafana.com>
* Quota: Extend service to set limit on alerts
* Add test for applying quota to alert rules
* Apply suggestions from code review
Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com>
* Get used alert quota only if naglert is enabled
* Set alert limit to zero if nglalert is not enabled
Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com>
* Fix fialure when adding state annotations
* Fix get org rules API
Do not fail response if user has no access to view a namespace.
Do not include the namespace in the response instead.
* lint
* update swagger json files match datasourceUid change
underlying change made in https://github.com/grafana/grafana/pull/33282
* Document DatasourceUID field in AlertQuery model
* Run spec generation from inside a docker container
* Generate latest spec
Co-authored-by: Sofia Papagiannaki <sofia@grafana.com>
* set processing time
* merge labels and set on response
* use state cache for adding alerts to rules
* minor cleanup
* add support for NoData and Error results
* rename test
* bring in changes from other PRs tha have been merged
* pr feedback
* add integration test
* close state tracker cleanup on context.Done
* fixup test
* rename state tracker
* set EvaluationDuration on Result
* default labels set as constants
* separate cache and state from manager
* use RWMutex in cache
* set processing time
* merge labels and set on response
* use state cache for adding alerts to rules
* minor cleanup
* add support for NoData and Error results
* rename test
* bring in changes from other PRs tha have been merged
* pr feedback
* add integration test
* close state tracker cleanup on context.Done
* fixup test
* not those annotations
* [Alerting]: Add alerting endpoint for Query Evaluation
* Fix passing down now parameter
* Add validations and test
* Fix eval queries and expressions test
* Add eval tests
* set processing time
* merge labels and set on response
* use state cache for adding alerts to rules
* minor cleanup
* pr feedback
* Do not initialize mutex unnecessarily
Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com>
* linter
Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com>
* [Alerting]: Fix updating rule group and add test
* Fix updating rule labels
* Set default values for rule no data and error states
if they are missing
* Add test for updating rule
* Test updating annotations
* Apply suggestions from code review
Co-authored-by: gotjosh <josue@grafana.com>
* add test for posting an unknown rule UID
* Fix alert rule validation and add tests
* Remove org id from PostableGrafanaRule
This field was not used; each rule gets the organisation of the user making
the rerquest
* Update pkg/tests/api/alerting/api_alertmanager_test.go
Co-authored-by: gotjosh <josue@grafana.com>
A set of fixes for the GET alert and groups endpoints.
- First, is the fact that the default values where not being for the query params. I've introduced a new method in the Grafana context that allow us to do this.
- Second, is the fact that alerts were never being transitioned to active. To my surprise this is actually done by the inhibitor in the pipeline - if an alert is not muted, or inhibited then it's active.
- Third, I have added an integration test to cover for regressions.
Signed-off-by: Josue Abreu <josue@grafana.com>
