AlertingNG: Create a scheduler to evaluate alert definitions (#29305)

* Always use cache: stop passing skipCache among ngalert functions

* Add updated column

* Scheduler initial draft

* Add retry on failure

* Allow settting/updating alert definition interval

Set default interval if no interval is provided during alert definition creation.
Keep existing alert definition interval if no interval is provided during alert definition update.

* Parameterise alerting.Ticker to run on custom interval

* Allow updating alert definition interval without having to provide the queries and expressions

* Add schedule tests

* Use xorm tags for having initialisms with consistent case in Go

* Add ability to pause/unpause the scheduler

* Add alert definition versioning

* Optimise scheduler to fetch alert definition only when it's necessary

* Change MySQL data column to mediumtext

* Delete alert definition versions

* Increase default scheduler interval to 10 seconds

* Fix setting OrgID on updates

* Add validation for alert definition name length

* Recreate tables
This commit is contained in:
Sofia Papagiannaki
2020-12-17 16:00:09 +02:00
committed by GitHub
parent 7d5c07549a
commit 3cac10e598
17 changed files with 1204 additions and 260 deletions

View File

@@ -46,7 +46,7 @@ func (e *AlertEngine) IsDisabled() bool {
// Init initializes the AlertingService.
func (e *AlertEngine) Init() error {
e.ticker = NewTicker(time.Now(), time.Second*0, clock.New())
e.ticker = NewTicker(time.Now(), time.Second*0, clock.New(), 1)
e.execQueue = make(chan *Job, 1000)
e.scheduler = newScheduler()
e.evalHandler = NewEvalHandler()