Commit Graph

4 Commits

Author SHA1 Message Date
Matthew Jacobson
e7c6e9c5c9 Alerting: Fix migration edge-case race condition for silences (#81206)
If the db already has an entry in the kvstore for the silences of an
alertmanager before the migration has taken place, then it's possible that the
active alertmanager will overwrite the silence file created by the migration
before it has a chance to load it into memory.

This should not happen normally but is possible in edge-cases.

This change opts to bypass the unnecessary step of writing the silences to disk
during the migration and instead write them directly to the kvstore. This avoids
 the race condition entirely and is more correct as we treat the database as the
  source of truth for AM state.
2024-02-16 10:47:34 -05:00
Sofia Papagiannaki
89d3b55bec Folders: Reduce DB queries when counting and deleting resources under folders (#81153)
* Add folder store method for fetching all folder descendants

* Modify GetDescendantCounts() to fetch folder descendants at once

* Reduce DB calls when counting library panels under dashboard

* Reduce DB calls when counting dashboards under folder

* Reduce DB calls during folder delete

* Modify folder registry to count/delete entities under multiple folders

* Reduce DB calls when counting

* Reduce DB calls when deleting
2024-01-30 18:26:34 +02:00
idafurjes
f44592a97a Remove folderID from service tests (#80615)
* Remove folderID from service tests

* Remove folderID from ngalert migration tests

* Remove tests related to folderIDs

* Roll back change

Before removing FolderID from this test, we need to adjust the code

* Remove FolderID from publicdashboard pkg

* Add back annotations test
2024-01-26 17:36:35 +02:00
Matthew Jacobson
71e70c424f Alerting: During legacy migration reduce the number of created silences (#78505)
* Alerting: During legacy migration reduce the number of created silences

During legacy migration every migrated rule was given a label rule_uid=<uid>.
This was used to silence DatasourceError/DatasourceNoData alerts for
migrated rules that had either ExecutionErrorState/NoDataState set to
keep_state, respectively.

This could potentially create a large amount of silences and a high cardinality
label. Both of these scenarios have poor outcomes for CPU load and latency in
unified alerting.

Instead, this change creates one label per ExecutionErrorState/NoDataState when
they are set to keep_state as well as two silence rules, if rules with said
labels were created during migration. These silence rules are:

- __legacy_silence_error_keep_state__ = true
- __legacy_silence_nodata_keep_state__ = true

This will drastically reduce the number of created silence rules in most cases
as well as not create the potentially high cardinality label `rule_uid`.
2024-01-24 15:56:19 -05:00