Steve Simpson 73873f5a8a Alerting: Optimize rule status gathering APIs when a limit is applied. (#86568) ... * Alerting: Optimize rule status gathering APIs when a limit is applied. The frontend very commonly calls the `/rules` API with `limit_alerts=16`. When there are a very large number of alert instances present, this API is quite slow to respond, and profiling suggests that a big part of the problem is sorting the alerts by importance, in order to select the first 16. This changes the application of the limit to use a more efficient heap-based top-k algorithm. This maintains a slice of only the highest ranked items whilst iterating the full set of alert instances, which substantially reduces the number of comparisons needed. This is particularly effective, as the `AlertsByImportance` comparison is quite complex. I've included a benchmark to compare the new TopK function to the existing Sort/limit strategy. It shows that for small limits, the new approach is much faster, especially at high numbers of alerts, e.g. 100K alerts / limit 16: 1.91s vs 0.02s (-99%) For situations where there is no effective limit, sorting is marginally faster, therefore in the API implementation, if there is either a) no limit or b) no effective limit, then we just sort the alerts as before. There is also a space overhead using a heap which would matter for large limits. * Remove commented test cases * Make linter happy		2024-04-19 11:51:22 +02:00
..
accesscontrol	IAM - Fix error messages for resource permissions endpoints (#85773)	2024-04-17 08:53:28 -05:00
annotations	Chore: Replace sqlstore with db interface (#85366)	2024-04-04 15:04:47 +02:00
anonymous	Auth: Add IsClientEnabled and IsEnabled for the authn.Service and authn.Client interfaces (#86034)	2024-04-15 10:54:50 +02:00
apikey	Chore: Update test database initialization (#81673)	2024-02-09 09:35:39 -05:00
apiserver	Plugins: Pass cancellable context during API server creation (#86545)	2024-04-19 09:22:14 +03:00
auth	Session: set authID and authenticatedBy (#85806)	2024-04-11 10:25:29 +02:00
authn	AuthZ: Further protect admin endpoints (#86285)	2024-04-16 15:48:12 +01:00
caching	Fixing typos (#70487)	2023-06-22 09:43:38 +01:00
cleanup	Auth: Fix email verification bypass when using basic authentication (#82914)	2024-02-16 18:54:59 +01:00
cloudmigration	CloudMigrations: Move business logic out of api layer (#86406)	2024-04-17 15:43:09 -04:00
contexthandler	Tracing: Improve HTTP request/middleware spans and standalone apiserver (#85715)	2024-04-11 13:28:23 +02:00
correlations	Chore: Replace sqlstore with db interface (#85366)	2024-04-04 15:04:47 +02:00
dashboardimport	Add FolderUID for library elements (#83819)	2024-04-09 12:27:43 +02:00
dashboards	Dashboard: Allow auto refresh option when saving a dashboard (#85581)	2024-04-11 12:31:47 +03:00
dashboardsnapshots	Snapshots: Viewers can not create a Snapshot (#84952)	2024-03-22 14:31:01 -03:00
dashboardversion	Chore: Update test database initialization (#81673)	2024-02-09 09:35:39 -05:00
datasourceproxy	Errors: Make errors the same in dev as prod (#77366)	2023-10-30 14:06:26 -04:00
datasources	Provisioning: datasources auto deletion (#83034)	2024-04-08 11:45:39 +02:00
encryption	Chore: use any rather than interface{} (#74066)	2023-08-30 18:46:47 +03:00
extsvcauth	Better tracing during extSvcAcc registration (#84719)	2024-03-21 16:41:10 +01:00
featuremgmt	QueryService: Add feature toggles to better support testing (#86493)	2024-04-19 12:26:21 +03:00
folder	Add FolderUID for library elements (#83819)	2024-04-09 12:27:43 +02:00
gcom	Cloud migrations: create endpoint to create an access token (#84690)	2024-03-25 12:43:28 -03:00
grpcserver	Storage Api: Adds traces (#85391)	2024-04-16 08:30:51 -06:00
guardian	Folders: Allow listing folders with write permission (#83527)	2024-03-15 14:05:27 +02:00
hooks	Hooks: Remove AddLoginHook and RunLogin hooks (#73227)	2023-08-17 09:51:45 +02:00
kmsproviders	Secrets: Make the Migrator extensible (#67307)	2023-06-19 23:44:01 +02:00
ldap	User: Clean up update functions (#86341)	2024-04-16 16:33:50 +02:00
libraryelements	Add FolderUID for library elements (#83819)	2024-04-09 12:27:43 +02:00
librarypanels	Add FolderUID for library elements (#83819)	2024-04-09 12:27:43 +02:00
licensing	Chore: Clean up old navigation (#66287)	2023-04-14 09:43:11 +01:00
live	Grafana: Replace magic number with a constant variable in response status (#80132)	2024-02-27 18:39:51 +02:00
login	User: Fix GetByID (#86282)	2024-04-16 15:24:34 +02:00
loginattempt	Chore: Update test database initialization (#81673)	2024-02-09 09:35:39 -05:00
navtree	Explore: Move app to under explore > traces (#86436)	2024-04-17 14:18:06 +01:00
ngalert	Alerting: Optimize rule status gathering APIs when a limit is applied. (#86568)	2024-04-19 11:51:22 +02:00
notifications	Refactor: Email verification (#84393)	2024-03-14 13:25:28 +01:00
oauthtoken	Fix: Refresh token when id_token is expired (#79569)	2024-02-05 16:44:25 +01:00
org	User: use update function for password updates (#86419)	2024-04-17 15:24:36 +02:00
playlist	Chore: Update test database initialization (#81673)	2024-02-09 09:35:39 -05:00
plugindashboards	Chore: removing folderId from plugindashboard service (#80570)	2024-01-16 11:08:39 +01:00
pluginsintegration	Return plugin error when requesting settings (#86052)	2024-04-18 14:29:02 +02:00
preference	Chore: Update test database initialization (#81673)	2024-02-09 09:35:39 -05:00
provisioning	Provisioning: datasources auto deletion (#83034)	2024-04-08 11:45:39 +02:00
publicdashboards	Chore: Replace sqlstore with db interface (#85366)	2024-04-04 15:04:47 +02:00
query	Feature Flags: use FeatureToggles interface where possible (#85131)	2024-04-04 12:22:31 -04:00
queryhistory	Query History: Count using SQL, not post query (#82208)	2024-02-20 07:32:40 -06:00
quota	Chore: Replace sqlstore with db interface (#85366)	2024-04-04 15:04:47 +02:00
rendering	Feature Flags: use FeatureToggles interface where possible (#85131)	2024-04-04 12:22:31 -04:00
screenshot	Remove extra defers in screenshot sevice (#84697)	2024-03-20 07:13:16 +00:00
search	Add MFolderIDsServiceCount to count folderIDs in services pkg (#81237)	2024-01-25 11:10:35 +01:00
searchusers	Grafana: Replace magic number with a constant variable in response status (#80132)	2024-02-27 18:39:51 +02:00
searchV2	Chore: Replace sqlstore with db interface (#85366)	2024-04-04 15:04:47 +02:00
secrets	Chore: Replace sqlstore with db interface (#85366)	2024-04-04 15:04:47 +02:00
serviceaccounts	Feature Flags: use FeatureToggles interface where possible (#85131)	2024-04-04 12:22:31 -04:00
shorturls	Chore: Update test database initialization (#81673)	2024-02-09 09:35:39 -05:00
signingkeys	SigningKeys: Add added_at when creating new signing key (#85060)	2024-03-25 09:58:40 +01:00
sqlstore	SQLStore: Improve recursive CTE support detection (#86397)	2024-04-17 08:37:47 -04:00
ssosettings	Auth: encrypt/decrypt SAML secrets in SSO settings service (#85253)	2024-04-18 15:16:59 +03:00
star	Chore: Update test database initialization (#81673)	2024-02-09 09:35:39 -05:00
stats	Chore: Replace sqlstore with db interface (#85366)	2024-04-04 15:04:47 +02:00
store	Storage Api: Adds traces (#85391)	2024-04-16 08:30:51 -06:00
supportbundles	Feature Flags: use FeatureToggles interface where possible (#85131)	2024-04-04 12:22:31 -04:00
tag	Chore: Update test database initialization (#81673)	2024-02-09 09:35:39 -05:00
team	Chore: Replace sqlstore with db interface (#85366)	2024-04-04 15:04:47 +02:00
temp_user	User: email verification completion (#85259)	2024-03-28 16:05:33 +01:00
updatechecker	Chore: Making versioncheck url rely on config instead of being hardcoded (#85855)	2024-04-11 12:04:53 +02:00
user	User: use update function for password updates (#86419)	2024-04-17 15:24:36 +02:00
validations	chore: move validations model into the validations service (#61953)	2023-01-23 15:10:14 -05:00