* add version to time-interval models
* set time interval fingerprint as version
* update to check provided version
* delete to check if version is provided in query parameter 'version'
* update integration tests
* update specs
* Support record struct in provisioning API
* Update api spec
* Use record field
* Restrict API endpoints following toggle
* Fix swagger spec
* Add recording rule validation to store validator
* Alerting: Update grafana/alerting
* make tests pass by implementing yaml unmarshallers and deleting fields with omitempty in their yaml tags
* go mod tidy
* fix tests by implementing not calling GettableApiAlertingConfig.UnmarshalYAML from GettableApiAlertingConfig.UnmarshalJSON
* cleanup, reduce diff
* fix more tests
* update grafana/alerting to latest commit, delete global section from configs in tests
* bring back YAML unmarshaller for GettableApiAlertingConfig
* update alerting package dependency to point to main
* skip test for sns notifier
* Placeholder commit with rule_uid change
* Add new filters to grafana rule state API
* Revert type change
* Split rule_group and rule_name params
* remove debug line
* Change how query params are parsed
* Comment
* Folders: Optionally include fullpath in service responses
* Alerting: Export folder fullpath instead of title
* Escape separator in folder title
* Add support for provisiong alret rules into subfolders
* Use FolderService for creating folders during provisioning
* Export WithFullpath() folder service function
---------
Co-authored-by: Tania B <yalyna.ts@gmail.com>
Co-authored-by: Yuri Tseretyan <yuriy.tseretyan@grafana.com>
* Alerting: Add optional metadata to GET silence responses
- ruleMetadata: to request rule metadata.
- accesscontrol: to request access control metadata.
* Alerting: Add single rule checks to alert rule access control
Modifies ruler api single rule read to no longer fetch entire groups and instead
use the new single rule ac check.
Simplifies provisioning api getAlertRuleAuthorized logic to always load a single
rule instead of conditionally loading the entire group when provisioning
permissions are not present.
* Swap out Has/AuthorizeAccessToRule for Has/AuthorizeAccessInFolder
* Read path, main API
* Define record field for incoming requests
* Refactor several alerting specific validators into two paths
* Refactor validateCondition actually contain all the condition validation logic
* Move condition validation inside rule path
* Validators for recording rules
* Wire feature flag through to validators
* Test for accepting a valid recording rule
* Tests for negative case, no UID
* Test for ignoring alerting fields
* Build conditions based on recording rules as well
* Regenerate swagger docs
* Fix CRUD test to cover the right thing
* Re-generate swagger docs with backdated v0.30.2 version
* Regenerate base spec
* Regenerate ngalert specs
* Regenerate top level specs
* Comment and rename
* Return struct instead of modifying ref
* Create recording rule fields in model
* Add migration
* Write to database, support in version table
* extend fingerprint
* Force fields to be empty on validate
* Another storage spot, tests for fingerprint
* Explicitly set defaults in provisioning API
* Tests for main API validation
* Add diff tests even though fields are unpopulated for now
* Use struct tag approach instead of FromDB/ToDB hooks as it better handles nulls when deserializing
* test for deser
* Backout RecordTo for now since it's not decided in the doc
* back out of migration too
* Drop datasourceref for now
* address linter complaints
* Try a single outer struct with all fields embedded
* Alerting: Implement GetStatus in the remote Alertmanager struct
* update tests
* fix tests, extract AlertmanagerConfig from PostableConfig
* get the remote AM config instead of the Grafana one from the remote AM
* pass grafana AM config in test
* return error in GetStatus instead of logging it (internal AM)
* Add auth checks and test
* Check user is authorized to view rule and add tests
* Change naming
* Update Swagger params
* Update auth test and swagger gen
* Update swagger gen
* Change response to GettableExtendedRuleNode
* openapi3-gen
* Update tests with refactors models pkg
* Move alertmanager api silence code to separate files unchanged
* Replace with silence model instead interface
---------
Co-authored-by: Matt Jacobson <matthew.jacobson@grafana.com>
* Alerting: Improve error when receiver used by rule is deleted
* Remove RuleUID from public error and data
* Improve fallback error in am config post
* Refactor to expand to time intervals
* Fix message on unchecked errors to be same as before
This splits the request handlers into two functions, one which is the actual
handler and one which is independent from the Grafana `ReqContext` object. This
is to make it easier to reuse the implementation in other code.
Part of the refactoring changes the functions which get query parameters from
the request to operate on a `url.Values` instead of the request object.
The change also makes the code consistently use `req.Form` instead of a
combination of `req.URL.Query()` and `req.Form`, though I have left
`api_ruler` as-is to avoid this PR growing too large.
This test has been skipped for a long time, so it doesn't work anymore. I've
fixed the test so it works again, but left some tests disabled which were
apparently flaky. If we see the other test cases flaking, we'll have to
disable it again.
Fixes:
- Use fake access control for most test cases, and real one for FGAC test cases.
- Check that "file" in API responses the full folder path, not folder title.
* Alerting: Consistently return Prometheus-style responses from rules APIs.
This commit is part refactor and part fix. The /rules API occasionally returns
error responses which are inconsistent with other error responses. This fixes
that, and adds a function to map from Prometheus error type and HTTP code.
* Fix integration tests
* Linter happiness
* Make linter more happy
* Fix up one more place returning non-Prometheus responses
Preparing these functions to be used by some other part of the codebase,
which does not have a `contextmodel.ReqContext`, only the normal request
structure (`url.Values`, etc). This is slightly messy because of how
Grafana allows url parameters to be in the URL or in the request body,
so we need to make sure to invoke the form parsing logic in `ReqContext`.
* Alerting: Optimize rule status gathering APIs when a limit is applied.
The frontend very commonly calls the `/rules` API with `limit_alerts=16`. When
there are a very large number of alert instances present, this API is quite
slow to respond, and profiling suggests that a big part of the problem is
sorting the alerts by importance, in order to select the first 16.
This changes the application of the limit to use a more efficient heap-based
top-k algorithm. This maintains a slice of only the highest ranked items whilst
iterating the full set of alert instances, which substantially reduces the
number of comparisons needed. This is particularly effective, as the
`AlertsByImportance` comparison is quite complex.
I've included a benchmark to compare the new TopK function to the existing
Sort/limit strategy. It shows that for small limits, the new approach is
much faster, especially at high numbers of alerts, e.g.
100K alerts / limit 16: 1.91s vs 0.02s (-99%)
For situations where there is no effective limit, sorting is marginally faster,
therefore in the API implementation, if there is either a) no limit or b) no
effective limit, then we just sort the alerts as before. There is also a space
overhead using a heap which would matter for large limits.
* Remove commented test cases
* Make linter happy
* Alerting: Fix simplified routes '...' groupBy creating invalid routes
There were a few ways to go about this fix:
1. Modifying our copy of upstream validation to allow this
2. Modify our notification settings validation to prevent this
3. Normalize group by on save
4. Normalized group by on generate
Option 4. was chosen as the others have a mix of the following cons:
- Generated routes risk being incompatible with upstream/remote AM
- Awkward FE UX when using '...'
- Rule definition changing after save and potential pitfalls with TF
With option 4. generated routes stay compatible with external/remote AMs, FE
doesn't need to change as we allow mixed '...' and custom label groupBys, and
settings we save to db are the same ones requested.
In addition, it has the slight benefit of allowing us to hide the internal
implementation details of `alertname, grafana_folder` from the user in the
future, since we don't need to send them with every FE or TF request.
* Safer use of DefaultNotificationSettingsGroupBy
* Fix missed API tests
* Alerting: Persist silence state immediately on Create/Delete
Persists the silence state to the kvstore immediately instead of waiting for the
next maintenance run. This is used after Create/Delete to prevent silences from
being lost when a new Alertmanager is started before the state has persisted.
This can happen, for example, in a rolling deployment scenario.
* Fix test that requires real data
* Don't error if silence state persist fails, maintenance will correct
* Feature Flags: use FeatureToggles interface where possible
Signed-off-by: Dave Henderson <dave.henderson@grafana.com>
* Replace TestFeatureToggles with existing WithFeatures
Signed-off-by: Dave Henderson <dave.henderson@grafana.com>
---------
Signed-off-by: Dave Henderson <dave.henderson@grafana.com>
* allow users with regular actions access provisioning API paths
* update methods that read rules
skip new authorization logic if user CanReadAllRules to avoid performance impact on file-provisioning
update all methods to accept identity.Requester that contains all permissions and is required by access control.
* create deltas for single rul e
* update modify methods
skip new authorization logic if user CanWriteAllRules to avoid performance impact on file-provisioning
update all methods to accept identity.Requester that contains all permissions and is required by access control.
* implement RuleAccessControlService in provisioning
* update file provisioning user to have all permissions to bypass authz
* update provisioning API to return errutil errors correctly
---------
Co-authored-by: Alexander Weaver <weaver.alex.d@gmail.com>
* require "folders:read" and "alert.rules:read" in all rules API requests (write and read).
* add check for permissions "folders:read" and "alert.rules:read" to AuthorizeAccessToRuleGroup and HasAccessToRuleGroup
* check only access to datasource in rule testing API
---------
Co-authored-by: William Wernert <william.wernert@grafana.com>
* (WIP) Alerting: Decrypt secrets before sending configuration to the remote Alertmanager
* refactor, fix tests
* test decrypting secrets
* tidy up
* test SendConfiguration, quote keys, refactor tests
* make linter happy
* decrypt configuration before comparing
* copy configuration struct before decrypting
* reduce diff in TestCompareAndSendConfiguration
* clean up remote/alertmanager.go
* make linter happy
* avoid serializing into JSON to copy struct
* codeowners
