* change where folder checks are done for dash creation/updates
* add test for folder not being found
* test fixes
* more test fixes
* add nlint directive to where folder IDs are used
* fix bad merge
* fix test
* WIP
* Refactor, plus update source of error in response_parser
* Adjust test
* Use methods and httpclient from errorsource
* Update pkg/tsdb/elasticsearch/data_query.go
Co-authored-by: Scott Lepper <scott.lepper@gmail.com>
* Return nil error
* Fix test
* Fix integration test
---------
Co-authored-by: Scott Lepper <scott.lepper@gmail.com>
* Dashboards: Add integration tests for creating a dashboard
* Fix creating dashboard under folder using deprecated API
* Update swagger response
* Fix comments
When running in dev mode, error messages would contain an additional "error" property alongside "message". Since this causes confusion, that has been removed and now error messages are the same both modes (using "message").
* update storage's method InstertRules to return ids of added rules as slice to keep the same order as rules in the argument
* schematize response of update rule group endpoint, add created, updated, deleted fields that contain UID of affected rules.
* update integration tests to use the new fields
* Allow creating correlations for provisioned data sources
* Update docs
* Fix linting
* Add missing props
* Add missing props
* Fix linting
* Fix linting
* Clarify error name
* Removed error handling for a non-existing use case
* Create a list of deleted data datasources based on all configs
* Add org_id to correlations
* Add tests
* Allow org_id to be null in case org_id=0 is used
* Create organization to ensure stable id is generated
* Fix linting
* Ensure backwards compatibility
* Add deprecation information
* Update comments
* Override existing datasSource variable so the UID is retrieved correctly
* Migrate correlations indices
* Default org_id when migrating
* Remove redundant default
* Make PK non-nullable
* Post merge fixes
* Separate data sources / correlations provisioning
* Adjust comments
* Store new data sources in spy store so it can be used to test correlations as well
* Fix linting
* Update tests
* Ensure response is closed
* Avoid creating duplicates during provisioning
* Fix updating provisioned column and update tests
* Rename error message
* Fix linting errors
* Fix linting errors and rename variable
* Update test
* Update pkg/services/sqlstore/migrations/correlations_mig.go
Co-authored-by: Giordano Ricci <me@giordanoricci.com>
* Remove unused error
* Fix lining
---------
Co-authored-by: Giordano Ricci <me@giordanoricci.com>
* feat: add ability to launch targeted dskit modules in the grafana server CLI command
This commit adds a ModuleServer and ModuleRunner suitable for launching dskit services and updates the server cli command to use this instead of the full Server. The default behavior is unchanged and will launch the full Grafana server. Individual services are targeted by setting target=comma,seperated,list in the config file.
* require dev mode to target dskit modules
* remove unused type
* replace setting.CommandLineArgs w/setting.Cfg; the caller can deal with calling setting.NewCfg
* Update pkg/server/module_server.go
Co-authored-by: Serge Zaitsev <serge.zaitsev@grafana.com>
---------
Co-authored-by: Serge Zaitsev <serge.zaitsev@grafana.com>
* chore(plugins): bump systemjs to latest version
* refactor(plugins): switch runtime over to use latest systemjs, add typings
* refactor(plugins): use latest systemjs APIs for runtime dependency resolution
* refactor(plugins): return valid urls from backend for latest systemjs import to work
* revert(plugins): remove cfg from assetpath in favour of relative paths
* fix(plugins): useDefault for systemjs deps to solve undefined errors
* feat(plugins): add basic support for loading plugins via CDN
* fix(plugins): load nested plugins with latest systemjs
* feat(plugins): add back ability to transform plugin src for cdns
* feat(plugins): get caching for module.js working, clean up
* refactor(plugin_loader): create buildImportMap fn and more clean up
* refactor(angularapp): use buildImportMap for dependencies and clean up
* test(plugin_loader): fix failing test due to systemjs update
* test(jest): mock systemjs amd extra in tests to prevent it breaking tests
* chore(plugins): remove systemjs module-types extra, already included with system
* fix(plugins): update key for invalidating cached plugins
* fix: move systemjs amd define to another global so monaco can load
* refactor(plugins): clean up cache buster and tests
* chore(plugins): remove debug
* refactor(plugins): move systemjs define to keep global cleaner
* fix(plugins): set useDefault so system modules lodash references dont fail
* feat(plugins): hook systemjs onload so stylesheets are applied to the dom
* refactor(plugins): wrap amd formatted plugins in iife to prevent define collision
* feat(plugins): support system module format for legacy plugins
* test(plugincachebuster): update tests to match latest implementation
* test(plugins-loader): fix up tests post module property change
* fix test
* Update pkg/plugins/manager/loader/assetpath/assetpath.go
Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
* chore(plugin_loader): remove stray import from merge conflict
* Revert "Update pkg/plugins/manager/loader/assetpath/assetpath.go"
This reverts commit 0df57d1cf20f49c22c93369001c70aae46a97c42.
* fix(plugin_loader): set use default for shared plugin dependencies
* refactor(plugins): use leading slash for Module and BaseUrl
* fix(plugins): fix resolve appending extension to cache query param
* refactor(plugins): align baseurl and module paths
* refactor(plugins): update builtInPlugins keys to match naming convention
* refactor(plugins): minor loader clean up, fix up types
* test(plugins): fix failing tests
* refactor(plugins): rename cache buster systemjs plugin to cache
* refactor(plugins): separate plugin_loader into smaller files
* chore(plugins): clean up plugin_loader types
* chore(plugin_sandbox): fix typescript error
* chore(npm): remove unplug debug and pin systemjs to 6.14.1
* refactor(plugins-cdn): update loaders to use absolute module url from backend
* fix(plugins): escape period in systemjs module regex
* chore(plugins): delete redundant systemjs plugins
* refactor(plugin_loader): move hooks into own file, add types
* test(plugins): add tests for systemjs loader hooks
* chore(plugins): rename systemjshooks file
* chore(plugins): remove redundant systemjs cdn backend code
* fix(plugins): handle loading with config.appSubUrl
* chore(plugins): delete redundant plugin-cdn angular code
* test(plugins): fix failing systemjs test missing pluginsCDNBaseUrl
* refactor(plugins): backend provides base and module properties with AppSubUrl
* fix(plugins): consider AppSubUrl for plugin logos
* fix(plugins): use isHostedOnCDN util when checking for cdn hosted plugins
* add new appSubURL field to config
* refactor relative URL func
* fix path for core app
* refactor asset path input
* fix(plugins): catch errors in loadPluginCss
* feat(plugins-cdn): selectively transform sourceMapURL
* re-add deleted test case
* chore(plugins): bump to latest systemjs@6.14.2
* feat(plugins): add systemjs-cjs-extra for loading commonjs plugins
---------
Co-authored-by: Will Browne <will.browne@grafana.com>
Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
- The util/converter Prometheus response json parse was not checking for errors while parsing. It now does. In particular, if `[dataproxy]/response_limit` is set in Grafana's config, it will now recognize the limit error.
- Fixes#73747
- Adds `jsonitere` package, which wraps json-iterator/go's Iterator's Methods with methods that return errors, so errcheck linting can be relied upon
- Impact:
- If something was sending malformed JSON to the prometheus or loki datasources, the previous code might have accepted that and partially processed the data
- Before there may have been partial data with no error, where as no there may be errors but they will have no partial results, just the error.
* Add org_id to correlations
* Add tests
* Allow org_id to be null in case org_id=0 is used
* Create organization to ensure stable id is generated
* Fix linting
* Ensure backwards compatibility
* Add deprecation information
* Migrate correlations indices
* Default org_id when migrating
* Remove redundant default
* Make PK non-nullable
* Alerting: Fix contact point testing with secure settings
Fixes double encryption of secure settings during contact point testing and removes code duplication
that helped cause the drift between alertmanager and test endpoint. Also adds integration tests to cover
the regression.
Note: provisioningStore is created to remove cycle and the unnecessary dependency.
* Chore: Add AwaitHealthy to ModuleEngine and Server
* switch from fmt.Errorf to errors.New
Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
---------
Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
* add a feature toggle
* add the fields for attribute, kind and identifier to permission
Co-authored-by: Kalle Persson <kalle.persson@grafana.com>
* set the new fields when new permissions are stored
* add migrations
Co-authored-by: Kalle Persson <kalle.persson@grafana.com>
* remove comments
* Update pkg/services/accesscontrol/migrator/migrator.go
Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>
* feedback: put column migrations behind the feature toggle, added an index, changed how wildcard scopes are split
* PR feedback: add a comment and revert an accidentally changed file
* PR feedback: handle the case with : in resource identifier
* switch from checking feature toggle through cfg to checking it through featuremgmt
* don't put the column migrations behind a feature toggle after all - this breaks permission queries from db
---------
Co-authored-by: Kalle Persson <kalle.persson@grafana.com>
Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>
* Add pagination params and apply to sql
* Create getCorrelationsResponse that returns metadata
* Set up pagination, change correlations fetch to only get source datasource correlations
* Move correlations from root to pane, only fetch correlations for one datasource when initialized or datasource is changed
* Fix tests
* Fix remaining tests
* Use functional component to handle state
* Remove unneeded mocks, fix tests
* Change perPage to limit
* Fix Go Tests
* Fix linter
* Remove parameter
* Account for mixed datasources
* Delete unused hook
* add source UID filter to API, start backing out front end hook changes
* add source IDs to API, use when loading or changing datasource
* Fix prettier
* Mock correlations response
* Get correlations for all datasources in mixed scenario
* Add documentation for new parameters
* Attempt to fix swagger
* Fix correlations page
* add swagger and openapi docs
* Add mocks to failing test
* Change API for consistency, remove extra hooks and unused function
* Add max to limit and re-gen api docs
* Move the page to the previous page if deleting all the rows on the page
* Only fetch if remove does not have value
* Change page to a reference hook
* Fix documentation, a test and some logic thinking page could be 0
* replace condition validation with just structural validation
* validate conditions of only new and updated rules
* add integration tests for rule update\delete API
Co-authored-by: George Robinson <george.robinson@grafana.com>
* Alerting: Repurpose rule testing endpoint to return potential alerts
This feature replaces the existing no-longer in-use grafana ruler testing API endpoint /api/v1/rule/test/grafana. The new endpoint returns a list of potential alerts created by the given alert rule, including built-in + interpolated labels and annotations.
The key priority of this endpoint is that it is intended to be as true as possible to what would be generated by the ruler except that the resulting alerts are not filtered to only Resolved / Firing and ready to be sent.
This means that the endpoint will, among other things:
- Attach static annotations and labels from the rule configuration to the alert instances.
- Attach dynamic annotations from the datasource to the alert instances.
- Attach built-in labels and annotations created by the Grafana Ruler (such as alertname and grafana_folder) to the alert instances.
- Interpolate templated annotations / labels and accept allowed template functions.
* Alerting: Fix unique violation when updating rule group with title chains/cycles
The uniqueness constraint for titles within an org+folder is enforced on every update within a transaction instead of on commit (deferred constraint). This means that there could be a set of updates that will throw a unique constraint violation in an intermediate step even though the final state is valid. For example, a chain of updates RuleA -> RuleB -> RuleC could fail if not executed in the correct order, or a swap of titles RuleA <-> RuleB cannot be executed in any order without violating the constraint.
The exact solution to this is complex and requires determining directed paths and cycles in the update graph, adding in temporary updates to break cycles, and then executing the updates in reverse topological order (see first commit in PR if curious).
This is not implemented here.
Instead, we choose a simpler solution that works in all cases but might perform more updates than necessary. This simpler solution makes a determination of whether an intermediate collision could occur and if so, adds a temporary title on all updated rules to break any cycles and remove the need for specific ordering.
In addition, we make sure diffs are executed in the following order: DELETES, UPDATES, INSERTS.
* Append analytics identifier upon authenticate session
* Add id and module upon syncing user to identity
* Add authModule & id to `IdentityFromSignedInUser`
* Allow req calls in test to use basic auth
* Add `intercom_secret` to grafana config in tests
* Add test for analytics render in html view
Moving call resource API stream handling within plugin management as a utility/wrapper.
Closes#66889
Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
* update to alerting 20230418161049-5f374e58cb32
* rename renamed structs in https://github.com/grafana/alerting/pull/73
* update ValidateContactPoint to use BuildReceiverConfiguration
* update logger factory according to changes
* rewrite integration builder
Co-authored-by: Santiago <santiagohernandez.1997@gmail.com>
This commit adds support for limits and filters to the Prometheus Rules
API.
Limits:
It adds a number of limits to the Grafana flavour of the Prometheus Rules
API:
- `limit` limits the maximum number of Rule Groups returned
- `limit_rules` limits the maximum number of rules per Rule Group
- `limit_alerts` limits the maximum number of alerts per rule
It sorts Rule Groups and rules within Rule Groups such that data in the
response is stable across requests. It also returns summaries (totals)
for all Rule Groups, individual Rule Groups and rules.
Filters:
Alerts can be filtered by state with the `state` query string. An example
of an HTTP request asking for just firing alerts might be
`/api/prometheus/grafana/api/v1/rules?state=alerting`.
A request can filter by two or more states by adding additional `state`
query strings to the URL. For example `?state=alerting&state=normal`.
Like the alert list panel, the `firing`, `pending` and `normal` state are
first compared against the state of each alert rule. All other states are
ignored. If the alert rule matches then its alert instances are filtered
against states once more.
Alerts can also be filtered by labels using the `matcher` query string.
Like `state`, multiple matchers can be provided by adding additional
`matcher` query strings to the URL.
The match expression should be parsed using existing regular expression
and sent to the API as URL-encoded JSON in the format:
{
"name": "test",
"value": "value1",
"isRegex": false,
"isEqual": true
}
The `isRegex` and `isEqual` options work as follows:
| IsEqual | IsRegex | Operator |
| ------- | -------- | -------- |
| true | false | = |
| true | true | =~ |
| false | true | !~ |
| false | false | != |
This commit adds a number of limits to the Grafana flavor of the
Prometheus Rules API:
1. `limit` limits the maximum number of Rule Groups returned
2. `limit_rules` limits the maximum number of rules per Rule Group
3. `limit_alerts` limits the maximum number of alerts per rule
It sorts Rule Groups and rules within Rule Groups such that data in the
response is stable across requests. It also returns summaries (totals) for
all Rule Groups, individual Rule Groups and rules.
* copy AlertQuery from ngmodels to the definition package
* replaces usages of ngmodels.AlertQuery in API models
* create a converter between models of AlertQuery
---------
Co-authored-by: Alex Moreno <alexander.moreno@grafana.com>
* BarGauge: New value options
* Fix typings for cell options, add new value mode option for bar gauge cells
* Add BarGauge panel option, tests, and update test dashboard
* Updated
* Added default
* Goodbye trusty console.log
* Update
* Merge changes from main
* Update docs
* Add valuemode doc changes
* Update gdev dashboard
* Update valueMode symbol name to valueDisplayMode
* Use Enums as Opposed to literals, don't calculate values when hidden
* Remove double import
* Fix tests
* One more test fix
* Remove erroneous targets field, fix type of maxDataPoints
* Strip nulls and add index field to Thresholds
* Gen cue
* remove bad targets again
* Fixes
---------
Co-authored-by: Kyle Cunningham <kyle@codeincarnate.com>
Co-authored-by: sam boyer <sdboyer@grafana.com>
* bring in source from database
* bring in transformations from database
* add regex transformations to scopevar
* Consolidate types, add better example, cleanup
* Add var only if match
* Change ScopedVar to not require text, do not leak transformation-made variables between links
* Add mappings and start implementing logfmt
* Add mappings and start implementing logfmt
* Remove mappings, turn off global regex
* Add example yaml and omit transformations if empty
* Fix the yaml
* Add logfmt transformation
* Cleanup transformations and yaml
* add transformation field to FE types and use it, safeStringify logfmt values
* Add tests, only safe stringify if non-string, fix bug with safe stringify where it would return empty string with false value
* Add test for transformation field
* Do not add null transformations object
* Break out transformation logic, add tests to backend code
* Fix lint errors I understand 😅
* Fix the backend lint error
* Remove unnecessary code and mark new Transformations object as internal
* Add support for named capture groups
* Remove type assertion
* Remove variable name from transformation
* Add test for overriding regexes
* Add back variable name field, but change to mapValue
* fix go api test
* Change transformation types to enum, add better provisioning checks for bad type name and format
* Check for expression with regex transformations
* Remove Result field from AddDataSourceCommand
* Remove DatasourcesPermissionFilterQuery Result
* Remove GetDataSourceQuery Result
* Remove GetDataSourcesByTypeQuery Result
* Remove GetDataSourcesQuery Result
* Remove GetDefaultDataSourceQuery Result
* Remove UpdateDataSourceCommand Result
* add bundle registry service to avoid dependency cycles
* move user support bundle collector to user service
* move usage stat bundle implementation to usage stats
* add info for background service
* fix remaining imports
* whitespace
* Initial schema
- Add types based off of current frontend
* Rename and field-level comments
* Update report and regenerate files
* Rename frontend Azure folder
- Doing this for consistency and to ensure code-generation works
- Update betterer results due to file renames
* Remove default and add back enum vals that I deleted
* Set workspace prop as optional
* Replace template variable types
* Connect frontend query types
- Keep properties optional for now to avoid major changes
- Rename AzureMetricResource
- Correctly use ResultFormat
* Add TSVeneer decorator
* Update schema
* Update type
* Update CODEOWNERS
* Fix gen-cue issue
* Fix backend test
* Fix e2e test
* Update code coverage
* Remove references to old Azure Monitor path
* Review
* Regen files
* Add is_paused attr to the POST alert rule group endpoint
* Add is_paused to alerting API POST alert rule group
* Fixed tests
* Add is_paused to alerting gettable endpoints
* Fix integration tests
* Alerting: allow to pause existing rules (#62401)
* Display Pause Rule switch in Editing Rule form
* add isPaused property to form interface and dto
* map isPaused prop with is_paused value from DTO
Also update test snapshots
* Append '(Paused)' text on alert list state column when appropriate
* Change Switch styles according to discussion with UX
Also adding a tooltip with info what this means
* Adjust styles
* Fix alignment and isPaused type definition
Co-authored-by: gillesdemey <gilles.de.mey@gmail.com>
* Fix test
* Fix test
* Fix RuleList test
---------
Co-authored-by: gillesdemey <gilles.de.mey@gmail.com>
* wip
* Fix tests and add comments to clarify AlertRuleWithOptionals
* Fix one more test
* Fix tests
* Fix typo in comment
* Fix alert rule(s) cannot be paused via API
* Add integration tests for alerting api pausing flow
* Remove duplicated integration test
---------
Co-authored-by: Virginia Cepeda <virginia.cepeda@grafana.com>
Co-authored-by: gillesdemey <gilles.de.mey@gmail.com>
Co-authored-by: George Robinson <george.robinson@grafana.com>
* Use suggested value for uid
* update the snapshot
* use __expr__
* replace all -100 with __expr__
* update snapshot
* more changes
* revert redundant change
* Use expr.DatasourceUID where it's possible
* generate files
* schematize data query
* add the stuff you dingus
* feat(testdatasource): add scenario to generated types
* use generated testdata query in frontend
* update code owners
* Add path exception for testdata datasource
* use specific numeric data types
* fix test
* fix e2e smoketest
* add test data query type
* use test data query type
* fix betterer
* Fix typo
* move to experimental
Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>
Co-authored-by: Jack Westbrook <jack.westbrook@gmail.com>
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
Co-authored-by: sam boyer <sdboyer@grafana.com>
Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>
* Alerting: Fix Test Receivers when settings are non-strings
As part of the Alerting extraction, we want to make sure we don't have circular depedencies. As such, I had to move `PostableGrafanaReceiver` to a new struct in `grafana/alerting` called `GrafanaReceiver`.
`PostableGrafanaReceiver` has an attribute called `Settings` that uses a Grafana-propietary struct called `RawMessage`, this struct shadows `json.RawMessage`.
When I created `GrafanaReceiver`, I turned settings into a `map[string]string` thinking all settings would end up as strings. This was a mistake, and this test proves that it doesn't work, and breaks the API.
* Return errors from data parsing
* Better error handling
* Fix the tests
* When there is no frame add empty frame to get metadata attached to it
* Fix tests
* Update testdata
