This pull request re-applies the refactoring of ConditionsCmd from a
reverted fix#56812 for mathexp.noData. It does not add the fix, or
tests for the fix, because those were added in #56816. We use the
additional test coverage added in #56816 and #58650 to avoid the
reoccurrence of regressions that caused us to revert #56812 the
first time.
Adding support for backend plugin client middlewares. This allows headers in outgoing
backend plugin and HTTP requests to be modified using client middlewares.
The following client middlewares added:
Forward cookies: Will forward incoming HTTP request Cookies to outgoing plugins.Client
and HTTP requests if the datasource has enabled forwarding of cookies (keepCookies).
Forward OAuth token: Will set OAuth token headers on outgoing plugins.Client and HTTP
requests if the datasource has enabled Forward OAuth Identity (oauthPassThru).
Clear auth headers: Will clear any outgoing HTTP headers that was part of the incoming
HTTP request and used when authenticating to Grafana.
The current suggested way to register client middlewares is to have a separate package,
pluginsintegration, responsible for bootstrap/instantiate the backend plugin client with
middlewares and/or longer term bootstrap/instantiate plugin management.
Fixes#54135
Related to #47734
Related to #57870
Related to #41623
Related to #57065
This commit fixes an issue where mathexp.NoData would return an error
in ConditionsCmd (Classic Condition) instead of no data. It further
refactors the Execute method to make it easier to understand.
* create contextual log context provider
* use contextual provider in scheduler
* init logger in the package
* use context for log context
* use context in state manager
* make TimeRange interface and add relative range
* make Execute methods support the current time
* update resample to support relative time range
* update DSNode to support relative time range
* update query service to create queries with absolute time
* make alerting evaluator create relative time ranges
This commit fixes a bug where queries from datasources such as InfluxDB that returned no data would not create a DatasourceNoData alert, but instead an error "can only reduce type series, got type noData".
* WIP
* Set public_suffix to a pre Ruby 2.6 version
* we don't need to install python
* Stretch->Buster
* Bump versions in lib.star
* Manually update linter
Sort of messy, but the .mod-file need to contain all dependencies that
use 1.16+ features, otherwise they're assumed to be compiled with
-lang=go1.16 and cannot access generics et al.
Bingo doesn't seem to understand that, but it's possible to manually
update things to get Bingo happy.
* undo reformatting
* Various lint improvements
* More from the linter
* goimports -w ./pkg/
* Disable gocritic
* Add/modify linter exceptions
* lint + flatten nested list
Go 1.19 doesn't support nested lists, and there wasn't an obvious workaround.
https://go.dev/doc/comment#lists
When there is a single frame with no fields (e.g. splunk datasource) SSE errors when trying to figure out the data type. This frame needs to exist since this is where the executedQueryString metadata exists.
This adds a new return type to SSE to represent no data, so the original frame with its metadata can still be maintained.
* update decrypt secrets function signature and add secrets error handling
* remove a couple instances of unnecessary logging since errors are properly handled now
* add unit test
* fix linting issues
* wip: Implement kvstore for secrets
* wip: Refactor kvstore for secrets
* wip: Add format key function to secrets kvstore sql
* wip: Add migration for secrets kvstore
* Remove unused Key field from secrets kvstore
* Remove secret values from debug logs
* Integrate unified secrets with datasources
* Fix minor issues and tests for kvstore
* Create test service helper for secret store
* Remove encryption tests from datasources
* Move secret operations after datasources
* Fix datasource proxy tests
* Fix legacy data tests
* Add Name to all delete data source commands
* Implement decryption cache on sql secret store
* Fix minor issue with cache and tests
* Use secret type on secret store datasource operations
* Add comments to make create and update clear
* Rename itemFound variable to isFound
* Improve secret deletion and cache management
* Add base64 encoding to sql secret store
* Move secret retrieval to decrypted values function
* Refactor decrypt secure json data functions
* Fix expr tests
* Fix datasource tests
* Fix plugin proxy tests
* Fix query tests
* Fix metrics api tests
* Remove unused fake secrets service from query tests
* Add rename function to secret store
* Add check for error renaming secret
* Remove bus from tests to fix merge conflicts
* Add background secrets migration to datasources
* Get datasource secure json fields from secrets
* Move migration to secret store
* Revert "Move migration to secret store"
This reverts commit 7c3f872072.
* Add secret service to datasource service on tests
* Fix datasource tests
* Remove merge conflict on wire
* Add ctx to data source http transport on prometheus stats collector
* Add ctx to data source http transport on stats collector test
* chore: remove all remaining uses of golang.org/x/net/context
This PR finishes the work started in #47532, replacing all calls to that package with the stdlib context and using http.NewRequestWithContext to include the context where necessary.
Bonus: small formatting fixes to goimports in these files.
closes#44178
* tweak: use context.Background in favor of TODO for tests
* refactor datasource loading
* refactor datasource loading
* pass uid
* use dscache in alerting to get DS
* remove expr/translate pacakge
* remove dup injection entry
* fix DS type on metrics endpoint, remove SQL DS lookup inside SSE
* update test and adapter
* comment fix
* Make eval run as admin when getting datasource info
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
* fmt and comment
* remove unncessary/redundant code
Co-authored-by: Kyle Brandt <kyle@grafana.com>
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
Co-authored-by: Santiago <santiagohernandez.1997@gmail.com>
Get rid of using legacydata.RequestHandler in HTTPServer, /api/tsdb/query and pkg/expr
with the goal of deprecating /api/tsdb/query and remove it completely eventually. This is
the first step of cleaning up the HTTP API query endpoint.
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
Co-authored-by: Alexander Emelin <frvzmb@gmail.com>
Instead of using the frame name as the series name, we now use the name of the series value column if the frame name is not set. Queries to some datasources may end up with an empty frame name, which leads to an empty series name. As the series name is used in the frontend to display which metrics were queried etc. it shouldn't be empty.
* Replace encryption.Service by secrets.Service on expr.Service
* Replace encryption.Service by secrets.Service on live pkg
* Rename encryption.Service to encryption.Internal to clarify it must be not used
Refactor usage of legacy data contracts. Moves legacy data contracts
to pkg/tsdb/legacydata package.
Refactor pkg/expr to be a proper service/dependency that can be provided
to wire to remove some unneeded dependencies to SSE in ngalert and other places.
Refactor pkg/expr to not use the legacydata,RequestHandler and use
backend.QueryDataHandler instead.
* context all the things
* apply feedback
* rollback some alerting changes
* rollback some alerting changes #2
* more rollbacks
* more rollbacks #2
* more rollbacks #3
* more rollbacks #4
* fix integration test
* add missing context
* add missing and remove incorrect dispatch
Fixes#30144
Co-authored-by: dsotirakis <sotirakis.dim@gmail.com>
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
Co-authored-by: Ida Furjesova <ida.furjesova@grafana.com>
Co-authored-by: Jack Westbrook <jack.westbrook@gmail.com>
Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
Co-authored-by: Leon Sorokin <leeoniya@gmail.com>
Co-authored-by: Andrej Ocenas <mr.ocenas@gmail.com>
Co-authored-by: spinillos <selenepinillos@gmail.com>
Co-authored-by: Karl Persson <kalle.persson@grafana.com>
Co-authored-by: Leonard Gram <leo@xlson.com>
Move QueryData method into backend plugin manager which HandleRequest uses to
query data from plugin SDK supported data sources. This allowed us to remove a lot
of code no longer needed.
Ref #21510
Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
removes "rpc error: code = InvalidArgument desc =" like strings from error messages.
This came from old grpc stuff left from when SSE was a plugin (that probably should
not have been used even when it was a plugin)
* starting to add eval logic.
* wip
* first version of test rule.
* reverted file.
* add info colum to result to show error or (with CC evalmatches)
* fix labels in evalmatch
* fix be test
* refactored using observables.
* moved widht/height div to outside panel rendere.
* adding docs api level.
* adding container styles to error div.
* increasing size of preview.
Co-authored-by: kyle <kyle@grafana.com>
* Remove use of backend SDKs backend.Logger
* reorder imports
* FIx logger name
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
* [Alerting]: Add alerting endpoint for Query Evaluation
* Fix passing down now parameter
* Add validations and test
* Fix eval queries and expressions test
* Add eval tests
* SSE: fix reduce to handle non-null
* add type for data.Field that is float64 or *float64
* resample fix for non-null input case, add couple non-null tests
- Add the ability to stick metadata attached to a value in expressions. Currently uses Frame.Meta.Custom.
- None of this is consumed by anything yet, so an incremental step.
- Takes the conditions property from the settings column of an alert from alerts table and turns into an ng alerting condition with the queries and classic condition.
- Has temp API rest endpoint that will take the dashboard conditions json, translate it to SEE queries + classic condition, and execute it (only enabled in dev mode).
- Changes expressions to catch query responses with a non-nil error property
- Adds two new states for an NG instance result (NoData, Error) and updates evaluation to match those states
- Changes the AsDataFrame (for frontend) from Bool to string to represent additional states
- Fix bug in condition model to accept first Operator as empty string.
- In ngalert, adds GetQueryDataRequest, which was part of execute and is still called from there. But this allows me to get the Expression request from a condition to make the "pipeline" can be built.
- Update AsDataFrame for evalresult to be row based so it displays a little better for now
This is a translation of services/alerting/conditions. Main Changes:
- Work with types in SSE (series/number) which are data frames (no more null.Float).
- The query part from alerting/conditions is handled by SSE logic
- Convey / simplejson removed.
- Time range no longer part of "query" in the condition
* Measure transformation number and elapsed time
* Change histogram to summary
* Apply suggestions from code review
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
* Pass skipCache from context
* Use macaron Params instead of ParamsEscape for UIDs
* Modify queries and transform to get datasource UIDs
* Update github.com/grafana/grafana-plugin-sdk-go to v0.83.0
* Expr: fix failure to execute due to OrgID
Get orgID from the plugin context, which makes more sense anyways.
makes expressions work again after https://github.com/grafana/grafana/pull/29449 changes.
* Do not save organisation on its alert query model
Co-authored-by: Sofia Papagiannaki <sofia@grafana.com>
* comes from grafana/gel-app
* remove transform plugin code
* move __expr__ and -100 constants to expr pkg
* set OrgID on request plugin context
* use gtime for resample duration
* in resample, rename "rule" to "window", use gtime for duration, parse duration before exec
* remove gel entry from plugins-bundled/external.json
which creates an empty array for plugins
