* Data query: Allo logging panel plugin id when executing queries
* Update tracing header middleware
* Test fix
* Add panelPluginType to query analytics
* Cleanup
* Make identity.Requester available at Context
* Clean pkg/services/guardian/guardian.go
* Clean guardian provider and guardian AC
* Clean pkg/api/team.go
* Clean ctxhandler, datasources, plugin and live
* Clean dashboards and guardian
* Implement NewUserDisplayDTOFromRequester
* Change status code numbers for http constants
* Upgrade signature of ngalert services
* log parsing errors instead of throwing error
* Make identity.Requester available at Context
* Clean pkg/services/guardian/guardian.go
* Clean guardian provider and guardian AC
* Clean pkg/api/team.go
* Clean ctxhandler, datasources, plugin and live
* Question: what to do with the UserDisplayDTO?
* Clean dashboards and guardian
* Remove identity.Requester from ReqContext
* Implement NewUserDisplayDTOFromRequester
* Fix tests
* Change status code numbers for http constants
* Upgrade signature of ngalert services
* log parsing errors instead of throwing error
* Fix tests and add logs
* linting
* split queries and merge responses
* increase concurrency again
* update unit test to verify the headers are merged
* fix lint issue
* fix race condition in unit test
* Fix function name and add a bit more documentation about how the func should be used
* update function call after rename
* check for duplicate header vals
* make concurrent query limit configurable
* Update conf/sample.ini
Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>
---------
Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>
* add NodeTypeFromDatasourceUID and DataSourceModelFromNodeType()
* deprecate expr.DataSourceModel
* replace usages of IsDataSource to NodeTypeFromDatasourceUID
* replace usages of DataSourceModel to DataSourceModelFromNodeType()
* define initial service and add to wire
* update caching service interface
* add skipQueryCache header handler and update metrics query function to use it
* add caching service as a dependency to query service
* working caching impl
* propagate cache status to frontend in response
* beginning of improvements suggested by Lean - separate caching logic from query logic.
* more changes to simplify query function
* Decided to revert renaming of function
* Remove error status from cache request
* add extra documentation
* Move query caching duration metric to query package
* add a little bit of documentation
* wip: convert resource caching
* Change return type of query service QueryData to a QueryDataResponse with Headers
* update codeowners
* change X-Cache value to const
* use resource caching in endpoint handlers
* write resource headers to response even if it's not a cache hit
* fix panic caused by lack of nil check
* update unit test
* remove NONE header - shouldn't show up in OSS
* Convert everything to use the plugin middleware
* revert a few more things
* clean up unused vars
* start reverting resource caching, start to implement in plugin middleware
* revert more, fix typo
* Update caching interfaces - resource caching now has a separate cache method
* continue wiring up new resource caching conventions - still in progress
* add more safety to implementation
* remove some unused objects
* remove some code that I left in by accident
* add some comments, fix codeowners, fix duplicate registration
* fix source of panic in resource middleware
* Update client decorator test to provide an empty response object
* create tests for caching middleware
* fix unit test
* Update pkg/services/caching/service.go
Co-authored-by: Arati R. <33031346+suntala@users.noreply.github.com>
* improve error message in error log
* quick docs update
* Remove use of mockery. Update return signature to return an explicit hit/miss bool
* create unit test for empty request context
* rename caching metrics to make it clear they pertain to caching
* Update pkg/services/pluginsintegration/clientmiddleware/caching_middleware.go
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
* Add clarifying comments to cache skip middleware func
* Add comment pointing to the resource cache update call
* fix unit tests (missing dependency)
* try to fix mystery syntax error
* fix a panic
* Caching: Introduce feature toggle to caching service refactor (#66323)
* introduce new feature toggle
* hide calls to new service behind a feature flag
* remove licensing flag from toggle (misunderstood what it was for)
* fix unit tests
* rerun toggle gen
---------
Co-authored-by: Arati R. <33031346+suntala@users.noreply.github.com>
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
* add `correlationId` to queries
* trace correlation id in backend
* add correlation id to loki's span
* add correlation id to query chunks
* fix test
* fix DataSourceWithBackend test
* change to `queryGroupId`
* remove empty line
* fix test in `DataSourceWithBackend`
Adding support for backend plugin client middlewares. This allows headers in outgoing
backend plugin and HTTP requests to be modified using client middlewares.
The following client middlewares added:
Forward cookies: Will forward incoming HTTP request Cookies to outgoing plugins.Client
and HTTP requests if the datasource has enabled forwarding of cookies (keepCookies).
Forward OAuth token: Will set OAuth token headers on outgoing plugins.Client and HTTP
requests if the datasource has enabled Forward OAuth Identity (oauthPassThru).
Clear auth headers: Will clear any outgoing HTTP headers that was part of the incoming
HTTP request and used when authenticating to Grafana.
The current suggested way to register client middlewares is to have a separate package,
pluginsintegration, responsible for bootstrap/instantiate the backend plugin client with
middlewares and/or longer term bootstrap/instantiate plugin management.
Fixes#54135
Related to #47734
Related to #57870
Related to #41623
Related to #57065
* recover from panic inside mixed ds query loop
* remove accidental commit
* add messages from panics and errors to the query response
* refactor based on PR comments
* quick update to unit test to verify mixed errors and successes
* reduce concurrency limit
* make TimeRange interface and add relative range
* make Execute methods support the current time
* update resample to support relative time range
* update DSNode to support relative time range
* update query service to create queries with absolute time
* make alerting evaluator create relative time ranges
* Plugins: Remove support for V1 manifests
* Plugins: Make proxy endpoints not leak sensitive HTTP headers
* Security: Fix do not forward login cookie in outgoing requests
(cherry picked from commit 4539c33fce)
Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
* initial cut at refactor - need to run more tests
* fix unit tests
* change newly unused function to test helper
* create unit tests for parsing query requests that cover a range of cases
* add some comments
* rename function to avoid dev confusion
* plugin client returns error base
* fix api test
* add plugin client test
* add fallback err
* fix linting
* wip
* replace bad query
* template is an error
* failing test of templated error
* add one test passing
* fix failing test
* move test
* rename ErrBadQuery to ErrQueryValidationFailure
* tidy diff
* Change to one error per specific error kind
* last err + fix test
* fix imports
* more tests
* keep req vars together
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
Removes various custom headers logic sprinkled around in the backend.
It should automatically be applied to outgoing HTTP requests via the
CustomHeadersMiddleware.
This also removes decryption of SecureJSONData to populate custom
headers in ngalert which seemed to have caused a ton of CPU usage.
* Chore: Add user service method SetUsingOrg
* Chore: Add user service method GetSignedInUserWithCacheCtx
* Use method GetSignedInUserWithCacheCtx from user service
* Fix lint after rebase
* Fix lint
* Fix lint error
* roll back some changes
* Roll back changes in api and middleware
* Add xorm tags to SignedInUser ID fields
* Move SignedInUser to user service and RoleType and Roles to org
* Use go naming convention for roles
* Fix some imports and leftovers
* Fix ldap debug test
* Fix lint
* Fix lint 2
* Fix lint 3
* Fix type and not needed conversion
* Clean up messages in api tests
* Clean up api tests 2
* update decrypt secrets function signature and add secrets error handling
* remove a couple instances of unnecessary logging since errors are properly handled now
* add unit test
* fix linting issues
This PR adds endpoints for public dashboards to retrieve data from the backend (trusted) query engine. It works by executing queries defined on the backend without any user input and does not support template variables.
* Public dashboard query API
* Create new API on service for building metric request
* Flesh out testing, implement BuildPublicDashboardMetricRequest
* Test for errors and missing panels
* Refactor tests, add supporting code for multiple datasources
* Handle queries from multiple datasources
* Explicitly pass no user for querying public dashboard
Co-authored-by: Jeff Levin <jeff@levinology.com>
* wip: Implement kvstore for secrets
* wip: Refactor kvstore for secrets
* wip: Add format key function to secrets kvstore sql
* wip: Add migration for secrets kvstore
* Remove unused Key field from secrets kvstore
* Remove secret values from debug logs
* Integrate unified secrets with datasources
* Fix minor issues and tests for kvstore
* Create test service helper for secret store
* Remove encryption tests from datasources
* Move secret operations after datasources
* Fix datasource proxy tests
* Fix legacy data tests
* Add Name to all delete data source commands
* Implement decryption cache on sql secret store
* Fix minor issue with cache and tests
* Use secret type on secret store datasource operations
* Add comments to make create and update clear
* Rename itemFound variable to isFound
* Improve secret deletion and cache management
* Add base64 encoding to sql secret store
* Move secret retrieval to decrypted values function
* Refactor decrypt secure json data functions
* Fix expr tests
* Fix datasource tests
* Fix plugin proxy tests
* Fix query tests
* Fix metrics api tests
* Remove unused fake secrets service from query tests
* Add rename function to secret store
* Add check for error renaming secret
* Remove bus from tests to fix merge conflicts
* Add background secrets migration to datasources
* Get datasource secure json fields from secrets
* Move migration to secret store
* Revert "Move migration to secret store"
This reverts commit 7c3f872072.
* Add secret service to datasource service on tests
* Fix datasource tests
* Remove merge conflict on wire
* Add ctx to data source http transport on prometheus stats collector
* Add ctx to data source http transport on stats collector test
* refactor datasource loading
* refactor datasource loading
* pass uid
* use dscache in alerting to get DS
* remove expr/translate pacakge
* remove dup injection entry
* fix DS type on metrics endpoint, remove SQL DS lookup inside SSE
* update test and adapter
* comment fix
* Make eval run as admin when getting datasource info
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
* fmt and comment
* remove unncessary/redundant code
Co-authored-by: Kyle Brandt <kyle@grafana.com>
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
Co-authored-by: Santiago <santiagohernandez.1997@gmail.com>
