This commit replaces `os.Setenv` with `t.Setenv` in tests. The
environment variable is automatically restored to its original value
when the test and all its subtests complete.
Reference: https://pkg.go.dev/testing#T.Setenv
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
* remove opentracing and use otel instead
* add various samplers for jaeger
* remove useless test that is covered in otel now
* we do not need a struct there
* remove old tests
* restore tests that parse various configurations
* check errors in tests
* Update pkg/infra/tracing/tracing_test.go
fix typo
Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>
* add test for both legacy and new config formats
* use named constants
---------
Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>
* define initial service and add to wire
* update caching service interface
* add skipQueryCache header handler and update metrics query function to use it
* add caching service as a dependency to query service
* working caching impl
* propagate cache status to frontend in response
* beginning of improvements suggested by Lean - separate caching logic from query logic.
* more changes to simplify query function
* Decided to revert renaming of function
* Remove error status from cache request
* add extra documentation
* Move query caching duration metric to query package
* add a little bit of documentation
* wip: convert resource caching
* Change return type of query service QueryData to a QueryDataResponse with Headers
* update codeowners
* change X-Cache value to const
* use resource caching in endpoint handlers
* write resource headers to response even if it's not a cache hit
* fix panic caused by lack of nil check
* update unit test
* remove NONE header - shouldn't show up in OSS
* Convert everything to use the plugin middleware
* revert a few more things
* clean up unused vars
* start reverting resource caching, start to implement in plugin middleware
* revert more, fix typo
* Update caching interfaces - resource caching now has a separate cache method
* continue wiring up new resource caching conventions - still in progress
* add more safety to implementation
* remove some unused objects
* remove some code that I left in by accident
* add some comments, fix codeowners, fix duplicate registration
* fix source of panic in resource middleware
* Update client decorator test to provide an empty response object
* create tests for caching middleware
* fix unit test
* Update pkg/services/caching/service.go
Co-authored-by: Arati R. <33031346+suntala@users.noreply.github.com>
* improve error message in error log
* quick docs update
* Remove use of mockery. Update return signature to return an explicit hit/miss bool
* create unit test for empty request context
* rename caching metrics to make it clear they pertain to caching
* Update pkg/services/pluginsintegration/clientmiddleware/caching_middleware.go
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
* Add clarifying comments to cache skip middleware func
* Add comment pointing to the resource cache update call
* fix unit tests (missing dependency)
* try to fix mystery syntax error
* fix a panic
* Caching: Introduce feature toggle to caching service refactor (#66323)
* introduce new feature toggle
* hide calls to new service behind a feature flag
* remove licensing flag from toggle (misunderstood what it was for)
* fix unit tests
* rerun toggle gen
---------
Co-authored-by: Arati R. <33031346+suntala@users.noreply.github.com>
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
* Add features dependency to SQLBuilder
* Add features dependency to AccessControlDashboardPermissionFilter
* Add test for folder inheritance
* Dashboard permissions: Return recursive query
* Recursive query for inherited folders
* Modify search builder
* Adjust db.SQLBuilder
* Pass flag to SQLbuilder if CTEs are supported
* Add support for mysql < 8.0
* Add benchmarking for search with nested folders
* Set features to AlertStore
* Update pkg/infra/db/sqlbuilder.go
Co-authored-by: Ieva <ieva.vasiljeva@grafana.com>
* Set features to LibraryElementService
* SQLBuilder tests with nested folder flag set
* Apply suggestion from code review
Co-authored-by: IevaVasiljeva <ieva.vasiljeva@grafana.com>
Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>
* Add fresh context with timeout and same log properties, re-derive logger
* Unify timeout constants
* Move ctx after shortcut that got added through rebasing
* Unify timeouts
* Port opentracing's SpanFromContext and ContextFromSpan to the grafana tracing package
* Support both opentracing and otel variants
* Better document why we're creating a new ctx
* Add new func to FakeSpan which was added after rebase
* Support grafana-specific traceID key in both tracer implementations
* Tracing: Pass OTLP address and propagation format to plugins
* Fix unit tests
* Fix indentation
* Fix plugin manager integration tests
* Goimports
* Pass plugin version to plugins
* Do not add GF_PLUGIN_VERSION if plugin version is not set, add tests
* Allow disabling plugins distributed tracing on a per-plugin basis
* Moved disabled plugins to tracing.opentelemetry config section
* Pre-allocate DisabledPlugins map to the correct size
* Moved disable tracing setting flags in plugin settings
* Renamed plugin env vars for tracing endpoint and propagation
* Fix plugin initializer tests
* Refactoring: Moved OpentelemetryCfg from pkg/infra to pkg/plugins
* Changed GetSection to Section in parseSettingsOpentelemetry
* Add tests for NewOpentelemetryCfg
* Fix test case names in TestNewOpentelemetryCfg
* OpenTelemetry: Remove redundant error checks
* tracing: Support multiple OTel propagators
Signed-off-by: Dave Henderson <dave.henderson@grafana.com>
* tracing: add TraceIDString method to fix up tests
This method will be useful elsewhere if we want to log the trace ID.
* improve propagation docs
Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com>
* doc style fix
Signed-off-by: Dave Henderson <dave.henderson@grafana.com>
* Use tracing.TraceIDFromContext instead of adding TraceIDString method
Signed-off-by: Dave Henderson <dave.henderson@grafana.com>
---------
Signed-off-by: Dave Henderson <dave.henderson@grafana.com>
Co-authored-by: Bryan Boreham <bjboreham@gmail.com>
Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com>
* fix: disable orgrolepicker if externaluser is synced
* add disable to role picker
* just took me 2 hours to center the icon
* wip
* fix: check externallySyncedUser for API call
* remove check from store
* add: tests
* refactor authproxy and made tests run
* add: feature toggle
* set feature toggle for tests
* add: IsProviderEnabled
* refactor: featuretoggle name
* IsProviderEnabled tests
* add specific tests for isProviderEnabled
* fix: org_user tests
* add: owner to featuretoggle
* add missing authlabels
* remove fmt
* feature toggle
* change config
* add test for a different authmodule
* test refactor
* gen feature toggle again
* fix basic auth user able to change the org role
* test for basic auth role
* make err.base to error
* lowered lvl of log and input mesg
* add anon sessions package
* add usage stat fn
* implement count for cache
* add anonservice to authn broker
* lint
* add tests for remote cache count
* move anon service to services
* wrap tagging in goroutine
* make func used
* Remove Result field from AddDataSourceCommand
* Remove DatasourcesPermissionFilterQuery Result
* Remove GetDataSourceQuery Result
* Remove GetDataSourcesByTypeQuery Result
* Remove GetDataSourcesQuery Result
* Remove GetDefaultDataSourceQuery Result
* Remove UpdateDataSourceCommand Result
* add bundle registry service to avoid dependency cycles
* move user support bundle collector to user service
* move usage stat bundle implementation to usage stats
* add info for background service
* fix remaining imports
* whitespace
* reorganize auth usage stats
* usage stat privilege elevators
* stat count of modified role
* cfg related info
* add authn anon client
* kv store
* ensure anon enabled is collected even if client is not registered
* fix usage stats test
- Create new ctxLogProviders for each scenario to support
go test -count=n for n>1. Currently, it just kept adding providers for
every new run, which would make it add way more key-value pairs to
the provider than necessary.
- Adding a helper method to the scenario to make it easier to set up tests
for logging and easier to read the tests. I also flattened the test file, to
reduce the complexity of each test function.
Automatically forward core plugin request HTTP headers in outgoing HTTP requests.
Core datasource plugin authors don't have to specifically handle forwarding of HTTP
headers, e.g. do not have to "hardcode" the header-names in the datasource plugin,
if not having custom needs.
Fixes#57065
* chore: remove unused test helper from sqlstore
TimeNow() is no longer used in any tests in this package.
* chore: move sqlstore.SQLBuilder to the infra/db package
This required some minor refactoring; we need to be a little more explicit about passing around the dialect and engine. On the other hand, that's a few fewer uses of the `dialect` global constant!
* chore: move UserDeletions into the only package using it
* cleanup around moving sqlbuilder
* remove dialect and sqlog global vars
* rename userDeletions to serviceAccountDeletions
Adding support for backend plugin client middlewares. This allows headers in outgoing
backend plugin and HTTP requests to be modified using client middlewares.
The following client middlewares added:
Forward cookies: Will forward incoming HTTP request Cookies to outgoing plugins.Client
and HTTP requests if the datasource has enabled forwarding of cookies (keepCookies).
Forward OAuth token: Will set OAuth token headers on outgoing plugins.Client and HTTP
requests if the datasource has enabled Forward OAuth Identity (oauthPassThru).
Clear auth headers: Will clear any outgoing HTTP headers that was part of the incoming
HTTP request and used when authenticating to Grafana.
The current suggested way to register client middlewares is to have a separate package,
pluginsintegration, responsible for bootstrap/instantiate the backend plugin client with
middlewares and/or longer term bootstrap/instantiate plugin management.
Fixes#54135
Related to #47734
Related to #57870
Related to #41623
Related to #57065
* move original stats service into a separate package
* add stats service to wire
* move GetAdminStats
* switch to using stats.Service
* add missing package
* fix api tests
* Plugins: Remove support for V1 manifests
* Plugins: Make proxy endpoints not leak sensitive HTTP headers
* Security: Fix do not forward login cookie in outgoing requests
(cherry picked from commit 4539c33fce)
Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
* chore: add alias for InitTestDB and Session
Adds an alias for the sqlstore InitTestDB and Session, and updates tests using these to reduce dependencies on the sqlstore.Store.
* next pass of removing sqlstore imports
* last little bit
* remove mockstore where possible
* SQLStore: Ensure that sessions are always closed
Delete `NewSession()` in favour of `WithDbSession()`
* Add WithDbSessionForceNewSession to the interface
* Apply suggestions from code review
* NewIA: Plugin nav config
* progress
* Progress
* Things are working
* Add monitoring node
* Add alerts and incidents
* added experiment with standalone page
* Refactoring by adding a type for navtree root
* First test working
* More tests
* more tests
* Progress on richer config and sorting
* Sort weight working
* Path config
* Improving logic for not including admin or cfg nodes, making it the last step so that enterprise can add admin nodes without having to worry about the section not existing
* fixed index routes
* removed file
* Fixes
* Fixing tests
* Fixing more tests and adding support for weight config
* Updates
* Remove unused fake
* More fixes
* Minor tweak
* Minor fix
* Can now control position using sortweight even when existing items have no sortweight
* Added tests for frontend standalone page logic
* more tests
* Remove unused fake and fixed lint issue
* Moving reading settings to navtree impl package
* remove nav_id setting prefix
* Remove old test file
* Fix trailing newline
* Fixed bug with adding nil node
* fixing lint issue
* remove some code we have to rethink
* move read settings to PrivideService and switch to util.SplitString
Introduces a FromContext method on the log.Logger interface that
allows contextual key/value pairs to be attached, e.g. per request,
so that any logger using this API will automatically get the per request
context attached. The proposal makes the traceID available for
contextual logger , if available, and would allow logs originating from
a certain HTTP request to be correlated with traceID.
In addition, when tracing not enabled, skip adding
traceID=00000000000000000000000000000000
to logs.
* WIP
* Set public_suffix to a pre Ruby 2.6 version
* we don't need to install python
* Stretch->Buster
* Bump versions in lib.star
* Manually update linter
Sort of messy, but the .mod-file need to contain all dependencies that
use 1.16+ features, otherwise they're assumed to be compiled with
-lang=go1.16 and cannot access generics et al.
Bingo doesn't seem to understand that, but it's possible to manually
update things to get Bingo happy.
* undo reformatting
* Various lint improvements
* More from the linter
* goimports -w ./pkg/
* Disable gocritic
* Add/modify linter exceptions
* lint + flatten nested list
Go 1.19 doesn't support nested lists, and there wasn't an obvious workaround.
https://go.dev/doc/comment#lists
* initial cut at migration from plugin
* create new migration from plugin
* only migrate to or from, not both
* remove cfg check from plugin migration itself
* update comments, clean up secret after migration
* add better error handling
* hook up REST API with migrations
* Minor fixes
* fix wire injection issue
* modify migrator to access plugin calls directly. create unit tests
* change pre-migration checks in admin api
* stop plugin after migrating from it
* fix compile issues after merge
* add comment about migration
* fix linting issue
* bleh, fix unit test
* fix another unit test
* update plugin error fatal flag after a migration from the plugin
* add extra logging to migration
* make linter happy
Co-authored-by: Leandro Deveikis <leandro.deveikis@gmail.com>
* Chore: Add user service method SetUsingOrg
* Chore: Add user service method GetSignedInUserWithCacheCtx
* Use method GetSignedInUserWithCacheCtx from user service
* Fix lint after rebase
* Fix lint
* Fix lint error
* roll back some changes
* Roll back changes in api and middleware
* Add xorm tags to SignedInUser ID fields
* add depguard rule for ioutil
* replace ioutil.ReadDir with os.ReadDir
* use legacy option in depguard supported in golangci-lint v1.40
* replace ioutil.ReadDir with os.ReadDir
* return error for file info
* Move SignedInUser to user service and RoleType and Roles to org
* Use go naming convention for roles
* Fix some imports and leftovers
* Fix ldap debug test
* Fix lint
* Fix lint 2
* Fix lint 3
* Fix type and not needed conversion
* Clean up messages in api tests
* Clean up api tests 2
* Created PluginSecretMigrationService to be able to migrate from the secrets table from the database to the secret plugin. Added migration which takes all the secrets at the sql store and stores it in the plugin. Then deletes all the secrets from the sql
* Added secretsKVStoreSQL.GetAll() method to return all the secrets at the sql table
* Renaming kvstore_test.go as sql_test.go, adding GetAll test case. Fixing decryption of keys
* UsageStats: fixed elasticsearch version number
- The version numbering was changed from plain numbers to a semver-ish approach
* added missing version assertion
* adapted tests
* Extend kvstore to retrieve all items
* Fix comment
* Fix tests
* Change test order
* Move test outside to avoid order conditions
* Update Items to GetAll function and return a map
* Add explanation of map result
* Add description comment
Co-authored-by: Tania B <yalyna.ts@gmail.com>
In #50420 a few summary metrics were converted to histograms,
but the Help text in a couple of them still referred to summaries.
This fixes that help text.
Signed-off-by: Dave Henderson <dave.henderson@grafana.com>
Because Summary metrics can not be aggreated, convert them to histograms
so that users with HA deployments can use these metrics.
* Convert metrics registration to promauto.
* Improve help text style.
Signed-off-by: SuperQ <superq@gmail.com>
* Chore: Exclude integration tests from running on test-backend step
* Remove -v from go test command
* Add check to skip integration tests before each integration test
* Try to restart pipeline
* Retrying to make pipeline run
* pkg/web: store http.Handler internally
* pkg/web: remove injection
Removes any injection code from pkg/web.
It already was no longer functional, as we already only injected into
`http.Handler`, meaning we only inject ctx.Req and ctx.Resp.
Any other types (*Context, *ReqContext) were already accessed using the
http.Request.Context.Value() method.
* *: remove type mappings
Removes any call to the previously removed TypeMapper, as those were
non-functional already.
* pkg/web: remove Context.Invoke
was no longer used outside of pkg/web and also no longer functional
* Add database migrations
* Use short uids as data key ids
* Add support for manual data key rotation
* Fix duplicated mutex unlocks
* Fix migration
* Manage current data keys per name
* Adjust key re-encryption and test
* Modify rename column migration for MySQL compatibility
* Refactor secrets manager and data keys cache
* Multiple o11y adjustments
* Fix stats query
* Apply suggestions from code review
Co-authored-by: Tania <yalyna.ts@gmail.com>
* Fix linter
* Docs: Rotate data encryption keys API endpoint
Co-authored-by: Tania <yalyna.ts@gmail.com>
Makes `pkg/web` only accept handles from the following set:
```go
handlerStd = func(http.ResponseWriter, *http.Request)
handlerStdCtx = func(http.ResponseWriter, *http.Request, *web.Context)
handlerStdReqCtx = func(http.ResponseWriter, *http.Request, *models.ReqContext)
handlerReqCtx = func(*models.ReqContext)
handlerReqCtxRes = func(*models.ReqContext) Response
handlerCtx = func(*web.Context)
```
This is a first step to reducing above set to only `http.Handler`.
---
Due to a cyclic import situation between `pkg/models` and `pkg/web`, parts of this PR were put into `pkg/api/response`, even though they definitely do not belong there. This however is _temporary_ until we untangle `models.ReqContext`.
