* wip: Implement kvstore for secrets
* wip: Refactor kvstore for secrets
* wip: Add format key function to secrets kvstore sql
* wip: Add migration for secrets kvstore
* Remove unused Key field from secrets kvstore
* Remove secret values from debug logs
* Integrate unified secrets with datasources
* Fix minor issues and tests for kvstore
* Create test service helper for secret store
* Remove encryption tests from datasources
* Move secret operations after datasources
* Fix datasource proxy tests
* Fix legacy data tests
* Add Name to all delete data source commands
* Implement decryption cache on sql secret store
* Fix minor issue with cache and tests
* Use secret type on secret store datasource operations
* Add comments to make create and update clear
* Rename itemFound variable to isFound
* Improve secret deletion and cache management
* Add base64 encoding to sql secret store
* Move secret retrieval to decrypted values function
* Refactor decrypt secure json data functions
* Fix expr tests
* Fix datasource tests
* Fix plugin proxy tests
* Fix query tests
* Fix metrics api tests
* Remove unused fake secrets service from query tests
* Add rename function to secret store
* Add check for error renaming secret
* Remove bus from tests to fix merge conflicts
* Add background secrets migration to datasources
* Get datasource secure json fields from secrets
* Move migration to secret store
* Revert "Move migration to secret store"
This reverts commit 7c3f872072.
* Add secret service to datasource service on tests
* Fix datasource tests
* Remove merge conflict on wire
* Add ctx to data source http transport on prometheus stats collector
* Add ctx to data source http transport on stats collector test
* use common traceID context value for opentracing and opentelemetry
* support sampled trace IDs as well
* inject traceID into NormalResponse on errors
* Finally the test passed
* fix the test
* fix linter
* change the function parameter
Co-authored-by: Ying WANG <ying.wang@grafana.com>
* add feature toggle with new format
* fix some comments ❤️
* Update pkg/infra/log/term/terminal_logger.go
Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>
Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>
* Remove specific stats from usage stats service
* Create statscollector service
* refactor
* Update and move tests
Mostly equivalent tests to before, but they've been divided over the two
services and removed the behavior driven legacy from GoConvey to
reduce the complexity of the tests.
* Collect featuremgmr metrics (copied over from #47407)
I removed the metrics registration from the feature manager in the merge
and re-add them in this commit. Separated to make things easier to
review.
The directory created by `T.TempDir` is automatically removed when the
test and all its subtests complete.
Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
* Stats: do not count SAs as users
* Stats: implement basic service account metrics
* Stats: do not count service account tokens as api keys
* Stats: fix metric names
* Stats: add SA stats test
* rename user to sa
* silence errors
* s3 fix - don't retrieve files with path equal to the root
* Storage: unify list queries
* Storage: add `IsFolder` method to file obj
* Storage: API consistency - always refer `File` as a pointer rather than a value
Fixing a couple bugs, adds some tests and hopefully decrease
lock contention when logging.
Switching from using sync.RWMutex to go-kit SwapLogger.
Fixes bug when creating a new logger from an existing one that
screwed up the keyvals and/or lost the logger name.
Ref #44681
* Separate Tracer interface to TracerService and Tracer
* Fix lint
* Fix:Make it possible to start spans for both opentracing and opentelemetry in ds proxy
* Add span methods, use span interface for rest of tracing
* Fix logs in tracing
* Fix tests that are related to tracing
* Fix resourcepermissions test
* Fix some tests
* Fix more tests
* Add TracingService to wire cli runner
* Remove GlobalTracer from bus
* Renaming test function
* Remove GlobalTracer from TSDB
* Replace GlobalTracer in api
* Adjust tests to the InitializeForTests func
* Remove GlobalTracer from services
* Remove GlobalTracer
* Remove bus.NewTest
* Remove Tracer interface
* Add InitializeForBus
* Simplify tests
* Clean up tests
* Rename TracerService to Tracer
* Update pkg/middleware/request_tracing.go
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
* Initialize tracer before passing it to SQLStore initialization in commands
* Remove tests for opentracing
* Set span attributes correctly, remove unnecessary trace initiliazation form test
* Add tracer instance to newSQLStore
* Fix changes due to rebase
* Add modified tracing middleware test
* Fix opentracing implementation tags
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
* add caller and stack Valuer functions
* Add WithPrefix and WithSuffix similar to what go-kit offers
* replace New with just `with`. Remove filter wrapper because the first argument of the context argument is not logger but additional context.
* update Xorm logger to use custom depth to display the datastore code instead of xorm
* Add interface Tracer, add Opentelemetry
* Fix lint
* Fix failing tests and return error if config not parsed fo opentelemetry
* Update defaults.ini
Add comment with jaeger url
* go mod tidy
* Remove comments that are not needed
* Move OpentracingSpan to tracing.go
* Add opentelemetry to sample.ini
When a request going through Grafana data source proxy responds with a websocket
upgrade response we cannot override the response body since it produces an error.
This problem seems to have been introduced in Grafana v8.0 by #38962.
In addition #40303 added same problem.
Fixes#41292
* context all the things
* apply feedback
* rollback some alerting changes
* rollback some alerting changes #2
* more rollbacks
* more rollbacks #2
* more rollbacks #3
* more rollbacks #4
* fix integration test
* add missing context
* add missing and remove incorrect dispatch
* Add context to star and stats
* Use WithTransactionalDbSession
* Add additional ctx
* Remove convey
* Fix star handler name
* Use WithDbSession, use DispatchCtx
* Remove xorm from star
* added errors in plugin list.
* added error to details page.
* adding badge on details page.
* added some more tests.
* Renamed to disabled and will handle the scenario in the plugin catalog.
* Update public/app/features/plugins/admin/components/PluginDetailsDisabledError.tsx
Co-authored-by: Levente Balogh <balogh.levente.hu@gmail.com>
* fixing some nits
* added missing isDisabeld to the mock.
* adding tests to verify scenarios when plugin is disabled.
* fixed issue with formatting after file changed on GH.
Co-authored-by: Levente Balogh <balogh.levente.hu@gmail.com>
* warn on linux
* add warning for grpc plugins
* add windows support
* update go.mod
* reorganize imports
* update naming
* remove Windows logic
* simplify and add check for when UID and EUID don't match
* fix build
* tidy go.mod
* feedback
* cleanup + migrate
* Introduce response_limit for datasource responses
* Fix lint
* Fix tests
* Add case where limit <= 0 - added parametrized tests
* Add max_bytes_reader.go
* Use new httpclient.MaxBytesReader instead of net/http one
* Fixes according to reviewer's comments
* Add tests for max_bytes_reader
* Add small piece in configuration.md
* Further fixes according to reviewer's comments
* Fix linting - fix test
* add key/value store service
* don't export kvStoreSQL, consumers should interact with KVStore & NamespacedKVStore
* add del method, avoid ErrNotFound (#38627)
* switch value column to medium text
Co-authored-by: Alexander Emelin <frvzmb@gmail.com>
Fixes#30144
Co-authored-by: dsotirakis <sotirakis.dim@gmail.com>
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
Co-authored-by: Ida Furjesova <ida.furjesova@grafana.com>
Co-authored-by: Jack Westbrook <jack.westbrook@gmail.com>
Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
Co-authored-by: Leon Sorokin <leeoniya@gmail.com>
Co-authored-by: Andrej Ocenas <mr.ocenas@gmail.com>
Co-authored-by: spinillos <selenepinillos@gmail.com>
Co-authored-by: Karl Persson <kalle.persson@grafana.com>
Co-authored-by: Leonard Gram <leo@xlson.com>
* Azure middleware in HttpClientProxy
* Azure authentication under feature flag
* Minor fixes
* Add prefixes to not clash with JsonData
* Return error if JsonData cannot be parsed
* Return original string if URL invalid
* Tests for datasource_cache
Make sure that default timeout settings are based on configuration
parameters. This now applies for core data sources using old TSDB
contracts and new SDK contracts. Before it was only applied for old TSDB
contracts.
Also moves global setting variables to non-global (setting.Cfg).
* Usage Stats: Rename service to use a more idiomatic name
* Usage Stats: Update MetricsFunc definition and implementations
* Revert "Usage Stats: Rename service to use a more idiomatic name"
This reverts commit 910ecce3e8.
* Usage Stats: Update MetricsFunc definition and implementations
Uses new httpclient package from grafana-plugin-sdk-go introduced
via grafana/grafana-plugin-sdk-go#328.
Replaces the GetHTTPClient, GetTransport, GetTLSConfig methods defined
on DataSource model.
Longer-term the goal is to migrate core HTTP backend data sources to use the
SDK contracts and using httpclient.Provider for creating HTTP clients and such.
Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>
This patch adds metrics to support instrumenting the accesscontrols package.
It also instruments the accesscontrol evaluator and the permissions function.
Co-authored-by: Vardan Torosyan <vardants@gmail.com>
* add uninstall flow
* add install flow
* small cleanup
* smaller-footprint solution
* cleanup + make bp start auto
* fix interface contract
* improve naming
* accept version arg
* ensure use of shared logger
* make installer a field
* add plugin decommissioning
* add basic error checking
* fix api docs
* making initialization idempotent
* add mutex
* fix comment
* fix test
* add test for decommission
* improve existing test
* add more test coverage
* more tests
* change test func to use read lock
* refactoring + adding test asserts
* improve purging old install flow
* improve dupe checking
* change log name
* skip over dupe scanned
* make test assertion more flexible
* remove trailing line
* fix pointer receiver name
* update comment
* add context to API
* add config flag
* add base http api test + fix update functionality
* simplify existing check
* clean up test
* refactor tests based on feedback
* add single quotes to errs
* use gcmp in tests + fix logo issue
* make plugin list testing more flexible
* address feedback
* fix API test
* fix linter
* undo preallocate
* Update docs/sources/administration/configuration.md
Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>
* Update docs/sources/administration/configuration.md
Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>
* Update docs/sources/administration/configuration.md
Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>
* fix linting issue in test
* add docs placeholder
* update install notes
* Update docs/sources/plugins/marketplace.md
Co-authored-by: Marcus Olsson <marcus.olsson@hey.com>
* update access wording
* add more placeholder docs
* add link to more info
* PR feedback - improved errors, refactor, lock fix
* improve err details
* propagate plugin version errors
* don't autostart renderer
* add H1
* fix imports
Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>
Co-authored-by: Marcus Olsson <marcus.olsson@hey.com>
* Rendering: add CSV rendering support
* Rendering: save csv files into a separate folder
* add missing field
* Renderer: get filename from renderer plugin
* apply PR suggestions
* Rendering: remove old PhantomJS error
* Rendering: separate RenderCSV and Render functions
* fix alerting test
* Rendering: fix handling error in HTTP mode
* apply PR feedback
* Update pkg/services/rendering/http_mode.go
Co-authored-by: Joan López de la Franca Beltran <joanjan14@gmail.com>
* apply PR feedback
* Update rendering metrics with type label
* Rendering: return error if not able to parse header
* Rendering: update grpc generated file
* Rendering: use context.WithTimeout to render CSV too
Co-authored-by: Joan López de la Franca Beltran <joanjan14@gmail.com>
* use a common way to redact sensitive values before logging them
* fix panic on missing testCase.err, simplify require checks
* fix a silly typo
* combine readConfig and buildConnectionString methods, as they are closely related
* check for jaeger env variables if tracing address configuration is empty
* add tests to ensure jaeger host/port variables override the settings
* allow default host and custom port, too
* disallow missing tracing.jaeger section in the config
* check for all errors in tests
* make parseSettings() return an error
* FrontendMetrics: Adds new backend api that frontend can use to push frontend measurements and counters to prometheus
* FrontendMetrics: Adds new backend api that frontend can use to push frontend measurements and counters to prometheus
* Fix naming
* change to histogram
* Fixed go lint
* Chore: Collect elasticsearch version usage stats
* Fix lint error
* use GetDataSources from sqlstore
* Apply review suggestions
* Return error if datasource type is not specified
* Update pkg/services/sqlstore/datasource.go
* fix undefined var
Usage Stats: Calculate concurrent users as a histogram
Metric will help to understand if users are sharing their credentials or using one account for concurrent sessions. This will help to make more informed decisions when enforcing any license limitations on credentials usage.
Adding an interface type for usage stats service allows us to not depend on the implementation outside of the package, for example when testing we can easily mock the service
The change allows to add external, additional metrics which will be reported with the rest of hardcoded metrics, allowing to add metrics without a need to change the service itself.
* middleware: Move context handler to own service
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
Co-authored-by: Emil Tullsted <sakjur@users.noreply.github.com>
Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
