Commit Graph

226 Commits

Author SHA1 Message Date
Emil Tullstedt
d76e5d7c6a Usage stats: Detect Prometheus flavors (#47942)
* Naïve Prometheus flavor detector

* Add concurrency and memoization

* Remove concurrency

* Fix tests

* close response body

* Add tests
2022-04-20 13:51:33 +00:00
Serge Zaitsev
41012af997 Tracing: Use common traceID context value for opentracing and opentelemetry (#46411)
* use common traceID context value for opentracing and opentelemetry

* support sampled trace IDs as well

* inject traceID into NormalResponse on errors

* Finally the test passed

* fix the test

* fix linter

* change the function parameter

Co-authored-by: Ying WANG <ying.wang@grafana.com>
2022-04-14 17:54:49 +02:00
ying-jeanne
f62c261900 fix terminal format (#47718) 2022-04-13 17:37:56 +02:00
ying-jeanne
06d2d44031 put the default logger format and feature toggle to fallback to old logger (#47584) 2022-04-13 11:02:19 +02:00
Artur Wierzbicki
712b239d5a Storage: remove pointer on write path (#47580)
* remove pointer to contents in UpsertFileCommand

* add command to UpsertFileCommand

* fix client after API refactor
2022-04-12 16:58:09 +02:00
ying-jeanne
0bf889e058 Fix unmaarshal of double pointer (#47586)
* Fix unmaarshal of double pointer

* update sdk version
2022-04-12 09:30:34 +02:00
ying-jeanne
03ba91e8a4 add feature toggle with new format (#47336)
* add feature toggle with new format

* fix some comments ❤️

* Update pkg/infra/log/term/terminal_logger.go

Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>

Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>
2022-04-11 18:50:29 +02:00
Emil Tullstedt
3df625e9f4 UsageStats: Move stats collection to separate service (#47381)
* Remove specific stats from usage stats service

* Create statscollector service

* refactor

* Update and move tests

Mostly equivalent tests to before, but they've been divided over the two
services and removed the behavior driven legacy from GoConvey to
reduce the complexity of the tests.

* Collect featuremgmr metrics (copied over from #47407)

I removed the metrics registration from the feature manager in the merge
and re-add them in this commit. Separated to make things easier to
review.
2022-04-08 13:41:26 +02:00
Artur Wierzbicki
f1a1070d41 UsageStats: track enabled features (#47407)
* #47127: include enabled features in the usage stats reports

* #47127: convert feature names to snake cased metric names

* #47127: remove dead code

* #47127: lint fix

* #47127: convert GetUsageStats to return `map[string]interface{}`

* #47127: fix testssssssss

* #47127: fix testssssssss
2022-04-08 12:42:33 +02:00
ying-jeanne
c525db9cd8 fallback to the v8 logging behavior (#47353) 2022-04-05 20:15:41 +02:00
Marcus Efraimsson
659cf17e10 Logging: Fix syslog messages should be sent with correct severity (#47209) 2022-04-04 12:51:14 +02:00
Marcus Efraimsson
b4346a5613 Logging: Fixes filtering logs based on gokitlog levels (#47195) 2022-04-04 12:36:17 +02:00
Sergey Kostrukov
656ade9884 Shared Azure middleware between Azure Monitor and Prometheus datasources (#46002)
* Scopes in Azure middleware

* Enable Azure middleware without feature flag

* Use common Azure middleware in Azure Monitor

* Apply feature flag to JsonData configuration of Azure auth

* Enforce feature flag in Prometheus datasource

* Prometheus provider tests

* Datasource service tests

* Fix http client provider tests

* Pass sdkhttpclient.Options by reference

* Add middleware to httpclient.Options

* Remove dependency on Grafana settings

* Unit-tests updated

* Fix ds_proxy_test

* Fix service_test
2022-04-01 13:26:49 +02:00
Marcus Efraimsson
16db1ad46d API: Fixes request logging middleware to include userId, orgId, uname (#47183)
Fixes #47006
2022-04-01 13:24:39 +02:00
Emil Tullstedt
7bc821baf2 Logger: Use specified format for file logger (#46970)
Fixes #45941
2022-03-26 11:40:08 +01:00
Eng Zer Jun
b56848f006 test: use T.TempDir to create temporary test directory (#44947)
The directory created by `T.TempDir` is automatically removed when the
test and all its subtests complete.

Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-03-22 15:43:29 +01:00
Ryan McKinley
1cfb9a4a19 Storage: add basic storage service (#46604) 2022-03-17 10:19:23 -07:00
Jguer
d5883c1b27 Service Accounts: Implement basic usage stats (#46619)
* Stats: do not count SAs as users

* Stats: implement basic service account metrics

* Stats: do not count service account tokens as api keys

* Stats: fix metric names

* Stats: add SA stats test

* rename user to sa
2022-03-16 16:54:34 +01:00
Artur Wierzbicki
b8fba41d74 Storage: unify List queries (#46572)
* silence errors

* s3 fix - don't retrieve files with path equal to the root

* Storage: unify list queries

* Storage: add `IsFolder` method to file obj

* Storage: API consistency - always refer `File` as a pointer rather than a value
2022-03-15 21:21:22 +04:00
Artur Wierzbicki
5881127319 Storage: fix filestorage.Wrapper PathFiltering and non-recursive folder listing (#46469)
* fix pathFilters in wrapper - apply rootPath changes before filtering

* fix non-recursive folder listing
2022-03-11 23:16:46 +04:00
Artur Wierzbicki
ed924b3d0c Storage: list filtering and root folder support (#46453)
* git the things: FS api internal changes

* remove filestorage/service.go

* remove filestore flag

* remove dummy fs

* readd fileblob import
2022-03-11 22:08:19 +04:00
gotjosh
b1e6f7126a Logging: Helper to create NopLogger (#46376) 2022-03-09 17:03:58 +00:00
Artur Wierzbicki
a8b90d9a25 FileStore: add basic file storage API (#46051)
* #45498: fs API alpha

* #45498: remove grafanaDS changes for filestorage.go

* #45498: fix lint

* #45498: fix lint

* #45498: remove db file storage migration

* #45498: linting

* #45498: linting

* #45498: linting

* #45498: fix imports

* #45498: add comment

* remove StorageName abstractions

* FileStore: add dummy implementation (#46071)

* #45498: bring back grafanaDs changes, add dummy filestorage

* #45498: rename grafanaDs to public

* #45498: modify join

* #45498: review fix

* #45498:  unnecessary leading newline (whitespace) IMPORTANT FIX

* #45498: fix belongsToStorage

* #45498: fix removeStoragePrefix so that it works with abs paths

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
2022-03-03 10:53:26 +04:00
Selene
2e6f14d17f Bus: Tests cleanup (#46025)
* Delete unused bus from usagestats

* Few updates to remove bus from searchusers test

* Fix import
2022-03-02 11:05:31 +01:00
idafurjes
d8a56d08ba Remove bus from usage stats (#45275) 2022-02-11 14:04:15 +01:00
Marcus Efraimsson
ca311bf94f Logging: Fix server hangs when using log.WithSuffix (#45213)
Fixes a problem where server hangs if setting 
GF_DATABASE_LOG_QUERIES=true.

Ref #44925
2022-02-10 12:51:51 +01:00
Marcus Efraimsson
2082eeb72f Logging: Fix so that filters can contain commented lines (#45159)
Fixes log filters that contains comments should not be enabled
2022-02-10 09:52:37 +01:00
Will Browne
fc42dfe396 Auth: Guarantee consistency of signed SigV4 headers (#45054)
* use latest sigv4 changes

* update configuration docs

* lint

* reformat lint ignore

* specific version for docs
2022-02-08 14:48:17 +01:00
Marcus Efraimsson
05ea825c76 Chore: Logging improvements (#44925)
Fixing a couple bugs, adds some tests and hopefully decrease 
lock contention when logging.
Switching from using sync.RWMutex to go-kit SwapLogger.
Fixes bug when creating a new logger from an existing one that 
screwed up the keyvals and/or lost the logger name. 

Ref #44681
2022-02-07 16:00:08 +01:00
bolshoytoster
7f2d3055d8 Logging: Fix don't return an error trying to initialize the file logger (#42952)
Fixes #42897

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2022-02-03 18:17:25 +01:00
Marcus Efraimsson
bc7e55d99b Chore: Fix log filters (#44681) 2022-02-03 23:20:02 +08:00
Marcus Efraimsson
a23900c4ba CodeQL: Ignore go/log-injection for terminal logger (#44481) 2022-01-27 19:01:59 +01:00
Ryan McKinley
3c334cd8ba Logger: avoid panic when time is a string (not fmt.Stringer) (#44497) 2022-01-26 13:17:42 -08:00
Ryan McKinley
5d66194ec5 FeatureFlags: define features outside settings.Cfg (take 3) (#44443) 2022-01-26 09:44:20 -08:00
Marcus Efraimsson
50fabe8a87 Chore: Implements log15 terminal logger (#44447)
Nicer development/terminal output based on log15 terminal logger.
2022-01-26 12:19:43 +01:00
Sergey Kostrukov
cb09162cde Datasources: Simplify the AzureCredentials structure in datasource config (#39209)
Related #35857

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2022-01-25 14:23:32 +01:00
Agnès Toulet
65bdb3a899 FeatureFlags: Revert managing feature flags outside of settings.Cfg (#44382)
* Revert "FeatureToggles: register all enterprise feature toggles (#44336)"

This reverts commit f53b3fb007.

* Revert "FeatureFlags: manage feature flags outside of settings.Cfg (#43692)"

This reverts commit f94c0decbd.
2022-01-24 16:08:05 +01:00
Ryan McKinley
f94c0decbd FeatureFlags: manage feature flags outside of settings.Cfg (#43692) 2022-01-20 13:42:05 -08:00
idafurjes
30aa24a183 Chore: Implement OpenTelemtry in Grafana (#42674)
* Separate Tracer interface to TracerService and Tracer

* Fix lint

* Fix:Make it possible to start spans for both opentracing and opentelemetry in ds proxy

* Add span methods, use span interface for rest of tracing

* Fix logs in tracing

* Fix tests that are related to tracing

* Fix resourcepermissions test

* Fix some tests

* Fix more tests

* Add TracingService to wire cli runner

* Remove GlobalTracer from bus

* Renaming test function

* Remove GlobalTracer from TSDB

* Replace GlobalTracer in api

* Adjust tests to the InitializeForTests func

* Remove GlobalTracer from services

* Remove GlobalTracer

* Remove bus.NewTest

* Remove Tracer interface

* Add InitializeForBus

* Simplify tests

* Clean up tests

* Rename TracerService to Tracer

* Update pkg/middleware/request_tracing.go

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>

* Initialize tracer before passing it to SQLStore initialization in commands

* Remove tests for opentracing

* Set span attributes correctly, remove unnecessary trace initiliazation form test

* Add tracer instance to newSQLStore

* Fix changes due to rebase

* Add modified tracing middleware test

* Fix opentracing implementation tags

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2022-01-20 11:10:12 +01:00
J Guerreiro
1e9818a69f Add usage stats preview endpoint (#43899)
* Stats: add preview route for usage statistics

* Stats: respect reporting settings

* Stats: add tests to api endpoint

* Stats: unregister route

* Stats: always possible to preview stats

* Update pkg/infra/usagestats/service/api.go

Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>

Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>
2022-01-14 10:16:07 +01:00
Yuriy Tseretyan
8114f6b065 Use stack trace context in XORM trace logger (#43780)
* add caller and stack Valuer functions
* Add WithPrefix and WithSuffix similar to what go-kit offers
* replace New with just `with`. Remove filter wrapper because the first argument of the context argument is not logger but additional context.
* update Xorm logger to use custom depth to display the datastore code instead of xorm
2022-01-13 13:30:28 -05:00
ying-jeanne
a8eef45a44 Logger migration from log15 to gokit/log (#41636)
* migrate log15 to gokit/log

* fix console log

* update some unittest

* fix all unittest

* fix the build

* Update pkg/infra/log/log.go

Co-authored-by: Yuriy Tseretyan <tceretian@gmail.com>

* general type vector

* correct the level key

Co-authored-by: Yuriy Tseretyan <tceretian@gmail.com>
2022-01-06 22:28:05 +08:00
idafurjes
8e6d6af744 Rename DispatchCtx to Dispatch (#43563) 2021-12-28 17:36:22 +01:00
idafurjes
7936c4c522 Rename AddHandlerCtx to AddHandler (#43557) 2021-12-28 16:08:07 +01:00
idafurjes
56c3875bb9 Chore: Remove context.TODO (#43458)
* Remove context.TODO() from services

* Fix live test
2021-12-28 10:26:18 +01:00
J Guerreiro
a1b8b5d123 Usage stats: Count API keys (#42883)
* Stats: add api key count to usage stats

* Chore: correctly capitalize struct member
2021-12-22 17:37:45 +01:00
idafurjes
b8852ef6a3 Chore: Remove context.TODO() (#43409)
* Remove context.TODO() from services

* Fix live test

* Remove context.TODO
2021-12-22 11:02:42 +01:00
Marcus Efraimsson
54fa7b57fe Chore: Fix flaky serverlock integration test (#42633) 2021-12-02 12:53:21 +01:00
Piotr Jamróz
bb24e8d578 Chore: Report frontend metrics for JS/CSS load time (#42421)
* Report frontend metrics for js/css load time

* Add FCP metric

* Change comment

* Simplify reporting frontend metrics
2021-12-02 09:34:39 +01:00
idafurjes
d993b12415 Add interface Tracer, add Opentelemetry (#41963)
* Add interface Tracer, add Opentelemetry

* Fix lint

* Fix failing tests and return error if config not parsed fo opentelemetry

* Update defaults.ini

Add comment with jaeger url

* go mod tidy

* Remove comments that are not needed

* Move OpentracingSpan to tracing.go

* Add opentelemetry to sample.ini
2021-12-01 17:05:08 +01:00