Commit Graph

226 Commits

Author SHA1 Message Date
Emil Tullstedt
d76e5d7c6a
Usage stats: Detect Prometheus flavors (#47942)
* Naïve Prometheus flavor detector

* Add concurrency and memoization

* Remove concurrency

* Fix tests

* close response body

* Add tests
2022-04-20 13:51:33 +00:00
Serge Zaitsev
41012af997
Tracing: Use common traceID context value for opentracing and opentelemetry (#46411)
* use common traceID context value for opentracing and opentelemetry

* support sampled trace IDs as well

* inject traceID into NormalResponse on errors

* Finally the test passed

* fix the test

* fix linter

* change the function parameter

Co-authored-by: Ying WANG <ying.wang@grafana.com>
2022-04-14 17:54:49 +02:00
ying-jeanne
f62c261900
fix terminal format (#47718) 2022-04-13 17:37:56 +02:00
ying-jeanne
06d2d44031
put the default logger format and feature toggle to fallback to old logger (#47584) 2022-04-13 11:02:19 +02:00
Artur Wierzbicki
712b239d5a
Storage: remove pointer on write path (#47580)
* remove pointer to contents in UpsertFileCommand

* add command to UpsertFileCommand

* fix client after API refactor
2022-04-12 16:58:09 +02:00
ying-jeanne
0bf889e058
Fix unmaarshal of double pointer (#47586)
* Fix unmaarshal of double pointer

* update sdk version
2022-04-12 09:30:34 +02:00
ying-jeanne
03ba91e8a4
add feature toggle with new format (#47336)
* add feature toggle with new format

* fix some comments ❤️

* Update pkg/infra/log/term/terminal_logger.go

Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>

Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>
2022-04-11 18:50:29 +02:00
Emil Tullstedt
3df625e9f4
UsageStats: Move stats collection to separate service (#47381)
* Remove specific stats from usage stats service

* Create statscollector service

* refactor

* Update and move tests

Mostly equivalent tests to before, but they've been divided over the two
services and removed the behavior driven legacy from GoConvey to
reduce the complexity of the tests.

* Collect featuremgmr metrics (copied over from #47407)

I removed the metrics registration from the feature manager in the merge
and re-add them in this commit. Separated to make things easier to
review.
2022-04-08 13:41:26 +02:00
Artur Wierzbicki
f1a1070d41
UsageStats: track enabled features (#47407)
* #47127: include enabled features in the usage stats reports

* #47127: convert feature names to snake cased metric names

* #47127: remove dead code

* #47127: lint fix

* #47127: convert GetUsageStats to return `map[string]interface{}`

* #47127: fix testssssssss

* #47127: fix testssssssss
2022-04-08 12:42:33 +02:00
ying-jeanne
c525db9cd8
fallback to the v8 logging behavior (#47353) 2022-04-05 20:15:41 +02:00
Marcus Efraimsson
659cf17e10
Logging: Fix syslog messages should be sent with correct severity (#47209) 2022-04-04 12:51:14 +02:00
Marcus Efraimsson
b4346a5613
Logging: Fixes filtering logs based on gokitlog levels (#47195) 2022-04-04 12:36:17 +02:00
Sergey Kostrukov
656ade9884
Shared Azure middleware between Azure Monitor and Prometheus datasources (#46002)
* Scopes in Azure middleware

* Enable Azure middleware without feature flag

* Use common Azure middleware in Azure Monitor

* Apply feature flag to JsonData configuration of Azure auth

* Enforce feature flag in Prometheus datasource

* Prometheus provider tests

* Datasource service tests

* Fix http client provider tests

* Pass sdkhttpclient.Options by reference

* Add middleware to httpclient.Options

* Remove dependency on Grafana settings

* Unit-tests updated

* Fix ds_proxy_test

* Fix service_test
2022-04-01 13:26:49 +02:00
Marcus Efraimsson
16db1ad46d
API: Fixes request logging middleware to include userId, orgId, uname (#47183)
Fixes #47006
2022-04-01 13:24:39 +02:00
Emil Tullstedt
7bc821baf2
Logger: Use specified format for file logger (#46970)
Fixes #45941
2022-03-26 11:40:08 +01:00
Eng Zer Jun
b56848f006
test: use T.TempDir to create temporary test directory (#44947)
The directory created by `T.TempDir` is automatically removed when the
test and all its subtests complete.

Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-03-22 15:43:29 +01:00
Ryan McKinley
1cfb9a4a19
Storage: add basic storage service (#46604) 2022-03-17 10:19:23 -07:00
Jguer
d5883c1b27
Service Accounts: Implement basic usage stats (#46619)
* Stats: do not count SAs as users

* Stats: implement basic service account metrics

* Stats: do not count service account tokens as api keys

* Stats: fix metric names

* Stats: add SA stats test

* rename user to sa
2022-03-16 16:54:34 +01:00
Artur Wierzbicki
b8fba41d74
Storage: unify List queries (#46572)
* silence errors

* s3 fix - don't retrieve files with path equal to the root

* Storage: unify list queries

* Storage: add `IsFolder` method to file obj

* Storage: API consistency - always refer `File` as a pointer rather than a value
2022-03-15 21:21:22 +04:00
Artur Wierzbicki
5881127319
Storage: fix filestorage.Wrapper PathFiltering and non-recursive folder listing (#46469)
* fix pathFilters in wrapper - apply rootPath changes before filtering

* fix non-recursive folder listing
2022-03-11 23:16:46 +04:00
Artur Wierzbicki
ed924b3d0c
Storage: list filtering and root folder support (#46453)
* git the things: FS api internal changes

* remove filestorage/service.go

* remove filestore flag

* remove dummy fs

* readd fileblob import
2022-03-11 22:08:19 +04:00
gotjosh
b1e6f7126a
Logging: Helper to create NopLogger (#46376) 2022-03-09 17:03:58 +00:00
Artur Wierzbicki
a8b90d9a25
FileStore: add basic file storage API (#46051)
* #45498: fs API alpha

* #45498: remove grafanaDS changes for filestorage.go

* #45498: fix lint

* #45498: fix lint

* #45498: remove db file storage migration

* #45498: linting

* #45498: linting

* #45498: linting

* #45498: fix imports

* #45498: add comment

* remove StorageName abstractions

* FileStore: add dummy implementation (#46071)

* #45498: bring back grafanaDs changes, add dummy filestorage

* #45498: rename grafanaDs to public

* #45498: modify join

* #45498: review fix

* #45498:  unnecessary leading newline (whitespace) IMPORTANT FIX

* #45498: fix belongsToStorage

* #45498: fix removeStoragePrefix so that it works with abs paths

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
2022-03-03 10:53:26 +04:00
Selene
2e6f14d17f
Bus: Tests cleanup (#46025)
* Delete unused bus from usagestats

* Few updates to remove bus from searchusers test

* Fix import
2022-03-02 11:05:31 +01:00
idafurjes
d8a56d08ba
Remove bus from usage stats (#45275) 2022-02-11 14:04:15 +01:00
Marcus Efraimsson
ca311bf94f
Logging: Fix server hangs when using log.WithSuffix (#45213)
Fixes a problem where server hangs if setting 
GF_DATABASE_LOG_QUERIES=true.

Ref #44925
2022-02-10 12:51:51 +01:00
Marcus Efraimsson
2082eeb72f
Logging: Fix so that filters can contain commented lines (#45159)
Fixes log filters that contains comments should not be enabled
2022-02-10 09:52:37 +01:00
Will Browne
fc42dfe396
Auth: Guarantee consistency of signed SigV4 headers (#45054)
* use latest sigv4 changes

* update configuration docs

* lint

* reformat lint ignore

* specific version for docs
2022-02-08 14:48:17 +01:00
Marcus Efraimsson
05ea825c76
Chore: Logging improvements (#44925)
Fixing a couple bugs, adds some tests and hopefully decrease 
lock contention when logging.
Switching from using sync.RWMutex to go-kit SwapLogger.
Fixes bug when creating a new logger from an existing one that 
screwed up the keyvals and/or lost the logger name. 

Ref #44681
2022-02-07 16:00:08 +01:00
bolshoytoster
7f2d3055d8
Logging: Fix don't return an error trying to initialize the file logger (#42952)
Fixes #42897

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2022-02-03 18:17:25 +01:00
Marcus Efraimsson
bc7e55d99b
Chore: Fix log filters (#44681) 2022-02-03 23:20:02 +08:00
Marcus Efraimsson
a23900c4ba
CodeQL: Ignore go/log-injection for terminal logger (#44481) 2022-01-27 19:01:59 +01:00
Ryan McKinley
3c334cd8ba
Logger: avoid panic when time is a string (not fmt.Stringer) (#44497) 2022-01-26 13:17:42 -08:00
Ryan McKinley
5d66194ec5
FeatureFlags: define features outside settings.Cfg (take 3) (#44443) 2022-01-26 09:44:20 -08:00
Marcus Efraimsson
50fabe8a87
Chore: Implements log15 terminal logger (#44447)
Nicer development/terminal output based on log15 terminal logger.
2022-01-26 12:19:43 +01:00
Sergey Kostrukov
cb09162cde
Datasources: Simplify the AzureCredentials structure in datasource config (#39209)
Related #35857

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2022-01-25 14:23:32 +01:00
Agnès Toulet
65bdb3a899
FeatureFlags: Revert managing feature flags outside of settings.Cfg (#44382)
* Revert "FeatureToggles: register all enterprise feature toggles (#44336)"

This reverts commit f53b3fb007.

* Revert "FeatureFlags: manage feature flags outside of settings.Cfg (#43692)"

This reverts commit f94c0decbd.
2022-01-24 16:08:05 +01:00
Ryan McKinley
f94c0decbd
FeatureFlags: manage feature flags outside of settings.Cfg (#43692) 2022-01-20 13:42:05 -08:00
idafurjes
30aa24a183
Chore: Implement OpenTelemtry in Grafana (#42674)
* Separate Tracer interface to TracerService and Tracer

* Fix lint

* Fix:Make it possible to start spans for both opentracing and opentelemetry in ds proxy

* Add span methods, use span interface for rest of tracing

* Fix logs in tracing

* Fix tests that are related to tracing

* Fix resourcepermissions test

* Fix some tests

* Fix more tests

* Add TracingService to wire cli runner

* Remove GlobalTracer from bus

* Renaming test function

* Remove GlobalTracer from TSDB

* Replace GlobalTracer in api

* Adjust tests to the InitializeForTests func

* Remove GlobalTracer from services

* Remove GlobalTracer

* Remove bus.NewTest

* Remove Tracer interface

* Add InitializeForBus

* Simplify tests

* Clean up tests

* Rename TracerService to Tracer

* Update pkg/middleware/request_tracing.go

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>

* Initialize tracer before passing it to SQLStore initialization in commands

* Remove tests for opentracing

* Set span attributes correctly, remove unnecessary trace initiliazation form test

* Add tracer instance to newSQLStore

* Fix changes due to rebase

* Add modified tracing middleware test

* Fix opentracing implementation tags

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2022-01-20 11:10:12 +01:00
J Guerreiro
1e9818a69f
Add usage stats preview endpoint (#43899)
* Stats: add preview route for usage statistics

* Stats: respect reporting settings

* Stats: add tests to api endpoint

* Stats: unregister route

* Stats: always possible to preview stats

* Update pkg/infra/usagestats/service/api.go

Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>

Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>
2022-01-14 10:16:07 +01:00
Yuriy Tseretyan
8114f6b065
Use stack trace context in XORM trace logger (#43780)
* add caller and stack Valuer functions
* Add WithPrefix and WithSuffix similar to what go-kit offers
* replace New with just `with`. Remove filter wrapper because the first argument of the context argument is not logger but additional context.
* update Xorm logger to use custom depth to display the datastore code instead of xorm
2022-01-13 13:30:28 -05:00
ying-jeanne
a8eef45a44
Logger migration from log15 to gokit/log (#41636)
* migrate log15 to gokit/log

* fix console log

* update some unittest

* fix all unittest

* fix the build

* Update pkg/infra/log/log.go

Co-authored-by: Yuriy Tseretyan <tceretian@gmail.com>

* general type vector

* correct the level key

Co-authored-by: Yuriy Tseretyan <tceretian@gmail.com>
2022-01-06 22:28:05 +08:00
idafurjes
8e6d6af744
Rename DispatchCtx to Dispatch (#43563) 2021-12-28 17:36:22 +01:00
idafurjes
7936c4c522
Rename AddHandlerCtx to AddHandler (#43557) 2021-12-28 16:08:07 +01:00
idafurjes
56c3875bb9
Chore: Remove context.TODO (#43458)
* Remove context.TODO() from services

* Fix live test
2021-12-28 10:26:18 +01:00
J Guerreiro
a1b8b5d123
Usage stats: Count API keys (#42883)
* Stats: add api key count to usage stats

* Chore: correctly capitalize struct member
2021-12-22 17:37:45 +01:00
idafurjes
b8852ef6a3
Chore: Remove context.TODO() (#43409)
* Remove context.TODO() from services

* Fix live test

* Remove context.TODO
2021-12-22 11:02:42 +01:00
Marcus Efraimsson
54fa7b57fe
Chore: Fix flaky serverlock integration test (#42633) 2021-12-02 12:53:21 +01:00
Piotr Jamróz
bb24e8d578
Chore: Report frontend metrics for JS/CSS load time (#42421)
* Report frontend metrics for js/css load time

* Add FCP metric

* Change comment

* Simplify reporting frontend metrics
2021-12-02 09:34:39 +01:00
idafurjes
d993b12415
Add interface Tracer, add Opentelemetry (#41963)
* Add interface Tracer, add Opentelemetry

* Fix lint

* Fix failing tests and return error if config not parsed fo opentelemetry

* Update defaults.ini

Add comment with jaeger url

* go mod tidy

* Remove comments that are not needed

* Move OpentracingSpan to tracing.go

* Add opentelemetry to sample.ini
2021-12-01 17:05:08 +01:00