Commit Graph

262 Commits

Author SHA1 Message Date
Artur Wierzbicki
e96f67ae2e
Renderer: Add sanitize API (#50936)
* svg fun

* #50597: add proto

* #50597: add sanitizer methods

* #50597: add provider

* #50597: use sanitizer

* #50597: use sanitizer

* update grafana to match new api

* add comments

* add capability check

* add timing

* update sanitize path

* improve log message

* strings.HasPrefix rather than filepath.IsAbs

* filepath.Clean + filepath.ToSlash for windows

* read 404

* remove `path.clean` from `getPathAndScope`

* add resp body close

* remove unneeded prop

* Update pkg/services/rendering/rendering.go

Co-authored-by: Agnès Toulet <35176601+AgnesToulet@users.noreply.github.com>

* remove test files

* filepath.ToSlash correct wrapping

* filepath.ToSlash correct wrapping

* filepath.ToSlash comment

* compilation error

* lint fix

* fix error message

* Update pkg/services/rendering/rendering.go

Co-authored-by: Agnès Toulet <35176601+AgnesToulet@users.noreply.github.com>

* add `image/svg+xml` mime type

* refactored log

* refactored log

Co-authored-by: Agnès Toulet <35176601+AgnesToulet@users.noreply.github.com>
2022-07-07 15:32:18 +04:00
Sven Grossmann
b7e22c37a8
UsageStats: fixed elasticsearch version number to semver (#49054)
* UsageStats: fixed elasticsearch version number

- The version numbering was changed from plain numbers to a semver-ish approach

* added missing version assertion

* adapted tests
2022-07-04 11:08:01 +02:00
Yuriy Tseretyan
739d3469bc
Chore: Update ConcreteLogger to implement gokit Logger interface (#51599) 2022-06-29 18:16:10 -04:00
Kristin Laemmert
945f015770
backend/datasources: move datasources models into the datasources service package (#51267)
* backend/datasources: move datasources models into the datasources service pkg
2022-06-27 12:23:15 -04:00
Ezequiel Victorero
4c4d6fd425
PublicDashboards: collect stats for public dashboards (#50553)
* PublicDashboards: collect stats for public dashboards
2022-06-23 12:02:57 -03:00
Sofia Papagiannaki
84e2e1ec8e
Log: Fix text logging for unsupported types (#51306)
* Fix text log for unsupported types

* Apply suggestions from code review

Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>
2022-06-23 13:24:26 +02:00
Selene
ecc15a2f71
KVStore: Extend kvstore to retrieve all items (#50848)
* Extend kvstore to retrieve all items

* Fix comment

* Fix tests

* Change test order

* Move test outside to avoid order conditions

* Update Items to GetAll function and return a map

* Add explanation of map result

* Add description comment

Co-authored-by: Tania B <yalyna.ts@gmail.com>
2022-06-23 11:12:07 +02:00
Dave Henderson
096330c53b
datasource metrics: Correct help text for histograms (#50942)
In #50420 a few summary metrics were converted to histograms,
but the Help text in a couple of them still referred to summaries.
This fixes that help text.

Signed-off-by: Dave Henderson <dave.henderson@grafana.com>
2022-06-16 14:18:09 +02:00
Ben Kochie
68691d7775
Convert some metrics to Histograms (#50420)
Because Summary metrics can not be aggreated, convert them to histograms
so that users with HA deployments can use these metrics.
* Convert metrics registration to promauto.
* Improve help text style.

Signed-off-by: SuperQ <superq@gmail.com>
2022-06-15 13:19:43 +02:00
Serge Zaitsev
ae9491c3a7
Chore: Make test tracer noop and return no errors (#50797) 2022-06-15 12:40:41 +02:00
Joan López de la Franca Beltran
97baa6911d
Metrics: Expose functions to initialize counters at zero (#50122) 2022-06-13 17:35:10 +02:00
Kat Yang
bd35e6917a
Chore: Exclude integration tests from running on test-backend step (#50359)
* Chore: Exclude integration tests from running on test-backend step

* Remove -v from go test command

* Add check to skip integration tests before each integration test

* Try to restart pipeline

* Retrying to make pipeline run
2022-06-10 11:46:21 -04:00
Todd Treece
99db588919
Data Sources: Add QueryData OAuth & cookie forwarding middleware (#50466) 2022-06-10 08:41:29 -04:00
Artur Wierzbicki
ef401f5d62
Storage: improve path validation, add tests (#50441)
* improve path validation

* add test case
2022-06-09 21:09:06 +04:00
Marcus Efraimsson
cddab15769
Chore: Upgrade grafana-plugin-sdk-go to v0.137.0 (#50443)
Co-authored-by: Todd Treece <todd.treece@grafana.com>
2022-06-09 10:10:58 +02:00
Kat Yang
31630edf0c
Chore: Remove Wrapf (#50128)
* Chore: Remove Wrapf

* Remove all Wrapf refs

* Remove last Wrapf ref

* Fix lint errors

* Remove Wrap and Wrapf definitions

* Remove unnecessary colon
2022-06-06 22:30:31 +02:00
Kat Yang
3c3039f5b3
Chore: Remove Wrap (#50048)
* Chore: Remove Wrap and Wrapf

* Fix: Add error check
2022-06-03 09:24:24 +02:00
idafurjes
24c6a73095
Tracing: Deprecate opentracing (#50058)
* Deprecate opentracing

* Fix comment

* Adjust comment

* Fix docs for opentelemetry

* Add deprecated to sample.ini
2022-06-02 14:13:00 +02:00
Kat Yang
c63ebc887b
Chore: Run integration tests without grabpl (#49448)
* Chore: Run integration tests without grabpl

* Add new step for integration tests in lib.star

* Remove old integration test step from lib.star

* Change drone signature

* Fix: Edit starlark integration step to not affect enterprise

* Remove all build tags & rename starlark integration test step

* Resync .drone.yml with .drone.star

* Fix lint errors

* Fix lint errors

* Fix lint errors

* Fix more lint errors

* Fix another lint error

* Rename integration test step

* Fix last lint error

* Recomment enterprise step

* Remove comment from Makefile

Co-authored-by: Ida Furjesova <ida.furjesova@grafana.com>
2022-06-01 14:55:22 -04:00
Carl Bergquist
9562fb389f
Metrics: Remove support for using summaries instead of histogram for HTTP instrumentation (#49985)
Signed-off-by: bergquist <carl.bergquist@gmail.com>
2022-06-01 14:33:59 +02:00
Emil Tullstedt
d3ffb9e245
Usage stats: Divide collection into multiple functions to isolate failures (#49928) 2022-06-01 10:27:06 +00:00
Dimitris Sotirakis
0d7a3209e7
etcd: Fix vuln CVE-2018-1098 (#49976)
* Update etcd

* Update go.sum

* Replace etcd with etcd/v3
2022-06-01 09:31:34 +02:00
sh0rez
3ca3a59079
pkg/web: remove dependency injection (#49123)
* pkg/web: store http.Handler internally

* pkg/web: remove injection

Removes any injection code from pkg/web.

It already was no longer functional, as we already only injected into
`http.Handler`, meaning we only inject ctx.Req and ctx.Resp.

Any other types (*Context, *ReqContext) were already accessed using the
http.Request.Context.Value() method.

* *: remove type mappings

Removes any call to the previously removed TypeMapper, as those were
non-functional already.

* pkg/web: remove Context.Invoke

was no longer used outside of pkg/web and also no longer functional
2022-05-24 15:35:08 -04:00
Kat Yang
50c2b4682a
Chore: Rename integration tests (#49438)
* Chore: Rename integration tests

* Remove one Integration

Co-authored-by: Ida Furjesova <ida.furjesova@grafana.com>
2022-05-24 11:04:03 +02:00
Joan López de la Franca Beltran
e43879e55d
Encryption: Add support for multiple data keys per day (#47765)
* Add database migrations

* Use short uids as data key ids

* Add support for manual data key rotation

* Fix duplicated mutex unlocks

* Fix migration

* Manage current data keys per name

* Adjust key re-encryption and test

* Modify rename column migration for MySQL compatibility

* Refactor secrets manager and data keys cache

* Multiple o11y adjustments

* Fix stats query

* Apply suggestions from code review

Co-authored-by: Tania <yalyna.ts@gmail.com>

* Fix linter

* Docs: Rotate data encryption keys API endpoint

Co-authored-by: Tania <yalyna.ts@gmail.com>
2022-05-23 13:13:55 +02:00
sh0rez
3d5d8c785b
pkg/web: restrict handler types (#48495)
Makes `pkg/web` only accept handles from the following set:

```go
	handlerStd       = func(http.ResponseWriter, *http.Request)
	handlerStdCtx    = func(http.ResponseWriter, *http.Request, *web.Context)
	handlerStdReqCtx = func(http.ResponseWriter, *http.Request, *models.ReqContext)
	handlerReqCtx    = func(*models.ReqContext)
	handlerReqCtxRes = func(*models.ReqContext) Response
	handlerCtx       = func(*web.Context)
```

This is a first step to reducing above set to only `http.Handler`.

---

Due to a cyclic import situation between `pkg/models` and `pkg/web`, parts of this PR were put into `pkg/api/response`, even though they definitely do not belong there. This however is _temporary_ until we untangle `models.ReqContext`.
2022-05-20 12:45:18 -04:00
Leonard Gram
aad2983350
Security: fixes CVE-2022-29170 (#49240)
* Request interceptor: block redirects

* handle location missing

* Update pkg/infra/httpclient/httpclientprovider/host_redirect_validation_middleware.go

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>

* Update pkg/infra/httpclient/httpclientprovider/host_redirect_validation_middleware.go

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>

* linter

* fixes tests

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2022-05-19 17:01:09 +02:00
ying-jeanne
4e3f53f106
Chore: Make logger mutable for regression test (#49146)
* Log: make logger mutable

* remove flag from public configuration
2022-05-18 17:04:52 +02:00
Artur Wierzbicki
5c321599c8
Storage: enable SQL backend (#48095)
* #45498: add String util to ListResponse for better UX

* #45498: refactor db_filestorage FS API backend - use path_hash in DB schema

* #45498: enable DB backend fs api tests

* #45498: add comment

* #45498: enable Storage feature flag during integration tests

* remove fmt.println

* #45498: reduce sizes of hash columns

* separate conditions

* #45498: make it easy to ignore backends when running fs api integration tests

* #45498: quote `key` column name

* #45498: reduce path_hash size

* #45498: verify `/{orgId}/{storageName}/` prefix convention in integration tests

* #45498: add etag to the sql table

* #45498: add etag to the sql table

* remove feature flag check (storage isn't dev-mode only)

* add cacheControl and content disposition

* add comments

* add path_hash comment

* explicitly set `path` column collation in `file` table for postgres
2022-05-16 10:26:40 -07:00
Serge Zaitsev
31ff23f542
Fix: add default nop trace exporter to opentelemetry (#48869) 2022-05-10 11:29:36 +02:00
Emil Tullstedt
51ff2b8c58
Logging: Unify logging fakes (#48822) 2022-05-06 17:44:22 +02:00
Joan López de la Franca Beltran
9826a694a8
Encryption: Add Prometheus metrics (#48603) 2022-05-06 10:21:55 +02:00
idafurjes
abb1618291
Add OTLP exporter for OpenTelemetry (#47987)
* Add OTLP exporter for OpenTelemtry

* Fix lint

* Refactore parse settings

* Add configuration for propagation + fix tests

* Fix tests and lint

* Fix alerting tests

* Add coments to config

* Add propagation to custom.ini
2022-05-05 10:37:26 +02:00
Artur Wierzbicki
736be74128
Chore: Add Usage stats providers registry (#48357)
* add usage stats providers

* migrate thumbnails service to `registry.ProvidesUsageStats`

* snake -> camel

* lint fix

* migrate accesscontrol to `registry.ProvidesUsageStats`

* add accesscontrol to usage stats providers registry

* fix test

* added a note about errors
2022-04-28 13:06:49 +04:00
Guilherme Caulada
a367ad730c
Secrets: Implement basic unified secret store service (#45804)
* wip: Implement kvstore for secrets

* wip: Refactor kvstore for secrets

* wip: Add format key function to secrets kvstore sql

* wip: Add migration for secrets kvstore

* Remove unused Key field from secrets kvstore

* Remove secret values from debug logs

* Integrate unified secrets with datasources

* Fix minor issues and tests for kvstore

* Create test service helper for secret store

* Remove encryption tests from datasources

* Move secret operations after datasources

* Fix datasource proxy tests

* Fix legacy data tests

* Add Name to all delete data source commands

* Implement decryption cache on sql secret store

* Fix minor issue with cache and tests

* Use secret type on secret store datasource operations

* Add comments to make create and update clear

* Rename itemFound variable to isFound

* Improve secret deletion and cache management

* Add base64 encoding to sql secret store

* Move secret retrieval to decrypted values function

* Refactor decrypt secure json data functions

* Fix expr tests

* Fix datasource tests

* Fix plugin proxy tests

* Fix query tests

* Fix metrics api tests

* Remove unused fake secrets service from query tests

* Add rename function to secret store

* Add check for error renaming secret

* Remove bus from tests to fix merge conflicts

* Add background secrets migration to datasources

* Get datasource secure json fields from secrets

* Move migration to secret store

* Revert "Move migration to secret store"

This reverts commit 7c3f872072.

* Add secret service to datasource service on tests

* Fix datasource tests

* Remove merge conflict on wire

* Add ctx to data source http transport on prometheus stats collector

* Add ctx to data source http transport on stats collector test
2022-04-25 13:57:45 -03:00
Artur Wierzbicki
9f0b6a5754
Storage: refactor filtering, improve performance (#47403)
* Storage: refactor filtering, improve performance

* added a comment to `newpathfilter`

* after merge fixes
2022-04-21 23:27:43 +04:00
Emil Tullstedt
d76e5d7c6a
Usage stats: Detect Prometheus flavors (#47942)
* Naïve Prometheus flavor detector

* Add concurrency and memoization

* Remove concurrency

* Fix tests

* close response body

* Add tests
2022-04-20 13:51:33 +00:00
Serge Zaitsev
41012af997
Tracing: Use common traceID context value for opentracing and opentelemetry (#46411)
* use common traceID context value for opentracing and opentelemetry

* support sampled trace IDs as well

* inject traceID into NormalResponse on errors

* Finally the test passed

* fix the test

* fix linter

* change the function parameter

Co-authored-by: Ying WANG <ying.wang@grafana.com>
2022-04-14 17:54:49 +02:00
ying-jeanne
f62c261900
fix terminal format (#47718) 2022-04-13 17:37:56 +02:00
ying-jeanne
06d2d44031
put the default logger format and feature toggle to fallback to old logger (#47584) 2022-04-13 11:02:19 +02:00
Artur Wierzbicki
712b239d5a
Storage: remove pointer on write path (#47580)
* remove pointer to contents in UpsertFileCommand

* add command to UpsertFileCommand

* fix client after API refactor
2022-04-12 16:58:09 +02:00
ying-jeanne
0bf889e058
Fix unmaarshal of double pointer (#47586)
* Fix unmaarshal of double pointer

* update sdk version
2022-04-12 09:30:34 +02:00
ying-jeanne
03ba91e8a4
add feature toggle with new format (#47336)
* add feature toggle with new format

* fix some comments ❤️

* Update pkg/infra/log/term/terminal_logger.go

Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>

Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>
2022-04-11 18:50:29 +02:00
Emil Tullstedt
3df625e9f4
UsageStats: Move stats collection to separate service (#47381)
* Remove specific stats from usage stats service

* Create statscollector service

* refactor

* Update and move tests

Mostly equivalent tests to before, but they've been divided over the two
services and removed the behavior driven legacy from GoConvey to
reduce the complexity of the tests.

* Collect featuremgmr metrics (copied over from #47407)

I removed the metrics registration from the feature manager in the merge
and re-add them in this commit. Separated to make things easier to
review.
2022-04-08 13:41:26 +02:00
Artur Wierzbicki
f1a1070d41
UsageStats: track enabled features (#47407)
* #47127: include enabled features in the usage stats reports

* #47127: convert feature names to snake cased metric names

* #47127: remove dead code

* #47127: lint fix

* #47127: convert GetUsageStats to return `map[string]interface{}`

* #47127: fix testssssssss

* #47127: fix testssssssss
2022-04-08 12:42:33 +02:00
ying-jeanne
c525db9cd8
fallback to the v8 logging behavior (#47353) 2022-04-05 20:15:41 +02:00
Marcus Efraimsson
659cf17e10
Logging: Fix syslog messages should be sent with correct severity (#47209) 2022-04-04 12:51:14 +02:00
Marcus Efraimsson
b4346a5613
Logging: Fixes filtering logs based on gokitlog levels (#47195) 2022-04-04 12:36:17 +02:00
Sergey Kostrukov
656ade9884
Shared Azure middleware between Azure Monitor and Prometheus datasources (#46002)
* Scopes in Azure middleware

* Enable Azure middleware without feature flag

* Use common Azure middleware in Azure Monitor

* Apply feature flag to JsonData configuration of Azure auth

* Enforce feature flag in Prometheus datasource

* Prometheus provider tests

* Datasource service tests

* Fix http client provider tests

* Pass sdkhttpclient.Options by reference

* Add middleware to httpclient.Options

* Remove dependency on Grafana settings

* Unit-tests updated

* Fix ds_proxy_test

* Fix service_test
2022-04-01 13:26:49 +02:00
Marcus Efraimsson
16db1ad46d
API: Fixes request logging middleware to include userId, orgId, uname (#47183)
Fixes #47006
2022-04-01 13:24:39 +02:00