Commit Graph

96 Commits

Author SHA1 Message Date
Kristin Laemmert
05709ce411
chore: remove sqlstore & mockstore dependencies from (most) packages (#57087)
* chore: add alias for InitTestDB and Session

Adds an alias for the sqlstore InitTestDB and Session, and updates tests using these to reduce dependencies on the sqlstore.Store.

* next pass of removing sqlstore imports
* last little bit
* remove mockstore where possible
2022-10-19 09:02:15 -04:00
Torkel Ödegaard
e31cb93ec0
NavTree: Make it possible to configure where in nav tree plugins live (#55484)
* NewIA: Plugin nav config

* progress

* Progress

* Things are working

* Add monitoring node

* Add alerts and incidents

* added experiment with standalone page

* Refactoring by adding a type for navtree root

* First test working

* More tests

* more tests

* Progress on richer config and sorting

* Sort weight working

* Path config

* Improving logic for not including admin or cfg nodes, making it the last step so that enterprise can add admin nodes without having to worry about the section not existing

* fixed index routes

* removed file

* Fixes

* Fixing tests

* Fixing more tests and adding support for weight config

* Updates

* Remove unused fake

* More fixes

* Minor tweak

* Minor fix

* Can now control position using sortweight even when existing items have no sortweight

* Added tests for frontend standalone page logic

* more tests

* Remove unused fake and fixed lint issue

* Moving reading settings to navtree impl package

* remove nav_id setting prefix

* Remove old test file

* Fix trailing newline

* Fixed bug with adding nil node

* fixing lint issue

* remove some code we have to rethink

* move read settings to PrivideService and switch to util.SplitString
2022-09-28 08:29:35 +02:00
Emil Tullstedt
f7a3e50b23
UsageStats: Add traces when sending usage stats (#55474) 2022-09-26 15:10:54 +02:00
idafurjes
a14621fff6
Chore: Add user service method SetUsingOrg and GetSignedInUserWithCacheCtx (#53343)
* Chore: Add user service method SetUsingOrg

* Chore: Add user service method GetSignedInUserWithCacheCtx

* Use method GetSignedInUserWithCacheCtx from user service

* Fix lint after rebase

* Fix lint

* Fix lint error

* roll back some changes

* Roll back changes in api and middleware

* Add xorm tags to SignedInUser ID fields
2022-08-11 13:28:55 +02:00
Jo
062d255124
Handle ioutil deprecations (#53526)
* replace ioutil.ReadFile -> os.ReadFile

* replace ioutil.ReadAll -> io.ReadAll

* replace ioutil.TempFile -> os.CreateTemp

* replace ioutil.NopCloser -> io.NopCloser

* replace ioutil.WriteFile -> os.WriteFile

* replace ioutil.TempDir -> os.MkdirTemp

* replace ioutil.Discard -> io.Discard
2022-08-10 15:37:51 +02:00
Serge Zaitsev
6e4b537ba6
Chore: Remove datasources from the Store interface (#53515) 2022-08-10 15:04:51 +02:00
idafurjes
6afad51761
Move SignedInUser to user service and RoleType and Roles to org (#53445)
* Move SignedInUser to user service and RoleType and Roles to org

* Use go naming convention for roles

* Fix some imports and leftovers

* Fix ldap debug test

* Fix lint

* Fix lint 2

* Fix lint 3

* Fix type and not needed conversion

* Clean up messages in api tests

* Clean up api tests 2
2022-08-10 11:56:48 +02:00
Sven Grossmann
b7e22c37a8
UsageStats: fixed elasticsearch version number to semver (#49054)
* UsageStats: fixed elasticsearch version number

- The version numbering was changed from plain numbers to a semver-ish approach

* added missing version assertion

* adapted tests
2022-07-04 11:08:01 +02:00
Kristin Laemmert
945f015770
backend/datasources: move datasources models into the datasources service package (#51267)
* backend/datasources: move datasources models into the datasources service pkg
2022-06-27 12:23:15 -04:00
Ezequiel Victorero
4c4d6fd425
PublicDashboards: collect stats for public dashboards (#50553)
* PublicDashboards: collect stats for public dashboards
2022-06-23 12:02:57 -03:00
Emil Tullstedt
d3ffb9e245
Usage stats: Divide collection into multiple functions to isolate failures (#49928) 2022-06-01 10:27:06 +00:00
sh0rez
3ca3a59079
pkg/web: remove dependency injection (#49123)
* pkg/web: store http.Handler internally

* pkg/web: remove injection

Removes any injection code from pkg/web.

It already was no longer functional, as we already only injected into
`http.Handler`, meaning we only inject ctx.Req and ctx.Resp.

Any other types (*Context, *ReqContext) were already accessed using the
http.Request.Context.Value() method.

* *: remove type mappings

Removes any call to the previously removed TypeMapper, as those were
non-functional already.

* pkg/web: remove Context.Invoke

was no longer used outside of pkg/web and also no longer functional
2022-05-24 15:35:08 -04:00
Joan López de la Franca Beltran
e43879e55d
Encryption: Add support for multiple data keys per day (#47765)
* Add database migrations

* Use short uids as data key ids

* Add support for manual data key rotation

* Fix duplicated mutex unlocks

* Fix migration

* Manage current data keys per name

* Adjust key re-encryption and test

* Modify rename column migration for MySQL compatibility

* Refactor secrets manager and data keys cache

* Multiple o11y adjustments

* Fix stats query

* Apply suggestions from code review

Co-authored-by: Tania <yalyna.ts@gmail.com>

* Fix linter

* Docs: Rotate data encryption keys API endpoint

Co-authored-by: Tania <yalyna.ts@gmail.com>
2022-05-23 13:13:55 +02:00
sh0rez
3d5d8c785b
pkg/web: restrict handler types (#48495)
Makes `pkg/web` only accept handles from the following set:

```go
	handlerStd       = func(http.ResponseWriter, *http.Request)
	handlerStdCtx    = func(http.ResponseWriter, *http.Request, *web.Context)
	handlerStdReqCtx = func(http.ResponseWriter, *http.Request, *models.ReqContext)
	handlerReqCtx    = func(*models.ReqContext)
	handlerReqCtxRes = func(*models.ReqContext) Response
	handlerCtx       = func(*web.Context)
```

This is a first step to reducing above set to only `http.Handler`.

---

Due to a cyclic import situation between `pkg/models` and `pkg/web`, parts of this PR were put into `pkg/api/response`, even though they definitely do not belong there. This however is _temporary_ until we untangle `models.ReqContext`.
2022-05-20 12:45:18 -04:00
Joan López de la Franca Beltran
9826a694a8
Encryption: Add Prometheus metrics (#48603) 2022-05-06 10:21:55 +02:00
Artur Wierzbicki
736be74128
Chore: Add Usage stats providers registry (#48357)
* add usage stats providers

* migrate thumbnails service to `registry.ProvidesUsageStats`

* snake -> camel

* lint fix

* migrate accesscontrol to `registry.ProvidesUsageStats`

* add accesscontrol to usage stats providers registry

* fix test

* added a note about errors
2022-04-28 13:06:49 +04:00
Guilherme Caulada
a367ad730c
Secrets: Implement basic unified secret store service (#45804)
* wip: Implement kvstore for secrets

* wip: Refactor kvstore for secrets

* wip: Add format key function to secrets kvstore sql

* wip: Add migration for secrets kvstore

* Remove unused Key field from secrets kvstore

* Remove secret values from debug logs

* Integrate unified secrets with datasources

* Fix minor issues and tests for kvstore

* Create test service helper for secret store

* Remove encryption tests from datasources

* Move secret operations after datasources

* Fix datasource proxy tests

* Fix legacy data tests

* Add Name to all delete data source commands

* Implement decryption cache on sql secret store

* Fix minor issue with cache and tests

* Use secret type on secret store datasource operations

* Add comments to make create and update clear

* Rename itemFound variable to isFound

* Improve secret deletion and cache management

* Add base64 encoding to sql secret store

* Move secret retrieval to decrypted values function

* Refactor decrypt secure json data functions

* Fix expr tests

* Fix datasource tests

* Fix plugin proxy tests

* Fix query tests

* Fix metrics api tests

* Remove unused fake secrets service from query tests

* Add rename function to secret store

* Add check for error renaming secret

* Remove bus from tests to fix merge conflicts

* Add background secrets migration to datasources

* Get datasource secure json fields from secrets

* Move migration to secret store

* Revert "Move migration to secret store"

This reverts commit 7c3f872072.

* Add secret service to datasource service on tests

* Fix datasource tests

* Remove merge conflict on wire

* Add ctx to data source http transport on prometheus stats collector

* Add ctx to data source http transport on stats collector test
2022-04-25 13:57:45 -03:00
Emil Tullstedt
d76e5d7c6a
Usage stats: Detect Prometheus flavors (#47942)
* Naïve Prometheus flavor detector

* Add concurrency and memoization

* Remove concurrency

* Fix tests

* close response body

* Add tests
2022-04-20 13:51:33 +00:00
ying-jeanne
0bf889e058
Fix unmaarshal of double pointer (#47586)
* Fix unmaarshal of double pointer

* update sdk version
2022-04-12 09:30:34 +02:00
Emil Tullstedt
3df625e9f4
UsageStats: Move stats collection to separate service (#47381)
* Remove specific stats from usage stats service

* Create statscollector service

* refactor

* Update and move tests

Mostly equivalent tests to before, but they've been divided over the two
services and removed the behavior driven legacy from GoConvey to
reduce the complexity of the tests.

* Collect featuremgmr metrics (copied over from #47407)

I removed the metrics registration from the feature manager in the merge
and re-add them in this commit. Separated to make things easier to
review.
2022-04-08 13:41:26 +02:00
Artur Wierzbicki
f1a1070d41
UsageStats: track enabled features (#47407)
* #47127: include enabled features in the usage stats reports

* #47127: convert feature names to snake cased metric names

* #47127: remove dead code

* #47127: lint fix

* #47127: convert GetUsageStats to return `map[string]interface{}`

* #47127: fix testssssssss

* #47127: fix testssssssss
2022-04-08 12:42:33 +02:00
Jguer
d5883c1b27
Service Accounts: Implement basic usage stats (#46619)
* Stats: do not count SAs as users

* Stats: implement basic service account metrics

* Stats: do not count service account tokens as api keys

* Stats: fix metric names

* Stats: add SA stats test

* rename user to sa
2022-03-16 16:54:34 +01:00
Selene
2e6f14d17f
Bus: Tests cleanup (#46025)
* Delete unused bus from usagestats

* Few updates to remove bus from searchusers test

* Fix import
2022-03-02 11:05:31 +01:00
idafurjes
d8a56d08ba
Remove bus from usage stats (#45275) 2022-02-11 14:04:15 +01:00
Marcus Efraimsson
bc7e55d99b
Chore: Fix log filters (#44681) 2022-02-03 23:20:02 +08:00
J Guerreiro
1e9818a69f
Add usage stats preview endpoint (#43899)
* Stats: add preview route for usage statistics

* Stats: respect reporting settings

* Stats: add tests to api endpoint

* Stats: unregister route

* Stats: always possible to preview stats

* Update pkg/infra/usagestats/service/api.go

Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>

Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>
2022-01-14 10:16:07 +01:00
ying-jeanne
a8eef45a44
Logger migration from log15 to gokit/log (#41636)
* migrate log15 to gokit/log

* fix console log

* update some unittest

* fix all unittest

* fix the build

* Update pkg/infra/log/log.go

Co-authored-by: Yuriy Tseretyan <tceretian@gmail.com>

* general type vector

* correct the level key

Co-authored-by: Yuriy Tseretyan <tceretian@gmail.com>
2022-01-06 22:28:05 +08:00
idafurjes
8e6d6af744
Rename DispatchCtx to Dispatch (#43563) 2021-12-28 17:36:22 +01:00
idafurjes
7936c4c522
Rename AddHandlerCtx to AddHandler (#43557) 2021-12-28 16:08:07 +01:00
J Guerreiro
a1b8b5d123
Usage stats: Count API keys (#42883)
* Stats: add api key count to usage stats

* Chore: correctly capitalize struct member
2021-12-22 17:37:45 +01:00
idafurjes
b8852ef6a3
Chore: Remove context.TODO() (#43409)
* Remove context.TODO() from services

* Fix live test

* Remove context.TODO
2021-12-22 11:02:42 +01:00
idafurjes
e6123bc3ef
Replace AddHandler with AddHandlerCtx in tests (#42585) 2021-12-01 15:43:31 +01:00
idafurjes
725dbf8d95
Chore: Add context to datasource service (#42294)
* Add context to datasource service

* Adjust wire for ShouldBeReported method

* Replace inTransactionCtx
2021-11-26 18:10:36 +01:00
Will Browne
2e3e7a7e55
Plugins: Plugin Store API returns DTO model (#41340)
* toying around

* fix refs

* remove unused fields

* go further

* add context

* ensure streaming handler is set
2021-11-17 12:04:22 +01:00
Guilherme Caulada
5afaf8930c
Add monthlyActiveUsers to stats API and usage report (#41289)
Closes grafana/grafana-enterprise-partnerships-team#7
2021-11-03 18:59:05 -03:00
idafurjes
9340430723
Chore: Add alert ctx (#41161)
* Add context for alert

* Remove context.TODO

* Remove xorm

* Remove context.TODO

* Fix UsageStatsQuerier interface
2021-11-03 14:10:39 +01:00
Will Browne
b80fbe03f0
Plugins: Refactor Plugin Management (#40477)
* add core plugin flow

* add instrumentation

* move func

* remove cruft

* support external backend plugins

* refactor + clean up

* remove comments

* refactor loader

* simplify core plugin path arg

* cleanup loggers

* move signature validator to plugins package

* fix sig packaging

* cleanup plugin model

* remove unnecessary plugin field

* add start+stop for pm

* fix failures

* add decommissioned state

* export fields just to get things flowing

* fix comments

* set static routes

* make image loading idempotent

* merge with backend plugin manager

* re-use funcs

* reorder imports + remove unnecessary interface

* add some TODOs + remove unused func

* remove unused instrumentation func

* simplify client usage

* remove import alias

* re-use backendplugin.Plugin interface

* re order funcs

* improve var name

* fix log statements

* refactor data model

* add logic for dupe check during loading

* cleanup state setting

* refactor loader

* cleanup manager interface

* add rendering flow

* refactor loading + init

* add renderer support

* fix renderer plugin

* reformat imports

* track errors

* fix plugin signature inheritance

* name param in interface

* update func comment

* fix func arg name

* introduce class concept

* remove func

* fix external plugin check

* apply changes from pm-experiment

* fix core plugins

* fix imports

* rename interface

* comment API interface

* add support for testdata plugin

* enable alerting + use correct core plugin contracts

* slim manager API

* fix param name

* fix filter

* support static routes

* fix rendering

* tidy rendering

* get tests compiling

* fix install+uninstall

* start finder test

* add finder test coverage

* start loader tests

* add test for core plugins

* load core + bundled test

* add test for nested plugin loading

* add test files

* clean interface + fix registering some core plugins

* refactoring

* reformat and create sub packages

* simplify core plugin init

* fix ctx cancel scenario

* migrate initializer

* remove Init() funcs

* add test starter

* new logger

* flesh out initializer tests

* refactoring

* remove unused svc

* refactor rendering flow

* fixup loader tests

* add enabled helper func

* fix logger name

* fix data fetchers

* fix case where plugin dir doesn't exist

* improve coverage + move dupe checking to loader

* remove noisy debug logs

* register core plugins automagically

* add support for renderer in catalog

* make private func + fix req validation

* use interface

* re-add check for renderer in catalog

* tidy up from moving to auto reg core plugins

* core plugin registrar

* guards

* copy over core plugins for test infra

* all tests green

* renames

* propagate new interfaces

* kill old manager

* get compiling

* tidy up

* update naming

* refactor manager test + cleanup

* add more cases to finder test

* migrate validator to field

* more coverage

* refactor dupe checking

* add test for plugin class

* add coverage for initializer

* split out rendering

* move

* fixup tests

* fix uss test

* fix frontend settings

* fix grafanads test

* add check when checking sig errors

* fix enabled map

* fixup

* allow manual setup of CM

* rename to cloud-monitoring

* remove TODO

* add installer interface for testing

* loader interface returns

* tests passing

* refactor + add more coverage

* support 'stackdriver'

* fix frontend settings loading

* improve naming based on package name

* small tidy

* refactor test

* fix renderer start

* make cloud-monitoring plugin ID clearer

* add plugin update test

* add integration tests

* don't break all if sig can't be calculated

* add root URL check test

* add more signature verification tests

* update DTO name

* update enabled plugins comment

* update comments

* fix linter

* revert fe naming change

* fix errors endpoint

* reset error code field name

* re-order test to help verify

* assert -> require

* pm check

* add missing entry + re-order

* re-check

* dump icon log

* verify manager contents first

* reformat

* apply PR feedback

* apply style changes

* fix one vs all loading err

* improve log output

* only start when no signature error

* move log

* rework plugin update check

* fix test

* fix multi loading from cfg.PluginSettings

* improve log output #2

* add error abstraction to capture errors without registering a plugin

* add debug log

* add unsigned warning

* e2e test attempt

* fix logger

* set home path

* prevent panic

* alternate

* ugh.. fix home path

* return renderer even if not started

* make renderer plugin managed

* add fallback renderer icon, update renderer badge + prevent changes when renderer is installed

* fix icon loading

* rollback renderer changes

* use correct field

* remove unneccessary block

* remove newline

* remove unused func

* fix bundled plugins base + module fields

* remove unused field since refactor

* add authorizer abstraction

* loader only returns plugins expected to run

* fix multi log output
2021-11-01 10:53:33 +01:00
Will Browne
e4297006f6
Chore: Propagate context for data source provisioning (#40235)
* context all the things

* apply feedback

* rollback some alerting changes

* rollback some alerting changes #2

* more rollbacks

* more rollbacks #2

* more rollbacks #3

* more rollbacks #4

* fix integration test

* add missing context

* add missing and remove incorrect dispatch
2021-10-18 17:06:19 +02:00
Tania B
990911a3b9
Chore: Cleanup usageStatsMock duplication (#39710)
* Chore: Cleanup usageStatsMock duplication

* Remove making a new slice for UsageStatsMock
2021-09-29 09:59:02 +03:00
idafurjes
b255c1b992
Chore: Add context to star and stats (#39591)
* Add context to star and stats

* Use WithTransactionalDbSession

* Add additional ctx

* Remove convey

* Fix star handler name

* Use WithDbSession, use DispatchCtx

* Remove xorm from star
2021-09-28 17:54:45 +02:00
Tania B
b9acdce837
Chore: Fix grafana live stats reset when sending usage stats (#39558) 2021-09-23 12:55:00 +03:00
Joan López de la Franca Beltran
b891af935a
Usage Stats: Decouple from alerting.UsageStatsQuerier (#39515)
* Usage Stats: Decouple from alerting.UsageStatsQuerier

Co-authored-by: Tania B <yalyna.ts@gmail.com>
2021-09-23 03:12:12 +02:00
Joan López de la Franca Beltran
c75737c808
Usage Stats: Decouple from GrafanaLive (#39512)
* Usage Stats: Decouple from GrafanaLive
2021-09-22 16:28:40 +02:00
Joan López de la Franca Beltran
a680162792
Usage Stats: Split domain & service packages (#39488) 2021-09-21 20:50:37 +02:00
Dan Cech
f8ae71af5b
Usage Stats Updates (#39235)
* add randomly-generated anonymous id to usage reports
* include uptime in stats
* add last_sent tracking, remove metricsLogger
2021-09-15 11:47:44 -04:00
Dan Cech
9dfd469afc
add daily active counts to stats (#38842)
* add daily active counts to stats

* standardize on int64, update tests
2021-09-13 10:29:35 -04:00
Torkel Ödegaard
51776e6bd3
UsageStats: Add connected users and client (#38811) 2021-09-07 18:50:28 +02:00
Arve Knudsen
78596a6756
Migrate to Wire for dependency injection (#32289)
Fixes #30144

Co-authored-by: dsotirakis <sotirakis.dim@gmail.com>
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
Co-authored-by: Ida Furjesova <ida.furjesova@grafana.com>
Co-authored-by: Jack Westbrook <jack.westbrook@gmail.com>
Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
Co-authored-by: Leon Sorokin <leeoniya@gmail.com>
Co-authored-by: Andrej Ocenas <mr.ocenas@gmail.com>
Co-authored-by: spinillos <selenepinillos@gmail.com>
Co-authored-by: Karl Persson <kalle.persson@grafana.com>
Co-authored-by: Leonard Gram <leo@xlson.com>
2021-08-25 15:11:22 +02:00
Karl Persson
4127db3d3a
Remove Licensing as dependency of UsageStats (#37259)
* Remove Licensing as dependency

Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>
2021-07-27 17:19:03 +02:00
Selene
1c74bb3992
UsageStats: Extend usage stats for count permissions in folders and dashborads (#36065)
* Add usage stats for permissions for dashboards and folders

* Change double quotes for simple ones
2021-07-16 10:14:33 +02:00