Commit Graph

174 Commits

Author SHA1 Message Date
Artur Wierzbicki
25e153e4e7 Search: in-memory index (#47709)
* #45498: add entity events table

* #45498: add entity events service

* #45498: hook up entity events service to http server

* #45498: use `dashboards.id` rather than `uid` and `org_id` in grn

* Update pkg/services/entityevents/service.go

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>

* #45498: move entityeventsservice to services/store

* #45498: add null check

* #45498: rename

* #45498: fix comment

* #45498: switch grn back to uid

* Search: listen for updates (#47719)

* #45498: wire entity event service with searchv2

* load last event id before building index for org 1

* fix service init in integration tests

* depend on required subset of event store methods

* Update pkg/services/sqlstore/migrations/entity_events_mig.go

Co-authored-by: Alexander Emelin <frvzmb@gmail.com>

* #45498: pointer receiver

* #45498: mockery!

* #45498: add entity events service to background services

* dashboard query pagination, allow queries while re-indexing

* log level cleanups, use rlock, add comments

* fix lint, check feature toggle in search v2 service

* use unix time for event created column

* add missing changes for created column

* fix integration tests init

* log re-index execution times on info level

* #45498: fix entityEventsService tests

* #45498: save events on dashboard delete

* use camel case for log labels

* formatting

* #45498: rename grn to entityid

* #45498: add `IsDisabled` to entityEventsService

* #45498: remove feature flag from migration

* better context usage, fix capacity, comments/cleanups

* replace print with logger

* Revert "#45498: remove feature flag from migration"

This reverts commit ed23968898.

* revert:revert:revert conditional feature flag

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
Co-authored-by: Alexander Emelin <frvzmb@gmail.com>
2022-04-27 12:29:39 +04:00
Kat Yang
098563179b Chore: Remove final x from sqlstore (#48086)
* Chore: Remove final x from everywhere

* Fix errors

* Fix: fix lint and nil pointer err

* Remove x from the sqlstore 🎉
2022-04-25 13:07:11 -04:00
Serge Zaitsev
e86b6662a1 Chore: Remove bus.Bus field (#47695)
* Chore: Remove bus.Bus field

* fix integration test
2022-04-13 15:24:13 +02:00
Serge Zaitsev
18e93c7077 Chore: Remove bus (#47511)
* Chore: Remove bus

* remove unused const
2022-04-08 16:15:06 +02:00
Karl Persson
ee9badecdf Access Control: Add test flag to enable access control (#46780)
* Add feature flags options to test db
2022-03-22 20:48:32 +01:00
ying-jeanne
adc0cbf176 remove global variable in annotation (#46746)
* remove global varaible in annotation

* remove todo

* replace intransaction with withdbtransaction

* fix typo
2022-03-22 19:20:57 +08:00
Serge Zaitsev
a231c6861c Chore: Remove bus.Dispatch from provisioning services (#44989)
* make getordbyname a method

* remove one dispatch from plugins provisioner

* remove bus from the plugins provisioner, skip test for now

* remove bus from datasource provisioning

* resolve tests in notifier provisioning

* remove bus from the dashboards provisioning service

* fix missing struct field

* fix getorgbyid method calls

* pass org store into dashboard provisioner

* fix test function prototype

* fix tests

* attempt to fix tests after the rebase

* fix integration test

* avoid using transaction

* remove comments
2022-02-23 11:12:37 +01:00
Alexander Emelin
28c30a34ad Comments: support live comments in dashboards and annotations (#44980) 2022-02-22 10:47:42 +03:00
Sofia Papagiannaki
6a38ce2307 Resolve depedency cycle (#45427) 2022-02-16 10:59:34 +02:00
Sofia Papagiannaki
d718ee1918 SQLStore: Prevent concurrent migrations (#44101)
* SQLStore: Prevent concurrent migrations

* Hide behind a feature toggle

* Configurable locking attempt timeout

* Update docs/sources/administration/configuration.md

Co-authored-by: Igor Suleymanov <radiohead@users.noreply.github.com>
Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>
2022-02-15 18:54:27 +02:00
Artur Wierzbicki
a025109647 Dash previews: populate crawler queue from SQL query (#44083)
* add SQL migrations

* dashboard previews from sql: poc

* added todos

* refactor: use the same enums where possible

* use useEffect, always return json

* added todo

* refactor + delete files after use

* refactor + fix manual thumbnail upload

* refactor: move all interactions with sqlStore to thumbnail repo

* refactor: remove file operations in thumb crawler/service

* refactor: fix dashboard_thumbs sql store

* refactor: extracted thumbnail fetching/updating to a hook

* refactor: store thumbnails in redux store

* refactor: store thumbnails in redux store

* refactor: private'd repo methods

* removed redux storage, saving images as blobs

* allow for configurable rendering timeouts

* added 1) query for dashboards with stale thumbnails, 2) command for marking thumbnails as stale

* use sql-based queue in crawler

* ui for marking thumbnails as stale

* replaced `stale` boolean prop with `state` enum

* introduce rendering session

* compilation errors

* fix crawler stop button

* rename thumbnail state frozen to locked

* #44449: fix merge conflicts

* #44449: remove thumb methods from `Store` interface

* #44449: clean filepath, defer file closing

* #44449: fix rendering.Theme cyclic import

* #44449: linting

* #44449: linting

* #44449: mutex'd crawlerStatus access

* #44449: added integration tests for `sqlstore.dashboard_thumbs`

* #44449: added comments to explain the `ThumbnailState` enum

* #44449: use os.ReadFile rather then os.Open

* #44449: always enable dashboardPreviews feature during integration tests

* #44449: remove sleep time, adjust number of threads

* #44449: review fix: add `orgId` to `DashboardThumbnailMeta`

* #44449: review fix: automatic parsing of thumbnailState

* #44449: lint fixes

* #44449: review fix: prefer `WithDbSession` over `WithTransactionalDbSession`

* #44449: review fix: add a comment explaining source of the filepath

* #44449: review fix: added filepath validation

* #44449: review fixes https://github.com/grafana/grafana/pull/45063/files @fzambia

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
Co-authored-by: Alexander Emelin <frvzmb@gmail.com>
2022-02-09 13:23:32 +04:00
Kat Yang
0e6300fb49 Chore: Remove bus from admin (#44920)
* Chore: Remove bus from admin

* fix test

Co-authored-by: Ying WANG <ying.wang@grafana.com>
2022-02-04 17:53:58 +01:00
Serge Zaitsev
61533a3cb4 Chore: Remove bus from health api (#44897)
* remove bus from health api

* fix health api tests

* use db health query as a method

* use pointer in sqlstore mock
2022-02-04 14:33:35 +01:00
Ryan McKinley
5d66194ec5 FeatureFlags: define features outside settings.Cfg (take 3) (#44443) 2022-01-26 09:44:20 -08:00
Katarina Yang
92ca38bedf Refactor: Change sqlstore.inTransaction to SQLStore.WithTransactionalDBSession in misc files (#43926)
* Refactor: Change sqlstore.inTransaction to SQLStore.WithTransactionalDBSession in misc files

* Refactor: Change .inTransaction in org.go file

* Refactor: Update init() to proper SQLStore handlers

* Refactor: Update funcs in tests to be sqlStore methods

* Refactor: Update API funcs to receive HTTPServer

* Fix: define methods on sqlstore

* Adjust GetSignedInUser calls

* Refactor: Add sqlStore to Service struct

* Chore: Add back black spaces to remove file from PR

Co-authored-by: Ida Furjesova <ida.furjesova@grafana.com>
2022-01-25 20:30:08 +01:00
Agnès Toulet
65bdb3a899 FeatureFlags: Revert managing feature flags outside of settings.Cfg (#44382)
* Revert "FeatureToggles: register all enterprise feature toggles (#44336)"

This reverts commit f53b3fb007.

* Revert "FeatureFlags: manage feature flags outside of settings.Cfg (#43692)"

This reverts commit f94c0decbd.
2022-01-24 16:08:05 +01:00
idafurjes
00c389933b Chore: Remove bus from team (#44218)
* Remove bus from team

* Fix api team test

* Remove bus from team members
2022-01-24 11:52:35 +01:00
Ryan McKinley
f94c0decbd FeatureFlags: manage feature flags outside of settings.Cfg (#43692) 2022-01-20 13:42:05 -08:00
idafurjes
30aa24a183 Chore: Implement OpenTelemtry in Grafana (#42674)
* Separate Tracer interface to TracerService and Tracer

* Fix lint

* Fix:Make it possible to start spans for both opentracing and opentelemetry in ds proxy

* Add span methods, use span interface for rest of tracing

* Fix logs in tracing

* Fix tests that are related to tracing

* Fix resourcepermissions test

* Fix some tests

* Fix more tests

* Add TracingService to wire cli runner

* Remove GlobalTracer from bus

* Renaming test function

* Remove GlobalTracer from TSDB

* Replace GlobalTracer in api

* Adjust tests to the InitializeForTests func

* Remove GlobalTracer from services

* Remove GlobalTracer

* Remove bus.NewTest

* Remove Tracer interface

* Add InitializeForBus

* Simplify tests

* Clean up tests

* Rename TracerService to Tracer

* Update pkg/middleware/request_tracing.go

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>

* Initialize tracer before passing it to SQLStore initialization in commands

* Remove tests for opentracing

* Set span attributes correctly, remove unnecessary trace initiliazation form test

* Add tracer instance to newSQLStore

* Fix changes due to rebase

* Add modified tracing middleware test

* Fix opentracing implementation tags

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2022-01-20 11:10:12 +01:00
Yuriy Tseretyan
8114f6b065 Use stack trace context in XORM trace logger (#43780)
* add caller and stack Valuer functions
* Add WithPrefix and WithSuffix similar to what go-kit offers
* replace New with just `with`. Remove filter wrapper because the first argument of the context argument is not logger but additional context.
* update Xorm logger to use custom depth to display the datastore code instead of xorm
2022-01-13 13:30:28 -05:00
idafurjes
87f148aa27 Chore: Add context to playlist (#41337)
* Add context to playlist

* Add AddEventListenerCtx and more context

* Update pkg/services/sqlstore/playlist.go

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>

* Update pkg/services/sqlstore/playlist.go

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>

* Update pkg/services/sqlstore/playlist.go

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>

* Fix lint

* Rename listener help function

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2021-11-19 14:32:14 +01:00
idafurjes
f40c1d0808 Chore: Add context apikey (#41698)
* Add context apikey

* Rename DeleteApiKeyCtx to DeleteApiKey
2021-11-18 09:50:03 +01:00
idafurjes
bb01f8c4cf Chore: Add context dashboard version (#41672)
* Add context dashboard version

* Fix codeql
2021-11-17 10:57:37 +01:00
idafurjes
da5033f3fb Chore: Add context to temp user (#41284)
* Add context to temp user

* Remove xorm and InTransaction
2021-11-04 11:17:07 +01:00
idafurjes
9340430723 Chore: Add alert ctx (#41161)
* Add context for alert

* Remove context.TODO

* Remove xorm

* Remove context.TODO

* Fix UsageStatsQuerier interface
2021-11-03 14:10:39 +01:00
Jean-Philippe Quéméner
d1aefa1792 Alerting: fix ngalert alertmanager SQL Syntax Errors (#40827)
* test kvstore in intregration tests with different databases

* escape 'key' in delete query

* export quote and use it in kvstore
2021-10-25 11:53:41 +02:00
idafurjes
b255c1b992 Chore: Add context to star and stats (#39591)
* Add context to star and stats

* Use WithTransactionalDbSession

* Add additional ctx

* Remove convey

* Fix star handler name

* Use WithDbSession, use DispatchCtx

* Remove xorm from star
2021-09-28 17:54:45 +02:00
idafurjes
65ebb04cf3 Chore: Add context to org users (#39526)
* Add context to org users

* Fix go lint

* Roll back xorm refactor

* Use WithTransactionalDbSession

* Update sqlstore.go

Fix typo

* Update org_users.go

Fix typo
2021-09-27 16:43:16 +02:00
Marcus Efraimsson
518a0d0458 Chore: Propagate context for dashboard guardian (#39201)
Require guardian.New to take context.Context as first argument. 
Migrates the GetDashboardAclInfoListQuery to be dispatched using context.

Ref #36734

Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>
Co-authored-by: sam boyer <sam.boyer@grafana.com>
2021-09-23 17:43:32 +02:00
Marcus Efraimsson
1e4a660ea7 Chore: Propagate context for quotas (#39205)
Propagate context for quotas and queries/commands.

Ref #36734
2021-09-20 17:05:30 +02:00
Arve Knudsen
78596a6756 Migrate to Wire for dependency injection (#32289)
Fixes #30144

Co-authored-by: dsotirakis <sotirakis.dim@gmail.com>
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
Co-authored-by: Ida Furjesova <ida.furjesova@grafana.com>
Co-authored-by: Jack Westbrook <jack.westbrook@gmail.com>
Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
Co-authored-by: Leon Sorokin <leeoniya@gmail.com>
Co-authored-by: Andrej Ocenas <mr.ocenas@gmail.com>
Co-authored-by: spinillos <selenepinillos@gmail.com>
Co-authored-by: Karl Persson <kalle.persson@grafana.com>
Co-authored-by: Leonard Gram <leo@xlson.com>
2021-08-25 15:11:22 +02:00
Marcus Efraimsson
11d03f7961 Chore: Use context.Context for the get home dashboard API endpoint (#36735)
Use context.Context for the get home dashboard API endpoint.

Ref #36734
2021-07-16 12:40:03 +02:00
Marcus Efraimsson
ca2223f705 Database: Fix incorrect format of isolation level configuration parameter for MySQL (#36565) 2021-07-09 19:05:48 +02:00
Kyle Brandt
dc3e17ba5b Alerting: Change rule migration to be based on feature flag (#33792)
makes it so the feature flag can be turned on off, and the migration will be cleared and rerun. All existing NG alert rules, configuration settings, etc are removed when disabling the feature flag.
for https://github.com/grafana/alerting-squad/issues/142

Co-authored-by: Sofia Papagiannaki <sofia@grafana.com>
2021-05-11 08:08:39 -04:00
Serge Zaitsev
7367cfc0a3 Add isolation level db configuration parameter (#33830)
* add isolation level db configuration parameter

* add isolation_level to default.ini and sample.ini

* add note that only mysql supports isolation levels for now

* mention isolation_level in the documentation

* Update docs/sources/administration/configuration.md

Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>

Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>
2021-05-10 17:54:13 +02:00
Serge Zaitsev
da13f88862 Redact sensitive values before logging them (#33829)
* use a common way to redact sensitive values before logging them

* fix panic on missing testCase.err, simplify require checks

* fix a silly typo

* combine readConfig and buildConnectionString methods, as they are closely related
2021-05-10 17:03:10 +02:00
ying-jeanne
bd66c8dde3 SQL data sources: Convert to return data frames (#32257)
Convert SQL data sources to return data frames.

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>
Co-authored-by: Will Browne <will.browne@grafana.com>
Co-authored-by: Hugo Häggmark <hugo.haggmark@gmail.com>
2021-05-05 16:46:07 +02:00
Sofia Papagiannaki
540f110220 [Alerting]: Extend quota service to optionally set limits on alerts (#33283)
* Quota: Extend service to set limit on alerts

* Add test for applying quota to alert rules

* Apply suggestions from code review

Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com>

* Get used alert quota only if naglert is enabled

* Set alert limit to zero if nglalert is not enabled
Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com>
2021-05-04 19:16:28 +03:00
Dafydd
01150ae962 Allow skip migrations in tests via environment variable (#32958) 2021-04-15 11:32:49 +02:00
Arve Knudsen
5a0780801b SQLStore: Close session in withDbSession (#31775)
* SQLStore: Close session in withDbSession

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* SQLStore.WithDbSession: Never use session from context

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2021-03-18 14:27:59 +01:00
Arve Knudsen
87c3a2b790 PluginManager: Make Plugins, Renderer and DataSources non-global (#31866)
* PluginManager: Make Plugins and DataSources non-global

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Fix integration tests

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Replace outdated command

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* DashboardService: Ensure it gets constructed with necessary parameters

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Fix build

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* DashboardService: Ensure it gets constructed with necessary parameters

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Remove dead code

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Fix test

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Fix test

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Remove FocusConvey

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Fix test

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Remove dead code

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Undo interface changes

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Backend: Move tsdbifaces.RequestHandler to plugins.DataRequestHandler

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Rename to DataSourceCount

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Consolidate dashboard interfaces into one

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Fix tests

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Fix tests

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Fix test

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Fix tests

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Fix tests

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Fix tests

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Fix tests

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Fix dashboard integration tests

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2021-03-17 16:06:10 +01:00
Carl Bergquist
3b36636318 Annotations: Make the annotation clean up batch size configurable (#31487)
Signed-off-by: bergquist <carl.bergquist@gmail.com>

Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com>
2021-03-08 14:16:07 +01:00
Dafydd
e17198af83 WIP: Skip this call when we skip migrations (#31216) 2021-02-15 12:57:20 +01:00
Tom Wilkie
07aa956667 Add option in database config to skip migrations for faster startup. (#30146)
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
2021-01-13 13:46:16 +00:00
Arve Knudsen
25048ebdf8 Chore: Add CloudWatch HTTP API tests (#29691)
* CloudWatch: Add HTTP API tests

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2021-01-07 11:36:13 +01:00
Dafydd
08fbe345a9 Instrumentation: re-enable database wrapper feature to expose counter and histogram for database queries (#29662)
ref https://github.com/grafana/grafana/issues/29489
2021-01-07 08:21:35 +01:00
ying-jeanne
375e8e4fd0 SQLStore: customise the limit of retrieved datasources per organisation (#29358)
* SQLStore: customise the limit of retrieved datasources per organisation

* update all suggestions regarding nil or 0 as default

* Apply suggestions from code review

Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>

* correct default.ini description + adding unittest

* Apply suggestions from code review

Co-authored-by: Sofia Papagiannaki <papagian@users.noreply.github.com>

* modify unittest name

Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>
Co-authored-by: Sofia Papagiannaki <papagian@users.noreply.github.com>
2020-12-28 12:24:42 +01:00
Arve Knudsen
dd2d206d99 Backend: Remove more globals (#29644)
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2020-12-15 19:09:04 +01:00
Arve Knudsen
12661e8a9d Move middleware context handler logic to service (#29605)
* middleware: Move context handler to own service

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

Co-authored-by: Emil Tullsted <sakjur@users.noreply.github.com>
Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
2020-12-11 11:44:44 +01:00
taciomcosta
10ff4eecef Backend: fix IPv6 address parsing erroneous (#28585)
* Backend: Fix parsing of client IP address

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>
2020-11-25 07:55:22 +01:00