Commit Graph

42 Commits

Author SHA1 Message Date
Marcus Efraimsson
e4c1a7a141
Tracing: Standardize on otel tracing (#75528) 2023-10-03 14:54:20 +02:00
Serge Zaitsev
bd12ce0cbc
Chore: capitalise log messages for app platform (#74336) 2023-09-04 18:46:08 +02:00
Ryan McKinley
025b2f3011
Chore: use any rather than interface{} (#74066) 2023-08-30 18:46:47 +03:00
Carl Bergquist
692bb9ed1a
Instrumentation: Add support for instrumenting database queries (#66022)
Signed-off-by: bergquist <carl.bergquist@gmail.com>
Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com>
2023-04-28 15:19:06 +02:00
Karl Persson
c931b8031e
SearchV2: Set correct batch limit when loading dashboards (#62314)
SearchV2: Set correct limit
2023-01-27 14:40:04 +01:00
Ieva
eb9ef34272
RBAC: Permission check performance improvements for the new search (#60729)
* Add checker and update the resource filter function for new search

* Add tests for checker

* small fixes

* handle location for panels correctly

* clean up checker code and extend the tests for it

* more fixes, but tests don't quite work yet

* a small change to return error

* cleanup

* more simplification

* fix tests

* correct wrong argument ordering & use constant

* Apply suggestions from code review

Co-authored-by: Artur Wierzbicki <artur.wierzbicki@grafana.com>

* import

* check general folder from permission checker function

* handle root folder aka general folder properly

* update tests

* clean up

* lint

* add fix from main

Co-authored-by: Karl Persson <kalle.persson@grafana.com>
Co-authored-by: Artur Wierzbicki <artur.wierzbicki@grafana.com>
2023-01-27 12:12:30 +00:00
Kristin Laemmert
dd147a3c31
chore: move entity models into entity store service (#62145) 2023-01-25 12:43:22 -05:00
Denis Limarev
e6dee8a723
Perfomance: Preallocate slices (#61580) 2023-01-17 11:50:17 +00:00
Ryan McKinley
5dc7b01f9d
EntityStore: Rename ObjectStore to EntityStore (part 1) (#59613) 2022-11-30 13:42:42 -08:00
Artur Wierzbicki
eff5450ff1
Search: Revert "load dashboard performance improvements" (#58730)
Revert "Search: load dashboard performance improvements (#57509)"

This reverts commit 1df8a85a42.
2022-11-15 02:25:13 -05:00
Artur Wierzbicki
1df8a85a42
Search: load dashboard performance improvements (#57509)
* fix windows storage init

* fixxx

* close session asap

* fix span end

* error handling

* add comment

* simplify

* simplify

* fix limit

* pass readDashboardCtx to summary builder

* add string slice validation
2022-10-27 14:52:42 +08:00
Artur Wierzbicki
e485ddd13d
Store: don't marshall/unmarshall the dashboard when building a summary (#57520)
* dont pretty print the dashboard

* stop marshalling the dashboard

* do not sanitize for searchV2

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
2022-10-25 09:39:17 +08:00
Kristin Laemmert
05709ce411
chore: remove sqlstore & mockstore dependencies from (most) packages (#57087)
* chore: add alias for InitTestDB and Session

Adds an alias for the sqlstore InitTestDB and Session, and updates tests using these to reduce dependencies on the sqlstore.Store.

* next pass of removing sqlstore imports
* last little bit
* remove mockstore where possible
2022-10-19 09:02:15 -04:00
Artur Wierzbicki
984ec00aac
Search: load dashboards optimization (#56933)
* search: load dashboards optimization

* search: load dashboards optimization

* search: close dashboard channel, return error when context is done

* search: refactor

* search: return err on ctx done

* search: remove sleep
2022-10-18 23:53:15 +08:00
Ryan McKinley
b24be6c0fc
ObjectStore: add a kind registry (#56507) 2022-10-08 12:05:46 -04:00
Ryan McKinley
e7e7763252
Store/Search: Explore a general interface to extract summary data from a blob (#55598) 2022-09-27 18:08:47 -04:00
Artur Wierzbicki
1f2cab9036
SearchV2: add more tracing spans (#55488)
* add additional spans to searchv2

* add initial setup span

* add dashboardCount attribute to readDashboard span

* add dashboard count to init org index span

* fix merge

Co-authored-by: Todd Treece <todd.treece@grafana.com>
2022-09-21 16:14:52 +02:00
Artur Wierzbicki
c3ca5405ce
Search: Add search index configuration options (#55525)
* Search: externalize config

* Search: update config descriptions

* Search: fix value

* Search: fix

* update

* Search: revert config values

* Search: rename copy/paste

* Search: fix tests
2022-09-20 19:09:55 -04:00
Artur Wierzbicki
9db2720016
Search: improve logging in case of failed dashboard loading phase (#55523) 2022-09-20 23:50:29 +02:00
Artur Wierzbicki
03af63d52e
SearchV2: add tracing to background jobs (#55250)
* searchv2: add tracing to background jobs

* searchv2: lint

* searchv2: lint

* searchv2: fix context passing

* searchv2: add init org index span

* searchv2: add traceid to logs

* searchv2: add db count to logs
2022-09-20 16:49:44 +02:00
Artur Wierzbicki
74158ed66b
Search: use SQL search as a fallback during bluge's initial indexing (#54095)
* Search: use SQL search as a fallback when bluge indexing is ongoing

* Search: lint

* Search: feedback fixes - return an empty frame with a special name

* Search: revert readiness check query type

* Search: remove println

* remove sleep, get coffee
2022-08-26 12:36:41 +04:00
Jo
062d255124
Handle ioutil deprecations (#53526)
* replace ioutil.ReadFile -> os.ReadFile

* replace ioutil.ReadAll -> io.ReadAll

* replace ioutil.TempFile -> os.CreateTemp

* replace ioutil.NopCloser -> io.NopCloser

* replace ioutil.WriteFile -> os.WriteFile

* replace ioutil.TempDir -> os.MkdirTemp

* replace ioutil.Discard -> io.Discard
2022-08-10 15:37:51 +02:00
Alexander Emelin
e791a4e576
Search: Move entity events to transaction (#52394) 2022-08-01 18:56:36 +03:00
Artur Wierzbicki
18daa6754c
Previews: datasource permissions (#52747)
* Previews: datasource permissions

* lint

* simplify - force non-null `ds_uids`

* add `canBeDisabled` to search service

* add `IncludeThumbnailsWithEmptyDsUids`

* remove force refresh migration

* refactor main preview service

* add safeguard

* revert ticker interval

* update testdata

* fix test

* add mock search service

* add datasources lookup test

* update migration

* extract ds lookup to its own package to avoid cyclic imports

* lint

* fix dashbaord extract, use the real datasource lookup in tests. IS IT BULLETPROOF YET?!

* fix dashbaord extract, use the real datasource lookup in tests. IS IT BULLETPROOF YET?!

* remove stale log

* consistent casing

* pass context to `createServiceAccount`

* filter out the special grafana ds
2022-07-28 16:40:26 +04:00
Artur Wierzbicki
17ea5f4f3e
Search: Handle special datasource types (#52446) 2022-07-22 21:10:53 -04:00
Artur Wierzbicki
63366615bb
Search: support datasource template variables when parsing dashboard JSON models (#51587)
* init

* support template variables

* support variables without curly braces

* add todo for `__all` case

* fix `$__all` case for non-multivalue

* extract some functions

* fix flakinesss

* support `$__all` and `default` template variables

* add todo

* compilation fix

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
2022-07-08 01:59:24 +04:00
Alexander Emelin
f233a74b70
Search: Downgrade Bluge, refactor code (#51560)
Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
2022-07-04 11:33:07 +03:00
Alexander Emelin
120c92b950
Search: Fix indexing - re-index after initial provisioning (#50959) 2022-06-27 18:11:08 +03:00
Alexander Emelin
cef81a6478
Search: Update bluge version (#50377) 2022-06-22 15:49:26 +03:00
Alexander Emelin
54e9408cfd
Search: Build indexes on start for all orgs (#50762) 2022-06-22 13:12:07 +03:00
Alexander Emelin
1ca2e2b6c2
Search: Sync state on read for HA consistency (#50152) 2022-06-22 12:21:43 +03:00
Alexander Emelin
da49f907bb
SearchV2: explicit dashboard loading order and cleanups (#50210) 2022-06-05 23:46:35 +02:00
Alexander Emelin
d2868a1ce7
SearchV2: instant local updates, folder events (#50001) 2022-06-03 13:11:32 -07:00
Alexander Emelin
1e7e149ed6
search: close unused reader and writer on re-indexing (#49458) 2022-05-24 09:42:21 +03:00
Ryan McKinley
9a59ccd644
Search: remove internal ID from bluge index (#49205)
Co-authored-by: Alexander Emelin <frvzmb@gmail.com>
2022-05-23 09:22:35 -07:00
Todd Treece
f0f33733a5
Search: Add DashboardIndexExtender interface (#49045)
Co-authored-by: Alexander Emelin <frvzmb@gmail.com>
2022-05-19 12:46:18 -04:00
Alexander Emelin
444c585c99
Search: limit max size of batches during indexing (#49187) 2022-05-19 18:57:26 +03:00
Ryan McKinley
9bbe951ec6
Usage: add gauge for panel/datasource/transformer types (#48991) 2022-05-18 00:25:28 +02:00
Alexander Emelin
baa50c58d0
Search: use only bluge-based search (#48968) 2022-05-16 16:22:45 -07:00
Ryan McKinley
d6d358ef26
Search: create bluge based index (#48606) 2022-05-09 01:00:09 -07:00
Alexander Emelin
4551f05994
Search: fix missing dashboards due to empty uid, log errors (#48361) 2022-04-28 19:29:09 +03:00
Artur Wierzbicki
25e153e4e7
Search: in-memory index (#47709)
* #45498: add entity events table

* #45498: add entity events service

* #45498: hook up entity events service to http server

* #45498: use `dashboards.id` rather than `uid` and `org_id` in grn

* Update pkg/services/entityevents/service.go

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>

* #45498: move entityeventsservice to services/store

* #45498: add null check

* #45498: rename

* #45498: fix comment

* #45498: switch grn back to uid

* Search: listen for updates (#47719)

* #45498: wire entity event service with searchv2

* load last event id before building index for org 1

* fix service init in integration tests

* depend on required subset of event store methods

* Update pkg/services/sqlstore/migrations/entity_events_mig.go

Co-authored-by: Alexander Emelin <frvzmb@gmail.com>

* #45498: pointer receiver

* #45498: mockery!

* #45498: add entity events service to background services

* dashboard query pagination, allow queries while re-indexing

* log level cleanups, use rlock, add comments

* fix lint, check feature toggle in search v2 service

* use unix time for event created column

* add missing changes for created column

* fix integration tests init

* log re-index execution times on info level

* #45498: fix entityEventsService tests

* #45498: save events on dashboard delete

* use camel case for log labels

* formatting

* #45498: rename grn to entityid

* #45498: add `IsDisabled` to entityEventsService

* #45498: remove feature flag from migration

* better context usage, fix capacity, comments/cleanups

* replace print with logger

* Revert "#45498: remove feature flag from migration"

This reverts commit ed23968898.

* revert:revert:revert conditional feature flag

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
Co-authored-by: Alexander Emelin <frvzmb@gmail.com>
2022-04-27 12:29:39 +04:00