IntenseWebs/grafana
3
0
Fork 0
mirror of https://github.com/grafana/grafana.git synced 2026-07-30 00:08:10 -05:00
Code Issues Packages Projects Releases Wiki Activity
38,951 Commits 1,123 Branches 715 Tags
c090de9ed9e3e3865d36a27256a28c8f64179736
Commit Graph
90 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Artur WierzbickiandTodd Treece c26cf6a517 Search: Sort alphabetically in the folder view, increase the limit of the folder search from 50 to 1000 (#57078) 
* search: sort folders

* search: increase the limit for folder search to 1000

* add folder view sort test

* search: getFolderViewSort

* search: revert test

Co-authored-by: Todd Treece <todd.treece@grafana.com>
 2022-10-18 14:10:53 +08:00
Artur Wierzbicki 4b2a940659 Search: wrap data.Frame in a backend.DataResponse and return it directly when the index is not ready (#56522) 2022-10-07 11:32:37 -07:00
Artur Wierzbicki bf264d2f76 Query library: requiresDevMode dummy backend (#56466) 
* query library - dummy backend

* fix tests

* dont explicitly marshall backend dataresponse

* skip integration tests

* null check for tests

* added query library to codeowners

* null check for tests

* lint
 2022-10-07 11:31:45 -07:00
Artur WierzbickiandTodd Treece a94acb7f61 Object store: get user from context (#56346) 
* GRPC Server: Add signedInUser to context after auth

* add permissions to signedInUser

* add access control permissions test

* add additional signedInUser checks

* get user from context

* move `UserFromContext` to object/auth.go

Co-authored-by: Todd Treece <todd.treece@grafana.com>
 2022-10-05 17:00:34 +00:00
Artur WierzbickiandRyan McKinley 85b965cbec Storage: Dummy object server and basic integration tests (#56014) 
* object extractors

* update bluge to use summary values

* gosec

* move to store/object package

* references

* references

* references

* same thign but with protobuf

* now the service

* now with summary

* now with summary

* from protobuf

* from protobuf

* cleanup

* remove hand crafted file

* update proto definitions

* update comments

* remove properties

* remove properties

* re-generate

* add batch

* move ref to raw struct

* GRPC test infra

* fix merge

* add delete

* lint

* rename to dummyobjectserver

* update comment

* refactor collection, simplify dummy server

* update

* refactor test structure

* more tests

* more tests

* replace collection with infra/persistentcollection

* skip if not integration test suite

* very important lint fix

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
 2022-09-30 15:56:07 -04:00
Artur Wierzbicki ea1334c01d Chore: Persistent collection (#56074) 
* persistent collection

* dont remove temp dir

* temp dir change

* lint

* add experimental comment

* move to X package

* lint

* orgID -> namespace
 2022-09-30 21:34:44 +04:00
Artur Wierzbicki 64eff8196c Chore: expose grpc server address (#56013) 2022-09-29 08:42:01 -04:00
Artur Wierzbicki 34f18aacd6 Chore: add more input parameters to serviceaccounts/test/common.go (#55994) 2022-09-29 12:11:53 +01:00
Artur Wierzbicki 202dce66ff Search: add basic e2e test for the folder view (#55820) 2022-09-28 09:46:24 +04:00
Artur Wierzbicki 814211282a Search: Fix tags query (#55851) 
* Search: fix tags query

* Search: fix type
 2022-09-27 21:16:49 +04:00
Artur Wierzbicki af9e58eeeb Search: fix cache in the frontend search engine (#55681) 2022-09-23 10:13:14 -07:00
Artur Wierzbicki f8d69415ca Search: create a separate HTTP endpoint (#55634) 
* search: create a separate http endpoint

* search: extract api uri

* search: rename uri

* search: replicate the readiness check

* search: replicate the readiness check metric

* search: update mock
 2022-09-22 19:02:09 -04:00
Artur Wierzbicki e341aa4dab Search: fix front end query search condition (#55611) 
* fix frontend query condition

* refactor to `inludes`
 2022-09-22 05:34:09 -04:00
Artur WierzbickiandTodd Treece 1f2cab9036 SearchV2: add more tracing spans (#55488) 
* add additional spans to searchv2

* add initial setup span

* add dashboardCount attribute to readDashboard span

* add dashboard count to init org index span

* fix merge

Co-authored-by: Todd Treece <todd.treece@grafana.com>
 2022-09-21 16:14:52 +02:00
Artur Wierzbicki 5a1f004fb1 Live: Fix live streaming with live-service-web-worker feature flag enabled (#55528) 2022-09-21 13:09:16 +02:00
Artur Wierzbicki c3ca5405ce Search: Add search index configuration options (#55525) 
* Search: externalize config

* Search: update config descriptions

* Search: fix value

* Search: fix

* update

* Search: revert config values

* Search: rename copy/paste

* Search: fix tests
 2022-09-20 19:09:55 -04:00
Artur Wierzbicki 9db2720016 Search: improve logging in case of failed dashboard loading phase (#55523) 2022-09-20 23:50:29 +02:00
Artur Wierzbicki 03af63d52e SearchV2: add tracing to background jobs (#55250) 
* searchv2: add tracing to background jobs

* searchv2: lint

* searchv2: lint

* searchv2: fix context passing

* searchv2: add init org index span

* searchv2: add traceid to logs

* searchv2: add db count to logs
 2022-09-20 16:49:44 +02:00
Artur WierzbickiandRyan McKinley e19f36649f Storage: Content root storage (#54929) 
Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
 2022-09-12 18:34:46 -04:00
Artur Wierzbicki 1353177e15 Search: Add substring matcher (#54813) 
* search: bring back substring search

* lint fix

* search: bring back ngram

* search: bring back ngram

* search: more tests
 2022-09-08 12:10:30 +02:00
Artur Wierzbicki 9284216880 Search: ordering tests (#54432) 
* search ordering tests

* lintt
 2022-08-30 18:58:19 +02:00
Artur WierzbickiandRyan McKinley 7a340f486b Storage: add WithContents option to storage.Get() (#53105) 
* Storage: add `WithContents` option to `storage.Get()`

* fix tests

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
 2022-08-30 15:23:16 +02:00
Artur Wierzbicki 74158ed66b Search: use SQL search as a fallback during bluge's initial indexing (#54095) 
* Search: use SQL search as a fallback when bluge indexing is ongoing

* Search: lint

* Search: feedback fixes - return an empty frame with a special name

* Search: revert readiness check query type

* Search: remove println

* remove sleep, get coffee
 2022-08-26 12:36:41 +04:00
Artur Wierzbicki 5c0bf2ba39 Storage: fix creating a new dashboard with dashboardsFromStorage feature flag enabled 2022-08-25 14:04:09 -04:00
Artur Wierzbicki 2a0ae74f96 Live: fix StreamingDataFrame length on replace action and schema change (#53796) 
* streaming-data-frame

* add more expect.length in tests
 2022-08-18 00:58:38 +04:00
Artur Wierzbicki f61a97a0ab Search: rename remove search constraints (#53190) 2022-08-03 16:44:51 +04:00
Artur Wierzbicki 0d817987ec Storage: move file quota check to service (#53058) 
* Storage: move file quota check to service

* Fix tests
 2022-08-01 22:53:44 +04:00
Artur Wierzbicki 784cfcf2b0 Search: add request metrics (#53028) 
* Search: add request metrics

* Search: consistent naming

* Search: remove success counter

* Search: update buckets

* Search: remove log
 2022-08-01 15:23:41 +04:00
Artur Wierzbicki 046a2602ff Storage: optimize sql list operation (#53009) 2022-07-30 00:19:28 +04:00
Artur Wierzbickiandnmarrs 08dabfaffc Storage: add global resources/system (#53000) 
Co-authored-by: nmarrs <nathanielmarrs@gmail.com>
 2022-07-29 15:58:47 -04:00
Artur Wierzbicki 273c37c27d Previews: return an empty array, rather than a null, if a dashboard has no datasources (#52961) 2022-07-28 22:49:06 +04:00
Artur Wierzbicki 449acb56a9 Previews: fix dashboard.thumbnails.ds_uids migration (#52960) 2022-07-28 21:50:02 +04:00
Artur Wierzbicki 18daa6754c Previews: datasource permissions (#52747) 
* Previews: datasource permissions

* lint

* simplify - force non-null `ds_uids`

* add `canBeDisabled` to search service

* add `IncludeThumbnailsWithEmptyDsUids`

* remove force refresh migration

* refactor main preview service

* add safeguard

* revert ticker interval

* update testdata

* fix test

* add mock search service

* add datasources lookup test

* update migration

* extract ds lookup to its own package to avoid cyclic imports

* lint

* fix dashbaord extract, use the real datasource lookup in tests. IS IT BULLETPROOF YET?!

* fix dashbaord extract, use the real datasource lookup in tests. IS IT BULLETPROOF YET?!

* remove stale log

* consistent casing

* pass context to `createServiceAccount`

* filter out the special grafana ds
 2022-07-28 16:40:26 +04:00
Artur Wierzbicki 9e841ef21d Storage: fix delete folder (#52860) 2022-07-28 12:27:26 +03:00
Artur Wierzbicki 3cd43bd7ea Storage: externalize allow_unsanitized_svg_upload (#52703) 2022-07-25 21:11:17 +04:00
Artur Wierzbicki 33fc3a683b Datasources: add datasourceUids filter 2022-07-25 19:24:22 +04:00
Artur Wierzbicki d9db155394 Storage: Mime type detection (#52512) 
* Storage: implement mime type detection

* lint
 2022-07-25 11:30:20 +04:00
Artur Wierzbicki 17ea5f4f3e Search: Handle special datasource types (#52446) 2022-07-22 21:10:53 -04:00
Artur Wierzbicki bcfa4fabf6 Search: Allowed actions (#52529) 2022-07-22 17:42:43 -07:00
Artur Wierzbicki 0300b1c674 Storage: branding access (#52412) 
* Storage: add special users for system branding access

* Storage: explicit global org id

* Storage: initialize org storages with global org id

* Storage: add comments

* Storage: simplify - use orgId: 1 for systembranding
 2022-07-19 18:47:11 +04:00
Artur Wierzbicki 841e5ae8ad Storage: system/branding storage (#51987) 
* Storage: add `system` branding storage root, expose `system/branding` subfolder

* Storage: merge
 2022-07-18 19:37:22 +04:00
Artur Wierzbicki 9afe845d5c Sanitization: fix log message (#52401) 2022-07-18 06:46:57 -07:00
Artur Wierzbicki b2736ac1fe Storage: limit the number of uploaded files (#50796) 
* #50608: sql file upload quotas

* rename `files_in_sql` to `file`

* merge conflict
 2022-07-18 15:24:39 +04:00
Artur Wierzbicki 67ea2da57e Storage: fix svg upload (#52395) 2022-07-18 06:36:54 -04:00
Artur Wierzbicki 6188526e1d Storage: use static access rules (#52334) 
* Storage: use static access rules

* Storage: use static access rules

* Storage: add tests
 2022-07-17 22:41:54 +04:00
Artur Wierzbicki 1d2aa7c69b Storage: add delete / deleteFolder / createFolder (#51887) 
* delete / delete folder / create folder

* add backend tests

* implement force delete

* fix merge

* lint fix

* fix delete root folder

* fix folder name validation

* fix mysql path_hash issue

* Fix returning error
 2022-07-08 22:23:16 +04:00
Artur Wierzbicki c372b892ec fix nil pointer dereference (#51978) 2022-07-08 19:27:53 +04:00
Artur WierzbickiandRyan McKinley 63366615bb Search: support datasource template variables when parsing dashboard JSON models (#51587) 
* init

* support template variables

* support variables without curly braces

* add todo for `__all` case

* fix `$__all` case for non-multivalue

* extract some functions

* fix flakinesss

* support `$__all` and `default` template variables

* add todo

* compilation fix

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
 2022-07-08 01:59:24 +04:00
Artur WierzbickiandAgnès Toulet e96f67ae2e Renderer: Add sanitize API (#50936) 
* svg fun

* #50597: add proto

* #50597: add sanitizer methods

* #50597: add provider

* #50597: use sanitizer

* #50597: use sanitizer

* update grafana to match new api

* add comments

* add capability check

* add timing

* update sanitize path

* improve log message

* strings.HasPrefix rather than filepath.IsAbs

* filepath.Clean + filepath.ToSlash for windows

* read 404

* remove `path.clean` from `getPathAndScope`

* add resp body close

* remove unneeded prop

* Update pkg/services/rendering/rendering.go

Co-authored-by: Agnès Toulet <35176601+AgnesToulet@users.noreply.github.com>

* remove test files

* filepath.ToSlash correct wrapping

* filepath.ToSlash correct wrapping

* filepath.ToSlash comment

* compilation error

* lint fix

* fix error message

* Update pkg/services/rendering/rendering.go

Co-authored-by: Agnès Toulet <35176601+AgnesToulet@users.noreply.github.com>

* add `image/svg+xml` mime type

* refactored log

* refactored log

Co-authored-by: Agnès Toulet <35176601+AgnesToulet@users.noreply.github.com>
 2022-07-07 15:32:18 +04:00
Artur Wierzbicki 052132be46 Storage: simplify retrieving the file names from the response frame (#51805) 
* add util method for storage list frame

* use go:embed

* remove enum
 2022-07-07 10:47:05 +04:00
Artur Wierzbicki 0ca4ccfa96 Search: fix logging statements (#51633) 2022-07-01 15:38:31 -07:00
Artur Wierzbicki c496b3e712 Search: Fix pagination in the new search page (#51366) 
* Fix pagination in the new search

* update betterer betterer betterer betterer betterer betterer

* revert packagejson
 2022-06-24 21:21:59 +04:00
Artur Wierzbicki 40b152e813 Search: fix anonymous user access (#51045) 2022-06-17 20:36:10 +04:00
Artur Wierzbicki 14f8e18978 fix broken link (#51040) 2022-06-17 08:32:55 -07:00
Artur Wierzbicki 644503f5e6 Previews: use ETag header (#51008) 
* use etag header for previews

* use unix millis rather than tz-dependant string
 2022-06-17 18:02:03 +04:00
Artur Wierzbicki b2852205a0 Previews: update documentation for enterprise setup (#50950) 
* add `enable service accounts` step

* add `enable service accounts` step
 2022-06-16 16:38:18 +02:00
Artur WierzbickiTania BRyan McKinley
 		cc4473faf3 Storage: validation and sanitization stubs (#50523) 
* add `IsPathValidationError` util to fs api

* refactor storage.Upload method

* remove unused struct

* extract `RootUpload` constant

* move file validation outside of the service

* Make UploadErrorToStatusCode exported

* validation/sanitization

* refactor pathValidationError check

* refactor, rename sanitize to transform

* add a todo

* refactor

* transform -> sanitize

* lint fix

* #50608: fix jpg/jpeg

Co-authored-by: Tania B <yalyna.ts@gmail.com>
Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
 2022-06-15 12:32:29 +04:00
Artur WierzbickiandTania B 9779f684d1 Storage: refactor - decouple the Upload method from HTTP concepts (#50378) 
* add `IsPathValidationError` util to fs api

* refactor storage.Upload method

* remove unused struct

* extract `RootUpload` constant

* move file validation outside of the service

* Make UploadErrorToStatusCode exported

* refactor pathValidationError check

* if -> switch

Co-authored-by: Tania B <yalyna.ts@gmail.com>
 2022-06-13 21:21:50 +04:00
Artur Wierzbicki ef401f5d62 Storage: improve path validation, add tests (#50441) 
* improve path validation

* add test case
 2022-06-09 21:09:06 +04:00
Artur Wierzbicki 58a783c0e4 Previews: improve docs phrasing (#50405) 2022-06-08 13:50:59 +02:00
Artur WierzbickiandRyan McKinley 49112c8356 Search: fix 'Clear filters' button (#50227) 
Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
 2022-06-07 11:57:23 +04:00
Artur Wierzbicki fac8db8ff6 Previews: update docs for the new search layout (#50130) 
* #44449: update docs, remove the mention of `Show previews` toggle

* #44449: remove formatting of `grid`

* #44449: review

* review https://github.com/grafana/grafana/pull/50130#discussion_r888449621
 2022-06-06 11:09:16 +04:00
Artur Wierzbicki 03fe1435a0 Storage: store uploaded files in SQL rather than on the disk (#49034) 
* #48259: set up storages per org id

* #48259: migrate to storage_sql
 2022-05-21 16:55:11 -07:00
Artur WierzbickiandJitaC 156e14e296 Previews: beta docs (#47993) 
* #44449: preview docs draft

* #44449: remove localhost

* #44449: fixes

* #44449: remove localhost

* Update docs/sources/dashboards/previews.md

Co-authored-by: JitaC <70489351+achatterjee-grafana@users.noreply.github.com>

* Update docs/sources/dashboards/previews.md

Co-authored-by: JitaC <70489351+achatterjee-grafana@users.noreply.github.com>

* Update docs/sources/dashboards/previews.md

Co-authored-by: JitaC <70489351+achatterjee-grafana@users.noreply.github.com>

* Update docs/sources/dashboards/previews.md

Co-authored-by: JitaC <70489351+achatterjee-grafana@users.noreply.github.com>

* Update docs/sources/dashboards/previews.md

Co-authored-by: JitaC <70489351+achatterjee-grafana@users.noreply.github.com>

* Update docs/sources/dashboards/previews.md

Co-authored-by: JitaC <70489351+achatterjee-grafana@users.noreply.github.com>

* Update docs/sources/dashboards/previews.md

Co-authored-by: JitaC <70489351+achatterjee-grafana@users.noreply.github.com>

* Update docs/sources/dashboards/previews.md

Co-authored-by: JitaC <70489351+achatterjee-grafana@users.noreply.github.com>

* Update docs/sources/dashboards/previews.md

Co-authored-by: JitaC <70489351+achatterjee-grafana@users.noreply.github.com>

* Update docs/sources/dashboards/previews.md

Co-authored-by: JitaC <70489351+achatterjee-grafana@users.noreply.github.com>

* Update docs/sources/dashboards/previews.md

Co-authored-by: JitaC <70489351+achatterjee-grafana@users.noreply.github.com>

* #44449: update link

* #44449: review comment https://github.com/grafana/grafana/pull/47993#discussion_r866363311

* #44449: https://github.com/grafana/grafana/pull/47993#discussion_r866363825

* #44449: https://github.com/grafana/grafana/pull/47993#discussion_r866364182

* #44449: https://github.com/grafana/grafana/pull/47993#discussion_r866365003

* #44449: https://github.com/grafana/grafana/pull/47993#discussion_r866359611

* #44449: https://github.com/grafana/grafana/pull/47993#discussion_r866361949

* #44449: add note about the first crawler run

* #44449: https://github.com/grafana/grafana/pull/47993/files#r866355629

* Update docs/sources/dashboards/previews.md

Co-authored-by: JitaC <70489351+achatterjee-grafana@users.noreply.github.com>

* fix up

* Update docs/sources/dashboards/previews.md

Co-authored-by: JitaC <70489351+achatterjee-grafana@users.noreply.github.com>

* typo

* Update docs/sources/dashboards/previews.md

Co-authored-by: JitaC <70489351+achatterjee-grafana@users.noreply.github.com>

* Update docs/sources/dashboards/previews.md

Co-authored-by: JitaC <70489351+achatterjee-grafana@users.noreply.github.com>

* rephrase

* replace gifs with webms

* reword

Co-authored-by: JitaC <70489351+achatterjee-grafana@users.noreply.github.com>
 2022-05-17 13:02:50 -07:00
Artur Wierzbicki 7a05941ea0 Previews: improve logging, add time measurements to the crawler (#49043) 
* #44449: log times taken for various things in previews service / crawler

* #44449: remove stale log
 2022-05-16 18:09:46 -07:00
Artur Wierzbicki 5c321599c8 Storage: enable SQL backend (#48095) 
* #45498: add String util to ListResponse for better UX

* #45498: refactor db_filestorage FS API backend - use path_hash in DB schema

* #45498: enable DB backend fs api tests

* #45498: add comment

* #45498: enable Storage feature flag during integration tests

* remove fmt.println

* #45498: reduce sizes of hash columns

* separate conditions

* #45498: make it easy to ignore backends when running fs api integration tests

* #45498: quote `key` column name

* #45498: reduce path_hash size

* #45498: verify `/{orgId}/{storageName}/` prefix convention in integration tests

* #45498: add etag to the sql table

* #45498: add etag to the sql table

* remove feature flag check (storage isn't dev-mode only)

* add cacheControl and content disposition

* add comments

* add path_hash comment

* explicitly set `path` column collation in `file` table for postgres
 2022-05-16 10:26:40 -07:00
Artur Wierzbicki de75dd6aad Search: remove dev-mode from the feature flag 2022-05-02 23:01:29 +04:00
Artur Wierzbicki d4616cfe26 Search: support auth filter based on access control/rbac (#48350) 
* #45498: add rbac support in searchv2

* to revert: fix dependency cycle

* Revert "to revert: fix dependency cycle"

This reverts commit 1ffbee73ec.

* added a TODO for caching user permissions

* add orgId to `getDashboardReadFilter`

* use orgId from signedInUser

* goimport
 2022-04-28 22:16:23 +04:00
Artur Wierzbicki 484bd0b557 Storage: use FlagStorageLocalUpload for HTTP API 2022-04-28 18:33:02 +04:00
Artur Wierzbicki 736be74128 Chore: Add Usage stats providers registry (#48357) 
* add usage stats providers

* migrate thumbnails service to `registry.ProvidesUsageStats`

* snake -> camel

* lint fix

* migrate accesscontrol to `registry.ProvidesUsageStats`

* add accesscontrol to usage stats providers registry

* fix test

* added a note about errors
 2022-04-28 13:06:49 +04:00
Artur WierzbickiRyan McKinleyAlexander Emelin
 		25e153e4e7 Search: in-memory index (#47709) 
* #45498: add entity events table

* #45498: add entity events service

* #45498: hook up entity events service to http server

* #45498: use `dashboards.id` rather than `uid` and `org_id` in grn

* Update pkg/services/entityevents/service.go

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>

* #45498: move entityeventsservice to services/store

* #45498: add null check

* #45498: rename

* #45498: fix comment

* #45498: switch grn back to uid

* Search: listen for updates (#47719)

* #45498: wire entity event service with searchv2

* load last event id before building index for org 1

* fix service init in integration tests

* depend on required subset of event store methods

* Update pkg/services/sqlstore/migrations/entity_events_mig.go

Co-authored-by: Alexander Emelin <frvzmb@gmail.com>

* #45498: pointer receiver

* #45498: mockery!

* #45498: add entity events service to background services

* dashboard query pagination, allow queries while re-indexing

* log level cleanups, use rlock, add comments

* fix lint, check feature toggle in search v2 service

* use unix time for event created column

* add missing changes for created column

* fix integration tests init

* log re-index execution times on info level

* #45498: fix entityEventsService tests

* #45498: save events on dashboard delete

* use camel case for log labels

* formatting

* #45498: rename grn to entityid

* #45498: add `IsDisabled` to entityEventsService

* #45498: remove feature flag from migration

* better context usage, fix capacity, comments/cleanups

* replace print with logger

* Revert "#45498: remove feature flag from migration"

This reverts commit ed23968898.

* revert:revert:revert conditional feature flag

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
Co-authored-by: Alexander Emelin <frvzmb@gmail.com>
 2022-04-27 12:29:39 +04:00
Artur Wierzbicki 2e599643f6 Previews: refactor (#47728) 
* #44449: return standard thumb service even if auth setup fails

* #44449: remove dashboardPreviewsScheduler feature flag

* #44449: externalize dashboardPreviews config

* #44449: disable previews by default

* #44449: rename logger

* #44449: dashboardPreviewsAdmin feature requires dev mode

* #44449: retrigger CII
 2022-04-25 01:55:10 +04:00
Artur Wierzbicki 87dc1bfc88 Chore: enforce feature naming convention (#48053) 
* enforce feature naming convention

* fix link

* add comment

* fix lint

* move the feature name check test to toggles_gen_test
 2022-04-22 08:23:30 +04:00
Artur Wierzbicki 9f0b6a5754 Storage: refactor filtering, improve performance (#47403) 
* Storage: refactor filtering, improve performance

* added a comment to `newpathfilter`

* after merge fixes
 2022-04-21 23:27:43 +04:00
Artur Wierzbicki 46dc1e98fa Previews: track oss adoption 2022-04-13 11:54:49 +02:00
Artur Wierzbicki a4381ebc91 Previews: create crawler auth setup service (#47349) 
* #46968: add `RetrieveServiceAccountIdByName` to serviceaccounts service

* #46968: improve error logging in rendering service

* #46968: add oss crawler account setup

* #46968: fix tests

* #46968: switch back to ROLE_ADMIN

* #46968: rename to crawlerAuth

* comment crawler_auth.go
 2022-04-12 19:34:04 +02:00
Artur Wierzbicki 712b239d5a Storage: remove pointer on write path (#47580) 
* remove pointer to contents in UpsertFileCommand

* add command to UpsertFileCommand

* fix client after API refactor
 2022-04-12 16:58:09 +02:00
Artur Wierzbicki f1a1070d41 UsageStats: track enabled features (#47407) 
* #47127: include enabled features in the usage stats reports

* #47127: convert feature names to snake cased metric names

* #47127: remove dead code

* #47127: lint fix

* #47127: convert GetUsageStats to return `map[string]interface{}`

* #47127: fix testssssssss

* #47127: fix testssssssss
 2022-04-08 12:42:33 +02:00
Artur Wierzbicki 60984881ec Previews: fix crawler in multi-org deployments (#46967) 
* Previews: fix crawler in multi-org deployments

* Previews: fix statement ordering
 2022-03-29 13:40:11 +01:00
Artur Wierzbicki 5881127319 Storage: fix filestorage.Wrapper PathFiltering and non-recursive folder listing (#46469) 
* fix pathFilters in wrapper - apply rootPath changes before filtering

* fix non-recursive folder listing
 2022-03-11 23:16:46 +04:00
Artur Wierzbicki ed924b3d0c Storage: list filtering and root folder support (#46453) 
* git the things: FS api internal changes

* remove filestorage/service.go

* remove filestore flag

* remove dummy fs

* readd fileblob import
 2022-03-11 22:08:19 +04:00
Artur Wierzbicki bdbb9ed54a Live: Fix support for StreamingFrameAction.Replace (#46086) 
* fix StreamingFrameAction.replace

* simplify

* beautify

* cover `streamingDataFrame.ts` changes with unit tests

* formatting

* cover `LiveDataStream.ts` changes with unit tests

* update frame length after pushing new values
 2022-03-04 14:04:30 +04:00
Artur WierzbickiandRyan McKinley a8b90d9a25 FileStore: add basic file storage API (#46051) 
* #45498: fs API alpha

* #45498: remove grafanaDS changes for filestorage.go

* #45498: fix lint

* #45498: fix lint

* #45498: remove db file storage migration

* #45498: linting

* #45498: linting

* #45498: linting

* #45498: fix imports

* #45498: add comment

* remove StorageName abstractions

* FileStore: add dummy implementation (#46071)

* #45498: bring back grafanaDs changes, add dummy filestorage

* #45498: rename grafanaDs to public

* #45498: modify join

* #45498: review fix

* #45498:  unnecessary leading newline (whitespace) IMPORTANT FIX

* #45498: fix belongsToStorage

* #45498: fix removeStoragePrefix so that it works with abs paths

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
 2022-03-03 10:53:26 +04:00
Artur Wierzbicki c04b0179ff Previews: prevent scheduler from running when image renderer is outdated (#45623) 
* #44449: prevent crawler from running when rendering service is outdated

* #44449: simplify
 2022-02-18 21:29:18 +04:00
Artur Wierzbicki 7643ae6c5e Rendering: periodically refresh remote plugin version (#45505) 
* #44449: refresh image renderer version on a schedule

* #44449: simplify
 2022-02-18 20:25:01 +04:00
Artur Wierzbicki 3e53a08090 #44449: hide dashboard preview admin UI (#45493) 2022-02-17 12:34:07 +04:00
Artur WierzbickiRyan McKinleyAlexander Emelin
 		6c76aa71e8 Previews: capability check (#44601) 
* add SQL migrations

* dashboard previews from sql: poc

* added todos

* refactor: use the same enums where possible

* use useEffect, always return json

* added todo

* refactor + delete files after use

* refactor + fix manual thumbnail upload

* refactor: move all interactions with sqlStore to thumbnail repo

* refactor: remove file operations in thumb crawler/service

* refactor: fix dashboard_thumbs sql store

* refactor: extracted thumbnail fetching/updating to a hook

* refactor: store thumbnails in redux store

* refactor: store thumbnails in redux store

* refactor: private'd repo methods

* removed redux storage, saving images as blobs

* allow for configurable rendering timeouts

* added 1) query for dashboards with stale thumbnails, 2) command for marking thumbnails as stale

* use sql-based queue in crawler

* ui for marking thumbnails as stale

* replaced `stale` boolean prop with `state` enum

* introduce rendering session

* compilation errors

* fix crawler stop button

* rename thumbnail state frozen to locked

* #44449: fix merge conflicts

* #44449: remove thumb methods from `Store` interface

* #44449: clean filepath, defer file closing

* #44449: fix rendering.Theme cyclic import

* #44449: linting

* #44449: linting

* #44449: mutex'd crawlerStatus access

* #44449: added integration tests for `sqlstore.dashboard_thumbs`

* #44449: added comments to explain the `ThumbnailState` enum

* #44449: use os.ReadFile rather then os.Open

* #44449: always enable dashboardPreviews feature during integration tests

* #44449: add /previews/system-requirements API

* #44449: remove sleep time, adjust number of threads

* #44449: review fix: add `orgId` to `DashboardThumbnailMeta`

* #44449: review fix: automatic parsing of thumbnailState

* #44449: update returned json

* #44449: UI changes - dashboard previews sytem req check

* #44449: lint fixes

* #44449: fix tests

* #44449: typo

* #44449: fix getSystemRequirements API: return 200 even if we plugin version is invalid

* #44449: fix getSystemRequirements API: don't return SemverConstraint on error

* #44449: fix getSystemRequirements API

* #44449: fix previews sytem requirements text

* #44449: add `doThumbnailsExist` to repo

* #44449: remove redux api

* #44449: add missing model

* #44449: implement frontedsettings-driven capability check

* #44449: simplify

* #44449: revert test changes

* #44449: add dummy setup settings

* #44449: implicit typing over `FC<Props>`

* #44449: refactor conditionals

* #44449: replace `getText` with a react component

* #44449: fix component interface

* #44449: add onRemove to `PreviewsSystemRequirements` alert

* #44449: add bottom/top margin to previewSystemRequirements modal

* #44449: merge conflict fix

* #44449: remove console.log

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
Co-authored-by: Alexander Emelin <frvzmb@gmail.com>
 2022-02-16 21:49:50 +04:00
Artur WierzbickiRyan McKinleyAlexander Emelin
 		0276b029fc Previews: crawler as a background service (#44891) 
* add SQL migrations

* dashboard previews from sql: poc

* added todos

* refactor: use the same enums where possible

* use useEffect, always return json

* added todo

* refactor + delete files after use

* refactor + fix manual thumbnail upload

* refactor: move all interactions with sqlStore to thumbnail repo

* refactor: remove file operations in thumb crawler/service

* refactor: fix dashboard_thumbs sql store

* refactor: extracted thumbnail fetching/updating to a hook

* refactor: store thumbnails in redux store

* refactor: store thumbnails in redux store

* refactor: private'd repo methods

* removed redux storage, saving images as blobs

* allow for configurable rendering timeouts

* added 1) query for dashboards with stale thumbnails, 2) command for marking thumbnails as stale

* use sql-based queue in crawler

* ui for marking thumbnails as stale

* replaced `stale` boolean prop with `state` enum

* introduce rendering session

* compilation errors

* fix crawler stop button

* rename thumbnail state frozen to locked

* #44449: fix merge conflicts

* #44449: remove thumb methods from `Store` interface

* #44449: clean filepath, defer file closing

* #44449: fix rendering.Theme cyclic import

* #44449: linting

* #44449: linting

* #44449: mutex'd crawlerStatus access

* #44449: added integration tests for `sqlstore.dashboard_thumbs`

* #44449: added comments to explain the `ThumbnailState` enum

* #44449: use os.ReadFile rather then os.Open

* #44449: always enable dashboardPreviews feature during integration tests

* #44449: remove sleep time, adjust number of threads

* #44449: review fix: add `orgId` to `DashboardThumbnailMeta`

* #44449: review fix: automatic parsing of thumbnailState

* #44449: lint fixes

* #44449: crawler as a background service v0.1

* #44449: use ServerLockService

* #44449: use ServerLockService

* #44449: review fix: prefer `WithDbSession` over `WithTransactionalDbSession`

* #44449: review fix: add a comment explaining source of the filepath

* #44449: review fix: added filepath validation

* #44449: fix FindDashboardsWithStaleThumbnails to include `theme` and `kind` in search params

* #44449: fix FindDashboardsWithStaleThumbnails to include `theme` and `kind` in search params

* #44449: create function for crawler on demand

* #44449: improve crawler logging

* #44449: fix wire

* #44449: uncomment dummy thumb service, fix ticker interval

* #44449: prevent race condition

* #44449: improve logging

* #44449: fix theme

* #44449: review fixes https://github.com/grafana/grafana/pull/45063/files @fzambia

* #44449: add missing unlock

* #44449: merge

* #44449: review fix - logger @fzambia https://github.com/grafana/grafana/pull/45063/files

* #44449: formatting

* #44449: merge conflict fix

* #44449: merge conflict fix

* #44449: merge conflict fix

* #44449: naming fix

* #44449: update authOpts

* #44449: change authOpts.role back to admin

* #44449: fix `walk` signature, move ctx to a first argument

* #44449: add `dashboardPreviewsScheduler` feature flag

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
Co-authored-by: Alexander Emelin <frvzmb@gmail.com>
 2022-02-10 22:45:00 +04:00
Artur Wierzbicki d60c2dff40 Chore: export closestIdx from StreamingDataFrame (#45162) 
* #41993: export closestIdx function

* #41993: formatting

* #41993: formatting v2
 2022-02-09 22:12:24 +04:00
ArturWierzbicki 45a8701cb5 Chore: Updated the link to the plugin developer guide docs - it used to point to an old version (#40710) 2021-10-25 12:11:44 +04:00