Commit Graph

223 Commits

Author SHA1 Message Date
Dan Cech
c4c9bfaf2e Storage: Unified Storage based on Entity API (#71977)
* first round of entityapi updates

- quote column names and clean up insert/update queries
- replace grn with guid
- streamline table structure

fixes

streamline entity history

move EntitySummary into proto

remove EntitySummary

add guid to json

fix tests

change DB_Uuid to DB_NVarchar

fix folder test

convert interface to any

more cleanup

start entity store under grafana-apiserver dskit target

CRUD working, kind of

rough cut of wiring entity api to kube-apiserver

fake grafana user in context

add key to entity

list working

revert unnecessary changes

move entity storage files to their own package, clean up

use accessor to read/write grafana annotations

implement separate Create and Update functions

* go mod tidy

* switch from Kind to resource

* basic grpc storage server

* basic support for grpc entity store

* don't connect to database unless it's needed, pass user identity over grpc

* support getting user from k8s context, fix some mysql issues

* assign owner to snowflake dependency

* switch from ulid to uuid for guids

* cleanup, rename Search to List

* remove entityListResult

* EntityAPI: remove extra user abstraction (#79033)

* remove extra user abstraction

* add test stub (but

* move grpc context setup into client wrapper, fix lint issue

* remove unused constants

* remove custom json stuff

* basic list filtering, add todo

* change target to storage-server, allow entityStore flag in prod mode

* fix issue with Update

* EntityAPI: make test work, need to resolve expected differences (#79123)

* make test work, need to resolve expected differences

* remove the fields not supported by legacy

* sanitize out the bits legacy does not support

* sanitize out the bits legacy does not support

---------

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>

* update feature toggle generated files

* remove unused http headers

* update feature flag strategy

* devmode

* update readme

* spelling

* readme

---------

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
2023-12-06 15:21:21 -05:00
Dan Cech
318f51eaee Chore: Remove unused CacheService dependency from sqlstore (#78507)
remove unused CacheService dependency from sqlstore
2023-12-04 10:00:45 -05:00
Ryan McKinley
a221c1d754 Chore: Remvoe newDBLibrary feature flag (#78074) 2023-11-14 06:51:35 -08:00
Ryan McKinley
dec9a07738 Settings: Actually deprecate access to feature flags (#78073) 2023-11-13 11:39:01 -08:00
George Robinson
15f6e8a500 SQLStore: Fix race condition in RecursiveQueriesAreSupported (#75274) 2023-09-22 11:30:14 +01:00
Serge Zaitsev
7197ed9fbb Fix: avoid panics on duplicate metrics registration (#74359) 2023-09-05 10:02:11 +02:00
Ryan McKinley
025b2f3011 Chore: use any rather than interface{} (#74066) 2023-08-30 18:46:47 +03:00
Sofia Papagiannaki
e21872bb15 SQLStore: Fix regression in PostgreSQL connection string (#71368)
* SQLStore: Fix connection string for PostgreSQL

* better testing
2023-07-11 16:47:35 +03:00
Guillaume Jacquet
b947222bf5 Platform: Add support for Postgresql pgpass file (#61517)
lib/pq has built-in support to use pgpass file for authentication when
no password has been provided. However this requires that the connection
does not contain the password parameter at all.

Removing password parameter when postgresql password is empty in
SQL store.
2023-07-07 11:17:35 +02:00
Dan Cech
a6279b2d62 Database: Make dialects independent of xorm Engine (#69955)
* make dialects independent of xorm Engine

* goimports
2023-06-14 16:13:36 -04:00
owensmallwood
b43206e26b MySQL: Use transaction_isolation instead of tx_isolation (#68575)
will use the correct system var name for transaction isolation
2023-05-23 09:22:05 -06:00
Ryan McKinley
26658d172f Migrations: Support independent migration log for entity api (#68511) 2023-05-22 11:31:07 -07:00
Carl Bergquist
692bb9ed1a Instrumentation: Add support for instrumenting database queries (#66022)
Signed-off-by: bergquist <carl.bergquist@gmail.com>
Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com>
2023-04-28 15:19:06 +02:00
Artur Wierzbicki
2136e680c4 Previews: remove dashboard previews backend (#66176)
* remove dashboard previews backend

* remove dashboard previews backend

* bring back the migration

* bring back the migration

* bring back the migration
2023-04-13 21:42:24 +04:00
Sofia Papagiannaki
4ee0be6fdf Chore: Add function for detecting if the SQL driver supported CTEs (#64441)
* Add interface method for detecting if the SQL driver supported CTEs

* Update nested folder store to call RecursiveQueriesAreSupported()
2023-03-20 10:27:08 +02:00
Sofia Papagiannaki
f5cb8c660e SQLStore: Fix setting query tries for integration tests (#64944)
* SQLStore: Pass testinfra database configuration to the test database

* Add test

* Bypass gocyclo check for initTestDB
2023-03-17 17:31:03 +02:00
Ryan McKinley
d5a9a0cea0 Chore: remove comments feature (#64644) 2023-03-11 04:28:12 -08:00
Alexander Weaver
8ea71d37c2 SQLStore: Enable clientFoundRows for MySQL connections (#64070)
Enable clientFoundRows for MySQL connections
2023-03-02 16:31:01 -06:00
Serge Zaitsev
efed0151f8 Chore: Remove pkg/models/stats.go (#61613)
* remove pkg/models/stats.go

* rename models package in tests
2023-01-17 14:17:54 +01:00
Kristin Laemmert
cc007e9727 chore: sqlstore cleanup (#60415)
* chore: remove unused test helper from sqlstore

TimeNow() is no longer used in any tests in this package.

* chore: move sqlstore.SQLBuilder to the infra/db package

This required some minor refactoring; we need to be a little more explicit about passing around the dialect and engine. On the other hand, that's a few fewer uses of the `dialect` global constant!

* chore: move UserDeletions into the only package using it

* cleanup around moving sqlbuilder

* remove dialect and sqlog global vars

* rename userDeletions to serviceAccountDeletions
2022-12-16 11:09:06 -05:00
Kristin Laemmert
e4a877b123 chore: attempt to fix flapping "organization not found" tests (#60418)
we live in hope!
2022-12-16 10:10:59 -05:00
Kristin Laemmert
70fbf47022 chore: remove CreateUser from sqlstore & replace with userService.CreateUserForTests (#59910) 2022-12-07 11:03:22 -05:00
Serge Zaitsev
b3284a8330 Chore: Make getUserOrgList private to sqlstore (#59654)
* make getUserOrgList private in sqlstore

* make other identifiers private
2022-12-01 15:46:42 +01:00
Ryan McKinley
14a080ec12 EntityStore: Rename ObjectStore to EntityStore (part 2) (#59616) 2022-11-30 17:52:15 -05:00
Marcus Efraimsson
79f1a7a4fd Database: Adds support for enable/disable SQLite Write-Ahead Logging (WAL) via configuration (#58268)
Adds support for enable/disable SQLite Write-Ahead Logging (WAL) via configuration.
Enables SQLite WAL for E2E tests.
2022-11-16 19:29:33 +01:00
Ryan McKinley
eb1cc80941 Storage: include SQL implementation (#58018) 2022-11-04 17:30:22 -04:00
Ryan McKinley
a3acfb1a48 Chore: Register a feature flag for "newDBLibrary" (#57468) 2022-10-25 18:20:41 -07:00
Kristin Laemmert
05709ce411 chore: remove sqlstore & mockstore dependencies from (most) packages (#57087)
* chore: add alias for InitTestDB and Session

Adds an alias for the sqlstore InitTestDB and Session, and updates tests using these to reduce dependencies on the sqlstore.Store.

* next pass of removing sqlstore imports
* last little bit
* remove mockstore where possible
2022-10-19 09:02:15 -04:00
Sofia Papagiannaki
46fb4081ba SQLStore: Optionally retry queries if sqlite returns database is locked (#56096)
* SQLStore: Retry queries if sqlite returns database is locked

* Configurable retries

* Apply suggestions from code review

Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com>
2022-10-17 21:23:44 +03:00
ying-jeanne
80ede174dd update the feature toggle usage in backend (#56787) 2022-10-12 13:43:35 +02:00
idafurjes
a45ef61d25 Chore: Remove GetUserByEmail and GetUserByLogin from sqlstore (#55903)
* Chore: Remove GetUserByEmail and GetUserByLogin from sqlstore
 Rename GetUserProfile to GetProfile

* Fix lint

* Skip test for mysql

* Add missing method to sqlstore mock
2022-09-28 13:18:19 +02:00
Sofia Papagiannaki
2f14575dd3 Chore: Move annotations cleanup to the annotations service (#55618) 2022-09-22 15:27:48 +03:00
Sofia Papagiannaki
754eea20b3 Chore: SQL store split for annotations (#55089)
* Chore: SQL store split for annotations

* Apply suggestion from code review
2022-09-19 10:54:37 +03:00
mhuangwm
39102c6656 Admin: Add support to configure default admin email (#54363) 2022-09-07 14:38:40 +02:00
Dave Henderson
713075c494 Metrics: Fixed grafana_database_conn_* metrics, and added new go_sql_stats_* metrics as eventual replacement (#54405)
* metrics: Fix broken DBStats metrics

Signed-off-by: Dave Henderson <dave.henderson@grafana.com>

* Register the sqlstats metrics by default

Signed-off-by: Dave Henderson <dave.henderson@grafana.com>

Signed-off-by: Dave Henderson <dave.henderson@grafana.com>
2022-08-31 08:54:32 -04:00
ying-jeanne
82b63688d2 Chore: remove xorm from preference (#53803)
* Chore: remove xorm from preference

* separte feature toggle

* fix comments

* fix comments

* remove the dublicated namedexec
2022-08-17 22:07:20 -04:00
ying-jeanne
25de383540 Chore: Replace xorm with sqlx (#52575)
* Change of sqlstore to use sqlx

* Use sqlx in the playlist store

* Refectory of the interface

* update playlist service

* go mod tidy

* some refectory on interface

* fix kyle
2022-08-16 13:17:14 -05:00
Karl Persson
c08fe3a53c RBAC: Enable rbac when creating new settings (#53531)
* Settings: Set RBACEnabled to true by default

* Remove accessControlEnabledFlag and explicitly set to false when needed

* Disable rbac for tests
2022-08-11 15:37:31 +02:00
Serge Zaitsev
72367f99cf Chore: Move plugin settings store code into a dedicated service (#53560) 2022-08-11 09:50:53 +02:00
ying-jeanne
ba89471598 Chore: Enable ANSI_QUOTES for Mysql Database (#53277)
* remove the quote dialect

* add environment variable
2022-08-08 12:14:17 -04:00
Carl Bergquist
7825ad6aa3 Instrumentation: Collect database connection stats (#52797)
Signed-off-by: bergquist <carl.bergquist@gmail.com>
2022-07-27 13:42:28 +02:00
idafurjes
6c43eb0b4d Split Create User (#50502)
* Split Create User

* Use new create user and User from package user

* Add service to wire

* Making create user work

* Replace user from user pkg

* One more

* Move Insert to orguser Service/Store

* Remove unnecessary conversion

* Cleaunp

* Fix Get User and add fakes

* Fixing get org id for user logic, adding fakes and other adjustments

* Add some tests for ourguser service and store

* Fix insert org logic

* Add comment about deprecation

* Fix after merge with main

* Move orguser service/store to org service/store

* Remove orguser from wire

* Unimplement new Create user and use User from pkg user

* Fix wire generation

* Fix lint

* Fix lint - use only User and CrateUserCommand from user pkg

* Remove User and CreateUserCommand from models

* Fix lint 2
2022-06-28 14:32:25 +02:00
Serge Zaitsev
ae9491c3a7 Chore: Make test tracer noop and return no errors (#50797) 2022-06-15 12:40:41 +02:00
Serge Zaitsev
0b55c41d05 Chore: Remove global bus variable (#50765)
* Chore: Remove global bus variable

* fix bus in tests
2022-06-14 16:07:41 +02:00
Dan Cech
9350ab781c clean up duplicated user creation code (#50178)
* clean up duplicated user creation code

* remove unused duplicate getOrCreateOrg function

* fix up tests
2022-06-07 09:49:18 -04:00
Kat Yang
31630edf0c Chore: Remove Wrapf (#50128)
* Chore: Remove Wrapf

* Remove all Wrapf refs

* Remove last Wrapf ref

* Fix lint errors

* Remove Wrap and Wrapf definitions

* Remove unnecessary colon
2022-06-06 22:30:31 +02:00
Kat Yang
3c3039f5b3 Chore: Remove Wrap (#50048)
* Chore: Remove Wrap and Wrapf

* Fix: Add error check
2022-06-03 09:24:24 +02:00
idafurjes
bdf50f3dd2 Dashboards: Split GetDashboardVersions method (#49967)
* Split GetDashboarVersions method

* Add sqlstore dialect and tests

* Fix signature of PAtchPreference

* Add GetDialect to sqlstore and remove GetDashboardVersions

* Add GetDialect to db interface

* Implement List

* add deleted test function

* Remove GetDialect from sqlstore interface

* Remove deleted method from mock

* Refactor test
2022-06-02 15:59:05 +02:00
Ieva
f256f625d8 AccessControl: Enable RBAC by default (#48813)
* Add RBAC section to settings

* Default to RBAC enabled settings to true

* Update tests to respect RBAC

Co-authored-by: Karl Persson <kalle.persson@grafana.com>
2022-05-16 12:45:41 +02:00
Artur Wierzbicki
25e153e4e7 Search: in-memory index (#47709)
* #45498: add entity events table

* #45498: add entity events service

* #45498: hook up entity events service to http server

* #45498: use `dashboards.id` rather than `uid` and `org_id` in grn

* Update pkg/services/entityevents/service.go

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>

* #45498: move entityeventsservice to services/store

* #45498: add null check

* #45498: rename

* #45498: fix comment

* #45498: switch grn back to uid

* Search: listen for updates (#47719)

* #45498: wire entity event service with searchv2

* load last event id before building index for org 1

* fix service init in integration tests

* depend on required subset of event store methods

* Update pkg/services/sqlstore/migrations/entity_events_mig.go

Co-authored-by: Alexander Emelin <frvzmb@gmail.com>

* #45498: pointer receiver

* #45498: mockery!

* #45498: add entity events service to background services

* dashboard query pagination, allow queries while re-indexing

* log level cleanups, use rlock, add comments

* fix lint, check feature toggle in search v2 service

* use unix time for event created column

* add missing changes for created column

* fix integration tests init

* log re-index execution times on info level

* #45498: fix entityEventsService tests

* #45498: save events on dashboard delete

* use camel case for log labels

* formatting

* #45498: rename grn to entityid

* #45498: add `IsDisabled` to entityEventsService

* #45498: remove feature flag from migration

* better context usage, fix capacity, comments/cleanups

* replace print with logger

* Revert "#45498: remove feature flag from migration"

This reverts commit ed23968898.

* revert:revert:revert conditional feature flag

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
Co-authored-by: Alexander Emelin <frvzmb@gmail.com>
2022-04-27 12:29:39 +04:00