This adds a version of the SQLStore that includes a ReadReplica. The primary DB can be accessed directly - from the caller's standpoint, there is no difference between the SQLStore and ReplStore unless they wish to explicitly call the ReadReplica() and use that for the DB sessions.
Currently only the stats service GetSystemStats and GetAdminStats are using the ReadReplica(); if it's misconfigured or if the databaseReadReplica feature flag is not turned on, it will fall back to the usual (SQLStore) behavior.
Testing requires a database and read replica - the replication should already be configured. I have been testing this locally with a docker mysql setup (https://medium.com/@vbabak/docker-mysql-master-slave-replication-setup-2ff553fceef2) and the following config:
[feature_toggles]
databaseReadReplica = true
[database]
type = mysql
name = grafana
user = grafana
password = password
host = 127.0.0.1:3306
[database_replica]
type = mysql
name = grafana
user = grafana
password = password
host = 127.0.0.1:3307
* Zanana: Initial work to run zanana as ebeddedn or standalone
* Add addr settings for when remote client is used.
* sync dependencies
* Lock mysql driver version
---------
Co-authored-by: Dan Cech <dcech@grafana.com>
* remove unused action set code, refactor the existing code
* fix import ordering
* use a separate interface for permission expansion after all, to avoid circular dependencies
* add comments, fix a test
* Social: link to OrgRoleMapper
* OIDC: support Generic Oauth org to role mappings
Fixes: #73448
Signed-off-by: Mathieu Parent <math.parent@gmail.com>
* Handle when getAllOrgs fails in the org_role_mapper
* Add more tests
* OIDC: ensure orgs are evaluated from API when not from token
Signed-off-by: Mathieu Parent <math.parent@gmail.com>
* OIDC: ensure AutoAssignOrg is applied with OrgMapping without RoleAttributeStrict
Signed-off-by: Mathieu Parent <math.parent@gmail.com>
* Extend docs
* Fix test, lint
---------
Signed-off-by: Mathieu Parent <math.parent@gmail.com>
Co-authored-by: Mihaly Gyongyosi <mgyongyosi@users.noreply.github.com>
* logic to expand action set to the underlying actions when permissions are fetched from the DB
* updates needed for dependency injection
* clean up some code, also deduplicate scopes when grouping scopes and actions
* expand on a comment
* rename a method
* moving folders to slog
* trying to fix the tests
* fix per-logger filters in slog
* allow using slog.Default()
* bring cfg back to keep the pr small
* fix tests
* back to the roots
* make cfg private in sqlstore
* fix db init in tests
* fix case
* fix folder test init
* fix imports
* make another Cfg private
* remove another Cfg
* remove unused variable
* use store cfg, it has side-effects
* fix mutated cfg in tests
* Storage server runs own instrumentation server if its the sole target. Starts adding some sample metrics for now.
* adds metric for failed optimistic locks
* refactors metrics registration to own method on service for testability. Adds tests.
* Register sql storage server metrics from within the service
* fixes test
* troubleshooting drone test failures. Maybe timing when starting instrumentation server?
* Waits until instrumentation server has started. Updates tests.
* defer wont get called unless theres an error. removing.
* wait for instrumentation server to be running
* linter - close res body
* use port 3000 for metrics and removes test metric inc() call
* fixes test - updates port
* refactors module server to provide an instrumentation server module when there is no ALL or CORE target provided and running as single target
* make instrumentation server a dependency of all modules that do not run their own http server
* adds module server test
* adds tests for instrumentation service and removes old tests that aren't needed
* ignore error in test
* uses helper to start and run service
* when running wait on ctx done or http server err
* wait for http server
* removes println
* updates module server test to be integration test
* require no error in goroutine
* skips integration test when GRAFANA_TEST_DB not defined
* move http server start into start, verify returned content
* make test error when run fails
* try waiting longer and see if drone tests pass
* update integration test mysql creds to match drone
* go back to only waiting half second
* debug log drone mysql connection string
* use same db connection config as drone
* try using same hostname as drone
* cant use localhost as mysql hostname in drone tests. Need to parse it from the cfg db connection string
---------
Co-authored-by: Dan Cech <dcech@grafana.com>
* replace sqlstore with db interface in a few packages
* remove from stats
* remove sqlstore in admin test
* remove sqlstore from api plugin tests
* fix another createUser
* remove sqlstore in publicdashboards
* remove sqlstore from orgs
* clean up orguser test
* more clean up in sso
* clean up service accounts
* further cleanup
* more cleanup in accesscontrol
* last cleanup in accesscontrol
* clean up teams
* more removals
* split cfg from db in testenv
* few remaining fixes
* fix test with bus
* pass cfg for testing inside db as an option
* set query retries when no opts provided
* revert golden test data
* rebase and rollback
Adds support for logs (specify level), metrics (enable metrics and Prometheus /metrics endpoint
and traces (jaeger or otlp) for standalone API server. This will allow any grafana core service
part of standalone apiserver to use logging, metrics and traces as normal.
* Move some thema code inside grafana
* Use new codegen instead of thema for core kinds
* Replace TS generator
* Use new generator for go types
* Remove thema from oapi generator
* Remove thema from generators
* Don't use kindsys/thema for core kinds
* Remove kindsys/thema from plugins
* Remove last thema related
* Remove most of cuectx and move utils_ts into codegen. It also deletes wire dependency
* Merge plugins generators
* Delete thema dependency 🎉
* Fix CODEOWNERS
* Fix package name
* Fix TS output names
* More path fixes
* Fix mod codeowners
* Use original plugin's name
* Remove kindsys dependency 🎉
* Modify oapi schema and create an apply function to fix elasticsearch errors
* cue.mod was deleted by mistake
* Fix TS panels
* sort imports
* Fixing elasticsearch output
* Downgrade oapi-codegen library
* Update output ts files
* More fixes
* Restore old elasticsearch generated file and skip its generation. Remove core imports into plugins
* More lint fixes
* Add codeowners
* restore embed.go file
* Fix embed.go
Removes legacy alerting, so long and thanks for all the fish! 🐟
---------
Co-authored-by: Matthew Jacobson <matthew.jacobson@grafana.com>
Co-authored-by: Sonia Aguilar <soniaAguilarPeiron@users.noreply.github.com>
Co-authored-by: Armand Grillet <armandgrillet@users.noreply.github.com>
Co-authored-by: William Wernert <rwwiv@users.noreply.github.com>
Co-authored-by: Yuri Tseretyan <yuriy.tseretyan@grafana.com>
* Update template names
* Add verifier that we can use to start verify process
* Use userVerifier when verifying email on update
* Add tests
---------
Co-authored-by: Ieva <ieva.vasiljeva@grafana.com>
* AuthN: Remove embedded oauth server
* Restore main
* go mod tidy
* Fix problem
* Remove permission intersection
* Fix test and lint
* Fix TestData test
* Revert to origin/main
* Update go.mod
* Update go.mod
* Update go.sum
* streamline initialization of test databases, support on-disk sqlite test db
* clean up test databases
* introduce testsuite helper
* use testsuite everywhere we use a test db
* update documentation
* improve error handling
* disable entity integration test until we can figure out locking error
* add support for sortBy field selector
* use label selectors instead of field selectors
* set entity_labels on create & update
* make entity server integration tests work
* test fixes
* be more consistent with handling of empty body, meta or status
* workaround for database is locked errors during migration
* fix double import of sqlite3
* rename functions and tidy up
* refactor update
* disable integration tests until we can fix the database locking issue
* Add test for create method
Co-authored-by: Tania B <10127682+undef1nd@users.noreply.github.com>
* Change structure of entity package to break import cycle
* Update wire file
---------
Co-authored-by: Tania B <10127682+undef1nd@users.noreply.github.com>
* introduce feature toggle
* create base service structure
* fix sample metric
* register metrics
* add to codeowners
* separate api dtos from service models
* remove leading newline
* Refactor to prevent cyclic dependencies
* Move list authorization to the API layer
* Init connectors using the SSO settings service in case the ssoSettingsApi feature toggle is enabled
* wip, need to handle the cyclic dep
* Remove cyclic dependency
* Align tests + refactor
* Move back OAuthInfo to social
* Delete pkg/login/social/constants
* Move reloadable registration to the social providers
* Rename connectors.Error to connectors.SocialError
* first round of entityapi updates
- quote column names and clean up insert/update queries
- replace grn with guid
- streamline table structure
fixes
streamline entity history
move EntitySummary into proto
remove EntitySummary
add guid to json
fix tests
change DB_Uuid to DB_NVarchar
fix folder test
convert interface to any
more cleanup
start entity store under grafana-apiserver dskit target
CRUD working, kind of
rough cut of wiring entity api to kube-apiserver
fake grafana user in context
add key to entity
list working
revert unnecessary changes
move entity storage files to their own package, clean up
use accessor to read/write grafana annotations
implement separate Create and Update functions
* go mod tidy
* switch from Kind to resource
* basic grpc storage server
* basic support for grpc entity store
* don't connect to database unless it's needed, pass user identity over grpc
* support getting user from k8s context, fix some mysql issues
* assign owner to snowflake dependency
* switch from ulid to uuid for guids
* cleanup, rename Search to List
* remove entityListResult
* EntityAPI: remove extra user abstraction (#79033)
* remove extra user abstraction
* add test stub (but
* move grpc context setup into client wrapper, fix lint issue
* remove unused constants
* remove custom json stuff
* basic list filtering, add todo
* change target to storage-server, allow entityStore flag in prod mode
* fix issue with Update
* EntityAPI: make test work, need to resolve expected differences (#79123)
* make test work, need to resolve expected differences
* remove the fields not supported by legacy
* sanitize out the bits legacy does not support
* sanitize out the bits legacy does not support
---------
Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
* update feature toggle generated files
* remove unused http headers
* update feature flag strategy
* devmode
* update readme
* spelling
* readme
---------
Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
* inital changes, db migration
* changes
* Implement basic GetAll, Delete
* Add first batch of tests
* Add more tests
* Add service tests for GetForProvider, List
* Update http_server.go + wire.go
* Lint + update fixed role
* Update CODEOWNERS
* Change API init
* Change roles, rename
* Review with @kalleep
* Revert a mistakenly changed part
* Updates based on @dmihai 's feedback
---------
Co-authored-by: Karl Persson <kalle.persson@grafana.com>
* Add interface verification compliance
* rework service account api to a provider
* wire the service accounts api
* rewire the implementation of sa srv for the proxy
---------
Co-authored-by: Misi <mgyongyosi@users.noreply.github.com>
