This adds a version of the SQLStore that includes a ReadReplica. The primary DB can be accessed directly - from the caller's standpoint, there is no difference between the SQLStore and ReplStore unless they wish to explicitly call the ReadReplica() and use that for the DB sessions.
Currently only the stats service GetSystemStats and GetAdminStats are using the ReadReplica(); if it's misconfigured or if the databaseReadReplica feature flag is not turned on, it will fall back to the usual (SQLStore) behavior.
Testing requires a database and read replica - the replication should already be configured. I have been testing this locally with a docker mysql setup (https://medium.com/@vbabak/docker-mysql-master-slave-replication-setup-2ff553fceef2) and the following config:
[feature_toggles]
databaseReadReplica = true
[database]
type = mysql
name = grafana
user = grafana
password = password
host = 127.0.0.1:3306
[database_replica]
type = mysql
name = grafana
user = grafana
password = password
host = 127.0.0.1:3307
* make cfg private in sqlstore
* fix db init in tests
* fix case
* fix folder test init
* fix imports
* make another Cfg private
* remove another Cfg
* remove unused variable
* use store cfg, it has side-effects
* fix mutated cfg in tests
* separate context and threads for usage stats
* use constants
* ignore original context
* fix runMetricsFunc
* fix collector registration
Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>
* change background ctx
* fix test randomness
* Add traces to support bundle collector
* Remove unecessay span
* Add trace to usagestats api
* Close spans
* Mv trace to bundle
* Change span name
* use parent context
* fix runtime declare of stats
* Fix pointer dereference problem on usage stat func
Co-authored-by: Karl Persson <kalle.persson@grafana.com>
Co-authored-by: jguer <joao.guerreiro@grafana.com>
* fix broken support bundle tests by tracer
---------
Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>
Co-authored-by: gamab <gabriel.mabille@grafana.com>
Co-authored-by: Karl Persson <kalle.persson@grafana.com>
* streamline initialization of test databases, support on-disk sqlite test db
* clean up test databases
* introduce testsuite helper
* use testsuite everywhere we use a test db
* update documentation
* improve error handling
* disable entity integration test until we can figure out locking error
* fix timeout issues when gathering prometheus flavor stats
* workaround data race in sdk tracing middleware
* cap concurrency at 10
---------
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
* Do not update statistics at service collector startup
* Configurable collector interval
* Introduce initial random delay
* Prevent reporting metrics until the stats have been collected
* Apply suggestion from code review
* define initial service and add to wire
* update caching service interface
* add skipQueryCache header handler and update metrics query function to use it
* add caching service as a dependency to query service
* working caching impl
* propagate cache status to frontend in response
* beginning of improvements suggested by Lean - separate caching logic from query logic.
* more changes to simplify query function
* Decided to revert renaming of function
* Remove error status from cache request
* add extra documentation
* Move query caching duration metric to query package
* add a little bit of documentation
* wip: convert resource caching
* Change return type of query service QueryData to a QueryDataResponse with Headers
* update codeowners
* change X-Cache value to const
* use resource caching in endpoint handlers
* write resource headers to response even if it's not a cache hit
* fix panic caused by lack of nil check
* update unit test
* remove NONE header - shouldn't show up in OSS
* Convert everything to use the plugin middleware
* revert a few more things
* clean up unused vars
* start reverting resource caching, start to implement in plugin middleware
* revert more, fix typo
* Update caching interfaces - resource caching now has a separate cache method
* continue wiring up new resource caching conventions - still in progress
* add more safety to implementation
* remove some unused objects
* remove some code that I left in by accident
* add some comments, fix codeowners, fix duplicate registration
* fix source of panic in resource middleware
* Update client decorator test to provide an empty response object
* create tests for caching middleware
* fix unit test
* Update pkg/services/caching/service.go
Co-authored-by: Arati R. <33031346+suntala@users.noreply.github.com>
* improve error message in error log
* quick docs update
* Remove use of mockery. Update return signature to return an explicit hit/miss bool
* create unit test for empty request context
* rename caching metrics to make it clear they pertain to caching
* Update pkg/services/pluginsintegration/clientmiddleware/caching_middleware.go
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
* Add clarifying comments to cache skip middleware func
* Add comment pointing to the resource cache update call
* fix unit tests (missing dependency)
* try to fix mystery syntax error
* fix a panic
* Caching: Introduce feature toggle to caching service refactor (#66323)
* introduce new feature toggle
* hide calls to new service behind a feature flag
* remove licensing flag from toggle (misunderstood what it was for)
* fix unit tests
* rerun toggle gen
---------
Co-authored-by: Arati R. <33031346+suntala@users.noreply.github.com>
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
* fix: disable orgrolepicker if externaluser is synced
* add disable to role picker
* just took me 2 hours to center the icon
* wip
* fix: check externallySyncedUser for API call
* remove check from store
* add: tests
* refactor authproxy and made tests run
* add: feature toggle
* set feature toggle for tests
* add: IsProviderEnabled
* refactor: featuretoggle name
* IsProviderEnabled tests
* add specific tests for isProviderEnabled
* fix: org_user tests
* add: owner to featuretoggle
* add missing authlabels
* remove fmt
* feature toggle
* change config
* add test for a different authmodule
* test refactor
* gen feature toggle again
* fix basic auth user able to change the org role
* test for basic auth role
* make err.base to error
* lowered lvl of log and input mesg
* Remove Result field from AddDataSourceCommand
* Remove DatasourcesPermissionFilterQuery Result
* Remove GetDataSourceQuery Result
* Remove GetDataSourcesByTypeQuery Result
* Remove GetDataSourcesQuery Result
* Remove GetDefaultDataSourceQuery Result
* Remove UpdateDataSourceCommand Result
* add bundle registry service to avoid dependency cycles
* move user support bundle collector to user service
* move usage stat bundle implementation to usage stats
* add info for background service
* fix remaining imports
* whitespace
* reorganize auth usage stats
* usage stat privilege elevators
* stat count of modified role
* cfg related info
* add authn anon client
* kv store
* ensure anon enabled is collected even if client is not registered
* fix usage stats test
* move original stats service into a separate package
* add stats service to wire
* move GetAdminStats
* switch to using stats.Service
* add missing package
* fix api tests
* chore: add alias for InitTestDB and Session
Adds an alias for the sqlstore InitTestDB and Session, and updates tests using these to reduce dependencies on the sqlstore.Store.
* next pass of removing sqlstore imports
* last little bit
* remove mockstore where possible
* NewIA: Plugin nav config
* progress
* Progress
* Things are working
* Add monitoring node
* Add alerts and incidents
* added experiment with standalone page
* Refactoring by adding a type for navtree root
* First test working
* More tests
* more tests
* Progress on richer config and sorting
* Sort weight working
* Path config
* Improving logic for not including admin or cfg nodes, making it the last step so that enterprise can add admin nodes without having to worry about the section not existing
* fixed index routes
* removed file
* Fixes
* Fixing tests
* Fixing more tests and adding support for weight config
* Updates
* Remove unused fake
* More fixes
* Minor tweak
* Minor fix
* Can now control position using sortweight even when existing items have no sortweight
* Added tests for frontend standalone page logic
* more tests
* Remove unused fake and fixed lint issue
* Moving reading settings to navtree impl package
* remove nav_id setting prefix
* Remove old test file
* Fix trailing newline
* Fixed bug with adding nil node
* fixing lint issue
* remove some code we have to rethink
* move read settings to PrivideService and switch to util.SplitString
