* Add getter to FolderAPIBuilder so that we can access it in admission
* Remove deprecated return
* Fix test
* Update pkg/registry/apis/folders/register_test.go
Co-authored-by: maicon <maiconscosta@gmail.com>
* Fix maxNested folder test
* Remove log
---------
Co-authored-by: maicon <maiconscosta@gmail.com>
* WIP: setup to test folders GetAuthorizer
* Setup test
* Extract authorizer fn for tests
* Setup internal test fn
* Better define test inputs
* Add FolderAPI builder to the test
* First test passing
* Test getAuthorize for the create method
* Change authorizerFunc's signature
* [REVIEW] code readability
* Name error
* [REVIEW] add one more test case. Lint
* Remove empty line
* All objects should have an UID
* Now with a different error message
* Simplify create on DW 2: use the same object to write to both storages
* Run only one test
* Add check for status code
* Add name if it's not present in mode2
* Populate UID in legacy
* Remove logs and commented code
* Change dualwriter1
* Remove commented code
* Fix list test
* remove get on update from dualwriter 2
* Get object before updating. Better var renaming
* Finish rebasing
* Comment test
* Uncomment tests
* Update legacy first. Add preconditions
* Remove preconditions
* Fix update test
* copy RV from unified to legacy objects
* revert changes to playlist xorm store
* Improve logging. Add go routines for mode3
* Add tests for async funcs in mode3
* Lint
* Lint
* Lint. Start to fix tests
* Fix watcher tests
* Fix store tests
* Fiinish fixing watcher tests
* Fix server tests
* add name check
* Update pkg/apiserver/rest/dualwriter_mode1.go
Co-authored-by: Bruno Abrantes <bruno.abrantes@grafana.com>
* All objects should have an UID
* Now with a different error message
* Simplify create on DW 2: use the same object to write to both storages
* Run only one test
* Add check for status code
* Add name if it's not present in mode2
* Populate UID in legacy
* Remove logs and commented code
* Change dualwriter1
* Remove commented code
* Fix list test
* remove get on update from dualwriter 2
* Get object before updating. Better var renaming
* Finish rebasing
* Comment test
* Uncomment tests
* Fix update test
* revert changes to playlist xorm store
* Improve logging. Add go routines for mode3
* Lint
* Fix watcher tests
* Fiinish fixing watcher tests
* Add mode 5 with etcd test case. Add early check to fail on populated RV in payload
* we can't set RV to the found object when updating
* Lint
* Don't fail on update playlists
* Name should not be different when updating and it should be not empty on creating
* Fix tests
* Update pkg/apiserver/rest/dualwriter_mode2.go
Co-authored-by: Todd Treece <360020+toddtreece@users.noreply.github.com>
* Lint
* Fix mode 5 tests
* Lint
* Add generateName condition on every mode. Fix tests
* Lint
* Add condition on where name or generate name have to be set
* Fix test
* Lint
* Fix folders test
* We dont need to send name for mode1
* Fail if UID is not present
* Remove change from not running test
* Remove unused line
* Lint
* Update pkg/storage/unified/apistore/store.go
Co-authored-by: Todd Treece <360020+toddtreece@users.noreply.github.com>
* Improve error message
* Fix broken watcher test
* Fail on name mismatch on update
* Remove log
* Make sure UIDs match on create in both stores
* Lint
* Write first to unified storage
* Remove uid setting
* Remove RV only in mode2
* Fix test. Remove log line
* test
* No need to asser on RV in mode3
* Remove RV check due to race condition
* Update dualwriter.go
Co-authored-by: Georges Chaudy <chaudyg@gmail.com>
* Update pkg/storage/unified/client.go
* remove unused parameter
* log an error for object is missing UID instead of returning an error
* remove obj.SetResourceVersion("")
* log an error for object is missing UID instead of returning an error
* FInalise merge
* Move RV check to where it was
* Remove name check
* Remove server check for backwards compatibility
* Remove unused fn
* Move test checks for another PR
* Dont commit go work sum changes
* Only log error if RV is present for now.
---------
Co-authored-by: Todd Treece <todd.treece@grafana.com>
Co-authored-by: Bruno Abrantes <bruno.abrantes@grafana.com>
Co-authored-by: Todd Treece <360020+toddtreece@users.noreply.github.com>
Co-authored-by: Georges Chaudy <chaudyg@gmail.com>
* Remove kubernetesPlaylists feature_toggle
* Remove unified_storage_mode
* Remove double import
* Read from config instead from feature_toggle
* cover scenario for when unified storage is not defined
* Be temporarily retro compatible with previous feature toggle
* Properly read unified_storage section
* [WIP] Read new format of config
* Fix test
* Fix other tests
* Generate feature flags file
* Use <group>.<resource> schema
* Use <group>.resource format on the FE as well
* Hide UniStore config from Frontend
Signed-off-by: Maicon Costa <maiconscosta@gmail.com>
* unwanted changes
* Use feature toggles in the FE. Enforce FTs are present before enabling dual writing
Co-authored-by: Ryan McKinley <ryantxu@users.noreply.github.com>
* use kubernetes playlists feature toggle on the FE
* Remove unwanted code
* Remove configs from the FE
* Remove commented code
* Add more explicit example
---------
Signed-off-by: Maicon Costa <maiconscosta@gmail.com>
Co-authored-by: Maicon Costa <maiconscosta@gmail.com>
* Remove kubernetesPlaylists feature_toggle
* Remove unified_storage_mode
* Remove double import
* Regenerate feature-toggles
* Read from config instead from feature_toggle
* cover scenario for when unified storage is not defined
* Itroduce watcher and mode4
* Logging
* Mode4 should be initialized from the dual writer for observability
* Comment watch while it's not implemented
* Lint
* Use mode log when dual writer is initiated
* Use error from logger
* Dual writer: mode 3
* Add integration tests for playlits in mode 3
* Remove todo
* Update pkg/apiserver/rest/dualwriter_mode3.go
Co-authored-by: Arati R. <33031346+suntala@users.noreply.github.com>
* Admin: Fixes an issue where user accounts could not be enabled (#88117)
Fix: unable to enable user
* [REVIEW] FInish mode 3 and add tests
* Improve logging
* Update dependencies
* Update pkg/apiserver/rest/dualwriter_mode3_test.go
Co-authored-by: maicon <maiconscosta@gmail.com>
* remove test assertion
* Use mode log when dual writer is initiated
---------
Co-authored-by: Arati R. <33031346+suntala@users.noreply.github.com>
Co-authored-by: gonvee <gonvee@qq.com>
Co-authored-by: maicon <maiconscosta@gmail.com>
* Compare results when reading/writing between unified_storage and legacy
* Always use name when comparing objects
* Compare on get method
* Update pkg/apiserver/rest/dualwriter.go
Co-authored-by: Dan Cech <dcech@grafana.com>
* Add new metric to count how many times we read from legacy in mode 2
* Move counter
* Add name in mode1
---------
Co-authored-by: Dan Cech <dcech@grafana.com>
* WIP implement generic compare interface
* Use global compare fn for all entities
* Lint
* Update pkg/apiserver/rest/dualwriter.go
Co-authored-by: Dan Cech <dcech@grafana.com>
* Don't need to hash, just compare bytes
* Fix tests
---------
Co-authored-by: Dan Cech <dcech@grafana.com>
* Pass prometheus registerer to the dual writer
* Fix tests
* Remove unused var
* Fix tests
* Uncomment test
* Remove leading line
* Fix tests. Reuse registerer if there's already one
* Lint
* Improve double registering logic
* Rebase main
* Add and fix tests for playlists in mode1
* Make etcd tests pass mode1 for now
* Fix mode1 and add more tests for playlists in mode 1
* Remove repeated test
* Fix test setup
* Also call storage on mode1. Add metrics
* Update comment
* Don't use compare function for now
* Remove very important space
* Improve code readability on modes 1 and 2
* Fix tests
* Return error from legacy write
* Renume useless defer
* Start adding logging for mode2
* Retrurn errors is storage read fails
* Finish adding latency metrics to mode2
* Emit kind to metrics instead
* Fix var name
* Return if legacy delete list fails
* Unify latency calls
* Fix get function
* Extract returned object hydration into function
* Finish writing tests for utils func
* Lint
* Update pkg/apiserver/rest/dualwriter_mode2.go
Co-authored-by: Arati R. <33031346+suntala@users.noreply.github.com>
* Better var naming
* Remove duplicated logic
* Lint
* Fix test
* Lint
* Make type private
* Fix one more test
* Fix test
---------
Co-authored-by: Arati R. <33031346+suntala@users.noreply.github.com>
* Also call storage on mode1. Add metrics
* Update comment
* Don't use compare function for now
* Remove very important space
* Finish add logging in mode2.
Also call US in mode1 in a non blocking way
* Improve code readability on modes 1 and 2
* Fix tests
* Rename vars
* Lint
* Return error from legacy write
* Renume useless defer
* [REVIEW] improvements
* Pass kind instead of name
* Use kind instead of name in metrics
* Only call latency metrics once
* Return error on writes to legacystore in mode1
* Move accesssor logic into the goroutine as well
* Choose mode. Add log field on top level dualwriter
* Add logs
* Turn DualWriter into a full interface. Fix tests
* Lint
* Use struct for dualWriter interface
* Use struct
* Default should be legacyStore for all entities
* Fix test. Get rid of extra concrete type
* Remove comment
* Add comment
* Temp set dualwriter mode 2 for playlists while configs are not in place
* Add modes type + add comment on what each mode does
* Don't require watcher interface for now
* Use storage implementation on mode 2
* Update pkg/apiserver/rest/dualwriter_mode2.go
Co-authored-by: Todd Treece <360020+toddtreece@users.noreply.github.com>
* Pass log values to the context
* test
* Update pkg/apiserver/rest/dualwriter_mode3.go
Co-authored-by: Dan Cech <dcech@grafana.com>
---------
Co-authored-by: Todd Treece <360020+toddtreece@users.noreply.github.com>
Co-authored-by: Dan Cech <dcech@grafana.com>
* Add update methods for the dual writer
* improve errors
* [WIP] add tests for the update method
* Move example package to its own package so it can be used by the rest package. Finish tests
* Add codeowners
* Use Pod as a dummy resource
* :int
* Lint
* [REVIEW] rename var
* [REVIEW] don't rely on legacy storage at all in mode4
* Update pkg/apiserver/rest/dualwriter_mode2.go
Co-authored-by: Arati R. <33031346+suntala@users.noreply.github.com>
* [REVIEW] improve comment
* Update pkg/apiserver/rest/dualwriter_mode1.go
Co-authored-by: Dan Cech <dcech@grafana.com>
* [REVIEW] improve mode3
* Lint
* Move test files
* Update pkg/apiserver/rest/dualwriter_mode2.go
Co-authored-by: Arati R. <33031346+suntala@users.noreply.github.com>
* Lint
* Update pkg/apiserver/rest/dualwriter_mode4_test.go
Co-authored-by: Arati R. <33031346+suntala@users.noreply.github.com>
* Fix error
* Lint
* Update pkg/apiserver/rest/dualwriter_mode2.go
Co-authored-by: Arati R. <33031346+suntala@users.noreply.github.com>
* Don't set the flag to true as updatedObj creates an object in case it's not found
* Lint
* Lint
* Add tests on update
* Lint
---------
Co-authored-by: Arati R. <33031346+suntala@users.noreply.github.com>
Co-authored-by: Dan Cech <dcech@grafana.com>
* WIP: add mocks for dual writer
* Test dualwriter mode1
* Re-add non implementation errors
* Use testify assert
* Write tests for mode2.
* Lint
* Remove comment
* Update pkg/apiserver/rest/dualwriter_mode2_test.go
Co-authored-by: Arati R. <33031346+suntala@users.noreply.github.com>
* [REVIEW improve readability and call fn legacy fn
* [REVIEW] rename mocks and setupFns
* [REVIEW add missing test case
* [REVIEW] add test case with getting object from storage and not legacy
* Add deleteCollection
* Test deleteCollection
* Rename file
---------
Co-authored-by: Arati R. <33031346+suntala@users.noreply.github.com>
* Add Get method in dual writer
* Update pkg/apiserver/rest/dualwriter_mode3.go
Co-authored-by: Dan Cech <dcech@grafana.com>
* Actually run the tests
* no need for t.Run()
* Update pkg/apiserver/rest/dualwriter_mode3.go
Co-authored-by: Dan Cech <dcech@grafana.com>
* Update pkg/apiserver/rest/dualwriter_mode3.go
Co-authored-by: Dan Cech <dcech@grafana.com>
* Update pkg/apiserver/rest/dualwriter_test.go
Co-authored-by: Dan Cech <dcech@grafana.com>
* Log improvements
* Fix method name
---------
Co-authored-by: Dan Cech <dcech@grafana.com>
Leonor Oliveira
