Commit Graph

17 Commits

Author SHA1 Message Date
Leonor Oliveira
ec6a939815
Compare results when reading/writing between unified_storage and legacy (#89773)
* Compare results when reading/writing between unified_storage and legacy

* Always use name when comparing objects

* Compare on get method

* Update pkg/apiserver/rest/dualwriter.go

Co-authored-by: Dan Cech <dcech@grafana.com>

* Add new metric to count how many times we read from legacy in mode 2

* Move counter

* Add name in mode1

---------

Co-authored-by: Dan Cech <dcech@grafana.com>
2024-07-05 14:01:05 +02:00
Todd Treece
9c6d3590a2
Chore: Fix apiserver circular dependencies (#89918) 2024-07-01 11:42:34 -04:00
Leonor Oliveira
2645958c8c
Use global function to compare any entity to both stores (#89282)
* WIP implement generic compare interface

* Use global compare fn for all entities

* Lint

* Update pkg/apiserver/rest/dualwriter.go

Co-authored-by: Dan Cech <dcech@grafana.com>

* Don't need to hash, just compare bytes

* Fix tests

---------

Co-authored-by: Dan Cech <dcech@grafana.com>
2024-06-26 10:19:40 +02:00
Leonor Oliveira
fd44f2ee4f
Init dualwriter metrics (#89003)
* Pass prometheus registerer to the dual writer

* Fix tests

* Remove unused var

* Fix tests

* Uncomment test

* Remove leading line

* Fix tests. Reuse registerer if there's already one

* Lint

* Improve double registering logic

* Rebase main
2024-06-14 11:01:49 +02:00
Arati R
36f42853dd
Storage: Read desired mode from config instead of feature flags (#88353)
* Read desired mode from config
* Update playlist integration tests
* Add mode 1 playlist integration tests
* Add mode 0 dual writing to playlist integration tests
* Add documentation for the different dual writing modes
2024-05-31 19:29:59 +02:00
Leonor Oliveira
ade96dbdbd
Introduce Comparator interface (#88016)
* Introduce Comparator interface

* Add compare implementation everywhere

* Add comment explaining what Compare should do

* Lint
2024-05-29 09:42:24 +02:00
Arati R
0ffc4c441b
Storage: Add mode reconciliation for modes 1 and 2 (#87919)
* Add skeleton implementation for mode reconciliation between 1 and 2
* Track mode for each dual writer
* Add test for setting dual writer
* Include context when setting dual writing mode

---------

Co-authored-by: Dan Cech <dcech@grafana.com>
2024-05-23 00:12:46 +02:00
Arati R
6836bfe1ea
Storage: Customise setting dual writing modes (#87668)
* Add feature toggles for mode 2 and 3 playlist dual writing
* Make current mode customised based on kind
* Check feature flags when initialising dual writer
* Fix linting
* Refactor NewDualWriter
2024-05-14 14:11:56 +02:00
Leonor Oliveira
ee2f6a7b49
Force interface implementation also on legacy storage (#87414)
* Force interface implementation also on legacy storage

* Add DeleteCollection to folders and dashboards

* Fix integration tests

* Fix tests
2024-05-07 15:02:30 +02:00
Leonor Oliveira
0a2c5065a0
Choose mode. Turn DualWriter into a real interface. Better Logging (#87291)
* Choose mode. Add log field on top level dualwriter

* Add logs

* Turn DualWriter into a full interface. Fix tests

* Lint

* Use struct for dualWriter interface

* Use struct

* Default should be legacyStore for all entities

* Fix test. Get rid of extra concrete type

* Remove comment

* Add comment

* Temp set dualwriter mode 2 for playlists while configs are not in place

* Add modes type + add comment on what each mode does

* Don't require watcher interface for now

* Use storage implementation on mode 2

* Update pkg/apiserver/rest/dualwriter_mode2.go

Co-authored-by: Todd Treece <360020+toddtreece@users.noreply.github.com>

* Pass log values to the context

* test

* Update pkg/apiserver/rest/dualwriter_mode3.go

Co-authored-by: Dan Cech <dcech@grafana.com>

---------

Co-authored-by: Todd Treece <360020+toddtreece@users.noreply.github.com>
Co-authored-by: Dan Cech <dcech@grafana.com>
2024-05-06 23:18:28 +02:00
Leonor Oliveira
9baf96dbd2
Storage: dualwriter update implementation (#85844)
* Add update methods for the dual writer

* improve errors

* [WIP] add tests for the update method

* Move example package to its own package so it can be used by the rest package. Finish tests

* Add codeowners

* Use Pod as a dummy resource

* :int

* Lint

* [REVIEW] rename var

* [REVIEW] don't rely on legacy storage at all in mode4

* Update pkg/apiserver/rest/dualwriter_mode2.go

Co-authored-by: Arati R. <33031346+suntala@users.noreply.github.com>

* [REVIEW] improve comment

* Update pkg/apiserver/rest/dualwriter_mode1.go

Co-authored-by: Dan Cech <dcech@grafana.com>

* [REVIEW] improve mode3

* Lint

* Move test files

* Update pkg/apiserver/rest/dualwriter_mode2.go

Co-authored-by: Arati R. <33031346+suntala@users.noreply.github.com>

* Lint

* Update pkg/apiserver/rest/dualwriter_mode4_test.go

Co-authored-by: Arati R. <33031346+suntala@users.noreply.github.com>

* Fix error

* Lint

* Update pkg/apiserver/rest/dualwriter_mode2.go

Co-authored-by: Arati R. <33031346+suntala@users.noreply.github.com>

* Don't set the flag to true as updatedObj creates an object in case it's not found

* Lint

* Lint

* Add tests on update

* Lint

---------

Co-authored-by: Arati R. <33031346+suntala@users.noreply.github.com>
Co-authored-by: Dan Cech <dcech@grafana.com>
2024-04-17 08:32:34 -06:00
Arati R
79eab65f94
Storage: Add mode-specific DeleteCollection implementations (#86065)
* Add DeleteCollection methods to modes 1 and 4
* Add DeleteCollection implementations for modes 2 and 3
2024-04-16 17:05:31 +02:00
Leonor Oliveira
d17af78e79
Storage: dualwriter delete implementation (#86000)
* Add delete methods

* Remove duplicated const

* Add tests

* Lint

* Lint

* Remove duplicated test file

* Update pkg/apiserver/rest/dualwriter_mode2.go

Co-authored-by: Arati R. <33031346+suntala@users.noreply.github.com>

* Update pkg/apiserver/rest/dualwriter.go

Co-authored-by: Arati R. <33031346+suntala@users.noreply.github.com>

* Update pkg/apiserver/rest/dualwriter_mode2.go

Co-authored-by: Arati R. <33031346+suntala@users.noreply.github.com>

* Add missing dependency

* Return if object deletion goes wrong

* Add a more complete log

---------

Co-authored-by: Arati R. <33031346+suntala@users.noreply.github.com>
2024-04-15 10:48:31 +02:00
Arati R
4e532b00e1
Storage: Add mode-specific List implementations (#85814)
* Add List methods to mode 1 and 4
* Refactor errors
2024-04-11 13:41:05 +02:00
Leonor Oliveira
8d75dcec04
Add better logging to the dual writer (#85594)
* Make Legacy a public field

* Remove duplicated Create method

* Add logger to dualwriter

* Use klog

* Add comment about selecting the dual writer

* Update pkg/apiserver/rest/dualwriter_mode1.go

Co-authored-by: Arati R. <33031346+suntala@users.noreply.github.com>

* Update pkg/apiserver/rest/dualwriter_mode2.go

Co-authored-by: Arati R. <33031346+suntala@users.noreply.github.com>

* Update pkg/apiserver/rest/dualwriter_mode3.go

Co-authored-by: Arati R. <33031346+suntala@users.noreply.github.com>

* Update pkg/apiserver/rest/dualwriter_mode3.go

Co-authored-by: Arati R. <33031346+suntala@users.noreply.github.com>

* Update pkg/apiserver/rest/dualwriter_mode2.go

Co-authored-by: Arati R. <33031346+suntala@users.noreply.github.com>

* Create error var

* Lint

---------

Co-authored-by: Arati R. <33031346+suntala@users.noreply.github.com>
2024-04-09 10:54:09 +02:00
Arati R
2232fe033b
Storage: Add mode-specific dual writers (#85551)
* Set up skeleton dual writers for each mode
* Add Create functionality to each of the mode-specific DualWriters
* Add switch for selecting DualWriter
2024-04-04 14:02:51 +02:00
Todd Treece
e5a26a3f7c
K8s: Add apimachinery and apiserver packages (#83190) 2024-02-23 15:15:43 -05:00