Commit Graph

87 Commits

Author SHA1 Message Date
owensmallwood
d96baaa878
Search PoC: Add logging (#94567) 2024-10-10 20:34:57 +03:00
Prem Saraswat
5c03c14b25
resource-api: Loosen name validation to match K8s requirements (#93404)
* resource-api: Loosen name validation to match K8s requirements

This patch modifies some of the requirements for name validation of
objects in Resource API to match Kubernetes.

The limit we have on characters in name is 64, but some resources allow
upto 253 characters. Similarly we also include `:` in the regex, as many
objects in default K8s setup use it in the name (the group
`system:masters` for example)

Signed-off-by: Prem Kumar <prem.saraswat@grafana.com>

* Update the name column length in migrator and update e2e test to verify

---------

Signed-off-by: Prem Kumar <prem.saraswat@grafana.com>
2024-10-10 11:03:18 -04:00
Diego Augusto Molina
4a800eda9f
Unistore Chore: add inocuous preparative changes for otel-based db observability (#94473)
add inocuous preparative changes for otel tracing
2024-10-09 20:32:09 +00:00
owensmallwood
612b864772
Indexing PoC: Add search/browse (#94126)
* adds Filter gRPC and make protobuf

* adds route for querying the filter gRPC

* wires up Filter gRPC call

* [WIP] index from start

* renames gRPC endpoint to "Search"

* adds /apis/search route into k8s routes. Hacky for now.

* updates readme - wrong casing

* adds feature toggle for unified storage search

* hides US search behind feature flag. Clean up print statements.

* removes indexer - will be added in another PR

* Search: Add API Builder

* adds required method

* implementing UpdateAPIGroupInfo (WIP)

* adds groupversion

* commenting out for now

* remove unneeded code from experimenting and update register.go to match interface required

* list resources and load into index

* pass context

* namespaces search route

* lint

* watch

* add todo

* add todo

* merge

* cleanup

* add todo

* gen protobuf

* lint; fix migration issue

* Updates index mapping function to map unified storage object Value

* Changes Index() to pointer receiver - fixes panic

* add delete

* cleanup

* gets search/browse functioning. Results show up as base64 encoded. Still a WIP.

* Doesnt json re-encode gRPC response in search handler

* add kind to SearchRequest proto

* Updates query interface to be more generic. Make proto. Parses query params in api server.

* make protobuf

* removes unused method and imports

* Returns all indexed fields in search results. Adds pagination support (limit + offset).

* remove comment

* remove unused struct

* gets tenant in search k8s api handler

* adds hardcoded spec field mappings - starting with playlists

* adds all spec fields to search results

* moved helper function for field mappings into index

* only includes allowed spec fields in search results

* cleans up error handling

* removes debug log

---------

Co-authored-by: leonorfmartins <leonorfmartins@gmail.com>
Co-authored-by: Todd Treece <todd.treece@grafana.com>
Co-authored-by: Scott Lepper <scott.lepper@gmail.com>
2024-10-09 11:20:05 -06:00
Scott Lepper
a3764ebeba
[Search] fix: add and delete (#94438)
[search] fix: add and delete from index
2024-10-08 16:17:31 -04:00
Scott Lepper
c2fb2dcfbe
wire up unified search from the ui; add basic search support (#94358)
* wire up search from the ui;  add basic search support
2024-10-08 13:09:56 -04:00
Scott Lepper
52b70ca976
Grafana indexing poc - load and watch index (#93843)
* list resources and load into index
* watch for changes and update index
2024-10-08 09:43:23 -04:00
Arati R.
ea5cf7c51f
Unified Storage /Folders: Allow Unified Storage subfolders creation (#94327)
* Add parents field to folder DTO
* Allow subfolder creation when folder flag is enabled
* Update UnstructuredToLegacyFolder
* Include parents field when creating folder
2024-10-07 16:48:56 +02:00
Matheus Macabu
4bc7a35f56
UnifiedStorage: fix watcher test (#94301) 2024-10-07 12:07:48 +03:00
Georges Chaudy
03f55e5288
Unistore : Ensure Watch works in HA mode (#94120)
* Revert "Revert "Unistore : Ensure Watch works in HA mode." (#94097)"

This reverts commit 7c3fc2f261.

* make previous_resource_version nullable

* handle nil case
2024-10-07 10:01:53 +02:00
Diego Augusto Molina
7d32d5eff4
Unistore: Reuse MySQL and Postgres Grafana core config instead of the object (#94223)
* Reuse MySQL and Postgres Grafana config instead of the object

- Only reuse the Grafana DB object for SQLite. Support for SQLite will be added in a different PR
- Fail when reusing the Grafana DB object if it is using DB instrumentation
- In the case that we have to reuse a Grafana DB with its instrumentation, fail with an error that describes a workaround
- Add regression tests to reproduce incident 2144

* remove temp file

* fix linter

* fix linter x2

* fix linter x3
2024-10-04 12:07:20 +00:00
Todd Treece
a4d919c157
Chore: Update k8s.io dependencies to v0.31.1 (#93696) 2024-10-03 15:50:15 -04:00
Georges Chaudy
7c3fc2f261
Revert "Unistore : Ensure Watch works in HA mode." (#94097)
Revert "Unistore : Ensure Watch works in HA mode. (#93428)"

This reverts commit 0a26c9e9ae.
2024-10-01 18:45:47 +00:00
owensmallwood
6a3eb276ef
Grafana Indexing PoC: Adds feature flag and gRPC endpoint (#93356)
* adds Filter gRPC and make protobuf

* adds route for querying the filter gRPC

* wires up Filter gRPC call

* [WIP] index from start

* renames gRPC endpoint to "Search"

* adds /apis/search route into k8s routes. Hacky for now.

* updates readme - wrong casing

* adds feature toggle for unified storage search

* hides US search behind feature flag. Clean up print statements.

* removes indexer - will be added in another PR

* Search: Add API Builder

* adds required method

* implementing UpdateAPIGroupInfo (WIP)

* adds groupversion

* commenting out for now

* remove unneeded code from experimenting and update register.go to match interface required

* namespaces search route

---------

Co-authored-by: leonorfmartins <leonorfmartins@gmail.com>
Co-authored-by: Todd Treece <todd.treece@grafana.com>
2024-09-30 13:46:14 -06:00
Georges Chaudy
0a26c9e9ae
Unistore : Ensure Watch works in HA mode. (#93428)
* Replace Watch with WatchNext

* remove watchset

* fix previous page and closing the channel

* Remove the broadcaster cache to prevent dupplicated events

* add watch bookmark

* add watch bookmark

* cleanup comments

* disable the tests for bookmarks for now

* Ensure we send previosu events

* lint

* re-introduce the cache

* load from cache

* disabling legacy test

* disabling legacy test

* Update pkg/storage/unified/resource/server.go

Co-authored-by: Diego Augusto Molina <diegoaugustomolina@gmail.com>

* Could not read previous events

* add proper migration

* Add previous_resource_version to both history and resource

* First event should have an RV of 2 and not 1

* Test both storage backends

* fix the inital RV for the sql backend

* ensure graceful stop of the stream decoder

* gocyclo

---------

Co-authored-by: Diego Augusto Molina <diegoaugustomolina@gmail.com>
2024-09-30 13:14:07 +02:00
Ryan McKinley
cdbc04ab2b
GRPC/Authn: Remove org name header (#93780) 2024-09-26 10:58:59 +03:00
Claudiu Dragalina-Paraipan
a8b07b0c81
[authn] use authlib client+interceptors for in-proc mode (#93124)
* Add authlib gRPC authenticators for in-proc mode

* implement `StaticRequester` signing in the unified resource client
- [x] when the `claims.AuthInfo` value type is `identity.StaticRequester`, and there's no ID token set, create an internal token and sign it with symmetrical key. This is a workaround for `go-jose` not offering the possibility to create an unsigned token.
- [x] update `IDClaimsWrapper` to support the scenario above
- [x] Switch to using `claims.From()` in `dashboardSqlAccess.SaveDashboard()`

---------

Co-authored-by: gamab <gabriel.mabille@grafana.com>
2024-09-24 09:03:48 +03:00
owensmallwood
8bbfbc61be
Unified Storage: Adds a few readme updates from issues I ran into with local dev (#93553)
adds a few readme updates from issues I ran into with local dev
2024-09-23 08:25:19 -06:00
Georges Chaudy
216b63549d
unistore: fix watch events forever looping (#93517) 2024-09-20 15:51:09 +02:00
Andres Martinez Gotor
74b2d38e37
Chore: Update grafana-plugin-sdk (#93426) 2024-09-20 10:20:59 +02:00
Ryan McKinley
542105b680
ResourceClient: Exercise resource client in k8s apis tests (#93473) 2024-09-19 17:16:48 +03:00
Todd Treece
d1ffcc22d9
Playlists: Migrate to App SDK codegen (#93246) 2024-09-13 16:27:40 -04:00
Ryan McKinley
45eb72e95a
K8s: Use wire to initalize the resource client (#93221) 2024-09-12 17:22:27 +03:00
Ryan McKinley
7efadb0a00
Testing: Integration tests for unified-grpc start a local gRPC server (#93201) 2024-09-11 11:50:14 +03:00
Arati R.
b12a29a1da
K8s: Implement partial folders api with k8s client (#93089)
* Add kubernetes folder feature toggle
* Add kubernetes routes for getting and creating a folder
* Add documentation for interacting with k8s folders
2024-09-10 11:22:08 +02:00
Claudiu Dragalina-Paraipan
3aeb8d390e
[authn] update authlib and claims versions (#93098)
* update authlib version to latest

* make update-workspace -- 2nd run

* manual cleanup of old version
2024-09-09 15:45:59 +03:00
ismail simsek
6548ea377d
Chore: Bump grafana-plugin-sdk-go version to v0.247.0 (#93095)
* Bump grafana-plugin-sdk-go version to v0.247.0

* make update-workspace

* make update-workspace
2024-09-09 14:36:59 +03:00
Dave Henderson
e1090db5d9
Chore: Bump Go to 1.23.1 (#93007)
Signed-off-by: Dave Henderson <dave.henderson@grafana.com>
2024-09-06 20:58:50 +03:00
Ryan McKinley
3cf84d1921
Chore: update centrifuge library (#93020) 2024-09-06 12:17:43 +03:00
Claudiu Dragalina-Paraipan
1ce43b4c42
[authn] update authlib and claims versions (#92987)
* update authlib and claims versions

* 2nd make update-workspace

* manual fix
2024-09-05 19:17:15 +03:00
Santiago
87f4df4bc3
Remote Alertmanager: update github.com/go-openapi/runtime (v0.27.1 -> v0.28.0) (#92951) 2024-09-05 08:47:38 -03:00
Ryan McKinley
5441e4c752
Storage: create a client with access to all interfaces, not just ResourceStore (#92967) 2024-09-05 12:52:30 +03:00
Marcus Efraimsson
0a337ff3b3
Chore: Update SDK to v0.246.0 (#92938) 2024-09-04 17:30:37 +02:00
maicon
9e6a464328
UniStore: Remove unused feature toggles from docs (#92688)
`grafanaAPIServer` got removed on https://github.com/grafana/grafana/pull/81030
`idForwarding` got removed on https://github.com/grafana/grafana/pull/92209

Signed-off-by: Maicon Costa <maiconscosta@gmail.com>
2024-08-29 18:27:38 +03:00
maicon
2b3d2e5b40
UniStore: Remove leftover references to unifiedStorage feature toggle (#92685)
it got removed on https://github.com/grafana/grafana/pull/92192

Signed-off-by: Maicon Costa <maiconscosta@gmail.com>
2024-08-29 18:04:36 +03:00
Charandas
af2e79aa83
K8s: namespace mapper should use authlib's util (#92332) 2024-08-27 15:01:42 -07:00
Ryan McKinley
c59dddf7af
MySQL: Add parseTime=true to SQL connections (#92469) 2024-08-27 14:16:04 +03:00
Ryan McKinley
5a30e12a10
SQLTemplate: Make Ident only work for identifiers (not any string) (#92387) 2024-08-27 13:22:40 +03:00
Todd Treece
2f01286034
Storage: Add go.mod for apistore (#92224) 2024-08-21 14:32:01 -04:00
Ryan McKinley
2136fd9a92
Storage: Remove unified storage feature flag (#92192)
remove unified storage flag
2024-08-21 19:28:30 +03:00
Dave Henderson
df3d8915ba
Chore: Bump Go to 1.23.0 (#92105)
* chore: Bump Go to 1.23.0

Signed-off-by: Dave Henderson <dave.henderson@grafana.com>

* update swagger files

Signed-off-by: Dave Henderson <dave.henderson@grafana.com>

* chore: update .bingo/README.md formatting to satisfy prettier

Signed-off-by: Dave Henderson <dave.henderson@grafana.com>

* chore(lint): Fix new lint errors found by golangci-lint 1.60.1 and Go 1.23

Signed-off-by: Dave Henderson <dave.henderson@grafana.com>

* keep golden file

* update openapi

* add name to expected output

* chore(lint): rearrange imports to a sensible order

Signed-off-by: Dave Henderson <dave.henderson@grafana.com>

---------

Signed-off-by: Dave Henderson <dave.henderson@grafana.com>
Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
2024-08-21 11:40:42 -04:00
Diego Augusto Molina
9a65d0a977
Storage: remove sqlstore.WrapDatabaseDriverWithHooks (#92111)
remove sqlstore.WrapDatabaseDriverWithHooks from Unified Storage after related incident
2024-08-21 10:31:34 -03:00
Diego Augusto Molina
e788df921c
Storage: always use transactions and make them readonly when possible (#92110)
* always use transactions and make them readonly when possible

* fix linters

* fix reference
2024-08-20 09:29:06 -03:00
owensmallwood
1cb3470e3e
Unified Storage: Wrapping the mysql driver with hooks causes issues (#92099)
dont wrap the mysql driver with hooks. Its causing the error 'begin tx: sql: driver does not support non-default isolation level'.
2024-08-19 15:09:25 -06:00
Ryan McKinley
9567275365
SQL: Add more legacy helpers (#92006) 2024-08-19 12:15:43 +03:00
Ryan McKinley
f432a1713b
SQLTemplate: Swap the IFace with the concrete struct (#92008) 2024-08-16 15:12:37 +03:00
Ryan McKinley
d9cabe5e14
SQL: Add sql template test helper (#91953) 2024-08-16 14:36:56 +03:00
owensmallwood
28f2a4078d
Unified Storage: Records grafana database metrics (#91932)
* records grafana database metrics for unified storage

* update type to tracing.Tracer

* use nil for tracer in tests
2024-08-15 11:13:36 -06:00
Ryan McKinley
a0cd89860e
Identity: Add endpoint to get display info for an identifier (#91828) 2024-08-15 14:38:43 +03:00
Todd Treece
4ce82d3f14
Chore: Update k8s.io dependencies to v0.30.0 (#91851) 2024-08-13 14:01:48 -04:00