* Create recording rule fields in model
* Add migration
* Write to database, support in version table
* extend fingerprint
* Force fields to be empty on validate
* Another storage spot, tests for fingerprint
* Explicitly set defaults in provisioning API
* Tests for main API validation
* Add diff tests even though fields are unpopulated for now
* Use struct tag approach instead of FromDB/ToDB hooks as it better handles nulls when deserializing
* test for deser
* Backout RecordTo for now since it's not decided in the doc
* back out of migration too
* Drop datasourceref for now
* address linter complaints
* Try a single outer struct with all fields embedded
* AuthN: Remove embedded oauth server
* Restore main
* go mod tidy
* Fix problem
* Remove permission intersection
* Fix test and lint
* Fix TestData test
* Revert to origin/main
* Update go.mod
* Update go.mod
* Update go.sum
* Add notification settings to storage\domain and API models. Settings are a slice to workaround XORM mapping
* Support validation of notification settings when rules are updated
* Implement route generator for Alertmanager configuration. That fetches all notification settings.
* Update multi-tenant Alertmanager to run the generator before applying the configuration.
* Add notification settings labels to state calculation
* update the Multi-tenant Alertmanager to provide validation for notification settings
* update GET API so only admins can see auto-gen
* add annotation permissions to dashboard managed role and add migrations for annotation permissions
* fix a bug with conditional access level definitions
* add tests
* Update pkg/services/sqlstore/migrations/accesscontrol/dashboard_permissions.go
Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>
* apply feedback
* add batching, fix tests and a typo
* add one more test
* undo unneeded change
* undo unwanted change
* only check the default basic permissions for non-OSS instances
* account for all wildcards and simplify the check a bit
* error handling and extra conditionals to avoid test failures
* fix a bug with admin permissions not appearing for folders
* fix the OSS check
---------
Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>
* Alerting: Increase size of kvstore value type for MySQL to LONGTEXT
alertmanager uses the kvstore to persist its notification log and the current
column limit for MySQL (16.7mb) puts the maximum entries at a level that is
potentially achievable for heavy alerting users (~40-80k entries).
In comparison, the current type for PSQL (TEXT) is effectively unlimited and
I believe SQLIte defaults to 2gb which is also plenty of leeway.
* Alerting: Keep track of individual org migration status
Save migration status per migrated org.
Change the meaning (and key/value) of the org_id=0 entry
to store the current (previous) config value used by alerting.
This is so we can know when to upgrade/downgrade by
comparing with the new config value in
UnifiedAlerting.IsEnabled.
* inital changes, db migration
* changes
* Implement basic GetAll, Delete
* Add first batch of tests
* Add more tests
* Add service tests for GetForProvider, List
* Update http_server.go + wire.go
* Lint + update fixed role
* Update CODEOWNERS
* Change API init
* Change roles, rename
* Review with @kalleep
* Revert a mistakenly changed part
* Updates based on @dmihai 's feedback
---------
Co-authored-by: Karl Persson <kalle.persson@grafana.com>
This PR replaces the vendored models in the migration with their equivalent ngalert models. It also replaces the raw SQL selects and inserts with service calls.
It also fills in some gaps in the testing suite around:
- Migration of alert rules: verifying that the actual data model (queries, conditions) are correct 9a7cfa9
- Secure settings migration: verifying that secure fields remain encrypted for all available notifiers and certain fields migrate from plain text to encrypted secure settings correctly e7d3993
Replacing the checks for custom dashboard ACLs will be replaced in a separate targeted PR as it will be complex enough alone.
* signing key wip
use db keyset storage
add signing_key table
add testing for key storage
add ES256 key tests
Remove caching and implement UpdateOrCreate
Stabilize interfaces
* Encrypt private keys
* Fixup signer
* Fixup ext_jwt
* Add GetOrCreatePrivate with automatic key rotation
* use GetOrCreate for ext_jwt
* use GetOrCreate in id
* catch invalid block type
* fix broken test
* remove key generator
* reduce public interface of signing service
* remove API tagging method and authed tagging
* add anonstore
move debug to after cache
change test order
fix issue where mysql trims to second
* add old device cleanup
lint
utc-ize everything
trim whitespace
* remove dangling setting
* Add delete devices
* Move anonymous authnclient to anonimpl
* Add simple post login hook
* move registration of Background Service
cleanup
* add updated_at index
* do not untag device if login err
* add delete device integration test
* Moving POC files from #64283 to a new branch
Co-authored-by: Mihály Gyöngyösi <mgyongyosi@users.noreply.github.com>
* Adding missing permission definition
Co-authored-by: Mihály Gyöngyösi <mgyongyosi@users.noreply.github.com>
* Force the service instantiation while client isn't merged
Co-authored-by: Mihály Gyöngyösi <mgyongyosi@users.noreply.github.com>
* Merge conf with main
Co-authored-by: Mihály Gyöngyösi <mgyongyosi@users.noreply.github.com>
* Leave go-sqlite3 version unchanged
Co-authored-by: Mihály Gyöngyösi <mgyongyosi@users.noreply.github.com>
* tidy
Co-authored-by: Mihály Gyöngyösi <mgyongyosi@users.noreply.github.com>
* User SearchUserPermissions instead of SearchUsersPermissions
* Replace DummyKeyService with signingkeys.Service
* Use user🆔<id> as subject
* Fix introspection endpoint issue
* Add X-Grafana-Org-Id to get_resources.bash script
* Regenerate toggles_gen.go
* Fix basic.go
* Add GetExternalService tests
* Add GetPublicKeyScopes tests
* Add GetScopesOnUser tests
* Add GetScopes tests
* Add ParsePublicKeyPem tests
* Add database test for GetByName
* re-add comments
* client tests added
* Add GetExternalServicePublicKey tests
* Add other test case to GetExternalServicePublicKey
* client_credentials grant test
* Add test to jwtbearer grant
* Test Comments
* Add handleKeyOptions tests
* Add RSA key generation test
* Add ECDSA by default to EmbeddedSigningKeysService
* Clean up org id scope and audiences
* Add audiences to the DB
* Fix check on Audience
* Fix double import
* Add AC Store mock and align oauthserver tests
* Fix test after rebase
* Adding missing store function to mock
* Fix double import
* Add CODEOWNER
* Fix some linting errors
* errors don't need type assertion
* Typo codeowners
* use mockery for oauthserver store
* Add feature toggle check
* Fix db tests to handle the feature flag
* Adding call to DeleteExternalServiceRole
* Fix flaky test
* Re-organize routes comments and plan futur work
* Add client_id check to Extended JWT client
* Clean up
* Fix
* Remove background service registry instantiation of the OAuth server
* Comment cleanup
* Remove unused client function
* Update go.mod to use the latest ory/fosite commit
* Remove oauth2_server related configs from defaults.ini
* Add audiences to DTO
* Fix flaky test
* Remove registration endpoint and demo scripts. Document code
* Rename packages
* Remove the OAuthService vs OAuthServer confusion
* fix incorrect import ext_jwt_test
* Comments and order
* Comment basic auth
* Remove unecessary todo
* Clean api
* Moving ParsePublicKeyPem to utils
* re ordering functions in service.go
* Fix comment
* comment on the redirect uri
* Add RBAC actions, not only scopes
* Fix tests
* re-import featuremgmt in migrations
* Fix wire
* Fix scopes in test
* Fix flaky test
* Remove todo, the intersection should always return the minimal set
* Remove unecessary check from intersection code
* Allow env overrides on settings
* remove the term app name
* Remove app keyword for client instead and use Name instead of ExternalServiceName
* LogID remove ExternalService ref
* Use Name instead of ExternalServiceName
* Imports order
* Inline
* Using ExternalService and ExternalServiceDTO
* Remove xorm tags
* comment
* Rename client files
* client -> external service
* comments
* Move test to correct package
* slimmer test
* cachedUser -> cachedExternalService
* Fix aggregate store test
* PluginAuthSession -> AuthSession
* Revert the nil cehcks
* Remove unecessary extra
* Removing custom session
* fix typo in test
* Use constants for tests
* Simplify HandleToken tests
* Refactor the HandleTokenRequest test
* test message
* Review test
* Prevent flacky test on client as well
* go imports
* Revert changes from 526e48ad45
* AuthN: Change the External Service registration form (#68649)
* AuthN: change the External Service registration form
* Gen default permissions
* Change demo script registration form
* Remove unecessary comment
* Nit.
* Reduce cyclomatic complexity
* Remove demo_scripts
* Handle case with no service account
* Comments
* Group key gen
* Nit.
* Check the SaveExternalService test
* Rename cachedUser to cachedClient in test
* One more test case to database test
* Comments
* Remove last org scope
Co-authored-by: Mihály Gyöngyösi <mgyongyosi@users.noreply.github.com>
* Update pkg/services/oauthserver/utils/utils_test.go
* Update pkg/services/sqlstore/migrations/oauthserver/migrations.go
Remove comment
* Update pkg/setting/setting.go
Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>
---------
Co-authored-by: Mihály Gyöngyösi <mgyongyosi@users.noreply.github.com>
* RBAC: Stop reading enabeld from ini file and always set to true
* Migrations: Add a migration for rbac to reset data migrations if rbac
was disabled
* If rbac was disabled we reset the data and data migrations that rbac
has to perform to get it to a correct state
* Migrator: Store migration logs on migrator and add function to clear it from the
in-memory stored logs
* update tests
---------
Co-authored-by: Karl Persson <kalle.persson@grafana.com>
* remove dashboard previews backend
* remove dashboard previews backend
* bring back the migration
* bring back the migration
* bring back the migration
* Remove URL-based alertmanagers from endpoint config
* WIP
* Add migration and alertmanagers from admin_configuration
* Empty comment removed
* set BasicAuth true when user is present in url
* Remove Alertmanagers from GET /admin_config payload
* Remove URL-based alertmanager configuration from UI
* Fix new uid generation in external alertmanagers migration
* Fix tests for URL-based external alertmanagers
* Fix API tests
* Add more tests, move migration code to separate file, and remove possible am duplicate urls
* Fix edge cases in migration
* Fix imports
* Remove useless fields and fix created_at/updated_at retrieval
Co-authored-by: George Robinson <george.robinson@grafana.com>
Co-authored-by: Konrad Lalik <konrad.lalik@grafana.com>
* add: added a repeat migration w. fixed checks for permissions
* add: migration to migrations
* refactor: fix migration instead of making a new one
* fix: removed the old id
* fix: keep old name but change id
* add: migration for patched previous migration
* add: migration from missing file
* feat: add new Folder table migration
Add a new folder table to support the Nested Folders feature.
https://github.com/grafana/grafana/issues/56880
* register nested folders feature flag (unused)
* feat: nested folder service (experiment)
This commit adds a NestedFolderSvc interface and stubbed out implementation as an alternative to the existing folder service. This is an experimental feature to try out different methods for backwards compatibility and parallelization, so that Grafana can continue to store folders in the existing (non-nested) manner while also using the new nested folder service.
Eventually the new service will (hopefully) become _the_ service, at which point the legacy service can be deprecated (or remain, with the new service methods replacing the original. whatever makes sense at the time).
* nested folders: don't run the new migration
This commit removes the nested folder migration from the list of active migrations so we can merge this branch and continue development without impacting Grafana instances built off main.
* add the migration
* Update pkg/services/sqlstore/migrations/accesscontrol/dashboard_permissions.go
Co-authored-by: Eric Leijonmarck <eric.leijonmarck@gmail.com>
Co-authored-by: Eric Leijonmarck <eric.leijonmarck@gmail.com>
Migrations:
* add a new column alert_group_idx to alert_rule table
* add a new column alert_group_idx to alert_rule_version table
* re-index existing rules during migration
API:
* set group index on update. Use the natural order of items in the array as group index
* sort rules in the group on GET
* update the version of all rules of all affected groups. This will make optimistic lock work in the case of multiple concurrent request touching the same groups.
UI:
* update UI to keep the order of alerts in a group
* backend/api: refactor PlaylistId to PlaylistUid
* Add org_id to Get and Update playlist functions
Fix migration - no longer pad the uid; fix mysql syntax
The relevant tests are passing using postgres, mysql and the default sqllite backends, but there are a number of other failing tests when using postgres and myself so I'm not entirely confident with those results.
* fix bad query in GetPlaylistItem and add a test that would have caught the mistake in the first place. Reverted the playlist_uid column addition in playlist_item; it became unnecessary after this PR.
Added default value to the new UID column based on PR feedback.
* break this PRs migration into its own function
* Playlists: Update UI to use the updated API
Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>
* update action names
* correctly retrieve teams for signed in user
* remove test
* undo swagger changes
* undo swagger changes pt2
* add migration from old action names to the new ones
* rename from list to read
* linting
* also update alertign actions
* fix migration
* AccessControl: Add migration for seeding managed inherited permissions
Co-authored-by: Karl Persson <kalle.persson@grafana.com>
* AccessControl: move to single file
* AccessControl: Add tests for managed permission migration
Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>
* AccessControl: Ensure no duplicate insertion
Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>
* Remove commented code
Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>
* Add code migration constant
Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>
* Ensure DB is clean between tests
Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>
* Update pkg/services/sqlstore/migrations/accesscontrol/managed_permission_migrator.go
Co-authored-by: Karl Persson <kalle.persson@grafana.com>
Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>
* #45498: add String util to ListResponse for better UX
* #45498: refactor db_filestorage FS API backend - use path_hash in DB schema
* #45498: enable DB backend fs api tests
* #45498: add comment
* #45498: enable Storage feature flag during integration tests
* remove fmt.println
* #45498: reduce sizes of hash columns
* separate conditions
* #45498: make it easy to ignore backends when running fs api integration tests
* #45498: quote `key` column name
* #45498: reduce path_hash size
* #45498: verify `/{orgId}/{storageName}/` prefix convention in integration tests
* #45498: add etag to the sql table
* #45498: add etag to the sql table
* remove feature flag check (storage isn't dev-mode only)
* add cacheControl and content disposition
* add comments
* add path_hash comment
* explicitly set `path` column collation in `file` table for postgres
* Add RBAC section to settings
* Default to RBAC enabled settings to true
* Update tests to respect RBAC
Co-authored-by: Karl Persson <kalle.persson@grafana.com>
