* Add disabled column to Org's Users table
* fix typo
* Change column order
* Add test for testing whether GetOrgUsers populates the DTO correctly
* Remove type assertion
* Correlations: Get Single correlations
* Correlations: Get all correlations for given source ds
* Correlations: Get all correlations
* add tests
* add DB indices
* fix lint errors
* remove skip from tests
* use DatasourceService in test
* Chore: Add user service method SetUsingOrg
* Chore: Add user service method GetSignedInUserWithCacheCtx
* Use method GetSignedInUserWithCacheCtx from user service
* Fix lint after rebase
* Fix lint
* Fix lint error
* roll back some changes
* Roll back changes in api and middleware
* Add xorm tags to SignedInUser ID fields
* Move SignedInUser to user service and RoleType and Roles to org
* Use go naming convention for roles
* Fix some imports and leftovers
* Fix ldap debug test
* Fix lint
* Fix lint 2
* Fix lint 3
* Fix type and not needed conversion
* Clean up messages in api tests
* Clean up api tests 2
* Add wrapper around sqlstore method GetUserByLogin
* Use new method from user service
* Fix lint
* Fix lint 2
* fix middleware basic auth test
* Fix grafana login returning a user by login
* Remove GetUserByLogin from store interface
* Merge commit
* move apikey store into a separate service
* add apikey service to wire graph
* fix linter
* switch api to use apikey service
* fix provideservice in tests
* add apikey service test double
* try different sql syntax
* rolling back the dialect
* trigger drone
* trigger drone
* Previews: datasource permissions
* lint
* simplify - force non-null `ds_uids`
* add `canBeDisabled` to search service
* add `IncludeThumbnailsWithEmptyDsUids`
* remove force refresh migration
* refactor main preview service
* add safeguard
* revert ticker interval
* update testdata
* fix test
* add mock search service
* add datasources lookup test
* update migration
* extract ds lookup to its own package to avoid cyclic imports
* lint
* fix dashbaord extract, use the real datasource lookup in tests. IS IT BULLETPROOF YET?!
* fix dashbaord extract, use the real datasource lookup in tests. IS IT BULLETPROOF YET?!
* remove stale log
* consistent casing
* pass context to `createServiceAccount`
* filter out the special grafana ds
* Move user not found err to user service
* User ErrCaseInsensitive from user pkg
* User ErrUserAlreadyExists from user pkg
* User ErrLastGrafanaAdmin from user pkg
* Remove errors from model
* add the migration
* Update pkg/services/sqlstore/migrations/accesscontrol/dashboard_permissions.go
Co-authored-by: Eric Leijonmarck <eric.leijonmarck@gmail.com>
Co-authored-by: Eric Leijonmarck <eric.leijonmarck@gmail.com>
* SQLstore: Fix fetching and deleting an inexistent playlist
xorm's session.Get() does not return an error if the raw does not exist.
It returns a boolean instead.
The playlist `sqlstore.GetPlaylist()` used to check only the error and in case
of inexistent UID didn't return an error.
* Implement disableSecretsCompatibility flag
* Allow secret deletion right after migration
* Use dialect.Quote for secure_json_data on secret deletion
Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>
* Set secure_json_data to NULL instead of empty json
* Run toggles_gen_test and use generated flag variable
* Add ID to delete data source secrets command on function call
Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>
* Remove extra query to get datasource on secret deletion
* Fix linting issues with CHANGELOG.md
* Use empty json string when deleting secure json data
* Implement secret migration as a background process
* Refactor secret migration as a background service
* Refactor migration to be inside secret store
* Re-add secret deletion function removed on merge
* Try using transaction to fix db lock during tests
* Disable migration for pipeline debugging
* Try adding sleep to fix database lock
* Remove unecessary time sleep from migration
* Fix merge issue, replace models with datasources
* Try event listener approach
* Fix merge issue, replace models with datasources
* Fix linting issues with unchecked error
* Remove unecessary trainling new line
* Increase wait interval on background secret migration
* Rename secret store migration folder for consistency
* Convert background migration to blocking
* Fix number of arguments on server tests
* Check error value of secret migration provider
* Fix linting issue with method varaible
* Revert unintended change on background services
* Move secret migration service provider to wire.go
* Remove unecessary else from datasource service
* Move transaction inside loop on secret migration
* Remove unecessary GetServices function
* Remove unecessary interface after method removal
* Rename Run to Migrate on secret migration interface
* Rename secret migrations service variable on server
* Use MustBool on datasource secret migration
* Revert changes to GetDataSources
* Implement GetAllDataSources function
* Remove DeleteDataSourceSecrets function
* Move datasource secret migration to datasource service
* Remove unecessary properties from datasource secret migration
* Make DecryptLegacySecrets a private method
* Remove context canceled check on secret migrator
* Log error when fail to unmarshal datasource secret
* Add necessary fields to update command on migration
* Handle high availability on secret migration
* Use kvstore for datasource secret migration status
* Add error check for migration status set on kvstore
* Remove NewSecretMigrationService from server tests
* Use const for strings on datasource secrets migration
* Test all cases for datasources secret migrations
Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>
* Alerting: Update default route groupBy to [grafana_folder, alertname]
Default group by for new routes and migrations is now [grafana_folder, alertname]
* Org: use constants for status codes
* ServiceAccounts: Avoid creating new orgs for service accounts
* Document createUserBehavior
* Update pkg/services/sqlstore/org_users_test.go
* add doc string to flag
* Revert "Serviceaccounts: #48995
Do not display service accounts assigned to team (#48995)"
This reverts commit cbf71fbd7f.
* fix: test to not include more actions than necessary
* adding service accounts to teams - backend and frontend changes
* also support SA addition through the old team membership endpoints
* fix tests
* tests
* serviceaccounts permission tests
* serviceaccounts permission service tests run
* added back test that was removed by accident
* lint
* refactor: add testoptionsTeams
* fix a bug
* service account picker change
* explicitly set SA managed permissions to false for dash and folders
* lint
* allow team creator to list service accounts
Co-authored-by: IevaVasiljeva <ieva.vasiljeva@grafana.com>
* UsageStats: fixed elasticsearch version number
- The version numbering was changed from plain numbers to a semver-ish approach
* added missing version assertion
* adapted tests
* chore/backend: move dashboard errors to dashboard service
Dashboard-related models are slowly moving out of the models package and into dashboard services. This commit moves dashboard-related errors; the rest will come in later commits.
There are no logical code changes, this is only a structural (package) move.
* lint lint lint
* Split Create User
* Use new create user and User from package user
* Add service to wire
* Making create user work
* Replace user from user pkg
* One more
* Move Insert to orguser Service/Store
* Remove unnecessary conversion
* Cleaunp
* Fix Get User and add fakes
* Fixing get org id for user logic, adding fakes and other adjustments
* Add some tests for ourguser service and store
* Fix insert org logic
* Add comment about deprecation
* Fix after merge with main
* Move orguser service/store to org service/store
* Remove orguser from wire
* Unimplement new Create user and use User from pkg user
* Fix wire generation
* Fix lint
* Fix lint - use only User and CrateUserCommand from user pkg
* Remove User and CreateUserCommand from models
* Fix lint 2
* add case insensitive option
* treat id as case insensitive
* Users: Add integration tests for case insensitive querying
* Prefer config struct to global variable
* change key to case_insensitive_login
* impede conflicting users from logging in
* add tests for impeding user retrieval if conflicting
* nits and picks
Co-authored-by: gamab <gabi.mabs@gmail.com>
* Add check in transaction for conflicting user
Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>
* add update tests
* skip on mysql
* add custom messages for user admin view
Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>
* nit: extra else
* linting mistake
Co-authored-by: gamab <gabi.mabs@gmail.com>
Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>
This PR completes public dashboards v1 functionality and simplifies public dashboard conventions. It exists as a large PR so that we are not making constant changes to the database schema.
models.PublicDashboardConfig model replaced with models.PublicDashboard directly
dashboard_public_config table renamed to dashboard_public
models.Dashboard.IsPublic removed from the dashboard and replaced with models.PublicDashboard.isEnabled
Routing now uses a uuid v4 as an access token for viewing a public dashboard anonymously, PublicDashboard.Uid only used as database identifier
Frontend utilizes uuid for auth'd operations and access token for anonymous access
Default to time range defined on dashboard when viewing public dashboard
Add audit fields to public dashboard
Co-authored-by: Owen Smallwood <owen.smallwood@grafana.com>, Ezequiel Victorero <ezequiel.victorero@grafana.com>, Jesse Weaver <jesse.weaver@grafana.com>
* Alerting: Don't stop the migration when alert rule tags are invalid
As we migrate we expect the `alertRuleTags` on a dashboard alert to be a JSON object. However, it seems this is not really validated by Grafana and an user can change the format to something else that the JSON parser is not able to marshal into a `map[string]string`.
Let's do a bit better by "attempting" to parse the tags and if we can't we'll simple return an empty map. The data is still there so if the user wishes they can go back, fix the data and attemp the migration again.
Migrations:
* add a new column alert_group_idx to alert_rule table
* add a new column alert_group_idx to alert_rule_version table
* re-index existing rules during migration
API:
* set group index on update. Use the natural order of items in the array as group index
* sort rules in the group on GET
* update the version of all rules of all affected groups. This will make optimistic lock work in the case of multiple concurrent request touching the same groups.
UI:
* update UI to keep the order of alerts in a group
* WIP
* update for prometheus
* usagestats: tests pass for user duplicate entries
* metrics: added duplicate user entries
* usagestats: adds metrics gauge for duplicate users
* usagestats: skip test for mysql
* sql in oneplace
* only use prometheus register to not panic
* usagestats: RegisterMetricsFunc with loginstats
* fix: remove unused commited code
* refactor: move test to authinfoservice
* Update pkg/models/stats.go
Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>
* removed prometheus metrics, due to sql cpu requirement
* Added: has_duplicate_user_entries and fix tests
* remove unused test
* fix: empty else statement removal
* missing argument to authinfoservice
Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>
* backend/dashboard snapshots: refactor leftover models and mocks
* Move all dashboard snapshot-related models into the dashboardsnapshotservice package
* Remove leftover dashboard-related mocks from the mockstore
* Add protobuf config and generated code, and client wrapper
* wire up loading of secretsmanager plugin, using renderer plugin as a model
* update kvstore provider to check if we should use the grpc plugin. return false always in OSS
* add OSS remote plugin check
* refactor wire gen file
* log which secrets manager is being used
* Fix argument types for remote checker
* Turns out if err != nil, then the result is always nil. Return empty values if there is an error.
* remove duplicate import
* ensure atomicity by adding secret management as a step to sql operations and rolling back if necessary
* Update pkg/services/secrets/kvstore/kvstore.go
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
* Update pkg/services/secrets/kvstore/kvstore.go
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
* refactor RemotePluginCheck interface to just return the Plugin client directly
* rename struct to something less silly
* add special error handling for remote secrets management
* switch to errors.as instead of type inference
* remove unnecessary rollback call
* just declare error once
* refactor .proto file according to prior PR suggestions
* re-generate protobuf files and fix compilation errors
* only wrap (ergo display in the front end) errors that are user friendly from the plugin
* rename error type to suggest user friendly only
* rename plugin functions to be more descriptive
* change delete message name
* Revert "change delete message name"
This reverts commit 8ca978301e.
* Revert "rename plugin functions to be more descriptive"
This reverts commit 4355c9b9ff.
* fix pointer to pointer problem
* change plugin user error to just hold a string
* fix sequencing problem with datasource updates
* clean up some return statements
* need to wrap multiple transactions with the InTransaction() func in order to keep the lock
* make linter happy
* revert input var name
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
* ServiceAccounts: able to get upgrade status
* Banner with API keys migration info
* Show API keys migration info on Service accounts page
* Migrate individual API keys
* Use transaction for key migration
* Migrate all api keys to service accounts
* Hide api keys after migration
* Migrate API keys separately for each org
* Revert API key
* Revert key API method
* Rename migration actions and reducers
* Fix linter errors
* Tests for migrating single API key
* Tests for migrating all api keys
* More tests
* Fix reverting tokens
* API: rename convert to migrate
* Add api route descriptions to methods
* rearrange methods in api.go
* Refactor: rename and move some methods
* Prevent assigning tokens to non-existing service accounts
* Refactor: ID TO Id
* Refactor: fix error message
* Delete service account if migration failed
* Fix linter errors
* backend/api: refactor PlaylistId to PlaylistUid
* Add org_id to Get and Update playlist functions
Fix migration - no longer pad the uid; fix mysql syntax
The relevant tests are passing using postgres, mysql and the default sqllite backends, but there are a number of other failing tests when using postgres and myself so I'm not entirely confident with those results.
* fix bad query in GetPlaylistItem and add a test that would have caught the mistake in the first place. Reverted the playlist_uid column addition in playlist_item; it became unnecessary after this PR.
Added default value to the new UID column based on PR feedback.
* break this PRs migration into its own function
* Playlists: Update UI to use the updated API
Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>
* backend/sqlstore split: move dashboard snapshot funcs to dashboardsnapshotservice
This commit moves the dashboard snapshot related sql functions in the dashboardsnapshots service. I split the dashboards package up so the interfaces live in dashboarsnapshots and the store and service implementations are in their own packages. This took some minor refactoring, but none of the actual underlying code has changed, just where it lives.
* Chore: Exclude integration tests from running on test-backend step
* Remove -v from go test command
* Add check to skip integration tests before each integration test
* Try to restart pipeline
* Retrying to make pipeline run
* Make inheritance permission migration more robust
* Better fix
* Add more tests to the migration
* Add removed test case
* Add test case for empty role in empty org
* Handling the role.ID 0 case with a log
Co-authored-by: Jguer <joao.guerreiro@grafana.com>
Co-authored-by: Jguer <joao.guerreiro@grafana.com>
* Alerting: Fix force_migration when alerting is disabled
This commit fixes a bug where force_migration must be set to true
when both unified and legacy alerting is disabled.
* Update comment
* Fix typo in comment
Co-authored-by: Armand Grillet <armand.grillet@outlook.com>
After migrating to unified alerting, users must explicitly allow rolling
back to legacy alerting by setting force_migration = true in config.
This updates the panic message to clarify why that's required and what
the consequences of rolling back will be.
Fixes#50469
* Remove dashbpard version from models
* Fix lint
* Fix api & sqlstore tests
* Remove integration tags
* Fix lint again
* Add integration test to correct namespace
* Lont fix 2
* Change Id to ID in dashVersionMeta
* change migration logic
* linting
* linting
* fix an issue with the migration logic
* make tests runnable against other DBs
Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>
Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>
* Add migration
* Migrator: Extend support to rename columns
* Fix getting current key
* Fix column name in migration
* Fix deks reencryption
* Fix caching
* Add back separate caches for byName and byPrefix
* Do not concatenate prefix with uid
* Rename DataKey struc fields
* SQLStore: Add deprecation comments for breaking migrations
* Add comment
* Minor corrections
Co-authored-by: Joan López de la Franca Beltran <joanjan14@gmail.com>
This PR adds endpoints for saving and retrieving a public dashboard configuration and and api endpoint to retrieve the public dashboard.
All of this is highly experimental and APIs will change. Notably, we will be removing isPublic from the dashboard model and moving it over to the public dashboard table in the next release.
Further context can be found here: https://github.com/grafana/grafana/pull/49131#issuecomment-1145456952
* Split GetDashboarVersions method
* Add sqlstore dialect and tests
* Fix signature of PAtchPreference
* Add GetDialect to sqlstore and remove GetDashboardVersions
* Add GetDialect to db interface
* Implement List
* add deleted test function
* Remove GetDialect from sqlstore interface
* Remove deleted method from mock
* Refactor test
* update action names
* correctly retrieve teams for signed in user
* remove test
* undo swagger changes
* undo swagger changes pt2
* add migration from old action names to the new ones
* rename from list to read
* linting
* also update alertign actions
* fix migration
* backend/sqlstore split: remove unused GetDashboardPermissionsForUser from sqlstore
* remove debugging line
* backend/sqlstore: move dashboard permission related functions to dashboard service
* Add store split for Get Dashboard version method
* Implement dashboard version service
* Fix api tests
* Remove GetDashboarVersion from sqlstore
* Add fakes for Get dashboard version
* Fix sqlstore test
* Add Get Dashboard store test
* Add dashver service test
* Remove useless comments
* chore: replace artisnal FakeDashboardService with generated mock
Maintaining a handcrafted FakeDashboardService is not sustainable now that we are in the process of moving the dashboard-related functions out of sqlstore.
* sqlstore: finish removing Find and SearchDashboards
Find and SearchDashboards were previously copied into the dashboard service. This commit completes that work, removing Find and SearchDashboards from the sqlstore and updating callers to use the dashboard service.
* dashboards: remove SearchDashboards from Store interface
SearchDashboards is a wrapper around FindDashboard that transforms the results, so it's been moved out of the Store entirely and the functionality moved into the Dashboard Service's search implementation.
The database tests depended heavily on the transformation, so I added testSearchDashboards, a copy of search dashboards, instead of (heavily) refactoring all the tests.
* Add IsServiceAccount to query
* test for excluding service accounts in query
* align test
* test update
* added logging
* should fix tests
* moved test to bottom
* add back setup function
* Remove IsServiceAccount from query
Instead use false for is_service_account
and when the time comes to implement service accounts
for team. we deal with it
* missed teamtest
* formatting
* AccessControl: Add migration for seeding managed inherited permissions
Co-authored-by: Karl Persson <kalle.persson@grafana.com>
* AccessControl: move to single file
* AccessControl: Add tests for managed permission migration
Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>
* AccessControl: Ensure no duplicate insertion
Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>
* Remove commented code
Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>
* Add code migration constant
Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>
* Ensure DB is clean between tests
Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>
* Update pkg/services/sqlstore/migrations/accesscontrol/managed_permission_migrator.go
Co-authored-by: Karl Persson <kalle.persson@grafana.com>
Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>
* Add prefixed API key gen.
* Retrieve API Key by hash
* Handle prefixed API keys for login
* Add placeholder key generator
* fix spelling
* add get by hash sqlstore test
* reformat query
* quote usage of reserved keyword key
* use constant
* improve error handling and pre-select key type
Co-authored-by: Victor Cinaglia <victor@grafana.com>
* nits
Co-authored-by: Victor Cinaglia <victor@grafana.com>
* Add database migrations
* Use short uids as data key ids
* Add support for manual data key rotation
* Fix duplicated mutex unlocks
* Fix migration
* Manage current data keys per name
* Adjust key re-encryption and test
* Modify rename column migration for MySQL compatibility
* Refactor secrets manager and data keys cache
* Multiple o11y adjustments
* Fix stats query
* Apply suggestions from code review
Co-authored-by: Tania <yalyna.ts@gmail.com>
* Fix linter
* Docs: Rotate data encryption keys API endpoint
Co-authored-by: Tania <yalyna.ts@gmail.com>
This change adds a field to state.State and models.AlertInstance
that indicate the "Reason" that an instance has its current state. This
helps us account for cases where the state is "Normal" but the
underlying evaluation returned "NoData" or "Error", for example.
Fixes#42606
Signed-off-by: Joe Blubaugh <joe.blubaugh@grafana.com>
This change extracts screenshot data from alert messages via a private annotation `__alertScreenshotToken__` and attaches a URL to a Slack message or uploads the data to an image upload endpoint if needed.
This change also implements a few foundational functions for use in other notifiers.
