grafana/pkg/services/secrets/kvstore/kvstore.go

119 lines
4.0 KiB
Go
Raw Normal View History

Secrets: Implement basic unified secret store service (#45804) * wip: Implement kvstore for secrets * wip: Refactor kvstore for secrets * wip: Add format key function to secrets kvstore sql * wip: Add migration for secrets kvstore * Remove unused Key field from secrets kvstore * Remove secret values from debug logs * Integrate unified secrets with datasources * Fix minor issues and tests for kvstore * Create test service helper for secret store * Remove encryption tests from datasources * Move secret operations after datasources * Fix datasource proxy tests * Fix legacy data tests * Add Name to all delete data source commands * Implement decryption cache on sql secret store * Fix minor issue with cache and tests * Use secret type on secret store datasource operations * Add comments to make create and update clear * Rename itemFound variable to isFound * Improve secret deletion and cache management * Add base64 encoding to sql secret store * Move secret retrieval to decrypted values function * Refactor decrypt secure json data functions * Fix expr tests * Fix datasource tests * Fix plugin proxy tests * Fix query tests * Fix metrics api tests * Remove unused fake secrets service from query tests * Add rename function to secret store * Add check for error renaming secret * Remove bus from tests to fix merge conflicts * Add background secrets migration to datasources * Get datasource secure json fields from secrets * Move migration to secret store * Revert "Move migration to secret store" This reverts commit 7c3f872072e9aff601fb9d639127d468c03f97ef. * Add secret service to datasource service on tests * Fix datasource tests * Remove merge conflict on wire * Add ctx to data source http transport on prometheus stats collector * Add ctx to data source http transport on stats collector test
2022-04-25 11:57:45 -05:00
package kvstore
import (
"context"
"time"
Secrets: Implement basic unified secret store service (#45804) * wip: Implement kvstore for secrets * wip: Refactor kvstore for secrets * wip: Add format key function to secrets kvstore sql * wip: Add migration for secrets kvstore * Remove unused Key field from secrets kvstore * Remove secret values from debug logs * Integrate unified secrets with datasources * Fix minor issues and tests for kvstore * Create test service helper for secret store * Remove encryption tests from datasources * Move secret operations after datasources * Fix datasource proxy tests * Fix legacy data tests * Add Name to all delete data source commands * Implement decryption cache on sql secret store * Fix minor issue with cache and tests * Use secret type on secret store datasource operations * Add comments to make create and update clear * Rename itemFound variable to isFound * Improve secret deletion and cache management * Add base64 encoding to sql secret store * Move secret retrieval to decrypted values function * Refactor decrypt secure json data functions * Fix expr tests * Fix datasource tests * Fix plugin proxy tests * Fix query tests * Fix metrics api tests * Remove unused fake secrets service from query tests * Add rename function to secret store * Add check for error renaming secret * Remove bus from tests to fix merge conflicts * Add background secrets migration to datasources * Get datasource secure json fields from secrets * Move migration to secret store * Revert "Move migration to secret store" This reverts commit 7c3f872072e9aff601fb9d639127d468c03f97ef. * Add secret service to datasource service on tests * Fix datasource tests * Remove merge conflict on wire * Add ctx to data source http transport on prometheus stats collector * Add ctx to data source http transport on stats collector test
2022-04-25 11:57:45 -05:00
"github.com/grafana/grafana/pkg/infra/db"
Secrets: Implement Secret Plugin required flag and fatal crash on startup (#52552) * add special handling on the plugin gathering side to check whether secrets manager plugins are enabled or not * show disabled badge in front end if the plugin is not enabled * Only show error in disabled badge hover if one is present (otherwise it shows "undefined") * refactor to make use of fields already available in the DTO * fix typo * if there is no error returned for the plugin, just show 'disabled' * fix typo * Update public/app/features/plugins/admin/components/Badges/PluginDisabledBadge.tsx Co-authored-by: Levente Balogh <balogh.levente.hu@gmail.com> * Update frontendsettings.go add clarifying comment * fix unit test * rework task to use new frontend property combined with plugin type to determine if the plugin should be disabled * Update helpers.test.ts revert test change * fix unit test * show custom uninstall message if the plugin is a secrets manager * bogus commit to trigger precommit * undo commit * run precommit manually * add some consts * refactor a bit to pull plugin error management up a level * re-add code squashed in merge * fix compile issues * add code to set plugin error fatal flag after secret migration * refactor to move plugin startup out of Should Check func * re-add important check * make plugin startup errors fatal the first time we set a secret on the plugin * rename func to make intent clearler * remove unnecessary duplicate code from plugin mig * fix compile error * fix more compile errors * add some extra logging to secrets migration * have remote_plugin secret service managed plugin error fatal flag directly * add blank file for eventual unit tests * fix linting issues * changes from PR review * quick bit of cleanup * add comment explaining design decision * move more common test helpers to file * slightly update to first time Get secret call * add unit tests * remove override func from provider * fix linting issues * add test cleanup step * add some comments about refactoring to hacky test function Co-authored-by: Levente Balogh <balogh.levente.hu@gmail.com>
2022-07-25 11:37:47 -05:00
"github.com/grafana/grafana/pkg/infra/kvstore"
Secrets: Implement basic unified secret store service (#45804) * wip: Implement kvstore for secrets * wip: Refactor kvstore for secrets * wip: Add format key function to secrets kvstore sql * wip: Add migration for secrets kvstore * Remove unused Key field from secrets kvstore * Remove secret values from debug logs * Integrate unified secrets with datasources * Fix minor issues and tests for kvstore * Create test service helper for secret store * Remove encryption tests from datasources * Move secret operations after datasources * Fix datasource proxy tests * Fix legacy data tests * Add Name to all delete data source commands * Implement decryption cache on sql secret store * Fix minor issue with cache and tests * Use secret type on secret store datasource operations * Add comments to make create and update clear * Rename itemFound variable to isFound * Improve secret deletion and cache management * Add base64 encoding to sql secret store * Move secret retrieval to decrypted values function * Refactor decrypt secure json data functions * Fix expr tests * Fix datasource tests * Fix plugin proxy tests * Fix query tests * Fix metrics api tests * Remove unused fake secrets service from query tests * Add rename function to secret store * Add check for error renaming secret * Remove bus from tests to fix merge conflicts * Add background secrets migration to datasources * Get datasource secure json fields from secrets * Move migration to secret store * Revert "Move migration to secret store" This reverts commit 7c3f872072e9aff601fb9d639127d468c03f97ef. * Add secret service to datasource service on tests * Fix datasource tests * Remove merge conflict on wire * Add ctx to data source http transport on prometheus stats collector * Add ctx to data source http transport on stats collector test
2022-04-25 11:57:45 -05:00
"github.com/grafana/grafana/pkg/infra/log"
"github.com/grafana/grafana/pkg/plugins"
"github.com/grafana/grafana/pkg/plugins/backendplugin/secretsmanagerplugin"
Secrets: Implement Secret Plugin required flag and fatal crash on startup (#52552) * add special handling on the plugin gathering side to check whether secrets manager plugins are enabled or not * show disabled badge in front end if the plugin is not enabled * Only show error in disabled badge hover if one is present (otherwise it shows "undefined") * refactor to make use of fields already available in the DTO * fix typo * if there is no error returned for the plugin, just show 'disabled' * fix typo * Update public/app/features/plugins/admin/components/Badges/PluginDisabledBadge.tsx Co-authored-by: Levente Balogh <balogh.levente.hu@gmail.com> * Update frontendsettings.go add clarifying comment * fix unit test * rework task to use new frontend property combined with plugin type to determine if the plugin should be disabled * Update helpers.test.ts revert test change * fix unit test * show custom uninstall message if the plugin is a secrets manager * bogus commit to trigger precommit * undo commit * run precommit manually * add some consts * refactor a bit to pull plugin error management up a level * re-add code squashed in merge * fix compile issues * add code to set plugin error fatal flag after secret migration * refactor to move plugin startup out of Should Check func * re-add important check * make plugin startup errors fatal the first time we set a secret on the plugin * rename func to make intent clearler * remove unnecessary duplicate code from plugin mig * fix compile error * fix more compile errors * add some extra logging to secrets migration * have remote_plugin secret service managed plugin error fatal flag directly * add blank file for eventual unit tests * fix linting issues * changes from PR review * quick bit of cleanup * add comment explaining design decision * move more common test helpers to file * slightly update to first time Get secret call * add unit tests * remove override func from provider * fix linting issues * add test cleanup step * add some comments about refactoring to hacky test function Co-authored-by: Levente Balogh <balogh.levente.hu@gmail.com>
2022-07-25 11:37:47 -05:00
"github.com/grafana/grafana/pkg/services/featuremgmt"
Secrets: Implement basic unified secret store service (#45804) * wip: Implement kvstore for secrets * wip: Refactor kvstore for secrets * wip: Add format key function to secrets kvstore sql * wip: Add migration for secrets kvstore * Remove unused Key field from secrets kvstore * Remove secret values from debug logs * Integrate unified secrets with datasources * Fix minor issues and tests for kvstore * Create test service helper for secret store * Remove encryption tests from datasources * Move secret operations after datasources * Fix datasource proxy tests * Fix legacy data tests * Add Name to all delete data source commands * Implement decryption cache on sql secret store * Fix minor issue with cache and tests * Use secret type on secret store datasource operations * Add comments to make create and update clear * Rename itemFound variable to isFound * Improve secret deletion and cache management * Add base64 encoding to sql secret store * Move secret retrieval to decrypted values function * Refactor decrypt secure json data functions * Fix expr tests * Fix datasource tests * Fix plugin proxy tests * Fix query tests * Fix metrics api tests * Remove unused fake secrets service from query tests * Add rename function to secret store * Add check for error renaming secret * Remove bus from tests to fix merge conflicts * Add background secrets migration to datasources * Get datasource secure json fields from secrets * Move migration to secret store * Revert "Move migration to secret store" This reverts commit 7c3f872072e9aff601fb9d639127d468c03f97ef. * Add secret service to datasource service on tests * Fix datasource tests * Remove merge conflict on wire * Add ctx to data source http transport on prometheus stats collector * Add ctx to data source http transport on stats collector test
2022-04-25 11:57:45 -05:00
"github.com/grafana/grafana/pkg/services/secrets"
"github.com/grafana/grafana/pkg/setting"
Secrets: Implement basic unified secret store service (#45804) * wip: Implement kvstore for secrets * wip: Refactor kvstore for secrets * wip: Add format key function to secrets kvstore sql * wip: Add migration for secrets kvstore * Remove unused Key field from secrets kvstore * Remove secret values from debug logs * Integrate unified secrets with datasources * Fix minor issues and tests for kvstore * Create test service helper for secret store * Remove encryption tests from datasources * Move secret operations after datasources * Fix datasource proxy tests * Fix legacy data tests * Add Name to all delete data source commands * Implement decryption cache on sql secret store * Fix minor issue with cache and tests * Use secret type on secret store datasource operations * Add comments to make create and update clear * Rename itemFound variable to isFound * Improve secret deletion and cache management * Add base64 encoding to sql secret store * Move secret retrieval to decrypted values function * Refactor decrypt secure json data functions * Fix expr tests * Fix datasource tests * Fix plugin proxy tests * Fix query tests * Fix metrics api tests * Remove unused fake secrets service from query tests * Add rename function to secret store * Add check for error renaming secret * Remove bus from tests to fix merge conflicts * Add background secrets migration to datasources * Get datasource secure json fields from secrets * Move migration to secret store * Revert "Move migration to secret store" This reverts commit 7c3f872072e9aff601fb9d639127d468c03f97ef. * Add secret service to datasource service on tests * Fix datasource tests * Remove merge conflict on wire * Add ctx to data source http transport on prometheus stats collector * Add ctx to data source http transport on stats collector test
2022-04-25 11:57:45 -05:00
)
const (
// Wildcard to query all organizations
AllOrganizations = -1
)
Secrets: Implement Secret Plugin required flag and fatal crash on startup (#52552) * add special handling on the plugin gathering side to check whether secrets manager plugins are enabled or not * show disabled badge in front end if the plugin is not enabled * Only show error in disabled badge hover if one is present (otherwise it shows "undefined") * refactor to make use of fields already available in the DTO * fix typo * if there is no error returned for the plugin, just show 'disabled' * fix typo * Update public/app/features/plugins/admin/components/Badges/PluginDisabledBadge.tsx Co-authored-by: Levente Balogh <balogh.levente.hu@gmail.com> * Update frontendsettings.go add clarifying comment * fix unit test * rework task to use new frontend property combined with plugin type to determine if the plugin should be disabled * Update helpers.test.ts revert test change * fix unit test * show custom uninstall message if the plugin is a secrets manager * bogus commit to trigger precommit * undo commit * run precommit manually * add some consts * refactor a bit to pull plugin error management up a level * re-add code squashed in merge * fix compile issues * add code to set plugin error fatal flag after secret migration * refactor to move plugin startup out of Should Check func * re-add important check * make plugin startup errors fatal the first time we set a secret on the plugin * rename func to make intent clearler * remove unnecessary duplicate code from plugin mig * fix compile error * fix more compile errors * add some extra logging to secrets migration * have remote_plugin secret service managed plugin error fatal flag directly * add blank file for eventual unit tests * fix linting issues * changes from PR review * quick bit of cleanup * add comment explaining design decision * move more common test helpers to file * slightly update to first time Get secret call * add unit tests * remove override func from provider * fix linting issues * add test cleanup step * add some comments about refactoring to hacky test function Co-authored-by: Levente Balogh <balogh.levente.hu@gmail.com>
2022-07-25 11:37:47 -05:00
func ProvideService(
sqlStore db.DB,
Secrets: Implement Secret Plugin required flag and fatal crash on startup (#52552) * add special handling on the plugin gathering side to check whether secrets manager plugins are enabled or not * show disabled badge in front end if the plugin is not enabled * Only show error in disabled badge hover if one is present (otherwise it shows "undefined") * refactor to make use of fields already available in the DTO * fix typo * if there is no error returned for the plugin, just show 'disabled' * fix typo * Update public/app/features/plugins/admin/components/Badges/PluginDisabledBadge.tsx Co-authored-by: Levente Balogh <balogh.levente.hu@gmail.com> * Update frontendsettings.go add clarifying comment * fix unit test * rework task to use new frontend property combined with plugin type to determine if the plugin should be disabled * Update helpers.test.ts revert test change * fix unit test * show custom uninstall message if the plugin is a secrets manager * bogus commit to trigger precommit * undo commit * run precommit manually * add some consts * refactor a bit to pull plugin error management up a level * re-add code squashed in merge * fix compile issues * add code to set plugin error fatal flag after secret migration * refactor to move plugin startup out of Should Check func * re-add important check * make plugin startup errors fatal the first time we set a secret on the plugin * rename func to make intent clearler * remove unnecessary duplicate code from plugin mig * fix compile error * fix more compile errors * add some extra logging to secrets migration * have remote_plugin secret service managed plugin error fatal flag directly * add blank file for eventual unit tests * fix linting issues * changes from PR review * quick bit of cleanup * add comment explaining design decision * move more common test helpers to file * slightly update to first time Get secret call * add unit tests * remove override func from provider * fix linting issues * add test cleanup step * add some comments about refactoring to hacky test function Co-authored-by: Levente Balogh <balogh.levente.hu@gmail.com>
2022-07-25 11:37:47 -05:00
secretsService secrets.Service,
pluginsManager plugins.SecretsPluginManager,
Secrets: Implement Secret Plugin required flag and fatal crash on startup (#52552) * add special handling on the plugin gathering side to check whether secrets manager plugins are enabled or not * show disabled badge in front end if the plugin is not enabled * Only show error in disabled badge hover if one is present (otherwise it shows "undefined") * refactor to make use of fields already available in the DTO * fix typo * if there is no error returned for the plugin, just show 'disabled' * fix typo * Update public/app/features/plugins/admin/components/Badges/PluginDisabledBadge.tsx Co-authored-by: Levente Balogh <balogh.levente.hu@gmail.com> * Update frontendsettings.go add clarifying comment * fix unit test * rework task to use new frontend property combined with plugin type to determine if the plugin should be disabled * Update helpers.test.ts revert test change * fix unit test * show custom uninstall message if the plugin is a secrets manager * bogus commit to trigger precommit * undo commit * run precommit manually * add some consts * refactor a bit to pull plugin error management up a level * re-add code squashed in merge * fix compile issues * add code to set plugin error fatal flag after secret migration * refactor to move plugin startup out of Should Check func * re-add important check * make plugin startup errors fatal the first time we set a secret on the plugin * rename func to make intent clearler * remove unnecessary duplicate code from plugin mig * fix compile error * fix more compile errors * add some extra logging to secrets migration * have remote_plugin secret service managed plugin error fatal flag directly * add blank file for eventual unit tests * fix linting issues * changes from PR review * quick bit of cleanup * add comment explaining design decision * move more common test helpers to file * slightly update to first time Get secret call * add unit tests * remove override func from provider * fix linting issues * add test cleanup step * add some comments about refactoring to hacky test function Co-authored-by: Levente Balogh <balogh.levente.hu@gmail.com>
2022-07-25 11:37:47 -05:00
kvstore kvstore.KVStore,
features featuremgmt.FeatureToggles,
cfg *setting.Cfg,
Secrets: Implement Secret Plugin required flag and fatal crash on startup (#52552) * add special handling on the plugin gathering side to check whether secrets manager plugins are enabled or not * show disabled badge in front end if the plugin is not enabled * Only show error in disabled badge hover if one is present (otherwise it shows "undefined") * refactor to make use of fields already available in the DTO * fix typo * if there is no error returned for the plugin, just show 'disabled' * fix typo * Update public/app/features/plugins/admin/components/Badges/PluginDisabledBadge.tsx Co-authored-by: Levente Balogh <balogh.levente.hu@gmail.com> * Update frontendsettings.go add clarifying comment * fix unit test * rework task to use new frontend property combined with plugin type to determine if the plugin should be disabled * Update helpers.test.ts revert test change * fix unit test * show custom uninstall message if the plugin is a secrets manager * bogus commit to trigger precommit * undo commit * run precommit manually * add some consts * refactor a bit to pull plugin error management up a level * re-add code squashed in merge * fix compile issues * add code to set plugin error fatal flag after secret migration * refactor to move plugin startup out of Should Check func * re-add important check * make plugin startup errors fatal the first time we set a secret on the plugin * rename func to make intent clearler * remove unnecessary duplicate code from plugin mig * fix compile error * fix more compile errors * add some extra logging to secrets migration * have remote_plugin secret service managed plugin error fatal flag directly * add blank file for eventual unit tests * fix linting issues * changes from PR review * quick bit of cleanup * add comment explaining design decision * move more common test helpers to file * slightly update to first time Get secret call * add unit tests * remove override func from provider * fix linting issues * add test cleanup step * add some comments about refactoring to hacky test function Co-authored-by: Levente Balogh <balogh.levente.hu@gmail.com>
2022-07-25 11:37:47 -05:00
) (SecretsKVStore, error) {
var logger = log.New("secrets.kvstore")
var store SecretsKVStore
ctx := context.Background()
store = NewSQLSecretsKVStore(sqlStore, secretsService, logger)
err := EvaluateRemoteSecretsPlugin(ctx, pluginsManager, cfg)
if err != nil {
2022-09-01 12:47:37 -05:00
logger.Debug("secrets manager evaluator returned false", "reason", err.Error())
} else {
Secrets: Implement Secret Plugin required flag and fatal crash on startup (#52552) * add special handling on the plugin gathering side to check whether secrets manager plugins are enabled or not * show disabled badge in front end if the plugin is not enabled * Only show error in disabled badge hover if one is present (otherwise it shows "undefined") * refactor to make use of fields already available in the DTO * fix typo * if there is no error returned for the plugin, just show 'disabled' * fix typo * Update public/app/features/plugins/admin/components/Badges/PluginDisabledBadge.tsx Co-authored-by: Levente Balogh <balogh.levente.hu@gmail.com> * Update frontendsettings.go add clarifying comment * fix unit test * rework task to use new frontend property combined with plugin type to determine if the plugin should be disabled * Update helpers.test.ts revert test change * fix unit test * show custom uninstall message if the plugin is a secrets manager * bogus commit to trigger precommit * undo commit * run precommit manually * add some consts * refactor a bit to pull plugin error management up a level * re-add code squashed in merge * fix compile issues * add code to set plugin error fatal flag after secret migration * refactor to move plugin startup out of Should Check func * re-add important check * make plugin startup errors fatal the first time we set a secret on the plugin * rename func to make intent clearler * remove unnecessary duplicate code from plugin mig * fix compile error * fix more compile errors * add some extra logging to secrets migration * have remote_plugin secret service managed plugin error fatal flag directly * add blank file for eventual unit tests * fix linting issues * changes from PR review * quick bit of cleanup * add comment explaining design decision * move more common test helpers to file * slightly update to first time Get secret call * add unit tests * remove override func from provider * fix linting issues * add test cleanup step * add some comments about refactoring to hacky test function Co-authored-by: Levente Balogh <balogh.levente.hu@gmail.com>
2022-07-25 11:37:47 -05:00
// Attempt to start the plugin
var secretsPlugin secretsmanagerplugin.SecretsManagerPlugin
secretsPlugin, err = StartAndReturnPlugin(pluginsManager, ctx)
Secrets: Implement Secret Plugin required flag and fatal crash on startup (#52552) * add special handling on the plugin gathering side to check whether secrets manager plugins are enabled or not * show disabled badge in front end if the plugin is not enabled * Only show error in disabled badge hover if one is present (otherwise it shows "undefined") * refactor to make use of fields already available in the DTO * fix typo * if there is no error returned for the plugin, just show 'disabled' * fix typo * Update public/app/features/plugins/admin/components/Badges/PluginDisabledBadge.tsx Co-authored-by: Levente Balogh <balogh.levente.hu@gmail.com> * Update frontendsettings.go add clarifying comment * fix unit test * rework task to use new frontend property combined with plugin type to determine if the plugin should be disabled * Update helpers.test.ts revert test change * fix unit test * show custom uninstall message if the plugin is a secrets manager * bogus commit to trigger precommit * undo commit * run precommit manually * add some consts * refactor a bit to pull plugin error management up a level * re-add code squashed in merge * fix compile issues * add code to set plugin error fatal flag after secret migration * refactor to move plugin startup out of Should Check func * re-add important check * make plugin startup errors fatal the first time we set a secret on the plugin * rename func to make intent clearler * remove unnecessary duplicate code from plugin mig * fix compile error * fix more compile errors * add some extra logging to secrets migration * have remote_plugin secret service managed plugin error fatal flag directly * add blank file for eventual unit tests * fix linting issues * changes from PR review * quick bit of cleanup * add comment explaining design decision * move more common test helpers to file * slightly update to first time Get secret call * add unit tests * remove override func from provider * fix linting issues * add test cleanup step * add some comments about refactoring to hacky test function Co-authored-by: Levente Balogh <balogh.levente.hu@gmail.com>
2022-07-25 11:37:47 -05:00
namespacedKVStore := GetNamespacedKVStore(kvstore)
if err != nil || secretsPlugin == nil {
logger.Error("failed to start remote secrets management plugin")
if isFatal, readErr := IsPluginStartupErrorFatal(ctx, namespacedKVStore); isFatal || readErr != nil {
Secrets: Implement Secret Plugin required flag and fatal crash on startup (#52552) * add special handling on the plugin gathering side to check whether secrets manager plugins are enabled or not * show disabled badge in front end if the plugin is not enabled * Only show error in disabled badge hover if one is present (otherwise it shows "undefined") * refactor to make use of fields already available in the DTO * fix typo * if there is no error returned for the plugin, just show 'disabled' * fix typo * Update public/app/features/plugins/admin/components/Badges/PluginDisabledBadge.tsx Co-authored-by: Levente Balogh <balogh.levente.hu@gmail.com> * Update frontendsettings.go add clarifying comment * fix unit test * rework task to use new frontend property combined with plugin type to determine if the plugin should be disabled * Update helpers.test.ts revert test change * fix unit test * show custom uninstall message if the plugin is a secrets manager * bogus commit to trigger precommit * undo commit * run precommit manually * add some consts * refactor a bit to pull plugin error management up a level * re-add code squashed in merge * fix compile issues * add code to set plugin error fatal flag after secret migration * refactor to move plugin startup out of Should Check func * re-add important check * make plugin startup errors fatal the first time we set a secret on the plugin * rename func to make intent clearler * remove unnecessary duplicate code from plugin mig * fix compile error * fix more compile errors * add some extra logging to secrets migration * have remote_plugin secret service managed plugin error fatal flag directly * add blank file for eventual unit tests * fix linting issues * changes from PR review * quick bit of cleanup * add comment explaining design decision * move more common test helpers to file * slightly update to first time Get secret call * add unit tests * remove override func from provider * fix linting issues * add test cleanup step * add some comments about refactoring to hacky test function Co-authored-by: Levente Balogh <balogh.levente.hu@gmail.com>
2022-07-25 11:37:47 -05:00
// plugin error was fatal or there was an error determining if the error was fatal
logger.Error("secrets management plugin is required to start -- exiting app")
if readErr != nil {
return nil, readErr
Secrets: Implement Secret Plugin required flag and fatal crash on startup (#52552) * add special handling on the plugin gathering side to check whether secrets manager plugins are enabled or not * show disabled badge in front end if the plugin is not enabled * Only show error in disabled badge hover if one is present (otherwise it shows "undefined") * refactor to make use of fields already available in the DTO * fix typo * if there is no error returned for the plugin, just show 'disabled' * fix typo * Update public/app/features/plugins/admin/components/Badges/PluginDisabledBadge.tsx Co-authored-by: Levente Balogh <balogh.levente.hu@gmail.com> * Update frontendsettings.go add clarifying comment * fix unit test * rework task to use new frontend property combined with plugin type to determine if the plugin should be disabled * Update helpers.test.ts revert test change * fix unit test * show custom uninstall message if the plugin is a secrets manager * bogus commit to trigger precommit * undo commit * run precommit manually * add some consts * refactor a bit to pull plugin error management up a level * re-add code squashed in merge * fix compile issues * add code to set plugin error fatal flag after secret migration * refactor to move plugin startup out of Should Check func * re-add important check * make plugin startup errors fatal the first time we set a secret on the plugin * rename func to make intent clearler * remove unnecessary duplicate code from plugin mig * fix compile error * fix more compile errors * add some extra logging to secrets migration * have remote_plugin secret service managed plugin error fatal flag directly * add blank file for eventual unit tests * fix linting issues * changes from PR review * quick bit of cleanup * add comment explaining design decision * move more common test helpers to file * slightly update to first time Get secret call * add unit tests * remove override func from provider * fix linting issues * add test cleanup step * add some comments about refactoring to hacky test function Co-authored-by: Levente Balogh <balogh.levente.hu@gmail.com>
2022-07-25 11:37:47 -05:00
}
return nil, err
}
} else {
// as the plugin is installed, SecretsKVStoreSQL is now replaced with
// an instance of SecretsKVStorePlugin with the sql store as a fallback
// (used for migration and in case a secret is not found).
store = NewPluginSecretsKVStore(secretsPlugin, secretsService, namespacedKVStore, features, WithCache(store, 5*time.Second, 5*time.Minute), logger)
}
}
if err != nil {
logger.Debug("secrets kvstore is using the default (SQL) implementation for secrets management")
}
Secrets: Implement Secret Plugin required flag and fatal crash on startup (#52552) * add special handling on the plugin gathering side to check whether secrets manager plugins are enabled or not * show disabled badge in front end if the plugin is not enabled * Only show error in disabled badge hover if one is present (otherwise it shows "undefined") * refactor to make use of fields already available in the DTO * fix typo * if there is no error returned for the plugin, just show 'disabled' * fix typo * Update public/app/features/plugins/admin/components/Badges/PluginDisabledBadge.tsx Co-authored-by: Levente Balogh <balogh.levente.hu@gmail.com> * Update frontendsettings.go add clarifying comment * fix unit test * rework task to use new frontend property combined with plugin type to determine if the plugin should be disabled * Update helpers.test.ts revert test change * fix unit test * show custom uninstall message if the plugin is a secrets manager * bogus commit to trigger precommit * undo commit * run precommit manually * add some consts * refactor a bit to pull plugin error management up a level * re-add code squashed in merge * fix compile issues * add code to set plugin error fatal flag after secret migration * refactor to move plugin startup out of Should Check func * re-add important check * make plugin startup errors fatal the first time we set a secret on the plugin * rename func to make intent clearler * remove unnecessary duplicate code from plugin mig * fix compile error * fix more compile errors * add some extra logging to secrets migration * have remote_plugin secret service managed plugin error fatal flag directly * add blank file for eventual unit tests * fix linting issues * changes from PR review * quick bit of cleanup * add comment explaining design decision * move more common test helpers to file * slightly update to first time Get secret call * add unit tests * remove override func from provider * fix linting issues * add test cleanup step * add some comments about refactoring to hacky test function Co-authored-by: Levente Balogh <balogh.levente.hu@gmail.com>
2022-07-25 11:37:47 -05:00
return WithCache(store, 5*time.Second, 5*time.Minute), nil
Secrets: Implement basic unified secret store service (#45804) * wip: Implement kvstore for secrets * wip: Refactor kvstore for secrets * wip: Add format key function to secrets kvstore sql * wip: Add migration for secrets kvstore * Remove unused Key field from secrets kvstore * Remove secret values from debug logs * Integrate unified secrets with datasources * Fix minor issues and tests for kvstore * Create test service helper for secret store * Remove encryption tests from datasources * Move secret operations after datasources * Fix datasource proxy tests * Fix legacy data tests * Add Name to all delete data source commands * Implement decryption cache on sql secret store * Fix minor issue with cache and tests * Use secret type on secret store datasource operations * Add comments to make create and update clear * Rename itemFound variable to isFound * Improve secret deletion and cache management * Add base64 encoding to sql secret store * Move secret retrieval to decrypted values function * Refactor decrypt secure json data functions * Fix expr tests * Fix datasource tests * Fix plugin proxy tests * Fix query tests * Fix metrics api tests * Remove unused fake secrets service from query tests * Add rename function to secret store * Add check for error renaming secret * Remove bus from tests to fix merge conflicts * Add background secrets migration to datasources * Get datasource secure json fields from secrets * Move migration to secret store * Revert "Move migration to secret store" This reverts commit 7c3f872072e9aff601fb9d639127d468c03f97ef. * Add secret service to datasource service on tests * Fix datasource tests * Remove merge conflict on wire * Add ctx to data source http transport on prometheus stats collector * Add ctx to data source http transport on stats collector test
2022-04-25 11:57:45 -05:00
}
// SecretsKVStore is an interface for k/v store.
type SecretsKVStore interface {
Get(ctx context.Context, orgId int64, namespace string, typ string) (string, bool, error)
Set(ctx context.Context, orgId int64, namespace string, typ string, value string) error
Del(ctx context.Context, orgId int64, namespace string, typ string) error
Keys(ctx context.Context, orgId int64, namespace string, typ string) ([]Key, error)
Rename(ctx context.Context, orgId int64, namespace string, typ string, newNamespace string) error
GetAll(ctx context.Context) ([]Item, error)
Secrets: Implement basic unified secret store service (#45804) * wip: Implement kvstore for secrets * wip: Refactor kvstore for secrets * wip: Add format key function to secrets kvstore sql * wip: Add migration for secrets kvstore * Remove unused Key field from secrets kvstore * Remove secret values from debug logs * Integrate unified secrets with datasources * Fix minor issues and tests for kvstore * Create test service helper for secret store * Remove encryption tests from datasources * Move secret operations after datasources * Fix datasource proxy tests * Fix legacy data tests * Add Name to all delete data source commands * Implement decryption cache on sql secret store * Fix minor issue with cache and tests * Use secret type on secret store datasource operations * Add comments to make create and update clear * Rename itemFound variable to isFound * Improve secret deletion and cache management * Add base64 encoding to sql secret store * Move secret retrieval to decrypted values function * Refactor decrypt secure json data functions * Fix expr tests * Fix datasource tests * Fix plugin proxy tests * Fix query tests * Fix metrics api tests * Remove unused fake secrets service from query tests * Add rename function to secret store * Add check for error renaming secret * Remove bus from tests to fix merge conflicts * Add background secrets migration to datasources * Get datasource secure json fields from secrets * Move migration to secret store * Revert "Move migration to secret store" This reverts commit 7c3f872072e9aff601fb9d639127d468c03f97ef. * Add secret service to datasource service on tests * Fix datasource tests * Remove merge conflict on wire * Add ctx to data source http transport on prometheus stats collector * Add ctx to data source http transport on stats collector test
2022-04-25 11:57:45 -05:00
}
// WithType returns a kvstore wrapper with fixed orgId and type.
func With(kv SecretsKVStore, orgId int64, namespace string, typ string) *FixedKVStore {
return &FixedKVStore{
kvStore: kv,
OrgId: orgId,
Namespace: namespace,
Type: typ,
}
}
// FixedKVStore is a SecretsKVStore wrapper with fixed orgId, namespace and type.
type FixedKVStore struct {
kvStore SecretsKVStore
OrgId int64
Namespace string
Type string
}
func (kv *FixedKVStore) Get(ctx context.Context) (string, bool, error) {
return kv.kvStore.Get(ctx, kv.OrgId, kv.Namespace, kv.Type)
}
func (kv *FixedKVStore) Set(ctx context.Context, value string) error {
return kv.kvStore.Set(ctx, kv.OrgId, kv.Namespace, kv.Type, value)
}
func (kv *FixedKVStore) Del(ctx context.Context) error {
return kv.kvStore.Del(ctx, kv.OrgId, kv.Namespace, kv.Type)
}
func (kv *FixedKVStore) Keys(ctx context.Context) ([]Key, error) {
return kv.kvStore.Keys(ctx, kv.OrgId, kv.Namespace, kv.Type)
}
func (kv *FixedKVStore) Rename(ctx context.Context, newNamespace string) error {
err := kv.kvStore.Rename(ctx, kv.OrgId, kv.Namespace, kv.Type, newNamespace)
if err != nil {
return err
}
kv.Namespace = newNamespace
return nil
}