grafana/pkg/tsdb/mssql/mssql.go

280 lines
7.6 KiB
Go
Raw Normal View History

2017-12-02 05:40:12 -06:00
package mssql
import (
"context"
"encoding/json"
2017-12-02 05:40:12 -06:00
"fmt"
"net/url"
"reflect"
"regexp"
"strconv"
"strings"
2017-12-02 05:40:12 -06:00
Encryption: Refactor securejsondata.SecureJsonData to stop relying on global functions (#38865) * Encryption: Add support to encrypt/decrypt sjd * Add datasources.Service as a proxy to datasources db operations * Encrypt ds.SecureJsonData before calling SQLStore * Move ds cache code into ds service * Fix tlsmanager tests * Fix pluginproxy tests * Remove some securejsondata.GetEncryptedJsonData usages * Add pluginsettings.Service as a proxy for plugin settings db operations * Add AlertNotificationService as a proxy for alert notification db operations * Remove some securejsondata.GetEncryptedJsonData usages * Remove more securejsondata.GetEncryptedJsonData usages * Fix lint errors * Minor fixes * Remove encryption global functions usages from ngalert * Fix lint errors * Minor fixes * Minor fixes * Remove securejsondata.DecryptedValue usage * Refactor the refactor * Remove securejsondata.DecryptedValue usage * Move securejsondata to migrations package * Move securejsondata to migrations package * Minor fix * Fix integration test * Fix integration tests * Undo undesired changes * Fix tests * Add context.Context into encryption methods * Fix tests * Fix tests * Fix tests * Trigger CI * Fix test * Add names to params of encryption service interface * Remove bus from CacheServiceImpl * Add logging * Add keys to logger Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com> * Add missing key to logger Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com> * Undo changes in markdown files * Fix formatting * Add context to secrets service * Rename decryptSecureJsonData to decryptSecureJsonDataFn * Name args in GetDecryptedValueFn * Add template back to NewAlertmanagerNotifier * Copy GetDecryptedValueFn to ngalert * Add logging to pluginsettings * Fix pluginsettings test Co-authored-by: Tania B <yalyna.ts@gmail.com> Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>
2021-10-07 09:33:50 -05:00
mssql "github.com/denisenkom/go-mssqldb"
"github.com/grafana/grafana-plugin-sdk-go/backend"
"github.com/grafana/grafana-plugin-sdk-go/backend/datasource"
"github.com/grafana/grafana-plugin-sdk-go/backend/instancemgmt"
"github.com/grafana/grafana-plugin-sdk-go/data"
"github.com/grafana/grafana-plugin-sdk-go/data/sqlutil"
Encryption: Refactor securejsondata.SecureJsonData to stop relying on global functions (#38865) * Encryption: Add support to encrypt/decrypt sjd * Add datasources.Service as a proxy to datasources db operations * Encrypt ds.SecureJsonData before calling SQLStore * Move ds cache code into ds service * Fix tlsmanager tests * Fix pluginproxy tests * Remove some securejsondata.GetEncryptedJsonData usages * Add pluginsettings.Service as a proxy for plugin settings db operations * Add AlertNotificationService as a proxy for alert notification db operations * Remove some securejsondata.GetEncryptedJsonData usages * Remove more securejsondata.GetEncryptedJsonData usages * Fix lint errors * Minor fixes * Remove encryption global functions usages from ngalert * Fix lint errors * Minor fixes * Minor fixes * Remove securejsondata.DecryptedValue usage * Refactor the refactor * Remove securejsondata.DecryptedValue usage * Move securejsondata to migrations package * Move securejsondata to migrations package * Minor fix * Fix integration test * Fix integration tests * Undo undesired changes * Fix tests * Add context.Context into encryption methods * Fix tests * Fix tests * Fix tests * Trigger CI * Fix test * Add names to params of encryption service interface * Remove bus from CacheServiceImpl * Add logging * Add keys to logger Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com> * Add missing key to logger Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com> * Undo changes in markdown files * Fix formatting * Add context to secrets service * Rename decryptSecureJsonData to decryptSecureJsonDataFn * Name args in GetDecryptedValueFn * Add template back to NewAlertmanagerNotifier * Copy GetDecryptedValueFn to ngalert * Add logging to pluginsettings * Fix pluginsettings test Co-authored-by: Tania B <yalyna.ts@gmail.com> Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>
2021-10-07 09:33:50 -05:00
"github.com/grafana/grafana/pkg/infra/log"
Plugins: Refactor Plugin Management (#40477) * add core plugin flow * add instrumentation * move func * remove cruft * support external backend plugins * refactor + clean up * remove comments * refactor loader * simplify core plugin path arg * cleanup loggers * move signature validator to plugins package * fix sig packaging * cleanup plugin model * remove unnecessary plugin field * add start+stop for pm * fix failures * add decommissioned state * export fields just to get things flowing * fix comments * set static routes * make image loading idempotent * merge with backend plugin manager * re-use funcs * reorder imports + remove unnecessary interface * add some TODOs + remove unused func * remove unused instrumentation func * simplify client usage * remove import alias * re-use backendplugin.Plugin interface * re order funcs * improve var name * fix log statements * refactor data model * add logic for dupe check during loading * cleanup state setting * refactor loader * cleanup manager interface * add rendering flow * refactor loading + init * add renderer support * fix renderer plugin * reformat imports * track errors * fix plugin signature inheritance * name param in interface * update func comment * fix func arg name * introduce class concept * remove func * fix external plugin check * apply changes from pm-experiment * fix core plugins * fix imports * rename interface * comment API interface * add support for testdata plugin * enable alerting + use correct core plugin contracts * slim manager API * fix param name * fix filter * support static routes * fix rendering * tidy rendering * get tests compiling * fix install+uninstall * start finder test * add finder test coverage * start loader tests * add test for core plugins * load core + bundled test * add test for nested plugin loading * add test files * clean interface + fix registering some core plugins * refactoring * reformat and create sub packages * simplify core plugin init * fix ctx cancel scenario * migrate initializer * remove Init() funcs * add test starter * new logger * flesh out initializer tests * refactoring * remove unused svc * refactor rendering flow * fixup loader tests * add enabled helper func * fix logger name * fix data fetchers * fix case where plugin dir doesn't exist * improve coverage + move dupe checking to loader * remove noisy debug logs * register core plugins automagically * add support for renderer in catalog * make private func + fix req validation * use interface * re-add check for renderer in catalog * tidy up from moving to auto reg core plugins * core plugin registrar * guards * copy over core plugins for test infra * all tests green * renames * propagate new interfaces * kill old manager * get compiling * tidy up * update naming * refactor manager test + cleanup * add more cases to finder test * migrate validator to field * more coverage * refactor dupe checking * add test for plugin class * add coverage for initializer * split out rendering * move * fixup tests * fix uss test * fix frontend settings * fix grafanads test * add check when checking sig errors * fix enabled map * fixup * allow manual setup of CM * rename to cloud-monitoring * remove TODO * add installer interface for testing * loader interface returns * tests passing * refactor + add more coverage * support 'stackdriver' * fix frontend settings loading * improve naming based on package name * small tidy * refactor test * fix renderer start * make cloud-monitoring plugin ID clearer * add plugin update test * add integration tests * don't break all if sig can't be calculated * add root URL check test * add more signature verification tests * update DTO name * update enabled plugins comment * update comments * fix linter * revert fe naming change * fix errors endpoint * reset error code field name * re-order test to help verify * assert -> require * pm check * add missing entry + re-order * re-check * dump icon log * verify manager contents first * reformat * apply PR feedback * apply style changes * fix one vs all loading err * improve log output * only start when no signature error * move log * rework plugin update check * fix test * fix multi loading from cfg.PluginSettings * improve log output #2 * add error abstraction to capture errors without registering a plugin * add debug log * add unsigned warning * e2e test attempt * fix logger * set home path * prevent panic * alternate * ugh.. fix home path * return renderer even if not started * make renderer plugin managed * add fallback renderer icon, update renderer badge + prevent changes when renderer is installed * fix icon loading * rollback renderer changes * use correct field * remove unneccessary block * remove newline * remove unused func * fix bundled plugins base + module fields * remove unused field since refactor * add authorizer abstraction * loader only returns plugins expected to run * fix multi log output
2021-11-01 04:53:33 -05:00
"github.com/grafana/grafana/pkg/plugins"
"github.com/grafana/grafana/pkg/plugins/backendplugin/coreplugin"
"github.com/grafana/grafana/pkg/setting"
"github.com/grafana/grafana/pkg/tsdb/sqleng"
Encryption: Refactor securejsondata.SecureJsonData to stop relying on global functions (#38865) * Encryption: Add support to encrypt/decrypt sjd * Add datasources.Service as a proxy to datasources db operations * Encrypt ds.SecureJsonData before calling SQLStore * Move ds cache code into ds service * Fix tlsmanager tests * Fix pluginproxy tests * Remove some securejsondata.GetEncryptedJsonData usages * Add pluginsettings.Service as a proxy for plugin settings db operations * Add AlertNotificationService as a proxy for alert notification db operations * Remove some securejsondata.GetEncryptedJsonData usages * Remove more securejsondata.GetEncryptedJsonData usages * Fix lint errors * Minor fixes * Remove encryption global functions usages from ngalert * Fix lint errors * Minor fixes * Minor fixes * Remove securejsondata.DecryptedValue usage * Refactor the refactor * Remove securejsondata.DecryptedValue usage * Move securejsondata to migrations package * Move securejsondata to migrations package * Minor fix * Fix integration test * Fix integration tests * Undo undesired changes * Fix tests * Add context.Context into encryption methods * Fix tests * Fix tests * Fix tests * Trigger CI * Fix test * Add names to params of encryption service interface * Remove bus from CacheServiceImpl * Add logging * Add keys to logger Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com> * Add missing key to logger Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com> * Undo changes in markdown files * Fix formatting * Add context to secrets service * Rename decryptSecureJsonData to decryptSecureJsonDataFn * Name args in GetDecryptedValueFn * Add template back to NewAlertmanagerNotifier * Copy GetDecryptedValueFn to ngalert * Add logging to pluginsettings * Fix pluginsettings test Co-authored-by: Tania B <yalyna.ts@gmail.com> Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>
2021-10-07 09:33:50 -05:00
"github.com/grafana/grafana/pkg/util"
2017-12-02 05:40:12 -06:00
)
var logger = log.New("tsdb.mssql")
2017-12-02 05:40:12 -06:00
2021-12-14 08:22:40 -06:00
const pluginID = "mssql"
type Service struct {
im instancemgmt.InstanceManager
}
2021-12-14 08:22:40 -06:00
func ProvideService(cfg *setting.Cfg, pluginStore plugins.Store) (*Service, error) {
s := &Service{
im: datasource.NewInstanceManager(newInstanceSettings(cfg)),
}
factory := coreplugin.New(backend.ServeOpts{
QueryDataHandler: s,
})
2021-12-14 08:22:40 -06:00
resolver := plugins.CoreDataSourcePathResolver(cfg, pluginID)
if err := pluginStore.AddWithFactory(context.Background(), pluginID, factory, resolver); err != nil {
logger.Error("Failed to register plugin", "error", err)
2019-03-15 05:53:30 -05:00
}
return s, nil
}
2018-03-19 11:32:51 -05:00
func (s *Service) getDataSourceHandler(pluginCtx backend.PluginContext) (*sqleng.DataSourceHandler, error) {
i, err := s.im.Get(pluginCtx)
if err != nil {
return nil, err
2018-07-26 11:11:10 -05:00
}
instance := i.(*sqleng.DataSourceHandler)
return instance, nil
}
2018-07-26 11:11:10 -05:00
func (s *Service) QueryData(ctx context.Context, req *backend.QueryDataRequest) (*backend.QueryDataResponse, error) {
dsHandler, err := s.getDataSourceHandler(req.PluginContext)
if err != nil {
return nil, err
2018-03-19 11:32:51 -05:00
}
return dsHandler.QueryData(ctx, req)
}
2018-03-19 11:32:51 -05:00
func newInstanceSettings(cfg *setting.Cfg) datasource.InstanceFactoryFunc {
return func(settings backend.DataSourceInstanceSettings) (instancemgmt.Instance, error) {
jsonData := sqleng.JsonData{
MaxOpenConns: 0,
MaxIdleConns: 2,
ConnMaxLifetime: 14400,
Encrypt: "false",
}
err := json.Unmarshal(settings.JSONData, &jsonData)
if err != nil {
return nil, fmt.Errorf("error reading settings: %w", err)
}
dsInfo := sqleng.DataSourceInfo{
JsonData: jsonData,
URL: settings.URL,
User: settings.User,
Database: settings.Database,
ID: settings.ID,
Updated: settings.Updated,
UID: settings.UID,
DecryptedSecureJSONData: settings.DecryptedSecureJSONData,
}
cnnstr, err := generateConnectionString(dsInfo)
if err != nil {
return nil, err
}
if cfg.Env == setting.Dev {
logger.Debug("getEngine", "connection", cnnstr)
}
config := sqleng.DataPluginConfiguration{
DriverName: "mssql",
ConnectionString: cnnstr,
DSInfo: dsInfo,
MetricColumnTypes: []string{"VARCHAR", "CHAR", "NVARCHAR", "NCHAR"},
RowLimit: cfg.DataProxyRowLimit,
}
queryResultTransformer := mssqlQueryResultTransformer{
log: logger,
}
return sqleng.NewQueryDataHandler(config, &queryResultTransformer, newMssqlMacroEngine(), logger)
}
2018-03-19 11:32:51 -05:00
}
// ParseURL tries to parse an MSSQL URL string into a URL object.
func ParseURL(u string) (*url.URL, error) {
logger.Debug("Parsing MSSQL URL", "url", u)
// Recognize ODBC connection strings like host\instance:1234
reODBC := regexp.MustCompile(`^[^\\:]+(?:\\[^:]+)?(?::\d+)?(?:;.+)?$`)
var host string
switch {
case reODBC.MatchString(u):
logger.Debug("Recognized as ODBC URL format", "url", u)
host = u
default:
logger.Debug("Couldn't recognize as valid MSSQL URL", "url", u)
return nil, fmt.Errorf("unrecognized MSSQL URL format: %q", u)
}
return &url.URL{
Scheme: "sqlserver",
Host: host,
}, nil
}
func generateConnectionString(dsInfo sqleng.DataSourceInfo) (string, error) {
const dfltPort = "0"
var addr util.NetworkAddress
if dsInfo.URL != "" {
u, err := ParseURL(dsInfo.URL)
if err != nil {
return "", err
}
addr, err = util.SplitHostPortDefault(u.Host, "localhost", dfltPort)
if err != nil {
return "", err
}
} else {
addr = util.NetworkAddress{
Host: "localhost",
Port: dfltPort,
}
}
args := []interface{}{
"url", dsInfo.URL, "host", addr.Host,
}
if addr.Port != "0" {
args = append(args, "port", addr.Port)
}
logger.Debug("Generating connection string", args...)
connStr := fmt.Sprintf("server=%s;database=%s;user id=%s;password=%s;",
addr.Host,
dsInfo.Database,
dsInfo.User,
dsInfo.DecryptedSecureJSONData["password"],
)
// Port number 0 means to determine the port automatically, so we can let the driver choose
if addr.Port != "0" {
connStr += fmt.Sprintf("port=%s;", addr.Port)
}
if dsInfo.JsonData.Encrypt == "" {
dsInfo.JsonData.Encrypt = "false"
}
if dsInfo.JsonData.Encrypt != "false" {
connStr += fmt.Sprintf("encrypt=%s;", dsInfo.JsonData.Encrypt)
}
return connStr, nil
2017-12-02 05:40:12 -06:00
}
type mssqlQueryResultTransformer struct {
2018-07-26 11:11:10 -05:00
log log.Logger
2017-12-02 05:40:12 -06:00
}
func (t *mssqlQueryResultTransformer) TransformQueryError(err error) error {
// go-mssql overrides source error, so we currently match on string
// ref https://github.com/denisenkom/go-mssqldb/blob/045585d74f9069afe2e115b6235eb043c8047043/tds.go#L904
if strings.HasPrefix(strings.ToLower(err.Error()), "unable to open tcp connection with host") {
t.log.Error("query error", "err", err)
return sqleng.ErrConnectionFailed
}
return err
}
func (t *mssqlQueryResultTransformer) GetConverterList() []sqlutil.StringConverter {
return []sqlutil.StringConverter{
{
Name: "handle MONEY",
InputScanKind: reflect.Slice,
InputTypeName: "MONEY",
ConversionFunc: func(in *string) (*string, error) { return in, nil },
Replacer: &sqlutil.StringFieldReplacer{
OutputFieldType: data.FieldTypeNullableFloat64,
ReplaceFunc: func(in *string) (interface{}, error) {
if in == nil {
return nil, nil
}
v, err := strconv.ParseFloat(*in, 64)
if err != nil {
return nil, err
}
return &v, nil
},
},
},
{
Name: "handle SMALLMONEY",
InputScanKind: reflect.Slice,
InputTypeName: "SMALLMONEY",
ConversionFunc: func(in *string) (*string, error) { return in, nil },
Replacer: &sqlutil.StringFieldReplacer{
OutputFieldType: data.FieldTypeNullableFloat64,
ReplaceFunc: func(in *string) (interface{}, error) {
if in == nil {
return nil, nil
}
v, err := strconv.ParseFloat(*in, 64)
if err != nil {
return nil, err
}
return &v, nil
},
},
},
{
Name: "handle DECIMAL",
InputScanKind: reflect.Slice,
InputTypeName: "DECIMAL",
ConversionFunc: func(in *string) (*string, error) { return in, nil },
Replacer: &sqlutil.StringFieldReplacer{
OutputFieldType: data.FieldTypeNullableFloat64,
ReplaceFunc: func(in *string) (interface{}, error) {
if in == nil {
return nil, nil
}
v, err := strconv.ParseFloat(*in, 64)
if err != nil {
return nil, err
}
return &v, nil
},
},
},
{
Name: "handle UNIQUEIDENTIFIER",
InputScanKind: reflect.Slice,
InputTypeName: "UNIQUEIDENTIFIER",
ConversionFunc: func(in *string) (*string, error) { return in, nil },
Replacer: &sqlutil.StringFieldReplacer{
OutputFieldType: data.FieldTypeNullableString,
ReplaceFunc: func(in *string) (interface{}, error) {
if in == nil {
return nil, nil
}
uuid := &mssql.UniqueIdentifier{}
if err := uuid.Scan([]byte(*in)); err != nil {
return nil, err
}
v := uuid.String()
return &v, nil
},
},
},
}
}