grafana/pkg/tsdb/postgres/postgres.go

205 lines
5.9 KiB
Go
Raw Normal View History

Postgres Data Source (#9475) * add postgresql datasource * add rest of files for postgres datasource * fix timeseries query, remove unused code * consistent naming, refactoring * s/mysql/postgres/ * s/mysql/postgres/ * couple more tests * tests for more datatypes * fix macros for postgres * add __timeSec macro * add frontend for postgres datasource * adjust documentation * fix formatting * add proper plugin description * merge editor changes from mysql * port changes from mysql datasource * set proper defaultQuery for postgres * add time_sec to timeseries query accept int for value for timeseries query * revert allowing time_sec and handle int or float values as unix timestamp for "time" column * fix tslint error * handle decimal values in timeseries query * allow setting sslmode for postgres datasource * use type switch for handling data types * fix value for timeseries query * refactor timeseries queries to make them more flexible * remove debug statement from inner loop in type conversion * use plain for loop in getTypedRowData * fix timeseries queries * adjust postgres datasource to tsdb refactoring * adjust postgres datasource to frontend changes * update lib/pq to latest version * move type conversion to getTypedRowData * handle address types cidr, inet and macaddr * adjust response parser and docs for annotations * convert unknown types to string * add documentation for postgres datasource * add another example query with metric column * set more helpful default query * update help text in query editor * handle NULL in value column of timeseries query * add __timeGroup macro * add test for __timeGroup macro * document __timeGroup and set proper default query for annotations * fix typos in docs * add postgres to list of datasources * add postgres to builtInPlugins * mysql: refactoring as prep for merging postgres Refactors out the initialization of the xorm engine and the query logic for an sql data source. * mysql: rename refactoring + test update * postgres:refactor to use SqlEngine(same as mysql) Refactored to use a common base class with the MySql data source. Other changes from the original PR: - Changed time column to be time_sec to allow other time units in the future and to be the same as MySQL - Changed integration test to test the main Query method rather than the private transformToTable method - Changed the __timeSec macro name to __timeEpoch - Renamed PostgresExecutor to PostgresQueryEndpoint Fixes #9209 (the original PR) * postgres: encrypt password on config page With some other cosmetic changes to the config page: - placeholder texts - reset button for the password after it has been encrypted. - default value for the sslmode field. * postgres: change back col name to time from time_sec * postgres mysql: remove annotation title Title has been removed from annotations * postgres: fix images for docs page * postgres mysql: fix specs
2017-10-10 08:19:14 -05:00
package postgres
import (
"fmt"
"reflect"
Postgres Data Source (#9475) * add postgresql datasource * add rest of files for postgres datasource * fix timeseries query, remove unused code * consistent naming, refactoring * s/mysql/postgres/ * s/mysql/postgres/ * couple more tests * tests for more datatypes * fix macros for postgres * add __timeSec macro * add frontend for postgres datasource * adjust documentation * fix formatting * add proper plugin description * merge editor changes from mysql * port changes from mysql datasource * set proper defaultQuery for postgres * add time_sec to timeseries query accept int for value for timeseries query * revert allowing time_sec and handle int or float values as unix timestamp for "time" column * fix tslint error * handle decimal values in timeseries query * allow setting sslmode for postgres datasource * use type switch for handling data types * fix value for timeseries query * refactor timeseries queries to make them more flexible * remove debug statement from inner loop in type conversion * use plain for loop in getTypedRowData * fix timeseries queries * adjust postgres datasource to tsdb refactoring * adjust postgres datasource to frontend changes * update lib/pq to latest version * move type conversion to getTypedRowData * handle address types cidr, inet and macaddr * adjust response parser and docs for annotations * convert unknown types to string * add documentation for postgres datasource * add another example query with metric column * set more helpful default query * update help text in query editor * handle NULL in value column of timeseries query * add __timeGroup macro * add test for __timeGroup macro * document __timeGroup and set proper default query for annotations * fix typos in docs * add postgres to list of datasources * add postgres to builtInPlugins * mysql: refactoring as prep for merging postgres Refactors out the initialization of the xorm engine and the query logic for an sql data source. * mysql: rename refactoring + test update * postgres:refactor to use SqlEngine(same as mysql) Refactored to use a common base class with the MySql data source. Other changes from the original PR: - Changed time column to be time_sec to allow other time units in the future and to be the same as MySQL - Changed integration test to test the main Query method rather than the private transformToTable method - Changed the __timeSec macro name to __timeEpoch - Renamed PostgresExecutor to PostgresQueryEndpoint Fixes #9209 (the original PR) * postgres: encrypt password on config page With some other cosmetic changes to the config page: - placeholder texts - reset button for the password after it has been encrypted. - default value for the sslmode field. * postgres: change back col name to time from time_sec * postgres mysql: remove annotation title Title has been removed from annotations * postgres: fix images for docs page * postgres mysql: fix specs
2017-10-10 08:19:14 -05:00
"strconv"
"strings"
Postgres Data Source (#9475) * add postgresql datasource * add rest of files for postgres datasource * fix timeseries query, remove unused code * consistent naming, refactoring * s/mysql/postgres/ * s/mysql/postgres/ * couple more tests * tests for more datatypes * fix macros for postgres * add __timeSec macro * add frontend for postgres datasource * adjust documentation * fix formatting * add proper plugin description * merge editor changes from mysql * port changes from mysql datasource * set proper defaultQuery for postgres * add time_sec to timeseries query accept int for value for timeseries query * revert allowing time_sec and handle int or float values as unix timestamp for "time" column * fix tslint error * handle decimal values in timeseries query * allow setting sslmode for postgres datasource * use type switch for handling data types * fix value for timeseries query * refactor timeseries queries to make them more flexible * remove debug statement from inner loop in type conversion * use plain for loop in getTypedRowData * fix timeseries queries * adjust postgres datasource to tsdb refactoring * adjust postgres datasource to frontend changes * update lib/pq to latest version * move type conversion to getTypedRowData * handle address types cidr, inet and macaddr * adjust response parser and docs for annotations * convert unknown types to string * add documentation for postgres datasource * add another example query with metric column * set more helpful default query * update help text in query editor * handle NULL in value column of timeseries query * add __timeGroup macro * add test for __timeGroup macro * document __timeGroup and set proper default query for annotations * fix typos in docs * add postgres to list of datasources * add postgres to builtInPlugins * mysql: refactoring as prep for merging postgres Refactors out the initialization of the xorm engine and the query logic for an sql data source. * mysql: rename refactoring + test update * postgres:refactor to use SqlEngine(same as mysql) Refactored to use a common base class with the MySql data source. Other changes from the original PR: - Changed time column to be time_sec to allow other time units in the future and to be the same as MySQL - Changed integration test to test the main Query method rather than the private transformToTable method - Changed the __timeSec macro name to __timeEpoch - Renamed PostgresExecutor to PostgresQueryEndpoint Fixes #9209 (the original PR) * postgres: encrypt password on config page With some other cosmetic changes to the config page: - placeholder texts - reset button for the password after it has been encrypted. - default value for the sslmode field. * postgres: change back col name to time from time_sec * postgres mysql: remove annotation title Title has been removed from annotations * postgres: fix images for docs page * postgres mysql: fix specs
2017-10-10 08:19:14 -05:00
"github.com/grafana/grafana-plugin-sdk-go/data"
"github.com/grafana/grafana-plugin-sdk-go/data/sqlutil"
Postgres: allow providing TLS/SSL certificates as text in addition to file paths (#30353) * postgres SSL certification * add back the UI to configure SSL Authentication files by file path * add backend logic * correct unittest * mini changes * Update public/app/plugins/datasource/postgres/config_ctrl.ts Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> * mutex * check file exist before remove * change permission * change default configuremethod to file-path * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * rename sslconfiguremethod to sslconfigurationmethod * frontend update * solve comments * Postgres: Convert tests to stdlib Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Postgres: Be consistent about TLS/SSL terminology Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * fix init inconsistancy * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * naming convention * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Undo change Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix TLS issue Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * change permissions * Fix data source field names Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Clean up HTML Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Improve popover text Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix SSL input bug Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Undo unnecessary change Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Clean up backend code Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix build Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * More consistent naming Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Clean up code Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Enforce certificate file permissions Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * add settings * Undo changes Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * fix windows file path * PostgresDataSource: Fix mutex usage Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Update pkg/tsdb/postgres/postgres.go Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Apply suggestions from code review Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * fix compilation * fix unittest * Apply suggestions from code review Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Apply suggestions from code review Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * mock function * change kmutex package * add kmutex into middleware * lock connection file per datasource * add unittest regarding concurrency * version should be equal * adding unittest * fix the loop * fix unitest * fix postgres unittst * remove comments * move dataPath from arg to tlsManager struct field * Use DecryptedValues method Use cached decrypted values instead of using secure json data decrypt which will decrypt unchanged values over and over again. * remove unneeded mutex in tests and cleanup tests * fix the lint Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2021-02-23 15:10:55 -06:00
"github.com/grafana/grafana/pkg/registry"
"github.com/grafana/grafana/pkg/setting"
"github.com/grafana/grafana/pkg/util/errutil"
"github.com/grafana/grafana/pkg/infra/log"
Postgres Data Source (#9475) * add postgresql datasource * add rest of files for postgres datasource * fix timeseries query, remove unused code * consistent naming, refactoring * s/mysql/postgres/ * s/mysql/postgres/ * couple more tests * tests for more datatypes * fix macros for postgres * add __timeSec macro * add frontend for postgres datasource * adjust documentation * fix formatting * add proper plugin description * merge editor changes from mysql * port changes from mysql datasource * set proper defaultQuery for postgres * add time_sec to timeseries query accept int for value for timeseries query * revert allowing time_sec and handle int or float values as unix timestamp for "time" column * fix tslint error * handle decimal values in timeseries query * allow setting sslmode for postgres datasource * use type switch for handling data types * fix value for timeseries query * refactor timeseries queries to make them more flexible * remove debug statement from inner loop in type conversion * use plain for loop in getTypedRowData * fix timeseries queries * adjust postgres datasource to tsdb refactoring * adjust postgres datasource to frontend changes * update lib/pq to latest version * move type conversion to getTypedRowData * handle address types cidr, inet and macaddr * adjust response parser and docs for annotations * convert unknown types to string * add documentation for postgres datasource * add another example query with metric column * set more helpful default query * update help text in query editor * handle NULL in value column of timeseries query * add __timeGroup macro * add test for __timeGroup macro * document __timeGroup and set proper default query for annotations * fix typos in docs * add postgres to list of datasources * add postgres to builtInPlugins * mysql: refactoring as prep for merging postgres Refactors out the initialization of the xorm engine and the query logic for an sql data source. * mysql: rename refactoring + test update * postgres:refactor to use SqlEngine(same as mysql) Refactored to use a common base class with the MySql data source. Other changes from the original PR: - Changed time column to be time_sec to allow other time units in the future and to be the same as MySQL - Changed integration test to test the main Query method rather than the private transformToTable method - Changed the __timeSec macro name to __timeEpoch - Renamed PostgresExecutor to PostgresQueryEndpoint Fixes #9209 (the original PR) * postgres: encrypt password on config page With some other cosmetic changes to the config page: - placeholder texts - reset button for the password after it has been encrypted. - default value for the sslmode field. * postgres: change back col name to time from time_sec * postgres mysql: remove annotation title Title has been removed from annotations * postgres: fix images for docs page * postgres mysql: fix specs
2017-10-10 08:19:14 -05:00
"github.com/grafana/grafana/pkg/models"
"github.com/grafana/grafana/pkg/plugins"
"github.com/grafana/grafana/pkg/tsdb/sqleng"
Postgres Data Source (#9475) * add postgresql datasource * add rest of files for postgres datasource * fix timeseries query, remove unused code * consistent naming, refactoring * s/mysql/postgres/ * s/mysql/postgres/ * couple more tests * tests for more datatypes * fix macros for postgres * add __timeSec macro * add frontend for postgres datasource * adjust documentation * fix formatting * add proper plugin description * merge editor changes from mysql * port changes from mysql datasource * set proper defaultQuery for postgres * add time_sec to timeseries query accept int for value for timeseries query * revert allowing time_sec and handle int or float values as unix timestamp for "time" column * fix tslint error * handle decimal values in timeseries query * allow setting sslmode for postgres datasource * use type switch for handling data types * fix value for timeseries query * refactor timeseries queries to make them more flexible * remove debug statement from inner loop in type conversion * use plain for loop in getTypedRowData * fix timeseries queries * adjust postgres datasource to tsdb refactoring * adjust postgres datasource to frontend changes * update lib/pq to latest version * move type conversion to getTypedRowData * handle address types cidr, inet and macaddr * adjust response parser and docs for annotations * convert unknown types to string * add documentation for postgres datasource * add another example query with metric column * set more helpful default query * update help text in query editor * handle NULL in value column of timeseries query * add __timeGroup macro * add test for __timeGroup macro * document __timeGroup and set proper default query for annotations * fix typos in docs * add postgres to list of datasources * add postgres to builtInPlugins * mysql: refactoring as prep for merging postgres Refactors out the initialization of the xorm engine and the query logic for an sql data source. * mysql: rename refactoring + test update * postgres:refactor to use SqlEngine(same as mysql) Refactored to use a common base class with the MySql data source. Other changes from the original PR: - Changed time column to be time_sec to allow other time units in the future and to be the same as MySQL - Changed integration test to test the main Query method rather than the private transformToTable method - Changed the __timeSec macro name to __timeEpoch - Renamed PostgresExecutor to PostgresQueryEndpoint Fixes #9209 (the original PR) * postgres: encrypt password on config page With some other cosmetic changes to the config page: - placeholder texts - reset button for the password after it has been encrypted. - default value for the sslmode field. * postgres: change back col name to time from time_sec * postgres mysql: remove annotation title Title has been removed from annotations * postgres: fix images for docs page * postgres mysql: fix specs
2017-10-10 08:19:14 -05:00
)
func init() {
Postgres: allow providing TLS/SSL certificates as text in addition to file paths (#30353) * postgres SSL certification * add back the UI to configure SSL Authentication files by file path * add backend logic * correct unittest * mini changes * Update public/app/plugins/datasource/postgres/config_ctrl.ts Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> * mutex * check file exist before remove * change permission * change default configuremethod to file-path * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * rename sslconfiguremethod to sslconfigurationmethod * frontend update * solve comments * Postgres: Convert tests to stdlib Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Postgres: Be consistent about TLS/SSL terminology Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * fix init inconsistancy * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * naming convention * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Undo change Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix TLS issue Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * change permissions * Fix data source field names Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Clean up HTML Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Improve popover text Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix SSL input bug Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Undo unnecessary change Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Clean up backend code Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix build Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * More consistent naming Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Clean up code Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Enforce certificate file permissions Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * add settings * Undo changes Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * fix windows file path * PostgresDataSource: Fix mutex usage Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Update pkg/tsdb/postgres/postgres.go Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Apply suggestions from code review Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * fix compilation * fix unittest * Apply suggestions from code review Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Apply suggestions from code review Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * mock function * change kmutex package * add kmutex into middleware * lock connection file per datasource * add unittest regarding concurrency * version should be equal * adding unittest * fix the loop * fix unitest * fix postgres unittst * remove comments * move dataPath from arg to tlsManager struct field * Use DecryptedValues method Use cached decrypted values instead of using secure json data decrypt which will decrypt unchanged values over and over again. * remove unneeded mutex in tests and cleanup tests * fix the lint Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2021-02-23 15:10:55 -06:00
registry.Register(&registry.Descriptor{
Name: "PostgresService",
InitPriority: registry.Low,
Instance: &PostgresService{},
Postgres: allow providing TLS/SSL certificates as text in addition to file paths (#30353) * postgres SSL certification * add back the UI to configure SSL Authentication files by file path * add backend logic * correct unittest * mini changes * Update public/app/plugins/datasource/postgres/config_ctrl.ts Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> * mutex * check file exist before remove * change permission * change default configuremethod to file-path * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * rename sslconfiguremethod to sslconfigurationmethod * frontend update * solve comments * Postgres: Convert tests to stdlib Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Postgres: Be consistent about TLS/SSL terminology Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * fix init inconsistancy * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * naming convention * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Undo change Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix TLS issue Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * change permissions * Fix data source field names Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Clean up HTML Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Improve popover text Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix SSL input bug Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Undo unnecessary change Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Clean up backend code Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix build Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * More consistent naming Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Clean up code Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Enforce certificate file permissions Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * add settings * Undo changes Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * fix windows file path * PostgresDataSource: Fix mutex usage Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Update pkg/tsdb/postgres/postgres.go Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Apply suggestions from code review Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * fix compilation * fix unittest * Apply suggestions from code review Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Apply suggestions from code review Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * mock function * change kmutex package * add kmutex into middleware * lock connection file per datasource * add unittest regarding concurrency * version should be equal * adding unittest * fix the loop * fix unitest * fix postgres unittst * remove comments * move dataPath from arg to tlsManager struct field * Use DecryptedValues method Use cached decrypted values instead of using secure json data decrypt which will decrypt unchanged values over and over again. * remove unneeded mutex in tests and cleanup tests * fix the lint Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2021-02-23 15:10:55 -06:00
})
}
type PostgresService struct {
Postgres: allow providing TLS/SSL certificates as text in addition to file paths (#30353) * postgres SSL certification * add back the UI to configure SSL Authentication files by file path * add backend logic * correct unittest * mini changes * Update public/app/plugins/datasource/postgres/config_ctrl.ts Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> * mutex * check file exist before remove * change permission * change default configuremethod to file-path * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * rename sslconfiguremethod to sslconfigurationmethod * frontend update * solve comments * Postgres: Convert tests to stdlib Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Postgres: Be consistent about TLS/SSL terminology Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * fix init inconsistancy * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * naming convention * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Undo change Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix TLS issue Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * change permissions * Fix data source field names Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Clean up HTML Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Improve popover text Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix SSL input bug Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Undo unnecessary change Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Clean up backend code Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix build Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * More consistent naming Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Clean up code Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Enforce certificate file permissions Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * add settings * Undo changes Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * fix windows file path * PostgresDataSource: Fix mutex usage Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Update pkg/tsdb/postgres/postgres.go Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Apply suggestions from code review Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * fix compilation * fix unittest * Apply suggestions from code review Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Apply suggestions from code review Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * mock function * change kmutex package * add kmutex into middleware * lock connection file per datasource * add unittest regarding concurrency * version should be equal * adding unittest * fix the loop * fix unitest * fix postgres unittst * remove comments * move dataPath from arg to tlsManager struct field * Use DecryptedValues method Use cached decrypted values instead of using secure json data decrypt which will decrypt unchanged values over and over again. * remove unneeded mutex in tests and cleanup tests * fix the lint Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2021-02-23 15:10:55 -06:00
Cfg *setting.Cfg `inject:""`
logger log.Logger
tlsManager tlsSettingsProvider
Postgres Data Source (#9475) * add postgresql datasource * add rest of files for postgres datasource * fix timeseries query, remove unused code * consistent naming, refactoring * s/mysql/postgres/ * s/mysql/postgres/ * couple more tests * tests for more datatypes * fix macros for postgres * add __timeSec macro * add frontend for postgres datasource * adjust documentation * fix formatting * add proper plugin description * merge editor changes from mysql * port changes from mysql datasource * set proper defaultQuery for postgres * add time_sec to timeseries query accept int for value for timeseries query * revert allowing time_sec and handle int or float values as unix timestamp for "time" column * fix tslint error * handle decimal values in timeseries query * allow setting sslmode for postgres datasource * use type switch for handling data types * fix value for timeseries query * refactor timeseries queries to make them more flexible * remove debug statement from inner loop in type conversion * use plain for loop in getTypedRowData * fix timeseries queries * adjust postgres datasource to tsdb refactoring * adjust postgres datasource to frontend changes * update lib/pq to latest version * move type conversion to getTypedRowData * handle address types cidr, inet and macaddr * adjust response parser and docs for annotations * convert unknown types to string * add documentation for postgres datasource * add another example query with metric column * set more helpful default query * update help text in query editor * handle NULL in value column of timeseries query * add __timeGroup macro * add test for __timeGroup macro * document __timeGroup and set proper default query for annotations * fix typos in docs * add postgres to list of datasources * add postgres to builtInPlugins * mysql: refactoring as prep for merging postgres Refactors out the initialization of the xorm engine and the query logic for an sql data source. * mysql: rename refactoring + test update * postgres:refactor to use SqlEngine(same as mysql) Refactored to use a common base class with the MySql data source. Other changes from the original PR: - Changed time column to be time_sec to allow other time units in the future and to be the same as MySQL - Changed integration test to test the main Query method rather than the private transformToTable method - Changed the __timeSec macro name to __timeEpoch - Renamed PostgresExecutor to PostgresQueryEndpoint Fixes #9209 (the original PR) * postgres: encrypt password on config page With some other cosmetic changes to the config page: - placeholder texts - reset button for the password after it has been encrypted. - default value for the sslmode field. * postgres: change back col name to time from time_sec * postgres mysql: remove annotation title Title has been removed from annotations * postgres: fix images for docs page * postgres mysql: fix specs
2017-10-10 08:19:14 -05:00
}
func (s *PostgresService) Init() error {
Postgres: allow providing TLS/SSL certificates as text in addition to file paths (#30353) * postgres SSL certification * add back the UI to configure SSL Authentication files by file path * add backend logic * correct unittest * mini changes * Update public/app/plugins/datasource/postgres/config_ctrl.ts Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> * mutex * check file exist before remove * change permission * change default configuremethod to file-path * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * rename sslconfiguremethod to sslconfigurationmethod * frontend update * solve comments * Postgres: Convert tests to stdlib Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Postgres: Be consistent about TLS/SSL terminology Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * fix init inconsistancy * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * naming convention * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Undo change Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix TLS issue Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * change permissions * Fix data source field names Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Clean up HTML Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Improve popover text Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix SSL input bug Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Undo unnecessary change Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Clean up backend code Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix build Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * More consistent naming Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Clean up code Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Enforce certificate file permissions Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * add settings * Undo changes Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * fix windows file path * PostgresDataSource: Fix mutex usage Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Update pkg/tsdb/postgres/postgres.go Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Apply suggestions from code review Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * fix compilation * fix unittest * Apply suggestions from code review Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Apply suggestions from code review Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * mock function * change kmutex package * add kmutex into middleware * lock connection file per datasource * add unittest regarding concurrency * version should be equal * adding unittest * fix the loop * fix unitest * fix postgres unittst * remove comments * move dataPath from arg to tlsManager struct field * Use DecryptedValues method Use cached decrypted values instead of using secure json data decrypt which will decrypt unchanged values over and over again. * remove unneeded mutex in tests and cleanup tests * fix the lint Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2021-02-23 15:10:55 -06:00
s.logger = log.New("tsdb.postgres")
s.tlsManager = newTLSManager(s.logger, s.Cfg.DataPath)
return nil
}
//nolint: staticcheck // plugins.DataPlugin deprecated
func (s *PostgresService) NewExecutor(datasource *models.DataSource) (plugins.DataPlugin, error) {
Postgres: allow providing TLS/SSL certificates as text in addition to file paths (#30353) * postgres SSL certification * add back the UI to configure SSL Authentication files by file path * add backend logic * correct unittest * mini changes * Update public/app/plugins/datasource/postgres/config_ctrl.ts Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> * mutex * check file exist before remove * change permission * change default configuremethod to file-path * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * rename sslconfiguremethod to sslconfigurationmethod * frontend update * solve comments * Postgres: Convert tests to stdlib Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Postgres: Be consistent about TLS/SSL terminology Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * fix init inconsistancy * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * naming convention * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Undo change Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix TLS issue Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * change permissions * Fix data source field names Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Clean up HTML Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Improve popover text Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix SSL input bug Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Undo unnecessary change Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Clean up backend code Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix build Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * More consistent naming Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Clean up code Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Enforce certificate file permissions Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * add settings * Undo changes Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * fix windows file path * PostgresDataSource: Fix mutex usage Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Update pkg/tsdb/postgres/postgres.go Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Apply suggestions from code review Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * fix compilation * fix unittest * Apply suggestions from code review Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Apply suggestions from code review Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * mock function * change kmutex package * add kmutex into middleware * lock connection file per datasource * add unittest regarding concurrency * version should be equal * adding unittest * fix the loop * fix unitest * fix postgres unittst * remove comments * move dataPath from arg to tlsManager struct field * Use DecryptedValues method Use cached decrypted values instead of using secure json data decrypt which will decrypt unchanged values over and over again. * remove unneeded mutex in tests and cleanup tests * fix the lint Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2021-02-23 15:10:55 -06:00
s.logger.Debug("Creating Postgres query endpoint")
cnnstr, err := s.generateConnectionString(datasource)
if err != nil {
return nil, err
}
Postgres Data Source (#9475) * add postgresql datasource * add rest of files for postgres datasource * fix timeseries query, remove unused code * consistent naming, refactoring * s/mysql/postgres/ * s/mysql/postgres/ * couple more tests * tests for more datatypes * fix macros for postgres * add __timeSec macro * add frontend for postgres datasource * adjust documentation * fix formatting * add proper plugin description * merge editor changes from mysql * port changes from mysql datasource * set proper defaultQuery for postgres * add time_sec to timeseries query accept int for value for timeseries query * revert allowing time_sec and handle int or float values as unix timestamp for "time" column * fix tslint error * handle decimal values in timeseries query * allow setting sslmode for postgres datasource * use type switch for handling data types * fix value for timeseries query * refactor timeseries queries to make them more flexible * remove debug statement from inner loop in type conversion * use plain for loop in getTypedRowData * fix timeseries queries * adjust postgres datasource to tsdb refactoring * adjust postgres datasource to frontend changes * update lib/pq to latest version * move type conversion to getTypedRowData * handle address types cidr, inet and macaddr * adjust response parser and docs for annotations * convert unknown types to string * add documentation for postgres datasource * add another example query with metric column * set more helpful default query * update help text in query editor * handle NULL in value column of timeseries query * add __timeGroup macro * add test for __timeGroup macro * document __timeGroup and set proper default query for annotations * fix typos in docs * add postgres to list of datasources * add postgres to builtInPlugins * mysql: refactoring as prep for merging postgres Refactors out the initialization of the xorm engine and the query logic for an sql data source. * mysql: rename refactoring + test update * postgres:refactor to use SqlEngine(same as mysql) Refactored to use a common base class with the MySql data source. Other changes from the original PR: - Changed time column to be time_sec to allow other time units in the future and to be the same as MySQL - Changed integration test to test the main Query method rather than the private transformToTable method - Changed the __timeSec macro name to __timeEpoch - Renamed PostgresExecutor to PostgresQueryEndpoint Fixes #9209 (the original PR) * postgres: encrypt password on config page With some other cosmetic changes to the config page: - placeholder texts - reset button for the password after it has been encrypted. - default value for the sslmode field. * postgres: change back col name to time from time_sec * postgres mysql: remove annotation title Title has been removed from annotations * postgres: fix images for docs page * postgres mysql: fix specs
2017-10-10 08:19:14 -05:00
Postgres: allow providing TLS/SSL certificates as text in addition to file paths (#30353) * postgres SSL certification * add back the UI to configure SSL Authentication files by file path * add backend logic * correct unittest * mini changes * Update public/app/plugins/datasource/postgres/config_ctrl.ts Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> * mutex * check file exist before remove * change permission * change default configuremethod to file-path * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * rename sslconfiguremethod to sslconfigurationmethod * frontend update * solve comments * Postgres: Convert tests to stdlib Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Postgres: Be consistent about TLS/SSL terminology Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * fix init inconsistancy * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * naming convention * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Undo change Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix TLS issue Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * change permissions * Fix data source field names Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Clean up HTML Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Improve popover text Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix SSL input bug Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Undo unnecessary change Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Clean up backend code Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix build Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * More consistent naming Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Clean up code Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Enforce certificate file permissions Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * add settings * Undo changes Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * fix windows file path * PostgresDataSource: Fix mutex usage Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Update pkg/tsdb/postgres/postgres.go Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Apply suggestions from code review Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * fix compilation * fix unittest * Apply suggestions from code review Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Apply suggestions from code review Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * mock function * change kmutex package * add kmutex into middleware * lock connection file per datasource * add unittest regarding concurrency * version should be equal * adding unittest * fix the loop * fix unitest * fix postgres unittst * remove comments * move dataPath from arg to tlsManager struct field * Use DecryptedValues method Use cached decrypted values instead of using secure json data decrypt which will decrypt unchanged values over and over again. * remove unneeded mutex in tests and cleanup tests * fix the lint Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2021-02-23 15:10:55 -06:00
if s.Cfg.Env == setting.Dev {
s.logger.Debug("getEngine", "connection", cnnstr)
2019-03-15 05:53:30 -05:00
}
Postgres Data Source (#9475) * add postgresql datasource * add rest of files for postgres datasource * fix timeseries query, remove unused code * consistent naming, refactoring * s/mysql/postgres/ * s/mysql/postgres/ * couple more tests * tests for more datatypes * fix macros for postgres * add __timeSec macro * add frontend for postgres datasource * adjust documentation * fix formatting * add proper plugin description * merge editor changes from mysql * port changes from mysql datasource * set proper defaultQuery for postgres * add time_sec to timeseries query accept int for value for timeseries query * revert allowing time_sec and handle int or float values as unix timestamp for "time" column * fix tslint error * handle decimal values in timeseries query * allow setting sslmode for postgres datasource * use type switch for handling data types * fix value for timeseries query * refactor timeseries queries to make them more flexible * remove debug statement from inner loop in type conversion * use plain for loop in getTypedRowData * fix timeseries queries * adjust postgres datasource to tsdb refactoring * adjust postgres datasource to frontend changes * update lib/pq to latest version * move type conversion to getTypedRowData * handle address types cidr, inet and macaddr * adjust response parser and docs for annotations * convert unknown types to string * add documentation for postgres datasource * add another example query with metric column * set more helpful default query * update help text in query editor * handle NULL in value column of timeseries query * add __timeGroup macro * add test for __timeGroup macro * document __timeGroup and set proper default query for annotations * fix typos in docs * add postgres to list of datasources * add postgres to builtInPlugins * mysql: refactoring as prep for merging postgres Refactors out the initialization of the xorm engine and the query logic for an sql data source. * mysql: rename refactoring + test update * postgres:refactor to use SqlEngine(same as mysql) Refactored to use a common base class with the MySql data source. Other changes from the original PR: - Changed time column to be time_sec to allow other time units in the future and to be the same as MySQL - Changed integration test to test the main Query method rather than the private transformToTable method - Changed the __timeSec macro name to __timeEpoch - Renamed PostgresExecutor to PostgresQueryEndpoint Fixes #9209 (the original PR) * postgres: encrypt password on config page With some other cosmetic changes to the config page: - placeholder texts - reset button for the password after it has been encrypted. - default value for the sslmode field. * postgres: change back col name to time from time_sec * postgres mysql: remove annotation title Title has been removed from annotations * postgres: fix images for docs page * postgres mysql: fix specs
2017-10-10 08:19:14 -05:00
config := sqleng.DataPluginConfiguration{
2018-07-26 11:10:17 -05:00
DriverName: "postgres",
ConnectionString: cnnstr,
Datasource: datasource,
MetricColumnTypes: []string{"UNKNOWN", "TEXT", "VARCHAR", "CHAR"},
Postgres Data Source (#9475) * add postgresql datasource * add rest of files for postgres datasource * fix timeseries query, remove unused code * consistent naming, refactoring * s/mysql/postgres/ * s/mysql/postgres/ * couple more tests * tests for more datatypes * fix macros for postgres * add __timeSec macro * add frontend for postgres datasource * adjust documentation * fix formatting * add proper plugin description * merge editor changes from mysql * port changes from mysql datasource * set proper defaultQuery for postgres * add time_sec to timeseries query accept int for value for timeseries query * revert allowing time_sec and handle int or float values as unix timestamp for "time" column * fix tslint error * handle decimal values in timeseries query * allow setting sslmode for postgres datasource * use type switch for handling data types * fix value for timeseries query * refactor timeseries queries to make them more flexible * remove debug statement from inner loop in type conversion * use plain for loop in getTypedRowData * fix timeseries queries * adjust postgres datasource to tsdb refactoring * adjust postgres datasource to frontend changes * update lib/pq to latest version * move type conversion to getTypedRowData * handle address types cidr, inet and macaddr * adjust response parser and docs for annotations * convert unknown types to string * add documentation for postgres datasource * add another example query with metric column * set more helpful default query * update help text in query editor * handle NULL in value column of timeseries query * add __timeGroup macro * add test for __timeGroup macro * document __timeGroup and set proper default query for annotations * fix typos in docs * add postgres to list of datasources * add postgres to builtInPlugins * mysql: refactoring as prep for merging postgres Refactors out the initialization of the xorm engine and the query logic for an sql data source. * mysql: rename refactoring + test update * postgres:refactor to use SqlEngine(same as mysql) Refactored to use a common base class with the MySql data source. Other changes from the original PR: - Changed time column to be time_sec to allow other time units in the future and to be the same as MySQL - Changed integration test to test the main Query method rather than the private transformToTable method - Changed the __timeSec macro name to __timeEpoch - Renamed PostgresExecutor to PostgresQueryEndpoint Fixes #9209 (the original PR) * postgres: encrypt password on config page With some other cosmetic changes to the config page: - placeholder texts - reset button for the password after it has been encrypted. - default value for the sslmode field. * postgres: change back col name to time from time_sec * postgres mysql: remove annotation title Title has been removed from annotations * postgres: fix images for docs page * postgres mysql: fix specs
2017-10-10 08:19:14 -05:00
}
queryResultTransformer := postgresQueryResultTransformer{
Postgres: allow providing TLS/SSL certificates as text in addition to file paths (#30353) * postgres SSL certification * add back the UI to configure SSL Authentication files by file path * add backend logic * correct unittest * mini changes * Update public/app/plugins/datasource/postgres/config_ctrl.ts Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> * mutex * check file exist before remove * change permission * change default configuremethod to file-path * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * rename sslconfiguremethod to sslconfigurationmethod * frontend update * solve comments * Postgres: Convert tests to stdlib Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Postgres: Be consistent about TLS/SSL terminology Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * fix init inconsistancy * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * naming convention * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Undo change Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix TLS issue Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * change permissions * Fix data source field names Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Clean up HTML Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Improve popover text Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix SSL input bug Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Undo unnecessary change Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Clean up backend code Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix build Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * More consistent naming Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Clean up code Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Enforce certificate file permissions Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * add settings * Undo changes Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * fix windows file path * PostgresDataSource: Fix mutex usage Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Update pkg/tsdb/postgres/postgres.go Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Apply suggestions from code review Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * fix compilation * fix unittest * Apply suggestions from code review Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Apply suggestions from code review Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * mock function * change kmutex package * add kmutex into middleware * lock connection file per datasource * add unittest regarding concurrency * version should be equal * adding unittest * fix the loop * fix unitest * fix postgres unittst * remove comments * move dataPath from arg to tlsManager struct field * Use DecryptedValues method Use cached decrypted values instead of using secure json data decrypt which will decrypt unchanged values over and over again. * remove unneeded mutex in tests and cleanup tests * fix the lint Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2021-02-23 15:10:55 -06:00
log: s.logger,
2018-07-26 11:10:17 -05:00
}
timescaledb := datasource.JsonData.Get("timescaledb").MustBool(false)
plugin, err := sqleng.NewDataPlugin(config, &queryResultTransformer, newPostgresMacroEngine(timescaledb),
Postgres: allow providing TLS/SSL certificates as text in addition to file paths (#30353) * postgres SSL certification * add back the UI to configure SSL Authentication files by file path * add backend logic * correct unittest * mini changes * Update public/app/plugins/datasource/postgres/config_ctrl.ts Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> * mutex * check file exist before remove * change permission * change default configuremethod to file-path * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * rename sslconfiguremethod to sslconfigurationmethod * frontend update * solve comments * Postgres: Convert tests to stdlib Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Postgres: Be consistent about TLS/SSL terminology Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * fix init inconsistancy * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * naming convention * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Undo change Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix TLS issue Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * change permissions * Fix data source field names Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Clean up HTML Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Improve popover text Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix SSL input bug Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Undo unnecessary change Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Clean up backend code Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix build Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * More consistent naming Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Clean up code Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Enforce certificate file permissions Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * add settings * Undo changes Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * fix windows file path * PostgresDataSource: Fix mutex usage Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Update pkg/tsdb/postgres/postgres.go Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Apply suggestions from code review Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * fix compilation * fix unittest * Apply suggestions from code review Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Apply suggestions from code review Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * mock function * change kmutex package * add kmutex into middleware * lock connection file per datasource * add unittest regarding concurrency * version should be equal * adding unittest * fix the loop * fix unitest * fix postgres unittst * remove comments * move dataPath from arg to tlsManager struct field * Use DecryptedValues method Use cached decrypted values instead of using secure json data decrypt which will decrypt unchanged values over and over again. * remove unneeded mutex in tests and cleanup tests * fix the lint Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2021-02-23 15:10:55 -06:00
s.logger)
if err != nil {
Postgres: allow providing TLS/SSL certificates as text in addition to file paths (#30353) * postgres SSL certification * add back the UI to configure SSL Authentication files by file path * add backend logic * correct unittest * mini changes * Update public/app/plugins/datasource/postgres/config_ctrl.ts Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> * mutex * check file exist before remove * change permission * change default configuremethod to file-path * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * rename sslconfiguremethod to sslconfigurationmethod * frontend update * solve comments * Postgres: Convert tests to stdlib Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Postgres: Be consistent about TLS/SSL terminology Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * fix init inconsistancy * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * naming convention * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Undo change Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix TLS issue Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * change permissions * Fix data source field names Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Clean up HTML Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Improve popover text Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix SSL input bug Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Undo unnecessary change Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Clean up backend code Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix build Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * More consistent naming Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Clean up code Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Enforce certificate file permissions Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * add settings * Undo changes Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * fix windows file path * PostgresDataSource: Fix mutex usage Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Update pkg/tsdb/postgres/postgres.go Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Apply suggestions from code review Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * fix compilation * fix unittest * Apply suggestions from code review Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Apply suggestions from code review Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * mock function * change kmutex package * add kmutex into middleware * lock connection file per datasource * add unittest regarding concurrency * version should be equal * adding unittest * fix the loop * fix unitest * fix postgres unittst * remove comments * move dataPath from arg to tlsManager struct field * Use DecryptedValues method Use cached decrypted values instead of using secure json data decrypt which will decrypt unchanged values over and over again. * remove unneeded mutex in tests and cleanup tests * fix the lint Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2021-02-23 15:10:55 -06:00
s.logger.Error("Failed connecting to Postgres", "err", err)
return nil, err
}
Postgres: allow providing TLS/SSL certificates as text in addition to file paths (#30353) * postgres SSL certification * add back the UI to configure SSL Authentication files by file path * add backend logic * correct unittest * mini changes * Update public/app/plugins/datasource/postgres/config_ctrl.ts Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> * mutex * check file exist before remove * change permission * change default configuremethod to file-path * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * rename sslconfiguremethod to sslconfigurationmethod * frontend update * solve comments * Postgres: Convert tests to stdlib Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Postgres: Be consistent about TLS/SSL terminology Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * fix init inconsistancy * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * naming convention * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Undo change Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix TLS issue Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * change permissions * Fix data source field names Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Clean up HTML Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Improve popover text Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix SSL input bug Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Undo unnecessary change Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Clean up backend code Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix build Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * More consistent naming Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Clean up code Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Enforce certificate file permissions Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * add settings * Undo changes Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * fix windows file path * PostgresDataSource: Fix mutex usage Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Update pkg/tsdb/postgres/postgres.go Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Apply suggestions from code review Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * fix compilation * fix unittest * Apply suggestions from code review Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Apply suggestions from code review Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * mock function * change kmutex package * add kmutex into middleware * lock connection file per datasource * add unittest regarding concurrency * version should be equal * adding unittest * fix the loop * fix unitest * fix postgres unittst * remove comments * move dataPath from arg to tlsManager struct field * Use DecryptedValues method Use cached decrypted values instead of using secure json data decrypt which will decrypt unchanged values over and over again. * remove unneeded mutex in tests and cleanup tests * fix the lint Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2021-02-23 15:10:55 -06:00
s.logger.Debug("Successfully connected to Postgres")
return plugin, nil
Postgres Data Source (#9475) * add postgresql datasource * add rest of files for postgres datasource * fix timeseries query, remove unused code * consistent naming, refactoring * s/mysql/postgres/ * s/mysql/postgres/ * couple more tests * tests for more datatypes * fix macros for postgres * add __timeSec macro * add frontend for postgres datasource * adjust documentation * fix formatting * add proper plugin description * merge editor changes from mysql * port changes from mysql datasource * set proper defaultQuery for postgres * add time_sec to timeseries query accept int for value for timeseries query * revert allowing time_sec and handle int or float values as unix timestamp for "time" column * fix tslint error * handle decimal values in timeseries query * allow setting sslmode for postgres datasource * use type switch for handling data types * fix value for timeseries query * refactor timeseries queries to make them more flexible * remove debug statement from inner loop in type conversion * use plain for loop in getTypedRowData * fix timeseries queries * adjust postgres datasource to tsdb refactoring * adjust postgres datasource to frontend changes * update lib/pq to latest version * move type conversion to getTypedRowData * handle address types cidr, inet and macaddr * adjust response parser and docs for annotations * convert unknown types to string * add documentation for postgres datasource * add another example query with metric column * set more helpful default query * update help text in query editor * handle NULL in value column of timeseries query * add __timeGroup macro * add test for __timeGroup macro * document __timeGroup and set proper default query for annotations * fix typos in docs * add postgres to list of datasources * add postgres to builtInPlugins * mysql: refactoring as prep for merging postgres Refactors out the initialization of the xorm engine and the query logic for an sql data source. * mysql: rename refactoring + test update * postgres:refactor to use SqlEngine(same as mysql) Refactored to use a common base class with the MySql data source. Other changes from the original PR: - Changed time column to be time_sec to allow other time units in the future and to be the same as MySQL - Changed integration test to test the main Query method rather than the private transformToTable method - Changed the __timeSec macro name to __timeEpoch - Renamed PostgresExecutor to PostgresQueryEndpoint Fixes #9209 (the original PR) * postgres: encrypt password on config page With some other cosmetic changes to the config page: - placeholder texts - reset button for the password after it has been encrypted. - default value for the sslmode field. * postgres: change back col name to time from time_sec * postgres mysql: remove annotation title Title has been removed from annotations * postgres: fix images for docs page * postgres mysql: fix specs
2017-10-10 08:19:14 -05:00
}
// escape single quotes and backslashes in Postgres connection string parameters.
func escape(input string) string {
return strings.ReplaceAll(strings.ReplaceAll(input, `\`, `\\`), "'", `\'`)
}
func (s *PostgresService) generateConnectionString(datasource *models.DataSource) (string, error) {
var host string
var port int
if strings.HasPrefix(datasource.Url, "/") {
host = datasource.Url
Postgres: allow providing TLS/SSL certificates as text in addition to file paths (#30353) * postgres SSL certification * add back the UI to configure SSL Authentication files by file path * add backend logic * correct unittest * mini changes * Update public/app/plugins/datasource/postgres/config_ctrl.ts Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> * mutex * check file exist before remove * change permission * change default configuremethod to file-path * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * rename sslconfiguremethod to sslconfigurationmethod * frontend update * solve comments * Postgres: Convert tests to stdlib Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Postgres: Be consistent about TLS/SSL terminology Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * fix init inconsistancy * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * naming convention * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Undo change Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix TLS issue Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * change permissions * Fix data source field names Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Clean up HTML Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Improve popover text Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix SSL input bug Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Undo unnecessary change Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Clean up backend code Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix build Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * More consistent naming Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Clean up code Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Enforce certificate file permissions Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * add settings * Undo changes Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * fix windows file path * PostgresDataSource: Fix mutex usage Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Update pkg/tsdb/postgres/postgres.go Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Apply suggestions from code review Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * fix compilation * fix unittest * Apply suggestions from code review Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Apply suggestions from code review Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * mock function * change kmutex package * add kmutex into middleware * lock connection file per datasource * add unittest regarding concurrency * version should be equal * adding unittest * fix the loop * fix unitest * fix postgres unittst * remove comments * move dataPath from arg to tlsManager struct field * Use DecryptedValues method Use cached decrypted values instead of using secure json data decrypt which will decrypt unchanged values over and over again. * remove unneeded mutex in tests and cleanup tests * fix the lint Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2021-02-23 15:10:55 -06:00
s.logger.Debug("Generating connection string with Unix socket specifier", "socket", host)
} else {
sp := strings.SplitN(datasource.Url, ":", 2)
host = sp[0]
if len(sp) > 1 {
var err error
port, err = strconv.Atoi(sp[1])
if err != nil {
return "", errutil.Wrapf(err, "invalid port in host specifier %q", sp[1])
}
Postgres: allow providing TLS/SSL certificates as text in addition to file paths (#30353) * postgres SSL certification * add back the UI to configure SSL Authentication files by file path * add backend logic * correct unittest * mini changes * Update public/app/plugins/datasource/postgres/config_ctrl.ts Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> * mutex * check file exist before remove * change permission * change default configuremethod to file-path * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * rename sslconfiguremethod to sslconfigurationmethod * frontend update * solve comments * Postgres: Convert tests to stdlib Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Postgres: Be consistent about TLS/SSL terminology Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * fix init inconsistancy * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * naming convention * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Undo change Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix TLS issue Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * change permissions * Fix data source field names Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Clean up HTML Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Improve popover text Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix SSL input bug Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Undo unnecessary change Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Clean up backend code Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix build Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * More consistent naming Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Clean up code Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Enforce certificate file permissions Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * add settings * Undo changes Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * fix windows file path * PostgresDataSource: Fix mutex usage Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Update pkg/tsdb/postgres/postgres.go Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Apply suggestions from code review Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * fix compilation * fix unittest * Apply suggestions from code review Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Apply suggestions from code review Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * mock function * change kmutex package * add kmutex into middleware * lock connection file per datasource * add unittest regarding concurrency * version should be equal * adding unittest * fix the loop * fix unitest * fix postgres unittst * remove comments * move dataPath from arg to tlsManager struct field * Use DecryptedValues method Use cached decrypted values instead of using secure json data decrypt which will decrypt unchanged values over and over again. * remove unneeded mutex in tests and cleanup tests * fix the lint Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2021-02-23 15:10:55 -06:00
s.logger.Debug("Generating connection string with network host/port pair", "host", host, "port", port)
} else {
Postgres: allow providing TLS/SSL certificates as text in addition to file paths (#30353) * postgres SSL certification * add back the UI to configure SSL Authentication files by file path * add backend logic * correct unittest * mini changes * Update public/app/plugins/datasource/postgres/config_ctrl.ts Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> * mutex * check file exist before remove * change permission * change default configuremethod to file-path * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * rename sslconfiguremethod to sslconfigurationmethod * frontend update * solve comments * Postgres: Convert tests to stdlib Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Postgres: Be consistent about TLS/SSL terminology Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * fix init inconsistancy * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * naming convention * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Undo change Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix TLS issue Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * change permissions * Fix data source field names Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Clean up HTML Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Improve popover text Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix SSL input bug Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Undo unnecessary change Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Clean up backend code Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix build Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * More consistent naming Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Clean up code Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Enforce certificate file permissions Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * add settings * Undo changes Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * fix windows file path * PostgresDataSource: Fix mutex usage Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Update pkg/tsdb/postgres/postgres.go Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Apply suggestions from code review Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * fix compilation * fix unittest * Apply suggestions from code review Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Apply suggestions from code review Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * mock function * change kmutex package * add kmutex into middleware * lock connection file per datasource * add unittest regarding concurrency * version should be equal * adding unittest * fix the loop * fix unitest * fix postgres unittst * remove comments * move dataPath from arg to tlsManager struct field * Use DecryptedValues method Use cached decrypted values instead of using secure json data decrypt which will decrypt unchanged values over and over again. * remove unneeded mutex in tests and cleanup tests * fix the lint Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2021-02-23 15:10:55 -06:00
s.logger.Debug("Generating connection string with network host", "host", host)
}
}
Postgres: allow providing TLS/SSL certificates as text in addition to file paths (#30353) * postgres SSL certification * add back the UI to configure SSL Authentication files by file path * add backend logic * correct unittest * mini changes * Update public/app/plugins/datasource/postgres/config_ctrl.ts Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> * mutex * check file exist before remove * change permission * change default configuremethod to file-path * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * rename sslconfiguremethod to sslconfigurationmethod * frontend update * solve comments * Postgres: Convert tests to stdlib Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Postgres: Be consistent about TLS/SSL terminology Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * fix init inconsistancy * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * naming convention * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Undo change Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix TLS issue Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * change permissions * Fix data source field names Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Clean up HTML Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Improve popover text Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix SSL input bug Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Undo unnecessary change Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Clean up backend code Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix build Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * More consistent naming Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Clean up code Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Enforce certificate file permissions Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * add settings * Undo changes Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * fix windows file path * PostgresDataSource: Fix mutex usage Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Update pkg/tsdb/postgres/postgres.go Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Apply suggestions from code review Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * fix compilation * fix unittest * Apply suggestions from code review Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Apply suggestions from code review Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * mock function * change kmutex package * add kmutex into middleware * lock connection file per datasource * add unittest regarding concurrency * version should be equal * adding unittest * fix the loop * fix unitest * fix postgres unittst * remove comments * move dataPath from arg to tlsManager struct field * Use DecryptedValues method Use cached decrypted values instead of using secure json data decrypt which will decrypt unchanged values over and over again. * remove unneeded mutex in tests and cleanup tests * fix the lint Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2021-02-23 15:10:55 -06:00
connStr := fmt.Sprintf("user='%s' password='%s' host='%s' dbname='%s'",
escape(datasource.User), escape(datasource.DecryptedPassword()), escape(host), escape(datasource.Database))
if port > 0 {
connStr += fmt.Sprintf(" port=%d", port)
}
Postgres: allow providing TLS/SSL certificates as text in addition to file paths (#30353) * postgres SSL certification * add back the UI to configure SSL Authentication files by file path * add backend logic * correct unittest * mini changes * Update public/app/plugins/datasource/postgres/config_ctrl.ts Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> * mutex * check file exist before remove * change permission * change default configuremethod to file-path * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * rename sslconfiguremethod to sslconfigurationmethod * frontend update * solve comments * Postgres: Convert tests to stdlib Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Postgres: Be consistent about TLS/SSL terminology Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * fix init inconsistancy * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * naming convention * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Undo change Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix TLS issue Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * change permissions * Fix data source field names Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Clean up HTML Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Improve popover text Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix SSL input bug Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Undo unnecessary change Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Clean up backend code Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix build Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * More consistent naming Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Clean up code Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Enforce certificate file permissions Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * add settings * Undo changes Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * fix windows file path * PostgresDataSource: Fix mutex usage Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Update pkg/tsdb/postgres/postgres.go Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Apply suggestions from code review Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * fix compilation * fix unittest * Apply suggestions from code review Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Apply suggestions from code review Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * mock function * change kmutex package * add kmutex into middleware * lock connection file per datasource * add unittest regarding concurrency * version should be equal * adding unittest * fix the loop * fix unitest * fix postgres unittst * remove comments * move dataPath from arg to tlsManager struct field * Use DecryptedValues method Use cached decrypted values instead of using secure json data decrypt which will decrypt unchanged values over and over again. * remove unneeded mutex in tests and cleanup tests * fix the lint Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2021-02-23 15:10:55 -06:00
tlsSettings, err := s.tlsManager.getTLSSettings(datasource)
if err != nil {
return "", err
}
Postgres: allow providing TLS/SSL certificates as text in addition to file paths (#30353) * postgres SSL certification * add back the UI to configure SSL Authentication files by file path * add backend logic * correct unittest * mini changes * Update public/app/plugins/datasource/postgres/config_ctrl.ts Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> * mutex * check file exist before remove * change permission * change default configuremethod to file-path * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * rename sslconfiguremethod to sslconfigurationmethod * frontend update * solve comments * Postgres: Convert tests to stdlib Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Postgres: Be consistent about TLS/SSL terminology Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * fix init inconsistancy * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * naming convention * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Undo change Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix TLS issue Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * change permissions * Fix data source field names Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Clean up HTML Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Improve popover text Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix SSL input bug Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Undo unnecessary change Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Clean up backend code Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix build Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * More consistent naming Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Clean up code Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Enforce certificate file permissions Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * add settings * Undo changes Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * fix windows file path * PostgresDataSource: Fix mutex usage Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Update pkg/tsdb/postgres/postgres.go Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Apply suggestions from code review Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * fix compilation * fix unittest * Apply suggestions from code review Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Apply suggestions from code review Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * mock function * change kmutex package * add kmutex into middleware * lock connection file per datasource * add unittest regarding concurrency * version should be equal * adding unittest * fix the loop * fix unitest * fix postgres unittst * remove comments * move dataPath from arg to tlsManager struct field * Use DecryptedValues method Use cached decrypted values instead of using secure json data decrypt which will decrypt unchanged values over and over again. * remove unneeded mutex in tests and cleanup tests * fix the lint Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2021-02-23 15:10:55 -06:00
connStr += fmt.Sprintf(" sslmode='%s'", escape(tlsSettings.Mode))
// Attach root certificate if provided
if tlsSettings.RootCertFile != "" {
s.logger.Debug("Setting server root certificate", "tlsRootCert", tlsSettings.RootCertFile)
connStr += fmt.Sprintf(" sslrootcert='%s'", escape(tlsSettings.RootCertFile))
}
// Attach client certificate and key if both are provided
if tlsSettings.CertFile != "" && tlsSettings.CertKeyFile != "" {
s.logger.Debug("Setting TLS/SSL client auth", "tlsCert", tlsSettings.CertFile, "tlsKey", tlsSettings.CertKeyFile)
connStr += fmt.Sprintf(" sslcert='%s' sslkey='%s'", escape(tlsSettings.CertFile), escape(tlsSettings.CertKeyFile))
} else if tlsSettings.CertFile != "" || tlsSettings.CertKeyFile != "" {
return "", fmt.Errorf("TLS/SSL client certificate and key must both be specified")
}
Postgres: allow providing TLS/SSL certificates as text in addition to file paths (#30353) * postgres SSL certification * add back the UI to configure SSL Authentication files by file path * add backend logic * correct unittest * mini changes * Update public/app/plugins/datasource/postgres/config_ctrl.ts Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> * mutex * check file exist before remove * change permission * change default configuremethod to file-path * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Update public/app/plugins/datasource/postgres/partials/config.html Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * rename sslconfiguremethod to sslconfigurationmethod * frontend update * solve comments * Postgres: Convert tests to stdlib Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Postgres: Be consistent about TLS/SSL terminology Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * fix init inconsistancy * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * naming convention * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Undo change Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix TLS issue Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * change permissions * Fix data source field names Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Clean up HTML Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Improve popover text Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix SSL input bug Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Undo unnecessary change Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Clean up backend code Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix build Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * More consistent naming Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Clean up code Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Enforce certificate file permissions Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * add settings * Undo changes Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * fix windows file path * PostgresDataSource: Fix mutex usage Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Update pkg/tsdb/postgres/postgres.go Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Apply suggestions from code review Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * fix compilation * fix unittest * Apply suggestions from code review Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * Apply suggestions from code review Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * mock function * change kmutex package * add kmutex into middleware * lock connection file per datasource * add unittest regarding concurrency * version should be equal * adding unittest * fix the loop * fix unitest * fix postgres unittst * remove comments * move dataPath from arg to tlsManager struct field * Use DecryptedValues method Use cached decrypted values instead of using secure json data decrypt which will decrypt unchanged values over and over again. * remove unneeded mutex in tests and cleanup tests * fix the lint Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2021-02-23 15:10:55 -06:00
s.logger.Debug("Generated Postgres connection string successfully")
return connStr, nil
Postgres Data Source (#9475) * add postgresql datasource * add rest of files for postgres datasource * fix timeseries query, remove unused code * consistent naming, refactoring * s/mysql/postgres/ * s/mysql/postgres/ * couple more tests * tests for more datatypes * fix macros for postgres * add __timeSec macro * add frontend for postgres datasource * adjust documentation * fix formatting * add proper plugin description * merge editor changes from mysql * port changes from mysql datasource * set proper defaultQuery for postgres * add time_sec to timeseries query accept int for value for timeseries query * revert allowing time_sec and handle int or float values as unix timestamp for "time" column * fix tslint error * handle decimal values in timeseries query * allow setting sslmode for postgres datasource * use type switch for handling data types * fix value for timeseries query * refactor timeseries queries to make them more flexible * remove debug statement from inner loop in type conversion * use plain for loop in getTypedRowData * fix timeseries queries * adjust postgres datasource to tsdb refactoring * adjust postgres datasource to frontend changes * update lib/pq to latest version * move type conversion to getTypedRowData * handle address types cidr, inet and macaddr * adjust response parser and docs for annotations * convert unknown types to string * add documentation for postgres datasource * add another example query with metric column * set more helpful default query * update help text in query editor * handle NULL in value column of timeseries query * add __timeGroup macro * add test for __timeGroup macro * document __timeGroup and set proper default query for annotations * fix typos in docs * add postgres to list of datasources * add postgres to builtInPlugins * mysql: refactoring as prep for merging postgres Refactors out the initialization of the xorm engine and the query logic for an sql data source. * mysql: rename refactoring + test update * postgres:refactor to use SqlEngine(same as mysql) Refactored to use a common base class with the MySql data source. Other changes from the original PR: - Changed time column to be time_sec to allow other time units in the future and to be the same as MySQL - Changed integration test to test the main Query method rather than the private transformToTable method - Changed the __timeSec macro name to __timeEpoch - Renamed PostgresExecutor to PostgresQueryEndpoint Fixes #9209 (the original PR) * postgres: encrypt password on config page With some other cosmetic changes to the config page: - placeholder texts - reset button for the password after it has been encrypted. - default value for the sslmode field. * postgres: change back col name to time from time_sec * postgres mysql: remove annotation title Title has been removed from annotations * postgres: fix images for docs page * postgres mysql: fix specs
2017-10-10 08:19:14 -05:00
}
type postgresQueryResultTransformer struct {
2018-07-26 11:10:17 -05:00
log log.Logger
Postgres Data Source (#9475) * add postgresql datasource * add rest of files for postgres datasource * fix timeseries query, remove unused code * consistent naming, refactoring * s/mysql/postgres/ * s/mysql/postgres/ * couple more tests * tests for more datatypes * fix macros for postgres * add __timeSec macro * add frontend for postgres datasource * adjust documentation * fix formatting * add proper plugin description * merge editor changes from mysql * port changes from mysql datasource * set proper defaultQuery for postgres * add time_sec to timeseries query accept int for value for timeseries query * revert allowing time_sec and handle int or float values as unix timestamp for "time" column * fix tslint error * handle decimal values in timeseries query * allow setting sslmode for postgres datasource * use type switch for handling data types * fix value for timeseries query * refactor timeseries queries to make them more flexible * remove debug statement from inner loop in type conversion * use plain for loop in getTypedRowData * fix timeseries queries * adjust postgres datasource to tsdb refactoring * adjust postgres datasource to frontend changes * update lib/pq to latest version * move type conversion to getTypedRowData * handle address types cidr, inet and macaddr * adjust response parser and docs for annotations * convert unknown types to string * add documentation for postgres datasource * add another example query with metric column * set more helpful default query * update help text in query editor * handle NULL in value column of timeseries query * add __timeGroup macro * add test for __timeGroup macro * document __timeGroup and set proper default query for annotations * fix typos in docs * add postgres to list of datasources * add postgres to builtInPlugins * mysql: refactoring as prep for merging postgres Refactors out the initialization of the xorm engine and the query logic for an sql data source. * mysql: rename refactoring + test update * postgres:refactor to use SqlEngine(same as mysql) Refactored to use a common base class with the MySql data source. Other changes from the original PR: - Changed time column to be time_sec to allow other time units in the future and to be the same as MySQL - Changed integration test to test the main Query method rather than the private transformToTable method - Changed the __timeSec macro name to __timeEpoch - Renamed PostgresExecutor to PostgresQueryEndpoint Fixes #9209 (the original PR) * postgres: encrypt password on config page With some other cosmetic changes to the config page: - placeholder texts - reset button for the password after it has been encrypted. - default value for the sslmode field. * postgres: change back col name to time from time_sec * postgres mysql: remove annotation title Title has been removed from annotations * postgres: fix images for docs page * postgres mysql: fix specs
2017-10-10 08:19:14 -05:00
}
func (t *postgresQueryResultTransformer) TransformQueryError(err error) error {
return err
}
func (t *postgresQueryResultTransformer) GetConverterList() []sqlutil.StringConverter {
return []sqlutil.StringConverter{
{
Name: "handle FLOAT4",
InputScanKind: reflect.Interface,
InputTypeName: "FLOAT4",
ConversionFunc: func(in *string) (*string, error) { return in, nil },
Replacer: &sqlutil.StringFieldReplacer{
OutputFieldType: data.FieldTypeNullableFloat64,
ReplaceFunc: func(in *string) (interface{}, error) {
if in == nil {
return nil, nil
}
v, err := strconv.ParseFloat(*in, 64)
if err != nil {
return nil, err
}
return &v, nil
},
},
},
{
Name: "handle FLOAT8",
InputScanKind: reflect.Interface,
InputTypeName: "FLOAT8",
ConversionFunc: func(in *string) (*string, error) { return in, nil },
Replacer: &sqlutil.StringFieldReplacer{
OutputFieldType: data.FieldTypeNullableFloat64,
ReplaceFunc: func(in *string) (interface{}, error) {
if in == nil {
return nil, nil
}
v, err := strconv.ParseFloat(*in, 64)
if err != nil {
return nil, err
}
return &v, nil
},
},
},
{
Name: "handle NUMERIC",
InputScanKind: reflect.Interface,
InputTypeName: "NUMERIC",
ConversionFunc: func(in *string) (*string, error) { return in, nil },
Replacer: &sqlutil.StringFieldReplacer{
OutputFieldType: data.FieldTypeNullableFloat64,
ReplaceFunc: func(in *string) (interface{}, error) {
if in == nil {
return nil, nil
}
v, err := strconv.ParseFloat(*in, 64)
if err != nil {
return nil, err
}
return &v, nil
},
},
},
}
}