Commit Graph

157 Commits

Author SHA1 Message Date
Gábor Farkas
9f0fc60477
sql: remove xorm dependency from postgres/mysql/mssql (#77870)
* sql: remove xorm

* sql: remove pkg/util dependency (#78821)

sql: removed md5-util dependency
2023-12-06 09:35:05 +01:00
Gábor Farkas
033446523b
mysql/postgres: vendor in unit-test helper function (#78744) 2023-11-29 08:07:14 +01:00
Gábor Farkas
4f899e3576
sql: removed dependency on test-db-check (#78717) 2023-11-28 12:46:01 +01:00
Gábor Farkas
b97e485fe0
mysql: update unit test to be ready for the xorm-removal (#78064)
* mysql: update unit test to be ready for the xorm-removal

* removed unused code
2023-11-15 09:46:18 +01:00
Gábor Farkas
fe18c63204
mysql: do not import httpClient from core grafana (#77628)
mysql: do not use core-only exports
2023-11-13 10:25:19 +01:00
Will Browne
9d92818cae
Plugins: Update PDC pattern from latest plugin SDK changes (#76576)
* update with sdk

* do sql

* fix core plugins

* fix proxy settings

* bump SDK version

* tidy

* enable pdc for test

* add codeowners

* bump dep

* go mod tidy

* bump SDK
2023-10-16 16:40:04 +02:00
Dan Cech
91cf4f0c1c
Revert "Plugins: Update PDC pattern from latest plugin SDK changes" (#76565)
Revert "Plugins: Update PDC pattern from latest plugin SDK changes (#76036)"

This reverts commit 151f6d6216.
2023-10-13 12:35:16 -04:00
Will Browne
151f6d6216
Plugins: Update PDC pattern from latest plugin SDK changes (#76036)
* update with sdk

* do sql

* fix core plugins

* fix proxy settings

* bump SDK version

* tidy

* enable pdc for test

* add codeowners
2023-10-13 14:42:15 +02:00
Will Browne
7fca1bde54
Plugins: Set grafana config, plugin version and user agent on plugin requests (#75171)
* first pass

* fixup

* remove test line

* fix tests

* use new fields

* fix imports + formatting

* fix tests

* rollback changes

* undo whitespace

* apply pr feedback
2023-09-21 11:33:31 +02:00
Serge Zaitsev
93cdc94a94
Chore: capitalise logs in other backend packages (#74344)
* capitalise logs in observability logs

* capitalise oss-bit-tent packages

* capitalise logs in aws-datasources

* capitalise logs for traces and profiling

* capitalise logs for partner datasources

* capitalise logs in plugins platform

* capitalise logs for observability metrics
2023-09-04 22:25:43 +02:00
Ryan McKinley
025b2f3011
Chore: use any rather than interface{} (#74066) 2023-08-30 18:46:47 +03:00
Stephanie Hingtgen
4ece133fce
Secure socks proxy: use Grafana Plugin SDK (#71616) 2023-07-18 15:23:02 -06:00
Michael Mandrus
66d2214c3b
Config: Add configuration option to define custom user-facing general error message for certain error types (#70023)
---------

Co-authored-by: Summer Wollin <summer.wollin@grafana.com>
Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com>
Co-authored-by: Arati R. <33031346+suntala@users.noreply.github.com>
2023-06-16 10:46:47 -05:00
enginecan
6758fd4888
MySQL: Add option to allow cleartext passwords (#63232)
* Add "Allow Cleartext Passwords" checkbox to MySQL connection settings

* Fix lint issues

* Add docs

* Add line break and bold text

---------

Co-authored-by: Zoltán Bedi <zoltan.bedi@gmail.com>
2023-05-26 12:33:55 +02:00
Will Browne
286b9e08e9
Plugins: Bump Plugin SDK version and address instance management breaking changes (#68900)
* bump sdk and propagate ctx

* fix tests
2023-05-24 10:19:34 +02:00
Emil Tullstedt
b0e681c339
Chore: Update target in go.mod to 1.20 and golangci-lint to v1.52.2 (#67102) 2023-05-10 13:47:35 +02:00
Jev Forsberg
86228ba1a0
Datasource: Fix missing raw SQL query in Query Inspector when query returns zero rows (#67844)
* baldm0mma/issue5799/ add empty frame with all meta data

* baldm0mma/issue5799/ add null test

* write an integration test around query results with empty rows

* baldm0mma/issue5799/ add tests to mssql and postgres

* remove use of apparently reserved keyword in mysql8

* baldm0mma/issue5799

* baldm0mma/issue5799/ update tests

* baldm0mma/issue5799/ update test structs

* baldm0mma/issue5799/ update annotation

---------

Co-authored-by: Michael Mandrus <michael.mandrus@grafana.com>
2023-05-09 08:29:02 -06:00
Stephanie Hingtgen
255d8f3326
Plugins: Remove secure socks proxy feature toggle (#66611) 2023-04-18 11:11:23 -06:00
Kyle Cunningham
92d92187d9
SQL Datasources: Update Max Connection and Max Idle Connection Defaults to 100 and add auto mode (#65834)
* Update connection configuration for SQL datasources

* Working auto state for connection numbers

* Add migration

* Use defaults from constants file

* Remove dead code

* Add tests and restructure useMigrateDatabaseField

* Update function names

* Update docs

* Make sure we don't continually issue updates

* Update docs

* Use onOptionsChnage in ConnectionLimits

* Update docs

* Clean up docs

* Update migration

* Fix default values in docs

* Fix spacing issue

* Fix test

* Update default values for SQL connections

* Include consts

* Allow override for default SQL datasource connection parameters

* Fix linter errors

* Remove extra @ts-ignore

* Centralize logic for default values

* Remove debugging

* Remove unecessary function

* Update configuration docs

* minor suggested change

* Fix comment misspelling

* Remove unecessary default setting code

* Update docs to indicate that code was included for backport version

* Remove dead code

---------

Co-authored-by: lwandz13 <larissa.wandzura@grafana.com>
2023-04-17 11:44:05 +03:00
Stephanie Hingtgen
10db808ea1
Plugins: Add sql support for the secure socks proxy (#64630) 2023-03-16 06:54:19 -06:00
Zoltán Bedi
c25fe34ac0
SQL Datasources: Use health check for config test (#59867)
* SQL Datasources: Use health check for config test

* Remove unnecessary test

* Fix test errors

* Revert mysql go driver update

* Use transform query error

* Use TransformQueryError from sql_engine
2022-12-19 17:17:52 +01:00
Zoltán Bedi
d65899de7b
SQL Datasources: Move database setting to jsonData (#58649)
* Datasource settings: Add deprecation notice for database field

* SQL Datasources: Migrate from settings.database to settings.jsonData.database

* Check jsonData first

* Remove comment from docs
2022-12-13 09:56:52 +01:00
Sasha Melentyev
c02003af3c
Refactor time durations (#58484)
This change uses `time.Second` in place of `1000 * time.Millisecond` and `time.Minute` in place of `60*time.Second`.
2022-11-22 15:09:15 +08:00
Yuriy Tseretyan
d9c40ca41e
Chore: Update sqleng, elasticsearch, tempo and opentsdb plugins to support contextual logs. (#57777)
* make sql engine use pick log context for logs
* update tempo to get log context
* update opentsdb to use log context
* update es client to use log context
2022-11-02 10:03:50 -04:00
Kristin Laemmert
05709ce411
chore: remove sqlstore & mockstore dependencies from (most) packages (#57087)
* chore: add alias for InitTestDB and Session

Adds an alias for the sqlstore InitTestDB and Session, and updates tests using these to reduce dependencies on the sqlstore.Store.

* next pass of removing sqlstore imports
* last little bit
* remove mockstore where possible
2022-10-19 09:02:15 -04:00
Kat Yang
bd35e6917a
Chore: Exclude integration tests from running on test-backend step (#50359)
* Chore: Exclude integration tests from running on test-backend step

* Remove -v from go test command

* Add check to skip integration tests before each integration test

* Try to restart pipeline

* Retrying to make pipeline run
2022-06-10 11:46:21 -04:00
Kat Yang
c63ebc887b
Chore: Run integration tests without grabpl (#49448)
* Chore: Run integration tests without grabpl

* Add new step for integration tests in lib.star

* Remove old integration test step from lib.star

* Change drone signature

* Fix: Edit starlark integration step to not affect enterprise

* Remove all build tags & rename starlark integration test step

* Resync .drone.yml with .drone.star

* Fix lint errors

* Fix lint errors

* Fix lint errors

* Fix more lint errors

* Fix another lint error

* Rename integration test step

* Fix last lint error

* Recomment enterprise step

* Remove comment from Makefile

Co-authored-by: Ida Furjesova <ida.furjesova@grafana.com>
2022-06-01 14:55:22 -04:00
Kat Yang
50c2b4682a
Chore: Rename integration tests (#49438)
* Chore: Rename integration tests

* Remove one Integration

Co-authored-by: Ida Furjesova <ida.furjesova@grafana.com>
2022-05-24 11:04:03 +02:00
matt abrams
76da017d00
MySQL: Prevent pre-epoch timestamps from breaking __timeFilter (#45345)
* add logic to change timeFilter query when FROM time is pre-EPOCH

* fix timeFilter tests where FROM time is pre-EPOCH
2022-02-22 20:06:04 +01:00
Will Browne
7fbc7d019a
Plugins: Core plugins register via backend factory provider (#43171)
* refactoring store interface and init flow

* fix import

* fix linter

* refactor resource calling

* load with class

* re-order args

* fix tests

* fix linter

* remove old creator

* add custom config struct

* fix some tests

* cleanup

* fix

* tackle plugins

* fix linter

* refactor and fix test

* add connect failure error

* add fix for azure, cloud monitoring and test data

* restructure

* remove unused err

* add fake tracer for test

* fix grafana ds plugin
2022-01-20 18:16:22 +01:00
Will Browne
e4ba5f17dd
add core plugins thru store (#43085) 2021-12-14 15:22:40 +01:00
Will Browne
b80fbe03f0
Plugins: Refactor Plugin Management (#40477)
* add core plugin flow

* add instrumentation

* move func

* remove cruft

* support external backend plugins

* refactor + clean up

* remove comments

* refactor loader

* simplify core plugin path arg

* cleanup loggers

* move signature validator to plugins package

* fix sig packaging

* cleanup plugin model

* remove unnecessary plugin field

* add start+stop for pm

* fix failures

* add decommissioned state

* export fields just to get things flowing

* fix comments

* set static routes

* make image loading idempotent

* merge with backend plugin manager

* re-use funcs

* reorder imports + remove unnecessary interface

* add some TODOs + remove unused func

* remove unused instrumentation func

* simplify client usage

* remove import alias

* re-use backendplugin.Plugin interface

* re order funcs

* improve var name

* fix log statements

* refactor data model

* add logic for dupe check during loading

* cleanup state setting

* refactor loader

* cleanup manager interface

* add rendering flow

* refactor loading + init

* add renderer support

* fix renderer plugin

* reformat imports

* track errors

* fix plugin signature inheritance

* name param in interface

* update func comment

* fix func arg name

* introduce class concept

* remove func

* fix external plugin check

* apply changes from pm-experiment

* fix core plugins

* fix imports

* rename interface

* comment API interface

* add support for testdata plugin

* enable alerting + use correct core plugin contracts

* slim manager API

* fix param name

* fix filter

* support static routes

* fix rendering

* tidy rendering

* get tests compiling

* fix install+uninstall

* start finder test

* add finder test coverage

* start loader tests

* add test for core plugins

* load core + bundled test

* add test for nested plugin loading

* add test files

* clean interface + fix registering some core plugins

* refactoring

* reformat and create sub packages

* simplify core plugin init

* fix ctx cancel scenario

* migrate initializer

* remove Init() funcs

* add test starter

* new logger

* flesh out initializer tests

* refactoring

* remove unused svc

* refactor rendering flow

* fixup loader tests

* add enabled helper func

* fix logger name

* fix data fetchers

* fix case where plugin dir doesn't exist

* improve coverage + move dupe checking to loader

* remove noisy debug logs

* register core plugins automagically

* add support for renderer in catalog

* make private func + fix req validation

* use interface

* re-add check for renderer in catalog

* tidy up from moving to auto reg core plugins

* core plugin registrar

* guards

* copy over core plugins for test infra

* all tests green

* renames

* propagate new interfaces

* kill old manager

* get compiling

* tidy up

* update naming

* refactor manager test + cleanup

* add more cases to finder test

* migrate validator to field

* more coverage

* refactor dupe checking

* add test for plugin class

* add coverage for initializer

* split out rendering

* move

* fixup tests

* fix uss test

* fix frontend settings

* fix grafanads test

* add check when checking sig errors

* fix enabled map

* fixup

* allow manual setup of CM

* rename to cloud-monitoring

* remove TODO

* add installer interface for testing

* loader interface returns

* tests passing

* refactor + add more coverage

* support 'stackdriver'

* fix frontend settings loading

* improve naming based on package name

* small tidy

* refactor test

* fix renderer start

* make cloud-monitoring plugin ID clearer

* add plugin update test

* add integration tests

* don't break all if sig can't be calculated

* add root URL check test

* add more signature verification tests

* update DTO name

* update enabled plugins comment

* update comments

* fix linter

* revert fe naming change

* fix errors endpoint

* reset error code field name

* re-order test to help verify

* assert -> require

* pm check

* add missing entry + re-order

* re-check

* dump icon log

* verify manager contents first

* reformat

* apply PR feedback

* apply style changes

* fix one vs all loading err

* improve log output

* only start when no signature error

* move log

* rework plugin update check

* fix test

* fix multi loading from cfg.PluginSettings

* improve log output #2

* add error abstraction to capture errors without registering a plugin

* add debug log

* add unsigned warning

* e2e test attempt

* fix logger

* set home path

* prevent panic

* alternate

* ugh.. fix home path

* return renderer even if not started

* make renderer plugin managed

* add fallback renderer icon, update renderer badge + prevent changes when renderer is installed

* fix icon loading

* rollback renderer changes

* use correct field

* remove unneccessary block

* remove newline

* remove unused func

* fix bundled plugins base + module fields

* remove unused field since refactor

* add authorizer abstraction

* loader only returns plugins expected to run

* fix multi log output
2021-11-01 10:53:33 +01:00
Serge Zaitsev
249856e527
Chore: Refactor GoConvey in mysql package tests (#40855) 2021-10-26 13:18:17 +02:00
Joan López de la Franca Beltran
722c414fef
Encryption: Refactor securejsondata.SecureJsonData to stop relying on global functions (#38865)
* Encryption: Add support to encrypt/decrypt sjd

* Add datasources.Service as a proxy to datasources db operations

* Encrypt ds.SecureJsonData before calling SQLStore

* Move ds cache code into ds service

* Fix tlsmanager tests

* Fix pluginproxy tests

* Remove some securejsondata.GetEncryptedJsonData usages

* Add pluginsettings.Service as a proxy for plugin settings db operations

* Add AlertNotificationService as a proxy for alert notification db operations

* Remove some securejsondata.GetEncryptedJsonData usages

* Remove more securejsondata.GetEncryptedJsonData usages

* Fix lint errors

* Minor fixes

* Remove encryption global functions usages from ngalert

* Fix lint errors

* Minor fixes

* Minor fixes

* Remove securejsondata.DecryptedValue usage

* Refactor the refactor

* Remove securejsondata.DecryptedValue usage

* Move securejsondata to migrations package

* Move securejsondata to migrations package

* Minor fix

* Fix integration test

* Fix integration tests

* Undo undesired changes

* Fix tests

* Add context.Context into encryption methods

* Fix tests

* Fix tests

* Fix tests

* Trigger CI

* Fix test

* Add names to params of encryption service interface

* Remove bus from CacheServiceImpl

* Add logging

* Add keys to logger

Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>

* Add missing key to logger

Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>

* Undo changes in markdown files

* Fix formatting

* Add context to secrets service

* Rename decryptSecureJsonData to decryptSecureJsonDataFn

* Name args in GetDecryptedValueFn

* Add template back to NewAlertmanagerNotifier

* Copy GetDecryptedValueFn to ngalert

* Add logging to pluginsettings

* Fix pluginsettings test

Co-authored-by: Tania B <yalyna.ts@gmail.com>
Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>
2021-10-07 17:33:50 +03:00
Andres Martinez Gotor
64c8d32fe7
Use sdk pkg for gtime (#39354) 2021-09-21 13:08:52 +02:00
Marcus Efraimsson
1c892a2fc4
Postgres/MySQL/MSSQL: Add setting to limit maximum amount of rows processed (#38986)
Adds a new setting dataproxy.row_limit that allows an operator to limit the 
amount of rows being processed/accepted in response to database queries 
originating from SQL data sources.

Closes #38975
Ref #39095

Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>
2021-09-13 15:27:51 +02:00
Marcus Efraimsson
fbdaf56a84
Postgres/MySQL/MSSQL: Fix region annotations not displayed correctly (#38936)
Fix region annotations not displayed correctly when returning timeend column 
as epoch timestamp and by that making sure that the returned data frame field 
named timeend is treated as time type.

Fixes #38533
2021-09-08 01:54:48 +02:00
ying-jeanne
c989f0fbbe
SQL: Migrate to use SDK contracts (#36635)
* convert SQLs to use sdk contracts

* make draft

* postgres

* intermedia

* get datasourceinfo filled at the beginning of the service

* move the interval into package because of cyclict  import and fix all postgres tests

* fix mysql test

* fix mssql

* fix the test for pr https://github.com/grafana/grafana/issues/35839

* fix some issue about intervalv2 package

* update sql test

* wire migration for SQLs

* add sqls to the background process

* make it register instead of register and start

* revert formatting

* fix tests

* fix linter

* remove integration test

* Postgres test fix

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2021-09-07 09:35:37 +02:00
Serge Zaitsev
643c7fa0cb
Chore: update all +build statements (#38782) 2021-09-01 17:38:56 +03:00
Marcus Efraimsson
10c892fa5b
Postgres/MySQL/MSSQL: Fix name of time field should be named Time for time series queries (#36720)
Name of time field changed in v8 for time series queries from Time to the name of the selected 
time column, i.e. time or time_sec. These changes should make sure that the name of time field 
is always returned as Time for time series queries.

Fixes #36059

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
2021-07-14 11:29:51 +02:00
ying-jeanne
73fdd9dc22
[MYSQL] convert explicitely smallint and tinyint (#35897)
* [MYSQL] convert explicitely smallint and tinyint

* clean up code

* fix comments
2021-06-22 00:10:45 +08:00
Ryan McKinley
1d6e99bc95
SQL: Fixes issues with showing value column name prefix in legends (#35839)
* rename strategy

* Update pkg/tsdb/sqleng/sql_engine.go

Co-authored-by: Torkel Ödegaard <torkel@grafana.org>

* more strict constraints

* Fixed so that it works on multi series results

* only apply the logic when original query returns 3 fields

* removed part of comment

* Update mysql test

Co-authored-by: Torkel Ödegaard <torkel@grafana.org>
Co-authored-by: Torkel Ödegaard <torkel@grafana.com>
2021-06-18 08:05:23 +02:00
Marcus Efraimsson
0611207f3b
Postgres/MySQL/MSSQL: Fix concurrent map writes panic (#35510)
Fixes panic/fatal error concurrent map writes in SQL data sources when multiple 
queries are executed concurrently and you interpolate SQL query before executing it.

Fixes #35469
2021-06-11 14:56:29 +02:00
Andi Pabst
29be741f26
MySQL Datasource: Add timezone parameter (#27535)
* Add MySQL timezone parameter

* Reword docs/sources/features/datasources/mysql.md

Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com>

* update documentation

* fix formatting mistake from merge

* enhance documentation with reason for parameter usage

* enhance documentation with reason for parameter usage

Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com>
2021-05-27 10:03:03 +02:00
Marcus Efraimsson
348e76fc8e
Datasource: Shared HTTP client provider for core backend data sources and any data source using the data source proxy (#33439)
Uses new httpclient package from grafana-plugin-sdk-go introduced 
via grafana/grafana-plugin-sdk-go#328. 
Replaces the GetHTTPClient, GetTransport, GetTLSConfig methods defined 
on DataSource model.
Longer-term the goal is to migrate core HTTP backend data sources to use the 
SDK contracts and using httpclient.Provider for creating HTTP clients and such.

Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>
2021-05-19 23:53:41 +02:00
Marcus Efraimsson
6c728d6a78
SQL: Use SDK sqlutil package refactor (#33758)
Upgrades grafana-plugin-sdk-go dependency to v0.94.0 to get 
the refactored sqlutil package. 

Co-authored-by: Kevin Minehart <kmineh0151@gmail.com>
2021-05-11 14:59:33 +02:00
ying-jeanne
bd66c8dde3
SQL data sources: Convert to return data frames (#32257)
Convert SQL data sources to return data frames.

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>
Co-authored-by: Will Browne <will.browne@grafana.com>
Co-authored-by: Hugo Häggmark <hugo.haggmark@gmail.com>
2021-05-05 16:46:07 +02:00
Ryan McKinley
7627b55ef4
TSDB: add deprecation comments to many tsdb structs (#33281) 2021-04-23 03:03:11 +02:00
Arve Knudsen
b79e61656a
Introduce TSDB service (#31520)
* Introduce TSDB service

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

Co-authored-by: Erik Sundell <erik.sundell87@gmail.com>
Co-authored-by: Will Browne <will.browne@grafana.com>
Co-authored-by: Torkel Ödegaard <torkel@grafana.org>
Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
Co-authored-by: Zoltán Bedi <zoltan.bedi@gmail.com>
2021-03-08 07:02:49 +01:00
Arve Knudsen
9593d57914
Chore: Enable errorlint linter (#29227)
* Enable errorlint linter
* Handle wrapped errors

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>
2020-11-19 14:47:17 +01:00