Commit Graph

173 Commits

Author SHA1 Message Date
Sven Klemm
94d6d51726 move timeFrom, timeTo, unixEpochFrom and unixEpochTo macros to sql_engine 2018-10-02 20:19:34 +02:00
Sven Klemm
0254a29e35 Interpolate $__interval in backend for alerting with sql datasources (#13156)
add support for interpolate $__interval and  $__interval_ms in sql datasources
2018-09-13 16:51:00 +02:00
Sven Klemm
fbc67a1c64 add $__unixEpochGroup to mysql datasource 2018-08-13 12:17:05 +02:00
Sven Klemm
d81a23becf Refactor setting fillmode
This adds SetupFillmode to the tsdb package to be used by the sql
datasources.
2018-08-12 10:51:58 +02:00
Sven Klemm
ee7602ec1f change fillmode from last to previous 2018-08-07 21:01:41 +02:00
Sven Klemm
bfc66a7ed0 add fillmode "last" to sql datasource
This adds a new fill mode last (last observation carried forward) for grafana
to the sql datasources. This fill mode will fill in the last seen value in a
series when a timepoint is missing or NULL if no value for that series has
been seen yet.
2018-08-02 20:35:09 +02:00
Sven Klemm
b71d10a7a4 add $__timeGroupAlias to mysql and mssql 2018-08-01 20:58:51 +02:00
Sven Klemm
e487fabcd5 add metric column prefix test for mysql 2018-07-27 17:59:59 +02:00
Marcus Efraimsson
27db454012
mysql: use new sql engine 2018-07-26 18:10:45 +02:00
Marcus Efraimsson
d90d4ce55d
Merge pull request #12402 from AustinWinstanley/macros/sql
mysql,mssql: improve $__timeFilter, $__timeFrom, $__timeTo macros

The $__timeFilter macro now uses a BETWEEN with timestamps instead of 
comparing epochs
The $__timeFrom and $__timeTo macros now return strings of format 
YYYY-MM-DDThh:mm:ssZ instead of epoch
2018-07-04 20:01:43 +02:00
Marcus Efraimsson
d2f31a716f
remove unnecessary conversions 2018-07-04 12:16:39 +02:00
Sven Klemm
1601f6d17c [mysql] fix $__timeGroup rounding (#12469)
* fix $__timeGroup rounding for mysql

* revert accidentally commented out line
2018-07-01 06:59:05 -07:00
Austin Winstanley
a0e8437f8f Used PostgreSQL TSDB as a model the set up the __timeFilter, __timeFrom, and __timeTo macros for Microsoft SQL and MySQL 2018-06-29 03:48:14 +00:00
Austin Winstanley
105b3d6804 Switched MySQL and MSSQL macros for timeFilter and related to use BETWEEN and calculate UNIX time server side instead of database side. Fixes #11618 #11619 2018-06-25 20:11:58 +00:00
Leonard Gram
50d1519a91 build: mysql integration testing on ci. 2018-05-30 16:27:47 +02:00
Marcus Efraimsson
de0d409a23
Revert "Opportunities to unindent code (unindent)" 2018-05-02 14:06:46 +02:00
Karsten Weiss
893a91af3a Use opportunities to unindent code (unindent)
This commit fixes the following unindent findings:
pkg/api/common.go:102:2: "if x { if y" should be "if x && y"
pkg/components/dynmap/dynmap.go:642:2: invert condition and early return
pkg/components/dynmap/dynmap.go:681:2: invert condition and early return
pkg/components/simplejson/simplejson.go:171:2: "if x { if y" should be "if x && y"
pkg/middleware/dashboard_redirect.go:42:3: invert condition and early return
pkg/tsdb/mssql/mssql.go:301:3: invert condition and early break
pkg/tsdb/mysql/mysql.go:312:3: invert condition and early break
pkg/tsdb/postgres/postgres.go:292:3: invert condition and early break
pkg/tsdb/sql_engine.go:144:2: invert condition and early return
2018-04-28 10:53:16 +02:00
Marcus Efraimsson
346577b664
mysql: fix value columns conversion to float when using timeseries query 2018-04-24 19:53:06 +02:00
Karsten Weiss
f61e69ce75 Simplify comparison to bool constant (gosimple)
This fixes:
build.go:553:6: should omit comparison to bool constant, can be simplified to !strings.Contains(path, ".sha256") (S1002)
pkg/cmd/grafana-cli/commands/ls_command.go:27:5: should omit comparison to bool constant, can be simplified to !pluginDirInfo.IsDir() (S1002)
pkg/components/dynmap/dynmap_test.go:24:5: should omit comparison to bool constant, can be simplified to !value (S1002)
pkg/components/dynmap/dynmap_test.go:122:14: should omit comparison to bool constant, can be simplified to b (S1002)
pkg/components/dynmap/dynmap_test.go:125:14: should omit comparison to bool constant, can be simplified to !b (S1002)
pkg/components/dynmap/dynmap_test.go:128:14: should omit comparison to bool constant, can be simplified to !b (S1002)
pkg/models/org_user.go:51:5: should omit comparison to bool constant, can be simplified to !(*r).IsValid() (S1002)
pkg/plugins/datasource/wrapper/datasource_plugin_wrapper_test.go:77:12: should omit comparison to bool constant, can be simplified to !haveBool (S1002)
pkg/services/alerting/conditions/evaluator.go:23:9: should omit comparison to bool constant, can be simplified to !reducedValue.Valid (S1002)
pkg/services/alerting/conditions/evaluator.go:48:5: should omit comparison to bool constant, can be simplified to !reducedValue.Valid (S1002)
pkg/services/alerting/conditions/evaluator.go:91:5: should omit comparison to bool constant, can be simplified to !reducedValue.Valid (S1002)
pkg/services/alerting/conditions/query.go:56:6: should omit comparison to bool constant, can be simplified to !reducedValue.Valid (S1002)
pkg/services/alerting/extractor.go:107:20: should omit comparison to bool constant, can be simplified to !enabled.MustBool() (S1002)
pkg/services/alerting/notifiers/telegram.go:222:41: should omit comparison to bool constant, can be simplified to this.UploadImage (S1002)
pkg/services/sqlstore/apikey.go:58:12: should omit comparison to bool constant, can be simplified to !has (S1002)
pkg/services/sqlstore/apikey.go:72:12: should omit comparison to bool constant, can be simplified to !has (S1002)
pkg/services/sqlstore/dashboard.go:66:33: should omit comparison to bool constant, can be simplified to !cmd.Overwrite (S1002)
pkg/services/sqlstore/dashboard.go:175:12: should omit comparison to bool constant, can be simplified to !has (S1002)
pkg/services/sqlstore/dashboard.go:311:13: should omit comparison to bool constant, can be simplified to !has (S1002)
pkg/services/sqlstore/dashboard.go:444:12: should omit comparison to bool constant, can be simplified to !exists (S1002)
pkg/services/sqlstore/dashboard.go:472:12: should omit comparison to bool constant, can be simplified to !exists (S1002)
pkg/services/sqlstore/dashboard.go:554:32: should omit comparison to bool constant, can be simplified to !cmd.Overwrite (S1002)
pkg/services/sqlstore/dashboard_snapshot.go:83:12: should omit comparison to bool constant, can be simplified to !has (S1002)
pkg/services/sqlstore/plugin_setting.go:39:12: should omit comparison to bool constant, can be simplified to !has (S1002)
pkg/services/sqlstore/quota.go:34:12: should omit comparison to bool constant, can be simplified to !has (S1002)
pkg/services/sqlstore/quota.go:111:6: should omit comparison to bool constant, can be simplified to !has (S1002)
pkg/services/sqlstore/quota.go:136:12: should omit comparison to bool constant, can be simplified to !has (S1002)
pkg/services/sqlstore/quota.go:213:6: should omit comparison to bool constant, can be simplified to !has (S1002)
pkg/services/sqlstore/temp_user.go:129:12: should omit comparison to bool constant, can be simplified to !has (S1002)
pkg/services/sqlstore/user.go:157:12: should omit comparison to bool constant, can be simplified to !has (S1002)
pkg/services/sqlstore/user.go:182:5: should omit comparison to bool constant, can be simplified to !has (S1002)
pkg/services/sqlstore/user.go:191:12: should omit comparison to bool constant, can be simplified to !has (S1002)
pkg/services/sqlstore/user.go:212:12: should omit comparison to bool constant, can be simplified to !has (S1002)
pkg/services/sqlstore/user.go:307:12: should omit comparison to bool constant, can be simplified to !has (S1002)
pkg/social/generic_oauth.go:185:5: should omit comparison to bool constant, can be simplified to !s.extractToken(&data, token) (S1002)
pkg/tsdb/mssql/mssql.go:148:39: should omit comparison to bool constant, can be simplified to ok (S1002)
pkg/tsdb/mssql/mssql.go:212:6: should omit comparison to bool constant, can be simplified to !query.Model.Get("fillNull").MustBool(false) (S1002)
pkg/tsdb/mssql/mssql.go:247:56: should omit comparison to bool constant, can be simplified to ok (S1002)
pkg/tsdb/mssql/mssql.go:274:7: should omit comparison to bool constant, can be simplified to !exist (S1002)
pkg/tsdb/mssql/mssql.go:282:8: should omit comparison to bool constant, can be simplified to !exist (S1002)
pkg/tsdb/mysql/mysql.go:221:6: should omit comparison to bool constant, can be simplified to !query.Model.Get("fillNull").MustBool(false) (S1002)
pkg/tsdb/mysql/mysql.go:256:56: should omit comparison to bool constant, can be simplified to ok (S1002)
pkg/tsdb/mysql/mysql.go:283:7: should omit comparison to bool constant, can be simplified to !exist (S1002)
pkg/tsdb/mysql/mysql.go:291:8: should omit comparison to bool constant, can be simplified to !exist (S1002)
pkg/tsdb/postgres/postgres.go:134:39: should omit comparison to bool constant, can be simplified to ok (S1002)
pkg/tsdb/postgres/postgres.go:201:6: should omit comparison to bool constant, can be simplified to !query.Model.Get("fillNull").MustBool(false) (S1002)
pkg/tsdb/postgres/postgres.go:236:56: should omit comparison to bool constant, can be simplified to ok (S1002)
pkg/tsdb/postgres/postgres.go:263:7: should omit comparison to bool constant, can be simplified to !exist (S1002)
pkg/tsdb/postgres/postgres.go:271:8: should omit comparison to bool constant, can be simplified to !exist (S1002)
2018-04-16 21:04:57 +02:00
Daniel Lee
9337972a0f sqlds: fix text in comments for tests 2018-04-16 12:55:27 +02:00
Daniel Lee
ee303c03e9
Merge pull request #11550 from grafana/sql_ts_precision
sql tsdb: fix precision and data type support for time columns
2018-04-16 11:04:46 +02:00
Marcus Efraimsson
a86ee304ff
tsdb: remove unnecessary type casts in sql data sources macro engines 2018-04-12 19:08:35 +02:00
Marcus Efraimsson
f5586b1270
tsdb: sql data sources should handle time ranges before epoch start correctly 2018-04-12 18:53:12 +02:00
Marcus Efraimsson
6cb891dca8
mysql: use a datetime column with microsecond precision in test 2018-04-10 16:37:39 +02:00
Marcus Efraimsson
af62646624
mysql: fix precision for time columns in time series query mode 2018-04-10 11:01:43 +02:00
Marcus Efraimsson
5c120c2c11
mysql: mysql tests should use a db server with UTC
To get rid of issues involving date/time when testing.
Also, makes it possible to run mysql integration tests for both
grafana config db and tsdb at the same time using
GRAFANA_TEST_DB=mysql go test ./pkg/...
2018-04-10 10:58:45 +02:00
ryan
7d6c8aa612 add mssql and mysql 2018-04-09 13:34:35 +02:00
Marcus Efraimsson
f0f41c2a8e mysql: skip tests by default 2018-03-22 16:44:55 +01:00
Marcus Efraimsson
f5654f88e2 mysql: fix precision for the time column in table/annotation query mode
Use the ConvertSqlTimeColumnToEpochMs function to convert any native
datetime data type or epoch time (millisecond precision).
Refactored mysql implementation to make it more similar to postgres
and mssql implementations.
Added $__timeEpoch macro function with same implementation as $__time.
Added possibility to use a time column named time in addition to
the currectly supported time_sec.
Additional tests and update of existing.
Added test dashboard.
2018-03-22 15:40:46 +01:00
Sven Klemm
4e826ff8f7 remove spaces around arguments of macros 2018-03-02 19:24:15 +01:00
Sven Klemm
0f931f9e5e Merge remote-tracking branch 'upstream/master' into macroengine-interpolate 2018-03-01 18:01:54 +01:00
Daniel Lee
6d13645206 mysql: update to use ColumnTypes interface in new version
The custom code in vendor is not needed anymore and most of the
mapping code can be replaced.
2018-01-18 11:50:16 +01:00
Sven Klemm
cb15d9cbdd add missing value fill code to mysql datasource 2017-12-10 19:50:01 +01:00
Sven Klemm
b86a42fffe pass tsdbQuery to transformToTimeSeries and transformToTable to get
access to selected frontend timerange
2017-12-09 20:35:00 +01:00
Sven Klemm
b6df91d56b pass Query to MacroEngine Interpolate 2017-12-08 23:30:33 +01:00
Daniel Lee
5dd9582520 mysql: add data source support for Azure MySql
Fixes #9649
2017-11-21 13:21:19 +01:00
Sven Klemm
34da0711ab add __timeGroup macro for mysql (#9596)
* add __timeGroup macro for mysql

* put example __timeGroup query in frontend help

* do __timeGroup interval parsing in go similar to mysql

* ignore whitespace around interval
2017-10-27 11:26:25 +02:00
Daniel Lee
d1c9760fa8 Postgres Data Source (#9475)
* add postgresql datasource

* add rest of files for postgres datasource

* fix timeseries query, remove unused code

* consistent naming, refactoring

* s/mysql/postgres/

* s/mysql/postgres/

* couple more tests

* tests for more datatypes

* fix macros for postgres

* add __timeSec macro

* add frontend for postgres datasource

* adjust documentation

* fix formatting

* add proper plugin description

* merge editor changes from mysql

* port changes from mysql datasource

* set proper defaultQuery for postgres

* add time_sec to timeseries query
accept int for value for timeseries query

* revert allowing time_sec and handle int or float values as unix
timestamp for "time" column

* fix tslint error

* handle decimal values in timeseries query

* allow setting sslmode for postgres datasource

* use type switch for handling data types

* fix value for timeseries query

* refactor timeseries queries to make them more flexible

* remove debug statement from inner loop in type conversion

* use plain for loop in getTypedRowData

* fix timeseries queries

* adjust postgres datasource to tsdb refactoring

* adjust postgres datasource to frontend changes

* update lib/pq to latest version

* move type conversion to getTypedRowData

* handle address types cidr, inet and macaddr

* adjust response parser and docs for annotations

* convert unknown types to string

* add documentation for postgres datasource

* add another example query with metric column

* set more helpful default query

* update help text in query editor

* handle NULL in value column of timeseries query

* add __timeGroup macro

* add test for __timeGroup macro

* document __timeGroup and set proper default query for annotations

* fix typos in docs

* add postgres to list of datasources

* add postgres to builtInPlugins

* mysql: refactoring as prep for merging postgres

Refactors out the initialization of the xorm engine and the query logic
for an sql data source.

* mysql: rename refactoring + test update

* postgres:refactor to use SqlEngine(same as mysql)

Refactored to use a common base class with the MySql data source.

Other changes from the original PR:
- Changed time column to be time_sec to allow other time units in the
future and to be the same as MySQL
- Changed integration test to test the main Query method rather than
the private transformToTable method
- Changed the __timeSec macro name to __timeEpoch
- Renamed PostgresExecutor to PostgresQueryEndpoint

Fixes #9209 (the original PR)

* postgres: encrypt password on config page

With some other cosmetic changes to the config page:
- placeholder texts
- reset button for the password after it has been encrypted.
- default value for the sslmode field.

* postgres: change back col name to time from time_sec

* postgres mysql: remove annotation title

Title has been removed from annotations

* postgres: fix images for docs page

* postgres mysql: fix specs
2017-10-10 15:19:14 +02:00
bergquist
ed661767f8 follow go idiom and return error as second param 2017-09-21 18:04:16 +02:00
bergquist
a45e2ec9c3 context is reserved for go's context 2017-09-21 15:23:34 +02:00
bergquist
16b5b9f6be make ds a param for Query 2017-09-21 15:09:14 +02:00
bergquist
7f9f388293 rename executor into tsdbqueryendpoint 2017-09-21 15:02:17 +02:00
bergquist
0229d28d64 remove unused structs 2017-09-21 15:02:17 +02:00
bergquist
55f1b36e31 refactor response flow 2017-09-21 15:02:17 +02:00
mxlxm
c5400ffe76 mysql: use collation instead of charset (#9156)
- use charset would issue additional 'SET NAMES <VALUE>' queries, set
collation would be better. see: https://github.com/go-sql-driver/mysql#charset
2017-09-04 12:32:07 +02:00
Torkel Ödegaard
c92317bafa fix: fixed gofmt formating for #9091 2017-08-28 13:29:24 +02:00
pdoan017
3eea5d235d Added import for container/list 2017-08-24 15:42:54 -04:00
pdoan017
d6085755c4 Added series order according to query results
Raw query results would return as expected. But when appending new series to pointsBySeries map, order of keys is random. So I've added a list to keep track of series order. Should implement FIFO rules.
2017-08-24 15:38:36 -04:00
Daniel Lee
43fa852cc1 mysql: change logging from into to debug for scan 2017-08-09 19:33:09 +02:00
Daniel Lee
563795245a mysqlds: support for more column types
Includes integration test for most column types. Fixes #8918
2017-08-08 15:24:41 +02:00
Daniel Lee
2fb25d8fa4 mysqlds: add tests for new macros 2017-08-08 14:40:27 +02:00
Brad Lhotsky
a8ac215039 Add a few more MySQL macros:
* $__timeFrom() -> Returns the dashboard 'from' suitable for use querying
  against a MySQL TIMESTAMP field.
* $__timeTo() -> Returns the dashboard 'to' suitable for use querying
  against a MySQL TIMESTAMP field.
* $__unixEpochFiler(column) -> If you store timestamps as UNIX epoch's,
  this builds: column > 'from' AND column < 'to'
* $__unixEpochFrom() -> Returns the dashboard 'from' suitable for use querying
  against a MySQL integer field (UNIX epochs)
* $__unixEpochTo() -> Returns the dashboard 'to' suitable for use querying
  against a MySQL integer field (UNIX epochs)
2017-08-08 14:40:27 +02:00
Torkel Ödegaard
60da730c95 mysql: fix for TIME columns, fixes #8534 2017-06-23 12:55:40 -04:00
Daniel Lee
2479e51a6b mysql: Null value should not be considered as previous value
fixes #8655
2017-06-20 02:02:05 +02:00
Denis Doria
41d300f69d Fix timeInterval for mysql datasource (#8651)
* Fix timeInterval for mysql datasource

This changes the > to >= and the < to <=, so the intervals are inclusive.
This should fix the #8635

* Fix validation
2017-06-19 08:58:22 -04:00
Daniel Lee
e8b798914d mysql: adds mapping for int/long 2017-05-16 16:07:42 +02:00
Daniel Lee
a9c535e551 mysql: add some more type mapping
Decimals mapped to floats for now. No mapping for bit or
any of the blob types. Tinyint not mapped to bool.
2017-05-12 11:37:51 +02:00
Torkel Ödegaard
ae5e004b69 mysql: time filter macro updated 2017-04-24 14:55:43 +02:00
Daniel Lee
7784e4e24b mysql: add datetime type to table data 2017-04-23 21:56:58 +02:00
Daniel Lee
413ee33d5d mysql: fix go vet error 2017-04-23 20:07:20 +02:00
Torkel Ödegaard
97e2d75f51 mysql: added support for tables in mysql queries 2017-04-21 15:07:43 +02:00
Torkel Ödegaard
c78c460f79 mysql: worked on mysql data soruce 2017-04-20 17:10:23 +02:00
Torkel Ödegaard
50e70cf3db build: fixed unit test 2017-04-20 13:04:42 +02:00
Torkel Ödegaard
fc878bc8ad build: fixed order 2017-04-20 11:59:11 +02:00
Torkel Ödegaard
11806dfa78 mysql: progress 2017-04-19 17:26:37 +02:00
Torkel Ödegaard
e164eba677 mysql: began work on backend macro engine 2017-04-19 10:10:08 +02:00
Torkel Ödegaard
5af81b974e mysql: graph showing 2017-03-31 13:57:10 +02:00
Torkel Ödegaard
a7babfb7cf tech: updated xorm libs 2017-03-31 12:54:39 +02:00
Torkel Ödegaard
e5a5bef200 mysql: minor update 2017-03-31 12:49:05 +02:00
Torkel Ödegaard
d6d2080f11 mysql: minor progress on response processing 2017-03-31 11:45:25 +02:00
Torkel Ödegaard
080d2a2054 mysql: reading arbitrary SQL data in go is really strange, data is only in strings? 2017-03-30 13:46:46 +02:00
Torkel Ödegaard
00fcaaf171 mysql: commented out old code from old PR 2017-03-29 22:55:37 +02:00
Torkel Ödegaard
8f90c6115d mysql: progress on mysql data source 2017-03-29 22:54:18 +02:00