Commit Graph

40 Commits

Author SHA1 Message Date
matt abrams
76da017d00
MySQL: Prevent pre-epoch timestamps from breaking __timeFilter (#45345)
* add logic to change timeFilter query when FROM time is pre-EPOCH

* fix timeFilter tests where FROM time is pre-EPOCH
2022-02-22 20:06:04 +01:00
Andres Martinez Gotor
64c8d32fe7
Use sdk pkg for gtime (#39354) 2021-09-21 13:08:52 +02:00
ying-jeanne
c989f0fbbe
SQL: Migrate to use SDK contracts (#36635)
* convert SQLs to use sdk contracts

* make draft

* postgres

* intermedia

* get datasourceinfo filled at the beginning of the service

* move the interval into package because of cyclict  import and fix all postgres tests

* fix mysql test

* fix mssql

* fix the test for pr https://github.com/grafana/grafana/issues/35839

* fix some issue about intervalv2 package

* update sql test

* wire migration for SQLs

* add sqls to the background process

* make it register instead of register and start

* revert formatting

* fix tests

* fix linter

* remove integration test

* Postgres test fix

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2021-09-07 09:35:37 +02:00
Marcus Efraimsson
0611207f3b
Postgres/MySQL/MSSQL: Fix concurrent map writes panic (#35510)
Fixes panic/fatal error concurrent map writes in SQL data sources when multiple 
queries are executed concurrently and you interpolate SQL query before executing it.

Fixes #35469
2021-06-11 14:56:29 +02:00
Arve Knudsen
b79e61656a
Introduce TSDB service (#31520)
* Introduce TSDB service

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

Co-authored-by: Erik Sundell <erik.sundell87@gmail.com>
Co-authored-by: Will Browne <will.browne@grafana.com>
Co-authored-by: Torkel Ödegaard <torkel@grafana.org>
Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
Co-authored-by: Zoltán Bedi <zoltan.bedi@gmail.com>
2021-03-08 07:02:49 +01:00
Arve Knudsen
2bf964c61e
Chore: Fix linting issues caught by ruleguard (#28799)
* Chore: Fix linting issues caught by ruleguard

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Improve error check

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2020-11-17 11:27:45 +01:00
Arve Knudsen
574553ec7b
Chore: Fix issues found by staticcheck (#28802)
* Fix linting issues

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2020-11-05 11:29:39 +01:00
Marcus Efraimsson
3de693af49
MySQL: Limit datasource error details returned from the backend (#19373)
Only return certain mysql errors from backend.
The following errors is returned as is from backend:
error code 1064 (parse error)
error code 1054 (bad column/field selected)
error code 1146 (table not exists)
Any other errors is logged and returned as a generic
error.
Restrict use of certain functions:
Do not allow usage of the following in query:
system_user()
session_user()
current_user() or current_user
user()
show grants

Fixes #19360
2019-09-24 20:50:49 +02:00
Kyle Brandt
e5bf3027fc
Refactor: Move sql_engine to sub package of tsdb (#18991)
this way importing the tsdb package does not come with xorm dependencies
2019-09-10 15:50:04 -04:00
Bernard Duggan
a0bb01103e MySQL/Postgres/MSSQL: Add parsing for day, weeks and year intervals in macros (#13086)
closes #11431
2019-05-30 09:58:29 +02:00
Ander Arguiñano
089d85af63 Added macros to mysql 2019-01-03 22:29:57 +01:00
Marcus Efraimsson
624e5e5b3d
let each sql datasource handle timeFrom and timeTo macros 2018-11-30 14:30:02 +01:00
Marcus Efraimsson
97b22aa5a9
mysql: fix timeFilter macro should respect local time zone 2018-10-25 10:29:40 +02:00
Sven Klemm
94d6d51726 move timeFrom, timeTo, unixEpochFrom and unixEpochTo macros to sql_engine 2018-10-02 20:19:34 +02:00
Sven Klemm
0254a29e35 Interpolate $__interval in backend for alerting with sql datasources (#13156)
add support for interpolate $__interval and  $__interval_ms in sql datasources
2018-09-13 16:51:00 +02:00
Sven Klemm
fbc67a1c64 add $__unixEpochGroup to mysql datasource 2018-08-13 12:17:05 +02:00
Sven Klemm
d81a23becf Refactor setting fillmode
This adds SetupFillmode to the tsdb package to be used by the sql
datasources.
2018-08-12 10:51:58 +02:00
Sven Klemm
ee7602ec1f change fillmode from last to previous 2018-08-07 21:01:41 +02:00
Sven Klemm
bfc66a7ed0 add fillmode "last" to sql datasource
This adds a new fill mode last (last observation carried forward) for grafana
to the sql datasources. This fill mode will fill in the last seen value in a
series when a timepoint is missing or NULL if no value for that series has
been seen yet.
2018-08-02 20:35:09 +02:00
Sven Klemm
b71d10a7a4 add $__timeGroupAlias to mysql and mssql 2018-08-01 20:58:51 +02:00
Marcus Efraimsson
27db454012
mysql: use new sql engine 2018-07-26 18:10:45 +02:00
Marcus Efraimsson
d90d4ce55d
Merge pull request #12402 from AustinWinstanley/macros/sql
mysql,mssql: improve $__timeFilter, $__timeFrom, $__timeTo macros

The $__timeFilter macro now uses a BETWEEN with timestamps instead of 
comparing epochs
The $__timeFrom and $__timeTo macros now return strings of format 
YYYY-MM-DDThh:mm:ssZ instead of epoch
2018-07-04 20:01:43 +02:00
Sven Klemm
1601f6d17c [mysql] fix $__timeGroup rounding (#12469)
* fix $__timeGroup rounding for mysql

* revert accidentally commented out line
2018-07-01 06:59:05 -07:00
Austin Winstanley
a0e8437f8f Used PostgreSQL TSDB as a model the set up the __timeFilter, __timeFrom, and __timeTo macros for Microsoft SQL and MySQL 2018-06-29 03:48:14 +00:00
Austin Winstanley
105b3d6804 Switched MySQL and MSSQL macros for timeFilter and related to use BETWEEN and calculate UNIX time server side instead of database side. Fixes #11618 #11619 2018-06-25 20:11:58 +00:00
Marcus Efraimsson
a86ee304ff
tsdb: remove unnecessary type casts in sql data sources macro engines 2018-04-12 19:08:35 +02:00
Marcus Efraimsson
f5586b1270
tsdb: sql data sources should handle time ranges before epoch start correctly 2018-04-12 18:53:12 +02:00
ryan
7d6c8aa612 add mssql and mysql 2018-04-09 13:34:35 +02:00
Marcus Efraimsson
f5654f88e2 mysql: fix precision for the time column in table/annotation query mode
Use the ConvertSqlTimeColumnToEpochMs function to convert any native
datetime data type or epoch time (millisecond precision).
Refactored mysql implementation to make it more similar to postgres
and mssql implementations.
Added $__timeEpoch macro function with same implementation as $__time.
Added possibility to use a time column named time in addition to
the currectly supported time_sec.
Additional tests and update of existing.
Added test dashboard.
2018-03-22 15:40:46 +01:00
Sven Klemm
4e826ff8f7 remove spaces around arguments of macros 2018-03-02 19:24:15 +01:00
Sven Klemm
cb15d9cbdd add missing value fill code to mysql datasource 2017-12-10 19:50:01 +01:00
Sven Klemm
b6df91d56b pass Query to MacroEngine Interpolate 2017-12-08 23:30:33 +01:00
Sven Klemm
34da0711ab add __timeGroup macro for mysql (#9596)
* add __timeGroup macro for mysql

* put example __timeGroup query in frontend help

* do __timeGroup interval parsing in go similar to mysql

* ignore whitespace around interval
2017-10-27 11:26:25 +02:00
Daniel Lee
d1c9760fa8 Postgres Data Source (#9475)
* add postgresql datasource

* add rest of files for postgres datasource

* fix timeseries query, remove unused code

* consistent naming, refactoring

* s/mysql/postgres/

* s/mysql/postgres/

* couple more tests

* tests for more datatypes

* fix macros for postgres

* add __timeSec macro

* add frontend for postgres datasource

* adjust documentation

* fix formatting

* add proper plugin description

* merge editor changes from mysql

* port changes from mysql datasource

* set proper defaultQuery for postgres

* add time_sec to timeseries query
accept int for value for timeseries query

* revert allowing time_sec and handle int or float values as unix
timestamp for "time" column

* fix tslint error

* handle decimal values in timeseries query

* allow setting sslmode for postgres datasource

* use type switch for handling data types

* fix value for timeseries query

* refactor timeseries queries to make them more flexible

* remove debug statement from inner loop in type conversion

* use plain for loop in getTypedRowData

* fix timeseries queries

* adjust postgres datasource to tsdb refactoring

* adjust postgres datasource to frontend changes

* update lib/pq to latest version

* move type conversion to getTypedRowData

* handle address types cidr, inet and macaddr

* adjust response parser and docs for annotations

* convert unknown types to string

* add documentation for postgres datasource

* add another example query with metric column

* set more helpful default query

* update help text in query editor

* handle NULL in value column of timeseries query

* add __timeGroup macro

* add test for __timeGroup macro

* document __timeGroup and set proper default query for annotations

* fix typos in docs

* add postgres to list of datasources

* add postgres to builtInPlugins

* mysql: refactoring as prep for merging postgres

Refactors out the initialization of the xorm engine and the query logic
for an sql data source.

* mysql: rename refactoring + test update

* postgres:refactor to use SqlEngine(same as mysql)

Refactored to use a common base class with the MySql data source.

Other changes from the original PR:
- Changed time column to be time_sec to allow other time units in the
future and to be the same as MySQL
- Changed integration test to test the main Query method rather than
the private transformToTable method
- Changed the __timeSec macro name to __timeEpoch
- Renamed PostgresExecutor to PostgresQueryEndpoint

Fixes #9209 (the original PR)

* postgres: encrypt password on config page

With some other cosmetic changes to the config page:
- placeholder texts
- reset button for the password after it has been encrypted.
- default value for the sslmode field.

* postgres: change back col name to time from time_sec

* postgres mysql: remove annotation title

Title has been removed from annotations

* postgres: fix images for docs page

* postgres mysql: fix specs
2017-10-10 15:19:14 +02:00
Brad Lhotsky
a8ac215039 Add a few more MySQL macros:
* $__timeFrom() -> Returns the dashboard 'from' suitable for use querying
  against a MySQL TIMESTAMP field.
* $__timeTo() -> Returns the dashboard 'to' suitable for use querying
  against a MySQL TIMESTAMP field.
* $__unixEpochFiler(column) -> If you store timestamps as UNIX epoch's,
  this builds: column > 'from' AND column < 'to'
* $__unixEpochFrom() -> Returns the dashboard 'from' suitable for use querying
  against a MySQL integer field (UNIX epochs)
* $__unixEpochTo() -> Returns the dashboard 'to' suitable for use querying
  against a MySQL integer field (UNIX epochs)
2017-08-08 14:40:27 +02:00
Denis Doria
41d300f69d Fix timeInterval for mysql datasource (#8651)
* Fix timeInterval for mysql datasource

This changes the > to >= and the < to <=, so the intervals are inclusive.
This should fix the #8635

* Fix validation
2017-06-19 08:58:22 -04:00
Torkel Ödegaard
ae5e004b69 mysql: time filter macro updated 2017-04-24 14:55:43 +02:00
Torkel Ödegaard
fc878bc8ad build: fixed order 2017-04-20 11:59:11 +02:00
Torkel Ödegaard
11806dfa78 mysql: progress 2017-04-19 17:26:37 +02:00
Torkel Ödegaard
e164eba677 mysql: began work on backend macro engine 2017-04-19 10:10:08 +02:00