Commit Graph

37 Commits

Author SHA1 Message Date
Marcus Efraimsson
8927a3ca20
Chore: Query endpoint refactor (#41637)
Get rid of using legacydata.RequestHandler in HTTPServer, /api/tsdb/query and pkg/expr 
with the goal of deprecating /api/tsdb/query and remove it completely eventually. This is 
the first step of cleaning up the HTTP API query endpoint.

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
Co-authored-by: Alexander Emelin <frvzmb@gmail.com>
2021-11-29 14:21:54 +01:00
Marcus Efraimsson
baab021fec
Chore: Refactor usage of legacy data contracts (#41218)
Refactor usage of legacy data contracts. Moves legacy data contracts 
to pkg/tsdb/legacydata package.
Refactor pkg/expr to be a proper service/dependency that can be provided 
to wire to remove some unneeded dependencies to SSE in ngalert and other places.
Refactor pkg/expr to not use the legacydata,RequestHandler and use 
backend.QueryDataHandler instead.
2021-11-10 11:52:16 +01:00
Marcus Efraimsson
e31b8f4d33
Chore: Remove SQL engine cache (#39254)
Fix bug with engine cache not updating after data source updated, introduced in #36635.
Removes the SQL engine cache because since #36635 we already have instance caching in place using the SDK instance manager.
This also adds support for disposing cached SQL/xorm engines before a query happens to a data source that has been updated. Done by letting the SQL engine implementing the InstanceDisposer interface.

Fixes #37299
2021-09-15 23:03:25 +02:00
Marcus Efraimsson
1c892a2fc4
Postgres/MySQL/MSSQL: Add setting to limit maximum amount of rows processed (#38986)
Adds a new setting dataproxy.row_limit that allows an operator to limit the 
amount of rows being processed/accepted in response to database queries 
originating from SQL data sources.

Closes #38975
Ref #39095

Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>
2021-09-13 15:27:51 +02:00
Ivana Huckova
f172701043
IntervalV2: Use maxDataPoints to calculate correct interval (#39036)
* Use max data points to calculate interval

* Fix mockCalculator to mach Calculator

* Fix incorrect merge, replace queryModel with query
2021-09-13 14:58:32 +02:00
Giordano Ricci
dc36f15fbb
Prometheus: Revert stepMode functionality (#38982)
* Revert "Prometheus: add functionality to specify desired step interval in dashboards panels (#36422)"

This reverts commit ddf5b65c51.
Co-authored-by: Ivana Huckova <ivana.huckova@gmail.com>

* Revert "Explore: add functionality for supporting different step modes in prometheus (#37829)"

This reverts commit f433cfd8d9.
Co-authored-by: Ivana Huckova <ivana.huckova@gmail.com>

* Revert stepMode BE implementation from #36796
Co-authored-by: "Ivana Huckova" <ivana.huckova@gmail.com>
2021-09-09 14:05:08 +02:00
Marcus Efraimsson
fbdaf56a84
Postgres/MySQL/MSSQL: Fix region annotations not displayed correctly (#38936)
Fix region annotations not displayed correctly when returning timeend column 
as epoch timestamp and by that making sure that the returned data frame field 
named timeend is treated as time type.

Fixes #38533
2021-09-08 01:54:48 +02:00
ying-jeanne
c989f0fbbe
SQL: Migrate to use SDK contracts (#36635)
* convert SQLs to use sdk contracts

* make draft

* postgres

* intermedia

* get datasourceinfo filled at the beginning of the service

* move the interval into package because of cyclict  import and fix all postgres tests

* fix mysql test

* fix mssql

* fix the test for pr https://github.com/grafana/grafana/issues/35839

* fix some issue about intervalv2 package

* update sql test

* wire migration for SQLs

* add sqls to the background process

* make it register instead of register and start

* revert formatting

* fix tests

* fix linter

* remove integration test

* Postgres test fix

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2021-09-07 09:35:37 +02:00
Ivana Huckova
1083bef030
Prometheus: Implement stepMode for alerting queries (#36796)
* Add select component for choosing step option

* Add onStepChange

* Add functionality for max step

* Rename minInterval to stepInterval to describe min, max and exact step interval

* Change select option from standard to exact

* Add new type StepType for better type safety

* Add tests for adjustInterval

* Add functionality and tests for exact step option

* Prometheus: Spell out min and max in select component

* Prometheus: Change width of step select component and add placeholder

* Prometheus: Adjust for the factor in exact step

* Prometheus: Update tooltip of step lable to include max and exact options and add padding to select component to give it some breathing room from other components

* Update snapshot for step tooltip

* Prometheus: make tooltip more informative

* Prometheus: add tooltip to interval input element

* Prometheus: extract default step option

* Prometheus: update snapshot for PromQueryEditor

* Prometheus: change step labels to uppercase

* Prometheus: define a default step option

* Prometheus: use default step option in both ui component and logic

* Prometheus: update snapshot for PromQueryEditor

* Prometheus: refactor datasource.ts for better readability

* Prometheus: change tool tip for step

* Prometheus: update snapshots

* Prometheus: add correct styling

* Prometheus: update snapshots

* Prometheus change variable name to something less superfluous

* Prometheus: refactor

* Prometheus: add new test for adjustInterval

* Docs: Update docummentation on the step parameter for prometheus

* Prometheus: make step input field smaller and change placeholder text to 15s

* Prometheus: update snapshots

* Prometheus: Make stepMode uniform in all places in the code

* Adjust step based on stepMode

* Adjust comment

* Check if we have queryInterval

* Refactor, add safe interval

* Fix merge resolutions

* Fix tests and add tests

* Update snapshot

* Update docs/sources/datasources/prometheus.md

Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>

* Update docs/sources/datasources/prometheus.md

Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>

* Update docs/sources/datasources/prometheus.md

Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>

* Update docs/sources/datasources/prometheus.md

Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>

* Update docs/sources/datasources/prometheus.md

Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>

* Update docs/sources/datasources/prometheus.md

Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>

* Update docs/sources/datasources/prometheus.md

Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>

* Implement calculation with intervalMode in calculator.go

* Update tests, add calculate safe interval method

* Replace panic with error

* Update pkg/tsdb/interval/interval_test.go

Co-authored-by: idafurjes <36131195+idafurjes@users.noreply.github.com>

* Update pkg/tsdb/calculator_test.go

Co-authored-by: idafurjes <36131195+idafurjes@users.noreply.github.com>

* Impotrt require

* Remove lint errors

Co-authored-by: Olof Bourghardt <ob655088@gmail.com>
Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>
Co-authored-by: idafurjes <36131195+idafurjes@users.noreply.github.com>
2021-08-05 09:09:49 +02:00
idafurjes
180b1973e0
SQL: Fix SQL dataframe resampling (fill mode + time intervals) (#36937)
* Refactor resample logic

* Adjust test to have one more timestamp out of range

* adjust test + ensure filling

* revert flag flip

* Undo logic - should be timeseries only

* change data calculation based on previous interval

* fix the logics

* fix typo

* fix resample start time, to reuse what sql api returned

* calculate the start point with from truncate by interval

Co-authored-by: Will Browne <will.browne@grafana.com>
Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
Co-authored-by: Ying WANG <ying.wang@grafana.com>
2021-07-29 08:38:41 +02:00
Kyle Brandt
3ab9249f16
SQL Datasources: Allow multiple string/labels columns with time series (#36485)
for #36370
2021-07-20 12:42:54 -04:00
Marcus Efraimsson
10c892fa5b
Postgres/MySQL/MSSQL: Fix name of time field should be named Time for time series queries (#36720)
Name of time field changed in v8 for time series queries from Time to the name of the selected 
time column, i.e. time or time_sec. These changes should make sure that the name of time field 
is always returned as Time for time series queries.

Fixes #36059

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
2021-07-14 11:29:51 +02:00
Ryan McKinley
1d6e99bc95
SQL: Fixes issues with showing value column name prefix in legends (#35839)
* rename strategy

* Update pkg/tsdb/sqleng/sql_engine.go

Co-authored-by: Torkel Ödegaard <torkel@grafana.org>

* more strict constraints

* Fixed so that it works on multi series results

* only apply the logic when original query returns 3 fields

* removed part of comment

* Update mysql test

Co-authored-by: Torkel Ödegaard <torkel@grafana.org>
Co-authored-by: Torkel Ödegaard <torkel@grafana.com>
2021-06-18 08:05:23 +02:00
Marc Bachmann
9a5e10e6fb
Annotations: Fix sql annotation parsing for empty responses (#35367)
* fix sql annotation parsing for empty responses

* fix backend when no data returned

* add back frontend changes

Co-authored-by: Ying WANG <ying.wang@grafana.com>
2021-06-09 21:14:28 +08:00
Marcus Efraimsson
6c728d6a78
SQL: Use SDK sqlutil package refactor (#33758)
Upgrades grafana-plugin-sdk-go dependency to v0.94.0 to get 
the refactored sqlutil package. 

Co-authored-by: Kevin Minehart <kmineh0151@gmail.com>
2021-05-11 14:59:33 +02:00
ying-jeanne
bd66c8dde3
SQL data sources: Convert to return data frames (#32257)
Convert SQL data sources to return data frames.

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>
Co-authored-by: Will Browne <will.browne@grafana.com>
Co-authored-by: Hugo Häggmark <hugo.haggmark@gmail.com>
2021-05-05 16:46:07 +02:00
Ryan McKinley
7627b55ef4
TSDB: add deprecation comments to many tsdb structs (#33281) 2021-04-23 03:03:11 +02:00
Marcus Efraimsson
b82c510581
Postgres/MySQL/MSSQL: Don't return connection failure details to the client (#32408)
For security reasons, log any SQL connection error 
details rather than returning it to the client.

Fixes #26623
Fixes #22000

Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>
2021-03-29 13:03:01 +02:00
Marcus Efraimsson
5985c199ae
SQL: Fix so that all captured errors are returned from sql engine (#32353) 2021-03-26 13:17:18 +01:00
Arve Knudsen
b79e61656a
Introduce TSDB service (#31520)
* Introduce TSDB service

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

Co-authored-by: Erik Sundell <erik.sundell87@gmail.com>
Co-authored-by: Will Browne <will.browne@grafana.com>
Co-authored-by: Torkel Ödegaard <torkel@grafana.org>
Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
Co-authored-by: Zoltán Bedi <zoltan.bedi@gmail.com>
2021-03-08 07:02:49 +01:00
Arve Knudsen
c2cad26ca9
Chore: Disable default golangci-lint filter (#29751)
* Disable default golangci-lint filter

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Chore: Fix linter warnings

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2020-12-15 09:32:06 +01:00
Emil Hessman
a5e43535b3
Chore: Rewrite tsdb sql engine test to standard library (#29590)
* Chore: Rewrite tsdb sql engine test to standard library

Signed-off-by: Emil Hessman <emil@hessman.se>
2020-12-07 09:10:33 +01:00
Kyle Brandt
0cb29d337a
Expressions: Move GEL into core as expressions (#29072)
* comes from grafana/gel-app
* remove transform plugin code
* move __expr__ and -100 constants to expr pkg
* set OrgID on request plugin context
* use gtime for resample duration
* in resample, rename "rule" to "window", use gtime for duration, parse duration before exec
* remove gel entry from plugins-bundled/external.json
which creates an empty array for plugins
2020-11-19 07:17:00 -05:00
Arve Knudsen
676d393ec9
Chore: Fix issues reported by staticcheck; enable stylecheck linter (#28866)
* Chore: Fix issues reported by staticcheck

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Apply suggestions from code review

Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>
2020-11-05 15:37:11 +01:00
Arve Knudsen
574553ec7b
Chore: Fix issues found by staticcheck (#28802)
* Fix linting issues

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2020-11-05 11:29:39 +01:00
Arve Knudsen
ec40e49dcb
Chore: Rewrite some tests to use testify (#28420)
* Chore: Improve tests to use testify

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2020-10-21 10:09:42 +02:00
Arve Knudsen
a078e40238
Settings: Rename constants/variables to follow Go naming standards (#28002)
* settings: Rename constants/variables to follow Go naming standards
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2020-10-02 15:45:45 +02:00
Arve Knudsen
a5d9196a53
Chore/fix lint issues (#27704)
* Chore: Fix linting issues

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2020-09-22 16:22:19 +02:00
Arve Knudsen
7896836deb
Chore: Reduce TSDB Go code complexity (#26401)
* tsdb: Make code less complex
2020-08-18 14:43:18 +02:00
Arve Knudsen
d4e4cb4c71
Chore: Enable Go linter gocritic (#26224)
* Chore: Enable gocritic linter

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2020-07-16 14:39:01 +02:00
Arve Knudsen
41d432b5ae
Chore: Enable whitespace linter (#25903)
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2020-07-06 20:17:28 +02:00
Ryan McKinley
2a6ac88a73
QueryInspector: add common way to show the raw query (#25204) 2020-05-29 11:06:33 -07:00
Noval Agung Prayogo
ea35ae4d1f
Chore: Update the xorm dependency (#22376)
* Upgrade xorm

Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>
2020-04-01 15:57:21 +02:00
Jerry Ylilammi
e27ab89aed Postgres/MySQL/MSSQL: Adds support for region annotations (#20752)
Adds support for region annotations in Postgres, MySQL and 
MSSQL data sources by adding a column named timeend to 
annotation query.

Co-Authored-By: Diana Payton <52059945+oddlittlebird@users.noreply.github.com>

Closes #20918
Ref #10589
2019-12-13 17:25:36 +01:00
Marcus Efraimsson
3de693af49
MySQL: Limit datasource error details returned from the backend (#19373)
Only return certain mysql errors from backend.
The following errors is returned as is from backend:
error code 1064 (parse error)
error code 1054 (bad column/field selected)
error code 1146 (table not exists)
Any other errors is logged and returned as a generic
error.
Restrict use of certain functions:
Do not allow usage of the following in query:
system_user()
session_user()
current_user() or current_user
user()
show grants

Fixes #19360
2019-09-24 20:50:49 +02:00
Marcus Efraimsson
19f3ec4891
MySQL, Postgres, MSSQL: Only debug log when in development (#19239)
Found some additional debug statements in relation to #19049 that 
can cause memory issues.

Ref #19049
2019-09-20 15:00:28 +02:00
Kyle Brandt
e5bf3027fc
Refactor: Move sql_engine to sub package of tsdb (#18991)
this way importing the tsdb package does not come with xorm dependencies
2019-09-10 15:50:04 -04:00