Commit Graph

62 Commits

Author SHA1 Message Date
Kyle Brandt
82aa000e9d
SSE: (Chore) Update log line with context (trace) (#89172)
Improves log line to help with debugging in Server Side Expressions. In particular, the traceId, datasourceType, and datasourceUid will now be included.
2024-06-13 10:58:39 -04:00
Yuri Tseretyan
9dc4221508
Alerting: Log expression command types during evaluation (#84614) 2024-03-19 10:00:03 -04:00
Ryan McKinley
d82f3be6f7
QueryService: Use types from sdk (#84029) 2024-03-08 18:12:59 +02:00
Ryan McKinley
3036b50df3
Expressions: expose ConvertDataFramesToResults (#83805) 2024-03-04 18:22:56 +02:00
Scott Lepper
70009201d4
Expressions: Sql expressions with Duckdb (#81666)
duckdb temp storage of dataframes using parquet and querying from sql expressions
---------

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
2024-02-27 16:16:00 -05:00
Ryan McKinley
f23f50f58d
Expressions: Add model struct for the query types (not map[string]any) (#82745) 2024-02-17 02:59:11 +02:00
Yuri Tseretyan
bf8be46e6f
SSE: Add utility methods for HysteresisCommand (#79157)
* add GetCommandsFromPipeline
* refactor method GetCommandType to func GetExpressionCommandType
* add function to create fingerprint frames
* add function to determine whether raw query represents a hysteresis command and a function to patch it with loaded metrics
2023-12-11 22:40:31 +02:00
Ryan McKinley
f69fd3726b
FeatureToggles: Add context and and an explicit global check (#78081) 2023-11-14 12:50:27 -08:00
Yuri Tseretyan
810fbc3327
SSE: Support hysteresis threshold expression (#70998)
* extend threshold command with second evaluator called `unloadEvaluator` 
* Introduce a new expression command Hysteresis and update Threshold unmarshaller to create the HysteresisCommand if the second eval
* add feature flag `recoveryThreshold`
* update unmarshal threshold command to not re-marshall because it breaks frame definition by shuffling the schema and data fields
2023-10-10 17:51:50 +03:00
Marcus Efraimsson
e4c1a7a141
Tracing: Standardize on otel tracing (#75528) 2023-10-03 14:54:20 +02:00
Kyle Brandt
35e488b22b
SSE: Localize/Contain Errors within an Expression (#73163)
Changes SSE to not always fail all queries when one fails. Now only the query itself, and nodes that depend on it will error.
---------

Co-authored-by: Gilles De Mey <gilles.de.mey@gmail.com>
2023-09-13 13:58:16 -04:00
Kyle Brandt
5cc737bb24
SSE: Put data source query grouping behind feature flag (#74551)
change was originally merged in commit: 720d716 via PR: https://github.com/grafana/grafana/pull/72935 with no flag
flag is: sseGroupByDatasource
2023-09-07 23:02:07 +03:00
Ryan McKinley
025b2f3011
Chore: use any rather than interface{} (#74066) 2023-08-30 18:46:47 +03:00
Kyle Brandt
720d716e45
SSE: Group data source node execution by data source (#72935)
Execute all queries to the same datasource in a single request.

Uses the query index and the graph node ID index, and then a stable dependency graph sort based on node input index number in attempt to keep the original query order intact.
2023-08-18 07:49:59 -04:00
Kyle Brandt
ceb694f165
SSE: (Instrumentation/Chore) Add datasource_type label to grafana_sse (#72370)
SSE: (Instrumentation/Chore) Add datasource_type label to the grafana_sse_ds_queries_total metric
2023-07-26 10:51:02 -04:00
Kyle Brandt
1df4d332c9
SSE: Use errutil to show better error messages in prod (#71658)
- include public message
- propagate data source query errors so they are shown as well to which fixes #70026
2023-07-21 06:38:29 -04:00
Yuri Tseretyan
cbbbe2e6f6
SSE: DSNode to update result with names to make each value identifiable by labels (only Graphite and TestData) (#71246)
* introduce a function checkIfSeriesNeedToBeFixed to scan all value fields in the response and provide a function that updates Series so they can be uniquely identifiable. Only Graphite and TestData are checked.

* update `convertDataFramesToResults` to run this function and provide it to WideToMany
* update WideToMany to run the fix function if it is not nil
2023-07-20 14:44:12 -04:00
Yuri Tseretyan
541bfe636d
SSE: Support for ML query node (#69963)
* introduce a new node-type ML and implement a command outlier that uses ML plugin as a source of data.
* add feature flag mlExpressions that guards the feature
2023-07-13 20:37:50 +03:00
Yuri Tseretyan
9e76d6372c
SSE: improve error handling in DSNode (#71468)
* add data source uid to QueryError

* add error and datasource uid to tracing

* split queryDataResponseToResults to two functions one to extract frame from the result, and another to convert it

* propagate logger with context to convertDataFramesToResults
2023-07-12 14:59:02 -04:00
Yuri Tseretyan
cc72fe17d5
SSE: Refactor DSNode and extract function to convert backend response to mathexp.Results (#70098) 2023-06-15 09:20:08 -04:00
Will Browne
624777258b
Plugins: Refactor creation of plugin context to dedicated service (#66451)
* first pass

* fix tests

* return errs

* change signature

* tidy

* delete unnecessary fields from test

* tidy

* fix tests

* simplify

* separate error check in API

* apply nits
2023-06-08 13:59:51 +02:00
Kyle Brandt
840fb32ad8
SSE: (Instrumentation) Add Tracing (#66700)
spans are prefixed `SSE.`
2023-04-18 08:04:51 -04:00
Kyle Brandt
2f13c851e4
SSE: (Chore/Instrumentation) Add ds_queries_total metric and move met… (#66695)
* SSE: (Chore/Instrumentation) Add ds_queries_total metric and move metrics to service
2023-04-17 16:12:44 -07:00
Kyle Brandt
e78be44e1a
SSE: Dataplane Compliance (#65927)
Takes a specific code path for data that identifies itself as dataplane instead of "guessing" what the data is.

The data must identify itself by being in the dataplane by having both the following frame metadata properties:

- TypeVersion property that is greater than 0.0
- 'Type' property

The flag is disableSSEDataplane and disables this functionality and uses the old code for all queries regardless.

See https://github.com/grafana/grafana-plugin-sdk-go/blob/main/data/contract_docs/contract.md for dataplane details.
2023-04-12 12:24:34 -04:00
Shirley
972e611f76
CloudWatch Logs: Revert "Queries in an expression should run synchronously (#64443)" (#65036)
Revert "CloudWatch Logs: Queries in an expression should run synchronously (#64443)"

This reverts commit 74436d31de.
2023-03-20 15:54:30 +01:00
Isabella Siu
74436d31de
CloudWatch Logs: Queries in an expression should run synchronously (#64443) 2023-03-13 13:31:46 -04:00
Will Browne
94f39e69a3
Plugins: Migrate plugincontext, adapters and pluginsettings to pkg/services/pluginsintegration package (#64154)
* migrate plugincontext, adapter and pluginsettings

* add to CODEOWNERS

* fix imports

* fix CODEOWNERS

* take pluginsettings

* migrate wire stuff
2023-03-07 11:22:30 -05:00
idafurjes
23c27cffb3
Chore: Rename Id to ID in alerting models (#62777)
* Chore: Rename Id to ID in alerting models

* Add xorm tags for datasource

* Add xorm tag for uid
2023-02-02 17:22:43 +01:00
Serge Zaitsev
f8ec35e643
Chore: Fix goimports grouping (#62427)
* fix goimports order

* fix goimports order
2023-01-30 09:38:51 +01:00
Yuri Tseretyan
5e8866ed5a
SSE: DSNode to parse only one response (#61644)
* do not iterate through responses
* log type of the response in all exit cases
2023-01-18 13:06:10 -05:00
Marcus Efraimsson
6dbe3b555f
Plugins: Refactor forward of cookies, OAuth token and header modifications by introducing client middlewares (#58132)
Adding support for backend plugin client middlewares. This allows headers in outgoing 
backend plugin and HTTP requests to be modified using client middlewares.

The following client middlewares added:
Forward cookies: Will forward incoming HTTP request Cookies to outgoing plugins.Client 
and HTTP requests if the datasource has enabled forwarding of cookies (keepCookies).
Forward OAuth token: Will set OAuth token headers on outgoing plugins.Client and HTTP 
requests if the datasource has enabled Forward OAuth Identity (oauthPassThru).
Clear auth headers: Will clear any outgoing HTTP headers that was part of the incoming 
HTTP request and used when authenticating to Grafana.
The current suggested way to register client middlewares is to have a separate package, 
pluginsintegration, responsible for bootstrap/instantiate the backend plugin client with 
middlewares and/or longer term bootstrap/instantiate plugin management. 

Fixes #54135
Related to #47734
Related to #57870
Related to #41623
Related to #57065
2022-12-01 19:08:36 +01:00
Kyle Brandt
5623b5afaf
SSE: Make sure to forward headers, user and cookies/OAuth token (#58897)
Fixes #58793 and Fixes https://github.com/grafana/azure-data-explorer-datasource/issues/513

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2022-11-28 07:40:06 -05:00
Kyle Brandt
1953d473c0
SSE: Keep value name from numeric table (#58831)
fixes #48868
2022-11-16 10:39:28 -05:00
Yuriy Tseretyan
0a4121cef8
Alerting: Contextual log provider for rule key (#57476)
* create contextual log context provider
* use contextual provider in scheduler
* init logger in the package
* use context for log context
* use context in state manager
2022-10-26 19:16:02 -04:00
Yuriy Tseretyan
2d20c8db7b
Chore: Expression engine to support relative time range (#57474)
* make TimeRange interface and add relative range
* make Execute methods support the current time
* update resample to support relative time range
* update DSNode to support relative time range
* update query service to create queries with absolute time
* make alerting evaluator create relative time ranges
2022-10-26 16:13:58 -04:00
Gilles De Mey
9aa61ddd0e
Alerting: Add threshold expression (#55102) 2022-09-26 16:05:44 +02:00
Yuriy Tseretyan
2d38664fe6
Alerting: Improve validation of query and expressions on rule submit (#53258)
* Improve error messages of server-side expression 
* move validation of alert queries and a condition to eval package
2022-09-21 15:14:11 -04:00
Kyle Brandt
05fd7eb047
SSE: Add noData type (#51973)
When there is a single frame with no fields (e.g. splunk datasource) SSE errors when trying to figure out the data type. This frame needs to exist since this is where the executedQueryString metadata exists.

This adds a new return type to SSE to represent no data, so the original frame with its metadata can still be maintained.
2022-07-14 09:18:12 -04:00
Kristin Laemmert
945f015770
backend/datasources: move datasources models into the datasources service package (#51267)
* backend/datasources: move datasources models into the datasources service pkg
2022-06-27 12:23:15 -04:00
Kat Yang
3c3039f5b3
Chore: Remove Wrap (#50048)
* Chore: Remove Wrap and Wrapf

* Fix: Add error check
2022-06-03 09:24:24 +02:00
Kyle Brandt
01ef899753
SSE/Alerting: Support prom instant vector responses (#44865)
* SSE/Alerting: (Draft) Support prom instant vector responses
fixes #35663
* reduce\classic expressions to handle mathexp.Number
* use Notice for warning

Co-authored-by: Yuriy Tseretyan <yuriy.tseretyan@grafana.com>
2022-05-23 10:08:14 -04:00
Kyle Brandt
cab509fbe8
SSE: Keep FieldConfig for data source queries (#46328)
for #46325
2022-03-10 10:03:26 -05:00
Santiago
1bed84ea56
Fix InfluxDB unhealthy alerts and false "no data"s (#43441)
* check for TimeSeriesTypeNot to fix InfluxDB alerts

* log a warning when a data frame is ignored
2022-01-27 10:53:27 -03:00
Ryan McKinley
2754e4fdf0
Expressions: use datasource model from the query (#41376)
* refactor datasource loading

* refactor datasource loading

* pass uid

* use dscache in alerting to get DS

* remove expr/translate pacakge

* remove dup injection entry

* fix DS type on metrics endpoint, remove SQL DS lookup inside SSE

* update test and adapter

* comment fix

* Make eval run as admin when getting datasource info

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>

* fmt and comment

* remove unncessary/redundant code

Co-authored-by: Kyle Brandt <kyle@grafana.com>
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
Co-authored-by: Santiago <santiagohernandez.1997@gmail.com>
2021-12-16 13:51:46 -03:00
George Robinson
708bdc80cb
Alerting: Support Unwrap for QueryError in expr package (#41743) 2021-11-17 10:07:24 +00:00
George Robinson
543b1a7e39
Alerting: Add QueryError to expr package (#41737) 2021-11-16 13:42:22 +00:00
Ryan McKinley
3489721ed6
api/ds/query: simplify data sources lookup for queries and expressions (#41172) 2021-11-05 08:12:55 -07:00
Ryan McKinley
7319efe077
Dashboard: replace datasource name with a reference object (#33817)
Co-authored-by: Torkel Ödegaard <torkel@grafana.com>
Co-authored-by: Elfo404 <me@giordanoricci.com>
2021-10-29 10:57:24 -07:00
Ikko Ashimine
202f295e51
Chore: Fix typo in nodes.go (#38636)
commmon -> common
2021-08-27 14:57:41 +02:00
Domas
3e124c854e
Alerting: interpolate variables when creating alert rule from dashboard panel (#37201) 2021-08-10 10:59:48 +03:00