Commit Graph

74 Commits

Author SHA1 Message Date
Yuriy Tseretyan
2d38664fe6
Alerting: Improve validation of query and expressions on rule submit (#53258)
* Improve error messages of server-side expression 
* move validation of alert queries and a condition to eval package
2022-09-21 15:14:11 -04:00
George Robinson
7d20766ae9
Alerting: Fix mathexp.NoData cannot be reduced (#55347)
This commit fixes a bug where queries from datasources such as InfluxDB that returned no data would not create a DatasourceNoData alert, but instead an error "can only reduce type series, got type noData".
2022-09-21 10:01:51 +01:00
Emil Tullstedt
b287047052
Chore: Upgrade Go to 1.19.1 (#54902)
* WIP

* Set public_suffix to a pre Ruby 2.6 version

* we don't need to install python

* Stretch->Buster

* Bump versions in lib.star

* Manually update linter

Sort of messy, but the .mod-file need to contain all dependencies that
use 1.16+ features, otherwise they're assumed to be compiled with
-lang=go1.16 and cannot access generics et al.

Bingo doesn't seem to understand that, but it's possible to manually
update things to get Bingo happy.

* undo reformatting

* Various lint improvements

* More from the linter

* goimports -w ./pkg/

* Disable gocritic

* Add/modify linter exceptions

* lint + flatten nested list

Go 1.19 doesn't support nested lists, and there wasn't an obvious workaround.
https://go.dev/doc/comment#lists
2022-09-12 12:03:49 +02:00
Kyle Brandt
05fd7eb047
SSE: Add noData type (#51973)
When there is a single frame with no fields (e.g. splunk datasource) SSE errors when trying to figure out the data type. This frame needs to exist since this is where the executedQueryString metadata exists.

This adds a new return type to SSE to represent no data, so the original frame with its metadata can still be maintained.
2022-07-14 09:18:12 -04:00
Michael Mandrus
9aa6ce2a50
Datasource: Propagate datasource secret decryption errors to the frontend (#52068)
* update decrypt secrets function signature and add secrets error handling

* remove a couple instances of unnecessary logging since errors are properly handled now

* add unit test

* fix linting issues
2022-07-13 09:27:03 -04:00
Kristin Laemmert
945f015770
backend/datasources: move datasources models into the datasources service package (#51267)
* backend/datasources: move datasources models into the datasources service pkg
2022-06-27 12:23:15 -04:00
Kat Yang
3c3039f5b3
Chore: Remove Wrap (#50048)
* Chore: Remove Wrap and Wrapf

* Fix: Add error check
2022-06-03 09:24:24 +02:00
Kyle Brandt
01ef899753
SSE/Alerting: Support prom instant vector responses (#44865)
* SSE/Alerting: (Draft) Support prom instant vector responses
fixes #35663
* reduce\classic expressions to handle mathexp.Number
* use Notice for warning

Co-authored-by: Yuriy Tseretyan <yuriy.tseretyan@grafana.com>
2022-05-23 10:08:14 -04:00
Guilherme Caulada
a367ad730c
Secrets: Implement basic unified secret store service (#45804)
* wip: Implement kvstore for secrets

* wip: Refactor kvstore for secrets

* wip: Add format key function to secrets kvstore sql

* wip: Add migration for secrets kvstore

* Remove unused Key field from secrets kvstore

* Remove secret values from debug logs

* Integrate unified secrets with datasources

* Fix minor issues and tests for kvstore

* Create test service helper for secret store

* Remove encryption tests from datasources

* Move secret operations after datasources

* Fix datasource proxy tests

* Fix legacy data tests

* Add Name to all delete data source commands

* Implement decryption cache on sql secret store

* Fix minor issue with cache and tests

* Use secret type on secret store datasource operations

* Add comments to make create and update clear

* Rename itemFound variable to isFound

* Improve secret deletion and cache management

* Add base64 encoding to sql secret store

* Move secret retrieval to decrypted values function

* Refactor decrypt secure json data functions

* Fix expr tests

* Fix datasource tests

* Fix plugin proxy tests

* Fix query tests

* Fix metrics api tests

* Remove unused fake secrets service from query tests

* Add rename function to secret store

* Add check for error renaming secret

* Remove bus from tests to fix merge conflicts

* Add background secrets migration to datasources

* Get datasource secure json fields from secrets

* Move migration to secret store

* Revert "Move migration to secret store"

This reverts commit 7c3f872072.

* Add secret service to datasource service on tests

* Fix datasource tests

* Remove merge conflict on wire

* Add ctx to data source http transport on prometheus stats collector

* Add ctx to data source http transport on stats collector test
2022-04-25 13:57:45 -03:00
Kristin Laemmert
8f6877e12a
chore: remove all remaining uses of golang.org/x/net/context (#47564)
* chore: remove all remaining uses of  golang.org/x/net/context

This PR finishes the work started in #47532, replacing all calls to that package with the stdlib context and using http.NewRequestWithContext to include the context where necessary.

Bonus: small formatting fixes to goimports in these files.

closes #44178

* tweak: use context.Background in favor of TODO for tests
2022-04-11 14:20:10 -04:00
Serge Zaitsev
18e93c7077
Chore: Remove bus (#47511)
* Chore: Remove bus

* remove unused const
2022-04-08 16:15:06 +02:00
Kyle Brandt
cab509fbe8
SSE: Keep FieldConfig for data source queries (#46328)
for #46325
2022-03-10 10:03:26 -05:00
Kyle Brandt
a578cf0f7c
SSE: Fix NoData when some series were no data but others not (#45867)
Co-authored-by: Santiago <santiagohernandez.1997@gmail.com>
2022-02-24 21:01:04 -03:00
Kyle Brandt
040ce40113
SSE: Mode to drop NaN/Inf/Null in Reduction operations (#43583)
Co-authored-by: Yuriy Tseretyan <yuriy.tseretyan@grafana.com>
Co-authored-by: gillesdemey <gilles.de.mey@gmail.com>
2022-02-02 14:50:44 +01:00
Santiago
1bed84ea56
Fix InfluxDB unhealthy alerts and false "no data"s (#43441)
* check for TimeSeriesTypeNot to fix InfluxDB alerts

* log a warning when a data frame is ignored
2022-01-27 10:53:27 -03:00
Marius Bezuidenhout
9fc0aee02b
Feature/mathandreduce (#41608)
* Added new math functions: round, ceil, floor 
* Added a new reduce function: last.
2022-01-21 13:15:50 -05:00
Kyle Brandt
f17fb76b5e
SSE: Add is_nan() and other like functions and allow '_' in func names (#43555)
* SSE: Add is_nan() and allow '_' in func names

is_null() infn(), is_inf(), is_number()
2021-12-29 11:40:52 -05:00
idafurjes
7936c4c522
Rename AddHandlerCtx to AddHandler (#43557) 2021-12-28 16:08:07 +01:00
Ryan McKinley
2754e4fdf0
Expressions: use datasource model from the query (#41376)
* refactor datasource loading

* refactor datasource loading

* pass uid

* use dscache in alerting to get DS

* remove expr/translate pacakge

* remove dup injection entry

* fix DS type on metrics endpoint, remove SQL DS lookup inside SSE

* update test and adapter

* comment fix

* Make eval run as admin when getting datasource info

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>

* fmt and comment

* remove unncessary/redundant code

Co-authored-by: Kyle Brandt <kyle@grafana.com>
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
Co-authored-by: Santiago <santiagohernandez.1997@gmail.com>
2021-12-16 13:51:46 -03:00
Marcus Efraimsson
8927a3ca20
Chore: Query endpoint refactor (#41637)
Get rid of using legacydata.RequestHandler in HTTPServer, /api/tsdb/query and pkg/expr 
with the goal of deprecating /api/tsdb/query and remove it completely eventually. This is 
the first step of cleaning up the HTTP API query endpoint.

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
Co-authored-by: Alexander Emelin <frvzmb@gmail.com>
2021-11-29 14:21:54 +01:00
Tania B
4014891971
Encryption: Cleanup and add logging (#42084)
* Encryption: Add more logs

* Add logging and checks

* Removed unused methods from secrets service

* Refactor and update tests

* Address review feedback
2021-11-24 15:01:44 +02:00
George Robinson
708bdc80cb
Alerting: Support Unwrap for QueryError in expr package (#41743) 2021-11-17 10:07:24 +00:00
Jean-Philippe Quéméner
2319c52c85
Alerting: fix a bug where the metric in the evaluation string was not correctly populated (#41731)
Instead of using the frame name as the series name, we now use the name of the series value column if the frame name is not set. Queries to some datasources may end up with an empty frame name, which leads to an empty series name. As the series name is used in the frontend to display which metrics were queried etc. it shouldn't be empty.
2021-11-17 11:00:59 +01:00
Tania B
bc60ae3c66
Encryption: Refactor secrets service (#41771)
* Refactor kmsproviders pkg

* Update tests

* Fix linting

Co-authored-by: Joan López de la Franca Beltran <joanjan14@gmail.com>
2021-11-17 11:52:45 +02:00
George Robinson
543b1a7e39
Alerting: Add QueryError to expr package (#41737) 2021-11-16 13:42:22 +00:00
Joan López de la Franca Beltran
44837fc592
Replace encryption.Service usages by secrets.Service (#41625)
* Replace encryption.Service by secrets.Service on expr.Service

* Replace encryption.Service by secrets.Service on live pkg

* Rename encryption.Service to encryption.Internal to clarify it must be not used
2021-11-12 12:16:39 +01:00
Marcus Efraimsson
baab021fec
Chore: Refactor usage of legacy data contracts (#41218)
Refactor usage of legacy data contracts. Moves legacy data contracts 
to pkg/tsdb/legacydata package.
Refactor pkg/expr to be a proper service/dependency that can be provided 
to wire to remove some unneeded dependencies to SSE in ngalert and other places.
Refactor pkg/expr to not use the legacydata,RequestHandler and use 
backend.QueryDataHandler instead.
2021-11-10 11:52:16 +01:00
Ryan McKinley
3489721ed6
api/ds/query: simplify data sources lookup for queries and expressions (#41172) 2021-11-05 08:12:55 -07:00
Dimitris Sotirakis
469a5e4a85
Comment out "Queries with new datasource ref object" test (#41187) 2021-11-02 13:09:51 +02:00
Ryan McKinley
7319efe077
Dashboard: replace datasource name with a reference object (#33817)
Co-authored-by: Torkel Ödegaard <torkel@grafana.com>
Co-authored-by: Elfo404 <me@giordanoricci.com>
2021-10-29 10:57:24 -07:00
Will Browne
e4297006f6
Chore: Propagate context for data source provisioning (#40235)
* context all the things

* apply feedback

* rollback some alerting changes

* rollback some alerting changes #2

* more rollbacks

* more rollbacks #2

* more rollbacks #3

* more rollbacks #4

* fix integration test

* add missing context

* add missing and remove incorrect dispatch
2021-10-18 17:06:19 +02:00
Andres Martinez Gotor
64c8d32fe7
Use sdk pkg for gtime (#39354) 2021-09-21 13:08:52 +02:00
Ikko Ashimine
202f295e51
Chore: Fix typo in nodes.go (#38636)
commmon -> common
2021-08-27 14:57:41 +02:00
Arve Knudsen
78596a6756
Migrate to Wire for dependency injection (#32289)
Fixes #30144

Co-authored-by: dsotirakis <sotirakis.dim@gmail.com>
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
Co-authored-by: Ida Furjesova <ida.furjesova@grafana.com>
Co-authored-by: Jack Westbrook <jack.westbrook@gmail.com>
Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
Co-authored-by: Leon Sorokin <leeoniya@gmail.com>
Co-authored-by: Andrej Ocenas <mr.ocenas@gmail.com>
Co-authored-by: spinillos <selenepinillos@gmail.com>
Co-authored-by: Karl Persson <kalle.persson@grafana.com>
Co-authored-by: Leonard Gram <leo@xlson.com>
2021-08-25 15:11:22 +02:00
Domas
3e124c854e
Alerting: interpolate variables when creating alert rule from dashboard panel (#37201) 2021-08-10 10:59:48 +03:00
Kyle Brandt
b6a7156300
Alerting: Handle marshaling Inf values (#36947)
* SSE: change EvalMatch json value encoding to string
* update MarshalJSON on null.Float to handle inf as null
fixes #36424
2021-07-21 10:04:40 -04:00
Andrej Ocenas
ea2ba06b93
CloudWatch/Logs: Fix log alerts in new unified alerting (#36558)
* Pass FromAlert header from new alerting

* Add better error messages
2021-07-09 13:43:22 +02:00
Sofia Papagiannaki
abe35c8c01
Alerting: Add error recovery during rule evaluations (#35450)
* Alerting: Eval recovery after query failure

* Apply suggestions from code review
2021-06-15 19:30:21 +03:00
Sofia Papagiannaki
c997f646cc
Alerting: Fix alert migration RefID generation (#35667)
* Alerting: Fix alert migration RefID generation

if the alert has more than 26 conditions

* Remove TODO comment

* Make similar change under pkg/expr
2021-06-14 22:13:45 +03:00
Ikko Ashimine
e3afb63e62
Chore: Fix typo in nodes.go (#35312)
formating -> formatting
2021-06-09 16:59:44 -07:00
Kyle Brandt
f4e9dcd40b
Alerting/SSE: Fix "count_non_null" reducer validation (#35451)
fixes error "reducer 'count_non_null' in condition 2 is not a valid reducer"
2021-06-09 14:42:48 -04:00
Sofia Papagiannaki
eeb84d09c2
SSE: Change math expression to accept any value convertible to float (#34996)
* SSE: Change math expression to accept any scalar value

* Apply suggestions from code review

* Update test

* Remove TODO
2021-06-03 18:06:12 +03:00
Marcus Efraimsson
b3e9087557
Chore: Refactor backend plugin manager/tsdb query data (#34944)
Move QueryData method into backend plugin manager which HandleRequest uses to 
query data from plugin SDK supported data sources. This allowed us to remove a lot 
of code no longer needed.

Ref #21510

Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
2021-06-03 14:16:58 +02:00
Kyle Brandt
4093fae99a
SSE: Refactor to simplify Series type (#35063)
Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>
2021-06-02 12:29:19 -04:00
Kyle Brandt
9016d20c4c
SSE/Chore: cleanup some error messages (#34738)
removes "rpc error: code = InvalidArgument desc =" like strings from error messages.

This came from old grpc stuff left from when SSE was a plugin (that probably should
not have been used even when it was a plugin)
2021-05-26 15:15:21 +02:00
Marcus Andersson
e19b3df1a9
Alerting: added possibility to preview grafana managed alert rules. (#34600)
* starting to add eval logic.

* wip

* first version of test rule.

* reverted file.

* add info colum to result to show error or (with CC evalmatches)

* fix labels in evalmatch

* fix be test

* refactored using observables.

* moved widht/height div to outside panel rendere.

* adding docs api level.

* adding container styles to error div.

* increasing size of preview.

Co-authored-by: kyle <kyle@grafana.com>
2021-05-26 10:06:28 +02:00
Will Browne
9726c76429
Chore: Swap out use of backend plugin SDK's backend.Logger (#33783)
* Remove use of backend SDKs backend.Logger

* reorder imports

* FIx logger name

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2021-05-07 15:16:21 +02:00
Kyle Brandt
f5efe97763
SSE: limit allowed input/output to/from classic conditions (#33337)
* add tests for graph and specific cases in this pr
2021-04-27 07:22:11 -04:00
Kyle Brandt
5e818146de
Alerting/Expr: New SSE Request/QueryType, alerting move data source UID (#33282) 2021-04-23 16:52:32 +02:00
Ryan McKinley
7627b55ef4
TSDB: add deprecation comments to many tsdb structs (#33281) 2021-04-23 03:03:11 +02:00