Commit Graph

83 Commits

Author SHA1 Message Date
Marcus Efraimsson
348e76fc8e
Datasource: Shared HTTP client provider for core backend data sources and any data source using the data source proxy (#33439)
Uses new httpclient package from grafana-plugin-sdk-go introduced 
via grafana/grafana-plugin-sdk-go#328. 
Replaces the GetHTTPClient, GetTransport, GetTLSConfig methods defined 
on DataSource model.
Longer-term the goal is to migrate core HTTP backend data sources to use the 
SDK contracts and using httpclient.Provider for creating HTTP clients and such.

Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>
2021-05-19 23:53:41 +02:00
Giordano Ricci
8ec87250c1
Elasticsearch: Fix handling of inline scripts in different ES versions (#34070)
* Devenv: add block for es 5.0, provisioned datasource & dashboard

* Trasnsform script property based on running ES version

* Handle different scripts format in BE
2021-05-14 11:50:15 +01:00
Giordano Ricci
e98a8bd11b
Elasticsearch: use semver strings to identify ES version (#33646)
* Elasticsearch: use proper semver strings to identify ES version

* Update BE & tests

* refactor BE tests

* refactor isValidOption check

* update test

* Update pkg/tsdb/elasticsearch/client/client.go

Co-authored-by: Piotr Jamróz <pm.jamroz@gmail.com>

* Update pkg/tsdb/elasticsearch/client/search_request_test.go

Co-authored-by: Piotr Jamróz <pm.jamroz@gmail.com>

* Remove leftover FIXME comment

* Add new test cases for new version format

* Docs: add documentation about version dropdown

* Update docs/sources/datasources/elasticsearch.md

Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>

* Update docs/sources/datasources/elasticsearch.md

Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>

* Update docs/sources/datasources/elasticsearch.md

Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>

* Update provisioning documentation

Co-authored-by: Piotr Jamróz <pm.jamroz@gmail.com>
Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>
2021-05-11 09:44:00 +01:00
Giordano Ricci
c88af6e221
Elasticsearch: Add generic support for template variables (#32762)
* Elasticsearch: Add generic support for template variables

* format MovingAverage settings as numbers

* Move formatting logic to query builder & forma serial_diff settings as numbers

* modify presence check

* add todo

* minor fixes

* transform string values to numbers

* Move casting logic

* Slightly cleaner implementation

* Add BE tests

* Leverage elastic validation when string doesn't resolve to a numeric value

* move newly introduced test to testify

* add FE query_builder tests

* check error

* Parse values to float instead of int

* Fix tests & ParseFloat bit size
2021-04-26 16:54:23 +01:00
Ryan McKinley
7627b55ef4
TSDB: add deprecation comments to many tsdb structs (#33281) 2021-04-23 03:03:11 +02:00
Giordano Ricci
cdb4785496
Elasticsearch: Allow omitting field when metric supports inline script (#32839)
* Elasticsearch: Allow omitting field when metric supports inline script

* Add tests for MetricEditor to show a None option

* Add tests for useFields hook

* Alerting: allow elasticsearch metrics without field
2021-04-14 15:18:06 +01:00
Giordano Ricci
f7b408f99f
Elasticsearch: use application/x-ndjson content type for multisearch requests (#32282) 2021-03-29 23:41:45 +01:00
Arve Knudsen
b79e61656a
Introduce TSDB service (#31520)
* Introduce TSDB service

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

Co-authored-by: Erik Sundell <erik.sundell87@gmail.com>
Co-authored-by: Will Browne <will.browne@grafana.com>
Co-authored-by: Torkel Ödegaard <torkel@grafana.org>
Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
Co-authored-by: Zoltán Bedi <zoltan.bedi@gmail.com>
2021-03-08 07:02:49 +01:00
Chris Cowan
5088e2044a
Elasticsearch: Support extended stats and percentiles in terms order by (#28910)
Adds support to the terms aggregation for ordering by percentiles and extended stats. 

Closes #5148

Co-authored-by: Giordano Ricci <grdnricci@gmail.com>
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2021-01-15 12:10:16 +01:00
Chris Cowan
3d7748d9f8
Elasticsearch: Use minimum interval for alerts (#30049)
The current backend code doesn't honor the minimum interval set in the UI for alerts using 
the Elasticsearch data source. This means that the data the alerts are triggering against 
will never match the data in the visualization if auto is used in the date histogram as interval. 
This fixes the problem to make sure that date histogram auto interval is set according to 
min interval set in UI for the query or fallback to data source min interval setting.

Fixes #22082

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2021-01-14 10:48:59 +01:00
Chris Cowan
cf1c01dd8b
Elasticsearch: Add Support for Serial Differencing Pipeline Aggregation (#28618)
* Elasticsearch: Add support for serial diff pipeline aggregation

* Removing settings transsforms

* Removing unused deps

* removing unused dep

* Fixing type in test

* Adding backend support for serial_diff
2021-01-08 11:52:23 +00:00
Emil Hessman
b2b3a603e8
Chore: Rewrite elasticsearch client test to standard library (#30093) 2021-01-07 09:35:56 +01:00
Arve Knudsen
c2cad26ca9
Chore: Disable default golangci-lint filter (#29751)
* Disable default golangci-lint filter

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Chore: Fix linter warnings

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2020-12-15 09:32:06 +01:00
Arve Knudsen
ac09baae7d
Chore: Use Header.Set method instead of Header.Add (#29804)
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2020-12-14 15:13:01 +01:00
Arve Knudsen
f55818ca70
Chore: Enable exhaustive linter (#29458)
* Chore: Enable exhaustive linter

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2020-12-01 09:53:27 +01:00
Tom Daly
ee3475d900
Elasticsearch: Fix index pattern not working with multiple base sections (#28348)
* Add logic to allow for multiple [] sections in pattern

* Small refactor

* Fix goimport error

* Refactor formateDate
2020-11-27 10:50:35 +01:00
Arve Knudsen
4dd7b7a82d
Chore: Remove unused Go code (#28852)
* Chore: Remove more unused Go code

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2020-11-17 11:51:31 +01:00
Chris Cowan
4ed7612e57
Elasticsearch: Add Moving Function Pipeline Aggregation (#28131)
* Elasticsearch: Add Moving Function Pipeline Aggregation

* Removing support for moving average in 7.x

* Fixing getMetricAggTypes to handle undefined esVersion

* Adding moving_fn to go code

* Update public/app/plugins/datasource/elasticsearch/metric_agg.ts

Co-authored-by: Giordano Ricci <grdnricci@gmail.com>

* Adding test for esversion 70

* Removing default value for script, adding placeholder instead

* Fixing formatting

* Adding code for handling missing or obsolete aggregations

Co-authored-by: Giordano Ricci <grdnricci@gmail.com>
2020-11-11 17:42:28 +00:00
Giordano Ricci
4c7131425b
Revert "Elasticsearch: add frozen indices search support (#27472)" (#27726)
This reverts commit 61a3160c34.
2020-09-23 12:24:46 +01:00
Arve Knudsen
a5d9196a53
Chore/fix lint issues (#27704)
* Chore: Fix linting issues

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2020-09-22 16:22:19 +02:00
Giordano Ricci
61a3160c34
Elasticsearch: add frozen indices search support (#27472)
* Elasticsearch: Add frozen indices support for alerting
2020-09-18 18:27:25 +01:00
Arve Knudsen
7896836deb
Chore: Reduce TSDB Go code complexity (#26401)
* tsdb: Make code less complex
2020-08-18 14:43:18 +02:00
Arve Knudsen
d4e4cb4c71
Chore: Enable Go linter gocritic (#26224)
* Chore: Enable gocritic linter

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2020-07-16 14:39:01 +02:00
Marcelo Nunes Alves
be961c5466
Elasticsearch: Fix using multiple bucket script aggregations when only grouping by terms (#24064)
* In the parser when it was configurate more bucket it put the formula in name column

* Tests

* Tests

* Tests

* Tests

* Tests

* Tests

* Process/Tests - backend (Go)

* Update pkg/tsdb/elasticsearch/response_parser.go

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>

* reverse

* Update pkg/tsdb/elasticsearch/response_parser.go

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2020-07-09 15:21:19 +02:00
Arve Knudsen
41d432b5ae
Chore: Enable whitespace linter (#25903)
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2020-07-06 20:17:28 +02:00
Arve Knudsen
d1e6214a4a
Chore: Enable scopelint Go linter (#25896)
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2020-06-29 17:04:38 +02:00
Arve Knudsen
5070f7a75b
Chore: Start harmonizing linting with plugin SDK (#25854)
* Chore: Harmonize linting with plugin SDK

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Chore: Fix linting issues

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2020-06-29 14:08:32 +02:00
Marcus Efraimsson
58cefe73ee
Datasources: Handle URL parsing error (#25742)
Adds handling of error returned from URL parsing.

Fixes #25714
2020-06-22 16:34:40 +02:00
Kevin Retzke
51d1261a4b
Elasticsearch: Adds cumulative sum aggregation support (#24820)
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2020-06-01 23:55:15 +02:00
Arve Knudsen
bb50fc5965
pkg/tsdb: Check errors (#19837)
Check errors in pkg/tsdb.
2019-10-22 16:16:53 +02:00
Arve Knudsen
39d8e73412
Chore: Upgrade to Go 1.13 (#19502)
* Chore: Upgrade to Go 1.13

Fixes: #18878

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Chore: Upgrade lint tools in order to work with Go 1.13

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Chore: Fix Go linting issues

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2019-10-02 13:59:05 +02:00
Oleg Gaidarenko
caa1314f44
Build: use golangci-lint as a make command (#17739)
* Build: use golangci-lint as a make command

* Since gometalinter was deprecated in favor of golangci-lint so it was
  replaced by it. Responsibilities held by the gometalinter was moved to
  golangci-lint

* There was some changes in implementation (that was also mentioned in
  the code comment) between the tools, which uncovered couple errors
  in the code. Those issues were either solved or disabled by
  the inline comments

* Introduce the golangci-lint config, to make their
  configuration more manageable

* Build: replace backend-lint.sh script with make
2019-07-02 16:06:59 +03:00
Marcus Efraimsson
5713048f48
Alerting: Improve alert rule testing (#16286)
* tsdb: add support for setting debug flag of tsdb query

* alerting: adds debug flag in eval context

Debug flag is set when testing an alert rule and this debug
flag is used to return more debug information in test aler rule
response. This debug flag is also provided to tsdb queries so
datasources can optionally add support for returning additional
debug data

* alerting: improve test alert rule ui

Adds buttons for expand/collapse json and copy json to clipboard,
very similar to how the query inspector works.

* elasticsearch: implement support for tsdb query debug flag

* elasticsearch: embedding client response in struct

* alerting: return proper query model when testing rule
2019-06-25 08:52:17 +02:00
zhulongcheng
2fff8f77dc move log package to /infra (#17023)
ref #14679

Signed-off-by: zhulongcheng <zhulongcheng.me@gmail.com>
2019-05-13 08:45:54 +02:00
Marcus Efraimsson
32a45077e7
Elasticsearch: Fix pre-v7.0 and alerting error (#16904)
This fixes a regression introduced in #16646 where using Elasticsearch 
pre-v7.0 and alerting resulted in an error when trying to deserialize the 
response of total number of hits.
Total number of hits is not in use by the backend so we're removing it 
for now to make ES 6 and 7 being able to deserialize search responses 
without errors.

Closes #15622
2019-05-06 15:12:18 +02:00
Alcides Viamontes E
cff2be0d66 elasticsearch: add 7.x version support (#16646)
Adds a new version option 7.0+ (70 internally).
Version 7.0+ doesn't include document types in index mappings 
so support for handling this have been added.
Version 7.0+ returns number of hits in a different way so 
support for handling this have been added.
Version 7.0+ doesn't support sending 
max_concurrent_shard_requests in multisearch header so 
support for sending this in query string have been added.
Update elastic6 docker block and dashboards (devenv) to use 
6.7.1 images, filebeat index name is now filebeat-YYYY.MM.DD 
and dashboard include correct tags and links.
Add elastic7 docker block and provisioning (devenv).
Updates documentation regarding new version.

Closes #15622
2019-04-25 09:41:13 +02:00
Andrej Ocenas
66f6e16916
Security: Store datasource passwords encrypted in secureJsonData (#16175)
* Store passwords in secureJsonData

* Revert unnecessary refactors

* Fix for nil jsonSecureData value

* Remove copied encryption code from migration

* Fix wrong field reference

* Remove migration and provisioning changes

* Use password getters in datasource proxy

* Refactor password handling in datasource configs

* Add provisioning warnings

* Update documentation

* Remove migration command, moved to separate PR

* Remove unused code

* Set the upgrade version

* Remove unused code

* Remove double reference
2019-04-15 11:11:17 +02:00
sandeep
e04218e550 Fix: #14706 Incorrect index pattern padding in alerting queries
When weekly index pattern is used, indices names contain single digit week number for week 1-9
This fix makes sure indice names always contain 2 digit week number for weekly pattern
2019-03-03 18:10:38 +05:30
Marcus Efraimsson
b45f72a140
elasticsearch: support bucket script pipeline aggregations 2018-12-20 01:24:36 +01:00
Marcus Efraimsson
c46d01dc83
Merge pull request #14090 from marefr/8843_pipeline_doc_count
Fix pipeline aggregations on doc count
2018-12-12 13:21:13 +01:00
Mario Trangoni
fadd816d89 pkg/*: Fix misspell issues
See,
$ find . -type f | xargs misspell -locale US | grep -vi -e vendor -e devenv -e Unknwon -e Destory | grep pkg
./pkg/services/sqlstore/org_test.go:190:15: "frome" is a misspelling of "from"
./pkg/tsdb/elasticsearch/response_parser.go:544:24: "Unkown" is a misspelling of "Unknown"
./pkg/tsdb/opentsdb/opentsdb.go:87:20: "marshalling" is a misspelling of "marshaling"
./pkg/components/dynmap/dynmap_test.go:2:7: "Licence" is a misspelling of "License"
./pkg/components/dynmap/dynmap.go:2:7: "Licence" is a misspelling of "License"
2018-12-04 23:29:34 +01:00
Mattia Rossi
ce74b1ddc2 Requested Backend changes, removed link in popover description for the offset field 2018-11-27 03:42:20 -05:00
Mattia Rossi
f6fc7f7b64 Requested Backend changes, added details to popover description for the offset field 2018-11-26 08:58:25 -05:00
Marcus Efraimsson
a022284cb0
fix handle of elasticsearch 6.0+ version 2018-11-23 15:24:16 +01:00
Marcus Efraimsson
9a8ad70013
fix pipeline aggregations on doc count 2018-11-15 19:06:47 +01:00
Marcus Efraimsson
5803bfd2c7
fix terms agg order deprecation warning on es 6+ 2018-11-02 17:53:47 +01:00
Yuan Liu
18dfdc4f0d
add test for es alert when group by has no limit 2018-10-11 16:04:57 +08:00
Yuan Liu
f8a8b213f9
remove tab 2018-10-11 11:40:23 +08:00
Yuan Liu
567db87c3a
bug fix
bug fix
2018-10-11 11:36:40 +08:00
Yuan Liu
3b9ab6e204
Update time_series_query.go
fix alert no data when elasticsearch group by terms size is set to no limit
2018-10-11 11:21:06 +08:00