Commit Graph

221 Commits

Author SHA1 Message Date
ismail simsek
994aedaac3
Chore: Update InfluxDB response parser test files (#81899)
* add show diagnostics tests

* update test files
2024-02-06 11:31:55 +01:00
ismail simsek
536153c336
InfluxDB: Run queries in parallel behind influxdbRunQueriesInParallel feature toggle (#81209)
* create the feature flag

* bring the concurrency in to the play

* Update feature flag

* Use concurrency number from settings

* update influxdb dependency

* use ConcurrentQueryCount from plugin-sdk-go

* use helper method for concurrent query count

* log the error

* add value guard

* add unit tests

* handle concurrency error
2024-02-01 11:58:24 +01:00
ismail simsek
177fa1b947
InfluxDB: Fix template variable interpolation (#80971)
* use regex as templateSrv replace format

* use regex as templateSrv replace format for raw queries

* import path fix

* don't use regex formatter

* tag value escape

* tag value escape with wrappers

* polished interpolation logic

* update unit tests

* comments and more place to update

* unit test update

* fix escaping

* handle the string and array of string type separately

* update variable type
2024-02-01 00:30:21 +01:00
ismail simsek
7d0017f3f2
InfluxDB: Use grafana timeout value for flux queries (#81252)
* use grafana timeout for flux queries

* get timeout from settings

* fix test
2024-01-25 17:05:08 +01:00
ismail simsek
14e55fefbb
InfluxDB: Check the value type before casting it to the string (#80986)
* Check the value type before casting it to the string

* set visualization as table by default

* append all values for show diagnostics

* golangci-lint

* append metadata only to first frame
2024-01-25 11:28:25 +01:00
Ryan McKinley
1caaa56de0
FeatureFlags: Use interface rather than manager (#80000) 2024-01-09 10:38:06 -08:00
ismail simsek
2edcf0edbd
InfluxDB: Use database input for SQL configuration instead of metadata (#79579)
* introduce constants file

* update frontend to use database field

* use dbName field instead of metadata

* use secureGrpc

* betterer
2023-12-15 18:41:32 +01:00
ismail simsek
03da1a3f3f
InfluxDB: Update SQL language url specification (#79408)
* accept urls with or without port

* remove redundant comment

* remove redundant function
2023-12-13 16:01:01 +01:00
ismail simsek
c088d003f2
InfluxDB: Implement InfluxQL json streaming parser (#76934)
* Have the first iteration

* Prepare bench testing

* rename the test files

* Remove unnecessary test file

* Introduce influxqlStreamingParser feature flag

* Apply streaming parser feature flag

* Add new tests

* More tests

* return executedQueryString only in first frame

* add frame meta and config

* Update golden json files

* Support tags/labels

* more tests

* more tests

* Don't change original response_parser.go

* provide context

* create util package

* don't pass the row

* update converter with formatted frameName

* add executedQueryString info only to first frame

* update golden files

* rename

* update test file

* use pointer values

* update testdata

* update parsing

* update converter for null values

* prepare converter for table response

* clean up

* return timeField in fields

* handle no time column responses

* better nil field handling

* refactor the code

* add table tests

* fix config for table

* table response format

* fix value

* if there is no time column set name

* linting

* refactoring

* handle the status code

* add tracing

* Update pkg/tsdb/influxdb/influxql/converter/converter_test.go

Co-authored-by: İnanç Gümüş <m@inanc.io>

* fix import

* update test data

* sanity

* sanity

* linting

* simplicity

* return empty rsp

* rename to prevent confusion

* nullableJson field type for null values

* better handling null values

* remove duplicate test file

* fix healthcheck

* use util for pointer

* move bench test to root

* provide fake feature manager

* add more tests

* partial fix for null values in table response format

* handle partial null fields

* comments for easy testing

* move frameName allocation in readSeries

* one less append operation

* performance improvement by making string conversion once

pkg: github.com/grafana/grafana/pkg/tsdb/influxdb/influxql
             │ stream2.txt │            stream3.txt             │
             │   sec/op    │   sec/op     vs base               │
ParseJson-10   314.4m ± 1%   303.9m ± 1%  -3.34% (p=0.000 n=10)

             │ stream2.txt  │             stream3.txt              │
             │     B/op     │     B/op      vs base                │
ParseJson-10   425.2Mi ± 0%   382.7Mi ± 0%  -10.00% (p=0.000 n=10)

             │ stream2.txt │            stream3.txt             │
             │  allocs/op  │  allocs/op   vs base               │
ParseJson-10   7.224M ± 0%   6.689M ± 0%  -7.41% (p=0.000 n=10)

* add comment lines

---------

Co-authored-by: İnanç Gümüş <m@inanc.io>
2023-12-06 12:39:05 +01:00
ismail simsek
2b0e7afb4e
InfluxDB: Fix an edge case while parsing table response (#78802)
* Prepare the test files

* use json files everywhere

* update golden json files

* disable update

* update test file

* fix naming

* lint

* InfluxDB: Add metadata information to first frame only (#78664)

* executedString in first frame only

* lint

* fix tests

* update tests

* don't update

* linting

* update

* update again

* handle nil values

* append in the right array

* add comments

* remove redundant if condition
2023-12-01 18:31:49 +01:00
ismail simsek
dff5022021
Chore: Return executedString information even with frames has no time column (#78906)
* return executedString for no time column responses

* remove comment
2023-11-30 14:25:15 +01:00
ismail simsek
aea5a9f01a
Chore: Introduce util package for InfluxDB backend testing (#78826)
have a util package
2023-11-29 17:48:53 +01:00
ismail simsek
23c7211f1d
Chore: InlfuxDB testing update (#78663)
* Prepare the test files

* use json files everywhere

* update golden json files

* disable update

* update test file

* fix naming

* lint

* InfluxDB: Add metadata information to first frame only (#78664)

* executedString in first frame only

* lint

* fix tests

* update tests

* don't update

* linting

* update

* update again
2023-11-29 16:19:46 +01:00
ismail simsek
c5f3ce1371
InfluxDB: Parse data for table view to have parity with frontend parser (#78365)
* Use TimeSeriesWide format for table response

* fix group by query result parsing

* handle labels

* provide a test where result has no tags

* parsing results without time column

* clean the code

* remove the comment line

* more cleaning

* lint
2023-11-22 19:27:42 +01:00
ismail simsek
5096806eab
InfluxDB: Don't sort retention policies on the backend (#78252)
don't sort retention policies
2023-11-16 10:39:15 +02:00
Ben Tasker
f38f657f87
InfluxDB: Add new truthiness operators (Is and Is Not) to InfluxQL Query Builder (#77923)
* InfluxDB: Add new truthiness operators (`Is` and `Is Not`) to InfluxQL Editor for use with boolean fields

* InfluxDB: Make the front-end aware of the new operators so that translation between raw and builder works

* Chore: Add tests

* feat: identify type of field value to allow other types to work with Is/Is Not

Tags: always quote
Integer: Don't quote
Float: Don't quote
Boolean: Don't quote
String: Quote

* Chore: Add test-cases for type inference

* Update front-end to infer type for operators Is and Is Not

* chore: Add front-end tests

* chore: add additional front-end tests

* chore: fix failing lint test

* chore: fix tests (run prettier)
2023-11-15 21:10:37 +01:00
Kyle Brandt
a94acf4b63
Revert "InfluxDB: Response parser improvements (#76852)" (#78224)
This reverts commit 046791e2be.

Is causing malformed frames, and resulting in errors about mismatched field lengths in the logs, No data in the UI, and will cause SSE to panic.

I think this is because of the global slice vars since it seems to take concurrency to replicate it.
2023-11-15 14:01:36 -05:00
Ben Tasker
c06debe200
InfluxDB: Add support for >= and <= comparison operators to IQL Query Builder (#77917)
* InfluxDB: Add support for `>=` and `<=` comparison operators to InfluxQL Query Builder

* Add front-end support for the new operators

This ensures that the query translates correctly between raw and builder mode

* Chore: add test for new operators

* chore: add front-end tests

* fix: don't skip quoting on `<>`

This preserves the pre-existing behaviour, fixing a failing test

* chore: fix tests
2023-11-15 12:21:41 +01:00
ismail simsek
6b13064cf6
InfluxDB: Fix parsing multiple tags on backend mode (#77340)
* Multiple tags separated by comma in the result

* A non-flaky unit test
2023-10-30 15:45:20 +01:00
ismail simsek
4ed36cbc1d
InfluxDB: Fix parsing empty response (#77353)
Fix parsing empty response
2023-10-30 11:39:15 +01:00
ismail simsek
046791e2be
InfluxDB: Response parser improvements (#76852)
* remove retention policy lookup

* Back to one big function

* %10 less memory allocation

pkg: github.com/grafana/grafana/pkg/tsdb/influxdb/influxql
                │    1.txt    │                2.txt                │
                │   sec/op    │   sec/op     vs base                │
ParseBigJson-10   540.9m ± 3%   474.0m ± 2%  -12.37% (p=0.000 n=10)

                │    1.txt     │                2.txt                │
                │     B/op     │     B/op      vs base               │
ParseBigJson-10   580.6Mi ± 0%   573.2Mi ± 0%  -1.28% (p=0.000 n=10)

                │    1.txt     │                2.txt                │
                │  allocs/op   │  allocs/op   vs base                │
ParseBigJson-10   10.123M ± 0%   9.086M ± 0%  -10.25% (p=0.000 n=10)

* Slightly better results comparing with the previous commit
pkg: github.com/grafana/grafana/pkg/tsdb/influxdb/influxql
                │    2.txt    │               3.txt                │
                │   sec/op    │   sec/op     vs base               │
ParseBigJson-10   474.0m ± 1%   503.4m ± 3%  +6.21% (p=0.000 n=10)

                │    2.txt     │                3.txt                │
                │     B/op     │     B/op      vs base               │
ParseBigJson-10   573.2Mi ± 0%   564.0Mi ± 0%  -1.60% (p=0.000 n=10)

                │    2.txt    │               3.txt                │
                │  allocs/op  │  allocs/op   vs base               │
ParseBigJson-10   9.086M ± 0%   9.052M ± 0%  -0.37% (p=0.000 n=10)

* Split into smaller functions

* Unit test for parseTimestamp
2023-10-27 17:17:19 +02:00
ismail simsek
63abe25b74
InfluxDB: Fix table parsing with backend mode (#76899)
* Add resultFormat to query model

* don't add row name if the result format is table

* No need special formatting since we use unified dataframes

* betterer

* specify visualization type in response

* Unit tests

* fix unit tests
2023-10-23 19:21:30 +02:00
ismail simsek
5eb0b2bedb
InfluxDB: Fix aliasing with $measurement or $m on backend mode (#76917)
* better interpolation $measurement aliasing

* unit tests
2023-10-23 17:36:51 +02:00
Will Browne
9d92818cae
Plugins: Update PDC pattern from latest plugin SDK changes (#76576)
* update with sdk

* do sql

* fix core plugins

* fix proxy settings

* bump SDK version

* tidy

* enable pdc for test

* add codeowners

* bump dep

* go mod tidy

* bump SDK
2023-10-16 16:40:04 +02:00
Dan Cech
91cf4f0c1c
Revert "Plugins: Update PDC pattern from latest plugin SDK changes" (#76565)
Revert "Plugins: Update PDC pattern from latest plugin SDK changes (#76036)"

This reverts commit 151f6d6216.
2023-10-13 12:35:16 -04:00
ismail simsek
2877723648
InfluxDB: Use apache/arrow v13 (#76548)
* Use apache/arrow v13

* remove apache/thrift

* go mod tidy with go1.21.1

* add metrics team as owner

---------

Co-authored-by: Kyle Brandt <kyle@grafana.com>
2023-10-13 17:06:10 +02:00
Will Browne
151f6d6216
Plugins: Update PDC pattern from latest plugin SDK changes (#76036)
* update with sdk

* do sql

* fix core plugins

* fix proxy settings

* bump SDK version

* tidy

* enable pdc for test

* add codeowners
2023-10-13 14:42:15 +02:00
ismail simsek
2d603bed22
InfluxDB SQL: Add time filter automatically (#75953)
* Use double quotes instead of backticks

* render time field as datetime

* add @react-awesome-query-builder/core

* Add time range filtering

* Update test

* Update macro test
2023-10-04 17:00:24 +02:00
ismail simsek
a64bfdfb94
InfluxDB SQL: Provide raw query preview for query history (#75030)
* log the sql query

* render the raw sql

* Update rawQuery

* unit tests
2023-09-22 17:40:18 +02:00
Will Browne
7fca1bde54
Plugins: Set grafana config, plugin version and user agent on plugin requests (#75171)
* first pass

* fixup

* remove test line

* fix tests

* use new fields

* fix imports + formatting

* fix tests

* rollback changes

* undo whitespace

* apply pr feedback
2023-09-21 11:33:31 +02:00
Serge Zaitsev
93cdc94a94
Chore: capitalise logs in other backend packages (#74344)
* capitalise logs in observability logs

* capitalise oss-bit-tent packages

* capitalise logs in aws-datasources

* capitalise logs for traces and profiling

* capitalise logs for partner datasources

* capitalise logs in plugins platform

* capitalise logs for observability metrics
2023-09-04 22:25:43 +02:00
Ryan McKinley
025b2f3011
Chore: use any rather than interface{} (#74066) 2023-08-30 18:46:47 +03:00
ismail simsek
29ea0886e3
InfluxDB: Fix sending retention policy with the backend request (#72763)
* Add retention policy to the request

* refactor

* refactor influxql query flow

* fix healthcheck

* organize imports

* handle queries separately

* fix tests

* update bench test
2023-08-30 12:42:02 +02:00
ismail simsek
1ac9e7eaab
InfluxDB: Fix handling flux response with no time and value column (#72833)
* Upgrade the influxdb-client-go version

* Handle flux response with no time and value column
2023-08-08 15:00:52 +02:00
ismail simsek
d333c09418
InfluxDB: SQL Query Editor (#72168)
* Add influxdbSqlSupport feature toggle

* Add SQL option to the config page

* Add SQL backend

* Add metadata support in config page

* Implement unified querying

* Fix healthcheck query

* fsql tests

* secure grpc by default

* code cleanup

* Query handing for sql mode

* Implement a placeholder sql editor

* Fix query language dropdown

* drop in SQL editor

* switch to use rawSql, get sql editor working

* fix healthcheck

* WIP

* memoize component to stop unwanted rerender onQuery

* dont reinit datasource on each render of the editor

* remove useless memo

* clean up

* Fix the link

* Alpha state warning

* Remove console.logs

* update model for fsql

* remove unused

---------

Co-authored-by: Galen <galen.kistler@grafana.com>
2023-08-02 19:04:16 +02:00
ismail simsek
77e7ae2a1b
InfluxDB: SQL Support (#72167)
* Add influxdbSqlSupport feature toggle

* Add SQL option to the config page

* Add SQL backend

* Add metadata support in config page

* Implement unified querying

* Fix healthcheck query

* fsql tests

* secure grpc by default

* code cleanup

* Query handing for sql mode

* Implement a placeholder sql editor

* Fix query language dropdown

* go mod updates

* make lint-go

* more make lint-go

* remove unused runQuery

* switch statements with default case

* linting again
2023-08-02 18:27:43 +02:00
ismail simsek
61a77b4f6b
InfluxDB: Return better error messages from backend (#70066)
Co-authored-by: ludovio <ludovic.viaud@gmail.com>
2023-07-04 13:55:08 +02:00
ismail simsek
80c432e524
InfluxDB: Fix backend mode table result with aliases (#69943)
Co-authored-by: ludovio <ludovic.viaud@gmail.com>
2023-07-03 14:50:08 +02:00
ismail simsek
903af7e29c
InfluxDB: Backend parser compatibility with frontend parser (#69865)
* Reformatting and restructuring

* Update unit test

* Always send the default retention policy as first element

* Fix typo

* Update test

* Update test once more

* Field names start with capital letters

* Simplify the condition

* Case-insensitive checks

* Fix typo

* Update response_parser test

* Update imports
2023-06-26 14:03:02 +02:00
Andreas Deininger
95b1f3c875
Fixing typos (#70487) 2023-06-22 09:43:38 +01:00
ismail simsek
e3ea5422ea
InfluxDB: Return default retention policy from backend as first element (#69818)
* Reformatting and restructuring

* Update unit test

* Always send the default retention policy as first element

* Fix typo

* Update test

* Update test once more
2023-06-17 14:20:54 +02:00
Eng Zer Jun
7f7b03d794
Chore: Avoid unnecessary byte/string conversions (#69001)
Avoid unnecessary byte/string conversion

We can use alternative functions/methods to avoid unnecessary
byte/string conversion calls.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2023-05-26 10:08:50 +00:00
Will Browne
286b9e08e9
Plugins: Bump Plugin SDK version and address instance management breaking changes (#68900)
* bump sdk and propagate ctx

* fix tests
2023-05-24 10:19:34 +02:00
Kyle Brandt
6c6427e63f
Chore: Use latest go-sdk (v0.158.0) (#66054)
SDK release notes:
* Logs contract: ignore remaining fields by @gabor in #659
* Logs contract: more robust field finding, explicit approach by @gabor in #660
* Tracing: Support multiple OTel propagators by @xnyo in #663
* Tracing: Add more details to HTTP Outgoing Request by @xnyo in #664
* Data: Encode Nanosecond into JSON by @kylebrandt in #647
* Data: cmp tests using FrameTestCompareOptions() will no longer ignore time differences beyond millisecond resolution
2023-04-05 21:00:31 +03:00
ismail simsek
ae07bf7ce5
Influxdb: Response parser performance improvement (#64776)
* Influxdb response parser allocation improvements

* More improvements

* more improvements

* embed file

* use json-iterator

* rename

* use encoding/json
2023-03-17 20:45:13 +01:00
Brendan O'Handley
50ef183208
InfluxDB: Fix type assertion panics for interface conversion: interface {} is nil (#64556)
* fix type assertions

* fix chk

* add nil and fix tests

* add test for null bool and strings in the JSON response to be parsed as nil in Go

* reduce complexity

* reduce complexity

* reduce complexity again

* switch case for valTypes
2023-03-10 11:11:00 -05:00
Serge Zaitsev
0bdb105df2
Chore: Remove xorcare/pointer dependency (#63900)
* Chore: remove pointer dependency

* fix type casts

* deprecate xorcare/pointer library in linter

* rooky mistake
2023-03-06 05:23:15 -05:00
Ludovic Viaud
5bd2fac9c8
InfluxDB: Update InfluxQL querybuilder to qualify identifiers (#62230) 2023-03-02 15:05:24 +01:00
Kyle Brandt
52ed297ed5
Backend: Use latest plugin-sdk-go v0.149.1 (#63498)
Use latest github.com/grafana/grafana-plugin-sdk-go which includes changes to the TypeVersion property (always present in JSON).

Also included is squtil changes: SQL util - allow using the database scan type for converters
2023-02-21 09:06:15 -05:00
ismail simsek
8684fa75c7
Remove explicit user agent definition (#63019) 2023-02-07 07:49:20 -08:00