Commit Graph

244 Commits

Author SHA1 Message Date
Andreas Christou
488e71226b
InfluxDB: Add support for LIST arrow type (#93227)
Add support for LIST arrow type
2024-09-24 18:36:07 +01:00
Andreas Christou
f5926af99a
InfluxDB: Update macro regexp (#93040)
Update macro and tests
2024-09-11 14:26:09 +01:00
Dave Henderson
df3d8915ba
Chore: Bump Go to 1.23.0 (#92105)
* chore: Bump Go to 1.23.0

Signed-off-by: Dave Henderson <dave.henderson@grafana.com>

* update swagger files

Signed-off-by: Dave Henderson <dave.henderson@grafana.com>

* chore: update .bingo/README.md formatting to satisfy prettier

Signed-off-by: Dave Henderson <dave.henderson@grafana.com>

* chore(lint): Fix new lint errors found by golangci-lint 1.60.1 and Go 1.23

Signed-off-by: Dave Henderson <dave.henderson@grafana.com>

* keep golden file

* update openapi

* add name to expected output

* chore(lint): rearrange imports to a sensible order

Signed-off-by: Dave Henderson <dave.henderson@grafana.com>

---------

Signed-off-by: Dave Henderson <dave.henderson@grafana.com>
Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
2024-08-21 11:40:42 -04:00
ismail simsek
4fcd348366
InfluxDB: Fix returning InfluxDB error messages (#89973)
* Revert "Chore: Return influxdb query error early before parsing the result (#88549)"

This reverts commit a87c155c06.

* Handle error in buffered parser

* handle error message in streaming parser
2024-07-04 17:45:36 +02:00
ismail simsek
dfcda2b14a
InfluxDB: Don't interpolate bucket keyword in flux language if it is part of a join query (#89298)
don't interpolate bucket keyword if it is part of a join query
2024-06-19 08:22:16 +02:00
ismail simsek
92233350f8
InfluxDB: Add custom metadata to the first frame via header (#88698)
* add custom metadata via header to the response

* use strings.CutPrefix instead of regex
2024-06-17 20:02:53 +03:00
Brendan O'Handley
8fddf30621
InfluxDB: Flight SQL test, add function to search for free port (#89255)
* add function to search for free port

* Update pkg/tsdb/influxdb/fsql/fsql_test.go

Co-authored-by: Dave Henderson <dave.henderson@grafana.com>

* Update pkg/tsdb/influxdb/fsql/fsql_test.go

Co-authored-by: Dave Henderson <dave.henderson@grafana.com>

* fix test

* fix go linting issue

* fix go lint

---------

Co-authored-by: Dave Henderson <dave.henderson@grafana.com>
2024-06-17 09:16:52 -05:00
Dave Henderson
6262c56132
chore(perf): Pre-allocate where possible (enable prealloc linter) (#88952)
* chore(perf): Pre-allocate where possible (enable prealloc linter)

Signed-off-by: Dave Henderson <dave.henderson@grafana.com>

* fix TestAlertManagers_buildRedactedAMs

Signed-off-by: Dave Henderson <dave.henderson@grafana.com>

* prealloc a slice that appeared after rebase

Signed-off-by: Dave Henderson <dave.henderson@grafana.com>

---------

Signed-off-by: Dave Henderson <dave.henderson@grafana.com>
2024-06-14 14:16:36 -04:00
ismail simsek
808cf75ff8
InfluxDB: Use json-iterator package for json operations (#88562)
* return error early

* enable gzip between grafana and influxdb

* use json-iterator package for json operations

* revert gzip changes

* update test

* go mod tidy
go work sync
2024-06-06 20:14:53 +03:00
ismail simsek
08276bd8e7
InfluxDB: Revert calculation of maxPointsEnforced value (#88809)
revert calculation of maxPointsEnforced
2024-06-05 21:43:04 +02:00
ismail simsek
a87c155c06
Chore: Return influxdb query error early before parsing the result (#88549)
return error early
2024-06-05 21:36:13 +02:00
ismail simsek
c73bbf19a1
InfluxDB: Introduce maxDataPoints setting for flux variable query editor (#87935)
* Introduce custom variable support

* Remove comment lines

* introduce maxDataPoints

* fix tests

* update

* fix unit tests

* remove new line
2024-06-03 12:09:33 +03:00
ismail simsek
194039c429
InfluxDB: Update alias regex pattern (#87713)
* use frontend mode alias regex

* remove comment
2024-05-14 09:53:18 +02:00
ismail simsek
3317691615
InfluxDB: Support cardinality queries with backend mode (#87264)
support cardinality queries
2024-05-03 12:50:00 +02:00
ismail simsek
96f49bfaee
InfluxDB: Fix panic when reading concurrency count during influxql health check (#85771)
fix panic when reading concurrency count during influxql health check
2024-04-10 15:32:38 +02:00
ismail simsek
fec7765111
Chore: InfluxQL stream parser improvements (#85041)
* don't iterate over first column as it is a time column already

* don't iterate over first column as it was handled earlier

* add more flexibility to run the commands

* Update pkg/tsdb/influxdb/influxql/converter/converter.go

Co-authored-by: Nick Richmond <5732000+NWRichmond@users.noreply.github.com>

---------

Co-authored-by: Nick Richmond <5732000+NWRichmond@users.noreply.github.com>
2024-03-26 19:25:03 +02:00
ismail simsek
016d6f2f6d
InfluxDB: Fix alias interpolation when it has $__interval or multiple tags (#84940)
fix $__interval interpolating in alias
2024-03-22 11:27:04 +01:00
Josh Michielsen
297d73a7df
InfluxDB: Add configuration option for enabling insecure gRPC connections (#83834)
* InfluxDB: add configuration option for enabling insecure gRPC connections

* fix: add insecureGrpc to InfluxOptions

* rename options label 'gRPC' -> 'Connection'

Co-authored-by: ismail simsek <ismailsimsek09@gmail.com>

* update docs: rename options label 'gRPC' -> 'Connection'

Co-authored-by: ismail simsek <ismailsimsek09@gmail.com>

* default insecure connection boolean to false in frontend

Co-authored-by: ismail simsek <ismailsimsek09@gmail.com>

* run prettier:write

---------

Co-authored-by: ismail simsek <ismailsimsek09@gmail.com>
2024-03-12 13:36:11 +01:00
ismail simsek
2cce9aa2f7
Chore: Move tracing function in influxdb package (#83899)
move tracing function in influxdb package
2024-03-05 07:34:08 -06:00
ismail simsek
94f544c9f6
InfluxDB: Fix tag interpolation when varable used within a regex pattern (#82785)
* fix tag interpolation

* remove redundant variables
2024-02-16 10:20:09 +01:00
ismail simsek
3f940f4da1
Chore: Use jsoniter in influxdb from grafana-plugin-sdk-go (#82360)
use jsoniter from grafana-plugin-sdk-go
2024-02-13 17:47:00 +01:00
Ryan McKinley
0c6e409350
Chore: Update arrow and prometheus dependencies (#82215)
* update arrow and prometheus

* keep codeowner

* use compare

* use grafana-plugin-sdk-go v0.210.0

---------

Co-authored-by: ismail simsek <ismailsimsek09@gmail.com>
2024-02-13 01:50:25 +01:00
ismail simsek
b0dfeb1911
Chore: Clean up intervalv2 functions (#82074)
* clean up intervalv2 functions

* use roundInterval from grafana-plugin-sdk-go

* use from grafana-plugin-sdk-go

* have intervalv2 in publicdashboards and remove tsdb/intervalv2

* legacydata cleanup

* remove unused variables

* Update pkg/tsdb/legacydata/interval/interval.go

Co-authored-by: Arati R. <33031346+suntala@users.noreply.github.com>

---------

Co-authored-by: Arati R. <33031346+suntala@users.noreply.github.com>
2024-02-09 11:39:21 +01:00
ismail simsek
994aedaac3
Chore: Update InfluxDB response parser test files (#81899)
* add show diagnostics tests

* update test files
2024-02-06 11:31:55 +01:00
ismail simsek
536153c336
InfluxDB: Run queries in parallel behind influxdbRunQueriesInParallel feature toggle (#81209)
* create the feature flag

* bring the concurrency in to the play

* Update feature flag

* Use concurrency number from settings

* update influxdb dependency

* use ConcurrentQueryCount from plugin-sdk-go

* use helper method for concurrent query count

* log the error

* add value guard

* add unit tests

* handle concurrency error
2024-02-01 11:58:24 +01:00
ismail simsek
177fa1b947
InfluxDB: Fix template variable interpolation (#80971)
* use regex as templateSrv replace format

* use regex as templateSrv replace format for raw queries

* import path fix

* don't use regex formatter

* tag value escape

* tag value escape with wrappers

* polished interpolation logic

* update unit tests

* comments and more place to update

* unit test update

* fix escaping

* handle the string and array of string type separately

* update variable type
2024-02-01 00:30:21 +01:00
ismail simsek
7d0017f3f2
InfluxDB: Use grafana timeout value for flux queries (#81252)
* use grafana timeout for flux queries

* get timeout from settings

* fix test
2024-01-25 17:05:08 +01:00
ismail simsek
14e55fefbb
InfluxDB: Check the value type before casting it to the string (#80986)
* Check the value type before casting it to the string

* set visualization as table by default

* append all values for show diagnostics

* golangci-lint

* append metadata only to first frame
2024-01-25 11:28:25 +01:00
Ryan McKinley
1caaa56de0
FeatureFlags: Use interface rather than manager (#80000) 2024-01-09 10:38:06 -08:00
ismail simsek
2edcf0edbd
InfluxDB: Use database input for SQL configuration instead of metadata (#79579)
* introduce constants file

* update frontend to use database field

* use dbName field instead of metadata

* use secureGrpc

* betterer
2023-12-15 18:41:32 +01:00
ismail simsek
03da1a3f3f
InfluxDB: Update SQL language url specification (#79408)
* accept urls with or without port

* remove redundant comment

* remove redundant function
2023-12-13 16:01:01 +01:00
ismail simsek
c088d003f2
InfluxDB: Implement InfluxQL json streaming parser (#76934)
* Have the first iteration

* Prepare bench testing

* rename the test files

* Remove unnecessary test file

* Introduce influxqlStreamingParser feature flag

* Apply streaming parser feature flag

* Add new tests

* More tests

* return executedQueryString only in first frame

* add frame meta and config

* Update golden json files

* Support tags/labels

* more tests

* more tests

* Don't change original response_parser.go

* provide context

* create util package

* don't pass the row

* update converter with formatted frameName

* add executedQueryString info only to first frame

* update golden files

* rename

* update test file

* use pointer values

* update testdata

* update parsing

* update converter for null values

* prepare converter for table response

* clean up

* return timeField in fields

* handle no time column responses

* better nil field handling

* refactor the code

* add table tests

* fix config for table

* table response format

* fix value

* if there is no time column set name

* linting

* refactoring

* handle the status code

* add tracing

* Update pkg/tsdb/influxdb/influxql/converter/converter_test.go

Co-authored-by: İnanç Gümüş <m@inanc.io>

* fix import

* update test data

* sanity

* sanity

* linting

* simplicity

* return empty rsp

* rename to prevent confusion

* nullableJson field type for null values

* better handling null values

* remove duplicate test file

* fix healthcheck

* use util for pointer

* move bench test to root

* provide fake feature manager

* add more tests

* partial fix for null values in table response format

* handle partial null fields

* comments for easy testing

* move frameName allocation in readSeries

* one less append operation

* performance improvement by making string conversion once

pkg: github.com/grafana/grafana/pkg/tsdb/influxdb/influxql
             │ stream2.txt │            stream3.txt             │
             │   sec/op    │   sec/op     vs base               │
ParseJson-10   314.4m ± 1%   303.9m ± 1%  -3.34% (p=0.000 n=10)

             │ stream2.txt  │             stream3.txt              │
             │     B/op     │     B/op      vs base                │
ParseJson-10   425.2Mi ± 0%   382.7Mi ± 0%  -10.00% (p=0.000 n=10)

             │ stream2.txt │            stream3.txt             │
             │  allocs/op  │  allocs/op   vs base               │
ParseJson-10   7.224M ± 0%   6.689M ± 0%  -7.41% (p=0.000 n=10)

* add comment lines

---------

Co-authored-by: İnanç Gümüş <m@inanc.io>
2023-12-06 12:39:05 +01:00
ismail simsek
2b0e7afb4e
InfluxDB: Fix an edge case while parsing table response (#78802)
* Prepare the test files

* use json files everywhere

* update golden json files

* disable update

* update test file

* fix naming

* lint

* InfluxDB: Add metadata information to first frame only (#78664)

* executedString in first frame only

* lint

* fix tests

* update tests

* don't update

* linting

* update

* update again

* handle nil values

* append in the right array

* add comments

* remove redundant if condition
2023-12-01 18:31:49 +01:00
ismail simsek
dff5022021
Chore: Return executedString information even with frames has no time column (#78906)
* return executedString for no time column responses

* remove comment
2023-11-30 14:25:15 +01:00
ismail simsek
aea5a9f01a
Chore: Introduce util package for InfluxDB backend testing (#78826)
have a util package
2023-11-29 17:48:53 +01:00
ismail simsek
23c7211f1d
Chore: InlfuxDB testing update (#78663)
* Prepare the test files

* use json files everywhere

* update golden json files

* disable update

* update test file

* fix naming

* lint

* InfluxDB: Add metadata information to first frame only (#78664)

* executedString in first frame only

* lint

* fix tests

* update tests

* don't update

* linting

* update

* update again
2023-11-29 16:19:46 +01:00
ismail simsek
c5f3ce1371
InfluxDB: Parse data for table view to have parity with frontend parser (#78365)
* Use TimeSeriesWide format for table response

* fix group by query result parsing

* handle labels

* provide a test where result has no tags

* parsing results without time column

* clean the code

* remove the comment line

* more cleaning

* lint
2023-11-22 19:27:42 +01:00
ismail simsek
5096806eab
InfluxDB: Don't sort retention policies on the backend (#78252)
don't sort retention policies
2023-11-16 10:39:15 +02:00
Ben Tasker
f38f657f87
InfluxDB: Add new truthiness operators (Is and Is Not) to InfluxQL Query Builder (#77923)
* InfluxDB: Add new truthiness operators (`Is` and `Is Not`) to InfluxQL Editor for use with boolean fields

* InfluxDB: Make the front-end aware of the new operators so that translation between raw and builder works

* Chore: Add tests

* feat: identify type of field value to allow other types to work with Is/Is Not

Tags: always quote
Integer: Don't quote
Float: Don't quote
Boolean: Don't quote
String: Quote

* Chore: Add test-cases for type inference

* Update front-end to infer type for operators Is and Is Not

* chore: Add front-end tests

* chore: add additional front-end tests

* chore: fix failing lint test

* chore: fix tests (run prettier)
2023-11-15 21:10:37 +01:00
Kyle Brandt
a94acf4b63
Revert "InfluxDB: Response parser improvements (#76852)" (#78224)
This reverts commit 046791e2be.

Is causing malformed frames, and resulting in errors about mismatched field lengths in the logs, No data in the UI, and will cause SSE to panic.

I think this is because of the global slice vars since it seems to take concurrency to replicate it.
2023-11-15 14:01:36 -05:00
Ben Tasker
c06debe200
InfluxDB: Add support for >= and <= comparison operators to IQL Query Builder (#77917)
* InfluxDB: Add support for `>=` and `<=` comparison operators to InfluxQL Query Builder

* Add front-end support for the new operators

This ensures that the query translates correctly between raw and builder mode

* Chore: add test for new operators

* chore: add front-end tests

* fix: don't skip quoting on `<>`

This preserves the pre-existing behaviour, fixing a failing test

* chore: fix tests
2023-11-15 12:21:41 +01:00
ismail simsek
6b13064cf6
InfluxDB: Fix parsing multiple tags on backend mode (#77340)
* Multiple tags separated by comma in the result

* A non-flaky unit test
2023-10-30 15:45:20 +01:00
ismail simsek
4ed36cbc1d
InfluxDB: Fix parsing empty response (#77353)
Fix parsing empty response
2023-10-30 11:39:15 +01:00
ismail simsek
046791e2be
InfluxDB: Response parser improvements (#76852)
* remove retention policy lookup

* Back to one big function

* %10 less memory allocation

pkg: github.com/grafana/grafana/pkg/tsdb/influxdb/influxql
                │    1.txt    │                2.txt                │
                │   sec/op    │   sec/op     vs base                │
ParseBigJson-10   540.9m ± 3%   474.0m ± 2%  -12.37% (p=0.000 n=10)

                │    1.txt     │                2.txt                │
                │     B/op     │     B/op      vs base               │
ParseBigJson-10   580.6Mi ± 0%   573.2Mi ± 0%  -1.28% (p=0.000 n=10)

                │    1.txt     │                2.txt                │
                │  allocs/op   │  allocs/op   vs base                │
ParseBigJson-10   10.123M ± 0%   9.086M ± 0%  -10.25% (p=0.000 n=10)

* Slightly better results comparing with the previous commit
pkg: github.com/grafana/grafana/pkg/tsdb/influxdb/influxql
                │    2.txt    │               3.txt                │
                │   sec/op    │   sec/op     vs base               │
ParseBigJson-10   474.0m ± 1%   503.4m ± 3%  +6.21% (p=0.000 n=10)

                │    2.txt     │                3.txt                │
                │     B/op     │     B/op      vs base               │
ParseBigJson-10   573.2Mi ± 0%   564.0Mi ± 0%  -1.60% (p=0.000 n=10)

                │    2.txt    │               3.txt                │
                │  allocs/op  │  allocs/op   vs base               │
ParseBigJson-10   9.086M ± 0%   9.052M ± 0%  -0.37% (p=0.000 n=10)

* Split into smaller functions

* Unit test for parseTimestamp
2023-10-27 17:17:19 +02:00
ismail simsek
63abe25b74
InfluxDB: Fix table parsing with backend mode (#76899)
* Add resultFormat to query model

* don't add row name if the result format is table

* No need special formatting since we use unified dataframes

* betterer

* specify visualization type in response

* Unit tests

* fix unit tests
2023-10-23 19:21:30 +02:00
ismail simsek
5eb0b2bedb
InfluxDB: Fix aliasing with $measurement or $m on backend mode (#76917)
* better interpolation $measurement aliasing

* unit tests
2023-10-23 17:36:51 +02:00
Will Browne
9d92818cae
Plugins: Update PDC pattern from latest plugin SDK changes (#76576)
* update with sdk

* do sql

* fix core plugins

* fix proxy settings

* bump SDK version

* tidy

* enable pdc for test

* add codeowners

* bump dep

* go mod tidy

* bump SDK
2023-10-16 16:40:04 +02:00
Dan Cech
91cf4f0c1c
Revert "Plugins: Update PDC pattern from latest plugin SDK changes" (#76565)
Revert "Plugins: Update PDC pattern from latest plugin SDK changes (#76036)"

This reverts commit 151f6d6216.
2023-10-13 12:35:16 -04:00
ismail simsek
2877723648
InfluxDB: Use apache/arrow v13 (#76548)
* Use apache/arrow v13

* remove apache/thrift

* go mod tidy with go1.21.1

* add metrics team as owner

---------

Co-authored-by: Kyle Brandt <kyle@grafana.com>
2023-10-13 17:06:10 +02:00
Will Browne
151f6d6216
Plugins: Update PDC pattern from latest plugin SDK changes (#76036)
* update with sdk

* do sql

* fix core plugins

* fix proxy settings

* bump SDK version

* tidy

* enable pdc for test

* add codeowners
2023-10-13 14:42:15 +02:00