Commit Graph

26 Commits

Author SHA1 Message Date
Matias Chomicki
6f3dfb2d8b
Loki Query Splitting: Ignore empty queries like hidden queries (#72376)
Loki query splitting: ignore empty queries like hidden queries
2023-07-26 15:18:55 +01:00
Ivana Huckova
82c125d450
Loki: Implement step editor (#69648)
* Loki: Implement step editor

* Update to keep value

* Remove console.log

* Remove white space

* Update public/app/plugins/datasource/loki/querybuilder/components/LokiQueryBuilderOptions.tsx

Co-authored-by: Matias Chomicki <matyax@gmail.com>

* Import trim

* Update using of step in split queries

* Add tests

* Add tests

* Remove step interpolation

---------

Co-authored-by: Matias Chomicki <matyax@gmail.com>
2023-06-16 19:08:29 +03:00
Sven Grossmann
15d55c371c
Loki: Add feature tracking to reduce repetition (#69900)
add tracking to predefined queries
2023-06-12 13:03:47 +02:00
Ivana Huckova
38fc22538a
Loki: Don't split queries if they use $__range variables (#69852)
* Loki: Don't split queries if they use  variables

* Update public/app/plugins/datasource/loki/querySplitting.ts

Co-authored-by: Matias Chomicki <matyax@gmail.com>

---------

Co-authored-by: Matias Chomicki <matyax@gmail.com>
2023-06-12 11:22:04 +02:00
Matias Chomicki
e31ea1da9d
Query Splitting: Display progress of sub-requests using an annotation frame (#69574)
* Loki Query Splitting: Add and update loading frame to merged response

* Update unit test

* Loki Query Splitting: use data frame constructor function
2023-06-07 13:52:06 +00:00
Matias Chomicki
ee0ee70aa1
Loki: Exclude queries using DISTINCT from query splitting (#69377)
* Query utils: add function to identify distinct queries

* Chore: return false when the expected node is found

* Query splitting: exclude distinct queries from splitting

* Group queries excluded from splitting
2023-06-02 12:50:23 +02:00
Matias Chomicki
2bfd415c07
Loki Query Splitting: Enable tracking for split queries (#68645)
* Loki datasource: move tracking out of runQuery

* Types: move LokiGroupedRequest to types file

* Tracking: add function to track grouped queries

* Query splitting: add tracking to split queries runner

* Remove unnecessary types

* Add unit test

* Tracking: include test case with hidden query

* Update public/app/plugins/datasource/loki/tracking.ts

Co-authored-by: Ivana Huckova <30407135+ivanahuckova@users.noreply.github.com>

* Update public/app/plugins/datasource/loki/tracking.ts

Co-authored-by: Ivana Huckova <30407135+ivanahuckova@users.noreply.github.com>

* Update public/app/plugins/datasource/loki/tracking.ts

Co-authored-by: Ivana Huckova <30407135+ivanahuckova@users.noreply.github.com>

* Update public/app/plugins/datasource/loki/tracking.ts

Co-authored-by: Ivana Huckova <30407135+ivanahuckova@users.noreply.github.com>

* Tracking: add is_split dimension

---------

Co-authored-by: Ivana Huckova <30407135+ivanahuckova@users.noreply.github.com>
2023-05-25 18:39:16 +03:00
Domas
5dc529a55d
Query Splitting: Fix for handling queries with no requestId (#66161)
Loki query splitting fix for when there's no request id
2023-04-07 13:31:18 +03:00
Matias Chomicki
44beef2e41
Loki Query Splitting: Fix bug for mixed split durations (#65925)
* Query splitting: fix next request group pointer calculation

* Update unit test
2023-04-04 15:51:41 +02:00
Matias Chomicki
663ed7ba82
Loki Query Splitting: Rename from "chunk" to "splitting" (#65630)
* Update chunking mentions in the code and ui

* Rename files and exported functions

* Rename configuration attribute

* Rename grouped querying function name

* Update more function names

* Update unit test

* Update unit tests

* More renames

* Rename time splitting functions
2023-04-03 12:30:08 +00:00
Matias Chomicki
6093e45178
Loki Query Chunking: Refactor naming conventions and reorganize code (#65056)
* Rename splitting files to chunking

* Rename time chunking functions

* Move response functions to response utils

* Remove some blank spaces

* Add an extra test case for frame refIds and names
2023-03-21 15:54:18 +00:00
Sven Grossmann
7261c6f7cd
DataSourceWithBackend: Add queryGroupId to find correlated/related queries (#64587)
* add `correlationId` to queries

* trace correlation id in backend

* add correlation id to loki's span

* add correlation id to query chunks

* fix test

* fix DataSourceWithBackend test

* change to `queryGroupId`

* remove empty line

* fix test in `DataSourceWithBackend`
2023-03-17 15:48:13 +01:00
Sven Grossmann
40014f1454
Loki: Add option to define chunk duration per query (#64834)
* add query option to configure chunk ranges

* remove `isValidDuration` check

* Update public/app/plugins/datasource/loki/querybuilder/components/LokiQueryBuilderOptions.tsx

Co-authored-by: Matias Chomicki <matyax@gmail.com>

* change to `chunkDuration`
added tests

* no need to call `toString`

---------

Co-authored-by: Matias Chomicki <matyax@gmail.com>
2023-03-16 16:30:12 +01:00
Gábor Farkas
5d8ec2756e
loki: query chunking: better error handling (#64811)
* loki: query chunking: better error handling

* better comment text

Co-authored-by: Matias Chomicki <matyax@gmail.com>

* better comment text

Co-authored-by: Matias Chomicki <matyax@gmail.com>

* changed merge approach

* simplified code

* fixed test

* removed forgotten code

---------

Co-authored-by: Matias Chomicki <matyax@gmail.com>
2023-03-16 08:54:32 +01:00
Ashley Harrison
53186c14a4
Chore: Improve some types (#64675)
* some type fixes

* few more

* more type fixes

* fix the majority of (window as any) calls

* don't make new variable for event

* few more

* MOAR
2023-03-14 09:51:44 +00:00
Matias Chomicki
5ffb28989e
Range splitting: remove hardcoded chunk limit (#64625)
* Range splitting: allow splits larger than 30 days while keeping the fail safe

* Remove limit

* Fix test

* Remove unnecessary try/catch

* Remove unused export
2023-03-13 05:33:01 -04:00
Matias Chomicki
ede3e9e5c4
Range Splitting: Process instant queries as an independent query group (#64049)
* Query splitting: enable instant queries

* Range splitting: send instant queries as another request group

* Range splitting: increase grouped splitted requests stability

We were defaulting to the `0` index as the first group for the next request batch, but there was no guarantee that the group `0` had a `.partition` entry for `requestN-1`. Now we find the first defined and use that index as the next starting group.

* Range splitting: update unit test
2023-03-07 07:44:13 -05:00
Matias Chomicki
accef84ca5
Range splitting: Call subscriber.next only when there are new results to report (#64171) 2023-03-07 13:05:40 +01:00
Matias Chomicki
40ac0fa14b
Query Splitting: Add support for multiple queries (#63663)
* Range splitting: group metric and logs queries

* Range splitting: intercalate logs and metric queries when > 1 queries

* Range splitting: fix possibly undefined error and remove console log

* Range splitting: update test imports

* Range splitting: add unit tests for multiple queries

* Query splitting: use lodash partition function

* Chore: rename variable

* Chore: attempt to improve readability
2023-03-02 10:32:45 +01:00
Matias Chomicki
a0bea04a02
Range splitting: Read errors from the received response and report them (#63368) 2023-02-16 12:39:26 -05:00
Matias Chomicki
0240f4eb45
Loki Range Splitting: Create a new instance for the initial frame (#63361)
* Range splitting: create a new instance of the initial frame

* Chore: rename variable
2023-02-16 12:55:31 +01:00
Gábor Farkas
0ee9d11a91
loki: query splitting: better canceling (#63315)
loki: query splitting: better cancelling
2023-02-13 17:52:30 +01:00
Matias Chomicki
84fdb7f908
Loki Range Splitting: Calculate dynamic maxLines per target based on the current response state (#63248)
* Range splitting: update maxLines for logs queries

* Range splitting: add unit tests for dynamic limit requesting
2023-02-13 07:59:20 +01:00
Matias Chomicki
8cd3d1bfda
Loki Range splitting: Ignore hidden queries (#63250)
Range splitting: ignore hidden queries
2023-02-10 18:19:17 +01:00
Gábor Farkas
27d70819cc
loki: query splitting: split logs queries (#63091)
* loki: calculate logs-chunk-boundaries

* renamed function
2023-02-10 12:01:33 +01:00
Matias Chomicki
94241f6676
Loki Query Splitting: Split queries into sub-queries with smaller time interval (#62767)
* Range splitting: range splitting function

* Range splitting: experiment with 1 hour splits

* Range splitting: reorganize code

* Range splitting: improve code readability and meaning

* Range splitting: add partition limit to prevent infinite loops

* Range splitting: add error handling

* Range splitting: disable for logs queries

* Range splitting: support any arbitrary time splitting + respect original from/to in the partition

* Chore: remove console logs

* Chore: delete unused import

* Range splitting: actually send requests in sequence

* Range splitting: do not split when > 1 query

* Range splitting: combine frames

* Chore: rename function

* split in reverse

* polished reversing

* keep reference to the right frame in the result

* Range splitting: change request state to Streaming

* Range splitting: fix moving only 1 unit of time instead of the provided one

* Chore: change default parameter to timeShift = 1

* Range splitting: do not split for range queqries

* Range splitting: add initial support for log queries

* Range splitting: do not use MutableDataFrame

It has bad performance and it's not required

* Chore: remove unused export

* Query Splitting: move to module

* loki: split: fix off-by-one error (#62966)

loki: split: fix off-by-one loop

* Range splitting: disable for logs volume queries

* Range splitting: combine any number of fields, not just hardcoded 2

* Range splitting: optimize frame-combining function

* Range splitting: further optimize

* Range splitting: combine frame length

* Range splitting: combine stats

* Range splitting: combine stats without assuming the same order

* Query splitting: catch and raise errors

* Range splitting: create feature flag

* Range splitting: implement feature flag

* Range splitting: add unit test for datasource query

* Range splitting: add basic test for runPartitionedQuery

* Range splitting: add unit test for resultLimitReached

* Range splitting: test frame merging

* Chore: fix unit test

---------

Co-authored-by: Sven Grossmann <svennergr@gmail.com>
Co-authored-by: Gábor Farkas <gabor.farkas@gmail.com>
2023-02-09 12:27:02 -05:00