Commit Graph

665 Commits

Author SHA1 Message Date
Matias Chomicki
663ed7ba82
Loki Query Splitting: Rename from "chunk" to "splitting" (#65630)
* Update chunking mentions in the code and ui

* Rename files and exported functions

* Rename configuration attribute

* Rename grouped querying function name

* Update more function names

* Update unit test

* Update unit tests

* More renames

* Rename time splitting functions
2023-04-03 12:30:08 +00:00
Sven Grossmann
d790cb1c09
Loki: Add unpack query builder hint (#65608)
* add unpack hint

* use `hasOwnProperty`

* add unpack completion to monaco

* adjust test name

* fix tests

* fix tests
2023-03-30 15:18:02 +02:00
Matias Chomicki
fc16fb0407
Loki Query Chunking: Group queries by resolution (#65353)
* Loki chunking: group queries by resolution

* Update unit tests

* Add chunked + grouped test case
2023-03-30 10:29:19 +02:00
Sven Grossmann
b01194f81e
Loki: Fix autocomplete situations with multiple escaped quotes (#65520)
fix situations with multiple quotes
2023-03-29 15:00:42 +02:00
Torkel Ödegaard
f43ef18732
Revert "Explore: Show log line if there is an interpolated link" (#65315)
Revert "Explore: Show log line if there is an interpolated link (#62926)"

This reverts commit aa857e2a4f.
2023-03-27 19:19:42 +02:00
Matias Chomicki
9874a2d48b
Loki Logs sample: use the selected time range in sample request (#65126)
Logs sample: use the selected time range in sample request

There are parts of the query, such as label values, that could only be present during the current selected time interval. If we request using a time range other than the selected one, we are not providing builder suggestions based on the same data that the user is seeing.
2023-03-22 18:56:40 +02:00
Kristina
aa857e2a4f
Explore: Show log line if there is an interpolated link (#62926)
* bring in source from database

* bring in transformations from database

* add regex transformations to scopevar

* Consolidate types, add better example, cleanup

* Add var only if match

* Change ScopedVar to not require text, do not leak transformation-made variables between links

* Add mappings and start implementing logfmt

* Add mappings and start implementing logfmt

* Remove mappings, turn off global regex

* Add example yaml and omit transformations if empty

* Fix the yaml

* Add logfmt transformation

* Cleanup transformations and yaml

* add transformation field to FE types and use it, safeStringify logfmt values

* Add tests, only safe stringify if non-string, fix bug with safe stringify where it would return empty string with false value

* Add test for transformation field

* Do not add null transformations object

* Add provisioning (to be removed) and show log lines with links

* Only display links if change to query was made

* Break out transformation logic, add tests to backend code

* Fix lint errors I understand 😅

* Fix the backend lint error

* Remove unnecessary code and mark new Transformations object as internal

* Add support for named capture groups

* Remove type assertion

* Remove variable name from transformation

* Add test for overriding regexes

* Add back variable name field, but change to mapValue

* fix go api test

* Change transformation types to enum, add better provisioning checks for bad type name and format

* Change transformation types to enum, add better provisioning checks for bad type name and format

* Check for expression with regex transformations

* Remove isInterpolated variable, add option to always use format function

* Add template variable check to links

* Use new functions

* Filter log line at render, remove extra createSpanLink imports

* Add scrollable to long log messages

* Remove test that is no longer accurate

* Remove test correlation

* Add tests, fix duplicate key issue

* WIP: show log line links key/value pairs

* Some not great style changes

* Change LogDetailsRow for better multi value formatting

* Cleanup

* Add additional information around variable regex, implement PR feedback

* Display name with fieldPath if applicable

* Add variables with fieldPaths to test

* Count empty string as undefined variable

* Add better commented version of function, fix tests by removing new variable

* Modify when links show

* Remove sample yaml

* If a link has no variables, set value to field name, and some formatting issues

* Add comments and change variable names to be more clear, add back logic where needed, add test coverage for new scenario

* Fix formatting of replaceInVariableRegex comment

* Remove changes from Grafana-data, move logic into explore

* Rename function and property to match similar format

* Move types to type files and consolidate definitions, rename functions, change field definitions to accept arrays of keys/values, move function to parser, hide actions on multi key/value rows

* Add tests to logParser’s new function
2023-03-22 08:01:04 -05:00
Matias Chomicki
6093e45178
Loki Query Chunking: Refactor naming conventions and reorganize code (#65056)
* Rename splitting files to chunking

* Rename time chunking functions

* Move response functions to response utils

* Remove some blank spaces

* Add an extra test case for frame refIds and names
2023-03-21 15:54:18 +00:00
Torkel Ödegaard
395890c357
Prometheus/Loki: Run query explicitly instead of onblur in panel edit (#64815)
* Prometheus: Run query explicitly instead of onblur in panel edit, and highlight Run qqueries when query changed

* Update loki to do the same

* Cleanup unused prop

* Remove test
2023-03-21 07:55:34 +01:00
Sven Grossmann
68551ac9ca
Loki: Add lokiQueryChunkingOption feature flag (#65057)
* add `lokiQueryChunkingOption` feature flag

* be safe and also rely on `config.featureToggles.lokiQuerySplitting`

* renamed to `lokiQuerySplittingConfig`

* fix typo
2023-03-20 16:51:36 +01:00
Sven Grossmann
7a17a8f02d
QueryStats: Fix comparison of relative and absolute timeranges (#65035)
* add missing fix of relative timeranges

* add doc
2023-03-20 11:35:03 +01:00
Sven Grossmann
6ae373b8c7
QueryStats: Fix comparison of timeranges (#65025)
* fix timerange comparison

* remove import
2023-03-20 10:04:41 +01:00
Sven Grossmann
7261c6f7cd
DataSourceWithBackend: Add queryGroupId to find correlated/related queries (#64587)
* add `correlationId` to queries

* trace correlation id in backend

* add correlation id to loki's span

* add correlation id to query chunks

* fix test

* fix DataSourceWithBackend test

* change to `queryGroupId`

* remove empty line

* fix test in `DataSourceWithBackend`
2023-03-17 15:48:13 +01:00
Gareth Dawson
c16280a4e8
Loki: Improve the display of loki query stats (#63623)
* fix: refresh query stats on timerange change

* partial fix: request stats update on type in code mode

* complete fix: request stats update on type in code mode

* fix: update failing tests and props

* refactor: pass only essential query string to getQueryStats

* refactor: remove unused variables

* test: fix datasource.getTimeRange is not a function

* refactor: use lodash debounce instead of setTimeout

* refactor: add suggestions from code review

* test: shouldUpdateStats and makeStatsRequest

* refactor: use more appropriate variable names

* refactor: make setQueryStats required instead of optional

* refactor: move setQueryStats into LokiQueryEditor

* fix: add missing props to LokiQueryField

* revert changes

* refactor: use inversion of control to request stats

* refactor: remove unnecessary code
2023-03-16 18:50:42 +00:00
Sven Grossmann
40014f1454
Loki: Add option to define chunk duration per query (#64834)
* add query option to configure chunk ranges

* remove `isValidDuration` check

* Update public/app/plugins/datasource/loki/querybuilder/components/LokiQueryBuilderOptions.tsx

Co-authored-by: Matias Chomicki <matyax@gmail.com>

* change to `chunkDuration`
added tests

* no need to call `toString`

---------

Co-authored-by: Matias Chomicki <matyax@gmail.com>
2023-03-16 16:30:12 +01:00
Sven Grossmann
652e7334c8
Loki: Remove unnecessary type extension (#64883)
remove unnecessary type extension
2023-03-16 11:28:40 +01:00
Gábor Farkas
5d8ec2756e
loki: query chunking: better error handling (#64811)
* loki: query chunking: better error handling

* better comment text

Co-authored-by: Matias Chomicki <matyax@gmail.com>

* better comment text

Co-authored-by: Matias Chomicki <matyax@gmail.com>

* changed merge approach

* simplified code

* fixed test

* removed forgotten code

---------

Co-authored-by: Matias Chomicki <matyax@gmail.com>
2023-03-16 08:54:32 +01:00
Ivana Huckova
a996344e14
Loki: Add autocomplete updates for improved suggestions (#64744)
* Loki: Add autocomplete updates for improved suggestions

* Use trimEnd for trailing pipeline

* Update public/app/plugins/datasource/loki/components/monaco-query-field/monaco-completion-provider/completions.ts

Co-authored-by: Matias Chomicki <matyax@gmail.com>

* Fix unused imports

---------

Co-authored-by: Matias Chomicki <matyax@gmail.com>
2023-03-15 14:26:46 +01:00
Ashley Harrison
e0e6165f70
Chore: adjust more unit tests to work with react 18 (#64759)
adjust unit tests to work with react 18
2023-03-15 10:12:13 +00:00
Ashley Harrison
85e5326040
Chore: Adjust unit tests so they work with react 18 (#64698)
tweak unit tests so they work with react 18
2023-03-14 17:24:04 +00:00
Ryan McKinley
d710507bc5
Chore: Avoid explicit React.FC<Props> when possible (#64722) 2023-03-14 16:38:21 +02:00
Gareth Dawson
d72c39c587
Loki: Add descriptions to query builder operations (#64046)
* feat: add descriptions to query builder operations

* refactor: add suggestion from review

* fix: update width of label operator seelct
2023-03-14 12:14:32 +00:00
Ashley Harrison
53186c14a4
Chore: Improve some types (#64675)
* some type fixes

* few more

* more type fixes

* fix the majority of (window as any) calls

* don't make new variable for event

* few more

* MOAR
2023-03-14 09:51:44 +00:00
Matias Chomicki
5ffb28989e
Range splitting: remove hardcoded chunk limit (#64625)
* Range splitting: allow splits larger than 30 days while keeping the fail safe

* Remove limit

* Fix test

* Remove unnecessary try/catch

* Remove unused export
2023-03-13 05:33:01 -04:00
Sven Grossmann
b46771cbfe
Loki: Always fetch for new label keys in the QueryBuilder (#64597)
* always fetch new labels

* remove refreshLogLabels
2023-03-10 13:13:51 +01:00
Sven Grossmann
976568a337
Loki: Fix labels in LabelBrowser being wrongly cached (#64482)
* remove label cache from label browser

* fix spelling

Co-authored-by: Gareth Dawson <gareth.dawson@grafana.com>

---------

Co-authored-by: Gareth Dawson <gareth.dawson@grafana.com>
2023-03-09 11:36:07 +01:00
Matias Chomicki
ede3e9e5c4
Range Splitting: Process instant queries as an independent query group (#64049)
* Query splitting: enable instant queries

* Range splitting: send instant queries as another request group

* Range splitting: increase grouped splitted requests stability

We were defaulting to the `0` index as the first group for the next request batch, but there was no guarantee that the group `0` had a `.partition` entry for `requestN-1`. Now we find the first defined and use that index as the next starting group.

* Range splitting: update unit test
2023-03-07 07:44:13 -05:00
Matias Chomicki
69d3ae8ec1
Range Splitting: ignore empty queries from splitting and fix result resetting (#64053)
Range splitting: ignore empty queries from splitting
2023-03-07 13:09:55 +01:00
Matias Chomicki
accef84ca5
Range splitting: Call subscriber.next only when there are new results to report (#64171) 2023-03-07 13:05:40 +01:00
Sven Grossmann
5db0d14606
LokiContextUi: Add loading indicator (#64167)
add loading indicator to LokiContextUi
2023-03-06 17:25:04 +01:00
Sven Grossmann
2076282064
Logs: Log samples not being ordered correctly (#64097)
* fix log samples being unsorted and multiple results when chunking

* use `SortDirection` enum

* changed to `sortDataFrame` to support other datasources than loki

* update tests

* change capitalization
2023-03-03 16:02:14 +01:00
Gábor Farkas
192308aac7
loki: query splitting: better stats (#64105) 2023-03-03 04:58:28 -05:00
李国忠
9521b0d2d2
Explore: Fix vector(5) syntax error in loki log explore (#63994)
* [fix] loki log explore : fix vector(5) err

* changelog

* changelog fmt

* rollback change log

* fix test and pretty fmt

* Update package.json

update lezer-logql depedency to 0.1.2

* Update package.json

fix conflict

➤ YN0000: │  "@grafana/monaco-logql@npm:^0.0.6":
➤ YN0000: │ @@ -21988,9 +21987,9 @@
➤ YN0000: │      "@grafana/experimental": 1.1.0
➤ YN0000: │      "@grafana/faro-core": 1.0.0-beta2
➤ YN0000: │      "@grafana/faro-web-sdk": 1.0.0-beta2
➤ YN0000: │      "@grafana/google-sdk": 0.0.4
➤ YN0028: │ -    "@grafana/lezer-logql": 0.1.1
➤ YN0028: │ +    "@grafana/lezer-logql": 0.1.2
➤ YN0000: │      "@grafana/monaco-logql": ^0.0.6
➤ YN0000: │      "@grafana/runtime": "workspace:*"
➤ YN0000: │      "@grafana/scenes": ^0.0.14
➤ YN0000: │      "@grafana/schema": "workspace:*"

* Upgrade lezer-logql

* fix logql test

---------

Co-authored-by: Matias Chomicki <matyax@gmail.com>
2023-03-03 10:09:34 +01:00
Gábor Farkas
fdc4973b77
loki: query chunking: consider refId when merging frames (#64103) 2023-03-03 10:06:25 +01:00
Matias Chomicki
40ac0fa14b
Query Splitting: Add support for multiple queries (#63663)
* Range splitting: group metric and logs queries

* Range splitting: intercalate logs and metric queries when > 1 queries

* Range splitting: fix possibly undefined error and remove console log

* Range splitting: update test imports

* Range splitting: add unit tests for multiple queries

* Query splitting: use lodash partition function

* Chore: rename variable

* Chore: attempt to improve readability
2023-03-02 10:32:45 +01:00
Matias Chomicki
d56896cf72
Range splitting: Omit instant queries from splitting (#63705)
Range splitting: omit instant queries from splitting
2023-02-27 11:09:46 +01:00
Ivan Ortega Alba
14e29451cf
Loki ds: Interpolation breaks in Loki queries when no scoped vars (#63694) 2023-02-24 00:18:08 -08:00
Ashley Harrison
4a2695349d
Chore: more ts test fixes (#63438)
* replace anys in tests

* more test fixes

* clean up any's in tests
2023-02-23 10:07:44 +00:00
Ivana Huckova
a1c0feab7d
Loki: Remove comment (#63565)
Remove comment
2023-02-22 14:57:18 +01:00
Gareth Dawson
c5ed9391b6
Loki: Display error with label filter conflicts (#63109)
* feat: add logic to check for conflicting operator

* feat: display error if there is a conflict

* feat: check if label filter conflicts with other filters

* fix: remove capitalisation from "no data"

* test: add tests for isConflictingFilter function

* feat(labels): add validation for label matchers

* test(labels): add tests for isConflictingSelector

* refactor(labels): add suggestions from code review

* test(labels): add case for labels without op

* refactor(operations): add suggestions from code review

* feat(operations): display tooltip when you have conflicts

* fix: remove unused props from test

* fix: remove old test

* fix(labels): error message now displays in the correct location

* fix(operations): operations can be dragged, even with error

* fix: remove unused vars

* fix: failing tests

* fix(operations): hide error message whilst dragging

* refactor: use more appropriate variable naming

* refactor: add suggestions from code review

* perf: use array.some instead of array.find
2023-02-22 09:56:20 +00:00
Gábor Farkas
b069fd81b2
loki: query splitting: more robust check (#63542) 2023-02-22 09:16:05 +01:00
Gábor Farkas
8f7b980c95
Loki: query splitting: handle stats (#63436)
* loki: query splitting: handle stats

* update tests
2023-02-20 13:55:56 +00:00
Gábor Farkas
7a8f1ab566
loki: query splitting: stricter types (#63387) 2023-02-17 19:43:08 +01:00
Matias Chomicki
a0bea04a02
Range splitting: Read errors from the received response and report them (#63368) 2023-02-16 12:39:26 -05:00
Matias Chomicki
0240f4eb45
Loki Range Splitting: Create a new instance for the initial frame (#63361)
* Range splitting: create a new instance of the initial frame

* Chore: rename variable
2023-02-16 12:55:31 +01:00
Gábor Farkas
0ee9d11a91
loki: query splitting: better canceling (#63315)
loki: query splitting: better cancelling
2023-02-13 17:52:30 +01:00
Matias Chomicki
84fdb7f908
Loki Range Splitting: Calculate dynamic maxLines per target based on the current response state (#63248)
* Range splitting: update maxLines for logs queries

* Range splitting: add unit tests for dynamic limit requesting
2023-02-13 07:59:20 +01:00
Matias Chomicki
8cd3d1bfda
Loki Range splitting: Ignore hidden queries (#63250)
Range splitting: ignore hidden queries
2023-02-10 18:19:17 +01:00
Sven Grossmann
3f144924d1
Loki: Do not chunk when refid contains do-not-chunk (#63276)
add ref-id to not chunk requests
2023-02-10 18:00:49 +01:00
Sven Grossmann
0bd326d846
LogsVolume: Make log volume work with chunking (#63181)
* aggregate logs volume on streaming

* enable log volume queries to be split

* add streaming indicator

* added tests

* also chunk logs queries

* change extraInfo back to undefined
2023-02-10 14:43:41 +01:00