* Logs: Use result range instead of timepicker range for log histogram
If a logs datasource does not send histogram data for the requested
time range, the logs model computes a timeseries based on the log row
counts, bucketed by an automcatically calculated time interval. Even
when this histogram time series did not span the whole requested time
range it was still rendered in the graph across the whole range, leaving
an empty area at its start. Users find this confusing and are lead to
believe their log data is missing.
This change fixes this by anchoring the start of the timeseries on the
first log row's timestamp from the result, and adds this smaller range
as `visibleRange` to the logs model and passes it through to the logs
component that optionally takes it into account to not render the empty
area.
The interval (bucket size) is also adjusted to account for a potentially
finer resolution on the shorter visible time interval.
The bucketsize multiplier was also changed from 10 to 20 to account for
the space between the chart's bars.
* Aligned visible range with buckets
* Extract bucket size calculation and add test
- log stats for Loki are per query
- this change tracks the query stats by refId, preventing the summing of
the same stats across multiple series of the same response.
* Field: Change getFieldTitle to getFieldDisplayNamne and change the NAME of the title field config from Title to Display name
* Review feedback
* fixed unit tests
* Rename fieldConfig.title to displayName
* Fixed tests
* Added migration
* Renamed getFrameDisplayTitle to getFrameDisplayName
* Chore: adds timeEpochNs to LogRowModel in @grafana/data
* Chore: updates explore utils ResultProcessor getLogsResult and explore utils tests
* Chore: updates core/logs_model to include nanoseconds
* Chore: updates LogRowModel sorting key from milliseconds to nanoseconds and adds timeEpochNs to tests
* Chore: adds timeEpochNs to LogRowModel mock in Explore LiveLogs test
* Chore: fixes logs model timeEpochNs padding
* Chore: updates timeEpochNs padding in tests
* Chore: updates LogRowModel mocks
* Chore: changes isLoki to datasourceId
* Chore: adds hasFieldWithNameAndType method to FieldCache in grafana-data dataframe
* Chore: changes timeEpochNs from number to string as it can overflow Number.MAX_SAFE_INTEGER
* Chore: updates LogRowModel sorting to use milliseconds and nanoseconds
* Chore: removes datasourceId from logSeriesToLogsModel method
* Chore: updates ResultProcessor tests to include nanosecond-level precision log rows sorting
* Loki: Show loki datasource stats in panel inspector
- puts the loki query result stats into the query results meta stat API
of Grafana, this allows the display of all backend loki stats in the
panel inspector in the dashboards
- added a hack to also display one of those values in Explore as a meta
label using the dataframe meta `custom` mechanims to point to a single
stat entry for each series which is then added together to show total
bytes processed across all query row results (this should be changed for
7.1 to make full use of the panel inspector in Explore)
* Fix test
* nicer stats labels for loki stats with units
* added moment timezone package.
* added a qnd way of selecting timezone.
* added a first draft to display how it can be used.
* fixed failing tests.
* made moment.local to be in utc when running tests.
* added tests to verify that the timeZone support works as expected.
* Fixed so we use the formatter in the graph context menu.
* changed so we will format d3 according to timeZone.
* changed from class base to function based for easier consumption.
* fixed so tests got green.
* renamed to make it shorter.
* fixed formatting in logRow.
* removed unused value.
* added time formatter to flot.
* fixed failing tests.
* changed so history will use the formatting with support for timezone.
* added todo.
* added so we append the correct abbrivation behind time.
* added time zone abbrevation in timepicker.
* adding timezone in rangeutil tool.
* will use timezone when formatting range.
* changed so we use new functions to format date so timezone is respected.
* wip - dashboard settings.
* changed so the time picker settings is in react.
* added force update.
* wip to get the react graph to work.
* fixed formatting and parsing on the timepicker.
* updated snap to be correct.
* fixed so we format values properly in time picker.
* make sure we pass timezone on all the proper places.
* fixed so we use correct timeZone in explore.
* fixed failing tests.
* fixed so we always parse from local to selected timezone.
* removed unused variable.
* reverted back.
* trying to fix issue with directive.
* fixed issue.
* fixed strict null errors.
* fixed so we still can select default.
* make sure we reads the time zone from getTimezone
* Datasource/Cloudwatch: Adds support for Cloudwatch Logs
* Fix rebase leftover
* Use jsurl for AWS url serialization
* WIP: Temporary workaround for CLIQ metrics
* Only allow up to 20 log groups to be selected
* WIP additional changes
* More changes based on feedback
* More changes based on PR feedback
* Fix strict null errors
* Move xss and sanitize packages to grafana-data
* Move text, url and location utils to grafana-data
* Move grafana config types to grafana-data
* Move field display value proxy to grafana-data
* Fix
* Move data links built in vars to grafana-data
* Attach links supplier to when applying field overrides
* Prep tests
* Use links suppliers attached via field overrides
* locationUtil dependencies type
* Move sanitize-url declaration to grafana-data
* Revert "Move sanitize-url declaration to grafana-data"
This reverts commit 11db9f5e55.
* Fix typo
* fix ts vol1
* Remove import from runtime in data.... Make TS happy at the same time ;)
* Lovely TS, please shut up
* Lovely TS, please shut up vol2
* fix tests
* Fixes
* minor refactor
* Attach get links to FieldDisplayValue for seamless usage
* Update packages/grafana-data/src/field/fieldOverrides.ts
* Make storybook build
* Explore: updates grafana-data explore query field props with explore mode
* Explore: updates query row to pass down explore mode to query fields
* Explore: adds LokiExploreQueryEditor
* Explore: updates loki query field form to render children
* Explore: adds loki explore extra field component
* Explore: adds extra field element to loki query field form
* Explore: updates loki explore query editor to use extra field element
* Explore: moves ExploreMode to grafana-data
* Explore: updates query row limit string
* Explore: adds maxLines to DataQuery
* Explore: adds maxLines to loki datasource runRangeQueryWithFallback
* Explore: adds onChangeQueryLimit to LokiExploreQueryEditor
* Explore: updates loki explore query editor to render extra field only in logs mode
* Explore: fixes query limits for live and legacy queries
* Explore: fixes result processor max lines limit in get logs result
* Explore: fixes Loki datasource limit test
* Explore: removes unnecessary ExploreMode from Loki language provider
* Explore: fixes formatting
* Explore: updates grafana-data datasource types - replaces strings with explore mode enum
* Explore: updates loki explore query field props to take ReactNode
* Explore: updates the way we calculate loki query lines limit to fall back to 0 lines on negative or invalid input instead of datasource maxLines
* Explore: updates result processor get logs result method to avoid counting invalid/negative line limits
* Explore: updates loki result transformer to process only an appropriate slice of a result instead of an entire one
* Explore: adds a method for query limit preprocessing/mapping
* Explore: updates loki datasource run range query with fallback method to use options.maxDataPoints in dashboards
* Explore: removes unnecessary maxlineslimt from getLogsResult in resultProcessor
* Explore: moves line limit to metadata
* Explore: adds an ability to specify input type of extra field
* Explore: updates LokiExploreQueryEditor - adds an input type
* Explore: updates LokiExploreQueryEditor to run queries when maxLines is positive
* Explore: fixes failing import of ExploreMode
* Explore: fixes reducers test imports formatting
* Explore: updates Loki extra field with min value set to 0
* Explore: exports LokiExploreExtraFieldProps
* Explore: adds render test of LokiExploreQueryEditor
* Explore: adds LokiExploreQueryEditor snapshot
* Explore: updates LokiExploreQueryEditor onChangeQueryLimit method to prevent it from running when query input is empty - fixes cheatsheet display issue
* Explore: updates Loki editor snapshots
* Explore: fixes typo in test set name in LokiExploreQueryEditor
* Explore: adds a render test of LokiExploreExtraField
* Explore: fixes typo in LokiExploreQueryEditor
* Explore: updates LokiExploreQueryEditor snapshot due to timezone issues
* Explore: updates LokiExploreExtraField to export both functional component and a version using memo
* Explore: updates LokiExploreQueryEditor to export both functional component and memoized function
* Explore: updates LokiExploreQueryEditor - removes unnecessary react fragment
* Explore: updates LokiExploreQueryEditor snapshot
* Explore: adds LokiExploreQueryEditor tests for different explore mode cases
* Explore: fixes Loki datasource and result transformer
* Explore: updates LokiExploreQueryEditor snapshot
* Explore: updates LokiExploreQueryEditor tests and test setup
* Explore: updates LokiExploreQueryEditor - refactors component
* Explore: updates LokiExploreQueryEditor to use default import from LokiExploreExtraField
* Explore: updates LokiExploreQueryEditor snapshot
* Explore: fixes formatting
* Explore: updates LokiExploreQueryEditor max lines change
* Explore: updates LokiExploreQueryEditor tests checking ExtraFieldElement
* Explore: adds mock loki datasource to LokiExploreQueryEditor
* Explore: updates LokiExploreQueryEditor test mock - adds language provider
* Explore: updates LokiExploreQueryEditor snapshot
* Explore: updates Loki ResultTransformer to filter out rows on limit - logic to be moved into a component with new form styles
* Explore: updates LokiExploreQueryEditor tests
Adds a config section with derived fields which is a config that allows you to create a new field based on a regex matcher run on a log message create DataLink to it which is the clickable in the log detail.
* POC: trying to see if there is a way to support objects in template interpolations
* Added support for nested objects, and arrays
* Added accessor cache
* fixed unit tests
* First take
* Use links supplier in graph
* Add field's index to cache items
* Get field index from field cache
* CHange FiledCacheItem to FieldWithIndex
* Add refId to TimeSeries class
* Make field link supplier work with _series, _field and _value vars
* use field link supplier in graph
* Fix yaxis settings
* Update dashboard schema version and add migration for data links variables
* Update snapshots
* Update build in data link variables
* FieldCache - idx -> index
* Add current query results to panel editor
* WIP Updated data links dropdown to display new variables
* Fix build
* Update variables syntac in field display, update migration
* Field links supplier: review updates
* Add data frame view and field name to TimeSeries for later inspection
* Retrieve data frame from TimeSeries when clicking on plot graph
* Use data frame's index instead of view
* Retrieve data frame by index instead of view on TimeSeries
* Update data links prism regex
* Fix typecheck
* Add value variables to suggestions list
* UI update
* Rename field to config in DisplayProcessorOptions
* Proces single value of a field instead of entire data frame
* Updated font size from 10px to 12px for auto complete
* Replace fieldName with fieldIndex in TimeSeries
* Don't use .entries() for iterating in field cache
* Don't use FieldCache when retrieving field for datalinks in graph
* Add value calculation variable to data links (#19031)
* Add support for labels with dots in the name (#19033)
* Docs update
* Use field name instead of removed series.fieldName
* Add test dashboard
* Typos fix
* Make visualization tab subscribe to query results
* Added tags to dashboard so it shows up in lists
* minor docs fix
* Update singlestat-ish variables suggestions to contain series variables
* Decrease suggestions update debounce
* Enable whitespace characters(new line, space) in links and strip them when processing the data link
* minor data links UI update
* DataLinks: Add __from and __to variables suggestions to data links (#19093)
* Add from and to variables suggestions to data links
* Update docs
* UI update and added info text
* Change ESC global bind to bind (doesn't capture ESC on input)
* Close datalinks suggestions on ESC
* Remove unnecessary fragment
* WIP: Use data frames in explore
* Explore: everything seems to be working again
* Reworked ResultProcessor tests
* Fixed unit test
* Add some typings and comments
* add appending utility
* add appending utility
* update comment
* rename to mutable
* move mutable functions out of DataFrameHelper
* move mutable functions out of DataFrameHelper
* move mutable functions out of DataFrameHelper
* turn DataFrameHelper into FieldCache
* guess time from name
* graph the numbers
* return the timeField, not just the index
* just warn on duplicate field names
* only use a parser if the input is a string
* append init all fields to the same length
* typo
* only parse string if value is a string
* DataFrame: test fixes
* Switch to null for missing values
* Fixed tests
* Wip: Compiles and runs
* WIP: Logs Graph partially working
* Refactor: Adds GraphSeriesToggler
* Refactor: Adds tickDecimals to YAxis
* Refactor: Adds TimeZone and PlotSelection to Graph
* Refactor: Makes the graphResult work in Explore
* Refactor: Adds ExploreGraphPanel that is used by Logs and Explore
* Fix: Fixes strange behaviour with ExploreMode not beeing changed
* Fix: Adds onSelectionChanged to GraphWithLegend
* Refactor: Cleans up unused comments
* ExploreGraph: Disable colorpicker
* Refactor: Removes switches for TimeStamp and LocalTime and displays time as configured for user
* Styles: Updates classname used to display timeUtc
* Refactor: Adds switch for Time column
If a field or a label named level is returned from datasource that is
used as log level for the logs result instead of parsing the log level
from the message.
Closes#17122
* Move log's typings into grafana/ui
* Update the way context is retrieved for Loki
Major changes:
1. getLogRowContext expects row to be of LogRowModel type
2. getLogRowContext accepts generic options object, specific to a datasource of interest. limit option has been removed, and now it's a part of Loki's context query options (see below)
3. LogRowContextProvider performs two queries now. Before, it was Loki ds that performed queries in both directions when getLogRowContext.
4. Loki's getLogRowContext accepts options object of a type:
interface LokiContextQueryOptions {
direction?: 'BACKWARD' | 'FORWARD';
limit?: number;
}
This will enable querying in either direction independently and also slightly simplifies the way query errors are handled.
LogRowContextProvider maps the results to a Loki specific context types, basically string[][], as raw log lines are displayed in first version.
* (feat/explore): Support for new LogQL filtering syntax
Loki is adding syntax to support chained filtering.
This PR adapts Grafana to support this.
- Send only `query` parameter in loki request
- Automatically wrap search text in simple syntax, e.g., `{} foo` is
sent as `{} |~ "foo"`.
* Adapted to regexp parameter staying on in Loki
* Dont wrap single regexp in new filter syntax
* Fix datasource test
* Fallback regexp parameter for legacy queries
* Fix search highlighting
* Make highlighting work for filter chains
* Fix datasource test
This is the first step moving towards Explore supporting logs for
more datasources than Loki. In the first step we move all the log
processing from Loki into Explore.
- Make explore convert logs result returned from datasources to SeriesData,
if needed, and for now convert SeriesData to LogsModel.
- Loki datasource query now returns SeriesData and all
processing have been moved into explore instead.
- Removed key from LogRowModel and use log row indexes as
the unique key instead.
- Removed id from LogsModel since it looks like it's not in use.
- Introduced a new class FieldCache which is convenient to use when
looking up multiple fields and/or field types and series values.
Closes#16287
The log rows were mutated in the store, what resulted in wrong deduplication results as well as made the selector memoization quirky - for "the same results" the selector was running twice.
