Adds a new setting dataproxy.row_limit that allows an operator to limit the
amount of rows being processed/accepted in response to database queries
originating from SQL data sources.
Closes#38975
Ref #39095
Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>
* Revert "Prometheus: add functionality to specify desired step interval in dashboards panels (#36422)"
This reverts commit ddf5b65c51.
Co-authored-by: Ivana Huckova <ivana.huckova@gmail.com>
* Revert "Explore: add functionality for supporting different step modes in prometheus (#37829)"
This reverts commit f433cfd8d9.
Co-authored-by: Ivana Huckova <ivana.huckova@gmail.com>
* Revert stepMode BE implementation from #36796
Co-authored-by: "Ivana Huckova" <ivana.huckova@gmail.com>
Fix region annotations not displayed correctly when returning timeend column
as epoch timestamp and by that making sure that the returned data frame field
named timeend is treated as time type.
Fixes#38533
* convert SQLs to use sdk contracts
* make draft
* postgres
* intermedia
* get datasourceinfo filled at the beginning of the service
* move the interval into package because of cyclict import and fix all postgres tests
* fix mysql test
* fix mssql
* fix the test for pr https://github.com/grafana/grafana/issues/35839
* fix some issue about intervalv2 package
* update sql test
* wire migration for SQLs
* add sqls to the background process
* make it register instead of register and start
* revert formatting
* fix tests
* fix linter
* remove integration test
* Postgres test fix
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
* influxdb: influxql: alert: use already-computed interval-value when available
* reverted by-mistake changed import
* simplify code
now we can rely on DataQuery.Interval always being good
* removed unnecessary test
* fix lint-error
* now to get it to build
* modified names, added the 455 day case, and added a comment
* removed wireguard override i used for my local
* updated the docs
Fixes#30144
Co-authored-by: dsotirakis <sotirakis.dim@gmail.com>
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
Co-authored-by: Ida Furjesova <ida.furjesova@grafana.com>
Co-authored-by: Jack Westbrook <jack.westbrook@gmail.com>
Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
Co-authored-by: Leon Sorokin <leeoniya@gmail.com>
Co-authored-by: Andrej Ocenas <mr.ocenas@gmail.com>
Co-authored-by: spinillos <selenepinillos@gmail.com>
Co-authored-by: Karl Persson <kalle.persson@grafana.com>
Co-authored-by: Leonard Gram <leo@xlson.com>
* Convert tag values returned as numbers to string
This is a bug in Graphite <= 1.1.7. Since 1.1.8 all values are converted to strings.
* Simplify type conversions
* Fix linting errors
* Add input to specify min step
* Add stepInterval to as input to component
* Add onBlur to Input component
* Loki: add functionality for min step
* Loki: change name on props to step to make it more clear
* Loki: add resolution as a query option
* Loki: Add min,max,exact as step options
* Loki: add functionality for different step modes
* Loki: fix bug where step function isn't working
* Loki: fix bug where exact step isn't working
* Loki: change width of step input field
* Loki: add tests for adjustInterval function
* Loki: add check for max step oprio to make sure it's not below the safe interval
* Loki: fix bug with some tests
* Loki: fix bug with tests
* Explore: add tooltip to loki step function
* Loki: remove resolution as a logs option
* Loki: update snapshots
* Fix failing tests
* Loki: add select component for choosing resolution
* Loki: add functionality for calculating correct interval with resolution applied
* Loki: remove functionality for step mode
* Loki: remove tests for step mode
* Loki: add tooltip to line limit and resolution
* Loki: add backend support for resolution
* Loki: fixed backend bug where resolution was undefined
* Loki: add check for resolution
* Add deep link in ARG
* add soverigh clouds
* add query to Azure portal link
* move deep link to backend
* cloud specific to ARG
* Add unit test
* fix lint
* fix escape issue
* use constant & fix test
* fix constant & use pathEscape
* use cmp.Equal for test
* Refactor loki to use SDK contracts
* Register with service name
* Initialize interval calculator
* Return always created result
* Use go library instead of simplejson
* Update pkg/tsdb/loki/loki.go
Co-authored-by: Ivana Huckova <30407135+ivanahuckova@users.noreply.github.com>
* Remove newline
* Merge with main
Co-authored-by: Ivana Huckova <30407135+ivanahuckova@users.noreply.github.com>
* Add select component for choosing step option
* Add onStepChange
* Add functionality for max step
* Rename minInterval to stepInterval to describe min, max and exact step interval
* Change select option from standard to exact
* Add new type StepType for better type safety
* Add tests for adjustInterval
* Add functionality and tests for exact step option
* Prometheus: Spell out min and max in select component
* Prometheus: Change width of step select component and add placeholder
* Prometheus: Adjust for the factor in exact step
* Prometheus: Update tooltip of step lable to include max and exact options and add padding to select component to give it some breathing room from other components
* Update snapshot for step tooltip
* Prometheus: make tooltip more informative
* Prometheus: add tooltip to interval input element
* Prometheus: extract default step option
* Prometheus: update snapshot for PromQueryEditor
* Prometheus: change step labels to uppercase
* Prometheus: define a default step option
* Prometheus: use default step option in both ui component and logic
* Prometheus: update snapshot for PromQueryEditor
* Prometheus: refactor datasource.ts for better readability
* Prometheus: change tool tip for step
* Prometheus: update snapshots
* Prometheus: add correct styling
* Prometheus: update snapshots
* Prometheus change variable name to something less superfluous
* Prometheus: refactor
* Prometheus: add new test for adjustInterval
* Docs: Update docummentation on the step parameter for prometheus
* Prometheus: make step input field smaller and change placeholder text to 15s
* Prometheus: update snapshots
* Prometheus: Make stepMode uniform in all places in the code
* Adjust step based on stepMode
* Adjust comment
* Check if we have queryInterval
* Refactor, add safe interval
* Fix merge resolutions
* Fix tests and add tests
* Update snapshot
* Update docs/sources/datasources/prometheus.md
Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>
* Update docs/sources/datasources/prometheus.md
Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>
* Update docs/sources/datasources/prometheus.md
Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>
* Update docs/sources/datasources/prometheus.md
Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>
* Update docs/sources/datasources/prometheus.md
Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>
* Update docs/sources/datasources/prometheus.md
Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>
* Update docs/sources/datasources/prometheus.md
Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>
* Implement calculation with intervalMode in calculator.go
* Update tests, add calculate safe interval method
* Replace panic with error
* Update pkg/tsdb/interval/interval_test.go
Co-authored-by: idafurjes <36131195+idafurjes@users.noreply.github.com>
* Update pkg/tsdb/calculator_test.go
Co-authored-by: idafurjes <36131195+idafurjes@users.noreply.github.com>
* Impotrt require
* Remove lint errors
Co-authored-by: Olof Bourghardt <ob655088@gmail.com>
Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>
Co-authored-by: idafurjes <36131195+idafurjes@users.noreply.github.com>
* Refactor resample logic
* Adjust test to have one more timestamp out of range
* adjust test + ensure filling
* revert flag flip
* Undo logic - should be timeseries only
* change data calculation based on previous interval
* fix the logics
* fix typo
* fix resample start time, to reuse what sql api returned
* calculate the start point with from truncate by interval
Co-authored-by: Will Browne <will.browne@grafana.com>
Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
Co-authored-by: Ying WANG <ying.wang@grafana.com>
* Azure middleware in HttpClientProxy
* Azure authentication under feature flag
* Minor fixes
* Add prefixes to not clash with JsonData
* Return error if JsonData cannot be parsed
* Return original string if URL invalid
* Tests for datasource_cache
* Use backend SDK for influxdb
* Remove BasicAuth condition, some comments
* Remove not used fields from datasource info
* Register InfluxDBService
* Fix casting and make HTTPClientProvider exported
* Remove unused function
* Remove empty line
* Update pkg/tsdb/influxdb/flux/query_models.go
Co-authored-by: Gábor Farkas <gabor.farkas@gmail.com>
* Read interval from TimeRange instead of Interval
* Change pkg name from datasource->models, minor changes
* Use testify instead of convey
* Add new calculator logic and fix pointer semantic for dsInfo
* Initialise parsers, use tsdb interval pkg
Co-authored-by: Gábor Farkas <gabor.farkas@gmail.com>
* Migrate Elasticsearch to backend plugin SDK
* Fix linting
* Move away from Convey!
* Rebase commit
* Small logger fix
* Fixes according to reviewer's comments
* Fixes according to reviewer's comments
* Fixes according to reviewer's comments
* More cleanup
* Move things around - small refactoring
* Fix typo
* Update calculator - add tests
* Fixes according to reviewer's comments
* Revert "Revert "Elasticsearch: add frozen indices search support (#27472)" (#27726)"
This reverts commit 4c7131425b.
* Make label width a bit more consistent
* Add documentation for X-Pack & Frozen Indices support in Elasticsearch
* Change UI & docs casing
* create default empty dataframe
* Remove backticks and simplify regex
* small doc improvement
Name of time field changed in v8 for time series queries from Time to the name of the selected
time column, i.e. time or time_sec. These changes should make sure that the name of time field
is always returned as Time for time series queries.
Fixes#36059
Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
* initial steps for config from data
* Moving to core and separate transforms
* Progress
* Rows to fields are starting to work
* Config from query transform working
* UI progress
* More scenarios working
* Update public/app/core/components/TransformersUI/rowsToFields/rowsToFields.ts
Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
* transform all
* Refactor
* UI starting to work
* Add matcher UI to transform
* Apply to self
* Adding a reducer option
* Value mapping via new all values reducer
* value mappings workg add -A
* Minor changes
* Improving UI and adding test dashboards
* RowsToFieldsTransformerEditor tests
* Added tests for FieldToConfigMapping Editor
* Added test for ConfigFromQueryTransformerEditor
* Minor UI tweaks
* Added missing test
* Added label extraction
* unified mapping
* Progress refactoring
* Updates
* UI tweaks
* Rename
* Updates
Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
* Use Dataframes and extract tags from response
* Fix timestamp conversion
* Add tests for data frame conversion
* Add missing RefID and simplify returning an error
* draft dataframe/sdk convertion for graphite
* intermedia
* modify init because registration failed
* Allocate memory for each data point value
* Remove redundant memory aliasing
* Remove redundant new line
* Sort imports
* Simplify returning nil values
* fix lint
* remove unused jsondata
* add checks on query length
* remove basic auth from request
Co-authored-by: Piotr Jamróz <pm.jamroz@gmail.com>
* Refactor OpenTSDB using backend SDK
* Adjust tests to the SDK refactor
* Remove openTSDB from service
* Rename OpenTASDB service to Service, use AuthPassword from DecryptedSecureJson
* Devenv: Add opentsdb v2.3 data source and dashboard
* Letting http client provider to set basic auth,
renaming,
casting datasource direclty to pointer
* Update pkg/tsdb/opentsdb/opentsdb.go
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
* Update pkg/tsdb/opentsdb/opentsdb.go
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
* Format struct
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
* Elasticsearch: add first version of rate aggregation
* Add test for rate aggreation settings editor
* Fix unit
* Add default value for mode for rate agg
* Add todo
* Revert "Add default value for mode for rate agg"
This reverts commit a44c985d8b.
* Fix width when select is empty
* Add rate aggregation to elasticsearch models
* Remove rate aggregation from extended stats
* Move elastic2 docker block & add elastic (latest)
* Add versionRange to rate aggregation
* add 7.10 elasticsearch version option
* Set supportsInlineScript to true for rate aggregation
* Remove fixed width from select
* Change seconds label for rate aggregation unit options dropdown
Co-authored-by: Piotr Jamróz <pm.jamroz@gmail.com>
Co-authored-by: Elfo404 <me@giordanoricci.com>
Co-authored-by: Piotr Jamróz <pm.jamroz@gmail.com>
* Rebase (broken tests)
* Removed tables - refactored processAggregationDocs func
* Tests cleanup
* Nit - add space in percentile legend titles
* Fix labels naming - use metricAggType map
* Fix bug which appended same fields over and over again
* Replace test with dataframes
* Fix tests
* Add nolint:gocyclo - will need refactoring
* Move frames tags assignment
* Add failing test fo when ES server is booting up
* Revert "Add failing test fo when ES server is booting up"
This reverts commit fd14a1fd5e.
Co-authored-by: Elfo404 <me@giordanoricci.com>
* rename strategy
* Update pkg/tsdb/sqleng/sql_engine.go
Co-authored-by: Torkel Ödegaard <torkel@grafana.org>
* more strict constraints
* Fixed so that it works on multi series results
* only apply the logic when original query returns 3 fields
* removed part of comment
* Update mysql test
Co-authored-by: Torkel Ödegaard <torkel@grafana.org>
Co-authored-by: Torkel Ödegaard <torkel@grafana.com>
Fixes panic/fatal error concurrent map writes in SQL data sources when multiple
queries are executed concurrently and you interpolate SQL query before executing it.
Fixes#35469
* draft pr convert opentsdb response to dataframes
* Add test for parse response and fix go lint
* Add test case for create request
* Use go-cmp in test
* Remove comment
Co-authored-by: Ida Furjesova <ida.furjesova@grafana.com>
* fix sql annotation parsing for empty responses
* fix backend when no data returned
* add back frontend changes
Co-authored-by: Ying WANG <ying.wang@grafana.com>
Move QueryData method into backend plugin manager which HandleRequest uses to
query data from plugin SDK supported data sources. This allowed us to remove a lot
of code no longer needed.
Ref #21510
Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
The /api/ds/query and /api/tsdb/query endpoints extract the intervalMs field from
each query in the request, but it currently seems to be ignored, at
least for queries to a Prometheus datasource. This changes
GetIntervalFrom function to check for the presence of intervalMs in the
query model if interval is missing.
* Migrate template variables straight into resource field,
Display error if unable to migrate
* Update workspace() template variable query function to return a resource URI
* update backend to support resource uris in workspace field
* don't load resource names for variables
* throw error when workspace migration returns no results
* update testDatasource to work with either resource URI or workspace GUID
* fix tests
* cleanup types just a little bit, as a treat
Uses new httpclient package from grafana-plugin-sdk-go introduced
via grafana/grafana-plugin-sdk-go#328.
Replaces the GetHTTPClient, GetTransport, GetTLSConfig methods defined
on DataSource model.
Longer-term the goal is to migrate core HTTP backend data sources to use the
SDK contracts and using httpclient.Provider for creating HTTP clients and such.
Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>
* wip
* wip:
* ui work for resource picker
* disable rows when others are selected
* resource picker open button
* Azure Monitor: Connect to backend with real data (#34024)
* Connect to backend with real data
* Fixes from code review.
* WIP:Begin thinking about how to make queries with scope determined by the resource picker
* More fixes post code review.
* Fixes after rebasing
* Remove outdated todo
* AzureMonitor: Support any resource for Logs queries (#33762)
* Apply button for resource picker
* scroll table body
* secondary cancel button
* loading state for nested rows
* Display resource components in picker button
* fix tests
* fix icons
* move route function
* Migrate from workspace to resource uri for log analytics (#34337)
* reword backwards compat comment
* remove base url suffix
* fix lint error
* move migrations to seperate file
* cleanup
* update regex
* cleanup
* update plugin routes to use new azure auth type
* use AzureDataSourceInstanceSettings alias
Co-authored-by: Sarah Zinger <sarahzinger@users.noreply.github.com>
* Bar chart label positioning and sizing
* Dev dashbard
* Improve autosizing
* Remove sync option
* Unify text sizing options between stat-ish visualizations and bar chart
* Add simple categorical data scenario and update dev dashboard
* Remove unused options builder
* Add docs annotations
* Fix go lint
* Determine Azure cloud based on authentication type
* Route resolution for clouds
* Fix tests
* Constants for cloud names
* move arg cloud route logic to routes.go
Co-authored-by: Erik Sundell <erik.sundell87@gmail.com>
* Add Azure Resource Graph in Azure Plugin
* fix lodash import
* Fix mock queries
* use "backend" sdk
* Address comments
* add converter for object type
* Query error should cause 400 & apply template var
* fix backend test & add documentation
* update image
* Address comments
* marshal body from map
* use interpolated query instead of raw query
* fix test
* filter out empty queries
* fix go linting problem
* use new query field language name
* improve variable tests
* add better tests for interpolate variable
Co-authored-by: joshhunt <josh@trtr.co>
Co-authored-by: Erik Sundell <erik.sundell87@gmail.com>
* add support for handling preprocessors in the backend
* add preprocessor tests
* use uppercase for constants
* add super label component
* remove error message from query editor since its not working (probably cause onDataError doesnt work anymore)
* use cheat sheet instead of help
* add return type annotation for projects
* add support for preprocessing. replace segment comp with select. change components names and refactoring
* cleanup
* more pr feedback
* fix annotations editor
* rename aggregation component
* fix broken test
* remove unnecessary cast
* fix strict errors
* fix more strict errors
* remove not used prop
* update docs
* use same inline label for annotation editor
* fix react prop warning
* disable preprocessing for distribution types
* using new default values for reducer
* auto select 'rate' if metric kind is not gauge
* fix create label format
* pr feedback
* more pr feedback
* update images
* add tests
* CloudWatch: Allow use of missing AWS namespaces using custom metrics
* CloudWatch: Allow use of missing AWS namespaces using custom metrics
* Devenv: add block for es 5.0, provisioned datasource & dashboard
* Trasnsform script property based on running ES version
* Handle different scripts format in BE
* add uninstall flow
* add install flow
* small cleanup
* smaller-footprint solution
* cleanup + make bp start auto
* fix interface contract
* improve naming
* accept version arg
* ensure use of shared logger
* make installer a field
* add plugin decommissioning
* add basic error checking
* fix api docs
* making initialization idempotent
* add mutex
* fix comment
* fix test
* add test for decommission
* improve existing test
* add more test coverage
* more tests
* change test func to use read lock
* refactoring + adding test asserts
* improve purging old install flow
* improve dupe checking
* change log name
* skip over dupe scanned
* make test assertion more flexible
* remove trailing line
* fix pointer receiver name
* update comment
* add context to API
* add config flag
* add base http api test + fix update functionality
* simplify existing check
* clean up test
* refactor tests based on feedback
* add single quotes to errs
* use gcmp in tests + fix logo issue
* make plugin list testing more flexible
* address feedback
* fix API test
* fix linter
* undo preallocate
* Update docs/sources/administration/configuration.md
Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>
* Update docs/sources/administration/configuration.md
Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>
* Update docs/sources/administration/configuration.md
Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>
* fix linting issue in test
* add docs placeholder
* update install notes
* Update docs/sources/plugins/marketplace.md
Co-authored-by: Marcus Olsson <marcus.olsson@hey.com>
* update access wording
* add more placeholder docs
* add link to more info
* PR feedback - improved errors, refactor, lock fix
* improve err details
* propagate plugin version errors
* don't autostart renderer
* add H1
* fix imports
Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>
Co-authored-by: Marcus Olsson <marcus.olsson@hey.com>
* Elasticsearch: use proper semver strings to identify ES version
* Update BE & tests
* refactor BE tests
* refactor isValidOption check
* update test
* Update pkg/tsdb/elasticsearch/client/client.go
Co-authored-by: Piotr Jamróz <pm.jamroz@gmail.com>
* Update pkg/tsdb/elasticsearch/client/search_request_test.go
Co-authored-by: Piotr Jamróz <pm.jamroz@gmail.com>
* Remove leftover FIXME comment
* Add new test cases for new version format
* Docs: add documentation about version dropdown
* Update docs/sources/datasources/elasticsearch.md
Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>
* Update docs/sources/datasources/elasticsearch.md
Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>
* Update docs/sources/datasources/elasticsearch.md
Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>
* Update provisioning documentation
Co-authored-by: Piotr Jamróz <pm.jamroz@gmail.com>
Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>
Convert SQL data sources to return data frames.
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>
Co-authored-by: Will Browne <will.browne@grafana.com>
Co-authored-by: Hugo Häggmark <hugo.haggmark@gmail.com>
* Elasticsearch: Add generic support for template variables
* format MovingAverage settings as numbers
* Move formatting logic to query builder & forma serial_diff settings as numbers
* modify presence check
* add todo
* minor fixes
* transform string values to numbers
* Move casting logic
* Slightly cleaner implementation
* Add BE tests
* Leverage elastic validation when string doesn't resolve to a numeric value
* move newly introduced test to testify
* add FE query_builder tests
* check error
* Parse values to float instead of int
* Fix tests & ParseFloat bit size
