* Prometheus: Run Explore both queries trough backend
* Refactor, simplify
* Set default values for query type selector
* Run multiple queries as one query trough backend
* Remove trailing newlines
* Pass utcOffset
* Remove trailing comma
* WIP: Implementatioon of exemplars
* add sampling for exemplars
* Refactor to use response as custom metadata
* Simplify processing of exemplars
* Update, clean up
* Refactor the way how we get available exemplars
* Simplify exemplars disabling and running on frontend
* Add tests
* Update toggle
* Remove console log
* Fix go linting
* Fix e2e test
* Trigger Build
* Compare lengts, small fix
* Remove duplicated time check
* Address feedback
* Remove redundant ! as not needed
* Update
* Encryption: Add support to encrypt/decrypt sjd
* Add datasources.Service as a proxy to datasources db operations
* Encrypt ds.SecureJsonData before calling SQLStore
* Move ds cache code into ds service
* Fix tlsmanager tests
* Fix pluginproxy tests
* Remove some securejsondata.GetEncryptedJsonData usages
* Add pluginsettings.Service as a proxy for plugin settings db operations
* Add AlertNotificationService as a proxy for alert notification db operations
* Remove some securejsondata.GetEncryptedJsonData usages
* Remove more securejsondata.GetEncryptedJsonData usages
* Fix lint errors
* Minor fixes
* Remove encryption global functions usages from ngalert
* Fix lint errors
* Minor fixes
* Minor fixes
* Remove securejsondata.DecryptedValue usage
* Refactor the refactor
* Remove securejsondata.DecryptedValue usage
* Move securejsondata to migrations package
* Move securejsondata to migrations package
* Minor fix
* Fix integration test
* Fix integration tests
* Undo undesired changes
* Fix tests
* Add context.Context into encryption methods
* Fix tests
* Fix tests
* Fix tests
* Trigger CI
* Fix test
* Add names to params of encryption service interface
* Remove bus from CacheServiceImpl
* Add logging
* Add keys to logger
Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>
* Add missing key to logger
Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>
* Undo changes in markdown files
* Fix formatting
* Add context to secrets service
* Rename decryptSecureJsonData to decryptSecureJsonDataFn
* Name args in GetDecryptedValueFn
* Add template back to NewAlertmanagerNotifier
* Copy GetDecryptedValueFn to ngalert
* Add logging to pluginsettings
* Fix pluginsettings test
Co-authored-by: Tania B <yalyna.ts@gmail.com>
Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>
* Prometheus: Run Explore both queries trough backend
* Refactor, simplify
* Set default values for query type selector
* Run multiple queries as one query trough backend
* Remove trailing newlines
* Pass utcOffset
* Remove trailing comma
* Add meta to frames only at 1 place
* Set exemplars to falsee if they are disabled
* Revert "Set exemplars to falsee if they are disabled"
This reverts commit e7b697c9f3.
* influxdb: flux: handle $__interval and $__interval_ms in alert-queries
* influxdb: flux: do not handle interval-variable in the frontend
* $__interval should be rounded
* added comment
* Prometheus: Add running of instant queries trough backend
* Rename response, keep zero length frames
* Remove debug log
* Simplify and fix value text
* Update duplicated and redundant go test
* Update public/app/plugins/datasource/prometheus/datasource.ts
Co-authored-by: Giordano Ricci <me@giordanoricci.com>
* Refactor using model to create Promquery
* Fix the response length
Co-authored-by: Giordano Ricci <me@giordanoricci.com>
* Run Explore range queries trough backend
* Remove trailing comma
* Add timeRange step alignment to backend
* Remove creation of instant query on backend as it is not supported ATM
* Remove non-related frontend changes
* Pass offset to calculate aligned range trough prom query
* Update order in query error message
* tableRefIds shouldn't contain undefined refIds
* Remove cloning of dataframes when processing
* Don't mutate response
* Remove ordering of processed frames
* Remove df because not needed
Fix bug with engine cache not updating after data source updated, introduced in #36635.
Removes the SQL engine cache because since #36635 we already have instance caching in place using the SDK instance manager.
This also adds support for disposing cached SQL/xorm engines before a query happens to a data source that has been updated. Done by letting the SQL engine implementing the InstanceDisposer interface.
Fixes#37299
Adds a new setting dataproxy.row_limit that allows an operator to limit the
amount of rows being processed/accepted in response to database queries
originating from SQL data sources.
Closes#38975
Ref #39095
Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>
* Revert "Prometheus: add functionality to specify desired step interval in dashboards panels (#36422)"
This reverts commit ddf5b65c51.
Co-authored-by: Ivana Huckova <ivana.huckova@gmail.com>
* Revert "Explore: add functionality for supporting different step modes in prometheus (#37829)"
This reverts commit f433cfd8d9.
Co-authored-by: Ivana Huckova <ivana.huckova@gmail.com>
* Revert stepMode BE implementation from #36796
Co-authored-by: "Ivana Huckova" <ivana.huckova@gmail.com>
Fix region annotations not displayed correctly when returning timeend column
as epoch timestamp and by that making sure that the returned data frame field
named timeend is treated as time type.
Fixes#38533
* convert SQLs to use sdk contracts
* make draft
* postgres
* intermedia
* get datasourceinfo filled at the beginning of the service
* move the interval into package because of cyclict import and fix all postgres tests
* fix mysql test
* fix mssql
* fix the test for pr https://github.com/grafana/grafana/issues/35839
* fix some issue about intervalv2 package
* update sql test
* wire migration for SQLs
* add sqls to the background process
* make it register instead of register and start
* revert formatting
* fix tests
* fix linter
* remove integration test
* Postgres test fix
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
* influxdb: influxql: alert: use already-computed interval-value when available
* reverted by-mistake changed import
* simplify code
now we can rely on DataQuery.Interval always being good
* removed unnecessary test
* fix lint-error
* now to get it to build
* modified names, added the 455 day case, and added a comment
* removed wireguard override i used for my local
* updated the docs
Fixes#30144
Co-authored-by: dsotirakis <sotirakis.dim@gmail.com>
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
Co-authored-by: Ida Furjesova <ida.furjesova@grafana.com>
Co-authored-by: Jack Westbrook <jack.westbrook@gmail.com>
Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
Co-authored-by: Leon Sorokin <leeoniya@gmail.com>
Co-authored-by: Andrej Ocenas <mr.ocenas@gmail.com>
Co-authored-by: spinillos <selenepinillos@gmail.com>
Co-authored-by: Karl Persson <kalle.persson@grafana.com>
Co-authored-by: Leonard Gram <leo@xlson.com>
* Convert tag values returned as numbers to string
This is a bug in Graphite <= 1.1.7. Since 1.1.8 all values are converted to strings.
* Simplify type conversions
* Fix linting errors
* Add input to specify min step
* Add stepInterval to as input to component
* Add onBlur to Input component
* Loki: add functionality for min step
* Loki: change name on props to step to make it more clear
* Loki: add resolution as a query option
* Loki: Add min,max,exact as step options
* Loki: add functionality for different step modes
* Loki: fix bug where step function isn't working
* Loki: fix bug where exact step isn't working
* Loki: change width of step input field
* Loki: add tests for adjustInterval function
* Loki: add check for max step oprio to make sure it's not below the safe interval
* Loki: fix bug with some tests
* Loki: fix bug with tests
* Explore: add tooltip to loki step function
* Loki: remove resolution as a logs option
* Loki: update snapshots
* Fix failing tests
* Loki: add select component for choosing resolution
* Loki: add functionality for calculating correct interval with resolution applied
* Loki: remove functionality for step mode
* Loki: remove tests for step mode
* Loki: add tooltip to line limit and resolution
* Loki: add backend support for resolution
* Loki: fixed backend bug where resolution was undefined
* Loki: add check for resolution
* Add deep link in ARG
* add soverigh clouds
* add query to Azure portal link
* move deep link to backend
* cloud specific to ARG
* Add unit test
* fix lint
* fix escape issue
* use constant & fix test
* fix constant & use pathEscape
* use cmp.Equal for test
* Refactor loki to use SDK contracts
* Register with service name
* Initialize interval calculator
* Return always created result
* Use go library instead of simplejson
* Update pkg/tsdb/loki/loki.go
Co-authored-by: Ivana Huckova <30407135+ivanahuckova@users.noreply.github.com>
* Remove newline
* Merge with main
Co-authored-by: Ivana Huckova <30407135+ivanahuckova@users.noreply.github.com>
* Add select component for choosing step option
* Add onStepChange
* Add functionality for max step
* Rename minInterval to stepInterval to describe min, max and exact step interval
* Change select option from standard to exact
* Add new type StepType for better type safety
* Add tests for adjustInterval
* Add functionality and tests for exact step option
* Prometheus: Spell out min and max in select component
* Prometheus: Change width of step select component and add placeholder
* Prometheus: Adjust for the factor in exact step
* Prometheus: Update tooltip of step lable to include max and exact options and add padding to select component to give it some breathing room from other components
* Update snapshot for step tooltip
* Prometheus: make tooltip more informative
* Prometheus: add tooltip to interval input element
* Prometheus: extract default step option
* Prometheus: update snapshot for PromQueryEditor
* Prometheus: change step labels to uppercase
* Prometheus: define a default step option
* Prometheus: use default step option in both ui component and logic
* Prometheus: update snapshot for PromQueryEditor
* Prometheus: refactor datasource.ts for better readability
* Prometheus: change tool tip for step
* Prometheus: update snapshots
* Prometheus: add correct styling
* Prometheus: update snapshots
* Prometheus change variable name to something less superfluous
* Prometheus: refactor
* Prometheus: add new test for adjustInterval
* Docs: Update docummentation on the step parameter for prometheus
* Prometheus: make step input field smaller and change placeholder text to 15s
* Prometheus: update snapshots
* Prometheus: Make stepMode uniform in all places in the code
* Adjust step based on stepMode
* Adjust comment
* Check if we have queryInterval
* Refactor, add safe interval
* Fix merge resolutions
* Fix tests and add tests
* Update snapshot
* Update docs/sources/datasources/prometheus.md
Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>
* Update docs/sources/datasources/prometheus.md
Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>
* Update docs/sources/datasources/prometheus.md
Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>
* Update docs/sources/datasources/prometheus.md
Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>
* Update docs/sources/datasources/prometheus.md
Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>
* Update docs/sources/datasources/prometheus.md
Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>
* Update docs/sources/datasources/prometheus.md
Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>
* Implement calculation with intervalMode in calculator.go
* Update tests, add calculate safe interval method
* Replace panic with error
* Update pkg/tsdb/interval/interval_test.go
Co-authored-by: idafurjes <36131195+idafurjes@users.noreply.github.com>
* Update pkg/tsdb/calculator_test.go
Co-authored-by: idafurjes <36131195+idafurjes@users.noreply.github.com>
* Impotrt require
* Remove lint errors
Co-authored-by: Olof Bourghardt <ob655088@gmail.com>
Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>
Co-authored-by: idafurjes <36131195+idafurjes@users.noreply.github.com>
* Refactor resample logic
* Adjust test to have one more timestamp out of range
* adjust test + ensure filling
* revert flag flip
* Undo logic - should be timeseries only
* change data calculation based on previous interval
* fix the logics
* fix typo
* fix resample start time, to reuse what sql api returned
* calculate the start point with from truncate by interval
Co-authored-by: Will Browne <will.browne@grafana.com>
Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
Co-authored-by: Ying WANG <ying.wang@grafana.com>
* Azure middleware in HttpClientProxy
* Azure authentication under feature flag
* Minor fixes
* Add prefixes to not clash with JsonData
* Return error if JsonData cannot be parsed
* Return original string if URL invalid
* Tests for datasource_cache
* Use backend SDK for influxdb
* Remove BasicAuth condition, some comments
* Remove not used fields from datasource info
* Register InfluxDBService
* Fix casting and make HTTPClientProvider exported
* Remove unused function
* Remove empty line
* Update pkg/tsdb/influxdb/flux/query_models.go
Co-authored-by: Gábor Farkas <gabor.farkas@gmail.com>
* Read interval from TimeRange instead of Interval
* Change pkg name from datasource->models, minor changes
* Use testify instead of convey
* Add new calculator logic and fix pointer semantic for dsInfo
* Initialise parsers, use tsdb interval pkg
Co-authored-by: Gábor Farkas <gabor.farkas@gmail.com>
* Migrate Elasticsearch to backend plugin SDK
* Fix linting
* Move away from Convey!
* Rebase commit
* Small logger fix
* Fixes according to reviewer's comments
* Fixes according to reviewer's comments
* Fixes according to reviewer's comments
* More cleanup
* Move things around - small refactoring
* Fix typo
* Update calculator - add tests
* Fixes according to reviewer's comments
* Revert "Revert "Elasticsearch: add frozen indices search support (#27472)" (#27726)"
This reverts commit 4c7131425b.
* Make label width a bit more consistent
* Add documentation for X-Pack & Frozen Indices support in Elasticsearch
* Change UI & docs casing
* create default empty dataframe
* Remove backticks and simplify regex
* small doc improvement
Name of time field changed in v8 for time series queries from Time to the name of the selected
time column, i.e. time or time_sec. These changes should make sure that the name of time field
is always returned as Time for time series queries.
Fixes#36059
Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
* initial steps for config from data
* Moving to core and separate transforms
* Progress
* Rows to fields are starting to work
* Config from query transform working
* UI progress
* More scenarios working
* Update public/app/core/components/TransformersUI/rowsToFields/rowsToFields.ts
Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
* transform all
* Refactor
* UI starting to work
* Add matcher UI to transform
* Apply to self
* Adding a reducer option
* Value mapping via new all values reducer
* value mappings workg add -A
* Minor changes
* Improving UI and adding test dashboards
* RowsToFieldsTransformerEditor tests
* Added tests for FieldToConfigMapping Editor
* Added test for ConfigFromQueryTransformerEditor
* Minor UI tweaks
* Added missing test
* Added label extraction
* unified mapping
* Progress refactoring
* Updates
* UI tweaks
* Rename
* Updates
Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
* Use Dataframes and extract tags from response
* Fix timestamp conversion
* Add tests for data frame conversion
* Add missing RefID and simplify returning an error
* draft dataframe/sdk convertion for graphite
* intermedia
* modify init because registration failed
* Allocate memory for each data point value
* Remove redundant memory aliasing
* Remove redundant new line
* Sort imports
* Simplify returning nil values
* fix lint
* remove unused jsondata
* add checks on query length
* remove basic auth from request
Co-authored-by: Piotr Jamróz <pm.jamroz@gmail.com>
* Refactor OpenTSDB using backend SDK
* Adjust tests to the SDK refactor
* Remove openTSDB from service
* Rename OpenTASDB service to Service, use AuthPassword from DecryptedSecureJson
* Devenv: Add opentsdb v2.3 data source and dashboard
* Letting http client provider to set basic auth,
renaming,
casting datasource direclty to pointer
* Update pkg/tsdb/opentsdb/opentsdb.go
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
* Update pkg/tsdb/opentsdb/opentsdb.go
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
* Format struct
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
* Elasticsearch: add first version of rate aggregation
* Add test for rate aggreation settings editor
* Fix unit
* Add default value for mode for rate agg
* Add todo
* Revert "Add default value for mode for rate agg"
This reverts commit a44c985d8b.
* Fix width when select is empty
* Add rate aggregation to elasticsearch models
* Remove rate aggregation from extended stats
* Move elastic2 docker block & add elastic (latest)
* Add versionRange to rate aggregation
* add 7.10 elasticsearch version option
* Set supportsInlineScript to true for rate aggregation
* Remove fixed width from select
* Change seconds label for rate aggregation unit options dropdown
Co-authored-by: Piotr Jamróz <pm.jamroz@gmail.com>
Co-authored-by: Elfo404 <me@giordanoricci.com>
Co-authored-by: Piotr Jamróz <pm.jamroz@gmail.com>
* Rebase (broken tests)
* Removed tables - refactored processAggregationDocs func
* Tests cleanup
* Nit - add space in percentile legend titles
* Fix labels naming - use metricAggType map
* Fix bug which appended same fields over and over again
* Replace test with dataframes
* Fix tests
* Add nolint:gocyclo - will need refactoring
* Move frames tags assignment
* Add failing test fo when ES server is booting up
* Revert "Add failing test fo when ES server is booting up"
This reverts commit fd14a1fd5e.
Co-authored-by: Elfo404 <me@giordanoricci.com>
* rename strategy
* Update pkg/tsdb/sqleng/sql_engine.go
Co-authored-by: Torkel Ödegaard <torkel@grafana.org>
* more strict constraints
* Fixed so that it works on multi series results
* only apply the logic when original query returns 3 fields
* removed part of comment
* Update mysql test
Co-authored-by: Torkel Ödegaard <torkel@grafana.org>
Co-authored-by: Torkel Ödegaard <torkel@grafana.com>
Fixes panic/fatal error concurrent map writes in SQL data sources when multiple
queries are executed concurrently and you interpolate SQL query before executing it.
Fixes#35469
* draft pr convert opentsdb response to dataframes
* Add test for parse response and fix go lint
* Add test case for create request
* Use go-cmp in test
* Remove comment
Co-authored-by: Ida Furjesova <ida.furjesova@grafana.com>
* fix sql annotation parsing for empty responses
* fix backend when no data returned
* add back frontend changes
Co-authored-by: Ying WANG <ying.wang@grafana.com>
Move QueryData method into backend plugin manager which HandleRequest uses to
query data from plugin SDK supported data sources. This allowed us to remove a lot
of code no longer needed.
Ref #21510
Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
The /api/ds/query and /api/tsdb/query endpoints extract the intervalMs field from
each query in the request, but it currently seems to be ignored, at
least for queries to a Prometheus datasource. This changes
GetIntervalFrom function to check for the presence of intervalMs in the
query model if interval is missing.
* Migrate template variables straight into resource field,
Display error if unable to migrate
* Update workspace() template variable query function to return a resource URI
* update backend to support resource uris in workspace field
* don't load resource names for variables
* throw error when workspace migration returns no results
* update testDatasource to work with either resource URI or workspace GUID
* fix tests
* cleanup types just a little bit, as a treat
Uses new httpclient package from grafana-plugin-sdk-go introduced
via grafana/grafana-plugin-sdk-go#328.
Replaces the GetHTTPClient, GetTransport, GetTLSConfig methods defined
on DataSource model.
Longer-term the goal is to migrate core HTTP backend data sources to use the
SDK contracts and using httpclient.Provider for creating HTTP clients and such.
Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>
* wip
* wip:
* ui work for resource picker
* disable rows when others are selected
* resource picker open button
* Azure Monitor: Connect to backend with real data (#34024)
* Connect to backend with real data
* Fixes from code review.
* WIP:Begin thinking about how to make queries with scope determined by the resource picker
* More fixes post code review.
* Fixes after rebasing
* Remove outdated todo
* AzureMonitor: Support any resource for Logs queries (#33762)
* Apply button for resource picker
* scroll table body
* secondary cancel button
* loading state for nested rows
* Display resource components in picker button
* fix tests
* fix icons
* move route function
* Migrate from workspace to resource uri for log analytics (#34337)
* reword backwards compat comment
* remove base url suffix
* fix lint error
* move migrations to seperate file
* cleanup
* update regex
* cleanup
* update plugin routes to use new azure auth type
* use AzureDataSourceInstanceSettings alias
Co-authored-by: Sarah Zinger <sarahzinger@users.noreply.github.com>
* Bar chart label positioning and sizing
* Dev dashbard
* Improve autosizing
* Remove sync option
* Unify text sizing options between stat-ish visualizations and bar chart
* Add simple categorical data scenario and update dev dashboard
* Remove unused options builder
* Add docs annotations
* Fix go lint
* Determine Azure cloud based on authentication type
* Route resolution for clouds
* Fix tests
* Constants for cloud names
* move arg cloud route logic to routes.go
Co-authored-by: Erik Sundell <erik.sundell87@gmail.com>
* Add Azure Resource Graph in Azure Plugin
* fix lodash import
* Fix mock queries
* use "backend" sdk
* Address comments
* add converter for object type
* Query error should cause 400 & apply template var
* fix backend test & add documentation
* update image
* Address comments
* marshal body from map
* use interpolated query instead of raw query
* fix test
* filter out empty queries
* fix go linting problem
* use new query field language name
* improve variable tests
* add better tests for interpolate variable
Co-authored-by: joshhunt <josh@trtr.co>
Co-authored-by: Erik Sundell <erik.sundell87@gmail.com>
* add support for handling preprocessors in the backend
* add preprocessor tests
* use uppercase for constants
* add super label component
* remove error message from query editor since its not working (probably cause onDataError doesnt work anymore)
* use cheat sheet instead of help
* add return type annotation for projects
* add support for preprocessing. replace segment comp with select. change components names and refactoring
* cleanup
* more pr feedback
* fix annotations editor
* rename aggregation component
* fix broken test
* remove unnecessary cast
* fix strict errors
* fix more strict errors
* remove not used prop
* update docs
* use same inline label for annotation editor
* fix react prop warning
* disable preprocessing for distribution types
* using new default values for reducer
* auto select 'rate' if metric kind is not gauge
* fix create label format
* pr feedback
* more pr feedback
* update images
* add tests
* CloudWatch: Allow use of missing AWS namespaces using custom metrics
* CloudWatch: Allow use of missing AWS namespaces using custom metrics
* Devenv: add block for es 5.0, provisioned datasource & dashboard
* Trasnsform script property based on running ES version
* Handle different scripts format in BE
