grafana/pkg/tsdb/azuremonitor/metrics/azuremonitor-datasource_test.go
Andreas Christou 63383ef545
AzureMonitor: Application Insights Traces (#64859)
* Build out barebones Traces editor

- Add Traces query type and operation ID prop to query type
- Add necessary header types
- Update resource picker to appropriately work with traces query type
- Build out TracesQueryEditor component
- Include logic to retrieve operationId's for AI Workspaces
- Add backend route mapping
- Update macro to use timestamp as default time field for traces

* AzureMonitor: Traces - Response parsing (#65442)

* Update FormatAsField component

- Add trace ResultFormat type
- Generalise FormatAsField component
- Add component to TracesQueryEditor
- Remove duplicate code in setQueryValue

* Add custom filter function to improve performance

* Add basic conversion for logs to trace

- Add serviceTags converter
- Pass through required parameters (queryType and resultFormat)
- Appropriately set visualisation

* Update parsing to also fill trace tags

- Add constant values for each table schema (include legacy mapping for now if needed)
- Add constant for list of table tags
- Set the foundation for dynamic query building
- Update query to build tags value
- Appropriately set operationName
- Update tagsConverter to filter empty values

* Fix lint and test issues

* AzureMonitor: Traces - Data links (#65566)

* Add portal link for traces

- Pull out necessary values (itemId and itemType)
- Appropriately construct
- Fix ordering

* Set default format as value

- Also set default visualisation

* Fix event schema

* Set default formatAsField value

* Include logs link on traces results

- Adapt config links to allow custom title to be set

* Correctly set operationId for query

* Update backend types

- Include OperationID in query
- Pass forward datasource name and UID

* Ensure setTime doesn't consistently get called if operationID is defined

* Add explore link

- Update util functions to allow setting custom datalinks

* Fix tests

* AzureMonitor: Traces - Query and Editor updates (#66076)

* Add initial query

- Will query the resource as soon as a resource has been selected
- Updates the data links for the query without operationId
- Remove initial operationId query and timeRange dependency
- Update query building

* Add entirely separate traces query property

- Update shared types (also including future types for Azure traces)
- Update backend log analytics datasource to accept both azureLogAnalytics and azureTraces queries
- Update backend specific types
- Update frontend datasource for new properties
- Update mock query

* Update FormatAsField to be entirely generic

* Update query building to be done in backend

- Add required mappings in backend
- Update frontend querying

* Fix query and explore data link

* Add trace type selection

* Better method for setting explore link

* Fix operationId updating

* Run go mod tidy

* Unnecessary changes

* Fix tests

* AzureMonitor: Traces - Add correlation API support (#65855)

Add correlation API support

- Add necessary types
- Add correlation API request when conditions are met
- Update query

* Fix property from merge

* AzureMonitor: Traces - Filtering (#66303)

* Add initial query

- Will query the resource as soon as a resource has been selected
- Updates the data links for the query without operationId
- Remove initial operationId query and timeRange dependency
- Update query building

* Add entirely separate traces query property

- Update shared types (also including future types for Azure traces)
- Update backend log analytics datasource to accept both azureLogAnalytics and azureTraces queries
- Update backend specific types
- Update frontend datasource for new properties
- Update mock query

* Update FormatAsField to be entirely generic

* Update query building to be done in backend

- Add required mappings in backend
- Update frontend querying

* Fix query and explore data link

* Add trace type selection

* Better method for setting explore link

* Fix operationId updating

* Run go mod tidy

* Unnecessary changes

* Fix tests

* Start building out Filters component

- Configure component to query for Filter property values when a filter property is set
- Add setFilters function
- Add typing to tablesSchema
- Use component in TracesQueryEditor

* Update Filters

- Asynchronously pull property options
- Setup list of Filter components

* Update filters component

- Remove unused imports
- Have local filters state and query filters
- Correctly set filters values
- Don't update query every time a filter property changes (not performant)

* Update properties query

- Use current timeRange
- Get count to provide informative labels

* Reset map when time changes

* Add operation selection

* Reset filters when property changes

* Appropriate label name for empty values

* Add filtering to query

* Update filter components

- Fix rendering issue
- Correctly compare and update timeRange
- Split out files for simplicity

* Add checkbox option to multiselect

- Add custom option component
- Correctly call onChange
- Add variableOptionGroup for template variable selection

* Fix adding template vars

* Improve labels and refresh labels on query prop changes

* AzureMonitor: Traces - Testing (#66474)

* Select ds for template variable interpolation

* Update az logs ds tests

- Add templateVariables test
- Add filter test
- Update mock
- Remove anys

* Update QueryEditor test

- Update mocks with timeSrv for log analytics datasource
- Fix query mock
- Use appropriate and consistent selectors

* Add TracesQueryEditor test

- Update resourcePickerRows mock to include app insights resources
- Remove comments and extra new line

* Add FormatAsField test

- Remove unneeded condition

* Update resourcePicker utils test

* Don't hide selected options in filters

* Fix multi-selection on filters

* Add TraceTypeField test

- Add test file
- Update selectors (remove copy/paste mistake)
- Update placeholder text for select and add label

* Add basic filters test

* Begin filters test

* Update filters test

* Add final tests and simplify/generalise addFilter helper

* Minor update to datasource test

* Update macros test

* Update selectors in tests

* Add response-table-frame tests

* Add datasource tests

- Use sorting where JSON models are inconsistent
- Update filters clause
- Dedupe tags
- Correct operationId conditions

* Don't set a default value for blurInputOnSelect

* Simplify datasource test

* Update to use CheckGoldenJSON utils

- Update with generated frame files
- Remove redundant expected frame code
- Update all usages

* Fix lint

* AzureMonitor: Traces feedback (#67292)

* Filter traces if the visualisation is set to trace

- Update build query logic
- Added additional test cases
- Return an error if the traces type is set by itself with the trace visualisation
- Add descriptions to event types
- Update tests

* Fix bug for error displaying traces

* Update mappings and add error field

- Update tests
- Remove unnecessary comments

* Switch location of Operation ID field

* Re-order fields

* Update link title

* Update label for event type selection

* Update correct link title

* Update logs datalink to link to Azure Logs in explore

* Fix lint
2023-04-27 20:24:11 +01:00

640 lines
36 KiB
Go

package metrics
import (
"context"
"encoding/json"
"fmt"
"net/http"
"net/url"
"os"
"path/filepath"
"strings"
"testing"
"time"
"github.com/google/go-cmp/cmp"
"github.com/google/go-cmp/cmp/cmpopts"
"github.com/grafana/grafana-plugin-sdk-go/backend"
"github.com/grafana/grafana-plugin-sdk-go/data"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/grafana/grafana/pkg/components/simplejson"
"github.com/grafana/grafana/pkg/infra/log"
"github.com/grafana/grafana/pkg/tsdb/azuremonitor/testdata"
azTime "github.com/grafana/grafana/pkg/tsdb/azuremonitor/time"
"github.com/grafana/grafana/pkg/tsdb/azuremonitor/types"
"github.com/grafana/grafana/pkg/util"
)
func TestAzureMonitorBuildQueries(t *testing.T) {
datasource := &AzureMonitorDatasource{}
dsInfo := types.DatasourceInfo{
Settings: types.AzureMonitorSettings{
SubscriptionId: "default-subscription",
},
}
fromStart := time.Date(2018, 3, 15, 13, 0, 0, 0, time.UTC).In(time.Local)
duration, _ := time.ParseDuration("400s")
wildcardFilter := "*"
testFilter := "test"
tests := []struct {
name string
azureMonitorVariedProperties map[string]interface{}
azureMonitorQueryTarget string
expectedInterval string
queryInterval time.Duration
expectedURL string
expectedBodyFilter string
expectedParamFilter string
expectedPortalURL *string
}{
{
name: "Parse queries from frontend and build AzureMonitor API queries",
azureMonitorVariedProperties: map[string]interface{}{
"resourceURI": "/subscriptions/12345678-aaaa-bbbb-cccc-123456789abc/resourceGroups/grafanastaging/providers/Microsoft.Compute/virtualMachines/grafana",
"timeGrain": "PT1M",
"top": "10",
},
expectedInterval: "PT1M",
azureMonitorQueryTarget: "aggregation=Average&api-version=2021-05-01&interval=PT1M&metricnames=Percentage+CPU&metricnamespace=Microsoft.Compute%2FvirtualMachines&timespan=2018-03-15T13%3A00%3A00Z%2F2018-03-15T13%3A34%3A00Z",
},
{
name: "legacy query without resourceURI and time grain set to auto",
azureMonitorVariedProperties: map[string]interface{}{
"timeGrain": "auto",
"top": "10",
},
queryInterval: duration,
expectedInterval: "PT15M",
azureMonitorQueryTarget: "aggregation=Average&api-version=2021-05-01&interval=PT15M&metricnames=Percentage+CPU&metricnamespace=Microsoft.Compute%2FvirtualMachines&timespan=2018-03-15T13%3A00%3A00Z%2F2018-03-15T13%3A34%3A00Z",
},
{
name: "legacy query without resourceURI and time grain set to auto",
azureMonitorVariedProperties: map[string]interface{}{
"timeGrain": "auto",
"allowedTimeGrainsMs": []int64{60000, 300000},
"top": "10",
},
queryInterval: duration,
expectedInterval: "PT5M",
azureMonitorQueryTarget: "aggregation=Average&api-version=2021-05-01&interval=PT5M&metricnames=Percentage+CPU&metricnamespace=Microsoft.Compute%2FvirtualMachines&timespan=2018-03-15T13%3A00%3A00Z%2F2018-03-15T13%3A34%3A00Z",
},
{
name: "legacy query without resourceURI and has a dimension filter",
azureMonitorVariedProperties: map[string]interface{}{
"timeGrain": "PT1M",
"dimension": "blob",
"dimensionFilter": "*",
"top": "30",
},
queryInterval: duration,
expectedInterval: "PT1M",
azureMonitorQueryTarget: "aggregation=Average&api-version=2021-05-01&interval=PT1M&metricnames=Percentage+CPU&metricnamespace=Microsoft.Compute%2FvirtualMachines&timespan=2018-03-15T13%3A00%3A00Z%2F2018-03-15T13%3A34%3A00Z&top=30",
expectedParamFilter: "blob eq '*'",
},
{
name: "legacy query without resourceURI and has a dimension filter and none Dimension",
azureMonitorVariedProperties: map[string]interface{}{
"timeGrain": "PT1M",
"dimension": "None",
"dimensionFilter": "*",
"top": "10",
},
queryInterval: duration,
expectedInterval: "PT1M",
azureMonitorQueryTarget: "aggregation=Average&api-version=2021-05-01&interval=PT1M&metricnames=Percentage+CPU&metricnamespace=Microsoft.Compute%2FvirtualMachines&timespan=2018-03-15T13%3A00%3A00Z%2F2018-03-15T13%3A34%3A00Z",
},
{
name: "legacy query without resourceURI and has dimensionFilter*s* property with one dimension",
azureMonitorVariedProperties: map[string]interface{}{
"timeGrain": "PT1M",
"dimensionFilters": []types.AzureMonitorDimensionFilter{{Dimension: "blob", Operator: "eq", Filter: &wildcardFilter}},
"top": "30",
},
queryInterval: duration,
expectedInterval: "PT1M",
azureMonitorQueryTarget: "aggregation=Average&api-version=2021-05-01&interval=PT1M&metricnames=Percentage+CPU&metricnamespace=Microsoft.Compute%2FvirtualMachines&timespan=2018-03-15T13%3A00%3A00Z%2F2018-03-15T13%3A34%3A00Z&top=30",
expectedParamFilter: "blob eq '*'",
expectedPortalURL: util.Pointer("http://ds/#blade/Microsoft_Azure_MonitoringMetrics/Metrics.ReactView/Referer/MetricsExplorer/TimeContext/%7B%22absolute%22%3A%7B%22startTime%22%3A%222018-03-15T13%3A00%3A00Z%22%2C%22endTime%22%3A%222018-03-15T13%3A34%3A00Z%22%7D%7D/ChartDefinition/%7B%22v2charts%22%3A%5B%7B%22grouping%22%3A%7B%22dimension%22%3A%22blob%22%2C%22sort%22%3A2%2C%22top%22%3A10%7D%2C%22metrics%22%3A%5B%7B%22resourceMetadata%22%3A%7B%22id%22%3A%22%2Fsubscriptions%2F12345678-aaaa-bbbb-cccc-123456789abc%2FresourceGroups%2Fgrafanastaging%2Fproviders%2FMicrosoft.Compute%2FvirtualMachines%2Fgrafana%22%7D%2C%22name%22%3A%22Percentage%20CPU%22%2C%22aggregationType%22%3A4%2C%22namespace%22%3A%22Microsoft.Compute%2FvirtualMachines%22%2C%22metricVisualization%22%3A%7B%22displayName%22%3A%22Percentage%20CPU%22%2C%22resourceDisplayName%22%3A%22grafana%22%7D%7D%5D%7D%5D%7D"),
},
{
name: "legacy query without resourceURI and has dimensionFilter*s* property with two dimensions",
azureMonitorVariedProperties: map[string]interface{}{
"timeGrain": "PT1M",
"dimensionFilters": []types.AzureMonitorDimensionFilter{{Dimension: "blob", Operator: "eq", Filter: &wildcardFilter}, {Dimension: "tier", Operator: "eq", Filter: &wildcardFilter}},
"top": "30",
},
queryInterval: duration,
expectedInterval: "PT1M",
azureMonitorQueryTarget: "aggregation=Average&api-version=2021-05-01&interval=PT1M&metricnames=Percentage+CPU&metricnamespace=Microsoft.Compute%2FvirtualMachines&timespan=2018-03-15T13%3A00%3A00Z%2F2018-03-15T13%3A34%3A00Z&top=30",
expectedParamFilter: "blob eq '*' and tier eq '*'",
expectedPortalURL: util.Pointer("http://ds/#blade/Microsoft_Azure_MonitoringMetrics/Metrics.ReactView/Referer/MetricsExplorer/TimeContext/%7B%22absolute%22%3A%7B%22startTime%22%3A%222018-03-15T13%3A00%3A00Z%22%2C%22endTime%22%3A%222018-03-15T13%3A34%3A00Z%22%7D%7D/ChartDefinition/%7B%22v2charts%22%3A%5B%7B%22grouping%22%3A%7B%22dimension%22%3A%22blob%22%2C%22sort%22%3A2%2C%22top%22%3A10%7D%2C%22metrics%22%3A%5B%7B%22resourceMetadata%22%3A%7B%22id%22%3A%22%2Fsubscriptions%2F12345678-aaaa-bbbb-cccc-123456789abc%2FresourceGroups%2Fgrafanastaging%2Fproviders%2FMicrosoft.Compute%2FvirtualMachines%2Fgrafana%22%7D%2C%22name%22%3A%22Percentage%20CPU%22%2C%22aggregationType%22%3A4%2C%22namespace%22%3A%22Microsoft.Compute%2FvirtualMachines%22%2C%22metricVisualization%22%3A%7B%22displayName%22%3A%22Percentage%20CPU%22%2C%22resourceDisplayName%22%3A%22grafana%22%7D%7D%5D%7D%5D%7D"),
},
{
name: "legacy query without resourceURI and has a dimension filter without specifying a top",
azureMonitorVariedProperties: map[string]interface{}{
"timeGrain": "PT1M",
"dimension": "blob",
"dimensionFilter": "*",
},
queryInterval: duration,
expectedInterval: "PT1M",
azureMonitorQueryTarget: "aggregation=Average&api-version=2021-05-01&interval=PT1M&metricnames=Percentage+CPU&metricnamespace=Microsoft.Compute%2FvirtualMachines&timespan=2018-03-15T13%3A00%3A00Z%2F2018-03-15T13%3A34%3A00Z",
expectedParamFilter: "blob eq '*'",
},
{
name: "has dimensionFilter*s* property with not equals operator",
azureMonitorVariedProperties: map[string]interface{}{
"timeGrain": "PT1M",
"dimensionFilters": []types.AzureMonitorDimensionFilter{{Dimension: "blob", Operator: "ne", Filter: &wildcardFilter, Filters: []string{"test"}}},
"top": "30",
},
queryInterval: duration,
expectedInterval: "PT1M",
azureMonitorQueryTarget: "aggregation=Average&api-version=2021-05-01&interval=PT1M&metricnames=Percentage+CPU&metricnamespace=Microsoft.Compute%2FvirtualMachines&timespan=2018-03-15T13%3A00%3A00Z%2F2018-03-15T13%3A34%3A00Z&top=30",
expectedParamFilter: "blob ne 'test'",
expectedPortalURL: util.Pointer("http://ds/#blade/Microsoft_Azure_MonitoringMetrics/Metrics.ReactView/Referer/MetricsExplorer/TimeContext/%7B%22absolute%22%3A%7B%22startTime%22%3A%222018-03-15T13%3A00%3A00Z%22%2C%22endTime%22%3A%222018-03-15T13%3A34%3A00Z%22%7D%7D/ChartDefinition/%7B%22v2charts%22%3A%5B%7B%22filterCollection%22%3A%7B%22filters%22%3A%5B%7B%22key%22%3A%22blob%22%2C%22operator%22%3A1%2C%22values%22%3A%5B%22test%22%5D%7D%5D%7D%2C%22grouping%22%3A%7B%22dimension%22%3A%22blob%22%2C%22sort%22%3A2%2C%22top%22%3A10%7D%2C%22metrics%22%3A%5B%7B%22resourceMetadata%22%3A%7B%22id%22%3A%22%2Fsubscriptions%2F12345678-aaaa-bbbb-cccc-123456789abc%2FresourceGroups%2Fgrafanastaging%2Fproviders%2FMicrosoft.Compute%2FvirtualMachines%2Fgrafana%22%7D%2C%22name%22%3A%22Percentage%20CPU%22%2C%22aggregationType%22%3A4%2C%22namespace%22%3A%22Microsoft.Compute%2FvirtualMachines%22%2C%22metricVisualization%22%3A%7B%22displayName%22%3A%22Percentage%20CPU%22%2C%22resourceDisplayName%22%3A%22grafana%22%7D%7D%5D%7D%5D%7D"),
},
{
name: "has dimensionFilter*s* property with startsWith operator",
azureMonitorVariedProperties: map[string]interface{}{
"timeGrain": "PT1M",
"dimensionFilters": []types.AzureMonitorDimensionFilter{{Dimension: "blob", Operator: "sw", Filter: &testFilter}},
"top": "30",
},
queryInterval: duration,
expectedInterval: "PT1M",
azureMonitorQueryTarget: "aggregation=Average&api-version=2021-05-01&interval=PT1M&metricnames=Percentage+CPU&metricnamespace=Microsoft.Compute%2FvirtualMachines&timespan=2018-03-15T13%3A00%3A00Z%2F2018-03-15T13%3A34%3A00Z&top=30",
expectedParamFilter: "blob sw 'test'",
expectedPortalURL: util.Pointer("http://ds/#blade/Microsoft_Azure_MonitoringMetrics/Metrics.ReactView/Referer/MetricsExplorer/TimeContext/%7B%22absolute%22%3A%7B%22startTime%22%3A%222018-03-15T13%3A00%3A00Z%22%2C%22endTime%22%3A%222018-03-15T13%3A34%3A00Z%22%7D%7D/ChartDefinition/%7B%22v2charts%22%3A%5B%7B%22filterCollection%22%3A%7B%22filters%22%3A%5B%7B%22key%22%3A%22blob%22%2C%22operator%22%3A3%2C%22values%22%3A%5B%22test%22%5D%7D%5D%7D%2C%22grouping%22%3A%7B%22dimension%22%3A%22blob%22%2C%22sort%22%3A2%2C%22top%22%3A10%7D%2C%22metrics%22%3A%5B%7B%22resourceMetadata%22%3A%7B%22id%22%3A%22%2Fsubscriptions%2F12345678-aaaa-bbbb-cccc-123456789abc%2FresourceGroups%2Fgrafanastaging%2Fproviders%2FMicrosoft.Compute%2FvirtualMachines%2Fgrafana%22%7D%2C%22name%22%3A%22Percentage%20CPU%22%2C%22aggregationType%22%3A4%2C%22namespace%22%3A%22Microsoft.Compute%2FvirtualMachines%22%2C%22metricVisualization%22%3A%7B%22displayName%22%3A%22Percentage%20CPU%22%2C%22resourceDisplayName%22%3A%22grafana%22%7D%7D%5D%7D%5D%7D"),
},
{
name: "correctly sets dimension operator to eq (irrespective of operator) when filter value is '*'",
azureMonitorVariedProperties: map[string]interface{}{
"timeGrain": "PT1M",
"dimensionFilters": []types.AzureMonitorDimensionFilter{{Dimension: "blob", Operator: "sw", Filter: &wildcardFilter}, {Dimension: "tier", Operator: "ne", Filter: &wildcardFilter}},
"top": "30",
},
queryInterval: duration,
expectedInterval: "PT1M",
azureMonitorQueryTarget: "aggregation=Average&api-version=2021-05-01&interval=PT1M&metricnames=Percentage+CPU&metricnamespace=Microsoft.Compute%2FvirtualMachines&timespan=2018-03-15T13%3A00%3A00Z%2F2018-03-15T13%3A34%3A00Z&top=30",
expectedParamFilter: "blob eq '*' and tier eq '*'",
expectedPortalURL: util.Pointer("http://ds/#blade/Microsoft_Azure_MonitoringMetrics/Metrics.ReactView/Referer/MetricsExplorer/TimeContext/%7B%22absolute%22%3A%7B%22startTime%22%3A%222018-03-15T13%3A00%3A00Z%22%2C%22endTime%22%3A%222018-03-15T13%3A34%3A00Z%22%7D%7D/ChartDefinition/%7B%22v2charts%22%3A%5B%7B%22grouping%22%3A%7B%22dimension%22%3A%22blob%22%2C%22sort%22%3A2%2C%22top%22%3A10%7D%2C%22metrics%22%3A%5B%7B%22resourceMetadata%22%3A%7B%22id%22%3A%22%2Fsubscriptions%2F12345678-aaaa-bbbb-cccc-123456789abc%2FresourceGroups%2Fgrafanastaging%2Fproviders%2FMicrosoft.Compute%2FvirtualMachines%2Fgrafana%22%7D%2C%22name%22%3A%22Percentage%20CPU%22%2C%22aggregationType%22%3A4%2C%22namespace%22%3A%22Microsoft.Compute%2FvirtualMachines%22%2C%22metricVisualization%22%3A%7B%22displayName%22%3A%22Percentage%20CPU%22%2C%22resourceDisplayName%22%3A%22grafana%22%7D%7D%5D%7D%5D%7D"),
},
{
name: "correctly constructs target when multiple filter values are provided for the 'eq' operator",
azureMonitorVariedProperties: map[string]interface{}{
"timeGrain": "PT1M",
"dimensionFilters": []types.AzureMonitorDimensionFilter{{Dimension: "blob", Operator: "eq", Filter: &wildcardFilter, Filters: []string{"test", "test2"}}},
"top": "30",
},
queryInterval: duration,
expectedInterval: "PT1M",
azureMonitorQueryTarget: "aggregation=Average&api-version=2021-05-01&interval=PT1M&metricnames=Percentage+CPU&metricnamespace=Microsoft.Compute%2FvirtualMachines&timespan=2018-03-15T13%3A00%3A00Z%2F2018-03-15T13%3A34%3A00Z&top=30",
expectedParamFilter: "blob eq 'test' or blob eq 'test2'",
expectedPortalURL: util.Pointer("http://ds/#blade/Microsoft_Azure_MonitoringMetrics/Metrics.ReactView/Referer/MetricsExplorer/TimeContext/%7B%22absolute%22%3A%7B%22startTime%22%3A%222018-03-15T13%3A00%3A00Z%22%2C%22endTime%22%3A%222018-03-15T13%3A34%3A00Z%22%7D%7D/ChartDefinition/%7B%22v2charts%22%3A%5B%7B%22filterCollection%22%3A%7B%22filters%22%3A%5B%7B%22key%22%3A%22blob%22%2C%22operator%22%3A0%2C%22values%22%3A%5B%22test%22%2C%22test2%22%5D%7D%5D%7D%2C%22grouping%22%3A%7B%22dimension%22%3A%22blob%22%2C%22sort%22%3A2%2C%22top%22%3A10%7D%2C%22metrics%22%3A%5B%7B%22resourceMetadata%22%3A%7B%22id%22%3A%22%2Fsubscriptions%2F12345678-aaaa-bbbb-cccc-123456789abc%2FresourceGroups%2Fgrafanastaging%2Fproviders%2FMicrosoft.Compute%2FvirtualMachines%2Fgrafana%22%7D%2C%22name%22%3A%22Percentage%20CPU%22%2C%22aggregationType%22%3A4%2C%22namespace%22%3A%22Microsoft.Compute%2FvirtualMachines%22%2C%22metricVisualization%22%3A%7B%22displayName%22%3A%22Percentage%20CPU%22%2C%22resourceDisplayName%22%3A%22grafana%22%7D%7D%5D%7D%5D%7D"),
},
{
name: "correctly constructs target when multiple filter values are provided for ne 'eq' operator",
azureMonitorVariedProperties: map[string]interface{}{
"timeGrain": "PT1M",
"dimensionFilters": []types.AzureMonitorDimensionFilter{{Dimension: "blob", Operator: "ne", Filter: &wildcardFilter, Filters: []string{"test", "test2"}}},
"top": "30",
},
queryInterval: duration,
expectedInterval: "PT1M",
azureMonitorQueryTarget: "aggregation=Average&api-version=2021-05-01&interval=PT1M&metricnames=Percentage+CPU&metricnamespace=Microsoft.Compute%2FvirtualMachines&timespan=2018-03-15T13%3A00%3A00Z%2F2018-03-15T13%3A34%3A00Z&top=30",
expectedParamFilter: "blob ne 'test' and blob ne 'test2'",
expectedPortalURL: util.Pointer("http://ds/#blade/Microsoft_Azure_MonitoringMetrics/Metrics.ReactView/Referer/MetricsExplorer/TimeContext/%7B%22absolute%22%3A%7B%22startTime%22%3A%222018-03-15T13%3A00%3A00Z%22%2C%22endTime%22%3A%222018-03-15T13%3A34%3A00Z%22%7D%7D/ChartDefinition/%7B%22v2charts%22%3A%5B%7B%22filterCollection%22%3A%7B%22filters%22%3A%5B%7B%22key%22%3A%22blob%22%2C%22operator%22%3A1%2C%22values%22%3A%5B%22test%22%2C%22test2%22%5D%7D%5D%7D%2C%22grouping%22%3A%7B%22dimension%22%3A%22blob%22%2C%22sort%22%3A2%2C%22top%22%3A10%7D%2C%22metrics%22%3A%5B%7B%22resourceMetadata%22%3A%7B%22id%22%3A%22%2Fsubscriptions%2F12345678-aaaa-bbbb-cccc-123456789abc%2FresourceGroups%2Fgrafanastaging%2Fproviders%2FMicrosoft.Compute%2FvirtualMachines%2Fgrafana%22%7D%2C%22name%22%3A%22Percentage%20CPU%22%2C%22aggregationType%22%3A4%2C%22namespace%22%3A%22Microsoft.Compute%2FvirtualMachines%22%2C%22metricVisualization%22%3A%7B%22displayName%22%3A%22Percentage%20CPU%22%2C%22resourceDisplayName%22%3A%22grafana%22%7D%7D%5D%7D%5D%7D"),
},
{
name: "Includes a region",
azureMonitorVariedProperties: map[string]interface{}{
"timeGrain": "PT1M",
"top": "10",
"region": "westus",
},
expectedInterval: "PT1M",
azureMonitorQueryTarget: "aggregation=Average&api-version=2021-05-01&interval=PT1M&metricnames=Percentage+CPU&metricnamespace=Microsoft.Compute%2FvirtualMachines&region=westus&timespan=2018-03-15T13%3A00%3A00Z%2F2018-03-15T13%3A34%3A00Z",
expectedURL: "/subscriptions/12345678-aaaa-bbbb-cccc-123456789abc/resourceGroups/grafanastaging/providers/Microsoft.Compute/virtualMachines/grafana/providers/microsoft.insights/metrics",
},
{
name: "Includes a region and a filter in the body with multiple resources",
azureMonitorVariedProperties: map[string]interface{}{
"timeGrain": "PT1M",
"top": "10",
"region": "westus",
"resources": []types.AzureMonitorResource{{ResourceGroup: "rg", ResourceName: "vm"}, {ResourceGroup: "rg2", ResourceName: "vm2"}},
},
expectedInterval: "PT1M",
azureMonitorQueryTarget: "aggregation=Average&api-version=2021-05-01&interval=PT1M&metricnames=Percentage+CPU&metricnamespace=Microsoft.Compute%2FvirtualMachines&region=westus&timespan=2018-03-15T13%3A00%3A00Z%2F2018-03-15T13%3A34%3A00Z",
expectedURL: "/subscriptions/12345678-aaaa-bbbb-cccc-123456789abc/providers/microsoft.insights/metrics",
expectedBodyFilter: "Microsoft.ResourceId eq '/subscriptions/12345678-aaaa-bbbb-cccc-123456789abc/resourceGroups/rg/providers/Microsoft.Compute/virtualMachines/vm' or Microsoft.ResourceId eq '/subscriptions/12345678-aaaa-bbbb-cccc-123456789abc/resourceGroups/rg2/providers/Microsoft.Compute/virtualMachines/vm2'",
},
{
name: "includes a single resource as a parameter filter",
azureMonitorVariedProperties: map[string]interface{}{
"timeGrain": "PT1M",
"resources": []types.AzureMonitorResource{{ResourceGroup: "rg", ResourceName: "vm"}},
},
queryInterval: duration,
expectedInterval: "PT1M",
azureMonitorQueryTarget: "aggregation=Average&api-version=2021-05-01&interval=PT1M&metricnames=Percentage+CPU&metricnamespace=Microsoft.Compute%2FvirtualMachines&timespan=2018-03-15T13%3A00%3A00Z%2F2018-03-15T13%3A34%3A00Z",
expectedURL: "/subscriptions/12345678-aaaa-bbbb-cccc-123456789abc/resourceGroups/rg/providers/Microsoft.Compute/virtualMachines/vm/providers/microsoft.insights/metrics",
},
{
name: "includes a resource and a dimesion as filters",
azureMonitorVariedProperties: map[string]interface{}{
"timeGrain": "PT1M",
"resources": []types.AzureMonitorResource{{ResourceGroup: "rg", ResourceName: "vm"}, {ResourceGroup: "rg2", ResourceName: "vm2"}},
"dimensionFilters": []types.AzureMonitorDimensionFilter{{Dimension: "blob", Operator: "ne", Filter: &wildcardFilter, Filters: []string{"test", "test2"}}},
"top": "30",
},
queryInterval: duration,
expectedInterval: "PT1M",
expectedURL: "/subscriptions/12345678-aaaa-bbbb-cccc-123456789abc/providers/microsoft.insights/metrics",
azureMonitorQueryTarget: "aggregation=Average&api-version=2021-05-01&interval=PT1M&metricnames=Percentage+CPU&metricnamespace=Microsoft.Compute%2FvirtualMachines&timespan=2018-03-15T13%3A00%3A00Z%2F2018-03-15T13%3A34%3A00Z&top=30",
expectedBodyFilter: "(Microsoft.ResourceId eq '/subscriptions/12345678-aaaa-bbbb-cccc-123456789abc/resourceGroups/rg/providers/Microsoft.Compute/virtualMachines/vm' or Microsoft.ResourceId eq '/subscriptions/12345678-aaaa-bbbb-cccc-123456789abc/resourceGroups/rg2/providers/Microsoft.Compute/virtualMachines/vm2') and (blob ne 'test' and blob ne 'test2')",
expectedPortalURL: util.Pointer("http://ds/#blade/Microsoft_Azure_MonitoringMetrics/Metrics.ReactView/Referer/MetricsExplorer/TimeContext/%7B%22absolute%22%3A%7B%22startTime%22%3A%222018-03-15T13%3A00%3A00Z%22%2C%22endTime%22%3A%222018-03-15T13%3A34%3A00Z%22%7D%7D/ChartDefinition/%7B%22v2charts%22%3A%5B%7B%22filterCollection%22%3A%7B%22filters%22%3A%5B%7B%22key%22%3A%22blob%22%2C%22operator%22%3A1%2C%22values%22%3A%5B%22test%22%2C%22test2%22%5D%7D%5D%7D%2C%22grouping%22%3A%7B%22dimension%22%3A%22blob%22%2C%22sort%22%3A2%2C%22top%22%3A10%7D%2C%22metrics%22%3A%5B%7B%22resourceMetadata%22%3A%7B%22id%22%3A%22%2Fsubscriptions%2F12345678-aaaa-bbbb-cccc-123456789abc%2FresourceGroups%2Fgrafanastaging%2Fproviders%2FMicrosoft.Compute%2FvirtualMachines%2Fgrafana%22%7D%2C%22name%22%3A%22Percentage%20CPU%22%2C%22aggregationType%22%3A4%2C%22namespace%22%3A%22Microsoft.Compute%2FvirtualMachines%22%2C%22metricVisualization%22%3A%7B%22displayName%22%3A%22Percentage%20CPU%22%2C%22resourceDisplayName%22%3A%22grafana%22%7D%7D%5D%7D%5D%7D"),
},
}
commonAzureModelProps := map[string]interface{}{
"aggregation": "Average",
"resourceGroup": "grafanastaging",
"resourceName": "grafana",
"metricNamespace": "Microsoft.Compute/virtualMachines",
"metricName": "Percentage CPU",
"alias": "testalias",
"queryType": "Azure Monitor",
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
for k, v := range commonAzureModelProps {
tt.azureMonitorVariedProperties[k] = v
}
azureMonitorJSON, _ := json.Marshal(tt.azureMonitorVariedProperties)
tsdbQuery := []backend.DataQuery{
{
JSON: []byte(fmt.Sprintf(`{
"subscription": "12345678-aaaa-bbbb-cccc-123456789abc",
"azureMonitor": %s
}`, string(azureMonitorJSON))),
RefID: "A",
Interval: tt.queryInterval,
TimeRange: backend.TimeRange{
From: fromStart,
To: fromStart.Add(34 * time.Minute),
},
},
}
queries, err := datasource.buildQueries(log.New("test"), tsdbQuery, dsInfo)
require.NoError(t, err)
azureMonitorQuery := &types.AzureMonitorQuery{
URL: tt.expectedURL,
Target: tt.azureMonitorQueryTarget,
RefID: "A",
Alias: "testalias",
TimeRange: backend.TimeRange{
From: fromStart,
To: fromStart.Add(34 * time.Minute),
},
BodyFilter: tt.expectedBodyFilter,
}
assert.Equal(t, tt.expectedParamFilter, queries[0].Params.Get("$filter"))
if azureMonitorQuery.URL == "" {
azureMonitorQuery.URL = "/subscriptions/12345678-aaaa-bbbb-cccc-123456789abc/resourceGroups/grafanastaging/providers/Microsoft.Compute/virtualMachines/grafana/providers/microsoft.insights/metrics"
}
if diff := cmp.Diff(azureMonitorQuery, queries[0], cmpopts.IgnoreUnexported(simplejson.Json{}), cmpopts.IgnoreFields(types.AzureMonitorQuery{}, "Params", "Dimensions")); diff != "" {
t.Errorf("Result mismatch (-want +got):\n%s", diff)
}
expectedPortalURL := `http://ds/#blade/Microsoft_Azure_MonitoringMetrics/Metrics.ReactView/Referer/MetricsExplorer/` +
`TimeContext/%7B%22absolute%22%3A%7B%22startTime%22%3A%222018-03-15T13%3A00%3A00Z%22%2C%22endTime%22%3A%222018-03-15T13%3A34%3A00Z%22%7D%7D/` +
`ChartDefinition/%7B%22v2charts%22%3A%5B%7B%22metrics%22%3A%5B%7B%22resourceMetadata%22%3A%7B%22id%22%3A%22%2Fsubscriptions%2F12345678-aaaa-bbbb-cccc-123456789abc%2FresourceGroups%2Fgrafanastaging%2Fproviders%2FMicrosoft.Compute%2FvirtualMachines%2Fgrafana%22%7D%2C` +
`%22name%22%3A%22Percentage%20CPU%22%2C%22aggregationType%22%3A4%2C%22namespace%22%3A%22Microsoft.Compute%2FvirtualMachines%22%2C%22metricVisualization%22%3A%7B%22displayName%22%3A%22Percentage%20CPU%22%2C%22resourceDisplayName%22%3A%22grafana%22%7D%7D%5D%7D%5D%7D`
if tt.expectedPortalURL != nil {
expectedPortalURL = *tt.expectedPortalURL
}
actual, err := getQueryUrl(queries[0], "http://ds", "/subscriptions/12345678-aaaa-bbbb-cccc-123456789abc/resourceGroups/grafanastaging/providers/Microsoft.Compute/virtualMachines/grafana", "grafana")
require.NoError(t, err)
require.Equal(t, expectedPortalURL, actual)
})
}
}
func TestCustomNamespace(t *testing.T) {
datasource := &AzureMonitorDatasource{}
t.Run("it should set the metricNamespace to a customNamespace value if customNamespace is present as a parameter", func(t *testing.T) {
q := []backend.DataQuery{
{
JSON: []byte(`{
"azureMonitor": {
"customNamespace": "custom/namespace"
}
}`),
},
}
result, err := datasource.buildQueries(log.New("test"), q, types.DatasourceInfo{})
require.NoError(t, err)
expected := "custom/namespace"
require.Equal(t, expected, result[0].Params.Get("metricnamespace"))
})
}
func TestAzureMonitorParseResponse(t *testing.T) {
tests := []struct {
name string
responseFile string
mockQuery *types.AzureMonitorQuery
expectedFrames data.Frames
queryIntervalMS int64
}{
{
name: "average aggregate time series response",
responseFile: "azuremonitor/1-azure-monitor-response-avg.json",
mockQuery: &types.AzureMonitorQuery{
URL: "/subscriptions/12345678-aaaa-bbbb-cccc-123456789abc/resourceGroups/grafanastaging/providers/Microsoft.Compute/virtualMachines/grafana/providers/microsoft.insights/metrics",
Params: url.Values{
"aggregation": {"Average"},
},
},
},
{
name: "total aggregate time series response",
responseFile: "azuremonitor/2-azure-monitor-response-total.json",
mockQuery: &types.AzureMonitorQuery{
URL: "/subscriptions/12345678-aaaa-bbbb-cccc-123456789abc/resourceGroups/grafanastaging/providers/Microsoft.Compute/virtualMachines/grafana/providers/microsoft.insights/metrics",
Params: url.Values{
"aggregation": {"Total"},
},
},
},
{
name: "maximum aggregate time series response",
responseFile: "azuremonitor/3-azure-monitor-response-maximum.json",
mockQuery: &types.AzureMonitorQuery{
URL: "/subscriptions/12345678-aaaa-bbbb-cccc-123456789abc/resourceGroups/grafanastaging/providers/Microsoft.Compute/virtualMachines/grafana/providers/microsoft.insights/metrics",
Params: url.Values{
"aggregation": {"Maximum"},
},
},
},
{
name: "minimum aggregate time series response",
responseFile: "azuremonitor/4-azure-monitor-response-minimum.json",
mockQuery: &types.AzureMonitorQuery{
URL: "/subscriptions/12345678-aaaa-bbbb-cccc-123456789abc/resourceGroups/grafanastaging/providers/Microsoft.Compute/virtualMachines/grafana/providers/microsoft.insights/metrics",
Params: url.Values{
"aggregation": {"Minimum"},
},
},
},
{
name: "count aggregate time series response",
responseFile: "azuremonitor/5-azure-monitor-response-count.json",
mockQuery: &types.AzureMonitorQuery{
URL: "/subscriptions/12345678-aaaa-bbbb-cccc-123456789abc/resourceGroups/grafanastaging/providers/Microsoft.Compute/virtualMachines/grafana/providers/microsoft.insights/metrics",
Params: url.Values{
"aggregation": {"Count"},
},
},
},
{
name: "single dimension time series response",
responseFile: "azuremonitor/6-azure-monitor-response-single-dimension.json",
mockQuery: &types.AzureMonitorQuery{
URL: "/subscriptions/12345678-aaaa-bbbb-cccc-123456789abc/resourceGroups/grafanastaging/providers/Microsoft.Compute/virtualMachines/grafana/providers/microsoft.insights/metrics",
Params: url.Values{
"aggregation": {"Average"},
},
},
},
{
name: "with alias patterns in the query",
responseFile: "azuremonitor/2-azure-monitor-response-total.json",
mockQuery: &types.AzureMonitorQuery{
URL: "/subscriptions/12345678-aaaa-bbbb-cccc-123456789abc/resourceGroups/grafanastaging/providers/Microsoft.Compute/virtualMachines/grafana/providers/microsoft.insights/metrics",
Alias: "custom {{resourcegroup}} {{namespace}} {{resourceName}} {{metric}}",
Params: url.Values{
"aggregation": {"Total"},
},
},
},
{
name: "single dimension with alias",
responseFile: "azuremonitor/6-azure-monitor-response-single-dimension.json",
mockQuery: &types.AzureMonitorQuery{
URL: "/subscriptions/12345678-aaaa-bbbb-cccc-123456789abc/resourceGroups/grafanastaging/providers/Microsoft.Compute/virtualMachines/grafana/providers/microsoft.insights/metrics",
Alias: "{{dimensionname}}={{DimensionValue}}",
Params: url.Values{
"aggregation": {"Average"},
},
},
},
{
name: "multiple dimension time series response with label alias",
responseFile: "azuremonitor/7-azure-monitor-response-multi-dimension.json",
mockQuery: &types.AzureMonitorQuery{
URL: "/subscriptions/12345678-aaaa-bbbb-cccc-123456789abc/resourceGroups/grafanastaging/providers/Microsoft.Compute/virtualMachines/grafana/providers/microsoft.insights/metrics",
Alias: "{{resourcegroup}} {Blob Type={{blobtype}}, Tier={{Tier}}}",
Params: url.Values{
"aggregation": {"Average"},
},
},
},
{
name: "unspecified unit with alias should not panic",
responseFile: "azuremonitor/8-azure-monitor-response-unspecified-unit.json",
mockQuery: &types.AzureMonitorQuery{
URL: "/subscriptions/12345678-aaaa-bbbb-cccc-123456789abc/resourceGroups/grafanastaging/providers/Microsoft.Compute/virtualMachines/grafana/providers/microsoft.insights/metrics",
Alias: "custom",
Params: url.Values{
"aggregation": {"Average"},
},
},
},
{
name: "with legacy azure monitor query properties and without a resource uri",
responseFile: "azuremonitor/2-azure-monitor-response-total.json",
mockQuery: &types.AzureMonitorQuery{
URL: "/subscriptions/12345678-aaaa-bbbb-cccc-123456789abc/resourceGroups/grafanastaging/providers/Microsoft.Compute/virtualMachines/grafana/providers/microsoft.insights/metrics",
Alias: "custom {{resourcegroup}} {{namespace}} {{resourceName}} {{metric}}",
Params: url.Values{
"aggregation": {"Total"},
},
},
},
{
name: "with legacy azure monitor query properties and with a resource uri it should use the resource uri",
responseFile: "azuremonitor/2-azure-monitor-response-total.json",
mockQuery: &types.AzureMonitorQuery{
URL: "/subscriptions/12345678-aaaa-bbbb-cccc-123456789abc/resourceGroups/grafanastaging/providers/Microsoft.Compute/virtualMachines/grafana/providers/microsoft.insights/metrics",
Alias: "custom {{resourcegroup}} {{namespace}} {{resourceName}} {{metric}}",
Params: url.Values{
"aggregation": {"Total"},
},
},
},
{
name: "multiple time series response",
responseFile: "azuremonitor/9-azure-monitor-response-multi.json",
mockQuery: &types.AzureMonitorQuery{
URL: "/subscriptions/12345678-aaaa-bbbb-cccc-123456789abc/providers/microsoft.insights/metrics",
Params: url.Values{
"aggregation": {"Average"},
},
},
},
{
name: "multiple time series response with multiple dimensions",
responseFile: "azuremonitor/10-azure-monitor-response-multi-with-dimensions.json",
mockQuery: &types.AzureMonitorQuery{
URL: "/subscriptions/12345678-aaaa-bbbb-cccc-123456789abc/providers/microsoft.insights/metrics",
Params: url.Values{
"aggregation": {"Average"},
},
},
},
}
datasource := &AzureMonitorDatasource{}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
azData := loadTestFile(t, tt.responseFile)
dframes, err := datasource.parseResponse(azData, tt.mockQuery, "http://ds")
require.NoError(t, err)
require.NotNil(t, dframes)
testdata.CheckGoldenFrames(t, "../testdata", fmt.Sprintf("%s.%s", tt.responseFile, strings.ReplaceAll(tt.name, " ", "-")), dframes)
})
}
}
func TestFindClosestAllowIntervalMS(t *testing.T) {
humanIntervalToMS := map[string]int64{
"3m": 180000,
"5m": 300000,
"10m": 600000,
"15m": 900000,
"1d": 86400000,
"2d": 172800000,
}
tests := []struct {
name string
allowedTimeGrains []int64 // Note: Uses defaults when empty list
inputInterval int64
expectedInterval int64
}{
{
name: "closest to 3m is 5m",
allowedTimeGrains: []int64{},
inputInterval: humanIntervalToMS["3m"],
expectedInterval: humanIntervalToMS["5m"],
},
{
name: "closest to 10m is 15m",
allowedTimeGrains: []int64{},
inputInterval: humanIntervalToMS["10m"],
expectedInterval: humanIntervalToMS["15m"],
},
{
name: "closest to 2d is 1d",
allowedTimeGrains: []int64{},
inputInterval: humanIntervalToMS["2d"],
expectedInterval: humanIntervalToMS["1d"],
},
{
name: "closest to 3m is 1d when 1d is only allowed interval",
allowedTimeGrains: []int64{humanIntervalToMS["1d"]},
inputInterval: humanIntervalToMS["2d"],
expectedInterval: humanIntervalToMS["1d"],
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
interval := azTime.FindClosestAllowedIntervalMS(tt.inputInterval, tt.allowedTimeGrains)
require.Equal(t, tt.expectedInterval, interval)
})
}
}
func loadTestFile(t *testing.T, name string) types.AzureMonitorResponse {
t.Helper()
path := filepath.Join("../testdata", name)
// Ignore gosec warning G304 since it's a test
// nolint:gosec
jsonBody, err := os.ReadFile(path)
require.NoError(t, err)
var azData types.AzureMonitorResponse
err = json.Unmarshal(jsonBody, &azData)
require.NoError(t, err)
return azData
}
func TestAzureMonitorCreateRequest(t *testing.T) {
ctx := context.Background()
url := "http://ds/"
tests := []struct {
name string
expectedURL string
expectedHeaders http.Header
Err require.ErrorAssertionFunc
}{
{
name: "creates a request",
expectedURL: "http://ds/",
expectedHeaders: http.Header{
"Content-Type": []string{"application/json"},
},
Err: require.NoError,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
ds := AzureMonitorDatasource{}
req, err := ds.createRequest(ctx, log.New("test"), url)
tt.Err(t, err)
if req.URL.String() != tt.expectedURL {
t.Errorf("Expecting %s, got %s", tt.expectedURL, req.URL.String())
}
if !cmp.Equal(req.Header, tt.expectedHeaders) {
t.Errorf("Unexpected HTTP headers: %v", cmp.Diff(req.Header, tt.expectedHeaders))
}
})
}
}
func TestExtractResourceNameFromMetricsURL(t *testing.T) {
t.Run("it should extract the resourceName from a well-formed Metrics URL", func(t *testing.T) {
url := "/subscriptions/12345678-aaaa-bbbb-cccc-123456789abc/resourceGroups/grafanastaging/providers/Microsoft.Compute/virtualMachines/Grafana-Test.VM/providers/microsoft.insights/metrics"
expected := "Grafana-Test.VM"
require.Equal(t, expected, extractResourceNameFromMetricsURL((url)))
})
t.Run("it should extract the resourceName from a well-formed Metrics URL in a case insensitive manner", func(t *testing.T) {
url := "/subscriptions/12345678-aaaa-bbbb-cccc-123456789abc/resourceGroups/grafanastaging/providers/Microsoft.Compute/virtualMachines/Grafana-Test.VM/pRoViDeRs/MiCrOsOfT.iNsIgHtS/mEtRiCs"
expected := "Grafana-Test.VM"
require.Equal(t, expected, extractResourceNameFromMetricsURL((url)))
})
t.Run("it should return an empty string if no match is found", func(t *testing.T) {
url := "/subscriptions/12345678-aaaa-bbbb-cccc-123456789abc/resourceGroups/grafanastaging/providers/Microsoft.Compute/virtualMachines/Grafana-Test.VM/providers/microsoft.insights/nope-this-part-does-not-match"
expected := ""
require.Equal(t, expected, extractResourceNameFromMetricsURL((url)))
})
}