grafana/pkg/tsdb/cloudwatch/metric_data_query_builder.go
Erik Sundell 254577ba56
CloudWatch: Cross-account querying support (#59362)
* Lattice: Point to private prerelease of aws-sdk-go (#515)

* point to private prerelease of aws-sdk-go

* fix build issue

* Lattice: Adding a feature toggle (#549)

* Adding a feature toggle for lattice

* Change name of feature toggle

* Lattice: List accounts (#543)

* Separate layers

* Introduce testify/mock library

Co-authored-by: Shirley Leu <4163034+fridgepoet@users.noreply.github.com>

* point to version that includes metric api changes (#574)

* add accounts component (#575)

* Test refactor: remove unneeded clientFactoryMock (#581)

* Lattice: Add monitoring badge (#576)

* add monitoring badge

* fix tests

* solve conflict

* Lattice: Add dynamic label for account display name (#579)

* Build: Automatically sync lattice-main with OSS

* Lattice: Point to private prerelease of aws-sdk-go (#515)

* point to private prerelease of aws-sdk-go

* fix build issue

* Lattice: Adding a feature toggle (#549)

* Adding a feature toggle for lattice

* Change name of feature toggle

* Lattice: List accounts (#543)

* Separate layers

* Introduce testify/mock library

Co-authored-by: Shirley Leu <4163034+fridgepoet@users.noreply.github.com>

* point to version that includes metric api changes (#574)

* add accounts component (#575)

* Test refactor: remove unneeded clientFactoryMock (#581)

* Lattice: Add monitoring badge (#576)

* add monitoring badge

* fix tests

* solve conflict

* add account label

Co-authored-by: Shirley Leu <4163034+fridgepoet@users.noreply.github.com>
Co-authored-by: Sarah Zinger <sarah.zinger@grafana.com>

* fix import

* solve merge related problem

* add account info (#608)

* add back namespaces handler

* Lattice: Parse account id and return it to frontend (#609)

* parse account id and return to frontend

* fix route test

* only show badge when feature toggle is enabled (#615)

* Lattice: Refactor resource response type and return account (#613)

* refactor resource response type

* remove not used file.

* go lint

* fix tests

* remove commented code

* Lattice: Use account as input when listing metric names and dimensions (#611)

* use account in resource requests

* add account to response

* revert accountInfo to accountId

* PR feedback

* unit test account in list metrics response

* remove not used asserts

* don't assert on response that is not relevant to the test

* removed dupe test

* pr feedback

* rename request package (#626)

* Lattice: Move account component and add tooltip (#630)

* move accounts component to the top of metric stat editor

* add tooltip

* CloudWatch: add account to GetMetricData queries (#627)

* Add AccountId to metric stat query

* Lattice: Account variable support  (#625)

* add variable support in accounts component

* add account variable query type

* update variables

* interpolate variable before its sent to backend

* handle variable change in hooks

* remove not used import

* Update public/app/plugins/datasource/cloudwatch/components/Account.tsx

Co-authored-by: Sarah Zinger <sarah.zinger@grafana.com>

* Update public/app/plugins/datasource/cloudwatch/hooks.ts

Co-authored-by: Sarah Zinger <sarah.zinger@grafana.com>

* add one more unit test

Co-authored-by: Sarah Zinger <sarah.zinger@grafana.com>

* cleanup (#629)

* Set account Id according to crossAccountQuerying feature flag in backend (#632)

* CloudWatch: Change spelling of feature-toggle (#634)

* Lattice Logs (#631)

* Lattice Logs

* Fixes after CR

* Lattice: Bug: fix dimension keys request (#644)

* fix dimension keys

* fix lint

* more lint

* CloudWatch: Add tests for QueryData with AccountId (#637)

* Update from breaking change (#645)

* Update from breaking change

* Remove extra interface and methods

Co-authored-by: Shirley Leu <4163034+fridgepoet@users.noreply.github.com>

* CloudWatch: Add business logic layer for getting log groups (#642)



Co-authored-by: Sarah Zinger <sarah.zinger@grafana.com>

* Lattice: Fix - unset account id in region change handler (#646)

* move reset of account to region change handler

* fix broken test

* Lattice: Add account id to metric stat query deep link (#656)

add account id to metric stat link

* CloudWatch: Add new log groups handler for cross-account querying (#643)

* Lattice: Add feature tracking (#660)

* add tracking for account id prescense in metrics query

* also check feature toggle

* fix broken test

* CloudWatch: Add route for DescribeLogGroups for cross-account querying (#647)

Co-authored-by: Erik Sundell <erik.sundell87@gmail.com>

* Lattice: Handle account id default value (#662)

* make sure right type is returned

* set right default values

* Suggestions to lattice changes (#663)

* Change ListMetricsWithPageLimit response to slice of non-pointers

* Change GetAccountsForCurrentUserOrRole response to be not pointer

* Clean test Cleanup calls in test

* Remove CloudWatchAPI as part of mock

* Resolve conflicts

* Add Latest SDK (#672)

* add tooltip (#674)

* Docs: Add documentation for CloudWatch cross account querying (#676)

* wip docs

* change wordings

* add sections about metrics and logs

* change from monitoring to observability

* Update docs/sources/datasources/aws-cloudwatch/_index.md

Co-authored-by: Sarah Zinger <sarah.zinger@grafana.com>

* Update docs/sources/datasources/aws-cloudwatch/query-editor/index.md

Co-authored-by: Fiona Artiaga <89225282+GrafanaWriter@users.noreply.github.com>

* Update docs/sources/datasources/aws-cloudwatch/query-editor/index.md

Co-authored-by: Fiona Artiaga <89225282+GrafanaWriter@users.noreply.github.com>

* Update docs/sources/datasources/aws-cloudwatch/query-editor/index.md

Co-authored-by: Sarah Zinger <sarah.zinger@grafana.com>

* Update docs/sources/datasources/aws-cloudwatch/query-editor/index.md

Co-authored-by: Fiona Artiaga <89225282+GrafanaWriter@users.noreply.github.com>

* apply pr feedback

* fix file name

* more pr feedback

* pr feedback

Co-authored-by: Sarah Zinger <sarah.zinger@grafana.com>
Co-authored-by: Fiona Artiaga <89225282+GrafanaWriter@users.noreply.github.com>

* use latest version of the aws-sdk-go

* Fix tests' mock response type

* Remove change in Azure Monitor

Co-authored-by: Sarah Zinger <sarah.zinger@grafana.com>
Co-authored-by: Shirley Leu <4163034+fridgepoet@users.noreply.github.com>
Co-authored-by: Fiona Artiaga <89225282+GrafanaWriter@users.noreply.github.com>
2022-11-28 12:39:12 +01:00

157 lines
4.6 KiB
Go

package cloudwatch
import (
"fmt"
"sort"
"strconv"
"strings"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/service/cloudwatch"
"github.com/grafana/grafana/pkg/infra/log"
"github.com/grafana/grafana/pkg/services/featuremgmt"
"github.com/grafana/grafana/pkg/tsdb/cloudwatch/models"
)
func (e *cloudWatchExecutor) buildMetricDataQuery(logger log.Logger, query *models.CloudWatchQuery) (*cloudwatch.MetricDataQuery, error) {
mdq := &cloudwatch.MetricDataQuery{
Id: aws.String(query.Id),
ReturnData: aws.Bool(query.ReturnData),
}
if e.features.IsEnabled(featuremgmt.FlagCloudWatchDynamicLabels) && len(query.Label) > 0 {
mdq.Label = &query.Label
}
switch query.GetGMDAPIMode(logger) {
case models.GMDApiModeMathExpression:
mdq.Period = aws.Int64(int64(query.Period))
mdq.Expression = aws.String(query.Expression)
case models.GMDApiModeSQLExpression:
mdq.Period = aws.Int64(int64(query.Period))
mdq.Expression = aws.String(query.SqlExpression)
case models.GMDApiModeInferredSearchExpression:
mdq.Expression = aws.String(buildSearchExpression(query, query.Statistic))
case models.GMDApiModeMetricStat:
mdq.MetricStat = &cloudwatch.MetricStat{
Metric: &cloudwatch.Metric{
Namespace: aws.String(query.Namespace),
MetricName: aws.String(query.MetricName),
Dimensions: make([]*cloudwatch.Dimension, 0),
},
Period: aws.Int64(int64(query.Period)),
}
for key, values := range query.Dimensions {
mdq.MetricStat.Metric.Dimensions = append(mdq.MetricStat.Metric.Dimensions,
&cloudwatch.Dimension{
Name: aws.String(key),
Value: aws.String(values[0]),
})
}
mdq.MetricStat.Stat = aws.String(query.Statistic)
mdq.AccountId = query.AccountId
}
if mdq.Expression != nil {
query.UsedExpression = *mdq.Expression
} else {
query.UsedExpression = ""
}
return mdq, nil
}
func buildSearchExpression(query *models.CloudWatchQuery, stat string) string {
knownDimensions := make(map[string][]string)
dimensionNames := []string{}
dimensionNamesWithoutKnownValues := []string{}
for key, values := range query.Dimensions {
dimensionNames = append(dimensionNames, key)
hasWildcard := false
for _, value := range values {
if value == "*" {
hasWildcard = true
break
}
}
if hasWildcard {
dimensionNamesWithoutKnownValues = append(dimensionNamesWithoutKnownValues, key)
} else {
knownDimensions[key] = values
}
}
searchTerm := fmt.Sprintf(`MetricName="%s"`, query.MetricName)
keys := []string{}
for k := range knownDimensions {
keys = append(keys, k)
}
sort.Strings(keys)
for _, key := range keys {
values := escapeDoubleQuotes(knownDimensions[key])
valueExpression := join(values, " OR ", `"`, `"`)
if len(knownDimensions[key]) > 1 {
valueExpression = fmt.Sprintf(`(%s)`, valueExpression)
}
keyFilter := fmt.Sprintf(`"%s"=%s`, key, valueExpression)
searchTerm = appendSearch(searchTerm, keyFilter)
}
var account string
if query.AccountId != nil && *query.AccountId != "all" {
account = fmt.Sprintf(":aws.AccountId=%q", *query.AccountId)
}
if query.MatchExact {
schema := fmt.Sprintf("%q", query.Namespace)
if len(dimensionNames) > 0 {
sort.Strings(dimensionNames)
schema += fmt.Sprintf(",%s", join(dimensionNames, ",", `"`, `"`))
}
schema = fmt.Sprintf("{%s}", schema)
schemaSearchTermAndAccount := strings.TrimSpace(strings.Join([]string{schema, searchTerm, account}, " "))
return fmt.Sprintf("REMOVE_EMPTY(SEARCH('%s', '%s', %s))", schemaSearchTermAndAccount, stat, strconv.Itoa(query.Period))
}
sort.Strings(dimensionNamesWithoutKnownValues)
searchTerm = appendSearch(searchTerm, join(dimensionNamesWithoutKnownValues, " ", `"`, `"`))
namespace := fmt.Sprintf("Namespace=%q", query.Namespace)
namespaceSearchTermAndAccount := strings.TrimSpace(strings.Join([]string{namespace, searchTerm, account}, " "))
return fmt.Sprintf(`REMOVE_EMPTY(SEARCH('%s', '%s', %s))`, namespaceSearchTermAndAccount, stat, strconv.Itoa(query.Period))
}
func escapeDoubleQuotes(arr []string) []string {
result := []string{}
for _, value := range arr {
value = strings.ReplaceAll(value, `"`, `\"`)
result = append(result, value)
}
return result
}
func join(arr []string, delimiter string, valuePrefix string, valueSuffix string) string {
result := ""
for index, value := range arr {
result += valuePrefix + value + valueSuffix
if index+1 != len(arr) {
result += delimiter
}
}
return result
}
func appendSearch(target string, value string) string {
if value != "" {
if target == "" {
return value
}
return fmt.Sprintf("%v %v", target, value)
}
return target
}