grafana/pkg/tsdb/cloudwatch/time_series_query.go
Erik Sundell 254577ba56
CloudWatch: Cross-account querying support (#59362)
* Lattice: Point to private prerelease of aws-sdk-go (#515)

* point to private prerelease of aws-sdk-go

* fix build issue

* Lattice: Adding a feature toggle (#549)

* Adding a feature toggle for lattice

* Change name of feature toggle

* Lattice: List accounts (#543)

* Separate layers

* Introduce testify/mock library

Co-authored-by: Shirley Leu <4163034+fridgepoet@users.noreply.github.com>

* point to version that includes metric api changes (#574)

* add accounts component (#575)

* Test refactor: remove unneeded clientFactoryMock (#581)

* Lattice: Add monitoring badge (#576)

* add monitoring badge

* fix tests

* solve conflict

* Lattice: Add dynamic label for account display name (#579)

* Build: Automatically sync lattice-main with OSS

* Lattice: Point to private prerelease of aws-sdk-go (#515)

* point to private prerelease of aws-sdk-go

* fix build issue

* Lattice: Adding a feature toggle (#549)

* Adding a feature toggle for lattice

* Change name of feature toggle

* Lattice: List accounts (#543)

* Separate layers

* Introduce testify/mock library

Co-authored-by: Shirley Leu <4163034+fridgepoet@users.noreply.github.com>

* point to version that includes metric api changes (#574)

* add accounts component (#575)

* Test refactor: remove unneeded clientFactoryMock (#581)

* Lattice: Add monitoring badge (#576)

* add monitoring badge

* fix tests

* solve conflict

* add account label

Co-authored-by: Shirley Leu <4163034+fridgepoet@users.noreply.github.com>
Co-authored-by: Sarah Zinger <sarah.zinger@grafana.com>

* fix import

* solve merge related problem

* add account info (#608)

* add back namespaces handler

* Lattice: Parse account id and return it to frontend (#609)

* parse account id and return to frontend

* fix route test

* only show badge when feature toggle is enabled (#615)

* Lattice: Refactor resource response type and return account (#613)

* refactor resource response type

* remove not used file.

* go lint

* fix tests

* remove commented code

* Lattice: Use account as input when listing metric names and dimensions (#611)

* use account in resource requests

* add account to response

* revert accountInfo to accountId

* PR feedback

* unit test account in list metrics response

* remove not used asserts

* don't assert on response that is not relevant to the test

* removed dupe test

* pr feedback

* rename request package (#626)

* Lattice: Move account component and add tooltip (#630)

* move accounts component to the top of metric stat editor

* add tooltip

* CloudWatch: add account to GetMetricData queries (#627)

* Add AccountId to metric stat query

* Lattice: Account variable support  (#625)

* add variable support in accounts component

* add account variable query type

* update variables

* interpolate variable before its sent to backend

* handle variable change in hooks

* remove not used import

* Update public/app/plugins/datasource/cloudwatch/components/Account.tsx

Co-authored-by: Sarah Zinger <sarah.zinger@grafana.com>

* Update public/app/plugins/datasource/cloudwatch/hooks.ts

Co-authored-by: Sarah Zinger <sarah.zinger@grafana.com>

* add one more unit test

Co-authored-by: Sarah Zinger <sarah.zinger@grafana.com>

* cleanup (#629)

* Set account Id according to crossAccountQuerying feature flag in backend (#632)

* CloudWatch: Change spelling of feature-toggle (#634)

* Lattice Logs (#631)

* Lattice Logs

* Fixes after CR

* Lattice: Bug: fix dimension keys request (#644)

* fix dimension keys

* fix lint

* more lint

* CloudWatch: Add tests for QueryData with AccountId (#637)

* Update from breaking change (#645)

* Update from breaking change

* Remove extra interface and methods

Co-authored-by: Shirley Leu <4163034+fridgepoet@users.noreply.github.com>

* CloudWatch: Add business logic layer for getting log groups (#642)



Co-authored-by: Sarah Zinger <sarah.zinger@grafana.com>

* Lattice: Fix - unset account id in region change handler (#646)

* move reset of account to region change handler

* fix broken test

* Lattice: Add account id to metric stat query deep link (#656)

add account id to metric stat link

* CloudWatch: Add new log groups handler for cross-account querying (#643)

* Lattice: Add feature tracking (#660)

* add tracking for account id prescense in metrics query

* also check feature toggle

* fix broken test

* CloudWatch: Add route for DescribeLogGroups for cross-account querying (#647)

Co-authored-by: Erik Sundell <erik.sundell87@gmail.com>

* Lattice: Handle account id default value (#662)

* make sure right type is returned

* set right default values

* Suggestions to lattice changes (#663)

* Change ListMetricsWithPageLimit response to slice of non-pointers

* Change GetAccountsForCurrentUserOrRole response to be not pointer

* Clean test Cleanup calls in test

* Remove CloudWatchAPI as part of mock

* Resolve conflicts

* Add Latest SDK (#672)

* add tooltip (#674)

* Docs: Add documentation for CloudWatch cross account querying (#676)

* wip docs

* change wordings

* add sections about metrics and logs

* change from monitoring to observability

* Update docs/sources/datasources/aws-cloudwatch/_index.md

Co-authored-by: Sarah Zinger <sarah.zinger@grafana.com>

* Update docs/sources/datasources/aws-cloudwatch/query-editor/index.md

Co-authored-by: Fiona Artiaga <89225282+GrafanaWriter@users.noreply.github.com>

* Update docs/sources/datasources/aws-cloudwatch/query-editor/index.md

Co-authored-by: Fiona Artiaga <89225282+GrafanaWriter@users.noreply.github.com>

* Update docs/sources/datasources/aws-cloudwatch/query-editor/index.md

Co-authored-by: Sarah Zinger <sarah.zinger@grafana.com>

* Update docs/sources/datasources/aws-cloudwatch/query-editor/index.md

Co-authored-by: Fiona Artiaga <89225282+GrafanaWriter@users.noreply.github.com>

* apply pr feedback

* fix file name

* more pr feedback

* pr feedback

Co-authored-by: Sarah Zinger <sarah.zinger@grafana.com>
Co-authored-by: Fiona Artiaga <89225282+GrafanaWriter@users.noreply.github.com>

* use latest version of the aws-sdk-go

* Fix tests' mock response type

* Remove change in Azure Monitor

Co-authored-by: Sarah Zinger <sarah.zinger@grafana.com>
Co-authored-by: Shirley Leu <4163034+fridgepoet@users.noreply.github.com>
Co-authored-by: Fiona Artiaga <89225282+GrafanaWriter@users.noreply.github.com>
2022-11-28 12:39:12 +01:00

116 lines
3.0 KiB
Go

package cloudwatch
import (
"context"
"fmt"
"github.com/grafana/grafana-plugin-sdk-go/backend"
"golang.org/x/sync/errgroup"
"github.com/grafana/grafana/pkg/infra/log"
"github.com/grafana/grafana/pkg/services/featuremgmt"
"github.com/grafana/grafana/pkg/tsdb/cloudwatch/models"
)
type responseWrapper struct {
DataResponse *backend.DataResponse
RefId string
}
func (e *cloudWatchExecutor) executeTimeSeriesQuery(ctx context.Context, logger log.Logger, req *backend.QueryDataRequest) (*backend.QueryDataResponse, error) {
logger.Debug("Executing time series query")
resp := backend.NewQueryDataResponse()
if len(req.Queries) == 0 {
return nil, fmt.Errorf("request contains no queries")
}
// startTime and endTime are always the same for all queries
startTime := req.Queries[0].TimeRange.From
endTime := req.Queries[0].TimeRange.To
if !startTime.Before(endTime) {
return nil, fmt.Errorf("invalid time range: start time must be before end time")
}
requestQueries, err := models.ParseMetricDataQueries(req.Queries, startTime, endTime,
e.features.IsEnabled(featuremgmt.FlagCloudWatchDynamicLabels),
e.features.IsEnabled(featuremgmt.FlagCloudWatchCrossAccountQuerying))
if err != nil {
return nil, err
}
if len(requestQueries) == 0 {
return backend.NewQueryDataResponse(), nil
}
requestQueriesByRegion := make(map[string][]*models.CloudWatchQuery)
for _, query := range requestQueries {
if _, exist := requestQueriesByRegion[query.Region]; !exist {
requestQueriesByRegion[query.Region] = []*models.CloudWatchQuery{}
}
requestQueriesByRegion[query.Region] = append(requestQueriesByRegion[query.Region], query)
}
resultChan := make(chan *responseWrapper, len(req.Queries))
eg, ectx := errgroup.WithContext(ctx)
for r, q := range requestQueriesByRegion {
requestQueries := q
region := r
eg.Go(func() error {
defer func() {
if err := recover(); err != nil {
logger.Error("Execute Get Metric Data Query Panic", "error", err, "stack", log.Stack(1))
if theErr, ok := err.(error); ok {
resultChan <- &responseWrapper{
DataResponse: &backend.DataResponse{
Error: theErr,
},
}
}
}
}()
client, err := e.getCWClient(req.PluginContext, region)
if err != nil {
return err
}
metricDataInput, err := e.buildMetricDataInput(logger, startTime, endTime, requestQueries)
if err != nil {
return err
}
mdo, err := e.executeRequest(ectx, client, metricDataInput)
if err != nil {
return err
}
res, err := e.parseResponse(startTime, endTime, mdo, requestQueries)
if err != nil {
return err
}
for _, responseWrapper := range res {
resultChan <- responseWrapper
}
return nil
})
}
if err := eg.Wait(); err != nil {
dataResponse := backend.DataResponse{
Error: fmt.Errorf("metric request error: %q", err),
}
resultChan <- &responseWrapper{
DataResponse: &dataResponse,
}
}
close(resultChan)
for result := range resultChan {
resp.Responses[result.RefId] = *result.DataResponse
}
return resp, nil
}