CloudWatch: Cross-account querying support (#59362)

* Lattice: Point to private prerelease of aws-sdk-go (#515)

* point to private prerelease of aws-sdk-go

* fix build issue

* Lattice: Adding a feature toggle (#549)

* Adding a feature toggle for lattice

* Change name of feature toggle

* Lattice: List accounts (#543)

* Separate layers

* Introduce testify/mock library

Co-authored-by: Shirley Leu <4163034+fridgepoet@users.noreply.github.com>

* point to version that includes metric api changes (#574)

* add accounts component (#575)

* Test refactor: remove unneeded clientFactoryMock (#581)

* Lattice: Add monitoring badge (#576)

* add monitoring badge

* fix tests

* solve conflict

* Lattice: Add dynamic label for account display name (#579)

* Build: Automatically sync lattice-main with OSS

* Lattice: Point to private prerelease of aws-sdk-go (#515)

* point to private prerelease of aws-sdk-go

* fix build issue

* Lattice: Adding a feature toggle (#549)

* Adding a feature toggle for lattice

* Change name of feature toggle

* Lattice: List accounts (#543)

* Separate layers

* Introduce testify/mock library

Co-authored-by: Shirley Leu <4163034+fridgepoet@users.noreply.github.com>

* point to version that includes metric api changes (#574)

* add accounts component (#575)

* Test refactor: remove unneeded clientFactoryMock (#581)

* Lattice: Add monitoring badge (#576)

* add monitoring badge

* fix tests

* solve conflict

* add account label

Co-authored-by: Shirley Leu <4163034+fridgepoet@users.noreply.github.com>
Co-authored-by: Sarah Zinger <sarah.zinger@grafana.com>

* fix import

* solve merge related problem

* add account info (#608)

* add back namespaces handler

* Lattice: Parse account id and return it to frontend (#609)

* parse account id and return to frontend

* fix route test

* only show badge when feature toggle is enabled (#615)

* Lattice: Refactor resource response type and return account (#613)

* refactor resource response type

* remove not used file.

* go lint

* fix tests

* remove commented code

* Lattice: Use account as input when listing metric names and dimensions (#611)

* use account in resource requests

* add account to response

* revert accountInfo to accountId

* PR feedback

* unit test account in list metrics response

* remove not used asserts

* don't assert on response that is not relevant to the test

* removed dupe test

* pr feedback

* rename request package (#626)

* Lattice: Move account component and add tooltip (#630)

* move accounts component to the top of metric stat editor

* add tooltip

* CloudWatch: add account to GetMetricData queries (#627)

* Add AccountId to metric stat query

* Lattice: Account variable support  (#625)

* add variable support in accounts component

* add account variable query type

* update variables

* interpolate variable before its sent to backend

* handle variable change in hooks

* remove not used import

* Update public/app/plugins/datasource/cloudwatch/components/Account.tsx

Co-authored-by: Sarah Zinger <sarah.zinger@grafana.com>

* Update public/app/plugins/datasource/cloudwatch/hooks.ts

Co-authored-by: Sarah Zinger <sarah.zinger@grafana.com>

* add one more unit test

Co-authored-by: Sarah Zinger <sarah.zinger@grafana.com>

* cleanup (#629)

* Set account Id according to crossAccountQuerying feature flag in backend (#632)

* CloudWatch: Change spelling of feature-toggle (#634)

* Lattice Logs (#631)

* Lattice Logs

* Fixes after CR

* Lattice: Bug: fix dimension keys request (#644)

* fix dimension keys

* fix lint

* more lint

* CloudWatch: Add tests for QueryData with AccountId (#637)

* Update from breaking change (#645)

* Update from breaking change

* Remove extra interface and methods

Co-authored-by: Shirley Leu <4163034+fridgepoet@users.noreply.github.com>

* CloudWatch: Add business logic layer for getting log groups (#642)



Co-authored-by: Sarah Zinger <sarah.zinger@grafana.com>

* Lattice: Fix - unset account id in region change handler (#646)

* move reset of account to region change handler

* fix broken test

* Lattice: Add account id to metric stat query deep link (#656)

add account id to metric stat link

* CloudWatch: Add new log groups handler for cross-account querying (#643)

* Lattice: Add feature tracking (#660)

* add tracking for account id prescense in metrics query

* also check feature toggle

* fix broken test

* CloudWatch: Add route for DescribeLogGroups for cross-account querying (#647)

Co-authored-by: Erik Sundell <erik.sundell87@gmail.com>

* Lattice: Handle account id default value (#662)

* make sure right type is returned

* set right default values

* Suggestions to lattice changes (#663)

* Change ListMetricsWithPageLimit response to slice of non-pointers

* Change GetAccountsForCurrentUserOrRole response to be not pointer

* Clean test Cleanup calls in test

* Remove CloudWatchAPI as part of mock

* Resolve conflicts

* Add Latest SDK (#672)

* add tooltip (#674)

* Docs: Add documentation for CloudWatch cross account querying (#676)

* wip docs

* change wordings

* add sections about metrics and logs

* change from monitoring to observability

* Update docs/sources/datasources/aws-cloudwatch/_index.md

Co-authored-by: Sarah Zinger <sarah.zinger@grafana.com>

* Update docs/sources/datasources/aws-cloudwatch/query-editor/index.md

Co-authored-by: Fiona Artiaga <89225282+GrafanaWriter@users.noreply.github.com>

* Update docs/sources/datasources/aws-cloudwatch/query-editor/index.md

Co-authored-by: Fiona Artiaga <89225282+GrafanaWriter@users.noreply.github.com>

* Update docs/sources/datasources/aws-cloudwatch/query-editor/index.md

Co-authored-by: Sarah Zinger <sarah.zinger@grafana.com>

* Update docs/sources/datasources/aws-cloudwatch/query-editor/index.md

Co-authored-by: Fiona Artiaga <89225282+GrafanaWriter@users.noreply.github.com>

* apply pr feedback

* fix file name

* more pr feedback

* pr feedback

Co-authored-by: Sarah Zinger <sarah.zinger@grafana.com>
Co-authored-by: Fiona Artiaga <89225282+GrafanaWriter@users.noreply.github.com>

* use latest version of the aws-sdk-go

* Fix tests' mock response type

* Remove change in Azure Monitor

Co-authored-by: Sarah Zinger <sarah.zinger@grafana.com>
Co-authored-by: Shirley Leu <4163034+fridgepoet@users.noreply.github.com>
Co-authored-by: Fiona Artiaga <89225282+GrafanaWriter@users.noreply.github.com>
This commit is contained in:
Erik Sundell
2022-11-28 12:39:12 +01:00
committed by GitHub
parent 5b861faec3
commit 254577ba56
100 changed files with 3945 additions and 475 deletions

View File

@@ -295,6 +295,11 @@ var (
Description: "Show the flame graph",
State: FeatureStateAlpha,
},
{
Name: "cloudWatchCrossAccountQuerying",
Description: "Use cross-account querying in CloudWatch datasource",
State: FeatureStateAlpha,
},
{
Name: "redshiftAsyncQueryDataSupport",
Description: "Enable async query data support for Redshift",

View File

@@ -215,6 +215,10 @@ const (
// Show the flame graph
FlagFlameGraph = "flameGraph"
// FlagCloudWatchCrossAccountQuerying
// Use cross-account querying in CloudWatch datasource
FlagCloudWatchCrossAccountQuerying = "cloudWatchCrossAccountQuerying"
// FlagRedshiftAsyncQueryDataSupport
// Enable async query data support for Redshift
FlagRedshiftAsyncQueryDataSupport = "redshiftAsyncQueryDataSupport"

View File

@@ -6,6 +6,7 @@ import (
"github.com/grafana/grafana/pkg/infra/metrics"
"github.com/grafana/grafana/pkg/setting"
"github.com/grafana/grafana/pkg/tsdb/cloudwatch/models"
"github.com/grafana/grafana/pkg/tsdb/cloudwatch/models/resources"
)
type metricsClient struct {
@@ -17,16 +18,20 @@ func NewMetricsClient(api models.CloudWatchMetricsAPIProvider, config *setting.C
return &metricsClient{CloudWatchMetricsAPIProvider: api, config: config}
}
func (l *metricsClient) ListMetricsWithPageLimit(params *cloudwatch.ListMetricsInput) ([]*cloudwatch.Metric, error) {
var cloudWatchMetrics []*cloudwatch.Metric
func (l *metricsClient) ListMetricsWithPageLimit(params *cloudwatch.ListMetricsInput) ([]resources.MetricResponse, error) {
var cloudWatchMetrics []resources.MetricResponse
pageNum := 0
err := l.ListMetricsPages(params, func(page *cloudwatch.ListMetricsOutput, lastPage bool) bool {
pageNum++
metrics.MAwsCloudWatchListMetrics.Inc()
metrics, err := awsutil.ValuesAtPath(page, "Metrics")
if err == nil {
for _, metric := range metrics {
cloudWatchMetrics = append(cloudWatchMetrics, metric.(*cloudwatch.Metric))
for idx, metric := range metrics {
metric := resources.MetricResponse{Metric: metric.(*cloudwatch.Metric)}
if len(page.OwningAccounts) >= idx && params.IncludeLinkedAccounts != nil && *params.IncludeLinkedAccounts {
metric.AccountId = page.OwningAccounts[idx]
}
cloudWatchMetrics = append(cloudWatchMetrics, metric)
}
}
return !lastPage && pageNum < l.config.AWSListMetricsPageLimit

View File

@@ -7,6 +7,7 @@ import (
"github.com/aws/aws-sdk-go/service/cloudwatch"
"github.com/grafana/grafana/pkg/setting"
"github.com/grafana/grafana/pkg/tsdb/cloudwatch/mocks"
"github.com/grafana/grafana/pkg/tsdb/cloudwatch/models/resources"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
@@ -46,4 +47,33 @@ func TestMetricsClient(t *testing.T) {
assert.Equal(t, len(metrics), len(response))
})
t.Run("Should return account id in case IncludeLinkedAccounts is set to true", func(t *testing.T) {
fakeApi := &mocks.FakeMetricsAPI{Metrics: []*cloudwatch.Metric{
{MetricName: aws.String("Test_MetricName1")},
{MetricName: aws.String("Test_MetricName2")},
{MetricName: aws.String("Test_MetricName3")},
}, OwningAccounts: []*string{aws.String("1234567890"), aws.String("1234567890"), aws.String("1234567895")}}
client := NewMetricsClient(fakeApi, &setting.Cfg{AWSListMetricsPageLimit: 100})
response, err := client.ListMetricsWithPageLimit(&cloudwatch.ListMetricsInput{IncludeLinkedAccounts: aws.Bool(true)})
require.NoError(t, err)
expected := []resources.MetricResponse{
{Metric: &cloudwatch.Metric{MetricName: aws.String("Test_MetricName1")}, AccountId: stringPtr("1234567890")},
{Metric: &cloudwatch.Metric{MetricName: aws.String("Test_MetricName2")}, AccountId: stringPtr("1234567890")},
{Metric: &cloudwatch.Metric{MetricName: aws.String("Test_MetricName3")}, AccountId: stringPtr("1234567895")},
}
assert.Equal(t, expected, response)
})
t.Run("Should not return account id in case IncludeLinkedAccounts is set to false", func(t *testing.T) {
fakeApi := &mocks.FakeMetricsAPI{Metrics: []*cloudwatch.Metric{{MetricName: aws.String("Test_MetricName1")}}, OwningAccounts: []*string{aws.String("1234567890")}}
client := NewMetricsClient(fakeApi, &setting.Cfg{AWSListMetricsPageLimit: 100})
response, err := client.ListMetricsWithPageLimit(&cloudwatch.ListMetricsInput{IncludeLinkedAccounts: aws.Bool(false)})
require.NoError(t, err)
assert.Nil(t, response[0].AccountId)
})
}
func stringPtr(s string) *string { return &s }

View File

@@ -5,7 +5,6 @@ import (
"encoding/json"
"fmt"
"net/http"
"net/url"
"regexp"
"time"
@@ -18,6 +17,7 @@ import (
"github.com/aws/aws-sdk-go/service/cloudwatchlogs/cloudwatchlogsiface"
"github.com/aws/aws-sdk-go/service/ec2"
"github.com/aws/aws-sdk-go/service/ec2/ec2iface"
"github.com/aws/aws-sdk-go/service/oam"
"github.com/aws/aws-sdk-go/service/resourcegroupstaggingapi"
"github.com/aws/aws-sdk-go/service/resourcegroupstaggingapi/resourcegroupstaggingapiiface"
"github.com/grafana/grafana-aws-sdk/pkg/awsds"
@@ -26,7 +26,6 @@ import (
"github.com/grafana/grafana-plugin-sdk-go/backend/instancemgmt"
"github.com/grafana/grafana-plugin-sdk-go/backend/resource/httpadapter"
"github.com/grafana/grafana-plugin-sdk-go/data"
"github.com/grafana/grafana/pkg/infra/httpclient"
"github.com/grafana/grafana/pkg/infra/log"
"github.com/grafana/grafana/pkg/services/featuremgmt"
@@ -122,8 +121,11 @@ func (e *cloudWatchExecutor) getRequestContext(pluginCtx backend.PluginContext,
return models.RequestContext{}, err
}
return models.RequestContext{
OAMClientProvider: NewOAMAPI(sess),
MetricsClientProvider: clients.NewMetricsClient(NewMetricsAPI(sess), e.cfg),
LogsAPIProvider: NewLogsAPI(sess),
Settings: instance.Settings,
Features: e.features,
}, nil
}
@@ -178,11 +180,12 @@ func (e *cloudWatchExecutor) checkHealthMetrics(pluginCtx backend.PluginContext)
}
func (e *cloudWatchExecutor) checkHealthLogs(pluginCtx backend.PluginContext) error {
parameters := url.Values{
"limit": []string{"1"},
session, err := e.newSession(pluginCtx, defaultRegion)
if err != nil {
return err
}
_, err := e.handleGetLogGroups(pluginCtx, parameters)
logsClient := NewLogsAPI(session)
_, err = logsClient.DescribeLogGroups(&cloudwatchlogs.DescribeLogGroupsInput{Limit: aws.Int64(1)})
return err
}
@@ -423,6 +426,20 @@ var NewMetricsAPI = func(sess *session.Session) models.CloudWatchMetricsAPIProvi
return cloudwatch.New(sess)
}
// NewLogsAPI is a CloudWatch logs api factory.
//
// Stubbable by tests.
var NewLogsAPI = func(sess *session.Session) models.CloudWatchLogsAPIProvider {
return cloudwatchlogs.New(sess)
}
// NewOAMAPI is a CloudWatch OAM api factory.
//
// Stubbable by tests.
var NewOAMAPI = func(sess *session.Session) models.OAMClientProvider {
return oam.New(sess)
}
// NewCWClient is a CloudWatch client factory.
//
// Stubbable by tests.

View File

@@ -24,7 +24,9 @@ import (
"github.com/grafana/grafana/pkg/tsdb/cloudwatch/mocks"
"github.com/grafana/grafana/pkg/tsdb/cloudwatch/models"
"github.com/grafana/grafana/pkg/tsdb/cloudwatch/models/resources"
"github.com/grafana/grafana/pkg/tsdb/cloudwatch/utils"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/mock"
"github.com/stretchr/testify/require"
)
@@ -97,16 +99,18 @@ func TestNewInstanceSettings(t *testing.T) {
func Test_CheckHealth(t *testing.T) {
origNewMetricsAPI := NewMetricsAPI
origNewCWLogsClient := NewCWLogsClient
origNewLogsAPI := NewLogsAPI
t.Cleanup(func() {
NewMetricsAPI = origNewMetricsAPI
NewCWLogsClient = origNewCWLogsClient
NewLogsAPI = origNewLogsAPI
})
var client fakeCheckHealthClient
NewMetricsAPI = func(sess *session.Session) models.CloudWatchMetricsAPIProvider {
return client
}
NewCWLogsClient = func(sess *session.Session) cloudwatchlogsiface.CloudWatchLogsAPI {
NewLogsAPI = func(sess *session.Session) models.CloudWatchLogsAPIProvider {
return client
}
@@ -536,16 +540,86 @@ func TestQuery_ResourceRequest_DescribeLogGroups(t *testing.T) {
})
}
func TestQuery_ResourceRequest_DescribeLogGroups_with_CrossAccountQuerying(t *testing.T) {
sender := &mockedCallResourceResponseSenderForOauth{}
origNewMetricsAPI := NewMetricsAPI
origNewOAMAPI := NewOAMAPI
origNewLogsAPI := NewLogsAPI
NewMetricsAPI = func(sess *session.Session) models.CloudWatchMetricsAPIProvider { return nil }
NewOAMAPI = func(sess *session.Session) models.OAMClientProvider { return nil }
t.Cleanup(func() {
NewOAMAPI = origNewOAMAPI
NewMetricsAPI = origNewMetricsAPI
NewLogsAPI = origNewLogsAPI
})
var logsApi mocks.LogsAPI
NewLogsAPI = func(sess *session.Session) models.CloudWatchLogsAPIProvider {
return &logsApi
}
im := datasource.NewInstanceManager(func(s backend.DataSourceInstanceSettings) (instancemgmt.Instance, error) {
return DataSource{Settings: models.CloudWatchSettings{}}, nil
})
t.Run("maps log group api response to resource response of describe-log-groups", func(t *testing.T) {
logsApi = mocks.LogsAPI{}
logsApi.On("DescribeLogGroups", mock.Anything).Return(&cloudwatchlogs.DescribeLogGroupsOutput{
LogGroups: []*cloudwatchlogs.LogGroup{
{Arn: aws.String("arn:aws:logs:us-east-1:111:log-group:group_a"), LogGroupName: aws.String("group_a")},
},
}, nil)
req := &backend.CallResourceRequest{
Method: "GET",
Path: `/describe-log-groups?logGroupPattern=some-pattern&accountId=some-account-id`,
PluginContext: backend.PluginContext{
DataSourceInstanceSettings: &backend.DataSourceInstanceSettings{ID: 0},
PluginID: "cloudwatch",
},
}
executor := newExecutor(im, newTestConfig(), &fakeSessionCache{}, featuremgmt.WithFeatures(featuremgmt.FlagCloudWatchCrossAccountQuerying))
err := executor.CallResource(context.Background(), req, sender)
assert.NoError(t, err)
assert.JSONEq(t, `[
{
"accountId":"111",
"value":{
"arn":"arn:aws:logs:us-east-1:111:log-group:group_a",
"name":"group_a"
}
}
]`, string(sender.Response.Body))
logsApi.AssertCalled(t, "DescribeLogGroups",
&cloudwatchlogs.DescribeLogGroupsInput{
AccountIdentifiers: []*string{utils.Pointer("some-account-id")},
IncludeLinkedAccounts: utils.Pointer(true),
Limit: utils.Pointer(int64(50)),
LogGroupNamePrefix: utils.Pointer("some-pattern"),
})
})
}
func Test_CloudWatch_CallResource_Integration_Test(t *testing.T) {
sender := &mockedCallResourceResponseSenderForOauth{}
origNewMetricsAPI := NewMetricsAPI
origNewOAMAPI := NewOAMAPI
origNewLogsAPI := NewLogsAPI
NewOAMAPI = func(sess *session.Session) models.OAMClientProvider { return nil }
NewLogsAPI = func(sess *session.Session) models.CloudWatchLogsAPIProvider { return nil }
t.Cleanup(func() {
NewOAMAPI = origNewOAMAPI
NewMetricsAPI = origNewMetricsAPI
NewLogsAPI = origNewLogsAPI
})
var api mocks.FakeMetricsAPI
NewMetricsAPI = func(sess *session.Session) models.CloudWatchMetricsAPIProvider {
return &api
}
im := datasource.NewInstanceManager(func(s backend.DataSourceInstanceSettings) (instancemgmt.Instance, error) {
return DataSource{Settings: models.CloudWatchSettings{}}, nil
})
@@ -580,10 +654,10 @@ func Test_CloudWatch_CallResource_Integration_Test(t *testing.T) {
sent := sender.Response
require.NotNil(t, sent)
require.Equal(t, http.StatusOK, sent.Status)
res := []string{}
res := []resources.ResourceResponse[string]{}
err = json.Unmarshal(sent.Body, &res)
require.Nil(t, err)
assert.Equal(t, []string{"Value1", "Value2", "Value7"}, res)
assert.Equal(t, []resources.ResourceResponse[string]{{Value: "Value1"}, {Value: "Value2"}, {Value: "Value7"}}, res)
})
t.Run("Should handle dimension key filter query and return keys from the api", func(t *testing.T) {
@@ -616,10 +690,10 @@ func Test_CloudWatch_CallResource_Integration_Test(t *testing.T) {
sent := sender.Response
require.NotNil(t, sent)
require.Equal(t, http.StatusOK, sent.Status)
res := []string{}
res := []resources.ResourceResponse[string]{}
err = json.Unmarshal(sent.Body, &res)
require.Nil(t, err)
assert.Equal(t, []string{"Test_DimensionName1", "Test_DimensionName2", "Test_DimensionName4", "Test_DimensionName5"}, res)
assert.Equal(t, []resources.ResourceResponse[string]{{Value: "Test_DimensionName1"}, {Value: "Test_DimensionName2"}, {Value: "Test_DimensionName4"}, {Value: "Test_DimensionName5"}}, res)
})
t.Run("Should handle standard dimension key query and return hard coded keys", func(t *testing.T) {
@@ -640,10 +714,10 @@ func Test_CloudWatch_CallResource_Integration_Test(t *testing.T) {
sent := sender.Response
require.NotNil(t, sent)
require.Equal(t, http.StatusOK, sent.Status)
res := []string{}
res := []resources.ResourceResponse[string]{}
err = json.Unmarshal(sent.Body, &res)
require.Nil(t, err)
assert.Equal(t, []string{"ClientId", "DomainName"}, res)
assert.Equal(t, []resources.ResourceResponse[string]{{Value: "ClientId"}, {Value: "DomainName"}}, res)
})
t.Run("Should handle custom namespace dimension key query and return hard coded keys", func(t *testing.T) {
@@ -664,10 +738,10 @@ func Test_CloudWatch_CallResource_Integration_Test(t *testing.T) {
sent := sender.Response
require.NotNil(t, sent)
require.Equal(t, http.StatusOK, sent.Status)
res := []string{}
res := []resources.ResourceResponse[string]{}
err = json.Unmarshal(sent.Body, &res)
require.Nil(t, err)
assert.Equal(t, []string{"ClientId", "DomainName"}, res)
assert.Equal(t, []resources.ResourceResponse[string]{{Value: "ClientId"}, {Value: "DomainName"}}, res)
})
t.Run("Should handle custom namespace metrics query and return metrics from api", func(t *testing.T) {
@@ -700,10 +774,10 @@ func Test_CloudWatch_CallResource_Integration_Test(t *testing.T) {
sent := sender.Response
require.NotNil(t, sent)
require.Equal(t, http.StatusOK, sent.Status)
res := []resources.Metric{}
res := []resources.ResourceResponse[resources.Metric]{}
err = json.Unmarshal(sent.Body, &res)
require.Nil(t, err)
assert.Equal(t, []resources.Metric{{Name: "Test_MetricName1", Namespace: "AWS/EC2"}, {Name: "Test_MetricName2", Namespace: "AWS/EC2"}, {Name: "Test_MetricName3", Namespace: "AWS/ECS"}, {Name: "Test_MetricName10", Namespace: "AWS/ECS"}, {Name: "Test_MetricName4", Namespace: "AWS/ECS"}, {Name: "Test_MetricName5", Namespace: "AWS/Redshift"}}, res)
assert.Equal(t, []resources.ResourceResponse[resources.Metric]{{Value: resources.Metric{Name: "Test_MetricName1", Namespace: "AWS/EC2"}}, {Value: resources.Metric{Name: "Test_MetricName2", Namespace: "AWS/EC2"}}, {Value: resources.Metric{Name: "Test_MetricName3", Namespace: "AWS/ECS"}}, {Value: resources.Metric{Name: "Test_MetricName10", Namespace: "AWS/ECS"}}, {Value: resources.Metric{Name: "Test_MetricName4", Namespace: "AWS/ECS"}}, {Value: resources.Metric{Name: "Test_MetricName5", Namespace: "AWS/Redshift"}}}, res)
})
}

View File

@@ -7,6 +7,7 @@ import (
"fmt"
"math"
"sort"
"strings"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/awserr"
@@ -17,6 +18,7 @@ import (
"golang.org/x/sync/errgroup"
"github.com/grafana/grafana/pkg/infra/log"
"github.com/grafana/grafana/pkg/services/featuremgmt"
)
const (
@@ -40,6 +42,7 @@ type LogQueryJson struct {
EndTime *int64
LogGroupName string
LogGroupNames []string
LogGroups []suggestData
LogGroupNamePrefix string
LogStreamName string
StartFromHead bool
@@ -224,15 +227,31 @@ func (e *cloudWatchExecutor) executeStartQuery(ctx context.Context, logsClient c
// StartTime is effectively floored while here EndTime is ceiled and so we should get the logs user wants
// and also a little bit more but as CW logs accept only seconds as integers there is not much to do about
// that.
EndTime: aws.Int64(int64(math.Ceil(float64(endTime.UnixNano()) / 1e9))),
LogGroupNames: aws.StringSlice(parameters.LogGroupNames),
QueryString: aws.String(modifiedQueryString),
EndTime: aws.Int64(int64(math.Ceil(float64(endTime.UnixNano()) / 1e9))),
QueryString: aws.String(modifiedQueryString),
}
if e.features.IsEnabled(featuremgmt.FlagCloudWatchCrossAccountQuerying) {
if parameters.LogGroups != nil && len(parameters.LogGroups) > 0 {
var logGroupIdentifiers []string
for _, lg := range parameters.LogGroups {
arn := lg.Value
// due to a bug in the startQuery api, we remove * from the arn, otherwise it throws an error
logGroupIdentifiers = append(logGroupIdentifiers, strings.TrimSuffix(arn, "*"))
}
startQueryInput.LogGroupIdentifiers = aws.StringSlice(logGroupIdentifiers)
}
}
if startQueryInput.LogGroupIdentifiers == nil {
startQueryInput.LogGroupNames = aws.StringSlice(parameters.LogGroupNames)
}
if parameters.Limit != nil {
startQueryInput.Limit = aws.Int64(*parameters.Limit)
}
logger.Debug("calling startquery with context with input", "input", startQueryInput)
return logsClient.StartQueryWithContext(ctx, startQueryInput)
}

View File

@@ -391,6 +391,78 @@ func Test_executeStartQuery(t *testing.T) {
require.Len(t, cli.calls.startQueryWithContext, 1)
assert.Nil(t, cli.calls.startQueryWithContext[0].Limit)
})
t.Run("attaches logGroupIdentifiers if the crossAccount feature is enabled", func(t *testing.T) {
cli = fakeCWLogsClient{}
im := datasource.NewInstanceManager(func(s backend.DataSourceInstanceSettings) (instancemgmt.Instance, error) {
return DataSource{Settings: models.CloudWatchSettings{}}, nil
})
executor := newExecutor(im, newTestConfig(), &fakeSessionCache{}, featuremgmt.WithFeatures(featuremgmt.FlagCloudWatchCrossAccountQuerying))
_, err := executor.QueryData(context.Background(), &backend.QueryDataRequest{
PluginContext: backend.PluginContext{DataSourceInstanceSettings: &backend.DataSourceInstanceSettings{}},
Queries: []backend.DataQuery{
{
RefID: "A",
TimeRange: backend.TimeRange{From: time.Unix(0, 0), To: time.Unix(1, 0)},
JSON: json.RawMessage(`{
"type": "logAction",
"subtype": "StartQuery",
"limit": 12,
"queryString":"fields @message",
"logGroups":[{"value": "fakeARN"}]
}`),
},
},
})
assert.NoError(t, err)
assert.Equal(t, []*cloudwatchlogs.StartQueryInput{
{
StartTime: aws.Int64(0),
EndTime: aws.Int64(1),
Limit: aws.Int64(12),
QueryString: aws.String("fields @timestamp,ltrim(@log) as __log__grafana_internal__,ltrim(@logStream) as __logstream__grafana_internal__|fields @message"),
LogGroupIdentifiers: []*string{aws.String("fakeARN")},
},
}, cli.calls.startQueryWithContext)
})
t.Run("attaches logGroupIdentifiers if the crossAccount feature is enabled and strips out trailing *", func(t *testing.T) {
cli = fakeCWLogsClient{}
im := datasource.NewInstanceManager(func(s backend.DataSourceInstanceSettings) (instancemgmt.Instance, error) {
return DataSource{Settings: models.CloudWatchSettings{}}, nil
})
executor := newExecutor(im, newTestConfig(), &fakeSessionCache{}, featuremgmt.WithFeatures(featuremgmt.FlagCloudWatchCrossAccountQuerying))
_, err := executor.QueryData(context.Background(), &backend.QueryDataRequest{
PluginContext: backend.PluginContext{DataSourceInstanceSettings: &backend.DataSourceInstanceSettings{}},
Queries: []backend.DataQuery{
{
RefID: "A",
TimeRange: backend.TimeRange{From: time.Unix(0, 0), To: time.Unix(1, 0)},
JSON: json.RawMessage(`{
"type": "logAction",
"subtype": "StartQuery",
"limit": 12,
"queryString":"fields @message",
"logGroups":[{"value": "*fake**ARN*"}]
}`),
},
},
})
assert.NoError(t, err)
assert.Equal(t, []*cloudwatchlogs.StartQueryInput{
{
StartTime: aws.Int64(0),
EndTime: aws.Int64(1),
Limit: aws.Int64(12),
QueryString: aws.String("fields @timestamp,ltrim(@log) as __log__grafana_internal__,ltrim(@logStream) as __logstream__grafana_internal__|fields @message"),
LogGroupIdentifiers: []*string{aws.String("*fake**ARN")},
},
}, cli.calls.startQueryWithContext)
})
}
func TestQuery_StopQuery(t *testing.T) {

View File

@@ -50,6 +50,7 @@ func (e *cloudWatchExecutor) buildMetricDataQuery(logger log.Logger, query *mode
})
}
mdq.MetricStat.Stat = aws.String(query.Statistic)
mdq.AccountId = query.AccountId
}
if mdq.Expression != nil {
@@ -98,18 +99,27 @@ func buildSearchExpression(query *models.CloudWatchQuery, stat string) string {
searchTerm = appendSearch(searchTerm, keyFilter)
}
var account string
if query.AccountId != nil && *query.AccountId != "all" {
account = fmt.Sprintf(":aws.AccountId=%q", *query.AccountId)
}
if query.MatchExact {
schema := fmt.Sprintf("%q", query.Namespace)
if len(dimensionNames) > 0 {
sort.Strings(dimensionNames)
schema += fmt.Sprintf(",%s", join(dimensionNames, ",", `"`, `"`))
}
return fmt.Sprintf("REMOVE_EMPTY(SEARCH('{%s} %s', '%s', %s))", schema, searchTerm, stat, strconv.Itoa(query.Period))
schema = fmt.Sprintf("{%s}", schema)
schemaSearchTermAndAccount := strings.TrimSpace(strings.Join([]string{schema, searchTerm, account}, " "))
return fmt.Sprintf("REMOVE_EMPTY(SEARCH('%s', '%s', %s))", schemaSearchTermAndAccount, stat, strconv.Itoa(query.Period))
}
sort.Strings(dimensionNamesWithoutKnownValues)
searchTerm = appendSearch(searchTerm, join(dimensionNamesWithoutKnownValues, " ", `"`, `"`))
return fmt.Sprintf(`REMOVE_EMPTY(SEARCH('Namespace="%s" %s', '%s', %s))`, query.Namespace, searchTerm, stat, strconv.Itoa(query.Period))
namespace := fmt.Sprintf("Namespace=%q", query.Namespace)
namespaceSearchTermAndAccount := strings.TrimSpace(strings.Join([]string{namespace, searchTerm, account}, " "))
return fmt.Sprintf(`REMOVE_EMPTY(SEARCH('%s', '%s', %s))`, namespaceSearchTermAndAccount, stat, strconv.Itoa(query.Period))
}
func escapeDoubleQuotes(arr []string) []string {

View File

@@ -3,11 +3,11 @@ package cloudwatch
import (
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/aws/aws-sdk-go/aws"
"github.com/grafana/grafana/pkg/services/featuremgmt"
"github.com/grafana/grafana/pkg/tsdb/cloudwatch/models"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestMetricDataQueryBuilder(t *testing.T) {
@@ -24,6 +24,27 @@ func TestMetricDataQueryBuilder(t *testing.T) {
assert.Equal(t, query.Namespace, *mdq.MetricStat.Metric.Namespace)
})
t.Run("should pass AccountId in metric stat query", func(t *testing.T) {
executor := newExecutor(nil, newTestConfig(), &fakeSessionCache{}, featuremgmt.WithFeatures())
query := getBaseQuery()
query.MetricEditorMode = models.MetricEditorModeBuilder
query.MetricQueryType = models.MetricQueryTypeSearch
query.AccountId = aws.String("some account id")
mdq, err := executor.buildMetricDataQuery(logger, query)
require.NoError(t, err)
assert.Equal(t, "some account id", *mdq.AccountId)
})
t.Run("should leave AccountId in metric stat query", func(t *testing.T) {
executor := newExecutor(nil, newTestConfig(), &fakeSessionCache{}, featuremgmt.WithFeatures())
query := getBaseQuery()
query.MetricEditorMode = models.MetricEditorModeBuilder
query.MetricQueryType = models.MetricQueryTypeSearch
mdq, err := executor.buildMetricDataQuery(logger, query)
require.NoError(t, err)
assert.Nil(t, mdq.AccountId)
})
t.Run("should use custom built expression", func(t *testing.T) {
executor := newExecutor(nil, newTestConfig(), &fakeSessionCache{}, featuremgmt.WithFeatures())
query := getBaseQuery()
@@ -112,6 +133,42 @@ func TestMetricDataQueryBuilder(t *testing.T) {
assert.Nil(t, mdq.Label)
})
}
t.Run(`should not specify accountId when it is "all"`, func(t *testing.T) {
executor := newExecutor(nil, newTestConfig(), &fakeSessionCache{}, featuremgmt.WithFeatures(featuremgmt.FlagCloudWatchDynamicLabels))
query := &models.CloudWatchQuery{
Namespace: "AWS/EC2",
MetricName: "CPUUtilization",
Statistic: "Average",
Period: 60,
MatchExact: false,
AccountId: aws.String("all"),
}
mdq, err := executor.buildMetricDataQuery(logger, query)
assert.NoError(t, err)
require.Nil(t, mdq.MetricStat)
assert.Equal(t, `REMOVE_EMPTY(SEARCH('Namespace="AWS/EC2" MetricName="CPUUtilization"', 'Average', 60))`, *mdq.Expression)
})
t.Run("should set accountId when it is specified", func(t *testing.T) {
executor := newExecutor(nil, newTestConfig(), &fakeSessionCache{}, featuremgmt.WithFeatures(featuremgmt.FlagCloudWatchDynamicLabels))
query := &models.CloudWatchQuery{
Namespace: "AWS/EC2",
MetricName: "CPUUtilization",
Statistic: "Average",
Period: 60,
MatchExact: false,
AccountId: aws.String("12345"),
}
mdq, err := executor.buildMetricDataQuery(logger, query)
assert.NoError(t, err)
require.Nil(t, mdq.MetricStat)
assert.Equal(t, `REMOVE_EMPTY(SEARCH('Namespace="AWS/EC2" MetricName="CPUUtilization" :aws.AccountId="12345"', 'Average', 60))`, *mdq.Expression)
})
})
t.Run("Query should be matched exact", func(t *testing.T) {
@@ -199,6 +256,24 @@ func TestMetricDataQueryBuilder(t *testing.T) {
assert.Equal(t, `REMOVE_EMPTY(SEARCH('{"AWS/EC2","InstanceId","LoadBalancer"} MetricName="CPUUtilization" "LoadBalancer"=("lb1" OR "lb2" OR "lb3")', 'Average', 300))`, res)
})
t.Run("Query has multiple dimensions and an account Id", func(t *testing.T) {
query := &models.CloudWatchQuery{
Namespace: "AWS/EC2",
MetricName: "CPUUtilization",
Dimensions: map[string][]string{
"LoadBalancer": {"lb1", "lb2", "lb3"},
"InstanceId": {"i-123", "*", "i-789"},
},
Period: 300,
Expression: "",
MatchExact: matchExact,
AccountId: aws.String("some account id"),
}
res := buildSearchExpression(query, "Average")
assert.Equal(t, `REMOVE_EMPTY(SEARCH('{"AWS/EC2","InstanceId","LoadBalancer"} MetricName="CPUUtilization" "LoadBalancer"=("lb1" OR "lb2" OR "lb3") :aws.AccountId="some account id"', 'Average', 300))`, res)
})
t.Run("Query has a dimension key with a space", func(t *testing.T) {
query := &models.CloudWatchQuery{
Namespace: "AWS/Kafka",
@@ -301,6 +376,24 @@ func TestMetricDataQueryBuilder(t *testing.T) {
res := buildSearchExpression(query, "Average")
assert.Equal(t, `REMOVE_EMPTY(SEARCH('Namespace="AWS/EC2" MetricName="CPUUtilization" "LoadBalancer"=("lb1" OR "lb2" OR "lb3") "InstanceId"', 'Average', 300))`, res)
})
t.Run("query has multiple dimensions and an account Id", func(t *testing.T) {
query := &models.CloudWatchQuery{
Namespace: "AWS/EC2",
MetricName: "CPUUtilization",
Dimensions: map[string][]string{
"LoadBalancer": {"lb1", "lb2", "lb3"},
"InstanceId": {"i-123", "*", "i-789"},
},
Period: 300,
Expression: "",
MatchExact: matchExact,
AccountId: aws.String("some account id"),
}
res := buildSearchExpression(query, "Average")
assert.Equal(t, `REMOVE_EMPTY(SEARCH('Namespace="AWS/EC2" MetricName="CPUUtilization" "LoadBalancer"=("lb1" OR "lb2" OR "lb3") "InstanceId" :aws.AccountId="some account id"', 'Average', 300))`, res)
})
})
t.Run("Query has invalid characters in dimension values", func(t *testing.T) {

View File

@@ -17,7 +17,6 @@ import (
"github.com/aws/aws-sdk-go/service/ec2"
"github.com/aws/aws-sdk-go/service/resourcegroupstaggingapi"
"github.com/grafana/grafana-plugin-sdk-go/backend"
"github.com/grafana/grafana/pkg/tsdb/cloudwatch/constants"
)
@@ -316,7 +315,6 @@ func (e *cloudWatchExecutor) handleGetLogGroups(pluginCtx backend.PluginContext,
if err != nil || response == nil {
return nil, err
}
result := make([]suggestData, 0)
for _, logGroup := range response.LogGroups {
logGroupName := *logGroup.LogGroupName

View File

@@ -0,0 +1,16 @@
package mocks
import (
"github.com/grafana/grafana/pkg/tsdb/cloudwatch/models/resources"
"github.com/stretchr/testify/mock"
)
type AccountsServiceMock struct {
mock.Mock
}
func (a *AccountsServiceMock) GetAccountsForCurrentUserOrRole() ([]resources.ResourceResponse[resources.Account], error) {
args := a.Called()
return args.Get(0).([]resources.ResourceResponse[resources.Account]), args.Error(1)
}

View File

@@ -9,9 +9,8 @@ import (
)
type FakeMetricsAPI struct {
cloudwatchiface.CloudWatchAPI
Metrics []*cloudwatch.Metric
OwningAccounts []*string
MetricsPerPage int
}
@@ -23,7 +22,8 @@ func (c *FakeMetricsAPI) ListMetricsPages(input *cloudwatch.ListMetricsInput, fn
for i, metrics := range chunks {
response := fn(&cloudwatch.ListMetricsOutput{
Metrics: metrics,
Metrics: metrics,
OwningAccounts: c.OwningAccounts,
}, i+1 == len(chunks))
if !response {
break

View File

@@ -9,26 +9,20 @@ type ListMetricsServiceMock struct {
mock.Mock
}
func (a *ListMetricsServiceMock) GetDimensionKeysByDimensionFilter(r resources.DimensionKeysRequest) ([]string, error) {
func (a *ListMetricsServiceMock) GetDimensionKeysByDimensionFilter(r resources.DimensionKeysRequest) ([]resources.ResourceResponse[string], error) {
args := a.Called(r)
return args.Get(0).([]string), args.Error(1)
return args.Get(0).([]resources.ResourceResponse[string]), args.Error(1)
}
func (a *ListMetricsServiceMock) GetDimensionValuesByDimensionFilter(r resources.DimensionValuesRequest) ([]string, error) {
func (a *ListMetricsServiceMock) GetDimensionValuesByDimensionFilter(r resources.DimensionValuesRequest) ([]resources.ResourceResponse[string], error) {
args := a.Called(r)
return args.Get(0).([]string), args.Error(1)
return args.Get(0).([]resources.ResourceResponse[string]), args.Error(1)
}
func (a *ListMetricsServiceMock) GetDimensionKeysByNamespace(namespace string) ([]string, error) {
args := a.Called(namespace)
func (a *ListMetricsServiceMock) GetMetricsByNamespace(r resources.MetricsRequest) ([]resources.ResourceResponse[resources.Metric], error) {
args := a.Called(r)
return args.Get(0).([]string), args.Error(1)
}
func (a *ListMetricsServiceMock) GetMetricsByNamespace(namespace string) ([]resources.Metric, error) {
args := a.Called(namespace)
return args.Get(0).([]resources.Metric), args.Error(1)
return args.Get(0).([]resources.ResourceResponse[resources.Metric]), args.Error(1)
}

View File

@@ -0,0 +1,37 @@
package mocks
import (
"github.com/aws/aws-sdk-go/service/cloudwatchlogs"
"github.com/grafana/grafana/pkg/tsdb/cloudwatch/models/resources"
"github.com/stretchr/testify/mock"
)
type LogsAPI struct {
mock.Mock
}
func (l *LogsAPI) DescribeLogGroups(input *cloudwatchlogs.DescribeLogGroupsInput) (*cloudwatchlogs.DescribeLogGroupsOutput, error) {
args := l.Called(input)
return args.Get(0).(*cloudwatchlogs.DescribeLogGroupsOutput), args.Error(1)
}
type LogsService struct {
mock.Mock
}
func (l *LogsService) GetLogGroups(request resources.LogGroupsRequest) ([]resources.ResourceResponse[resources.LogGroup], error) {
args := l.Called(request)
return args.Get(0).([]resources.ResourceResponse[resources.LogGroup]), args.Error(1)
}
type MockFeatures struct {
mock.Mock
}
func (f *MockFeatures) IsEnabled(feature string) bool {
args := f.Called(feature)
return args.Bool(0)
}

View File

@@ -2,6 +2,7 @@ package mocks
import (
"github.com/aws/aws-sdk-go/service/cloudwatch"
"github.com/grafana/grafana/pkg/tsdb/cloudwatch/models/resources"
"github.com/stretchr/testify/mock"
)
@@ -9,7 +10,7 @@ type FakeMetricsClient struct {
mock.Mock
}
func (m *FakeMetricsClient) ListMetricsWithPageLimit(params *cloudwatch.ListMetricsInput) ([]*cloudwatch.Metric, error) {
func (m *FakeMetricsClient) ListMetricsWithPageLimit(params *cloudwatch.ListMetricsInput) ([]resources.MetricResponse, error) {
args := m.Called(params)
return args.Get(0).([]*cloudwatch.Metric), args.Error(1)
return args.Get(0).([]resources.MetricResponse), args.Error(1)
}

View File

@@ -0,0 +1,20 @@
package mocks
import (
"github.com/aws/aws-sdk-go/service/oam"
"github.com/stretchr/testify/mock"
)
type FakeOAMClient struct {
mock.Mock
}
func (o *FakeOAMClient) ListSinks(input *oam.ListSinksInput) (*oam.ListSinksOutput, error) {
args := o.Called(input)
return args.Get(0).(*oam.ListSinksOutput), args.Error(1)
}
func (o *FakeOAMClient) ListAttachedLinks(input *oam.ListAttachedLinksInput) (*oam.ListAttachedLinksOutput, error) {
args := o.Called(input)
return args.Get(0).(*oam.ListAttachedLinksOutput), args.Error(1)
}

View File

@@ -1,21 +1,55 @@
package models
import (
"net/url"
"github.com/aws/aws-sdk-go/service/cloudwatch"
"github.com/aws/aws-sdk-go/service/cloudwatchlogs"
"github.com/aws/aws-sdk-go/service/oam"
"github.com/grafana/grafana-plugin-sdk-go/backend"
"github.com/grafana/grafana/pkg/services/featuremgmt"
"github.com/grafana/grafana/pkg/tsdb/cloudwatch/models/resources"
)
type RequestContextFactoryFunc func(pluginCtx backend.PluginContext, region string) (reqCtx RequestContext, err error)
type RouteHandlerFunc func(pluginCtx backend.PluginContext, reqContextFactory RequestContextFactoryFunc, parameters url.Values) ([]byte, *HttpError)
type RequestContext struct {
MetricsClientProvider MetricsClientProvider
LogsAPIProvider CloudWatchLogsAPIProvider
OAMClientProvider OAMClientProvider
Settings CloudWatchSettings
Features featuremgmt.FeatureToggles
}
type ListMetricsProvider interface {
GetDimensionKeysByDimensionFilter(resources.DimensionKeysRequest) ([]string, error)
GetDimensionKeysByNamespace(string) ([]string, error)
GetDimensionValuesByDimensionFilter(resources.DimensionValuesRequest) ([]string, error)
GetMetricsByNamespace(namespace string) ([]resources.Metric, error)
GetDimensionKeysByDimensionFilter(resources.DimensionKeysRequest) ([]resources.ResourceResponse[string], error)
GetDimensionValuesByDimensionFilter(resources.DimensionValuesRequest) ([]resources.ResourceResponse[string], error)
GetMetricsByNamespace(r resources.MetricsRequest) ([]resources.ResourceResponse[resources.Metric], error)
}
type MetricsClientProvider interface {
ListMetricsWithPageLimit(params *cloudwatch.ListMetricsInput) ([]*cloudwatch.Metric, error)
ListMetricsWithPageLimit(params *cloudwatch.ListMetricsInput) ([]resources.MetricResponse, error)
}
type CloudWatchMetricsAPIProvider interface {
ListMetricsPages(*cloudwatch.ListMetricsInput, func(*cloudwatch.ListMetricsOutput, bool) bool) error
}
type CloudWatchLogsAPIProvider interface {
DescribeLogGroups(*cloudwatchlogs.DescribeLogGroupsInput) (*cloudwatchlogs.DescribeLogGroupsOutput, error)
}
type OAMClientProvider interface {
ListSinks(*oam.ListSinksInput) (*oam.ListSinksOutput, error)
ListAttachedLinks(*oam.ListAttachedLinksInput) (*oam.ListAttachedLinksOutput, error)
}
type LogGroupsProvider interface {
GetLogGroups(request resources.LogGroupsRequest) ([]resources.ResourceResponse[resources.LogGroup], error)
}
type AccountsProvider interface {
GetAccountsForCurrentUserOrRole() ([]resources.ResourceResponse[resources.Account], error)
}

View File

@@ -60,6 +60,7 @@ type CloudWatchQuery struct {
TimezoneUTCOffset string
MetricQueryType MetricQueryType
MetricEditorMode MetricEditorMode
AccountId *string
}
func (q *CloudWatchQuery) GetGMDAPIMode(logger log.Logger) GMDApiMode {
@@ -95,6 +96,10 @@ func (q *CloudWatchQuery) IsInferredSearchExpression() bool {
return false
}
if q.AccountId != nil && *q.AccountId == "all" {
return true
}
if len(q.Dimensions) == 0 {
return !q.MatchExact
}
@@ -167,6 +172,9 @@ func (q *CloudWatchQuery) BuildDeepLink(startTime time.Time, endTime time.Time,
if dynamicLabelEnabled {
metricStatMeta.Label = q.Label
}
if q.AccountId != nil {
metricStatMeta.AccountId = *q.AccountId
}
metricStat = append(metricStat, metricStatMeta)
link.Metrics = []interface{}{metricStat}
}
@@ -214,11 +222,13 @@ type metricsDataQuery struct {
QueryType string `json:"type"`
Hide *bool `json:"hide"`
Alias string `json:"alias"`
AccountId *string `json:"accountId"`
}
// ParseMetricDataQueries decodes the metric data queries json, validates, sets default values and returns an array of CloudWatchQueries.
// The CloudWatchQuery has a 1 to 1 mapping to a query editor row
func ParseMetricDataQueries(dataQueries []backend.DataQuery, startTime time.Time, endTime time.Time, dynamicLabelsEnabled bool) ([]*CloudWatchQuery, error) {
func ParseMetricDataQueries(dataQueries []backend.DataQuery, startTime time.Time, endTime time.Time, dynamicLabelsEnabled,
crossAccountQueryingEnabled bool) ([]*CloudWatchQuery, error) {
var metricDataQueries = make(map[string]metricsDataQuery)
for _, query := range dataQueries {
var metricsDataQuery metricsDataQuery
@@ -250,7 +260,7 @@ func ParseMetricDataQueries(dataQueries []backend.DataQuery, startTime time.Time
Expression: mdq.Expression,
}
if err := cwQuery.validateAndSetDefaults(refId, mdq, startTime, endTime); err != nil {
if err := cwQuery.validateAndSetDefaults(refId, mdq, startTime, endTime, crossAccountQueryingEnabled); err != nil {
return nil, &QueryError{Err: err, RefID: refId}
}
@@ -267,7 +277,8 @@ func (q *CloudWatchQuery) migrateLegacyQuery(query metricsDataQuery, dynamicLabe
q.Label = getLabel(query, dynamicLabelsEnabled)
}
func (q *CloudWatchQuery) validateAndSetDefaults(refId string, metricsDataQuery metricsDataQuery, startTime, endTime time.Time) error {
func (q *CloudWatchQuery) validateAndSetDefaults(refId string, metricsDataQuery metricsDataQuery, startTime, endTime time.Time,
crossAccountQueryingEnabled bool) error {
if metricsDataQuery.Statistic == nil && metricsDataQuery.Statistics == nil {
return fmt.Errorf("query must have either statistic or statistics field")
}
@@ -283,6 +294,10 @@ func (q *CloudWatchQuery) validateAndSetDefaults(refId string, metricsDataQuery
return fmt.Errorf("failed to parse dimensions: %v", err)
}
if crossAccountQueryingEnabled {
q.AccountId = metricsDataQuery.AccountId
}
if metricsDataQuery.Id == "" {
// Why not just use refId if id is not specified in the frontend? When specifying an id in the editor,
// and alphabetical must be used. The id must be unique, so if an id like for example a, b or c would be used,

View File

@@ -107,6 +107,53 @@ func TestCloudWatchQuery(t *testing.T) {
require.NoError(t, err)
assert.NotContains(t, deepLink, "label")
})
t.Run("includes account id in case its a metric stat query and an account id is set", func(t *testing.T) {
startTime := time.Now()
endTime := startTime.Add(2 * time.Hour)
query := &CloudWatchQuery{
RefId: "A",
Region: "us-east-1",
Expression: "",
Statistic: "Average",
Period: 300,
Id: "id1",
MatchExact: true,
AccountId: pointer("123456789"),
Label: "${PROP('Namespace')}",
Dimensions: map[string][]string{
"InstanceId": {"i-12345678"},
},
MetricQueryType: MetricQueryTypeSearch,
MetricEditorMode: MetricEditorModeBuilder,
}
deepLink, err := query.BuildDeepLink(startTime, endTime, false)
require.NoError(t, err)
assert.Contains(t, deepLink, "accountId%22%3A%22123456789")
})
t.Run("does not include account id in case its not a metric stat query", func(t *testing.T) {
startTime := time.Now()
endTime := startTime.Add(2 * time.Hour)
query := &CloudWatchQuery{
RefId: "A",
Region: "us-east-1",
Statistic: "Average",
Expression: "SEARCH(someexpression)",
AccountId: pointer("123456789"),
Period: 300,
Id: "id1",
MatchExact: true,
Label: "${PROP('Namespace')}",
MetricQueryType: MetricQueryTypeSearch,
MetricEditorMode: MetricEditorModeRaw,
}
deepLink, err := query.BuildDeepLink(startTime, endTime, false)
require.NoError(t, err)
assert.NotContains(t, deepLink, "accountId%22%3A%22123456789")
})
})
t.Run("SEARCH(someexpression) was specified in the query editor", func(t *testing.T) {
@@ -269,7 +316,7 @@ func TestRequestParser(t *testing.T) {
},
}
migratedQueries, err := ParseMetricDataQueries(oldQuery, time.Now(), time.Now(), false)
migratedQueries, err := ParseMetricDataQueries(oldQuery, time.Now(), time.Now(), false, false)
assert.NoError(t, err)
require.Len(t, migratedQueries, 1)
require.NotNil(t, migratedQueries[0])
@@ -300,7 +347,7 @@ func TestRequestParser(t *testing.T) {
},
}
results, err := ParseMetricDataQueries(query, time.Now().Add(-2*time.Hour), time.Now().Add(-time.Hour), false)
results, err := ParseMetricDataQueries(query, time.Now().Add(-2*time.Hour), time.Now().Add(-time.Hour), false, false)
require.NoError(t, err)
require.Len(t, results, 1)
res := results[0]
@@ -343,7 +390,7 @@ func TestRequestParser(t *testing.T) {
},
}
results, err := ParseMetricDataQueries(query, time.Now().Add(-2*time.Hour), time.Now().Add(-time.Hour), false)
results, err := ParseMetricDataQueries(query, time.Now().Add(-2*time.Hour), time.Now().Add(-time.Hour), false, false)
assert.NoError(t, err)
require.Len(t, results, 1)
res := results[0]
@@ -376,7 +423,7 @@ func TestRequestParser(t *testing.T) {
},
}
_, err := ParseMetricDataQueries(query, time.Now().Add(-2*time.Hour), time.Now().Add(-time.Hour), false)
_, err := ParseMetricDataQueries(query, time.Now().Add(-2*time.Hour), time.Now().Add(-time.Hour), false, false)
require.Error(t, err)
assert.Equal(t, `error parsing query "", failed to parse dimensions: unknown type as dimension value`, err.Error())
@@ -405,7 +452,7 @@ func Test_ParseMetricDataQueries_periods(t *testing.T) {
},
}
res, err := ParseMetricDataQueries(query, time.Now().Add(-2*time.Hour), time.Now().Add(-time.Hour), false)
res, err := ParseMetricDataQueries(query, time.Now().Add(-2*time.Hour), time.Now().Add(-time.Hour), false, false)
assert.NoError(t, err)
require.Len(t, res, 1)
require.NotNil(t, res[0])
@@ -437,7 +484,7 @@ func Test_ParseMetricDataQueries_periods(t *testing.T) {
to := time.Now()
from := to.Local().Add(time.Minute * time.Duration(5))
res, err := ParseMetricDataQueries(query, from, to, false)
res, err := ParseMetricDataQueries(query, from, to, false, false)
require.NoError(t, err)
require.Len(t, res, 1)
assert.Equal(t, 60, res[0].Period)
@@ -447,7 +494,7 @@ func Test_ParseMetricDataQueries_periods(t *testing.T) {
to := time.Now()
from := to.AddDate(0, 0, -1)
res, err := ParseMetricDataQueries(query, from, to, false)
res, err := ParseMetricDataQueries(query, from, to, false, false)
require.NoError(t, err)
require.Len(t, res, 1)
assert.Equal(t, 60, res[0].Period)
@@ -456,7 +503,7 @@ func Test_ParseMetricDataQueries_periods(t *testing.T) {
t.Run("Time range is 2 days", func(t *testing.T) {
to := time.Now()
from := to.AddDate(0, 0, -2)
res, err := ParseMetricDataQueries(query, from, to, false)
res, err := ParseMetricDataQueries(query, from, to, false, false)
require.NoError(t, err)
require.Len(t, res, 1)
assert.Equal(t, 300, res[0].Period)
@@ -466,7 +513,7 @@ func Test_ParseMetricDataQueries_periods(t *testing.T) {
to := time.Now()
from := to.AddDate(0, 0, -7)
res, err := ParseMetricDataQueries(query, from, to, false)
res, err := ParseMetricDataQueries(query, from, to, false, false)
require.NoError(t, err)
require.Len(t, res, 1)
assert.Equal(t, 900, res[0].Period)
@@ -476,7 +523,7 @@ func Test_ParseMetricDataQueries_periods(t *testing.T) {
to := time.Now()
from := to.AddDate(0, 0, -30)
res, err := ParseMetricDataQueries(query, from, to, false)
res, err := ParseMetricDataQueries(query, from, to, false, false)
require.NoError(t, err)
require.Len(t, res, 1)
assert.Equal(t, 3600, res[0].Period)
@@ -486,7 +533,7 @@ func Test_ParseMetricDataQueries_periods(t *testing.T) {
to := time.Now()
from := to.AddDate(0, 0, -90)
res, err := ParseMetricDataQueries(query, from, to, false)
res, err := ParseMetricDataQueries(query, from, to, false, false)
require.NoError(t, err)
require.Len(t, res, 1)
assert.Equal(t, 21600, res[0].Period)
@@ -496,7 +543,7 @@ func Test_ParseMetricDataQueries_periods(t *testing.T) {
to := time.Now()
from := to.AddDate(-1, 0, 0)
res, err := ParseMetricDataQueries(query, from, to, false)
res, err := ParseMetricDataQueries(query, from, to, false, false)
require.Nil(t, err)
require.Len(t, res, 1)
assert.Equal(t, 21600, res[0].Period)
@@ -506,7 +553,7 @@ func Test_ParseMetricDataQueries_periods(t *testing.T) {
to := time.Now()
from := to.AddDate(-2, 0, 0)
res, err := ParseMetricDataQueries(query, from, to, false)
res, err := ParseMetricDataQueries(query, from, to, false, false)
require.NoError(t, err)
require.Len(t, res, 1)
assert.Equal(t, 86400, res[0].Period)
@@ -515,7 +562,7 @@ func Test_ParseMetricDataQueries_periods(t *testing.T) {
t.Run("Time range is 2 days, but 16 days ago", func(t *testing.T) {
to := time.Now().AddDate(0, 0, -14)
from := to.AddDate(0, 0, -2)
res, err := ParseMetricDataQueries(query, from, to, false)
res, err := ParseMetricDataQueries(query, from, to, false, false)
require.NoError(t, err)
require.Len(t, res, 1)
assert.Equal(t, 300, res[0].Period)
@@ -524,7 +571,7 @@ func Test_ParseMetricDataQueries_periods(t *testing.T) {
t.Run("Time range is 2 days, but 90 days ago", func(t *testing.T) {
to := time.Now().AddDate(0, 0, -88)
from := to.AddDate(0, 0, -2)
res, err := ParseMetricDataQueries(query, from, to, false)
res, err := ParseMetricDataQueries(query, from, to, false, false)
require.NoError(t, err)
require.Len(t, res, 1)
assert.Equal(t, 3600, res[0].Period)
@@ -533,7 +580,7 @@ func Test_ParseMetricDataQueries_periods(t *testing.T) {
t.Run("Time range is 2 days, but 456 days ago", func(t *testing.T) {
to := time.Now().AddDate(0, 0, -454)
from := to.AddDate(0, 0, -2)
res, err := ParseMetricDataQueries(query, from, to, false)
res, err := ParseMetricDataQueries(query, from, to, false, false)
require.NoError(t, err)
require.Len(t, res, 1)
assert.Equal(t, 21600, res[0].Period)
@@ -548,7 +595,7 @@ func Test_ParseMetricDataQueries_periods(t *testing.T) {
}`),
},
}
_, err := ParseMetricDataQueries(query, time.Now().Add(-2*time.Hour), time.Now().Add(-time.Hour), false)
_, err := ParseMetricDataQueries(query, time.Now().Add(-2*time.Hour), time.Now().Add(-time.Hour), false, false)
require.Error(t, err)
assert.Equal(t, `error parsing query "", failed to parse period as duration: time: invalid duration "invalid"`, err.Error())
})
@@ -563,7 +610,7 @@ func Test_ParseMetricDataQueries_periods(t *testing.T) {
},
}
res, err := ParseMetricDataQueries(query, time.Now().Add(-2*time.Hour), time.Now().Add(-time.Hour), false)
res, err := ParseMetricDataQueries(query, time.Now().Add(-2*time.Hour), time.Now().Add(-time.Hour), false, false)
assert.NoError(t, err)
require.Len(t, res, 1)
@@ -620,6 +667,18 @@ func Test_ParseMetricDataQueries_query_type_and_metric_editor_mode_and_GMD_query
expectedMetricEditorMode: dummyTestEditorMode,
expectedGMDApiMode: GMDApiModeMetricStat,
},
"no dimensions, matchExact is false": {
extraDataQueryJson: `"matchExact":false,`,
expectedMetricQueryType: MetricQueryTypeSearch,
expectedMetricEditorMode: MetricEditorModeBuilder,
expectedGMDApiMode: GMDApiModeInferredSearchExpression,
},
"query metricQueryType": {
extraDataQueryJson: `"metricQueryType":1,`,
expectedMetricQueryType: MetricQueryTypeQuery,
expectedMetricEditorMode: MetricEditorModeBuilder,
expectedGMDApiMode: GMDApiModeSQLExpression,
},
}
for name, tc := range testCases {
t.Run(name, func(t *testing.T) {
@@ -638,7 +697,7 @@ func Test_ParseMetricDataQueries_query_type_and_metric_editor_mode_and_GMD_query
),
},
}
res, err := ParseMetricDataQueries(query, time.Now(), time.Now(), false)
res, err := ParseMetricDataQueries(query, time.Now(), time.Now(), false, false)
require.NoError(t, err)
require.Len(t, res, 1)
require.NotNil(t, res[0])
@@ -664,7 +723,7 @@ func Test_ParseMetricDataQueries_hide_and_ReturnData(t *testing.T) {
}`),
},
}
res, err := ParseMetricDataQueries(query, time.Now().Add(-2*time.Hour), time.Now().Add(-time.Hour), false)
res, err := ParseMetricDataQueries(query, time.Now().Add(-2*time.Hour), time.Now().Add(-time.Hour), false, false)
require.NoError(t, err)
require.Len(t, res, 1)
require.NotNil(t, res[0])
@@ -685,7 +744,7 @@ func Test_ParseMetricDataQueries_hide_and_ReturnData(t *testing.T) {
}`),
},
}
res, err := ParseMetricDataQueries(query, time.Now().Add(-2*time.Hour), time.Now().Add(-time.Hour), false)
res, err := ParseMetricDataQueries(query, time.Now().Add(-2*time.Hour), time.Now().Add(-time.Hour), false, false)
require.NoError(t, err)
require.Len(t, res, 1)
require.NotNil(t, res[0])
@@ -706,7 +765,7 @@ func Test_ParseMetricDataQueries_hide_and_ReturnData(t *testing.T) {
}`),
},
}
res, err := ParseMetricDataQueries(query, time.Now().Add(-2*time.Hour), time.Now().Add(-time.Hour), false)
res, err := ParseMetricDataQueries(query, time.Now().Add(-2*time.Hour), time.Now().Add(-time.Hour), false, false)
require.NoError(t, err)
require.Len(t, res, 1)
require.NotNil(t, res[0])
@@ -725,7 +784,7 @@ func Test_ParseMetricDataQueries_hide_and_ReturnData(t *testing.T) {
}`),
},
}
res, err := ParseMetricDataQueries(query, time.Now().Add(-2*time.Hour), time.Now().Add(-time.Hour), false)
res, err := ParseMetricDataQueries(query, time.Now().Add(-2*time.Hour), time.Now().Add(-time.Hour), false, false)
require.NoError(t, err)
require.Len(t, res, 1)
require.NotNil(t, res[0])
@@ -746,7 +805,7 @@ func Test_ParseMetricDataQueries_hide_and_ReturnData(t *testing.T) {
}`),
},
}
res, err := ParseMetricDataQueries(query, time.Now().Add(-2*time.Hour), time.Now().Add(-time.Hour), false)
res, err := ParseMetricDataQueries(query, time.Now().Add(-2*time.Hour), time.Now().Add(-time.Hour), false, false)
require.NoError(t, err)
require.Len(t, res, 1)
require.NotNil(t, res[0])
@@ -767,7 +826,7 @@ func Test_ParseMetricDataQueries_hide_and_ReturnData(t *testing.T) {
}`),
},
}
res, err := ParseMetricDataQueries(query, time.Now().Add(-2*time.Hour), time.Now().Add(-time.Hour), false)
res, err := ParseMetricDataQueries(query, time.Now().Add(-2*time.Hour), time.Now().Add(-time.Hour), false, false)
require.NoError(t, err)
require.Len(t, res, 1)
require.NotNil(t, res[0])
@@ -790,7 +849,7 @@ func Test_ParseMetricDataQueries_ID(t *testing.T) {
}`),
},
}
res, err := ParseMetricDataQueries(query, time.Now().Add(-2*time.Hour), time.Now().Add(-time.Hour), false)
res, err := ParseMetricDataQueries(query, time.Now().Add(-2*time.Hour), time.Now().Add(-time.Hour), false, false)
require.NoError(t, err)
require.Len(t, res, 1)
require.NotNil(t, res[0])
@@ -811,7 +870,7 @@ func Test_ParseMetricDataQueries_ID(t *testing.T) {
}`),
},
}
res, err := ParseMetricDataQueries(query, time.Now().Add(-2*time.Hour), time.Now().Add(-time.Hour), false)
res, err := ParseMetricDataQueries(query, time.Now().Add(-2*time.Hour), time.Now().Add(-time.Hour), false, false)
require.NoError(t, err)
require.Len(t, res, 1)
require.NotNil(t, res[0])
@@ -838,7 +897,7 @@ func Test_ParseMetricDataQueries_sets_label_when_label_is_present_in_json_query(
},
}
res, err := ParseMetricDataQueries(query, time.Now(), time.Now(), true)
res, err := ParseMetricDataQueries(query, time.Now(), time.Now(), true, false)
assert.NoError(t, err)
require.Len(t, res, 1)
require.NotNil(t, res[0])
@@ -902,7 +961,7 @@ func Test_ParseMetricDataQueries_migrate_alias_to_label(t *testing.T) {
},
}
res, err := ParseMetricDataQueries(query, time.Now(), time.Now(), true)
res, err := ParseMetricDataQueries(query, time.Now(), time.Now(), true, false)
assert.NoError(t, err)
require.Len(t, res, 1)
@@ -949,7 +1008,7 @@ func Test_ParseMetricDataQueries_migrate_alias_to_label(t *testing.T) {
},
}
res, err := ParseMetricDataQueries(query, time.Now(), time.Now(), true)
res, err := ParseMetricDataQueries(query, time.Now(), time.Now(), true, false)
assert.NoError(t, err)
require.Len(t, res, 2)
@@ -1019,7 +1078,7 @@ func Test_ParseMetricDataQueries_migrate_alias_to_label(t *testing.T) {
}`, tc.labelJson)),
},
}
res, err := ParseMetricDataQueries(query, time.Now(), time.Now(), tc.dynamicLabelsFeatureToggleEnabled)
res, err := ParseMetricDataQueries(query, time.Now(), time.Now(), tc.dynamicLabelsFeatureToggleEnabled, false)
assert.NoError(t, err)
require.Len(t, res, 1)
@@ -1046,7 +1105,7 @@ func Test_ParseMetricDataQueries_statistics_and_query_type_validation_and_MatchE
{
JSON: []byte("{}"),
},
}, time.Now(), time.Now(), false)
}, time.Now(), time.Now(), false, false)
assert.Error(t, err)
assert.Equal(t, `error parsing query "", query must have either statistic or statistics field`, err.Error())
@@ -1059,7 +1118,7 @@ func Test_ParseMetricDataQueries_statistics_and_query_type_validation_and_MatchE
{
JSON: []byte(`{"type":"some other type", "statistic":"Average", "matchExact":false}`),
},
}, time.Now(), time.Now(), false)
}, time.Now(), time.Now(), false, false)
assert.NoError(t, err)
assert.Empty(t, actual)
@@ -1071,7 +1130,7 @@ func Test_ParseMetricDataQueries_statistics_and_query_type_validation_and_MatchE
{
JSON: []byte(`{"statistic":"Average"}`),
},
}, time.Now(), time.Now(), false)
}, time.Now(), time.Now(), false, false)
assert.NoError(t, err)
assert.NotEmpty(t, actual)
@@ -1083,7 +1142,7 @@ func Test_ParseMetricDataQueries_statistics_and_query_type_validation_and_MatchE
{
JSON: []byte(`{"statistic":"Average"}`),
},
}, time.Now(), time.Now(), false)
}, time.Now(), time.Now(), false, false)
assert.NoError(t, err)
assert.Len(t, actual, 1)
@@ -1097,7 +1156,7 @@ func Test_ParseMetricDataQueries_statistics_and_query_type_validation_and_MatchE
{
JSON: []byte(`{"statistic":"Average","matchExact":false}`),
},
}, time.Now(), time.Now(), false)
}, time.Now(), time.Now(), false, false)
assert.NoError(t, err)
assert.Len(t, actual, 1)
@@ -1105,3 +1164,36 @@ func Test_ParseMetricDataQueries_statistics_and_query_type_validation_and_MatchE
assert.False(t, actual[0].MatchExact)
})
}
func Test_ParseMetricDataQueries_account_Id(t *testing.T) {
t.Run("account is set when cross account querying enabled", func(t *testing.T) {
actual, err := ParseMetricDataQueries(
[]backend.DataQuery{
{
JSON: []byte(`{"accountId":"some account id", "statistic":"Average"}`),
},
}, time.Now(), time.Now(), false, true)
assert.NoError(t, err)
require.Len(t, actual, 1)
require.NotNil(t, actual[0])
require.NotNil(t, actual[0].AccountId)
assert.Equal(t, "some account id", *actual[0].AccountId)
})
t.Run("account is not set when cross account querying disabled", func(t *testing.T) {
actual, err := ParseMetricDataQueries(
[]backend.DataQuery{
{
JSON: []byte(`{"accountId":"some account id", "statistic":"Average"}`),
},
}, time.Now(), time.Now(), false, false)
assert.NoError(t, err)
require.Len(t, actual, 1)
require.NotNil(t, actual[0])
assert.Nil(t, actual[0].AccountId)
})
}
func pointer[T any](arg T) *T { return &arg }

View File

@@ -0,0 +1,53 @@
package resources
import (
"fmt"
"net/url"
"strconv"
)
const defaultLogGroupLimit = int64(50)
type LogGroupsRequest struct {
ResourceRequest
Limit int64
LogGroupNamePrefix, LogGroupNamePattern *string
}
func (r LogGroupsRequest) IsTargetingAllAccounts() bool {
return *r.AccountId == "all"
}
func ParseLogGroupsRequest(parameters url.Values) (LogGroupsRequest, error) {
logGroupNamePrefix := setIfNotEmptyString(parameters.Get("logGroupNamePrefix"))
logGroupPattern := setIfNotEmptyString(parameters.Get("logGroupPattern"))
if logGroupNamePrefix != nil && logGroupPattern != nil {
return LogGroupsRequest{}, fmt.Errorf("cannot set both log group name prefix and pattern")
}
return LogGroupsRequest{
Limit: getLimit(parameters.Get("limit")),
ResourceRequest: ResourceRequest{
Region: parameters.Get("region"),
AccountId: setIfNotEmptyString(parameters.Get("accountId")),
},
LogGroupNamePrefix: logGroupNamePrefix,
LogGroupNamePattern: logGroupPattern,
}, nil
}
func setIfNotEmptyString(paramValue string) *string {
if paramValue == "" {
return nil
}
return &paramValue
}
func getLimit(limit string) int64 {
logGroupLimit := defaultLogGroupLimit
intLimit, err := strconv.ParseInt(limit, 10, 64)
if err == nil && intLimit > 0 {
logGroupLimit = intLimit
}
return logGroupLimit
}

View File

@@ -17,13 +17,13 @@ type MetricsRequest struct {
Namespace string
}
func GetMetricsRequest(parameters url.Values) (*MetricsRequest, error) {
func GetMetricsRequest(parameters url.Values) (MetricsRequest, error) {
resourceRequest, err := getResourceRequest(parameters)
if err != nil {
return nil, err
return MetricsRequest{}, err
}
return &MetricsRequest{
return MetricsRequest{
ResourceRequest: resourceRequest,
Namespace: parameters.Get("namespace"),
}, nil

View File

@@ -5,8 +5,15 @@ import (
"net/url"
)
const useLinkedAccountsId = "all"
type ResourceRequest struct {
Region string
Region string
AccountId *string
}
func (r *ResourceRequest) ShouldTargetAllAccounts() bool {
return r.AccountId != nil && *r.AccountId == useLinkedAccountsId
}
func getResourceRequest(parameters url.Values) (*ResourceRequest, error) {
@@ -14,9 +21,24 @@ func getResourceRequest(parameters url.Values) (*ResourceRequest, error) {
Region: parameters.Get("region"),
}
accountId := parameters.Get("accountId")
if accountId != "" {
request.AccountId = &accountId
}
if request.Region == "" {
return nil, fmt.Errorf("region is required")
}
return request, nil
}
type LogsRequest struct {
Limit int64
AccountId, LogGroupNamePrefix, LogGroupNamePattern *string
IsCrossAccountQueryingEnabled bool
}
func (r LogsRequest) IsTargetingAllAccounts() bool {
return *r.AccountId == useLinkedAccountsId
}

View File

@@ -1,11 +1,35 @@
package resources
import "github.com/aws/aws-sdk-go/service/cloudwatch"
type Dimension struct {
Name string
Value string
}
type ResourceResponse[T any] struct {
AccountId *string `json:"accountId,omitempty"`
Value T `json:"value"`
}
type MetricResponse struct {
*cloudwatch.Metric
AccountId *string `json:"accountId,omitempty"`
}
type Account struct {
Id string `json:"id"`
Arn string `json:"arn"`
Label string `json:"label"`
IsMonitoringAccount bool `json:"isMonitoringAccount"`
}
type Metric struct {
Name string `json:"name"`
Namespace string `json:"namespace"`
}
type LogGroup struct {
Arn string `json:"arn"`
Name string `json:"name"`
}

View File

@@ -1,20 +1,5 @@
package models
import (
"net/url"
"github.com/grafana/grafana-plugin-sdk-go/backend"
)
type RequestContext struct {
MetricsClientProvider MetricsClientProvider
Settings CloudWatchSettings
}
type RequestContextFactoryFunc func(pluginCtx backend.PluginContext, region string) (reqCtx RequestContext, err error)
type RouteHandlerFunc func(pluginCtx backend.PluginContext, reqContextFactory RequestContextFactoryFunc, parameters url.Values) ([]byte, *HttpError)
type cloudWatchLink struct {
View string `json:"view"`
Stacked bool `json:"stacked"`
@@ -31,7 +16,8 @@ type metricExpression struct {
}
type metricStatMeta struct {
Stat string `json:"stat"`
Period int `json:"period"`
Label string `json:"label,omitempty"`
Stat string `json:"stat"`
Period int `json:"period"`
Label string `json:"label,omitempty"`
AccountId string `json:"accountId,omitempty"`
}

View File

@@ -19,10 +19,12 @@ func (e *cloudWatchExecutor) newResourceMux() *http.ServeMux {
mux.HandleFunc("/ec2-instance-attribute", handleResourceReq(e.handleGetEc2InstanceAttribute))
mux.HandleFunc("/resource-arns", handleResourceReq(e.handleGetResourceArns))
mux.HandleFunc("/log-groups", handleResourceReq(e.handleGetLogGroups))
mux.HandleFunc("/describe-log-groups", routes.ResourceRequestMiddleware(routes.LogGroupsHandler, logger, e.getRequestContext)) // supports CrossAccountQuerying
mux.HandleFunc("/all-log-groups", handleResourceReq(e.handleGetAllLogGroups))
mux.HandleFunc("/metrics", routes.ResourceRequestMiddleware(routes.MetricsHandler, logger, e.getRequestContext))
mux.HandleFunc("/dimension-values", routes.ResourceRequestMiddleware(routes.DimensionValuesHandler, logger, e.getRequestContext))
mux.HandleFunc("/dimension-keys", routes.ResourceRequestMiddleware(routes.DimensionKeysHandler, logger, e.getRequestContext))
mux.HandleFunc("/accounts", routes.ResourceRequestMiddleware(routes.AccountsHandler, logger, e.getRequestContext))
mux.HandleFunc("/namespaces", routes.ResourceRequestMiddleware(routes.NamespacesHandler, logger, e.getRequestContext))
return mux
}

View File

@@ -0,0 +1,55 @@
package routes
import (
"encoding/json"
"errors"
"fmt"
"net/http"
"net/url"
"github.com/grafana/grafana-plugin-sdk-go/backend"
"github.com/grafana/grafana/pkg/tsdb/cloudwatch/models"
"github.com/grafana/grafana/pkg/tsdb/cloudwatch/services"
)
func AccountsHandler(pluginCtx backend.PluginContext, reqCtxFactory models.RequestContextFactoryFunc, parameters url.Values) ([]byte, *models.HttpError) {
region := parameters.Get("region")
if region == "" {
return nil, models.NewHttpError("error in AccountsHandler", http.StatusBadRequest, fmt.Errorf("region is required"))
}
service, err := newAccountsService(pluginCtx, reqCtxFactory, region)
if err != nil {
return nil, models.NewHttpError("error in AccountsHandler", http.StatusInternalServerError, err)
}
accounts, err := service.GetAccountsForCurrentUserOrRole()
if err != nil {
msg := "error getting accounts for current user or role"
switch {
case errors.Is(err, services.ErrAccessDeniedException):
return nil, models.NewHttpError(msg, http.StatusForbidden, err)
default:
return nil, models.NewHttpError(msg, http.StatusInternalServerError, err)
}
}
accountsResponse, err := json.Marshal(accounts)
if err != nil {
return nil, models.NewHttpError("error in AccountsHandler", http.StatusInternalServerError, err)
}
return accountsResponse, nil
}
// newAccountService is an account service factory.
//
// Stubbable by tests.
var newAccountsService = func(pluginCtx backend.PluginContext, reqCtxFactory models.RequestContextFactoryFunc, region string) (models.AccountsProvider, error) {
oamClient, err := reqCtxFactory(pluginCtx, region)
if err != nil {
return nil, err
}
return services.NewAccountsService(oamClient.OAMClientProvider), nil
}

View File

@@ -0,0 +1,96 @@
package routes
import (
"fmt"
"net/http"
"net/http/httptest"
"testing"
"github.com/grafana/grafana-plugin-sdk-go/backend"
"github.com/grafana/grafana/pkg/tsdb/cloudwatch/mocks"
"github.com/grafana/grafana/pkg/tsdb/cloudwatch/models"
"github.com/grafana/grafana/pkg/tsdb/cloudwatch/models/resources"
"github.com/grafana/grafana/pkg/tsdb/cloudwatch/services"
"github.com/stretchr/testify/assert"
)
func Test_accounts_route(t *testing.T) {
origNewAccountsService := newAccountsService
t.Cleanup(func() {
newAccountsService = origNewAccountsService
})
t.Run("successfully returns array of accounts json", func(t *testing.T) {
mockAccountsService := mocks.AccountsServiceMock{}
mockAccountsService.On("GetAccountsForCurrentUserOrRole").Return([]resources.ResourceResponse[resources.Account]{{
Value: resources.Account{
Id: "123456789012",
Arn: "some arn",
Label: "some label",
IsMonitoringAccount: true,
},
}}, nil)
newAccountsService = func(pluginCtx backend.PluginContext, reqCtxFactory models.RequestContextFactoryFunc, region string) (models.AccountsProvider, error) {
return &mockAccountsService, nil
}
rr := httptest.NewRecorder()
req := httptest.NewRequest("GET", "/accounts?region=us-east-1", nil)
handler := http.HandlerFunc(ResourceRequestMiddleware(AccountsHandler, logger, nil))
handler.ServeHTTP(rr, req)
assert.Equal(t, http.StatusOK, rr.Code)
assert.JSONEq(t, `[{"value":{"id":"123456789012", "arn":"some arn", "isMonitoringAccount":true, "label":"some label"}}]`, rr.Body.String())
})
t.Run("rejects POST method", func(t *testing.T) {
rr := httptest.NewRecorder()
req := httptest.NewRequest("POST", "/accounts?region=us-east-1", nil)
handler := http.HandlerFunc(ResourceRequestMiddleware(AccountsHandler, logger, nil))
handler.ServeHTTP(rr, req)
assert.Equal(t, http.StatusMethodNotAllowed, rr.Code)
})
t.Run("requires region query value", func(t *testing.T) {
rr := httptest.NewRecorder()
req := httptest.NewRequest("GET", "/accounts", nil)
handler := http.HandlerFunc(ResourceRequestMiddleware(AccountsHandler, logger, nil))
handler.ServeHTTP(rr, req)
assert.Equal(t, http.StatusBadRequest, rr.Code)
})
t.Run("returns 403 when accounts service returns ErrAccessDeniedException", func(t *testing.T) {
mockAccountsService := mocks.AccountsServiceMock{}
mockAccountsService.On("GetAccountsForCurrentUserOrRole").Return([]resources.ResourceResponse[resources.Account](nil),
fmt.Errorf("%w: %s", services.ErrAccessDeniedException, "some AWS message"))
newAccountsService = func(pluginCtx backend.PluginContext, reqCtxFactory models.RequestContextFactoryFunc, region string) (models.AccountsProvider, error) {
return &mockAccountsService, nil
}
rr := httptest.NewRecorder()
req := httptest.NewRequest("GET", "/accounts?region=us-east-1", nil)
handler := http.HandlerFunc(ResourceRequestMiddleware(AccountsHandler, logger, nil))
handler.ServeHTTP(rr, req)
assert.Equal(t, http.StatusForbidden, rr.Code)
assert.JSONEq(t,
`{"Message":"error getting accounts for current user or role: access denied. please check your IAM policy: some AWS message",
"Error":"access denied. please check your IAM policy: some AWS message","StatusCode":403}`, rr.Body.String())
})
t.Run("returns 500 when accounts service returns unknown error", func(t *testing.T) {
mockAccountsService := mocks.AccountsServiceMock{}
mockAccountsService.On("GetAccountsForCurrentUserOrRole").Return([]resources.ResourceResponse[resources.Account](nil), fmt.Errorf("some error"))
newAccountsService = func(pluginCtx backend.PluginContext, reqCtxFactory models.RequestContextFactoryFunc, region string) (models.AccountsProvider, error) {
return &mockAccountsService, nil
}
rr := httptest.NewRecorder()
req := httptest.NewRequest("GET", "/accounts?region=us-east-1", nil)
handler := http.HandlerFunc(ResourceRequestMiddleware(AccountsHandler, logger, nil))
handler.ServeHTTP(rr, req)
assert.Equal(t, http.StatusInternalServerError, rr.Code)
assert.Equal(t, `{"Message":"error getting accounts for current user or role: some error","Error":"some error","StatusCode":500}`, rr.Body.String())
})
}

View File

@@ -22,7 +22,7 @@ func DimensionKeysHandler(pluginCtx backend.PluginContext, reqCtxFactory models.
return nil, models.NewHttpError("error in DimensionKeyHandler", http.StatusInternalServerError, err)
}
var response []string
var response []resources.ResourceResponse[string]
switch dimensionKeysRequest.Type() {
case resources.FilterDimensionKeysRequest:
response, err = service.GetDimensionKeysByDimensionFilter(dimensionKeysRequest)

View File

@@ -31,7 +31,7 @@ func Test_DimensionKeys_Route(t *testing.T) {
len(r.DimensionFilter) == 2 &&
assert.Contains(t, r.DimensionFilter, &resources.Dimension{Name: "NodeID", Value: "Shared"}) &&
assert.Contains(t, r.DimensionFilter, &resources.Dimension{Name: "stage", Value: "QueryCommit"})
})).Return([]string{}, nil).Once()
})).Return([]resources.ResourceResponse[string]{}, nil).Once()
newListMetricsService = func(pluginCtx backend.PluginContext, reqCtxFactory models.RequestContextFactoryFunc, region string) (models.ListMetricsProvider, error) {
return &mockListMetricsService, nil
}
@@ -48,10 +48,10 @@ func Test_DimensionKeys_Route(t *testing.T) {
})
haveBeenCalled := false
usedNamespace := ""
services.GetHardCodedDimensionKeysByNamespace = func(namespace string) ([]string, error) {
services.GetHardCodedDimensionKeysByNamespace = func(namespace string) ([]resources.ResourceResponse[string], error) {
haveBeenCalled = true
usedNamespace = namespace
return []string{}, nil
return []resources.ResourceResponse[string]{}, nil
}
rr := httptest.NewRecorder()
req := httptest.NewRequest("GET", "/dimension-keys?region=us-east-2&namespace=AWS/EC2&metricName=CPUUtilization", nil)
@@ -66,7 +66,7 @@ func Test_DimensionKeys_Route(t *testing.T) {
t.Run("return 500 if GetDimensionKeysByDimensionFilter returns an error", func(t *testing.T) {
mockListMetricsService := mocks.ListMetricsServiceMock{}
mockListMetricsService.On("GetDimensionKeysByDimensionFilter", mock.Anything).Return([]string{}, fmt.Errorf("some error"))
mockListMetricsService.On("GetDimensionKeysByDimensionFilter", mock.Anything).Return([]resources.ResourceResponse[string]{}, fmt.Errorf("some error"))
newListMetricsService = func(pluginCtx backend.PluginContext, reqCtxFactory models.RequestContextFactoryFunc, region string) (models.ListMetricsProvider, error) {
return &mockListMetricsService, nil
}

View File

@@ -26,7 +26,7 @@ func Test_DimensionValues_Route(t *testing.T) {
len(r.DimensionFilter) == 2 &&
assert.Contains(t, r.DimensionFilter, &resources.Dimension{Name: "NodeID", Value: "Shared"}) &&
assert.Contains(t, r.DimensionFilter, &resources.Dimension{Name: "stage", Value: "QueryCommit"})
})).Return([]string{}, nil).Once()
})).Return([]resources.ResourceResponse[string]{}, nil).Once()
newListMetricsService = func(pluginCtx backend.PluginContext, reqCtxFactory models.RequestContextFactoryFunc, region string) (models.ListMetricsProvider, error) {
return &mockListMetricsService, nil
}
@@ -38,7 +38,7 @@ func Test_DimensionValues_Route(t *testing.T) {
t.Run("returns 500 if GetDimensionValuesByDimensionFilter returns an error", func(t *testing.T) {
mockListMetricsService := mocks.ListMetricsServiceMock{}
mockListMetricsService.On("GetDimensionValuesByDimensionFilter", mock.Anything).Return([]string{}, fmt.Errorf("some error"))
mockListMetricsService.On("GetDimensionValuesByDimensionFilter", mock.Anything).Return([]resources.ResourceResponse[string]{}, fmt.Errorf("some error"))
newListMetricsService = func(pluginCtx backend.PluginContext, reqCtxFactory models.RequestContextFactoryFunc, region string) (models.ListMetricsProvider, error) {
return &mockListMetricsService, nil
}

View File

@@ -0,0 +1,49 @@
package routes
import (
"encoding/json"
"net/http"
"net/url"
"github.com/grafana/grafana-plugin-sdk-go/backend"
"github.com/grafana/grafana/pkg/services/featuremgmt"
"github.com/grafana/grafana/pkg/tsdb/cloudwatch/models"
"github.com/grafana/grafana/pkg/tsdb/cloudwatch/models/resources"
"github.com/grafana/grafana/pkg/tsdb/cloudwatch/services"
)
func LogGroupsHandler(pluginCtx backend.PluginContext, reqCtxFactory models.RequestContextFactoryFunc, parameters url.Values) ([]byte, *models.HttpError) {
request, err := resources.ParseLogGroupsRequest(parameters)
if err != nil {
return nil, models.NewHttpError("cannot set both log group name prefix and pattern", http.StatusBadRequest, err)
}
service, err := newLogGroupsService(pluginCtx, reqCtxFactory, request.Region)
if err != nil {
return nil, models.NewHttpError("newLogGroupsService error", http.StatusInternalServerError, err)
}
logGroups, err := service.GetLogGroups(request)
if err != nil {
return nil, models.NewHttpError("GetLogGroups error", http.StatusInternalServerError, err)
}
logGroupsResponse, err := json.Marshal(logGroups)
if err != nil {
return nil, models.NewHttpError("LogGroupsHandler json error", http.StatusInternalServerError, err)
}
return logGroupsResponse, nil
}
// newLogGroupsService is a describe log groups service factory.
//
// Stubbable by tests.
var newLogGroupsService = func(pluginCtx backend.PluginContext, reqCtxFactory models.RequestContextFactoryFunc, region string) (models.LogGroupsProvider, error) {
reqCtx, err := reqCtxFactory(pluginCtx, region)
if err != nil {
return nil, err
}
return services.NewLogGroupsService(reqCtx.LogsAPIProvider, reqCtx.Features.IsEnabled(featuremgmt.FlagCloudWatchCrossAccountQuerying)), nil
}

View File

@@ -0,0 +1,240 @@
package routes
import (
"fmt"
"net/http"
"net/http/httptest"
"testing"
"github.com/grafana/grafana-plugin-sdk-go/backend"
"github.com/grafana/grafana/pkg/services/featuremgmt"
"github.com/grafana/grafana/pkg/tsdb/cloudwatch/mocks"
"github.com/grafana/grafana/pkg/tsdb/cloudwatch/models"
"github.com/grafana/grafana/pkg/tsdb/cloudwatch/models/resources"
"github.com/grafana/grafana/pkg/tsdb/cloudwatch/utils"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/mock"
)
func Test_log_groups_route(t *testing.T) {
origLogGroupsService := newLogGroupsService
t.Cleanup(func() {
newLogGroupsService = origLogGroupsService
})
mockFeatures := mocks.MockFeatures{}
mockFeatures.On("IsEnabled", featuremgmt.FlagCloudWatchCrossAccountQuerying).Return(false)
reqCtxFunc := func(pluginCtx backend.PluginContext, region string) (reqCtx models.RequestContext, err error) {
return models.RequestContext{Features: &mockFeatures}, err
}
t.Run("successfully returns 1 log group with account id", func(t *testing.T) {
mockLogsService := mocks.LogsService{}
mockLogsService.On("GetLogGroups", mock.Anything).Return([]resources.ResourceResponse[resources.LogGroup]{{
Value: resources.LogGroup{
Arn: "some arn",
Name: "some name",
},
AccountId: utils.Pointer("111"),
}}, nil)
newLogGroupsService = func(pluginCtx backend.PluginContext, reqCtxFactory models.RequestContextFactoryFunc, region string) (models.LogGroupsProvider, error) {
return &mockLogsService, nil
}
rr := httptest.NewRecorder()
req := httptest.NewRequest("GET", "/log-groups", nil)
handler := http.HandlerFunc(ResourceRequestMiddleware(LogGroupsHandler, logger, reqCtxFunc))
handler.ServeHTTP(rr, req)
assert.Equal(t, http.StatusOK, rr.Code)
assert.JSONEq(t, `[{"value":{"name":"some name", "arn":"some arn"},"accountId":"111"}]`, rr.Body.String())
})
t.Run("successfully returns multiple log groups with account id", func(t *testing.T) {
mockLogsService := mocks.LogsService{}
mockLogsService.On("GetLogGroups", mock.Anything).Return(
[]resources.ResourceResponse[resources.LogGroup]{
{
Value: resources.LogGroup{
Arn: "arn 1",
Name: "name 1",
},
AccountId: utils.Pointer("111"),
}, {
Value: resources.LogGroup{
Arn: "arn 2",
Name: "name 2",
},
AccountId: utils.Pointer("222"),
},
}, nil)
newLogGroupsService = func(pluginCtx backend.PluginContext, reqCtxFactory models.RequestContextFactoryFunc, region string) (models.LogGroupsProvider, error) {
return &mockLogsService, nil
}
rr := httptest.NewRecorder()
req := httptest.NewRequest("GET", "/log-groups", nil)
handler := http.HandlerFunc(ResourceRequestMiddleware(LogGroupsHandler, logger, reqCtxFunc))
handler.ServeHTTP(rr, req)
assert.Equal(t, http.StatusOK, rr.Code)
assert.JSONEq(t, `[
{
"value":{
"name":"name 1",
"arn":"arn 1"
},
"accountId":"111"
},
{
"value":{
"name":"name 2",
"arn":"arn 2"
},
"accountId":"222"
}
]`, rr.Body.String())
})
t.Run("returns error when both logGroupPrefix and logGroup Pattern are provided", func(t *testing.T) {
mockLogsService := mocks.LogsService{}
mockLogsService.On("GetLogGroups", mock.Anything).Return([]resources.ResourceResponse[resources.LogGroup]{}, nil)
newLogGroupsService = func(pluginCtx backend.PluginContext, reqCtxFactory models.RequestContextFactoryFunc, region string) (models.LogGroupsProvider, error) {
return &mockLogsService, nil
}
rr := httptest.NewRecorder()
req := httptest.NewRequest("GET", "/log-groups?logGroupNamePrefix=some-prefix&logGroupPattern=some-pattern", nil)
handler := http.HandlerFunc(ResourceRequestMiddleware(LogGroupsHandler, logger, reqCtxFunc))
handler.ServeHTTP(rr, req)
assert.Equal(t, http.StatusBadRequest, rr.Code)
assert.JSONEq(t, `{"Error":"cannot set both log group name prefix and pattern", "Message":"cannot set both log group name prefix and pattern: cannot set both log group name prefix and pattern", "StatusCode":400}`, rr.Body.String())
})
t.Run("passes default log group limit and nil for logGroupNamePrefix, accountId, and logGroupPattern", func(t *testing.T) {
mockLogsService := mocks.LogsService{}
mockLogsService.On("GetLogGroups", mock.Anything).Return([]resources.ResourceResponse[resources.LogGroup]{}, nil)
newLogGroupsService = func(pluginCtx backend.PluginContext, reqCtxFactory models.RequestContextFactoryFunc, region string) (models.LogGroupsProvider, error) {
return &mockLogsService, nil
}
rr := httptest.NewRecorder()
req := httptest.NewRequest("GET", "/log-groups", nil)
handler := http.HandlerFunc(ResourceRequestMiddleware(LogGroupsHandler, logger, reqCtxFunc))
handler.ServeHTTP(rr, req)
mockLogsService.AssertCalled(t, "GetLogGroups", resources.LogGroupsRequest{
Limit: 50,
ResourceRequest: resources.ResourceRequest{},
LogGroupNamePrefix: nil,
LogGroupNamePattern: nil,
})
})
t.Run("passes default log group limit and nil for logGroupNamePrefix when both are absent", func(t *testing.T) {
mockLogsService := mocks.LogsService{}
mockLogsService.On("GetLogGroups", mock.Anything).Return([]resources.ResourceResponse[resources.LogGroup]{}, nil)
newLogGroupsService = func(pluginCtx backend.PluginContext, reqCtxFactory models.RequestContextFactoryFunc, region string) (models.LogGroupsProvider, error) {
return &mockLogsService, nil
}
rr := httptest.NewRecorder()
req := httptest.NewRequest("GET", "/log-groups", nil)
handler := http.HandlerFunc(ResourceRequestMiddleware(LogGroupsHandler, logger, reqCtxFunc))
handler.ServeHTTP(rr, req)
mockLogsService.AssertCalled(t, "GetLogGroups", resources.LogGroupsRequest{
Limit: 50,
LogGroupNamePrefix: nil,
})
})
t.Run("passes log group limit from query parameter", func(t *testing.T) {
mockLogsService := mocks.LogsService{}
mockLogsService.On("GetLogGroups", mock.Anything).Return([]resources.ResourceResponse[resources.LogGroup]{}, nil)
newLogGroupsService = func(pluginCtx backend.PluginContext, reqCtxFactory models.RequestContextFactoryFunc, region string) (models.LogGroupsProvider, error) {
return &mockLogsService, nil
}
rr := httptest.NewRecorder()
req := httptest.NewRequest("GET", "/log-groups?limit=2", nil)
handler := http.HandlerFunc(ResourceRequestMiddleware(LogGroupsHandler, logger, reqCtxFunc))
handler.ServeHTTP(rr, req)
mockLogsService.AssertCalled(t, "GetLogGroups", resources.LogGroupsRequest{
Limit: 2,
})
})
t.Run("passes logGroupPrefix from query parameter", func(t *testing.T) {
mockLogsService := mocks.LogsService{}
mockLogsService.On("GetLogGroups", mock.Anything).Return([]resources.ResourceResponse[resources.LogGroup]{}, nil)
newLogGroupsService = func(pluginCtx backend.PluginContext, reqCtxFactory models.RequestContextFactoryFunc, region string) (models.LogGroupsProvider, error) {
return &mockLogsService, nil
}
rr := httptest.NewRecorder()
req := httptest.NewRequest("GET", "/log-groups?logGroupNamePrefix=some-prefix", nil)
handler := http.HandlerFunc(ResourceRequestMiddleware(LogGroupsHandler, logger, reqCtxFunc))
handler.ServeHTTP(rr, req)
mockLogsService.AssertCalled(t, "GetLogGroups", resources.LogGroupsRequest{
Limit: 50,
LogGroupNamePrefix: utils.Pointer("some-prefix"),
})
})
t.Run("passes logGroupPattern from query parameter", func(t *testing.T) {
mockLogsService := mocks.LogsService{}
mockLogsService.On("GetLogGroups", mock.Anything).Return([]resources.ResourceResponse[resources.LogGroup]{}, nil)
newLogGroupsService = func(pluginCtx backend.PluginContext, reqCtxFactory models.RequestContextFactoryFunc, region string) (models.LogGroupsProvider, error) {
return &mockLogsService, nil
}
rr := httptest.NewRecorder()
req := httptest.NewRequest("GET", "/log-groups?logGroupPattern=some-pattern", nil)
handler := http.HandlerFunc(ResourceRequestMiddleware(LogGroupsHandler, logger, reqCtxFunc))
handler.ServeHTTP(rr, req)
mockLogsService.AssertCalled(t, "GetLogGroups", resources.LogGroupsRequest{
Limit: 50,
LogGroupNamePattern: utils.Pointer("some-pattern"),
})
})
t.Run("passes logGroupPattern from query parameter", func(t *testing.T) {
mockLogsService := mocks.LogsService{}
mockLogsService.On("GetLogGroups", mock.Anything).Return([]resources.ResourceResponse[resources.LogGroup]{}, nil)
newLogGroupsService = func(pluginCtx backend.PluginContext, reqCtxFactory models.RequestContextFactoryFunc, region string) (models.LogGroupsProvider, error) {
return &mockLogsService, nil
}
rr := httptest.NewRecorder()
req := httptest.NewRequest("GET", "/log-groups?accountId=some-account-id", nil)
handler := http.HandlerFunc(ResourceRequestMiddleware(LogGroupsHandler, logger, reqCtxFunc))
handler.ServeHTTP(rr, req)
mockLogsService.AssertCalled(t, "GetLogGroups", resources.LogGroupsRequest{
Limit: 50,
ResourceRequest: resources.ResourceRequest{AccountId: utils.Pointer("some-account-id")},
})
})
t.Run("returns error if service returns error", func(t *testing.T) {
mockLogsService := mocks.LogsService{}
mockLogsService.On("GetLogGroups", mock.Anything).
Return([]resources.ResourceResponse[resources.LogGroup]{}, fmt.Errorf("some error"))
newLogGroupsService = func(pluginCtx backend.PluginContext, reqCtxFactory models.RequestContextFactoryFunc, region string) (models.LogGroupsProvider, error) {
return &mockLogsService, nil
}
rr := httptest.NewRecorder()
req := httptest.NewRequest("GET", "/log-groups", nil)
handler := http.HandlerFunc(ResourceRequestMiddleware(LogGroupsHandler, logger, reqCtxFunc))
handler.ServeHTTP(rr, req)
assert.Equal(t, http.StatusInternalServerError, rr.Code)
assert.JSONEq(t, `{"Error":"some error","Message":"GetLogGroups error: some error","StatusCode":500}`, rr.Body.String())
})
}

View File

@@ -22,20 +22,20 @@ func MetricsHandler(pluginCtx backend.PluginContext, reqCtxFactory models.Reques
return nil, models.NewHttpError("error in MetricsHandler", http.StatusInternalServerError, err)
}
var metrics []resources.Metric
var response []resources.ResourceResponse[resources.Metric]
switch metricsRequest.Type() {
case resources.AllMetricsRequestType:
metrics = services.GetAllHardCodedMetrics()
response = services.GetAllHardCodedMetrics()
case resources.MetricsByNamespaceRequestType:
metrics, err = services.GetHardCodedMetricsByNamespace(metricsRequest.Namespace)
response, err = services.GetHardCodedMetricsByNamespace(metricsRequest.Namespace)
case resources.CustomNamespaceRequestType:
metrics, err = service.GetMetricsByNamespace(metricsRequest.Namespace)
response, err = service.GetMetricsByNamespace(metricsRequest)
}
if err != nil {
return nil, models.NewHttpError("error in MetricsHandler", http.StatusInternalServerError, err)
}
metricsResponse, err := json.Marshal(metrics)
metricsResponse, err := json.Marshal(response)
if err != nil {
return nil, models.NewHttpError("error in MetricsHandler", http.StatusInternalServerError, err)
}

View File

@@ -1,27 +1,24 @@
package routes
import (
"encoding/json"
"fmt"
"net/http"
"net/http/httptest"
"testing"
"github.com/grafana/grafana-plugin-sdk-go/backend"
"github.com/grafana/grafana/pkg/tsdb/cloudwatch/models/resources"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/mock"
"github.com/stretchr/testify/require"
"github.com/grafana/grafana/pkg/tsdb/cloudwatch/mocks"
"github.com/grafana/grafana/pkg/tsdb/cloudwatch/models"
"github.com/grafana/grafana/pkg/tsdb/cloudwatch/models/resources"
"github.com/grafana/grafana/pkg/tsdb/cloudwatch/services"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/mock"
)
func Test_Metrics_Route(t *testing.T) {
t.Run("calls GetMetricsByNamespace when a CustomNamespaceRequestType is passed", func(t *testing.T) {
mockListMetricsService := mocks.ListMetricsServiceMock{}
mockListMetricsService.On("GetMetricsByNamespace", mock.Anything).Return([]resources.Metric{}, nil)
mockListMetricsService.On("GetMetricsByNamespace", mock.Anything).Return([]resources.ResourceResponse[resources.Metric]{}, nil)
newListMetricsService = func(pluginCtx backend.PluginContext, reqCtxFactory models.RequestContextFactoryFunc, region string) (models.ListMetricsProvider, error) {
return &mockListMetricsService, nil
}
@@ -38,17 +35,14 @@ func Test_Metrics_Route(t *testing.T) {
services.GetAllHardCodedMetrics = origGetAllHardCodedMetrics
})
haveBeenCalled := false
services.GetAllHardCodedMetrics = func() []resources.Metric {
services.GetAllHardCodedMetrics = func() []resources.ResourceResponse[resources.Metric] {
haveBeenCalled = true
return []resources.Metric{}
return []resources.ResourceResponse[resources.Metric]{}
}
rr := httptest.NewRecorder()
req := httptest.NewRequest("GET", "/metrics?region=us-east-2", nil)
handler := http.HandlerFunc(ResourceRequestMiddleware(MetricsHandler, logger, nil))
handler.ServeHTTP(rr, req)
res := []resources.Metric{}
err := json.Unmarshal(rr.Body.Bytes(), &res)
require.Nil(t, err)
assert.True(t, haveBeenCalled)
})
@@ -59,25 +53,22 @@ func Test_Metrics_Route(t *testing.T) {
})
haveBeenCalled := false
usedNamespace := ""
services.GetHardCodedMetricsByNamespace = func(namespace string) ([]resources.Metric, error) {
services.GetHardCodedMetricsByNamespace = func(namespace string) ([]resources.ResourceResponse[resources.Metric], error) {
haveBeenCalled = true
usedNamespace = namespace
return []resources.Metric{}, nil
return []resources.ResourceResponse[resources.Metric]{}, nil
}
rr := httptest.NewRecorder()
req := httptest.NewRequest("GET", "/metrics?region=us-east-2&namespace=AWS/DMS", nil)
handler := http.HandlerFunc(ResourceRequestMiddleware(MetricsHandler, logger, nil))
handler.ServeHTTP(rr, req)
res := []resources.Metric{}
err := json.Unmarshal(rr.Body.Bytes(), &res)
require.Nil(t, err)
assert.True(t, haveBeenCalled)
assert.Equal(t, "AWS/DMS", usedNamespace)
})
t.Run("returns 500 if GetMetricsByNamespace returns an error", func(t *testing.T) {
mockListMetricsService := mocks.ListMetricsServiceMock{}
mockListMetricsService.On("GetMetricsByNamespace", mock.Anything).Return([]resources.Metric{}, fmt.Errorf("some error"))
mockListMetricsService.On("GetMetricsByNamespace", mock.Anything).Return([]resources.ResourceResponse[resources.Metric]{}, fmt.Errorf("some error"))
newListMetricsService = func(pluginCtx backend.PluginContext, reqCtxFactory models.RequestContextFactoryFunc, region string) (models.ListMetricsProvider, error) {
return &mockListMetricsService, nil
}

View File

@@ -9,6 +9,7 @@ import (
"github.com/grafana/grafana-plugin-sdk-go/backend"
"github.com/grafana/grafana/pkg/tsdb/cloudwatch/models"
"github.com/grafana/grafana/pkg/tsdb/cloudwatch/models/resources"
"github.com/grafana/grafana/pkg/tsdb/cloudwatch/services"
)
@@ -18,14 +19,19 @@ func NamespacesHandler(pluginCtx backend.PluginContext, reqCtxFactory models.Req
return nil, models.NewHttpError("error in NamespacesHandler", http.StatusInternalServerError, err)
}
result := services.GetHardCodedNamespaces()
response := services.GetHardCodedNamespaces()
customNamespace := reqCtx.Settings.Namespace
if customNamespace != "" {
result = append(result, strings.Split(customNamespace, ",")...)
customNamespaces := strings.Split(customNamespace, ",")
for _, customNamespace := range customNamespaces {
response = append(response, resources.ResourceResponse[string]{Value: customNamespace})
}
}
sort.Strings(result)
sort.Slice(response, func(i, j int) bool {
return response[i].Value < response[j].Value
})
namespacesResponse, err := json.Marshal(result)
namespacesResponse, err := json.Marshal(response)
if err != nil {
return nil, models.NewHttpError("error in NamespacesHandler", http.StatusInternalServerError, err)
}

View File

@@ -9,6 +9,7 @@ import (
"github.com/stretchr/testify/assert"
"github.com/grafana/grafana/pkg/tsdb/cloudwatch/models"
"github.com/grafana/grafana/pkg/tsdb/cloudwatch/models/resources"
"github.com/grafana/grafana/pkg/tsdb/cloudwatch/services"
)
@@ -28,9 +29,9 @@ func Test_Namespaces_Route(t *testing.T) {
services.GetHardCodedNamespaces = origGetHardCodedNamespaces
})
haveBeenCalled := false
services.GetHardCodedNamespaces = func() []string {
services.GetHardCodedNamespaces = func() []resources.ResourceResponse[string] {
haveBeenCalled = true
return []string{}
return []resources.ResourceResponse[string]{}
}
rr := httptest.NewRecorder()
req := httptest.NewRequest("GET", "/namespaces", nil)
@@ -44,15 +45,15 @@ func Test_Namespaces_Route(t *testing.T) {
t.Cleanup(func() {
services.GetHardCodedNamespaces = origGetHardCodedNamespaces
})
services.GetHardCodedNamespaces = func() []string {
return []string{"AWS/EC2", "AWS/ELB"}
services.GetHardCodedNamespaces = func() []resources.ResourceResponse[string] {
return []resources.ResourceResponse[string]{{Value: "AWS/EC2"}, {Value: "AWS/ELB"}}
}
rr := httptest.NewRecorder()
req := httptest.NewRequest("GET", "/namespaces", nil)
customNamespaces = "customNamespace1,customNamespace2"
handler := http.HandlerFunc(ResourceRequestMiddleware(NamespacesHandler, logger, factoryFunc))
handler.ServeHTTP(rr, req)
assert.JSONEq(t, `["AWS/EC2", "AWS/ELB", "customNamespace1", "customNamespace2"]`, rr.Body.String())
assert.JSONEq(t, `[{"value":"AWS/EC2"}, {"value":"AWS/ELB"}, {"value":"customNamespace1"}, {"value":"customNamespace2"}]`, rr.Body.String())
})
t.Run("sorts result", func(t *testing.T) {
@@ -60,14 +61,14 @@ func Test_Namespaces_Route(t *testing.T) {
t.Cleanup(func() {
services.GetHardCodedNamespaces = origGetHardCodedNamespaces
})
services.GetHardCodedNamespaces = func() []string {
return []string{"AWS/XYZ", "AWS/ELB"}
services.GetHardCodedNamespaces = func() []resources.ResourceResponse[string] {
return []resources.ResourceResponse[string]{{Value: "AWS/XYZ"}, {Value: "AWS/ELB"}}
}
rr := httptest.NewRecorder()
req := httptest.NewRequest("GET", "/namespaces", nil)
customNamespaces = "DCustomNamespace1,ACustomNamespace2"
handler := http.HandlerFunc(ResourceRequestMiddleware(NamespacesHandler, logger, factoryFunc))
handler.ServeHTTP(rr, req)
assert.JSONEq(t, `["ACustomNamespace2", "AWS/ELB", "AWS/XYZ", "DCustomNamespace1"]`, rr.Body.String())
assert.JSONEq(t, `[{"value":"ACustomNamespace2"}, {"value":"AWS/ELB"}, {"value":"AWS/XYZ"}, {"value":"DCustomNamespace1"}]`, rr.Body.String())
})
}

View File

@@ -0,0 +1,89 @@
package services
import (
"errors"
"fmt"
"github.com/aws/aws-sdk-go/aws/awserr"
"github.com/aws/aws-sdk-go/service/oam"
"github.com/grafana/grafana/pkg/tsdb/cloudwatch/models"
"github.com/grafana/grafana/pkg/tsdb/cloudwatch/models/resources"
)
var ErrAccessDeniedException = errors.New("access denied. please check your IAM policy")
type AccountsService struct {
models.OAMClientProvider
}
func NewAccountsService(oamClient models.OAMClientProvider) models.AccountsProvider {
return &AccountsService{oamClient}
}
func (a *AccountsService) GetAccountsForCurrentUserOrRole() ([]resources.ResourceResponse[resources.Account], error) {
var nextToken *string
sinks := []*oam.ListSinksItem{}
for {
response, err := a.ListSinks(&oam.ListSinksInput{NextToken: nextToken})
if err != nil {
var aerr awserr.Error
if errors.As(err, &aerr) {
switch aerr.Code() {
// unlike many other services, OAM doesn't define this error code. however, it's returned in case calling role/user has insufficient permissions
case "AccessDeniedException":
return nil, fmt.Errorf("%w: %s", ErrAccessDeniedException, aerr.Message())
}
}
}
if err != nil {
return nil, fmt.Errorf("ListSinks error: %w", err)
}
sinks = append(sinks, response.Items...)
if response.NextToken == nil {
break
}
nextToken = response.NextToken
}
if len(sinks) == 0 {
return nil, nil
}
sinkIdentifier := sinks[0].Arn
response := []resources.Account{{
Id: getAccountId(*sinkIdentifier),
Label: *sinks[0].Name,
Arn: *sinkIdentifier,
IsMonitoringAccount: true,
}}
nextToken = nil
for {
links, err := a.ListAttachedLinks(&oam.ListAttachedLinksInput{
SinkIdentifier: sinkIdentifier,
NextToken: nextToken,
})
if err != nil {
return nil, fmt.Errorf("ListAttachedLinks error: %w", err)
}
for _, link := range links.Items {
arn := *link.LinkArn
response = append(response, resources.Account{
Id: getAccountId(arn),
Label: *link.Label,
Arn: arn,
IsMonitoringAccount: false,
})
}
if links.NextToken == nil {
break
}
nextToken = links.NextToken
}
return valuesToListMetricRespone(response), nil
}

View File

@@ -0,0 +1,165 @@
package services
import (
"fmt"
"testing"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/awserr"
"github.com/aws/aws-sdk-go/service/oam"
"github.com/grafana/grafana/pkg/tsdb/cloudwatch/mocks"
"github.com/grafana/grafana/pkg/tsdb/cloudwatch/models/resources"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/mock"
"github.com/stretchr/testify/require"
)
func TestHandleGetAccounts(t *testing.T) {
t.Run("Should return an error in case of insufficient permissions from ListSinks", func(t *testing.T) {
fakeOAMClient := &mocks.FakeOAMClient{}
fakeOAMClient.On("ListSinks", mock.Anything).Return(&oam.ListSinksOutput{}, awserr.New("AccessDeniedException",
"AWS message", nil))
accounts := NewAccountsService(fakeOAMClient)
resp, err := accounts.GetAccountsForCurrentUserOrRole()
assert.Error(t, err)
assert.Nil(t, resp)
assert.Equal(t, err.Error(), "access denied. please check your IAM policy: AWS message")
assert.ErrorIs(t, err, ErrAccessDeniedException)
})
t.Run("Should return an error in case of any error from ListSinks", func(t *testing.T) {
fakeOAMClient := &mocks.FakeOAMClient{}
fakeOAMClient.On("ListSinks", mock.Anything).Return(&oam.ListSinksOutput{}, fmt.Errorf("some error"))
accounts := NewAccountsService(fakeOAMClient)
resp, err := accounts.GetAccountsForCurrentUserOrRole()
assert.Error(t, err)
assert.Nil(t, resp)
assert.Equal(t, err.Error(), "ListSinks error: some error")
})
t.Run("Should return empty array in case no monitoring account exists", func(t *testing.T) {
fakeOAMClient := &mocks.FakeOAMClient{}
fakeOAMClient.On("ListSinks", mock.Anything).Return(&oam.ListSinksOutput{}, nil)
accounts := NewAccountsService(fakeOAMClient)
resp, err := accounts.GetAccountsForCurrentUserOrRole()
assert.NoError(t, err)
assert.Empty(t, resp)
})
t.Run("Should return one monitoring account (the first) even though ListSinks returns multiple sinks", func(t *testing.T) {
fakeOAMClient := &mocks.FakeOAMClient{}
fakeOAMClient.On("ListSinks", mock.Anything).Return(&oam.ListSinksOutput{
Items: []*oam.ListSinksItem{
{Name: aws.String("Account 1"), Arn: aws.String("arn:aws:logs:us-east-1:123456789012:log-group:my-log-group1")},
{Name: aws.String("Account 2"), Arn: aws.String("arn:aws:logs:us-east-1:123456789012:log-group:my-log-group2")},
},
NextToken: new(string),
}, nil).Once()
fakeOAMClient.On("ListSinks", mock.Anything).Return(&oam.ListSinksOutput{
Items: []*oam.ListSinksItem{
{Name: aws.String("Account 3"), Arn: aws.String("arn:aws:logs:us-east-1:123456789012:log-group:my-log-group3")},
},
NextToken: nil,
}, nil)
fakeOAMClient.On("ListAttachedLinks", mock.Anything).Return(&oam.ListAttachedLinksOutput{}, nil)
accounts := NewAccountsService(fakeOAMClient)
resp, err := accounts.GetAccountsForCurrentUserOrRole()
assert.NoError(t, err)
fakeOAMClient.AssertNumberOfCalls(t, "ListSinks", 2)
require.Len(t, resp, 1)
assert.True(t, resp[0].Value.IsMonitoringAccount)
assert.Equal(t, "Account 1", resp[0].Value.Label)
assert.Equal(t, "arn:aws:logs:us-east-1:123456789012:log-group:my-log-group1", resp[0].Value.Arn)
})
t.Run("Should merge the first sink with attached links", func(t *testing.T) {
fakeOAMClient := &mocks.FakeOAMClient{}
fakeOAMClient.On("ListSinks", mock.Anything).Return(&oam.ListSinksOutput{
Items: []*oam.ListSinksItem{
{Name: aws.String("Account 1"), Arn: aws.String("arn:aws:logs:us-east-1:123456789012:log-group:my-log-group1")},
{Name: aws.String("Account 2"), Arn: aws.String("arn:aws:logs:us-east-1:123456789012:log-group:my-log-group2")},
},
NextToken: new(string),
}, nil).Once()
fakeOAMClient.On("ListSinks", mock.Anything).Return(&oam.ListSinksOutput{
Items: []*oam.ListSinksItem{
{Name: aws.String("Account 3"), Arn: aws.String("arn:aws:logs:us-east-1:123456789012:log-group:my-log-group3")},
},
NextToken: nil,
}, nil)
fakeOAMClient.On("ListAttachedLinks", mock.Anything).Return(&oam.ListAttachedLinksOutput{
Items: []*oam.ListAttachedLinksItem{
{Label: aws.String("Account 10"), LinkArn: aws.String("arn:aws:logs:us-east-1:123456789013:log-group:my-log-group10")},
{Label: aws.String("Account 11"), LinkArn: aws.String("arn:aws:logs:us-east-1:123456789014:log-group:my-log-group11")},
},
NextToken: new(string),
}, nil).Once()
fakeOAMClient.On("ListAttachedLinks", mock.Anything).Return(&oam.ListAttachedLinksOutput{
Items: []*oam.ListAttachedLinksItem{
{Label: aws.String("Account 12"), LinkArn: aws.String("arn:aws:logs:us-east-1:123456789012:log-group:my-log-group12")},
},
NextToken: nil,
}, nil)
accounts := NewAccountsService(fakeOAMClient)
resp, err := accounts.GetAccountsForCurrentUserOrRole()
assert.NoError(t, err)
fakeOAMClient.AssertNumberOfCalls(t, "ListSinks", 2)
fakeOAMClient.AssertNumberOfCalls(t, "ListAttachedLinks", 2)
expectedAccounts := []resources.ResourceResponse[resources.Account]{
{Value: resources.Account{Id: "123456789012", Label: "Account 1", Arn: "arn:aws:logs:us-east-1:123456789012:log-group:my-log-group1", IsMonitoringAccount: true}},
{Value: resources.Account{Id: "123456789013", Label: "Account 10", Arn: "arn:aws:logs:us-east-1:123456789013:log-group:my-log-group10", IsMonitoringAccount: false}},
{Value: resources.Account{Id: "123456789014", Label: "Account 11", Arn: "arn:aws:logs:us-east-1:123456789014:log-group:my-log-group11", IsMonitoringAccount: false}},
{Value: resources.Account{Id: "123456789012", Label: "Account 12", Arn: "arn:aws:logs:us-east-1:123456789012:log-group:my-log-group12", IsMonitoringAccount: false}},
}
assert.Equal(t, expectedAccounts, resp)
})
t.Run("Should call ListAttachedLinks with arn of first sink", func(t *testing.T) {
fakeOAMClient := &mocks.FakeOAMClient{}
fakeOAMClient.On("ListSinks", mock.Anything).Return(&oam.ListSinksOutput{
Items: []*oam.ListSinksItem{
{Name: aws.String("Account 1"), Arn: aws.String("arn:aws:logs:us-east-1:123456789012:log-group:my-log-group1")},
},
NextToken: new(string),
}, nil).Once()
fakeOAMClient.On("ListSinks", mock.Anything).Return(&oam.ListSinksOutput{
Items: []*oam.ListSinksItem{
{Name: aws.String("Account 3"), Arn: aws.String("arn:aws:logs:us-east-1:123456789012:log-group:my-log-group3")},
},
NextToken: nil,
}, nil).Once()
fakeOAMClient.On("ListAttachedLinks", mock.Anything).Return(&oam.ListAttachedLinksOutput{}, nil)
accounts := NewAccountsService(fakeOAMClient)
_, _ = accounts.GetAccountsForCurrentUserOrRole()
fakeOAMClient.AssertCalled(t, "ListAttachedLinks", &oam.ListAttachedLinksInput{
SinkIdentifier: aws.String("arn:aws:logs:us-east-1:123456789012:log-group:my-log-group1"),
})
})
t.Run("Should return an error in case of any error from ListAttachedLinks", func(t *testing.T) {
fakeOAMClient := &mocks.FakeOAMClient{}
fakeOAMClient.On("ListSinks", mock.Anything).Return(&oam.ListSinksOutput{
Items: []*oam.ListSinksItem{{Name: aws.String("Account 1"), Arn: aws.String("arn:aws:logs:us-east-1:123456789012:log-group:my-log-group1")}},
}, nil)
fakeOAMClient.On("ListAttachedLinks", mock.Anything).Return(&oam.ListAttachedLinksOutput{}, fmt.Errorf("some error")).Once()
accounts := NewAccountsService(fakeOAMClient)
resp, err := accounts.GetAccountsForCurrentUserOrRole()
assert.Error(t, err)
assert.Nil(t, resp)
assert.Equal(t, err.Error(), "ListAttachedLinks error: some error")
})
}

View File

@@ -7,16 +7,16 @@ import (
"github.com/grafana/grafana/pkg/tsdb/cloudwatch/models/resources"
)
var GetHardCodedDimensionKeysByNamespace = func(namespace string) ([]string, error) {
var dimensionKeys []string
var GetHardCodedDimensionKeysByNamespace = func(namespace string) ([]resources.ResourceResponse[string], error) {
var response []string
exists := false
if dimensionKeys, exists = constants.NamespaceDimensionKeysMap[namespace]; !exists {
if response, exists = constants.NamespaceDimensionKeysMap[namespace]; !exists {
return nil, fmt.Errorf("unable to find dimensions for namespace '%q'", namespace)
}
return dimensionKeys, nil
return valuesToListMetricRespone(response), nil
}
var GetHardCodedMetricsByNamespace = func(namespace string) ([]resources.Metric, error) {
var GetHardCodedMetricsByNamespace = func(namespace string) ([]resources.ResourceResponse[resources.Metric], error) {
response := []resources.Metric{}
exists := false
var metrics []string
@@ -28,10 +28,10 @@ var GetHardCodedMetricsByNamespace = func(namespace string) ([]resources.Metric,
response = append(response, resources.Metric{Namespace: namespace, Name: metric})
}
return response, nil
return valuesToListMetricRespone(response), nil
}
var GetAllHardCodedMetrics = func() []resources.Metric {
var GetAllHardCodedMetrics = func() []resources.ResourceResponse[resources.Metric] {
response := []resources.Metric{}
for namespace, metrics := range constants.NamespaceMetricsMap {
for _, metric := range metrics {
@@ -39,14 +39,14 @@ var GetAllHardCodedMetrics = func() []resources.Metric {
}
}
return response
return valuesToListMetricRespone(response)
}
var GetHardCodedNamespaces = func() []string {
var namespaces []string
var GetHardCodedNamespaces = func() []resources.ResourceResponse[string] {
response := []string{}
for key := range constants.NamespaceMetricsMap {
namespaces = append(namespaces, key)
response = append(response, key)
}
return namespaces
return valuesToListMetricRespone(response)
}

View File

@@ -19,7 +19,7 @@ func TestHardcodedMetrics_GetHardCodedDimensionKeysByNamespace(t *testing.T) {
t.Run("Should return keys if namespace exist", func(t *testing.T) {
resp, err := GetHardCodedDimensionKeysByNamespace("AWS/EC2")
require.NoError(t, err)
assert.Equal(t, []string{"AutoScalingGroupName", "ImageId", "InstanceId", "InstanceType"}, resp)
assert.Equal(t, []resources.ResourceResponse[string]{{Value: "AutoScalingGroupName"}, {Value: "ImageId"}, {Value: "InstanceId"}, {Value: "InstanceType"}}, resp)
})
}
@@ -34,6 +34,6 @@ func TestHardcodedMetrics_GetHardCodedMetricsByNamespace(t *testing.T) {
t.Run("Should return metrics if namespace exist", func(t *testing.T) {
resp, err := GetHardCodedMetricsByNamespace("AWS/IoTAnalytics")
require.NoError(t, err)
assert.Equal(t, []resources.Metric{{Name: "ActionExecution", Namespace: "AWS/IoTAnalytics"}, {Name: "ActivityExecutionError", Namespace: "AWS/IoTAnalytics"}, {Name: "IncomingMessages", Namespace: "AWS/IoTAnalytics"}}, resp)
assert.Equal(t, []resources.ResourceResponse[resources.Metric]{{Value: resources.Metric{Name: "ActionExecution", Namespace: "AWS/IoTAnalytics"}}, {Value: resources.Metric{Name: "ActivityExecutionError", Namespace: "AWS/IoTAnalytics"}}, {Value: resources.Metric{Name: "IncomingMessages", Namespace: "AWS/IoTAnalytics"}}}, resp)
})
}

View File

@@ -18,7 +18,7 @@ func NewListMetricsService(metricsClient models.MetricsClientProvider) models.Li
return &ListMetricsService{metricsClient}
}
func (l *ListMetricsService) GetDimensionKeysByDimensionFilter(r resources.DimensionKeysRequest) ([]string, error) {
func (l *ListMetricsService) GetDimensionKeysByDimensionFilter(r resources.DimensionKeysRequest) ([]resources.ResourceResponse[string], error) {
input := &cloudwatch.ListMetricsInput{}
if r.Namespace != "" {
input.Namespace = aws.String(r.Namespace)
@@ -27,13 +27,14 @@ func (l *ListMetricsService) GetDimensionKeysByDimensionFilter(r resources.Dimen
input.MetricName = aws.String(r.MetricName)
}
setDimensionFilter(input, r.DimensionFilter)
setAccount(input, r.ResourceRequest)
metrics, err := l.ListMetricsWithPageLimit(input)
if err != nil {
return nil, fmt.Errorf("%v: %w", "unable to call AWS API", err)
}
var dimensionKeys []string
response := []resources.ResourceResponse[string]{}
// remove duplicates
dupCheck := make(map[string]struct{})
for _, metric := range metrics {
@@ -56,26 +57,27 @@ func (l *ListMetricsService) GetDimensionKeysByDimensionFilter(r resources.Dimen
}
dupCheck[*dim.Name] = struct{}{}
dimensionKeys = append(dimensionKeys, *dim.Name)
response = append(response, resources.ResourceResponse[string]{AccountId: metric.AccountId, Value: *dim.Name})
}
}
return dimensionKeys, nil
return response, nil
}
func (l *ListMetricsService) GetDimensionValuesByDimensionFilter(r resources.DimensionValuesRequest) ([]string, error) {
func (l *ListMetricsService) GetDimensionValuesByDimensionFilter(r resources.DimensionValuesRequest) ([]resources.ResourceResponse[string], error) {
input := &cloudwatch.ListMetricsInput{
Namespace: aws.String(r.Namespace),
MetricName: aws.String(r.MetricName),
}
setDimensionFilter(input, r.DimensionFilter)
setAccount(input, r.ResourceRequest)
metrics, err := l.ListMetricsWithPageLimit(input)
if err != nil {
return nil, fmt.Errorf("%v: %w", "unable to call AWS API", err)
}
var dimensionValues []string
response := []resources.ResourceResponse[string]{}
dupCheck := make(map[string]bool)
for _, metric := range metrics {
for _, dim := range metric.Dimensions {
@@ -85,51 +87,33 @@ func (l *ListMetricsService) GetDimensionValuesByDimensionFilter(r resources.Dim
}
dupCheck[*dim.Value] = true
dimensionValues = append(dimensionValues, *dim.Value)
response = append(response, resources.ResourceResponse[string]{AccountId: metric.AccountId, Value: *dim.Value})
}
}
}
sort.Strings(dimensionValues)
return dimensionValues, nil
sort.Slice(response, func(i, j int) bool {
return response[i].Value < response[j].Value
})
return response, nil
}
func (l *ListMetricsService) GetDimensionKeysByNamespace(namespace string) ([]string, error) {
metrics, err := l.ListMetricsWithPageLimit(&cloudwatch.ListMetricsInput{Namespace: aws.String(namespace)})
if err != nil {
return []string{}, err
}
var dimensionKeys []string
dupCheck := make(map[string]struct{})
for _, metric := range metrics {
for _, dim := range metric.Dimensions {
if _, exists := dupCheck[*dim.Name]; exists {
continue
}
dupCheck[*dim.Name] = struct{}{}
dimensionKeys = append(dimensionKeys, *dim.Name)
}
}
return dimensionKeys, nil
}
func (l *ListMetricsService) GetMetricsByNamespace(namespace string) ([]resources.Metric, error) {
metrics, err := l.ListMetricsWithPageLimit(&cloudwatch.ListMetricsInput{Namespace: aws.String(namespace)})
func (l *ListMetricsService) GetMetricsByNamespace(r resources.MetricsRequest) ([]resources.ResourceResponse[resources.Metric], error) {
input := &cloudwatch.ListMetricsInput{Namespace: aws.String(r.Namespace)}
setAccount(input, r.ResourceRequest)
metrics, err := l.ListMetricsWithPageLimit(input)
if err != nil {
return nil, err
}
response := []resources.Metric{}
response := []resources.ResourceResponse[resources.Metric]{}
dupCheck := make(map[string]struct{})
for _, metric := range metrics {
if _, exists := dupCheck[*metric.MetricName]; exists {
continue
}
dupCheck[*metric.MetricName] = struct{}{}
response = append(response, resources.Metric{Name: *metric.MetricName, Namespace: *metric.Namespace})
response = append(response, resources.ResourceResponse[resources.Metric]{AccountId: metric.AccountId, Value: resources.Metric{Name: *metric.MetricName, Namespace: *metric.Namespace}})
}
return response, nil
@@ -146,3 +130,12 @@ func setDimensionFilter(input *cloudwatch.ListMetricsInput, dimensionFilter []*r
input.Dimensions = append(input.Dimensions, df)
}
}
func setAccount(input *cloudwatch.ListMetricsInput, r *resources.ResourceRequest) {
if r != nil && r.AccountId != nil {
input.IncludeLinkedAccounts = aws.Bool(true)
if !r.ShouldTargetAllAccounts() {
input.OwningAccount = r.AccountId
}
}
}

View File

@@ -7,40 +7,55 @@ import (
"github.com/aws/aws-sdk-go/service/cloudwatch"
"github.com/grafana/grafana/pkg/tsdb/cloudwatch/mocks"
"github.com/grafana/grafana/pkg/tsdb/cloudwatch/models/resources"
"github.com/grafana/grafana/pkg/tsdb/cloudwatch/utils"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/mock"
"github.com/stretchr/testify/require"
)
var metricResponse = []*cloudwatch.Metric{
const useLinkedAccountsId = "all"
var metricResponse = []resources.MetricResponse{
{
MetricName: aws.String("CPUUtilization"),
Namespace: aws.String("AWS/EC2"),
Dimensions: []*cloudwatch.Dimension{
{Name: aws.String("InstanceId"), Value: aws.String("i-1234567890abcdef0")},
{Name: aws.String("InstanceType"), Value: aws.String("t2.micro")},
Metric: &cloudwatch.Metric{
MetricName: aws.String("CPUUtilization"),
Namespace: aws.String("AWS/EC2"),
Dimensions: []*cloudwatch.Dimension{
{Name: aws.String("InstanceId"), Value: aws.String("i-1234567890abcdef0")},
{Name: aws.String("InstanceType"), Value: aws.String("t2.micro")},
},
},
},
{
MetricName: aws.String("CPUUtilization"),
Namespace: aws.String("AWS/EC2"),
Dimensions: []*cloudwatch.Dimension{
{Name: aws.String("InstanceId"), Value: aws.String("i-5234567890abcdef0")},
{Name: aws.String("InstanceType"), Value: aws.String("t2.micro")},
{Name: aws.String("AutoScalingGroupName"), Value: aws.String("my-asg")},
Metric: &cloudwatch.Metric{
MetricName: aws.String("CPUUtilization"),
Namespace: aws.String("AWS/EC2"),
Dimensions: []*cloudwatch.Dimension{
{Name: aws.String("InstanceId"), Value: aws.String("i-5234567890abcdef0")},
{Name: aws.String("InstanceType"), Value: aws.String("t2.micro")},
{Name: aws.String("AutoScalingGroupName"), Value: aws.String("my-asg")},
},
},
},
{
MetricName: aws.String("CPUUtilization"),
Namespace: aws.String("AWS/EC2"),
Dimensions: []*cloudwatch.Dimension{
{Name: aws.String("InstanceId"), Value: aws.String("i-64234567890abcdef0")},
{Name: aws.String("InstanceType"), Value: aws.String("t3.micro")},
{Name: aws.String("AutoScalingGroupName"), Value: aws.String("my-asg2")},
Metric: &cloudwatch.Metric{
MetricName: aws.String("CPUUtilization"),
Namespace: aws.String("AWS/EC2"),
Dimensions: []*cloudwatch.Dimension{
{Name: aws.String("InstanceId"), Value: aws.String("i-64234567890abcdef0")},
{Name: aws.String("InstanceType"), Value: aws.String("t3.micro")},
{Name: aws.String("AutoScalingGroupName"), Value: aws.String("my-asg2")},
},
},
},
}
type validateInputTestCase[T resources.DimensionKeysRequest | resources.DimensionValuesRequest] struct {
name string
input T
listMetricsWithPageLimitInput *cloudwatch.ListMetricsInput
}
func TestListMetricsService_GetDimensionKeysByDimensionFilter(t *testing.T) {
t.Run("Should filter out duplicates and keys matching dimension filter keys", func(t *testing.T) {
fakeMetricsClient := &mocks.FakeMetricsClient{}
@@ -51,27 +66,68 @@ func TestListMetricsService_GetDimensionKeysByDimensionFilter(t *testing.T) {
ResourceRequest: &resources.ResourceRequest{Region: "us-east-1"},
Namespace: "AWS/EC2",
MetricName: "CPUUtilization",
DimensionFilter: []*resources.Dimension{
{Name: "InstanceId", Value: ""},
},
DimensionFilter: []*resources.Dimension{{Name: "InstanceId", Value: ""}},
})
require.NoError(t, err)
assert.Equal(t, []string{"InstanceType", "AutoScalingGroupName"}, resp)
assert.Equal(t, []resources.ResourceResponse[string]{{Value: "InstanceType"}, {Value: "AutoScalingGroupName"}}, resp)
})
}
func TestListMetricsService_GetDimensionKeysByNamespace(t *testing.T) {
t.Run("Should filter out duplicates and keys matching dimension filter keys", func(t *testing.T) {
fakeMetricsClient := &mocks.FakeMetricsClient{}
fakeMetricsClient.On("ListMetricsWithPageLimit", mock.Anything).Return(metricResponse, nil)
listMetricsService := NewListMetricsService(fakeMetricsClient)
testCases := []validateInputTestCase[resources.DimensionKeysRequest]{
{
name: "Should set account correctly on list metric input if it cross account is defined on the request",
input: resources.DimensionKeysRequest{
ResourceRequest: &resources.ResourceRequest{Region: "us-east-1", AccountId: utils.Pointer(useLinkedAccountsId)},
Namespace: "AWS/EC2",
MetricName: "CPUUtilization",
DimensionFilter: []*resources.Dimension{{Name: "InstanceId", Value: ""}},
},
listMetricsWithPageLimitInput: &cloudwatch.ListMetricsInput{
MetricName: aws.String("CPUUtilization"),
Namespace: aws.String("AWS/EC2"),
Dimensions: []*cloudwatch.DimensionFilter{{Name: aws.String("InstanceId")}},
IncludeLinkedAccounts: aws.Bool(true),
},
},
{
name: "Should set account correctly on list metric input if single account is defined on the request",
input: resources.DimensionKeysRequest{
ResourceRequest: &resources.ResourceRequest{Region: "us-east-1", AccountId: utils.Pointer("1234567890")},
Namespace: "AWS/EC2",
MetricName: "CPUUtilization",
DimensionFilter: []*resources.Dimension{{Name: "InstanceId", Value: ""}},
},
listMetricsWithPageLimitInput: &cloudwatch.ListMetricsInput{
MetricName: aws.String("CPUUtilization"),
Namespace: aws.String("AWS/EC2"),
Dimensions: []*cloudwatch.DimensionFilter{{Name: aws.String("InstanceId")}},
IncludeLinkedAccounts: aws.Bool(true),
OwningAccount: aws.String("1234567890"),
},
},
{
name: "Should not set namespace and metricName on list metric input if empty strings are set for these in the request",
input: resources.DimensionKeysRequest{
ResourceRequest: &resources.ResourceRequest{Region: "us-east-1"},
Namespace: "",
MetricName: "",
DimensionFilter: []*resources.Dimension{{Name: "InstanceId", Value: ""}},
},
listMetricsWithPageLimitInput: &cloudwatch.ListMetricsInput{Dimensions: []*cloudwatch.DimensionFilter{{Name: aws.String("InstanceId")}}},
},
}
resp, err := listMetricsService.GetDimensionKeysByNamespace("AWS/EC2")
require.NoError(t, err)
assert.Equal(t, []string{"InstanceId", "InstanceType", "AutoScalingGroupName"}, resp)
})
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
fakeMetricsClient := &mocks.FakeMetricsClient{}
fakeMetricsClient.On("ListMetricsWithPageLimit", mock.Anything).Return(metricResponse, nil)
listMetricsService := NewListMetricsService(fakeMetricsClient)
res, err := listMetricsService.GetDimensionKeysByDimensionFilter(tc.input)
require.NoError(t, err)
require.NotEmpty(t, res)
fakeMetricsClient.AssertCalled(t, "ListMetricsWithPageLimit", tc.listMetricsWithPageLimitInput)
})
}
}
func TestListMetricsService_GetDimensionValuesByDimensionFilter(t *testing.T) {
@@ -91,6 +147,52 @@ func TestListMetricsService_GetDimensionValuesByDimensionFilter(t *testing.T) {
})
require.NoError(t, err)
assert.Equal(t, []string{"i-1234567890abcdef0", "i-5234567890abcdef0", "i-64234567890abcdef0"}, resp)
assert.Equal(t, []resources.ResourceResponse[string]{{Value: "i-1234567890abcdef0"}, {Value: "i-5234567890abcdef0"}, {Value: "i-64234567890abcdef0"}}, resp)
})
testCases := []validateInputTestCase[resources.DimensionValuesRequest]{
{
name: "Should set account correctly on list metric input if it cross account is defined on the request",
input: resources.DimensionValuesRequest{
ResourceRequest: &resources.ResourceRequest{Region: "us-east-1", AccountId: utils.Pointer(useLinkedAccountsId)},
Namespace: "AWS/EC2",
MetricName: "CPUUtilization",
DimensionFilter: []*resources.Dimension{{Name: "InstanceId", Value: ""}},
},
listMetricsWithPageLimitInput: &cloudwatch.ListMetricsInput{
MetricName: aws.String("CPUUtilization"),
Namespace: aws.String("AWS/EC2"),
Dimensions: []*cloudwatch.DimensionFilter{{Name: aws.String("InstanceId")}},
IncludeLinkedAccounts: aws.Bool(true),
},
},
{
name: "Should set account correctly on list metric input if single account is defined on the request",
input: resources.DimensionValuesRequest{
ResourceRequest: &resources.ResourceRequest{Region: "us-east-1", AccountId: utils.Pointer("1234567890")},
Namespace: "AWS/EC2",
MetricName: "CPUUtilization",
DimensionFilter: []*resources.Dimension{{Name: "InstanceId", Value: ""}},
},
listMetricsWithPageLimitInput: &cloudwatch.ListMetricsInput{
MetricName: aws.String("CPUUtilization"),
Namespace: aws.String("AWS/EC2"),
Dimensions: []*cloudwatch.DimensionFilter{{Name: aws.String("InstanceId")}},
IncludeLinkedAccounts: aws.Bool(true),
OwningAccount: aws.String("1234567890"),
},
},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
fakeMetricsClient := &mocks.FakeMetricsClient{}
fakeMetricsClient.On("ListMetricsWithPageLimit", mock.Anything).Return(metricResponse, nil)
listMetricsService := NewListMetricsService(fakeMetricsClient)
res, err := listMetricsService.GetDimensionValuesByDimensionFilter(tc.input)
require.NoError(t, err)
require.Empty(t, res)
fakeMetricsClient.AssertCalled(t, "ListMetricsWithPageLimit", tc.listMetricsWithPageLimitInput)
})
}
}

View File

@@ -0,0 +1,53 @@
package services
import (
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/service/cloudwatchlogs"
"github.com/grafana/grafana/pkg/tsdb/cloudwatch/models"
"github.com/grafana/grafana/pkg/tsdb/cloudwatch/models/resources"
"github.com/grafana/grafana/pkg/tsdb/cloudwatch/utils"
)
type LogGroupsService struct {
logGroupsAPI models.CloudWatchLogsAPIProvider
isCrossAccountEnabled bool
}
func NewLogGroupsService(logsClient models.CloudWatchLogsAPIProvider, isCrossAccountEnabled bool) models.LogGroupsProvider {
return &LogGroupsService{logGroupsAPI: logsClient, isCrossAccountEnabled: isCrossAccountEnabled}
}
func (s *LogGroupsService) GetLogGroups(req resources.LogGroupsRequest) ([]resources.ResourceResponse[resources.LogGroup], error) {
input := &cloudwatchlogs.DescribeLogGroupsInput{
Limit: aws.Int64(req.Limit),
LogGroupNamePrefix: req.LogGroupNamePrefix,
}
if s.isCrossAccountEnabled && req.AccountId != nil {
input.IncludeLinkedAccounts = aws.Bool(true)
if req.LogGroupNamePattern != nil {
input.LogGroupNamePrefix = req.LogGroupNamePattern
}
if !req.IsTargetingAllAccounts() {
// TODO: accept more than one account id in search
input.AccountIdentifiers = []*string{req.AccountId}
}
}
response, err := s.logGroupsAPI.DescribeLogGroups(input)
if err != nil || response == nil {
return nil, err
}
var result []resources.ResourceResponse[resources.LogGroup]
for _, logGroup := range response.LogGroups {
result = append(result, resources.ResourceResponse[resources.LogGroup]{
Value: resources.LogGroup{
Arn: *logGroup.Arn,
Name: *logGroup.LogGroupName,
},
AccountId: utils.Pointer(getAccountId(*logGroup.Arn)),
})
}
return result, nil
}

View File

@@ -0,0 +1,200 @@
package services
import (
"fmt"
"testing"
"github.com/aws/aws-sdk-go/service/cloudwatchlogs"
"github.com/grafana/grafana/pkg/tsdb/cloudwatch/mocks"
"github.com/grafana/grafana/pkg/tsdb/cloudwatch/models/resources"
"github.com/grafana/grafana/pkg/tsdb/cloudwatch/utils"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/mock"
)
func Test_GetLogGroups(t *testing.T) {
t.Run("Should map log groups response", func(t *testing.T) {
mockLogsAPI := &mocks.LogsAPI{}
mockLogsAPI.On("DescribeLogGroups", mock.Anything).Return(
&cloudwatchlogs.DescribeLogGroupsOutput{
LogGroups: []*cloudwatchlogs.LogGroup{
{Arn: utils.Pointer("arn:aws:logs:us-east-1:111:log-group:group_a"), LogGroupName: utils.Pointer("group_a")},
{Arn: utils.Pointer("arn:aws:logs:us-east-1:222:log-group:group_b"), LogGroupName: utils.Pointer("group_b")},
{Arn: utils.Pointer("arn:aws:logs:us-east-1:333:log-group:group_c"), LogGroupName: utils.Pointer("group_c")},
},
}, nil)
service := NewLogGroupsService(mockLogsAPI, false)
resp, err := service.GetLogGroups(resources.LogGroupsRequest{})
assert.NoError(t, err)
assert.Equal(t, []resources.ResourceResponse[resources.LogGroup]{
{
AccountId: utils.Pointer("111"),
Value: resources.LogGroup{Arn: "arn:aws:logs:us-east-1:111:log-group:group_a", Name: "group_a"},
},
{
AccountId: utils.Pointer("222"),
Value: resources.LogGroup{Arn: "arn:aws:logs:us-east-1:222:log-group:group_b", Name: "group_b"},
},
{
AccountId: utils.Pointer("333"),
Value: resources.LogGroup{Arn: "arn:aws:logs:us-east-1:333:log-group:group_c", Name: "group_c"},
},
}, resp)
})
t.Run("Should only use LogGroupNamePrefix even if LogGroupNamePattern passed in resource call", func(t *testing.T) {
// TODO: use LogGroupNamePattern when we have accounted for its behavior, still a little unexpected at the moment
mockLogsAPI := &mocks.LogsAPI{}
mockLogsAPI.On("DescribeLogGroups", mock.Anything).Return(&cloudwatchlogs.DescribeLogGroupsOutput{}, nil)
service := NewLogGroupsService(mockLogsAPI, false)
_, err := service.GetLogGroups(resources.LogGroupsRequest{
Limit: 0,
LogGroupNamePrefix: utils.Pointer("test"),
})
assert.NoError(t, err)
mockLogsAPI.AssertCalled(t, "DescribeLogGroups", &cloudwatchlogs.DescribeLogGroupsInput{
Limit: utils.Pointer(int64(0)),
LogGroupNamePrefix: utils.Pointer("test"),
})
})
t.Run("Should call api without LogGroupNamePrefix nor LogGroupNamePattern if not passed in resource call", func(t *testing.T) {
mockLogsAPI := &mocks.LogsAPI{}
mockLogsAPI.On("DescribeLogGroups", mock.Anything).Return(&cloudwatchlogs.DescribeLogGroupsOutput{}, nil)
service := NewLogGroupsService(mockLogsAPI, false)
_, err := service.GetLogGroups(resources.LogGroupsRequest{})
assert.NoError(t, err)
mockLogsAPI.AssertCalled(t, "DescribeLogGroups", &cloudwatchlogs.DescribeLogGroupsInput{
Limit: utils.Pointer(int64(0)),
})
})
t.Run("Should return an error when API returns error", func(t *testing.T) {
mockLogsAPI := &mocks.LogsAPI{}
mockLogsAPI.On("DescribeLogGroups", mock.Anything).Return(&cloudwatchlogs.DescribeLogGroupsOutput{},
fmt.Errorf("some error"))
service := NewLogGroupsService(mockLogsAPI, false)
_, err := service.GetLogGroups(resources.LogGroupsRequest{})
assert.Error(t, err)
assert.Equal(t, "some error", err.Error())
})
}
func Test_GetLogGroups_crossAccountQuerying(t *testing.T) {
t.Run("Should not includeLinkedAccounts or accountId if isCrossAccountEnabled is set to false", func(t *testing.T) {
mockLogsAPI := &mocks.LogsAPI{}
mockLogsAPI.On("DescribeLogGroups", mock.Anything).Return(&cloudwatchlogs.DescribeLogGroupsOutput{}, nil)
service := NewLogGroupsService(mockLogsAPI, false)
_, err := service.GetLogGroups(resources.LogGroupsRequest{
ResourceRequest: resources.ResourceRequest{AccountId: utils.Pointer("accountId")},
LogGroupNamePrefix: utils.Pointer("prefix"),
})
assert.NoError(t, err)
mockLogsAPI.AssertCalled(t, "DescribeLogGroups", &cloudwatchlogs.DescribeLogGroupsInput{
Limit: utils.Pointer(int64(0)),
LogGroupNamePrefix: utils.Pointer("prefix"),
})
})
t.Run("Should replace LogGroupNamePrefix if LogGroupNamePattern passed in resource call", func(t *testing.T) {
mockLogsAPI := &mocks.LogsAPI{}
mockLogsAPI.On("DescribeLogGroups", mock.Anything).Return(&cloudwatchlogs.DescribeLogGroupsOutput{}, nil)
service := NewLogGroupsService(mockLogsAPI, true)
_, err := service.GetLogGroups(resources.LogGroupsRequest{
ResourceRequest: resources.ResourceRequest{AccountId: utils.Pointer("accountId")},
LogGroupNamePrefix: utils.Pointer("prefix"),
LogGroupNamePattern: utils.Pointer("pattern"),
})
assert.NoError(t, err)
mockLogsAPI.AssertCalled(t, "DescribeLogGroups", &cloudwatchlogs.DescribeLogGroupsInput{
AccountIdentifiers: []*string{utils.Pointer("accountId")},
Limit: utils.Pointer(int64(0)),
LogGroupNamePrefix: utils.Pointer("pattern"),
IncludeLinkedAccounts: utils.Pointer(true),
})
})
t.Run("Should includeLinkedAccounts,and accountId if isCrossAccountEnabled is set to true", func(t *testing.T) {
mockLogsAPI := &mocks.LogsAPI{}
mockLogsAPI.On("DescribeLogGroups", mock.Anything).Return(&cloudwatchlogs.DescribeLogGroupsOutput{}, nil)
service := NewLogGroupsService(mockLogsAPI, true)
_, err := service.GetLogGroups(resources.LogGroupsRequest{
ResourceRequest: resources.ResourceRequest{AccountId: utils.Pointer("accountId")},
})
assert.NoError(t, err)
mockLogsAPI.AssertCalled(t, "DescribeLogGroups", &cloudwatchlogs.DescribeLogGroupsInput{
Limit: utils.Pointer(int64(0)),
IncludeLinkedAccounts: utils.Pointer(true),
AccountIdentifiers: []*string{utils.Pointer("accountId")},
})
})
t.Run("Should should not override prefix is there is no logGroupNamePattern", func(t *testing.T) {
mockLogsAPI := &mocks.LogsAPI{}
mockLogsAPI.On("DescribeLogGroups", mock.Anything).Return(&cloudwatchlogs.DescribeLogGroupsOutput{}, nil)
service := NewLogGroupsService(mockLogsAPI, true)
_, err := service.GetLogGroups(resources.LogGroupsRequest{
ResourceRequest: resources.ResourceRequest{AccountId: utils.Pointer("accountId")},
LogGroupNamePrefix: utils.Pointer("prefix"),
})
assert.NoError(t, err)
mockLogsAPI.AssertCalled(t, "DescribeLogGroups", &cloudwatchlogs.DescribeLogGroupsInput{
AccountIdentifiers: []*string{utils.Pointer("accountId")},
Limit: utils.Pointer(int64(0)),
LogGroupNamePrefix: utils.Pointer("prefix"),
IncludeLinkedAccounts: utils.Pointer(true),
})
})
t.Run("Should not includeLinkedAccounts, or accountId if accountId is nil", func(t *testing.T) {
mockLogsAPI := &mocks.LogsAPI{}
mockLogsAPI.On("DescribeLogGroups", mock.Anything).Return(&cloudwatchlogs.DescribeLogGroupsOutput{}, nil)
service := NewLogGroupsService(mockLogsAPI, true)
_, err := service.GetLogGroups(resources.LogGroupsRequest{
LogGroupNamePrefix: utils.Pointer("prefix"),
})
assert.NoError(t, err)
mockLogsAPI.AssertCalled(t, "DescribeLogGroups", &cloudwatchlogs.DescribeLogGroupsInput{
Limit: utils.Pointer(int64(0)),
LogGroupNamePrefix: utils.Pointer("prefix"),
})
})
t.Run("Should should not override prefix is there is no logGroupNamePattern", func(t *testing.T) {
mockLogsAPI := &mocks.LogsAPI{}
mockLogsAPI.On("DescribeLogGroups", mock.Anything).Return(&cloudwatchlogs.DescribeLogGroupsOutput{}, nil)
service := NewLogGroupsService(mockLogsAPI, true)
_, err := service.GetLogGroups(resources.LogGroupsRequest{
ResourceRequest: resources.ResourceRequest{
AccountId: utils.Pointer("accountId"),
},
LogGroupNamePrefix: utils.Pointer("prefix"),
})
assert.NoError(t, err)
mockLogsAPI.AssertCalled(t, "DescribeLogGroups", &cloudwatchlogs.DescribeLogGroupsInput{
AccountIdentifiers: []*string{utils.Pointer("accountId")},
IncludeLinkedAccounts: utils.Pointer(true),
Limit: utils.Pointer(int64(0)),
LogGroupNamePrefix: utils.Pointer("prefix"),
})
})
}

View File

@@ -0,0 +1,27 @@
package services
import (
"strings"
"github.com/grafana/grafana/pkg/tsdb/cloudwatch/models/resources"
)
func valuesToListMetricRespone[T any](values []T) []resources.ResourceResponse[T] {
var response []resources.ResourceResponse[T]
for _, value := range values {
response = append(response, resources.ResourceResponse[T]{Value: value})
}
return response
}
func getAccountId(arn string) string {
// format: arn:partition:service:region:account-id:resource-id
parts := strings.Split(arn, ":")
if len(parts) >= 4 {
return parts[4]
}
return ""
}

View File

@@ -31,7 +31,9 @@ func (e *cloudWatchExecutor) executeTimeSeriesQuery(ctx context.Context, logger
return nil, fmt.Errorf("invalid time range: start time must be before end time")
}
requestQueries, err := models.ParseMetricDataQueries(req.Queries, startTime, endTime, e.features.IsEnabled(featuremgmt.FlagCloudWatchDynamicLabels))
requestQueries, err := models.ParseMetricDataQueries(req.Queries, startTime, endTime,
e.features.IsEnabled(featuremgmt.FlagCloudWatchDynamicLabels),
e.features.IsEnabled(featuremgmt.FlagCloudWatchCrossAccountQuerying))
if err != nil {
return nil, err
}

View File

@@ -588,3 +588,186 @@ func Test_QueryData_response_data_frame_names(t *testing.T) {
})
}
}
func TestTimeSeriesQuery_CrossAccountQuerying(t *testing.T) {
origNewCWClient := NewCWClient
t.Cleanup(func() {
NewCWClient = origNewCWClient
})
var api mocks.MetricsAPI
NewCWClient = func(sess *session.Session) cloudwatchiface.CloudWatchAPI {
return &api
}
im := datasource.NewInstanceManager(func(s backend.DataSourceInstanceSettings) (instancemgmt.Instance, error) {
return DataSource{Settings: models.CloudWatchSettings{}}, nil
})
t.Run("should call GetMetricDataInput with AccountId nil when no AccountId is provided", func(t *testing.T) {
api = mocks.MetricsAPI{}
api.On("GetMetricDataWithContext", mock.Anything, mock.Anything, mock.Anything).Return(&cloudwatch.GetMetricDataOutput{}, nil)
executor := newExecutor(im, newTestConfig(), &fakeSessionCache{}, featuremgmt.WithFeatures(featuremgmt.FlagCloudWatchCrossAccountQuerying))
_, err := executor.QueryData(context.Background(), &backend.QueryDataRequest{
PluginContext: backend.PluginContext{
DataSourceInstanceSettings: &backend.DataSourceInstanceSettings{},
},
Queries: []backend.DataQuery{
{
RefID: "A",
TimeRange: backend.TimeRange{From: time.Now().Add(time.Hour * -2), To: time.Now().Add(time.Hour * -1)},
JSON: json.RawMessage(`{
"type": "timeSeriesQuery",
"subtype": "metrics",
"namespace": "AWS/EC2",
"metricName": "NetworkOut",
"dimensions": {
"InstanceId": "i-00645d91ed77d87ac"
},
"region": "us-east-2",
"id": "a",
"alias": "NetworkOut",
"statistic": "Maximum",
"period": "300",
"hide": false,
"matchExact": true,
"refId": "A"
}`),
},
},
})
require.NoError(t, err)
actualInput, ok := api.Calls[0].Arguments[1].(*cloudwatch.GetMetricDataInput)
require.True(t, ok)
require.Len(t, actualInput.MetricDataQueries, 1)
assert.Nil(t, actualInput.MetricDataQueries[0].Expression)
assert.Nil(t, actualInput.MetricDataQueries[0].AccountId)
})
t.Run("should call GetMetricDataInput with AccountId nil when feature flag is false", func(t *testing.T) {
api = mocks.MetricsAPI{}
api.On("GetMetricDataWithContext", mock.Anything, mock.Anything, mock.Anything).Return(&cloudwatch.GetMetricDataOutput{}, nil)
executor := newExecutor(im, newTestConfig(), &fakeSessionCache{}, featuremgmt.WithFeatures())
_, err := executor.QueryData(context.Background(), &backend.QueryDataRequest{
PluginContext: backend.PluginContext{
DataSourceInstanceSettings: &backend.DataSourceInstanceSettings{},
},
Queries: []backend.DataQuery{
{
RefID: "A",
TimeRange: backend.TimeRange{From: time.Now().Add(time.Hour * -2), To: time.Now().Add(time.Hour * -1)},
JSON: json.RawMessage(`{
"type": "timeSeriesQuery",
"subtype": "metrics",
"namespace": "AWS/EC2",
"metricName": "NetworkOut",
"dimensions": {
"InstanceId": "i-00645d91ed77d87ac"
},
"region": "us-east-2",
"id": "a",
"alias": "NetworkOut",
"statistic": "Maximum",
"period": "300",
"hide": false,
"matchExact": true,
"refId": "A",
"accountId":"some account Id"
}`),
},
},
})
require.NoError(t, err)
actualInput, ok := api.Calls[0].Arguments[1].(*cloudwatch.GetMetricDataInput)
require.True(t, ok)
require.Len(t, actualInput.MetricDataQueries, 1)
assert.Nil(t, actualInput.MetricDataQueries[0].Expression)
assert.Nil(t, actualInput.MetricDataQueries[0].AccountId)
})
t.Run("should call GetMetricDataInput with AccountId in a MetricStat query", func(t *testing.T) {
api = mocks.MetricsAPI{}
api.On("GetMetricDataWithContext", mock.Anything, mock.Anything, mock.Anything).Return(&cloudwatch.GetMetricDataOutput{}, nil)
executor := newExecutor(im, newTestConfig(), &fakeSessionCache{}, featuremgmt.WithFeatures(featuremgmt.FlagCloudWatchCrossAccountQuerying))
_, err := executor.QueryData(context.Background(), &backend.QueryDataRequest{
PluginContext: backend.PluginContext{
DataSourceInstanceSettings: &backend.DataSourceInstanceSettings{},
},
Queries: []backend.DataQuery{
{
RefID: "A",
TimeRange: backend.TimeRange{From: time.Now().Add(time.Hour * -2), To: time.Now().Add(time.Hour * -1)},
JSON: json.RawMessage(`{
"type": "timeSeriesQuery",
"subtype": "metrics",
"namespace": "AWS/EC2",
"metricName": "NetworkOut",
"dimensions": {
"InstanceId": "i-00645d91ed77d87ac"
},
"region": "us-east-2",
"id": "a",
"alias": "NetworkOut",
"statistic": "Maximum",
"period": "300",
"hide": false,
"matchExact": true,
"refId": "A",
"accountId":"some account Id"
}`),
},
},
})
require.NoError(t, err)
actualInput, ok := api.Calls[0].Arguments[1].(*cloudwatch.GetMetricDataInput)
require.True(t, ok)
require.Len(t, actualInput.MetricDataQueries, 1)
require.NotNil(t, actualInput.MetricDataQueries[0].AccountId)
assert.Equal(t, "some account Id", *actualInput.MetricDataQueries[0].AccountId)
})
t.Run("should GetMetricDataInput with AccountId in an inferred search expression query", func(t *testing.T) {
api = mocks.MetricsAPI{}
api.On("GetMetricDataWithContext", mock.Anything, mock.Anything, mock.Anything).Return(&cloudwatch.GetMetricDataOutput{}, nil)
executor := newExecutor(im, newTestConfig(), &fakeSessionCache{}, featuremgmt.WithFeatures(featuremgmt.FlagCloudWatchCrossAccountQuerying))
_, err := executor.QueryData(context.Background(), &backend.QueryDataRequest{
PluginContext: backend.PluginContext{
DataSourceInstanceSettings: &backend.DataSourceInstanceSettings{},
},
Queries: []backend.DataQuery{
{
RefID: "A",
TimeRange: backend.TimeRange{From: time.Now().Add(time.Hour * -2), To: time.Now().Add(time.Hour * -1)},
JSON: json.RawMessage(`{
"type": "timeSeriesQuery",
"subtype": "metrics",
"namespace": "AWS/EC2",
"metricName": "NetworkOut",
"dimensions": {
"InstanceId": "*"
},
"region": "us-east-2",
"id": "a",
"alias": "NetworkOut",
"statistic": "Maximum",
"period": "300",
"hide": false,
"matchExact": true,
"refId": "A",
"accountId":"some account Id"
}`),
},
},
})
require.NoError(t, err)
actualInput, ok := api.Calls[0].Arguments[1].(*cloudwatch.GetMetricDataInput)
require.True(t, ok)
require.Len(t, actualInput.MetricDataQueries, 1)
require.NotNil(t, actualInput.MetricDataQueries[0].Expression)
assert.Equal(t, `REMOVE_EMPTY(SEARCH('{"AWS/EC2","InstanceId"} MetricName="NetworkOut" :aws.AccountId="some account Id"', 'Maximum', 300))`, *actualInput.MetricDataQueries[0].Expression)
})
}

View File

@@ -0,0 +1,3 @@
package utils
func Pointer[T any](arg T) *T { return &arg }

View File

@@ -166,9 +166,6 @@ func (c fakeRGTAClient) GetResourcesPages(in *resourcegroupstaggingapi.GetResour
}
type fakeCheckHealthClient struct {
cloudwatchiface.CloudWatchAPI
cloudwatchlogsiface.CloudWatchLogsAPI
listMetricsPages func(input *cloudwatch.ListMetricsInput, fn func(*cloudwatch.ListMetricsOutput, bool) bool) error
describeLogGroups func(input *cloudwatchlogs.DescribeLogGroupsInput) (*cloudwatchlogs.DescribeLogGroupsOutput, error)
}