IntenseWebs/grafana
3
0
Fork 0
mirror of https://github.com/grafana/grafana.git synced 2024-11-23 09:26:43 -06:00
Code Issues Packages Projects Releases Wiki Activity

CloudWatch: Add support for AWS Metric Insights (#42487)

Browse Source 
* add support for code editor and builder

* refactor cloudwatch migration

* Add tooltip to editor field (#56)

* add tooltip

* add old tooltips

* Bug bash feedback fixes (#58)

* make ASC the default option

* update sql preview whenever sql changes

* don't allow queries without aggregation

* set default value for aggregation

* use new input field

* cleanup

* pr feedback

* prevent unnecessary rerenders

* use frame error instead of main error

* remove not used snapshot

* Use dimension filter in schema picker  (#63)

* use dimension key filter in group by and schema labels

* add dimension filter also to code editor

* add tests

* fix build error

* fix strict error

* remove debug code

* fix annotation editor (#64)

* fix annotation editor

* fix broken test

* revert annotation backend change

* PR feedback (#67)

* pr feedback

* removed dimension filter from group by

* add spacing between common fields and rest

* do not generate deep link for metric queries (#70)

* update docs (#69)

Co-authored-by: Erik Sundell <erik.sundell87@gmail.com>

* fix lint problem caused by merge conflict

Co-authored-by: achatterjee-grafana <70489351+achatterjee-grafana@users.noreply.github.com>
This commit is contained in:
Erik Sundell 2021-11-30 10:53:31 +01:00 committed by GitHub
parent 2a50c029b2
commit bab78a9e64
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
86 changed files with 6487 additions and 1010 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

419
docs/sources/datasources/aws-cloudwatch/_index.md
View File

@ -8,48 +8,23 @@ weight = 200
# AWS CloudWatch data source
Grafana ships with built-in support for CloudWatch. Add it as a data source, then you are ready to build dashboards or use Explore with CloudWatch metrics and CloudWatch Logs.
Grafana ships with built-in support for CloudWatch. This topic describes queries, templates, variables, and other configuration specific to the CloudWatch data source. For instructions on how to add a data source to Grafana, refer to [Add a data source]({{< relref "../add-a-data-source.md" >}}). Only users with the organization admin role can add data sources.
This topic describes queries, templates, variables, and other configuration specific to the CloudWatch data source. For instructions on how to add a data source to Grafana, refer to [Add a data source]({{< relref "../add-a-data-source.md" >}}). Only users with the organization admin role can add data sources.
Once you have added the Cloudwatch data source, you can build dashboards or use Explore with CloudWatch metrics and CloudWatch Logs.
> **Note:** If you are having issues setting up the data source and Grafana is returning undescriptive errors, then check the log file located in /var/log/grafana/grafana.log).
> **Note:** For troubleshooting issues when setting up the Cloudwatch data source, check the `/var/log/grafana/grafana.log` file.
## Cloudwatch settings
## Configure the CloudWatch data source
To access data source settings, hover your mouse over the **Configuration** (gear) icon, then click **Data Sources**, and then click the AWS Cloudwatch data source.
| Name | Description |
| -------------------------- | ----------------------------------------------------------------------------------------------------------------------- |
| `Name` | The data source name. This is how you refer to the data source in panels and queries. |
| `Default` | Default data source means that it will be pre-selected for new panels. |
| `Default Region` | Used in query editor to set region (can be changed on per query basis) |
| `Custom Metrics namespace` | Specify the CloudWatch namespace of Custom metrics |
| `Auth Provider` | Specify the provider to get credentials. |
| `Credentials` profile name | Specify the name of the profile to use (if you use `~/.aws/credentials` file), leave blank for default. |
| `Assume Role Arn` | Specify the ARN of the role to assume |
| `External ID` | If you are assuming a role in another account, that has been created with an external ID, specify the external ID here. |
### X-Ray trace links
Link an X-Ray data source in the "X-Ray trace link" section of the configuration page to automatically add links in your logs when the log contains `@xrayTraceId` field.
![Trace link configuration](/static/img/docs/cloudwatch/xray-trace-link-configuration-8-2.png 'Trace link configuration')
The data source select will contain only existing data source instances of type X-Ray so in order to use this feature you need to have existing X-Ray data source already configured, see [X-Ray docs](https://grafana.com/grafana/plugins/grafana-x-ray-datasource/) for details.
The X-Ray link will then appear in the log details section which is accessible by clicking on the log row either in Explore or in dashboard [Logs panel]({{< relref "../../visualizations/logs-panel.md" >}}). To log the `@xrayTraceId` in your logs see the [AWS X-Ray documentation](https://docs.amazonaws.cn/en_us/xray/latest/devguide/xray-services.html). To provide the field to Grafana your log queries also have to contain the `@xrayTraceId` field, for example using query `fields @message, @xrayTraceId`.
![Trace link in log details](/static/img/docs/cloudwatch/xray-link-log-details-8-2.png 'Trace link in log details')
## Authentication
For authentication options and configuration details, see [AWS authentication]({{< relref "aws-authentication.md" >}}) topic.
## IAM policies
### CloudWatch specific data source configuration
Grafana needs permissions granted via IAM to be able to read CloudWatch metrics
and EC2 tags/instances/regions. You can attach these permissions to IAM roles and
utilize Grafana's built-in support for assuming roles.
#### IAM policies
Grafana needs permissions granted via IAM to be able to read CloudWatch metrics and EC2 tags/instances/regions/alarms. You can attach these permissions to the IAM role or IAM user configured in the previous step.
Here is a minimal policy example:
@ -100,30 +75,53 @@ Here is a minimal policy example:
}
```
## Using the Query Editor
#### Namespaces of Custom Metrics
Grafana is not able to load custom namespaces through the GetMetricData API. If you still want your custom metrics to show up in the fields in the query editor, you can specify the names of the namespaces containing the custom metrics in the _Namespaces of Custom Metrics_ field. The field accepts a multiple namespaces, separated by a comma.
#### X-Ray trace links
Link an X-Ray data source in the "X-Ray trace link" section of the configuration page to automatically add links in your logs when the log contains `@xrayTraceId` field.
![Trace link configuration](/static/img/docs/cloudwatch/xray-trace-link-configuration-8-2.png 'Trace link configuration')
The data source select will contain only existing data source instances of type X-Ray so in order to use this feature you need to have existing X-Ray data source already configured, see [X-Ray docs](https://grafana.com/grafana/plugins/grafana-x-ray-datasource/) for details.
The X-Ray link will then appear in the log details section which is accessible by clicking on the log row either in Explore or in dashboard [Logs panel]({{< relref "../../visualizations/logs-panel.md" >}}). To log the `@xrayTraceId` in your logs see the [AWS X-Ray documentation](https://docs.amazonaws.cn/en_us/xray/latest/devguide/xray-services.html). To provide the field to Grafana your log queries also have to contain the `@xrayTraceId` field, for example using query `fields @message, @xrayTraceId`.
![Trace link in log details](/static/img/docs/cloudwatch/xray-link-log-details-8-2.png 'Trace link in log details')
## CloudWatch query editor
The CloudWatch data source can query data from both CloudWatch metrics and CloudWatch Logs APIs, each with its own specialized query editor. You select which API you want to query with using the query mode switch on top of the editor.
{{< figure src="/static/img/docs/v70/cloudwatch-metrics-query-field.png" max-width="800px" class="docs-image--left" caption="CloudWatch metrics query field" >}}
{{< figure src="/static/img/docs/v70/cloudwatch-logs-query-field.png" max-width="800px" class="docs-image--right" caption="CloudWatch Logs query field" >}}
![CloudWatch API modes](/static/img/docs/cloudwatch/cloudwatch-query-editor-api-modes-8.3.0.png)
## Using the Metric Query Editor
### Metrics query editor
To create a valid query, you need to specify the namespace, metric name and at least one statistic. If `Match Exact` is enabled, you also need to specify all the dimensions of the metric you’re querying, so that the [metric schema](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/search-expression-syntax.html) matches exactly. If `Match Exact` is off, you can specify any number of dimensions by which you’d like to filter. Up to 100 metrics matching your filter criteria will be returned.
The metrics query editor allows you to build two types of queries - **Metric Search** and **Metric Query**.
### Dynamic queries using dimension wildcards
#### Using the Metric Search option
To create a valid Metric Search query specify the namespace, metric name and at least one statistic.
If `Match Exact` is enabled, you also need to specify all the dimensions of the metric you’re querying, so that the [metric schema](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/search-expression-syntax.html) matches exactly. If `Match Exact` is disabled, you can specify any number of dimensions by which you’d like to filter. Up to 100 metrics matching your filter criteria will be returned.
##### Dynamic queries using dimension wildcards
You can monitor a dynamic list of metrics by using the asterisk (\*) wildcard for one or more dimension values.
{{< figure src="/static/img/docs/v65/cloudwatch-dimension-wildcard.png" max-width="800px" class="docs-image--right" caption="CloudWatch dimension wildcard" >}}
![CloudWatch dimension wildcard](/static/img/docs/cloudwatch/cloudwatch-dimension-wildcard-8.3.0.png)
In this example, the query returns all metrics in the namespace `AWS/EC2` with a metric name of `CPUUtilization` and ANY value for the `InstanceId` dimension are queried. This can help you monitor metrics for AWS resources, like EC2 instances or containers. For example, when new instances are created as part of an auto scaling event, they will automatically appear in the graph without needing to track the new instance IDs. This capability is currently limited to retrieving up to 100 metrics.
In this example, the query returns all metrics in the namespace `AWS/EC2` with a metric name of `CPUUtilization` and ANY value for the `InstanceId` dimension are queried. This can help you monitor metrics for AWS resources, like EC2 instances or containers. When new instances are created as part of an auto scaling event, they will automatically appear in the graph without you having to track the new instance IDs. This capability is currently limited to retrieving up to 100 metrics.
Click on `Show Query Preview` to see the search expression that is automatically built to support wildcards. To learn more about search expressions, visit the [CloudWatch documentation](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/search-expression-syntax.html). By default, the search expression is defined in such a way that the queried metrics must match the defined dimension names exactly. This means that in the example only metrics with exactly one dimension with name ‘InstanceId’ will be returned.
You can expand the [Query inspector](https://grafana.com/docs/grafana/latest/panels/queries/#query-inspector-button) button and click `Meta Data` to see the search expression that is automatically built to support wildcards. To learn more about search expressions, visit the [CloudWatch documentation](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/search-expression-syntax.html). By default, the search expression is defined in such a way that the queried metrics must match the defined dimension names exactly. This means that in the example only metrics with exactly one dimension with the name ‘InstanceId’ will be returned.
You can untoggle `Match Exact` to include metrics that have other dimensions defined. Disabling `Match Exact` also creates a search expression even if you don’t use wildcards. We simply search for any metric that matches at least the namespace, metric name, and all defined dimensions.
![CloudWatch Meta Inspector](/static/img/docs/cloudwatch/cloudwatch-meta-inspector-8.3.0.png)
### Multi-value template variables
You can disable `Match Exact` to include metrics that have other dimensions defined. Disabling `Match Exact` also creates a search expression even if you don’t use wildcards. We simply search for any metric that matches at least the namespace, metric name, and all defined dimensions.
##### Multi-value template variables
When defining dimension values based on multi-valued template variables, a search expression is used to query for the matching metrics. This enables the use of multiple template variables in one query and also allows you to use template variables for queries that have the `Match Exact` option disabled.
@ -131,7 +129,7 @@ Search expressions are currently limited to 1024 characters, so your query may f
The use of multi-valued template variables is only supported for dimension values. Using multi-valued template variables for `Region`, `Namespace`, or `Metric Name` is not supported.
### Metric math expressions
##### Metric math expressions
You can create new time series metrics by operating on top of CloudWatch metrics using mathematical functions. Arithmetic operators, unary subtraction and other functions are supported and can be applied to CloudWatch metrics. More details on the available functions can be found on [AWS Metric Math](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/using-metric-math.html)
@ -139,13 +137,7 @@ As an example, if you want to apply arithmetic operations on a metric, you can d
Please note that in the case you use the expression field to reference another query, like `queryA * 2`, it will not be possible to create an alert rule based on that query.
### Period
A period is the length of time associated with a specific Amazon CloudWatch statistic. Periods are defined in numbers of seconds, and valid values for period are 1, 5, 10, 30, or any multiple of 60.
If the period field is left blank or set to `auto`, then it calculates automatically based on the time range and [cloudwatch's retention policy](https://aws.amazon.com/about-aws/whats-new/2016/11/cloudwatch-extends-metrics-retention-and-new-user-interface/). The formula used is `time range in seconds / 2000`, and then it snaps to the next higher value in an array of predefined periods `[60, 300, 900, 3600, 21600, 86400]` after removing periods based on retention. By clicking `Show Query Preview` in the query editor, you can see what period Grafana used.
### Deep linking from Grafana panels to the CloudWatch console
##### Deep linking from Grafana panels to the CloudWatch console
{{< figure src="/static/img/docs/v65/cloudwatch-deep-linking.png" max-width="500px" class="docs-image--right" caption="CloudWatch deep linking" >}}
@ -153,257 +145,140 @@ Left clicking a time series in the panel shows a context menu with a link to `Vi
This feature is not available for metrics that are based on metric math expressions.
## Using the Logs Query Editor
### Using the Metric Query option
To query CloudWatch Logs, select the region and up to 20 log groups which you want to query. Use the main input area to write your query in [CloudWatch Logs Query Language](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CWL_QuerySyntax.html)
> **Note:** This query option is available in Grafana 8.3 and higher versions only.
Metrics Query in the CloudWatch plugin is what is referred to as **Metric Insights** in the AWS console. It's a fast, flexible, SQL-based query engine that enables you to identify trends and patterns across millions of operational metrics in real time. It uses a dialect of SQL. The query syntax is as follows.
```
SELECT FUNCTION(MetricName)
FROM Namespace | SCHEMA(...)
[ WHERE labelKey OPERATOR labelValue [AND|...]]
[ GROUP BY labelKey [, ...]]
[ ORDER BY FUNCTION() [DESC | ASC] ]
[ LIMIT number]
```
The following table provides basic explanation of the query keywords. For details about the Metrics Insights syntax, refer to the [AWS documentation](https://docs.aws.amazon.com/console/cloudwatch/metricsinsights-syntax).
| Keyword | Description |
| ------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `FUNCTION` | Required. Specifies the aggregate function to use, and also specifies the name of the metric that is to be queried. Valid values are AVG, COUNT, MAX, MIN, and SUM |
| `MetricName` | Required. For example, `CPUUtilization`. |
| `FROM` | Required. Specifies the source of the metric. You can specify either the metric namespace that contains the metric that is to be queried, or a SCHEMA table function. Some namespace examples are 1`AWS/EC2`, `AWS/Lambda`. |
| `SCHEMA` | Optional. Allows you to narrow down the query results to only the metrics that is an exact match or to metrics that do noy match. |
| `WHERE` | Optional. Filters the results to only those metrics that match your specified expression. For example, `WHERE InstanceType != 'c3.4xlarge'`. |
| `GROUP BY` | Optional. Groups the query results into multiple time series. For example, `GROUP BY ServiceName`. |
| `ORDER BY` | Optional. Specifies the order of time series that are returned. Options are `ASC`, `DESC`. |
| `LIMIT` | Optional. Limits the number of time series returned. |
For information about limits for the Metrics Insights, please refer to the [AWS documentation](https://docs.aws.amazon.com/console/cloudwatch/metricsinsights).
**Builder mode**
To create a query in Builder mode:
1. Browse and select a metric namespace, metric name, filter, group, and order options using information from the table above.
1. For each of these options, choose from the list of possible options.
Grafana automatically constructs a SQL query based on your selections.
**Code mode**
To create a query in the Code mode:
1. Write your SQL query.
1. To run the query, click the **Run query** above the code editor.
The code editor has a built in autocomplete feature that gives suggestions for keywords, aggregations, namespaces, metrics, labels and label values. The suggestions are shown when hitting space, comma or dollar character. You can also use the keyboard combination CTRL+Space.
![Code editor autocomplete](/static/img/docs/cloudwatch/cloudwatch-code-editor-autocomplete-8.3.0.gif)
> **Note:** Usage of template variables in the code editor might interfere the autocompletion.
### Common metric query editor fields
At the bottom of the metric query editor, you'll find three fields that are common to both _Metric Search_ and _Metric Query_.
#### Id
The GetMetricData API requires that all queries have a unique ID. Use this field to specify an ID of choice. The ID can include numbers, letters, and underscore, and must start with a lowercase letter. If no ID is specified, grafana will generate an ID using the following pattern `query[refId of the current query row]`, e.g `queryA` for the first query row in the panel editor.
The ID can be used to reference queries in Metric Math expressions.
#### Period
A period is the length of time associated with a specific Amazon CloudWatch statistic. Periods are defined in numbers of seconds, and valid values for period are 1, 5, 10, 30, or any multiple of 60.
If the period field is left blank or set to `auto`, then it calculates automatically based on the time range and [cloudwatch's retention policy](https://aws.amazon.com/about-aws/whats-new/2016/11/cloudwatch-extends-metrics-retention-and-new-user-interface/). The formula used is `time range in seconds / 2000`, and then it snaps to the next higher value in an array of predefined periods `[60, 300, 900, 3600, 21600, 86400]` after removing periods based on retention. By clicking `Show Query Preview` in the query editor, you can see what period Grafana used.
#### Alias
The alias field allows you to override the default name of the metric legend.
##### Alias patterns
| Alias Pattern | Description | Example Result |
| ---------------------- | ------------------------------------------------------------- | ---------------- |
| `{{region}}` | returns the region | `us-east-1` |
| `{{period}}` | returns the period | `3000` |
| `{{metric}}` | returns the metric | `CPUUtilization` |
| `{{label}}` | returns the label returned by the API (only in Metric Search) | `i-01343` |
| `{{namespace}}` | returns the namespace (only in Metric Search) | `AWS/EC2` |
| `{{stat}}` | returns the statistic (only in Metric Search) | `Average` |
| `{{[dimension name]}}` | returns the dimension name (only in Metric Search) | `i-01343` |
## Using the Logs query editor
To query CloudWatch Logs:
1. Select the region and up to 20 log groups which you want to query.
1. Use the main input area to write your query in [CloudWatch Logs Query Language](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CWL_QuerySyntax.html).
You can also write queries returning time series data by using the [`stats` command](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CWL_Insights-Visualizing-Log-Data.html). When making `stats` queries in Explore, you have to make sure you are in Metrics Explore mode.
{{< figure src="/static/img/docs/v70/explore-mode-switcher.png" max-width="500px" class="docs-image--right" caption="Explore mode switcher" >}}
To the right of the query input field is a CloudWatch Logs Insights link that opens the CloudWatch Logs Insights console with your query. You can continue exploration there if necessary.
{{< figure src="/static/img/docs/v70/cloudwatch-logs-deep-linking.png" max-width="500px" class="docs-image--right" caption="CloudWatch Logs deep linking" >}}
### Using template variables
The CloudWatch data source supports use of template variables in queries.
For an introduction to templating and template variables, refer to the [Templating]({{< relref "../../variables/_index.md" >}}) documentation.
### Deep linking from Grafana panels to the CloudWatch console
{{< figure src="/static/img/docs/v70/cloudwatch-logs-deep-linking.png" max-width="500px" class="docs-image--right" caption="CloudWatch Logs deep linking" >}}
If you'd like to view your query in the CloudWatch Logs Insights console, simply click the `CloudWatch Logs Insights` button next to the query editor.
If you're not currently logged in to the CloudWatch console, the link will forward you to the login page. The provided link is valid for any account but will only display the right metrics if you're logged in to the account that corresponds to the selected data source in Grafana.
### Alerting
## Alerting
Since CloudWatch Logs queries can return numeric data, for example through the use of the `stats` command, alerts are supported.
For more information on Grafana alerts, refer to [Alerting]({{< relref "../../alerting/_index.md" >}}) documentation.
## Curated dashboards
## Configure CloudWatch with grafana.ini
The updated CloudWatch data source ships with pre-configured dashboards for five of the most popular AWS services:
The Grafana [configuration]({{< relref "../../administration/configuration.md#aws" >}}) file includes an `AWS` section where you can customize the data source.
- Amazon Elastic Compute Cloud `Amazon EC2`,
- Amazon Elastic Block Store `Amazon EBS`,
- AWS Lambda `AWS Lambda`,
- Amazon CloudWatch Logs `Amazon CloudWatch Logs`, and
- Amazon Relational Database Service `Amazon RDS`.
To import the pre-configured dashboards, go to the configuration page of your CloudWatch data source and click on the `Dashboards` tab. Click `Import` for the dashboard you would like to use. To customize the dashboard, we recommend saving the dashboard under a different name, because otherwise the dashboard will be overwritten when a new version of the dashboard is released.
{{< figure src="/static/img/docs/v65/cloudwatch-dashboard-import.png" caption="CloudWatch dashboard import" >}}
## Templated queries
Instead of hard-coding server, application, and sensor names in your metric queries, you can use variables. The variables are listed as dropdown select boxes at the top of the dashboard. These dropdowns make it easy to change the display of data in your dashboard.
For an introduction to templating and template variables, refer to the [Templating]({{< relref "../../variables/_index.md" >}}) documentation.
### Query variable
The CloudWatch data source provides the following queries that you can specify in the `Query` field in the Variable edit view. They allow you to fill a variable's options list with things like `region`, `namespaces`, `metric names` and `dimension keys/values`.
In place of `region` you can specify `default` to use the default region configured in the data source for the query,
e.g. `metrics(AWS/DynamoDB, default)` or `dimension_values(default, ..., ..., ...)`.
Read more about the available dimensions in the [CloudWatch Metrics and Dimensions Reference](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CW_Support_For_AWS.html).
| Name | Description |
| ----------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `regions()` | Returns a list of all AWS regions |
| `namespaces()` | Returns a list of namespaces CloudWatch support. |
| `metrics(namespace, [region])` | Returns a list of metrics in the namespace. (specify region or use "default" for custom metrics) |
| `dimension_keys(namespace)` | Returns a list of dimension keys in the namespace. |
| `dimension_values(region, namespace, metric, dimension_key, [filters])` | Returns a list of dimension values matching the specified `region`, `namespace`, `metric`, `dimension_key` or you can use dimension `filters` to get more specific result as well. |
| `ebs_volume_ids(region, instance_id)` | Returns a list of volume ids matching the specified `region`, `instance_id`. |
| `ec2_instance_attribute(region, attribute_name, filters)` | Returns a list of attributes matching the specified `region`, `attribute_name`, `filters`. |
| `resource_arns(region, resource_type, tags)` | Returns a list of ARNs matching the specified `region`, `resource_type` and `tags`. |
| `statistics()` | Returns a list of all the standard statistics |
For details about the metrics CloudWatch provides, please refer to the [CloudWatch documentation](https://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/CW_Support_For_AWS.html).
#### Examples templated queries
Example dimension queries which will return list of resources for individual AWS Services:
| Query | Service |
| ----------------------------------------------------------------------------------------------------------------------------- | ---------------- |
| `dimension_values(us-east-1,AWS/ELB,RequestCount,LoadBalancerName)` | ELB |
| `dimension_values(us-east-1,AWS/ElastiCache,CPUUtilization,CacheClusterId)` | ElastiCache |
| `dimension_values(us-east-1,AWS/Redshift,CPUUtilization,ClusterIdentifier)` | RedShift |
| `dimension_values(us-east-1,AWS/RDS,CPUUtilization,DBInstanceIdentifier)` | RDS |
| `dimension_values(us-east-1,AWS/S3,BucketSizeBytes,BucketName)` | S3 |
| `dimension_values(us-east-1,CWAgent,disk_used_percent,device,{"InstanceId":"$instance_id"})` | CloudWatch Agent |
| `resource_arns(eu-west-1,elasticloadbalancing:loadbalancer,{"elasticbeanstalk:environment-name":["myApp-dev","myApp-prod"]})` | ELB |
| `resource_arns(eu-west-1,elasticloadbalancing:loadbalancer,{"Component":["$service"],"Environment":["$environment"]})` | ELB |
| `resource_arns(eu-west-1,ec2:instance,{"elasticbeanstalk:environment-name":["myApp-dev","myApp-prod"]})` | EC2 |
## ec2_instance_attribute examples
### JSON filters
The `ec2_instance_attribute` query takes `filters` in JSON format.
You can specify [pre-defined filters of ec2:DescribeInstances](http://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_DescribeInstances.html).
Note that the actual filtering takes place on Amazon's servers, not in Grafana.
Filters syntax:
```javascript
{ "filter_name1": [ "filter_value1" ], "filter_name2": [ "filter_value2" ] }
```
Example `ec2_instance_attribute()` query
```javascript
ec2_instance_attribute(us - east - 1, InstanceId, { 'tag:Environment': ['production'] });
```
### Selecting attributes
Only 1 attribute per instance can be returned. Any flat attribute can be selected (i.e. if the attribute has a single value and isn't an object or array). Below is a list of available flat attributes:
- `AmiLaunchIndex`
- `Architecture`
- `ClientToken`
- `EbsOptimized`
- `EnaSupport`
- `Hypervisor`
- `IamInstanceProfile`
- `ImageId`
- `InstanceId`
- `InstanceLifecycle`
- `InstanceType`
- `KernelId`
- `KeyName`
- `LaunchTime`
- `Platform`
- `PrivateDnsName`
- `PrivateIpAddress`
- `PublicDnsName`
- `PublicIpAddress`
- `RamdiskId`
- `RootDeviceName`
- `RootDeviceType`
- `SourceDestCheck`
- `SpotInstanceRequestId`
- `SriovNetSupport`
- `SubnetId`
- `VirtualizationType`
- `VpcId`
Tags can be selected by prepending the tag name with `Tags.`
Example `ec2_instance_attribute()` query
```javascript
ec2_instance_attribute(us - east - 1, Tags.Name, { 'tag:Team': ['sysops'] });
```
## Using JSON format template variables
Some queries accept filters in JSON format and Grafana supports the conversion of template variables to JSON.
If `env = 'production', 'staging'`, following query will return ARNs of EC2 instances which `Environment` tag is `production` or `staging`.
```javascript
resource_arns(us-east-1, ec2:instance, {"Environment":${env:json}})
```
| Configuration option | Description |
| ------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `allowed_auth_providers` | Specifies which authentication providers are allowed for the CloudWatch data source. The following providers are enabled by default in OSS Grafana: `default` (AWS SDK default), keys (Access and secret key), credentials (Credentials file), ec2_IAM_role (EC2 IAM role). |
| `assume_role_enabled` | Allows you to disable `assume role (ARN)` in the CloudWatch data source. By default, assume role (ARN) is enabled for OSS Grafana. |
| `list_metrics_page_limit` | When a custom namespace is specified in the query editor, the [List Metrics API](https://docs.aws.amazon.com/AmazonCloudWatch/latest/APIReference/API_ListMetrics.html) is used to populate the _Metrics_ field and the _Dimension_ fields. The API is paginated and returns up to 500 results per page. The CloudWatch data source also limits the number of pages to 500. However, you can change this limit using the `list_metrics_page_limit` variable in the [grafana configuration file](https://grafana.com/docs/grafana/latest/administration/configuration/#aws). |
## Pricing
The Amazon CloudWatch data source for Grafana uses the `ListMetrics` and `GetMetricData` CloudWatch API calls to list and retrieve metrics.
Pricing for CloudWatch Logs is based on the amount of data ingested, archived, and analyzed via CloudWatch Logs Insights queries.
Please see the [CloudWatch pricing page](https://aws.amazon.com/cloudwatch/pricing/) for more details.
Every time you pick a dimension in the query editor Grafana will issue a ListMetrics request. Whenever you make a change to the queries in the query editor, one new request to GetMetricData will be issued.
Every time you pick a dimension in the query editor Grafana will issue a ListMetrics request.
Whenever you make a change to the queries in the query editor, one new request to GetMetricData will be issued.
In Grafana version 6.5 or higher, all API requests to GetMetricStatistics have been replaced with calls to GetMetricData to provide better support for CloudWatch metric math and enables the automatic generation of search expressions when using wildcards or disabling the `Match Exact` option. While GetMetricStatistics qualified for the CloudWatch API free tier, this is not the case for GetMetricData calls.
Please note that for Grafana version 6.5 or higher, all API requests to GetMetricStatistics have been replaced with calls to GetMetricData. This change enables better support for CloudWatch metric math and enables the automatic generation of search expressions when using wildcards or disabling the `Match Exact` option. While GetMetricStatistics qualified for the CloudWatch API free tier, this is not the case for GetMetricData calls. For more information, please refer to the [CloudWatch pricing page](https://aws.amazon.com/cloudwatch/pricing/).
For more information, please refer to the [CloudWatch pricing page](https://aws.amazon.com/cloudwatch/pricing/).
## Service quotas
AWS defines quotas, or limits, for resources, actions, and items in your AWS account. Depending on the number of queries in your dashboard and the number of users accessing the dashboard, you may reach the usage limits for various CloudWatch and CloudWatch Logs resources. Note that quotas are defined per account and per region. If you're using multiple regions or have set up more than one CloudWatch data source to query against multiple accounts, you need to request a quota increase for each account and each region in which you hit the limit.
To request a quota increase, visit the [AWS Service Quotas console](https://console.aws.amazon.com/servicequotas/home?r#!/services/monitoring/quotas/L-5E141212).
To request a quota increase, visit the [AWS Service Quotas console](https://console.aws.amazon.com/servicequotas/home?r#!/services/monitoring/quotas/L-5E141212). For more information, refer to the AWS documentation for [Service Quotas](https://docs.aws.amazon.com/servicequotas/latest/userguide/intro.html) and [CloudWatch limits](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/cloudwatch_limits.html).
Please see the AWS documentation for [Service Quotas](https://docs.aws.amazon.com/servicequotas/latest/userguide/intro.html) and [CloudWatch limits](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/cloudwatch_limits.html) for more information.
## Configure the data source with grafana.ini
The Grafana [configuration]({{< relref "../../administration/configuration.md#aws" >}}) file includes an `AWS` section where you can customize the data source.
### allowed_auth_providers
Specify which authentication providers are allowed for the CloudWatch data source. The following providers are enabled by default in OSS Grafana: `default` (AWS SDK default), keys (Access and secret key), credentials (Credentials file), ec2_IAM_role (EC2 IAM role).
### assume_role_enabled
Allows you to disable `assume role (ARN)` in the CloudWatch data source. By default, assume role (ARN) is enabled for OSS Grafana.
### list_metrics_page_limit
When a custom namespace is specified in the query editor, the [List Metrics API](https://docs.aws.amazon.com/AmazonCloudWatch/latest/APIReference/API_ListMetrics.html) is used to populate the _Metrics_ field and the _Dimension_ fields. The API is paginated and returns up to 500 results per page. The CloudWatch data source also limits the number of pages to 500. However, you can change this limit using the `list_metrics_page_limit` variable in the [grafana configuration file](https://grafana.com/docs/grafana/latest/administration/configuration/#aws).
## Configure the data source with provisioning
You can configure the CloudWatch data source by customizing configuration files in Grafana's provisioning system. To know more about provisioning and learn about available configuration options, refer to the [Provisioning Grafana]({{< relref "../../administration/provisioning/#datasources" >}}) topic.
Here are some provisioning examples for this data source.
### Using AWS SDK (default)
```yaml
apiVersion: 1
datasources:
- name: CloudWatch
type: cloudwatch
jsonData:
authType: default
defaultRegion: eu-west-2
```
### Using credentials' profile name (non-default)
```yaml
apiVersion: 1
datasources:
- name: CloudWatch
type: cloudwatch
jsonData:
authType: credentials
defaultRegion: eu-west-2
customMetricsNamespaces: 'CWAgent,CustomNameSpace'
profile: secondary
```
### Using `accessKey` and `secretKey`
```yaml
apiVersion: 1
datasources:
- name: CloudWatch
type: cloudwatch
jsonData:
authType: keys
defaultRegion: eu-west-2
secureJsonData:
accessKey: '<your access key>'
secretKey: '<your secret key>'
```
### Using AWS SDK Default and ARN of IAM Role to Assume
```yaml
apiVersion: 1
datasources:
- name: CloudWatch
type: cloudwatch
jsonData:
authType: default
assumeRoleArn: arn:aws:iam::123456789012:root
defaultRegion: eu-west-2
```

34
docs/sources/datasources/aws-cloudwatch/aws-authentication.md
View File

@ -3,7 +3,7 @@ title = "Authentication"
description = "AWS authentication"
keywords = ["grafana", "aws", "authentication"]
aliases = ["/docs/grafana/latest/datasources/cloudwatch"]
weight = 205
weight = 05
+++
# AWS authentication
@ -12,6 +12,14 @@ Requests from a Grafana plugin to AWS are made on behalf of an IAM role or an IA
All requests to AWS APIs are performed on the server side by the Grafana backend using the official AWS SDK.
This topic has the following sections:
- [Authentication methods](#authentication-methods)
- [Assuming a role](#assuming-a-role)
- [Endpoint](#endpoint)
- [AWS credentials file](#aws-credentials-file)
- [EKS IAM roles for service accounts](#eks-iam-roles-for-service-accounts)
## Authentication methods
You can use one of the following authentication methods. Currently, `AWS SDK Default`, `Credentials file` and `Access and secret key` are enabled by default in open source Grafana. You can enable/disable them if necessary if you have server configuration access. For more information, refer to [allowed_auth_providers]({{< relref "../../administration/configuration.md#allowed_auth_providers" >}}) documentation.
@ -36,18 +44,7 @@ If you are assuming a role in another account that was created with an external
## Endpoint
The `Endpoint` field allows you to specify a custom endpoint URL that overrides the default generated endpoint for the CloudWatch API. Leave this field blank if you want to use the default generated endpoint. For more information on why and how to use Service endpoints, refer to the [AWS service endpoints documentation](https://docs.aws.amazon.com/general/latest/gr/rande.html).
## EKS IAM roles for service accounts
The Grafana process in the container runs as user 472 (called "grafana"). When Kubernetes mounts your projected credentials, they will by default only be available to the root user. To allow user 472 to access the credentials (and avoid falling back to the IAM role attached to the EC2 instance), you need to provide a [security context](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/) for your pod.
```yaml
securityContext:
fsGroup: 472
runAsUser: 472
runAsGroup: 472
```
The `Endpoint` field allows you to specify a custom endpoint URL that overrides the default generated endpoint for the AWS service API. Leave this field blank if you want to use the default generated endpoint. For more information on why and how to use Service endpoints, refer to the [AWS service endpoints documentation](https://docs.aws.amazon.com/general/latest/gr/rande.html).
## AWS credentials file
@ -63,3 +60,14 @@ aws_access_key_id = asdsadasdasdasd
aws_secret_access_key = dasdasdsadasdasdasdsa
region = us-west-2
```
## EKS IAM roles for service accounts
The Grafana process in the container runs as user 472 (called "grafana"). When Kubernetes mounts your projected credentials, they will by default only be available to the root user. To allow user 472 to access the credentials (and avoid falling back to the IAM role attached to the EC2 instance), you need to provide a [security context](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/) for your pod.
```yaml
securityContext:
fsGroup: 472
runAsUser: 472
runAsGroup: 472
```

27
docs/sources/datasources/aws-cloudwatch/preconfig-cloudwatch-dashboards.md Normal file
View File

@ -0,0 +1,27 @@
+++
title = "Curated CloudWatch dashboards"
description = "Guide for using AWS CloudWatch in Grafana"
keywords = ["grafana", "stackdriver", "google", "guide", "cloud", "monitoring"]
aliases = ["/docs/grafana/latest/datasources/cloudwatch"]
weight = 15
+++
# Curated CloudWatch dashboards
The updated CloudWatch data source ships with pre-configured dashboards for five of the most popular AWS services:
- Amazon Elastic Compute Cloud `Amazon EC2`,
- Amazon Elastic Block Store `Amazon EBS`,
- AWS Lambda `AWS Lambda`,
- Amazon CloudWatch Logs `Amazon CloudWatch Logs`, and
- Amazon Relational Database Service `Amazon RDS`.
To import curatedd dashboards:
1. On the configuration page of your CloudWatch data source, click the **Dashboards** tab.
1. Click **Import** for the dashboard you would like to use.
In case you want to customize a dashboard, we recommend that you save it under a different name. Otherwise the dashboard will be overwritten when a new version of the dashboard is released.
{{< figure src="/static/img/docs/v65/cloudwatch-dashboard-import.png" caption="CloudWatch dashboard import" >}}

68
docs/sources/datasources/aws-cloudwatch/provision-cloudwatch.md Normal file
View File

@ -0,0 +1,68 @@
+++
title = "Provision CloudWatch"
description = "Guide for provisioning CloudWatch"
weight = 400
aliases = ["/docs/grafana/latest/datasources/cloudwatch"]
+++
# Provision CloudWatch data source
You can configure the CloudWatch data source by customizing configuration files in Grafana's provisioning system. To know more about provisioning and learn about available configuration options, refer to the [Provisioning Grafana]({{< relref "../../administration/provisioning/#datasources" >}}) topic.
Here are some provisioning examples for this data source.
## Using AWS SDK (default)
```yaml
apiVersion: 1
datasources:
- name: CloudWatch
type: cloudwatch
jsonData:
authType: default
defaultRegion: eu-west-2
```
## Using credentials' profile name (non-default)
```yaml
apiVersion: 1
datasources:
- name: CloudWatch
type: cloudwatch
jsonData:
authType: credentials
defaultRegion: eu-west-2
customMetricsNamespaces: 'CWAgent,CustomNameSpace'
profile: secondary
```
## Using accessKey and secretKey
```yaml
apiVersion: 1
datasources:
- name: CloudWatch
type: cloudwatch
jsonData:
authType: keys
defaultRegion: eu-west-2
secureJsonData:
accessKey: '<your access key>'
secretKey: '<your secret key>'
```
## Using AWS SDK Default and ARN of IAM Role to Assume
```yaml
apiVersion: 1
datasources:
- name: CloudWatch
type: cloudwatch
jsonData:
authType: default
assumeRoleArn: arn:aws:iam::123456789012:root
defaultRegion: eu-west-2
```

122
docs/sources/datasources/aws-cloudwatch/template-queries-cloudwatch.md Normal file
View File

@ -0,0 +1,122 @@
+++
title = "Template variables in CloudWatch query"
description = "Template variables in CloudWatch queryh"
weight = 10
aliases = ["/docs/grafana/latest/datasources/cloudwatch"]
+++
# Using template variables in CloudWatch queries
Instead of hard-coding server, application, and sensor names in your metric queries, you can use variables. The variables are listed as dropdown select boxes at the top of the dashboard. These dropdowns make it easy to change the display of data in your dashboard.
For an introduction to templating and template variables, refer to the [Templating]({{< relref "../../variables/_index.md" >}}) documentation.
## Query variable
The CloudWatch data source provides the following queries that you can specify in the `Query` field in the Variable edit view. They allow you to fill a variable's options list with things like `region`, `namespaces`, `metric names` and `dimension keys/values`.
In place of `region` you can specify `default` to use the default region configured in the data source for the query,
e.g. `metrics(AWS/DynamoDB, default)` or `dimension_values(default, ..., ..., ...)`.
Read more about the available dimensions in the [CloudWatch Metrics and Dimensions Reference](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CW_Support_For_AWS.html).
| Name | Description |
| ----------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `regions()` | Returns a list of all AWS regions |
| `namespaces()` | Returns a list of namespaces CloudWatch support. |
| `metrics(namespace, [region])` | Returns a list of metrics in the namespace. (specify region or use "default" for custom metrics) |
| `dimension_keys(namespace)` | Returns a list of dimension keys in the namespace. |
| `dimension_values(region, namespace, metric, dimension_key, [filters])` | Returns a list of dimension values matching the specified `region`, `namespace`, `metric`, `dimension_key` or you can use dimension `filters` to get more specific result as well. |
| `ebs_volume_ids(region, instance_id)` | Returns a list of volume ids matching the specified `region`, `instance_id`. |
| `ec2_instance_attribute(region, attribute_name, filters)` | Returns a list of attributes matching the specified `region`, `attribute_name`, `filters`. |
| `resource_arns(region, resource_type, tags)` | Returns a list of ARNs matching the specified `region`, `resource_type` and `tags`. |
| `statistics()` | Returns a list of all the standard statistics |
For details about the metrics CloudWatch provides, please refer to the [CloudWatch documentation](https://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/CW_Support_For_AWS.html).
## Example of templated queries
Here is an example of the dimension queries which will return list of resources for individual AWS Services:
| Query | Service |
| ----------------------------------------------------------------------------------------------------------------------------- | ---------------- |
| `dimension_values(us-east-1,AWS/ELB,RequestCount,LoadBalancerName)` | ELB |
| `dimension_values(us-east-1,AWS/ElastiCache,CPUUtilization,CacheClusterId)` | ElastiCache |
| `dimension_values(us-east-1,AWS/Redshift,CPUUtilization,ClusterIdentifier)` | RedShift |
| `dimension_values(us-east-1,AWS/RDS,CPUUtilization,DBInstanceIdentifier)` | RDS |
| `dimension_values(us-east-1,AWS/S3,BucketSizeBytes,BucketName)` | S3 |
| `dimension_values(us-east-1,CWAgent,disk_used_percent,device,{"InstanceId":"$instance_id"})` | CloudWatch Agent |
| `resource_arns(eu-west-1,elasticloadbalancing:loadbalancer,{"elasticbeanstalk:environment-name":["myApp-dev","myApp-prod"]})` | ELB |
| `resource_arns(eu-west-1,elasticloadbalancing:loadbalancer,{"Component":["$service"],"Environment":["$environment"]})` | ELB |
| `resource_arns(eu-west-1,ec2:instance,{"elasticbeanstalk:environment-name":["myApp-dev","myApp-prod"]})` | EC2 |
## Using JSON format template variables
Some queries accept filters in JSON format and Grafana supports the conversion of template variables to JSON.
If `env = 'production', 'staging'`, following query will return ARNs of EC2 instances which `Environment` tag is `production` or `staging`.
```javascript
resource_arns(us-east-1, ec2:instance, {"Environment":${env:json}})
```
## ec2_instance_attribute examples
### JSON filters
The `ec2_instance_attribute` query takes `filters` in JSON format.
You can specify [pre-defined filters of ec2:DescribeInstances](http://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_DescribeInstances.html).
Note that the actual filtering takes place on Amazon's servers, not in Grafana.
Filters syntax:
```javascript
{ "filter_name1": [ "filter_value1" ], "filter_name2": [ "filter_value2" ] }
```
Example `ec2_instance_attribute()` query
```javascript
ec2_instance_attribute(us - east - 1, InstanceId, { 'tag:Environment': ['production'] });
```
### Selecting attributes
Only 1 attribute per instance can be returned. Any flat attribute can be selected (i.e. if the attribute has a single value and isn't an object or array). Below is a list of available flat attributes:
- `AmiLaunchIndex`
- `Architecture`
- `ClientToken`
- `EbsOptimized`
- `EnaSupport`
- `Hypervisor`
- `IamInstanceProfile`
- `ImageId`
- `InstanceId`
- `InstanceLifecycle`
- `InstanceType`
- `KernelId`
- `KeyName`
- `LaunchTime`
- `Platform`
- `PrivateDnsName`
- `PrivateIpAddress`
- `PublicDnsName`
- `PublicIpAddress`
- `RamdiskId`
- `RootDeviceName`
- `RootDeviceType`
- `SourceDestCheck`
- `SpotInstanceRequestId`
- `SriovNetSupport`
- `SubnetId`
- `VirtualizationType`
- `VpcId`
Tags can be selected by prepending the tag name with `Tags.`
Example `ec2_instance_attribute()` query
```javascript
ec2_instance_attribute(us - east - 1, Tags.Name, { 'tag:Team': ['sysops'] });
```

61
pkg/tsdb/cloudwatch/cloudwatch_query.go
View File

@ -9,34 +9,57 @@ import (
)
type cloudWatchQuery struct {
RefId string
Region string
Id string
Namespace string
MetricName string
Statistic string
Expression string
ReturnData bool
Dimensions map[string][]string
Period int
Alias string
MatchExact bool
UsedExpression string
RefId string
Region string
Id string
Namespace string
MetricName string
Statistic string
Expression string
SqlExpression string
ReturnData bool
Dimensions map[string][]string
Period int
Alias string
MatchExact bool
UsedExpression string
MetricQueryType metricQueryType
MetricEditorMode metricEditorMode
}
func (q *cloudWatchQuery) getGMDAPIMode() gmdApiMode {
if q.MetricQueryType == MetricQueryTypeSearch && q.MetricEditorMode == MetricEditorModeBuilder {
if q.isInferredSearchExpression() {
return GMDApiModeInferredSearchExpression
}
return GMDApiModeMetricStat
} else if q.MetricQueryType == MetricQueryTypeSearch && q.MetricEditorMode == MetricEditorModeRaw {
return GMDApiModeMathExpression
} else if q.MetricQueryType == MetricQueryTypeQuery {
return GMDApiModeSQLExpression
}
plog.Warn("Could not resolve CloudWatch metric query type. Falling back to metric stat.", "query", q)
return GMDApiModeMetricStat
}
func (q *cloudWatchQuery) isMathExpression() bool {
return q.Expression != "" && !q.isUserDefinedSearchExpression()
return q.MetricQueryType == MetricQueryTypeSearch && q.MetricEditorMode == MetricEditorModeRaw && !q.isUserDefinedSearchExpression()
}
func (q *cloudWatchQuery) isSearchExpression() bool {
return q.isUserDefinedSearchExpression() || q.isInferredSearchExpression()
return q.MetricQueryType == MetricQueryTypeSearch && (q.isUserDefinedSearchExpression() || q.isInferredSearchExpression())
}
func (q *cloudWatchQuery) isUserDefinedSearchExpression() bool {
return strings.Contains(q.Expression, "SEARCH(")
return q.MetricQueryType == MetricQueryTypeSearch && q.MetricEditorMode == MetricEditorModeRaw && strings.Contains(q.Expression, "SEARCH(")
}
func (q *cloudWatchQuery) isInferredSearchExpression() bool {
if q.MetricQueryType != MetricQueryTypeSearch || q.MetricEditorMode != MetricEditorModeBuilder {
return false
}
if len(q.Dimensions) == 0 {
return !q.MatchExact
}
@ -58,6 +81,10 @@ func (q *cloudWatchQuery) isInferredSearchExpression() bool {
}
func (q *cloudWatchQuery) isMultiValuedDimensionExpression() bool {
if q.MetricQueryType != MetricQueryTypeSearch || q.MetricEditorMode != MetricEditorModeBuilder {
return false
}
for _, values := range q.Dimensions {
for _, v := range values {
if v == "*" {
@ -74,7 +101,7 @@ func (q *cloudWatchQuery) isMultiValuedDimensionExpression() bool {
}
func (q *cloudWatchQuery) buildDeepLink(startTime time.Time, endTime time.Time) (string, error) {
if q.isMathExpression() {
if q.isMathExpression() || q.MetricQueryType == MetricQueryTypeQuery {
return "", nil
}

32
pkg/tsdb/cloudwatch/cloudwatch_query_test.go
View File

@ -2,11 +2,36 @@ package cloudwatch
import (
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestCloudWatchQuery(t *testing.T) {
t.Run("Deeplink is not generated for MetricQueryTypeQuery", func(t *testing.T) {
startTime := time.Now()
endTime := startTime.Add(2 * time.Hour)
query := &cloudWatchQuery{
RefId: "A",
Region: "us-east-1",
Expression: "",
Statistic: "Average",
Period: 300,
Id: "id1",
MatchExact: true,
Dimensions: map[string][]string{
"InstanceId": {"i-12345678"},
},
MetricQueryType: MetricQueryTypeQuery,
MetricEditorMode: MetricEditorModeBuilder,
}
deepLink, err := query.buildDeepLink(startTime, endTime)
require.NoError(t, err)
assert.Empty(t, deepLink)
})
t.Run("SEARCH(someexpression) was specified in the query editor", func(t *testing.T) {
query := &cloudWatchQuery{
RefId: "A",
@ -107,14 +132,12 @@ func TestCloudWatchQuery(t *testing.T) {
query.MatchExact = false
assert.True(t, query.isSearchExpression(), "Expected a search expression")
assert.False(t, query.isMathExpression(), "Expected not math expression")
assert.False(t, query.isMetricStat(), "Expected not metric stat")
})
t.Run("Match exact is true", func(t *testing.T) {
query.MatchExact = true
assert.False(t, query.isSearchExpression(), "Exxpected not search expression")
assert.False(t, query.isMathExpression(), "Expected not math expression")
assert.True(t, query.isMetricStat(), "Expected a metric stat")
})
})
@ -134,10 +157,5 @@ func TestCloudWatchQuery(t *testing.T) {
assert.True(t, query.isSearchExpression(), "Expected search expression")
assert.False(t, query.isMathExpression(), "Expected not math expression")
assert.False(t, query.isMetricStat(), "Expected not metric stat")
})
}
func (q *cloudWatchQuery) isMetricStat() bool {
return !q.isSearchExpression() && !q.isMathExpression()
}

46
pkg/tsdb/cloudwatch/metric_data_query_builder.go
View File

@ -16,30 +16,32 @@ func (e *cloudWatchExecutor) buildMetricDataQuery(query *cloudWatchQuery) (*clou
ReturnData: aws.Bool(query.ReturnData),
}
if query.Expression != "" {
mdq.Expression = aws.String(query.Expression)
switch query.getGMDAPIMode() {
case GMDApiModeMathExpression:
mdq.Period = aws.Int64(int64(query.Period))
} else {
if query.isSearchExpression() {
mdq.Expression = aws.String(buildSearchExpression(query, query.Statistic))
} else {
mdq.MetricStat = &cloudwatch.MetricStat{
Metric: &cloudwatch.Metric{
Namespace: aws.String(query.Namespace),
MetricName: aws.String(query.MetricName),
Dimensions: make([]*cloudwatch.Dimension, 0),
},
Period: aws.Int64(int64(query.Period)),
}
for key, values := range query.Dimensions {
mdq.MetricStat.Metric.Dimensions = append(mdq.MetricStat.Metric.Dimensions,
&cloudwatch.Dimension{
Name: aws.String(key),
Value: aws.String(values[0]),
})
}
mdq.MetricStat.Stat = aws.String(query.Statistic)
mdq.Expression = aws.String(query.Expression)
case GMDApiModeSQLExpression:
mdq.Period = aws.Int64(int64(query.Period))
mdq.Expression = aws.String(query.SqlExpression)
case GMDApiModeInferredSearchExpression:
mdq.Expression = aws.String(buildSearchExpression(query, query.Statistic))
case GMDApiModeMetricStat:
mdq.MetricStat = &cloudwatch.MetricStat{
Metric: &cloudwatch.Metric{
Namespace: aws.String(query.Namespace),
MetricName: aws.String(query.MetricName),
Dimensions: make([]*cloudwatch.Dimension, 0),
},
Period: aws.Int64(int64(query.Period)),
}
for key, values := range query.Dimensions {
mdq.MetricStat.Metric.Dimensions = append(mdq.MetricStat.Metric.Dimensions,
&cloudwatch.Dimension{
Name: aws.String(key),
Value: aws.String(values[0]),
})
}
mdq.MetricStat.Stat = aws.String(query.Statistic)
}
if mdq.Expression != nil {

78
pkg/tsdb/cloudwatch/metric_data_query_builder_test.go
View File

@ -7,21 +7,63 @@ import (
"github.com/stretchr/testify/require"
)
func TestMetricDataQueryBuilder_buildSearchExpression(t *testing.T) {
func TestMetricDataQueryBuilder(t *testing.T) {
t.Run("buildMetricDataQuery", func(t *testing.T) {
t.Run("should use metric stat", func(t *testing.T) {
executor := newExecutor(nil, nil, newTestConfig(), fakeSessionCache{})
query := getBaseQuery()
query.MetricEditorMode = MetricEditorModeBuilder
query.MetricQueryType = MetricQueryTypeSearch
mdq, err := executor.buildMetricDataQuery(query)
require.NoError(t, err)
require.Empty(t, mdq.Expression)
assert.Equal(t, query.MetricName, *mdq.MetricStat.Metric.MetricName)
assert.Equal(t, query.Namespace, *mdq.MetricStat.Metric.Namespace)
})
t.Run("should use custom built expression", func(t *testing.T) {
executor := newExecutor(nil, nil, newTestConfig(), fakeSessionCache{})
query := getBaseQuery()
query.MetricEditorMode = MetricEditorModeBuilder
query.MetricQueryType = MetricQueryTypeSearch
query.MatchExact = false
mdq, err := executor.buildMetricDataQuery(query)
require.NoError(t, err)
require.Nil(t, mdq.MetricStat)
assert.Equal(t, `REMOVE_EMPTY(SEARCH('Namespace="AWS/EC2" MetricName="CPUUtilization" "LoadBalancer"="lb1"', '', 300))`, *mdq.Expression)
})
t.Run("should use sql expression", func(t *testing.T) {
executor := newExecutor(nil, nil, newTestConfig(), fakeSessionCache{})
query := getBaseQuery()
query.MetricEditorMode = MetricEditorModeRaw
query.MetricQueryType = MetricQueryTypeQuery
query.SqlExpression = `SELECT SUM(CPUUTilization) FROM "AWS/EC2"`
mdq, err := executor.buildMetricDataQuery(query)
require.NoError(t, err)
require.Nil(t, mdq.MetricStat)
assert.Equal(t, query.SqlExpression, *mdq.Expression)
})
t.Run("should use user defined math expression", func(t *testing.T) {
executor := newExecutor(nil, nil, newTestConfig(), fakeSessionCache{})
query := getBaseQuery()
query.MetricEditorMode = MetricEditorModeRaw
query.MetricQueryType = MetricQueryTypeSearch
query.Expression = `SUM(x+y)`
mdq, err := executor.buildMetricDataQuery(query)
require.NoError(t, err)
require.Nil(t, mdq.MetricStat)
assert.Equal(t, query.Expression, *mdq.Expression)
})
t.Run("should set period in user defined expression", func(t *testing.T) {
executor := newExecutor(nil, nil, newTestConfig(), fakeSessionCache{})
query := &cloudWatchQuery{
Namespace: "AWS/EC2",
MetricName: "CPUUtilization",
Dimensions: map[string][]string{
"LoadBalancer": {"lb1"},
},
Period: 300,
Expression: "SUM([a,b])",
MatchExact: true,
}
query := getBaseQuery()
query.MetricEditorMode = MetricEditorModeRaw
query.MetricQueryType = MetricQueryTypeSearch
query.MatchExact = false
query.Expression = `SUM([a,b])`
mdq, err := executor.buildMetricDataQuery(query)
require.NoError(t, err)
require.Nil(t, mdq.MetricStat)
@ -235,3 +277,17 @@ func TestMetricDataQueryBuilder_buildSearchExpression(t *testing.T) {
assert.Contains(t, res, `lb4\"\"`, "Expected escape double quotes")
})
}
func getBaseQuery() *cloudWatchQuery {
query := &cloudWatchQuery{
Namespace: "AWS/EC2",
MetricName: "CPUUtilization",
Dimensions: map[string][]string{
"LoadBalancer": {"lb1"},
},
Period: 300,
Expression: "",
MatchExact: true,
}
return query
}

40
pkg/tsdb/cloudwatch/request_parser.go
View File

@ -143,6 +143,7 @@ func parseRequestQuery(model *simplejson.Json, refId string, startTime time.Time
id = fmt.Sprintf("query%s", refId)
}
expression := model.Get("expression").MustString("")
sqlExpression := model.Get("sqlExpression").MustString("")
alias := model.Get("alias").MustString()
returnData := !model.Get("hide").MustBool(false)
queryType := model.Get("type").MustString()
@ -154,21 +155,34 @@ func parseRequestQuery(model *simplejson.Json, refId string, startTime time.Time
}
matchExact := model.Get("matchExact").MustBool(true)
metricQueryType := metricQueryType(model.Get("metricQueryType").MustInt(0))
var metricEditorModeValue metricEditorMode
memv, err := model.Get("metricEditorMode").Int()
if err != nil && len(expression) > 0 {
// this should only ever happen if this is an alerting query that has not yet been migrated in the frontend
metricEditorModeValue = MetricEditorModeRaw
} else {
metricEditorModeValue = metricEditorMode(memv)
}
return &cloudWatchQuery{
RefId: refId,
Region: region,
Id: id,
Namespace: namespace,
MetricName: metricName,
Statistic: statistic,
Expression: expression,
ReturnData: returnData,
Dimensions: dimensions,
Period: period,
Alias: alias,
MatchExact: matchExact,
UsedExpression: "",
RefId: refId,
Region: region,
Id: id,
Namespace: namespace,
MetricName: metricName,
Statistic: statistic,
Expression: expression,
ReturnData: returnData,
Dimensions: dimensions,
Period: period,
Alias: alias,
MatchExact: matchExact,
UsedExpression: "",
MetricQueryType: metricQueryType,
MetricEditorMode: metricEditorModeValue,
SqlExpression: sqlExpression,
}, nil
}

50
pkg/tsdb/cloudwatch/request_parser_test.go
View File

@ -272,4 +272,54 @@ func TestRequestParser(t *testing.T) {
assert.Equal(t, 21600, res.Period)
})
})
t.Run("Metric query type, metric editor mode and query api mode", func(t *testing.T) {
timeRange := legacydata.NewDataTimeRange("now-1h", "now-2h")
from, err := timeRange.ParseFrom()
require.NoError(t, err)
to, err := timeRange.ParseTo()
require.NoError(t, err)
t.Run("when metric query type and metric editor mode is not specified", func(t *testing.T) {
t.Run("it should be metric search builder", func(t *testing.T) {
query := getBaseJsonQuery()
res, err := parseRequestQuery(query, "ref1", from, to)
require.NoError(t, err)
assert.Equal(t, MetricQueryTypeSearch, res.MetricQueryType)
assert.Equal(t, MetricEditorModeBuilder, res.MetricEditorMode)
assert.Equal(t, GMDApiModeMetricStat, res.getGMDAPIMode())
})
t.Run("and an expression is specified it should be metric search builder", func(t *testing.T) {
query := getBaseJsonQuery()
query.Set("expression", "SUM(a)")
res, err := parseRequestQuery(query, "ref1", from, to)
require.NoError(t, err)
assert.Equal(t, MetricQueryTypeSearch, res.MetricQueryType)
assert.Equal(t, MetricEditorModeRaw, res.MetricEditorMode)
assert.Equal(t, GMDApiModeMathExpression, res.getGMDAPIMode())
})
})
t.Run("and an expression is specified it should be metric search builder", func(t *testing.T) {
query := getBaseJsonQuery()
query.Set("expression", "SUM(a)")
res, err := parseRequestQuery(query, "ref1", from, to)
require.NoError(t, err)
assert.Equal(t, MetricQueryTypeSearch, res.MetricQueryType)
assert.Equal(t, MetricEditorModeRaw, res.MetricEditorMode)
assert.Equal(t, GMDApiModeMathExpression, res.getGMDAPIMode())
})
})
}
func getBaseJsonQuery() *simplejson.Json {
return simplejson.NewFromAny(map[string]interface{}{
"refId": "ref1",
"region": "us-east-1",
"namespace": "ec2",
"metricName": "CPUUtilization",
"statistic": "Average",
"period": "900",
})
}

22
pkg/tsdb/cloudwatch/response_parser.go
View File

@ -229,19 +229,27 @@ func formatAlias(query *cloudWatchQuery, stat string, dimensions map[string]stri
if len(query.Alias) == 0 && query.isInferredSearchExpression() && !query.isMultiValuedDimensionExpression() {
return label
}
if len(query.Alias) == 0 && query.MetricQueryType == MetricQueryTypeQuery {
return label
}
// common fields
data := map[string]string{
"region": region,
"namespace": namespace,
"metric": metricName,
"stat": stat,
"period": period,
"region": region,
"period": period,
}
if len(label) != 0 {
data["label"] = label
}
for k, v := range dimensions {
data[k] = v
// since the SQL query string is not (yet) parsed, we don't know what namespace, metric, statistic and labels it's using at this point
if query.MetricQueryType != MetricQueryTypeQuery {
data["namespace"] = namespace
data["metric"] = metricName
data["stat"] = stat
for k, v := range dimensions {
data[k] = v
}
}
result := aliasFormat.ReplaceAllFunc([]byte(query.Alias), func(in []byte) []byte {

94
pkg/tsdb/cloudwatch/response_parser_test.go
View File

@ -3,6 +3,7 @@ package cloudwatch
import (
"encoding/json"
"io/ioutil"
"strings"
"testing"
"time"
@ -104,9 +105,11 @@ func TestCloudWatchResponseParser(t *testing.T) {
"LoadBalancer": {"lb1", "lb2"},
"TargetGroup": {"tg"},
},
Statistic: "Average",
Period: 60,
Alias: "{{LoadBalancer}} Expanded",
Statistic: "Average",
Period: 60,
Alias: "{{LoadBalancer}} Expanded",
MetricQueryType: MetricQueryTypeSearch,
MetricEditorMode: MetricEditorModeBuilder,
}
frames, err := buildDataFrames(startTime, endTime, *response, query)
require.NoError(t, err)
@ -166,9 +169,11 @@ func TestCloudWatchResponseParser(t *testing.T) {
"LoadBalancer": {"lb1", "lb2"},
"TargetGroup": {"tg"},
},
Statistic: "Average",
Period: 60,
Alias: "{{LoadBalancer}} Expanded",
Statistic: "Average",
Period: 60,
Alias: "{{LoadBalancer}} Expanded",
MetricQueryType: MetricQueryTypeSearch,
MetricEditorMode: MetricEditorModeBuilder,
}
frames, err := buildDataFrames(startTime, endTime, *response, query)
require.NoError(t, err)
@ -229,9 +234,11 @@ func TestCloudWatchResponseParser(t *testing.T) {
"LoadBalancer": {"*"},
"TargetGroup": {"tg"},
},
Statistic: "Average",
Period: 60,
Alias: "{{LoadBalancer}} Expanded",
Statistic: "Average",
Period: 60,
Alias: "{{LoadBalancer}} Expanded",
MetricQueryType: MetricQueryTypeSearch,
MetricEditorMode: MetricEditorModeBuilder,
}
frames, err := buildDataFrames(startTime, endTime, *response, query)
require.NoError(t, err)
@ -266,9 +273,11 @@ func TestCloudWatchResponseParser(t *testing.T) {
Dimensions: map[string][]string{
"LoadBalancer": {"lb1", "lb2"},
},
Statistic: "Average",
Period: 60,
Alias: "{{LoadBalancer}} Expanded",
Statistic: "Average",
Period: 60,
Alias: "{{LoadBalancer}} Expanded",
MetricQueryType: MetricQueryTypeSearch,
MetricEditorMode: MetricEditorModeBuilder,
}
frames, err := buildDataFrames(startTime, endTime, *response, query)
require.NoError(t, err)
@ -307,9 +316,11 @@ func TestCloudWatchResponseParser(t *testing.T) {
"InstanceType": {"micro"},
"Resource": {"res"},
},
Statistic: "Average",
Period: 60,
Alias: "{{LoadBalancer}} Expanded {{InstanceType}} - {{Resource}}",
Statistic: "Average",
Period: 60,
Alias: "{{LoadBalancer}} Expanded {{InstanceType}} - {{Resource}}",
MetricQueryType: MetricQueryTypeSearch,
MetricEditorMode: MetricEditorModeBuilder,
}
frames, err := buildDataFrames(startTime, endTime, *response, query)
require.NoError(t, err)
@ -319,6 +330,51 @@ func TestCloudWatchResponseParser(t *testing.T) {
assert.Equal(t, "lb2 Expanded micro - res", frames[1].Name)
})
t.Run("Should only expand certain fields when using SQL queries", func(t *testing.T) {
timestamp := time.Unix(0, 0)
response := &queryRowResponse{
Labels: []string{"lb3"},
Metrics: map[string]*cloudwatch.MetricDataResult{
"lb3": {
Id: aws.String("lb3"),
Label: aws.String("lb3"),
Timestamps: []*time.Time{
aws.Time(timestamp),
},
Values: []*float64{aws.Float64(23)},
StatusCode: aws.String("Complete"),
},
},
}
query := &cloudWatchQuery{
RefId: "refId1",
Region: "us-east-1",
Namespace: "AWS/ApplicationELB",
MetricName: "TargetResponseTime",
Dimensions: map[string][]string{
"LoadBalancer": {"lb1"},
"InstanceType": {"micro"},
"Resource": {"res"},
},
Statistic: "Average",
Period: 60,
Alias: "{{LoadBalancer}} {{InstanceType}} {{metric}} {{namespace}} {{stat}} {{region}} {{period}}",
MetricQueryType: MetricQueryTypeQuery,
MetricEditorMode: MetricEditorModeRaw,
}
frames, err := buildDataFrames(startTime, endTime, *response, query)
require.NoError(t, err)
assert.False(t, strings.Contains(frames[0].Name, "AWS/ApplicationELB"))
assert.False(t, strings.Contains(frames[0].Name, "lb1"))
assert.False(t, strings.Contains(frames[0].Name, "micro"))
assert.False(t, strings.Contains(frames[0].Name, "AWS/ApplicationELB"))
assert.True(t, strings.Contains(frames[0].Name, "us-east-1"))
assert.True(t, strings.Contains(frames[0].Name, "60"))
})
t.Run("Parse cloudwatch response", func(t *testing.T) {
timestamp := time.Unix(0, 0)
response := &queryRowResponse{
@ -351,9 +407,11 @@ func TestCloudWatchResponseParser(t *testing.T) {
"LoadBalancer": {"lb"},
"TargetGroup": {"tg"},
},
Statistic: "Average",
Period: 60,
Alias: "{{namespace}}_{{metric}}_{{stat}}",
Statistic: "Average",
Period: 60,
Alias: "{{namespace}}_{{metric}}_{{stat}}",
MetricQueryType: MetricQueryTypeSearch,
MetricEditorMode: MetricEditorModeBuilder,
}
frames, err := buildDataFrames(startTime, endTime, *response, query)
require.NoError(t, err)

1
pkg/tsdb/cloudwatch/time_series_query.go
View File

@ -91,7 +91,6 @@ func (e *cloudWatchExecutor) executeTimeSeriesQuery(ctx context.Context, req *ba
resultChan <- &responseWrapper{
DataResponse: &dataResponse,
}
return nil, err
}
close(resultChan)

23
pkg/tsdb/cloudwatch/types.go
View File

@ -31,3 +31,26 @@ type metricStatMeta struct {
Stat string `json:"stat"`
Period int `json:"period"`
}
type metricQueryType uint32
const (
MetricQueryTypeSearch metricQueryType = iota
MetricQueryTypeQuery
)
type metricEditorMode uint32
const (
MetricEditorModeBuilder metricEditorMode = iota
MetricEditorModeRaw
)
type gmdApiMode uint32
const (
GMDApiModeMetricStat gmdApiMode = iota
GMDApiModeInferredSearchExpression
GMDApiModeMathExpression
GMDApiModeSQLExpression
)

18
public/app/features/dashboard/state/DashboardMigrator.ts
View File

@ -48,6 +48,7 @@ import { mergeTransformer } from '../../../../../packages/grafana-data/src/trans
import {
migrateMultipleStatsMetricsQuery,
migrateMultipleStatsAnnotationQuery,
migrateCloudWatchQuery,
} from 'app/plugins/datasource/cloudwatch/migrations';
import { CloudWatchMetricsQuery, CloudWatchAnnotationQuery } from 'app/plugins/datasource/cloudwatch/types';
@ -747,10 +748,14 @@ export class DashboardMigrator {
// New queries, that were created during migration, are put at the end of the array.
migrateCloudWatchQueries(panel: PanelModel) {
for (const target of panel.targets || []) {
if (isLegacyCloudWatchQuery(target)) {
const newQueries = migrateMultipleStatsMetricsQuery(target, [...panel.targets]);
for (const newQuery of newQueries) {
panel.targets.push(newQuery);
if (isCloudWatchQuery(target)) {
migrateCloudWatchQuery(target);
if (target.hasOwnProperty('statistics')) {
// New queries, that were created during migration, are put at the end of the array.
const newQueries = migrateMultipleStatsMetricsQuery(target, [...panel.targets]);
for (const newQuery of newQueries) {
panel.targets.push(newQuery);
}
}
}
}
@ -1099,12 +1104,13 @@ function upgradeValueMappingsForPanel(panel: PanelModel) {
return panel;
}
function isLegacyCloudWatchQuery(target: DataQuery): target is CloudWatchMetricsQuery {
function isCloudWatchQuery(target: DataQuery): target is CloudWatchMetricsQuery {
return (
target.hasOwnProperty('dimensions') &&
target.hasOwnProperty('namespace') &&
target.hasOwnProperty('region') &&
target.hasOwnProperty('statistics')
target.hasOwnProperty('period') &&
target.hasOwnProperty('metricName')
);
}

38
public/app/plugins/datasource/cloudwatch/__mocks__/cloudwatch-sql/Monaco.ts Normal file
View File

@ -0,0 +1,38 @@
import { monacoTypes } from '@grafana/ui';
import { Monaco } from '../../cloudwatch-sql/completion/types';
import {
multiLineFullQuery,
singleLineFullQuery,
singleLineEmptyQuery,
singleLineTwoQueries,
multiLineIncompleteQueryWithoutNamespace,
} from './test-data';
const TestData = {
[multiLineFullQuery.query]: multiLineFullQuery.tokens,
[singleLineFullQuery.query]: singleLineFullQuery.tokens,
[singleLineEmptyQuery.query]: singleLineEmptyQuery.tokens,
[singleLineTwoQueries.query]: singleLineTwoQueries.tokens,
[multiLineIncompleteQueryWithoutNamespace.query]: multiLineIncompleteQueryWithoutNamespace.tokens,
};
// Stub for the Monaco instance. Only implements the parts that are used in cloudwatch sql
const MonacoMock: Monaco = {
editor: {
tokenize: (value: string, languageId: string) => {
return TestData[value];
},
},
Range: {
containsPosition: (range: monacoTypes.IRange, position: monacoTypes.IPosition) => {
return (
position.lineNumber >= range.startLineNumber &&
position.lineNumber <= range.endLineNumber &&
position.column >= range.startColumn &&
position.column <= range.endColumn
);
},
},
};
export default MonacoMock;

21
public/app/plugins/datasource/cloudwatch/__mocks__/cloudwatch-sql/TextModel.ts Normal file
View File

@ -0,0 +1,21 @@
import { monacoTypes } from '@grafana/ui';
// Stub for monacoTypes.editor.ITextModel. Only implements the parts that are used in cloudwatch sql
function TextModel(value: string) {
return {
getValue: function (eol?: monacoTypes.editor.EndOfLinePreference, preserveBOM?: boolean): string {
return value;
},
getValueInRange: function (range: monacoTypes.IRange, eol?: monacoTypes.editor.EndOfLinePreference): string {
const lines = value.split('\n');
const line = lines[range.startLineNumber - 1];
return line.trim().slice(range.startColumn === 0 ? 0 : range.startColumn - 1, range.endColumn - 1);
},
getLineLength: function (lineNumber: number): number {
const lines = value.split('\n');
return lines[lineNumber - 1].trim().length;
},
};
}
export default TextModel;

5
public/app/plugins/datasource/cloudwatch/__mocks__/cloudwatch-sql/test-data/index.ts Normal file
View File

@ -0,0 +1,5 @@
export { multiLineFullQuery } from './multiLineFullQuery';
export { singleLineFullQuery } from './singleLineFullQuery';
export { singleLineEmptyQuery } from './singleLineEmptyQuery';
export { singleLineTwoQueries } from './singleLineTwoQueries';
export { multiLineIncompleteQueryWithoutNamespace } from './multiLineIncompleteQueryWithoutNamespace';

238
public/app/plugins/datasource/cloudwatch/__mocks__/cloudwatch-sql/test-data/multiLineFullQuery.ts Normal file
View File

@ -0,0 +1,238 @@
import { monacoTypes } from '@grafana/ui';
export const multiLineFullQuery = {
query: `SELECT AVG(CPUUtilization)
FROM SCHEMA("AWS/ECS", InstanceId)
WHERE InstanceId = 'i-03c6908092db17ac9'
GROUP BY InstanceId ORDER BY AVG() DESC
LIMIT 10`,
tokens: [
[
{
offset: 0,
type: 'keyword.sql',
language: 'cloudwatch-sql',
},
{
offset: 6,
type: 'white.sql',
language: 'cloudwatch-sql',
},
{
offset: 7,
type: 'predefined.sql',
language: 'cloudwatch-sql',
},
{
offset: 10,
type: 'delimiter.parenthesis.sql',
language: 'cloudwatch-sql',
},
{
offset: 11,
type: 'identifier.sql',
language: 'cloudwatch-sql',
},
{
offset: 25,
type: 'delimiter.parenthesis.sql',
language: 'cloudwatch-sql',
},
{
offset: 26,
type: 'white.sql',
language: 'cloudwatch-sql',
},
],
[
{
offset: 0,
type: 'keyword.sql',
language: 'cloudwatch-sql',
},
{
offset: 4,
type: 'white.sql',
language: 'cloudwatch-sql',
},
{
offset: 5,
type: 'keyword.sql',
language: 'cloudwatch-sql',
},
{
offset: 11,
type: 'delimiter.parenthesis.sql',
language: 'cloudwatch-sql',
},
{
offset: 12,
type: 'type.sql',
language: 'cloudwatch-sql',
},
{
offset: 21,
type: 'delimiter.sql',
language: 'cloudwatch-sql',
},
{
offset: 22,
type: 'white.sql',
language: 'cloudwatch-sql',
},
{
offset: 23,
type: 'identifier.sql',
language: 'cloudwatch-sql',
},
{
offset: 33,
type: 'delimiter.parenthesis.sql',
language: 'cloudwatch-sql',
},
{
offset: 34,
type: 'white.sql',
language: 'cloudwatch-sql',
},
],
[],
[
{
offset: 0,
type: 'keyword.sql',
language: 'cloudwatch-sql',
},
{
offset: 5,
type: 'white.sql',
language: 'cloudwatch-sql',
},
{
offset: 6,
type: 'identifier.sql',
language: 'cloudwatch-sql',
},
{
offset: 16,
type: 'white.sql',
language: 'cloudwatch-sql',
},
{
offset: 17,
type: 'operator.sql',
language: 'cloudwatch-sql',
},
{
offset: 18,
type: 'white.sql',
language: 'cloudwatch-sql',
},
{
offset: 19,
type: 'string.sql',
language: 'cloudwatch-sql',
},
{
offset: 40,
type: 'white.sql',
language: 'cloudwatch-sql',
},
],
[
{
offset: 0,
type: 'keyword.sql',
language: 'cloudwatch-sql',
},
{
offset: 5,
type: 'white.sql',
language: 'cloudwatch-sql',
},
{
offset: 6,
type: 'keyword.sql',
language: 'cloudwatch-sql',
},
{
offset: 8,
type: 'white.sql',
language: 'cloudwatch-sql',
},
{
offset: 9,
type: 'identifier.sql',
language: 'cloudwatch-sql',
},
{
offset: 19,
type: 'white.sql',
language: 'cloudwatch-sql',
},
{
offset: 20,
type: 'keyword.sql',
language: 'cloudwatch-sql',
},
{
offset: 25,
type: 'white.sql',
language: 'cloudwatch-sql',
},
{
offset: 26,
type: 'keyword.sql',
language: 'cloudwatch-sql',
},
{
offset: 28,
type: 'white.sql',
language: 'cloudwatch-sql',
},
{
offset: 29,
type: 'predefined.sql',
language: 'cloudwatch-sql',
},
{
offset: 32,
type: 'delimiter.parenthesis.sql',
language: 'cloudwatch-sql',
},
{
offset: 34,
type: 'white.sql',
language: 'cloudwatch-sql',
},
{
offset: 35,
type: 'keyword.sql',
language: 'cloudwatch-sql',
},
{
offset: 39,
type: 'white.sql',
language: 'cloudwatch-sql',
},
],
[
{
offset: 0,
type: 'keyword.sql',
language: 'cloudwatch-sql',
},
{
offset: 5,
type: 'white.sql',
language: 'cloudwatch-sql',
},
{
offset: 6,
type: 'number.sql',
language: 'cloudwatch-sql',
},
],
] as monacoTypes.Token[][],
};

57
public/app/plugins/datasource/cloudwatch/__mocks__/cloudwatch-sql/test-data/multiLineIncompleteQueryWithoutNamespace.ts Normal file
View File

@ -0,0 +1,57 @@
import { monacoTypes } from '@grafana/ui';
export const multiLineIncompleteQueryWithoutNamespace = {
query: `SELECT AVG(CPUUtilization)
FROM `,
tokens: [
[
{
offset: 0,
type: 'keyword.sql',
language: 'cloudwatch-sql',
},
{
offset: 6,
type: 'white.sql',
language: 'cloudwatch-sql',
},
{
offset: 7,
type: 'predefined.sql',
language: 'cloudwatch-sql',
},
{
offset: 10,
type: 'delimiter.parenthesis.sql',
language: 'cloudwatch-sql',
},
{
offset: 11,
type: 'identifier.sql',
language: 'cloudwatch-sql',
},
{
offset: 25,
type: 'delimiter.parenthesis.sql',
language: 'cloudwatch-sql',
},
{
offset: 26,
type: 'white.sql',
language: 'cloudwatch-sql',
},
],
[
{
offset: 0,
type: 'keyword.sql',
language: 'cloudwatch-sql',
},
{
offset: 4,
type: 'white.sql',
language: 'cloudwatch-sql',
},
],
] as monacoTypes.Token[][],
};

6
public/app/plugins/datasource/cloudwatch/__mocks__/cloudwatch-sql/test-data/singleLineEmptyQuery.ts Normal file
View File

@ -0,0 +1,6 @@
import { monacoTypes } from '@grafana/ui';
export const singleLineEmptyQuery = {
query: '',
tokens: [] as monacoTypes.Token[][],
};

224
public/app/plugins/datasource/cloudwatch/__mocks__/cloudwatch-sql/test-data/singleLineFullQuery.ts Normal file
View File

@ -0,0 +1,224 @@
import { monacoTypes } from '@grafana/ui';
export const singleLineFullQuery = {
query: `SELECT AVG(CPUUtilization) FROM SCHEMA("AWS/EC2", InstanceId) WHERE InstanceId = 'i-03c6908092db17ac9' GROUP BY InstanceId ORDER BY AVG() DESC LIMIT 10`,
tokens: [
[
{
offset: 0,
type: 'keyword.sql',
language: 'cloudwatch-sql',
},
{
offset: 6,
type: 'white.sql',
language: 'cloudwatch-sql',
},
{
offset: 7,
type: 'predefined.sql',
language: 'cloudwatch-sql',
},
{
offset: 10,
type: 'delimiter.parenthesis.sql',
language: 'cloudwatch-sql',
},
{
offset: 11,
type: 'identifier.sql',
language: 'cloudwatch-sql',
},
{
offset: 25,
type: 'delimiter.parenthesis.sql',
language: 'cloudwatch-sql',
},
{
offset: 26,
type: 'white.sql',
language: 'cloudwatch-sql',
},
{
offset: 27,
type: 'keyword.sql',
language: 'cloudwatch-sql',
},
{
offset: 31,
type: 'white.sql',
language: 'cloudwatch-sql',
},
{
offset: 32,
type: 'keyword.sql',
language: 'cloudwatch-sql',
},
{
offset: 38,
type: 'delimiter.parenthesis.sql',
language: 'cloudwatch-sql',
},
{
offset: 39,
type: 'type.sql',
language: 'cloudwatch-sql',
},
{
offset: 48,
type: 'delimiter.sql',
language: 'cloudwatch-sql',
},
{
offset: 49,
type: 'white.sql',
language: 'cloudwatch-sql',
},
{
offset: 50,
type: 'identifier.sql',
language: 'cloudwatch-sql',
},
{
offset: 60,
type: 'delimiter.parenthesis.sql',
language: 'cloudwatch-sql',
},
{
offset: 61,
type: 'white.sql',
language: 'cloudwatch-sql',
},
{
offset: 62,
type: 'keyword.sql',
language: 'cloudwatch-sql',
},
{
offset: 67,
type: 'white.sql',
language: 'cloudwatch-sql',
},
{
offset: 68,
type: 'identifier.sql',
language: 'cloudwatch-sql',
},
{
offset: 78,
type: 'white.sql',
language: 'cloudwatch-sql',
},
{
offset: 79,
type: 'operator.sql',
language: 'cloudwatch-sql',
},
{
offset: 80,
type: 'white.sql',
language: 'cloudwatch-sql',
},
{
offset: 81,
type: 'string.sql',
language: 'cloudwatch-sql',
},
{
offset: 102,
type: 'white.sql',
language: 'cloudwatch-sql',
},
{
offset: 103,
type: 'keyword.sql',
language: 'cloudwatch-sql',
},
{
offset: 108,
type: 'white.sql',
language: 'cloudwatch-sql',
},
{
offset: 109,
type: 'keyword.sql',
language: 'cloudwatch-sql',
},
{
offset: 111,
type: 'white.sql',
language: 'cloudwatch-sql',
},
{
offset: 112,
type: 'identifier.sql',
language: 'cloudwatch-sql',
},
{
offset: 122,
type: 'white.sql',
language: 'cloudwatch-sql',
},
{
offset: 123,
type: 'keyword.sql',
language: 'cloudwatch-sql',
},
{
offset: 128,
type: 'white.sql',
language: 'cloudwatch-sql',
},
{
offset: 129,
type: 'keyword.sql',
language: 'cloudwatch-sql',
},
{
offset: 131,
type: 'white.sql',
language: 'cloudwatch-sql',
},
{
offset: 132,
type: 'predefined.sql',
language: 'cloudwatch-sql',
},
{
offset: 135,
type: 'delimiter.parenthesis.sql',
language: 'cloudwatch-sql',
},
{
offset: 137,
type: 'white.sql',
language: 'cloudwatch-sql',
},
{
offset: 138,
type: 'keyword.sql',
language: 'cloudwatch-sql',
},
{
offset: 142,
type: 'white.sql',
language: 'cloudwatch-sql',
},
{
offset: 143,
type: 'keyword.sql',
language: 'cloudwatch-sql',
},
{
offset: 148,
type: 'white.sql',
language: 'cloudwatch-sql',
},
{
offset: 149,
type: 'number.sql',
language: 'cloudwatch-sql',
},
],
] as monacoTypes.Token[][],
};

289
public/app/plugins/datasource/cloudwatch/__mocks__/cloudwatch-sql/test-data/singleLineTwoQueries.ts Normal file
View File

@ -0,0 +1,289 @@
import { monacoTypes } from '@grafana/ui';
export const singleLineTwoQueries = {
query: `SELECT AVG(CPUUtilization) FROM SCHEMA("AWS/EC2", InstanceId) WHERE InstanceId = 'i-03c6908092db17ac9' GROUP BY InstanceId ORDER BY AVG() DESC LIMIT 10 / SELECT SUM(CPUCreditUsage) FROM "AWS/ECS"`,
tokens: [
[
{
offset: 0,
type: 'keyword.sql',
language: 'cloudwatch-sql',
},
{
offset: 6,
type: 'white.sql',
language: 'cloudwatch-sql',
},
{
offset: 7,
type: 'predefined.sql',
language: 'cloudwatch-sql',
},
{
offset: 10,
type: 'delimiter.parenthesis.sql',
language: 'cloudwatch-sql',
},
{
offset: 11,
type: 'identifier.sql',
language: 'cloudwatch-sql',
},
{
offset: 25,
type: 'delimiter.parenthesis.sql',
language: 'cloudwatch-sql',
},
{
offset: 26,
type: 'white.sql',
language: 'cloudwatch-sql',
},
{
offset: 27,
type: 'keyword.sql',
language: 'cloudwatch-sql',
},
{
offset: 31,
type: 'white.sql',
language: 'cloudwatch-sql',
},
{
offset: 32,
type: 'keyword.sql',
language: 'cloudwatch-sql',
},
{
offset: 38,
type: 'delimiter.parenthesis.sql',
language: 'cloudwatch-sql',
},
{
offset: 39,
type: 'type.sql',
language: 'cloudwatch-sql',
},
{
offset: 48,
type: 'delimiter.sql',
language: 'cloudwatch-sql',
},
{
offset: 49,
type: 'white.sql',
language: 'cloudwatch-sql',
},
{
offset: 50,
type: 'identifier.sql',
language: 'cloudwatch-sql',
},
{
offset: 60,
type: 'delimiter.parenthesis.sql',
language: 'cloudwatch-sql',
},
{
offset: 61,
type: 'white.sql',
language: 'cloudwatch-sql',
},
{
offset: 62,
type: 'keyword.sql',
language: 'cloudwatch-sql',
},
{
offset: 67,
type: 'white.sql',
language: 'cloudwatch-sql',
},
{
offset: 68,
type: 'identifier.sql',
language: 'cloudwatch-sql',
},
{
offset: 78,
type: 'white.sql',
language: 'cloudwatch-sql',
},
{
offset: 79,
type: 'operator.sql',
language: 'cloudwatch-sql',
},
{
offset: 80,
type: 'white.sql',
language: 'cloudwatch-sql',
},
{
offset: 81,
type: 'string.sql',
language: 'cloudwatch-sql',
},
{
offset: 102,
type: 'white.sql',
language: 'cloudwatch-sql',
},
{
offset: 103,
type: 'keyword.sql',
language: 'cloudwatch-sql',
},
{
offset: 108,
type: 'white.sql',
language: 'cloudwatch-sql',
},
{
offset: 109,
type: 'keyword.sql',
language: 'cloudwatch-sql',
},
{
offset: 111,
type: 'white.sql',
language: 'cloudwatch-sql',
},
{
offset: 112,
type: 'identifier.sql',
language: 'cloudwatch-sql',
},
{
offset: 122,
type: 'white.sql',
language: 'cloudwatch-sql',
},
{
offset: 123,
type: 'keyword.sql',
language: 'cloudwatch-sql',
},
{
offset: 128,
type: 'white.sql',
language: 'cloudwatch-sql',
},
{
offset: 129,
type: 'keyword.sql',
language: 'cloudwatch-sql',
},
{
offset: 131,
type: 'white.sql',
language: 'cloudwatch-sql',
},
{
offset: 132,
type: 'predefined.sql',
language: 'cloudwatch-sql',
},
{
offset: 135,
type: 'delimiter.parenthesis.sql',
language: 'cloudwatch-sql',
},
{
offset: 137,
type: 'white.sql',
language: 'cloudwatch-sql',
},
{
offset: 138,
type: 'keyword.sql',
language: 'cloudwatch-sql',
},
{
offset: 142,
type: 'white.sql',
language: 'cloudwatch-sql',
},
{
offset: 143,
type: 'keyword.sql',
language: 'cloudwatch-sql',
},
{
offset: 148,
type: 'white.sql',
language: 'cloudwatch-sql',
},
{
offset: 149,
type: 'number.sql',
language: 'cloudwatch-sql',
},
{
offset: 151,
type: 'white.sql',
language: 'cloudwatch-sql',
},
{
offset: 152,
type: 'operator.sql',
language: 'cloudwatch-sql',
},
{
offset: 153,
type: 'white.sql',
language: 'cloudwatch-sql',
},
{
offset: 154,
type: 'keyword.sql',
language: 'cloudwatch-sql',
},
{
offset: 160,
type: 'white.sql',
language: 'cloudwatch-sql',
},
{
offset: 161,
type: 'predefined.sql',
language: 'cloudwatch-sql',
},
{
offset: 164,
type: 'delimiter.parenthesis.sql',
language: 'cloudwatch-sql',
},
{
offset: 165,
type: 'identifier.sql',
language: 'cloudwatch-sql',
},
{
offset: 179,
type: 'delimiter.parenthesis.sql',
language: 'cloudwatch-sql',
},
{
offset: 180,
type: 'white.sql',
language: 'cloudwatch-sql',
},
{
offset: 181,
type: 'keyword.sql',
language: 'cloudwatch-sql',
},
{
offset: 185,
type: 'white.sql',
language: 'cloudwatch-sql',
},
{
offset: 186,
type: 'type.sql',
language: 'cloudwatch-sql',
},
],
] as monacoTypes.Token[][],
};

76
public/app/plugins/datasource/cloudwatch/__mocks__/sqlUtils.ts Normal file
View File

@ -0,0 +1,76 @@
import {
QueryEditorExpression,
QueryEditorExpressionType,
QueryEditorArrayExpression,
QueryEditorOperatorExpression,
QueryEditorPropertyType,
QueryEditorGroupByExpression,
QueryEditorFunctionExpression,
QueryEditorFunctionParameterExpression,
QueryEditorPropertyExpression,
} from '../expressions';
export function createArray(
expressions: QueryEditorExpression[],
type: QueryEditorExpressionType.And | QueryEditorExpressionType.Or = QueryEditorExpressionType.And
): QueryEditorArrayExpression {
const array = {
type,
expressions,
};
return array;
}
export function createOperator(property: string, operator: string, value?: string): QueryEditorOperatorExpression {
return {
type: QueryEditorExpressionType.Operator,
property: {
name: property,
type: QueryEditorPropertyType.String,
},
operator: {
name: operator,
value: value,
},
};
}
export function createGroupBy(column: string): QueryEditorGroupByExpression {
return {
type: QueryEditorExpressionType.GroupBy,
property: {
type: QueryEditorPropertyType.String,
name: column,
},
};
}
export function createFunction(name: string): QueryEditorFunctionExpression {
return {
type: QueryEditorExpressionType.Function,
name,
};
}
export function createFunctionWithParameter(functionName: string, params: string[]): QueryEditorFunctionExpression {
const reduce = createFunction(functionName);
reduce.parameters = params.map((name) => {
const param: QueryEditorFunctionParameterExpression = {
type: QueryEditorExpressionType.FunctionParameter,
name,
};
return param;
});
return reduce;
}
export function createProperty(name: string): QueryEditorPropertyExpression {
return {
type: QueryEditorExpressionType.Property,
property: {
type: QueryEditorPropertyType.String,
name: name,
},
};
}

377
public/app/plugins/datasource/cloudwatch/cloudwatch-sql/SQLGenerator.test.ts Normal file
View File

@ -0,0 +1,377 @@
import { TemplateSrv } from 'app/features/templating/template_srv';
import { QueryEditorExpressionType } from '../expressions';
import { SQLExpression } from '../types';
import {
aggregationvariable,
labelsVariable,
metricVariable,
namespaceVariable,
} from '../__mocks__/CloudWatchDataSource';
import {
createFunctionWithParameter,
createArray,
createOperator,
createGroupBy,
createFunction,
createProperty,
} from '../__mocks__/sqlUtils';
import SQLGenerator from './SQLGenerator';
describe('SQLGenerator', () => {
let baseQuery: SQLExpression = {
select: createFunctionWithParameter('SUM', ['CPUUtilization']),
from: createFunctionWithParameter('SCHEMA', ['AWS/EC2']),
orderByDirection: 'DESC',
};
describe('mandatory fields check', () => {
it('should return undefined if metric and aggregation is missing', () => {
expect(
new SQLGenerator().expressionToSqlQuery({
from: createFunctionWithParameter('SCHEMA', ['AWS/EC2']),
})
).toBeUndefined();
});
it('should return undefined if aggregation is missing', () => {
expect(
new SQLGenerator().expressionToSqlQuery({
from: createFunctionWithParameter('SCHEMA', []),
})
).toBeUndefined();
});
});
it('should return query if mandatory fields are provided', () => {
expect(new SQLGenerator().expressionToSqlQuery(baseQuery)).not.toBeUndefined();
});
describe('select', () => {
it('should use statistic and metric name', () => {
const select = createFunctionWithParameter('COUNT', ['BytesPerSecond']);
expect(new SQLGenerator().expressionToSqlQuery({ ...baseQuery, select })).toEqual(
`SELECT COUNT(BytesPerSecond) FROM SCHEMA("AWS/EC2")`
);
});
it('should wrap in double quotes if metric name contains illegal characters ', () => {
const select = createFunctionWithParameter('COUNT', ['Bytes-Per-Second']);
expect(new SQLGenerator().expressionToSqlQuery({ ...baseQuery, select })).toEqual(
`SELECT COUNT("Bytes-Per-Second") FROM SCHEMA("AWS/EC2")`
);
});
});
describe('from', () => {
describe('with schema contraint', () => {
it('should handle schema without dimensions', () => {
const from = createFunctionWithParameter('SCHEMA', ['AWS/MQ']);
expect(new SQLGenerator().expressionToSqlQuery({ ...baseQuery, from })).toEqual(
`SELECT SUM(CPUUtilization) FROM SCHEMA("AWS/MQ")`
);
});
it('should handle schema with dimensions', () => {
const from = createFunctionWithParameter('SCHEMA', ['AWS/MQ', 'InstanceId', 'InstanceType']);
expect(new SQLGenerator().expressionToSqlQuery({ ...baseQuery, from })).toEqual(
`SELECT SUM(CPUUtilization) FROM SCHEMA("AWS/MQ", InstanceId, InstanceType)`
);
});
it('should handle schema with dimensions that has special characters', () => {
const from = createFunctionWithParameter('SCHEMA', [
'AWS/MQ',
'Instance Id',
'Instance.Type',
'Instance-Group',
]);
expect(new SQLGenerator().expressionToSqlQuery({ ...baseQuery, from })).toEqual(
`SELECT SUM(CPUUtilization) FROM SCHEMA("AWS/MQ", "Instance Id", "Instance.Type", "Instance-Group")`
);
});
});
describe('without schema', () => {
it('should use the specified namespace', () => {
const from = createProperty('AWS/MQ');
expect(new SQLGenerator().expressionToSqlQuery({ ...baseQuery, from })).toEqual(
`SELECT SUM(CPUUtilization) FROM "AWS/MQ"`
);
});
});
});
function assertQueryEndsWith(rest: Partial<SQLExpression>, expectedFilter: string) {
expect(new SQLGenerator().expressionToSqlQuery({ ...baseQuery, ...rest })).toEqual(
`SELECT SUM(CPUUtilization) FROM SCHEMA("AWS/EC2") ${expectedFilter}`
);
}
describe('filter', () => {
it('should not add WHERE clause in case its empty', () => {
expect(new SQLGenerator().expressionToSqlQuery({ ...baseQuery })).not.toContain('WHERE');
});
it('should not add WHERE clause when there is no filter conditions', () => {
const where = createArray([]);
expect(new SQLGenerator().expressionToSqlQuery({ ...baseQuery, where })).not.toContain('WHERE');
});
// TODO: We should handle this scenario
it.skip('should not add WHERE clause when the operator is incomplete', () => {
const where = createArray([createOperator('Instance-Id', '=')]);
expect(new SQLGenerator().expressionToSqlQuery({ ...baseQuery, where })).not.toContain('WHERE');
});
it('should handle one top level filter with AND', () => {
const where = createArray([createOperator('Instance-Id', '=', 'I-123')]);
assertQueryEndsWith({ where }, `WHERE "Instance-Id" = 'I-123'`);
});
it('should handle one top level filter with OR', () => {
assertQueryEndsWith(
{ where: createArray([createOperator('InstanceId', '=', 'I-123')]) },
`WHERE InstanceId = 'I-123'`
);
});
it('should handle multiple top level filters combined with AND', () => {
const filter = createArray(
[createOperator('InstanceId', '=', 'I-123'), createOperator('Instance-Id', '!=', 'I-456')],
QueryEditorExpressionType.And
);
assertQueryEndsWith({ where: filter }, `WHERE InstanceId = 'I-123' AND "Instance-Id" != 'I-456'`);
});
it('should handle multiple top level filters combined with OR', () => {
const filter = createArray(
[createOperator('InstanceId', '=', 'I-123'), createOperator('InstanceId', '!=', 'I-456')],
QueryEditorExpressionType.Or
);
assertQueryEndsWith({ where: filter }, `WHERE InstanceId = 'I-123' OR InstanceId != 'I-456'`);
});
it('should handle one top level filters with one nested filter', () => {
const filter = createArray(
[
createOperator('InstanceId', '=', 'I-123'),
createArray([createOperator('InstanceId', '!=', 'I-456')], QueryEditorExpressionType.And),
],
QueryEditorExpressionType.And
);
assertQueryEndsWith({ where: filter }, `WHERE InstanceId = 'I-123' AND InstanceId != 'I-456'`);
});
it('should handle one top level filter with two nested filters combined with AND', () => {
const filter = createArray(
[
createOperator('Instance.Type', '=', 'I-123'),
createArray(
[createOperator('InstanceId', '!=', 'I-456'), createOperator('Type', '!=', 'some-type')],
QueryEditorExpressionType.And
),
],
QueryEditorExpressionType.And
);
// In this scenario, the parenthesis are redundant. However, they're not doing any harm and it would be really complicated to remove them
assertQueryEndsWith(
{ where: filter },
`WHERE "Instance.Type" = 'I-123' AND (InstanceId != 'I-456' AND Type != 'some-type')`
);
});
it('should handle one top level filter with two nested filters combined with OR', () => {
const filter = createArray(
[
createOperator('InstanceId', '=', 'I-123'),
createArray(
[createOperator('InstanceId', '!=', 'I-456'), createOperator('Type', '!=', 'some-type')],
QueryEditorExpressionType.Or
),
],
QueryEditorExpressionType.And
);
assertQueryEndsWith(
{ where: filter },
`WHERE InstanceId = 'I-123' AND (InstanceId != 'I-456' OR Type != 'some-type')`
);
});
it('should handle two top level filters with two nested filters combined with AND', () => {
const filter = createArray(
[
createArray(
[createOperator('InstanceId', '=', 'I-123'), createOperator('Type', '!=', 'some-type')],
QueryEditorExpressionType.And
),
createArray(
[createOperator('InstanceId', '!=', 'I-456'), createOperator('Type', '!=', 'some-type')],
QueryEditorExpressionType.Or
),
],
QueryEditorExpressionType.And
);
assertQueryEndsWith(
{ where: filter },
`WHERE (InstanceId = 'I-123' AND Type != 'some-type') AND (InstanceId != 'I-456' OR Type != 'some-type')`
);
});
it('should handle two top level filters with two nested filters combined with OR', () => {
const filter = createArray(
[
createArray(
[createOperator('InstanceId', '=', 'I-123'), createOperator('Type', '!=', 'some-type')],
QueryEditorExpressionType.Or
),
createArray(
[createOperator('InstanceId', '!=', 'I-456'), createOperator('Type', '!=', 'some-type')],
QueryEditorExpressionType.Or
),
],
QueryEditorExpressionType.Or
);
assertQueryEndsWith(
{ where: filter },
`WHERE (InstanceId = 'I-123' OR Type != 'some-type') OR (InstanceId != 'I-456' OR Type != 'some-type')`
);
});
it('should handle three top level filters with one nested filters combined with OR', () => {
const filter = createArray(
[
createArray([createOperator('InstanceId', '=', 'I-123')], QueryEditorExpressionType.Or),
createArray([createOperator('Type', '!=', 'some-type')], QueryEditorExpressionType.Or),
createArray([createOperator('InstanceId', '!=', 'I-456')], QueryEditorExpressionType.Or),
],
QueryEditorExpressionType.Or
);
assertQueryEndsWith(
{ where: filter },
`WHERE InstanceId = 'I-123' OR Type != 'some-type' OR InstanceId != 'I-456'`
);
});
it('should handle three top level filters with one nested filters combined with AND', () => {
const filter = createArray(
[
createArray([createOperator('InstanceId', '=', 'I-123')], QueryEditorExpressionType.Or),
createArray([createOperator('Type', '!=', 'some-type')], QueryEditorExpressionType.Or),
createArray([createOperator('InstanceId', '!=', 'I-456')], QueryEditorExpressionType.Or),
],
QueryEditorExpressionType.And
);
assertQueryEndsWith(
{ where: filter },
`WHERE InstanceId = 'I-123' AND Type != 'some-type' AND InstanceId != 'I-456'`
);
});
});
describe('group by', () => {
it('should not add GROUP BY clause in case its empty', () => {
expect(new SQLGenerator().expressionToSqlQuery({ ...baseQuery })).not.toContain('GROUP BY');
});
it('should handle single label', () => {
const groupBy = createArray([createGroupBy('InstanceId')], QueryEditorExpressionType.And);
assertQueryEndsWith({ groupBy }, `GROUP BY InstanceId`);
});
it('should handle multiple label', () => {
const groupBy = createArray(
[createGroupBy('InstanceId'), createGroupBy('Type'), createGroupBy('Group')],
QueryEditorExpressionType.And
);
assertQueryEndsWith({ groupBy }, `GROUP BY InstanceId, Type, Group`);
});
});
describe('order by', () => {
it('should not add ORDER BY clause in case its empty', () => {
expect(new SQLGenerator().expressionToSqlQuery({ ...baseQuery })).not.toContain('ORDER BY');
});
it('should handle SUM ASC', () => {
const orderBy = createFunction('SUM');
assertQueryEndsWith({ orderBy, orderByDirection: 'ASC' }, `ORDER BY SUM() ASC`);
});
it('should handle SUM ASC', () => {
const orderBy = createFunction('SUM');
assertQueryEndsWith({ orderBy, orderByDirection: 'ASC' }, `ORDER BY SUM() ASC`);
});
it('should handle COUNT DESC', () => {
const orderBy = createFunction('COUNT');
assertQueryEndsWith({ orderBy, orderByDirection: 'DESC' }, `ORDER BY COUNT() DESC`);
});
});
describe('limit', () => {
it('should not add LIMIT clause in case its empty', () => {
expect(new SQLGenerator().expressionToSqlQuery({ ...baseQuery })).not.toContain('LIMIT');
});
it('should be added in case its specified', () => {
assertQueryEndsWith({ limit: 10 }, `LIMIT 10`);
});
});
describe('full query', () => {
it('should not add LIMIT clause in case its empty', () => {
let query: SQLExpression = {
select: createFunctionWithParameter('COUNT', ['DroppedBytes']),
from: createFunctionWithParameter('SCHEMA', ['AWS/MQ', 'InstanceId', 'Instance-Group']),
where: createArray(
[
createArray(
[createOperator('InstanceId', '=', 'I-123'), createOperator('Type', '!=', 'some-type')],
QueryEditorExpressionType.Or
),
createArray(
[createOperator('InstanceId', '!=', 'I-456'), createOperator('Type', '!=', 'some-type')],
QueryEditorExpressionType.Or
),
],
QueryEditorExpressionType.And
),
groupBy: createArray([createGroupBy('InstanceId'), createGroupBy('InstanceType')]),
orderBy: createFunction('COUNT'),
orderByDirection: 'DESC',
limit: 100,
};
expect(new SQLGenerator().expressionToSqlQuery(query)).toEqual(
`SELECT COUNT(DroppedBytes) FROM SCHEMA("AWS/MQ", InstanceId, "Instance-Group") WHERE (InstanceId = 'I-123' OR Type != 'some-type') AND (InstanceId != 'I-456' OR Type != 'some-type') GROUP BY InstanceId, InstanceType ORDER BY COUNT() DESC LIMIT 100`
);
});
});
describe('using variables', () => {
const templateService = new TemplateSrv();
templateService.init([metricVariable, namespaceVariable, labelsVariable, aggregationvariable]);
it('should interpolate variables correctly', () => {
let query: SQLExpression = {
select: createFunctionWithParameter('$aggregation', ['$metric']),
from: createFunctionWithParameter('SCHEMA', ['$namespace', '$labels']),
where: createArray(
[
createArray(
[createOperator('InstanceId', '=', 'I-123'), createOperator('Type', '!=', 'some-type')],
QueryEditorExpressionType.Or
),
createArray(
[createOperator('InstanceId', '!=', 'I-456'), createOperator('Type', '!=', 'some-type')],
QueryEditorExpressionType.Or
),
],
QueryEditorExpressionType.And
),
groupBy: createArray([createGroupBy('$labels')]),
orderBy: createFunction('$aggregation'),
orderByDirection: 'DESC',
limit: 100,
};
expect(new SQLGenerator(templateService).expressionToSqlQuery(query)).toEqual(
`SELECT $aggregation($metric) FROM SCHEMA(\"$namespace\", $labels) WHERE (InstanceId = 'I-123' OR Type != 'some-type') AND (InstanceId != 'I-456' OR Type != 'some-type') GROUP BY $labels ORDER BY $aggregation() DESC LIMIT 100`
);
});
});
});

157
public/app/plugins/datasource/cloudwatch/cloudwatch-sql/SQLGenerator.ts Normal file
View File

@ -0,0 +1,157 @@
import { getTemplateSrv, TemplateSrv } from 'app/features/templating/template_srv';
import { SQLExpression } from '../types';
import {
QueryEditorArrayExpression,
QueryEditorExpression,
QueryEditorExpressionType,
QueryEditorFunctionExpression,
QueryEditorOperatorExpression,
QueryEditorPropertyExpression,
} from '../expressions';
export default class SQLGenerator {
constructor(private templateSrv: TemplateSrv = getTemplateSrv()) {}
expressionToSqlQuery({
select,
from,
where,
groupBy,
orderBy,
orderByDirection,
limit,
}: SQLExpression): string | undefined {
if (!from || !select?.name || !select?.parameters?.length) {
return undefined;
}
let parts: string[] = [];
this.appendSelect(select, parts);
this.appendFrom(from, parts);
this.appendWhere(where, parts, true, where?.expressions?.length ?? 0);
this.appendGroupBy(groupBy, parts);
this.appendOrderBy(orderBy, orderByDirection, parts);
this.appendLimit(limit, parts);
return parts.join(' ');
}
private appendSelect(select: QueryEditorFunctionExpression | undefined, parts: string[]) {
parts.push('SELECT');
this.appendFunction(select, parts);
}
private appendFrom(from: QueryEditorPropertyExpression | QueryEditorFunctionExpression | undefined, parts: string[]) {
parts.push('FROM');
from?.type === QueryEditorExpressionType.Function
? this.appendFunction(from, parts)
: parts.push(this.formatValue(from?.property?.name ?? ''));
}
private appendWhere(
filter: QueryEditorExpression | undefined,
parts: string[],
isTopLevelExpression: boolean,
topLevelExpressionsCount: number
) {
if (!filter) {
return;
}
const hasChildExpressions = 'expressions' in filter && filter.expressions.length > 0;
if (isTopLevelExpression && hasChildExpressions) {
parts.push('WHERE');
}
if (filter.type === QueryEditorExpressionType.And) {
const andParts: string[] = [];
filter.expressions.map((exp) => this.appendWhere(exp, andParts, false, topLevelExpressionsCount));
if (andParts.length === 0) {
return;
}
const andCombined = andParts.join(' AND ');
const wrapInParentheses = !isTopLevelExpression && topLevelExpressionsCount > 1 && andParts.length > 1;
return parts.push(wrapInParentheses ? `(${andCombined})` : andCombined);
}
if (filter.type === QueryEditorExpressionType.Or) {
const orParts: string[] = [];
filter.expressions.map((exp) => this.appendWhere(exp, orParts, false, topLevelExpressionsCount));
if (orParts.length === 0) {
return;
}
const orCombined = orParts.join(' OR ');
const wrapInParentheses = !isTopLevelExpression && topLevelExpressionsCount > 1 && orParts.length > 1;
parts.push(wrapInParentheses ? `(${orCombined})` : orCombined);
return;
}
if (filter.type === QueryEditorExpressionType.Operator) {
return this.appendOperator(filter, parts);
}
}
private appendGroupBy(groupBy: QueryEditorArrayExpression | undefined, parts: string[]) {
const groupByParts: string[] = [];
for (const expression of groupBy?.expressions ?? []) {
if (expression?.type !== QueryEditorExpressionType.GroupBy || !expression.property.name) {
continue;
}
groupByParts.push(this.formatValue(expression.property.name));
}
if (groupByParts.length > 0) {
parts.push(`GROUP BY ${groupByParts.join(', ')}`);
}
}
private appendOrderBy(
orderBy: QueryEditorFunctionExpression | undefined,
orderByDirection: string | undefined,
parts: string[]
) {
if (orderBy) {
parts.push('ORDER BY');
this.appendFunction(orderBy, parts);
parts.push(orderByDirection ?? 'ASC');
}
}
private appendLimit(limit: number | undefined, parts: string[]) {
limit && parts.push(`LIMIT ${limit}`);
}
private appendOperator(expression: QueryEditorOperatorExpression, parts: string[], prefix?: string) {
const { property, operator } = expression;
if (!property.name || !operator.name || !operator.value) {
return;
}
parts.push(`${this.formatValue(property.name)} ${operator.name} '${operator.value}'`);
}
private appendFunction(select: QueryEditorFunctionExpression | undefined, parts: string[]) {
if (!select?.name) {
return;
}
const params = (select.parameters ?? [])
.map((p) => p.name && this.formatValue(p.name))
.filter(Boolean)
.join(', ');
parts.push(`${select.name}(${params})`);
}
private formatValue(label: string): string {
const specialCharacters = /[/\s\.-]/; // slash, space, dot or dash
const interpolated = this.templateSrv.replace(label, {}, 'raw');
if (specialCharacters.test(interpolated)) {
return `"${label}"`;
}
return label;
}
}

298
public/app/plugins/datasource/cloudwatch/cloudwatch-sql/completion/CompletionItemProvider.ts Normal file
View File

@ -0,0 +1,298 @@
import type { Monaco, monacoTypes } from '@grafana/ui';
import { getTemplateSrv, TemplateSrv } from '@grafana/runtime';
import { uniq } from 'lodash';
import { CloudWatchDatasource } from '../../datasource';
import { linkedTokenBuilder } from './linkedTokenBuilder';
import { getSuggestionKinds } from './suggestionKind';
import { getStatementPosition } from './statementPosition';
import { TRIGGER_SUGGEST } from './commands';
import { TokenType, SuggestionKind, CompletionItemPriority, StatementPosition } from './types';
import { LinkedToken } from './LinkedToken';
import {
BY,
FROM,
GROUP,
LIMIT,
ORDER,
SCHEMA,
SELECT,
ASC,
DESC,
WHERE,
COMPARISON_OPERATORS,
LOGICAL_OPERATORS,
STATISTICS,
} from '../language';
import { getMetricNameToken, getNamespaceToken } from './tokenUtils';
type CompletionItem = monacoTypes.languages.CompletionItem;
export class CompletionItemProvider {
region: string;
templateVariables: string[];
constructor(private datasource: CloudWatchDatasource, private templateSrv: TemplateSrv = getTemplateSrv()) {
this.templateVariables = this.datasource.getVariables();
this.region = datasource.getActualRegion();
}
setRegion(region: string) {
this.region = region;
}
getCompletionProvider(monaco: Monaco) {
return {
triggerCharacters: [' ', '$', ',', '(', "'"],
provideCompletionItems: async (model: monacoTypes.editor.ITextModel, position: monacoTypes.IPosition) => {
const currentToken = linkedTokenBuilder(monaco, model, position);
const statementPosition = getStatementPosition(currentToken);
const suggestionKinds = getSuggestionKinds(statementPosition);
const suggestions = await this.getSuggestions(
monaco,
currentToken,
suggestionKinds,
statementPosition,
position
);
return {
suggestions,
};
},
};
}
private async getSuggestions(
monaco: Monaco,
currentToken: LinkedToken | null,
suggestionKinds: SuggestionKind[],
statementPosition: StatementPosition,
position: monacoTypes.IPosition
): Promise<CompletionItem[]> {
let suggestions: CompletionItem[] = [];
const invalidRangeToken = currentToken?.isWhiteSpace() || currentToken?.isParenthesis();
const range =
invalidRangeToken || !currentToken?.range ? monaco.Range.fromPositions(position) : currentToken?.range;
const toCompletionItem = (value: string, rest: Partial<CompletionItem> = {}) => {
const item: CompletionItem = {
label: value,
insertText: value,
kind: monaco.languages.CompletionItemKind.Field,
range,
sortText: CompletionItemPriority.Medium,
...rest,
};
return item;
};
function addSuggestion(value: string, rest: Partial<CompletionItem> = {}) {
suggestions = [...suggestions, toCompletionItem(value, rest)];
}
for (const suggestion of suggestionKinds) {
switch (suggestion) {
case SuggestionKind.SelectKeyword:
addSuggestion(SELECT, {
insertText: `${SELECT} $0`,
insertTextRules: monaco.languages.CompletionItemInsertTextRule.InsertAsSnippet,
kind: monaco.languages.CompletionItemKind.Keyword,
command: TRIGGER_SUGGEST,
});
break;
case SuggestionKind.FunctionsWithArguments:
STATISTICS.map((s) =>
addSuggestion(s, {
insertText: `${s}($0)`,
insertTextRules: monaco.languages.CompletionItemInsertTextRule.InsertAsSnippet,
command: TRIGGER_SUGGEST,
kind: monaco.languages.CompletionItemKind.Function,
})
);
break;
case SuggestionKind.FunctionsWithoutArguments:
STATISTICS.map((s) =>
addSuggestion(s, {
insertText: `${s}() `,
insertTextRules: monaco.languages.CompletionItemInsertTextRule.InsertAsSnippet,
command: TRIGGER_SUGGEST,
kind: monaco.languages.CompletionItemKind.Function,
})
);
break;
case SuggestionKind.Metrics:
{
const namespaceToken = getNamespaceToken(currentToken);
if (namespaceToken?.value) {
// if a namespace is specified, only suggest metrics for the namespace
const metrics = await this.datasource.getMetrics(
this.templateSrv.replace(namespaceToken?.value.replace(/\"/g, '')),
this.templateSrv.replace(this.region)
);
metrics.map((m) => addSuggestion(m.value));
} else {
// If no namespace is specified in the query, just list all metrics
const metrics = await this.datasource.getAllMetrics(this.templateSrv.replace(this.region));
uniq(metrics.map((m) => m.metricName)).map((m) => addSuggestion(m, { insertText: m }));
}
}
break;
case SuggestionKind.FromKeyword:
addSuggestion(FROM, {
insertText: `${FROM} `,
command: TRIGGER_SUGGEST,
});
break;
case SuggestionKind.SchemaKeyword:
addSuggestion(SCHEMA, {
sortText: CompletionItemPriority.High,
insertText: `${SCHEMA}($0)`,
insertTextRules: monaco.languages.CompletionItemInsertTextRule.InsertAsSnippet,
command: TRIGGER_SUGGEST,
kind: monaco.languages.CompletionItemKind.Function,
});
break;
case SuggestionKind.Namespaces:
const metricNameToken = getMetricNameToken(currentToken);
let namespaces = [];
if (metricNameToken?.value) {
// if a metric is specified, only suggest namespaces that actually have that metric
const metrics = await this.datasource.getAllMetrics(this.region);
const metricName = this.templateSrv.replace(metricNameToken.value);
namespaces = metrics.filter((m) => m.metricName === metricName).map((m) => m.namespace);
} else {
// if no metric is specified, just suggest all namespaces
const ns = await this.datasource.getNamespaces();
namespaces = ns.map((n) => n.value);
}
namespaces.map((n) => addSuggestion(`"${n}"`, { insertText: `"${n}"` }));
break;
case SuggestionKind.LabelKeys:
{
const metricNameToken = getMetricNameToken(currentToken);
const namespaceToken = getNamespaceToken(currentToken);
if (namespaceToken?.value) {
let dimensionFilter = {};
let labelKeyTokens;
if (statementPosition === StatementPosition.SchemaFuncExtraArgument) {
labelKeyTokens = namespaceToken?.getNextUntil(TokenType.Parenthesis, [
TokenType.Delimiter,
TokenType.Whitespace,
]);
} else if (statementPosition === StatementPosition.AfterGroupByKeywords) {
labelKeyTokens = currentToken?.getPreviousUntil(TokenType.Keyword, [
TokenType.Delimiter,
TokenType.Whitespace,
]);
}
dimensionFilter = (labelKeyTokens || []).reduce((acc, curr) => {
return { ...acc, [curr.value]: null };
}, {});
const keys = await this.datasource.getDimensionKeys(
this.templateSrv.replace(namespaceToken.value.replace(/\"/g, '')),
this.templateSrv.replace(this.region),
dimensionFilter,
metricNameToken?.value ?? ''
);
keys.map((m) => {
const key = /[\s\.-]/.test(m.value) ? `"${m.value}"` : m.value;
addSuggestion(key);
});
}
}
break;
case SuggestionKind.LabelValues:
{
const namespaceToken = getNamespaceToken(currentToken);
const metricNameToken = getMetricNameToken(currentToken);
const labelKey = currentToken?.getPreviousNonWhiteSpaceToken()?.getPreviousNonWhiteSpaceToken();
if (namespaceToken?.value && labelKey?.value && metricNameToken?.value) {
const values = await this.datasource.getDimensionValues(
this.templateSrv.replace(this.region),
this.templateSrv.replace(namespaceToken.value.replace(/\"/g, '')),
this.templateSrv.replace(metricNameToken.value),
this.templateSrv.replace(labelKey.value),
{}
);
values.map((o) =>
addSuggestion(`'${o.value}'`, { insertText: `'${o.value}' `, command: TRIGGER_SUGGEST })
);
}
}
break;
case SuggestionKind.LogicalOperators:
LOGICAL_OPERATORS.map((o) =>
addSuggestion(`${o}`, {
insertText: `${o} `,
command: TRIGGER_SUGGEST,
sortText: CompletionItemPriority.MediumHigh,
})
);
break;
case SuggestionKind.WhereKeyword:
addSuggestion(`${WHERE}`, {
insertText: `${WHERE} `,
command: TRIGGER_SUGGEST,
sortText: CompletionItemPriority.High,
});
break;
case SuggestionKind.ComparisonOperators:
COMPARISON_OPERATORS.map((o) => addSuggestion(`${o}`, { insertText: `${o} `, command: TRIGGER_SUGGEST }));
break;
case SuggestionKind.GroupByKeywords:
addSuggestion(`${GROUP} ${BY}`, {
insertText: `${GROUP} ${BY} `,
command: TRIGGER_SUGGEST,
sortText: CompletionItemPriority.MediumHigh,
});
break;
case SuggestionKind.OrderByKeywords:
addSuggestion(`${ORDER} ${BY}`, {
insertText: `${ORDER} ${BY} `,
command: TRIGGER_SUGGEST,
sortText: CompletionItemPriority.Medium,
});
break;
case SuggestionKind.LimitKeyword:
addSuggestion(LIMIT, { insertText: `${LIMIT} `, sortText: CompletionItemPriority.MediumLow });
break;
case SuggestionKind.SortOrderDirectionKeyword:
[ASC, DESC].map((s) =>
addSuggestion(s, {
insertText: `${s} `,
command: TRIGGER_SUGGEST,
})
);
break;
}
}
// always suggest template variables
this.templateVariables.map((v) => {
addSuggestion(v, {
range,
label: v,
insertText: v,
kind: monaco.languages.CompletionItemKind.Variable,
sortText: CompletionItemPriority.Low,
});
});
return suggestions;
}
}

150
public/app/plugins/datasource/cloudwatch/cloudwatch-sql/completion/LinkedToken.ts Normal file
View File

@ -0,0 +1,150 @@
import { monacoTypes } from '@grafana/ui';
import { TokenType } from './types';
export class LinkedToken {
constructor(
public type: string,
public value: string,
public range: monacoTypes.IRange,
public previous: LinkedToken | null,
public next: LinkedToken | null
) {}
isKeyword(): boolean {
return this.type === TokenType.Keyword;
}
isWhiteSpace(): boolean {
return this.type === TokenType.Whitespace;
}
isParenthesis(): boolean {
return this.type === TokenType.Parenthesis;
}
isIdentifier(): boolean {
return this.type === TokenType.Identifier;
}
isString(): boolean {
return this.type === TokenType.String;
}
isDoubleQuotedString(): boolean {
return this.type === TokenType.Type;
}
isVariable(): boolean {
return this.type === TokenType.Variable;
}
isFunction(): boolean {
return this.type === TokenType.Function;
}
is(type: TokenType, value?: string | number | boolean): boolean {
const isType = this.type === type;
return value !== undefined ? isType && this.value === value : isType;
}
getPreviousNonWhiteSpaceToken(): LinkedToken | null {
let curr = this.previous;
while (curr != null) {
if (!curr.isWhiteSpace()) {
return curr;
}
curr = curr.previous;
}
return null;
}
getPreviousOfType(type: TokenType, value?: string): LinkedToken | null {
let curr = this.previous;
while (curr != null) {
const isType = curr.type === type;
if (value !== undefined ? isType && curr.value === value : isType) {
return curr;
}
curr = curr.previous;
}
return null;
}
getPreviousUntil(type: TokenType, ignoreTypes: TokenType[], value?: string): LinkedToken[] | null {
let tokens: LinkedToken[] = [];
let curr = this.previous;
while (curr != null) {
if (ignoreTypes.some((t) => t === curr?.type)) {
curr = curr.previous;
continue;
}
const isType = curr.type === type;
if (value !== undefined ? isType && curr.value === value : isType) {
return tokens;
}
if (!curr.isWhiteSpace()) {
tokens.push(curr);
}
curr = curr.previous;
}
return tokens;
}
getNextUntil(type: TokenType, ignoreTypes: TokenType[], value?: string): LinkedToken[] | null {
let tokens: LinkedToken[] = [];
let curr = this.next;
while (curr != null) {
if (ignoreTypes.some((t) => t === curr?.type)) {
curr = curr.next;
continue;
}
const isType = curr.type === type;
if (value !== undefined ? isType && curr.value === value : isType) {
return tokens;
}
if (!curr.isWhiteSpace()) {
tokens.push(curr);
}
curr = curr.next;
}
return tokens;
}
getPreviousKeyword(): LinkedToken | null {
let curr = this.previous;
while (curr != null) {
if (curr.isKeyword()) {
return curr;
}
curr = curr.previous;
}
return null;
}
getNextNonWhiteSpaceToken(): LinkedToken | null {
let curr = this.next;
while (curr != null) {
if (!curr.isWhiteSpace()) {
return curr;
}
curr = curr.next;
}
return null;
}
getNextOfType(type: TokenType, value?: string): LinkedToken | null {
let curr = this.next;
while (curr != null) {
const isType = curr.type === type;
if (value !== undefined ? isType && curr.value === value : isType) {
return curr;
}
curr = curr.next;
}
return null;
}
}

4
public/app/plugins/datasource/cloudwatch/cloudwatch-sql/completion/commands.ts Normal file
View File

@ -0,0 +1,4 @@
export const TRIGGER_SUGGEST = {
id: 'editor.action.triggerSuggest',
title: '',
};

58
public/app/plugins/datasource/cloudwatch/cloudwatch-sql/completion/linkedTokenBuilder.test.ts Normal file
View File

@ -0,0 +1,58 @@
import { monacoTypes } from '@grafana/ui';
import MonacoMock from '../../__mocks__/cloudwatch-sql/Monaco';
import TextModel from '../../__mocks__/cloudwatch-sql/TextModel';
import { multiLineFullQuery, singleLineFullQuery } from '../../__mocks__/cloudwatch-sql/test-data';
import { linkedTokenBuilder } from './linkedTokenBuilder';
import { TokenType } from './types';
import { DESC, SELECT } from '../language';
describe('linkedTokenBuilder', () => {
describe('singleLineFullQuery', () => {
const testModel = TextModel(singleLineFullQuery.query);
it('should add correct references to next LinkedToken', () => {
const position: monacoTypes.IPosition = { lineNumber: 1, column: 0 };
const current = linkedTokenBuilder(MonacoMock, testModel as monacoTypes.editor.ITextModel, position);
expect(current?.is(TokenType.Keyword, SELECT)).toBeTruthy();
expect(current?.getNextNonWhiteSpaceToken()?.is(TokenType.Function, 'AVG')).toBeTruthy();
});
it('should add correct references to previous LinkedToken', () => {
const position: monacoTypes.IPosition = { lineNumber: 1, column: singleLineFullQuery.query.length };
const current = linkedTokenBuilder(MonacoMock, testModel as monacoTypes.editor.ITextModel, position);
expect(current?.is(TokenType.Number, '10')).toBeTruthy();
expect(current?.getPreviousNonWhiteSpaceToken()?.is(TokenType.Keyword, 'LIMIT')).toBeTruthy();
expect(
current?.getPreviousNonWhiteSpaceToken()?.getPreviousNonWhiteSpaceToken()?.is(TokenType.Keyword, DESC)
).toBeTruthy();
});
});
describe('multiLineFullQuery', () => {
const testModel = TextModel(multiLineFullQuery.query);
it('should add LinkedToken with whitespace in case empty lines', () => {
const position: monacoTypes.IPosition = { lineNumber: 3, column: 0 };
const current = linkedTokenBuilder(MonacoMock, testModel as monacoTypes.editor.ITextModel, position);
expect(current).not.toBeNull();
expect(current?.isWhiteSpace()).toBeTruthy();
});
it('should add correct references to next LinkedToken', () => {
const position: monacoTypes.IPosition = { lineNumber: 1, column: 0 };
const current = linkedTokenBuilder(MonacoMock, testModel as monacoTypes.editor.ITextModel, position);
expect(current?.is(TokenType.Keyword, SELECT)).toBeTruthy();
expect(current?.getNextNonWhiteSpaceToken()?.is(TokenType.Function, 'AVG')).toBeTruthy();
});
it('should add correct references to previous LinkedToken even when references spans over multiple lines', () => {
const position: monacoTypes.IPosition = { lineNumber: 6, column: 7 };
const current = linkedTokenBuilder(MonacoMock, testModel as monacoTypes.editor.ITextModel, position);
expect(current?.is(TokenType.Number, '10')).toBeTruthy();
expect(current?.getPreviousNonWhiteSpaceToken()?.is(TokenType.Keyword, 'LIMIT')).toBeTruthy();
expect(
current?.getPreviousNonWhiteSpaceToken()?.getPreviousNonWhiteSpaceToken()?.is(TokenType.Keyword, DESC)
).toBeTruthy();
});
});
});

55
public/app/plugins/datasource/cloudwatch/cloudwatch-sql/completion/linkedTokenBuilder.ts Normal file
View File

@ -0,0 +1,55 @@
import type { monacoTypes } from '@grafana/ui';
import language from '../definition';
import { LinkedToken } from './LinkedToken';
import { Monaco, TokenType } from './types';
export function linkedTokenBuilder(
monaco: Monaco,
model: monacoTypes.editor.ITextModel,
position: monacoTypes.IPosition
) {
let current: LinkedToken | null = null;
let previous: LinkedToken | null = null;
const tokensPerLine = monaco.editor.tokenize(model.getValue() ?? '', language.id);
for (let lineIndex = 0; lineIndex < tokensPerLine.length; lineIndex++) {
const tokens = tokensPerLine[lineIndex];
// In case position is first column in new line, add empty whitespace token so that links are not broken
if (!tokens.length && previous) {
const token: monacoTypes.Token = {
offset: 0,
type: TokenType.Whitespace,
language: language.id,
_tokenBrand: undefined,
};
tokens.push(token);
}
for (let columnIndex = 0; columnIndex < tokens.length; columnIndex++) {
const token = tokens[columnIndex];
let endColumn =
tokens.length > columnIndex + 1 ? tokens[columnIndex + 1].offset + 1 : model.getLineLength(lineIndex + 1) + 1;
const range: monacoTypes.IRange = {
startLineNumber: lineIndex + 1,
startColumn: token.offset === 0 ? 0 : token.offset + 1,
endLineNumber: lineIndex + 1,
endColumn,
};
const value = model.getValueInRange(range);
const sqlToken: LinkedToken = new LinkedToken(token.type, value, range, previous, null);
if (monaco.Range.containsPosition(range, position)) {
current = sqlToken;
}
if (previous) {
previous.next = sqlToken;
}
previous = sqlToken;
}
}
return current;
}

157
public/app/plugins/datasource/cloudwatch/cloudwatch-sql/completion/statementPosition.test.ts Normal file
View File

@ -0,0 +1,157 @@
import { monacoTypes } from '@grafana/ui';
import MonacoMock from '../../__mocks__/cloudwatch-sql/Monaco';
import TextModel from '../../__mocks__/cloudwatch-sql/TextModel';
import {
multiLineFullQuery,
singleLineFullQuery,
singleLineEmptyQuery,
singleLineTwoQueries,
} from '../../__mocks__/cloudwatch-sql/test-data';
import { linkedTokenBuilder } from './linkedTokenBuilder';
import { StatementPosition } from './types';
import { getStatementPosition } from './statementPosition';
describe('statementPosition', () => {
function assertPosition(query: string, position: monacoTypes.IPosition, expected: StatementPosition) {
const testModel = TextModel(query);
const current = linkedTokenBuilder(MonacoMock, testModel as monacoTypes.editor.ITextModel, position);
const statementPosition = getStatementPosition(current);
expect(statementPosition).toBe(expected);
}
test.each([
[singleLineFullQuery.query, { lineNumber: 1, column: 0 }],
[multiLineFullQuery.query, { lineNumber: 1, column: 0 }],
[singleLineEmptyQuery.query, { lineNumber: 1, column: 0 }],
[singleLineTwoQueries.query, { lineNumber: 1, column: 154 }],
])('should be before select keyword', (query: string, position: monacoTypes.IPosition) => {
assertPosition(query, position, StatementPosition.SelectKeyword);
});
test.each([
[singleLineFullQuery.query, { lineNumber: 1, column: 7 }],
[multiLineFullQuery.query, { lineNumber: 1, column: 7 }],
[singleLineTwoQueries.query, { lineNumber: 1, column: 161 }],
])('should be after select keyword', (query: string, position: monacoTypes.IPosition) => {
assertPosition(query, position, StatementPosition.AfterSelectKeyword);
});
test.each([
[singleLineFullQuery.query, { lineNumber: 1, column: 12 }],
[multiLineFullQuery.query, { lineNumber: 1, column: 12 }],
[singleLineTwoQueries.query, { lineNumber: 1, column: 166 }],
])('should be first argument in select statistic function', (query: string, position: monacoTypes.IPosition) => {
assertPosition(query, position, StatementPosition.AfterSelectFuncFirstArgument);
});
test.each([
[singleLineFullQuery.query, { lineNumber: 1, column: 27 }],
[multiLineFullQuery.query, { lineNumber: 2, column: 0 }],
[singleLineTwoQueries.query, { lineNumber: 1, column: 181 }],
])('should be before the FROM keyword', (query: string, position: monacoTypes.IPosition) => {
assertPosition(query, position, StatementPosition.FromKeyword);
});
test.each([
[singleLineFullQuery.query, { lineNumber: 1, column: 32 }],
[multiLineFullQuery.query, { lineNumber: 2, column: 5 }],
[singleLineTwoQueries.query, { lineNumber: 1, column: 186 }],
])('should after the FROM keyword', (query: string, position: monacoTypes.IPosition) => {
assertPosition(query, position, StatementPosition.AfterFromKeyword);
});
test.each([
[singleLineFullQuery.query, { lineNumber: 1, column: 40 }],
[multiLineFullQuery.query, { lineNumber: 2, column: 13 }],
[singleLineTwoQueries.query, { lineNumber: 1, column: 40 }],
])('should be namespace arg in the schema func', (query: string, position: monacoTypes.IPosition) => {
assertPosition(query, position, StatementPosition.SchemaFuncFirstArgument);
});
test.each([
[singleLineFullQuery.query, { lineNumber: 1, column: 50 }],
[multiLineFullQuery.query, { lineNumber: 2, column: 23 }],
[singleLineTwoQueries.query, { lineNumber: 1, column: 50 }],
])('should be label key args within the schema func', (query: string, position: monacoTypes.IPosition) => {
assertPosition(query, position, StatementPosition.SchemaFuncExtraArgument);
});
test.each([
[singleLineFullQuery.query, { lineNumber: 1, column: 63 }],
[multiLineFullQuery.query, { lineNumber: 3, column: 0 }],
[singleLineTwoQueries.query, { lineNumber: 1, column: 63 }],
])('should be after from schema/namespace', (query: string, position: monacoTypes.IPosition) => {
assertPosition(query, position, StatementPosition.AfterFrom);
});
test.each([
[singleLineFullQuery.query, { lineNumber: 1, column: 69 }],
[multiLineFullQuery.query, { lineNumber: 4, column: 6 }],
[singleLineTwoQueries.query, { lineNumber: 1, column: 69 }],
])('should after where keyword and before label key', (query: string, position: monacoTypes.IPosition) => {
assertPosition(query, position, StatementPosition.WhereKey);
});
test.each([
[singleLineFullQuery.query, { lineNumber: 1, column: 79 }],
[multiLineFullQuery.query, { lineNumber: 4, column: 17 }],
[singleLineTwoQueries.query, { lineNumber: 1, column: 79 }],
])('should be before the comparison operator in a where filter', (query: string, position: monacoTypes.IPosition) => {
assertPosition(query, position, StatementPosition.WhereComparisonOperator);
});
test.each([
[singleLineFullQuery.query, { lineNumber: 1, column: 81 }],
[multiLineFullQuery.query, { lineNumber: 4, column: 19 }],
[singleLineTwoQueries.query, { lineNumber: 1, column: 81 }],
])('should be before or in the value in a where filter', (query: string, position: monacoTypes.IPosition) => {
assertPosition(query, position, StatementPosition.WhereValue);
});
test.each([
[singleLineFullQuery.query, { lineNumber: 1, column: 105 }],
[multiLineFullQuery.query, { lineNumber: 5, column: 0 }],
[singleLineTwoQueries.query, { lineNumber: 1, column: 105 }],
])('should be after a where value', (query: string, position: monacoTypes.IPosition) => {
assertPosition(query, position, StatementPosition.AfterWhereValue);
});
test.each([
[singleLineFullQuery.query, { lineNumber: 1, column: 115 }],
[multiLineFullQuery.query, { lineNumber: 5, column: 10 }],
[singleLineTwoQueries.query, { lineNumber: 1, column: 115 }],
])('should be after group by keywords', (query: string, position: monacoTypes.IPosition) => {
assertPosition(query, position, StatementPosition.AfterGroupByKeywords);
});
test.each([
[singleLineFullQuery.query, { lineNumber: 1, column: 123 }],
[multiLineFullQuery.query, { lineNumber: 5, column: 22 }],
[singleLineTwoQueries.query, { lineNumber: 1, column: 123 }],
])('should be after group by labels', (query: string, position: monacoTypes.IPosition) => {
assertPosition(query, position, StatementPosition.AfterGroupBy);
});
test.each([
[singleLineFullQuery.query, { lineNumber: 1, column: 132 }],
[multiLineFullQuery.query, { lineNumber: 5, column: 31 }],
[singleLineTwoQueries.query, { lineNumber: 1, column: 132 }],
])('should be after order by keywords', (query: string, position: monacoTypes.IPosition) => {
assertPosition(query, position, StatementPosition.AfterOrderByKeywords);
});
test.each([
[singleLineFullQuery.query, { lineNumber: 1, column: 138 }],
[multiLineFullQuery.query, { lineNumber: 5, column: 37 }],
[singleLineTwoQueries.query, { lineNumber: 1, column: 138 }],
])('should be after order by function', (query: string, position: monacoTypes.IPosition) => {
assertPosition(query, position, StatementPosition.AfterOrderByFunction);
});
test.each([
[singleLineFullQuery.query, { lineNumber: 1, column: 143 }],
[multiLineFullQuery.query, { lineNumber: 6, column: 0 }],
[singleLineTwoQueries.query, { lineNumber: 1, column: 145 }],
])('should be after order by direction', (query: string, position: monacoTypes.IPosition) => {
assertPosition(query, position, StatementPosition.AfterOrderByDirection);
});
});

125
public/app/plugins/datasource/cloudwatch/cloudwatch-sql/completion/statementPosition.ts Normal file
View File

@ -0,0 +1,125 @@
import { AND, ASC, BY, DESC, EQUALS, FROM, GROUP, NOT_EQUALS, ORDER, SCHEMA, SELECT, WHERE } from '../language';
import { LinkedToken } from './LinkedToken';
import { StatementPosition, TokenType } from './types';
export function getStatementPosition(currentToken: LinkedToken | null): StatementPosition {
const previousNonWhiteSpace = currentToken?.getPreviousNonWhiteSpaceToken();
const previousKeyword = currentToken?.getPreviousKeyword();
const previousIsSlash = currentToken?.getPreviousNonWhiteSpaceToken()?.is(TokenType.Operator, '/');
if (
currentToken === null ||
(currentToken.isWhiteSpace() && currentToken.previous === null) ||
(currentToken.is(TokenType.Keyword, SELECT) && currentToken.previous === null) ||
previousIsSlash ||
(currentToken.isIdentifier() && (previousIsSlash || currentToken?.previous === null))
) {
return StatementPosition.SelectKeyword;
}
if (previousNonWhiteSpace?.value === SELECT) {
return StatementPosition.AfterSelectKeyword;
}
if (
(previousNonWhiteSpace?.is(TokenType.Parenthesis, '(') || currentToken?.is(TokenType.Parenthesis, '()')) &&
previousKeyword?.value === SELECT
) {
return StatementPosition.AfterSelectFuncFirstArgument;
}
if (previousKeyword?.value === SELECT && previousNonWhiteSpace?.isParenthesis()) {
return StatementPosition.FromKeyword;
}
if (previousNonWhiteSpace?.value === FROM) {
return StatementPosition.AfterFromKeyword;
}
if (
(previousNonWhiteSpace?.is(TokenType.Parenthesis, '(') || currentToken?.is(TokenType.Parenthesis, '()')) &&
previousKeyword?.value === SCHEMA
) {
return StatementPosition.SchemaFuncFirstArgument;
}
if (previousKeyword?.value === SCHEMA && previousNonWhiteSpace?.is(TokenType.Delimiter, ',')) {
return StatementPosition.SchemaFuncExtraArgument;
}
if (
(previousKeyword?.value === FROM && previousNonWhiteSpace?.isDoubleQuotedString()) ||
(previousKeyword?.value === FROM && previousNonWhiteSpace?.isVariable()) ||
(previousKeyword?.value === SCHEMA && previousNonWhiteSpace?.is(TokenType.Parenthesis, ')'))
) {
return StatementPosition.AfterFrom;
}
if (
previousKeyword?.value === WHERE &&
(previousNonWhiteSpace?.isKeyword() ||
previousNonWhiteSpace?.is(TokenType.Parenthesis, '(') ||
previousNonWhiteSpace?.is(TokenType.Operator, AND))
) {
return StatementPosition.WhereKey;
}
if (
previousKeyword?.value === WHERE &&
(previousNonWhiteSpace?.isIdentifier() || previousNonWhiteSpace?.isDoubleQuotedString())
) {
return StatementPosition.WhereComparisonOperator;
}
if (
previousKeyword?.value === WHERE &&
(previousNonWhiteSpace?.is(TokenType.Operator, EQUALS) || previousNonWhiteSpace?.is(TokenType.Operator, NOT_EQUALS))
) {
return StatementPosition.WhereValue;
}
if (
previousKeyword?.value === WHERE &&
(previousNonWhiteSpace?.isString() || previousNonWhiteSpace?.is(TokenType.Parenthesis, ')'))
) {
return StatementPosition.AfterWhereValue;
}
if (
previousKeyword?.is(TokenType.Keyword, BY) &&
previousKeyword?.getPreviousKeyword()?.is(TokenType.Keyword, GROUP) &&
(previousNonWhiteSpace?.is(TokenType.Keyword, BY) || previousNonWhiteSpace?.is(TokenType.Delimiter, ','))
) {
return StatementPosition.AfterGroupByKeywords;
}
if (
previousKeyword?.is(TokenType.Keyword, BY) &&
previousKeyword?.getPreviousKeyword()?.is(TokenType.Keyword, GROUP) &&
(previousNonWhiteSpace?.isIdentifier() || previousNonWhiteSpace?.isDoubleQuotedString())
) {
return StatementPosition.AfterGroupBy;
}
if (
previousNonWhiteSpace?.is(TokenType.Keyword, BY) &&
previousNonWhiteSpace?.getPreviousKeyword()?.is(TokenType.Keyword, ORDER)
) {
return StatementPosition.AfterOrderByKeywords;
}
if (
previousKeyword?.is(TokenType.Keyword, BY) &&
previousKeyword?.getPreviousKeyword()?.is(TokenType.Keyword, ORDER) &&
previousNonWhiteSpace?.is(TokenType.Parenthesis) &&
previousNonWhiteSpace?.getPreviousNonWhiteSpaceToken()?.is(TokenType.Function)
) {
return StatementPosition.AfterOrderByFunction;
}
if (previousKeyword?.is(TokenType.Keyword, DESC) || previousKeyword?.is(TokenType.Keyword, ASC)) {
return StatementPosition.AfterOrderByDirection;
}
return StatementPosition.Unknown;
}

52
public/app/plugins/datasource/cloudwatch/cloudwatch-sql/completion/suggestionKind.ts Normal file
View File

@ -0,0 +1,52 @@
import { StatementPosition, SuggestionKind } from './types';
export function getSuggestionKinds(statementPosition: StatementPosition): SuggestionKind[] {
switch (statementPosition) {
case StatementPosition.SelectKeyword:
return [SuggestionKind.SelectKeyword];
case StatementPosition.AfterSelectKeyword:
return [SuggestionKind.FunctionsWithArguments];
case StatementPosition.AfterSelectFuncFirstArgument:
return [SuggestionKind.Metrics];
case StatementPosition.AfterFromKeyword:
return [SuggestionKind.Namespaces, SuggestionKind.SchemaKeyword];
case StatementPosition.SchemaFuncFirstArgument:
return [SuggestionKind.Namespaces];
case StatementPosition.SchemaFuncExtraArgument:
return [SuggestionKind.LabelKeys];
case StatementPosition.FromKeyword:
return [SuggestionKind.FromKeyword];
case StatementPosition.AfterFrom:
return [
SuggestionKind.WhereKeyword,
SuggestionKind.GroupByKeywords,
SuggestionKind.OrderByKeywords,
SuggestionKind.LimitKeyword,
];
case StatementPosition.WhereKey:
return [SuggestionKind.LabelKeys];
case StatementPosition.WhereComparisonOperator:
return [SuggestionKind.ComparisonOperators];
case StatementPosition.WhereValue:
return [SuggestionKind.LabelValues];
case StatementPosition.AfterWhereValue:
return [
SuggestionKind.LogicalOperators,
SuggestionKind.GroupByKeywords,
SuggestionKind.OrderByKeywords,
SuggestionKind.LimitKeyword,
];
case StatementPosition.AfterGroupByKeywords:
return [SuggestionKind.LabelKeys];
case StatementPosition.AfterGroupBy:
return [SuggestionKind.OrderByKeywords, SuggestionKind.LimitKeyword];
case StatementPosition.AfterOrderByKeywords:
return [SuggestionKind.FunctionsWithoutArguments];
case StatementPosition.AfterOrderByFunction:
return [SuggestionKind.SortOrderDirectionKeyword, SuggestionKind.LimitKeyword];
case StatementPosition.AfterOrderByDirection:
return [SuggestionKind.LimitKeyword];
}
return [];
}

94
public/app/plugins/datasource/cloudwatch/cloudwatch-sql/completion/tokenUtils.test.ts Normal file
View File

@ -0,0 +1,94 @@
import { monacoTypes } from '@grafana/ui';
import { LinkedToken } from './LinkedToken';
import MonacoMock from '../../__mocks__/cloudwatch-sql/Monaco';
import TextModel from '../../__mocks__/cloudwatch-sql/TextModel';
import {
multiLineFullQuery,
singleLineFullQuery,
singleLineTwoQueries,
multiLineIncompleteQueryWithoutNamespace,
} from '../../__mocks__/cloudwatch-sql/test-data';
import { linkedTokenBuilder } from './linkedTokenBuilder';
import { TokenType } from './types';
import { getMetricNameToken, getNamespaceToken, getSelectStatisticToken, getSelectToken } from './tokenUtils';
import { SELECT } from '../language';
const getToken = (
query: string,
position: monacoTypes.IPosition,
invokeFunction: (token: LinkedToken | null) => LinkedToken | null
) => {
const testModel = TextModel(query);
const current = linkedTokenBuilder(MonacoMock, testModel as monacoTypes.editor.ITextModel, position);
return invokeFunction(current);
};
describe('tokenUtils', () => {
test.each([
[singleLineFullQuery.query, { lineNumber: 1, column: 50 }],
[multiLineFullQuery.query, { lineNumber: 5, column: 10 }],
[singleLineTwoQueries.query, { lineNumber: 1, column: 30 }],
[singleLineTwoQueries.query, { lineNumber: 1, column: 185 }],
])('getSelectToken should return the right token', (query: string, position: monacoTypes.IPosition) => {
const token = getToken(query, position, getSelectToken);
expect(token).not.toBeNull();
expect(token?.value).toBe(SELECT);
expect(token?.type).toBe(TokenType.Keyword);
});
test.each([
[singleLineFullQuery.query, { lineNumber: 1, column: 50 }],
[multiLineFullQuery.query, { lineNumber: 5, column: 10 }],
[singleLineTwoQueries.query, { lineNumber: 1, column: 30 }],
[singleLineTwoQueries.query, { lineNumber: 1, column: 185 }],
])('getSelectToken should return the right token', (query: string, position: monacoTypes.IPosition) => {
const token = getToken(query, position, getSelectStatisticToken);
expect(token).not.toBeNull();
expect(token?.type).toBe(TokenType.Function);
});
test.each([
[singleLineFullQuery.query, 'AVG', { lineNumber: 1, column: 50 }],
[multiLineFullQuery.query, 'AVG', { lineNumber: 5, column: 10 }],
[singleLineTwoQueries.query, 'AVG', { lineNumber: 1, column: 30 }],
[singleLineTwoQueries.query, 'SUM', { lineNumber: 1, column: 185 }],
])(
'getSelectStatisticToken should return the right token',
(query: string, value: string, position: monacoTypes.IPosition) => {
const token = getToken(query, position, getSelectStatisticToken);
expect(token).not.toBeNull();
expect(token?.value).toBe(value);
expect(token?.type).toBe(TokenType.Function);
}
);
test.each([
[singleLineFullQuery.query, 'CPUUtilization', { lineNumber: 1, column: 50 }],
[multiLineFullQuery.query, 'CPUUtilization', { lineNumber: 5, column: 10 }],
[singleLineTwoQueries.query, 'CPUUtilization', { lineNumber: 1, column: 30 }],
[singleLineTwoQueries.query, 'CPUCreditUsage', { lineNumber: 1, column: 185 }],
])(
'getMetricNameToken should return the right token',
(query: string, value: string, position: monacoTypes.IPosition) => {
const token = getToken(query, position, getMetricNameToken);
expect(token).not.toBeNull();
expect(token?.value).toBe(value);
expect(token?.type).toBe(TokenType.Identifier);
}
);
test.each([
[singleLineFullQuery.query, '"AWS/EC2"', TokenType.Type, { lineNumber: 1, column: 50 }],
[multiLineFullQuery.query, '"AWS/ECS"', TokenType.Type, { lineNumber: 5, column: 10 }],
[singleLineTwoQueries.query, '"AWS/EC2"', TokenType.Type, { lineNumber: 1, column: 30 }],
[singleLineTwoQueries.query, '"AWS/ECS"', TokenType.Type, { lineNumber: 1, column: 185 }],
[multiLineIncompleteQueryWithoutNamespace.query, undefined, undefined, { lineNumber: 2, column: 5 }],
])(
'getNamespaceToken should return the right token',
(query: string, value: string | undefined, tokenType: TokenType | undefined, position: monacoTypes.IPosition) => {
const token = getToken(query, position, getNamespaceToken);
expect(token?.value).toBe(value);
expect(token?.type).toBe(tokenType);
}
);
});

41
public/app/plugins/datasource/cloudwatch/cloudwatch-sql/completion/tokenUtils.ts Normal file
View File

@ -0,0 +1,41 @@
import { LinkedToken } from './LinkedToken';
import { FROM, SCHEMA, SELECT } from '../language';
import { TokenType } from './types';
export const getSelectToken = (currentToken: LinkedToken | null) =>
currentToken?.getPreviousOfType(TokenType.Keyword, SELECT) ?? null;
export const getSelectStatisticToken = (currentToken: LinkedToken | null) => {
const assumedStatisticToken = getSelectToken(currentToken)?.getNextNonWhiteSpaceToken();
return assumedStatisticToken?.isVariable() || assumedStatisticToken?.isFunction() ? assumedStatisticToken : null;
};
export const getMetricNameToken = (currentToken: LinkedToken | null) => {
// statistic function is followed by `(` and then an argument
const assumedMetricNameToken = getSelectStatisticToken(currentToken)?.next?.next;
return assumedMetricNameToken?.isVariable() || assumedMetricNameToken?.isIdentifier() ? assumedMetricNameToken : null;
};
export const getFromKeywordToken = (currentToken: LinkedToken | null) => {
const selectToken = getSelectToken(currentToken);
return selectToken?.getNextOfType(TokenType.Keyword, FROM);
};
export const getNamespaceToken = (currentToken: LinkedToken | null) => {
const fromToken = getFromKeywordToken(currentToken);
const nextNonWhiteSpace = fromToken?.getNextNonWhiteSpaceToken();
if (
nextNonWhiteSpace?.isDoubleQuotedString() ||
(nextNonWhiteSpace?.isVariable() && nextNonWhiteSpace?.value.toUpperCase() !== SCHEMA)
) {
// schema is not used
return nextNonWhiteSpace;
} else if (nextNonWhiteSpace?.isKeyword() && nextNonWhiteSpace.next?.is(TokenType.Parenthesis, '(')) {
// schema is specified
const assumedNamespaceToken = nextNonWhiteSpace.next?.next;
if (assumedNamespaceToken?.isDoubleQuotedString() || assumedNamespaceToken?.isVariable()) {
return assumedNamespaceToken;
}
}
return null;
};

76
public/app/plugins/datasource/cloudwatch/cloudwatch-sql/completion/types.ts Normal file
View File

@ -0,0 +1,76 @@
import { monacoTypes } from '@grafana/ui';
export enum TokenType {
Parenthesis = 'delimiter.parenthesis.sql',
Whitespace = 'white.sql',
Keyword = 'keyword.sql',
Delimiter = 'delimiter.sql',
Operator = 'operator.sql',
Identifier = 'identifier.sql',
Type = 'type.sql',
Function = 'predefined.sql',
Number = 'number.sql',
String = 'string.sql',
Variable = 'variable.sql',
}
export enum StatementPosition {
Unknown,
SelectKeyword,
AfterSelectKeyword,
AfterSelectFuncFirstArgument,
AfterFromKeyword,
SchemaFuncFirstArgument,
SchemaFuncExtraArgument,
FromKeyword,
AfterFrom,
WhereKey,
WhereComparisonOperator,
WhereValue,
AfterWhereValue,
AfterGroupByKeywords,
AfterGroupBy,
AfterOrderByKeywords,
AfterOrderByFunction,
AfterOrderByDirection,
}
export enum SuggestionKind {
SelectKeyword,
FunctionsWithArguments,
Metrics,
FromKeyword,
SchemaKeyword,
Namespaces,
LabelKeys,
WhereKeyword,
GroupByKeywords,
OrderByKeywords,
FunctionsWithoutArguments,
LimitKeyword,
SortOrderDirectionKeyword,
ComparisonOperators,
LabelValues,
LogicalOperators,
}
export enum CompletionItemPriority {
High = 'a',
MediumHigh = 'd',
Medium = 'g',
MediumLow = 'k',
Low = 'q',
}
export interface Editor {
tokenize: (value: string, languageId: string) => monacoTypes.Token[][];
}
export interface Range {
containsPosition: (range: monacoTypes.IRange, position: monacoTypes.IPosition) => boolean;
}
export interface Monaco {
editor: Editor;
Range: Range;
}

7
public/app/plugins/datasource/cloudwatch/cloudwatch-sql/definition.ts Normal file
View File

@ -0,0 +1,7 @@
export default {
id: 'cloudwatch-sql',
extensions: ['.cloudwatchSql'],
aliases: ['CloudWatch', 'cloudwatch', 'CloudWatchSQL'],
mimetypes: [],
loader: () => import('./language'),
};

131
public/app/plugins/datasource/cloudwatch/cloudwatch-sql/language.ts Normal file
View File

@ -0,0 +1,131 @@
import type * as monacoType from 'monaco-editor/esm/vs/editor/editor.api';
interface CloudWatchLanguage extends monacoType.languages.IMonarchLanguage {
keywords: string[];
operators: string[];
builtinFunctions: string[];
}
export const SELECT = 'SELECT';
export const FROM = 'FROM';
export const WHERE = 'WHERE';
export const GROUP = 'GROUP';
export const ORDER = 'ORDER';
export const BY = 'BY';
export const DESC = 'DESC';
export const ASC = 'ASC';
export const LIMIT = 'LIMIT';
export const WITH = 'WITH';
export const SCHEMA = 'SCHEMA';
export const KEYWORDS = [SELECT, FROM, WHERE, GROUP, ORDER, BY, DESC, ASC, LIMIT, WITH, SCHEMA];
export const STATISTICS = ['AVG', 'COUNT', 'MAX', 'MIN', 'SUM'];
export const AND = 'AND';
export const LOGICAL_OPERATORS = [AND];
export const EQUALS = '=';
export const NOT_EQUALS = '!=';
export const COMPARISON_OPERATORS = [EQUALS, NOT_EQUALS];
export const language: CloudWatchLanguage = {
defaultToken: '',
tokenPostfix: '.sql',
ignoreCase: true,
brackets: [
{ open: '[', close: ']', token: 'delimiter.square' },
{ open: '(', close: ')', token: 'delimiter.parenthesis' },
],
keywords: KEYWORDS,
operators: LOGICAL_OPERATORS,
builtinFunctions: STATISTICS,
tokenizer: {
root: [
[/\$[a-zA-Z0-9-_]+/, 'variable'],
{ include: '@comments' },
{ include: '@whitespace' },
{ include: '@numbers' },
{ include: '@strings' },
{ include: '@complexIdentifiers' },
[/[;,.]/, 'delimiter'],
[/[()]/, '@brackets'],
[
/[\w@#$]+/,
{
cases: {
'@keywords': 'keyword',
'@operators': 'operator',
'@builtinFunctions': 'predefined',
'@default': 'identifier',
},
},
],
[/[=!%&+\-*/|~^]/, 'operator'], // TODO: strip these options
],
whitespace: [[/\s+/, 'white']],
comments: [[/--+.*/, 'comment']],
comment: [
[/[^*/]+/, 'comment'],
[/./, 'comment'],
],
numbers: [
[/0[xX][0-9a-fA-F]*/, 'number'],
[/[$][+-]*\d*(\.\d*)?/, 'number'],
[/((\d+(\.\d*)?)|(\.\d+))([eE][\-+]?\d+)?/, 'number'],
],
strings: [
[/N'/, { token: 'string', next: '@string' }],
[/'/, { token: 'string', next: '@string' }],
[/"/, { token: 'type', next: '@string_double' }],
],
string: [
[/[^']+/, 'string'],
[/''/, 'string'],
[/'/, { token: 'string', next: '@pop' }],
],
string_double: [
[/[^\\"]+/, 'type'],
[/"/, 'type', '@pop'],
],
complexIdentifiers: [
[/\[/, { token: 'identifier.quote', next: '@bracketedIdentifier' }],
[/"/, { token: 'identifier.quote', next: '@quotedIdentifier' }],
],
bracketedIdentifier: [
[/[^\]]+/, 'identifier'],
[/]]/, 'identifier'],
[/]/, { token: 'identifier.quote', next: '@pop' }],
],
quotedIdentifier: [
[/[^"]+/, 'identifier'],
[/""/, 'identifier'],
[/"/, { token: 'identifier.quote', next: '@pop' }],
],
},
};
export const conf: monacoType.languages.LanguageConfiguration = {
comments: {
lineComment: '--',
blockComment: ['/*', '*/'],
},
brackets: [
['{', '}'],
['[', ']'],
['(', ')'],
],
autoClosingPairs: [
{ open: '{', close: '}' },
{ open: '[', close: ']' },
{ open: '(', close: ')' },
{ open: '"', close: '"' },
{ open: "'", close: "'" },
],
surroundingPairs: [
{ open: '{', close: '}' },
{ open: '[', close: ']' },
{ open: '(', close: ')' },
{ open: '"', close: '"' },
{ open: "'", close: "'" },
],
};

19
public/app/plugins/datasource/cloudwatch/cloudwatch-sql/register.ts Normal file
View File

@ -0,0 +1,19 @@
import { Monaco } from '@grafana/ui';
import { CompletionItemProvider } from './completion/CompletionItemProvider';
import language from './definition';
export const registerLanguage = (monaco: Monaco, sqlCompletionItemProvider: CompletionItemProvider) => {
const { id, loader } = language;
const languages = monaco.languages.getLanguages();
if (languages.find((l) => l.id === id)) {
return;
}
monaco.languages.register({ id });
loader().then((monarch) => {
monaco.languages.setMonarchTokensProvider(id, monarch.language);
monaco.languages.setLanguageConfiguration(id, monarch.conf);
monaco.languages.registerCompletionItemProvider(id, sqlCompletionItemProvider.getCompletionProvider(monaco));
});
};

10
public/app/plugins/datasource/cloudwatch/components/Alias.test.tsx
View File

@ -1,10 +0,0 @@
import React from 'react';
import renderer from 'react-test-renderer';
import { Alias } from './Alias';
describe('Alias', () => {
it('should render component', () => {
const tree = renderer.create(<Alias value={'legend'} onChange={() => {}} />).toJSON();
expect(tree).toMatchSnapshot();
});
});

5
public/app/plugins/datasource/cloudwatch/components/Alias.tsx
View File

@ -1,7 +1,6 @@
import React, { FunctionComponent, useState } from 'react';
import { debounce } from 'lodash';
import { LegacyForms } from '@grafana/ui';
const { Input } = LegacyForms;
import { Input } from '@grafana/ui';
export interface Props {
onChange: (alias: any) => void;
@ -18,5 +17,5 @@ export const Alias: FunctionComponent<Props> = ({ value = '', onChange }) => {
propagateOnChange(e.target.value);
};
return <Input type="text" className="gf-form-input width-16" value={alias} onChange={onChange} />;
return <Input type="text" value={alias} onChange={onChange} />;
};

69
public/app/plugins/datasource/cloudwatch/components/AnnotationQueryEditor.test.tsx Normal file
View File

@ -0,0 +1,69 @@
import React from 'react';
import '@testing-library/jest-dom';
import { cleanup, render, screen, waitFor } from '@testing-library/react';
import { setupMockedDataSource } from '../__mocks__/CloudWatchDataSource';
import { CloudWatchAnnotationQuery } from '../types';
import { AnnotationQueryEditor } from './AnnotationQueryEditor';
import { act } from 'react-dom/test-utils';
const ds = setupMockedDataSource({
variables: [],
});
const q: CloudWatchAnnotationQuery = {
id: '',
region: 'us-east-2',
namespace: '',
period: '',
alias: '',
metricName: '',
dimensions: {},
matchExact: true,
statistic: '',
expression: '',
refId: '',
enable: true,
name: '',
iconColor: '',
prefixMatching: false,
actionPrefix: '',
alarmNamePrefix: '',
};
ds.datasource.getRegions = jest.fn().mockResolvedValue([]);
ds.datasource.getNamespaces = jest.fn().mockResolvedValue([]);
ds.datasource.getMetrics = jest.fn().mockResolvedValue([]);
ds.datasource.getDimensionKeys = jest.fn().mockResolvedValue([]);
ds.datasource.getVariables = jest.fn().mockReturnValue([]);
const props = {
datasource: ds.datasource,
query: q,
onChange: jest.fn(),
onRunQuery: jest.fn(),
};
afterEach(cleanup);
describe('AnnotationQueryEditor', () => {
it('should not display match exact switch', () => {
render(<AnnotationQueryEditor {...props} />);
expect(screen.queryByText('Match exact')).toBeNull();
});
it('shoud not display wildcard option in dimension value dropdown', async () => {
ds.datasource.getDimensionValues = jest.fn().mockResolvedValue([[{ label: 'dimVal1', value: 'dimVal1' }]]);
props.query.dimensions = { instanceId: 'instance-123' };
render(<AnnotationQueryEditor {...props} />);
const valueElement = screen.getByText('instance-123');
expect(valueElement).toBeInTheDocument();
expect(screen.queryByText('*')).toBeNull();
act(async () => {
await valueElement.click();
await waitFor(() => {
expect(screen.queryByText('*')).toBeNull();
});
});
});
});

112
public/app/plugins/datasource/cloudwatch/components/AnnotationQueryEditor.tsx
View File

@ -1,10 +1,15 @@
import React, { ChangeEvent } from 'react';
import { LegacyForms } from '@grafana/ui';
const { Switch } = LegacyForms;
import { Switch, Input } from '@grafana/ui';
import { CloudWatchAnnotationQuery, CloudWatchMetricsQuery } from '../types';
import { PanelData } from '@grafana/data';
import { CloudWatchAnnotationQuery, CloudWatchQuery } from '../types';
import { CloudWatchDatasource } from '../datasource';
import { QueryField, PanelQueryEditor } from './';
import { MetricStatEditor } from './MetricStatEditor';
import EditorHeader from './ui/EditorHeader';
import InlineSelect from './ui/InlineSelect';
import { Space } from './ui/Space';
import { useRegions } from '../hooks';
import EditorRow from './ui/EditorRow';
import EditorField from './ui/EditorField';
export type Props = {
query: CloudWatchAnnotationQuery;
@ -14,50 +19,69 @@ export type Props = {
};
export function AnnotationQueryEditor(props: React.PropsWithChildren<Props>) {
const { query, onChange } = props;
const { query, onChange, datasource } = props;
const [regions, regionIsLoading] = useRegions(datasource);
return (
<>
<PanelQueryEditor
{...props}
onChange={(editorQuery: CloudWatchQuery) => onChange({ ...query, ...editorQuery })}
onRunQuery={() => {}}
history={[]}
></PanelQueryEditor>
<div className="gf-form-inline">
<Switch
label="Enable Prefix Matching"
labelClass="query-keyword"
checked={query.prefixMatching}
onChange={() => onChange({ ...query, prefixMatching: !query.prefixMatching })}
<EditorHeader>
<InlineSelect
label="Region"
value={regions.find((v) => v.value === query.region)}
placeholder="Select region"
allowCustomValue
onChange={({ value: region }) => region && onChange({ ...query, region })}
options={regions}
isLoading={regionIsLoading}
/>
<div className="gf-form gf-form--grow">
<QueryField label="Action">
<input
disabled={!query.prefixMatching}
className="gf-form-input width-12"
value={query.actionPrefix || ''}
onChange={(event: ChangeEvent<HTMLInputElement>) =>
onChange({ ...query, actionPrefix: event.target.value })
}
/>
</QueryField>
<QueryField label="Alarm Name">
<input
disabled={!query.prefixMatching}
className="gf-form-input width-12"
value={query.alarmNamePrefix || ''}
onChange={(event: ChangeEvent<HTMLInputElement>) =>
onChange({ ...query, alarmNamePrefix: event.target.value })
}
/>
</QueryField>
<div className="gf-form gf-form--grow">
<div className="gf-form-label gf-form-label--grow" />
</div>
</div>
</div>
</EditorHeader>
<Space v={0.5} />
<MetricStatEditor
{...props}
disableExpressions={true}
onChange={(editorQuery: CloudWatchMetricsQuery) => onChange({ ...query, ...editorQuery })}
onRunQuery={() => {}}
></MetricStatEditor>
<Space v={0.5} />
<EditorRow>
<EditorField label="Period" width={26} tooltip="Minimum interval between points in seconds.">
<Input
value={query.period || ''}
placeholder="auto"
onChange={(event: ChangeEvent<HTMLInputElement>) => onChange({ ...query, period: event.target.value })}
/>
</EditorField>
<EditorField label="Enable Prefix Matching" optional={true}>
<Switch
value={query.prefixMatching}
onChange={(e) => {
onChange({
...query,
prefixMatching: e.currentTarget.checked,
});
}}
/>
</EditorField>
<EditorField label="Action" optional={true}>
<Input
disabled={!query.prefixMatching}
value={query.actionPrefix || ''}
onChange={(event: ChangeEvent<HTMLInputElement>) =>
onChange({ ...query, actionPrefix: event.target.value })
}
/>
</EditorField>
<EditorField label="Alarm Name" optional={true}>
<Input
disabled={!query.prefixMatching}
value={query.alarmNamePrefix || ''}
onChange={(event: ChangeEvent<HTMLInputElement>) =>
onChange({ ...query, alarmNamePrefix: event.target.value })
}
/>
</EditorField>
</EditorRow>
</>
);
}

46
public/app/plugins/datasource/cloudwatch/components/Dimensions.test.tsx
View File

@ -1,46 +0,0 @@
import React from 'react';
import { mount, shallow } from 'enzyme';
import { Dimensions } from './';
import { SelectableStrings } from '../types';
describe('Dimensions', () => {
it('renders', () => {
mount(
<Dimensions
dimensions={{}}
onChange={(dimensions) => console.log(dimensions)}
loadKeys={() => Promise.resolve<SelectableStrings>([])}
loadValues={() => Promise.resolve<SelectableStrings>([])}
/>
);
});
describe('and no dimension were passed to the component', () => {
it('initially displays just an add button', () => {
const wrapper = shallow(
<Dimensions
dimensions={{}}
onChange={() => {}}
loadKeys={() => Promise.resolve<SelectableStrings>([])}
loadValues={() => Promise.resolve<SelectableStrings>([])}
/>
);
expect(wrapper.html()).toEqual(expect.stringContaining(`gf-form`));
});
});
describe('and one dimension key along with a value were passed to the component', () => {
it('initially displays the dimension key, value and an add button', () => {
const wrapper = shallow(
<Dimensions
dimensions={{ somekey: 'somevalue' }}
onChange={() => {}}
loadKeys={() => Promise.resolve<SelectableStrings>([])}
loadValues={() => Promise.resolve<SelectableStrings>([])}
/>
);
expect(wrapper.html()).toEqual(expect.stringContaining(`gf-form`));
});
});
});

81
public/app/plugins/datasource/cloudwatch/components/Dimensions.tsx
View File

@ -1,81 +0,0 @@
import React, { FunctionComponent, Fragment, useState, useEffect } from 'react';
import { isEqual } from 'lodash';
import { SelectableValue } from '@grafana/data';
import { SegmentAsync, Icon } from '@grafana/ui';
import { SelectableStrings } from '../types';
export interface Props {
dimensions: { [key: string]: string | string[] };
onChange: (dimensions: { [key: string]: string }) => void;
loadValues: (key: string) => Promise<SelectableStrings>;
loadKeys: () => Promise<SelectableStrings>;
}
const removeText = '-- remove dimension --';
const removeOption: SelectableValue<string> = { label: removeText, value: removeText };
// The idea of this component is that is should only trigger the onChange event in the case
// there is a complete dimension object. E.g, when a new key is added is doesn't have a value.
// That should not trigger onChange.
export const Dimensions: FunctionComponent<Props> = ({ dimensions, loadValues, loadKeys, onChange }) => {
const [data, setData] = useState(dimensions);
useEffect(() => {
const completeDimensions = Object.entries(data).reduce(
(res, [key, value]) => (value ? { ...res, [key]: value } : res),
{}
);
if (!isEqual(completeDimensions, dimensions)) {
onChange(completeDimensions);
}
}, [data, dimensions, onChange]);
const excludeUsedKeys = (options: SelectableStrings) => {
return options.filter(({ value }) => !Object.keys(data).includes(value!));
};
return (
<>
{Object.entries(data).map(([key, value], index) => (
<Fragment key={index}>
<SegmentAsync
allowCustomValue
value={key}
loadOptions={() => loadKeys().then((keys) => [removeOption, ...excludeUsedKeys(keys)])}
onChange={({ value: newKey }) => {
const { [key]: value, ...newDimensions } = data;
if (newKey === removeText) {
setData({ ...newDimensions });
} else {
setData({ ...newDimensions, [newKey!]: '' });
}
}}
/>
<label className="gf-form-label query-segment-operator">=</label>
<SegmentAsync
allowCustomValue
value={value}
placeholder="select dimension value"
loadOptions={() => loadValues(key)}
onChange={({ value: newValue }) => setData({ ...data, [key]: newValue! })}
/>
{Object.values(data).length > 1 && index + 1 !== Object.values(data).length && (
<label className="gf-form-label query-keyword">AND</label>
)}
</Fragment>
))}
{Object.values(data).every((v) => v) && (
<SegmentAsync
allowCustomValue
Component={
<a className="gf-form-label query-part">
<Icon name="plus" />
</a>
}
loadOptions={() => loadKeys().then(excludeUsedKeys)}
onChange={({ value: newKey }) => setData({ ...data, [newKey!]: '' })}
/>
)}
</>
);
};

35
public/app/plugins/datasource/cloudwatch/components/LogsQueryField.tsx
View File

@ -171,25 +171,26 @@ export class CloudWatchLogsQueryField extends React.PureComponent<CloudWatchLogs
loadingLogGroups: true,
});
this.fetchLogGroupOptions(query.region).then((logGroups) => {
this.setState((state) => {
const selectedLogGroups = state.selectedLogGroups;
if (onChange) {
const nextQuery = {
...query,
logGroupNames: selectedLogGroups.map((group) => group.value!),
query.region &&
this.fetchLogGroupOptions(query.region).then((logGroups) => {
this.setState((state) => {
const selectedLogGroups = state.selectedLogGroups;
if (onChange) {
const nextQuery = {
...query,
logGroupNames: selectedLogGroups.map((group) => group.value!),
};
onChange(nextQuery);
}
return {
loadingLogGroups: false,
availableLogGroups: logGroups,
selectedLogGroups,
};
onChange(nextQuery);
}
return {
loadingLogGroups: false,
availableLogGroups: logGroups,
selectedLogGroups,
};
});
});
});
datasource.getRegions().then((regions) => {
this.setState({

28
public/app/plugins/datasource/cloudwatch/components/MathExpressionQueryField.tsx Normal file
View File

@ -0,0 +1,28 @@
import React from 'react';
import { Input } from '@grafana/ui';
export interface Props {
onChange: (query: string) => void;
onRunQuery: () => void;
expression: string;
}
export function MathExpressionQueryField({ expression: query, onChange, onRunQuery }: React.PropsWithChildren<Props>) {
const onKeyDown = (event: React.KeyboardEvent<HTMLInputElement>) => {
if (event.key === 'Enter' && (event.shiftKey || event.ctrlKey)) {
event.preventDefault();
onRunQuery();
}
};
return (
<Input
name="Query"
value={query}
placeholder="Enter a math expression"
onBlur={onRunQuery}
onChange={(e) => onChange(e.currentTarget.value)}
onKeyDown={onKeyDown}
/>
);
}

49
public/app/plugins/datasource/cloudwatch/components/MetaInspector.tsx Normal file
View File

@ -0,0 +1,49 @@
import React, { useMemo } from 'react';
import { MetadataInspectorProps } from '@grafana/data';
import { CloudWatchDatasource } from '../datasource';
import { CloudWatchQuery, CloudWatchJsonData } from '../types';
import { groupBy } from 'lodash';
export type Props = MetadataInspectorProps<CloudWatchDatasource, CloudWatchQuery, CloudWatchJsonData>;
export function MetaInspector({ data = [] }: Props) {
const rows = useMemo(() => groupBy(data, 'refId'), [data]);
return (
<>
<table className="filter-table form-inline">
<thead>
<tr>
<th>RefId</th>
<th>Metric Data Query ID</th>
<th>Metric Data Query Expression</th>
<th>Period</th>
<th />
</tr>
</thead>
{Object.entries(rows).map(([refId, frames], idx) => {
if (!frames.length) {
return null;
}
const frame = frames[0];
const custom = frame.meta?.custom;
if (!custom) {
return null;
}
return (
<tbody key={idx}>
<tr>
<td>{refId}</td>
<td>{custom.id}</td>
<td>{frame.meta?.executedQueryString}</td>
<td>{custom.period}</td>
</tr>
</tbody>
);
})}
</table>
</>
);
}

122
public/app/plugins/datasource/cloudwatch/components/MetricStatEditor/Dimensions.test.tsx Normal file
View File

@ -0,0 +1,122 @@
import React from 'react';
import { cleanup, fireEvent, render, screen } from '@testing-library/react';
import { setupMockedDataSource } from '../../__mocks__/CloudWatchDataSource';
import '@testing-library/jest-dom';
import { CloudWatchMetricsQuery } from '../../types';
import userEvent from '@testing-library/user-event';
import { Dimensions } from '..';
import { within } from '@testing-library/dom';
const ds = setupMockedDataSource({
variables: [],
});
ds.datasource.getNamespaces = jest.fn().mockResolvedValue([]);
ds.datasource.getMetrics = jest.fn().mockResolvedValue([]);
ds.datasource.getDimensionKeys = jest.fn().mockResolvedValue([]);
ds.datasource.getVariables = jest.fn().mockReturnValue([]);
const q: CloudWatchMetricsQuery = {
id: '',
region: 'us-east-2',
namespace: '',
period: '',
alias: '',
metricName: '',
dimensions: {},
matchExact: true,
statistic: '',
expression: '',
refId: '',
};
const props = {
datasource: ds.datasource,
query: q,
disableExpressions: false,
onChange: jest.fn(),
onRunQuery: jest.fn(),
};
afterEach(cleanup);
describe('Dimensions', () => {
describe('when rendered with two existing dimensions', () => {
it('should render two filter items', async () => {
props.query.dimensions = {
InstanceId: '*',
InstanceGroup: 'Group1',
};
render(<Dimensions {...props} query={props.query} dimensionKeys={[]} />);
const filterItems = screen.getAllByTestId('cloudwatch-dimensions-filter-item');
expect(filterItems.length).toBe(2);
expect(within(filterItems[0]).getByText('InstanceId')).toBeInTheDocument();
expect(within(filterItems[0]).getByText('*')).toBeInTheDocument();
expect(within(filterItems[1]).getByText('InstanceGroup')).toBeInTheDocument();
expect(within(filterItems[1]).getByText('Group1')).toBeInTheDocument();
});
});
describe('when adding a new filter item', () => {
it('it should add the new item but not call onChange', async () => {
props.query.dimensions = {};
const onChange = jest.fn();
render(<Dimensions {...props} query={props.query} onChange={onChange} dimensionKeys={[]} />);
userEvent.click(screen.getByLabelText('Add'));
expect(screen.getByTestId('cloudwatch-dimensions-filter-item')).toBeInTheDocument();
expect(onChange).not.toHaveBeenCalled();
});
});
describe('when adding a new filter item with key', () => {
it('it should add the new item but not call onChange', async () => {
props.query.dimensions = {};
const onChange = jest.fn();
const { container } = render(
<Dimensions {...props} query={props.query} onChange={onChange} dimensionKeys={[]} />
);
userEvent.click(screen.getByLabelText('Add'));
const filterItemElement = screen.getByTestId('cloudwatch-dimensions-filter-item');
expect(filterItemElement).toBeInTheDocument();
const keyElement = container.querySelector('#cloudwatch-dimensions-filter-item-key');
expect(keyElement).toBeInTheDocument();
userEvent.type(keyElement!, 'my-key');
fireEvent.keyDown(keyElement!, { keyCode: 13 });
expect(onChange).not.toHaveBeenCalled();
});
});
describe('when adding a new filter item with key and value', () => {
it('it should add the new item and trigger onChange', async () => {
props.query.dimensions = {};
const onChange = jest.fn();
const { container } = render(
<Dimensions {...props} query={props.query} onChange={onChange} dimensionKeys={[]} />
);
userEvent.click(screen.getByLabelText('Add'));
const filterItemElement = screen.getByTestId('cloudwatch-dimensions-filter-item');
expect(filterItemElement).toBeInTheDocument();
const keyElement = container.querySelector('#cloudwatch-dimensions-filter-item-key');
expect(keyElement).toBeInTheDocument();
userEvent.type(keyElement!, 'my-key');
fireEvent.keyDown(keyElement!, { keyCode: 13 });
expect(onChange).not.toHaveBeenCalled();
const valueElement = container.querySelector('#cloudwatch-dimensions-filter-item-value');
expect(valueElement).toBeInTheDocument();
userEvent.type(valueElement!, 'my-value');
fireEvent.keyDown(valueElement!, { keyCode: 13 });
expect(onChange).not.toHaveBeenCalledWith({
...props.query,
dimensions: {
'my-key': 'my-value',
},
});
});
});
});

93
public/app/plugins/datasource/cloudwatch/components/MetricStatEditor/Dimensions.tsx Normal file
View File

@ -0,0 +1,93 @@
import React, { useMemo, useState } from 'react';
import { isEqual } from 'lodash';
import { SelectableValue } from '@grafana/data';
import { Dimensions as DimensionsType, CloudWatchMetricsQuery } from '../../types';
import EditorList from '../ui/EditorList';
import { CloudWatchDatasource } from '../../datasource';
import { FilterItem } from './FilterItem';
export interface Props {
query: CloudWatchMetricsQuery;
onChange: (dimensions: DimensionsType) => void;
datasource: CloudWatchDatasource;
dimensionKeys: Array<SelectableValue<string>>;
disableExpressions: boolean;
}
export interface DimensionFilterCondition {
key?: string;
operator?: string;
value?: string;
}
const dimensionsToFilterConditions = (dimensions: DimensionsType | undefined) =>
Object.entries(dimensions ?? {}).reduce<DimensionFilterCondition[]>((acc, [key, value]) => {
if (value && typeof value === 'string') {
const filter = {
key,
value,
operator: '=',
};
return [...acc, filter];
}
return acc;
}, []);
const filterConditionsToDimensions = (filters: DimensionFilterCondition[]) => {
return filters.reduce<DimensionsType>((acc, { key, value }) => {
if (key && value) {
return { ...acc, [key]: value };
}
return acc;
}, {});
};
export const Dimensions: React.FC<Props> = ({ query, datasource, dimensionKeys, disableExpressions, onChange }) => {
const dimensionFilters = useMemo(() => dimensionsToFilterConditions(query.dimensions), [query.dimensions]);
const [items, setItems] = useState<DimensionFilterCondition[]>(dimensionFilters);
const onDimensionsChange = (newItems: Array<Partial<DimensionFilterCondition>>) => {
setItems(newItems);
// The onChange event should only be triggered in the case there is a complete dimension object.
// So when a new key is added that does not yet have a value, it should not trigger an onChange event.
const newDimensions = filterConditionsToDimensions(newItems);
if (!isEqual(newDimensions, query.dimensions)) {
onChange(newDimensions);
}
};
return (
<EditorList
items={items}
onChange={onDimensionsChange}
renderItem={makeRenderFilter(datasource, query, dimensionKeys, disableExpressions)}
/>
);
};
function makeRenderFilter(
datasource: CloudWatchDatasource,
query: CloudWatchMetricsQuery,
dimensionKeys: Array<SelectableValue<string>>,
disableExpressions: boolean
) {
function renderFilter(
item: DimensionFilterCondition,
onChange: (item: DimensionFilterCondition) => void,
onDelete: () => void
) {
return (
<FilterItem
filter={item}
onChange={(item) => onChange(item)}
datasource={datasource}
query={query}
disableExpressions={disableExpressions}
dimensionKeys={dimensionKeys}
onDelete={onDelete}
/>
);
}
return renderFilter;
}

110
public/app/plugins/datasource/cloudwatch/components/MetricStatEditor/FilterItem.tsx Normal file
View File

@ -0,0 +1,110 @@
import React, { FunctionComponent, useMemo } from 'react';
import { css, cx } from '@emotion/css';
import { Select, stylesFactory, useTheme2 } from '@grafana/ui';
import { useAsyncFn } from 'react-use';
import { GrafanaTheme2, SelectableValue, toOption } from '@grafana/data';
import { CloudWatchDatasource } from '../../datasource';
import { CloudWatchMetricsQuery, Dimensions } from '../../types';
import { appendTemplateVariables } from '../../utils/utils';
import { DimensionFilterCondition } from './Dimensions';
import InputGroup from '../ui/InputGroup';
import AccessoryButton from '../ui/AccessoryButton';
export interface Props {
query: CloudWatchMetricsQuery;
datasource: CloudWatchDatasource;
filter: DimensionFilterCondition;
dimensionKeys: Array<SelectableValue<string>>;
disableExpressions: boolean;
onChange: (value: DimensionFilterCondition) => void;
onDelete: () => void;
}
const wildcardOption = { value: '*', label: '*' };
const excludeCurrentKey = (dimensions: Dimensions, currentKey: string | undefined) =>
Object.entries(dimensions ?? {}).reduce<Dimensions>((acc, [key, value]) => {
if (key !== currentKey) {
return { ...acc, [key]: value };
}
return acc;
}, {});
export const FilterItem: FunctionComponent<Props> = ({
filter,
query: { region, namespace, metricName, dimensions },
datasource,
dimensionKeys,
disableExpressions,
onChange,
onDelete,
}) => {
const dimensionsExcludingCurrentKey = useMemo(() => excludeCurrentKey(dimensions ?? {}, filter.key), [
dimensions,
filter,
]);
const loadDimensionValues = async () => {
if (!filter.key) {
return [];
}
return datasource
.getDimensionValues(region, namespace, metricName, filter.key, dimensionsExcludingCurrentKey)
.then((result: Array<SelectableValue<string>>) => {
if (result.length && !disableExpressions) {
result.unshift(wildcardOption);
}
return appendTemplateVariables(datasource, result);
});
};
const [state, loadOptions] = useAsyncFn(loadDimensionValues, [filter.key, dimensions]);
const theme = useTheme2();
const styles = getOperatorStyles(theme);
return (
<div data-testid="cloudwatch-dimensions-filter-item">
<InputGroup>
<Select
inputId="cloudwatch-dimensions-filter-item-key"
width="auto"
value={filter.key ? toOption(filter.key) : null}
allowCustomValue
options={dimensionKeys}
onChange={(change) => {
if (change.label) {
onChange({ key: change.label, value: undefined });
}
}}
/>
<span className={cx(styles.root)}>=</span>
<Select
inputId="cloudwatch-dimensions-filter-item-value"
onOpenMenu={loadOptions}
width="auto"
value={filter.value ? toOption(filter.value) : null}
allowCustomValue
isLoading={state.loading}
options={state.value}
onChange={(change) => {
if (change.value) {
onChange({ ...filter, value: change.value });
}
}}
/>
<AccessoryButton aria-label="remove" icon="times" variant="secondary" onClick={onDelete} />
</InputGroup>
</div>
);
};
const getOperatorStyles = stylesFactory((theme: GrafanaTheme2) => ({
root: css({
padding: theme.spacing(0, 1),
alignSelf: 'center',
}),
}));

90
public/app/plugins/datasource/cloudwatch/components/MetricStatEditor/MetricStatEditor.test.tsx Normal file
View File

@ -0,0 +1,90 @@
import React from 'react';
import { cleanup, fireEvent, render, screen } from '@testing-library/react';
import { setupMockedDataSource } from '../../__mocks__/CloudWatchDataSource';
import '@testing-library/jest-dom';
import { CloudWatchMetricsQuery } from '../../types';
import userEvent from '@testing-library/user-event';
import { MetricStatEditor } from '..';
const ds = setupMockedDataSource({
variables: [],
});
ds.datasource.getNamespaces = jest.fn().mockResolvedValue([]);
ds.datasource.getMetrics = jest.fn().mockResolvedValue([]);
ds.datasource.getDimensionKeys = jest.fn().mockResolvedValue([]);
ds.datasource.getVariables = jest.fn().mockReturnValue([]);
const q: CloudWatchMetricsQuery = {
id: '',
region: 'us-east-2',
namespace: '',
period: '',
alias: '',
metricName: '',
dimensions: {},
matchExact: true,
statistic: '',
expression: '',
refId: '',
};
const props = {
datasource: ds.datasource,
query: q,
onChange: jest.fn(),
onRunQuery: jest.fn(),
};
afterEach(cleanup);
describe('MetricStatEditor', () => {
describe('statistics field', () => {
test.each([['Average', 'p23.23', 'p34', '$statistic']])('should accept valid values', (statistic) => {
const onChange = jest.fn();
const onRunQuery = jest.fn();
props.datasource.getVariables = jest.fn().mockReturnValue(['$statistic']);
render(<MetricStatEditor {...props} onChange={onChange} onRunQuery={onRunQuery} />);
const statisticElement = screen.getByLabelText('Statistic');
expect(statisticElement).toBeInTheDocument();
userEvent.type(statisticElement!, statistic);
fireEvent.keyDown(statisticElement!, { keyCode: 13 });
expect(onChange).toHaveBeenCalledWith({ ...props.query, statistic });
expect(onRunQuery).toHaveBeenCalled();
});
test.each([['CustomStat', 'p23,23', '$statistic']])('should not accept invalid values', (statistic) => {
const onChange = jest.fn();
const onRunQuery = jest.fn();
render(<MetricStatEditor {...props} onChange={onChange} onRunQuery={onRunQuery} />);
const statisticElement = screen.getByLabelText('Statistic');
expect(statisticElement).toBeInTheDocument();
userEvent.type(statisticElement!, statistic);
fireEvent.keyDown(statisticElement!, { keyCode: 13 });
expect(onChange).not.toHaveBeenCalled();
expect(onRunQuery).not.toHaveBeenCalled();
});
});
describe('expressions', () => {
it('should display match exact switch is not set', () => {
render(<MetricStatEditor {...props} />);
expect(screen.getByText('Match exact')).toBeInTheDocument();
});
it('should display match exact switch if prop is set to false', () => {
render(<MetricStatEditor {...props} disableExpressions={false} />);
expect(screen.getByText('Match exact')).toBeInTheDocument();
});
it('should not display match exact switch if prop is set to true', async () => {
render(<MetricStatEditor {...props} disableExpressions={true} />);
expect(screen.queryByText('Match exact')).toBeNull();
});
});
});

125
public/app/plugins/datasource/cloudwatch/components/MetricStatEditor/MetricStatEditor.tsx Normal file
View File

@ -0,0 +1,125 @@
import React from 'react';
import { Switch, Select } from '@grafana/ui';
import { CloudWatchMetricsQuery } from '../../types';
import { CloudWatchDatasource } from '../../datasource';
import EditorRows from '../ui/EditorRows';
import EditorRow from '../ui/EditorRow';
import EditorFieldGroup from '../ui/EditorFieldGroup';
import EditorField from '../ui/EditorField';
import { appendTemplateVariables, toOption } from '../../utils/utils';
import { useDimensionKeys, useMetrics, useNamespaces } from '../../hooks';
import { Dimensions } from '..';
export type Props = {
query: CloudWatchMetricsQuery;
datasource: CloudWatchDatasource;
disableExpressions?: boolean;
onChange: (value: CloudWatchMetricsQuery) => void;
onRunQuery: () => void;
};
export function MetricStatEditor({
query,
datasource,
disableExpressions = false,
onChange,
onRunQuery,
}: React.PropsWithChildren<Props>) {
const { region, namespace, metricName, dimensions } = query;
const namespaces = useNamespaces(datasource);
const metrics = useMetrics(datasource, region, namespace);
const dimensionKeys = useDimensionKeys(datasource, region, namespace, metricName, dimensions ?? {});
const onQueryChange = (query: CloudWatchMetricsQuery) => {
onChange(query);
onRunQuery();
};
return (
<EditorRows>
<EditorRow>
<EditorFieldGroup>
<EditorField label="Namespace" width={26}>
<Select
value={query.namespace}
allowCustomValue
options={namespaces}
onChange={({ value: namespace }) => {
if (namespace) {
onQueryChange({ ...query, namespace });
}
}}
/>
</EditorField>
<EditorField label="Metric name" width={16}>
<Select
value={query.metricName}
allowCustomValue
options={metrics}
onChange={({ value: metricName }) => {
if (metricName) {
onQueryChange({ ...query, metricName });
}
}}
/>
</EditorField>
<EditorField label="Statistic" width={16}>
<Select
inputId="metric-stat-editor-select-statistic"
allowCustomValue
value={toOption(query.statistic ?? datasource.standardStatistics[0])}
options={appendTemplateVariables(
datasource,
datasource.standardStatistics.filter((s) => s !== query.statistic).map(toOption)
)}
onChange={({ value: statistic }) => {
if (
!statistic ||
(!datasource.standardStatistics.includes(statistic) &&
!/^p\d{2}(?:\.\d{1,2})?$/.test(statistic) &&
!statistic.startsWith('$'))
) {
return;
}
onQueryChange({ ...query, statistic });
}}
/>
</EditorField>
</EditorFieldGroup>
</EditorRow>
<EditorRow>
<EditorField label="Dimensions">
<Dimensions
query={query}
onChange={(dimensions) => onQueryChange({ ...query, dimensions })}
dimensionKeys={dimensionKeys}
disableExpressions={disableExpressions}
datasource={datasource}
/>
</EditorField>
</EditorRow>
{!disableExpressions && (
<EditorRow>
<EditorField
label="Match exact"
optional={true}
tooltip="Only show metrics that exactly match all defined dimension names."
>
<Switch
checked={!!query.matchExact}
onChange={(e) => {
onQueryChange({
...query,
matchExact: e.currentTarget.checked,
});
}}
/>
</EditorField>
</EditorRow>
)}
</EditorRows>
);
}

1
public/app/plugins/datasource/cloudwatch/components/MetricStatEditor/index.ts Normal file
View File

@ -0,0 +1 @@
export { MetricStatEditor } from './MetricStatEditor';

110
public/app/plugins/datasource/cloudwatch/components/MetricsQueryEditor.test.tsx
View File

@ -1,13 +1,13 @@
import React from 'react';
import renderer from 'react-test-renderer';
import { mount } from 'enzyme';
import { render, screen, waitFor } from '@testing-library/react';
import { act } from 'react-dom/test-utils';
import { DataSourceInstanceSettings } from '@grafana/data';
import { TemplateSrv } from 'app/features/templating/template_srv';
import { MetricsQueryEditor, normalizeQuery, Props } from './MetricsQueryEditor';
import { CloudWatchDatasource } from '../datasource';
import { CustomVariableModel, initialVariableModelState } from '../../../../features/variables/types';
import { CloudWatchJsonData } from '../types';
import { CloudWatchJsonData, CloudWatchMetricsQuery, MetricEditorMode, MetricQueryType } from '../types';
const setup = () => {
const instanceSettings = {
@ -35,6 +35,10 @@ const setup = () => {
const datasource = new CloudWatchDatasource(instanceSettings, templateSrv as any, {} as any);
datasource.metricFindQuery = async () => [{ value: 'test', label: 'test', text: 'test' }];
datasource.getNamespaces = jest.fn().mockResolvedValue([]);
datasource.getMetrics = jest.fn().mockResolvedValue([]);
datasource.getRegions = jest.fn().mockResolvedValue([]);
datasource.getDimensionKeys = jest.fn().mockResolvedValue([]);
const props: Props = {
query: {
@ -50,6 +54,8 @@ const setup = () => {
expression: '',
alias: '',
matchExact: true,
metricQueryType: MetricQueryType.Search,
metricEditorMode: MetricEditorMode.Builder,
},
datasource,
history: [],
@ -80,6 +86,8 @@ describe('QueryEditor', () => {
refId: '',
expression: '',
matchExact: true,
metricQueryType: MetricQueryType.Search,
metricEditorMode: MetricEditorMode.Builder,
} as any;
await act(async () => {
renderer.create(<MetricsQueryEditor {...props} />);
@ -88,6 +96,7 @@ describe('QueryEditor', () => {
namespace: '',
metricName: '',
expression: '',
sqlExpression: '',
dimensions: {},
region: 'default',
id: '',
@ -98,27 +107,18 @@ describe('QueryEditor', () => {
apiMode: 'Metrics',
refId: '',
matchExact: true,
metricQueryType: MetricQueryType.Search,
metricEditorMode: MetricEditorMode.Builder,
});
});
describe('should use correct default values', () => {
it('when region is null is display default in the label', async () => {
// @ts-ignore strict null error TS2345: Argument of type '() => Promise<void>' is not assignable to parameter of type '() => void | undefined'.
await act(async () => {
const props = setup();
props.query.region = (null as unknown) as string;
const wrapper = mount(<MetricsQueryEditor {...props} />);
expect(
wrapper.find('.gf-form-inline').first().find('Segment').find('InlineLabel').find('label').text()
).toEqual('default');
});
});
it('should normalize query with default values', () => {
expect(normalizeQuery({ refId: '42' } as any)).toEqual({
namespace: '',
metricName: '',
expression: '',
sqlExpression: '',
dimensions: {},
region: 'default',
id: '',
@ -126,7 +126,89 @@ describe('QueryEditor', () => {
statistic: 'Average',
matchExact: true,
period: '',
queryMode: 'Metrics',
refId: '42',
metricQueryType: MetricQueryType.Search,
metricEditorMode: MetricEditorMode.Builder,
});
});
});
describe('should handle editor modes correctly', () => {
it('when metric query type is metric search and editor mode is builder', async () => {
await act(async () => {
const props = setup();
render(<MetricsQueryEditor {...props} />);
expect(screen.getByText('Metric Search')).toBeInTheDocument();
const radio = screen.getByLabelText('Builder');
expect(radio instanceof HTMLInputElement && radio.checked).toBeTruthy();
});
});
it('when metric query type is metric search and editor mode is raw', async () => {
await act(async () => {
const props = setup();
(props.query as CloudWatchMetricsQuery).metricEditorMode = MetricEditorMode.Code;
render(<MetricsQueryEditor {...props} />);
expect(screen.getByText('Metric Search')).toBeInTheDocument();
const radio = screen.getByLabelText('Code');
expect(radio instanceof HTMLInputElement && radio.checked).toBeTruthy();
});
});
it('when metric query type is metric query and editor mode is builder', async () => {
await act(async () => {
const props = setup();
(props.query as CloudWatchMetricsQuery).metricQueryType = MetricQueryType.Query;
(props.query as CloudWatchMetricsQuery).metricEditorMode = MetricEditorMode.Builder;
render(<MetricsQueryEditor {...props} />);
expect(screen.getByText('Metric Query')).toBeInTheDocument();
const radio = screen.getByLabelText('Builder');
expect(radio instanceof HTMLInputElement && radio.checked).toBeTruthy();
});
});
it('when metric query type is metric query and editor mode is raw', async () => {
await act(async () => {
const props = setup();
(props.query as CloudWatchMetricsQuery).metricQueryType = MetricQueryType.Query;
(props.query as CloudWatchMetricsQuery).metricEditorMode = MetricEditorMode.Code;
render(<MetricsQueryEditor {...props} />);
expect(screen.getByText('Metric Query')).toBeInTheDocument();
const radio = screen.getByLabelText('Code');
expect(radio instanceof HTMLInputElement && radio.checked).toBeTruthy();
});
});
});
describe('should handle expression options correctly', () => {
it('should display match exact switch', () => {
const props = setup();
render(<MetricsQueryEditor {...props} />);
expect(screen.getByText('Match exact')).toBeInTheDocument();
});
it('shoud display wildcard option in dimension value dropdown', async () => {
const props = setup();
props.datasource.getDimensionValues = jest.fn().mockResolvedValue([[{ label: 'dimVal1', value: 'dimVal1' }]]);
(props.query as CloudWatchMetricsQuery).metricQueryType = MetricQueryType.Search;
(props.query as CloudWatchMetricsQuery).metricEditorMode = MetricEditorMode.Builder;
(props.query as CloudWatchMetricsQuery).dimensions = { instanceId: 'instance-123' };
render(<MetricsQueryEditor {...props} />);
expect(screen.getByText('Match exact')).toBeInTheDocument();
const valueElement = screen.getByText('instance-123');
expect(valueElement).toBeInTheDocument();
expect(screen.queryByText('*')).toBeNull();
act(async () => {
await valueElement.click();
await waitFor(() => {
expect(screen.getByText('*')).toBeInTheDocument();
});
});
});
});

288
public/app/plugins/datasource/cloudwatch/components/MetricsQueryEditor.tsx
View File

@ -1,27 +1,29 @@
import React, { PureComponent, ChangeEvent } from 'react';
import { QueryEditorProps, PanelData } from '@grafana/data';
import { LegacyForms, ValidationEvents, EventsWithValidation, Icon } from '@grafana/ui';
const { Input, Switch } = LegacyForms;
import { CloudWatchQuery, CloudWatchMetricsQuery, CloudWatchJsonData, ExecutedQueryPreview } from '../types';
import { QueryEditorProps } from '@grafana/data';
import { Input } from '@grafana/ui';
import {
CloudWatchQuery,
CloudWatchMetricsQuery,
CloudWatchJsonData,
MetricQueryType,
MetricEditorMode,
} from '../types';
import { CloudWatchDatasource } from '../datasource';
import { QueryField, Alias, MetricsQueryFieldsEditor } from './';
import { Alias, MetricStatEditor, MathExpressionQueryField, SQLBuilderEditor, SQLCodeEditor } from './';
import EditorRow from './ui/EditorRow';
import EditorField from './ui/EditorField';
import { Space } from './ui/Space';
import QueryHeader from './QueryHeader';
import { isMetricsQuery } from '../guards';
export type Props = QueryEditorProps<CloudWatchDatasource, CloudWatchQuery, CloudWatchJsonData>;
interface State {
showMeta: boolean;
sqlCodeEditorIsDirty: boolean;
}
const idValidationEvents: ValidationEvents = {
[EventsWithValidation.onBlur]: [
{
rule: (value) => new RegExp(/^$|^[a-z][a-zA-Z0-9_]*$/).test(value),
errorMessage: 'Invalid format. Only alphanumeric characters and underscores are allowed',
},
],
};
export const normalizeQuery = ({
namespace,
metricName,
@ -32,169 +34,153 @@ export const normalizeQuery = ({
alias,
statistic,
period,
sqlExpression,
metricQueryType,
metricEditorMode,
...rest
}: CloudWatchMetricsQuery): CloudWatchMetricsQuery => {
const normalizedQuery = {
namespace: namespace || '',
metricName: metricName || '',
expression: expression || '',
dimensions: dimensions || {},
region: region || 'default',
id: id || '',
alias: alias || '',
queryMode: 'Metrics' as const,
namespace: namespace ?? '',
metricName: metricName ?? '',
expression: expression ?? '',
dimensions: dimensions ?? {},
region: region ?? 'default',
id: id ?? '',
alias: alias ?? '',
statistic: statistic ?? 'Average',
period: period || '',
period: period ?? '',
metricQueryType: metricQueryType ?? MetricQueryType.Search,
metricEditorMode: metricEditorMode ?? MetricEditorMode.Builder,
sqlExpression: sqlExpression ?? '',
...rest,
};
return !rest.hasOwnProperty('matchExact') ? { ...normalizedQuery, matchExact: true } : normalizedQuery;
};
export class MetricsQueryEditor extends PureComponent<Props, State> {
state: State = { showMeta: false };
state = {
sqlCodeEditorIsDirty: false,
};
componentDidMount(): void {
componentDidMount = () => {
const metricsQuery = this.props.query as CloudWatchMetricsQuery;
const query = normalizeQuery(metricsQuery);
this.props.onChange(query);
}
};
onChange(query: CloudWatchMetricsQuery) {
onChange = (query: CloudWatchQuery) => {
const { onChange, onRunQuery } = this.props;
onChange(query);
onRunQuery();
}
getExecutedQueryPreview(data?: PanelData): ExecutedQueryPreview {
if (!(data?.series.length && data?.series[0].meta?.custom)) {
return {
executedQuery: '',
period: '',
id: '',
};
}
return {
executedQuery: data?.series[0].meta.executedQueryString ?? '',
period: data.series[0].meta.custom['period'],
id: data.series[0].meta.custom['id'],
};
}
};
render() {
const { data, onRunQuery } = this.props;
const { onRunQuery, datasource } = this.props;
const metricsQuery = this.props.query as CloudWatchMetricsQuery;
const { showMeta } = this.state;
const query = normalizeQuery(metricsQuery);
const executedQueryPreview = this.getExecutedQueryPreview(data);
return (
<>
<MetricsQueryFieldsEditor {...{ ...this.props, query }}></MetricsQueryFieldsEditor>
<div className="gf-form-inline">
<div className="gf-form">
<QueryField
label="Id"
tooltip="Id can include numbers, letters, and underscore, and must start with a lowercase letter."
>
<Input
className="gf-form-input width-8"
onBlur={onRunQuery}
onChange={(event: ChangeEvent<HTMLInputElement>) =>
this.onChange({ ...metricsQuery, id: event.target.value })
}
validationEvents={idValidationEvents}
value={query.id}
<QueryHeader
query={query}
onRunQuery={onRunQuery}
datasource={datasource}
onChange={(newQuery) => {
if (isMetricsQuery(newQuery) && newQuery.metricEditorMode !== query.metricEditorMode) {
this.setState({ sqlCodeEditorIsDirty: false });
}
this.onChange(newQuery);
}}
sqlCodeEditorIsDirty={this.state.sqlCodeEditorIsDirty}
/>
<Space v={0.5} />
{query.metricQueryType === MetricQueryType.Search && (
<>
{query.metricEditorMode === MetricEditorMode.Builder && (
<MetricStatEditor {...{ ...this.props, query }}></MetricStatEditor>
)}
{query.metricEditorMode === MetricEditorMode.Code && (
<MathExpressionQueryField
onRunQuery={onRunQuery}
expression={query.expression ?? ''}
onChange={(expression) => this.props.onChange({ ...query, expression })}
></MathExpressionQueryField>
)}
</>
)}
{query.metricQueryType === MetricQueryType.Query && (
<>
{query.metricEditorMode === MetricEditorMode.Code && (
<SQLCodeEditor
region={query.region}
sql={query.sqlExpression ?? ''}
onChange={(sqlExpression) => {
if (!this.state.sqlCodeEditorIsDirty) {
this.setState({ sqlCodeEditorIsDirty: true });
}
this.props.onChange({ ...metricsQuery, sqlExpression });
}}
onRunQuery={onRunQuery}
datasource={datasource}
/>
</QueryField>
</div>
<div className="gf-form gf-form--grow">
<QueryField
className="gf-form--grow"
label="Expression"
tooltip="Optionally you can add an expression here. Please note that if a math expression that is referencing other queries is being used, it will not be possible to create an alert rule based on this query"
>
<Input
className="gf-form-input"
onBlur={onRunQuery}
value={query.expression || ''}
onChange={(event: ChangeEvent<HTMLInputElement>) =>
this.onChange({ ...metricsQuery, expression: event.target.value })
}
/>
</QueryField>
</div>
</div>
<div className="gf-form-inline">
<div className="gf-form">
<QueryField label="Period" tooltip="Minimum interval between points in seconds">
<Input
className="gf-form-input width-8"
value={query.period || ''}
placeholder="auto"
onBlur={onRunQuery}
onChange={(event: ChangeEvent<HTMLInputElement>) =>
this.onChange({ ...metricsQuery, period: event.target.value })
}
/>
</QueryField>
</div>
<div className="gf-form">
<QueryField
label="Alias"
tooltip="Alias replacement variables: {{metric}}, {{stat}}, {{namespace}}, {{region}}, {{period}}, {{label}}, {{YOUR_DIMENSION_NAME}}"
>
<Alias
value={metricsQuery.alias}
onChange={(value: string) => this.onChange({ ...metricsQuery, alias: value })}
/>
</QueryField>
<Switch
label="Match Exact"
labelClass="query-keyword"
tooltip="Only show metrics that exactly match all defined dimension names."
checked={metricsQuery.matchExact}
onChange={() =>
this.onChange({
...metricsQuery,
matchExact: !metricsQuery.matchExact,
})
)}
{query.metricEditorMode === MetricEditorMode.Builder && (
<>
<SQLBuilderEditor
query={query}
onChange={this.props.onChange}
onRunQuery={onRunQuery}
datasource={datasource}
></SQLBuilderEditor>
</>
)}
</>
)}
<Space v={0.5} />
<EditorRow>
<EditorField
label="ID"
width={26}
optional
tooltip="ID can be used to reference other queries in math expressions. The ID can include numbers, letters, and underscore, and must start with a lowercase letter."
>
<Input
onBlur={onRunQuery}
onChange={(event: ChangeEvent<HTMLInputElement>) =>
this.onChange({ ...metricsQuery, id: event.target.value })
}
type="text"
invalid={!!query.id && !/^$|^[a-z][a-zA-Z0-9_]*$/.test(query.id)}
value={query.id}
/>
</EditorField>
<EditorField label="Period" width={26} tooltip="Minimum interval between points in seconds.">
<Input
value={query.period || ''}
placeholder="auto"
onBlur={onRunQuery}
onChange={(event: ChangeEvent<HTMLInputElement>) =>
this.onChange({ ...metricsQuery, period: event.target.value })
}
/>
<label className="gf-form-label">
<a
onClick={() =>
executedQueryPreview &&
this.setState({
showMeta: !showMeta,
})
}
>
<Icon name={showMeta ? 'angle-down' : 'angle-right'} /> {showMeta ? 'Hide' : 'Show'} Query Preview
</a>
</label>
</div>
<div className="gf-form gf-form--grow">
<div className="gf-form-label gf-form-label--grow" />
</div>
{showMeta && (
<table className="filter-table form-inline">
<thead>
<tr>
<th>Metric Data Query ID</th>
<th>Metric Data Query Expression</th>
<th>Period</th>
<th />
</tr>
</thead>
<tbody>
<tr>
<td>{executedQueryPreview.id}</td>
<td>{executedQueryPreview.executedQuery}</td>
<td>{executedQueryPreview.period}</td>
</tr>
</tbody>
</table>
)}
</div>
</EditorField>
<EditorField
label="Alias"
width={26}
optional
tooltip="Change time series legend name using this field. See documentation for replacement variable formats."
>
<Alias
value={metricsQuery.alias ?? ''}
onChange={(value: string) => this.onChange({ ...metricsQuery, alias: value })}
/>
</EditorField>
</EditorRow>
</>
);
}

157
public/app/plugins/datasource/cloudwatch/components/MetricsQueryFieldsEditor.tsx
View File

@ -1,157 +0,0 @@
import React, { useEffect, useState } from 'react';
import { SelectableValue } from '@grafana/data';
import { Segment, SegmentAsync } from '@grafana/ui';
import { CloudWatchMetricsQuery, SelectableStrings } from '../types';
import { CloudWatchDatasource } from '../datasource';
import { Dimensions, QueryInlineField } from '.';
export type Props = {
query: CloudWatchMetricsQuery;
datasource: CloudWatchDatasource;
onRunQuery?: () => void;
onChange: (value: CloudWatchMetricsQuery) => void;
};
interface State {
regions: SelectableStrings;
namespaces: SelectableStrings;
metricNames: SelectableStrings;
variableOptionGroup: SelectableValue<string>;
showMeta: boolean;
}
export function MetricsQueryFieldsEditor({
query,
datasource,
onChange,
onRunQuery = () => {},
}: React.PropsWithChildren<Props>) {
const metricsQuery = query as CloudWatchMetricsQuery;
const [state, setState] = useState<State>({
regions: [],
namespaces: [],
metricNames: [],
variableOptionGroup: {},
showMeta: false,
});
useEffect(() => {
const variableOptionGroup = {
label: 'Template Variables',
options: datasource.getVariables().map(toOption),
};
Promise.all([datasource.metricFindQuery('regions()'), datasource.metricFindQuery('namespaces()')]).then(
([regions, namespaces]) => {
setState((prevState) => ({
...prevState,
regions: [...regions, variableOptionGroup],
namespaces: [...namespaces, variableOptionGroup],
variableOptionGroup,
}));
}
);
}, [datasource]);
const loadMetricNames = async () => {
const { namespace, region } = query;
return datasource.metricFindQuery(`metrics(${namespace},${region})`).then(appendTemplateVariables);
};
const appendTemplateVariables = (values: SelectableValue[]) => [
...values,
{ label: 'Template Variables', options: datasource.getVariables().map(toOption) },
];
const toOption = (value: any) => ({ label: value, value });
const onQueryChange = (query: CloudWatchMetricsQuery) => {
onChange(query);
onRunQuery();
};
// Load dimension values based on current selected dimensions.
// Remove the new dimension key and all dimensions that has a wildcard as selected value
const loadDimensionValues = (newKey: string) => {
const { [newKey]: value, ...dim } = metricsQuery.dimensions;
const newDimensions = Object.entries(dim).reduce(
(result, [key, value]) => (value === '*' ? result : { ...result, [key]: value }),
{}
);
return datasource
.getDimensionValues(query.region, query.namespace, metricsQuery.metricName, newKey, newDimensions)
.then((values) => (values.length ? [{ value: '*', text: '*', label: '*' }, ...values] : values))
.then(appendTemplateVariables);
};
const { regions, namespaces, variableOptionGroup } = state;
return (
<>
<QueryInlineField label="Region">
<Segment
value={query.region}
placeholder="Select region"
options={regions}
allowCustomValue
onChange={({ value: region }) => onQueryChange({ ...query, region: region! })}
/>
</QueryInlineField>
{query.expression?.length === 0 && (
<>
<QueryInlineField label="Namespace">
<Segment
value={query.namespace}
placeholder="Select namespace"
allowCustomValue
options={namespaces}
onChange={({ value: namespace }) => onQueryChange({ ...query, namespace: namespace! })}
/>
</QueryInlineField>
<QueryInlineField label="Metric Name">
<SegmentAsync
value={metricsQuery.metricName}
placeholder="Select metric name"
allowCustomValue
loadOptions={loadMetricNames}
onChange={({ value: metricName }) => onQueryChange({ ...metricsQuery, metricName })}
/>
</QueryInlineField>
<QueryInlineField label="Statistic">
<Segment
allowCustomValue
value={query.statistic}
options={[
...datasource.standardStatistics.filter((s) => s !== query.statistic).map(toOption),
variableOptionGroup,
]}
onChange={({ value: statistic }) => {
if (
!datasource.standardStatistics.includes(statistic) &&
!/^p\d{2}(?:\.\d{1,2})?$/.test(statistic) &&
!statistic.startsWith('$')
) {
return;
}
onQueryChange({ ...metricsQuery, statistic });
}}
/>
</QueryInlineField>
<QueryInlineField label="Dimensions">
<Dimensions
dimensions={metricsQuery.dimensions}
onChange={(dimensions) => onQueryChange({ ...metricsQuery, dimensions })}
loadKeys={() => datasource.getDimensionKeys(query.namespace, query.region).then(appendTemplateVariables)}
loadValues={loadDimensionValues}
/>
</QueryInlineField>
</>
)}
</>
);
}

55
public/app/plugins/datasource/cloudwatch/components/PanelQueryEditor.tsx
View File

@ -22,33 +22,36 @@ export class PanelQueryEditor extends PureComponent<Props> {
return (
<>
<QueryInlineField label="Query Mode">
<Segment
value={apiModes[apiMode]}
options={Object.values(apiModes)}
onChange={({ value }) => {
const newMode = (value as 'Metrics' | 'Logs') ?? 'Metrics';
if (newMode !== apiModes[apiMode].value) {
const commonProps = pick(
query,
'id',
'region',
'namespace',
'refId',
'hide',
'key',
'queryType',
'datasource'
);
{/* TODO: Remove this in favor of the QueryHeader */}
{apiMode === ExploreMode.Logs && (
<QueryInlineField label="Query Mode">
<Segment
value={apiModes[apiMode]}
options={Object.values(apiModes)}
onChange={({ value }) => {
const newMode = (value as 'Metrics' | 'Logs') ?? 'Metrics';
if (newMode !== apiModes[apiMode].value) {
const commonProps = pick(
query,
'id',
'region',
'namespace',
'refId',
'hide',
'key',
'queryType',
'datasource'
);
this.props.onChange({
...commonProps,
queryMode: newMode,
} as CloudWatchQuery);
}
}}
/>
</QueryInlineField>
this.props.onChange({
...commonProps,
queryMode: newMode,
} as CloudWatchQuery);
}
}}
/>
</QueryInlineField>
)}
{apiMode === ExploreMode.Logs ? (
<LogsQueryEditor {...this.props} allowCustomValue />
) : (

133
public/app/plugins/datasource/cloudwatch/components/QueryHeader.test.tsx Normal file
View File

@ -0,0 +1,133 @@
import React from 'react';
import { render, screen } from '@testing-library/react';
import { act } from 'react-dom/test-utils';
import { CloudWatchMetricsQuery, MetricEditorMode, MetricQueryType } from '../types';
import { setupMockedDataSource } from '../__mocks__/CloudWatchDataSource';
import QueryHeader from './QueryHeader';
const ds = setupMockedDataSource({
variables: [],
});
ds.datasource.getRegions = jest.fn().mockResolvedValue([]);
const query: CloudWatchMetricsQuery = {
id: '',
region: 'us-east-2',
namespace: '',
period: '',
alias: '',
metricName: '',
dimensions: {},
matchExact: true,
statistic: '',
expression: '',
refId: '',
};
describe('QueryHeader', () => {
describe('confirm modal', () => {
it('should be shown when moving from code editor to builder when in sql mode', async () => {
const onChange = jest.fn();
const onRunQuery = jest.fn();
query.metricEditorMode = MetricEditorMode.Code;
query.metricQueryType = MetricQueryType.Query;
render(
<QueryHeader
sqlCodeEditorIsDirty={true}
datasource={ds.datasource}
query={query}
onChange={onChange}
onRunQuery={onRunQuery}
/>
);
const builderElement = screen.getByLabelText('Builder');
expect(builderElement).toBeInTheDocument();
await act(async () => {
await builderElement.click();
});
const modalTitleElem = screen.getByText('Are you sure?');
expect(modalTitleElem).toBeInTheDocument();
expect(onChange).not.toHaveBeenCalled();
});
it('should not be shown when moving from builder to code when in sql mode', async () => {
const onChange = jest.fn();
const onRunQuery = jest.fn();
query.metricEditorMode = MetricEditorMode.Builder;
query.metricQueryType = MetricQueryType.Query;
render(
<QueryHeader
sqlCodeEditorIsDirty={true}
datasource={ds.datasource}
query={query}
onChange={onChange}
onRunQuery={onRunQuery}
/>
);
const builderElement = screen.getByLabelText('Code');
expect(builderElement).toBeInTheDocument();
await act(async () => {
await builderElement.click();
});
const modalTitleElem = screen.queryByText('Are you sure?');
expect(modalTitleElem).toBeNull();
expect(onChange).toHaveBeenCalled();
});
it('should not be shown when moving from code to builder when in standard mode', async () => {
const onChange = jest.fn();
const onRunQuery = jest.fn();
query.metricEditorMode = MetricEditorMode.Code;
query.metricQueryType = MetricQueryType.Search;
render(
<QueryHeader
sqlCodeEditorIsDirty={true}
datasource={ds.datasource}
query={query}
onChange={onChange}
onRunQuery={onRunQuery}
/>
);
const builderElement = screen.getByLabelText('Builder');
expect(builderElement).toBeInTheDocument();
await act(async () => {
await builderElement.click();
});
const modalTitleElem = screen.queryByText('Are you sure?');
expect(modalTitleElem).toBeNull();
expect(onChange).toHaveBeenCalled();
});
});
it('run button should be displayed in code editor in metric query mode', async () => {
const onChange = jest.fn();
const onRunQuery = jest.fn();
query.metricEditorMode = MetricEditorMode.Code;
query.metricQueryType = MetricQueryType.Query;
render(
<QueryHeader
sqlCodeEditorIsDirty={true}
datasource={ds.datasource}
query={query}
onChange={onChange}
onRunQuery={onRunQuery}
/>
);
const runQueryButton = screen.getByText('Run query');
expect(runQueryButton).toBeInTheDocument();
await act(async () => {
await runQueryButton.click();
});
expect(onRunQuery).toHaveBeenCalled();
});
});

124
public/app/plugins/datasource/cloudwatch/components/QueryHeader.tsx Normal file
View File

@ -0,0 +1,124 @@
import React, { useCallback, useState } from 'react';
import { pick } from 'lodash';
import { SelectableValue } from '@grafana/data';
import { Button, ConfirmModal, RadioButtonGroup } from '@grafana/ui';
import { CloudWatchDatasource } from '../datasource';
import {
CloudWatchMetricsQuery,
CloudWatchQuery,
CloudWatchQueryMode,
MetricEditorMode,
MetricQueryType,
} from '../types';
import EditorHeader from './ui/EditorHeader';
import InlineSelect from './ui/InlineSelect';
import FlexItem from './ui/FlexItem';
import { useRegions } from '../hooks';
interface QueryHeaderProps {
query: CloudWatchMetricsQuery;
datasource: CloudWatchDatasource;
onChange: (query: CloudWatchQuery) => void;
onRunQuery: () => void;
sqlCodeEditorIsDirty: boolean;
}
const apiModes: Array<SelectableValue<CloudWatchQueryMode>> = [
{ label: 'CloudWatch Metrics', value: 'Metrics' },
{ label: 'CloudWatch Logs', value: 'Logs' },
];
const metricEditorModes: Array<SelectableValue<MetricQueryType>> = [
{ label: 'Metric Search', value: MetricQueryType.Search },
{ label: 'Metric Query', value: MetricQueryType.Query },
];
const editorModes = [
{ label: 'Builder', value: MetricEditorMode.Builder },
{ label: 'Code', value: MetricEditorMode.Code },
];
const QueryHeader: React.FC<QueryHeaderProps> = ({ query, sqlCodeEditorIsDirty, datasource, onChange, onRunQuery }) => {
const { metricEditorMode, metricQueryType, queryMode, region } = query;
const [showConfirm, setShowConfirm] = useState(false);
const [regions, regionIsLoading] = useRegions(datasource);
const onEditorModeChange = useCallback(
(newMetricEditorMode: MetricEditorMode) => {
if (
sqlCodeEditorIsDirty &&
metricQueryType === MetricQueryType.Query &&
metricEditorMode === MetricEditorMode.Code
) {
setShowConfirm(true);
return;
}
onChange({ ...query, metricEditorMode: newMetricEditorMode });
},
[setShowConfirm, onChange, sqlCodeEditorIsDirty, query, metricEditorMode, metricQueryType]
);
const onQueryModeChange = ({ value }: SelectableValue<CloudWatchQueryMode>) => {
if (value !== queryMode) {
const commonProps = pick(query, 'id', 'region', 'namespace', 'refId', 'hide', 'key', 'queryType', 'datasource');
onChange({
...commonProps,
queryMode: value,
});
}
};
return (
<EditorHeader>
<InlineSelect
label="Region"
value={regions.find((v) => v.value === region)}
placeholder="Select region"
allowCustomValue
onChange={({ value: region }) => region && onChange({ ...query, region: region })}
options={regions}
isLoading={regionIsLoading}
/>
<InlineSelect value={queryMode} options={apiModes} onChange={onQueryModeChange} />
<InlineSelect
value={metricEditorModes.find((m) => m.value === metricQueryType)}
options={metricEditorModes}
onChange={({ value }) => {
onChange({ ...query, metricQueryType: value });
}}
/>
<FlexItem grow={1} />
<RadioButtonGroup options={editorModes} size="sm" value={metricEditorMode} onChange={onEditorModeChange} />
{query.metricQueryType === MetricQueryType.Query && query.metricEditorMode === MetricEditorMode.Code && (
<Button variant="secondary" size="sm" onClick={() => onRunQuery()}>
Run query
</Button>
)}
<ConfirmModal
isOpen={showConfirm}
title="Are you sure?"
body="You will lose manual changes done to the query if you go back to the visual builder."
confirmText="Yes, I am sure."
dismissText="No, continue editing the query manually."
icon="exclamation-triangle"
onConfirm={() => {
setShowConfirm(false);
onChange({ ...query, metricEditorMode: MetricEditorMode.Builder });
}}
onDismiss={() => setShowConfirm(false)}
/>
</EditorHeader>
);
};
export default QueryHeader;

163
public/app/plugins/datasource/cloudwatch/components/SQLBuilderEditor/SQLBuilderEditor.test.tsx Normal file
View File

@ -0,0 +1,163 @@
import React from 'react';
import { SQLBuilderEditor } from '..';
import { act, render, screen, waitFor } from '@testing-library/react';
import { CloudWatchMetricsQuery, MetricEditorMode, MetricQueryType, SQLExpression } from '../../types';
import { setupMockedDataSource } from '../../__mocks__/CloudWatchDataSource';
import { QueryEditorExpressionType, QueryEditorPropertyType } from '../../expressions';
const { datasource } = setupMockedDataSource();
const makeSQLQuery = (sql?: SQLExpression): CloudWatchMetricsQuery => ({
queryMode: 'Metrics',
refId: '',
id: '',
region: 'us-east-1',
namespace: 'ec2',
dimensions: { somekey: 'somevalue' },
metricQueryType: MetricQueryType.Query,
metricEditorMode: MetricEditorMode.Builder,
sql: sql,
});
describe('Cloudwatch SQLBuilderEditor', () => {
beforeEach(() => {
datasource.getNamespaces = jest.fn().mockResolvedValue([]);
datasource.getMetrics = jest.fn().mockResolvedValue([]);
datasource.getDimensionKeys = jest.fn().mockResolvedValue([]);
datasource.getDimensionValues = jest.fn().mockResolvedValue([]);
});
const baseProps = {
query: makeSQLQuery(),
datasource,
onChange: () => {},
onRunQuery: () => {},
};
it('Displays the namespace', async () => {
const query = makeSQLQuery({
from: {
type: QueryEditorExpressionType.Property,
property: {
type: QueryEditorPropertyType.String,
name: 'AWS/EC2',
},
},
});
render(<SQLBuilderEditor {...baseProps} query={query} />);
await waitFor(() => expect(datasource.getNamespaces).toHaveBeenCalled());
expect(screen.getByText('AWS/EC2')).toBeInTheDocument();
expect(screen.getByLabelText('With schema')).not.toBeChecked();
});
it('Displays withSchema namespace', async () => {
const query = makeSQLQuery({
from: {
type: QueryEditorExpressionType.Function,
name: 'SCHEMA',
parameters: [
{
type: QueryEditorExpressionType.FunctionParameter,
name: 'AWS/EC2',
},
],
},
});
render(<SQLBuilderEditor {...baseProps} query={query} />);
await waitFor(() => expect(datasource.getNamespaces).toHaveBeenCalled());
expect(screen.getByText('AWS/EC2')).toBeInTheDocument();
expect(screen.getByLabelText('With schema')).toBeChecked();
expect(screen.getByText('Schema labels')).toBeInTheDocument();
});
it('Uses dimension filter when loading dimension keys', async () => {
const query = makeSQLQuery({
from: {
type: QueryEditorExpressionType.Function,
name: 'SCHEMA',
parameters: [
{
type: QueryEditorExpressionType.FunctionParameter,
name: 'AWS/EC2',
},
{
type: QueryEditorExpressionType.FunctionParameter,
name: 'InstanceId',
},
],
},
});
render(<SQLBuilderEditor {...baseProps} query={query} />);
act(async () => {
expect(screen.getByText('AWS/EC2')).toBeInTheDocument();
expect(screen.getByLabelText('With schema')).toBeChecked();
expect(screen.getByText('Schema labels')).toBeInTheDocument();
await waitFor(() =>
expect(datasource.getDimensionKeys).toHaveBeenCalledWith(
query.namespace,
query.region,
{ InstanceId: null },
undefined
)
);
});
});
it('Displays the SELECT correctly', async () => {
const query = makeSQLQuery({
select: {
type: QueryEditorExpressionType.Function,
name: 'AVERAGE',
parameters: [
{
type: QueryEditorExpressionType.FunctionParameter,
name: 'CPUUtilization',
},
],
},
});
render(<SQLBuilderEditor {...baseProps} query={query} />);
await waitFor(() => expect(datasource.getNamespaces).toHaveBeenCalled());
expect(screen.getByText('AVERAGE')).toBeInTheDocument();
expect(screen.getByText('CPUUtilization')).toBeInTheDocument();
});
describe('ORDER BY', async () => {
it('should display it correctly when its specified', async () => {
const query = makeSQLQuery({
orderBy: {
type: QueryEditorExpressionType.Function,
name: 'AVG',
},
});
render(<SQLBuilderEditor {...baseProps} query={query} />);
await waitFor(() => expect(datasource.getNamespaces).toHaveBeenCalled());
expect(screen.getByText('AVG')).toBeInTheDocument();
const directionElement = screen.getByLabelText('Direction');
expect(directionElement).toBeInTheDocument();
expect(directionElement).not.toBeDisabled();
});
it('should display it correctly when its not specified', async () => {
const query = makeSQLQuery({});
render(<SQLBuilderEditor {...baseProps} query={query} />);
await waitFor(() => expect(datasource.getNamespaces).toHaveBeenCalled());
expect(screen.queryByText('AVG')).toBeNull();
const directionElement = screen.getByLabelText('Direction');
expect(directionElement).toBeInTheDocument();
expect(directionElement).toBeDisabled();
});
});
});

89
public/app/plugins/datasource/cloudwatch/components/SQLBuilderEditor/SQLBuilderEditor.tsx Normal file
View File

@ -0,0 +1,89 @@
import React, { useCallback, useEffect, useState } from 'react';
import { CloudWatchMetricsQuery } from '../../types';
import { CloudWatchDatasource } from '../../datasource';
import EditorRow from '../ui/EditorRow';
import EditorRows from '../ui/EditorRows';
import EditorField from '../ui/EditorField';
import SQLFilter from './SQLFilter';
import SQLGroupBy from './SQLGroupBy';
import SQLBuilderSelectRow from './SQLBuilderSelectRow';
import SQLGenerator from '../../cloudwatch-sql/SQLGenerator';
import SQLOrderByGroup from './SQLOrderByGroup';
import { Input } from '@grafana/ui';
import { setSql } from './utils';
export type Props = {
query: CloudWatchMetricsQuery;
datasource: CloudWatchDatasource;
onChange: (value: CloudWatchMetricsQuery) => void;
onRunQuery: () => void;
};
export function SQLBuilderEditor({ query, datasource, onChange, onRunQuery }: React.PropsWithChildren<Props>) {
const sql = query.sql ?? {};
const onQueryChange = useCallback(
(query: CloudWatchMetricsQuery) => {
const sqlGenerator = new SQLGenerator();
const sqlString = sqlGenerator.expressionToSqlQuery(query.sql ?? {});
const fullQuery = {
...query,
sqlExpression: sqlString,
};
onChange(fullQuery);
onRunQuery();
},
[onChange, onRunQuery]
);
const [sqlPreview, setSQLPreview] = useState<string | undefined>();
useEffect(() => {
const sqlGenerator = new SQLGenerator();
const sqlString = sqlGenerator.expressionToSqlQuery(query.sql ?? {});
if (sqlPreview !== sqlString) {
setSQLPreview(sqlString);
}
}, [query, sqlPreview, setSQLPreview]);
return (
<EditorRows>
<EditorRow>
<SQLBuilderSelectRow query={query} onQueryChange={onQueryChange} datasource={datasource} />
</EditorRow>
<EditorRow>
<EditorField label="Filter" optional={true}>
<SQLFilter query={query} onQueryChange={onQueryChange} datasource={datasource} />
</EditorField>
</EditorRow>
<EditorRow>
<EditorField label="Group by" optional>
<SQLGroupBy query={query} onQueryChange={onQueryChange} datasource={datasource} />
</EditorField>
<SQLOrderByGroup query={query} onQueryChange={onQueryChange} datasource={datasource}></SQLOrderByGroup>
<EditorField label="Limit" optional>
<Input
value={sql.limit}
onChange={(e) => {
const val = e.currentTarget.valueAsNumber;
onQueryChange(setSql(query, { limit: isNaN(val) ? undefined : val }));
}}
type="number"
min={1}
/>
</EditorField>
</EditorRow>
{sqlPreview && (
<EditorRow>
{process.env.NODE_ENV === 'development' && <pre>{JSON.stringify(query.sql ?? {}, null, 2)}</pre>}
<pre>{sqlPreview ?? ''}</pre>
</EditorRow>
)}
</EditorRows>
);
}

121
public/app/plugins/datasource/cloudwatch/components/SQLBuilderEditor/SQLBuilderSelectRow.tsx Normal file
View File

@ -0,0 +1,121 @@
import { toOption } from '@grafana/data';
import { Select, Switch } from '@grafana/ui';
import React, { useEffect, useMemo } from 'react';
import { STATISTICS } from '../../cloudwatch-sql/language';
import { CloudWatchDatasource } from '../../datasource';
import { useDimensionKeys, useMetrics, useNamespaces } from '../../hooks';
import { CloudWatchMetricsQuery } from '../../types';
import { appendTemplateVariables } from '../../utils/utils';
import EditorField from '../ui/EditorField';
import EditorFieldGroup from '../ui/EditorFieldGroup';
import {
stringArrayToDimensions,
getMetricNameFromExpression,
getNamespaceFromExpression,
getSchemaLabelKeys as getSchemaLabels,
isUsingWithSchema,
setAggregation,
setMetricName,
setNamespace,
setSchemaLabels,
setWithSchema,
} from './utils';
interface SQLBuilderSelectRowProps {
query: CloudWatchMetricsQuery;
datasource: CloudWatchDatasource;
onQueryChange: (query: CloudWatchMetricsQuery) => void;
}
const AGGREGATIONS = STATISTICS.map(toOption);
const SQLBuilderSelectRow: React.FC<SQLBuilderSelectRowProps> = ({ datasource, query, onQueryChange }) => {
const sql = query.sql ?? {};
const aggregation = sql.select?.name;
useEffect(() => {
if (!aggregation) {
onQueryChange(setAggregation(query, STATISTICS[0]));
}
}, [aggregation, onQueryChange, query]);
const metricName = getMetricNameFromExpression(sql.select);
const namespace = getNamespaceFromExpression(sql.from);
const schemaLabels = getSchemaLabels(sql.from);
const withSchemaEnabled = isUsingWithSchema(sql.from);
const namespaceOptions = useNamespaces(datasource);
const metricOptions = useMetrics(datasource, query.region, namespace);
const existingFilters = useMemo(() => stringArrayToDimensions(schemaLabels ?? []), [schemaLabels]);
const unusedDimensionKeys = useDimensionKeys(datasource, query.region, namespace, metricName, existingFilters);
const dimensionKeys = useMemo(
() => (schemaLabels?.length ? [...unusedDimensionKeys, ...schemaLabels.map(toOption)] : unusedDimensionKeys),
[unusedDimensionKeys, schemaLabels]
);
return (
<>
<EditorFieldGroup>
<EditorField label="Namespace" width={16}>
<Select
value={namespace ? toOption(namespace) : null}
inputId="cloudwatch-sql-namespace"
options={namespaceOptions}
allowCustomValue
onChange={({ value }) => value && onQueryChange(setNamespace(query, value))}
menuShouldPortal
/>
</EditorField>
<EditorField label="With schema">
<Switch
id="cloudwatch-sql-withSchema"
value={withSchemaEnabled}
onChange={(ev) =>
ev.target instanceof HTMLInputElement && onQueryChange(setWithSchema(query, ev.target.checked))
}
/>
</EditorField>
{withSchemaEnabled && (
<EditorField label="Schema labels">
<Select
id="cloudwatch-sql-schema-label-keys"
width="auto"
isMulti={true}
disabled={!namespace}
value={schemaLabels ? schemaLabels.map(toOption) : null}
options={dimensionKeys}
allowCustomValue
onChange={(item) => item && onQueryChange(setSchemaLabels(query, item))}
menuShouldPortal
/>
</EditorField>
)}
</EditorFieldGroup>
<EditorFieldGroup>
<EditorField label="Metric name" width={16}>
<Select
value={metricName ? toOption(metricName) : null}
options={metricOptions}
allowCustomValue
onChange={({ value }) => value && onQueryChange(setMetricName(query, value))}
menuShouldPortal
/>
</EditorField>
<EditorField label="Aggregation" width={16}>
<Select
value={aggregation ? toOption(aggregation) : null}
options={appendTemplateVariables(datasource, AGGREGATIONS)}
onChange={({ value }) => value && onQueryChange(setAggregation(query, value))}
menuShouldPortal
/>
</EditorField>
</EditorFieldGroup>
</>
);
};
export default SQLBuilderSelectRow;

160
public/app/plugins/datasource/cloudwatch/components/SQLBuilderEditor/SQLFilter.tsx Normal file
View File

@ -0,0 +1,160 @@
import { SelectableValue, toOption } from '@grafana/data';
import { Select } from '@grafana/ui';
import React, { useMemo, useState } from 'react';
import { useAsyncFn } from 'react-use';
import { COMPARISON_OPERATORS, EQUALS } from '../../cloudwatch-sql/language';
import { CloudWatchDatasource } from '../../datasource';
import { QueryEditorExpressionType, QueryEditorOperatorExpression, QueryEditorPropertyType } from '../../expressions';
import { useDimensionKeys } from '../../hooks';
import { CloudWatchMetricsQuery } from '../../types';
import { appendTemplateVariables } from '../../utils/utils';
import AccessoryButton from '../ui/AccessoryButton';
import EditorList from '../ui/EditorList';
import InputGroup from '../ui/InputGroup';
import {
getFlattenedFilters,
getMetricNameFromExpression,
getNamespaceFromExpression,
sanitizeOperator,
setOperatorExpressionName,
setOperatorExpressionProperty,
setOperatorExpressionValue,
setSql,
} from './utils';
interface SQLFilterProps {
query: CloudWatchMetricsQuery;
datasource: CloudWatchDatasource;
onQueryChange: (query: CloudWatchMetricsQuery) => void;
}
const OPERATORS = COMPARISON_OPERATORS.map(toOption);
const SQLFilter: React.FC<SQLFilterProps> = ({ query, onQueryChange, datasource }) => {
const filtersFromQuery = useMemo(() => getFlattenedFilters(query.sql ?? {}), [query.sql]);
const [filters, setFilters] = useState<QueryEditorOperatorExpression[]>(filtersFromQuery);
const onChange = (newItems: Array<Partial<QueryEditorOperatorExpression>>) => {
// As new (empty object) items come in, with need to make sure they have the correct type
const cleaned = newItems.map(
(v): QueryEditorOperatorExpression => ({
type: QueryEditorExpressionType.Operator,
property: v.property ?? { type: QueryEditorPropertyType.String },
operator: v.operator ?? {
name: EQUALS,
},
})
);
setFilters(cleaned);
// Only save valid and complete filters into the query state
const validExpressions: QueryEditorOperatorExpression[] = [];
for (const operatorExpression of cleaned) {
const validated = sanitizeOperator(operatorExpression);
if (validated) {
validExpressions.push(validated);
}
}
const where = validExpressions.length
? {
type: QueryEditorExpressionType.And as const,
expressions: validExpressions,
}
: undefined;
onQueryChange(setSql(query, { where }));
};
return <EditorList items={filters} onChange={onChange} renderItem={makeRenderFilter(datasource, query)} />;
};
// Making component functions in the render body is not recommended, but it works for now.
// If some problems arise (perhaps with state going missing), consider this to be a potential cause
function makeRenderFilter(datasource: CloudWatchDatasource, query: CloudWatchMetricsQuery) {
function renderFilter(
item: Partial<QueryEditorOperatorExpression>,
onChange: (item: QueryEditorOperatorExpression) => void,
onDelete: () => void
) {
return <FilterItem datasource={datasource} query={query} filter={item} onChange={onChange} onDelete={onDelete} />;
}
return renderFilter;
}
export default SQLFilter;
interface FilterItemProps {
datasource: CloudWatchDatasource;
query: CloudWatchMetricsQuery;
filter: Partial<QueryEditorOperatorExpression>;
onChange: (item: QueryEditorOperatorExpression) => void;
onDelete: () => void;
}
const FilterItem: React.FC<FilterItemProps> = (props) => {
const { datasource, query, filter, onChange, onDelete } = props;
const sql = query.sql ?? {};
const namespace = getNamespaceFromExpression(sql.from);
const metricName = getMetricNameFromExpression(sql.select);
const dimensionKeys = useDimensionKeys(datasource, query.region, namespace, metricName);
const loadDimensionValues = async () => {
if (!filter.property?.name) {
return [];
}
return datasource
.getDimensionValues(query.region, namespace, metricName, filter.property.name, {})
.then((result: Array<SelectableValue<string>>) => {
return appendTemplateVariables(datasource, result);
});
};
const [state, loadOptions] = useAsyncFn(loadDimensionValues, [
query.region,
namespace,
metricName,
filter.property?.name,
]);
return (
<InputGroup>
<Select
width="auto"
value={filter.property?.name ? toOption(filter.property?.name) : null}
options={dimensionKeys}
allowCustomValue
onChange={({ value }) => value && onChange(setOperatorExpressionProperty(filter, value))}
menuShouldPortal
/>
<Select
width="auto"
value={filter.operator?.name && toOption(filter.operator.name)}
options={OPERATORS}
onChange={({ value }) => value && onChange(setOperatorExpressionName(filter, value))}
menuShouldPortal
/>
<Select
width="auto"
isLoading={state.loading}
value={
filter.operator?.value && typeof filter.operator?.value === 'string' ? toOption(filter.operator?.value) : null
}
options={state.value}
allowCustomValue
onOpenMenu={loadOptions}
onChange={({ value }) => value && onChange(setOperatorExpressionValue(filter, value))}
menuShouldPortal
/>
<AccessoryButton aria-label="remove" icon="times" variant="secondary" onClick={onDelete} />
</InputGroup>
);
};

61
public/app/plugins/datasource/cloudwatch/components/SQLBuilderEditor/SQLGroupBy.test.tsx Normal file
View File

@ -0,0 +1,61 @@
import React from 'react';
import { act, render, screen, waitFor } from '@testing-library/react';
import { CloudWatchMetricsQuery, MetricEditorMode, MetricQueryType, SQLExpression } from '../../types';
import { setupMockedDataSource } from '../../__mocks__/CloudWatchDataSource';
import { createArray, createGroupBy } from '../../__mocks__/sqlUtils';
import SQLGroupBy from './SQLGroupBy';
const { datasource } = setupMockedDataSource();
const makeSQLQuery = (sql?: SQLExpression): CloudWatchMetricsQuery => ({
queryMode: 'Metrics',
refId: '',
id: '',
region: 'us-east-1',
namespace: 'ec2',
dimensions: { somekey: 'somevalue' },
metricQueryType: MetricQueryType.Query,
metricEditorMode: MetricEditorMode.Builder,
sql: sql,
});
describe('Cloudwatch SQLGroupBy', () => {
const baseProps = {
query: makeSQLQuery(),
datasource,
onQueryChange: () => {},
};
it('should load dimension keys with an empty dimension filter in case no group bys exist', async () => {
const query = makeSQLQuery({
groupBy: undefined,
});
render(<SQLGroupBy {...baseProps} query={query} />);
act(async () => {
await waitFor(() =>
expect(datasource.getDimensionKeys).toHaveBeenCalledWith(query.namespace, query.region, {}, undefined)
);
});
});
it('should load dimension keys with a dimension filter in case a group bys exist', async () => {
const query = makeSQLQuery({
groupBy: createArray([createGroupBy('InstanceId'), createGroupBy('InstanceType')]),
});
render(<SQLGroupBy {...baseProps} query={query} />);
act(async () => {
expect(screen.getByText('InstanceId')).toBeInTheDocument();
expect(screen.getByText('InstanceType')).toBeInTheDocument();
await waitFor(() =>
expect(datasource.getDimensionKeys).toHaveBeenCalledWith(
query.namespace,
query.region,
{ InstanceId: null, InstanceType: null },
undefined
)
);
});
});
});

109
public/app/plugins/datasource/cloudwatch/components/SQLBuilderEditor/SQLGroupBy.tsx Normal file
View File

@ -0,0 +1,109 @@
import { SelectableValue, toOption } from '@grafana/data';
import { Select } from '@grafana/ui';
import React, { useMemo, useState } from 'react';
import { CloudWatchDatasource } from '../../datasource';
import { QueryEditorExpressionType, QueryEditorGroupByExpression, QueryEditorPropertyType } from '../../expressions';
import { useDimensionKeys } from '../../hooks';
import { CloudWatchMetricsQuery } from '../../types';
import AccessoryButton from '../ui/AccessoryButton';
import EditorList from '../ui/EditorList';
import InputGroup from '../ui/InputGroup';
import {
getFlattenedGroupBys,
getMetricNameFromExpression,
getNamespaceFromExpression,
setGroupByField,
setSql,
} from './utils';
interface SQLGroupByProps {
query: CloudWatchMetricsQuery;
datasource: CloudWatchDatasource;
onQueryChange: (query: CloudWatchMetricsQuery) => void;
}
const SQLGroupBy: React.FC<SQLGroupByProps> = ({ query, datasource, onQueryChange }) => {
const sql = query.sql ?? {};
const groupBysFromQuery = useMemo(() => getFlattenedGroupBys(query.sql ?? {}), [query.sql]);
const [items, setItems] = useState<QueryEditorGroupByExpression[]>(groupBysFromQuery);
const namespace = getNamespaceFromExpression(sql.from);
const metricName = getMetricNameFromExpression(sql.select);
const baseOptions = useDimensionKeys(datasource, query.region, namespace, metricName);
const options = useMemo(
// Exclude options we've already selected
() => baseOptions.filter((option) => !groupBysFromQuery.some((v) => v.property.name === option.value)),
[baseOptions, groupBysFromQuery]
);
const onChange = (newItems: Array<Partial<QueryEditorGroupByExpression>>) => {
// As new (empty object) items come in, with need to make sure they have the correct type
const cleaned = newItems.map(
(v): QueryEditorGroupByExpression => ({
type: QueryEditorExpressionType.GroupBy,
property: {
type: QueryEditorPropertyType.String,
name: v.property?.name,
},
})
);
setItems(cleaned);
// Only save complete expressions into the query state;
const completeExpressions = cleaned.filter((v) => v.property?.name);
const groupBy = completeExpressions.length
? {
type: QueryEditorExpressionType.And as const,
expressions: completeExpressions,
}
: undefined;
onQueryChange(setSql(query, { groupBy }));
};
return <EditorList items={items} onChange={onChange} renderItem={makeRenderItem(options)} />;
};
function makeRenderItem(options: Array<SelectableValue<string>>) {
function renderItem(
item: Partial<QueryEditorGroupByExpression>,
onChange: (item: QueryEditorGroupByExpression) => void,
onDelete: () => void
) {
return <GroupByItem options={options} item={item} onChange={onChange} onDelete={onDelete} />;
}
return renderItem;
}
interface GroupByItemProps {
options: Array<SelectableValue<string>>;
item: Partial<QueryEditorGroupByExpression>;
onChange: (item: QueryEditorGroupByExpression) => void;
onDelete: () => void;
}
const GroupByItem: React.FC<GroupByItemProps> = (props) => {
const { options, item, onChange, onDelete } = props;
const fieldName = item.property?.name;
return (
<InputGroup>
<Select
width="auto"
value={fieldName ? toOption(fieldName) : null}
options={options}
allowCustomValue
onChange={({ value }) => value && onChange(setGroupByField(value))}
menuShouldPortal
/>
<AccessoryButton aria-label="remove" icon="times" variant="secondary" onClick={onDelete} />
</InputGroup>
);
};
export default SQLGroupBy;

64
public/app/plugins/datasource/cloudwatch/components/SQLBuilderEditor/SQLOrderByGroup.tsx Normal file
View File

@ -0,0 +1,64 @@
import { SelectableValue, toOption } from '@grafana/data';
import { Select } from '@grafana/ui';
import React from 'react';
import { ASC, DESC, STATISTICS } from '../../cloudwatch-sql/language';
import { CloudWatchDatasource } from '../../datasource';
import { CloudWatchMetricsQuery } from '../../types';
import { appendTemplateVariables } from '../../utils/utils';
import AccessoryButton from '../ui/AccessoryButton';
import EditorField from '../ui/EditorField';
import EditorFieldGroup from '../ui/EditorFieldGroup';
import { setOrderBy, setSql } from './utils';
interface SQLBuilderSelectRowProps {
query: CloudWatchMetricsQuery;
datasource: CloudWatchDatasource;
onQueryChange: (query: CloudWatchMetricsQuery) => void;
}
const orderByDirections: Array<SelectableValue<string>> = [
{ label: ASC, value: ASC },
{ label: DESC, value: DESC },
];
const SQLOrderByGroup: React.FC<SQLBuilderSelectRowProps> = ({ query, onQueryChange, datasource }) => {
const sql = query.sql ?? {};
const orderBy = sql.orderBy?.name;
const orderByDirection = sql.orderByDirection;
return (
<EditorFieldGroup>
<EditorField label="Order by" optional width={16}>
<>
<Select
onChange={({ value }) => value && onQueryChange(setOrderBy(query, value))}
options={appendTemplateVariables(datasource, STATISTICS.map(toOption))}
value={orderBy ? toOption(orderBy) : null}
menuShouldPortal
/>
{orderBy && (
<AccessoryButton
aria-label="remove"
icon="times"
variant="secondary"
onClick={() => onQueryChange(setSql(query, { orderBy: undefined }))}
/>
)}
</>
</EditorField>
<EditorField label="Direction" width={16}>
<Select
inputId="cloudwatch-sql-order-by-direction"
disabled={!orderBy}
value={orderByDirection ? toOption(orderByDirection) : orderByDirections[0]}
options={appendTemplateVariables(datasource, orderByDirections)}
onChange={(item) => item && onQueryChange(setSql(query, { orderByDirection: item.value }))}
menuShouldPortal
/>
</EditorField>
</EditorFieldGroup>
);
};
export default SQLOrderByGroup;

1
public/app/plugins/datasource/cloudwatch/components/SQLBuilderEditor/index.tsx Normal file
View File

@ -0,0 +1 @@
export { SQLBuilderEditor } from './SQLBuilderEditor';

346
public/app/plugins/datasource/cloudwatch/components/SQLBuilderEditor/utils.ts Normal file
View File

@ -0,0 +1,346 @@
import { SelectableValue } from './../../../../../../../packages/grafana-data/src/types/select';
import { SCHEMA } from '../../cloudwatch-sql/language';
import {
QueryEditorExpressionType,
QueryEditorPropertyType,
QueryEditorFunctionParameterExpression,
QueryEditorArrayExpression,
QueryEditorOperatorExpression,
QueryEditorGroupByExpression,
} from '../../expressions';
import { SQLExpression, CloudWatchMetricsQuery, Dimensions } from '../../types';
export function getMetricNameFromExpression(selectExpression: SQLExpression['select']): string | undefined {
return selectExpression?.parameters?.[0].name;
}
export function getNamespaceFromExpression(fromExpression: SQLExpression['from']): string | undefined {
// It's just a simple `FROM "AWS/EC2"` expression
if (fromExpression?.type === QueryEditorExpressionType.Property) {
return fromExpression.property.name; // PR TODO: do we need to test the type here? It can only be string?
}
// It's a more complicated `FROM SCHEMA("AWS/EC2", ...)` expression
if (fromExpression?.type === QueryEditorExpressionType.Function) {
// TODO: do we need to test the name of the function?
return fromExpression.parameters?.[0].name;
}
return undefined;
}
export function getSchemaLabelKeys(fromExpression: SQLExpression['from']): string[] | undefined {
// Schema label keys are second to n arguments in the from expression function
if (fromExpression?.type === QueryEditorExpressionType.Function && fromExpression?.parameters?.length) {
if (fromExpression?.parameters?.length <= 1) {
return [];
}
// ignore the first arg (the namespace)
const paramExpressions = fromExpression?.parameters.slice(1);
return paramExpressions.reduce<string[]>((acc, curr) => (curr.name ? [...acc, curr.name] : acc), []);
}
return undefined;
}
export function isUsingWithSchema(fromExpression: SQLExpression['from']): boolean {
return fromExpression?.type === QueryEditorExpressionType.Function && fromExpression.name === SCHEMA;
}
/** Given a partial operator expression, return a non-partial if it's valid, or undefined */
export function sanitizeOperator(
expression: Partial<QueryEditorOperatorExpression>
): QueryEditorOperatorExpression | undefined {
const key = expression.property?.name;
const value = expression.operator?.value;
const operator = expression.operator?.name;
if (key && value && operator) {
return {
type: QueryEditorExpressionType.Operator,
property: {
type: QueryEditorPropertyType.String,
name: key,
},
operator: {
value,
name: operator,
},
};
}
return undefined;
}
/**
* Given an array of Expressions, flattens them to the leaf Operator expressions.
* Note, this loses context of any nested ANDs or ORs, so will not be useful once we support nested conditions */
function flattenOperatorExpressions(
expressions: QueryEditorArrayExpression['expressions']
): QueryEditorOperatorExpression[] {
return expressions.flatMap((expression) => {
if (expression.type === QueryEditorExpressionType.Operator) {
return expression;
}
if (expression.type === QueryEditorExpressionType.And || expression.type === QueryEditorExpressionType.Or) {
return flattenOperatorExpressions(expression.expressions);
}
// Expressions that we don't expect to find in the WHERE filter will be ignored
return [];
});
}
/** Returns a flattened list of WHERE filters, losing all context of nested filters or AND vs OR. Not suitable
* if the UI supports nested conditions
*/
export function getFlattenedFilters(sql: SQLExpression): QueryEditorOperatorExpression[] {
const where = sql.where;
return flattenOperatorExpressions(where?.expressions ?? []);
}
/**
* Given an array of Expressions, flattens them to the leaf Operator expressions.
* Note, this loses context of any nested ANDs or ORs, so will not be useful once we support nested conditions */
function flattenGroupByExpressions(
expressions: QueryEditorArrayExpression['expressions']
): QueryEditorGroupByExpression[] {
return expressions.flatMap((expression) => {
if (expression.type === QueryEditorExpressionType.GroupBy) {
return expression;
}
// Expressions that we don't expect to find in the GROUP BY will be ignored
return [];
});
}
/** Returns a flattened list of GROUP BY expressions, losing all context of nested filters or AND vs OR.
*/
export function getFlattenedGroupBys(sql: SQLExpression): QueryEditorGroupByExpression[] {
const groupBy = sql.groupBy;
return flattenGroupByExpressions(groupBy?.expressions ?? []);
}
/** Converts a string array to a Dimensions object with null values **/
export function stringArrayToDimensions(arr: string[]): Dimensions {
return arr.reduce((acc, curr) => {
if (curr) {
return { ...acc, [curr]: null };
}
return acc;
}, {});
}
export function setSql(query: CloudWatchMetricsQuery, sql: SQLExpression): CloudWatchMetricsQuery {
return {
...query,
sql: {
...(query.sql ?? {}),
...sql,
},
};
}
export function setNamespace(query: CloudWatchMetricsQuery, namespace: string | undefined): CloudWatchMetricsQuery {
const sql = query.sql ?? {};
if (namespace === undefined) {
return setSql(query, {
from: undefined,
});
}
// It's just a simple `FROM "AWS/EC2"` expression
if (!sql.from || sql.from.type === QueryEditorExpressionType.Property) {
return setSql(query, {
from: {
type: QueryEditorExpressionType.Property,
property: {
type: QueryEditorPropertyType.String,
name: namespace,
},
},
});
}
// It's a more complicated `FROM SCHEMA("AWS/EC2", ...)` expression
if (sql.from.type === QueryEditorExpressionType.Function) {
const namespaceParam: QueryEditorFunctionParameterExpression = {
type: QueryEditorExpressionType.FunctionParameter,
name: namespace,
};
const labelKeys = (sql.from.parameters ?? []).slice(1);
return setSql(query, {
from: {
type: QueryEditorExpressionType.Function,
name: SCHEMA,
parameters: [namespaceParam, ...labelKeys],
},
});
}
// TODO: do the with schema bit
return query;
}
export function setSchemaLabels(
query: CloudWatchMetricsQuery,
schemaLabels: Array<SelectableValue<string>> | SelectableValue<string>
): CloudWatchMetricsQuery {
const sql = query.sql ?? {};
schemaLabels = Array.isArray(schemaLabels) ? schemaLabels.map((l) => l.value) : [schemaLabels.value];
// schema labels are the second parameter in the schema function. `... FROM SCHEMA("AWS/EC2", label1, label2 ...)`
if (sql.from?.type === QueryEditorExpressionType.Function && sql.from.parameters?.length) {
const parameters: QueryEditorFunctionParameterExpression[] = (schemaLabels ?? []).map((label: string) => ({
type: QueryEditorExpressionType.FunctionParameter,
name: label,
}));
const namespaceParam = (sql.from.parameters ?? [])[0];
return setSql(query, {
from: {
type: QueryEditorExpressionType.Function,
name: SCHEMA,
parameters: [namespaceParam, ...parameters],
},
});
}
return query;
}
export function setMetricName(query: CloudWatchMetricsQuery, metricName: string): CloudWatchMetricsQuery {
const param: QueryEditorFunctionParameterExpression = {
type: QueryEditorExpressionType.FunctionParameter,
name: metricName,
};
return setSql(query, {
select: {
type: QueryEditorExpressionType.Function,
...(query.sql?.select ?? {}),
parameters: [param],
},
});
}
export function setAggregation(query: CloudWatchMetricsQuery, aggregation: string): CloudWatchMetricsQuery {
return setSql(query, {
select: {
type: QueryEditorExpressionType.Function,
...(query.sql?.select ?? {}),
name: aggregation,
},
});
}
export function setOrderBy(query: CloudWatchMetricsQuery, aggregation: string): CloudWatchMetricsQuery {
return setSql(query, {
orderBy: {
type: QueryEditorExpressionType.Function,
name: aggregation,
},
});
}
export function setWithSchema(query: CloudWatchMetricsQuery, withSchema: boolean): CloudWatchMetricsQuery {
const namespace = getNamespaceFromExpression((query.sql ?? {}).from);
if (withSchema) {
const namespaceParam: QueryEditorFunctionParameterExpression = {
type: QueryEditorExpressionType.FunctionParameter,
name: namespace,
};
return setSql(query, {
from: {
type: QueryEditorExpressionType.Function,
name: SCHEMA,
parameters: [namespaceParam],
},
});
}
return setSql(query, {
from: {
type: QueryEditorExpressionType.Property,
property: {
type: QueryEditorPropertyType.String,
name: namespace,
},
},
});
}
/** Sets the left hand side (InstanceId) in an OperatorExpression
* Accepts a partial expression to use in an editor
*/
export function setOperatorExpressionProperty(
expression: Partial<QueryEditorOperatorExpression>,
property: string
): QueryEditorOperatorExpression {
return {
type: QueryEditorExpressionType.Operator,
property: {
type: QueryEditorPropertyType.String,
name: property,
},
operator: expression.operator ?? {},
};
}
/** Sets the operator ("==") in an OperatorExpression
* Accepts a partial expression to use in an editor
*/
export function setOperatorExpressionName(
expression: Partial<QueryEditorOperatorExpression>,
name: string
): QueryEditorOperatorExpression {
return {
type: QueryEditorExpressionType.Operator,
property: expression.property ?? {
type: QueryEditorPropertyType.String,
},
operator: {
...expression.operator,
name,
},
};
}
/** Sets the right hand side ("i-abc123445") in an OperatorExpression
* Accepts a partial expression to use in an editor
*/
export function setOperatorExpressionValue(
expression: Partial<QueryEditorOperatorExpression>,
value: string
): QueryEditorOperatorExpression {
return {
type: QueryEditorExpressionType.Operator,
property: expression.property ?? {
type: QueryEditorPropertyType.String,
},
operator: {
...expression.operator,
value,
},
};
}
/** Creates a GroupByExpression for a specified field
*/
export function setGroupByField(field: string): QueryEditorGroupByExpression {
return {
type: QueryEditorExpressionType.GroupBy,
property: {
type: QueryEditorPropertyType.String,
name: field,
},
};
}

50
public/app/plugins/datasource/cloudwatch/components/SQLCodeEditor.tsx Normal file
View File

@ -0,0 +1,50 @@
import React, { FunctionComponent, useCallback, useEffect } from 'react';
import type * as monacoType from 'monaco-editor/esm/vs/editor/editor.api';
import { CodeEditor, Monaco } from '@grafana/ui';
import { CloudWatchDatasource } from '../datasource';
import language from '../cloudwatch-sql/definition';
import { TRIGGER_SUGGEST } from '../cloudwatch-sql/completion/commands';
import { registerLanguage } from '../cloudwatch-sql/register';
export interface Props {
region: string;
sql: string;
onChange: (sql: string) => void;
onRunQuery: () => void;
datasource: CloudWatchDatasource;
}
export const SQLCodeEditor: FunctionComponent<Props> = ({ region, sql, onChange, onRunQuery, datasource }) => {
useEffect(() => {
datasource.sqlCompletionItemProvider.setRegion(region);
}, [region, datasource]);
const onEditorMount = useCallback(
(editor: monacoType.editor.IStandaloneCodeEditor, monaco: Monaco) => {
editor.onDidFocusEditorText(() => editor.trigger(TRIGGER_SUGGEST.id, TRIGGER_SUGGEST.id, {}));
editor.addCommand(monaco.KeyMod.Shift | monaco.KeyCode.Enter, () => {
const text = editor.getValue();
onChange(text);
onRunQuery();
});
},
[onChange, onRunQuery]
);
return (
<CodeEditor
height={'150px'}
language={language.id}
value={sql}
onBlur={(value) => {
if (value !== sql) {
onChange(value);
}
}}
showMiniMap={false}
showLineNumbers={true}
onBeforeEditorMount={(monaco: Monaco) => registerLanguage(monaco, datasource.sqlCompletionItemProvider)}
onEditorDidMount={onEditorMount}
/>
);
};

18
public/app/plugins/datasource/cloudwatch/components/__snapshots__/Alias.test.tsx.snap
View File

@ -1,18 +0,0 @@
// Jest Snapshot v1, https://goo.gl/fbAQLP
exports[`Alias should render component 1`] = `
<div
style={
Object {
"flexGrow": 1,
}
}
>
<input
className="gf-form-input gf-form-input width-16"
onChange={[Function]}
type="text"
value="legend"
/>
</div>
`;

7
public/app/plugins/datasource/cloudwatch/components/index.ts
View File

@ -1,6 +1,9 @@
export { Dimensions } from './Dimensions';
export { Dimensions } from './MetricStatEditor/Dimensions';
export { QueryInlineField, QueryField } from './Forms';
export { Alias } from './Alias';
export { MetricsQueryFieldsEditor } from './MetricsQueryFieldsEditor';
export { PanelQueryEditor } from './PanelQueryEditor';
export { CloudWatchLogsQueryEditor } from './LogsQueryEditor';
export { MetricStatEditor } from './MetricStatEditor';
export { SQLBuilderEditor } from './SQLBuilderEditor';
export { MathExpressionQueryField } from './MathExpressionQueryField';
export { SQLCodeEditor } from './SQLCodeEditor';

152
public/app/plugins/datasource/cloudwatch/datasource.test.ts
View File

@ -3,8 +3,15 @@ import { setDataSourceSrv } from '@grafana/runtime';
import { ArrayVector, DataFrame, dataFrameToJSON, dateTime, Field, MutableDataFrame } from '@grafana/data';
import { toArray } from 'rxjs/operators';
import { setupMockedDataSource } from './__mocks__/CloudWatchDataSource';
import { CloudWatchLogsQueryStatus } from './types';
import { CloudWatchMetricsQuery, MetricEditorMode, MetricQueryType, CloudWatchLogsQueryStatus } from './types';
import {
setupMockedDataSource,
namespaceVariable,
metricVariable,
labelsVariable,
limitVariable,
} from './__mocks__/CloudWatchDataSource';
import { CloudWatchDatasource } from './datasource';
describe('datasource', () => {
describe('query', () => {
@ -87,6 +94,107 @@ describe('datasource', () => {
});
});
describe('filterMetricQuery', () => {
let baseQuery: CloudWatchMetricsQuery;
let datasource: CloudWatchDatasource;
beforeEach(() => {
datasource = setupMockedDataSource().datasource;
baseQuery = {
id: '',
region: 'us-east-2',
namespace: '',
period: '',
alias: '',
metricName: '',
dimensions: {},
matchExact: true,
statistic: '',
expression: '',
refId: '',
};
});
it('should error if invalid mode', async () => {
expect(() => datasource.filterMetricQuery(baseQuery)).toThrowError('invalid metric editor mode');
});
describe('metric search queries', () => {
beforeEach(() => {
datasource = setupMockedDataSource().datasource;
baseQuery = {
...baseQuery,
namespace: 'AWS/EC2',
metricName: 'CPUUtilization',
statistic: 'Average',
metricQueryType: MetricQueryType.Search,
metricEditorMode: MetricEditorMode.Builder,
};
});
it('should not allow queries that dont have `matchExact` or dimensions', async () => {
const valid = datasource.filterMetricQuery(baseQuery);
expect(valid).toBeFalsy();
});
it('should allow queries that have `matchExact`', async () => {
baseQuery.matchExact = false;
const valid = datasource.filterMetricQuery(baseQuery);
expect(valid).toBeTruthy();
});
it('should allow queries that have dimensions', async () => {
baseQuery.dimensions = { instanceId: ['xyz'] };
const valid = datasource.filterMetricQuery(baseQuery);
expect(valid).toBeTruthy();
});
});
describe('metric search expression queries', () => {
beforeEach(() => {
datasource = setupMockedDataSource().datasource;
baseQuery = {
...baseQuery,
metricQueryType: MetricQueryType.Search,
metricEditorMode: MetricEditorMode.Code,
};
});
it('should not allow queries that dont have an expresssion', async () => {
const valid = datasource.filterMetricQuery(baseQuery);
expect(valid).toBeFalsy();
});
it('should allow queries that have an expresssion', async () => {
baseQuery.expression = 'SUM([a,x])';
const valid = datasource.filterMetricQuery(baseQuery);
expect(valid).toBeTruthy();
});
});
describe('metric query queries', () => {
beforeEach(() => {
datasource = setupMockedDataSource().datasource;
baseQuery = {
...baseQuery,
metricQueryType: MetricQueryType.Query,
metricEditorMode: MetricEditorMode.Code,
};
});
it('should not allow queries that dont have a sql expresssion', async () => {
const valid = datasource.filterMetricQuery(baseQuery);
expect(valid).toBeFalsy();
});
it('should allow queries that have a sql expresssion', async () => {
baseQuery.sqlExpression = 'select SUM(CPUUtilization) from "AWS/EC2"';
const valid = datasource.filterMetricQuery(baseQuery);
expect(valid).toBeTruthy();
});
});
});
describe('performTimeSeriesQuery', () => {
it('should return the same length of data as result', async () => {
const { datasource } = setupMockedDataSource({
@ -126,6 +234,46 @@ describe('datasource', () => {
});
});
describe('template variable interpolation', () => {
it('interpolates variables correctly', async () => {
const { datasource, fetchMock } = setupMockedDataSource({
variables: [namespaceVariable, metricVariable, labelsVariable, limitVariable],
});
datasource.handleMetricQueries(
[
{
id: '',
refId: 'a',
region: 'us-east-2',
namespace: '',
period: '',
alias: '',
metricName: '',
dimensions: {},
matchExact: true,
statistic: '',
expression: '',
metricQueryType: MetricQueryType.Query,
metricEditorMode: MetricEditorMode.Code,
sqlExpression: 'SELECT SUM($metric) FROM "$namespace" GROUP BY ${labels:raw} LIMIT $limit',
},
],
{ range: { from: dateTime(), to: dateTime() } } as any
);
expect(fetchMock).toHaveBeenCalledWith(
expect.objectContaining({
data: expect.objectContaining({
queries: expect.arrayContaining([
expect.objectContaining({
sqlExpression: `SELECT SUM(CPUUtilization) FROM "AWS/EC2" GROUP BY InstanceId,InstanceType LIMIT 100`,
}),
]),
}),
})
);
});
});
describe('getLogGroupFields', () => {
it('passes region correctly', async () => {
const { datasource, fetchMock } = setupMockedDataSource();

88
public/app/plugins/datasource/cloudwatch/datasource.ts
View File

@ -32,6 +32,7 @@ import { getTimeSrv, TimeSrv } from 'app/features/dashboard/services/TimeSrv';
import { ThrottlingErrorMessage } from './components/ThrottlingErrorMessage';
import memoizedDebounce from './memoizedDebounce';
import {
MetricEditorMode,
CloudWatchJsonData,
CloudWatchLogsQuery,
CloudWatchLogsQueryStatus,
@ -43,6 +44,7 @@ import {
GetLogGroupFieldsResponse,
isCloudWatchLogsQuery,
LogAction,
MetricQueryType,
MetricQuery,
MetricRequest,
StartQueryRequest,
@ -57,6 +59,7 @@ import { increasingInterval } from './utils/rxjs/increasingInterval';
import { toTestingStatus } from '@grafana/runtime/src/utils/queryResponse';
import { addDataLinksToLogsResponse } from './utils/datalinks';
import { runWithRetry } from './utils/logsRetry';
import { CompletionItemProvider } from './cloudwatch-sql/completion/CompletionItemProvider';
const DS_QUERY_ENDPOINT = '/api/ds/query';
@ -87,11 +90,13 @@ export class CloudWatchDatasource
defaultRegion: any;
datasourceName: string;
languageProvider: CloudWatchLanguageProvider;
sqlCompletionItemProvider: CompletionItemProvider;
tracingDataSourceUid?: string;
logsTimeout: string;
type = 'cloudwatch';
standardStatistics = ['Average', 'Maximum', 'Minimum', 'Sum', 'SampleCount'];
debouncedAlert: (datasourceName: string, region: string) => void = memoizedDebounce(
displayAlert,
AppNotificationTimeout.Error
@ -114,6 +119,7 @@ export class CloudWatchDatasource
this.languageProvider = new CloudWatchLanguageProvider(this);
this.tracingDataSourceUid = instanceSettings.jsonData.tracingDatasourceUid;
this.logsTimeout = instanceSettings.jsonData.logsTimeout || '15m';
this.sqlCompletionItemProvider = new CompletionItemProvider(this);
}
query(options: DataQueryRequest<CloudWatchQuery>): Observable<DataQueryResponse> {
@ -220,35 +226,64 @@ export class CloudWatchDatasource
);
};
filterMetricQuery({
region,
metricQueryType,
metricEditorMode,
expression,
metricName,
namespace,
sqlExpression,
statistic,
dimensions,
...rest
}: CloudWatchMetricsQuery): boolean {
if (!region) {
return false;
}
if (metricQueryType === MetricQueryType.Search && metricEditorMode === MetricEditorMode.Builder) {
return (
!!namespace &&
!!metricName &&
!!statistic &&
(('matchExact' in rest && !rest.matchExact) || !isEmpty(dimensions))
);
} else if (metricQueryType === MetricQueryType.Search && metricEditorMode === MetricEditorMode.Code) {
return !!expression;
} else if (metricQueryType === MetricQueryType.Query) {
// still TBD how to validate the visual query builder for SQL
return !!sqlExpression;
}
throw new Error('invalid metric editor mode');
}
handleMetricQueries = (
metricQueries: CloudWatchMetricsQuery[],
options: DataQueryRequest<CloudWatchQuery>
): Observable<DataQueryResponse> => {
const validMetricsQueries = metricQueries
.filter(
(item) =>
(!!item.region && !!item.namespace && !!item.metricName && !!item.statistic) || item.expression?.length > 0
)
.map(
(item: CloudWatchMetricsQuery): MetricQuery => {
item.region = this.replace(this.getActualRegion(item.region), options.scopedVars, true, 'region');
item.namespace = this.replace(item.namespace, options.scopedVars, true, 'namespace');
item.metricName = this.replace(item.metricName, options.scopedVars, true, 'metric name');
item.dimensions = this.convertDimensionFormat(item.dimensions, options.scopedVars);
item.statistic = this.templateSrv.replace(item.statistic, options.scopedVars);
item.period = String(this.getPeriod(item, options)); // use string format for period in graph query, and alerting
item.id = this.templateSrv.replace(item.id, options.scopedVars);
item.expression = this.templateSrv.replace(item.expression, options.scopedVars);
const validMetricsQueries = metricQueries.filter(this.filterMetricQuery).map(
(item: CloudWatchMetricsQuery): MetricQuery => {
item.region = this.replace(this.getActualRegion(item.region), options.scopedVars, true, 'region');
item.namespace = this.replace(item.namespace, options.scopedVars, true, 'namespace');
item.metricName = this.replace(item.metricName, options.scopedVars, true, 'metric name');
item.dimensions = this.convertDimensionFormat(item.dimensions ?? {}, options.scopedVars);
item.statistic = this.templateSrv.replace(item.statistic, options.scopedVars);
item.period = String(this.getPeriod(item, options)); // use string format for period in graph query, and alerting
item.id = this.templateSrv.replace(item.id, options.scopedVars);
item.expression = this.templateSrv.replace(item.expression, options.scopedVars);
item.sqlExpression = this.templateSrv.replace(item.sqlExpression, options.scopedVars, 'raw');
return {
intervalMs: options.intervalMs,
maxDataPoints: options.maxDataPoints,
type: 'timeSeriesQuery',
...item,
datasource: this.getRef(),
};
}
);
return {
intervalMs: options.intervalMs,
maxDataPoints: options.maxDataPoints,
...item,
type: 'timeSeriesQuery',
datasource: this.getRef(),
};
}
);
// No valid targets, return the empty result to save a round trip.
if (isEmpty(validMetricsQueries)) {
@ -1001,13 +1036,14 @@ export class CloudWatchDatasource
interpolateMetricsQueryVariables(
query: CloudWatchMetricsQuery,
scopedVars: ScopedVars
): Pick<CloudWatchMetricsQuery, 'alias' | 'metricName' | 'namespace' | 'period' | 'dimensions'> {
): Pick<CloudWatchMetricsQuery, 'alias' | 'metricName' | 'namespace' | 'period' | 'dimensions' | 'sqlExpression'> {
return {
alias: this.replace(query.alias, scopedVars),
metricName: this.replace(query.metricName, scopedVars),
namespace: this.replace(query.namespace, scopedVars),
period: this.replace(query.period, scopedVars),
dimensions: Object.entries(query.dimensions).reduce((prev, [key, value]) => {
sqlExpression: this.replace(query.sqlExpression, scopedVars),
dimensions: Object.entries(query.dimensions ?? {}).reduce((prev, [key, value]) => {
if (Array.isArray(value)) {
return { ...prev, [key]: value };
}

66
public/app/plugins/datasource/cloudwatch/expressions.ts Normal file
View File

@ -0,0 +1,66 @@
export enum QueryEditorPropertyType {
String = 'string',
}
export interface QueryEditorProperty {
type: QueryEditorPropertyType;
name?: string;
}
export type QueryEditorOperatorType = string | boolean | number;
type QueryEditorOperatorValueType = QueryEditorOperatorType | QueryEditorOperatorType[];
export interface QueryEditorOperator<T extends QueryEditorOperatorValueType> {
name?: string;
value?: T;
}
export interface QueryEditorOperatorExpression {
type: QueryEditorExpressionType.Operator;
property: QueryEditorProperty;
operator: QueryEditorOperator<QueryEditorOperatorValueType>;
}
export interface QueryEditorArrayExpression {
type: QueryEditorExpressionType.And | QueryEditorExpressionType.Or;
expressions: QueryEditorExpression[] | QueryEditorArrayExpression[];
}
export interface QueryEditorPropertyExpression {
type: QueryEditorExpressionType.Property;
property: QueryEditorProperty;
}
export enum QueryEditorExpressionType {
Property = 'property',
Operator = 'operator',
Or = 'or',
And = 'and',
GroupBy = 'groupBy',
Function = 'function',
FunctionParameter = 'functionParameter',
}
export type QueryEditorExpression =
| QueryEditorArrayExpression
| QueryEditorPropertyExpression
| QueryEditorGroupByExpression
| QueryEditorFunctionExpression
| QueryEditorFunctionParameterExpression
| QueryEditorOperatorExpression;
export interface QueryEditorGroupByExpression {
type: QueryEditorExpressionType.GroupBy;
property: QueryEditorProperty;
}
export interface QueryEditorFunctionExpression {
type: QueryEditorExpressionType.Function;
name?: string;
parameters?: QueryEditorFunctionParameterExpression[];
}
export interface QueryEditorFunctionParameterExpression {
type: QueryEditorExpressionType.FunctionParameter;
name?: string;
}

5
public/app/plugins/datasource/cloudwatch/guards.ts Normal file
View File

@ -0,0 +1,5 @@
import { CloudWatchMetricsQuery, CloudWatchQuery } from './types';
export const isMetricsQuery = (query: CloudWatchQuery): query is CloudWatchMetricsQuery => {
return query.queryMode === 'Metrics';
};

70
public/app/plugins/datasource/cloudwatch/hooks.ts Normal file
View File

@ -0,0 +1,70 @@
import { useEffect, useState } from 'react';
import { SelectableValue, toOption } from '@grafana/data';
import { appendTemplateVariables } from './utils/utils';
import { Dimensions } from './types';
import { CloudWatchDatasource } from './datasource';
import { useDeepCompareEffect } from 'react-use';
export const useRegions = (datasource: CloudWatchDatasource): [Array<SelectableValue<string>>, boolean] => {
const [regionsIsLoading, setRegionsIsLoading] = useState<boolean>(false);
const [regions, setRegions] = useState<Array<SelectableValue<string>>>([{ label: 'default', value: 'default' }]);
useEffect(() => {
setRegionsIsLoading(true);
const variableOptionGroup = {
label: 'Template Variables',
options: datasource.getVariables().map(toOption),
};
datasource
.getRegions()
.then((regions: Array<SelectableValue<string>>) => setRegions([...regions, variableOptionGroup]))
.finally(() => setRegionsIsLoading(false));
}, [datasource]);
return [regions, regionsIsLoading];
};
export const useNamespaces = (datasource: CloudWatchDatasource) => {
const [namespaces, setNamespaces] = useState<Array<SelectableValue<string>>>([]);
useEffect(() => {
datasource.getNamespaces().then((namespaces) => {
setNamespaces(appendTemplateVariables(datasource, namespaces));
});
}, [datasource]);
return namespaces;
};
export const useMetrics = (datasource: CloudWatchDatasource, region: string, namespace: string | undefined) => {
const [metrics, setMetrics] = useState<Array<SelectableValue<string>>>([]);
useEffect(() => {
datasource.getMetrics(namespace, region).then((result: Array<SelectableValue<string>>) => {
setMetrics(appendTemplateVariables(datasource, result));
});
}, [datasource, region, namespace]);
return metrics;
};
export const useDimensionKeys = (
datasource: CloudWatchDatasource,
region: string,
namespace: string | undefined,
metricName: string | undefined,
dimensionFilter?: Dimensions
) => {
const [dimensionKeys, setDimensionKeys] = useState<Array<SelectableValue<string>>>([]);
// doing deep comparison to avoid making new api calls to list metrics unless dimension filter object props changes
useDeepCompareEffect(() => {
datasource
.getDimensionKeys(namespace, region, dimensionFilter, metricName)
.then((result: Array<SelectableValue<string>>) => {
setDimensionKeys(appendTemplateVariables(datasource, result));
});
}, [datasource, region, namespace, metricName, dimensionFilter]);
return dimensionKeys;
};

61
public/app/plugins/datasource/cloudwatch/migration.test.ts
View File

@ -1,6 +1,10 @@
import { DataQuery } from '@grafana/data';
import { migrateMultipleStatsAnnotationQuery, migrateMultipleStatsMetricsQuery } from './migrations';
import { CloudWatchAnnotationQuery, CloudWatchMetricsAnnotationQuery, CloudWatchMetricsQuery } from './types';
import {
migrateMultipleStatsAnnotationQuery,
migrateMultipleStatsMetricsQuery,
migrateCloudWatchQuery,
} from './migrations';
import { CloudWatchAnnotationQuery, CloudWatchMetricsQuery, MetricQueryType, MetricEditorMode } from './types';
describe('migration', () => {
describe('migrateMultipleStatsMetricsQuery', () => {
@ -71,7 +75,7 @@ describe('migration', () => {
};
const newAnnotations = migrateMultipleStatsAnnotationQuery(annotationToMigrate as CloudWatchAnnotationQuery);
const newCloudWatchAnnotations = newAnnotations as CloudWatchMetricsAnnotationQuery[];
const newCloudWatchAnnotations = newAnnotations as CloudWatchAnnotationQuery[];
it('should create one new annotation for each stat', () => {
expect(newAnnotations.length).toBe(1);
@ -114,5 +118,56 @@ describe('migration', () => {
expect(annotationToMigrate).not.toHaveProperty('statistics');
});
});
describe('migrateCloudWatchQuery', () => {
describe('and query doesnt have an expression', () => {
const query: CloudWatchMetricsQuery = {
statistic: 'Average',
refId: 'A',
id: '',
region: '',
namespace: '',
period: '',
alias: '',
metricName: '',
dimensions: {},
matchExact: false,
expression: '',
};
migrateCloudWatchQuery(query);
it('should have basic metricEditorMode', () => {
expect(query.metricQueryType).toBe(MetricQueryType.Search);
});
it('should have Builder BasicEditorMode', () => {
expect(query.metricEditorMode).toBe(MetricEditorMode.Builder);
});
});
describe('and query has an expression', () => {
const query: CloudWatchMetricsQuery = {
statistic: 'Average',
refId: 'A',
id: '',
region: '',
namespace: '',
period: '',
alias: '',
metricName: '',
dimensions: {},
matchExact: false,
expression: 'SUM(x)',
};
migrateCloudWatchQuery(query);
migrateCloudWatchQuery(query);
it('should have basic metricEditorMode', () => {
expect(query.metricQueryType).toBe(MetricQueryType.Search);
});
it('should have Expression BasicEditorMode', () => {
expect(query.metricEditorMode).toBe(MetricEditorMode.Code);
});
});
});
});
});

20
public/app/plugins/datasource/cloudwatch/migrations.ts
View File

@ -1,7 +1,9 @@
import { AnnotationQuery, DataQuery } from '@grafana/data';
import { getNextRefIdChar } from 'app/core/utils/query';
import { CloudWatchAnnotationQuery, CloudWatchMetricsQuery } from './types';
import { MetricEditorMode, CloudWatchAnnotationQuery, CloudWatchMetricsQuery, MetricQueryType } from './types';
// Migrates a metric query that use more than one statistic into multiple queries
// E.g query.statistics = ['Max', 'Min'] will be migrated to two queries - query1.statistic = 'Max' and query2.statistic = 'Min'
export function migrateMultipleStatsMetricsQuery(
query: CloudWatchMetricsQuery,
panelQueries: DataQuery[]
@ -23,6 +25,8 @@ export function migrateMultipleStatsMetricsQuery(
return newQueries;
}
// Migrates an annotation query that use more than one statistic into multiple queries
// E.g query.statistics = ['Max', 'Min'] will be migrated to two queries - query1.statistic = 'Max' and query2.statistic = 'Min'
export function migrateMultipleStatsAnnotationQuery(
annotationQuery: CloudWatchAnnotationQuery
): Array<AnnotationQuery<DataQuery>> {
@ -43,3 +47,17 @@ export function migrateMultipleStatsAnnotationQuery(
return newAnnotations as Array<AnnotationQuery<DataQuery>>;
}
export function migrateCloudWatchQuery(query: CloudWatchMetricsQuery) {
if (!query.hasOwnProperty('metricQueryType')) {
query.metricQueryType = MetricQueryType.Search;
}
if (!query.hasOwnProperty('metricEditorMode')) {
if (query.metricQueryType === MetricQueryType.Query) {
query.metricEditorMode = MetricEditorMode.Code;
} else {
query.metricEditorMode = query.expression ? MetricEditorMode.Code : MetricEditorMode.Builder;
}
}
}

2
public/app/plugins/datasource/cloudwatch/module.tsx
View File

@ -5,6 +5,7 @@ import { CloudWatchAnnotationsQueryCtrl } from './annotations_query_ctrl';
import { CloudWatchJsonData, CloudWatchQuery } from './types';
import { CloudWatchLogsQueryEditor } from './components/LogsQueryEditor';
import { PanelQueryEditor } from './components/PanelQueryEditor';
import { MetaInspector } from './components/MetaInspector';
import LogsCheatSheet from './components/LogsCheatSheet';
export const plugin = new DataSourcePlugin<CloudWatchDatasource, CloudWatchQuery, CloudWatchJsonData>(
@ -13,6 +14,7 @@ export const plugin = new DataSourcePlugin<CloudWatchDatasource, CloudWatchQuery
.setQueryEditorHelp(LogsCheatSheet)
.setConfigEditor(ConfigEditor)
.setQueryEditor(PanelQueryEditor)
.setMetadataInspector(MetaInspector)
.setExploreMetricsQueryField(PanelQueryEditor)
.setExploreLogsQueryField(CloudWatchLogsQueryEditor)
.setAnnotationQueryCtrl(CloudWatchAnnotationsQueryCtrl);

29
public/app/plugins/datasource/cloudwatch/specs/datasource.test.ts
View File

@ -11,11 +11,13 @@ import * as redux from 'app/store/store';
import { CloudWatchDatasource, MAX_ATTEMPTS } from '../datasource';
import { TemplateSrv } from 'app/features/templating/template_srv';
import {
MetricEditorMode,
CloudWatchJsonData,
CloudWatchLogsQuery,
CloudWatchLogsQueryStatus,
CloudWatchMetricsQuery,
LogAction,
MetricQueryType,
} from '../types';
import { backendSrv } from 'app/core/services/backend_srv'; // will use the version in __mocks__
import { TimeSrv } from 'app/features/dashboard/services/TimeSrv';
@ -298,6 +300,8 @@ describe('CloudWatchDatasource', () => {
rangeRaw: { from: 1483228800, to: 1483232400 },
targets: [
{
metricQueryType: MetricQueryType.Search,
metricEditorMode: MetricEditorMode.Builder,
type: 'Metrics',
expression: '',
refId: 'A',
@ -378,6 +382,8 @@ describe('CloudWatchDatasource', () => {
rangeRaw: { from: 1483228800, to: 1483232400 },
targets: [
{
metricQueryType: MetricQueryType.Search,
metricEditorMode: MetricEditorMode.Builder,
type: 'Metrics',
refId: 'A',
region: 'us-east-1',
@ -411,6 +417,8 @@ describe('CloudWatchDatasource', () => {
describe('and throttling exception is thrown', () => {
const partialQuery = {
metricQueryType: MetricQueryType.Search,
metricEditorMode: MetricEditorMode.Builder,
type: 'Metrics',
namespace: 'AWS/EC2',
metricName: 'CPUUtilization',
@ -542,6 +550,8 @@ describe('CloudWatchDatasource', () => {
rangeRaw: { from: 1483228800, to: 1483232400 },
targets: [
{
metricQueryType: MetricQueryType.Search,
metricEditorMode: MetricEditorMode.Builder,
type: 'Metrics',
refId: 'A',
region: 'default',
@ -566,14 +576,14 @@ describe('CloudWatchDatasource', () => {
describe('When interpolating variables', () => {
it('should return an empty array if no queries are provided', () => {
const templateSrv: any = { replace: jest.fn() };
const templateSrv: any = { replace: jest.fn(), getVariables: () => [] };
const { ds } = getTestContext({ templateSrv });
expect(ds.interpolateVariablesInQueries([], {})).toHaveLength(0);
});
it('should replace correct variables in CloudWatchLogsQuery', () => {
const templateSrv: any = { replace: jest.fn() };
const templateSrv: any = { replace: jest.fn(), getVariables: () => [] };
const { ds } = getTestContext({ templateSrv });
const variableName = 'someVar';
const logQuery: CloudWatchLogsQuery = {
@ -592,7 +602,7 @@ describe('CloudWatchDatasource', () => {
});
it('should replace correct variables in CloudWatchMetricsQuery', () => {
const templateSrv: any = { replace: jest.fn() };
const templateSrv: any = { replace: jest.fn(), getVariables: () => [] };
const { ds } = getTestContext({ templateSrv });
const variableName = 'someVar';
const logQuery: CloudWatchMetricsQuery = {
@ -610,13 +620,14 @@ describe('CloudWatchDatasource', () => {
},
matchExact: false,
statistic: '',
sqlExpression: `$${variableName}`,
};
ds.interpolateVariablesInQueries([logQuery], {});
// We interpolate `expression`, `region`, `period`, `alias`, `metricName`, `nameSpace` and `dimensions` in CloudWatchMetricsQuery
expect(templateSrv.replace).toHaveBeenCalledWith(`$${variableName}`, {});
expect(templateSrv.replace).toHaveBeenCalledTimes(8);
expect(templateSrv.replace).toHaveBeenCalledTimes(9);
});
});
@ -626,6 +637,8 @@ describe('CloudWatchDatasource', () => {
rangeRaw: { from: 1483228800, to: 1483232400 },
targets: [
{
metricQueryType: MetricQueryType.Search,
metricEditorMode: MetricEditorMode.Builder,
type: 'Metrics',
refId: 'A',
region: 'us-east-1',
@ -753,6 +766,8 @@ describe('CloudWatchDatasource', () => {
rangeRaw: { from: 1483228800, to: 1483232400 },
targets: [
{
metricQueryType: MetricQueryType.Search,
metricEditorMode: MetricEditorMode.Builder,
type: 'Metrics',
refId: 'A',
region: 'us-east-1',
@ -779,6 +794,8 @@ describe('CloudWatchDatasource', () => {
rangeRaw: { from: 1483228800, to: 1483232400 },
targets: [
{
metricQueryType: MetricQueryType.Search,
metricEditorMode: MetricEditorMode.Builder,
type: 'Metrics',
refId: 'A',
region: 'us-east-1',
@ -813,6 +830,8 @@ describe('CloudWatchDatasource', () => {
rangeRaw: { from: 1483228800, to: 1483232400 },
targets: [
{
metricQueryType: MetricQueryType.Search,
metricEditorMode: MetricEditorMode.Builder,
type: 'Metrics',
refId: 'A',
region: 'us-east-1',
@ -843,6 +862,8 @@ describe('CloudWatchDatasource', () => {
rangeRaw: { from: 1483228800, to: 1483232400 },
targets: [
{
metricQueryType: MetricQueryType.Search,
metricEditorMode: MetricEditorMode.Builder,
type: 'Metrics',
refId: 'A',
region: 'us-east-1',

69
public/app/plugins/datasource/cloudwatch/types.ts
View File

@ -1,27 +1,72 @@
import { DataQuery, DataSourceRef, SelectableValue } from '@grafana/data';
import { AwsAuthDataSourceSecureJsonData, AwsAuthDataSourceJsonData } from '@grafana/aws-sdk';
export interface Dimensions {
[key: string]: string | string[];
}
import {
QueryEditorArrayExpression,
QueryEditorFunctionExpression,
QueryEditorPropertyExpression,
} from './expressions';
export type CloudWatchQueryMode = 'Metrics' | 'Logs';
export enum MetricQueryType {
'Search',
'Query',
}
export enum MetricEditorMode {
'Builder',
'Code',
}
export type Direction = 'ASC' | 'DESC';
export interface SQLExpression {
select?: QueryEditorFunctionExpression;
from?: QueryEditorPropertyExpression | QueryEditorFunctionExpression;
where?: QueryEditorArrayExpression;
groupBy?: QueryEditorArrayExpression;
orderBy?: QueryEditorFunctionExpression;
orderByDirection?: string;
limit?: number;
}
export interface CloudWatchMetricsQuery extends DataQuery {
queryMode?: 'Metrics';
metricQueryType?: MetricQueryType;
metricEditorMode?: MetricEditorMode;
//common props
id: string;
region: string;
namespace: string;
expression: string;
period?: string;
alias?: string;
metricName: string;
dimensions: { [key: string]: string | string[] };
statistic: string;
//Basic editor builder props
metricName?: string;
dimensions?: Dimensions;
matchExact?: boolean;
statistic?: string;
/**
* @deprecated use statistic
*/
statistics?: string[];
period: string;
alias: string;
matchExact: boolean;
// Math expression query
expression?: string;
sqlExpression?: string;
sql?: SQLExpression;
}
export interface CloudWatchMathExpressionQuery extends DataQuery {
expression: string;
}
export type LogAction =
@ -65,9 +110,7 @@ interface AnnotationProperties {
alarmNamePrefix: string;
}
export type CloudWatchLogsAnnotationQuery = CloudWatchLogsQuery & AnnotationProperties;
export type CloudWatchMetricsAnnotationQuery = CloudWatchMetricsQuery & AnnotationProperties;
export type CloudWatchAnnotationQuery = CloudWatchLogsAnnotationQuery | CloudWatchMetricsAnnotationQuery;
export type CloudWatchAnnotationQuery = CloudWatchMetricsQuery & AnnotationProperties;
export type SelectableStrings = Array<SelectableValue<string>>;
@ -325,12 +368,6 @@ export interface MetricQuery {
intervalMs?: number;
}
export interface ExecutedQueryPreview {
id: string;
executedQuery: string;
period: string;
}
export interface MetricFindSuggestData {
text: string;
label: string;

4
public/app/plugins/datasource/cloudwatch/utils/datalinks.ts
View File

@ -22,11 +22,11 @@ export async function addDataLinksToLogsResponse(
for (const dataFrame of response.data as DataFrame[]) {
const curTarget = request.targets.find((target) => target.refId === dataFrame.refId) as CloudWatchLogsQuery;
const interpolatedRegion = getRegion(replace(curTarget.region, 'region'));
const interpolatedRegion = getRegion(replace(curTarget.region ?? '', 'region'));
for (const field of dataFrame.fields) {
if (field.name === '@xrayTraceId' && tracingDatasourceUid) {
getRegion(replace(curTarget.region, 'region'));
getRegion(replace(curTarget.region ?? '', 'region'));
const xrayLink = await createInternalXrayLink(tracingDatasourceUid, interpolatedRegion);
if (xrayLink) {
field.config.links = [xrayLink];

9
public/app/plugins/datasource/cloudwatch/utils/utils.ts Normal file
View File

@ -0,0 +1,9 @@
import { SelectableValue } from '@grafana/data';
import { CloudWatchDatasource } from './../datasource';
export const toOption = (value: string) => ({ label: value, value });
export const appendTemplateVariables = (datasource: CloudWatchDatasource, values: SelectableValue[]) => [
...values,
{ label: 'Template Variables', options: datasource.getVariables().map(toOption) },
];