IntenseWebs/grafana
3
0
Fork 0
mirror of https://github.com/grafana/grafana.git synced 2024-11-26 02:40:26 -06:00
Code Issues Packages Projects Releases Wiki Activity

Cloudwatch: Docs improvements (#20100)

Browse Source 
* Add Service Quota part

* Update docs after pr feedback

* Describing new features

* Divide into subheaders

* Changes after feedback

* cloudwatch: add min time interval section to docs

* docs: json format fix

* cloudwatch: move min time interval section in docs

* cloudwatch: docs fix

* docs: cloudwatch fixes

* docs: s/you are/you're

* docs: Cloudwatch - escapes underscores in tables
This commit is contained in:
Erik Sundell 2019-11-22 14:51:35 +01:00 committed by Daniel Lee
parent ccc7f39b8a
commit de94298d3a
1 changed files with 192 additions and 121 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

313
docs/sources/features/datasources/cloudwatch.md
View File

@ -20,20 +20,39 @@ Grafana ships with built in support for CloudWatch. You just have to add it as a
1. Open the side menu by clicking the Grafana icon in the top header.
2. In the side menu under the `Dashboards` link you should find a link named `Data Sources`.
3. Click the `+ Add data source` button in the top header.
4. Select `Cloudwatch` from the *Type* dropdown.
4. Select `Cloudwatch` from the _Type_ dropdown.
> NOTE: If at any moment you have issues with getting this data source to work and Grafana is giving you undescriptive errors then don't
forget to check your log file (try looking in /var/log/grafana/grafana.log).
> forget to check your log file (try looking in /var/log/grafana/grafana.log).
Name | Description
------------ | -------------
*Name* | The data source name. This is how you refer to the data source in panels and queries.
*Default* | Default data source means that it will be pre-selected for new panels.
*Default Region* | Used in query editor to set region (can be changed on per query basis)
*Custom Metrics namespace* | Specify the CloudWatch namespace of Custom metrics
*Auth Provider* | Specify the provider to get credentials.
*Credentials* profile name | Specify the name of the profile to use (if you use `~/.aws/credentials` file), leave blank for default.
*Assume Role Arn* | Specify the ARN of the role to assume
| Name | Description |
| -------------------------- | ------------------------------------------------------------------------------------------------------- |
| _Name_ | The data source name. This is how you refer to the data source in panels and queries. |
| _Default_ | Default data source means that it will be pre-selected for new panels. |
| _Default Region_ | Used in query editor to set region (can be changed on per query basis) |
| _Custom Metrics namespace_ | Specify the CloudWatch namespace of Custom metrics |
| _Auth Provider_ | Specify the provider to get credentials. |
| _Credentials_ profile name | Specify the name of the profile to use (if you use `~/.aws/credentials` file), leave blank for default. |
| _Assume Role Arn_ | Specify the ARN of the role to assume |
### Min time interval
> Only available in Grafana v6.5+.
A lower limit for the auto group by time interval. Recommended to be set to write frequency, for example `1m` if your data is written every minute.
This option can also be overridden/configured in a dashboard panel under data source options. It's important to note that this value **needs** to be formatted as a
number followed by a valid time identifier, e.g. `1m` (1 minute) or `30s` (30 seconds). The following time identifiers are supported:
| Identifier | Description |
| ---------- | ----------- |
| `y` | year |
| `M` | month |
| `w` | week |
| `d` | day |
| `h` | hour |
| `m` | minute |
| `s` | second |
| `ms` | millisecond |
## Authentication
@ -44,8 +63,7 @@ server is running on AWS you can use IAM Roles and authentication will be handle
Checkout AWS docs on [IAM Roles](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html)
> NOTE: AWS Role Switching as described [here](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-cli.html) it not supported at the moment.
> NOTE: AWS Role Switching as described [here](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-cli.html) it not supported at the moment.
## IAM Policies
@ -55,42 +73,39 @@ utilize Grafana's built-in support for assuming roles.
Here is a minimal policy example:
```json
```bash
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AllowReadingMetricsFromCloudWatch",
"Effect": "Allow",
"Action": [
"cloudwatch:DescribeAlarmsForMetric",
"cloudwatch:ListMetrics",
"cloudwatch:GetMetricStatistics",
"cloudwatch:GetMetricData"
],
"Resource": "*"
},
{
"Sid": "AllowReadingTagsInstancesRegionsFromEC2",
"Effect": "Allow",
"Action": [
"ec2:DescribeTags",
"ec2:DescribeInstances",
"ec2:DescribeRegions"
],
"Resource": "*"
},
{
"Sid": "AllowReadingResourcesForTags",
"Effect" : "Allow",
"Action" : "tag:GetResources",
"Resource" : "*"
}
]
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AllowReadingMetricsFromCloudWatch",
"Effect": "Allow",
"Action": [
"cloudwatch:DescribeAlarmsForMetric",
"cloudwatch:ListMetrics",
"cloudwatch:GetMetricStatistics",
"cloudwatch:GetMetricData"
],
"Resource": "*"
},
{
"Sid": "AllowReadingTagsInstancesRegionsFromEC2",
"Effect": "Allow",
"Action": ["ec2:DescribeTags", "ec2:DescribeInstances", "ec2:DescribeRegions"],
"Resource": "*"
},
{
"Sid": "AllowReadingResourcesForTags",
"Effect": "Allow",
"Action": "tag:GetResources",
"Resource": "*"
}
]
}
```
### AWS credentials
If Auth Provider is `Credentials file`, Grafana try to get credentials by following order.
- Environment variables. (`AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`)
@ -103,7 +118,8 @@ Checkout AWS docs on [Configuring the AWS SDK for Go](https://docs.aws.amazon.co
### AWS credentials file
Create a file at `~/.aws/credentials`. That is the `HOME` path for user running grafana-server.
> NOTE: If you think you have the credentials file in the right place but it is still not working then you might try moving your .aws file to '/usr/share/grafana/' and make sure your credentials file has at most 0644 permissions.
> NOTE: If you think you have the credentials file in the right place but it is still not working then you might try moving your .aws file to '/usr/share/grafana/' and make sure your credentials file has at most 0644 permissions.
Example content:
@ -114,50 +130,94 @@ aws_secret_access_key = dasdasdsadasdasdasdsa
region = us-west-2
```
## Metric Query Editor
## Using the Metric Query Editor
![](/img/docs/v43/cloudwatch_editor.png)
To create a valid query, you need to specify the namespace, metric name and at least one statistic. If `Match Exact` is enabled, you also need to specify all the dimensions of the metric you’re querying, so that the [metric schema](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/search-expression-syntax.html) matches exactly. If `Match Exact` is off, you can specify any number of dimensions by which you’d like to filter. Up to 100 metrics matching your filter criteria will be returned.
You need to specify a namespace, metric, at least one stat, and at least one dimension.
### Dynamic queries using dimension wildcards
## Metric Math
> Only available in Grafana v6.5+.
You can now create new time series metrics by operating on top of Cloudwatch metrics using mathematical functions. Arithmetic operators, unary subtraction and other functions are supported to be applied on cloudwatch metrics. More details on the available functions can be found on [AWS Metric Math](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/using-metric-math.html)
In Grafana 6.5 or higher, you’re able to monitor a dynamic list of metrics by using the asterisk (\*) wildcard for one or more dimension values.
As an example, if you want to apply arithmetic operator on a metric, you can do it by giving an alias(a unique string) to the raw metric as shown below. Then you can use this alias and apply arithmetic operator to it in the Expression field of created metric.
{{< docs-imagebox img="/img/docs/v65/cloudwatch-dimension-wildcard.png" max-width="800px" class="docs-image--right" caption="CloudWatch dimension wildcard" >}}
![](/img/docs/v60/cloudwatch_metric_math.png)
In the example, all metrics in the namespace `AWS/EC2` with a metric name of `CPUUtilization` and ANY value for the `InstanceId` dimension are queried. This can help you monitor metrics for AWS resources, like EC2 instances or containers. For example, when new instances get created as part of an auto scaling event, they will automatically appear in the graph without you having to track the new instance IDs. This capability is currently limited to retrieving up to 100 metrics. You can click on `Show Query Preview` to see the search expression that is automatically built to support wildcards. To learn more about search expressions, visit the [CloudWatch documentation](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/search-expression-syntax.html).
By default, the search expression is defined in such a way that the queried metrics must match the defined dimension names exactly. This means that in the example only metrics with exactly one dimension with name ‘InstanceId’ will be returned.
You can untoggle `Match Exact` to include metrics that have other dimensions defined. Disabling `Match Exact` also creates a search expression even if you don’t use wildcards. We simply search for any metric that matches at least the namespace, metric name, and all defined dimensions.
### Multi-value template variables
> Only available in Grafana v6.5+.
When defining dimension values based on multi-valued template variables, a search expression is used to query for the matching metrics. This enables the use of multiple template variables in one query and also allows you to use template variables for queries that have the `Match Exact` option disabled.
Search expressions are currently limited to 1024 characters, so your query may fail if you have a long list of values. We recommend using the asterisk (`*`) wildcard instead of the `All` option if you want to query all metrics that have any value for a certain dimension name.
The use of multi-valued template variables is only supported for dimension values. Using multi-valued template variables for `Region`, `Namespace`, or `Metric Name` is not supported.
### Metric Math Expressions
You can create new time series metrics by operating on top of CloudWatch metrics using mathematical functions. Arithmetic operators, unary subtraction and other functions are supported and can be applied to CloudWatch metrics. More details on the available functions can be found on [AWS Metric Math](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/using-metric-math.html)
As an example, if you want to apply arithmetic operations on a metric, you can do it by giving an id (a unique string) to the raw metric as shown below. You can then use this id and apply arithmetic operations to it in the Expression field of the new metric.
Please note that in the case you use the expression field to reference another query, like `queryA * 2`, it will not be possible to create an alert rule based on that query.
### Deep linking from Grafana panels to the CloudWatch console
> Only available in Grafana v6.5+.
{{< docs-imagebox img="/img/docs/v65/cloudwatch-deep-linking.png" max-width="500px" class="docs-image--right" caption="CloudWatch deep linking" >}}
Left clicking a time series in the panel shows a context menu with a link to `View in CloudWatch console`. Clicking that link will open a new tab that will take you to the CloudWatch console and display all the metrics for that query. If you're not currently logged in to the CloudWatch console, the link will forward you to the login page. The provided link is valid for any account but will only display the right metrics if you're logged in to the account that corresponds to the selected data source in Grafana.
This feature is not available for metrics that are based on math expressions.
## Curated Dashboards
> Only available in Grafana v6.5+.
The updated CloudWatch data source ships with pre-configured dashboards for five of the most popular AWS services:
- Amazon Elastic Compute Cloud `Amazon EC2`,
- Amazon Elastic Block Store `Amazon EBS`,
- AWS Lambda `AWS Lambda`,
- Amazon CloudWatch Logs `Amazon CloudWatch Logs`, and
- Amazon Relational Database Service `Amazon RDS`.
To import the pre-configured dashboards, go to the configuration page of your CloudWatch data source and click on the `Dashboards` tab. Click `Import` for the dashboard you would like to use. To customize the dashboard, we recommend saving the dashboard under a different name, because otherwise the dashboard will be overwritten when a new version of the dashboard is released.
{{< docs-imagebox img="/img/docs/v65/cloudwatch-dashboard-import.png" caption="CloudWatch dashboard import" >}}
## Templated queries
Instead of hard-coding things like server, application and sensor name in you metric queries you can use variables in their place.
Variables are shown as dropdown select boxes at the top of the dashboard. These dropdowns makes it easy to change the data
being displayed in your dashboard.
Instead of hard-coding things like server, application and sensor name in you metric queries you can use variables in their place. Variables are shown as dropdown select boxes at the top of the dashboard. These dropdowns makes it easy to change the data being displayed in your dashboard.
Checkout the [Templating]({{< relref "../../reference/templating.md" >}}) documentation for an introduction to the templating feature and the different
types of template variables.
See the [Templating]({{< relref "../../reference/templating.md" >}}) documentation for an introduction to the templating feature and the different types of template variables.
### Query variable
CloudWatch data source plugin provides the following queries you can specify in the `Query` field in the Variable
edit view. They allow you to fill a variable's options list with things like `region`, `namespaces`, `metric names`
and `dimension keys/values`.
The CloudWatch data source provides the following queries that you can specify in the `Query` field in the Variable edit view. They allow you to fill a variable's options list with things like `region`, `namespaces`, `metric names` and `dimension keys/values`.
In place of `region` you can specify `default` to use the default region configured in the data source for the query,
e.g. `metrics(AWS/DynamoDB, default)` or `dimension_values(default, ..., ..., ...)`.
Read more about the available dimensions in the [CloudWatch Metrics and Dimensions Reference](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CW_Support_For_AWS.html).
Read more about the available dimensions in the [CloudWatch Metrics and Dimensions Reference](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CW_Support_For_AWS.html).
Name | Description
------- | --------
*regions()* | Returns a list of regions AWS provides their service.
*namespaces()* | Returns a list of namespaces CloudWatch support.
*metrics(namespace, [region])* | Returns a list of metrics in the namespace. (specify region or use "default" for custom metrics)
*dimension_keys(namespace)* | Returns a list of dimension keys in the namespace.
*dimension_values(region, namespace, metric, dimension_key, [filters])* | Returns a list of dimension values matching the specified `region`, `namespace`, `metric`, `dimension_key` or you can use dimension `filters` to get more specific result as well.
*ebs_volume_ids(region, instance_id)* | Returns a list of volume ids matching the specified `region`, `instance_id`.
*ec2_instance_attribute(region, attribute_name, filters)* | Returns a list of attributes matching the specified `region`, `attribute_name`, `filters`.
*resource_arns(region, resource_type, tags)* | Returns a list of ARNs matching the specified `region`, `resource_type` and `tags`.
| Name | Description |
| ----------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| _regions()_ | Returns a list of all AWS regions |
| _namespaces()_ | Returns a list of namespaces CloudWatch support. |
| _metrics(namespace, [region])_ | Returns a list of metrics in the namespace. (specify region or use "default" for custom metrics) |
| _dimension_\__keys(namespace)_ | Returns a list of dimension keys in the namespace. |
| _dimension_\__values(region, namespace, metric, dimension_\__key, [filters])_ | Returns a list of dimension values matching the specified `region`, `namespace`, `metric`, `dimension_key` or you can use dimension `filters` to get more specific result as well. |
| _ebs_\__volume_\__ids(region, instance_\__id)_ | Returns a list of volume ids matching the specified `region`, `instance_id`. |
| _ec2_\__instance_\__attribute(region, attribute_\__name, filters)_ | Returns a list of attributes matching the specified `region`, `attribute_name`, `filters`. |
| _resource_\__arns(region, resource_\__type, tags)_ | Returns a list of ARNs matching the specified `region`, `resource_type` and `tags`. |
| _statistics()_ | Returns a list of all the standard statistics |
For details about the metrics CloudWatch provides, please refer to the [CloudWatch documentation](https://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/CW_Support_For_AWS.html).
@ -165,16 +225,16 @@ For details about the metrics CloudWatch provides, please refer to the [CloudWat
Example dimension queries which will return list of resources for individual AWS Services:
Query | Service
------- | -----
*dimension_values(us-east-1,AWS/ELB,RequestCount,LoadBalancerName)* | ELB
*dimension_values(us-east-1,AWS/ElastiCache,CPUUtilization,CacheClusterId)* | ElastiCache
*dimension_values(us-east-1,AWS/Redshift,CPUUtilization,ClusterIdentifier)* | RedShift
*dimension_values(us-east-1,AWS/RDS,CPUUtilization,DBInstanceIdentifier)* | RDS
*dimension_values(us-east-1,AWS/S3,BucketSizeBytes,BucketName)* | S3
*dimension_values(us-east-1,CWAgent,disk_used_percent,device,{"InstanceId":"$instance_id"})* | CloudWatch Agent
*resource_arns(eu-west-1,elasticloadbalancing:loadbalancer,{"elasticbeanstalk:environment-name":["myApp-dev","myApp-prod"]})* | ELB
*resource_arns(eu-west-1,ec2:instance,{"elasticbeanstalk:environment-name":["myApp-dev","myApp-prod"]})* | EC2
| Query | Service |
| -------------------------------------------------------------------------------------------------------------------------------- | ---------------- |
| _dimension_\__values(us-east-1,AWS/ELB,RequestCount,LoadBalancerName)_ | ELB |
| _dimension_\__values(us-east-1,AWS/ElastiCache,CPUUtilization,CacheClusterId)_ | ElastiCache |
| _dimension_\__values(us-east-1,AWS/Redshift,CPUUtilization,ClusterIdentifier)_ | RedShift |
| _dimension_\__values(us-east-1,AWS/RDS,CPUUtilization,DBInstanceIdentifier)_ | RDS |
| _dimension_\__values(us-east-1,AWS/S3,BucketSizeBytes,BucketName)_ | S3 |
| _dimension_\__values(us-east-1,CWAgent,disk_\__used_\__percent,device,{"InstanceId":"\$instance_\__id"})_ | CloudWatch Agent |
| _resource_\__arns(eu-west-1,elasticloadbalancing:loadbalancer,{"elasticbeanstalk:environment-name":["myApp-dev","myApp-prod"]})_ | ELB |
| _resource_\__arns(eu-west-1,ec2:instance,{"elasticbeanstalk:environment-name":["myApp-dev","myApp-prod"]})_ | EC2 |
## ec2_instance_attribute examples
@ -193,53 +253,53 @@ Filters syntax:
Example `ec2_instance_attribute()` query
```javascript
ec2_instance_attribute(us-east-1, InstanceId, { "tag:Environment": [ "production" ] })
ec2_instance_attribute(us - east - 1, InstanceId, { 'tag:Environment': ['production'] });
```
### Selecting Attributes
Only 1 attribute per instance can be returned. Any flat attribute can be selected (i.e. if the attribute has a single value and isn't an object or array). Below is a list of available flat attributes:
* `AmiLaunchIndex`
* `Architecture`
* `ClientToken`
* `EbsOptimized`
* `EnaSupport`
* `Hypervisor`
* `IamInstanceProfile`
* `ImageId`
* `InstanceId`
* `InstanceLifecycle`
* `InstanceType`
* `KernelId`
* `KeyName`
* `LaunchTime`
* `Platform`
* `PrivateDnsName`
* `PrivateIpAddress`
* `PublicDnsName`
* `PublicIpAddress`
* `RamdiskId`
* `RootDeviceName`
* `RootDeviceType`
* `SourceDestCheck`
* `SpotInstanceRequestId`
* `SriovNetSupport`
* `SubnetId`
* `VirtualizationType`
* `VpcId`
- `AmiLaunchIndex`
- `Architecture`
- `ClientToken`
- `EbsOptimized`
- `EnaSupport`
- `Hypervisor`
- `IamInstanceProfile`
- `ImageId`
- `InstanceId`
- `InstanceLifecycle`
- `InstanceType`
- `KernelId`
- `KeyName`
- `LaunchTime`
- `Platform`
- `PrivateDnsName`
- `PrivateIpAddress`
- `PublicDnsName`
- `PublicIpAddress`
- `RamdiskId`
- `RootDeviceName`
- `RootDeviceType`
- `SourceDestCheck`
- `SpotInstanceRequestId`
- `SriovNetSupport`
- `SubnetId`
- `VirtualizationType`
- `VpcId`
Tags can be selected by prepending the tag name with `Tags.`
Example `ec2_instance_attribute()` query
```javascript
ec2_instance_attribute(us-east-1, Tags.Name, { "tag:Team": [ "sysops" ] })
ec2_instance_attribute(us - east - 1, Tags.Name, { 'tag:Team': ['sysops'] });
```
## Using json format template variables
Some of query takes JSON format filter. Grafana support to interpolate template variable to JSON format string, it can use as filter string.
Some queries accept filters in JSON format and Grafana supports the conversion of template variables to JSON.
If `env = 'production', 'staging'`, following query will return ARNs of EC2 instances which `Environment` tag is `production` or `staging`.
@ -247,12 +307,22 @@ If `env = 'production', 'staging'`, following query will return ARNs of EC2 inst
resource_arns(us-east-1, ec2:instance, {"Environment":${env:json}})
```
## Cost
## Pricing
Amazon provides 1 million CloudWatch API requests each month at no additional charge. Past this,
it costs $0.01 per 1,000 GetMetricStatistics or ListMetrics requests. For each query Grafana will
issue a GetMetricStatistics request and every time you pick a dimension in the query editor
Grafana will issue a ListMetrics request.
The Amazon CloudWatch data source for Grafana uses the `ListMetrics` and `GetMetricData` CloudWatch API calls to list and retrieve metrics. Please see the [CloudWatch pricing page](https://aws.amazon.com/cloudwatch/pricing/) for pricing information about these API calls.
Every time you pick a dimension in the query editor Grafana will issue a ListMetrics request.
Whenever you make a change to the queries in the query editor, one new request to GetMetricData will be issued.
Please note that for Grafana version 6.5 or higher, all API requests to GetMetricStatistics have been replaced with calls to GetMetricData. This change enables better support for CloudWatch metric math and enables the use of search expressions. While GetMetricStatistics qualified for the CloudWatch API free tier, this is not the case for GetMetricData calls. For more information, please refer to the [CloudWatch pricing page](https://aws.amazon.com/cloudwatch/pricing/).
## Service Quotas
AWS defines quotas, or limits, for resources, actions, and items in your AWS account. Depending on the number of queries in your dashboard and the amount of users accessing the dashboard, you may reach the limit for the allowed number of CloudWatch GetMetricData requests per second. Note that quotas are defined per account and per region. If you're using multiple regions or have set up more than one CloudWatch data source to query against multiple accounts, you need to request a quota increase for each account and each region in which you hit the limit.
To request a quota increase, visit the [AWS Service Quotas console](https://console.aws.amazon.com/servicequotas/home?r#!/services/monitoring/quotas/L-5E141212).
Please see the AWS documentation for [Service Quotas](https://docs.aws.amazon.com/servicequotas/latest/userguide/intro.html) and [CloudWatch limits](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/cloudwatch_limits.html) for more information.
## Configure the data source with provisioning
@ -260,7 +330,8 @@ It's now possible to configure data sources using config files with Grafana's pr
Here are some provisioning examples for this data source.
Using a credentials file
### Using a credentials file
```yaml
apiVersion: 1
@ -272,7 +343,7 @@ datasources:
defaultRegion: eu-west-2
```
Using `accessKey` and `secretKey`
### Using `accessKey` and `secretKey`
```yaml
apiVersion: 1
@ -284,6 +355,6 @@ datasources:
authType: keys
defaultRegion: eu-west-2
secureJsonData:
accessKey: "<your access key>"
secretKey: "<your secret key>"
accessKey: '<your access key>'
secretKey: '<your secret key>'
```