grafana/docs/sources/administration/provisioning.md

573 lines
28 KiB
Markdown
Raw Normal View History

+++
title = "Provisioning"
description = ""
keywords = ["grafana", "provisioning"]
aliases = ["/docs/grafana/latest/installation/provisioning"]
weight = 8
+++
# Provisioning Grafana
In previous versions of Grafana, you could only use the API for provisioning data sources and dashboards. But that required the service to be running before you started creating dashboards and you also needed to set up credentials for the HTTP API. In v5.0 we decided to improve this experience by adding a new active provisioning system that uses config files. This will make GitOps more natural as data sources and dashboards can be defined via files that can be version controlled. We hope to extend this system to later add support for users, orgs and alerts as well.
2018-03-23 04:50:34 -05:00
## Config File
Check out the [configuration]({{< relref "configuration.md" >}}) page for more information on what you can configure in `grafana.ini`
2018-03-23 04:50:34 -05:00
### Config File Locations
- Default configuration from `$WORKING_DIR/conf/defaults.ini`
- Custom configuration from `$WORKING_DIR/conf/custom.ini`
- The custom configuration file path can be overridden using the `--config` parameter
> **Note:** If you have installed Grafana using the `deb` or `rpm`
> packages, then your configuration file is located at
> `/etc/grafana/grafana.ini`. This path is specified in the Grafana
> init.d script using `--config` file parameter.
2018-03-23 04:50:34 -05:00
### Using Environment Variables
It is possible to use environment variable interpolation in all 3 provisioning configuration types. Allowed syntax
is either `$ENV_VAR_NAME` or `${ENV_VAR_NAME}` and can be used only for values not for keys or bigger parts
of the configurations. It is not available in the dashboard's definition files just the dashboard provisioning
configuration.
Example:
```yaml
datasources:
- name: Graphite
url: http://localhost:$PORT
user: $USER
secureJsonData:
password: $PASSWORD
```
If you have a literal `$` in your value and want to avoid interpolation, `$$` can be used.
<hr />
2018-03-23 04:50:34 -05:00
## Configuration Management Tools
2020-05-02 12:58:06 -05:00
Currently we do not provide any scripts/manifests for configuring Grafana. Rather than spending time learning and creating scripts/manifests for each tool, we think our time is better spent making Grafana easier to provision. Therefore, we heavily rely on the expertise of the community.
| Tool | Project |
| --------- | -------------------------------------------------------------------------------------------------------------- |
| Puppet | [https://forge.puppet.com/puppet/grafana](https://forge.puppet.com/puppet/grafana) |
| Ansible | [https://github.com/cloudalchemy/ansible-grafana](https://github.com/cloudalchemy/ansible-grafana) |
| Chef | [https://github.com/JonathanTron/chef-grafana](https://github.com/JonathanTron/chef-grafana) |
| Saltstack | [https://github.com/salt-formulas/salt-formula-grafana](https://github.com/salt-formulas/salt-formula-grafana) |
| Jsonnet | [https://github.com/grafana/grafonnet-lib/](https://github.com/grafana/grafonnet-lib/) |
## Data sources
> This feature is available from v5.0
It's possible to manage data sources in Grafana by adding one or more YAML config files in the [`provisioning/datasources`](/administration/configuration/#provisioning) directory. Each config file can contain a list of `datasources` that will get added or updated during start up. If the data source already exists, then Grafana updates it to match the configuration file. The config file can also contain a list of data sources that should be deleted. That list is called `deleteDatasources`. Grafana will delete data sources listed in `deleteDatasources` before inserting/updating those in the `datasource` list.
2018-03-23 04:50:34 -05:00
### Running Multiple Grafana Instances
2018-03-23 03:04:50 -05:00
2018-01-14 00:57:48 -06:00
If you are running multiple instances of Grafana you might run into problems if they have different versions of the `datasource.yaml` configuration file. The best way to solve this problem is to add a version number to each datasource in the configuration and increase it when you update the config. Grafana will only update datasources with the same or lower version number than specified in the config. That way, old configs cannot overwrite newer configs if they restart at the same time.
### Example data source Config File
2018-03-23 03:04:50 -05:00
```yaml
# config file version
apiVersion: 1
# list of datasources that should be deleted from the database
deleteDatasources:
- name: Graphite
orgId: 1
2018-01-14 00:57:48 -06:00
# list of datasources to insert/update depending
2018-05-18 04:49:00 -05:00
# what's available in the database
datasources:
# <string, required> name of the datasource. Required
- name: Graphite
# <string, required> datasource type. Required
type: graphite
# <string, required> access mode. proxy or direct (Server or Browser in the UI). Required
access: proxy
# <int> org id. will default to orgId 1 if not specified
orgId: 1
# <string> custom UID which can be used to reference this datasource in other parts of the configuration, if not specified will be generated automatically
uid: my_unique_uid
# <string> url
url: http://localhost:8080
# <string> Deprecated, use secureJsonData.password
password:
# <string> database user, if used
user:
# <string> database name, if used
database:
# <bool> enable/disable basic auth
basicAuth:
# <string> basic auth username
basicAuthUser:
# <string> Deprecated, use secureJsonData.basicAuthPassword
basicAuthPassword:
# <bool> enable/disable with credentials headers
withCredentials:
# <bool> mark as default datasource. Max one per org
isDefault:
# <map> fields that will be converted to json and stored in jsonData
jsonData:
graphiteVersion: '1.1'
tlsAuth: true
tlsAuthWithCACert: true
# <string> json object of data that will be encrypted.
secureJsonData:
tlsCACert: '...'
tlsClientCert: '...'
tlsClientKey: '...'
# <string> database password, if used
password:
# <string> basic auth password
basicAuthPassword:
version: 1
# <bool> allow users to edit datasources from the UI.
editable: false
```
2018-03-23 04:50:34 -05:00
#### Custom Settings per Datasource
Please refer to each datasource documentation for specific provisioning examples.
| Datasource | Misc |
| ------------- | ---------------------------------------------------------------------------------- |
2018-03-23 03:04:50 -05:00
| Elasticsearch | Elasticsearch uses the `database` property to configure the index for a datasource |
CloudWatch: Re-implement authentication (#25548) * CloudWatch: Revisit authentication Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * CloudWatch: Simplify auth code Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Use ARN Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Add Drone configuration Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Remove unused code Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Remove .drone.yml Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix external ID usage Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * CloudWatch: Fix issues after merge Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Remove stale code Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Remove stale code Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Use auth type enum Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix test snapshot * Coordinate frontend and backend option names Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Remove old comments Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix front-end tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Introduce session cache Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Use constants Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix field alignment * CloudWatch: Fix log message Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Tidy go.mod Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * CloudWatch: Handle arn auth type Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * CloudWatch: Fix role assumption duration Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix test Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * CloudWatch: Inline unnecessary constants Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * CloudWatch: Use serial comma in UI Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * CloudWatch: Inline unnecessary constants Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * CloudWatch: Fail if missing region Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * CloudWatch: Handle unconfigured region Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * CloudWatch: Log when using cached session Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * CloudWatch: Include region in cache key Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Add UI warnings for lecagy support * Do not clear ARN fields whenging change authentication provider * Graph NG: annotations display (#27972) * Annotations support POC * Fix markers memoization * dev dashboard update * Update public/app/plugins/panel/graph3/plugins/AnnotationsPlugin.tsx * CloudWatch: Remove errors.BadRequest Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * CloudWatch: Undo unintentional change Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Remove log line Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix cache key computation Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Add region to cache key Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Improve log messages Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * CloudWatch: Add documentation Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Improve tooltip Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Improve docs Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Improve docs Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Improve docs Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Improve tooltip Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Add role assumption provisioning example Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Add upgrade notes Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Improve docs Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Apply suggestions from code review Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * backend: use latest sdk (#28147) fixes #27713 via https://github.com/grafana/grafana-plugin-sdk-go/pull/227 * Docs: Update Permissions documentation (#28144) * removed overview.md * content updates * Update datasource_permissions.md * update content * content updates * Update organization_roles.md * Update docs/sources/enterprise/saml.md Co-authored-by: Kyle Brandt <kyle@grafana.com> * Update dashboard_folder_permissions.md Co-authored-by: Kyle Brandt <kyle@grafana.com> * area/grafana/toolkit: ci-package needs to use synchronous writes (#28148) * ci needs to use synchronous writes or the file ends up with zero length * <Enterprise Docs> Add instructions to upload license via UI (#28067) * Add UI license upload option, reformat Enterprise license activation section Added the option to upload a license file through the Server Admin UI, and did a little reformatting to make license activation look more like a process. * Headers not bold, hyphens not asterisks * Github: run metrics collector workflow every 10min (#28153) * GithubActions: Updated cron schedule * Updated * Docs: Update explore docs: remove dot at the end of line (#28151) HI - Removed Dot(.) at the end of line to make it consistent with other 2 points. Thanks, Ashish * Fix frontend tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix frontend tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Docs: Update upgrade notes Co-authored-by: Sofia Papagiannaki <sofia@grafana.com> Co-authored-by: Dominik Prokop <dominik.prokop@grafana.com> Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> Co-authored-by: Kyle Brandt <kyle@grafana.com> Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> Co-authored-by: Brian Gann <briangann@users.noreply.github.com> Co-authored-by: Mitch Seaman <mjseaman@users.noreply.github.com> Co-authored-by: Torkel Ödegaard <torkel@grafana.org> Co-authored-by: Torkel Ödegaard <torkel@grafana.com> Co-authored-by: ashishagarwal06 <34888589+ashishagarwal06@users.noreply.github.com>
2020-10-12 10:58:58 -05:00
#### JSON Data
2018-10-09 05:24:23 -05:00
Since not all datasources have the same configuration settings we only have the most common ones as fields. The rest should be stored as a json blob in the `jsonData` field. Here are the most common settings that the core datasources use.
| Name | Type | Datasource | Description |
| ----------------------- | ------- | ---------------------------------------------------------------- | ------------------------------------------------------------------------------------------- |
| tlsAuth | boolean | _All_ | Enable TLS authentication using client cert configured in secure json data |
| tlsAuthWithCACert | boolean | _All_ | Enable TLS authentication using CA cert |
| tlsSkipVerify | boolean | _All_ | Controls whether a client verifies the server's certificate chain and host name. |
DataSourceSettings: Add servername field to DataSource TLS config (#29279) * DataSourceSettings: Add servername field to DataSource TLS config A DNS lookup URL can be provided in the DataSource URL field in order to dynamically load balance between multiple instances of a DataSource. When using mutual TLS, Golang's TLS config implementation checks that the certificate's common name (< 1.15) or subject alternative name (>= 1.15) has the same value as the domain being accessed. If the DNS entry is dynamically generated for a specific environment, the certificate cannot be generated with a name matching the dynamic DNS URL. As such, Golang offers a servername field that can be set to overwrite what value is used when checking against the certificate's common name (or subject alternative name). Without this change, Skip TLS Verify must be set to true in order for the DataSource to work, removing some of the benefits gained by using mutual TLS. This commit adds the ability to set Grafana's internal Golang TLS config servername field from the UI or a provisioned DataSource. The servername field is optional and the existing behavior is retained if the field is not set. Co-authored-by: Dana Pruitt <dpruitt@vmware.com> Co-authored-by: Jeremy Alvis <jalvis@pivotal.io> * Update docs with PR review changes Co-authored-by: Jeremy Alvis <jalvis@pivotal.io> Co-authored-by: Dana Pruitt <dpruitt@vmware.com> * Update with additional PR requested changes * Minor updates based on PR change requests Co-authored-by: Dana Pruitt <dpruitt@vmware.com>
2020-12-10 09:07:05 -06:00
| serverName | string | _All_ | Optional. Controls the server name used for certificate common name/subject alternative name verification. Defaults to using the data source URL. |
| graphiteVersion | string | Graphite | Graphite version |
| timeInterval | string | Prometheus, Elasticsearch, InfluxDB, MySQL, PostgreSQL and MSSQL | Lowest interval/step value that should be used for this data source. |
| httpMode | string | Influxdb | HTTP Method. 'GET', 'POST', defaults to GET |
| httpMethod | string | Prometheus | HTTP Method. 'GET', 'POST', defaults to GET |
| esVersion | number | Elasticsearch | Elasticsearch version as a number (2/5/56/60/70) |
| timeField | string | Elasticsearch | Which field that should be used as timestamp |
| interval | string | Elasticsearch | Index date time format. nil(No Pattern), 'Hourly', 'Daily', 'Weekly', 'Monthly' or 'Yearly' |
| logMessageField | string | Elasticsearch | Which field should be used as the log message |
| logLevelField | string | Elasticsearch | Which field should be used to indicate the priority of the log message |
| sigV4Auth | boolean | Elasticsearch | Enable usage of SigV4 |
| sigV4AuthType | string | Elasticsearch | SigV4 auth provider. default/credentials/keys |
| sigV4ExternalId | string | Elasticsearch | Optional SigV4 External ID |
| sigV4AssumeRoleArn | string | Elasticsearch | Optional SigV4 ARN role to assume |
| sigV4Region | string | Elasticsearch | SigV4 AWS region |
| sigV4Profile | string | Elasticsearch | Optional SigV4 credentials profile |
CloudWatch: Re-implement authentication (#25548) * CloudWatch: Revisit authentication Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * CloudWatch: Simplify auth code Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Use ARN Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Add Drone configuration Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Remove unused code Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Remove .drone.yml Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix external ID usage Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * CloudWatch: Fix issues after merge Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Remove stale code Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Remove stale code Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Use auth type enum Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix test snapshot * Coordinate frontend and backend option names Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Remove old comments Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix front-end tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Introduce session cache Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Use constants Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix field alignment * CloudWatch: Fix log message Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Tidy go.mod Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * CloudWatch: Handle arn auth type Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * CloudWatch: Fix role assumption duration Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix test Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * CloudWatch: Inline unnecessary constants Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * CloudWatch: Use serial comma in UI Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * CloudWatch: Inline unnecessary constants Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * CloudWatch: Fail if missing region Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * CloudWatch: Handle unconfigured region Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * CloudWatch: Log when using cached session Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * CloudWatch: Include region in cache key Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Add UI warnings for lecagy support * Do not clear ARN fields whenging change authentication provider * Graph NG: annotations display (#27972) * Annotations support POC * Fix markers memoization * dev dashboard update * Update public/app/plugins/panel/graph3/plugins/AnnotationsPlugin.tsx * CloudWatch: Remove errors.BadRequest Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * CloudWatch: Undo unintentional change Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Remove log line Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix cache key computation Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Add region to cache key Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Improve log messages Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * CloudWatch: Add documentation Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Improve tooltip Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Improve docs Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Improve docs Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Improve docs Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Improve tooltip Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Add role assumption provisioning example Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Add upgrade notes Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Improve docs Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Apply suggestions from code review Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * backend: use latest sdk (#28147) fixes #27713 via https://github.com/grafana/grafana-plugin-sdk-go/pull/227 * Docs: Update Permissions documentation (#28144) * removed overview.md * content updates * Update datasource_permissions.md * update content * content updates * Update organization_roles.md * Update docs/sources/enterprise/saml.md Co-authored-by: Kyle Brandt <kyle@grafana.com> * Update dashboard_folder_permissions.md Co-authored-by: Kyle Brandt <kyle@grafana.com> * area/grafana/toolkit: ci-package needs to use synchronous writes (#28148) * ci needs to use synchronous writes or the file ends up with zero length * <Enterprise Docs> Add instructions to upload license via UI (#28067) * Add UI license upload option, reformat Enterprise license activation section Added the option to upload a license file through the Server Admin UI, and did a little reformatting to make license activation look more like a process. * Headers not bold, hyphens not asterisks * Github: run metrics collector workflow every 10min (#28153) * GithubActions: Updated cron schedule * Updated * Docs: Update explore docs: remove dot at the end of line (#28151) HI - Removed Dot(.) at the end of line to make it consistent with other 2 points. Thanks, Ashish * Fix frontend tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix frontend tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Docs: Update upgrade notes Co-authored-by: Sofia Papagiannaki <sofia@grafana.com> Co-authored-by: Dominik Prokop <dominik.prokop@grafana.com> Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> Co-authored-by: Kyle Brandt <kyle@grafana.com> Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> Co-authored-by: Brian Gann <briangann@users.noreply.github.com> Co-authored-by: Mitch Seaman <mjseaman@users.noreply.github.com> Co-authored-by: Torkel Ödegaard <torkel@grafana.org> Co-authored-by: Torkel Ödegaard <torkel@grafana.com> Co-authored-by: ashishagarwal06 <34888589+ashishagarwal06@users.noreply.github.com>
2020-10-12 10:58:58 -05:00
| authType | string | Cloudwatch | Auth provider. default/credentials/keys |
| externalId | string | Cloudwatch | Optional External ID |
| assumeRoleArn | string | Cloudwatch | Optional ARN role to assume |
| defaultRegion | string | Cloudwatch | Optional default AWS region |
| customMetricsNamespaces | string | Cloudwatch | Namespaces of Custom Metrics |
CloudWatch: Re-implement authentication (#25548) * CloudWatch: Revisit authentication Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * CloudWatch: Simplify auth code Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Use ARN Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Add Drone configuration Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Remove unused code Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Remove .drone.yml Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix external ID usage Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * CloudWatch: Fix issues after merge Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Remove stale code Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Remove stale code Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Use auth type enum Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix test snapshot * Coordinate frontend and backend option names Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Remove old comments Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix front-end tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Introduce session cache Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Use constants Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix field alignment * CloudWatch: Fix log message Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Tidy go.mod Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * CloudWatch: Handle arn auth type Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * CloudWatch: Fix role assumption duration Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix test Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * CloudWatch: Inline unnecessary constants Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * CloudWatch: Use serial comma in UI Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * CloudWatch: Inline unnecessary constants Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * CloudWatch: Fail if missing region Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * CloudWatch: Handle unconfigured region Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * CloudWatch: Log when using cached session Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * CloudWatch: Include region in cache key Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Add UI warnings for lecagy support * Do not clear ARN fields whenging change authentication provider * Graph NG: annotations display (#27972) * Annotations support POC * Fix markers memoization * dev dashboard update * Update public/app/plugins/panel/graph3/plugins/AnnotationsPlugin.tsx * CloudWatch: Remove errors.BadRequest Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * CloudWatch: Undo unintentional change Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Remove log line Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix cache key computation Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Add region to cache key Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Improve log messages Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * CloudWatch: Add documentation Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Improve tooltip Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Improve docs Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Improve docs Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Improve docs Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Improve tooltip Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Add role assumption provisioning example Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Add upgrade notes Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Improve docs Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Apply suggestions from code review Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * backend: use latest sdk (#28147) fixes #27713 via https://github.com/grafana/grafana-plugin-sdk-go/pull/227 * Docs: Update Permissions documentation (#28144) * removed overview.md * content updates * Update datasource_permissions.md * update content * content updates * Update organization_roles.md * Update docs/sources/enterprise/saml.md Co-authored-by: Kyle Brandt <kyle@grafana.com> * Update dashboard_folder_permissions.md Co-authored-by: Kyle Brandt <kyle@grafana.com> * area/grafana/toolkit: ci-package needs to use synchronous writes (#28148) * ci needs to use synchronous writes or the file ends up with zero length * <Enterprise Docs> Add instructions to upload license via UI (#28067) * Add UI license upload option, reformat Enterprise license activation section Added the option to upload a license file through the Server Admin UI, and did a little reformatting to make license activation look more like a process. * Headers not bold, hyphens not asterisks * Github: run metrics collector workflow every 10min (#28153) * GithubActions: Updated cron schedule * Updated * Docs: Update explore docs: remove dot at the end of line (#28151) HI - Removed Dot(.) at the end of line to make it consistent with other 2 points. Thanks, Ashish * Fix frontend tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix frontend tests Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Docs: Update upgrade notes Co-authored-by: Sofia Papagiannaki <sofia@grafana.com> Co-authored-by: Dominik Prokop <dominik.prokop@grafana.com> Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> Co-authored-by: Kyle Brandt <kyle@grafana.com> Co-authored-by: Diana Payton <52059945+oddlittlebird@users.noreply.github.com> Co-authored-by: Brian Gann <briangann@users.noreply.github.com> Co-authored-by: Mitch Seaman <mjseaman@users.noreply.github.com> Co-authored-by: Torkel Ödegaard <torkel@grafana.org> Co-authored-by: Torkel Ödegaard <torkel@grafana.com> Co-authored-by: ashishagarwal06 <34888589+ashishagarwal06@users.noreply.github.com>
2020-10-12 10:58:58 -05:00
| profile | string | Cloudwatch | Optional credentials profile |
| tsdbVersion | string | OpenTSDB | Version |
| tsdbResolution | string | OpenTSDB | Resolution |
| sslmode | string | PostgreSQL | SSLmode. 'disable', 'require', 'verify-ca' or 'verify-full' |
| sslRootCertFile | string | PostgreSQL | SSL server root certificate file, must be readable by the Grafana user |
| sslCertFile | string | PostgreSQL | SSL client certificate file, must be readable by the Grafana user |
| sslKeyFile | string | PostgreSQL | SSL client key file, must be readable by _only_ the Grafana user |
| encrypt | string | MSSQL | Connection SSL encryption handling. 'disable', 'false' or 'true' |
| postgresVersion | number | PostgreSQL | Postgres version as a number (903/904/905/906/1000) meaning v9.3, v9.4, ..., v10 |
| timescaledb | boolean | PostgreSQL | Enable usage of TimescaleDB extension |
| maxOpenConns | number | MySQL, PostgreSQL and MSSQL | Maximum number of open connections to the database (Grafana v5.4+) |
| maxIdleConns | number | MySQL, PostgreSQL and MSSQL | Maximum number of connections in the idle connection pool (Grafana v5.4+) |
| connMaxLifetime | number | MySQL, PostgreSQL and MSSQL | Maximum amount of time in seconds a connection may be reused (Grafana v5.4+) |
2018-03-23 04:50:34 -05:00
#### Secure Json Data
`{"authType":"keys","defaultRegion":"us-west-2","timeField":"@timestamp"}`
Secure json data is a map of settings that will be encrypted with [secret key]({{< relref "configuration.md#secret-key" >}}) from the Grafana config. The purpose of this is only to hide content from the users of the application. This should be used for storing TLS Cert and password that Grafana will append to the request on the server side. All of these settings are optional.
| Name | Type | Datasource | Description |
| ----------------- | ------ | ---------- | --------------------------------------- |
| tlsCACert | string | _All_ | CA cert for out going requests |
| tlsClientCert | string | _All_ | TLS Client cert for outgoing requests |
| tlsClientKey | string | _All_ | TLS Client key for outgoing requests |
| password | string | _All_ | password |
| basicAuthPassword | string | _All_ | password for basic authentication |
| accessKey | string | Cloudwatch | Access key for connecting to Cloudwatch |
| secretKey | string | Cloudwatch | Secret key for connecting to Cloudwatch |
| sigV4AccessKey | string | Elasticsearch | SigV4 access key. Required when using keys auth provider |
| sigV4SecretKey | string | Elasticsearch | SigV4 secret key. Required when using keys auth provider |
#### Custom HTTP headers for datasources
Data sources managed by Grafanas provisioning can be configured to add HTTP headers to all requests
going to that datasource. The header name is configured in the `jsonData` field and the header value should be
configured in `secureJsonData`.
```yaml
apiVersion: 1
datasources:
- name: Graphite
jsonData:
httpHeaderName1: 'HeaderName'
httpHeaderName2: 'Authorization'
secureJsonData:
httpHeaderValue1: 'HeaderValue'
httpHeaderValue2: 'Bearer XXXXXXXXX'
```
## Plugins
> This feature is available from v7.1
You can manage plugins in Grafana by adding one or more YAML config files in the [`provisioning/plugins`]({{< relref "configuration.md#provisioning" >}}) directory. Each config file can contain a list of `apps` that will be updated during start up. Grafana updates each app to match the configuration file.
### Example plugin configuration file
```yaml
apiVersion: 1
apps:
# <string> the type of app, plugin identifier. Required
- type: raintank-worldping-app
# <int> Org ID. Default to 1, unless org_name is specified
org_id: 1
# <string> Org name. Overrides org_id unless org_id not specified
org_name: Main Org.
# <bool> disable the app. Default to false.
disabled: false
# <map> fields that will be converted to json and stored in jsonData. Custom per app.
jsonData:
# key/value pairs of string to object
key: value
# <map> fields that will be converted to json, encrypted and stored in secureJsonData. Custom per app.
secureJsonData:
# key/value pairs of string to string
key: value
```
## Dashboards
You can manage dashboards in Grafana by adding one or more YAML config files in the [`provisioning/dashboards`]({{< relref "configuration.md" >}}) directory. Each config file can contain a list of `dashboards providers` that load dashboards into Grafana from the local filesystem.
2018-01-14 00:57:48 -06:00
The dashboard provider config file looks somewhat like this:
```yaml
apiVersion: 1
providers:
# <string> an unique provider name. Required
- name: 'a unique provider name'
# <int> Org id. Default to 1
orgId: 1
# <string> name of the dashboard folder.
folder: ''
# <string> folder UID. will be automatically generated if not specified
folderUid: ''
# <string> provider type. Default to 'file'
type: file
# <bool> disable dashboard deletion
disableDeletion: false
# <int> how often Grafana will scan for changed dashboards
updateIntervalSeconds: 10
# <bool> allow updating provisioned dashboards from the UI
allowUiUpdates: false
options:
# <string, required> path to dashboard files on disk. Required when using the 'file' type
path: /var/lib/grafana/dashboards
# <bool> use folder names from filesystem to create folders in Grafana
foldersFromFilesStructure: true
```
2019-03-22 05:49:27 -05:00
When Grafana starts, it will update/insert all dashboards available in the configured path. Then later on poll that path every **updateIntervalSeconds** and look for updated json files and update/insert those into the database.
> **Note:** Dashboards are provisioned to the General folder if the `folder` option is missing or empty.
#### Making changes to a provisioned dashboard
It's possible to make changes to a provisioned dashboard in the Grafana UI. However, it is not possible to automatically save the changes back to the provisioning source.
If `allowUiUpdates` is set to `true` and you make changes to a provisioned dashboard, you can `Save` the dashboard then changes will be persisted to the Grafana database.
> **Note:**
> If a provisioned dashboard is saved from the UI and then later updated from the source, the dashboard stored in the database will always be overwritten. The `version` property in the JSON file will not affect this, even if it is lower than the existing dashboard.
>
> If a provisioned dashboard is saved from the UI and the source is removed, the dashboard stored in the database will be deleted unless the configuration option `disableDeletion` is set to true.
If `allowUiUpdates` is configured to `false`, you are not able to make changes to a provisioned dashboard. When you click `Save`, Grafana brings up a _Cannot save provisioned dashboard_ dialog. The screenshot below illustrates this behavior.
Grafana offers options to export the JSON definition of a dashboard. Either `Copy JSON to Clipboard` or `Save JSON to file` can help you synchronize your dashboard changes back to the provisioning source.
Note: The JSON definition in the input field when using `Copy JSON to Clipboard` or `Save JSON to file` will have the `id` field automatically removed to aid the provisioning workflow.
{{< docs-imagebox img="/img/docs/v51/provisioning_cannot_save_dashboard.png" max-width="500px" class="docs-image--no-shadow" >}}
### Reusable Dashboard URLs
If the dashboard in the json file contains an [uid](/reference/dashboard/#json-fields), Grafana will force insert/update on that uid. This allows you to migrate dashboards between Grafana instances and provisioning Grafana from configuration without breaking the URLs given since the new dashboard URL uses the uid as identifier.
2018-01-14 00:57:48 -06:00
When Grafana starts, it will update/insert all dashboards available in the configured folders. If you modify the file, the dashboard will also be updated.
By default, Grafana will delete dashboards in the database if the file is removed. You can disable this behavior using the `disableDeletion` setting.
> **Note:** Provisioning allows you to overwrite existing dashboards
> which leads to problems if you re-use settings that are supposed to be unique.
> Be careful not to re-use the same `title` multiple times within a folder
> or `uid` within the same installation as this will cause weird behaviors.
2018-11-28 08:35:42 -06:00
### Provision folders structure from filesystem to Grafana
If you already store your dashboards using folders in a git repo or on a filesystem, and also you want to have the same folder names in the Grafana menu, you can use `foldersFromFilesStructure` option.
For example, to replicate these dashboards structure from the filesystem to Grafana,
```
/etc/dashboards
├── /server
│ ├── /common_dashboard.json
│ └── /network_dashboard.json
└── /application
├── /requests_dashboard.json
└── /resources_dashboard.json
```
you need to specify just this short provision configuration file.
```yaml
apiVersion: 1
providers:
- name: dashboards
type: file
updateIntervalSeconds: 30
options:
path: /etc/dashboards
foldersFromFilesStructure: true
```
`server` and `application` will become new folders in Grafana menu.
> **Note:** `folder` and `folderUid` options should be empty or missing to make `foldersFromFilesStructure` work.
> **Note:** To provision dashboards to the General folder, store them in the root of your `path`.
2018-11-28 08:35:42 -06:00
## Alert Notification Channels
Alert Notification Channels can be provisioned by adding one or more YAML config files in the [`provisioning/notifiers`](/administration/configuration/#provisioning) directory.
2018-11-28 08:35:42 -06:00
Each config file can contain the following top-level fields:
- `notifiers`, a list of alert notifications that will be added or updated during start up. If the notification channel already exists, Grafana will update it to match the configuration file.
- `delete_notifiers`, a list of alert notifications to be deleted before inserting/updating those in the `notifiers` list.
2018-11-28 08:35:42 -06:00
Provisioning looks up alert notifications by uid, and will update any existing notification with the provided uid.
2018-11-28 08:35:42 -06:00
By default, exporting a dashboard as JSON will use a sequential identifier to refer to alert notifications. The field `uid` can be optionally specified to specify a string identifier for the alert name.
2018-11-28 08:35:42 -06:00
```json
{
...
"alert": {
...,
"conditions": [...],
"frequency": "24h",
"noDataState": "ok",
"notifications": [
2018-12-14 03:53:50 -06:00
{"uid": "notifier1"},
{"uid": "notifier2"},
2018-11-28 08:35:42 -06:00
]
}
...
}
```
### Example Alert Notification Channels Config File
```yaml
notifiers:
2018-11-28 08:35:42 -06:00
- name: notification-channel-1
type: slack
2018-12-14 03:53:50 -06:00
uid: notifier1
# either
2018-11-28 08:35:42 -06:00
org_id: 2
# or
org_name: Main Org.
2018-11-28 08:35:42 -06:00
is_default: true
send_reminder: true
frequency: 1h
disable_resolve_message: false
# See `Supported Settings` section for settings supported for each
2018-11-28 08:35:42 -06:00
# alert notification type.
settings:
recipient: 'XXX'
2018-11-28 08:35:42 -06:00
uploadImage: true
token: 'xoxb' # legacy setting since Grafana v7.2 (stored non-encrypted)
url: https://slack.com # legacy setting since Grafana v7.2 (stored non-encrypted)
# Secure settings that will be encrypted in the database (supported since Grafana v7.2). See `Supported Settings` section for secure settings supported for each notifier.
secure_settings:
token: 'xoxb'
url: https://slack.com
2018-11-28 08:35:42 -06:00
delete_notifiers:
2018-11-28 08:35:42 -06:00
- name: notification-channel-1
2018-12-14 03:53:50 -06:00
uid: notifier1
# either
2018-11-28 08:35:42 -06:00
org_id: 2
# or
org_name: Main Org.
2018-11-28 08:35:42 -06:00
- name: notification-channel-2
# default org_id: 1
2018-11-28 08:35:42 -06:00
```
### Supported Settings
The following sections detail the supported settings and secure settings for each alert notification type. Secure settings are stored encrypted in the database and you add them to `secure_settings` in the YAML file instead of `settings`.
> **Note:** Secure settings is supported since Grafana v7.2.
2018-11-28 08:35:42 -06:00
#### Alert notification `pushover`
| Name | Secure setting |
| -------- | -------------- |
| apiToken | yes |
| userKey | yes |
| device | |
| priority | |
| okPriority | |
| retry | |
| expire | |
| sound | |
| okSound | |
2018-11-28 08:35:42 -06:00
#### Alert notification `slack`
| Name | Secure setting |
| -------------- | -------------- |
| url | yes |
| recipient | |
| username | |
| icon_emoji | |
| icon_url | |
| uploadImage | |
| mentionUsers | |
| mentionGroups | |
| mentionChannel | |
| token | yes |
2018-11-28 08:35:42 -06:00
#### Alert notification `victorops`
| Name |
| ----------- |
| url |
| autoResolve |
2018-11-28 08:35:42 -06:00
#### Alert notification `kafka`
| Name |
| -------------- |
2018-11-28 08:35:42 -06:00
| kafkaRestProxy |
| kafkaTopic |
2018-11-28 08:35:42 -06:00
#### Alert notification `LINE`
| Name | Secure setting |
| ----- | -------------- |
| token | yes |
2018-11-28 08:35:42 -06:00
#### Alert notification `pagerduty`
| Name | Secure setting |
| -------------- | -------------- |
| integrationKey | yes |
| autoResolve | |
2018-11-28 08:35:42 -06:00
#### Alert notification `sensu`
| Name | Secure setting |
| -------- | -------------- |
| url | |
| source | |
| handler | |
| username | |
| password | yes |
2018-11-28 08:35:42 -06:00
Alerting: Add support for Sensu Go notification channel (#28012) * Add support for Sensu Go notification channel Similar to current support for the older "Core" version of Sensu, this commit add support for the newer version. Closes #19908 Signed-off-by: Todd Campbell <todd@sensu.io> * fix linter errors Signed-off-by: Todd Campbell <todd@sensu.io> * Apply suggestions from code review PR review suggestions Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> * Fix no new variables error * Replace convey testing with testify Signed-off-by: Todd Campbell <todd@sensu.io> * Wording suggestions Signed-off-by: Todd Campbell <todd@sensu.io> * Add docker compose environment for testing/maintenance Signed-off-by: Todd Campbell <todd@sensu.io> * Renamed and fixed docker-compose.yaml to work in devenv Signed-off-by: Todd Campbell <todd@sensu.io> * Change sensugo web UI port to 3080 so as not to conflict with grafana Signed-off-by: Todd Campbell <todd@sensu.io> * Apply suggestions from code review Set the API key as a secure value. Co-authored-by: Sofia Papagiannaki <papagian@users.noreply.github.com> * Add Sensu Go information to notifications doc Signed-off-by: Todd Campbell <todd@sensu.io> * Update pkg/services/alerting/notifiers/sensugo.go Co-authored-by: Sofia Papagiannaki <papagian@users.noreply.github.com> * change assert to require for Error/NoError tests Signed-off-by: Todd Campbell <todd@sensu.io> Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> Co-authored-by: Sofia Papagiannaki <papagian@users.noreply.github.com>
2020-11-27 11:09:24 -06:00
#### Alert notification `sensugo`
| Name | Secure setting |
| -------- | -------------- |
| url | |
| apikey | yes |
| entity | |
| check | |
| handler | |
| namespace | |
2018-11-28 08:35:42 -06:00
#### Alert notification `prometheus-alertmanager`
| Name | Secure setting |
| ----------------- | -------------- |
| url | |
| basicAuthUser | |
| basicAuthPassword | yes |
2018-11-28 08:35:42 -06:00
#### Alert notification `teams`
| Name |
| ---- |
| url |
2018-11-28 08:35:42 -06:00
#### Alert notification `dingding`
| Name |
| ---- |
| url |
2018-11-28 08:35:42 -06:00
#### Alert notification `email`
| Name |
| ----------- |
| singleEmail |
| addresses |
2018-11-28 08:35:42 -06:00
#### Alert notification `hipchat`
| Name |
| ------ |
| url |
2018-11-28 08:35:42 -06:00
| apikey |
| roomid |
#### Alert notification `opsgenie`
| Name | Secure setting |
| ---------------- | -------------- |
| apiKey | yes |
| apiUrl | |
| autoClose | |
| overridePriority | |
2018-11-28 08:35:42 -06:00
#### Alert notification `telegram`
| Name | Secure setting |
| ----------- | -------------- |
| bottoken | yes |
| chatid | |
| uploadImage | |
2018-11-28 08:35:42 -06:00
#### Alert notification `threema`
| Name | Secure setting |
| ------------ | -------------- |
| gateway_id | |
| recipient_id | |
| api_secret | yes |
2018-11-28 08:35:42 -06:00
#### Alert notification `webhook`
| Name | Secure setting |
| ---------- | -------------- |
| url | |
| httpMethod | |
| username | |
| password | yes |
#### Alert notification `googlechat`
| Name |
| ---- |
| url |