IntenseWebs/grafana
3
0
Fork 0
mirror of https://github.com/grafana/grafana.git synced 2025-02-25 18:55:37 -06:00
Code Issues Packages Projects Releases Wiki Activity

Provisioning: Makes file the default dashboard provisioner type (#24856)

Browse Source 
Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>
This commit is contained in:
Carl Bergquist
2020-05-19 15:01:04 +02:00
committed by GitHub
parent 2f01f1320f
commit c62591e77b
5 changed files with 246 additions and 219 deletions
Download Patch File Download Diff File Expand all files Collapse all files

102
docs/sources/administration/provisioning.md
View File

@@ -53,13 +53,13 @@ If you have a literal `$` in your value and want to avoid interpolation, `$$` ca
Currently we do not provide any scripts/manifests for configuring Grafana. Rather than spending time learning and creating scripts/manifests for each tool, we think our time is better spent making Grafana easier to provision. Therefore, we heavily rely on the expertise of the community.
Tool | Project
-----|------------
Puppet | [https://forge.puppet.com/puppet/grafana](https://forge.puppet.com/puppet/grafana)
Ansible | [https://github.com/cloudalchemy/ansible-grafana](https://github.com/cloudalchemy/ansible-grafana)
Chef | [https://github.com/JonathanTron/chef-grafana](https://github.com/JonathanTron/chef-grafana)
Saltstack | [https://github.com/salt-formulas/salt-formula-grafana](https://github.com/salt-formulas/salt-formula-grafana)
Jsonnet | [https://github.com/grafana/grafonnet-lib/](https://github.com/grafana/grafonnet-lib/)
| Tool | Project |
| --------- | -------------------------------------------------------------------------------------------------------------- |
| Puppet | [https://forge.puppet.com/puppet/grafana](https://forge.puppet.com/puppet/grafana) |
| Ansible | [https://github.com/cloudalchemy/ansible-grafana](https://github.com/cloudalchemy/ansible-grafana) |
| Chef | [https://github.com/JonathanTron/chef-grafana](https://github.com/JonathanTron/chef-grafana) |
| Saltstack | [https://github.com/salt-formulas/salt-formula-grafana](https://github.com/salt-formulas/salt-formula-grafana) |
| Jsonnet | [https://github.com/grafana/grafonnet-lib/](https://github.com/grafana/grafonnet-lib/) |
## Datasources
@@ -115,14 +115,14 @@ datasources:
isDefault:
# <map> fields that will be converted to json and stored in jsonData
jsonData:
graphiteVersion: "1.1"
graphiteVersion: '1.1'
tlsAuth: true
tlsAuthWithCACert: true
# <string> json object of data that will be encrypted.
secureJsonData:
tlsCACert: "..."
tlsClientCert: "..."
tlsClientKey: "..."
tlsCACert: '...'
tlsClientCert: '...'
tlsClientKey: '...'
# <string> database password, if used
password:
# <string> basic auth password
@@ -133,10 +133,11 @@ datasources:
```
#### Custom Settings per Datasource
Please refer to each datasource documentation for specific provisioning examples.
| Datasource | Misc |
| ---- | ---- |
| ------------- | ---------------------------------------------------------------------------------- |
| Elasticsearch | Elasticsearch uses the `database` property to configure the index for a datasource |
#### Json Data
@@ -144,10 +145,10 @@ Please refer to each datasource documentation for specific provisioning examples
Since not all datasources have the same configuration settings we only have the most common ones as fields. The rest should be stored as a json blob in the `jsonData` field. Here are the most common settings that the core datasources use.
| Name | Type | Datasource | Description |
| ---- | ---- | ---- | ---- |
| tlsAuth | boolean | *All* | Enable TLS authentication using client cert configured in secure json data |
| tlsAuthWithCACert | boolean | *All* | Enable TLS authentication using CA cert |
| tlsSkipVerify | boolean | *All* | Controls whether a client verifies the server's certificate chain and host name. |
| ----------------------- | ------- | ---------------------------------------------------------------- | ------------------------------------------------------------------------------------------- |
| tlsAuth | boolean | _All_ | Enable TLS authentication using client cert configured in secure json data |
| tlsAuthWithCACert | boolean | _All_ | Enable TLS authentication using CA cert |
| tlsSkipVerify | boolean | _All_ | Controls whether a client verifies the server's certificate chain and host name. |
| graphiteVersion | string | Graphite | Graphite version |
| timeInterval | string | Prometheus, Elasticsearch, InfluxDB, MySQL, PostgreSQL and MSSQL | Lowest interval/step value that should be used for this data source |
| esVersion | number | Elasticsearch | Elasticsearch version as a number (2/5/56/60/70) |
@@ -164,7 +165,7 @@ Since not all datasources have the same configuration settings we only have the
| sslmode | string | PostgreSQL | SSLmode. 'disable', 'require', 'verify-ca' or 'verify-full' |
| sslRootCertFile | string | PostgreSQL | SSL server root certificate file, must be readable by the Grafana user |
| sslCertFile | string | PostgreSQL | SSL client certificate file, must be readable by the Grafana user |
| sslKeyFile | string | PostgreSQL | SSL client key file, must be readable by *only* the Grafana user |
| sslKeyFile | string | PostgreSQL | SSL client key file, must be readable by _only_ the Grafana user |
| encrypt | string | MSSQL | Connection SSL encryption handling. 'disable', 'false' or 'true' |
| postgresVersion | number | PostgreSQL | Postgres version as a number (903/904/905/906/1000) meaning v9.3, v9.4, ..., v10 |
| timescaledb | boolean | PostgreSQL | Enable usage of TimescaleDB extension |
@@ -179,16 +180,17 @@ Since not all datasources have the same configuration settings we only have the
Secure json data is a map of settings that will be encrypted with [secret key]({{< relref "../installation/configuration/#secret-key" >}}) from the Grafana config. The purpose of this is only to hide content from the users of the application. This should be used for storing TLS Cert and password that Grafana will append to the request on the server side. All of these settings are optional.
| Name | Type | Datasource | Description |
| ----| ---- | ---- | --- |
| tlsCACert | string | *All* |CA cert for out going requests |
| tlsClientCert | string | *All* |TLS Client cert for outgoing requests |
| tlsClientKey | string | *All* |TLS Client key for outgoing requests |
| password | string | *All* | password |
| basicAuthPassword | string | *All* | password for basic authentication |
| ----------------- | ------ | ---------- | --------------------------------------- |
| tlsCACert | string | _All_ | CA cert for out going requests |
| tlsClientCert | string | _All_ | TLS Client cert for outgoing requests |
| tlsClientKey | string | _All_ | TLS Client key for outgoing requests |
| password | string | _All_ | password |
| basicAuthPassword | string | _All_ | password for basic authentication |
| accessKey | string | Cloudwatch | Access key for connecting to Cloudwatch |
| secretKey | string | Cloudwatch | Secret key for connecting to Cloudwatch |
#### Custom HTTP headers for datasources
Datasources managed by Grafanas provisioning can be configured to add HTTP headers to all requests
going to that datasource. The header name is configured in the `jsonData` field and the header value should be
configured in `secureJsonData`.
@@ -199,11 +201,11 @@ apiVersion: 1
datasources:
- name: Graphite
jsonData:
httpHeaderName1: "HeaderName"
httpHeaderName2: "Authorization"
httpHeaderName1: 'HeaderName'
httpHeaderName2: 'Authorization'
secureJsonData:
httpHeaderValue1: "HeaderValue"
httpHeaderValue2: "Bearer XXXXXXXXX"
httpHeaderValue1: 'HeaderValue'
httpHeaderValue2: 'Bearer XXXXXXXXX'
```
## Dashboards
@@ -216,15 +218,15 @@ The dashboard provider config file looks somewhat like this:
apiVersion: 1
providers:
# <string> an unique provider name
# <string> an unique provider name. Required
- name: 'a unique provider name'
# <int> org id. will default to orgId 1 if not specified
# <int> Org id. Default to 1
orgId: 1
# <string, required> name of the dashboard folder. Required
# <string> name of the dashboard folder.
folder: ''
# <string> folder UID. will be automatically generated if not specified
folderUid: ''
# <string, required> provider type. Required
# <string> provider type. Default to 'file'
type: file
# <bool> disable dashboard deletion
disableDeletion: false
@@ -235,13 +237,14 @@ providers:
# <bool> allow updating provisioned dashboards from the UI
allowUiUpdates: false
options:
# <string, required> path to dashboard files on disk. Required
# <string, required> path to dashboard files on disk. Required when using the 'file' type
path: /var/lib/grafana/dashboards
```
When Grafana starts, it will update/insert all dashboards available in the configured path. Then later on poll that path every **updateIntervalSeconds** and look for updated json files and update/insert those into the database.
#### Making changes to a provisioned dashboard
It's possible to make changes to a provisioned dashboard in the Grafana UI. However, it is not possible to automatically save the changes back to the provisioning source.
If `allowUiUpdates` is set to `true` and you make changes to a provisioned dashboard, you can `Save` the dashboard then changes will be persisted to the Grafana database.
@@ -250,7 +253,7 @@ If `allowUiUpdates` is set to `true` and you make changes to a provisioned dashb
>
> If a provisioned dashboard is saved from the UI and the source is removed, the dashboard stored in the database will be deleted unless the configuration option `disableDeletion` is set to true.
If `allowUiUpdates` is configured to `false`, you are not able to make changes to a provisioned dashboard. When you click `Save`, Grafana brings up a *Cannot save provisioned dashboard* dialog. The screenshot below illustrates this behavior.
If `allowUiUpdates` is configured to `false`, you are not able to make changes to a provisioned dashboard. When you click `Save`, Grafana brings up a _Cannot save provisioned dashboard_ dialog. The screenshot below illustrates this behavior.
Grafana offers options to export the JSON definition of a dashboard. Either `Copy JSON to Clipboard` or `Save JSON to file` can help you synchronize your dashboard changes back to the provisioning source.
@@ -274,6 +277,7 @@ By default Grafana will delete dashboards in the database if the file is removed
Alert Notification Channels can be provisioned by adding one or more yaml config files in the [`provisioning/notifiers`](/installation/configuration/#provisioning) directory.
Each config file can contain the following top-level fields:
- `notifiers`, a list of alert notifications that will be added or updated during start up. If the notification channel already exists, Grafana will update it to match the configuration file.
- `delete_notifiers`, a list of alert notifications to be deleted before inserting/updating those in the `notifiers` list.
@@ -316,8 +320,8 @@ notifiers:
# See `Supported Settings` section for settings supporter for each
# alert notification type.
settings:
recipient: "XXX"
token: "xoxb"
recipient: 'XXX'
token: 'xoxb'
uploadImage: true
url: https://slack.com
@@ -339,7 +343,7 @@ The following sections detail the supported settings for each alert notification
#### Alert notification `pushover`
| Name |
| ---- |
| -------- |
| apiToken |
| userKey |
| device |
@@ -349,7 +353,7 @@ The following sections detail the supported settings for each alert notification
#### Alert notification `slack`
| Name |
| ---- |
| -------------- |
| url |
| recipient |
| username |
@@ -364,34 +368,34 @@ The following sections detail the supported settings for each alert notification
#### Alert notification `victorops`
| Name |
| ---- |
| ----------- |
| url |
| autoResolve |
#### Alert notification `kafka`
| Name |
| ---- |
| -------------- |
| kafkaRestProxy |
| kafkaTopic |
#### Alert notification `LINE`
| Name |
| ---- |
| ----- |
| token |
#### Alert notification `pagerduty`
| Name |
| ---- |
| -------------- |
| integrationKey |
| autoResolve |
#### Alert notification `sensu`
| Name |
| ---- |
| -------- |
| url |
| source |
| handler |
@@ -401,7 +405,7 @@ The following sections detail the supported settings for each alert notification
#### Alert notification `prometheus-alertmanager`
| Name |
| ---- |
| ----------------- |
| url |
| basicAuthUser |
| basicAuthPassword |
@@ -421,14 +425,14 @@ The following sections detail the supported settings for each alert notification
#### Alert notification `email`
| Name |
| ---- |
| ----------- |
| singleEmail |
| addresses |
#### Alert notification `hipchat`
| Name |
| ---- |
| ------ |
| url |
| apikey |
| roomid |
@@ -436,7 +440,7 @@ The following sections detail the supported settings for each alert notification
#### Alert notification `opsgenie`
| Name |
| ---- |
| ---------------- |
| apiKey |
| apiUrl |
| autoClose |
@@ -445,7 +449,7 @@ The following sections detail the supported settings for each alert notification
#### Alert notification `telegram`
| Name |
| ---- |
| ----------- |
| bottoken |
| chatid |
| uploadImage |
@@ -453,7 +457,7 @@ The following sections detail the supported settings for each alert notification
#### Alert notification `threema`
| Name |
| ---- |
| ------------ |
| gateway_id |
| recipient_id |
| api_secret |
@@ -461,7 +465,7 @@ The following sections detail the supported settings for each alert notification
#### Alert notification `webhook`
| Name |
| ---- |
| -------- |
| url |
| username |
| password |

1
go.sum
View File

@@ -297,6 +297,7 @@ github.com/smartystreets/goconvey v0.0.0-20181108003508-044398e4856c/go.mod h1:X
github.com/smartystreets/goconvey v0.0.0-20190731233626-505e41936337 h1:WN9BUFbdyOsSH/XohnWpXOlq9NBD5sGAB2FciQMUEe8=
github.com/smartystreets/goconvey v0.0.0-20190731233626-505e41936337/go.mod h1:syvi0/a8iFYH4r/RixwvyeAJjdLS9QV7WQ/tjFTllLA=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.1.1 h1:2vfRuCMp5sSVIDSqO8oNnWJq7mPa6KVP3iPIwFBuy8A=
github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.2.0/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=

4
pkg/services/provisioning/dashboards/config_reader.go
View File

@@ -88,6 +88,10 @@ func (cr *configReader) readConfig() ([]*config, error) {
dashboard.OrgID = 1
}
if dashboard.Type == "" {
dashboard.Type = "file"
}
if dashboard.UpdateIntervalSeconds == 0 {
dashboard.UpdateIntervalSeconds = 10
}

78
pkg/services/provisioning/dashboards/config_reader_test.go
View File

@@ -4,87 +4,99 @@ import (
"os"
"testing"
"github.com/stretchr/testify/require"
"github.com/grafana/grafana/pkg/infra/log"
. "github.com/smartystreets/goconvey/convey"
)
var (
simpleDashboardConfig = "./testdata/test-configs/dashboards-from-disk"
oldVersion = "./testdata/test-configs/version-0"
brokenConfigs = "./testdata/test-configs/broken-configs"
appliedDefaults = "./testdata/test-configs/applied-defaults"
)
func TestDashboardsAsConfig(t *testing.T) {
Convey("Dashboards as configuration", t, func() {
t.Run("Dashboards as configuration", func(t *testing.T) {
logger := log.New("test-logger")
Convey("Can read config file version 1 format", func() {
t.Run("default values should be applied", func(t *testing.T) {
cfgProvider := configReader{path: appliedDefaults, log: logger}
cfg, err := cfgProvider.readConfig()
require.NoError(t, err)
require.Equal(t, "file", cfg[0].Type)
require.Equal(t, int64(1), cfg[0].OrgID)
require.Equal(t, int64(10), cfg[0].UpdateIntervalSeconds)
})
t.Run("Can read config file version 1 format", func(t *testing.T) {
_ = os.Setenv("TEST_VAR", "general")
cfgProvider := configReader{path: simpleDashboardConfig, log: logger}
cfg, err := cfgProvider.readConfig()
_ = os.Unsetenv("TEST_VAR")
So(err, ShouldBeNil)
require.NoError(t, err)
validateDashboardAsConfig(t, cfg)
})
Convey("Can read config file in version 0 format", func() {
t.Run("Can read config file in version 0 format", func(t *testing.T) {
cfgProvider := configReader{path: oldVersion, log: logger}
cfg, err := cfgProvider.readConfig()
So(err, ShouldBeNil)
require.NoError(t, err)
validateDashboardAsConfig(t, cfg)
})
Convey("Should skip invalid path", func() {
t.Run("Should skip invalid path", func(t *testing.T) {
cfgProvider := configReader{path: "/invalid-directory", log: logger}
cfg, err := cfgProvider.readConfig()
if err != nil {
t.Fatalf("readConfig return an error %v", err)
}
So(len(cfg), ShouldEqual, 0)
require.Equal(t, 0, len(cfg))
})
Convey("Should skip broken config files", func() {
t.Run("Should skip broken config files", func(t *testing.T) {
cfgProvider := configReader{path: brokenConfigs, log: logger}
cfg, err := cfgProvider.readConfig()
if err != nil {
t.Fatalf("readConfig return an error %v", err)
}
So(len(cfg), ShouldEqual, 0)
require.Equal(t, 0, len(cfg))
})
})
}
func validateDashboardAsConfig(t *testing.T, cfg []*config) {
t.Helper()
So(len(cfg), ShouldEqual, 2)
require.Equal(t, 2, len(cfg))
ds := cfg[0]
So(ds.Name, ShouldEqual, "general dashboards")
So(ds.Type, ShouldEqual, "file")
So(ds.OrgID, ShouldEqual, 2)
So(ds.Folder, ShouldEqual, "developers")
So(ds.FolderUID, ShouldEqual, "xyz")
So(ds.Editable, ShouldBeTrue)
So(len(ds.Options), ShouldEqual, 1)
So(ds.Options["path"], ShouldEqual, "/var/lib/grafana/dashboards")
So(ds.DisableDeletion, ShouldBeTrue)
So(ds.UpdateIntervalSeconds, ShouldEqual, 15)
require.Equal(t, ds.Name, "general dashboards")
require.Equal(t, ds.Type, "file")
require.Equal(t, ds.OrgID, int64(2))
require.Equal(t, ds.Folder, "developers")
require.Equal(t, ds.FolderUID, "xyz")
require.True(t, ds.Editable)
require.Equal(t, len(ds.Options), 1)
require.Equal(t, ds.Options["path"], "/var/lib/grafana/dashboards")
require.True(t, ds.DisableDeletion)
require.Equal(t, ds.UpdateIntervalSeconds, int64(15))
ds2 := cfg[1]
So(ds2.Name, ShouldEqual, "default")
So(ds2.Type, ShouldEqual, "file")
So(ds2.OrgID, ShouldEqual, 1)
So(ds2.Folder, ShouldEqual, "")
So(ds2.FolderUID, ShouldEqual, "")
So(ds2.Editable, ShouldBeFalse)
So(len(ds2.Options), ShouldEqual, 1)
So(ds2.Options["path"], ShouldEqual, "/var/lib/grafana/dashboards")
So(ds2.DisableDeletion, ShouldBeFalse)
So(ds2.UpdateIntervalSeconds, ShouldEqual, 10)
require.Equal(t, ds2.Name, "default")
require.Equal(t, ds2.Type, "file")
require.Equal(t, ds2.OrgID, int64(1))
require.Equal(t, ds2.Folder, "")
require.Equal(t, ds2.FolderUID, "")
require.False(t, ds2.Editable)
require.Equal(t, len(ds2.Options), 1)
require.Equal(t, ds2.Options["path"], "/var/lib/grafana/dashboards")
require.False(t, ds2.DisableDeletion)
require.Equal(t, ds2.UpdateIntervalSeconds, int64(10))
}

6
pkg/services/provisioning/dashboards/testdata/test-configs/applied-defaults/dev-dashboards.yaml vendored Normal file
View File

@@ -0,0 +1,6 @@
apiVersion: 1
providers:
- name: 'applied-defaults'
options:
path: /var/lib/grafana/dashboards