Merge remote-tracking branch 'grafana/master' into proxy-slash

* grafana/master: (127 commits)
  alerting: move all notification conditions to defaultShouldNotify
  filter NULL values for column value suggestions
  imguploader: Add support for ECS credential provider for S3
  Remove .dropdown-menu-open on body click fixes #13409
  Remove option r from ln command since its not working everywhere
  fix: updated tests
  Fix spelling of your and you're
  Changed setting to be an alerting setting
  Remove non-existing css prop
  fix: Legend to the right, as table, should follow the width prop. Removing css conflicting with baron's width calculation. #13312
  rendering: Added concurrent rendering limits
  devenv: fix uid for bulk alert dashboards
  Explore: moved code to app/features/explore
  target gfdev-prometheus datasource
  devenv: adds script for creating many dashboards with alerts
  Fix goconst issues
  When stacking graphs, always include the y-offset so that tooltips can render proper values for individual points
  provisioning: changed provisioning default update interval from 3 to 10 seconds
  Fix https://github.com/grafana/grafana/issues/13387 metric segment options displays after blur
  docs: improve oauth generic azure ad instructions
  ...

.circleci/config.yml

@@ -32,7 +32,7 @@ jobs:
         - run: sudo apt update
         - run: sudo apt install -y mysql-client
         - run: dockerize -wait tcp://127.0.0.1:3306 -timeout 120s
-        - run: cat docker/blocks/mysql_tests/setup.sql | mysql -h 127.0.0.1 -P 3306 -u root -prootpass
+        - run: cat devenv/docker/blocks/mysql_tests/setup.sql | mysql -h 127.0.0.1 -P 3306 -u root -prootpass
         - run:
             name: mysql integration tests
             command: 'GRAFANA_TEST_DB=mysql go test ./pkg/services/sqlstore/... ./pkg/tsdb/mysql/... '
@@ -51,7 +51,7 @@ jobs:
         - run: sudo apt update
         - run: sudo apt install -y postgresql-client
         - run: dockerize -wait tcp://127.0.0.1:5432 -timeout 120s
-        - run: 'PGPASSWORD=grafanatest psql -p 5432 -h 127.0.0.1 -U grafanatest -d grafanatest -f docker/blocks/postgres_tests/setup.sql'
+        - run: 'PGPASSWORD=grafanatest psql -p 5432 -h 127.0.0.1 -U grafanatest -d grafanatest -f devenv/docker/blocks/postgres_tests/setup.sql'
         - run:
             name: postgres integration tests
             command: 'GRAFANA_TEST_DB=postgres go test ./pkg/services/sqlstore/... ./pkg/tsdb/postgres/...'

.gitignore

@@ -40,8 +40,8 @@ public/css/*.min.css
 
 conf/custom.ini
 fig.yml
-docker-compose.yml
+devenv/docker-compose.yml
-docker-compose.yaml
+devenv/docker-compose.yaml
 /conf/provisioning/**/custom.yaml
 /conf/provisioning/**/dev.yaml
 /conf/ldap_dev.toml
@@ -72,3 +72,4 @@ debug.test
 *.orig
 
 /devenv/bulk-dashboards/*.json
+/devenv/bulk_alerting_dashboards/*.json

CHANGELOG.md

@@ -1,15 +1,24 @@
 # 5.4.0 (unreleased)
 
+### New Features
+
+* **Annotations**: Enable template variables in tagged annotations queries [#9735](https://github.com/grafana/grafana/issues/9735)
+
 ### Minor
 
 * **OAuth**: Allow oauth email attribute name to be configurable [#12986](https://github.com/grafana/grafana/issues/12986), thx [@bobmshannon](https://github.com/bobmshannon)
 * **Tags**: Default sort order for GetDashboardTags [#11681](https://github.com/grafana/grafana/pull/11681), thx [@Jonnymcc](https://github.com/Jonnymcc)
+* **Prometheus**: Label completion queries respect dashboard time range  [#12251](https://github.com/grafana/grafana/pull/12251), thx [@mtanda](https://github.com/mtanda)
+* **Prometheus**: Allow to display annotations based on Prometheus series value [#10159](https://github.com/grafana/grafana/issues/10159), thx [@mtanda](https://github.com/mtanda)
+* **Prometheus**: Adhoc-filtering for Prometheus dashboards [#13212](https://github.com/grafana/grafana/issues/13212)
+* **Singlestat**: Fix gauge display accuracy for percents [#13270](https://github.com/grafana/grafana/issues/13270), thx [@tianon](https://github.com/tianon)
 
 # 5.3.0 (unreleased)
 
 ### Minor
 
 * **Alerting**: Link to view full size image in Microsoft Teams alert notifier [#13121](https://github.com/grafana/grafana/issues/13121), thx [@holiiveira](https://github.com/holiiveira)
+* **Postgres/MySQL/MSSQL**: Add support for replacing $__interval and  $__interval_ms in alert queries [#11555](https://github.com/grafana/grafana/issues/11555), thx [@svenklemm](https://github.com/svenklemm)
 
 # 5.3.0-beta1 (2018-09-06)
 
@@ -309,7 +318,7 @@ See [security announcement](https://community.grafana.com/t/grafana-5-2-3-and-4-
 * **Dashboard**: Sizing and positioning of settings menu icons [#11572](https://github.com/grafana/grafana/pull/11572)
 * **Dashboard**: Add search filter/tabs to new panel control [#10427](https://github.com/grafana/grafana/issues/10427)
 * **Folders**: User with org viewer role should not be able to save/move dashboards in/to general folder [#11553](https://github.com/grafana/grafana/issues/11553)
-* **Influxdb**: Dont assume the first column in table response is time. [#11476](https://github.com/grafana/grafana/issues/11476), thx [@hahnjo](https://github.com/hahnjo)
+* **Influxdb**: Don't assume the first column in table response is time. [#11476](https://github.com/grafana/grafana/issues/11476), thx [@hahnjo](https://github.com/hahnjo)
 
 ### Tech
 * Backend code simplification [#11613](https://github.com/grafana/grafana/pull/11613), thx [@knweiss](https://github.com/knweiss)
@@ -496,7 +505,7 @@ See [security announcement](https://community.grafana.com/t/grafana-5-2-3-and-4-
 # 4.6.2 (2017-11-16)
 
 ## Important
-* **Prometheus**: Fixes bug with new prometheus alerts in Grafana. Make sure to download this version if your using Prometheus for alerting. More details in the issue. [#9777](https://github.com/grafana/grafana/issues/9777)
+* **Prometheus**: Fixes bug with new prometheus alerts in Grafana. Make sure to download this version if you're using Prometheus for alerting. More details in the issue. [#9777](https://github.com/grafana/grafana/issues/9777)
 
 ## Fixes
 * **Color picker**: Bug after using textbox input field to change/paste color string [#9769](https://github.com/grafana/grafana/issues/9769)
@@ -1455,7 +1464,7 @@ Grafana 2.x is fundamentally different from 1.x; it now ships with an integrated
 
 **New features**
 - [Issue #1623](https://github.com/grafana/grafana/issues/1623). Share Dashboard: Dashboard snapshot sharing (dash and data snapshot), save to local or save to public snapshot dashboard snapshots.raintank.io site
-- [Issue #1622](https://github.com/grafana/grafana/issues/1622). Share Panel: The share modal now has an embed option, gives you an iframe that you can use to embedd a single graph on another web site
+- [Issue #1622](https://github.com/grafana/grafana/issues/1622). Share Panel: The share modal now has an embed option, gives you an iframe that you can use to embed a single graph on another web site
 - [Issue #718](https://github.com/grafana/grafana/issues/718).   Dashboard: When saving a dashboard and another user has made changes in between the user is prompted with a warning if he really wants to overwrite the other's changes
 - [Issue #1331](https://github.com/grafana/grafana/issues/1331). Graph & Singlestat: New axis/unit format selector and more units (kbytes, Joule, Watt, eV), and new design for graph axis & grid tab and single stat options tab views
 - [Issue #1241](https://github.com/grafana/grafana/issues/1242). Timepicker: New option in timepicker (under dashboard settings), to change ``now`` to be for example ``now-1m``, useful when you want to ignore last minute because it contains incomplete data

Gruntfile.js

@@ -25,7 +25,6 @@ module.exports = function (grunt) {
     }
   }
 
-  config.coverage = grunt.option('coverage');
   config.phjs = grunt.option('phjsToRelease');
   config.pkg.version = grunt.option('pkgVer') || config.pkg.version;
 

build.go

@@ -22,6 +22,11 @@ import (
 	"time"
 )
 
+const (
+	windows = "windows"
+	linux   = "linux"
+)
+
 var (
 	//versionRe = regexp.MustCompile(`-[0-9]{1,3}-g[0-9a-f]{5,10}`)
 	goarch  string
@@ -110,17 +115,16 @@ func main() {
 		case "package":
 			grunt(gruntBuildArg("build")...)
 			grunt(gruntBuildArg("package")...)
-			if goos == "linux" {
+			if goos == linux {
 				createLinuxPackages()
 			}
 
 		case "package-only":
 			grunt(gruntBuildArg("package")...)
-			if goos == "linux" {
+			if goos == linux {
 				createLinuxPackages()
 			}
 
-
 		case "pkg-rpm":
 			grunt(gruntBuildArg("release")...)
 			createRpmPackages()
@@ -379,7 +383,7 @@ func ensureGoPath() {
 }
 
 func grunt(params ...string) {
-	if runtime.GOOS == "windows" {
+	if runtime.GOOS == windows {
 		runPrint(`.\node_modules\.bin\grunt`, params...)
 	} else {
 		runPrint("./node_modules/.bin/grunt", params...)
@@ -417,11 +421,11 @@ func test(pkg string) {
 func build(binaryName, pkg string, tags []string) {
 	binary := fmt.Sprintf("./bin/%s-%s/%s", goos, goarch, binaryName)
 	if isDev {
-		//dont include os and arch in output path in dev environment
+		//don't include os and arch in output path in dev environment
 		binary = fmt.Sprintf("./bin/%s", binaryName)
 	}
 
-	if goos == "windows" {
+	if goos == windows {
 		binary += ".exe"
 	}
 
@@ -485,11 +489,11 @@ func clean() {
 
 func setBuildEnv() {
 	os.Setenv("GOOS", goos)
-	if goos == "windows" {
+	if goos == windows {
 		// require windows >=7
 		os.Setenv("CGO_CFLAGS", "-D_WIN32_WINNT=0x0601")
 	}
-	if goarch != "amd64" || goos != "linux" {
+	if goarch != "amd64" || goos != linux {
 		// needed for all other archs
 		cgo = true
 	}

codecov.yml

@@ -1,11 +0,0 @@
-coverage:
-  precision: 2
-  round: down
-  range: "50...100"
-
-  status:
-    project: yes
-    patch: yes
-    changes: no
-
-comment: off

conf/defaults.ini

@@ -474,6 +474,10 @@ error_or_timeout = alerting
 # Default setting for how Grafana handles nodata or null values in alerting. (alerting, no_data, keep_state, ok)
 nodata_or_nullvalues = no_data
 
+# Alert notifications can include images, but rendering many images at the same time can overload the server
+# This limit will protect the server from render overloading and make sure notifications are sent out quickly
+concurrent_render_limit = 5
+
 #################################### Explore #############################
 [explore]
 # Enable the Explore section

conf/ldap.toml

@@ -31,37 +31,11 @@ search_filter = "(cn=%s)"
 # An array of base dns to search through
 search_base_dns = ["dc=grafana,dc=org"]
 
-# In POSIX LDAP schemas, without memberOf attribute a secondary query must be made for groups.
+## For Posix or LDAP setups that does not support member_of attribute you can define the below settings
-# This is done by enabling group_search_filter below. You must also set member_of= "cn"
+## Please check grafana LDAP docs for examples
-# in [servers.attributes] below.
-
-# Users with nested/recursive group membership and an LDAP server that supports LDAP_MATCHING_RULE_IN_CHAIN
-# can set group_search_filter, group_search_filter_user_attribute, group_search_base_dns and member_of
-# below in such a way that the user's recursive group membership is considered.
-#
-# Nested Groups + Active Directory (AD) Example:
-#
-#   AD groups store the Distinguished Names (DNs) of members, so your filter must
-#   recursively search your groups for the authenticating user's DN. For example:
-#
-#     group_search_filter = "(member:1.2.840.113556.1.4.1941:=%s)"
-#     group_search_filter_user_attribute = "distinguishedName"
-#     group_search_base_dns = ["ou=groups,dc=grafana,dc=org"]
-#
-#     [servers.attributes]
-#     ...
-#     member_of = "distinguishedName"
-
-## Group search filter, to retrieve the groups of which the user is a member (only set if memberOf attribute is not available)
 # group_search_filter = "(&(objectClass=posixGroup)(memberUid=%s))"
-## Group search filter user attribute defines what user attribute gets substituted for %s in group_search_filter.
-## Defaults to the value of username in [server.attributes]
-## Valid options are any of your values in [servers.attributes]
-## If you are using nested groups you probably want to set this and member_of in
-## [servers.attributes] to "distinguishedName"
-# group_search_filter_user_attribute = "distinguishedName"
-## An array of the base DNs to search through for groups. Typically uses ou=groups
 # group_search_base_dns = ["ou=groups,dc=grafana,dc=org"]
+# group_search_filter_user_attribute = "uid"
 
 # Specify names of the ldap attributes your ldap uses
 [servers.attributes]

conf/sample.ini

@@ -393,6 +393,10 @@ log_queries =
 # Default setting for how Grafana handles nodata or null values in alerting. (alerting, no_data, keep_state, ok)
 ;nodata_or_nullvalues = no_data
 
+# Alert notifications can include images, but rendering many images at the same time can overload the server
+# This limit will protect the server from render overloading and make sure notifications are sent out quickly
+;concurrent_render_limit = 5
+
 #################################### Explore #############################
 [explore]
 # Enable the Explore section

devenv/bulk_alerting_dashboards/bulk_alerting_dashboards.yaml

@@ -0,0 +1,9 @@
+apiVersion: 1
+
+providers:
+ - name: 'Bulk alerting dashboards'
+   folder: 'Bulk alerting dashboards'
+   type: file
+   options:
+     path: devenv/bulk_alerting_dashboards
+

devenv/bulk_alerting_dashboards/bulkdash_alerting.jsonnet

@@ -0,0 +1,168 @@
+{
+  "editable": true,
+  "gnetId": null,
+  "graphTooltip": 0,
+  "id": null,
+  "links": [],
+  "panels": [
+    {
+      "alert": {
+        "conditions": [
+          {
+            "evaluator": {
+              "params": [
+                65
+              ],
+              "type": "gt"
+            },
+            "operator": {
+              "type": "and"
+            },
+            "query": {
+              "params": [
+                "A",
+                "5m",
+                "now"
+              ]
+            },
+            "reducer": {
+              "params": [],
+              "type": "avg"
+            },
+            "type": "query"
+          }
+        ],
+        "executionErrorState": "alerting",
+        "frequency": "10s",
+        "handler": 1,
+        "name": "bulk alerting",
+        "noDataState": "no_data",
+        "notifications": []
+      },
+      "aliasColors": {},
+      "bars": false,
+      "dashLength": 10,
+      "dashes": false,
+      "datasource": "gdev-prometheus",
+      "fill": 1,
+      "gridPos": {
+        "h": 9,
+        "w": 12,
+        "x": 0,
+        "y": 0
+      },
+      "id": 2,
+      "legend": {
+        "avg": false,
+        "current": false,
+        "max": false,
+        "min": false,
+        "show": true,
+        "total": false,
+        "values": false
+      },
+      "lines": true,
+      "linewidth": 1,
+      "nullPointMode": "null",
+      "percentage": false,
+      "pointradius": 5,
+      "points": false,
+      "renderer": "flot",
+      "seriesOverrides": [],
+      "spaceLength": 10,
+      "stack": false,
+      "steppedLine": false,
+      "targets": [
+        {
+          "$$hashKey": "object:117",
+          "expr": "go_goroutines",
+          "format": "time_series",
+          "intervalFactor": 1,
+          "refId": "A"
+        }
+      ],
+      "thresholds": [
+        {
+          "colorMode": "critical",
+          "fill": true,
+          "line": true,
+          "op": "gt",
+          "value": 50
+        }
+      ],
+      "timeFrom": null,
+      "timeShift": null,
+      "title": "Panel Title",
+      "tooltip": {
+        "shared": true,
+        "sort": 0,
+        "value_type": "individual"
+      },
+      "type": "graph",
+      "xaxis": {
+        "buckets": null,
+        "mode": "time",
+        "name": null,
+        "show": true,
+        "values": []
+      },
+      "yaxes": [
+        {
+          "format": "short",
+          "label": null,
+          "logBase": 1,
+          "max": null,
+          "min": null,
+          "show": true
+        },
+        {
+          "format": "short",
+          "label": null,
+          "logBase": 1,
+          "max": null,
+          "min": null,
+          "show": true
+        }
+      ]
+    }
+  ],
+  "schemaVersion": 16,
+  "style": "dark",
+  "tags": [],
+  "templating": {
+    "list": []
+  },
+  "time": {
+    "from": "now-6h",
+    "to": "now"
+  },
+  "timepicker": {
+    "refresh_intervals": [
+      "5s",
+      "10s",
+      "30s",
+      "1m",
+      "5m",
+      "15m",
+      "30m",
+      "1h",
+      "2h",
+      "1d"
+    ],
+    "time_options": [
+      "5m",
+      "15m",
+      "1h",
+      "6h",
+      "12h",
+      "24h",
+      "2d",
+      "7d",
+      "30d"
+    ]
+  },
+  "timezone": "",
+  "title": "New dashboard",
+  "uid": null,
+  "version": 0
+}

devenv/create_docker_compose.sh

@@ -1,13 +1,13 @@
 #!/bin/bash
 
-blocks_dir=blocks
+blocks_dir=docker/blocks
 docker_dir=docker
 template_dir=templates
 
 grafana_config_file=conf.tmp
 grafana_config=config
 
-compose_header_file=compose_header.yml
+compose_header_file=docker/compose_header.yml
 fig_file=docker-compose.yaml
 fig_config=docker-compose.yaml
 

devenv/docker/blocks/apache_proxy/docker-compose.yaml

@@ -5,5 +5,5 @@
 # root_url = %(protocol)s://%(domain)s:10081/grafana/
 
   apacheproxy:
-    build: blocks/apache_proxy
+    build: docker/blocks/apache_proxy
     network_mode: host

devenv/docker/blocks/collectd/docker-compose.yaml

@@ -1,5 +1,5 @@
   collectd:
-    build: blocks/collectd
+    build: docker/blocks/collectd
     environment:
       HOST_NAME: myserver
       GRAPHITE_HOST: graphite

devenv/docker/blocks/graphite/docker-compose.yaml

@@ -1,5 +1,5 @@
   graphite09:
-    build: blocks/graphite
+    build: docker/blocks/graphite
     ports:
       - "8080:80"
       - "2003:2003"

devenv/docker/blocks/graphite1/conf/opt/graphite/conf/aggregation-rules.conf

@@ -8,7 +8,7 @@
 # 'avg'. The name of the aggregate metric will be derived from
 # 'output_template' filling in any captured fields from 'input_pattern'.
 #
-# For example, if you're metric naming scheme is:
+# For example, if your metric naming scheme is:
 #
 #   <env>.applications.<app>.<server>.<metric>
 #

devenv/docker/blocks/graphite1/docker-compose.yaml

@@ -1,6 +1,6 @@
   graphite:
     build:
-      context: blocks/graphite1
+      context: docker/blocks/graphite1
       args:
         version: master
     ports:

devenv/docker/blocks/mssql/docker-compose.yaml

@@ -1,6 +1,6 @@
   mssql:
     build:
-      context: blocks/mssql/build
+      context: docker/blocks/mssql/build
     environment:
       ACCEPT_EULA: Y
       MSSQL_SA_PASSWORD: Password!

devenv/docker/blocks/mssql_tests/docker-compose.yaml

@@ -1,6 +1,6 @@
   mssqltests:
     build:
-      context: blocks/mssql/build
+      context: docker/blocks/mssql/build
     environment:
       ACCEPT_EULA: Y
       MSSQL_SA_PASSWORD: Password!

devenv/docker/blocks/mysql_opendata/docker-compose.yaml

@@ -1,5 +1,5 @@
   mysql_opendata:
-    build: blocks/mysql_opendata
+    build: docker/blocks/mysql_opendata
     environment:
       MYSQL_ROOT_PASSWORD: rootpass
       MYSQL_DATABASE: testdata

devenv/docker/blocks/mysql_tests/docker-compose.yaml

@@ -1,6 +1,6 @@
   mysqltests:
     build:
-      context: blocks/mysql_tests
+      context: docker/blocks/mysql_tests
     environment:
       MYSQL_ROOT_PASSWORD: rootpass
       MYSQL_DATABASE: grafana_tests

devenv/docker/blocks/nginx_proxy/docker-compose.yaml

@@ -5,5 +5,5 @@
 # root_url = %(protocol)s://%(domain)s:10080/grafana/
 
   nginxproxy:
-    build: blocks/nginx_proxy
+    build: docker/blocks/nginx_proxy
     network_mode: host

devenv/docker/blocks/openldap/docker-compose.yaml

@@ -1,5 +1,5 @@
   openldap:
-    build: blocks/openldap
+    build: docker/blocks/openldap
     environment:
       SLAPD_PASSWORD: grafana
       SLAPD_DOMAIN: grafana.org