IntenseWebs/grafana
3
0
Fork 0
mirror of https://github.com/grafana/grafana.git synced 2024-11-22 17:06:57 -06:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'master' into mssql_datasource

Browse Source
This commit is contained in:
Marcus Efraimsson 2018-03-19 17:14:33 +01:00
commit 6044b3aeeb
7 changed files with 664 additions and 597 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
CHANGELOG.md
View File

@ -5,6 +5,7 @@
* **Graph**: Thresholds for Right Y axis [#7107](https://github.com/grafana/grafana/issues/7107), thx [@ilgizar](https://github.com/ilgizar)
* **Graph**: Support multiple series stacking in histogram mode [#8151](https://github.com/grafana/grafana/issues/8151), thx [@mtanda](https://github.com/mtanda)
* **Alerting**: Pausing/un alerts now updates new_state_date [#10942](https://github.com/grafana/grafana/pull/10942)
* **Alerting**: Support Pagerduty notification channel using Pagerduty V2 API [#10531](https://github.com/grafana/grafana/issues/10531), thx [@jbaublitz](https://github.com/jbaublitz)
* **Templating**: Add comma templating format [#10632](https://github.com/grafana/grafana/issues/10632), thx [@mtanda](https://github.com/mtanda)
* **Prometheus**: Support POST for query and query_range [#9859](https://github.com/grafana/grafana/pull/9859), thx [@mtanda](https://github.com/mtanda)

9
ROADMAP.md
View File

@ -6,18 +6,21 @@ But it will give you an idea of our current vision and plan.
### Short term (1-2 months)
- v5.1
- Crossplatform builds & build speed improvements
- Build speed improvements & integration test execution
- Kubernetes friendly docker container
- Enterprise LDAP
- Provisioning workflow
- First login registration view
- IFQL Initial support
- MSSQL datasource
### Mid term (2-4 months)
- v5.2
- Azure monitor backend rewrite
- Elasticsearch alerting
- First login registration view
- Backend plugins? (alert notifiers, auth)
- Crossplatform builds
- IFQL Initial support
### Long term (4 - 8 months)

36
docs/sources/tutorials/ha_setup.md
View File

@ -9,30 +9,38 @@ weight = 10
# How to setup Grafana for high availability
> Alerting does not support high availability yet.
Setting up Grafana for high availability is fairly simple. It comes down to two things:
* Use a shared database for multiple grafana instances.
* Consider how user sessions are stored.
1. Use a shared database for storing dashboard, users, and other persistent data
2. Decide how to store session data.
<div class="text-center">
<img src="/img/docs/tutorials/grafana-high-availability.png" max-width= "800px" class="center"></img>
</div>
## Configure multiple servers to use the same database
First you need to do is to setup mysql or postgres on another server and configure Grafana to use that database.
First, you need to do is to setup MySQL or Postgres on another server and configure Grafana to use that database.
You can find the configuration for doing that in the [[database]]({{< relref "configuration.md" >}}#database) section in the grafana config.
Grafana will now persist all long term data in the database.
It also worth considering how to setup the database for high availability but thats outside the scope of this guide.
Grafana will now persist all long term data in the database. How to configure the database for high availability is out of scope for this guide. We recommend finding an expert on for the database your using.
## User sessions
The second thing to consider is how to deal with user sessions and how to balance the load between servers.
By default Grafana stores user sessions on disk which works fine if you use `sticky sessions` in your load balancer.
Grafana also supports storing the session data in the database, redis or memcache which makes it possible to use round robin in your load balancer.
If you use mysql/postgres for session storage you first need a table to store the session data in. More details about that in [[sessions]]({{< relref "configuration.md" >}}#session)
The second thing to consider is how to deal with user sessions and how to configure your load balancer infront of Grafana.
Grafana support two says of storing session data locally on disk or in a database/cache-server.
If you want to store sessions on disk you can use `sticky sessions` in your load balanacer. If you prefer to store session data in a database/cache-server
you can use any stateless routing strategy in your load balancer (ex round robin or least connections).
For Grafana itself it doesn't really matter if you store your sessions on disk or database/redis/memcache.
But we suggest that you store the session in redis/memcache since it makes it easier to add/remote instances from the group.
### Sticky sessions
Using sticky sessions, all traffic for one user will always be sent to the same server. Which means that session related data can be
stored on disk rather than on a shared database. This is the default behavior for Grafana and if only want multiple servers for fail over this is a good solution since it requires the least amount of work.
### Stateless sessions
You can also choose to store session data in a Redis/Memcache/Postgres/MySQL which means that the load balancer can send a user to any Grafana server without having to log in on each server. This requires a little bit more work from the operator but enables you to remove/add grafana servers without impacting the user experience.
If you use MySQL/Postgres for session storage, you first need a table to store the session data in. More details about that in [[sessions]]({{< relref "configuration.md" >}}#session)
For Grafana itself it doesn't really matter if you store the session data on disk or database/redis/memcache. But we recommend using a database/redis/memcache since it makes it easier manage the grafana servers.
## Alerting
Currently alerting supports a limited form of high availability. Since v4.2.0 of Grafana, alert notifications are deduped when running multiple servers. This means all alerts are executed on every server but no duplicate alert notifications are sent due to the deduping logic. Proper load balancing of alerts will be introduced in the future.
Currently alerting supports a limited form of high availability. Since v4.2.0, alert notifications are deduped when running multiple servers. This means all alerts are executed on every server but alert notifications are only sent once per alert. Grafana does not support distributing the alert rule execution between servers. That might be added in the future but right now prefer to keep it simple.

108
public/app/plugins/datasource/elasticsearch/bucket_agg.js → public/app/plugins/datasource/elasticsearch/bucket_agg.ts
View File

@ -1,28 +1,24 @@
define([
'angular',
'lodash',
'./query_def',
],
function (angular, _, queryDef) {
'use strict';
import angular from 'angular';
import _ from 'lodash';
import * as queryDef from './query_def';
var module = angular.module('grafana.directives');
export function elasticBucketAgg() {
return {
templateUrl: 'public/app/plugins/datasource/elasticsearch/partials/bucket_agg.html',
controller: 'ElasticBucketAggCtrl',
restrict: 'E',
scope: {
target: '=',
index: '=',
onChange: '&',
getFields: '&',
},
};
}
module.directive('elasticBucketAgg', function() {
return {
templateUrl: 'public/app/plugins/datasource/elasticsearch/partials/bucket_agg.html',
controller: 'ElasticBucketAggCtrl',
restrict: 'E',
scope: {
target: "=",
index: "=",
onChange: "&",
getFields: "&",
}
};
});
module.controller('ElasticBucketAggCtrl', function($scope, uiSegmentSrv, $q, $rootScope) {
export class ElasticBucketAggCtrl {
/** @nginject */
constructor($scope, uiSegmentSrv, $q, $rootScope) {
var bucketAggs = $scope.target.bucketAggs;
$scope.orderByOptions = [];
@ -39,9 +35,13 @@ function (angular, _, queryDef) {
return queryDef.sizeOptions;
};
$rootScope.onAppEvent('elastic-query-updated', function() {
$scope.validateModel();
}, $scope);
$rootScope.onAppEvent(
'elastic-query-updated',
function() {
$scope.validateModel();
},
$scope
);
$scope.init = function() {
$scope.agg = bucketAggs[$scope.index];
@ -56,10 +56,10 @@ function (angular, _, queryDef) {
$scope.agg.settings = {};
$scope.showOptions = false;
switch($scope.agg.type) {
switch ($scope.agg.type) {
case 'date_histogram':
case 'histogram':
case 'terms': {
case 'terms': {
delete $scope.agg.query;
$scope.agg.field = 'select field';
break;
@ -84,15 +84,15 @@ function (angular, _, queryDef) {
$scope.isFirst = $scope.index === 0;
$scope.bucketAggCount = bucketAggs.length;
var settingsLinkText = "";
var settingsLinkText = '';
var settings = $scope.agg.settings || {};
switch($scope.agg.type) {
switch ($scope.agg.type) {
case 'terms': {
settings.order = settings.order || "desc";
settings.size = settings.size || "10";
settings.order = settings.order || 'desc';
settings.size = settings.size || '10';
settings.min_doc_count = settings.min_doc_count || 1;
settings.orderBy = settings.orderBy || "_term";
settings.orderBy = settings.orderBy || '_term';
if (settings.size !== '0') {
settingsLinkText = queryDef.describeOrder(settings.order) + ' ' + settings.size + ', ';
@ -111,13 +111,17 @@ function (angular, _, queryDef) {
break;
}
case 'filters': {
settings.filters = settings.filters || [{query: '*'}];
settingsLinkText = _.reduce(settings.filters, function(memo, value, index) {
memo += 'Q' + (index + 1) + ' = ' + value.query + ' ';
return memo;
}, '');
settings.filters = settings.filters || [{ query: '*' }];
settingsLinkText = _.reduce(
settings.filters,
function(memo, value, index) {
memo += 'Q' + (index + 1) + ' = ' + value.query + ' ';
return memo;
},
''
);
if (settingsLinkText.length > 50) {
settingsLinkText = settingsLinkText.substr(0, 50) + "...";
settingsLinkText = settingsLinkText.substr(0, 50) + '...';
}
settingsLinkText = 'Filter Queries (' + settings.filters.length + ')';
break;
@ -165,7 +169,7 @@ function (angular, _, queryDef) {
};
$scope.addFiltersQuery = function() {
$scope.agg.settings.filters.push({query: '*'});
$scope.agg.settings.filters.push({ query: '*' });
};
$scope.removeFiltersQuery = function(filter) {
@ -182,7 +186,7 @@ function (angular, _, queryDef) {
$scope.getFieldsInternal = function() {
if ($scope.agg.type === 'date_histogram') {
return $scope.getFields({$fieldType: 'date'});
return $scope.getFields({ $fieldType: 'date' });
} else {
return $scope.getFields();
}
@ -198,14 +202,18 @@ function (angular, _, queryDef) {
var addIndex = bucketAggs.length - 1;
if (lastBucket && lastBucket.type === 'date_histogram') {
addIndex - 1;
addIndex -= 1;
}
var id = _.reduce($scope.target.bucketAggs.concat($scope.target.metrics), function(max, val) {
return parseInt(val.id) > max ? parseInt(val.id) : max;
}, 0);
var id = _.reduce(
$scope.target.bucketAggs.concat($scope.target.metrics),
function(max, val) {
return parseInt(val.id) > max ? parseInt(val.id) : max;
},
0
);
bucketAggs.splice(addIndex, 0, {type: "terms", field: "select field", id: (id+1).toString(), fake: true});
bucketAggs.splice(addIndex, 0, { type: 'terms', field: 'select field', id: (id + 1).toString(), fake: true });
$scope.onChange();
};
@ -215,7 +223,9 @@ function (angular, _, queryDef) {
};
$scope.init();
}
}
});
});
var module = angular.module('grafana.directives');
module.directive('elasticBucketAgg', elasticBucketAgg);
module.controller('ElasticBucketAggCtrl', ElasticBucketAggCtrl);

118
public/app/plugins/datasource/elasticsearch/metric_agg.js → public/app/plugins/datasource/elasticsearch/metric_agg.ts
View File

@ -1,31 +1,25 @@
define([
'angular',
'lodash',
'./query_def'
],
function (angular, _, queryDef) {
'use strict';
import angular from 'angular';
import _ from 'lodash';
import * as queryDef from './query_def';
var module = angular.module('grafana.directives');
export function elasticMetricAgg() {
return {
templateUrl: 'public/app/plugins/datasource/elasticsearch/partials/metric_agg.html',
controller: 'ElasticMetricAggCtrl',
restrict: 'E',
scope: {
target: '=',
index: '=',
onChange: '&',
getFields: '&',
esVersion: '=',
},
};
}
module.directive('elasticMetricAgg', function() {
return {
templateUrl: 'public/app/plugins/datasource/elasticsearch/partials/metric_agg.html',
controller: 'ElasticMetricAggCtrl',
restrict: 'E',
scope: {
target: "=",
index: "=",
onChange: "&",
getFields: "&",
esVersion: '='
}
};
});
module.controller('ElasticMetricAggCtrl', function($scope, uiSegmentSrv, $q, $rootScope) {
export class ElasticMetricAggCtrl {
constructor($scope, uiSegmentSrv, $q, $rootScope) {
var metricAggs = $scope.target.metrics;
$scope.metricAggTypes = queryDef.getMetricAggTypes($scope.esVersion);
$scope.extendedStats = queryDef.extendedStats;
$scope.pipelineAggOptions = [];
@ -41,17 +35,21 @@ function (angular, _, queryDef) {
$scope.pipelineAggOptions = queryDef.getPipelineAggOptions($scope.target);
};
$rootScope.onAppEvent('elastic-query-updated', function() {
$scope.index = _.indexOf(metricAggs, $scope.agg);
$scope.updatePipelineAggOptions();
$scope.validateModel();
}, $scope);
$rootScope.onAppEvent(
'elastic-query-updated',
function() {
$scope.index = _.indexOf(metricAggs, $scope.agg);
$scope.updatePipelineAggOptions();
$scope.validateModel();
},
$scope
);
$scope.validateModel = function() {
$scope.isFirst = $scope.index === 0;
$scope.isSingle = metricAggs.length === 1;
$scope.settingsLinkText = '';
$scope.aggDef = _.find($scope.metricAggTypes, {value: $scope.agg.type});
$scope.aggDef = _.find($scope.metricAggTypes, { value: $scope.agg.type });
if (queryDef.isPipelineAgg($scope.agg.type)) {
$scope.agg.pipelineAgg = $scope.agg.pipelineAgg || 'select metric';
@ -67,30 +65,34 @@ function (angular, _, queryDef) {
} else if (!$scope.agg.field) {
$scope.agg.field = 'select field';
}
switch($scope.agg.type) {
switch ($scope.agg.type) {
case 'cardinality': {
var precision_threshold = $scope.agg.settings.precision_threshold || '';
$scope.settingsLinkText = 'Precision threshold: ' + precision_threshold;
break;
}
case 'percentiles': {
$scope.agg.settings.percents = $scope.agg.settings.percents || [25,50,75,95,99];
$scope.agg.settings.percents = $scope.agg.settings.percents || [25, 50, 75, 95, 99];
$scope.settingsLinkText = 'Values: ' + $scope.agg.settings.percents.join(',');
break;
}
case 'extended_stats': {
if (_.keys($scope.agg.meta).length === 0) {
if (_.keys($scope.agg.meta).length === 0) {
$scope.agg.meta.std_deviation_bounds_lower = true;
$scope.agg.meta.std_deviation_bounds_upper = true;
}
var stats = _.reduce($scope.agg.meta, function(memo, val, key) {
if (val) {
var def = _.find($scope.extendedStats, {value: key});
memo.push(def.text);
}
return memo;
}, []);
var stats = _.reduce(
$scope.agg.meta,
function(memo, val, key) {
if (val) {
var def = _.find($scope.extendedStats, { value: key });
memo.push(def.text);
}
return memo;
},
[]
);
$scope.settingsLinkText = 'Stats: ' + stats.join(', ');
break;
@ -103,8 +105,8 @@ function (angular, _, queryDef) {
}
case 'raw_document': {
$scope.agg.settings.size = $scope.agg.settings.size || 500;
$scope.settingsLinkText = 'Size: ' + $scope.agg.settings.size ;
$scope.target.metrics.splice(0,$scope.target.metrics.length, $scope.agg);
$scope.settingsLinkText = 'Size: ' + $scope.agg.settings.size;
$scope.target.metrics.splice(0, $scope.target.metrics.length, $scope.agg);
$scope.target.bucketAggs = [];
break;
@ -115,7 +117,7 @@ function (angular, _, queryDef) {
// but having it like this simplifes the query_builder
var inlineScript = $scope.agg.inlineScript;
if (inlineScript) {
$scope.agg.settings.script = {inline: inlineScript};
$scope.agg.settings.script = { inline: inlineScript };
} else {
delete $scope.agg.settings.script;
}
@ -135,15 +137,15 @@ function (angular, _, queryDef) {
$scope.onChange();
};
$scope.updateMovingAvgModelSettings = function () {
$scope.updateMovingAvgModelSettings = function() {
var modelSettingsKeys = [];
var modelSettings = queryDef.getMovingAvgSettings($scope.agg.settings.model, false);
for (var i=0; i < modelSettings.length; i++) {
for (var i = 0; i < modelSettings.length; i++) {
modelSettingsKeys.push(modelSettings[i].value);
}
for (var key in $scope.agg.settings.settings) {
if (($scope.agg.settings.settings[key] === null) || (modelSettingsKeys.indexOf(key) === -1)) {
if ($scope.agg.settings.settings[key] === null || modelSettingsKeys.indexOf(key) === -1) {
delete $scope.agg.settings.settings[key];
}
}
@ -166,17 +168,21 @@ function (angular, _, queryDef) {
if ($scope.agg.type === 'cardinality') {
return $scope.getFields();
}
return $scope.getFields({$fieldType: 'number'});
return $scope.getFields({ $fieldType: 'number' });
};
$scope.addMetricAgg = function() {
var addIndex = metricAggs.length;
var id = _.reduce($scope.target.bucketAggs.concat($scope.target.metrics), function(max, val) {
return parseInt(val.id) > max ? parseInt(val.id) : max;
}, 0);
var id = _.reduce(
$scope.target.bucketAggs.concat($scope.target.metrics),
function(max, val) {
return parseInt(val.id) > max ? parseInt(val.id) : max;
},
0
);
metricAggs.splice(addIndex, 0, {type: "count", field: "select field", id: (id+1).toString()});
metricAggs.splice(addIndex, 0, { type: 'count', field: 'select field', id: (id + 1).toString() });
$scope.onChange();
};
@ -194,7 +200,9 @@ function (angular, _, queryDef) {
};
$scope.init();
}
}
});
});
var module = angular.module('grafana.directives');
module.directive('elasticMetricAgg', elasticMetricAgg);
module.controller('ElasticMetricAggCtrl', ElasticMetricAggCtrl);

476
public/app/plugins/datasource/opentsdb/datasource.js
View File

@ -1,476 +0,0 @@
define([
'angular',
'lodash',
'app/core/utils/datemath',
'moment',
],
function (angular, _, dateMath) {
'use strict';
/** @ngInject */
function OpenTsDatasource(instanceSettings, $q, backendSrv, templateSrv) {
this.type = 'opentsdb';
this.url = instanceSettings.url;
this.name = instanceSettings.name;
this.withCredentials = instanceSettings.withCredentials;
this.basicAuth = instanceSettings.basicAuth;
instanceSettings.jsonData = instanceSettings.jsonData || {};
this.tsdbVersion = instanceSettings.jsonData.tsdbVersion || 1;
this.tsdbResolution = instanceSettings.jsonData.tsdbResolution || 1;
this.supportMetrics = true;
this.tagKeys = {};
// Called once per panel (graph)
this.query = function(options) {
var start = convertToTSDBTime(options.rangeRaw.from, false);
var end = convertToTSDBTime(options.rangeRaw.to, true);
var qs = [];
_.each(options.targets, function(target) {
if (!target.metric) { return; }
qs.push(convertTargetToQuery(target, options, this.tsdbVersion));
}.bind(this));
var queries = _.compact(qs);
// No valid targets, return the empty result to save a round trip.
if (_.isEmpty(queries)) {
var d = $q.defer();
d.resolve({ data: [] });
return d.promise;
}
var groupByTags = {};
_.each(queries, function(query) {
if (query.filters && query.filters.length > 0) {
_.each(query.filters, function(val) {
groupByTags[val.tagk] = true;
});
} else {
_.each(query.tags, function(val, key) {
groupByTags[key] = true;
});
}
});
options.targets = _.filter(options.targets, function(query) {
return query.hide !== true;
});
return this.performTimeSeriesQuery(queries, start, end).then(function(response) {
var metricToTargetMapping = mapMetricsToTargets(response.data, options, this.tsdbVersion);
var result = _.map(response.data, function(metricData, index) {
index = metricToTargetMapping[index];
if (index === -1) {
index = 0;
}
this._saveTagKeys(metricData);
return transformMetricData(metricData, groupByTags, options.targets[index], options, this.tsdbResolution);
}.bind(this));
return { data: result };
}.bind(this));
};
this.annotationQuery = function(options) {
var start = convertToTSDBTime(options.rangeRaw.from, false);
var end = convertToTSDBTime(options.rangeRaw.to, true);
var qs = [];
var eventList = [];
qs.push({ aggregator:"sum", metric:options.annotation.target });
var queries = _.compact(qs);
return this.performTimeSeriesQuery(queries, start, end).then(function(results) {
if(results.data[0]) {
var annotationObject = results.data[0].annotations;
if(options.annotation.isGlobal){
annotationObject = results.data[0].globalAnnotations;
}
if(annotationObject) {
_.each(annotationObject, function(annotation) {
var event = {
text: annotation.description,
time: Math.floor(annotation.startTime) * 1000,
annotation: options.annotation
};
eventList.push(event);
});
}
}
return eventList;
}.bind(this));
};
this.targetContainsTemplate = function(target) {
if (target.filters && target.filters.length > 0) {
for (var i = 0; i < target.filters.length; i++) {
if (templateSrv.variableExists(target.filters[i].filter)) {
return true;
}
}
}
if (target.tags && Object.keys(target.tags).length > 0) {
for (var tagKey in target.tags) {
if (templateSrv.variableExists(target.tags[tagKey])) {
return true;
}
}
}
return false;
};
this.performTimeSeriesQuery = function(queries, start, end) {
var msResolution = false;
if (this.tsdbResolution === 2) {
msResolution = true;
}
var reqBody = {
start: start,
queries: queries,
msResolution: msResolution,
globalAnnotations: true
};
if (this.tsdbVersion === 3) {
reqBody.showQuery = true;
}
// Relative queries (e.g. last hour) don't include an end time
if (end) {
reqBody.end = end;
}
var options = {
method: 'POST',
url: this.url + '/api/query',
data: reqBody
};
this._addCredentialOptions(options);
return backendSrv.datasourceRequest(options);
};
this.suggestTagKeys = function(metric) {
return $q.when(this.tagKeys[metric] || []);
};
this._saveTagKeys = function(metricData) {
var tagKeys = Object.keys(metricData.tags);
_.each(metricData.aggregateTags, function(tag) {
tagKeys.push(tag);
});
this.tagKeys[metricData.metric] = tagKeys;
};
this._performSuggestQuery = function(query, type) {
return this._get('/api/suggest', {type: type, q: query, max: 1000}).then(function(result) {
return result.data;
});
};
this._performMetricKeyValueLookup = function(metric, keys) {
if(!metric || !keys) {
return $q.when([]);
}
var keysArray = keys.split(",").map(function(key) {
return key.trim();
});
var key = keysArray[0];
var keysQuery = key + "=*";
if (keysArray.length > 1) {
keysQuery += "," + keysArray.splice(1).join(",");
}
var m = metric + "{" + keysQuery + "}";
return this._get('/api/search/lookup', {m: m, limit: 3000}).then(function(result) {
result = result.data.results;
var tagvs = [];
_.each(result, function(r) {
if (tagvs.indexOf(r.tags[key]) === -1) {
tagvs.push(r.tags[key]);
}
});
return tagvs;
});
};
this._performMetricKeyLookup = function(metric) {
if(!metric) { return $q.when([]); }
return this._get('/api/search/lookup', {m: metric, limit: 1000}).then(function(result) {
result = result.data.results;
var tagks = [];
_.each(result, function(r) {
_.each(r.tags, function(tagv, tagk) {
if(tagks.indexOf(tagk) === -1) {
tagks.push(tagk);
}
});
});
return tagks;
});
};
this._get = function(relativeUrl, params) {
var options = {
method: 'GET',
url: this.url + relativeUrl,
params: params,
};
this._addCredentialOptions(options);
return backendSrv.datasourceRequest(options);
};
this._addCredentialOptions = function(options) {
if (this.basicAuth || this.withCredentials) {
options.withCredentials = true;
}
if (this.basicAuth) {
options.headers = {"Authorization": this.basicAuth};
}
};
this.metricFindQuery = function(query) {
if (!query) { return $q.when([]); }
var interpolated;
try {
interpolated = templateSrv.replace(query, {}, 'distributed');
}
catch (err) {
return $q.reject(err);
}
var responseTransform = function(result) {
return _.map(result, function(value) {
return {text: value};
});
};
var metrics_regex = /metrics\((.*)\)/;
var tag_names_regex = /tag_names\((.*)\)/;
var tag_values_regex = /tag_values\((.*?),\s?(.*)\)/;
var tag_names_suggest_regex = /suggest_tagk\((.*)\)/;
var tag_values_suggest_regex = /suggest_tagv\((.*)\)/;
var metrics_query = interpolated.match(metrics_regex);
if (metrics_query) {
return this._performSuggestQuery(metrics_query[1], 'metrics').then(responseTransform);
}
var tag_names_query = interpolated.match(tag_names_regex);
if (tag_names_query) {
return this._performMetricKeyLookup(tag_names_query[1]).then(responseTransform);
}
var tag_values_query = interpolated.match(tag_values_regex);
if (tag_values_query) {
return this._performMetricKeyValueLookup(tag_values_query[1], tag_values_query[2]).then(responseTransform);
}
var tag_names_suggest_query = interpolated.match(tag_names_suggest_regex);
if (tag_names_suggest_query) {
return this._performSuggestQuery(tag_names_suggest_query[1], 'tagk').then(responseTransform);
}
var tag_values_suggest_query = interpolated.match(tag_values_suggest_regex);
if (tag_values_suggest_query) {
return this._performSuggestQuery(tag_values_suggest_query[1], 'tagv').then(responseTransform);
}
return $q.when([]);
};
this.testDatasource = function() {
return this._performSuggestQuery('cpu', 'metrics').then(function () {
return { status: "success", message: "Data source is working" };
});
};
var aggregatorsPromise = null;
this.getAggregators = function() {
if (aggregatorsPromise) { return aggregatorsPromise; }
aggregatorsPromise = this._get('/api/aggregators').then(function(result) {
if (result.data && _.isArray(result.data)) {
return result.data.sort();
}
return [];
});
return aggregatorsPromise;
};
var filterTypesPromise = null;
this.getFilterTypes = function() {
if (filterTypesPromise) { return filterTypesPromise; }
filterTypesPromise = this._get('/api/config/filters').then(function(result) {
if (result.data) {
return Object.keys(result.data).sort();
}
return [];
});
return filterTypesPromise;
};
function transformMetricData(md, groupByTags, target, options, tsdbResolution) {
var metricLabel = createMetricLabel(md, target, groupByTags, options);
var dps = [];
// TSDB returns datapoints has a hash of ts => value.
// Can't use _.pairs(invert()) because it stringifies keys/values
_.each(md.dps, function (v, k) {
if (tsdbResolution === 2) {
dps.push([v, k * 1]);
} else {
dps.push([v, k * 1000]);
}
});
return { target: metricLabel, datapoints: dps };
}
function createMetricLabel(md, target, groupByTags, options) {
if (target.alias) {
var scopedVars = _.clone(options.scopedVars || {});
_.each(md.tags, function(value, key) {
scopedVars['tag_' + key] = {value: value};
});
return templateSrv.replace(target.alias, scopedVars);
}
var label = md.metric;
var tagData = [];
if (!_.isEmpty(md.tags)) {
_.each(_.toPairs(md.tags), function(tag) {
if (_.has(groupByTags, tag[0])) {
tagData.push(tag[0] + "=" + tag[1]);
}
});
}
if (!_.isEmpty(tagData)) {
label += "{" + tagData.join(", ") + "}";
}
return label;
}
function convertTargetToQuery(target, options, tsdbVersion) {
if (!target.metric || target.hide) {
return null;
}
var query = {
metric: templateSrv.replace(target.metric, options.scopedVars, 'pipe'),
aggregator: "avg"
};
if (target.aggregator) {
query.aggregator = templateSrv.replace(target.aggregator);
}
if (target.shouldComputeRate) {
query.rate = true;
query.rateOptions = {
counter: !!target.isCounter
};
if (target.counterMax && target.counterMax.length) {
query.rateOptions.counterMax = parseInt(target.counterMax);
}
if (target.counterResetValue && target.counterResetValue.length) {
query.rateOptions.resetValue = parseInt(target.counterResetValue);
}
if(tsdbVersion >= 2) {
query.rateOptions.dropResets = !query.rateOptions.counterMax &&
(!query.rateOptions.ResetValue || query.rateOptions.ResetValue === 0);
}
}
if (!target.disableDownsampling) {
var interval = templateSrv.replace(target.downsampleInterval || options.interval);
if (interval.match(/\.[0-9]+s/)) {
interval = parseFloat(interval)*1000 + "ms";
}
query.downsample = interval + "-" + target.downsampleAggregator;
if (target.downsampleFillPolicy && target.downsampleFillPolicy !== "none") {
query.downsample += "-" + target.downsampleFillPolicy;
}
}
if (target.filters && target.filters.length > 0) {
query.filters = angular.copy(target.filters);
if (query.filters){
for (var filter_key in query.filters) {
query.filters[filter_key].filter = templateSrv.replace(query.filters[filter_key].filter, options.scopedVars, 'pipe');
}
}
} else {
query.tags = angular.copy(target.tags);
if (query.tags){
for (var tag_key in query.tags) {
query.tags[tag_key] = templateSrv.replace(query.tags[tag_key], options.scopedVars, 'pipe');
}
}
}
if (target.explicitTags) {
query.explicitTags = true;
}
return query;
}
function mapMetricsToTargets(metrics, options, tsdbVersion) {
var interpolatedTagValue, arrTagV;
return _.map(metrics, function(metricData) {
if (tsdbVersion === 3) {
return metricData.query.index;
} else {
return _.findIndex(options.targets, function(target) {
if (target.filters && target.filters.length > 0) {
return target.metric === metricData.metric;
} else {
return target.metric === metricData.metric &&
_.every(target.tags, function(tagV, tagK) {
interpolatedTagValue = templateSrv.replace(tagV, options.scopedVars, 'pipe');
arrTagV = interpolatedTagValue.split('|');
return _.includes(arrTagV, metricData.tags[tagK]) || interpolatedTagValue === "*";
});
}
});
}
});
}
function convertToTSDBTime(date, roundUp) {
if (date === 'now') {
return null;
}
date = dateMath.parse(date, roundUp);
return date.valueOf();
}
}
return OpenTsDatasource;
});

513
public/app/plugins/datasource/opentsdb/datasource.ts Normal file
View File

@ -0,0 +1,513 @@
import angular from 'angular';
import _ from 'lodash';
import * as dateMath from 'app/core/utils/datemath';
export default class OpenTsDatasource {
type: any;
url: any;
name: any;
withCredentials: any;
basicAuth: any;
tsdbVersion: any;
tsdbResolution: any;
supportMetrics: any;
tagKeys: any;
aggregatorsPromise: any;
filterTypesPromise: any;
/** @ngInject */
constructor(instanceSettings, private $q, private backendSrv, private templateSrv) {
this.type = 'opentsdb';
this.url = instanceSettings.url;
this.name = instanceSettings.name;
this.withCredentials = instanceSettings.withCredentials;
this.basicAuth = instanceSettings.basicAuth;
instanceSettings.jsonData = instanceSettings.jsonData || {};
this.tsdbVersion = instanceSettings.jsonData.tsdbVersion || 1;
this.tsdbResolution = instanceSettings.jsonData.tsdbResolution || 1;
this.supportMetrics = true;
this.tagKeys = {};
this.aggregatorsPromise = null;
this.filterTypesPromise = null;
}
// Called once per panel (graph)
query(options) {
var start = this.convertToTSDBTime(options.rangeRaw.from, false);
var end = this.convertToTSDBTime(options.rangeRaw.to, true);
var qs = [];
_.each(
options.targets,
function(target) {
if (!target.metric) {
return;
}
qs.push(this.convertTargetToQuery(target, options, this.tsdbVersion));
}.bind(this)
);
var queries = _.compact(qs);
// No valid targets, return the empty result to save a round trip.
if (_.isEmpty(queries)) {
var d = this.$q.defer();
d.resolve({ data: [] });
return d.promise;
}
var groupByTags = {};
_.each(queries, function(query) {
if (query.filters && query.filters.length > 0) {
_.each(query.filters, function(val) {
groupByTags[val.tagk] = true;
});
} else {
_.each(query.tags, function(val, key) {
groupByTags[key] = true;
});
}
});
options.targets = _.filter(options.targets, function(query) {
return query.hide !== true;
});
return this.performTimeSeriesQuery(queries, start, end).then(
function(response) {
var metricToTargetMapping = this.mapMetricsToTargets(response.data, options, this.tsdbVersion);
var result = _.map(
response.data,
function(metricData, index) {
index = metricToTargetMapping[index];
if (index === -1) {
index = 0;
}
this._saveTagKeys(metricData);
return this.transformMetricData(
metricData,
groupByTags,
options.targets[index],
options,
this.tsdbResolution
);
}.bind(this)
);
return { data: result };
}.bind(this)
);
}
annotationQuery(options) {
var start = this.convertToTSDBTime(options.rangeRaw.from, false);
var end = this.convertToTSDBTime(options.rangeRaw.to, true);
var qs = [];
var eventList = [];
qs.push({ aggregator: 'sum', metric: options.annotation.target });
var queries = _.compact(qs);
return this.performTimeSeriesQuery(queries, start, end).then(
function(results) {
if (results.data[0]) {
var annotationObject = results.data[0].annotations;
if (options.annotation.isGlobal) {
annotationObject = results.data[0].globalAnnotations;
}
if (annotationObject) {
_.each(annotationObject, function(annotation) {
var event = {
text: annotation.description,
time: Math.floor(annotation.startTime) * 1000,
annotation: options.annotation,
};
eventList.push(event);
});
}
}
return eventList;
}.bind(this)
);
}
targetContainsTemplate(target) {
if (target.filters && target.filters.length > 0) {
for (var i = 0; i < target.filters.length; i++) {
if (this.templateSrv.variableExists(target.filters[i].filter)) {
return true;
}
}
}
if (target.tags && Object.keys(target.tags).length > 0) {
for (var tagKey in target.tags) {
if (this.templateSrv.variableExists(target.tags[tagKey])) {
return true;
}
}
}
return false;
}
performTimeSeriesQuery(queries, start, end) {
var msResolution = false;
if (this.tsdbResolution === 2) {
msResolution = true;
}
var reqBody: any = {
start: start,
queries: queries,
msResolution: msResolution,
globalAnnotations: true,
};
if (this.tsdbVersion === 3) {
reqBody.showQuery = true;
}
// Relative queries (e.g. last hour) don't include an end time
if (end) {
reqBody.end = end;
}
var options = {
method: 'POST',
url: this.url + '/api/query',
data: reqBody,
};
this._addCredentialOptions(options);
return this.backendSrv.datasourceRequest(options);
}
suggestTagKeys(metric) {
return this.$q.when(this.tagKeys[metric] || []);
}
_saveTagKeys(metricData) {
var tagKeys = Object.keys(metricData.tags);
_.each(metricData.aggregateTags, function(tag) {
tagKeys.push(tag);
});
this.tagKeys[metricData.metric] = tagKeys;
}
_performSuggestQuery(query, type) {
return this._get('/api/suggest', { type: type, q: query, max: 1000 }).then(function(result) {
return result.data;
});
}
_performMetricKeyValueLookup(metric, keys) {
if (!metric || !keys) {
return this.$q.when([]);
}
var keysArray = keys.split(',').map(function(key) {
return key.trim();
});
var key = keysArray[0];
var keysQuery = key + '=*';
if (keysArray.length > 1) {
keysQuery += ',' + keysArray.splice(1).join(',');
}
var m = metric + '{' + keysQuery + '}';
return this._get('/api/search/lookup', { m: m, limit: 3000 }).then(function(result) {
result = result.data.results;
var tagvs = [];
_.each(result, function(r) {
if (tagvs.indexOf(r.tags[key]) === -1) {
tagvs.push(r.tags[key]);
}
});
return tagvs;
});
}
_performMetricKeyLookup(metric) {
if (!metric) {
return this.$q.when([]);
}
return this._get('/api/search/lookup', { m: metric, limit: 1000 }).then(function(result) {
result = result.data.results;
var tagks = [];
_.each(result, function(r) {
_.each(r.tags, function(tagv, tagk) {
if (tagks.indexOf(tagk) === -1) {
tagks.push(tagk);
}
});
});
return tagks;
});
}
_get(relativeUrl, params?) {
var options = {
method: 'GET',
url: this.url + relativeUrl,
params: params,
};
this._addCredentialOptions(options);
return this.backendSrv.datasourceRequest(options);
}
_addCredentialOptions(options) {
if (this.basicAuth || this.withCredentials) {
options.withCredentials = true;
}
if (this.basicAuth) {
options.headers = { Authorization: this.basicAuth };
}
}
metricFindQuery(query) {
if (!query) {
return this.$q.when([]);
}
var interpolated;
try {
interpolated = this.templateSrv.replace(query, {}, 'distributed');
} catch (err) {
return this.$q.reject(err);
}
var responseTransform = function(result) {
return _.map(result, function(value) {
return { text: value };
});
};
var metrics_regex = /metrics\((.*)\)/;
var tag_names_regex = /tag_names\((.*)\)/;
var tag_values_regex = /tag_values\((.*?),\s?(.*)\)/;
var tag_names_suggest_regex = /suggest_tagk\((.*)\)/;
var tag_values_suggest_regex = /suggest_tagv\((.*)\)/;
var metrics_query = interpolated.match(metrics_regex);
if (metrics_query) {
return this._performSuggestQuery(metrics_query[1], 'metrics').then(responseTransform);
}
var tag_names_query = interpolated.match(tag_names_regex);
if (tag_names_query) {
return this._performMetricKeyLookup(tag_names_query[1]).then(responseTransform);
}
var tag_values_query = interpolated.match(tag_values_regex);
if (tag_values_query) {
return this._performMetricKeyValueLookup(tag_values_query[1], tag_values_query[2]).then(responseTransform);
}
var tag_names_suggest_query = interpolated.match(tag_names_suggest_regex);
if (tag_names_suggest_query) {
return this._performSuggestQuery(tag_names_suggest_query[1], 'tagk').then(responseTransform);
}
var tag_values_suggest_query = interpolated.match(tag_values_suggest_regex);
if (tag_values_suggest_query) {
return this._performSuggestQuery(tag_values_suggest_query[1], 'tagv').then(responseTransform);
}
return this.$q.when([]);
}
testDatasource() {
return this._performSuggestQuery('cpu', 'metrics').then(function() {
return { status: 'success', message: 'Data source is working' };
});
}
getAggregators() {
if (this.aggregatorsPromise) {
return this.aggregatorsPromise;
}
this.aggregatorsPromise = this._get('/api/aggregators').then(function(result) {
if (result.data && _.isArray(result.data)) {
return result.data.sort();
}
return [];
});
return this.aggregatorsPromise;
}
getFilterTypes() {
if (this.filterTypesPromise) {
return this.filterTypesPromise;
}
this.filterTypesPromise = this._get('/api/config/filters').then(function(result) {
if (result.data) {
return Object.keys(result.data).sort();
}
return [];
});
return this.filterTypesPromise;
}
transformMetricData(md, groupByTags, target, options, tsdbResolution) {
var metricLabel = this.createMetricLabel(md, target, groupByTags, options);
var dps = [];
// TSDB returns datapoints has a hash of ts => value.
// Can't use _.pairs(invert()) because it stringifies keys/values
_.each(md.dps, function(v, k) {
if (tsdbResolution === 2) {
dps.push([v, k * 1]);
} else {
dps.push([v, k * 1000]);
}
});
return { target: metricLabel, datapoints: dps };
}
createMetricLabel(md, target, groupByTags, options) {
if (target.alias) {
var scopedVars = _.clone(options.scopedVars || {});
_.each(md.tags, function(value, key) {
scopedVars['tag_' + key] = { value: value };
});
return this.templateSrv.replace(target.alias, scopedVars);
}
var label = md.metric;
var tagData = [];
if (!_.isEmpty(md.tags)) {
_.each(_.toPairs(md.tags), function(tag) {
if (_.has(groupByTags, tag[0])) {
tagData.push(tag[0] + '=' + tag[1]);
}
});
}
if (!_.isEmpty(tagData)) {
label += '{' + tagData.join(', ') + '}';
}
return label;
}
convertTargetToQuery(target, options, tsdbVersion) {
if (!target.metric || target.hide) {
return null;
}
var query: any = {
metric: this.templateSrv.replace(target.metric, options.scopedVars, 'pipe'),
aggregator: 'avg',
};
if (target.aggregator) {
query.aggregator = this.templateSrv.replace(target.aggregator);
}
if (target.shouldComputeRate) {
query.rate = true;
query.rateOptions = {
counter: !!target.isCounter,
};
if (target.counterMax && target.counterMax.length) {
query.rateOptions.counterMax = parseInt(target.counterMax);
}
if (target.counterResetValue && target.counterResetValue.length) {
query.rateOptions.resetValue = parseInt(target.counterResetValue);
}
if (tsdbVersion >= 2) {
query.rateOptions.dropResets =
!query.rateOptions.counterMax && (!query.rateOptions.ResetValue || query.rateOptions.ResetValue === 0);
}
}
if (!target.disableDownsampling) {
var interval = this.templateSrv.replace(target.downsampleInterval || options.interval);
if (interval.match(/\.[0-9]+s/)) {
interval = parseFloat(interval) * 1000 + 'ms';
}
query.downsample = interval + '-' + target.downsampleAggregator;
if (target.downsampleFillPolicy && target.downsampleFillPolicy !== 'none') {
query.downsample += '-' + target.downsampleFillPolicy;
}
}
if (target.filters && target.filters.length > 0) {
query.filters = angular.copy(target.filters);
if (query.filters) {
for (var filter_key in query.filters) {
query.filters[filter_key].filter = this.templateSrv.replace(
query.filters[filter_key].filter,
options.scopedVars,
'pipe'
);
}
}
} else {
query.tags = angular.copy(target.tags);
if (query.tags) {
for (var tag_key in query.tags) {
query.tags[tag_key] = this.templateSrv.replace(query.tags[tag_key], options.scopedVars, 'pipe');
}
}
}
if (target.explicitTags) {
query.explicitTags = true;
}
return query;
}
mapMetricsToTargets(metrics, options, tsdbVersion) {
var interpolatedTagValue, arrTagV;
return _.map(metrics, function(metricData) {
if (tsdbVersion === 3) {
return metricData.query.index;
} else {
return _.findIndex(options.targets, function(target) {
if (target.filters && target.filters.length > 0) {
return target.metric === metricData.metric;
} else {
return (
target.metric === metricData.metric &&
_.every(target.tags, function(tagV, tagK) {
interpolatedTagValue = this.templateSrv.replace(tagV, options.scopedVars, 'pipe');
arrTagV = interpolatedTagValue.split('|');
return _.includes(arrTagV, metricData.tags[tagK]) || interpolatedTagValue === '*';
})
);
}
});
}
});
}
convertToTSDBTime(date, roundUp) {
if (date === 'now') {
return null;
}
date = dateMath.parse(date, roundUp);
return date.valueOf();
}
}