IntenseWebs/grafana
3
0
Fork 0
mirror of https://github.com/grafana/grafana.git synced 2026-07-30 08:18:10 -05:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #14711 from ander26/nanosecond-postgresql

Browse Source 
sql datasources: nanosecond timestamp support
This commit is contained in:
Marcus Efraimsson
2019-01-28 20:26:45 +01:00
committed by GitHub
parent f09c4608c2 7023c957d7
commit fce354875c
15 changed files with 140 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/sources/features/datasources/mssql.md
+3
View File
@@ -110,6 +110,9 @@ Macro example | Description
*$__unixEpochFilter(dateColumn)* | Will be replaced by a time range filter using the specified column name with times represented as unix timestamp. For example, *dateColumn > 1494410783 AND dateColumn < 1494497183*
*$__unixEpochFrom()* | Will be replaced by the start of the currently active time selection as unix timestamp. For example, *1494410783*
*$__unixEpochTo()* | Will be replaced by the end of the currently active time selection as unix timestamp. For example, *1494497183*
*$__unixEpochNanoFilter(dateColumn)* | Will be replaced by a time range filter using the specified column name with times represented as nanosecond timestamp. For example, *dateColumn > 1494410783152415214 AND dateColumn < 1494497183142514872*
*$__unixEpochNanoFrom()* | Will be replaced by the start of the currently active time selection as nanosecond timestamp. For example, *1494410783152415214*
*$__unixEpochNanoTo()* | Will be replaced by the end of the currently active time selection as nanosecond timestamp. For example, *1494497183142514872*
*$__unixEpochGroup(dateColumn,'5m', [fillmode])* | Same as $__timeGroup but for times stored as unix timestamp (only available in Grafana 5.3+).
*$__unixEpochGroupAlias(dateColumn,'5m', [fillmode])* | Same as above but also adds a column alias (only available in Grafana 5.3+).
docs/sources/features/datasources/mysql.md
+3
View File
@@ -144,6 +144,9 @@ Macro example | Description
*$__unixEpochFilter(dateColumn)* | Will be replaced by a time range filter using the specified column name with times represented as unix timestamp. For example, *dateColumn > 1494410783 AND dateColumn < 1494497183*
*$__unixEpochFrom()* | Will be replaced by the start of the currently active time selection as unix timestamp. For example, *1494410783*
*$__unixEpochTo()* | Will be replaced by the end of the currently active time selection as unix timestamp. For example, *1494497183*
*$__unixEpochNanoFilter(dateColumn)* | Will be replaced by a time range filter using the specified column name with times represented as nanosecond timestamp. For example, *dateColumn > 1494410783152415214 AND dateColumn < 1494497183142514872*
*$__unixEpochNanoFrom()* | Will be replaced by the start of the currently active time selection as nanosecond timestamp. For example, *1494410783152415214*
*$__unixEpochNanoTo()* | Will be replaced by the end of the currently active time selection as nanosecond timestamp. For example, *1494497183142514872*
*$__unixEpochGroup(dateColumn,'5m', [fillmode])* | Same as $__timeGroup but for times stored as unix timestamp (only available in Grafana 5.3+).
*$__unixEpochGroupAlias(dateColumn,'5m', [fillmode])* | Same as above but also adds a column alias (only available in Grafana 5.3+).
docs/sources/features/datasources/postgres.md
+3
View File
@@ -154,6 +154,9 @@ Macro example | Description
*$__unixEpochFilter(dateColumn)* | Will be replaced by a time range filter using the specified column name with times represented as unix timestamps. For example, *dateColumn >= 1494410783 AND dateColumn <= 1494497183*
*$__unixEpochFrom()* | Will be replaced by the start of the currently active time selection as unix timestamp. For example, *1494410783*
*$__unixEpochTo()* | Will be replaced by the end of the currently active time selection as unix timestamp. For example, *1494497183*
*$__unixEpochNanoFilter(dateColumn)* | Will be replaced by a time range filter using the specified column name with times represented as nanosecond timestamps. For example, *dateColumn >= 1494410783152415214 AND dateColumn <= 1494497183142514872*
*$__unixEpochNanoFrom()* | Will be replaced by the start of the currently active time selection as nanosecond timestamp. For example, *1494410783152415214*
*$__unixEpochNanoTo()* | Will be replaced by the end of the currently active time selection as unix timestamp. For example, *1494497183142514872*
*$__unixEpochGroup(dateColumn,'5m', [fillmode])* | Same as $__timeGroup, but for times stored as unix timestamp (only available in Grafana 5.3+).
*$__unixEpochGroupAlias(dateColumn,'5m', [fillmode])* | Same as above, but also adds a column alias (only available in Grafana 5.3+).
pkg/tsdb/mssql/macros.go
+9
View File
@@ -96,6 +96,15 @@ func (m *msSqlMacroEngine) evaluateMacro(name string, args []string) (string, er
return "", fmt.Errorf("missing time column argument for macro %v", name)
}
return fmt.Sprintf("%s >= %d AND %s <= %d", args[0], m.timeRange.GetFromAsSecondsEpoch(), args[0], m.timeRange.GetToAsSecondsEpoch()), nil
case "__unixEpochNanoFilter":
if len(args) == 0 {
return "", fmt.Errorf("missing time column argument for macro %v", name)
}
return fmt.Sprintf("%s >= %d AND %s <= %d", args[0], m.timeRange.GetFromAsTimeUTC().UnixNano(), args[0], m.timeRange.GetToAsTimeUTC().UnixNano()), nil
case "__unixEpochNanoFrom":
return fmt.Sprintf("%d", m.timeRange.GetFromAsTimeUTC().UnixNano()), nil
case "__unixEpochNanoTo":
return fmt.Sprintf("%d", m.timeRange.GetToAsTimeUTC().UnixNano()), nil
case "__unixEpochGroup":
if len(args) < 2 {
return "", fmt.Errorf("macro %v needs time column and interval and optional fill value", name)
pkg/tsdb/mssql/macros_test.go
+34
View File
@@ -132,6 +132,26 @@ func TestMacroEngine(t *testing.T) {
So(sql, ShouldEqual, fmt.Sprintf("select time_column >= %d AND time_column <= %d", from.Unix(), to.Unix()))
})
Convey("interpolate __unixEpochNanoFilter function", func() {
sql, err := engine.Interpolate(query, timeRange, "select $__unixEpochNanoFilter(time_column)")
So(err, ShouldBeNil)
So(sql, ShouldEqual, fmt.Sprintf("select time_column >= %d AND time_column <= %d", from.UnixNano(), to.UnixNano()))
})
Convey("interpolate __unixEpochNanoFrom function", func() {
sql, err := engine.Interpolate(query, timeRange, "select $__unixEpochNanoFrom()")
So(err, ShouldBeNil)
So(sql, ShouldEqual, fmt.Sprintf("select %d", from.UnixNano()))
})
Convey("interpolate __unixEpochNanoTo function", func() {
sql, err := engine.Interpolate(query, timeRange, "select $__unixEpochNanoTo()")
So(err, ShouldBeNil)
So(sql, ShouldEqual, fmt.Sprintf("select %d", to.UnixNano()))
})
Convey("interpolate __unixEpochGroup function", func() {
sql, err := engine.Interpolate(query, timeRange, "SELECT $__unixEpochGroup(time_column,'5m')")
@@ -163,6 +183,13 @@ func TestMacroEngine(t *testing.T) {
So(sql, ShouldEqual, fmt.Sprintf("select time_column >= %d AND time_column <= %d", from.Unix(), to.Unix()))
})
Convey("interpolate __unixEpochNanoFilter function", func() {
sql, err := engine.Interpolate(query, timeRange, "select $__unixEpochNanoFilter(time_column)")
So(err, ShouldBeNil)
So(sql, ShouldEqual, fmt.Sprintf("select time_column >= %d AND time_column <= %d", from.UnixNano(), to.UnixNano()))
})
})
Convey("Given a time range between 1960-02-01 07:00 and 1980-02-03 08:00", func() {
@@ -183,6 +210,13 @@ func TestMacroEngine(t *testing.T) {
So(sql, ShouldEqual, fmt.Sprintf("select time_column >= %d AND time_column <= %d", from.Unix(), to.Unix()))
})
Convey("interpolate __unixEpochNanoFilter function", func() {
sql, err := engine.Interpolate(query, timeRange, "select $__unixEpochNanoFilter(time_column)")
So(err, ShouldBeNil)
So(sql, ShouldEqual, fmt.Sprintf("select time_column >= %d AND time_column <= %d", from.UnixNano(), to.UnixNano()))
})
})
})
}
pkg/tsdb/mysql/macros.go
+9
View File
@@ -91,6 +91,15 @@ func (m *mySqlMacroEngine) evaluateMacro(name string, args []string) (string, er
return "", fmt.Errorf("missing time column argument for macro %v", name)
}
return fmt.Sprintf("%s >= %d AND %s <= %d", args[0], m.timeRange.GetFromAsSecondsEpoch(), args[0], m.timeRange.GetToAsSecondsEpoch()), nil
case "__unixEpochNanoFilter":
if len(args) == 0 {
return "", fmt.Errorf("missing time column argument for macro %v", name)
}
return fmt.Sprintf("%s >= %d AND %s <= %d", args[0], m.timeRange.GetFromAsTimeUTC().UnixNano(), args[0], m.timeRange.GetToAsTimeUTC().UnixNano()), nil
case "__unixEpochNanoFrom":
return fmt.Sprintf("%d", m.timeRange.GetFromAsTimeUTC().UnixNano()), nil
case "__unixEpochNanoTo":
return fmt.Sprintf("%d", m.timeRange.GetToAsTimeUTC().UnixNano()), nil
case "__unixEpochGroup":
if len(args) < 2 {
return "", fmt.Errorf("macro %v needs time column and interval and optional fill value", name)
pkg/tsdb/mysql/macros_test.go
+21
View File
@@ -84,6 +84,27 @@ func TestMacroEngine(t *testing.T) {
So(sql, ShouldEqual, fmt.Sprintf("select time >= %d AND time <= %d", from.Unix(), to.Unix()))
})
Convey("interpolate __unixEpochNanoFilter function", func() {
sql, err := engine.Interpolate(query, timeRange, "select $__unixEpochNanoFilter(time)")
So(err, ShouldBeNil)
So(sql, ShouldEqual, fmt.Sprintf("select time >= %d AND time <= %d", from.UnixNano(), to.UnixNano()))
})
Convey("interpolate __unixEpochNanoFrom function", func() {
sql, err := engine.Interpolate(query, timeRange, "select $__unixEpochNanoFrom()")
So(err, ShouldBeNil)
So(sql, ShouldEqual, fmt.Sprintf("select %d", from.UnixNano()))
})
Convey("interpolate __unixEpochNanoTo function", func() {
sql, err := engine.Interpolate(query, timeRange, "select $__unixEpochNanoTo()")
So(err, ShouldBeNil)
So(sql, ShouldEqual, fmt.Sprintf("select %d", to.UnixNano()))
})
Convey("interpolate __unixEpochGroup function", func() {
sql, err := engine.Interpolate(query, timeRange, "SELECT $__unixEpochGroup(time_column,'5m')")
pkg/tsdb/postgres/macros.go
+9
View File
@@ -122,6 +122,15 @@ func (m *postgresMacroEngine) evaluateMacro(name string, args []string) (string,
return "", fmt.Errorf("missing time column argument for macro %v", name)
}
return fmt.Sprintf("%s >= %d AND %s <= %d", args[0], m.timeRange.GetFromAsSecondsEpoch(), args[0], m.timeRange.GetToAsSecondsEpoch()), nil
case "__unixEpochNanoFilter":
if len(args) == 0 {
return "", fmt.Errorf("missing time column argument for macro %v", name)
}
return fmt.Sprintf("%s >= %d AND %s <= %d", args[0], m.timeRange.GetFromAsTimeUTC().UnixNano(), args[0], m.timeRange.GetToAsTimeUTC().UnixNano()), nil
case "__unixEpochNanoFrom":
return fmt.Sprintf("%d", m.timeRange.GetFromAsTimeUTC().UnixNano()), nil
case "__unixEpochNanoTo":
return fmt.Sprintf("%d", m.timeRange.GetToAsTimeUTC().UnixNano()), nil
case "__unixEpochGroup":
if len(args) < 2 {
return "", fmt.Errorf("macro %v needs time column and interval and optional fill value", name)
pkg/tsdb/postgres/macros_test.go
+31
View File
@@ -115,6 +115,25 @@ func TestMacroEngine(t *testing.T) {
So(sql, ShouldEqual, fmt.Sprintf("select time >= %d AND time <= %d", from.Unix(), to.Unix()))
})
Convey("interpolate __unixEpochNanoFilter function", func() {
sql, err := engine.Interpolate(query, timeRange, "select $__unixEpochNanoFilter(time)")
So(err, ShouldBeNil)
So(sql, ShouldEqual, fmt.Sprintf("select time >= %d AND time <= %d", from.UnixNano(), to.UnixNano()))
})
Convey("interpolate __unixEpochNanoFrom function", func() {
sql, err := engine.Interpolate(query, timeRange, "select $__unixEpochNanoFrom()")
So(err, ShouldBeNil)
So(sql, ShouldEqual, fmt.Sprintf("select %d", from.UnixNano()))
})
Convey("interpolate __unixEpochNanoTo function", func() {
sql, err := engine.Interpolate(query, timeRange, "select $__unixEpochNanoTo()")
So(err, ShouldBeNil)
So(sql, ShouldEqual, fmt.Sprintf("select %d", to.UnixNano()))
})
Convey("interpolate __unixEpochGroup function", func() {
@@ -147,6 +166,12 @@ func TestMacroEngine(t *testing.T) {
So(sql, ShouldEqual, fmt.Sprintf("select time >= %d AND time <= %d", from.Unix(), to.Unix()))
})
Convey("interpolate __unixEpochNanoFilter function", func() {
sql, err := engine.Interpolate(query, timeRange, "select $__unixEpochNanoFilter(time)")
So(err, ShouldBeNil)
So(sql, ShouldEqual, fmt.Sprintf("select time >= %d AND time <= %d", from.UnixNano(), to.UnixNano()))
})
})
Convey("Given a time range between 1960-02-01 07:00 and 1980-02-03 08:00", func() {
@@ -167,6 +192,12 @@ func TestMacroEngine(t *testing.T) {
So(sql, ShouldEqual, fmt.Sprintf("select time >= %d AND time <= %d", from.Unix(), to.Unix()))
})
Convey("interpolate __unixEpochNanoFilter function", func() {
sql, err := engine.Interpolate(query, timeRange, "select $__unixEpochNanoFilter(time)")
So(err, ShouldBeNil)
So(sql, ShouldEqual, fmt.Sprintf("select time >= %d AND time <= %d", from.UnixNano(), to.UnixNano()))
})
})
Convey("Given a time range between 1960-02-01 07:00:00.5 and 1980-02-03 08:00:00.5", func() {
public/app/plugins/datasource/mssql/partials/annotations.editor.html
+3
View File
@@ -30,12 +30,15 @@ Macros:
- $__timeEpoch(column) -&gt; DATEDIFF(second, '1970-01-01', column) AS time
- $__timeFilter(column) -&gt; column BETWEEN '2017-04-21T05:01:17Z' AND '2017-04-21T05:01:17Z'
- $__unixEpochFilter(column) -&gt; column &gt;= 1492750877 AND column &lt;= 1492750877
- $__unixEpochNanoFilter(column) -&gt; column &gt;= 1494410783152415214 AND column &lt;= 1494497183142514872
Or build your own conditionals using these macros which just return the values:
- $__timeFrom() -&gt; '2017-04-21T05:01:17Z'
- $__timeTo() -&gt; '2017-04-21T05:01:17Z'
- $__unixEpochFrom() -&gt; 1492750877
- $__unixEpochTo() -&gt; 1492750877
- $__unixEpochNanoFrom() -&gt; 1494410783152415214
- $__unixEpochNanoTo() -&gt; 1494497183142514872
</pre>
</div>
</div>
public/app/plugins/datasource/mssql/partials/query.editor.html
+3
View File
@@ -55,6 +55,7 @@ Macros:
- $__timeEpoch(column) -&gt; DATEDIFF(second, '1970-01-01', column) AS time
- $__timeFilter(column) -&gt; column BETWEEN '2017-04-21T05:01:17Z' AND '2017-04-21T05:01:17Z'
- $__unixEpochFilter(column) -&gt; column &gt;= 1492750877 AND column &lt;= 1492750877
- $__unixEpochNanoFilter(column) -&gt; column &gt;= 1494410783152415214 AND column &lt;= 1494497183142514872
- $__timeGroup(column, '5m'[, fillvalue]) -&gt; CAST(ROUND(DATEDIFF(second, '1970-01-01', column)/300.0, 0) as bigint)*300.
by setting fillvalue grafana will fill in missing values according to the interval
fillvalue can be either a literal value, NULL or previous; previous will fill in the previous seen value or NULL if none has been seen yet
@@ -75,6 +76,8 @@ Or build your own conditionals using these macros which just return the values:
- $__timeTo() -&gt; '2017-04-21T05:01:17Z'
- $__unixEpochFrom() -&gt; 1492750877
- $__unixEpochTo() -&gt; 1492750877
- $__unixEpochNanoFrom() -&gt; 1494410783152415214
- $__unixEpochNanoTo() -&gt; 1494497183142514872
</pre>
</div>
public/app/plugins/datasource/mysql/partials/annotations.editor.html
+3
View File
@@ -30,12 +30,15 @@ Macros:
- $__timeEpoch(column) -&gt; UNIX_TIMESTAMP(column) as time (or as time_sec)
- $__timeFilter(column) -&gt; column BETWEEN FROM_UNIXTIME(1492750877) AND FROM_UNIXTIME(1492750877)
- $__unixEpochFilter(column) -&gt; time_unix_epoch &gt; 1492750877 AND time_unix_epoch &lt; 1492750877
- $__unixEpochNanoFilter(column) -&gt; column &gt;= 1494410783152415214 AND column &lt;= 1494497183142514872
Or build your own conditionals using these macros which just return the values:
- $__timeFrom() -&gt; FROM_UNIXTIME(1492750877)
- $__timeTo() -&gt; FROM_UNIXTIME(1492750877)
- $__unixEpochFrom() -&gt; 1492750877
- $__unixEpochTo() -&gt; 1492750877
- $__unixEpochNanoFrom() -&gt; 1494410783152415214
- $__unixEpochNanoTo() -&gt; 1494497183142514872
</pre>
</div>
</div>
public/app/plugins/datasource/mysql/partials/query.editor.html
+3
View File
@@ -153,6 +153,7 @@ Macros:
- $__timeEpoch(column) -&gt; UNIX_TIMESTAMP(column) as time_sec
- $__timeFilter(column) -&gt; column BETWEEN FROM_UNIXTIME(1492750877) AND FROM_UNIXTIME(1492750877)
- $__unixEpochFilter(column) -&gt; time_unix_epoch &gt; 1492750877 AND time_unix_epoch &lt; 1492750877
- $__unixEpochNanoFilter(column) -&gt; column &gt;= 1494410783152415214 AND column &lt;= 1494497183142514872
- $__timeGroup(column,'5m'[, fillvalue]) -&gt; cast(cast(UNIX_TIMESTAMP(column)/(300) as signed)*300 as signed)
by setting fillvalue grafana will fill in missing values according to the interval
fillvalue can be either a literal value, NULL or previous; previous will fill in the previous seen value or NULL if none has been seen yet
@@ -173,6 +174,8 @@ Or build your own conditionals using these macros which just return the values:
- $__timeTo() -&gt; FROM_UNIXTIME(1492750877)
- $__unixEpochFrom() -&gt; 1492750877
- $__unixEpochTo() -&gt; 1492750877
- $__unixEpochNanoFrom() -&gt; 1494410783152415214
- $__unixEpochNanoTo() -&gt; 1494497183142514872
</pre>
</div>
public/app/plugins/datasource/postgres/partials/annotations.editor.html
+3
View File
@@ -30,12 +30,15 @@ Macros:
- $__timeEpoch -&gt; extract(epoch from column) as "time"
- $__timeFilter(column) -&gt; column BETWEEN '2017-04-21T05:01:17Z' AND '2017-04-21T05:01:17Z'
- $__unixEpochFilter(column) -&gt; column &gt;= 1492750877 AND column &lt;= 1492750877
- $__unixEpochNanoFilter(column) -&gt; column &gt;= 1494410783152415214 AND column &lt;= 1494497183142514872
Or build your own conditionals using these macros which just return the values:
- $__timeFrom() -&gt; '2017-04-21T05:01:17Z'
- $__timeTo() -&gt; '2017-04-21T05:01:17Z'
- $__unixEpochFrom() -&gt; 1492750877
- $__unixEpochTo() -&gt; 1492750877
- $__unixEpochNanoFrom() -&gt; 1494410783152415214
- $__unixEpochNanoTo() -&gt; 1494497183142514872
</pre>
</div>
</div>
public/app/plugins/datasource/postgres/partials/query.editor.html
+3
View File
@@ -153,6 +153,7 @@ Macros:
- $__timeEpoch -&gt; extract(epoch from column) as "time"
- $__timeFilter(column) -&gt; column BETWEEN '2017-04-21T05:01:17Z' AND '2017-04-21T05:01:17Z'
- $__unixEpochFilter(column) -&gt; column &gt;= 1492750877 AND column &lt;= 1492750877
- $__unixEpochNanoFilter(column) -&gt; column &gt;= 1494410783152415214 AND column &lt;= 1494497183142514872
- $__timeGroup(column,'5m'[, fillvalue]) -&gt; (extract(epoch from column)/300)::bigint*300
by setting fillvalue grafana will fill in missing values according to the interval
fillvalue can be either a literal value, NULL or previous; previous will fill in the previous seen value or NULL if none has been seen yet
@@ -173,6 +174,8 @@ Or build your own conditionals using these macros which just return the values:
- $__timeTo() -&gt; '2017-04-21T05:01:17Z'
- $__unixEpochFrom() -&gt; 1492750877
- $__unixEpochTo() -&gt; 1492750877
- $__unixEpochNanoFrom() -&gt; 1494410783152415214
- $__unixEpochNanoTo() -&gt; 1494497183142514872
</pre>
</div>