grafana/pkg/services/live/live.go
Ryan McKinley 2aafa39879
Live: support real time measurements (alpha) (#28022)
* improve reduce transformer

* add measurment classes

* sync with new grafana measure format

* use address for live

* use plural in URL

* set the field name

* fix build

* find changes

* POST http to channel

* Yarn: Update lock file (#28014)

* Loki: Run instant query only in Explore (#27974)

* Run instant query only in Explore

* Replace forEach with for loop

* don't cast

* Docs: Fixed row display in table (#28031)

* Plugins: Let descendant plugins inherit their root's signature (#27970)

* plugins: Let descendant plugins inherit their root's signature

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Registry: Fix service shutdown mode trigger location (#28025)

* Add Alex Khomenko as member (#28032)

* show history

* fix confirm

* fix confirm

* add tests

* fix lint

* add more errors

* set values

* remove unrelated changes

* unrelated changes

* Update pkg/models/live.go

Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>

* Update pkg/models/live.go

Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>

* Update pkg/services/live/live.go

Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>

* Update pkg/services/live/pluginHandler.go

Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>

* Update pkg/services/live/pluginHandler.go

Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>

* Update pkg/services/live/pluginHandler.go

Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>

* use measurments for testdata endpoints

* add live to testdata

* add live to testdata

* Update pkg/services/live/channel.go

Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>

* Apply suggestions from code review

Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>

* update comment formats

* uprevert testdata

* Apply suggestions from code review

Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
Co-authored-by: Hugo Häggmark <hugo.haggmark@grafana.com>

* Apply suggestions from code review

* CloudWatch: Add EC2CapacityReservations Namespace (#28309)

* API: Fix short URLs (#28300)

* API: Fix short URLs

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Chore: Add cloud-middleware as code owners (#28310)

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* SQLStore: Run tests as integration tests (#28265)

* sqlstore: Run tests as integration tests

* Truncate database instead of re-creating it on each test

* Fix test description

See https://github.com/grafana/grafana/pull/12129

* Fix lint issues

* Fix postgres dialect after review suggestion

* Rename and document functions after review suggestion

* Add periods

* Fix auto-increment value for mysql dialect

Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>

* Drone: Fix grafana-mixin linting (#28308)

* Drone: Fix Starlark script

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* grafana-mixin: Move build logic to scripts

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Drone: Use mixin scripts

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* CI build image: Install jsonnetfmt and mixtool

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Makefile: Print commands

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* should only ignore the file in the grafana mixin root folder (#28306)

Signed-off-by: bergquist <carl.bergquist@gmail.com>

* fix: for graph size not taking up full height or width

* Graph NG: fix toggling queries and extract Graph component from graph3 panel (#28290)

* Fix issue when data and config is not in sync

* Extract GraphNG component from graph panel and add some tests coverage

* Update packages/grafana-ui/src/components/uPlot/hooks.test.ts

* Update packages/grafana-ui/src/components/uPlot/hooks.test.ts

* Update packages/grafana-ui/src/components/uPlot/hooks.test.ts

* Fix grid color and annotations refresh

* Drone: Use ${DRONE_TAG} in release pipelines, since it should work (#28299)

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Explore: respect min_refresh_interval (#27988)

* Explore: respect min_refresh_interval

Fixes #27494

* fixup! Explore: respect min_refresh_interval

* fixup! Explore: respect min_refresh_interval

* UI: export defaultIntervals from refresh picker

* fixup! Explore: respect min_refresh_interval

Co-authored-by: Zoltán Bedi <zoltan.bedi@gmail.com>

* Loki: Base maxDataPoints limits on query type (#28298)

* Base maxLines and maxDataPoints based on query type

* Allow overriding the limit to higher value

* Bump tree-kill from 1.2.1 to 1.2.2 (#27405)

Bumps [tree-kill](https://github.com/pkrumins/node-tree-kill) from 1.2.1 to 1.2.2.
- [Release notes](https://github.com/pkrumins/node-tree-kill/releases)
- [Commits](https://github.com/pkrumins/node-tree-kill/compare/v1.2.1...v1.2.2)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump handlebars from 4.4.3 to 4.7.6 (#27416)

Bumps [handlebars](https://github.com/wycats/handlebars.js) from 4.4.3 to 4.7.6.
- [Release notes](https://github.com/wycats/handlebars.js/releases)
- [Changelog](https://github.com/handlebars-lang/handlebars.js/blob/master/release-notes.md)
- [Commits](https://github.com/wycats/handlebars.js/compare/v4.4.3...v4.7.6)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Build(deps): Bump http-proxy from 1.18.0 to 1.18.1 (#27507)

Bumps [http-proxy](https://github.com/http-party/node-http-proxy) from 1.18.0 to 1.18.1.
- [Release notes](https://github.com/http-party/node-http-proxy/releases)
- [Changelog](https://github.com/http-party/node-http-proxy/blob/master/CHANGELOG.md)
- [Commits](https://github.com/http-party/node-http-proxy/compare/1.18.0...1.18.1)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Automation: Add backport github action (#28318)

* BackendSrv: Fixes queue countdown when unsubscribe is before response (#28323)

* GraphNG: Use AxisSide enum (#28320)

* IssueTriage: Needs more info automation and messages (#28137)

* IssueTriage: Needs more info automation and messages

* Updated

* Updated

* Updated wording

* SAML: IdP-initiated SSO docs (#28280)

* SAML: IdP-initiated SSO docs

* Update docs/sources/enterprise/saml.md

Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>

* Apply suggestions from code review

Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>

Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>

* Loki: Run instant query only when doing metric query (#28325)

* Run instant query only when doing metric query

* Update public/app/plugins/datasource/loki/datasource.ts

Co-authored-by: Ivana Huckova <30407135+ivanahuckova@users.noreply.github.com>

Co-authored-by: Ivana Huckova <30407135+ivanahuckova@users.noreply.github.com>

* Automation: Tweaks to more info message (#28332)

* AlertingNG: remove warn/crit from eval prototype (#28334)

and misc cleanup

* area/grafana/toolkit: update e2e docker image (#28335)

* add xvfb to image

* comment out toolkit inclusion

* add latest tag

* update packages for cypress

* cleanup script

* Update auth-proxy.md (#28339)

Fix a minor grammar mistake: 'handling' to 'handle'.

* Git: Create .gitattributes for windows line endings (#28340)

With this set, Windows users will have text files converted from Windows style line endings (\r\n) to Unix style line endings (\n) when they’re added to the repository.
https://www.edwardthomson.com/blog/git_for_windows_line_endings.html

* Docs: Add docs for valuepicker (#28327)

* Templating: Replace all '$tag' in tag values query (#28343)

* Docs: Add missing records from grafana-ui 7.2.1 CHANGELOG (#28302)

* Dashboard links: Places drop down list so it's always visible (#28330)

* calculating whether to place the list on the right or left edge of the parent

* change naming and add import of createRef

* Automation: Update backport github action trigger (#28352)

It seems like GitHub has solved the problem of running github actions on PRs from forks with access to secrets. 

https://github.blog/2020-08-03-github-actions-improvements-for-fork-and-pull-request-workflows/#improvements-for-public-repository-forks

If I change the event that triggers it to pull_request_target the action is run in the context of the base instead of the merged PR branch

* ColorSchemes: Adds more color schemes and text colors that depend on the background (#28305)

* Adding more color modes and text colors that depend on the background color

* Updates

* Updated

* Another big value fix

* Fixing unit tests

* Updated

* Updated test

* Update

* Updated

* Updated

* Updated

* Updated

* Added new demo dashboard

* Updated

* updated

* Updated

* Updateed

* added beta notice

* Fixed e2e test

* Fix typos

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* revert pseduo code

* apply feedback

* remove HTTP for now

* fix backend test

* change to datasource

* clear input for streams

* fix docs?

* consistent measure vs measurements

* better jsdocs

* fix a few jsdoc errors

* fix comment style

* Remove commented out code

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Clean up code

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Clean up code

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Clean up code

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Clean up code

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Clean up code

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Clean up code

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Clean up code

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Update pkg/models/live.go

Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>

* Fix build

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* set the stringField

Co-authored-by: Torkel Ödegaard <torkel@grafana.org>
Co-authored-by: Ivana Huckova <30407135+ivanahuckova@users.noreply.github.com>
Co-authored-by: ozhuang <ozhuang.95@gmail.com>
Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>
Co-authored-by: Amos Law <ahlaw.dev@gmail.com>
Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>
Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
Co-authored-by: Hugo Häggmark <hugo.haggmark@grafana.com>
Co-authored-by: The Rock Guy <fabian.bracco@gvcgroup.com.au>
Co-authored-by: Sofia Papagiannaki <papagian@users.noreply.github.com>
Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>
Co-authored-by: Carl Bergquist <carl@grafana.com>
Co-authored-by: Jack Westbrook <jack.westbrook@gmail.com>
Co-authored-by: Dominik Prokop <dominik.prokop@grafana.com>
Co-authored-by: Elliot Pryde <elliot.pryde@elliotpryde.com>
Co-authored-by: Zoltán Bedi <zoltan.bedi@gmail.com>
Co-authored-by: Andrej Ocenas <mr.ocenas@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Alexander Zobnin <alexanderzobnin@gmail.com>
Co-authored-by: Kyle Brandt <kyle@grafana.com>
Co-authored-by: Brian Gann <briangann@users.noreply.github.com>
Co-authored-by: J-F-Far <joel.f.farthing@gmail.com>
Co-authored-by: acoder77 <73009264+acoder77@users.noreply.github.com>
Co-authored-by: Peter Holmberg <peterholmberg@users.noreply.github.com>
Co-authored-by: Krzysztof Dąbrowski <krzysdabro@live.com>
Co-authored-by: maknik <mooniczkam@gmail.com>
2020-10-22 09:10:26 +02:00

323 lines
8.9 KiB
Go

package live
import (
"encoding/json"
"fmt"
"strings"
"sync"
"github.com/centrifugal/centrifuge"
"github.com/grafana/grafana/pkg/infra/log"
"github.com/grafana/grafana/pkg/models"
"github.com/grafana/grafana/pkg/plugins"
"github.com/grafana/grafana/pkg/services/live/features"
)
var (
logger = log.New("live")
loggerCF = log.New("live.centrifuge")
)
// CoreGrafanaScope list of core features
type CoreGrafanaScope struct {
Features map[string]models.ChannelHandlerFactory
// The generic service to advertise dashboard changes
Dashboards models.DashboardActivityChannel
}
// GrafanaLive pretends to be the server
type GrafanaLive struct {
node *centrifuge.Node
// The websocket handler
WebsocketHandler interface{}
// Full channel handler
channels map[string]models.ChannelHandler
channelsMu sync.RWMutex
// The core internal features
GrafanaScope CoreGrafanaScope
}
// InitializeBroker initializes the broker and starts listening for requests.
func InitializeBroker() (*GrafanaLive, error) {
glive := &GrafanaLive{
channels: make(map[string]models.ChannelHandler),
channelsMu: sync.RWMutex{},
GrafanaScope: CoreGrafanaScope{
Features: make(map[string]models.ChannelHandlerFactory),
},
}
// We use default config here as starting point. Default config contains
// reasonable values for available options.
cfg := centrifuge.DefaultConfig
// cfg.LogLevel = centrifuge.LogLevelDebug
cfg.LogHandler = handleLog
// This function is called fast and often -- it must be sychronized
cfg.ChannelOptionsFunc = func(channel string) (centrifuge.ChannelOptions, bool, error) {
handler, err := glive.GetChannelHandler(channel)
if err != nil {
logger.Error("ChannelOptionsFunc", "channel", channel, "err", err)
if err.Error() == "404" { // ????
return centrifuge.ChannelOptions{}, false, nil
}
return centrifuge.ChannelOptions{}, true, err
}
opts := handler.GetChannelOptions(channel)
return opts, true, nil
}
// Node is the core object in Centrifuge library responsible for many useful
// things. For example Node allows to publish messages to channels from server
// side with its Publish method, but in this example we will publish messages
// only from client side.
node, err := centrifuge.New(cfg)
if err != nil {
return nil, err
}
glive.node = node
// Initialize the main features
dash := &features.DashboardHandler{
Publisher: glive.Publish,
}
glive.GrafanaScope.Dashboards = dash
glive.GrafanaScope.Features["dashboard"] = dash
glive.GrafanaScope.Features["testdata"] = &features.TestDataSupplier{
Publisher: glive.Publish,
}
glive.GrafanaScope.Features["broadcast"] = &features.BroadcastRunner{}
glive.GrafanaScope.Features["measurements"] = &features.MeasurementsRunner{}
// Set ConnectHandler called when client successfully connected to Node. Your code
// inside handler must be synchronized since it will be called concurrently from
// different goroutines (belonging to different client connections). This is also
// true for other event handlers.
node.OnConnect(func(c *centrifuge.Client) {
// In our example transport will always be Websocket but it can also be SockJS.
transportName := c.Transport().Name()
// In our example clients connect with JSON protocol but it can also be Protobuf.
transportEncoding := c.Transport().Encoding()
logger.Debug("client connected", "transport", transportName, "encoding", transportEncoding)
})
// Set Disconnect handler to react on client disconnect events.
node.OnDisconnect(func(c *centrifuge.Client, e centrifuge.DisconnectEvent) {
logger.Info("client disconnected")
})
// Set SubscribeHandler to react on every channel subscription attempt
// initiated by client. Here you can theoretically return an error or
// disconnect client from server if needed. But now we just accept
// all subscriptions to all channels. In real life you may use a more
// complex permission check here.
node.OnSubscribe(func(c *centrifuge.Client, e centrifuge.SubscribeEvent) (centrifuge.SubscribeReply, error) {
reply := centrifuge.SubscribeReply{}
handler, err := glive.GetChannelHandler(e.Channel)
if err != nil {
return reply, err
}
err = handler.OnSubscribe(c, e)
if err != nil {
return reply, err
}
return reply, nil
})
node.OnUnsubscribe(func(c *centrifuge.Client, e centrifuge.UnsubscribeEvent) {
logger.Debug("unsubscribe from channel", "channel", e.Channel, "user", c.UserID())
})
// Called when something is written to the websocket
node.OnPublish(func(c *centrifuge.Client, e centrifuge.PublishEvent) (centrifuge.PublishReply, error) {
reply := centrifuge.PublishReply{}
handler, err := glive.GetChannelHandler(e.Channel)
if err != nil {
return reply, err
}
data, err := handler.OnPublish(c, e)
if err != nil {
return reply, err
}
if len(data) > 0 {
_, err = node.Publish(e.Channel, e.Data)
}
return centrifuge.PublishReply{}, err // returns an error if it could not publish
})
// Run node. This method does not block.
if err := node.Run(); err != nil {
return nil, err
}
// SockJS will find the best protocol possible for the browser
sockJsPrefix := "/live/sockjs"
sockjsHandler := centrifuge.NewSockjsHandler(node, centrifuge.SockjsConfig{
HandlerPrefix: sockJsPrefix,
WebsocketReadBufferSize: 1024,
WebsocketWriteBufferSize: 1024,
})
// Use a direct websocket from go clients
wsHandler := centrifuge.NewWebsocketHandler(node, centrifuge.WebsocketConfig{
ReadBufferSize: 1024,
WriteBufferSize: 1024,
})
glive.WebsocketHandler = func(ctx *models.ReqContext) {
user := ctx.SignedInUser
if user == nil {
ctx.Resp.WriteHeader(401)
return
}
dto := models.UserProfileDTO{
Id: user.UserId,
Name: user.Name,
Email: user.Email,
Login: user.Login,
IsGrafanaAdmin: user.IsGrafanaAdmin,
OrgId: user.OrgId,
}
jsonData, err := json.Marshal(dto)
if err != nil {
logger.Debug("error reading user", "dto", dto)
ctx.Resp.WriteHeader(404)
return
}
logger.Info("Logged in user", "user", user)
cred := &centrifuge.Credentials{
UserID: fmt.Sprintf("%d", user.UserId),
Info: jsonData,
}
newCtx := centrifuge.SetCredentials(ctx.Req.Context(), cred)
r := ctx.Req.Request
r = r.WithContext(newCtx) // Set a user ID
// Check if this is a direct websocket connection
path := ctx.Req.URL.Path
if strings.Contains(path, "live/ws") {
wsHandler.ServeHTTP(ctx.Resp, r)
return
}
if strings.Contains(path, sockJsPrefix) {
sockjsHandler.ServeHTTP(ctx.Resp, r)
return
}
// Unknown path
ctx.Resp.WriteHeader(404)
}
return glive, nil
}
// GetChannelHandler gives threadsafe access to the channel
func (g *GrafanaLive) GetChannelHandler(channel string) (models.ChannelHandler, error) {
g.channelsMu.RLock()
c, ok := g.channels[channel]
g.channelsMu.RUnlock() // defer? but then you can't lock further down
if ok {
return c, nil
}
// Parse the identifier ${scope}/${namespace}/${path}
addr := ParseChannelAddress(channel)
if !addr.IsValid() {
return nil, fmt.Errorf("invalid channel: %q", channel)
}
logger.Info("initChannel", "channel", channel, "address", addr)
g.channelsMu.Lock()
defer g.channelsMu.Unlock()
c, ok = g.channels[channel] // may have filled in while locked
if ok {
return c, nil
}
getter, err := g.GetChannelHandlerFactory(addr.Scope, addr.Namespace)
if err != nil {
return nil, err
}
// First access will initialize
c, err = getter.GetHandlerForPath(addr.Path)
if err != nil {
return nil, err
}
g.channels[channel] = c
return c, nil
}
// GetChannelHandlerFactory gets a ChannelHandlerFactory for a namespace.
// It gives threadsafe access to the channel.
func (g *GrafanaLive) GetChannelHandlerFactory(scope string, name string) (models.ChannelHandlerFactory, error) {
if scope == "grafana" {
p, ok := g.GrafanaScope.Features[name]
if ok {
return p, nil
}
return nil, fmt.Errorf("unknown feature: %q", name)
}
if scope == "ds" {
return nil, fmt.Errorf("todo... look up datasource: %q", name)
}
if scope == "plugin" {
p, ok := plugins.Plugins[name]
if ok {
h := &PluginHandler{
Plugin: p,
}
return h, nil
}
return nil, fmt.Errorf("unknown plugin: %q", name)
}
return nil, fmt.Errorf("invalid scope: %q", scope)
}
// Publish sends the data to the channel without checking permissions etc
func (g *GrafanaLive) Publish(channel string, data []byte) error {
_, err := g.node.Publish(channel, data)
return err
}
// Write to the standard log15 logger
func handleLog(msg centrifuge.LogEntry) {
arr := make([]interface{}, 0)
for k, v := range msg.Fields {
if v == nil {
v = "<nil>"
} else if v == "" {
v = "<empty>"
}
arr = append(arr, k, v)
}
switch msg.Level {
case centrifuge.LogLevelDebug:
loggerCF.Debug(msg.Message, arr...)
case centrifuge.LogLevelError:
loggerCF.Error(msg.Message, arr...)
case centrifuge.LogLevelInfo:
loggerCF.Info(msg.Message, arr...)
}
}