mirror of
https://github.com/grafana/grafana.git
synced 2025-02-25 18:55:37 -06:00
Docs: Plugin doc review - backend topics - chunk 5 (#68662)
* Initial commit Signed-off-by: Joe Perez <joseph.perez@grafana.com> * Minor fixes Signed-off-by: Joe Perez <joseph.perez@grafana.com> * Prettier fixes Signed-off-by: Joe Perez <joseph.perez@grafana.com> * Incorporating review feedback Signed-off-by: Joe Perez <joseph.perez@grafana.com> * Minor fix Signed-off-by: Joe Perez <joseph.perez@grafana.com> * Update docs/sources/developers/plugins/backend/_index.md Co-authored-by: Giuseppe Guerra <giuseppe.guerra@grafana.com> * Update docs/sources/developers/plugins/backend/_index.md Co-authored-by: Clayton Cornell <131809008+clayton-cornell@users.noreply.github.com> * Update docs/sources/developers/plugins/backend/_index.md Co-authored-by: Clayton Cornell <131809008+clayton-cornell@users.noreply.github.com> * Update docs/sources/developers/plugins/backend/_index.md Co-authored-by: Clayton Cornell <131809008+clayton-cornell@users.noreply.github.com> * Update docs/sources/developers/plugins/backend/_index.md Co-authored-by: Clayton Cornell <131809008+clayton-cornell@users.noreply.github.com> * Update docs/sources/developers/plugins/backend/_index.md Co-authored-by: Clayton Cornell <131809008+clayton-cornell@users.noreply.github.com> * Update docs/sources/developers/plugins/backend/_index.md Co-authored-by: Clayton Cornell <131809008+clayton-cornell@users.noreply.github.com> * Update docs/sources/developers/plugins/backend/_index.md Co-authored-by: Clayton Cornell <131809008+clayton-cornell@users.noreply.github.com> * Update docs/sources/developers/plugins/backend/_index.md Co-authored-by: Clayton Cornell <131809008+clayton-cornell@users.noreply.github.com> * Update docs/sources/developers/plugins/backend/_index.md Co-authored-by: Clayton Cornell <131809008+clayton-cornell@users.noreply.github.com> * Update docs/sources/developers/plugins/backend/grafana-plugin-sdk-for-go.md Co-authored-by: Clayton Cornell <131809008+clayton-cornell@users.noreply.github.com> * Update docs/sources/developers/plugins/backend/grafana-plugin-sdk-for-go.md Co-authored-by: Clayton Cornell <131809008+clayton-cornell@users.noreply.github.com> * Update docs/sources/developers/plugins/backend/plugin-protocol.md Co-authored-by: Clayton Cornell <131809008+clayton-cornell@users.noreply.github.com> * Update docs/sources/developers/plugins/backend/plugin-protocol.md Co-authored-by: Clayton Cornell <131809008+clayton-cornell@users.noreply.github.com> * Update docs/sources/developers/plugins/backend/plugin-protocol.md Co-authored-by: Clayton Cornell <131809008+clayton-cornell@users.noreply.github.com> * Update docs/sources/developers/plugins/backend/_index.md Co-authored-by: David Harris <david.harris@grafana.com> * Indentation fix Signed-off-by: Joe Perez <joseph.perez@grafana.com> * Review feedback Signed-off-by: Joe Perez <joseph.perez@grafana.com> * Bug fix Signed-off-by: Joe Perez <joseph.perez@grafana.com> --------- Signed-off-by: Joe Perez <joseph.perez@grafana.com> Co-authored-by: Giuseppe Guerra <giuseppe.guerra@grafana.com> Co-authored-by: Clayton Cornell <131809008+clayton-cornell@users.noreply.github.com> Co-authored-by: David Harris <david.harris@grafana.com>
This commit is contained in:
Joseph Perez
GitHub
parent
0600c5f5e7
commit
b7e2f3ad3f
@ -13,63 +13,85 @@ title: Backend plugins
|
||||
|
||||
# Backend plugins
|
||||
|
||||
Grafana added support for plugins in version 3.0 and this enabled the Grafana community to create panel plugins and data source plugins. It was wildly successful and has made Grafana much more useful as you can integrate it with anything and do any type of custom visualization that you want.
|
||||
The Grafana plugin system for backend development allows you to integrate Grafana with virtually anything and offer custom visualizations. This document explains the system's background, use cases, and key features.
|
||||
|
||||
However, one limitation with these plugins are that they execute on the client-side (in the browser) which makes it hard to support certain use cases/features, e.g. enable Grafana Alerting for data sources. Grafana v7.0 adds official support for backend plugins which removes this limitation. At the same time it gives plugin developers the possibility to extend Grafana in new and interesting ways, with code running in the backend (server side).
|
||||
## Background
|
||||
|
||||
We use the term _backend plugin_ to denote that a plugin has a backend component. Still, normally a backend plugin requires frontend components as well. This is for example true for backend data source plugins which normally need configuration and query editor components implemented for the frontend.
|
||||
Grafana added support for _frontend plugins_ in version 3.0 so that the Grafana community could create custom panels and data sources. It was wildly successful and has made Grafana much more useful for our user community.
|
||||
|
||||
Data source plugins can be extended with a backend component. In the future we plan to support additional types and possibly new kinds of plugins, such as notifiers for Grafana Alerting and custom authentication to name a few.
|
||||
However, one limitation of these plugins is that they run on the client side, in the browser. Therefore, they can't support use cases that require server-side features.
|
||||
|
||||
Since Grafana v7.0, we have supported server-side plugins that remove this limitation. We use the term _backend plugin_ to denote that a plugin has a backend component. A backend plugin usually requires frontend components as well. For example, some backend data source plugins need query editor components on the frontend.
|
||||
|
||||
## Use cases for implementing a backend plugin
|
||||
|
||||
The following examples gives you an idea of why you'd consider implementing a backend plugin:
|
||||
The following examples give some common use cases for backend plugins:
|
||||
|
||||
- Enable [Grafana Alerting]({{< relref "../../../alerting/" >}}) for data sources.
|
||||
- Connect to non-HTTP services that normally can't be connected to from a web browser, e.g. SQL database servers.
|
||||
- Keep state between users, e.g. query caching for data sources.
|
||||
- Connect to SQL database servers and other non-HTTP services that normally can't be connected to from a browser.
|
||||
- Keep state between users, for example, by query caching for data sources.
|
||||
- Use custom authentication methods and/or authorization checks that aren't supported in Grafana.
|
||||
- Use a custom data source request proxy, see [Resources]({{< relref "#resources" >}}).
|
||||
- Use a custom data source request proxy (refer to [Resources]({{< relref "#resources" >}}) for more information).
|
||||
|
||||
## Grafana’s backend plugin system
|
||||
## Grafana backend plugin system
|
||||
|
||||
The Grafana backend plugin system is based on the [go-plugin library by HashiCorp](https://github.com/hashicorp/go-plugin). The Grafana server launches each backend plugin as a subprocess and communicates with it over [gRPC](https://grpc.io/). This approach has a number of benefits:
|
||||
The Grafana backend plugin system is based on HashiCorp's [Go Plugin System over RPC](https://github.com/hashicorp/go-plugin). Our implementation of the Grafana server launches each backend plugin as a subprocess and communicates with it over [gRPC](https://grpc.io/).
|
||||
|
||||
- Plugins can’t crash your grafana process: a panic in a plugin doesn’t panic the server.
|
||||
- Plugins are easy to develop: just write a Go application and run `go build` (or use any other language which supports gRPC).
|
||||
- Plugins can be relatively secure: The plugin only has access to the interfaces and arguments that are given to it, not to the entire memory space of the process.
|
||||
### Benefits for plugin development
|
||||
|
||||
Grafana's backend plugin system exposes a couple of different capabilities, or building blocks, that a backend plugin can implement:
|
||||
Grafana's approach has benefits for developers:
|
||||
|
||||
- **Stability:** Plugins can't crash your Grafana process: a panic in a plugin doesn't panic the server.
|
||||
- **Ease of development:** Plugins can be written in any language that supports gRPC (for example, write a Go application and run `go build`).
|
||||
- **Security:** Plugins only have access to the interfaces and arguments given to them, not to the entire memory space of the process.
|
||||
|
||||
### Capabilities of the backend plugin system
|
||||
|
||||
Grafana's backend plugin system exposes several key capabilities, or building blocks, that your backend plugin can implement:
|
||||
|
||||
- Query data
|
||||
- Resources
|
||||
- Health checks
|
||||
- Collect metrics
|
||||
- Streaming
|
||||
|
||||
### Query data
|
||||
#### Query data
|
||||
|
||||
The query data capability allows a backend plugin to handle data source queries that are submitted from a [dashboard]({{< relref "../../../dashboards/" >}}), [Explore]({{< relref "../../../explore/" >}}) or [Grafana Alerting]({{< relref "../../../alerting/" >}}). The response contains [data frames]({{< relref "../data-frames/" >}}), which are used to visualize metrics, logs, and traces. The query data capability is required to implement for a backend data source plugin.
|
||||
The query data capability allows a backend plugin to handle data source queries that are submitted from a [dashboard]({{< relref "../../../dashboards/" >}}), [Explore]({{< relref "../../../explore/" >}}) or [Grafana Alerting]({{< relref "../../../alerting/" >}}). The response contains [data frames]({{< relref "../data-frames/" >}}), which are used to visualize metrics, logs, and traces.
|
||||
|
||||
### Resources
|
||||
{{% admonition type="note" %}} Backend data source plugins are required to implement the query data capability.{{%
|
||||
/admonition %}}
|
||||
|
||||
The resources capability allows a backend plugin to handle custom HTTP requests sent to the Grafana HTTP API and respond with custom HTTP responses. Here, the request and response formats can vary, e.g. JSON, plain text, HTML or static resources (files, images) etc. Compared to the query data capability where the response contains data frames, resources give the plugin developer a lot of flexibility for extending and open up Grafana for new and interesting use cases.
|
||||
#### Resources
|
||||
|
||||
The resources capability allows a backend plugin to handle custom HTTP requests sent to the Grafana HTTP API and respond with custom HTTP responses. Here, the request and response formats can vary. For example, you can use JSON, plain text, HTML, or static resources such as images and files, and so on.
|
||||
|
||||
Compared to the query data capability, where the response contains data frames, the resources capability gives the plugin developer more flexibility for extending and opening up Grafana for new and interesting use cases.
|
||||
|
||||
### Use cases
|
||||
|
||||
Examples of use cases for implementing resources:
|
||||
|
||||
- Implement a custom data source proxy in case certain authentication/authorization or other requirements are required/needed that are not supported in Grafana's [built-in data proxy]({{< relref "../../http_api/data_source/#data-source-proxy-calls" >}}).
|
||||
- Return data or information in a format suitable to use within a data source query editor to provide auto-complete functionality.
|
||||
- Return static resources, such as images or files.
|
||||
- Send a command to a device, such as a micro controller or IOT device.
|
||||
- Request information from a device, such as a micro controller or IOT device.
|
||||
- Implement a custom data source proxy to provide certain authentication, authorization, or other requirements that are not supported in Grafana's [built-in data proxy]({{< relref "../../http_api/data_source/#data-source-proxy-calls" >}}).
|
||||
- Return data or information in a format suitable for use within a data source query editor to provide auto-complete functionality.
|
||||
- Return static resources such as images or files.
|
||||
- Send a command to a device, such as a microcontroller or IoT device.
|
||||
- Request information from a device, such as a microcontroller or IoT device.
|
||||
- Extend Grafana's HTTP API with custom resources, methods and actions.
|
||||
- Use [chunked transfer encoding](https://en.wikipedia.org/wiki/Chunked_transfer_encoding) to return large data responses in chunks or to enable "basic" streaming capabilities.
|
||||
- Use [chunked transfer encoding](https://en.wikipedia.org/wiki/Chunked_transfer_encoding) to return large data responses in chunks or to enable certain streaming capabilities.
|
||||
|
||||
### Health checks
|
||||
#### Health checks
|
||||
|
||||
The health checks capability allows a backend plugin to return the status of the plugin. For data source backend plugins the health check will automatically be called when you do _Save & Test_ in the UI when editing a data source. A plugin's health check endpoint is exposed in the Grafana HTTP API and allows external systems to continuously poll the plugin's health to make sure it's running and working as expected.
|
||||
The health checks capability allows a backend plugin to return the status of the plugin. For data source backend plugins, the health check is automatically called when a user edits a data source and selects _Save & Test_ in the UI.
|
||||
|
||||
### Collect metrics
|
||||
A plugin's health check endpoint is exposed in the Grafana HTTP API and allows external systems to continuously poll the plugin's health to make sure that it's running and working as expected.
|
||||
|
||||
A backend plugin can collect and return runtime, process and custom metrics using the text-based Prometheus [exposition format](https://prometheus.io/docs/instrumenting/exposition_formats/). If you’re using the [Grafana Plugin SDK for Go]({{< relref "grafana-plugin-sdk-for-go/" >}}) to implement your backend plugin, then the [Prometheus instrumentation library for Go applications](https://github.com/prometheus/client_golang) is built-in, and gives you Go runtime metrics and process metrics out of the box. By using the [Prometheus instrumentation library](https://github.com/prometheus/client_golang) you can add custom metrics to instrument your backend plugin.
|
||||
#### Collect metrics
|
||||
|
||||
A metrics endpoint (`/api/plugins/<plugin id>/metrics`) for a plugin is available in the Grafana HTTP API and allows a Prometheus instance to be configured to scrape the metrics.
|
||||
A backend plugin can collect and return runtime, process, and custom metrics using the text-based Prometheus [exposition format](https://prometheus.io/docs/instrumenting/exposition_formats/). If you're using the [Grafana Plugin SDK for Go]({{< relref "grafana-plugin-sdk-for-go/" >}}) to implement your backend plugin, then the [Prometheus instrumentation library for Go applications](https://github.com/prometheus/client_golang) is built-in. This SDK gives you Go runtime metrics and process metrics out of the box. You can use the [Prometheus instrumentation library](https://github.com/prometheus/client_golang) to add custom metrics to instrument your backend plugin.
|
||||
|
||||
The Grafana HTTP API offers an endpoint (`/api/plugins/<plugin id>/metrics`) that allows you to configure a Prometheus instance to scrape the metrics.
|
||||
|
||||
#### Streaming
|
||||
|
||||
The streaming capability allows a backend plugin to handle data source queries that are streaming. For more information, refer to [Build a streaming data source plugin]({{<relref "../build-a-streaming-data-source-plugin.md">}})
|
||||
|
||||
@ -12,15 +12,17 @@ title: Grafana Plugin SDK for Go
|
||||
|
||||
# Grafana plugin SDK for Go
|
||||
|
||||
The Grafana plugin SDK for Go enables building Grafana backend plugins using [Go](https://golang.org/). The SDK provides a high-level framework with APIs, utilities and tooling that abstract away the details of the [plugin protocol]({{< relref "plugin-protocol/" >}}) and RPC communication so plugin developers do not need to manage either.
|
||||
The [Grafana plugin SDK for Go](https://pkg.go.dev/mod/github.com/grafana/grafana-plugin-sdk-go?tab=overview) is a [Go](https://golang.org/) module that provides a set of [packages](https://pkg.go.dev/mod/github.com/grafana/grafana-plugin-sdk-go?tab=packages) that you can use to implement a backend plugin.
|
||||
|
||||
The [github.com/grafana/grafana-plugin-sdk-go](https://pkg.go.dev/mod/github.com/grafana/grafana-plugin-sdk-go?tab=overview) is a Go module that provides a set of [Go packages](https://pkg.go.dev/mod/github.com/grafana/grafana-plugin-sdk-go?tab=packages) that can be used to implement a backend plugin.
|
||||
The plugin SDK provides a high-level framework with APIs, utilities, and tooling. By using the SDK, you can avoid the need to learn the details of the [plugin protocol]({{< relref "plugin-protocol/" >}}) and RPC communication protocol, so you don't have to manage either one.
|
||||
|
||||
## Versioning
|
||||
|
||||
The SDK is still in development. The [plugin protocol]({{< relref "plugin-protocol/" >}}) between Grafana and the plugin SDK is versioned separately and considered stable. However, there might be breaking changes introduced in the SDK. This means that plugins using an older version of the SDK should still work with Grafana, but might lose out on new features and capabilities introduced in the SDK.
|
||||
The Grafana plugin Go SDK is still in development. It is based on the [plugin protocol]({{< relref "plugin-protocol/" >}}), which is versioned separately and is considered stable. However, from time to time, we might introduce breaking changes in the SDK.
|
||||
|
||||
When we update the plugin SDK, those plugins that use an older version of the SDK should still work with Grafana. However, these older plugins may be unable to use the new features and capabilities we introduce in updated SDK versions.
|
||||
|
||||
## See also
|
||||
|
||||
- [Source code](https://github.com/grafana/grafana-plugin-sdk-go)
|
||||
- [SDK source code](https://github.com/grafana/grafana-plugin-sdk-go)
|
||||
- [Go reference documentation](https://pkg.go.dev/github.com/grafana/grafana-plugin-sdk-go)
|
||||
|
||||
@ -11,24 +11,35 @@ title: Plugin protocol
|
||||
|
||||
# Plugin protocol
|
||||
|
||||
There’s a physical wire protocol that Grafana server uses to communicate with backend plugins. This is the contract between Grafana and backend plugins, that must be agreed upon for Grafana and a backend plugin to be able to communicate with each other. The plugin protocol is built on [gRPC](https://grpc.io/) and is defined in [Protocol Buffers (a.k.a., protobuf)](https://developers.google.com/protocol-buffers).
|
||||
The Grafana server uses a physical wire protocol to communicate with backend plugins. This protocol establishes a contract between Grafana and backend plugins to allow them to communicate with each other.
|
||||
|
||||
We advise for backend plugins to not be implemented directly against this protocol. Instead, prefer to use the [Grafana Plugin SDK for Go]({{< relref "grafana-plugin-sdk-for-go/" >}}) that implements this protocol and provides higher level APIs.
|
||||
## Developing with the plugin protocol
|
||||
|
||||
The plugin protocol is available in the [GitHub repository](https://github.com/grafana/grafana-plugin-sdk-go/blob/master/proto/backend.proto). The plugin protocol lives in the [Grafana Plugin SDK for Go]({{< relref "grafana-plugin-sdk-for-go/" >}}) since Grafana itself uses parts of the SDK as a dependency.
|
||||
{{% admonition type="caution" %}} We strongly recommend that backend plugin development not be implemented directly against the protocol. Instead, we prefer that you use the [Grafana Plugin SDK for Go]({{< relref "grafana-plugin-sdk-for-go/" >}}) that implements this protocol and provides higher-level APIs. {{%
|
||||
/admonition %}}
|
||||
|
||||
If you choose to develop against the plugin protocol directly, you can do so using [Protocol Buffers](https://developers.google.com/protocol-buffers) (that is, protobufs) with [gRPC](https://grpc.io/).
|
||||
|
||||
Grafana's plugin protocol protobufs are available in the [GitHub repository](https://github.com/grafana/grafana-plugin-sdk-go/blob/master/proto/backend.proto).
|
||||
|
||||
{{% admonition type="note" %}}
|
||||
The plugin protocol lives in the [Grafana Plugin SDK for Go]({{< relref "grafana-plugin-sdk-for-go/" >}}) because Grafana itself uses parts of the SDK as a dependency.
|
||||
{{% /admonition %}}
|
||||
|
||||
## Versioning
|
||||
|
||||
Additions of services, messages and fields in the latest version of the plugin protocol are expected to happen, but should not introduce any breaking changes. If breaking changes to the plugin protocol is needed, a new major version of the plugin protocol will be created and released together with a new major Grafana release. Grafana will then support both the old and the new plugin protocol for some time to make sure existing backend plugins continue to work.
|
||||
From time to time, Grafana will offer additions of services, messages, and fields in the latest version of the plugin protocol. We don't expect these updates to introduce any breaking changes. However, if we must introduce breaking changes to the plugin protocol, we'll create a new major version of the plugin protocol.
|
||||
|
||||
Because Grafana maintains the plugin protocol, the plugin protocol attempts to follow Grafana's versioning, However, that doesn't automatically mean that a new major version of the plugin protocol is created when a new major release of Grafana is released.
|
||||
Grafana will release new major versions of the plugin protocol alongside new major Grafana releases. When this happens, we'll support both the old and the new plugin protocol for some time to make sure existing backend plugins continue to work.
|
||||
|
||||
The plugin protocol attempts to follow Grafana's versioning. However, that doesn't mean we will automatically create a new major version of the plugin protocol when a new major release of Grafana is released.
|
||||
|
||||
## Writing plugins without Go
|
||||
|
||||
If you want to write a backend plugin in another language than Go, then it’s possible as long as the language supports [gRPC](https://grpc.io/). However, writing a plugin in Go is recommended and has several advantages that should be carefully taken into account before proceeding:
|
||||
If you want to write a backend plugin in a language other than Go, then it's possible as long as the language supports gRPC. However, we recommend that you develop your plugin in Go for several reasons:
|
||||
|
||||
- There's an official [SDK]({{< relref "grafana-plugin-sdk-for-go/" >}}) available.
|
||||
- Single binary as the compiled output.
|
||||
- Building and compiling for multiple platforms is easy.
|
||||
- A statically compiled binary (in most cases) doesn't require any additional dependencies installed on the target platform enabling it to run “everywhere”.
|
||||
- Small footprint in regards to binary size and resource usage.
|
||||
- We offer an official plugin SDK.
|
||||
- The compiled output is a single binary.
|
||||
- Writing for multiple platforms is easy. Typically, no additional dependencies must be installed on the target platform.
|
||||
- Small footprint for binary size.
|
||||
- Small footprint for resource usage.
|
||||
|
||||
Loading…
Reference in New Issue
Block a user