grafana/pkg/api
Giuseppe Guerra a89202eab2
Plugins: Improve instrumentation by adding metrics and tracing (#61035)
* WIP: Plugins tracing

* Trace ID middleware

* Add prometheus metrics and tracing to plugins updater

* Add TODOs

* Add instrumented http client

* Add tracing to grafana update checker

* Goimports

* Moved plugins tracing to middleware

* goimports, fix tests

* Removed X-Trace-Id header

* Fix comment in NewTracingHeaderMiddleware

* Add metrics to instrumented http client

* Add instrumented http client options

* Removed unused function

* Switch to contextual logger

* Refactoring, fix tests

* Moved InstrumentedHTTPClient and PrometheusMetrics to their own package

* Tracing middleware: handle errors

* Report span status codes when recording errors

* Add tests for tracing middleware

* Moved fakeSpan and fakeTracer to pkg/infra/tracing

* Add TestHTTPClientTracing

* Lint

* Changes after PR review

* Tests: Made "ended" in FakeSpan private, allow calling End only once

* Testing: panic in FakeSpan if span already ended

* Refactoring: Simplify Grafana updater checks

* Refactoring: Simplify plugins updater error checks and logs

* Fix wrong call to checkForUpdates -> instrumentedCheckForUpdates

* Tests: Fix wrong call to checkForUpdates -> instrumentedCheckForUpdates

* Log update checks duration, use Info log level for check succeeded logs

* Add plugin context span attributes in tracing_middleware

* Refactor prometheus metrics as httpclient middleware

* Fix call to ProvidePluginsService in plugins_test.go

* Propagate context to update checker outgoing http requests

* Plugin client tracing middleware: Removed operation name in status

* Fix tests

* Goimports tracing_middleware.go

* Goimports

* Fix imports

* Changed span name to plugins client middleware

* Add span name assertion in TestTracingMiddleware

* Removed Prometheus metrics middleware from grafana and plugins updatechecker

* Add span attributes for ds name, type, uid, panel and dashboard ids

* Fix http header reading in tracing middlewares

* Use contexthandler.FromContext, add X-Query-Group-Id

* Add test for RunStream

* Fix imports

* Changes from PR review

* TestTracingMiddleware: Changed assert to require for didPanic assertion

* Lint

* Fix imports
2023-03-28 11:01:06 +02:00
..
apierrors Chore: Fix status codes for nested folders (#59087) 2022-11-22 16:06:39 +02:00
avatar Chore: Fix goimports grouping in pkg/api (#62419) 2023-01-30 08:18:26 +00:00
datasource backend/datasources: move datasources models into the datasources service package (#51267) 2022-06-27 12:23:15 -04:00
dtos Navigation: Fix Home logo always going to /login (#62658) 2023-03-09 16:42:45 +00:00
frontendlogging Chore: Fix goimports grouping in pkg/api (#62419) 2023-01-30 08:18:26 +00:00
pluginproxy Plugins: Migrate plugincontext, adapters and pluginsettings to pkg/services/pluginsintegration package (#64154) 2023-03-07 11:22:30 -05:00
response Chore: Fix goimports grouping in pkg/api (#62419) 2023-01-30 08:18:26 +00:00
routing Chore: Move ReqContext to contexthandler service (#62102) 2023-01-27 08:50:36 +01:00
static API: Extract OpenAPI specification from source code using go-swagger (#40528) 2022-02-08 13:38:43 +01:00
accesscontrol.go Plugins: Migrate licensing and access control to pkg/services/pluginsintegration package (#65258) 2023-03-27 11:15:37 +02:00
admin_encryption.go Chore: Move ReqContext to contexthandler service (#62102) 2023-01-27 08:50:36 +01:00
admin_provisioning_test.go RBAC: Rewrite provisioning rbac tests (#61752) 2023-01-19 13:49:57 +01:00
admin_provisioning.go Chore: Move ReqContext to contexthandler service (#62102) 2023-01-27 08:50:36 +01:00
admin_test.go Chore: Fix goimports grouping in pkg/api (#62419) 2023-01-30 08:18:26 +00:00
admin_users_test.go Chore: Differentiate the ErrOrgNotFound error messages (#64131) 2023-03-06 09:57:46 +02:00
admin_users.go Chore: Differentiate the ErrOrgNotFound error messages (#64131) 2023-03-06 09:57:46 +02:00
admin.go Chore: Move ReqContext to contexthandler service (#62102) 2023-01-27 08:50:36 +01:00
alerting.go Chore: rename Id to ID in alert notification models (#62868) 2023-02-03 15:46:55 +01:00
annotations_test.go Cfg: Move ViewersCanEdit into cfg (#64876) 2023-03-16 10:54:01 +01:00
annotations.go Chore: remove dashboardsFromStorage (#65058) 2023-03-20 18:36:49 +02:00
api.go Plugins: Migrate licensing and access control to pkg/services/pluginsintegration package (#65258) 2023-03-27 11:15:37 +02:00
apikey.go Chore: Remove result field from API keys commands and queries (#65055) 2023-03-21 13:26:33 +01:00
basic_auth_test.go Macaron: remove custom Request type (#37874) 2021-09-01 11:18:30 +02:00
basic_auth.go Macaron: remove custom Request type (#37874) 2021-09-01 11:18:30 +02:00
common_test.go Cfg: Move ViewersCanEdit into cfg (#64876) 2023-03-16 10:54:01 +01:00
dashboard_permission_test.go Chore: cleanup dashboard service names (#64442) 2023-03-08 14:37:45 -05:00
dashboard_permission.go Chore: Move ReqContext to contexthandler service (#62102) 2023-01-27 08:50:36 +01:00
dashboard_snapshot_test.go Cfg: Move ViewersCanEdit into cfg (#64876) 2023-03-16 10:54:01 +01:00
dashboard_snapshot.go Snapshots: Fix deleting snapshot with non existent dashboard ID (#64345) 2023-03-08 10:12:02 +02:00
dashboard_test.go Cfg: Move ViewersCanEdit into cfg (#64876) 2023-03-16 10:54:01 +01:00
dashboard.go Chore: remove querylibrary feature toggle (#65021) 2023-03-20 20:00:14 +04:00
dataproxy.go Chore: Move ReqContext to contexthandler service (#62102) 2023-01-27 08:50:36 +01:00
datasources_test.go Chore: Remove Result field from datasources (#63048) 2023-02-09 15:49:44 +01:00
datasources.go Plugins: Migrate plugincontext, adapters and pluginsettings to pkg/services/pluginsintegration package (#64154) 2023-03-07 11:22:30 -05:00
fakes.go NavTree: Make it possible to configure where in nav tree plugins live (#55484) 2022-09-28 08:29:35 +02:00
folder_permission_test.go Chore: cleanup dashboard service names (#64442) 2023-03-08 14:37:45 -05:00
folder_permission.go Chore: Move ReqContext to contexthandler service (#62102) 2023-01-27 08:50:36 +01:00
folder_test.go RBAC: protect folder creation and moving (#64636) 2023-03-20 11:04:22 +00:00
folder.go RBAC: protect folder creation and moving (#64636) 2023-03-20 11:04:22 +00:00
frontend_logging_test.go Chore: Fix goimports grouping in pkg/api (#62419) 2023-01-30 08:18:26 +00:00
frontend_logging.go Chore: Add deprecation warnings for Sentry (#60165) 2022-12-13 16:41:42 +02:00
frontend_metrics.go Chore: Move ReqContext to contexthandler service (#62102) 2023-01-27 08:50:36 +01:00
frontendsettings_test.go SupportBundles: Add OAuth bundle collectors (#64810) 2023-03-16 09:46:25 +02:00
frontendsettings.go Auth: Fix orgrole picker disabled if isSynced user (#64033) 2023-03-22 17:41:59 +00:00
grafana_com_proxy.go API: don't re-add /api suffix to grafana.com API URL (#62280) 2023-01-27 10:20:55 +01:00
health_test.go Chore: Fix goimports grouping in pkg/api (#62419) 2023-01-30 08:18:26 +00:00
health.go Chore: Remove Store interface and use db.DB instead (#60160) 2022-12-13 11:03:36 +01:00
http_server_test.go Chore: Fix goimports grouping in pkg/api (#62419) 2023-01-30 08:18:26 +00:00
http_server.go Chore: remove querylibrary feature toggle (#65021) 2023-03-20 20:00:14 +04:00
index.go Chore: Pass signed user_hash to Intercom via Rudderstack (#63921) 2023-03-03 14:39:53 +00:00
login_oauth_test.go SupportBundles: Add OAuth bundle collectors (#64810) 2023-03-16 09:46:25 +02:00
login_oauth.go Auth: Add feature flag to move token rotation to client (#65060) 2023-03-23 14:39:04 +01:00
login_test.go Auth: Add feature flag to move token rotation to client (#65060) 2023-03-23 14:39:04 +01:00
login.go Auth: Add feature flag to move token rotation to client (#65060) 2023-03-23 14:39:04 +01:00
metrics_test.go Chore: Change fmt.Errorf to errors.New when there is no formatting required (#58600) 2022-12-01 20:51:12 +01:00
metrics.go Chore: Move ReqContext to contexthandler service (#62102) 2023-01-27 08:50:36 +01:00
openapi3.go Chore: Move ReqContext to contexthandler service (#62102) 2023-01-27 08:50:36 +01:00
org_invite_test.go Chore: Fix goimports grouping in pkg/api (#62419) 2023-01-30 08:18:26 +00:00
org_invite.go Settings: Remove global variables for auth settings (#63795) 2023-02-27 15:28:49 +01:00
org_test.go RBAC: remove access control mock from org quota tests (#61574) 2023-01-17 10:33:01 +00:00
org_users_test.go Auth: Fix orgrole picker disabled if isSynced user (#64033) 2023-03-22 17:41:59 +00:00
org_users.go Auth: Fix orgrole picker disabled if isSynced user (#64033) 2023-03-22 17:41:59 +00:00
org.go Chore: Move ReqContext to contexthandler service (#62102) 2023-01-27 08:50:36 +01:00
password.go Settings: Remove global variables for auth settings (#63795) 2023-02-27 15:28:49 +01:00
playlist_play.go Chore: Update oapi-codegen library (#62962) 2023-02-21 12:34:24 +01:00
playlist.go Chore: Move ReqContext to contexthandler service (#62102) 2023-01-27 08:50:36 +01:00
plugin_dashboards_test.go Chore: Fix goimports grouping in pkg/api (#62419) 2023-01-30 08:18:26 +00:00
plugin_dashboards.go Chore: Move ReqContext to contexthandler service (#62102) 2023-01-27 08:50:36 +01:00
plugin_metrics_test.go Chore: Fix goimports grouping in pkg/api (#62419) 2023-01-30 08:18:26 +00:00
plugin_metrics.go Chore: Fix goimports grouping in pkg/api (#62419) 2023-01-30 08:18:26 +00:00
plugin_proxy_test.go Plugins: Handle app plugin proxy routes per request (#51835) 2022-08-23 13:05:31 +02:00
plugin_proxy.go Plugins: Migrate plugincontext, adapters and pluginsettings to pkg/services/pluginsintegration package (#64154) 2023-03-07 11:22:30 -05:00
plugin_resource_test.go Plugins: Improve instrumentation by adding metrics and tracing (#61035) 2023-03-28 11:01:06 +02:00
plugin_resource.go Chore: Move ReqContext to contexthandler service (#62102) 2023-01-27 08:50:36 +01:00
plugins_test.go Plugins: Improve instrumentation by adding metrics and tracing (#61035) 2023-03-28 11:01:06 +02:00
plugins.go Plugins: Markdown fetch retry with lowercase (#65384) 2023-03-27 19:44:06 +03:00
preferences_test.go Chore: Remove Result from dashboard models (#61997) 2023-01-25 10:36:26 +01:00
preferences.go Cookies: Provide a mechanism for per user control over cookies (#61566) 2023-02-21 11:19:07 +01:00
quota_test.go Chore: Upgrade golangci-lint to v1.51.2 (#63630) 2023-02-23 15:10:03 +01:00
quota.go Chore: Move ReqContext to contexthandler service (#62102) 2023-01-27 08:50:36 +01:00
README.md API: Enable serving Swagger UI by default and add docs and guidelines (#63489) 2023-03-01 16:36:37 +02:00
render.go Chore: Move ReqContext to contexthandler service (#62102) 2023-01-27 08:50:36 +01:00
search_test.go RBAC: Rewrite search and plugin list rbac test (#63483) 2023-02-21 11:13:35 +01:00
search.go Chore: Move search model from models package to search service (#62215) 2023-01-30 15:17:53 +01:00
short_url_test.go Chore: Fix goimports grouping in pkg/api (#62419) 2023-01-30 08:18:26 +00:00
short_url.go Chore: Move ReqContext to contexthandler service (#62102) 2023-01-27 08:50:36 +01:00
signup.go Orgs: Remove auto assign globals (#63754) 2023-02-27 10:23:38 +01:00
swagger_responses.go Chore: Move swagger definitions to the handlers (#52643) 2022-07-27 09:54:37 -04:00
swagger_tags.json Chore: Move swagger definitions to the handlers (#52643) 2022-07-27 09:54:37 -04:00
swagger.go Chore: Move ReqContext to contexthandler service (#62102) 2023-01-27 08:50:36 +01:00
team_members_test.go Chore: Remove CreateUserForTests() (#64125) 2023-03-03 11:01:23 -05:00
team_members.go Chore: Move ReqContext to contexthandler service (#62102) 2023-01-27 08:50:36 +01:00
team_test.go Chore: Move ReqContext to contexthandler service (#62102) 2023-01-27 08:50:36 +01:00
team.go Chore: Move ReqContext to contexthandler service (#62102) 2023-01-27 08:50:36 +01:00
user_test.go Chore: Remove CreateUserForTests() (#64125) 2023-03-03 11:01:23 -05:00
user_token_test.go Auth: Add feature flag to move token rotation to client (#65060) 2023-03-23 14:39:04 +01:00
user_token.go Auth: Add feature flag to move token rotation to client (#65060) 2023-03-23 14:39:04 +01:00
user.go Auth: Fix orgrole picker disabled if isSynced user (#64033) 2023-03-22 17:41:59 +00:00
utils.go Auth: Add feature flag to move token rotation to client (#65060) 2023-03-23 14:39:04 +01:00

OpenAPI specifications

Since version 8.4, HTTP API details are specified using OpenAPI v2. Starting from version 9.1, there is also an OpenAPI v3 specification (generated by the v2 one using this script).

OpenAPI annotations

The OpenAPI v2 specification is generated automatically from the annotated Go code using go-swagger which scans the source code for annotation rules. Refer to this getting started guide for getting familiar with the toolkit.

Developers modifying the HTTP API endpoints need to make sure to add the necessary annotations so that their changes are reflected into the generated specifications.

Example of endpoint annotation

The following route defines a PATCH endpoint under the /serviceaccounts/{serviceAccountId} path with tag service_accounts (used for grouping together several routes) and operation ID updateServiceAccount (used for uniquely identifying routes and associate parameters and response with them).


// swagger:route PATCH /serviceaccounts/{serviceAccountId} service_accounts updateServiceAccount
//
// # Update service account
//
// Required permissions (See note in the [introduction](https://grafana.com/docs/grafana/latest/developers/http_api/serviceaccount/#service-account-api) for an explanation):
// action: `serviceaccounts:write` scope: `serviceaccounts:id:1` (single service account)
//
// Responses:
// 200: updateServiceAccountResponse
// 400: badRequestError
// 401: unauthorisedError
// 403: forbiddenError
// 404: notFoundError
// 500: internalServerError

The go-swagger can discover such annotations by scanning any code imported by pkg/server but by convention we place the endpoint annotations above the endpoint definition.

Example of endpoint parameters

The following struct defines the route parameters for the updateServiceAccount endpoint. The route expects:

  • a path parameter denoting the service account identifier and
  • a body parameter with the new values for the specific service account

// swagger:parameters updateServiceAccount
type UpdateServiceAccountParams struct {
	// in:path
	ServiceAccountId int64 `json:"serviceAccountId"`
	// in:body
	Body serviceaccounts.UpdateServiceAccountForm
}

Example of endpoint response

The following struct defines the response for the updateServiceAccount endpoint in case of a successful 200 response.


// swagger:response updateServiceAccountResponse
type UpdateServiceAccountResponse struct {
	// in:body
	Body struct {
		Message        string                                    `json:"message"`
		ID             int64                                     `json:"id"`
		Name           string                                    `json:"name"`
		ServiceAccount *serviceaccounts.ServiceAccountProfileDTO `json:"serviceaccount"`
	}
}

OpenAPI generation

Developers can re-create the OpenAPI v2 and v3 specifications using the following command:


make clean-api-spec && make openapi3-gen

They can observe its output into the public/api-merged.json and public/openapi3.json files.

Finally, they can browser and try out both the OpenAPI v2 and v3 via the Swagger UI editor (served by the grafana server) by navigating to /swagger-ui and /openapi3 respectivally.