Commit Graph

53 Commits

Author SHA1 Message Date
Marcus Efraimsson
06e67bc573
API Server: Fix stack overflow panic when tracing is disabled (#90075) 2024-07-04 19:11:59 +03:00
Ryan McKinley
99d8025829
Chore: Move identity and errutil to apimachinery module (#89116) 2024-06-13 07:11:35 +03:00
Marcus Efraimsson
1abaa825c6
Tracing: Only enable traces to profiles for api servers for now (#89126) 2024-06-12 19:36:31 +03:00
Marcus Efraimsson
40931b6da6
Tracing: Enable traces to profiles (#88896) 2024-06-07 16:58:24 +02:00
Marcus Efraimsson
0b1aec6767
Tracing: Various improvements (#88308) 2024-05-27 14:21:40 +02:00
Marcus Efraimsson
78f460c4de
Tracing: Allow otel service name and attributes to be overridden from env (#85937)
Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com>
2024-04-11 15:18:46 +02:00
Marcus Efraimsson
6c1de260a2
API Server: Standalone observability (#84789)
Adds support for logs (specify level), metrics (enable metrics and Prometheus /metrics endpoint 
and traces (jaeger or otlp) for standalone API server. This will allow any grafana core service 
part of standalone apiserver to use logging, metrics and traces as normal.
2024-03-21 17:06:32 +01:00
Andres Martinez Gotor
fcc2f01c7e
Chore: Update otel deps (#78069)
Co-authored-by: Giuseppe Guerra <giuseppe@guerra.in>
Co-authored-by: Giuseppe Guerra <giuseppe.guerra@grafana.com>
2023-11-14 14:14:15 +01:00
Giuseppe Guerra
7b4bcc86f1
Plugins: Pass OTEL sampling config to plugins (#76834)
* Pass OTEL sampling config to plugins

* fix capital letters

* Do not pass sampler env vars if sampling is not configured

* Add tests

* PR review feedback

* Simplify tracing env vars logic

* Update test to reflect pkg/infra/tracing behaviour
2023-11-10 14:56:08 +01:00
Todd Treece
bf8af608a7
K8s: Add tracer provider to config (#77003) 2023-10-25 15:19:44 -04:00
Marcus Efraimsson
e4c1a7a141
Tracing: Standardize on otel tracing (#75528) 2023-10-03 14:54:20 +02:00
Marcus Efraimsson
0128d0403f
Tracing: Use tracing.InitializeTracerForTest (#75479) 2023-09-27 09:51:57 +02:00
Marcus Efraimsson
534e3ebf0c
Plugins: Use a Grafana specific SDK tracer implementation for core plugins (#75388) 2023-09-26 14:46:31 +02:00
Dave Henderson
ce1169f8b7
Tracing: Support remote, rate-limited, and probabilistic sampling in tracing.opentelemetry config section (#73587)
* tracing: Support remote sampling server

Signed-off-by: Dave Henderson <dave.henderson@grafana.com>

* Update docs/sources/setup-grafana/configure-grafana/_index.md

Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com>

* Update docs/sources/setup-grafana/configure-grafana/_index.md

Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com>

* Update docs/sources/setup-grafana/configure-grafana/_index.md

Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com>

* Update docs/sources/setup-grafana/configure-grafana/_index.md

Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com>

* Update docs/sources/setup-grafana/configure-grafana/_index.md

* Update docs/sources/setup-grafana/configure-grafana/_index.md

* Update docs/sources/setup-grafana/configure-grafana/_index.md

* Satisfying the doc-validator check

* satisfy prettier

Signed-off-by: Dave Henderson <dave.henderson@grafana.com>

* back out unnecessary change

Signed-off-by: Dave Henderson <dave.henderson@grafana.com>

---------

Signed-off-by: Dave Henderson <dave.henderson@grafana.com>
Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com>
2023-09-11 12:13:29 -04:00
Ryan McKinley
025b2f3011
Chore: use any rather than interface{} (#74066) 2023-08-30 18:46:47 +03:00
Alexander Weaver
2245a3d0d1
Tracer: Export ContextWithSpan (#73715)
Export ContextWithSpan
2023-08-25 13:48:58 -05:00
Serge Zaitsev
2bfef9e916
Tracing: Differentiate collector and agent better (#71851)
* differentiate collector and agent better

* Add back the max packet size

Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>

---------

Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>
2023-08-01 14:23:31 +02:00
Juraci Paixão Kröhling
c4be5cd747
Tracing: Reduce the max packet size for Jaeger exporter (#71659)
Reduce the max packet size for Jaeger exporter

Signed-off-by: Juraci Paixão Kröhling <juraci@kroehling.de>
2023-07-14 16:31:50 +01:00
Bryan Boreham
c45ff94806
Tracing: supply Grafana build version (#69733)
* tracing: supply Grafana build version

Not etcd client version.

* go mod tidy
2023-06-22 11:55:30 +03:00
Andreas Deininger
95b1f3c875
Fixing typos (#70487) 2023-06-22 09:43:38 +01:00
Eng Zer Jun
cf1945d0c3
Tests: use t.Setenv to set env vars (#69516)
This commit replaces `os.Setenv` with `t.Setenv` in tests. The
environment variable is automatically restored to its original value
when the test and all its subtests complete.

Reference: https://pkg.go.dev/testing#T.Setenv

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2023-06-05 11:31:03 +02:00
Serge Zaitsev
6d8f9c5bf4
Chore: Remove opentracing and use opentelemetry instead (#67200)
* remove opentracing and use otel instead

* add various samplers for jaeger

* remove useless test that is covered in otel now

* we do not need a struct there

* remove old tests

* restore tests that parse various configurations

* check errors in tests

* Update pkg/infra/tracing/tracing_test.go

fix typo

Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>

* add test for both legacy and new config formats

* use named constants

---------

Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>
2023-04-27 15:04:43 +02:00
Will Browne
bff9f4c890
Plugins: Move config factory to pluginsintegration package (#65716)
* move config to pluginsintegration package

* change to all plugin toggle

* fixes

* fixes

* fix lerna

* fix test
2023-04-05 14:40:08 +02:00
Alexander Weaver
fb520edd72
Alerting: Use a completely isolated context for state history writes (#64989)
* Add fresh context with timeout and same log properties, re-derive logger

* Unify timeout constants

* Move ctx after shortcut that got added through rebasing

* Unify timeouts

* Port opentracing's SpanFromContext and ContextFromSpan to the grafana tracing package

* Support both opentracing and otel variants

* Better document why we're creating a new ctx

* Add new func to FakeSpan which was added after rebase

* Support grafana-specific traceID key in both tracer implementations
2023-04-04 16:41:46 -05:00
Giuseppe Guerra
09078b14e1
Plugins: Support for distributed tracing in backend plugins SDK (#63714)
* Tracing: Pass OTLP address and propagation format to plugins

* Fix unit tests

* Fix indentation

* Fix plugin manager integration tests

* Goimports

* Pass plugin version to plugins

* Do not add GF_PLUGIN_VERSION if plugin version is not set, add tests

* Allow disabling plugins distributed tracing on a per-plugin basis

* Moved disabled plugins to tracing.opentelemetry config section

* Pre-allocate DisabledPlugins map to the correct size

* Moved disable tracing setting flags in plugin settings

* Renamed plugin env vars for tracing endpoint and propagation

* Fix plugin initializer tests

* Refactoring: Moved OpentelemetryCfg from pkg/infra to pkg/plugins

* Changed GetSection to Section in parseSettingsOpentelemetry

* Add tests for NewOpentelemetryCfg

* Fix test case names in TestNewOpentelemetryCfg

* OpenTelemetry: Remove redundant error checks
2023-03-30 23:31:14 +02:00
Giuseppe Guerra
a89202eab2
Plugins: Improve instrumentation by adding metrics and tracing (#61035)
* WIP: Plugins tracing

* Trace ID middleware

* Add prometheus metrics and tracing to plugins updater

* Add TODOs

* Add instrumented http client

* Add tracing to grafana update checker

* Goimports

* Moved plugins tracing to middleware

* goimports, fix tests

* Removed X-Trace-Id header

* Fix comment in NewTracingHeaderMiddleware

* Add metrics to instrumented http client

* Add instrumented http client options

* Removed unused function

* Switch to contextual logger

* Refactoring, fix tests

* Moved InstrumentedHTTPClient and PrometheusMetrics to their own package

* Tracing middleware: handle errors

* Report span status codes when recording errors

* Add tests for tracing middleware

* Moved fakeSpan and fakeTracer to pkg/infra/tracing

* Add TestHTTPClientTracing

* Lint

* Changes after PR review

* Tests: Made "ended" in FakeSpan private, allow calling End only once

* Testing: panic in FakeSpan if span already ended

* Refactoring: Simplify Grafana updater checks

* Refactoring: Simplify plugins updater error checks and logs

* Fix wrong call to checkForUpdates -> instrumentedCheckForUpdates

* Tests: Fix wrong call to checkForUpdates -> instrumentedCheckForUpdates

* Log update checks duration, use Info log level for check succeeded logs

* Add plugin context span attributes in tracing_middleware

* Refactor prometheus metrics as httpclient middleware

* Fix call to ProvidePluginsService in plugins_test.go

* Propagate context to update checker outgoing http requests

* Plugin client tracing middleware: Removed operation name in status

* Fix tests

* Goimports tracing_middleware.go

* Goimports

* Fix imports

* Changed span name to plugins client middleware

* Add span name assertion in TestTracingMiddleware

* Removed Prometheus metrics middleware from grafana and plugins updatechecker

* Add span attributes for ds name, type, uid, panel and dashboard ids

* Fix http header reading in tracing middlewares

* Use contexthandler.FromContext, add X-Query-Group-Id

* Add test for RunStream

* Fix imports

* Changes from PR review

* TestTracingMiddleware: Changed assert to require for didPanic assertion

* Lint

* Fix imports
2023-03-28 11:01:06 +02:00
Dave Henderson
c0b5d2dfaa
Tracing: Support multiple OTel propagators (#61199)
* tracing: Support multiple OTel propagators

Signed-off-by: Dave Henderson <dave.henderson@grafana.com>

* tracing: add TraceIDString method to fix up tests

This method will be useful elsewhere if we want to log the trace ID.

* improve propagation docs

Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com>

* doc style fix

Signed-off-by: Dave Henderson <dave.henderson@grafana.com>

* Use tracing.TraceIDFromContext instead of adding TraceIDString method

Signed-off-by: Dave Henderson <dave.henderson@grafana.com>

---------

Signed-off-by: Dave Henderson <dave.henderson@grafana.com>
Co-authored-by: Bryan Boreham <bjboreham@gmail.com>
Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com>
2023-03-27 14:56:24 -04:00
Serge Zaitsev
f531074d89
Chore: Fix goimports grouping in pkg/infra (#62421)
* fix goimports

* fix goimports order
2023-01-30 08:32:25 +00:00
Emil Tullstedt
6aef353b59
Tracing: Fix bug where errors are not reported to OTel (#55925) 2022-09-28 15:54:47 +02:00
Emil Tullstedt
b0b1e8d568
Tracing: Document Tracer and Span interfaces (#55926) 2022-09-28 15:53:48 +02:00
Marcus Efraimsson
862a6a2fa6
Logging: Introduce API for contextual logging (#55198)
Introduces a FromContext method on the log.Logger interface that 
allows contextual key/value pairs to be attached, e.g. per request, 
so that any logger using this API will automatically get the per request 
context attached. The proposal makes the traceID available for 
contextual logger , if available, and would allow logs originating from 
a certain HTTP request to be correlated with traceID.
In addition, when tracing not enabled, skip adding
traceID=00000000000000000000000000000000
to logs.
2022-09-20 18:32:06 +02:00
Dave Henderson
801b61c963
Tracing: Add new [tracing.opentelemetry] custom_attributes config setting (#54110)
* tracing: Add new [tracing.opentelemetry] custom_attributes config setting

Signed-off-by: Dave Henderson <dave.henderson@grafana.com>

* Fix typos in config

Signed-off-by: Dave Henderson <dave.henderson@grafana.com>

* Return error when custom_attributes contains malformed entries

Signed-off-by: Dave Henderson <dave.henderson@grafana.com>

Signed-off-by: Dave Henderson <dave.henderson@grafana.com>
2022-09-16 09:54:25 -04:00
ying-jeanne
4d4ecd7fec
fix the event attributes (#54117) 2022-08-24 09:59:06 -05:00
zhichli
806fb8ab7a
Tracing: Fix OpenTelemetry Jaeger context propagation (#53269)
* fix otel jaeger context propagation

* add back launch.json

* add back launch.json
2022-08-11 16:11:34 +02:00
ying-jeanne
56f3f3fe69
Chore: Remove the old log format (#51526)
* remove the old log format

* fix CI

Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>
2022-07-08 09:56:30 -05:00
Serge Zaitsev
ae9491c3a7
Chore: Make test tracer noop and return no errors (#50797) 2022-06-15 12:40:41 +02:00
idafurjes
24c6a73095
Tracing: Deprecate opentracing (#50058)
* Deprecate opentracing

* Fix comment

* Adjust comment

* Fix docs for opentelemetry

* Add deprecated to sample.ini
2022-06-02 14:13:00 +02:00
Dimitris Sotirakis
0d7a3209e7
etcd: Fix vuln CVE-2018-1098 (#49976)
* Update etcd

* Update go.sum

* Replace etcd with etcd/v3
2022-06-01 09:31:34 +02:00
Serge Zaitsev
31ff23f542
Fix: add default nop trace exporter to opentelemetry (#48869) 2022-05-10 11:29:36 +02:00
idafurjes
abb1618291
Add OTLP exporter for OpenTelemetry (#47987)
* Add OTLP exporter for OpenTelemtry

* Fix lint

* Refactore parse settings

* Add configuration for propagation + fix tests

* Fix tests and lint

* Fix alerting tests

* Add coments to config

* Add propagation to custom.ini
2022-05-05 10:37:26 +02:00
Serge Zaitsev
41012af997
Tracing: Use common traceID context value for opentracing and opentelemetry (#46411)
* use common traceID context value for opentracing and opentelemetry

* support sampled trace IDs as well

* inject traceID into NormalResponse on errors

* Finally the test passed

* fix the test

* fix linter

* change the function parameter

Co-authored-by: Ying WANG <ying.wang@grafana.com>
2022-04-14 17:54:49 +02:00
idafurjes
30aa24a183
Chore: Implement OpenTelemtry in Grafana (#42674)
* Separate Tracer interface to TracerService and Tracer

* Fix lint

* Fix:Make it possible to start spans for both opentracing and opentelemetry in ds proxy

* Add span methods, use span interface for rest of tracing

* Fix logs in tracing

* Fix tests that are related to tracing

* Fix resourcepermissions test

* Fix some tests

* Fix more tests

* Add TracingService to wire cli runner

* Remove GlobalTracer from bus

* Renaming test function

* Remove GlobalTracer from TSDB

* Replace GlobalTracer in api

* Adjust tests to the InitializeForTests func

* Remove GlobalTracer from services

* Remove GlobalTracer

* Remove bus.NewTest

* Remove Tracer interface

* Add InitializeForBus

* Simplify tests

* Clean up tests

* Rename TracerService to Tracer

* Update pkg/middleware/request_tracing.go

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>

* Initialize tracer before passing it to SQLStore initialization in commands

* Remove tests for opentracing

* Set span attributes correctly, remove unnecessary trace initiliazation form test

* Add tracer instance to newSQLStore

* Fix changes due to rebase

* Add modified tracing middleware test

* Fix opentracing implementation tags

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2022-01-20 11:10:12 +01:00
idafurjes
d993b12415
Add interface Tracer, add Opentelemetry (#41963)
* Add interface Tracer, add Opentelemetry

* Fix lint

* Fix failing tests and return error if config not parsed fo opentelemetry

* Update defaults.ini

Add comment with jaeger url

* go mod tidy

* Remove comments that are not needed

* Move OpentracingSpan to tracing.go

* Add opentelemetry to sample.ini
2021-12-01 17:05:08 +01:00
Arve Knudsen
78596a6756
Migrate to Wire for dependency injection (#32289)
Fixes #30144

Co-authored-by: dsotirakis <sotirakis.dim@gmail.com>
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
Co-authored-by: Ida Furjesova <ida.furjesova@grafana.com>
Co-authored-by: Jack Westbrook <jack.westbrook@gmail.com>
Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
Co-authored-by: Leon Sorokin <leeoniya@gmail.com>
Co-authored-by: Andrej Ocenas <mr.ocenas@gmail.com>
Co-authored-by: spinillos <selenepinillos@gmail.com>
Co-authored-by: Karl Persson <kalle.persson@grafana.com>
Co-authored-by: Leonard Gram <leo@xlson.com>
2021-08-25 15:11:22 +02:00
Serge Zaitsev
e244267b7d
Enable tracing when jaeger host and port are set (#33682)
* check for jaeger env variables if tracing address configuration is empty

* add tests to ensure jaeger host/port variables override the settings

* allow default host and custom port, too

* disallow missing tracing.jaeger section in the config

* check for all errors in tests

* make parseSettings() return an error
2021-05-05 11:34:13 +02:00
Arve Knudsen
c2cad26ca9
Chore: Disable default golangci-lint filter (#29751)
* Disable default golangci-lint filter

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Chore: Fix linter warnings

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2020-12-15 09:32:06 +01:00
Carl Bergquist
e3c7d66324
Tracing: Add setting for sampling server (#29011)
Signed-off-by: bergquist <carl.bergquist@gmail.com>
Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com>
Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>
2020-11-11 10:21:52 +01:00
Arve Knudsen
5070f7a75b
Chore: Start harmonizing linting with plugin SDK (#25854)
* Chore: Harmonize linting with plugin SDK

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Chore: Fix linting issues

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
2020-06-29 14:08:32 +02:00
Dave Henderson
66b70b50b8 Tracing: Support configuring Jaeger client from environment (#21103)
* Tracing: Support configuring Jaeger client from environment
* Tracing: Replace deprecated Jaeger Client call
2019-12-18 20:09:15 +01:00
Arve Knudsen
7349a6b96c
pkg/infra: Check errors (#19705)
* pkg/infra: Check errors

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* pkg/infra: Handle errors

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>

* Update pkg/infra/usagestats/usage_stats.go

Co-Authored-By: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2019-10-10 16:42:11 +02:00