Commit Graph

631 Commits

Author SHA1 Message Date
Giuseppe Guerra
1c3ad81826
Plugins: Fix module.js file not being closed when loading plugins (#66288)
* Plugins: Loader: Fix module.js file not being closed

* Plugins: LocalFS: Add comments, ensure same Read() behaviour as os.File's

* Changed comment for Close()

* Add tests for LocalFS

* "Fix" linter error

* "Fix" linter error again
2023-04-13 10:48:15 +02:00
Michael Mandrus
5626461b3c
Caching: Refactor enterprise query caching middleware to a wire service (#65616)
* define initial service and add to wire

* update caching service interface

* add skipQueryCache header handler and update metrics query function to use it

* add caching service as a dependency to query service

* working caching impl

* propagate cache status to frontend in response

* beginning of improvements suggested by Lean - separate caching logic from query logic.

* more changes to simplify query function

* Decided to revert renaming of function

* Remove error status from cache request

* add extra documentation

* Move query caching duration metric to query package

* add a little bit of documentation

* wip: convert resource caching

* Change return type of query service QueryData to a QueryDataResponse with Headers

* update codeowners

* change X-Cache value to const

* use resource caching in endpoint handlers

* write resource headers to response even if it's not a cache hit

* fix panic caused by lack of nil check

* update unit test

* remove NONE header - shouldn't show up in OSS

* Convert everything to use the plugin middleware

* revert a few more things

* clean up unused vars

* start reverting resource caching, start to implement in plugin middleware

* revert more, fix typo

* Update caching interfaces - resource caching now has a separate cache method

* continue wiring up new resource caching conventions - still in progress

* add more safety to implementation

* remove some unused objects

* remove some code that I left in by accident

* add some comments, fix codeowners, fix duplicate registration

* fix source of panic in resource middleware

* Update client decorator test to provide an empty response object

* create tests for caching middleware

* fix unit test

* Update pkg/services/caching/service.go

Co-authored-by: Arati R. <33031346+suntala@users.noreply.github.com>

* improve error message in error log

* quick docs update

* Remove use of mockery. Update return signature to return an explicit hit/miss bool

* create unit test for empty request context

* rename caching metrics to make it clear they pertain to caching

* Update pkg/services/pluginsintegration/clientmiddleware/caching_middleware.go

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>

* Add clarifying comments to cache skip middleware func

* Add comment pointing to the resource cache update call

* fix unit tests (missing dependency)

* try to fix mystery syntax error

* fix a panic

* Caching: Introduce feature toggle to caching service refactor (#66323)

* introduce new feature toggle

* hide calls to new service behind a feature flag

* remove licensing flag from toggle (misunderstood what it was for)

* fix unit tests

* rerun toggle gen

---------

Co-authored-by: Arati R. <33031346+suntala@users.noreply.github.com>
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2023-04-12 12:30:33 -04:00
Andres Martinez Gotor
52f39e6fa0
Chore: Soft failure in plugin initialization (#66210) 2023-04-12 12:34:16 +03:00
Andres Martinez Gotor
48de17945f
Chore: Use maintained crypto library (#66304) 2023-04-12 11:18:38 +02:00
ismail simsek
a1bc227228
Instrumentation: Report the size of plugin request (#66149)
* Report the size of the plugin request

* Remove middleware, report directly

* PR review updates
2023-04-11 22:04:32 +02:00
Giuseppe Guerra
facd4eca7e
Plugins: Fix plugin signature calculation not working on Windows (#66273)
* Plugins: Fix plugin signature calculation not working on Windows

* Plugins: Added test for NTFS path separators in signature verification

* Use filepath.ToSlash and replace its implementation in tests

* Fix typo
2023-04-11 17:15:34 +02:00
Giuseppe Guerra
652c6dbd22
Plugins: Refactoring: Move plugins tracing env vars to separate func (#66170) 2023-04-07 17:11:47 +02:00
Stephanie Hingtgen
536fad5b61
Plugins: add env variables needed for the secure socks proxy in the plugins sdk (#65878) 2023-04-07 08:20:25 -06:00
Will Browne
7808e74260
Plugins: Skip instrumenting plugin build info for core and bundled plugins (#66105)
* only instrument plugin build for non core/bundled plugins

* fix import
2023-04-06 12:50:50 +02:00
Will Browne
bff9f4c890
Plugins: Move config factory to pluginsintegration package (#65716)
* move config to pluginsintegration package

* change to all plugin toggle

* fixes

* fixes

* fix lerna

* fix test
2023-04-05 14:40:08 +02:00
Ryan McKinley
d974e5f25a
TrendPanel: Add new trend panel (Alpha) (#65740) 2023-04-04 20:52:20 -07:00
Giuseppe Guerra
f1eddbad06
Plugins: Tracing: Add test for composite propagation (#65917) 2023-04-04 23:16:57 +02:00
Will Browne
819c2f4ad8
Plugins: Remove unnecessary CDN code (#65929)
remove unnecessary code
2023-04-04 17:01:49 +02:00
Giuseppe Guerra
2647b2e0f9
Plugins: Tracing: Disable distributed tracing for plugins by default (#65796)
* Plugins: Tracing: Disable tracing by default, pass GF_VERSION only if tracing is enabled

* Plugins: Tracing: Fix GF_PLUGIN_VERSION instead of GF_VERSION
2023-04-03 12:48:24 +02:00
Giuseppe Guerra
09078b14e1
Plugins: Support for distributed tracing in backend plugins SDK (#63714)
* Tracing: Pass OTLP address and propagation format to plugins

* Fix unit tests

* Fix indentation

* Fix plugin manager integration tests

* Goimports

* Pass plugin version to plugins

* Do not add GF_PLUGIN_VERSION if plugin version is not set, add tests

* Allow disabling plugins distributed tracing on a per-plugin basis

* Moved disabled plugins to tracing.opentelemetry config section

* Pre-allocate DisabledPlugins map to the correct size

* Moved disable tracing setting flags in plugin settings

* Renamed plugin env vars for tracing endpoint and propagation

* Fix plugin initializer tests

* Refactoring: Moved OpentelemetryCfg from pkg/infra to pkg/plugins

* Changed GetSection to Section in parseSettingsOpentelemetry

* Add tests for NewOpentelemetryCfg

* Fix test case names in TestNewOpentelemetryCfg

* OpenTelemetry: Remove redundant error checks
2023-03-30 23:31:14 +02:00
Will Browne
7bbe255150
Plugins: Add file store abstraction for handling plugin files (#65432)
* add file store

* fix markdown fetch bug

* add markdown tests

* fix var name
2023-03-29 12:55:55 +02:00
Giuseppe Guerra
a89202eab2
Plugins: Improve instrumentation by adding metrics and tracing (#61035)
* WIP: Plugins tracing

* Trace ID middleware

* Add prometheus metrics and tracing to plugins updater

* Add TODOs

* Add instrumented http client

* Add tracing to grafana update checker

* Goimports

* Moved plugins tracing to middleware

* goimports, fix tests

* Removed X-Trace-Id header

* Fix comment in NewTracingHeaderMiddleware

* Add metrics to instrumented http client

* Add instrumented http client options

* Removed unused function

* Switch to contextual logger

* Refactoring, fix tests

* Moved InstrumentedHTTPClient and PrometheusMetrics to their own package

* Tracing middleware: handle errors

* Report span status codes when recording errors

* Add tests for tracing middleware

* Moved fakeSpan and fakeTracer to pkg/infra/tracing

* Add TestHTTPClientTracing

* Lint

* Changes after PR review

* Tests: Made "ended" in FakeSpan private, allow calling End only once

* Testing: panic in FakeSpan if span already ended

* Refactoring: Simplify Grafana updater checks

* Refactoring: Simplify plugins updater error checks and logs

* Fix wrong call to checkForUpdates -> instrumentedCheckForUpdates

* Tests: Fix wrong call to checkForUpdates -> instrumentedCheckForUpdates

* Log update checks duration, use Info log level for check succeeded logs

* Add plugin context span attributes in tracing_middleware

* Refactor prometheus metrics as httpclient middleware

* Fix call to ProvidePluginsService in plugins_test.go

* Propagate context to update checker outgoing http requests

* Plugin client tracing middleware: Removed operation name in status

* Fix tests

* Goimports tracing_middleware.go

* Goimports

* Fix imports

* Changed span name to plugins client middleware

* Add span name assertion in TestTracingMiddleware

* Removed Prometheus metrics middleware from grafana and plugins updatechecker

* Add span attributes for ds name, type, uid, panel and dashboard ids

* Fix http header reading in tracing middlewares

* Use contexthandler.FromContext, add X-Query-Group-Id

* Add test for RunStream

* Fix imports

* Changes from PR review

* TestTracingMiddleware: Changed assert to require for didPanic assertion

* Lint

* Fix imports
2023-03-28 11:01:06 +02:00
Will Browne
1387fec51d
Plugins: Markdown fetch retry with lowercase (#65384)
* retry with lowercase

* undo incorrect err check

* re-add defer to close file

* fix test
2023-03-27 19:44:06 +03:00
Will Browne
31d6416157
Plugins: Migrate licensing and access control to pkg/services/pluginsintegration package (#65258)
* migrate licensing + access control

* update package name
2023-03-27 11:15:37 +02:00
Ryan McKinley
7835aacc8f
Schema: Remove exclusion for timeseries and update imports (#65242) 2023-03-23 02:14:54 +02:00
Will Browne
23e3ac0ac9
Plugins: Improve client instrumentation by adding cancelled status and logging errors (#65099)
add cancelled status and log err
2023-03-21 13:42:57 +01:00
Artur Wierzbicki
4274b9414f
Chore: remove querylibrary feature toggle (#65021)
* chore: remove querylibrary

* chore: remove querylibrary

* chore: remove querylibrary
2023-03-20 20:00:14 +04:00
Will Browne
ee2dd62a1f
Plugins: Refactor loader + finder to support multiple sourcing methods (#64735)
* it's cdn time

* tidy body closing

* auto signed

* fix close

* update log name

* remove comments
2023-03-20 14:35:49 +01:00
Kristin Laemmert
8745d7ef1b
extract kindsys to external library (#64562)
* extract kindsys

* reinstate kindsys report

This may end up living somewhere else (or not! who knows!), but the important part is that I don't get rid of it right now :)

I hate the package layout (kindsysreport/codegen) for the main function and will take pretty much any alternative suggestion, but we can change also change it later.

Note that the generated report.json is in a different location - anything using this (ops something) needs to be updated.

* kindsysreport in codeowners
2023-03-15 12:04:28 -04:00
Andres Martinez Gotor
80e8ac640e
Docs: Update plugin.json docs and schema (#64297) 2023-03-10 15:07:35 +01:00
lean.dev
0b0140b761
Licensing: Send the app url to plugin (#64258) 2023-03-08 14:44:04 -03:00
Will Browne
94f39e69a3
Plugins: Migrate plugincontext, adapters and pluginsettings to pkg/services/pluginsintegration package (#64154)
* migrate plugincontext, adapter and pluginsettings

* add to CODEOWNERS

* fix imports

* fix CODEOWNERS

* take pluginsettings

* migrate wire stuff
2023-03-07 11:22:30 -05:00
Will Browne
68df83c86d
Plugins: Add Plugin FS abstraction (#63734)
* unexport pluginDir from dto

* first pass

* tidy

* naming + add mutex

* add dupe checking

* fix func typo

* interface + move logic from renderer

* remote finder

* remote signing

* fix tests

* tidy up

* tidy markdown logic

* split changes

* fix tests

* slim interface down

* fix status code

* tidy exec path func

* fixup

* undo changes

* remove unused func

* remove unused func

* fix goimports

* fetch remotely

* simultaneous support

* fix linter

* use var

* add exception for gosec warning

* fixup

* fix tests

* tidy

* rework cfg pattern

* simplify

* PR feedback

* fix dupe field

* remove g304 nolint

* apply PR feedback

* remove unnecessary gosec nolint

* fix finder loop and update comment

* fix map alloc

* fix test

* remove commented code
2023-03-07 16:47:02 +01:00
Andres Martinez Gotor
c67bb07968
Docs: sync plugin JSON info in the different files (first part) (#63750) 2023-03-07 09:14:51 +01:00
Jack Westbrook
8c8f584b41
Plugins: Extend panel menu with links from plugins (#63089)
* feat(plugins): introduce dashboard panel menu placement for adding menu items

* test: add test for getPanelMenu()

* added an unique identifier for each extension.

* added context to getPluginExtensions.

* wip

* Wip

* wiwip

* Wip

* feat: WWWIIIIPPPP 🧨

* Wip

* Renamed some of the types to align a bit better.

* added limit to how many extensions a plugin can register per placement.

* decreased number of items to 2

* will trim the lenght of titles to max 25 chars.

* wrapping configure function with error handling.

* added error handling for all scenarios.

* moved extension menu items to the bottom of the more sub menu.

* added tests for configuring the title.

* minor refactorings.

* changed so you need to specify the full path in package.json.

* wip

* removed unused type.

* big refactor to make things simpler and to centralize all configure error/validation handling.

* added missing import.

* fixed failing tests.

* fixed tests.

* revert(extensions): remove static extensions config in favour of registering via AppPlugin APIs

* removed the compose that didn't work for some reason.

* added tests just to verify that validation and error handling is tied together in configuration function.

* adding some more values to the context.

* draft validation.

* added missing tests for getPanelMenu.

* added more tests.

* refactor(extensions): move logic for validating extension link config to function

* Fixed ts errors.

* Update packages/grafana-data/src/types/app.ts

Co-authored-by: Levente Balogh <balogh.levente.hu@gmail.com>

* Update packages/grafana-runtime/src/services/pluginExtensions/extensions.test.ts

Co-authored-by: Levente Balogh <balogh.levente.hu@gmail.com>

* refactor(extensions): rename limiter -> pluginPlacementCount

* refactor(getpanelmenu): remove redundant continue statement

---------

Co-authored-by: Levente Balogh <balogh.levente.hu@gmail.com>
Co-authored-by: Marcus Andersson <marcus.andersson@grafana.com>
2023-03-02 15:42:00 +01:00
Victor Marin
a91e0a49c9
Migrate table to cue model (#61852)
* WIP table cue model

* WIP table types migration

* refactor

* WIP table cue

* docs

* veneer fix, docs

* docs
2023-03-01 17:48:36 +02:00
Victor Marin
e1bc3fad83
Heatmap: Migrate to CUE (#62173)
* IP migrate heatmap to cue

* WIP

* refactor

* change usages and refactor panelcfg

* CR modifications and docs

* refactor, enable heatmap cue generation, overwrite enum values in types.ts

* small refactor

* refactor types.ts

* cr modifications

* docs
2023-03-01 17:47:21 +02:00
Will Browne
ec82719372
Plugins: Introduce plugin package specific logger (#62204)
* refactor

* implement with infra log for now

* undo moving

* update package name

* update name

* fix tests

* update pretty signature

* update naming

* simplify

* fix typo

* delete comment

* fix import

* retrigger
2023-02-28 16:10:27 +01:00
Will Browne
ab8de1a0e3
Plugins: Add simple plugin sources service (#63814)
add simple plugin sources svc
2023-02-28 15:27:11 +01:00
Will Browne
32aa0ad272
Plugins: Remove stream handler from plugins DTO (#63812)
remove stream handler from DTO
2023-02-27 19:48:18 +01:00
Will Browne
25b3abece9
Plugins: Remove unused funcs and re-use enum types (#63813)
plugins tidy
2023-02-27 13:03:55 -05:00
Giuseppe Guerra
966bcd3545
Plugins: Fix plugins CDN not working when plugins are not in domain's root path (#63202)
* Plugins CDN: Add support for different CDN root path

* Plugins CDN: Make frontendsettings return the correct CDN base path

* Update comments

* Fix version detection

* Undo frontend changes

* Fix system.js asset path construction

* fix(plugins): translate all plugin css asset paths loaded via cdn

* refactor(plugins): rename extractPluginNameVersionFromUrl and add comments

* Fix typo in comment

Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>

* Hardcode CDN URL structure

/{id}/{version}/public/plugins/{id}/{assetPath} is not required anymore in the cdn url template config

---------

Co-authored-by: Jack Westbrook <jack.westbrook@gmail.com>
Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
2023-02-24 14:28:13 +01:00
Will Browne
200d2ad249
Plugins: Invalidate plugin context app settings cache post update (#63279)
invalid plugin settings
2023-02-24 12:13:22 +01:00
Selene
7ace44c8b5
Chore: Update oapi-codegen library (#62962)
* Update oapi library and thema

* Use fork commit to fix elasticsearch and cloudwatch generators

* Update thema

* Fixes

* Update thema with last fixes

* Sync

* Fix test

* Update thema and schemas

* Update thema
2023-02-21 12:34:24 +01:00
Marcus Andersson
f46f8bdd3a
Plugins: Renamed parts of the UI extension APIs (#63070)
* Renamed target -> id and href -> path after feedback.

* fixed type issues in test page.

* chore(pluginschemajson): update extensions props target -> id

* this is the final.

* fixed typings...again...

---------

Co-authored-by: Jack Westbrook <jack.westbrook@gmail.com>
2023-02-08 05:33:28 -05:00
Marcus Andersson
1cfd3f81fb
Plugins: Support for link extensions (#61663)
* added extensions to plugin.json and exposing it via frontend settings.

* added extensions to the plugin.json schema.

* changing the extensions in frontend settings to a map instead of an array.

* wip

* feat(pluginregistry): begin wiring up registry

* feat(pluginextensions): prevent duplicate links and clean up

* added test case for link extensions.

* added tests and implemented the getPluginLink function.

* wip

* feat(pluginextensions): expose plugin extension registry

* fix(pluginextensions): appease the typescript gods post rename

* renamed file and will throw error if trying to call setExtensionsRegistry if trying to call it twice.

* added reafactorings.

* fixed failing test.

* minor refactorings to make sure we only include extensions if the app is enabled.

* fixed some nits.

* Update public/app/features/plugins/extensions/registry.test.ts

Co-authored-by: Levente Balogh <balogh.levente.hu@gmail.com>

* Update packages/grafana-runtime/src/services/pluginExtensions/registry.ts

Co-authored-by: Levente Balogh <balogh.levente.hu@gmail.com>

* Update packages/grafana-runtime/src/services/pluginExtensions/registry.ts

Co-authored-by: Levente Balogh <balogh.levente.hu@gmail.com>

* Update public/app/features/plugins/extensions/registry.test.ts

Co-authored-by: Levente Balogh <balogh.levente.hu@gmail.com>

* Moved types for extensions from data to runtime.

* added a small example on how you could consume link extensions.

* renamed after feedback from levi.

* updated the plugindef.cue.

* using the generated plugin def.

* added tests for apps and extensions.

* fixed linting issues.

* wip

* wip

* wip

* wip

* test(extensions): fix up failing tests

* feat(extensions): freeze registry extension arrays, include type in registry items

* added restrictions in the pugindef cue schema.

* wip

* added required fields.

* added key to uniquely identify each item.

* test(pluginextensions): align tests with implementation

* chore(schema): refresh reference.md

---------

Co-authored-by: Jack Westbrook <jack.westbrook@gmail.com>
Co-authored-by: Levente Balogh <balogh.levente.hu@gmail.com>
2023-02-07 17:20:05 +01:00
Carl Bergquist
2a29a07465
Plugins: Add optional logger for plugin requests sent to backend plugins (#62981)
Signed-off-by: bergquist <carl.bergquist@gmail.com>
2023-02-07 12:49:16 +01:00
Ieva
49ccae7489
RBAC: cascaded nested folder permissions for search v2 (#62608)
* fetch inherited folder scopes for search v2

* fix error and don't fetch parents for the general folder

* remove feature toggle check and lower log level
2023-02-06 17:38:15 +00:00
Andreas Christou
d7f7cd1e61
Schema: Initial Azure Monitor query schema (#62018)
* Initial schema

- Add types based off of current frontend

* Rename and field-level comments

* Update report and regenerate files

* Rename frontend Azure folder

- Doing this for consistency and to ensure code-generation works
- Update betterer results due to file renames

* Remove default and add back enum vals that I deleted

* Set workspace prop as optional

* Replace template variable types

* Connect frontend query types

- Keep properties optional for now to avoid major changes
- Rename AzureMetricResource
- Correctly use ResultFormat

* Add TSVeneer decorator

* Update schema

* Update type

* Update CODEOWNERS

* Fix gen-cue issue

* Fix backend test

* Fix e2e test

* Update code coverage

* Remove references to old Azure Monitor path

* Review

* Regen files
2023-02-03 16:06:54 +00:00
Joe Blubaugh
41dc88bd25
Adds a description field to plugin.json's pages:role field (#62439)
* Adds a description field to plugin.json's pages:role field

There's no description of what a page object's "role" setting does. It
controls whether a page will appear in the navigation menu for a user
with a given role.

* Update plugindef.cue with comments to match documentation.

* make gen-cue
2023-02-03 14:50:24 +01:00
Michael Mandrus
7391793504
Query Caching: Add per-panel query caching TTL (#61968)
* *Create Caching Config interface and OSS impl
*Create front-end facing DS Cache config
*Populate Caching Config on Datasource DTO
*Update OSS wire deps

* fix unit test

* handle query caching TTL override on the frontend

* Make sure the override works with pubdash

* move caching config to the right place in the ds info

* move caching config logic to enterprise index hook

* move queryCachingTTL to pubdash query payload

* Remove  from metadata (not needed)

* rename struct and add comment

* remove invalid wire dependency

* manual revert of 395c74b

* fix frontend test

* fix backend test

* fix tests for real this time

* truly fix frontend test

* fix back end unit test for real
2023-02-02 23:39:54 -05:00
Agnès Toulet
121e384fab
kindsys: Fix CUE code generation on Windows (#62812)
* fix Windows paths

* fix remaining paths issues

* clean up multiple use of Dir()

* fix headers in generated files
2023-02-02 15:06:55 -05:00
idafurjes
23c27cffb3
Chore: Rename Id to ID in alerting models (#62777)
* Chore: Rename Id to ID in alerting models

* Add xorm tags for datasource

* Add xorm tag for uid
2023-02-02 17:22:43 +01:00
sam boyer
30b4205521
Kindsys: Replace DefForGen with kindsys.Kind (#62642)
* Kindsys: Replace DeclForGen with kindsys.Kind

DeclForGen was always unnecessary - it just wasn't obvious on initial
implementation, when we were focused on generating unique types for each
core kind. This removes it, considerably simplifying interactions with
kindsys - virtually everything now just relies on kindsys.Kind and its
derived interfaces.

* Removed unused jenny

* Rename params in jennies
2023-01-31 19:40:15 -05:00
sam boyer
dbe2f0c8f6
Kindsys: Replace "Declaration" with "Definition" (#62515)
* s/Declaration/Definition/g

* s/DeclForGen/DefForGen/g

* Rename some local vars
2023-01-31 09:50:08 +00:00