Commit Graph

26 Commits

Author SHA1 Message Date
Kristin Laemmert
8745d7ef1b
extract kindsys to external library (#64562)
* extract kindsys

* reinstate kindsys report

This may end up living somewhere else (or not! who knows!), but the important part is that I don't get rid of it right now :)

I hate the package layout (kindsysreport/codegen) for the main function and will take pretty much any alternative suggestion, but we can change also change it later.

Note that the generated report.json is in a different location - anything using this (ops something) needs to be updated.

* kindsysreport in codeowners
2023-03-15 12:04:28 -04:00
Jack Westbrook
8c8f584b41
Plugins: Extend panel menu with links from plugins (#63089)
* feat(plugins): introduce dashboard panel menu placement for adding menu items

* test: add test for getPanelMenu()

* added an unique identifier for each extension.

* added context to getPluginExtensions.

* wip

* Wip

* wiwip

* Wip

* feat: WWWIIIIPPPP 🧨

* Wip

* Renamed some of the types to align a bit better.

* added limit to how many extensions a plugin can register per placement.

* decreased number of items to 2

* will trim the lenght of titles to max 25 chars.

* wrapping configure function with error handling.

* added error handling for all scenarios.

* moved extension menu items to the bottom of the more sub menu.

* added tests for configuring the title.

* minor refactorings.

* changed so you need to specify the full path in package.json.

* wip

* removed unused type.

* big refactor to make things simpler and to centralize all configure error/validation handling.

* added missing import.

* fixed failing tests.

* fixed tests.

* revert(extensions): remove static extensions config in favour of registering via AppPlugin APIs

* removed the compose that didn't work for some reason.

* added tests just to verify that validation and error handling is tied together in configuration function.

* adding some more values to the context.

* draft validation.

* added missing tests for getPanelMenu.

* added more tests.

* refactor(extensions): move logic for validating extension link config to function

* Fixed ts errors.

* Update packages/grafana-data/src/types/app.ts

Co-authored-by: Levente Balogh <balogh.levente.hu@gmail.com>

* Update packages/grafana-runtime/src/services/pluginExtensions/extensions.test.ts

Co-authored-by: Levente Balogh <balogh.levente.hu@gmail.com>

* refactor(extensions): rename limiter -> pluginPlacementCount

* refactor(getpanelmenu): remove redundant continue statement

---------

Co-authored-by: Levente Balogh <balogh.levente.hu@gmail.com>
Co-authored-by: Marcus Andersson <marcus.andersson@grafana.com>
2023-03-02 15:42:00 +01:00
Victor Marin
a91e0a49c9
Migrate table to cue model (#61852)
* WIP table cue model

* WIP table types migration

* refactor

* WIP table cue

* docs

* veneer fix, docs

* docs
2023-03-01 17:48:36 +02:00
Victor Marin
e1bc3fad83
Heatmap: Migrate to CUE (#62173)
* IP migrate heatmap to cue

* WIP

* refactor

* change usages and refactor panelcfg

* CR modifications and docs

* refactor, enable heatmap cue generation, overwrite enum values in types.ts

* small refactor

* refactor types.ts

* cr modifications

* docs
2023-03-01 17:47:21 +02:00
Marcus Andersson
1cfd3f81fb
Plugins: Support for link extensions (#61663)
* added extensions to plugin.json and exposing it via frontend settings.

* added extensions to the plugin.json schema.

* changing the extensions in frontend settings to a map instead of an array.

* wip

* feat(pluginregistry): begin wiring up registry

* feat(pluginextensions): prevent duplicate links and clean up

* added test case for link extensions.

* added tests and implemented the getPluginLink function.

* wip

* feat(pluginextensions): expose plugin extension registry

* fix(pluginextensions): appease the typescript gods post rename

* renamed file and will throw error if trying to call setExtensionsRegistry if trying to call it twice.

* added reafactorings.

* fixed failing test.

* minor refactorings to make sure we only include extensions if the app is enabled.

* fixed some nits.

* Update public/app/features/plugins/extensions/registry.test.ts

Co-authored-by: Levente Balogh <balogh.levente.hu@gmail.com>

* Update packages/grafana-runtime/src/services/pluginExtensions/registry.ts

Co-authored-by: Levente Balogh <balogh.levente.hu@gmail.com>

* Update packages/grafana-runtime/src/services/pluginExtensions/registry.ts

Co-authored-by: Levente Balogh <balogh.levente.hu@gmail.com>

* Update public/app/features/plugins/extensions/registry.test.ts

Co-authored-by: Levente Balogh <balogh.levente.hu@gmail.com>

* Moved types for extensions from data to runtime.

* added a small example on how you could consume link extensions.

* renamed after feedback from levi.

* updated the plugindef.cue.

* using the generated plugin def.

* added tests for apps and extensions.

* fixed linting issues.

* wip

* wip

* wip

* wip

* test(extensions): fix up failing tests

* feat(extensions): freeze registry extension arrays, include type in registry items

* added restrictions in the pugindef cue schema.

* wip

* added required fields.

* added key to uniquely identify each item.

* test(pluginextensions): align tests with implementation

* chore(schema): refresh reference.md

---------

Co-authored-by: Jack Westbrook <jack.westbrook@gmail.com>
Co-authored-by: Levente Balogh <balogh.levente.hu@gmail.com>
2023-02-07 17:20:05 +01:00
Andreas Christou
d7f7cd1e61
Schema: Initial Azure Monitor query schema (#62018)
* Initial schema

- Add types based off of current frontend

* Rename and field-level comments

* Update report and regenerate files

* Rename frontend Azure folder

- Doing this for consistency and to ensure code-generation works
- Update betterer results due to file renames

* Remove default and add back enum vals that I deleted

* Set workspace prop as optional

* Replace template variable types

* Connect frontend query types

- Keep properties optional for now to avoid major changes
- Rename AzureMetricResource
- Correctly use ResultFormat

* Add TSVeneer decorator

* Update schema

* Update type

* Update CODEOWNERS

* Fix gen-cue issue

* Fix backend test

* Fix e2e test

* Update code coverage

* Remove references to old Azure Monitor path

* Review

* Regen files
2023-02-03 16:06:54 +00:00
Agnès Toulet
121e384fab
kindsys: Fix CUE code generation on Windows (#62812)
* fix Windows paths

* fix remaining paths issues

* clean up multiple use of Dir()

* fix headers in generated files
2023-02-02 15:06:55 -05:00
sam boyer
dbe2f0c8f6
Kindsys: Replace "Declaration" with "Definition" (#62515)
* s/Declaration/Definition/g

* s/DeclForGen/DefForGen/g

* Rename some local vars
2023-01-31 09:50:08 +00:00
Giuseppe Guerra
af1e2d68da
Plugins: Allow loading panel plugins from a CDN (#59096)
* POC: Plugins CDN reverse proxy

* CDN proxy POC: changed env var names

* Add authorization: false for /public path in frontend plugin loader

* Moved CDN settings to Cfg, add some comments

* Fix error 500 in asset fetch if plugin is not using CDN

* Fix EnterpriseLicensePath declared twice

* Fix linter complaining about whitespaces

* Plugins CDN: Skip signature verification for CDN plugins

* Plugins CDN: Skip manifest and signature check for cdn plugins

* Plugins: use IsValid() and IsInternal() rather than equality checks

* Plugins CDN: remove comment

* Plugins CDN: Fix seeker can't seek when serving plugins from local fs

* Plugins CDN: add back error codes in getLocalPluginAssets

* Plugins CDN: call asset.Close() rather than asset.readSeekCloser.Close()

* Plugins CDN: Fix panic in JsonApiErr when errorMessageCoder wraps a nil error

* Plugins CDN: Add error handling to proxyCDNPluginAsset

* Plugins CDN: replace errorMessageCoder with errutil

* Plugins CDN POC: expose cdn plugin paths to frontend for system.js

* Plugins CDN: Fix cdn plugins showing as unsigned in frontend

* WIP: Add support for formatted URL

* Fix missing cdnPluginsBaseURLs in GrafanaConfig

* Plugins CDN: Remove reverse proxy mode and reverse proxy references

* Plugins CDN: Simplify asset serving logic

* Plugins CDN: sanitize redirect path

* Plugins CDN: Removed unused pluginAsset type

* Plugins CDN: Removed system.js changes

* Plugins CDN: Return different system.js baseURL and module for cdn plugins

* Plugins CDN: Ensure CDN is disabled for non-external plugins

* lint

* Plugins CDN: serve images and screenshots from CDN, refactoring

* Lint

* Plugins CDN: Fix URLs for system.js (baseUrl and module)

* Plugins CDN: Add more tests for RelativeURLForSystemJS

* Plugins CDN: Iterate only on apps when preloading

* Plugins CDN: Refactoring

* Plugins CDN: Add comments to url_constructor.go

* Plugins CDN: Update defaultHGPluginsCDNBaseURL

* Plugins CDN: undo extract meta from system js config

* refactor(plugins): migrate systemjs css plugin to typescript

* feat(plugins): introduce systemjs cdn loader plugin

* feat(plugins): add systemjs load type

* Plugins CDN: Removed RelativeURLForSystemJS

* Plugins CDN: Log backend redirect hits along with plugin info

* Plugins CDN: Add pluginsCDNBasePath to getFrontendSettingsMap

* feat(plugins): introduce cdn loading for angular plugins

* refactor(plugins): move systemjs cache buster into systemjsplugins directory

* Plugins CDN: Rename pluginsCDNBasePath to pluginsCDNBaseURL

* refactor(plugins): introduce pluginsCDNBaseURL to the frontend

* Plugins CDN: Renamed "cdn base path" to "cdn url template" in backend

* Plugins CDN: lint

* merge with main

* Instrumentation: Add prometheus counter for backend hits, log from Info to Warn

* Config: Changed key from plugins_cdn.url to plugins.plugins_cdn_base_url

* CDN: Add backend tests

* Lint: goimports

* Default CDN URL to empty string,

* Do not use CDN in setImages and module if the url template is empty

* CDN: Backend: Add test for frontend settings

* CDN: Do not log missing module.js warn if plugin is being loaded from CDN

* CDN: Add backend test for CDN plugin loader

* Removed 'cdn' signature level, switch to 'valid'

* Fix pfs.TestParseTreeTestdata for cdn plugin testdata dir

* Fix TestLoader_Load

* Fix gocyclo complexity of loadPlugins

* Plugins CDN: Moved prometheus metric to api package, removed asset_path label

* Fix missing  in config

* Changes after review

* Add pluginscdn.Service

* Fix tests

* Refactoring

* Moved all remaining CDN checks inside pluginscdn.Service

* CDN url constructor: Renamed stringURLFor to stringPath

* CDN: Moved asset URL functionality to assetpath service

* CDN: Renamed HasCDN() to IsEnabled()

* CDN: Replace assert with require

* CDN: Changes after review

* Assetpath: Handle url.Parse error

* Fix plugin_resource_test

* CDN: Change fallback redirect from 302 to 307

* goimports

* Fix tests

* Switch to contextmodel.ReqContext in plugins.go

Co-authored-by: Will Browne <will.browne@grafana.com>
Co-authored-by: Jack Westbrook <jack.westbrook@gmail.com>
2023-01-27 15:08:17 +01:00
Victor Marin
6a93c77082
Refactor state timeline/status history to cue model and refactor TimelineChart component (#61631)
* Adapt state timeline to scuemata

* Refactor status history to cue model

* Refactor

* Refactor TimelineChart as a core component

* wip

* Change as per CR

Co-authored-by: sam boyer <sdboyer@grafana.com>
2023-01-26 07:03:59 +00:00
Tania
5f93e67419
Codegen: Generate docs for composable kinds (#61719)
* Generate docs for composable kinds

* Fix getting the kind schema

* Generate composable kinds docs

* Add disclaimer to the generated docs

* Change docs disclaimer
2023-01-22 18:58:49 -05:00
sam boyer
3b3059c9ce
Kindsys: Unify plugins, pfs with kind framework (#61192)
* New pfs impl

* Reached codegen parity with old system

* Update all models.cue inputs

* Rename all models.cue files

* Remove unused prefixfs

* Changes Queries->DataQuery schema interface

* Recodegen

* All tests passing, nearly good now

* Add SchemaInterface to kindsys props

* Add pascal name deriver

* Relocate plugin cue files again

* Clarify use of injected fields

* Remove unnecessary aliasing

* Move DataQuery into mudball

* Allow forcing ExpandReferences on go type generation

* Move DataQuery def into kindsys, add generator to copy it to common

* Fix copy generator to replace package name correctly

* Fix duplicate type, test failure

* Fix linting issues
2023-01-20 09:41:35 +00:00
sam boyer
492bdbd585
Kindsys: Rename mudball to common schemas, allow multiple input cue files (#61703)
* kindsys: Rename mudball to common schemas

* Update all core plugin imports

* Add missed table types

* Remove invalid JSON comment

* Remove coremodel dir from codeowners
2023-01-18 17:40:22 +00:00
sam boyer
4db3b2fd5c
Kindsys: Remove defs, Slot->SchemaInterface (#61069)
* kindsys: Remove defs, Slot->SchemaInterface

* Remove excess file

* Fix up tests

* Regenerate kinds report

* Final bits of cleanup

* Stop complaining, linter

* Update pkg/kindsys/kindcat_composable.cue

Co-authored-by: Tania <yalyna.ts@gmail.com>

Co-authored-by: Tania <yalyna.ts@gmail.com>
2023-01-06 12:37:32 -05:00
Jack Baldry
410b23c3d9
Use preferred package header for generated code (#61059)
* Use preferred package header for generated code

> To convey to humans and machine tools that code is generated, generated source should have a line that matches the following regular expression (in Go syntax):
> `^// Code generated .* DO NOT EDIT\.$`

https://pkg.go.dev/cmd/go#hdr-Generate_Go_files_by_processing_source

Co-authored-by: sam boyer <sdboyer@grafana.com>
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Regenerate files with updated header

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
Co-authored-by: sam boyer <sdboyer@grafana.com>
2023-01-06 04:57:45 -05:00
Will Browne
48b33ab521
Plugins: Unsigned chromium file should not invalidate signature for Renderer plugin (#59104)
* Plugins: Unsigned chromium file should not invalidate signature for Renderer plugin

* fix test

* re-work solution
2022-12-19 12:46:27 +01:00
Marcus Andersson
7f92f1df00
Schema: Refactor plugin code generation (#58901)
* wip

* wip

* almost there..

* wip - change so it can run.

* treelist is working.

* support CODEGEN_VERIFY env variable

* use log.fatal

* comment out old PluginTreeList code generation

* cleanup

* rename corelist package files

* fix makefile

* move pkg/codegen/pluggen.go to pkg/plugins/codegen

* copy and refactor files to pkg/plugins/codegen

* use pkg/plugins/codegen instead of pkg/codegen for core plugins code gen

* remove unneeded files

* remove unused code to resolve linting errors

* adapters first hack

* added flattener

* add back ignore build tags to go generate file

* cleaned up the code a bit.

* seems to work, needs to do some refactoring of the GoTypesJenns and TSTypesJenny.

* one more step, going to get upstream changes in this branch.

* working but need to run import tmpl in jenny_schemapath to have the proper imports.

* added header to generated files.

* added missing jenny.

* preventing plugins with multiple decls/schemas to insert multiple lines in corelist.

* fixed so we use Slot type from kindsys to detect if its group.

* adding a go jenny that only runs if the plugin has a backend.

* added version object to generated ts.

* generating the ts types with the same output as prior to this refactoring.

* removed code that is replaced by the jenny pattern.

* removed the go code that isn't used anymore.

* removed some more unused code and renamed pluggen to util_ts

* fixed linting issue.

* removed unused vars.

* use a jenny list postprocessor for header injection

* moved decl and decl_parser to pfs.

* removed the pre-pended header in the gotypes jenny since it is done in the postprocess.

* moved decl to pfs.

* removed unused template.

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2022-12-02 08:22:28 +01:00
Denis Limarev
b814c66c1d
Perfomance: add preallocation for some slices (#59263)
add preallocation for some slices
2022-11-28 13:10:24 +01:00
sam boyer
78f0340031
plugindef: Move pluginmeta out of coremodels as standalone thema lineage (#56765)
* Get pluginmeta mostly moved over to pkg/plugins/plugindef

* Remove dead func

* Fix up pfs, use sync.Once in plugindef

* Update to latest thema

* Chase Endec->Codec conversion in Thema

* Comments on slash header gen; use ToSlash

* Also generate JSON schema for plugindef

* Generate JSON Schema as well

* Fix slot loading from kindsys cue decls

* Remove unused vars

* skip generating plugin.schema.json for now

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2022-11-15 14:48:31 +01:00
Gabriel MABILLE
30fae33f66
RBAC: Allow role registration for plugins (#57387)
* Picking role registration from OnCall POC branch

* Fix test

* Remove include actions from this PR

* Removing unused permission

* Adding test to DeclarePluginRoles

* Add testcase to RegisterFixed role

* Additional test case

* Adding tests to validate plugins roles

* Add test to plugin loader

* Nit.

* Scuemata validation

* Changing the design to decouple accesscontrol from plugin management

Co-authored-by: Kalle Persson <kalle.persson@grafana.com>

* Fixing tests

Co-authored-by: Jguer <joao.guerreiro@grafana.com>

* Add missing files

Co-authored-by: Jguer <joao.guerreiro@grafana.com>

* Remove feature toggle check from loader

* Remove feature toggleimport

* Feedback

Co-Authored-By: marefr <marcus.efraimsson@gmail.com>

* Fix test'

* Make plugins.RoleRegistry interface typed

* Remove comment question

* No need for json tags anymore

* Nit. log

* Adding the schema validation

* Remove group to take plugin Name instead

* Revert sqlstore -> db

* Nit.

* Nit. on tests

Co-authored-by: ievaVasiljeva <ieva.vasiljeva@grafana.com>

* Update pkg/services/accesscontrol/plugins.go

Co-authored-by: Ieva <ieva.vasiljeva@grafana.com>

* Log message

Co-Authored-By: marefr <marcus.efraimsson@gmail.com>

* Log message

Co-Authored-By: marefr <marcus.efraimsson@gmail.com>

* Remove unecessary method. Update test name.

Co-authored-by: ievaVasiljeva <ieva.vasiljeva@grafana.com>

* Fix linting

* Update cue descriptions

* Fix test

Co-authored-by: Kalle Persson <kalle.persson@grafana.com>
Co-authored-by: Jguer <joao.guerreiro@grafana.com>
Co-authored-by: marefr <marcus.efraimsson@gmail.com>
Co-authored-by: ievaVasiljeva <ieva.vasiljeva@grafana.com>
2022-11-07 11:30:45 +01:00
Andrej Ocenas
0845ac2f53
Profiling: Add Phlare and Parca datasources (#57809)
* Add phlare datasource

* Rename

* Add parca

* Add self field to parca

* Make sure phlare works with add to dashboard flow

* Add profiling category and hide behind feature flag

* Update description and logos

* Update phlare icon

* Cleanup logging

* Clean up logging

* Fix for shift+enter

* onRunQuery to set label

* Update type naming

* Fix lint

* Fix test and quality issues

Co-authored-by: Joey Tawadrous <joey.tawadrous@grafana.com>
2022-10-28 13:33:37 +02:00
sam boyer
e5a6547a94
coremodels: Update to latest Thema with generics (#56602)
* Update thema to latest

* Deal with s/Library/*Runtime/

* Commit new, working results of codegen
2022-10-11 09:45:07 +01:00
Joey Tawadrous
74c809f544
Plugins: Introduce new Flame graph panel (#56376)
* Flamegraph

* Updated flame graph width/height values

* Fix top table rendering issue

* Add feature toggle for flamegraph in explore

* Update tests

* Hide flamegraph from dash panel viz list if feature toggle not enabled

* Show table if no flameGraphFrames

* Add flame graph to testdata ds

* Minor improvement
2022-10-07 11:39:14 +01:00
sam boyer
ced53a8dc2
plugins: Introduce generated, static core plugin registry (#54118)
* Refactor towards template/codegen framework

* Add templates for plugin gen

* Add Go codegen for plugins; overhaul framework, too

* Add new codegen output; assorted framework fixes

* Regenerate after merge

* Remove accidental commit file, update templates

* Export the pfs.Tree loader from plugin types

* Print details from cuetsy errors

* Generate loaders for all plugins and list in registry

* Use pfs_gen.go over lineage_gen.go

* Un-un-ignore main file

* Introduce simple List static registry for plugins

* Last tweaks to codegen

* remove unused tvars

* Ensure loop-local instances for both vars

* Generate pfs parsing in-place in registry

* Stop generating pfs_gen.go

* Move Tree into pfs, rename subdir

* Change package name to match dir

* Ignore gocyclo on HTTPServer.getNavTree
2022-09-14 10:15:09 -04:00
Will Browne
26dfdd5af3
Plugins: Refactor plugin download/installation (#43046)
* installer -> repo

* add semver format checking

* add plugin callbacks in test

* remove newline

* post install only scans new directories

* remove unused stuff

* everything in own package

* add missing cli params

* make grafana version part of the API

* resolve conflicts

* tidy up logger

* fix cli and tidy log statements

* rename log package

* update struct name

* fix linter issue

* fs -> filestore

* reorder imports

* alias import

* fix test

* fix test

* inline var

* revert jsonc file

* make repo dep of manager

* actually inject the thing

* accept all args for compatability checks

* accept compat from store

* pass os + arch vals

* don't inject fs

* tidy up

* tidy up

* merge with main and tidy fs storage

* fix test

* fix packages

* fix comment + field name

* update fs naming

* fixed wire

* remove unused func

* fix mocks

* fix storage test

* renaming

* fix log line

* fix test

* re-order field

* tidying

* add test for update with same version

* fix wire for CLI

* remove use of ioutil

* don't pass field

* small tidy

* ignore code scanning warn

* fix testdata link

* update lgtm code
2022-08-23 11:50:50 +02:00
sam boyer
4d433084a5
plugins: New static scanner and validator, with Thema slot support (#53754)
* coremodels: Convert plugin-metadata schema to a coremodel

* Newer cuetsy; try quoting field name

* Add slot definitions

* Start sketching out pfs package

* Rerun codegen with fixes, new cuetsy

* Catch up dashboard with new cuetsy

* Update to go1.18

* Use new vmuxers in thema

* Add slot system in Go

* Draft finished implementation of pfs

* Collapse slot pkg into coremodel dir; add PluginInfo

* Add the mux type on top of kernel

* Refactor plugin generator for extensibility

* Change models.cue package, numerous debugs

* Bring new output to parity with old

* Remove old plugin generation logic

* Misc tweaking

* Reintroduce generation of shared schemas

* Drop back to go1.17

* Add globbing to tsconfig exclude

* Introduce pfs test on existing testdata

* Make most existing testdata tests pass with pfs

* coremodels: Convert plugin-metadata schema to a coremodel

* Newer cuetsy; try quoting field name

* Add APIType control concept, regen pluginmeta

* Use proper numeric types for schema fields

* Make pluginmeta schema follow Go type breakdown

* More decomposition into distinct types

* Add test case for no plugin.json file

* Fix missing ref to #Dependencies

* Remove generated TS for pluginmeta

* Update dependencies, rearrange go.mod

* Regenerate without Model prefix

* Use updated thema loader; this is now runnable

* Skip app plugin with weird include

* Make plugin tree extractor reusable

* Split out slot lineage load/validate logic

* Add myriad tests for new plugin validation failures

* Add test for zip fixtures

* One last run of codegen

* Proper delinting

* Ensure validation order is deterministic

* Let there actually be sorting

* Undo reliance on builtIn field (#54009)

* undo builtIn reliance

* fix tests

Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
2022-08-22 12:11:45 -04:00