grafana/pkg/plugins/pfs/pfs_test.go

298 lines
7.3 KiB
Go
Raw Normal View History

plugins: New static scanner and validator, with Thema slot support (#53754) * coremodels: Convert plugin-metadata schema to a coremodel * Newer cuetsy; try quoting field name * Add slot definitions * Start sketching out pfs package * Rerun codegen with fixes, new cuetsy * Catch up dashboard with new cuetsy * Update to go1.18 * Use new vmuxers in thema * Add slot system in Go * Draft finished implementation of pfs * Collapse slot pkg into coremodel dir; add PluginInfo * Add the mux type on top of kernel * Refactor plugin generator for extensibility * Change models.cue package, numerous debugs * Bring new output to parity with old * Remove old plugin generation logic * Misc tweaking * Reintroduce generation of shared schemas * Drop back to go1.17 * Add globbing to tsconfig exclude * Introduce pfs test on existing testdata * Make most existing testdata tests pass with pfs * coremodels: Convert plugin-metadata schema to a coremodel * Newer cuetsy; try quoting field name * Add APIType control concept, regen pluginmeta * Use proper numeric types for schema fields * Make pluginmeta schema follow Go type breakdown * More decomposition into distinct types * Add test case for no plugin.json file * Fix missing ref to #Dependencies * Remove generated TS for pluginmeta * Update dependencies, rearrange go.mod * Regenerate without Model prefix * Use updated thema loader; this is now runnable * Skip app plugin with weird include * Make plugin tree extractor reusable * Split out slot lineage load/validate logic * Add myriad tests for new plugin validation failures * Add test for zip fixtures * One last run of codegen * Proper delinting * Ensure validation order is deterministic * Let there actually be sorting * Undo reliance on builtIn field (#54009) * undo builtIn reliance * fix tests Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
2022-08-22 11:11:45 -05:00
package pfs
import (
"archive/zip"
"io/fs"
"os"
"path/filepath"
"sort"
"testing"
"github.com/google/go-cmp/cmp"
"github.com/grafana/grafana/pkg/cuectx"
"github.com/stretchr/testify/require"
)
func TestParsePluginTestdata(t *testing.T) {
plugins: New static scanner and validator, with Thema slot support (#53754) * coremodels: Convert plugin-metadata schema to a coremodel * Newer cuetsy; try quoting field name * Add slot definitions * Start sketching out pfs package * Rerun codegen with fixes, new cuetsy * Catch up dashboard with new cuetsy * Update to go1.18 * Use new vmuxers in thema * Add slot system in Go * Draft finished implementation of pfs * Collapse slot pkg into coremodel dir; add PluginInfo * Add the mux type on top of kernel * Refactor plugin generator for extensibility * Change models.cue package, numerous debugs * Bring new output to parity with old * Remove old plugin generation logic * Misc tweaking * Reintroduce generation of shared schemas * Drop back to go1.17 * Add globbing to tsconfig exclude * Introduce pfs test on existing testdata * Make most existing testdata tests pass with pfs * coremodels: Convert plugin-metadata schema to a coremodel * Newer cuetsy; try quoting field name * Add APIType control concept, regen pluginmeta * Use proper numeric types for schema fields * Make pluginmeta schema follow Go type breakdown * More decomposition into distinct types * Add test case for no plugin.json file * Fix missing ref to #Dependencies * Remove generated TS for pluginmeta * Update dependencies, rearrange go.mod * Regenerate without Model prefix * Use updated thema loader; this is now runnable * Skip app plugin with weird include * Make plugin tree extractor reusable * Split out slot lineage load/validate logic * Add myriad tests for new plugin validation failures * Add test for zip fixtures * One last run of codegen * Proper delinting * Ensure validation order is deterministic * Let there actually be sorting * Undo reliance on builtIn field (#54009) * undo builtIn reliance * fix tests Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
2022-08-22 11:11:45 -05:00
type tt struct {
tfs fs.FS
// TODO could remove this by getting rid of inconsistent subdirs
subpath string
skip string
err error
// TODO could remove this by expecting that dirname == id
rootid string
}
tab := map[string]tt{
"app-with-child": {
rootid: "myorgid-simple-app",
subpath: "dist",
skip: "schema violation, weirdness in info.version field",
},
"duplicate-plugins": {
rootid: "test-app",
subpath: "nested",
skip: "schema violation, dependencies don't follow naming constraints",
},
"includes-symlinks": {
skip: "schema violation, dependencies don't follow naming constraints",
},
"installer": {
rootid: "test-datasource",
subpath: "plugin",
},
"invalid-plugin-json": {
rootid: "test-app",
err: ErrInvalidRootFile,
},
"invalid-v1-signature": {
rootid: "test-datasource",
subpath: "plugin",
},
"invalid-v2-extra-file": {
rootid: "test-datasource",
subpath: "plugin",
},
"invalid-v2-missing-file": {
rootid: "test-datasource",
subpath: "plugin",
},
"lacking-files": {
rootid: "test-datasource",
subpath: "plugin",
},
"nested-plugins": {
rootid: "test-datasource",
subpath: "parent",
},
"non-pvt-with-root-url": {
rootid: "test-datasource",
subpath: "plugin",
},
"renderer-added-file": {
rootid: "test-renderer",
subpath: "plugin",
},
plugins: New static scanner and validator, with Thema slot support (#53754) * coremodels: Convert plugin-metadata schema to a coremodel * Newer cuetsy; try quoting field name * Add slot definitions * Start sketching out pfs package * Rerun codegen with fixes, new cuetsy * Catch up dashboard with new cuetsy * Update to go1.18 * Use new vmuxers in thema * Add slot system in Go * Draft finished implementation of pfs * Collapse slot pkg into coremodel dir; add PluginInfo * Add the mux type on top of kernel * Refactor plugin generator for extensibility * Change models.cue package, numerous debugs * Bring new output to parity with old * Remove old plugin generation logic * Misc tweaking * Reintroduce generation of shared schemas * Drop back to go1.17 * Add globbing to tsconfig exclude * Introduce pfs test on existing testdata * Make most existing testdata tests pass with pfs * coremodels: Convert plugin-metadata schema to a coremodel * Newer cuetsy; try quoting field name * Add APIType control concept, regen pluginmeta * Use proper numeric types for schema fields * Make pluginmeta schema follow Go type breakdown * More decomposition into distinct types * Add test case for no plugin.json file * Fix missing ref to #Dependencies * Remove generated TS for pluginmeta * Update dependencies, rearrange go.mod * Regenerate without Model prefix * Use updated thema loader; this is now runnable * Skip app plugin with weird include * Make plugin tree extractor reusable * Split out slot lineage load/validate logic * Add myriad tests for new plugin validation failures * Add test for zip fixtures * One last run of codegen * Proper delinting * Ensure validation order is deterministic * Let there actually be sorting * Undo reliance on builtIn field (#54009) * undo builtIn reliance * fix tests Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
2022-08-22 11:11:45 -05:00
"symbolic-plugin-dirs": {
skip: "io/fs-based scanner will not traverse symlinks; caller of ParsePluginFS() must do it",
},
"test-app": {
skip: "schema violation, dependencies don't follow naming constraints",
rootid: "test-app",
},
"test-app-with-includes": {
rootid: "test-app",
skip: "has a 'page'-type include which isn't a known part of spec",
},
RBAC: Allow role registration for plugins (#57387) * Picking role registration from OnCall POC branch * Fix test * Remove include actions from this PR * Removing unused permission * Adding test to DeclarePluginRoles * Add testcase to RegisterFixed role * Additional test case * Adding tests to validate plugins roles * Add test to plugin loader * Nit. * Scuemata validation * Changing the design to decouple accesscontrol from plugin management Co-authored-by: Kalle Persson <kalle.persson@grafana.com> * Fixing tests Co-authored-by: Jguer <joao.guerreiro@grafana.com> * Add missing files Co-authored-by: Jguer <joao.guerreiro@grafana.com> * Remove feature toggle check from loader * Remove feature toggleimport * Feedback Co-Authored-By: marefr <marcus.efraimsson@gmail.com> * Fix test' * Make plugins.RoleRegistry interface typed * Remove comment question * No need for json tags anymore * Nit. log * Adding the schema validation * Remove group to take plugin Name instead * Revert sqlstore -> db * Nit. * Nit. on tests Co-authored-by: ievaVasiljeva <ieva.vasiljeva@grafana.com> * Update pkg/services/accesscontrol/plugins.go Co-authored-by: Ieva <ieva.vasiljeva@grafana.com> * Log message Co-Authored-By: marefr <marcus.efraimsson@gmail.com> * Log message Co-Authored-By: marefr <marcus.efraimsson@gmail.com> * Remove unecessary method. Update test name. Co-authored-by: ievaVasiljeva <ieva.vasiljeva@grafana.com> * Fix linting * Update cue descriptions * Fix test Co-authored-by: Kalle Persson <kalle.persson@grafana.com> Co-authored-by: Jguer <joao.guerreiro@grafana.com> Co-authored-by: marefr <marcus.efraimsson@gmail.com> Co-authored-by: ievaVasiljeva <ieva.vasiljeva@grafana.com>
2022-11-07 04:30:45 -06:00
"test-app-with-roles": {
rootid: "test-app",
},
plugins: New static scanner and validator, with Thema slot support (#53754) * coremodels: Convert plugin-metadata schema to a coremodel * Newer cuetsy; try quoting field name * Add slot definitions * Start sketching out pfs package * Rerun codegen with fixes, new cuetsy * Catch up dashboard with new cuetsy * Update to go1.18 * Use new vmuxers in thema * Add slot system in Go * Draft finished implementation of pfs * Collapse slot pkg into coremodel dir; add PluginInfo * Add the mux type on top of kernel * Refactor plugin generator for extensibility * Change models.cue package, numerous debugs * Bring new output to parity with old * Remove old plugin generation logic * Misc tweaking * Reintroduce generation of shared schemas * Drop back to go1.17 * Add globbing to tsconfig exclude * Introduce pfs test on existing testdata * Make most existing testdata tests pass with pfs * coremodels: Convert plugin-metadata schema to a coremodel * Newer cuetsy; try quoting field name * Add APIType control concept, regen pluginmeta * Use proper numeric types for schema fields * Make pluginmeta schema follow Go type breakdown * More decomposition into distinct types * Add test case for no plugin.json file * Fix missing ref to #Dependencies * Remove generated TS for pluginmeta * Update dependencies, rearrange go.mod * Regenerate without Model prefix * Use updated thema loader; this is now runnable * Skip app plugin with weird include * Make plugin tree extractor reusable * Split out slot lineage load/validate logic * Add myriad tests for new plugin validation failures * Add test for zip fixtures * One last run of codegen * Proper delinting * Ensure validation order is deterministic * Let there actually be sorting * Undo reliance on builtIn field (#54009) * undo builtIn reliance * fix tests Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
2022-08-22 11:11:45 -05:00
"unsigned-datasource": {
rootid: "test-datasource",
subpath: "plugin",
},
"unsigned-panel": {
rootid: "test-panel",
subpath: "plugin",
},
"valid-v2-pvt-signature": {
rootid: "test-datasource",
subpath: "plugin",
},
"valid-v2-pvt-signature-root-url-uri": {
rootid: "test-datasource",
subpath: "plugin",
},
"valid-v2-signature": {
rootid: "test-datasource",
subpath: "plugin",
},
"plugin-with-dist": {
rootid: "test-datasource",
subpath: "plugin",
},
plugins: New static scanner and validator, with Thema slot support (#53754) * coremodels: Convert plugin-metadata schema to a coremodel * Newer cuetsy; try quoting field name * Add slot definitions * Start sketching out pfs package * Rerun codegen with fixes, new cuetsy * Catch up dashboard with new cuetsy * Update to go1.18 * Use new vmuxers in thema * Add slot system in Go * Draft finished implementation of pfs * Collapse slot pkg into coremodel dir; add PluginInfo * Add the mux type on top of kernel * Refactor plugin generator for extensibility * Change models.cue package, numerous debugs * Bring new output to parity with old * Remove old plugin generation logic * Misc tweaking * Reintroduce generation of shared schemas * Drop back to go1.17 * Add globbing to tsconfig exclude * Introduce pfs test on existing testdata * Make most existing testdata tests pass with pfs * coremodels: Convert plugin-metadata schema to a coremodel * Newer cuetsy; try quoting field name * Add APIType control concept, regen pluginmeta * Use proper numeric types for schema fields * Make pluginmeta schema follow Go type breakdown * More decomposition into distinct types * Add test case for no plugin.json file * Fix missing ref to #Dependencies * Remove generated TS for pluginmeta * Update dependencies, rearrange go.mod * Regenerate without Model prefix * Use updated thema loader; this is now runnable * Skip app plugin with weird include * Make plugin tree extractor reusable * Split out slot lineage load/validate logic * Add myriad tests for new plugin validation failures * Add test for zip fixtures * One last run of codegen * Proper delinting * Ensure validation order is deterministic * Let there actually be sorting * Undo reliance on builtIn field (#54009) * undo builtIn reliance * fix tests Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
2022-08-22 11:11:45 -05:00
"no-rootfile": {
err: ErrNoRootFile,
},
"valid-model-panel": {},
"valid-model-datasource": {},
"missing-kind-datasource": {},
plugins: New static scanner and validator, with Thema slot support (#53754) * coremodels: Convert plugin-metadata schema to a coremodel * Newer cuetsy; try quoting field name * Add slot definitions * Start sketching out pfs package * Rerun codegen with fixes, new cuetsy * Catch up dashboard with new cuetsy * Update to go1.18 * Use new vmuxers in thema * Add slot system in Go * Draft finished implementation of pfs * Collapse slot pkg into coremodel dir; add PluginInfo * Add the mux type on top of kernel * Refactor plugin generator for extensibility * Change models.cue package, numerous debugs * Bring new output to parity with old * Remove old plugin generation logic * Misc tweaking * Reintroduce generation of shared schemas * Drop back to go1.17 * Add globbing to tsconfig exclude * Introduce pfs test on existing testdata * Make most existing testdata tests pass with pfs * coremodels: Convert plugin-metadata schema to a coremodel * Newer cuetsy; try quoting field name * Add APIType control concept, regen pluginmeta * Use proper numeric types for schema fields * Make pluginmeta schema follow Go type breakdown * More decomposition into distinct types * Add test case for no plugin.json file * Fix missing ref to #Dependencies * Remove generated TS for pluginmeta * Update dependencies, rearrange go.mod * Regenerate without Model prefix * Use updated thema loader; this is now runnable * Skip app plugin with weird include * Make plugin tree extractor reusable * Split out slot lineage load/validate logic * Add myriad tests for new plugin validation failures * Add test for zip fixtures * One last run of codegen * Proper delinting * Ensure validation order is deterministic * Let there actually be sorting * Undo reliance on builtIn field (#54009) * undo builtIn reliance * fix tests Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
2022-08-22 11:11:45 -05:00
"panel-conflicting-joinschema": {
err: ErrInvalidLineage,
skip: "TODO implement BindOption in thema, SatisfiesJoinSchema, then use it here",
},
"panel-does-not-follow-slot-joinschema": {
err: ErrInvalidLineage,
skip: "TODO implement BindOption in thema, SatisfiesJoinSchema, then use it here",
},
"pluginRootWithDist": {
err: ErrNoRootFile,
skip: "This folder is used to test multiple plugins in the same folder",
},
"name-mismatch-panel": {
err: ErrInvalidGrafanaPluginInstance,
plugins: New static scanner and validator, with Thema slot support (#53754) * coremodels: Convert plugin-metadata schema to a coremodel * Newer cuetsy; try quoting field name * Add slot definitions * Start sketching out pfs package * Rerun codegen with fixes, new cuetsy * Catch up dashboard with new cuetsy * Update to go1.18 * Use new vmuxers in thema * Add slot system in Go * Draft finished implementation of pfs * Collapse slot pkg into coremodel dir; add PluginInfo * Add the mux type on top of kernel * Refactor plugin generator for extensibility * Change models.cue package, numerous debugs * Bring new output to parity with old * Remove old plugin generation logic * Misc tweaking * Reintroduce generation of shared schemas * Drop back to go1.17 * Add globbing to tsconfig exclude * Introduce pfs test on existing testdata * Make most existing testdata tests pass with pfs * coremodels: Convert plugin-metadata schema to a coremodel * Newer cuetsy; try quoting field name * Add APIType control concept, regen pluginmeta * Use proper numeric types for schema fields * Make pluginmeta schema follow Go type breakdown * More decomposition into distinct types * Add test case for no plugin.json file * Fix missing ref to #Dependencies * Remove generated TS for pluginmeta * Update dependencies, rearrange go.mod * Regenerate without Model prefix * Use updated thema loader; this is now runnable * Skip app plugin with weird include * Make plugin tree extractor reusable * Split out slot lineage load/validate logic * Add myriad tests for new plugin validation failures * Add test for zip fixtures * One last run of codegen * Proper delinting * Ensure validation order is deterministic * Let there actually be sorting * Undo reliance on builtIn field (#54009) * undo builtIn reliance * fix tests Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
2022-08-22 11:11:45 -05:00
},
"disallowed-cue-import": {
err: ErrDisallowedCUEImport,
},
Plugins: Allow loading panel plugins from a CDN (#59096) * POC: Plugins CDN reverse proxy * CDN proxy POC: changed env var names * Add authorization: false for /public path in frontend plugin loader * Moved CDN settings to Cfg, add some comments * Fix error 500 in asset fetch if plugin is not using CDN * Fix EnterpriseLicensePath declared twice * Fix linter complaining about whitespaces * Plugins CDN: Skip signature verification for CDN plugins * Plugins CDN: Skip manifest and signature check for cdn plugins * Plugins: use IsValid() and IsInternal() rather than equality checks * Plugins CDN: remove comment * Plugins CDN: Fix seeker can't seek when serving plugins from local fs * Plugins CDN: add back error codes in getLocalPluginAssets * Plugins CDN: call asset.Close() rather than asset.readSeekCloser.Close() * Plugins CDN: Fix panic in JsonApiErr when errorMessageCoder wraps a nil error * Plugins CDN: Add error handling to proxyCDNPluginAsset * Plugins CDN: replace errorMessageCoder with errutil * Plugins CDN POC: expose cdn plugin paths to frontend for system.js * Plugins CDN: Fix cdn plugins showing as unsigned in frontend * WIP: Add support for formatted URL * Fix missing cdnPluginsBaseURLs in GrafanaConfig * Plugins CDN: Remove reverse proxy mode and reverse proxy references * Plugins CDN: Simplify asset serving logic * Plugins CDN: sanitize redirect path * Plugins CDN: Removed unused pluginAsset type * Plugins CDN: Removed system.js changes * Plugins CDN: Return different system.js baseURL and module for cdn plugins * Plugins CDN: Ensure CDN is disabled for non-external plugins * lint * Plugins CDN: serve images and screenshots from CDN, refactoring * Lint * Plugins CDN: Fix URLs for system.js (baseUrl and module) * Plugins CDN: Add more tests for RelativeURLForSystemJS * Plugins CDN: Iterate only on apps when preloading * Plugins CDN: Refactoring * Plugins CDN: Add comments to url_constructor.go * Plugins CDN: Update defaultHGPluginsCDNBaseURL * Plugins CDN: undo extract meta from system js config * refactor(plugins): migrate systemjs css plugin to typescript * feat(plugins): introduce systemjs cdn loader plugin * feat(plugins): add systemjs load type * Plugins CDN: Removed RelativeURLForSystemJS * Plugins CDN: Log backend redirect hits along with plugin info * Plugins CDN: Add pluginsCDNBasePath to getFrontendSettingsMap * feat(plugins): introduce cdn loading for angular plugins * refactor(plugins): move systemjs cache buster into systemjsplugins directory * Plugins CDN: Rename pluginsCDNBasePath to pluginsCDNBaseURL * refactor(plugins): introduce pluginsCDNBaseURL to the frontend * Plugins CDN: Renamed "cdn base path" to "cdn url template" in backend * Plugins CDN: lint * merge with main * Instrumentation: Add prometheus counter for backend hits, log from Info to Warn * Config: Changed key from plugins_cdn.url to plugins.plugins_cdn_base_url * CDN: Add backend tests * Lint: goimports * Default CDN URL to empty string, * Do not use CDN in setImages and module if the url template is empty * CDN: Backend: Add test for frontend settings * CDN: Do not log missing module.js warn if plugin is being loaded from CDN * CDN: Add backend test for CDN plugin loader * Removed 'cdn' signature level, switch to 'valid' * Fix pfs.TestParseTreeTestdata for cdn plugin testdata dir * Fix TestLoader_Load * Fix gocyclo complexity of loadPlugins * Plugins CDN: Moved prometheus metric to api package, removed asset_path label * Fix missing in config * Changes after review * Add pluginscdn.Service * Fix tests * Refactoring * Moved all remaining CDN checks inside pluginscdn.Service * CDN url constructor: Renamed stringURLFor to stringPath * CDN: Moved asset URL functionality to assetpath service * CDN: Renamed HasCDN() to IsEnabled() * CDN: Replace assert with require * CDN: Changes after review * Assetpath: Handle url.Parse error * Fix plugin_resource_test * CDN: Change fallback redirect from 302 to 307 * goimports * Fix tests * Switch to contextmodel.ReqContext in plugins.go Co-authored-by: Will Browne <will.browne@grafana.com> Co-authored-by: Jack Westbrook <jack.westbrook@gmail.com>
2023-01-27 08:08:17 -06:00
"cdn": {
rootid: "grafana-worldmap-panel",
subpath: "plugin",
},
"external-registration": {
rootid: "grafana-test-datasource",
},
plugins: New static scanner and validator, with Thema slot support (#53754) * coremodels: Convert plugin-metadata schema to a coremodel * Newer cuetsy; try quoting field name * Add slot definitions * Start sketching out pfs package * Rerun codegen with fixes, new cuetsy * Catch up dashboard with new cuetsy * Update to go1.18 * Use new vmuxers in thema * Add slot system in Go * Draft finished implementation of pfs * Collapse slot pkg into coremodel dir; add PluginInfo * Add the mux type on top of kernel * Refactor plugin generator for extensibility * Change models.cue package, numerous debugs * Bring new output to parity with old * Remove old plugin generation logic * Misc tweaking * Reintroduce generation of shared schemas * Drop back to go1.17 * Add globbing to tsconfig exclude * Introduce pfs test on existing testdata * Make most existing testdata tests pass with pfs * coremodels: Convert plugin-metadata schema to a coremodel * Newer cuetsy; try quoting field name * Add APIType control concept, regen pluginmeta * Use proper numeric types for schema fields * Make pluginmeta schema follow Go type breakdown * More decomposition into distinct types * Add test case for no plugin.json file * Fix missing ref to #Dependencies * Remove generated TS for pluginmeta * Update dependencies, rearrange go.mod * Regenerate without Model prefix * Use updated thema loader; this is now runnable * Skip app plugin with weird include * Make plugin tree extractor reusable * Split out slot lineage load/validate logic * Add myriad tests for new plugin validation failures * Add test for zip fixtures * One last run of codegen * Proper delinting * Ensure validation order is deterministic * Let there actually be sorting * Undo reliance on builtIn field (#54009) * undo builtIn reliance * fix tests Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
2022-08-22 11:11:45 -05:00
}
staticRootPath, err := filepath.Abs(filepath.Join("..", "manager", "testdata"))
plugins: New static scanner and validator, with Thema slot support (#53754) * coremodels: Convert plugin-metadata schema to a coremodel * Newer cuetsy; try quoting field name * Add slot definitions * Start sketching out pfs package * Rerun codegen with fixes, new cuetsy * Catch up dashboard with new cuetsy * Update to go1.18 * Use new vmuxers in thema * Add slot system in Go * Draft finished implementation of pfs * Collapse slot pkg into coremodel dir; add PluginInfo * Add the mux type on top of kernel * Refactor plugin generator for extensibility * Change models.cue package, numerous debugs * Bring new output to parity with old * Remove old plugin generation logic * Misc tweaking * Reintroduce generation of shared schemas * Drop back to go1.17 * Add globbing to tsconfig exclude * Introduce pfs test on existing testdata * Make most existing testdata tests pass with pfs * coremodels: Convert plugin-metadata schema to a coremodel * Newer cuetsy; try quoting field name * Add APIType control concept, regen pluginmeta * Use proper numeric types for schema fields * Make pluginmeta schema follow Go type breakdown * More decomposition into distinct types * Add test case for no plugin.json file * Fix missing ref to #Dependencies * Remove generated TS for pluginmeta * Update dependencies, rearrange go.mod * Regenerate without Model prefix * Use updated thema loader; this is now runnable * Skip app plugin with weird include * Make plugin tree extractor reusable * Split out slot lineage load/validate logic * Add myriad tests for new plugin validation failures * Add test for zip fixtures * One last run of codegen * Proper delinting * Ensure validation order is deterministic * Let there actually be sorting * Undo reliance on builtIn field (#54009) * undo builtIn reliance * fix tests Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
2022-08-22 11:11:45 -05:00
require.NoError(t, err)
dfs := os.DirFS(staticRootPath)
ents, err := fs.ReadDir(dfs, ".")
require.NoError(t, err)
// Ensure table test and dir list are ==
var dirs, tts []string
for k := range tab {
tts = append(tts, k)
}
for _, ent := range ents {
dirs = append(dirs, ent.Name())
}
sort.Strings(tts)
sort.Strings(dirs)
if !cmp.Equal(tts, dirs) {
t.Fatalf("table test map (-) and pkg/plugins/manager/testdata dirs (+) differ: %s", cmp.Diff(tts, dirs))
}
for _, ent := range ents {
tst := tab[ent.Name()]
tst.tfs, err = fs.Sub(dfs, filepath.Join(ent.Name(), tst.subpath))
require.NoError(t, err)
tab[ent.Name()] = tst
}
lib := cuectx.GrafanaThemaRuntime()
plugins: New static scanner and validator, with Thema slot support (#53754) * coremodels: Convert plugin-metadata schema to a coremodel * Newer cuetsy; try quoting field name * Add slot definitions * Start sketching out pfs package * Rerun codegen with fixes, new cuetsy * Catch up dashboard with new cuetsy * Update to go1.18 * Use new vmuxers in thema * Add slot system in Go * Draft finished implementation of pfs * Collapse slot pkg into coremodel dir; add PluginInfo * Add the mux type on top of kernel * Refactor plugin generator for extensibility * Change models.cue package, numerous debugs * Bring new output to parity with old * Remove old plugin generation logic * Misc tweaking * Reintroduce generation of shared schemas * Drop back to go1.17 * Add globbing to tsconfig exclude * Introduce pfs test on existing testdata * Make most existing testdata tests pass with pfs * coremodels: Convert plugin-metadata schema to a coremodel * Newer cuetsy; try quoting field name * Add APIType control concept, regen pluginmeta * Use proper numeric types for schema fields * Make pluginmeta schema follow Go type breakdown * More decomposition into distinct types * Add test case for no plugin.json file * Fix missing ref to #Dependencies * Remove generated TS for pluginmeta * Update dependencies, rearrange go.mod * Regenerate without Model prefix * Use updated thema loader; this is now runnable * Skip app plugin with weird include * Make plugin tree extractor reusable * Split out slot lineage load/validate logic * Add myriad tests for new plugin validation failures * Add test for zip fixtures * One last run of codegen * Proper delinting * Ensure validation order is deterministic * Let there actually be sorting * Undo reliance on builtIn field (#54009) * undo builtIn reliance * fix tests Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
2022-08-22 11:11:45 -05:00
for name, otst := range tab {
tst := otst // otherwise var is shadowed within func by looping
t.Run(name, func(t *testing.T) {
if tst.skip != "" {
t.Skip(tst.skip)
}
pp, err := ParsePluginFS(tst.tfs, lib)
plugins: New static scanner and validator, with Thema slot support (#53754) * coremodels: Convert plugin-metadata schema to a coremodel * Newer cuetsy; try quoting field name * Add slot definitions * Start sketching out pfs package * Rerun codegen with fixes, new cuetsy * Catch up dashboard with new cuetsy * Update to go1.18 * Use new vmuxers in thema * Add slot system in Go * Draft finished implementation of pfs * Collapse slot pkg into coremodel dir; add PluginInfo * Add the mux type on top of kernel * Refactor plugin generator for extensibility * Change models.cue package, numerous debugs * Bring new output to parity with old * Remove old plugin generation logic * Misc tweaking * Reintroduce generation of shared schemas * Drop back to go1.17 * Add globbing to tsconfig exclude * Introduce pfs test on existing testdata * Make most existing testdata tests pass with pfs * coremodels: Convert plugin-metadata schema to a coremodel * Newer cuetsy; try quoting field name * Add APIType control concept, regen pluginmeta * Use proper numeric types for schema fields * Make pluginmeta schema follow Go type breakdown * More decomposition into distinct types * Add test case for no plugin.json file * Fix missing ref to #Dependencies * Remove generated TS for pluginmeta * Update dependencies, rearrange go.mod * Regenerate without Model prefix * Use updated thema loader; this is now runnable * Skip app plugin with weird include * Make plugin tree extractor reusable * Split out slot lineage load/validate logic * Add myriad tests for new plugin validation failures * Add test for zip fixtures * One last run of codegen * Proper delinting * Ensure validation order is deterministic * Let there actually be sorting * Undo reliance on builtIn field (#54009) * undo builtIn reliance * fix tests Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
2022-08-22 11:11:45 -05:00
if tst.err == nil {
require.NoError(t, err, "unexpected error while parsing plugin tree")
} else {
require.Error(t, err)
t.Logf("%T %s", err, err)
plugins: New static scanner and validator, with Thema slot support (#53754) * coremodels: Convert plugin-metadata schema to a coremodel * Newer cuetsy; try quoting field name * Add slot definitions * Start sketching out pfs package * Rerun codegen with fixes, new cuetsy * Catch up dashboard with new cuetsy * Update to go1.18 * Use new vmuxers in thema * Add slot system in Go * Draft finished implementation of pfs * Collapse slot pkg into coremodel dir; add PluginInfo * Add the mux type on top of kernel * Refactor plugin generator for extensibility * Change models.cue package, numerous debugs * Bring new output to parity with old * Remove old plugin generation logic * Misc tweaking * Reintroduce generation of shared schemas * Drop back to go1.17 * Add globbing to tsconfig exclude * Introduce pfs test on existing testdata * Make most existing testdata tests pass with pfs * coremodels: Convert plugin-metadata schema to a coremodel * Newer cuetsy; try quoting field name * Add APIType control concept, regen pluginmeta * Use proper numeric types for schema fields * Make pluginmeta schema follow Go type breakdown * More decomposition into distinct types * Add test case for no plugin.json file * Fix missing ref to #Dependencies * Remove generated TS for pluginmeta * Update dependencies, rearrange go.mod * Regenerate without Model prefix * Use updated thema loader; this is now runnable * Skip app plugin with weird include * Make plugin tree extractor reusable * Split out slot lineage load/validate logic * Add myriad tests for new plugin validation failures * Add test for zip fixtures * One last run of codegen * Proper delinting * Ensure validation order is deterministic * Let there actually be sorting * Undo reliance on builtIn field (#54009) * undo builtIn reliance * fix tests Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
2022-08-22 11:11:45 -05:00
require.ErrorIs(t, err, tst.err, "unexpected error type while parsing plugin tree")
return
}
if tst.rootid == "" {
tst.rootid = name
}
require.Equal(t, tst.rootid, pp.Properties.Id, "expected plugin id and actual plugin id differ")
plugins: New static scanner and validator, with Thema slot support (#53754) * coremodels: Convert plugin-metadata schema to a coremodel * Newer cuetsy; try quoting field name * Add slot definitions * Start sketching out pfs package * Rerun codegen with fixes, new cuetsy * Catch up dashboard with new cuetsy * Update to go1.18 * Use new vmuxers in thema * Add slot system in Go * Draft finished implementation of pfs * Collapse slot pkg into coremodel dir; add PluginInfo * Add the mux type on top of kernel * Refactor plugin generator for extensibility * Change models.cue package, numerous debugs * Bring new output to parity with old * Remove old plugin generation logic * Misc tweaking * Reintroduce generation of shared schemas * Drop back to go1.17 * Add globbing to tsconfig exclude * Introduce pfs test on existing testdata * Make most existing testdata tests pass with pfs * coremodels: Convert plugin-metadata schema to a coremodel * Newer cuetsy; try quoting field name * Add APIType control concept, regen pluginmeta * Use proper numeric types for schema fields * Make pluginmeta schema follow Go type breakdown * More decomposition into distinct types * Add test case for no plugin.json file * Fix missing ref to #Dependencies * Remove generated TS for pluginmeta * Update dependencies, rearrange go.mod * Regenerate without Model prefix * Use updated thema loader; this is now runnable * Skip app plugin with weird include * Make plugin tree extractor reusable * Split out slot lineage load/validate logic * Add myriad tests for new plugin validation failures * Add test for zip fixtures * One last run of codegen * Proper delinting * Ensure validation order is deterministic * Let there actually be sorting * Undo reliance on builtIn field (#54009) * undo builtIn reliance * fix tests Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
2022-08-22 11:11:45 -05:00
})
}
}
func TestParseTreeZips(t *testing.T) {
type tt struct {
tfs fs.FS
// TODO could remove this by getting rid of inconsistent subdirs
subpath string
skip string
err error
// TODO could remove this by expecting that dirname == id
rootid string
}
tab := map[string]tt{
"grafana-simple-json-datasource-ec18fa4da8096a952608a7e4c7782b4260b41bcf.zip": {
skip: "binary plugin",
},
"plugin-with-absolute-member.zip": {
skip: "not actually a plugin, no plugin.json?",
},
"plugin-with-absolute-symlink-dir.zip": {
skip: "not actually a plugin, no plugin.json?",
},
"plugin-with-absolute-symlink.zip": {
skip: "not actually a plugin, no plugin.json?",
},
"plugin-with-parent-member.zip": {
skip: "not actually a plugin, no plugin.json?",
},
"plugin-with-symlink-dir.zip": {
skip: "not actually a plugin, no plugin.json?",
},
"plugin-with-symlink.zip": {
skip: "not actually a plugin, no plugin.json?",
},
"plugin-with-symlinks.zip": {
subpath: "test-app",
rootid: "test-app",
},
}
staticRootPath, err := filepath.Abs(filepath.Join("..", "storage", "testdata"))
plugins: New static scanner and validator, with Thema slot support (#53754) * coremodels: Convert plugin-metadata schema to a coremodel * Newer cuetsy; try quoting field name * Add slot definitions * Start sketching out pfs package * Rerun codegen with fixes, new cuetsy * Catch up dashboard with new cuetsy * Update to go1.18 * Use new vmuxers in thema * Add slot system in Go * Draft finished implementation of pfs * Collapse slot pkg into coremodel dir; add PluginInfo * Add the mux type on top of kernel * Refactor plugin generator for extensibility * Change models.cue package, numerous debugs * Bring new output to parity with old * Remove old plugin generation logic * Misc tweaking * Reintroduce generation of shared schemas * Drop back to go1.17 * Add globbing to tsconfig exclude * Introduce pfs test on existing testdata * Make most existing testdata tests pass with pfs * coremodels: Convert plugin-metadata schema to a coremodel * Newer cuetsy; try quoting field name * Add APIType control concept, regen pluginmeta * Use proper numeric types for schema fields * Make pluginmeta schema follow Go type breakdown * More decomposition into distinct types * Add test case for no plugin.json file * Fix missing ref to #Dependencies * Remove generated TS for pluginmeta * Update dependencies, rearrange go.mod * Regenerate without Model prefix * Use updated thema loader; this is now runnable * Skip app plugin with weird include * Make plugin tree extractor reusable * Split out slot lineage load/validate logic * Add myriad tests for new plugin validation failures * Add test for zip fixtures * One last run of codegen * Proper delinting * Ensure validation order is deterministic * Let there actually be sorting * Undo reliance on builtIn field (#54009) * undo builtIn reliance * fix tests Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
2022-08-22 11:11:45 -05:00
require.NoError(t, err)
ents, err := os.ReadDir(staticRootPath)
require.NoError(t, err)
// Ensure table test and dir list are ==
var dirs, tts []string
for k := range tab {
tts = append(tts, k)
}
for _, ent := range ents {
dirs = append(dirs, ent.Name())
}
sort.Strings(tts)
sort.Strings(dirs)
if !cmp.Equal(tts, dirs) {
t.Fatalf("table test map (-) and pkg/plugins/installer/testdata dirs (+) differ: %s", cmp.Diff(tts, dirs))
}
for _, ent := range ents {
tst := tab[ent.Name()]
r, err := zip.OpenReader(filepath.Join(staticRootPath, ent.Name()))
require.NoError(t, err)
defer r.Close() //nolint:errcheck
if tst.subpath != "" {
tst.tfs, err = fs.Sub(r, tst.subpath)
require.NoError(t, err)
} else {
tst.tfs = r
}
tab[ent.Name()] = tst
}
lib := cuectx.GrafanaThemaRuntime()
plugins: New static scanner and validator, with Thema slot support (#53754) * coremodels: Convert plugin-metadata schema to a coremodel * Newer cuetsy; try quoting field name * Add slot definitions * Start sketching out pfs package * Rerun codegen with fixes, new cuetsy * Catch up dashboard with new cuetsy * Update to go1.18 * Use new vmuxers in thema * Add slot system in Go * Draft finished implementation of pfs * Collapse slot pkg into coremodel dir; add PluginInfo * Add the mux type on top of kernel * Refactor plugin generator for extensibility * Change models.cue package, numerous debugs * Bring new output to parity with old * Remove old plugin generation logic * Misc tweaking * Reintroduce generation of shared schemas * Drop back to go1.17 * Add globbing to tsconfig exclude * Introduce pfs test on existing testdata * Make most existing testdata tests pass with pfs * coremodels: Convert plugin-metadata schema to a coremodel * Newer cuetsy; try quoting field name * Add APIType control concept, regen pluginmeta * Use proper numeric types for schema fields * Make pluginmeta schema follow Go type breakdown * More decomposition into distinct types * Add test case for no plugin.json file * Fix missing ref to #Dependencies * Remove generated TS for pluginmeta * Update dependencies, rearrange go.mod * Regenerate without Model prefix * Use updated thema loader; this is now runnable * Skip app plugin with weird include * Make plugin tree extractor reusable * Split out slot lineage load/validate logic * Add myriad tests for new plugin validation failures * Add test for zip fixtures * One last run of codegen * Proper delinting * Ensure validation order is deterministic * Let there actually be sorting * Undo reliance on builtIn field (#54009) * undo builtIn reliance * fix tests Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
2022-08-22 11:11:45 -05:00
for name, otst := range tab {
tst := otst // otherwise var is shadowed within func by looping
t.Run(name, func(t *testing.T) {
if tst.skip != "" {
t.Skip(tst.skip)
}
pp, err := ParsePluginFS(tst.tfs, lib)
plugins: New static scanner and validator, with Thema slot support (#53754) * coremodels: Convert plugin-metadata schema to a coremodel * Newer cuetsy; try quoting field name * Add slot definitions * Start sketching out pfs package * Rerun codegen with fixes, new cuetsy * Catch up dashboard with new cuetsy * Update to go1.18 * Use new vmuxers in thema * Add slot system in Go * Draft finished implementation of pfs * Collapse slot pkg into coremodel dir; add PluginInfo * Add the mux type on top of kernel * Refactor plugin generator for extensibility * Change models.cue package, numerous debugs * Bring new output to parity with old * Remove old plugin generation logic * Misc tweaking * Reintroduce generation of shared schemas * Drop back to go1.17 * Add globbing to tsconfig exclude * Introduce pfs test on existing testdata * Make most existing testdata tests pass with pfs * coremodels: Convert plugin-metadata schema to a coremodel * Newer cuetsy; try quoting field name * Add APIType control concept, regen pluginmeta * Use proper numeric types for schema fields * Make pluginmeta schema follow Go type breakdown * More decomposition into distinct types * Add test case for no plugin.json file * Fix missing ref to #Dependencies * Remove generated TS for pluginmeta * Update dependencies, rearrange go.mod * Regenerate without Model prefix * Use updated thema loader; this is now runnable * Skip app plugin with weird include * Make plugin tree extractor reusable * Split out slot lineage load/validate logic * Add myriad tests for new plugin validation failures * Add test for zip fixtures * One last run of codegen * Proper delinting * Ensure validation order is deterministic * Let there actually be sorting * Undo reliance on builtIn field (#54009) * undo builtIn reliance * fix tests Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
2022-08-22 11:11:45 -05:00
if tst.err == nil {
require.NoError(t, err, "unexpected error while parsing plugin fs")
plugins: New static scanner and validator, with Thema slot support (#53754) * coremodels: Convert plugin-metadata schema to a coremodel * Newer cuetsy; try quoting field name * Add slot definitions * Start sketching out pfs package * Rerun codegen with fixes, new cuetsy * Catch up dashboard with new cuetsy * Update to go1.18 * Use new vmuxers in thema * Add slot system in Go * Draft finished implementation of pfs * Collapse slot pkg into coremodel dir; add PluginInfo * Add the mux type on top of kernel * Refactor plugin generator for extensibility * Change models.cue package, numerous debugs * Bring new output to parity with old * Remove old plugin generation logic * Misc tweaking * Reintroduce generation of shared schemas * Drop back to go1.17 * Add globbing to tsconfig exclude * Introduce pfs test on existing testdata * Make most existing testdata tests pass with pfs * coremodels: Convert plugin-metadata schema to a coremodel * Newer cuetsy; try quoting field name * Add APIType control concept, regen pluginmeta * Use proper numeric types for schema fields * Make pluginmeta schema follow Go type breakdown * More decomposition into distinct types * Add test case for no plugin.json file * Fix missing ref to #Dependencies * Remove generated TS for pluginmeta * Update dependencies, rearrange go.mod * Regenerate without Model prefix * Use updated thema loader; this is now runnable * Skip app plugin with weird include * Make plugin tree extractor reusable * Split out slot lineage load/validate logic * Add myriad tests for new plugin validation failures * Add test for zip fixtures * One last run of codegen * Proper delinting * Ensure validation order is deterministic * Let there actually be sorting * Undo reliance on builtIn field (#54009) * undo builtIn reliance * fix tests Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
2022-08-22 11:11:45 -05:00
} else {
require.ErrorIs(t, err, tst.err, "unexpected error type while parsing plugin fs")
plugins: New static scanner and validator, with Thema slot support (#53754) * coremodels: Convert plugin-metadata schema to a coremodel * Newer cuetsy; try quoting field name * Add slot definitions * Start sketching out pfs package * Rerun codegen with fixes, new cuetsy * Catch up dashboard with new cuetsy * Update to go1.18 * Use new vmuxers in thema * Add slot system in Go * Draft finished implementation of pfs * Collapse slot pkg into coremodel dir; add PluginInfo * Add the mux type on top of kernel * Refactor plugin generator for extensibility * Change models.cue package, numerous debugs * Bring new output to parity with old * Remove old plugin generation logic * Misc tweaking * Reintroduce generation of shared schemas * Drop back to go1.17 * Add globbing to tsconfig exclude * Introduce pfs test on existing testdata * Make most existing testdata tests pass with pfs * coremodels: Convert plugin-metadata schema to a coremodel * Newer cuetsy; try quoting field name * Add APIType control concept, regen pluginmeta * Use proper numeric types for schema fields * Make pluginmeta schema follow Go type breakdown * More decomposition into distinct types * Add test case for no plugin.json file * Fix missing ref to #Dependencies * Remove generated TS for pluginmeta * Update dependencies, rearrange go.mod * Regenerate without Model prefix * Use updated thema loader; this is now runnable * Skip app plugin with weird include * Make plugin tree extractor reusable * Split out slot lineage load/validate logic * Add myriad tests for new plugin validation failures * Add test for zip fixtures * One last run of codegen * Proper delinting * Ensure validation order is deterministic * Let there actually be sorting * Undo reliance on builtIn field (#54009) * undo builtIn reliance * fix tests Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
2022-08-22 11:11:45 -05:00
return
}
if tst.rootid == "" {
tst.rootid = name
}
require.Equal(t, tst.rootid, pp.Properties.Id, "expected plugin id and actual plugin id differ")
plugins: New static scanner and validator, with Thema slot support (#53754) * coremodels: Convert plugin-metadata schema to a coremodel * Newer cuetsy; try quoting field name * Add slot definitions * Start sketching out pfs package * Rerun codegen with fixes, new cuetsy * Catch up dashboard with new cuetsy * Update to go1.18 * Use new vmuxers in thema * Add slot system in Go * Draft finished implementation of pfs * Collapse slot pkg into coremodel dir; add PluginInfo * Add the mux type on top of kernel * Refactor plugin generator for extensibility * Change models.cue package, numerous debugs * Bring new output to parity with old * Remove old plugin generation logic * Misc tweaking * Reintroduce generation of shared schemas * Drop back to go1.17 * Add globbing to tsconfig exclude * Introduce pfs test on existing testdata * Make most existing testdata tests pass with pfs * coremodels: Convert plugin-metadata schema to a coremodel * Newer cuetsy; try quoting field name * Add APIType control concept, regen pluginmeta * Use proper numeric types for schema fields * Make pluginmeta schema follow Go type breakdown * More decomposition into distinct types * Add test case for no plugin.json file * Fix missing ref to #Dependencies * Remove generated TS for pluginmeta * Update dependencies, rearrange go.mod * Regenerate without Model prefix * Use updated thema loader; this is now runnable * Skip app plugin with weird include * Make plugin tree extractor reusable * Split out slot lineage load/validate logic * Add myriad tests for new plugin validation failures * Add test for zip fixtures * One last run of codegen * Proper delinting * Ensure validation order is deterministic * Let there actually be sorting * Undo reliance on builtIn field (#54009) * undo builtIn reliance * fix tests Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
2022-08-22 11:11:45 -05:00
})
}
}