Commit Graph

26 Commits

Author SHA1 Message Date
sam boyer
46143b0764
kindsys: Change "Meta" to "Properties" (#59852) 2022-12-05 14:48:47 -05:00
Marcus Andersson
7f92f1df00
Schema: Refactor plugin code generation (#58901)
* wip

* wip

* almost there..

* wip - change so it can run.

* treelist is working.

* support CODEGEN_VERIFY env variable

* use log.fatal

* comment out old PluginTreeList code generation

* cleanup

* rename corelist package files

* fix makefile

* move pkg/codegen/pluggen.go to pkg/plugins/codegen

* copy and refactor files to pkg/plugins/codegen

* use pkg/plugins/codegen instead of pkg/codegen for core plugins code gen

* remove unneeded files

* remove unused code to resolve linting errors

* adapters first hack

* added flattener

* add back ignore build tags to go generate file

* cleaned up the code a bit.

* seems to work, needs to do some refactoring of the GoTypesJenns and TSTypesJenny.

* one more step, going to get upstream changes in this branch.

* working but need to run import tmpl in jenny_schemapath to have the proper imports.

* added header to generated files.

* added missing jenny.

* preventing plugins with multiple decls/schemas to insert multiple lines in corelist.

* fixed so we use Slot type from kindsys to detect if its group.

* adding a go jenny that only runs if the plugin has a backend.

* added version object to generated ts.

* generating the ts types with the same output as prior to this refactoring.

* removed code that is replaced by the jenny pattern.

* removed the go code that isn't used anymore.

* removed some more unused code and renamed pluggen to util_ts

* fixed linting issue.

* removed unused vars.

* use a jenny list postprocessor for header injection

* moved decl and decl_parser to pfs.

* removed the pre-pended header in the gotypes jenny since it is done in the postprocess.

* moved decl to pfs.

* removed unused template.

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2022-12-02 08:22:28 +01:00
Denis Limarev
b814c66c1d
Perfomance: add preallocation for some slices (#59263)
add preallocation for some slices
2022-11-28 13:10:24 +01:00
sam boyer
42baad837a
codegen: Refactor core jennies for reusability, add version-picking metajennies (#58995)
* Add each-major jenny, refactor TS jenny for it

* Introduce LatestJenny, refactor GoTypesJenny for it
2022-11-22 15:00:29 +01:00
sam boyer
6ac43c9cb4
kindsys: Return Decl from Kinds (#59075)
* kindsys: Return Decl from Kinds

* Add funcs to extract DeclForGen from kinds
2022-11-21 17:50:06 -05:00
sam boyer
c4528f9bd8
codejen: Update to latest codejen (#58866) 2022-11-16 15:08:01 -05:00
sam boyer
78f0340031
plugindef: Move pluginmeta out of coremodels as standalone thema lineage (#56765)
* Get pluginmeta mostly moved over to pkg/plugins/plugindef

* Remove dead func

* Fix up pfs, use sync.Once in plugindef

* Update to latest thema

* Chase Endec->Codec conversion in Thema

* Comments on slash header gen; use ToSlash

* Also generate JSON schema for plugindef

* Generate JSON Schema as well

* Fix slot loading from kindsys cue decls

* Remove unused vars

* skip generating plugin.schema.json for now

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2022-11-15 14:48:31 +01:00
sam boyer
07e5f8117f
Reconcile coremodels, entities, objects under new kind framework (#56492)
* Update thema to latest

* Deal with s/Library/*Runtime/

* Commit new, working results of codegen

* We like pointers now

* Always take runtime arg for NewBase()

* Sketchy handwavy pass at entity meta framework

* Little nibbles

* Update pkg/framework/coremodel/entityframework.cue

Co-authored-by: Artur Wierzbicki <wierzbicki.artur.94@gmail.com>

* Move file into new framework location

* Introduce loaders, Go code

* Complete rename to kind

* Flesh out framework, add svg/dashboard examples

* Cruft removal

* Remove generated kind go files from gitignore

* Refine maturity concept, add SlotKind

* Update embed and go deps

* Export PrefixWithGrafanaCUE

* Make the loader actually work, holy crap

* Many small tweaks to type.cue

* Add Apache 2 licensing exceptions for kinds

* Add new kinds dir, start of generator

* Roll back to earlier oapi-codegen

* Introduce new grafana-specific CUE loaders

* Introduce new tidy code generators framework

* Catch up kind framework with tinkering

* Add slices for the generators

* Add write/verify step to main generator

* Many renames

* Split up kind framework cue files

* Use kind.Decl within generated kinds

* Create kind.SomeDecl wrapper type to cache lineages

* Better names again

* Get one generated implemented, hopefully

* Copy dashboard schema into new kind.cue

* Small fixes to make the initial gen work

* Put svg kind in its new home

* Add generated Go dashboard type

* More renames and cleanups

* Add base kind registry and generator

* Stop blacklisting *_gen.go files

This is not the Go best practice, anyway. All we actually want to ignore
for enterprise is generated wire files.

* Change codegen output directories

pkg/kind -> pkg/kinds
pkg/registry/kindreg -> pkg/registry/corekind

* Rename pkg/framework/kind to pkg/kindsys

* Add core structured kind generator

* Add plural and machine names to kind spec

* Copy playlist over to kind system

* Consolidate kindsys files

* Add raw kind generator

* Update CODEOWNERS for kind framework

* Touch up comments a bit

* More docs tweaks

* Remove generated types to reduce noise for review

* Split each generator into its own file

* Rename Slot kind to Composable kind

* Add handwavy types for customkind loading

* Guard against init calls to framework loader

* First pass at doc on extending the kind system

* Improve attribute example in docs

* Fix wire imports

* Add basic TS types generator

* Fix composable kind category def

* No need for a separate file with generate directive

* Catch dashboard schema up

* Rename generator types to something saner and generic

* Make version configurable in ts/go generators

* Add CommonMeta to ease property access

* Add kindsys prop indicating whether lineage is group

* Put all kind categories back in a single file

* Finish with kindsys group props

* Refactor maturity progression per discussion

- Replace "committed" with "merged"
- All kindcats can use all maturity levels, at least for now

* Convert ts veneer index generator to modular system

* Move over to new jennywrites framework

* Strip down old coremodel generator

* Use public version of jennywrites

* Pull latest thema

* Commit generated Go types

* Add header injection postprocessor

* Move sdboyer/jennywrites to grafana/codejen

* Tweak header output

* Remove dashboard and playlist coremodels

* Fix up backend dashboards devenv test

* Fix TS import patterns to new gen filename

* Update internal imports, remove coremodel registry

* Fix compilation errors, wire generation

* Export and replace the prefix dropper

* More Go struct and field name changes

* Last name fixes, hopefully

* Fix lint errors

* Last lint error

Co-authored-by: Artur Wierzbicki <wierzbicki.artur.94@gmail.com>
2022-11-10 12:36:40 -08:00
sam boyer
e5a6547a94
coremodels: Update to latest Thema with generics (#56602)
* Update thema to latest

* Deal with s/Library/*Runtime/

* Commit new, working results of codegen
2022-10-11 09:45:07 +01:00
sam boyer
59a3e18bb2
Fix prefix dropper in Go codegen (#56041) 2022-09-29 12:01:02 -04:00
sam boyer
e2ff875976
codegen: Introduce TS codegen veneer (#54816)
* Split all named types out into defs, etc.

* Use latest cuetsy, refactor generators accordingly

* Return AST type from plugin TS generator

* Near-complete checkin of TS veneer code generator

* First full completed pass

* Improve the attribute name

* Defer use of the dashboard veneer type to follow-up

* Remove dummy index, prettier on veneer

* Fix merge errors in gen.go

* Add match field to SpecialValueMap

* Fix backend lint errors
2022-09-26 11:26:18 -04:00
sam boyer
ced53a8dc2
plugins: Introduce generated, static core plugin registry (#54118)
* Refactor towards template/codegen framework

* Add templates for plugin gen

* Add Go codegen for plugins; overhaul framework, too

* Add new codegen output; assorted framework fixes

* Regenerate after merge

* Remove accidental commit file, update templates

* Export the pfs.Tree loader from plugin types

* Print details from cuetsy errors

* Generate loaders for all plugins and list in registry

* Use pfs_gen.go over lineage_gen.go

* Un-un-ignore main file

* Introduce simple List static registry for plugins

* Last tweaks to codegen

* remove unused tvars

* Ensure loop-local instances for both vars

* Generate pfs parsing in-place in registry

* Stop generating pfs_gen.go

* Move Tree into pfs, rename subdir

* Change package name to match dir

* Ignore gocyclo on HTTPServer.getNavTree
2022-09-14 10:15:09 -04:00
sam boyer
4d433084a5
plugins: New static scanner and validator, with Thema slot support (#53754)
* coremodels: Convert plugin-metadata schema to a coremodel

* Newer cuetsy; try quoting field name

* Add slot definitions

* Start sketching out pfs package

* Rerun codegen with fixes, new cuetsy

* Catch up dashboard with new cuetsy

* Update to go1.18

* Use new vmuxers in thema

* Add slot system in Go

* Draft finished implementation of pfs

* Collapse slot pkg into coremodel dir; add PluginInfo

* Add the mux type on top of kernel

* Refactor plugin generator for extensibility

* Change models.cue package, numerous debugs

* Bring new output to parity with old

* Remove old plugin generation logic

* Misc tweaking

* Reintroduce generation of shared schemas

* Drop back to go1.17

* Add globbing to tsconfig exclude

* Introduce pfs test on existing testdata

* Make most existing testdata tests pass with pfs

* coremodels: Convert plugin-metadata schema to a coremodel

* Newer cuetsy; try quoting field name

* Add APIType control concept, regen pluginmeta

* Use proper numeric types for schema fields

* Make pluginmeta schema follow Go type breakdown

* More decomposition into distinct types

* Add test case for no plugin.json file

* Fix missing ref to #Dependencies

* Remove generated TS for pluginmeta

* Update dependencies, rearrange go.mod

* Regenerate without Model prefix

* Use updated thema loader; this is now runnable

* Skip app plugin with weird include

* Make plugin tree extractor reusable

* Split out slot lineage load/validate logic

* Add myriad tests for new plugin validation failures

* Add test for zip fixtures

* One last run of codegen

* Proper delinting

* Ensure validation order is deterministic

* Let there actually be sorting

* Undo reliance on builtIn field (#54009)

* undo builtIn reliance

* fix tests

Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
2022-08-22 12:11:45 -04:00
sam boyer
a87c685345
coremodels: Convert plugin-metadata schema to a coremodel (#52121)
* coremodels: Convert plugin-metadata schema to a coremodel

* Newer cuetsy; try quoting field name

* Add APIType control concept, regen pluginmeta

* Use proper numeric types for schema fields
2022-08-19 12:11:13 +02:00
Victor Marin
ae69b1d99c
HistogramPanel: Generate .cue model (#53826)
* Generate .cue model for histogram panel

* Docs

* Refactor PR
2022-08-18 16:19:11 +03:00
Zoltán Bedi
ea5e58a051
BarChart/PieChart: create/update models.cue (#52988)
* Reverse engineering barchart models.cue from models.gen.ts

* Barchart models.cue and import fixes

* Add models.cue for piechart

* Use single-item syntax to dodge cuetsy bug

* Add some docs

Co-authored-by: sam boyer <sdboyer@grafana.com>
2022-08-18 14:46:36 +02:00
sam boyer
8ef6820f06
coremodels: Remove "Model" prefix from Go types (#53801)
Using the prefix "Model" on all generated types was adding more
verbosity to already lengthy generated types. This removed that prefix,
only replacing the base generated struct with the literal "Model" to
avoid a redundant name, e.g. dashboard.Dashboard.
2022-08-17 05:42:41 -04:00
Jo
062d255124
Handle ioutil deprecations (#53526)
* replace ioutil.ReadFile -> os.ReadFile

* replace ioutil.ReadAll -> io.ReadAll

* replace ioutil.TempFile -> os.CreateTemp

* replace ioutil.NopCloser -> io.NopCloser

* replace ioutil.WriteFile -> os.WriteFile

* replace ioutil.TempDir -> os.MkdirTemp

* replace ioutil.Discard -> io.Discard
2022-08-10 15:37:51 +02:00
sam boyer
b11f66b4bb
coremodels: Combine static and generic registries (#53246)
* Stop generating non-dynamic registry code

* Remove generic, errors, s/static/base/

* Sort during codegen, not runtime

* Not a method call

* Precisiate a comment

* Remove generic registry, fix assignability test
2022-08-04 00:04:54 +04:00
sam boyer
566d0a7148
coremodel: Fix hardcoded text and overly broad embedding (#52802)
* Replace hardcoded 'dashboard' references

* Only embed cue.mod/module.cue

* Add quotes
2022-07-26 09:14:07 -04:00
sam boyer
80f6560c9c
coremodels: Change input filename (#51349) 2022-06-23 12:47:47 -07:00
Ryan McKinley
dd5a3b7747
Heatmap: Replace the heatmap panel with new implementation (#50229) 2022-06-23 11:48:28 -07:00
sam boyer
e6d91235a7
codegen: Correctly gen TS without defaults (#51155) 2022-06-22 03:55:37 +02:00
sam boyer
4c4aa95d38
coremodels: Automatically generate coremodel registries (#50057)
* coremodel: Generate static registry

* Actually make codegen work

Also, remove the per-coremodel assignability test from generator set.

* Make wire gen depend on cue gen

This is necessary now that we're generating a wire set as part of
coremodel registry generation.

* Add wire inject bits to http server

* s/staticregistry/registry/

* move to static and dynamic wording

* Move registry type into registry package

* Use static registry in http handler

* Oi comments
2022-06-15 15:47:04 +02:00
sam boyer
8876d56495
schema: Migrate from scuemata to thema (#49805)
* Remove crufty scuemata bits

Buhbye to: cue/ dir with old definitions, CI steps for checking unnecessary
things, and the original dashboard scuemata file.

* Remove grafana-cli cue subcommand

* Remove old testdata

* Don't swallow errors from codegen

* Small nits and tweaks to cuectx package

* WIP - refactor pluggen to use Thema

Also consolidate the embed.FS in the repo root.

* Finish halfway rename

* Convert all panel models.cue to thema

* Rewrite pluggen to use Thema

* Remove pkg/schema, and trim command

* Remove schemaloader service and usages

Will be replaced by coremodel-centric hydrate/dehydrate system Soon™.

* Remove schemaloader from wire

* Remove hangover field on histogram models.cue

* Fix lint errors, some vestiges of trim service

* Remove unused cuetsify cli command
2022-06-06 17:52:44 -07:00
sam boyer
be06d37a20
schema: Generate Go and Typescript from Thema coremodels (#49193)
* Add go code generator for coremodels

* Just generate the entire coremodel for now

Maybe we'll need more flexibility as more coremodels are added, but for
now this is fine.

* Add note on type comment about stability, grodkit

* Remove local replace directive for thema

* Generate typescript from coremodel

* Update pkg/coremodel/dashboard/addenda.go

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>

* Update cuetsy to new release

* Update thema to latest

* Fix enum generation for FieldColorModeId

* Put main generated object at the end of the file

* Tweaks to generated Go output

* Retweak back to var

* Add generated coremodel test

* Remove local replace statement again

* Add Make target and call into cuetsy cmd from gen

* Rename and comment linsrc for readability

* Move key codegen bits into reusable package

* Move body of cuetsifier into codegen pkg

Also genericize the diffing output into reusable WriteDiffer.

* Refactor coremodel generator to use WriteDiffer

* Add gen-cue step to CI

* Whip all the codegen automation into shape

* Add simplistic coremodel canonicality controls

* Remove erroneously committed test

* Bump thema version

* Remove dead code

* Improve wording of non-canonicality comment

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
2022-05-27 03:21:37 +02:00