Commit Graph

33 Commits

Author SHA1 Message Date
Selene
473898e47c
Core: Remove thema and kindsys dependencies (#84499)
* Move some thema code inside grafana

* Use new codegen instead of thema for core kinds

* Replace TS generator

* Use new generator for go types

* Remove thema from oapi generator

* Remove thema from generators

* Don't use kindsys/thema for core kinds

* Remove kindsys/thema from plugins

* Remove last thema related

* Remove most of cuectx and move utils_ts into codegen. It also deletes wire dependency

* Merge plugins generators

* Delete thema dependency 🎉

* Fix CODEOWNERS

* Fix package name

* Fix TS output names

* More path fixes

* Fix mod codeowners

* Use original plugin's name

* Remove kindsys dependency 🎉

* Modify oapi schema and create an apply function to fix elasticsearch errors

* cue.mod was deleted by mistake

* Fix TS panels

* sort imports

* Fixing elasticsearch output

* Downgrade oapi-codegen library

* Update output ts files

* More fixes

* Restore old elasticsearch generated file and skip its generation. Remove core imports into plugins

* More lint fixes

* Add codeowners

* restore embed.go file

* Fix embed.go
2024-03-21 11:11:29 +01:00
Selene
5c7849417b
Schemas: Replace registry generation and github workflow (#83490)
* Create small registries for core and composable kinds

* Update workflow with new registries

* Fix imports in plugin schemas and deleted old registry generation files

* Remove verification and maturity

* Modify registries and add missing composable information to make schemas in kind-registry work

* Add missing aliases

* Remove unused templates

* Remove kinds verification

* Format generated code

* Add gen header

* Delete unused code and clean path in composable template

* Delete kind-registry loader

* Delete unused code

* Update License link

* Update codeowners path

* Sort imports

* More cleanup

* Remove verify-kinds.yml from codeowners

* Fix lint

* Update composable_kidns

* Fix cue extension

* Restore verify-kinds to avoid to push outdated kind's registry

* Fix composable format

* Restore code owners for verify-kinds

* Remove verify check
2024-03-13 18:05:21 +02:00
Selene
75ea33e0cd
Schemas: Delete unused code from the previous refactors (#84254)
* Delete unused code from the previous refactors

* Sort imports
2024-03-13 12:30:05 +01:00
Selene
275ccf994b
Schemas: Reduce duplicated jenny code (#84061)
* Remove jenny_ts_resources and use jenny_ts_types for all cases

* Unify TS generated files into one jenny

* Add missing imports to versioned files

* Update Parca plugin

* Fix loki

* Use LokiQuery

* Fix pyroscope tests

* Fix prettier

* 😒 fix default pyroscope value name

* Set the LokiQuery

* Update Elasticsearch and TestData

* Missed files from testdata

* Order imports
2024-03-11 12:51:44 +01:00
Selene
1181141b40
Schemas: Refactor plugin's metadata (#83696)
* Remove kinds verification for kind-registry

* Read plugin's json information with json library instead of use thema binding

* Remove grafanaplugin unification

* Don't use kindsys for extract the slot name

* Fix IsGroup

* Remove all plugindef generation

* Refactor schema interfaces

* Pushed this change from a different branch by mistake...

* Create small plugin definition structure adding additional information for plugins registration

* Add some validation checks

* Delete unused code

* Fix imports lint
2024-03-07 11:09:19 +01:00
Tania
8895b43c85
Chore: Remove docs and kinds report generators (#83277)
* Chore: Remove codegen for docs

* Remove kindsysreport
2024-02-23 20:06:43 +01:00
Agnès Toulet
ac445a25d5
Kindsys: update generated doc and add check in CI (#71871)
* clean up docs

* fix Windows path issue

* add verify-gen-cue to doc pipeline

* fix linting error
2023-07-20 11:48:12 +02:00
Zoltán Bedi
39433b5b9a
Candlestick: Migrate to schema (#62113)
Co-authored-by: nmarrs <nathanielmarrs@gmail.com>
2023-06-20 07:18:40 +03:00
Nathan Marrs
d7a8892f3a
Canvas: Migrate to schema (#70205) 2023-06-17 05:56:23 +03:00
sam boyer
09895c26b6
Chore: Hacky first pass on generating composable kinds within grafana/schema (#64723)
* Super hacky first pass on gen types

* First sketchy pass at generated compo kinds in TS, grok-style

* Merge fix

* Create jenny for MajorsOrX for plugins

* Re-generate files with imports

* Delete invalid generated file

* Fix cue

* Update go.mod

* Update schemas

* Fix go-imports

* Regenerate cue files

---------

Co-authored-by: spinillos <selenepinillos@gmail.com>
2023-05-30 05:56:18 -07:00
Tania
dcb3b4d97a
Codegen: Fix composable kinds docs generation (#68087)
* Fix plugins docs and regenerate

* fixup! Fix plugins docs and regenerate
2023-05-10 11:27:13 +02:00
Ryan McKinley
7835aacc8f
Schema: Remove exclusion for timeseries and update imports (#65242) 2023-03-23 02:14:54 +02:00
Kristin Laemmert
8745d7ef1b
extract kindsys to external library (#64562)
* extract kindsys

* reinstate kindsys report

This may end up living somewhere else (or not! who knows!), but the important part is that I don't get rid of it right now :)

I hate the package layout (kindsysreport/codegen) for the main function and will take pretty much any alternative suggestion, but we can change also change it later.

Note that the generated report.json is in a different location - anything using this (ops something) needs to be updated.

* kindsysreport in codeowners
2023-03-15 12:04:28 -04:00
Victor Marin
a91e0a49c9
Migrate table to cue model (#61852)
* WIP table cue model

* WIP table types migration

* refactor

* WIP table cue

* docs

* veneer fix, docs

* docs
2023-03-01 17:48:36 +02:00
Victor Marin
e1bc3fad83
Heatmap: Migrate to CUE (#62173)
* IP migrate heatmap to cue

* WIP

* refactor

* change usages and refactor panelcfg

* CR modifications and docs

* refactor, enable heatmap cue generation, overwrite enum values in types.ts

* small refactor

* refactor types.ts

* cr modifications

* docs
2023-03-01 17:47:21 +02:00
Agnès Toulet
121e384fab
kindsys: Fix CUE code generation on Windows (#62812)
* fix Windows paths

* fix remaining paths issues

* clean up multiple use of Dir()

* fix headers in generated files
2023-02-02 15:06:55 -05:00
sam boyer
30b4205521
Kindsys: Replace DefForGen with kindsys.Kind (#62642)
* Kindsys: Replace DeclForGen with kindsys.Kind

DeclForGen was always unnecessary - it just wasn't obvious on initial
implementation, when we were focused on generating unique types for each
core kind. This removes it, considerably simplifying interactions with
kindsys - virtually everything now just relies on kindsys.Kind and its
derived interfaces.

* Removed unused jenny

* Rename params in jennies
2023-01-31 19:40:15 -05:00
sam boyer
dbe2f0c8f6
Kindsys: Replace "Declaration" with "Definition" (#62515)
* s/Declaration/Definition/g

* s/DeclForGen/DefForGen/g

* Rename some local vars
2023-01-31 09:50:08 +00:00
Victor Marin
6a93c77082
Refactor state timeline/status history to cue model and refactor TimelineChart component (#61631)
* Adapt state timeline to scuemata

* Refactor status history to cue model

* Refactor

* Refactor TimelineChart as a core component

* wip

* Change as per CR

Co-authored-by: sam boyer <sdboyer@grafana.com>
2023-01-26 07:03:59 +00:00
Selene
3c073b00ec
Schema: Fix interfaces with blank spaces (#62144)
Fixes interfaces with blank spaces
2023-01-25 17:03:46 +00:00
sam boyer
f62f3cb0e9
Kindsys: Generate plugin Go types under a kinds/ dir (#61899)
Kindsys: Generate plugin Go types under kinds/
2023-01-23 08:56:00 -05:00
Tania
5f93e67419
Codegen: Generate docs for composable kinds (#61719)
* Generate docs for composable kinds

* Fix getting the kind schema

* Generate composable kinds docs

* Add disclaimer to the generated docs

* Change docs disclaimer
2023-01-22 18:58:49 -05:00
sam boyer
3b3059c9ce
Kindsys: Unify plugins, pfs with kind framework (#61192)
* New pfs impl

* Reached codegen parity with old system

* Update all models.cue inputs

* Rename all models.cue files

* Remove unused prefixfs

* Changes Queries->DataQuery schema interface

* Recodegen

* All tests passing, nearly good now

* Add SchemaInterface to kindsys props

* Add pascal name deriver

* Relocate plugin cue files again

* Clarify use of injected fields

* Remove unnecessary aliasing

* Move DataQuery into mudball

* Allow forcing ExpandReferences on go type generation

* Move DataQuery def into kindsys, add generator to copy it to common

* Fix copy generator to replace package name correctly

* Fix duplicate type, test failure

* Fix linting issues
2023-01-20 09:41:35 +00:00
sam boyer
4db3b2fd5c
Kindsys: Remove defs, Slot->SchemaInterface (#61069)
* kindsys: Remove defs, Slot->SchemaInterface

* Remove excess file

* Fix up tests

* Regenerate kinds report

* Final bits of cleanup

* Stop complaining, linter

* Update pkg/kindsys/kindcat_composable.cue

Co-authored-by: Tania <yalyna.ts@gmail.com>

Co-authored-by: Tania <yalyna.ts@gmail.com>
2023-01-06 12:37:32 -05:00
Marcus Andersson
7f92f1df00
Schema: Refactor plugin code generation (#58901)
* wip

* wip

* almost there..

* wip - change so it can run.

* treelist is working.

* support CODEGEN_VERIFY env variable

* use log.fatal

* comment out old PluginTreeList code generation

* cleanup

* rename corelist package files

* fix makefile

* move pkg/codegen/pluggen.go to pkg/plugins/codegen

* copy and refactor files to pkg/plugins/codegen

* use pkg/plugins/codegen instead of pkg/codegen for core plugins code gen

* remove unneeded files

* remove unused code to resolve linting errors

* adapters first hack

* added flattener

* add back ignore build tags to go generate file

* cleaned up the code a bit.

* seems to work, needs to do some refactoring of the GoTypesJenns and TSTypesJenny.

* one more step, going to get upstream changes in this branch.

* working but need to run import tmpl in jenny_schemapath to have the proper imports.

* added header to generated files.

* added missing jenny.

* preventing plugins with multiple decls/schemas to insert multiple lines in corelist.

* fixed so we use Slot type from kindsys to detect if its group.

* adding a go jenny that only runs if the plugin has a backend.

* added version object to generated ts.

* generating the ts types with the same output as prior to this refactoring.

* removed code that is replaced by the jenny pattern.

* removed the go code that isn't used anymore.

* removed some more unused code and renamed pluggen to util_ts

* fixed linting issue.

* removed unused vars.

* use a jenny list postprocessor for header injection

* moved decl and decl_parser to pfs.

* removed the pre-pended header in the gotypes jenny since it is done in the postprocess.

* moved decl to pfs.

* removed unused template.

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2022-12-02 08:22:28 +01:00
Ryan McKinley
0a7a5b13fe
Heatmap: Remove legacy angular based implementation (#59249)
remove angular heatmap
2022-11-23 10:46:21 -08:00
sam boyer
07e5f8117f
Reconcile coremodels, entities, objects under new kind framework (#56492)
* Update thema to latest

* Deal with s/Library/*Runtime/

* Commit new, working results of codegen

* We like pointers now

* Always take runtime arg for NewBase()

* Sketchy handwavy pass at entity meta framework

* Little nibbles

* Update pkg/framework/coremodel/entityframework.cue

Co-authored-by: Artur Wierzbicki <wierzbicki.artur.94@gmail.com>

* Move file into new framework location

* Introduce loaders, Go code

* Complete rename to kind

* Flesh out framework, add svg/dashboard examples

* Cruft removal

* Remove generated kind go files from gitignore

* Refine maturity concept, add SlotKind

* Update embed and go deps

* Export PrefixWithGrafanaCUE

* Make the loader actually work, holy crap

* Many small tweaks to type.cue

* Add Apache 2 licensing exceptions for kinds

* Add new kinds dir, start of generator

* Roll back to earlier oapi-codegen

* Introduce new grafana-specific CUE loaders

* Introduce new tidy code generators framework

* Catch up kind framework with tinkering

* Add slices for the generators

* Add write/verify step to main generator

* Many renames

* Split up kind framework cue files

* Use kind.Decl within generated kinds

* Create kind.SomeDecl wrapper type to cache lineages

* Better names again

* Get one generated implemented, hopefully

* Copy dashboard schema into new kind.cue

* Small fixes to make the initial gen work

* Put svg kind in its new home

* Add generated Go dashboard type

* More renames and cleanups

* Add base kind registry and generator

* Stop blacklisting *_gen.go files

This is not the Go best practice, anyway. All we actually want to ignore
for enterprise is generated wire files.

* Change codegen output directories

pkg/kind -> pkg/kinds
pkg/registry/kindreg -> pkg/registry/corekind

* Rename pkg/framework/kind to pkg/kindsys

* Add core structured kind generator

* Add plural and machine names to kind spec

* Copy playlist over to kind system

* Consolidate kindsys files

* Add raw kind generator

* Update CODEOWNERS for kind framework

* Touch up comments a bit

* More docs tweaks

* Remove generated types to reduce noise for review

* Split each generator into its own file

* Rename Slot kind to Composable kind

* Add handwavy types for customkind loading

* Guard against init calls to framework loader

* First pass at doc on extending the kind system

* Improve attribute example in docs

* Fix wire imports

* Add basic TS types generator

* Fix composable kind category def

* No need for a separate file with generate directive

* Catch dashboard schema up

* Rename generator types to something saner and generic

* Make version configurable in ts/go generators

* Add CommonMeta to ease property access

* Add kindsys prop indicating whether lineage is group

* Put all kind categories back in a single file

* Finish with kindsys group props

* Refactor maturity progression per discussion

- Replace "committed" with "merged"
- All kindcats can use all maturity levels, at least for now

* Convert ts veneer index generator to modular system

* Move over to new jennywrites framework

* Strip down old coremodel generator

* Use public version of jennywrites

* Pull latest thema

* Commit generated Go types

* Add header injection postprocessor

* Move sdboyer/jennywrites to grafana/codejen

* Tweak header output

* Remove dashboard and playlist coremodels

* Fix up backend dashboards devenv test

* Fix TS import patterns to new gen filename

* Update internal imports, remove coremodel registry

* Fix compilation errors, wire generation

* Export and replace the prefix dropper

* More Go struct and field name changes

* Last name fixes, hopefully

* Fix lint errors

* Last lint error

Co-authored-by: Artur Wierzbicki <wierzbicki.artur.94@gmail.com>
2022-11-10 12:36:40 -08:00
sam boyer
e5a6547a94
coremodels: Update to latest Thema with generics (#56602)
* Update thema to latest

* Deal with s/Library/*Runtime/

* Commit new, working results of codegen
2022-10-11 09:45:07 +01:00
sam boyer
e2ff875976
codegen: Introduce TS codegen veneer (#54816)
* Split all named types out into defs, etc.

* Use latest cuetsy, refactor generators accordingly

* Return AST type from plugin TS generator

* Near-complete checkin of TS veneer code generator

* First full completed pass

* Improve the attribute name

* Defer use of the dashboard veneer type to follow-up

* Remove dummy index, prettier on veneer

* Fix merge errors in gen.go

* Add match field to SpecialValueMap

* Fix backend lint errors
2022-09-26 11:26:18 -04:00
sam boyer
ced53a8dc2
plugins: Introduce generated, static core plugin registry (#54118)
* Refactor towards template/codegen framework

* Add templates for plugin gen

* Add Go codegen for plugins; overhaul framework, too

* Add new codegen output; assorted framework fixes

* Regenerate after merge

* Remove accidental commit file, update templates

* Export the pfs.Tree loader from plugin types

* Print details from cuetsy errors

* Generate loaders for all plugins and list in registry

* Use pfs_gen.go over lineage_gen.go

* Un-un-ignore main file

* Introduce simple List static registry for plugins

* Last tweaks to codegen

* remove unused tvars

* Ensure loop-local instances for both vars

* Generate pfs parsing in-place in registry

* Stop generating pfs_gen.go

* Move Tree into pfs, rename subdir

* Change package name to match dir

* Ignore gocyclo on HTTPServer.getNavTree
2022-09-14 10:15:09 -04:00
sam boyer
4d433084a5
plugins: New static scanner and validator, with Thema slot support (#53754)
* coremodels: Convert plugin-metadata schema to a coremodel

* Newer cuetsy; try quoting field name

* Add slot definitions

* Start sketching out pfs package

* Rerun codegen with fixes, new cuetsy

* Catch up dashboard with new cuetsy

* Update to go1.18

* Use new vmuxers in thema

* Add slot system in Go

* Draft finished implementation of pfs

* Collapse slot pkg into coremodel dir; add PluginInfo

* Add the mux type on top of kernel

* Refactor plugin generator for extensibility

* Change models.cue package, numerous debugs

* Bring new output to parity with old

* Remove old plugin generation logic

* Misc tweaking

* Reintroduce generation of shared schemas

* Drop back to go1.17

* Add globbing to tsconfig exclude

* Introduce pfs test on existing testdata

* Make most existing testdata tests pass with pfs

* coremodels: Convert plugin-metadata schema to a coremodel

* Newer cuetsy; try quoting field name

* Add APIType control concept, regen pluginmeta

* Use proper numeric types for schema fields

* Make pluginmeta schema follow Go type breakdown

* More decomposition into distinct types

* Add test case for no plugin.json file

* Fix missing ref to #Dependencies

* Remove generated TS for pluginmeta

* Update dependencies, rearrange go.mod

* Regenerate without Model prefix

* Use updated thema loader; this is now runnable

* Skip app plugin with weird include

* Make plugin tree extractor reusable

* Split out slot lineage load/validate logic

* Add myriad tests for new plugin validation failures

* Add test for zip fixtures

* One last run of codegen

* Proper delinting

* Ensure validation order is deterministic

* Let there actually be sorting

* Undo reliance on builtIn field (#54009)

* undo builtIn reliance

* fix tests

Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
2022-08-22 12:11:45 -04:00
sam boyer
8876d56495
schema: Migrate from scuemata to thema (#49805)
* Remove crufty scuemata bits

Buhbye to: cue/ dir with old definitions, CI steps for checking unnecessary
things, and the original dashboard scuemata file.

* Remove grafana-cli cue subcommand

* Remove old testdata

* Don't swallow errors from codegen

* Small nits and tweaks to cuectx package

* WIP - refactor pluggen to use Thema

Also consolidate the embed.FS in the repo root.

* Finish halfway rename

* Convert all panel models.cue to thema

* Rewrite pluggen to use Thema

* Remove pkg/schema, and trim command

* Remove schemaloader service and usages

Will be replaced by coremodel-centric hydrate/dehydrate system Soon™.

* Remove schemaloader from wire

* Remove hangover field on histogram models.cue

* Fix lint errors, some vestiges of trim service

* Remove unused cuetsify cli command
2022-06-06 17:52:44 -07:00
sam boyer
be06d37a20
schema: Generate Go and Typescript from Thema coremodels (#49193)
* Add go code generator for coremodels

* Just generate the entire coremodel for now

Maybe we'll need more flexibility as more coremodels are added, but for
now this is fine.

* Add note on type comment about stability, grodkit

* Remove local replace directive for thema

* Generate typescript from coremodel

* Update pkg/coremodel/dashboard/addenda.go

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>

* Update cuetsy to new release

* Update thema to latest

* Fix enum generation for FieldColorModeId

* Put main generated object at the end of the file

* Tweaks to generated Go output

* Retweak back to var

* Add generated coremodel test

* Remove local replace statement again

* Add Make target and call into cuetsy cmd from gen

* Rename and comment linsrc for readability

* Move key codegen bits into reusable package

* Move body of cuetsifier into codegen pkg

Also genericize the diffing output into reusable WriteDiffer.

* Refactor coremodel generator to use WriteDiffer

* Add gen-cue step to CI

* Whip all the codegen automation into shape

* Add simplistic coremodel canonicality controls

* Remove erroneously committed test

* Bump thema version

* Remove dead code

* Improve wording of non-canonicality comment

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
2022-05-27 03:21:37 +02:00