* Add verify-starlark build action that returns an error for starlark files with lint
Relies on `buildifier` tool.
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Add verify_starlark_step to PR pipeline
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Manually fetch buildifier in curl_image until a new build_image is created
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Format with buildifier
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Remove all unused variables retaining one unused function
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Use snake_case for variable
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Replace deprecated dictionary concatenation with .update() method
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Start adding docstrings for all modules and functions
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Prefer os.WriteFile as ioutil.WriteFile has been deprecated since go 1.16
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Attempt to document the behavior of the init_enterprise_step
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Document test_backend pipeline
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Document enterprise_downstream_step
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Document the pipeline utility function
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Document publish_images_step
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Document publish_images_steps
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Document enterprise2_pipelines function
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Add tags table for Starlark files.
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Document test_frontend
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Document windows function
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Add docstrings to verifystarlark functions
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Refactor error handling to be more clear and document complex behavior
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Split errors into execution errors and verification errors
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Document all other library functions
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Add local variables to TAGS
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Add blank line between all Args and Returns sections
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Fix new linting errors
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Lint new Starlark files
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Correct buildifier binary mv
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Document the need to set nofile ulimit to at least 2048
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Update build-container to include buildifier
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Ensure buildifier binary is executable
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Fix valid content test
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Simply return execution error
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Only check files rather than fixing them
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Use updated build-container with executable buildifier
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Test that context cancellation stops execution
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Simplify error handling
Return execution errors that short circuit WalkDir rather than
separately tracking that error.
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Remove fetching of buildifier binary now that it is in the build-container
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Use build image in verify-starlark step
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Use semver tag
The image is the same but uses a semver tag to make it clearer that
this is a forward upgrade from the old version.
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Use node 18 image with buildifier
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
---------
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Kinds: Generate JSON report
* Kinds: Add Validate/Write support for JSON report
* Kinds: Report format
* Kinds: Report new line
* Kinds: Use kindsys.SomeKindMeta to generate the report
* Move kinds report generation to the end
* Re-structure kinds report JSON output
* Make prettier to ignore kinds json report
* Minor on kinds report generation
* Fix toggles_gen test
* wip
* wip
* almost there..
* wip - change so it can run.
* treelist is working.
* support CODEGEN_VERIFY env variable
* use log.fatal
* comment out old PluginTreeList code generation
* cleanup
* rename corelist package files
* fix makefile
* move pkg/codegen/pluggen.go to pkg/plugins/codegen
* copy and refactor files to pkg/plugins/codegen
* use pkg/plugins/codegen instead of pkg/codegen for core plugins code gen
* remove unneeded files
* remove unused code to resolve linting errors
* adapters first hack
* added flattener
* add back ignore build tags to go generate file
* cleaned up the code a bit.
* seems to work, needs to do some refactoring of the GoTypesJenns and TSTypesJenny.
* one more step, going to get upstream changes in this branch.
* working but need to run import tmpl in jenny_schemapath to have the proper imports.
* added header to generated files.
* added missing jenny.
* preventing plugins with multiple decls/schemas to insert multiple lines in corelist.
* fixed so we use Slot type from kindsys to detect if its group.
* adding a go jenny that only runs if the plugin has a backend.
* added version object to generated ts.
* generating the ts types with the same output as prior to this refactoring.
* removed code that is replaced by the jenny pattern.
* removed the go code that isn't used anymore.
* removed some more unused code and renamed pluggen to util_ts
* fixed linting issue.
* removed unused vars.
* use a jenny list postprocessor for header injection
* moved decl and decl_parser to pfs.
* removed the pre-pended header in the gotypes jenny since it is done in the postprocess.
* moved decl to pfs.
* removed unused template.
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
* streamline and sync dockerfiles
* improve go dependency cacheability
* unify alpine and ubuntu Dockerfiles
* include glibc support in locally-built alpine images
* avoid the need for a second bulky binary for grafana-cli
* look for grafana-server in $PATH as well as same directory
* implement unified "grafana" command
* update dockerfiles, fix grafana-cli -v
* update packaging to work with single binary
- add wrapper scripts for grafana and grafana-server
- update and sync package files
- implement --sign flag of build package command
- stop packaging scripts folder, they are not useful for end users
- add support for --configOverrides in server command
- remove unused nfpm.yaml config file
* windows support
* Get pluginmeta mostly moved over to pkg/plugins/plugindef
* Remove dead func
* Fix up pfs, use sync.Once in plugindef
* Update to latest thema
* Chase Endec->Codec conversion in Thema
* Comments on slash header gen; use ToSlash
* Also generate JSON schema for plugindef
* Generate JSON Schema as well
* Fix slot loading from kindsys cue decls
* Remove unused vars
* skip generating plugin.schema.json for now
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
* Update thema to latest
* Deal with s/Library/*Runtime/
* Commit new, working results of codegen
* We like pointers now
* Always take runtime arg for NewBase()
* Sketchy handwavy pass at entity meta framework
* Little nibbles
* Update pkg/framework/coremodel/entityframework.cue
Co-authored-by: Artur Wierzbicki <wierzbicki.artur.94@gmail.com>
* Move file into new framework location
* Introduce loaders, Go code
* Complete rename to kind
* Flesh out framework, add svg/dashboard examples
* Cruft removal
* Remove generated kind go files from gitignore
* Refine maturity concept, add SlotKind
* Update embed and go deps
* Export PrefixWithGrafanaCUE
* Make the loader actually work, holy crap
* Many small tweaks to type.cue
* Add Apache 2 licensing exceptions for kinds
* Add new kinds dir, start of generator
* Roll back to earlier oapi-codegen
* Introduce new grafana-specific CUE loaders
* Introduce new tidy code generators framework
* Catch up kind framework with tinkering
* Add slices for the generators
* Add write/verify step to main generator
* Many renames
* Split up kind framework cue files
* Use kind.Decl within generated kinds
* Create kind.SomeDecl wrapper type to cache lineages
* Better names again
* Get one generated implemented, hopefully
* Copy dashboard schema into new kind.cue
* Small fixes to make the initial gen work
* Put svg kind in its new home
* Add generated Go dashboard type
* More renames and cleanups
* Add base kind registry and generator
* Stop blacklisting *_gen.go files
This is not the Go best practice, anyway. All we actually want to ignore
for enterprise is generated wire files.
* Change codegen output directories
pkg/kind -> pkg/kinds
pkg/registry/kindreg -> pkg/registry/corekind
* Rename pkg/framework/kind to pkg/kindsys
* Add core structured kind generator
* Add plural and machine names to kind spec
* Copy playlist over to kind system
* Consolidate kindsys files
* Add raw kind generator
* Update CODEOWNERS for kind framework
* Touch up comments a bit
* More docs tweaks
* Remove generated types to reduce noise for review
* Split each generator into its own file
* Rename Slot kind to Composable kind
* Add handwavy types for customkind loading
* Guard against init calls to framework loader
* First pass at doc on extending the kind system
* Improve attribute example in docs
* Fix wire imports
* Add basic TS types generator
* Fix composable kind category def
* No need for a separate file with generate directive
* Catch dashboard schema up
* Rename generator types to something saner and generic
* Make version configurable in ts/go generators
* Add CommonMeta to ease property access
* Add kindsys prop indicating whether lineage is group
* Put all kind categories back in a single file
* Finish with kindsys group props
* Refactor maturity progression per discussion
- Replace "committed" with "merged"
- All kindcats can use all maturity levels, at least for now
* Convert ts veneer index generator to modular system
* Move over to new jennywrites framework
* Strip down old coremodel generator
* Use public version of jennywrites
* Pull latest thema
* Commit generated Go types
* Add header injection postprocessor
* Move sdboyer/jennywrites to grafana/codejen
* Tweak header output
* Remove dashboard and playlist coremodels
* Fix up backend dashboards devenv test
* Fix TS import patterns to new gen filename
* Update internal imports, remove coremodel registry
* Fix compilation errors, wire generation
* Export and replace the prefix dropper
* More Go struct and field name changes
* Last name fixes, hopefully
* Fix lint errors
* Last lint error
Co-authored-by: Artur Wierzbicki <wierzbicki.artur.94@gmail.com>
* add api route for validating a dashboard json
* add feature flag for showDashboardValidationWarnings
* tidy up
* comments and messages
* swagger specs
* fix typo
* more swagger
* tests!
* tidy test a little bit
* no more ioutil
* api will return different status code depending on validation error
* clean up
* handle 4xx errors
* remove console.log
* fix backend tests
* tidy up
* Swagger: Exclude alpha endpoints
Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>
* WIP
* Set public_suffix to a pre Ruby 2.6 version
* we don't need to install python
* Stretch->Buster
* Bump versions in lib.star
* Manually update linter
Sort of messy, but the .mod-file need to contain all dependencies that
use 1.16+ features, otherwise they're assumed to be compiled with
-lang=go1.16 and cannot access generics et al.
Bingo doesn't seem to understand that, but it's possible to manually
update things to get Bingo happy.
* undo reformatting
* Various lint improvements
* More from the linter
* goimports -w ./pkg/
* Disable gocritic
* Add/modify linter exceptions
* lint + flatten nested list
Go 1.19 doesn't support nested lists, and there wasn't an obvious workaround.
https://go.dev/doc/comment#lists
* Adding OpenApi target to makefile, to covert swagger spec into OpenAPI 3
* Adding endpoint to server swaggerui with new openapi3 specs
* Passing output file as parameter for OpenApi3 specs
* Implement workaround for missing host and prefix domain
* Fix get legacy alert response
* Swagger: Fix get folder by UID response
* Fix conflicting swagger model Alert
Reanme legacy alerting swagger model to LegacyAlert to differentiate it
from the prometheus Alert
* Bump grafana-plugin-sdk-go
* Fix get folder response
* Use go-swagger command for merging the specifications and remove merge_specs script
* Chore: Add new go test commands for unit, integration tests to makefile
* Add test-go-unit and test-go-integration targets as dependencies of the test-go target
* Add makefile target for mysql & postgres backends
* Set GRAFANA_TEST_DB variable for the xargs postgres command
Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>
* Set GRAFANA_TEST_DB variable for the xargs mysql command
Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>
* Use postgres_tests as source
Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>
* Set postgres_tests as source
Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>
* Clean test cache before postgres and mysql integration test makefile commands
Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>
* coremodel: Generate static registry
* Actually make codegen work
Also, remove the per-coremodel assignability test from generator set.
* Make wire gen depend on cue gen
This is necessary now that we're generating a wire set as part of
coremodel registry generation.
* Add wire inject bits to http server
* s/staticregistry/registry/
* move to static and dynamic wording
* Move registry type into registry package
* Use static registry in http handler
* Oi comments
* Fix swagger generation targer using mac
Validate using the swagger binarry instead of the docker container
* API: Make swagger specification more consistent
Fix operation parameter ordering
* Add go code generator for coremodels
* Just generate the entire coremodel for now
Maybe we'll need more flexibility as more coremodels are added, but for
now this is fine.
* Add note on type comment about stability, grodkit
* Remove local replace directive for thema
* Generate typescript from coremodel
* Update pkg/coremodel/dashboard/addenda.go
Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
* Update cuetsy to new release
* Update thema to latest
* Fix enum generation for FieldColorModeId
* Put main generated object at the end of the file
* Tweaks to generated Go output
* Retweak back to var
* Add generated coremodel test
* Remove local replace statement again
* Add Make target and call into cuetsy cmd from gen
* Rename and comment linsrc for readability
* Move key codegen bits into reusable package
* Move body of cuetsifier into codegen pkg
Also genericize the diffing output into reusable WriteDiffer.
* Refactor coremodel generator to use WriteDiffer
* Add gen-cue step to CI
* Whip all the codegen automation into shape
* Add simplistic coremodel canonicality controls
* Remove erroneously committed test
* Bump thema version
* Remove dead code
* Improve wording of non-canonicality comment
Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
* Support for documenting stable vs unstable alerting routes
* empty commit, restart drone
* Touch-up references in root makefile and drop trailing escape newline
* Rebase and regenerate
* Extend README with docs for this change
* API: Using go-swagger for extracting OpenAPI specification from source code
* Merge Grafana Alerting spec
* Include enterprise endpoints (if enabled)
* Serve SwaggerUI under feature flag
* Fix building dev docker images
* Configure swaggerUI
* Add missing json tags
Co-authored-by: Ying WANG <ying.wang@grafana.com>
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
* Set up Wire build graph
* Remove enterprise Wire set
* Move runner package outside commands
* Update Makefile (gen-go path)
* Minor prettier fix
* Include new Wire enterprise file into .gitignore
* Update Wire deps
* Update the grabpl version
Co-authored-by: Dan Cech <dcech@grafana.com>
* Force use drone-cli 1.4.0
* Small modifications to prevent null generation
* Sign drone
* Use bingo to declare drone-cli version
* Update message in Makefile
* Update to latest grabpl version
Fixes#30144
Co-authored-by: dsotirakis <sotirakis.dim@gmail.com>
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
Co-authored-by: Ida Furjesova <ida.furjesova@grafana.com>
Co-authored-by: Jack Westbrook <jack.westbrook@gmail.com>
Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
Co-authored-by: Leon Sorokin <leeoniya@gmail.com>
Co-authored-by: Andrej Ocenas <mr.ocenas@gmail.com>
Co-authored-by: spinillos <selenepinillos@gmail.com>
Co-authored-by: Karl Persson <kalle.persson@grafana.com>
Co-authored-by: Leonard Gram <leo@xlson.com>
* Sign Drone's configuration YAML file
This is needed to protect the repository
* Update grabpl to 2.2.8
* Add comment about generating the drone.yml file
* Remove Revive standalone linter - use it via golangci-lint
* Add revive settings to ignore lint errors for now
* Small fix
* Update to grabpl v0.5.53
* Remove errorCode and warningCode from config
Refactor to allow shared contract between core and external backend plugins
allowing core backend data sources in Grafana to be implemented in same
way as an external backend plugin.
Use v0.67.0 of sdk.
Add tests for verifying plugin is restarted when process is killed.
Enable strict linting for backendplugin packages
grafana-plugin-model is legacy and is replaced by new backend
plugins SDK and architecture. Renderer is not part of SDK and
we want to keep it that way for now since it's highly unlikely there
will be more than one kind of renderer plugin.
So this PR adds support for renderer plugin v2.
Also adds support sending a Device Scale Factor parameter to the
plugin v2 remote rendering service and by that replaces #22474.
Adds support sending a Headers parameter to the plugin v2 and
remote rendering service which for now only include
Accect-Language header (the user locale in browser when using
Grafana), ref grafana/grafana-image-renderer#45.
Fixes health check json details response.
Adds image renderer plugin configuration settings in defaults.ini
and sample.ini.
Co-Authored-By: Arve Knudsen <arve.knudsen@gmail.com>
* build: update scripts go.(mod|sum)
* build: exclude gosec rule G108
Gosec rule G108: Profiling endpoint automatically exposed on /debug/pprof
is not applicable since we only enable /debug/pprof endpoint if -profile
flag is provided when starting grafana.
Prompted by the @slim-bean talk (thanks!). With that definition
`GO_FILES` var can be dynamically changed, that will be very helpful
in grafana enterprise repo
* Build: allow dynamically change docker image
Right now, only for couple DBs.
With the 2.1, you can do `${TAG:-5.6}` without needing the `.env` file, but
it seems it would be better for back-compat to support `.env` right now.
Fixes#17891
* Build: add prefix for removal process
* Docs: mention makefile command instead of the bash
Since we are moving to makefile for things and make command
simplifies some of the actions you have to perform to work with
different environments
* Docs: mention dynamic tags in the docs
* Docs: avoid name collision
Fixes#16107
Before this applied, go binary is referenced inconsistently in makefile.
Sometimes with go modules enabled and sometimes not, sometimes through
a variable and sometimes not.
This change makes all the references in makefile consistent
* Build: use golangci-lint as a make command
* Since gometalinter was deprecated in favor of golangci-lint so it was
replaced by it. Responsibilities held by the gometalinter was moved to
golangci-lint
* There was some changes in implementation (that was also mentioned in
the code comment) between the tools, which uncovered couple errors
in the code. Those issues were either solved or disabled by
the inline comments
* Introduce the golangci-lint config, to make their
configuration more manageable
* Build: replace backend-lint.sh script with make
Same idea as with the `gosec`, `revive` and etc.
Allows you to execute `make bra` as pinned local dependency instead
as global one. And therefore alleviates dependency installation issues.
README also updated - I was thinking to provide verbose installation instructions
for it. Since with that way we assume make is installed on the
developer system.
But it might be premature right now?
Otherwise changed data will not get refreshed to the previous
state.
Which I think is not evident (unless you are familar with docker-compose).
and therefore error-prone (at least it did for me)
* Chore: explore possibilities of using makefile
This is an exploratory commit - I wanted to see how
revive/gosec linters could be integrated with makefile and our build scripts.
Looks better then I expected :)
* Chore: make revive happy
Revive execution was not supplied with path, if you restore there is couple
errors that were popping up - so I fixed them
* Chore: make revive happy
- guide shamelessly stolen from prometheus/prometheus
- updates local interface of oauth exchange
- updates local impl of hclogger
- bump jaeger client version
closes#16088
Since we do not like some of the default golint rules,
this commit proposes to use https://github.com/mgechev/revive.
And potential revive speed-up should't hurt :).
Right now, presented config (./conf/revive.toml) is permissive,
we might improve it over time however. Fixes for found revive
issues in the code are very limited so it wouldn't be large to review.
Also in this commit:
* Add annotations for makefile commands and declare phony targets
* Rename "gometalinter" script and CI command to "lint"
since we are doing there a bit more then using gometalinter package
* Add Makefile rules to .editorconfig
* Documentation which mentioned "golint" replaced with revive
Fixes#16109
Ref #16160
- added `node_modules` as new target
- dependency on `package.json` and `yarn.lock` allows for quick `make
node_modules` after a branch change, which noops when the deps have
not changed
- also added `clean` target
