* bumping grafanas go version to 1.21.5
* adding how to upgrade go doc
* removing the whole ci-build and corresponding pipelines
* clarifying that we have to run make drone
* fixing starlark linter and removing unused code
* Adding note about enterprise
* trying to change the underyling image to avoid musl issues
* only need to golang one
* remove GRN and switch tenant to namespace
* clean up remaining references
* simplify and remove inconsistency in With* parameters
* parse listing keys so we can use db index
* bump the schema version
---------
Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
* Alerting: Introduce a Mimir client as part of the Remote Alertmanager
This is our first attempt at making Grafana communicate use Mimir as a backend - it uses a new set of APIs that we've developed on the Mimir side to upload the grafana configuration and alertmanager state so that it can then be ported over.
Codewise, we've introduced a couple of things:
A client to isolate in its own package all the communication that happens with Mimir
A few changes to the remote/alertmanager to include uploading the configuration and state when it starts
A few refactors that align a bit better with the design approach that we're thinking
An integration tests again these newly developed APIs using a custom image
---------
Signed-off-by: gotjosh <josue.abreu@gmail.com>
Co-authored-by: Santiago <santiagohernandez.1997@gmail.com>
* Alerting: Use Mimir image to run integration tests for the remote Alertmanager
* skip integration test when running all tests
* skipping integration test when no Alertmanager URL is provided
* fix bad host for mimir_backend
* remove basic auth testing until we have an nginx image in our CI
* Switch from husky+precommit to lefthook
Migrate from husky/precommit to lefthook
h
old husky precommit
lefthook so far
remove husky folder
switch to new lefthook package
Add postinstall script to clean up after husky
preinstall to remove husky config
package.json
package.json
package.json
script
v
Reduce lefthook output
Cleanup
testing
reduce output
testing betterer
testing betterer
testing betterer
testing betterer
testing betterer
testing betterer
testing betterer
skip skipping execution info
small cleanup
remove comment from testing
clean up old husky hooks in .git/hooks path
run scripts in parallel
update codeowners
don't auto-install lefthook
install lefthook with make command
fix yarn checksum
* update codeowners
* don't use source because it isn't available in ubuntu/debian's /bin/sh
* Ensure lefthook commits files fixed by precommit hooks
* add comment to lefthook.rc explaining what it does
* add i18n:pseudo precommit
* Don't create .husky/safe-to-delete file anymore
* cleanup old lint-staged config
* contribute docs
* update lefthook to 1.4.8
* Move frontend encouragement docs
* rewrite husky cleanup script in bash so we can run it with make
* Make old husky precommit script reject commits until husky is removed
* log precommit warning for everyone
* fix package.json
* run lefthook hooks from old husky hook
* run lefthook hooks from old husky hook
* comments
* codeowners
* codeowners
* try a different way to run integration tests
* fix formatting
* clean test cache again
* use previous command
* playing around with random commands
* dont run tests in parallel yet
* use parallel option instead of gomaxprocs
* use same package set for redis/memcached; use p=1
* Replace tmp folder for tar stdin on build-docker-full
* Fix building linux/arm64 images using the Dockerfile
Introduces JS_PLATFORM variable with linux/amd64 as default to allow building node packages.
* Revert unintended white line changes on Dockerfile
* Add verify-starlark build action that returns an error for starlark files with lint
Relies on `buildifier` tool.
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Add verify_starlark_step to PR pipeline
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Manually fetch buildifier in curl_image until a new build_image is created
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Format with buildifier
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Remove all unused variables retaining one unused function
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Use snake_case for variable
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Replace deprecated dictionary concatenation with .update() method
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Start adding docstrings for all modules and functions
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Prefer os.WriteFile as ioutil.WriteFile has been deprecated since go 1.16
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Attempt to document the behavior of the init_enterprise_step
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Document test_backend pipeline
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Document enterprise_downstream_step
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Document the pipeline utility function
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Document publish_images_step
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Document publish_images_steps
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Document enterprise2_pipelines function
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Add tags table for Starlark files.
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Document test_frontend
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Document windows function
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Add docstrings to verifystarlark functions
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Refactor error handling to be more clear and document complex behavior
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Split errors into execution errors and verification errors
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Document all other library functions
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Add local variables to TAGS
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Add blank line between all Args and Returns sections
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Fix new linting errors
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Lint new Starlark files
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Correct buildifier binary mv
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Document the need to set nofile ulimit to at least 2048
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Update build-container to include buildifier
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Ensure buildifier binary is executable
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Fix valid content test
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Simply return execution error
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Only check files rather than fixing them
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Use updated build-container with executable buildifier
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Test that context cancellation stops execution
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Simplify error handling
Return execution errors that short circuit WalkDir rather than
separately tracking that error.
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Remove fetching of buildifier binary now that it is in the build-container
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Use build image in verify-starlark step
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Use semver tag
The image is the same but uses a semver tag to make it clearer that
this is a forward upgrade from the old version.
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Use node 18 image with buildifier
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
---------
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Kinds: Generate JSON report
* Kinds: Add Validate/Write support for JSON report
* Kinds: Report format
* Kinds: Report new line
* Kinds: Use kindsys.SomeKindMeta to generate the report
* Move kinds report generation to the end
* Re-structure kinds report JSON output
* Make prettier to ignore kinds json report
* Minor on kinds report generation
* Fix toggles_gen test
* wip
* wip
* almost there..
* wip - change so it can run.
* treelist is working.
* support CODEGEN_VERIFY env variable
* use log.fatal
* comment out old PluginTreeList code generation
* cleanup
* rename corelist package files
* fix makefile
* move pkg/codegen/pluggen.go to pkg/plugins/codegen
* copy and refactor files to pkg/plugins/codegen
* use pkg/plugins/codegen instead of pkg/codegen for core plugins code gen
* remove unneeded files
* remove unused code to resolve linting errors
* adapters first hack
* added flattener
* add back ignore build tags to go generate file
* cleaned up the code a bit.
* seems to work, needs to do some refactoring of the GoTypesJenns and TSTypesJenny.
* one more step, going to get upstream changes in this branch.
* working but need to run import tmpl in jenny_schemapath to have the proper imports.
* added header to generated files.
* added missing jenny.
* preventing plugins with multiple decls/schemas to insert multiple lines in corelist.
* fixed so we use Slot type from kindsys to detect if its group.
* adding a go jenny that only runs if the plugin has a backend.
* added version object to generated ts.
* generating the ts types with the same output as prior to this refactoring.
* removed code that is replaced by the jenny pattern.
* removed the go code that isn't used anymore.
* removed some more unused code and renamed pluggen to util_ts
* fixed linting issue.
* removed unused vars.
* use a jenny list postprocessor for header injection
* moved decl and decl_parser to pfs.
* removed the pre-pended header in the gotypes jenny since it is done in the postprocess.
* moved decl to pfs.
* removed unused template.
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
* streamline and sync dockerfiles
* improve go dependency cacheability
* unify alpine and ubuntu Dockerfiles
* include glibc support in locally-built alpine images
* avoid the need for a second bulky binary for grafana-cli
* look for grafana-server in $PATH as well as same directory
* implement unified "grafana" command
* update dockerfiles, fix grafana-cli -v
* update packaging to work with single binary
- add wrapper scripts for grafana and grafana-server
- update and sync package files
- implement --sign flag of build package command
- stop packaging scripts folder, they are not useful for end users
- add support for --configOverrides in server command
- remove unused nfpm.yaml config file
* windows support
* Get pluginmeta mostly moved over to pkg/plugins/plugindef
* Remove dead func
* Fix up pfs, use sync.Once in plugindef
* Update to latest thema
* Chase Endec->Codec conversion in Thema
* Comments on slash header gen; use ToSlash
* Also generate JSON schema for plugindef
* Generate JSON Schema as well
* Fix slot loading from kindsys cue decls
* Remove unused vars
* skip generating plugin.schema.json for now
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
* Update thema to latest
* Deal with s/Library/*Runtime/
* Commit new, working results of codegen
* We like pointers now
* Always take runtime arg for NewBase()
* Sketchy handwavy pass at entity meta framework
* Little nibbles
* Update pkg/framework/coremodel/entityframework.cue
Co-authored-by: Artur Wierzbicki <wierzbicki.artur.94@gmail.com>
* Move file into new framework location
* Introduce loaders, Go code
* Complete rename to kind
* Flesh out framework, add svg/dashboard examples
* Cruft removal
* Remove generated kind go files from gitignore
* Refine maturity concept, add SlotKind
* Update embed and go deps
* Export PrefixWithGrafanaCUE
* Make the loader actually work, holy crap
* Many small tweaks to type.cue
* Add Apache 2 licensing exceptions for kinds
* Add new kinds dir, start of generator
* Roll back to earlier oapi-codegen
* Introduce new grafana-specific CUE loaders
* Introduce new tidy code generators framework
* Catch up kind framework with tinkering
* Add slices for the generators
* Add write/verify step to main generator
* Many renames
* Split up kind framework cue files
* Use kind.Decl within generated kinds
* Create kind.SomeDecl wrapper type to cache lineages
* Better names again
* Get one generated implemented, hopefully
* Copy dashboard schema into new kind.cue
* Small fixes to make the initial gen work
* Put svg kind in its new home
* Add generated Go dashboard type
* More renames and cleanups
* Add base kind registry and generator
* Stop blacklisting *_gen.go files
This is not the Go best practice, anyway. All we actually want to ignore
for enterprise is generated wire files.
* Change codegen output directories
pkg/kind -> pkg/kinds
pkg/registry/kindreg -> pkg/registry/corekind
* Rename pkg/framework/kind to pkg/kindsys
* Add core structured kind generator
* Add plural and machine names to kind spec
* Copy playlist over to kind system
* Consolidate kindsys files
* Add raw kind generator
* Update CODEOWNERS for kind framework
* Touch up comments a bit
* More docs tweaks
* Remove generated types to reduce noise for review
* Split each generator into its own file
* Rename Slot kind to Composable kind
* Add handwavy types for customkind loading
* Guard against init calls to framework loader
* First pass at doc on extending the kind system
* Improve attribute example in docs
* Fix wire imports
* Add basic TS types generator
* Fix composable kind category def
* No need for a separate file with generate directive
* Catch dashboard schema up
* Rename generator types to something saner and generic
* Make version configurable in ts/go generators
* Add CommonMeta to ease property access
* Add kindsys prop indicating whether lineage is group
* Put all kind categories back in a single file
* Finish with kindsys group props
* Refactor maturity progression per discussion
- Replace "committed" with "merged"
- All kindcats can use all maturity levels, at least for now
* Convert ts veneer index generator to modular system
* Move over to new jennywrites framework
* Strip down old coremodel generator
* Use public version of jennywrites
* Pull latest thema
* Commit generated Go types
* Add header injection postprocessor
* Move sdboyer/jennywrites to grafana/codejen
* Tweak header output
* Remove dashboard and playlist coremodels
* Fix up backend dashboards devenv test
* Fix TS import patterns to new gen filename
* Update internal imports, remove coremodel registry
* Fix compilation errors, wire generation
* Export and replace the prefix dropper
* More Go struct and field name changes
* Last name fixes, hopefully
* Fix lint errors
* Last lint error
Co-authored-by: Artur Wierzbicki <wierzbicki.artur.94@gmail.com>
* add api route for validating a dashboard json
* add feature flag for showDashboardValidationWarnings
* tidy up
* comments and messages
* swagger specs
* fix typo
* more swagger
* tests!
* tidy test a little bit
* no more ioutil
* api will return different status code depending on validation error
* clean up
* handle 4xx errors
* remove console.log
* fix backend tests
* tidy up
* Swagger: Exclude alpha endpoints
Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>
* WIP
* Set public_suffix to a pre Ruby 2.6 version
* we don't need to install python
* Stretch->Buster
* Bump versions in lib.star
* Manually update linter
Sort of messy, but the .mod-file need to contain all dependencies that
use 1.16+ features, otherwise they're assumed to be compiled with
-lang=go1.16 and cannot access generics et al.
Bingo doesn't seem to understand that, but it's possible to manually
update things to get Bingo happy.
* undo reformatting
* Various lint improvements
* More from the linter
* goimports -w ./pkg/
* Disable gocritic
* Add/modify linter exceptions
* lint + flatten nested list
Go 1.19 doesn't support nested lists, and there wasn't an obvious workaround.
https://go.dev/doc/comment#lists
* Adding OpenApi target to makefile, to covert swagger spec into OpenAPI 3
* Adding endpoint to server swaggerui with new openapi3 specs
* Passing output file as parameter for OpenApi3 specs
* Implement workaround for missing host and prefix domain
* Fix get legacy alert response
* Swagger: Fix get folder by UID response
* Fix conflicting swagger model Alert
Reanme legacy alerting swagger model to LegacyAlert to differentiate it
from the prometheus Alert
* Bump grafana-plugin-sdk-go
* Fix get folder response
* Use go-swagger command for merging the specifications and remove merge_specs script
* Chore: Add new go test commands for unit, integration tests to makefile
* Add test-go-unit and test-go-integration targets as dependencies of the test-go target
* Add makefile target for mysql & postgres backends
* Set GRAFANA_TEST_DB variable for the xargs postgres command
Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>
* Set GRAFANA_TEST_DB variable for the xargs mysql command
Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>
* Use postgres_tests as source
Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>
* Set postgres_tests as source
Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>
* Clean test cache before postgres and mysql integration test makefile commands
Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>
* coremodel: Generate static registry
* Actually make codegen work
Also, remove the per-coremodel assignability test from generator set.
* Make wire gen depend on cue gen
This is necessary now that we're generating a wire set as part of
coremodel registry generation.
* Add wire inject bits to http server
* s/staticregistry/registry/
* move to static and dynamic wording
* Move registry type into registry package
* Use static registry in http handler
* Oi comments
* Fix swagger generation targer using mac
Validate using the swagger binarry instead of the docker container
* API: Make swagger specification more consistent
Fix operation parameter ordering
* Add go code generator for coremodels
* Just generate the entire coremodel for now
Maybe we'll need more flexibility as more coremodels are added, but for
now this is fine.
* Add note on type comment about stability, grodkit
* Remove local replace directive for thema
* Generate typescript from coremodel
* Update pkg/coremodel/dashboard/addenda.go
Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
* Update cuetsy to new release
* Update thema to latest
* Fix enum generation for FieldColorModeId
* Put main generated object at the end of the file
* Tweaks to generated Go output
* Retweak back to var
* Add generated coremodel test
* Remove local replace statement again
* Add Make target and call into cuetsy cmd from gen
* Rename and comment linsrc for readability
* Move key codegen bits into reusable package
* Move body of cuetsifier into codegen pkg
Also genericize the diffing output into reusable WriteDiffer.
* Refactor coremodel generator to use WriteDiffer
* Add gen-cue step to CI
* Whip all the codegen automation into shape
* Add simplistic coremodel canonicality controls
* Remove erroneously committed test
* Bump thema version
* Remove dead code
* Improve wording of non-canonicality comment
Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
* Support for documenting stable vs unstable alerting routes
* empty commit, restart drone
* Touch-up references in root makefile and drop trailing escape newline
* Rebase and regenerate
* Extend README with docs for this change
* API: Using go-swagger for extracting OpenAPI specification from source code
* Merge Grafana Alerting spec
* Include enterprise endpoints (if enabled)
* Serve SwaggerUI under feature flag
* Fix building dev docker images
* Configure swaggerUI
* Add missing json tags
Co-authored-by: Ying WANG <ying.wang@grafana.com>
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
* Set up Wire build graph
* Remove enterprise Wire set
* Move runner package outside commands
* Update Makefile (gen-go path)
* Minor prettier fix
* Include new Wire enterprise file into .gitignore
* Update Wire deps
* Update the grabpl version
Co-authored-by: Dan Cech <dcech@grafana.com>
* Force use drone-cli 1.4.0
* Small modifications to prevent null generation
* Sign drone
* Use bingo to declare drone-cli version
* Update message in Makefile
* Update to latest grabpl version
Fixes#30144
Co-authored-by: dsotirakis <sotirakis.dim@gmail.com>
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
Co-authored-by: Ida Furjesova <ida.furjesova@grafana.com>
Co-authored-by: Jack Westbrook <jack.westbrook@gmail.com>
Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
Co-authored-by: Leon Sorokin <leeoniya@gmail.com>
Co-authored-by: Andrej Ocenas <mr.ocenas@gmail.com>
Co-authored-by: spinillos <selenepinillos@gmail.com>
Co-authored-by: Karl Persson <kalle.persson@grafana.com>
Co-authored-by: Leonard Gram <leo@xlson.com>
* Sign Drone's configuration YAML file
This is needed to protect the repository
* Update grabpl to 2.2.8
* Add comment about generating the drone.yml file
* Remove Revive standalone linter - use it via golangci-lint
* Add revive settings to ignore lint errors for now
* Small fix
* Update to grabpl v0.5.53
* Remove errorCode and warningCode from config
Refactor to allow shared contract between core and external backend plugins
allowing core backend data sources in Grafana to be implemented in same
way as an external backend plugin.
Use v0.67.0 of sdk.
Add tests for verifying plugin is restarted when process is killed.
Enable strict linting for backendplugin packages
