Commit Graph

123 Commits

Author SHA1 Message Date
Emil Tullstedt
e07fbeef60
Chore: Upgrade Go to 1.20.6 (#71440) 2023-07-12 15:32:40 +02:00
Agnès Toulet
f96ceb7804
Kinds: Fix report generation (#71048)
Fix kindsys report generation
2023-07-05 14:58:00 -04:00
OdysseyGuy
da54fe8ae8
Removed duplicate build flags. (#70653) 2023-06-27 10:09:44 +03:00
sam boyer
2bef09bfcb
All: Remove make gen-cue from main dev flow (#67838) 2023-05-04 16:40:35 +03:00
Sofia Papagiannaki
9812ad2cb2
Chore: Upgrade Go to 1.20.4 (#67748)
Upgrade Go to 1.20.4
2023-05-04 10:52:43 +03:00
Guilherme Caulada
614427c602
Build: Add optional build-args for COMMIT_SHA and BUILD_BRANCH (#66672)
* Remove .git from .dockerignore

* Revert "Remove .git from .dockerignore"

This reverts commit 5a4658f270.

* Add optional COMMIT_SHA and BUILD_BRANCH environment variables
2023-04-18 10:03:27 -03:00
Serge Zaitsev
391a192310
Chore: Try a different way to run integration tests (#66653)
* try a different way to run integration tests

* fix formatting

* clean test cache again

* use previous command

* playing around with random commands

* dont run tests in parallel yet

* use parallel option instead of gomaxprocs

* use same package set for redis/memcached; use p=1
2023-04-17 18:33:43 +02:00
Emil Tullstedt
202afb9041
Chore: Upgrade Go to 1.20.3 (#66264) 2023-04-14 10:01:14 +02:00
Sofia Papagiannaki
caac9838d8
Build: Fix Redis/Memcached integration tests (#64298)
* Build: Fix integration cache tests

* Allow REDIS_URL with scheme

* Reduce cache integration tests timeout to 5m

* Apply suggestion from code review

* Run redis/memcached integration tests in OSS pipelines

* Change redis image
2023-04-05 11:55:55 +03:00
Guilherme Caulada
ac03c0efed
Docker: Allow docker-build-full to work for BSD and GNU based systems (#65216)
* Replace tmp folder for tar stdin on build-docker-full

* Fix building linux/arm64 images using the Dockerfile
   Introduces JS_PLATFORM variable with linux/amd64 as default to allow building node packages.

* Revert unintended white line changes on Dockerfile
2023-03-23 12:03:25 -03:00
Serge Zaitsev
2d30f33ad8
Chore: Unite wire graphs for cli and server (#61143)
* Chore: unite wire graphs for cli and server

* remove cli wire once again

* try to fix runner wire injection

* unify runner initialisation
2023-03-17 16:36:39 +01:00
Todd Treece
c1f1cb8524
Chore: Remove apiserver from devenv sources (#64847) 2023-03-15 14:46:35 -04:00
Guilherme Caulada
69aba3eed3
Dockerfile: Allow building docker images locally for different editions (#63893)
* Allow to build different editions from dockerfile

* Fix missing argument on dockerfile
2023-02-28 17:18:00 -03:00
Emil Tullstedt
110b05a0c0
Chore: Upgrade Go to 1.20.1 and Alpine to 3.17 (#63639) 2023-02-27 14:50:32 +01:00
Emil Tullstedt
c72d18f5e6
Revert "Chore: Upgrade Go to 1.20.1 and Alpine to 3.17 (#63506)" (#63567)
This reverts commit 747ddf7b33.
2023-02-22 08:14:26 -05:00
Emil Tullstedt
747ddf7b33
Chore: Upgrade Go to 1.20.1 and Alpine to 3.17 (#63506) 2023-02-22 11:53:06 +01:00
Todd Treece
dea108a8b9
DevEnv: Add k8s apiserver (#63031)
Adds kind to bingo dependencies, and makes it available via `make devenv sources=apiserver`.
2023-02-07 20:34:05 -05:00
Sofia Papagiannaki
29d3e696f0
Swagger: Fix circular reference (#62673)
* Swagger: Fix circular reference

* gitignore intermediate swagger file

* Update specs

* Do not generate specs when building the backend
2023-02-02 13:18:04 +02:00
Sofia Papagiannaki
f8bd13ffad
Chore: Regenerate swagger and openapi specifications (#62561) 2023-01-31 11:00:45 +00:00
Jack Baldry
8379a5338c
CI: Lint starlark files with buildifier (#59157)
* Add verify-starlark build action that returns an error for starlark files with lint

Relies on `buildifier` tool.

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Add verify_starlark_step to PR pipeline

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Manually fetch buildifier in curl_image until a new build_image is created

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Format with buildifier

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Remove all unused variables retaining one unused function

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Use snake_case for variable

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Replace deprecated dictionary concatenation with .update() method

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Start adding docstrings for all modules and functions

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Prefer os.WriteFile as ioutil.WriteFile has been deprecated since go 1.16

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Attempt to document the behavior of the init_enterprise_step

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Document test_backend pipeline

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Document enterprise_downstream_step

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Document the pipeline utility function

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Document publish_images_step

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Document publish_images_steps

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Document enterprise2_pipelines function

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Add tags table for Starlark files.

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Document test_frontend

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Document windows function

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Add docstrings to verifystarlark functions

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Refactor error handling to be more clear and document complex behavior

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Split errors into execution errors and verification errors

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Document all other library functions

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Add local variables to TAGS

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Add blank line between all Args and Returns sections

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Fix new linting errors

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Lint new Starlark files

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Correct buildifier binary mv

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Document the need to set nofile ulimit to at least 2048

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Update build-container to include buildifier

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Ensure buildifier binary is executable

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Fix valid content test

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Simply return execution error

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Only check files rather than fixing them

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Use updated build-container with executable buildifier

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Test that context cancellation stops execution

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Simplify error handling

Return execution errors that short circuit WalkDir rather than
separately tracking that error.

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Remove fetching of buildifier binary now that it is in the build-container

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Use build image in verify-starlark step

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Use semver tag

The image is the same but uses a semver tag to make it clearer that
this is a forward upgrade from the old version.

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Use node 18 image with buildifier

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

---------

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
2023-01-30 09:27:11 +00:00
Selene
6f26333e96
Chore: Add cuefix hook (#61941)
* Add cue

* Add fix-cue target

* Add pre-commit hook

* fixup! Add fix-cue target

* Update the cue version

* Add datasources and panels folders

* Update cue files

Co-authored-by: Tania B <yalyna.ts@gmail.com>
2023-01-23 20:36:46 -05:00
sam boyer
492bdbd585
Kindsys: Rename mudball to common schemas, allow multiple input cue files (#61703)
* kindsys: Rename mudball to common schemas

* Update all core plugin imports

* Add missed table types

* Remove invalid JSON comment

* Remove coremodel dir from codeowners
2023-01-18 17:40:22 +00:00
Emil Tullstedt
bf574e98c8
Chore: Upgrade Go to 1.19.4 (#60794) 2022-12-29 13:03:56 +01:00
Joan López de la Franca Beltran
329fedb55d
Codegen: Fix mudball gen (#60812)
* Codegen: Fix mudball gen

* Fix Go build tags annotations
2022-12-29 12:48:27 +01:00
sam boyer
cc4d18f626
kindsys: Include composable kinds and planned maturity in report (#60477)
kindsys: Include composable kinds + planned maturity
2022-12-19 09:14:51 +01:00
Joan López de la Franca Beltran
61cd9eeb24
Kinds: Generate JSON report (#59742)
* Kinds: Generate JSON report

* Kinds: Add Validate/Write support for JSON report

* Kinds: Report format

* Kinds: Report new line

* Kinds: Use kindsys.SomeKindMeta to generate the report

* Move kinds report generation to the end

* Re-structure kinds report JSON output

* Make prettier to ignore kinds json report

* Minor on kinds report generation

* Fix toggles_gen test
2022-12-12 15:47:19 +01:00
Dan Cech
440d8a3d88
Build: Drone starlark file cleanup (#59919)
* format drone starlark files with black

* clean up unused params

* more simplification

* more cleanup

* more cleanup
2022-12-07 09:13:57 +02:00
Marcus Andersson
7f92f1df00
Schema: Refactor plugin code generation (#58901)
* wip

* wip

* almost there..

* wip - change so it can run.

* treelist is working.

* support CODEGEN_VERIFY env variable

* use log.fatal

* comment out old PluginTreeList code generation

* cleanup

* rename corelist package files

* fix makefile

* move pkg/codegen/pluggen.go to pkg/plugins/codegen

* copy and refactor files to pkg/plugins/codegen

* use pkg/plugins/codegen instead of pkg/codegen for core plugins code gen

* remove unneeded files

* remove unused code to resolve linting errors

* adapters first hack

* added flattener

* add back ignore build tags to go generate file

* cleaned up the code a bit.

* seems to work, needs to do some refactoring of the GoTypesJenns and TSTypesJenny.

* one more step, going to get upstream changes in this branch.

* working but need to run import tmpl in jenny_schemapath to have the proper imports.

* added header to generated files.

* added missing jenny.

* preventing plugins with multiple decls/schemas to insert multiple lines in corelist.

* fixed so we use Slot type from kindsys to detect if its group.

* adding a go jenny that only runs if the plugin has a backend.

* added version object to generated ts.

* generating the ts types with the same output as prior to this refactoring.

* removed code that is replaced by the jenny pattern.

* removed the go code that isn't used anymore.

* removed some more unused code and renamed pluggen to util_ts

* fixed linting issue.

* removed unused vars.

* use a jenny list postprocessor for header injection

* moved decl and decl_parser to pfs.

* removed the pre-pended header in the gotypes jenny since it is done in the postprocess.

* moved decl to pfs.

* removed unused template.

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2022-12-02 08:22:28 +01:00
Dan Cech
7436872dab
Build: Unified dockerfile for all builds (#59173)
* unified dockerfile for all builds

* update builder

* include docker commands and output in logs

* quiet docker build

* quiet gsutil
2022-11-28 16:18:53 -05:00
Dan Cech
9fec54df2d
Build: Streamline and sync dockerfiles (#58101)
* streamline and sync dockerfiles

* improve go dependency cacheability

* unify alpine and ubuntu Dockerfiles

* include glibc support in locally-built alpine images
2022-11-28 14:43:42 -05:00
Dan Cech
de99ce139c
Server: Switch from separate server & cli to a unified grafana binary (#58286)
* avoid the need for a second bulky binary for grafana-cli

* look for grafana-server in $PATH as well as same directory

* implement unified "grafana" command

* update dockerfiles, fix grafana-cli -v

* update packaging to work with single binary

- add wrapper scripts for grafana and grafana-server
- update and sync package files
- implement --sign flag of build package command
- stop packaging scripts folder, they are not useful for end users
- add support for --configOverrides in server command
- remove unused nfpm.yaml config file

* windows support
2022-11-22 11:53:43 -05:00
sam boyer
78f0340031
plugindef: Move pluginmeta out of coremodels as standalone thema lineage (#56765)
* Get pluginmeta mostly moved over to pkg/plugins/plugindef

* Remove dead func

* Fix up pfs, use sync.Once in plugindef

* Update to latest thema

* Chase Endec->Codec conversion in Thema

* Comments on slash header gen; use ToSlash

* Also generate JSON schema for plugindef

* Generate JSON Schema as well

* Fix slot loading from kindsys cue decls

* Remove unused vars

* skip generating plugin.schema.json for now

Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
2022-11-15 14:48:31 +01:00
Todd Treece
c090de9ed9
Chore: Move dev-dashboards jsonnet into separate directory (#58619) 2022-11-11 04:19:29 -05:00
sam boyer
07e5f8117f
Reconcile coremodels, entities, objects under new kind framework (#56492)
* Update thema to latest

* Deal with s/Library/*Runtime/

* Commit new, working results of codegen

* We like pointers now

* Always take runtime arg for NewBase()

* Sketchy handwavy pass at entity meta framework

* Little nibbles

* Update pkg/framework/coremodel/entityframework.cue

Co-authored-by: Artur Wierzbicki <wierzbicki.artur.94@gmail.com>

* Move file into new framework location

* Introduce loaders, Go code

* Complete rename to kind

* Flesh out framework, add svg/dashboard examples

* Cruft removal

* Remove generated kind go files from gitignore

* Refine maturity concept, add SlotKind

* Update embed and go deps

* Export PrefixWithGrafanaCUE

* Make the loader actually work, holy crap

* Many small tweaks to type.cue

* Add Apache 2 licensing exceptions for kinds

* Add new kinds dir, start of generator

* Roll back to earlier oapi-codegen

* Introduce new grafana-specific CUE loaders

* Introduce new tidy code generators framework

* Catch up kind framework with tinkering

* Add slices for the generators

* Add write/verify step to main generator

* Many renames

* Split up kind framework cue files

* Use kind.Decl within generated kinds

* Create kind.SomeDecl wrapper type to cache lineages

* Better names again

* Get one generated implemented, hopefully

* Copy dashboard schema into new kind.cue

* Small fixes to make the initial gen work

* Put svg kind in its new home

* Add generated Go dashboard type

* More renames and cleanups

* Add base kind registry and generator

* Stop blacklisting *_gen.go files

This is not the Go best practice, anyway. All we actually want to ignore
for enterprise is generated wire files.

* Change codegen output directories

pkg/kind -> pkg/kinds
pkg/registry/kindreg -> pkg/registry/corekind

* Rename pkg/framework/kind to pkg/kindsys

* Add core structured kind generator

* Add plural and machine names to kind spec

* Copy playlist over to kind system

* Consolidate kindsys files

* Add raw kind generator

* Update CODEOWNERS for kind framework

* Touch up comments a bit

* More docs tweaks

* Remove generated types to reduce noise for review

* Split each generator into its own file

* Rename Slot kind to Composable kind

* Add handwavy types for customkind loading

* Guard against init calls to framework loader

* First pass at doc on extending the kind system

* Improve attribute example in docs

* Fix wire imports

* Add basic TS types generator

* Fix composable kind category def

* No need for a separate file with generate directive

* Catch dashboard schema up

* Rename generator types to something saner and generic

* Make version configurable in ts/go generators

* Add CommonMeta to ease property access

* Add kindsys prop indicating whether lineage is group

* Put all kind categories back in a single file

* Finish with kindsys group props

* Refactor maturity progression per discussion

- Replace "committed" with "merged"
- All kindcats can use all maturity levels, at least for now

* Convert ts veneer index generator to modular system

* Move over to new jennywrites framework

* Strip down old coremodel generator

* Use public version of jennywrites

* Pull latest thema

* Commit generated Go types

* Add header injection postprocessor

* Move sdboyer/jennywrites to grafana/codejen

* Tweak header output

* Remove dashboard and playlist coremodels

* Fix up backend dashboards devenv test

* Fix TS import patterns to new gen filename

* Update internal imports, remove coremodel registry

* Fix compilation errors, wire generation

* Export and replace the prefix dropper

* More Go struct and field name changes

* Last name fixes, hopefully

* Fix lint errors

* Last lint error

Co-authored-by: Artur Wierzbicki <wierzbicki.artur.94@gmail.com>
2022-11-10 12:36:40 -08:00
Todd Treece
e105b5a107
Chore: Add jsonnet library for devenv dashboards (#57928) 2022-11-03 11:04:39 -04:00
Josh Hunt
2e16d5499e
Dashboard: Add dashboard validation warning to save drawer (#55732)
* add api route for validating a dashboard json

* add feature flag for showDashboardValidationWarnings

* tidy up

* comments and messages

* swagger specs

* fix typo

* more swagger

* tests!

* tidy test a little bit

* no more ioutil

* api will return different status code depending on validation error

* clean up

* handle 4xx errors

* remove console.log

* fix backend tests

* tidy up

* Swagger: Exclude alpha endpoints

Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>
2022-10-14 09:51:05 -04:00
Sofia Papagiannaki
4cdbfb7606
Chore: Timeout integration package tests faster (#56287) 2022-10-10 09:27:04 +02:00
Emil Tullstedt
b287047052
Chore: Upgrade Go to 1.19.1 (#54902)
* WIP

* Set public_suffix to a pre Ruby 2.6 version

* we don't need to install python

* Stretch->Buster

* Bump versions in lib.star

* Manually update linter

Sort of messy, but the .mod-file need to contain all dependencies that
use 1.16+ features, otherwise they're assumed to be compiled with
-lang=go1.16 and cannot access generics et al.

Bingo doesn't seem to understand that, but it's possible to manually
update things to get Bingo happy.

* undo reformatting

* Various lint improvements

* More from the linter

* goimports -w ./pkg/

* Disable gocritic

* Add/modify linter exceptions

* lint + flatten nested list

Go 1.19 doesn't support nested lists, and there wasn't an obvious workaround.
https://go.dev/doc/comment#lists
2022-09-12 12:03:49 +02:00
linoman
3f0beee362
Update README build references (#54394) 2022-08-29 16:36:06 +02:00
lean.dev
e8e7b59bb0
Implementing OpenApi 3 specs (converted from Swagger 2) (#53243)
* Adding OpenApi target to makefile, to covert swagger spec into OpenAPI 3
* Adding endpoint to server swaggerui with new openapi3 specs
* Passing output file as parameter for OpenApi3 specs
* Implement workaround for missing host and prefix domain
2022-08-04 13:51:12 -03:00
Sofia Papagiannaki
3713a193d8
Chore: Fix Makefile target dependency (#53165) 2022-08-02 19:42:48 +03:00
Sofia Papagiannaki
3f58e8ec2a
Chore: Cleanup Makefile (#53037)
Remove redundant reference to deleted folder pkg/api/docs/definitions
2022-08-01 16:31:49 +03:00
Sofia Papagiannaki
7ba076de10
Chore: Move swagger definitions to the handlers (#52643) 2022-07-27 09:54:37 -04:00
Sofia Papagiannaki
f7c5eceb21
Chore: Update to the latest grafana-plugin-sdk-go and more swagger fixes (#52445)
* Fix get legacy alert response

* Swagger: Fix get folder by UID response

* Fix conflicting swagger model Alert

Reanme legacy alerting swagger model to LegacyAlert to differentiate it
from the prometheus Alert

* Bump grafana-plugin-sdk-go

* Fix get folder response

* Use go-swagger command for merging the specifications and remove merge_specs script
2022-07-20 16:09:42 +03:00
Kat Yang
ab8ad1bb42
Chore: Add new go test commands for unit, integration, and pro tests to makefile (#51202)
* Chore: Add new go test commands for unit, integration tests to makefile

* Add test-go-unit and test-go-integration targets as dependencies of the test-go target

* Add makefile target for mysql & postgres backends

* Set GRAFANA_TEST_DB variable for the xargs postgres command

Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>

* Set GRAFANA_TEST_DB variable for the xargs mysql command

Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>

* Use postgres_tests as source

Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>

* Set postgres_tests as source

Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>

* Clean test cache before postgres and mysql integration test makefile commands

Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>
2022-07-19 11:04:38 +03:00
Sofia Papagiannaki
a5924315f8
API: Fix failure to generate swagger specification due to missing binary (#51551)
* Fix swagger generation

Add installing binary as dependency to the target

* Some more fixes
2022-06-30 09:58:07 +03:00
Sofia Papagiannaki
1399ab50b3
API: Universal swagger generation (#51033) 2022-06-27 10:54:31 +03:00
ying-jeanne
62531715b3
Chore: remove bra and scripts/go from grafana (#51130) 2022-06-21 11:08:08 +02:00
ying-jeanne
4489f331b8
Chore: Use the bingo built golangci-lint (#51048)
* use the bingo built golangci-lint

* remove grapl usage

* add dependancy
2022-06-17 19:46:20 +02:00
sam boyer
c8239a3a73
dev: Ignore generated go files in bra (#50889) 2022-06-15 16:33:40 +01:00