Commit Graph

171 Commits

Author SHA1 Message Date
Kevin Minehart
cf6bed7ae5
Trigger downstream from grafana and not the mirror (#84381) 2024-03-13 12:21:08 -05:00
Sofia Papagiannaki
b8d8662bd9
Swagger: Re-generate the enterprise specification if enterprise is cloned (#81730)
* Swagger: Re-generate the enterprise specification if enterprise is cloned successfully

* API change to trigger the swagger CI step execution

* Swagger: Silence logs
2024-03-07 12:07:35 +02:00
Alexander Weaver
7a171fd14a
Regenerate openapidocs at 1.21.8 to match ci (#84037)
* Regenerate openapidocs at 1.21.8 to match ci

* Adjust trigger to work on the actual outputted files

* Also put go.mod and go.sum in the triggers

* manually fix

* Make an arbitrary change rather than touching the trigger to force a run

* Drop all triggers - run all the time

* Print diff - taken from @papagian's PR

* Manual fixes to swagger doc

---------

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
2024-03-06 16:08:45 -06:00
Erik Sundell
3e456127cb
E2E: Add plugin-e2e scenario verification tests (#79969)
* add playwright test and plugin-e2e

* run tests in ci

* add ds config tests

* add panel edit tests

* add annotation test

* add variable edit page tests

* add explore page tests

* add panel plugin tests

* add readme

* remove comments

* fix broken test

* remove user.json

* remove newline in starlark

* fix lint issue

* ignore failure of playwright tests

* update code owners

* add detailed error messages in every expect

* update message frame

* fix link

* upload report to gcp

* echo url

* add playwright developer guide

* bump plugin-e2e

* add custom provisioning dir

* update plugin-e2e

* remove not used imports

* fix typo

* minor fixes

* use latest version of plugin-e2e

* fix broken link

* use latest plugin-e2e

* add feature toggle scenario verification tests

* bump version

* use auth file from package

* fix type error

* add panel data assertions

* rename parent dir and bump version

* fix codeowners

* reset files

* remove not used file

* update plugin-e2e

* separate tests per role

* pass prov dir

* skip using provisioning fixture

* wip

* fix permission test

* move to e2e dir

* fix path to readme

* post comment with report url

* format starlark

* post comment with report url

* post comment with report url

* fix token

* make test fail

* fix exit code

* bump version

* bump to latest plugin-e2e

* revert reporting message

* remove comments

* readding report comment

* change exit code

* format starlark

* force test to fail

* add new step that posts comment

* fix link

* use latest playwright image

* fix failing test

* format starlark

* remove unused fixture

Co-authored-by: Marcus Andersson <marcus.andersson@grafana.com>

---------

Co-authored-by: Marcus Andersson <marcus.andersson@grafana.com>
2024-02-23 12:39:30 +01:00
Victor Marin
9dc015b595
Re-add arm32 builds (#79496)
* Re-add arm32 builds

* update grabpl version

* make drone

* make drone

* format drone
2023-12-15 16:56:22 +02:00
Timur Olzhabayev
4991e71b85
Chore: Upgrade Go to 1.21.5 (#79329)
* bumping grafanas go version to 1.21.5

* adding how to upgrade go doc

* removing the whole ci-build and corresponding pipelines

* clarifying that we have to run make drone

* fixing starlark linter and removing unused code

* Adding note about enterprise

* trying to change the underyling image to avoid musl issues

* only need to golang one
2023-12-15 10:36:49 +01:00
Kevin Minehart
f6d3238505
CI: Fix race condition when building docker on main (#77504)
* build docker after packages are updated

* use my branch for main pipelines for testing

* use my branch for main pipelines for testing

* use main instead now

* formatting
2023-11-01 18:25:49 +02:00
Kevin Minehart
8a5d4c4c6e
CI: Update RGM steps to use the artifacts command (#77470)
* update rgm steps to use artifacts subcmd

* format-drone

* make drone
2023-10-31 22:52:09 +02:00
Niki Manoledaki
9a0af13dbc
Remove pr-swagger-gen dependency on clone-enterprise (#77217)
clone-enterprise step can fail in swagger-gen pipeline
2023-10-26 12:55:56 -04:00
Kat Yang
bbcc81405b
Chore: Add CI Pipeline to generate Grafana's OpenAPI specification (#75393)
* chore: move over initial changes from sofia's pr #58029

* chore: remove go_image

* chore: begin removing edition, remove unused imports

* chore: remove edition from swagger_gen.star and generate .drone.yml

* chore: regen drone.yml

* fix: fix order of load statements

* fix: try #2 fix order of load statements

* linter fixes

* chore: add doc comment explaining purpose of new clone_pr_branch step

* fix: add placeholder documentation for ver_mode arg

* attempt #1 to import and use clone_enterprise_step_pr

* Update scripts/drone/pipelines/swagger_gen.star

Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>

* attempt #2 to import and use clone_enterprise_step_pr

* fix: fix drone lint err invalid or unknown step dependency

* fix: regen hmac to make drone run

* fix: fix drone lint err

* fix: update swagger-clean cmd in step

* attempt to return non zero exit code

* test to see if pipeline fails

* fix: add git to clone pr branch step

* fix: add git and make to swagger-gen step

* try to rerun drone

* debug: figure out why cannot find make swagger-clean

* debug: see if cd grafana/grafana fixes things

* debug: undo cd grafana/grafana

* debug: add more logging statements

* debug: try and remove cd grafana in swagger-gen

* debug: removed grafana after clone statement; add debug before cloning

* fix: remove disable clone

* regen specs to see if swagger-gen step passes now

* add descriptive error message to swagger-gen step

* remove api-spec.json from .gitignore

* revert backend change, regen spec

* add back backend change, regen specs

* Update scripts/drone/pipelines/swagger_gen.star

Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>

* Update scripts/drone/pipelines/swagger_gen.star

Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>

* revert gitignore change, set enterprise source to drone source branch

* chore: remove unused variable, change committish var name to source

* testing functionality: make a new BE change without gen spec, pipeline should fail

* test functionality: does removing err msg cause step to fail properly

* test functionality: add back err msg and && after

* chore: remove debug statements from swagger gen step

* chore: remove debug lines from clone_pr_branch step

* test functionality: regen specs, swagger_gen step should pass

* test funcionality: regen specs again ????

* chore: update swagger-gen step err msg

* test functionality: make BE change dont regen spec, swagger gen should fail

* test functionality: regen the specs, swagger-gen should pass

* chore: revert test BE change, regen spec

* chore: remove unused clone step

* chore: regen drone.yml

---------

Co-authored-by: Timur Olzhabayev <timur.olzhabayev@grafana.com>
Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>
2023-10-24 13:01:04 -04:00
Horst Gutmann
bd2b4e956b
CI: rgm-package must wait for update-package-json-version in main pip… (#77022)
CI: rgm-package must wait for update-package-json-version in main pipeline
2023-10-24 11:52:14 +03:00
Dimitris Sotirakis
faa22b8f20
CI: Remove 0.0.0-test tag event (#76662)
Remove 0.0.0-test tag event
2023-10-17 10:20:14 +03:00
Santiago
726260b2f6
CI: Add test Mimir backend image for integration tests (#76051)
* test mimir image drone

* add mimir service to services.star

* change name for mimir step

* change remote alertmanager step names, add commands for integration tests

* add  tag to mimir image
2023-10-05 15:09:26 -03:00
Dimitris Sotirakis
c08606d2c5
Bug: Exclude 32-bit arm builds (#75448)
* Exclude 32 bit arm builds

# Conflicts:
#	pkg/build/config/versions.go

* Bump grabpl - remove arm/v7 from build.star

# Conflicts:
#	.drone.yml
2023-09-29 12:33:01 -05:00
Kat Yang
24a1f12826
Chore: Add new drone step to validate OpenAPI specification (#75034)
* chore: add new drone step to validate openapi specification

* fix: update make before running make command
2023-09-20 11:46:15 -04:00
Kevin Minehart
c7ef496750
CI: Add a step on main pipelines to temporarily update the package.json v… (#75018)
* Add a step on main pipelines to temporarily update the package.json version to include the build number
2023-09-18 18:53:42 -03:00
Kevin Minehart
69737cba6d
CI: Fixes for various main steps (#74768)
Fixes for various main steps
2023-09-13 00:44:33 +03:00
Kevin Minehart
9221822dfe
CI: fix docker tag format on main branch and missing dependency on ci-frontend-metrics.sh (#74753)
* On main, set the docker tag format to be the same as it is in PRs

* Install bash when trying to push frontend metrics
2023-09-12 19:05:02 +03:00
Kevin Minehart
915f198ecb
Refactor starlark to remove references to the build_image (#74624)
* Refactor starlark to remove references to the build_image
2023-09-12 12:19:06 -03:00
Kevin Minehart
6669d0a704
CI: Update secret refs to vault ones; update downstream repo origin (#72515)
Update secret refs to vault ones; update downstream repo origin
2023-07-28 20:36:31 +03:00
Kevin Minehart
c3d6f795ea
Only trigger downstream builds on Grafana (#72356) 2023-07-26 15:26:58 +02:00
Agnès Toulet
ac445a25d5
Kindsys: update generated doc and add check in CI (#71871)
* clean up docs

* fix Windows path issue

* add verify-gen-cue to doc pipeline

* fix linting error
2023-07-20 11:48:12 +02:00
Kevin Minehart
6d6cd2011e
CI: Only run main pipeliens on Grafana (#71949)
Only run main pipeliens on Grafana
2023-07-19 14:03:07 +00:00
Dimitris Sotirakis
b13939b9af
[WIP] CI: Removes enterprise specific pipelines and steps (#70815)
* Removes enterprise specific pipelines and steps (#123)

* Comment out enterprise related pipelines and steps

* Suppress unused variable warning

* Removes all edition arguments

* Remove leftover comments

* Remove redundant oss on pipelines and steps names

* Remove leftover unused variable

* Remove leftovers

* Remove pipeline dependencies

* Rename pipelines

* Fix starlark

---------

Co-authored-by: dsotirakis <dimitrios.sotirakis@grafana.com>
(cherry picked from commit 642a81ba75e79138246797302aba5c35575f030d)

# Conflicts:
#	.drone.yml
#	scripts/drone/steps/lib.star

# Conflicts:
#	.drone.yml

* Add editions for static assets

# Conflicts:
#	.drone.yml

# Conflicts:
#	.drone.yml
2023-07-14 14:18:09 +03:00
Emil Tullstedt
7a9847e196
CI: Provide a Drone promotion to build the build-container (#71133) 2023-07-11 13:08:05 +02:00
Sofia Papagiannaki
e82d437e0e
Drone: Enable running go benchmarks on demand (#70359)
* Add benchmark

* Run pkg/api benchmarks in CI

* Conditionally run benchmarks for provided go packages

* Bypass fork check for promotes
2023-07-07 11:55:01 +03:00
Dimitris Sotirakis
7f55ba9c6e
CI: Cleanup - Remove security related steps (#70788)
* Remove security related steps

* More cleanup
2023-06-28 11:12:47 +03:00
Dimitris Sotirakis
91272ee4f9
CI: Add CI check for what's new link (#70046)
* Add whatsnewchecker

* Add whatsnewchecker_test

* Small fixes

* Add step in CI

* Fix lint

* Fix starlark

* t.Cleanup instead of separate func

* Skip check for test tags
2023-06-14 13:51:01 +03:00
Dimitris Sotirakis
fa70fba0e3
Security Scans: Exclude windows container scans (#69977)
* Exclude windows container scans

* Fixes according to reviewer's comments
2023-06-13 10:38:18 +03:00
Dimitris Sotirakis
2cda971796
Security Scans: Add trivy scans to every docker image used for building/testing/publishing (#69911)
* Created images.star

* Fix typo

* Add cronjobs for build-images
2023-06-12 16:41:18 +03:00
Tobias Skarhed
134839d701
CI: Add i18n verification step (#69384)
* Add i18n verification step

* Add check for extracted translations

* Run extract

* Add newline for pseudo

* Format JSON output with prettier

* Print file diff

* Actually run make drone

* Only deo git diff on locales

* Update error message

* Verify that step fails on dynamic phrases

* Verify that it passes when fixed

* Verify that dynamic phrase fails, for real this time

* Extract error messages

* Change echo

* Fix string format

* Try double escaping

* Escape for Drone substitution

* Update character escape

* Remove dynamic phrase

* Verify multiple keys

* Remove double keys

* Readd en locale because of reasons

* Undo changes

* Format lint_frontend.star

* Update error message

* Update .drone.yml

* Add quotes for echo

* Verify fail to extract

* Fix diff stat command

* Reset footer changes
2023-06-06 09:29:04 +02:00
Kat Yang
293993ac09
CI: Add validate-modfile step in drone (#69232)
* chore: test drone step

* revert changes to lint_backend_step()

* try to add validate_modifle_step

* try to run check func

* remove CGO and wire-install

* update filepath to scripts/modowners

* make drone again

* correct the filepath

* try to ignore validate-modfile step if fail

* chore: remove log that says i am a programmer

* fix: spell modfile correctly

* chore: remove unecessary comment

* fix: check go.mod instead of go.txd
2023-05-31 16:57:45 -04:00
Dimitris Sotirakis
26bacf6d4c
G10: Post docker pull commands for local testing (#67489)
* Add grabpl publish images

Update grabpl - add github pr token

* Replace github_pr_token with gh app

# Conflicts:
#	.drone.yml
#	scripts/drone/vault.star

# Conflicts:
#	.drone.yml

# Conflicts:
#	.drone.yml

* Lint starlark

Add forgotten secrets

Lint starlark

Change secrets

* Update grabpl to v3.0.35

* Lint starlark

* Remove unnecessary secrets
2023-05-17 17:57:37 +02:00
Horst Gutmann
acd02e8a48
CI: Move windows-tests into promotion pipeline (#68218) 2023-05-10 17:16:59 +03:00
Horst Gutmann
b2fc285a5d
CI: Add Windows backend tests in more places (#66438)
* CI: Add Windows backend tests in more places

* CI: Add promotion pipeline for publishing new windows-test-images

* CI: Ignore windows backend-test failures for now

* CI: Fix linting issue in ci_images.star file
2023-05-08 15:22:34 +02:00
owensmallwood
a4c7450c42
Grafana: Upgrades mysql images from 5.7 to 8 (#67604)
adds mysql integration build step for mysql 8
2023-05-04 11:42:48 -06:00
Sofia Papagiannaki
caac9838d8
Build: Fix Redis/Memcached integration tests (#64298)
* Build: Fix integration cache tests

* Allow REDIS_URL with scheme

* Reduce cache integration tests timeout to 5m

* Apply suggestion from code review

* Run redis/memcached integration tests in OSS pipelines

* Change redis image
2023-04-05 11:55:55 +03:00
Dimitris Sotirakis
821633c796
CI: Make init-enterprise depend on grabpl (#65797)
Make init-enterprise depend on grabpl

# Conflicts:
#	.drone.yml
2023-04-03 14:04:30 +03:00
Dimitris Sotirakis
89d642e001
CI: Cleanup unnecessary grabpl dependencies (#65330)
* Remove unnecessary grabpl dependencies

* Revert test-frontend change
2023-03-27 12:55:45 +02:00
Dimitris Sotirakis
d0be9610f0
CD: Remove fetch-images dependency from AWS marketplace automation (#65173)
* Remove fetch-images dependency

* Fix starlark link
2023-03-22 17:35:57 +02:00
Dimitris Sotirakis
e5c48ac945
CI: Remove variants arg from package step (#62858)
* Remove variants arg from package step

# Conflicts:
#	.drone.yml

* Fix starlark lint

* Default to releaseModeConfig.Variants

* Check for empty variants arg
2023-02-06 17:15:42 +02:00
Kevin Minehart
3cbaa58b1b
CI: Replace enterprise check (#62359)
* Add steps in the pipeline for cloning enterprise when running PR tests.
* Removed enterprise-check from the github action workflows, and removed it from the codeowners file.
2023-02-01 16:55:49 +00:00
Dimitris Sotirakis
1a5b54cff3
CI: Replace grafana/grafana-oss with grafana-oss when publishing to dockerhub repo (#62651)
Replace grafana/grafana-oss with grafana-oss
2023-02-01 09:53:42 +00:00
Jack Baldry
8379a5338c
CI: Lint starlark files with buildifier (#59157)
* Add verify-starlark build action that returns an error for starlark files with lint

Relies on `buildifier` tool.

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Add verify_starlark_step to PR pipeline

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Manually fetch buildifier in curl_image until a new build_image is created

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Format with buildifier

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Remove all unused variables retaining one unused function

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Use snake_case for variable

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Replace deprecated dictionary concatenation with .update() method

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Start adding docstrings for all modules and functions

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Prefer os.WriteFile as ioutil.WriteFile has been deprecated since go 1.16

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Attempt to document the behavior of the init_enterprise_step

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Document test_backend pipeline

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Document enterprise_downstream_step

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Document the pipeline utility function

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Document publish_images_step

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Document publish_images_steps

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Document enterprise2_pipelines function

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Add tags table for Starlark files.

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Document test_frontend

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Document windows function

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Add docstrings to verifystarlark functions

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Refactor error handling to be more clear and document complex behavior

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Split errors into execution errors and verification errors

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Document all other library functions

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Add local variables to TAGS

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Add blank line between all Args and Returns sections

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Fix new linting errors

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Lint new Starlark files

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Correct buildifier binary mv

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Document the need to set nofile ulimit to at least 2048

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Update build-container to include buildifier

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Ensure buildifier binary is executable

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Fix valid content test

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Simply return execution error

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Only check files rather than fixing them

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Use updated build-container with executable buildifier

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Test that context cancellation stops execution

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Simplify error handling

Return execution errors that short circuit WalkDir rather than
separately tracking that error.

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Remove fetching of buildifier binary now that it is in the build-container

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Use build image in verify-starlark step

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Use semver tag

The image is the same but uses a semver tag to make it clearer that
this is a forward upgrade from the old version.

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

* Use node 18 image with buildifier

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>

---------

Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
2023-01-30 09:27:11 +00:00
Dimitris Sotirakis
158502572e
CI: Add aws-marketplace pipeline (#60484)
* Add aws marketplace automation

# Conflicts:
#	.drone.yml

* Fix secret paths

# Conflicts:
#	.drone.yml

* Add docker socket

# Conflicts:
#	.drone.yml

# Conflicts:
#	.drone.yml

* s/enterprise2/enterprise

* Add dependency on the enterprise docker publish

# Conflicts:
#	.drone.yml

* Replace testing args with prod args

# Conflicts:
#	.drone.yml

* Fix path

# Conflicts:
#	.drone.yml
2022-12-19 11:25:48 -05:00
Dimitris Sotirakis
e9cf8fa751
CI: Add automation for github assets publishing (#59491)
* Add github.star

# Conflicts:
#	.drone.star
#	.drone.yml

* Make step depend on fetch images

# Conflicts:
#	.drone.yml

* artifacts -> path

# Conflicts:
#	.drone.yml

* Add github token

# Conflicts:
#	.drone.yml

* Fix custom path

# Conflicts:
#	.drone.yml

* Add case where path is absent

# Conflicts:
#	.drone.yml

* Add GH_REGISTRY secret

# Conflicts:
#	.drone.yml
2022-12-12 09:51:27 -05:00
Dan Cech
440d8a3d88
Build: Drone starlark file cleanup (#59919)
* format drone starlark files with black

* clean up unused params

* more simplification

* more cleanup

* more cleanup
2022-12-07 09:13:57 +02:00
Dimitris Sotirakis
7deaeb0f9e
CI: Add identify-runner step for publish images (#59827)
Add missing identify-runner step
2022-12-05 09:49:33 -05:00
Dimitris Sotirakis
44402ff726
CI: Split enterprise2 pipelines (#57997)
* Add publishimages_enterprise2.go file

* Add push.go

* Modify build files

* Modify starlark files

* Remove promotion pipeline

* Remove unused args

* Handle tag for custom events
2022-11-21 20:06:00 +02:00
Dimitris Sotirakis
0bf3c98eec
CI: Add environment variables for edition to all pipelines (#58964)
Add environment variables to pipelines
2022-11-18 16:15:04 +02:00