IntenseWebs/veilid
3
0
Fork 0
mirror of https://gitlab.com/veilid/veilid.git synced 2024-11-22 08:56:58 -06:00
Code Issues Packages Projects Releases Wiki Activity
4d9787fb86
veilid/.gitlab-ci.yml

306 lines
8.2 KiB
YAML
Raw Normal View History

cleanup and stats accounting organization
2021-11-26 08:54:38 -06:00
variables:
use new runner
2022-11-22 16:16:53 -06:00
NO_DOCKER: 1
FORCE_COLOR: 1
EARTHLY_EXEC_CMD: "/bin/sh"
Add submodule strategyt to build
2022-11-26 23:18:08 -06:00
GIT_SUBMODULE_STRATEGY: normal
cleanup and stats accounting organization
2021-11-26 08:54:38 -06:00
Add merge_requests as build target, split into two architectures
2023-02-20 13:14:05 -06:00
stages:
Refactored Earthfile to improve caching Targets have been parallelized so that initial push of the container cache should build the whole build a little quicker, plus the container should now use the cache for more of the build and so speed up normal builds to just the compilation and test of the code that has changed The `build_cache` target now builds a `build-cache:latest` container that is stored in the GitLab project Container Registry, and then used (if it exists) by the `test_build` target. The `build_cache` task runs under 3 conditions, 1. the container does not exist, 2. if scheduled, 3. if run manually from the Pipelines page in the GitLab interface. It is recommended that the build is set up to run on a weekly schedule via the `Pipeline schedules` page in GitLab with the schedule of `0 2 * * 6`.
2023-10-31 06:17:13 -05:00
- prepare
Update .gitlab-ci.yml file to the intermediary package distribution pipeline.
2023-06-20 21:46:05 -05:00
- test
Update .gitlab-ci.yml file
2023-07-16 12:04:23 -05:00
- build_packages
Adding release stage triggered by existence of a tag that matches the v+semver format
2023-07-31 19:59:23 -05:00
- release
Update .gitlab-ci.yml file to the intermediary package distribution pipeline.
2023-06-20 21:46:05 -05:00
- distribute
initial CI attempt
2021-11-24 15:49:13 -06:00
Refactored Earthfile to improve caching Targets have been parallelized so that initial push of the container cache should build the whole build a little quicker, plus the container should now use the cache for more of the build and so speed up normal builds to just the compilation and test of the code that has changed The `build_cache` target now builds a `build-cache:latest` container that is stored in the GitLab project Container Registry, and then used (if it exists) by the `test_build` target. The `build_cache` task runs under 3 conditions, 1. the container does not exist, 2. if scheduled, 3. if run manually from the Pipelines page in the GitLab interface. It is recommended that the build is set up to run on a weekly schedule via the `Pipeline schedules` page in GitLab with the schedule of `0 2 * * 6`.
2023-10-31 06:17:13 -05:00
# base earthly setup for jobs
.base:
tags: [ saas-linux-medium-amd64 ]
Earthly Build Caching in GitLab Container Registry
2023-08-27 12:17:29 -05:00
image: docker
services:
- docker:dind
Refactored Earthfile to improve caching Targets have been parallelized so that initial push of the container cache should build the whole build a little quicker, plus the container should now use the cache for more of the build and so speed up normal builds to just the compilation and test of the code that has changed The `build_cache` target now builds a `build-cache:latest` container that is stored in the GitLab project Container Registry, and then used (if it exists) by the `test_build` target. The `build_cache` task runs under 3 conditions, 1. the container does not exist, 2. if scheduled, 3. if run manually from the Pipelines page in the GitLab interface. It is recommended that the build is set up to run on a weekly schedule via the `Pipeline schedules` page in GitLab with the schedule of `0 2 * * 6`.
2023-10-31 06:17:13 -05:00
.earthly: &earthly_setup
- apk update && apk add git
- wget https://github.com/earthly/earthly/releases/download/v0.7.15/earthly-linux-amd64 -O /usr/local/bin/earthly
- chmod +x /usr/local/bin/earthly
- earthly bootstrap
- echo $CI_REGISTRY_PASSWORD | docker login -u $CI_REGISTRY_USER --password-stdin $CI_REGISTRY
Use $CI_REGISTRY_IMAGE for the registry path GitLab will return an error if you have an upper case letter at the start of your username. ``` invalid reference format: repository name must be lowercase ``` The built-in `$CI_REGISTRY_IMAGE` variable does the right thing. Closes https://gitlab.com/veilid/veilid/-/issues/352
2023-12-25 15:14:23 -06:00
- test "$CI_PROJECT_PATH" != "veilid/veilid" && project_args="--CI_REGISTRY_IMAGE=$CI_REGISTRY_IMAGE"
Refactored Earthfile to improve caching Targets have been parallelized so that initial push of the container cache should build the whole build a little quicker, plus the container should now use the cache for more of the build and so speed up normal builds to just the compilation and test of the code that has changed The `build_cache` target now builds a `build-cache:latest` container that is stored in the GitLab project Container Registry, and then used (if it exists) by the `test_build` target. The `build_cache` task runs under 3 conditions, 1. the container does not exist, 2. if scheduled, 3. if run manually from the Pipelines page in the GitLab interface. It is recommended that the build is set up to run on a weekly schedule via the `Pipeline schedules` page in GitLab with the schedule of `0 2 * * 6`.
2023-10-31 06:17:13 -05:00
# Create the build container if:
# - no container in the registry
# - run as part of a schedule
# - run manually from the pipelines web page
build_cache:
extends: .base
stage: prepare
script:
Use $CI_REGISTRY_IMAGE for the registry path GitLab will return an error if you have an upper case letter at the start of your username. ``` invalid reference format: repository name must be lowercase ``` The built-in `$CI_REGISTRY_IMAGE` variable does the right thing. Closes https://gitlab.com/veilid/veilid/-/issues/352
2023-12-25 15:14:23 -06:00
- if ! docker manifest inspect $CI_REGISTRY_IMAGE/build-cache:latest > /dev/null || [[ "$CI_PIPELINE_SOURCE" == "schedule" ]] || [[ "$CI_PIPELINE_SOURCE" == "web" ]] ; then
Refactored Earthfile to improve caching Targets have been parallelized so that initial push of the container cache should build the whole build a little quicker, plus the container should now use the cache for more of the build and so speed up normal builds to just the compilation and test of the code that has changed The `build_cache` target now builds a `build-cache:latest` container that is stored in the GitLab project Container Registry, and then used (if it exists) by the `test_build` target. The `build_cache` task runs under 3 conditions, 1. the container does not exist, 2. if scheduled, 3. if run manually from the Pipelines page in the GitLab interface. It is recommended that the build is set up to run on a weekly schedule via the `Pipeline schedules` page in GitLab with the schedule of `0 2 * * 6`.
2023-10-31 06:17:13 -05:00
- *earthly_setup
- earthly --use-inline-cache --save-inline-cache --strict --push -P +build-linux-cache $project_args
- else
- echo "No need to rebuild"
- fi
Fixes scp from build machines to orchestrator Each machine needed a script this specified that machine's arch and OS type. Also added a rule on the cache and test jobs to not run if the dry run trigger is present. [ci dryrun]
2024-04-29 17:07:40 -05:00
rules:
- if: $CI_COMMIT_MESSAGE =~ /\[ci dryrun]/
- when: never
Refactored Earthfile to improve caching Targets have been parallelized so that initial push of the container cache should build the whole build a little quicker, plus the container should now use the cache for more of the build and so speed up normal builds to just the compilation and test of the code that has changed The `build_cache` target now builds a `build-cache:latest` container that is stored in the GitLab project Container Registry, and then used (if it exists) by the `test_build` target. The `build_cache` task runs under 3 conditions, 1. the container does not exist, 2. if scheduled, 3. if run manually from the Pipelines page in the GitLab interface. It is recommended that the build is set up to run on a weekly schedule via the `Pipeline schedules` page in GitLab with the schedule of `0 2 * * 6`.
2023-10-31 06:17:13 -05:00
# Runs a basic unit test build, this task will use the `build-cache:latest` as set up in the projects Container Registry
test_build:
extends: .base
stage: test
Tests passed. This commit is a clean up of .gitlab-ci.yml
2023-08-22 16:49:55 -05:00
only:
CI config and Changelog tweaks Fixed a few typos before releasing v0.2.1
2023-09-03 19:43:46 -05:00
- main
Refactored Earthfile to improve caching Targets have been parallelized so that initial push of the container cache should build the whole build a little quicker, plus the container should now use the cache for more of the build and so speed up normal builds to just the compilation and test of the code that has changed The `build_cache` target now builds a `build-cache:latest` container that is stored in the GitLab project Container Registry, and then used (if it exists) by the `test_build` target. The `build_cache` task runs under 3 conditions, 1. the container does not exist, 2. if scheduled, 3. if run manually from the Pipelines page in the GitLab interface. It is recommended that the build is set up to run on a weekly schedule via the `Pipeline schedules` page in GitLab with the schedule of `0 2 * * 6`.
2023-10-31 06:17:13 -05:00
- pushes
Earthly Build Caching in GitLab Container Registry
2023-08-27 12:17:29 -05:00
- merge_requests
script:
Refactored Earthfile to improve caching Targets have been parallelized so that initial push of the container cache should build the whole build a little quicker, plus the container should now use the cache for more of the build and so speed up normal builds to just the compilation and test of the code that has changed The `build_cache` target now builds a `build-cache:latest` container that is stored in the GitLab project Container Registry, and then used (if it exists) by the `test_build` target. The `build_cache` task runs under 3 conditions, 1. the container does not exist, 2. if scheduled, 3. if run manually from the Pipelines page in the GitLab interface. It is recommended that the build is set up to run on a weekly schedule via the `Pipeline schedules` page in GitLab with the schedule of `0 2 * * 6`.
2023-10-31 06:17:13 -05:00
- *earthly_setup
- earthly --use-inline-cache +unit-tests-linux --BASE=container $project_args
Earthly Build Caching in GitLab Container Registry
2023-08-27 12:17:29 -05:00
resource_group: test
Fixes scp from build machines to orchestrator Each machine needed a script this specified that machine's arch and OS type. Also added a rule on the cache and test jobs to not run if the dry run trigger is present. [ci dryrun]
2024-04-29 17:07:40 -05:00
rules:
- if: $CI_COMMIT_MESSAGE =~ /\[ci dryrun]/
- when: never
Added ephemeral create/delete to test stage.
2023-07-22 12:07:00 -05:00
Added CICD dry run This is the first attempt at setting up a dry run pipeline to test changes to the CICD config without actually publishing the compiled binaries and packages built by the process. The dry run should be triggered by any changes to .gitlab-ci.yml or changes to any of the scripts under scripts/cicd/.
2024-04-29 14:40:07 -05:00
# Actual release -- triggered by pushing a new version tag
Update .gitlab-ci.yml file
2023-07-31 20:23:36 -05:00
release_job:
stage: release
Earthly Build Caching in GitLab Container Registry
2023-08-27 12:17:29 -05:00
image: registry.gitlab.com/gitlab-org/release-cli:latest
Update .gitlab-ci.yml file
2023-07-31 20:23:36 -05:00
script:
- echo "running release_job"
release: # See https://docs.gitlab.com/ee/ci/yaml/#release for available properties
tag_name: '$CI_COMMIT_TAG'
description: '$CI_COMMIT_TAG'
rules:
- if: '$CI_COMMIT_TAG =~ /v\d.+/'
Added jobs to create and delete the ephemeral build machines.
2023-07-21 14:30:33 -05:00
create_build_machines:
stage: build_packages
tags:
- build-orchestration
script:
Calling bash explicitly The paths work now, but the scripts are not set with execute permission. I don't know if that permission will survive the transfer from my machine across the git and runner stuff so I added an explicit call for bash instead.
2024-04-28 20:39:18 -05:00
- bash scripts/cicd/build-orchestration/build-machine-ctrl.sh create amd64-deb
- bash scripts/cicd/build-orchestration/build-machine-ctrl.sh create arm64-deb
- bash scripts/cicd/build-orchestration/build-machine-ctrl.sh create amd64-rpm
Adding release stage triggered by existence of a tag that matches the v+semver format
2023-07-31 20:08:41 -05:00
rules:
Update .gitlab-ci.yml file
2023-07-31 20:23:36 -05:00
- if: '$CI_COMMIT_TAG =~ /v\d.+/'
Adding release stage triggered by existence of a tag that matches the v+semver format
2023-07-31 20:08:41 -05:00
Update .gitlab-ci.yml file
2023-07-16 12:04:23 -05:00
package_amd64_deb:
stage: build_packages
Added jobs to create and delete the ephemeral build machines.
2023-07-21 14:30:33 -05:00
needs:
- create_build_machines
Update .gitlab-ci.yml file to the intermediary package distribution pipeline.
2023-06-20 21:46:05 -05:00
tags:
Update .gitlab-ci.yml file
2023-07-16 12:04:23 -05:00
- build-amd64-deb
Update .gitlab-ci.yml file to the intermediary package distribution pipeline.
2023-06-20 21:46:05 -05:00
script:
Update .gitlab-ci.yml file
2023-07-04 13:31:02 -05:00
- earthly bootstrap
Update .gitlab-ci.yml file
2023-07-03 21:50:50 -05:00
- earthly +package-linux-amd64-deb
Fixes scp from build machines to orchestrator Each machine needed a script this specified that machine's arch and OS type. Also added a rule on the cache and test jobs to not run if the dry run trigger is present. [ci dryrun]
2024-04-29 17:07:40 -05:00
- bash scripts/cicd/build-machine/scp-amd64-debs-to-orchestrator.sh
Adding release stage triggered by existence of a tag that matches the v+semver format
2023-07-31 20:08:41 -05:00
rules:
Update .gitlab-ci.yml file
2023-07-31 20:23:36 -05:00
- if: '$CI_COMMIT_TAG =~ /v\d.+/'
Earthly Build Caching in GitLab Container Registry
2023-08-27 12:17:29 -05:00
Update .gitlab-ci.yml file
2023-07-16 12:04:23 -05:00
package_arm64_deb:
stage: build_packages
Added jobs to create and delete the ephemeral build machines.
2023-07-21 14:30:33 -05:00
needs:
- create_build_machines
Update .gitlab-ci.yml file
2023-07-16 12:04:23 -05:00
tags:
- build-arm64-deb
script:
- earthly bootstrap
Removed amd64 from package job name, added arm64.deb to package job scripts
2023-07-14 23:37:30 -05:00
- earthly +package-linux-arm64-deb
Fixes scp from build machines to orchestrator Each machine needed a script this specified that machine's arch and OS type. Also added a rule on the cache and test jobs to not run if the dry run trigger is present. [ci dryrun]
2024-04-29 17:07:40 -05:00
- bash scripts/cicd/build-machine/scp-arm64-debs-to-orchestrator.sh
Adding release stage triggered by existence of a tag that matches the v+semver format
2023-07-31 20:08:41 -05:00
rules:
Update .gitlab-ci.yml file
2023-07-31 20:23:36 -05:00
- if: '$CI_COMMIT_TAG =~ /v\d.+/'
Update .gitlab-ci.yml file
2023-07-04 11:34:48 -05:00
Update .gitlab-ci.yml file
2023-07-16 12:04:23 -05:00
package_amd64_rpm:
stage: build_packages
Added jobs to create and delete the ephemeral build machines.
2023-07-21 14:30:33 -05:00
needs:
- create_build_machines
Update .gitlab-ci.yml file
2023-07-16 12:04:23 -05:00
tags:
- build-amd64-rpm
script:
- earthly bootstrap
- earthly +package-linux-amd64-rpm
Fixes scp from build machines to orchestrator Each machine needed a script this specified that machine's arch and OS type. Also added a rule on the cache and test jobs to not run if the dry run trigger is present. [ci dryrun]
2024-04-29 17:07:40 -05:00
- bash scripts/cicd/build-machine/scp-amd64-rmps-to-orchestrator.sh
Adding release stage triggered by existence of a tag that matches the v+semver format
2023-07-31 20:08:41 -05:00
rules:
Update .gitlab-ci.yml file
2023-07-31 20:23:36 -05:00
- if: '$CI_COMMIT_TAG =~ /v\d.+/'
Adding release stage triggered by existence of a tag that matches the v+semver format
2023-07-31 19:59:23 -05:00
Moved crate and python publishing to a dev machine. The orchestration machine doesn't have the resources for the cargo build steps.
2024-04-04 18:22:10 -05:00
publish_crates:
moved crates and pypi publishing to build_packages stage
2024-04-04 19:53:28 -05:00
stage: build_packages
Update .gitlab-ci.yml file
2023-07-16 14:18:35 -05:00
tags:
Moved crate and python publishing to a dev machine. The orchestration machine doesn't have the resources for the cargo build steps.
2024-04-04 18:22:10 -05:00
- build-amd64-deb
Update .gitlab-ci.yml file
2023-07-16 14:18:35 -05:00
script:
Added crates.io and Pypi release steps to build_repositories job. Maybe it works, maybe it doesn't. We'll find out at next release.
2024-03-03 15:19:21 -06:00
- vlt login
- vlt run --command="cargo publish -p veilid-tools --dry-run"
- vlt run --command="cargo publish -p veilid-tools"
- vlt run --command="cargo publish -p veilid-core --dry-run"
- vlt run --command="cargo publish -p veilid-core"
Moved crate and python publishing to a dev machine. The orchestration machine doesn't have the resources for the cargo build steps.
2024-04-04 18:22:10 -05:00
rules:
- if: '$CI_COMMIT_TAG =~ /v\d.+/'
publish_python:
moved crates and pypi publishing to build_packages stage
2024-04-04 19:53:28 -05:00
stage: build_packages
Moved crate and python publishing to a dev machine. The orchestration machine doesn't have the resources for the cargo build steps.
2024-04-04 18:22:10 -05:00
tags:
- build-amd64-deb
script:
- vlt login
It was a typo in the script name Was missing the r in ctrl then spread via copy paste. Fixed the typo and reverted the paths to relative because that should work just fine.
2024-04-28 19:24:45 -05:00
- cd veilid-python && /home/gitlab-runner/.local/bin/poetry build
Added crates.io and Pypi release steps to build_repositories job. Maybe it works, maybe it doesn't. We'll find out at next release.
2024-03-03 15:19:21 -06:00
- vlt run --command="/home/gitlab-runner/.local/bin/poetry publish"
Adding release stage triggered by existence of a tag that matches the v+semver format
2023-07-31 20:08:41 -05:00
rules:
Update .gitlab-ci.yml file
2023-07-31 20:23:36 -05:00
- if: '$CI_COMMIT_TAG =~ /v\d.+/'
Update .gitlab-ci.yml file
2023-07-16 14:18:35 -05:00
Moved crate and python publishing to a dev machine. The orchestration machine doesn't have the resources for the cargo build steps.
2024-04-04 18:22:10 -05:00
build_repositories:
stage: distribute
Correcting distribute stage order of execution
2024-04-04 19:46:47 -05:00
needs:
- publish_python
Moved crate and python publishing to a dev machine. The orchestration machine doesn't have the resources for the cargo build steps.
2024-04-04 18:22:10 -05:00
tags:
- build-orchestration
Migrating CICD scripts to Veilid repo Copied CICD scripts into the repository so that the community can make contributions to the build system. Wrote a brief description of the build and distribute process. Modified the CICD config to use the repo hosted scripts. [ci skip]
2024-04-21 13:37:45 -05:00
variables:
SECURE_FILES_DOWNLOAD_PATH: './'
Moved crate and python publishing to a dev machine. The orchestration machine doesn't have the resources for the cargo build steps.
2024-04-04 18:22:10 -05:00
script:
It was a typo in the script name Was missing the r in ctrl then spread via copy paste. Fixed the typo and reverted the paths to relative because that should work just fine.
2024-04-28 19:24:45 -05:00
- cp scripts/cicd/build-orchestration/generate-release.sh ~
Calling bash explicitly The paths work now, but the scripts are not set with execute permission. I don't know if that permission will survive the transfer from my machine across the git and runner stuff so I added an explicit call for bash instead.
2024-04-28 20:39:18 -05:00
- bash scripts/cicd/build-orchestration/distribute-packages.sh
Moved crate and python publishing to a dev machine. The orchestration machine doesn't have the resources for the cargo build steps.
2024-04-04 18:22:10 -05:00
rules:
- if: '$CI_COMMIT_TAG =~ /v\d.+/'
Update .gitlab-ci.yml file
2023-07-16 14:18:35 -05:00
deploy_repos:
stage: distribute
needs:
- build_repositories
tags:
- repo-server
script:
Calling bash explicitly The paths work now, but the scripts are not set with execute permission. I don't know if that permission will survive the transfer from my machine across the git and runner stuff so I added an explicit call for bash instead.
2024-04-28 20:39:18 -05:00
- bash scripts/cicd/repo-server/deploy-repo.sh
Update .gitlab-ci.yml file
2023-08-02 09:14:18 -05:00
rules:
- if: '$CI_COMMIT_TAG =~ /v\d.+/'
delete_build_machines:
stage: distribute
needs:
- deploy_repos
tags:
- build-orchestration
script:
Calling bash explicitly The paths work now, but the scripts are not set with execute permission. I don't know if that permission will survive the transfer from my machine across the git and runner stuff so I added an explicit call for bash instead.
2024-04-28 20:39:18 -05:00
- bash scripts/cicd/build-orchestration/build-machine-ctrl.sh delete amd64-deb
- bash scripts/cicd/build-orchestration/build-machine-ctrl.sh delete arm64-deb
- bash scripts/cicd/build-orchestration/build-machine-ctrl.sh delete amd64-rpm
Adding release stage triggered by existence of a tag that matches the v+semver format
2023-07-31 20:08:41 -05:00
rules:
Earthly Build Caching in GitLab Container Registry
2023-08-27 12:17:29 -05:00
- if: '$CI_COMMIT_TAG =~ /v\d.+/'
Added CICD dry run This is the first attempt at setting up a dry run pipeline to test changes to the CICD config without actually publishing the compiled binaries and packages built by the process. The dry run should be triggered by any changes to .gitlab-ci.yml or changes to any of the scripts under scripts/cicd/.
2024-04-29 14:40:07 -05:00
Added Earthfile condition and RPM spec fix Dry run will now trigger for Earthfile changes. I also reordered the changelog in veilid-server.spec to descending to correct that error. Commented out the crates.io publishing dry run. This branch does not have access to the protected variables.
2024-04-29 15:24:04 -05:00
# Dryrun release -- triggered by changes in .gitlab-ci.yml, CICD scripts, or Earthfile
Added CICD dry run This is the first attempt at setting up a dry run pipeline to test changes to the CICD config without actually publishing the compiled binaries and packages built by the process. The dry run should be triggered by any changes to .gitlab-ci.yml or changes to any of the scripts under scripts/cicd/.
2024-04-29 14:40:07 -05:00
dryrun_create_build_machines:
stage: build_packages
tags:
- build-orchestration
script:
- bash scripts/cicd/build-orchestration/build-machine-ctrl.sh create amd64-deb
- bash scripts/cicd/build-orchestration/build-machine-ctrl.sh create arm64-deb
- bash scripts/cicd/build-orchestration/build-machine-ctrl.sh create amd64-rpm
rules:
Switched to regex trigger The previous trigger condition wasn't liked by Gitlab's yaml parser. I'm switching to trying a commit message based regex trigger. This message does not contain the trigger and so should not fire the dry run.
2024-04-29 16:22:58 -05:00
- if: $CI_COMMIT_MESSAGE =~ /\[ci dryrun]/
Added CICD dry run This is the first attempt at setting up a dry run pipeline to test changes to the CICD config without actually publishing the compiled binaries and packages built by the process. The dry run should be triggered by any changes to .gitlab-ci.yml or changes to any of the scripts under scripts/cicd/.
2024-04-29 14:40:07 -05:00
dryrun_package_amd64_deb:
stage: build_packages
needs:
- dryrun_create_build_machines
tags:
- build-amd64-deb
script:
- earthly bootstrap
- earthly +package-linux-amd64-deb
Fixes scp from build machines to orchestrator Each machine needed a script this specified that machine's arch and OS type. Also added a rule on the cache and test jobs to not run if the dry run trigger is present. [ci dryrun]
2024-04-29 17:07:40 -05:00
- bash scripts/cicd/build-machine/scp-amd64-debs-to-orchestrator.sh
Added CICD dry run This is the first attempt at setting up a dry run pipeline to test changes to the CICD config without actually publishing the compiled binaries and packages built by the process. The dry run should be triggered by any changes to .gitlab-ci.yml or changes to any of the scripts under scripts/cicd/.
2024-04-29 14:40:07 -05:00
rules:
Switched to regex trigger The previous trigger condition wasn't liked by Gitlab's yaml parser. I'm switching to trying a commit message based regex trigger. This message does not contain the trigger and so should not fire the dry run.
2024-04-29 16:22:58 -05:00
- if: $CI_COMMIT_MESSAGE =~ /\[ci dryrun]/
Added CICD dry run This is the first attempt at setting up a dry run pipeline to test changes to the CICD config without actually publishing the compiled binaries and packages built by the process. The dry run should be triggered by any changes to .gitlab-ci.yml or changes to any of the scripts under scripts/cicd/.
2024-04-29 14:40:07 -05:00
dryrun_package_arm64_deb:
stage: build_packages
needs:
- dryrun_create_build_machines
tags:
- build-arm64-deb
script:
- earthly bootstrap
- earthly +package-linux-arm64-deb
Fixes scp from build machines to orchestrator Each machine needed a script this specified that machine's arch and OS type. Also added a rule on the cache and test jobs to not run if the dry run trigger is present. [ci dryrun]
2024-04-29 17:07:40 -05:00
- bash scripts/cicd/build-machine/scp-arm64-debs-to-orchestrator.sh
Added CICD dry run This is the first attempt at setting up a dry run pipeline to test changes to the CICD config without actually publishing the compiled binaries and packages built by the process. The dry run should be triggered by any changes to .gitlab-ci.yml or changes to any of the scripts under scripts/cicd/.
2024-04-29 14:40:07 -05:00
rules:
Switched to regex trigger The previous trigger condition wasn't liked by Gitlab's yaml parser. I'm switching to trying a commit message based regex trigger. This message does not contain the trigger and so should not fire the dry run.
2024-04-29 16:22:58 -05:00
- if: $CI_COMMIT_MESSAGE =~ /\[ci dryrun]/
Further dry run trigger refinements The dry run should now be configured if changes to .gitlab-ci.yml, Earthfile, contents of scripts/cicd, or contents of package arrive as the result of a push OR merge but NOT when a tag is present. I think that will prevent the dry run executing alongside the actual release process when a new release tag is pushed but that also contains changes to the above mentioned files. <fingers crossed emoji>
2024-04-29 16:03:38 -05:00
Added CICD dry run This is the first attempt at setting up a dry run pipeline to test changes to the CICD config without actually publishing the compiled binaries and packages built by the process. The dry run should be triggered by any changes to .gitlab-ci.yml or changes to any of the scripts under scripts/cicd/.
2024-04-29 14:40:07 -05:00
dryrun_package_amd64_rpm:
stage: build_packages
needs:
- dryrun_create_build_machines
tags:
- build-amd64-rpm
script:
- earthly bootstrap
- earthly +package-linux-amd64-rpm
Fixes scp from build machines to orchestrator Each machine needed a script this specified that machine's arch and OS type. Also added a rule on the cache and test jobs to not run if the dry run trigger is present. [ci dryrun]
2024-04-29 17:07:40 -05:00
- bash scripts/cicd/build-machine/scp-amd64-rmps-to-orchestrator.sh
Added CICD dry run This is the first attempt at setting up a dry run pipeline to test changes to the CICD config without actually publishing the compiled binaries and packages built by the process. The dry run should be triggered by any changes to .gitlab-ci.yml or changes to any of the scripts under scripts/cicd/.
2024-04-29 14:40:07 -05:00
rules:
Switched to regex trigger The previous trigger condition wasn't liked by Gitlab's yaml parser. I'm switching to trying a commit message based regex trigger. This message does not contain the trigger and so should not fire the dry run.
2024-04-29 16:22:58 -05:00
- if: $CI_COMMIT_MESSAGE =~ /\[ci dryrun]/
Added CICD dry run This is the first attempt at setting up a dry run pipeline to test changes to the CICD config without actually publishing the compiled binaries and packages built by the process. The dry run should be triggered by any changes to .gitlab-ci.yml or changes to any of the scripts under scripts/cicd/.
2024-04-29 14:40:07 -05:00
Added Earthfile condition and RPM spec fix Dry run will now trigger for Earthfile changes. I also reordered the changelog in veilid-server.spec to descending to correct that error. Commented out the crates.io publishing dry run. This branch does not have access to the protected variables.
2024-04-29 15:24:04 -05:00
# dryrun_publish_crates:
# stage: build_packages
# needs:
# - dryrun_create_build_machines
# tags:
# - build-amd64-deb
# script:
# - vlt login
# - vlt run --command="cargo publish -p veilid-tools --dry-run"
# - vlt run --command="cargo publish -p veilid-core --dry-run"
# rules:
Further dry run trigger refinements The dry run should now be configured if changes to .gitlab-ci.yml, Earthfile, contents of scripts/cicd, or contents of package arrive as the result of a push OR merge but NOT when a tag is present. I think that will prevent the dry run executing alongside the actual release process when a new release tag is pushed but that also contains changes to the above mentioned files. <fingers crossed emoji>
2024-04-29 16:03:38 -05:00
# - if: ($CI_PIPELINE_SOURCE == "push" || $CI_PIPELINE_SOURCE == "merge_request_event") && $CI_COMMIT_TAG == NULL
Added Earthfile condition and RPM spec fix Dry run will now trigger for Earthfile changes. I also reordered the changelog in veilid-server.spec to descending to correct that error. Commented out the crates.io publishing dry run. This branch does not have access to the protected variables.
2024-04-29 15:24:04 -05:00
# changes:
# - .gitlab-ci.yml
# - scripts/cicd/**/*
# - Earthfile
Further dry run trigger refinements The dry run should now be configured if changes to .gitlab-ci.yml, Earthfile, contents of scripts/cicd, or contents of package arrive as the result of a push OR merge but NOT when a tag is present. I think that will prevent the dry run executing alongside the actual release process when a new release tag is pushed but that also contains changes to the above mentioned files. <fingers crossed emoji>
2024-04-29 16:03:38 -05:00
# - package/**/*
Added CICD dry run This is the first attempt at setting up a dry run pipeline to test changes to the CICD config without actually publishing the compiled binaries and packages built by the process. The dry run should be triggered by any changes to .gitlab-ci.yml or changes to any of the scripts under scripts/cicd/.
2024-04-29 14:40:07 -05:00
dryrun_publish_python:
stage: build_packages
needs:
- dryrun_create_build_machines
tags:
- build-amd64-deb
script:
- cd veilid-python && /home/gitlab-runner/.local/bin/poetry build
rules:
Switched to regex trigger The previous trigger condition wasn't liked by Gitlab's yaml parser. I'm switching to trying a commit message based regex trigger. This message does not contain the trigger and so should not fire the dry run.
2024-04-29 16:22:58 -05:00
- if: $CI_COMMIT_MESSAGE =~ /\[ci dryrun]/
Added CICD dry run This is the first attempt at setting up a dry run pipeline to test changes to the CICD config without actually publishing the compiled binaries and packages built by the process. The dry run should be triggered by any changes to .gitlab-ci.yml or changes to any of the scripts under scripts/cicd/.
2024-04-29 14:40:07 -05:00
dryrun_build_repositories:
stage: distribute
tags:
- build-orchestration
variables:
SECURE_FILES_DOWNLOAD_PATH: './'
script:
- cp scripts/cicd/build-orchestration/generate-release.sh ~
- bash scripts/cicd/build-orchestration/distribute-packages.sh
rules:
Switched to regex trigger The previous trigger condition wasn't liked by Gitlab's yaml parser. I'm switching to trying a commit message based regex trigger. This message does not contain the trigger and so should not fire the dry run.
2024-04-29 16:22:58 -05:00
- if: $CI_COMMIT_MESSAGE =~ /\[ci dryrun]/
Added CICD dry run This is the first attempt at setting up a dry run pipeline to test changes to the CICD config without actually publishing the compiled binaries and packages built by the process. The dry run should be triggered by any changes to .gitlab-ci.yml or changes to any of the scripts under scripts/cicd/.
2024-04-29 14:40:07 -05:00
dryrun_deploy_repos:
stage: distribute
needs:
- dryrun_build_repositories
tags:
- repo-server
script:
- ls -al repo.tar
rules:
Switched to regex trigger The previous trigger condition wasn't liked by Gitlab's yaml parser. I'm switching to trying a commit message based regex trigger. This message does not contain the trigger and so should not fire the dry run.
2024-04-29 16:22:58 -05:00
- if: $CI_COMMIT_MESSAGE =~ /\[ci dryrun]/
Added CICD dry run This is the first attempt at setting up a dry run pipeline to test changes to the CICD config without actually publishing the compiled binaries and packages built by the process. The dry run should be triggered by any changes to .gitlab-ci.yml or changes to any of the scripts under scripts/cicd/.
2024-04-29 14:40:07 -05:00
dryrun_delete_build_machines:
stage: distribute
needs:
- dryrun_deploy_repos
tags:
- build-orchestration
script:
- bash scripts/cicd/build-orchestration/build-machine-ctrl.sh delete amd64-deb
- bash scripts/cicd/build-orchestration/build-machine-ctrl.sh delete arm64-deb
- bash scripts/cicd/build-orchestration/build-machine-ctrl.sh delete amd64-rpm
rules:
Testing dry run trigger The trigger is [ci dryrun]
2024-04-29 16:26:13 -05:00
- if: $CI_COMMIT_MESSAGE =~ /\[ci dryrun]/
# Added to test commit message dry run trigger
Reference in New Issue Copy Permalink