mirror of
https://github.com/pgadmin-org/pgadmin4.git
synced 2026-10-31 16:37:42 -05:00
Publish the assembled snapshot to the staging server
The last missing link: assemble produced a tree and nothing carried it anywhere. The publish job uploads it over the rrsync-confined key and then asks pga-publish to sign and index it, so the runner names a verb and a datestamp rather than a command, and the signing happens where the key is. The staging directory can be named explicitly, because the buildfarm writes <date> on that same server and a test run has no business landing on the directory Jenkins is still filling. Publication is off by default for the same reason: it takes an explicit input, so a dispatch that only wants to know whether everything builds cannot reach the servers at all. Also adds the deployment notes, which is where the four files go on each host, what the two configuration files hold, and why no key gets a shell.
This commit is contained in:
Dave Page committed
1 parent
596396acaf
commit
9f7dff40f1
2 files changed
+196
No files matched your search
@@ -18,6 +18,19 @@ name: Snapshot
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
location:
|
||||
description: >-
|
||||
Staging directory name, normally today's date. Jenkins writes
|
||||
<date> on the same server, so a test run wants a -N suffix,
|
||||
for example 2026-09-17-2. stage-list shows what is taken.
|
||||
required: false
|
||||
default: ''
|
||||
publish:
|
||||
description: Upload to the staging server when the build succeeds
|
||||
type: boolean
|
||||
required: false
|
||||
default: false
|
||||
|
||||
# See snapshot-macos.yml: no dispatch button exists for a workflow off the
|
||||
# default branch, so whilst this lives on a working branch it fires on
|
||||
@@ -162,3 +175,85 @@ jobs:
|
||||
with:
|
||||
name: pgadmin4-snapshot
|
||||
path: stage/
|
||||
|
||||
# Upload the assembled tree to procyon and have it signed and indexed there.
|
||||
#
|
||||
# Everything this job does on the far side goes through pga-publish behind a
|
||||
# forced command, so the request is a verb and a datestamp rather than a
|
||||
# shell command; the wrapper decides what that means. The bytes travel over a
|
||||
# second key confined by rrsync to the staging root, write-only and with
|
||||
# deletion refused, so a compromised run can add to a staging build but
|
||||
# cannot read the rest of the server or empty it.
|
||||
#
|
||||
# The signing happens on the far side too, which is the point: the key never
|
||||
# comes here.
|
||||
publish:
|
||||
needs: assemble
|
||||
if: inputs.publish
|
||||
runs-on: ubuntu-24.04
|
||||
|
||||
# Restricts the keys to this job, and lets a deployment branch policy
|
||||
# decide which refs may reach the servers at all.
|
||||
environment: publishing
|
||||
|
||||
steps:
|
||||
- name: Collect the assembled tree
|
||||
uses: actions/download-artifact@v7
|
||||
with:
|
||||
name: pgadmin4-snapshot
|
||||
path: stage
|
||||
|
||||
- name: Configure SSH
|
||||
env:
|
||||
PUBLISH_KEY: ${{ secrets.PUBLISH_SSH_KEY }}
|
||||
UPLOAD_KEY: ${{ secrets.UPLOAD_SSH_KEY }}
|
||||
KNOWN_HOSTS: ${{ secrets.SSH_KNOWN_HOSTS }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
mkdir -m 700 -p ~/.ssh
|
||||
printf '%s\n' "${PUBLISH_KEY}" > ~/.ssh/publish
|
||||
printf '%s\n' "${UPLOAD_KEY}" > ~/.ssh/upload
|
||||
printf '%s\n' "${KNOWN_HOSTS}" > ~/.ssh/known_hosts
|
||||
chmod 600 ~/.ssh/publish ~/.ssh/upload ~/.ssh/known_hosts
|
||||
|
||||
- name: Publish to staging
|
||||
env:
|
||||
STAGING: ${{ vars.STAGING_SSH_TARGET }}
|
||||
LOCATION: ${{ inputs.location }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
if [ -z "${STAGING}" ]; then
|
||||
echo "The STAGING_SSH_TARGET variable is not set." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# One directory came out of assemble, named for the date. An explicit
|
||||
# location overrides it so that a test run does not land on the one
|
||||
# the buildfarm is still writing to on the same server.
|
||||
BUILT=$(ls stage)
|
||||
DEST=${LOCATION:-${BUILT}}
|
||||
echo "Publishing ${BUILT} as ${DEST}"
|
||||
|
||||
publish() {
|
||||
ssh -i ~/.ssh/publish -o BatchMode=yes \
|
||||
-o StrictHostKeyChecking=yes "${STAGING}" "$@"
|
||||
}
|
||||
|
||||
publish stage-create "${DEST}"
|
||||
|
||||
# No --delete, deliberately: rrsync refuses it anyway, and a
|
||||
# publishing step that can remove things is a publishing step that
|
||||
# can remove the wrong things.
|
||||
rsync -rlt --safe-links --no-specials --no-devices \
|
||||
-e "ssh -i ~/.ssh/upload -o BatchMode=yes -o StrictHostKeyChecking=yes" \
|
||||
"stage/${BUILT}/" "${STAGING}:${DEST}/"
|
||||
|
||||
# Signing before indexing: the metadata has to describe the signed
|
||||
# packages rather than the unsigned ones just uploaded.
|
||||
publish stage-sign "${DEST}"
|
||||
publish stage-index-apt "${DEST}"
|
||||
publish stage-index-yum "${DEST}"
|
||||
|
||||
- name: Remove the keys
|
||||
if: always()
|
||||
run: rm -f ~/.ssh/publish ~/.ssh/upload ~/.ssh/known_hosts
|
||||
@@ -0,0 +1,101 @@
|
||||
# Publishing
|
||||
|
||||
Everything that puts a pgAdmin build in front of a user lives here: the scripts
|
||||
that index and sign the APT and YUM repositories, the README that sits at the
|
||||
top of each of them, and the wrapper that lets a GitHub Actions runner ask for
|
||||
those things to happen without being given a shell on the servers.
|
||||
|
||||
Two servers are involved. The *staging* server holds build output at
|
||||
`/var/www/html/builds/<date>` and serves it over HTTP for testing. The
|
||||
*download* server holds the published tree at `/var/ftp/pgadmin4` and is what
|
||||
the world mirrors. Each runs the same wrapper with a different role.
|
||||
|
||||
## Files
|
||||
|
||||
| File | What it is |
|
||||
| --- | --- |
|
||||
| `rebuild-apt-repo.sh` | Index and sign one APT tree |
|
||||
| `rebuild-yum-repo.sh` | Index and sign one YUM tree, and make the EL compatibility links |
|
||||
| `install-repo-readme.py` | Write the README at the top of an APT or YUM tree |
|
||||
| `README.apt.in`, `README.yum.in` | The prose those READMEs are built from |
|
||||
| `aptftp.conf` | `apt-ftparchive` settings: Origin, Label, Suite, Description |
|
||||
| `pga-publish` | The forced-command wrapper |
|
||||
| `authorized_keys.procyon`, `authorized_keys.paxsor` | Annotated templates for the two servers |
|
||||
| `selftest.sh` | Exercises the wrapper's parser without touching anything |
|
||||
|
||||
The three scripts work standalone. Rebuilding a repository by hand after
|
||||
purging old releases is what `rebuild-apt-repo.sh bookworm` has always been
|
||||
for, and that has not changed; the wrapper calls the same script rather than
|
||||
carrying its own copy of the logic.
|
||||
|
||||
## Installing
|
||||
|
||||
On both servers, as root:
|
||||
|
||||
```sh
|
||||
install -d -m 755 /usr/local/lib/pga-publish
|
||||
install -m 755 rebuild-apt-repo.sh rebuild-yum-repo.sh install-repo-readme.py \
|
||||
/usr/local/lib/pga-publish/
|
||||
install -m 644 aptftp.conf README.apt.in README.yum.in /usr/local/lib/pga-publish/
|
||||
install -m 755 pga-publish /usr/local/sbin/pga-publish
|
||||
```
|
||||
|
||||
Then the role, which the wrapper refuses to start without, since defaulting it
|
||||
would mean a misconfigured host quietly accepting the other one's verbs:
|
||||
|
||||
```sh
|
||||
echo staging > /etc/pga-publish.role # on the staging server
|
||||
echo download > /etc/pga-publish.role # on the download server
|
||||
```
|
||||
|
||||
On the download server only, the CloudFront distribution that `sync-s3`
|
||||
invalidates. It is an identifier rather than a credential, but it is site
|
||||
configuration, so it lives here rather than in the source tree:
|
||||
|
||||
```sh
|
||||
echo '<distribution id>' > /etc/pga-publish.cloudfront
|
||||
```
|
||||
|
||||
Also on the download server, move `aptftp.conf` out of `~pgaupload`: it is an
|
||||
input to what gets indexed and therefore to what gets signed, and nothing
|
||||
needs the publishing account to be able to write it.
|
||||
|
||||
## Access
|
||||
|
||||
Three key pairs. `roadie-publish` and `roadie-upload` belong to the runner;
|
||||
`paxsor-pull` belongs to the download server, which uses it to fetch staged
|
||||
content from the staging server during publication.
|
||||
|
||||
`authorized_keys.procyon` and `authorized_keys.paxsor` are annotated templates:
|
||||
substitute the addresses and the public keys, and read the annotations before
|
||||
changing an option, since each is there for a reason that is easier to write
|
||||
down than to rediscover.
|
||||
|
||||
The shape of it is that no key gets a shell. The publishing key runs
|
||||
`pga-publish`, which parses `SSH_ORIGINAL_COMMAND` itself and never passes it
|
||||
to one. The upload key runs `rrsync`, confined to the staging root,
|
||||
write-only, with deletion and symlink-following refused. `restrict` turns off
|
||||
everything optional, PTY allocation included, and keeps doing so as OpenSSH
|
||||
grows new options.
|
||||
|
||||
## Checking it
|
||||
|
||||
```sh
|
||||
./selftest.sh # 56 cases, touches nothing
|
||||
pga-publish --dry-run --role download rebuild-apt bookworm
|
||||
```
|
||||
|
||||
`--dry-run` prints the argument vector each step would have run instead of
|
||||
running it, and is only honoured when `SSH_ORIGINAL_COMMAND` is unset, so a
|
||||
client cannot reach it.
|
||||
|
||||
## What signs what, and where
|
||||
|
||||
The GPG key that signs packages and repository metadata stays on the servers.
|
||||
A workflow uploads unsigned artefacts and asks for `stage-sign`, so a
|
||||
compromised run can ask for a signature over something it has just uploaded,
|
||||
but never holds the key, signs out of band, or signs with a different one.
|
||||
|
||||
The exceptions are the macOS notarisation and the Windows Authenticode
|
||||
signature, which happen on the runners because the Apple credentials and the
|
||||
Certum token cannot follow the GPG key onto these machines.
|
||||
Reference in new issue
Block a user