Compare commits

...
Author SHA1 Message Date
Dave Page 6f322ea160 Address the pre-merge review
Ten fixes from a review pass with fresh eyes. The ones that mattered:

rebuild-yum-repo.sh signed repomd.xml with no -u, so it used whatever
the keyring's default key resolved to. That is the packaging key today
only because there is one secret key in it. A repository whose metadata
is signed by the wrong key is one every client with gpgcheck=1 rejects,
produced by a run that exits 0.

The snapshot publish gate required only deb and rpm, whilst its comment
justified exactly one exception. A failed docs, tarball or wheel job
published a snapshot missing those files, and a failed repo-rpm job was
worse: install-repo-readme.py then fails after the packages are uploaded
and signed, leaving half a snapshot on the download server.

The location input was interpolated into shell on the runner holding both
publishing keys. It now travels through the environment, and is validated
against the shape of a datestamp before anything downstream sees it, so a
bad one fails in five seconds rather than after an hour of building.

Make.bat tested %ERRORLEVEL% inside a parenthesised block, where it is
expanded at parse time, so a failed Authenticode signature on
pgAdmin4.exe was undetectable; the handler was PAUSE, which on a headless
signing host either returns at once or blocks until the job times out.
Both sites now fail the build.

A called workflow inherits the caller's ref, so once this is on master an
ordinary push would have shared a concurrency group with the nightly
snapshot's builds and cancelled them.

And PULL_KEY named a file that does not exist on the download server,
which would have failed the first release promotion.

The rest: if-no-files-found on every build artefact, so a build that
produces nothing fails rather than silently shipping a snapshot without
it; pinned syft and helm installers, which were piping a default branch
into a shell as root in the container that builds signed packages; the
Helm chart no longer built and discarded; and the comments that had
stopped describing the code.

The self-test grows from 56 cases to 70, covering the verbs that delete
things, arguments that look like options, and the SSH_ORIGINAL_COMMAND
entry point rather than only the dry-run path.
2026-09-21 09:30:46 +01:00
Dave Page b8ac3485ad Give rpmsign the key identity rather than relying on a dotfile
rpmsign refuses to run without %_gpg_name, and the buildfarm satisfied
that with a ~/.rpmmacros on each build agent. Signing has moved to the
download server, which has never signed an RPM and has no such file, so
the first real snapshot-sign failed there.

Passing it explicitly suits this better than adding the dotfile: the key
is already a constant in this file, for the same reason the client cannot
name one, and the result no longer depends on whose home directory the
wrapper runs in.
2026-09-18 14:36:30 +01:00
Dave Page 1a2bbac26e Clear the staging directory before assembling into it
roadie's workspace persists between jobs, so yesterday's aborted run
left stage/2026-09-17-2 behind and today's run found two dated
directories where it expected one. The guard caught it and refused to
guess, which is the right outcome, but the tree should not have been
there to find.

The merged assemble job begins with a checkout, which cleans the
workspace as a side effect, and this does not depend on that.
2026-09-18 11:40:55 +01:00
Dave Page 67086aabff Filter the branch's own build checks too
The path filters that just landed on master cover the workflows that
exist there. These five are new on this branch and were still firing on
every push, which is what starved the macOS runners repeatedly today:
each documentation or workflow commit started two macOS appbundle builds
that no change could possibly have affected.

Each now lists what it builds from. The two that consume pgbuild also
list the composite action that fetches it, since a change to how the
dependencies are installed changes the build as surely as the packaging
does.
2026-09-18 11:35:18 +01:00
Dave Page d6262db305 Merge remote-tracking branch 'upstream/master' into worktree-qa-builds-on-actions 2026-09-18 11:34:31 +01:00
Dave Page d317a93019 Only run each CI workflow when something it tests has changed (#10427)
* Only run each CI workflow when something it tests has changed

Every workflow fired on every push and every pull request, so a commit
touching nothing but documentation or a workflow file started the whole
matrix: two macOS jobs, the EPAS and PostgreSQL feature tests, the
container build and the rest. That is slow for the contributor waiting
on a green tick and wasteful of the hosted runners, and macOS capacity
in particular is the one resource we can actually exhaust.

Each workflow now lists the paths it depends on, and its own file, so
editing it still runs it. The scope is deliberately per workflow rather
than shared: eslint and Jest care about the JavaScript and the files
that configure it, pycodestyle cares about Python anywhere in the tree,
the documentation build cares about docs plus the version and branding
it renders, and the packaging checks care about their own pkg
subdirectory alongside the application.

sonarqube-scan is left alone deliberately, being a whole-repository
quality scan rather than a test of any particular thing.

* Widen the path filters to inputs I had missed

All eight review findings check out against the tree, so all eight are
applied:

- The container build copies docs into the image, and .dockerignore
  decides what the build context contains at all.
- The documentation build imports web/config.py from conf.py.
- make install-python runs tools/setup-python-env.sh, so it belongs in
  every workflow that calls it, which is eight of them.
- pkg/pip/build.sh packages LICENSE and README.md and builds the docs;
  pkg/src/build.sh does much the same for the tarball.
- The Yarn and Babel configuration, web/.yarnrc.yml and
  web/babel.config.json, change what eslint and Jest do, and the
  JavaScript workflows filter on JavaScript rather than all of web/.
- tools/get_sb_package.py is fetched by the PostgreSQL and EPAS Python
  test workflows.

One deviation: get_sb_package.py goes to run-python-tests-pg and
run-python-tests-epas, which use it, rather than to server-mode, which
does not.
2026-09-18 11:33:50 +01:00
Dave Page 992ce03b43 Assemble on roadie, alongside the publishing
Splitting the two meant assembling a tree of several hundred megabytes
on a hosted runner, uploading it as an artifact, and immediately
downloading it again on roadie, which is where it was going anyway. The
hosted runner also has to be waited for: the second end-to-end run had
every build finished and then sat queued for assemble behind nine
unrelated master workflows.

Roadie is dedicated and nothing else queues for it, so the job now runs
there start to finish. Only the publishing steps are guarded by the
publish input, so a dispatch without it still proves the tree assembles,
which was the other reason for keeping them apart.

What goes is the assembled tree as a downloadable artifact. It was a
copy of something that ends up on the download server, where it can be
looked at directly.
2026-09-18 11:23:08 +01:00
Dave Page 14b3137ef1 Refuse a public key where a private one is needed
The pre-flight check tested only that the key file was readable, which a
.pub file also is, so a variable naming the wrong half of the pair passed
here and failed later inside ssh with a message that does not say which
file it objected to. Both halves live in one directory with names a
suffix apart, so it is an easy thing to get wrong and a cheap thing to
catch.
2026-09-18 09:18:44 +01:00
Dave Page 1f6076f888 Put every RPM in its distribution's tree, noarch included
The first end-to-end run assembled yum/redhat/rhel-9-noarch and three
more like it, because the architecture was taken from each package's own
filename. pgadmin4 and pgadmin4-web are noarch, so on a real publish the
metapackage and the web package would have been missing from every
repository: dnf asks for <name>-<version>-$basearch and never looks in a
noarch directory. The live snapshots have them in rhel-9-x86_64
alongside the rest, which is what this now produces.

x86_64 is a constant here because that is the whole RPM matrix today; it
becomes a property of the build when aarch64 arrives.

Found by diffing the assembled tree against a snapshot the buildfarm
produced, rather than by the build failing, which it would not have.
2026-09-17 16:58:09 +01:00
Dave Page 2a4492ea5d Bring the platform build READMEs up to date
The Windows one still pointed at dpage/winpgbuild and told people to dig
a binaries asset out of a workflow run. pgbuild publishes proper releases
now, each tagged *-latest, so both platforms get a predictable URL and
the same binaries CI uses. syft is added to the Chocolatey list, the
build having needed it since the SBOM step went in.

The macOS one claimed Python 3.6+, which has not been true for a while
and is now doubly confusing given the build downloads the interpreter it
bundles: the two are separated, and the version that ships is pointed at
pkg/python-version.txt. It also never mentioned syft or wget, both of
which the build needs, or that Homebrew's PostgreSQL lacks the zstd and
lz4 support ours is built with.
2026-09-17 15:14:09 +01:00
Dave Page f703f10d83 Give each workflow its own concurrency group
github.workflow in a reusable workflow is the CALLER's name, so every
build Snapshot calls landed in one group, 'Snapshot @ <ref>', with
cancel-in-progress set. Each called build therefore cancelled the one
before it: the first dispatched run finished with deb, rpm, docs,
tarball, wheel and helm all cancelled, and only the two jobs defined
inline in the orchestrator surviving.

Each workflow now names its own group literally.

Also restores snapshot-macos.yml, which the previous commit truncated to
its trigger block: the expression used to strip the push trigger matched
across the rest of the file and took the concurrency settings, the
permissions and the entire appbundle job with it.
2026-09-17 15:07:32 +01:00
Dave Page f7cd7f4d63 Run snapshots nightly rather than on push
A push trigger was never the intent; it was there because a workflow off
the default branch has no dispatch button, and it is what caused the
orchestrator to run alongside the standalone signing workflows it calls.

Nightly at 01:01, which is when pgadmin4-all-snapshot ran, plus
workflow_dispatch for testing and reruns.
2026-09-17 15:06:05 +01:00
Dave Page 6f7685a111 Stop the signing workflows firing on push as well
They each had a push trigger, added when they were standalone and there
was no dispatch button for a workflow off the default branch. Now that
the orchestrator calls them, a push touching pkg/mac or pkg/win32 starts
them twice: once directly and once through Snapshot. The first push
after they became callable did exactly that, and the resulting contention
for the macOS runners and the one Windows signing host left the real
dispatched run queued behind its own duplicate.

They keep workflow_dispatch, so either can still be run alone.
2026-09-17 14:58:21 +01:00
Dave Page 2d8479feab Build the repository configuration RPMs for the snapshot
Every snapshot on the download server carries
pgadmin4-fedora-repo-2-1.noarch.rpm and its redhat counterpart at the
root of yum/, and the README tells people to install one before anything
else. Nothing in the new flow produced them, so install-repo-readme.py
could not name them and snapshot-index-yum would have failed.

They are not a build of pgAdmin, which is why no QA workflow makes them:
each is a single .repo file pointing at the tree it ships in. That is
also why the destination has to be known before they are built, and why
the directory name is now decided once by its own job rather than by
each job calling date. Two jobs calling date independently would, on a
run straddling midnight, have shipped a repository configuration
pointing at a snapshot that does not exist.

They are left unsigned here and signed on the server with everything
else, since they land under yum/ where snapshot-sign finds them.
2026-09-17 14:50:34 +01:00
Dave Page e46d9cb08e Match the layout the servers actually use
Verifying against paxsor and procyon before the first run turned up that
the flow had the wrong shape, which would have produced a snapshot
nobody could install from.

A staging build and a snapshot are flat: apt/, yum/, and the
downloadable files loose beside them. It is only a published release
that is divided into docs/, macos/, pip/, source/ and windows/, and the
promotion job sorts files into those on the way to v<VERSION>. We had
the release layout applied to all three, so stage-create and
snapshot-create made directories nothing uses, the assembled tree put
every file one level too deep, and sign_packages looked for things to
sign in directories that would have been empty. A snapshot would have
gone out unsigned.

Signing is now driven by extension and location rather than by a list of
directories, so it is right for both layouts: RPMs under yum/ signed in
place, and detached signatures for the tarball, wheel, documentation
tarball, PDF and ePub wherever they sit. Debs are left alone, apt
verifying them through the signed Release file, as are the disk images,
ZIPs and installer, which carry signatures of their own.

The documentation also gets versioned filenames, the build leaving them
as pgadmin4.pdf and pgAdmin4.epub whilst the download site has always
carried pgadmin4-<version>.pdf and .epub.
2026-09-17 14:45:37 +01:00
Dave Page df76b1ba34 Build the RPMs against PostgreSQL 18
pkg/redhat/build.sh bundles postgresqlNN-libs into the yum trees,
choosing NN from the highest /usr/pgsql-1* on the build host rather than
from a constant, so the version that ships is decided by what the build
environment installs. Ours installed postgresql17-devel, so the
repositories carried postgresql17-libs alongside a libpq5 from 18.

pkg/redhat/setup.sh was staler again at postgresql15-devel, in all three
of its distribution branches.
2026-09-17 14:45:37 +01:00
Dave Page 6920a58d91 Do not enforce snapshot retention on a test run
snapshot-purge ran at the end of every publish, so the first test run
would have deleted the older snapshots the buildfarm put on the download
server, which is exactly what we agreed a test must not touch. A run
that names its own directory is a test by definition, so that is the
signal to leave retention alone.

Also fails rather than guessing when the artefact does not contain
exactly one dated directory, and checks for ssh and rsync up front.
2026-09-17 14:16:56 +01:00
Dave Page 1545bcc91c Send snapshots to the download server, not to staging
I had the two flows conflated. Release candidates are staged on the
development server and promoted; snapshots are published straight to the
download server under snapshots/<date>, which is what every
*-snapshot job in the buildfarm targets.

So the publish job talks to the download server, the wrapper grows the
snapshot verbs it needed and loses snapshot-fetch, which existed only to
copy a staging build that is never made, and the variable is named for
the server it actually names.

It also reverses my advice about the upload key on the download server,
which I had said to drop on the reasoning that publication always pulls
from procyon. It does not, for snapshots. The key comes back, confined
to the snapshots tree, which is the right root on two counts: it is
where snapshots belong, and it is the one part of the download site that
the S3 sync excludes, so nothing written through it can reach the
archive or the mirrors.

snapshot-purge replaces the buildfarm's ls | tail | xargs rm. It is the
only verb that deletes anything, so it takes no argument: it cannot be
asked to remove a particular snapshot, only to enforce the retention the
server already has, and it ignores directories that do not look like a
snapshot.
2026-09-17 13:25:06 +01:00
Dave Page e4eb334599 Reference the publishing keys by path rather than copying them
They are already on roadie. Putting copies in the secret store would
mean two places to rotate them and two places to leak them from, so the
workflow names the paths and never handles key material.

What changes with it is where the boundary sits. An environment secret
is gated by a deployment branch policy; a file on disk is gated by which
workflows may run on that machine. This repository is public, so roadie
needs to be in a runner group restricted to these workflows, with
nothing triggered by pull_request targeting it. That is the same bargain
the Windows signing host already makes with the Certum token, but it is
now load-bearing in a second place, so it is written down.

The environment stays for the branch policy and the deployment record.
2026-09-17 13:19:13 +01:00
Dave Page 9f7dff40f1 Publish the assembled snapshot to the staging server
The last missing link: assemble produced a tree and nothing carried it
anywhere. The publish job uploads it over the rrsync-confined key and
then asks pga-publish to sign and index it, so the runner names a verb
and a datestamp rather than a command, and the signing happens where the
key is.

The staging directory can be named explicitly, because the buildfarm
writes <date> on that same server and a test run has no business landing
on the directory Jenkins is still filling. Publication is off by default
for the same reason: it takes an explicit input, so a dispatch that only
wants to know whether everything builds cannot reach the servers at all.

Also adds the deployment notes, which is where the four files go on each
host, what the two configuration files hold, and why no key gets a
shell.
2026-09-17 13:13:04 +01:00
Dave Page 596396acaf Move the publishing wrapper into pkg/publish and correct it
It was drafted from the Jenkins job definitions before anybody had
looked at the servers, and most of what it assumed about them was wrong.

The apt layout is per distribution release, with the packages sitting
directly in binary-<arch> and no shared pool, so packages-fetch and
stage-create were writing to paths that do not exist. The allowlists
were stale in both directions: plucky and RHEL 8 are not built, whilst
resolute, Fedora 43 and 44 were missing. sync-ftp-to-s3.py takes the
CloudFront distribution to invalidate rather than a choice of what to
sync, so that verb loses its argument entirely and reads the identifier
from the server, since the client has nothing to choose.

The indexing is no longer implemented here. It calls the scripts in this
directory instead, which are the same ones an administrator runs by hand
after purging old releases, so there is one implementation rather than
the four the buildfarm had. That also settles the doubled signing the
draft worried about: the scripts sign as part of indexing.

New verb: stage-sign. With the GPG key staying on the servers rather
than moving into Actions secrets, the workflow uploads unsigned
artefacts and asks for them to be signed here. A compromised workflow
can therefore ask for a signature over something it just uploaded, which
is real and acknowledged; what it cannot do is hold the key, sign out of
band, or sign with any other key.

The self-test covers 56 cases and passes.
2026-09-17 13:10:13 +01:00
Dave Page 3c038f8fd5 Add the snapshot orchestrator, calling the build workflows
Replaces pgadmin4-all-snapshot and the dozen per-artefact snapshot jobs
it drove. Those were separate job definitions carrying the same commands
as their QA counterparts, which is how a snapshot comes to differ from
what QA tested; here the build workflows gain a workflow_call trigger
and the orchestrator calls them, so there is one definition of each
build.

Nothing in it signs anything beyond the macOS notarisation and the
Windows signature, which have to happen where their credentials are. The
GPG key that signs RPMs and repository metadata stays on the servers, so
the workflow assembles the tree and uploads it, and procyon indexes and
signs it. A compromised run can ask for a signature; it cannot make one.

The assembled layout is the one the publishing scripts and the download
site expect. Debian architectures come from the package filenames, since
dpkg encodes them there; the codename does not follow from the build
name, so that mapping is explicit and fails loudly on an unknown one.

The Helm chart is built but not assembled: it goes to an OCI registry
rather than the download tree.
2026-09-17 12:49:54 +01:00
Dave Page f149cf4b81 Derive the repository README platform tables
The support matrix was maintained by hand in five Jenkins heredocs,
which is exactly why the published one had drifted: it stops at Noble
and still lists Mantic, whilst the job that writes it goes up to
Resolute.

It is now derived, and the two sources are deliberately different. Which
platforms are supported comes from the repositories present in the tree
being written, since that is what a user can actually install from.
Which releases each platform carried comes from the S3 archive rather
than the download server, because old releases are purged from the
server as they age: bookworm reads as 7.8 there against 7.0 in the
archive, so deriving history from the server would quietly rewrite it
every time somebody tidied up. The sync has no --delete, so the bucket
keeps everything.

Platforms that are no longer supported get their own table with the last
release each carried, which turns the standing note about the archive
into something a user on an old distribution can act on. For snapshot
and pre-release trees --no-archive drops that section and the release
columns with it, since every package there is from the one build.

Cross-checked against the hand-written tables: every value they assert
is reproduced, including bookworm 7.0, jammy 6.9, bullseye 4.30 to 9.14
and fedora-42 9.3 to 9.17. Two they got wrong come out right, RHEL 10
having been copied from the RHEL 9 row as 6.12 when it is 9.6.

Two traps worth naming. aws s3 sync follows symlinks, so the EL
compatibility links exist in the bucket as full copies and would
otherwise invent platforms like fedora-31Client. And Debian versions
carry a packaging revision now, as in 9.16-1.resolute, which sorts below
9.3 unless it is stripped first.
2026-09-17 12:41:22 +01:00
Dave Page 7ea9a6bb35 Bring aptftp.conf across from the download server
It lived unversioned in pgaupload's home directory on paxsor, which also
made it writable by the account the publishing scripts run as, despite
being an input to what gets indexed and therefore to what gets signed.

Taking it here settles both: changing the repository's Origin, Label or
architectures becomes a reviewed commit, and the copy on the server can
become root owned.

The default architecture list drops i386 to match. apt-ftparchive has
been writing dists/pgadmin4/main/binary-i386/Packages for years, but
Architectures in the Release file has only ever said 'amd64 all', so no
client has been able to reach that index through the signed metadata.
Nothing that works today stops working.
2026-09-17 12:21:38 +01:00
Dave Page 2db86f9d0a Drop the bash step from the pgbuild dependency action
The action ran a final step whose only job was to publish whichever of
the two platform steps' outputs had been set, and it did that in bash.
The hosted Windows image has bash by way of Git for Windows; the signing
host does not, so the Windows snapshot build failed with 'bash: command
not found' before it reached anything interesting.

An expression does the same job without needing a shell to exist.
2026-09-17 12:14:14 +01:00
Dave Page 5a694d2f48 Add pkg/publish: one implementation of the repository indexing
The sequence that indexes and signs an APT repository existed in four
places that differed only in which directory they ran against: inline in
the pgadmin4-deb-build, pgadmin4-deb-snapshot and
pgadmin4-deb-publish-qa-to-prod Jenkins jobs, and in rebuild-apt-repo.sh
in the pgaweb repository. The YUM equivalent existed in five, and the
repository READMEs were heredocs in five more jobs.

These are those sequences, once each, with the tree root as an argument
so that the staging, snapshot and production trees share one
implementation, and with the architecture parameterised rather than
x86_64 spliced into the paths, since we will want to publish for arm64
before long. The APT script passes the architecture list to
apt-ftparchive as an override rather than letting it come from
aptftp.conf, so the set indexed and the set advertised in the signed
Release file cannot drift apart; a package whose architecture is missing
from Release is ignored by every client, silently.

Run with no options they behave exactly as the pgaweb copies did, so
rebuilding a repository by hand after purging old releases is unchanged.

aptftp.conf still needs to be brought across from the download server.
2026-09-17 12:13:32 +01:00
Dave Page 432d74bf1b Decide the Python version in one file rather than three places
Which Python pgAdmin builds against and ships has until now been settled in
three unrelated places. On Windows, Make.bat used whatever %PGADMIN_PYTHON_DIR%
pointed at, which the buildfarm's Jenkins node configuration injected, and then
derived the embeddable distribution it actually ships from that interpreter's
own sys.version. On macOS, pkg/mac/build.sh defaulted to a version of its own
whilst the workflows overrode it with a third, copied from the buildfarm's
DEFAULT_PYTHON_VERSION. Locally it was whatever the developer happened to have.
A single commit could therefore ship different interpreters on different
platforms with nothing anywhere announcing it, which nearly happened: a workflow
that discovered Python from the PATH would have shipped 3.14 on Windows against
3.13 on macOS.

The version now lives in pkg/python-version.txt, which every platform's
packaging reads. It is a file in the tree rather than a GitHub variable because
a local build has to read it too, and because a change of interpreter should
arrive as a commit somebody can review rather than as a settings change made on
an agent. The format is KEY=value with # comments, dull on purpose: it has to be
parsed by a batch file, by a shell script and occasionally by a person, and
anything richer would need a parser in at least one of those. It carries both
the exact CPython version and the python-build-standalone release tag, since
that project's releases are dated rather than versioned and the archive names
embed both.

Make.bat no longer depends on an installed Python at all. It downloads the
relocatable distribution named in that file, unpacks it into the build tree and
installs virtualenv into it, then fetches the matching embeddable distribution
from python.org for what actually ships, so the two can no longer disagree.
python-build-standalone suits the signing host particularly well, being a plain
tarball rather than an installer and so raising no UAC prompt, which matters
because an elevation dialog in that machine's auto-logged-in session waits
forever for somebody who is not there. curl.exe and tar.exe have shipped in
Windows since 1803, so nothing needs installing to bootstrap it. Both Windows
workflows consequently lose their Python setup, and the signing host needs
neither a Python, nor a virtualenv installed into it, nor the
PGADMIN_PYTHON_DIR repository variable.

macOS only reads the version. It still builds its framework with
relocatable-python, because python-build-standalone's relocatable distributions
are not framework-shaped and replacing what is shipped there would mean
reworking the codesigning and the @loader_path rewriting in _fixup_imports,
which is a job of its own.

The version itself is unchanged: 3.13.15 is what the released installers
already carry, and this makes that existing choice explicit rather than a
different one.
2026-09-17 10:16:50 +01:00
Dave Page 892681f78d Queue rather than cancel concurrent macOS signing builds
Copied from the QA builds without thinking, where cancelling an in-flight run
on a new push is exactly right: the tip is what matters and a superseded build
is wasted effort.

A signing build is not that. Cancelling one throws away forty minutes of work,
and worse, it can abandon a notarisation already submitted to Apple, leaving a
request in their queue for a build nobody will ever collect. The Windows
snapshot already queues rather than cancels, for the related reason that there
is one hardware token, so this brings the two into line.

Found by cancelling my own in-flight Intel notarisation with an unrelated push.
2026-09-16 15:48:23 +01:00
Dave Page ad5b6ae2c7 Extract the pgbuild dependency install into a composite action
Four workflows fetch these — the QA and snapshot builds on each of macOS and
Windows — and the duplication has already cost something: the first version
used the gh CLI, which turned out not to exist on the self-hosted Windows
runner, and the same fix had to be made in two places. There will be more of
them as the snapshot and release builds fill in.

The action carries the parts that are easy to get wrong and tedious to
rediscover. On macOS the archives must land in /opt/pgbuild precisely, because
the install names inside are absolute and _fixup_imports copies from those
paths whilst rewriting the references. On Windows the location only has to
avoid the C:\build64 that the buildfarm's Jenkins worker deletes and
re-extracts, hence the runner's temp directory. Downloads are plain HTTPS
rather than a CLI, since pgbuild is public and its URLs are predictable, and
both curl --fail and Invoke-WebRequest fail on a 404 rather than writing an
error page to disk and failing later at the unpack.

Deliberately not reusable workflows, which was the first idea. The signed
builds need environment: signing and the QA builds must not have it, and there
is no clean way to express "no environment" as an input, so factoring at that
level would have meant either handing the QA builds access to the signing
certificate or contorting around it. The duplication that actually hurt was the
steps rather than the workflows.
2026-09-16 15:47:46 +01:00
Dave Page 5326c1524f Take the signing credentials from a "signing" environment
"snapshot" was the wrong name for the boundary. The same Developer ID and
notarisation credentials serve both the snapshot and the release builds, so
naming the environment after one occasion would have meant either duplicating
the secrets or referring to "snapshot" from a release workflow. Naming it for
what it holds rather than when it runs avoids both.

Worth knowing that the old environment was never created deliberately: GitHub
creates one the first time a workflow references it, so an environment
appearing in the settings is not evidence that anyone intended it.
2026-09-16 14:57:50 +01:00
Dave Page 96b72d76f1 Fetch the Windows build dependencies without the gh CLI
The signed build failed with CommandNotFoundException on gh. Hosted runners
ship the GitHub CLI; a plain Windows machine does not, and the signing host is
the one machine in the pipeline that cannot be rebuilt, so the fewer things it
must have installed the better.

pgbuild is public and its asset URLs are predictable, so this needs neither a
CLI nor authentication. Invoke-WebRequest throws on a 404, so a missing release
still fails immediately rather than writing an error page to disk and failing
later at Expand-Archive, which was the reason for preferring gh in the first
place.

The hosted QA build changes the same way, though it has gh and did not need to.
One mechanism for the same job is worth more than the small convenience,
particularly if that build ever moves to a self-hosted runner.

ProgressPreference is silenced because Invoke-WebRequest otherwise spends most
of a 90 MB download rendering a progress bar into a log nobody will read.
2026-09-16 13:58:09 +01:00
Dave Page d37e8fe798 State which Python the Windows build uses rather than discovering it
The previous attempt took whatever python was on PATH, which was wrong in a way
that would not have announced itself. The signing host has 3.14 on PATH, whilst
the buildfarm's Jenkins job injects PGADMIN_PYTHON_DIR pointing at 3.13, so the
two systems would have built the same commit against different interpreters and
shipped different runtimes, with nothing failing to indicate it.

Which Python we ship is a decision. It now comes from a repository variable,
set to match Jenkins, so it is stated once rather than inherited from whichever
Python was installed on a machine most recently. The build fails if the
variable is unset rather than falling back to something plausible, because a
plausible fallback is exactly how this went unnoticed in the first place.

This is the interim arrangement. The version belongs in a file in this
repository, with each platform fetching a private Python of that version at
build time, so that local builds cannot drift either and the Windows host needs
no system Python at all. That wants doing before the system Python goes rather
than after, and deserves its own change rather than being done in the middle of
getting signing working.
2026-09-16 13:52:30 +01:00
Dave Page b60ac47346 Use Windows PowerShell rather than pwsh on the signing host
The second run failed immediately with "pwsh: command not found". PowerShell 7
is not on that machine: GitHub's hosted Windows images ship it, a self-hosted
Windows box has only Windows PowerShell 5.1 unless somebody installed it, and
nothing should be installing it on this machine to suit a CI job.

Nothing here needs PowerShell 7, so the steps use the shell that is present.
One thing does change with it, and it is the sort of difference that produces a
baffling failure rather than an obvious one: Out-File in 5.1 defaults to
UTF-16, so writing to GITHUB_ENV without an explicit encoding produces a file
the runner cannot parse and every variable set that way silently goes missing.
Hence -Encoding utf8 on each, which is what GitHub's own documentation
prescribes for this shell.

The hosted Windows QA build keeps using pwsh, since hosted images have it and
it is the better shell where there is a choice.
2026-09-16 12:59:27 +01:00
Dave Page 5dcf165a0a Use the signing host's own Python rather than actions/setup-python
The first run on the self-hosted runner hung on setup-python for thirty
minutes, against eleven seconds for setup-node, and had to be cancelled. On
Windows that action installs Python rather than unpacking it, and anything
raising a UAC prompt in the auto-logged-in session waits forever with nobody to
answer it. Which is the PIN dialog problem wearing different clothes, and worth
remembering as a property of this machine: anything that might prompt will hang
rather than fail.

The machine has a Python already, which is what the buildfarm's Jenkins job
uses, so the workflow finds it on PATH and derives PGADMIN_PYTHON_DIR from
that. It is also closer to what we are actually testing here, since the point
of building on this box rather than a hosted runner is to exercise its
environment.

virtualenv is checked rather than installed, consistent with the other
prerequisites: a CI job should not be modifying the one machine in the pipeline
that cannot be rebuilt, so it stops with the command to run instead.
2026-09-16 12:57:57 +01:00
Dave Page b6ed164952 Document the Windows signing host in pkg/win32
The signing host is about to become the only machine in the release pipeline
that cannot be rebuilt from something. Everything else either disappears or
becomes an ephemeral runner described entirely by a workflow file; this one is
configured by hand and holds a certificate on a hardware card that cannot be
copied or exported.

What documentation existed lived in the buildfarm repository, which is being
retired, and was written for the Jenkins arrangement. It also opened by saying
it was "preserved for reference only" because the configuration was "now
managed by Ansible roles", which was never true of this machine.

So this consolidates both notes into one document next to the build they serve,
rewritten for the GitHub Actions arrangement: the runner in an interactive
session rather than as a service, the Startup folder shortcut, the PIN helper,
the prerequisites the workflow checks for, and the PGADMIN_WINDOWS_CSC
repository variable.

The Certum material is preserved largely as it was, because it is hard-won and
none of it changes: CSP mode rather than Minidriver, importing to the machine
store, and above all the certificate-to-key repair via keyprov.inf, which the
card's own registration does not do correctly and without which signing either
fails or quietly degrades to SHA1. The troubleshooting section is kept for the
same reason, particularly that NTE_KEYSET_NOT_DEF almost always means someone
is on Remote Desktop.

Everything specific to an individual, a machine or a certificate stays out:
subject names, thumbprints, container IDs and passwords are placeholders, as
they were in the originals.
2026-09-16 12:25:27 +01:00
Dave Page 68536e70ec Add the signed Windows snapshot build
Replaces the buildfarm's pgadmin4-windows-x64-snapshot job, and it is the one
workflow that cannot use a hosted runner: the certificate is on a Certum
hardware token, reached through the machine certificate store, with a PIN
handler answering the prompt from the interactive desktop session. None of that
can move to a runner GitHub provides.

Setting PGADMIN_WINDOWS_CSC is the whole of what makes Make.bat sign, so the
build fails immediately if the variable is absent rather than quietly producing
an unsigned installer that looks like a successful snapshot. It is a repository
variable rather than a value in the tree: it is a person's name, and a
certificate renewal should be a settings change rather than a commit.

Three things differ from the QA build beyond the signing.

Nothing is installed. The hosted build runs choco; this machine is provisioned
by the buildfarm's Ansible and a CI job has no business mutating it, so the
prerequisites are checked and the build stops with a useful message if one is
missing.

Visual Studio, the VC++ redistributable and signtool are discovered rather than
hardcoded. The buildfarm pins all three in its Jenkins node configuration,
which then needs editing every time Visual Studio or the Windows SDK updates,
and signtool in particular lives at a versioned path.

Concurrency is a fixed group with cancel-in-progress off. There is one hardware
token, Jenkins may also be using it, and two builds signing through it at once
is a good way to discover what proCertum does under contention.

The verification step checks the executable inside the installer as well as the
installer itself. Make.bat verifies only the latter, and the former is the
thing users actually run.
2026-09-16 12:20:46 +01:00
Dave Page 65557f1814 Trigger the new workflows on the working branch
These all trigger on pushes to master, which means that on a working branch
they never run and the branch cannot be tested at all. So the branch is named
alongside master in the triggers, to be removed when it merges.

The snapshot build needs different treatment. workflow_dispatch only offers a
"Run workflow" button for workflows that are on the default branch, so whilst
it lives on a branch there is no way to trigger it by hand at all. A plain push
trigger would mean signing and notarising on every commit, which is both
wasteful and rude to Apple's notary service, so it fires only on changes to
itself or to pkg/mac, which is to say only when the thing under test changes.
2026-09-16 12:04:32 +01:00
Dave Page c436287365 Remove the DMG diagnostic spike
Its own header said to delete it once the cause was understood, and it is: the
unmount cost scales with the bundle, hdiutil's 120 second DiskArbitration
timeout cannot be raised, and detach -force waits on DiskArbitration exactly as
the polite detach does. The build now creates the image with hdiutil create
-srcfolder and never mounts a volume at all.

What the spike established is recorded where it will actually be read, in the
comments on _create_dmg and in the commit that replaced create-dmg, rather than
in a workflow nobody will run again: create-dmg's flags, the volume size, the
file count, Gatekeeper, Spotlight and the choice of runner image were each
tested and each exonerated.
2026-09-16 12:03:47 +01:00
Dave Page 473fe47895 Pass --deep when signing ad-hoc
The first ad-hoc run failed on the very first binary:

    Contents/MacOS/pgAdmin 4: replacing existing signature
    Contents/MacOS/pgAdmin 4: code object is not signed at all
    In subcomponent: Contents/LICENSE

The Developer ID path passes --deep on every codesign call and the ad-hoc one
did not. These are bundles with nested code, and without --deep codesign will
not replace the signature on one whose nested code is unsigned. Matching the
existing path is both the fix and the smaller surprise for anyone reading the
two branches side by side.
2026-09-16 12:03:20 +01:00
Dave Page c76fd6b3ee Add the signed, notarised macOS snapshot build
Replaces the buildfarm's pgadmin4-appbundle-snapshot job. Where the QA build
proves the bundle assembles, this produces something distributable: signed with
the Developer ID, notarised by Apple, stapled, and with a ZIP alongside the DMG
because the Electron auto-updater consumes the ZIP whilst the DMG is the
download people click.

The buildfarm keeps the Developer ID in the worker's login keychain and unlocks
it with devkeys/unlock-csc.sh. A hosted runner has no such keychain, so one is
created for the job and destroyed with it, under a generated password that
never outlives the run. set-key-partition-list matters more than it looks:
without it codesign asks permission to use the key, and on a headless runner
asking means hanging until the job times out.

The credentials sit on a "snapshot" environment rather than loose on the
repository, so only workflows that ought to be signing can reach them, and the
cleanup step runs even on failure, since neither a keychain holding a Developer
ID key nor a conf file holding an app-specific password should survive a failed
build.

Nothing is uploaded yet. This proves the signed build works; publishing is the
next piece. It is also workflow_dispatch only rather than nightly, so it cannot
start signing things before we intend it to.

Requires six secrets on the snapshot environment: MACOS_CERTIFICATE (the
Developer ID .p12, base64 encoded), MACOS_CERTIFICATE_PASSWORD,
MACOS_DEVELOPER_ID, APPLE_ID, APPLE_APP_SPECIFIC_PASSWORD and APPLE_TEAM_ID.
2026-09-16 12:03:20 +01:00
Dave Page 01cd0454a3 Unpack the Windows build dependencies under the runner's temp directory
C:\build64 was the obvious place to put them and the wrong one. It is what the
buildfarm's winpgbuild-update job manages on the Windows worker, where it
deletes and re-extracts postgresql and krb5 inside that directory.

That does not matter whilst these builds only run on hosted runners, which are
ephemeral and have no Jenkins on them. It matters a great deal as soon as the
signing builds run on that same machine as a self-hosted runner, because the
two systems would then take turns overwriting each other's dependencies, and
the Jenkins-side failures would look nothing like their cause: a build that
worked yesterday, with a PostgreSQL that is subtly not the one it was given.

So the path is derived from RUNNER_TEMP and passed on through GITHUB_ENV rather
than being a fixed constant. Nothing else changes, and QA stays on the hosted
runner.
2026-09-16 12:03:20 +01:00
Dave Page 778e00fc3b Sign the macOS bundle ad-hoc when there is no Developer ID
Without codesign.conf the build skipped signing entirely, which leaves a bundle
that cannot be run rather than merely one that cannot be distributed.

_fixup_imports rewrites install names across several hundred binaries, which
invalidates whatever signatures they arrived with, Electron's included. On
Apple Silicon the kernel refuses to execute a binary whose signature is broken
or absent, so an unsigned bundle will not launch at all there, even once the
quarantine attribute has been removed. On Intel it is less severe but no more
correct: every binary ends up carrying a signature that no longer matches its
contents.

So with no Developer ID we now sign ad-hoc. This is not a substitute for a real
signature and is not meant to be: the result is still refused by Gatekeeper and
still cannot be notarised, and the build says so. It just means an unsigned
build is something a developer can actually open after allowing it, which is
the entire point of producing one.

The signing arguments move into a helper so the two paths cannot drift, and the
dylib loop stops using find -exec so it can share them. Signing the disk image
ad-hoc would achieve nothing, since a signature there is about distribution, so
that one still does nothing without a Developer ID.

Worth noting this path had never run anywhere before: the buildfarm's macOS QA
job signs properly, because it runs on a machine with the certificate, so the
unsigned branch existed only in principle. The same was true of the Windows
installer's unsigned path until last week.
2026-09-16 12:03:20 +01:00
Dave Page 6a287fdb87 Stop installing the Python dependencies twice in the macOS build
_create_python_env installs the whole of requirements.txt into the framework
that gets shipped, and then _build_docs created a fresh venv and installed all
hundred-odd packages again. Both installs show up in every build log as
identical "Successfully installed Authlib-1.8.0 Babel-2.18.0 ..." lines a few
hundred lines apart.

The second install was not gratuitous: the doc build imports pgAdmin itself,
since docs/en_US/build_code_snippet.py pulls in config, pgadmin.utils,
pgadmin.browser.utils and the driver abstractions, so the dependencies have to
be importable. But they already are, in the framework populated moments
earlier, so the venv can inherit them instead. Sphinx still goes into the venv
rather than the framework, which is the reason the venv exists at all: it must
not end up in the shipped bundle.

--system-site-packages is safe here in a way it is not in the Linux build,
which deliberately avoids it. There the parent is a shared system Python that
may carry other packages' stale .pth files, and one of those corrupted sys.path
badly enough to break pip's own bootstrap on the el-10 builder. Here the parent
is a framework we built ourselves in this run, containing only our
dependencies. The comment says so, to save someone importing the Linux
workaround into a situation that does not call for it.

Windows already avoided the duplication by adding sphinx to the same
environment, though without the separation that keeps it out of the shipped
files.
2026-09-16 12:03:20 +01:00
Dave Page 5b1d7ec954 Build the Windows QA installer against pgbuild's dependencies
pgadmin-org/pgbuild supersedes dpage/winpgbuild, and now that a full
build-all-windows run has published the Windows artefacts, this points at them
instead. Both platforms then build against a single manifest of pinned
versions, which is the main reason for merging the two repos in the first
place: until now the macOS and Windows dependency versions were kept in step by
hand, in two places, with nothing enforcing it.

Verified against what Make.bat actually needs before switching. The krb5
archive unpacks to krb5/ and carries kinit.exe, krb5_64.dll, comerr64.dll,
k5sprt64.dll and gssapi64.dll; the PostgreSQL one unpacks to postgresql/ and
carries libpq.dll, zlib1.dll, the client binaries, libcrypto-3-x64.dll and
libssl-3-x64.dll, along with the optional liblz4.dll, libzstd.dll, libintl-8.dll
and libiconv-2.dll. So the directory layout the workflow already assumed is
unchanged and every file the build copies is present.

Downloads now go through gh rather than Invoke-WebRequest, which keeps the tag
and the asset name in one place and fails loudly if a release is missing rather
than writing an error page to disk and failing later at Expand-Archive.
2026-09-16 12:03:20 +01:00
Dave Page 346e4deff4 Build the macOS QA appbundle against pgbuild's dependencies
Homebrew's PostgreSQL was only ever a stand-in for the build check, and a poor
one: it has no zstd or lz4 support, which is issue #9425, it links Homebrew's
OpenSSL rather than the one we ship, and it carries whatever minimum macOS
Homebrew happened to build for. So the QA build proved that the bundle
assembles, but not that the binaries in it are the ones we intend to ship.

The dependencies now come from pgadmin-org/pgbuild, which builds OpenSSL,
Kerberos, zstd, lz4 and PostgreSQL against each other with a 14.0 deployment
target, matching the support policy. That makes the QA build the same thing the
release build will produce, minus the signing, and it exercises those artefacts
continuously rather than discovering a problem with them at release time.

They are unpacked into /opt/pgbuild specifically, because the install names are
absolute: pg_dump asks for /opt/pgbuild/zstd/lib/libzstd.1.dylib and libpq for
/opt/pgbuild/krb5/lib/libgssapi_krb5.2.2.dylib, among others. _fixup_imports
copies from those paths into the bundle and rewrites the references, so they
have to resolve during the build.

Which is also why /opt/pgbuild joins the prefixes _verify_bundle_linkage
rejects. An unrewritten reference to it is precisely the defect that check
exists to catch, no different from a dangling Homebrew path, and it would
otherwise pass unnoticed: the shipped binary would look fine on the build
machine and fail on a user's.

OPENSSL_DIR now points at our OpenSSL too, so the Intel cryptography build,
which compiles from source because there is no x86_64 wheel, statically links
the same OpenSSL as everything else in the bundle.
2026-09-16 12:03:20 +01:00
Dave Page c049488a11 Build the macOS disk image without mounting it, and drop create-dmg
The DMG was built by mounting a read/write volume, populating it, unmounting it
and converting the result. The unmount is the step that does not survive
contact with a slow machine: hdiutil waits on DiskArbitration with a fixed 120
second timeout that nothing exposes a way to raise, and the cost scales with the
bundle. Measured on GitHub's hosted runners with an identical payload it takes
around 20 seconds on arm64, 41 on macos-26-intel and 90 on macos-15-intel, and
with the real bundle it exceeds 120 on both Intel images every time. It has been
seen intermittently on developers' own machines for the same reason.

`hdiutil detach -force` is no help, which was worth finding out the hard way
since it is usually described as the escape hatch: it waits on DiskArbitration
as well, times out identically, and the conversion then fails with "Resource
temporarily unavailable" because the image is still attached.

`hdiutil create -srcfolder` builds the image directly from a directory and never
mounts a volume, so none of this can happen. Verified on both architectures
against a real Electron bundle: the image is created and verifies, and mounting
it read-only afterwards shows the volume named correctly with the .DS_Store,
the background directory, the volume icon and the Applications symlink all
present. hdiutil preserves that symlink rather than following it. Creation takes
107 seconds on arm64 and 312 on macos-26-intel, slower than before, but with no
fixed wall anywhere in it.

That also disposes of create-dmg, which was cloned unpinned from its default
branch on every build and run against the signed bundle. Since we passed
--skip-jenkins, none of its Finder automation ran and everything it did for us
was staging, all of which is now done directly. Its EULA template is vendored
into pkg/mac/ rather than cloned for one XML file, and the licence is attached
with udifrez exactly as create-dmg did.

Ruled out before arriving here: create-dmg's flags, the volume size, the file
count, Gatekeeper, Spotlight and the choice of runner image. A 400 MB image of
30,000 text files and one holding Electron and a Python framework unmount in
comparable time, so only the speed of the machine and the size of the bundle
move the number.

One thing CI cannot check is whether the volume icon and background actually
render, since that needs Finder. The DMG is kept as a build artefact so it can
be opened and looked at.
2026-09-16 12:03:20 +01:00
Dave Page cde8851993 Prototype replacing create-dmg with hdiutil create -srcfolder
The recovery added earlier does not work, and the run proved it plainly:

    Forcing detach of /dev/disk22...
    hdiutil: detach: timeout for DiskArbitration expired
    hdiutil: detach: drive not detached
    hdiutil: convert failed - Resource temporarily unavailable

So hdiutil detach -force waits on DiskArbitration exactly as the polite detach
does, same 120 seconds and same message, contrary to how it is usually
described, and the convert then fails because the image is still attached. That
was the one assumption I could not verify beforehand, the spike having never
reproduced the failure and its diagnostics therefore never having run.

hdiutil create -srcfolder builds the image directly from a directory and never
mounts a volume, so DiskArbitration is never involved and there is nothing to
detach. Since we pass --skip-jenkins, create-dmg does no Finder automation for
us anyway: what it actually provides is the volume name, an Applications
symlink, the background directory, the pre-built .DS_Store and a hidden
extension, all of which are staging. The two that are not are the volume icon,
which needs the custom-icon flag on the volume root, and the EULA, which is
attached to the finished image rather than placed inside it.

Rather than learn which of those survive through forty minute Intel builds,
test F stages a directory exactly as the replacement would and creates the
image from it, then mounts the result read-only to confirm the styling is
really there. Mounting a finished compressed image briefly is cheap; it is
unmounting a large populated read/write volume that is not.

The dead recovery code in _create_dmg stays put until this proves out, so it is
replaced in one change rather than reverted and then rewritten.
2026-09-16 12:03:20 +01:00
Dave Page 4d76b6190e Recover the macOS DMG build when hdiutil's unmount times out
create-dmg builds a read/write image, mounts it, populates it, unmounts it, and
only then converts it to the compressed image we ship. The unmount is the
fragile step: hdiutil waits on DiskArbitration with a fixed 120 second timeout
that nothing exposes a way to raise, and the cost of unmounting scales with the
bundle.

Measured on GitHub's hosted runners with an identical payload, unmount to "Disk
image done" takes about 20 seconds on arm64, 41 on macos-26-intel and 90 on
macos-15-intel; with the real bundle it exceeds 120 on both Intel images, every
time rather than occasionally. It has also been seen intermittently on
developers' own machines, which is the same problem with a smaller multiplier.

Retrying create-dmg would achieve nothing, since a deterministic failure just
recurs. What matters is that by the time the unmount fails the working image is
fully populated, and the only step create-dmg did not reach is the conversion.
So on failure we force the volume off, which does not wait on DiskArbitration,
and perform that last step ourselves. The result is verified rather than
assumed, because this path only runs when something has already gone wrong.

The successful path is untouched: when create-dmg returns 0 we return
immediately and none of this executes.

A fair amount was ruled out before landing here. create-dmg's flags, the volume
size, the file count, Gatekeeper, Spotlight and the choice of runner image were
each tested and each exonerated: a 400 MB image of 30,000 text files and one of
the same size holding Electron and a Python framework unmount in comparable
time, and only the speed of the machine and the size of the bundle move the
number.
2026-09-16 12:03:20 +01:00
Dave Page e49593d005 Actually remove the Spotlight step
The previous commit message said this was going and then didn't remove it.
Spotlight was ruled out two runs ago: the step demonstrably disabled indexing
and the detach failed exactly as before, so the comment above it claimed a
cause that had already been disproved.
2026-09-16 12:03:20 +01:00
Dave Page e7027e2186 Test file count as the driver of the DMG detach cost, and move Intel to macos-26
Tests C and D pinned the cost to something about a real bundle and, more
usefully, showed how differently the three runners handle it. Unmount to "Disk
image done", identical payloads: 19s on arm64, 41s on macos-26-intel, 82s on
macos-15-intel, against hdiutil's 120 second timeout, and that was with a
lighter stand-in than the real thing.

So the Intel QA job moves to macos-26-intel, which has roughly double the
headroom and is the newer image we would end up on anyway, macOS 15 being the
first to retire. The mdutil step goes at the same time: Spotlight was ruled out
two runs ago and leaving a step behind whose comment claims it fixes something
is worse than not having it.

What C and D could not separate is content type from file count, and the second
looks the more likely of the two. Test B held ~409 MB in 201 files and test D
~415 MB in a few thousand; the real bundle holds a comparable amount of data
across far more files, since web/ alone is 4,626 of them, 2,043 being .sql
templates, with docs/ adding 773 and site-packages and the Python stdlib on top.
Thirty thousand is a conservative estimate.

Test E therefore puts the same ~400 MB into 30,000 small text files with no
Mach-O content at all. If the unmount climbs, the driver is the number of files,
not Gatekeeper or LaunchServices inspecting binaries, and no amount of signing
or scanning policy would have helped. It also tells us whether macos-26-intel
survives a realistic file count, which decides whether the runner change above
is a fix or merely a stay of execution.
2026-09-16 12:03:20 +01:00
Dave Page 2d7e8aedb1 Bisect the macOS DMG detach failure down to the bundle
The first pass of the spike came back clean on all three runners, which is more
informative than it sounds. Bare hdiutil detached a 100 MB image in six seconds
and a 1.5 GB one in under a second, and create-dmg with pgAdmin's exact flags
and real assets over a 409 MB payload exited 0 in forty-four seconds, on the
very runner label where the real build has now failed three times out of three.

So it is not the runner image, not hdiutil, not create-dmg's invocation and not
the size of the volume. What is left is the bundle. Test B's stand-in was a
random blob in Contents/MacOS with no Info.plist and no Mach-O, which the system
does not regard as an application at all, whereas a genuine bundle landing on a
freshly mounted volume gets picked up by LaunchServices and Gatekeeper in ways a
folder of noise does not.

Test C is therefore a real Electron.app, which is precisely what the real build
starts from, and test D adds a relocatable Python framework on top, that being
the other symlink-heavy thing the build layers on and the reason
_prune_dangling_symlinks exists. Neither installs anything through pip, so this
stays in the region of ten minutes rather than the forty the real build needs on
Intel to compile cryptography.

If C fails we have it narrowed to real app bundles; if C passes and D fails, to
the Python framework.
2026-09-16 12:03:20 +01:00
Dave Page d5c71d7063 Add a spike workflow to diagnose the macOS DMG detach failure
Three Intel appbundle runs have now failed identically, always exactly 120
seconds after create-dmg starts unmounting, with the bundle otherwise complete
and verified; arm64 has passed four times through the same code, and disabling
Spotlight changed nothing. Reproducing it through `make appbundle` costs forty
minutes a go, nearly all of it compiling cryptography on Intel, which is a
miserable way to test a hypothesis.

So this reproduces the failing step directly, in a couple of minutes, and
answers three questions at once: whether plain hdiutil fails or only create-dmg
does, whether it depends on image size, and whether it is specific to the
macos-15-intel image. arm64 is included as the control that is known to work.

When something fails it captures evidence rather than leaving us guessing:
hdiutil's own view of what is attached, lsof against the volume, the processes
that typically hold one, the diskarbitrationd and hdiutil unified log for the
preceding few minutes, and whether a forced detach succeeds where the polite
one timed out. That last one should be the most telling, since it distinguishes
something holding a reference from DiskArbitration itself not answering.

The mdutil step added to check-macos-build.yml stays for now so the two runs
remain comparable; it should come out once we know the real cause.
2026-09-16 12:03:20 +01:00
Dave Page 68a5f7781f Disable Spotlight indexing for the macOS build checks
The Intel appbundle build fails at the very last step, every time rather than
intermittently: the bundle is assembled, _verify_bundle_linkage passes, the SBOM
is written, the disk image is created and populated, and then create-dmg cannot
unmount it.

    Unmounting disk image...
    hdiutil: detach: timeout for DiskArbitration expired
    hdiutil: detach: drive not detached

Two runs, identical failure, and the gap between the unmount starting and the
timeout is exactly 120 seconds, so something is waiting out a timeout rather
than racing. Spotlight indexing the freshly mounted volume is the usual cause,
holding a DiskArbitration reservation that the detach then sits behind, and
nothing on a build runner wants a Spotlight index anyway.

arm64 has never shown this, so the step is harmless there rather than needed.
If this doesn't settle it, the next thing to try is macos-26-intel, and after
that a retry around the detach, which would mean changing pkg/mac rather than
the workflow.
2026-09-16 12:03:20 +01:00
Dave Page 35f3615644 Restore yum-utils to the RPM build container
Dropped by accident when the compiler packages were added, and pkg/redhat's
build.sh shells out to yumdownloader at line 229, near the very end. So the
container built the docs, ran webpack, produced the RPMs, and only then died
with a bare exit 127 about twelve minutes in, which is a slow and unhelpful way
to learn that a package is missing.

Also check for yumdownloader explicitly once the packages are installed, so the
same class of mistake fails immediately and says what is wrong. That matters
beyond this one slip: the dnf5 distributions (EL 10, Fedora 41 and later) get
yumdownloader via yum-utils' compatibility shims rather than from yum itself,
which is the sort of thing that changes under us without warning.
2026-09-16 12:03:20 +01:00
Dave Page c82e8038ee Fix the unsigned Windows installer build, and install syft for it
Two problems, one of them in the packaging rather than the workflow.

installer.iss.in has always carried an unconditional `SignTool=pgAdminSigntool`,
but Make.bat only defines that SignTool when PGADMIN_WINDOWS_CSC is set. So the
unsigned path, which Make.bat explicitly supports and announces ("Skipping code
signing"), cannot actually produce an installer: ISCC aborts with "Value of
[Setup] section directive SignTool is invalid". It has presumably never been
exercised, because the Jenkins Windows job runs on the box with the token
plugged in, where the variable is always set.

Both signing directives are now inside `#ifdef SIGNED`, and the signing branch
of Make.bat passes /DSIGNED next to the existing /SpgAdminSigntool=. The signed
build emits exactly what it emitted before; the unsigned build now works.

Second, and more mundane, syft isn't on the hosted image. That one was silent:
:GENERATE_SBOM redirects syft's output into sbom.json and returns 0 whatever
happens, so a machine without syft ships an empty SBOM rather than failing. The
buildfarm's Windows worker installs it via Chocolatey, so this does too.
2026-09-16 12:03:20 +01:00
Dave Page 997b6c52f9 Statically link OpenSSL into cryptography for the Intel macOS build
cryptography publishes arm64-only macOS wheels (checked against 50.0.1), so on
macos-15-intel pip builds it from the sdist and its openssl-sys crate links
whatever OpenSSL it finds on the build host, which on a hosted runner is
Homebrew's. _fixup_imports deliberately skips _rust.abi3.so, so the reference
survives into the bundle and _verify_bundle_linkage stops the build:

    ERROR: .../cryptography/hazmat/bindings/_rust.abi3.so links against
    build-host libraries:
        /usr/local/opt/openssl@3/lib/libssl.3.dylib
        /usr/local/opt/openssl@3/lib/libcrypto.3.dylib

That's issue #10123 reproducing exactly, and the release build already guards
against it by setting OPENSSL_DIR and OPENSSL_STATIC against its own OpenSSL.
This does the same against Homebrew's, which leaves no external reference at
all. The arm64 job was unaffected throughout, since it gets a wheel.

Worth noting for scheduling: the Rust compile makes the Intel build roughly
twice the length of the arm64 one, 50 minutes against 27.
2026-09-16 12:03:20 +01:00
Dave Page b6627315b4 Install wget for the Windows build check
wget isn't on the hosted Windows image, and Make.bat's Electron download is an
unbounded retry loop (Make.bat:316), so the build doesn't fail, it just spins on
error 9009 until the job hits its timeout. The buildfarm's Windows worker has
wget from the same Chocolatey line in pkg/win32/README.md, which is where this
one now gets it too.
2026-09-16 12:03:20 +01:00
Dave Page f1681d78d3 Fix the DEB and RPM container builds
Two faults from the first run, neither of them deep.

All nine jobs would have hit the git ownership check: the container runs as
root whilst the workspace is owned by the runner user, so git refuses to touch
it, and `yarn run bundle` shells out to git for the commit hash and dies with
exit 128. The DEB builds got all the way through webpack before falling over on
that, which at least confirms the rest of the environment is sound. Marking the
workspace as a safe directory after checkout is the usual remedy.

The RPM builds failed earlier and more prosaically, with no compiler in the
container: psycopg-c and gssapi are both built from source, since
pkg/linux/build-functions.sh passes --no-binary psycopg, and rpmbuild wants
redhat-rpm-config. The buildfarm picks these up from the @development-tools
group; they're listed individually here to keep the container small. The DEB
side already had build-essential, which is why it got so much further.
2026-09-16 12:03:20 +01:00
Dave Page b55f5997dd Update the GitHub Actions to their latest major versions
The runner has started warning about Node.js 20 actions being deprecated, which
covers every first-party action we use, since v4 of checkout and upload-artifact
still run on the Node 20 runtime.

So: checkout v4 to v7, upload-artifact v4 to v7, setup-node v4 to v7,
setup-python v5 to v7 and docker/build-push-action v6 to v7. Nothing behavioural
changes for us. The intervening majors are almost entirely the Node 24 runtime
move, with checkout v6 persisting credentials to a separate file, checkout v7
refusing to check out a fork's head for pull_request_target and workflow_run
(neither of which we use), and upload-artifact v7 adding an opt-in unzipped
single-file upload.

One thing to be aware of before we stand up a self-hosted runner for the signing
builds: checkout v5 and later need runner 2.327.1 or newer, so that host will
have to be reasonably current.

The SonarQube action is left pinned to its commit hash, which is deliberate.
2026-09-16 12:03:20 +01:00
Dave Page d0ce7ad552 Add DEB, RPM, macOS, Windows and Helm build checks
These are the five QA builds that the Jenkins buildfarm still owns, and between
them they're the last thing standing in the way of retiring the QA half of it:
the container, tarball, wheel and documentation checks already run here.

The DEB and RPM builds run in a container per distribution on a hosted runner
rather than on a dedicated EC2 VM per distribution, which is the same build in
a reproducible environment and nine fewer machines to look after. The package
lists come from the buildfarm's own Ansible packages_build role. PGDG is
mandatory for RPM, because pkg/redhat/build.sh opens by globbing /usr/pgsql-1*
under `set -e` and dies without it; the DEB build is happy with the distro's
libpq-dev.

macOS builds on both architectures and is already proven: an earlier spike ran
it on macos-15 in 22 minutes with webpack peaking at 2991 MB against its 4096 MB
ceiling and no swap used at all, so the 7 GB arm64 runners have adequate
headroom. That spike workflow is removed here, superseded by the real one.

Windows runs on a hosted runner too, which wasn't obvious: signing in Make.bat
is conditional on PGADMIN_WINDOWS_CSC, so an unsigned build check needs neither
the Certum hardware token nor the machine it's plugged into, and only the
signing release builds need a self-hosted runner. Its dependencies come from
dpage/winpgbuild, the same place the buildfarm's winpgbuild-update job gets
them. This one hasn't had a successful run yet and should be expected to need a
few passes.

All five trigger on master and by hand, not on pull requests, matching what the
Jenkins jobs do today; whether the full matrix should also gate PRs is a
separate question.
2026-09-16 12:03:20 +01:00
Dave Page d486241b06 Add a throwaway spike workflow for the macOS appbundle build
We're assessing a move of the buildfarm to GitHub Actions, and the one thing
that isn't already proven by the existing workflows is whether the appbundle
build fits inside a hosted macOS runner. The arm64 images have 3 vCPU and 7 GB
of RAM against the 8 cores and 16 GiB of the mac2.metal worker they'd replace,
and pkg/mac/build-functions.sh:320 records webpack being OS-OOM-killed inside
TerserPlugin at a 3 GB old-space ceiling, with 6 GB then starving the earlier
steps. Nothing we currently run on macOS builds a production bundle, so that
history is untested on hosted hardware.

The workflow is workflow_dispatch only, builds unsigned (build.sh skips signing
and notarisation when the two conf files are absent, so it needs no secrets),
and samples resident set sizes and swap every ten seconds so the logs show
whether the machine actually ran out of headroom rather than just whether the
build happened to pass. The runner image, the Python version and the webpack
memory ceiling are all inputs, so the ceiling can be bisected across images.

To be deleted once the question is answered.
2026-09-16 12:03:20 +01:00
44 changed files with 5063 additions and 705 deletions

No files matched your search

@@ -0,0 +1,99 @@
# Installs the PostgreSQL build dependencies published by pgadmin-org/pgbuild.
#
# Four workflows need these — the QA and snapshot builds on each of macOS and
# Windows — and keeping the logic in one place is not merely tidiness: the
# first version used the gh CLI, which turned out not to exist on the
# self-hosted Windows runner, and that same fix had to be made twice.
#
# On macOS the archives must land in /opt/pgbuild specifically, because the
# install names inside them are absolute: pg_dump asks for
# /opt/pgbuild/zstd/lib/libzstd.1.dylib and libpq for
# /opt/pgbuild/krb5/lib/libgssapi_krb5.2.2.dylib, among others. _fixup_imports
# copies from those paths into the bundle and rewrites the references, so they
# have to resolve at build time. On Windows there are no absolute references,
# so the location only has to avoid colliding with anything else, which is why
# it goes under the runner's temp directory rather than the C:\build64 the
# buildfarm's Jenkins worker manages.
#
# Everything is fetched over plain HTTPS. pgbuild is public and its asset URLs
# are predictable, so this needs neither a CLI nor authentication, and both
# curl --fail and Invoke-WebRequest fail on a 404 rather than writing an error
# page to disk and failing later at the unpack.
name: Install pgbuild dependencies
description: Download and unpack the PostgreSQL build dependencies from pgadmin-org/pgbuild
inputs:
arch:
description: Architecture as pgbuild names it, e.g. arm64, x86_64
required: true
postgres-major:
description: PostgreSQL major version to install
required: false
default: '18'
outputs:
prefix:
description: Directory the dependencies were unpacked into
# Whichever of the two platform steps ran. Merging these in a shell step
# instead would mean picking a shell that exists on every runner, and the
# Windows signing host has no bash.
value: ${{ steps.install-macos.outputs.prefix || steps.install-windows.outputs.prefix }}
runs:
using: composite
steps:
- name: Install the pgbuild dependencies (macOS)
id: install-macos
if: runner.os == 'macOS'
shell: bash
env:
PGBUILD_ARCH: ${{ inputs.arch }}
POSTGRES_MAJOR: ${{ inputs.postgres-major }}
run: |
set -euo pipefail
BASE=https://github.com/pgadmin-org/pgbuild/releases/download
sudo mkdir -p /opt/pgbuild
sudo chown "$(id -u):$(id -g)" /opt/pgbuild
for PKG in openssl krb5 zstd lz4 "postgresql-${POSTGRES_MAJOR}"; do
TAG="${PKG}-macos-${PGBUILD_ARCH}-latest"
echo "Fetching ${TAG}..."
curl --fail --silent --show-error --location \
-o "${RUNNER_TEMP}/${TAG}.tar.gz" \
"${BASE}/${TAG}/${TAG}.tar.gz"
tar -xzf "${RUNNER_TEMP}/${TAG}.tar.gz" -C /opt/pgbuild
done
ls -l /opt/pgbuild
echo "prefix=/opt/pgbuild" >> "${GITHUB_OUTPUT}"
- name: Install the pgbuild dependencies (Windows)
id: install-windows
if: runner.os == 'Windows'
shell: powershell
env:
PGBUILD_ARCH: ${{ inputs.arch }}
POSTGRES_MAJOR: ${{ inputs.postgres-major }}
run: |
# Without this, Invoke-WebRequest spends most of a 90 MB download
# rendering a progress bar into a log nobody will read.
$ProgressPreference = 'SilentlyContinue'
$depsDir = Join-Path $env:RUNNER_TEMP 'pgbuild-deps'
New-Item -ItemType Directory -Force -Path $depsDir | Out-Null
$base = 'https://github.com/pgadmin-org/pgbuild/releases/download'
foreach ($pkg in @("postgresql-$env:POSTGRES_MAJOR", 'krb5')) {
$tag = "$pkg-windows-$env:PGBUILD_ARCH-latest"
$zip = Join-Path $env:RUNNER_TEMP "$tag.zip"
Write-Output "Fetching $tag..."
Invoke-WebRequest -Uri "$base/$tag/$tag.zip" -OutFile $zip -UseBasicParsing
Expand-Archive -Path $zip -DestinationPath $depsDir -Force
}
Get-ChildItem $depsDir
# Out-File in Windows PowerShell defaults to UTF-16, which the runner
# cannot parse, so the encoding is explicit.
"prefix=$depsDir" | Out-File -FilePath $env:GITHUB_OUTPUT -Encoding utf8 -Append
+35 -3
View File
@@ -1,15 +1,47 @@
name: Check container build
on:
# Callable, so that a workflow which needs a container build gets this one
# rather than a second definition of it that can drift. Snapshot does not
# call it today, the image not being part of the download tree.
workflow_call:
push:
branches: [ "master" ]
paths:
- 'Dockerfile'
- 'pkg/docker/**'
- 'web/**'
- 'requirements.txt'
- 'docs/**'
- '.dockerignore'
- 'LICENSE'
- '.github/workflows/check-container-build.yml'
pull_request:
branches: [ "master" ]
paths:
- 'Dockerfile'
- 'pkg/docker/**'
- 'web/**'
- 'requirements.txt'
- 'docs/**'
- '.dockerignore'
- 'LICENSE'
- '.github/workflows/check-container-build.yml'
workflow_dispatch:
concurrency:
group: '${{ github.workflow }} @ ${{ github.event.pull_request.head.label || github.head_ref || github.ref }}'
# The workflow's own name, spelled out rather than taken from
# github.workflow, because in a reusable workflow that expression is
# the CALLER's name. Every workflow Snapshot calls would otherwise
# share one group and, with cancel-in-progress, cancel each other.
# github.event_name is in the key because a called workflow inherits the
# caller's ref: once this is on master, the nightly Snapshot's deb build
# and an ordinary push's deb build would otherwise share a group and,
# with cancel-in-progress, kill each other. A push landing mid-nightly
# would cancel the snapshot's build and skip the publish entirely.
group: 'check-container-build @ ${{ github.event.pull_request.head.label || github.head_ref || github.ref }} @ ${{ github.event_name }}'
cancel-in-progress: true
jobs:
@@ -17,12 +49,12 @@ jobs:
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v7
with:
fetch-depth: 2
- name: Build the container
uses: docker/build-push-action@v6
uses: docker/build-push-action@v7
with:
push: false
context: .
+136
View File
@@ -0,0 +1,136 @@
# Replaces the pgadmin4-deb-qa Jenkins job. That job runs `make debian` on a
# dedicated EC2 VM per distribution; here each distribution is a container on a
# hosted runner instead, which is the same build in a reproducible environment
# and no VMs to maintain.
#
# Triggered on master and by hand rather than on pull requests, matching what
# the Jenkins job does today (it polls git every five minutes). Running the
# full matrix on every PR is a separate decision.
name: Check DEB builds
on:
# Callable from snapshot.yml, so that what gets published is the
# same build that runs on every push rather than a second
# definition of it that can drift.
workflow_call:
push:
branches: [ "master", "buildfarm-to-actions" ]
paths:
- 'pkg/debian/**'
- 'pkg/linux/**'
- 'web/**'
- 'docs/**'
- 'requirements.txt'
- 'Makefile'
- 'tools/setup-python-env.sh'
- '.github/workflows/check-deb-build.yml'
workflow_dispatch:
concurrency:
# The workflow's own name, spelled out rather than taken from
# github.workflow, because in a reusable workflow that expression is
# the CALLER's name. Every workflow Snapshot calls would otherwise
# share one group and, with cancel-in-progress, cancel each other.
# github.event_name is in the key because a called workflow inherits the
# caller's ref: once this is on master, the nightly Snapshot's deb build
# and an ordinary push's deb build would otherwise share a group and,
# with cancel-in-progress, kill each other. A push landing mid-nightly
# would cancel the snapshot's build and skip the publish entirely.
group: 'check-deb-build @ ${{ github.ref }} @ ${{ github.event_name }}'
cancel-in-progress: true
permissions:
contents: read
jobs:
build-deb:
runs-on: ubuntu-24.04
strategy:
fail-fast: false
matrix:
include:
# "name" exists only because artifact names may not contain a colon.
- { name: 'debian-12', image: 'debian:12' }
- { name: 'debian-13', image: 'debian:13' }
- { name: 'ubuntu-2204', image: 'ubuntu:22.04' }
- { name: 'ubuntu-2404', image: 'ubuntu:24.04' }
- { name: 'ubuntu-2604', image: 'ubuntu:26.04' }
container: ${{ matrix.image }}
steps:
# Before checkout, so the action has git available and the container has
# the certificates it needs to fetch anything.
- name: Install the build environment
env:
DEBIAN_FRONTEND: noninteractive
run: |
apt-get update
apt-get install -y --no-install-recommends \
build-essential \
ca-certificates \
curl \
dh-exec \
dpkg-dev \
fakeroot \
git \
libffi-dev \
libkrb5-dev \
libpq-dev \
libreadline-dev \
libssl-dev \
lsb-release \
python3-dev \
python3-pip \
python3-sphinx \
python3-venv \
python3-wheel \
unzip \
wget \
zlib1g-dev
# sphinxcontrib-youtube has no distro package. --break-system-packages is
# needed from Debian 12 and Ubuntu 24.04 onwards (PEP 668), and is
# harmless before that, but the flag itself only exists on newer pip, so
# fall back rather than fail.
- name: Install the Sphinx extensions
run: |
python3 -m pip install --break-system-packages sphinxcontrib-youtube \
|| python3 -m pip install sphinxcontrib-youtube
- name: Install syft
run: curl -sSfL https://raw.githubusercontent.com/anchore/syft/v1.52.0/install.sh | sh -s -- -b /usr/local/bin v1.52.0
- uses: actions/setup-node@v7
with:
node-version: '20.x'
- uses: actions/checkout@v7
# The container runs as root while the workspace is owned by the runner
# user, so git refuses to operate on it ("detected dubious ownership").
# The build shells out to git for the commit hash during `yarn run
# bundle`, which fails with exit 128 without this.
- name: Mark the workspace as a safe git directory
run: git config --global --add safe.directory "${GITHUB_WORKSPACE}"
# Corepack provisions the pinned Yarn from the packageManager field,
# matching run-javascript-tests.yml. The distro nodejs packages are
# avoided entirely: on Ubuntu 26.04 the distro node ships without npm,
# which is why the buildfarm has to pin NodeSource above it.
- name: Enable Corepack
run: corepack enable
- name: Check the DEB build
run: make debian
- name: Archive the packages
uses: actions/upload-artifact@v7
with:
name: pgadmin4-deb-build-output-${{ matrix.name }}
if-no-files-found: error
path: dist/*.deb
+40 -5
View File
@@ -1,15 +1,47 @@
name: Check documentation builds
on:
# Callable from snapshot.yml, so that what gets published is the
# same build that runs on every push rather than a second
# definition of it that can drift.
workflow_call:
push:
branches: [ "master" ]
paths:
- 'docs/**'
- 'web/version.py'
- 'web/branding.py'
- 'requirements.txt'
- 'Makefile'
- 'web/config.py'
- 'tools/setup-python-env.sh'
- '.github/workflows/check-doc-builds.yml'
pull_request:
branches: [ "master" ]
paths:
- 'docs/**'
- 'web/version.py'
- 'web/branding.py'
- 'requirements.txt'
- 'Makefile'
- 'web/config.py'
- 'tools/setup-python-env.sh'
- '.github/workflows/check-doc-builds.yml'
workflow_dispatch:
concurrency:
group: '${{ github.workflow }} @ ${{ github.event.pull_request.head.label || github.head_ref || github.ref }}'
# The workflow's own name, spelled out rather than taken from
# github.workflow, because in a reusable workflow that expression is
# the CALLER's name. Every workflow Snapshot calls would otherwise
# share one group and, with cancel-in-progress, cancel each other.
# github.event_name is in the key because a called workflow inherits the
# caller's ref: once this is on master, the nightly Snapshot's deb build
# and an ordinary push's deb build would otherwise share a group and,
# with cancel-in-progress, kill each other. A push landing mid-nightly
# would cancel the snapshot's build and skip the publish entirely.
group: 'check-doc-builds @ ${{ github.event.pull_request.head.label || github.head_ref || github.ref }} @ ${{ github.event_name }}'
cancel-in-progress: true
jobs:
@@ -17,7 +49,7 @@ jobs:
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v7
- name: Install platform dependencies
run: |
@@ -33,9 +65,10 @@ jobs:
make docs
- name: Archive HTML
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v7
with:
name: pgadmin4-html-build-output
if-no-files-found: error
path: docs/en_US/_build/html/*
- name: Check the ePub build
@@ -44,9 +77,10 @@ jobs:
make docs-epub
- name: Archive ePub
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v7
with:
name: pgadmin4-epub-build-output
if-no-files-found: error
path: docs/en_US/_build/epub/pgAdmin4.epub
- name: Check the PDF build
@@ -55,7 +89,8 @@ jobs:
make docs-pdf
- name: Archive PDF
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v7
with:
name: pgadmin4-pdf-build-output
if-no-files-found: error
path: docs/en_US/_build/latex/pgadmin4.pdf
+59
View File
@@ -0,0 +1,59 @@
# Replaces the pgadmin4-helm-buildx-qa Jenkins job, which runs
# `helm package pkg/helm` and nothing else. Helm is installed explicitly rather
# than relied upon from the runner image, since what the images ship changes
# between versions.
name: Check Helm chart build
on:
# Callable from snapshot.yml, so that what gets published is the
# same build that runs on every push rather than a second
# definition of it that can drift.
workflow_call:
push:
branches: [ "master", "buildfarm-to-actions" ]
paths:
- 'pkg/helm/**'
- '.github/workflows/check-helm-build.yml'
workflow_dispatch:
concurrency:
# The workflow's own name, spelled out rather than taken from
# github.workflow, because in a reusable workflow that expression is
# the CALLER's name. Every workflow Snapshot calls would otherwise
# share one group and, with cancel-in-progress, cancel each other.
# github.event_name is in the key because a called workflow inherits the
# caller's ref: once this is on master, the nightly Snapshot's deb build
# and an ordinary push's deb build would otherwise share a group and,
# with cancel-in-progress, kill each other. A push landing mid-nightly
# would cancel the snapshot's build and skip the publish entirely.
group: 'check-helm-build @ ${{ github.ref }} @ ${{ github.event_name }}'
cancel-in-progress: true
permissions:
contents: read
jobs:
build-chart:
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v7
- name: Install Helm
run: |
curl -fsSL https://raw.githubusercontent.com/helm/helm/v4.3.0/scripts/get-helm-3 -o get-helm-3
chmod 700 get-helm-3
./get-helm-3
- name: Check the chart build
run: helm package pkg/helm
- name: Archive the chart
uses: actions/upload-artifact@v7
with:
name: pgadmin4-helm-build-output
if-no-files-found: error
path: pgadmin4-helm-*.tgz
+17 -1
View File
@@ -3,8 +3,24 @@ name: Check Javascript style
on:
push:
branches: [ "master" ]
paths:
- 'web/**/*.js'
- 'web/**/*.jsx'
- 'web/package.json'
- 'web/yarn.lock'
- 'web/.eslintrc.js'
- 'web/.yarnrc.yml'
- '.github/workflows/check-javascript-style.yml'
pull_request:
branches: [ "master" ]
paths:
- 'web/**/*.js'
- 'web/**/*.jsx'
- 'web/package.json'
- 'web/yarn.lock'
- 'web/.eslintrc.js'
- 'web/.yarnrc.yml'
- '.github/workflows/check-javascript-style.yml'
workflow_dispatch:
@@ -17,7 +33,7 @@ jobs:
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v7
# Corepack reads packageManager from web/package.json and provisions
# exactly that Yarn: a single pinned download, with nothing written
+119
View File
@@ -0,0 +1,119 @@
# Replaces the pgadmin4-macos-qa Jenkins job. That job runs `make appbundle` on
# the two EC2 Mac workers; this runs the same build on hosted runners, which
# cover both architectures.
#
# The build is unsigned and unnotarised: pkg/mac/build.sh sets CODESIGN=0 and
# NOTARIZE=0 when pkg/mac/codesign.conf and pkg/mac/notarization.conf are
# absent, so this needs no secrets. Signing belongs to the release and snapshot
# builds.
#
# PostgreSQL comes from Homebrew for pg_config (psycopg builds against it) and
# for the four client binaries copied into the bundle. The release build uses
# a PostgreSQL built against its own OpenSSL and Kerberos; that difference
# does not matter for a build check, and _fixup_imports relocates Homebrew's
# libraries into the bundle correctly (verified by _verify_bundle_linkage,
# which fails the build if anything still points outside it).
name: Check macOS builds
on:
push:
branches: [ "master", "buildfarm-to-actions" ]
paths:
- 'pkg/mac/**'
- '.github/actions/install-pgbuild-deps/action.yml'
- 'web/**'
- 'docs/**'
- 'requirements.txt'
- 'Makefile'
- 'tools/setup-python-env.sh'
- '.github/workflows/check-macos-build.yml'
workflow_dispatch:
concurrency:
# The workflow's own name, spelled out rather than taken from
# github.workflow, because in a reusable workflow that expression is
# the CALLER's name. Every workflow Snapshot calls would otherwise
# share one group and, with cancel-in-progress, cancel each other.
group: 'check-macos-build @ ${{ github.ref }}'
cancel-in-progress: true
permissions:
contents: read
jobs:
build-appbundle:
runs-on: ${{ matrix.runner }}
strategy:
fail-fast: false
matrix:
include:
# "arch" is pgbuild's spelling of the architecture, which differs
# from the one this build uses internally (x64 rather than x86_64).
- { name: 'arm64', arch: 'arm64', runner: 'macos-15' }
# macos-26-intel rather than macos-15-intel: create-dmg's unmount
# scales with how much real Mach-O content lands on the volume, and
# the macos-15-intel image is slow enough that the real bundle
# crosses hdiutil's 120-second DiskArbitration timeout. Measured with
# an identical payload, unmount to "Disk image done": 19s on arm64,
# 41s on macos-26-intel, 82s on macos-15-intel, and that was with a
# lighter stand-in than the real bundle. It is also the newer image,
# which we would move to regardless since macOS 15 retires first.
- { name: 'x64', arch: 'x86_64', runner: 'macos-26-intel' }
timeout-minutes: 120
steps:
- uses: actions/checkout@v7
- uses: actions/setup-node@v7
with:
node-version: '20.x'
- name: Enable Corepack
run: corepack enable
# syft generates the SBOM, wget fetches Electron. PostgreSQL and its
# dependencies no longer come from Homebrew; see below.
- name: Install the build dependencies
run: brew install syft wget
# The dependencies come from pgadmin-org/pgbuild, built against each
# other with --with-zstd --with-lz4 and a 14.0 deployment target, which
# is what the release build ships. Homebrew's PostgreSQL was only ever a
# stand-in: it has no zstd or lz4 support (issue #9425), links Homebrew's
# OpenSSL rather than ours, and carries whatever minimum OS Homebrew
# happened to build for.
- name: Install the PostgreSQL build dependencies
uses: ./.github/actions/install-pgbuild-deps
with:
arch: ${{ matrix.arch }}
- name: Show what pg_dump expects
run: otool -L /opt/pgbuild/postgresql/bin/pg_dump
- name: Check the appbundle build
run: |
export PGADMIN_POSTGRES_DIR=/opt/pgbuild/postgresql
# cryptography publishes arm64-only macOS wheels (checked against
# 50.0.1), so on Intel pip builds it from the sdist and its
# openssl-sys crate links whatever OpenSSL it finds on the build
# host. _fixup_imports deliberately skips _rust.abi3.so, so that
# reference would survive into the bundle and _verify_bundle_linkage
# would fail the build. Linking statically leaves no external
# reference at all, and against our own OpenSSL rather than a
# different one from Homebrew. See issue #10123.
export OPENSSL_DIR=/opt/pgbuild/openssl
export OPENSSL_STATIC=1
make appbundle
- name: Archive the disk image
uses: actions/upload-artifact@v7
with:
name: pgadmin4-macos-build-output-${{ matrix.name }}
if-no-files-found: error
path: dist/*.dmg
+38 -3
View File
@@ -1,15 +1,49 @@
name: Check Python build
on:
# Callable from snapshot.yml, so that what gets published is the
# same build that runs on every push rather than a second
# definition of it that can drift.
workflow_call:
push:
branches: [ "master" ]
paths:
- 'web/**'
- 'pkg/pip/**'
- 'requirements.txt'
- 'Makefile'
- 'docs/**'
- 'LICENSE'
- 'README.md'
- 'tools/setup-python-env.sh'
- '.github/workflows/check-python-build.yml'
pull_request:
branches: [ "master" ]
paths:
- 'web/**'
- 'pkg/pip/**'
- 'requirements.txt'
- 'Makefile'
- 'docs/**'
- 'LICENSE'
- 'README.md'
- 'tools/setup-python-env.sh'
- '.github/workflows/check-python-build.yml'
workflow_dispatch:
concurrency:
group: '${{ github.workflow }} @ ${{ github.event.pull_request.head.label || github.head_ref || github.ref }}'
# The workflow's own name, spelled out rather than taken from
# github.workflow, because in a reusable workflow that expression is
# the CALLER's name. Every workflow Snapshot calls would otherwise
# share one group and, with cancel-in-progress, cancel each other.
# github.event_name is in the key because a called workflow inherits the
# caller's ref: once this is on master, the nightly Snapshot's deb build
# and an ordinary push's deb build would otherwise share a group and,
# with cancel-in-progress, kill each other. A push landing mid-nightly
# would cancel the snapshot's build and skip the publish entirely.
group: 'check-python-build @ ${{ github.event.pull_request.head.label || github.head_ref || github.ref }} @ ${{ github.event_name }}'
cancel-in-progress: true
jobs:
@@ -17,7 +51,7 @@ jobs:
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v7
- name: Install platform dependencies
run: |
@@ -33,7 +67,8 @@ jobs:
make pip
- name: Archive Python wheel
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v7
with:
name: pgadmin4-python-build-output
if-no-files-found: error
path: dist/*
+9 -1
View File
@@ -3,8 +3,16 @@ name: Check Python style
on:
push:
branches: [ "master" ]
paths:
- '**.py'
- '.pycodestyle'
- '.github/workflows/check-python-style.yml'
pull_request:
branches: [ "master" ]
paths:
- '**.py'
- '.pycodestyle'
- '.github/workflows/check-python-style.yml'
workflow_dispatch:
@@ -17,7 +25,7 @@ jobs:
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v7
- name: Install Python dependencies
run: |
+169
View File
@@ -0,0 +1,169 @@
# Replaces the pgadmin4-rpm-qa Jenkins job, as check-deb-build.yml replaces the
# DEB one: a container per distribution on a hosted runner, rather than a
# dedicated EC2 VM per distribution.
#
# Unlike the DEB build, the PGDG repository is not optional here.
# pkg/redhat/build.sh opens with `export PATH=$(ls -d /usr/pgsql-1* | ...)` and
# runs under `set -e`, so the build dies immediately without a PGDG PostgreSQL
# installed.
#
# The QA job does not sign anything; `rpmsign --resign` is a separate step in
# the release and snapshot jobs.
name: Check RPM builds
on:
# Callable from snapshot.yml, so that what gets published is the
# same build that runs on every push rather than a second
# definition of it that can drift.
workflow_call:
push:
branches: [ "master", "buildfarm-to-actions" ]
paths:
- 'pkg/redhat/**'
- 'pkg/linux/**'
- 'web/**'
- 'docs/**'
- 'requirements.txt'
- 'Makefile'
- 'tools/setup-python-env.sh'
- '.github/workflows/check-rpm-build.yml'
workflow_dispatch:
concurrency:
# The workflow's own name, spelled out rather than taken from
# github.workflow, because in a reusable workflow that expression is
# the CALLER's name. Every workflow Snapshot calls would otherwise
# share one group and, with cancel-in-progress, cancel each other.
# github.event_name is in the key because a called workflow inherits the
# caller's ref: once this is on master, the nightly Snapshot's deb build
# and an ordinary push's deb build would otherwise share a group and,
# with cancel-in-progress, kill each other. A push landing mid-nightly
# would cancel the snapshot's build and skip the publish entirely.
group: 'check-rpm-build @ ${{ github.ref }} @ ${{ github.event_name }}'
cancel-in-progress: true
permissions:
contents: read
jobs:
build-rpm:
runs-on: ubuntu-24.04
strategy:
fail-fast: false
matrix:
include:
# "name" exists only because artifact names may not contain a colon.
# "repo" selects the PGDG repository RPM, which is built per
# distribution family and major version.
- name: 'el-9'
image: 'almalinux:9'
repo: 'https://download.postgresql.org/pub/repos/yum/reporpms/EL-9-x86_64/pgdg-redhat-repo-latest.noarch.rpm'
- name: 'el-10'
image: 'almalinux:10'
repo: 'https://download.postgresql.org/pub/repos/yum/reporpms/EL-10-x86_64/pgdg-redhat-repo-latest.noarch.rpm'
- name: 'fedora-43'
image: 'fedora:43'
repo: 'https://download.postgresql.org/pub/repos/yum/reporpms/F-43-x86_64/pgdg-fedora-repo-latest.noarch.rpm'
- name: 'fedora-44'
image: 'fedora:44'
repo: 'https://download.postgresql.org/pub/repos/yum/reporpms/F-44-x86_64/pgdg-fedora-repo-latest.noarch.rpm'
container: ${{ matrix.image }}
steps:
# Before checkout, so the action has git available.
- name: Install the build environment
env:
PGDG_REPO: ${{ matrix.repo }}
run: |
dnf install -y dnf-plugins-core
# EL needs EPEL and CRB for the build dependencies; Fedora has
# everything in the main repositories.
if [ ! -f /etc/fedora-release ]; then
dnf install -y epel-release
dnf config-manager --set-enabled crb
fi
# The distro's own PostgreSQL module would shadow PGDG's packages.
dnf -qy module disable postgresql || true
dnf install -y --nogpgcheck "${PGDG_REPO}"
# The compiler toolchain is not optional: psycopg-c and gssapi are
# both built from source (pkg/linux/build-functions.sh passes
# --no-binary psycopg), and rpmbuild needs redhat-rpm-config. The
# buildfarm gets these from the @development-tools group; they are
# listed individually here so the container stays small.
dnf install -y \
diffutils \
expect \
fakeroot \
file \
findutils \
gcc \
gcc-c++ \
git \
krb5-devel \
libpq5-devel \
make \
openssl-devel \
patch \
postgresql18-devel \
python3-devel \
python3-pip \
python3-sphinx \
redhat-rpm-config \
rpm-build \
rpm-sign \
unzip \
wget \
yum-utils \
zlib-devel
# pkg/redhat/build.sh:229 shells out to yumdownloader, and a missing
# one surfaces as a bare exit 127 twelve minutes into the build,
# after the docs and the webpack bundle. Fail here instead, where the
# message is obvious. dnf5 distributions (EL10, Fedora 41+) get
# yumdownloader through yum-utils' compatibility shims rather than
# from yum itself, so it is worth checking rather than assuming.
command -v yumdownloader >/dev/null || {
echo "ERROR: yumdownloader is not on PATH; pkg/redhat/build.sh needs it." >&2
exit 1
}
- name: Install the Sphinx extensions
run: python3 -m pip install --break-system-packages sphinxcontrib-youtube || python3 -m pip install sphinxcontrib-youtube
- name: Install syft
run: curl -sSfL https://raw.githubusercontent.com/anchore/syft/v1.52.0/install.sh | sh -s -- -b /usr/local/bin v1.52.0
- uses: actions/setup-node@v7
with:
node-version: '20.x'
- uses: actions/checkout@v7
# The container runs as root while the workspace is owned by the runner
# user, so git refuses to operate on it ("detected dubious ownership").
# The build shells out to git for the commit hash during `yarn run
# bundle`, which fails with exit 128 without this.
- name: Mark the workspace as a safe git directory
run: git config --global --add safe.directory "${GITHUB_WORKSPACE}"
- name: Enable Corepack
run: corepack enable
- name: Check the RPM build
run: make redhat
- name: Archive the packages
uses: actions/upload-artifact@v7
with:
name: pgadmin4-rpm-build-output-${{ matrix.name }}
if-no-files-found: error
path: dist/*.rpm
+38 -3
View File
@@ -1,15 +1,49 @@
name: Check tarball build
on:
# Callable from snapshot.yml, so that what gets published is the
# same build that runs on every push rather than a second
# definition of it that can drift.
workflow_call:
push:
branches: [ "master" ]
paths:
- 'web/**'
- 'docs/**'
- 'pkg/src/**'
- 'requirements.txt'
- 'Makefile'
- 'LICENSE'
- 'README.md'
- 'tools/setup-python-env.sh'
- '.github/workflows/check-tarball-build.yml'
pull_request:
branches: [ "master" ]
paths:
- 'web/**'
- 'docs/**'
- 'pkg/src/**'
- 'requirements.txt'
- 'Makefile'
- 'LICENSE'
- 'README.md'
- 'tools/setup-python-env.sh'
- '.github/workflows/check-tarball-build.yml'
workflow_dispatch:
concurrency:
group: '${{ github.workflow }} @ ${{ github.event.pull_request.head.label || github.head_ref || github.ref }}'
# The workflow's own name, spelled out rather than taken from
# github.workflow, because in a reusable workflow that expression is
# the CALLER's name. Every workflow Snapshot calls would otherwise
# share one group and, with cancel-in-progress, cancel each other.
# github.event_name is in the key because a called workflow inherits the
# caller's ref: once this is on master, the nightly Snapshot's deb build
# and an ordinary push's deb build would otherwise share a group and,
# with cancel-in-progress, kill each other. A push landing mid-nightly
# would cancel the snapshot's build and skip the publish entirely.
group: 'check-tarball-build @ ${{ github.event.pull_request.head.label || github.head_ref || github.ref }} @ ${{ github.event_name }}'
cancel-in-progress: true
jobs:
@@ -17,7 +51,7 @@ jobs:
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v7
- name: Install platform dependencies
run: |
@@ -33,7 +67,8 @@ jobs:
make src
- name: Archive source tarball
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v7
with:
name: pgadmin4-tarball-build-output
if-no-files-found: error
path: dist/*
+9 -1
View File
@@ -3,8 +3,16 @@ name: Check translations
on:
push:
branches: [ "master" ]
paths:
- 'web/**'
- 'Makefile'
- '.github/workflows/check-translations.yml'
pull_request:
branches: [ "master" ]
paths:
- 'web/**'
- 'Makefile'
- '.github/workflows/check-translations.yml'
workflow_dispatch:
@@ -17,7 +25,7 @@ jobs:
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v7
- name: Install platform dependencies
run: |
+141
View File
@@ -0,0 +1,141 @@
# Replaces the pgadmin4-windows-x64-qa Jenkins job.
#
# This runs on a hosted runner, not a self-hosted one. Code signing in Make.bat
# is conditional on PGADMIN_WINDOWS_CSC being set (see Make.bat:393, :447 and
# :469), so an unsigned build check needs neither the Certum hardware token nor
# the machine it is plugged into. Only the release and snapshot builds, which
# do sign, need the self-hosted runner.
#
# Dependencies come from dpage/winpgbuild, which is exactly where the
# buildfarm's winpgbuild-update job gets them, so the toolchain matches what
# the release build uses rather than being an approximation of it.
#
# UNTESTED as written: unlike the macOS equivalent this has not yet had a
# successful run, so expect the first few attempts to need adjustment.
name: Check Windows build
on:
push:
branches: [ "master", "buildfarm-to-actions" ]
paths:
- 'Make.bat'
- 'pkg/win32/**'
- 'pkg/python-version.txt'
- '.github/actions/install-pgbuild-deps/action.yml'
- 'web/**'
- 'docs/**'
- 'requirements.txt'
- '.github/workflows/check-windows-build.yml'
workflow_dispatch:
concurrency:
# The workflow's own name, spelled out rather than taken from
# github.workflow, because in a reusable workflow that expression is
# the CALLER's name. Every workflow Snapshot calls would otherwise
# share one group and, with cancel-in-progress, cancel each other.
group: 'check-windows-build @ ${{ github.ref }}'
cancel-in-progress: true
permissions:
contents: read
jobs:
build-installer:
runs-on: windows-2025
timeout-minutes: 120
env:
# The major version of the PostgreSQL release to pull from pgbuild,
# matching DEFAULT_POSTGRESQL_VERSION on the buildfarm.
POSTGRES_MAJOR: '18'
steps:
- uses: actions/checkout@v7
- uses: actions/setup-node@v7
with:
node-version: '20.x'
- name: Enable Corepack
run: corepack enable
# None of these are on the hosted image. wget matters most: Make.bat's
# Electron download is an unbounded retry loop (Make.bat:370), so without
# it the build spins on error 9009 until the job times out rather than
# failing. syft generates the SBOM, and its absence is silent, since
# :GENERATE_SBOM redirects into sbom.json and returns 0 regardless. The
# buildfarm's Windows worker installs all three via Chocolatey too.
- name: Install Inno Setup, syft and wget
run: choco install innosetup syft wget -y --no-progress
# From pgadmin-org/pgbuild, which supersedes dpage/winpgbuild and is
# where the macOS dependencies come from too, so both platforms build
# against one manifest of pinned versions. The archives unpack to
# <DEPS_DIR>\postgresql and <DEPS_DIR>\krb5.
# Unpacked under the runner's own temp directory rather than a fixed
# path. C:\build64 was the obvious choice and the wrong one: it is what
# the buildfarm's winpgbuild-update job manages on the Windows worker,
# deleting and re-extracting postgresql and krb5 inside it. Since the
# signing builds will run on that same machine as a self-hosted runner,
# a shared path would have the two systems overwriting each other's
# dependencies, and the resulting Jenkins failures would look nothing
# like their cause.
- name: Install the PostgreSQL and Kerberos dependencies
id: deps
uses: ./.github/actions/install-pgbuild-deps
with:
arch: x86_64
postgres-major: ${{ env.POSTGRES_MAJOR }}
- name: Record the dependency location
shell: pwsh
run: '"DEPS_DIR=${{ steps.deps.outputs.prefix }}" | Out-File -FilePath $env:GITHUB_ENV -Append'
# The hosted image ships Visual Studio Enterprise at a path that differs
# from the Community install Make.bat defaults to, and the redistributable
# version directory changes with every VS update, so both are discovered
# rather than hardcoded.
- name: Locate the Visual Studio toolchain
shell: pwsh
run: |
$vswhere = "${env:ProgramFiles(x86)}\Microsoft Visual Studio\Installer\vswhere.exe"
$vs = & $vswhere -latest -products * -property installationPath
if (-not $vs) { throw "Could not locate a Visual Studio installation" }
Write-Output "Visual Studio: $vs"
$redist = Get-ChildItem "$vs\VC\Redist\MSVC" -Directory |
Where-Object { Test-Path "$($_.FullName)\vc_redist.x64.exe" } |
Sort-Object Name -Descending |
Select-Object -First 1
if (-not $redist) { throw "Could not locate vc_redist.x64.exe under $vs" }
Write-Output "VC++ redistributable: $($redist.FullName)"
"VS_INSTALL_DIR=$vs" | Out-File -FilePath $env:GITHUB_ENV -Append
"PGADMIN_VCREDIST_DIR=$($redist.FullName)" | Out-File -FilePath $env:GITHUB_ENV -Append
# psycopg[c] compiles against libpq, so the build needs the MSVC
# environment. vcvars64.bat only sets it for its own process, hence
# invoking it and Make.bat in a single cmd session, as the Jenkins job
# does.
#
# No Python is set up for this job: Make.bat downloads the version named
# in pkg/python-version.txt, so the build gets the same interpreter here
# as it does on the signing host and on a developer's machine.
- name: Check the installer build
shell: cmd
run: |
SET "PGADMIN_POSTGRES_DIR=%DEPS_DIR%\postgresql"
SET "PGADMIN_KRB5_DIR=%DEPS_DIR%\krb5"
SET "PGADMIN_INNOTOOL_DIR=C:\Program Files (x86)\Inno Setup 6"
CALL "%VS_INSTALL_DIR%\VC\Auxiliary\Build\vcvars64.bat" || EXIT /b 1
CALL Make.bat || EXIT /b 1
- name: Archive the installer
uses: actions/upload-artifact@v7
with:
name: pgadmin4-windows-build-output
if-no-files-found: error
path: dist/*.exe
+16 -4
View File
@@ -3,8 +3,20 @@ name: Run feature tests on EPAS
on:
push:
branches: [ "master" ]
paths:
- 'web/**'
- 'requirements.txt'
- 'Makefile'
- 'tools/setup-python-env.sh'
- '.github/workflows/run-feature-tests-epas.yml'
pull_request:
branches: [ "master" ]
paths:
- 'web/**'
- 'requirements.txt'
- 'Makefile'
- 'tools/setup-python-env.sh'
- '.github/workflows/run-feature-tests-epas.yml'
workflow_dispatch:
@@ -26,7 +38,7 @@ jobs:
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v7
- name: Setup the EDB APT repo on Linux
run: |
@@ -163,21 +175,21 @@ jobs:
- name: Archive server log
if: success() || failure()
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v7
with:
name: server-log-pg${{ matrix.pgver }}
path: var/pgadmin4.log
- name: Archive regression log
if: success() || failure()
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v7
with:
name: regression-log-pg${{ matrix.pgver }}
path: web/regression/regression.log
- name: Archive screenshots
if: success() || failure()
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v7
with:
name: screenshots-pg${{ matrix.pgver }}
path: web/regression/screenshots
+16 -4
View File
@@ -6,8 +6,20 @@ name: Run feature tests on PostgreSQL
on:
push:
branches: [ "master" ]
paths:
- 'web/**'
- 'requirements.txt'
- 'Makefile'
- 'tools/setup-python-env.sh'
- '.github/workflows/run-feature-tests-pg.yml'
pull_request:
branches: [ "master" ]
paths:
- 'web/**'
- 'requirements.txt'
- 'Makefile'
- 'tools/setup-python-env.sh'
- '.github/workflows/run-feature-tests-pg.yml'
workflow_dispatch:
@@ -25,7 +37,7 @@ jobs:
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v7
- name: Setup the PGDG APT repo
run: |
@@ -173,21 +185,21 @@ jobs:
- name: Archive server log
if: success() || failure()
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v7
with:
name: server-log-pg${{ matrix.pgver }}
path: var/pgadmin4.log
- name: Archive regression log
if: success() || failure()
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v7
with:
name: regression-log-pg${{ matrix.pgver }}
path: web/regression/regression.log
- name: Archive screenshots
if: success() || failure()
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v7
with:
name: screenshots-pg${{ matrix.pgver }}
path: web/regression/screenshots
+24 -2
View File
@@ -3,8 +3,30 @@ name: Run Javascript tests
on:
push:
branches: [ "master" ]
paths:
- 'web/**/*.js'
- 'web/**/*.jsx'
- 'web/package.json'
- 'web/yarn.lock'
- 'web/jest.config.js'
- 'web/webpack.shim.js'
- 'web/regression/javascript/**'
- 'web/.yarnrc.yml'
- 'web/babel.config.json'
- '.github/workflows/run-javascript-tests.yml'
pull_request:
branches: [ "master" ]
paths:
- 'web/**/*.js'
- 'web/**/*.jsx'
- 'web/package.json'
- 'web/yarn.lock'
- 'web/jest.config.js'
- 'web/webpack.shim.js'
- 'web/regression/javascript/**'
- 'web/.yarnrc.yml'
- 'web/babel.config.json'
- '.github/workflows/run-javascript-tests.yml'
workflow_dispatch:
@@ -22,11 +44,11 @@ jobs:
runs-on: ${{ matrix.os }}
steps:
- uses: actions/setup-node@v4
- uses: actions/setup-node@v7
with:
node-version: '20.x'
- uses: actions/checkout@v4
- uses: actions/checkout@v7
# Corepack reads packageManager from web/package.json and provisions
# exactly that Yarn: a single pinned download, with nothing written
+18 -4
View File
@@ -13,8 +13,22 @@ name: Run Python tests on EPAS
on:
push:
branches: [ "master" ]
paths:
- 'web/**'
- 'requirements.txt'
- 'Makefile'
- 'tools/setup-python-env.sh'
- 'tools/get_sb_package.py'
- '.github/workflows/run-python-tests-epas.yml'
pull_request:
branches: [ "master" ]
paths:
- 'web/**'
- 'requirements.txt'
- 'Makefile'
- 'tools/setup-python-env.sh'
- 'tools/get_sb_package.py'
- '.github/workflows/run-python-tests-epas.yml'
workflow_dispatch:
@@ -48,10 +62,10 @@ jobs:
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v7
- name: Update python version
uses: actions/setup-python@v5
uses: actions/setup-python@v7
with:
python-version: '3.10'
@@ -311,14 +325,14 @@ jobs:
- name: Archive server log
if: success() || failure()
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v7
with:
name: server-log-epas-${{ matrix.os }}-pg${{ matrix.pgver }}
path: var/pgadmin4.log
- name: Archive regression log
if: success() || failure()
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v7
with:
name: regression-log-epas-${{ matrix.os }}-pg${{ matrix.pgver }}
path: web/regression/regression.log
+18 -4
View File
@@ -6,8 +6,22 @@ name: Run Python tests on PostgreSQL
on:
push:
branches: [ "master" ]
paths:
- 'web/**'
- 'requirements.txt'
- 'Makefile'
- 'tools/setup-python-env.sh'
- 'tools/get_sb_package.py'
- '.github/workflows/run-python-tests-pg.yml'
pull_request:
branches: [ "master" ]
paths:
- 'web/**'
- 'requirements.txt'
- 'Makefile'
- 'tools/setup-python-env.sh'
- 'tools/get_sb_package.py'
- '.github/workflows/run-python-tests-pg.yml'
workflow_dispatch:
@@ -26,10 +40,10 @@ jobs:
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v7
- name: Update python version
uses: actions/setup-python@v5
uses: actions/setup-python@v7
with:
python-version: '3.10'
@@ -320,14 +334,14 @@ jobs:
- name: Archive server log
if: success() || failure()
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v7
with:
name: server-log-${{ matrix.os }}-pg${{ matrix.pgver }}
path: var/pgadmin4.log
- name: Archive regression log
if: success() || failure()
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v7
with:
name: regression-log-${{ matrix.os }}-pg${{ matrix.pgver }}
path: web/regression/regression.log
@@ -15,8 +15,20 @@ name: Run Python tests in server mode (data isolation)
on:
push:
branches: [ "master" ]
paths:
- 'web/**'
- 'requirements.txt'
- 'Makefile'
- 'tools/setup-python-env.sh'
- '.github/workflows/run-python-tests-server-mode.yml'
pull_request:
branches: [ "master" ]
paths:
- 'web/**'
- 'requirements.txt'
- 'Makefile'
- 'tools/setup-python-env.sh'
- '.github/workflows/run-python-tests-server-mode.yml'
workflow_dispatch:
@@ -32,7 +44,7 @@ jobs:
PG_VER: 18
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v7
with:
# We don't run any authenticated git operations after checkout
# (no push, no submodule fetch, no comment back to the PR), so
@@ -41,7 +53,7 @@ jobs:
persist-credentials: false
- name: Update python version
uses: actions/setup-python@v5
uses: actions/setup-python@v7
with:
python-version: '3.10'
@@ -191,14 +203,14 @@ jobs:
- name: Archive server log
if: success() || failure()
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v7
with:
name: server-log-server-mode
path: var/pgadmin4.log
- name: Archive regression log
if: success() || failure()
uses: actions/upload-artifact@v4
uses: actions/upload-artifact@v7
with:
name: regression-log-server-mode
path: web/regression/regression.log
+168
View File
@@ -0,0 +1,168 @@
# Signed, notarised macOS snapshot builds, replacing the buildfarm's
# pgadmin4-appbundle-snapshot job.
#
# Unlike check-macos-build.yml this produces something distributable: the
# bundle is signed with the Developer ID, notarised by Apple and stapled, and a
# ZIP is built alongside the DMG because the Electron auto-updater consumes the
# ZIP whilst the DMG is the primary download.
#
# Uploading is deliberately not wired up yet. This proves the signed build
# works; publishing comes after.
#
# Triggered by hand for now rather than nightly, so it cannot start signing
# things before we mean it to.
name: Snapshot - macOS
on:
# Callable from snapshot.yml so that the orchestrator produces one
# complete tree. Signing stays here rather than moving to the
# servers like the GPG signing did, because the Apple and Certum
# credentials cannot follow it.
workflow_call:
workflow_dispatch:
concurrency:
# Not cancel-in-progress, unlike the QA builds. Cancelling a signing build
# partway through wastes forty minutes of work and, worse, can abandon a
# notarisation already submitted to Apple, leaving a request sitting in their
# queue for a build nobody will collect. Queue instead.
# Spelled out rather than taken from github.workflow, which in a
# reusable workflow is the CALLER's name, so every workflow Snapshot
# calls would otherwise share one group.
group: 'snapshot-macos @ ${{ github.ref }}'
cancel-in-progress: false
permissions:
contents: read
jobs:
appbundle:
runs-on: ${{ matrix.runner }}
# The signing credentials live on an environment rather than loose on the
# repository, so only the workflows that should be signing can reach them.
environment: signing
strategy:
fail-fast: false
matrix:
include:
- { name: 'arm64', arch: 'arm64', runner: 'macos-15' }
- { name: 'x64', arch: 'x86_64', runner: 'macos-26-intel' }
timeout-minutes: 180
steps:
- uses: actions/checkout@v7
- uses: actions/setup-node@v7
with:
node-version: '20.x'
- name: Enable Corepack
run: corepack enable
- name: Install the build dependencies
run: brew install syft wget
- name: Install the PostgreSQL build dependencies
uses: ./.github/actions/install-pgbuild-deps
with:
arch: ${{ matrix.arch }}
# The buildfarm keeps the Developer ID in the worker's login keychain and
# unlocks it with devkeys/unlock-csc.sh. A hosted runner has no such
# keychain, so one is created for this job alone and destroyed after it,
# which is also why the password is generated rather than configured: it
# never outlives the job.
- name: Import the signing certificate
env:
MACOS_CERTIFICATE: ${{ secrets.MACOS_CERTIFICATE }}
MACOS_CERTIFICATE_PASSWORD: ${{ secrets.MACOS_CERTIFICATE_PASSWORD }}
run: |
KEYCHAIN="${RUNNER_TEMP}/build.keychain"
KEYCHAIN_PASSWORD=$(uuidgen)
echo "KEYCHAIN=${KEYCHAIN}" >> "${GITHUB_ENV}"
echo "${MACOS_CERTIFICATE}" | base64 --decode > "${RUNNER_TEMP}/cert.p12"
security create-keychain -p "${KEYCHAIN_PASSWORD}" "${KEYCHAIN}"
security set-keychain-settings -lut 21600 "${KEYCHAIN}"
security unlock-keychain -p "${KEYCHAIN_PASSWORD}" "${KEYCHAIN}"
security import "${RUNNER_TEMP}/cert.p12" \
-k "${KEYCHAIN}" \
-P "${MACOS_CERTIFICATE_PASSWORD}" \
-T /usr/bin/codesign
# Without this, codesign prompts for permission to use the key, which
# on a headless runner means hanging until the job times out.
security set-key-partition-list -S apple-tool:,apple:,codesign: \
-s -k "${KEYCHAIN_PASSWORD}" "${KEYCHAIN}" > /dev/null
security list-keychain -d user -s "${KEYCHAIN}" login.keychain
rm -f "${RUNNER_TEMP}/cert.p12"
echo "Identities available to codesign:"
security find-identity -v -p codesigning "${KEYCHAIN}"
# build.sh decides whether to sign and notarise purely by whether these
# files exist, which is what keeps the QA build credential-free.
- name: Write the signing configuration
env:
MACOS_DEVELOPER_ID: ${{ secrets.MACOS_DEVELOPER_ID }}
APPLE_ID: ${{ secrets.APPLE_ID }}
APPLE_APP_SPECIFIC_PASSWORD: ${{ secrets.APPLE_APP_SPECIFIC_PASSWORD }}
APPLE_TEAM_ID: ${{ secrets.APPLE_TEAM_ID }}
run: |
printf 'DEVELOPER_ID="%s"\n' "${MACOS_DEVELOPER_ID}" > pkg/mac/codesign.conf
{
printf 'DEVELOPER_USER=%s\n' "${APPLE_ID}"
printf 'DEVELOPER_ASP=%s\n' "${APPLE_APP_SPECIFIC_PASSWORD}"
printf 'DEVELOPER_TEAM_ID=%s\n' "${APPLE_TEAM_ID}"
} > pkg/mac/notarization.conf
# --zip as well as the DMG: the DMG is the download, the ZIP is what the
# Electron auto-updater fetches. The buildfarm's snapshot job does the
# same.
- name: Build the signed appbundle
run: |
export PGADMIN_POSTGRES_DIR=/opt/pgbuild/postgresql
export OPENSSL_DIR=/opt/pgbuild/openssl
export OPENSSL_STATIC=1
make appbundle BUILD_OPTS="--zip"
- name: Verify the signature and notarisation
run: |
APP="mac-build/pgAdmin 4.app"
echo "=== codesign ==="
codesign --verify --deep --strict --verbose=2 "${APP}"
codesign -dvvv "${APP}" 2>&1 | grep -E 'Authority|TeamIdentifier|Identifier='
echo "=== gatekeeper assessment ==="
spctl --assess --type execute --verbose=4 "${APP}"
echo "=== stapled notarisation ==="
xcrun stapler validate "${APP}"
for F in dist/*.dmg; do
echo "--- ${F} ---"
xcrun stapler validate "${F}"
done
- name: Archive the disk image and zip
uses: actions/upload-artifact@v7
with:
name: pgadmin4-macos-snapshot-${{ matrix.name }}
if-no-files-found: error
path: |
dist/*.dmg
dist/*.zip
# Runs even on failure: a keychain holding a Developer ID key must not
# outlive the job that created it, and the conf files carry the Apple
# credentials.
- name: Clean up the credentials
if: always()
run: |
rm -f pkg/mac/codesign.conf pkg/mac/notarization.conf
if [ -n "${KEYCHAIN:-}" ] && [ -f "${KEYCHAIN}" ]; then
security delete-keychain "${KEYCHAIN}"
fi
+180
View File
@@ -0,0 +1,180 @@
# Signed Windows snapshot builds, replacing the buildfarm's
# pgadmin4-windows-x64-snapshot job.
#
# This is the one workflow that cannot use a hosted runner. The certificate is
# on a Certum hardware token plugged into the machine in Dave's rack, reached
# through the Windows machine certificate store, and a PIN handler answers the
# prompt from the interactive desktop session. None of that can be moved to a
# runner GitHub provides.
#
# Uploading is not wired up yet; this proves the signed build works.
#
# The runner must be running in the logged-in interactive session, not as a
# Windows service. A service runs in session 0 with no desktop, so the PIN
# prompt appears where the handler cannot see it and signing hangs rather than
# failing.
name: Snapshot - Windows
on:
# Callable from snapshot.yml so that the orchestrator produces one
# complete tree. Signing stays here rather than moving to the
# servers like the GPG signing did, because the Apple and Certum
# credentials cannot follow it.
workflow_call:
workflow_dispatch:
concurrency:
# Serialised deliberately, and not cancel-in-progress. There is one hardware
# token, and two builds signing through it at once is a good way to find out
# what proCertum does under contention. Jenkins may also be using it.
group: 'windows-signing'
cancel-in-progress: false
permissions:
contents: read
jobs:
installer:
runs-on: [self-hosted, windows-x64]
timeout-minutes: 120
environment: signing
env:
POSTGRES_MAJOR: '18'
steps:
- uses: actions/checkout@v7
- uses: actions/setup-node@v7
with:
node-version: '20.x'
- name: Enable Corepack
run: corepack enable
# No Python step, and none needed. actions/setup-python hangs on this
# runner: it installs Python rather than unpacking it, and anything
# raising a UAC prompt in the auto-logged-in session waits forever with
# nobody to answer it, exactly as the card's PIN dialog would. The first
# attempt sat on that step for thirty minutes against eleven seconds for
# setup-node. Using whatever Python is on PATH avoids the hang and is
# worse: this machine has 3.14 installed whilst the released installers
# carry 3.13, so it would have quietly shipped a different interpreter
# from the one the macOS build ships.
#
# Make.bat now downloads the version named in pkg/python-version.txt
# itself, as a plain tarball with no installer and so no prompt, which
# settles both problems and means this host needs no Python at all.
# Unlike the hosted build, nothing is installed here. This machine is
# configured by hand rather than by Ansible, it is the only one in the
# pipeline that cannot simply be rebuilt, and a CI job has no business
# mutating it. So check, and stop with something useful to act on rather
# than failing halfway through a build.
- name: Verify the build prerequisites
shell: powershell
run: |
$missing = @()
if (-not (Test-Path "C:\Program Files (x86)\Inno Setup 6\ISCC.exe")) { $missing += "innosetup" }
foreach ($tool in @('syft', 'wget')) {
if (-not (Get-Command $tool -ErrorAction SilentlyContinue)) { $missing += $tool }
}
if ($missing.Count) {
throw ("Missing build prerequisites on this runner: $($missing -join ', '). " +
"Install with: choco install $($missing -join ' ') -y")
}
Write-Output "All prerequisites present."
- name: Install the PostgreSQL and Kerberos dependencies
id: deps
uses: ./.github/actions/install-pgbuild-deps
with:
arch: x86_64
postgres-major: ${{ env.POSTGRES_MAJOR }}
- name: Record the dependency location
shell: powershell
run: '"DEPS_DIR=${{ steps.deps.outputs.prefix }}" | Out-File -FilePath $env:GITHUB_ENV -Encoding utf8 -Append'
# Discovered rather than hardcoded. The buildfarm pins these to specific
# paths in its Jenkins node configuration, which needs editing every time
# Visual Studio or the Windows SDK updates, and that configuration is
# going away with the buildfarm.
- name: Locate the toolchain
shell: powershell
run: |
$vswhere = "${env:ProgramFiles(x86)}\Microsoft Visual Studio\Installer\vswhere.exe"
$vs = & $vswhere -latest -products * -property installationPath
if (-not $vs) { throw "Could not locate a Visual Studio installation" }
$redist = Get-ChildItem "$vs\VC\Redist\MSVC" -Directory |
Where-Object { Test-Path "$($_.FullName)\vc_redist.x64.exe" } |
Sort-Object Name -Descending | Select-Object -First 1
if (-not $redist) { throw "Could not locate vc_redist.x64.exe under $vs" }
# signtool lives under the SDK's versioned bin directory, and also in
# the App Certification Kit, which is where the buildfarm points at.
# Take the newest SDK copy, falling back to the ACK one.
$signtool = Get-ChildItem "${env:ProgramFiles(x86)}\Windows Kits\10\bin" `
-Filter signtool.exe -Recurse -ErrorAction SilentlyContinue |
Where-Object { $_.FullName -match '\\x64\\' } |
Sort-Object FullName -Descending | Select-Object -First 1
if (-not $signtool) {
$ack = "${env:ProgramFiles(x86)}\Windows Kits\10\App Certification Kit\signtool.exe"
if (Test-Path $ack) { $signtool = Get-Item $ack }
}
if (-not $signtool) { throw "Could not locate signtool.exe" }
Write-Output "Visual Studio: $vs"
Write-Output "VC++ redist: $($redist.FullName)"
Write-Output "signtool: $($signtool.DirectoryName)"
"VS_INSTALL_DIR=$vs" | Out-File -FilePath $env:GITHUB_ENV -Encoding utf8 -Append
"PGADMIN_VCREDIST_DIR=$($redist.FullName)" | Out-File -FilePath $env:GITHUB_ENV -Encoding utf8 -Append
"PGADMIN_SIGNTOOL_DIR=$($signtool.DirectoryName)" | Out-File -FilePath $env:GITHUB_ENV -Encoding utf8 -Append
# Setting PGADMIN_WINDOWS_CSC is what makes Make.bat sign: it selects the
# certificate by subject name from the machine store, where proCertum
# installed it. Kept in a repository variable rather than the tree, so a
# certificate renewal is a settings change rather than a commit.
- name: Build the signed installer
shell: cmd
env:
PGADMIN_WINDOWS_CSC: ${{ vars.PGADMIN_WINDOWS_CSC }}
run: |
IF "%PGADMIN_WINDOWS_CSC%" == "" (
ECHO ERROR: the PGADMIN_WINDOWS_CSC repository variable is not set,
ECHO so Make.bat would silently produce an unsigned build.
EXIT /b 1
)
SET "PGADMIN_POSTGRES_DIR=%DEPS_DIR%\postgresql"
SET "PGADMIN_KRB5_DIR=%DEPS_DIR%\krb5"
SET "PGADMIN_INNOTOOL_DIR=C:\Program Files (x86)\Inno Setup 6"
CALL "%VS_INSTALL_DIR%\VC\Auxiliary\Build\vcvars64.bat" || EXIT /b 1
CALL Make.bat || EXIT /b 1
# Make.bat now fails the build if either signature fails, the executable
# inside the installer included, so this is a second opinion rather than
# the only check. It verifies what is in dist/, which is the installer;
# the executable within it is covered by Make.bat refusing to continue.
- name: Verify the signatures
shell: powershell
run: |
$signtool = Join-Path $env:PGADMIN_SIGNTOOL_DIR 'signtool.exe'
Get-ChildItem dist\*.exe | ForEach-Object {
Write-Output "--- $($_.Name) ---"
& $signtool verify /pa /v $_.FullName
if ($LASTEXITCODE -ne 0) { throw "Signature verification failed for $($_.Name)" }
}
- name: Archive the installer
uses: actions/upload-artifact@v7
with:
name: pgadmin4-windows-snapshot
if-no-files-found: error
path: dist/*.exe
+391
View File
@@ -0,0 +1,391 @@
# Nightly snapshot builds, replacing the buildfarm's pgadmin4-all-snapshot job
# and the dozen per-artefact snapshot jobs it drives.
#
# Nothing here is a second definition of a build. Every artefact comes from the
# same workflow that runs on each push, called rather than copied, so a
# snapshot cannot quietly diverge from what QA tested. That was not true of the
# buildfarm, where pgadmin4-deb-build and pgadmin4-deb-snapshot were separate
# job definitions carrying the same commands.
#
# Nothing here signs anything either, beyond the macOS notarisation and the
# Windows Authenticode signature, which have to happen where their credentials
# are. The GPG key that signs the RPMs and the repository metadata stays on the
# servers: this workflow assembles the tree and uploads it, and asks procyon to
# index and sign, so a compromised workflow run can ask for a signature but
# never holds the means to make one.
name: Snapshot
on:
workflow_dispatch:
inputs:
location:
description: >-
Snapshot directory name, normally today's date. The buildfarm
writes <date> on the same server, so a test run wants a -N
suffix, for example 2026-09-17-2.
required: false
default: ''
publish:
description: Upload to the download server when the build succeeds
type: boolean
required: false
default: false
# Nightly, matching the buildfarm's pgadmin4-all-snapshot, which ran at
# 01:01. GitHub's scheduler is best-effort and runs late under load, which
# does not matter for a snapshot: the directory is named for the date the
# run assembles it, not for the minute it started.
schedule:
- cron: '1 1 * * *'
concurrency:
# One snapshot at a time, and never cancelled: a cancelled run can abandon a
# notarisation already submitted to Apple, and leaves the signing token in an
# unknown state.
group: 'snapshot'
cancel-in-progress: false
permissions:
contents: read
jobs:
# The directory name, decided once. Two jobs need it and they must agree:
# the repo RPMs carry the snapshot's own URL inside them, so if that job and
# the assemble job each called date(1) a run straddling midnight would ship a
# repository configuration pointing at a snapshot that does not exist.
location:
runs-on: ubuntu-24.04
outputs:
name: ${{ steps.pick.outputs.name }}
steps:
- id: pick
env:
REQUESTED: ${{ inputs.location }}
run: |
set -euo pipefail
NAME="${REQUESTED:-}"
[ -n "${NAME}" ] || NAME=$(date +%Y-%m-%d)
# Validated here so that everything downstream, including a shell on
# the machine holding the publishing keys, is working with a name
# that cannot be anything but a date and an optional suffix. The
# wrapper on the server applies its own rule to the same value; this
# one exists so a bad input fails in five seconds rather than after
# an hour of building.
if ! printf '%s' "${NAME}" | grep -Eq '^[0-9]{4}-[0-9]{2}-[0-9]{2}(-[1-9][0-9]?)?$'; then
echo "Not a valid snapshot location: ${NAME}" >&2
echo "Expected YYYY-MM-DD, optionally with a -N suffix." >&2
exit 1
fi
echo "Snapshot location: ${NAME}"
echo "name=${NAME}" >> "${GITHUB_OUTPUT}"
deb:
uses: ./.github/workflows/check-deb-build.yml
rpm:
uses: ./.github/workflows/check-rpm-build.yml
tarball:
uses: ./.github/workflows/check-tarball-build.yml
wheel:
uses: ./.github/workflows/check-python-build.yml
docs:
uses: ./.github/workflows/check-doc-builds.yml
# No helm job here. The chart is not part of the download tree: the
# buildfarm pushed it to an OCI registry, which nothing in this workflow
# does yet. Calling the build and then discarding its artefact, which is what
# happened before, cost a runner and proved nothing that check-helm-build
# does not already prove on every push.
# The repository configuration RPMs. These are not built by the QA
# workflows, because they are not a build of pgAdmin: they are a one-file
# package carrying a .repo pointing at the tree they ship in, which is why
# the URL has to be known before they are built.
repo-rpm:
needs: location
runs-on: ubuntu-24.04
container: almalinux:9
steps:
- name: Install the build environment
run: dnf install -y --setopt=install_weak_deps=False rpm-build python3 git
- uses: actions/checkout@v7
- name: Mark the workspace as a safe git directory
run: git config --global --add safe.directory "${GITHUB_WORKSPACE}"
- name: Build the repository RPMs
env:
LOCATION: ${{ needs.location.outputs.name }}
run: |
# Through the environment rather than interpolated into the script.
# The location is a dispatch input, so it is text somebody chose, and
# ${{ }} substitution happens before the shell ever sees the line.
export PGADMIN_REPO_DIR="https://ftp.postgresql.org/pub/pgadmin/pgadmin4/snapshots/${LOCATION}/yum"
echo "Repositories will point at ${PGADMIN_REPO_DIR}"
pkg/redhat/repo-rpm.sh
- name: Archive the repository RPMs
uses: actions/upload-artifact@v7
with:
name: pgadmin4-repo-rpm-build-output
if-no-files-found: error
path: dist/*.noarch.rpm
macos:
uses: ./.github/workflows/snapshot-macos.yml
secrets: inherit
windows:
uses: ./.github/workflows/snapshot-windows.yml
secrets: inherit
assemble:
needs: [location, deb, rpm, tarball, wheel, docs, repo-rpm, macos, windows]
# roadie, not a hosted runner. This job assembles a tree of several
# hundred megabytes and then, when publishing, sends it to the download
# server; splitting the two meant uploading that tree as an artifact and
# immediately downloading it again on roadie, having first queued for a
# hosted runner behind whatever else the organisation was building.
runs-on: [self-hosted, roadie]
# For the variables naming the keys and the target, which live here. The
# environment carries no protection rules, so a run that is not publishing
# is not gated by it.
environment: publishing
# Everything must have built except Windows. That is the one runner which
# can legitimately be offline, and a snapshot without the installer is
# still worth having; a snapshot missing its documentation, source tarball,
# wheel or repository RPMs is not, and the repository RPMs are worse than
# missing, since install-repo-readme.py then fails after the packages have
# been uploaded and signed, leaving half a snapshot on the download server.
if: >-
always()
&& needs.location.result == 'success'
&& needs.deb.result == 'success'
&& needs.rpm.result == 'success'
&& needs.tarball.result == 'success'
&& needs.wheel.result == 'success'
&& needs.docs.result == 'success'
&& needs.repo-rpm.result == 'success'
&& needs.macos.result == 'success'
steps:
- uses: actions/checkout@v7
- name: Collect the build artefacts
uses: actions/download-artifact@v7
with:
path: artefacts
# The layout is the one the publishing scripts and the download site
# expect, and is what the wrapper's stage-create verb creates on procyon.
# The Debian architecture comes from the package filename rather than a
# table, since dpkg already encodes it there; the codename does not, so
# that mapping is explicit.
- name: Assemble the staging tree
env:
LOCATION: ${{ needs.location.outputs.name }}
run: |
set -euo pipefail
# Flat, beside apt/ and yum/, which is how both the staging server
# and the snapshots tree are laid out. The docs/macos/pip/source/
# windows subdivision belongs to a published release, and it is the
# promotion step that sorts files into it on the way there.
# Explicitly, rather than trusting the checkout to have cleaned it.
# This runs on a self-hosted runner whose workspace persists between
# jobs, and a previous run's tree left behind here is indistinguishable
# from this one's until something downstream has to choose between
# them.
rm -rf stage
STAGE="stage/${LOCATION}"
mkdir -p "${STAGE}/yum"
# An artefact can be legitimately absent: the Windows signing host is
# allowed to be offline, and a failed matrix leg is reported by its
# own job rather than by a confusing cp error here.
copy_into() {
DEST=$1; shift
for SRC in "$@"; do
[ -e "${SRC}" ] || continue
mkdir -p "${DEST}"
cp "${SRC}" "${DEST}/"
done
}
codename_for() {
case "$1" in
debian-12) echo bookworm ;;
debian-13) echo trixie ;;
ubuntu-2204) echo jammy ;;
ubuntu-2404) echo noble ;;
ubuntu-2604) echo resolute ;;
*) echo "Unknown Debian build name: $1" >&2; exit 1 ;;
esac
}
for DIR in artefacts/pgadmin4-deb-build-output-*; do
[ -d "${DIR}" ] || continue
CODENAME=$(codename_for "${DIR##*-output-}")
for DEB in "${DIR}"/*.deb; do
ARCH=$(basename "${DEB}" .deb); ARCH=${ARCH##*_}
TARGET="${STAGE}/apt/${CODENAME}/dists/pgadmin4/main/binary-${ARCH}"
mkdir -p "${TARGET}"
cp "${DEB}" "${TARGET}/"
done
done
# RPM trees are named for the distribution, which the build name
# already carries: el-9 becomes redhat/rhel-9-x86_64.
for DIR in artefacts/pgadmin4-rpm-build-output-*; do
[ -d "${DIR}" ] || continue
NAME=${DIR##*-output-}
case "${NAME}" in
el-*) FAMILY=redhat; TREE="rhel-${NAME#el-}" ;;
fedora-*) FAMILY=fedora; TREE="${NAME}" ;;
*) echo "Unknown RPM build name: ${NAME}" >&2; exit 1 ;;
esac
# Every RPM for a distribution goes in that distribution's
# tree, including the noarch ones. Taking the directory from the
# package's own architecture would file pgadmin4 and
# pgadmin4-web, both noarch, under rhel-9-noarch, where no
# client looks: dnf asks for <name>-<version>-$basearch. The
# RPM matrix is x86_64 only for now, so this is a constant; when
# aarch64 arrives it becomes a property of the build.
TARGET="${STAGE}/yum/${FAMILY}/${TREE}-x86_64"
mkdir -p "${TARGET}"
cp "${DIR}"/*.rpm "${TARGET}/"
done
copy_into "${STAGE}" artefacts/pgadmin4-tarball-build-output/*
copy_into "${STAGE}" artefacts/pgadmin4-python-build-output/*
# The version is read once here and used for the documentation
# filenames, which the build leaves unversioned as pgadmin4.pdf and
# pgAdmin4.epub whilst the download site has always carried
# pgadmin4-<version>.pdf and .epub.
VERSION=$(python3 -c "import sys; sys.path.insert(0, 'web'); import version; print(version.APP_VERSION)")
for SRC in artefacts/pgadmin4-pdf-build-output/*.pdf; do
[ -e "${SRC}" ] && cp "${SRC}" "${STAGE}/pgadmin4-${VERSION}.pdf"
done
for SRC in artefacts/pgadmin4-epub-build-output/*.epub; do
[ -e "${SRC}" ] && cp "${SRC}" "${STAGE}/pgadmin4-${VERSION}.epub"
done
# The docs tarball is what load-docs.sh unpacks on the web server, so
# it ships as a tarball rather than as the directory of files the
# build produces.
if [ -d artefacts/pgadmin4-html-build-output ]; then
tar -czf "${STAGE}/pgadmin4-${VERSION}-docs.tar.gz" \
-C artefacts/pgadmin4-html-build-output .
fi
# At the root of yum/, beside the family directories, which is
# where the README instructions tell people to fetch them from.
copy_into "${STAGE}/yum" artefacts/pgadmin4-repo-rpm-build-output/*
copy_into "${STAGE}" artefacts/pgadmin4-macos-snapshot-*/*
copy_into "${STAGE}" artefacts/pgadmin4-windows-snapshot/*
find "${STAGE}" -type f | sort
- name: Check the keys are present
if: inputs.publish
env:
PUBLISH_KEY: ${{ vars.PUBLISH_SSH_KEY || '~/.ssh/roadie-publish' }}
UPLOAD_KEY: ${{ vars.UPLOAD_SSH_KEY || '~/.ssh/roadie-upload' }}
run: |
set -euo pipefail
for TOOL in ssh rsync; do
command -v "${TOOL}" >/dev/null || {
echo "This runner has no ${TOOL}." >&2; exit 1; }
done
for KEY in "${PUBLISH_KEY/#\~/$HOME}" "${UPLOAD_KEY/#\~/$HOME}"; do
if [ ! -r "${KEY}" ]; then
echo "Missing or unreadable SSH key: ${KEY}" >&2
exit 1
fi
# A public key is readable too, and pointing -i at one fails
# later inside ssh with a message that does not mention which
# file was wrong. Both halves of a pair sit in the same
# directory with names one suffix apart, so this is an easy
# thing to configure wrongly and a cheap thing to catch.
if head -c 4 "${KEY}" | grep -q '^ssh-'; then
echo "${KEY} is a public key; -i needs the private one." >&2
echo "Drop the .pub suffix from the variable that names it." >&2
exit 1
fi
done
if [ ! -r ~/.ssh/known_hosts ]; then
echo "This runner has no ~/.ssh/known_hosts, so host key" >&2
echo "checking would fail. Add the servers' host keys." >&2
exit 1
fi
- name: Publish the snapshot
if: inputs.publish
env:
DOWNLOAD: ${{ vars.DOWNLOAD_SSH_TARGET }}
LOCATION: ${{ inputs.location }}
PUBLISH_KEY: ${{ vars.PUBLISH_SSH_KEY || '~/.ssh/roadie-publish' }}
UPLOAD_KEY: ${{ vars.UPLOAD_SSH_KEY || '~/.ssh/roadie-upload' }}
run: |
set -euo pipefail
if [ -z "${DOWNLOAD}" ]; then
echo "The DOWNLOAD_SSH_TARGET variable is not set." >&2
exit 1
fi
# One directory came out of assemble, named for the date. An explicit
# location overrides it so that a test run does not land on the
# snapshot the buildfarm is still writing on the same server.
# assemble produces exactly one directory, named for the date.
# Anything else means the artefact is not what this step thinks it
# is, and guessing which one to publish is not a decision to make at
# three in the morning.
if [ "$(find stage -mindepth 1 -maxdepth 1 -type d | wc -l)" != "1" ]; then
echo "Expected one directory in the snapshot artefact:" >&2
ls -l stage >&2
exit 1
fi
BUILT=$(basename "$(find stage -mindepth 1 -maxdepth 1 -type d)")
DEST=${LOCATION:-${BUILT}}
echo "Publishing ${BUILT} as ${DEST}"
publish() {
ssh -i "${PUBLISH_KEY/#\~/$HOME}" -o BatchMode=yes \
-o StrictHostKeyChecking=yes "${DOWNLOAD}" "$@"
}
publish snapshot-create "${DEST}"
# No --delete, deliberately: rrsync refuses it anyway, and a
# publishing step that can remove things is a publishing step that
# can remove the wrong things.
rsync -rlt --safe-links --no-specials --no-devices \
-e "ssh -i ${UPLOAD_KEY/#\~/$HOME} -o BatchMode=yes -o StrictHostKeyChecking=yes" \
"stage/${BUILT}/" "${DOWNLOAD}:${DEST}/"
# Signing before indexing: the metadata has to describe the signed
# packages rather than the unsigned ones just uploaded.
publish snapshot-sign "${DEST}"
publish snapshot-index-apt "${DEST}"
publish snapshot-index-yum "${DEST}"
# Retention, which the buildfarm did with ls | tail | xargs rm. The
# verb takes no argument: it cannot be asked to remove a particular
# snapshot, only to enforce the retention the server already has.
#
# Skipped when the directory was named by hand, because that is a
# test run, and a test run has no business deciding which of the
# real snapshots are old enough to delete.
if [ -n "${LOCATION}" ]; then
echo "Named location, so leaving retention alone."
else
publish snapshot-purge
fi
+1 -1
View File
@@ -25,7 +25,7 @@ jobs:
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v7
with:
fetch-depth: 0
+507 -443
View File
@@ -1,443 +1,507 @@
@ECHO off
SETLOCAL
SET WD=%CD%
SET "BUILDROOT=%WD%\win-build"
SET "TMPDIR=%WD%\win-temp"
SET "DISTROOT=%WD%\dist"
SET CMDOPTIONS=""
IF "%1" == "clean" SET CMDOPTIONS="VALID"
IF "%1" == "" SET CMDOPTIONS="VALID"
IF NOT %CMDOPTIONS%=="VALID" (
GOTO USAGE
)
IF "%1" == "clean" (
CALL :CLEAN
EXIT /B %ERRORLEVEL%
)
REM Main build sequence
CALL :SET_ENVIRONMENT
CALL :VALIDATE_ENVIRONMENT || EXIT /B 1
CALL :CLEAN || EXIT /B 1
CALL :CREATE_VIRTUAL_ENV || EXIT /B 1
CALL :CREATE_PYTHON_ENV || EXIT /B 1
CALL :CREATE_RUNTIME_ENV || EXIT /B 1
CALL :GENERATE_SBOM || EXIT /B 1
CALL :CREATE_INSTALLER || EXIT /B 1
CALL :VERIFY_SIGNATURE || EXIT /B 1
EXIT /B %ERRORLEVEL%
REM Main build sequence Ends
:CLEAN
ECHO Removing build directory...
IF EXIST "%BUILDROOT%" RD "%BUILDROOT%" /S /Q > nul || EXIT /B 1
ECHO Removing tmp directory...
IF EXIST "%TMPDIR%" RD "%TMPDIR%" /S /Q > nul || EXIT /B 1
ECHO Removing installer build directory...
IF EXIST "%WD%\pkg\win32\Output" rd "%WD%\pkg\win32\Output" /S /Q > nul || EXIT /B 1
ECHO Removing installer configuration script...
IF EXIST DEL /q "%WD%\pkg\win32\installer.iss" > nul || EXIT /B 1
EXIT /B 0
:SET_ENVIRONMENT
ECHO Configuring the environment...
IF "%PGADMIN_PYTHON_DIR%" == "" SET "PGADMIN_PYTHON_DIR=C:\Python314"
IF "%PGADMIN_KRB5_DIR%" == "" SET "PGADMIN_KRB5_DIR=C:\Program Files\MIT\Kerberos"
IF "%PGADMIN_POSTGRES_DIR%" == "" SET "PGADMIN_POSTGRES_DIR=C:\Program Files\PostgreSQL\17"
IF "%PGADMIN_INNOTOOL_DIR%" == "" SET "PGADMIN_INNOTOOL_DIR=C:\Program Files (x86)\Inno Setup 6"
IF "%PGADMIN_VCREDIST_DIR%" == "" SET "PGADMIN_VCREDIST_DIR=C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Redist\MSVC\14.40.33807"
IF "%PGADMIN_VCREDIST_FILE%" == "" SET "PGADMIN_VCREDIST_FILE=vc_redist.x64.exe"
IF "%PGADMIN_SIGNTOOL_DIR%" == "" SET "PGADMIN_SIGNTOOL_DIR=C:\Program Files (x86)\Windows Kits\10\bin\10.0.22621.0\x64"
IF "%PGADMIN_WINDOWS_CSC%" == "" SET "PGADMIN_WINDOWS_CSC="
REM Set additional variables we need
FOR /F "tokens=3" %%a IN ('findstr /C:"APP_RELEASE =" %WD%\web\version.py') DO SET APP_MAJOR=%%a
FOR /F "tokens=3" %%a IN ('findstr /C:"APP_REVISION =" %WD%\web\version.py') DO SET APP_MINOR=%%a
FOR /F "tokens=3" %%a IN ('findstr /C:"APP_SUFFIX =" %WD%\web\version.py') DO SET APP_VERSION_SUFFIX=%%a
REM remove single quote from the string
SET APP_VERSION_SUFFIX=%APP_VERSION_SUFFIX:'=%
SET APP_NAME=""
FOR /F "tokens=2* DELims='" %%a IN ('findstr /C:"APP_NAME =" web\branding.py') DO SET APP_NAME=%%a
FOR /f "tokens=1 DELims=." %%G IN ('%PGADMIN_PYTHON_DIR%/python.exe -c "print('%APP_NAME%'.lower().replace(' ', ''))"') DO SET APP_SHORTNAME=%%G
SET APP_VERSION=%APP_MAJOR%.%APP_MINOR%
SET INSTALLERNAME=%APP_SHORTNAME%-%APP_MAJOR%.%APP_MINOR%-%APP_VERSION_SUFFIX%-x64.exe
IF "%APP_VERSION_SUFFIX%" == "" SET INSTALLERNAME=%APP_SHORTNAME%-%APP_MAJOR%.%APP_MINOR%-x64.exe
REM get Python version for the runtime build ex. 3.9.2 will be 39
FOR /f "tokens=1 DELims=." %%G IN ('%PGADMIN_PYTHON_DIR%/python.exe -c "import sys; print(sys.version.split(' ')[0])"') DO SET PYTHON_MAJOR=%%G
FOR /f "tokens=2 DELims=." %%G IN ('%PGADMIN_PYTHON_DIR%/python.exe -c "import sys; print(sys.version.split(' ')[0])"') DO SET PYTHON_MINOR=%%G
FOR /f "tokens=3 DELims=." %%G IN ('%PGADMIN_PYTHON_DIR%/python.exe -c "import sys; print(sys.version.split(' ')[0])"') DO SET PYTHON_REVISION=%%G
EXIT /B 0
:VALIDATE_ENVIRONMENT
ECHO ****************************************************************
ECHO * Build summary
ECHO ****************************************************************
ECHO Build path: %BUILDROOT%
ECHO Output directory: %DISTROOT%
ECHO Installer name: %INSTALLERNAME%
ECHO.
ECHO Python directory: %PGADMIN_PYTHON_DIR%
ECHO Python version: %PYTHON_MAJOR%.%PYTHON_MINOR%.%PYTHON_REVISION%
ECHO.
ECHO KRB5 directory: %PGADMIN_KRB5_DIR%
ECHO PostgreSQL directory: %PGADMIN_POSTGRES_DIR%
ECHO.
ECHO VC++ redist directory: %PGADMIN_VCREDIST_DIR%
ECHO VC++ redist file: %PGADMIN_VCREDIST_FILE%
ECHO InnoTool directory: %PGADMIN_INNOTOOL_DIR%
ECHO signtool directory: %PGADMIN_SIGNTOOL_DIR%
IF "%PGADMIN_WINDOWS_CSC%" == "" (
ECHO Code signing certificate: [NONE - Signing disabled]
) ELSE (
ECHO Code signing certificate: %PGADMIN_WINDOWS_CSC%
)
ECHO.
ECHO App version: %APP_VERSION%
ECHO App version suffix: %APP_VERSION_SUFFIX%
ECHO App short name: %APP_SHORTNAME%
ECHO App name: %APP_NAME%
ECHO ****************************************************************
ECHO Checking the environment...
IF NOT EXIST "%PGADMIN_INNOTOOL_DIR%" (
ECHO !PGADMIN_INNOTOOL_DIR! does not exist
ECHO Please install InnoTool and set the PGADMIN_INNOTOOL_DIR environment variable.
EXIT /B 1
)
IF NOT EXIST "%PGADMIN_VCREDIST_DIR%" (
ECHO !PGADMIN_VCREDIST_DIR! does not exist
ECHO Please install Microsoft Visual studio and set the PGADMIN_VCREDIST_DIR environment variable.
EXIT /B 1
)
IF NOT EXIST "%PGADMIN_KRB5_DIR%" (
ECHO !PGADMIN_KRB5_DIR! does not exist.
ECHO Please install MIT Kerberos for Windows and set the PGADMIN_KRB5_DIR environment variable.
EXIT /B 1
)
IF NOT EXIST "%PGADMIN_PYTHON_DIR%" (
ECHO !PGADMIN_PYTHON_DIR! does not exist.
ECHO Please install Python and set the PGADMIN_PYTHON_DIR environment variable.
EXIT /B 1
)
IF NOT EXIST "%PGADMIN_POSTGRES_DIR%" (
ECHO !PGADMIN_POSTGRES_DIR! does not exist.
ECHO Please install PostgreSQL and set the PGADMIN_POSTGRES_DIR environment variable.
EXIT /B 1
)
IF NOT EXIST "%PGADMIN_PYTHON_DIR%\Scripts\virtualenv.exe" (
ECHO !PGADMIN_PYTHON_DIR!\Scripts\virtualenv.exe does not exist.
ECHO Please install the virtualenv package in Python.
EXIT /B 1
)
SET "PATH=%PGADMIN_POSTGRES_DIR%\bin;%PATH%"
EXIT /B 0
:CREATE_VIRTUAL_ENV
ECHO Creating virtual environment...
IF NOT EXIST "%TMPDIR%" MKDIR "%TMPDIR%"
CD "%TMPDIR%"
REM Note that we must use virtualenv.exe here, as the venv module doesn't allow python.exe to relocate.
"%PGADMIN_PYTHON_DIR%\Scripts\virtualenv.exe" venv || EXIT /B 1
XCOPY /S /I /E /H /Y "%PGADMIN_PYTHON_DIR%\DLLs" "%TMPDIR%\venv\DLLs" > nul || EXIT /B 1
REM Copy the standard library, but NOT site-packages: the venv already has its
REM own seeded pip there, and overwriting only the files the system Python also
REM has leaves a mix of two pip versions behind.
ROBOCOPY /E /R:3 /W:5 /NFL /NDL /NP "%PGADMIN_PYTHON_DIR%\Lib" "%TMPDIR%\venv\Lib" /XD site-packages
CALL :CHECK_ROBOCOPY_ERROR || EXIT /B 1
ECHO Activating virtual environment - %TMPDIR%\venv...
CALL "%TMPDIR%\venv\Scripts\activate" || EXIT /B 1
ECHO Installing dependencies...
CALL python -m pip install --upgrade pip || EXIT /B 1
CALL pip install --only-binary=cryptography -r "%WD%\requirements.txt" || EXIT /B 1
CD %WD%
EXIT /B 0
:CREATE_PYTHON_ENV
ECHO Staging Python...
MKDIR "%BUILDROOT%\python\Lib" || EXIT /B 1
ECHO Downloading embedded Python...
REM Get the python embeddable and extract it to %BUILDROOT%\python
CD "%TMPDIR%
%PGADMIN_PYTHON_DIR%\python -c "import sys; from urllib.request import urlretrieve; urlretrieve('https://www.python.org/ftp/python/' + sys.version.split(' ')[0] + '/python-' + sys.version.split(' ')[0] + '-embed-amd64.zip', 'python-embedded.zip')" || EXIT /B 1
%PGADMIN_PYTHON_DIR%\python -c "import zipfile; z = zipfile.ZipFile('python-embedded.zip', 'r'); z.extractall('../win-build/python/')" || EXIT /B 1
ECHO Copying site-packages...
XCOPY /S /I /E /H /Y "%TMPDIR%\venv\Lib\site-packages" "%BUILDROOT%\python\Lib\site-packages" > nul || EXIT /B 1
REM NOTE: There is intentionally no space after "site" in the line below, to prevent Python barfing if there's one in the file
ECHO import site>> "%BUILDROOT%\python\python%PYTHON_MAJOR%%PYTHON_MINOR%._pth"
ECHO Staging Kerberos components...
COPY "%PGADMIN_KRB5_DIR%\bin\kinit.exe" "%BUILDROOT%\python" > nul || EXIT /B 1
COPY "%PGADMIN_KRB5_DIR%\bin\krb5_64.dll" "%BUILDROOT%\python" > nul || EXIT /B 1
COPY "%PGADMIN_KRB5_DIR%\bin\comerr64.dll" "%BUILDROOT%\python" > nul || EXIT /B 1
COPY "%PGADMIN_KRB5_DIR%\bin\k5sprt64.dll" "%BUILDROOT%\python" > nul || EXIT /B 1
COPY "%PGADMIN_KRB5_DIR%\bin\gssapi64.dll" "%BUILDROOT%\python" > nul || EXIT /B 1
ECHO Cleaning up unnecessary .pyc and .pyo files...
FOR /R "%BUILDROOT%\python" %%f in (*.pyc *.pyo) do DEL /q "%%f" 1> nul 2>&1
ECHO Removing tests...
FOR /R "%BUILDROOT%\python\Lib" %%f in (test tests) do RD /Q /S "%%f" 1> nul 2>&1
EXIT /B 0
:CREATE_RUNTIME_ENV
IF NOT EXIST "%BUILDROOT%" MKDIR "%BUILDROOT%"
MKDIR "%BUILDROOT%\runtime"
ECHO Removing webpack caches...
RD /Q /S "%WD%\web\pgadmin\static\js\generated\.cache" 1> nul 2>&1
ECHO Copying web directory...
ROBOCOPY /S /NFL /NDL /NP "%WD%\web" "%BUILDROOT%\web"
CALL :CHECK_ROBOCOPY_ERROR || EXIT /B 1
ECHO Installing javascript dependencies...
CD "%BUILDROOT%\web"
SET "YARN_VERSION="
FOR /f "delims=" %%v IN ('node -p "require('./package.json').packageManager.split('@')[1]"') DO SET "YARN_VERSION=%%v"
IF "%YARN_VERSION%"=="" (
ECHO ERROR: Could not determine Yarn version from package.json packageManager field.
EXIT /B 1
)
CALL yarn set version %YARN_VERSION% || EXIT /B 1
CALL yarn install || EXIT /B 1
CALL npm rebuild || EXIT /B 1
ECHO Bundling javascript...
CALL yarn run bundle || EXIT /B 1
ECHO Cleaning up unnecessary .pyc and .pyo files...
FOR /R "%BUILDROOT%\web" %%f in (*.pyc *.pyo) do DEL /q "%%f" 1> nul 2>&1
ECHO Removing tests, Python caches and node modules...
FOR /R "%BUILDROOT%\web" %%f in (tests feature_tests __pycache__ node_modules) do RD /Q /S "%%f" 1> nul 2>&1
ECHO Removing the test framework...
RD /Q /S "%BUILDROOT%\web\regression" 1> nul 2>&1
ECHO Removing tools...
RD /Q /S "%BUILDROOT%\web\tools" 1> nul 2>&1
ECHO Removing the JavaScript build configuration...
RD /Q /S "%BUILDROOT%\web\.yarn" 1> nul 2>&1
DEL /q "%BUILDROOT%\web\yarn.lock" 1> nul 2>&1
DEL /q "%BUILDROOT%\web\.yarnrc.yml" 1> nul 2>&1
DEL /q "%BUILDROOT%\web\package.json" 1> nul 2>&1
DEL /q "%BUILDROOT%\web\jest.config.js" 1> nul 2>&1
DEL /q "%BUILDROOT%\web\babel.cfg" 1> nul 2>&1
DEL /q "%BUILDROOT%\web\babel.config.json" 1> nul 2>&1
DEL /q "%BUILDROOT%\web\webpack.config.js" 1> nul 2>&1
DEL /q "%BUILDROOT%\web\webpack.shim.js" 1> nul 2>&1
DEL /q "%BUILDROOT%\web\.eslintrc.js" 1> nul 2>&1
DEL /q "%BUILDROOT%\web\.editorconfig" 1> nul 2>&1
ECHO Removing any existing configurations...
DEL /q "%BUILDROOT%\web\pgadmin4.db" 1> nul 2>&1
DEL /q "%BUILDROOT%\web\config_local.py" 1> nul 2>&1
ECHO Staging license files...
COPY "%WD%\LICENSE" "%BUILDROOT%\" > nul || EXIT /B 1
ECHO Creating config_distro.py
ECHO SERVER_MODE = False > "%BUILDROOT%\web\config_distro.py"
ECHO HELP_PATH = '../../../docs/en_US/html/' >> "%BUILDROOT%\web\config_distro.py"
ECHO DEFAULT_BINARY_PATHS = { >> "%BUILDROOT%\web\config_distro.py"
ECHO 'pg': '$DIR/../runtime', >> "%BUILDROOT%\web\config_distro.py"
ECHO 'ppas': '' >> "%BUILDROOT%\web\config_distro.py"
ECHO } >> "%BUILDROOT%\web\config_distro.py"
ECHO Building docs...
CALL pip install sphinx || EXIT /B 1
CALL pip install sphinxcontrib-youtube || EXIT /B 1
MKDIR "%BUILDROOT%\docs\en_US\html"
CD "%WD%\docs\en_US"
CALL "%TMPDIR%\venv\Scripts\python.exe" build_code_snippet.py || EXIT /B 1
CALL "%TMPDIR%\venv\Scripts\sphinx-build.exe" "%WD%\docs\en_US" "%BUILDROOT%\docs\en_US\html" || EXIT /B 1
REM Remove unnecessary doc files
DEL /q "%BUILDROOT%\docs\en_US\html\_static\*.png" 1> nul 2>&1
RD /Q /S "%BUILDROOT%\docs\en_US\html\_sources" 1> nul 2>&1
ECHO Staging runtime components...
MKDIR "%BUILDROOT%\runtime\resources\app"
XCOPY /S /I /E /H /Y "%WD%\runtime\assets" "%BUILDROOT%\runtime\resources\app\assets" > nul || EXIT /B 1
XCOPY /S /I /E /H /Y "%WD%\runtime\src" "%BUILDROOT%\runtime\resources\app\src" > nul || EXIT /B 1
COPY "%WD%\runtime\package.json" "%BUILDROOT%\runtime\resources\app\" > nul || EXIT /B 1
COPY "%WD%\runtime\.yarnrc.yml" "%BUILDROOT%\runtime\resources\app\" > nul || EXIT /B 1
CD "%BUILDROOT%\runtime\resources\app\"
SET "YARN_VERSION="
FOR /f "delims=" %%v IN ('node -p "require('./package.json').packageManager.split('@')[1]"') DO SET "YARN_VERSION=%%v"
IF "%YARN_VERSION%"=="" (
ECHO ERROR: Could not determine Yarn version from package.json packageManager field.
EXIT /B 1
)
CALL yarn set version %YARN_VERSION% || EXIT /B 1
CALL yarn workspaces focus --production || EXIT /B 1
ECHO Removing yarn cache...
RD /Q /S "%BUILDROOT%\runtime\resources\app\.yarn" 1> nul 2>&1
ECHO Downloading Electron to %TMPDIR%...
REM Get a fresh copy of electron.
REM WGET
FOR /f "tokens=*" %%i IN ('npm info electron version') DO SET "ELECTRON_VERSION=%%i"
:GET_NW
wget https://github.com/electron/electron/releases/download/v%ELECTRON_VERSION%/electron-v%ELECTRON_VERSION%-win32-x64.zip -O "%TMPDIR%\electron-v%ELECTRON_VERSION%-win32-x64.zip"
IF %ERRORLEVEL% NEQ 0 GOTO GET_NW
MKDIR "%TMPDIR%\electron-v%ELECTRON_VERSION%-win32-x64" || EXIT /B 1
tar -C "%TMPDIR%\electron-v%ELECTRON_VERSION%-win32-x64" -xvf "%TMPDIR%\electron-v%ELECTRON_VERSION%-win32-x64.zip" || EXIT /B 1
REM WGET END
REM XCOPY
XCOPY /S /I /E /H /Y "%TMPDIR%\electron-v%ELECTRON_VERSION%-win32-x64\*" "%BUILDROOT%\runtime" > nul || EXIT /B 1
REM XCOPY END
MOVE "%BUILDROOT%\runtime\electron.exe" "%BUILDROOT%\runtime\pgAdmin4.exe"
ECHO Downloading rcedit.exe...
wget https://github.com/electron/rcedit/releases/download/v2.0.0/rcedit-x64.exe -O "%TMPDIR%\rcedit-x64.exe"
ECHO Replacing executable icon, description, version...
%TMPDIR%\rcedit-x64.exe "%BUILDROOT%\runtime\pgAdmin4.exe" --set-icon "%WD%\pkg\win32\Resources\pgAdmin4.ico"
%TMPDIR%\rcedit-x64.exe "%BUILDROOT%\runtime\pgAdmin4.exe" --set-version-string "FileDescription" "%APP_NAME%"
%TMPDIR%\rcedit-x64.exe "%BUILDROOT%\runtime\pgAdmin4.exe" --set-version-string "ProductName" "%APP_NAME%"
%TMPDIR%\rcedit-x64.exe "%BUILDROOT%\runtime\pgAdmin4.exe" --set-product-version "%APP_VERSION%"
IF NOT "%PGADMIN_WINDOWS_CSC%" == "" (
ECHO Attempting to sign the pgAdmin4.exe...
CALL "%PGADMIN_SIGNTOOL_DIR%\signtool.exe" sign /sm /n "%PGADMIN_WINDOWS_CSC%" /tr http://timestamp.digicert.com /td sha256 /fd sha1 /v "%BUILDROOT%\runtime\pgAdmin4.exe"
IF %ERRORLEVEL% NEQ 0 (
ECHO.
ECHO ************************************************************
ECHO * Failed to sign the pgAdmin4.exe
ECHO ************************************************************
PAUSE
)
) ELSE (
ECHO Skipping code signing ^(PGADMIN_WINDOWS_CSC is not set^)...
)
ECHO Staging PostgreSQL components...
COPY "%PGADMIN_POSTGRES_DIR%\bin\libpq.dll" "%BUILDROOT%\runtime" > nul || EXIT /B 1
COPY "%PGADMIN_POSTGRES_DIR%\bin\libcrypto-*-x64.dll" "%BUILDROOT%\runtime" > nul || EXIT /B 1
COPY "%PGADMIN_POSTGRES_DIR%\bin\libssl-*-x64.dll" "%BUILDROOT%\runtime" > nul || EXIT /B 1
IF EXIST "%PGADMIN_POSTGRES_DIR%\bin\libintl-*.dll" COPY "%PGADMIN_POSTGRES_DIR%\bin\libintl-*.dll" "%BUILDROOT%\runtime" > nul
IF EXIST "%PGADMIN_POSTGRES_DIR%\bin\libiconv-*.dll" COPY "%PGADMIN_POSTGRES_DIR%\bin\libiconv-*.dll" "%BUILDROOT%\runtime" > nul
IF EXIST "%PGADMIN_POSTGRES_DIR%\bin\liblz4.dll" COPY "%PGADMIN_POSTGRES_DIR%\bin\liblz4.dll" "%BUILDROOT%\runtime" > nul
IF EXIST "%PGADMIN_POSTGRES_DIR%\bin\libzstd.dll" COPY "%PGADMIN_POSTGRES_DIR%\bin\libzstd.dll" "%BUILDROOT%\runtime" > nul
COPY "%PGADMIN_POSTGRES_DIR%\bin\zlib1.dll" "%BUILDROOT%\runtime" > nul || EXIT /B 1
COPY "%PGADMIN_POSTGRES_DIR%\bin\pg_dump.exe" "%BUILDROOT%\runtime" > nul || EXIT /B 1
COPY "%PGADMIN_POSTGRES_DIR%\bin\pg_dumpall.exe" "%BUILDROOT%\runtime" > nul || EXIT /B 1L%
COPY "%PGADMIN_POSTGRES_DIR%\bin\pg_restore.exe" "%BUILDROOT%\runtime" > nul || EXIT /B 1
COPY "%PGADMIN_POSTGRES_DIR%\bin\psql.exe" "%BUILDROOT%\runtime" > nul || EXIT /B 1
ECHO Staging VC++ runtime...
MKDIR "%BUILDROOT%\installer" || EXIT /B 1
COPY "%PGADMIN_VCREDIST_DIR%\%PGADMIN_VCREDIST_FILE%" "%BUILDROOT%\installer" > nul || EXIT /B 1
CD %WD%
EXIT /B 0
:CREATE_INSTALLER
ECHO Preparing for creation of windows installer...
IF NOT EXIST "%DISTROOT%" MKDIR "%DISTROOT%"
ECHO Copying icon file...
COPY "%WD%\pkg\win32\Resources\pgAdmin4.ico" "%BUILDROOT%" > nul || EXIT /B 1
CD "%WD%\pkg\win32"
ECHO Processing installer configuration script...
CALL "%PGADMIN_PYTHON_DIR%\python" "%WD%\pkg\win32\replace.py" "-i" "%WD%\pkg\win32\installer.iss.in" "-o" "%WD%\pkg\win32\installer.iss.in_stage1" "-s" MYAPP_FULLVERSION -r """%APP_VERSION%"""
CALL "%PGADMIN_PYTHON_DIR%\python" "%WD%\pkg\win32\replace.py" "-i" "%WD%\pkg\win32\installer.iss.in_stage1" "-o" "%WD%\pkg\win32\installer.iss.in_stage2" "-s" MYAPP_VERSION -r """v%APP_MAJOR%"""
CALL "%PGADMIN_PYTHON_DIR%\python" "%WD%\pkg\win32\replace.py" "-i" "%WD%\pkg\win32\installer.iss.in_stage2" "-o" "%WD%\pkg\win32\installer.iss" "-s" MYAPP_VCDIST -r """%PGADMIN_VCREDIST_DIRNAME%\%PGADMIN_VCREDIST_FILE%"""
ECHO Cleaning up...
DEL /s "%WD%\pkg\win32\installer.iss.in_stage*" > nul
ECHO Creating windows installer using INNO tool...
IF NOT "%PGADMIN_WINDOWS_CSC%" == "" (
CALL "%PGADMIN_INNOTOOL_DIR%\ISCC.exe" "%WD%\pkg\win32\installer.iss" "/SpgAdminSigntool=%PGADMIN_SIGNTOOL_DIR%\signtool.exe sign /sm /n $q%PGADMIN_WINDOWS_CSC%$q /tr http://timestamp.digicert.com /td sha256 /fd sha1 /v $f" || EXIT /B 1
) ELSE (
CALL "%PGADMIN_INNOTOOL_DIR%\ISCC.exe" "%WD%\pkg\win32\installer.iss" || EXIT /B 1
)
ECHO Renaming installer...
MOVE "%WD%\pkg\win32\Output\pgadmin4-setup.exe" "%DISTROOT%\%INSTALLERNAME%" > nul || EXIT /B 1
ECHO Location - %DISTROOT%\%INSTALLERNAME%
ECHO Installer generated successfully.
CD %WD%
EXIT /B 0
:GENERATE_SBOM
ECHO Generating SBOM...
CALL syft "%BUILDROOT%" -o cyclonedx-json > "%BUILDROOT%\sbom.json"
EXIT /B 0
:VERIFY_SIGNATURE
IF "%PGADMIN_WINDOWS_CSC%" == "" (
ECHO Skipping signature verification ^(PGADMIN_WINDOWS_CSC is not set^)...
EXIT /B 0
)
ECHO Verifying the installer signature...
CALL "%PGADMIN_SIGNTOOL_DIR%\signtool.exe" verify /pa /v "%DISTROOT%\%INSTALLERNAME%"
IF %ERRORLEVEL% NEQ 0 (
ECHO.
ECHO ************************************************************
ECHO * Failed to verify signature of the installer
ECHO ************************************************************
PAUSE
)
EXIT /B 0
:USAGE
ECHO Invalid command line options.
ECHO Usage: "Make.bat [clean]"
ECHO.
EXIT /B 1
:CHECK_ROBOCOPY_ERROR
IF %ERRORLEVEL% GEQ 8 EXIT /B %ERRORLEVEL%
EXIT /B 0
@ECHO off
SETLOCAL
SET WD=%CD%
SET "BUILDROOT=%WD%\win-build"
SET "TMPDIR=%WD%\win-temp"
SET "DISTROOT=%WD%\dist"
SET CMDOPTIONS=""
IF "%1" == "clean" SET CMDOPTIONS="VALID"
IF "%1" == "" SET CMDOPTIONS="VALID"
IF NOT %CMDOPTIONS%=="VALID" (
GOTO USAGE
)
IF "%1" == "clean" (
CALL :CLEAN
EXIT /B %ERRORLEVEL%
)
REM Main build sequence
REM
REM The clean runs first because :FETCH_PYTHON unpacks into the temp directory
REM that it removes, and the fetch runs before :SET_ENVIRONMENT because the
REM latter has no interpreter of its own to ask about the build.
CALL :CLEAN || EXIT /B 1
CALL :FETCH_PYTHON || EXIT /B 1
CALL :SET_ENVIRONMENT
CALL :VALIDATE_ENVIRONMENT || EXIT /B 1
CALL :CREATE_VIRTUAL_ENV || EXIT /B 1
CALL :CREATE_PYTHON_ENV || EXIT /B 1
CALL :CREATE_RUNTIME_ENV || EXIT /B 1
CALL :GENERATE_SBOM || EXIT /B 1
CALL :CREATE_INSTALLER || EXIT /B 1
CALL :VERIFY_SIGNATURE || EXIT /B 1
EXIT /B %ERRORLEVEL%
REM Main build sequence Ends
:CLEAN
ECHO Removing build directory...
IF EXIST "%BUILDROOT%" RD "%BUILDROOT%" /S /Q > nul || EXIT /B 1
ECHO Removing tmp directory...
IF EXIST "%TMPDIR%" RD "%TMPDIR%" /S /Q > nul || EXIT /B 1
ECHO Removing installer build directory...
IF EXIST "%WD%\pkg\win32\Output" rd "%WD%\pkg\win32\Output" /S /Q > nul || EXIT /B 1
ECHO Removing installer configuration script...
IF EXIST DEL /q "%WD%\pkg\win32\installer.iss" > nul || EXIT /B 1
EXIT /B 0
:FETCH_PYTHON
REM Fetch the Python this build uses, rather than using whatever happens to
REM be installed on the machine running it. Which interpreter pgAdmin ships
REM is a decision recorded in pkg\python-version.txt, not a property of the
REM build agent, and this is the only place on Windows that reads it.
REM
REM The distribution comes from astral-sh/python-build-standalone: it is
REM relocatable, published per exact CPython version, and is a plain tarball
REM rather than an installer, so unpacking it raises no UAC prompt. That
REM matters on the signing host, where the desktop session is logged in
REM automatically and an elevation dialog waits forever for somebody who is
REM not there.
REM
REM curl.exe and tar.exe have shipped in Windows since 1803, so this needs
REM nothing installed to bootstrap itself, not even a Python.
ECHO Reading the Python version...
SET "PYTHON_VERSION="
SET "PYTHON_BUILD_STANDALONE_TAG="
FOR /F "eol=# usebackq tokens=1,2 delims==" %%a IN ("%WD%\pkg\python-version.txt") DO (
IF "%%a" == "PYTHON_VERSION" SET "PYTHON_VERSION=%%b"
IF "%%a" == "PYTHON_BUILD_STANDALONE_TAG" SET "PYTHON_BUILD_STANDALONE_TAG=%%b"
)
IF "%PYTHON_VERSION%" == "" (
ECHO PYTHON_VERSION is not set in pkg\python-version.txt.
EXIT /B 1
)
IF "%PYTHON_BUILD_STANDALONE_TAG%" == "" (
ECHO PYTHON_BUILD_STANDALONE_TAG is not set in pkg\python-version.txt.
EXIT /B 1
)
REM Split for the runtime build, where e.g. 3.13.15 becomes 313
FOR /F "tokens=1,2,3 delims=." %%a IN ("%PYTHON_VERSION%") DO (
SET "PYTHON_MAJOR=%%a"
SET "PYTHON_MINOR=%%b"
SET "PYTHON_REVISION=%%c"
)
SET "PGADMIN_PYTHON_DIR=%TMPDIR%\python"
SET "PYTHON_ARCHIVE=cpython-%PYTHON_VERSION%+%PYTHON_BUILD_STANDALONE_TAG%-x86_64-pc-windows-msvc-install_only.tar.gz"
IF NOT EXIST "%TMPDIR%" MKDIR "%TMPDIR%"
ECHO Downloading Python %PYTHON_VERSION%...
curl.exe --fail --location --silent --show-error --output "%TMPDIR%\python.tar.gz" "https://github.com/astral-sh/python-build-standalone/releases/download/%PYTHON_BUILD_STANDALONE_TAG%/%PYTHON_ARCHIVE%" || EXIT /B 1
ECHO Unpacking Python...
REM The archive unpacks to a single python\ directory.
tar -x -f "%TMPDIR%\python.tar.gz" -C "%TMPDIR%" || EXIT /B 1
DEL /q "%TMPDIR%\python.tar.gz" > nul 2>&1
"%PGADMIN_PYTHON_DIR%\python.exe" --version || EXIT /B 1
REM :CREATE_VIRTUAL_ENV needs virtualenv rather than the venv module, as it
REM relocates the result. pip is bundled with the distribution, but ask
REM ensurepip for it anyway so this does not quietly depend on that.
ECHO Installing virtualenv...
"%PGADMIN_PYTHON_DIR%\python.exe" -m ensurepip --upgrade || EXIT /B 1
"%PGADMIN_PYTHON_DIR%\python.exe" -m pip install --upgrade pip virtualenv || EXIT /B 1
EXIT /B 0
:SET_ENVIRONMENT
ECHO Configuring the environment...
IF "%PGADMIN_KRB5_DIR%" == "" SET "PGADMIN_KRB5_DIR=C:\Program Files\MIT\Kerberos"
IF "%PGADMIN_POSTGRES_DIR%" == "" SET "PGADMIN_POSTGRES_DIR=C:\Program Files\PostgreSQL\17"
IF "%PGADMIN_INNOTOOL_DIR%" == "" SET "PGADMIN_INNOTOOL_DIR=C:\Program Files (x86)\Inno Setup 6"
IF "%PGADMIN_VCREDIST_DIR%" == "" SET "PGADMIN_VCREDIST_DIR=C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Redist\MSVC\14.40.33807"
IF "%PGADMIN_VCREDIST_FILE%" == "" SET "PGADMIN_VCREDIST_FILE=vc_redist.x64.exe"
IF "%PGADMIN_SIGNTOOL_DIR%" == "" SET "PGADMIN_SIGNTOOL_DIR=C:\Program Files (x86)\Windows Kits\10\bin\10.0.22621.0\x64"
IF "%PGADMIN_WINDOWS_CSC%" == "" SET "PGADMIN_WINDOWS_CSC="
REM Set additional variables we need
FOR /F "tokens=3" %%a IN ('findstr /C:"APP_RELEASE =" %WD%\web\version.py') DO SET APP_MAJOR=%%a
FOR /F "tokens=3" %%a IN ('findstr /C:"APP_REVISION =" %WD%\web\version.py') DO SET APP_MINOR=%%a
FOR /F "tokens=3" %%a IN ('findstr /C:"APP_SUFFIX =" %WD%\web\version.py') DO SET APP_VERSION_SUFFIX=%%a
REM remove single quote from the string
SET APP_VERSION_SUFFIX=%APP_VERSION_SUFFIX:'=%
SET APP_NAME=""
FOR /F "tokens=2* DELims='" %%a IN ('findstr /C:"APP_NAME =" web\branding.py') DO SET APP_NAME=%%a
FOR /f "tokens=1 DELims=." %%G IN ('%PGADMIN_PYTHON_DIR%/python.exe -c "print('%APP_NAME%'.lower().replace(' ', ''))"') DO SET APP_SHORTNAME=%%G
SET APP_VERSION=%APP_MAJOR%.%APP_MINOR%
SET INSTALLERNAME=%APP_SHORTNAME%-%APP_MAJOR%.%APP_MINOR%-%APP_VERSION_SUFFIX%-x64.exe
IF "%APP_VERSION_SUFFIX%" == "" SET INSTALLERNAME=%APP_SHORTNAME%-%APP_MAJOR%.%APP_MINOR%-x64.exe
EXIT /B 0
:VALIDATE_ENVIRONMENT
ECHO ****************************************************************
ECHO * Build summary
ECHO ****************************************************************
ECHO Build path: %BUILDROOT%
ECHO Output directory: %DISTROOT%
ECHO Installer name: %INSTALLERNAME%
ECHO.
ECHO Python version: %PYTHON_MAJOR%.%PYTHON_MINOR%.%PYTHON_REVISION%
ECHO Python directory: %PGADMIN_PYTHON_DIR%
ECHO.
ECHO KRB5 directory: %PGADMIN_KRB5_DIR%
ECHO PostgreSQL directory: %PGADMIN_POSTGRES_DIR%
ECHO.
ECHO VC++ redist directory: %PGADMIN_VCREDIST_DIR%
ECHO VC++ redist file: %PGADMIN_VCREDIST_FILE%
ECHO InnoTool directory: %PGADMIN_INNOTOOL_DIR%
ECHO signtool directory: %PGADMIN_SIGNTOOL_DIR%
IF "%PGADMIN_WINDOWS_CSC%" == "" (
ECHO Code signing certificate: [NONE - Signing disabled]
) ELSE (
ECHO Code signing certificate: %PGADMIN_WINDOWS_CSC%
)
ECHO.
ECHO App version: %APP_VERSION%
ECHO App version suffix: %APP_VERSION_SUFFIX%
ECHO App short name: %APP_SHORTNAME%
ECHO App name: %APP_NAME%
ECHO ****************************************************************
ECHO Checking the environment...
IF NOT EXIST "%PGADMIN_INNOTOOL_DIR%" (
ECHO !PGADMIN_INNOTOOL_DIR! does not exist
ECHO Please install InnoTool and set the PGADMIN_INNOTOOL_DIR environment variable.
EXIT /B 1
)
IF NOT EXIST "%PGADMIN_VCREDIST_DIR%" (
ECHO !PGADMIN_VCREDIST_DIR! does not exist
ECHO Please install Microsoft Visual studio and set the PGADMIN_VCREDIST_DIR environment variable.
EXIT /B 1
)
IF NOT EXIST "%PGADMIN_KRB5_DIR%" (
ECHO !PGADMIN_KRB5_DIR! does not exist.
ECHO Please install MIT Kerberos for Windows and set the PGADMIN_KRB5_DIR environment variable.
EXIT /B 1
)
IF NOT EXIST "%PGADMIN_POSTGRES_DIR%" (
ECHO !PGADMIN_POSTGRES_DIR! does not exist.
ECHO Please install PostgreSQL and set the PGADMIN_POSTGRES_DIR environment variable.
EXIT /B 1
)
SET "PATH=%PGADMIN_POSTGRES_DIR%\bin;%PATH%"
EXIT /B 0
:CREATE_VIRTUAL_ENV
ECHO Creating virtual environment...
IF NOT EXIST "%TMPDIR%" MKDIR "%TMPDIR%"
CD "%TMPDIR%"
REM Note that we must use virtualenv.exe here, as the venv module doesn't allow python.exe to relocate.
"%PGADMIN_PYTHON_DIR%\Scripts\virtualenv.exe" venv || EXIT /B 1
XCOPY /S /I /E /H /Y "%PGADMIN_PYTHON_DIR%\DLLs" "%TMPDIR%\venv\DLLs" > nul || EXIT /B 1
REM Copy the standard library, but NOT site-packages: the venv already has its
REM own seeded pip there, and overwriting only the files the system Python also
REM has leaves a mix of two pip versions behind.
ROBOCOPY /E /R:3 /W:5 /NFL /NDL /NP "%PGADMIN_PYTHON_DIR%\Lib" "%TMPDIR%\venv\Lib" /XD site-packages
CALL :CHECK_ROBOCOPY_ERROR || EXIT /B 1
ECHO Activating virtual environment - %TMPDIR%\venv...
CALL "%TMPDIR%\venv\Scripts\activate" || EXIT /B 1
ECHO Installing dependencies...
CALL python -m pip install --upgrade pip || EXIT /B 1
CALL pip install --only-binary=cryptography -r "%WD%\requirements.txt" || EXIT /B 1
CD %WD%
EXIT /B 0
:CREATE_PYTHON_ENV
ECHO Staging Python...
MKDIR "%BUILDROOT%\python\Lib" || EXIT /B 1
ECHO Downloading embedded Python %PYTHON_VERSION%...
REM What ships is the embeddable distribution from python.org, a different
REM build from the one :FETCH_PYTHON downloaded, and it has to be the same
REM CPython version: the extension modules in site-packages were compiled
REM against the build Python and are about to be run under this one. Both
REM therefore read pkg\python-version.txt, rather than either asking the
REM other what it is.
curl.exe --fail --location --silent --show-error --output "%TMPDIR%\python-embedded.zip" "https://www.python.org/ftp/python/%PYTHON_VERSION%/python-%PYTHON_VERSION%-embed-amd64.zip" || EXIT /B 1
tar -x -f "%TMPDIR%\python-embedded.zip" -C "%BUILDROOT%\python" || EXIT /B 1
ECHO Copying site-packages...
XCOPY /S /I /E /H /Y "%TMPDIR%\venv\Lib\site-packages" "%BUILDROOT%\python\Lib\site-packages" > nul || EXIT /B 1
REM NOTE: There is intentionally no space after "site" in the line below, to prevent Python barfing if there's one in the file
ECHO import site>> "%BUILDROOT%\python\python%PYTHON_MAJOR%%PYTHON_MINOR%._pth"
ECHO Staging Kerberos components...
COPY "%PGADMIN_KRB5_DIR%\bin\kinit.exe" "%BUILDROOT%\python" > nul || EXIT /B 1
COPY "%PGADMIN_KRB5_DIR%\bin\krb5_64.dll" "%BUILDROOT%\python" > nul || EXIT /B 1
COPY "%PGADMIN_KRB5_DIR%\bin\comerr64.dll" "%BUILDROOT%\python" > nul || EXIT /B 1
COPY "%PGADMIN_KRB5_DIR%\bin\k5sprt64.dll" "%BUILDROOT%\python" > nul || EXIT /B 1
COPY "%PGADMIN_KRB5_DIR%\bin\gssapi64.dll" "%BUILDROOT%\python" > nul || EXIT /B 1
ECHO Cleaning up unnecessary .pyc and .pyo files...
FOR /R "%BUILDROOT%\python" %%f in (*.pyc *.pyo) do DEL /q "%%f" 1> nul 2>&1
ECHO Removing tests...
FOR /R "%BUILDROOT%\python\Lib" %%f in (test tests) do RD /Q /S "%%f" 1> nul 2>&1
EXIT /B 0
:CREATE_RUNTIME_ENV
IF NOT EXIST "%BUILDROOT%" MKDIR "%BUILDROOT%"
MKDIR "%BUILDROOT%\runtime"
ECHO Removing webpack caches...
RD /Q /S "%WD%\web\pgadmin\static\js\generated\.cache" 1> nul 2>&1
ECHO Copying web directory...
ROBOCOPY /S /NFL /NDL /NP "%WD%\web" "%BUILDROOT%\web"
CALL :CHECK_ROBOCOPY_ERROR || EXIT /B 1
ECHO Installing javascript dependencies...
CD "%BUILDROOT%\web"
SET "YARN_VERSION="
FOR /f "delims=" %%v IN ('node -p "require('./package.json').packageManager.split('@')[1]"') DO SET "YARN_VERSION=%%v"
IF "%YARN_VERSION%"=="" (
ECHO ERROR: Could not determine Yarn version from package.json packageManager field.
EXIT /B 1
)
CALL yarn set version %YARN_VERSION% || EXIT /B 1
CALL yarn install || EXIT /B 1
CALL npm rebuild || EXIT /B 1
ECHO Bundling javascript...
CALL yarn run bundle || EXIT /B 1
ECHO Cleaning up unnecessary .pyc and .pyo files...
FOR /R "%BUILDROOT%\web" %%f in (*.pyc *.pyo) do DEL /q "%%f" 1> nul 2>&1
ECHO Removing tests, Python caches and node modules...
FOR /R "%BUILDROOT%\web" %%f in (tests feature_tests __pycache__ node_modules) do RD /Q /S "%%f" 1> nul 2>&1
ECHO Removing the test framework...
RD /Q /S "%BUILDROOT%\web\regression" 1> nul 2>&1
ECHO Removing tools...
RD /Q /S "%BUILDROOT%\web\tools" 1> nul 2>&1
ECHO Removing the JavaScript build configuration...
RD /Q /S "%BUILDROOT%\web\.yarn" 1> nul 2>&1
DEL /q "%BUILDROOT%\web\yarn.lock" 1> nul 2>&1
DEL /q "%BUILDROOT%\web\.yarnrc.yml" 1> nul 2>&1
DEL /q "%BUILDROOT%\web\package.json" 1> nul 2>&1
DEL /q "%BUILDROOT%\web\jest.config.js" 1> nul 2>&1
DEL /q "%BUILDROOT%\web\babel.cfg" 1> nul 2>&1
DEL /q "%BUILDROOT%\web\babel.config.json" 1> nul 2>&1
DEL /q "%BUILDROOT%\web\webpack.config.js" 1> nul 2>&1
DEL /q "%BUILDROOT%\web\webpack.shim.js" 1> nul 2>&1
DEL /q "%BUILDROOT%\web\.eslintrc.js" 1> nul 2>&1
DEL /q "%BUILDROOT%\web\.editorconfig" 1> nul 2>&1
ECHO Removing any existing configurations...
DEL /q "%BUILDROOT%\web\pgadmin4.db" 1> nul 2>&1
DEL /q "%BUILDROOT%\web\config_local.py" 1> nul 2>&1
ECHO Staging license files...
COPY "%WD%\LICENSE" "%BUILDROOT%\" > nul || EXIT /B 1
ECHO Creating config_distro.py
ECHO SERVER_MODE = False > "%BUILDROOT%\web\config_distro.py"
ECHO HELP_PATH = '../../../docs/en_US/html/' >> "%BUILDROOT%\web\config_distro.py"
ECHO DEFAULT_BINARY_PATHS = { >> "%BUILDROOT%\web\config_distro.py"
ECHO 'pg': '$DIR/../runtime', >> "%BUILDROOT%\web\config_distro.py"
ECHO 'ppas': '' >> "%BUILDROOT%\web\config_distro.py"
ECHO } >> "%BUILDROOT%\web\config_distro.py"
ECHO Building docs...
CALL pip install sphinx || EXIT /B 1
CALL pip install sphinxcontrib-youtube || EXIT /B 1
MKDIR "%BUILDROOT%\docs\en_US\html"
CD "%WD%\docs\en_US"
CALL "%TMPDIR%\venv\Scripts\python.exe" build_code_snippet.py || EXIT /B 1
CALL "%TMPDIR%\venv\Scripts\sphinx-build.exe" "%WD%\docs\en_US" "%BUILDROOT%\docs\en_US\html" || EXIT /B 1
REM Remove unnecessary doc files
DEL /q "%BUILDROOT%\docs\en_US\html\_static\*.png" 1> nul 2>&1
RD /Q /S "%BUILDROOT%\docs\en_US\html\_sources" 1> nul 2>&1
ECHO Staging runtime components...
MKDIR "%BUILDROOT%\runtime\resources\app"
XCOPY /S /I /E /H /Y "%WD%\runtime\assets" "%BUILDROOT%\runtime\resources\app\assets" > nul || EXIT /B 1
XCOPY /S /I /E /H /Y "%WD%\runtime\src" "%BUILDROOT%\runtime\resources\app\src" > nul || EXIT /B 1
COPY "%WD%\runtime\package.json" "%BUILDROOT%\runtime\resources\app\" > nul || EXIT /B 1
COPY "%WD%\runtime\.yarnrc.yml" "%BUILDROOT%\runtime\resources\app\" > nul || EXIT /B 1
CD "%BUILDROOT%\runtime\resources\app\"
SET "YARN_VERSION="
FOR /f "delims=" %%v IN ('node -p "require('./package.json').packageManager.split('@')[1]"') DO SET "YARN_VERSION=%%v"
IF "%YARN_VERSION%"=="" (
ECHO ERROR: Could not determine Yarn version from package.json packageManager field.
EXIT /B 1
)
CALL yarn set version %YARN_VERSION% || EXIT /B 1
CALL yarn workspaces focus --production || EXIT /B 1
ECHO Removing yarn cache...
RD /Q /S "%BUILDROOT%\runtime\resources\app\.yarn" 1> nul 2>&1
ECHO Downloading Electron to %TMPDIR%...
REM Get a fresh copy of electron.
REM WGET
FOR /f "tokens=*" %%i IN ('npm info electron version') DO SET "ELECTRON_VERSION=%%i"
:GET_NW
wget https://github.com/electron/electron/releases/download/v%ELECTRON_VERSION%/electron-v%ELECTRON_VERSION%-win32-x64.zip -O "%TMPDIR%\electron-v%ELECTRON_VERSION%-win32-x64.zip"
IF %ERRORLEVEL% NEQ 0 GOTO GET_NW
MKDIR "%TMPDIR%\electron-v%ELECTRON_VERSION%-win32-x64" || EXIT /B 1
tar -C "%TMPDIR%\electron-v%ELECTRON_VERSION%-win32-x64" -xvf "%TMPDIR%\electron-v%ELECTRON_VERSION%-win32-x64.zip" || EXIT /B 1
REM WGET END
REM XCOPY
XCOPY /S /I /E /H /Y "%TMPDIR%\electron-v%ELECTRON_VERSION%-win32-x64\*" "%BUILDROOT%\runtime" > nul || EXIT /B 1
REM XCOPY END
MOVE "%BUILDROOT%\runtime\electron.exe" "%BUILDROOT%\runtime\pgAdmin4.exe"
ECHO Downloading rcedit.exe...
wget https://github.com/electron/rcedit/releases/download/v2.0.0/rcedit-x64.exe -O "%TMPDIR%\rcedit-x64.exe"
ECHO Replacing executable icon, description, version...
%TMPDIR%\rcedit-x64.exe "%BUILDROOT%\runtime\pgAdmin4.exe" --set-icon "%WD%\pkg\win32\Resources\pgAdmin4.ico"
%TMPDIR%\rcedit-x64.exe "%BUILDROOT%\runtime\pgAdmin4.exe" --set-version-string "FileDescription" "%APP_NAME%"
%TMPDIR%\rcedit-x64.exe "%BUILDROOT%\runtime\pgAdmin4.exe" --set-version-string "ProductName" "%APP_NAME%"
%TMPDIR%\rcedit-x64.exe "%BUILDROOT%\runtime\pgAdmin4.exe" --set-product-version "%APP_VERSION%"
IF NOT "%PGADMIN_WINDOWS_CSC%" == "" (
ECHO Attempting to sign the pgAdmin4.exe...
CALL "%PGADMIN_SIGNTOOL_DIR%\signtool.exe" sign /sm /n "%PGADMIN_WINDOWS_CSC%" /tr http://timestamp.digicert.com /td sha256 /fd sha1 /v "%BUILDROOT%\runtime\pgAdmin4.exe"
REM IF ERRORLEVEL, not IF %ERRORLEVEL%: inside a parenthesised block the
REM latter is expanded when the block is parsed, before signtool has run,
REM so it tested a stale value and this branch never fired. A failed
REM signature on the executable users actually run was undetectable, and
REM the installer around it was signed and verified regardless.
IF ERRORLEVEL 1 (
ECHO.
ECHO ************************************************************
ECHO * Failed to sign the pgAdmin4.exe
ECHO ************************************************************
EXIT /B 1
)
) ELSE (
ECHO Skipping code signing ^(PGADMIN_WINDOWS_CSC is not set^)...
)
ECHO Staging PostgreSQL components...
COPY "%PGADMIN_POSTGRES_DIR%\bin\libpq.dll" "%BUILDROOT%\runtime" > nul || EXIT /B 1
COPY "%PGADMIN_POSTGRES_DIR%\bin\libcrypto-*-x64.dll" "%BUILDROOT%\runtime" > nul || EXIT /B 1
COPY "%PGADMIN_POSTGRES_DIR%\bin\libssl-*-x64.dll" "%BUILDROOT%\runtime" > nul || EXIT /B 1
IF EXIST "%PGADMIN_POSTGRES_DIR%\bin\libintl-*.dll" COPY "%PGADMIN_POSTGRES_DIR%\bin\libintl-*.dll" "%BUILDROOT%\runtime" > nul
IF EXIST "%PGADMIN_POSTGRES_DIR%\bin\libiconv-*.dll" COPY "%PGADMIN_POSTGRES_DIR%\bin\libiconv-*.dll" "%BUILDROOT%\runtime" > nul
IF EXIST "%PGADMIN_POSTGRES_DIR%\bin\liblz4.dll" COPY "%PGADMIN_POSTGRES_DIR%\bin\liblz4.dll" "%BUILDROOT%\runtime" > nul
IF EXIST "%PGADMIN_POSTGRES_DIR%\bin\libzstd.dll" COPY "%PGADMIN_POSTGRES_DIR%\bin\libzstd.dll" "%BUILDROOT%\runtime" > nul
COPY "%PGADMIN_POSTGRES_DIR%\bin\zlib1.dll" "%BUILDROOT%\runtime" > nul || EXIT /B 1
COPY "%PGADMIN_POSTGRES_DIR%\bin\pg_dump.exe" "%BUILDROOT%\runtime" > nul || EXIT /B 1
COPY "%PGADMIN_POSTGRES_DIR%\bin\pg_dumpall.exe" "%BUILDROOT%\runtime" > nul || EXIT /B 1L%
COPY "%PGADMIN_POSTGRES_DIR%\bin\pg_restore.exe" "%BUILDROOT%\runtime" > nul || EXIT /B 1
COPY "%PGADMIN_POSTGRES_DIR%\bin\psql.exe" "%BUILDROOT%\runtime" > nul || EXIT /B 1
ECHO Staging VC++ runtime...
MKDIR "%BUILDROOT%\installer" || EXIT /B 1
COPY "%PGADMIN_VCREDIST_DIR%\%PGADMIN_VCREDIST_FILE%" "%BUILDROOT%\installer" > nul || EXIT /B 1
CD %WD%
EXIT /B 0
:CREATE_INSTALLER
ECHO Preparing for creation of windows installer...
IF NOT EXIST "%DISTROOT%" MKDIR "%DISTROOT%"
ECHO Copying icon file...
COPY "%WD%\pkg\win32\Resources\pgAdmin4.ico" "%BUILDROOT%" > nul || EXIT /B 1
CD "%WD%\pkg\win32"
ECHO Processing installer configuration script...
CALL "%PGADMIN_PYTHON_DIR%\python" "%WD%\pkg\win32\replace.py" "-i" "%WD%\pkg\win32\installer.iss.in" "-o" "%WD%\pkg\win32\installer.iss.in_stage1" "-s" MYAPP_FULLVERSION -r """%APP_VERSION%"""
CALL "%PGADMIN_PYTHON_DIR%\python" "%WD%\pkg\win32\replace.py" "-i" "%WD%\pkg\win32\installer.iss.in_stage1" "-o" "%WD%\pkg\win32\installer.iss.in_stage2" "-s" MYAPP_VERSION -r """v%APP_MAJOR%"""
CALL "%PGADMIN_PYTHON_DIR%\python" "%WD%\pkg\win32\replace.py" "-i" "%WD%\pkg\win32\installer.iss.in_stage2" "-o" "%WD%\pkg\win32\installer.iss" "-s" MYAPP_VCDIST -r """%PGADMIN_VCREDIST_DIRNAME%\%PGADMIN_VCREDIST_FILE%"""
ECHO Cleaning up...
DEL /s "%WD%\pkg\win32\installer.iss.in_stage*" > nul
ECHO Creating windows installer using INNO tool...
IF NOT "%PGADMIN_WINDOWS_CSC%" == "" (
CALL "%PGADMIN_INNOTOOL_DIR%\ISCC.exe" "%WD%\pkg\win32\installer.iss" "/DSIGNED" "/SpgAdminSigntool=%PGADMIN_SIGNTOOL_DIR%\signtool.exe sign /sm /n $q%PGADMIN_WINDOWS_CSC%$q /tr http://timestamp.digicert.com /td sha256 /fd sha1 /v $f" || EXIT /B 1
) ELSE (
CALL "%PGADMIN_INNOTOOL_DIR%\ISCC.exe" "%WD%\pkg\win32\installer.iss" || EXIT /B 1
)
ECHO Renaming installer...
MOVE "%WD%\pkg\win32\Output\pgadmin4-setup.exe" "%DISTROOT%\%INSTALLERNAME%" > nul || EXIT /B 1
ECHO Location - %DISTROOT%\%INSTALLERNAME%
ECHO Installer generated successfully.
CD %WD%
EXIT /B 0
:GENERATE_SBOM
ECHO Generating SBOM...
CALL syft "%BUILDROOT%" -o cyclonedx-json > "%BUILDROOT%\sbom.json"
EXIT /B 0
:VERIFY_SIGNATURE
IF "%PGADMIN_WINDOWS_CSC%" == "" (
ECHO Skipping signature verification ^(PGADMIN_WINDOWS_CSC is not set^)...
EXIT /B 0
)
ECHO Verifying the installer signature...
CALL "%PGADMIN_SIGNTOOL_DIR%\signtool.exe" verify /pa /v "%DISTROOT%\%INSTALLERNAME%"
REM PAUSE was the old handler here, which on the signing host either returns
REM immediately, leaving the build to carry on with an unverified installer,
REM or blocks until the job times out. There is nobody at that machine to
REM press a key.
IF ERRORLEVEL 1 (
ECHO.
ECHO ************************************************************
ECHO * Failed to verify signature of the installer
ECHO ************************************************************
EXIT /B 1
)
EXIT /B 0
:USAGE
ECHO Invalid command line options.
ECHO Usage: "Make.bat [clean]"
ECHO.
EXIT /B 1
:CHECK_ROBOCOPY_ERROR
IF %ERRORLEVEL% GEQ 8 EXIT /B %ERRORLEVEL%
EXIT /B 0
+32 -8
View File
@@ -6,17 +6,41 @@ Either build the sources or get them from macports or similar:
1. Yarn & NodeJS
2. PostgreSQL 12 or above from http://www.postgresql.org/
2. PostgreSQL 12 or above from http://www.postgresql.org/, or the pre-built
dependencies from the
[pgbuild project](https://github.com/pgadmin-org/pgbuild), which is what the
release builds use. Five are needed, openssl, krb5, zstd, lz4 and
postgresql-18, each published as a release whose tag ends in *-latest* so
that the current build is always at a predictable URL:
https://github.com/pgadmin-org/pgbuild/releases/download/<pkg>-macos-<arch>-latest/<pkg>-macos-<arch>-latest.tar.gz
where <arch> is arm64 or x86_64. They unpack into subdirectories of a
common prefix, /opt/pgbuild in CI, which is then given to the build as
*PGADMIN_POSTGRES_DIR=/opt/pgbuild/postgresql*.
These matter beyond convenience: they are built against each other with
*--with-zstd* and *--with-lz4* and a consistent deployment target, which
Homebrew's PostgreSQL is not. The build checks in *.github/workflows* fetch
exactly these through the *install-pgbuild-deps* action, so a local build
and a CI build work from the same binaries.
3. Python 3.9+ (required for running the build; the version that gets bundled
is a separate thing, see below). The build environment should run this
version of python in response to the *python* command.
4. syft, to generate the software bill of materials, and wget, which fetches
Electron:
brew install syft wget
3. Python 3.6+ (required for building). The build environment should run this
version of python in response to the *python* command.
## Building
1. To bundle a different version of Python from the default of 3.14.7, set the
*PGADMIN_PYTHON_VERSION* environment variable, e.g:
export PGADMIN_PYTHON_VERSION=3.13.11
1. The version of Python that gets bundled is read from
*pkg/python-version.txt*, which every platform's packaging shares so that
one commit cannot ship different interpreters on different systems. To
bundle a different version, edit that file.
2. If a path different from the default of /usr/local/pgsql for the PostgreSQL
installation has been used, set the *PGADMIN_POSTGRES_DIR* environment variable
+167 -53
View File
@@ -145,13 +145,24 @@ _create_python_env() {
_build_docs() {
echo "Building the docs..."
# Create a temporary venv for the doc build, so we don't contaminate the one
# that we're going to ship.
"${BUNDLE_DIR}/Contents/Frameworks/Python.framework/Versions/Current/bin/python3" -m venv "${BUILD_ROOT}/venv"
# The doc build imports pgAdmin itself: docs/en_US/build_code_snippet.py
# pulls in config, pgadmin.utils, pgadmin.browser.utils and the driver
# abstractions, so the whole of requirements.txt has to be importable. It
# already is, in the framework we just populated, so the venv inherits
# those rather than installing all hundred-odd packages a second time.
# Sphinx still goes in the venv rather than the framework, which is the
# point of having one: it must not end up in the shipped bundle.
#
# --system-site-packages is safe here in a way it is not in the Linux
# build, which deliberately avoids it (see the long comment in
# pkg/linux/build-functions.sh). There the parent is a shared system Python
# that may carry other packages' stale .pth files; here it is a framework
# we built ourselves moments ago and which contains only our dependencies.
"${BUNDLE_DIR}/Contents/Frameworks/Python.framework/Versions/Current/bin/python3" \
-m venv --system-site-packages "${BUILD_ROOT}/venv"
# shellcheck disable=SC1091
source "${BUILD_ROOT}/venv/bin/activate"
pip3 install --upgrade pip
pip3 install --no-cache-dir -r "${SOURCE_DIR}/requirements.txt"
pip3 install sphinx==7.4.7
pip3 install sphinxcontrib-youtube
@@ -497,7 +508,11 @@ _verify_bundle_linkage() {
# Build-host prefixes that must never appear in a shipped bundle. SLAVE_HOME
# (the Jenkins workspace root, under which the self-built OpenSSL and
# PostgreSQL live) is only added when set, i.e. on the CI builders.
local PREFIXES='/usr/local|/opt/homebrew|/opt/local'
# /opt/pgbuild is where the PostgreSQL, OpenSSL, Kerberos, zstd and lz4
# builds from pgadmin-org/pgbuild are unpacked. Their install names are
# absolute, so an unrewritten reference to one is exactly the kind of
# build-host path this check exists to catch, no different from Homebrew's.
local PREFIXES='/usr/local|/opt/homebrew|/opt/local|/opt/pgbuild'
if [ -n "${SLAVE_HOME}" ]; then
PREFIXES="${PREFIXES}|${SLAVE_HOME}"
fi
@@ -535,15 +550,32 @@ _generate_sbom() {
syft "${BUNDLE_DIR}/Contents/" -o cyclonedx-json > "${BUNDLE_DIR}/Contents/sbom.json"
}
_codesign_binaries() {
_set_codesign_args() {
# Populates CODESIGN_ARGS with the arguments shared by every codesign call.
#
# Without a Developer ID we sign ad-hoc rather than not signing at all.
# That is emphatically not a substitute for a real signature: an ad-hoc
# signed bundle is still refused by Gatekeeper and cannot be notarised. It
# exists because _fixup_imports has just rewritten install names across
# several hundred binaries, which invalidates whatever signatures they
# arrived with, Electron's included. On Apple Silicon the kernel refuses to
# execute a binary whose signature is broken or absent, so without this an
# unsigned bundle cannot be launched at all on arm64, even once the
# quarantine attribute has been removed. On Intel it merely leaves every
# binary in the bundle carrying a signature that no longer matches its
# contents.
if [ "${CODESIGN}" -eq 0 ]; then
echo "Signing ad-hoc: the result will run locally once allowed, but"
echo "cannot be distributed. Provide pkg/mac/codesign.conf to sign"
echo "properly."
# --deep for the same reason the Developer ID path uses it: these are
# bundles with nested code, and without it codesign refuses to replace
# the signature on one whose nested code is unsigned, reporting "code
# object is not signed at all" against a subcomponent.
CODESIGN_ARGS=(--deep --force --sign -)
return
fi
echo "Purging build-machine pollution (pycache) before signing..."
find "${BUNDLE_DIR}" -name "__pycache__" -type d -exec rm -rf {} +
find "${BUNDLE_DIR}" -name "*.pyc" -delete
if [ -z "${DEVELOPER_ID}" ] ; then
echo "Developer ID Application not found in codesign.conf" >&2
exit 1
@@ -554,6 +586,20 @@ _codesign_binaries() {
TEAM_ID=$(echo "${DEVELOPER_ID}" | awk -F"[()]" '{print $2}')
sed -i '' "s/%TEAMID%/${TEAM_ID}/g" "${BUILD_ROOT}/entitlements.plist"
CODESIGN_ARGS=(--deep --force --verify --verbose --timestamp
--options runtime
--entitlements "${BUILD_ROOT}/entitlements.plist"
-i org.pgadmin.pgadmin4
--sign "${DEVELOPER_ID}")
}
_codesign_binaries() {
echo "Purging build-machine pollution (pycache) before signing..."
find "${BUNDLE_DIR}" -name "__pycache__" -type d -exec rm -rf {} +
find "${BUNDLE_DIR}" -name "*.pyc" -delete
_set_codesign_args
echo Signing "${BUNDLE_DIR}" binaries...
IFS=$'\n'
for i in $(find "${BUNDLE_DIR}" -type f -perm +111 -exec file "{}" \; | \
@@ -563,37 +609,21 @@ _codesign_binaries() {
awk -F":" '{print $1}' | \
uniq)
do
codesign --deep --force --verify --verbose --timestamp \
--options runtime \
--entitlements "${BUILD_ROOT}/entitlements.plist" \
-i org.pgadmin.pgadmin4 \
--sign "${DEVELOPER_ID}" \
"$i"
codesign "${CODESIGN_ARGS[@]}" "$i"
done
unset IFS
echo Signing "${BUNDLE_DIR}" libraries...
find "${BUNDLE_DIR}" -type f -name "*.dylib*" -exec \
codesign --deep --force --verify --verbose --timestamp \
--options runtime \
--entitlements "${BUILD_ROOT}/entitlements.plist" \
-i org.pgadmin.pgadmin4 \
--sign "${DEVELOPER_ID}" \
{} \;
while IFS= read -r lib; do
codesign "${CODESIGN_ARGS[@]}" "${lib}"
done < <(find "${BUNDLE_DIR}" -type f -name "*.dylib*")
}
_codesign_bundle() {
if [ "${CODESIGN}" -eq 0 ]; then
return
fi
# Sign the .app
# CODESIGN_ARGS is set by _codesign_binaries, which always runs first and
# signs ad-hoc when there is no Developer ID; see _set_codesign_args.
echo Signing "${BUNDLE_DIR}"...
codesign --deep --force --verify --verbose --timestamp \
--options runtime \
--entitlements "${BUILD_ROOT}/entitlements.plist" \
-i org.pgadmin.pgadmin4 \
--sign "${DEVELOPER_ID}" \
"${BUNDLE_DIR}"
codesign "${CODESIGN_ARGS[@]}" "${BUNDLE_DIR}"
echo "Verifying the signature from bundle dir..."
codesign --verify --deep --verbose=4 "${BUNDLE_DIR}"
@@ -614,32 +644,116 @@ _create_zip() {
echo "Successfully created ZIP file: ${ZIP_NAME}"
}
_attach_dmg_eula() {
# The licence shown when the image is opened is a resource attached to the
# image, not a file inside it. Lifted from create-dmg, which in turn took
# the udifrez approach from https://developer.apple.com/forums/thread/668084
# after the older flatten/rez/unflatten route stopped working. The template
# is vendored alongside this script rather than cloned, so the build does
# not depend on another repository for one XML file.
echo "Attaching the licence agreement..."
local template="${SCRIPT_DIR}/eula-resources-template.xml"
local plist="${TEMP_DIR}/eula-resources.xml"
local licence="${SCRIPT_DIR}/licence.rtf"
local format
test -d "${TEMP_DIR}" || mkdir -p "${TEMP_DIR}"
case "$(file -b "${licence}")" in
'Rich Text Format data'*) format='RTF ' ;;
*) format='TEXT' ;;
esac
# The payload goes into a plist <data> element, so it is base64 wrapped at
# 52 columns and indented to sit inside the element.
EULA_FORMAT="${format}" \
EULA_DATA="$(openssl base64 -in "${licence}" | tr -d '\n' \
| awk '{gsub(/.{52}/, "&\n")}1' | sed $'s/^\\(.*\\)$/\t\t\t\\1/')" \
python3 -c 'import os, sys
tpl = open(sys.argv[1]).read()
tpl = tpl.replace("${EULA_FORMAT}", os.environ["EULA_FORMAT"])
tpl = tpl.replace("${EULA_DATA}", os.environ["EULA_DATA"])
open(sys.argv[2], "w").write(tpl)' "${template}" "${plist}"
hdiutil udifrez -xml "${plist}" '' -quiet "${DMG_NAME}" || {
echo 'ERROR: Failed to attach the licence agreement.'
exit 1
}
}
_create_dmg() {
# move to the directory where we want to create the DMG
# Build the image straight from a staged directory, rather than mounting a
# read/write volume, populating it and unmounting again.
#
# The unmount was the problem. hdiutil waits on DiskArbitration with a
# fixed 120 second timeout that nothing exposes a way to raise, and the
# cost of unmounting scales with the bundle: measured on GitHub's hosted
# runners with an identical payload it takes about 20 seconds on arm64, 41
# on macos-26-intel and 90 on macos-15-intel, and with the real bundle it
# exceeds 120 on both Intel images every time. `hdiutil detach -force` is
# no help, contrary to how it is usually described: it waits on
# DiskArbitration too, times out identically, and the conversion then fails
# with "Resource temporarily unavailable" because the image is still
# attached. `hdiutil create -srcfolder` never mounts anything, so none of
# that can arise.
#
# This also replaces create-dmg, which was cloned unpinned from its default
# branch on every build and run against the signed bundle. Everything it
# did for us was staging, because we passed --skip-jenkins and so none of
# its Finder automation ran: the volume name, the Applications symlink, the
# background directory, the pre-built .DS_Store carrying the window and
# icon layout, and the hidden bundle extension. The one part that is not a
# file in the volume is the licence agreement, attached afterwards.
test -d "${DIST_ROOT}" || mkdir "${DIST_ROOT}"
echo "Checking out create-dmg..."
git clone https://github.com/create-dmg/create-dmg.git "${BUILD_ROOT}/create-dmg"
local stage="${BUILD_ROOT}/dmg-staging"
"${BUILD_ROOT}/create-dmg/create-dmg" \
--volname "${APP_NAME}" \
--volicon "${SCRIPT_DIR}/dmg-icon.icns" \
--eula "${SCRIPT_DIR}/licence.rtf" \
--background "${SCRIPT_DIR}/dmg-background.png" \
--app-drop-link 600 220 \
--icon "${APP_NAME}.app" 200 220 \
--window-pos 200 120 \
--window-size 800 400 \
--hide-extension "${APP_NAME}.app" \
--add-file .DS_Store "${SCRIPT_DIR}/dmg.DS_Store" 5 5 \
--format UDBZ \
--skip-jenkins \
--no-internet-enable \
"${DMG_NAME}" \
"${BUNDLE_DIR}"
echo "Staging the disk image contents..."
rm -rf "${stage}"
mkdir -p "${stage}/.background"
# -c clones rather than copies on APFS, which makes staging a bundle of
# this size effectively free; fall back for any filesystem that cannot.
cp -Rc "${BUNDLE_DIR}" "${stage}/" 2>/dev/null || cp -R "${BUNDLE_DIR}" "${stage}/"
cp "${SCRIPT_DIR}/dmg-background.png" "${stage}/.background/"
cp "${SCRIPT_DIR}/dmg.DS_Store" "${stage}/.DS_Store"
cp "${SCRIPT_DIR}/dmg-icon.icns" "${stage}/.VolumeIcon.icns"
# hdiutil preserves this as a symlink rather than following it, so the
# volume gets the usual drag-to-install target without carrying a copy of
# /Applications.
ln -s /Applications "${stage}/Applications"
# C marks the volume root as having a custom icon, so .VolumeIcon.icns is
# used; E hides the bundle's extension in Finder. These are what create-dmg
# set via --volicon and --hide-extension.
SetFile -a C "${stage}" || echo "WARNING: could not set the custom icon attribute"
SetFile -a E "${stage}/${APP_NAME}.app" || echo "WARNING: could not hide the bundle extension"
echo "Creating the disk image..."
rm -f "${DMG_NAME}"
hdiutil create \
-srcfolder "${stage}" \
-volname "${APP_NAME}" \
-fs HFS+ \
-format UDBZ \
-ov \
"${DMG_NAME}" || { echo 'ERROR: Failed to create the disk image.'; exit 1; }
_attach_dmg_eula
hdiutil verify "${DMG_NAME}" || { echo 'ERROR: The disk image did not verify.'; exit 1; }
rm -rf "${stage}"
}
_codesign_dmg() {
# Unlike the bundle, there is no point signing the image ad-hoc: the
# signature on a disk image is about distribution, and an ad-hoc one
# satisfies nothing that a missing one does not. What matters for running
# the build locally is that the app inside it is signed, which
# _codesign_binaries and _codesign_bundle have already seen to.
if [ "${CODESIGN}" -eq 0 ]; then
return
fi
+17 -3
View File
@@ -52,10 +52,24 @@ if [ "${PGADMIN_POSTGRES_DIR}" == "" ]; then
export PGADMIN_POSTGRES_DIR=/usr/local/pgsql
fi
if [ "${PGADMIN_PYTHON_VERSION}" == "" ]; then
echo "PGADMIN_PYTHON_VERSION not set. Setting it to the default: 3.14.7"
export PGADMIN_PYTHON_VERSION=3.14.7
# Which Python gets bundled is a property of this commit rather than of the
# machine or the environment the build happens to run in, so it is read from
# pkg/python-version.txt and cannot be overridden from outside. The Windows
# build reads the same file.
#
# Unlike Windows, this still builds the framework with relocatable-python,
# which downloads from python.org itself and so only needs the version.
# python-build-standalone's relocatable distributions are not framework-shaped,
# and shipping one instead would mean reworking both the codesigning and the
# @loader_path rewriting in _fixup_imports, which is a job of its own.
PYTHON_VERSION_FILE="${SOURCE_DIR}/pkg/python-version.txt"
PGADMIN_PYTHON_VERSION=$(sed -nE 's/^PYTHON_VERSION=([0-9.]+)[[:space:]]*$/\1/p' "${PYTHON_VERSION_FILE}")
if [ -z "${PGADMIN_PYTHON_VERSION}" ]; then
echo "ERROR: could not read PYTHON_VERSION from ${PYTHON_VERSION_FILE}" >&2
exit 1
fi
export PGADMIN_PYTHON_VERSION
echo "Bundling Python ${PGADMIN_PYTHON_VERSION}, per ${PYTHON_VERSION_FILE}."
# Initialize variables
CREATE_ZIP=0
+105
View File
@@ -0,0 +1,105 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>LPic</key>
<array>
<dict>
<key>Attributes</key>
<string>0x0000</string>
<key>Data</key>
<data>
AAAAAgAAAAAAAAAAAAQAAA==
</data>
<key>ID</key>
<string>5000</string>
<key>Name</key>
<string></string>
</dict>
</array>
<key>STR#</key>
<array>
<dict>
<key>Attributes</key>
<string>0x0000</string>
<key>Data</key>
<data>
AAYNRW5nbGlzaCB0ZXN0MQVBZ3JlZQhEaXNhZ3JlZQVQcmludAdT
YXZlLi4ueklmIHlvdSBhZ3JlZSB3aXRoIHRoZSB0ZXJtcyBvZiB0
aGlzIGxpY2Vuc2UsIGNsaWNrICJBZ3JlZSIgdG8gYWNjZXNzIHRo
ZSBzb2Z0d2FyZS4gSWYgeW91IGRvIG5vdCBhZ3JlZSwgY2xpY2sg
IkRpc2FncmVlIi4=
</data>
<key>ID</key>
<string>5000</string>
<key>Name</key>
<string>English buttons</string>
</dict>
<dict>
<key>Attributes</key>
<string>0x0000</string>
<key>Data</key>
<data>
AAYHRW5nbGlzaAVBZ3JlZQhEaXNhZ3JlZQVQcmludAdTYXZlLi4u
e0lmIHlvdSBhZ3JlZSB3aXRoIHRoZSB0ZXJtcyBvZiB0aGlzIGxp
Y2Vuc2UsIHByZXNzICJBZ3JlZSIgdG8gaW5zdGFsbCB0aGUgc29m
dHdhcmUuIElmIHlvdSBkbyBub3QgYWdyZWUsIGNsaWNrICJEaXNh
Z3JlZSIu
</data>
<key>ID</key>
<string>5002</string>
<key>Name</key>
<string>English</string>
</dict>
</array>
<key>${EULA_FORMAT}</key>
<array>
<dict>
<key>Attributes</key>
<string>0x0000</string>
<key>Data</key>
<data>
${EULA_DATA}
</data>
<key>ID</key>
<string>5000</string>
<key>Name</key>
<string>English</string>
</dict>
</array>
<key>TMPL</key>
<array>
<dict>
<key>Attributes</key>
<string>0x0000</string>
<key>Data</key>
<data>
E0RlZmF1bHQgTGFuZ3VhZ2UgSUREV1JEBUNvdW50T0NOVAQqKioq
TFNUQwtzeXMgbGFuZyBJRERXUkQebG9jYWwgcmVzIElEIChvZmZz
ZXQgZnJvbSA1MDAwRFdSRBAyLWJ5dGUgbGFuZ3VhZ2U/RFdSRAQq
KioqTFNURQ==
</data>
<key>ID</key>
<string>128</string>
<key>Name</key>
<string>LPic</string>
</dict>
</array>
<key>styl</key>
<array>
<dict>
<key>Attributes</key>
<string>0x0000</string>
<key>Data</key>
<data>
AAMAAAAAAAwACQAUAAAAAAAAAAAAAAAAACcADAAJABQBAAAAAAAA
AAAAAAAAKgAMAAkAFAAAAAAAAAAAAAA=
</data>
<key>ID</key>
<string>5000</string>
<key>Name</key>
<string>English</string>
</dict>
</array>
</dict>
</plist>
+41
View File
@@ -0,0 +1,41 @@
pgAdmin 4: APT repository
Supported platform matrix:
@SUPPORTED_TABLE@
@ARCHIVE_SECTION_START@
Platforms that are no longer supported. Packages for these remain available
from https://pgadmin-archive.postgresql.org/, from where they can be manually
downloaded; the archive site cannot be used as a repository.
@ARCHIVED_TABLE@
@ARCHIVE_SECTION_END@
To use this repository, run the following commands:
#
# Setup the repository
#
# Install the public key for the repository (if not done previously):
curl -fsS https://www.pgadmin.org/static/packages_pgadmin_org.pub | sudo gpg --dearmor -o /usr/share/keyrings/packages-pgadmin-org.gpg
# Create the repository configuration file:
sudo sh -c 'echo "deb [signed-by=/usr/share/keyrings/packages-pgadmin-org.gpg] @BASE_URL@/apt/$(lsb_release -cs) pgadmin4 main" > /etc/apt/sources.list.d/pgadmin4.list && apt update'
#
# Install pgAdmin
#
# Install for both desktop and web modes:
sudo apt install pgadmin4
# Install for desktop mode only:
sudo apt install pgadmin4-desktop
# Install for web mode only:
sudo apt install pgadmin4-web
# Configure the webserver, if you installed pgadmin4-web:
sudo /usr/pgadmin4/bin/setup-web.sh
+128
View File
@@ -0,0 +1,128 @@
# Publishing
Everything that puts a pgAdmin build in front of a user lives here: the scripts
that index and sign the APT and YUM repositories, the README that sits at the
top of each of them, and the wrapper that lets a GitHub Actions runner ask for
those things to happen without being given a shell on the servers.
Two servers are involved, and which one a build goes to depends on what kind
of build it is. The *staging* server holds release candidates at
`/var/www/html/builds/<date>` and serves them over HTTP for testing, until they
are promoted and copied across. The *download* server holds the published tree
at `/var/ftp/pgadmin4`, which the world mirrors, and it is also where nightly
snapshots go directly, under `snapshots/<date>`: snapshots are not staged and
promoted, they are simply published somewhere nobody mirrors.
That last point is why the snapshots tree is the confinement root for the
upload key on the download server. It is the one part of the download site
that `sync-ftp-to-s3.py` excludes, so nothing written through that key can
reach the archive bucket or the PostgreSQL mirrors.
Each server runs the same wrapper with a different role, and the role decides
which verbs exist.
## Files
| File | What it is |
| --- | --- |
| `rebuild-apt-repo.sh` | Index and sign one APT tree |
| `rebuild-yum-repo.sh` | Index and sign one YUM tree, and make the EL compatibility links |
| `install-repo-readme.py` | Write the README at the top of an APT or YUM tree |
| `README.apt.in`, `README.yum.in` | The prose those READMEs are built from |
| `aptftp.conf` | `apt-ftparchive` settings: Origin, Label, Suite, Description |
| `pga-publish` | The forced-command wrapper |
| `authorized_keys.procyon`, `authorized_keys.paxsor` | Annotated templates for the two servers |
| `selftest.sh` | Exercises the wrapper's parser without touching anything |
The three scripts work standalone. Rebuilding a repository by hand after
purging old releases is what `rebuild-apt-repo.sh bookworm` has always been
for, and that has not changed; the wrapper calls the same script rather than
carrying its own copy of the logic.
## Installing
On both servers, as root:
```sh
install -d -m 755 /usr/local/lib/pga-publish
install -m 755 rebuild-apt-repo.sh rebuild-yum-repo.sh install-repo-readme.py \
/usr/local/lib/pga-publish/
install -m 644 aptftp.conf README.apt.in README.yum.in /usr/local/lib/pga-publish/
install -m 755 pga-publish /usr/local/sbin/pga-publish
```
Then the role, which the wrapper refuses to start without, since defaulting it
would mean a misconfigured host quietly accepting the other one's verbs:
```sh
echo staging > /etc/pga-publish.role # on the staging server
echo download > /etc/pga-publish.role # on the download server
```
On the download server only, the CloudFront distribution that `sync-s3`
invalidates. It is an identifier rather than a credential, but it is site
configuration, so it lives here rather than in the source tree:
```sh
echo '<distribution id>' > /etc/pga-publish.cloudfront
```
Also on the download server, move `aptftp.conf` out of `~pgaupload`: it is an
input to what gets indexed and therefore to what gets signed, and nothing
needs the publishing account to be able to write it.
## Access
Three key pairs. `roadie-publish` and `roadie-upload` belong to the runner;
`paxsor-pull` belongs to the download server, which uses it to fetch staged
content from the staging server during publication. Its private half must be at
`~pgaupload/.ssh/id_ed25519_paxsor_pull` on the download server, which is
where `pga-publish` looks for it; the name in the key management system is
only a label, and the path is what matters.
`authorized_keys.procyon` and `authorized_keys.paxsor` are annotated templates:
substitute the addresses and the public keys, and read the annotations before
changing an option, since each is there for a reason that is easier to write
down than to rediscover.
The private keys live on the runner's filesystem, at `~/.ssh/roadie-publish`
and `~/.ssh/roadie-upload`, and the workflow references them by path rather
than carrying copies in GitHub's secret store: they are already on the machine,
and a second copy would mean a second place to rotate and a second place to
leak from. The runner also needs the two servers in its `~/.ssh/known_hosts`,
since host key checking is on and there is nobody to answer a prompt.
The trade-off is that the boundary moves. An environment secret is gated by a
deployment branch policy; a file on disk is gated by which workflows may run on
that machine. This repository is public, so the runner belongs in a runner
group restricted to these workflows, and nothing triggered by `pull_request`
should target it.
The shape of it is that no key gets a shell. The publishing key runs
`pga-publish`, which parses `SSH_ORIGINAL_COMMAND` itself and never passes it
to one. The upload key runs `rrsync`, confined to the staging root,
write-only, with deletion and symlink-following refused. `restrict` turns off
everything optional, PTY allocation included, and keeps doing so as OpenSSH
grows new options.
## Checking it
```sh
./selftest.sh # 56 cases, touches nothing
pga-publish --dry-run --role download rebuild-apt bookworm
```
`--dry-run` prints the argument vector each step would have run instead of
running it, and is only honoured when `SSH_ORIGINAL_COMMAND` is unset, so a
client cannot reach it.
## What signs what, and where
The GPG key that signs packages and repository metadata stays on the servers.
A workflow uploads unsigned artefacts and asks for `stage-sign`, so a
compromised run can ask for a signature over something it has just uploaded,
but never holds the key, signs out of band, or signs with a different one.
The exceptions are the macOS notarisation and the Windows Authenticode
signature, which happen on the runners because the Apple credentials and the
Certum token cannot follow the GPG key onto these machines.
+41
View File
@@ -0,0 +1,41 @@
pgAdmin 4: YUM repository
Supported platforms:
@SUPPORTED_TABLE@
@ARCHIVE_SECTION_START@
Platforms that are no longer supported. Packages for these remain available
from https://pgadmin-archive.postgresql.org/, from where they can be manually
downloaded; the archive site cannot be used as a repository.
@ARCHIVED_TABLE@
@ARCHIVE_SECTION_END@
To use this repository, first uninstall any pgAdmin repo packages that you may
already have installed, for example; 'sudo rpm -e pgadmin4-fedora-repo' or
'sudo rpm -e pgadmin4-redhat-repo'.
Then to setup the repository for this set of builds on Fedora run:
sudo rpm -i @BASE_URL@/yum/@FEDORA_REPO_RPM@
On AlmaLinux, RHEL, or Rocky, run:
sudo rpm -i @BASE_URL@/yum/@REDHAT_REPO_RPM@
To install pgAdmin, run one of the following commands:
# Install for both desktop and web modes.
sudo yum install pgadmin4
# Install for desktop mode only.
sudo yum install pgadmin4-desktop
# Install for web mode only.
sudo yum install pgadmin4-web
Finally, if you have installed pgadmin4 or pgadmin4-web, run the web setup
script to configure the system to run in web mode:
sudo /usr/pgadmin4/bin/setup-web.sh
+22
View File
@@ -0,0 +1,22 @@
// Configuration for apt-ftparchive, brought across from
// /home/pgaupload/aptftp.conf on the download server so that it is version
// controlled alongside the scripts that use it.
//
// Architectures is overridden on the apt-ftparchive command line by
// rebuild-apt-repo.sh, from the same list it indexes, so that the set of
// architectures indexed and the set advertised in the signed Release file
// cannot disagree. The value here is the default that override supplies.
APT::FTPArchive::Release {
Origin "pgAdmin Development Team";
Label "pgAdmin 4";
Suite "pgadmin4";
Architectures "amd64 all";
Components "main";
Description "pgAdmin 4 - Development Tools for PostgreSQL";
};
Default {
Packages::Compress ". gzip bzip2";
Sources::Compress ". gzip bzip2";
Contents::Compress ". gzip bzip2";
};
+52
View File
@@ -0,0 +1,52 @@
# ~pgaupload/.ssh/authorized_keys on paxsor (ftp.pgadmin.org)
#
# The same two keys as on procyon, with the same options and for the same
# reasons; read the annotations in authorized_keys.procyon first, since they
# are not repeated here. What differs is the blast radius, because this host
# serves the packages that end users install, and the yum and apt signing keys
# live here.
#
# Placeholders in angle brackets must be substituted at install time:
#
# <ROADIE-IPV4>, <ROADIE-IPV6> the runner's addresses
# <...KEY...> the public key, one line, no line breaks
#
# Note that these are the same two key pairs as on procyon, not four. One
# publishing key and one upload key, each trusted by both hosts, each confined
# differently on each host by the options below. Two more key pairs would mean
# two more things to rotate and no reduction in what a compromised runner can
# reach, since a compromised runner has both of them anyway.
# ---------------------------------------------------------------------------
# Key 1: publishing. Verbs only.
# ---------------------------------------------------------------------------
# What a holder of this key can do on paxsor: create a v<VERSION> or snapshot
# directory (refusing if it already exists), pull the matching staging content
# from procyon, rebuild and sign the apt and yum metadata for an allowlisted
# distribution, and run the four existing publication scripts with arguments
# that have been validated or allowlisted first.
#
# What it cannot do, and this is the point of the exercise: delete anything,
# name a file, name a path, name a host, name a signing key, choose which
# binary runs, or supply a single character that reaches a shell.
restrict,from="<ROADIE-IPV4>,<ROADIE-IPV6>",command="/usr/local/sbin/pga-publish" ssh-ed25519 <...PUBLISH-KEY...> pga-publish@roadie
# ---------------------------------------------------------------------------
# Key 2: upload. Bytes only, and only into the snapshots tree.
# ---------------------------------------------------------------------------
# Snapshots are built straight onto this server rather than staged on procyon
# and promoted, so unlike the release flow they do need an upload path here.
#
# The confinement root is the snapshots tree itself, which is the right place
# on two counts. It is where snapshots belong, and it is the one part of the
# download tree that sync-ftp-to-s3.py excludes, so nothing written through
# this key can reach the archive bucket or the PostgreSQL mirrors. A key
# confined anywhere else under /var/ftp would have both of those as a side
# effect of a single bad upload.
#
# Retention is not this key's business either: the wrapper's snapshot-purge
# verb enforces it, and -no-del means this key cannot remove a snapshot even
# if asked to.
#
# SITE: check `command -v rrsync`; the path moved between rsync packages.
restrict,from="<ROADIE-IPV4>,<ROADIE-IPV6>",command="/usr/bin/rrsync -wo -no-del -munge /var/ftp/pgadmin4/snapshots" ssh-ed25519 <...UPLOAD-KEY...> pga-upload@roadie
+89
View File
@@ -0,0 +1,89 @@
# ~pgaupload/.ssh/authorized_keys on procyon (developer.pgadmin.org)
#
# Two keys, two jobs, two blast radii. Neither of them can give you a shell.
#
# Placeholders in angle brackets must be substituted at install time. Do not
# commit the substituted file to a public repository: the addresses are not
# secret, but publishing the exact shape of the firewall is free reconnaissance
# for somebody who has not earned it.
#
# <ROADIE-IPV4>, <ROADIE-IPV6> the runner's addresses
# <PAXSOR-IPV4> the download server's address
# <...KEY...> the public key, one line, no line breaks
#
# Options used, and why:
#
# restrict Refuse everything optional, then add back nothing. This is
# the modern shorthand for no-agent-forwarding, no-pty,
# no-port-forwarding, no-X11-forwarding and no-user-rc, and
# crucially it is future-proof: when OpenSSH grows a new
# forwarding feature, `restrict` disables that too, whereas an
# explicit list of no-* options silently would not.
#
# no-pty matters more than it looks. Without it, a client can
# ask for a terminal, and a terminal is an interactive channel
# into whatever the forced command turns out to be.
#
# from= Defence in depth behind the firewall, not instead of it. The
# firewall can be misconfigured during maintenance; this line
# is in the same file as the key it constrains, so the two are
# reviewed together and rotated together.
#
# command= The forced command. Whatever the client asked to run is put
# in SSH_ORIGINAL_COMMAND and this runs instead. The wrapper
# parses that variable itself and never lets a shell near it.
#
# There is deliberately no environment= option on either key. The wrapper
# builds its children's environment from scratch, but not offering the client
# an input channel at all is better than filtering one.
# ---------------------------------------------------------------------------
# Key 1: publishing. Verbs only. No file content passes through this key.
# ---------------------------------------------------------------------------
# What a holder of this key can do on procyon: create a staging directory from
# a fixed template, list staging directories, and rebuild and sign the apt and
# yum metadata inside one. That is the entire capability set; it is enumerated
# in the VERBS table in pga-publish, on one screen, on purpose.
restrict,from="<ROADIE-IPV4>,<ROADIE-IPV6>",command="/usr/local/sbin/pga-publish" ssh-ed25519 <...PUBLISH-KEY...> pga-publish@roadie
# ---------------------------------------------------------------------------
# Key 2: upload. Bytes only. No verbs, no metadata, no signing.
# ---------------------------------------------------------------------------
# rrsync ships with rsync (on Debian and Ubuntu, /usr/share/doc/rsync/scripts,
# or /usr/bin/rrsync on newer packages: check before writing this line). It is
# itself a forced-command wrapper, written and reviewed by the rsync
# maintainers, whose whole job is to inspect the rsync argument list the client
# sent and refuse anything outside the subtree it was given.
#
# -wo write-only. The client may send files and may not fetch them.
# A compromised runner therefore cannot read back whatever else
# lives under the staging root, and cannot use this key to
# exfiltrate anything.
# -no-del no deletion. Uploads add and overwrite; they never remove. A
# wrong --delete in a workflow, or a deliberate one from an
# attacker, stops here rather than emptying developer.pgadmin.org.
# -munge munge incoming symlinks so that they cannot be followed. Without
# it, an upload can drop a symlink pointing at, say, the account's
# ~/.ssh, and the next thing that writes through that path writes
# somewhere it should not.
#
# The trailing path is the root the client is confined to. Every path the
# client names is resolved relative to it, and rrsync refuses anything that
# climbs out, so the client never sees an absolute path at all.
restrict,from="<ROADIE-IPV4>,<ROADIE-IPV6>",command="/usr/bin/rrsync -wo -no-del -munge /var/www/html/builds" ssh-ed25519 <...UPLOAD-KEY...> pga-upload@roadie
# ---------------------------------------------------------------------------
# Key 3: the download server's read-only pull.
# ---------------------------------------------------------------------------
# paxsor fetches staging content from procyon during publication, so it holds a
# key here. It is read-only and confined to the same staging root, which is a
# tree that developer.pgadmin.org already serves over plain HTTP; this edge
# therefore grants paxsor nothing it could not have fetched with curl, and it
# is worth saying so explicitly in review, because "the download server can SSH
# to the staging server" sounds worse than it is.
#
# If that reasoning is ever rejected, the alternative is for roadie to pull the
# content down and push it up again over its own two keys, at the cost of
# moving every byte twice. The wrapper would lose release-fetch,
# packages-fetch and snapshot-fetch, and gain nothing else.
restrict,from="<PAXSOR-IPV4>",command="/usr/bin/rrsync -ro /var/www/html/builds" ssh-ed25519 <...PULL-KEY...> pga-pull@paxsor
+284
View File
@@ -0,0 +1,284 @@
#!/usr/bin/python3
#
# Write the README that sits at the top of an APT or YUM repository tree.
#
# These were heredocs inline in five Jenkins jobs: pgadmin4-deb-readme-build
# and pgadmin4-deb-readme-snapshot for APT, and pgadmin4-repo-rpm-build,
# pgadmin4-repo-rpm-snapshot and pgadmin4-qa-repo-rpm-build for YUM. Only the
# base URL differed between them, so that is an argument and the prose is a
# template.
#
# The platform support table used to be maintained by hand in those heredocs,
# which is why the published one had drifted: it stopped at Noble whilst the
# job that wrote it listed Resolute. It is now derived.
#
# Where the facts come from:
#
# Which platforms are supported the repository directories that exist in
# the tree being written. That is what a user
# can actually install from today.
#
# Which releases each carried the S3 archive, not the download server.
# Old releases are purged from the server as
# they age, so deriving "first supported
# release" from it would rewrite history
# every time somebody tidied up: bookworm
# reads as 7.8 on the server against 7.0 in
# the archive. The sync has no --delete, so
# the bucket keeps everything.
#
# For a snapshot or pre-release tree there is no history to show, every package
# being from the one build, so --no-archive emits a plain list of what is
# present and skips the archive entirely.
import argparse
import json
import os
import re
import subprocess
import sys
BUCKET = "pgadmin-archive.postgresql.org"
# The packages that carry the pgAdmin version. Deliberately a list rather than
# a prefix match: the repository RPMs have their own versioning (2-1), and
# pgadmin4-python3-mod_wsgi carries mod_wsgi's, so both would otherwise turn up
# as implausible pgAdmin releases.
VERSIONED_PACKAGES = ("pgadmin4", "pgadmin4-desktop", "pgadmin4-server",
"pgadmin4-web")
# aws s3 sync follows symlinks, so the EL compatibility links that
# rebuild-yum-repo.sh creates exist in the bucket as complete copies of the
# tree they point at. Counting them would invent platforms.
EL_COMPAT = re.compile(r"\d(?:Server|Workstation|Client)-")
DEB = re.compile(r"\A(?P<package>[a-z0-9.+-]+)_(?P<version>[^_]+)_"
r"(?P<arch>[a-z0-9]+)\.deb\Z")
RPM = re.compile(r"\A(?P<package>.+?)-(?P<version>[0-9][^-]*)-[^-]+\.rpm\Z")
def platform_key(platform):
"""Sort rhel-8 before rhel-10, and codenames alphabetically."""
match = re.match(r"\A(.*?)-(\d+)\Z", platform)
if match:
return (match.group(1), int(match.group(2)))
return (platform, 0)
def upstream_version(version):
"""Strip the packaging revision, leaving the pgAdmin release.
Debian versions have carried one since 9.10 or so, as in 9.16-1.resolute,
and RPM releases are separated the same way. What the table wants is the
pgAdmin version, and mixing the two forms also sorts badly: 9.16-1.resolute
compares below 9.3 unless the suffix goes first.
"""
return version.split("-", 1)[0]
def version_key(version):
"""Sort 9.17 above 9.5, and anything unparseable below everything."""
parts = []
for piece in upstream_version(version).split("."):
parts.append(int(piece) if piece.isdigit() else -1)
return parts
def archive_keys(object_list=None):
"""Every object key in the archive, from S3 or from a captured listing."""
if object_list:
with open(object_list) as handle:
return [line.strip() for line in handle if line.strip()]
out = subprocess.run(
["/usr/bin/aws", "s3api", "list-objects-v2",
"--bucket", BUCKET, "--prefix", "pgadmin4/",
"--query", "Contents[].Key"],
stdout=subprocess.PIPE, check=True).stdout
return json.loads(out) or []
def parse_apt(key):
"""pgadmin4/apt/<codename>/dists/pgadmin4/main/binary-<arch>/<file>"""
parts = key.split("/")
if len(parts) != 8 or parts[1] != "apt" or not parts[6].startswith("binary-"):
return None
match = DEB.match(parts[7])
if not match or match.group("package") not in VERSIONED_PACKAGES:
return None
return parts[2], parts[6][len("binary-"):], match.group("version")
def parse_yum(key):
"""pgadmin4/yum/<family>/<name>-<version>-<arch>/<file>"""
parts = key.split("/")
if len(parts) != 5 or parts[1] != "yum" or EL_COMPAT.search(parts[3]):
return None
tree = parts[3]
if "-" not in tree:
return None
platform, _, arch = tree.rpartition("-")
match = RPM.match(parts[4])
if not match or match.group("package") not in VERSIONED_PACKAGES:
return None
return platform, arch, match.group("version")
def collect(keys, kind):
"""{platform: {"arches": set, "versions": set}} from archive keys."""
parse = parse_apt if kind == "apt" else parse_yum
found = {}
for key in keys:
parsed = parse(key)
if not parsed:
continue
platform, arch, version = parsed
entry = found.setdefault(platform, {"arches": set(), "versions": set()})
entry["arches"].add(arch)
entry["versions"].add(upstream_version(version))
return found
def scan_tree(root, kind):
"""{platform: set(arches)} for the repositories present in a tree."""
base = os.path.join(root, kind)
if not os.path.isdir(base):
sys.exit("No such repository tree: %s" % base)
found = {}
if kind == "apt":
for codename in sorted(os.listdir(base)):
main = os.path.join(base, codename, "dists", "pgadmin4", "main")
if not os.path.isdir(main):
continue
for entry in sorted(os.listdir(main)):
if not entry.startswith("binary-"):
continue
directory = os.path.join(main, entry)
if any(DEB.match(f) and DEB.match(f).group("package")
in VERSIONED_PACKAGES for f in os.listdir(directory)):
found.setdefault(codename, set()).add(
entry[len("binary-"):])
else:
for family in sorted(os.listdir(base)):
directory = os.path.join(base, family)
if not os.path.isdir(directory):
continue
for tree in sorted(os.listdir(directory)):
if EL_COMPAT.search(tree) or "-" not in tree:
continue
full = os.path.join(directory, tree)
if not os.path.isdir(full) or os.path.islink(full):
continue
platform, _, arch = tree.rpartition("-")
if any(RPM.match(f) and RPM.match(f).group("package")
in VERSIONED_PACKAGES for f in os.listdir(full)):
found.setdefault(platform, set()).add(arch)
return found
def render_table(headings, rows):
"""An ASCII table in the style the hand-written READMEs used."""
if not rows:
return "(none)"
widths = [max(len(str(r[i])) for r in [headings] + rows)
for i in range(len(headings))]
rule = "+" + "+".join("-" * (w + 2) for w in widths) + "+"
def line(cells, centre):
out = []
for cell, width in zip(cells, widths):
text = str(cell)
out.append(" %s " % (text.center(width) if centre
else text.ljust(width)))
return "|" + "|".join(out) + "|"
return "\n".join([rule, line(headings, False), rule]
+ [line(r, True) for r in rows] + [rule])
def main():
parser = argparse.ArgumentParser(
description="Write a pgAdmin repository README.")
parser.add_argument("kind", choices=("apt", "yum"))
parser.add_argument("-r", "--root", default="/var/ftp/pgadmin4",
help="the tree holding apt/ and yum/")
parser.add_argument("-u", "--base-url",
default="https://ftp.postgresql.org/pub/pgadmin/pgadmin4",
help="what the instructions tell users to fetch from")
parser.add_argument("--no-archive", action="store_true",
help="list only what this tree holds, with no release "
"history; for snapshot and pre-release trees")
parser.add_argument("--object-list",
help="read archive keys from a file rather than S3")
args = parser.parse_args()
here = os.path.dirname(os.path.abspath(__file__))
template_path = os.path.join(here, "README.%s.in" % args.kind)
if not os.path.isfile(template_path):
sys.exit("Missing %s" % template_path)
with open(template_path) as handle:
text = handle.read()
present = scan_tree(args.root, args.kind)
if args.no_archive:
supported = render_table(
["Platform", "Architecture"],
[[p, " ".join(sorted(present[p]))] for p in sorted(present, key=platform_key)])
else:
history = collect(archive_keys(args.object_list), args.kind)
for platform in present:
if platform not in history:
sys.exit("%s is published but absent from the archive; the "
"archive sync has probably not run since it was "
"added" % platform)
supported_rows, archived_rows = [], []
for platform in sorted(history, key=platform_key):
versions = sorted(history[platform]["versions"], key=version_key)
arches = " ".join(sorted(history[platform]["arches"]))
if platform in present:
supported_rows.append([platform, arches, versions[0]])
else:
archived_rows.append([platform, arches,
versions[0], versions[-1]])
headings = ["Platform", "Architecture", "First supported release",
"Last supported release"]
supported = render_table(headings[:3], supported_rows)
archived = render_table(headings, archived_rows)
# A snapshot or pre-release tree has no history to show, so the whole
# section goes rather than being left saying "(none)".
if args.no_archive:
text = re.sub(r"@ARCHIVE_SECTION_START@\n.*?@ARCHIVE_SECTION_END@\n\n?",
"", text, flags=re.S)
else:
text = text.replace("@ARCHIVE_SECTION_START@\n", "")
text = text.replace("@ARCHIVE_SECTION_END@\n", "")
text = text.replace("@ARCHIVED_TABLE@", archived)
text = text.replace("@SUPPORTED_TABLE@", supported)
text = text.replace("@BASE_URL@", args.base_url)
if args.kind == "yum":
tree = os.path.join(args.root, "yum")
for name, placeholder in (("fedora", "@FEDORA_REPO_RPM@"),
("redhat", "@REDHAT_REPO_RPM@")):
matches = sorted(f for f in os.listdir(tree)
if f.startswith("pgadmin4-%s-repo-" % name)
and f.endswith(".noarch.rpm"))
if not matches:
sys.exit("Could not find the %s repository RPM in %s"
% (name, tree))
text = text.replace(placeholder, matches[-1])
destination = os.path.join(args.root, args.kind, "README")
with open(destination, "w") as handle:
handle.write(text)
print("Wrote %s" % destination)
if __name__ == "__main__":
main()
+1167
View File
File diff suppressed because it is too large. Load diff
+82
View File
@@ -0,0 +1,82 @@
#!/bin/sh
#
# Index and sign one APT repository tree.
#
# This is the only implementation of that sequence. It previously existed five
# times: inline in the pgadmin4-deb-build, pgadmin4-deb-snapshot and
# pgadmin4-deb-publish-qa-to-prod Jenkins jobs, which differed from each other
# only in which directory they ran against, and again in the copy of this
# script that lived in the pgaweb repository for manual use after old releases
# were purged by hand.
#
# Run by hand with just a codename it does exactly what that copy did, against
# the production tree. The publishing automation passes -r to point it at a
# staging or snapshot tree instead.
set -e
ROOT=/var/ftp/pgadmin4
ARCHES="amd64 all"
DEFAULT_ROOT=${ROOT}
DEFAULT_ARCHES=${ARCHES}
usage() {
cat >&2 <<USAGE
Usage: $0 [-r ROOT] [-a "ARCH ..."] <distro codename>
-r ROOT the tree holding apt/<codename> (default: ${DEFAULT_ROOT})
-a ARCHES space separated architectures to index (default: ${DEFAULT_ARCHES})
Examples:
$0 bookworm
$0 -r /var/ftp/pgadmin4/snapshots/2026-09-17 bookworm
$0 -a "all amd64 arm64" bookworm
USAGE
exit 1
}
while getopts 'r:a:' OPT; do
case "${OPT}" in
r) ROOT=${OPTARG} ;;
a) ARCHES=${OPTARG} ;;
*) usage ;;
esac
done
shift $((OPTIND - 1))
[ $# -eq 1 ] || usage
CODENAME=$1
# apt-ftparchive reads its Origin, Label, Suite and Description from this file.
# The architecture list is deliberately not read from it: it is overridden
# below from ARCHES, so that the set of architectures indexed and the set
# advertised in the signed Release file cannot drift apart. A package whose
# architecture is missing from Release is ignored by every client, silently.
CONF=$(dirname "$0")/aptftp.conf
[ -f "${CONF}" ] || { echo "Missing ${CONF}" >&2; exit 1; }
TREE=${ROOT}/apt/${CODENAME}
DISTS=${TREE}/dists/pgadmin4
for ARCH in ${ARCHES}; do
mkdir -p "${DISTS}/main/binary-${ARCH}"
done
cd "${TREE}"
for ARCH in ${ARCHES}; do
apt-ftparchive packages -c="${CONF}" "dists/pgadmin4/main/binary-${ARCH}" \
> "dists/pgadmin4/main/binary-${ARCH}/Packages"
gzip -f -k "dists/pgadmin4/main/binary-${ARCH}/Packages"
done
cd "${DISTS}"
apt-ftparchive -c="${CONF}" \
-o "APT::FTPArchive::Release::Architectures=${ARCHES}" \
release . > Release
gzip -f -k Release
rm -f Release.gpg
gpg -u packages@pgadmin.org -bao Release.gpg Release
rm -f InRelease
gpg -u packages@pgadmin.org --clear-sign --output InRelease Release
+71
View File
@@ -0,0 +1,71 @@
#!/bin/sh
#
# Index and sign one YUM repository tree, and create the EL compatibility
# symlinks that go with it.
#
# As with the APT script, this is now the only implementation. The same
# sequence was inline in pgadmin4-rpm-build, pgadmin4-rpm-snapshot,
# pgadmin4-rpm-publish-qa-to-prod and pgadmin4-repo-rpm-copy-to-repos, and in
# the pgaweb copy of this script.
#
# createrepo_c runs a full rebuild rather than --update, which is what both the
# manual and the automated paths have always done, and is the safe choice after
# packages have been removed by hand.
set -e
ROOT=/var/ftp/pgadmin4
ARCH=x86_64
DEFAULT_ROOT=${ROOT}
DEFAULT_ARCH=${ARCH}
usage() {
cat >&2 <<USAGE
Usage: $0 [-r ROOT] [-a ARCH] <distro family> <distro name> <distro version>
-r ROOT the tree holding yum/<family> (default: ${DEFAULT_ROOT})
-a ARCH the package architecture (default: ${DEFAULT_ARCH})
family fedora or redhat
name fedora or rhel
version the major version, e.g. 10
Examples:
$0 redhat rhel 10
$0 -a aarch64 redhat rhel 10
$0 -r /var/ftp/pgadmin4/snapshots/2026-09-17 fedora fedora 44
USAGE
exit 1
}
while getopts 'r:a:' OPT; do
case "${OPT}" in
r) ROOT=${OPTARG} ;;
a) ARCH=${OPTARG} ;;
*) usage ;;
esac
done
shift $((OPTIND - 1))
[ $# -eq 3 ] || usage
FAMILY=$1
NAME=$2
VERSION=$3
TREE=${ROOT}/yum/${FAMILY}/${NAME}-${VERSION}-${ARCH}
/usr/bin/createrepo_c "${TREE}"
# -u, because without it gpg signs with whatever the keyring's default
# key resolves to. That is the packaging key today only because there is
# one secret key in it, and it stops being true the moment a second one
# lands. A repository whose metadata is signed by the wrong key is one
# every client with gpgcheck=1 rejects, produced by a run that exits 0.
/usr/bin/gpg --yes -u packages@pgadmin.org --detach-sign --armor \
"${TREE}/repodata/repomd.xml"
# Some EL variants report $releasever with a variant suffix, so the repository
# has to answer to each of the resulting directory names.
cd "${ROOT}/yum/${FAMILY}"
for VARIANT in Server Workstation Client; do
ln -snf "${NAME}-${VERSION}-${ARCH}" "${NAME}-${VERSION}${VARIANT}-${ARCH}"
done
+179
View File
@@ -0,0 +1,179 @@
#!/bin/bash
#
# selftest.sh: exercise the pga-publish request parser without touching
# anything. Every case runs the wrapper in --dry-run mode, which prints the
# argv it would have executed rather than executing it, so this is safe to run
# on a workstation, on a staging host, or on the download server itself.
#
# The interesting half of the table is the rejections. A parser that accepts
# the right things is table stakes; a parser that refuses a semicolon, a
# backtick, a newline and a `..` is the reason this wrapper exists, and those
# cases should be extended every time somebody thinks of a new way in.
#
# Usage: ./selftest.sh [path-to-pga-publish]
set -u
WRAPPER="${1:-$(dirname "$0")/pga-publish}"
pass=0
fail=0
# check <expected-exit> <role> <description> <request...>
check() {
local expected="$1"; shift
local role="$1"; shift
local description="$1"; shift
local output
output="$("$WRAPPER" --dry-run --role "$role" "$@" 2>&1)"
local rc=$?
if [ "$rc" = "$expected" ]; then
pass=$((pass + 1))
printf 'ok %-3s %s\n' "$rc" "$description"
else
fail=$((fail + 1))
printf 'FAIL %-3s (wanted %s) %s\n' "$rc" "$expected" "$description"
printf ' %s\n' "$output"
fi
}
# check_ssh <expected-exit> <description> <raw SSH_ORIGINAL_COMMAND>
#
# The other helper drives --dry-run, which takes its request from argv and so
# joins it with single spaces. Production takes the raw string sshd puts in
# SSH_ORIGINAL_COMMAND, which keeps runs of whitespace, tabs, and whatever else
# the client sent. Only rejections are tested this way: an accepted request
# would run for real, and the point of these cases is that none of them gets
# that far.
check_ssh() {
local expected="$1"; shift
local description="$1"; shift
local output
output="$(SSH_ORIGINAL_COMMAND="$1" "$WRAPPER" 2>&1)"
local rc=$?
if [ "$rc" = "$expected" ]; then
pass=$((pass + 1))
printf 'ok %-3s %s\n' "$rc" "$description"
else
fail=$((fail + 1))
printf 'FAIL %-3s (wanted %s) %s\n' "$rc" "$expected" "$description"
printf ' %s\n' "$output"
fi
}
echo "== accepted requests: staging role (procyon) =="
check 0 staging "hello" hello
check 0 staging "stage-create" stage-create 2026-09-17
check 0 staging "stage-create with -N suffix" stage-create 2026-09-17-2
check 0 staging "stage-list" stage-list
check 0 staging "stage-exists" stage-exists 2026-09-17
check 0 staging "stage-index-apt" stage-index-apt 2026-09-17
check 0 staging "stage-index-yum" stage-index-yum 2026-09-17
echo
echo "== accepted requests: download role (paxsor) =="
check 0 download "snapshot-create" snapshot-create 2026-09-17
check 0 download "snapshot-sign" snapshot-sign 2026-09-17
check 0 download "snapshot-index-apt" snapshot-index-apt 2026-09-17
check 0 download "snapshot-index-yum" snapshot-index-yum 2026-09-17
check 0 download "snapshot-purge" snapshot-purge
check 0 download "release-exists" release-exists 9.18
check 0 download "release-create" release-create 9.18
check 0 download "release-create with patch" release-create 9.18.1
check 0 download "release-fetch" release-fetch 9.18 2026-09-17
check 0 download "packages-fetch" packages-fetch 2026-09-17
check 0 download "snapshot-create" snapshot-create 2026-09-17
check 0 download "rebuild-apt" rebuild-apt bookworm
check 0 download "rebuild-yum" rebuild-yum redhat rhel 9 x86_64
check 0 download "create-release" create-release 9.18
check 0 download "load-docs" load-docs 9.18
check 0 download "purge-cache" purge-cache
check 0 download "sync-s3" sync-s3
echo
echo "== rejected: shell metacharacters (exit 64) =="
check 64 staging "semicolon" "stage-create 2026-09-17; id"
check 64 staging "ampersands" "stage-create 2026-09-17 && id"
check 64 staging "pipe" "stage-create 2026-09-17 | sh"
check 64 staging "command substitution" 'stage-create $(id)'
check 64 staging "backticks" 'stage-create `id`'
check 64 staging "variable expansion" 'stage-create $HOME'
check 64 staging "newline" "$(printf 'stage-create 2026-09-17\nrebuild-apt bookworm')"
check 64 staging "tab" "$(printf 'stage-create\t2026-09-17')"
check 64 staging "single quotes" "stage-create '2026-09-17'"
check 64 staging "redirection" "stage-create 2026-09-17 > /etc/passwd"
check 64 staging "NUL-adjacent control char" "$(printf 'stage-create 2026-09-17\r')"
echo
echo "== rejected: path traversal (exit 64) =="
check 64 staging "absolute path" "stage-create /etc"
check 64 staging "relative traversal" "stage-create ../../etc"
check 64 staging "bare dot-dot" "stage-create .."
check 64 staging "dot-dot inside a name" "stage-create 2026-09-17..x"
check 64 staging "hidden directory" "stage-create .ssh"
check 64 download "traversal in version" "release-create ../../etc"
echo
echo "== rejected: malformed arguments (exit 64) =="
check 64 staging "unknown verb" "no-such-verb"
check 64 staging "empty request" ""
check 64 staging "missing argument" "stage-create"
check 64 staging "extra argument" "stage-create 2026-09-17 2026-09-18"
check 64 staging "impossible date" "stage-create 2026-13-45"
check 64 staging "date before the build farm" "stage-create 1999-01-01"
check 64 staging "date far in the future" "stage-create 2099-01-01"
check 64 staging "datestamp that is not a date" "stage-create latest"
check 64 download "version with four parts" "release-create 9.18.1.2"
check 64 download "version with letters" "release-create 9.18beta"
check 64 staging "over-long request" "stage-create $(printf '2%.0s' $(seq 1 300))"
echo
echo "== rejected: not permitted (exit 77) =="
check 77 staging "download verb on staging" "release-create 9.18"
check 77 download "staging verb on download" "stage-create 2026-09-17"
check 77 download "codename not allowlisted" "rebuild-apt sarge"
check 77 download "yum target not allowlisted" "rebuild-yum redhat rhel 7 x86_64"
check 77 download "mismatched yum tuple" "rebuild-yum fedora rhel 9 x86_64"
check 64 download "sync-s3 takes no argument" "sync-s3 release"
check 77 download "unpublished yum arch" "rebuild-yum redhat rhel 9 aarch64"
check 77 download "unpublished apt codename" "rebuild-apt plucky"
echo
echo "== through the real entry point, SSH_ORIGINAL_COMMAND =="
check_ssh 64 "tab instead of a space" "$(printf 'stage-create\t2026-09-17')"
check_ssh 64 "embedded newline" "$(printf 'hello\nstage-create 2026-09-17')"
check_ssh 64 "trailing carriage return" "$(printf 'hello\r')"
check_ssh 64 "Cyrillic homoglyph in the verb" "ѕtage-create 2026-09-17"
check_ssh 64 "non-ASCII in an argument" "rebuild-apt bookwоrm"
check_ssh 64 "NUL-ish escape in the request" "$(printf 'hello\\x00')"
echo
echo "== an argument may not look like an option =="
check 64 download "leading hyphen as a codename" "rebuild-apt -r"
check 64 download "leading hyphen as a datestamp" "snapshot-sign -rf"
check 64 staging "lone hyphen" "stage-exists -"
echo
echo "== the dry-run flag is not reachable from a client =="
# With SSH_ORIGINAL_COMMAND set, our own argv is ignored entirely, so a client
# that somehow contrived to influence it still cannot turn on dry-run, cannot
# choose a role, and cannot smuggle in a second request. On a host with no
# /etc/pga-publish.role this exits 69; on a real server it runs `hello`. What
# matters is that release-create does not appear in the output either way.
output="$(SSH_ORIGINAL_COMMAND=hello "$WRAPPER" --dry-run --role download \
release-create 9.18 2>&1)"
if printf '%s' "$output" | grep -q "would create"; then
fail=$((fail + 1))
echo "FAIL client reached --dry-run and release-create"
else
pass=$((pass + 1))
echo "ok argv ignored when SSH_ORIGINAL_COMMAND is set"
fi
echo
echo "passed $pass, failed $fail"
[ "$fail" = 0 ]
+26
View File
@@ -0,0 +1,26 @@
# The Python that pgAdmin builds against and ships.
#
# Every platform's packaging reads this file, so that a given commit produces
# the same interpreter everywhere rather than whichever one happened to be
# installed on the machine that ran the build, and so that a change of version
# arrives as a reviewable commit rather than as a settings change somebody made
# on a build agent.
#
# The format is deliberately dull: KEY=value, one per line, with # comments. It
# has to be read by a batch file, a shell script and occasionally a person, and
# anything richer than this would need a parser in at least one of those.
#
# PYTHON_VERSION is the exact CPython release, and is the only value the macOS
# build needs, since relocatable-python fetches the framework from python.org
# by version. The Windows build fetches a relocatable distribution from
# astral-sh/python-build-standalone, whose releases are dated rather than
# versioned, hence the second value; it then ships the matching embeddable
# distribution from python.org.
#
# The two must be changed together, because the archive names embed both, as
# cpython-<PYTHON_VERSION>+<PYTHON_BUILD_STANDALONE_TAG>-<platform>.tar.gz, and
# any given tag only carries the CPython releases that were current when it was
# cut.
PYTHON_VERSION=3.13.15
PYTHON_BUILD_STANDALONE_TAG=20260901
+3 -3
View File
@@ -38,15 +38,15 @@ echo "Installing build pre-requisites..."
yum groupinstall -y "Development Tools"
if [ "${OS_VERSION}" == "8" ]; then
yum install -y expect fakeroot postgresql15-devel python3.9-devel nodejs yarn rpm-build rpm-sign yum-utils krb5-devel
yum install -y expect fakeroot postgresql18-devel python3.9-devel nodejs yarn rpm-build rpm-sign yum-utils krb5-devel
pip3.9 install sphinx==7.4.7
pip3.9 install sphinxcontrib-youtube
elif [ "${OS_VERSION}" == "9" ]; then
yum install -y expect libpq5-devel postgresql15-devel python3-devel nodejs yarn rpm-build rpm-sign yum-utils krb5-devel
yum install -y expect libpq5-devel postgresql18-devel python3-devel nodejs yarn rpm-build rpm-sign yum-utils krb5-devel
pip3 install sphinx==7.4.7
pip3 install sphinxcontrib-youtube
else
yum install -y expect fakeroot postgresql15-devel python3-devel nodejs yarn rpm-build rpm-sign yum-utils krb5-devel
yum install -y expect fakeroot postgresql18-devel python3-devel nodejs yarn rpm-build rpm-sign yum-utils krb5-devel
pip3 install sphinx==7.4.7
pip3 install sphinxcontrib-youtube
fi
+55 -155
View File
@@ -9,165 +9,65 @@ builds may still work with suitable adjustments.
1. Install Chocolatey from https://chocolatey.org/install#individual
2. Install Visual Studio 2017 Pro (PostgreSQL 16 and below) from
https://my.visualstudio.com/Downloads?q=Visual%20Studio%202017.
Choose the Desktop development with C++ option, and ensure that you add the
'Visual C++ MFC for x86 and x64' option.
3. Install Visual Studio (PostgreSQL 17 and above):
2. Install Visual Studio:
choco install visualstudio2022community --add Microsoft.VisualStudio.Component.VC.ATLMFC;includeRecommended --add Microsoft.VisualStudio.Workload.NativeDesktop;includeRecommended --add Microsoft.VisualStudio.Component.VC.CMake.Project;includeRecommended
4. Install various command line tools:
3. Install the command line tools the build uses:
choco install -y awk bzip2 cmake diffutils dotnet3.5 gnuwin32-coreutils.install gzip git html-help-workshop meson innosetup ninja nodejs-lts python sed strawberryperl wget winflexbison3 yarn
choco install -y git innosetup nodejs-lts python syft wget yarn
5. Ensure the GNU CoreUtils and Microsoft HTML Help Workshop are in the system path - add:
That is the whole list for building pgAdmin. The build calls yarn, node,
npm, wget for the Electron download, syft for the SBOM, ISCC from Inno
Setup, and signtool if you are signing; git is used indirectly, the bundle
step recording the commit hash. curl and tar are used too, but Windows has
shipped both since 1803, so they need no installing.
Python is here for a development environment only. Building an installer
does not need one, because Make.bat downloads the version named in
*pkg/python-version.txt* into the build tree.
If you intend to build the dependencies from source, rather than taking
them from pgbuild as described below, you will also want the toolchain
that needs, and the GnuWin32 binaries and HTML Help Workshop on the
system path:
choco install -y awk bzip2 cmake diffutils dotnet3.5 gnuwin32-coreutils.install gzip html-help-workshop meson ninja sed strawberryperl winflexbison3
* C:\Program Files (x86)\GnuWin32\bin
* C:\Program Files (x86)\HTML Help Workshop
6. Upgrade pip:
4. Upgrade pip:
python -m pip install --upgrade pip
7. Install virtualenv:
5. Install virtualenv:
pip install virtualenv
## Building dependencies (PostgreSQL 16 and below)
## Getting the dependencies
The following steps should be run from a Visual Studio 64bit command
prompt.
Download a pre-built set of dependencies from the
[pgbuild project](https://github.com/pgadmin-org/pgbuild), which is what the
release builds use. Building them from source is possible but not documented
here: PostgreSQL 17 and later use Meson and pkg-config, which takes a fair
amount of setting up, and doing it by hand gets you binaries built against
different libraries from the ones we ship. Each dependency is published as a
release whose tag ends in *-latest*, so the current build is always at a
predictable URL rather than buried in a workflow run:
1. Create a directory for the dependencies:
https://github.com/pgadmin-org/pgbuild/releases/download/postgresql-18-windows-x86_64-latest/postgresql-18-windows-x86_64-latest.zip
https://github.com/pgadmin-org/pgbuild/releases/download/krb5-windows-x86_64-latest/krb5-windows-x86_64-latest.zip
mkdir c:\build64
Extract both into the same directory, such as `C:\Build64`; they unpack into
*postgresql* and *krb5* subdirectories. Kerberos is a separate download
because the PostgreSQL build doesn't include gssapi support, using native SSPI
instead.
2. Download the zlib source code, unpack, and build it:
wget https://zlib.net/zlib-1.3.11.tar.gz
tar -zxvf zlib-1.3.1.tar.gz
cd zlib-1.3.1
nmake -f win32/Makefile.msc
mkdir c:\build64\zlib
mkdir c:\build64\zlib\bin
copy zlib1.dll c:\build64\zlib\bin\
copy zlib1.pdb c:\build64\zlib\bin\
mkdir c:\build64\zlib\include
copy zlib.h c:\build64\zlib\include\
copy zconf.h c:\build64\zlib\include\
mkdir c:\build64\zlib\lib
copy zlib.lib c:\build64\zlib\lib\
copy zlib.pdb c:\build64\zlib\lib\
copy zdll.lib c:\build64\zlib\lib\
copy zdll.exp c:\build64\zlib\lib\
cd ..
3. Download the OpenSSL source code, unpack and build it:
wget https://www.openssl.org/source/openssl-3.0.13.tar.gz
tar -zxvf openssl-3.0.13.tar.gz
cd openssl-3.0.13
perl Configure VC-WIN64A no-asm --prefix=C:\build64\openssl no-ssl3 no-comp
nmake
nmake test
nmake install
cd ..
Note that if you are not working in an administrative account, you may need to
create and give your regular account appropriate permissions to write/modify
files in *C:\Program Files\Common Files\SSL*. This is the default directory used
for the *OPENSSLDIR*, and should not be changed to a directory that un-privileged
users could potentially write to.
4. Download the MIT Kerberos source code, unpack and build it:
In a *32bit* Visual Studio command prompt:
wget https://kerberos.org/dist/krb5/1.21/krb5-1.21.2.tar.gz
tar -zxvf krb5-1.21.2.tar.gz
mkdir C:\build64\krb5
cd krb5-1.21.2\src
set KRB_INSTALL_DIR=C:\build64\krb5
nmake -f Makefile.in prep-windows
Optionally, if you want 32bit binaries as well as 64bit:
nmake NODEBUG=1
nmake install NODEBUG=1
In a *64bit* Visual Studio command prompt:
cd krb5-1.21.2\src
set PATH=%PATH%;"%WindowsSdkVerBinPath%"\x86
set KRB_INSTALL_DIR=C:\build64\krb5
nmake NODEBUG=1
nmake install NODEBUG=1
cd ..\..
5. Download the PostgreSQL source code, unpack and build it:
wget https://ftp.postgresql.org/pub/source/v13.3/postgresql-16.3.tar.gz
tar -zxvf postgresql-16.3.tar.gz
cd postgresql-16.3\src\tools\msvc
>> config.pl echo # Configuration arguments for vcbuild.
>> config.pl echo use strict;
>> config.pl echo use warnings;
>> config.pl echo.
>> config.pl echo our $config = {
>> config.pl echo asserts =^> 0, # --enable-cassert
>> config.pl echo ldap =^> 1, # --with-ldap
>> config.pl echo extraver =^> undef, # --with-extra-version=^<string^>
>> config.pl echo gss =^> undef, # --with-gssapi=^<path^>
>> config.pl echo icu =^> undef, # --with-icu=^<path^>
>> config.pl echo nls =^> undef, # --enable-nls=^<path^>
>> config.pl echo tap_tests =^> undef, # --enable-tap-tests
>> config.pl echo tcl =^> undef, # --with-tcl=^<path^>
>> config.pl echo perl =^> undef, # --with-perl
>> config.pl echo python =^> undef, # --with-python=^<path^>
>> config.pl echo openssl =^> 'C:\build64\openssl', # --with-openssl=^<path^>
>> config.pl echo uuid =^> undef, # --with-ossp-uuid
>> config.pl echo xml =^> undef, # --with-libxml=^<path^>
>> config.pl echo xslt =^> undef, # --with-libxslt=^<path^>
>> config.pl echo iconv =^> undef, # (not in configure, path to iconv)
>> config.pl echo zlib =^> 'C:\build64\zlib' # --with-zlib=^<path^>
>> config.pl echo };
>> config.pl echo.
>> config.pl echo 1;
>> buildenv.pl echo $ENV{PATH} = "C:\\build64\\openssl\\bin;C:\\build64\\zlib\\bin;$ENV{PATH}";
perl build.pl Release
perl vcregress.pl check
perl install.pl c:\build64\pgsql
copy c:\build64\zlib\bin\zlib1.dll c:\build64\pgsql\bin\
copy c:\build64\openssl\bin\libssl-3-x64.dll c:\build64\pgsql\bin\
copy c:\build64\openssl\bin\libcrypto-3-x64.dll c:\build64\pgsql\bin\
## Building dependencies (PostgreSQL 17 and above)
PostgreSQL 17 and later use Meson for generating project/build files, and
makes use of pkg-config to locate dependencies and configure the build
accordingly. Whilst this is arguably more reliably and flexible, it also
takes some effort to setup.
It is therefore recommended that you simply download a pre-built set of
PostgreSQL binaries from the
[winpgbuild project](https://github.com/dpage/winpgbuild/actions/workflows/postgresql.yml).
Locate the binaries asset for the version of PostgreSQL you wish to use
in the most recent workflow run, and extract the contents to a suitable
directory such as `C:\Build64`.
Repeat the process with the latest build of
[MIT Kerberos](https://github.com/dpage/winpgbuild/actions/workflows/krb5.yml),
merging the files into the same set of directories. This is required because
the PostgreSQL build doesn't include Kerberos (gssapi) support as it uses
native SSPI instead.
The build checks in *.github/workflows* fetch exactly these, through the
*install-pgbuild-deps* action, so a local build and a CI build are working
from the same binaries.
## Setting up a dev environment
@@ -199,34 +99,34 @@ desktop runtime.
## Building an installer
1. Set the required environment variables, either system-wide, or in a Visual
Studio 2017 (or 2022 with PostgreSQL 17+) 64bit command prompt. Note that the
examples shown below are the defaults for the build system, so if they match
your requirements you don't need to set them. For PostgreSQL 16 and below:
Studio 2022 64bit command prompt. The values below are examples rather than
defaults, so check them against where you unpacked the dependencies:
SET "PGADMIN_POSTGRES_DIR=C:\build64\pgsql"
SET "PGADMIN_PYTHON_DIR=C:\Python314"
SET "PGADMIN_POSTGRES_DIR=C:\build64\postgresql"
SET "PGADMIN_KRB5_DIR=C:\build64\krb5"
SET "PGADMIN_INNOTOOL_DIR=C:\Program Files (x86)\Inno Setup 6"
SET "PGADMIN_SIGNTOOL_DIR=C:\Program Files (x86)\Windows Kits\10\bin\10.0.17763.0\x64"
SET "PGADMIN_VCREDIST_DIR=C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\VC\Redist\MSVC\14.16.27012"
SET "PGADMIN_VCREDIST_FILE=vcredist_x64.exe"
For PostgreSQL 17 and later:
SET "PGADMIN_POSTGRES_DIR=C:\build64"
SET "PGADMIN_PYTHON_DIR=C:\Python314"
SET "PGADMIN_KRB5_DIR=C:\build64"
SET "PGADMIN_INNOTOOL_DIR=C:\Program Files (x86)\Inno Setup 6"
SET "PGADMIN_SIGNTOOL_DIR=C:\Program Files (x86)\Windows Kits\10\bin\10.0.22621.0\x64"
SET "PGADMIN_VCREDIST_DIR=C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Redist\MSVC\14.40.33807"
SET "PGADMIN_VCREDIST_FILE=vc_redist.x64.exe"
Note that Python is not among them. Make.bat downloads the exact version
named in *pkg/python-version.txt* into the build tree, both to build the
virtual environment with and to ship, so no Python needs to be installed
for the installer build and the one on the PATH is not consulted.
2. Run:
make
If you have a code signing certificate, this will automatically be used if
found in the Windows Certificate Store to sign the installer.
found in the Windows Certificate Store to sign the installer. Signing is
enabled by setting PGADMIN_WINDOWS_CSC to the certificate's subject name;
without it the build produces an unsigned installer.
See SIGNING.md for how the release signing host is set up, including the
Certum hardware token, the certificate-to-key repair that its "simple"
registration does not do correctly, and why the build agent must run in an
interactive session rather than as a Windows service.
3. Find the completed installer in the dist/ subdirectory of your source tree.
+232
View File
@@ -0,0 +1,232 @@
# pgAdmin Windows Signing Host
The Windows installer is signed with a Certum Open Source code signing
certificate held on a hardware cryptographic card, which cannot be copied,
exported, or handed to a cloud runner. Everything else in the release pipeline
runs on ephemeral, disposable machines described entirely by the workflow files
in `.github/workflows/`; this one machine is configured by hand and holds
something irreplaceable, so this document is the record of how it is set up.
If that machine ever dies, this document is what stands between you and a very
bad week. Please keep it current.
## What the machine does
It runs a self-hosted GitHub Actions runner, which builds and signs the
snapshot and release installers (`.github/workflows/snapshot-windows.yml` and
its release equivalent). The unsigned QA build in `check-windows-build.yml`
deliberately does *not* run here: it uses a GitHub-hosted runner, because it
needs no certificate and there is no reason to put ordinary CI traffic on the
machine that holds the signing key.
## Hardware
A Certum cryptographic card in an ACS ACR40T reader, holding the code signing
certificate and its private key. The key never leaves the card; signing
operations happen on it.
## Why the runner must not be a Windows service
This is the single most important thing on this page, because the failure is
silent rather than loud.
Signing prompts for the card's PIN, and that prompt is a window on the
interactive desktop. An AutoHotkey helper answers it automatically. A Windows
service runs in session 0, which has no desktop, so the prompt appears
somewhere the helper cannot see it and the build hangs until its timeout rather
than failing with anything useful.
So the machine logs in automatically, and both the runner and the PIN helper
start in that session. When configuring the runner, answer **N** to
`config.cmd`'s "run as a service" question, and never pass `--runasservice`,
`--windowslogonaccount` or `--windowslogonpassword`.
The same applies to Task Scheduler, if you use it instead of the Startup
folder: choose "Run only when user is logged on". The adjacent option, "Run
whether user is logged on or not", is the service problem wearing a different
hat.
## Machine configuration
### Automatic login
```powershell
$regPath = "HKLM:\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Winlogon"
Set-ItemProperty -Path $regPath -Name "AutoAdminLogon" -Value "1"
Set-ItemProperty -Path $regPath -Name "DefaultUserName" -Value "<build user>"
Set-ItemProperty -Path $regPath -Name "DefaultPassword" -Value "<password>"
```
### Never sleep
A sleeping machine is an offline runner, and a release that waits.
```powershell
powercfg -change -standby-timeout-ac 0
powercfg -change -monitor-timeout-ac 0
powercfg -change -hibernate-timeout-ac 0
```
### The Actions runner
Install under a short path, `C:\actions-runner`, to stay clear of Windows path
length limits. Register it against `pgadmin-org/pgadmin4`, answering **N** to
the service question, then start it from the logged-in session:
```powershell
$startup = [Environment]::GetFolderPath('Startup')
$s = (New-Object -ComObject WScript.Shell).CreateShortcut("$startup\actions-runner.lnk")
$s.TargetPath = 'C:\actions-runner\run.cmd'
$s.WorkingDirectory = 'C:\actions-runner'
$s.Save()
```
Startup folder ordering is not guaranteed, so if you want to be certain the PIN
helper is up before any job can arrive, point the shortcut at a batch file that
sleeps briefly first.
### The PIN helper
An AutoHotkey executable that watches for the card's PIN dialog and answers it.
It must start in the same session, by the same mechanism, at login.
### Build prerequisites
The build needs Inno Setup, syft and wget beyond what Visual Studio and the
Windows SDK provide. The workflow checks for these and stops with the command
to install them rather than installing anything itself, on the grounds that a
CI job should not be modifying the one machine that cannot be rebuilt.
```
choco install innosetup syft wget -y
```
Visual Studio, the VC++ redistributable and `signtool.exe` are located by the
workflow at build time rather than configured here, so an SDK or Visual Studio
update does not require editing anything.
## Certificate setup
The card's "simple" registration through proCertum Card Manager does not
correctly link the certificate to its private key. Without the repair below,
signing either fails or silently degrades to SHA1.
**All of this needs local console access.** Smart card operations do not work
over Remote Desktop, and the usual symptom is `certutil -key` failing with
`NTE_KEYSET_NOT_DEF (0x80090019)`.
### 1. Put proCertum in CSP mode
Open proCertum Card Manager, go to Options, select **CSP driver** rather than
Minidriver, apply, and restart Windows.
### 2. Import the certificate to the machine store
In proCertum Card Manager: Read Card, then the Common Profile tab, select the
code signing certificate, Show certificate details, and install it into the
Windows **machine** store, under Personal. The machine store is why the build
signs with `/sm`.
### 3. Link the certificate to the key
`certutil -store My` will show the certificate with `No key provider
information`, which is expected at this point.
Find the key container:
```batch
certutil -key -csp "crypto3 CSP"
```
Create `keyprov.inf`, substituting the container ID. The trailing `&` on each
line is required INF syntax, not a typo:
```ini
[Properties]
2 = "{text}"
_continue_ = "Container=<key container id>&"
_continue_ = "Provider=crypto3 CSP&"
_continue_ = "ProviderType=1&"
_continue_ = "Flags=0&"
_continue_ = "KeySpec=2"
```
Find the certificate's SHA1 thumbprint with `certutil -store My`, looking for
`Cert Hash(sha1)`, and repair the store as Administrator:
```batch
certutil -repairstore My "<thumbprint>" keyprov.inf
```
`certutil -store My` should now report `Provider = crypto3 CSP` and a
`Key Container`.
## How the build signs
`Make.bat` signs only when `PGADMIN_WINDOWS_CSC` is set, and selects the
certificate by subject name from the machine store. It is held as a repository
variable on `pgadmin-org/pgadmin4` rather than in this repository, so that a
certificate renewal is a settings change rather than a commit. The snapshot
workflow fails immediately if it is unset, rather than quietly producing an
unsigned installer that looks like a successful build.
The signing call it makes is equivalent to:
```batch
signtool sign /sm /n "<certificate subject>" /tr http://timestamp.digicert.com /td sha256 /fd sha256 /v <file>
```
| Parameter | Why |
|---|---|
| `/sm` | machine store, which is where the certificate was installed |
| `/n` | selects the certificate by subject name |
| `/fd sha256` | file digest algorithm |
| `/tr` | RFC 3161 timestamp server; prefer this over the legacy `/t` |
| `/td sha256` | timestamp digest algorithm |
Timestamping matters: without it, signatures stop validating when the
certificate expires. If DigiCert's server is unavailable,
`http://timestamp.sectigo.com` and `http://ts.ssl.com` are alternatives.
Certum's own, `http://time.certum.pl`, was unreliable during setup.
Verify with:
```batch
signtool verify /pa /v <file>
```
## Troubleshooting
**`certutil -key` fails with `NTE_KEYSET_NOT_DEF (0x80090019)`.** Almost always
Remote Desktop: smart card operations need a local console session. Otherwise
check proCertum is in CSP mode rather than Minidriver, that Windows was
restarted after changing it, and that the card is seated and readable.
**"No certificates were found that met all the given criteria".** Check `/sm`
is being passed, that `certutil -store My` shows the certificate, and that the
key provider link from step 3 exists.
**`SignerSign() failed`.** Check the card is inserted and readable, that the
PIN prompt has not appeared behind another window, and that the
certificate-to-key link is intact.
**The build hangs rather than failing.** The runner is almost certainly running
as a service, or as a scheduled task set to "run whether user is logged on or
not", so the PIN prompt is somewhere nothing can answer it. See above.
**The first signature after the card is inserted prompts for a PIN.** Expected.
Subsequent operations in the same session may be cached, depending on proCertum
settings.
## Fully unattended signing
There isn't a clean answer. PIN entry is the obstacle, and the arrangement here
works around it rather than solving it: the card is unlocked once per boot in
an interactive session and a helper answers the dialog. Options investigated
and rejected were PIN caching in proCertum alone, which does not survive a
reboot, and `CRYPTOAPI_PIN`-style environment variables, which the CSP ignored.
If this becomes painful enough to be worth removing, the way out is a cloud
signing service such as Azure Trusted Signing, SignPath, or Certum's own cloud
offering, all of which replace the token with an API credential and would let
this machine disappear along with the rest of the buildfarm.
+6
View File
@@ -36,8 +36,14 @@ ArchitecturesInstallIn64BitMode={#MyAppArchitecturesMode}
AllowNoIcons=yes
WizardImageFile=sidebar.bmp
MinVersion={#MinimumWindowsVer}
; Only emitted for a signing build. Make.bat passes /DSIGNED alongside the
; /SpgAdminSigntool= definition; without both, ISCC aborts with "Value of
; [Setup] section directive SignTool is invalid" because the named SignTool
; was never defined.
#ifdef SIGNED
SignTool=pgAdminSigntool
SignedUninstaller=yes
#endif
CloseApplications=yes
[Languages]