pgadmin4-appbundle-build #114 (macos-arm64) failed on the same
"pgAdmin 4 Helper (GPU)" binary as before, but this time the
diagnostics from d4364555a actually ran and proved the path itself is
not the problem:
Raw bytes of the path (od -c): plain ASCII throughout, single
spaces, no non-breaking/unicode look-alikes, no trailing garbage.
ls -la of the exact path: file exists, correct size/perms,
timestamped seconds before otool ran - not a race condition either.
otool -L still failed via its internal otool-classic re-exec on that
exact, verified-correct, verified-present file. That's conclusively a
bug in otool's own internal dispatch, not anything this script builds
or passes to it - we can't fix Apple's tool, but we can avoid
triggering its broken code path.
Retry via `otool-classic -L "${TODO_OBJ}"` called directly (our own
argv-based quoting, not otool's internal re-exec, whatever that does
differently) when the initial `otool -L` fails. Every other binary in
the bundle keeps going through plain `otool -L` unchanged - this only
engages for the specific case that's already failing anyway. The
diagnostic dump (od -c / ls -la / otool-classic's own output) is kept
as the final fallback in case even the direct call fails too.
Not yet verified against a real buildfarm run.
pgadmin4-appbundle-build #113 (macos-arm64) failed again on the same
otool -L call, but none of the diagnostic output from 401afbfa3 showed
up. Instead:
The command "OTOOL_OUTPUT=$(otool -L "${TODO_OBJ}" 2>&1)" failed in
"_fixup_imports" with exit code 1.
That's this script's own ERR trap firing on the assignment statement
itself. Under set -e, `VAR=$(cmd)` still propagates cmd's non-zero
exit to the trap immediately - the previous diagnostic's separate
`OTOOL_STATUS=$?` line right after never got a chance to run, so
nothing was ever printed, same as the original bare failure.
Move the assignment into the condition of the `if` itself
(`if ! OTOOL_OUTPUT=$(...); then`) - bash explicitly exempts a
command being tested by if/while/until (or negated with !) from
errexit, so this time the diagnostic block will actually execute.
Still not verified - watching the next buildfarm run.
pgadmin4-appbundle-build #112 (macos-arm64) failed with otool-classic
unable to open a path that our own logging showed as correct
immediately beforehand ("pgAdmin 4 Helper (GPU)"). Rather than work
around it blindly (an otool-classic fallback was tried and reverted -
unverified, and risks masking a real problem instead of fixing it),
capture real evidence the next time this happens:
- raw byte dump (od -c) of ${TODO_OBJ} right before otool runs, and
again on failure - catches non-ASCII or invisible whitespace that
a plain `echo` can't show
- ls -la of the exact path and its containing directory on failure -
confirms whether the file actually exists at that exact moment
(rules out/in a race with something still writing it)
- otool's actual stdout/stderr and exit code, instead of letting its
own error text be the only signal
No fallback behavior - fails loudly (exit 1) with this diagnostic
dump so we can tell what's actually happening from the next real
buildfarm run instead of speculating further.
pgadmin4-appbundle-build #110 (macos-x64) failed with:
error: otool-classic: can't open file: ./Contents/Frameworks/pgAdmin
error: otool-classic: can't open file: ./Contents/Frameworks/Electron
_fixup_imports built its worklist with `awk -F':| '`, a regex
alternation that splits on a literal ':' OR any bare space. `file`'s
actual output format is "path: description" (colon-space as one
token), so any space *inside* the path itself - "pgAdmin 4 Helper
(Plugin).app", "Electron Framework.framework", both routine in macOS
app bundles - also got treated as a split point, truncating $1 to
whatever preceded the first space.
The resulting list was then joined with a single space (ORS=" ") and
iterated via an unquoted `for x in $list`, which word-splits on
spaces again - doubly ambiguous between a path's own spaces and the
list's separator, unrecoverable however it's parsed.
Fix: split only on the literal ": " token file emits (`awk -F': '`,
not `-F ':| '`), and switch the whole worklist to newline-separated
instead of space-separated, iterated with `while IFS= read -r`
instead of `for x in $unquoted_list`, at both the outer (executable)
level and inner (library-copy) level (`TODO="${TODO}"$'\n'"..."`).
Not yet verified against a real buildfarm run - can't reproduce the
macOS codesigning environment locally. Watching the next
pgadmin4-appbundle-build run.
macOS build (build-functions.sh): _notarize_pkg only printed
"Notarization failed." on rejection, giving no indication why. Print
the actual REQUEST_STATUS and fetch the full notary log via
`notarytool log` so future failures are actually diagnosable from the
Jenkins console instead of just "status: Invalid".
Linux build (build-functions.sh): _create_python_virtualenv creates
venvs with --system-site-packages, which pulls in the entire system
site-packages dir - including any stale, improperly-uninstalled
package's namespace-package .pth hook. That legacy pip/setuptools
mechanism runs at interpreter startup, before core stdlib is
guaranteed to resolve; a leftover sphinxcontrib-jsmath nspkg.pth on
the el-10 build node corrupted sys.path early enough to break pip's
own subprocess bootstrap, failing the whole build with a misleading
"No module named 'importlib'"/"'traceback'" error
(pgadmin4-rpm-build #176). Remove that exact known-broken file before
creating the venv - not every *-nspkg.pth, since this touches the
*system* Python install on a build node shared by other jobs, and any
other such file could still be load-bearing for something unrelated.
--system-site-packages itself is left untouched - it's required so
venvs can see OS-provided packages not available as clean pip wheels
on every target platform.
_verify_bundle_linkage's host-library scan only matched files by
.so/.dylib name suffix, so a shipped executable (Contents/MacOS/*)
or a Python.framework payload with a host-linked dependency but no
matching suffix could slip through unchecked — the same class of
bug #10135 was added to catch.
Detect Mach-O executables/libraries by content via `file` instead
(same pattern the codesign step in this file already uses),
regardless of extension or executable permission bits, so readable
non-executable dylibs are covered too.
9.16 failed to start on Intel Macs (#10123): cryptography had no
prebuilt Intel macOS wheel, pip compiled it from source, and its
openssl-sys build linked the builder's Homebrew OpenSSL into
_rust.abi3.so instead of the bundled one. _fixup_imports deliberately
skips _rust.abi3.so, so the dangling reference shipped unnoticed and
the app died on startup for anyone without that external dylib.
Add _verify_bundle_linkage, run after the bundle is assembled and
relocated but before code-signing: walks every .so/.dylib and fails
the build if any install-name points at a build-host prefix
(/usr/local, /opt/homebrew, /opt/local, $SLAVE_HOME). OS libraries
and @loader_path/@rpath/@executable_path references pass untouched.
Turns this class of bug into a build-time failure instead of a
runtime one.
#10156 fixed CI build failures caused by Yarn 4.x fetching a newer patch
with different builtin compat hashes, breaking --immutable lockfile
validation, but only patched pkg/linux/build-functions.sh and the GHA
workflows. Make.bat, pkg/mac/build-functions.sh, pkg/pip/build.sh, and
Dockerfile still hardcoded 'yarn set version 4', which is what broke the
Windows Jenkins snapshot build (job 1669).
A signed, notarised and stapled pgAdmin 4.app was rejected by Gatekeeper
with "invalid destination for symbolic link in bundle". Gatekeeper walks
every symlink in the bundle and rejects the whole app if any link does
not resolve to a real file inside it; notarisation does not catch this, so
a broken link slips through stapling and only surfaces as a Gatekeeper
failure on the end user's machine.
The embedded Python.framework ships such links: an arm64-only build still
carries a bin/python3-intel64 launcher symlink (whose target
_strip_architecture deletes when it removes the foreign-arch files), and
the bundled Tcl/Tk frameworks carry PrivateHeaders links pointing at a
Versions/Current that has none.
Add a _prune_dangling_symlinks step that removes every dangling symlink in
the bundle after architecture stripping and before signing, then fails the
build if any remain, so this cannot slip past notarisation again.
The macOS app is built for a single architecture (matching the build
machine, via ${ARCH}), but relocatable-python pulls the python.org
universal2 installer, so the entire Python.framework ships both arm64
and x86_64 slices. PostgreSQL-sourced dylibs may be universal too. The
foreign slice is dead weight that bloats the bundle and DMG.
Add a _strip_architecture step, run after _complete_bundle and before
code-signing (lipo invalidates signatures, so the existing sign passes
re-sign the thinned binaries). It removes the universal2 stragglers
(python*-intel64 launcher, config-*-darwin/python.o) and lipo-thins
every fat Mach-O in the bundle to the build arch, preserving file modes
and warning on anything lacking the target slice. Already single-arch
inputs (Electron and its helpers) are skipped.
Bumps the desktop runtime to electron 42 (dependabot PR #9945) and
closes a supply-chain gap in the Linux/Mac packaging scripts that
predated this bump.
Why the bump is safe:
- macOS UNNotification API change — pgAdmin's runtime does not use
Electron's Notification API (only a UI toast comment in
src/js/pgadmin.js:211; no `new Notification(...)` anywhere).
- postinstall no longer downloads electron — production packaging
fetches the binary directly via wget from GitHub releases, never
via electron's postinstall script.
- Offscreen rendering scale-factor change — no OSR usage anywhere
in runtime/src/js/.
While verifying, found that pkg/linux/build-functions.sh and
pkg/mac/build-functions.sh resolve the packaged electron version
via:
ELECTRON_VERSION="$(npm info electron version)"
This pulls whatever currently carries the `latest` dist-tag on the
npm registry. Any newly published electron release — including a
hypothetical malicious one — would land in shipped binaries without
review, regardless of what runtime/package.json pins.
Replace with sed-based extraction from runtime/package.json and
fail loudly if extraction returns empty. The Windows installer
(pkg/win32/installer.iss.in) does not have this issue (it bundles a
pre-built tree, no electron download step).
Net change in runtime/yarn.lock is mostly deletions — electron 42
ships with @electron/get 5.x, which dropped a large transitive
dependency tree associated with the old postinstall download path.
Verified:
- eslint (runtime): clean (silent)
- yarn install (runtime): resolved to electron 42.2.0 within
^42.1.0 range
- sed extraction smoke-tested: returns 42.1.0 from current
runtime/package.json
The 6 GB ceiling set by #9967 was too aggressive for the macOS x64
VM's total RAM. Build #1295 on `pgabf-macos-x64` failed in
`_build_runtime` at `unzip electron-vX.X.X-darwin-x64.zip` with exit
code 2 — never even reached webpack. That points at OS-level memory
pressure spilling out of the Node process and starving the rest of
the build: at 6 GB reserved, the box runs out of RAM long before
Terser actually needs the full ceiling.
Drop back to 4 GB, which still gives Terser a full extra gigabyte
beyond the original 3 GB setting that OOM-killed webpack in #1294,
but leaves enough headroom for the other steps in the appbundle
build to coexist.
Only the macOS appbundle path changes (see pkg/mac/build-functions.sh);
linux/pip/Makefile and dev-machine builds keep the 3 GB the `bundle`
npm script ships with.
macOS x64 appbundle builds keep dying inside webpack's TerserPlugin at
92% (asset processing). Build #1294 on `pgabf-macos-x64` reached
`<s> [webpack.Progress] 92% [0] sealing asset processing TerserPlugin`
and was killed without producing a V8 fatal-error preamble, which
points at the OS reaping the Node process under memory pressure rather
than V8 hitting its own heap ceiling.
TerserPlugin is already running single-threaded (see
web/webpack.config.js, `parallel: false`), so we can't claw memory back
by reducing parallelism. Bump the V8 old-space ceiling from 3072 MB to
6144 MB inside the macOS appbundle build only — the helper in
pkg/mac/build-functions.sh bypasses `yarn run bundle` and calls
`yarn run webpacker` directly (see commit d96e8634), so this knob is
independent of the npm script and does not affect linux/pip/Makefile
or dev-machine builds. They keep the 3 GB the `bundle` script has been
shipping with for years.
If this still doesn't get the x64 box past Terser we'll switch the
minimiser to esbuild via terser-webpack-plugin's `minify` option; that
is a larger and more invasive change so we are trying the cheap fix
first.
The macOS x64 appbundle build can fail inside `yarn run bundle` while
producing zero console output -- the Jenkins log goes straight from
"yarn install ... Done with warnings" to the EXIT trap's failure
message, leaving no signal as to whether linter, webpack, or a native
module load was the culprit (build #1293 on pgabf-macos-x64 is the
prompting example).
Split the bundled script into its constituent steps and merge stderr
into stdout so any error text reaches the console even if Jenkins'
shell step drops a tail buffer:
yarn install
yarn run git:hash # cheap source-hash capture, moved up front
yarn run linter
yarn run webpacker
`git:hash` is a pure `git log` redirect (see web/package.json) with no
node-module dependency, but `yarn run` needs node_modules so it stays
after install. Pulling it before the heavy steps means the commit_hash
file lands on disk even if webpack later bails out.
Env vars NODE_ENV=production and NODE_OPTIONS=--max-old-space-size=3072
are set explicitly to mirror the cross-env wrapper inside the top-level
"bundle" npm script, so build output stays byte-identical to before.
No-op for successful builds; pure diagnostics win on failure.
1. Fixed the issue where auto-update was not working for macOS x64 arch machines as pgadmin4 zip file name has x86_64 in it.
2. Improved error handling in the /upgrade_check API by replacing the static “Failed to check for update” message for Windows users with a dynamic error message.
3. Fixed the CSS issue affecting the close icon in the warning notifier.
4. Removed trailing periods from helper texts and notifier messages in the app’s auto-update workflow. #9133
1. Replace the current layout library wcDocker with ReactJS based rc-dock. #6479
2. Have close buttons on individual panel tabs instead of common. #2821
3. Changes in the context menu on panel tabs - Add close, close all and close others menu items. #5394
4. Allow closing all the tabs, including SQL and Properties. #4733
5. Changes in docking behaviour of different tabs based on user requests and remove lock layout menu.
6. Fix an issue where the scroll position of panels was not remembered on Firefox. #2986
7. Reset layout now will not require page refresh and is done spontaneously.
8. Use the zustand store for storing preferences instead of plain JS objects. This will help reflecting preferences immediately.
9. The above fix incorrect format (no indent) of SQL stored functions/procedures. #6720
10. New version check is moved to an async request now instead of app start to improve startup performance.
11. Remove jQuery and Bootstrap completely.
12. Replace jasmine and karma test runner with jest. Migrate all the JS test cases to jest. This will save time in writing and debugging JS tests.
13. Other important code improvements and cleanup.
Ashesh Vashi
Dave Page
Anil Sahoo
Akshay Joshi
Pravesh Sharma
Khushboo Vashi
Aditya Toshniwal
Ashesh Vashi