chore(upstream): sync portable PostgreSQL releases (16,17,18) - #5
Closed
github-actions[bot] wants to merge 30 commits into
Closed
github-actions[bot] wants to merge 30 commits into
github-actions[bot] wants to merge 30 commits into
Conversation
Build system that produces fully self-contained, relocatable PostgreSQL archives for Linux (CentOS 7+), macOS (x86_64/aarch64), and Windows. Designed for enterprise environments with no internet access. Key features: - manylinux2014 container build with patchelf RPATH normalization for glibc 2.17+ portability - SHA256 checksum verification of upstream source tarballs - Full feature set bundled: readline, zlib, ICU, LDAP, NLS - Separate overlay archives for pg_hint_plan, pg_partman, pg_cron, pgvector - Automated upstream sync (cron) with config validation and auto-merge - CI optimized with ccache, pre-built container image, and package caching - Local builds auto-containerized on Linux via Docker/Podman - GitHub Actions expression injection hardened (env: intermediary pattern)
GitHub Container Registry requires all-lowercase repository names.
github.repository resolves to mixed-case (4mGLn/postgresql-portable),
which fails in image tags. Compute lowercase name via bash ${REPO,,}
in a run: block and pass it as a step/job output for YAML-level fields.
- Add FORCE_JAVASCRIPT_ACTIONS_TO_NODE24=true to all workflows - Add PostgreSQL 16.13 (REL_16_13) to tracked majors - Add pg_hint_plan REL16_1_6_2 ref for PG 16
The container: directive executes GitHub Actions' Node.js runner inside the container. manylinux2014 (CentOS 7, glibc 2.17) is too old for Node.js 20+, which requires glibc 2.27+. Switch to docker run with bind-mounted workspace so checkout/cache/upload actions run on the modern Ubuntu host while compilation runs inside the old glibc container.
- Replace container: directive with docker run for Linux builds to avoid Node.js 20+ glibc requirement inside manylinux2014 (CentOS 7) - Remove macOS build jobs (macos_x86_64, macos_aarch64) - Add mingw-w64-x86_64-pkg-config to Windows MSYS2 packages so configure can find ICU, readline, zlib via pkg-config - Add PG 16.13 to tracked majors - Opt all workflows into Node.js 24 (FORCE_JAVASCRIPT_ACTIONS_TO_NODE24)
- Add if: always() to publish job so it runs even when one platform fails, publishing whatever artifacts are available - Guard against publishing when all builds failed - Add mingw-w64-x86_64-pkg-config for Windows ICU/library detection - Remove macOS builds and targets
MSYS2 jq may output \r\n line endings. mapfile captures the \r as part of the value, causing extension/flag lookups to fail with "pg_cron\r not found". Add tr -d '\r' to all mapfile+jq pipelines in build_portable_postgresql.sh.
- Add skip_targets config field for per-extension platform exclusion - pg_cron requires Unix process management and cannot build on Windows - Build loop checks skip_targets before attempting extension build
On MSYS2/MINGW64, path representations can differ between Perl's realpath, bash pwd, and pg_config's baked-in prefix. Instead of computing ext_stage_prefix from install_prefix (which may not match where PGXS actually installed files), discover the staged .control file with find and derive the prefix from its location. Also handle win32ver.rc: copy it into the PGXS tree after PostgreSQL install, and create a stub in extension source dirs that lack one.
On MSYS2, pg_config returns Windows-style paths (D:/a/...) but DESTDIR is Unix-style. PGXS concatenates $(DESTDIR)$(prefix) directly, producing broken paths like "stageD:/a/..." where MSYS2 interprets D: as a drive letter and installs files outside the staging area entirely. Fix by creating a pg_config wrapper on MSYS2 that converts Windows paths to Unix-style via cygpath, so DESTDIR + prefix concatenation is valid. Also provide win32ver.rc stubs for PGXS extension builds on Windows.
Add 16 configure flags for a complete PostgreSQL feature set and bundle all required runtime dependencies on both Linux and Windows. Build integrity: - SHA256 checksum failures are now fatal; add --skip-checksum-verify for air-gapped builds - Always clean build_dir before configure to prevent stale object files causing duplicate-symbol linker errors - Add host-target compatibility guard (clear error + CI suggestion when attempting Linux→Windows cross-compile) Configure flags: - Add explicit configure_flags to config: --with-openssl, --with-python, --with-perl, --with-tcl, --with-lz4, --with-zstd, --with-gssapi, --with-ldap, --with-pam, --with-libxml, --with-libxslt, --with-uuid=e2fs, --enable-nls, --with-icu, --with-readline, --with-zlib - Add configure_flags_exclude schema for per-target exclusions (Windows skips --with-pam and --with-uuid=e2fs) - Update Dockerfile.manylinux2014 and MSYS2 package lists accordingly - Add validate-config.yml validation for configure_flags_exclude Windows DLL bundling: - Convert bundle_windows() to BFS queue to catch transitive deps (libgcc_s_seh-1.dll → libwinpthread-1.dll) - Use find -iname for case-insensitive DLL name resolution - Explicitly bundle libicudt*.dll (not in PE import table, loaded via u_init()) - Expand system DLL skip list: MSVCRT/UCRTBASE/MSVCP*/VCRUNTIME*, API-MS-WIN-*/EXT-MS-WIN-*, networking (DNSAPI, IPHLPAPI, WLDAP32, etc.) Developer tooling: - run_local_build.sh: build from Dockerfile instead of inline yum install - Add --jobs, --target, --no-ccache flags to run_local_build.sh - Add ccache support with per-target host cache (.cache/ccache-<target>) - Remove LD_LIBRARY_PATH from env.sh (redundant with $ORIGIN RPATHs) Documentation: - Add Known Limitations for PL/Python, PL/Perl, PL/Tcl, PAM, Windows CRT
Co-authored-by: 4mGLn <127629+4mGLn@users.noreply.github.com> Co-authored-by: aMgLn <aadaa88@gmail.com>
CentOS 7's openssl-devel is 1.0.2k; PostgreSQL requires >= 1.1.1. Switch to EPEL's openssl11-devel/openssl11-libs and set PKG_CONFIG_PATH to /usr/lib64/openssl11/pkgconfig so configure finds the correct version. MSYS2 Python 3.14 no longer exposes a shared-library stub in the form PostgreSQL's configure expects. Add --with-python to configure_flags_exclude for windows_x86_64 to skip PL/Python on Windows.
OPENSSL_CFLAGS/OPENSSL_LIBS only bypass PKG_CHECK_MODULES (PG16/17 path). PG18 added an AC_COMPILE_IFELSE probe for SSL_CTX_set_ciphersuites which uses CPPFLAGS/LDFLAGS, not OPENSSL_*, so it was still finding the system OpenSSL 1.0.2k headers and failing the version check. Set both CPPFLAGS/LDFLAGS (for all AC_CHECK_* probes) and OPENSSL_*/LIBS (for PKG_CHECK_MODULES) so every configure path across PG16-18 resolves against openssl11's 1.1.1 headers and libraries.
…ed flags MinGW64 GCC on GHA runners does not resolve /mingw64/lib in its default library search path when LDFLAGS is empty at configure time, causing every AC_CHECK_LIB and AC_SEARCH_LIBS conftest link to fail even though all packages are installed. Additionally, GHA Windows runners uppercase environment variable names, so lowercase autoconf cache variables set via the GHA env: block are invisible to configure's subshell. Fix: export every mandatory library cache variable directly in the msys2 run: block before invoking the build script. The exported values bypass the broken link tests and cause configure to run ACTION-IF-FOUND for each library, correctly populating LIBS. Cache variables bypassed (all use AC_CHECK_LIB or AC_SEARCH_LIBS): --with-openssl : ac_cv_search_CRYPTO_new_ex_data, ac_cv_search_SSL_new --with-libxml : ac_cv_lib_xml2_xmlSaveToBuffer --with-libxslt : ac_cv_lib_xslt_xsltCleanupGlobals --with-lz4 : ac_cv_lib_lz4_LZ4_compress_default --with-zstd : ac_cv_lib_zstd_ZSTD_compress --with-zlib : ac_cv_lib_z_inflate --with-readline : ac_cv_search_readline --with-ldap : ac_cv_lib_wldap32_ldap_bind (win32 uses wldap32) --enable-nls : ac_cv_search_ngettext Also exclude --with-gssapi for windows_x86_64: gssapi.h requires a MINGW64 Kerberos package (e.g. mingw-w64-x86_64-heimdal) which is not installed. Windows GSSAPI/SSPI is not supported in this portable build.
Building PL/Perl and PL/Tcl on Windows requires gendef to generate .def export files from perl538.dll and tcl86.dll before creating import libraries. gendef is provided by mingw-w64-x86_64-tools-git.
…deps.sh MSYS2 Perl emits locale warnings on Windows (falls back to "English_United States.1252") which under set -e causes the realpath_existing() call to exit non-zero, firing the ERR trap. Replace with the coreutils 'realpath' command (available on both Linux and MSYS2) and fall back to python3 for macOS where GNU realpath requires Homebrew.
(( qi++ )) evaluates to the old value of qi; on the first iteration (qi=0) the expression is falsy, exit status 1, and set -e silently kills the script before any output is written. (( ++qi )) always evaluates to the new value (≥1), so it is safe under set -e. Affects bundle_macos() and bundle_windows().
libcrypt.so.1 was treated as a universally-present system library and excluded from bundling. On RHEL9+, glibc no longer ships it — it moved to a separate libxcrypt package where the default soname is .so.2, making .so.1 only available via libxcrypt-compat. Removing it from the skip list lets the bundler copy it into lib/ alongside other runtime deps, making the portable build self-contained on modern RHEL/derivatives.
The three shell activation helpers were unreliable: env.sh used set -euo pipefail which leaks strict error mode into the caller's shell when sourced, env.ps1 used $MyInvocation.MyCommand.Path which breaks under dot-sourcing (should be $PSScriptRoot), and all three silently no-op when executed directly instead of sourced. The helpers are unnecessary since patchelf-set RPATHs already make the bundle self-contained without any PATH manipulation.
Co-authored-by: 4mGLn <127629+4mGLn@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR was created automatically.
Updated file:
ci/postgresql-release-config.jsonChanged PostgreSQL majors:
16,17,18