Skip to content

fix(scaffold): cli.service stop_signal; io.pilot.postgres 17.5.1 runs its server under the app - #122

Draft
TeoSlayer wants to merge 1 commit into
fix/redis-lifecyclefrom
fix/postgres-lifecycle
Draft

TeoSlayer wants to merge 1 commit into
fix/redis-lifecyclefrom
fix/postgres-lifecycle

Conversation

@TeoSlayer

Copy link
Copy Markdown
Contributor

Stacked on #115 (fix/redis-lifecycle, which adds cli.service), and through it on #114 and #111. This PR adds one commit. Once those merge, it retargets to main.

Problem

The harness sweep ran the published io.pilot.postgres 17.5.0 bundles on darwin/arm64, darwin/amd64 (Rosetta), linux/arm64 and linux/amd64 (docker --init, as a non-root user). I reran the published bundle through this PR's harness for a same-harness baseline:

Published 17.5.0 (measured)
Server outlives the app postgres.start runs pg_ctl start, which setsid()s the postmaster out of the adapter's process group. After the app's SIGTERM: LEFT pid=39689 ppid=1 pgid=39689 …/bin/postgres -D … + checkpointer, background writer, walwriter, autovacuum launcher, logical replication launcher (darwin-arm64). The same 6 processes are left after a SIGKILL of the app pid (the supervisor's stop path) and on linux-arm64. On Linux with Pdeathsig, the app dies 0 ms after its parent, but the postmaster (pid=62 ppid=1) and 5 children are left. #114's guard, a group kill and reapStale can't reach it.
Shell injection pg_ctl runs -o "-p ${port} -k ${datadir} -h 127.0.0.1" through /bin/sh. A $(…) in port created a marker file (audit).
Passthrough escape The pg dispatcher only checks [ -x "$dir/$tool" ], so postgres.exec ["../../…/bin/echo", …] runs any host binary (audit).
Long datadir -k ${datadir}: the Unix socket path goes over sun_path (103 bytes) and the start fails: pg_ctl: could not start server with the 117-byte scratch root.
Orphan (template, fixed by #111) darwin: ORPHAN (app 40783 alive 6s later, answering IPC=true), plus the postmaster.

Fix

Template (small): cli.service gains stop_signal: SIGTERM (default), SIGINT or SIGQUIT. It replaces SIGTERM in the adapter's clean stop (StopServices) and in a failed readiness wait; SIGKILL still follows after 8 s. PostgreSQL's SIGTERM is a smart shutdown that waits for every client to disconnect. One open connection would run out the grace and end in a SIGKILL and crash recovery on the next start. Its SIGINT is the fast shutdown with a checkpoint. Validation rejects anything else. Documented in docs/CLI-ADAPTER.md, with a note that a server must not be started through a daemonizing wrapper (pg_ctl start).

submissions/io.pilot.postgres → 17.5.1 (the bundled PostgreSQL is still 17.5, same native assets and shas):

  • postgres.start is a cli.service that runs the postgres binary itself in the foreground: postgres -D ${datadir} -p ${port} -h 127.0.0.1 -c unix_socket_directories= -c logging_collector=on -c log_directory=${datadir} -c log_filename=postgres.log, with ready_tcp 127.0.0.1:${port}, ready_timeout 45s, log_file ${datadir}/postgres.log and stop_signal SIGINT.
    • No pg_ctl and no /bin/sh: port and datadir are plain argv, so the injection is gone. postgres rejects a bad port: FATAL: invalid value for parameter "port".
    • TCP only, so any datadir length works (tested with 247 bytes).
    • The log stays at <datadir>/postgres.log.
    • A busy port fails fast.
  • postgres.exec gets a bare-name cli.tools allowlist of 21 bundled tools. It leaves out pg_ctl, pg_upgrade and pg_createsubscriber, each of which starts a detached postmaster through pg_ctl. The allowlist closes the ../ escape. A passthrough postgres is a service (ready_port_flag -p, stop_signal SIGINT). postgres.stop/postgres.status still use pg_ctl stop/status on their enumerated routes. postgres.stop stops the service, and a later postgres.start works.
  • Descriptions and listing text say that the server runs under the app and has to be started again after the app restarts. They no longer mention the Unix socket or pg_ctl -o for exec.

Tests

  • TestCLIServiceLifetime/stop-signal: a service with stop_signal: SIGINT gets SIGINT and not SIGTERM on the adapter's SIGTERM, and the group is left empty. Mutation check: with StopServices sending SIGTERM again, the subtest fails. Restored, it passes -count=3.
  • TestCLIServiceConfig: stop_signal: SIGINT is ok; SIGKILL is rejected.
  • go vet ./... and go test ./... -count=1 pass on darwin/arm64. internal/scaffold and internal/publish pass on linux/arm64 (golang:1.25.13-bookworm, non-root).
  • pilot-app verify-update submissions/io.pilot.postgres/submission.json: UPDATE GATE OK (17.5.1 ≥ 17.5.0). demo-score: postgres 100.0 ok.

Verified on real postgres bundles built from this commit

I built all 4 DefaultPlatforms with publish.BuildBundle from submissions/io.pilot.postgres/submission.json at 17.5.1 and signed them with a TEST-ONLY key; nothing was published or uploaded. pilot-app verify passes on each tarball (VERIFY OK), and the adapter binaries are Mach-O arm64/x86_64 and static ELF aarch64/x86-64. A lifecycle harness drove the real staged PostgreSQL 17.5 from R2, spawning the app with the supervisor's exact flags in its own process group. On Linux it runs in docker --init as node, with Pdeathsig from a locked thread like the supervisor.

scenario darwin-arm64 darwin-amd64 (Rosetta) linux-arm64 linux-amd64 (emulated)
start → query → app SIGTERM exit 0 in 26 ms, 0 left, log: received fast shutdown request … database system is shut down exit 0 in 73 ms, 0 left exit 0, 0 left exit 0 in 103 ms, 0 left
SIGKILL of the app pid (supervisor stop) 0 left after 73 ms 0 left after 121 ms 0 left (Pdeathsig on and off) 0 left (Pdeathsig on and off)
supervisor SIGKILLed SELF-EXIT 197 ms, 0 left SELF-EXIT 155 ms, 0 left Pdeathsig: 12 ms; without: SELF-EXIT 35 ms; 0 left Pdeathsig: 0 ms; without: SELF-EXIT 509 ms; 0 left
postgres.stop, then postgres.start again postmaster gone, restart ok, 0 left ok ok ok
passthrough postgres.exec ["postgres",…,"-p",P], then app SIGKILL serves select 7; 0 left 0 left 0 left 0 left
probes: $(touch …) in port, ../…/bin/echo, pg_ctl, sh, ./psql; busy port no marker; all 5 refused; busy port refused same same same
247-byte datadir starts, 0 left ok ok ok
postgres.query_csv ×200 200/200, fd 9→9 200/200, fd 9→9 200/200, fd 11→11 200/200, fd 19→18

The same harness on the published 17.5.0 (darwin-arm64 and linux-arm64): SIGTERM, SIGKILL of the pid, and supervisor death each leave the postmaster plus 5 children (LEAK). darwin's supervisor-death case is also ORPHAN.

Not in this PR

  • The native assets are unchanged, so the /tmp/pilot-pg/17.5.0 symlink remains. The conda-built binaries don't relocate. With no link, initdb fails with file "/private/tmp/pilot-pg/17.5.0/share/postgres.bki" does not exist (darwin-arm64, reproduced here), so the pg dispatcher keeps creating that shared link in world-writable /tmp. It's never removed. Concurrent installs re-point it, and on a multi-user host a second user's server reads share/ and loads $libdir modules from whichever tree the link points at. The fix is new, relocatable PostgreSQL 17.5 assets built from upstream source, plus a dispatcher without the link. That needs an R2 upload, so it's tracked separately (see the follow-up comment on this PR).
  • Behaviour change: the local server now stops whenever the app stops (daemon restart, upgrade, crash, uninstall), so a caller needs postgres.start again. The method description says so.
  • Shipping 17.5.1 needs the app's publisher key (ed25519:N1uQkAJ3…) through publish-server, and the catalogue PR.

🤖 Generated with Claude Code

…res 17.5.1 runs its server under the app

postgres.start ran pg_ctl, which setsid()s the postmaster out of the
adapter's process group: it outlived the app's SIGTERM, the supervisor's
SIGKILL and daemon death on all 4 platforms. It is now a cli.service that
runs the postgres binary itself in the foreground (no pg_ctl, no /bin/sh,
so the port/datadir params can no longer reach a shell), TCP only (no Unix
socket, so a long datadir no longer hits the sun_path limit).

cli.service gains stop_signal (SIGTERM default, SIGINT, SIGQUIT):
PostgreSQL's SIGTERM is a smart shutdown that waits for every client;
SIGINT is its fast, clean one.

postgres.exec gets a bare-name tools allowlist without pg_ctl, pg_upgrade
and pg_createsubscriber (each daemonizes a postmaster), which also closes
the '../' escape to any host binary; a passthrough postgres is a service.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@TeoSlayer
TeoSlayer marked this pull request as draft October 1, 2026 14:31
@TeoSlayer

Copy link
Copy Markdown
Contributor Author

Converted to draft rather than merged. It depends on draft #115 (service.go.tmpl is not on main) and the postgres 17.5.1 bundles are not in R2 yet, so publish-on-merge would fail. The code itself reviewed fine.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants