From 7213c8fed22b5e63ca2e7929ca5d8ef1eca9d751 Mon Sep 17 00:00:00 2001 From: Georgy Butaev <41178744+g-but@users.noreply.github.com> Date: Sat, 29 Aug 2026 08:02:14 +0200 Subject: [PATCH] fix(deploy): boot-test cleanup used pkill -f, which cannot match a Next server MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Found 2026-08-29 auditing the box for services bound to 0.0.0.0: a leaked boot-test instance was squatting on port 4099, actively serving 500s from a stale build, with no systemd unit, no owner, alive since an earlier deploy. Root cause, verified with an isolated reproduction (not just reasoned about): Next's standalone server rewrites its own process title to "next-server (v...)" during startup, and — confirmed by reading /proc/PID/cmdline directly, both on the box and in a minimal local repro — that rewrite changes the process's actual argv as seen by /proc, not just what `ps` displays. `pkill -f 'app-next/server.js'` matches against that same argv, so once the server finishes booting (routinely before this line runs, since the health-check loop above can take up to 20s), there is nothing left containing "app-next/server.js" to match. Confirmed both directions in the repro: pkill -f fails to kill a title-rewritten process; fuser -k on the actual bound port kills it every time. This is not cosmetic. A leaked boot-test process on $OC_BOOT_PORT silently threatens every subsequent deploy: the next boot-test's own health check can hit the STALE process instead of the new build, or fail outright on a port collision — either way, a deploy could report success (or a confusing failure) without ever having actually tested the new code. Replaced with `fuser -k "$BOOT/tcp"` — kills whatever is actually listening on the scratch port, independent of what the process calls itself. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01Y9rKLxddothnXEtY6KDziN --- scripts/deploy-selfhost.sh | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/scripts/deploy-selfhost.sh b/scripts/deploy-selfhost.sh index 740e99d0b..4ff7c8958 100755 --- a/scripts/deploy-selfhost.sh +++ b/scripts/deploy-selfhost.sh @@ -150,7 +150,19 @@ for _ in $(seq 1 20); do [ "$code" = "200" ] && { ok=200; break; } read -t 1 _ /dev/null || true done -pkill -f 'app-next/server.js' 2>/dev/null || true +# `pkill -f 'app-next/server.js'` does NOT reliably match the running +# process: Next's standalone server rewrites its own process title to +# "next-server (v...)" during startup, and that rewrite changes +# /proc/PID/cmdline itself, not just what `ps` displays — confirmed +# 2026-08-29 by reading /proc/PID/cmdline directly on a leaked instance. +# Once boot succeeds (usually well before this line runs, since the +# health-check loop above can take up to 20s), pkill -f has nothing left to +# match, and the boot-test process leaks forever, squatting on $BOOT. That +# then silently wedges every FUTURE deploy: the next boot-test binds a +# different port collision or hits the stale process's stale health +# response instead of the new build. Kill by what's actually listening on +# the port instead — immune to whatever the process calls itself. +fuser -k "$BOOT/tcp" 2>/dev/null || true [ "$ok" != "200" ] && { echo "BOOT-TEST FAILED ($ok)"; tail -25 "$LOG"; exit 1; } echo "boot-test: OK"