[tnunamak]

The "Lifecycle owner oracle" CI step (the test file scripts/test-scratch/run-command.test.ts, which checks how scripts/test-scratch/run-command.ts, the wrapper that runs a test command inside a private scratch directory, starts, signals and cleans up commands) stopped finishing when GitHub's ubuntu-latest runner label moved to Ubuntu 26.04: two tests failed at their 20 s timeout, and then the step stayed open until someone cancelled the job. This PR fixes both tests and gives the test file a fixture that makes the two mistakes behind the hang fail fast and leave nothing running.

What was wrong

1. Two tests gave the sandboxed command a path it could not see.

scripts/test-sandbox/test-sandbox is the vendored script that runs a command inside bubblewrap (bwrap, a Linux tool that starts a process in a restricted namespace). When bwrap works, the command gets a private, empty /tmp, and the script binds back only the top-level /tmp/<name> directory that contains its --scratch root (the per-run directory the wrapper creates for the command) and the one that contains the working directory. The tests "a sidecar killed right after spawning the command…" and "owner SIGKILL then a command that leaves a background child…" (the owner is the wrapper process; the sidecar is the helper script that starts the command inside the sandbox) put the file their command writes (command.record, background.pid) in a second directory made with mkdtemp (Node's temporary-directory call) under /tmp. Inside the sandbox that directory does not exist, so the command could not create the file and the test timed out at 20 s while polling for it.

Only the new image exposed this. The 26.04 image ships bubblewrap 0.11.1 and it works for the runner user. The 24.04 image has no bwrap, so the sandbox fell back to weaker isolation, /tmp stayed shared and the helper directory was visible.

2. A failed test left a process group alive, and that kept the step open.

These two tests kill an owner or sidecar on purpose, so when an assertion fails nothing stops the command's process group. In the second test the wrapper is started with piped stdout and stderr, and the leftover command inherits those pipes. The test process then cannot exit, so Node's built-in test runner (node --test) never reports. On the runner after the failure, test-sandbox was reparented to PID 1 and bwrap, node …/sandbox-child.ts, sh -c "sleep 300 …" and sleep 300 were all still running; an outer timeout 90 had to kill the run (exit 124). In production CI, the step ran until the job was cancelled.

What this changes

• scripts/test-scratch/fixtures/lifecycle-fixture.ts adds LifecycleFixture, a per-test helper created with lifecycleFixture(t) (t is Node's test context). It owns a unique runner temp directory: the directory the wrapper puts its scratch root in, and so the one directory tree the sandbox binds back. fixture.dir() and fixture.path() return paths only inside it and throw for anything else.
• fixture.spawn() starts a process. Its optional helpers list names files the command will use; if one is outside the runner temp, spawn throws before launching. It also throws if the environment's RUNNER_TEMP (the variable that tells the wrapper where to put its scratch root) is not the fixture's. The check covers only the paths a test declares. It does not read argument strings or sh -c script bodies.
• Teardown, in the test's after hook (the Node test-runner callback that runs once the test ends, pass or fail), closes the pipe ends the fixture opened and kills every process whose environment carries this fixture's unique RUNNER_TEMP value. Every descendant of a process the fixture launched inherits that value, including a command inside the sandbox (checked on this branch with the sandbox on and off). Teardown reads /proc/<pid>/environ, never kills its own process, and scans again because a dying parent may fork once more. It never signals a stored process or group id. It matches each process and signals it in one synchronous step, so a pid read earlier cannot be reassigned while other reads are pending. A small window remains between one read and its kill, the same one Jenkins' ProcessTreeKiller accepts. Where /proc is unreadable it kills nothing and reports that through the test diagnostics.
• fixture.useRunnerTempForThisTest() points this process's RUNNER_TEMP at the runner temp for tests that call runScratchCommand (the wrapper's function form) in-process; calling it twice changes nothing, and teardown restores the previous value.
• Every wrapper launch in run-command.test.ts goes through the fixture; the per-test rm calls and the hand-written group reaper are gone.
• The timeout-minutes: 5 limit on the oracle step stays as a backstop for a hang with some other cause. The file runs in under a minute.

No product or sandbox code changed. The three test files total 2089 lines (2093 at the previous revision).

Evidence

• Original failure: the hung run at 8374fd936b is https://github.com/PDP-Connect/pdpp/actions/runs/42271324505 (ubuntu-26.04, Node 24.22.2). The last green run before the image change is https://github.com/PDP-Connect/pdpp/actions/runs/40720715938 (ubuntu-24.04, e6099bd067). No lifecycle code, workflow or lockfile differed between them.
• A helper outside the runner temp is rejected by a test through fixture.spawn, before any process starts. A helper made with fixture.path is accepted, and a wrapper-launched cat reads it.
• Cleanup after a failed assertion is tested by a child node --test run (fixtures/leaked-group-after-failure.ts, run only by the parent test). It starts a wrapper whose command is a Node program that starts a Node keep-alive child. Each program writes its own pid to a file from inside the running process; the command writes its file only after the child's. The child run waits for both files (20 s deadline), requires both pids alive and the wrapper still running (otherwise it fails with "setup failed"), kills the wrapper's owner (the command and its keep-alive child still hold the wrapper's pipes), then fails an assertion. The parent requires exit code 1 within 60 s, requires the deliberate-failure message and no "setup failed" in the output (so a setup failure cannot stand in for it) and requires both pids to be gone, once with the sandbox enabled and once disabled. With the reaping line disabled, both runs fail, so the test does detect a leak. A keep-alive that exits before or right after it reports ready fails the run as a setup failure.
• The /tmp visibility probe, the test "a helper made with the fixture is readable through the sandboxed wrapper below /tmp", always runs with TEST_SANDBOX_REQUIRE=1, so a host without usable bubblewrap fails it instead of running on the host /tmp. The workflow installs bubblewrap and checks it before the oracle step. Bubblewrap cannot nest, so inside the outer sandbox of the test-accounting run (TEST_SANDBOX_ACTIVE set) the probe is skipped with that reason rather than run without isolation; the lifecycle workflow, which runs outside any sandbox, is its gate.
• A second test launches a sleeping process through the fixture and starts another that the fixture did not launch; teardown kills the first, destroys its pipe, removes the runner temp, and leaves the second running.
• The oracle runs 59/59 and the canonical-entrypoint ratchet (it checks that package scripts and workflows start tests through the scratch wrapper) 11/11 locally on Linux with bubblewrap. CI results for the latest head are in the checks list below.

What I did not verify

• Teardown relies on /proc. On a system without it the fixture kills nothing it did not start by handle and reports it; the leak protection does not apply there. The workflow runs on Ubuntu only.
• A process that clears or replaces its own RUNNER_TEMP is not recognised as the fixture's and is not killed. The timeout-minutes: 5 limit still applies.
• The path check does not look inside sh -c scripts or argument strings, so a script that builds a hidden /tmp/... path itself still fails by timeout, as before.
• 28 other tests still create a directory with mkdtemp directly; they exercise scratch allocation and recovery functions in-process and start no command through the wrapper.
• I did not run the suite on an ubuntu-24.04 image after the fixture change.

Assisted-by: AI

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
pdpp	Ready	Preview	Nov 7, 2026 9:59am UTC