diff --git a/docs/PLAN.md b/docs/PLAN.md index 9a03a3a..0d22b5b 100644 --- a/docs/PLAN.md +++ b/docs/PLAN.md @@ -159,7 +159,7 @@ An opted-in AI Gateway key (the NAME `AI_GATEWAY_API_KEY`) is never handed to th - **What a shutdown does:** the pass ends over budget, so the stage Blocks there, and the relay refuses every later call, the model list included. The dollars are marked a lower bound as well, since a run of calls none of which could be priced gives no evidence the estimates cover them. The Block reason and the progress line name it first (`gateway relay shut: 7 calls in a row could not be priced (); Factory's max_usd was not reached`, `gateway relay shut: 6 calls in this stage could not be priced ()`, or `gateway relay shut: a complete answer could not be priced ()`); only an actual `max_usd` overrun reads "dollar cap reached". Nothing has to persist across stages: the Block stops the Objective until a person re-applies it. - **The call log:** each call the relay answers is one JSON line in the stage's usage directory (`relay-calls.jsonl`, which no sandbox writes, mode 600): its time, method, relayed path, model (the allowlisted model header, else a JSON body's or a form's `model`, kept only with a model id's shape: `provider/name`, at most 80 characters), status, the dollars booked and how (`reported`, `lookup`, `refused`, `unsent`, `redirect`, `unavailable`, `gateway-refused`, `estimate`, `shut`), its generation id, why it was not priced, or why the relay refused it (a route it does not relay is `refused: route`, never the path asked for; a call whose client hung up while it waited is `refused: gone`, never sent). Never a body, a header's value, a message or the key, and the stage's held secrets are masked in it when the stage ends, as in `events.jsonl`. At most 2,000 lines; one closing line counts the calls past them. A lookup that settles later adds a line of its own, with `reconciled`, the dollars it changed the booking by: one that raises a reported cost, one that prices a refusal's generation, and the one for a failed call booked at the estimate (its record's cost, or 0 when no record came and the estimate stands). `result.json`'s `workerKey` summarises it: the estimate (`estimated`, `estimatedUsd`), the shutdown (`shut`: calls, why, and `complete` for a complete answer) and `callLog` (the file, its line count and any calls left out). - **The cap is live and shared with the harness:** calls are relayed only while a pass runs. With a cap, billable calls go one at a time (at most 8 wait their turn, more get 429 with a `Retry-After`; a pass's end refuses those still waiting, and one whose pass ended while its turn came; one whose client hung up while it waited is never sent), each only once the one before it is priced or booked at its estimate, and only while the pass's spend through the key plus the harness's own spend in the pass so far is under what `max_usd` left when the pass started; past it the relay answers 402 with a message the worker sees, and the pass counts as over budget, so the stage stops or Blocks as when the harness reaches its own cap. Model spend plus key spend can pass the cap by about two calls and what the standing estimates understate (the money bound above), and by what the harness spends after the check. The harness's own spend is read live (`lib/live-spend.mjs`): Claude Code's per-message usage from its stream, priced from `lib/claude-prices.json` (an unlisted model at the dearest row); Codex's newest thread rollout (its `token_usage_record` lines, one per model response) and its turns, priced from the Codex table (an unset or unlisted model at its dearest rates), sub-agent threads once the pass ends; Copilot's usage checkpoints (its running AI credits). A live figure that cannot be read refuses the call (503, with a `Retry-After`) rather than count as $0. A parallel pipeline is serialised while a cap is on: N concurrent calls take about N times one call's time. With no cap nothing is serialised, and each call of unknown cost still books its own estimate. -- **How commands reach it:** the relay listens on a Unix socket in a directory the agent cannot write (a directory of the stage's own under `/var/lib/factory-relay` on a runner and in a cloud sandbox, the socket group-readable by the agent at 660, removed when the stage ends). Locally, and in a developer's own session, the socket goes in a short directory of the stage's own, `/tmp/factory-relay-/<12 hex>`, named by a hash of the stage's usage directory: both levels 700, the socket 600, removed when the stage ends. A Unix socket's path is limited to 107 bytes on Linux (103 on macOS), and the usage directory's path grows with the checkout's. A parent that is a link, a file, another user's or open to others is refused, and a socket path that would still pass the limit Blocks the stage before the run, naming the limit. Claude Code's and Codex's sandboxes start every command with the `bwrap` first on PATH. Factory puts its shim there (`lib/relay-shim.mjs`, `bash -p`, builtins only), which starts a forwarder on the command's own network namespace's `127.0.0.1:` to the socket before the command runs; it walks bubblewrap's options by their arity, so only `--unshare-net` before the command's separator marks a command sandbox. The harness's network policy, proxy and seccomp filter are unchanged; the forwarder carries one port to the relay alone, and the commands themselves cannot open the socket (both harnesses' filters refuse Unix sockets). Copilot's own bwrap shim (`lib/copilot-runner.mjs` `bwrapShim`, already first on its PATH) hands each call to the relay's shim for Copilot, at a path in the stage's own relay directory written into Copilot's shim for the stage (on a runner and in a cloud sandbox once the relay opens, locally by `serve`), never read from an environment; concurrent stages on one host each hand to their own, and a Copilot stage removes the one shared path an earlier Factory used. For a Copilot command sandbox that shim also binds the socket's directory read-only and sets `NO_PROXY`/`no_proxy` to `127.0.0.1`, so loopback stays off Copilot's proxy and reaches the namespace's own forwarder. `allowLocalNetwork` stays false and Copilot's host allow-list is unchanged. Copilot's filter lets commands open a Unix socket, so a Copilot command can reach the relay's socket directly too: it is the same relay behind the same token check. The worker needs no egress to the gateway's host. Locally the outer sandbox keeps the socket's directory reachable, never writable; it is the one socket directory it does not hide. +- **How commands reach it:** the relay listens on a Unix socket in a directory the agent cannot write (a directory of the stage's own under `/var/lib/factory-relay` on a runner and in a cloud sandbox, the socket group-readable by the agent at 660, removed when the stage ends). Locally, and in a developer's own session, the socket goes in a short directory of the stage's own, `/tmp/factory-relay-/<12 hex>`, named by a hash of the stage's usage directory: both levels 700, the socket 600. A Unix socket's path is limited to 107 bytes on Linux (103 on macOS), and the usage directory's path grows with the checkout's. `/tmp` (its real path) must be root's with the sticky bit, which alone keeps another user from swapping this user's directory there between Factory's checks and its listen; otherwise the stage Blocks before the run, naming why. A per-user directory that is a link, a file, another user's or open to others is refused, and the stage's socket goes in a private directory instead, `/tmp/factory-relay--XXXXXX` (made by `mkdtemp`, 700). Locally `serve` makes the directory before the stage starts, records it in the stage's usage directory, names it in the outer sandbox and hands it to the stage, which takes no other; a stage that relays nothing has no socket directory reachable. After the listen the stage confirms the socket is in the very directory it checked, or Blocks before any command starts. The directory is removed when the stage ends; one a dead stage left, `serve` removes when it reaps the stage, and only if it still passes those checks at every level. A socket path that would still pass the limit Blocks the stage before the run, naming the limit. Claude Code's and Codex's sandboxes start every command with the `bwrap` first on PATH. Factory puts its shim there (`lib/relay-shim.mjs`, `bash -p`, builtins only), which starts a forwarder on the command's own network namespace's `127.0.0.1:` to the socket before the command runs; it walks bubblewrap's options by their arity, so only `--unshare-net` before the command's separator marks a command sandbox. The harness's network policy, proxy and seccomp filter are unchanged; the forwarder carries one port to the relay alone, and the commands themselves cannot open the socket (both harnesses' filters refuse Unix sockets). Copilot's own bwrap shim (`lib/copilot-runner.mjs` `bwrapShim`, already first on its PATH) hands each call to the relay's shim for Copilot, at a path in the stage's own relay directory written into Copilot's shim for the stage (on a runner and in a cloud sandbox once the relay opens, locally by `serve`), never read from an environment; concurrent stages on one host each hand to their own, and a Copilot stage removes the one shared path an earlier Factory used. For a Copilot command sandbox that shim also binds the socket's directory read-only and sets `NO_PROXY`/`no_proxy` to `127.0.0.1`, so loopback stays off Copilot's proxy and reaches the namespace's own forwarder. `allowLocalNetwork` stays false and Copilot's host allow-list is unchanged. Copilot's filter lets commands open a Unix socket, so a Copilot command can reach the relay's socket directly too: it is the same relay behind the same token check. The worker needs no egress to the gateway's host. Locally the outer sandbox keeps the socket's directory reachable, never writable; it is the one socket directory it does not hide. - **Probes and fail-closed starts:** before a stage's first relayed Codex pass, Codex's own sandbox (`codex sandbox`, the run's profile, the shim on PATH) must reach the relay's ping, or the stage Blocks (`relay`) before the run. Copilot's probe before every start (Copilot's sandbox, above) gets a rule of its own: a command in Copilot's sandbox reaches this relay's ping through the forwarder, or the stage Blocks (`relay`). Each command sandbox's forwarder pings the relay when it comes up, and a pass whose commands ran with no ping is logged. On a machine without `bwrap` or `socat` the stage Blocks before any run, with the key out of the shell. Factory never falls back to handing the key over. - **Other opted-in secrets** keep the behaviour above: they reach the worker's shell as they are, and rely on their provider's own cap. diff --git a/plugin/scripts/lib/harness.mjs b/plugin/scripts/lib/harness.mjs index 6c03e02..52b91be 100644 --- a/plugin/scripts/lib/harness.mjs +++ b/plugin/scripts/lib/harness.mjs @@ -57,7 +57,7 @@ export const hiddenFromShell = (shellSecrets = []) => SHELL_SECRET_KEYS.filter(( * acts as, its progress comment's key, its read token, local mode's host-only settings. Each is read by the process it was * given to (from process.env), never by a child, so scrubbedEnv removes them from every child. */ -export const FACTORY_CONTROL_KEYS = ["FACTORY_STAGE", "FACTORY_STAGE_ISSUE", "FACTORY_STAGE_CLAIM", "FACTORY_STAGE_PR", "FACTORY_STAGE_RECORD", "FACTORY_STAGE_COMMAND_MS", "FACTORY_STAGE_SESSION_END", "FACTORY_STAGE_SESSION_CAP", "FACTORY_APP_SLUG", "FACTORY_PROGRESS_RUN", "FACTORY_READ_TOKEN", "FACTORY_IMAGE_CHECKOUT", "FACTORY_IMAGE_ENV_FILE", "FACTORY_ADAPTER_INSTALL", "FACTORY_SANDBOX_VCPUS", "FACTORY_LOCAL_FIRST_SETUP_MS", "FACTORY_LOCAL_APP_ID", "FACTORY_LOCAL_APP_KEY", "FACTORY_GATEWAY_URL", "FACTORY_STATE_DIR"]; +export const FACTORY_CONTROL_KEYS = ["FACTORY_STAGE", "FACTORY_STAGE_ISSUE", "FACTORY_STAGE_CLAIM", "FACTORY_STAGE_PR", "FACTORY_STAGE_RECORD", "FACTORY_STAGE_COMMAND_MS", "FACTORY_STAGE_SESSION_END", "FACTORY_STAGE_SESSION_CAP", "FACTORY_APP_SLUG", "FACTORY_PROGRESS_RUN", "FACTORY_READ_TOKEN", "FACTORY_IMAGE_CHECKOUT", "FACTORY_IMAGE_ENV_FILE", "FACTORY_ADAPTER_INSTALL", "FACTORY_SANDBOX_VCPUS", "FACTORY_LOCAL_FIRST_SETUP_MS", "FACTORY_LOCAL_APP_ID", "FACTORY_LOCAL_APP_KEY", "FACTORY_GATEWAY_URL", "FACTORY_STATE_DIR", "FACTORY_RELAY_SOCKET_DIR"]; /** * Factory's internal names that do reach children, on purpose: the hand-off and the task card's (FACTORY_IN, FACTORY_OUT, diff --git a/plugin/scripts/lib/local.mjs b/plugin/scripts/lib/local.mjs index 150d668..74e8019 100644 --- a/plugin/scripts/lib/local.mjs +++ b/plugin/scripts/lib/local.mjs @@ -1518,8 +1518,9 @@ export function hostSockets(env = {}, uid = process.getuid?.() ?? 0, { roots = S * `projectDir`: the directory claudeProjectLink links the clone's Claude Code project to, writable here (Claude Code only). * `bwrapPath`: the bubblewrap srt runs (Copilot: writeBwrapTun's wrapper). * `ripgrep`: nestedGitScanScript's path (writeNestedGitScan). - * `relaySocketDir`: the directory of the stage's gateway relay socket (worker-relay.mjs relaySocketDir): the one socket - * directory that stays reachable inside, for the forwarder Factory's bwrap shim starts in each command's sandbox + * `relaySocketDir`: the directory of the stage's gateway relay socket, as serve made it (worker-relay.mjs handSocketDir; + * null for a stage that relays nothing, which then reaches no socket directory at all): the one socket directory that + * stays reachable inside, for the forwarder Factory's bwrap shim starts in each command's sandbox * (relay-shim.mjs); never writable. */ export function harnessOuterSettings({ harness, domains = LOCAL_DOMAINS, writable = [], clone = null, home = homedir(), env = {}, readOnly = [], install = [], projectDir = null, bwrapPath = null, relaySocketDir = null, ripgrep = null } = {}) { diff --git a/plugin/scripts/lib/worker-relay.mjs b/plugin/scripts/lib/worker-relay.mjs index d904985..d9ff4d0 100644 --- a/plugin/scripts/lib/worker-relay.mjs +++ b/plugin/scripts/lib/worker-relay.mjs @@ -3,14 +3,14 @@ // place and the relay's address in FACTORY_GATEWAY_URL (harness.mjs). Never falls back to handing the key over: where the // relay cannot reach the harness's commands, the stage Blocks before the run. import { createHash, randomBytes } from "node:crypto"; -import { existsSync, lstatSync, mkdirSync, realpathSync, rmSync } from "node:fs"; +import { existsSync, lstatSync, mkdirSync, mkdtempSync, readFileSync, realpathSync, rmSync, statSync, writeFileSync } from "node:fs"; import { dirname, join, resolve } from "node:path"; import { backend } from "./backend.mjs"; import { prepareFactoryHome } from "./claude-runner.mjs"; import { CALL_LOG, GATEWAY_KEY_SHAPE, RELAYED_NAME, openRelay, setActiveRelay } from "./gateway-relay.mjs"; import { nameLabel } from "./worker-secrets.mjs"; import { holdSecrets, log } from "./inputs.mjs"; -import { SAFE_PATH, agentUser, exec, whichSync } from "./proc.mjs"; +import { SAFE_PATH, agentUser, exec, processRecord, whichSync } from "./proc.mjs"; import { copilotRelayExtra, copilotRelayShimIn, localCopilotRelayShim, writeRelayShim } from "./relay-shim.mjs"; import { COPILOT_RUNNER_HOME, bakeCopilotRelay, copilotRunnerLayout } from "./copilot-runner.mjs"; @@ -36,9 +36,9 @@ export const localRelayDir = (usageDir) => join(usageDir, "relay"); * Where the relay's socket goes when its directory is under the stage's usage directory (locally, and in a developer's * own session): a short directory of the stage's own, `/tmp/factory-relay-/<12 hex>`. A Unix socket's path is * limited (107 bytes on Linux, 103 on macOS) and the usage directory's grows with the checkout's path, the issue and the - * stage's name. Its name is a hash of the usage directory, so local mode's outer sandbox, written before the relay opens, - * names it as the one socket directory it keeps reachable (local.mjs harnessOuterSettings), as it named the relay - * directory before. No sandbox writes /tmp: the outer one writes only its allowed paths, and the harnesses' own write + * stage's name. Its name is a hash of the usage directory. Locally the loop makes it (handSocketDir) before it writes + * local mode's outer sandbox, which names it as the one socket directory it keeps reachable (local.mjs + * harnessOuterSettings). No sandbox writes /tmp: the outer one writes only its allowed paths, and the harnesses' own write * the checkout and the stage's temp directory. */ export const RELAY_SOCKET_ROOT = "/tmp"; @@ -53,6 +53,33 @@ export function relaySocketDir(usageDir, { uid = process.getuid?.() ?? 0, root = return join(base, `factory-relay-${uid}`, createHash("sha256").update(resolve(usageDir)).digest("hex").slice(0, 12)); } +/** The env name that carries the socket's directory the loop made for a local stage (handSocketDir) to the stage. */ +export const RELAY_SOCKET_DIR_ENV = "FACTORY_RELAY_SOCKET_DIR"; +/** The file in a stage's usage directory naming the socket's directory the loop made for it, and the loop (handSocketDir). */ +export const SOCKET_DIR_RECORD = "relay-socket.json"; + +/** + * The directory the socket's directories go in (RELAY_SOCKET_ROOT), resolved: refused (throws, naming why) unless it is + * a directory owned by root (`rootUid`; a test's stand-in its own) with the sticky bit. Every user writes it, and only the + * sticky bit keeps another from renaming this user's directory there between Factory's checks and its `listen`, which + * would put the socket, and the commands' calls with their token, in a directory of theirs. + */ +export function checkSocketRoot(root = RELAY_SOCKET_ROOT, { rootUid = 0 } = {}) { + let real; + let st; + try { + real = realpathSync(root); + st = statSync(real); + } catch (e) { + throw new Error(`${root} could not be read (${e?.code || e?.message || e})`); + } + const owner = rootUid === 0 ? "root" : `uid ${rootUid}`; + if (!st.isDirectory()) throw new Error(`${real} is not a directory`); + if (st.uid !== rootUid) throw new Error(`${real} is not owned by ${owner}, so nothing keeps another user from swapping the socket's directory there`); + if (!(st.mode & 0o1000)) throw new Error(`${real} has no sticky bit (mode ${(st.mode & 0o7777).toString(8)}), so another user could swap the socket's directory there for one of theirs`); + return real; +} + /** * Make relaySocketDir's directory and its parent, each a real directory of this user's alone (mode 700): the parent is * in a directory every user writes, so one that is a link, a file, another user's or open to others is refused (throws, @@ -65,14 +92,135 @@ export function makeSocketDir(dir, { uid = process.getuid?.() ?? 0 } = {}) { } catch (e) { if (e?.code !== "EEXIST") throw new Error(`${d} could not be made (${e?.code || e?.message || e})`); } - const st = lstatSync(d); - if (st.isSymbolicLink() || !st.isDirectory()) throw new Error(`${d} is not a directory (a link or a file stands there)`); - if (st.uid !== uid) throw new Error(`${d} belongs to another user`); - if (st.mode & 0o077) throw new Error(`${d} is open to other users (mode ${(st.mode & 0o777).toString(8)})`); + checkOwnDir(d, uid); } return dir; } +function checkOwnDir(d, uid) { + const st = lstatSync(d); + if (st.isSymbolicLink() || !st.isDirectory()) throw new Error(`${d} is not a directory (a link or a file stands there)`); + if (st.uid !== uid) throw new Error(`${d} belongs to another user`); + if (st.mode & 0o077) throw new Error(`${d} is open to other users (mode ${(st.mode & 0o777).toString(8)})`); + return st; +} + +/** + * The levels of a socket's directory under the resolved root `real`: [`factory-relay-`, `<12 hex>`] for + * relaySocketDir's, [`factory-relay--XXXXXX`] for the private fallback (stageSocketDir); null for any other path. + */ +function socketDirLevels(dir, real, uid) { + const own = `${real}/factory-relay-${uid}`; + if (dir.startsWith(`${own}/`) && /^[0-9a-f]{12}$/.test(dir.slice(own.length + 1))) return [own, dir]; + if (dir.startsWith(`${own}-`) && /^[A-Za-z0-9]{6}$/.test(dir.slice(own.length + 1))) return [dir]; + return null; +} + +/** + * Check a socket's directory before the relay listens in it: the root is root's and sticky (checkSocketRoot), the path is + * one of the two this user's socket directories take there (socketDirLevels), and each level is a real directory of this + * user's alone. Returns {dir, dev, ino}, which confirmSocketDir holds the socket's directory to after the listen. + */ +export function checkSocketDir(dir, { uid = process.getuid?.() ?? 0, root = RELAY_SOCKET_ROOT, rootUid = 0 } = {}) { + const real = checkSocketRoot(root, { rootUid }); + const levels = socketDirLevels(String(dir || ""), real, uid); + if (!levels) throw new Error(`${dir} is not a socket directory of this user's under ${real}`); + let st = null; + for (const d of levels) { + try { + st = checkOwnDir(d, uid); + } catch (e) { + if (e?.code) throw new Error(`${d} could not be read (${e.code})`); + throw e; + } + } + return { dir, dev: st.dev, ino: st.ino }; +} + +/** + * After the listen: the socket is in the very directory checkSocketDir checked (`checked`), which still passes its + * checks: the same directory (device and inode), the socket's real directory, and the socket itself this user's. + * Throws, naming what differs. + */ +export function confirmSocketDir(socketPath, checked, opts = {}) { + const uid = opts.uid ?? process.getuid?.() ?? 0; + const now = checkSocketDir(checked.dir, opts); + if (now.dev !== checked.dev || now.ino !== checked.ino) throw new Error(`${checked.dir} was replaced after it was checked`); + let real = null; + try { + real = realpathSync(dirname(socketPath)); + } catch { + // named below + } + if (real !== checked.dir) throw new Error(`the socket's directory is ${real ?? "gone"}, not ${checked.dir}`); + const st = lstatSync(socketPath, { throwIfNoEntry: false }); + if (!st?.isSocket() || st.uid !== uid) throw new Error(`no socket of this user's is at ${socketPath} after the listen (its directory was swapped while the socket was made)`); +} + +/** + * The socket's directory for a stage: relaySocketDir's, made by makeSocketDir; when that is refused (another user made + * `factory-relay-` first, or it fails a check) and `fallback`, a private one of the stage's own made by mkdtemp in the + * root instead (`factory-relay--XXXXXX`, mode 700, a name no one can take first). Returns {dir, refused}: `refused` + * the per-user directory's refusal when the fallback was taken. Throws when the root is refused (checkSocketRoot), which + * no fallback there escapes, or the per-user directory is refused without `fallback`. + */ +export function stageSocketDir(usageDir, { uid = process.getuid?.() ?? 0, root = RELAY_SOCKET_ROOT, rootUid = 0, fallback = true } = {}) { + const real = checkSocketRoot(root, { rootUid }); + try { + return { dir: makeSocketDir(relaySocketDir(usageDir, { uid, root: real }), { uid }), refused: null }; + } catch (e) { + if (!fallback) throw e; + return { dir: mkdtempSync(join(real, `factory-relay-${uid}-`)), refused: String(e?.message || e) }; + } +} + +/** + * Local mode's loop, before a stage that relays the gateway key starts: its socket's directory (stageSocketDir), recorded + * in the stage's usage directory (SOCKET_DIR_RECORD, with this process as its owner) so a later loop removes it when the + * stage died (local.mjs reapSocketDirs). The loop names it in the outer sandbox and hands it to the stage (RELAY_SOCKET_DIR_ENV). + */ +export function handSocketDir(usageDir, opts = {}) { + const made = stageSocketDir(usageDir, opts); + try { + writeFileSync(join(usageDir, SOCKET_DIR_RECORD), JSON.stringify({ dir: made.dir, owner: processRecord() }), { mode: 0o600 }); + } catch (e) { + // Unrecorded, no later loop could find it: it goes now. + rmSync(made.dir, { recursive: true, force: true }); + throw e; + } + return made; +} + +/** The record handSocketDir left in `usageDir`, or null. */ +export function readSocketDirRecord(usageDir) { + try { + const rec = JSON.parse(readFileSync(join(usageDir, SOCKET_DIR_RECORD), "utf8")); + return typeof rec?.dir === "string" ? rec : null; + } catch { + return null; + } +} + +/** + * Remove a stage's socket directory and its record (handSocketDir) from `usageDir`: only one that passes checkSocketDir + * (the root root's and sticky, a path of the two shapes this user's socket directories take there, and every level a + * real directory of this user's alone, so no link anywhere on the way), and only the stage's own level (the per-user + * parent stays). Returns whether a directory was removed. + */ +export function dropSocketDir(usageDir, { uid = process.getuid?.() ?? 0, root = RELAY_SOCKET_ROOT, rootUid = 0 } = {}) { + const rec = readSocketDirRecord(usageDir); + rmSync(join(usageDir, SOCKET_DIR_RECORD), { force: true }); + if (!rec) return false; + try { + checkSocketDir(rec.dir, { uid, root, rootUid }); + } catch { + // Gone already, or no longer this user's own all the way down (a reboot cleared /tmp and someone planted a link). + return false; + } + rmSync(rec.dir, { recursive: true, force: true }); + return true; +} + /** * The worker secrets an AI Gateway key must not reach as they are: any but RELAYED_NAME itself whose NAME is it in * another case, or whose value has a gateway key's shape (GATEWAY_KEY_SHAPE), whatever its NAME. Such a key is only ever @@ -85,9 +233,10 @@ export const unrelayedGatewayKeys = (shell = {}) => Object.entries(shell).filter * {shell, relay, blocked, masked}: `shell` with the token in the key's place and any other gateway-shaped key withheld * (unrelayedGatewayKeys); `blocked` a reason when the key cannot be relayed here (the key is then out of `shell`, and the * caller Blocks before any run); `masked` the values the stage masks in what it saves (the key, which `shell` no longer - * holds). `socketRoot`: where relaySocketDir's directory goes (RELAY_SOCKET_ROOT; a test sets its own). + * holds). `socketRoot`: where the socket's directory goes (RELAY_SOCKET_ROOT), `socketRootUid` its owner (root; a test's + * stand-in its own). */ -export async function openWorkerRelay({ inputs, shell, usageDir, env = process.env, open = openRelay, which = whichSync, runnerDir = RELAY_RUNNER_DIR, copilotHome = COPILOT_RUNNER_HOME, socketRoot = RELAY_SOCKET_ROOT }) { +export async function openWorkerRelay({ inputs, shell, usageDir, env = process.env, open = openRelay, which = whichSync, runnerDir = RELAY_RUNNER_DIR, copilotHome = COPILOT_RUNNER_HOME, socketRoot = RELAY_SOCKET_ROOT, socketRootUid = 0 }) { if (inputs.harness === "copilot" && agentUser() && backend(env).name !== "local") dropSharedCopilotRelay(runnerDir); const withheld = unrelayedGatewayKeys(shell || {}); const masked = withheld.map((n) => shell[n]); @@ -124,13 +273,30 @@ export async function openWorkerRelay({ inputs, shell, usageDir, env = process.e if (r.code !== 0) throw new Error(`the relay's socket could not be handed to ${g}'s group: ${String(r.stderr).trim().split("\n").at(-1)?.slice(0, 160) || `exit ${r.code}`}`); }; } else dir = local ? localRelayDir(usageDir) : join(usageDir, "relay"); - // Under the usage directory the socket's path could pass its limit: it goes in a short directory of its own. + // Under the usage directory the socket's path could pass its limit: it goes in a short directory of its own. Locally + // the loop made it and named it in the outer sandbox (handSocketDir); elsewhere the stage makes it, falling back to a + // private one when the per-user directory is refused. Checked before the listen and confirmed after it. let socketDir = null; + let checked = null; + // The directory this process made, which it removes when the relay does not start (the loop removes the one it made). + let made = null; + const dirOpts = { root: socketRoot, rootUid: socketRootUid }; + const dirBlocked = (e) => out({ shell: rest, relay: null, blocked: `the gateway relay for ${RELAYED_NAME} could not start: its socket's directory: ${String(e?.message || e).slice(0, 200)}` }); if (!(agent && !local)) { try { - socketDir = makeSocketDir(relaySocketDir(usageDir, { root: socketRoot })); + const handed = local ? env[RELAY_SOCKET_DIR_ENV] || "" : ""; + // Only the one serve made for this stage, as its record in the usage directory (which no sandbox writes) names. + if (handed && handed !== readSocketDirRecord(usageDir)?.dir) throw new Error(`${handed} is not the directory made for this stage`); + if (handed) socketDir = handed; + else { + const m = stageSocketDir(usageDir, { ...dirOpts, fallback: !local }); + made = socketDir = m.dir; + if (m.refused) log(`gateway relay: ${m.refused}; the socket goes in a private directory of this stage's instead`); + } + checked = checkSocketDir(socketDir, dirOpts); } catch (e) { - return out({ shell: rest, relay: null, blocked: `the gateway relay for ${RELAYED_NAME} could not start: its socket's directory: ${String(e?.message || e).slice(0, 200)}` }); + if (made) rmSync(made, { recursive: true, force: true }); + return dirBlocked(e); } } let relay; @@ -140,6 +306,17 @@ export async function openWorkerRelay({ inputs, shell, usageDir, env = process.e if (socketDir) rmSync(socketDir, { recursive: true, force: true }); return out({ shell: rest, relay: null, blocked: `the gateway relay for ${RELAYED_NAME} could not start: ${String(e?.message || e).slice(0, 240)}` }); } + if (checked) { + try { + confirmSocketDir(relay.socketPath, checked, dirOpts); + } catch (e) { + // Nothing was handed out yet: the relay goes before any command could reach it. + await relay.close().catch(() => {}); + rmSync(dir, { recursive: true, force: true }); + if (made) rmSync(made, { recursive: true, force: true }); + return dirBlocked(e); + } + } // Claude Code and Codex: the relay's shim first on their PATH. Copilot: its own shim (copilot-runner.mjs bwrapShim), // already first on its PATH, hands each call to the relay's, which adds Copilot's extra options (copilotRelayExtra). if (sandboxed && inputs.harness === "copilot") { diff --git a/plugin/scripts/local.mjs b/plugin/scripts/local.mjs index 3c7150e..11fe623 100644 --- a/plugin/scripts/local.mjs +++ b/plugin/scripts/local.mjs @@ -26,8 +26,9 @@ import { dispatch } from "./dispatch.mjs"; import { PLUGIN_ROOT } from "./lib/checkout.mjs"; import { activeWorkIssues, addLabels, ensureLabel, hasMergeQueue, findPr, findProject, getIssue, getPr, getPrSettled, listComments, listIssuesByLabel, moveCard, postReview, prComment, prDiff, primeSelf, primeWriters, removeLabel, repoOf, rest, setDraft, setStateLabel, transientError } from "./lib/github.mjs"; import { holdSecrets, input, log, redactSecrets, resolveInputs, workflowDriftLines, workflowSettings } from "./lib/inputs.mjs"; -import { localSecretValues, parseSecretValues } from "./lib/worker-secrets.mjs"; -import { localRelayDir, relaySocketDir } from "./lib/worker-relay.mjs"; +import { localSecretValues, parseSecretValues, parseWorkerSecrets } from "./lib/worker-secrets.mjs"; +import { RELAY_SOCKET_DIR_ENV, dropSocketDir, handSocketDir, localRelayDir, readSocketDirRecord } from "./lib/worker-relay.mjs"; +import { RELAYED_NAME } from "./lib/gateway-relay.mjs"; import { localCopilotRelayShim } from "./lib/relay-shim.mjs"; import { GROUP_RECORD, STOP_SIGNALS, exec, processRecord, reapRecorded, sameProcess, whichSync } from "./lib/proc.mjs"; import { HEARTBEAT_MS, LOCAL_DOMAINS, LOCAL_LABEL, claudeProjectLink, codexLocalConfig, copilotHomeFiles, copilotLocalSettings, copilotPrivateTmp, dropClaim, dropClaudeProjectLink, ensureHarnessHomes, gitDirChanged, gitIdentityKey, gitIdentityReason, gitPinFile, pinGitDir, readGitPin, harnessInstall, writeNestedGitScan, gitGuardScript, harnessOuterSettings, outerProbe, sandboxDroppedHosts, gatewayKeyProblem, localApp, localBaseEnv, localClaim, prepareClone, committedWorkflowEnv, readWorkflowEnv, staleGitLocks, stageTemp, sandboxStartProblems, srtSettings, stopHold, workflowEnv, writeBwrapTun, writeClaim } from "./lib/local.mjs"; @@ -344,6 +345,38 @@ export function reapLeftRuns(ws) { /** The file in a stage's usage directory that names its process while it runs (proc.mjs processRecord). */ export const STAGE_RECORD = "stage-process.json"; +/** Whether a stage running `script` with this env relays the gateway key: the work and review stages that opt it in. */ +export const relaysGatewayKey = (script, env) => (script === "worker.mjs" || script === "review.mjs") && parseWorkerSecrets(input("worker_secrets", { env })).entries.some((e) => e.name === RELAYED_NAME); + +/** + * Remove the gateway relay socket directories dead stages of this Objective left (lib/worker-relay.mjs handSocketDir's + * record in each usage directory): a stage whose process (STAGE_RECORD) is gone, or, where it never started, whose loop is. + * A live one, or one whose process cannot be told, is left. Returns how many were removed. + */ +export function reapSocketDirs(ws, opts = {}) { + let n = 0; + let dirs = []; + try { + dirs = readdirSync(ws).filter((d) => d.startsWith("usage-")); + } catch { + return 0; + } + for (const d of dirs) { + const usageDir = join(ws, d); + const rec = readSocketDirRecord(usageDir); + if (!rec) continue; + let stage = null; + try { + stage = JSON.parse(readFileSync(join(usageDir, STAGE_RECORD), "utf8")); + } catch { + // never started, or ended + } + if (sameProcess(stage ?? rec.owner) !== false) continue; + if (dropSocketDir(usageDir, opts)) n += 1; + } + return n; +} + /** * A stage of this Objective still running on this machine (this loop's own, or one whose loop died), as {pid, file, * sure}, or null. A stage counts as running when its record's process is still that process (`sure`), or cannot be told @@ -521,9 +554,9 @@ export class Local { /** * One stage's env: the base, the scoped token, and the sandbox both the harness and Factory's own commands run in. - * `stageTmp` is the stage's own temp directory (stage() makes it and removes it on every path). + * `stageTmp` is the stage's own temp directory (stage() makes it and removes it on every path). `script`: the stage's. */ - stageEnv(issue, usageDir, clone, stageTmp) { + stageEnv(issue, usageDir, clone, stageTmp, script = null) { const ws = this.ws(issue); // The package managers' caches, one set per Objective: the sandboxes cannot write the developer's own (pnpm's // store sits in the home directory), and a cache shared between Objectives would let one plant code in another. @@ -573,6 +606,21 @@ export class Local { chmodSync(join(binDir, "git-lfs"), 0o755); } const base = this.baseEnv(); + // A stage that relays the gateway key (lib/worker-relay.mjs): its socket's directory, made here so the outer sandbox + // written below names it, and handed to the stage, from this loop alone. A private one when the per-user directory is + // refused; none when /tmp itself is refused, and the stage then Blocks naming why. Removed when the stage ends + // (stage()), or by a later loop's reap when this one dies with it (reapSocketDirs). + delete base[RELAY_SOCKET_DIR_ENV]; + let socketDir = null; + if (relaysGatewayKey(script, base)) { + try { + const m = handSocketDir(usageDir); + socketDir = base[RELAY_SOCKET_DIR_ENV] = m.dir; + if (m.refused) log(`#${issue}: the gateway relay's socket goes in a private directory: ${m.refused}`); + } catch (e) { + log(`#${issue}: no directory for the gateway relay's socket: ${String(e?.message || e).slice(0, 200)}`); + } + } const addDirs = [...caches, stageTmp]; // Linux: the stage's node (#111) and the repository's package manager (#137) for Claude Code's and Copilot's commands, // read-only: their sandboxes show nothing of the home but what their settings name. Never in or above what a sandbox @@ -608,7 +656,7 @@ export class Local { // Copilot's outer sandbox runs bwrap through Factory's wrapper (lib/local.mjs bwrapTunScript), in the usage directory, // which no sandbox may write: srt runs it on the host. const bwrapPath = this.harness === "copilot" ? writeBwrapTun(join(usageDir, "outer")) : null; - writeFileSync(outerFile, JSON.stringify(harnessOuterSettings({ harness: this.harness, domains: this.domains(), writable: [clone, ...addDirs, join(usageDir, "harness")], clone, env: { ...process.env, ...extra }, install: harnessInstall(whichSync(this.harness)), projectDir, bwrapPath, relaySocketDir: relaySocketDir(usageDir), ripgrep }), null, 2)); + writeFileSync(outerFile, JSON.stringify(harnessOuterSettings({ harness: this.harness, domains: this.domains(), writable: [clone, ...addDirs, join(usageDir, "harness")], clone, env: { ...process.env, ...extra }, install: harnessInstall(whichSync(this.harness)), projectDir, bwrapPath, relaySocketDir: socketDir, ripgrep }), null, 2)); } // Last on PATH, so nothing there can ever stand in for git, gh, srt or claude in Factory's own unsandboxed calls; // and neither sandbox may write it (FACTORY_LOCAL_SANDBOX.readOnly, srt denyWrite), so nothing can be planted. @@ -730,9 +778,14 @@ export class Local { // a symlink into the home directory before the next sandbox opens it. Removed when the stage ends or fails to // start; one a killed loop left is cleared when the next stage on this machine makes its own (lib/local.mjs stageTemp). const stageTmp = stageTemp(`${issue}-`); + // The gateway relay's socket directory, when stageEnv made one (handSocketDir's record): removed with the stage. + const dropStage = () => { + dropOwned(stageTmp); + dropSocketDir(usageDir); + }; let env; try { - env = this.stageEnv(issue, usageDir, clone, stageTmp); + env = this.stageEnv(issue, usageDir, clone, stageTmp, script); // Git locks an interrupted stage left (a reboot, a killed serve) go first, unless a live process may hold them // (lib/local.mjs staleGitLocks); then, fresh: what an earlier stage or run left untracked in this reused clone (prepareClone). const locks = staleGitLocks(clone); @@ -740,7 +793,7 @@ export class Local { const prepared = await prepareClone(clone, [env.pnpm_config_store_dir, env.BUN_INSTALL_CACHE_DIR], (inner) => exec(...BACKENDS.local.sandbox.git(inner, env), { env }), { fresh: true, locks: locks.stale }); for (const l of prepared.locks) log(`#${issue}: removed stale git lock ${l} (no live process holds it)`); } catch (e) { - dropOwned(stageTmp); + dropStage(); const reason = { "outer-sandbox": copilotOuterReason, "sandbox-scan": sandboxScanReason }[e.code]; if (!reason) throw e; log(`#${issue}: refusing to run: ${e.message}`); @@ -748,7 +801,7 @@ export class Local { return { code: null, exit: "blocked", usd: null, usageDir }; } if (this.stopping) { - dropOwned(stageTmp); + dropStage(); return { code: null, exit: null, usd: null, usageDir }; } return new Promise((done) => { @@ -789,7 +842,7 @@ export class Local { } catch { // no record: the child failed before writing one } - dropOwned(stageTmp); + dropStage(); done({ code, signal, exit: result?.exit ?? null, usd: result?.usd ?? null, usageDir, stage: name, log: logFile }); }); }); @@ -860,6 +913,8 @@ export class Local { reapHere(issue) { const reaped = reapLeftRuns(this.ws(issue)); if (reaped) log(`#${issue}: stopped ${reaped} harness run(s) an earlier stage left running on this machine`); + const dirs = reapSocketDirs(this.ws(issue)); + if (dirs) log(`#${issue}: removed ${dirs} gateway relay socket director${dirs === 1 ? "y" : "ies"} a dead stage left`); } /** diff --git a/plugin/test/gateway-relay.test.mjs b/plugin/test/gateway-relay.test.mjs index b9d6605..a46e2ac 100644 --- a/plugin/test/gateway-relay.test.mjs +++ b/plugin/test/gateway-relay.test.mjs @@ -1,7 +1,7 @@ import "./temp.mjs"; import assert from "node:assert/strict"; import { execFileSync, spawn } from "node:child_process"; -import { chmodSync, existsSync, mkdirSync, mkdtempSync, readFileSync, realpathSync, rmSync, statSync, symlinkSync, writeFileSync } from "node:fs"; +import { chmodSync, existsSync, mkdirSync, mkdtempSync, readFileSync, readdirSync, realpathSync, renameSync, rmSync, statSync, symlinkSync, writeFileSync } from "node:fs"; import http from "node:http"; import net from "node:net"; import { tmpdir } from "node:os"; @@ -9,7 +9,9 @@ import { dirname, join } from "node:path"; import { test } from "node:test"; import { CALL_LOG, UNIX_SOCKET_PATH_MAX, EDITS_MAX_BYTES, FAST_ERROR_MS, FAST_ERROR_STATUS, HOLD_MAX_MS, IMAGE_UNKNOWN_FLOOR_USD, SLOW_FAILURE_MS, SLOW_UNKNOWN_FLOOR_USD, RELAY_RETRY_AFTER_S, UNKNOWN_STAGE_SHUT, UNKNOWN_STREAK_SHUT, UNKNOWN_WINDOW_MS, UPSTREAM_KEEPALIVE_MS, upstreamRequest, RELAYED_NAME, RELAY_PING, addKeySpend, costOf, gatewayRefusalOf, keyCutter, modelOf, openRelay, relayShutText, routeOf, stageBlockReason, tokenMatches, withGatewayRefusal, withRelayShut } from "../scripts/lib/gateway-relay.mjs"; import { relayProbeCommand, relayShimScript, writeRelayShim } from "../scripts/lib/relay-shim.mjs"; -import { RELAY_SOCKET_ROOT, closeWorkerRelay, localRelayDir, makeSocketDir, openWorkerRelay, relaySocketDir, unrelayedGatewayKeys } from "../scripts/lib/worker-relay.mjs"; +import { RELAY_SOCKET_DIR_ENV, RELAY_SOCKET_ROOT, SOCKET_DIR_RECORD, checkSocketRoot, closeWorkerRelay, dropSocketDir, handSocketDir, localRelayDir, makeSocketDir, openWorkerRelay, readSocketDirRecord, relaySocketDir, unrelayedGatewayKeys } from "../scripts/lib/worker-relay.mjs"; +import { STAGE_RECORD, reapSocketDirs, relaysGatewayKey } from "../scripts/local.mjs"; +import { processRecord } from "../scripts/lib/proc.mjs"; import { keyBreakdown, maskFile, writeUsage } from "../scripts/lib/session.mjs"; import { maxUsdReached, usdText } from "../scripts/lib/logic.mjs"; import { harnessOuterSettings } from "../scripts/lib/local.mjs"; @@ -432,10 +434,12 @@ test("the relay's socket goes in a short directory of the stage's own: made 700 rmSync(dirname(dir), { force: true }); // A socket path still over the limit: the stage Blocks before the run, naming the limit, with nothing left behind. + // A stand-in for /tmp: sticky, and its owner taken for root's (socketRootUid). const deep = join(root, "d".repeat(80)); - mkdirSync(deep); + mkdirSync(deep, { mode: 0o1700 }); + chmodSync(deep, 0o1700); const shell = { [RELAYED_NAME]: KEY }; - const r = await openWorkerRelay({ inputs: { harness: "codex", maxUsd: 5 }, shell, usageDir: join(root, "u"), env: { FACTORY_LOCAL: "1", PATH: "/nope" }, which: (n) => `/usr/bin/${n}`, socketRoot: deep }); + const r = await openWorkerRelay({ inputs: { harness: "codex", maxUsd: 5 }, shell, usageDir: join(root, "u"), env: { FACTORY_LOCAL: "1", PATH: "/nope" }, which: (n) => `/usr/bin/${n}`, socketRoot: deep, socketRootUid: uid }); assert.equal(r.relay, null); assert.match(r.blocked, new RegExp(`^the gateway relay for ${RELAYED_NAME} could not start: its socket's path would be \\d+ bytes, over this system's limit of ${UNIX_SOCKET_PATH_MAX} for a Unix socket`)); assert.ok(!(RELAYED_NAME in r.shell), "the key is out of the shell"); @@ -460,7 +464,8 @@ test("a long checkout path: the relay starts, a call through Factory's forwarder const open = (o) => openRelay({ ...o, origin: gw.origin, lookupDelaysMs: [1] }); let opened = null; try { - opened = await openWorkerRelay({ inputs: { harness: "codex", maxUsd: 5 }, shell, usageDir, env: { FACTORY_LOCAL: "1", PATH: "/nope" }, which: (n) => (n === "bwrap" ? fake : "/usr/bin/socat"), open, socketRoot: root }); + chmodSync(root, 0o1700); + opened = await openWorkerRelay({ inputs: { harness: "codex", maxUsd: 5 }, shell, usageDir, env: { FACTORY_LOCAL: "1", PATH: "/nope" }, which: (n) => (n === "bwrap" ? fake : "/usr/bin/socat"), open, socketRoot: root, socketRootUid: process.getuid() }); assert.equal(opened.blocked, null, opened.blocked); const relay = opened.relay; assert.equal(dirname(relay.socketPath), relaySocketDir(usageDir, { root })); @@ -2081,3 +2086,249 @@ test("a fast 408 or 409 is a fast error status, as a fast 5xx is: only the run c rmSync(dir, { recursive: true, force: true }); } }); + +/** A stand-in for /tmp under the real one: sticky (mode 1700), its owner taken for root's in the calls below. */ +function stickyRoot() { + const root = mkdtempSync(join(realpathSync("/tmp"), "fxr-")); + chmodSync(root, 0o1700); + return root; +} + +test("the socket's root must be root's and sticky: a non-sticky stand-in is refused with a named reason, and the stage Blocks with nothing made (#185)", async () => { + const uid = process.getuid(); + assert.equal(checkSocketRoot(RELAY_SOCKET_ROOT), realpathSync(RELAY_SOCKET_ROOT), "this machine's /tmp passes"); + const root = mkdtempSync(join(realpathSync("/tmp"), "fxr-")); + try { + chmodSync(root, 0o777); + assert.throws(() => checkSocketRoot(root, { rootUid: uid }), new RegExp(`^Error: ${root} has no sticky bit \\(mode 777\\), so another user could swap the socket's directory there`)); + chmodSync(root, 0o1777); + assert.throws(() => checkSocketRoot(root), new RegExp(`^Error: ${root} is not owned by root`), "sticky, but not root's"); + assert.equal(checkSocketRoot(root, { rootUid: uid }), root); + chmodSync(root, 0o777); + const shell = { [RELAYED_NAME]: KEY }; + for (const env of [{ FACTORY_LOCAL: "1", PATH: "/nope" }, { PATH: "/nope" }]) { + const r = await openWorkerRelay({ inputs: { harness: "codex", maxUsd: 5 }, shell, usageDir: join(root, "u"), env, which: (n) => `/usr/bin/${n}`, socketRoot: root, socketRootUid: uid }); + assert.equal(r.relay, null); + assert.match(r.blocked, new RegExp(`^the gateway relay for ${RELAYED_NAME} could not start: its socket's directory: ${root} has no sticky bit`)); + assert.ok(!(RELAYED_NAME in r.shell), "the key is out of the shell"); + assert.deepEqual(readdirSync(root), [], "no directory made, no fallback either"); + } + // The loop refuses it too, and hands nothing. + assert.throws(() => handSocketDir(join(root, "u"), { root, rootUid: uid }), /has no sticky bit/); + } finally { + rmSync(root, { recursive: true, force: true }); + } +}); + +test("a refused per-user directory: the stage's socket goes in a private mkdtemp directory, handed from the loop locally, checked, and removed with the stage; the path limit still holds there (#185)", { skip: process.platform === "win32" && "unix" }, async () => { + const uid = process.getuid(); + const root = stickyRoot(); + const shell = { [RELAYED_NAME]: KEY }; + const opts = { socketRoot: root, socketRootUid: uid }; + try { + // Another user's stand-in at the per-user name: a file (a directory of theirs, or one open to others, is refused the same way). + writeFileSync(join(root, `factory-relay-${uid}`), "taken"); + // A developer's own session: the stage falls back itself. + const own = await openWorkerRelay({ inputs: { harness: "claude", maxUsd: 5 }, shell, usageDir: join(root, "u1"), env: { PATH: "/nope" }, ...opts }); + assert.equal(own.blocked, null, own.blocked); + const fallback = dirname(own.relay.socketPath); + assert.match(fallback, new RegExp(`^${root}/factory-relay-${uid}-[A-Za-z0-9]{6}$`)); + assert.equal(statSync(fallback).mode & 0o7777, 0o700); + assert.equal(statSync(own.relay.socketPath).mode & 0o777, 0o600); + await closeWorkerRelay(own.relay); + assert.ok(!existsSync(fallback), "removed with the stage"); + + // Local mode with nothing handed: refused, never a directory the outer sandbox does not name. + const unhanded = await openWorkerRelay({ inputs: { harness: "codex", maxUsd: 5 }, shell, usageDir: join(root, "u2"), env: { FACTORY_LOCAL: "1", PATH: "/nope" }, which: (n) => `/usr/bin/${n}`, ...opts }); + assert.match(unhanded.blocked, /its socket's directory: .*factory-relay-\d+ is not a directory/); + + // Local mode: the loop makes the private directory, records it, names it in the outer sandbox and hands it over. + const usageDir = join(root, "u3"); + mkdirSync(usageDir); + const m = handSocketDir(usageDir, { root, rootUid: uid }); + assert.match(m.refused, /factory-relay-\d+ is not a directory/); + assert.match(m.dir, new RegExp(`^${root}/factory-relay-${uid}-[A-Za-z0-9]{6}$`)); + assert.equal(readSocketDirRecord(usageDir).dir, m.dir); + const outer = harnessOuterSettings({ harness: "codex", writable: ["/w"], relaySocketDir: m.dir, env: {} }); + assert.ok(!outer.filesystem.denyRead.some((p) => m.dir === p || m.dir.startsWith(`${p}/`)), "the outer sandbox keeps it reachable"); + const handed = await openWorkerRelay({ inputs: { harness: "codex", maxUsd: 5 }, shell, usageDir, env: { FACTORY_LOCAL: "1", PATH: "/nope", [RELAY_SOCKET_DIR_ENV]: m.dir }, which: (n) => `/usr/bin/${n}`, ...opts }); + assert.equal(handed.blocked, null, handed.blocked); + assert.equal(dirname(handed.relay.socketPath), m.dir); + await closeWorkerRelay(handed.relay); + assert.equal(dropSocketDir(usageDir, { root, rootUid: uid }), false, "already gone with the stage"); + assert.equal(readSocketDirRecord(usageDir), null); + + // A handed path that is not one of this user's socket directories there is refused, even where recorded. + writeFileSync(join(usageDir, SOCKET_DIR_RECORD), JSON.stringify({ dir: join(root, "elsewhere") })); + const odd = await openWorkerRelay({ inputs: { harness: "codex", maxUsd: 5 }, shell, usageDir, env: { FACTORY_LOCAL: "1", PATH: "/nope", [RELAY_SOCKET_DIR_ENV]: join(root, "elsewhere") }, which: (n) => `/usr/bin/${n}`, ...opts }); + assert.match(odd.blocked, /is not a socket directory of this user's under/); + + // The path limit, for the fallback too: a stand-in root deep enough that the private directory's socket passes it. + const deep = join(root, "d".repeat(80)); + mkdirSync(deep, { mode: 0o1700 }); + chmodSync(deep, 0o1700); + writeFileSync(join(deep, `factory-relay-${uid}`), "taken"); + const long = await openWorkerRelay({ inputs: { harness: "claude", maxUsd: 5 }, shell, usageDir: join(root, "u4"), env: { PATH: "/nope" }, socketRoot: deep, socketRootUid: uid }); + assert.match(long.blocked, new RegExp(`could not start: its socket's path would be \\d+ bytes, over this system's limit of ${UNIX_SOCKET_PATH_MAX} for a Unix socket \\(${deep}/factory-relay-${uid}-`)); + assert.deepEqual(readdirSync(deep), [`factory-relay-${uid}`], "the private directory removed"); + } finally { + rmSync(root, { recursive: true, force: true }); + } +}); + +test("after the listen the socket must be in the very directory checked: a directory swapped before the listen, or swapped back after it, Blocks with nothing handed over (#185)", { skip: process.platform === "win32" && "unix" }, async () => { + const uid = process.getuid(); + const root = stickyRoot(); + const shell = { [RELAYED_NAME]: KEY }; + const opened = []; + try { + // What only a writable, non-sticky /tmp would allow: the checked directory renamed away, another put in its place. + const swap = (back) => async (o) => { + renameSync(o.socketDir, `${o.socketDir}.aside`); + mkdirSync(o.socketDir, { mode: 0o700 }); + const relay = await openRelay(o); + if (back) { + renameSync(o.socketDir, `${o.socketDir}.other`); + renameSync(`${o.socketDir}.aside`, o.socketDir); + } + return relay; + }; + // Any relay a failed check let start is closed below, so a failure here never hangs the run. + for (const [back, why] of [[false, /was replaced after it was checked/], [true, /no socket of this user's is at .* after the listen/]]) { + const usageDir = join(root, `u-${back}`); + const r = await openWorkerRelay({ inputs: { harness: "claude", maxUsd: 5 }, shell, usageDir, env: { PATH: "/nope" }, open: swap(back), socketRoot: root, socketRootUid: uid }); + if (r.relay) opened.push(r.relay); + assert.equal(r.relay, null); + assert.match(r.blocked, new RegExp(`^the gateway relay for ${RELAYED_NAME} could not start: its socket's directory: `)); + assert.match(r.blocked, why); + assert.ok(!(RELAYED_NAME in r.shell), "the key is out of the shell"); + } + } finally { + for (const r of opened) await closeWorkerRelay(r); + rmSync(root, { recursive: true, force: true }); + } +}); + +test("a loop's reap removes the socket directory of a dead stage, never a live one's or a path of another shape (#185)", { skip: process.platform !== "linux" && "process records read /proc" }, async () => { + const uid = process.getuid(); + const root = stickyRoot(); + const ws = join(root, "ws"); + // A process record of a process that has exited. + const gone = spawn("sleep", ["30"]); + const dead = processRecord(gone.pid); + gone.kill("SIGKILL"); + await new Promise((r) => gone.on("exit", r)); + const stages = { + "usage-work-dead": { stage: dead }, + "usage-work-live": { stage: processRecord() }, + "usage-work-unstarted-dead-loop": { owner: dead }, + "usage-work-unstarted-live-loop": {}, + }; + try { + const dirs = {}; + for (const [name, rec] of Object.entries(stages)) { + const usageDir = join(ws, name); + mkdirSync(usageDir, { recursive: true }); + dirs[name] = handSocketDir(usageDir, { root, rootUid: uid }).dir; + if (rec.stage) writeFileSync(join(usageDir, STAGE_RECORD), JSON.stringify(rec.stage)); + if (rec.owner) writeFileSync(join(usageDir, SOCKET_DIR_RECORD), JSON.stringify({ dir: dirs[name], owner: rec.owner })); + } + // A record naming some other directory of this user's: never removed. + const other = join(root, "keep-me"); + mkdirSync(other); + mkdirSync(join(ws, "usage-odd")); + writeFileSync(join(ws, "usage-odd", SOCKET_DIR_RECORD), JSON.stringify({ dir: other, owner: dead })); + + assert.equal(reapSocketDirs(ws, { root, rootUid: uid }), 2); + assert.ok(!existsSync(dirs["usage-work-dead"]), "a dead stage's: removed"); + assert.ok(!existsSync(dirs["usage-work-unstarted-dead-loop"]), "one whose loop died before the stage started: removed"); + assert.ok(existsSync(dirs["usage-work-live"]), "a live stage's: kept"); + assert.ok(existsSync(dirs["usage-work-unstarted-live-loop"]), "one a live loop is starting a stage with: kept"); + assert.ok(existsSync(other), "a path of another shape: kept"); + assert.ok(existsSync(join(root, `factory-relay-${uid}`)), "the per-user directory stays"); + assert.equal(reapSocketDirs(ws, { root, rootUid: uid }), 0, "done once"); + } finally { + rmSync(root, { recursive: true, force: true }); + } +}); + +test("local mode hands a socket directory only to a stage that relays the gateway key (#185)", () => { + const env = (v) => ({ FACTORY_WORKER_SECRETS: v }); + assert.equal(relaysGatewayKey("worker.mjs", env("AI_GATEWAY_API_KEY=WORKER_AI_GATEWAY_API_KEY")), true); + assert.equal(relaysGatewayKey("review.mjs", env("FAL_KEY, AI_GATEWAY_API_KEY=W")), true); + assert.equal(relaysGatewayKey("merge.mjs", env("AI_GATEWAY_API_KEY=W")), false); + assert.equal(relaysGatewayKey("worker.mjs", env("FAL_KEY")), false); + assert.equal(relaysGatewayKey("worker.mjs", {}), false); +}); + +test("removing a socket directory never goes through a link: a recorded path whose per-user parent is now a link to elsewhere stays, by reap and at the stage's end (#185)", { skip: process.platform !== "linux" && "process records read /proc" }, async () => { + const uid = process.getuid(); + const root = stickyRoot(); + try { + // A dead stage's record; then /tmp cleared (a reboot) and another user's link planted at the per-user name. + const gone = spawn("sleep", ["30"]); + const dead = processRecord(gone.pid); + gone.kill("SIGKILL"); + await new Promise((r) => gone.on("exit", r)); + const hash = "0123456789ab"; + const elsewhere = join(root, "clone", "x"); + mkdirSync(join(elsewhere, hash), { recursive: true }); + for (const d of [join(root, "clone"), elsewhere, join(elsewhere, hash)]) chmodSync(d, 0o700); + writeFileSync(join(elsewhere, hash, "precious.txt"), "keep"); + symlinkSync(elsewhere, join(root, `factory-relay-${uid}`)); + const dir = join(root, `factory-relay-${uid}`, hash); + const ws = join(root, "ws"); + for (const name of ["usage-work-dead", "usage-work-ended"]) { + mkdirSync(join(ws, name), { recursive: true }); + writeFileSync(join(ws, name, SOCKET_DIR_RECORD), JSON.stringify({ dir, owner: dead })); + } + assert.equal(reapSocketDirs(ws, { root, rootUid: uid }), 0); + assert.equal(dropSocketDir(join(ws, "usage-work-ended"), { root, rootUid: uid }), false); + assert.ok(existsSync(join(elsewhere, hash, "precious.txt")), "nothing removed through the link"); + for (const name of ["usage-work-dead", "usage-work-ended"]) assert.equal(readSocketDirRecord(join(ws, name)), null, "the records go"); + } finally { + rmSync(root, { recursive: true, force: true }); + } +}); + +test("locally the stage takes only the socket directory made for it: another stage's live one, or one with no record, is refused; an unrecorded directory is never left behind (#185)", { skip: process.platform === "win32" && "unix" }, async () => { + const uid = process.getuid(); + const root = stickyRoot(); + const shell = { [RELAYED_NAME]: KEY }; + const opts = { socketRoot: root, socketRootUid: uid }; + const env = (dir) => ({ FACTORY_LOCAL: "1", PATH: "/nope", [RELAY_SOCKET_DIR_ENV]: dir }); + // Any relay a failed check let start is closed below, so a failure here never hangs the run. + const opened = []; + try { + const [a, b] = ["a", "b"].map((n) => { + const usageDir = join(root, `usage-${n}`); + mkdirSync(usageDir); + return { usageDir, dir: handSocketDir(usageDir, { root, rootUid: uid }).dir }; + }); + // Stage a handed stage b's directory: one of this user's, of the right shape, yet not its own. + const other = await openWorkerRelay({ inputs: { harness: "codex", maxUsd: 5 }, shell, usageDir: a.usageDir, env: env(b.dir), which: (n) => `/usr/bin/${n}`, ...opts }); + if (other.relay) opened.push(other.relay); + assert.equal(other.relay, null); + assert.match(other.blocked, new RegExp(`its socket's directory: ${b.dir} is not the directory made for this stage`)); + assert.ok(!(RELAYED_NAME in other.shell)); + const unrecorded = join(root, "usage-c"); + mkdirSync(unrecorded); + const none = await openWorkerRelay({ inputs: { harness: "codex", maxUsd: 5 }, shell, usageDir: unrecorded, env: env(a.dir), which: (n) => `/usr/bin/${n}`, ...opts }); + if (none.relay) opened.push(none.relay); + assert.match(none.blocked, /is not the directory made for this stage/); + const own = await openWorkerRelay({ inputs: { harness: "codex", maxUsd: 5 }, shell, usageDir: a.usageDir, env: env(a.dir), which: (n) => `/usr/bin/${n}`, ...opts }); + if (own.relay) opened.push(own.relay); + assert.equal(own.blocked, null, own.blocked); + assert.equal(dirname(own.relay.socketPath), a.dir); + await closeWorkerRelay(own.relay); + assert.ok(existsSync(b.dir), "the other stage's directory untouched"); + + // The record cannot be written (here: no usage directory): the directory just made goes too. + const missing = join(root, "usage-missing"); + assert.throws(() => handSocketDir(missing, { root, rootUid: uid }), /ENOENT/); + assert.ok(!existsSync(relaySocketDir(missing, { root })), "no directory left without a record"); + } finally { + for (const r of opened) await closeWorkerRelay(r); + rmSync(root, { recursive: true, force: true }); + } +});