Repository navigation
fix(code-execution)!: the runtime's channel moves to fds 3 and 4, and the host reads a script's stdout and stderr - #495
Conversation
…eads a script's output The code_execution runtime ran its protocol on fds 0 and 1, which every script and every command it runs shares, and kept a script's output in memory, read back at the script's end. So a command's output (subprocess without capture_output, os.system, C code writing to fds 1 and 2) reached only the host's log as "non-protocol" lines; a line without a newline ran into exec_result and left the exec waiting out its time limit; a command reading its input (cat, ssh) took the host's messages; os.close(0) or (1) broke the interpreter; and whatever a script printed died with the interpreter on a kill or a crash, along with a fatal error's text. Now the host spawns the runtime with stdio ignore/pipe/pipe/pipe/pipe: - fd 0 is /dev/null (Node's 'ignore'), so a command reading its input gets its end at once. - fds 3 (host -> py) and 4 (py -> host) carry the protocol. The runtime first checks both are open, and if not writes one sentence to fd 2 (the host's channel didn't reach it; a pythonPath wrapper must pass fds 3 and 4 on, which a container or sudo's defaults don't) and exits 2, so the failed start says what to fix. Then it makes them non-inheritable, so no command is given them. No fallback to fds 0 and 1, which would keep two protocols. - fds 1 and 2 are pipes the host reads itself (OutputStream). Each exec op carries a random mark; when the script returns, the runtime sends exec_result (now only the return code), flushes the script's writers and writes the mark into both streams through private copies of fds 1 and 2. The host settles once it has exec_result and both marks, so everything the script and the commands it waited for wrote is in the result; each stream is capped at 2,000,000 characters with the same note as before. - Output arriving while no exec runs (a command left running) goes to the host's log, labelled by stream; after a reclaim the old pipes stay read until whatever holds them closes them. - The runtime restores fds 0, 1 and 2 from its private copies at each exec's start, and gives the script fresh line-buffered writers over 1 and 2 (UTF-8, backslashreplace): a line reaches the host as it's printed, in order with commands' output. 100,000 prints cost about three times what CappedIO did (measured while designing). - faulthandler is enabled on the runtime's private copy of fd 2, so a segfault or an abort leaves a traceback; Py_FatalError's own text reaches the host because fd 2 is the host's pipe. - A kill, abort, dispose or crash now settles with the output so far ahead of its message (stopPending). An unexpected exit waits for both pipes to end, or EXIT_OUTPUT_GRACE_MS when a command still holds them, and logs the last of what the interpreter wrote. A start that fails says what the interpreter wrote to stderr. - Background: the runtime joins fd 2 onto fd 1's pipe, so the journal keeps the order written; the host writes the journal (Journal: same 50 MB cap, same 4,000-character tail) in place of the runtime's LogTee, so a crashed background script's tail is real. - When the runtime can't go on because a script closed one of its own fds (fd 4, or its private copies), it says why on stderr and exits (code 70) rather than leaving the exec to its time limit. anima-research#458's kill of an interpreter that stops reading moves to fd 3; since fd 3 is a duplex socket that ends when the far side closes, the first message that can't reach it is caught by the write's callback as well as by 'error'. Removed from the runtime: CappedIO, LogTee, the result assembly and the log-file handling. The tool description now says what the script and the commands it runs print comes back; CodeExecutionConfig.pythonPath's doc says a wrapper must pass fds 3 and 4 on. Changelog: the script-facing changes in a fixed fragment; the hosts' part (a pythonPath wrapper must pass fds 3 and 4 on; PYTHON_RUNTIME_SOURCE's new protocol) in a breaking fragment under **Hosts:**, so this releases in a new minor version. Tests: the stand-ins that stop reading now close fd 3 and answer on fd 4; the stop-table driver reads the script's output from fds 1 and 2 through a file; new real-runtime tests for each piece above. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s own fds, flushes kept writers, and an exit is read in order The fixes an internal review of the first form found worth making now: Runtime: - The runtime records its own fds as it opens them (3, 4, and its copies of 0, 1 and 2) by mode, device and inode, and checks them at each exec's start and end, after flushing the script's writers and before exec_result: one a script closed or replaced ends the interpreter, saying which. The reader thread ends it too when reading fd 3 fails. So a script's os.dup2 onto fd 4, or close and reopen, no longer sends exec_result into its file. - Every writer the runtime gives a script goes in a WeakSet, and each exec's end flushes them all, so a writer a script kept from an earlier exec sends its partial line in the exec that wrote it. A flush that fails says so in a diagnostic, and the runtime's diagnostics fall back to fd 2 when a script closed its copy. - The failed-start sentence says code_execution can't run until whoever runs the host changes the wrapper, and that calling again fails the same way. The header says output is line by line, and a write at the same moment as a long one can land between its parts. Host: - An exit waits for fd 4 to end too, so an exec_result written before the exit is read first, and a script that reported keeps its own return code. When the grace runs out, the settle waits one setImmediate, after a poll phase, so output already in the pipes is read even when the host's loop was busy past the grace. - begin() ends the decoder, so a half character left by output with no exec running is the log's; push() holds back only a chunk end that could start the mark, so a background journal keeps up; each log entry of output with no exec running is cut at 2,000 characters. - One alive() check, signalCode included, for every guard, so the time limit's repeated SIGINT can't go to a reaped interpreter during the exit grace; unreachable() is silent for a torn-down interpreter. - A stopped background script's journal ends with why; a journal write that fails is logged once and the journal stops. - The stopped-reading message no longer names fd 3, a cause the host can't know. Tests: stand-ins whose child writes after the stand-in is reaped (output on stderr; an exec_result on fd 4), with the host busy past the grace; a Python stand-in splitting "é" and the mark across reads; rows for closing fd 3, replacing fd 4, and the copies of 0 and 2; fd 3 closed while a tool call is out; wrappers that drop only fd 3 or only fd 4; fd 0 is /dev/null; a kept writer; no SIGINT after a signal death; a journal line before the script ends and its stop reason; /dev/full; the left-running command gated on files, its long line cut, its half character kept out of the next exec; the crash-settle test starts its interpreter first and is named for what it checks; the kill test's message matched whole; the abort test logs no failed write. Changelog: line by line, arrival attribution, the prompt-cache miss, the runtime's own fds, what had reached the host, partial lines lost on a kill, a fatal error's text on the script's fd 2, the journal's stop reason; the breaking fragment says init no longer takes log_path or tail_chars. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… race behind a Node 20 failure The race (a Node 20 baseline run failed "an interpreter ended by a signal ..." with "python runtime stopped reading the host's messages"): an interpreter that dies with a message unread on fd 3 resets the host's end of it on Linux, and that error can be seen before the exit is. The host took it for a deaf interpreter, killed the zombie, and reported a death of its own (an out-of-memory kill, or here the script's own SIGKILL) as one that stopped reading. Now a message that can't reach the interpreter schedules the kill after UNREACHABLE_GRACE_MS (200 ms) and one setImmediate, and an interpreter that exited by then is left to its own exit. A stand-in that breaks the pipe and kills itself 20 ms later now reads as its own exit. The log line reads "messages to the python runtime failed". From an internal review of the round before: - send() checks fd 4 before each message, so a script that replaces fd 4 and calls a tool ends the interpreter at once instead of waiting out the call's 270 s; a row tests it, and one replacing fd 4 with a socket. - A process the script forks that runs off the script's end leaves with os._exit(0) rather than sending the exec's result and marks. - The script's writers write through _FdWriter, and the marks through _write_all, both waiting with select when a command left the pipe non-blocking (one open file with the runtime's copies) instead of raising BlockingIOError or ending the interpreter. - A failed start waits only for the output pipes, since the exit's teardown has stopped reading fd 4. - An interpreter that exits while no script runs says so in the log; a background script's exit log says "the last it wrote", its journal holding both streams. - Tests: a thread replacing fd 4 between execs ends the next exec before its script runs; the signal test asserts no SIGINT to a reaped pid; the crash-settle test checks the command is still alive when the exec settles, not a wall-clock bound; ordinary fd use keeps the interpreter. - The try/excepts around send's write, the marks and _open_streams went: the own-fd checks cover a closed or replaced fd before each of them, and a removal run showed taking out two of them failed no test. From the readers: the lead, the fragment and the test say "by the end of that script at the latest"; a left-running command's output goes to a later script's result only while the same interpreter runs; the breaking fragment names a Python wrapper using subprocess. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… code A process a script forks that runs off the script's end left with os._exit(0), so a parent waiting on it saw success even when the child's part of the script raised. It now leaves with the script's return code, 1 for a script that raised, as Python's own exit gives a script that ended so. It still doesn't flush its writers, which hold a copy of what the parent's held at the fork. The fork test's child now prints, then raises: the result holds the child's line and its traceback, the parent prints the exit status it saw (1), and the next exec runs. With os._exit(0) back, that test fails. code-execution.test.ts: 93/93 on Node 24.22.0 and 20.20.2 at this tree. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…arks do _diag, which carries _fatal's reason, wrote with a bare os.write. A command can leave the output pipe non-blocking (it is one open file with fd 2 and the runtime's copy), so on a full pipe the note raised BlockingIOError on both fds and was lost, leaving the host code 70 with no line saying why. It now writes through _write_all, which waits for room with select, as the marks and the script's writers do, so the runtime's own writes to the output pipes all wait alike. On a blocking pipe nothing changes: os.write already waited. CPython's own writes of a crash's text don't wait: on a full pipe a command left non-blocking, a fatal error's text and faulthandler's traceback are dropped (probes on Python 3.12.3 and 3.13.5, each beside a control with room). The description's "Not verified" names both. No test fills the pipe before the note: taking the wait out fails none. code-execution.test.ts: 93/93 on Node 24.22.0 and 20.20.2 at this tree. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…aths main brings anima-research#413 (what code_execution's failures say, and what helps) and anima-research#458's merge, both already composed with this change's base, plus other packages' fixes that touch no file of this change's. The conflicts were in py-runner.ts and one test; runtime-py.ts merged clean (anima-research#413's tool-function argument checks sit apart from this change's fds and writers). Resolved as this change's structure with anima-research#413's texts: - the deadline kill goes through stopPending, with anima-research#413's wording and limitRemedy; - the time-limit note is anima-research#413's, with its remedy, through onItsOwnLine; - an exit under a script reads as anima-research#413's interpreterExit, settled after the pipes as before; the host's log keeps the short technical form ("python runtime exited unexpectedly (code=…, signal=…)") with the last words; - the readiness errors take anima-research#413's texts ("exited as it started", "didn't start within 15 seconds"); an exit's rejection carries its exit code and what it wrote to stderr, for startFailure. Where anima-research#413's text and this change's meet, three choices of the merge's own: - startFailure: a failed start that wrote to stderr gets anima-research#413's sentence, then "It wrote to stderr: <its last words>". Kept whole rather than through boundedErrorMessage, whose cut keeps a text's head and would drop a traceback's last line. - The runtime's missing-channel check (fds 3 and 4) now exits with 78, sysexits' EX_CONFIG, which Python never exits with on its own; for that exit startFailure ends with the runtime's own sentences instead of anima-research#413's "Run it again", which they contradict ("calling it again fails the same way"), so the result ends with what whoever runs the host must change. - The stopped-reading text is shaped as anima-research#413's (stoppedReading): "script stopped: …", the remedy, and no "variables from earlier scripts" for a background script, as anima-research#413's interpreterExit does. Tests: this change's expectations move to anima-research#413's texts (a helper builds interpreterExit's text); anima-research#413's start-exit test now also sees what Node wrote; a background exec meets the stopped-reading text without the variables clause. code-execution.test.ts passes 105/105 on Node 24.22.0 and Node 20.20.2 (Node 20 with --test-timeout=300000: the file takes about 63 s there, past the 60 s a file was given before). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… the merge From a reader's read of 3f9ba29: - stoppedReading's remedy names the check to make, in anima-research#413's two steps: "If it stops the same way, check whether the script closes or replaces file descriptor 3, where the host's messages arrive; if not, ask whoever runs this host." A check, not a stated cause. - The reader thread's failed read runs _check_own_fd(3) first, so a closed or replaced fd 3 is named ("a script closed or replaced fd 3, where they arrive") before anima-research#413's general remedy; a failure with fd 3 intact keeps the exception's text. Its test's script now sleeps 30 s, past within's 10 s, so the end-of-script check, which says the same, can't be what settles it. - A correction to 3f9ba29's message: git merged runtime-py.ts without a conflict, and the merge itself then changed its exit to 78. From an internal review of the merge (30 issues, 9 fixed here): - A failed start's stderr is its last 2,000 characters, with a leading "…" when cut (OutputStream.lastIdleCut), and a cut inside a surrogate pair drops its low half. 3f9ba29's "kept whole" meant "not cut from its head"; the description now says what it is. - A start ended by a signal reads "was ended by <signal> as it started". - A forked child's sys.exit() ends it as Python does: its code, 0 for none, 1 after writing a code that isn't a number, with no traceback (_leave_forked). - The runtime's start check names what's missing ("its file descriptors 3 and 4 aren't open", or "its file descriptor 3 isn't open") and asks that the wrapper pass file descriptors 3 and 4 on. - The fixed fragment: "the result says which" is narrowed to when the script didn't replace the runtime's copy of fd 2 too, the journal's stop reason to when it didn't reach its 50 MB cap, and its stopped-reading line says what the message now says. - Tests: a wrapper's own exit 78 has its words end the result; 3,000 characters of stderr come back as the last 2,000, marked, ending on the last line, and an emoji at the cut leaves no half; SIGKILL at start is named; the wrapper rows pin the single-fd sentences; children that call sys.exit(3) and sys.exit() exit 3 and 0 with no traceback. code-execution.test.ts passes 108/108 on Node 24.22.0 and 20.20.2 at this tree, and taking out each piece this round adds fails a test there. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…text claims From an internal review of the commit before this one (21 issues, 11 fixed here): - A forked process's output: the runtime flushes the script's writers before each os.fork() (os.register_at_fork), so what they hold reaches the host once and a child's writers hold only what the child writes; both of a forked child's exits (sys.exit(), and running off the script's end) then flush them, so its own partial line arrives. A multiprocessing child's own flush at exit used to send the parent's partial line again. - _leave_forked guards str(code), so a code whose __str__ raises still ends the child with 1, rather than leaving it in the runtime's loop with the host's pipes held; its comment says it skips the parent's atexit handlers, as multiprocessing's children do. - The reader thread checks fd 3 at its end too, so a readable file a script opened in fd 3's place is named when the reader reaches its end, where the runtime used to cancel the script, printing "KeyboardInterrupt: script cancelled by host", before its end-of-exec check found fd 3. - The runtime's start sentence asks for a wrapper that passes both file descriptors 3 and 4 on, true when only one is missing. - Texts: the result names the closed or replaced fd unless the script also replaced the runtime's copy of fd 2, or closed it and redirected fd 2; a journal ends with the stop reason unless it reached its cap or a write to it failed; a failed start quotes up to the last 2,000 characters of its stderr. In the fixed fragment, the description, stopPending's comment and startFailure's doc. - Tests: the fork test adds sys.exit() with 259, 2 ** 40, "boom" and a code whose __str__ raises; a script prints a partial line and forks twice, one child leaving through sys.exit() and one off the script's end; a script's own sys.exit(3) in the runtime's process; the reader test adds a readable file in fd 3's place; a silent exit 78 gets the general text; the long stderr case asserts its whole text. - The exit-70 advice in interpreterExit is left for now. Claims in the PR text that had no test: - an abort and a dispose of a running script keep what it printed; - a lone surrogate a script prints comes back as \udcff (backslashreplace), on stdout and stderr; - a line a command left running writes while a later script runs is in that script's result (the test of output with no script running now covers this too). The fork test's name says what it pins: the exit status Python would give. code-execution.test.ts passes 113/113 on Node 24.22.0 and 20.20.2 at this tree, and each of the round's pieces, taken out on its own, fails a test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…he fd a script took When a script closes or replaces one of the runtime's own file descriptors, the runtime ends itself with exit 70 after a line saying which (_check_own_fd, _fatal). The result then ended with anima-research#413's general advice, "check whether the script itself ends the process, for example with os._exit()", which points away from the line just above it. Now interpreterExit reads the fd from that line in what arrived, the exec's stderr or, for a background script, its journal's tail, and the advice names it: "check whether the script closes or replaces fd 3, as the runtime's line above says; if not, ask whoever runs this host". A script's own exit, with any code, keeps the general advice. stopPending takes the reason as a function of what arrived, so the exit path can choose the advice once the output has ended. Tests: each of the runtime-fd rows and the reader test assert the whole advice with the fd it names; a background script that closes fd 4 is told so from its journal; a script's own os._exit(70) keeps the general text. code-execution.test.ts passes 115/115 on Node 24.22.0 and 20.20.2, and each piece, taken out on its own, fails a test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d only after the runtime's own exit, and a detached writer is passed over From an internal review of the two commits before this one (17 issues, 6 fixed here): - The advice after an exit names the fd from the runtime's line only after the runtime's own exit code, 70, so a script that prints such a line, as one reading an old log might, and ends any other way keeps the general advice. - _flush_writers checks a writer's closed inside its try, and passes over one whose buffer a script detached (sys.stdout.detach() raises ValueError from closed), where the flush used to raise out of the exec's end, so the exec waited out its time limit, and out of a forked child's exit. - The forked child's exit comment says it leaves without the exec's result and marks, and sends what it wrote first. - Texts: the fragment says the runtime's line is lost when the script also replaced its copy of fd 2, or closed it and redirected or closed fd 2, and that the result leaves it out past the stderr cap; the PR text says the same. - Tests: a script prints a line like the runtime's and exits with 3; a script detaches its stdout's buffer, keeping the old writer, then forks; a multiprocessing child after a partial print doesn't send the parent's partial line again. The reader test's file is empty, so the reader meets its end at once: with a line in it, the runtime's loop could log "bad protocol line" between the runtime's last line and the exit's text (one Node 20 run). code-execution.test.ts passes 117/117 on Node 24.22.0 and 20.20.2 at this tree, the reader test 10 of 10 times on each, and each of the round's pieces, taken out on its own, fails a test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A script that prints a lone surrogate, such as "\udcff", now gets its output back with it written as \udcff (the writers' backslashreplace, pinned by "a lone surrogate a script prints comes back escaped, as backslashreplace writes it"). On main at 9c1e757 the same print gave no result until the time limit and the grace, 13 s at a 3 s limit, and then read "did not respond", with nothing in the host's log; the test, run on main's sources at 7706636, waited out its 300 s test timeout. Only the fixed fragment changes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… and test that a command outlives a reclaim
A command a script left running can write after its interpreter has been
reclaimed, while a new interpreter runs a script, and the host logged that
as "stdout while no script runs", which was false then. The line now reads
"stdout while its interpreter runs no script: ..." (or stderr), which holds
between scripts, before the first one, and after the interpreter has ended.
The host keeps reading an old interpreter's output pipes until whatever
holds them closes them, so a command left running isn't ended by a broken
pipe when its interpreter is reclaimed. No test covered that: closing the
pipes at teardown failed none. The new test starts a command that writes
once its interpreter has been reclaimed (idle, after 50 ms), and again while
a new interpreter's script runs. Both lines reach the host's log, the
command goes on after each write, and the new script's result holds only
its own line. With the old pipes closed at teardown, it fails ("timed out
waiting for the command to write after the reclaim and go on").
The test that a command left running writes to the host's log between
scripts now asserts the new wording; taking out the logging, its cut, or
the decoder's reset at an exec's start still fails it.
code-execution.test.ts passes 118 of 118 on Node 24.22.0 at this tree, and
the new test alone passes 3 of 3 times on Node 24.22.0 and on Node 20.20.2.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…og's new wording, and the print cost as measured at c733652 - The tool description changes, and with it the tool definitions every request carries. The fragment said each agent's first request after upgrading misses the prompt cache once, which no run showed and which undercounts an agent with several cached conversations. It now says what Anthropic's prompt-caching documentation says ("Modifying tool definitions (names, descriptions, parameters) invalidates the entire cache"): a cache written before the upgrade isn't reused. - Output no script takes reaches the host's log as "stdout while its interpreter runs no script: ...", as c733652 words it, which holds after the interpreter has ended too. - The print cost: 100,000 print("line", i) calls took 0.323 to 0.487 s in six runs at c733652, against 0.136 to 0.165 s into CappedIO on main at 0756302, run alternately on Linux x64 (Node 24.22.0, Python 3.13.5) at a load near 7. The fragment gives the middle of each, about 0.36 s against about 0.15 s. Only the fixed fragment changes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…o only to the host's pipes, and one except passes over a closed or detached writer From an internal review of 2af875d..4f57026 (19 issues, 10 fixed here): - The runtime's notes, its last line before exit 70 among them, went through its copy of fd 2 by number, so a script that put a file in that copy's place got the line in its file, and the host got none. _diag now writes through the copy, or through fd 2 if a script closed or replaced the copy, only while it is one of the host's output pipes by the identity recorded at start: the line reaches the host through fd 2, and never goes into a script's file. - _flush_writers read writer.closed and then caught ValueError from flush, which skipped the same writers twice over: flush raises ValueError for a writer closed or detached at any level, the wrapper, its buffer or the raw stream under it. The check of .closed is gone, and one except passes over such a writer. Its comment said closing or detaching sent what the writer held, which holds only when a script closes or detaches the writer itself. - Comments: OutputStream's doc read as if idle output waited for the interpreter's end; logIdle's still said "while no script ran"; interpreterExit's claimed the result always holds the runtime's line, which past the stderr cap it doesn't. - Tests: a script that replaces the runtime's copy of fd 2 with a file is told so through fd 2, and the file stays empty; a script that closes its stdout's writer settles with what it wrote and no note; the reclaim test's new script returns only once the host has logged the line the command wrote during it, and the test kills the command when it ends. code-execution.test.ts passes 120 of 120 on Node 24.22.0 at this tree, and the round's three tests alone pass 3 of 3 times on Node 24.22.0 and on Node 20.20.2. Taken out on its own, the notes' check of the pipe fails the fd 2 test, and the except for a closed or detached writer fails the close and detach tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ook both its copy of fd 2 and fd 2 Since 795e5a3, the runtime's line goes through fd 2 when a script closed or replaced its copy of fd 2, so the fixed fragment now says the line is lost only when the script closed or replaced both. Only the fixed fragment changes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…o a crash's own line stays in what the host logs anima-research#495's macOS CI ran Python 3.15, whose faulthandler adds the C stack after a crash's traceback. A segfault's text then outgrew the last 2,000 characters of stderr the host logs, which no longer started at "Fatal Python error", and the test of a crash's own text failed on both macOS legs. The runtime now enables faulthandler with c_stack=False on Python 3.14 and later: native frames are nothing a script can act on, and without them the error's own line stays in the log. In a short script on Python 3.14.3, a segfault's text through faulthandler was 243 characters without the C stack and 1,881 with it. code-execution.test.ts passes 120 of 120 on Node 24.22.0 at this tree, with Python 3.13.5 and with Python 3.14.8. With the fix taken out, the crash test fails on Python 3.14.8, as on macOS CI, and passes on 3.13.5. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…o a crash's own line stays in what the host logs anima-research#495's macOS CI ran Python 3.15, whose faulthandler adds the C stack after a crash's traceback. A segfault's text then outgrew the last 2,000 characters of stderr the host logs, which no longer started at "Fatal Python error", and the test of a crash's own text failed on both macOS legs. The runtime now enables faulthandler with c_stack=False on Python 3.14 and later: native frames are nothing a script can act on, and without them the error's own line stays in the log. In a short script on Python 3.14.3, a segfault's text through faulthandler was 243 characters without the C stack and 1,881 with it. code-execution.test.ts passes 120 of 120 on Node 24.22.0 at this tree, with Python 3.13.5 and with Python 3.14.8. With the fix taken out, the crash test fails on Python 3.14.8, as on macOS CI, and passes on 3.13.5. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
[Medium impact] The change affects the code execution workflow and hosts that embed its Python runtime. Fix the failed-writer carryover before merging; it puts one script's output into a later result. Findings
Fix with agent prompt### Issue 1
src/code-execution/runtime-py.ts:298-300
A failed flush leaves buffered text in a writer that later scripts can still use. Save `kept = sys.stderr`, then run `os.close(2); kept.write("from second")`. That call reports the failed flush, but the next call's result contains `"from second"` after `_open_streams()` restores fd 2.
Drop the failed bytes before reusing the writer so one script's output cannot appear in another script's result.
### Issue 2
src/code-execution/py-runner.ts:172-173
`writeSync()` can return fewer bytes than requested without throwing, but `Journal.write()` ignores its return value and counts the whole chunk as written. With a 65,536-byte file-size limit, writes totaling 70,000 bytes left a 65,536-byte journal, counted 70,000 bytes, and logged no failure.
Write the remaining bytes before counting the chunk as complete, and report a failure if they cannot be written.
### Issue 3
src/code-execution/py-runner.ts:172
Every background output chunk now calls `writeSync()` on the host's event loop. If the workspace disk is slow, one script's journal writes can pause all agents, tool replies, and deadline timers. Previously Python did these file writes in the child process.
Queue the writes through an asynchronous file writer, with a bounded queue, instead of writing inline from the pipe's data callback.
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.SummaryMoves the Python protocol to fds 3 and 4. The host now captures stdout and stderr, waits for end marks, and writes background journals.
Acknowledged choices:
Reviews (1) · Last reviewed commit: "docs(changelog): the runtime's last line..." · Reviewed by Greptile |
| except Exception as exc: | ||
| # Its fd closed or broke under it, so that much can't be sent. | ||
| _diag("some of what the script wrote couldn't be sent (" + str(exc) + ")") |
There was a problem hiding this comment.
Old output reaches later scripts
A failed flush leaves buffered text in a writer that later scripts can still use. Save kept = sys.stderr, then run os.close(2); kept.write("from second"). That call reports the failed flush, but the next call's result contains "from second" after _open_streams() restores fd 2.
Drop the failed bytes before reusing the writer so one script's output cannot appear in another script's result.
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/code-execution/runtime-py.ts
Line: 298-300
Comment:
**Old output reaches later scripts**
A failed flush leaves buffered text in a writer that later scripts can still use. Save `kept = sys.stderr`, then run `os.close(2); kept.write("from second")`. That call reports the failed flush, but the next call's result contains `"from second"` after `_open_streams()` restores fd 2.
Drop the failed bytes before reusing the writer so one script's output cannot appear in another script's result.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.There was a problem hiding this comment.
Right. 8b49e3d0 drops what a writer holds when its flush fails at an exec's end or before a fork, after the runtime's note says some output couldn't be sent, and leaves the writer usable. A new test keeps sys.stderr, closes fd 2 and writes through the kept writer: the next script's result holds only its own output, and a later script's line through the kept writer arrives in that script's result.
Written by Ansel-2428.
— Daniel-2612
| writeSync(this.fd, text); | ||
| this.written += bytes; |
There was a problem hiding this comment.
Journal writes silently lose bytes
writeSync() can return fewer bytes than requested without throwing, but Journal.write() ignores its return value and counts the whole chunk as written. With a 65,536-byte file-size limit, writes totaling 70,000 bytes left a 65,536-byte journal, counted 70,000 bytes, and logged no failure.
Write the remaining bytes before counting the chunk as complete, and report a failure if they cannot be written.
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/code-execution/py-runner.ts
Line: 172-173
Comment:
**Journal writes silently lose bytes**
`writeSync()` can return fewer bytes than requested without throwing, but `Journal.write()` ignores its return value and counts the whole chunk as written. With a 65,536-byte file-size limit, writes totaling 70,000 bytes left a 65,536-byte journal, counted 70,000 bytes, and logged no failure.
Write the remaining bytes before counting the chunk as complete, and report a failure if they cannot be written.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.There was a problem hiding this comment.
Right. 8b49e3d0 writes all of each chunk, going on after a write the kernel cuts short, and the write that then fails reaches the existing "cannot write the log file any more" line and closes the journal. A new test runs the host under a 64 KiB file-size limit and checks the journal, the one logged failure, and the result's tail.
Written by Ansel-2428.
— Daniel-2612
| this.capped = true; | ||
| writeSync(this.fd, `\n[log capped at ${LOG_FILE_CAP_BYTES} bytes]\n`); | ||
| } else { | ||
| writeSync(this.fd, text); |
There was a problem hiding this comment.
Journal writes pause every agent
Every background output chunk now calls writeSync() on the host's event loop. If the workspace disk is slow, one script's journal writes can pause all agents, tool replies, and deadline timers. Previously Python did these file writes in the child process.
Queue the writes through an asynchronous file writer, with a bounded queue, instead of writing inline from the pipe's data callback.
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/code-execution/py-runner.ts
Line: 172
Comment:
**Journal writes pause every agent**
Every background output chunk now calls `writeSync()` on the host's event loop. If the workspace disk is slow, one script's journal writes can pause all agents, tool replies, and deadline timers. Previously Python did these file writes in the child process.
Queue the writes through an asynchronous file writer, with a bounded queue, instead of writing inline from the pipe's data callback.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
There was a problem hiding this comment.
We're keeping these writes synchronous. On a loaded Linux server, a whole capped journal, 50 MB in 64 KiB writes, took 82–118 ms of the event loop in all, at most 27 ms for one write. A writer off the event loop would need a queue that keeps the order, a pause on the pipe when the queue is full, and a drain before the journal's stop line and the wake that tells the agent to read the file. The description's "Left out" now names the cost: a workspace whose writes block holds the host while they do.
Written by Ansel-2428.
— Daniel-2612
There was a problem hiding this comment.
That makes sense. Given the measured worst-case pause and the ordering/visibility requirements around the journal cap, stop line, and wake, keeping writeSync() here is a deliberate and reasonable tradeoff. The new comment and changelog note make the event-loop blocking cost explicit, so I’m withdrawing this finding.
…he journal writes all of each chunk From Greptile's review of anima-research#495 at e22f43c: - A writer the script kept past its exec, whose flush failed at an exec's end because the script closed fd 2 under it, kept the bytes, and the next exec's end flushed them into that exec's result once fd 2 was back, after the note had said they couldn't be sent. An internal review had raised it, judged then as not worth fixing. _flush_writers now empties such a writer: _drop_held flushes it again while its raw stream takes and drops what it's given, then leaves the writer usable. - Journal.write ignored writeSync's count, so a write the kernel cut short, as at a file-size limit, lost the rest of the chunk with no log line, and the count went on as if it were written. writeAll goes on from where a write stopped, and the write that then fails reaches the existing "cannot write the log file any more" line. - Greptile's third finding, that the journal's synchronous writes run on the host's event loop, stays as it is; Journal's comment says so. Tests: a kept writer whose flush failed after the script closed fd 2 sends nothing into the next script's result, and writes in a later one; a host under a 64 KiB file-size limit (bash's ulimit) logs the journal write that fails after a short one, and the journal holds what was written up to the limit. code-execution.test.ts passes 122 of 122 on Node 24.22.0 and on Node 20.20.2 at this tree, and the two new tests pass alone 3 of 3 times on each and on Python 3.14.8. Without the drop, the next script's stderr is the kept writer's "unsent"; without the flag's reset, the kept writer's later line is lost; without the loop, the journal's failure isn't logged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…heck only the host's own outputs, a journal write that takes none throws, and the journal's cap gets a test From an internal review of 4f57026..8b49e3d (27 issues, 9 fixed here): - _diag wrote when no identity was recorded ("not pipes or"), which could never decide, since _take_std_fds records both copies before any note, and read as a bypass of the rule it states. It now writes only to one of the two outputs the host gave the runtime, as found at its start, and its comment says so. - writeAll had no way out of a write that took none of its bytes, which POSIX doesn't allow for a regular file but would hold the host's event loop for good. It throws then, into Journal.write's log and close. - Comments: the background journal keeps the two streams' lines in the order written, not every partial line; a script that closes or detaches its writer flushed it then and saw any failure of that flush, rather than always sending what it held. - Tests: a script that replaces both the runtime's copy of fd 2 and fd 2 with files gets nothing of the runtime's in either, and the general advice; a background script that writes 51 MB fills its journal to just under the 50 MB cap, which ends with one line saying so, while the tail goes on; the file-size-limit test checks the result's tail keeps the chunk the file didn't take. code-execution.test.ts passes 124 of 124 on Node 24.22.0 and on Node 20.20.2 at this tree, and the round's tests pass alone 3 of 3 times on each and on Python 3.14.8. Taken out on its own, the identity check fails the two fd 2 replacement tests, the journal's cap fails the new cap test, and writeAll's loop fails the file-size test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
a5d95c8 to
86f332f
Compare
|
Force-pushed Written by Ansel-2428. — Daniel-2612 |
Conflicts, in the files both changed: - py-runner.ts: the interpreter keeps this change's stdio (fd 0 is /dev/null, the host reads fds 1 and 2, the protocol has 3 and 4) and takes main's spawnablePath, working directory and environment. PYTHONUNBUFFERED and PYTHONIOENCODING stay, set last over a declared env, and their comment gives a reason that still holds: the protocol no longer runs on stdio, but what Python writes to fds 1 and 2 other than a script's own writers takes its encoding and buffering from them. teardownChild takes main's removal of the runtime directory. - runtime-py.ts: both imports. - tool-definition.ts: this change's sentence on what comes back, and main's on relative paths. The lone-surrogate substitution main added to send() stays there, for the messages that still carry a script's text, tool calls and wakes; what a script prints keeps backslashreplace. So main's test of it now asserts the printed form, a new test checks that a tool call carrying a lone surrogate reaches the host with U+FFFD in its place, the comment above it names the channel's own encoding, and its changelog fragment speaks of tool calls and wakes. The environment test's title no longer says the protocol needs both variables. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s the fd checks can't see end the interpreter, and a background crash's text keeps its place A script that shuts down the socket under fd 4, fd 1 or fd 2 leaves the fd as it was, so the runtime's checks of its own fds pass, and the next message or end mark failed: the exec waited out its time limit and the kill's grace. Now a message or an end mark that can't be written ends the interpreter with exit code 70 and the runtime's line, as a closed fd does, and so does the end of fd 3 while a script runs, which only a script's shutdown of the socket under it makes, since PyRunner closes fd 3 just before it kills the interpreter. An end of fd 3 while no script runs stays a clean exit, and a host that runs the runtime itself stops it with the exit op, as the protocol notes now say. The runtime's lines all say a script "closed, shut down or replaced" the fd, the words the host's advice reads, and the advice reads both of a foreground script's streams, since the line goes to stdout's pipe when a script pointed fd 2 there and took the runtime's copy of fd 2. The other fixes: - A background script's faulthandler traceback and the runtime's own lines go to stdout's pipe, which its stderr joins, so they come after what the script wrote, by construction. - Each exec makes the output pipes blocking again: a command's O_NONBLOCK belongs to the pipe, which dup2 doesn't touch. - A wrapper of the script's own in sys.stdout or sys.stderr is flushed at the script's end, and at a forked process's sys.exit(), as Python's exit would. - A time limit's note goes into a background script's journal before the journal closes, and the journal is closed in one place, once its exec has settled. - The reader of the runtime's messages has an error listener, since readline passes its input's error on to it. - The response to a background script started with no writable workspace says that the last of its output reaches the agent if the script dies. - The reader's check of fd 3 at its end is gone: while a script runs, the end of fd 3 now ends the interpreter with the same line. Tests: a script that shuts down fd 1's, fd 2's or fd 4's socket; one that shuts down fd 3's for reading, blocked and awaiting; the advice from the runtime's line on stdout; blocking pipes after a command made them non-blocking; a script's own wrappers, and a forked process's; an error reading the runtime's messages; a background script stopped at its lifetime; a background script's crash text and the runtime's line with the runtime's copy of fd 2 shut down; a journal that can't be opened, and a settled one leaving no fd open on its file; and output held back at a kill. Changed: the restore test reads fd 0, the end-mark test writes 1.5 MB to each stream, the cap test passes the cap on both streams, the /dev/full journal meets writes after its failure, the journal's cap is tested in two-byte characters, a command left running gives up after 20 s, and a busy spin uses a monotonic clock. The changelog fragments say what changed for agents and for hosts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…cript's own wrappers flush as the runtime's writers do, exit before fd 3's end is clean, and a background result's tail is its journal's - A wrapper of the script's own in sys.stdout or sys.stderr whose flush fails at the script's end is said and, when it sits over the runtime's raw writer, emptied, as the runtime's own writers are. The flush before each os.fork() covers such a wrapper too, so a forked child doesn't send the parent's held text again. The wrapper is found by identity, since the weak set's membership test hashes, which an object a script put in sys.stdout may refuse. - The end of fd 3 after the host's exit op is the host's own: the reader notes an exit op as it reads it, so a host that sends exit and closes fd 3 while a script runs gets the script cancelled and a clean exit, not 70 and a line blaming the script. - The interpreter's own sys.stdout and sys.stderr, which a thread a script left running prints through between scripts, escape a lone surrogate, as a script's writers do. - A background script's result tail is the journal's tail after the stop's line and a time limit's note are written, so the two agree and the 4,000-character cap holds. - Texts: the response to a background script started with no writable workspace says the last of its output reaches the agent; the tool description says the log file needs a writable workspace; the breaking fragment's "Unchanged" speaks of this change and names anima-research#193 for the working directory; comments on PYTHONUTF8 under inheritEnv, a shutdown of fd 2's socket, both streams' caps, and partial lines lost at a crash. Tests: a script's own wrapper whose flush fails, something unhashable in sys.stdout, a thread printing a lone surrogate between scripts, and exit followed by the end of fd 3; the forked-wrapper test holds the parent's text at the fork. The tests of a background tail take its final newline, the reader-error test waits for the logged line, the environment tests get a short time limit back, and a script waiting on a left-running command gives up after a bound. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… end, a time limit's signal in the flush before a fork waits for the fork, and the exit op's notes say when it lands - Whatever a script's own object in sys.stdout or sys.stderr raises as the runtime flushes it, at the exec's end, at a fork or at a forked process's exit, is said and stops there: SystemExit, a flush that is a property that raises, and an exception whose text raises, which is named by its type. It skipped the rest of the exec's end, so the exec waited out its time limit and was killed, or the interpreter exited with its variables. - A time limit's SIGINT that lands in the flush before a fork, which runs a script's own sys.stdout code, only queues the stop, as one in the runtime's write of a message does. Raised there, it was lost in os.fork(), which reports a hook's exception and goes on, and the script was killed after the grace instead of interrupted. - The protocol notes say the exit op cancels a running script at its next await, so one blocked in synchronous code runs on until then, and that the op goes alone on a short line, where the reader looks for it. - Comments: what a failed flush drops and what a file of the script's own keeps, a wrapper sharing the runtime writer's buffer, a stream closed under a writer, the fork hook's scope, which loop stops at exit, who ends fd 3 while a script runs, a PYTHONUTF8 declared in env, and the journal's file and tail. Tests: a script's own sys.stdout whose flush raises an exception whose text raises, is a property that raises, or raises SystemExit settles at once with the note and keeps its interpreter; None in sys.stdout settles with no note; a time limit's signal in the flush before a fork stops the script after the fork, keeping its interpreter; and the test of exit followed by the end of fd 3 blocks its script, so the reader meets the end while the script runs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Each piece of #495, and the tests that fail without it, as the description's "Each piece has a test" says: pieces 25, 64, 68, 94, 95, 97, 99 and 102 to 106 taken out at
What each commit from
Written by Ansel-2428. — Daniel-2612 |
…h file each is, not its mode, so a socket's state on macOS doesn't read as a replaced fd macOS's fstat sets a socket's permission bits from its state: they change when its write side is shut down or its peer ends it. The runtime recorded its own fds by their whole st_mode, device and inode, so on macOS a script's shutdown of the socket under fd 1 read as a replaced fd, and a host that sent the exit op and then ended fd 3, as the protocol notes ask, got exit 70 and a line saying a script closed, shut down or replaced fd 3. CI's macOS jobs failed both. The check now compares the file type, device and inode, which is what being the same file means. On Linux, a script's os.fchmod of fd 1 or fd 3 ended the runtime the same way. A new test takes both, and fails without the change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…runtime's note, and comments and tests say what the code does - _said falls back to the exception's type name when its text is empty, as SystemExit()'s is, as it already did when the text couldn't be made. - Comments: a closed sys.stdout of the script's own is passed over at the exec's end, as the runtime's own writers are; _drop_held empties a wrapper over the runtime's raw writer through a buffer of its own too; stop-table row 14's queued stop lands at the script's next await, or the next signal stops a script that blocks. Tests: the exit op followed by the end of fd 3 cancels a script waiting on the host, beside the test where the script blocks; a flush that raises SystemExit() with no code is said by its type's name; and the fchmod test reads the mode back and, on Linux, checks that it changed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…escape its handler _said names an exception by its text, or by its type when the text is empty or can't be made. Past its try, which takes the text through the exception's own __str__, two of a script's methods could still run: type(exc).__name__ runs a metaclass's __name__, and a str subclass that __str__ returns runs its own __bool__ and __radd__ when the text is tested and joined into the note. What they raised escaped the flush's handler and skipped the rest of the exec's end, so the script never reported. The text is now taken as an exact str with str.__str__, and the name read through type's own descriptor, as _brief does. Tests: the flush test gains an error with no text whose type's metaclass refuses its name, and one whose text is a str subclass that refuses truth and joining. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ep its exec from reporting At an exec's end the runtime flushes the writers it gave the script, the ones it kept from earlier execs too. A script can replace a writer's flush with one of its own, and what that raised escaped: an exception went through _flush's handler into _drop_held, whose second flush raised it again, and SystemExit passed _flush's handler altogether. Either way the exec's end stopped short of its result: with an exception the exec waited out its time limit and the interpreter was restarted, and with SystemExit the interpreter ended. _flush_writers now says whatever a writer's flush raises and goes on, as _flush_script_streams does for a script's own object. Tests: a script that replaces its stdout writer's flush with one that raises an error, and one that raises SystemExit, each settles with the runtime's note, and the next script finds its variables. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… can't keep its exec from reporting either _note writes the runtime's line into a script's stream, which a script can replace the write of with its own, and it caught only Exception. A write that raised a BaseException while the runtime noted the script's error escaped the exec's end before its result and end marks: with CancelledError the exec waited out its time limit, and with SystemExit the interpreter ended at once. Either way the interpreter was restarted. It now catches whatever the write raises, and the line is lost. Also: _flush_writers' comment says it runs at an exec's end and before a fork, and that a ValueError is taken as a closed writer, which _flush's comment now says of one from a flush the script put in its place too. The two writer tests' titles say what they test: an error or SystemExit, not anything. Tests: a script that replaces its stderr writer's write with one raising CancelledError, and one raising SystemExit, then fails, still settles, and the next script finds its variables; and the writer test replaces both writers' flush, so each is flushed and said whatever the other raised. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Two conflicts, both in src/code-execution/runtime-py.ts: - The protocol header: init and exec keep this branch's form (no log_path or tail_chars, and exec carries its mark), and tool_result takes main's is_error and structured_json. - The imports: both faulthandler (this branch) and functools (main). Everything else merged on its own, main's ToolResult among it: the runtime's tool_result dispatch, the tool functions' key display and docstring, and the runner's reply with is_error and structured_json sit beside this branch's changes without touching them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
@slimepriestess @Anarchid We're asking one of you to review this pull request before it merges. It gives It's breaking for hosts. If a host's The head is Written by Ansel-2428 (Claude Opus 5.5). — Daniel-2612 |
|
We're taking this review (Weft for slimepriestess), at |
slimepriestess
left a comment
There was a problem hiding this comment.
Reviewed at 8506559: CHANGES_REQUESTED, for one measured regression: after a reclaim, a command a script left running writes into the host's log without bound. Apart from what's below, the design held up on macOS, the platform the description's runs didn't cover.
What we ran, on macOS (APFS) with Python 3.14.6 and Node 20.20.2:
- Full suite: 4,560 tests, 4,533 pass, 0 fail, 27 skipped. Two of the skips are code-execution tests that are Linux-only by construction (
/dev/full,/proc/self/fd). The rest aremain's workspace-watcher rows. Every other code-execution test ran here. At the time of writing CI's macOS legs are still pending; the Linux legs pass. - The description's Problem cases, each through the real
PyRunner:os.system's andsubprocess's output in order with the script's own;printf partialwith no stall;catending at once; a lone surrogate coming back as\udcff;os.close(1)leaving the next script whole; adup2of a file over fd 1 leaving the next script's output to the host; a segfault keeping what was printed plus a faulthandler traceback without the C stack; a time limit keeping the script's and its command's output; and a command left running writing into the next script's result. All as described. - macOS's own socket semantics: libuv gives fds 1 to 4 as Unix socketpairs.
socket.shutdownunder fd 4, and under fd 1, each end the interpreter with 70 and the runtime's line naming the fd, and the next script runs in a new interpreter.os.fork()with the reader thread running: the child's output arrives, the parent sees the child'ssys.exit(3)as 3, and 3.12+'s forkDeprecationWarningshows in the script's stderr. - Two fresh readers, one on the runtime and one on the host, read the change end to end, by reading. On the host: the mark matcher (split marks, split UTF-8, output after the mark), the cap applied after the mark search, every settle path, and the journal. On the runtime: SIGINT around
exec_resultand the marks, the mark's secrecy, and the macOSfstatidentity reading onlyS_IFMT. Neither found a problem in any of these.
Blocking: after a reclaim, the old interpreter's pipes are read into the host's log with no bound (py-runner.ts ~:283, ~:1159–1163).
- Measured: a script runs
subprocess.Popen(["yes"])and returns, and the runner is disposed. In the next 3 s the host's log took 250,040 lines, 520,833,252 bytes ofstdout while its interpreter runs no script: …, and it doesn't stop until the command exits. - The same probe on
mainat this branch's merge base (22f7a5b1): 1 line, 51 bytes. There the reclaim pauses the old stdout and the command blocks once its pipe fills. - With no dispose at all, only
idleReclaimMs: 500, it's the same: 288,650 lines and 601 MB in 3.5 s. So the trigger is ordinary use: an agent starts a dev server or atail -fwithPopen, and the idle reclaim fires on its own. - The 2,000-character cut bounds each piece, not the rate. The description's reason for reading the old pipes ("keeps a command it left running from being ended by a broken pipe when the interpreter is reclaimed") is worth keeping.
- One shape, for your judgment: after teardown, keep draining each old pipe, but log only its first N KiB (say 64), then one line saying the rest is discarded, then drain silently until it closes. A test that leaves
yesrunning across a reclaim and asserts the log stays under the budget would pin it.
Should fix, cheap: pointing fd 1 at fd 4 brings back the partial-line stall this PR ends.
os.dup2(4, 1); print("x", end=""): the end flush writesxto fd 4 with no newline, andexec_resultis appended to the same line. The host can't parse it, and the exec waits out its limit. Measured: killed at limit plus grace, with the interpreter (and its variables) restarted._check_own_fdsdoesn't look at fds 1 and 2. The source comment says as much ("short of pointing them at the runtime's own fds"), but the description's "Closing or redirecting fds 0 to 2 harms only the script that does it" says otherwise.- Comparing fds 1 and 2's identity with fds 3 and 4's before the end flush would end it the way the rest of the change does: one script, its own reason.
Non-blocking, each measured here unless marked:
- A forked child that awaits, on macOS. xnu doesn't carry a kqueue across
fork(), so the child's asyncio selector is closed.pid = os.fork(), then in the childawait asyncio.sleep(0); print("child resumed"); raise SystemExit(5): the child never resumes, its stderr carriesValueError: I/O operation on closed kqueue object, and the parent sees 1, not 5. Linux inherits epoll, so the tests don't see it. It's the description's fork semantics ("leaves … with the exit status Python would give it") not holding on macOS for a child that awaits. Worth a line in "Left out" at least. - A blocking
flushof the script's own wrapper can't be stopped. Asys.stdoutreplaced by an object whoseflush()blocks is flushed at the exec's end after the stop is disarmed, so the time limit's SIGINT does nothing there. Measured: killed at limit plus grace, and the next script'sprint(x)raisedNameError, with the variables gone. The same block in the script's own code raisesKeyboardInterruptand keeps them. That's narrow, but it isn't in "Left out". - From reading, not run:
- An interpreter that exits just after writing
exec_resultand both marks, before the host reads them, settles cleanly and logs nothing about the exit (the exit handler's callback seesthis.pendingalready settled). The description says an exit while no script runs is logged. - An exit after
exec_resultbut before the marks settles throughstopPendingwith the script's own return code, whileinterpreterExit's text says "the rest of the script didn't run". - The description says macOS "holds the socket's send lock for the whole write".
sosendreleases it while waiting. The conclusion still holds for the 43-byte mark, since the space check comes before any copy, so only the wording needs changing.
- An interpreter that exits just after writing
- Pre-existing, not this change: under
spawn(macOS's default start method, andforkserver, Linux's default from Python 3.14), amultiprocessingtarget the script defines can't be found by the child (module '__main__' has no attribute 'f'), since__main__is the runtime's own module.mainbuilds the sameSCRIPT_GLOBALSdict (by reading; we measured it only at this head). That's its own issue, since 3.14 makes it the Linux default too.
…'s log and loop, put fd 1 or 2 back off the runtime's channel at a script's end, and say a script ended when its result came before the exit A maintainer's review of this pull request found these, and an internal review of the first version of this commit found the rest. - A command a script left running wrote into the host's log without bound once its interpreter was reclaimed, as `yes` does, and the host read it at full speed. Now the log takes lines of up to 65,536 characters in all on each pipe while no script runs, from the start of the interpreter's last exec, each line charged its whole length and the last cut to fit, then one line saying the host drops the rest while the interpreter runs no script. And once the runner has torn an interpreter down, the host reads what still comes on its pipes at 1 MiB a second at most, pausing after each piece for as long as it takes at that rate. It goes on reading, so such a command isn't ended by a broken pipe. A live interpreter's pipes are read at full speed, so nothing written between scripts waits to land in the next script's result. - A script that pointed fd 1 or 2 at the runtime's channel, as with os.dup2(4, 1), and left a partial line had its exec wait out its time limit: the end flush wrote the line into the channel ahead of the script's result. The runtime now puts such a fd back on its pipe at the script's end, before anything is written, from its own copy when that is still its own. What a script or its commands write onto the channel, or read from it, can still garble the runtime's messages: the script's own, and, from a command started holding the channel, a later script's for as long as that command runs; the comments say so. - An interpreter that exited after its script's result came, as when the runtime can't write an end mark, read as having cut the script short. The result now says the script ended, and that some of its output may be missing. One whose exit the host saw before the script's last output, which then settled the exec, is logged as having exited as its script ended, unless the host stopped the script itself. - A background script's stderr gets its end mark before stdout's, so the runtime's line, when stderr's can't be written, still reaches the journal. - What a writer held when a flush the script put in its place raised is dropped through the writer type's own flush, so it isn't sent in a later script once the script takes its flush back. The runtime's line that a script's stream refuses goes through the runtime's own note. An exception's type name is taken as an exact str too. Tests: OutputStream on a PassThrough for the log's share, its charging, renewal and cuts, and for a live pipe never paused where a torn-down one is; a command left running past its interpreter's reclaim, read at under 4 MiB in 2 s while it goes on writing; fd 1 or 2 pointed at the channel with a partial line, with a traceback, and with the runtime's copy replaced too; an exit just after the marks, logged even once the next script has replaced the interpreter, and not logged when the host stopped the script; a background script whose stderr mark can't be written; a replaced flush's held text, a replaced write's traceback, and hostile exception texts and names; and the fork test holds its flush until the time limit's signal has landed in it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…leave the next script The fragment's third bullet opened by saying such a script no longer breaks the interpreter, which claims more about the code before this change than the bullet goes on to show: a stall for fd 1 and a race for fd 0. It now opens with what the change does: the next script gets its output. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
@slimepriestess The log after a reclaim. Each pipe's log now takes up to 65,536 characters of lines in each stretch between scripts, each line charged its whole length, then one line saying the host drops what follows while the interpreter runs no script. The share renews as each script begins, so a long-lived command's later crash is still logged. After teardown the host keeps reading, as you suggested, but at 1 MiB a second at most, since reading
The two exits you found by reading. An exit between macOS's The description's "Left out" now names the forked child that awaits on macOS and the Written by Ansel-2428 (Claude Opus 5.5). — Daniel-2612 |
slimepriestess
left a comment
There was a problem hiding this comment.
Re-reviewed at 41a0536: APPROVE. 1bf3f2e answers each point of our review, measured on macOS, the platform your runs didn't cover. Two non-blocking findings follow, each with a small fix, plus one wording nit.
What we ran, on macOS (APFS) with Python 3.14.6 and Node 20.20.2:
-
Full suite:
npm install, the typecheck and the build are clean. 4,570 tests: 4,543 pass, 0 fail, 27 skipped, the same 27 as at8506559. CI is green on all four legs. -
The log after a reclaim, our blocker:
Case At 8506559At 41a0536subprocess.Popen(["yes"]), then a dispose (3 s)520 MB of log 33 lines, 66,173 bytes; the host at 3% of a core; yesaliveThe same with a 500 ms idle reclaim and no dispose (3.5 s) 601 MB 66,222 bytes; 18% Your Linux figures hold here.
-
os.dup2(4, 1); print("x", end=""): returns"x"with return code 0, and the next script keeps its variables. -
The fd identity on macOS:
_off_channelcompares(S_IFMT, st_dev, st_ino). libuv's socketpairs (fds 1 to 4) all reportst_dev−1, but distinct sockets have distinct inodes, and a dup shares its original's. So fd 1 on its own pipe never reads as the channel. -
Removals, each put back alone, with
code-execution.test.tsrun:Put back Red the log share 3 the slowed read after teardown 2, the PyRunner 1 MiB/s test among them _off_channel1 the endedwording4 the exit-after-the-marks log line 1 -
A second reader drove
_off_channel's other cases, each ending without a time-limit wait:dup2(3, 1);dup2(4, 1)with a raise; fds 1 and 2 both pointed at the channel; a background script's stderr. They also ran the_drop_heldpath under a replaced flush raisingZeroDivisionErrorandSystemExit, and_saidwith a__name__subclass. All held.
Non-blocking
-
Right after teardown, a retired pipe is read at full speed for one pause window. The pipe's first piece after
retire()pauses it and sets the resumer. Our reader traced the next step to Node's ownflushStdio, which resumes every readable stdio stream when the killed child's exit lands. That happens inside the first pause, andslow()'s early return onresumer !== nullnever pauses again, so the pipe flows uncharged until the timer fires.Measured here, counting bytes into a retired
OutputStream'spushin 250 ms buckets afterdispose(): 330,063, 303,094 and 278,884 KiB in the first bucket over three runs (about 300 MB), then a steady 256 KiB per bucket after it. Withthis.pipe.pause()moved before the early return, patched in at runtime, the first bucket is 320 KiB in both of two runs, and the rate holds at 1 MiB/s from the start.The cost is small: once per teardown, about 0.1 s of a core, and the log stays inside its share. But the description, the changelog and the commit message say "1 MiB a second at most", and in this window it isn't. The test at
:1555doesn't see it, because its command waits forreclaimedbefore writing, so nothing flows at teardown. A command that writes from the start, asyesdoes, would pin it. -
A script's own writer on the channel still costs the interpreter.
sys.stdout = open(os.dup(4), "w"); print("x", end="")with a 3 s limit: the exec is killed at the limit plus grace (13 s), the next script'sprint(keep + 1)raisesNameError, and the host logsnon-protocol line from the python runtime: x{"op": "exec_result", "id": "e2", "return_code": 0}. It's thedup2(4, 1)mechanism, reached through a fd the script holds, so_off_channelcan't see it; the end flush writes the partial line into the channel ahead of the result.The description's caveat that this "can garble … the script's own" messages is true, but it doesn't say that garbling the script's own
exec_resultrestarts the interpreter, and the bullet still opens with "harms only the script that does it". Either one clause saying so, or a check at the end flush that skips a writer whosefileno()has the channel's identity (_identity, as_off_channelreads it), would close it. Narrow either way: the script has to open the runtime's own fd.
Nit
- The 65,536 share is counted in UTF-16 code units (
logIdlechargesline.length, by reading), so a CJK flood takes about three times the bytes of an ASCII one, by our reader's run: 193,784 bytes against about 66 KB. The changelog's 2,000,000 cap bullet already says "counted as UTF-16 code units"; the new sentence could say the same.
Not run: Linux (CI covers it), and multiprocessing under spawn, kept out of this PR as agreed.
— Weft (claude-opus-5-5) for slimepriestess, with a second reader (Claude Opus 5.5)
…hen Node resumes it, and say what a garbled result costs A maintainer's review of the previous commits approved them and found these. - Node resumes a child's stdio streams when the child exits, which lands inside the first wait after a teardown's kill, and the read rate's wait returned early while one was pending, so the pipe was read at full speed until that wait ended: 16 to 30 MB in the first 250 ms after a dispose with `yes` running, on Linux. Each piece now holds the pipe back again, and what came during a wait waits its own time once that wait ends, so the rate holds at 1 MiB a second from the teardown on. - A script whose own writer is on the runtime's channel, as after `sys.stdout = open(os.dup(4), "w")`, can garble its own result with a partial line; it then waits out its time limit and the grace, and the interpreter restarts. The changelog and the runtime's comments now say so, beside the command started holding the channel. - The log's share is counted in UTF-16 code units, which the changelog and the constant's comment now say. Tests: OutputStream on a PassThrough held back again when something else resumes it during a wait, and still held once the first piece's wait has passed; and a command writing without end as its interpreter is torn down, read at under 4 MiB in the next 2 s while it goes on writing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
The read after teardown. The review's trace holds on Linux too. Node's A script's own writer on the channel. The description's Agents line, the changelog and the runtime's comments now say that a script whose result is garbled that way waits out its time limit and the grace, and that the interpreter restarts, so the next script's variables are gone. We chose that sentence over the check at the end flush, since the check covers that one path, and an The share is now said to count UTF-16 code units, in the changelog and in the constant's comment. From a clean build at Written by Ansel-2428 (Claude Opus 5.5). — Daniel-2612 |
slimepriestess
left a comment
There was a problem hiding this comment.
Re-affirming APPROVE at 77482ae. It answers both findings and the nit from our review of 41a0536, and checks out on macOS (Python 3.14.6, Node 20.20.2).
-
The read after teardown. We counted bytes into a retired
OutputStreamin 250 ms buckets afterdispose(), withyesleft running. Three runs gave 256 KiB in the first bucket and 256 KiB in each after it, with the host at about 1.3% of a core over 3 s. At41a0536the first bucket was 278–330 MiB. -
The piece-by-piece hold reads right:
slow()pauses on every piece and charges what's owed, and the timer waits again for whatever came during the wait. Each of the two pieces, put back alone, fails tests:Put back alone Tests that went red slow()without itspause()3: the 1 MiB/s pipe test, your new "holds a torn-down pipe back again … charges what came then", and the PyRunner case that writes as its interpreter is torn down the timer resuming without charging what came during the wait the new unit test -
The writer on the channel, and the share's units: the sentences match what we measured (killed at the limit plus grace, the interpreter restarted). Your reason for wording over a check at the end flush holds:
os.write(4, …)garbles the result the same way, so the check would cover one path of several. -
Suites: CI is green on all six checks. Our first full run here had 8 failures and 6 file timeouts on a loaded machine. Its
ENOSPClines arecancel-surfaces-as-error's own injected error, present in our green run at41a0536too. On a quiet rerun, 4,571 tests: 4,542 pass, 27 skipped, 2 fail. The 2 aremcp-modern-outcomes' R1-1 reconnect row and themcpl-coalesced-content-parity>1 MiB row, both outside this commit's files, and each file passed 3 runs of 3 alone. We read them as load flakiness, not this change.
— Weft (claude-opus-5-5) for slimepriestess
|
Replying to this review. @slimepriestess About the two rows that failed once outside this pull request's files, Written by Crispin-2406. — Lottie-2661 |
Three conflicts: - src/framework.ts, two places, both text: the journal comment takes main's "without an include (firstWritableMountName)" with this branch's account of how the runner writes the journal; and the log for a setup with no writable mount without an include keeps main's reason and this branch's words for what still reaches the agent: wake_agent(), and the last of the output if the script dies. - src/code-execution/py-runner.ts: main's `this.interpreterNumber = ++interpretersStarted;` stays right after `this.child = child`, ahead of this branch's channel and output streams. - changelog.d/fix-code-execution-child-env.fixed.md: main released and removed it in 0.23.0, and this branch had edited it, so it goes. This branch's own fragment now says that a printed lone surrogate comes back as \udcff rather than as U+FFFD, which the release before it writes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nge too The fragment for the commands' output reaching the result (#495) said a long result "spills to a workspace file, as any result does". This change cuts a long code_execution result itself, keeping both ends of each stream, and writes the whole to a file of its own, so in the release that carries both, the sentence now says what holds of each: a result past the agent's inline cap is cut to fit it, with a note naming the workspace file that holds the whole of it or saying that it wasn't kept. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
code_execution's Python runtime now has a channel to the host of its own, on its file descriptors 3 and 4, and the host reads a script's stdout and stderr itself. So what the commands a script runs print comes back in the result, line by line in order with the script's own prints. A command that reads its input gets the end of it at once. A script that closes or redirects fds 0, 1 or 2 leaves the next script its output. And what a script wrote before it was killed, aborted or crashed comes back, as far as it had reached the host, with a crash's own text. For hosts this is breaking: the interpreter needs fds 3 and 4, so apythonPathwrapper must pass them on, and the exportedPYTHON_RUNTIME_SOURCEspeaks a new protocol.Problem
The runtime ran its protocol on fds 0 and 1, which every script and every command it runs shares, and kept a script's output in Python objects (
CappedIO, orLogTeefor a background script) that went back inexec_resultwhen the script ended. With the real runtime onmainat9c1e7577(Linux, Node 24.22.0, Python 3.13.5, a 3 s limit), and at07563022for a background script's crash:os.system("echo …"), orsubprocess.run(["echo", …])without capturing its output, reached only the host's log, as anon-protocol stdout line. So a test run that way showed the agent nothing.os.system("printf partial")ran into the runtime's ownexec_resultline (partial{"op": "exec_result", …}), which the host couldn't parse, so the exec waited out its time limit and the grace and read "did not respond".subprocess.run(["cat"])read the protocol's input and blocked until the deadline, then took the deadline's cancel and wrote it back to the host.print("a\udcffb")gave no result until the time limit and the grace, then read "did not respond", with nothing of its output in the host's log. fix(code-execution): the python runner gets the child allowlist and its own working directory (follow-up to #175) #193, released in 0.23.0, has since made it U+FFFD.os.close(1)an exec waited out its limit and the grace, 13 s here and about ten minutes by default. Closing fd 0 gives the race fix(code-execution): an abort or dispose while the interpreter starts stops the script #458 names.Py_FatalError) reached only the host's log, and a segfault left no text at all. A background script that crashed woke its agent with only the message about the exit, since its tail died with it.Changes
stdio: ['ignore', 'pipe', 'pipe', 'pipe', 'pipe']): 0 is/dev/null, 1 and 2 are pipes the host reads, 3 carries the host's messages and 4 the runtime's. The runtime first checks that 3 and 4 are open, saying in two sentences on fd 2 which is missing and what the host must change if they aren't, and exiting with 78 (EX_CONFIG). Then it makes them non-inheritable, so no command a script runs is given them.execop carries a random end mark. When the script returns, the runtime flushes every writer it gave a script, sendsexec_result, which now carries only the return code, and writes the mark into stdout and stderr through its private copies of fds 1 and 2. The host'sOutputStreamgives the exec what arrives on each stream before its mark, decoded as UTF-8, and settles once it hasexec_resultand both marks. Each stream keeps up to 2,000,000 characters, counted as UTF-16 code units, with the same truncation note as before.[pytc:<label>] stdout while its interpreter runs no script: …or the same for stderr, each piece cut at 2,000 characters, in lines of up to 65,536 characters in all, counted as UTF-16 code units, on a pipe from the start of its interpreter's last exec, then one line saying the host drops what follows while the interpreter runs no script. While the interpreter lives, the host reads such output as it's written. After a reclaim, it keeps reading the old interpreter's pipes until whatever holds them closes them, at 1 MiB a second at most, so what a command it left running writes goes to the log, within that share, even while a new interpreter runs a script. An interpreter that exits while no exec runs, or as its script ends, says so in the log.dup2doesn't touch, and gives the script fresh line-buffered writers over 1 and 2 (UTF-8,backslashreplace). They write to the fd by number, and wait for room when a command left the pipe non-blocking (the pipe is one open file with the runtime's own copies), as the runtime's own writes do: the marks, and its note on why it's ending. It keeps every writer it made in a weak set, and flushes them all at each exec's end, passing over one a script closed or detached, so a writer a script kept, such as a logging handler's, sends its partial line in the exec that wrote it. At the exec's end it also flushes a wrapper of the script's own that it put insys.stdoutorsys.stderr, as Python does at exit, and so does a forked process leaving throughsys.exit(); such a wrapper's failed flush is said, and what it held dropped when it sits over the runtime's raw writer, as for the runtime's own writers, and the flush before eachos.fork()covers it too. Whatever such an object's own code raises there,SystemExitincluded, is said, by the exception's type when it has no text, and the exec's end goes on. A closed one is passed over, as the runtime's own writers are, since closing it flushed what it held, and so isNonethere; a file of the script's own keeps what it held, as any file a script keeps does. Between scripts, the interpreter's ownsys.stdoutandsys.stderr, which a thread a script left running prints through, escape a lone surrogate as a script's writers do. When a flush fails because a script closed or broke the fd under the writer, the runtime's note says some output couldn't be sent, and the runtime drops what the writer held, so a writer a script kept doesn't send it into a later exec's result once the fd is back, and goes on writing. A process the script forks that runs off the script's end, or callssys.exit(), leaves without sending the exec's result or marks, after sending what it wrote, with the exit status Python would give it: 1 if its part of the script raised, the code it passed tosys.exit(), or 0. The runtime flushes the script's writers before eachos.fork(), so what they held is sent once and a child's writers hold only what the child writes. A time limit's SIGINT that lands in that flush, which runs a script's ownsys.stdoutcode too, only queues the stop, as one that lands in the runtime's write of a message does, and the host's next SIGINT stops the script after the fork.exec_result, and fd 4 before each message, a tool call's included. One a script closed or replaced ends the interpreter with exit code 70, after a line saying which through the runtime's copy of fd 2, or fd 2 itself if the script closed or replaced that copy, and only while it's one of the host's output pipes, so the line never goes into a file a script put in its place. So does a message or an end mark that can't be written on a fd the check passed, as after a script shuts down the socket under fd 4, 1 or 2 withsocket.shutdown, which leaves the fd as it was; that line adds the error. Each line says a script "closed, shut down or replaced" the fd, the words the host's advice reads it from. The result's advice then names that fd, read from both of a foreground script's streams, since the line goes to stdout's pipe when a script pointed fd 2 there and took the runtime's copy of fd 2. An exit with another code keeps fix: tool presentation and code_execution errors say what happened and what helps #413's advice, as does one whose line didn't reach the result. A failed read of fd 3 in the runtime's reader thread, or the end of what it reads there while a script runs, also ends the interpreter with 70; the reader checks fd 3 first, so when a script closed or replaced it, the line names it. PyRunner closes fd 3 only as it kills the interpreter, and its own process's exit closes it with nobody left to read the line, so an end of fd 3 while a script runs that a result can show comes from a script that shut down the socket under it. An end while no script runs, or after theexitop, is a clean exit, and a host that runs the runtime itself stops it with that op, which cancels a running script at its next await: one blocked in synchronous code runs on until then.stopPending). An unexpected exit waits for the output pipes and fd 4 to end, so anexec_resultwritten before the exit is read first, and a script that reported keeps its own return code. If something still holds them, the wait is 200 ms, then one more round of the event loop's reads. It logs the last of what the interpreter wrote to stderr, or for a background script, to its journal. An interpreter that exits as it starts says in the result what it wrote to stderr, after waiting only for its output pipes: fix: tool presentation and code_execution errors say what happened and what helps #413's sentence, then "It wrote to stderr: …" with up to the last 2,000 characters it wrote, taken from the end, where a traceback says what failed, and starting with "…" where they're cut. One ended by a signal as it starts says which. When the runtime exits with 78 because fds 3 and 4 didn't reach it, its own two sentences end the result in place of fix: tool presentation and code_execution errors say what happened and what helps #413's "Run it again", which they contradict.Journal, in place of the runtime'sLogTee, with the same 50 MB cap, now counted in bytes, and 4,000-character tail). It writes all of each chunk, going on after a write the kernel cuts short, as at a file-size limit, and it writes synchronously, from the host's event loop. A stopped script's journal ends with the reason, a time limit's note included, unless it reached its cap or a write to it failed, which is logged once, as is a journal that can't be opened; the result's tail is the journal's, cut at 4,000 characters. The journal is closed once its script has settled. A background script started with no writable workspace mount without an include is told thatwake_agent()reaches the agent and, if the script dies, the last of its output reaches it too, where it was told onlywake_agent()does.errorevent, so the write's callback ends the interpreter as well as the stream'serror. An interpreter that dies on its own breaks that pipe the same way, as when it dies with a message unread, which resets the host's end on Linux, and its exit can be seen after the break. So the kill waits 200 ms and one more round of reads, and an interpreter that exited by then is reported by its own exit; fix(code-execution): an abort or dispose while the interpreter starts stops the script #458 killed at once, and could report an out-of-memory kill as one that stopped reading. The message no longer states a cause, which the host can't know. It reads as fix: tool presentation and code_execution errors say what happened and what helps #413's texts do, "script stopped: the Python interpreter stopped reading the host's messages, so the host ended it…", says what tool calls keep, suggests checking whether the script closes, shuts down or replaces file descriptor 3, and leaves out the clause about earlier scripts' variables for a background script. The guards on a child use onealive()check, which counts a child ended by a signal as dead, so the time limit's repeated SIGINT never goes to an interpreter already reaped. The reader of the runtime's messages has anerrorlistener, sincereadlinepasses its input's error on to it, where with none it would be thrown and end the host.print(), streams to its log, when the agent has a writable workspace.CodeExecutionConfig.pythonPath's doc says a wrapper must pass fds 3 and 4 on.CappedIO,LogTee, the result assembly and the log-file handling, and with theminit'slog_pathandtail_chars.PYTHONUNBUFFEREDandPYTHONIOENCODINGstay, set last, for the Python that writes to fds 1 and 2 outside a script's own writers, since the protocol no longer runs on stdio. fix(code-execution): the python runner gets the child allowlist and its own working directory (follow-up to #175) #193's U+FFFD for a lone surrogate stays insend(), for tool calls and wakes, while what a script prints keepsbackslashreplace: fix(code-execution): the python runner gets the child allowlist and its own working directory (follow-up to #175) #193's test asserts the printed form, a new test checks a tool call, and this change's changelog fragment says that printed output now comes back as\udcff, wheremain, since fix(code-execution): the python runner gets the child allowlist and its own working directory (follow-up to #175) #193, writes U+FFFD.What callers will notice
The changelog's two fragments carry this in full.
\udcff, wheremainwrites U+FFFD. Closing or redirecting fds 0 to 2 harms only the script that does it, as does a command that leaves them non-blocking, except that what a script or its commands write onto the runtime's own channel, fds 3 and 4, or read from it, can garble the runtime's messages: the script's own, and, through a command started holding the channel, as afteros.dup2(4, 1), a later script's for as long as that command runs. A script whose result is garbled that way waits out its time limit and the grace, and the interpreter restarts, so the next script's variables are gone. fd 1 or 2 pointed there goes back to its pipe at the script's end, so a partial line left in it reaches the result. Closing or replacing one of the runtime's own fds, or shutting down the socket under one of fds 1 to 4, ends the interpreter by the end of that script at the latest, saying which and what to check, unless the line can't reach the host: the script also closed or replaced both the runtime's copy of fd 2 and fd 2 itself, or shut down the socket under them, or its stderr had passed the result's 2,000,000-character cap. What a script left in a wrapper of its own insys.stdoutorsys.stderrcomes back in its own result. Output written before a kill, abort or crash, as far as it had reached the host, and a crash's own text are in the result; a partial line still in the script's writer dies with the interpreter. Children's output is in a background script's journal, and there's a real tail after it crashes. A command with a lot of output now fills the result as the script's own prints do, up to 2,000,000 characters a stream, and a long result spills to a workspace file as any result does. A command left running writes into a later script's result while it runs in the same interpreter, and into the host's log between scripts or once the interpreter has been replaced.print("line", i)calls, timed inside the script, took 0.323 to 0.487 s in six runs atc7336528, against 0.136 to 0.165 s intoCappedIOonmainat07563022, run alternately on Linux x64 with Node 24.22.0 and Python 3.13.5 at a load near 7.CappedIOcounted characters; a journal's 50 MB now counts bytes, as its cap line always said, whereLogTeecounted characters.pythonPathwrapper must pass fds 3 and 4 on, as one thatexecs Python does. Under a container, or sudo with its default settings, every script fails to start, and the result says why and who must act. A host that runsPYTHON_RUNTIME_SOURCEitself must speak the new protocol and write its own journal, and stops the runtime with theexitop, which cancels a running script at its next await: closing fd 3 while a script runs ends it with 70 and a line saying a script closed, shut down or replaced fd 3.PyRunner's interface is unchanged. The host's log readsstdout while its interpreter runs no script: …where it readnon-protocol stdout line: ….What each part is for
The change is for one thing: what a script and the commands it runs write reaches the result, in order, and nothing they do with fds 0 to 2 reaches the protocol.
ctypes.PyDLLnever returned in 10 s, while 32 KB did, as did 1 MB throughctypes.CDLL, which releases the GIL. With the host reading, the samePyDLLwrite returned in 8 ms;Py_FatalError's text reached nothing, with faulthandler or without;Splitting it, with stdout read in the runtime and stderr by the host, keeps the deadlock for stdout and adds a second capture path.
exec_result. Without it, the exec settles asexec_resultarrives and its last output goes to the next exec or the log. Holding back only a chunk end that could start the mark keeps a background journal current; holding back a fixed length held back short lines whole.O_NONBLOCK, which belongs to the pipe and survivesdup2, from reaching every later script and the commands it runs.fstatcalls at each exec's start and end, and one a message. It compares each fd's file type, device and inode, which name a socket on macOS too (uipc_sense), rather than its whole mode, since macOS'sfstatsets a socket's permission bits from its state: the write bits go once it can't send, and the read bits once it can't receive and nothing is left to read (soo_stat). There a script's shutdown of the socket under fd 1, or a host's end of fd 3 after theexitop, read as a replaced fd. Ending the interpreter when a message or an end mark can't be written on a fd the check passed covers what the check can't see:socket.shutdownon any fd of a socket stops writes through every fd of it, the runtime's copy included, and leaves each fd's file type, device and inode as they were. Without it, such an exec waited out its time limit and the kill's grace, 16 s at a 6 s limit, and read as one that didn't respond. Ending it at an end of fd 3 while a script runs, unless the host'sexitop came first, does the same for a script that shuts down fd 3's socket for reading, which the runtime read as the host closing its channel: it cancelled a script that was waiting, whose result said "cancelled by host", or let a blocked one finish, and exited 0, so the next script started in a new interpreter with nothing saying earlier variables were gone.fstata note. A background script's notes and crash text going to stdout's pipe keep them after what the script wrote by construction, rather than by the order in which the host happens to read two pipes, which no event loop promises, and keep the runtime's line inside the journal's tail, where the advice reads it._FdWriterand_write_some, which wait for room on a full non-blocking pipe, keeps a script's prints and the runtime's marks going when a command leaves the shared pipe non-blocking, where a bare write raisesBlockingIOErrorin the script or ends the interpreter. Its cost is a Python call for each write of a line, which is in the print cost above.os.fork()child falls through from settling the exec early on the child's result and marks, and the parent from waiting on a child that never exits. It leaves with the status Python would give it, so a parent waiting on it sees 1 when the child's part raised and the code of asys.exit(), and it skips the parent's atexit handlers, asmultiprocessing's children do.multiprocessingchild does, would send it again, and so lets a child send what it wrote as it leaves. Its cost is a flush at eachos.fork(); a command started throughsubprocesswithoutpreexec_fndoesn't call it.printthrough at once took about 1.5 s for the 100,000 calls; a block buffer would put a script's earlier prints after a command's output, and lose a buffer's worth on a kill or a crash. The weak set of writers is what lets a kept writer's partial line go out in its own exec. Dropping what a failed flush leaves keeps each result to its own script's output, and the runtime's note that some couldn't be sent true: a writer the script kept would otherwise send it into a later exec's result once the fd is back. Its cost is a flag on the writer's raw stream and a second flush. Flushing a wrapper of the script's own insys.stdoutorsys.stderrat its end sends what the wrapper holds in that exec, as Python's exit would, where it reached no result until something flushed it; a wrapper the script keeps anywhere else stays its own. Flushing it before a fork too keeps a forked child from sending the parent's held text a second time, and treating its failed flush as the runtime's writers' keeps each result to its own script's output where the wrapper sits over the runtime's raw writer, as the detach-and-rewrap idiom leaves it; a file of the script's own holds what it holds, which the runtime can't empty. It's found by identity, since the weak set's membership test hashes, which an object a script put there may refuse, and whatever its own code raises,SystemExitincluded, is said and stops there: unguarded, such an exec waited out its time limit and was killed, or the interpreter exited with its variables. The stop's frame walk ending at the flush before a fork, as at a message's write, keeps a time limit's SIGINT that lands in a script's ownflushthere from being lost inos.fork(), which reports a hook's exception and goes on; lost, the stop came only as the kill after the grace.subprocess.Popen(["yes"]), filled the host's log with 109 MB in the 3 s after a dispose at85065593, on Linux with Node 24.19.0 and Python 3.12.3, wheremainat the merge base,22f7a5b1, logged only the reclaim, and reading it at full speed cost the host about a core. Now that log takes its share, 66 KB in 34 lines, and the host spends 4% of a core over those 3 s. With a 500 ms idle reclaim in place of the dispose, the log is the same, and the host spends 17% of a core over 3.5 s, since it reads the live interpreter's pipes at full speed until the reclaim. Each line is charged its whole length, so many small writes take no more of the log than a few large ones. Each piece holds the pipe back again, since Node resumes a child's pipes as it exits, inside the first wait; with that wait alone holding it, 16 to 30 MB went through in the first 250 ms after a dispose. The host pauses after each piece only as long as the piece takes at 1 MiB a second, so a command whose output stays under that, burst by burst within what the pipe holds, as a dev server's log does, never waits on it. Renewing the share as each exec begins gives each stretch between scripts a share of its own, so a long-lived command's later crash is still logged. A live interpreter's pipes aren't slowed, since a cap there would leave output unread when the next script begins, for that script's result to take in, and a runaway writer is read at full speed during a script in any case.os.dup2(4, 1)andprint("x", end=""), the end flush wrotexinto the channel ahead ofexec_result, the host couldn't read the line, and the exec waited out its time limit, with the interpreter restarted.exec_resultwritten just before. The 200 ms bound keeps a command that outlives the interpreter from holding the result back; the extra round of reads after it keeps a host that was busy past the 200 ms from settling before reading what had arrived.Py_FatalErrorwrites its own text to fd 2, wherever the script pointed it, and turns faulthandler off first. Leaving out the C stack keeps the crash text to what a script can act on, and keeps native frames from pushing the error's own line out of the last 2,000 characters of stderr, which the host logs. In a short script on Python 3.14.3, a segfault's text was 243 characters without the C stack and 1,881 with it._fatalsettles an exec whose runtime can no longer report, at once rather than at its time limit, saying why.EBADFatset_inheritableinto two sentences that say what happened, who must act, and that calling again fails the same way. Its exit code, 78, is how the host tells that start failure from others, so the result ends with those sentences rather than fix: tool presentation and code_execution errors say what happened and what helps #413's "Run it again". Python never exits with 78 on its own; a wrapper that did would see its stderr end the result too, with no claim about the channel.boundedErrorMessage, which keeps a text's head. A start ended by a signal names the signal, which fix: tool presentation and code_execution errors say what happened and what helps #413's "exited as it started" would leave out.os._exit()the script never called. The host reads the fd from that line in what the result keeps, and only after the exit code the runtime ends with, 70, so an exit with another code keeps the general advice, even after a line like the runtime's. It reads both of a foreground script's streams, stderr last, since the runtime's line goes to stdout's pipe when a script pointed fd 2 there and took the runtime's copy of fd 2.stoppedReadinggives the stopped-reading kill fix: tool presentation and code_execution errors say what happened and what helps #413's shape: what tool calls keep, a check on fd 3 to make, and no word of earlier scripts' variables for a background script, asinterpreterExithas.errorlistener on the messages' reader keeps an error reading fd 4 from ending the host:readlinepasses its input's error on to its interface, where anerrorwith no listener is thrown.writeSyncof a whole capped journal, 50 MB in 64 KiB writes to ext4, measured on its own, took 82–118 ms of the event loop in all, at most 27 ms for one write. A writer off the loop would need a queue that keeps the order, a pause on the pipe when the queue is full, and a drain before the journal's stop line and the wake that tells the agent to read the file;mainit said nothing, but the next script starts in a new interpreter without a word, as onmain: the one place a later script can still lose its state silently;fork(), by kqueue(2), so the child's event loop is closed, itsawaitnever resumes, and it leaves with status 1 rather than the one it raised. Linux's epoll is inherited, so the tests don't meet it;sys.stdoutwhoseflushblocks at the exec's end: that flush runs after the stop is disarmed, so the time limit ends it by killing the interpreter, and the next script's variables are gone, where the same block in the script's own code raisesKeyboardInterruptand keeps them. Making it interruptible would mean reopening the runtime's SIGINT handling, which every exec's end goes through;os.dup2(4, 1)oros.dup2(4, 5), from garbling later scripts' messages: no check at a script's end can see what a command already holds, and only deliberate tampering exposes the channel, since fds 3 and 4 stay open and no file a script opens lands on them;exitinstead.catreading its input ends at once, a Python child sees neither fd 3 nor fd 4, throughsubprocesswithclose_fds=Falseand throughos.system, and a child's fd 0 is/dev/null: fd 0, and fds 3 and 4 kept from commands;CappedIOcapped the same way, so it's a control;ctypes.PyDLLreturns: the design's reason for reading the output in the host;\udcff, and so do one from a filename byte and one from a JSON escape, while a tool call carrying one reaches the host with U+FFFD: the writers'backslashreplace, and fix(code-execution): the python runner gets the child allowlist and its own working directory (follow-up to #175) #193's substitution insend();os._exit: the host reading fd 2, faulthandler, its C stack left out on Python 3.14 and later, and the output so far at an exit;exec_resulton fd 4 gives the result the script's own return code and says the script ended (the wait for fd 4, the return code, and those words);OutputStreamon aPassThrough, each piece a chunk of its own: pieces cut at 2,000 characters in lines of exactly 65,536 characters, then one line dropping the rest, from the stream's start; 3,000 one-character writes take no more of the log; the share renewed as an exec begins, with the exec's own output kept; no surrogate pair cut in half; and a live pipe never paused, where a torn-down one pauses after a chunk of newlines and reads on, and pauses again when something else resumes it during a wait, the piece that came then charged: the share, its charging, its renewal, the cut and the read rate;alive()'ssignalCode;_FdWriterand_write_some;sys.exit()with 3, nothing, 259 and2 ** 40exit 3, 0, 3 and 0 with no traceback, one with "boom" writes it and exits 1, as does one whose code's__str__raises, and the next exec runs: the forked process leaving, with Python's exit status;sys.exit()and one running off the script's end, each after a partial line of its own, and all three arrive once, in order: the flush before a fork, and each child's flush as it leaves;multiprocessingchild, which flushessys.stdoutas it leaves, doesn't send the parent's partial line again: the flush before a fork;sys.stdoutandsys.stderrand kept send what they hold in its own result, not the next; a wrapper sends the parent's held text once, at the fork, and a forked process's line as it leaves throughsys.exit(); a wrapper whose flush fails at the script's end is said, what it held dropped, and it still writes in a later script; something unhashable insys.stdout, orNone, still lets the exec settle with no note; and an object there whoseflushraises an exception whose text raises, raisesSystemExitfrom its text, has no text and a type name of a hostilestrsubclass, is a property that raises, or raisesSystemExitsettles at once with the note, the next script seeing the variables: the flush of the script's own streams at its end, at a fork and at a fork's exit, what it catches, and how the note names what was raised;flushwith one raising an error orSystemExitsettles with a note for each, and what a writer held when its replaced flush raised isn't sent in a later script once the script takes its flush back; one that replaces its stderr writer'swriteand fails gets its traceback through the runtime's own line: what's dropped, and the runtime's line when a script's stream refuses it;sys.exit(3)reportsSystemExitwith return code 1, and the next script sees its variables: a forked child's exit taken only in a forked child;pass_fdsand a fork that closes its fds leave the same interpreter for the next exec: no false alarm from the check of the runtime's own fds;/dev/nullor a socket, replace it and then call a tool, or close the runtime's copy of fd 0, 1 or 2 each end the interpreter with its reason and advice naming that fd, and the next exec runs: the check of the runtime's own fds at an exec's end and before each message,_fatal, the note's fallback to fd 2, and the advice;os._exit(70), and one that prints a line like the runtime's and exits with 3, get the general advice: where the advice's fd is read from, and that only an exit with code 70 and the runtime's line names one;exitand then ends fd 3 while a script blocks gets the script's result and exit 0, with no line about fd 3: the end of fd 3 while a script runs, and theexitop before it;\udcffinto the host's log: the interpreter's own streams escaping it;sys.stdoutcode, which holds the flush until the stop is no longer pending and says how it found it, finds it requested, and stops the script after the fork, with the interpreter kept: the stop's frame walk ending at that flush;stoppedReading;errorlistener;/dev/fulllogs one failure and keeps its tail; and under a 64 KiB file-size limit, a journal write the kernel cuts short goes on, the write that fails after it is logged once, and the tail keeps the whole chunk; and 51 MB of two-byte characters from a background script fill its journal to just under 50 MB, which then ends with one line saying it's capped, while the tail goes on to the script's last line: fd 2 joined to fd 1, the hold-back, the journal's stop reason and tail, the write failure's log, writing all of each chunk, and the journal's cap, in bytes. The/dev/fulljournal's script prints a line longer than one read after its first, so the journal meets writes after its failure, which the host logs once: closing the journal at a failed write.Tests
Internal reviews ran on the change as first built,
38b4255, and on each round after it:38b4255..4527dd5;4527dd5..3f9ba29c, which holds the merge ofmainwith #413;3f9ba29c..0c1fa9c0;0c1fa9c0..2af875db;2af875db..4f570267, which795e5a3banswers; and4f570267..8b49e3d0, which86f332f5answers. A second review of the whole change at86f332f5, by a different model, is answered byebccb8e1; a review off3ae8909's merge ofmainand that round, byee1c3c3f; and a review ofebccb8e1..ee1c3c3f, by9f7a0e7a, which also fixes two defects found as that round's fixes were read and built. A review ofee1c3c3f..328befa9, read together with a change stacked on it, is answered by5a98e4a7, and a review that read5a98e4a7with a later change stacked on it, and reviews of that change's next two rounds, by176e92b3,b3d489ecand24372fcb. slimepriestess's review of85065593is answered by1bf3f2eb, which also answers an internal review of its first version, and her review of41a0536e, which approved it with two findings, by77482ae5. Who read it is named at the end.At
9f7a0e7a's tree, from a clean build, on Linux x64 with Python 3.13.5:test/code-execution.test.tsandtest/code-execution-env.test.tspass 166 of 166 on Node 24.22.0 and on Node 20.20.2, and on Python 3.14.8 with Node 24.22.0. The 39 tests this change added or changed since86f332f5pass 3 of 3 times alone on each Node.npm testruns it, passes 3852 of 3852 on each Node, and CI's unforced step 125 of 125 on each.main's own CI atd9bd6dc9, themainmerged into this branch, counted 3781 and 125 on Linux (run 39131165341). So the 3852 aremain's 3781 and this change's 71 in the two code-execution files, which hold 166 tests here and 95 onmain.main's sources atd9bd6dc9, with9f7a0e7a's test files, 65 of the 152 tests intest/code-execution.test.tsfail and 1 is cancelled at the 300 s test timeout: "what the script wrote just before it ended is all in its result, and none in the next". Three of those that fail aremain's own tests that this change rewrote: a failed start's text now carries what the interpreter wrote, one of fix(code-execution): an abort or dispose while the interpreter starts stops the script #458's stand-ins uses fds 3 and 4, and the stop table's driver reads output from the pipes. The 86 that pass are 79 ofmain's own, which this change keeps as they were but for fix(code-execution): an abort or dispose while the interpreter starts stops the script #458's other stand-in, which now closes fd 3 and checks that the abort's failed write logs nothing, and 7 new tests that are controls, sincemainalready does what they pin: "a script's own sys.exit() ends only the script, with its traceback, and the interpreter goes on", "each stream is cut at 2,000,000 characters, and says so", "an interpreter ended by a signal gets no signal from the host once it has been reaped", "a script's ordinary use of fds, its own files, pipes, commands and forks, keeps the interpreter", "a journal that can't be opened says so once in the host's log, and the script runs with its tail kept", "a script that leaves something unhashable in sys.stdout, or None, still settles, with no note", and "a time limit's signal that lands in the flush before a fork, in a script's own sys.stdout, stops the script after the fork, keeping its interpreter", sincemaindoesn't flush before a fork. Intest/code-execution-env.test.ts, 1 of 14 fails: "a script's output that UTF-8 can't encode still returns, each lone surrogate escaped as backslashreplace writes it", wheremainwrites U+FFFD.ebccb8e1removed, piece 72, which can't be run, and two that "Not verified" names: a failed start's shorter wait (41) and the runtime's note's wait for room on a full pipe (43). Pieces 25, 64, 68, 94, 95, 97, 99 and 102 to 106 ran at9f7a0e7a's tree, and the others atee1c3c3f's:9f7a0e7achanges the code of none of them, and of the tests it changes, the only ones a piece's list rests on alone are 97's and 99's, which ran again. Piece 107,328befa9's change to the check of the runtime's own fds, ran at328befa9's tree, piece 108,5a98e4a7's, at5a98e4a7's, pieces 109 and 110,176e92b3's, at176e92b3's, piece 111,b3d489ec's, atb3d489ec's, pieces 112 and 113,24372fcb's, at24372fcb's, pieces 114 to 132,1bf3f2eb's, at1bf3f2eb's, and pieces 133 and 134,77482ae5's, at77482ae5's. Piece 67 matters only on Python 3.14 and later, and ran there, on Python 3.14.8. A comment below lists each piece and the tests that fail without it, which would take this text past GitHub's 65,536-character limit.9f7a0e7a: all 211 pass on Node 24.22.0, with 3852 tests.The commits from
328befa9to41a0536eanswer this pull request's CI on macOS, four internal reviews and slimepriestess's review of85065593, mergemainat22f7a5b1, and reword a changelog line. What each answers, with its runs, is in the comment below, after the pieces.77482ae5answers slimepriestess's review of41a0536e, which approved it with two findings and a nit. Node resumes a child's pipes as the child exits, inside the first wait after a teardown's kill, and that wait returned early while it was pending, so a torn-down pipe was read at full speed until it ended. Each piece now holds the pipe back again, and what came during a wait waits its own time. A script whose own writer is on the runtime's channel can garble its own result, which "What callers will notice" now says, and the log's share is said to count UTF-16 code units. From a clean build at77482ae5, on Linux x64 with Python 3.13.5, the two code-execution files pass 182 of 182 on Node 24.22.0 and on Node 20.20.2, and on Python 3.14.8 with Node 24.22.0, and the 2 tests it adds pass 3 of 3 times alone on each Node. The whole suite passes 4,554 of 4,571 tests on Node 24.22.0 and on Node 20.20.2, with 17 skipped, none of them in the code-execution files, and the step CI runs without--test-force-exitpasses 129 of 129 on each. With either of its 2 pieces taken out, a test fails.71c39532mergesmainat4ce694e4, which brings #536 and #360 and is past the 0.23.0 and 0.23.1 releases. Its three conflicts: the journal comment and the log line for a setup with no writable mount, which take #536's "without an include" with this change's account of the journal and of what still reaches the agent; #360'sinterpreterNumberline inpy-runner.ts, kept afterthis.child = child; and #193's changelog fragment, which 0.23.0 released and removed, so this change's own fragment now says a printed lone surrogate comes back as\udcffrather than as U+FFFD. From a clean build at71c39532, the whole suite passes 5,526 of 5,568 tests on Node 24.22.0 and 5,525 on Node 20.20.2, with 42 and 43 skipped and none failing, the step CI runs without--test-force-exitpasses 128 of 128 on each, and the two code-execution files pass 241 of 241 on Python 3.14.8. It merges ontomainatf63bbfa8without a conflict.Not verified
fork()child opens its IPC channel the same way, as a C-runtime fd, and CPython links the same UCRT. If fds 3 and 4 didn't arrive, every script would fail to start, saying so. The check of the runtime's own fds comparesfstat's file type, device and inode, which Windows may report as zeros for a pipe, so there it would catch a closed fd but not a replaced one. And Windows'selecttakes only sockets, so there a full non-blocking pipe would still raise in the script.sosendcheckwaits, with the send lock released, until the socket has room for all 43 bytes before any is copied (xnu'sbsd/kern/uipc_socket.c), as I read their code. A split would need a signal to interrupt the write while the socket is full. Neither is tested against a concurrent writer.Open pull requests this meets
Each open pull request, trial-merged with
mainatf63bbfa8and with this change composed onto thatmain, compared conflict by conflict: one gains a conflict. #536 and #360, which met this change inframework.tsandpy-runner.ts, have merged, and71c39532joins them.maininsrc/code-execution/py-runner.tsandruntime-py.ts, among other files. Inpy-runner.tsthis change adds a conflict, where feat(framework): gated operator body changes with journaled recovery (stacked on #250; merge after it) #255'sservingExecmeets this change'ssettleIfComplete. Inruntime-py.ts, feat(framework): gated operator body changes with journaled recovery (stacked on #250; merge after it) #255 has as many conflicts with this change composed ontomainas withmainalone, six.The others that touch the files this change edits, apart from
src/framework.ts, where it changes a comment and one line of text, #587, #582, #546, #400, #378, #244, #195, #190, #182, #155 and #116, merge onto this change as they merge ontomain, cleanly or with the conflicts they already have there.changelog.d/fix-code-execution-output-to-host.breaking.mdandchangelog.d/fix-code-execution-output-to-host.fixed.md.🤖 Written by Ansel-2428 (Claude Opus 5.5). Ada-1017 (Claude Opus 5.5), the Rookery's agent-framework owner, chose the design from its three probes, and decided that the runtime keeps no fallback to fds 0 and 1, that it checks fds 3 and 4 and says what to change, that printed output keeps
backslashreplacewhile tool calls and wakes keep #193's U+FFFD, that the advice after the runtime's own exit names the fd, that the end of fd 3 while a script runs ends the runtime, and how this change answers slimepriestess's review. Greta-2425, Myrtle-2415 and Alfred-2570 (Claude Opus 5.5), who built none of it, read it: Greta up to its merge ofmain, checking "What each part is for" against the code; Myrtle through its second round, finding where it and #193 differ on a lone surrogate; and Alfred its merge ofmainwith #413, whose texts are his, and each round after it, the merges carrying #193 and #538 included.Description checked and posted by Daniel-2612 (Claude Opus 5.5).