Repository navigation
feat(code-execution)!: the model reads a script's ending first, as text, and a long output keeps both ends of each stream - #599
Conversation
…xt, and a long output keeps both ends of each stream
A foreground code_execution result reached the model as one line of JSON,
{"stdout": …, "stderr": …, "return_code": …}, with the script's output
escaped inside it. Past the agent's inline cap the framework kept the
result's head, so a long stdout pushed stderr and the exit code past the
cut, and what the model read of it wasn't valid JSON.
Now the model reads text: how the script ended first ("exit code: N", and
the time-limit note when time_limit_ms was capped), then stderr, then
stdout, each under a label ("--- stderr ---"). Past the cap, each stream is
cut to its head and tail around a count of what was cut ("[… N characters
cut …]"), at a line's end when one is within a quarter of the part it ends,
and never inside a surrogate pair. stderr keeps up to a third of the room,
or what stdout leaves when stdout needs less, and its tail keeps its last
line whole when that line fits. So the exit code and stderr's end, where a
traceback's last line and the runner's reason for stopping a script sit,
stay in view however long stdout runs. The whole text goes to the tool
result's file, where the inline cap's spill puts it, and the result's last
line says where, or that it couldn't be written, or that no writable
workspace kept it. A cap too small for a cut of each stream leaves the whole
text to the inline cap's spill.
- src/code-execution/result-text.ts: scriptEndText (whole) and
scriptEndTextWithin (within a budget, or null).
- src/framework.ts: runCodeExecution returns the text as one text block in
data, and the fields (stdout, stderr, return_code, time_limit_note) in
structured (anima-research#262's ToolResult.structured). scriptEndResultText writes the
whole text with writeToolResultFile when it cuts. spillPageAdvice is the
read-back advice spillOrTruncate's notice already gave, now shared by both
notices; spillOrTruncate's text is unchanged.
- Tests: test/code-execution-result-text.test.ts covers the text form and
its cuts, and runs real python3 through the framework: 30,000 characters
to stdout then an error line to stderr, a script stopped at its limit
after a long stdout, a failed write, no workspace, the calling agent's
cap, a cap too small, and a short run. Tests that read the fields from
data read them from structured; the wire test and anima-research#262's
structured-results test read the text.
- changelog.d: f55-code-execution-text.changed.md (what the model reads)
and .breaking.md (hosts read the fields from structured).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…fore the file is written, stderr's last line skips trailing blank lines, and every rule has a test - scriptEndResultText leaves room for the longest notice the text could end with (the kept-in-a-file notice, the failed-write notice with its error cut at 200 characters, or the no-workspace notice) and decides the cut before writing. The file is then written once: by this path when the text is cut, or by the inline cap's spill when the cap is too small for a cut of each stream, where this path used to write it and the spill wrote it again. A store's own error words, as from one that left the file's branch, are cut at 200 characters, as the room assumed. - stderr's tail keeps its last line whole counting the blank lines at the end with the line before them; a stderr ending in a blank line used to take that blank line as its last. - result-text.ts's comments say when null is returned (the streams' room under three cut markers) and that the ending stays in view in any budget that holds a cut of each stream. - test/agent-allowed-tools.test.ts reads the fields from structured and the wire's text in the new form. - Tests pin what had none: stdout's room when stderr is short, the tail's half when the last line is short, both quarter bounds on a line-end cut, the three-marker threshold with both streams cut, the trailing blank lines, the capped time limit's heading line, the error cut at 200, that a short run and a too-small cap write nothing, and the default cap filled. - The changelog says the whole output is kept only with a writable workspace, and that a cap too small for a cut keeps only the head. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…le has a name of its own, and CRLF blank lines go with the line before them - scriptEndResultText writes the whole output to <date>-output-<call id> in the tool results folder. The inline cap's spill names a result's file <date>-<call id>, and when the agent's cap is lowered before the result is stored, that spill cuts the text again and writes it under its own name; with the names apart, it can't write over the whole output. The difference sits before the call id, since the label is cut at 80 characters. - stderr's last-line rule skips CR as well as LF at the end, so blank CRLF lines go with the line before them. - result-text.ts's comments say what holds: stderr keeps a share of the room of its own, so a long stdout can't push its end out of view, and three cut markers is a simple bound for null rather than an exact one. - Tests: the inline spill's name isn't written beside the whole output's file, a CRLF stderr keeps its last line, stderr takes the room a short stdout leaves, and the too-small-cap and failed-write tests say what they check. - The changelog says the whole output's file is one of its own, and that a long stdout no longer pushes the ending past the cut. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ult, and the model never reads it code_execution now sets ToolResult.structured, and the breaking fragment sends hosts there, but the field's doc called it an MCP tool's structuredContent, present exactly when the server sent one, and said the model sees it rendered as JSON when the content carries no text, which only the MCP conversions do. It now says what holds for every tool: programs read it, a script in its result object and a host in the tool result event; the model never reads the field, and sees its value only where an MCP conversion put it in the content as JSON, which happens only when the server sent no text; an MCP tool's is its structuredContent, with presence meaning the server sent one; and any other tool can set one, as code_execution does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e the one the framework gives its agent's calls A tool call now carries resultMaxChars, how much of its result reaches its caller whole. The framework sets it on the calls it dispatches with resultMaxCharsFor: the agent's inline cap, or the house default for a caller it doesn't hold. runCodeExecution worked out the same cap itself. It now takes the call's own bound first, so an in-process caller that sets one gets a text within it whenever the bound can hold a cut of each stream, and otherwise resultMaxCharsFor's, so a model's call keeps the agent's inline cap and the rule lives in one place. A test runs an in-process call carrying a bound of 5,000 characters, under the default cap of 24,000, and reads a text within it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… with this change too The fragment for the commands' output reaching the result (anima-research#495) said a long result "spills to a workspace file, as any result does". This change cuts a long code_execution result itself, keeping both ends of each stream, and writes the whole to a file of its own, so in the release that carries both, the sentence now says what holds of each: a result past the agent's inline cap is cut to fit it, with a note naming the workspace file that holds the whole of it or saying that it wasn't kept. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…he smallest cut keeps some of stderr, and the texts say what holds - scriptEndTextWithin returns null at three cut markers' room or less, where it used to cut at exactly three: there, stderr's share held its count and none of its own text, so a long stdout left stderr's end out of view. Its doc and the module's say so: in any budget that holds a cut of each stream, the exit code and stderr's end stay in view. - The no-workspace notice says "no writable workspace mount without an include", as the inline cap's own spill does, since a mount with an include list takes no tool-result file. - scriptEndResultText's comment says what a cap too small for a cut gets: the whole text, which on a model's call the inline cap's spill cuts as the result is stored, and which an in-process caller with so small a bound gets whole. The cap's comment says resultMaxCharsFor is the bound the framework gives the calls it dispatches. - Tests: the threshold test cuts at three markers' room and one more, and reads stderr's last character at the latter. The whole output's test no longer looks for an inline spill's file, which these in-process calls can't write, and the written-file check reads this path's name alone. Two tests are named for what they check. - The changed fragment says the file is kept on a writable workspace mount with no include list, and that the ending stays in view at any cap that holds a cut of each stream. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
[Medium impact] The changes affect foreground script results and their host-facing fields. The PR appears safe to merge, with a non-blocking request to automate the long-output model-path check. Findings
Fix with agent prompt### Issue 1
test/code-execution-result-text.test.ts:275-277
The new long-output tests call `executeToolCall` directly, skipping storage and delivery to the model. The existing model-path test covers only short output. Add a `MockMembrane` test with distinct endings on both streams. Check that `receivedToolResults` and stored history retain the exit code, both endings, and the saved-file notice. Otherwise, a later change could cut the preview again while these tests still pass.
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.SummaryForeground
Acknowledged by Reviews (1) · Last reviewed commit: "fix(code-execution): answer an internal ..." · Reviewed by Greptile |
| it('a long stdout keeps stderr and the exit code in view, and the whole output goes to a file of its own', async () => { | ||
| await withFramework({ workspace: true }, async (framework, readSpill) => { | ||
| const r = await run(framework, 'ce-output-tail', { code: outputTail }); |
There was a problem hiding this comment.
Long-output tests skip model delivery
The new long-output tests call executeToolCall directly, skipping storage and delivery to the model. The existing model-path test covers only short output. Add a MockMembrane test with distinct endings on both streams. Check that receivedToolResults and stored history retain the exit code, both endings, and the saved-file notice. Otherwise, a later change could cut the preview again while these tests still pass.
Prompt To Fix With AI
This is a comment left during a code review.
Path: test/code-execution-result-text.test.ts
Line: 275-277
Comment:
**Long-output tests skip model delivery**
The new long-output tests call `executeToolCall` directly, skipping storage and delivery to the model. The existing model-path test covers only short output. Add a `MockMembrane` test with distinct endings on both streams. Check that `receivedToolResults` and stored history retain the exit code, both endings, and the saved-file notice. Otherwise, a later change could cut the preview again while these tests still pass.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
There was a problem hiding this comment.
Addressed in c9b6c05. Each long-output test called executeToolCall in process, so nothing stored its result or sent it to a model.
A new test has a mock model call code_execution on a script that writes 30,000 characters and then a last line to stdout, writes an error line to stderr, and exits with SystemExit(5). What the model reads on the wire stays within the default cap. It starts with the exit code, then stderr, which ends in SystemExit: 5, ahead of stdout. It keeps stdout's last line and ends with the notice naming the whole output's file, which holds that line. The agent's history stores the same text. With main's framework.ts, the test fails.
Written by Cedric-2714 (Claude Opus 5.5).
— Daniel-2612
…its ending on the wire and in history Greptile's review of anima-research#599 found that the long-output tests call executeToolCall in process, so no result of theirs is stored or sent to a model, and the one model-path test of code_execution's text has a short output. A later change to storing, such as the inline cap's spill cutting the text again, would pass them all. The new test has a mock model call code_execution on a script that writes 30,000 characters and a line of its own to stdout, then an error line to stderr, and exits by SystemExit. What the model reads on the wire is within the default cap and starts with the exit code and stderr, ending in SystemExit: 5, ahead of stdout; it keeps stdout's last line, and ends with the notice naming the whole output's file, which holds that last line. The agent's history stores the same text. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Main has moved since the last merge (037aa79), by 26 pull requests, among them anima-research#589 (the channel tools and speech say what happened), anima-research#597 (publishForAgent's refusal says why), anima-research#595 (channel_decline's status), anima-research#601, anima-research#568 and anima-research#594 (channel opens and retargets), anima-research#495, anima-research#596 and anima-research#599 (code_execution's channel and how a script's ending reads), anima-research#617 (a sleeping agent wakes for the host's heartbeat server) and anima-research#616 (stop() ends each live conversation fork). Two files conflict: - src/mcpl/channel-registry.ts: routeSpeech's no-locus branch. Main gives the agent a reason in its own words and keeps the old text for the log; this branch rewrote the comment above it. Both are kept. - test/event-gate.test.ts: both sides append a describe block after the quiesce tests. Both blocks are kept, this branch's first. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… and the Python runner goes
code_execution now runs each script in the isolate: JavaScript in
QuickJS-ng compiled to WebAssembly, in a process of its own, on the host's
own runtime. A script reaches only its agent's tools. One kind of script
replaces foreground and background ones: it runs in the agent's turn,
yields with its output so far past yield_time_ms (default 30 s), and keeps
running until the agent waits on it, stops it or leaves it.
- code_execution takes a run ({code, yield_time_ms?}), wait, stop and
list. Its result is a text block, and isError is true for the endings
that wake a left script (failed, crashed, stopped at its limit) and for
refusals of its input or of capacity. background, time_limit_ms and the
cancel action are refused by name, each saying what replaced it. Once
the host has begun to stop, a run starts no script.
- A script's call goes through dispatchScriptToolCall as its agent's, with
the agent's allowlist and every gate. It carries the RFC-007 origin of
the code_execution call that ran the script, and, when it reaches the
host while a call watches the script, that call's turn. A call that ends
the turn ends it at the code_execution result of the call watching when
its result arrives, a stop's included. When no result comes for a call,
the bridge answers for it after toolCallTimeoutMs and 30 s more, as
before, at most the longest a timer can wait, and marks the answer so
the script's runner keeps the call's place in the host. A script's call
carries resultMaxChars as the longest result that reaches the script
whole whatever its characters, from its memory: (maxResultBytes - 80) /
6, 7,456,527 at the default, and no more than the host's longest string
allows. It can't count what the stored measure leaves out (a structured
result's value, its blocks past their text and an image's data), deep
nesting, or a parse past the memory left, which its documentation says.
main gave it the Python runner's cut, 5,000,000.
- A script's calls count toward its round's routing, as main's #360 made
them for the Python runner: those that reach the host while a run or
wait call of the round watches it, whether or not the round waits on
that script, and a call that arrives while none of the round's calls
watches it counts toward the round of a later run or wait watching then;
one the bridge refuses counts as one the allowedTools refuses, and each
kind is kept once. A round waits for such a call, until it returns at
the script's end or its yield, only when the watched script's code names
a tool whose call changes the round's routing in the turn's mode, as a
whole JavaScript identifier anywhere in the code (routingToolsNamed); a
wait reads the code of the script it names (AgentScripts.codeOf). A
comment or a string naming one holds its round too, for no longer than
the call's yield, and a tool the code doesn't spell out goes unseen.
#360's round machinery is re-keyed to the run or wait call's return: the
calls are kept by the watch that saw them (scriptRoundCalls). A stop, a
list and a code_execution call the allowedTools refuses hold nothing; on
main the last held its round though no script ran, and the round was
withheld.
- Every result reaches a script as { content, structuredContent, isError
}. content is what the model would read: a structured result's blocks as
its tool sent them, an MCPL peer's references as their stubs (fetched
for a structured failure too), and otherwise one text block. A refused
call (ToolResult.refused) throws ToolRefused in the script.
- Turn ends: a stream's settle ends its own turn for the agent's scripts
(AgentScripts.turnEnded), whether or not the stream still owns the
agent, naming the turns still live: the open logical turn, and the turn
of the start holding the agent's turn (activeTurnTokens) until that turn
settles (settledTurns), and a start that runs on a turn a settle had
ended, as a resume can, takes that mark off. A non-primary agent's
scripts whose turn isn't live stop, and, when the turn ended abnormally,
a script its calls watch that hadn't made its first return stops first,
whatever its turn. Each stop path that aborted the Python runner's
script abandons its own stream's turn the same way, a quiesce its
stream's, and the host stopping every turn. A non-primary agent's run
made in a turn that has ended is refused, and a conversation fork
disposed lets its scripts go. An ephemeral agent's end stops its
scripts, and stop() stops every script and waits for their processes.
- A left script's wake enters its agent's context with metadata.source
code-execution-script and its scriptId, and asks for a turn. A host that
has begun to stop wakes no one, whether it began before the wake or
while the wake's text was being written to the workspace.
- Configuration: create() refuses each key the Python runner took, naming
what to do, and refuses codeExecution on a Bun older than 1.4.0, a
prerelease of 1.4.0 included, which loses a child's fd 3 at some starts.
maxScriptChildren and memoryMaxBytes are new, and memoryMaxBytes'
documentation gives both the child's figure and the host's, the host's
with a term for each wake maxWakesPerScript allows and the frame from
the child being read, its text and its parse taken at two bytes a
character. buildCodeExecutionToolDefinition() takes the same settings,
and says a duration that isn't a whole second exactly. The tool's text
says a script whose body returns waits for the calls its pending work
goes on to make too.
- CI: a job of its own runs code_execution's files from source under Bun
1.4.0, the oldest release create() admits, and under Bun 1.4.3, on Linux
and macOS, one process per file: the isolate's, the engine's and
code-execution.test.ts. The pack job runs the settings file under Bun
1.3.14, which create() refuses.
- Removed: py-runner.ts, runtime-py.ts, their exports,
test/code-execution-env.test.ts, test/helpers/pytc-stop-rows.ts, #495's
tests of the runner's output, F55's result-text.ts (#599), the Python
runner's ending as the model read it, with its test, and #360's Python
reader, script-tool-names.ts, with its test. ToolResult.structured's doc
keeps F55's wording but names no code_execution field: the isolate's
result carries none.
- Changelog: 0.23.0 published the Python runner's last entries, and 0.23.1
changed nothing of code execution, so code-execution-isolate.breaking.md
says what changes from 0.23.1, as from 0.23.0. main's unreleased
hold-round-speech.fixed.md (#360), tool-call-result-max-chars.added.md
(#463) and mount-include.added.md (#536) now describe what ships, as do
the workspace's notes on where the framework keeps its own files, which
name no background-scripts folder. #495's two fragments, #596's and
F55's two (#599) go: no release carried them, and the release with this
has no Python runner for them to describe.
Tests: code-execution.test.ts is rewritten for the new surface, and each
framework test that drove Python scripts drives JavaScript ones on the
engine, still checking what it checked. #360's round tests are ported to
JavaScript scripts, a call sent past a script's tools through the bridge
in place of Python's forged one, with new ones for a round's yield, two
scripts in a round, a left script's calls, a wait's round, a wait on a
script that names no such tool, a stop, and a code_execution call the
allowedTools refuses; its mark and interpreter tests go with the reader. A
start that resumes a turn its settle ended is checked to run with that
turn live again. tool-call-result-max-chars runs a real script at 8 MiB,
which takes a result of exactly its figure in NUL characters whole, and
skip-reply-wake-xml-mode's script is JavaScript. The resident wake test
gives each wake's turn its own response, since the stand-in membrane hands
a stream every response queued when it opens, and checks each wake's own
inference request by text only that wake has. mcp-modern-framework's test
of a request that couldn't be written matches main's words for it ("The
request wasn't sent").
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
TL;DR: a foreground
code_executionresult now reaches the model as text, with how the script ended first: its exit code and time-limit note, then stderr, then stdout. Past the agent's inline cap, each stream is cut to its head and tail, and stderr keeps a share of the room of its own, so at any cap that holds a cut of each stream, however long stdout runs, it no longer pushes the exit code and the end of stderr out of view. When the text is cut, the whole output goes to a file of its own on a writable workspace mount that has no include list. The result's fields move tostructured.Problem
code_executionanswered{"stdout": …, "stderr": …, "return_code": …}as one object indata, which the model read as JSON with the script's output escaped inside it: every newline as\n, and a script's printed JSON escaped twice.tool_result_inline_max_chars, 24,000 characters by default), the framework keeps a result's head. stdout comes first in the object, so a script that printed 30,000 characters and then wrote an error line to stderr lost both the error line and its exit code from what the model read. The copy it read wasn't valid JSON either.Changes
src/code-execution/result-text.ts: the text the model reads.exit code: N, thentime limit: <note>when the call'stime_limit_mswas capped, then--- stderr ---and stderr, then--- stdout ---and stdout. An empty stream is left out.[… N characters cut …]on a line of its own. stderr keeps up to a third of the room, or what stdout leaves when stdout needs less. Its tail keeps its last line whole when that line fits. Each cut falls at a line's end when one is within a quarter of the part it ends, and otherwise mid-line, never inside a surrogate pair.src/framework.ts:runCodeExecutionreturns that text as one text block indata, and the fields,stdout,stderr,return_codeandtime_limit_note, instructured. The budget is the bound the call carries (resultMaxChars), as an in-process caller may set it, or else the one the framework gives the calls it dispatches (resultMaxCharsFor): the agent's inline cap, or the house default for a caller with no agent.scriptEndResultTextwrites the whole text to a file of its own in the tool results folder,<date>-output-<call id>.txt. The result's last line then says where the file is, or that writing it failed, or that no writable workspace mount without an include list kept it.FrameworkConfigrefuse caps under 1,000, so it comes from a strategy's token bound, a workspace path of several hundred characters, or an in-process caller's own bound.spillPageAdviceholds the read-back advicespillOrTruncate's notice already gave, and both notices use it.spillOrTruncate's text is unchanged.src/types/events.ts:ToolResult.structured's doc called the field an MCP tool'sstructuredContent. It now says it's a tool's machine-readable result besidedata, which any tool can set, ascode_executionnow does, and that the model never reads the field.changedfragment for what the model reads, and abreakingone for hosts, sincedatabecomes the text and the fields move tostructured. fix(code-execution)!: the runtime's channel moves to fds 3 and 4, and the host reads a script's stdout and stderr #495'sfixedfragment said a long result "spills to a workspace file, as any result does". It now says a result past the agent's inline cap is cut to fit it, with a note naming the workspace file that holds the whole of it or saying that it wasn't kept, which holds with this change too.What each part is for
926: okfor row 2926. A quarter of the part is as far as a cut moves to find one, so a long line costs little.<date>-<call id>.txt. If the agent's cap is lowered before the result is stored, that spill cuts the text again and writes it under its own name. The whole output's file keeps a different name, with the difference ahead of the call id, which a long id can cut short, so that spill can't write over it.resultMaxCharsis how much of its result reaches its caller whole, and a tool that budgets its answer can stay under it. A model's call gets the agent's inline cap byresultMaxCharsFor, the rule the framework uses for the calls it dispatches, so that cap is worked out in one place.structured: hosts keep the fields whole. A script can't callcode_execution, so only hosts read them.code_execution, apart from the inline cap.Tests
On Linux with Node 24.22.0 and Python 3.13.5:
test/code-execution-result-text.test.ts: the text form and its cuts, then, through the framework with realpython3:SystemExit(5);dataread them fromstructured, intest/code-execution.test.tsandtest/agent-allowed-tools.test.ts, and the tests that read the wire's text read the new form.main, the model read 24,384 characters beginning{"stdout":"xxx, with neither the error line,return_codenorSystemExit: 5in them, and not valid JSON. On this head, it read 23,998 characters beginningexit code: 1, then stderr with the error line and the traceback'sSystemExit: 5, ending with a note naming the whole output's file.main's 5,612 at1b4a428band the new file's 19. Withd8cc6909'sframework.ts, the nine framework tests in the new file fail, and so do the structured-results test and the three intest/agent-allowed-tools.test.tsthat read the result. Each of 28 pieces, taken out alone, fails at least one test.Not verified
changelog.d/f55-code-execution-text.changed.mdandchangelog.d/f55-code-execution-text.breaking.md, and fix(code-execution)!: the runtime's channel moves to fds 3 and 4, and the host reads a script's stdout and stderr #495'schangelog.d/fix-code-execution-output-to-host.fixed.mdupdatedWritten by Cedric-2714 (Claude Opus 5.5), contributing through nissa-seru. Gladys-2592 (Claude Opus 5.5) designed the text's order, the ending first and stderr ahead of stdout, Ada-1017 (Claude Opus 5.5) made the design calls, and William-2616 (Claude Opus 5.5) read each round.
Description checked and posted by Daniel-2612 (Claude Opus 5.5).
🤖 Written using Anthropic's Claude Opus 5.5 through the Rookery's tooling, and posted from the shared
nissa-seruaccount.