[nissa-seru]

When a code_execution script ends without the answers to some of its tool calls having been seen, its result now names them, up to ten, and counts the rest by tool, so the agent can tell which effects may still come, or came without being seen, before it runs anything again. Until now a stopped, failed or crashed script said nothing of its calls, and a call that timed out in a script raised a TimeoutError that didn't say the call could still take effect.

For a foreground script stopped at its time limit, the result now reads:

exit code: 1
script stopped: it reached its 30s time limit. Tool calls it had already made keep their effects, so check them before running it again. To give it longer, pass time_limit_ms up to 300000 (5 min).
tool calls: 7 made while the script ran, 4 answered while it waited; the other 3:
- timed out in the script and still running, so it may still take effect: workspace--write {"path":"notes/a.md","content":"# Notes\nfirst line of a long note xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx…
- answered with an error after its code stopped waiting: discord--send_message {"channel_id":"123","text":"Deploy started"}, answer "Error: rate limited"
- still running when the script ended, so it may still take effect: discord--send_message {"channel_id":"123","text":"Deploy finished at 12:04, all checks green"}
--- stderr ---

KeyboardInterrupt: script cancelled by host

The exit code and the --- stderr --- section are the text form #599 gave the result. This change puts the runner's own words and the tool calls lines after the exit code.

Changes

• The runtime (src/code-execution/runtime-py.ts). When the code that made a tool call stops waiting for it, at its timeout or because the script cancelled it, as with a shorter wait_for of its own or a TaskGroup whose other task failed, the runtime sends the host {"op": "tool_unwaited", "id": …, "cause": "timeout" | "cancelled"}. It sends none when the host asked the script to stop, or for a call it cancelled itself at the script's end. The TimeoutError keeps its first sentence and adds: "The call was sent and may still take effect, so check for its effect before calling it again."

• The runner (src/code-execution/py-runner.ts). The runner files each tool call under the script running when the host reads it, whatever code made it: the script's own, or a task or a thread an earlier script left running. It keys each call by its interpreter's number and the runtime's call id, since call ids start again with each interpreter. A call read while no script runs waits in the runner's own record, and the next script takes it on if its answer hasn't been seen by then. Once the host reads a script's end, it closes the script's record, so a later answer leaves a call as still running when the script ended, and a later call goes to the next script. At the end, ExecResult.calls carries how many calls were made while the script ran, how many it took on from before it started, and, in the order made, each call whose answer wasn't seen while its code waited:

  • running: no answer had come when the script ended;
  • timed-out-running: its code stopped waiting at the call's timeout, and no answer had come;
  • cancelled-running: the script cancelled the call's code, and no answer had come;
  • answered-unseen: the answer came before the script ended, but after the call's code stopped waiting, at the call's timeout, when the script cancelled it, or with its interpreter exited or killed. It carries the first 120 characters of the answer's text, and isError when the answer was an error.

  The lines name each call by its tool and its arguments as JSON, cut to 120 characters. Past the first 1,000 calls answered after their code stopped waiting, the record counts them by tool in calls.unlisted. ExecResult.runnerNote carries the runner's own words that end the result: why it stopped the script, the time-limit note on a line of its own, why a script didn't run, or why the interpreter didn't start. stderr still ends with them, as hosts get it today, and scriptStderr gives a result's stderr without them.

• The result (src/framework.ts, src/code-execution/result-text.ts). callsLine turns calls into the lines above. A foreground script's text gives, after its exit code and, when the call's time_limit_ms was capped, the note saying so, the runner's own words and then these lines, and its stderr without the runner's words, so they're said once. Its structured fields carry the lines first, as tool_calls, and stderr as before. The lines also come ahead of a background script's last output in the wake when it dies, and first in the result of a cancel. After an error, a stop or a crash, the first line also says how many calls were made, as "tool calls: 5 made while the script ran, all answered while it waited" when every answer came while its code waited, with "and 1 before it started" after the first count when the script took calls on. After a script ends on its own, the lines appear only when some call's answer went unseen. The runner's advice after a crash, or after the runtime ended itself over a file descriptor a script took, names what to look for, the traceback that starts "Fatal Python error:" or the runtime's line starting "[pytc-runtime]", since it now comes before stderr in the text and after it in ExecResult.stderr and a background script's journal.

• The tool's description says that how the script ended, what it printed, and a line naming the tool calls that timed out or were cancelled in it, or were still running when it ended, come back, other tool results staying in the script, and that a call that times out was sent and may still take effect. Changing it makes each agent offered the tool miss its prompt cache once, when its host upgrades.

• The changelog. An added fragment for the lines, the runtime's word and the description, and a changed one for the runner's words in the text's opening lines. Two sentences of feat(code-execution)!: the model reads a script's ending first, as text, and a long output keeps both ends of each stream #599's changed fragment held only without this change: that the end of stderr "holds a traceback's last line and the reason a stopped script gives", and that a cap too small for a cut of each stream is "a few hundred characters". They now say the cut no longer pushes "how the script ended, or the end of stderr, which holds a traceback's last line" out of view, and that the cap is too small at "a few hundred characters more than the opening lines". fix(code-execution): the advice after a signal fits the signal: a crash's points at native code, and only SIGKILL keeps the out-of-memory example #596's fragment quotes the crash advice in its new words.

What each part is for

• The record in the runner. PyRunner already sees each tool_call op, each answer it sends back, and the exec's end, so it can say which calls had no answer at the end without a new path through the framework. ScriptToolCallHandler stays as it is.
• tool_unwaited from the runtime. Only the runtime knows whether a call's answer was seen. A script that holds its event loop past a call's timeout has the answer queued behind the timeout, so it gets a TimeoutError even when the answer came long before. A call the host answered while the script held its loop past the call's 1 s timeout ended in a TimeoutError at the script's next await, in a probe with Python 3.12.3, 3.13.3 and 3.14.3, and in one of this change's tests. A clock on the host can't tell that case from one answered in time. It's also the case where a script's retry repeats an effect, so the call is named with its answer. A script that cancels its own call, with a shorter wait_for or a TaskGroup whose other task failed, may call again too, so the runtime says that as well, and the call is named the same way. A stop the host asked for, and the calls the runtime cancels at a script's end, are the host's to know: their calls read as still running when the script ended. It's a message of its own rather than a field of exec_result, since no exec_result reaches the host when the interpreter crashes, when the host kills a script that doesn't answer its cancel or aborts it, or when the runner is disposed, and a field there would be lost in those cases.
• The answer's text, and whether it was an error. The answer quoted is the text a script gets, the result's text. A structured result reaches a script as one JSON object with the server's content first and isError after it, so the cut can leave the error out of view on a long one. The host has isError with each answer, and whether the first call failed is what the agent needs before it calls again, so the line says so. A value a handler's type rules out is kept as no text, so keeping it can't stop the call's answer from going.
• Every call in the result of the script running when it's made. One script runs in an interpreter at a time, so the host files each call under the exec pending when it reads the call, and a call from a task or a thread an earlier script left running is named in the running script's result and counted as made while it ran. A call named in no result would leave an effect that may still come unsaid, and the call's tool and arguments let the agent tell it from the script's own.
• Calls made while no script runs. Such a call waits in the runner's own record, and the next exec takes on those whose answers haven't been seen, so nothing goes unnamed between scripts either. They're counted apart, "and 1 before it started", since the host read them while no script ran. Those answered while their code waited are left out of the count, since their effect came and was seen, but kept for word that their code had stopped waiting after all, as a held loop can bring once the next script has started, which names them then.
• A call's key holds its interpreter's number. The runtime's call ids start again with each interpreter, so a call taken on from one the runner has since replaced, as when it reclaims an idle one, would otherwise meet the next interpreter's call of the same id, and an answer to the one would settle the other. The number is the one fix(streaming): the outgoing stream carries only what delivery publishes, and a script's calls count as its round's #360 gives each interpreter as it starts. An answer that comes once the interpreter that made its call has exited or been killed counts as answered after its code stopped waiting, since that code can't see it, whether or not the runner has torn that interpreter down yet. After an exit, the runner waits for the output pipes before it settles the script, 200 ms and a round of reads at most when a command still holds them.
• The record closes when the host reads the script's end. The host reads the runtime's end of a script before the end marks of its output, so an answer read between the two would otherwise count as answered while the script waited, and a call read then would count in a script that had ended.
• "made while the script ran", "answered while it waited". The lines claim only what the host knows: when it read each call, and that it sent the answer while the call's code still waited, not that the script read it. A script stopped before the runtime read an answer queued behind a held loop never saw it, though the call's effect came.
• The bounds on the record. Word that a call's code stopped waiting can come after the answer has gone, as above, so the runner keeps the 1,000 most recent answered calls to find it. Once 1,000 more are answered before that word, the call counts as answered while it waited. The record lists the first 1,000 calls answered after their code stopped waiting and counts the rest. So a background script that runs for a day doesn't hold every call it made, and a call still waiting for its answer is held until the framework's answer comes, within the tool-call timeout and 30 s more.
• A brief copies the characters it keeps. On V8 a slice of a string keeps the whole string alive, so a kept brief of a long answer would hold the answer: fifty briefs of 5,000,000-character answers kept 254 MB with slices and 9 MB with copies (Node 24.19.0), and the record keeps up to 2,000.
• Each string and array of the arguments cut before the brief is made. Without it the brief would turn every call's whole arguments into JSON on the host's event loop: about 5 ms for a 5 MB argument and 66 ms for a 50 MB one, where cutting first takes about 0.002 ms (medians of five, Node 24.19.0 on Linux x64). An object with many keys, or a long key, is still made into JSON whole, as tool arguments rarely have.
• Ten named, the rest counted by tool. The lines come first in the result, which the agent reads before anything else and which a cap keeps, so naming every one of up to a thousand calls, a line each, would push out what the script printed. Ten covers a script with a few calls in flight, and the count by tool says what else to check.
• The lines after a clean finish. A script that catches a TimeoutError and goes on, or starts a call without awaiting it, ends with a result that reads as success while a call is still in flight or happened after its code gave up on it.
• The runner's words and the lines first in the text. Both say how the script ended, beside the exit code. stderr's end holds the reason today, ahead of stdout, but behind a long traceback it's in a tail after a cut marker. A result past the inline cap keeps its head, so what comes first stays in view.
• scriptStderr, beside the code that adds the words. The runner adds its words to stderr's end on lines of their own, so the text's stderr can go without them while ExecResult.stderr keeps them for a host that reads it, as it does today. If a result doesn't end with them, it's stderr whole.
• tool_calls first in structured. A host reads the same lines first, as the text has them.
• The description's two changes. A script's retry logic is written before the agent sees any TimeoutError, and repeating an effect is the case this change exists for, so that a timed-out call may still take effect is said before calling. The "Only … come back" sentence would be false with the lines and the runner's words in the result, so it names them, and names the calls the lines name: those that timed out or were cancelled in the script, or were still running when it ended. The result itself says where the lines go, and which result names a call from code an earlier script left running, when it happens.
• The cancel result and the wake of a dead background script. A background script's end reaches the agent only there, and an agent that stopped one is the one to tell which of its calls may still take effect.
• Cut: naming refused or failed calls apart from answered ones when their answers were seen; a host-side clock for timeouts; a separate field for each of the runner's endings; a context variable naming the script that made each call, since with one script at a time it differed from the exec running only when it named a script that had ended, and then its one effect was to drop the call from every result; the runner's own counter of interpreters, now fix(streaming): the outgoing stream carries only what delivery publishes, and a script's calls count as its round's #360's; a label such as stopped: before the runner's words, each of which already says what happened; taking the runner's words out of ExecResult.stderr, which would change what a host reads; and the description's sentences on where the lines go and on calls from code an earlier script left running, which the result says when it happens.
• Left out: telling the agent later when a call left running finishes, which would need a new notice path, where the agent can check the effect itself; a call answered while the script ran, which counts as seen even when the script never reads the answer, as with the other calls of a gather that failed, a shielded call, or a task never awaited, since the host can't tell those from calls the script saw; a call still waiting in the runner's own record when the runner is disposed; and a background script that finishes on its own, which has no result to carry the lines.

Tests

• New:
  • "a call the script stopped waiting for and that was answered before it ended is named with its answer's start, in the order the calls were made": the case where a script's retry repeats an effect, the order made, and the answer's cut, which leaves out a surrogate pair's first half;
  • "a call answered with an error after its code stopped waiting is named as one, and an answer is the start of the text a script gets": a structured error whose isError the cut hides, a handler's rejection, an answer with no error, and a plain string;
  • "a call the script stops waiting for with a shorter wait_for of its own is named with its answer, beside the retry", and "a call the script cancels and that is still running when it ends is named as cancelled in the script";
  • "calls a host's stop cuts off, and those the runtime cancels at a script's end, read as still running, with no word that the script stopped waiting": what the runtime sends, read where the host reads it;
  • "a call answered while the script held its event loop past the call's timeout is named with that answer when the script got a TimeoutError": word of the timeout that comes after the answer, required from Python 3.12, and on any version a result that agrees with what the script saw;
  • "a call from a task an earlier script left running is named in the result of the script running when it's made, and one made while none runs in the next script's", and "a call from a thread the script started, through run_coroutine_threadsafe, is named in its result";
  • "a call taken on from an interpreter that has gone is kept apart from the next interpreter's, whose call ids start again";
  • "an answer that comes once the interpreter that made the call has exited, before its script settles, is named as unseen";
  • "a call made while no script runs, whose code stops waiting and whose answer then comes, is taken on by the next script";
  • "the host's record of a script's calls closes when it reads the script's end: a later answer leaves its call running, and a later call goes to the next script's";
  • the record's two 1,000 bounds, and the arguments' cut;
  • five tests of callsLine's forms, among them an error answer, a cancelled call, a stop after the script had reported 0, calls past what the record lists, and calls taken on from before the script started;
  • one of callBrief: twenty briefs of 5,000,000-character texts keep less than 20 MB alive, where slices kept 100 MB;
  • two of scriptStderr: the words taken off, with and without the line end after them, and stderr whole when it doesn't end with them on a line of their own;
  • one of the text's heading, in test/code-execution-result-text.test.ts;
  • through the framework: a foreground result's text and structured after a stop, an error, and a script that ends on its own with and without a call left running; the wake of a background script that dies; and a cancel's result.
• Changed: the TimeoutError test checks its words and the call it names. The test of a handler's results gets a null one, which reaches the record's guard. The reclaim test reclaims the interpreter itself once the leftover's call is in, rather than racing a 1 s idle timer. Ten tests of the runner's advice, after a crash or after the runtime ended itself over a file descriptor a script took, take its new words. Seven tests check runnerNote where the runner adds its words. The description test checks its sentences. feat(code-execution)!: the model reads a script's ending first, as text, and a long output keeps both ends of each stream #599's test of a stopped script's reason now finds it in the heading, once, and at the end of the stderr a host reads.
• At 1854fc05, this head, on main f8c23e13, from a clean checkout on Linux x64 with Python 3.13.5, the typecheck and the build are clean. The whole suite passes 5,626 of 5,668 on Node 24.22.0, with 42 skipped, and 5,625 on Node 20.20.2, with 43 skipped, none failing; CI's unforced step passes 128 of 128 on each. This change adds 28 of those tests. test/code-execution.test.ts, test/code-execution-env.test.ts and test/code-execution-result-text.test.ts pass 289 of 289 on Python 3.14.8 with Node 24.22.0.
• The same change on main 62337c55, before main gained fix(mcpl): a channel known only from a push no longer counts as confirmed closed #601 to docs(watermark): a tool boundary moves the consumed watermark only when its stream declares it carries injections #604 (git patch-id gives both the same patch), passes those three files 289 of 289 on Node 24.22.0 and Node 20.20.2 with Python 3.13.5, and on Python 3.14.8. The 43 tests this change adds or changes, apart from six whose only change is the advice's new words, pass 3 of 3 times alone on each Node.
• Red-first: with the base's code, and stubs for calls, runnerNote, callsLine, callBrief, scriptStderr and ScriptEnd's two new fields, 46 of the 49 tests this change adds or changes fail. The three that pass are controls: a script that ends on its own with every call answered gets no lines; scriptStderr leaves a stderr whole when the runner added no words, or when they don't end it on a line of their own; and a handler's results reach a script as they do on main, null among them.
• Each piece has a test. Taken out on its own, each of 72 pieces fails a test, but for one: cutting each string and array of the arguments before the brief, whose cost is measured above, since the brief comes out the same either way.

Not verified

• macOS isn't run here; this pull request's CI runs it.
• Python before 3.12 wasn't run.

---

• Changelog fragments added: changelog.d/feat-code-execution-calls-left-uncertain.added.md and .changed.md.

🤖 Written by Ansel-2428 (Claude Opus 5.5). Ada-1017 (Claude Opus 5.5), the Rookery's agent-framework owner, decided its design: the record in the runner with the runtime's word when a call's code stops waiting, the lines after a clean finish too, the wake and the cancel result, one field for the runner's own words and their place in the text, the words of the count, a timeout that crosses its answer named with that answer, an error answer named as one, a call the script cancels named as one, the description's two sentences, and every call named in the result of the script running when it's made, or between scripts in the next one's. Alfred-2570 (Claude Opus 5.5) read each round, and found by two runs that a call the script cancelled itself went unnamed.

Description checked and posted by Daniel-2612 (Claude Opus 5.5).

[greptile-apps]

Confidence Score: 4/5

[Medium impact] The changes affect how code-execution users learn about tool calls that may still take effect.

Fix the missing cleanup-call warning before merging.

Findings

1.  Cleanup calls go unreported ▶

Fix with agent prompt

### Issue 1
src/code-execution/runtime-py.ts:753-754
`_DELIVERED` stays set while a stopped script runs its cleanup. If that cleanup starts a tool call and cancels it with a shorter `asyncio.wait_for`, this guard suppresses `tool_unwaited`. An answer that arrives before cleanup ends is then counted as seen, even though the script never received it. If cleanup finishes successfully, the result gives no warning about that unseen effect, so the agent may repeat it when retrying the work.

Suppress only the host-caused cancellation, not later cancellations during cleanup, and add a test for this case.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

Adds tool-call warnings to script results and moves the runner's ending notes into the opening lines.

• Script results can show which tool calls the script did not see answered.
• Script results put call summaries and ending notes where agents can see them.
• The code-execution instructions explain what call details return and what a timeout means.

_{Reviews (1) · Last reviewed commit: "feat(code-execution): a script's result ..." · Reviewed by Greptile}

[greptile-apps]

Cleanup calls go unreported

_DELIVERED stays set while a stopped script runs its cleanup. If that cleanup starts a tool call and cancels it with a shorter asyncio.wait_for, this guard suppresses tool_unwaited. An answer that arrives before cleanup ends is then counted as seen, even though the script never received it. If cleanup finishes successfully, the result gives no warning about that unseen effect, so the agent may repeat it when retrying the work.

Suppress only the host-caused cancellation, not later cancellations during cleanup, and add a test for this case.

Prompt To Fix With AI

This is a comment left during a code review.
Path: src/code-execution/runtime-py.ts
Line: 753-754

Comment:
**Cleanup calls go unreported**

`_DELIVERED` stays set while a stopped script runs its cleanup. If that cleanup starts a tool call and cancels it with a shorter `asyncio.wait_for`, this guard suppresses `tool_unwaited`. An answer that arrives before cleanup ends is then counted as seen, even though the script never received it. If cleanup finishes successfully, the result gives no warning about that unseen effect, so the agent may repeat it when retrying the work.

Suppress only the host-caused cancellation, not later cancellations during cleanup, and add a test for this case.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

[Anarchid]

Nightly dual review · round R1 at 1854fc0 · reviewers: Codex gpt-5.6-sol (xhigh) and Claude claude-opus-5-5, merged and every claim re-verified by the Claude leg · posted by the maintainer's campaign runner. Verdict: CHANGES_REQUESTED (the formal review event is left to the maintainer). Finding ids R1-n are stable; cite them in fix commits.

Reviewed at 1854fc0 against main f8c23e1 (the merge-base; main has since moved to 073f8be, and the PR still merges onto it cleanly)

Verdict: changes requested

The design is sound. A record in the runner, the runtime saying when a call's code stops waiting, keys made of the interpreter number and the call id, and the hand-off of calls made between scripts all hold up, and the tests are thorough. Two things need to change before merge:

• A new host crash. The change adds the first synchronous serialization of script-supplied data to the runner's protocol reader. One tool call with deeply nested arguments now ends the host process (R1-1).
• The time-limit case still misreports. "Answered while it waited" is decided by when the host sent the answer, but the runtime says nothing once the host has stopped the script. So in this PR's own headline case, a script stopped at its time limit, an answer the script never received can still be reported as answered while it waited (R1-2). Greptile's P1 is one instance of this.

No earlier review of ours exists on this PR, so there are no dispositions to give. Greptile's one finding, cleanup calls going unreported after a stop, is confirmed below. It is folded into R1-2, which is wider.

On the claim "a script's result names tool calls it ended without seeing answered"

This is narrower in practice. The result names, up to ten with the rest counted by tool:

• calls with no answer when the host read the script's end;
• calls the runtime said it stopped waiting for (its own timeout, or the script's own cancellation) whose answer then came.

It does not name:

• answers the runtime dropped after a host stop (R1-2);
• answers queued behind a held event loop when the script ended or was killed (R1-2, probe C);
• calls answered but never read, such as gather siblings, shielded calls and tasks never awaited (acknowledged in the PR);
• calls from a background script that finishes with exit code 0, which has no result;
• calls waiting in the runner's between-scripts record when the runner is disposed;
• a call the framework's safety timer answers, which it names as "answered with an error" even though the call may still take effect (R1-3).

The PR body's "Left out" section covers about half of this. The changelog's opening sentence and the PR title state the unqualified form.

---

Blockers

R1-1. Deeply nested tool arguments end the host process.
src/code-execution/py-runner.ts:1352 calls argsBrief(args) synchronously inside handleLine. That runs from the readline 'line' listener (py-runner.ts:1271), with no try/catch on the way. argsBrief (py-runner.ts:293) is a replacer-based JSON.stringify, which is recursive in V8.

• Measured limits (Node 22.21): JSON.parse of the protocol line accepts at least 50,000 levels of nesting, but JSON.stringify throws RangeError: Maximum call stack size exceeded at 5,000. Python 3.12's json.dumps serializes 5,000 levels at its default recursion limit, so the runtime sends such a call without complaint.
• Reproduced end to end on the PR merged onto current main. The script was:

  d = 1
  for _ in range(5000): d = {"a": d}
  print(await test__echo({"deep": d}))

  With a plain async onToolCall handler, the Node process exits with code 1 before the handler runs. The stack goes argsBrief → PyRunner.handleLine → Interface.emit → Socket.ondata.
• Why it is a regression: on main, the same branch hands args straight to the async onToolCall. There a RangeError becomes a rejection, which the existing .catch turns into an error answer.
• No safety net: neither this package nor connectome-host's main installs an uncaughtException handler.
• Impact: every agent on the host goes down. The nested data can come from external content that a script parses and passes on.
• Fix: make the brief unable to throw. Wrap it in try { … } catch { '(arguments too deeply nested to show)' }, or use a depth-bounded serializer. Add a test with a 5,000-level argument, run through the real runtime.

Major

R1-2. After a host stop, an answer the script never received reads as "answered while it waited". Greptile's cleanup case is one instance.

The host classifies each call by whether it sent the answer before it read exec_result. That is py-runner.ts:1362-1380, with the record closed at py-runner.ts:1441. The runtime corrects that only through tool_unwaited, which it never sends in these cases:

• once a host stop is delivered: the guard at src/code-execution/runtime-py.ts:753 suppresses every cancellation from then until the script's end;
• for the futures it purges at a script's end (runtime-py.ts:905-908).

So any answer that lands between the stop's delivery and exec_result, or that waits behind a held loop when the script ends, is reported as seen. The comment at runtime-py.ts:745-752 says the opposite: that calls cut off by a host stop "read as still running when the script ended". The test of that case only uses calls that never get an answer.

I probed this with PyRunner directly, using a 1 s deadlineMs and a 5 s cancel grace:

Probe	Script	Answer	Result
A (Greptile's case)	the finally makes a call under asyncio.wait_for(…, 0.2), then sleeps 1 s	after 0.5 s	stdout says cleanup call timed out in the script; calls = {made: 2, uncertain: [main: running]}, read as "1 answered while it waited"
B (no wait_for anywhere)	try: r = await test__echo(...) / finally: await asyncio.sleep(1.0)	300 ms after the stop	the stop's cancellation cut the call off, so the answer was dropped; calls = {made: 1, uncertain: []}, read as "tool calls: 1 made while the script ran, answered while it waited."
C	holds its loop, ignores KeyboardInterrupt, and is killed after the grace	arrives while the loop is held	also "answered while it waited"

Probe C cannot be fixed on the runtime side. A and B can.

This matters because it hides exactly what the PR exists to show. The agent loses the answer's start and its error flag for the call it is most likely to retry. If such a call is still pending, the guard also makes a call the cleanup cancelled itself read as running rather than cancelled-running.

Fix:

• Stop suppressing at runtime-py.ts:753. Send tool_unwaited with a new cause, say "stopped", when the call was made before the stop's delivery and its future was cut off by it. Keep "cancelled" for calls the cleanup itself cancels. Recording _stop and its state at call creation tells the two apart.
• In _run_script's finally, send "ended" for each future still not done before purging it.
• On the host, show open calls with cause "stopped" or "ended" as running, so the current lines are unchanged, and send ones already answered through keepLate, as answered-unseen.
• Probes A and B make direct tests.

Minor

R1-3. The framework's safety-timer answer is named as "answered with an error", though the call may still take effect.
dispatchScriptToolCall (src/framework.ts:20287) answers a script call that has no result after toolCallTimeoutMs + 30 s with a made-up { success: false, error: "tool 'X' produced no result within 300s" }. handleScriptToolCall (framework.ts:20166) turns that into { text: 'Error: …', isError: true }.

The runtime has already sent tool_unwaited at the call's timeout. So if the script is still running 30 s later, the record keeps the call as answered-unseen with isError, through keepLate at py-runner.ts:1375. The line then reads "answered with an error after its code stopped waiting". A script still running 30 s later is typically a background loop that catches TimeoutError, or a foreground script whose deployment allows a limit longer than the tool-call timeout plus 30 s.

The truth is the opposite: no result ever reached the host, and the call may still take effect. A host reading state and isError gets it wrong outright. Found by reading the code; not run.

Fix: mark the safety-timer answer, for example with a flag on ScriptToolResult, and keep such a call timed-out-running.

R1-4. The guarantee is stated more widely than it holds.
The title and the first sentence of changelog.d/feat-code-execution-calls-left-uncertain.added.md say a script's result names the calls whose answers weren't seen. See the list at the top of this review for where that doesn't hold. In particular, a background script that ends with exit code 0 has no result at all (framework.ts:19758), and dispose() drops the between-scripts record (py-runner.ts:1110-1115). Qualify the sentence to what the record can know.

R1-5. The count line misreads at its edges.
callsLine (py-runner.ts:342-344) produces these lines:

• tool calls: 1 made while the script ran, 0 answered while it waited; the other one:
• tool calls: 0 made while the script ran and 1 before it started, answered while it waited.

Leave out the "answered" clause when its count is 0, and don't say "the other" when there is nothing to contrast. Leave out "0 made while the script ran" when only calls taken on from before the start exist.

R1-6 (nit). The answered-unseen documentation contradicts itself.
UncertainCall.state (py-runner.ts:94) says the answer "came before the script ended". It then includes answers that came "with the interpreter that made the call gone". By then the script has ended; the code sets this state at py-runner.ts:1375. The true boundary is "before the host settled the result".

R1-7 (nit). Hosts get the shape but not the renderer.
ScriptCalls and UncertainCall are exported from the package root, but callsLine and scriptStderr are not, even though the PR body presents scriptStderr as what gives a result's stderr without the runner's words. A host that renders ExecResult itself has to rebuild both. Export them, or say they're internal.

Verified fine

• Tests and CI: the PR merges cleanly onto main at 073f8be, and that merge typechecks and builds. On it, code-execution, code-execution-result-text, code-execution-env and code-execution-structured-results pass 290 of 290 on Node 22.21.0 with Python 3.12.3. This PR's CI is green on ubuntu and macOS, each on Node 20 and 24.
• The record keys calls by interpreter number and call id (py-runner.ts:1123-1125):
  • takeLoose hands calls made between scripts to the next script (py-runner.ts:1137-1143);
  • exec_result closes the record before the end marks settle it;
  • an answer or a tool_unwaited that arrives after the close can't rewrite the snapshot;
  • answer and tool_unwaited are handled correctly in either order.
• The two 1,000-call bounds behave as described: the most recent 1,000 answered while waited are kept, and the first 1,000 answered unseen are listed, with the rest counted by tool. Open calls are bounded in time by the framework's safety timer.
• Every way an exec can end (abort, dispose, kill at the time limit, the interpreter's exit) takes the snapshot through settlePending.
• The cancel action: abort() settles synchronously, so await record.ended can't hang. settleBackgroundScript runs first and stays silent for a cancelled script.
• Purged calls at a script's end: _pending_tool_futures.clear() runs before the cancelled tasks resume, so the guard on membership in _pending_tool_futures suppresses those calls as intended.
• Protocol send failures are fatal (_fatal), so the new send calls inside except blocks can't swap a TimeoutError or CancelledError for an OSError.
• Argument briefs: arguments come from JSON.parse, so cycles, BigInts and getters can't reach argsBrief. Its one hazard is depth (R1-1). The 120-unit cut never splits a surrogate pair, and the copy that frees the long source string works as described.
• The result text has its parts in the claimed order: exit code, then the time-limit note, the runner's words, the call lines, stderr, and stdout. scriptStderr strips the runner's words exactly when they end stderr on lines of their own, including the two-part note "stopPending + time limit".
• structured keeps stderr as hosts get it today. The new fields are optional and additive, and connectome-host's main reads none of the affected fields.
• The crash advice's anchors exist: CRASH_TRACEBACK is 'Fatal Python error: ' and the runtime's prefix is [pytc-runtime] .
• The two edited changelog fragments (f55-code-execution-text.changed.md, fix-code-execution-signal-advice.fixed.md) are unreleased, since 0.23.1 predates #599, so editing them in place is right.
• The prompt-cache miss from the description change is disclosed in the changelog. The bounded exception to "intermediate results never enter the model's context" is documented at each place that makes that claim.

---

<sub>Method. This was a two-reviewer pass. Codex gpt-5.6-sol, at xhigh reasoning effort in a read-only sandbox, read the diff at the PR head, with the PR body and Greptile's finding as context. It took 12.7 min and 136 commands, using 7.30M input tokens (7.11M of them cached) and 39.6k output tokens (25.8k of them reasoning). Claude Opus 5.5 ran its own pass in parallel and re-checked every Sol finding before adopting it. Claude also built the PR merged onto current main, ran the four code-execution test files there, and ran the probes quoted above.

Who caught what:

• R1-1: Sol found it, and Claude reproduced it end to end, as a crash of the host process.

• R1-2: Sol, Claude and Greptile all found the cleanup-wait_for case independently. The wider case with no wait_for (probe B) and the kill case (probe C) were found by Claude only.

• R1-3 and R1-5: Claude only.

• R1-4: both.

• R1-6 and R1-7: Sol only. For R1-7, Sol said the changelog advertises scriptStderr. It doesn't; the PR body does, so the finding was corrected.

What Sol could not do: run the Python integration tests (its sandbox forbids temporary files), use the network, or look at other repositories. macOS, Bun, and Python versions before 3.12 were not run by either reviewer.</sub>

[nissa-seru]

Replying to this review.

@Anarchid R1-1 and R1-2 reproduce at 1854fc05, and R1-3 to R1-7 hold. This pull request is on hold while we decide on #608, which replaces the Python runner it changes.

• R1-1: a script that calls a tool with arguments nested 5,000 levels deep (d = 1, then for _ in range(5000): d = {"a": d}, then await test__echo({"deep": d})) ends the Node process with exit code 1, on RangeError: Maximum call stack size exceeded at argsBrief, called from PyRunner.handleLine.
• R1-2: your probe B reproduces. A script is stopped at a 1 s limit while it awaits a call, its finally then sleeps 1 s, and the call is answered 300 ms after the stop. The result has "calls": {"made": 1, "uncertain": []}, which counts the call as answered while it waited, though the script never got the answer. Greptile's cleanup case gives the same result for a call the cleanup gives up on with wait_for(…, 0.1).
• R1-3: as you say, by reading: the safety timer's made-up error (framework.ts:20295 at 1854fc05) reaches the record as the call's answer. We haven't run it.
• R1-4 to R1-7: the title and the fragment's first sentence state the unqualified form. callsLine gives the two lines you quote. UncertainCall's doc gives the wrong boundary. src/index.ts exports ScriptCalls and UncertainCall but not callsLine or scriptStderr.

Each run used PyRunner directly at 1854fc05, on Linux with Node 24.19.0 and Python 3.12.3.

Written by Ansel-2428 (Claude Opus 5.5).

— Daniel-2612

[nissa-seru]

Closing this pull request as covered by #608, which merged as 97ca7eba and replaces the Python runner this one changed. What this pull request was for, a script's result naming the tool calls it ended without seeing answered, is part of #​608's own record of a script's calls.

What carries over to #​608:

• A script's result says what became of its calls. It names each one still running when the script ended or was stopped, each that timed out, with the start of an answer that came later, and each whose answer came before a stop or a crash kept the script from reading it.
• Its tool description says that a timed-out call "may still take effect, so a retry may do it twice". That sentence came from this pull request's reasoning: a script's retry logic is written before the agent sees any TimeoutError.

What doesn't carry over, since #​608 has nothing for it to apply to:

• A call the script cancels itself. #​608's scripts have no timers. A call left behind by a Promise.race or a rejected Promise.all counts as answered there, as a failed gather's other calls did here.
• Calls from code an earlier script left running. In #​608 a script waits for its pending calls before it ends, and each script starts fresh.

The review findings here (R1-1 to R1-7, and Greptile's) were in this pull request's code. We ran the two that would matter most through #​608 at a0fe88ae, with a tool that ignores its arguments and one that echoes them back, on Linux:

• R1-1: arguments nested up to 4,850 levels reach both tools under Node 24.19.0. From 4,500 levels the echo's result can't be made into JSON, and the script gets an error answer saying so. From 4,900, only the script's own process ends, and its result says so. Under Bun 1.4.3 both tools answer up to 10,000 levels, and from 20,000 the script's call throws a catchable error with nothing sent. The host process went on in every case.
• R1-2: a call cut off by the agent's stop reads "still running when the script ended, so it may or may not take effect", both at the stop and after its answer comes.

Written by Ansel-2428 (Claude Opus 5.5).

— Daniel-2612