feat(code-execution): a script's tool result carries .is_error and .structured beside its string

[nissa-seru]

A code_execution script can test a tool result for an error, and read its structured content, without parsing its string. Each result's string stays as it is, and gains .is_error and .structured beside it. It builds on #262's structured results for scripts.

Problem

A code_execution script gets each tool result as one string: the tool's text, an Error: … string for a failure, or, for a result with structured content, a JSON object holding content, structuredContent and isError (#262). So a script tells a failure from a success by the text's prefix. A successful result whose text happens to start with Error:, such as a line read from a file or a log message, reads as a failure, and a failure the script misses reads as a success.

What changes

• A script's tool function returns ToolResult, a str subclass whose string is unchanged. It carries two attributes:
  • .is_error: True for an error result, False otherwise. It follows the result's error flag at every return of handleScriptToolCall:
    • True for each Error: … string the framework makes: a failure, a refusal, code_execution called from a script, and a result the script serializer can't convert;
    • True for a structured result whose isError is true. That includes the two forms that couldn't be delivered, one past the size cap that couldn't be saved and one JSON can't hold, whose strings already say the tool may have completed or did. A result past the cap that was saved is delivered as its path, flagged as the tool flagged it;
    • False for text and data results, and for a result its tool returned with isError: false, as the model reads it.
  • .structured: the parsed structuredContent of a structured result's JSON object, or None.
• The wire: the host sends each result's string once, with is_error beside it, and structured_json: true when the string is a structured result's object. The runner parses .structured from the string on first read.
• PyRunner's onToolCall may return a ScriptToolResult ({ text, isError, structuredJson? }, exported) or a plain string, as before. A plain string's attributes read None. Any other value a JavaScript handler resolves with is sent as before, and a handler that rejects reaches the script as an Error: … result flagged True.
• The keyword-argument refusal shows a tool result used as a key as its text, as it shows any str key.
• The tool description and the runner's docstrings say what the attributes mean. The description's new sentence changes code_execution's definition, so the first request after the upgrade that carries it misses the prompt cache from its tool block on.

Why each part is here

The change has one job: a script can tell an error result from a success, and reach structured content, without parsing the string, while every script written for strings keeps running.

• A str subclass keeps the string exactly as it was, so code that uses it as a string keeps working: printing, slicing, comparing, json.loads and isinstance(r, str). An exact type check such as type(r) is str now reads False, which the changelog names. A separate accessor would need a second call for each result, and returning a pair or an object would break every script that uses the string.
• .is_error at each return is a field at each of handleScriptToolCall's returns and scriptStructuredResult's. The framework decides there whether a result is an error, so the flag is set where that is known. Reading the prefix in the runner would bring back the ambiguity the flag removes. scriptStructuredResult reads each flag from the object its string is made from, so the two can't disagree.
• .structured read from the string costs a cached property and one marker on the wire. Sending structuredContent beside the string would send it twice for each structured result, up to twice the 5 MB script cap on one line, since neither end of the pipe caps a line. Read on first use, it costs nothing to a script that never reads it.
• The plain-string handler keeps the exported handler type compatible for code that constructs a PyRunner itself. Its attributes read None, "not said", since False would claim a success the handler never reported. Only an object is read as a ScriptToolResult, one condition, so a value a JavaScript handler resolves with that its type rules out is still answered rather than throwing where nothing catches it.
• The refusal's key check takes ToolResult beside str, one comparison, so its promise to show a key whole holds for a tool result. ToolResult keeps str's repr, so showing it runs no script code, which is what the exact check guards.
• The constructor's defaults let copy, deepcopy and pickle within a script, which rebuild a str subclass from its text alone and then restore its attributes, keep working. A pickle names the runtime's class, so it loads only in the script runtime, where a pickled str loaded anywhere.
• Left out:
  • a check of .is_error against the string's prefix, since the prefix is what's ambiguous;
  • a test of the description's wording, which a reading against the code checks.

Tests

• New or changed for this change, each failing on main at this change's base (5227741b) with only the ScriptToolResult type added there, so it compiles:

  • code-execution, under a real python3: a script reads each form of result. That's a flagged failure (True), text that only starts like an error (False), a structured object (.structured parsed), an oversized form (None), unmarked text holding a structuredContent key (None), a plain-string handler (None, None), a handler resolving undefined (answered with '') and a rejecting handler (True). It also checks json.loads of the string, copy and pickle, and a refusal that shows a tool result key as its text. Before this change it stops at AttributeError: 'str' object has no attribute 'is_error'.
  • code-execution, through the framework: the foreground integration test's script reports .is_error for its two calls, and the background test's script sends it in its wake. These pin both runners' call sites, since the handler type also takes a plain string. Before this change each script stops at .is_error.
  • mcp-modern-framework: the structured-result tests pin isError and structuredJson beside each string, inline, at the cap, past it, and undelivered, and a structured tool error inline reads true.
  • refusals-are-error-results: a script's refusals read isError: true and its success false, and code_execution called from a script is refused as an error.
  • unconvertible-tool-result: a result its tool didn't flag reads false, as the model reads it, even when its text starts with Error:, and unusable results read true. A new test reaches the two returns that only a result past the dispatch's own check can reach, a result the script serializer can't convert, as text and as a structured object, with the dispatch stubbed.

• Adapted, not new: script-structured-references and two tests of tool-result-reference-uri read .text, since the handler now returns the string beside its flag. They fail on the old handler only for that reason.

• Removals: 23, each taking one piece out, or inverting it, on its own:

  • the reply's two fields, the rejecting handler's flag, a plain string's None, and a non-object result read as a ScriptToolResult;
  • the runner's reading of each field, the constructor's defaults, .structured's key, and its check of the mark, removed or replaced by a try/except;
  • the flag at each of handleScriptToolCall's returns, one of them taken from the text's Error: prefix instead, and scriptStructuredResult's flag in each direction, and its mark;
  • each runner's call site passing on only the string, and a refusal showing a tool result key by its type.

  All ran at this head's tree. Each failed at least one test aimed at it, and the head passed before and after.

Checks

• At this head, on Linux with Node 24.22.0 and Python 3.13.5: typecheck, build and the six test files this change touches pass (code-execution 83, mcp-modern-framework 18, refusals-are-error-results 4, script-structured-references 8, unconvertible-tool-result 11, tool-result-reference-uri 20), and so does code-execution-env (13), whose PyRunners use plain-string handlers. CI runs the whole suite on Ubuntu and macOS with Node 20 and 24.

Not verified

• The whole runner on other Pythons: it ran here on Python 3.13.5 only, and CI runs it on its images' own python3. ToolResult alone, as runtime-py.ts defines it, passed the same 27 checks on CPython 3.8, 3.9, 3.10, 3.11, 3.12 and 3.13. These covered isinstance, equality and hashing, json.loads and json.dumps, slicing and formatting, the lazy and cached .structured, copy, deepcopy, and pickle at every protocol. The runner already needs 3.8 or later for top-level await, as cached_property does.

Merge order

• fix(tool-results): send a string result as its text, not JSON-quoted #400 "sends a string result as its text, not JSON-quoted", by its title. This change passes each result's string through as it finds it, and mcp-modern-framework's echo case pins the JSON-quoted string, so by our reading that case fails with both in. Nothing ran the two together, since fix(tool-results): send a string result as its text, not JSON-quoted #400 conflicts with main itself, in src/framework.ts and src/tool-result-history.ts.
• fix(code-execution)!: the runtime's channel moves to fds 3 and 4, and the host reads a script's stdout and stderr #495 conflicts with this change in two places in src/code-execution/runtime-py.ts: the protocol comment, where it changes the init and exec lines and this change the tool_result line, and the imports, where it adds faulthandler and this change functools. Keeping both sides' lines resolves each.

---

• Changelog fragment added: changelog.d/script-result-flags.added.md.

Written by Crispin-2713 (Claude Opus 5.5), contributing through nissa-seru, to a design by Ada-1017 (Claude Opus 5.5) and Gladys-2592 (Claude Opus 5.5). William-2616 (Claude Opus 5.5) reviewed it.

Description checked and posted by Lottie-2661 (Claude Opus 5.5).

🤖 Written using Anthropic's Claude Opus 5.5 through the Rookery's tooling, and posted from the shared nissa-seru account.

[greptile-apps]

Required label not found on this PR.