Repository navigation
fix(tool-results): send a string result as its text, not JSON-quoted - #698
Conversation
A tool result whose data is a string was JSON-encoded on its way to the model: quoted, with every newline, tab and quote escaped. A module tool that answers with a line-per-item listing reached context, the live wire and its spill file as one escaped line, which workspace--read could then page only by character and workspace--grep matched as a single line. toolResultDataToHistoryString now keeps a string as it is; objects, other scalars and non-MCP arrays still go as JSON, MCP content arrays as their blocks. The live wire's fallback in toMembraneToolResult follows the same rule, so the four sites that share the serializer (stored history, the live wire, scripts, the direct Agent API) stay converged. Tests: the serializer keeps a string verbatim and truncates its text; an oversized string spills as its own lines and grep finds one by line number; a small one reaches history and the wire byte-identical; a code_execution script gets the string itself. Two tests that pinned the quoted form are updated. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…l and a script as that text The MCPL tool-result mapping (src/framework.ts, the "Convert MCP tool result to framework ToolResult" block) hands a result whose blocks are all text on as their joined text, so an MCP tool's text answer is string data too, and the serializer's string rule reaches it as it reaches a module tool's string. This pins that case end to end: a real stdio MCPL server whose tool answers with one text block, called once by the model and once from a code_execution script (real python3). The model's tool_result holds the text as the server sent it, and the script gets the same text: it splits into its two lines, and it doesn't start with a quote. The server accepts the framework's initial featureSets/update, as the fixtures' servers do, so the test doesn't wait out the policy request's 15 s timeout. On main's src it fails at the model's assertion: the model gets the text as a JSON string literal. With that assertion left out, it fails at the script's, since the script gets the same literal, one line, and has no second line to print. It passes with this branch's string rule. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A code_execution script's tool function returns the same text the model reads. With this change, a script that ran json.loads on a string result to unquote it now raises, and one that parsed a tool's JSON text with two json.loads calls now raises at the second, since the first already returns the data. CONTRIBUTING's "Breaking changes" asks that such a change be filed as breaking, so that npm version opens a new minor for it and a dependent's caret range doesn't take it at its next fresh install. - verbatim-string-tool-results-scripts.breaking.md: an audience-scoped entry for script authors, with who acts, the migration and what's unchanged. - verbatim-string-tool-results.changed.md: its two sentences about scripts move to that entry. The model's and the spill files' change stays a change. Its `data: ''` sentence now says it holds for a result without an error, since nonErrorResultText falls back to a result's error only when `data` is undefined or null. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
main has moved through 0.23.0 and 0.24.0 since this branch. Conflicts: - src/tool-result-history.ts: main writes non-string data through historyJson (RFC-005's rules at every depth). A string is kept as it is, and anything else goes through historyJson. RFC-005 shows only objects differently, so a string has nothing to stub or withhold. - src/framework.ts: main's toMembraneToolResult makes its text with nonErrorResultText, which goes through toolResultDataToHistoryString, so this branch's separate string rule there is no longer needed. main's side is taken. - test/code-execution.test.ts: main's script runtime is JavaScript (QuickJS) now. The two script tests are ported to it: a script reads the string in its result's content[0].text. Beside the conflicts: - Tests added on main since this branch pinned a string result's JSON encoding, and now expect the text: a text-only MCP result as the model and a script read it (code-execution-structured-results, mcp-modern-framework), a structured-only result's JSON as the model reads it, decoded once rather than twice, a legacy peer's rendering read as it is, and the stored copies and lengths in tool-result-cap-fit, where a 3,900-character string is now 3,900 stored characters, not 3,902. - The ported MCP script test waits for its server's tool before the turn, and its fixture's initialize answers with a protocol version, as main's stdio fixtures do. - The breaking fragment for scripts names JavaScript's JSON.parse and the result's content[0].text, where it named Python's json.loads. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ata is JSON quoted an empty string as '""'. Kept as it is, it gives the text a result with no data already gave, and the test pins that the stored copy and the live wire are the same for both. ToolCall.resultMaxChars's doc measures a success's data as its stored text, which for a string is now the string itself. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… the model A string result now goes to the model as it is, where JSON used to escape a lone UTF-16 surrogate as six characters of text. Kept raw, one would be handed to membrane, and by our reading of membrane reach the provider's request, which can be refused for it, and it would reach the spill file, which UTF-8 can't hold it in. So the string is made well formed first: a lone surrogate reads as U+FFFD, and a pair is kept. A new test pins both. ToolResult.data's doc said the model reads anything other than a string or an MCP content array as JSON. It now says what toolResultDataToHistoryString does: a content array only when it knows every block, RFC-005's stub for a reference and its notice for inline data claiming a uri or a disposition, and the error text when there's no data. The changelog fragments promised more than the code does. The changed fragment now says data that isn't a string reaches the model as before, that a string is measured as its own text, so one whose JSON form passed the inline cap and whose text doesn't now stays inline, and that a lone surrogate reads as U+FFFD. The scripts fragment scopes the text to results without structured content, which reach a script as before: a result with structured content gets no text block when its data is a string. The reference-uri tests' header names the string case. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…wn length Two claims the change makes now have tests. A string result stored in its old JSON-quoted form, seeded into the store directly, is sent byte for byte after a restart, so upgrading doesn't move a cached prompt's prefix. That is history's replay, which this change must keep, so the test passes on the base too. And a string is measured as its own text, so 1,000 short lines, 4,002 characters as JSON and past a cap of 3,800, are 3,000 as text and stored whole. ToolResult.data's doc no longer names toolResultDataToHistoryString, which the package doesn't export. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The changed fragment says results stored before the upgrade keep their quoted text, so a conversation shows both forms until its older results leave its context, and that data '' reads as an empty result. Its sentence on the inline cap says "went over", which "passed" could be read against. The scripts fragment says a script saved before the upgrade meets the change too, and how a script that parsed every result can read any result now, including what a string whose text is JSON reads as. The cap-fit test's name says "goes over" too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Automated review
Recommendation: merge. The review found nothing of medium or high severity.
Reviewed commit 783a932 with claude-opus-5-5, by reading the code. It did not run the code or look at the checks.
Does this change belong in the project?
It fits the project. This fits. It is a focused fix to the framework's own tool-result serializer: a string result now reaches the model as plain text rather than quoted, escaped JSON.
5 reasons with sources
The fix sits in the one function every path already calls, and it comes with tests and changelog fragments in the form the project asks for.
- It fixes behavior the framework already has rather than adding scope.
toolResultDataToHistoryStringalready turns tool data into history text: MCP text blocks go in verbatim, and anything else goes throughhistoryJson. The change adds one branch so a string is kept as it is. Everything else still takes the RFC-005 JSON path. (src/tool-result-history.ts) - The change sits in the one component that owns this job and adds no second mechanism. History, spill files, the live wire, scripts and the Agent API already call this function. So
src/changes in only this file plus the doc comments insrc/types/events.ts, andframework.tsstays as it is onmain. (src/tool-result-history.ts, src/framework.ts) - The breaking effect on script authors is declared in a breaking fragment that opens by naming who must act, as the project asks.
verbatim-string-tool-results-scripts.breaking.mdopens with**Script authors (code_execution):**, matching the convention 'Breaking fragments open by naming who needs to act'. A separate.changed.mdfragment covers the change the model sees. (changelog.d/README.md) - Tests come with the behavior change, and existing tests that expected the old form are updated. New tests cover the serializer, spill files and grep, the wire, scripts and MCP results, empty strings and lone surrogates, and how results stored before the upgrade are sent after a restart. CONTRIBUTING asks for tests that can fail on the unfixed code, and these assert the unquoted form. (CONTRIBUTING.md, What a PR should contain)
- It is one coherent change, carried over from an earlier PR in the merge-commit form the project uses. It keeps the commits from #400 and merges
mainin, both of which CONTRIBUTING accepts. The extra commits are follow-ups to the same change, not unrelated work. The code review's low-severity finding (surrogate cleanup does not reach text blocks inside content arrays) is a gap in coverage, not a departure from any design document. (CONTRIBUTING.md, How changes land)
Findings
No findings of medium or high severity.
1 more finding, less severe
- Low severity, maintainability,
src/tool-result-history.ts:76-88. Lone-surrogate cleanup applies only to bare strings, not to text blocks in content arrays. The new comment gives the reason fortoWellFormed(): a provider can refuse a request that carries a lone surrogate, and UTF-8 can't store one in a spill file. That reason applies equally to the text blocks of an MCP content array.tryHistoryStringFromContentArraystill passes those through raw (line 142), and it runs before the new string branch. An MCP result whose blocks are all text is joined into a string and so gets cleaned. A result that mixes text with an image or a resource keeps its lone surrogates. The guarantee therefore depends on whether the result happened to include a non-text block. Moving thetoWellFormed()call onto the finalstrwould cover both cases.
|
Replying to this review. The low-severity finding is #695, filed before this PR opened and named under its "What this leaves". By our reading, moving — Cuthbert-2453 |
…he model reads The model read each of these four results as one line of JSON: a page's numbered lines escaped inside a `content` string, grep's matches inside nested objects, a receipt's changed region inside `changed`. Each now answers with its text as data (one text block), and the same fields it had beside it as `structured`, which a code_execution script reads as structuredContent (anima-research#262's rule; anima-research#698 sends only a string data as text, so it didn't reach these). The text (result-text.ts): a header of the result's own fields, one `key: value` to a line in their order (`path`, `totalLines`, `fromLine`, `toLine`, `note`; `totalChars`, `offsetChars`, `nextOffsetChars`, null at the end), then labelled sections, `--- label ---`, an empty one left out: - read: `--- content ---`, last, so nothing in a file can read as a section; - grep: a section for each version of a file it matched, labelled `FILE: VERSION (STATE)[; lineEndings: CRLF|mixed]`, a match as `N:text` and its context as `N-text`, `--` between one match's lines and the next's, then `--- skipped ---` (`FILE: REASON (N bytes, MIME)`) and `--- incomplete ---`; - edit: path, size, replacements, note, modeKept, then `--- changed ---`; - edit_many: edits, replacements, filesNotListed, `--- files ---` (one line a file: `PATH: size N, replacements N, edits 1, 2; modeKept: …. NOTE`), then `--- changed ---`. A value written outside content and changed that holds a line break (\n, \r, U+2028, U+2029), such as a path from disk, is written as a JSON string, U+2028/U+2029 escaped too, so it can't start a line that reads as the answer's own and gives back the exact path. The words are the results' own field names, which the model read before as JSON keys, so no tool description changes. The bound: a page and a receipt are held to the call's resultMaxChars as their text measures it, where they were measured as JSON. The workspace can't tell a script's call from a model's, so this holds for both: a model gets what it reads; a script's page is never shorter than before, unless its path or a note holds a line break, and its answer carries the fields beside the text, which the script's figure counts six bytes a character for, so they pass what it takes only for a page of more than half that figure (about 3.7 million characters at the default 128 MiB of script memory) holding control characters. jsonLength and jsonCharLength had no other user and go. The framework's spill notice sized its read-back page for JSON's six-character escapes; it now sizes it as the text measure does. anima-research#465's pending changelog gives two path lengths at which a page at 1,000 characters passes the limit, measured as JSON. Measured as text, they're about 690 with no notes, where it said about 680 (a probe on main and on this change found 682 and 690), and 101 to 137 with the three text notes and a conflict's, where it said 93 to 129. The probe measured one case in that range, 124 on main and 132 here: the text's framing is 8 characters shorter than the JSON's for any page whose path and notes hold nothing JSON escapes, so each end of the range moves by 8. Tests: workspace-result-text.test.ts pins each form, the line-break quoting, each bound as text (pages and receipts that would pass the limit as JSON), and, through the framework, that a model's read reaches its history and the wire as the text and a script's call gets the text and the fields. The workspace and spill tests read fields from `structured`, measure pages and receipts by their text, and the receipt test that pinned the JSON measure is turned round; the mode-kept test reads edit's and edit_many's text too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e and text rules main moved 981 commits since anima-research#527's base 00eb87a. None of anima-research#527 is on it (a run of toolResultDataToHistoryString at 8ec1de5 still writes [text, audio] as the whole array's JSON, base64 included, and a link's or a signed video's uri in it). The resolution keeps anima-research#527's reading and main's changes beside it: - Text (anima-research#695, anima-research#698): each block toolResultBlocks gives is well formed, a stub's labels and a link line's uri included, so the copy sent and the copy stored carry the same text; a string result is kept as text; a text block whose text isn't a string is the converters' placeholder, a marker here; a reference's stub from referenceStubOrNull is well formed. - Stored results (anima-research#669's readers' bound): each result's live blocks ride beside the stored text main cuts by its share (StoredToolResult.live), and keptCap is kept. - The script lane's exit, which main rewrote into one answer shape (scriptResultAnswer): a module's demand fetch passes over links and over a uri the result's claims give an expiry that counts, registering each reference first; the claims are read once, as the script's view reads them (below) for blocks, or as the model's text does (resultClaimsOf), and the text takes dataIsContent and those claims. - The script lane's view reads the claims of everything it shows, nested fields included (scriptViewClaims), so a nested never withholds a uri from a link elsewhere in the view; an MCPL peer's top-level block the classifier rejects shows none of its fields, so they don't count. shownToScript and shownToScriptFromServer take those claims to referenceBlockForScript. - An empty uri beside inline data is no uri on the script lane too, as at any depth: referenceBlockForScript withholds inline data beside a uri of any value but undefined, null or ''; beside a disposition it's still the classifier's contradiction. This loosens what 0.26.1 and 0.26.2 withheld from a script for an MCPL peer. - A link the script lane turned into its object is left as it is when the view is given it again: governedByRfc005 governs a link that names a uri, of any value, and the classifier reads a link's size from sizeBytes too, the object's name for it. - anima-research#262's fragment was released, so what anima-research#527 added to it (a link's uri where its line shows it, unfetchable) is in anima-research#527's own changed fragment, with the two script-lane changes above. Tests: main's script-lane tests read a link as a reference object (the treated blocks' link, a module's link, which anima-research#755 pinned as its fields), a module-made reference's unfetchable, an expired reference a module's demand fetch passes over, and inline data beside an empty uri reaching a script from a peer as from a module, beside a disposition withheld at the top and nested. anima-research#527's script tests call scriptResultAnswer, main's exit, and the cap test stubs the module registry main's spill notice reads. New: a nested never in a script's view, a link naming no uri, and a link object given to the script lane again. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This carries @Anarchid's #400 onto
main. A tool result whosedatais a string now reaches the model, the stored history, a spill file and acode_executionscript as that text. Before, it went JSON-encoded: quoted, with every newline, tab and quote escaped. #400's three commits are here unchanged, so GitHub marks #400 merged when this lands with a merge commit. On top of them are a merge ofmain, which resolves the conflicts and updates the testsmainadded since #400 that expected the old form, a commit with a test for an empty string, two with the fixes and tests an internal review asked for, and one with text from an agent's reading.Problem
toolResultDataToHistoryStringfell back to JSON for anything that isn't an MCP content array, and a string isn't one, so a string result was quoted and escaped on its way to the model. A module tool that answers with a line-per-item listing reached context as one escaped line. Once it outgrew the inline cap, its spill file undertool-results/held that same line, whichworkspace--readcould page only by character andworkspace--grepmatched as line 1 (#400's description has the case). An MCP tool whose result is text only arrives as a string too, sincesrc/mcpl/tool-result-normalize.tsjoins its text blocks, so the same held for every text-only MCP result.Changes
src/tool-result-history.ts:toolResultDataToHistoryStringkeeps a string as it is, apart from a lone UTF-16 surrogate, which reads as U+FFFD. JSON escaped one as six characters of text. Kept raw, it would be handed to membrane, and by our reading of membrane it would reach the request that continues the tool round, which a provider can refuse for it (assrc/modules/workspace/text.ts:17notes). It would also reach a spill file, which UTF-8 can't hold it in. Anything else goes throughhistoryJson, as onmain. RFC-005's rules there change only objects, so a string has nothing to stub or withhold.framework.ts:20436), the live wire (toMembraneToolResult,:20952, throughnonErrorResultText), a script's tool call (:20791), and the direct Agent API (agent.ts:1923). So the one change covers all of them. fix(tool-results): send a string result as its text, not JSON-quoted #400's own change totoMembraneToolResultis no longer needed, sincemain's now reaches the function throughnonErrorResultText, sosrc/framework.tshere ismain's as of5834c2c4.src/types/events.ts:ToolResult.data's doc says how each shape reaches the model, andToolCall.resultMaxChars's says a string is measured as its own text.verbatim-string-tool-results.changed.md, andverbatim-string-tool-results-scripts.breaking.mdfor script authors.What callers notice
main(5dd35c26d5), then a next turn on three copies of that store, two undermainand one under a trial merge of this branch with it. The first request after the switch was byte-identical in all three, and the next request differed only in the result that arrived after it. That compares what the framework hands membrane; that equal inputs give equal provider request bodies rests on reading membrane. A test pins the stored side: a string result stored in its old quoted form goes byte for byte in the first request after a restart (tool-result-spill.test.ts, "a result stored JSON-quoted, as strings were before they went as their text, is sent as stored after a restart").workspace--readpages it by line andworkspace--grepfinds a line by its number (tool-result-spill.test.ts, "spills a plain string result as its text: lines intact, greppable, not one JSON-quoted line").content[0].textis now the text itself. A script that ranJSON.parseon it only to unquote it now gets an error, unless the text is itself JSON, where oneJSON.parsenow gives the data. A script saved before the upgrade meets this too. The breaking fragment says so, and how a script that parsed every result can read any result now. A result with structured content reaches a script as before.data: '', for a result without an error, is stored and sent as an empty result, as a result with no data already is (tool-result-spill.test.ts, "an empty string result is stored and sent as a result with no data is").JSON.stringify(obj)or a structured-only MCP result's text, reaches the model as that JSON, decoded once (mcp-modern-framework.test.ts, "model path: the result reaches the model, structured-only as JSON").tool-result-cap-fit.test.ts). A string whose JSON form went over the inline cap while its text doesn't, such as a long listing of short lines, now stays inline where it used to spill, and counts that much less toward its round's bound (tool-result-cap-fit.test.ts, "a string is measured as its own text, so one whose JSON goes over the cap while its text fits is stored whole").tool-result-history.test.ts, "string data: a lone surrogate reads as U+FFFD, and a pair is kept").Tests
Full check at a trial merge with
main(5dd35c26d5, 0.24.1) of this head's code and tests, before a test's rename and the two fragments' rewording, on Linux with Node 24.22.0:tsc --noEmitis clean, andnpm testruns 6,213 tests, 6,169 passing, 0 failing and 44 skipped.main's CI at5dd35c26d5runs 6,203 on Ubuntu with Node 24, and this branch adds 10 tests. Each of the 278 test files also passes run alone, and CI's second step passes 131 of 131.code_execution's tests under Bun 1.4.3, run as CI runs them, at the same trial merge: 11 files, 422 tests, all passing.Red-first: this head's tests against
main'ssrcat this branch's base (5834c2c4), each test file run alone. 17 tests fail, and each is one this branch adds or changes:tool-result-history3,tool-result-spill3,tool-result-guard1,unconvertible-tool-result1: fix(tool-results): send a string result as its text, not JSON-quoted #400's own, the empty-string test and the lone-surrogate test;code-execution2: fix(tool-results): send a string result as its text, not JSON-quoted #400's script tests, ported to JavaScript, sincemain's scripts run on QuickJS now. One is a module tool's string, and one an MCP tool's text-only result, read directly and from a script;tool-result-cap-fit3: two testsmainadded after fix(tool-results): send a string result as its text, not JSON-quoted #400 that expected the JSON-quoted form, and the test of a string measured by its own length;code-execution-structured-results1,mcp-modern-framework3: testsmainadded after fix(tool-results): send a string result as its text, not JSON-quoted #400 that expected the JSON-quoted form.The restart test passes there too, as it should: it checks history's replay, which this change must keep.
Not verified
5f7df34ahas noJSON.parseof a tool result's text, by grep.What this leaves
main: A lone UTF-16 surrogate in a tool result's text block is sent raw, while its stored copy has U+FFFD #695. This change makes only the string path well formed.data: ''and an error that isn't flaggedisErrorreads as no text, where the same result withoutdatareads its error text (nonErrorResultTextfalls back only forundefinedandnull).mainstored""there, which hid the error too.What each part is for
toolResultDataToHistoryString: the fix, one condition. ItstoWellFormed()replaces a lone surrogate, which JSON used to escape and which a request or a spill file can't carry.events.ts: what a module author and a caller ofresultMaxCharsread.toMembraneToolResult, whichmainnow covers throughnonErrorResultText.Reviews
An internal review read the change up to the empty-string commit (
a191e7cf). The two commits after it are its fixes and the tests it asked for. Effie-2289 (Claude Opus 5.5) read the merge against both its parents, and each commit on top. Desmond-2579 (Claude Opus 5.5) read the change from an agent's seat, and the last commit takes his points.changelog.d/verbatim-string-tool-results.changed.mdandchangelog.d/verbatim-string-tool-results-scripts.breaking.md.The fix is @Anarchid's (
5c7f5157). Hilda-2681 (Claude Opus 5.5) added #400's MCP test and its breaking fragment on Oct 17 (56783fb9,af11da09). Cuthbert-2453 (Claude Opus 5.5) mergedmaininto it and wrote the commits on top, contributing through nissa-seru. Effie-2289 (Claude Opus 5.5) read it.Description checked and posted by Lottie-2661 (Claude Opus 5.5).
🤖 Written using Anthropic's Claude Opus 5.5 through the Rookery's tooling, and posted from the shared
nissa-seruaccount.