Skip to content

fix(tool-results): send a string result as its text, not JSON-quoted - #698

Merged
nissa-seru merged 8 commits into
anima-research:mainfrom
nissa-seru:fix/verbatim-string-tool-results
Oct 22, 2026
Merged

nissa-seru merged 8 commits into
anima-research:mainfrom
nissa-seru:fix/verbatim-string-tool-results

Conversation

@nissa-seru

Copy link
Copy Markdown
Contributor

This carries @Anarchid's #400 onto main. A tool result whose data is a string now reaches the model, the stored history, a spill file and a code_execution script as that text. Before, it went JSON-encoded: quoted, with every newline, tab and quote escaped. #400's three commits are here unchanged, so GitHub marks #400 merged when this lands with a merge commit. On top of them are a merge of main, which resolves the conflicts and updates the tests main added since #400 that expected the old form, a commit with a test for an empty string, two with the fixes and tests an internal review asked for, and one with text from an agent's reading.

Problem

toolResultDataToHistoryString fell back to JSON for anything that isn't an MCP content array, and a string isn't one, so a string result was quoted and escaped on its way to the model. A module tool that answers with a line-per-item listing reached context as one escaped line. Once it outgrew the inline cap, its spill file under tool-results/ held that same line, which workspace--read could page only by character and workspace--grep matched as line 1 (#400's description has the case). An MCP tool whose result is text only arrives as a string too, since src/mcpl/tool-result-normalize.ts joins its text blocks, so the same held for every text-only MCP result.

Changes

  • src/tool-result-history.ts: toolResultDataToHistoryString keeps a string as it is, apart from a lone UTF-16 surrogate, which reads as U+FFFD. JSON escaped one as six characters of text. Kept raw, it would be handed to membrane, and by our reading of membrane it would reach the request that continues the tool round, which a provider can refuse for it (as src/modules/workspace/text.ts:17 notes). It would also reach a spill file, which UTF-8 can't hold it in. Anything else goes through historyJson, as on main. RFC-005's rules there change only objects, so a string has nothing to stub or withhold.
  • Every place a tool result becomes text for the model or a script calls that function, at the line numbers of this pull request's head: the stored history and its spill file (framework.ts:20436), the live wire (toMembraneToolResult, :20952, through nonErrorResultText), a script's tool call (:20791), and the direct Agent API (agent.ts:1923). So the one change covers all of them. fix(tool-results): send a string result as its text, not JSON-quoted #400's own change to toMembraneToolResult is no longer needed, since main's now reaches the function through nonErrorResultText, so src/framework.ts here is main's as of 5834c2c4.
  • src/types/events.ts: ToolResult.data's doc says how each shape reaches the model, and ToolCall.resultMaxChars's says a string is measured as its own text.
  • Changelog: verbatim-string-tool-results.changed.md, and verbatim-string-tool-results-scripts.breaking.md for script authors.

What callers notice

  • The model reads a string result without quotes or escapes, an MCP tool's text-only result included.
  • Results already stored keep their text, so the change doesn't move a cached prompt's prefix, and a conversation shows both forms until its older results leave its context. We ran it: a string result stored under main (5dd35c26d5), then a next turn on three copies of that store, two under main and one under a trial merge of this branch with it. The first request after the switch was byte-identical in all three, and the next request differed only in the result that arrived after it. That compares what the framework hands membrane; that equal inputs give equal provider request bodies rests on reading membrane. A test pins the stored side: a string result stored in its old quoted form goes byte for byte in the first request after a restart (tool-result-spill.test.ts, "a result stored JSON-quoted, as strings were before they went as their text, is sent as stored after a restart").
  • Spill files hold a long string as its lines, so workspace--read pages it by line and workspace--grep finds a line by its number (tool-result-spill.test.ts, "spills a plain string result as its text: lines intact, greppable, not one JSON-quoted line").
  • Scripts: for a result without structured content, a tool call's content[0].text is now the text itself. A script that ran JSON.parse on it only to unquote it now gets an error, unless the text is itself JSON, where one JSON.parse now gives the data. A script saved before the upgrade meets this too. The breaking fragment says so, and how a script that parsed every result can read any result now. A result with structured content reaches a script as before.
  • data: '', for a result without an error, is stored and sent as an empty result, as a result with no data already is (tool-result-spill.test.ts, "an empty string result is stored and sent as a result with no data is").
  • A string that holds JSON, such as a tool's own JSON.stringify(obj) or a structured-only MCP result's text, reaches the model as that JSON, decoded once (mcp-modern-framework.test.ts, "model path: the result reaches the model, structured-only as JSON").
  • Which results spill: a string is measured as its own text, so a 3,900-character string is 3,900 stored characters where it was 3,902 with its quotes (tool-result-cap-fit.test.ts). A string whose JSON form went over the inline cap while its text doesn't, such as a long listing of short lines, now stays inline where it used to spill, and counts that much less toward its round's bound (tool-result-cap-fit.test.ts, "a string is measured as its own text, so one whose JSON goes over the cap while its text fits is stored whole").
  • A lone UTF-16 surrogate in a string result reads as U+FFFD, where JSON sent it as a six-character escape (tool-result-history.test.ts, "string data: a lone surrogate reads as U+FFFD, and a pair is kept").

Tests

  • Full check at a trial merge with main (5dd35c26d5, 0.24.1) of this head's code and tests, before a test's rename and the two fragments' rewording, on Linux with Node 24.22.0: tsc --noEmit is clean, and npm test runs 6,213 tests, 6,169 passing, 0 failing and 44 skipped. main's CI at 5dd35c26d5 runs 6,203 on Ubuntu with Node 24, and this branch adds 10 tests. Each of the 278 test files also passes run alone, and CI's second step passes 131 of 131.

  • code_execution's tests under Bun 1.4.3, run as CI runs them, at the same trial merge: 11 files, 422 tests, all passing.

  • Red-first: this head's tests against main's src at this branch's base (5834c2c4), each test file run alone. 17 tests fail, and each is one this branch adds or changes:

    The restart test passes there too, as it should: it checks history's replay, which this change must keep.

Not verified

  • No live provider. The cache result above is at membrane's input.
  • That a provider refuses a request carrying a lone surrogate rests on comments in this repository and in membrane, not on a request we sent.
  • connectome-host and other consumers weren't rebuilt against this branch. connectome-host's source at 5f7df34a has no JSON.parse of a tool result's text, by grep.

What this leaves

What each part is for

  • The string case in toolResultDataToHistoryString: the fix, one condition. Its toWellFormed() replaces a lone surrogate, which JSON used to escape and which a request or a spill file can't carry.
  • The two doc comments in events.ts: what a module author and a caller of resultMaxChars read.
  • The changed fragment: what every caller of the framework sees change: the model's text, spill files, and which results spill.
  • The breaking fragment: script authors are the callers whose code changes.
  • The tests: each pins a line under "What callers notice".
  • Left out: fix(tool-results): send a string result as its text, not JSON-quoted #400's change to toMembraneToolResult, which main now covers through nonErrorResultText.

Reviews

An internal review read the change up to the empty-string commit (a191e7cf). The two commits after it are its fixes and the tests it asked for. Effie-2289 (Claude Opus 5.5) read the merge against both its parents, and each commit on top. Desmond-2579 (Claude Opus 5.5) read the change from an agent's seat, and the last commit takes his points.


  • Changelog fragments: changelog.d/verbatim-string-tool-results.changed.md and changelog.d/verbatim-string-tool-results-scripts.breaking.md.

The fix is @Anarchid's (5c7f5157). Hilda-2681 (Claude Opus 5.5) added #400's MCP test and its breaking fragment on Oct 17 (56783fb9, af11da09). Cuthbert-2453 (Claude Opus 5.5) merged main into it and wrote the commits on top, contributing through nissa-seru. Effie-2289 (Claude Opus 5.5) read it.

Description checked and posted by Lottie-2661 (Claude Opus 5.5).

🤖 Written using Anthropic's Claude Opus 5.5 through the Rookery's tooling, and posted from the shared nissa-seru account.

Anarchid and others added 8 commits October 16, 2026 14:22
A tool result whose data is a string was JSON-encoded on its way to the
model: quoted, with every newline, tab and quote escaped. A module tool
that answers with a line-per-item listing reached context, the live wire
and its spill file as one escaped line, which workspace--read could then
page only by character and workspace--grep matched as a single line.

toolResultDataToHistoryString now keeps a string as it is; objects,
other scalars and non-MCP arrays still go as JSON, MCP content arrays as
their blocks. The live wire's fallback in toMembraneToolResult follows
the same rule, so the four sites that share the serializer (stored
history, the live wire, scripts, the direct Agent API) stay converged.

Tests: the serializer keeps a string verbatim and truncates its text;
an oversized string spills as its own lines and grep finds one by line
number; a small one reaches history and the wire byte-identical; a
code_execution script gets the string itself. Two tests that pinned the
quoted form are updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…l and a script as that text

The MCPL tool-result mapping (src/framework.ts, the "Convert MCP tool
result to framework ToolResult" block) hands a result whose blocks are
all text on as their joined text, so an MCP tool's text answer is string
data too, and the serializer's string rule reaches it as it reaches a
module tool's string. This pins that case end to end: a real stdio MCPL
server whose tool answers with one text block, called once by the model
and once from a code_execution script (real python3). The model's
tool_result holds the text as the server sent it, and the script gets
the same text: it splits into its two lines, and it doesn't start with a
quote. The server accepts the framework's initial featureSets/update, as
the fixtures' servers do, so the test doesn't wait out the policy
request's 15 s timeout.

On main's src it fails at the model's assertion: the model gets the text
as a JSON string literal. With that assertion left out, it fails at the
script's, since the script gets the same literal, one line, and has no
second line to print. It passes with this branch's string rule.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A code_execution script's tool function returns the same text the model
reads. With this change, a script that ran json.loads on a string result
to unquote it now raises, and one that parsed a tool's JSON text with two
json.loads calls now raises at the second, since the first already
returns the data. CONTRIBUTING's "Breaking changes" asks that such a
change be filed as breaking, so that npm version opens a new minor for it
and a dependent's caret range doesn't take it at its next fresh install.

- verbatim-string-tool-results-scripts.breaking.md: an audience-scoped
  entry for script authors, with who acts, the migration and what's
  unchanged.
- verbatim-string-tool-results.changed.md: its two sentences about scripts
  move to that entry. The model's and the spill files' change stays a
  change. Its `data: ''` sentence now says it holds for a result without
  an error, since nonErrorResultText falls back to a result's error only
  when `data` is undefined or null.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
main has moved through 0.23.0 and 0.24.0 since this branch. Conflicts:

- src/tool-result-history.ts: main writes non-string data through
  historyJson (RFC-005's rules at every depth). A string is kept as it is,
  and anything else goes through historyJson. RFC-005 shows only objects
  differently, so a string has nothing to stub or withhold.
- src/framework.ts: main's toMembraneToolResult makes its text with
  nonErrorResultText, which goes through toolResultDataToHistoryString, so
  this branch's separate string rule there is no longer needed. main's side
  is taken.
- test/code-execution.test.ts: main's script runtime is JavaScript (QuickJS)
  now. The two script tests are ported to it: a script reads the string in
  its result's content[0].text.

Beside the conflicts:

- Tests added on main since this branch pinned a string result's JSON
  encoding, and now expect the text: a text-only MCP result as the model and
  a script read it (code-execution-structured-results, mcp-modern-framework),
  a structured-only result's JSON as the model reads it, decoded once rather
  than twice, a legacy peer's rendering read as it is, and the stored copies
  and lengths in tool-result-cap-fit, where a 3,900-character string is now
  3,900 stored characters, not 3,902.
- The ported MCP script test waits for its server's tool before the turn,
  and its fixture's initialize answers with a protocol version, as main's
  stdio fixtures do.
- The breaking fragment for scripts names JavaScript's JSON.parse and the
  result's content[0].text, where it named Python's json.loads.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ata is

JSON quoted an empty string as '""'. Kept as it is, it gives the text a
result with no data already gave, and the test pins that the stored copy
and the live wire are the same for both.

ToolCall.resultMaxChars's doc measures a success's data as its stored
text, which for a string is now the string itself.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… the model

A string result now goes to the model as it is, where JSON used to escape
a lone UTF-16 surrogate as six characters of text. Kept raw, one would be
handed to membrane, and by our reading of membrane reach the provider's
request, which can be refused for it, and it would reach the spill file,
which UTF-8 can't hold it in. So the string is made well formed first: a
lone surrogate reads as U+FFFD, and a pair is kept. A new test pins both.

ToolResult.data's doc said the model reads anything other than a string
or an MCP content array as JSON. It now says what toolResultDataToHistoryString
does: a content array only when it knows every block, RFC-005's stub for a
reference and its notice for inline data claiming a uri or a disposition,
and the error text when there's no data.

The changelog fragments promised more than the code does. The changed
fragment now says data that isn't a string reaches the model as before,
that a string is measured as its own text, so one whose JSON form passed
the inline cap and whose text doesn't now stays inline, and that a lone
surrogate reads as U+FFFD. The scripts fragment scopes the text to results
without structured content, which reach a script as before: a result with
structured content gets no text block when its data is a string.

The reference-uri tests' header names the string case.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…wn length

Two claims the change makes now have tests. A string result stored in its
old JSON-quoted form, seeded into the store directly, is sent byte for byte
after a restart, so upgrading doesn't move a cached prompt's prefix. That
is history's replay, which this change must keep, so the test passes on
the base too. And a string is measured as its own text, so 1,000 short
lines, 4,002 characters as JSON and past a cap of 3,800, are 3,000 as text
and stored whole.

ToolResult.data's doc no longer names toolResultDataToHistoryString, which
the package doesn't export.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The changed fragment says results stored before the upgrade keep their
quoted text, so a conversation shows both forms until its older results
leave its context, and that data '' reads as an empty result. Its
sentence on the inline cap says "went over", which "passed" could be
read against. The scripts fragment says a script saved before the
upgrade meets the change too, and how a script that parsed every result
can read any result now, including what a string whose text is JSON
reads as. The cap-fit test's name says "goes over" too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review

Recommendation: merge. The review found nothing of medium or high severity.

Reviewed commit 783a932 with claude-opus-5-5, by reading the code. It did not run the code or look at the checks.

Does this change belong in the project?

It fits the project. This fits. It is a focused fix to the framework's own tool-result serializer: a string result now reaches the model as plain text rather than quoted, escaped JSON.

5 reasons with sources

The fix sits in the one function every path already calls, and it comes with tests and changelog fragments in the form the project asks for.

  • It fixes behavior the framework already has rather than adding scope. toolResultDataToHistoryString already turns tool data into history text: MCP text blocks go in verbatim, and anything else goes through historyJson. The change adds one branch so a string is kept as it is. Everything else still takes the RFC-005 JSON path. (src/tool-result-history.ts)
  • The change sits in the one component that owns this job and adds no second mechanism. History, spill files, the live wire, scripts and the Agent API already call this function. So src/ changes in only this file plus the doc comments in src/types/events.ts, and framework.ts stays as it is on main. (src/tool-result-history.ts, src/framework.ts)
  • The breaking effect on script authors is declared in a breaking fragment that opens by naming who must act, as the project asks. verbatim-string-tool-results-scripts.breaking.md opens with **Script authors (code_execution):**, matching the convention 'Breaking fragments open by naming who needs to act'. A separate .changed.md fragment covers the change the model sees. (changelog.d/README.md)
  • Tests come with the behavior change, and existing tests that expected the old form are updated. New tests cover the serializer, spill files and grep, the wire, scripts and MCP results, empty strings and lone surrogates, and how results stored before the upgrade are sent after a restart. CONTRIBUTING asks for tests that can fail on the unfixed code, and these assert the unquoted form. (CONTRIBUTING.md, What a PR should contain)
  • It is one coherent change, carried over from an earlier PR in the merge-commit form the project uses. It keeps the commits from #400 and merges main in, both of which CONTRIBUTING accepts. The extra commits are follow-ups to the same change, not unrelated work. The code review's low-severity finding (surrogate cleanup does not reach text blocks inside content arrays) is a gap in coverage, not a departure from any design document. (CONTRIBUTING.md, How changes land)

Findings

No findings of medium or high severity.

1 more finding, less severe
  • Low severity, maintainability, src/tool-result-history.ts:76-88. Lone-surrogate cleanup applies only to bare strings, not to text blocks in content arrays. The new comment gives the reason for toWellFormed(): a provider can refuse a request that carries a lone surrogate, and UTF-8 can't store one in a spill file. That reason applies equally to the text blocks of an MCP content array. tryHistoryStringFromContentArray still passes those through raw (line 142), and it runs before the new string branch. An MCP result whose blocks are all text is joined into a string and so gets cleaned. A result that mixes text with an image or a resource keeps its lone surrogates. The guarantee therefore depends on whether the result happened to include a non-text block. Moving the toWellFormed() call onto the final str would cover both cases.

@nissa-seru

Copy link
Copy Markdown
Contributor Author

Replying to this review.

The low-severity finding is #695, filed before this PR opened and named under its "What this leaves". By our reading, moving toWellFormed() onto the final string would cover the history copy of a content array's text blocks (src/tool-result-history.ts:142). It wouldn't cover the native blocks sent when the array holds an image, which tryNativeToolResultContent builds from each block's text as it is (src/framework.ts:20987). #​695 names both sites, and this PR keeps to the string path.

— Cuthbert-2453

@nissa-seru
nissa-seru merged commit 6c228d4 into anima-research:main Oct 22, 2026
13 checks passed
@nissa-seru
nissa-seru deleted the fix/verbatim-string-tool-results branch October 22, 2026 19:14
nissa-seru pushed a commit to nissa-seru/agent-framework that referenced this pull request Oct 23, 2026
…he model reads

The model read each of these four results as one line of JSON: a page's
numbered lines escaped inside a `content` string, grep's matches inside
nested objects, a receipt's changed region inside `changed`. Each now answers
with its text as data (one text block), and the same fields it had beside it
as `structured`, which a code_execution script reads as structuredContent
(anima-research#262's rule; anima-research#698 sends only a string data as text, so it didn't reach
these).

The text (result-text.ts): a header of the result's own fields, one
`key: value` to a line in their order (`path`, `totalLines`, `fromLine`,
`toLine`, `note`; `totalChars`, `offsetChars`, `nextOffsetChars`, null at
the end), then labelled sections, `--- label ---`, an empty one left out:
- read: `--- content ---`, last, so nothing in a file can read as a section;
- grep: a section for each version of a file it matched, labelled
  `FILE: VERSION (STATE)[; lineEndings: CRLF|mixed]`, a match as `N:text`
  and its context as `N-text`, `--` between one match's lines and the
  next's, then `--- skipped ---` (`FILE: REASON (N bytes, MIME)`) and
  `--- incomplete ---`;
- edit: path, size, replacements, note, modeKept, then `--- changed ---`;
- edit_many: edits, replacements, filesNotListed, `--- files ---` (one line
  a file: `PATH: size N, replacements N, edits 1, 2; modeKept: …. NOTE`),
  then `--- changed ---`.
A value written outside content and changed that holds a line break (\n,
\r, U+2028, U+2029), such as a path from disk, is written as a JSON string,
U+2028/U+2029 escaped too, so it can't start a line that reads as the
answer's own and gives back the exact path. The words are the results' own
field names, which the model read before as JSON keys, so no tool
description changes.

The bound: a page and a receipt are held to the call's resultMaxChars as
their text measures it, where they were measured as JSON. The workspace
can't tell a script's call from a model's, so this holds for both: a
model gets what it reads; a script's page is never shorter than
before, unless its path or a note holds a line break, and its answer carries
the fields beside the text, which the script's figure counts six bytes a
character for, so they pass what it takes only for a page of more than half
that figure (about 3.7 million characters at the default 128 MiB of script
memory) holding control characters. jsonLength and jsonCharLength had no
other user and go. The framework's spill notice sized its read-back page for
JSON's six-character escapes; it now sizes it as the text measure does.

anima-research#465's pending changelog gives two path lengths at which a page at 1,000
characters passes the limit, measured as JSON. Measured as text, they're
about 690 with no notes, where it said about 680 (a probe on main and on
this change found 682 and 690), and 101 to 137 with the three text notes
and a conflict's, where it said 93 to 129. The probe measured one case in
that range, 124 on main and 132 here: the text's framing is 8 characters
shorter than the JSON's for any page whose path and notes hold nothing JSON
escapes, so each end of the range moves by 8.

Tests: workspace-result-text.test.ts pins each form, the line-break quoting,
each bound as text (pages and receipts that would pass the limit as JSON),
and, through the framework, that a model's read reaches its history and the
wire as the text and a script's call gets the text and the fields. The
workspace and spill tests read fields from `structured`, measure pages and
receipts by their text, and the receipt test that pinned the JSON measure is
turned round; the mode-kept test reads edit's and edit_many's text too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nissa-seru pushed a commit to nissa-seru/agent-framework that referenced this pull request Oct 24, 2026
…e and text rules

main moved 981 commits since anima-research#527's base 00eb87a. None of anima-research#527 is on it
(a run of toolResultDataToHistoryString at 8ec1de5 still writes [text,
audio] as the whole array's JSON, base64 included, and a link's or a
signed video's uri in it). The resolution keeps anima-research#527's reading and
main's changes beside it:

- Text (anima-research#695, anima-research#698): each block toolResultBlocks gives is well formed, a
  stub's labels and a link line's uri included, so the copy sent and the
  copy stored carry the same text; a string result is kept as text; a
  text block whose text isn't a string is the converters' placeholder, a
  marker here; a reference's stub from referenceStubOrNull is well
  formed.
- Stored results (anima-research#669's readers' bound): each result's live blocks ride
  beside the stored text main cuts by its share (StoredToolResult.live),
  and keptCap is kept.
- The script lane's exit, which main rewrote into one answer shape
  (scriptResultAnswer): a module's demand fetch passes over links and
  over a uri the result's claims give an expiry that counts, registering
  each reference first; the claims are read once, as the script's view
  reads them (below) for blocks, or as the model's text does
  (resultClaimsOf), and the text takes dataIsContent and those claims.
- The script lane's view reads the claims of everything it shows, nested
  fields included (scriptViewClaims), so a nested never withholds a uri
  from a link elsewhere in the view; an MCPL peer's top-level block the
  classifier rejects shows none of its fields, so they don't count.
  shownToScript and shownToScriptFromServer take those claims to
  referenceBlockForScript.
- An empty uri beside inline data is no uri on the script lane too, as
  at any depth: referenceBlockForScript withholds inline data beside a
  uri of any value but undefined, null or ''; beside a disposition it's
  still the classifier's contradiction. This loosens what 0.26.1 and
  0.26.2 withheld from a script for an MCPL peer.
- A link the script lane turned into its object is left as it is when
  the view is given it again: governedByRfc005 governs a link that names
  a uri, of any value, and the classifier reads a link's size from
  sizeBytes too, the object's name for it.
- anima-research#262's fragment was released, so what anima-research#527 added to it (a link's uri
  where its line shows it, unfetchable) is in anima-research#527's own changed
  fragment, with the two script-lane changes above.

Tests: main's script-lane tests read a link as a reference object (the
treated blocks' link, a module's link, which anima-research#755 pinned as its fields),
a module-made reference's unfetchable, an expired reference a module's
demand fetch passes over, and inline data beside an empty uri reaching a
script from a peer as from a module, beside a disposition withheld at
the top and nested. anima-research#527's script tests call scriptResultAnswer, main's
exit, and the cap test stubs the module registry main's spill notice
reads. New: a nested never in a script's view, a link naming no uri, and
a link object given to the script lane again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants