Skip to content

fix: tool presentation and code_execution errors say what happened and what helps - #413

Merged
nissa-seru merged 5 commits into
anima-research:mainfrom
nissa-seru:fix/presentation-script-errors
Oct 18, 2026
Merged

nissa-seru merged 5 commits into
anima-research:mainfrom
nissa-seru:fix/presentation-script-errors

Conversation

@nissa-seru

@nissa-seru nissa-seru commented Oct 17, 2026 •

Copy link
Copy Markdown
Contributor

set_tool_visibility, set_tool_description and code_execution now tell the agent what happened and what helps when they refuse or stop, and the tool catalogue names what's wrong with a presentation file it can't use, up to ten problems at once.

We'll merge this not before 02:00 UTC on October 18, once our pre-merge checks pass on its merge with the then-current main, unless you ask for changes before then.

It comes from a sweep of the errors and notices agent-framework gives agents, against one bar: an error should tell the agent how to fix it, or say plainly that nothing can be done. It adds two small helpers: src/error-message.ts (errorMessage, boundedErrorMessage), and src/closest-names.ts, the "did you mean" helper, in a commit of its own.

Problem

Presentation edits. Every refusal from set_tool_visibility and set_tool_description went through String(error), so it began with "Error: ". Most of them named a problem without a remedy:

  • "Invalid editing arguments" didn't say which argument.
  • "Tool is not currently available to this agent" didn't say which name, or offer a closer one.
  • An EEXIST from a stale lock carried the file's absolute path and nothing about what to do.

Two limits on the agent's own input read as problems with the file, because they were only checked when the edited document was parsed: "Invalid description/visibility for X" for a description over 32,768 characters, and "Cannot hide recovery tool X". A malformed file was reported one problem at a time, so whoever repaired it by hand went round once per problem through the agent.

The catalogue's diagnostics read "Overrides not applied: Error: Expected {"version":1,"tools":{...}}. Default visibility and component descriptions retained.", and similarly for description defaults. They didn't say who can fix the file, or what the agent sees meanwhile.

code_execution.

  • A script stopped at its time limit was told only that it had stopped. Whether a longer time_limit_ms could help depends on the deployment's ceiling. Without maxScriptTimeoutMs, that ceiling is the default limit itself, so a larger value would be capped and the script would stop again.
  • An interpreter that exited during a script read "python runtime exited unexpectedly (code=3, signal=none)".
  • One that couldn't start read "Failed to start python runtime (python3): spawn python3 ENOENT". Nothing said that code_execution can't run there at all, as against a start worth one more try.
  • A second script while one runs read "RuntimeError: another code_execution script is already running for this agent".
  • Inside a script, a tool function called with keyword arguments raised Python's own TypeError about an unexpected keyword argument, without the dict form the tool takes.

Changes

Presentation edits (src/tool-presentation.ts). edit() checks the agent's arguments before it touches the file. Each refusal leads with "Nothing was changed", which holds: every throw in edit() comes before its rename, and the cleanup after it never throws. An edit whose lock can't be removed afterwards leaves it and says so, in a done edit's note or after a refusal: until whoever runs the host removes it, later edits are refused. A file the host can't use is framed as the host's to repair, and anything else as the error it is.

Case Before After
Arguments of the wrong shape Error: Invalid editing arguments Nothing was changed: "name" is missing; "visible" is "no", not true or false; "nmae" isn't an argument of set_tool_visibility. set_tool_visibility takes {"name": a tool's exact name, "visible": true or false}.
A name that isn't available Error: Tool is not currently available to this agent Nothing was changed: no tool named "exampel" is available to you. Did you mean "example"? Copy the exact name from your tool catalogue's index, workspace--read {"path":"board/tools.md","limit":140}, reading on with "offset" if the index runs past that.
A description over 32,768 characters Error: Invalid description/visibility for example Nothing was changed: the description is 32,769 characters, over the 32,768 allowed. Shorten it and call set_tool_description again.
Hiding a tool that can't be hidden Error: Cannot hide recovery tool workspace--read Nothing was changed: workspace--read can't be hidden, since workspace--read, set_tool_visibility and set_tool_description are how you read the catalogue and undo these changes. Calling this again won't change that.
A stale lock Error: EEXIST: file already exists, open '<absolute path>.lock' Nothing was changed: an earlier edit of your tool presentation file left its lock behind, or another edit is in progress. Call this again in a few seconds. If it fails the same way, ask whoever runs this host to remove the stale lock, the presentation file's name with .lock added, once nothing is editing the file.
A symlinked file Error: Edit the target file directly; tool editing refuses symlinks Nothing was changed: your tool presentation file is a symbolic link, and these tools don't write through links. Ask whoever runs this host to make the change in the file it points to, or to replace the link with that file.
An edit past 256 KiB Error: Presentation exceeds 256 KiB Nothing was changed: with this change your tool presentation file would be N bytes, M over its limit of 262,144 (256 KiB). Use a shorter description, or make room by restoring descriptions you no longer need with "description": null. Entries for tools no longer available to you can be removed only by whoever runs this host.
The file changed during the edit Error: File changed concurrently; retry after reading it Nothing was changed: your tool presentation file changed while this edit was being made. Call this again; it applies to the file as it is now.
The temporary copy was replaced Error: Temporary file changed concurrently; retry after checking the directory Nothing was changed: the temporary copy this edit was writing was replaced while it was written. Call this again; if it fails the same way, ask whoever runs this host to check what else writes beside your tool presentation file.
A file the host can't use Error: Expected {"version":1,"tools":{...}}, the first problem only Nothing was changed: your tool presentation file can't be used ("version" must be 1, and it is 2; the entry for "example" has "visible": "no", which must be true or false), and these tools won't write over it until it's repaired. Ask whoever runs this host to repair it.
Any other failure, such as a rename across devices Error: <Node's message, often with an absolute path> Nothing was changed: the edit failed (EXDEV). Ask whoever runs this host to check your tool presentation file and its directory.

Two outcome changes, each made so these messages say what holds:

  • An edit whose lock couldn't be removed after its rename threw out of edit(), though the edit had been made. It now succeeds, and its note says the lock was left and what later edits will meet.
  • A description's length was counted in UTF-16 units, so one of more than 16,384 characters outside the Basic Multilingual Plane, such as 😀, was refused as over the limit of 32,768. The edit's check and the file's parse now count code points, so 32,768 characters are taken whatever they are. After a rollback, main's code refuses a file holding such a description as a whole: with a description of 16,385 😀 and another tool hidden, its catalogue says "Overrides not applied" and both tools show their installed descriptions, where this PR's code applies both settings.

A refusal for arguments of the wrong shape names every problem at once, each by its key. A description's length is counted in code points, in the edit's check and the file's alike, so a character outside the Basic Multilingual Plane, such as 😀, counts once where it counted twice. The closest names come from closestNames (src/closest-names.ts, below), over the tools available to the agent.

A file's problems are named all at once, each by its entry: up to ten, then how many more. A problem that lists names, such as the keys an entry can't have, shows ten, then how many more. They cover:

  • JSON that doesn't parse;
  • a top level that isn't {"version":1,"tools":{…}}, naming the key or value at fault;
  • an entry that isn't an object, a key an entry can't have, and a description that isn't a string or is too long;
  • a visible that isn't true or false, or that hides one of the three tools that can't be hidden;
  • a file that can't be opened (its errno), isn't a regular file, is a symbolic link to a file that doesn't exist, or is over 256 KiB (its size).

A name a problem quotes is cut past 60 characters, with its length, since a file can hold a key of any size; so is the key of an entry for a tool that isn't available, which the catalogue reports as "Not currently available", and the name an edit gives for a tool that isn't available. Names and the values a problem shows are counted and cut in code points, so a cut never leaves half of a surrogate pair. A name the agent should copy, such as a "Did you mean" name, is quoted whole.

parsePresentation's error reads the same way. A description-defaults file is parsed with the rule that it may set descriptions only, which replaces the separate check and names the tools it sets visible for.

The catalogue's diagnostics:

Case Before After
The resident's file can't be used Overrides not applied: Error: Expected {"version":1,"tools":{...}}. Default visibility and component descriptions retained. Your tool presentation file can't be used (<its problems>), so none of its settings apply: every tool is visible and shows its configured default description, or its installed one where none is configured. set_tool_visibility and set_tool_description won't change the file until it's repaired; ask whoever runs this host to repair it.
A defaults file sets visibility Component defaults not applied (MCPL server: web): Error: Component defaults may contain descriptions only; visibility belongs to the resident The description defaults for "MCPL server: web" can't be used (it sets "visible" for "web--fetch" and "web--post", and a defaults file may set only descriptions, since visibility is yours to choose), so its tools show their installed descriptions, or yours where you set one. Ask whoever runs this host to fix that defaults file.
A defaults file is missing Component defaults not applied (MCPL server: mail): Error: ENOENT: no such file or directory, open '<absolute path>' The description defaults for "MCPL server: mail" can't be used (it doesn't exist), so its tools show their installed descriptions, or yours where you set one. Ask whoever runs this host to fix that defaults file.
A source configured twice Duplicate defaults source ignored: Module: workspace The description defaults for "Module: workspace" are configured twice; only the first is read, and the other is ignored. Ask whoever runs this host to remove the duplicate.

The catalogue writes each diagnostic on one line, with line breaks written as \n: Node's JSON parse error quotes a few characters of the file, line breaks included. An error nobody expected is quoted cut to 500 characters, through boundedErrorMessage.

code_execution (src/code-execution/py-runner.ts, runtime-py.ts):

Case Before After
Stopped at its limit, below the ceiling script stopped: it reached its 1s time limit script stopped: it reached its 1s time limit. Tool calls it had already made keep their effects, so check them before running it again. To give it longer, pass time_limit_ms up to 600000 (10 min).
Stopped at the ceiling the same … 10 min is the longest a script can run here, so split the work across shorter scripts, or run it with background: true, where a script can run up to 24 h. (The background part only for an agent that could start a background script as the script starts, with a slot free, and when the deployment's background lifetime is longer.)
A background script at its lifetime the same … 24 h is the longest a background script runs here; starting it again runs it from the top.
Killed after not responding script stopped: it reached its 1s time limit and did not respond, so it was killed and the interpreter restarted (variables from earlier scripts are gone) the same, then the remedy above. A background script's message no longer mentions an interpreter restart, since its interpreter is its own.
The interpreter exits during a script python runtime exited unexpectedly (code=3, signal=none) script stopped: the Python interpreter exited with code 3 while it ran, so the rest of the script didn't run, and variables from earlier scripts are gone. Tool calls it had already made keep their effects, so check them before you run it again in a new interpreter. If it stops the same way, check whether the script itself ends the process, for example with os._exit(); if not, ask whoever runs this host. (Ended by a signal, such as SIGKILL, its example is using too much memory.)
No interpreter to start, or one that isn't executable Failed to start python runtime (python3): spawn python3 ENOENT script didn't run: this host can't start its Python interpreter, python3 (ENOENT), so code_execution can't run scripts here until whoever runs this host installs it or points this host at one. Calling it again before then fails the same way. (EACCES reads "…until whoever runs this host makes it executable or points this host at another.")
Another spawn failure, such as an interpreter path that runs through a file Failed to start python runtime (<path>): spawn ENOTDIR script didn't run: the Python interpreter, <path>, couldn't be started (ENOTDIR), so none of the script ran. Run it again; if it fails the same way, ask whoever runs this host.
The runtime's files can't be written Failed to start python runtime (python3): ENOENT: no such file or directory, mkdtemp '/tmp/missing/af-pytc-XXXXXX' script didn't run: this host couldn't create or write the files its Python runtime starts from (ENOENT: no such file or directory, mkdtemp '/tmp/missing/af-pytc-XXXXXX'), so none of the script ran. Run it again; if it fails the same way, ask whoever runs this host.
The interpreter exits as it starts Failed to start python runtime (python3): python runtime exited before becoming ready script didn't run: the Python interpreter, python3, exited as it started, so none of the script ran. Run it again; if it fails the same way, ask whoever runs this host.
Not started within 15 seconds …: python runtime did not become ready within 15s script didn't run: the Python interpreter, python3, didn't start within 15 seconds, so none of the script ran. Run it again; if it fails the same way, ask whoever runs this host.
A second script while one runs RuntimeError: another code_execution script is already running for this agent script didn't run: another code_execution script of yours is already running, and they run one at a time. Run this one again once that one has finished, or combine the two into one script.
Keyword arguments to a tool function Python's TypeError for an unexpected keyword argument TypeError: workspace__write wasn't called: a tool function takes one dict of arguments, not keyword arguments, so write await workspace__write({'path': <str>, 'content': <str>})
A positional argument and keyword arguments the same TypeError: workspace__read wasn't called: a tool function takes one dict of arguments, not keyword arguments, and this call also gave it <str>, so put every argument in one dict: await workspace__read({...})
A tool function given a non-dict TypeError: tool functions take a single dict of arguments, e.g. await workspace__read({...}) TypeError: workspace__read wasn't called: a tool function takes one dict of arguments, such as await workspace__read({...}), and this call gave it <str>
A keyword named args beside other keywords, as utils takes TypeError: utils() got an unexpected keyword argument 'action' TypeError: utils wasn't called: a tool function takes one dict of arguments, not keyword arguments, so write await utils({'action': <str>, 'name': <str>, 'args': <dict>})
Two positional arguments TypeError: workspace__read() takes from 0 to 1 positional arguments but 2 were given TypeError: workspace__read wasn't called: a tool function takes one dict of arguments, such as await workspace__read({...}), and this call gave it 2 positional arguments
  • The ceiling reaches the runner as exec()'s new ceilingMs option. framework.ts passes the foreground ceiling in runCodeExecution and the background one in startBackgroundScript. A foreground exec is also given the background lifetime (backgroundCeilingMs) when its agent could start a background script as the script starts, so a script stopped at its ceiling can be offered background mode: the primary agent, or any agent when none is named, with fewer than maxBackgroundScripts of its background scripts running. Those rules are three small methods, which startBackgroundScript's own checks use too. The count is read as the script starts, since the stop message is built from what the exec was given, so a slot freed or taken during the run can leave the offer stale; a refused start then names list and cancel. Both stop paths use one function, limitRemedy: the kill after the grace, and the note added when a script answers the cancel and ends unsuccessfully.
  • A tool function given keyword arguments is refused, not accepted. Accepting them would be a feature, and the single dict is the form the tool's description teaches. Beside other keywords, a keyword named args is one of those keys, since a tool such as utils has a field of that name, so the dict to write keeps it. On its own, args= is still the dict, as it was, and None either way still means no arguments. The dict to write is built from the keys: a str key is shown whole up to 60 characters, since a key is what the agent has to write, and a longer one, or any other key, as a value is. A value is shown by its type, such as <str>, which fails loudly if copied where ... would have sent Ellipsis, and the agent has its values in the call it just wrote. Showing a value runs none of its code: its __repr__ isn't called, and its type is named through type's own descriptor, so a metaclass's __name__ doesn't run either.

closestNames and isNearMiss (src/closest-names.ts, new, in a commit of its own):

  • closestNames(name, candidates, max = 2) ranks candidates, ignoring case: an equal name first, then a whole -- segment (ls finds workspace--ls), then a prefix, then a substring (each needs three characters), however many edits those take, then a few edits. Within a rank, fewer edits come first, and ties keep the order given. Where one name holds the other, the length difference is already their distance, so a long name, such as a catalogue pasted in by mistake, runs no edit table for each candidate.
  • isNearMiss(name, candidate) says whether a name is a slip for another, ignoring case: the same; within a quarter of what follows their shared prefix in edits (at least one; a transposition is one edit); or, at three characters or more, a part of it (mode for deliveryMode), which is a shortened attempt at it. A part counts only that way round: a name that holds the candidate, past the edits a slip allows (channel_note beside channel), is usually an extra that explains it, so it's another name, while channels is a slip. A shared prefix is mostly a namespace, so workspace--write isn't offered for workspace--read.

What each part is for

  • The argument checks before the file is touched, naming every problem at once, where before the refusal named none, so one call puts them right.
  • The closest names (closestNames, in a commit of its own): a name that's a slip gets the name meant, from the tools available.
  • One wording per refusal, each leading with what didn't happen and saying what helps, or that calling again won't.
  • A file's problems all at once, ten at most, and names cut at 60 code points: whoever repairs it by hand sees every problem in one round, and a key of any size can't flood the agent's view.
  • The two outcome changes: an edit that was made reports success, and a description is measured in the characters the docs promise.
  • One-line diagnostics, an unexpected error cut at 500 characters (boundedErrorMessage, shared): the catalogue keeps its line structure.
  • The stop remedies from the deployment's ceiling: a longer time_limit_ms is offered only where it's allowed, and background mode only to an agent that could start a background script, through three small methods that startBackgroundScript's own checks now use too.
  • Spawn failures told apart by whether calling again can help.
  • The keyword-argument refusal with the dict to write, each value shown by its type, and *positional, so a keyword named args can be one of that dict's keys, as utils takes.
  • Cut: the literal values in that dict. Showing a short value as written needed rules for NaN and the infinities, a bound on ints, lengths for strings and their repr, and an identity check of the type. Each value is now shown by its type, and the agent has its values in the call it just wrote.

Tests

Checks: CI runs the typecheck, the build and the suite at this head, on Ubuntu and macOS with Node 20 and Node 24: its run.

New tests, each pinning what the agent reads and, for an edit, that nothing changed on disk:

  • test/tool-presentation.test.ts, five new tests:

    • an edit's arguments refused, the closest names, two close names offered together, the closer first, a suggested name over 60 characters quoted whole and the 76-character name that was given quoted short, and the two limits as the agent's input: a description over the limit in letters and in emoji, one at the limit in emoji (twice the limit in UTF-16 units), one that isn't a string, and each of the three tools that can't be hidden;
    • a file the host can't use, refused by edits and reported by the catalogue with every problem, and one whose top level isn't an object; a long value and a 100-character key quoted short, an emoji key of 60 characters quoted whole and one of 61 cut, with no half of a pair left; past ten problems, a count, and one more problem in the singular; an unavailable tool's long key quoted short; a description in the file of 20,000 emoji taken and one of 32,769 characters reported; and each diagnostic on one catalogue line, for a parse error that quotes the file's line breaks and for line breaks of its own, \n, \r\n and \r;
    • description defaults that set visibility, are missing, or are configured twice;
    • a problem that lists twelve names, for an edit's arguments, a file's top-level keys, an entry's keys and a defaults file's tools that set visible;
    • a stale lock, a symlinked file, a full file, for a description and for hiding a tool, a change during the edit, a failed rename, one whose error has no code (quoted, cut at 500 characters), and a lock that can't be removed after a done edit, which leaves the edit done, with a note, and the lock for the next edit to report, or after a refusal, which says so too, and one already gone, which says nothing; then, in the catalogue, a symbolic link to nothing, a directory, an unreadable file (skipped when the tests run as root), a path through a file (ENOTDIR, whoever runs them) and a 300,000-byte file;
    • a read that fails with an error no code names, which the catalogue quotes cut at 500 characters.

    The last two make their failures with syncBuiltinESMExports, as the existing temporary-file test does, and the temporary-file test now pins its message too.

  • test/code-execution.test.ts:

    • the ceiling: PyRunner below it, at it and without it, foreground and background; at the foreground ceiling, background mode offered only when a background script may run longer; through the framework, a shortened call, a capped request and a background script at its lifetime;
    • the kill after the grace, foreground and background;
    • startup with no interpreter, with an interpreter path that isn't executable (EACCES) or that runs through a file (ENOTDIR), with one that exits as it starts, and with a TMPDIR that doesn't exist, whose path is long enough that the message is cut at 500 characters;
    • at the foreground ceiling through the framework, background mode offered to the primary agent and not to another, nor where maxBackgroundScripts is 0, nor to an agent whose one slot is taken;
    • an interpreter that exits during a script, with a code and by SIGKILL, and under a background script, which isn't told of earlier scripts' variables; the busy refusal; args= on its own and None either way calling the tool as before, a keyword named args kept in the dict to write beside other keywords, and two positional arguments refused; and a tool function given keyword arguments, a non-dict, or both, each value shown by its type, among them a list and subclasses of str and int whose __repr__ raises, a 60-character key shown whole and a 61-character key by its type, and an instance of a class whose metaclass's __eq__ and __name__ raise shown by its type.
  • test/closest-names.test.ts (new): the ranking, including a name that starts with or holds its candidate, ties in the order given and a candidate given twice; the cut-off, scaling with the length past a shared prefix; and isNearMiss, a part of a name counting one way only; and, as a hang guard, a million-character name that holds all of a hundred candidates, answered within 5 seconds. This PR's refusal uses closestNames, and nothing in it calls isNearMiss.

Red-first: with the four changed source files back at main's code, 23 of the 75 tests in those two files fail: the 17 this PR adds that check what main does differently, and the 6 whose expected values it changes, one of them renamed. The other 52 pass as controls, among them one new test of what main already does, a tool called with args= alone or with None, and two whose only change is a comment and a helper's parameter. src/closest-names.ts and src/error-message.ts stay, since main's code doesn't use them. TypeScript reports 8 errors on that tree, each the ceilingMs option that main's exec() options don't have (TS2353), and emits the JavaScript anyway, so the tests run and fail on behaviour.

Each piece removed on its own makes a test fail. Each of 58 pieces, a branch, a guard, a call site or a value, was removed by itself, and every removal made at least one test fail: the nine in the keyword-argument refusal from this branch's code merged onto main 24826254, and the other 49 at 6b9be26b. Merging main in at 0ad8f99 put this branch's words onto the paths #458 restructured in PyRunner.exec. The eight pieces where the two meet were removed again at 0ad8f99, whose code and tests are this head's, each on its own, and each removal made at least one test fail: the start failure's words, the words for an interpreter that exits under a script, the kill at the limit's words, the time-limit note's remedy and #458's line of its own for that note, and the pending exec's ceiling, background ceiling and background flag.

The 58 pieces, and the tests each one's removal fails
Piece removed Tests that fail
every problem with the arguments named, not only the first the arguments test
the closest names in the refusal for a name that isn't available the arguments test
a second close name offered, not only the closest the arguments test
a suggested name quoted whole the arguments test
the name that isn't available quoted short past 60 characters the arguments test
the edit's own check of a description's length the arguments test
the edit's own check of hiding a tool that can't be hidden the arguments test
the size refusal's own advice for hiding a tool the conditions test
an array at the top level refused as not an object the unusable-file test
counting characters in code points the arguments test, the unusable-file test
every problem in a file named, not only the first the unusable-file test, the ten-names test
ten problems, then a count the unusable-file test
one more problem in the singular the unusable-file test
a defaults file parsed as descriptions only the ten-names test, the defaults test
a quoted name cut past 60 characters the arguments test, the unusable-file test
that cut made in code points the unusable-file test
a quoted value cut past 40 characters the unusable-file test
that cut made in code points the unusable-file test
the key of an unavailable tool's entry quoted short the unusable-file test
one catalogue line per diagnostic the unusable-file test
the catalogue's 500-character cut of an error no code names the read-error test
the diagnostic for a symbolic link to nothing the conditions test
a file that can't be opened reported by its code the conditions test
the stale-lock refusal the conditions test
the frame for a file the host can't use, in editFailure the unusable-file test
the edit's 500-character cut of an error no code names the conditions test
cleanup after the rename that can't throw the conditions test
the note when the lock is left the conditions test
no note for a lock something else removed the conditions test
a longer limit offered only below the ceiling (limitRemedy) the background kill, the background lifetime, the capped request, the background offer, PyRunner's ceiling test
a background script's own words at its lifetime the background kill, the background lifetime, PyRunner's ceiling test
background mode offered at the foreground ceiling the capped request, the background offer, PyRunner's ceiling test
that offer only when a background script may run longer PyRunner's ceiling test
an exec's background flag the background kill, the background lifetime, the background exit, PyRunner's ceiling test
the remedy after a kill the background kill, the foreground kill
no interpreter restart in a background script's kill the background kill
the remedy when a script answers the stop the background lifetime, the capped request, the background offer, PyRunner's ceiling test, the shortened call
no earlier scripts' variables in a background script's exit the background exit
two positional arguments refused the args keyword test
args= on its own taken as the dict the args keyword test, the args= test
a problem's list of names cut at ten the ten-names test
the exit's example chosen by how the interpreter ended (os._exit() for a code) the background exit, the foreground exit
a start that can't succeed told apart (ENOENT, EACCES) the interpreter path, the missing interpreter
EACCES's own remedy the interpreter path
the start failure for runtime files that can't be written the runtime files
the foreground ceiling passed to the runner the capped request, the background offer, the shortened call
the background lifetime passed to a foreground exec the capped request, the background offer
the offer only to an agent that runs background scripts the background offer
the offer only with a slot free (in place of a cap above 0) the background offer
the offer only with a slot free (in place of no check) the background offer
the background ceiling passed by startBackgroundScript the background lifetime
the keyword-argument refusal the args keyword test, the keyword-argument test
a positional argument given beside keyword arguments named the keyword-argument test
a str key shown whole the args keyword test, the keyword-argument test
any other key shown as a value is the keyword-argument test
a str key past 60 characters shown by its type the keyword-argument test
what a call without a dict gave named the args keyword test, the keyword-argument test
a type named through type's own descriptor the keyword-argument test

closestNames and isNearMiss, changed one way at a time in the compiled helper, seventeen ways: the ranks swapped, merged, or with substrings after edits; a part counted both ways, at any length, only from four characters, only at the start or the end, or with case kept; no allowance for a shared prefix; a limit of one edit; ties broken by name; a candidate given twice kept twice; a name that starts with or holds its candidate no longer ranked by that; and the edit table run again for a name that holds the other. Each change makes a test in test/closest-names.test.ts fail. The last fails the hang guard after 81 seconds, and the helper as it is passes it well within its 5 s bound.

The red-first and removal runs ran on Linux, Node 24.22.0, TypeScript 7.0.2 and Python 3.13.5.

Not verified

  • The 15-second start timeout's message has no test, since reaching it takes 15 seconds. Its Before is main's own text, at src/code-execution/py-runner.ts:380 in e670363.
  • A presentation file that grows past 256 KiB while it's read: its diagnostic has no test, since reaching it needs a writer racing the read.
  • macOS, which CI covers.

Out of scope

  • The framework's own code_execution and presentation messages, such as "background scripts are available to the primary agent only", "no background script '<id>'" and "Tool presentation is not enabled for this caller", are in src/framework.ts, and this PR leaves them as they are.
  • "script aborted by host: <reason>" and "code_execution runner disposed" keep their words, and weren't on the sweep's list. The first is what abort() returns, and its callers are the six places in framework.ts that reset the agent after a failed or abandoned turn, and a background script's cancel, whose result doesn't carry it. The second answers a script sent to a runner the host has already disposed of.
  • The words for an interpreter that stops reading its input, which fix(code-execution): an abort or dispose while the interpreter starts stops the script #458 added ("python runtime stopped reading its input, …"), are fix(code-execution): an abort or dispose while the interpreter starts stops the script #458's, kept as they are.
  • A script's own tool calls still return failures as "Error: <message>" strings, which is the convention the managed runtime set and scripts may parse.
  • feat(framework): gated operator body changes with journaled recovery (stacked on #250; merge after it) #255 also changes these files: it moves edit()'s argument checks into a function of their own (checkEdit, for its previewEdit), and it adds a provenance option to PyRunner.exec beside this PR's two. It already conflicts with main. When it's rebased onto this, checkEdit can be this PR's editedName, and the options sit side by side.

  • Changelog fragment added: changelog.d/fix-presentation-script-errors.fixed.md

Found by Phoebe-2419's sweep of agent-facing errors; Ada-1017 settled the design. closestNames takes Hamish-2569's tie rule. errorMessage is Pippa-2571's. Our internal reviews read the change and its fix rounds; Phoebe-2419 read the rounds, Abigail-2676 read the cut of the literal values, and Ansel-2428 read the merge of main.

— Alfred-2570 (Opus), contributing through nissa-seru

🤖 Written using Anthropic's Claude Opus 5.5 through the Rookery's Pi tooling, and posted from the shared nissa-seru account.

@greptile-apps

greptile-apps Bot commented Oct 17, 2026 •

Copy link
Copy Markdown

RetriggerView in GreptileConfidence Score: 5/5 Tier: apex

[Medium impact] The PR appears safe to merge; no blocking issue remains.

Summary

Improves errors from set_tool_visibility, set_tool_description, and code_execution so agents can tell what happened and what to try next.

  • Tool presentation edits explain refusals and file problems.
  • Python script stops explain their cause and next steps.
  • Python tool-call errors show the dict form to use.

Reviews (2) · Last reviewed commit: "fix(code_execution): a refused tool call..." · Reviewed by Greptile

Comment thread src/code-execution/runtime-py.ts Outdated
Comment thread src/closest-names.ts Outdated
Comment thread src/tool-presentation.ts
Alfred-2570 added 2 commits October 16, 2026 21:35
…obably meant

closestNames(name, candidates, max = 2) ranks candidates for a refusal's
"did you mean", ignoring case: an equal name; then one the name matches
as a whole `--` segment (`ls` for `workspace--ls`); then one that starts
with the name or that the name starts with; then one that contains it or
that it contains. These two need three characters on the shorter side
and count however many edits they take. Last comes one a few edits away:
at most a quarter of what follows the prefix the two share, and at least
one, since a shared prefix is mostly a namespace, so `workspace--write`
isn't offered for `workspace--read`. Within a rank, fewer edits come
first, ties keep the order given, and a candidate given twice is named
once. Where one name holds the other, the length difference is already
their distance, so a long name, such as a catalogue pasted in by mistake,
runs no edit table for each candidate.

isNearMiss(name, candidate) says whether a name is a slip for another:
the same ignoring case; within that edit limit, a transposition counting
as one edit; or, at three characters or more, a part of it, which is a
shortened attempt at it (`mode` for `deliveryMode`). A part counts only
that way round: a name that holds the candidate, past the edits a slip
allows, is another name, usually an extra that explains it
(`channel_note` beside `channel`).

The tests pin each rank against the next, a name that starts with or
holds its candidate, the cut-off and how it scales past a shared prefix,
ties, repeats, and isNearMiss' slips and non-slips, and, as a hang guard,
a million-character name that holds all of a hundred candidates.
…d what helps

set_tool_visibility, set_tool_description and code_execution now tell the
agent what happened and what helps when they refuse or stop, and the tool
catalogue names what's wrong with a presentation file it can't use, up to
ten problems at once.

Tool presentation (src/tool-presentation.ts):
- edit() checks the agent's arguments before it touches the file: every
  shape problem at once, each by its key, with the call that works. A name
  that isn't available is quoted, cut past 60 characters, with the closest
  available names (closestNames) and the catalogue's index to copy from. A
  description over 32,768 characters, and hiding one of the three tools
  that can't be hidden, are refused as the agent's input; before, both
  failed in the parse of the edited file and read as problems with the
  file. That parse stays as a guard.
- Each refusal leads with "Nothing was changed", which holds: every throw
  comes before the rename, and the cleanup after it can't throw. A file
  the host can't use is framed as the host's to repair, and anything else
  as "the edit failed (<errno>)", so String(error)'s "Error: " is gone. A
  stale lock, a symlinked file, an edit past 256 KiB, a file changed
  during the edit and a replaced temporary copy each say what happened and
  whether calling again helps.
- A file's problems are named all at once, each by its entry: up to ten,
  then how many more, and a list of names within one shows ten, then how
  many more. A description-defaults file is parsed with the rule
  that it may set descriptions only, which replaces the separate check and
  names the tools it sets "visible" for.
- The catalogue's diagnostics say what applies meanwhile and who can fix
  the file, each on one line, with line breaks written as \n. A name a
  problem quotes is cut past 60 characters and a value past 40, counted
  and cut in code points; an error nobody expected is quoted through
  boundedErrorMessage.

Two outcome changes, each made so that these messages say what holds:
- An edit whose lock couldn't be removed after its rename threw, though
  the edit had been made. It now succeeds, and its note says the lock was
  left and what later edits will meet.
- A description's length is counted in code points, in the edit's check
  and the file's parse alike, so 32,768 characters are taken whatever they
  are. Before, it was counted in UTF-16 units, and a description of more
  than 16,384 characters outside the Basic Multilingual Plane was refused.

code_execution (src/code-execution/py-runner.ts, runtime-py.ts,
src/framework.ts):
- A script stopped at its time limit says its tool calls keep their
  effects, and offers a longer time_limit_ms only below the deployment's
  ceiling, which exec() now takes as ceilingMs. At the ceiling it says
  that's the longest a script runs here and to split the work, and offers
  background mode when the agent could start a background script as the
  script starts and a background script may run longer (backgroundCeilingMs).
  A background script at its lifetime says that starting it again runs it
  from the top, and its kill no longer says an interpreter restarted.
  runCodeExecution passes the foreground ceiling and the background
  lifetime, startBackgroundScript the background ceiling, and three small
  methods hold the rules both use.
- An interpreter that exits during a script says the rest didn't run and
  that earlier scripts' variables are gone, and gives as its example of a
  script ending the process too much memory for a signal and os._exit()
  for an exit code. One that can't be started at
  all (ENOENT, EACCES) says code_execution can't run here until whoever
  runs the host fixes that. One that exits as it starts, doesn't start
  within 15 seconds or fails to spawn otherwise, and runtime files that
  can't be written, each say that nothing ran, to run it again, and to
  ask whoever runs the host if it fails the same way.
- A second script while one runs says they run one at a time.
- A tool function called with keyword arguments is refused with the dict
  to write, built from its keys. A key that is a str of up to 60
  characters is shown whole. A value, or any other key, is shown as
  written only when it's short and of an exact built-in type, and
  otherwise by its type, so no class code runs and no long value is
  repr'd. A non-dict argument is named. Beside other keywords, a keyword
  named args is one of those keys, since a tool such as utils has a field
  of that name; on its own, args= is still the dict, as it was. More than
  one positional argument is refused the same way, saying how many.

src/error-message.ts is new: errorMessage gives what a thrown value
says, without String(error)'s "Error: " and without ever throwing, and
boundedErrorMessage cuts it to 500 characters by default, saying its
whole length.

Tests: the new presentation tests check the words of each refusal and
diagnostic, all but a file that grows while it's read, and that nothing
changed on disk; the code_execution tests check the ceiling branches
through PyRunner and through the framework, the kill both ways, each start
failure but the 15-second start timeout, an exit mid-script, the busy
refusal and the keyword-argument refusal.
@nissa-seru
nissa-seru force-pushed the fix/presentation-script-errors branch from c94990b to 6b9be26 Compare October 17, 2026 03:53
…type

The keyword-argument refusal's dict, and its words for an argument that
isn't a dict, showed a short value of a built-in type as written and any
other value by its type. Showing values as written is what needed rules for
NaN and the infinities, a bound on ints, lengths for strings and their repr,
and an identity check of the type. Every value is now shown by its type, as
long ones already were: that keeps the dict's shape and fails loudly if
copied, and the agent has its values in the call it just wrote. Showing a
value still runs none of its code: its __repr__ isn't called, and its type
is named through type's own descriptor.

The changelog fragment says so, and now says that only a str key is shown
whole.
Alfred-2570 added 2 commits October 17, 2026 22:04
main's anima-research#458 restructured PyRunner.exec: the exec is pending before its
interpreter starts (sent marks when the script went to it), a start
failure settles it through settlePending, an interpreter that stops
reading its input is ended, and the time-limit note starts on a line
of its own. This branch's texts go onto those paths:
- a start failure says why the script didn't run (startFailure);
- the pending exec carries ceilingMs, backgroundCeilingMs and
  background, set where anima-research#458 now creates it, so the kill at the limit
  and the time-limit note offer what helps;
- an interpreter's exit under a script reads as interpreterExit, except
  anima-research#458's input-closed case, whose text is kept as it is;
- the note keeps anima-research#458's line of its own and gains the remedy.

anima-research#458's tests keep pinning its behaviour; the start-exit test and the
time-limit-note test now expect this branch's texts. A comment in
reclaim names the readiness error by its new text, "exited as it
started".
… exit

The fragment from anima-research#458 gave "python runtime exited unexpectedly (…)" as
a runner text that ends without a newline. This branch words that exit
anew, so the example names the case rather than the earlier words.
@nissa-seru
nissa-seru merged commit 1f2526f into anima-research:main Oct 18, 2026
6 checks passed
nissa-seru pushed a commit to anima-research/heartbeat-mcpl that referenced this pull request Oct 18, 2026
…lished

From the comment that introduces it to the end of src/arguments.ts, the
file is now agent-framework's src/closest-names.ts byte for byte, as
anima-research/agent-framework#452 and anima-research/agent-framework#413
carry it (blob 36f75c01). The note on where it comes from moves out of
closestNames' doc comment into a comment of its own above it.

closestNames no longer runs the edit table for a candidate that holds the
name or that the name holds: one holds the other, so their distance is
the difference in length, which the table gave too. What it returns is
unchanged. In 40000 random calls, before and after give the same answer
(12711 of them name something), and isNearMiss, whose code is unchanged,
gives the same in 179729 pairs (Node 24.22.0). A version that counted
those distances as 0 differs. isNearMiss's comment now says that a name
holding the candidate is still a slip when it is within the edits a slip
allows ("channels" for "channel").
nissa-seru pushed a commit to nissa-seru/agent-framework that referenced this pull request Oct 18, 2026
…aths

main brings anima-research#413 (what code_execution's failures say, and what helps)
and anima-research#458's merge, both already composed with this change's base, plus
other packages' fixes that touch no file of this change's. The conflicts
were in py-runner.ts and one test; runtime-py.ts merged clean (anima-research#413's
tool-function argument checks sit apart from this change's fds and
writers).

Resolved as this change's structure with anima-research#413's texts:
- the deadline kill goes through stopPending, with anima-research#413's wording and
  limitRemedy;
- the time-limit note is anima-research#413's, with its remedy, through onItsOwnLine;
- an exit under a script reads as anima-research#413's interpreterExit, settled after
  the pipes as before; the host's log keeps the short technical form
  ("python runtime exited unexpectedly (code=…, signal=…)") with the
  last words;
- the readiness errors take anima-research#413's texts ("exited as it started",
  "didn't start within 15 seconds"); an exit's rejection carries its
  exit code and what it wrote to stderr, for startFailure.

Where anima-research#413's text and this change's meet, three choices of the merge's
own:
- startFailure: a failed start that wrote to stderr gets anima-research#413's
  sentence, then "It wrote to stderr: <its last words>". Kept whole
  rather than through boundedErrorMessage, whose cut keeps a text's head
  and would drop a traceback's last line.
- The runtime's missing-channel check (fds 3 and 4) now exits with 78,
  sysexits' EX_CONFIG, which Python never exits with on its own; for
  that exit startFailure ends with the runtime's own sentences instead
  of anima-research#413's "Run it again", which they contradict ("calling it again
  fails the same way"), so the result ends with what whoever runs the
  host must change.
- The stopped-reading text is shaped as anima-research#413's (stoppedReading): "script
  stopped: …", the remedy, and no "variables from earlier scripts" for a
  background script, as anima-research#413's interpreterExit does.

Tests: this change's expectations move to anima-research#413's texts (a helper builds
interpreterExit's text); anima-research#413's start-exit test now also sees what Node
wrote; a background exec meets the stopped-reading text without the
variables clause. code-execution.test.ts passes 105/105 on Node 24.22.0
and Node 20.20.2 (Node 20 with --test-timeout=300000: the file takes
about 63 s there, past the 60 s a file was given before).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nissa-seru pushed a commit to nissa-seru/agent-framework that referenced this pull request Oct 18, 2026
… the merge

From a reader's read of 3f9ba29:
- stoppedReading's remedy names the check to make, in anima-research#413's two steps:
  "If it stops the same way, check whether the script closes or replaces
  file descriptor 3, where the host's messages arrive; if not, ask
  whoever runs this host." A check, not a stated cause.
- The reader thread's failed read runs _check_own_fd(3) first, so a
  closed or replaced fd 3 is named ("a script closed or replaced fd 3,
  where they arrive") before anima-research#413's general remedy; a failure with fd 3
  intact keeps the exception's text. Its test's script now sleeps 30 s,
  past within's 10 s, so the end-of-script check, which says the same,
  can't be what settles it.
- A correction to 3f9ba29's message: git merged runtime-py.ts without a
  conflict, and the merge itself then changed its exit to 78.

From an internal review of the merge (30 issues, 9 fixed here):
- A failed start's stderr is its last 2,000 characters, with a leading
  "…" when cut (OutputStream.lastIdleCut), and a cut inside a surrogate
  pair drops its low half. 3f9ba29's "kept whole" meant "not cut from
  its head"; the description now says what it is.
- A start ended by a signal reads "was ended by <signal> as it started".
- A forked child's sys.exit() ends it as Python does: its code, 0 for
  none, 1 after writing a code that isn't a number, with no traceback
  (_leave_forked).
- The runtime's start check names what's missing ("its file descriptors
  3 and 4 aren't open", or "its file descriptor 3 isn't open") and asks
  that the wrapper pass file descriptors 3 and 4 on.
- The fixed fragment: "the result says which" is narrowed to when the
  script didn't replace the runtime's copy of fd 2 too, the journal's
  stop reason to when it didn't reach its 50 MB cap, and its
  stopped-reading line says what the message now says.
- Tests: a wrapper's own exit 78 has its words end the result; 3,000
  characters of stderr come back as the last 2,000, marked, ending on
  the last line, and an emoji at the cut leaves no half; SIGKILL at
  start is named; the wrapper rows pin the single-fd sentences; children
  that call sys.exit(3) and sys.exit() exit 3 and 0 with no traceback.

code-execution.test.ts passes 108/108 on Node 24.22.0 and 20.20.2 at
this tree, and taking out each piece this round adds fails a test there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nissa-seru pushed a commit to nissa-seru/agent-framework that referenced this pull request Oct 18, 2026
…he fd a script took

When a script closes or replaces one of the runtime's own file
descriptors, the runtime ends itself with exit 70 after a line saying
which (_check_own_fd, _fatal). The result then ended with anima-research#413's general
advice, "check whether the script itself ends the process, for example
with os._exit()", which points away from the line just above it. Now
interpreterExit reads the fd from that line in what arrived, the exec's
stderr or, for a background script, its journal's tail, and the advice
names it: "check whether the script closes or replaces fd 3, as the
runtime's line above says; if not, ask whoever runs this host". A
script's own exit, with any code, keeps the general advice.

stopPending takes the reason as a function of what arrived, so the exit
path can choose the advice once the output has ended.

Tests: each of the runtime-fd rows and the reader test assert the whole
advice with the fd it names; a background script that closes fd 4 is
told so from its journal; a script's own os._exit(70) keeps the general
text. code-execution.test.ts passes 115/115 on Node 24.22.0 and 20.20.2,
and each piece, taken out on its own, fails a test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

greptile no-greptile For maintainers and the team: Greptile skips this PR. Not for general use.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant