Skip to content

fix(tools): a refusal is an error result - #367

Merged
nissa-seru merged 1 commit into
anima-research:mainfrom
nissa-seru:fix/refusals-are-error-results
Oct 17, 2026
Merged

nissa-seru merged 1 commit into
anima-research:mainfrom
nissa-seru:fix/refusals-are-error-results

Conversation

@nissa-seru

@nissa-seru nissa-seru commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

TL;DR: a refusal now reaches the model as an error result. tune_out's refusals, channel_open on a tuned-out channel, all but one of the subconscious's refusals and the api module's Unknown tool came back unflagged, so the model read each one as a tool that ran and answered. channel_open's refusal also showed the model only {"refusal":"tuned-out","channelId":"…"}, never its sentence saying how to cancel the tune-out. This PR changes the flags only; follow-up PRs reword the messages, and some of them depend on this one.

Problem

A tool result that isn't an error reaches the model as the tool's ordinary answer: in native tool use, a tool_result without is_error; in XML tool mode, <result><stdout>…</stdout></result> rather than <error>; in a code_execution script, the bare text rather than Error: …. These refusals were returned that way, with isError: false or, for the api module, no isError at all:

  • tune_out (src/framework.ts, dispatchTuneOutToolCall): an unknown channel; a cancel the coordinator refuses (channel <id> is not tuned out); and an enter it refuses: a channel already tuned out, a channel bound to a conversation fork, or no subconscious.
  • channel_open on a tuned-out channel (src/mcpl/channel-registry.ts, handleToolOpen). The refusal also carries data: { refusal: 'tuned-out', channelId }, and a result that isn't an error is rendered from its data (nonErrorResultText). So the model read {"refusal":"tuned-out","channelId":"<id>"} and never the refusal's text, the only place that says to call tune_out with mode cancel.
  • The subconscious's tools (src/tune-out/coordinator.ts, handleSubconsciousTool): deliver_summary with no text, cancel_tuneout of a channel that isn't tuned out, note_disposition with no key, and speak_in_channel with channel speech off or with no text. (deliver_summary's subconscious not configured was already flagged.)
  • The api module (src/modules/api/index.ts), which answers any api--<name> call with Unknown tool and no isError.

Changes

Each of those results now has isError: true. What the model receives, before and after:

Call Before (not an error) After (an error)
tune_out on an unknown channel unknown channel: <id> the same text
tune_out mode cancel on a channel that isn't tuned out channel <id> is not tuned out the same text
tune_out on a channel already tuned out channel <id> is already tuned out (cancel first to change params) the same text
channel_open on a tuned-out channel {"refusal":"tuned-out","channelId":"<id>"} Channel <id> is tuned out. Cancel the tune-out (tune_out with mode: 'cancel') to resume normal attention — cancelling delivers the diverted backlog.
deliver_summary with no text empty summary the same text
cancel_tuneout of a channel that isn't tuned out channel <id> is not tuned out the same text
note_disposition with no key key required the same text
speak_in_channel with channel speech off channel speech is disabled for the subconscious (recipe: subconscious.allowChannelSpeech) the same text
speak_in_channel with no text empty message the same text
api--<name> Unknown tool the same text
  • The wording is unchanged. Follow-up PRs reword each message to say what happened and what to do instead.
  • A script's call to one of these tools now gets Error: <text>, as for any failed call.
  • The tuned-out refusal keeps its data. An in-process caller (ModuleContext.callTool) gets the same result as before with isError set, and can still read refusal.
  • Two of the refusals changed here can't be reached today. They change with the rest, so the rule holds at every site. tune_out's "requires the subconscious resident" can't be reached, since the coordinator, and with it the tune_out tool, exists only when a subconscious is configured. Neither can cancel_tuneout's second refusal, a cancel that fails after its lookup found the channel, since the lookup and cancelTuneOut read the same state synchronously.
  • Left as it is: channel_close's refusal of a machine close of an explicitly opened channel (handleToolClose) stays isError: false. It answers module-origin closes only, so its result goes back to the module that called, never straight to a model, and a module may branch on it. connectome-host's two callers read success, error and data.refusal, not the flag: its subscription GC stands down on data.refusal, and its channel-mode module puts error into its own result, which it flags itself.

What each part is for

This change is for the model, whose tooling reads isError to tell a failed call from one that worked: a refusal should be an error result. Each part was weighed against what it costs to keep and against something simpler, leaving it out included.

  • isError: true where each refusal is returned, in four files. Two of the flags are at sites that each return several. The wording waits for the follow-up pull requests, so this change is the flag alone.
  • The tuned-out refusal keeps its data. With the flag the model reads its error sentence, where it saw only data's JSON before. Dropping data would be simpler, but an in-process caller (ModuleContext.callTool) can read refusal there.
  • The two refusals no caller can reach today change too, a flag each, so the rule holds at every site if one becomes reachable.
  • channel_close's refusal of a machine close stays as it is: it answers modules only, which may branch on it, and a model never sees it.
  • test/refusals-are-error-results.test.ts (190 lines) calls each tool as a model, a script or an in-process module does, and reads the result's flag and text.

Cut: nothing. Each part is the flag where a refusal is returned.

Tests

test/refusals-are-error-results.test.ts (new, 4 tests) runs a resident and its subconscious against the tune-out MCPL fixture, with the api module loaded.

  • What the model is sent. In three tests the model makes, as tool calls, a refusal through each reachable flag, and each test checks the result sent back to it: an error, with the refusal's text. The enter refusal for a channel bound to a conversation fork isn't made: it shares its flag with the already-tuned-out refusal, which is. One call is a control: a tune_out enter that succeeds still reads as a success, with its parameters.
  • A script and a module. The fourth test makes the same refusals from a script and from a module calling in process. A script's call of tune_out, the api module and channel_open gets Error: <text>, with the same control. A module's ctx.callTool of channel_open gets the registry's result whole, flagged and with its data, and of the api module the flagged Unknown tool.

The tests protect what an agent is told: a resident or the subconscious can tell a refused call from one that ran, and reads why it was refused. They also protect what an in-process caller reads: the flag, and the tuned-out refusal's data.

  • Red first: on main's src (92125b07) with this test file, all 4 tests fail. For channel_open, a probe on main showed the model receives {"refusal":"tuned-out","channelId":"disc:guild:noisy"} with isError: false.
  • Each piece: each of the 11 flags was put back as main has it, on its own, and the new file run after a rebuild. Each of the 10 reachable ones fails a test, and the unreachable cancel_tuneout one fails none. Removing the tuned-out refusal's data fails the fourth test.
  • The suite: at this head, on Linux, the typecheck and build are clean. Our run of CI's test step (node --test --test-force-exit over dist/test/*.test.js) passes 2233 of 2233 on Node 24.21.0 and on Node 20.20.2, and of its unforced step, 103 of 103 on each. GitHub's CI passed at this head on Ubuntu and macOS, each on Node 20 and Node 24: https://github.com/anima-research/agent-framework/actions/runs/38685007996. Each of the 145 test files, run on its own on Node 24.21.0, passes: 2233 tests in all. main (92125b07), run file by file the same way, passes 2229 tests in 144 files. The difference is this PR's new file and its 4 tests.
  • Who else reads these results.
    • connectome-host (main at ca512a0) shows a stored tool result's flag in its web UI (tool result · error) and in its /puppet command (→ error, with the error text in place of the data). Both now show these refusals as errors, and /puppet shows a puppeted channel_open refusal's sentence where it showed its data.
    • Its channel-mode and subscription GC modules call channel_open and channel_close in process and read success, error and data.refusal, which this leaves as they were.
    • The tool lifecycle events MCPL servers observe already counted a success: false result as an error (src/mcpl/tool-lifecycle.ts, onResult), so they don't change.
  • Open PRs: a git merge-tree trial against the 59 open PRs (Oct 15): the 27 that merge cleanly with main merge cleanly with this branch too, and the other 32 conflict with both. Three diffs come near these results, and none asserts their flags or texts. fix(tune-out): enforce diversion across live delivery and recovery #201's test calls speak_in_channel with channel speech off and asserts only success: false. feat(mcp): modern MCP (2026-07-28) servers beside MCPL, over stdio and Streamable HTTP #262 and feat(mcp): MCP servers on the 2025 revisions over Streamable HTTP #337 rework the script path's result handling and keep Error: <text> for an error result.

Not verified

  • A live provider. The tests read the result as the mock membrane receives it and send no request to a provider, in native or XML tool mode. From membrane 0.5.87's source, the version this resolves to:
    • native tool use sends isError as the tool_result block's is_error (src/formatters/native.ts, src/providers/anthropic.ts);
    • XML tool mode renders an error result as <error>…</error>, and any other result as <result><stdout>…</stdout></result> (formatToolResults, src/utils/tool-parser.ts).
  • The puppet path. puppetToolCall isn't run here. It calls executeToolCall and stores the pair as a model's call is stored.
  • Windows. Our runs were on Linux, and GitHub's CI on Ubuntu and macOS.

Out of scope

  • Each refusal's wording, which follow-up PRs change.
  • Tool traces for these refusals. dispatchChannelToolCall traces tool:completed for every result its promise resolves with. So a channel tool's error result, such as channel_open on an unknown channel, was already traced as completed on main, and the tuned-out refusal joins them. tune_out and the subconscious's tools trace none. Both are operator-facing and unchanged here.
  • feat(focus): single-channel focus mode with held backlog, autoreply, and expiry #195 (focus mode, open) adds refusals in the same isError: false shape, which would want the same flag when it lands.

Review

Reviewed in the house: an internal review of the change before publication, whose findings this head answers. Naomi-2422 then read it whole. She traced each changed flag to an agent by its code, through the model's path, a script's and an in-process caller's, with membrane 0.5.87 packed to read its rendering. She read connectome-host's readers of these results, and checked that channel_close's module-only refusal is rightly left. She ran the new file at this head on Node 24.21.0 and 20.20.2, where it passes 4 of 4, and on Node 24.21.0 with main's src, where all 4 fail.

The sites came from Phoebe-2419's sweep of agent-framework's agent-facing errors. Ada-1017 made the design calls: the tuned-out refusal keeps its data, and this PR changes the flags only.

— Hamish-2569 (Opus), contributing through nissa-seru

tune_out's refusals, channel_open on a tuned-out channel, all but one of
the subconscious's refusals and the api module's "Unknown tool" came back
unflagged: isError false, or for the api module no isError at all
(deliver_summary's "subconscious not configured" was already flagged). A
result that isn't an error reaches the model as a tool's ordinary answer:
in native tool use without is_error, in XML tool mode as
<result><stdout>, and in a script's inner call without "Error: ". So the
model could take a refusal for a success. channel_open's refusal also
carries data ({refusal: 'tuned-out', channelId}), and a result that isn't
an error is rendered from its data, so the model saw only that JSON and
never the sentence pointing it to tune_out's cancel mode.

Each now returns isError: true, and the model reads the refusal's text:
- framework.ts, dispatchTuneOutToolCall: an unknown channel, a cancel the
  coordinator refuses, and an enter it refuses (already tuned out, a
  channel bound to a conversation fork, or no subconscious, which can't
  happen, since the coordinator exists only with one).
- mcpl/channel-registry.ts, handleToolOpen: a tuned-out channel. The
  refusal keeps its data, so an in-process caller can still read why.
- tune-out/coordinator.ts, handleSubconsciousTool: deliver_summary with no
  text, cancel_tuneout of a channel that isn't tuned out, note_disposition
  with no key, and speak_in_channel with channel speech off or no text.
  cancel_tuneout's second refusal, the cancel failing after its lookup
  found the channel, can't be reached, since both read the same state
  synchronously; it changes with the rest.
- modules/api/index.ts: "Unknown tool".

Each message's wording is unchanged here; follow-up pull requests reword
them. channel_close's refusal of a machine close of an explicit open
stays isError false. It answers module-origin closes only, so its result
goes back to the module that called, never straight to a model.

test/refusals-are-error-results.test.ts makes a refusal through each
reachable flag as a model-issued call and checks what goes back to the
model: an error result, with the refusal's text. A fourth test makes the
same refusals from a script, which gets "Error: <text>", and from a
module calling in process, which gets the result flagged, with the
tuned-out refusal's data kept. All four tests fail on main's src.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@greptile-apps

greptile-apps Bot commented Oct 15, 2026 •

Copy link
Copy Markdown

RetriggerView in GreptileConfidence Score: 5/5 Tier: apex

[Medium impact] This PR appears safe to merge; no actionable issues were found.

Summary

Marks existing tune_out, tuned-out channel_open, subconscious-tool, and API refusals as errors.

  • Refused tool calls now reach agents as errors.

No actionable issues found. Review used code inspection; tests were not run.

Reviews (1) · Last reviewed commit: "fix(tools): a refusal is an error result" · Reviewed by Greptile

@nissa-seru
nissa-seru merged commit 5c472b1 into anima-research:main Oct 17, 2026
6 checks passed
nissa-seru pushed a commit to nissa-seru/agent-framework that referenced this pull request Oct 17, 2026
main had taken 26 commits since 3af13bf: anima-research#367 (a refusal is an error
result), anima-research#411 (tune-out errors), anima-research#453 (quiesce writes its record first)
and anima-research#207 (end-turn accounting).

One conflict, in src/framework.ts: both sides add a private field to
AgentFramework at the same place, anima-research#247's inboundAcceptanceObserver and
anima-research#207's streamBoundaryEnds. Both are kept.

None of main's new lines reads or writes anima-research#247's turn silence, its cause,
its drafts or its receipts. anima-research#207 records a tool-ended turn as a
completion at the endTurn boundary itself, and leaves driveStream's
turn_ended cancel, where the turn's receipt is written, as it was.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nissa-seru pushed a commit to nissa-seru/agent-framework that referenced this pull request Oct 17, 2026
…ima-research#367, anima-research#411, anima-research#453 and anima-research#207

The conflict, and what it keeps:
- runResume's finally: anima-research#453's reset of resumeInFlight, then this branch's
  reopen of the MCPL data planes with readyMcplPlanes(), with no data-plane
  gate. anima-research#453 changed only the flag there. The barrier funnel beside it on
  main is the gate this branch removes.

anima-research#453's other changes merge without a conflict. quiesce() writes the
host-mode record before it sets the flag, createOn marks a quiesced boot's
record as possibly quiesced, and a resume() on a serving host clears the
record when it may still say quiesced. anima-research#367, anima-research#411 and anima-research#207 merge without
a conflict, and in src/framework.ts and src/types/framework.ts they change
no member or declaration this branch changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant