Skip to content

fix: treat hallucinated tool calls as plain response when tools are removed - #9913

Open
FionaFaust wants to merge 2 commits into
AstrBotDevs:masterfrom
FionaFaust:fix/9912-hallucinated-tool-call
Open

fix: treat hallucinated tool calls as plain response when tools are removed#9913
FionaFaust wants to merge 2 commits into
AstrBotDevs:masterfrom
FionaFaust:fix/9912-hallucinated-tool-call

Conversation

@FionaFaust

@FionaFaust FionaFaust commented Sep 1, 2026

Copy link
Copy Markdown

Summary

When an agent run reaches the max_step limit, run_agent removes all tools (req.func_tool = None) and appends a user message asking the model to answer directly. If the model still returns tool calls (hallucination), the run used to:

  • silently produce no final reply (the runner stays RUNNING and the step loop exits without a response), and
  • append a dangling assistant(tool_calls) message (with no matching tool result) to the context, which then gets persisted into the conversation history and re-sent to the provider on every later turn.

Closes #9912

Root cause

  • _handle_function_tools returns immediately when req.func_tool is falsy (if not req.func_tool: return), appending no tool result.
  • step() then unconditionally extends the context with assistant(tool_calls) + (empty) results, creating a protocol-invalid dangling message, and never transitions the state away from RUNNING.

Changes

  1. step() hallucination guard: before finalizing/entering the tool-call branch, if the model returned tool calls while no tools are available, strip the tools_call_* fields and finalize via the existing _complete_with_assistant_response. The run reaches DONE, the user receives a reply, and no dangling tool-call message is appended. (This also covers skills_like mode, where the previous "remove tools" step was ineffective because tools were still resolved from the raw tool set.)
  2. Protocol safety net: when assembling ToolCallsResult, if any tool_call_id has no matching tool result block, append a placeholder tool message so the context always stays protocol-valid even if a future path yields partial results.

Tests

  • Added test_hallucinated_tool_call_when_tools_removed_finalizes_with_plain_response: tools removed + model hallucinates a tool call -> run completes, llm_result is produced, no dangling tool-call message in the context.
  • Added test_tool_calls_without_results_get_placeholder_tool_blocks: tool call produces no result -> placeholder tool block is appended and paired correctly.
  • tests/test_tool_loop_agent_runner.py: 42 passed.
  • ruff check / ruff format --check on the touched files: clean.

Summary by Sourcery

Prevent hallucinated or incomplete tool calls from leaving agent runs unfinished or conversation history protocol-invalid.

Bug Fixes:

  • Finalize runs with a plain assistant response when models emit hallucinated tool calls after tools have been removed.
  • Preserve protocol-valid tool-call context by adding placeholder results for tool calls that lack execution results.

Tests:

  • Add coverage for hallucinated tool calls during forced finalization and missing or mismatched tool results.

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 1 issue

Prompt for AI Agents
Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="astrbot/core/agent/runners/tool_loop_agent_runner.py" line_range="1014-1020" />
<code_context>
+            # Protocol safety net: every tool_call_id must have a matching tool
+            # result, otherwise the context would contain a dangling
+            # assistant(tool_calls) message that providers reject.
+            if len(tool_call_result_blocks) < len(llm_resp.tools_call_ids):
+                for tool_call_id in llm_resp.tools_call_ids:
+                    if not any(
+                        block.tool_call_id == tool_call_id
+                        for block in tool_call_result_blocks
+                    ):
+                        tool_call_result_blocks.append(
+                            ToolCallMessageSegment(
+                                role="tool",
</code_context>
<issue_to_address>
**issue (bug_risk):** The placeholder guard only runs when the number of result blocks is smaller than the number of tool-call IDs, so equal-length results with duplicate IDs still leave another call unmatched. A tool execution that emits multiple result blocks for one call while a later call produces none creates a protocol-invalid assistant/tool message pair without triggering the guard.

**Triggers:** When one tool call yields multiple result blocks and another tool call yields no result block.

**Suggested fix:** Check each `tool_call_id` independently regardless of list lengths, and append a placeholder for every ID absent from the result blocks.
</issue_to_address>

Sourcery assessment

Needs a human reviewer. 1 finding to address first, and if the fallback misclassifies a legitimate tool call or inserts an incorrect placeholder result, the current run's conversation context can contain a wrong tool outcome and produce a bad final answer. Reverting prevents future occurrences, but already affected runs may need to be retried or have their context cleared.

Blocking findings: astrbot/core/agent/runners/tool_loop_agent_runner.py:1020


Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Comment thread astrbot/core/agent/runners/tool_loop_agent_runner.py Outdated
Address Sourcery review: the guard must check each tool_call_id
independently, since equal block/id counts with duplicate ids can still
leave another call unmatched.
@FionaFaust

Copy link
Copy Markdown
Author

Thanks for the review - good catch. I've updated the fix to check each tool_call_id independently instead of comparing list lengths, so duplicate/equal-count result blocks can no longer leave a call unmatched:

  • The placeholder guard now builds the set of existing result ids and appends a placeholder for every id absent from it (no length precondition).
  • Added a regression test test_duplicate_tool_result_ids_still_trigger_placeholder_fill covering the "two declared ids, two result blocks both for the same id" case.

Validation: tests/test_tool_loop_agent_runner.py ? 43 passed; ruff check / ruff format --check clean on the touched files.

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sourcery assessment

Approved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Agent 达到最大步数强制收尾时,若大模型仍返回工具调用,将静默无回复并持久化悬空 tool_calls 消息

1 participant