Describe the bug
Agent.run documents exit_reason as, verbatim:
the name of the tool that satisfied a tool exit condition (in which case last_message is that tool's result)
When the model requests several tools in one assistant message and the exit-condition tool is not the one it listed last, that promise does not hold: exit_reason names tool A while last_message carries tool B's result.
_check_exit_conditions is deliberately order-agnostic — its own docstring says "Every tool call in the message is checked, so the order of parallel tool calls does not matter" — but last_message is simply messages[-1], and tool result messages are appended in the order the model requested them. The two are only consistent by luck.
This matters because the docs advertise exit_reason for exactly this purpose: "useful for routing the output downstream (e.g. with a ConditionalRouter)". A router that branches on exit_reason == "get_weather" and then reads last_message silently consumes a different tool's output — no error, no warning.
Expected behavior
Either last_message is the exit tool's result as documented, or the documentation says how to actually find it (by origin.tool_name among the step's tool messages).
To Reproduce
from typing import Any, Optional
from haystack import component
from haystack.components.agents import Agent
from haystack.dataclasses import ChatMessage, ToolCall
from haystack.tools import Tool
@component
class FakeChatGenerator:
"""Returns one assistant message requesting two tools, then a plain reply."""
def __init__(self) -> None:
self.calls = 0
@component.output_types(replies=list[ChatMessage])
def run(self, messages: list[ChatMessage], tools: Optional[Any] = None, **kwargs: Any) -> dict[str, Any]:
self.calls += 1
if self.calls == 1:
return {"replies": [ChatMessage.from_assistant(tool_calls=[
ToolCall(id="call_1", tool_name="get_weather", arguments={"city": "Berlin"}),
ToolCall(id="call_2", tool_name="add_to_calendar", arguments={"title": "trip"}),
])]}
return {"replies": [ChatMessage.from_assistant("done")]}
def to_dict(self) -> dict[str, Any]: return {"type": "FakeChatGenerator", "init_parameters": {}}
@classmethod
def from_dict(cls, data: dict[str, Any]) -> "FakeChatGenerator": return cls()
weather = Tool(name="get_weather", description="weather", function=lambda city: f"22C and sunny in {city}",
parameters={"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]})
calendar = Tool(name="add_to_calendar", description="calendar", function=lambda title: "event created",
parameters={"type": "object", "properties": {"title": {"type": "string"}}, "required": ["title"]})
agent = Agent(chat_generator=FakeChatGenerator(), tools=[weather, calendar], exit_conditions=["get_weather"])
agent.warm_up()
result = agent.run([ChatMessage.from_user("Weather in Berlin, and add a calendar entry")])
last = result["last_message"]
print("exit_reason :", result["exit_reason"])
print("last_message produced by :", last.tool_call_result.origin.tool_name)
print("last_message.result :", last.tool_call_result.result)
print("last_message.text :", repr(last.text))
Output on main (4505df3):
exit_reason : get_weather
last_message produced by : add_to_calendar
last_message.result : event created
last_message.text : None
Reproduces identically through Agent.run_async.
Additional context
The same claim appears in three more places, and one of them is independently inaccurate:
haystack/components/agents/agent.py:869 and :955 — the :returns: clause quoted above.
docs-website/docs/pipeline-components/agents-1/agent.mdx:69 — a stronger form: "In this case last_message is that tool's result — a tool-result ChatMessage whose text is empty — so exit_reason tells you how to consume it." text is None on a tool-result message, not empty, as the repro above shows.
test/components/agents/test_agent.py:1114 — a comment restating the same belief.
test_tool_exit_reports_the_first_matching_tool already builds exactly this parallel-call shape, but asserts only exit_reason and never last_message — which is why this went unnoticed.
Two ways to resolve it, and I'm happy to send either
- Documentation — correct the four places and say that the exit tool's result is found via
tool_call_result.origin.tool_name among the step's tool messages. No behaviour change.
- Behaviour — have
_check_exit_conditions report the exit-condition tool whose result is last among the step's tool messages, making the documented promise true. This changes which tool is reported when two exit-condition tools fire in one step, which test_tool_exit_reports_the_first_matching_tool currently pins.
I'd default to (1), because on #12621 @anakin87 raised the same preference for a neighbouring case — "just update the docstring to describe the actual behavior instead of introducing new complex logic". @sjrl, you added exit_reason in #12074, so you may have a view on whether the promise was meant to hold.
I'll open a PR for option (1) right after this; glad to switch it to (2) instead if you'd rather the behaviour changed.
FAQ Check
System:
- OS: macOS 15 (arm64)
- Haystack version:
main @ 4505df3 (3.3.0-rc0)
Describe the bug
Agent.rundocumentsexit_reasonas, verbatim:When the model requests several tools in one assistant message and the exit-condition tool is not the one it listed last, that promise does not hold:
exit_reasonnames tool A whilelast_messagecarries tool B's result._check_exit_conditionsis deliberately order-agnostic — its own docstring says "Every tool call in the message is checked, so the order of parallel tool calls does not matter" — butlast_messageis simplymessages[-1], and tool result messages are appended in the order the model requested them. The two are only consistent by luck.This matters because the docs advertise
exit_reasonfor exactly this purpose: "useful for routing the output downstream (e.g. with aConditionalRouter)". A router that branches onexit_reason == "get_weather"and then readslast_messagesilently consumes a different tool's output — no error, no warning.Expected behavior
Either
last_messageis the exit tool's result as documented, or the documentation says how to actually find it (byorigin.tool_nameamong the step's tool messages).To Reproduce
Output on
main(4505df3):Reproduces identically through
Agent.run_async.Additional context
The same claim appears in three more places, and one of them is independently inaccurate:
haystack/components/agents/agent.py:869and:955— the:returns:clause quoted above.docs-website/docs/pipeline-components/agents-1/agent.mdx:69— a stronger form: "In this caselast_messageis that tool's result — a tool-resultChatMessagewhosetextis empty — soexit_reasontells you how to consume it."textisNoneon a tool-result message, not empty, as the repro above shows.test/components/agents/test_agent.py:1114— a comment restating the same belief.test_tool_exit_reports_the_first_matching_toolalready builds exactly this parallel-call shape, but asserts onlyexit_reasonand neverlast_message— which is why this went unnoticed.Two ways to resolve it, and I'm happy to send either
tool_call_result.origin.tool_nameamong the step's tool messages. No behaviour change._check_exit_conditionsreport the exit-condition tool whose result is last among the step's tool messages, making the documented promise true. This changes which tool is reported when two exit-condition tools fire in one step, whichtest_tool_exit_reports_the_first_matching_toolcurrently pins.I'd default to (1), because on #12621 @anakin87 raised the same preference for a neighbouring case — "just update the docstring to describe the actual behavior instead of introducing new complex logic". @sjrl, you added
exit_reasonin #12074, so you may have a view on whether the promise was meant to hold.I'll open a PR for option (1) right after this; glad to switch it to (2) instead if you'd rather the behaviour changed.
FAQ Check
System:
main@ 4505df3 (3.3.0-rc0)