From e8b86654e897b12c80d01bac85cf226605ccd52b Mon Sep 17 00:00:00 2001 From: Ambuj Upadhyay <34904987+lets-order-some-fries@users.noreply.github.com> Date: Wed, 23 Sep 2026 21:33:39 +0530 Subject: [PATCH 1/3] docs: exit_reason's tool result is not always last_message Agent.run and Agent.run_async both documented exit_reason as "the name of the tool that satisfied a tool exit condition (in which case `last_message` is that tool's result)". That holds only when the exit tool is the last one the model requested. _check_exit_conditions is deliberately order-agnostic -- its docstring says "the order of parallel tool calls does not matter" -- while last_message is the final message of the step. With parallel tool calls the two disagree, and because exit_reason is documented as useful for routing downstream, a ConditionalRouter branching on it and then reading last_message silently consumes a different tool's output. The same claim was in three other places: the shipped docs page, which put it more strongly and also described the message's text as empty when it is None, and a comment in the tests. All four now say to locate the exit tool's result by matching tool_call_result.origin.tool_name against exit_reason. test_tool_exit_reports_the_first_matching_tool already built exactly this parallel-call shape but asserted only exit_reason, which is why the mismatch went unnoticed; it now pins last_message too. Behaviour is unchanged. See #12890 for the alternative -- making _check_exit_conditions report the exit tool whose result lands last, so the original promise becomes true -- which changes which tool is reported when two exit-condition tools fire in one step. --- .../docs/pipeline-components/agents-1/agent.mdx | 2 +- haystack/components/agents/agent.py | 12 ++++++++---- ...it-reason-last-message-docs-4b1c7a9e2f6d08c3.yaml | 11 +++++++++++ test/components/agents/test_agent.py | 11 ++++++++++- 4 files changed, 30 insertions(+), 6 deletions(-) create mode 100644 releasenotes/notes/agent-exit-reason-last-message-docs-4b1c7a9e2f6d08c3.yaml diff --git a/docs-website/docs/pipeline-components/agents-1/agent.mdx b/docs-website/docs/pipeline-components/agents-1/agent.mdx index 8d5eeec5746..49c32a2ee1f 100644 --- a/docs-website/docs/pipeline-components/agents-1/agent.mdx +++ b/docs-website/docs/pipeline-components/agents-1/agent.mdx @@ -66,7 +66,7 @@ The `exit_reason` output tells you why the agent stopped, which makes it easy to - `"text"`: the model returned a complete reply with no tool calls. - `"length"`: the model reached its output-token limit. `last_message` may contain a partial response. - `"content_filter"`: a content filter stopped the model response. `last_message` may contain a partial response. -- the name of the tool that satisfied a tool exit condition. In this case `last_message` is that tool's result — a tool-result `ChatMessage` whose `text` is empty — so `exit_reason` tells you how to consume it. +- the name of the tool that satisfied a tool exit condition. Its result is the tool-result `ChatMessage` in `messages` whose `tool_call_result.origin.tool_name` matches `exit_reason` (read `tool_call_result.result`; `text` is `None` on such a message). When the model requests several tools in one reply, that message is not necessarily `last_message`, which is always the final message of the step. - `"max_agent_steps"`: the agent reached `max_agent_steps` before meeting an exit condition. - a custom reason set by a hook through the `stop_run` state key, such as `"token_budget_exceeded"` from the [`TokenBudgetHook`](./token-budget.mdx). diff --git a/haystack/components/agents/agent.py b/haystack/components/agents/agent.py index dc72a9433e1..1df256025f0 100644 --- a/haystack/components/agents/agent.py +++ b/haystack/components/agents/agent.py @@ -866,8 +866,10 @@ def run( - "exit_reason": Why the Agent stopped, useful for routing the output downstream (e.g. with a `ConditionalRouter`). One of: `"text"` (the model returned a complete reply with no tool calls), `"length"` or `"content_filter"` (the model returned an incomplete reply, which may contain partial - text), the name of the tool that satisfied a tool exit condition (in which case `last_message` is that - tool's result), or `"max_agent_steps"` (the Agent hit `max_agent_steps` before meeting an exit + text), the name of the tool that satisfied a tool exit condition (its result is the tool message whose + `tool_call_result.origin.tool_name` matches it, which is not necessarily `last_message` when the model + requested several tools at once), or `"max_agent_steps"` (the Agent hit `max_agent_steps` before + meeting an exit condition), or a custom reason a hook supplied through the `stop_run` state key. - Any additional keys defined in the `state_schema`. """ @@ -952,8 +954,10 @@ async def run_async( - "exit_reason": Why the Agent stopped, useful for routing the output downstream (e.g. with a `ConditionalRouter`). One of: `"text"` (the model returned a complete reply with no tool calls), `"length"` or `"content_filter"` (the model returned an incomplete reply, which may contain partial - text), the name of the tool that satisfied a tool exit condition (in which case `last_message` is that - tool's result), or `"max_agent_steps"` (the Agent hit `max_agent_steps` before meeting an exit + text), the name of the tool that satisfied a tool exit condition (its result is the tool message whose + `tool_call_result.origin.tool_name` matches it, which is not necessarily `last_message` when the model + requested several tools at once), or `"max_agent_steps"` (the Agent hit `max_agent_steps` before + meeting an exit condition), or a custom reason a hook supplied through the `stop_run` state key. - Any additional keys defined in the `state_schema`. """ diff --git a/releasenotes/notes/agent-exit-reason-last-message-docs-4b1c7a9e2f6d08c3.yaml b/releasenotes/notes/agent-exit-reason-last-message-docs-4b1c7a9e2f6d08c3.yaml new file mode 100644 index 00000000000..ee7e7394783 --- /dev/null +++ b/releasenotes/notes/agent-exit-reason-last-message-docs-4b1c7a9e2f6d08c3.yaml @@ -0,0 +1,11 @@ +--- +fixes: + - | + Correct the documented relationship between ``Agent``'s ``exit_reason`` and ``last_message``. + Both ``Agent.run()`` and ``Agent.run_async()`` stated that when ``exit_reason`` names a tool, + ``last_message`` is that tool's result. ``last_message`` is the final message of the step, while + exit conditions are checked without regard to call order, so the two only agree when the exit + tool happens to be the last one the model requested. With parallel tool calls the promise fails + silently, and a downstream consumer routing on ``exit_reason`` reads a different tool's output. + The documentation now says to locate the exit tool's result by matching + ``tool_call_result.origin.tool_name`` against ``exit_reason``. Behaviour is unchanged. diff --git a/test/components/agents/test_agent.py b/test/components/agents/test_agent.py index 08e568fd3ac..8691592fb84 100644 --- a/test/components/agents/test_agent.py +++ b/test/components/agents/test_agent.py @@ -1111,7 +1111,8 @@ def test_exit_condition_exits(self, weather_tool): assert "last_message" in result assert isinstance(result["last_message"], ChatMessage) assert result["messages"][-1] == result["last_message"] - # The exit reason is the tool that triggered the exit, and `last_message` is that tool's result. + # Only one tool was called, so here the exit tool's result is also `last_message`. That coincidence does + # not hold for parallel tool calls -- see `test_tool_exit_reports_the_first_matching_tool`. assert result["exit_reason"] == "weather_tool" def test_exit_condition_on_tool_provided_at_runtime(self, weather_tool): @@ -1254,6 +1255,14 @@ def test_tool_exit_reports_the_first_matching_tool(self, weather_tool, component ) result = agent.run([ChatMessage.from_user("Go")]) assert result["exit_reason"] == "parrot" + # `last_message` is the final message of the step, not the exit tool's result: the model asked for `parrot` + # first and `weather_tool` second, so the weather result is what lands last. Locate the exit tool's result by + # matching `tool_call_result.origin.tool_name` against `exit_reason` instead of reading `last_message`. + assert result["last_message"].tool_call_result.origin.tool_name == "weather_tool" + exit_result = next( + m for m in result["messages"] if m.tool_call_result and m.tool_call_result.origin.tool_name == "parrot" + ) + assert exit_result is not result["last_message"] def test_max_steps_exit(self, weather_tool, caplog): """Exhausting `max_agent_steps` before meeting an exit condition reports `"max_agent_steps"`.""" From 3f02c5e54c318690e9518634cc19646e1fcf1a84 Mon Sep 17 00:00:00 2001 From: Ambuj Upadhyay <34904987+lets-order-some-fries@users.noreply.github.com> Date: Mon, 28 Sep 2026 20:26:08 +0530 Subject: [PATCH 2/3] docs: address review on the exit_reason wording - Rewrap the exit_reason paragraph in the run() and run_async() docstrings, and say the *last* matching tool message there too: messages covers the whole run, so the same tool can have results from earlier steps (an errored call, or an exit an on_exit hook vetoed). - docs page: take the suggested wording. - Tests: drop the comments and the exit_result lookup; the last_message assertion covers it. - Drop the release note; this is a docs-only change. Co-Authored-By: Claude Opus 5.5 --- .../pipeline-components/agents-1/agent.mdx | 2 +- haystack/components/agents/agent.py | 18 ++++++++---------- ...son-last-message-docs-4b1c7a9e2f6d08c3.yaml | 11 ----------- test/components/agents/test_agent.py | 9 --------- 4 files changed, 9 insertions(+), 31 deletions(-) delete mode 100644 releasenotes/notes/agent-exit-reason-last-message-docs-4b1c7a9e2f6d08c3.yaml diff --git a/docs-website/docs/pipeline-components/agents-1/agent.mdx b/docs-website/docs/pipeline-components/agents-1/agent.mdx index 49c32a2ee1f..c84d7c24595 100644 --- a/docs-website/docs/pipeline-components/agents-1/agent.mdx +++ b/docs-website/docs/pipeline-components/agents-1/agent.mdx @@ -66,7 +66,7 @@ The `exit_reason` output tells you why the agent stopped, which makes it easy to - `"text"`: the model returned a complete reply with no tool calls. - `"length"`: the model reached its output-token limit. `last_message` may contain a partial response. - `"content_filter"`: a content filter stopped the model response. `last_message` may contain a partial response. -- the name of the tool that satisfied a tool exit condition. Its result is the tool-result `ChatMessage` in `messages` whose `tool_call_result.origin.tool_name` matches `exit_reason` (read `tool_call_result.result`; `text` is `None` on such a message). When the model requests several tools in one reply, that message is not necessarily `last_message`, which is always the final message of the step. +- the name of the tool that satisfied a tool exit condition. Its result is the last tool-result `ChatMessage` in `messages` whose `tool_call_result.origin.tool_name` matches `exit_reason`; read it from `tool_call_result.result`. When the model calls several tools in one reply, this may not be `last_message`. - `"max_agent_steps"`: the agent reached `max_agent_steps` before meeting an exit condition. - a custom reason set by a hook through the `stop_run` state key, such as `"token_budget_exceeded"` from the [`TokenBudgetHook`](./token-budget.mdx). diff --git a/haystack/components/agents/agent.py b/haystack/components/agents/agent.py index 1df256025f0..ce556e28b5a 100644 --- a/haystack/components/agents/agent.py +++ b/haystack/components/agents/agent.py @@ -866,11 +866,10 @@ def run( - "exit_reason": Why the Agent stopped, useful for routing the output downstream (e.g. with a `ConditionalRouter`). One of: `"text"` (the model returned a complete reply with no tool calls), `"length"` or `"content_filter"` (the model returned an incomplete reply, which may contain partial - text), the name of the tool that satisfied a tool exit condition (its result is the tool message whose - `tool_call_result.origin.tool_name` matches it, which is not necessarily `last_message` when the model - requested several tools at once), or `"max_agent_steps"` (the Agent hit `max_agent_steps` before - meeting an exit - condition), or a custom reason a hook supplied through the `stop_run` state key. + text), the name of the tool that satisfied a tool exit condition (its result is the last tool message in + `messages` whose `tool_call_result.origin.tool_name` matches it, which may not be `last_message` when + the model called several tools at once), or `"max_agent_steps"` (the Agent hit `max_agent_steps` before + meeting an exit condition), or a custom reason a hook supplied through the `stop_run` state key. - Any additional keys defined in the `state_schema`. """ agent_inputs = {"messages": messages, "streaming_callback": streaming_callback, **kwargs} @@ -954,11 +953,10 @@ async def run_async( - "exit_reason": Why the Agent stopped, useful for routing the output downstream (e.g. with a `ConditionalRouter`). One of: `"text"` (the model returned a complete reply with no tool calls), `"length"` or `"content_filter"` (the model returned an incomplete reply, which may contain partial - text), the name of the tool that satisfied a tool exit condition (its result is the tool message whose - `tool_call_result.origin.tool_name` matches it, which is not necessarily `last_message` when the model - requested several tools at once), or `"max_agent_steps"` (the Agent hit `max_agent_steps` before - meeting an exit - condition), or a custom reason a hook supplied through the `stop_run` state key. + text), the name of the tool that satisfied a tool exit condition (its result is the last tool message in + `messages` whose `tool_call_result.origin.tool_name` matches it, which may not be `last_message` when + the model called several tools at once), or `"max_agent_steps"` (the Agent hit `max_agent_steps` before + meeting an exit condition), or a custom reason a hook supplied through the `stop_run` state key. - Any additional keys defined in the `state_schema`. """ agent_inputs = {"messages": messages, "streaming_callback": streaming_callback, **kwargs} diff --git a/releasenotes/notes/agent-exit-reason-last-message-docs-4b1c7a9e2f6d08c3.yaml b/releasenotes/notes/agent-exit-reason-last-message-docs-4b1c7a9e2f6d08c3.yaml deleted file mode 100644 index ee7e7394783..00000000000 --- a/releasenotes/notes/agent-exit-reason-last-message-docs-4b1c7a9e2f6d08c3.yaml +++ /dev/null @@ -1,11 +0,0 @@ ---- -fixes: - - | - Correct the documented relationship between ``Agent``'s ``exit_reason`` and ``last_message``. - Both ``Agent.run()`` and ``Agent.run_async()`` stated that when ``exit_reason`` names a tool, - ``last_message`` is that tool's result. ``last_message`` is the final message of the step, while - exit conditions are checked without regard to call order, so the two only agree when the exit - tool happens to be the last one the model requested. With parallel tool calls the promise fails - silently, and a downstream consumer routing on ``exit_reason`` reads a different tool's output. - The documentation now says to locate the exit tool's result by matching - ``tool_call_result.origin.tool_name`` against ``exit_reason``. Behaviour is unchanged. diff --git a/test/components/agents/test_agent.py b/test/components/agents/test_agent.py index 8691592fb84..54502d90380 100644 --- a/test/components/agents/test_agent.py +++ b/test/components/agents/test_agent.py @@ -1111,8 +1111,6 @@ def test_exit_condition_exits(self, weather_tool): assert "last_message" in result assert isinstance(result["last_message"], ChatMessage) assert result["messages"][-1] == result["last_message"] - # Only one tool was called, so here the exit tool's result is also `last_message`. That coincidence does - # not hold for parallel tool calls -- see `test_tool_exit_reports_the_first_matching_tool`. assert result["exit_reason"] == "weather_tool" def test_exit_condition_on_tool_provided_at_runtime(self, weather_tool): @@ -1255,14 +1253,7 @@ def test_tool_exit_reports_the_first_matching_tool(self, weather_tool, component ) result = agent.run([ChatMessage.from_user("Go")]) assert result["exit_reason"] == "parrot" - # `last_message` is the final message of the step, not the exit tool's result: the model asked for `parrot` - # first and `weather_tool` second, so the weather result is what lands last. Locate the exit tool's result by - # matching `tool_call_result.origin.tool_name` against `exit_reason` instead of reading `last_message`. assert result["last_message"].tool_call_result.origin.tool_name == "weather_tool" - exit_result = next( - m for m in result["messages"] if m.tool_call_result and m.tool_call_result.origin.tool_name == "parrot" - ) - assert exit_result is not result["last_message"] def test_max_steps_exit(self, weather_tool, caplog): """Exhausting `max_agent_steps` before meeting an exit condition reports `"max_agent_steps"`.""" From 30b294488e8e32a5101541dd19e0f6328061d6c5 Mon Sep 17 00:00:00 2001 From: Ambuj Upadhyay <34904987+lets-order-some-fries@users.noreply.github.com> Date: Mon, 28 Sep 2026 20:26:23 +0530 Subject: [PATCH 3/3] docs: apply the exit_reason wording to the 3.2 docs The review asked for version-3.2-unstable; #12918 has since promoted it to version-3.2, the current versioned page. Co-Authored-By: Claude Opus 5.5 --- .../version-3.2/pipeline-components/agents-1/agent.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs-website/versioned_docs/version-3.2/pipeline-components/agents-1/agent.mdx b/docs-website/versioned_docs/version-3.2/pipeline-components/agents-1/agent.mdx index 8d5eeec5746..c84d7c24595 100644 --- a/docs-website/versioned_docs/version-3.2/pipeline-components/agents-1/agent.mdx +++ b/docs-website/versioned_docs/version-3.2/pipeline-components/agents-1/agent.mdx @@ -66,7 +66,7 @@ The `exit_reason` output tells you why the agent stopped, which makes it easy to - `"text"`: the model returned a complete reply with no tool calls. - `"length"`: the model reached its output-token limit. `last_message` may contain a partial response. - `"content_filter"`: a content filter stopped the model response. `last_message` may contain a partial response. -- the name of the tool that satisfied a tool exit condition. In this case `last_message` is that tool's result — a tool-result `ChatMessage` whose `text` is empty — so `exit_reason` tells you how to consume it. +- the name of the tool that satisfied a tool exit condition. Its result is the last tool-result `ChatMessage` in `messages` whose `tool_call_result.origin.tool_name` matches `exit_reason`; read it from `tool_call_result.result`. When the model calls several tools in one reply, this may not be `last_message`. - `"max_agent_steps"`: the agent reached `max_agent_steps` before meeting an exit condition. - a custom reason set by a hook through the `stop_run` state key, such as `"token_budget_exceeded"` from the [`TokenBudgetHook`](./token-budget.mdx).