Skip to content

fix(agents): consult response terminators in streaming mode - #4400

Open
BlueX888 wants to merge 1 commit into
camel-ai:masterfrom
BlueX888:fix/prep-streaming-terminators-ignored
Open

BlueX888 wants to merge 1 commit into
camel-ai:masterfrom
BlueX888:fix/prep-streaming-terminators-ignored

Conversation

@BlueX888

Copy link
Copy Markdown

Related Issue

Closes #4399

Description

A ChatAgent configured with response_terminators and a streaming model backend (model_config_dict={"stream": True}) never consulted its terminators: is_terminated() was never called, agent.terminated stayed False, and every yielded response carried terminated=False, so the agent never reported completion in stream mode while the identical non-streaming agent terminates on the same model output. Consumers that break their loop on terminated (e.g. camel/societies/workforce/role_playing_worker.py:154-162) can never end their chat via a terminator in this configuration.

Root cause: the only terminator call sites were the non-streaming ones — _step_impl (camel/agents/chat_agent.py:3132-3150), _astep_non_streaming_task (:3440-3458) and _step_get_info (:3959-3972). The streaming chunk processors _process_stream_chunks_with_accumulator / _aprocess_stream_chunks_with_accumulator never read self.response_terminators and hard-coded terminated=False in the final ChatAgentResponse (parent of this branch: :4868 and :5998), and _stream_response / _astream_response had no equivalent of the "Please continue." continuation loop that _step_impl runs when a terminator reports not-terminated. The constructor docstring (camel/agents/chat_agent.py:423-426) promises the agent "will keep prompting the model until a terminator signals completion" with no streaming carve-out, and the streaming code's own non-streaming fallback branch (via _convert_to_chatagent_response -> _step_get_info) already terminates correctly — the gap was only in the chunk-processing paths.

Changes (camel/agents/chat_agent.py only):

  • Add _check_response_terminators, the same terminator pass _step_get_info performs (call is_terminated() on the completed message list, first firing terminator wins, update self.terminated).
  • Call it at the record_message sites of the four streaming completion paths — _process_stream_chunks_with_accumulator, _aprocess_stream_chunks_with_accumulator, and the structured-output branches of _stream_response / _astream_response — and propagate the resulting status to the final streamed response instead of the hard-coded False. Partial responses keep terminated=False.
  • In _stream_response / _astream_response, when a completed streamed response does not trigger a terminator, mirror _step_impl: stop with a warning at max_iteration, otherwise add the "Please continue." user message to memory and request the model again.

Out of scope: the streaming info dict keeps its existing schema (finish_reasons), so it still carries no termination_reasons entry when a terminator fires; that pre-existing schema difference is the secondary observation noted in the issue.

Testing

Reproduction first on unpatched master (24fde60a) with the four new regression tests added to test/agents/test_chat_agent.py (sync/async termination and sync/async continuation; fully offline via DummyModel + chunk stubs, no API key needed):

$ OPENAI_API_KEY=sk-dummy .venv/bin/pytest test/agents/test_chat_agent.py -k "stream_response_terminator" -q
FAILED test/agents/test_chat_agent.py::test_chat_agent_stream_response_terminators
FAILED test/agents/test_chat_agent.py::test_chat_agent_async_stream_response_terminators
FAILED test/agents/test_chat_agent.py::test_chat_agent_stream_response_terminator_continuation
FAILED test/agents/test_chat_agent.py::test_chat_agent_async_stream_response_terminator_continuation
4 failed, 38 deselected, 2 warnings in 1.13s

The core assertion failure is AssertionError: Expected 'mock' to have been called once. Called 0 times. — is_terminated() is never invoked in streaming mode. The issue's repro script also prints BUG PRESENT: terminator fired in non-streaming mode but was never consulted in streaming mode; agent never terminates. on the unpatched tree.

After the fix:

$ OPENAI_API_KEY=sk-dummy .venv/bin/pytest test/agents/test_chat_agent.py -k "stream_response_terminator" -q
4 passed, 38 deselected, 2 warnings in 0.71s

and the repro script now reports for the same scripted model output "Goodbye!":

non-streaming: response.terminated = True
streaming:    response.terminated = True (type: StreamingChatAgentResponse)
streaming:    agent.terminated   = True
streaming:    terminator fired   = True word_count_dict = [defaultdict(<class 'int'>, {'goodbye': 1})]

No existing test changes states:

$ OPENAI_API_KEY=sk-dummy .venv/bin/pytest test/agents/test_chat_agent.py -q
# unpatched master + new tests: 8 failed, 34 passed
# patched:                       4 failed, 38 passed
$ OPENAI_API_KEY=sk-dummy .venv/bin/pytest --fast-test-mode test/agents/test_chat_agent.py -q
14 passed, 28 skipped, 2 warnings in 1.09s
$ OPENAI_API_KEY=sk-dummy .venv/bin/pytest --fast-test-mode test/agents/ -q --ignore=test/agents/test_agent_e2e.py
3 failed, 77 passed, 49 skipped, 2 warnings in 6.51s

The 4 module failures (test_tool_calling_async, test_chat_agent_creation_methods, test_chat_agent_stream_with_structured_output, test_chat_agent_async_stream_with_structured_output) need a live OpenAI backend and fail identically on unpatched master. The 3 failures in the wider test/agents/ run (test_knowledge_agent.py::test_parse_graph_elements — ModuleNotFoundError for an optional dep, test_role_playing.py::test_role_playing_role_sequence, test_task_agent.py::test_task_planner_agent[model0]) also fail identically on unpatched master in this environment.

Lint and type check, run as CI runs them (.pre-commit-config.yaml):

$ pre-commit run --all-files
ruff...........Passed
ruff-format....Passed
Check mypy....Failed
Found 141 errors in 84 files (checked 852 source files)
Check License..Passed
codespell......Passed
(secrets / end-of-files / trailing-whitespace: Passed)

The mypy count is byte-identical to unpatched master in this environment (missing optional dependency stubs such as requests, playwright, nebula3, flask, pptx, sklearn); neither changed file reports an error.

What is the purpose of this pull request?

  • Bug fix
  • New Feature
  • Documentation update
  • Other

Checklist

  • I have read and agree to the AI-Generated Code Policy (required)
  • I have linked this PR to an issue (required)
  • I have checked if any dependencies need to be added or updated in pyproject.toml and run uv lock — no dependency changes
  • I have updated the tests accordingly (required for a bug fix or a new feature)
  • I have updated the documentation if needed — not needed, existing behaviour only
  • I have added examples if this is a new feature — n/a

AI assistance disclosure: this patch was drafted with AI assistance and reviewed by a human; the test output quoted above was produced by running the commands shown against this branch and unpatched master.

Streaming paths never called ResponseTerminator.is_terminated() and
hard-coded terminated=False in every yielded response, so an agent
configured with response_terminators and a streaming model backend
never reported termination and skipped the "Please continue."
continuation loop that non-streaming step() applies.

Mirror the non-streaming behavior in _stream_response,
_astream_response and their chunk processors / structured-output
branches: check terminators on each completed streamed message,
propagate the status to the final streamed response and agent state,
and prompt the model to continue until a terminator fires or
max_iteration is reached.

Add regression tests covering sync/async streaming termination and
continuation.
@coderabbitai

coderabbitai Bot commented Oct 11, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration
  • Configuration used: Repository UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: cf539b11-a5e7-4046-b7a5-e10bf67fee3c

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] ChatAgent never evaluates response_terminators in streaming mode (stream=True); agent never terminates

1 participant