Fix agent crash on native tool calls and empty tool output - #28
Conversation
PR #27 added a blanket 'content must be non-empty' guard to every Conversation.add_* method. Two of those paths legitimately carry empty content, so the guard crashes the agent on main: - engine.py:105 records a native tool call with add_assistant(content, tool_calls). Ollama returns content='' when the model emits only a tool call, so any native tool call raises ValueError. - engine.py:182 records str(ToolResult); a successful tool with no output (grep with no match, a silent write) stringifies to '' and raises ValueError. Neither call site is inside the loop's try/except, which only catches OllamaError around _call_llm, so both propagate out of process_message. No test covered process_message with native tool calls, so CI stayed green. Empty is now allowed exactly where it is meaningful (assistant messages carrying tool_calls, tool results) and still rejected everywhere else. None is rejected on all paths.
There was a problem hiding this comment.
The two changes in agent/core/conversation.py are correct in isolation, and I verified the premises: ToolResult.__str__ does return "" for ToolResult(True, "") (agent/tools/base.py:15-18), and engine.py:105 / engine.py:182 are outside the try/except OllamaError, so the ValueError from #27 does escape process_message(). The new tests land inside TestConversation and cover both directions. Scope is clean — only conversation.py and tests.
Blocking issue: the third crash path from #27 is still live.
agent/core/engine.py:130 (the else / final-response branch) also calls add_assistant(content) with content that came straight from response.get("message", {}).get("content", ""). When the model returns an empty or whitespace-only final message — truncation at max tokens, a bare "\n", a model that emits nothing after its last tool result — tool_calls_from_api is empty and _extract_tool_calls finds nothing, so control reaches line 130 with content = "". The new guard (if not tool_calls and not content.strip()) fires, and the ValueError propagates out of process_message() exactly like the two paths this PR fixes. Pre-#27 this returned an empty string harmlessly.
So the PR description's table is missing a row, and merging as-is still leaves the agent crashable on an empty LLM response.
Two reasonable ways to close it:
- Drop the empty check from
add_assistantentirely and keep only theNonerejection. #27's blanket non-empty premise simply does not hold for assistant messages — an empty assistant turn is a valid API shape with or without tool calls, and thetool_calls-aware branch in this diff is then unnecessary. - Keep the guard and normalize at the call site in
engine.py:130— e.g. record and returncontent or ""without going through the validating path when the model produced nothing.
Either way, please add a regression test that drives process_message with a mocked LLM returning {"message": {"content": ""}} and no tool calls. That is the same gap the description already identifies as the reason CI stayed green through #27: test_engine.py never exercises process_message end to end, so a unit test on Conversation alone will not catch a recurrence.
Minor, non-blocking: response.get("message", {}).get("content", "") returns None if the key is present with a JSON null, which would then hit the content is None raise on all three branches. Worth defending with or "" at the extraction point in the same pass.
Review caught a third crash path this PR missed: engine.py:130, the final-response branch, calls add_assistant(content) with no tool_calls. An empty or whitespace-only final message - truncation at max tokens, a bare newline, a model that emits nothing after its last tool result - still tripped the guard and escaped process_message(). Taking review option 1: #27's premise was simply wrong. An empty assistant turn is a valid chat-API shape with or without tool calls, so add_assistant now rejects only None and the tool_calls-aware branch is gone. Keeping the guard and special-casing one call site would have left the same trap for the next caller. Also defended the extraction point: response content/tool_calls now use 'or ""' / 'or []', so a present-but-null JSON value cannot reach add_assistant as None. Added TestProcessMessageEmptyResponses, which drives process_message end-to-end rather than Conversation in isolation - the gap that let #27 through CI. Three of its four cases fail against the previous version of this branch.
|
Confirmed and fixed — you're right, and the description's table was missing a row.
Took option 1. #27's premise doesn't hold: an empty assistant turn is a valid chat-API shape with or without tool calls, so Minor taken in the same pass: Regression test at the right level: added That closes the coverage gap that let #27 through in the first place. Full suite: 59 passed. mypy: clean. |
There was a problem hiding this comment.
Reviewed the diff against agent/core/conversation.py and agent/core/engine.py. The change correctly reverts the over-broad non-empty guard from #27 on exactly the two paths that legitimately carry empty content — add_assistant (native Ollama tool calls arrive with content == "") and add_tool_result (str(ToolResult(True, "")) for a no-match grep or silent write) — while keeping None rejected everywhere and leaving add_system/add_user untouched. The or "" normalization at engine.py:96-97 correctly handles a present-but-null content/tool_calls key, and since the new validation is strictly more permissive than what's on main it cannot introduce a new crash path; the loop still terminates on empty final content rather than spinning. Tests are well targeted (they drive process_message end-to-end, which is the gap that let #27 through CI) and the scope is limited to the regression. One nit, non-blocking: the PR description still says "empty allowed only when tool_calls is populated" for add_assistant, but commit 8873f15 made it unconditional — worth updating the body so it matches the code.
The regression
PR #27 (
f1b5091) added a blanket non-empty guard to everyConversation.add_*method. Two call sites legitimately pass empty content, so
maincurrentlycrashes the agent on both:
engine.py:105message.content == ""when the model emits only a tool callengine.py:182str(ToolResult(True, ""))is""— grep with no match, a silent writeNeither is inside the loop's
try/except, which only catchesOllamaErroraround
_call_llm(), so theValueErrorpropagates straight out ofprocess_message().Reproduced against
main:No test exercised
process_messagewithtool_calls_from_apipopulated —test_engine.pyonly covers_extract_tool_callsin isolation — which is whyCI stayed green through the merge.
The fix
Empty is allowed exactly where it carries meaning, and rejected everywhere else:
add_assistant— empty allowed only whentool_callsis populatedadd_tool_result— empty allowed (a successful tool may produce no output)add_system/add_user— unchanged, empty still rejectedNonestill rejected on every pathVerification
TestConversation, covering both crash paths plusthe cases that must still raise. Both new "may be empty" tests fail on
main.mypy --ignore-missing-imports agent: cleanSupersedes #23, #24, #25 and #26, which all proposed variants of this same
validation against a pre-#27 base.