fix(tools): stop retrying broken tool-call arguments with identical requests

[mudler-agent]

A nib session on a local LocalAI (llama.cpp, n_ctx 100k, ~14 tok/s) failed with:

failed to select tool: failed to pick tool: tool selection failed: failed to make a streaming decision after 3 attempts: unexpected end of JSON input

The prompt was about 73k tokens. The model then streamed a long tool call until prompt + output reached n_ctx. llama.cpp stopped with finish_reason=length, the call's arguments were cut, and json.Unmarshal failed. cogito then re-sent the identical request, and each attempt took about half an hour. The final error did not say why.

The decision loops treated every argument parse failure the same way and retried with an identical request. That cannot fix a deterministic cause, and the stream reader could also produce cut arguments by itself.

Changes

Errors (tool_args.go). New sentinels for errors.Is:

• ErrStreamInterrupted
• ErrToolArgumentsTruncated
• ErrToolArgumentsInvalid

*ToolArgumentsTruncatedError carries PromptTokens, CompletionTokens, MaxTokens, ToolName, ArgumentsBytes and ReasoningBytes, so a caller can tell "the window ran out" from "the cap was reached", and a long tool call from runaway reasoning. The final decision errors keep their text and wrap the cause.

LocalAI stream reader (clients/localai_client.go):

• A body that ends without [DONE] and without a finish_reason is an ErrStreamInterrupted error, not Done. Before, a cut connection looked like a finished reply, and the decision parsed partial arguments.
• A data: line that does not decode was silently dropped. It is now logged. If the stream carried a tool call, it ends with ErrStreamInterrupted, because the dropped line may have held part of the arguments.
• One SSE line may be up to 16 MiB (was the scanner's 64KB default). Some tool parsers send a call's whole arguments in one chunk.
• StreamEvent (on Done) and LLMReply carry the MaxTokens the client sent, so the decision can compare it with the usage.

Decisions (tools.go, streaming and non-streaming):

• Empty or whitespace-only arguments are {}.
• A parse failure is logged with the tool name, finish_reason, the argument length and the last 80 characters.
• finish_reason=length with a cut tool call:
  • If the completion stopped below the requested cap, the context window ran out. The decision fails at once with *ToolArgumentsTruncatedError and does not re-send.
  • If the cap was reached, or the figures are unknown, it retries once with a raised cap (as the no-content path already did), then fails with the typed error.
• Malformed JSON with a normal finish: the next attempt's request (a copy, not the caller's fragment) gets the assistant message with that call and a tool message giving the parse error and the text sent (first 300 and last 200 bytes). The echoed call's arguments are {}, because llama.cpp chat templates parse earlier tool-call arguments as JSON and would reject the correction request. After the attempts, the error wraps ErrToolArgumentsInvalid.
• An interrupted stream is retried with backoff, like other temporary failures, and the final error wraps ErrStreamInterrupted.

Not changed: clients/openai_client.go. go-openai returns io.EOF both after [DONE] and on a cut connection, so an interruption cannot be detected cleanly there. A comment explains this.

Tests

• clients/localai_stream_integrity_test.go: a stream ending without [DONE] or finish_reason is interrupted; with a finish_reason it is Done; a 200KB single chunk is accepted; a malformed tool-call chunk is not silently dropped.
• decision_tool_args_test.go:
  • Empty arguments.
  • Window-stopped truncation: exactly one request, and the typed error carries the usage, tool name and byte counts.
  • Cap-reached truncation: one retry with a raised cap.
  • Malformed JSON: the correction request pairs a {} call with the tool message, and every echoed tool call carries valid JSON arguments.
  • A persistent failure wraps ErrToolArgumentsInvalid.
  • An interrupted stream wraps ErrStreamInterrupted.
  • Streaming and non-streaming paths.

All tests were written first and failed before the change. make test: 74 passed, 0 failed. Plain go test . in the root is flaky on agent_test.go:206 ("should spawn agent in background"), which fails 2 of 3 runs on unchanged main too.

🤖 Generated with Claude Code