Repository navigation
Conversation
|
Thanks for this. Lenient matching helps local models, but as written it lets the name that approval sees differ from the tool that runs, which matters for embedders that gate tool calls by name. The approved name is not the executed tool. In nib (the main embedder), a PreToolUse hook configured for Ambiguous matches. Suggested fix:
The error-message change ( |
…ently
Local models routinely call a tool by a variant of its name ("Bash",
"functions.bash", "tools/bash", "coach-regeln"). Tools.Find now falls
back to a normalized match when there is no exact one, and a pickTool
miss names the tool the model chose.
The lenient match is made safe for embedders that gate tool calls by
name (review on mudler#64):
- Before any tool-call callback runs, every selected tool's name is
rewritten to the real name of the tool it resolves to, in pickTool
and for native tool calls, so what gets approved is what gets
executed. A model that calls "Bash" no longer runs "bash" past an
approval hook configured for "bash".
- A lenient match only counts when exactly one tool matches. When two
or more tools normalize to the same name ("github/list_issues" and
"gitlab/list_issues"), the name is not found and the error names the
candidates.
Tests: the callback receives the canonical name for lenient matches
(three spellings, end to end through ExecuteTools); an ambiguous name
resolves to nothing, the error names both candidates, and end to end
neither tool runs. Without the canonicalization the callback test sees
the model's spelling; without the uniqueness rule the ambiguous tests
pick and run the first tool.
Signed-off-by: Stefan Walcz <stefan.walcz@walcz.de>
Assisted-by: Claude:claude-opus-5-5
201beec to
b2f239c
Compare
|
Thanks for the careful review — both points were real. Updated and rebased on current main (single commit
The non-e2e suite passes; "Sub-Agent Spawning — Background agent spawning" fails occasionally, but it does so on untouched main as well (1 in 4 runs on each here), so it's pre-existing timing flakiness rather than this change. The |
Two small, related robustness fixes in
tools.gofor local-model tool selection. We hit these repeatedly in production (agent pool via LocalAI) — 4 hard turn-failures in one evening across two different Qwen-class local models.1. The error hides the chosen name (
pickTool)When the model's chosen tool can't be resolved,
pickToolreturned a barechosen tool not found. The name was onlyxlog.Debug-logged, so operators running at info level saw repeated failures with zero diagnosability. Now:chosen tool %q not found.2.
Tools.Findmatched exact-onlyLocal models routinely emit namespaced / case / separator variants —
functions.coach_regeln,Coach_Regeln,tools/coach_regeln— each of which missed and became a hard failure.Findnow keeps exact match as the fast path and adds a lenient fallback that only triggers on a miss: it strips a leading namespace segment (functions./tools//ns::), case-folds, and treats-≡_. When an exact match exists it is always returned, so existing resolutions are unaffected (covered by a test).Tests
Table-driven tests for
normalizeToolName, lenientFind, and the exact-wins-over-lenient guarantee.go test,go vet,gofmtclean.Related in spirit to #60 / #62: helper/selection steps benefit from diagnosability and tolerance of model quirks. Happy to adjust (e.g. gate the lenient match behind an option) if preferred.
🤖 Generated with Claude Code