Repository navigation
feat(tools): add EnableStrictToolSchemas and keep MCP additionalProperties - #81
Conversation
…rties A tool's schema only described the arguments; nothing made the model follow it. In nib, models called the bash tool with "command" instead of "script", the MCP server rejected every call, and the model gave up on the shell. LocalAI disables its own tool grammar for models that use the backend's chat template, and llama.cpp's grammar is lazy under tool_choice auto, so those calls were never constrained. EnableStrictToolSchemas wraps every tool usableTools returns so it is sent with "strict": true and a schema that follows strict mode's rules: additionalProperties false on every object, every property required, formerly optional ones nullable. The wrapper drops the nulls the model then sends before it calls the tool. A schema strict mode cannot express is sent unchanged. LocalAI switches its tool grammar on for a request with a strict tool; OpenAI enforces the schema. MCP tools were advertised with a jsonschema.Definition, which has no additionalProperties field, so a server's "additionalProperties": false never reached the model. They now carry it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Follow-up on the "Why" section above, after checking LocalAI and llama.cpp more closely: on LocalAI with a model that uses the backend's chat template, llama.cpp's autoparser does build a tool-call grammar with argument names as literals from the schema. The So on LocalAI, The Also: nib pinned this PR's branch head ( |
An MCP tool that takes no arguments has no "properties" in its input schema, so the unmarshal leaves the property map nil. Before #81 the tool was sent as a jsonschema.Definition, whose MarshalJSON encodes a nil map as {}. mcpToolParameters has no such hook, so the tool went out with "properties": null. vLLM validates every tool schema and rejects the whole request with a 400 ("None is not of type 'object'"), so one argument-less MCP tool broke every call to such a server. Strict mode was not affected: it already fills in an empty properties object. Tool() now sends an empty object when the map is nil. Co-authored-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
What changed
EnableStrictToolSchemas(opt-in). Every toolusableToolsreturns is wrapped so it is sent with"strict": trueand a schema that follows strict mode's rules:additionalProperties: falseon every object (nested, array items,anyOf/allOf,$defs);required; a formerly optional property becomes nullable (type: [T, "null"],nulladded to anenum, oranyOf: [..., {type: null}]when there is notype).The wrapper's
Executedrops thenullvalues the model then sends for an argument it leaves out, so the tool sees an omitted argument, as before. A schema strict mode cannot express (an object with schema-valued ortrueadditionalProperties,oneOf, a boolean schema, a non-object top level) is sent unchanged and not strict. Wrapping is idempotent.MCP tools keep
additionalProperties. They were advertised as a langchaingojsonschema.Definition, which has noadditionalPropertiesfield, so a server's"additionalProperties": false(the go-sdk infers it for every Go struct) never reached the model. They now use a smallmcpToolParametersstruct that carries it. This applies with or without the option.Why
In nib, models called the bash tool with
commandinstead ofscript. The MCP server rejected each call asunexpected additional properties ["command"], and the model concluded it had no shell. Nothing constrained the arguments while the model wrote them:core/config/gguf.go,model_config.go).tool_choice: auto: it applies only after the model emits its format's tool-call trigger.LocalAI turns its grammar back on for a request with a strict tool (
core/http/endpoints/openai/chat.go,strictMode). OpenAI enforces the schema.Caveats
strictfield, and on LocalAI a model using the backend's template can conflict with LocalAI's own grammar (the comment around issue #10052 inmodel_config.go). nib will expose it per endpoint.Background dispatchandshould spawn agent in backgroundspecs fail intermittently onmaintoo (5 runs on main: FFFPF; with this change: FPFFP). Not touched here.How to verify
go test . -ginkgo.label-filter='!e2e' -ginkgo.focus='strict'. The new specs instrict_test.goserve a typed go-sdk tool like nib's bash tool:additionalProperties: false;timeoutnullable and required, and a call with"timeout": nullsucceeds (the server rejects the null as "want integer" without the wrapper; verified by removing it);usableToolswraps only with the option set;anyOfwrapping, and the refused cases.Used by mudler/nib#132 (
strict_tools), which pins this branch's commit until this merges. The same branch also exists on mudler/cogito for that reason.🤖 Generated with Claude Code