Repository navigation
feat(config): add strict_tools to send tool schemas in strict mode - #132
Merged
Merged
Conversation
Models called nib's bash tool with "command" instead of "script". The tool's schema only described the arguments: LocalAI turns its tool grammar off for models that use the backend's chat template, and llama.cpp's grammar is lazy under tool_choice auto, so nothing stopped the model, and the MCP server rejected every call. strict_tools (top level, inherited by named endpoints, which can set it to false) turns on cogito's EnableStrictToolSchemas: tools are sent with "strict": true and a schema that follows strict mode's rules. LocalAI switches its tool grammar on for such a request, and OpenAI enforces the schema. It is off by default because some OpenAI-compatible servers reject the field, and on LocalAI it can conflict with a model that uses the backend's template. The cogito bump also makes MCP tools carry the additionalProperties their server declares, so bash now tells the model that unknown arguments are not accepted, with or without strict_tools. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
mudler-agent
added a commit
that referenced
this pull request
Sep 27, 2026
…133) #132 pinned cogito to bca971d, the head of cogito#81's branch. cogito squash-merged #81 as d35c7af, so bca971d is reachable only through that branch, and a build that fetches modules directly breaks once the branch is deleted. Both commits have the same tree. Co-authored-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
mudler
added a commit
that referenced
this pull request
Sep 28, 2026
… tools (#134) The cogito bump in #132 sent an MCP tool that takes no arguments with "properties": null. vLLM validates every tool schema and rejected the whole request with a 400 ("None is not of type 'object'"), so on regolo every turn failed while such a tool was configured. cogito#82 sends an empty object instead. Co-authored-by: Ettore Di Giacinto <mudler@localai.io> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
strict_tools(top-level, defaultfalse): turns on cogito'sEnableStrictToolSchemas(feat(tools): add EnableStrictToolSchemas and keep MCP additionalProperties cogito#81). Tools are sent with"strict": trueand a schema that follows strict mode's rules:additionalProperties: falseeverywhere, every property required, formerly optional ones nullable. cogito drops the resultingnulls before it calls the tool.reasoning_effort, and can setstrict_tools: falseto opt out./settingslists it.strict_tools, MCP tools now carry theadditionalPropertiestheir server declares, sobashtells the model that unknown arguments are rejected.selfdoc/.Why
Models called
bashwithcommandinstead ofscript, and the MCP server rejected every call (unexpected additional properties ["command"]). On LocalAI with a model that uses the backend's chat template, llama.cpp's autoparser builds the tool-call grammar, and under that grammar argument names are literals from the schema, socommandcannot be generated. The grammar is lazy undertool_choice: auto, though: it starts only when the model writes the format's trigger marker. A call written without the marker comes back from the autoparser as plain text, and LocalAI's Go-side text-parsing fallback (core/http/endpoints/openai/chat.go, "Content-based tool call fallback") turns it into a tool call without checking it against the schema. That is how{"command": ...}got through.Correction: an earlier version of this description said strict mode is the fix for LocalAI. It is not the right lever there: a strict tool makes LocalAI send its own grammar, which replaces the autoparser's pipeline and can break tool calls on template-based models (LocalAI issue #10052). The LocalAI-side fix is to validate fallback-parsed tool calls against the schema (PR in progress on mudler/LocalAI).
strict_toolsstays useful for OpenAI and for servers without a native tool parser, which is why it is off by default. #129 (acceptingcommand) covers the gap meanwhile.Before merging
go.modpins cogito atbca971dfrom the feat(chat): add ephemeral todo_write tool for task tracking #81 branch. feat(chat): add ephemeral todo_write tool for task tracking #81 has merged asd35c7af; chore(deps): pin cogito to the merged strict-schemas commit on main #133 re-pins to it.strict. On LocalAI, a model using the backend's chat template may conflict with LocalAI's grammar (see the comment about issue #10052 in LocalAI'smodel_config.go). This needs a real run against a LocalAI model before recommending it.How to verify
go test ./...with an emptyHOME. New tests:TestStrictToolsSendsBashAsAStrictTool: the real bash server behind a fake OpenAI endpoint; the first request sendsbashwithstrict: true,additionalProperties: falseandtimeouttyped[integer, null].TestStrictToolsOffByDefault: nothing is strict by default, andbashstill carriesadditionalProperties: false.TestConfigNamedEndpointInheritsStrictTools: the default endpoint, an inheriting endpoint and an opted-out one.strict_tools: trueon a LocalAI endpoint and check that tool calls still work, and that LocalAI's debug log shows grammar in use.🤖 Generated with Claude Code