Skip to content

feat(config): add strict_tools to send tool schemas in strict mode - #132

Merged
mudler merged 1 commit into
masterfrom
feat/strict-tool-schemas
Sep 27, 2026
Merged

mudler merged 1 commit into
masterfrom
feat/strict-tool-schemas

Conversation

@mudler-agent

@mudler-agent mudler-agent commented Sep 27, 2026 •

Copy link
Copy Markdown
Collaborator

What changed

  • strict_tools (top-level, default false): turns on cogito's EnableStrictToolSchemas (feat(tools): add EnableStrictToolSchemas and keep MCP additionalProperties cogito#81). Tools are sent with "strict": true and a schema that follows strict mode's rules: additionalProperties: false everywhere, every property required, formerly optional ones nullable. cogito drops the resulting nulls before it calls the tool.
  • Named endpoints inherit it, like reasoning_effort, and can set strict_tools: false to opt out. /settings lists it.
  • cogito bump to the feat(chat): add ephemeral todo_write tool for task tracking #81 commit. With or without strict_tools, MCP tools now carry the additionalProperties their server declares, so bash tells the model that unknown arguments are rejected.
  • README config reference and the named-endpoint inheritance paragraph, synced to selfdoc/.

Why

Models called bash with command instead of script, and the MCP server rejected every call (unexpected additional properties ["command"]). On LocalAI with a model that uses the backend's chat template, llama.cpp's autoparser builds the tool-call grammar, and under that grammar argument names are literals from the schema, so command cannot be generated. The grammar is lazy under tool_choice: auto, though: it starts only when the model writes the format's trigger marker. A call written without the marker comes back from the autoparser as plain text, and LocalAI's Go-side text-parsing fallback (core/http/endpoints/openai/chat.go, "Content-based tool call fallback") turns it into a tool call without checking it against the schema. That is how {"command": ...} got through.

Correction: an earlier version of this description said strict mode is the fix for LocalAI. It is not the right lever there: a strict tool makes LocalAI send its own grammar, which replaces the autoparser's pipeline and can break tool calls on template-based models (LocalAI issue #10052). The LocalAI-side fix is to validate fallback-parsed tool calls against the schema (PR in progress on mudler/LocalAI). strict_tools stays useful for OpenAI and for servers without a native tool parser, which is why it is off by default. #129 (accepting command) covers the gap meanwhile.

Before merging

How to verify

  • go test ./... with an empty HOME. New tests:
    • TestStrictToolsSendsBashAsAStrictTool: the real bash server behind a fake OpenAI endpoint; the first request sends bash with strict: true, additionalProperties: false and timeout typed [integer, null].
    • TestStrictToolsOffByDefault: nothing is strict by default, and bash still carries additionalProperties: false.
    • TestConfigNamedEndpointInheritsStrictTools: the default endpoint, an inheriting endpoint and an opted-out one.
  • By hand: set strict_tools: true on a LocalAI endpoint and check that tool calls still work, and that LocalAI's debug log shows grammar in use.

🤖 Generated with Claude Code

Models called nib's bash tool with "command" instead of "script". The
tool's schema only described the arguments: LocalAI turns its tool
grammar off for models that use the backend's chat template, and
llama.cpp's grammar is lazy under tool_choice auto, so nothing stopped
the model, and the MCP server rejected every call.

strict_tools (top level, inherited by named endpoints, which can set it
to false) turns on cogito's EnableStrictToolSchemas: tools are sent with
"strict": true and a schema that follows strict mode's rules. LocalAI
switches its tool grammar on for such a request, and OpenAI enforces
the schema. It is off by default because some OpenAI-compatible servers
reject the field, and on LocalAI it can conflict with a model that uses
the backend's template.

The cogito bump also makes MCP tools carry the additionalProperties
their server declares, so bash now tells the model that unknown
arguments are not accepted, with or without strict_tools.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@mudler
mudler merged commit 12a7ff2 into master Sep 27, 2026
2 checks passed
mudler-agent added a commit that referenced this pull request Sep 27, 2026
…133)

#132 pinned cogito to bca971d, the head of cogito#81's branch. cogito
squash-merged #81 as d35c7af, so bca971d is reachable only through that
branch, and a build that fetches modules directly breaks once the
branch is deleted. Both commits have the same tree.

Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
mudler added a commit that referenced this pull request Sep 28, 2026
… tools (#134)

The cogito bump in #132 sent an MCP tool that takes no arguments with
"properties": null. vLLM validates every tool schema and rejected the
whole request with a 400 ("None is not of type 'object'"), so on
regolo every turn failed while such a tool was configured.

cogito#82 sends an empty object instead.

Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants