fix(models): report real token usage on streamed replies from any openai-compatible provider

[marcelstenkewitz]

Summary

Upstream clodex 2.21.1 sets includeUsage itself but scopes it to one provider — ollama-cloud
(src/provider-factory.ts:268-272:
spec.providerId === OLLAMA_CLOUD_PROVIDER_ID ? { includeUsage: true } : {}) — so a custom
endpoint and other openai-compatible providers never send
stream_options: {"include_usage": true} on streamed requests. The installed
@ai-sdk/openai-compatible honours the option when it is passed
(@ai-sdk/openai-compatible/dist/index.js:702:
stream_options: this.config.includeUsage ? { include_usage: true } : void 0; :1813
includeUsage: options.includeUsage).

This PR changes the scoping from one hard-coded provider id to upstream's per-provider opt-in
shape: a supportsStreamUsage?: boolean flag on ModelRuntimeCompatibility
(src/model-runtime-compatibility.ts, beside its supportsStore, supportsDeveloperRole,
supportsCountTokens fields), set in the Ollama Cloud catalog's per-model compatibility, and
read by provider-factory.ts in place of the id check. Unset keeps the option off, so every
existing provider is unaffected.

Measured gap

5,690 of 5,692 glm53 requests record output_tokens: 0 (the plan's task 144 measurement,
2026-10-12). glm53's provider is custom-unquant (per ~/.clodex/config.json modelAliases), an
openai-compatible provider the ollama-cloud condition does not cover, so streamed turns never
report a real output token count.

Why the catalog flag and not a change to the scoping

Upstream's per-provider opt-in (supportsStore, supportsDeveloperRole, … on
ModelRuntimeCompatibility) is the shape upstream itself uses. The change is one flag on that
interface, not a change to provider-factory.ts's option plumbing. A model refresh keeps the
flag: the refresh path (src/registry/refresh-models.ts updateProviderCache) replaces the
model cache wholesale, but applyTemplateModelMetadata
(src/registry/fetch-template-models.ts:245-269) re-applies the template's curated metadata —
which carries compatibility — over the fetched list on the api-list path, and the Ollama Cloud
template is staticModelPolicy: 'allowlist' with staticModels carrying the compatibility
block, so the flag is re-applied on every refresh.

The custom-endpoint confirm

src/providers-custom-add.ts (which prompts for name, URL, HTTP and key only today) asks
"Send stream_options.include_usage with streamed requests?" and stores
supportsStreamUsage: true on its model entries' compatibility through
src/registry/custom-endpoint.ts (AddCustomEndpointInput.supportsStreamUsage). The provider
then sends stream_options: {"include_usage": true} and a streamed reply carries a final usage
chunk.

Test result

• pnpm typecheck and pnpm build pass.
• 181 tests across the 5 affected files pass
  (ollama-cloud-stream-usage, ollama-cloud, providers-custom-add, custom-endpoint,
  provider-factory), including a real-@ai-sdk/openai-compatible listener test that
  reproduces the include_usage contract end to end: the request carries
  stream_options: {include_usage: true} and the streamed reply reports the real completion
  token count; other providers' specs without the flag stay without stream_options.
• The full pnpm test suite's remaining failures are pre-existing on pristine origin/main
  (verified on a clean worktree at the same commit: the same test files fail there, in
  connection-pacing and pooled-socket retry paths unrelated to this change).

Commit

3c7a0c7 on fix/openai-compatible-include-usage, one commit, Conventional Commits style,
nothing pushed.

[bman654]

Thanks for this. The problem is real. A server that only sends streaming usage when asked leaves Claude Code with clodex's input-size guess (request bytes ÷ 4) and 0 output tokens on every streamed turn. vLLM, SGLang, llama.cpp, Ollama and LiteLLM all behave this way unless the request carries stream_options.include_usage. We checked their source and docs.

As written, though, the change doesn't reach the endpoint that motivated it, and storing the switch in per-model compatibility causes problems in three places. Details follow; each was reproduced with the real add, refresh, registry and SDK code against a local stub server.

Changes we'd need

1. A refresh erases the opt-in for a custom endpoint. A custom endpoint refreshes through syntheticTemplate (src/registry/resolve-template.ts:116), which carries no curated metadata, and updateProviderCache (src/registry/refresh-models.ts:202) replaces modelsCache wholesale. After clodex providers refresh-models <id>, refresh-all, or the menu's "Refresh model list", the flag is gone, stream_options disappears from the request and output goes back to 0. A failed refresh leaves it alone. The PR description's "A model refresh keeps the flag" holds for Ollama Cloud only, because its template re-applies the catalog. Our README and the Daybreak error message tell ChatGPT users to run clodex providers refresh-models. With no id, that refreshes every provider, so following that advice wipes this setting too.

2. Existing custom endpoints can't opt in. The add flow is the only writer, and there is no edit action. Your own provider would have to be removed and re-added (key re-entered), and the next refresh undoes it (point 1).

3. Opting in silently drops reasoning effort for several model families. A usage-only block { supportsStreamUsage: true } makes effortProviderOptions (src/provider-factory.ts:1054) take the compatibility branch. That branch finds no reasoning fields and returns nothing, so the model-id rules below it never run: Kimi prefix, DeepSeek V4, GLM-5.2, supported_parameters and OpenRouter. With the opt-in, kimi-k3 loses reasoning_effort: "high" in proxy and server mode, both streamed and not. Its patched picker levels also drop from low/medium/high to none. The rule that a compatibility block owns the effort decision is deliberate and pinned (tests/provider-factory.test.ts:334). The new problem is that this PR creates a block with no reasoning fields on routes that had no block before.

4. Existing Ollama Cloud installs lose real usage until a manual refresh. Installs from 2.20.0–2.21.1 store the catalog's compatibility without the new field, and nothing re-applies the catalog at load time (src/registry/materialize.ts:67). After upgrading, those users get no usage until they run a refresh. We reproduced this with the base commit's own catalog: the old provider-id check sends include_usage and reports 7 output tokens, while this head sends nothing and reports 0.

5. Tests don't cover the custom-endpoint half. Every test file still passes in either of these cases: the compatibility spread in addCustomEndpointProvider is removed, or runCustomEndpointAddFlow passes false for every answer. Flipping the prompt's default to true also passes. The existing real-add tests answer "yes" but never check what was stored. The three "leaves the X request without stream_options" negatives use a hand-built compatibility object, so none of them ties to OpenCode Go's real catalog.

Suggested design

Make this a provider-level setting, not a per-model compatibility field:

• Store it on the provider record. api already sits beside headers, and updateProviderCache keeps api across a refresh. Carry it to the spec the way api.headers already flows from materialize.ts through provider-catalog.ts to spec.headers. That fixes 1, and keeping it out of compatibility fixes 3.
• Add a supported way to switch it for an existing provider (a provider-menu action, or a providers subcommand). That fixes 2 and your own endpoint.
• Keep Ollama Cloud's catalog opt-in, but don't let existing caches regress. Either keep the provider-id check as the fallback when the setting is unset, or have the catalog's value apply at load time. That fixes 4.
• Defaults: turn it on by default for newly added custom endpoints and leave existing providers as they are (off) until the user switches it on. The servers people most often point a custom endpoint at need it, and someone who has just added a server will see a rejection on their first request and can turn it off. Nobody's working setup changes on upgrade. If you keep an add-time question, word it for a user and default it to yes, e.g. "Ask this server to report token usage on streamed replies? Most servers (vLLM, llama.cpp, Ollama, LiteLLM) need this. Choose No only if streamed requests fail."
• Add tests that go through the real producers: a real add with yes or no, then real registry reload, materialize, createLanguageModel and a stream against a stub. Assert the stored value, stream_options on the wire, and real output_tokens. Add one that survives refreshProviderModels, one for the reasoning-effort case in 3, and a negative over the real OpenCode Go catalog.

We considered sending include_usage by default to every OpenAI-compatible provider. pi-ai and OpenCode do this, and the self-hosted servers above all accept it. We decided against it for now. Rejections are documented, some recent and some model-specific: Databricks (2026), Mistral, SambaNova, an NVIDIA model, older xAI and older vLLM. A rejecting server would fail every streamed turn, which is worse than inaccurate counts. An automatic "retry without it" fallback doesn't fully cover that: it misses errors that don't name the field, and it sits outside the retry budget. If you have evidence the other way, we're open to it.

Smaller notes

• The JSDoc on supportsStreamUsage says "Unset keeps upstream's per-provider scoping", but no provider-id scoping remains on this head. The other supports* fields mean "unset = normal, false = turn off"; this one only acts on true. tests/ollama-cloud.test.ts:116 still describes keying on the provider id.
• The title says "from any openai-compatible provider", but the change only applies to providers that opt in.
• Description: please drop the references to a private plan ("task 144") and the "nothing pushed" line. Start with a paragraph a clodex user can read. Also, the full-suite failures you saw on main didn't reproduce for us. Typecheck, test and build all pass on head (3466 passed) and on base (3464 passed), Node 24.14.1, and CI is green.
• What does custom-unquant point at? If it is ollama.com or a relay to it, the built-in Ollama Cloud provider already requests usage today.

Checked and fine

• The stored flag reaches the request in every translated route: proxy mode (canonical id and alias), endpoint mode (catalog, alias, single model) and the clodex server Anthropic gateway. Every request carried include_usage and reported real output tokens.
• With only this flag, transformOpenAiCompatibleRequestBody changes nothing. Non-streamed requests get no stream_options. custom-anthropic endpoints are unaffected.
• All three add entry points use the new prompt: providers add, the providers hub and the first-run wizard.
• The OpenAI-format gateway route passes through unchanged and is outside this change's scope. That's expected.
• No conflict with main and no overlapping open PRs.