From 271071023c46813e97b1f097c0e0574041ac1701 Mon Sep 17 00:00:00 2001 From: Vincent Labreche Date: Sat, 24 Oct 2026 00:57:22 +0000 Subject: [PATCH] fix: per-model reasoning knob turns thinking off on the summary lane (#2666) A summary target that thinks by default spends its output budget on reasoning: finish reason "length", empty content, and every fold fails. summaryPayload() sends only model / max_tokens / messages / stream and no config could add a reasoning control (PR #2806: "it needs a typed reasoning knob"). Recipe models (config-file named recipes and agent-registry recipes) take `reasoning`: - "off": the protocol's own control -- openai reasoning_effort "none", responses reasoning.effort "none", anthropic thinking.type "disabled", google generationConfig.thinkingConfig.thinkingBudget 0; - "template-off": chat_template_kwargs.enable_thinking false, openai api only, for chat-template servers (vLLM, SGLang, llama.cpp + Qwen). Validated at recipe parse, chain expansion and on the request rail; applySummaryReasoning() writes it after the output budget. Unset leaves the body byte-identical. Opt-in because strict upstreams reject unknown fields. Closes #2666 --- CHANGELOG.md | 1 + CONFIGURATION.md | 4 +- CONFIGURATION.zh-CN.md | 4 +- src/agent-providers.ts | 8 +- src/config.ts | 9 +- src/external-summary-http.ts | 21 ++++ src/external-summary-settings.ts | 30 ++++- tests/issue2666-summary-reasoning.test.ts | 140 ++++++++++++++++++++++ 8 files changed, 206 insertions(+), 11 deletions(-) create mode 100644 tests/issue2666-summary-reasoning.test.ts diff --git a/CHANGELOG.md b/CHANGELOG.md index 3c6375674f..f77d45f3b3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,7 @@ Versions follow the merge of a `*_release-v*` branch; CI publishes to npm on tag ### Fixed +- **External summary: per-model `reasoning` knob turns thinking off on the summary lane (#2666).** A summary target that thinks by default (Qwen, DeepSeek, GLM thinking …) spent its output budget on reasoning — finish reason `length`, empty content — and every fold failed; bili sent no reasoning control and no config could add one. Recipe models (config-file and agent-registry) now accept `reasoning: "off"` (protocol-native: openai `reasoning_effort: "none"`, responses `reasoning.effort: "none"`, anthropic `thinking.disabled`, google `thinkingBudget: 0`) or `"template-off"` (`chat_template_kwargs.enable_thinking: false`, openai api only, for vLLM/SGLang/llama.cpp chat-template servers). Unset leaves the body unchanged. New tests/issue2666-summary-reasoning.test.ts (15 cases, every protocol positive + negative). - **增量车道:拉取回执修复 ack 传输盲区 + 漂移回退取富侧(#4352/#4323)**。2026-10-22 事故双根因修复:① pi 的 after_provider_response 只在部分 provider 传输下暴露响应头,x-bili-applied-seq 丢失时 noteIncrementalAck 永不执行,后续每轮都整包 re-seed(sinceSeq=0 重发 21K 条历史)。现在 GET /__bili/plugin/turn?conversation= 返回热 canonical 的 {appliedSeq, boundary}(boundary=末条 wire 消息哈希,冷重建诚实返回 null),pi 扩展在规划下一轮前拉取回执,仅在 boundary 证明 canonical 吸收了我方末条消息时采纳——绝不按 seq 前进盲目采纳(外来的全量重发也会推进 canonical);② #4323 多实例写同一目录时,漂移回退无条件吃旧信封,把索引里更新的折叠世代(26,971 refs/16 active)换成旧世代(11,849/12),覆盖与 canonical 失配→0 可压区间→preflight 8.29M 假性死局 502。现在漂移两侧按 mapped ref 数取富侧(平局信封赢,平局语义=原子整文件)。新增 tests/incremental-ack-pull.test.ts 3 例(响应头 ack 回归钉/拉取采纳/冷 boundary null 拒采纳)+ storage-full-flip 4b(陈旧信封让位新索引世代)。 ### Changes diff --git a/CONFIGURATION.md b/CONFIGURATION.md index a9de25ce50..6b12c14aab 100644 --- a/CONFIGURATION.md +++ b/CONFIGURATION.md @@ -1261,7 +1261,9 @@ Targets are **references into the `providers` table**: each entry is the string } ``` -A recipe carries: `baseUrl` (HTTPS; loopback HTTP allowed for local development; no embedded credentials, proxy-recursion paths, or arbitrary query parameters), `api` (one of `openai` | `anthropic` | `responses` | `google`; selects the wire protocol and derives the request path from `baseUrl`), exactly one credential reference — `apiKeyEnv: "VAR_NAME"` (an environment variable read at call time) or `credentialRef: "NAME"` (a value stored via the Web UI) — and a `models` map whose per-model fields are `contextWindow` (default 128000), `outputTokens` (default `min(8192, window/4)`), and `stream` (default false). Recipes can also be split into `recipe`/`bind` routing form like URL entries; a named entry is routing-inert unless bound. +A recipe carries: `baseUrl` (HTTPS; loopback HTTP allowed for local development; no embedded credentials, proxy-recursion paths, or arbitrary query parameters), `api` (one of `openai` | `anthropic` | `responses` | `google`; selects the wire protocol and derives the request path from `baseUrl`), exactly one credential reference — `apiKeyEnv: "VAR_NAME"` (an environment variable read at call time) or `credentialRef: "NAME"` (a value stored via the Web UI) — and a `models` map whose per-model fields are `contextWindow` (default 128000), `outputTokens` (default `min(8192, window/4)`), `stream` (default false), and `reasoning` (unset by default; see below). Recipes can also be split into `recipe`/`bind` routing form like URL entries; a named entry is routing-inert unless bound. + +`reasoning` turns thinking off on that model's summary requests (#2666). A summary model that thinks by default spends its output budget on reasoning: the reply ends with finish reason `length` and no summary text, and every fold fails. `"off"` sends the protocol's own control — `reasoning_effort: "none"` (openai), `reasoning: {effort: "none"}` (responses), `thinking: {type: "disabled"}` (anthropic), `generationConfig.thinkingConfig: {thinkingBudget: 0}` (google). `"template-off"` sends `chat_template_kwargs: {enable_thinking: false}` instead, for openai-compatible servers that render a chat template (vLLM, SGLang, llama.cpp serving Qwen-family models); it requires `api: "openai"`. The knob is opt-in because strict upstreams reject fields they do not know; an unset knob leaves the request body unchanged. Up to 16 targets per chain are supported. Saving an enabled chain through the Web API validates that every reference resolves (unknown provider or model → HTTP 400). At runtime an unresolvable reference is logged once and disables the chain until fixed — it never falls back to summarizing with the main model. `secret:` values are stored separately from the main JSON configuration in the private `billion-context.json.summary-credentials.json` file and are never returned by the configuration API. On Windows, protect this file and its parent directory with an administrator-only ACL; stale `.lock` files require manual removal after confirming no proxy process is writing the store. The total budget is shared across all targets and compression entry points, and cancellation or session-state changes discard generated results without folding. The agent-side plugin bridge (dsh / pi / opencode / codex MCP) sizes each `compress` call's HTTP wait to `totalTimeoutMs` plus a small overhead margin — learned from the proxy at bootstrap — so raising this budget no longer caps long folds at a hidden 60-second client-side timeout (#2652). diff --git a/CONFIGURATION.zh-CN.md b/CONFIGURATION.zh-CN.md index 9f295fed8f..cfae01bae6 100644 --- a/CONFIGURATION.zh-CN.md +++ b/CONFIGURATION.zh-CN.md @@ -1264,7 +1264,9 @@ URL 键通道(或带 `bind` 的命名条目)可以向上游发送**自己的 } ``` -配方包含:`baseUrl`(必须 HTTPS,本地开发可用回环 HTTP;拒绝内嵌凭据、代理递归路径与任意 query 参数);`api`(`openai` | `anthropic` | `responses` | `google` 四选一,决定线上协议并从 `baseUrl` 推导请求路径);恰好一个凭据引用 —— `apiKeyEnv: "变量名"`(调用时读环境变量)或 `credentialRef: "名字"`(通过 Web UI 存储的值);以及 `models` 表,每个模型可设 `contextWindow`(默认 128000)、`outputTokens`(默认 `min(8192, 窗口/4)`)、`stream`(默认 false)。配方也可以像 URL 条目一样拆成 `recipe`/`bind` 路由形态;具名条目未绑定时对路由不生效(routing-inert)。 +配方包含:`baseUrl`(必须 HTTPS,本地开发可用回环 HTTP;拒绝内嵌凭据、代理递归路径与任意 query 参数);`api`(`openai` | `anthropic` | `responses` | `google` 四选一,决定线上协议并从 `baseUrl` 推导请求路径);恰好一个凭据引用 —— `apiKeyEnv: "变量名"`(调用时读环境变量)或 `credentialRef: "名字"`(通过 Web UI 存储的值);以及 `models` 表,每个模型可设 `contextWindow`(默认 128000)、`outputTokens`(默认 `min(8192, 窗口/4)`)、`stream`(默认 false)、`reasoning`(默认不设,见下)。配方也可以像 URL 条目一样拆成 `recipe`/`bind` 路由形态;具名条目未绑定时对路由不生效(routing-inert)。 + +`reasoning` 用于关闭该模型摘要请求的思考(#2666)。默认会思考的摘要模型会把输出预算花在推理上:回复以 `length` 结束、没有摘要正文,每次折叠都失败。`"off"` 发送协议自身的控制字段 —— `reasoning_effort: "none"`(openai)、`reasoning: {effort: "none"}`(responses)、`thinking: {type: "disabled"}`(anthropic)、`generationConfig.thinkingConfig: {thinkingBudget: 0}`(google)。`"template-off"` 改为发送 `chat_template_kwargs: {enable_thinking: false}`,适用于渲染 chat template 的 openai 兼容服务(vLLM、SGLang、运行 Qwen 系模型的 llama.cpp),要求 `api: "openai"`。该字段需显式开启,因为严格的上游会拒绝不认识的字段;不设时请求体保持不变。 每条链最多 16 个目标。通过 Web API 保存启用的链时会校验每个引用可解析(未知 provider 或 model → HTTP 400)。运行时遇到不可解析的引用会记录一次警告并禁用整条链直至修复 —— 绝不会回退用主模型写摘要。`secret:` 值单独存储在私有的 `billion-context.json.summary-credentials.json` 文件中,不随主 JSON 配置下发,配置 API 也不会返回。Windows 上请用仅管理员的 ACL 保护该文件及其父目录;确认没有代理进程在写存储后,残留的 `.lock` 文件需手工删除。总预算由全部目标与压缩入口共享;取消或会话状态变化会丢弃已生成但未落盘的结果。代理端插件桥(dsh / pi / opencode / codex MCP)会把每次 `compress` 调用的 HTTP 等待时间设为 `totalTimeoutMs` 加少量开销余量 —— 启动时从代理学习得到 —— 因此调高该预算不会再让长折叠被隐藏的 60 秒客户端超时截断(#2652)。 diff --git a/src/agent-providers.ts b/src/agent-providers.ts index a5db2e78c1..1465eba4dc 100644 --- a/src/agent-providers.ts +++ b/src/agent-providers.ts @@ -1,5 +1,5 @@ import type { NamedProviderRecipe } from "./config.js"; -import { validSummaryCredentialName } from "./external-summary-settings.js"; +import { parseSummaryReasoning, validSummaryCredentialName, type SummaryReasoning } from "./external-summary-settings.js"; import { plainHttpTrusted } from "./private-host.js"; import { allowPrivateHttpProviders } from "./knobs.js"; @@ -72,13 +72,13 @@ function parseAgentProviderEntry(name: string, value: unknown): AgentProviderRec if ((url.protocol !== "https:" && !(url.protocol === "http:" && plainHttpTrusted(url, allowPrivateHttpProviders()))) || url.username || url.password || url.hash || url.pathname.includes("/bili/") || url.pathname.startsWith("/__bili/")) throw new Error(`Provider "${name}" baseUrl must be HTTPS (or loopback HTTP) without credentials or proxy recursion`); if (!recipe.models || typeof recipe.models !== "object" || Array.isArray(recipe.models)) throw new Error(`Provider "${name}" needs a models object`); - const models = recipe.models as Record; + const models = recipe.models as Record; const modelEntries = Object.entries(models); if (modelEntries.length === 0 || modelEntries.length > 64) throw new Error(`Provider "${name}" needs 1 to 64 models`); const outModels: NamedProviderRecipe["models"] = {}; for (const [model, knobs] of modelEntries) { if (model.length === 0 || model.length > 200) throw new Error(`Provider "${name}" has an invalid model name`); - const parsed: { contextWindow?: number; outputTokens?: number; stream?: boolean } = {}; + const parsed: { contextWindow?: number; outputTokens?: number; stream?: boolean; reasoning?: SummaryReasoning } = {}; if (knobs.contextWindow !== undefined) { if (typeof knobs.contextWindow !== "number" || !Number.isSafeInteger(knobs.contextWindow) || knobs.contextWindow < 2048 || knobs.contextWindow > 10_000_000) throw new Error(`Provider "${name}" model "${model}" has an invalid contextWindow`); parsed.contextWindow = knobs.contextWindow; @@ -91,6 +91,8 @@ function parseAgentProviderEntry(name: string, value: unknown): AgentProviderRec if (typeof knobs.stream !== "boolean") throw new Error(`Provider "${name}" model "${model}" has an invalid stream flag`); parsed.stream = knobs.stream; } + const reasoning = parseSummaryReasoning(knobs.reasoning, recipe.api, `Provider "${name}" model "${model}"`); + if (reasoning !== undefined) parsed.reasoning = reasoning; outModels[model] = parsed; } return { baseUrl: recipe.baseUrl.replace(/\/+$/, ""), api: recipe.api as NamedProviderRecipe["api"], apiKey: recipe.apiKey, models: outModels }; diff --git a/src/config.ts b/src/config.ts index b338b0955c..a62947155c 100644 --- a/src/config.ts +++ b/src/config.ts @@ -15,7 +15,7 @@ import { parseCompatDropFields } from "./compat-drop.js"; import type { ImageBillingMode } from "./image-tokens.js"; import type { ReasoningGuardConfig } from "./reasoning-guard.js"; import type { OutputSteeringConfig } from "./output-steering.js"; -import { parseExternalSummaryChain, type ExternalSummaryChain, validSummaryCredentialName } from "./external-summary-settings.js"; +import { parseExternalSummaryChain, parseSummaryReasoning, type ExternalSummaryChain, type SummaryReasoning, validSummaryCredentialName } from "./external-summary-settings.js"; export function safeReadJson(path: string): unknown { try { @@ -1140,6 +1140,8 @@ interface NamedProviderModel { contextWindow?: number; outputTokens?: number; stream?: boolean; + /** #2666: summary-lane thinking control (see SummaryReasoning). */ + reasoning?: SummaryReasoning; } /** Dialing recipe on a NAMED providers-table key (pi/opencode idiom): the @@ -1177,12 +1179,13 @@ export function parseNamedProviderRecipe(value: unknown): NamedProviderRecipe { const models: Record = {}; for (const [id, raw] of Object.entries(obj.models)) { const knobs = (raw && typeof raw === "object" && !Array.isArray(raw)) ? raw as Record : {}; - const unknown = Object.keys(knobs).filter((key) => !["contextWindow", "outputTokens", "stream"].includes(key)); + const unknown = Object.keys(knobs).filter((key) => !["contextWindow", "outputTokens", "stream", "reasoning"].includes(key)); if (unknown.length > 0) throw new Error(`unknown fields on recipe model "${id}" [${unknown.join(", ")}]`); if (knobs.contextWindow !== undefined && (typeof knobs.contextWindow !== "number" || !Number.isSafeInteger(knobs.contextWindow) || knobs.contextWindow < 2048 || knobs.contextWindow > 10_000_000)) throw new Error(`recipe model "${id}" contextWindow must be an integer in [2048, 10000000]`); if (knobs.outputTokens !== undefined && (typeof knobs.outputTokens !== "number" || !Number.isSafeInteger(knobs.outputTokens) || knobs.outputTokens < 128)) throw new Error(`recipe model "${id}" outputTokens must be an integer >= 128`); if (knobs.stream !== undefined && typeof knobs.stream !== "boolean") throw new Error(`recipe model "${id}" stream must be a boolean`); - models[id] = { ...(knobs.contextWindow !== undefined ? { contextWindow: knobs.contextWindow } : {}), ...(knobs.outputTokens !== undefined ? { outputTokens: knobs.outputTokens } : {}), ...(knobs.stream !== undefined ? { stream: knobs.stream } : {}) }; + const reasoning = parseSummaryReasoning(knobs.reasoning, obj.api, `recipe model "${id}"`); + models[id] = { ...(knobs.contextWindow !== undefined ? { contextWindow: knobs.contextWindow } : {}), ...(knobs.outputTokens !== undefined ? { outputTokens: knobs.outputTokens } : {}), ...(knobs.stream !== undefined ? { stream: knobs.stream } : {}), ...(reasoning !== undefined ? { reasoning } : {}) }; } return { baseUrl: obj.baseUrl.trim().replace(/\/+$/, ""), api: obj.api, ...(obj.apiKeyEnv !== undefined ? { apiKeyEnv: obj.apiKeyEnv } : {}), ...(obj.credentialRef !== undefined ? { credentialRef: obj.credentialRef } : {}), models }; } diff --git a/src/external-summary-http.ts b/src/external-summary-http.ts index c7d3d83c65..2938e2a389 100644 --- a/src/external-summary-http.ts +++ b/src/external-summary-http.ts @@ -8,6 +8,7 @@ import { maskUrlForLog, maskUrlsInText } from "./log-mask.js"; import { finalizeSseLineEndings } from "./sse-util.js"; import { proxyDispatcher } from "./upstream-proxy.js"; import { PACKAGE_NAME, VERSION } from "./version.js"; +import type { SummaryReasoning } from "./external-summary-settings.js"; /** #3488: bounded retry budget for one candidate call. Total dispatches * (initial + retries). Retries fire ONLY on transient verdicts and must fit @@ -88,6 +89,7 @@ export interface SummaryHttpTarget { readonly stream?: boolean; readonly contextWindow?: number; readonly outputTokens?: number; + readonly reasoning?: SummaryReasoning; readonly proxyUrl?: string; /** #2760: stable conversation id, stamped as x-opencode-session on OpenCode Go targets only. */ readonly sessionId?: string; @@ -184,6 +186,24 @@ function parseSummary(protocol: PreflightProtocol, text: string): string { return summary; } +/** #2666: write the configured thinking control into a summary payload. A + * no-op without the knob, so an unconfigured target's body is unchanged. */ +export function applySummaryReasoning(payload: Record, protocol: PreflightProtocol, reasoning: SummaryReasoning | undefined): void { + if (reasoning === undefined) return; + if (reasoning === "template-off") { + payload.chat_template_kwargs = { ...(record(payload.chat_template_kwargs) ?? {}), enable_thinking: false }; + return; + } + if (protocol === "openai") payload.reasoning_effort = "none"; + else if (protocol === "responses") payload.reasoning = { effort: "none" }; + else if (protocol === "anthropic") payload.thinking = { type: "disabled" }; + else { + const generationConfig = record(payload.generationConfig) ?? {}; + generationConfig.thinkingConfig = { thinkingBudget: 0 }; + payload.generationConfig = generationConfig; + } +} + /** #2760: OpenCode Go (`opencode.ai/zen/go`) requires a stable x-opencode-session * header for routing + prompt cache and answers HTTP 400 MissingSessionID without * one. Matched on host + exact `/zen/go` or `/zen/go/...` so adjacent endpoints @@ -232,6 +252,7 @@ export function createSummaryHttpCandidate(target: SummaryHttpTarget, maxRespons payload[plan.protocol === "responses" ? "max_output_tokens" : "max_tokens"] = plan.outputTokens; } } + applySummaryReasoning(payload, plan.protocol, plan.reasoning); // Reuse the existing codecs without carrying the main request's auth or // model. Bounded retry (#3488): transient verdicts (429/5xx/risk-control // body markers, fail-fast pre-response network deaths) re-dispatch up to diff --git a/src/external-summary-settings.ts b/src/external-summary-settings.ts index 2d9b7c0b84..8a4a046eef 100644 --- a/src/external-summary-settings.ts +++ b/src/external-summary-settings.ts @@ -5,6 +5,27 @@ import type { PreflightProtocol } from "./preflight.js"; import { plainHttpTrusted } from "./private-host.js"; import { allowPrivateHttpProviders } from "./knobs.js"; +/** #2666: summary-lane thinking control, per recipe model. A summary target + * that thinks by default spends its output budget on reasoning (finish + * reason "length", empty content) and every fold fails. Opt-in, because + * strict upstreams reject fields they do not know: + * - "off": the protocol's own control (openai `reasoning_effort: "none"`, + * responses `reasoning.effort: "none"`, anthropic `thinking.type: + * "disabled"`, google `thinkingConfig.thinkingBudget: 0`); + * - "template-off": `chat_template_kwargs.enable_thinking: false`, for + * openai-compatible servers that render a chat template (vLLM, SGLang, + * llama.cpp serving Qwen-family models). openai api only. */ +export type SummaryReasoning = "off" | "template-off"; + +/** Validate a `reasoning` knob against the target's protocol. THROWS with + * `where` naming the offending entry; undefined means "send nothing". */ +export function parseSummaryReasoning(value: unknown, protocol: string, where: string): SummaryReasoning | undefined { + if (value === undefined) return undefined; + if (value !== "off" && value !== "template-off") throw new Error(`${where} reasoning must be "off" or "template-off"`); + if (value === "template-off" && protocol !== "openai") throw new Error(`${where} reasoning "template-off" needs api "openai" (it sets chat_template_kwargs)`); + return value; +} + interface ExternalSummaryTarget { name: string; protocol: PreflightProtocol; @@ -20,6 +41,7 @@ interface ExternalSummaryTarget { contextWindow: number; outputTokens: number; stream: boolean; + reasoning?: SummaryReasoning; } export interface ExternalSummarySettings { @@ -210,9 +232,10 @@ export function expandExternalSummaryChain(chain: ExternalSummaryChain, recipes: const outputTokens = integer(knobs.outputTokens, Math.min(8192, Math.floor(contextWindow / 4)), 128, contextWindow - 1); if (knobs.stream !== undefined && typeof knobs.stream !== "boolean") throw new Error("External summary stream must be boolean"); const stream = knobs.stream === true; + const reasoning = parseSummaryReasoning(knobs.reasoning, recipe.api, `External summary target "${ref}"`); const inlineKey = recipe.apiKey; const credentialRef = recipe.apiKeyEnv ? `env:${recipe.apiKeyEnv}` : `secret:${recipe.credentialRef}`; - return { name, protocol: recipe.api, url: derivedSummaryEndpoint(recipe, model, stream), model, ...(inlineKey !== undefined ? { apiKey: inlineKey } : { credentialRef }), contextWindow, outputTokens, stream }; + return { name, protocol: recipe.api, url: derivedSummaryEndpoint(recipe, model, stream), model, ...(inlineKey !== undefined ? { apiKey: inlineKey } : { credentialRef }), contextWindow, outputTokens, stream, ...(reasoning !== undefined ? { reasoning } : {}) }; }); return { enabled: true, targets, budget: chain.budget ?? SUMMARY_DEFAULT_BUDGET , ...(chain.concurrency !== undefined ? { concurrency: chain.concurrency } : {}), ...parseAutoFold(chain as unknown as Record) }; @@ -263,7 +286,7 @@ export function parseExternalSummarySettings(value: unknown, options: { inlineKe const names = new Set(); const targets = values.map((value): ExternalSummaryTarget => { const target = object(value); - knownKeys(target, inlineKeys ? ["name", "protocol", "url", "model", "credentialRef", "apiKey", "contextWindow", "outputTokens", "stream"] : ["name", "protocol", "url", "model", "credentialRef", "contextWindow", "outputTokens", "stream"]); + knownKeys(target, inlineKeys ? ["name", "protocol", "url", "model", "credentialRef", "apiKey", "contextWindow", "outputTokens", "stream", "reasoning"] : ["name", "protocol", "url", "model", "credentialRef", "contextWindow", "outputTokens", "stream", "reasoning"]); const name = text(target.name, 64); if (!validSummaryCredentialName(name) || names.has(name)) throw new Error("External summary target names must be unique identifiers"); names.add(name); @@ -292,7 +315,8 @@ export function parseExternalSummarySettings(value: unknown, options: { inlineKe const suffix = target.stream === true ? "streamGenerateContent" : "generateContent"; if (!url.pathname.endsWith(`/models/${encodeURIComponent(model.replace(/^models\//, ""))}:${suffix}`)) throw new Error("Google summary endpoint must match the model and stream mode"); } - return { name, protocol, url: url.href, model, ...(inlineKey !== undefined ? { apiKey: inlineKey } : { credentialRef }), contextWindow, outputTokens, stream: target.stream === true }; + const reasoning = parseSummaryReasoning(target.reasoning, protocol, "External summary target"); + return { name, protocol, url: url.href, model, ...(inlineKey !== undefined ? { apiKey: inlineKey } : { credentialRef }), contextWindow, outputTokens, stream: target.stream === true, ...(reasoning !== undefined ? { reasoning } : {}) }; }); return { enabled: true, targets, budget: parseBudget(settings.budget), ...parseConcurrency(settings), ...parseAutoFold(settings) }; } diff --git a/tests/issue2666-summary-reasoning.test.ts b/tests/issue2666-summary-reasoning.test.ts new file mode 100644 index 0000000000..0a21e1e221 --- /dev/null +++ b/tests/issue2666-summary-reasoning.test.ts @@ -0,0 +1,140 @@ +// #2666: a summary target that thinks by default spent its whole output budget +// on reasoning (finish_reason "length", empty content), so every fold failed and +// nothing in config could turn it off. The per-model recipe knob `reasoning` +// ("off" | "template-off") writes the protocol's thinking control into the +// summary request; without the knob the request body is unchanged. +import assert from "node:assert/strict"; +import http from "node:http"; +import { test } from "node:test"; +import { ExternalSummaryExecutor, type SummaryWork } from "../src/external-summary.ts"; +import { applySummaryReasoning, createSummaryHttpCandidate, type SummaryHttpTarget } from "../src/external-summary-http.ts"; +import { expandExternalSummaryChain, parseExternalSummarySettings, parseSummaryReasoning, type SummaryReasoning } from "../src/external-summary-settings.ts"; +import { parseNamedProviderRecipe } from "../src/config.ts"; +import { parseAgentProviderReport } from "../src/agent-providers.ts"; +import type { PreflightProtocol } from "../src/preflight.ts"; + +process.env.NODE_ENV = "test"; +const work: SummaryWork = { content: "Prior evidence and decisions", reference: "Read-only current task", instructions: "Preserve unfinished objectives" }; +const budget = { totalTimeoutMs: 4000, targetTimeoutMs: 1500, maxSummaryBytes: 2048 }; +const summary = "Retained decisions, evidence, constraints and unfinished objectives."; +const THINKING_FIELDS = ["reasoning_effort", "reasoning", "thinking", "chat_template_kwargs"]; + +function completion(protocol: PreflightProtocol): object { + if (protocol === "responses") return { status: "completed", output_text: summary }; + if (protocol === "anthropic") return { stop_reason: "end_turn", content: [{ type: "text", text: summary }] }; + if (protocol === "openai") return { choices: [{ message: { content: summary }, finish_reason: "stop" }] }; + return { candidates: [{ content: { parts: [{ text: summary }] }, finishReason: "STOP" }] }; +} + +/** Send one summary request through the real candidate and return the body the upstream received. */ +async function capturedBody(protocol: PreflightProtocol, reasoning?: SummaryReasoning): Promise> { + let body: Record | undefined; + const server = http.createServer((req, res) => { + void (async () => { + const chunks: Buffer[] = []; + for await (const chunk of req) chunks.push(Buffer.from(chunk)); + body = JSON.parse(Buffer.concat(chunks).toString("utf8")) as Record; + res.setHeader("content-type", "application/json"); + res.end(JSON.stringify(completion(protocol))); + })().catch(() => { res.writeHead(500); res.end(); }); + }); + await new Promise((resolve, reject) => { server.once("error", reject); server.listen(0, "127.0.0.1", resolve); }); + const address = server.address(); + assert.ok(address && typeof address !== "string"); + try { + const target: SummaryHttpTarget = { + url: `http://127.0.0.1:${address.port}`, protocol, model: "external-test-model", + headers: { authorization: "Bearer test-summary-key" }, outputTokens: 1024, ...(reasoning !== undefined ? { reasoning } : {}), + }; + const result = await new ExternalSummaryExecutor(1).execute(work, [createSummaryHttpCandidate(target, 4096)], budget); + assert.equal(result.status, "success"); + } finally { + await new Promise((resolve) => { server.close(() => resolve()); server.closeAllConnections(); }); + } + assert.ok(body, "upstream received a request"); + return body; +} + +const OFF_EXPECTED: Record) => void> = { + openai: (body) => assert.equal(body.reasoning_effort, "none"), + responses: (body) => assert.deepEqual(body.reasoning, { effort: "none" }), + anthropic: (body) => assert.deepEqual(body.thinking, { type: "disabled" }), + google: (body) => { + const generationConfig = body.generationConfig as Record; + assert.deepEqual(generationConfig.thinkingConfig, { thinkingBudget: 0 }); + assert.equal(generationConfig.maxOutputTokens, 1024, "the output budget survives next to the thinking control"); + }, +}; + +for (const protocol of ["openai", "responses", "anthropic", "google"] as const) { + test(`#2666: ${protocol} — reasoning "off" sends the protocol's own thinking control`, async () => { + OFF_EXPECTED[protocol](await capturedBody(protocol, "off")); + }); + + test(`#2666: ${protocol} — no knob, no thinking field (body unchanged)`, async () => { + const body = await capturedBody(protocol); + for (const field of THINKING_FIELDS) assert.equal(body[field], undefined, `unconfigured ${protocol} body must not carry ${field}`); + if (protocol === "google") assert.equal((body.generationConfig as Record).thinkingConfig, undefined); + }); +} + +test("#2666: openai — reasoning \"template-off\" sends chat_template_kwargs, not reasoning_effort", async () => { + const body = await capturedBody("openai", "template-off"); + assert.deepEqual(body.chat_template_kwargs, { enable_thinking: false }); + assert.equal(body.reasoning_effort, undefined); +}); + +test("#2666: applySummaryReasoning keeps existing chat_template_kwargs and is a no-op without the knob", () => { + const payload: Record = { model: "m", chat_template_kwargs: { custom: 1 } }; + applySummaryReasoning(payload, "openai", "template-off"); + assert.deepEqual(payload.chat_template_kwargs, { custom: 1, enable_thinking: false }); + const untouched: Record = { model: "m", max_tokens: 10 }; + applySummaryReasoning(untouched, "openai", undefined); + assert.deepEqual(untouched, { model: "m", max_tokens: 10 }); +}); + +test("#2666: parseSummaryReasoning accepts the two values and rejects the rest", () => { + assert.equal(parseSummaryReasoning(undefined, "openai", "t"), undefined); + assert.equal(parseSummaryReasoning("off", "anthropic", "t"), "off"); + assert.equal(parseSummaryReasoning("template-off", "openai", "t"), "template-off"); + assert.throws(() => parseSummaryReasoning("none", "openai", "t"), /reasoning must be "off" or "template-off"/); + assert.throws(() => parseSummaryReasoning(false, "openai", "t"), /reasoning must be/); + for (const protocol of ["anthropic", "responses", "google"]) { + assert.throws(() => parseSummaryReasoning("template-off", protocol, "t"), /needs api "openai"/); + } +}); + +const recipeInput = (models: Record, api = "openai") => ({ baseUrl: "https://summary.example/v1", api, apiKeyEnv: "SUMMARY_KEY", models }); + +test("#2666: named recipe — reasoning reaches the expanded chain target", () => { + const recipe = parseNamedProviderRecipe(recipeInput({ thinker: { outputTokens: 2048, reasoning: "template-off" }, plain: { outputTokens: 2048 } })); + assert.equal(recipe.models.thinker.reasoning, "template-off"); + assert.equal("reasoning" in recipe.models.plain, false); + const expanded = expandExternalSummaryChain({ enabled: true, targets: ["local/thinker", "local/plain"] }, { local: recipe }); + assert.equal(expanded.targets[0].reasoning, "template-off"); + assert.equal("reasoning" in expanded.targets[1], false, "an unconfigured model stays unconfigured"); +}); + +test("#2666: named recipe — invalid reasoning values are rejected", () => { + assert.throws(() => parseNamedProviderRecipe(recipeInput({ m: { reasoning: "minimal" } })), /reasoning must be/); + assert.throws(() => parseNamedProviderRecipe(recipeInput({ m: { reasoning: "template-off" } }, "anthropic")), /needs api "openai"/); + assert.throws(() => parseNamedProviderRecipe(recipeInput({ m: { thinking: "off" } })), /unknown fields on recipe model/, "other keys stay unknown"); +}); + +test("#2666: request-rail settings accept reasoning and validate it against the protocol", () => { + const base = { name: "local", protocol: "openai", url: "https://summary.example/v1/chat/completions", model: "m", credentialRef: "env:SUMMARY_KEY" }; + const parsed = parseExternalSummarySettings({ enabled: true, targets: [{ ...base, reasoning: "template-off" }] }); + assert.equal(parsed.targets[0].reasoning, "template-off"); + assert.equal("reasoning" in parseExternalSummarySettings({ enabled: true, targets: [base] }).targets[0], false); + assert.throws(() => parseExternalSummarySettings({ enabled: true, targets: [{ ...base, protocol: "anthropic", url: "https://summary.example/v1/messages", reasoning: "template-off" }] }), /needs api "openai"/); +}); + +test("#2666: agent-registry recipes carry reasoning too; a bad value skips that provider", () => { + const report = parseAgentProviderReport({ agent: "pi", providers: { + good: { baseUrl: "https://summary.example/v1", api: "openai", apiKey: "k", models: { m: { reasoning: "off" } } }, + bad: { baseUrl: "https://summary.example/v1", api: "google", apiKey: "k", models: { m: { reasoning: "template-off" } } }, + } }); + assert.equal(report.registered.good.models.m.reasoning, "off"); + assert.equal(report.registered.bad, undefined); + assert.equal(report.skipped[0]?.name, "bad"); +});