Skip to content

[PR] fix: per-model reasoning knob turns thinking off on the summary lane (#2666) - #4463

Closed
naerymdan wants to merge 2 commits into
ranxianglei:masterfrom
naerymdan:2026-10-24_summary-reasoning-knob
Closed

naerymdan wants to merge 2 commits into
ranxianglei:masterfrom
naerymdan:2026-10-24_summary-reasoning-knob

Conversation

@naerymdan

Copy link
Copy Markdown
Contributor

Closes #2666

Needs owner sign-off on the config surface (AGENTS.md, "Configuration Surface Discipline"). PR #2806 said #2666 "needs a typed reasoning knob". This PR is a concrete proposal of that knob; the mapping is below. If you'd rather have a different shape, say which and I'll rework it.

What

A summary target that thinks by default spends its output budget on reasoning: the reply has finish reason length and empty content, and every fold fails. summaryPayload() sends only model, max_tokens, messages and stream, so nothing in config could turn thinking off.

Recipe models now accept reasoning:

value wire
"off" openai reasoning_effort: "none" · responses reasoning: {effort: "none"} · anthropic thinking: {type: "disabled"} · google generationConfig.thinkingConfig: {thinkingBudget: 0}
"template-off" chat_template_kwargs: {enable_thinking: false} (existing keys merged), api: "openai" only. This is for servers that render a chat template (vLLM, SGLang, llama.cpp serving Qwen-family models), which ignore reasoning_effort.
  • It's validated in three places: the config-file recipe parser (parseNamedProviderRecipe), the agent-registry parser (parseAgentProviderReport), and the request rail (parseExternalSummarySettings), via the shared parseSummaryReasoning.
  • Chain expansion carries it to the target. applySummaryReasoning() writes it after the output budget.
  • Unset means the request body is byte-identical to today.

Config surface

  • Added: an optional providers.<name>.models.<model>.reasoning: "off" | "template-off" on named recipes, the same per-model level as the existing contextWindow / outputTokens / stream. Also accepted on agent-registry recipes and on the expanded rail target, since both carry the same per-target knobs.
  • Why the existing surface can't express it:
    • compat.dropFields can only delete fields, not add them.
    • compress.reasoning governs dropping finished reasoning from history, a different concern.
    • pi.subagents.thinkingLevel only covers the sub-agent lane.
    • Env vars are process-global, but this has to be per target: a chain can mix a thinking target with a plain one, and mix protocols.
    • The three-level compress.externalSummary block is whole-chain-replace, so a chain-level knob would hit every target and every protocol at once.
    • The recipe model entry is where per-target summary knobs already live.
  • Typed, not a free-form extraBody. An arbitrary body passthrough would be a much wider surface, and harder to validate.
  • Two values: strict upstreams reject unknown fields, and "thinking off" is spelled differently by native APIs and chat-template servers. Both spellings were checked against a real server (below).
  • Compat: purely additive. Existing configs parse unchanged. A config that uses reasoning on an older bili fails that recipe's parse ("unknown fields on recipe model"), and loadNamedProviders already reports that loudly. The launcher↔extension channel is unchanged: pi doesn't send reasoning in its provider report yet (possible follow-up).

Measured

From my #2666 comment: summaries on a local gufo 0.22.0 / Qwen3.8 Flash-Next.

  • outputTokens 256: 92 of 92 summary calls ended finish_reason=length in reasoning, and 0 folds landed.
  • outputTokens 2048: 10 of 15 still ended length, at ~40 s each.

On the same server, reasoning_effort: "none", chat_template_kwargs: {enable_thinking: false} and thinking: {type: "disabled"} each return a direct answer with 0 reasoning characters, against 179 without.

Tests

  • New tests/issue2666-summary-reasoning.test.ts: 15 cases.
    • For every protocol, "off" writes the native field into the body the upstream actually receives, and no knob means no thinking field.
    • "template-off" sets chat_template_kwargs and not reasoning_effort, merging existing keys.
    • Validation rejects unknown values, and rejects template-off on non-openai apis.
    • Named recipe → chain-expansion → target propagation works, and an unconfigured model stays unconfigured.
    • Rail settings work, and an agent-registry entry with a bad value is skipped.
  • npm run typecheck, npm run build and check:unused-exports are clean.
  • In my sandbox, 5 tests in e2e/e2e-native-pi, issue3157-persist-durability and launcher-plugin-mode fail; they fail the same way on unmodified master there. That's environment-only, so the full-suite verdict is CI's.
  • No kernel/** change.

…anxianglei#2666)

A summary target that thinks by default spends its output budget on
reasoning: finish reason "length", empty content, and every fold fails.
summaryPayload() sends only model / max_tokens / messages / stream and no
config could add a reasoning control (PR ranxianglei#2806: "it needs a typed
reasoning knob").

Recipe models (config-file named recipes and agent-registry recipes) take
`reasoning`:
- "off": the protocol's own control -- openai reasoning_effort "none",
  responses reasoning.effort "none", anthropic thinking.type "disabled",
  google generationConfig.thinkingConfig.thinkingBudget 0;
- "template-off": chat_template_kwargs.enable_thinking false, openai api
  only, for chat-template servers (vLLM, SGLang, llama.cpp + Qwen).

Validated at recipe parse, chain expansion and on the request rail;
applySummaryReasoning() writes it after the output budget. Unset leaves the
body byte-identical. Opt-in because strict upstreams reject unknown fields.

Closes ranxianglei#2666
@ranxianglei ranxianglei changed the title fix: per-model reasoning knob turns thinking off on the summary lane (#2666) [PR] fix: per-model reasoning knob turns thinking off on the summary lane (#2666) Oct 24, 2026
@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重: 不重复。 This is the concrete implementation of the knob requested in #2666 (your floor-5/6 comments there ask for exactly this shape, and the earlier triage there recommended a typed per-model knob over free-form extraBody). Related open work checked:

标签: pr · scope/wiring(摘要请求管道 + 配置解析面)· p2(能用但思考型目标上摘要基本失效,有绕行——换非思考模型)。功能请求不打 bug。

Full review (direction vs #2666, regression surface, code/tests, diff cleanliness) is in progress — I'll post the structured verdict next. Note the PR body asks for owner sign-off on the config surface; my review will state whether the proposed shape holds up against the existing providers/recipe system.

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

PR #4463 审核结论(独立审读 diff + master 代码 + 本地定向验证)

1. 是什么问题:外部摘要链(summary lane)发出的请求体里没有任何思考控制字段。目标模型默认会思考时(Qwen/DeepSeek/GLM thinking 等),整个输出预算被 reasoning 吃掉——finish_reason=length、正文为空——每次折叠都失败,而配置面没有任何入口能关掉它。

2. 什么原因导致的:能力缺失,不是回归。master@e518363(v0.1.205)上 src/preflight.ts 的 summaryPayload 只写 max_tokens/max_output_tokens/generationConfig.maxOutputTokens;三个配置入口(parseNamedProviderRecipe、agent-registry 解析、request-rail settings)的模型字段白名单都没有 reasoning 槽位。与 naerymdan 在 0.1.205 的复现一致——机制确实不存在。修法打在决策点(config 解析 → target plan → payload),不是传输层打掩码;与 issue 楼层 2 triage 推荐的「typed per-model knob」方向一致(优于 free-form extraBody)。

3. 影响范围与严重程度:仅配置了 external summary 且目标模型默认思考的用户;该会话折叠整体不可用(主模型流量不受影响)。有绕行(换非思考模型、调大 outputTokens 碰运气)→ p2,标签已打。

4. 怎么修复的:新增 opt-in 类型化旋钮 reasoning: "off" | "template-off"。三个配置入口共用同一个校验器 parseSummaryReasoning(坏值直接拒绝、template-off 强制 api=openai);payload 构建后由 applySummaryReasoning 写入协议原生控制字段(openai reasoning_effort:"none" / responses reasoning:{effort:"none"} / anthropic thinking:{type:"disabled"} / google thinkingConfig.thinkingBudget:0;template-off → chat_template_kwargs.enable_thinking:false)。不设 = 请求体逐字节不变(早退 + 解析侧省略字段,双向都有测试钉住)。

5. 是否有回归问题:逐项清点过,无回归——

6. 是否破坏缓存:不涉及(不碰前缀缓存、会话存储、reconcile)。

7. 是否改了设计方向:不涉及(纯增量配置面,沿用既有 recipe model 字段模式)。

8. 是否引入了新的配置:是——recipe model 条目(providers 表配方 + agent-registry + request-rail targets)新增 reasoning("off"|"template-off"),默认不设=现状;template-off 要求 api:"openai" 否则报错。CONFIGURATION.md 中英同步更新,CHANGELOG [Unreleased] 条目按惯例放置。PR 正文含完整的 config surface 对照节。

验证证据(本地定向,全量判定以 CI 为准):PR head 上 npm run typecheck 干净;定向测试 186/186 通过(新 15 例为真行为测试——真实 HTTP server 随机端口 + 真 ExternalSummaryExecutor/createSummaryHttpCandidate 入口,捕获上游实收 body,每协议正+负向,校验在三入口分别钉死;外加 external-summary-http/settings/runtime/executor、agent-providers、named-provider-bind、config-surface-guard 全套邻域回归)。config-surface-guard、unused-exports 两个 CI gate 本地预跑均绿。PR 目前尚无 check runs。

判定:代码侧审核通过,可合入条件 = owner 对 config surface 拍板 + CI 转绿。 按 owner-gated 规则这是新配置字段,需要你一句「批准」;作者已在 #2666 楼层 6 明确请求签核。不合 auto-merge 门槛(new capability surface + config schema),人工合并。未发现需要顺手修的小问题。

@ranxianglei

Copy link
Copy Markdown
Owner

(ai: zhipuai-lb/glm-5.3)

陈旧票清理(第三批动作①):目标 issue #2666 已关。为避免重复劳动关闭本 PR;如判断有误请直接 reopen 并留言。

感谢 @naerymdan 的贡献与排查——这条修复路径已被其他方式覆盖;欢迎认领其他 open issue。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

2 participants