Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ Versions follow the merge of a `*_release-v*` branch; CI publishes to npm on tag

### Fixed

- **External summary: per-model `reasoning` knob turns thinking off on the summary lane (#2666).** A summary target that thinks by default (Qwen, DeepSeek, GLM thinking …) spent its output budget on reasoning — finish reason `length`, empty content — and every fold failed; bili sent no reasoning control and no config could add one. Recipe models (config-file and agent-registry) now accept `reasoning: "off"` (protocol-native: openai `reasoning_effort: "none"`, responses `reasoning.effort: "none"`, anthropic `thinking.disabled`, google `thinkingBudget: 0`) or `"template-off"` (`chat_template_kwargs.enable_thinking: false`, openai api only, for vLLM/SGLang/llama.cpp chat-template servers). Unset leaves the body unchanged. New tests/issue2666-summary-reasoning.test.ts (15 cases, every protocol positive + negative).
- **External summary: stop injecting the summary-writing rules into the conversation prompt (#4460).** With `compress.externalSummary.enabled`, the conversation model only selects ranges and the independent service writes the committed summary — yet every conversation request still carried `HOW TO COMPRESS`/`KEEP VERBATIM`/`DROP`/`PRIORITY` (~5.8K chars) plus the per-summary length budget, which the model's non-authoritative hint is never checked against. The conversation-side prompt now drops both while the chain is enabled (all five prompt assembly sites, function and hybrid builders, pack overrides included); philosophy, ACP tags, tools, summaries-in-context and marker integrity stay. The summary requests keep the full rules (`session.meta.summaryInstructions`), and classic mode is byte-identical. New tests/issue4460-external-summary-conversation-prompt.test.ts (7 cases, positive + negative).
- **增量车道:拉取回执修复 ack 传输盲区 + 漂移回退取富侧(#4352/#4323)**。2026-10-22 事故双根因修复:① pi 的 after_provider_response 只在部分 provider 传输下暴露响应头,x-bili-applied-seq 丢失时 noteIncrementalAck 永不执行,后续每轮都整包 re-seed(sinceSeq=0 重发 21K 条历史)。现在 GET /__bili/plugin/turn?conversation=<id> 返回热 canonical 的 {appliedSeq, boundary}(boundary=末条 wire 消息哈希,冷重建诚实返回 null),pi 扩展在规划下一轮前拉取回执,仅在 boundary 证明 canonical 吸收了我方末条消息时采纳——绝不按 seq 前进盲目采纳(外来的全量重发也会推进 canonical);② #4323 多实例写同一目录时,漂移回退无条件吃旧信封,把索引里更新的折叠世代(26,971 refs/16 active)换成旧世代(11,849/12),覆盖与 canonical 失配→0 可压区间→preflight 8.29M 假性死局 502。现在漂移两侧按 mapped ref 数取富侧(平局信封赢,平局语义=原子整文件)。新增 tests/incremental-ack-pull.test.ts 3 例(响应头 ack 回归钉/拉取采纳/冷 boundary null 拒采纳)+ storage-full-flip 4b(陈旧信封让位新索引世代)。

Expand Down
4 changes: 3 additions & 1 deletion CONFIGURATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -1261,7 +1261,9 @@ Targets are **references into the `providers` table**: each entry is the string
}
```

A recipe carries: `baseUrl` (HTTPS; loopback HTTP allowed for local development; no embedded credentials, proxy-recursion paths, or arbitrary query parameters), `api` (one of `openai` | `anthropic` | `responses` | `google`; selects the wire protocol and derives the request path from `baseUrl`), exactly one credential reference — `apiKeyEnv: "VAR_NAME"` (an environment variable read at call time) or `credentialRef: "NAME"` (a value stored via the Web UI) — and a `models` map whose per-model fields are `contextWindow` (default 128000), `outputTokens` (default `min(8192, window/4)`), and `stream` (default false). Recipes can also be split into `recipe`/`bind` routing form like URL entries; a named entry is routing-inert unless bound.
A recipe carries: `baseUrl` (HTTPS; loopback HTTP allowed for local development; no embedded credentials, proxy-recursion paths, or arbitrary query parameters), `api` (one of `openai` | `anthropic` | `responses` | `google`; selects the wire protocol and derives the request path from `baseUrl`), exactly one credential reference — `apiKeyEnv: "VAR_NAME"` (an environment variable read at call time) or `credentialRef: "NAME"` (a value stored via the Web UI) — and a `models` map whose per-model fields are `contextWindow` (default 128000), `outputTokens` (default `min(8192, window/4)`), `stream` (default false), and `reasoning` (unset by default; see below). Recipes can also be split into `recipe`/`bind` routing form like URL entries; a named entry is routing-inert unless bound.

`reasoning` turns thinking off on that model's summary requests (#2666). A summary model that thinks by default spends its output budget on reasoning: the reply ends with finish reason `length` and no summary text, and every fold fails. `"off"` sends the protocol's own control — `reasoning_effort: "none"` (openai), `reasoning: {effort: "none"}` (responses), `thinking: {type: "disabled"}` (anthropic), `generationConfig.thinkingConfig: {thinkingBudget: 0}` (google). `"template-off"` sends `chat_template_kwargs: {enable_thinking: false}` instead, for openai-compatible servers that render a chat template (vLLM, SGLang, llama.cpp serving Qwen-family models); it requires `api: "openai"`. The knob is opt-in because strict upstreams reject fields they do not know; an unset knob leaves the request body unchanged.

Up to 16 targets per chain are supported. Saving an enabled chain through the Web API validates that every reference resolves (unknown provider or model → HTTP 400). At runtime an unresolvable reference is logged once and disables the chain until fixed — it never falls back to summarizing with the main model. `secret:` values are stored separately from the main JSON configuration in the private `billion-context.json.summary-credentials.json` file and are never returned by the configuration API. On Windows, protect this file and its parent directory with an administrator-only ACL; stale `.lock` files require manual removal after confirming no proxy process is writing the store. The total budget is shared across all targets and compression entry points, and cancellation or session-state changes discard generated results without folding. The agent-side plugin bridge (dsh / pi / opencode / codex MCP) sizes each `compress` call's HTTP wait to `totalTimeoutMs` plus a small overhead margin — learned from the proxy at bootstrap — so raising this budget no longer caps long folds at a hidden 60-second client-side timeout (#2652).

Expand Down
4 changes: 3 additions & 1 deletion CONFIGURATION.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -1264,7 +1264,9 @@ URL 键通道(或带 `bind` 的命名条目)可以向上游发送**自己的
}
```

配方包含:`baseUrl`(必须 HTTPS,本地开发可用回环 HTTP;拒绝内嵌凭据、代理递归路径与任意 query 参数);`api`(`openai` | `anthropic` | `responses` | `google` 四选一,决定线上协议并从 `baseUrl` 推导请求路径);恰好一个凭据引用 —— `apiKeyEnv: "变量名"`(调用时读环境变量)或 `credentialRef: "名字"`(通过 Web UI 存储的值);以及 `models` 表,每个模型可设 `contextWindow`(默认 128000)、`outputTokens`(默认 `min(8192, 窗口/4)`)、`stream`(默认 false)。配方也可以像 URL 条目一样拆成 `recipe`/`bind` 路由形态;具名条目未绑定时对路由不生效(routing-inert)。
配方包含:`baseUrl`(必须 HTTPS,本地开发可用回环 HTTP;拒绝内嵌凭据、代理递归路径与任意 query 参数);`api`(`openai` | `anthropic` | `responses` | `google` 四选一,决定线上协议并从 `baseUrl` 推导请求路径);恰好一个凭据引用 —— `apiKeyEnv: "变量名"`(调用时读环境变量)或 `credentialRef: "名字"`(通过 Web UI 存储的值);以及 `models` 表,每个模型可设 `contextWindow`(默认 128000)、`outputTokens`(默认 `min(8192, 窗口/4)`)、`stream`(默认 false)、`reasoning`(默认不设,见下)。配方也可以像 URL 条目一样拆成 `recipe`/`bind` 路由形态;具名条目未绑定时对路由不生效(routing-inert)。

`reasoning` 用于关闭该模型摘要请求的思考(#2666)。默认会思考的摘要模型会把输出预算花在推理上:回复以 `length` 结束、没有摘要正文,每次折叠都失败。`"off"` 发送协议自身的控制字段 —— `reasoning_effort: "none"`(openai)、`reasoning: {effort: "none"}`(responses)、`thinking: {type: "disabled"}`(anthropic)、`generationConfig.thinkingConfig: {thinkingBudget: 0}`(google)。`"template-off"` 改为发送 `chat_template_kwargs: {enable_thinking: false}`,适用于渲染 chat template 的 openai 兼容服务(vLLM、SGLang、运行 Qwen 系模型的 llama.cpp),要求 `api: "openai"`。该字段需显式开启,因为严格的上游会拒绝不认识的字段;不设时请求体保持不变。

每条链最多 16 个目标。通过 Web API 保存启用的链时会校验每个引用可解析(未知 provider 或 model → HTTP 400)。运行时遇到不可解析的引用会记录一次警告并禁用整条链直至修复 —— 绝不会回退用主模型写摘要。`secret:` 值单独存储在私有的 `billion-context.json.summary-credentials.json` 文件中,不随主 JSON 配置下发,配置 API 也不会返回。Windows 上请用仅管理员的 ACL 保护该文件及其父目录;确认没有代理进程在写存储后,残留的 `.lock` 文件需手工删除。总预算由全部目标与压缩入口共享;取消或会话状态变化会丢弃已生成但未落盘的结果。代理端插件桥(dsh / pi / opencode / codex MCP)会把每次 `compress` 调用的 HTTP 等待时间设为 `totalTimeoutMs` 加少量开销余量 —— 启动时从代理学习得到 —— 因此调高该预算不会再让长折叠被隐藏的 60 秒客户端超时截断(#2652)。

Expand Down
8 changes: 5 additions & 3 deletions src/agent-providers.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import type { NamedProviderRecipe } from "./config.js";
import { validSummaryCredentialName } from "./external-summary-settings.js";
import { parseSummaryReasoning, validSummaryCredentialName, type SummaryReasoning } from "./external-summary-settings.js";
import { plainHttpTrusted } from "./private-host.js";
import { allowPrivateHttpProviders } from "./knobs.js";

Expand Down Expand Up @@ -72,13 +72,13 @@ function parseAgentProviderEntry(name: string, value: unknown): AgentProviderRec
if ((url.protocol !== "https:" && !(url.protocol === "http:" && plainHttpTrusted(url, allowPrivateHttpProviders()))) || url.username || url.password || url.hash
|| url.pathname.includes("/bili/") || url.pathname.startsWith("/__bili/")) throw new Error(`Provider "${name}" baseUrl must be HTTPS (or loopback HTTP) without credentials or proxy recursion`);
if (!recipe.models || typeof recipe.models !== "object" || Array.isArray(recipe.models)) throw new Error(`Provider "${name}" needs a models object`);
const models = recipe.models as Record<string, { contextWindow?: unknown; outputTokens?: unknown; stream?: unknown }>;
const models = recipe.models as Record<string, { contextWindow?: unknown; outputTokens?: unknown; stream?: unknown; reasoning?: unknown }>;
const modelEntries = Object.entries(models);
if (modelEntries.length === 0 || modelEntries.length > 64) throw new Error(`Provider "${name}" needs 1 to 64 models`);
const outModels: NamedProviderRecipe["models"] = {};
for (const [model, knobs] of modelEntries) {
if (model.length === 0 || model.length > 200) throw new Error(`Provider "${name}" has an invalid model name`);
const parsed: { contextWindow?: number; outputTokens?: number; stream?: boolean } = {};
const parsed: { contextWindow?: number; outputTokens?: number; stream?: boolean; reasoning?: SummaryReasoning } = {};
if (knobs.contextWindow !== undefined) {
if (typeof knobs.contextWindow !== "number" || !Number.isSafeInteger(knobs.contextWindow) || knobs.contextWindow < 2048 || knobs.contextWindow > 10_000_000) throw new Error(`Provider "${name}" model "${model}" has an invalid contextWindow`);
parsed.contextWindow = knobs.contextWindow;
Expand All @@ -91,6 +91,8 @@ function parseAgentProviderEntry(name: string, value: unknown): AgentProviderRec
if (typeof knobs.stream !== "boolean") throw new Error(`Provider "${name}" model "${model}" has an invalid stream flag`);
parsed.stream = knobs.stream;
}
const reasoning = parseSummaryReasoning(knobs.reasoning, recipe.api, `Provider "${name}" model "${model}"`);
if (reasoning !== undefined) parsed.reasoning = reasoning;
outModels[model] = parsed;
}
return { baseUrl: recipe.baseUrl.replace(/\/+$/, ""), api: recipe.api as NamedProviderRecipe["api"], apiKey: recipe.apiKey, models: outModels };
Expand Down
9 changes: 6 additions & 3 deletions src/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ import { parseCompatDropFields } from "./compat-drop.js";
import type { ImageBillingMode } from "./image-tokens.js";
import type { ReasoningGuardConfig } from "./reasoning-guard.js";
import type { OutputSteeringConfig } from "./output-steering.js";
import { parseExternalSummaryChain, type ExternalSummaryChain, validSummaryCredentialName } from "./external-summary-settings.js";
import { parseExternalSummaryChain, parseSummaryReasoning, type ExternalSummaryChain, type SummaryReasoning, validSummaryCredentialName } from "./external-summary-settings.js";

export function safeReadJson(path: string): unknown {
try {
Expand Down Expand Up @@ -1140,6 +1140,8 @@ interface NamedProviderModel {
contextWindow?: number;
outputTokens?: number;
stream?: boolean;
/** #2666: summary-lane thinking control (see SummaryReasoning). */
reasoning?: SummaryReasoning;
}

/** Dialing recipe on a NAMED providers-table key (pi/opencode idiom): the
Expand Down Expand Up @@ -1177,12 +1179,13 @@ export function parseNamedProviderRecipe(value: unknown): NamedProviderRecipe {
const models: Record<string, NamedProviderModel> = {};
for (const [id, raw] of Object.entries(obj.models)) {
const knobs = (raw && typeof raw === "object" && !Array.isArray(raw)) ? raw as Record<string, unknown> : {};
const unknown = Object.keys(knobs).filter((key) => !["contextWindow", "outputTokens", "stream"].includes(key));
const unknown = Object.keys(knobs).filter((key) => !["contextWindow", "outputTokens", "stream", "reasoning"].includes(key));
if (unknown.length > 0) throw new Error(`unknown fields on recipe model "${id}" [${unknown.join(", ")}]`);
if (knobs.contextWindow !== undefined && (typeof knobs.contextWindow !== "number" || !Number.isSafeInteger(knobs.contextWindow) || knobs.contextWindow < 2048 || knobs.contextWindow > 10_000_000)) throw new Error(`recipe model "${id}" contextWindow must be an integer in [2048, 10000000]`);
if (knobs.outputTokens !== undefined && (typeof knobs.outputTokens !== "number" || !Number.isSafeInteger(knobs.outputTokens) || knobs.outputTokens < 128)) throw new Error(`recipe model "${id}" outputTokens must be an integer >= 128`);
if (knobs.stream !== undefined && typeof knobs.stream !== "boolean") throw new Error(`recipe model "${id}" stream must be a boolean`);
models[id] = { ...(knobs.contextWindow !== undefined ? { contextWindow: knobs.contextWindow } : {}), ...(knobs.outputTokens !== undefined ? { outputTokens: knobs.outputTokens } : {}), ...(knobs.stream !== undefined ? { stream: knobs.stream } : {}) };
const reasoning = parseSummaryReasoning(knobs.reasoning, obj.api, `recipe model "${id}"`);
models[id] = { ...(knobs.contextWindow !== undefined ? { contextWindow: knobs.contextWindow } : {}), ...(knobs.outputTokens !== undefined ? { outputTokens: knobs.outputTokens } : {}), ...(knobs.stream !== undefined ? { stream: knobs.stream } : {}), ...(reasoning !== undefined ? { reasoning } : {}) };
}
return { baseUrl: obj.baseUrl.trim().replace(/\/+$/, ""), api: obj.api, ...(obj.apiKeyEnv !== undefined ? { apiKeyEnv: obj.apiKeyEnv } : {}), ...(obj.credentialRef !== undefined ? { credentialRef: obj.credentialRef } : {}), models };
}
Expand Down
21 changes: 21 additions & 0 deletions src/external-summary-http.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ import { maskUrlForLog, maskUrlsInText } from "./log-mask.js";
import { finalizeSseLineEndings } from "./sse-util.js";
import { proxyDispatcher } from "./upstream-proxy.js";
import { PACKAGE_NAME, VERSION } from "./version.js";
import type { SummaryReasoning } from "./external-summary-settings.js";

/** #3488: bounded retry budget for one candidate call. Total dispatches
* (initial + retries). Retries fire ONLY on transient verdicts and must fit
Expand Down Expand Up @@ -88,6 +89,7 @@ export interface SummaryHttpTarget {
readonly stream?: boolean;
readonly contextWindow?: number;
readonly outputTokens?: number;
readonly reasoning?: SummaryReasoning;
readonly proxyUrl?: string;
/** #2760: stable conversation id, stamped as x-opencode-session on OpenCode Go targets only. */
readonly sessionId?: string;
Expand Down Expand Up @@ -184,6 +186,24 @@ function parseSummary(protocol: PreflightProtocol, text: string): string {
return summary;
}

/** #2666: write the configured thinking control into a summary payload. A
* no-op without the knob, so an unconfigured target's body is unchanged. */
export function applySummaryReasoning(payload: Record<string, unknown>, protocol: PreflightProtocol, reasoning: SummaryReasoning | undefined): void {
if (reasoning === undefined) return;
if (reasoning === "template-off") {
payload.chat_template_kwargs = { ...(record(payload.chat_template_kwargs) ?? {}), enable_thinking: false };
return;
}
if (protocol === "openai") payload.reasoning_effort = "none";
else if (protocol === "responses") payload.reasoning = { effort: "none" };
else if (protocol === "anthropic") payload.thinking = { type: "disabled" };
else {
const generationConfig = record(payload.generationConfig) ?? {};
generationConfig.thinkingConfig = { thinkingBudget: 0 };
payload.generationConfig = generationConfig;
}
}

/** #2760: OpenCode Go (`opencode.ai/zen/go`) requires a stable x-opencode-session
* header for routing + prompt cache and answers HTTP 400 MissingSessionID without
* one. Matched on host + exact `/zen/go` or `/zen/go/...` so adjacent endpoints
Expand Down Expand Up @@ -232,6 +252,7 @@ export function createSummaryHttpCandidate(target: SummaryHttpTarget, maxRespons
payload[plan.protocol === "responses" ? "max_output_tokens" : "max_tokens"] = plan.outputTokens;
}
}
applySummaryReasoning(payload, plan.protocol, plan.reasoning);
// Reuse the existing codecs without carrying the main request's auth or
// model. Bounded retry (#3488): transient verdicts (429/5xx/risk-control
// body markers, fail-fast pre-response network deaths) re-dispatch up to
Expand Down
Loading
Loading