Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions CONFIGURATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -325,6 +325,25 @@ For each request, the proxy resolves the settings by longest-URL-prefix match (t
```
- `threshold: number` — character gate; runs **strictly greater** than this are dropped (`0` = drop any non-empty run). Invalid values fall back to the default instead of throwing.

#### `reasoningGuard`

- **Type:** `object` (`{ enabled?, maxContinue?, maxTierN?, markerText?, base?, offset?, debugLog? }`)
- **Default:** *(disabled — off unless you set `enabled: true` at some level)*
- **Status:** ACTIVE
- **Description:** Opt-in guard against gpt-5.x/gpt-6.x **"lattice" reasoning truncation** (issue #739; upstream [openai/codex#30364](https://github.com/openai/codex/issues/30364)). These models intermittently stop at exactly `base*n + offset` reasoning tokens (default `518n−2` → 516, 1034, 1552, …) mid-thought, then answer from a half-finished thought. When an in-scope terminal round hits the lattice **and** carries an `encrypted_content` blob, bili buffers the response, re-sends it replaying its own reasoning plus a continue nudge (up to `maxContinue` continuation rounds), and folds everything into ONE response whose usage is the true summed total. Reasoning streams live to the client during the fold (no full buffering); only the final clean round's non-reasoning output is passed through. Applies to Responses/SSE streaming requests only (bili is SSE-only); compress-injected turns are exempt (the loop owns those). Sub-fields (merged deepest-wins like every other CompressSettings field):
- `enabled: boolean` — master switch; anything other than `true` keeps the guard fully off. Scope is set by **where** this block sits in the three-level tree (global / provider / model) — there is no separate model list. The strict signature (exact lattice hit + `encrypted_content` + no tool calls) limits which rounds actually trigger recovery.
- `maxContinue: number` — max continuation rounds after the initial round (default `3`).
- `maxTierN: number` — highest lattice tier `n` allowed to continue (default `6`); `0` = unlimited. Raise for rare deep-tier truncations (e.g. an `n=11` hit observed on gpt-6-astra).
- `markerText: string` — nudge text appended as a commentary message each continued round (default `"Continue thinking..."`).
- `base: number` / `offset: number` — the lattice signature `tokens == base*n + offset` (defaults `518` / `-2`). Override if another model family truncates on a different lattice.
- `debugLog: boolean` — verbose per-round logging (default `false`).
```jsonc
// enable globally
{ "compress": { "reasoningGuard": { "enabled": true } } }
// tune per provider (placement scopes it to that provider's traffic)
{ "providers": { "https://your-relay.example": { "compress": { "reasoningGuard": { "enabled": true, "maxContinue": 2 } } } } }
```

#### `stripImages`

- **Type:** `boolean`
Expand Down
19 changes: 19 additions & 0 deletions CONFIGURATION.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -323,6 +323,25 @@
```
- `threshold: number` — 字符门槛;**严格大于**该值的段才被剥离(`0` = 只要非空就剥)。非法值回退默认而不是报错。

#### `reasoningGuard`

- **类型:** `object`(`{ enabled?, maxContinue?, maxTierN?, markerText?, base?, offset?, debugLog? }`)
- **默认值:** *(禁用 —— 除非在某一层设置 `enabled: true`)*
- **状态:** ACTIVE
- **说明:** gpt-5.x/gpt-6.x **"晶格"(lattice)推理截断**守卫(issue #739;上游 [openai/codex#30364](https://github.com/openai/codex/issues/30364))。这些模型会间歇性地在恰好 `base*n + offset` 个 reasoning token 处(默认 `518n−2` → 516、1034、1552 …)思考到一半就停下,然后基于未完成的思路作答。当作用域内的终止回合落在晶格上**且**携带 `encrypted_content` 块时,bili 会缓冲该响应、带着自己的 reasoning 加一条继续提示重发(最多 `maxContinue` 个续写回合),再把全部折叠成**一个**响应,其 usage 为真实求和值。折叠期间 reasoning 实时流式发给客户端(不做整段缓冲),只有最后一轮干净回合的非 reasoning 输出被透传。仅作用于 Responses/SSE 流式请求(bili 只支持 SSE);压缩注入的回合被豁免(由循环负责)。子字段与其他 CompressSettings 字段一样按“深层覆盖”合并:
- `enabled: boolean` — 总开关;非 `true` 时守卫完全关闭。作用域由该块在三级树(全局 / provider / model)**放在哪一层**决定——没有单独模型列表。严格签名(精确晶格命中 + `encrypted_content` + 无工具调用)限制哪些回合真正触发续写。
- `maxContinue: number` — 首轮之后最多续写的回合数(默认 `3`)。
- `maxTierN: number` — 允许续写的最高晶格层级 `n`(默认 `6`);`0` = 不限制。遇到罕见的深层截断时调高(例如在 gpt-6-astra 上观察到一次 `n=11`)。
- `markerText: string` — 每个续写回合追加的 commentary 提示文本(默认 `"Continue thinking..."`)。
- `base: number` / `offset: number` — 晶格签名 `tokens == base*n + offset`(默认 `518` / `-2`)。若其他模型家族在不同晶格上截断则覆盖。
- `debugLog: boolean` — 逐回合详细日志(默认 `false`)。
```jsonc
// 全局开启
{ "compress": { "reasoningGuard": { "enabled": true } } }
// 按 provider 调参(放在哪一层就作用于哪一层的流量)
{ "providers": { "https://your-relay.example": { "compress": { "reasoningGuard": { "enabled": true, "maxContinue": 2 } } } } }
```

#### `stripImages`

- **类型:** `boolean`
Expand Down
2 changes: 2 additions & 0 deletions src/compress-settings.ts
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@ export function mergeCompress(
// provider-level excludeTools.
const absorbLevels = [global?.absorb, provider?.absorb, model?.absorb].filter(Boolean) as NonNullable<CompressSettings["absorb"]>[];
const reasoningLevels = [global?.reasoning, provider?.reasoning, model?.reasoning].filter(Boolean) as NonNullable<CompressSettings["reasoning"]>[];
const reasoningGuardLevels = [global?.reasoningGuard, provider?.reasoningGuard, model?.reasoningGuard].filter(Boolean) as NonNullable<CompressSettings["reasoningGuard"]>[];
return {
modelContextLimit: pick("modelContextLimit"),
maxContextLimit: pick("maxContextLimit"),
Expand All @@ -73,6 +74,7 @@ stripImages: pick("stripImages"),
// exactly like `absorb`/`prompts`: a model-level `threshold` must not
// discard a provider-level `drop: false`.
reasoning: reasoningLevels.length > 0 ? Object.assign({}, ...reasoningLevels) : undefined,
reasoningGuard: reasoningGuardLevels.length > 0 ? Object.assign({}, ...reasoningGuardLevels) : undefined,
promptPack: pick("promptPack"),
};
}
Expand Down
37 changes: 37 additions & 0 deletions src/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import { log as loggerLog } from "./logger.js";
import { validateHttpProxy, type ProxyFallbackOptions } from "./upstream-proxy.js";

import { parseCompatRoles } from "./compat-roles.js";
import type { ReasoningGuardConfig } from "./reasoning-guard.js";

export function safeReadJson(path: string): unknown {
try {
Expand Down Expand Up @@ -187,6 +188,15 @@ export type CompressSettings = {
* dropped (default 2048). */
threshold?: number;
};
/** [#739] Opt-in guard against gpt-5.x/gpt-6.x "lattice" reasoning truncation
* (reasoning stops at exactly base*n+offset tokens, default 518n-2 -> 516,
* 1034, ..., mid-thought). When engaged on a matched-model terminal round that
* hits the lattice AND carries an encrypted_content blob, bili buffers the
* response, replays its own reasoning plus a continue nudge (up to maxContinue
* rounds), and folds to ONE response with true summed usage. Merged sub-field-wise
* across the three levels like `absorb`/`reasoning`; off unless enabled at some
* level. See src/reasoning-guard.ts. */
reasoningGuard?: ReasoningGuardConfig;
};
export type PromptCacheRouting = "auto" | "enabled" | "disabled";
export type UpstreamProxyMode = "auto" | "manual" | "direct";
Expand Down Expand Up @@ -699,6 +709,33 @@ export function parseCompressSettings(v: unknown): (CompressSettings & { injectT
if (typeof obj.promptPack !== "string" || obj.promptPack.trim().length === 0) ok = false;
else out.promptPack = obj.promptPack.trim();
}
if ("reasoningGuard" in obj && obj.reasoningGuard !== undefined) {
const rg = obj.reasoningGuard;
if (!rg || typeof rg !== "object" || Array.isArray(rg)) {
ok = false;
} else {
const rgo = rg as Record<string, unknown>;
const cleaned: ReasoningGuardConfig = {};
for (const key of ["enabled", "maxContinue", "maxTierN", "markerText", "base", "offset", "debugLog"] as const) {
if (!(key in rgo)) continue;
const v = rgo[key];
if (key === "enabled") {
if (typeof v !== "boolean") { ok = false; continue; }
cleaned.enabled = v;
} else if (key === "debugLog") {
if (typeof v !== "boolean") { ok = false; continue; }
cleaned.debugLog = v;
} else if (key === "markerText") {
if (typeof v !== "string" || v.trim().length === 0) { ok = false; continue; }
cleaned.markerText = v.trim();
} else {
if (typeof v !== "number" || !Number.isFinite(v)) { ok = false; continue; }
(cleaned as Record<string, unknown>)[key] = v;
}
}
if (ok) out.reasoningGuard = cleaned;
}
}
if (!ok) return undefined;
return out;
}
Expand Down
Loading
Loading