Skip to content

feat: opt-in gpt-5.x reasoning-truncation guard - #740

Merged
ranxianglei merged 5 commits into
masterfrom
2026-09-13_reasoning-guard
Sep 14, 2026
Merged

ranxianglei merged 5 commits into
masterfrom
2026-09-13_reasoning-guard

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Fixes #739

Ports the codexcomp / cpa-plugin detect → continue → fold fix for the gpt-5.x 518n−2 reasoning-truncation problem into bili as a single-process, default-off core module — no extra proxy process stacked on top of bili.

What it does

New module src/reasoning-guard.ts. The guard takes over the stream only when all of these hold:

  • config reasoningGuard.enabled: true (default off)
  • Responses/SSE wire (/responses, streaming)
  • model ∈ configured list (default gpt-5.5, gpt-5.6; empty list = all models)
  • terminal reasoning_tokens hits the lattice (rt+2) % 518 === 0 && rt >= 516 (i.e. 516, 1034, 1552, …)

On engage it buffers the round, replays that round's reasoning items (including encrypted_content) with a phase:"commentary" Continue thinking... nudge up to maxContinue rounds, folds everything into one reconstructed terminal response, and reports true summed usage under metadata.proxy_billed_usage (plus metadata.proxy_rounds). Reasoning is streamed live (Codex TTFB); non-reasoning items are buffered and flushed on a clean stop. If any gate fails — or the round is clean — the existing pipeline runs untouched, so there is zero behavior change when disabled.

Design notes

  • Generic recovery + data-driven detection: only the fingerprint (lattice + model scope) is model-specific; the continue/fold machinery is agent-agnostic, so adding a new affected model is a config edit, not code. (See discussion in [Question] Can we fix gpt 516 problem in bili proxy? #739.)
  • First-class core module, not a bolt-on plugin.
  • A small SseSink interface decouples the fold from node:http.ServerResponse so it is unit-testable.

Files

  • src/reasoning-guard.ts (new) — guard + live-fold state machine
  • src/config.ts — reasoningGuard option (ProxyOptions + FileConfig + loadOptions passthrough)
  • src/server.ts — forward() proxy-mode hook + config hot-reload
  • tests/reasoning-guard.test.ts (new) — 12 tests (pure functions + 2 local-server SSE integration tests of the fold)

Pre-flight

  • npm run typecheck ✓
  • npm test ✓ (1441 pass / 0 fail / 2 skipped e2e-gated)
  • npm run build ✓

@github-actions

github-actions Bot commented Sep 13, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-09-13_reasoning-guard (5f60153)

Option A — Install from npm PR tag (recommended)

npm install -g billion-context@pr-740

Each push to this PR publishes a new version under the pr-740 npm tag.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr740.tgz
npm install -g package

This comment is automatically updated on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 On it — reviewing PR #740 (reasoning-truncation guard). Pulling the branch, checking diff cleanliness vs master, and re-running typecheck / test / build before I give a verdict. Will report back shortly.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 Review of PR #740 (reasoning-guard)

Diff cleanliness — ✅ clean. Single commit, 4 files, +909/−0, purely additive (new module + config passthrough + one forward() hook + tests). No stray/generated files, no whitespace churn. Branch is 11 commits behind master (release v0.1.108 + persist-encryption #708/#711), but those touch none of the PR's files → clean rebase/merge, no conflict risk.

Pre-flight — ✅ all green (ran locally on branch head d8dba7e):

  • npm run typecheck — clean
  • npm test — 1441 pass / 0 fail / 2 skipped (e2e-gated)
  • npm run build — success (dist/index.js 2.79 MB, acp-kernel inlined)

⚠️ One blocking design question — does the guard ever engage for the target (codex + gpt-5.x)?

The forward() hook gates on !prepared.compressInjected (src/server.ts:3814). For Responses, compressInjected = injectTools && !isCompactionTrigger (src/server.ts:2604), where injectTools = opts.compress.injectTool && !pluginMode (src/server.ts:2412). injectTool defaults to true (src/config.ts:451). Codex runs in proxy mode via the MITM launcher (not plugin mode), so by default pluginMode=false → injectTools=true → compressInjected=true → the guard never engages.

It only engages when compress.injectTool:false / ACP_NO_INJECT_TOOL=1 / the turn is a compaction trigger. So out of the box, against a standard bili codex setup, this feature is inert — which is exactly the scenario #739/codexcomp targets. Two readings:

I'd rather not widen the gate myself — engaging alongside the rewriter silently drops the #371 retry and the compress-nudge channel for those turns (a functional regression for the compression loop). My recommendation for v1: keep the gate, but (a) document that enabling reasoningGuard requires compress.injectTool:false (global or per-route) for affected models, and (b) add a startup/log warning when reasoningGuard.enabled=true but tool injection is still on for a matching model, so the no-op can't be silent. But this changes the feature's scope, so I want your call first: should the guard be usable together with compression enabled, or is "pick one per model" acceptable for v1?

Non-blocking findings:

  1. Clean rounds are still reconstructed, not "untouched." Once a model matches, every turn (even a clean, non-truncated one) goes through the fold: rebuilt output array, sequence_number renumbered from 0, output_index remapped, plus metadata.proxy_rounds/proxy_billed_usage added — and because the hook returns before the [bug] 假完成:模型把工具调用写成文字(无 tool_use 块)→ 轮次提前终止,agent 显示正常完成(#361 根因 follow-up) #371 block (src/server.ts:3837) and the tag-strip pipe (src/server.ts:3895), those are bypassed too. The PR body says "or the round is clean — the existing pipeline runs untouched," which isn't accurate; only pre-engagement gate failures preserve the existing pipeline. Usage math is preserved for a single clean round (I verified agentUsage reproduces the original exactly), so it's benign, but the wire differs on every engaged-model turn. Suggest fixing that doc line + a short comment at the fold site.
  2. Output-order edge case. Reasoning items stream live while non-reasoning items buffer until stop; if upstream ever emits a message before reasoning, the reconstructed output reorders relative to the original output_index. gpt-5.x is reasoning-first, so low risk — worth a one-line note.
  3. Continuation fetches don't check response.ok and don't retry (they use fetchWithTimeout in the runReasoningGuard continuation loop). A transient 429/5xx on a continue round degrades to upstream_eof→incomplete (graceful, no crash) but is neither retried nor logged with status. Consider a single retry or at least logging the HTTP status.
  4. Test coverage gaps. The two integration tests cover the fold + clean passthrough well (summed usage rt=2450 / input=100 / output=2460, replay of nudge + encrypted_content, exactly one terminal). Missing: (a) an end-to-end test of the forward() hook itself — the compressInjected=false path in server.ts is untested — and (b) the clean-round test asserts reasoning_tokens + message presence but not full usage equality.

Code quality is good overall — pure-function extraction, SseSink decoupling for testability, hex-safe (no <acp> literals in source), no as any/@ts-ignore, loggerLog used, default-off correct.

Verdict: hold for your call on the applicability question above (⚠️) — it decides whether the PR delivers its stated goal. Happy to apply the doc/comment fix, the config warning, and the test additions on the branch once you confirm direction.

中文摘要:diff 干净、typecheck/test/build 全绿;但发现一个关键适用性问题——guard 以 !compressInjected 为门槛,而 codex 走 MITM 代理模式默认会注入压缩工具(compressInjected=true),因此默认配置下对目标场景(codex+gpt-5.x)guard 根本不触发。需你拍板:是"每个模型二选一(关掉工具注入才生效)"还是"与压缩共存"。据此建议合并前先补文档/启动告警,或调整门槛;其余为若干非阻塞小项(干净轮次仍被重建、续传不重试不查状态、缺 forward() 钩子端到端测试)。

@ranxianglei

Copy link
Copy Markdown
Owner Author

"reasoningGuard": { "enabled": true, "models": ["gpt-5", "gpt-6"],

这个配置是不是和三级是冲突的?

另外这个是临时配置?
如果是的话,我理解是不是可以加一个专门给临时的配置的字段,放在这个下面?

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 两个问题分开答,先给结论。

Q1:和三级压缩配置冲突吗?

结论:数据上不冲突(不会互相覆盖、不会打架),但有两处真实的耦合/不一致,你感觉到的"冲突"应该主要来自第一处。

① 功能耦合(这是真·依赖,也是上一轮我提的那个阻塞点)
reasoningGuard.enabled: true 单独开着不会生效。forward() 里的门槛是 !prepared.compressInjected(src/server.ts:3814),而 Responses 下 compressInjected = injectTools && !isCompactionTrigger,injectTools = opts.compress.injectTool && !pluginMode(src/server.ts:2411-2412,2604)。injectTool 默认 true(src/config.ts:451),codex 走 MITM 代理模式(pluginMode=false)→ 默认 compressInjected=true → guard 根本不触发。

也就是说:guard 能不能跑,取决于 compress 那套配置里的 injectTool。要让它工作,你得同时 compress.injectTool: false(或 ACP_NO_INJECT_TOOL=1)。这就是它和压缩配置的实质纠缠——不是字段层面撞,而是"guard 的激活被 compress 的门控卡着"。所以你在压缩开着的状态下放这段 reasoningGuard,它会静默无效。

② 作用域机制不一致 + 命名撞车
就算绕过了①,两者也不是一套模型作用域:

  • 三级级联(mergeCompress,src/compress-settings.ts:33):global → provider(URL 前缀 route)→ model(精确名 key,src/config.ts:278 findRoute(...)?.models?.[model]),逐字段最深覆盖。
  • reasoningGuard:一个全局 enabled 布尔 + 扁平 models: string[] 前缀匹配(src/reasoning-guard.ts:79-84 m === model || model.startsWith(m),空数组=全部)。不看 route、不分 provider、无逐级合并。
  • 后果:没法说"只在某条 route 上、对 gpt-5.x 开 guard";而且两边都叫 models 但语义不同(精确名 key vs 前缀族列表),同一个配置文件里容易看混。

我的建议(Q1):

  • 前缀族匹配对这个特性本身是合理的(指纹针对的是"gpt-5.x 这一族共病",硬塞进 route/精确名树反而别扭),所以我不主张把它并进三级级联。
  • 小改两点降低不一致感:(a) 字段 models 改名 modelPrefixes(或 matchModels),消掉撞名;(b) 文档写明它故意不进级联(全局 only)+ 为什么。
  • 真正要你拍板的是①:guard 到底要不要能和压缩共存?我上一轮的倾向是 v1 保持"每个模型二选一(开 guard 就得关工具注入)",并补一条启动告警(enabled 开着但 injectTool 还开着时 warn,避免静默无效)。如果你希望"压缩和 guard 同开",那就得改门槛,代价是这些轮次会绕过 [bug] 假完成:模型把工具调用写成文字(无 tool_use 块)→ 轮次提前终止,agent 显示正常完成(#361 根因 follow-up) #371 fake-completion retry 和压缩 nudge 通道——需要你确认能接受。

Q2:是临时配置吗?要不要加专门的临时字段?

我的判断:它是"workaround 形态",但耐久性一半一半,所以现在不建议新开一个"临时配置"层。

  • 拆开看耐久性:
    • 检测指纹(latticeBase/latticeOffset + 默认 models)绑死当前 gpt-5.x 的具体缺陷,OpenAI 一改就废 → 这部分确实是临时的。
    • 恢复机制(detect→continue nudge→fold 成一条响应→报真实 usage)是通用的,以后任何类似供应商截断都能复用 → 这部分是耐久的。
  • 因为一半一半,把整块标成 "temporary" 会低估那个可复用的恢复引擎,还会误导人以为整块会删。
  • 为单个特性新开顶层 "experimental/temporary" 维度:代码库里现在没有任何先例(我查过,没有 experimental/workaround/temporary 字段),"临时桶"又特别容易腐(要么变永久项、要么删了弄坏别人配置)——YAGNI。
  • 更合适的现成接缝:易变部分已经隔离在具名字段里(models/latticeBase/latticeOffset),这才是"bug 变了改这里"的维护点,把它写进文档即可,不必给整块打标签。OpenAI 修好后用户 enabled: false 一关,代码默认 off 安静待命,不用删。
  • 若要显式的"临时性"信号,bili 本来的实验/诊断惯例是 env(ACP_*);最低成本且符合习惯的做法是加一个文档小节("Workarounds / experimental"),而不是新 schema 层。

我的建议(Q2): 保持它作为正常、默认关的顶层选项(PR 现状)。不为单个特性新增"临时配置"层。 只有当你预期后面还有第二、第三个同类 workaround 时,再抽一个轻量 workarounds: 命名空间把它们归拢——到那时再动,现在先 YAGNI。

一句话串起来

reasoningGuard 最稳的心智定位是:一等公民的 opt-in workaround,故意放在三级级联之外(按模型族而非 route/模型 scope),易变部分已隔离在具名字段里。这样 Q1 的不一致和 Q2 的临时性就一致地解决了;唯一还需要你拍板的是①那条功能门控(共存 vs 二选一)。

要不要我现在就把 (a) 字段改名 + 文档说明 +(Q1①) 的启动告警补到分支上?还是你先定①的方向,我再动手?

ework-agent added 5 commits September 13, 2026 21:53
Port codexcomp/cpa-plugin detect->continue->fold fix for gpt-5.x
518n-2 reasoning truncation into bili as a single-process, default-off
core module (src/reasoning-guard.ts). When enabled and a Responses/SSE
stream from a matched model terminates with reasoning_tokens on the
518n-2 lattice, bili replays the round's reasoning items
(+encrypted_content) with a continuation nudge up to N rounds, folds
them into one response, and reports true summed usage. Otherwise the
existing pipeline is untouched.
Community telemetry (gpt-6-astra reasoning-token distribution) confirms the
same 518n-2 truncation lattice on GPT-6. Broaden DEFAULT_MODELS from the
specific 5.5/5.6 releases to family prefixes [gpt-5, gpt-6] so both lines are
covered out of the box; detection stays strict (exact lattice + encrypted
content), so breadth adds no false-positive action.
…#739)

The inner models prefix list overlapped with three-level scoping and could neuter a deeper-level enable (an inherited global models list would silently filter out a model-level enable). Scoping is now expressed purely by where the block sits in the global/provider/model tree; the strict detection signature (exact lattice hit + encrypted_content + no tool calls) limits actual action. A legacy models key in an existing config is ignored, not fatal.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Question] Can we fix gpt 516 problem in bili proxy?

1 participant