Skip to content

[PR] fix: remember local preflight 413 refusals so estimate demotion cannot reopen oversized forwards - #3867

Merged
ranxianglei merged 1 commit into
masterfrom
2026-10-19_local-refusal-evidence
Oct 20, 2026
Merged

ranxianglei merged 1 commit into
masterfrom
2026-10-19_local-refusal-evidence

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Closes #3846

The bug

A locally refused over-window request (deterministic exhaustion → local 413, payload never forwarded) was re-forwarded on retry once the usage baseline demoted to an estimate — which a preflight rebuild does (src/server.ts anchor demotion: rebuiltTextSize > lastInputTokens → source="estimate") and armFailureShrink also does. On retry, probeForEvidence saw baselineFloor = 0, ran before the #726 dead-end check, and forwarded the payload once "to acquire usage evidence" — billing the full payload upstream. The #726 dead-end cannot save it: its key is model␀limit␀sha256(inboundBody), and growing bodies (prod session: 606524 → 620664 → 633349 input tokens, only 32768 cached each) defeat the hash every turn. The local refusal verdict — independent evidence that this payload does not fit that window — evaporated with the baseline's source label.

The fix (host-only, src/server.ts)

  1. Record: noteLocalRefusal(session, origin, model, window) at all three local-413 sites — the probe doomed-bytes gate, the fresh-session doom gate, and the final deterministic-exhaustion fail-fast. Stored as plain JSON in session.metadata.localRefusals ({origin, model, window, at}[], capped at 16 records, deduped per scope keeping the widest window). Persists across restarts through the existing metadata persistence; survives the JSON round-trip.
  2. Honor: localRefusalCovers(metadata, origin, model, window) matches on both origin and model equal AND recorded window ≥ current limit (same-or-tighter coverage only). When covered, the four forward-once escape hatches are suppressed:

Fitting payloads still forward everywhere; another origin, model, or a larger window is evaluated independently (probe stays open). The refusal evidence is never written into lastInputTokens/lastInputTokensSource — it is not upstream usage and must not masquerade as one.

Behavior changes (old → new)

Path Old New
Estimate-only probe after a local 413, same origin+model+window forwards once (full payload billed upstream) suppressed; falls through to fold/fail-fast
Image arbitration (immediate / post-fold) after a local 413 forwards once suppressed
#1839 rebuilt-view divergence forward after a local 413 forwards once suppressed
#3146 fresh-session forward-once after a local 413 forwards once suppressed
Fitting payloads (any state) forwarded forwarded (unchanged)
Different origin / model / larger window after a local 413 probes probes (unchanged)

No config surface added (no new env var / config key / CLI flag). Kernel untouched. No persistence-format change beyond the additive metadata.localRefusals array.

Interaction with PR #2928

#2928 (open, reviewed mergeable) restructures the doomed branch inside the probe gate (foldable → fold first, blob → terminal 502). This PR's edits there are additive — record at the 413 site, suppression inside the probe branch — and orthogonal in intent. If #2928 lands first, expect a small textual conflict on rebase; resolve by union of intent (both changes compose cleanly).

Tests

New tests/issue3846-local-refusal-evidence.test.ts — 4 hermetic e2e tests (counting fake upstream, 20K-token window, non-foldable ~400KB system text):

  • a. issue repro verbatim: seed 200 → oversized 413 (zero stream calls beyond seed) → estimate demotion + metadata JSON round-trip → three identical retries all 413 with zero additional upstream stream calls → trimmed request 200. Hygiene assertions: lastInputTokens/Source untouched by the fix (estimate stays estimate), exactly one scoped refusal record.
  • b. production shape: changed bodies per retry defeat the dead-end hash — persisted evidence holds the line.
  • c. negative row: another model on the same origin still probes (independent evaluation).
  • d. negative row: another origin still probes.

A/B verified against unpatched master (stash → run → pop): tests a/b fail pre-fix with expected 413, actual 200 (the exact reported symptom); c/d fail on the new evidence-channel assertions. Post-fix: 4/4 green.

Adjacent suites pass locally (targeted runs): issue2313-probe-forward, issue2490-ledger-floor-doomed-probe, issue1800-image-arbitration-compress, issue1839-ghost-denominator, issue496-forward-once-learn, preflight-fail-fast, preflight-compress — 26/26. Full-suite verdict is CI.

… reopen oversized forwards (#3846)

A locally refused over-window request (deterministic exhaustion -> 413,
payload never forwarded) was re-forwarded on retry once the usage
baseline demoted to an estimate (preflight-rebuild anchor or failure
shrink): probeForEvidence saw baselineFloor=0, ran BEFORE the #726
dead-end check, and forwarded once for 'evidence' - billing the full
payload upstream. Growing bodies also defeated the dead-end body-hash
key, so every turn re-probed (prod: 606K/620K/633K input tokens, 32K
cached each).

- noteLocalRefusal() records {origin,model,window} at all three
  local-413 sites (probe doomed-bytes gate, fresh-session doom gate,
  final deterministic-exhaustion fail-fast); plain JSON in
  session.metadata, capped at 16 records, deduped per scope keeping
  the widest window; survives restart via existing metadata persistence.
- localRefusalCovers() suppresses the four forward-once escapes
  (estimate-only probe, image arbitration immediate+post-fold, #1839
  rebuilt-view divergence, #3146 fresh-session escape) at the same or
  tighter window for the same origin+model.

Fitting payloads still forward; other origin/model/larger-window
evaluated independently. Evidence is NEVER written into
lastInputTokens/Source (it is not upstream usage). No config surface
added; kernel untouched.
@ranxianglei ranxianglei changed the title fix: remember local preflight 413 refusals so estimate demotion cannot reopen oversized forwards [PR] fix: remember local preflight 413 refusals so estimate demotion cannot reopen oversized forwards Oct 19, 2026
@github-actions

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-19_local-refusal-evidence (f5af0dc)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.4568-pr.3867.2ba9827

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr3867.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重: 不重复。open+closed 全量检索(按失败签名: local 413 / estimate demotion / forward-once / localRefusals),#3867 是 #3846 的唯一修复 PR;相关但不同机制的只有 PR #2928(doomed-probe gate 重构,正交,PR 描述已声明 rebase 时的文本冲突预期)。无并列的第二个修复。

标签: bug + codex + scope/kernel + p2(缺陷;生产证据来自 Codex Desktop/Responses;preflight/probe 属上下文管理核心域;p2=会话可用但每次重试有真实计费损失,非不可用/缓存失控类)。

本 PR 是处理 #3846 的 daemon 会话交付的工作(单 commit f5af0dcdb,分支 2026-10-19_local-refusal-evidence)。我不重做,现在做独立审核:逐行读 diff、对照改动前代码清点既有行为(回归关)、审测试有效性、核对 CI 状态。分支基点落后 master 190 个 commit,会重点核这 190 个 commit 里对 src/server.ts preflight 域的语义交互。结论按结构化八问随后跟进。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

PR #3867 独立审核结论(不照抄 issue/提交侧八问,以下为审核侧独立重做:拉分支逐行读 diff、对照当前 master、本地 A/B 实测、测试有效性审查)

结构化八问

  1. 是什么问题:bili 在 preflight 阶段本地拒绝超窗请求(确定性耗尽 → 本地 413,从未转发上游)后,一旦 baseline 降级为 estimate,"forward once to learn" 探针会把这个拒收忘掉,把超大请求重新转发给上游——每次重试白烧一次真实上游调用(生产证据:606K input tokens 计费、仅 32K cached)。
  2. 什么原因导致的:本地 413 只通过 [Bug] Codex 额度中断后恢复长会话,preflight SSE 摘要为空并持续 502 #726 dead-end key(model␀limit␀sha256(body))锁住字节级相同的重试;两条路径都能绕过它——(a) estimate 降级重置探针门且该门在 dead-end 检查之前执行;(b) body 每轮增长后 hash 变化。而"这个 origin/model/window 组合已被证明装不下"这一事实本身,此前没有任何地方持久化。
  3. 影响范围与严重程度:影响 proxy 模式下 preflight 确定性耗尽 + usage 源降级到 estimate 的会话(Codex Desktop/Responses 生产形态);每次重试泄露一次超大上游调用 = 真金白银的计费损失 + 上游负载。p2:可用、有绕行(trim/换模型/提窗口),损失按次发生;非不可恢复类,不升 p1。
  4. 怎么修复的:把拒收判决持久化进 session metadata(按 origin+model 去重、同 scope 保留最宽 window、上限 16 条、随现有持久化跨重启存活);当前请求命中覆盖记录(同 origin+model 且记录 window ≥ 当前 limit)时,抑制全部四条 "forward once to learn" 逃生路径(estimate-only 探针、图像仲裁 ×2、任何一轮没有拿到 usage 报告的请求,都会把上下文分母放大 12–20×(nudge 读到 >1000% usage、假 EMERGENCY、基线冻结数分钟) #1839 rebuilt-view divergence、[Bug] 新会话(无 usage 基线)preflight 把字符数当 token——折叠其实成功(2263796→1111212)却 fail-fast 502,上游实际只计费 540828 #3146 fresh-session escape),落回既有本地 413。记录只发生在强判决点(确定性耗尽 / 字节密度 doom floor),绝不用 estimate 当证据。
  5. 是否有回归问题:逐项审过触碰区域的既有行为——(a) 能装下的 payload 不受影响:它们在到达任何被抑制门之前就提前返回(代码路径验证 + trimmed→200 测试钉住);(b) 其他 model/origin/更大窗口照常 fresh 评估(测试 c/d 钉住负向方向);(c) [Bug] Codex 额度中断后恢复长会话,preflight SSE 摘要为空并持续 502 #726 dead-end 锁、doom-probe floor、压缩阶梯、nudge 节奏均未动。已披露的行为变更(旧→新):estimate 降级区间内,若存在覆盖记录,一个"校准 estimate 超窗但实际可能装得下"的 payload 从"一次自愈探针"变为硬 413——这是有意的取舍(已知坏 scope 不再赌真实调用),边界窄(需 estimate 失准 + 无可折叠内容 + 覆盖记录三者同时),消息可操作。无 fail-fast 变吞错:413 状态与消息族不变。
  6. 是否破坏缓存:不破坏。新 metadata 是附加会话状态(localRefusals),防御性读取、JSON round-trip 有测试;转发请求的 wire 字节零改动,前缀缓存不受影响;未触碰任何缓存/重试/游标/去重键语义。
  7. 是否改了设计方向:无架构转向——沿用既有"证据存 session metadata"模式(与 dead-end marker、ledger floor 同族)。唯一有意的策略点:本地拒收证据现在能活过 estimate 降级(此前被遗忘)——这本身就是修复目标。
  8. 是否引入了新的配置:无。不加 env/config/CLI 面,记录纯在 session metadata 内。

独立验证证据(非转述 PR 声明)

CI 现状(重要,非本 PR 之过)

head f5af0dc 的运行:9 job 绿、2 job 红(unused-exports + docs-sync)、test matrix 被级联 skip——全量套件在本 head 上从未跑过。我在纯净 master(39fb841d3)本地复现确认:这两个红都是 master 自身带的,与本 PR 内容无关(bare head 本地两门皆绿):

  1. unused-exports:[security][web] Diagnostics blob prints private relay hosts verbatim in the config section — the "secrets masked" header only masks values, not hosts | 诊断包 config 段原样输出内网中转地址:标题说"密钥已遮蔽",但只遮值不遮主机 #3811 引入的 maskUrlForPublish 是 internal-only 却被 export;
  2. docs-sync:CONFIGURATION.md/.zh-CN.md 残留 Storage wrap-up: 6 fixes + switch default mode to full (single PR) | 存储收尾六修 + 缺省模式翻 full(一个 PR 统一处理) #3818/SQLite unified storage degraded to legacy-only (bili v0.1.201, node v20.19.5, win32-x64) #3788 合并期的重复旧行。

处置(按几行级小修纪律):已开修复 PR #3877(两个标注 commit:去掉 export + 提交重新生成的索引,共 5 行/3 文件,本地验证门绿 + tsc 绿),并立 [no-dispatch] 跟踪 issue #3878(查重:无)。#3877 合并后请 update/re-run 本 PR,即可拿到全量套件的最终裁决。

判定

代码审核通过、行为变更已披露、回归面逐项审毕——可合;唯一待办是等 master 红修复(#3877)合入后的全量 CI 绿灯。人工合并(行为变更档)。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working codex p2 scope/kernel

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug][preflight] Local 413 is forgotten when the baseline becomes an estimate, reopening oversized probes

1 participant