Skip to content

[PR] feat: name top-level keys when the egress window fuse fires (#7273) - #7275

Open
ranxianglei wants to merge 2 commits into
masterfrom
2026-11-19_egress-fuse-top-keys
Open

ranxianglei wants to merge 2 commits into
masterfrom
2026-11-19_egress-fuse-top-keys

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

What

When the egress window fuse fires (413 egress_window_fuse), the four-bucket breakdown (messages / tools[] / system / other) lumps every non-messages/tools/system field into other — a black box. The #7273 report (~1.7M tokens in other that the contextView never counts, blocking every turn of a DSH session) cannot be localized from that clause alone.

This PR adds estimateRawBodyTopKeys (per-top-level-key token + byte count, same CJK-aware meter + NON_TEXT_BODY_KEYS exclusions as the plain meter — the sum of every key's tokens equals estimateRawBodyTokens(parsed) exactly) and wires it into the egress fuse path:

  • logs the top-N top-level keys (key + bytes + est. tokens) when the fuse fires;
  • surfaces them in the 413 response body so the operator can name the offending field without a body dump.

Why

Ask #1 from the #7273 diagnostics report: "when egress fuse triggers, print estimateRawBodyBreakdown by top-level key (key name + byte size + est tokens), and log it. Currently only four-bucket total, 'other' is a black box, can't localize."

The sum invariant (top-key tokens == plain meter) is asserted in the regression test, so the new view cannot drift from the fuse's judgment basis.

Behavior change

None to the fuse's judgment basis. The fuse still fires on estimateRawBodyBreakdown(parsed).total > fuseWindow (unchanged). The top-key view is purely additive diagnostics — it names the field that already pushed the body over the window, it does not change what counts.

Ask #2 from the report (should a top-level field that doesn't count toward contextView.sentTotal be able to alone push the body beyond the window?) is a design question I'm raising in the issue thread for owner input — it is NOT addressed by this PR.

Closes #7273

@github-actions

github-actions Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-11-19_egress-fuse-top-keys (c6d3e8c)

Latest build — install this exact version

npm install -g billion-context@0.0.10957-pr.7275.bf1e44b

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Recent builds (commit → version)

commit version build
c6d3e8c billion-context@0.0.10957-pr.7275.bf1e44b build
0e26e89 billion-context@0.0.10956-pr.7275.882cf36 build
7d9fe0b billion-context@0.0.10955-pr.7275.00e2ce0 build

Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr7275.tgz
npm install -g package

Install instructions are refreshed on each push.

The four-bucket breakdown lumps every non-messages/tools/system field into
"other", so a body whose mass rides a single DSH-side top-level field
(~1.7M tokens in "other" that the contextView never counts) cannot be
localized. Add estimateRawBodyTopKeys (per-top-level-key token + byte count,
same CJK meter + NON_TEXT_BODY_KEYS exclusions as the plain meter — the sum
equals estimateRawBodyTokens exactly) and log the top-N keys in the egress
fuse path, also surfacing them in the 413 response so the operator can name
the offending field without a body dump.
@ranxianglei
ranxianglei force-pushed the 2026-11-19_egress-fuse-top-keys branch from 7d9fe0b to 0e26e89 Compare November 19, 2026 08:38
@ranxianglei ranxianglei changed the title feat: name top-level keys when the egress window fuse fires (#7273) [PR] feat: name top-level keys when the egress window fuse fires (#7273) Nov 19, 2026
The PR's relay wiring (top-key log line + 413 response clause) had no
test coverage — only the unit-level estimateRawBodyTopKeys was pinned.
The existing #4079 e2e case hits exactly that path (no model in body →
preflight skipped → relay fuse measured tier), so assert the new clause
is present in the 413 message: the mass rides messages[] there, so the
clause must name it.
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷

(ai: vllm-qwen/qwen3.8-27b)

查重: 不重复任何既有 issue/PR(检索 egress fuse / other bucket / top-level key / 413,open+closed)——但与在途 PR #7092 同机制家族重叠(egress fuse 定价 + 413 字段定位):同用户可见症状(DSH 会话被非-prompt 质量 413、字段无法定位)、同机制(fuse 判定基准 + "other" 桶可见性),且 #7092 的方案是超集(根因修复:prompt-mass 定价;诊断:otherTop 字段点名)。#7092 先到(2026-11-18 开;#7275 于 2026-11-19 08:36 开,仅晚于 #7092 最后一次更新 33 分钟)。按先来后到,本 PR 判 duplicate of #7092,已打 duplicate 标签——不关闭(关闭权在 owner);#7275 是在 #7092 之上返工还是被其取代,见下方 🔶 需要决策 评论。

独立代码审核(重读 diff,未复用 issue 侧分析):

阻断性发现——与在途 #7092 的冲突(为何不能按现状合):

  1. [PR] fix: egress fuse and doom gates price the prompt mass, not the whole outbound body (#6844 #7013 #7088) #7092 把 fuse 判定基准改为 wire-format 的 PROMPT mass(PROMPT_BODY_KEYS 按协议;measuredOutbound 增加 protocol 参数)。[PR] feat: name top-level keys when the egress window fuse fires (#7273) #7275 的 PR 正文声明 "None to the fuse's judgment basis"——两个 PR 对同一设计问题(bili diagnostics report (v0.1.216, node v24.21.0, win32-x64) #7273 的 Ask fix: address all code review findings (condense, routing, session-id, dead code) #2)持相反立场。字段证据([Bug] estimateRawBodyBreakdown counts non-prompt extension fields toward the fuse budget, causing legitimate requests to | [Bug] estimateRawBodyBreakdown 将非 prompt 扩展字段计入 fuse 预算,导致合法请求被 413 误杀 #7013: "a ~100K-token prompt with an 8 MiB upload field 413'd";dsh #7088: "88% of a refused body was dsh_session_log inside the other bucket")支持 [PR] fix: egress fuse and doom gates price the prompt mass, not the whole outbound body (#6844 #7013 #7088) #7092 的 prompt-mass 立场:窗口被 prompt tokens 消耗,不被同一 JSON 信封里的非-prompt 字段消耗。
  2. [PR] fix: egress fuse and doom gates price the prompt mass, not the whole outbound body (#6844 #7013 #7088) #7092 已经交付了 bili diagnostics report (v0.1.216, node v24.21.0, win32-x64) #7273 场景的诊断:RawBodyBreakdown.otherTop(other 桶内逐顶层字段 tokens+bytes)+ measuredBreakdownClause 点名 top field("other ~N (non-prompt, excluded from the window budget; top field: dsh_session_log ~N tokens ≈N B)")——正是 Ask fix: keep system prompt byte-stable for prefix cache #1。其根因修复同时解除 bili diagnostics report (v0.1.216, node v24.21.0, win32-x64) #7273 用户的会话阻塞(1.7M token 遥测字段不再 413 一个本可装下的 prompt)。
  3. 两者都合(无论顺序)会让 413 消息带两条重叠 clause,且 [PR] feat: name top-level keys when the egress window fuse fires (#7273) #7275 的 whole-body top-key 视图在基准变为 prompt mass 后会把非-prompt 字段点名成"肇事者"——主动误导("trim dsh_session_log" 并不能解除发送阻塞)。两个 PR 还改同一行 fuseMessage(src/server/relay.ts)与 src/preflight.ts / src/server.ts / 两个测试文件的同一区域——语义冲突,不是文本冲突。

标签: enhancement / p3 / dsh / scope/wiring 已打(按本 PR 自身 diff 判定:relay 接线路径上的增量诊断;用户侧 p2 影响归 #7092/#7273)。


结构化八问

  1. 是什么问题: egress fuse 触发时,四桶 breakdown 把所有非 messages/tools/system 的顶层字段全归入 "other"——黑箱;bili diagnostics report (v0.1.216, node v24.21.0, win32-x64) #7273 的 DSH 会话被 ~1.7M token 的 "other" 每轮阻塞,413 消息无法定位字段。
  2. 什么原因导致的: estimateRawBodyBreakdown(src/preflight.ts:503)只分四桶,无逐顶层 key 视图;且 fuse 判定基准(whole-body tokens)与 contextView.sentTotal 口径不同——"other" 质量在 fuse 里可见、在 contextView 里不可见。
  3. 影响范围与严重程度: 所有触发 egress fuse 的会话(出站 body 超模型窗口);严重度 = 诊断可见性缺口——fuse 正确拦截但原因无法点名,运维只能猜。非数据丢失、非缓存泄漏。
  4. 怎么修复的: 新增 estimateRawBodyTopKeys——同 CJK 计量器 + 同 NON_TEXT_BODY_KEYS 排除集,所有 key 的 tokens 之和精确等于 estimateRawBodyTokens(parsed)(回归测试钉死);fuse 触发时在日志 + 413 响应体点名 top-N 顶层 key(key + bytes + est. tokens)。纯增量诊断,判定基准不动。
  5. 是否有回归问题: fuse 判定基准不变;413 消息文本为增量(PR 已披露);既有 fuse e2e 用 regex 断言、本地 4/4 通过;另一 measuredOutbound 调用点只读 .tokens。⚠️ 与在途 [PR] fix: egress fuse and doom gates price the prompt mass, not the whole outbound body (#6844 #7013 #7088) #7092 存在语义冲突(同一 relay.ts 行、相反判定基准立场)——这是合并阻断点,不是代码缺陷。
  6. 是否破坏缓存: 不涉及——纯诊断路径,不触碰缓存/重试/游标/去重键。
  7. 是否改了设计方向: 本 PR 自身不改;但与 [PR] fix: egress fuse and doom gates price the prompt mass, not the whole outbound body (#6844 #7013 #7088) #7092 的 "price the prompt mass" 方向在同一设计问题(Ask fix: address all code review findings (condense, routing, session-id, dead code) #2)上相撞——机制家族以哪个 PR 为载体需要 owner 拍板。
  8. 是否引入了新的配置: 无。

合并判定: 暂不可合——代码本身过审(正确、有测试、diff 干净),但它是同机制家族中后到、且判定基准立场与在途 #7092 相反的 duplicate。等 owner 决策(见 🔶 需要决策 评论)。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

🔶 需要决策

背景一句话: PR #7275(本线程)与在途 PR #7092 是同机制家族(egress fuse 定价 + 413 字段定位)的两个 PR,改 src/server/relay.ts 同一行代码、对同一设计问题(Ask #2:非-prompt 顶层字段是否计入 fuse 预算)持相反立场——先合哪个、以哪个为载体,需要你拍板。

待决问题: 机制家族以哪个 PR 为载体、fuse 判定基准取哪个?(关联:#7273 上早先 triage 评论主张"保持 whole-body 基准"——该立场与 #7092 相反,且不被字段证据支持,建议撤回,以 #7092 的 prompt-mass 立场为准。)

选项(各带代价):

你的推荐 + 不决定的默认行为: 推荐 A——#7092 有字段证据(#7013/#7088:本可装下的 prompt 被 8 MiB 遥测字段 413;被拒 body 的 88% 是 dsh_session_log),whole-body 基准被实证有误;#7092 的 otherTop 已覆盖 #7273 的 Ask #1(点名 dsh_session_log + tokens + bytes);#7275 的独有增量(whole-body 视图)绑定旧基准,#7092 之后会误导。不决定的默认行为: #7275 保持 open 挂起(不合并),#7092 走正常 review 流程;#7273 用户的 413 由 #7092 解除。

八问(决策请求):

  1. 是什么问题: 同机制家族两个在途 PR 对判定基准持相反立场、改同一行代码——先合任一个都迫使另一个返工,两个都合则产生误导诊断。
  2. 什么原因导致的: [PR] feat: name top-level keys when the egress window fuse fires (#7273) #7275 因新的同症状用户报告(bili diagnostics report (v0.1.216, node v24.21.0, win32-x64) #7273)开出,查重按症状关键词检索 issue 时漏掉了在途 PR [PR] fix: egress fuse and doom gates price the prompt mass, not the whole outbound body (#6844 #7013 #7088) #7092(其标题 "price the prompt mass" 不含 "egress fuse / other bucket / top-level key" 等关键词)——两个 PR 互不可见。
  3. 影响范围与严重程度(按选项):
  4. 怎么修复(推荐选项 A 的思路): [PR] feat: name top-level keys when the egress window fuse fires (#7273) #7275 挂起;[PR] fix: egress fuse and doom gates price the prompt mass, not the whole outbound body (#6844 #7013 #7088) #7092 走正常 review/合并流程;[PR] fix: egress fuse and doom gates price the prompt mass, not the whole outbound body (#6844 #7013 #7088) #7092 合入后重估 [PR] feat: name top-level keys when the egress window fuse fires (#7273) #7275 的残余价值(prompt-mass 基准上的 whole-body top-key 视图)——返工或取代,由你定。
  5. 回归风险(按选项): A: 无(不新增合并);B: [PR] fix: egress fuse and doom gates price the prompt mass, not the whole outbound body (#6844 #7013 #7088) #7092 返工可能回归其字段证据修复;C: 诊断面本身回归(误导 413 指引)。
  6. 是否破坏缓存: 不涉及(三个选项都不触碰缓存/重试/游标)。
  7. 是否改了设计方向: A 以 [PR] fix: egress fuse and doom gates price the prompt mass, not the whole outbound body (#6844 #7013 #7088) #7092 的方向(prompt-mass 定价)为机制家族载体;B/C 维持 whole-body 基准方向、迫使 [PR] fix: egress fuse and doom gates price the prompt mass, not the whole outbound body (#6844 #7013 #7088) #7092 适配。
  8. 是否引入了新的配置: 无(三个选项都不新增配置)。

选项对比表:

A: #7275 挂起,先合 #7092 B: 先合 #7275 C: 两个都合
#7273 用户解阻 靠 #7092(根因修复) 不解阻(基准不变) 靠 #7092
根因修复(prompt-mass 定价) #7092 无返工 #7092 需返工 #7092 需返工
Ask #1 诊断(字段点名) #7092 otherTop(覆盖 #7273 场景) #7275 top-keys(旧基准) 两套(重叠)
误导诊断风险 #7092 之后无 窗口期 + 基准变化后 长期
返工代价 #7275(小,或被取代则零) #7092(大) #7092(大)

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷

(ai: vllm-qwen/qwen3.8-27b)

Delivery summary — review of this PR is complete; the thread now waits on your decision (see the 🔶 需要决策 comment above).

What was done:

  1. Independent review of the full diff (4 files, +121/−6): walker correctness, sum invariant, edge cases, fuse-basis invariance, test quality, diff cleanliness — all pass. Details in the review comment above.
  2. Fixed one wiring gap found in review: the new 413-body top-keys clause had no test pinning it → added the assertion to the existing Context Window Overflow Issue Summary: Raw Requests Sent Directly to Upstream Rejected + Local Intercept Cooldown Lockin | 上下文超窗问题汇总:请求裸冲上游被拒 + 本地拦截冷却锁死会话(统一分析入口) #4079 fuse e2e, commit c6d3e8c78c7 pushed to the PR branch (4/4 local pass, typecheck clean).
  3. Labeled the mirror (judged on this PR's own diff): enhancement / p3 / dsh / scope/wiring, plus needs-decision + duplicate. Not closed — close authority is yours.
  4. Cross-reference note posted on [PR] fix: egress fuse and doom gates price the prompt mass, not the whole outbound body (#6844 #7013 #7088) #7092 so that thread knows [PR] feat: name top-level keys when the egress window fuse fires (#7273) #7275 is on hold pending the decision.

What was verified (live): CI at head c6d3e8c78c7 — all e2e lanes green, static gates green, sharded test lanes in progress; both PRs' live state re-checked before writing the verdict.

What remains / risk: the merge blocker is not a code defect — it is the mechanism-family conflict with in-flight #7092 (same relay.ts lines, opposite judgment-basis positions; #7092 already ships both the #7273 diagnostic and the root fix). Risk of merging this PR now: #7092 gets reworked and the 413 message carries a misleading whole-body top-key view after the basis moves. My recommendation is option A (hold this PR, merge #7092 first); default if you don't decide: this PR stays open on hold, nothing merges.

@ranxianglei ranxianglei added duplicate This issue or pull request already exists needs-decision root-cause labels Nov 19, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dsh duplicate This issue or pull request already exists enhancement New feature or request needs-decision p3 root-cause scope/wiring

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bili diagnostics report (v0.1.216, node v24.21.0, win32-x64)

1 participant