Repository navigation
[PR] feat: name top-level keys when the egress window fuse fires (#7273) - #7275
ranxianglei wants to merge 2 commits into
Conversation
📦 Built Package ArtifactBranch: Latest build — install this exact versionnpm install -g billion-context@0.0.10957-pr.7275.bf1e44bThis exact version is immutable and reproducible. The rolling Recent builds (commit → version)
Download artifact
tar xzf billion-context-pr7275.tgz
npm install -g packageInstall instructions are refreshed on each push. |
The four-bucket breakdown lumps every non-messages/tools/system field into "other", so a body whose mass rides a single DSH-side top-level field (~1.7M tokens in "other" that the contextView never counts) cannot be localized. Add estimateRawBodyTopKeys (per-top-level-key token + byte count, same CJK meter + NON_TEXT_BODY_KEYS exclusions as the plain meter — the sum equals estimateRawBodyTokens exactly) and log the top-N keys in the egress fuse path, also surfacing them in the 413 response so the operator can name the offending field without a body dump.
7d9fe0b to
0e26e89
Compare
The PR's relay wiring (top-key log line + 413 response clause) had no test coverage — only the unit-level estimateRawBodyTopKeys was pinned. The existing #4079 e2e case hits exactly that path (no model in body → preflight skipped → relay fuse measured tier), so assert the new clause is present in the 413 message: the mass rides messages[] there, so the clause must name it.
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) 查重: 不重复任何既有 issue/PR(检索 egress fuse / other bucket / top-level key / 413,open+closed)——但与在途 PR #7092 同机制家族重叠(egress fuse 定价 + 413 字段定位):同用户可见症状(DSH 会话被非-prompt 质量 413、字段无法定位)、同机制(fuse 判定基准 + "other" 桶可见性),且 #7092 的方案是超集(根因修复:prompt-mass 定价;诊断: 独立代码审核(重读 diff,未复用 issue 侧分析):
阻断性发现——与在途 #7092 的冲突(为何不能按现状合):
标签: enhancement / p3 / dsh / scope/wiring 已打(按本 PR 自身 diff 判定:relay 接线路径上的增量诊断;用户侧 p2 影响归 #7092/#7273)。 结构化八问
合并判定: 暂不可合——代码本身过审(正确、有测试、diff 干净),但它是同机制家族中后到、且判定基准立场与在途 #7092 相反的 duplicate。等 owner 决策(见 🔶 需要决策 评论)。 |
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) Delivery summary — review of this PR is complete; the thread now waits on your decision (see the 🔶 需要决策 comment above). What was done:
What was verified (live): CI at head What remains / risk: the merge blocker is not a code defect — it is the mechanism-family conflict with in-flight #7092 (same relay.ts lines, opposite judgment-basis positions; #7092 already ships both the #7273 diagnostic and the root fix). Risk of merging this PR now: #7092 gets reworked and the 413 message carries a misleading whole-body top-key view after the basis moves. My recommendation is option A (hold this PR, merge #7092 first); default if you don't decide: this PR stays open on hold, nothing merges. |
What
When the egress window fuse fires (413
egress_window_fuse), the four-bucket breakdown (messages / tools[] / system / other) lumps every non-messages/tools/system field intoother— a black box. The #7273 report (~1.7M tokens inotherthat the contextView never counts, blocking every turn of a DSH session) cannot be localized from that clause alone.This PR adds
estimateRawBodyTopKeys(per-top-level-key token + byte count, same CJK-aware meter +NON_TEXT_BODY_KEYSexclusions as the plain meter — the sum of every key's tokens equalsestimateRawBodyTokens(parsed)exactly) and wires it into the egress fuse path:Why
Ask #1 from the #7273 diagnostics report: "when egress fuse triggers, print
estimateRawBodyBreakdownby top-level key (key name + byte size + est tokens), and log it. Currently only four-bucket total, 'other' is a black box, can't localize."The sum invariant (top-key tokens == plain meter) is asserted in the regression test, so the new view cannot drift from the fuse's judgment basis.
Behavior change
None to the fuse's judgment basis. The fuse still fires on
estimateRawBodyBreakdown(parsed).total > fuseWindow(unchanged). The top-key view is purely additive diagnostics — it names the field that already pushed the body over the window, it does not change what counts.Ask #2 from the report (should a top-level field that doesn't count toward
contextView.sentTotalbe able to alone push the body beyond the window?) is a design question I'm raising in the issue thread for owner input — it is NOT addressed by this PR.Closes #7273