Skip to content

[PR] fix: egress fuse and doom gates price the prompt mass, not the whole outbound body (#6844 #7013 #7088) - #7092

Open
ranxianglei wants to merge 10 commits into
masterfrom
2026-11-18_rcagg-doom-gate-pricing-mismatch
Open

ranxianglei wants to merge 10 commits into
masterfrom
2026-11-18_rcagg-doom-gate-pricing-mismatch

Conversation

@ranxianglei

@ranxianglei ranxianglei commented Nov 17, 2026 •

Copy link
Copy Markdown
Owner

Root-cause fix for the egress-fuse-pricing batch (aggregation thread #7230; supersedes the doom-gate-pricing-mismatch batch #7090 — same root cause, overlapping members): the measured-body egress fuse, the preflight doom gates, the side-request guard, and the client-local doom gate all priced the whole outbound body against the model window, but the window budget only applies to prompt mass. DSH's documented model-hidden top-level field dsh_session_log (up to 8 MiB, DeepSeek-adapter routes only) landed in the other catch-all bucket → ordinary DSH sessions 413 permanently (watermark never advances → deadlock); forked/seeded subagent sessions 413 on the FIRST request (inherited log); and the 413 message misdirected the user to clean up messages[] (the smallest bucket).

What this PR does

Server-side metering (cherry-picked from PR #7016 with -x, authorship preserved)

  1. The meter now prices exactly what the window consumes:
    • PROMPT_BODY_KEYS: Record<WireProtocol, ReadonlySet<string>> — per-wire prompt fields: openai messages+tools, anthropic messages+tools+system, google contents+systemInstruction+tools, responses input+instructions+tools.
    • estimateRawBodyBreakdown(parsed, promptKeys?) gains a prompt field (single walk, no extra pass).
    • measuredOutbound(prepared) now takes the wire protocol and returns tokens = breakdown.prompt; all three server gates price prompt mass (relay.ts egress fuse, server.ts preflight measured gates, side-request.ts guard).
    • promptMassForProtocol(parsed, protocol) — fail-closed: a protocol with no PROMPT_BODY_KEYS entry prices the whole body (never 0).
    • total and estimateRawBodyTokens are unchanged — the rawInputTokens metadata ([Feature] Default state file cleanup strategy #1082) keeps its whole-body semantics.
  2. The 413 breakdown now attributes the other bucket to its top-level fields (ported from PR [PR] fix: exclude model-hidden DSH fields from the measured body (#6844) #6860's otherTop diagnostic): otherTop (descending) added to RawBodyBreakdown; measuredBreakdownClause names the dominating field (top field: dsh_session_log ~1,634,970) with the (non-prompt, excluded from the window budget) suffix — the 413 no longer misdirects the user to clean up messages[].

Client-local doom gate (cherry-picked from PR #7119 with -x, authorship preserved — ZzXxYy1234)

  1. The client doom gate prices prompt mass, not raw body bytes (src/agent/prompt-mass.ts): promptMassOfWireBody(raw) shape-detects the wire protocol (contents→google, input→responses, system→anthropic, messages→openai) and prices only the prompt fields with the same CJK-aware walker as the server. bodyOverWindowDetail (native-intercept.ts) uses it when the body is parseable; non-JSON/unrecognized keeps the byte fallback. The DSH's unattributed over-limit gate prices base64 images as text (bytes/4) — media-heavy requests are falsely rejected wi | dsh 无归属超限门把 base64 图片当文本计价(bytes/4)——媒体重的请求被假拒 422,上游本会正常接受 #4111 two-tier contract (calibrated floor×1.14 / bare floor) is unchanged; media stays uncounted.

Why not PR #6860's approach

#6860's hardcoded MODEL_HIDDEN_BODY_KEYS exclusion would shrink total/estimateRawBodyTokens, contradicting #7016's deliberate whole-body total semantics. #7013 explicitly asked for no hardcoded one-plugin field list: the fix is to price the window budget by what the window consumes, which automatically excludes ANY host-attached non-prompt field, known or unknown. Only the otherTop diagnostic was ported.

Behavior changes (old → new)

  1. Gate criterion (server): whole-body token count → prompt token count (per-wire prompt keys). A body whose prompt fits but whose non-prompt fields are huge now FORWARDS (was: 413); a body whose prompt itself exceeds the window still 413s. Both directions pinned in e2e (tests/issue4079-egress-window-fuse.test.ts).
  2. Gate criterion (client): whole-body bytes/4 → prompt-mass tokens (shape-detected). A body carrying a huge non-prompt field (e.g. dsh_session_log) no longer dooms the client when the prompt fits; a genuinely over-window prompt still dooms. Pinned in tests/issue7106-client-doom-gate-prompt-mass.test.ts.
  3. 413 wording: outbound body measures ~N tokens … the body ITSELF exceeds the window → outbound prompt measures ~N tokens … the prompt ITSELF exceeds the window (relay fuse, both preflight gates); the breakdown clause's other bucket now carries the (non-prompt, excluded from the window budget) suffix and, when non-empty, names the dominating field.
  4. New diagnostic field otherTop in RawBodyBreakdown (additive; pinned in tests/raw-body-walk-depth.test.ts).

Tests

Targeted, local: 265/265 green across tests/raw-body-walk-depth.test.ts, tests/issue4079-egress-window-fuse.test.ts, tests/issue4185-identified-fresh-forward.test.ts, tests/preflight-zero-fold-calibration.test.ts, tests/side-request-isolation.test.ts, tests/issue7106-client-doom-gate-prompt-mass.test.ts, tests/dsh-native.test.ts, tests/issue4894-direct-window-guard.test.ts, tests/native-intercept.test.ts. Typecheck clean. Full-suite verdict = remote CI on the pushed head.

Relationship to existing PRs

Pending decision (NOT fixed by this PR)

Fixes #6844
Fixes #7013
Fixes #7099
Fixes #7106
Fixes #7189
Fixes #7088

Related (decision item, not fixed here): #7114

@ranxianglei ranxianglei changed the title fix: egress fuse and doom gates price the prompt mass, not the whole outbound body (#6844 #7013 #7088) [PR] fix: egress fuse and doom gates price the prompt mass, not the whole outbound body (#6844 #7013 #7088) Nov 17, 2026
@github-actions

github-actions Bot commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-11-18_rcagg-doom-gate-pricing-mismatch (32eda3a)

Latest build — install this exact version

npm install -g billion-context@0.0.10900-pr.7092.1da8f04

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Recent builds (commit → version)

commit version build
32eda3a billion-context@0.0.10900-pr.7092.1da8f04 build
2257410 billion-context@0.0.10735-pr.7092.0aea00c build
ccfee8a billion-context@0.0.10717-pr.7092.60ef9ab build
4763ab5 billion-context@0.0.10703-pr.7092.f0e6b11 build
60edc7c billion-context@0.0.10675-pr.7092.53040de build

Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr7092.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重: 不重复 — 本线程即修复 PR 本身。同主题在途 PR 核对: #7016 已被本 PR 完整包含(两个 commit 以 -x cherry-pick,补丁逐字节一致,已验证);#6860 的机制被本 PR 取代(其 MODEL_HIDDEN_BODY_KEYS 会缩小 total/estimateRawBodyTokens,与 #1082 的 whole-body 语义冲突,且其 e2e 负例"未知字段仍 413"钉的是旧语义),仅 otherTop 诊断被移植。三个 PR 同修一个线程,合并后建议 owner 关闭 #7016 与 #6860(agent 无关闭权)。

标签: 源 PR 已带 bug p1 dsh scope/wiring regression root-cause,无需补打(只加不删,已齐)。

Review — independent audit of the diff (not the issue's conclusions)

Direction (①): correct, root-cause level. The meter priced the wrong quantity: the window budget applies to prompt mass, but all three gates (egress fuse, both preflight measured tiers, side-request guard) compared the WHOLE outbound body against the window. DSH's model-hidden dsh_session_log (8 MiB, other bucket) therefore 413'd fitting prompts — the #6844/#7013/#7088 incident class. Pricing per-wire prompt keys (PROMPT_BODY_KEYS) is exactly #7013's "更彻底的方案": no hardcoded one-plugin field list, so the next host-attached telemetry field cannot recur it. No config surface added (correct — none was needed).

Code (③): verified against the PR head (60edc7c740, base a259e0b8fa = current master, no rebase needed):

  • PROMPT_BODY_KEYS is exhaustive over the WireProtocol union (anthropic|openai|responses|google) — a missing key would have silently fail-open (prompt=0 → gate never arms); it can't happen, and Prepared.protocol is required at all three call sites, so typecheck enforces it.
  • Single walk: prompt and otherTop are accumulated in the existing pass, no second traversal; total and the four buckets are byte-identical to before (invariant pinned in tests/raw-body-walk-depth.test.ts).
  • rawInputTokens ([Feature] Default state file cleanup strategy #1082, src/server/handle.ts:1834) still uses whole-body estimateRawBodyTokens — unchanged, as claimed.
  • Image exclusion (NON_TEXT_BODY_KEYS) and the "images never doom" rule untouched; side-request guard tolerance (1.15) and headroom reservation untouched; error JSON shapes (egress_window_fuse / preflight_compress_failed / side_request_payload_too_large, retryable:false) untouched.
  • otherTop is pure diagnostics (descending per-top-level-field counts inside other); measuredBreakdownClause gains the top-field name only in the three 413 messages.

Regression (②): existing behaviors on the touched paths, item by item: byte-floor fallback for unparseable bodies — kept; message-count gate — kept; output-budget read — kept; fail-fast 413 for a genuinely over-window prompt — kept and pinned by the new negative e2e (1.2M-CJK prompt + 2 MiB upload still dooms, upstream sees zero calls); the only semantic change is the gate criterion itself (whole-body → prompt mass), disclosed old→new in the PR body. No fail-fast→swallow, no threshold/timing drift, no cache/retry impact — the fix actually unblocks the watermark-deadlock path the aggregation thread describes.

Tests (④): real teeth. Two new e2e cases through the live proxy pin BOTH directions (#3617): 8 MiB dsh_session_log + fitting prompt → 200, forwarded exactly once, upload still rides the wire byte-exact; over-window prompt → 413, zero upstream calls. Five per-wire prompt-mass unit tests assert against direct leaf counts (including the honest note that google's contents/systemInstruction sit in the other bucket since the bucket split is openai-shaped). Five otherTop tests pin attribution, nesting inheritance, empty case, clause naming, and the bare clause. Stale old-wording pins: none left in tests/ (grep-verified).

Diff (⑤/⑤b): 9 files, +335/−47, every line in scope; no whitespace churn, no generated/lock files; naming, comments (why, with issue refs), and style match the file.

Findings (minor, PR-body only — I cannot edit the source PR body, owner to amend before merge if desired):

  1. The 413 doom wording is model-facing error text (§7.3 lists "error wordings" as prompt surface). The change is disclosed under "Behavior changes" fix: address all code review findings (condense, routing, session-id, dead code) #2, but the PR body does not open with the required ⚠️ MODEL-FACING TEXT CHANGE marker, and neither commit message carries it. I walked the consumption chain: the new quantity is the same class (more accurate), and the new top-field name+count is a real diagnostic with no phantom value — so no regression risk, only the missing flag.
  2. The body says the otherTop diagnostic was ported "from PR [PR] fix: exclude model-hidden DSH fields from the measured body (#6844) #6860's otherTop diagnostic, commit 60edc7c740" — 60edc7c740 is THIS PR's commit; the corresponding [PR] fix: exclude model-hidden DSH fields from the measured body (#6844) #6860 commit is 2542369c0b. One-line fix to the reference.

Local verification: targeted suites 79/79 green (the 8 files the PR lists, run on the PR head) + tsc --noEmit clean. Full-suite verdict = remote CI on the pushed head: 0 failed, 11 pending, 19 skipped (cache read 2026-11-17 20:19 UTC) — CI still running, not red.

八问

  1. 是什么问题: 出站三道门(egress fuse、preflight 两道 measured 门、side-request guard)拿"整个出站 body"去比模型窗口,但窗口预算只约束 prompt 质量——DSH 挂在 body 顶层的 model-hidden 字段 dsh_session_log(最大 8 MiB)被计入后,普通 DSH 会话被永久 413,水位不前进,死锁。
  2. 什么原因导致的: 计量口径错了——estimateRawBodyBreakdown 的 total 把非 prompt 的顶层字段(telemetry 上传、配置标量)和 prompt 一起计价,而窗口只消耗 prompt。根因在"用什么量去比窗口",不在某个具体字段,所以修法是按 wire 协议声明 prompt 键(PROMPT_BODY_KEYS),而不是硬编码排除某个插件的字段(后者是 [PR] fix: exclude model-hidden DSH fields from the measured body (#6844) #6860 被否掉的方案)。
  3. 影响范围与严重程度: 凡宿主在 body 顶层挂非 prompt 大字段且 prompt 本身超门限附近的会话——当前实证为 DSH 0.2.2-rc.x 的 session-log 插件(三个独立复现:0.1.214 冷载、0.1.215 增量死锁、0.1.214 会话中切模型);npm latest 0.1.216 未修。严重度:该场景下用户完全不能用(p1 成立);prompt 本身超窗的拒绝行为不变,无过度抑制。
  4. 怎么修复的: 计量只计 wire 协议的 prompt 顶层键(openai: messages+tools;anthropic: +system;google: contents+systemInstruction+tools;responses: input+instructions+tools),三道门改比 prompt 而非 total;total/estimateRawBodyTokens/rawInputTokens 保持 whole-body 语义不动;另把 413 分解里的 other 桶归属到具体顶层字段(otherTop,纯诊断,移植自 [PR] fix: exclude model-hidden DSH fields from the measured body (#6844) #6860)。
  5. 是否有回归问题: 无。逐项过:unparseable body 的字节兜底、消息数门、output-budget 读取、图片永不 doom、fail-fast 413 形状与 retryable:false、side-request 1.15 容差与 headroom 预留——全部保持;唯一语义变更就是门限判据本身(whole-body → prompt mass),PR 已按旧值→新值披露;负向 e2e 钉住"真超窗 prompt 仍 413"。
  6. 是否破坏缓存: 不涉及。缓存/重试/水位/去重键均未触碰;修复反而解开水位死锁路径。
  7. 是否改了设计方向: 不涉及架构转向;只把"窗口适配判定"的计量口径从 body 质量修正为 prompt 质量,与 [Bug] estimateRawBodyBreakdown counts non-prompt extension fields toward the fuse budget, causing legitimate requests to | [Bug] estimateRawBodyBreakdown 将非 prompt 扩展字段计入 fuse 预算,导致合法请求被 413 误杀 #7013 指定的方向一致,未引入任何新配置面。
  8. 是否引入了新的配置: 无。无新 env/config/默认值。

判定: 可合(mergeable,base 即当前 master;CI 0 失败/11 pending,待远端全量绿即可由 owner 合并;合并后请 owner 关闭被取代的 #7016 与 #6860)。两条 minor 发现均在 PR 正文层面(缺 model-facing 顶标、#6860 commit 引用笔误),不阻塞,留 owner 酌情修订。

@ZzXxYy1234

Copy link
Copy Markdown

真实 DSH 生产流量验证了本 PR 的方向,PROMPT_BODY_KEYS[anthropic] = {messages, tools, system} 与 DSH 的 wire body 完全对得上。数据来自给 fuse 分支加的一行零行为日志(DSH 0.2.2-rc.3 + bili 0.1.215,deepseek-account/deepseek-flash):

wireChars=9947234
keys=model:16|stream:4|messages:2561750|max_tokens:6|thinking:18|output_config:16|system:92785|tools:2498|dsh_session_log:7275441|dsh_plugin_packages:14564
  • 旧口径:~2,395,536 tokens (messages ~710,189 · tools ~549 · system ~24,937 · other ~1,755,712),窗口 1,048,576 → 413,且 retryable:false;
  • 本 PR 口径:prompt ≈ 710,189 + 24,937 + 549 ≈ 736K < 1,048,576 → 放行。修复正是我们需要的。

两个建议:

  1. 回归测试请用「真实形态」的混合 body(比纯构造更有说服力):{ model, messages:[…2.5MB…], system:"…", tools:[…], dsh_session_log:"x".repeat(7_000_000), dsh_plugin_packages:"{}" },窗口 1,048,576,断言 fuse 不触发、且 breakdown.prompt 不含 dsh_* 部分;再加一个对照:把同样的 7MB 内容搬进 messages 里,断言仍然触发。这样能同时钉住「剔除」和「没把排除做过头」两个方向。
  2. otherTop 字段级明细非常有用(我们就是靠它定位到 dsh_session_log 的),请保留。

一个待确认点(不在本 PR 改动文件里):客户端本地 doom gate 走的是 src/agent/native-intercept.ts 的 bodyOverWindowDetail()(src/agent/dsh-native.ts:613 调用),它按原始 body 字节估算:floorTokens = coreBytes/4,再乘 CALIBRATED_TEXT_TOKEN_RATIO = 1.14,只扣图片字节、不区分 prompt 与 transport 字段。9.9 MB 的 body 在这里 floor ≈ 2.4M/1.0M 都会被判超窗。我们这次是服务端 fuse 先响,没法判断 dsh lane 在「proxy 不可用 / 侧请求」路径上是否会被本地 gate 先拦。它算不算 #7090 这条根因的范围、是否需要同样改成 price prompt mass,麻烦维护者判断。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

感谢真实生产流量数据 —— 它独立印证了本 PR 的计量方向: PROMPT_BODY_KEYS[anthropic] = {messages, tools, system} 与 DSH wire body 顶层键完全对得上; 旧口径 ~2.4M tokens → 413, 本 PR 口径 prompt ≈ 736K < 1,048,576 → 放行, 与 PR 的计量模型一致。

关于两条建议:

  1. 真实形态混合 body 的回归测试: PR 现有 e2e 已同时钉住两个方向 —— 8MiB dsh_session_log + 窗内 prompt → 200 且只转发一次(钉"剔除"),1.2M CJK prompt + 2MiB 上传 → 413 且上游零调用(钉"没排除过头")。你建议的显式断言 breakdown.prompt 不含 dsh_* 部分是个好的小补充(把 prompt 组成本身钉死,而不只是行为结果),作者可酌情采纳;当前测试已覆盖行为契约,不构成合并阻塞。
  2. 保留 otherTop: 已是本 PR 的一部分(纯诊断,不改变 meter 语义),保留。

关于待确认点(客户端本地 doom gate): 我已对照代码逐行核实,判断准确 ——

  • bodyOverWindowDetail(src/agent/native-intercept.ts:957-973)按整个 body 字节(仅扣 media)计价: 早期放行门 :966 用 bodyBytes/4 × 1.14,:968 coreBytes = bodyBytes - mediaBytes 不区分 prompt 与 transport 字段;
  • 调用点: src/agent/dsh-native.ts:2362/:2393(takeover gate)、src/agent/native-intercept.ts:1606/:1812(proxy-down direct-fallback);
  • 你的 9.9MB body 在此: floorTokens ≈ ceil(9,947,234/4) ≈ 2.49M > 1.05M → 本地拒,而 prompt ≈ 736K 本应在窗内。

这属本 PR 之外的另一条代码路径(客户端字节估算 vs 服务端 token 分解,不同故障域),且是否改 price prompt mass 是设计决策(客户端 gate 的 cheap-first 字节估算是有意的,见 native-intercept.ts:941-956 注释),需 owner 拍板。按"顺提"规则我已单独立项跟踪: [Drive-by] client-local doom gate prices the whole body, not the prompt mass (proxy-down / side-request path),内含代码核实、实测数据、三个选项(各带代价)+ 推荐(选项 A: 复用 PROMPT_BODY_KEYS 让客户端也 price prompt mass,与服务端修复一致、成本同阶)。它不阻塞本 PR 合并(不同代码路径)。

判定不变: 可合。 CI 现已全绿(10 success / 20 skipped / 0 failed / 0 pending)。待确认点已转出本线程跟踪,属 #7090 根因家族的另一故障域,由 owner 决定是否并入 #7090 清单或单列修复。

@ranxianglei
ranxianglei force-pushed the 2026-11-18_rcagg-doom-gate-pricing-mismatch branch from 60edc7c to 4763ab5 Compare November 18, 2026 02:25

@ZzXxYy1234 ZzXxYy1234 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

逐行读了 PR head(4763ab5 fix: show the dominating other-field's byte size in the measured-doom 413)。结论:方向正确,服务端三个消费点(egress fuse / post-compress recheck / side-request guard)都改到了,回归测试也覆盖了「大非 prompt 字段放行」与「prompt 真超窗仍然拦」两个方向。我用真实 DSH 流量核对过,数值吻合:

wireChars=9,947,234
messages:2,561,750 | system:92,785 | tools:2,498 | dsh_session_log:7,275,441 | dsh_plugin_packages:14,564
旧口径 2,395,536 tokens → 413
新口径 prompt ≈ 736K < 1,048,576 → 放行

四条建议,按重要性排序:

1. handle.ts:1874 的 rawInputTokens 仍按整 body 计价

session.metadata.rawInputTokens = estimateRawBodyTokens(parsed) + imageTokensInParsedBody(...)

如果这个值参与 usage baseline / preflight 触发(usageGradeInputBaseline 那条链),那么 DSH 这类会话里它仍是 ~2.4M,而 fuse 现在量到 736K——同一份 body 两套口径。建议确认消费方;若它只是「原始体积」指标,建议改名(如 rawBodyTokens)或加注释,避免后续维护者拿它当窗口依据。

2. otherTop 在 responses / google 协议下会把 prompt 字段标成 non-prompt

estimateRawBodyBreakdown 的分桶是 openai/anthropic 形状:input、instructions(responses)、systemInstruction(google)都落在 bucket 3(other),但同时 isPrompt=true。于是 otherTop 会列出它们,而 measuredBreakdownClause 会给整条 other 贴上 "(non-prompt, excluded from the window budget)" —— 对这两种协议是错误描述,排查时会把一个 prompt 字段当成 host 上传字段。建议 otherTop 只收集 isPrompt === false 的字段,或在元素里带 prompt: boolean。

3. PROMPT_BODY_KEYS[protocol] 取不到时 fail-open

measuredOutbound 与 side-request.ts:959 都直接索引 PROMPT_BODY_KEYS[protocol]。若运行时协议串超出联合类型(头部推断、新协议上线),得到 undefined → prompt = 0 → fuse 永不触发。建议 undefined 时回退 estimateRawBodyTokens(parsed)(整 body,保守方向),或至少 log 一次。

4. 建议补一条贴近真实形态的边界回归

现有测试已覆盖「8MB upload + 小 prompt 放行」与「1.2M CJK prompt + 2MB upload 拦」。建议再加一条混合断言,把「剔除」和「没剔过头」一次钉死:

  • 2.5MB messages(≈710K tok)+ 92KB system + 7.3MB dsh_session_log,窗口 1,048,576 → 放行;
  • 把同一份 7.3MB 内容搬进 messages → 仍然 413。

一点范围外提醒

客户端本地 doom gate(src/agent/native-intercept.ts:957 的 bodyOverWindowDetail,整体字节 ÷4 ×1.14)在同一份 body 上仍会算 ~2.4M,proxy 不可用 / 侧请求路径上会本地误拒。那条已单独登记,由 owner 决定并入本根因还是单列;这里只提醒它和本 PR 是「同一根因、不同代码路径」。

@ZzXxYy1234

Copy link
Copy Markdown

更正我上面 review 的第 1 条:我把 rawInputTokens 的消费链追完了,它不参与窗口判断,只喂给 session GC 的尺寸闸门。

全仓(含 kernel/plugins,排除 node_modules)只有两处引用:

  • 写入:src/server/handle.ts:1874 session.metadata.rawInputTokens = estimateRawBodyTokens(parsed) + imageTokensInParsedBody(...)(per-turn 记录,[Feature] Default state file cleanup strategy #1082,随 envelope 持久化);
  • 读取:src/session-gc.ts:129 viewFromParsed 取 metadata.rawInputTokens → :165 const base = view.rawInputTokens ?? view.contextTokens → isGcEligible:只有「age > 7d 且从未压缩 且 base + storedTokens <= sessions.gc.maxTokens(默认 1M)」才删文件。

而且 GC 是 opt-in(BILI_SESSION_GC / sessions.gc.enabled,默认 off)。

所以实际影响只有一个、且方向安全:如果用户开了 GC,DSH 这类会话因为 upload 字段被算进 rawInputTokens(实测 2,395,536 > 默认 1M),会永远落不进「小会话」那一档 → 文件一直保留。不会误删、不影响熔断。

结论:这条不需要在本 PR 里改,最多在 rawInputTokens 的注释里说明「含 wire 上的非 prompt 上传字段,仅用于 GC 尺寸档」,避免以后有人把它当窗口依据。我原先说的『两套口径』只说对了一半——口径确实不一致,但不落在同一条判定链上。

@ZzXxYy1234

Copy link
Copy Markdown

第 2 条的实测影响面,已经跑出来了:只影响 413 文案,不影响判定数学;但文案在 google / responses 两种协议下自相矛盾。

影响面:全仓只有两个消费者用 measuredBreakdownClause —— relay.ts:1047 的 egress fuse 与 server.ts:4586/4690 的两个 measured-doom 消息,也就是操作者读到的全部 413 文本。

按 PR 的实现原样复算(1.2M CJK 字符):

GOOGLE    {contents:[…1.2M CJK…], systemInstruction:"you are helpful", tools:[…]}
  prompt=1,200,009  other=1,200,008  otherTop=[contents 1,200,001 / systemInstruction 4 / model 3]
  message: messages ~0 / tools[] ~4 / system ~0 / other ~1,200,008
           (non-prompt, excluded from the window budget; top field: contents ~1,200,001 tokens ≈ 3,600,004 B)

RESPONSES {input:[…1.2M CJK…], instructions:"be brief"}
  prompt=1,200,006  other=1,200,008  otherTop=[input 1,200,004 / instructions 2]
  message: … other ~1,200,008 (non-prompt, excluded from the window budget; top field: input ~1,200,004 tokens …)

ANTHROPIC(对照,文案正确)
  messages 1,200,001 / other ~1,750,004 (non-prompt … top field: dsh_session_log)

问题在于:google/responses 的 prompt 键(contents / systemInstruction / input / instructions)都落在 bucket 3,于是文案同时说了两件互相矛盾的事——「other 是 non-prompt、已排除出窗口预算」,而这次 413 恰恰就是这段 other 触发的;并且 messages ~0 tools[] ~0 system ~0 会让操作者以为没有任何维度超窗,实际 actionable 的方向(trim contents / input)被标成了「非 prompt、不计窗口」。anthropic 那行是对的,说明只是分桶命名形状绑定了 openai/anthropic。

最小修法(不改变判定数学)

给 otherTop 的每个元素加一个 prompt: boolean(walk 里 isPrompt 已有,只是没带出去),然后文案里把 other 拆成两段:

const nonPrompt = b.otherTop.filter(e => !e.prompt);
const promptInOther = b.other - nonPrompt.reduce((s, e) => s + e.tokens, 0);
const top = nonPrompt[0];
const other = b.other > 0
    ? promptInOther > 0
        ? `other ~${fmt(b.other)} (of which prompt ~${fmt(promptInOther)} — IS counted in the window budget; top non-prompt field: ${top ? `${top.field} ~${fmt(top.tokens)} tokens` : "none"})`
        : `other ~${fmt(b.other)} (non-prompt, excluded from the window budget; top field: ${top.field} ~${fmt(top.tokens)} tokens ≈ ${fmt(top.bytes)} B)`
    : `other ~${fmt(b.other)} (non-prompt, excluded from the window budget)`;

这样 anthropic 的文案一字不变(promptInOther===0 走原分支),google/responses 会变成「other 里有 1.2M 是 prompt、计入窗口」,和开头的 outbound prompt measures 自洽。

建议补的断言

在现有 #7013 的 e2e 里各加一个 google / responses 形态:断言 413 的 message 不包含 non-prompt, excluded from the window budget; top field: contents(或 input),且 breakdown.otherTop[0].prompt === true;另加一条单测钉住 otherTop 的 prompt 标志(现有 #6844 那条只覆盖 anthropic 形状)。

如果 owner 认为 other 在概念上就不该装 prompt,也可以反过来做成「bucket 按 prompt 归属划分」(contents/input/instructions/systemInstruction 归 messages/system 桶),但那样要改 messages/tools/system 三个标签的语义和现有测试,改动比加标志大。

@ZzXxYy1234

Copy link
Copy Markdown

Follow-up on my review item ③ (PROMPT_BODY_KEYS[protocol] undefined ⇒ prompt = 0 ⇒ token gate never fires). After tracing every producer of protocol I'm downgrading this from "bug" to "hardening suggestion" — as written it is not reachable today, and I don't want the review to leave a false alarm:

Sources of protocol, all union-typed and runtime-validated:

  1. route.explicitProtocol — only set by the /bili/<protocol>/http(s)://… marker, matched against KNOWN_PROTOCOLS = ["responses","anthropic","openai","google"] in src/server.ts:250-300;
  2. providers.<key>.protocol — parsed by parseDeclaredWireProtocol (src/config.ts:3143), which throws on anything outside those four members (strict [feature] configurable wire-path table (per-provider custom paths) - Claude Desktop's claude.ai private protocol as motivating case #1909 stance);
  3. declared.wire from modelEndpointPatterns — commandcode is mapped to openai at src/server/handle.ts:458-474;
  4. path-suffix fallbacks (/chat/completions, /messages, /responses, googlePathKind) and inferWireProtocol return literals.

So PROMPT_BODY_KEYS[prepared.protocol] is always defined, and all four gate call sites pass it (src/server.ts:3820, src/server.ts:4547, src/server/side-request.ts:959).

What is still worth hardening (opt-in, no behavior change for valid configs): estimateRawBodyBreakdown(parsed, promptKeys?) takes an optional key set, so a future protocol added to WireProtocol without a PROMPT_BODY_KEYS entry would silently price 0 and open the gate instead of closing it. Suggest falling back to whole-body pricing when promptKeys is absent (fail-closed), plus a test that enumerates every WireProtocol member and asserts its entry exists.

The client-side counterpart of this family is already tracked in #7106, so no new issue from me.

@ZzXxYy1234

Copy link
Copy Markdown

Concretizing my review item ② (google/responses prompt fields mislabeled as non-prompt) with a patch on top of this PR's head (4763ab5):

Before — a google body whose contents alone blows the window renders:

messages ~0 · tools[] ~0 · system ~0 · other ~1,200,008 (non-prompt, excluded from the window budget; top field: contents ~1,200,001 tokens ≈3,600,004 B)

…i.e. the 413 names the excluded bucket as the reason, while prompt (the number that actually tripped the gate) is 1,200,009 from that very field. Same for responses/input.

Patch (one commit, ZzXxYy1234:fix/7013-prompt-mass-clause, based on this branch head — cherry-pick c27f5ea or take as-is):

  • src/preflight.ts: RawBodyBreakdown gains otherPrompt — the part of other that is still prompt mass for the wire format (google contents/systemInstruction, responses input/instructions). Accumulated at the same string leaf where prompt is accumulated, so it never drifts from the gate's number.
  • src/server.ts: measuredBreakdownClause splits other on it →
    other ~1,201,100 (prompt mass ~1,200,009 counted toward the window, 1,091 excluded as non-prompt; top field: contents …).
    When otherPrompt === 0 (anthropic/openai, where other really is transport) the previous wording is byte-identical, so existing assertions hold.
  • tests/issue7013-prompt-mass-google-responses.test.ts (new, 6 cases): google/responses prompt-in-other attribution, the 413-clause wording for a prompt-bearing top field, and anthropic/openai controls that keep dsh_session_log classified as excluded transport.
  • tests/raw-body-walk-depth.test.ts: one expectation gains otherPrompt: 0 (the breakdown object gained a field).

Verification (from the patch branch):

  • node --import ./tests/preload-test-env.mjs --import tsx --test tests/issue7013-prompt-mass-google-responses.test.ts → 6/6 pass
  • … --test tests/raw-body-walk-depth.test.ts tests/issue4079-egress-window-fuse.test.ts → 22/22 pass (no wording/behavior regression for anthropic)
  • npx tsc --noEmit --project tsconfig.json → clean

I don't want to open a second PR for the same domain (#7106 already tracks the client-side half), so this is posted as a ready-to-apply patch; happy to turn it into a stacked PR if you prefer that over cherry-picking.

@ZzXxYy1234

Copy link
Copy Markdown

Opened the PR-shaped path after all: #7118 — base = 2026-11-18_rcagg-doom-gate-pricing-mismatch (this branch), head ZzXxYy1234:fix/7013-prompt-mass-clause, 1 commit, 4 files (+136/−4): the otherPrompt split in preflight.ts, the clause wording in server.ts, the new google/responses test file, and the raw-body-walk-depth expectation update.

Merge order: land #7092 first, then #7118 rebases onto master cleanly (it is one commit on top of 4763ab5). If you would rather keep a single PR thread, git cherry-pick c27f5ea into this branch gives the identical tree and #7118 can be closed.

@ZzXxYy1234

Copy link
Copy Markdown

ccfee8a 的「修分桶」把 otherPrompt 那条思路彻底取代了,基于这条分支的 stacked PR #7118 已关闭。

客户端那条残面(#7106)改成基于 master 独立实现:PR #7119(只动 src/agent/native-intercept.ts + 新 src/agent/prompt-mass.ts,按形状识别协议后只对 prompt 键计价,保持 #4111 两档数学与媒体排除)。本 PR 不涉及客户端 gate。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

逐条回复 + 状态更新:

ccfee8a(re-bucketing)取代 otherPrompt — 确认。prompt 键归入命名桶后 other 永远非 prompt,413 措辞自洽;不变量 prompt===messages+tools+system、prompt+other===total 按 lane 钉死。#7118 关闭、客户端残面转 #7119(基于 master 独立实现)均合理,本 PR 不涉及客户端 gate。

m00222(413 措辞自相矛盾) — 证实为真,ccfee8ad99 从根上修复:对 google/responses,prompt 字段(contents/input/systemInstruction/instructions)原先落入 anthropic 命名的 other 桶却同时被计入 breakdown.prompt,导致 413 措辞对触发它的字段说 "non-prompt, excluded"。re-bucketing 使 other 永远非 prompt,矛盾消除。

m00223(PROMPT_BODY_KEYS 加固) — 已顺手修掉,push 为 2257410:三个 gate 调用点(measuredOutbound、measured doom tier、sideRequestGuard)统一走新 promptMassForProtocol helper,协议无键表时回退整包计价(fail-closed,而非静默计 0 打开闸门);新增枚举测试补运行时非空守卫(Record<WireProtocol,…> 类型已在编译期强制存在性)。四协议行为不变,typecheck 干净,定向 83/83 全绿。

m00209(rawInputTokens 更正) — 结论正确,已核实:session.metadata.rawInputTokens(src/server/handle.ts:1834,整包 estimateRawBodyTokens,#1082)只被 session-GC 大小档消费(src/session-gc.ts:129/133/137/165),GC 默认关(BILI_SESSION_GC/sessions.gc.enabled),不参与窗口判定,本 PR 无需改动。

CI:0 failed / 7 pending(22574103f4 运行中),mergeable_state=behind(落后 master,非阻塞)。


八问(PR #7092 全量,含 ccfee8a + 2257410):

  1. 是什么问题:出站熔断(egress fuse)与 doom 闸门按整包体积计价,把非 prompt 字段(dsh_session_log 遥测上传、配置标量)算进窗口预算,DSH 88%-other 413 误伤([Bug][dsh] measured-body egress fuse meters a model-hidden DSH field: 88% of a refused body was dsh_session_log inside the other bucket #6844/[Bug] estimateRawBodyBreakdown counts non-prompt extension fields toward the fuse budget, causing legitimate requests to | [Bug] estimateRawBodyBreakdown 将非 prompt 扩展字段计入 fuse 预算,导致合法请求被 413 误杀 #7013/dsh #7088)。
  2. 什么原因导致的:计量器量错量——该量 prompt 质量(模型要读的上下文),却量了整包体积(含上传/遥测)。根因在计量口径,不在阈值。
  3. 影响范围与严重程度:带大非 prompt 上传且 prompt 本身在窗口内的出站请求被 413 误伤,prompt 能发也发不出去。p1(特定条件下用户不能用)。
  4. 怎么修复的:按 wire 协议定义 PROMPT_BODY_KEYS(openai {messages,tools}、anthropic {messages,tools,system}、google {contents,systemInstruction,tools}、responses {input,instructions,tools}),闸门只计 prompt 质量;ccfee8ad99 把 prompt 键归入命名桶(other 永远非 prompt,措辞自洽);22574103f4 加 fail-closed 回退 + 枚举测试。
  5. 是否有回归问题:无。byte-floor/count-gate/output-budget/image-exclusion/fail-fast 形状/1.15 容差全部保持;唯一语义变更 = 闸门判据从整包→prompt 质量(已披露旧值→新值);rawInputTokens([Feature] Default state file cleanup strategy #1082,session-GC 用)不变。
  6. 是否破坏缓存:不涉及。缓存/重试/增量游标/去重键均不受影响。
  7. 是否改了设计方向:不涉及。计量口径整包→prompt 质量是量对量(bug 修复),非架构转向;PROMPT_BODY_KEYS 是 wire 协议常量,非新配置面。
  8. 是否引入了新的配置:无。PROMPT_BODY_KEYS 是代码常量(按协议硬编码),非 env/config/CLI。

判定:可合。方向正确(根因 = 计量器量错量;按 wire prompt 键 = #7013 的更彻底方案,无新配置面),无回归,测试有牙(双向钉死 #3617),diff 干净。建议 owner 合并后关闭 #7016(被本 PR 完全吸收)与 #6860(机制被取代)。

ci-check and others added 10 commits November 19, 2026 05:44
…the whole body (#7013)

estimateRawBodyBreakdown now also reports the wire-format's prompt mass
(messages/tools/system per protocol), and the window-fit gates (egress
fuse, preflight doom tiers, side-request guard) use it instead of the
whole-body total. A non-prompt upload field (e.g. DSH's dsh_session_log,
telemetry that rides the wire but consumes no window) no longer dooms a
fitting prompt.

- preflight: estimateRawBodyBreakdown gains a promptKeys param and a
  `prompt` field on RawBodyBreakdown; PROMPT_BODY_KEYS maps each wire
  protocol to its prompt top-level keys.
- server: measuredOutbound now requires protocol and returns the prompt
  mass; the two preflight doom tiers use outboundBreakdown.prompt.
- relay: the egress fuse call passes prepared.protocol.
- side-request: the guard estimates the prompt mass, not the whole body.
- 413 messages reworded body→prompt, noting the other bucket is excluded.

Consumers kept as whole-body total (not window-fit):
- handle.ts rawInputTokens (#1082 GC rebuild-cost signal).
- threads-lane synthetic prompt-only shapes.

Tests: raw-body-walk-depth (prompt mass per protocol + a giant non-prompt
field excluded), side-request-isolation (guard uses prompt mass),
issue4079 egress fuse (both-ways: giant upload + small prompt forwards;
over-window prompt still dooms), issue4185 (doom wording updated).

Closes #7013

(cherry picked from commit ff258d0)
)

The preflight doom messages now say 'the prompt ITSELF exceeds the
window' (prompt mass, not whole body). The zero-fold calibration test
asserted the old 'body ITSELF' wording; update it to the new wording.

Closes #7013

(cherry picked from commit 8cd3690)
… dominating other (#6844)

#6844's 88%-other body took a manual body dump to attribute: which
host-attached field carries the non-prompt mass. estimateRawBodyBreakdown
now tracks per-top-level-field token counts inside the other bucket
(otherTop, descending) and measuredBreakdownClause names the dominating
field in the 413 reason. No meter semantics change: total, the four
buckets and prompt are byte-identical to before; otherTop is pure
diagnostics.

Ported from sx4im/billion-context PR #6860's otherTop diagnostic, which
is independent of that PR's hardcoded-field exemption (superseded by the
prompt-mass metering in bccd147).
… 413 (#7090)

The 413 clause names the top-level field dominating the non-prompt `other`
bucket, but only in tokens. DSH's dsh_session_log upload is capped in BYTES
(8 MiB), and 8 MiB of session log is routinely truncated or backlogged — so
bytes localize the truncation better than a token estimate.

Additive: RawBodyBreakdown.otherTop entries gain a `bytes` field (UTF-8 byte
length of the field's string content, sum of leaves — an approximation of the
field's JSON size, excluding syntax overhead). The clause now reads
`top field: dsh_session_log ~1,755,712 tokens ≈7,275,441 B`. No change to
total/prompt/estimateRawBodyTokens; the bytes are diagnostic-only and are never
priced against the window budget.

Field data motivating this (from an independent repro on DSH 0.2.2-rc.3 +
bili 0.1.215, anthropic lane): wireChars=9947234 with dsh_session_log alone at
7,275,441 chars (94% of the wire) — the real prompt (messages+system+tools) was
~736K tokens, under the 1,048,576 window, so the session should have forwarded
but 413'd on the whole-body meter.
…y lane (#7090)

The 413 clause says 'non-prompt fields in the other bucket are excluded from
the window budget', but the display buckets (messages/tools/system/other) were
split on the openai/anthropic key names only. On the responses lane the main
prompt content `input` (and on the google lane `contents` /
`systemInstruction`) fell into the `other` catch-all — so the clause would
label prompt mass as non-prompt, and an operator reading 'other ~1.1M
(non-prompt, excluded)' would misread their own conversation as the culprit.

The gate already priced `prompt` correctly (PROMPT_BODY_KEYS); this is the
display-side half of the same fix. With promptKeys supplied, each prompt key
now lands in its named display bucket (PROMPT_KEY_BUCKET: messages/contents/
input→messages, tools→tools, system/systemInstruction/instructions→system) and
everything else in `other`. The invariants now hold on every lane:
`prompt === messages + tools + system` and `prompt + other === total`.

total / estimateRawBodyTokens / prompt are unchanged; only the diagnostic
split moves. Model-facing: the 413 clause now shows the responses lane's
`input` under 'messages ~N' (where it belongs) instead of 'other ~N
(non-prompt, excluded)'.

Field data (independent repro on DSH 0.2.2-rc.3 + bili 0.1.215, responses
lane): a 4.7 MiB /v1/responses body whose top-level keys are model/input/
stream/prompt_cache_key/store/max_output_tokens/tools/reasoning/include —
no dsh_session_log on this lane; `input` alone was 1,141,965 tokens and
previously sat in `other`.
…Y_KEYS (#7013)

Route the three gate call sites (measuredOutbound, measured doom tier,
sideRequestGuard) through a promptMassForProtocol helper that falls back to
whole-body pricing (breakdown.total) when the protocol has no keys entry,
instead of silently pricing 0 (which would OPEN the gate). The
Record<WireProtocol, ...> type enforces existence at compile time; a new
enumeration test adds the runtime non-empty guard. No behavior change for the
four current protocols.
…body bytes (#7106)

The DSH client gate (bodyOverWindowDetail) estimated tokens from the whole
outbound body (bytes/4, x1.14) with no notion of which bytes are prompt. On
official DeepSeek lanes the body also carries dsh_session_log - a
transport-only session-log contribution of up to 8 MiB - so a prompt well
inside the declared window (e.g. 736K < 1,048,576) was refused locally with
doom-gate-pricing-mismatch.

When the body parses, the gate now prices only the protocol's prompt keys
(anthropic messages/tools/system, openai messages/tools, google
contents/systemInstruction/tools, responses input/instructions/tools) with
the same CJK-aware counter the server uses, and keeps the #4111 two-tier
math: calibrated tier = floor x 1.14, bare floor = provably-over tier.
Inline media stays excluded (as it has since #4111); transport-only fields
are dropped from the floor. Unparseable bodies keep the whole-body byte
estimate, with the early bound raised from bytes/4 to bytes/3 so CJK-dense
bodies are not missed. Refusal wording now says prompt mass is what counts.

Tests: tests/issue7106-client-doom-gate-prompt-mass.test.ts (6 cases:
transport-field-does-not-doom, genuinely-over prompt still dooms, CJK-aware
pricing, google/openai shape sniffing, byte fallback, image exclusion).
Regression: 179/180 across the gate suites; the one failure (#3578) is a
pre-existing Windows temp-dir EPERM cleanup flake, reproduced on clean
master.

(cherry picked from commit 266b286)
…e in the architecture map (#7106)

(cherry picked from commit a01db3a)
- drop the dead imageTokens field from WirePromptMass (computed but never
  consumed; the text walk already excludes image keys, #4111)
- remove the accidental UTF-8 BOM that landed on native-intercept.ts line 1
- fix the stale CJK test comment (400K/1.2MB -> 1.1M/3.3MB) and its mojibake
- rename the image test to match what it asserts (image bytes excluded, not
  "added back")
- restore the trailing newline and drop a doubled blank line in the test file

(cherry picked from commit 97c7c30)
@zhang-ming-hui

Copy link
Copy Markdown

(ai: deepseek/deepseek-flash)

Independent confirmation that this PR addresses what users hit on the released build, not just master — DSH 0.2.2-rc.3 + billion-context 0.1.216, anthropic lane, forked session's first turn:

413: outbound body measures ~1,836,642 tokens
     (messages ~212,225 · tools[] ~21,658 · system ~5,183 · other ~1,597,576)
     vs the effective model window 1,048,576

other = one dsh_session_log envelope (~8 MiB; the child's first request carries the whole ancestor log) + dsh_plugin_packages (~14 KB). Prompt mass was ~240K, so pricing breakdown.prompt is exactly the right axis. Same session after a rewind produced ~2,014,210 (messages ~738,747 · other ~1,249,484) — other dominant again.

Two things worth knowing for the merge/release note:

  1. dsh_plugin_packages is ~14 KB on every DSH request, so this isn't only a fork/first-request bug — it is simply survivable in small sessions and fatal once dsh_session_log grows.
  2. Until a release ships with this, 0.1.216 users cannot get past the fuse, and the 413 text still names messages[] (the smallest bucket) as the thing to trim; the otherTop part of this PR is what removes that misleading half.

If it helps, I can re-test the released build on the same two sessions (forked-child first turn, and the rewind case) once it's out.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

Thanks for the independent confirmation on the released build — this is exactly the data that validates the axis the PR picks.

Your numbers line up with the fix. Forked-child first turn: whole outbound body ~1.84M tokens, but the prompt mass (messages+tools+system) is only ~240K — well under the 1,048,576 window. The ~1.6M other is the dsh_session_log envelope (~8 MiB) + dsh_plugin_packages (~14 KB), which never reach the model. Pricing breakdown.prompt (what this PR does) lets the request through; pricing the whole body (0.1.216) trips the fuse. The rewind case (~2.01M, other ~1.25M dominant) is the same shape: prompt fits, transport-only other doesn't.

Both release-note points verified against the code:

  1. dsh_plugin_packages ~14 KB on every DSH request — consistent with your read; it means the bug is latent in small sessions and only fatal once dsh_session_log grows, not a fork-only path. (That's a DSH-side fact; the bili side just prices what it's given.)
  2. The 413 wording — confirmed. In 0.1.216 the doom clause's only trim hint is the generic "e.g. a giant paste or tool result in messages[], or tools[] schemas" (src/server.ts:4564 at the pre-PR base), which points at messages[] (~212K, far below the ~1.6M other) when the real culprit is other. The otherTop part of this PR (src/server.ts:3805) adds "top field: dsh_session_log ~1,597,576 tokens ≈8,000,000 B" so the named dimension is the actual one. That misleading half is gone once this ships.

Two things that changed since my last review (for the record):

On shipping: the release is the owner's call (I can't cut releases). Your offer to re-test the released build on the forked-child + rewind sessions once it's out is exactly the right validation — the two sessions you named are the repro.

Verdict unchanged: 可合. The data independently confirms the axis; the expanded client-side fix is correct and tested; CI green.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷

(ai: vllm-qwen/qwen3.8-27b)

交叉引用:本 PR 与后开的 PR #7275("name top-level keys when the egress window fuse fires",针对 #7273 的 DSH 413 报告)是同机制家族(egress fuse 定价 + 413 字段定位)的两个在途 PR——两者改 src/server/relay.ts 同一行 fuseMessage、src/preflight.ts / src/server.ts 同一区域,且对同一设计问题(非-prompt 顶层字段是否计入 fuse 预算)持相反立场(本 PR 定价 prompt mass;#7275 保持 whole-body 基准并新增 whole-body top-key 视图)。

#7275 的审核已完成,结论:代码过审,但判为 duplicate of 本 PR(先到者保留),已打 duplicate + needs-decision 标签、不关闭(关闭权在 owner)。owner 决策请求(选项 A/B/C + 推荐 A:#7275 挂起、先合本 PR)已发在 #7275 线程(🔶 需要决策 评论)。

本 PR 侧无需动作,继续正常 review 流程即可;#7275 在决策前保持 open 挂起、不合并。

@g-yixuan

Copy link
Copy Markdown

(ai: openai/gpt-6)

补充来自 g-yixuan/dsh-sidenote#103 的独立复现与验证。核对到本 PR 已覆盖同一根因后,沿用现有修复,没有另开重复 PR。

验证对象:本 PR head 32eda3a34fd11c54ccbf4f2bda7b00b4d75f9c48,以及 CI 发布的固定测试包 billion-context@0.0.10900-pr.7092.1da8f04。本地 Node 24.18.0 / macOS;参考宿主模块为 DSH 0.2.2-rc.5。原报告没有 DSH 精确版本,因此下面是独立复现,不冒充其 Windows 环境回验。

使用真实 DSH SessionStore、Session-log 插件和请求扩展注册器,构造长历史并提交压缩后的 surface replacement,然后原生 fork。父子 deriveMessages() 深比较完全相同,正文均为 598196 字符;父下一次日志只有 308 字节,子首次日志为 6599244 字节。无需加载 sidenote,即可在 0.1.216 重现子会话的 egress_window_fuse 413。

把模块实际生成的请求及正反对照送入真实 bili HTTP 代理,本地模拟上游记录收到的请求数和日志附件哈希:

场景 0.1.216 本 PR 本地构建 固定 PR 测试包
约 15 万 token 正文,不带大日志 200 200 200
同一正文,附加约 6.38 MB 日志 本地 413,上游 0 次 200 200
真实 DSH 父会话请求 200 200 200
真实 DSH fork 子会话请求 本地 413,上游 0 次 200 200
真正超窗的正文 本地 413 本地 413 本地 413
小输出预算辅助请求附带大日志 本地 413 200 200

所有成功转发的大日志请求都校验了附件 JSON 的 SHA-256,转发前后完全一致。子会话请求的原始总量仍估算为 1799184 token,而窗口判断使用的正文量为 149565 token;保留原始计数、单独计算 prompt 的设计有效。真正超窗的对照仍未触达模拟上游。

本地检查:

  • npm run typecheck、npm run build 通过,构建的 import-annotations / external-dependency 检查通过。
  • 使用仓库 preload-test-env.mjs 隔离环境,9 个相关测试文件共 265/265 通过,0 skipped:raw-body-walk-depth、issue4079-egress-window-fuse、issue4185-identified-fresh-forward、preflight-zero-fold-calibration、side-request-isolation、issue7106-client-doom-gate-prompt-mass、dsh-native、issue4894-direct-window-guard、native-intercept。
  • 上表 6 场景在本地构建和固定测试包中各通过一次。
  • 本地构建与固定测试包的 dist/index.js SHA-256 相同:15033f6c3c1d99c3ad0d354e87686299df09ba9c9ba025201c61d6dc055e1ebf。
  • 核对本 head 的既有 CI run 为 success。没有把其他提交或本地目标测试当成全量 CI 结论。

这些结果支持使用本 PR 作为 sidenote #103 的上游修复。相较于单独给快照加预算,按协议计算模型输入能解决误判,同时保留日志上传和真正超窗保护。这里未调用真实模型 API;HTTP 200 证明请求完整通过本地拦截并转发,不代表已验证模型回答、供应商缓存命中率或报告者的现场配置。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

感谢这份独立复现——是目前对根因覆盖最完整的一份证据,且你核对到本 PR 已覆盖同一根因后没有另开重复 PR,处理得很干净。

CI 复核:本 head 32eda3a34fd 的 CI 仍为绿(镜像缓存 2026-11-19T09:07Z:14 success / 0 failed / 16 skipped),与你引用的 run 44406086858 success 一致。

6 场景 A/B 表与 PR 行为逐行吻合:

  • 大日志 + 小正文(~150K 正文 + 6.38 MB 日志):0.1.216 本地 413、上游 0 次;本 PR 200 且只转发一次——正是「按 prompt mass 计价」的预期:日志是 transport-only,不进窗口预算,正文在窗内就放行。
  • fork 子会话(6.6 MB 日志):0.1.216 本地 413;本 PR 200——与 @zhang-ming-hui 的 released-build 数据同形。
  • 真正超窗正文:三个构建都本地 413、未触达上游——闸门对真实超窗仍生效,没有因「只算 prompt」漏放。
  • 小输出预算辅助请求 + 大日志:0.1.216 本地 413;本 PR 200——side-request 车道同样改按 prompt mass 计价(src/server/side-request.ts:959)。

两个关键不变量你都验到了:

  • 附件 JSON SHA-256 转发前后一致——wire fidelity(上传字节 byte-exact)没被计价改动破坏。
  • 子会话原始总量仍估 1,799,184 token,窗口判断用 149,565 token——「保留 raw 计数、单独算 prompt」正是 PR 核心(rawInputTokens 只喂 session-GC 尺寸档,不参与窗口判定,这点之前已核过)。

产物一致性:本地构建与固定测试包 dist/index.js SHA-256 相同(15033f6c…)——你验的就是 CI 发布的那个产物,不是另一个 build;本地 265/265(9 文件)+ typecheck/build 通过。

边界限定采纳:HTTP 200 只证明请求完整通过本地拦截并转发,不代表模型回答 / 供应商缓存命中 / 报告者现场配置已验证——这个限定是对的,别读成 e2e 模型级验证。

结论不变:可合。两份独立数据(@zhang-ming-hui 的 released-build 413 + 你的 A/B 复现)从两个方向钉死同一根因与修法;CI 绿;wire fidelity 与真实超窗保护都保留。发版仍是 owner 的决策(我不能切 release);「用本 PR 作为 sidenote #103 的上游修复」方向成立,等 owner 拍板发版即可。

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment