Skip to content

[PR] fix(preflight): price host transport fields as hop mass, not prompt mass (#7013) - #7269

Closed
zhang-ming-hui wants to merge 1 commit into
ranxianglei:masterfrom
zhang-ming-hui:2026-11-19_fuse-hop-mass-not-prompt
Closed

zhang-ming-hui wants to merge 1 commit into
ranxianglei:masterfrom
zhang-ming-hui:2026-11-19_fuse-hop-mass-not-prompt

Conversation

@zhang-ming-hui

@zhang-ming-hui zhang-ming-hui commented Nov 19, 2026 •

Copy link
Copy Markdown

是什么问题

DSH 客户端会在模型请求体里附带网关自用的传输字段:dsh_session_log(会话增量日志,分叉/撤回后的首包会带整份祖先日志,实测 ~8 MiB)与 dsh_plugin_packages(插件清单,~14 KB)。它们和模型请求共用同一个 HTTP body,但从不进入任何 tokenizer。

egress window fuse 目前把它们当 prompt 计价:estimateRawBodyBreakdown() 把所有非 messages/tools/system 的顶层字段归进 other 桶并逐叶计价,measuredOutbound() 直接拿 breakdown.total 去和窗口比。于是「模型要读多少 token」被换成了「body 里有几 token 的字符串」,请求在本地被 413 拦下、根本没出机。

实测(DSH 0.2.2-rc.3 + billion-context 0.1.216,anthropic 车道):

413: outbound body measures ~1,836,642 tokens
     (messages ~212,225 · tools[] ~21,658 · system ~5,183 · other ~1,597,576)
     vs the effective model window 1,048,576

other 的 1,597,576 全部来自那份 dsh_session_log;真实 prompt 只有 ~240K。同一形态已上报 5 次(#7013 #7189 #6844 #7099 #7106),#7114 还记录着「pass-through vs egress strip」的 owner 待决项。

根本原因

计价函数把「HTTP 体里的字符串」等同于「模型要读的 token」:

  • src/preflight.ts:503 estimateRawBodyBreakdown():顶层 key 分桶,非 messages/tools/system 一律进 other;NON_TEXT_BODY_KEYS 只排除了 base64/URL 形状(data/url/b64_json/file_data/image_url),排不掉「整段 JSON 日志」。
  • src/server.ts:3793 measuredOutbound():return { tokens: breakdown.total, breakdown }。
  • src/server/relay.ts:1041 if (fuseRawTokens > fuseWindow):判据因此是原始体量,不是 prompt 量。

如何修复

只修计价口径,不碰传输层(§7.3:不靠过滤 payload 掩盖症状;要不要把 hop 字段发出去属 #7114 的 owner 待决项,本 PR 不替它决定):

  1. src/preflight.ts:新增 HOP_ONLY_BODY_KEYS —— 证据白名单,当前只有 dsh_session_log、dsh_plugin_packages 两条(同 KDD release v0.1.2 #9 纪律:只按流量证据逐条收,不做 dsh_ 前缀泛化,避免将来真正的 prompt 字段藏进来)。RawBodyBreakdown 增加 hop(⊆ other)与 promptTotal = total − hop;total 语义不变,仍等于 estimateRawBodyTokens(parsed),四个桶仍穷尽。
  2. src/server.ts:measuredOutbound() 返回 tokens: breakdown.promptTotal。fuse 与 estimate-only probe 门问的都是「上游会不会收」,该按模型真读的量算;breakdown.total 仍留给日志与 breakdown 文案。
  3. src/server/relay.ts:仅在 hop > 0 时补一条 warn,把「测得的 X 里有 Y 是传输元数据」写进日志,省得下一个人对着 other ~1.2M 猜。

不改任何模型可见文案(因此没有 MODEL-FACING TEXT CHANGE 标记);不新增配置键;不动 package.json。

影响面

测试

新增 tests/egress-fuse-hop-mass.test.ts,正反两向:

  • 顶层 hop 字段从 promptTotal 扣除,total 不变、四个桶仍穷尽;
  • 反向:同一个 key 嵌在 messages[] 里不得算 hop(hop === 0、promptTotal === total)——白名单最怕的就是吞掉真正的 prompt;
  • measuredOutbound() 的 gate 值等于 prompt 量,而 breakdown.total 仍是原始量。

既有 tests/side-request-isolation.test.ts 里 #5000 的三条断言(total === estimateRawBodyTokens(body)、桶穷尽、tools 主导可判)原样通过。本机按 AGENTS.md 只跑了这两个目标文件,全量以 CI 为准。

配置面

无新增、无变更配置键。

风险

低:纯计价口径收窄,白名单只有两个有流量证据的字段,total 未改。剩余风险是「将来再出现新的 host 传输字段」——按证据纪律逐条追加即可;反过来放开前缀匹配才会把真正的 prompt 字段藏进白名单,那是更大的风险。

Fixes #7013
Fixes #7189
Fixes #6844
Fixes #7099
Refs #7106 #7114

…nglei#7013)

dsh_session_log / dsh_plugin_packages ride the model request body but never reach a tokenizer; the raw walk counted them into other, so the egress window fuse refused sends whose prompt fit the window (ranxianglei#7013/ranxianglei#7189/ranxianglei#6844/ranxianglei#7099/ranxianglei#7106). Adds an evidence-permitlist HOP_ONLY_BODY_KEYS and reports hop/promptTotal on RawBodyBreakdown (total unchanged, buckets still exhaustive); measuredOutbound now prices promptTotal.
@ranxianglei ranxianglei changed the title fix(preflight): price host transport fields as hop mass, not prompt mass (#7013) [PR] fix(preflight): price host transport fields as hop mass, not prompt mass (#7013) Nov 19, 2026
@zhang-ming-hui

Copy link
Copy Markdown
Author

(ai: deepseek/deepseek-flash)

Closing as a duplicate of #7092 — I missed that the batch's root fix was already in review there, and #7092's design is the better one: it prices the wire protocol's prompt keys (PROMPT_BODY_KEYS + promptMassForProtocol, fail-closed on unknown lanes) instead of subtracting a fixed permitlist like this PR did. Mine would have covered dsh_session_log and dsh_plugin_packages and broken again on the next host field; #7092 covers the whole class.

For the record, what this branch independently reproduced on the released 0.1.216 (not just master), DSH 0.2.2-rc.3, anthropic lane, a forked session's first turn:

413: outbound body measures ~1,836,642 tokens
     (messages ~212,225 · tools[] ~21,658 · system ~5,183 · other ~1,597,576)
     vs the effective model window 1,048,576

other there was one dsh_session_log envelope (~8 MiB — the child's first request carried the whole ancestor log) plus dsh_plugin_packages (~14 KB); the real prompt was ~240K. Same session, after a rewind: ~2,014,210 (messages ~738,747 · other ~1,249,484).

Two operational notes, not blocking anything:

Thanks — happy to re-test the released build once #7092 lands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment