Skip to content

[PR] fix: exclude model-hidden DSH fields from the measured body (#6844) - #6860

Open
sx4im wants to merge 5 commits into
ranxianglei:masterfrom
sx4im:fix-6844-egress-fuse-model-hidden-field
Open

sx4im wants to merge 5 commits into
ranxianglei:masterfrom
sx4im:fix-6844-egress-fuse-model-hidden-field

Conversation

@sx4im

@sx4im sx4im commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

The measured-body meter counted every top-level body field into other, including DSH's model-hidden dsh_session_log diagnostics. An ordinary request 413'd at the egress fuse with other at 88% of the measured body while the model-visible buckets used 28% of budget.

What changed: estimateRawBodyBreakdown and estimateRawBodyTokens now skip the two DSH model-hidden top-level fields (dsh_session_log, dsh_plugin_packages). The same key nested inside messages[] is model-visible content and still counts. Genuinely oversized other fields are still counted.

How verified: new regression test tests/issue6844-measured-body-model-hidden-fields.test.ts (4/4, failed on master before the fix); neighbor suites raw-body-walk-depth and side-request-isolation pass (45/45). Typecheck is clean in the changed files; two session-gc failures and five dsh-tool-registration-lifetime type errors reproduce on pristine master here (missing @deepseek-ai/cordis in this install) and are not from this change. Full suite is the CI gate.


Additive review commits (ework-daemon, 2026-11-16)

Three commits added on top of the original fix (direction verified correct; all meter consumers consistent):

  1. refactor: re-arranged the [Bug][dsh] measured-body egress fuse meters a model-hidden DSH field: 88% of a refused body was dsh_session_log inside the other bucket #6844 comment that the first commit had split out of the side request(max_tokens<=200)绕过 preflight 且不检查载荷体积,携带全量历史直转上游导致 400 #554 comment paragraph (no behavior change).
  2. fix: the 413 message now NAMES the field dominating the other bucket (issue item 3, independent of any exemption mechanism): estimateRawBodyBreakdown tracks per-top-level-field token counts in the same single walk (additive RawBodyBreakdown.otherTop, descending), and measuredBreakdownClause appends (top field: <k> ~N) — all three 413 sites (egress fuse, preflight measured gate, fresh-session gate) inherit it.
  3. test: e2e both-ways fuse regression in tests/issue4079-egress-window-fuse.test.ts — a body whose mass is a model-hidden DSH field forwards (the incident shape no longer 413s; the fuse is a meter, not a filter, so the field still rides byte-exact), while an equally sized UNKNOWN top-level field still 413s and names itself.

Targeted suites green: issue6844 (8/8), issue4079 fuse (5/5), side-request-isolation + raw-body-walk-depth + fix-1075 (51/51), issue4185 (5/5), typecheck clean. Full-suite verdict = CI on the pushed head.

Behavior changes (old → new)

  • Meter input set: top-level dsh_session_log / dsh_plugin_packages bytes were previously priced into the measured body; now excluded from estimateRawBodyTokens / estimateRawBodyBreakdown (top level only; nested occurrences still count; unknown fields still count — fail-safe). Affects every meter consumer: egress fuse, preflight measured gate, side-request guard, session.metadata.rawInputTokens. Reason: the fields are documented model-hidden provider metadata; the meter prices model input.
  • 413 message format: other ~X → other ~X (top field: <k> ~N) (additive; only when other is non-empty). Tests match by regex prefix — no parse dependency.
  • RawBodyBreakdown gains an additive otherTop field (the total invariant == estimateRawBodyTokens is preserved and pinned).

Open decision (see thread comment 🔶 需要决策)

The exemption is a library-maintained hardcoded set. An operator-configurable allowlist (config-surface change, needs owner sign-off) or a host-declared header (DSH-side change) are follow-up options — not implemented here.

Fixes #6844

@ranxianglei ranxianglei changed the title fix: exclude model-hidden DSH fields from the measured body (#6844) [PR] fix: exclude model-hidden DSH fields from the measured body (#6844) Nov 15, 2026
ci-check added 3 commits November 16, 2026 00:55
…ianglei#554 paragraph

The first commit inserted the MODEL_HIDDEN_BODY_KEYS declaration in the
middle of the ranxianglei#554 comment paragraph, orphaning the 'fails closed.' line
below the new const. Move the declaration (with its own complete comment)
after NON_TEXT_BODY_KEYS; no behavior change.
…ranxianglei#6844)

Issue ranxianglei#6844 item 3: the measured-doom 413 lists four buckets, but 'other'
is a catch-all — attributing it took manual arithmetic (the incident's
88%-other body needed a body dump to identify dsh_session_log). Independent
of any exemption mechanism: the next unknown host-attached field will still
413, and the operator-facing message must say WHICH field is metered.

- estimateRawBodyBreakdown tracks per-top-level-field token counts for the
  other bucket in the same single walk (new RawBodyBreakdown.otherTop,
  descending; empty when other is empty; top-level arrays attribute to no
  field, as before).
- measuredBreakdownClause appends '(top field: <k> ~N)' when other is
  non-empty — all three 413 sites (egress fuse, preflight measured gate,
  fresh-session gate) inherit it.
- raw-body-walk-depth shape pin updated for the additive otherTop field.
- unit tests: per-field attribution + sum-to-bucket, hidden fields leave no
  attribution, clause naming + bare clause when other is empty.
- a body whose mass is a model-hidden DSH top-level field (5M-char
  dsh_session_log, ~1.25M measured tokens vs the 1M floored bar) forwards
  to the upstream — the incident shape no longer 413s; the fuse is a meter,
  not a filter, so the field still rides byte-exact;
- an equally sized UNKNOWN top-level field still stops at the fuse (the
  exemption is a documented-field allowlist, not a hole — ranxianglei#3617 gate
  discipline) and the 413 names the dominating field.
@ranxianglei ranxianglei added bug Something isn't working dsh p1 regression labels Nov 15, 2026
@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

Duplicate screening: Not a duplicate. The closest open PR is #6840 (fixes #6828) — different symptom (a body carrying the conversation under two protocol shapes at once double-counts the history) and different mechanism (a protocol-aware skip of the foreign conversation field), so it stays a separate issue. However #6840 and this PR both restructure the same top-level loop in estimateRawBodyTokens / estimateRawBodyBreakdown (src/preflight.ts) and both touch measuredBreakdownClause / measuredOutbound (src/server.ts) — they will conflict on merge and need coordinated ordering (see the coordination note at the end).

Verified live (targeted, on the pushed head 92994368):

  • New regression test tests/issue6844-measured-body-model-hidden-fields.test.ts: 8/8 pass; run against the pre-fix source it is 6/8 fail (the 2 that pass don't depend on the fix), so it genuinely reproduces the bug.
  • Neighbor suites raw-body-walk-depth + side-request-isolation: 45/45.
  • E2E issue4079-egress-window-fuse.test.ts (incl. the new both-ways [Bug][dsh] measured-body egress fuse meters a model-hidden DSH field: 88% of a refused body was dsh_session_log inside the other bucket #6844 test): 5/5.
  • Typecheck: clean in every changed file. The only 5 errors are in tests/dsh-tool-registration-lifetime.test.ts, all from a missing @deepseek-ai/cordis module in this install — pre-existing, not from this change (confirmed: the module is declared in package.json but absent from node_modules).
  • Full-suite verdict is the CI gate (still running on the pushed head when I checked).

Review (八问):

  1. 是什么问题: 0.1.214 的 measured-body 出口熔断把 body 里每个顶层字段都算进 other 桶,包括 DSH 挂的 model-hidden 诊断字段 dsh_session_log(默认 8 MiB)和 dsh_plugin_packages。结果普通会话请求在模型可见内容只占窗口 28% 时,other 占 88% 就把请求 413 拒了。
  2. 什么原因导致的: 根因是 other 是个 catch-all 桶,没有"这个 payload 不会到模型"的概念——任何宿主合法挂的诊断字段都会落进 other 参与计量。表面现象是 413,病因是计量器把 model-hidden 字段当成 model-visible 内容。
  3. 影响范围与严重程度: 只影响 DSH native-plugin 会话且带了 dsh_session_log(默认开)的普通请求——这类请求被本地 413 拒、用户无法继续对话(需手动禁用 session-log 插件绕行)。属"特定条件下不能用"(p1 可用性缺陷)。非 DSH 客户端、不带该字段的请求不受影响。
  4. 怎么修复的: 在 estimateRawBodyTokens / estimateRawBodyBreakdown 的顶层遍历里跳过这两个 DSH model-hidden 字段(新增 MODEL_HIDDEN_BODY_KEYS 常量)。只在顶层跳——同名 key 嵌在 messages[] 里是 model-visible 内容照常计量;真正超大的未知 other 字段也照常计量(熔断是计量器不是过滤器,字段仍原样转发上游)。同时按 issue 第 3 条把 other 桶里占主导的顶层字段名写进 413 消息(otherTop + measuredBreakdownClause 后缀),下次未知字段触发时不用手工算。
  5. 是否有回归问题: 审了改动路径的既有行为——(a) estimateRawBodyTokens 其余调用点:threads-lane 用合成对象(不含这两个字段,不受影响);side-request 守卫是模型 token 预算,排除 model-hidden 字段是正确的(它们不进模型窗口);handle 的 rawInputTokens 是元数据,同理正确。(b) estimateRawBodyBreakdown 新增 otherTop 必填字段:全仓只有 estimateRawBodyBreakdown 一个构造点(已更新),唯一的 deepEqual 断言(raw-body-walk-depth)已同步,无其他消费方破坏。(c) 高危模式逐项过:无语义漂移(只跳两个具名字段)、无 fail-fast 变吞错(未知超大字段仍触发,负向测试钉住)、无阈值/时序变化、输出格式变化已披露(见下)。无回归。
  6. 是否破坏缓存: 不涉及。改动只在出口计量与 413 文案,不碰缓存/重试/增量游标/去重键。
  7. 是否改了设计方向: 不涉及架构转向。这是 issue 建议方向里的"具名顶层豁免",比 issue 列的第 4 条"扩 NON_TEXT_BODY_KEYS"更精确(后者是 value 级、会误伤任何叫 data/url 的字段;本 PR 是顶层级、只豁免两个 DSH 具名字段)。issue 首选的 config 白名单(egressFuse.exemptTopLevelKeys[])是 owner 门禁的配置面变更,本 PR 正确地没有擅自发明新配置。
  8. 是否引入了新的配置: 无新 env/config/CLI。MODEL_HIDDEN_BODY_KEYS 是模块内常量,不是配置面。

行为变更披露(旧值→新值+原因):

  • other 桶计量:旧=含 dsh_session_log/dsh_plugin_packages → 新=不含(仅顶层)。原因:这两个字段文档明确"不进模型输入",不该按模型窗口计量——本次修复的本意。
  • 413 measured-doom 文案:旧=other ~W → 新=other ~W (top field: F ~T)(仅当 other > 0 且有主导字段时)。四桶形状不变,纯增量。原因:issue 第 3 条要求报出占主导的字段。

协调注意(重要,需 owner 定合入顺序): 本 PR 与 #6840 都改了 src/preflight.ts 的 estimateRawBodyTokens/estimateRawBodyBreakdown——两者都把 pending 初始化改成"顶层遍历+跳过"的同一段代码,只是跳的 key 不同(本 PR 跳 MODEL_HIDDEN_BODY_KEYS,#6840 跳 FOREIGN_CONVERSATION_FIELDS[protocol]),也都碰了 src/server.ts 的 measuredBreakdownClause/measuredOutbound。两者合并必然冲突:需先合一个,另一个 rebase 时把两个跳过条件合并进同一个顶层循环(既跳 foreign 会话字段又跳 model-hidden DSH 字段),并把 otherTop 与 protocol 参数两个改动叠加。

判定: 代码干净、测试有牙(正/负双向 + e2e both-ways)、无回归、行为变更已披露。本 PR 自身可合;与 #6840 的合入顺序需 owner 协调。

ranxianglei pushed a commit that referenced this pull request Nov 18, 2026
… dominating other (#6844)

#6844's 88%-other body took a manual body dump to attribute: which
host-attached field carries the non-prompt mass. estimateRawBodyBreakdown
now tracks per-top-level-field token counts inside the other bucket
(otherTop, descending) and measuredBreakdownClause names the dominating
field in the 413 reason. No meter semantics change: total, the four
buckets and prompt are byte-identical to before; otherTop is pure
diagnostics.

Ported from sx4im/billion-context PR #6860's otherTop diagnostic, which
is independent of that PR's hardcoded-field exemption (superseded by the
prompt-mass metering in bccd147).
ranxianglei pushed a commit that referenced this pull request Nov 18, 2026
… dominating other (#6844)

#6844's 88%-other body took a manual body dump to attribute: which
host-attached field carries the non-prompt mass. estimateRawBodyBreakdown
now tracks per-top-level-field token counts inside the other bucket
(otherTop, descending) and measuredBreakdownClause names the dominating
field in the 413 reason. No meter semantics change: total, the four
buckets and prompt are byte-identical to before; otherTop is pure
diagnostics.

Ported from sx4im/billion-context PR #6860's otherTop diagnostic, which
is independent of that PR's hardcoded-field exemption (superseded by the
prompt-mass metering in bccd147).
Resolve conflicts in src/preflight.ts: keep the PR's ranxianglei#6844 MODEL_HIDDEN_BODY_KEYS skip and otherTop per-field breakdown alongside main's isNonModelTopLevelKey skip in estimateRawBodyTokens and estimateRawBodyBreakdown; the 4-tuple pending walk keeps the PR's field tracking, main's Array.isArray guard included. src/server.ts merged clean.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug][dsh] measured-body egress fuse meters a model-hidden DSH field: 88% of a refused body was dsh_session_log inside the other bucket

2 participants