Skip to content

[PR] fix: session-level compress.modelContextLimit now sizes every window consumer (#5937) - #5944

Open
ranxianglei wants to merge 3 commits into
masterfrom
2026-11-08_honour-compress-modelcontext-limit
Open

ranxianglei wants to merge 3 commits into
masterfrom
2026-11-08_honour-compress-modelcontext-limit

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Closes #5937

A session-level compress.modelContextLimit was honoured by the judging chain (preflight limit / headroom) but ignored by three window consumers, so a 450K cap could never stop an over-cap body (#5885 defect-4: 2M-token outbound, 31 preflight rounds chasing a deficit measured against the wrong wall):

  1. preflight post-fold fit proof (nativeFitAfterClamp) judged against the RAW resolved native window.
  2. relay egress fuse sized every declared window through its 1M anti-guess floor, so an explicit sub-1M cap could never doom a body.
  3. DSH native plugin lane: effectiveContextWindow read only the cached model info — both the host-side oversize gate and the x-bili-plugin-context-window stamp ([dsh-native] takeover-gate refusal (threw state) fails open with the FULL uncompressed transcript; a >window local history dies upstream with 400 (maximum context length) #2924) ignored the key entirely.

Changes

config surface

No new keys. Semantics extension of existing compress.modelContextLimit: it now governs the token-measured egress guards in addition to the preflight fold/headroom. Window priority, highest first: upstream 400 evidence (overflow arm) > this key > registry/native resolution > top-level modelContextLimit fallback. Route/model-level compress overrides unchanged (proxy-side only).

Behavior changes (old → new)

  1. Egress fuse bar for sessions carrying an explicit cap: max(cap, 1M) × 7.5 B/tok → cap × 7.5 B/tok (min 2 MB). Bodies between the two bars that used to forward now get a local 413 naming compress.modelContextLimit. Rationale: an explicit declaration is a deliberate operator wall, not the wrong-small guess the floor exists for (Context Window Overflow Issue Summary: Raw Requests Sent Directly to Upstream Rejected + Local Intercept Cooldown Lockin | 上下文超窗问题汇总:请求裸冲上游被拒 + 本地拦截冷却锁死会话(统一分析入口) #4079).
  2. Preflight forward-once fit proof: fires on the operator cap even when the native window is unknown, and drops its bar to the cap when cap < native. Rationale: a body over the operator's declared wall can never fit even if it fits the raw native window.
  3. DSH plugin lane: oversize gate + x-bili-plugin-context-window reflect the global cap instead of only the cached model info. Rationale: the key is documented as the effective window; the incident showed the plugin lane ignoring it end-to-end.
  4. Follow-up needed once this lands: PR [PR] Over-window requests no longer sent raw upstream: three forwarding gates metered by actual outbound body size + 32 MiB c | 超窗请求不再裸发上游:三道转发闸按真实出站体计量 + 大请求 32MiB 成本带准入 (#5000 #4543 #5885 #5794) #5932's armMeasuredBar/gateMeasuredBar (+ its relay fuse token tier) still use raw resolvedNativeWindow; they need a 2-line rewire to meteringWindow ?? limit. Not done here to avoid landing dependent code on another thread's open branch.

Tests

New tests/issue5937-operator-window.test.ts: main-path pin ([window] base=450000, [headroom] 450000→337500, stamps), fuse both-ways (over-cap 413 / under-cap forwards — verified to FAIL against master relay.ts), dsh-native resolution rows (abs/pct/no-info/garbage). Targeted suites green locally: issue4079-egress-window-fuse, preflight-client-abort, compress-settings, dsh-native, issue5191-arm-judge-line, headroom-behavior files, context-overflow, side-request-isolation, issue2317-route-miss-diagnostics. Full suite = CI.

No kernel/ changes; metadata key is additive (no persistence format change).

@github-actions

github-actions Bot commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-11-08_honour-compress-modelcontext-limit (74a0b93)

Latest build — install this exact version

npm install -g billion-context@0.0.9719-pr.5944.76f56c0

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Recent builds (commit → version)

commit version build
74a0b93 billion-context@0.0.9719-pr.5944.76f56c0 build
fc6494b billion-context@0.0.8626-pr.5944.a09121c build
b6a90ab billion-context@0.0.8624-pr.5944.833659e build

Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr5944.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei ranxianglei changed the title fix: session-level compress.modelContextLimit now sizes every window consumer (#5937) [PR] fix: session-level compress.modelContextLimit now sizes every window consumer (#5937) Nov 8, 2026
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

Duplicate screening: not a duplicate. No other open PR targets #5937 — checked all open [PR] rows; the closest relative is #5934 (three forwarding gates metered by outbound body size, same #5885 incident family but a different mechanism: body-size gates vs. window-consumer sizing) and #5932 (the measured-bar arms this PR explicitly defers rewiring to a follow-up). No parallel second fix for this thread.

Labels: PR #5944 already carries bug + p2 + dsh + scope/kernel, all judged against its own diff and all holding up: bug (defect fix), p2 (cap silently ignored — usable but the declared wall never bites; workaround exists via top-level fallback), dsh (one of the three fixed consumers is the DSH native plugin lane), scope/kernel = compression/context-management domain (the diff touches zero files under kernel/). No relabel needed.

Review in progress: branch b6a90ab fetched locally; reading the full diff file-by-file against current master, verifying the three claimed consumer wirings plus regression surface (non-operator sessions must stay byte-for-byte), then CI status via the mirror cache. Structured eight-question conclusion (八问) follows as a separate comment once verification is done.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

🔶 需要决策
(ai: vllm-qwen/qwen3.8-27b)

背景一句话:同一 issue #5943 有两个在途 PR(#5945 先到、#5946 后到),核心语义互斥——「成功折叠是否消耗 nudge 信用」;我用 issue 正文的事故时间线对两个方案各做了一次代码级回放,#5946 的「折叠即消耗」语义修不掉标题现象,需要你拍板以哪个为准。

查重结论:重复。 #5946 与 #5945 同症状(requireNudge 下多轮清理压缩被误拒)+ 同根因(prepare 每轮重盖 armed,资格是单轮布尔)+ 同机制族(单轮武装改 nudge 信用窗口),差异只在两处设计取舍:①成功折叠是否消耗信用;②窗口计数单位(#5945 按模型轮、非模型形态不衰减;#5946 按每次 prepare)。按先来后到,保留 #5945(11:21:30Z 开)、#5946(11:23:17Z 开)标 duplicate——agent 无关闭权,关不关由你定。

待决问题:信用语义取哪种?合哪个 PR?

我的推荐:C(实质 A 先行)。理由:#5946 经回放证明修不掉标题现象,合入即「修了个没修的」;#5945 修掉事故、测试含 K±1 边界、偏离已披露且正是在等这个裁决。不决定的默认行为:两个 PR 我都不能合也不能关(owner-only);方向性缺陷使 #5946 不宜合入,默认轨迹 = #5945 待 CI 转绿后等你确认合入,#5946 等你关闭。


为什么判断 #5946 修不掉事故(独立回放,非引用 #5945 结论)。issue 证据时间线(session 01a1b6a5,请求级 host,每模型轮 = 新 HTTP 请求 = 新 prepare):18:40 T1 nudge 注入(usage 72%)→ 18:41:11 模型当轮成功压 5 块(72%→38%)→ 18:42:27/18:43:29/18:44:15/18:46:21 连续 4 轮压剩余范围全部 REFUSED requireNudge。按 #5946 代码走:18:40 轮 stampUnpromptedGate(nudgeInjected=true) → armed、until=R+19;18:41 折叠提交走到 src/stream.ts 提交点 consumeNudgeCredit → consumed=true(同轮余量仍放行);18:42 起每轮 prepare 重 stamp,creditLive = prev.armed && prev.consumed !== true → false → armed=false → 4 连拒逐分复现。即 #5946 只救场景 1(延迟首次履约:nudge 轮没压、之后几轮回来压),救不了 issue 自认「更常见的现场」场景 3(主范围当轮压完、跨轮清理剩余范围)——而事故现场恰恰是场景 3。#5945 的不消耗语义下同时间线:18:42–18:46 信用仍在窗口内(20→16),4 次清理放行,事故修复。

结构化八问(审的是 #5946):

  1. 是什么问题:开 compress.requireNudge 后压缩资格只在 nudge 注入那一轮有效,模型隔轮回来把推荐范围的剩余部分压完 → 每轮被拒、白烧 tool call + 思考,回执还把「下轮再压」引导成「别压了」。
  2. 什么原因导致的:src/server/handle.ts prepare 每轮重盖 armed: nudgeOutcome === "injected"——单轮布尔,无跨轮延续、无履约概念,nudge 轮一结束资格蒸发。
  3. 影响范围与严重程度(按选项):仅显式开启该 opt-in 开关的会话(默认关 = 零变化);受影响会话多轮清理全拒、opt-in 功能实际不可用,p2(无数据丢失/缓存泄露,preflight 兜底不受此门影响)。若走 B 合入:事故现象保留,对 compress.requireNudge is only valid in the round where the nudge is injected: if the model comes back in a later round t | compress.requireNudge 只在 nudge 注入的那一轮有效:模型隔轮回来压缩就被拒,多轮清理式压缩全部白拒 #5943 验收标准不达成;走 A/C:零额外影响,由 [PR] fix(#5943): requireNudge judges a nudge credit window, not this-round arm #5945 承接。
  4. 怎么修复的([PR] compress.requireNudge single-round arming changed to 20-round nudge credit: the model returning on alternating rounds to | compress.requireNudge 的单轮武装改为 20 轮 nudge 信用:模型隔轮回来压缩不再被误拒(#5943) #5946 方案):armed 改 20 轮 nudge 信用(NUDGE_CREDIT_ROUNDS=20 常量)——真实注入开窗、新注入刷新、committed 折叠消耗(同轮放行、下一轮边界失效)、窗口尽过期;纯内存重启清零。我对实现的独立核验(状态机、preflight 经 loopTracking:false 豁免、所有模型车道汇入 applyRanges 单执行点、fork 子车道 gate-OFF 不变、默认关字节级不变)均成立;本地定向测试 21/21 + 相邻面 59/59 绿、typecheck 干净。缺陷在语义选择层,不在实现层。
  5. 是否有回归风险(按选项):[PR] compress.requireNudge single-round arming changed to 20-round nudge credit: the model returning on alternating rounds to | compress.requireNudge 的单轮武装改为 20 轮 nudge 信用:模型隔轮回来压缩不再被误拒(#5943) #5946 代码本身逐项审过既有行为——默认关路径字节不变、preflight 豁免不变、时序门先于范围门顺序不变、拒收回执文案变更已进 docs 四件套披露,未见无意回归。B 的风险是方向性的:合入 = 带着未修复的标题现象关 issue。A/C 无回归面。
  6. 是否破坏缓存:不涉及缓存/重试/增量游标/去重键。行为变更披露(若 B 合入):旧 = 资格仅 nudge 注入当轮 → 新 = 最近一次真实注入起 ≤20 轮内有效且首次成功折叠即消耗(仅开关开启时生效)。
  7. 是否改了设计方向:架构无转向(仍是 prepare 盖章 + applyRanges 单执行点),只是盖章事实从「单轮布尔」换成「信用状态机」。
  8. 是否引入新配置:无(K=20 为代码常量;i18n/yaml/CHANGELOG/docs 文案四件套已同步、生成器已跑)。

标签:GitHub PR #5946 = bug / p2 / scope/kernel / duplicate(按本 PR diff 独立判定:opt-in 功能缺陷、非 p1 类);镜像 5948 = pr + bug / scope/kernel / p2 / duplicate / needs-decision(同判定 + 待你裁决)。不打客户端标签——证据宿主为 qwen3.8-27b 请求级 host,无具体客户端指纹可判。

CI 状态(刚读的实时值,head 6ca0e91):0 failed,13 项运行中/排队(test shards + static-gates),已完成项全部 success/skipped。

我的动作披露:review 中向 #5946 分支补了 2 个标注 commit——c3b7381e1b(test:钉死「被拒的折叠不消耗信用」,该分支本地 22/22)与 6ca0e91(docs:澄清 armedUntilRound 缺省字段语义)。若走 A/C 两者随分支作废;若走 B 则保留有效。

收工判定:#5946 实现质量过关但方向有缺陷(修不掉标题现象),不建议合入;推荐 C,等你拍板。

ci-check added 3 commits November 14, 2026 08:34
…consumer (#5937)

judgments — they all ran off the raw model window while the judging chain
(preflight limit / headroom) already consumed the key:

- preflightCompressIfNeeded gains operatorWindow (post-self-heal tuned
  value); the post-fold measured-token fit proof (nativeFitAfterClamp) now
  judges a meteringWindow = operator cap > raw native > undefined instead
  of always the raw native window. No explicit setting => identical
  behavior. Proven upstream evidence still clamps lower (#5191 ruling).
- relay egress fuse: sessions stamped with an explicit cap (new
  session.metadata.operatorWindowTuned provenance twin, written beside
  effectiveContextLimit on every full-pipeline request) size the bar at
  cap x 7.5 B/tok (min 2MB) WITHOUT the 1M anti-guess floor; non-operator
  sessions keep the floor byte-for-byte. The 413 names the setting and how
  to release it. A no-model request no longer erases a prior modeled turn's
  operator stamp (tri-state operatorWindowTuned).
- dsh-native effectiveContextWindow honours the GLOBAL-level
  compress.modelContextLimit (resolveContextLimitValue moved to util.ts so
  the host bundle shares the exact resolution without pulling kernel pack
  machinery; compress-settings re-exports it). Fixes both the oversize gate
  doom math and the x-bili-plugin-context-window stamp at once: the proxy
  then sees the cap as its pluginWindow source and the whole existing chain
  sizes off it. Route/model-level overrides stay proxy-side only.

No new config keys — semantics extension of an existing one. Window
priority, highest first: upstream 400 evidence > this key >
registry/native resolution > top-level modelContextLimit fallback.

Tests: tests/issue5937-operator-window.test.ts (main-path pin [window]
base=450000 / [headroom] 450000->337500; fuse both-ways, verified to fail
against master relay.ts; dsh-native resolution rows). Docs:
configuration.md en+zh (guard wall + priority line).
…n-config-docs (#5937)

The summary tables in docs/configuration.md are generated from
website/config-reference/*.yaml (docs-sync lane: generate + drift check).
Edit the seed instead of the generated rows; site_en/site_zh carry the
same guard-wall semantics to website/main.js and website/docs/index.html.
… new operatorWindow param (rebase #5944)

Rebasing #5944 onto master surfaced two rebase-induced breaks:

1. master added a measured-body egress fuse (relay.ts) that fires BEFORE
   the PR's core-text fuse. Its 413 message hardcoded the non-operator
   wording, so an operator-declared session's over-cap body was refused
   without naming compress.modelContextLimit (#5937). Added the operator
   wall clause + remedy to the measured-body fuse message, mirroring the
   core-text fuse.

2. the PR added a required operatorWindow param to
   preflightCompressIfNeeded (pos 13). Two master-added test files still
   called it with 17 args → TS2554/TS7006. Inserted the operatorWindow
   undefined at pos 13 in both.

3. the PR's own egress-fuse test (issue5937-operator-window) was written
   against the core-text byte bar; with the measured-body fuse the bar is
   the token cap, so the over-cap direction now matches the measured-body
   message format and the under-bar body is sized below the token cap.
@ranxianglei
ranxianglei force-pushed the 2026-11-08_honour-compress-modelcontext-limit branch from fc6494b to 74a0b93 Compare November 14, 2026 00:49
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷

Rebase + red-CI fix for PR #5944 (mirror #5946) — done, force-pushed.

The PR was 1489 commits behind master at merge-base. I rebased onto current master, resolved 2 conflicts, and fixed 3 rebase-induced breaks the rebase itself didn't catch. All verified green locally; force-pushed fc6494ba6a…74a0b937f7.

What I did:

  1. Rebase onto master (merge-base 3ac3e95c0e → master; re-rebased once master moved 23 commits). Two conflicts, both resolved:

  2. Fixed the typecheck break the rebase exposed: the PR added a required operatorWindow param (pos 13) to preflightCompressIfNeeded. Two master-added test files still called it with 17 args → TS2554/TS7006. Inserted the operatorWindow undefined at pos 13 in both (tests/issue5907-anon-preflight-warn.test.ts:121, tests/preflight-zero-fold-calibration.test.ts:113).

  3. Fixed the test failure the rebase exposed: master added a measured-body egress fuse (src/server/relay.ts:1013-1031) that fires BEFORE the PR's core-text fuse. Its 413 message hardcoded the non-operator wording, so an operator-declared session's over-cap body was refused without naming compress.modelContextLimit ([Bug][config] Session-level compress.modelContextLimit is ignored by the window resolution chain: the compression condit | [Bug][config] 会话级 compress.modelContextLimit 被窗口解析链忽略:压缩判式按默认 1M 窗口算,450K 限制永不生效 (#5885-defect-4) #5937). Added the operator wall clause + remedy to the measured-body fuse message (mirroring the core-text fuse at relay.ts:1036-1041). The PR's own egress-fuse test was written against the core-text byte bar; with the measured-body fuse the bar is the token cap, so I updated the over-cap direction to match the measured-body message format and sized the under-bar body below the token cap (bigMessages(670→300, 4_450) — 760K→341K measured tokens, under the 450K cap).

Verification (all green on rebased head 74a0b937f7):

  • npm run typecheck → exit 0
  • node --import tsx --test tests/issue5937-operator-window.test.ts tests/issue5907-anon-preflight-warn.test.ts tests/preflight-zero-fold-calibration.test.ts → 18 pass, 0 fail
  • node tools/gen-config-docs.mjs generate → NO drift (docs-sync CI job)
  • Force-pushed: fc6494ba6a…74a0b937f7 (forced update)

The mirror cache still shows the old SHA (fc6494ba6a, fetched 2026-11-08) — the new CI run for 74a0b937f7 is in-flight and will appear once the mirror poller picks it up (~60s).


七问 (CI engineer summary):

  1. 是什么问题: PR [PR] fix: session-level compress.modelContextLimit now sizes every window consumer (#5937) #5944 与 master 冲突(merge-base 落后 1489 个 commit);rebase 后又暴露出 3 处 rebase 引入的破坏(typecheck 参数不匹配 + 测试失败)。
  2. 什么原因导致的: master 在 merge-base 之后新增了一个 measured-body egress fuse(relay.ts),它在 PR 的 core-text fuse 之前触发,且 413 消息硬编码了非 operator 措辞;PR 又给 preflightCompressIfNeeded 加了必填的 operatorWindow 参数,导致 master 新增的两个测试文件调用时参数数量不匹配(TS2554/TS7006)。
  3. 影响范围与严重程度: 仅影响 operator-declared(显式 compress.modelContextLimit)会话的 egress fuse 413 消息措辞——bar 本身(= cap,无 1M floor)是对的,只是消息没点名 compress.modelContextLimit。属于可诊断性/体验问题,不是行为回归。
  4. 怎么修复的: 给 measured-body fuse 的 413 消息补上 operator wall clause + remedy(镜像 core-text fuse 的写法);给两个 master 测试文件补上 operatorWindow 的 undefined 参数;把 PR 自己的 egress-fuse 测试的 over-cap 方向改成匹配 measured-body 消息格式,under-bar body 缩小到 token cap 以下。
  5. 是否有回归问题: 无。bar 的计算(fuseWindow = operatorDeclared ? declaredWindow : max(declaredWindow, 1M))没动,只改了消息措辞;非 operator 会话走原分支(措辞逐字不变)。typecheck + 3 个测试文件 + docs-sync 全绿。
  6. 是否破坏缓存: 不涉及。改动只触及 egress fuse 的 413 消息字符串和测试,不触及缓存/重试/游标/去重键。
  7. 是否改了设计方向: 不涉及。这是 rebase 适配,不是设计变更。PR 的意图(operator-declared 会话的 egress fuse 点名 compress.modelContextLimit,bar = cap 无 1M floor)完整保留。

收工判定: rebase 完成,2 处冲突已解,3 处 rebase 引入的破坏已修,本地 typecheck + 测试 + docs-sync 全绿,已 force-push。新 CI run 在途,等镜像 poller 拾取后即可看绿。

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working dsh p2 scope/kernel

Projects

None yet

1 participant