Skip to content

[PR] fix(preflight): fold probes blocked by the local egress fuse - #7009

Closed
WHUrubbish wants to merge 7 commits into
ranxianglei:masterfrom
WHUrubbish:2026-11-17_preflight-probe-egress
Closed

WHUrubbish wants to merge 7 commits into
ranxianglei:masterfrom
WHUrubbish:2026-11-17_preflight-probe-egress

Conversation

@WHUrubbish

Copy link
Copy Markdown
Contributor

Problem

中文摘要:修复“预压缩决定先发探针获取证据,但最终出站保护又在本地返回 413”的恢复死结;复用已有计量与阈值,让本地已无法发出的探针进入正常压缩。涉及取证策略衔接,请维护者人工审阅。

An estimate-only session can get stuck between two existing decisions:

  1. preflightCompressIfNeeded() chooses the billion-context 卡死问题 —— 上报材料 usage获取不到导致上下文估计失误 #2313 evidence probe and returns the prepared request without folding.
  2. The final egress window fuse refuses the same body locally with 413 egress_window_fuse.

The probe never reaches the upstream, so it cannot obtain the usage/overflow evidence that was supposed to restore the recovery path. A long resumed conversation reproduced this with approximately 1.30M locally measured tokens against a 1M window, below the existing core-byte doom bar. These are local meter readings, not upstream billed token counts.

Change

Before returning an estimate-only probe, reuse measuredOutbound() and the final egress fuse's existing declared-window validation and MIN_ASSUMED_WINDOW_TOKENS floor. If that existing fuse would reject the probe, fall through to the normal preflight compression path instead.

  • Keep the final egress fuse and the earlier byte-doom guard intact.
  • Preserve evidence probes below the final fuse's bar, including the existing dense-payload controls.
  • Add no persistent refusal record and do not write estimated values into usage evidence.
  • No config, version, lockfile, persistence format, kernel, identity, or message-shape changes.
  • The added progress line is server-side diagnostics; no model-facing prompt/receipt wording changes.

This runs through the common preflight path for proxy and plugin modes.

Regression and validation

Two hermetic fake-upstream cases cover proxy and plugin modes. Each seeds a non-current-route estimate baseline, then submits a CJK body below the byte-doom bar but above the final measured-body fuse.

Field validation on the original released 0.1.215 deployment with DSH 0.2.2-rc.3 restored the affected conversation without changing its Session ID. Long preflight still triggered two 600s DSH idle retries before eventual completion; this PR does not claim to fix stream-watchdog activity accounting or every upstream reset.

Related work and review boundary

AI-assisted contribution prepared with Codex; public changes are limited to the source fix and synthetic tests.

ci-check and others added 5 commits November 16, 2026 20:42
…ns {3,5,11,14}

The probe anchor PROXY_TOOL_AT=6 was calibrated on 2026-10-31 for ranxianglei#4521
(schedule {4,6,8,9}). The nudge scheduling has shifted again — nudge now
lands at {3,5,11,14}. Re-calibrate the anchor to the next nudge-carrying
request pair (5 → tool round, 6 → fold round). The fold/proxy-tool-round
mechanics under test are unchanged.

Closes: master-red on test (windows-latest, 22, 3) and (ubuntu-latest, 22, 1).
…ic wire shape

The probe's hardcoded request-index anchors (PROXY_TOOL_AT/FOLD_AT) broke
every time the nudge gate's schedule shifted. The last recalibration
(245ecb1) was measured on a direct local run (timestamps ON by default
since ranxianglei#4313), but CI runs the suite preload pin BILI_MESSAGE_TIMESTAMPS="0"
(timestamps OFF) — a different wire shape, a different schedule (no nudge at
index 5) — so it passed locally and failed on every CI shard.

Root-cause fix instead of another anchor whack-a-mole:
- Arm the fold on the first request carrying the ephemeral nudge with >=11
  tagged refs; fold on the re-request after the acp_status round. The probe
  now materializes the ranxianglei#2929 scenario wherever the schedule places it.
- Pin compress.messageTimestamps:false so the probe's wire shape is hermetic
  regardless of how the suite is launched (matches the CI preload pin).

Supersedes the 245ecb1 recalibration. Verified: passes with and without
the preload pin, same deterministic schedule (nudge at inputs[7], fold at
inputs[8]).
@ranxianglei ranxianglei changed the title fix(preflight): fold probes blocked by the local egress fuse [PR] fix(preflight): fold probes blocked by the local egress fuse Nov 17, 2026
@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

查重: 同家族但机制不同 — 与 #6844/#6845(measured-body fuse 误计 model-hidden 的 dsh_session_log)、#6877(fuse 把 Responses input 里未计费的 encrypted_content 当 token,导致 nudge/fuse 口径分歧)同属「egress fuse 413 卡死长会话」家族,但机制不同:

无并行 PR 针对同一线程(已查 open [PR] 列表)。开始正式 review:拉取分支、通读 diff、定向验证。

ci-check added 2 commits November 17, 2026 13:31
… unreachable from the side lane

The preflight egress-fuse probe block keys on !prepared.sidePassthrough, but
side requests never reach preflight — handle.ts forwards them with a minimal
prepared before the preflight call, and bodies over the effective window are
already rejected locally by the side payload guard (ranxianglei#554, 413
side_request_payload_too_large). Pin that exact shape (stuck estimate meter +
body above the fuse's 1M floor) so a future change that routes side traffic
through preflight, or makes the new block apply to side, fails here.

Lane-suite discipline per ranxianglei#2170: the lane-symbol line in src/server.ts
(ranxianglei#7009) lands with the dedicated side-lane suite.
@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

Review verdict: technically sound and well-tested. Two CI reds fixed on the spot as additive commits to the PR branch. Remaining merge gates: (a) CI green on the new head, (b) owner confirmation of the #5976 probe-policy direction.

查重(首条回复已给): 与 #6844/#6845/#6877 同属 egress-fuse-413 家族,但机制不同——本 PR 修的是预压缩取证探针与本地熔断的衔接,不是熔断计量口径。

What I verified

  1. The new check is an exact mirror of the final egress fuse's measured-body gate. Same applicability guards (!sidePassthrough, non-GET/HEAD, declared window > 0), same meter (measuredOutbound, CJK-aware), same bar max(declaredWindow, MIN_ASSUMED_WINDOW_TOKENS). The fuse itself is untouched and still writes no persistent refusal record (The context window suddenly hit its limit and threw an error. Starting a new session didn't help either. dsh | 上下文突然满了报错 新会话也没用 dsh #3974 contract).
  2. Side lane is structurally unaffected — and doubly so. Side requests never reach preflightCompressIfNeeded: handle.ts builds the minimal sidePassthrough prepared and forwards before the preflight call (src/server/handle.ts:2699-2764 vs the preflight call at :3263). Additionally, bodies over the effective window are already rejected locally by the pre-existing side payload guard (side request(max_tokens<=200)绕过 preflight 且不检查载荷体积,携带全量历史直转上游导致 400 #554, 413 side_request_payload_too_large), whose bar (effective window) is at least as strict as the fuse's measured bar (max(window, 1M)). The PR's !prepared.sidePassthrough conjunct is a defensive mirror of the fuse guard; I pinned the whole interaction in the side-lane suite (below).
  3. Both compression modes. The preflight call site has no plugin-mode exclusion, so the fix applies to proxy and plugin main lanes alike; the PR's regression tests loop both.
  4. Image path unaffected. Image mass is excluded from the meter (NON_TEXT_BODY_KEYS), so the PR #489 后续: 默认图片计费 b64/4 在像素 tile 计费上游(官方 Anthropic/OpenAI)会硬 502 误杀多模态会话 #496 image-arbitration path is untouched by the fuse's measured gate; the new check targets only the estimate-only probe arm.
  5. Regression tests have teeth. On clean master 301e1a54, both new cases fail with the original local 413 egress_window_fuse (~1.45M measured tokens vs 1M window); on the PR head they fold first and forward seed + one compressed request. 37 targeted tests pass (7 probe-forward + 30 related: billion-context 卡死问题 —— 上报材料 usage获取不到导致上下文估计失误 #2313, [Bug][preflight] Local 413 is forgotten when the baseline becomes an estimate, reopening oversized probes #3846, Context Window Overflow Issue Summary: Raw Requests Sent Directly to Upstream Rejected + Local Intercept Cooldown Lockin | 上下文超窗问题汇总:请求裸冲上游被拒 + 本地拦截冷却锁死会话(统一分析入口) #4079, PR #489 后续: 默认图片计费 b64/4 在像素 tile 计费上游(官方 Anthropic/OpenAI)会硬 502 误杀多模态会话 #496, [Bug][dsh][0.1.208] Kill switch / egress fuse only measures message.text: a session with 26,903 messages and only 1.44 M | [Bug][dsh][0.1.208] 判死/egress fuse 只量 message.text:26,903 条消息、core text 仅 1.44 MiB 的会话绕过 fuse 发出 55.2 MiB 请求体,被上游 413 拒收(ModelArts.… #5000, dsh压缩与bili压缩冲突 #2490, 模型切换后压缩块跨模型失配,新模型收到 61.3 万 token 全量上下文 #1001), tsc --noEmit clean.
  6. Residual pre-existing boundary (not introduced, not blocking): with output-headroom reservation, the preflight byte-doom guard uses the reserved limit while the fuse's byte floor uses the unreserved window (1M floor) — a body between limit×7.5 and declaredWindow×7.5 can pass the preflight byte-doom and still trip the fuse byte floor. The PR's meter-gated probe check does not change this; recorded as a boundary note.

Policy context (the human gate). This PR lands one direction of the open design issue #5976 (forensic probe that will inevitably exceed the window: 413-for-evidence contract vs metering gate): the probe now respects the already-enforced local egress decision and falls through to normal folding instead of dying in a local 413. That is the availability-first resolution (the turn is recovered, not killed), and it preserves every below-bar probe including the dense-payload controls. The PR explicitly asks for human review of the evidence-probe policy and requests no auto-merge — the direction call belongs to the owner.

CI fixes added on the spot (additive commits to the PR branch, per the external-PR fix rule):

CI state (verified live): both prior runs on 85fd89081b failed only on lane-guard + architecture-docs; the entire test(...) matrix was skipped by the fail-fast tiering (guard red → test legs skip), so the full-suite verdict was missing. A new run on 27d0349efe (synchronize) is in progress — with both guards fixed, the CORE matrix will run; the full-suite verdict is pending on that run.

8问

  1. 是什么问题:预压缩取证探针(billion-context 卡死问题 —— 上报材料 usage获取不到导致上下文估计失误 #2313)在本地被出站熔断拦截时,探针到不了上游、拿不到证据,会话卡在"要探针→本地 413"的死结,长会话完全无法继续。
  2. 什么原因导致的:预压缩"先发探针取证"的资格判定与最终出站熔断的计量口径之间没有衔接——探针被放行,但同一个 body 在最终出站时被本地熔断拒发。
  3. 影响范围与严重程度:仅 estimate-only(当前路由无 usage 证据)且本地计量超过 max(声明窗口, 1M) 的会话;触发时该会话完全不可用(每轮本地 413,无恢复路径)。触发条件窄,但一旦触发即毁灭性。
  4. 怎么修复的:返回 estimate-only 探针前,复用熔断既有的计量与阈值做同一判断;若熔断会拒,落回正常预压缩折叠路径。不改熔断、不加新阈值、不写持久拒绝记录、不写估算值进 usage 证据。
  5. 是否有回归问题:熔断本体、byte-doom 守卫、低栏探针与密集载荷控制、图像仲裁路径、side 车道、小窗口 1M 地板——逐项审过,均保持(既有测试全绿 + 新增 side-lane pin)。发现一个既有边界(非本 PR 引入):headroom 预留时 byte-doom 与熔断 byte floor 的窗口口径不一致,中间带 body 可穿过 preflight 却被熔断拦截——记录在案,不阻塞。
  6. 是否破坏缓存:不涉及(不触碰缓存/游标/去重键;折叠走既有压缩路径)。
  7. 是否改了设计方向:是——开放设计议题 [Design][preflight] Forensic probe (#2313) send will inevitably exceed window: one-400-for-evidence contract vs metering | [Design][preflight] 取证探针(#2313)发送必然超窗体:一次 400 换证据的契约 vs 计量档,需 owner 定方向 (#4543-③) #5976 的一个方向落地:探针尊重本地已强制的出站决策,超栏则折叠而非 413(比 [Design][preflight] Forensic probe (#2313) send will inevitably exceed window: one-400-for-evidence contract vs metering | [Design][preflight] 取证探针(#2313)发送必然超窗体:一次 400 换证据的契约 vs 计量档,需 owner 定方向 (#4543-③) #5976 的 413 选项更优:恢复轮次而非杀死轮次)。方向拍板归 owner,PR 已明确请求人工审阅、不请求 auto-merge。
  8. 是否引入了新的配置:不涉及(无新 env/config/默认值;MIN_ASSUMED_WINDOW_TOKENS 为既有常量)。

判定:代码与测试达标,两处 CI 红已当场修掉(继承红 merge ci-fixes + lane-guard 补 side-lane pin)。合并前置条件:① 新 head 27d0349efe 的 CI 全绿(运行中);② owner 对 #5976 探针策略方向的确认。

@ranxianglei

Copy link
Copy Markdown
Owner

此 PR 的内容已整体进入批量 PR #7107(逐 commit 原样 cherry-pick,作者与 commit message 保留,已合并)。

This PR's changes have been merged verbatim (commit-by-commit, authorship preserved) via the batch PR #7107. Closing this to avoid a duplicate merge. Thanks!

Afloat16 pushed a commit to Afloat16/billion-context that referenced this pull request Nov 18, 2026
… unreachable from the side lane

The preflight egress-fuse probe block keys on !prepared.sidePassthrough, but
side requests never reach preflight — handle.ts forwards them with a minimal
prepared before the preflight call, and bodies over the effective window are
already rejected locally by the side payload guard (ranxianglei#554, 413
side_request_payload_too_large). Pin that exact shape (stuck estimate meter +
body above the fuse's 1M floor) so a future change that routes side traffic
through preflight, or makes the new block apply to side, fails here.

Lane-suite discipline per ranxianglei#2170: the lane-symbol line in src/server.ts
(ranxianglei#7009) lands with the dedicated side-lane suite.

(cherry picked from commit 27d0349)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants