Skip to content

Over-limit exits: structurally doomed preflight answers 413, emergency nudge backs off per session - #3467

Merged
ranxianglei merged 1 commit into
masterfrom
wt-3461-ework-daemon
Oct 17, 2026
Merged

ranxianglei merged 1 commit into
masterfrom
wt-3461-ework-daemon

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Follows #3461 (fixes problem 2b + problem 2a host-side; problems 1 and 3 await owner decisions)

What & why

Real-model load testing (qwen3.8-27b @ sglang) found that an over-limit turn dead-ends in a 502 loop with no exit while EMERGENCY-tier nudge prompts get re-injected on every client retry (measured 7 injections in 1 second). Two independent fixes:

Problem 2a (host-side): structurally doomed preflight now answers 413 instead of 502. When preflight exhausts every foldable range and the gap cannot close by construction — futility math fired without any unusable draw, nothing-compressible exits, zero-dispatch exits — the final fail-fast answers 413 non-retryable (the #2847/#2921 convention: the content cannot fit, retrying cannot change the verdict). Relay-dependent exhaustion (empty/truncated/incomplete summaries, budget draws, pool saturation, sticky verdicts) deliberately stays 502 — that class is owned by retry/canary machinery. PreflightFailure gains an optional deterministic marker set only at the structurally doomed sites.

Problem 2b: per-session backoff on the emergency nudge arm. The EMERGENCY tier-1 arm was a pure function of contextUsage >= 0.7, so a headless client retrying an over-window turn re-injected the EMERGENCY prompt on EVERY request. emergencyNudgeArmed() gates the arm per session: the first pressure round injects, then re-arms only when a reduction landed (reductionKeyOf changed), a full flat growth step (default 50K, KDD#8) of tokens accumulated, or the 10-minute valve elapsed. Pressure admitted by every standing gate but held by the backoff reports nudgeOutcome: "gated", so self-heal streaks (#2155/#3004) keep crediting active rounds instead of starving — without this, zombie-lane D2 degrade would never arm under the new backoff.

Behavior changes (old → new)

  1. Structurally doomed preflight exhaustion: 502 → 413 non-retryable (futility math without unusable draws; nothing-compressible exits; zero-dispatch exits). Relay-dependent exhaustion unchanged (502). Headless clients stop spinning on a verdict that cannot change; the [Bug] Codex 额度中断后恢复长会话,preflight SSE 摘要为空并持续 502 #726 cooldown message is unchanged and now replays 413.
  2. Emergency tier-1 nudge: injected on every request ≥70% usage → injected once per state, re-arming only on a landed reduction / a full flat-50K growth step / the 10-minute valve. Normal (non-emergency) nudge cadence is untouched (kernel flat-50K discipline).
  3. Self-heal bookkeeping: pressure rounds held by the backoff count as active nudge rounds (gated) — D1 suppression and zombie-lane degrade still arm exactly as before; they would have starved under the new backoff otherwise.
  4. Structurally doomed image/byte-relay fail-fast lanes (issue1841 A, issue496 r2/r3, issue767, issue857) now assert 413; issue1841 B (fixture relay always returns empty summaries) stays 502 as relay-dependent.

Not covered here (owner decision needed, tracked in #3461)

Verification

查重: 问题一与在途 PR #3447 重复(同一机制,共享 egress 覆盖两条 WS lane),不另开修复;问题二/三无重复项(检索 open+closed issue/PR:超限死锁与紧急提示风暴、surface 成本均无既有条目,#2438/#3452 同家族但生产端不同)。

@github-actions

github-actions Bot commented Oct 17, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: wt-3461-ework-daemon (e54cabd)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.4137-pr.3467.76bbae7

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr3467.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重:不重复。本 PR 对应 #3461 的问题 2a/2b;正文已声明问题一与在途 PR #3447 重复(同一机制、共享 ResponsesWsUpstream egress),我会独立核实该重叠是否成立(读 #3447 的 egress 覆盖范围)。检索 open+closed issue/PR:超限死锁(413 出口)、紧急提示风暴(surface/backoff)均无既有条目;#2438/#3452 同家族但生产端不同,不构成重复。

开始独立审核:已拉取 wt-3461-ework-daemon(单 commit 21a30b527,16 文件 +411/−48,base 与 master 无分叉迹象)。按规范走:原 issue 方向关 → 回归审查 → 代码审查 → 测试有效性 → CI 状态,结论按结构化八问回复。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重:不重复(与首条回复一致)。另核实了与在途 PR #3447 的关系:#3447 只动 src/responses-ws.ts/src/fetch-util.ts + WS 测试,与本 PR 触碰的文件零交集,两个 PR 可任意顺序独立合并,无 hot-file 冲突。

审核方式:独立通读全部 16 个文件 diff(prpreflight/server/budget/handle/session-self-heal/4×prepare + 7 个测试),未照抄 PR 描述结论;在 PR head 本地重跑 typecheck 与定向测试;逐条对照代码验证行为变更披露。

审核中发现并已闭环的一个问题:最初推送的 commit 21a30b527 typecheck 红——prepare-google.ts:204 与 prepare-responses.ts:319 的 willInjectNudge 行漏了兄弟文件(anthropic/openai)都有的 !!turn.nudge && 收窄守卫,TS18048 ×2,导致 CI static-gates 双平台失败、test 车道全被取消。分支随后被重新提交为 9cda2c703(同一 commit message,仅这两行补上守卫),在我准备自己推修复之前已到位——未产生重复推送。以下审核结论均以最终 head 9cda2c703 为准。

八问

  1. 是什么问题:Three issues found during real-model stress testing: WS passthrough can't connect — HTTP upstream is guaranteed to die; | 真模型压测发现的三个问题:WS 直通连不上 HTTP 上游必死、超限时死锁+紧急提示风暴、compress 激活本身太重 #3461 问题二——上下文超限且折不够时,headless 客户端被卡进「发不出去也压不下去」循环:preflight 答 502(可重试语义)无限自旋,EMERGENCY 档提示秒级重复注入(实测 1 秒 7 次)。
  2. 什么原因导致的:两个独立机制缺陷——①「结构性无解」(折叠数学已证明任何组合都补不上缺口)与「relay 依赖型失败」(摘要空/截断、预算抽干、池饱和)被混用同一个 502,而前者对相同 payload 重试永远不可能成功;②紧急档触发条件是纯函数(usage≥70% + 有可执行 range),没有任何每会话状态,每次重试都重新武装。
  3. 影响范围与严重程度:任何车道(dsh/pi/codex/opencode,plugin 与 proxy 两种模式)命中「超限且可折叠量 < 需要减量」即触发;该 turn 完全不可用,风暴要磨满 5 轮 self-heal 才停(且只停风暴不解死锁)。
  4. 怎么修复的:① preflight 失败增加 deterministic 标记,只在纯折叠数学证明无解的 5 个出口置位(relay 依赖型一律不标);server 侧对 deterministic && !payloadFitsWindow 的耗尽答 413 不可重试(沿用 [Evidence] pi 车道切供应商(同 modelId)后 fold 不随会话迁移:裸上下文 2.23MB(≥298K est tokens)被 preflight 10 秒拒 3 次;账面 ≤157K 与裸水位背离 10.5h #2847/fix: doomed preflight rejection answers 413, not 502 (#2847) #2921/[Bug] 新会话(无 usage 基线)preflight 把字符数当 token——折叠其实成功(2263796→1111212)却 fail-fast 502,上游实际只计费 540828 #3146「can never fit as-is」既有约定,走同一个 failFast 通道,[Bug] Codex 额度中断后恢复长会话,preflight SSE 摘要为空并持续 502 #726 冷却期回放同样携带 413);② 新增 emergencyNudgeArmed() 每会话门:首个压力回合必发声,之后仅在「折叠落地(reductionKey 移动)/ 输入增长满一个 flat 50K 步 / 10 分钟阀门」时重新武装;被门控的压力回合计为 nudgeOutcome:"gated",self-heal 连击记账不饿死(会话卡死自愈机制:卡死模式检测与代理侧主动补救 #2155 zombie-lane 降级保持可用)。
  5. 是否有回归问题:逐项审过既有行为——
  6. 是否破坏缓存:不涉及。system prompt 字节稳定性未动,nudge 仍是尾部 ephemeral user message;门状态存 session.metadata.emergencyNudgeGate(随快照存活重启,/__bili/sessions 可见),不进 wire;前缀缓存/重试/去重键均未改。旧会话无此 key → 首个压力回合正常发声,向前兼容。
  7. 是否改了设计方向:没有转向——给既有 fail-fast 出口补上真实 HTTP 语义,给紧急档补上缺失的每会话状态;不引入新架构、新组件。
  8. 是否引入了新的配置:没有。退避常量复用既有 SELF_HEAL_SUPPRESS_CAP_MS(10min)与 loopConfig.nudge.growthFloor(flat 50K),无新 env/config/默认值。

行为变更清单(独立对照代码核验,与 PR 描述一致)

  1. 结构性无解的 preflight 耗尽:502(可重试语义)→ 413 不可重试;relay 依赖型保持 502。
  2. 紧急档 nudge:每次 ≥70% 请求都注入 → 每个状态注入一次(折叠落地/满 50K 增长/10min 阀门才重开)。
  3. self-heal 记账:被门控的压力回合计为活跃回合(gated)。
  4. byte-relay/image fail-fast 子例(issue1841 A、issue496 r2/r3、issue767 ×3、issue857 B/C)断言 502→413;issue1841 B 保持 502。

测试有效性

新测试断言有牙:单测钉死 [true,false×6] 序列、三个重开条件各自独立、低于阈值不落状态、shouldInject 绕过不消费门;e2e 走真代理(anthropic lane,window=8k)断言 413 + "NOT forwarded" + 冷却期零上游调用。既有测试改动均为「断言跟随有意行为变更」+ 注释说明为何某个相邻用例不变(issue1841 B、byte-relay 502 边界),不是放宽。

验证证据

  • 本地(head 9cda2c703):typecheck 干净;定向 12 个测试文件 78/78 绿(7 个改动文件 + 5 个相邻 nudge/emergency 套件);lane-decisions-matrix.test.ts 单独跑 4 次均 21/21。
  • CI:9cda2c703 全绿——19 success / 11 skipped(门控的真客户端 e2e 车道,常态)/ 0 failure。首轮 test (windows-latest, 22, 2) 曾红,定性为环境抖动而非回归:该 shard TAP 汇总 fail 0,7 个 not-ok 全是 cancelledByParent 的 teardown 型挂起(Promise resolution is still pending but the event loop has already resolved,起点 lane-decisions-matrix 第 1719 个子测),无任何断言失败;本 PR diff 不引入异步句柄/定时器,且该文件每个 row 只有单次压力请求、退避门对其观测上无效,harness 对非 200 会直接断言失败而非挂起;同文件的兄弟分支(wt-3342,WS client lane: fall back to HTTP/SSE egress when the upstream refuses the upgrade #3447)同平台同 shard 全绿。重跑后通过,佐证成立。

收工判定:未发现阻塞问题,CI 在 9cda2c703 全绿,建议合并(合并按仓库规则由人工执行)。本 PR 范围正确地不含 #3461 问题一(已由 #3447 覆盖)、问题二的 kernel 硬裁剪根治与问题三(surface 成本)——那三项仍在 #3461 线程等你拍板。

@ranxianglei

Copy link
Copy Markdown
Owner Author

冲突

…ergency nudge backs off per session (#3461)

Problem 2a (host-side): when preflight exhausts every foldable range and the gap cannot close by construction (futility math with no unusable draws, nothing-compressible exits, zero-dispatch exits), the final fail-fast now answers 413 non-retryable instead of 502. Relay-dependent exhaustion (empty/truncated/incomplete summaries, budget draws, pool saturation, sticky verdicts) deliberately stays 502 — retry/canary machinery owns that class. PreflightFailure gains an optional deterministic marker set only at the structurally doomed sites.

Problem 2b: the emergency tier-1 nudge arm was a pure function of contextUsage >= 0.7, so a headless client retrying an over-window turn re-injected the EMERGENCY prompt on EVERY request (measured 7 injections in 1 second). emergencyNudgeArmed() now gates the arm per session: first pressure round injects, then re-arms only when a reduction landed (reductionKeyOf changed), a full flat growth step (default 50K, KDD#8) of tokens accumulated, or the 10-minute valve elapsed. Pressure admitted by every standing gate but held by the backoff reports nudgeOutcome 'gated' so self-heal streaks (#2155/#3004) keep crediting active rounds instead of starving.

Test updates follow the status mapping: structurally doomed byte-relay/image lanes (issue1841 A, issue496 r2/r3, issue767, issue857) now assert 413; issue1841 B (fixture relay always returns empty summaries) reverts to 502 as relay-dependent; issue2155 idle-loop arithmetic pins the gated-round credit.
@ranxianglei
ranxianglei merged commit 63556d8 into master Oct 17, 2026
47 checks passed
@github-actions

Copy link
Copy Markdown
Contributor

📦 Merged Build Published

This merge to master was published as 0.0.525-master.63556d8 (rolling master tag).

npm install -g billion-context@0.0.525-master.63556d8  # this exact build
npm install -g billion-context@master  # always the newest merged build

npm view billion-context@master prints the current version at any time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant