Repository navigation
[PR] fix(wiring): feed the loop lane's egress sends into the 429 governor - #4772
ranxianglei wants to merge 7 commits into
Conversation
…4743) The upstream 429 governor (upstreamThrottleLocalReject / noteUpstreamThrottle429 / noteUpstreamThrottleResolved) was wired only on the relay forwarding path. Three loop-lane chokepoints bypassed it, so a 429-storming upstream never armed the egress hold on the lane most prone to storms: - src/compress-loop-responses.ts — Responses-wire compress loop round-2+ resends - src/preflight.ts — preflight refold summary calls - src/reasoning-guard.ts — reasoning-guard continuation rounds Each now reports its verdict to the observe pair: a 429 arms the egress hold for the next client turn; a settled non-429 clears it. Observe-only (no local gate), matching the main-loop fix in #4725 — a mid-loop local reject would change re-request behavior under load. The observation split follows each site's fetch primitive: fetchWithRetry sites report at the settled boundary (a 429 exhausting the budget throws UpstreamHttpError(429)); fetchWithTimeout sites report per response, like relay. Adds tests/issue4743-loop-lane-429-observe.test.ts — three wiring pins that drive the real code path against a 429 upstream and assert the hold armed; they fail without the fix (verified: 0/3 pass on the unpatched tree). Closes #4743
(ai: vllm-qwen/qwen3.8-27b) 这是 agent PR( 查重:不重复。 与 #4725(主循环,现为空壳 PR)、#4752(degenerate-retry,在途)同属 #4743 覆盖家族;本 PR 覆盖的三个文件(responses-wire compress loop / preflight refold / reasoning-guard rounds)与它们互不相交,无其他重叠条目。 标签: bug / scope/wiring / p2(wiring 遗漏;#4204 已覆盖主转发路径,此处是 429 风暴下 loop 车道放大问题,非整体不可用) 本地核验(PR head e7e38d4,base = 当前 master,落后 0 commit):
review 中发现一个小偏差,稍后补独立标注 commit:compress-loop 站点在 fetchWithRetry 抛出非 429 的 UpstreamHttpError(如 fail-fast 400)时没有上报 resolved——而 governor 契约(#4204 模块头注释「ANY resolved non-429 … clears the hold」)以及 relay 和 #4752 的写法都是任何已决定的非 429 都清 hold。一行对齐 + 一行测试钉住。 |
… non-429 (#4743 review fix) fetchWithRetry throws UpstreamHttpError for terminal non-ok verdicts; the boundary observation armed on 429 but stayed silent on e.g. a fail-fast 400, whereas the #4204 contract clears the hold on ANY resolved non-429 (the endpoint demonstrably processed a request) — relay parity, same as the sibling #4752 site. Transport deaths remain unobserved (circuit-breaker territory, #3148). The responses-wire pin gains a re-arm -> fail-fast-400 phase asserting the hold lifts (verified failing without the fix line).
(ai: vllm-qwen/qwen3.8-27b) #4772 审核结论(agent 自报 PR,审核模式:不重做,核对代码 + 盯 CI)。 做了什么
八问
状态与判定
|
📦 Built Package ArtifactBranch: Latest build — install this exact versionnpm install -g billion-context@0.0.9437-pr.4772.de40aa7This exact version is immutable and reproducible. The rolling Recent builds (commit → version)
Download artifact
tar xzf billion-context-pr4772.tgz
npm install -g packageInstall instructions are refreshed on each push. |
… per-response (#4743 review fix) The per-response arm added by this PR in the preflight walk self-blocked: the fits-case forward runs AFTER preflight and passes the relay gate, so a summary-probe 429 armed the hold that then answered the SAME request's own forward locally — violating the #301 contract (payload fits + preflight 429 → still forwarded; CI red on tests/preflight-fail-fast.test.ts, reproduced locally: 429 !== 200). Fix: the walk reports resolved per response (settled non-429 clears); the arm moves to the caller's withhold verdict in server.ts (kind === upstream && status === 429 → noteUpstreamThrottle429 with null retry-after). Nothing is forwarded on a withheld request, so the armed hold only answers the NEXT turn's relay forward locally — exactly the #4743 ask. Doomed-413 exits stay un-armed (they spend no summary call / different evidence). Pins: new e2e in tests/preflight-fail-fast.test.ts (503 withhold arms → next small turn gets local 429 stage:forward, zero egress; FAILS without the arm line — verified). The issue4743 preflight pin now asserts the anti-self-block (no arm from a 429 probe alone) + cleared-on-200.
(ai: vllm-qwen/qwen3.8-27b) Review update — found and fixed a self-block regression in this PR's preflight site. My earlier "mergeable once CI green" verdict is superseded: CI went red exactly where predicted, and it was this PR's code, not inherited master red. Fix is pushed as Mechanism. The per-response 429 arm added in the preflight walk ( Fix (
Pins. New e2e in Structured 八问:
行为变更披露(vs 本 PR 第一版,非 vs master):preflight 摘要 429 不再在请求中途武装 hold,改在 withhold 判决时武装。相对 master 的净效果:fits 场景行为不变(照常转发);withhold 场景从「不武装」变为「武装下一轮闸门」——后者即本 PR 的功能本身。 判定:代码审核完成(两轮 review fixes 均已落地: |
…e-429-observe # Conflicts: # src/server.ts # tests/preflight-fail-fast.test.ts
(ai: qwen/qwen3.8-27b) Merge conflict resolved and pushed — the PR is now clean against master. What was done this run
八问
收工判定:冲突已解决并推送( |
[ework] CI fix verified green ✓ |
…vernor Review fix for #4772: the PR wired the Responses-wire compress loop, the preflight refold, and the reasoning-guard continuation rounds into the 429 governor, but missed the structurally-identical non-stream Anthropic-wire compress loop (compress-loop-anthropic.ts) — its round-2+ resends bypassed the governor, so a 429-storming upstream on that lane never armed the egress hold for the next client turn. Mirrors the Responses-wire observe pair: a settled ok response clears the hold; a terminal 429 arms it; any other settled non-429 clears it (relay parity, #4204); transport death stays unobserved (circuit-breaker territory). Adds a wiring pin (tests/issue4743-loop-lane-429-observe.test.ts) that drives the real loop against a 429 upstream and asserts the governor armed — fails on the pre-fix tree, passes with the fix.
ranxianglei
left a comment
There was a problem hiding this comment.
(ai: vllm-qwen/qwen3.8-27b)
Verdict: APPROVE
The PR correctly feeds the loop lane's egress sends into the 429 governor. I verified all three chokepoints it wired (Responses-wire compress loop, preflight refold, reasoning-guard continuation rounds) against the governor contract: observe-only (no local gate), arm on terminal 429, clear on any resolved non-429 (relay parity, #4204), transport death unobserved (circuit-breaker territory, #3148). The preflight anti-self-block is correct — arm at the caller's withhold verdict (server.ts), not per-response, so the same request's own forward of a fitting payload is never blocked (#301 contract). Tests have teeth: reverting the 4 src files to master makes all 4 wiring pins + the e2e #4743 pin fail; with the PR they pass.
已顺手修:wired the non-stream Anthropic-wire compress loop (src/compress-loop-anthropic.ts:150) into the 429 governor — the structurally-identical loop-lane chokepoint the PR missed (it wired the Responses-wire sibling but not the Anthropic-wire one). Mirrored the Responses-wire observe pair (ok→clear / terminal-429→arm / other-terminal→clear / transport-death→unobserved) + added a wiring pin (tests/issue4743-loop-lane-429-observe.test.ts, verified fail-on-pre-fix / pass-with-fix). + commit 242aaae
八问(中文,供人工 reviewer)
- 是什么问题:上游 429 governor 此前只接在 relay 转发路径上;loop 车道的出站请求(压缩循环 round-2+ 重发、preflight refold、reasoning-guard 续轮)绕过 governor,429 风暴的上游无法为下一个 client 轮次 arm egress hold。
- 什么原因导致的:governor 的 observe pair(noteUpstreamThrottle429 / noteUpstreamThrottleResolved)只在 relay 转发路径被调用;loop 车道的 fetchWithRetry / fetchWithTimeout 调用点没有接 observe。
- 影响范围与严重程度:observe-only(无本地 gate),不影响正确性;影响 429 风暴场景下 loop 车道(最易风暴的车道)的 egress hold 覆盖度——该车道出站请求不 arm hold,下一个 client 轮次可能再次风暴上游。特定条件下有问题(429 风暴 + loop 车道)。
- 怎么修复的:在 loop 车道各出站 chokepoint 加 observe pair——ok 响应→clear、终态 429→arm、其它终态非 429→clear(relay parity #4204)、传输死亡→不观测(circuit-breaker 领域 #3148)。preflight 的 arm 放在 caller 的 withhold verdict(server.ts),不 per-response(避免 self-block #301)。
- 是否有回归问题:observe-only,不改 re-request 行为、不改 wire 形状、不改本地 gate。高危模式逐项过:无语义漂移、无 fail-fast 变吞错、无阈值/时序/输出格式变化。
- 是否破坏缓存:不涉及缓存/重试/增量游标/去重键失效策略。
- 是否改了设计方向:不涉及架构/边界/数据流转向。
- 是否引入了新的配置:不涉及新 env/config/默认值。
合并判定:可合(APPROVE)。已顺手修掉 PR 漏掉的 Anthropic-wire 压缩循环 chokepoint(commit 242aaae);CI 会在新 head 上重跑确认。
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) PR #4772 review complete — Verdict: APPROVE. Reviewed the full diff (6 files, 327 ins/14 del): the loop lane's egress sends (compress-loop-responses, preflight, reasoning-guard) are now wired into the 429 governor (observe-only: arm on terminal 429, clear on resolved non-429, transport-death unobserved). Found one small edge miss — Full review (APPROVE + 已顺手修 line + Chinese 8-question checklist): #4772 (review) |
[ework] CI fix verified green ✓ |
Summary
The upstream 429 governor (
upstreamThrottleLocalReject/noteUpstreamThrottle429/noteUpstreamThrottleResolved, #4204) was wired only on the relay forwarding path. Three loop-lane egress chokepoints bypassed it, so when the upstream continuously returns 429 for compress requests, the governor never arms on the lane most prone to storms. This wires them in.What changed
Each site now reports its verdict to the governor's observe pair — a 429 arms the egress hold for the next client turn; a settled non-429 clears it:
Observe-only, no local gate — matching the main-loop fix in #4725. A mid-loop local reject would change re-request behavior under load; the issue's ask ("stop amplifying") is met by arming the hold so the NEXT turn's relay forward answers locally instead of storming again.
The observation split follows each site's fetch primitive:
fetchWithRetryresolves only ok responses and throwsUpstreamHttpError(status)on a terminal non-ok, so those sites report at the boundary;fetchWithTimeouthands back the raw response, so those sites report per response exactly as the relay path does.Coverage map (#4743 family)
These three together close the loop-lane coverage for #4743; they touch disjoint files.
Tests
tests/issue4743-loop-lane-429-observe.test.ts — three wiring pins that drive the real code path against a 429 upstream and assert the hold armed (and cleared on a later 200). Unlike the #4725 governor-contract pin, these FAIL on the unpatched tree (verified locally: 0/3 pass without the fix, 3/3 with). Targeted regression suites for the touched paths (reasoning-guard, preflight-transport-recovery, preflight-transient-empty-retry, responses-guard): 25/25 green locally.
Closes #4743
Review fixes (agent review, #4772 thread)
50b2bf859— loop-lane boundary report: a terminal NON-429UpstreamHttpError(e.g. fail-fast 400) now reportsresolvedto the governor, matching the Provider 429 (concurrency-style limit): turn re-driven every few seconds for minutes, keeps the account pinned at the limit #4204 contract ("ANY resolved non-429 … clears the hold") and the relay / [PR] fix: review-findings wave — 7 fixes from the #4725 pre-release audit (P1×2 + P2×3 + P3×2) #4752 sites; transport deaths stay unobserved ([bug] 上游在会话中途不可用时,bili 把一次故障放大成四重代价:固定节奏 20 连击全部冷发、输出预算被静默钳制 83 次(最狠 65536→1360)、紧急收缩与 nudge 停摆均不可见 #3148 territory). The responses-wire pin gains a re-arm → fail-fast-400 phase asserting the hold lifts (verified failing without the fix line).c1e0b726b— self-block regression found in review and fixed. The per-response 429 arm in the preflight walk violated the fix: honor anthropic-beta context-1m to size the context window #301 contract: the fits-case forward runs AFTER preflight and passes the relay gate, so a summary-probe 429 armed the hold that then answered the SAME request's own forward locally (CI red: tests/preflight-fail-fast.test.ts fix: honor anthropic-beta context-1m to size the context window #301 e2e, reproduced locally429 !== 200). Fix: the walk reports resolved per response only; the arm moves to the caller's withhold verdict in server.ts (kind === "upstream" && status === 429→ note with null retry-after). Nothing is forwarded on a withheld request, so the armed hold answers only the NEXT turn's relay forward locally — the [wiring] 429 governor is invisible to the loop lane (compress round-2+ resends do not go through gate/observe) #4743 ask is preserved. Doomed-413 exits stay un-armed. New e2e pin (503 withhold arms → next small turn gets local 429 stage:forward, zero egress) verified failing without the fix line; issue4743 preflight pin now asserts the anti-self-block direction.Behavior change vs this PR's first revision (not vs master): preflight summary 429 no longer arms the hold mid-request; it arms at the withhold verdict instead. Net effect on master behavior: unchanged for the fits-case (still forwards); the withhold-case now arms where master did not (that is the feature).