Skip to content

[PR] fix: retain the longest active upstream throttle deadline - #7015

Closed
Afloat16 wants to merge 2 commits into
ranxianglei:masterfrom
Afloat16:2026-11-17_throttle-deadline-retention
Closed

Afloat16 wants to merge 2 commits into
ranxianglei:masterfrom
Afloat16:2026-11-17_throttle-deadline-retention

Conversation

@Afloat16

Copy link
Copy Markdown
Contributor

Problem

A later concurrent 429 can replace a 300-second Retry-After hold with a few seconds of ordinary backoff. That admits another provider request before the already-observed waiting period expires, on both account and model scopes.

Change

Keep the greater of the current deadline and the new observation's deadline when replacing a confirmed hold in the same scope. Streak counting, jitter, ownership, provisional classification, credential/model isolation, and successful-probe recovery remain intact.

Verification

  • Base: 301e1a5459a77d5578206927c8062a58381af642 (master, 0.1.215).
  • Six regression cases fail on the base and pass with the fix: account/model scopes crossed with absent, shorter, and invalid Retry-After headers.
  • The regressions exercise the public gate before expiry, at expiry, and through successful owned-probe recovery, and check other credentials/models remain independent.
  • An additional passing control verifies that a stronger HTTP-date Retry-After can extend the hold.
  • 53 tests passed across the 429 governor, configurable scope, loop observation, and degenerate refetch integration suites.
  • npm run typecheck, npm run build, and git diff --check: passed on Node 24.22.1.

Fixes #7014

@ranxianglei ranxianglei changed the title fix: retain the longest active upstream throttle deadline [PR] fix: retain the longest active upstream throttle deadline Nov 17, 2026
@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

Duplicate screening: no duplicate. Searched open PRs for throttle/429/deadline/Retry-After — #7015 is the only PR fixing this. (#4772 wires the governor into a lane; #6154 is credential-fingerprint — different mechanisms.)

Verdict: the fix is correct and mergeable. I verified the regression tests fail on the base and pass on the head, and that all 4 red CI lanes are inherited from master, not caused by this PR.

What I verified (evidence, not "should work"):

  • Base 301e1a5459 (current master): the 6 new regression tests FAIL — nextAttemptAt lands at 1002655 (T0+1000 + ~1.6s jittered backoff) instead of 1300000 (T0+300s). That is exactly the reported bug: a 300s hold shrunk to ~2.6s.
  • PR head ea4f9c7821: all 36 tests in tests/issue4204-throttle-governor.test.ts pass.
  • The control test (a stronger HTTP-date Retry-After still extends the hold) passes on both base and head — the fix does not over-suppress.
  • The 2-line fix sits in noteUpstreamThrottle429 (the direct observation path) and mirrors the Math.max clamp that already exists in the provisional settlement path (settleProvisionalObservation, lines 650/654) — it closes a real asymmetry, in the right place.

CI status (verified live): 4 lanes red — test (win22/3), test (ubu22/3), test (ubu24/3), architecture-docs. I checked out the master base and confirmed ALL FOUR are inherited red, not caused by this PR:

结构化八问(中文,供 reviewer):

  1. 是什么问题:同一 origin+credential(+model) 已有活跃 hold 时,一个并发 429 若缺 Retry-After / 带更短值 / 带无效值,会把已确认 hold 的截止时刻覆盖成几秒的普通退避,在已观测到的等待期结束前又放行一个上游请求(account 与 model 两个 scope 都中招)。
  2. 什么原因导致的:noteUpstreamThrottle429 直接路径里,createHoldState 用「本次观测」重算 nextAttemptAt(now + max(jittered 退避, 本次 Retry-After))后无条件 holds.set 覆盖旧状态,没有与旧 hold 截止时刻取 max。provisional 结算路径(settleProvisionalObservation)本就有 Math.max 钳制,直接路径漏了——这是真实存在的不对称,不是表面现象。
  3. 影响范围与严重程度:仅当「同 scope 已有活跃 hold + 又来并发 429 且其 Retry-After 缺失/更短/无效」时触发;后果是 hold 被缩短、提前放行对已限流 provider 的请求(放大限流压力)。特定条件下的行为缺陷,非全量不可用——p2。
  4. 怎么修复的:直接路径覆盖旧状态前对 nextAttemptAt 取 max(新, 旧)——只可能拉长、不可能缩短 hold,与 provisional 路径既有钳制对齐。更强的新 Retry-After(HTTP-date)仍能延长;到期与成功 owned-probe 仍能释放。
  5. 是否有回归问题:逐项审过既有行为——首次 429(previous=undefined,if (previous) 守卫,不受影响);streak 计数(仍 +1,测试钉 streak=2);jitter(仍作用于新算退避,被钳制后无副作用);probe 所有权(probeToken/probeAt 仍从 previous 继承,未动);provisional 分类(未动,且本就有钳制);credential/model 隔离(stateKey 含 scope,测试钉其他 cred/model 不受影响);到期释放 + 成功 probe 释放(均保留,测试钉)。钳制只增不减,且仍受 now+600s(RETRY_AFTER_CAP_MS)上界约束,无无限增长。双向测试钉住(保留更长 + 更强可延长)。无回归。
  6. 是否破坏缓存:不涉及。429 governor 是 egress 侧内存态限流,不动缓存/重试游标/去重键。
  7. 是否改了设计方向:不涉及。只是把直接路径补齐到与 provisional 路径一致的「截止时刻只增不减」语义,无架构/边界/数据流转向。
  8. 是否引入了新的配置:不涉及。无新 env/config/默认值。

Merge verdict: code is correct, minimal (2 lines + 36 test lines), well-tested in both directions, no regressions, no kernel/config/wire touch. The only blocker is inherited master CI red — not this PR's fault, and not clearable from here (no fork push access). Ready to merge once ci-fixes is folded into the branch (or after it lands on master).

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

Status check (my full review is floor 2): the code verdict stands — correct and mergeable, evidence in floor 2 (6 regression tests fail on base 301e1a5459, 36/36 pass on head ea4f9c7821, no regressions, no kernel/config/wire touch).

On CI: the 4 red lanes are all inherited from master (verified on master base, not this PR's fault). Per floor 3, the CI engineer has now taken over the red items + conflicts, which is exactly the resolution path — ci-fixes (7247410976) already carries both fixes and I verified the merged tree is green locally (36/36 + #2929 + arch-docs + typecheck). Once the CI engineer folds ci-fixes into the branch, CI should go green and this is ready to merge.

Nothing further needed from me on the code.

Merge upstream master 6a641f3, including the architecture module-map and schedule-independent ranxianglei#2929 probe fixes.

Preserve the existing PR head and exact contribution patch. Validation of this refreshed head runs in upstream CI; no local test rerun is claimed.
@Afloat16

Copy link
Copy Markdown
Contributor Author

(ai: OpenAI/gpt-6-astra)

The CI integration is now on this existing branch in c9e789d957c87cac1915d585b39a15c3666c7294. It merges current master 6a641f378537e06e7430ec90c9abf035ffa12377, which already contains ci-fixes, while preserving the original throttle fix, tests, and commit history.

The refreshed head has triggered CI run 44050365583. Its main workflow is still pending at 2026-11-17T09:30:54.486Z; the other nine workflows have passed. The final CI result remains pending.

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

Re-verified the live check state before touching anything: the failures this wake reported (test shard 3 on ubuntu-22 / ubuntu-24 / windows-22 + architecture-docs) were against the old head ea4f9c7821. The branch head is now c9e789d957 ("fix: sync PR #7015 with merged upstream CI fixes" — a merge of upstream master 6a641f3785, which carries the architecture module-map and schedule-independent #2929 probe fixes).

On c9e789d, all 48 check runs have completed: 36 success, 12 skipped, 0 failures — including the four originally-failing checks (test ubuntu-22 shard 3, test ubuntu-24 shard 3, test windows-22 shard 3, architecture-docs), all green.

No code changes or pushes were needed this run. CI is fully green on the current head; the PR is ready for human review/merge.

七问:

  1. 是什么问题: 本事件报告的 4 个 CI 失败针对的是旧 head ea4f9c7821,不是当前 head。
  2. 什么原因导致的: 旧 head 落后于 master,继承了 architecture-docs 与 codex 命中率问题分析 #2929 probe 的失败(陈旧基线,非本 PR 代码缺陷)。
  3. 影响范围与严重程度: 仅 CI 红,不影响运行行为;当前 head 已全绿。
  4. 怎么修复的: 作者已在本次唤醒前推送 c9e789d(同步 master),无需我再改代码。
  5. 是否有回归问题: 本次运行未做任何改动,无回归。
  6. 是否破坏缓存: 不涉及。
  7. 是否引入了新配置: 不涉及。

判定: 已收工,CI 全绿,等待人工 review/merge。

@ranxianglei

Copy link
Copy Markdown
Owner

This fix is included verbatim (original commits and authorship preserved) in the merged batch PR #7074 — the changes are already on master. Closing to avoid a duplicate merge. 该修复已随批量 PR #7074 原样合入 master,为避免重复合并,关闭此 PR。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

A later concurrent 429 shortens an active Retry-After hold

2 participants