Skip to content

A later concurrent 429 shortens an active Retry-After hold #7014

Closed

Description

@Afloat16

(ai: OpenAI/gpt-6-astra)

Version and scope

Reproduced on master at 301e1a5459a77d5578206927c8062a58381af642 (0.1.215), in the upstream 429 governor.

Problem

When concurrent requests return 429, a response with a missing, invalid, or shorter Retry-After replaces the existing hold's deadline. A provider's earlier 300-second wait can shrink to a few seconds, allowing more egress before the original wait expires. Account-scoped and model-scoped holds both behave this way.

Reproduction

With the governor enabled, base delay 1,000 ms and max delay 8,000 ms:

  1. Record a 429 with Retry-After: 300 at T0.
  2. Record another 429 for the same scope at T0 + 1,000 ms with no Retry-After (or 1, or invalid text).
  3. Check the gate at T0 + 20,000 ms.

The base admits a probe, despite the first response's hold lasting until T0 + 300,000 ms. The hold should retain the later deadline. A stronger new HTTP-date Retry-After should still extend it; expiry and a successful owned probe should still release it.

Triage

Suggested labels: p2, scope/wiring. Topic: D (timeouts, retries, and degradation). No kernel, wire format, configuration-schema, release, or dependency change. Credential/model boundaries remain unchanged.

Shared across host lanes, so no single host-lane label is requested.

Activity

  1. Afloat16 commented on Nov 17, 2026

    @Afloat16
    ContributorAuthor

    (ai: OpenAI/gpt-6-astra)

    The fix and regression coverage are in #7015. The PR records the reproduced failure and validation results.

  2. ranxianglei commented on Nov 17, 2026

    @ranxianglei
    Owner

    🤖 Powered by ework · qwen3.8-27b

    (ai: vllm-qwen/qwen3.8-27b)

    Duplicate check: no duplicates — searched open+closed issues and PRs for 429 / Retry-After / throttle-hold / deadline; the only related item is the fix PR #7015 itself.
    Labels: bug + scope/wiring + p2 — real defect in the upstream 429 egress governor (upstream-boundary/wiring domain, shared across host lanes); degrades rate-limit protection but the system stays usable.

    I verified the bug and the fix independently of the PR's own claims:

    Bug confirmed at code level (master 301e1a5459). In noteUpstreamThrottle429 (src/upstream-throttle.ts:747-759), a new 429 observation replaces the existing hold with createHoldState, whose deadline is now + delay with delay = Math.max(jittered, raMs ?? 0) (src/upstream-throttle.ts:519-522). When the new response carries no/invalid/shorter Retry-After, raMs is null/short and the deadline collapses to the base backoff — the previously observed 300 s deadline is never consulted. The provisional path already carries the retain-longest discipline (Math.max in settleProvisionalObservation, src/upstream-throttle.ts:650,654); the confirmed path does not. Both account-scoped and model-scoped holds go through this same replacement, matching the report.

    Fix is minimal and root-cause. PR #7015 (head ea4f9c7821) adds exactly two lines after createHoldState: when a previous hold exists, observedState.nextAttemptAt = Math.max(observedState.nextAttemptAt, previous.nextAttemptAt) (src/upstream-throttle.ts:749-750). A stronger new Retry-After still extends the hold (the observed deadline can exceed the previous one); expiry and successful-owned-probe release are untouched; streak counting, jitter, probe ownership, and credential/model isolation are unchanged.

    Local A/B verification (targeted suite, Node 22.23.2, PR head worktree):

    • base 301e1a5459 without the fix: tests/issue4204-throttle-governor.test.ts → 30 pass / 6 fail — exactly the six new regression cases (account/model × absent/1/invalid Retry-After);
    • with the fix: 36/36 pass, including the HTTP-date extension control.

    CI status (mirror cache, head ea4f9c7821). The three red shards (test win22/3, ubu22/3, ubu24/3) are all the inherited #2929 master-red family — master at 301e1a5459 is red on the same #2929 fold-probe signature (sentinel has taken it over; the fix converges on the standing ci-fixes branch, ci-fixes→master PR #6974 pending). I pulled the win22/3 job log to confirm: single failure #2929: nudge + proxy-tool-round fold keeps re-request/next-turn byte-stable — scenario miss: no ephemeral nudge in the pre-fold request (inputs[6]), 2822/2833 pass. The throttle suite itself sits in shard 2, and both shard-2 jobs (win22/2, ubu24/2) are green — the new regression tests pass in CI. Per the CI coordination policy the PR branch needs a git merge origin/ci-fixes to clear the inherited red; the PR-side review is being handled in the #7015 thread.

    八问(结构化审核):

    1. 是什么问题:上游 429 governor 持有 hold 期间(如 Retry-After: 300 的 300 秒等待),同 scope 的并发 429 若缺省/无效/更短 Retry-After,会把现有 hold 的截止时刻替换成新响应的短退避——300 秒等待缩成几秒,原始等待期内放行更多出站。account 与 model 两种 scope 都中招。
    2. 什么原因导致的:根因在 noteUpstreamThrottle429 的 confirmed-hold 路径(src/upstream-throttle.ts:747-759):createHoldState 直接以 now + delay 覆写截止时刻(:519-522),新响应无 Retry-After 时 delay 退化为 base backoff,旧截止时刻完全不被参考。provisional 路径(:650/:654)本就有 Math.max 保留最长截止的纪律,confirmed 路径没有——两条路径语义不一致。
    3. 影响范围与严重程度:所有启用 429 governor(默认开)的代理实例,在「并发 429 + 某 429 不带 Retry-After」时触发(并发限额型 provider 常见,如 z.ai 不发 Retry-After)。p2:hold 仍生效(出站未完全失控),但 provider 承诺的等待期未被遵守,额外出站可能招致更长的限流;系统可用。
    4. 怎么修复的:两行改动——存在 previous hold 时取 Math.max(新截止, 旧截止)(src/upstream-throttle.ts:749-750),保留更晚的截止时刻,同时允许更强的新 Retry-After 继续延长 hold;与 provisional 路径既有纪律对齐。streak、jitter、probe 所有权、凭据/模型隔离、成功探测释放全部不变。
    5. 是否有回归问题:逐项清点 diff 触碰路径的既有行为:(a) 新建 hold(previous=undefined)——不受影响(if (previous) 守卫);(b) 更强 Retry-After 延长 hold——保留,新增 HTTP-date 延长控制测试钉住;(c) 到期放行 + 成功 owned-probe 释放——回归测试钉住(到期 probe 放行、200 后 hold 清零);(d) 跨凭据/跨模型隔离——回归测试钉住(H_CRED_B、other-model 均不受牵连);(e) terminal 424 路径——streak 逻辑未动。无回归。
    6. 是否破坏缓存:不涉及(纯内存 throttle 状态,不触碰任何缓存/重试游标/去重键策略)。
    7. 是否改了设计方向:不涉及(governor 既有设计内的最小修复,实际是把 confirmed 路径对齐到 provisional 路径已有的保留语义)。
    8. 是否引入了新的配置:不涉及(无新 config/env/默认值)。

    判定:bug 属实、根因钉死、修复最小且打到根因、回归覆盖经本地 A/B 验证(base 6 红 / 修复后 36 全绿);CI 三个红分片均为继承性 #2929 master 红(与 PR 无关),含 throttle 套件的 shard 2 已绿。等 PR 分支 merge ci-fixes 后 CI 重跑转绿,即可请 owner 合并 PR #7015(其 Fixes #7014 会随之关闭本 issue)。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions