Skip to content

[PR] fix: clarify preflight fail-fast NOT-forwarded message is per-request (#7344) - #7368

Open
ranxianglei wants to merge 1 commit into
masterfrom
2026-11-20_failfast-log-precision
Open

ranxianglei wants to merge 1 commit into
masterfrom
2026-11-20_failfast-log-precision

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

⚠️ MODEL-FACING TEXT CHANGE: the preflight fail-fast error message now says "This request's over-window payload was NOT forwarded to the upstream." instead of "The over-window payload was NOT forwarded."

What

The preflight fail-fast log line and client-facing error message ended with "The over-window payload was NOT forwarded." The definite article "The" reads as a global statement — as if no over-window payload on the proxy was forwarded. In practice the guarantee is per-request: this specific request's payload was withheld, while a concurrent request (different session, or a client retry on the same session after the lock releases) can still forward its own over-window payload via a forward-once arm (probeForEvidence #2313, imageArbitration #496/#1800, post-fold #3146/#4185).

Why

Issue #7344: an operator saw preflight fail-fast 502 ... NOT forwarded and ← upstream 400: requested 1667733 tokens in the same millisecond and read them as contradictory. The 400 was from a different (larger) request, not the fail-fasted one. The old wording sent the operator down the wrong debugging path (checking window/threshold config that was actually correct).

Fix

Change the final clause from "The over-window payload was NOT forwarded." to "This request's over-window payload was NOT forwarded to the upstream." — making the scope (this request) and the destination (the upstream) explicit. The ← upstream log in relay.ts already carries the session id, so operators can now correlate the two lines and see they are different requests.

Regression test

tests/issue1993-preflight-400.test.ts:205 pinned the old exact string; updated to the new string. All other "NOT forwarded" assertions use substring/regex matches that still pass.

Closes #7344

@ranxianglei ranxianglei changed the title fix: clarify preflight fail-fast NOT-forwarded message is per-request (#7344) [PR] fix: clarify preflight fail-fast NOT-forwarded message is per-request (#7344) Nov 20, 2026
…#7344)

⚠️ MODEL-FACING TEXT CHANGE: the preflight fail-fast error message now
says "This request's over-window payload was NOT forwarded to the
upstream." instead of "The over-window payload was NOT forwarded."

The old wording ("The over-window payload") reads as a global statement,
which is misleading when a concurrent request (different session, or a
client retry) forwards its own over-window payload via a forward-once arm
(probeForEvidence #2313 / imageArbitration #496/#1800 / post-fold
#3146/#4185). The new wording makes it clear that the NOT-forwarded
guarantee is scoped to THIS specific request, not to all in-flight
traffic on the session or the proxy.

The upstream 400 log (relay.ts ← upstream) already carries the session
id, so operators can now correlate the two lines and see they are
different requests rather than a contradiction.

Closes #7344
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重: 不重复 — 本 PR 是 #7344 措辞问题的唯一修复,无并列 PR/issue 动同一处。

Reviewed PR #7368 (closes #7344). Clean 2-line change: src/server.ts:4222 (the fail-fast message) + tests/issue1993-preflight-400.test.ts:205 (the one test pinning the old exact string).

What I verified

  • The message feeds BOTH the log line (src/server.ts:4223) and the returned client-facing 502 error (src/server.ts:4224), matching the PR description.
  • The PR's test claim holds. I grepped every NOT forwarded occurrence in the repo: only tests/issue1993-preflight-400.test.ts:205 pinned the old exact full string. All other assertions use substring (includes("NOT forwarded")), regex (/NOT forwarded/), or are assertion-failure messages (not matched against output) — all still pass. No other test breaks.
  • The model-facing text change is safe: the new string carries no numbers/refs/positions the model could anchor on, and it is strictly more disambiguating (scope "this request" + destination "the upstream"). No phantom-value risk.
  • No semantic/behavior change — wording only. The fail-fast decision, status, retryable, and respond logic are untouched.

One finding (filed, non-blocking)
#7344 raised a deeper concern: the fail-fasted payload might have been already in flight and un-cancellable, in which case the message should say "already in flight, cannot cancel", not "NOT forwarded". PR #7368 resolves the immediate confusion by asserting the 400 was a different, larger request (supported by the token counts: fail-fasted ~1M vs the 400 at 1.67M), but it does not include a code-level verification that the proxy never forwards before the fail-fast decision. I filed a drive-by issue for that open verification (titled "[Drive-by] Verify preflight fail-fast has no forward-before-decision path", mirror #7377). It does not block this PR — the wording fix is correct either way for the fail-fasted request.

CI: still in progress — all 26 checks queued as of the last mirror-cache read (07:23 UTC). I'll confirm green once the test lanes complete; the code-review verdict below is independent of CI.


结构化八问

  1. 是什么问题: preflight fail-fast 的报错/日志结尾 "The over-window payload was NOT forwarded." 用定冠词 "The" 读起来像全局声明(仿佛代理上没有任何超窗 payload 被转发过),但真实保证是按请求的——只拦了这一个请求,并发请求仍可各自转发。措辞与保证的作用域不匹配,把排查带偏(去查本来就正确的窗口/阈值配置)。
  2. 什么原因导致的: 文案用了暗示全局作用域的定冠词,而机制的真实保证是 per-request。是文案与语义作用域不一致,不是机制 bug。
  3. 影响范围与严重程度: 影响读 fail-fast 日志的运维 + 收到 502 的 agent/模型。严重程度 = 体验瑕疵(仅误导排查方向);不影响正确性——fail-fast 场景下该请求的 payload 确实没被转发,无数据丢失、无功能破坏。
  4. 怎么修复的: 把结尾从句改为 "This request's over-window payload was NOT forwarded to the upstream.",显式点明作用域(本请求)与目的地(上游);同步更新唯一钉死旧完整串的测试。
  5. 是否有回归问题: 无。已逐项核验:① 全仓 grep NOT forwarded,只有 tests/issue1993-preflight-400.test.ts:205 钉死旧完整串,其余全是 substring/regex/断言失败文案,均不受影响;② 模型可见文本安全(新串无数字/ref/位置可被模型锚定,且更消歧);③ 无语义/行为变化,仅错误文案。
  6. 是否破坏缓存: 不涉及。纯错误文案改动,不触碰缓存/重试/增量游标/去重键。
  7. 是否改了设计方向: 不涉及。无架构/边界/数据流转向,只是既有 fail-fast 路径内的措辞澄清。
  8. 是否引入了新的配置: 不涉及。无新 env/config/默认值。

判定: 代码层面可合——方向对、diff 干净(2 行)、测试钉死、无回归。待 CI test lanes 转绿后由 owner 合入。#7344 的"在飞绕过"疑虑已单独立项(mirror #7377)跟进,不阻塞本 PR。

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

After fail-fast, the payload in the same round was still sent upstream. | fail-fast 之后,同一轮的 payload 仍然被发到了上游

1 participant