Repository navigation
[PR] docs: document the preflight fail-fast egress invariant (#7376) - #7385
ranxianglei wants to merge 1 commit into
Conversation
Record at the fail-fast decision point (the only egress gate for the main payload) that preflightCompressIfNeeded runs to completion BEFORE forward() is ever reached, and that it performs no upstream I/O for the main body — so a fail-fast verdict means the main payload was never sent upstream, and the only egress during preflight is the separate summarization call. Closes #7376
[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b) 查重: 不重复 — searched open+closed issues and PRs for the preflight fail-fast egress invariant topic; the only hit is this thread itself (closes #7376, the drive-by verification request from PR #7368's analysis of #7344). 标签: p3 + scope/docs(docs-only PR, no behavior change; no host lane involved). Verification (branch
|
What
Documents the preflight→forward egress invariant at the fail-fast decision point in
src/server/handle.ts. Comment-only; no behavior change.Context
#7376 (drive-by from PR #7368's analysis of #7344) asked for a code-level verification that the proxy never forwards the main payload to upstream BEFORE the preflight fail-fast decision is made.
Verification result (Option A — confirmed)
The main model request path is ordered so the preflight decision always completes before
forward():src/server/handle.ts:3572—runPreparedPipeline(prepare + preflight) runs under the session lock.src/server/handle.ts:3403—preflightCompressIfNeeded(...)fully decides.src/server/handle.ts:3425—if (isPreflightFailFast(outcome))→ respond locally (structured error) andreturn null(handle.ts:3468). No forward.src/server/handle.ts:3572→if (pendingForward)(i.e. NOT fail-fast) →handle.ts:3605await forward(...).So a fail-fast verdict ⇒
return null⇒pendingForwardis null ⇒forward()is never called. There is no forward-before-decision path for the main payload.The only upstream egress during preflight is the separate, smaller summarization call (external summary chain,
preflightCompressinsrc/preflight.ts), which is a different request — not the fail-fasted main payload.Therefore #7344's same-millisecond
400: requested 1667733 tokensmust be a different, larger request (matching PR #7368's assertion (b); the token counts support it: fail-fasted ~942,971–1,020,628 vs the 400 at 1,667,733). The "NOT forwarded" wording in the fail-fast message is always true for the main payload.Change
A comment-only addition at the fail-fast decision point (
src/server/handle.ts:3429-3439) recording the invariant. No behavior change.Closes #7376