Skip to content

[PR] docs: document the preflight fail-fast egress invariant (#7376) - #7385

Open
ranxianglei wants to merge 1 commit into
masterfrom
2026-11-20_preflight-failfast-egress-doc
Open

ranxianglei wants to merge 1 commit into
masterfrom
2026-11-20_preflight-failfast-egress-doc

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

What

Documents the preflight→forward egress invariant at the fail-fast decision point in src/server/handle.ts. Comment-only; no behavior change.

Context

#7376 (drive-by from PR #7368's analysis of #7344) asked for a code-level verification that the proxy never forwards the main payload to upstream BEFORE the preflight fail-fast decision is made.

Verification result (Option A — confirmed)

The main model request path is ordered so the preflight decision always completes before forward():

  1. src/server/handle.ts:3572 — runPreparedPipeline (prepare + preflight) runs under the session lock.
  2. Inside it, src/server/handle.ts:3403 — preflightCompressIfNeeded(...) fully decides.
  3. src/server/handle.ts:3425 — if (isPreflightFailFast(outcome)) → respond locally (structured error) and return null (handle.ts:3468). No forward.
  4. src/server/handle.ts:3572 → if (pendingForward) (i.e. NOT fail-fast) → handle.ts:3605 await forward(...).

So a fail-fast verdict ⇒ return null ⇒ pendingForward is null ⇒ forward() is never called. There is no forward-before-decision path for the main payload.

The only upstream egress during preflight is the separate, smaller summarization call (external summary chain, preflightCompress in src/preflight.ts), which is a different request — not the fail-fasted main payload.

Therefore #7344's same-millisecond 400: requested 1667733 tokens must be a different, larger request (matching PR #7368's assertion (b); the token counts support it: fail-fasted ~942,971–1,020,628 vs the 400 at 1,667,733). The "NOT forwarded" wording in the fail-fast message is always true for the main payload.

Change

A comment-only addition at the fail-fast decision point (src/server/handle.ts:3429-3439) recording the invariant. No behavior change.

Closes #7376

Record at the fail-fast decision point (the only egress gate for the main
payload) that preflightCompressIfNeeded runs to completion BEFORE forward()
is ever reached, and that it performs no upstream I/O for the main body —
so a fail-fast verdict means the main payload was never sent upstream, and
the only egress during preflight is the separate summarization call.

Closes #7376
@ranxianglei ranxianglei added documentation Improvements or additions to documentation p3 scope/docs root-cause labels Nov 20, 2026
@ranxianglei ranxianglei changed the title docs: document the preflight fail-fast egress invariant (#7376) [PR] docs: document the preflight fail-fast egress invariant (#7376) Nov 20, 2026
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

查重: 不重复 — searched open+closed issues and PRs for the preflight fail-fast egress invariant topic; the only hit is this thread itself (closes #7376, the drive-by verification request from PR #7368's analysis of #7344).

标签: p3 + scope/docs(docs-only PR, no behavior change; no host lane involved).

Verification (branch 2026-11-20_preflight-failfast-egress-doc, head 933dd2c)

Single commit, comment-only: 11 lines added at src/server/handle.ts:3429-3439, zero behavior change.

All four flow claims in the PR body verified against the code:

  1. src/server/handle.ts:3572 — withSessionLock(session, () => runPreparedPipeline!(true)) ✓ (the prepare+preflight pipeline runs under the session lock)
  2. src/server/handle.ts:3403 — preflightCompressIfNeeded(...) fully decides ✓
  3. src/server/handle.ts:3425 — if (isPreflightFailFast(outcome)) → local structured response → return null ✓ (at handle.ts:3479 on the branch; the body's :3468 is the pre-insertion line number — cosmetic, no action needed)
  4. src/server/handle.ts:3573 — if (pendingForward) → handle.ts:3605 await forward(...) ✓

Egress claim verified: src/preflight.ts contains no fetch/request for the main body; the only preflight egress is the external summary chain (deps.externalSummary.summarize(...), src/preflight.ts:1355) — a separate, smaller request. The dead-probe canary (#2832) runs through that same summary executor (its local probe is CPU-only, deps.core.processTurn at src/preflight.ts:1630).

Also audited every other forward() call site in src/server/handle.ts (616/686/735/762/862/1762/2741/2864/3558/3646): all are early passthrough / non-conversation shapes, side requests that skip preflight by design (#388), the forged auto-mode classifier one-shots (#2740), or the signed-passthrough escape hatch (#1884) — none is the main payload subject to fail-fast. The invariant "fail-fast ⇒ return null ⇒ pendingForward null ⇒ forward() never called" holds structurally; there is no forward-before-decision path for the main payload.

Master freshness: the branch is 7 commits behind master, but none of those commits touch src/server/handle.ts, src/preflight.ts, or src/server.ts (the #7173 merge touched launcher/storage/persist/docs only) — the comment's claims remain valid on current master. GitHub reports mergeable: true.

CI (head 933dd2c): all test lanes green (60 success / 31 skipped; the 80 cancelled entries are superseded runs from the 09:08/09:11 re-pushes). The single red is build-artifact — npm publish 404. That is the systemic NPM_TOKEN issue already escalated (the pr dist-tag has been stuck at 0.0.11045-pr.7336.3b3c789 since 06:02Z; same-repo PR #7368 failed identically at 07:28Z; see #4599 comment 09:26Z and PR #7352's decision comment). Unrelated to this diff — a comment-only change cannot affect npm publish; it needs the owner's NPM_TOKEN action, after which a build-artifact rerun goes green.

八问(review 结论)

  1. 是什么问题: 文档 PR —— 把 [Drive-by] Verify preflight fail-fast has no forward-before-decision path (#7344 in-flight concern) #7376 要求的代码级验证结论(preflight fail-fast 决策先于主 payload 转发上游)固化成决策点处的注释。
  2. 什么原因导致的: 无缺陷 —— [Drive-by] Verify preflight fail-fast has no forward-before-decision path (#7344 in-flight concern) #7376 是 After fail-fast, the payload in the same round was still sent upstream. | fail-fast 之后,同一轮的 payload 仍然被发到了上游 #7344 分析中派生的验证请求(fail-fast 消息里 "NOT forwarded" 的措辞需要代码级背书)。
  3. 影响范围与严重程度: 纯注释、零行为变更,无用户面影响。
  4. 怎么修复的: 在 fail-fast 决策点加 11 行注释记录不变量:锁内完成 preflight 决策 → fail-fast 返回 null → 仅 pendingForward 非空才 forward;preflight 期间唯一出网是独立的摘要调用(非主 payload)。
  5. 是否有回归问题: 无 —— diff 纯注释,既有行为全部保持;已核对 master 领先的 7 个 commit 均未触及 handle.ts/preflight.ts/server.ts;另逐点审计了 handle.ts 全部 11 个 forward() 调用点,确认其余路径均为旁路形状(透传/侧请求/伪造裁决/签名逃生舱),不受该不变量约束。
  6. 是否破坏缓存: 不涉及。
  7. 是否改了设计方向: 不涉及。
  8. 是否引入了新的配置: 不涉及。

Verdict: ready for human merge. Code verified, invariant holds, CI green except the systemic build-artifact red (already escalated, owner's NPM_TOKEN).

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation p3 root-cause scope/docs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Drive-by] Verify preflight fail-fast has no forward-before-decision path (#7344 in-flight concern)

1 participant