Skip to content

[PR] fix: reject proxy CONNECT handshakes on premature close - #6577

Merged
ranxianglei merged 14 commits into
ranxianglei:masterfrom
Afloat16:2026-11-13_proxy-handshake-close
Nov 14, 2026
Merged

ranxianglei merged 14 commits into
ranxianglei:masterfrom
Afloat16:2026-11-13_proxy-handshake-close

Conversation

@Afloat16

Copy link
Copy Markdown
Contributor

Fixes #6576

Problem

A proxy can accept CONNECT and then close gracefully without a complete HTTP response. The socket emits end/close, but the handshake only listens for data, error and its deadline. The caller waits the full ten seconds after the proxy has already disconnected.

Changes

Reject on end/close while waiting for the CONNECT response. Release the deadline and terminal handlers when the handshake settles. Once a 200 response hands the socket to the tunnel owner, remove those handshake listeners so normal tunnel completion retains its existing behavior.

Validation

  • Four real TCP regressions cover EOF without headers, EOF during headers, successful bidirectional tunnel traffic and a 407 refusal.
  • Unchanged master e1f31ad: 2 failing EOF cases and 2 passing controls. Both failures wait roughly ten seconds for the old timeout.
  • Fixed branch: 12 passing tests including the existing direct, timeout and IPv6 CONNECT coverage. The new EOF cases complete in about 17 and 2 ms in the local run.
  • Root typecheck, complete build, unused-export gate, test-import-graph gate and git diff --check pass. Hosted CI and real-model E2E remain separate merge gates.

This applies to the shared proxy CONNECT transport. The existing timeout still bounds a live peer which stays silent. Previous Afloat16 submissions and all-state CONNECT searches were checked for overlap.

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[bot] (ai: vllm-qwen/qwen3.8-27b)

查重: 不重复 — open+closed 全量检索 CONNECT/handshake/close/timeout 无既有条目;与既有连接死线、upstream-body 超时改动属不同生命周期条件(握手期 EOF),无重叠。

标签: bug + scope/wiring + p2(代理上游 CONNECT 传输缺陷:死掉的代理拖慢调用方 10 秒,能用但有明显问题)。

审核结论: APPROVE(代码与测试成立;CI 红为继承,非本 PR 缺陷;待 ci-fixes 收敛 + 人工合并)

① 方向:根因定位正确——connectThroughProxy(src/upstream-proxy.ts)的握手 Promise 只监听 data/error/deadline;代理接受 CONNECT 后优雅 FIN 只发 end/close 无 error,无终局路径 → 干等 10 秒 CONNECT_TIMEOUT_MS。修在决策点(握手状态机补终局事件),不是在中途滤字节,疏不是堵。

② 回归清点(既有行为逐项):直连(独立函数,未动)/ 非法 proxyUrl(未动)/ TCP error(ECONNREFUSED 等,finishError 先摘 end/close 监听再 destroy,无重入)/ 静默活对端 10 秒死线(timer 逻辑未动,保留)/ 200 移交(摘掉握手 once 监听,隧道属主自己的 end/close/error 监听不受影响;settled 守卫本就使 resolve 后的 finishError 为 no-op,无可观测变化)/ 非 200 如 407(未动,测试钉住)/ 头超 64KiB(未动)/ IPv6 权威格式(未动)。高危模式逐项:无 fail-fast 变吞错(变更方向是失败更快);无阈值漂移(10 秒死线对静默活对端保留,PR 正文已披露);无 wire 形状变化(客户端上游传输,非 LLM wire)。

③ 代码:src 仅 7 行。closedBeforeHeaders 引用后声明的 finishError —— 闭包只在 executor 完成后的异步事件里被调,无 TDZ 风险;end+close 双发由 settled 守卫去重;成功路径同步摘监听再 resolve,不可能有事件插队。命名达意,风格与文件一致,无死代码。

④ 测试有效性:4 条真 TCP 回归,listen(0) 无固定端口,确定性。#3617 双向门:EOF 无头/EOF 半头 → 1 秒内必须 reject(旧代码 1 秒时仍 pending,旧分支实测 ~10.05s/10.01s);成功隧道双向流量 + 407 拒绝 = 负向对照(新监听不得误伤移交后语义)。finally { await result; } 排干基线 10 秒定时器,不留活句柄。断言有牙:回归测试在旧 master 红、修复分支绿,bug 回归时抓得住。

⑤ diff:2 文件 +75 行,无跑题文件、无锁文件连带、无 version 触碰。干净。

CI 红项归因(head 00ed16e,4 个 test 分片,已逐 job 拉日志核签名):


八问(中文)

  1. 是什么问题:代理接受 CONNECT 后干净断开(优雅 FIN、无 error)时,connectThroughProxy 不失败、空等满 10 秒握手超时才 reject——调用方被一个已经死掉的代理拖住 10 秒。
  2. 什么原因导致的:握手 Promise 的终局路径只有 data/error/deadline 三条;优雅 FIN 只触发 end/close 且无 error,三条都不命中,只能落到 10 秒死线。
  3. 影响范围与严重程度:配置了上游 HTTP 代理、且该代理行为异常(接受后关闭)的用户;请求最终仍正确失败,只是慢 10 秒。p2——可用但有明显问题,无数据丢失/损坏。
  4. 怎么修复的:把 end/close 补为握手终局事件——头未收完时立即 reject(错误信息带 host:port 与 "closed before response headers");任何 settle 路径清死线并摘终局监听;200 成功后摘掉握手自己的 end/close 监听,隧道属主的正常关闭语义不变。
  5. 是否有回归问题:无。既有行为清单(直连/非法 URL/TCP error/静默死线/200 移交/非 200/超大头/IPv6)逐项保持;10 秒死线对静默活对端保留(PR 正文已披露);无 fail-fast 变吞错、无阈值/时序/输出格式漂移。
  6. 是否破坏缓存:不涉及(传输层,未触碰缓存/重试/游标/去重键)。
  7. 是否改了设计方向:不涉及(单函数内补终局事件,不转向)。
  8. 是否引入了新的配置:不涉及。

判定:代码可合;CI 4 红全为继承/在档 flake(归因已发 #4599),等 ci-fixes 收敛后 merge 重跑,再交 owner 人工合并。Review: #6577

@ranxianglei

Copy link
Copy Markdown
Owner

(ai: vllm-qwen/qwen3.8-27b)

Review verdict: code is correct, complete and verified — no changes needed.

  • Baseline on master e1f31ad: both EOF regressions fail at ~10.0s (promise pending after proxy EOF); the tunnel-echo and 407 controls pass. On head 00ed16e: 4/4 pass in milliseconds; adjacent CONNECT/direct/IPv6/timeout + mitm e2e suites 33/33 green; root typecheck and full build clean.
  • The end/close handlers attached at socket creation (so a pre-connect FIN is caught too), deadline + terminal listeners released on every settle path, and removed before the socket is handed to the tunnel owner on 2xx — tunnel completion semantics unchanged.
  • Labeled: bug, p2, scope/wiring.

CI note: this run's red is an inherited master red, not this PR — master's own CI run 43469650189 fails with the identical signature (tests/notice-registry.test.ts visibleNotices cases vs the BILI_STORAGE_UNIFIED=legacy suite baseline; #6480 tests not hermetic, the #4011 lesson). Diagnosis posted to #4599; the fix converges on the ci-fixes sentinel lane. The 11 cancelled e2e lanes are the janitor fork-repo bug (#6221 closed unmerged). Once ci-fixes lands and CI re-runs green, this is ready for a human merge.

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[bot] (ai: vllm-qwen/qwen3.8-27b)

Status after your verdict — code needs no changes (both reviews agree); one blocker remains, and I just re-verified the branch state:

  1. ci-fixes has not landed the visibleNotices fix yet. origin/ci-fixes is still 0 ahead of master (head a007974683 = the [PR] storage: pure-SQLite default — JSON envelopes stop being written unless dual-write is requested (#6479) #6480 merge). The hermetic fix for the visibleNotices tests (pin/unset the storage env) requested at CI Control Center: Centralized CI Failure Handling & Status Sync (Comment all CI issues here; do not open new issues) | CI 总控: CI 失败集中处理与状态同步(所有 CI 问题在此评论, 勿开新 issue) #4599 floor 1307 is still pending with the sentinel. Merging ci-fixes into the PR branch right now would be a no-op, so the branch is untouched.
  2. The 36 cancelled check runs at 18:34Z on head 00ed16ebdd are the janitor fork-repo cancellation ([PR] fix(ci): preserve live fork PR runs during janitor cleanup #6221), consistent with your note. The 4 test-shard failures from the first run (all inherited: visibleNotices ×2 on 3 shards + Windows EBUSY unlink index.db flakes on 2 shards) remain the latest conclusion for those lanes.
  3. Plan: the moment ci-fixes carries the visibleNotices fix, I merge it into the PR branch and push — a fresh push triggers a clean run (not subject to the janitor cancelling the old run) — and re-verify before the human merge.

八问(中文)

  1. 是什么问题:本轮无新代码问题——是 CI/合并状态同步:PR 代码已双审通过,唯一阻塞是继承红(visibleNotices born-red)的修复尚未收敛进 ci-fixes。
  2. 什么原因导致的:[PR] storage: pure-SQLite default — JSON envelopes stop being written unless dual-write is requested (#6479) #6480(pure-SQLite default)落 master 后,visibleNotices 测试在 CI(storage=full)下 born-red;hermetic 修复仍待哨兵收敛进 ci-fixes(当前 0 ahead master,head a007974)。
  3. 影响范围与严重程度:仅阻塞本 PR 的 CI 绿灯与合并;不影响 master 代码与用户;p2 继承红,全 fleet 多个 PR 同签名(CI Control Center: Centralized CI Failure Handling & Status Sync (Comment all CI issues here; do not open new issues) | CI 总控: CI 失败集中处理与状态同步(所有 CI 问题在此评论, 勿开新 issue) #4599 floor 1305/1307/1310/1311)。
  4. 怎么修复的:本轮零代码改动;既定动作 = ci-fixes 收敛后立即 merge 进 PR 分支并 push,触发全新 run 重验。
  5. 是否有回归问题:不涉及(无代码改动)。
  6. 是否破坏缓存:不涉及。
  7. 是否改了设计方向:不涉及。
  8. 是否引入了新的配置:不涉及。

判定:PR 代码可合;等 ci-fixes 收敛 → merge 重跑 → 交 owner 人工合并。

The suite baseline preload (preload-storage-baseline.mjs) pins
BILI_STORAGE_UNIFIED="legacy" for the broad corpus. After ranxianglei#6480 flipped the
storage default to pure-sqlite, the notice-registry suite's visibleNotices
assertions expect the pure-sqlite default notice, which is gated on
unifiedSelectionIsSqlite (unified "sqlite" or absent) — the preload's
"legacy" made the notice silent, so the suite was born-red on master. Pin
the selection explicitly (ranxianglei#4011 lesson: a test that depends on the production
default passes locally but fails in CI).
ework-agent and others added 12 commits November 14, 2026 10:20
…test

The dsh plugin lane (ci-e2e-dsh.yml) runs node --import tsx --test
WITHOUT the preload-storage-baseline.mjs preload, so BILI_STORAGE_UNIFIED
is unset and the storage mode defaults to 'full'. Under the pure-sqlite
default flip (5a80e2b, ranxianglei#6480), 'full' mode turns the JSON envelope
writer OFF — so the .content-store.json sidecar is never written, and
the test's readdirSync(namespace) hits ENOENT.

Pin BILI_STORAGE_JSON_WRITER=on in the test so the content-store
sidecar is written regardless of the storage mode. This keeps the
corrupted-store detection path exercised in both full and legacy modes.

Closes: dsh plugin lane red on master (run 43532980164, job 132255459726)
The pure-sqlite default flip (5a80e2b, ranxianglei#6480) turns the JSON envelope
writer OFF, so the .json session file this lane asserts on is never
written and 'proxy must persist the session' / 'no persisted session
carried compressed blocks' fail after the 20s wait. Pin the writer ON in
the spawned client's env (the bootstrapped proxy inherits it) so the
envelope this test was written for is still produced.

Closes: native opencode lane red on master (run 43547593803, job 132302438610)
Same pure-sqlite default flip (5a80e2b, ranxianglei#6480) as the opencode lane:
the JSON envelope writer is OFF, so 'proxy must persist the session
within 20000ms' fails — the on-disk .json session file is never written.
Pin the writer ON in the spawned dsh client's env (the bootstrapped
proxy inherits it).

Closes: native dsh lane red on master (run 43547497015, job 132302138843)
…e2e lane

Same pure-sqlite default flip (5a80e2b, ranxianglei#6480) as the opencode/dsh
lanes: the JSON envelope writer is OFF, so 'session persisted' fails —
the on-disk .json session file is never written. Pin the writer ON in
the spawned pi client's env (the bootstrapped proxy inherits it).

Closes: pi codex-responses WS lane red on master (run 43547626291, job 132302540960)
…lane

4th lane of the pure-sqlite default flip (5a80e2b, ranxianglei#6480) family: the
JSON envelope writer is OFF, so 'proxy must persist the session' and
'at least one persisted session must carry compressed blocks' fail — the
on-disk .json session file is never written. Pin the writer ON in the
spawned pi client's env (the bootstrapped proxy inherits it).

Closes: pi package-native lane red on ci-fixes (run 43558179054, job 132336647955)
Same pure-sqlite default flip (5a80e2b, ranxianglei#6480) family: the JSON writer
is OFF, so the per-session CCR content-store sidecar (.content-store.json)
that 'child persists its own companion and survives a proxy restart'
asserts on is never written (0 !== 1). The earlier b1bc17b pin only
covered tests/plugin-fork.test.ts, not this file. Pin the writer ON at
module load (before any SessionStore is constructed).

Closes: dsh plugin lane red on ci-fixes (run 43558179102, job 132336519539)
…lane

Pure-sqlite default flip (5a80e2b, ranxianglei#6480) family: the JSON envelope
writer is OFF, so 'proxy must persist the session' fails — the on-disk
.json session file is never written. Pin the writer ON in the spawned
opencode client's env (the bootstrapped proxy inherits it).
Pure-sqlite default flip (5a80e2b, ranxianglei#6480) family: the JSON envelope
writer is OFF, so the on-disk .json session envelope this lane asserts on
(exactly ONE persisted session under the raw key) is never written. Pin
the writer ON in the spawned dsh client's env.
Pure-sqlite default flip (5a80e2b, ranxianglei#6480) family: the JSON envelope
writer is OFF, so the on-disk .json session file this lane asserts on
(state survival across the OLD→NEW update) is never written. Pin the
writer ON in the spawned proxy's env (both update phases share envBase).
@ranxianglei
ranxianglei merged commit f3a9825 into ranxianglei:master Nov 14, 2026
50 checks passed
@ranxianglei

Copy link
Copy Markdown
Owner

Superseded by #6677 (merged as 3858b6956, 2026-11-14).

Landed into master via the real merge commit f3a982507af (Merge Afloat16/2026-11-13_proxy-handshake-close (PR #6577)), so your original commits are preserved verbatim in master history — only hashes of the batch wrapper changed, your authorship and commit messages are intact.

The fix for #6576 is now in master. Closing this PR as superseded — thanks for the careful, well-tested fix.

@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[bot] (ai: vllm-qwen/qwen3.8-27b)

Confirmed — verified on master:


八问(中文)

  1. 是什么问题:无——本轮为收尾确认:Proxy CONNECT waits ten seconds after the proxy has already closed #6576 的修复已经由 PR [PR] chore(branch): land 40 single-purpose fixes as one history-preserving batch (wave 5) #6677(batch wrapper)合入 master,本 PR 按 owner 决定以 superseded 关闭。
  2. 什么原因导致的:不涉及(修复已交付;原 bug 根因 = CONNECT 握手 Promise 缺 end/close 监听,修复已在 master 验证)。
  3. 影响范围与严重程度:已消除——代理提前关闭时 CONNECT 立即报错而非等满 10 秒,修复已在 master。
  4. 怎么修复的:已在 master(closedBeforeHeaders 守卫,src/upstream-proxy.ts:493-499);本轮零改动。
  5. 是否有回归问题:无——回归盘点(直连/无效 proxyUrl/TCP 错误/10s 静默对端/200 移交/407/>64KiB 头/IPv6)全保持,双审确认。
  6. 是否破坏缓存:不涉及。
  7. 是否改了设计方向:不涉及。
  8. 是否引入了新的配置:不涉及。

判定:线程收工——修复在 master,PR 由 owner 关闭,无遗留动作。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working p2 scope/wiring

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Proxy CONNECT waits ten seconds after the proxy has already closed

2 participants