Skip to content

[PR] fix: retire upstream dispatchers when CA identity changes - #7349

Open
Afloat16 wants to merge 1 commit into
ranxianglei:masterfrom
Afloat16:2026-11-20_retire-rotated-ca-dispatchers
Open

Afloat16 wants to merge 1 commit into
ranxianglei:masterfrom
Afloat16:2026-11-20_retire-rotated-ca-dispatchers

Conversation

@Afloat16

Copy link
Copy Markdown
Contributor

Problem

Both outbound dispatcher caches key entries by the configured CA identity but never remove older identities. Repeated CA changes retain obsolete direct Agents and ProxyAgents, their trust material and origin pools for the process lifetime.

Changes

When the effective CA identity changes, gracefully close and clear the prior identity's cached dispatchers. Both direct and proxied paths continue reusing the current identity across their existing timeout/route variants. The existing close() cleanup waits for in-flight responses; it does not destroy their streams.

The trust construction, CA identity refresh policy, default timeouts, wire payload and configuration surface are unchanged.

Verification

  • Tested base: 1e357bf2d6632bfbd2a74ffb70ecee1af1ec7cc3 (current master, 0.1.216).
  • Before the fix: 4 regression failures, each showing that an obsolete dispatcher remains open.
  • After the fix: 30 targeted tests pass, including direct/proxy cache reuse and retirement, returning to a prior identity, removing the configured CA, real HTTP and CONNECT responses finishing during rotation, existing TLS trust and wrong-host/untrusted-certificate rejection, and upstream timeout coverage.
  • npm run typecheck, npm run build, npm run check:unused-exports, npm run check:test-import-graph, and git diff --check: passed.
  • Node 24.22.3, Linux x86_64. No live host-client or Windows run. The full-suite verdict belongs to repository CI on this head.
  • Published fork tree c2ccc1e28f69bd345876a1fa0257493b677ee7a0 exactly matches the local validated tree.

Related #3786, #6096 and #7004 handle CA configuration, identity refresh and trust merging. This addresses the remaining cache lifecycle defect in both transport paths as one change.

Fixes #7348

@ranxianglei ranxianglei changed the title fix: retire upstream dispatchers when CA identity changes [PR] fix: retire upstream dispatchers when CA identity changes Nov 20, 2026
@ranxianglei

Copy link
Copy Markdown
Owner

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷

PR #7349 review — retire upstream dispatchers on CA identity change

I reviewed this independently (re-derived the bug, ran the suites, read the diff — not from the issue's conclusions).

The bug is real and reproduced. On the PR base (1e357bf2d66), all 4 new regression tests fail with retired … Agent must release its origin pools and CA material — expected true, actual false: the obsolete dispatchers stay open. With the fix, all 4 pass.

The fix is correct and complete. Both dispatcher caches (directDispatchers in src/fetch-util.ts, dispatcherCache in src/upstream-proxy.ts) are keyed by CA identity but never remove older identities. The fix adds a module-level "current CA identity" sentinel to each; on identity change it gracefully closes and clears the prior identity's cached dispatchers (close() waits for in-flight responses, does not destroy their streams), then builds fresh for the new identity. I confirmed these are the only two Agent/ProxyAgent caches keyed by CA identity — no third path was missed.

No regressions. Audited the existing behaviors the touched paths carry:

  • Reuse across timeout/proxy-route variants for an unchanged identity: preserved (key still includes timeout/route; tests assert reuse).
  • In-flight responses finish during rotation: preserved (close() is graceful; test asserts destroyed === false until the last response completes, then true).
  • Returning to a prior identity: now correctly builds a fresh agent (no resurrection of a retired one).
  • Removing the configured CA: correctly falls back to Node's default trust (identity none, no connect.ca).
  • No fail-fast→swallow, no threshold/timing/output-format drift.
  • One boundary (not a regression): retirement is lazy — it runs on the next request for a given path. If the CA rotates but a path (direct or proxy) gets no further request, that path's old dispatchers stay until the next request for it. This matches the dispatcher caches' existing lazy-evaluation design; the idle agents are bounded by keep-alive timeout.

Tests have teeth. They read the real undici closed/destroyed getters via Reflect.get (not fabricated state), use real rootCertificates PEM files, real HTTP + CONNECT fixtures, and assert the exact lifecycle. They reproduce the bug on the base and pass on the head.

Verification I ran locally (targeted, per repo discipline):

  • New retirement suite: 4/4 pass on head; 4/4 fail on base (pre-fix).
  • Related existing suites (upstream-ca-default-trust, upstream-proxy-ca-trust, fix-551-upstream-timeout, upstream-timeout-diagnostics): 29/29 pass.
  • npm run typecheck, npm run check:unused-exports, npm run check:test-import-graph, git diff --check: all clean.
  • CI on the head (c11a7ef): green — 84 check runs, 57 success / 27 skipped / 0 failed (verified live; the mirror cache was stale at the time of first look).

Diff is clean (2 src files +15/-2 each, 1 new test file +192; no unrelated changes, no whitespace churn). Code follows the file's existing patterns — the close logic is extracted into close*Dispatchers and reused by both the rotation path and the existing test-reset helpers. No config surface, wire payload, persistence, or kernel changes.

Verdict: ready to merge (CI green, bug reproduced + fixed, no regressions). Merge is human-only per repo rules.


结构化八问(供人工 reviewer):

  1. 是什么问题:CA 身份(network.caFile/BILI_CA_FILE)变化后,旧的 direct/proxy 出站 dispatcher 缓存(undici Agent/ProxyAgent,连同其信任材料与 origin 池)永不被清除,进程生命周期内持续累积。
  2. 什么原因导致的:两个 dispatcher 缓存(src/fetch-util.ts 的 directDispatchers、src/upstream-proxy.ts 的 dispatcherCache)按 CA 身份做缓存 key,但只增不删——身份变化时只为新身份新建 Agent,旧身份的 Agent 一直留在 Map 里直到进程退出。根因是缓存缺"身份变化即退役"的生命周期钩子,而非某个具体请求路径的 bug。
  3. 影响范围与严重程度:触发条件 = 反复变更 CA 文件路径或原地轮换该文件。每次轮换泄漏一组 Agent + CA 信任材料 + origin 池。按 owner 2026-10-19 缓存泄露规则定 p1;实际触发面较窄(需用户主动反复轮换 CA),但一旦触发是进程级持续泄漏。
  4. 怎么修复的:在两个 dispatcher 取用入口各加一个模块级"当前 CA 身份"哨兵;身份变化时先优雅关闭并清空旧身份全部缓存(close() 等待在途响应完成、不销毁流),再为新身份建缓存。把既有 _resetFetchUtilForTest/resetProxyCache 的清理逻辑抽成 close*Dispatchers 复用,避免重复。
  5. 是否有回归问题:逐项审过既有行为——同身份跨 timeout/proxy-route 变体复用(保持)、在途响应不被中断(保持,close() 优雅等待)、回到旧身份不复活旧 Agent(新增正确行为)、移除 CA 回落 Node 默认信任(正确)、无 fail-fast 变吞错、无阈值/时序/输出格式漂移。一个非回归边界:退役是惰性的(该路径下一次请求时才执行),与既有 dispatcher 缓存的惰性求值设计一致。
  6. 是否破坏缓存:不涉及前缀缓存/重试/增量游标/去重键——dispatcher 是传输层 Agent,不是上下文缓存。
  7. 是否改了设计方向:无架构转向;只给既有的身份键控缓存补上生命周期钩子。
  8. 是否引入了新的配置:无。BILI_CA_FILE/network.caFile 语义不变,无新 env/config/默认值。

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working p1 root-cause

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] CA rotation retains obsolete direct and proxy dispatchers

2 participants