Repository navigation
[PR] fix: make upstream TLS trust failures self-diagnosable; add network.caFile (#3776) - #3786
Conversation
…aFile (#3776) Security software doing HTTPS scanning (Kaspersky, enterprise DLP) or a corporate MITM proxy re-signs the upstream with its own CA. Browsers trust it via the OS certificate store, but bili's own Node fetch reads only its built-in roots plus NODE_EXTRA_CA_CERTS, so these users hit SELF_SIGNED_CERT_IN_CHAIN, the upstream circuit opens after 3 pre-response failures, and the client only ever sees a generic 502 (#3773/#3776). - Trust-code TLS failures carry a concrete remediation hint on every channel rendering formatUpstreamError (log, 502 detail, circuit/alert detail) plus the panel tls alert banner (en+zh): export the scanner/vendor root to PEM -> NODE_EXTRA_CA_CERTS or network.caFile, or exclude node.exe from scanning. The trust-code set and remediation text live once in src/upstream-fail.ts; preflight reuses them. - New network.caFile / BILI_CA_FILE knob: a PEM file whose roots are MERGED into bili's own upstream fetch trust (direct + proxied dispatchers), never replacing built-ins or NODE_EXTRA_CA_CERTS. Unset = byte-identical behavior; dispatcher caches key on CA identity so edits apply on rotation. - The dispatch-catch 502 body gains an additive upstreamFailure{kind,code,hint} field when the failure is structured (upstream request failed / circuit open), so clients can self-diagnose without reading bili.log. Absent for everything else. Docs: CONFIGURATION.md en+zh. Tests: new hermetic suite (unit + in-process proxy e2e: untrusted MITM upstream => structured 502; same upstream trusted via caFile => 200); preflight transport-recovery assertion updated to the shared remediation.
📦 Built Package ArtifactBranch: Option A — Install this exact build (recommended)npm install -g billion-context@0.0.4497-pr.3786.a4f49e8This exact version is immutable and reproducible. The rolling Option B — Download artifact
tar xzf billion-context-pr3786.tgz
npm install -g packageInstall instructions are refreshed on each push. |
(ai: vllm-qwen/qwen3.8-27b) 查重:不重复。 按失败签名(SELF_SIGNED_CERT_IN_CHAIN / network.caFile / BILI_CA_FILE / circuit 502 结构化出口)检索 open+closed issue 与 PR:唯一相关链条是来源 #3773(已关闭,用户自定位根因)→ 跟踪单 #3776(本 PR 关闭它),无其他在途同机制修复。 标签: bug、dsh、scope/wiring、p2(缺陷类;复现客户端 dsh-desktop;改动在代理上游出口/协议接线层;条件触发需 MITM 软件在环,与 #3776 定级一致)。 审核进行中,先报一个已本地复现的硬伤(完整八问结论随后): CI 必红 ×3 —— 新配置键没进 YAML seed。 本仓配置面纪律:
修法明确:补 |
…-coverage/config-surface-guard) (#3776) Review fix for PR #3786: the new knob was documented by hand-editing the GENERATED artifacts (CONFIGURATION.md / zh-CN) while the single source of truth website/config-reference/*.yaml stayed untouched, which trips all three config-doc CI gates (docs-sync strips the hand-added rows on regeneration; docs-coverage flags BILI_CA_FILE undocumented; config-surface-guard flags network.caFile without a seed entry). Add the seed entry and regenerate; the committed table rows now match the generator output byte-for-byte.
…n merge + circuit-kind coverage (#3776) Review fixes for PR #3786: - upstream-ca.ts: the merged trust list re-parses NODE_EXTRA_CA_CERTS itself (an explicit undici connect.ca replaces the default store, so every source Node would already have trusted must be folded in). It split on ':' only, which on Windows shreds drive-letter paths (C:\certs\ca.pem -> "C" + "\certs\ca.pem") and silently drops those certs with a warn log. Mirror Node's own win32 separator handling (';' also separates there). - tests: pin the previously uncovered corners of sub-defects 2 and 3 — NODE_EXTRA_CA_CERTS entries land in the merged list, and upstreamCircuitLastFailureKind reports the opener's kind only while the circuit is open (the kind source of the circuit-open egress hint).
Review fix for PR #3786: the interface is only consumed inside src/upstream-proxy.ts as the return type of describeUpstreamFailure — no other module imports it, so the unused-exports gate fails the PR head. Drop the leading export (INTERNAL_ONLY); callers keep the shape via structural inference, no behavior or typing change.
(ai: vllm-qwen/qwen3.8-27b) 进度汇报(上一轮长轮询 CI 被看门狗重启,现在恢复):审核已完成,发现的小问题已按「小问题直接修」在 PR 分支上补了 3 个独立标注 commit(均已 push,head
另外给 PR 描述补了缺失的 CI 现状(head |
…A merge (#3776) Review fix for PR #3786 (replaces the separator heuristic from 93288dc, which this commit supersedes): Node does NOT split NODE_EXTRA_CA_CERTS on any separator — verified against Node 22, the entire value is loaded as ONE file (multi-cert files carry several PEM blocks). The previous ':' / win32 ';:' splits diverged from Node on every platform and shredded drive-letter paths on Windows (which also broke the env-extra test there). Read the value as a single path so the merge trusts exactly what Node would already have trusted; pin the no-split semantics with an extra assertion.
(ai: vllm-qwen/qwen3.8-27b) 独立复核完成,判定:可合并。 CI 在 head 结构化八问
审核发现及处置(均为标注 commit,head
观察(不拦,记录在册):① proxy lane( 行为变更(作者已披露,复核确认):trust-code TLS 错误追加修复指引后缀;preflight caNote 去掉 判定:可合并。 head |
📦 Merged Build PublishedThis merge to master was published as npm install -g billion-context@0.0.621-master.7446891 # this exact build
npm install -g billion-context@master # always the newest merged build
|
Closes #3776
What & why
Security software doing HTTPS scanning (Kaspersky "encrypted connection scanning", enterprise DLP) or a corporate MITM proxy re-signs the upstream with its own CA. Browsers trust it via the OS certificate store; Node's fetch does not read the OS store — only its built-in roots plus
NODE_EXTRA_CA_CERTS. So bili's own upstream fetch fails withSELF_SIGNED_CERT_IN_CHAIN, the upstream circuit opens after 3 pre-response failures, and the client only ever sees a generic 502 (reported case: #3773 — Windows + dsh-desktop + Kaspersky; the user spent ~a day diagnosing from nothing visible client-side).One mechanism family (MITM trust + error transparency), three sub-defects, one PR:
SELF_SIGNED_CERT_IN_CHAIN,UNABLE_TO_GET_ISSUER_CERT_LOCALLY) and the remediation text now live once insrc/upstream-fail.ts.formatUpstreamErrorappends the remediation for trust-code TLS failures, so the log line, the 502detail, and the circuit/alert detail all carry it; preflight's summary-transport note reuses the same helpers (its local code set was deleted); the panel's tls alert banner (en+zh) names the scenario too.network.caFile/BILI_CA_FILE— a PEM file whose roots are merged into bili's OWN upstream fetch trust (both the direct dispatcher infetch-util.tsand the proxiedProxyAgentinupstream-proxy.ts). Explicit undiciconnect.caREPLACES default trust, so the implementation merges built-in roots +NODE_EXTRA_CA_CERTS+ the file (strictly additive over today's default); dispatcher caches key on a TTL-cached CA identity (path+mtime) so file edits rotate within ~30 s. Unset ⇒ dispatchers omitconnectentirely ⇒ byte-identical behavior.Audit finding from the issue (sub-defect 2):
combined-ca.pem(fix(ca): merge OS trust store into combined-ca.pem (#1807) #1808 OS-store merge) is consumed ONLY by launcher lanes (child-client env); bili's own process never points at it — by design, since bili also serves non-launcher installs. That is exactly why the Kaspersky root sitting in the Windows store was untrusted here. Auto-trusting the OS store inside bili's process would be a default trust-surface change (owner decision), deliberately NOT done; this knob is the explicit escape hatch.{kind, code, hint}to the error (non-enumerable symbol); the dispatch catch adds an additiveupstreamFailurefield to the 502 JSON body when present. Clients can now self-diagnose (kind=tls+ remediation hint) without reading bili.log. Body is byte-identical for every failure that isn't structured.config surface
network.caFile(string, default unset) ↔ envBILI_CA_FILE(env wins). Documented in CONFIGURATION.md en+zh (network table row, env mapping row, env detail entry, network-block prose).NODE_EXTRA_CA_CERTSis a process-global environment variable — users on host-managed installs (dsh profile bundles, opencode/pi trees) often cannot control the proxy process's environment at all, and there was no per-install file knob. Considered and rejected: (a) auto-merging the OS store like fix(ca): merge OS trust store into combined-ca.pem (#1807) #1808 — silently expands the trust surface of every install (product/security decision, not an agent call); (b) reusingcombined-ca.pem— launcher-lane artifact by design (dsh plugin-market installs never auto-update: the refresh chain's only trigger is a global bili self-update #1196 contract), unavailable/irrelevant for market-only installs.connect, Node default trust untouched).Behavior changes (old → new)
— CA/proxy MITM config→ same prefix plus appended remediation:…; the presented certificate chain is not trusted by this Node.js process — usually security software doing HTTPS/encrypted-connection scanning (Kaspersky, enterprise DLP) or a corporate MITM proxy re-signing the upstream. Fix: export that scanner/vendor root CA to a PEM file and point NODE_EXTRA_CA_CERTS=/path/to/ca.pem at it (or set bili config "network.caFile"), or exclude node.exe from the software's HTTPS inspection. Non-trust TLS failures (e.g. EPROTO) keep the old hint unchanged.This looks like TLS certificate interception (corporate proxy/MITM): install the corporate root CA for Node.js via NODE_EXTRA_CA_CERTS=/path/to/ca.pem, or run node with --use-system-ca.→ new—+ the shared remediation above. Dropped the--use-system-casuggestion (flag-based, unusable for a running daemon) in favor of the file-based fixes.upstreamFailure: {kind, code?, hint}on structured upstream failures (fetch failure / circuit open); absent otherwise — existing consumers see unchanged bodies.Tests
New hermetic suite
tests/issue3776-tls-mitm-diagnostics.test.ts: unit (hint present for trust codes / absent for EPROTO; structured attachment round-trip + non-enumerability; CA merge never replaces; unset ⇒ undefined) + in-process proxy e2e with a real self-signed MITM https upstream: untrusted ⇒ 502 carryingupstreamFailure{kind:"tls", code, hint}; the same upstream trusted viaBILI_CA_FILE⇒ 200 completion.tests/preflight-transport-recovery.test.tsassertion updated to the shared remediation text (the old pinned note string no longer exists by design).Rebased onto current master (original base was 80 commits behind; conflicts in relay.ts/config.ts/docs were union-of-intent resolutions with #3701's WS byte-shed work — the structured-failure change lands in the non-oversize branch only, the ws-shed path stays verbatim). Local on the rebased head: typecheck clean, build green (both dist gates), 95/95 across the 10 affected suites incl.
issue3701-ws-byte-budget. Full-suite verdict: CI.