Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions LIFECYCLE.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@
| `BILI_OPENCODE_INLINE` | #3278 opencode 移植(#4687) | opencode inline 模式(需手动开启,默认保持旧行为 spawn,owner 决定不改默认):与 pi 同构——不起 proxy 进程,模型 URL 被 V2 route/intercept 改写成伪 origin 后由进程内引擎接住(bun 宿主已验:引擎 boot/dispatch 全绿);`=1/on/yes/TRUE` 或 file 家 `"opencode": {"inline": true}` 显式开启(显式开启压过 attach 目标);`bili opencode-inline` 一键车道 | 同 pi inline 车道节奏 | opt-in(不设置时 attach/spawn 行为与旧版完全一致) |
| `BILI_INLINE_DEBUG` | #4669 | 仅 inline 车道:引擎 stderr 日志腿默认静默(stderr 是 pi 的交互终端,日志会挤掉输入框;默认只写 bili.log),设 1 恢复 stderr 输出用于排查 | 随 inline 车道(#3278)走 | 排查逃生口 |
| 测试钩子:`BILI_DSH_RECOVERY_INTERVAL_MS` / `BILI_DSH_RETRY_INTERVAL_MS` / `BILI_MODEL_INFO_RETRY_MS` | #2082 等 | 缩短测试/运维节奏(dsh-native.ts;NODE_TEST_CONTEXT 感知,避免生产误缩) | 永久测试设施,随宿主代码走 | 测试钩子 |
| `BILI_SESSION_CONTENT_STORE_PIN` | #4788 phase 2(c8d7a802) | idle-fold 逃生口:`=1/true` 让每会话 CCR content store({byRef}/{byHash} 原文)常驻内存、save 落盘后不折叠(默认折叠;读点 src/knobs.ts:451,store 可从 unified ccr mirror 或 .content-store.json sidecar 重新物化) | #4788 phase 2 idle-fold 生产站稳后评估退役(owner 拍板) | 在用(默认 fold) |

## 2. 常设面全量登记表(src/ 实际读取的其余 BILI_* 环境变量)

Expand All @@ -78,6 +79,7 @@
| `BILI_NON_HTTP_PROVIDERS` | #1392 非 http(s) baseUrl provider 白名单传递 |
| `BILI_PROVIDER_REWRITES` | #535 provider 改写 manifest JSON(存在即关 zcode native) |
| `BILI_UPGRADE_REPLACEMENT` | #3044 self-restart 替换进程交接标记(UPGRADE_REPLACEMENT_ENV) |
| `BILI_SPAWN_FINGERPRINT` | #4125 spawner 侧入口脚本 sha256 指纹(SPAWN_FINGERPRINT_ENV,src/instance.ts:343),spawn 时注入供子进程记录自身指纹,避免 auto-update 窗口内 re-hash 漂移 |

### 2.3 网络调优(#2030 迁移家族:env > 文件 network.* > 默认)

Expand Down
8 changes: 8 additions & 0 deletions src/acp-status.ts
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ import { forkAdoptionSkippedOf } from "./fork-adoption.js";
import { preCompactionArchiveOf, statusInputBaseline, type Session } from "./session.js";
import { compressBreakerDetail, compressLastFailureCause } from "./stream.js";
import { describeAdvisory, getAdvisoryState } from "./advisory.js";
import { getResolverWedgeState } from "./resolver-wedge.js";
import { getUpdateVisibility, updateBannerPayload } from "./update-notes.js";
import { loadOptions } from "./config.js";
import { pckNoticeActive } from "./pck-notice.js";
Expand Down Expand Up @@ -491,6 +492,13 @@ export function handleAcpStatus(args: Record<string, unknown>, ctx: AcpStatusCtx
extra.push("");
extra.push(`CRITICAL ADVISORY (instance-level): bili is auto-updating through the self-updater's safety chain — ${describeAdvisory(adv.active, adv.lastError)}. Live state: the advisory field of GET /__bili/status.`);
}
// #5221 phase 2 (owner ruling 2026-10-31): suspected persistent local-resolver
// wedge — detection-only alarm surface, same lane as the advisory above.
const rw = getResolverWedgeState();
if (rw) {
extra.push("");
extra.push(`RESOLVER WEDGE SUSPECTED (instance-level): this process's OS name resolution keeps failing for ${rw.hosts.length} host(s) that resolved fine before (${rw.hosts.slice(0, 5).join(", ")}${rw.hosts.length > 5 ? ", …" : ""}) — model requests 502 until the proxy process restarts or egress routes through an upstream proxy (-F/--forward-proxy). Live state: the resolverWedge field of GET /__bili/status.`);
}
const upd = getUpdateVisibility(VERSION);
const updateAction = updateBannerPayload(upd);
if (updateAction) {
Expand Down
98 changes: 98 additions & 0 deletions src/resolver-wedge.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
/**
* #5221 (Phase 2, owner ruling 2026-10-31): persistent local-resolver WEDGE
* detector + alarm surface. Detection only — the process NEVER restarts or
* retries differently on its own; the ruling was "if it heals, it heals
* without a restart; a restart at most earns an ALARM".
*
* Mechanism (#5221 evidence): a long-lived process can hold stale OS-resolver
* state after a WSL/VPN/MagicDNS change — glibc caches resolv.conf and libuv
* getaddrinfo ages out — so every tunnel admission's name lookup times out
* while a freshly spawned process on the same machine resolves fine. The
* wedge is per-PROCESS: it is proven by "this process resolved these hosts
* successfully before and has now failed every attempt for minutes straight".
*
* Evidence gates (all required, no config surface by design):
* 1. only THROWING resolutions count as failures — an empty answer is an
* authoritative NO-DATA response, i.e. the resolver IS reachable; a dead
* name must never look like resolver corruption (#1686 boundary);
* 2. a host qualifies only if this process saw it resolve OK at least once
* (a mistyped upstream that never resolved does not qualify);
* 3. the failure streak must run continuously for >= WEDGE_SUSTAIN_MS;
* 4. at least WEDGE_MIN_HOSTS distinct hosts must be wedged at once — one
* poisoned destination is configuration noise, two never-green-before
* hosts failing in lockstep after a network flip is the wedge signature.
*
* Recovery is derived, not latched: any successful resolution clears that
* host's streak; when fewer than two hosts remain wedged the alarm drops on
* its own (status field back to null, web banner hides).
*/
import { log as loggerLog } from "./logger.js";

const WEDGE_MIN_HOSTS = 2;
const WEDGE_SUSTAIN_MS = 5 * 60_000;
const LOG_DEDUP_MS = 10 * 60_000;

type ResolverWedgeWire = {
tripped: true;
/** Wall-clock ms when the earliest qualifying failure streak started. */
sinceMs: number;
elapsedMs: number;
/** Sorted hostnames currently wedged (full list, no truncation — the status reader owns display). */
hosts: string[];
};

type HostRecord = { lastGoodAt: number | null; failSince: number | null };

let hosts = new Map<string, HostRecord>();
let nowFn: () => number = () => Date.now();
let episodeLoggedAt = 0;

function evalAndLog(now: number): ResolverWedgeWire | null {
const wedged: Array<[string, number]> = [];
for (const [host, r] of hosts) {
if (r.failSince !== null && r.lastGoodAt !== null && now - r.failSince >= WEDGE_SUSTAIN_MS) {
wedged.push([host, r.failSince]);
}
}
if (wedged.length < WEDGE_MIN_HOSTS) return null;
wedged.sort((a, b) => a[0].localeCompare(b[0]));
const sinceMs = Math.min(...wedged.map(([, since]) => since));
if (now - episodeLoggedAt >= LOG_DEDUP_MS) {
episodeLoggedAt = now;
loggerLog(
"warn",
`[resolver-wedge] suspected local-resolver wedge: ${wedged.length} host(s) this process resolved before now fail every lookup for >=${WEDGE_SUSTAIN_MS / 60000} min (${wedged.slice(0, 5).map(([h]) => h).join(", ")}${wedged.length > 5 ? ", …" : ""}) — model requests 502 until the proxy process restarts or egress routes through an upstream proxy (-F/--forward-proxy); see #5221`,
);
}
return { tripped: true, sinceMs, elapsedMs: now - sinceMs, hosts: wedged.map(([h]) => h) };
}

/** Record ONE hostname-path resolution outcome from checkTunnelDestination.
* resolvedOk=true means "the resolver ANSWERED" (even with an empty set);
* false means every attempt THREW (ETIMEDOUT/EAI_AGAIN/…). IP literals and
* egress-proxied destinations never call this. */
export function resolverWedgeRecord(host: string, resolvedOk: boolean, nowMs?: number): void {
const key = host.toLowerCase();
const rec = hosts.get(key) ?? { lastGoodAt: null, failSince: null };
if (resolvedOk) {
rec.lastGoodAt = nowMs ?? nowFn();
rec.failSince = null;
} else if (rec.failSince === null) {
rec.failSince = nowMs ?? nowFn();
}
hosts.set(key, rec);
}

/** Current wedge verdict + its alarm log line. Cheap to call on every
* unresolvable admission AND from the 5s status poll — pure derived state. */
export function getResolverWedgeState(nowMs?: number): ResolverWedgeWire | null {
return evalAndLog(nowMs ?? nowFn());
}

/** Test seams only. */
export function _resolverWedgeTest(): { reset(): void; setNow(f: () => number): void } {
return {
reset() { hosts = new Map(); episodeLoggedAt = 0; },
setNow(f) { nowFn = f; },
};
}
4 changes: 3 additions & 1 deletion src/server/admin.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ import { effectivePromptCacheRouting, loadNamedProviders, loadOptions, loadRoute
import { applyCompressSettings, resolveCompressSurfaceDetailed } from "../compress-settings.js";
import { clearConflictEvents, summarizeConflicts } from "../conflict-watch.js";
import { cannotResolveTarget, getAdvisoryState } from "../advisory.js";
import { getResolverWedgeState } from "../resolver-wedge.js";
import { detectCostAdvisories } from "../plugin-advisory.js";
import { fetchWithTimeout } from "../fetch-util.js";
import { log as loggerLog, getLogPath } from "../logger.js";
Expand Down Expand Up @@ -1239,7 +1240,7 @@ async function sendStatus(res: http.ServerResponse, opts: ProxyOptions, laneHost
loggerLog("warn", `split-session canary (#2170): conversation ${w.base} has live traffic under multiple session keys (design persona forks are excluded): ${w.sessions.map((s) => `${s.id} (requests=${s.requests})`).join("; ")}. For a non-persona host this is the #2165 failure shape (stolen anchor / never-compressing split) — investigate if unexpected.`);
}
}
res.end(JSON.stringify({ version: VERSION, commit: BUILD_COMMIT, origin: laneHost ?? null, diskVersion, stale, lastInstall, updateReady: updateReadyPayload(), autoRestartOnUpdate: opts.autoRestartOnUpdate, advisory: currentAdvisoryPayload(), updatePolicy: updatePolicyPayload(), pluginAdvisories: detectCostAdvisories(process.env), inFlight: totalInFlight(), splitSessions: splitWarnings, conflicts: summarizeConflicts(listSessions()), storage: storageStatusWire(), snapshotShadow: snapshotShadowStatusWire(), attribution: attributionStatusWire() }, null, 2));
res.end(JSON.stringify({ version: VERSION, commit: BUILD_COMMIT, origin: laneHost ?? null, diskVersion, stale, lastInstall, updateReady: updateReadyPayload(), autoRestartOnUpdate: opts.autoRestartOnUpdate, advisory: currentAdvisoryPayload(), resolverWedge: getResolverWedgeState(), updatePolicy: updatePolicyPayload(), pluginAdvisories: detectCostAdvisories(process.env), inFlight: totalInFlight(), splitSessions: splitWarnings, conflicts: summarizeConflicts(listSessions()), storage: storageStatusWire(), snapshotShadow: snapshotShadowStatusWire(), attribution: attributionStatusWire() }, null, 2));
}

// #2090 plan A — read-only view backing the web UI's "Signed upstreams" card:
Expand Down Expand Up @@ -1295,6 +1296,7 @@ async function sendOverview(res: http.ServerResponse, opts: ProxyOptions, laneHo
updateReady: updateReadyPayload(),
autoRestartOnUpdate: opts.autoRestartOnUpdate,
advisory: currentAdvisoryPayload(),
resolverWedge: getResolverWedgeState(),
updatePolicy: updatePolicyPayload(),
pluginAdvisories: detectCostAdvisories(process.env),
hostPluginNotices: visibleHostPluginNotices(process.env),
Expand Down
26 changes: 18 additions & 8 deletions src/tunnel-guard.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import { lookup } from "node:dns/promises";
import { networkInterfaces } from "node:os";
import { getResolverWedgeState, resolverWedgeRecord } from "./resolver-wedge.js";

/**
* Tunnel admission for the `/bili/<absolute-url>` zero-config branch (#409).
Expand Down Expand Up @@ -204,14 +205,23 @@ export async function checkTunnelDestination(origin: string, ctx: TunnelCheckCon
await new Promise<void>((r) => setTimeout(r, backoff));
}
}
if (resolved === undefined) return {
ok: false,
code: "unresolvable",
// #5224: a long-lived process can hold stale OS-resolver state after a
// WSL/VPN change (glibc caches resolv.conf; libuv getaddrinfo ages out),
// so every admission fails until restart even though a fresh process resolves.
message: `cannot resolve tunnel destination ${host}${lastErrCode ? ` (${lastErrCode})` : ""}: local name resolution failed — the local resolver may be unreachable or taken over by another DNS service (VPN/MagicDNS); route egress through an upstream proxy to have the destination resolved remotely, or restart the bili proxy process (a long-lived process can hold a stale system-resolver state after a WSL/VPN network change; a fresh process re-reads it)`,
};
if (resolved === undefined) {
// #5221 phase 2: only THROWING resolutions are wedge evidence — feed
// the alarm detector (detection + alarm only; the process never acts).
resolverWedgeRecord(host, false);
getResolverWedgeState();
return {
ok: false,
code: "unresolvable",
// #5224: a long-lived process can hold stale OS-resolver state after a
// WSL/VPN change (glibc caches resolv.conf; libuv getaddrinfo ages out),
// so every admission fails until restart even though a fresh process resolves.
message: `cannot resolve tunnel destination ${host}${lastErrCode ? ` (${lastErrCode})` : ""}: local name resolution failed — the local resolver may be unreachable or taken over by another DNS service (VPN/MagicDNS); route egress through an upstream proxy to have the destination resolved remotely, or restart the bili proxy process (a long-lived process can hold a stale system-resolver state after a WSL/VPN network change; a fresh process re-reads it)`,
};
}
// The resolver ANSWERED (even an authoritative empty answer proves it is
// reachable) — this clears any failure streak for the host.
resolverWedgeRecord(host, true);
ips = resolved;
if (ips.length === 0) return { ok: false, code: "unresolvable", message: `no addresses for tunnel destination ${host}` };
// #2143: mDNS names (.local / single-label) answer with an unreachable
Expand Down
15 changes: 15 additions & 0 deletions src/web/client.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1454,6 +1454,21 @@ function changelogCatOf(rawLine) {
ab.innerHTML = "";
}
}
// #5221 phase 2 (owner ruling 2026-10-31): persistent local-resolver
// wedge — detection-only alarm banner beside the defect advisory.
const wb = $("resolver-wedge-banner");
if (wb) {
const w = d.resolverWedge;
if (w && w.tripped) {
wb.hidden = false;
wb.classList.add("show");
wb.innerHTML = '<strong>' + t("wedge.on") + '</strong> <span class="mono">' + w.hosts.join(", ") + '</span>' + t("wedge.desc") + t("wedge.hint");
} else {
wb.hidden = true;
wb.classList.remove("show");
wb.innerHTML = "";
}
}
const pb = $("plugin-advisory-banner");
if (pb) {
const advs = Array.isArray(d.pluginAdvisories) ? d.pluginAdvisories : [];
Expand Down
6 changes: 6 additions & 0 deletions src/web/i18n.ts
Original file line number Diff line number Diff line change
Expand Up @@ -256,6 +256,9 @@ export const MESSAGES: Record<Locale, Record<string, string>> = {
"advisory.on": "严重缺陷公告",
"advisory.desc": ":当前版本存在重大缺陷,bili 正在自动升级到安全版本。原因:",
"advisory.hint": "。若未自动完成,请手动执行:",
"wedge.on": "疑似本机 DNS 解析楔死",
"wedge.desc": ":本进程此前能正常解析的域名现在每次尝试都解析失败——重启前模型请求持续 502。涉及域名:",
"wedge.hint": "请重启本 bili 代理进程,或让出站走上游代理解析(-F/--forward-proxy);实时状态见 /__bili/status 的 resolverWedge 字段。",
"alert.title": "上游连接异常",
"alert.item": "无法连接 {host}({kind} ×{count},首次 {first})— {hint}",
"alert.note.skipped_proxy": "可用的代理 {proxy}({source})被跳过——未设置 upstreamProxyMode(默认直连是有意设计,#346/#357);设置 \"upstreamProxyMode\": \"auto\" 或 BILI_UPSTREAM_PROXY_MODE=auto,或为该 provider 配置 \"proxy\" 即可启用",
Expand Down Expand Up @@ -924,6 +927,9 @@ export const MESSAGES: Record<Locale, Record<string, string>> = {
"advisory.on": "Critical defect advisory",
"advisory.desc": ": the current version has a major defect; bili is upgrading it automatically to a safe version. Reason:",
"advisory.hint": ". If it did not complete automatically, run manually:",
"wedge.on": "Local DNS wedge suspected",
"wedge.desc": ": destination hostnames this process resolved fine before now fail every lookup attempt — model requests keep 502 until the process restarts. Hosts:",
"wedge.hint": " Restart the bili proxy process, or route egress through an upstream proxy (-F/--forward-proxy); live state: the resolverWedge field of /__bili/status.",
"alert.title": "Upstream connection problems",
"alert.item": "Cannot reach {host} ({kind} ×{count}, since {first}) — {hint}",
"alert.note.skipped_proxy": "available proxy {proxy} ({source}) was skipped — upstreamProxyMode is unset (direct is the default by design, #346/#357); set \"upstreamProxyMode\": \"auto\" or BILI_UPSTREAM_PROXY_MODE=auto, or give this provider a \"proxy\" lane to enable it",
Expand Down
1 change: 1 addition & 0 deletions src/web/page.ts
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ export function renderPage(origin: string, version: string, embed = false, theme
<div id="split-brain-banner" class="banner err" hidden></div>
<div id="conflicts-banner" class="banner warn" hidden></div>
<div id="advisory-banner" class="banner warn" hidden></div>
<div id="resolver-wedge-banner" class="banner warn" hidden></div>
<div id="update-policy-banner" class="banner warn" hidden></div>
<div id="plugin-advisory-banner" class="banner warn" hidden></div>
<div id="pck-banner" class="banner warn" hidden></div>
Expand Down
Loading
Loading