Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ Versions follow the merge of a `*_release-v*` branch; CI publishes to npm on tag
* **分代日志读取保持有界,「下载完整日志」不再暗中只取最后 100000 行**:Web 查询、诊断和用量读者逐行流式遍历,内存随输出/上下文和最长单行增长,不再随 256 MiB 保留预算整体增长。完整原始下载按时间顺序带背压发送所有可读代;预检发现不可读代会在界面与 `x-bili-log-incomplete` 标明,发头后读取失败则中断响应,不把截断当完整。时间窗口允许两次有界遍历但不承诺稳定快照,同一不可读代只计一次;显式 ACP `--log` 仍只读所选文件。看门狗等直接追加现在计入下一次封存的文件大小,活动文件超预算状态按当前字节重新计算,不再沿用打开时快照。预算回收还会等所有被替换流的 `end()` 完成,最后一个排水完成者才执行回收,避免刚封存的代在尾部缓冲落盘前被删除。

- **Hermes native plugin: the documented stock config shape (top-level `base_url` next to `provider:`) now joins the MITM whitelist (#6022).** Hermes' own config examples put the active provider's endpoint in a **top-level** `base_url:` with no `providers:` map and no `custom_providers:` list, but both of the plugin's `~/.hermes/config.yaml` readers scanned only those two map/list shapes — discovery returned `[]`, `_spawn_child()` never set `BILI_MITM_DOMAINS`, and every model request silently blind-tunneled: zero compression, no user-visible error beyond one BLIND TUNNEL WARNING log line per request. `mitm_domains_from_config()` now also reads the top-level `base_url`/`url`/`api` — additive, not fallback-only, since a config may carry both shapes and traffic can legally reach either host; `_provider_base_url()` resolves the same keys for the **active** provider (and unnamed requests — the hook surface often carries no provider kwarg) after its named-entry lookup, so the context-window / max-output header channels (#3261) stop degrading to `None` on this shape. A named non-active provider still gets only its own entry — the active endpoint never leaks into another provider's window resolution. The **launcher** had the same gap: `parseHermesYaml()` collected only `providers:`/`custom_providers:` entries, so `bili hermes` on the stock shape warned "no hermes providers found … traffic will NOT go through the proxy" and blind-tunneled everything; it now captures the indent-0 `provider:`/`base_url:`/`url:`/`api:` lines (the indent guard keeps a `model:`-nested `provider:` from binding) and merges the top-level URL into route discovery under the active provider's name without ever overwriting an existing entry — when both shapes carry different URLs, BOTH hosts stay routable rather than guessing which wins. Quoted endpoint values (`base_url: "https://…"`, as in Hermes' own examples) no longer die silently either: `new URL()` rejects surrounding quotes, so quoted entries were skipped as unparseable; quotes are now stripped at every capture site (unquoted values byte-identical). Behavior change (intended): dual-shape configs whitelist both hosts, and plugin instances spawned before this fix are refused by the attach gate (`mitm-domains-mismatch`) until restarted — the gate exists precisely to avoid riding a whitelist that does not cover the config's hosts. Regressions: `tests/hermes-native.test.ts` new `mitm-top-level` driver scenario (stock-shape whitelist + active/unnamed/other scoping truth table + mixed-shape order/isolation), `tests/launcher.test.ts` top-level shape + quoted-value cases.
* **「慢会话」告警只说实测事实、不再归因(#4528 遗留文案 / #5744 / #5481)**:这条告警的正文曾断言「每回合的 O(messages) 同步扫描正在饿死事件循环」,并建议用户「关闭/重启会话或开启压缩」。实测里大历史请求常常几十毫秒就完成本地准备(热缓存、请求体小),此时告警内容与同一行 `local=` 毫秒数直接自相矛盾,还会把用户引向「重启会话」这种既不必要又有损的操作。#5744 已把触发条件从「本轮入站消息数 > 1000」换成实测墙钟(`local > 1000ms`,常量 `SLOW_SESSION_MS_WARN`),因为同一会话在同样约 3k 条消息下实测既有 0.8s 也有 14.5s,条数无法把成本与体积分开;本 PR 接着把陈述改成纯事实:只报消息条数、入站字节与实测本地毫秒,明说「本行只报实测、不归因,不构成事件循环饥饿的证据」,并删掉重启/开压缩的建议。墙钟触发条件(严格 >1000ms)、warn 级别、`local=`/`inbound=`/`outbound=`/`view=` 字段与 #5408 的 `noteRequest` 观测点全部保留;标签仍是 `slow-session`(触发条件就是墙钟,改名不再必要)。历史长停顿的真实成因仍未证实,因此没有新增任何延迟阈值、重试、配置项或对会话历史的改写。该单元从 `src/server.ts` 抽出到 `src/server/request-cost.ts`(连同它的阈值常量与 `formatBytes`),`src/server/handle.ts` 的 `logRequestCost` 导入改指新模块,`src/server.ts` 不再转出。正常上游路径现在在兼容重写与 V1 封装后的最终 fetch body 上计量 `outbound`,不再拿转发前字符串代替实际上游字节;codex 本地 forge 路径仍记一行但不再携带 `outbound` 字段(该路径没有交给 fetch 的 body),auto-mode forge、side 与 fail-fast 的既有记账逐字不变。
- **KeyRing model-local 429s no longer throttle other models on the same credential.** Bili gives an independent origin+credential+model hold only to HTTP 429 responses whose structured KeyRing or Bili marker has an exact request-model match; `keyring_all_limited`, direct OpenCode monthly quotas, malformed bodies, and mismatches remain origin+credential holds. Large passthrough bodies use a top-level model scan, while Gemini/Vertex/v1 and side-lane paths carry their actual wire model. Account/model probe slots are request-owned and canceled sends release only their token; concurrent provisional 429 observations settle independently. Continuation retries reserve bounded classification capacity per origin+credential before egress; saturation skips only the optional refetch, without a global gate or discarding account-wide evidence. Continuation reads retain the upstream idle/abort policy. Side-lane SSE retains its stream mode so render-tag stripping still runs. Regressions: `tests/issue4204-throttle-governor.test.ts`, `tests/issue4743-degenerate-refetch-429.test.ts`, `tests/proxy-noinject-tag-strip.test.ts`, and `tests/upstream-error-log.test.ts`.

- **中断后/usage 丢失后的「幻影紧急压缩」:本轮拿不到计费样本时,压力档压缩延后一轮再决定**:中断的流(用户 ESC)或上游短 TTL 缓存把 usage 报告弄丢时,本轮的「计量表针」只能落回字符级本地估算——它量的是即将被改写丢掉的入站字节,实测比真实计费高约 3 倍(0.1.212 当天两起真实现场:ESC 的会话从 9478 跳成 228376、长跑会话被连续两轮注入 153% 幻影压力带,徒劳改写并烧掉前缀缓存)。此前这种坏表针直接按 EMERGENCY/OVER-LIMIT 扣扳机。现在:内核收到以「估算值」计量的压力带读数(宿主标记 tokenCountSource="estimate")时,本轮压力档注压延后一轮——下一次请求必然带着新鲜计费报告,拿真数再决定,代价≈0;计费级读数("usage")与不传 source 的老调用方字节不变,非压力档(增长/节拍)完全不受影响,硬溢出兜底臂全程照旧武装。宿主侧提前广告臂(emergencyNudge)同步镜像该位,不会拿着同一个坏读数抢跑。逃生阀:`compress.requireUsageSampleForPressure: false` 恢复旧的估算驱动行为。tests:kernel/tests/nudge-pressure-no-sample-hold.test.ts(6 例)、tests/nudge-pressure-no-sample-armed.test.ts(3 例);kernel 既有套件 1582/1582 绿。
Expand Down
1 change: 1 addition & 0 deletions reference/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ billion-context/
│ │ ├── prepare-openai.ts # OpenAI chat request preparation (#1440)
│ │ ├── prepare-responses.ts # Responses request preparation + codex compact/prompt-cache helpers (#1440)
│ │ ├── relay.ts # Upstream relay zone: forward() + fake-completion recovery (#1440 P2 cut 4)
│ │ ├── request-cost.ts # Per-request local-cost line (outbound measured at the real wire body) + #5408 heartbeat + wall-clock slow-session advisory, facts only (#903/#4528/#5744)
│ │ ├── responses-ordering.ts # #564 Responses assistant-run ordering repair + producer-owned turn-separator provenance (#2627)
│ │ ├── responses-separator.ts # Canonical Responses turn-separator literal and carrier predicates (#2627)
│ │ ├── side-request.ts # Tool-surface check for auxiliary side requests
Expand Down
58 changes: 0 additions & 58 deletions src/server.ts
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,6 @@ import { clearConflictEvents, recordConflict, summarizeConflicts } from "./confl
import { getStore, setGlobalSessionCommitListener } from "./persist.js";
import { log as loggerLog, configureLogger, getLogPath, closeLogger, isStreamWriteError, isBenignSocketRaceError, enterSessionContext, muteStderrLeg, onFileLegLive, shouldMuteStderrLegAfterFileLive, logTimestamp } from "./logger.js";
import { configureLogRetention } from "./log-retention.js";
import { noteRequest } from "./attach-heartbeat.js";
import { configFile, defaultLogFile, sessionsDir, stateDir } from "./paths.js";
import { atomicWriteInstanceFile, clearProxyInstanceFile, classifySameLanePeer, entryShape, findPresentUnverifiedSameLaneTwin, findSameLanePredecessor, findVerifiedSameLaneTwin, isPidAlive, listInstances, markPortReleased, probePid, readProxyInstanceFile, registerInstanceAndWarn, resolveOwnFingerprint, sameInstanceShape, SPAWN_FINGERPRINT_ENV, unregisterInstance, warnOnNewPeers, writeSpawnPortLog, type InstanceShape, type PidProbe, type ProxyInstanceFile, type RegistryEntry, type SameLanePeer } from "./instance.js";
import { createServedMarkerWriter, peerStillContests, processStartMs, reapStaleWriterMarkers, writerMarkerDir, type ServedMarkerWriter } from "./served-markers.js";
Expand Down Expand Up @@ -156,7 +155,6 @@ import { BILI_HOP_HEADER, anthropicBetaContextWindow, capRegistryWindowByStandar
import { connectionNamedHeaders, NO_IDENTITY_MESSAGE, stripRemoteCompactionBetaFeature, UPSTREAM_HOP_HEADERS } from "./server/headers.js";
import { installWebSocketBridge } from "./ws-bridge.js";
import { codexResponsesCodec, responsesCodec } from "./responses-ws.js";
import { currentFetchTransport } from "./fetch-transport.js";
import { demoteGate, hasLeakedBiliToolsOnly, isSideRequest, outputBudgetField, resolveSideLane, restoreOutputBudget, SIDE_REQUEST_MAX_TOKENS, sideRequestGuard, stripLeakedBiliTools } from "./server/side-request.js";
import { DSH_COMPACTION_SHAPE_MSGS, dshCompactionRefusal, isDshCompactionCall } from "./server/dsh-compaction-guard.js";
import { clampOutgoingOutput, countSystemAndToolsTokens, emergencyNudge, estimateInputTokens, estimateWireOverhead, projectThinkingMass, readOutputBudget } from "./server/budget.js";
Expand Down Expand Up @@ -4728,62 +4726,6 @@ export function headerValue(req: http.IncomingMessage, name: string): string | u
return undefined;
}

function formatBytes(n: number): string {
if (n < 1024) return `${n}B`;
if (n < 1024 * 1024) return `${(n / 1024).toFixed(1)}KiB`;
if (n < 1024 * 1024 * 1024) return `${(n / (1024 * 1024)).toFixed(1)}MiB`;
return `${(n / (1024 * 1024 * 1024)).toFixed(1)}GiB`;
}

// #903: per-request local-cost line, logged at every point where bili finishes
// its own processing and hands the request off (upstream forward, forged local
// response, or fail-fast error) — see handle(). outbound is the exact body value
// handed to forward() (string OR Buffer — Prepared.body is string|Buffer); wire
// bytes are counted with Buffer.byteLength since undici sends UTF-8 bytes even
// for string bodies (.length would count UTF-16 chars and undercount any
// non-ASCII injection). Omitted on paths that fail fast without forwarding.
// #4528: bili's own per-request processing past this WALL-CLOCK budget means
// synchronous main-thread work large enough to starve the event loop and
// freeze every endpoint on the same listener — warn so a pathological session
// shows up in the log instead of being found through a frozen UI.
// #5744: the trigger used to be message COUNT (>1000), which cannot separate
// cost from size — one reporter session saved at 0.8s AND 14.5s at the same
// ~3k msgs. Wall clock IS the cost; the info line above keeps reporting the
// count for correlation.
const SLOW_SESSION_MS_WARN = 1000;
// #5907 follow-up: the warn fires ONCE per session per process — a 3k-msg
// session exceeds the wall budget on EVERY turn, so re-logging each turn
// floods the log while adding nothing; the first occurrence carries the full
// size picture (msgs + inbound + est tokens) plus the cleanup command. The
// map is bounded: past the cap the earliest-warned session is evicted so a
// long-lived proxy with many sessions cannot grow it without bound.
const SLOW_SESSION_WARNED = new Map<string, number>();
const MAX_SLOW_SESSION_WARNED = 512;
export function _resetSlowSessionWarnedForTest(): void {
SLOW_SESSION_WARNED.clear();
}
export function logRequestCost(log: (level: string, msg: string) => void, sessionId: string, msgs: number | null, inboundBytes: number, t0: number, outbound?: string | Buffer): void {
const ms = Math.max(0, Math.round(performance.now() - t0));
const outboundField = outbound !== undefined ? `, outbound=${formatBytes(Buffer.byteLength(outbound))}` : "";
const view = currentFetchTransport() ? ", view=ws-expanded" : "";
log("info", `[${sessionId}] request: ${msgs ?? "?"} msgs, inbound=${formatBytes(inboundBytes)}${outboundField}, local=${ms}ms${view}`);
// #5408: this line is THE observation point — a hot-at-attach session that
// never produces one is silent compression loss (the incident signature).
noteRequest(sessionId);
if (ms > SLOW_SESSION_MS_WARN && !SLOW_SESSION_WARNED.has(sessionId)) {
SLOW_SESSION_WARNED.set(sessionId, ms);
while (SLOW_SESSION_WARNED.size > MAX_SLOW_SESSION_WARNED) {
const oldest = SLOW_SESSION_WARNED.keys().next().value;
if (oldest === undefined) break;
SLOW_SESSION_WARNED.delete(oldest);
}
// bytes/4 is the repo-wide optimistic token floor (same caliber as the
// oversize gates) — labeled est because JSON overhead skews it high.
const estTok = Math.round(inboundBytes / 4);
log("warn", `[${sessionId}] slow-session: ${msgs ?? "?"} msgs / inbound=${formatBytes(inboundBytes)} (~${estTok} est tok) took local=${ms}ms — synchronous per-turn work is starving the event loop (#4528/#5744); the session has grown large — consider \`bili sessions rm ${sessionId}\` or enabling compression (#5907)`);
}
}

/** Thrown by readBody when the request body exceeds the configured receive-body
* budget (limitsReceiveBytes). handle() catches this and attempts a 413 response;
* readBody also destroys the request socket so a client that keeps streaming a
Expand Down
10 changes: 5 additions & 5 deletions src/server/handle.ts
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,8 @@ import { handleIncrementalTurn, handleIncrementalProbe, incrementalTurnBodyShape
import { handlePluginRender, PLUGIN_RENDER_PATH } from "../plugin-render.js";
import { ccrPluginWireOk, storeEffectiveCcr, storeEffectiveImageRescale, type CcrSettings } from "../store.js";
import { DEFAULT_DECIDE_MAX_TOKENS } from "../nudge-decide.js";
import { BodyTooLargeError, buildForwardTarget, forward, forwardUpstreamUrl, googleModelFromPath, googlePathKind, headroomEffectiveLogged, headroomFallbackLogged, headerValue, imageBillingFor, imageReserveFor, imageTokenCapFor, isCountTokensRequest, isPreflightFailFast, logRequestCost, NON_CONVERSATION_RELAY_WARN_CAP, nonConversationRelayWarned, prepareCountTokens, preflightCompressIfNeeded, readBody, resolveKnownOutputCeiling, resolveUpstream, resignSettingsFor, selfAddressedLocalPath, scrubAnthropicPck, scrubCompatDrop, vertexAnthropicModelFromPath, warnModelNameMissIfNew, warnRouteMissIfNew, type Prepared } from "../server.js";
import { BodyTooLargeError, buildForwardTarget, forward, forwardUpstreamUrl, googleModelFromPath, googlePathKind, headroomEffectiveLogged, headroomFallbackLogged, headerValue, imageBillingFor, imageReserveFor, imageTokenCapFor, isCountTokensRequest, isPreflightFailFast, NON_CONVERSATION_RELAY_WARN_CAP, nonConversationRelayWarned, prepareCountTokens, preflightCompressIfNeeded, readBody, resolveKnownOutputCeiling, resolveUpstream, resignSettingsFor, selfAddressedLocalPath, scrubAnthropicPck, scrubCompatDrop, vertexAnthropicModelFromPath, warnModelNameMissIfNew, warnRouteMissIfNew, type Prepared } from "../server.js";
import { logRequestCost } from "./request-cost.js";
import { normalizeVertexModelId } from "../vertex.js";
import { CREDENTIAL_HEADER_NAMES } from "../cache-ledger.js";
import { requestModelFromBody } from "../upstream-throttle.js";
Expand Down Expand Up @@ -2846,7 +2847,6 @@ async function handleRequest(
if (mode === "intercept" && gatePre) prepared = await runPrepare();
if (prepared) memProbe(opts.debug, log, session.id, "prepared", typeof prepared.body === "string" ? Buffer.byteLength(prepared.body) : prepared.body.byteLength);
if (prepared?.codexForge) {
logRequestCost(log, session.id, inboundMsgs, inboundBytes, reqT0, prepared.body);
return { body: prepared.body, prepared };
}
// runPrepare may have mutated parsed before failing to forge.
Expand All @@ -2858,7 +2858,6 @@ async function handleRequest(
const forwardBody = normalized ? Buffer.from(JSON.stringify({ ...original, input: items })) : bodyBuffer;
const why = mode !== "intercept" ? "BILI_CODEX_COMPACT=pass" : !gatePre ? "gate preconditions not met" : "transform/forge failed";
log("info", `[${session.id}] codex compaction_trigger request not intercepted (${why}) — forwarding ${normalized ? `with bili summaries normalized (replaced=${replaced}, dropped=${dropped})` : "verbatim"} (no preflight, no rebuild, no window clamp)`);
logRequestCost(log, session.id, inboundMsgs, inboundBytes, reqT0, forwardBody);
return { body: forwardBody, prepared: null };
}
// #3450: ONE wall per request — on the refold path the
Expand Down Expand Up @@ -3060,7 +3059,6 @@ async function handleRequest(
// forward — anything that queued during the turn is served
// here rather than behind the response stream setup.
await yieldToEventLoop();
logRequestCost(log, session.id, inboundMsgs, inboundBytes, reqT0, prepared!.body);
return { body: prepared!.body, prepared: prepared! };
};
// #1884/#2090 (un-armed signed traffic): a request that already
Expand Down Expand Up @@ -3146,7 +3144,9 @@ async function handleRequest(
return next;
}
: undefined;
await forward(req, res, opts, pendingForward.body, pendingForward.prepared, core, reqConfig, log, route, instanceId, affinity, overflowRefold, v1Meta, throttleModel);
await forward(req, res, opts, pendingForward.body, pendingForward.prepared, core, reqConfig, log, route, instanceId, affinity, overflowRefold, v1Meta, throttleModel, (outbound) => {
logRequestCost(log, session.id, inboundMsgs, inboundBytes, reqT0, outbound);
});
// Remember for ALL modes (not just plugin): wire clients (dsh,
// hermes, unplug'd pi) read the same panel via /__bili/plugin/status
// and need the nudge/breakdown sections too; locked so a racing
Expand Down
Loading
Loading