diff --git a/CHANGELOG.md b/CHANGELOG.md index 68bfed35f3..b6f0b9ce5f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -34,6 +34,7 @@ Versions follow the merge of a `*_release-v*` branch; CI publishes to npm on tag * **分代日志读取保持有界,「下载完整日志」不再暗中只取最后 100000 行**:Web 查询、诊断和用量读者逐行流式遍历,内存随输出/上下文和最长单行增长,不再随 256 MiB 保留预算整体增长。完整原始下载按时间顺序带背压发送所有可读代;预检发现不可读代会在界面与 `x-bili-log-incomplete` 标明,发头后读取失败则中断响应,不把截断当完整。时间窗口允许两次有界遍历但不承诺稳定快照,同一不可读代只计一次;显式 ACP `--log` 仍只读所选文件。看门狗等直接追加现在计入下一次封存的文件大小,活动文件超预算状态按当前字节重新计算,不再沿用打开时快照。预算回收还会等所有被替换流的 `end()` 完成,最后一个排水完成者才执行回收,避免刚封存的代在尾部缓冲落盘前被删除。 - **Hermes native plugin: the documented stock config shape (top-level `base_url` next to `provider:`) now joins the MITM whitelist (#6022).** Hermes' own config examples put the active provider's endpoint in a **top-level** `base_url:` with no `providers:` map and no `custom_providers:` list, but both of the plugin's `~/.hermes/config.yaml` readers scanned only those two map/list shapes — discovery returned `[]`, `_spawn_child()` never set `BILI_MITM_DOMAINS`, and every model request silently blind-tunneled: zero compression, no user-visible error beyond one BLIND TUNNEL WARNING log line per request. `mitm_domains_from_config()` now also reads the top-level `base_url`/`url`/`api` — additive, not fallback-only, since a config may carry both shapes and traffic can legally reach either host; `_provider_base_url()` resolves the same keys for the **active** provider (and unnamed requests — the hook surface often carries no provider kwarg) after its named-entry lookup, so the context-window / max-output header channels (#3261) stop degrading to `None` on this shape. A named non-active provider still gets only its own entry — the active endpoint never leaks into another provider's window resolution. The **launcher** had the same gap: `parseHermesYaml()` collected only `providers:`/`custom_providers:` entries, so `bili hermes` on the stock shape warned "no hermes providers found … traffic will NOT go through the proxy" and blind-tunneled everything; it now captures the indent-0 `provider:`/`base_url:`/`url:`/`api:` lines (the indent guard keeps a `model:`-nested `provider:` from binding) and merges the top-level URL into route discovery under the active provider's name without ever overwriting an existing entry — when both shapes carry different URLs, BOTH hosts stay routable rather than guessing which wins. Quoted endpoint values (`base_url: "https://…"`, as in Hermes' own examples) no longer die silently either: `new URL()` rejects surrounding quotes, so quoted entries were skipped as unparseable; quotes are now stripped at every capture site (unquoted values byte-identical). Behavior change (intended): dual-shape configs whitelist both hosts, and plugin instances spawned before this fix are refused by the attach gate (`mitm-domains-mismatch`) until restarted — the gate exists precisely to avoid riding a whitelist that does not cover the config's hosts. Regressions: `tests/hermes-native.test.ts` new `mitm-top-level` driver scenario (stock-shape whitelist + active/unnamed/other scoping truth table + mixed-shape order/isolation), `tests/launcher.test.ts` top-level shape + quoted-value cases. +* **「慢会话」告警只说实测事实、不再归因(#4528 遗留文案 / #5744 / #5481)**:这条告警的正文曾断言「每回合的 O(messages) 同步扫描正在饿死事件循环」,并建议用户「关闭/重启会话或开启压缩」。实测里大历史请求常常几十毫秒就完成本地准备(热缓存、请求体小),此时告警内容与同一行 `local=` 毫秒数直接自相矛盾,还会把用户引向「重启会话」这种既不必要又有损的操作。#5744 已把触发条件从「本轮入站消息数 > 1000」换成实测墙钟(`local > 1000ms`,常量 `SLOW_SESSION_MS_WARN`),因为同一会话在同样约 3k 条消息下实测既有 0.8s 也有 14.5s,条数无法把成本与体积分开;本 PR 接着把陈述改成纯事实:只报消息条数、入站字节与实测本地毫秒,明说「本行只报实测、不归因,不构成事件循环饥饿的证据」,并删掉重启/开压缩的建议。墙钟触发条件(严格 >1000ms)、warn 级别、`local=`/`inbound=`/`outbound=`/`view=` 字段与 #5408 的 `noteRequest` 观测点全部保留;标签仍是 `slow-session`(触发条件就是墙钟,改名不再必要)。历史长停顿的真实成因仍未证实,因此没有新增任何延迟阈值、重试、配置项或对会话历史的改写。该单元从 `src/server.ts` 抽出到 `src/server/request-cost.ts`(连同它的阈值常量与 `formatBytes`),`src/server/handle.ts` 的 `logRequestCost` 导入改指新模块,`src/server.ts` 不再转出。正常上游路径现在在兼容重写与 V1 封装后的最终 fetch body 上计量 `outbound`,不再拿转发前字符串代替实际上游字节;codex 本地 forge 路径仍记一行但不再携带 `outbound` 字段(该路径没有交给 fetch 的 body),auto-mode forge、side 与 fail-fast 的既有记账逐字不变。 - **KeyRing model-local 429s no longer throttle other models on the same credential.** Bili gives an independent origin+credential+model hold only to HTTP 429 responses whose structured KeyRing or Bili marker has an exact request-model match; `keyring_all_limited`, direct OpenCode monthly quotas, malformed bodies, and mismatches remain origin+credential holds. Large passthrough bodies use a top-level model scan, while Gemini/Vertex/v1 and side-lane paths carry their actual wire model. Account/model probe slots are request-owned and canceled sends release only their token; concurrent provisional 429 observations settle independently. Continuation retries reserve bounded classification capacity per origin+credential before egress; saturation skips only the optional refetch, without a global gate or discarding account-wide evidence. Continuation reads retain the upstream idle/abort policy. Side-lane SSE retains its stream mode so render-tag stripping still runs. Regressions: `tests/issue4204-throttle-governor.test.ts`, `tests/issue4743-degenerate-refetch-429.test.ts`, `tests/proxy-noinject-tag-strip.test.ts`, and `tests/upstream-error-log.test.ts`. - **中断后/usage 丢失后的「幻影紧急压缩」:本轮拿不到计费样本时,压力档压缩延后一轮再决定**:中断的流(用户 ESC)或上游短 TTL 缓存把 usage 报告弄丢时,本轮的「计量表针」只能落回字符级本地估算——它量的是即将被改写丢掉的入站字节,实测比真实计费高约 3 倍(0.1.212 当天两起真实现场:ESC 的会话从 9478 跳成 228376、长跑会话被连续两轮注入 153% 幻影压力带,徒劳改写并烧掉前缀缓存)。此前这种坏表针直接按 EMERGENCY/OVER-LIMIT 扣扳机。现在:内核收到以「估算值」计量的压力带读数(宿主标记 tokenCountSource="estimate")时,本轮压力档注压延后一轮——下一次请求必然带着新鲜计费报告,拿真数再决定,代价≈0;计费级读数("usage")与不传 source 的老调用方字节不变,非压力档(增长/节拍)完全不受影响,硬溢出兜底臂全程照旧武装。宿主侧提前广告臂(emergencyNudge)同步镜像该位,不会拿着同一个坏读数抢跑。逃生阀:`compress.requireUsageSampleForPressure: false` 恢复旧的估算驱动行为。tests:kernel/tests/nudge-pressure-no-sample-hold.test.ts(6 例)、tests/nudge-pressure-no-sample-armed.test.ts(3 例);kernel 既有套件 1582/1582 绿。 diff --git a/reference/architecture.md b/reference/architecture.md index 064739ed92..9afea78f6a 100644 --- a/reference/architecture.md +++ b/reference/architecture.md @@ -28,6 +28,7 @@ billion-context/ │ │ ├── prepare-openai.ts # OpenAI chat request preparation (#1440) │ │ ├── prepare-responses.ts # Responses request preparation + codex compact/prompt-cache helpers (#1440) │ │ ├── relay.ts # Upstream relay zone: forward() + fake-completion recovery (#1440 P2 cut 4) +│ │ ├── request-cost.ts # Per-request local-cost line (outbound measured at the real wire body) + #5408 heartbeat + wall-clock slow-session advisory, facts only (#903/#4528/#5744) │ │ ├── responses-ordering.ts # #564 Responses assistant-run ordering repair + producer-owned turn-separator provenance (#2627) │ │ ├── responses-separator.ts # Canonical Responses turn-separator literal and carrier predicates (#2627) │ │ ├── side-request.ts # Tool-surface check for auxiliary side requests diff --git a/src/server.ts b/src/server.ts index 7e7343bc93..1c7064d073 100644 --- a/src/server.ts +++ b/src/server.ts @@ -97,7 +97,6 @@ import { clearConflictEvents, recordConflict, summarizeConflicts } from "./confl import { getStore, setGlobalSessionCommitListener } from "./persist.js"; import { log as loggerLog, configureLogger, getLogPath, closeLogger, isStreamWriteError, isBenignSocketRaceError, enterSessionContext, muteStderrLeg, onFileLegLive, shouldMuteStderrLegAfterFileLive, logTimestamp } from "./logger.js"; import { configureLogRetention } from "./log-retention.js"; -import { noteRequest } from "./attach-heartbeat.js"; import { configFile, defaultLogFile, sessionsDir, stateDir } from "./paths.js"; import { atomicWriteInstanceFile, clearProxyInstanceFile, classifySameLanePeer, entryShape, findPresentUnverifiedSameLaneTwin, findSameLanePredecessor, findVerifiedSameLaneTwin, isPidAlive, listInstances, markPortReleased, probePid, readProxyInstanceFile, registerInstanceAndWarn, resolveOwnFingerprint, sameInstanceShape, SPAWN_FINGERPRINT_ENV, unregisterInstance, warnOnNewPeers, writeSpawnPortLog, type InstanceShape, type PidProbe, type ProxyInstanceFile, type RegistryEntry, type SameLanePeer } from "./instance.js"; import { createServedMarkerWriter, peerStillContests, processStartMs, reapStaleWriterMarkers, writerMarkerDir, type ServedMarkerWriter } from "./served-markers.js"; @@ -156,7 +155,6 @@ import { BILI_HOP_HEADER, anthropicBetaContextWindow, capRegistryWindowByStandar import { connectionNamedHeaders, NO_IDENTITY_MESSAGE, stripRemoteCompactionBetaFeature, UPSTREAM_HOP_HEADERS } from "./server/headers.js"; import { installWebSocketBridge } from "./ws-bridge.js"; import { codexResponsesCodec, responsesCodec } from "./responses-ws.js"; -import { currentFetchTransport } from "./fetch-transport.js"; import { demoteGate, hasLeakedBiliToolsOnly, isSideRequest, outputBudgetField, resolveSideLane, restoreOutputBudget, SIDE_REQUEST_MAX_TOKENS, sideRequestGuard, stripLeakedBiliTools } from "./server/side-request.js"; import { DSH_COMPACTION_SHAPE_MSGS, dshCompactionRefusal, isDshCompactionCall } from "./server/dsh-compaction-guard.js"; import { clampOutgoingOutput, countSystemAndToolsTokens, emergencyNudge, estimateInputTokens, estimateWireOverhead, projectThinkingMass, readOutputBudget } from "./server/budget.js"; @@ -4728,62 +4726,6 @@ export function headerValue(req: http.IncomingMessage, name: string): string | u return undefined; } -function formatBytes(n: number): string { - if (n < 1024) return `${n}B`; - if (n < 1024 * 1024) return `${(n / 1024).toFixed(1)}KiB`; - if (n < 1024 * 1024 * 1024) return `${(n / (1024 * 1024)).toFixed(1)}MiB`; - return `${(n / (1024 * 1024 * 1024)).toFixed(1)}GiB`; -} - -// #903: per-request local-cost line, logged at every point where bili finishes -// its own processing and hands the request off (upstream forward, forged local -// response, or fail-fast error) — see handle(). outbound is the exact body value -// handed to forward() (string OR Buffer — Prepared.body is string|Buffer); wire -// bytes are counted with Buffer.byteLength since undici sends UTF-8 bytes even -// for string bodies (.length would count UTF-16 chars and undercount any -// non-ASCII injection). Omitted on paths that fail fast without forwarding. -// #4528: bili's own per-request processing past this WALL-CLOCK budget means -// synchronous main-thread work large enough to starve the event loop and -// freeze every endpoint on the same listener — warn so a pathological session -// shows up in the log instead of being found through a frozen UI. -// #5744: the trigger used to be message COUNT (>1000), which cannot separate -// cost from size — one reporter session saved at 0.8s AND 14.5s at the same -// ~3k msgs. Wall clock IS the cost; the info line above keeps reporting the -// count for correlation. -const SLOW_SESSION_MS_WARN = 1000; -// #5907 follow-up: the warn fires ONCE per session per process — a 3k-msg -// session exceeds the wall budget on EVERY turn, so re-logging each turn -// floods the log while adding nothing; the first occurrence carries the full -// size picture (msgs + inbound + est tokens) plus the cleanup command. The -// map is bounded: past the cap the earliest-warned session is evicted so a -// long-lived proxy with many sessions cannot grow it without bound. -const SLOW_SESSION_WARNED = new Map(); -const MAX_SLOW_SESSION_WARNED = 512; -export function _resetSlowSessionWarnedForTest(): void { - SLOW_SESSION_WARNED.clear(); -} -export function logRequestCost(log: (level: string, msg: string) => void, sessionId: string, msgs: number | null, inboundBytes: number, t0: number, outbound?: string | Buffer): void { - const ms = Math.max(0, Math.round(performance.now() - t0)); - const outboundField = outbound !== undefined ? `, outbound=${formatBytes(Buffer.byteLength(outbound))}` : ""; - const view = currentFetchTransport() ? ", view=ws-expanded" : ""; - log("info", `[${sessionId}] request: ${msgs ?? "?"} msgs, inbound=${formatBytes(inboundBytes)}${outboundField}, local=${ms}ms${view}`); - // #5408: this line is THE observation point — a hot-at-attach session that - // never produces one is silent compression loss (the incident signature). - noteRequest(sessionId); - if (ms > SLOW_SESSION_MS_WARN && !SLOW_SESSION_WARNED.has(sessionId)) { - SLOW_SESSION_WARNED.set(sessionId, ms); - while (SLOW_SESSION_WARNED.size > MAX_SLOW_SESSION_WARNED) { - const oldest = SLOW_SESSION_WARNED.keys().next().value; - if (oldest === undefined) break; - SLOW_SESSION_WARNED.delete(oldest); - } - // bytes/4 is the repo-wide optimistic token floor (same caliber as the - // oversize gates) — labeled est because JSON overhead skews it high. - const estTok = Math.round(inboundBytes / 4); - log("warn", `[${sessionId}] slow-session: ${msgs ?? "?"} msgs / inbound=${formatBytes(inboundBytes)} (~${estTok} est tok) took local=${ms}ms — synchronous per-turn work is starving the event loop (#4528/#5744); the session has grown large — consider \`bili sessions rm ${sessionId}\` or enabling compression (#5907)`); - } -} - /** Thrown by readBody when the request body exceeds the configured receive-body * budget (limitsReceiveBytes). handle() catches this and attempts a 413 response; * readBody also destroys the request socket so a client that keeps streaming a diff --git a/src/server/handle.ts b/src/server/handle.ts index 875cd8bbfd..ee339240ed 100644 --- a/src/server/handle.ts +++ b/src/server/handle.ts @@ -89,7 +89,8 @@ import { handleIncrementalTurn, handleIncrementalProbe, incrementalTurnBodyShape import { handlePluginRender, PLUGIN_RENDER_PATH } from "../plugin-render.js"; import { ccrPluginWireOk, storeEffectiveCcr, storeEffectiveImageRescale, type CcrSettings } from "../store.js"; import { DEFAULT_DECIDE_MAX_TOKENS } from "../nudge-decide.js"; -import { BodyTooLargeError, buildForwardTarget, forward, forwardUpstreamUrl, googleModelFromPath, googlePathKind, headroomEffectiveLogged, headroomFallbackLogged, headerValue, imageBillingFor, imageReserveFor, imageTokenCapFor, isCountTokensRequest, isPreflightFailFast, logRequestCost, NON_CONVERSATION_RELAY_WARN_CAP, nonConversationRelayWarned, prepareCountTokens, preflightCompressIfNeeded, readBody, resolveKnownOutputCeiling, resolveUpstream, resignSettingsFor, selfAddressedLocalPath, scrubAnthropicPck, scrubCompatDrop, vertexAnthropicModelFromPath, warnModelNameMissIfNew, warnRouteMissIfNew, type Prepared } from "../server.js"; +import { BodyTooLargeError, buildForwardTarget, forward, forwardUpstreamUrl, googleModelFromPath, googlePathKind, headroomEffectiveLogged, headroomFallbackLogged, headerValue, imageBillingFor, imageReserveFor, imageTokenCapFor, isCountTokensRequest, isPreflightFailFast, NON_CONVERSATION_RELAY_WARN_CAP, nonConversationRelayWarned, prepareCountTokens, preflightCompressIfNeeded, readBody, resolveKnownOutputCeiling, resolveUpstream, resignSettingsFor, selfAddressedLocalPath, scrubAnthropicPck, scrubCompatDrop, vertexAnthropicModelFromPath, warnModelNameMissIfNew, warnRouteMissIfNew, type Prepared } from "../server.js"; +import { logRequestCost } from "./request-cost.js"; import { normalizeVertexModelId } from "../vertex.js"; import { CREDENTIAL_HEADER_NAMES } from "../cache-ledger.js"; import { requestModelFromBody } from "../upstream-throttle.js"; @@ -2846,7 +2847,6 @@ async function handleRequest( if (mode === "intercept" && gatePre) prepared = await runPrepare(); if (prepared) memProbe(opts.debug, log, session.id, "prepared", typeof prepared.body === "string" ? Buffer.byteLength(prepared.body) : prepared.body.byteLength); if (prepared?.codexForge) { - logRequestCost(log, session.id, inboundMsgs, inboundBytes, reqT0, prepared.body); return { body: prepared.body, prepared }; } // runPrepare may have mutated parsed before failing to forge. @@ -2858,7 +2858,6 @@ async function handleRequest( const forwardBody = normalized ? Buffer.from(JSON.stringify({ ...original, input: items })) : bodyBuffer; const why = mode !== "intercept" ? "BILI_CODEX_COMPACT=pass" : !gatePre ? "gate preconditions not met" : "transform/forge failed"; log("info", `[${session.id}] codex compaction_trigger request not intercepted (${why}) — forwarding ${normalized ? `with bili summaries normalized (replaced=${replaced}, dropped=${dropped})` : "verbatim"} (no preflight, no rebuild, no window clamp)`); - logRequestCost(log, session.id, inboundMsgs, inboundBytes, reqT0, forwardBody); return { body: forwardBody, prepared: null }; } // #3450: ONE wall per request — on the refold path the @@ -3060,7 +3059,6 @@ async function handleRequest( // forward — anything that queued during the turn is served // here rather than behind the response stream setup. await yieldToEventLoop(); - logRequestCost(log, session.id, inboundMsgs, inboundBytes, reqT0, prepared!.body); return { body: prepared!.body, prepared: prepared! }; }; // #1884/#2090 (un-armed signed traffic): a request that already @@ -3146,7 +3144,9 @@ async function handleRequest( return next; } : undefined; - await forward(req, res, opts, pendingForward.body, pendingForward.prepared, core, reqConfig, log, route, instanceId, affinity, overflowRefold, v1Meta, throttleModel); + await forward(req, res, opts, pendingForward.body, pendingForward.prepared, core, reqConfig, log, route, instanceId, affinity, overflowRefold, v1Meta, throttleModel, (outbound) => { + logRequestCost(log, session.id, inboundMsgs, inboundBytes, reqT0, outbound); + }); // Remember for ALL modes (not just plugin): wire clients (dsh, // hermes, unplug'd pi) read the same panel via /__bili/plugin/status // and need the nudge/breakdown sections too; locked so a racing diff --git a/src/server/relay.ts b/src/server/relay.ts index 976ba5a33c..8be3b0981c 100644 --- a/src/server/relay.ts +++ b/src/server/relay.ts @@ -343,12 +343,14 @@ async function forwardCaptured( overflowRefold?: (realWindow: number | undefined) => Promise<{ body: string | Buffer; prepared: Prepared | null } | null>, v1Meta?: Record, throttleModelHint?: string, + onWireBody?: (body?: Buffer | string) => void, ): Promise { const forwardStartedAt = Date.now(); const reply = prepared && !prepared.sidePassthrough ? createHandoffReply(prepared.session, () => !res.destroyed) : undefined; // E2: a codex native-compaction request intercepted in prepare() carries a // forged success response — serve it without contacting upstream. if (prepared?.codexForge) { + onWireBody?.(); log("info", `[${prepared.session.id}] codex compact served locally (${prepared.codexForge.kind}); upstream not contacted`); if (!res.headersSent) res.writeHead(200, { "content-type": prepared.codexForge.contentType }); res.end(prepared.codexForge.body); @@ -989,11 +991,13 @@ async function forwardCaptured( log("debug", `[${prepared.session.id}] [thinking-posture] dropped ${dropped.removed} signature-less thinking block(s) before send (strict upstream learned for this session, #5998)`); } } + const requestBody = req.method === "GET" || req.method === "HEAD" ? undefined : v1Wrap(wireBody); const init: Omit & { dispatcher?: object } = { method: req.method ?? "GET", headers, - body: req.method === "GET" || req.method === "HEAD" ? undefined : v1Wrap(wireBody), + body: requestBody, }; + onWireBody?.(requestBody); if (dispatcher) init.dispatcher = dispatcher; // #1843 L1: capture this round's image facts at the SEND chokepoint so the // turn's usage settle pairs with the bytes actually sent — streamed turns @@ -1055,7 +1059,8 @@ async function forwardCaptured( // previous request's body. Loop re-fetches re-note their own rebuilt bodies // (loop/core.ts fetchUpstream); side requests never settle usage and must // not clobber the slot. - if (prepared?.session && !prepared.sidePassthrough && req.method !== "GET" && req.method !== "HEAD") { + const mainSend = prepared !== null && prepared.session !== undefined && !prepared.sidePassthrough && req.method !== "GET" && req.method !== "HEAD"; + if (mainSend && prepared?.session) { const sentBody = typeof wireBody === "string" ? wireBody : wireBody.toString("utf8"); // #2078: ONE parse of the outbound body for the whole seam block — the // estimate's projection, its wire-overhead term, and the image reserve diff --git a/src/server/request-cost.ts b/src/server/request-cost.ts new file mode 100644 index 0000000000..d3dfb69bd0 --- /dev/null +++ b/src/server/request-cost.ts @@ -0,0 +1,60 @@ +import { noteRequest } from "../attach-heartbeat.js"; +import { currentFetchTransport } from "../fetch-transport.js"; + +// #903: per-request local-cost line, logged at every point where bili finishes +// its own processing and hands the request off (upstream forward, forged local +// response, or fail-fast error) — see handle(). outbound is the final body value +// actually handed to fetch (string OR Buffer), measured inside forwardCaptured +// after compatibility rewriting, egress shedding, signing and V1 wrapping; wire +// bytes are counted with Buffer.byteLength since undici sends UTF-8 bytes even +// for string bodies (.length would count UTF-16 chars and undercount any +// non-ASCII injection). Omitted where no body is handed to fetch (local forge, +// fail fast). +export function formatBytes(n: number): string { + if (n < 1024) return `${n}B`; + if (n < 1024 * 1024) return `${(n / 1024).toFixed(1)}KiB`; + if (n < 1024 * 1024 * 1024) return `${(n / (1024 * 1024)).toFixed(1)}MiB`; + return `${(n / (1024 * 1024 * 1024)).toFixed(1)}GiB`; +} + +// #5744: the advisory budget is WALL CLOCK, never message COUNT — one reporter +// session cost 0.8s and 14.5s at the same ~3k msgs, so the count cannot separate +// cost from size (the info line still carries the count for correlation). +// #4528/#5481: this line reports measured facts only (count, inbound bytes, +// measured local time). It does not attribute the cost to event-loop starvation +// and does not advise restarting the session or enabling compression — neither +// follows from elapsed time alone, and a restart is a lossy action. +const SLOW_SESSION_MS_WARN = 1000; +// #5907 follow-up: the warn fires ONCE per session per process — a 3k-msg +// session exceeds the wall budget on EVERY turn, so re-logging each turn +// floods the log while adding nothing; the first occurrence carries the full +// size picture (msgs + inbound + est tokens) plus the cleanup command. The +// map is bounded: past the cap the earliest-warned session is evicted so a +// long-lived proxy with many sessions cannot grow it without bound. +const SLOW_SESSION_WARNED = new Map(); +const MAX_SLOW_SESSION_WARNED = 512; +export function _resetSlowSessionWarnedForTest(): void { + SLOW_SESSION_WARNED.clear(); +} + +export function logRequestCost(log: (level: string, msg: string) => void, sessionId: string, msgs: number | null, inboundBytes: number, t0: number, outbound?: string | Buffer): void { + const ms = Math.max(0, Math.round(performance.now() - t0)); + const outboundField = outbound !== undefined ? `, outbound=${formatBytes(Buffer.byteLength(outbound))}` : ""; + const view = currentFetchTransport() ? ", view=ws-expanded" : ""; + log("info", `[${sessionId}] request: ${msgs ?? "?"} msgs, inbound=${formatBytes(inboundBytes)}${outboundField}, local=${ms}ms${view}`); + // #5408: this line is THE observation point — a hot-at-attach session that + // never produces one is silent compression loss (the incident signature). + noteRequest(sessionId); + if (ms > SLOW_SESSION_MS_WARN && !SLOW_SESSION_WARNED.has(sessionId)) { + SLOW_SESSION_WARNED.set(sessionId, ms); + while (SLOW_SESSION_WARNED.size > MAX_SLOW_SESSION_WARNED) { + const oldest = SLOW_SESSION_WARNED.keys().next().value; + if (oldest === undefined) break; + SLOW_SESSION_WARNED.delete(oldest); + } + // bytes/4 is the repo-wide optimistic token floor (same caliber as the + // oversize gates) — labeled est because JSON overhead skews it high. + const estTok = Math.round(inboundBytes / 4); + log("warn", `[${sessionId}] slow-session: ${msgs ?? "?"} msgs / inbound=${formatBytes(inboundBytes)} (~${estTok} est tok) took local=${ms}ms — past the ${SLOW_SESSION_MS_WARN}ms wall-clock budget (#5744); measured facts only, cause not attributed — not evidence of event-loop starvation (#4528); the session has grown large — consider \`bili sessions rm ${sessionId}\` (#5907)`); + } +} diff --git a/src/web/sessions-data.ts b/src/web/sessions-data.ts index 1cf04f9117..aec78bc19a 100644 --- a/src/web/sessions-data.ts +++ b/src/web/sessions-data.ts @@ -38,7 +38,7 @@ import * as path from "node:path"; // the model, not the host's own history — so per-step cost climbs with it. On // plugin lanes (~2 host messages per request, dsh-measured) that becomes // visible around 10k messages ≈ 5k requests. The server log warns much earlier -// (SLOW_SESSION_MSG_WARN = 1000 messages, src/server.ts); this display gate is +// (SLOW_SESSION_MS_WARN = 1000ms local-processing budget, src/server/request-cost.ts); this display gate is // deliberately higher — the badge should mean "worth starting a new session", // not "notable". const SLOW_SESSION_REQUESTS_WARN = 5000; diff --git a/tests/issue4528-event-loop-yield.test.ts b/tests/issue4528-event-loop-yield.test.ts index 67a090f52b..2d158c66d6 100644 --- a/tests/issue4528-event-loop-yield.test.ts +++ b/tests/issue4528-event-loop-yield.test.ts @@ -2,7 +2,7 @@ import { describe, it } from "node:test"; import assert from "node:assert/strict"; import { performance } from "node:perf_hooks"; import { yieldToEventLoop } from "../src/yield-loop.js"; -import { logRequestCost, _resetSlowSessionWarnedForTest } from "../src/server.js"; +import { logRequestCost, _resetSlowSessionWarnedForTest } from "../src/server/request-cost.js"; // #4528: a large resent history makes bili run many O(messages) synchronous // passes back-to-back on Node's single main thread, starving every other @@ -43,27 +43,66 @@ describe("#4528 yieldToEventLoop hands control to the poll phase", () => { }); }); +// #5481: the wording IS the fix. The old advisory fired on inbound message +// COUNT (#4528) and asserted that per-turn synchronous work "is starving the +// event loop", advising a restart/compression — while the same line reported +// tens of milliseconds of local preparation. #5744 replaced the trigger with +// measured wall clock (count cannot separate cost from size: one reporter +// session cost 0.8s and 14.5s at the same ~3k msgs), so these tests pin the +// wall-clock trigger AND the factual, non-attributing wording together. describe("#4528 logRequestCost slow-session guard", () => { function capture(): { lines: Array<[string, string]>; log: (level: string, msg: string) => void } { const lines: Array<[string, string]> = []; return { lines, log: (level, msg) => lines.push([level, msg]) }; } + const warns = (lines: Array<[string, string]>) => lines.filter(([l]) => l === "warn"); - it("warns when the inbound history has grown past the message threshold", () => { + it("warns once past the wall-clock budget, reporting measured facts only", () => { const { lines, log } = capture(); - logRequestCost(log, "sess-big", 1500, 11_000_000, performance.now() - 8000); - const warns = lines.filter(([l]) => l === "warn"); - assert.equal(warns.length, 1); - assert.match(warns[0][1], /slow-session: 1500 msgs/); - assert.match(warns[0][1], /inbound=/); - assert.match(warns[0][1], /local=\d+ms/); + logRequestCost(log, "sess-slow", 1500, 11_000_000, performance.now() - 8000); + const w = warns(lines); + assert.equal(w.length, 1); + assert.match(w[0][1], /slow-session: 1500 msgs/); + assert.match(w[0][1], /inbound=/); + assert.match(w[0][1], /local=\d+ms/); + // Load-bearing: revert to the attributed wording and this goes red. + assert.match(w[0][1], /not evidence of event-loop starvation/); + assert.doesNotMatch(w[0][1], /starving the event loop/); + assert.doesNotMatch(w[0][1], /restart|enabling compression/); + assert.ok(lines.some(([l, m]) => l === "info" && m.includes("sess-slow") && m.includes("1500 msgs"))); }); - it("does not warn for a healthy-sized session", () => { + it("does not warn for a fast request, however large the history (#5744)", () => { const { lines, log } = capture(); logRequestCost(log, "sess-small", 500, 100_000, performance.now() - 40); - assert.equal(lines.filter(([l]) => l === "warn").length, 0); - assert.ok(lines.some(([l, m]) => l === "info" && m.includes("sess-small") && m.includes("500 msgs"))); + logRequestCost(log, "sess-big-fast", 1500, 11_000_000, performance.now() - 40); + assert.equal(warns(lines).length, 0); + assert.ok(lines.some(([l, m]) => l === "info" && m.includes("sess-big-fast") && m.includes("1500 msgs"))); + }); + + it("warns for a tiny history that is slow — the trigger is wall clock, not count (#5744)", () => { + const { lines, log } = capture(); + logRequestCost(log, "sess-tiny-slow", 13, 900_000, performance.now() - 39_000); + const w = warns(lines); + assert.equal(w.length, 1); + assert.match(w[0][1], /slow-session: 13 msgs/); + assert.match(w[0][1], /local=\d+ms/); + }); + + it("reports an unknown message count without dropping the slow warn", () => { + const { lines, log } = capture(); + logRequestCost(log, "sess-unknown", null, 1_000_000, performance.now() - 39_000); + const w = warns(lines); + assert.equal(w.length, 1); + assert.match(w[0][1], /slow-session: \? msgs/); + assert.ok(lines.some(([l, m]) => l === "info" && m.includes("? msgs"))); + }); + + it("does not warn for a fast request with an unknown message count (passthrough lane)", () => { + const { lines, log } = capture(); + logRequestCost(log, "sess-unknown-fast", null, 1_000_000, performance.now() - 5); + assert.equal(warns(lines).length, 0); + assert.ok(lines.some(([l, m]) => l === "info" && m.includes("? msgs"))); }); it("warns only ONCE per session and carries the size picture plus the cleanup command (#5907)", () => { diff --git a/tests/issue903-request-cost-log.test.ts b/tests/issue903-request-cost-log.test.ts index 5791b42060..fb63129da9 100644 --- a/tests/issue903-request-cost-log.test.ts +++ b/tests/issue903-request-cost-log.test.ts @@ -11,6 +11,7 @@ import type { ProxyOptions } from "../src/config.ts"; import { SessionStore, _setStoreForTest } from "../src/persist.ts"; import { _setForTest as setRegistryForTest } from "../src/registry.ts"; import { setLogCapture } from "../src/logger.ts"; +import { formatBytes } from "../src/server/request-cost.ts"; // #903: long sessions keep resending the full harness history to bili even // when the folded context stays small — the per-request cost line makes that @@ -36,21 +37,13 @@ function inboundBytesOf(line: string): number { return Math.round(v * mult); } -function outboundBytesOf(line: string): number { - const m = line.match(/outbound=(\d+(?:\.\d+)?)(B|KiB|MiB|GiB)/); - assert.ok(m, `no outbound field in: ${line}`); - const v = Number(m![1]); - const mult = m![2] === "B" ? 1 : m![2] === "KiB" ? 1024 : m![2] === "MiB" ? 1024 ** 2 : 1024 ** 3; - return Math.round(v * mult); -} - test("e2e: per-request cost line reports msg count, inbound wire size and local duration", async () => { - let upstreamSeen = 0; + const upstreamBodies: Buffer[] = []; const upstream = http.createServer((req, res) => { const chunks: Buffer[] = []; req.on("data", (c: Buffer) => chunks.push(c)); req.on("end", () => { - upstreamSeen = Buffer.concat(chunks).length; + upstreamBodies.push(Buffer.concat(chunks)); res.writeHead(200, { "content-type": "application/json" }); res.end(okJson()); }); @@ -79,7 +72,7 @@ test("e2e: per-request cost line reports msg count, inbound wire size and local chainContentDetection: false, autoUpdate: false, mitm: { enabled: false, domains: [] }, - compat: { roles: {} }, + compat: { roles: { assistant: "rewritten-assistant-role-with-a-deliberately-long-name" } }, streamErrorShape: "protocol", passthroughSource: null, autoRestartOnUpdate: false, @@ -114,13 +107,15 @@ test("e2e: per-request cost line reports msg count, inbound wire size and local const r1 = await fetch(url, { method: "POST", headers, body: first.raw }); assert.equal(r1.status, 200); await r1.text(); - const up1 = upstreamSeen; + assert.equal(upstreamBodies.length, 1); + const up1 = upstreamBodies[0]!.length; const second = mkBody(3); const r2 = await fetch(url, { method: "POST", headers, body: second.raw }); assert.equal(r2.status, 200); await r2.text(); - const up2 = upstreamSeen; + assert.equal(upstreamBodies.length, 2); + const up2 = upstreamBodies[1]!.length; // Blocked side request (#554 gate): tiny output budget + raw-history // estimate reaching the effective window -> 413 locally, upstream never @@ -138,12 +133,12 @@ test("e2e: per-request cost line reports msg count, inbound wire size and local const l1 = costLines[0]!; assert.match(l1, /^\[cost-sess\] request: 5 msgs, inbound=\d+(?:\.\d+)?(B|KiB|MiB|GiB), outbound=\d+(?:\.\d+)?(B|KiB|MiB|GiB), local=\d+ms$/); assert.ok(Math.abs(inboundBytesOf(l1) - Buffer.byteLength(first.raw)) <= 100, `inbound ${l1} vs sent ${Buffer.byteLength(first.raw)}`); - assert.ok(Math.abs(outboundBytesOf(l1) - up1) <= 100, `outbound ${l1} vs upstream-received ${up1}`); + assert.ok(l1.includes(`outbound=${formatBytes(up1)}`), `outbound ${l1} vs upstream-received ${up1}`); const l2 = costLines[1]!; assert.match(l2, /^\[cost-sess\] request: 7 msgs, inbound=\d+(?:\.\d+)?(B|KiB|MiB|GiB), outbound=\d+(?:\.\d+)?(B|KiB|MiB|GiB), local=\d+ms$/); assert.ok(Math.abs(inboundBytesOf(l2) - Buffer.byteLength(second.raw)) <= 100, `inbound ${l2} vs sent ${Buffer.byteLength(second.raw)}`); - assert.ok(Math.abs(outboundBytesOf(l2) - up2) <= 100, `outbound ${l2} vs upstream-received ${up2}`); + assert.ok(l2.includes(`outbound=${formatBytes(up2)}`), `outbound ${l2} vs upstream-received ${up2}`); assert.ok(inboundBytesOf(l2) > inboundBytesOf(l1), "growing history must show up as growing inbound"); const l3 = costLines[2]!;