diff --git a/.github/workflows/ci-gate-plugins.yml b/.github/workflows/ci-gate-plugins.yml index 6a14dc01c8..3eaad84aaf 100644 --- a/.github/workflows/ci-gate-plugins.yml +++ b/.github/workflows/ci-gate-plugins.yml @@ -8,7 +8,7 @@ on: workflow_dispatch: inputs: agents: - description: "comma-separated lanes: dsh,pi,codex,opencode,claude" + description: "comma-separated lanes: dsh,pi,codex,opencode,claude (+ dsh matrix/proxy #3475: dsh-responses,dsh-anthropic,dsh-google,dsh-proxy)" required: false default: "dsh,pi,codex,opencode,claude" diff --git a/.gitignore b/.gitignore index 7e2336bcda..799e541967 100644 --- a/.gitignore +++ b/.gitignore @@ -7,3 +7,4 @@ dist/ tmp/ tests/.pp-stamp-state-*/ .tmp/ +plugin-lanes/ diff --git a/CHANGELOG.md b/CHANGELOG.md index d543aac4b3..d25fcf0e73 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,12 +7,17 @@ Versions follow the merge of a `*_release-v*` branch; CI publishes to npm on tag ### Changes +- **Egress image downscale + `read_original_image` (#3400).** Redoes the image pre-compression removed in #3295 in simplified form — **behavior change**: since #3295 every image passed through byte-identical at original resolution; from this version on, large outbound images are sent SHRUNKEN by default. On every outbound request (after kernel folding, before protocol encoding, all protocols share one pass) an image whose longest edge exceeds 1600 px (`BILI_IMAGE_RESCALE_MAX_DIM`) or whose base64 body exceeds 100 KB is re-encoded fit-inside (never upscaled, original format kept) on an OUTBOUND COPY only — session storage keeps the original bytes byte-exact. A re-encode saving less than 20% of the bytes is discarded (keep-smaller), so small and low-benefit images pass through untouched; only PNG/JPEG/BMP are handled (pure-JS jimp encoder, zero native dependencies — sharp does not return). Each shrunken image carries a stable annotation beside it reporting current vs original size; lanes that register bili tools (proxy-injection lanes, plugin lanes via the plugin manifest) also get the tool named in the annotation — `read_original_image("mNNNNN")` returns the original bytes from local session storage — while tool-rejecting lanes (codex) get size-only notes and no tool. Tool results are structurally exempt from re-shrinking (zero-state loop guard against the #3266 guess-reject-resend disease). Output is deterministic (content-hash cached, one encode per unique image; same history → byte-identical outbound bodies) so upstream prefix caching is undisturbed. Observability: one log line per unique image (dims+bytes before/after) and one per original-image read (ref, sizes, session cumulative count+bytes) — no hard cap, the model was told the sizes. Knobs: `BILI_IMAGE_RESCALE` (`"0"`/`"false"` off; default on), `BILI_IMAGE_RESCALE_MAX_DIM` (default 1600, env-only), config-file key `imageRescale`. Legacy `compress.imageCompression.*` INERT warnings remain owned by #3344/#3355. +- **Lane-decisions 收尾:四处 B 类旁路迁入统一决策点 + nudgeOn 单点(#3482,承接 #3417)。** ①omp 身份戳通道(`stampPromptCacheKey` 的 `agent !== "omp"` 门)改为注册表属性 `stampsBodyPromptCacheKey`;OMP Cowork 直连传输的"是否可能"进注册表(`coworkTransportPossible`),逐请求真相仍由环境观察(`ompCoworkTransportBypassesFetch`)给出。②codex/claude-responses 的 instructions 指纹键控从散落头名比较收进 `WIRE_CLIENTS` 注册表(`instructionsFingerprintKeyed`;信号检测留在 session-id,决策表只在注册表)。③计费块(#2189)提取改由 `laneDecisions.billingBlockWanted` 门控:profile 声明(dsh)或 wire 观察到块任一成立即提取——证据优先,与 toolsReachable 同形状;零 wire 字节变更。④四份手抄 `willInjectNudge` 合取收敛为 `nudgeOn()` 九入参纯函数:responses 的两处有意差异(compaction-trigger 豁免、codex-intercept 健康窗)显式入参,openai/google 的冗余 `shouldInject` 合取被 `compressSurfaceOn` 吸收;1024 点布尔立方等价测试 + 四车道旧公式逐位对照钉死。结构守卫扩展:B 类站点禁裸 lane 名门控,`WIRE_CLIENTS` 注册表条目必须有 `matches()`。⑤HostProfile 注册表整体落在零导入叶子模块 `src/host-profiles.ts`(review 修正:首版让 agent 侧直接 import lane-decisions,preflight→kernel 整链被拖进 dist/agent/pi.js、omp.js,两个宿主扩展包体积翻倍;拆出后两者回到原尺寸,结构守卫钉死该模块必须无 import)。`wantsBillingBlock` 死属性落位为真实消费者(上述③)。dshLedgerFloorTokens 本无 lane 名查(issue 前提修正:#1729 守卫故意 identity-blind,#2193)——三处 ledger floor 调用点保持不动并在 PR 说明。 - **Lane decisions extracted to a single resolver (#3417, fixes #3401).** The four wire adapters each carried their own copy of the lane gates (`dshCompressSurfaceOff`, the dsh compact-rebase guard), and the copies drifted: a dsh profile that DOES declare `compress` on the wire still had the model-side compress surface dropped (#3401). New `src/server/lane-decisions.ts`: a `HostProfile` registry (the only place lane names are compared), per-request `WireObservation` (which bili tools this request actually declares), and `resolveLaneDecisions()` — the one gate the four prepares now consume. Observed reachability overrides the dsh host-scope default, so tool-carrying dsh profiles keep the nudge/standing prompt while bare dsh stays off (#3177 preserved). `biliToolsDeclaredOnWire` gained a named-tool variant (the nudge advertises `compress` specifically). Structural guard test pins that prepares carry no raw lane-name comparisons. The dsh native-compaction rebase gate on all four wires now reads the registry property `compactsClientSide`. - **Config epoch: stale effectiveConfig stamps are refused (incident class tracked as ⑥ on #2960).** Every wire request stamps the session's resolved config snapshot (`metadata.effectiveConfig`) — but that snapshot was only refreshed on the NEXT wire request, so tool-path readers (plugin tool API, status panels, nudge re-anchor) executed settings the user had already deleted from the config file, and with several instances on one machine a session could carry a foreign instance's stamp for days (the October externalSummary incident: a user-disabled summary chain kept running). The snapshot is now stamped with `metadata.effectiveConfigEpoch` — a content hash of the config file at resolution time. Readers refuse the snapshot when the epoch no longer matches the live file and re-resolve the file tier (global `compress.*`, including `externalSummary`) onto the base config instead; provider/model cascade levels are re-stamped by the session's next wire request. Epochs are content-keyed (byte-identical files validate cross-instance), legacy snapshots without an epoch keep the old merge semantics, forks inherit the epoch key, and the golden drives pin the new metadata digest. - **Fix: the Vertex managed block no longer wraps its own value.** `bili install claude` pins `ANTHROPIC_VERTEX_BASE_URL`, and the next derivation read that value back — the settings `env` block reaches the client process, which `readClaudeSettings` falls back to — wrapped it again, and added a `/bili/anthropic/` layer per launch. The extra hop targets the proxy's own port, which the tunnel guard denies (`403 detail:"self"`), so the lane died on the second start. Both derivations (`claudeNativeVertexBaseUrl` in `src/plugin-install.ts`, `claudeVertexUpstream` in `src/launcher.ts`) now collapse any nesting of bili's own wrapper first (`stripBiliWrappers`, `src/vertex.ts`) and are idempotent on their own output; a user's own endpoint behind a wrapper is preserved intact. The same pass closes an install/remove asymmetry: `bili plugin remove claude` now takes `ANTHROPIC_VERTEX_BASE_URL` back when it is ours (a foreign endpoint is left alone, as at install). - **Claude Code on Vertex AI (`CLAUDE_CODE_USE_VERTEX=1`) is routed through bili.** Under Vertex the plain `ANTHROPIC_BASE_URL` is inert — the client dials `/projects/…:rawPredict|streamRawPredict` instead — so the launcher, the managed native install, the upstream-discovery pass and the client config all gained that key: `bili claude` now sets `ANTHROPIC_VERTEX_BASE_URL` to `/bili/anthropic//v1`, where the protocol marker is required (`:rawPredict` matches no path suffix, so the suffix ladder cannot classify it) and the `/v1` is required because the client appends its own path starting at `/projects/…`; the endpoint host is derived from `ANTHROPIC_VERTEX_BASE_URL`/`CLOUD_ML_REGION` with the region validated against a strict pattern, host and `locations/` moving together; a dated model id (`claude-opus-5-5@…`) is normalized before the window lookup. The plain key stays set and untouched (inert under Vertex); with Vertex off nothing changes and nothing is emitted. Grounded in a captured Claude Code 2.1.300 wire (`tests/fixtures/vertex-anthropic/`, `tests/vertex-wire-fixture.test.ts`) plus a lane-scoped upstream fake (`tests/wire-contract-fakes.ts` rules `WC-019`/`WC-020`/`WC-021`, `tests/vertex-fake-upstream.test.ts`). - **Anthropic Threads lane: mirror + reforge (#2926, #2884).** `thread` bodies (`create`/`continue`) hold conversation state server-side: a continue wire body is delta-only no matter how big the thread grew, so bili's history-rewrite lever does not exist on this wire and the compress machinery now stands down for these turns entirely (byte-faithful passthrough, no preflight — the wire body says nothing about the real thread size). Instead every turn is mirrored into the unified store (`session_ext` keys `threads:mirror:*`: create replaces the mirror — it is the client's authoritative snapshot — continue appends its delta; oversized entries are elided, not stored as media). When the mirror estimate crosses the window on a continue and the rollout feature `anthropicThreadsReforge` is on (default 0%; force with `rollout.features.anthropicThreadsReforge: true` or `BILI_ROLLOUT_ANTHROPIC_THREADS_REFORGE=1`), bili swaps that continue for a fresh `create` carrying [summary of the folded prefix, verbatim tail ring (last 3 user turns), this turn's delta] — the only channel that can deliver a fold to a server-side thread. The client's system/tools are copied byte-faithfully from the captured create shape (host instructions never enter the summary); one summarizer side-call rides the client's auth and the same upstream; because the message-id chain IS the thread identity, the client's next continue lands in the new thread with zero redirection. Fail-open everywhere: legacy-only storage (no unified sidecar) captures nothing and never reforges; a failed summary side-call forwards the continue untouched. The lane also claims turns before the #2648 global-scope stand-down — the stand-down used to swallow whole Claude Code threads sessions and silently disable folding (#2884). -- **Stale-guard ownership fast path (#3049):** in shadow/full modes the per-save rollback guard no longer re-reads and re-parses the entire session envelope (the ~320 ms/turn, full-file read on >40 MiB sessions — a large slice of #3049's read amplification). The builder now carries an ownership token (the two monotonic counters it just ingested, plus its build time); when the unified row still matches those counters and the envelope file's mtime sits inside the build window, no other writer can have committed and the guard skips the read. Cross-instance commits are still caught through the shared `index.db` row, foreign file writes (legacy sibling, manual edit) through the mtime check, and legacy-only installs keep the exact per-save full read. Deliberate residual (documented on `ownershipFastPath`): a sibling whose sidecar is disabled writing within the ~1.5 s grace window is missed for one cycle — strictly narrower than an unconditional size/mtime skip. +- **Absorb distillation restored and scoped to image-bearing results by default (#3402, reverts #3309).** The `compress.absorb` block returns. New `compress.absorb.mediaOnly` (default `true`) restricts absorption to tool results that carry media (images): image bytes are not counted in text tokens, so such results are exempt from the `minToolTokens` floor and get marked `[ACP absorb]` even when small, while plain-text large results are no longer marked by default — this removes the old "always over-compresses" failure mode that motivated the original removal. Set `mediaOnly: false` to restore the pre-#3309 full-text behavior. `absorb.enabled` still defaults to `false` (the feature stays opt-in); once enabled, an absorbed result's call+result pair leaves subsequent context and its reclaimed tokens are credited against the compression budget. The plugin-lane prompt/tool gate is pinned consistent (the absorb prompt appears ⟺ the tool is advertised, both driven by the same base config block — #3293). Kernel `acp-kernel` 0.0.128 → 0.0.129 (new optional `mediaOnly` field; wire-contract goldens unchanged). +- **Absorb distillation removed (#3309).** The opt-in `compress.absorb` block (instant distillation of large tool results into short abstracts, originals to the content store) and its `absorb` tool were deleted from host and kernel. *(Backfill: #3309 shipped without a changelog line; this entry was added when the change was reverted. Superseded by the entry above.)* +- **Output-budget restore never exceeds the session's proven budget (#3406 item 4).** When no output ceiling is known for a model (no runtime-info, launcher, config, or registry row) and a starved main request must be restored, the #3071 absolute floor (32768) now applies ONLY when the session has no functional high-water of its own (born-starved / fork-inherited rattle, or a high-water below the 4096 trust line). A high-water at/above the trust line is the budget the provider already accepted at full value — restoring TO it instead of raising it to 32K fixes the hard-rejection on providers that cap `maxOutputTokens` below 32768 (the raised value 400'd outright, trading a degraded-but-working session for a failing one). The #3071 protections are unchanged: fresh sessions and death-rattle decay chains (…, 2061, 252) still get the floor, a known ceiling still floors the target, and non-starved values still pass through verbatim. +- **Stale-guard ownership fast path (#3049, hardened in #3406 item 2):** in shadow/full modes the per-save rollback guard no longer re-reads and re-parses the entire session envelope (the ~320 ms/turn, full-file read on >40 MiB sessions — a large slice of #3049's read amplification). The builder carries an ownership token (the two monotonic counters it just ingested); when the unified row still matches those counters and the envelope file is byte-identical to the stat stamped right after OUR write landed (same size, mtime not newer beyond sub-ms clock skew), no other writer can have committed and the guard skips the read. Cross-instance commits are still caught through the shared `index.db` row; foreign file writes (legacy sibling, manual edit) through the size/mtime stamp mismatch — now regardless of WHEN they land, closing the #3406 item-2 race where a sibling writing inside the old builtAtMs+1.5 s grace window was adopted as "ours" and clobbered on the next save. The debounce is host-owned so EVERY envelope write — the per-turn debounced save or an explicit one — gets stamped when it lands; a save whose write never landed observably (restore-replica seeds, crash before stamping) leaves the token unstamped and always takes the full read. Legacy-only installs keep the exact per-save full read. Residual (documented on `ownershipFastPath`): a foreign write landing in the microsecond gap between our awaited write resolving and the stamp's statSync can be adopted as ours for one cycle — the window shrank from 1.5 s to ~µs. - **Image pre-compression (`compress.imageCompression`) and the `image_full` tool removed (#3266).** The opt-in downscale pass and its restore channel are gone from host and kernel: no more arrival-time shrinking, no `image_full` injection, no `[Downscaled screenshots]` notes, no image stats — `sharp` is no longer an optional dependency. Rationale: the restore loop was structurally unreachable on hosts that reject bili-injected tools (codex), and where reachable the model had no reliable way to learn the right ref, producing the 38-round guess-reject-resend loop in #3266 (cache hit 75%→28%). Every image now passes through byte-identical at original resolution. Sessions persisted by older versions load unchanged; their legacy `imageShrinks`/`imageFullRestored` state keys and `imageShrunk*` stats are ignored on load (never materialize). A future redesign is tracked on the issue. - **The proxy now carries Claude Code on Vertex: the wire is classified and the model comes off the publisher path.** A Vertex turn arrives at `/bili/anthropic/https://aiplatform.googleapis.com/v1/projects/…/publishers/anthropic/models/:rawPredict`, whose body carries `anthropic_version` and no `model` field at all — so the model is read from the path (`vertexAnthropicModelFromPath`, a sibling of the Gemini parser so its `:countTokens` arm cannot start matching Anthropic paths), normalised once (`claude-…@YYYYMMDD` → bare publisher id, logged per id), and handed to every model-keyed decision — window resolution, compress thresholds, price profile, fallback logging — through one resolution point (`bodyModel ?? googleModel ?? vertexPathModel ?? v1Model`, body first, so the non-Vertex anthropic lane is byte-identical) that also rides on the prepared request into the relay's compress-loop rebuild. `aiplatform.googleapis.com` and both location-shaped host families now map to the `anthropic` provider; the incremental-lane parameter allowlist keeps `anthropic_version` (`src/incremental-lane.ts` and its mirror `src/agent/incremental-client.ts`) for native-extension hosts; and `bili doctor` reports `bypassed` on a Vertex session instead of under-reporting one. No GCP access was needed — acceptance is the captured fixture replayed at the emitted path against a mock upstream (`tests/e2e-vertex-anthropic.test.ts`), and the same request without the protocol marker is still forwarded verbatim. - **Incremental marker divert + native client helpers (#3248 phase 2).** A native extension can now send incremental turns on its ORDINARY model requests: stamp `x-bili-plugin-turn: 1` (requires the usual `x-bili-plugin`) and post the incremental payload — the body shape (top-level `conversationId`/`sinceSeq`/`messages`, no top-level `model`) is the final word, so a marker on a non-incremental body rides the normal pipeline unchanged and no provider URL changes. The lane forwards to the ORIGINAL request path; `GET /__bili/plugin/turn` probes capability (`{ok, protocol: 1, feature, enabled, storage}`). Gated by the rollout feature `pluginIncrementalTurns` — default 0% keeps every install byte-identical; force with `rollout.features.pluginIncrementalTurns: true` or `BILI_ROLLOUT_PLUGIN_INCREMENTAL_TURNS=1` (still requires a live unified sidecar). Seeds imply `resync: true` (a first incremental turn never 409s, and interop full-resend traffic is replaced client-authoritatively); a system-only change rides an empty delta. Reference wirings: pi (`before_provider_headers`/`before_provider_request`/`after_provider_response`) and dsh-native (fetch patch `beforeSend`/`headersFor`/`onResponse`), both on the shared state machine in `src/agent/incremental-client.ts` (probe TTL cache, prefix-hash diffing, sticky system, per-conversation dead flag) — new hosts should reuse it. Follow-up wiring ships the remaining native hosts: pi-native inherits the pi wiring unchanged (same extension factory), opencode rides its `http.request` hook (adapter callback swaps the request body after routing/stamping, acks on the fetch intercept), and omp — which has no header events — reads the extension-stamped `prompt_cache_key` body field as the conversation id inside `beforeSend` and stamps the plugin headers directly on the request init (identity upgrade, not a fork). Non-Anthropic wire shapes (OpenAI `tool_calls`) are detected and left on full resend; planned turns are single-in-flight per URL and age out after 30s. diff --git a/CLIENTS.md b/CLIENTS.md index dc99059109..9fb654fb62 100644 --- a/CLIENTS.md +++ b/CLIENTS.md @@ -17,8 +17,10 @@ Two lanes, same plugin (#941): `--patch` overlay (`~/.dsh-bili/.bili-acp.patch.yml`) — every profile boots with the bili tools registered natively, model requests carry `x-bili-plugin` + the dsh session id (plugin mode), and `/acp` is - session-bound. dsh's native auto-compaction is disabled in the same patch - (`compaction-basic` → `auto: false`); manual `/compact` stays available. The + session-bound. dsh's native auto-compaction is disabled in the same patch + (host-plane `compaction-basic` → `auto: false`; web-profile preset + instances are disabled at load time by the #3354 overlay); manual + `/compact` stays available in non-web profiles. The same patch also adds `MALFORMED_RESPONSE` to the built-in DeepSeek routes' retry whitelist, so one sampling flake that breaks a tool-call argument JSON retries instead of killing the turn (#2605). @@ -90,25 +92,34 @@ Two lanes, same plugin (#941): settings overlay rewrites those providers' `baseURL`s to `/bili/` URLs, so the traffic reaches the proxy regardless of which fetch the transport uses or what the attribution state is. - - **Web-profile caveat (#1772, corrected by #2474):** when a profile's - bundles include `@deepseek-ai/dsh-web-app`, the running `compaction-basic` - instance lives inside an agent preset (`preset-standard.config.plugins`). - No patch layer can address that NESTED row by id — dsh's patch engine - indexes only top-level rows and true group children — so the bundled - `auto: false` lands on web-app's already-disabled host-plane row and the - preset instance keeps auto-compaction ON. But `preset-standard` itself is - an ordinary bundle-layer entry: a profile-level patch (your profile's - `cordis.patch.yml`, applied after every bundle) overrides it by id with a - FULL-SNAPSHOT `config` — copy the complete preset config from - `dsh --profile --dump-config`, add `config.auto: false` under its - `compaction-basic` row, append the result to `cordis.patch.yml`. Config - replacement is WHOLESALE: a partial snippet silently drops everything not - restated — including the preset's own `id`/`order` identity, which can - leave the preset unrecognized by the roster — and `- insert:` with the - same id appends a second dead row instead of merging. The snapshot also - freezes the roster until re-copied after a dsh preset update. The plugin - logs a one-time `[dsh-client]` warning naming this recipe at boot in such - profiles; ACP compression is unaffected. + - **Web-profile compaction (#1772, resolved by #3354):** when a profile's + bundles include `@deepseek-ai/dsh-web-app`, the running `compaction-basic` + instance lives inside an agent preset (`preset-standard.config.plugins`). + No patch layer can address that NESTED row by id — dsh's patch engine + indexes only top-level rows and true group children — so the bundled + `auto: false` lands on web-app's already-disabled host-plane row and + changes nothing. Since #3354 bili closes the gap at LOAD TIME: the + `bili-native` bundle entry registers a cordis `internal/config` overlay + (vendored from deshark42/dsh-compaction-basic-disable, MIT) that marks + the preset's `compaction-basic` and `command-compact` rows + `disabled: true`, and the preset gate rows in the bundle patch make each + Web preset wait for that overlay before it can register. Native + auto-compaction is OFF in these profiles too (and `/compact` is + unavailable there); ACP compression is unaffected. The overlay rides an + INTERNAL dsh hook and is version-sensitive — if you still see a dsh + compaction checkpoint land on a newer/older dsh build, fall back to the + #2474 recipe: a profile-level patch (your profile's `cordis.patch.yml`, + applied after every bundle) overrides preset-standard by id with a + FULL-SNAPSHOT `config` — copy the complete preset config from + `dsh --profile --dump-config`, add `config.auto: false` under its + `compaction-basic` row, append the result to `cordis.patch.yml`. Config + replacement is WHOLESALE: a partial snippet silently drops everything not + restated — including the preset's own `id`/`order` identity, which can + leave the preset unrecognized by the roster — and `- insert:` with the + same id appends a second dead row instead of merging. The snapshot also + freezes the roster until re-copied after a dsh preset update. The plugin + logs a one-time `[dsh-client]` note naming both paths at boot in such + profiles. - **Global copy shadows the profile copy at load time (#2891):** dsh resolves the bare `billion-context` name in `dsh.profile.bundles` installation- anchor-first — from its own install tree (the global npm root) before the @@ -128,8 +139,9 @@ spawn mode (a loopback proxy target is never proxied, so the MITM envs are simply bypassed); already-routed `/bili/`-prefixed requests pass through untouched except for header stamping. Known limitation: manual `/compact` has no dsh-side event hook, so its boundary is left to the -kernel's natural ingest diff (auto-compaction is off in non-web profiles, so -this is rare; see the #1772 caveat above for web profiles). +kernel's natural ingest diff (auto-compaction is now off everywhere — host +plane via the bundle patch, web-profile presets via the #3354 load-time +overlay; see the web-profile note above). **Host-side context meters diverge from the model's view by design (#2837):** in plugin mode the host's durable session log @@ -536,9 +548,10 @@ the four have a base-URL env hook; one doesn't: override (verified in the v2.19.1 binary: *"Overriding CloudCodeServerURL via CLOUD_CODE_URL environment variable"*) — same shape as the gemini-cli base-URL hook. The launcher sets - `CLOUD_CODE_URL=/bili/` (default upstream - `https://cloudcode-pa.googleapis.com`; if you export your own - `CLOUD_CODE_URL`, that value is relayed through the proxy instead). No MITM, + `CLOUD_CODE_URL=/bili/` (default upstream + `https://daily-cloudcode-pa.googleapis.com` — Antigravity's real default; + the bare `cloudcode-pa` host 400s on `/v1internal:*`, #3129; if you export + your own `CLOUD_CODE_URL`, that value is relayed through the proxy instead). No MITM, no CA install. The proxy recognizes the wire **by path**: `:streamGenerateContent` / `:generateContent` / `:countTokens` requests are routed to the Google-native adapter and compressed like `bili gemini`. If @@ -546,9 +559,11 @@ the four have a base-URL env hook; one doesn't: (`/google.internal.cloud.code.v1internal.CloudCode/*`), those requests are not recognized and relay verbatim without compression — a graceful degrade, fixable with a per-lane `protocol` declaration (#1909) once confirmed. - Fallback if Google ever removes the env hook: cert-MITM — the - language_server honors `HTTPS_PROXY` and does not pin certificates, so add - `cloudcode-pa.googleapis.com` to `"mitm".domains` and trust bili's root CA. + Fallback if Google ever removes the env hook: cert-MITM — the + language_server honors `HTTPS_PROXY` and does not pin certificates, so add + `daily-cloudcode-pa.googleapis.com` (Antigravity's real default endpoint; + the bare `cloudcode-pa` name 400s on `/v1internal:*`, #3129) to + `"mitm".domains` and trust bili's root CA. The desktop app and the IDE extension share this same language_server channel; the launcher drives the CLI specifically — desktop users can export `CLOUD_CODE_URL` manually or use the MITM recipe. @@ -788,7 +803,7 @@ file (2.x rejects bare file paths in the config `plugin` array); **1.x** hosts get the bare file path. What the plugin does (both generations): registers the bili tools natively -in-host — compress / decompress / search_context / acp_status — +in-host — compress / decompress / search_context / acp_status (+ absorb) — and stamps the proxy headers on every outgoing provider request, including context-window / max-output read from the host's own model catalog (`ctx.catalog.model.list()`, refreshed every 60s) and reported to the proxy diff --git a/CLIENTS.zh-CN.md b/CLIENTS.zh-CN.md index 34d63eb8c9..729a4fe007 100644 --- a/CLIENTS.zh-CN.md +++ b/CLIENTS.zh-CN.md @@ -9,15 +9,15 @@ 两条通道,同一个插件(#941): -- **启动器:** `bili dsh` 经 `--patch` overlay(`~/.dsh-bili/.bili-acp.patch.yml`)注入完整原生插件 —— 每个 profile 启动即注册 bili 工具,模型请求盖 `x-bili-plugin` + dsh 会话 id(plugin 模式),`/acp` 会话绑定。同一份 patch 同时禁用 dsh 原生自动压缩(`compaction-basic` → `auto: false`);手动 `/compact` 仍可用。 +- **启动器:** `bili dsh` 经 `--patch` overlay(`~/.dsh-bili/.bili-acp.patch.yml`)注入完整原生插件 —— 每个 profile 启动即注册 bili 工具,模型请求盖 `x-bili-plugin` + dsh 会话 id(plugin 模式),`/acp` 会话绑定。同一份 patch 同时禁用 dsh 原生自动压缩(宿主平面 `compaction-basic` → `auto: false`,web profile 的 preset 实例由 #3354 加载期 overlay 关闭);非 web profile 下手动 `/compact` 仍可用。 - **Profile 安装(免启动器)——统一泳道(#966):** `bili plugin install dsh` 对每个已存在的 profile 执行 `dsh plugin --profile add billion-context` —— pnpm 把包装进各 profile 自己的 `node_modules`,dsh 自动挂载包内 patch 层(`dsh.bundle.patch.yml`)。装哪个源取决于 bili 自身的安装形态(#925):npm 安装传注册表名,checkout/dev 构建传绝对路径(`link:` 依赖,本地改动实时生效)。旧版受管块(`# bili begin` / `# bili end`,#966 之前的安装所写)在安装与卸载时都会被剥离 —— 用户条目与注释保留,清空的文件还原占位 `[]`。先在每个 profile 里跑过一次 dsh 让目录存在。插件加载时自拉起代理(已有健康实例则附看,不重复拉;父进程 pid 看门狗),经全局 fetch 补丁把模型流量改写为 `/bili/<上游URL>`,原样注册清单工具,并按工具就绪门控 plugin 模式头(第一轮走 wire 模式)。退出开关:`BILI_NATIVE_DSH=0`。卸载:`bili plugin remove dsh` 或 `dsh plugin --profile remove billion-context` —— 两者走同一通道。经注册表安装要求 npm 上已发布含 `dsh.bundle.patch.yml` 的版本;请装 npm 形态 —— 直接填 GitHub 地址装到的是没有 `dist/` 的源码(入口悬空、dsh 静默跳过该 bundle;安装期守卫现在会响亮失败并给出正确装法,#2471)。若 add 后 dsh 启动即报 `billion-context/dsh` 的 `ERR_MODULE_NOT_FOUND`,说明 profile 从陈旧的包元数据缓存里解析到了不含 bundle 子路径导出的旧版本(#953)——固定版本重装:`dsh plugin --profile add billion-context@latest`。 - **桌面应用(Electron 宿主):** 同一插件也运行在 deepseek-harness **desktop** 应用内(经其应用内插件管理器安装)。此时 bootstrap 在应用进程内拉起代理:该进程的 `process.execPath` 是 Electron 二进制而非 Node,GUI 的 PATH 又不含常规安装目录 —— `resolveNodeRuntime`(#819/#1429)先探测众所周知的安装位置(`/opt/homebrew/bin`、`/usr/local/bin`、Volta 等),再回退到以纯 Node 方式运行应用自身二进制(`ELECTRON_RUN_AS_NODE=1`,强制注入子进程 env),于是无需 PATH 里有独立 Node 也能零配置压缩;设 `BILLION_CONTEXT_NODE` 可强制指定某个 Node(优先于两者)。#1429 之前该路径在 spawn 前抛错,所有会话静默降级为直连(不压缩),只能从 bili.log 里发现。 - **自动更新保持各 profile 同步:** 刷新有两个触发器 —— 全局自更新完成后,以及**profile 自己的代理**在周期检查里发现注册表有新版本时(全局 bili 从不运行也一样刷新,#1196 —— 插件市场安装的用户往往根本没有全局安装)。两种触发器都扫描 `~/.dsh/profiles/*/package.json`,把注册表钉住的 `billion-context` 依赖刷新到目标版本(全局触发器刷到新全局版本,自触发刷到注册表最新),统一经 dsh 自己的 `plugin add` 通道,绝不在位覆盖 —— 加载的插件与代理从此不再漂移(#953);钉在本地源的 profile 不动。刷新是尽力而为,失败下个周期重试,绝不会让代理或更新本身失败。 - **已报告:Profile 安装下部分传输零代理流量(#1158,调查中):** dsh `llm-pi-ai` 层的部分传输服务会话**从未有任何模型请求到达代理**(日志无 `processTurn`,调用 bili 工具返回 404 "no model request has arrived"),而同宿主的其他 provider 正常。根因仍在用运行时证据定性 —— 候选:传输层 fetch 形态(SDK 注入 fetch / 非全局 dispatcher)或宿主侧归属缺口导致流量未被 takeover gate 认领。检测:此类情况会打一次性 `[plugin] NO MODEL REQUESTS seen for conversation …` 告警,dsh 插件还会把归属 gate 放行的每个端点各记一条日志(每进程一次)。期间可靠规避:改用 `bili dsh` 启动 —— 启动器的 settings overlay 会把那些 provider 的 `baseURL` 重写为 `/bili/` URL,无论传输层使用哪种 fetch、归属状态如何,流量都必然过代理。 -- **Web profile 说明(#1772,#2474 修正):** 当 profile 的 bundles 含 `@deepseek-ai/dsh-web-app` 时,真正运行的 `compaction-basic` 实例位于 agent preset(`preset-standard.config.plugins`)内部 —— dsh 的 patch 引擎只按 id 索引顶层条目与真 group(`group: true` 且 config 为数组)的子项,preset 内的嵌套行任何 patch 层都无法**按 id 直寻**。于是 bundle 里的 `auto: false` 落在 web-app 已禁用的宿主平面行上,preset 实例保持自动压缩开启。但 `preset-standard` 本身是 bundle 层的普通顶层条目:profile 层 patch(你 profile 自己的 `cordis.patch.yml`,在所有 bundle 之后生效)可用同 id + **完整快照** `config` 整体覆盖它 —— 从 `dsh --profile --dump-config` 抄下完整的 preset config,在其 `compaction-basic` 行下加 `config.auto: false`,把结果追加进 `cordis.patch.yml`。Config 替换是**整体语义**(片段会静默丢掉一切未重述的键 —— 不只是其他插件,连 preset 自己的 `id`/`order` 身份也会丢,可能使 roster 无法识别该 preset);`- insert:` 同 id 则是在末尾追加一条死行而非合并。dsh 升级 preset 后快照冻结 roster、需重抄一次。此类 profile 启动时插件打一条一次性 `[dsh-client]` 告警并给出该配方;ACP 压缩不受影响。 +- **Web profile 压缩(#1772,#3354 解决):** 当 profile 的 bundles 含 `@deepseek-ai/dsh-web-app` 时,真正运行的 `compaction-basic` 实例位于 agent preset(`preset-standard.config.plugins`)内部 —— dsh 的 patch 引擎只按 id 索引顶层条目与真 group(`group: true` 且 config 为数组)的子项,preset 内的嵌套行任何 patch 层都无法**按 id 直寻**。于是 bundle 里的 `auto: false` 落在 web-app 已禁用的宿主平面行上、什么都没改。#3354 起 bili 在**加载期**补上这一环:`bili-native` bundle 条目注册一个 cordis `internal/config` overlay(移植自 deshark42/dsh-compaction-basic-disable,MIT),把 preset 里的 `compaction-basic` 与 `command-compact` 行标记为 `disabled: true`;bundle patch 里的 preset 门控行让每个 Web preset 在注册前等待该 overlay。这些 profile 里原生自动压缩同样关闭(且 `/compact` 在那里不可用);ACP 压缩不受影响。overlay 依赖 dsh 的内部钩子、对版本敏感 —— 若在新/旧 dsh 构建上仍见到 dsh 压缩 checkpoint 落地,回退到 #2474 配方:profile 层 patch(你 profile 自己的 `cordis.patch.yml`,在所有 bundle 之后生效)用同 id + **完整快照** `config` 整体覆盖 preset-standard —— 从 `dsh --profile --dump-config` 抄下完整的 preset config,在其 `compaction-basic` 行下加 `config.auto: false`,把结果追加进 `cordis.patch.yml`。Config 替换是**整体语义**(片段会静默丢掉一切未重述的键 —— 不只是其他插件,连 preset 自己的 `id`/`order` 身份也会丢,可能使 roster 无法识别该 preset);`- insert:` 同 id 则是在末尾追加一条死行而非合并。dsh 升级 preset 后快照冻结 roster、需重抄一次。此类 profile 启动时插件打一条一次性 `[dsh-client]` 日志,说明两条路径。 - **全局副本在加载期抢占 profile 副本(#2891):** dsh 解析 `dsh.profile.bundles` 里的裸包名 `billion-context` 时是 installation-anchor-first —— 先在自己的安装树(全局 npm root)里找,再找 profile 的 `node_modules`。因此只要机器上存在一份全局 `billion-context`,dsh 就加载**全局那份**,profile 里装的被静默忽略 —— 不崩溃、无日志。两者版本不一致时,实际运行的可能与 profile 声明的不同(例如 #1590 之前的全局没有 `dsh.client` web 面,设置页菜单凭空消失)。这一事实默认情况下处处不可见:`bili doctor` 现在会按 profile 逐一标记任何版本不一致的全局/本地对(#2891);用 `dsh --profile --dump-config`(读该 bundle 的 `name:` 行)可核对 dsh 实际加载了哪份。 -`bili dsh` 启动下插件**附看**(attach)启动器的代理(不二次拉起)。裸上游 URL 与 spawn 模式一样重写为 `/bili/`(回环代理目标永不被代理 env 拦截,等于直接绕开 MITM);已经路由的 `/bili/` 前缀请求原样放行、只盖章。已知局限:手动 `/compact` 没有 dsh 侧事件钩子,其边界交给内核的自然 ingest diff(非 web profile 下自动压缩已关,影响罕见;web profile 见上方 #1772 说明)。 +`bili dsh` 启动下插件**附看**(attach)启动器的代理(不二次拉起)。裸上游 URL 与 spawn 模式一样重写为 `/bili/`(回环代理目标永不被代理 env 拦截,等于直接绕开 MITM);已经路由的 `/bili/` 前缀请求原样放行、只盖章。已知局限:手动 `/compact` 没有 dsh 侧事件钩子,其边界交给内核的自然 ingest diff(自动压缩现已处处关闭 —— 宿主平面经 bundle patch,web profile 的 preset 经 #3354 加载期 overlay;见上方 Web profile 说明)。 **宿主侧上下文计量与模型实际视图按设计不一致(#2837):**plugin 模式下宿主本地持久会话日志有意保持无损 —— 完整未折叠历史会因前缀缓存与严格回显而重发(见 TECHNICAL-NOTES.md「两种压缩模式」)。因此宿主或其插件从该本地日志算出的任何计量,量的都是**未折叠的本地表面**,而不是下一次请求实际携带的量;压缩生效期间本地数字远高于真实输入属预期,不是压缩卡死。有效尺寸以代理为准:读 `/acp`(或 `GET /__bili/plugin/status` 的 `contextTokens`,来源见 `contextTokensSource`),不要本地重新推算。 @@ -62,17 +62,19 @@ `CLOUD_CODE_URL` 环境变量覆盖(已在 v2.19.1 二进制中验证:*"Overriding CloudCodeServerURL via CLOUD_CODE_URL environment variable"*),形态与 gemini-cli 的 base-URL 钩子相同。启动器设置 - `CLOUD_CODE_URL=/bili/`(默认上游 - `https://cloudcode-pa.googleapis.com`;如果你自己导出了 `CLOUD_CODE_URL`, - 该值会被中继经过代理)。无 MITM、无需装 CA。代理**按 path 识别 wire**: + `CLOUD_CODE_URL=/bili/`(默认上游 + `https://daily-cloudcode-pa.googleapis.com` —— Antigravity 真实默认;裸 + `cloudcode-pa` 主机在 `/v1internal:*` 上 400,#3129;如果你自己导出了 + `CLOUD_CODE_URL`,该值会被中继经过代理)。无 MITM、无需装 CA。代理**按 path 识别 wire**: `:streamGenerateContent` / `:generateContent` / `:countTokens` 请求走 Google 原生适配器,与 `bili gemini` 一样被压缩。若服务端改用 gRPC 方法 path(`/google.internal.cloud.code.v1internal.CloudCode/*`),这些请求不被 识别、原样透传不压缩 —— 属优雅降级,确认后可用按 lane 的 `protocol` 声明(#1909)修复。若 Google 将来移除该环境钩子,退路是证书 MITM: - language_server 遵循 `HTTPS_PROXY` 且不锁定证书,把 - `cloudcode-pa.googleapis.com` 加进 `"mitm".domains` 并信任 bili 根 CA - 即可。桌面端与 IDE 扩展共用同一条 language_server 通道;启动器只驱动 + language_server 遵循 `HTTPS_PROXY` 且不锁定证书,把 + `daily-cloudcode-pa.googleapis.com`(Antigravity 真实默认端点;裸 + `cloudcode-pa` 名称在 `/v1internal:*` 上 400,#3129)加进 `"mitm".domains` + 并信任 bili 根 CA 即可。桌面端与 IDE 扩展共用同一条 language_server 通道;启动器只驱动 CLI —— 桌面用户可手动导出 `CLOUD_CODE_URL`,或直接用上面的 MITM 配方。 四者都没有 native 模式:均无环内工具注入接缝(gemini-cli 扩展只到自定义命令, @@ -258,7 +260,7 @@ bili 只压缩路径匹配已知 wire 协议(`/chat/completions`、`/llm_raw_cha HTTPS 走证书 MITM,HTTP 走临时 `opencode.json` 副本(`/bili/` 改写;JSONC 注释照单接受,合并方式与 opencode 自身一致;相对本地插件路径在副本里重新锚定为绝对路径 —— opencode 按声明所在配置文件目录解析,#826)。宿主代次用 `--version` 探测(探测失败默认按 1.x):**2.x** 宿主注入内置 V2 插件(`dist/agent/opencode.js`),以临时包装目录形式给出(目录入口 `index.js` 再 re-export 插件文件 —— 2.x 拒绝配置 `plugin` 数组里的裸文件路径);**1.x** 宿主直接给裸文件路径。 -插件在两代宿主上做的事相同:在宿主内原生注册 bili 工具 —— compress / decompress / search_context / acp_status—— 并在每个 provider 请求上盖章代理头,含从宿主自身模型目录(`ctx.catalog.model.list()`,每 60s 刷新)读取的 context-window / max-output,并以 runtime-info 上报给代理(#955)—— 压缩走插件模式,**不做** wire 级工具注入;原生 auto-compaction 自动关闭(`compaction.auto: false`)。所有注册都是防御式的(可选链):任一 2.x build 上接缝缺失或未触发时,插件保持惰性,会话透明回退纯代理模式而不是报错 —— 在相邻的 `dev` build 上观察到过 API 面互不相同(#754 评审探针)。 +插件在两代宿主上做的事相同:在宿主内原生注册 bili 工具 —— compress / decompress / search_context / acp_status(另加 absorb)—— 并在每个 provider 请求上盖章代理头,含从宿主自身模型目录(`ctx.catalog.model.list()`,每 60s 刷新)读取的 context-window / max-output,并以 runtime-info 上报给代理(#955)—— 压缩走插件模式,**不做** wire 级工具注入;原生 auto-compaction 自动关闭(`compaction.auto: false`)。所有注册都是防御式的(可选链):任一 2.x build 上接缝缺失或未触发时,插件保持惰性,会话透明回退纯代理模式而不是报错 —— 在相邻的 `dev` build 上观察到过 API 面互不相同(#754 评审探针)。 1.x 细节(1.14.46 + 1.18.31 验证):V1 `.server()` 钩子在进程内把每个 provider 的 `options.baseURL` 改写为 `/bili/…` 并设 `compaction.auto: false`;`chat.headers` 每次请求盖章插件头;`tool` 用真实 zod 形状注册 bili 工具(zod 是运行时依赖 —— 解析不到时降级为只改写)。没有显式 `baseURL` 的 provider(SDK 默认值,如裸 `@ai-sdk/openai` → api.openai.com)由全局 `fetch` 补丁兜住(日志:`v1: fetch patch installed`)—— 幂等,`/bili/` 包装过的 URL 原样直通;含 OpenAI Responses 端点端到端验证。 diff --git a/CONFIGURATION.md b/CONFIGURATION.md index d7824e1c7a..e91e6d2da7 100644 --- a/CONFIGURATION.md +++ b/CONFIGURATION.md @@ -87,6 +87,7 @@ This index is generated from `website/config-reference/*.yaml` — edit the seed | `modelContextLimit` | number | 200000 | ACP_MODEL_CONTEXT_LIMIT | Legacy global window ceiling (highest-priority window source and usage denominator); prefer compress.modelContextLimit. | | `imageBilling` | "auto" \| "pixels" \| "bytes" | auto (resolves to pixels) | BILI_IMAGE_BILLING | Global image-token estimation basis; live-read per request, beats every per-provider setting. | | `imageTokenCap` | number | unset (uncapped) | BILI_IMAGE_TOKEN_CAP | Global per-image token ceiling applied on top of the resolved billing mode; live-read per request. | +| `imageRescale` | boolean | true | BILI_IMAGE_RESCALE | Egress image downscale master switch (#3295/#3400): large outbound images are sent shrunken while originals stay untouched in session storage. Live-read per request. | **Behavior toggles** @@ -100,6 +101,7 @@ This index is generated from `website/config-reference/*.yaml` — edit the seed | `rollout.features.salt` | string | the feature name | — | Hash salt for bucketing. Changing it re-randomizes the whole cohort (the only way to reshuffle; raising percent alone never reshuffles). | | `rollout.features.base` | string | unset (roll out over everyone) | — | Nested rollout: evaluate this feature only inside the named feature's enabled cohort, so the percent is relative to that population (roll B out to A's users; chains allowed, cycles fail closed). | | `maskHosts` | boolean | true | BILI_LOG_MASK_HOSTS | Mask non-public target hosts in logs as ; credential headers are always masked regardless. | +| `security.adminRelay` | boolean | false | — | Let an embedding host (dsh web today) relay the ENTIRE /__bili/ admin surface through its own entry origin so a remote browser can open the panel (#3373). Fail-closed, file-only. The relayed hop reaches the proxy as a local non-browser client (loopback source + loopback Host, no Origin), so the admin gate stays unchanged; enabling it exposes config editing and credential writes to whoever can reach the host web entry — protect that entry (auth / reverse proxy) first. | | `subagentSplit` | boolean | true | BILI_SUBAGENT_SPLIT | Claude Code subagents get their own session namespace (\|sub:) instead of queueing behind the main session. | | `forkAdoption` | boolean | false | BILI_FORK_ADOPTION | Anonymous forks inherit the parent's compression blocks whose source content is fully present in the forked request. | | `autoModeForge.enabled` | boolean | true | BILI_AUTO_MODE_FORGE | Answer Claude Code auto-mode safety-classifier calls locally with a forged allow verdict instead of forwarding them upstream (#2740): zero egress, zero billing for that lane; every forged verdict is audit-logged and the session carries a notice badge. false / env 0 restores pure passthrough for users who pay for real verdicts. | @@ -170,7 +172,7 @@ This index is generated from `website/config-reference/*.yaml` — edit the seed | `persist.epermAlertRepeatMs` | number | 0 (no repeats) | BILI_PERSIST_EPERM_ALERT_REPEAT_MS | Repeat interval for the EPERM alert; 0 logs it once. | | `storage` | object | {} (all defaults) | — | Unified SQLite storage sidecar settings (#2671). | | `storage.unified` | legacy\|shadow\|full | unset (shadow) | BILI_STORAGE_UNIFIED | Tiered storage-migration switch (#2671): shadow = unified SQLite double-write dry-run with digest reconciliation; full = drop-in read path (unified-first loads with organic import, legacy fallback). Absent everywhere now defaults to shadow (JSON stays the sole source of truth; the index is digest-reconciled telemetry - set legacy to opt out). File tier is strict legacy/shadow/full; the env tier also accepts 1/true (=shadow) and unified (=full) and wins over the file. Resolved once per lane boot - hot edits need a restart. Encryption (BILI_ENCRYPTION_KEY) hard-blocks the sidecar. | -| `storage.sqliteAutoDownload` | boolean | true | BILI_SQLITE_AUTO_DOWNLOAD | Gate for the one-time SQLite engine self-download (#2833): when no engine is available in-process (no built-in node:sqlite, no host-installed better-sqlite3) and the unified sidecar is wanted on this platform, bili downloads the pinned better-sqlite3 prebuilt tarball from the npm registry once (~11 MB, sha512-verified) into the cache dir and reuses it on every later start. Engine resolution order: built-in node:sqlite (Node >= 22.5) -> host-provided better-sqlite3 -> cached self-download -> self-download (gated by this switch) -> none (legacy-only, the #2825 degrade contract). The switch gates only the network fetch - a cached copy is used even when it is false; unsupported platforms never download. | +| `storage.sqliteAutoDownload` | boolean | true | BILI_SQLITE_AUTO_DOWNLOAD | Gate for the one-time SQLite engine self-download (#2833): when no engine is available in-process (no built-in node:sqlite, no host-installed better-sqlite3) and the unified sidecar is wanted on this platform, bili downloads the pinned better-sqlite3 prebuilt tarball from the npm registry once (~11 MB, sha512-verified) into the cache dir and reuses it on every later start. Engine resolution order: built-in node:sqlite (Node >= 22.5) -> host-provided better-sqlite3 -> cached self-download -> self-download (gated by this switch) -> none (legacy-only, the #2825 degrade contract). The switch gates only the network fetch - a cached copy is used even when it is false; unsupported platforms never download. The tarball download walks a builtin mirror list (#3338): the configured registry (BILI_UPDATE_REGISTRY / update.registry) is tried first, then https://registry.npmjs.org, then https://registry.npmmirror.com - the first mirror that yields a sha512-verified tarball wins; a Node < 22.5 with no engine gets an explicit Node upgrade hint in the degrade reason (#3385). | | `storage.jsonWriter` | on\|off | on | BILI_STORAGE_JSON_WRITER | Phase 3 envelope switch (#2960): off stops writing the legacy JSON session envelope while storage.unified is full - the SQLite index becomes the only persistent copy (reads are already unified-first; boot enumerates the index union, so index-only sessions load normally). Only honored in effective-full mode: under shadow/legacy (or with encryption active) the request is ignored with a one-time warning because the JSON file remains the sole source of truth. Env tier 0/false/off = off and 1/true/on = on wins over the strict file spellings. Fail-safe: if the sidecar disables itself mid-run, envelope writes resume on the next save. Rollback: unset the knob - writing resumes, index-only sessions keep loading via the boot union, and replicas/restore re-materialize portable JSON envelopes. Downgrading the binary (or losing the SQLite engine) makes index-only sessions invisible to v0.1.196 and older until a >=v0.1.197 binary runs again - the data stays intact in index.db, it is just not read by the older code. The per-session content-store sidecar keeps being written either way. A jsonWriter=off lane must be the SOLE writer of its sessions directory: an off lane reads unified-first and never falls back to the file, so any peer instance running legacy/shadow against the same directory diverges silently (its JSON copy stops receiving updates, and the off lane never sees the peer's writes). | | `storage.walJournalSizeLimitBytes` | number | 8388608 | BILI_STORAGE_WAL_JOURNAL_SIZE_LIMIT_BYTES | WAL reclamation cap for index.db-wal (#3250), in bytes: every completed checkpoint truncates the WAL file back to this size, bounding disk use and crash-recovery replay in long-lived processes. NOT a per-session data quota - data lands in index.db as it is written and the WAL only stages frames between checkpoints, so session count/size does not scale this; a large burst can overshoot temporarily until the next completed checkpoint truncates back. Higher = rarer truncate IO but more steady disk use and a longer crash-replay tail. Env tier (positive integer) wins over the file tier; a garbage or non-positive env value falls through to the file/default tier instead of failing boot. Applied when the store opens at lane boot - hot edits need a restart. The effective value appears on the unified-enable log line and in the storage diagnostics blob. | | `sessions` | object | {} (all defaults) | — | Session-table size cap and opt-in garbage collection. | @@ -248,10 +250,11 @@ This index is generated from `website/config-reference/*.yaml` — edit the seed | `compress.neverPreserveRecentTools` | string[] ([] valid) | ["decompress", "search_context", "read", "bash"] (kernel) | — | Excluded from the recent protection zone (immediately compressible); an empty array excludes nothing (maximum protection). | | `compress.preserveRecentTools` | string[] | n/a (subtraction form) | — | Subtraction form: recent-zone tools minus this list get full protection; an empty array here is rejected as a typo. | | `compress.visibilityMarkers` | boolean | true | — | Append visibility markers after compress/decompress/search_context/acp_status results; off stops imitated narration. | -| `compress.rules` | boolean | false | — | Persistent model reminders delivered through an injected acp_rule tool, hard-protected from folds; pi/omp expose an /acp-rule command. | +| `compress.rules` | boolean | false | — | Persistent model reminders delivered through an injected acp_rule tool, hard-protected from folds; pi/omp expose an /acp-rule command. Proxy-mode only — inert in plugin mode, where the host owns the tool surface. | | `compress.injectTool` | boolean | true | ACP_COMPRESS_TOOL | Register the acp_compress tool for clients (global level only). | | `compress.injectNudge` | boolean | true | ACP_COMPRESS_NUDGE | Send growth nudges as the window fills (global level only). | | `compress.reasoning` | { drop?, threshold? } | drop true · threshold 2048 | — | Drop finished-round reasoning runs longer than 2048 chars (drop: true by default); strict-reasoning upstreams need drop:false. | +| `compress.absorb` | object | opt-in (disabled) | — | Opt-in instant distillation of tool results into short abstracts; originals go to the content store. By default only image-bearing results qualify. | | `compress.ccr` | object | enabled in proxy mode since v2 | — | Content cache & retrieve: large outputs stored losslessly beside the session, replaced by head excerpt + pointer fetched back via acp_retrieve; no cap or eviction. | | `compress.search` | object | off | — | Search-behavior settings for search_context; currently holds planAware re-ranking. | | `compress.search.planAware` | boolean | false | — | When on, search_context candidates are re-ranked against current plan state; off keeps byte-identical lexical results. | @@ -260,6 +263,12 @@ This index is generated from `website/config-reference/*.yaml` — edit the seed | `compress.outputSteering` | object { enabled?, verbosityLevel?, effortRouting? } | enabled false · verbosityLevel 2 | — | Appends a conciseness directive to the system-prompt tail; clamps mechanical continuation requests. | | `compress.priceProfile` | { w?, r?, q? } | unset (registry, kernel fallback {1, 0.1, 4}) | — | w/r/q ratios normalized to the input price (write/cache-read/output); report-only — never affects triggers or the wire. | | `compress.acknowledgePromptsRisk` | boolean | false | — | Must be true before custom prompts take effect. | +| `compress.absorb.enabled` | boolean | false | — | Enable the absorb distillation block. | +| `compress.absorb.minToolTokens` | number | 1000 | — | Minimum estimated token size of a tool result eligible for absorption. | +| `compress.absorb.contextThresholdPct` | number \| % | unset | — | Only absorb when context usage passes this share of the window. | +| `compress.absorb.excludeTools` | string[] | [] | — | Tool names excluded from absorption. | +| `compress.absorb.toolName` | string | "absorb" | — | Registered name of the absorb tool. | +| `compress.absorb.mediaOnly` | boolean | true | — | Restrict absorption to tool results that carry media (images). Image bytes are exempt from minToolTokens since they are not counted in text. Set false to restore legacy full-text absorption of any large result. | | `compress.ccr.enabled` | boolean | true (proxy mode) | — | Enable the content-cache-and-retrieve block. | | `compress.ccr.minToolTokens` | number | ≈4000 | — | Minimum estimated token size of a tool result eligible for CCR storage. | | `compress.ccr.excludeTools` | string[] | [] | — | Tool names excluded from CCR storage. | @@ -367,6 +376,7 @@ This index is generated from `website/config-reference/*.yaml` — edit the seed | `BILI_MCP_NO_ORPHAN_ADOPT` | boolean-ish | off (=1 disables orphan adoption) | — | Opt out of resumed-session orphan adoption when several host sessions share one proxy. | | `BILI_MITM_HOSTS` | string[] (csv) | unset | — | Launcher channel: MITM whitelist hosts handed to the spawned proxy (distinct from the config-file twin BILI_MITM_DOMAINS). | | `BILI_PLUGIN_AGENT` | boolean-ish | off | — | Spawn channel: marks a plugin-agent process. | +| `BILI_IMAGE_RESCALE_MAX_DIM` | integer | 1600 | — | Longest-edge trigger threshold (px) for the egress image downscale pass (#3400); non-numeric or below 128 ignored (default applies). Env-only by design. | @@ -388,7 +398,7 @@ Top-level keys that control how the proxy listens and behaves globally. - **Type:** `string` - **Default:** `127.0.0.1` - **Status:** ACTIVE -- **Description:** Network interface the proxy binds to. `127.0.0.1` (default) listens only on localhost — safe for a local sidecar. Use `::` for IPv4 + IPv6 dual-stack. Use `0.0.0.0` (or a LAN IP) to expose the proxy to other machines — typically inside a container or on a trusted LAN: remote agents then point their model `baseURL` at `http://:/bili/…`, and MITM-mode `CONNECT` accepts remote clients for whitelisted model hosts only (blind tunnels stay loopback-only, and `/__bili/` management endpoints remain loopback-only). There is no authentication — ensure the surrounding network is trusted. Overridden by `ACP_HOST` / `--host`. +- **Description:** Network interface the proxy binds to. `127.0.0.1` (default) listens only on localhost — safe for a local sidecar. Use `::` for IPv4 + IPv6 dual-stack. Use `0.0.0.0` (or a LAN IP) to expose the proxy to other machines — typically inside a container or on a trusted LAN: remote agents then point their model `baseURL` at `http://:/bili/…`, and MITM-mode `CONNECT` accepts remote clients for whitelisted model hosts only (blind tunnels stay loopback-only, and `/__bili/` management endpoints remain loopback-only unless the embedding-host relay below is enabled). There is no authentication — ensure the surrounding network is trusted. Overridden by `ACP_HOST` / `--host`. ### `upstream` @@ -524,6 +534,13 @@ Top-level keys that control how the proxy listens and behaves globally. - **Status:** ACTIVE - **Description:** Blanket ceiling on the per-image token estimate used by the preflight size gate, output clamp, and image-compression stats (#488/#496/#1843). Applied on top of whatever billing mode resolved — useful when a route's real encoder bills far below even the pixel prior. Precedence: `BILI_IMAGE_TOKEN_CAP` env var (live-read, no restart) > per-provider `providers..imageTokenCap` > this global. Non-numeric or non-positive values are dropped (lenient parse, same as `imageBilling`). +### `imageRescale` + +- **Type:** boolean +- **Default:** `true` +- **Status:** ACTIVE +- **Description:** Master switch for the egress image downscale pass (#3295 removed the old arrival-time pass; #3400 re-adds it in simplified form). On every outbound request, after kernel folding and before protocol encoding, images whose longest edge exceeds `BILI_IMAGE_RESCALE_MAX_DIM` (default 1600) OR whose base64 body exceeds 100 KB are re-encoded fit-inside (never upscaled, original format kept) on an OUTBOUND COPY only — session storage keeps the original bytes byte-exact. A re-encode that saves less than 20% of the bytes is discarded (keep-smaller), so small/low-benefit images pass through untouched. Only PNG/JPEG/BMP are handled (pure-JS encoder, zero native dependencies); other formats pass through. Each shrunken image gets a stable annotation beside it reporting current vs original size; on lanes that register bili tools (proxy-injection lanes and plugin lanes via the plugin manifest) it also names `read_original_image("")`, which returns the original bytes from local session storage — codex-style lanes that reject injected tools get the size-only annotation and no tool. Results of `read_original_image` are structurally exempt from re-shrinking (the loop guard against the #3266 guess-reject-resend disease). Output is deterministic: the same input bytes always produce the same shrunken bytes and the same annotation text, so repeated resends of one history stay byte-stable and upstream prefix caching is undisturbed. One log line per unique image (content-hash cached; cache hits stay silent). Precedence: `BILI_IMAGE_RESCALE` env var (`"0"`/`"false"` disables; live-read, no restart) > this key > default on. + ### `resign` - **Type:** `object` — scheme-keyed: `Record` where each key is a lowercase signature scheme token as detected on the wire (built-in: `"sdk-hmac-sha256"`; custom: an HMAC `Authorization` token like `"aws4-hmac-sha256"`, or a body-signature header name like `"x-ofm-signature"`) @@ -648,6 +665,19 @@ Top-level keys that control how the proxy listens and behaves globally. - **Status:** ACTIVE - **Description:** Override for the release-notes document URL. Overridden by `BILI_RELEASE_NOTES_URL`. +### `security` + +- **Type:** object +- **Default:** none of the knobs below are set (every one fails closed) +- **Status:** ACTIVE +- **Description:** Security-posture knobs — the place where turning something ON deliberately WIDENS a trust boundary, kept as one auditable block. File-only: no `BILI_*` env twin for anything in here (#2030). + +#### `security.adminRelay` (#3373) + +- **Type:** `boolean` +- **Default:** `false` (fail-closed) +- **Description:** Lets an embedding host — dsh's web panel today — relay the ENTIRE `/__bili/` admin surface through the host's own web entry, so a browser on another machine can open the panel without the proxy ever binding a non-loopback interface. Headless setup: run `dsh web`, point your reverse proxy at it, and open the bili tab from anywhere — the settings iframe and its API calls stay same-origin with the entry and ride the host's relay. Mechanics: the dsh-side bili plugin registers a `/__bili` prefix route that forwards to the loopback proxy (`127.0.0.1:18787` zone), `/bili/origin` starts reporting `{"relay":true}` so the panel UI builds same-origin URLs, and the panel's own API root derives itself from the page path — so prefix gateways (`/app/dsh/…`) work too. The admin gate itself is UNCHANGED (#1538): the relayed hop arrives at the proxy as a local non-browser client — loopback source address, loopback `Host`, no `Origin`/`Cookie`/`Referer` forwarded — exactly the shape the existing checks already admit. **Security note:** `/__bili/` is the full admin surface, not a read-only dashboard — it includes config editing and credential writes. Enabling this flag means whoever can reach the host web entry can reach all of that; protect the entry (auth / reverse proxy) before turning it on. Read live per request: toggling the file applies to the next call, no restart needed. No env var — deliberate posture decisions belong in the file. + ### `maskHosts` - **Type:** `boolean` @@ -1337,12 +1367,25 @@ For each request, the proxy resolves the settings by longest-URL-prefix match (t - **Status:** ACTIVE - **Description:** Controls **fold-state reconciliation** (#1921): what happens when a client resends history whose *content* has drifted from what was compressed — e.g. a fork re-serialized earlier turns (#1908), a client rewriting message bytes in place, or a provider normalizing whitespace. By default message identity is a content hash, so a one-byte churn silently drops the affected messages out of every compression block and the raw originals re-enter the wire ("fold drift"). In `"repair"` mode the proxy classifies the drift per message before the fold runs: stable tool-call anchors (`tool_use_id`) and normalized-identity anchors (NFC text, CRLF/whitespace-collapsed) re-anchor compression blocks onto the new message ids, so the fold survives client-side churn byte-for-byte-equivalent to a clean resend; genuinely *edited* messages (normalized text differs, or length drifts beyond `max(256, 25%)`) are left to re-enter the wire unfolded — honest, never a silent rewrite — and the next compression pass re-folds them. `"warn"` runs the same classifier but only logs what it found; `"off"` disables the layer entirely (pre-#1921 behavior). The environment variable `BILI_FOLD_RECONCILE` overrides the configured value for one process. Anchor metadata is persisted per session (`foldAnchors` in the session record, capped at 16384 entries) and covers the messages each block actually references. A one-line `info` is logged when the system prompt changes without any message churn (system-only drift never touches fold state). +#### `absorb` + +- **Type:** `object` (`{ enabled?, minToolTokens?, contextThresholdPct?, excludeTools?, toolName? }`) +- **Default:** *(disabled — the feature is off unless you set `enabled: true`)* +- **Status:** ACTIVE +- **Description:** Opt-in **instant tool-result compression** (issue #605, via the `acp-kernel` absorb API). When enabled, large tool results get a forced `[ACP absorb]` instruction at result time; the model distills the result into a compact summary via the `absorb` tool, and the original tool-call/tool-result pair is hidden from the wire from the next turn on — keeping mid-session pressure lower between fold rounds. Sub-fields (merged deepest-wins like every other CompressSettings field): + - `enabled: boolean` — master switch; anything other than `true` keeps the feature fully off (no tool, no prompt, no markers). + - `minToolTokens: number` — only results at or above this many tokens are prompted (kernel default 1000). + - `contextThresholdPct: number|percent-string` — only prompt once usage reaches this fraction of `modelContextLimit` (`0` = size gate alone; `"75%"` is accepted). + - `excludeTools: string[]` — tool-name patterns never absorbed. **Known limitation:** a no-op on tool *results* until [ranxianglei/acp-kernel#213](https://github.com/ranxianglei/acp-kernel/issues/213) ships (wire projections don't carry `toolName` on results, so the kernel's name guard can't fire). + - `toolName: string` — rename the injected tool (default `"absorb"`); the advertised/injected schema, the system-prompt section and per-session adjudication all follow the name in both lanes. **Lane governance (#1359):** plugin mode governs the *entire* `absorb` block by the **base** config, so a renamed tool is advertised under that name and executed under it (they can never diverge); provider/model-level `absorb.*` overrides are **proxy-lane-only** (the proxy injects and adjudicates the merged name). A load-time warning lists any provider/model `absorb.*` field whose value diverges from base. + Injection follows the wire's native-tool surface: proxy mode injects the tool (under the per-request resolved name) + a system-prompt section on the anthropic/openai/responses native-tools wires, plugin mode advertises it in the plugin manifest **only while enabled** (hosts register the manifest's tools verbatim — advertising a disabled tool breaks them, #1192; the MCP shell picks it up for free). Responses **marker/text-protocol** routes are not supported (no native tool surface — the REQUIRED absorb instruction would be unsatisfiable), and title-generation requests (`max_tokens ≤ 200`) skip injection like the compress prompt does. Absorbed pairs stay hidden across restarts (persisted in the session state). + #### `ccr` - **Type:** `object` (`{ enabled?, minToolTokens?, excludeTools?, toolName?, maxHeadChars? }`) - **Default:** *(on in proxy mode since v2 (#1179) — unset resolves to `enabled: true`; an explicit `enabled: false` at any level opts out. Plugin lanes stay opt-in: they arm only on an explicit global `enabled: true`, #1271/#1273.)* - **Status:** ACTIVE (v2 — proxy mode by default; plugin mode on the anthropic + openai wires when explicitly enabled) -- **Description:** **Content-addressed message store** / built-in CCR (issue #1097/#1179, via the `acp-kernel` CCR API, acp-kernel >= 0.0.84). Instead of carrying oversized tool results on the wire forever, the kernel ID-references them at arrival (the ccr-store node runs between prune and absorb within `processTurn` — ID-referencing wins over distillation): the wire keeps a deterministic, byte-stable placeholder (`📦 [acp-stored #m00423 · shell output · 4,213 tok] \`npm run build\`\n → acp_retrieve("m00423") returns the full text`) and the original goes into the session's content store. The model retrieves the full original on demand via the injected `acp_retrieve` tool; a retrieve is ephemeral (it rides the intra-request tool-result channel, never enters fold space, consumes no message ref). Lossless by default: a retrieve not made costs one cheap tool call. Sub-fields (merged deepest-wins like every other CompressSettings field): +- **Description:** **Content-addressed message store** / built-in CCR (issue #1097/#1179, via the `acp-kernel` CCR API, acp-kernel >= 0.0.84). Instead of force-distilling oversized tool results (like `absorb`) or carrying them on the wire forever, the kernel ID-references them at arrival (the ccr-store node runs between prune and absorb within `processTurn` — ID-referencing wins over distillation): the wire keeps a deterministic, byte-stable placeholder (`📦 [acp-stored #m00423 · shell output · 4,213 tok] \`npm run build\`\n → acp_retrieve("m00423") returns the full text`) and the original goes into the session's content store. The model retrieves the full original on demand via the injected `acp_retrieve` tool; a retrieve is ephemeral (it rides the intra-request tool-result channel, never enters fold space, consumes no message ref). Lossless by default: a retrieve not made costs one cheap tool call; a detail distilled away by absorb is gone for good. Sub-fields (merged deepest-wins like every other CompressSettings field): - `enabled: boolean` — master switch. **Defaults to `true`** when unset at every level (#1179); an explicit `false` at any level wins and keeps the feature fully off (no placeholders, no tool). Plugin-mode arming additionally requires an explicit global `true` (#1273) — the manifest only advertises what the operator explicitly enabled. - `minToolTokens: number` — only tool results at or above this many tokens are ID-referenced (kernel default `4000`); smaller results stay verbatim. - `excludeTools: string[]` — tool-name patterns never stored (glob suffixes allowed; kernel default: none). @@ -1364,7 +1407,7 @@ For each request, the proxy resolves the settings by longest-URL-prefix match (t - **Type:** `boolean` - **Default:** `false` - **Status:** ACTIVE -- **Description:** **Persistent model reminders** (issue [ranxianglei/billion-context-pi#433](https://github.com/ranxianglei/billion-context-pi/issues/433), via the `acp-kernel` rules API), **off by default** — set `compress.rules: true` to enable it (the #1399 decision: once enabled the model has full rights over session rules and may call it unprompted, including delete/clear). When enabled, an `acp_rule` tool is injected alongside the ACP tools: calling it with a short `rule` argument records a principle-level reminder that is **hard-protected** from compression (the call + result stay in context across every fold), and omitting the argument lists the recorded rules. Passing `delete` with a rule id (e.g. `"rule3"`) removes that one rule (`Removed ruleN: `); passing `clear: true` removes every rule. The two are mutually exclusive with each other and with `rule` — one operation per call — so a stale or accidentally-recorded rule can be deleted instead of hedged with a counter-rule. Guidance on *when* to record (user-emphasized lessons, behaviors the user asks to remember, major pitfalls hit) lives entirely in the tool description — nothing is added to the system prompt. Kernel limits apply (50 rules × 300 chars each); duplicate text is rejected pointing at the existing id. Injection follows the wire's native-tool surface: proxy mode injects the tool on the anthropic/openai/responses native-tools wires, plugin mode advertises it in the plugin manifest **only while enabled** (hosts register the manifest's tools verbatim — advertising a disabled tool breaks them, #1192; execution gated per session). Recorded rules persist across restarts in the session state; removals persist the same way. Requires `acp-kernel` >= 0.0.70 (>= 0.0.84 for delete/clear). Humans get the full operation set without going through the model: pi/omp expose a native `/acp-rule` command with the same semantics as the tool — bare `/acp-rule` lists every recorded rule, `/acp-rule ` records one directly, `/acp-rule remove ` deletes one, and bare `/acp-rule clear` wipes all recorded rules (`clear ` records instead of wiping) (#1251; remove/clear added in #1399). +- **Description:** **Persistent model reminders** (issue [ranxianglei/billion-context-pi#433](https://github.com/ranxianglei/billion-context-pi/issues/433), via the `acp-kernel` rules API), **off by default** — set `compress.rules: true` to enable it (the #1399 decision: once enabled the model has full rights over session rules and may call it unprompted, including delete/clear). When enabled, an `acp_rule` tool is injected alongside the ACP tools: calling it with a short `rule` argument records a principle-level reminder that is **hard-protected** from compression (the call + result stay in context across every fold), and omitting the argument lists the recorded rules. Passing `delete` with a rule id (e.g. `"rule3"`) removes that one rule (`Removed ruleN: `); passing `clear: true` removes every rule. The two are mutually exclusive with each other and with `rule` — one operation per call — so a stale or accidentally-recorded rule can be deleted instead of hedged with a counter-rule. Guidance on *when* to record (user-emphasized lessons, behaviors the user asks to remember, major pitfalls hit) lives entirely in the tool description — nothing is added to the system prompt. Kernel limits apply (50 rules × 300 chars each); duplicate text is rejected pointing at the existing id. Injection follows the wire's native-tool surface exactly like `absorb`: proxy mode injects the tool on the anthropic/openai/responses native-tools wires, plugin mode advertises it in the plugin manifest **only while enabled** (hosts register the manifest's tools verbatim — advertising a disabled tool breaks them, #1192; execution gated per session). Recorded rules persist across restarts in the session state; removals persist the same way. Requires `acp-kernel` >= 0.0.70 (>= 0.0.84 for delete/clear). Humans get the full operation set without going through the model: pi/omp expose a native `/acp-rule` command with the same semantics as the tool — bare `/acp-rule` lists every recorded rule, `/acp-rule ` records one directly, `/acp-rule remove ` deletes one, and bare `/acp-rule clear` wipes all recorded rules (`clear ` records instead of wiping) (#1251; remove/clear added in #1399). #### `reasoning` @@ -1647,6 +1690,7 @@ File keys resolve only when the matching env var is unset. Defaults in parenthes | `BILI_GUARDRAIL_REPETITION` | `guardrails.repetition` | true | | `BILI_GUARDRAIL_TEXT_RUN_THRESHOLD` | `guardrails.textRunThreshold` | 200 | | `BILI_IMAGE_BILLING` | `imageBilling` | auto (resolves to pixels) | +| `BILI_IMAGE_RESCALE` | `imageRescale` | true | | `BILI_IMAGE_TOKEN_CAP` | `imageTokenCap` | unset (uncapped) | | `BILI_KEEP_ALIVE_TIMEOUT_MS` | `network.keepAliveTimeoutMs` | 5000 | | `BILI_LOG_MASK_HOSTS` | `maskHosts` | true | @@ -1724,6 +1768,8 @@ File keys resolve only when the matching env var is unset. Defaults in parenthes | `BILI_CCR_RETRIEVAL_TTL_MS` | Expiry (ms) for a queued-but-undelivered `acp_retrieve` injection (#1343): if the full text stays queued this long without ever riding an upstream request, it is dropped **loudly** — warn log with refs + reason, `stats.retrieveDropped` bump, and a corrective note on the next request telling the model to re-issue `acp_retrieve`. Default `600000` (10 min); `0` disables expiry. Plugin-lane queue items only; range-restore riders (#1207) are exempt. | | `BILI_IMAGE_TOKEN_CAP` | Cap the per-image token estimate used by the preflight size gate, output clamp, and image-compression stats (#488/#496/#1843). Applied on top of whatever billing mode resolved — a blanket ceiling for routes whose real encoder bills below even the pixel prior. Wins over both config levels ([`imageTokenCap`](#imagetokencap)); live-read per request (no restart). Unset = fall back to config (`providers..imageTokenCap`, then global), default no cap. | | `BILI_IMAGE_BILLING` | Override the image billing mode used by the preflight size gate and output clamp (#767/#1843): `pixels` or `bytes`. Live-read per request (no restart); beats the global `imageBilling` and every per-provider `providers..imageBilling`. Since #1843 the default resolves to `pixels` for every host, so use `bytes` to force conservative byte billing process-wide (e.g. a byte-counting relay) — the per-route setting remains the narrower tool. See [`imageBilling`](#imagebilling). | +| `BILI_IMAGE_RESCALE` | Disable (`"0"` or `"false"`) or leave enabled (anything else) the egress image downscale pass (#3400): large outbound images are sent shrunken while originals stay byte-exact in session storage. Live-read per request (no restart); wins over the config-file key [`imageRescale`](#imagerescale), whose own default is on. | +| `BILI_IMAGE_RESCALE_MAX_DIM` | Longest-edge trigger threshold (px) for the egress downscale pass (#3400): an image is a rescale candidate when its longest edge exceeds this value OR its base64 body exceeds 100 KB; fit-inside resizing never upscales. Default `1600`; non-numeric or values below `128` are ignored (default applies). Env-only by design — no config-file mirror. See [`imageRescale`](#imagerescale). | | `BILI_PREFLIGHT_HOLD_MS` | Grace period (ms) before a long preflight compression starts holding the client with keep-alive bytes (default `30000`; see #568). | | `BILI_STREAM_KEEPALIVE_MS` | Streaming-phase client hold (#1647): when an SSE response has written zero bytes to the client for this many milliseconds, bili emits one SSE comment line (`: bili-keepalive`, a spec-level no-op) so the client's undici `bodyTimeout` (default 300s; Node built-in fetch cannot override it per request) cannot kill long prefills whose upstream pings the rewriter/strip pipes swallow. Default `15000`; `0` disables. Sibling of `BILI_PREFLIGHT_HOLD_MS`, which covers compression-preflight silence — this covers upstream-caused silence during streaming. | | `BILI_RECLAIM_FETCH_PATCH` | Set to `0` to disable the native-mode fetch self-heal re-arm (#1158). By default the native fetch intercept installs `globalThis.fetch` as a guarded accessor, so a third-party patch that re-installs `globalThis.fetch` (e.g. dsh-http-proxy's settings refresh writing its frozen pre-bili `originalFetch`) is re-chained as the downstream and model traffic keeps routing through bili. With `0` the classic direct install stays: a third-party re-arm then wins and bili stops seeing model traffic for the session. **Egress note:** while the guard holds, claimed model traffic is dispatched by the bili proxy itself — it no longer rides the third-party chain's egress (e.g. a SOCKS5 proxy configured in dsh-http-proxy; bili's own upstream proxying supports HTTP proxies only). If you need the third-party egress back, set `0` and configure the egress at bili's level (`"proxy": "http://…"`). | @@ -1769,7 +1815,8 @@ File keys resolve only when the matching env var is unset. Defaults in parenthes | `BILI_AFFINITY_SIMHASH` | Set `0` to turn OFF simhash chain-alignment adoption (#2265): anonymous requests whose exact hash chain was broken by a client-side pervasive decorative rewrite (e.g. Trae re-stamping model tags on every assistant message) re-attach the existing session with its compression state instead of minting fresh and re-folding from zero. Guards: ≥90% coverage within Hamming 10, ≥20% mutated-but-similar positions (point edits still fork, #629), ≥1 byte-identical user message, unique candidate only. Default ON. `"affinitySimhash": false` in the config file does the same; the env var wins. | | `BILI_RESUME_INHERITANCE` | Set `0` to turn OFF resume inheritance (ON by default) (#1486): when an identified client (sends its own session id, e.g. Claude Code's `x-claude-code-session-id`) resumes a conversation under a NEW session id while replaying the full transcript (`cc --resume` forks a fresh UUID), bili matches the incoming history byte-exactly against that client's tracked chains (≥8 messages, append-only tracking) and, on the resumed session's first request, inherits the parent's ref assignments — stale model citations then resolve to their ORIGINAL messages instead of mis-hitting renumbered ones — adopts the fully-present compression blocks (together with this inheritance, #1834: losing them on resume meant the folded originals came back on the wire and the upstream request ballooned; the old `forkAdoption` co-gate now applies only to anonymous forks, #629), and records the `derivedFrom` lineage. The parent is never modified and fresh messages number above the parent's ref space. A resume must strictly EXTEND the parent's history — an equal-depth byte-exact replay under another id is a duplicate, not a resume. Anonymous sessions are unaffected (they keep their own pfa-* world, #309). `"resumeInheritance": false` in the config file does the same; the env var wins. | | `BILI_STABLE_SYSTEM_ANCHOR` | Set `1` to enable stable-system anchoring (#1085) — a **best-effort wire-layer fallback** for prefix caching: the root fix belongs client-side (the client owns its history and decides how to present instruction changes), this only stops the proxy from letting a changed head invalidate the whole cached prefix. **Plain-proxy mode only**: plugin-mode agents (`x-bili-plugin`) own their context management and are never anchored, so clients that already inject cache-friendly updates (e.g. claude-code-style system reminders) are not double-processed. When enabled, bili remembers each session's first-seen head system/instructions block and keeps re-sending those exact bytes even when the client's system prompt later changes. A **localized** change (a file-style edit sharing ≥70% of lines with the version in effect so far) appends a trailing `[System context update] …` user note carrying a compact line diff (`-` removed / `+` added; each note composes sequentially onto the previous one). A **non-localized** change (structural reshuffle, tool-def churn, timestamped banners, heads over 400 lines) is adopted outright — one deliberate cache miss beats appending noise that would mislead the model about its instructions. Churn guard: more than 8 accumulated notes likewise replace the anchor with the newest text and clear the log. The anchor and note log persist with the session and survive compression/compaction (session metadata, not kernel state). Known residual limitation: client-placed `cache_control` breakpoints may still misalign after a head swap. Excluded from anchoring: title-gen micro-requests (OpenAI/Google), Responses compaction-trigger requests, auto-mode classifier requests. Clients implementing their own variant (stable prompt + in-history updates) get zero extra injection — such updates pass through as ordinary history. Default is OFF. `"stableSystemAnchor": true` in the config file does the same; the env var wins. | -| `BILI_ALLOW_DSH_COMPACTION` | Set `1` to let dsh's built-in auto-compaction run through bili (#2028). By default the wire-level guard (#1729) **refuses dsh native compaction calls locally** (422) on every lane bili serves — openai, anthropic, and responses (responses added by #2360; it was the lane the pre-#2360 whitelist short-circuited before the marker was ever examined, so dsh desktop's compaction calls passed silently): dsh's `compaction-basic` answers context pressure by replaying the conversation prefix plus a fixed summarization directive as the final user message, and when such a call LANDS its checkpoint durably shadows the raw history — irreversible, and it destroys the proxy's compression substrate. This switch lifts the refusal so dsh-native compaction can actually execute. Scope note: on non-web profiles the shipped bundle patch (`auto: false`) still suppresses AUTO-triggering, so only manual `/compact` benefits there; on web profiles no patch layer can address the preset-nested row by id (#1772), so auto-triggering works as-is once allowed unless you apply the #2474 full-snapshot override of preset-standard from your profile's cordis.patch.yml. The web config page exposes the same switch; the env var wins over the file. Set `"dsh": { "allowDshCompaction": true }` in the config file to the same effect (a bare top-level `"allowDshCompaction"` in an older file is auto-relocated there on load). | +| `BILI_ALLOW_DSH_COMPACTION` | Set `1` to let dsh's built-in auto-compaction run through bili (#2028). By default the wire-level guard (#1729) **refuses dsh native compaction calls locally** (422) on every lane bili serves — openai, anthropic, and responses (responses added by #2360; it was the lane the pre-#2360 whitelist short-circuited before the marker was ever examined, so dsh desktop's compaction calls passed silently): dsh's `compaction-basic` answers context pressure by replaying the conversation prefix plus a fixed summarization directive as the final user message, and when such a call LANDS its checkpoint durably shadows the raw history — irreversible, and it destroys the proxy's compression substrate. This switch lifts the refusal so dsh-native compaction can actually execute. While the guard refuses, dsh's own context-pressure counter (its raw ledger) only grows — refused checkpoints never land — while bili's folds keep the wire payload small, so the session keeps working past the point where the client's gauge says "full", and dsh keeps re-requesting compaction (refused every time). Since #3384 the outgoing output budget is also no longer planned against that raw ledger, EXCEPT when bili's own folded view has already reached the enforced headroom target or the requested budget is >= 25% of the native window (the #2490 protection regimes) — before that fix the growing ledger silently shrank the model's output budget every turn, ending in error-free empty replies on onlyReasoning models. Scope note: on non-web profiles the shipped bundle patch (`auto: false`) still suppresses AUTO-triggering, so only manual `/compact` benefits there; on web profiles no patch layer can address the preset-nested row by id (#1772), so auto-triggering works as-is once allowed unless you apply the #2474 full-snapshot override of preset-standard from your profile's cordis.patch.yml. The web config page exposes the same switch; the env var wins over the file. Set `"dsh": { "allowDshCompaction": true }` in the config file to the same effect (a bare top-level `"allowDshCompaction"` in an older file is auto-relocated there on load). | +| `BILI_ALLOW_DSH_COMPACTION` | Set `1` to let dsh's built-in auto-compaction run through bili (#2028). By default the wire-level guard (#1729) **refuses dsh native compaction calls locally** (422) on every lane bili serves — openai, anthropic, and responses (responses added by #2360; it was the lane the pre-#2360 whitelist short-circuited before the marker was ever examined, so dsh desktop's compaction calls passed silently): dsh's `compaction-basic` answers context pressure by replaying the conversation prefix plus a fixed summarization directive as the final user message, and when such a call LANDS its checkpoint durably shadows the raw history — irreversible, and it destroys the proxy's compression substrate. This switch lifts the refusal so dsh-native compaction can actually execute. Scope note: with bili ON, dsh native auto-compaction is now off on every plane — the host-plane row via the shipped bundle patch (`auto: false`), and web-profile preset instances at load time via the #3354 overlay — so this switch mainly governs manual `/compact` on non-web profiles (auto-triggering stays suppressed there) and any residual auto-triggering on dsh builds where the #3354 overlay does not apply. The web config page exposes the same switch; the env var wins over the file. Set `"dsh": { "allowDshCompaction": true }` in the config file to the same effect (a bare top-level `"allowDshCompaction"` in an older file is auto-relocated there on load). | | `BILI_NO_CACHE_CONTROL` | Set `1` to disable bili's Anthropic-lane `cache_control` breakpoint stamping (#1637, shipped in #1639). On by default: Anthropic-style upstreams cache only what is explicitly breakpointed (max 4 per request, counted across system + tools + message blocks combined), so bili marks the system block plus up to 3 cumulative message marks — a marked message stays marked (byte-stable prefix), marks die only by folding, and the last 3 stable messages carry the advancing frontier. Any client-set `cache_control` (message blocks, tools entries) suppresses bili's stamps entirely — client-managed caching wins. Marks persist with the session. This switch is the escape hatch for upstreams that reject the field or relays with their own breakpoint policy. Plain-proxy Anthropic lane only; OpenAI/Responses lanes cache implicitly and are never stamped. Config-file twin: [`compat.noCacheControl`](#compat) — the env var wins. | | `BILI_CHAIN_CONTENT` | Set `1` to turn ON the ACP-artifact / `` checkpoint **body-content** detection of bili→bili chain awareness (#1086/#1421): when an inbound request carries compression artifacts (render tags / historical `acp_status`+`search_context` tool calls) or a digest-bearing checkpoint in its BODY but neither the `x-bili-hop` header nor local compression state for its session, bili records an advisory observation and/or applies first-processor-wins passthrough. **Default is OFF** (#1683 follow-up): by default ONLY the `x-bili-hop` header drives chain recognition, because scanning the request BODY can false-positive on CCR/file-introduced text and model-echoed tags that look like real markers. Enable it only for the narrow multi-bili relay case where an intermediate box strips `x-bili-hop` and you accept that false-positive risk. `"chainContentDetection": true` in the config file does the same; the env var wins. The `x-bili-hop` signal itself is unaffected by this switch. | | `BILI_CHAIN_STAMP` | Set `1` to turn ON egress emission of the **model-visible** `` chain-integrity checkpoint carrier (#1683, default OFF): when enabled, every request THIS instance actually processes leaves with a digest-bearing stamp so a downstream bili applies first-processor-wins even when the `x-bili-hop` header was stripped in transit (#1421). The carrier lands in a slot the terminal model also reads (a trailing `user` message on OpenAI/Responses, a trailing text part on Anthropic/Google), so models treat it as phantom user input and burn tokens commenting on it — which is why it is off by default. Enable it only for the narrow multi-bili relay case where `x-bili-hop` is stripped by an intermediate box and the digest-verified body-stamp is the sole guard against double-processing. Independent of `BILI_CHAIN_CONTENT` (inbound body-detection is also default OFF) and of the `x-bili-hop` passthrough, which works either way. `"chainEgressStamp": true` in the config file does the same; the env var wins. | @@ -1789,7 +1836,7 @@ File keys resolve only when the matching env var is unset. Defaults in parenthes | `BILI_PERSIST_EPERM_ALERT_REPEAT_MS` | Re-alert window for the persist EPERM alert, in ms. `0` (default) = alert once then stay silent; `>0` = re-alert at most every that many ms while the failures continue. | | `BILI_TUNNEL_ALLOWED_HOSTS` | `/bili/` tunnel admission for **remote clients** (#409): comma-separated `host` or `host:port` entries that unlock loopback/private destinations (e.g. a LAN relay or the machine's own sglang) for non-loopback clients. The proxy itself and link-local/metadata addresses are always denied; local (loopback) clients always pass. | | `BILI_MAX_SESSIONS` | Max sessions held in memory (default `256`; LRU eviction — disk is the source of truth). | -| `BILI_SESSION_WARM_MAX` | Bounded startup (#2803): max most-recently-saved sessions hydrated at boot before listening (default `8`, capped by `sessions.max`); older state reloads on demand. `0` warms none. File twin: `sessions.warmMax`. | +| `BILI_SESSION_WARM_MAX` | Bounded startup (#2803): max most-recently-saved sessions hydrated at boot before listening (default `0` = warm none, capped by `sessions.max`); older state reloads on demand. Set e.g. `8` to restore eager warming of the recent set. File twin: `sessions.warmMax`. | | `BILI_SESSION_WARM_MAX_BYTES` | Decoded-byte budget for the boot warm set (#2803; default `67108864` = 64 MiB). Counts DECODED size, not compressed on-disk bytes, so zstd archives cannot sneak past the RAM bound. A single file over the budget is never warmed. File twin: `sessions.warmMaxBytes`. | | `BILI_SESSION_RESIDENT_MAX_BYTES` | Soft target for estimated resident session bytes (#2803; default `536870912` = 512 MiB). Active/in-flight state stays pinned above it (the byte budget limits the idle cache, not active workload); when over target, idle state discharges back to disk as requests complete. File twin: `sessions.residentMaxBytes`. | | `BILI_PUBLIC_SNAPSHOT_CAP_BYTES` | Retention cap (bytes) for the raw wire-history snapshot kept per plugin session to serve the public fork API (#2017). A plugin session whose serialized snapshot exceeds the cap stops being forkable — `GET /__bili/plugin/snapshot` and `POST /__bili/plugin/fork` fail closed with `409` and the capped reason — instead of retaining an unbounded raw copy of the history forever. The cap is re-evaluated on every plugin model request: a session that shrinks back under the cap (after a fork trimmed it, or the host shrank the history) resumes being forkable. Default `104857600` (100 MiB); `0` disables retention entirely (no session is forkable; existing snapshots are dropped on the next request). File twin: `plugin.snapshotCapBytes`. | @@ -2133,13 +2180,13 @@ Two compressors on one conversation double-compress and corrupt message refs, so - **Runtime evidence, tiered for display (#2545):** unannounced history rewrites (#1001) and orphan-gc deactivations (summarized content deleted out of the client's history) are recorded in the same ledger as *suspected* coexistence. Severity surfaces grade by EVIDENCE TIER: **confirmed** = a detected client-native compaction landing (bili rebases onto it, #2372/#2432) or a non-suspected plugin finding — only this triggers the imperative "keep one compressor" warning; **unconfirmed signals** = unannounced rewrites + orphan-gc deactivations — observed changes whose cause is NOT identified ("history changed, cause unknown"), never presented as native auto-compaction or a second compressor; name-only [suspected] matches stay in the soft "verify first" tier (#1736) and a PURE [suspected] ledger gets verify-first advice only — no imperative per-client fix footer ("disable auto-compaction / start a fresh session") while nothing is confirmed (#2969). Stock ledger records naming verified non-compressing plugins (read-only viewers, or protocol adapters such as #2969's ACP adapter) are neutralized at display time instead of persisting as alarms, and cross-session aggregate counts of unconfirmed signals never escalate to a same-conversation double-compression claim. - **dsh's `auto: false` disables only the automatic triggers.** The `compaction-basic: { auto: false }` line installed by the profile bundle patch (`dsh.bundle.patch.yml`) skips pressure/overflow auto-compaction — a manual `/compact` (and idle-session compaction) still fires. Calls routed through bili are refused by the server-side guard (#1729/#2360); calls that reach the provider without transiting bili (desktop-lane paths the plugin's takeover gate does not attribute) land, and bili detects them on the next replay — checkpoint framing + decimated fold coverage — rebasing its compression state in one turn instead of failing every later compress forever (#2432). -#### Turning dsh native auto-compaction OFF in a web profile (#1772/#2474) - -The bundled `dsh.bundle.patch.yml` cannot do this: in a profile bundling `@deepseek-ai/dsh-web-app` its `- id: compaction-basic` row lands on web-app's **host-plane row, which is already `disabled: true`**, while the instance that actually runs lives inside `preset-standard.config.plugins`. The patch applies silently and changes nothing. - -A profile-layer patch **can** turn it off — presets are ordinary top-level rows inserted by the bundle layer, and the profile patch layer is applied after it. What does not work is addressing the nested row by id; what does work is overriding the preset row's `config` wholesale. - -**Recipe** (verified on dsh `0.2.1-alpha.1`, Windows 11, profile bundles including `@deepseek-ai/dsh-web-app`): + #### Turning dsh native auto-compaction OFF in a web profile (#1772/#3354, automatic since #3354; #2474 recipe as fallback) + + **Automatic since #3354.** In a profile bundling `@deepseek-ai/dsh-web-app` the running compaction-basic instance lives inside an agent preset (`preset-standard.config.plugins`) where no patch layer can address the nested row by id (#1772) — the bundled `- id: compaction-basic` row lands on web-app's already-disabled host-plane row and changes nothing. bili now closes that gap at LOAD TIME: the `bili-native` bundle entry registers a cordis `internal/config` overlay (vendored from [deshark42/dsh-compaction-basic-disable](https://github.com/deshark42/dsh-compaction-basic-disable), MIT) that marks the preset's `compaction-basic` and `command-compact` rows `disabled: true`, and the preset gate rows in `dsh.bundle.patch.yml` / the launcher overlay make each Web preset wait for that overlay before it can register. Native auto-compaction is off in these profiles with no user action; ACP compression is unaffected. + + The overlay rides an INTERNAL dsh hook and is version-sensitive. If you still see a dsh compaction checkpoint land on a newer/older dsh build, use the fallback below. + + **Fallback recipe (#2474)** (verified on dsh `0.2.1-alpha.1`, Windows 11, profile bundles including `@deepseek-ai/dsh-web-app`). A profile-layer patch can turn it off — presets are ordinary top-level rows inserted by the bundle layer, and the profile patch layer is applied after it. What does not work is addressing the nested row by id; what does work is overriding the preset row's `config` wholesale. ```bash # 1. Dump the composed profile and copy preset-standard's COMPLETE config block. diff --git a/CONFIGURATION.zh-CN.md b/CONFIGURATION.zh-CN.md index da4ec4580e..f45422c0ca 100644 --- a/CONFIGURATION.zh-CN.md +++ b/CONFIGURATION.zh-CN.md @@ -87,6 +87,7 @@ | `modelContextLimit` | number | 200000 | ACP_MODEL_CONTEXT_LIMIT | 旧版全局窗口上限(窗口来源中优先级最高,兼作使用率分母);建议改用 compress.modelContextLimit。 | | `imageBilling` | "auto" \| "pixels" \| "bytes" | auto (resolves to pixels) | BILI_IMAGE_BILLING | 全局图片 token 估算口径;逐请求实时读取,优先于所有逐 provider 设置。 | | `imageTokenCap` | number | unset (uncapped) | BILI_IMAGE_TOKEN_CAP | 全局单图 token 上限,叠加在计费口径之上生效;逐请求实时读取。 | +| `imageRescale` | boolean | true | BILI_IMAGE_RESCALE | 出站图片降采样总开关(#3295/#3400):大图以缩小形态发给模型,原图在会话存储中保持字节级原样。逐请求实时读取。 | **行为开关** @@ -100,6 +101,7 @@ | `rollout.features.salt` | string | the feature name | — | 分桶哈希盐。修改它会重新随机整个分组(唯一的洗牌方式;仅上调 percent 永不洗牌)。 | | `rollout.features.base` | string | unset (roll out over everyone) | — | 嵌套灰度:仅在所指特性的已入组人群内评估本特性,percent 相对该人群(B 只放量给 A 的用户;允许链式,环路判为关)。 | | `maskHosts` | boolean | true | BILI_LOG_MASK_HOSTS | 日志中把非公开目标主机遮成 ;凭据头无论此开关如何总是遮盖。 | +| `security.adminRelay` | boolean | false | — | 允许宿主(当前为 dsh web)把它自己入口域名下的整个 /__bili/ 管理面中继到本机代理,使远程浏览器能打开面板(#3373)。默认关闭、仅配置文件。中继跳到达代理时是本机非浏览器客户端(回环源地址 + 回环 Host、无 Origin 头),管理端门槛不变;开启后,凡能访问宿主 web 入口的人都能改配置、写凭据——请先给入口加鉴权/反代。 | | `subagentSplit` | boolean | true | BILI_SUBAGENT_SPLIT | Claude 子代理拥有独立会话命名空间(\|sub:),不再排队在主会话锁后。 | | `forkAdoption` | boolean | false | BILI_FORK_ADOPTION | 匿名 fork 会话继承父会话中源内容完整存在于本次请求里的压缩块。 | | `autoModeForge.enabled` | boolean | true | BILI_AUTO_MODE_FORGE | Claude Code auto mode 安全分类器请求由 bili 在本地代答放行判定,不再转发上游(#2740):该旁路零流出、零计费;每条代答写审计日志,会话带提示徽标。false / env 0 恢复纯透传(愿意付费要真判定的用户)。 | @@ -170,7 +172,7 @@ | `persist.epermAlertRepeatMs` | number | 0 (no repeats) | BILI_PERSIST_EPERM_ALERT_REPEAT_MS | EPERM 告警的重复间隔;0 只记录一次。 | | `storage` | object | {} (all defaults) | — | 统一 SQLite 存储边车设置(#2671)。 | | `storage.unified` | legacy\|shadow\|full | unset (shadow) | BILI_STORAGE_UNIFIED | 分层存储迁移开关(#2671):shadow = 统一 SQLite 双写干跑 + digest 对账;full = 平替读路径(unified-first 加载 + 有机导入 + legacy 兜底)。两处均未设置时默认 shadow(JSON 仍是唯一事实源,索引只做对账遥测 —— 设 legacy 可退出)。文件层只接受 legacy/shadow/full;环境变量层还接受 1/true(=shadow)与 unified(=full)并优先于文件。每个 lane 启动时解析一次 —— 热改动需重启生效。加密(BILI_ENCRYPTION_KEY)会硬阻断边车。 | -| `storage.sqliteAutoDownload` | boolean | true | BILI_SQLITE_AUTO_DOWNLOAD | SQLite 引擎一次性自下载的闸门(#2833):进程内没有可用引擎(无内置 node:sqlite、宿主未装 better-sqlite3)且本平台希望启用统一边车时,bili 从 npm registry 一次性下载钉死的 better-sqlite3 预编译 tarball(约 11 MB,sha512 校验)到缓存目录,之后每次启动复用。引擎解析顺序:内置 node:sqlite(Node >= 22.5)-> 宿主机已装的 better-sqlite3 -> 缓存的自下载副本 -> 自下载(本开关控制)-> 无引擎(仅 legacy,即 #2825 降级契约)。开关只控制网络下载 —— 为 false 时缓存副本仍会使用;不支持的平台永不下载。 | +| `storage.sqliteAutoDownload` | boolean | true | BILI_SQLITE_AUTO_DOWNLOAD | SQLite 引擎一次性自下载的闸门(#2833):进程内没有可用引擎(无内置 node:sqlite、宿主未装 better-sqlite3)且本平台希望启用统一边车时,bili 从 npm registry 一次性下载钉死的 better-sqlite3 预编译 tarball(约 11 MB,sha512 校验)到缓存目录,之后每次启动复用。引擎解析顺序:内置 node:sqlite(Node >= 22.5)-> 宿主机已装的 better-sqlite3 -> 缓存的自下载副本 -> 自下载(本开关控制)-> 无引擎(仅 legacy,即 #2825 降级契约)。开关只控制网络下载 —— 为 false 时缓存副本仍会使用;不支持的平台永不下载。 tarball 下载内置镜像列表兜底(#3338):先试配置的 registry(BILI_UPDATE_REGISTRY / update.registry),再试 https://registry.npmjs.org、https://registry.npmmirror.com —— 第一个通过 sha512 校验的镜像生效;Node < 22.5 且无引擎时,降级原因会给出明确的 Node 升级提示(#3385)。 | | `storage.jsonWriter` | on\|off | on | BILI_STORAGE_JSON_WRITER | Phase 3 信封开关(#2960):off 时若 storage.unified 为 full 则不再写 legacy JSON 会话信封 —— SQLite 索引成为唯一持久化副本(读路径本就 unified-first;boot 枚举索引并集,纯索引会话可正常加载)。仅在有效 full 模式下生效:shadow/legacy(或加密激活)下请求会被忽略并警告一次,因为 JSON 文件仍是唯一事实源。环境变量层 0/false/off = off、1/true/on = on,优先于严格拼写的文件层。失败安全:若边车运行中自禁用,下次保存即恢复信封写入。回滚:去掉该开关 —— 写入恢复,纯索引会话经 boot 并集继续可加载,replicas/restore 可重新物化可移植的 JSON 信封。降级二进制(或丢失 SQLite 引擎)会让纯索引会话在 v0.1.196 及更早版本上不可见,直到重新运行 >=v0.1.197 —— 数据在 index.db 中完好无损,只是旧代码不读它。每会话的 content-store 边车无论开关如何都照写。jsonWriter=off 的 lane 必须是其会话目录的唯一写者:off lane 读路径 unified-first、永不回退到文件,因此同一目录上跑 legacy/shadow 的对等实例会静默分叉(它的 JSON 副本收不到更新,off lane 也看不到对等实例的写入)。 | | `storage.walJournalSizeLimitBytes` | number | 8388608 | BILI_STORAGE_WAL_JOURNAL_SIZE_LIMIT_BYTES | index.db-wal 的 WAL 回收上限(#3250),单位字节:每次完成的 checkpoint 把 WAL 文件截回该大小,约束长驻进程的磁盘占用与崩溃恢复重放。不是每会话数据配额 —— 数据写入时就已落进 index.db,WAL 只是两次 checkpoint 之间的帧暂存区,会话数量/大小不会放大它;大 burst 可能暂时超出,直到下一次完成的 checkpoint 截回。调大 = truncate IO 更稀疏,但稳态磁盘占用更多、崩溃恢复重放尾更长。环境变量层(正整数)优先于配置文件层;env 值非法或非正数时静默落到文件/默认层,boot 不崩。lane 启动、store open 时生效 —— 热改动需重启。实际取值会出现在 unified-enable 日志行与 storage 诊断块中。 | | `sessions` | object | {} (all defaults) | — | 会话表容量上限与可选垃圾回收。 | @@ -248,10 +250,11 @@ | `compress.neverPreserveRecentTools` | string[] ([] valid) | ["decompress", "search_context", "read", "bash"] (kernel) | — | 从近期保护区排除(立即可压);空数组合法=不排除任何工具(最大保护)。 | | `compress.preserveRecentTools` | string[] | n/a (subtraction form) | — | 减法形式:近期区工具减去本列表得到完全保护;此处空数组按笔误拒绝。 | | `compress.visibilityMarkers` | boolean | true | — | 在 compress/decompress/search_context/acp_status 结果后追加可见性标记;关闭可抑制模型模仿叙述。 | -| `compress.rules` | boolean | false | — | 通过注入的 acp_rule 工具提供持久模型提醒——对折叠硬保护;pi/omp 提供 /acp-rule 命令。 | +| `compress.rules` | boolean | false | — | 通过注入的 acp_rule 工具提供持久模型提醒——对折叠硬保护;pi/omp 提供 /acp-rule 命令。仅代理模式生效——插件模式下宿主自管工具面,此项无效。 | | `compress.injectTool` | boolean | true | ACP_COMPRESS_TOOL | 向客户端注册 acp_compress 工具(仅全局层级生效)。 | | `compress.injectNudge` | boolean | true | ACP_COMPRESS_NUDGE | 窗口填满过程中发送增长提醒(仅全局层级生效)。 | | `compress.reasoning` | { drop?, threshold? } | drop true · threshold 2048 | — | 丢弃超过 2048 字符的已结束轮次推理块(drop 默认 true);严格推理上游需设 drop:false。 | +| `compress.absorb` | object | opt-in (disabled) | — | 可选即时蒸馏:把工具结果蒸馏成短摘要,原文进内容库。默认仅图片类结果参与。 | | `compress.ccr` | object | enabled in proxy mode since v2 | — | 内容缓存与回取:大输出无损存到会话旁、替换为首段摘录+指针,模型用 acp_retrieve 取回原文;无上限、永不清除。 | | `compress.search` | object | off | — | search_context 的检索行为设置;当前含 planAware 重排。 | | `compress.search.planAware` | boolean | false | — | 开启后 search_context 候选按当前计划状态重排;关闭时结果逐字节不变。 | @@ -260,6 +263,12 @@ | `compress.outputSteering` | object { enabled?, verbosityLevel?, effortRouting? } | enabled false · verbosityLevel 2 | — | 向系统提示词尾部附加简洁度指令;只收紧机械延续请求。 | | `compress.priceProfile` | { w?, r?, q? } | unset (registry, kernel fallback {1, 0.1, 4}) | — | w/r/q 相对输入价的比率(写入/缓存读/输出);仅供报告,不影响触发条件或线上行为。 | | `compress.acknowledgePromptsRisk` | boolean | false | — | 必须先置 true,自定义 prompts 才会生效。 | +| `compress.absorb.enabled` | boolean | false | — | 启用 absorb 蒸馏块。 | +| `compress.absorb.minToolTokens` | number | 1000 | — | 可被吸收的工具结果的最小估算 token 大小。 | +| `compress.absorb.contextThresholdPct` | number \| % | unset | — | 仅当上下文使用率超过窗口该占比时才吸收。 | +| `compress.absorb.excludeTools` | string[] | [] | — | 排除出吸收范围的工具名。 | +| `compress.absorb.toolName` | string | "absorb" | — | absorb 工具的注册名。 | +| `compress.absorb.mediaOnly` | boolean | true | — | 仅吸收携带媒体(图片)的工具结果;图片字节不计入文本 token,故豁免 minToolTokens 门槛。置 false 恢复对任意大结果的全文吸收(旧行为)。 | | `compress.ccr.enabled` | boolean | true (proxy mode) | — | 启用内容缓存与回取块。 | | `compress.ccr.minToolTokens` | number | ≈4000 | — | 可进入 CCR 存储的工具结果的最小估算 token 大小。 | | `compress.ccr.excludeTools` | string[] | [] | — | 排除出 CCR 存储的工具名。 | @@ -367,6 +376,7 @@ | `BILI_MCP_NO_ORPHAN_ADOPT` | boolean-ish | off (=1 disables orphan adoption) | — | 多个宿主会话共享一个代理时,退出恢复会话的孤儿收养。 | | `BILI_MITM_HOSTS` | string[] (csv) | unset | — | 启动器通道:传给被派生代理的 MITM 白名单主机(区别于配置文件对应项 BILI_MITM_DOMAINS)。 | | `BILI_PLUGIN_AGENT` | boolean-ish | off | — | 派生通道:标记插件 agent 进程。 | +| `BILI_IMAGE_RESCALE_MAX_DIM` | integer | 1600 | — | 出站图片降采样 pass 的最长边触发阈值(像素,#3400);非数字或低于 128 的取值按默认处理。设计上仅环境变量。 | @@ -388,7 +398,7 @@ - **类型:** `string` - **默认值:** `127.0.0.1` - **状态:** ACTIVE -- **说明:** 代理绑定的网络接口。`127.0.0.1`(默认)仅监听本机 —— 适合本地 sidecar。使用 `::` 可同时监听 IPv4 + IPv6 双栈。使用 `0.0.0.0`(或局域网 IP)可将代理暴露给其他机器 —— 常见于容器或可信局域网: 远程 agent 把模型 `baseURL` 指向 `http://<本机>:<端口>/bili/…`,MITM 模式的 `CONNECT` 仅对白名单内的模型域名接受远程客户端(盲隧道仍仅限本机,`/__bili/` 管理端点也仍仅限本机)。没有任何鉴权 —— 请确保所在网络可信。可由 `ACP_HOST` / `--host` 覆盖。 +- **说明:** 代理绑定的网络接口。`127.0.0.1`(默认)仅监听本机 —— 适合本地 sidecar。使用 `::` 可同时监听 IPv4 + IPv6 双栈。使用 `0.0.0.0`(或局域网 IP)可将代理暴露给其他机器 —— 常见于容器或可信局域网: 远程 agent 把模型 `baseURL` 指向 `http://<本机>:<端口>/bili/…`,MITM 模式的 `CONNECT` 仅对白名单内的模型域名接受远程客户端(盲隧道仍仅限本机,`/__bili/` 管理端点也仍仅限本机,除非开启了下方的宿主中继 security.adminRelay)。没有任何鉴权 —— 请确保所在网络可信。可由 `ACP_HOST` / `--host` 覆盖。 ### `upstream` @@ -524,6 +534,13 @@ - **状态:** ACTIVE - **说明:** 预检尺寸门、输出钳制与图片压缩统计所用单图 token 估算的统一天花板(#488/#496/#1843)。叠加在任何计费模式之上 —— 适用于路由真实编码器计费远低于像素先验的场景。优先级:`BILI_IMAGE_TOKEN_CAP` 环境变量(实时读取,无需重启)> 按 provider 的 `providers..imageTokenCap` > 本全局项。非数字或非正值按宽松解析丢弃(与 `imageBilling` 一致)。 +### `imageRescale` + +- **类型:** boolean +- **默认值:** `true` +- **状态:** ACTIVE +- **说明:** 出站图片降采样 pass 的总开关(#3295 删除了旧的入站时降采样;#3400 以简化形态重新引入)。每个出站请求在 kernel 折叠之后、协议编码之前,对最长边超过 `BILI_IMAGE_RESCALE_MAX_DIM`(默认 1600)**或** base64 体积超过 100 KB 的图片做 fit-inside 重编码(不放大、保留原格式),且只作用于**出站副本**——会话存储中的原始字节保持逐字节原样。重编码节省不足 20% 字节的结果直接丢弃(keep-smaller),小图与低收益图原样通过。仅处理 PNG/JPEG/BMP(纯 JS 编码器,零原生依赖);其他格式原样通过。每张被缩小的图片旁附一段稳定注释,报告当前与原图尺寸;在注册 bili 工具的车道(代理注入车道,以及经插件 manifest 注册的插件车道)中还会指明 `read_original_image("")`——它从本地会话存储取回原始字节;拒绝注入工具的车道(如 codex)只拿到报尺寸的注释、没有工具。`read_original_image` 的结果**结构性豁免**再次降采样(防 #3266「猜→拒→重发」循环复发的环锁)。输出确定性:同一输入字节永远产出同一缩小字节与同一注释文本,同一历史反复重发保持逐字节稳定,上游前缀缓存不受扰动。每张唯一图片记一条日志(内容 hash 缓存;缓存命中静默)。优先级:`BILI_IMAGE_RESCALE` 环境变量(`"0"`/`"false"` 关闭;实时读取,无需重启)> 本配置键 > 默认开启。 + ### `resign` - **类型:** `object` —— 按签名方案分键:`Record<方案, { enabled?, passthrough?, credentialRef? }>`,键 = 电线上检测到的签名方案 token(小写;内置:`"sdk-hmac-sha256"`;自定义:HMAC `Authorization` token 如 `"aws4-hmac-sha256"`,或 body 签名头名如 `"x-ofm-signature"`) @@ -648,6 +665,19 @@ - **状态:** ACTIVE - **说明:** release notes 文档 URL 覆盖。可由 `BILI_RELEASE_NOTES_URL` 覆盖。 +### `security` + +- **类型:** 对象 +- **默认:** 各键均不设置(全部 fail-closed) +- **状态:** 生效中 +- **说明:** 安全姿态开关 —— 打开会刻意扩宽信任边界的地方,集中在一个块里便于审计。仅配置文件生效:本块任何键都没有 `BILI_*` 环境变量孪生(#2030)。 + +#### `security.adminRelay` (#3373) + +- **类型:** `boolean` +- **默认:** `false`(fail-closed) +- **说明:** 允许宿主 —— 当前是 dsh 的 web 面板 —— 把整个 `/__bili/` 管理面通过宿主自己的 web 入口中继到本机代理,让另一台机器上的浏览器无需代理绑定非回环网卡就能打开面板。无头部署:跑 `dsh web`,把反向代理指向它,在任何地方打开 bili 标签页 —— 设置 iframe 和它的 API 调用与入口同源,走宿主中继。机制:dsh 侧的 bili 插件注册 `/__bili` 前缀路由转发到回环代理(`127.0.0.1:18787` 区),`/bili/origin` 开始上报 `{"relay":true}`,面板 UI 据此改用同源 URL,面板自己的 API 根从页面路径推导 —— 前缀网关(`/app/dsh/…`)也能工作。管理端门槛本身不变(#1538):中继跳到达代理时是本机非浏览器客户端 —— 回环源地址、回环 `Host`、不携带 `Origin`/`Cookie`/`Referer` —— 与现有检查早已放行的形状完全一致。**安全提示:** `/__bili/` 是完整管理面而非只读仪表盘 —— 包含改配置与写凭据。开启此开关意味着凡能访问宿主 web 入口的人都能触及这些能力;开启前请先给入口加鉴权 / 反向代理。逐请求实时读取:改文件对下一次调用即生效,无需重启。无环境变量 —— 刻意的姿态决策属于文件。 + ### `maskHosts` - **类型:** `boolean` @@ -1342,12 +1372,25 @@ ACP 原生 agent(当前为 `pi` 扩展)会在每个进程内向代理上报 - **状态:** ACTIVE - **说明:** 控制**折叠状态和解**(#1921):当客户端重发的历史与已压缩内容发生漂移时怎么处理 —— 例如 fork 重新序列化了早前轮次(#1908)、客户端原地改写消息字节、或 provider 规范化空白。默认情况下消息身份是内容哈希,一字节漂移就会让受影响消息从所有压缩块中静默脱落,原文重回线上(“折叠漂移”)。`"repair"` 模式在折叠运行前逐条分类漂移:稳定的工具调用锚(`tool_use_id`)与规范化身份锚(NFC 文本、CRLF/空白折叠)把压缩块重锚到新消息 id 上,折叠在客户端漂移下存活、字节等价于一次干净重发;真正被**编辑**的消息(规范化文本不同,或长度漂移超过 `max(256, 25%)`)则诚实地重回线上(绝不静默改写)——由下一轮压缩重新折叠。`"warn"` 运行同一分类器但只记录发现;`"off"` 完全关闭该层(#1921 之前的行为)。环境变量 `BILI_FOLD_RECONCILE` 可为单个进程覆盖配置值。锚元数据按会话持久化(会话记录中的 `foldAnchors`,上限 16384 条),覆盖每个块实际引用的消息。系统提示词在无消息漂移时变化会记录一行 `info`(system-only 漂移永不影响折叠状态)。 +#### `absorb` + +- **类型:** `object`(`{ enabled?, minToolTokens?, contextThresholdPct?, excludeTools?, toolName? }`) +- **默认值:** *(禁用 — 除非显式设置 `enabled: true`,该特性完全关闭)* +- **状态:** ACTIVE +- **说明:** 可选开启的**即时工具结果压缩**(issue #605,经由 `acp-kernel` absorb API)。启用时,大工具结果在到达即被附带强制的 `[ACP absorb]` 指令;模型通过 `absorb` 工具将结果蒸馏为紧凑摘要,原 tool-call/tool-result 配对从下一轮起在线上隐藏 —— 使折叠轮之间的中间会话压力更低。子字段(按字段最深层级胜出,与其他 CompressSettings 字段一致): + - `enabled: boolean` — 主开关;任何值不为 `true` 时特性完全关闭(无工具、无提示、无标记)。 + - `minToolTokens: number` — 仅达到此 token 数的结果被附带提示(内核默认 1000)。 + - `contextThresholdPct: number|percent-string` — 仅当用量达到 `modelContextLimit` 的此比例时附带提示(`0` = 仅尺寸门槛;`"75%"` 接受)。 + - `excludeTools: string[]` — 永不吸收的工具名模式。**已知限制:对工具*结果*目前无效,直到 [ranxianglei/acp-kernel#213](https://github.com/ranxianglei/acp-kernel/issues/213) 修复发布**(wire 投影不把 `toolName` 携带在结果上,内核名称守卫无法命中)。 + - `toolName: string` — 重命名注入工具(默认 `"absorb"`);声明/注入的模式、系统提示段与按会话裁决在两条车道中都跟随该名称。**车道治理(#1359):** 插件模式用**基础**配置治理整个 `absorb` 块,因此重命名后的工具既以该名声明、也以该名执行(二者永不背离);provider/model 层的 `absorb.*` 覆盖**仅限代理车道**(代理注入并裁决合并后的名称)。加载时会输出一条警告,列出任何取值与基础值不同的 provider/model `absorb.*` 字段。 + 注入跟随线上原生工具面:代理模式在 anthropic/openai/responses 原生工具线上注入工具(按每请求解析的名称)+ 系统提示段,插件模式在插件清单中广告它(MCP shell 自动拾取)。Responses **marker/文本协议**路由不支持(无原生工具面 — 强制的 absorb 指令不可满足),标题生成请求(`max_tokens ≤ 200`)跳过注入如压缩提示一样。吸收配对在重启后保持隐藏(在会话状态持久化)。 + #### `ccr` - **类型:** `object`(`{ enabled?, minToolTokens?, excludeTools?, toolName?, maxHeadChars? }`) - **默认值:** *(全车道默认关闭(#1207 决策)— 未设置时保持关闭;在任意层级显式设置 `enabled: true` 方可启用(建议先本地验证)。插件通道同样需全局 `enabled: true` 才武装(#1271/#1273))* - **状态:** ACTIVE(v2 — 全车道 opt-in;插件通道限 anthropic + openai wire,显式开启后生效) -- **说明:** **内容寻址消息存储**/内置 CCR(issue #1097/#1179,经 `acp-kernel` CCR API,需 acp-kernel >= 0.0.84)。超大工具结果不再永远挂在线上:kernel 在到达时将其 ID 引用(ccr-store 节点位于 `processTurn` 内 prune 与 absorb 之间 —— ID 引用优先于蒸馏),线上保留一个确定性、字节稳定的占位符(`📦 [acp-stored #m00423 · shell output · 4,213 tok] \`npm run build\`\n → acp_retrieve("m00423") returns the full text`),原文进入该会话的内容存储。模型通过注入的 `acp_retrieve` 工具按需取回完整原文;retrieve 是临时的(走请求内工具结果通道,不进入折叠空间,不占用消息 ref)。默认无损:未执行的 retrieve 只花一次廉价工具调用。子字段(按字段最深层级胜出,与其他 CompressSettings 字段一致): +- **说明:** **内容寻址消息存储**/内置 CCR(issue #1097/#1179,经 `acp-kernel` CCR API,需 acp-kernel >= 0.0.84)。超大工具结果不再被强制蒸馏(如 `absorb`)或永远挂在线上:kernel 在到达时将其 ID 引用(ccr-store 节点位于 `processTurn` 内 prune 与 absorb 之间 —— ID 引用优先于蒸馏),线上保留一个确定性、字节稳定的占位符(`📦 [acp-stored #m00423 · shell output · 4,213 tok] \`npm run build\`\n → acp_retrieve("m00423") returns the full text`),原文进入该会话的内容存储。模型通过注入的 `acp_retrieve` 工具按需取回完整原文;retrieve 是临时的(走请求内工具结果通道,不进入折叠空间,不占用消息 ref)。默认无损:未执行的 retrieve 只花一次廉价工具调用;而被 absorb 蒸馏掉的细节则永久丢失。子字段(按字段最深层级胜出,与其他 CompressSettings 字段一致): - `enabled: boolean` — 主开关。**全层级均未设置时默认为 `true`**(#1179);任意层级显式 `false` 优先,特性完全关闭(无占位符、无工具)。插件模式武装还需全局显式 `true`(#1273)——插件清单只广播运维者显式开启的能力。 - `minToolTokens: number` — 仅达到此 token 数的工具结果被 ID 引用(kernel 默认 `4000`);更小的结果保持原样。 - `excludeTools: string[]` — 从不存储的工具名模式(允许 glob 后缀;kernel 默认为空)。 @@ -1369,7 +1412,7 @@ ACP 原生 agent(当前为 `pi` 扩展)会在每个进程内向代理上报 - **类型:** `boolean` - **默认值:** `false` - **状态:** ACTIVE -- **说明:** **持久化模型提醒**(issue [ranxianglei/billion-context-pi#433](https://github.com/ranxianglei/billion-context-pi/issues/433),经由 `acp-kernel` rules API),**默认关闭**——设 `compress.rules: true` 显式开启(#1399 定案:开启后模型对会话规则有全权,可未经提示自主调用,含 delete/clear)。启用后,一个 `acp_rule` 工具随 ACP 工具一同注入:传入简短的 `rule` 参数调用可记录一条**原则级提醒**,该提醒受**硬性保护**免于压缩(调用及结果在每次折叠中都保留在上下文中);省略参数则列出已记录的规则。传入 `delete`(规则 id,如 `"rule3"`)删除该条规则(返回 `Removed ruleN: `);传入 `clear: true` 清空全部规则。两者互斥,且均与 `rule` 互斥——一次调用只能执行一种操作——因此过时或误记的规则可以直接删除,而无需再写一条"作废声明"对冲。关于*何时*记录(用户强调的教训、用户要求记住的行为、遇到的重大陷阱)的指导完全写在工具描述里——不向系统提示词添加任何内容。受内核限制约束(50 条规则 × 每条 300 字符);重复文本会被拒绝并指向已有的 id。注入跟随线上原生工具面:代理模式在 anthropic/openai/responses 原生工具线上注入该工具,插件模式在插件清单中声明(执行按会话门控)。已记录的规则在会话状态中持久化,跨重启保留;删除同样持久化。要求 `acp-kernel` >= 0.0.70(delete/clear 需要 >= 0.0.84)。人也可以不经过模型执行全部操作:pi/omp 提供原生 `/acp-rule` 命令,语义与工具一致 —— 裸 `/acp-rule` 列出全部已记录规则,`/acp-rule <文本>` 直接记录一条,`/acp-rule remove ` 删除一条,裸 `/acp-rule clear` 清空全部(`clear <文本>` 是记录而非清空)(#1251;remove/clear 由 #1399 补齐)。 +- **说明:** **持久化模型提醒**(issue [ranxianglei/billion-context-pi#433](https://github.com/ranxianglei/billion-context-pi/issues/433),经由 `acp-kernel` rules API),**默认关闭**——设 `compress.rules: true` 显式开启(#1399 定案:开启后模型对会话规则有全权,可未经提示自主调用,含 delete/clear)。启用后,一个 `acp_rule` 工具随 ACP 工具一同注入:传入简短的 `rule` 参数调用可记录一条**原则级提醒**,该提醒受**硬性保护**免于压缩(调用及结果在每次折叠中都保留在上下文中);省略参数则列出已记录的规则。传入 `delete`(规则 id,如 `"rule3"`)删除该条规则(返回 `Removed ruleN: `);传入 `clear: true` 清空全部规则。两者互斥,且均与 `rule` 互斥——一次调用只能执行一种操作——因此过时或误记的规则可以直接删除,而无需再写一条"作废声明"对冲。关于*何时*记录(用户强调的教训、用户要求记住的行为、遇到的重大陷阱)的指导完全写在工具描述里——不向系统提示词添加任何内容。受内核限制约束(50 条规则 × 每条 300 字符);重复文本会被拒绝并指向已有的 id。注入跟随线上原生工具面,与 `absorb` 完全一致:代理模式在 anthropic/openai/responses 原生工具线上注入该工具,插件模式在插件清单中声明(执行按会话门控)。已记录的规则在会话状态中持久化,跨重启保留;删除同样持久化。要求 `acp-kernel` >= 0.0.70(delete/clear 需要 >= 0.0.84)。人也可以不经过模型执行全部操作:pi/omp 提供原生 `/acp-rule` 命令,语义与工具一致 —— 裸 `/acp-rule` 列出全部已记录规则,`/acp-rule <文本>` 直接记录一条,`/acp-rule remove ` 删除一条,裸 `/acp-rule clear` 清空全部(`clear <文本>` 是记录而非清空)(#1251;remove/clear 由 #1399 补齐)。 #### `reasoning` @@ -1646,6 +1689,7 @@ ACP 原生 agent(当前为 `pi` 扩展)会在每个进程内向代理上报 | `BILI_GUARDRAIL_REPETITION` | `guardrails.repetition` | true | | `BILI_GUARDRAIL_TEXT_RUN_THRESHOLD` | `guardrails.textRunThreshold` | 200 | | `BILI_IMAGE_BILLING` | `imageBilling` | auto (resolves to pixels) | +| `BILI_IMAGE_RESCALE` | `imageRescale` | true | | `BILI_IMAGE_TOKEN_CAP` | `imageTokenCap` | unset (uncapped) | | `BILI_KEEP_ALIVE_TIMEOUT_MS` | `network.keepAliveTimeoutMs` | 5000 | | `BILI_LOG_MASK_HOSTS` | `maskHosts` | true | @@ -1722,6 +1766,8 @@ ACP 原生 agent(当前为 `pi` 扩展)会在每个进程内向代理上报 | `BILLION_CONTEXT_NODE` | 非 Node 宿主进程拉起代理时显式指定的 Node 可执行文件路径(#819/#1429)。宿主自身不是 Node 时的解析顺序:本覆盖项 → PATH 搜索 + GUI PATH 会遗漏的众所周知安装位置(`/opt/homebrew/bin`、`/usr/local/bin`、Volta 等)→ Electron 宿主自身二进制以纯 Node 运行(`ELECTRON_RUN_AS_NODE=1`,最后手段 —— 因此 deepseek-harness desktop 这类桌面应用零配置即可用)。用于强制指定某个 Node(如比宿主捆绑运行时更新的版本);始终优先于上述回退。 | | `BILI_IMAGE_TOKEN_CAP` | 预检尺寸门、输出钳制与图片压缩统计所用单图 token 估算的统一天花板(#488/#496/#1843)。叠加在任何计费模式之上 —— 适用于路由真实编码器计费低于像素先验的场景。优先于两级配置([`imageTokenCap`](#imagetokencap));每次请求实时读取(无需重启)。不设置 = 回退到配置(`providers..imageTokenCap`,再全局),默认无上限。 | | `BILI_IMAGE_BILLING` | 覆盖预检尺寸门与输出钳制的图片计费模式(#767/#1843):`pixels` 或 `bytes`。每次请求实时读取(无需重启);优先于全局 `imageBilling` 与所有按 provider 的 `providers..imageBilling`。自 #1843 起默认对所有 host 解析为 `pixels`,故用 `bytes` 全进程强制保守字节计费(例如字节计数 relay)—— 更窄的场景仍用按路由设置。详见 [`imageBilling`](#imagebilling)。 | +| `BILI_IMAGE_RESCALE` | 关闭(`"0"` 或 `"false"`)或保持开启(其他取值)出站图片降采样 pass(#3400):大图以缩小形态发给模型,原图在会话存储中保持逐字节原样。每次请求实时读取(无需重启);优先于配置文件键 [`imageRescale`](#imagerescale),后者自身默认开启。 | +| `BILI_IMAGE_RESCALE_MAX_DIM` | 出站降采样 pass 的最长边触发阈值(像素,#3400):图片最长边超过该值**或** base64 体积超过 100 KB 即成为降采样候选;fit-inside 缩放从不放大。默认 `1600`;非数字或低于 `128` 的取值被忽略(按默认处理)。设计上仅环境变量、无配置文件镜像。详见 [`imageRescale`](#imagerescale)。 | | `BILI_PREFLIGHT_HOLD_MS` | 预压缩超过该宽限期(毫秒)后,代理提前提交响应并用保活字节挂住客户端(默认 `30000`;见 #568)。 | | `BILI_STREAM_KEEPALIVE_MS` | 流式阶段客户端保活(#1647):SSE 响应连续该毫秒数没有向客户端写出任何字节时,bili 发一条 SSE 注释行(`: bili-keepalive`,协议层 no-op),防止客户端 undici `bodyTimeout`(默认 300s;Node 内置 fetch 无法按请求覆盖)在长 prefill 时断连——上游的 ping 注释会被重写器/剥离管道吞掉。默认 `15000`;`0` 关闭。与 `BILI_PREFLIGHT_HOLD_MS` 互补:后者覆盖压缩预检期的静默,本变量覆盖流式期上游导致的静默。 | | `BILI_RECLAIM_FETCH_PATCH` | 设为 `0` 关闭 native 模式 fetch 自愈重武装(#1158)。默认情况下 native fetch 拦截会把 `globalThis.fetch` 装成受保护的访问器:第三方补丁重新赋值 `globalThis.fetch` 时(如 dsh-http-proxy 的 settings 刷新用冻结的 pre-bili `originalFetch` 盲覆盖),会被接链为下游,模型流量继续经过 bili。设 `0` 则回到经典直装:第三方重装生效,bili 将看不到本会话的模型流量。**出口提示:** 自愈生效期间,被认领的模型流量由 bili 代理自身派发——不再走第三方链的出口(例如 dsh-http-proxy 里配置的 SOCKS5;bili 自身的上游代理仅支持 HTTP 形式)。若需要回退第三方出口,设 `0` 并在 bili 层配置出口(`"proxy": "http://…"`)。 | @@ -1765,7 +1811,8 @@ ACP 原生 agent(当前为 `pi` 扩展)会在每个进程内向代理上报 | `BILI_AFFINITY_SIMHASH` | 设为 `0` 关闭 simhash 链对齐收养(#2265):匿名请求的精确哈希链被客户端侧大面积装饰性改写(如 Trae 切模型后给每条 assistant 消息重打模型标签)打断时,重新挂回既有会话并保留压缩状态,而不是每次新铸会话、从零重折。护栏:覆盖率 ≥90%(Hamming ≤10)、变异位置 ≥20%(单点编辑仍走 fork,#629)、至少一条字节相同的用户消息、双候选歧义拒猜。默认开启。配置文件中设 `"affinitySimhash": false` 效果相同;环境变量优先。 | | `BILI_RESUME_INHERITANCE` | 设为 `0` 关闭 resume 继承(默认开启)(#1486):当带自有会话 id 的客户端(如 Claude Code 的 `x-claude-code-session-id`)以**新**会话 id 重放完整历史来续接会话时(`cc --resume` 会 fork 出新 UUID),bili 通过字节级前缀匹配(≥8 条消息、append-only 跟踪)识别出它与该客户端已跟踪历史的父子关系,并在续接会话的首个请求上继承父会话的 ref 分配 —— 模型引用的旧代际 refs 因此命中**原始**消息、而不是错配到重新编号的新消息 —— 同时继承源内容完整存在的压缩块(随本继承一并生效,#1834:resume 丢块会导致被折叠原文重新回到线上、上游请求膨胀;旧的 `forkAdoption` 联动门现仅作用于匿名 fork,#629),并记录 `derivedFrom` 血缘。父会话不受影响;新消息在父会话 ref 空间之上继续编号。resume 必须**严格扩展**父历史 —— 同深度的字节级重放(不同 id)视为重复会话而非 resume。匿名会话不受影响(保留自己的 pfa-* 世界,#309)。配置文件中设 `"resumeInheritance": false` 效果相同;环境变量优先。 | | `BILI_STABLE_SYSTEM_ANCHOR` | 设为 `1` 开启稳定 system 锚定(#1085)—— **wire 层兜底(best-effort)**:根治在客户端(会话历史与指令变更的呈现方式由客户端决定),本开关只是阻止代理因头部变化而使整个已缓存前缀失效。**仅限 plain-proxy 模式**:plugin-mode agent(`x-bili-plugin`)自管上下文、永不参与锚定,避免对已自带 cache-friendly 更新注入的客户端(如 claude-code 的 system-reminder)做双重处理。开启后,bili 按会话记住客户端首次发送的头部 system/instructions 块并持续原样重发。**局部变更**(文件式编辑,与当前生效版本共享 ≥70% 行)追加末尾 `[System context update] …` user 注记,内含紧凑行级 diff(`-` 删除 / `+` 新增;每条注记顺序叠加在前一条之上)。**非局部变更**(结构性重排、tool 定义增删、带时间戳的 banner、超 400 行的头部)直接采用新文本 —— 一次有意的缓存失效好过追加会误导模型的噪声 diff。防抖保护:累积超过 8 条注记同样直接替换锚点为最新文本并清空日志。锚点与注记日志随会话持久化,不受压缩/compaction 影响(session metadata 而非 kernel state)。已知残留限制:客户端自放的 `cache_control` 断点在换头后仍可能错位。不参与锚定的请求:标题生成微请求(OpenAI/Google)、Responses compaction-trigger 请求、auto-mode classifier 请求。客户端自身已实现同类机制(稳定 prompt + 历史内更新)时零额外注入 —— 这类更新作为普通历史透传。默认关闭。配置文件中设 `"stableSystemAnchor": true` 效果相同;环境变量优先。 | -| `BILI_ALLOW_DSH_COMPACTION` | 设为 `1` 放行 dsh 内置自动压缩(#2028)。默认由 wire 级守卫(#1729)**在本地拒绝 dsh 原生压缩调用**(422),覆盖 bili 服务的全部线路——openai、anthropic、responses(responses 由 #2360 补上:此前的协议白名单在检查标记之前就把 responses 短路了,而 dsh 桌面端的压缩恰好走这条线路,调用因此静默穿过):dsh 的 `compaction-basic` 应对上下文压力时会重放会话前缀、并把固定摘要指令作为最后一条 user 消息发出,此类调用一旦落地,其 checkpoint 会永久覆盖原始历史——不可逆,且摧毁代理的压缩基底。本开关解除该拒绝,让 dsh 原生压缩真正执行。作用范围:非 web profile 下随附 bundle patch(`auto: false`)仍抑制**自动**触发,因此那里只有手动 `/compact` 受益;web profile 下任何 patch 层都无法按 id 直寻 preset 嵌套行(#1772),因此放行后自动触发照常工作——除非在你的 profile `cordis.patch.yml` 里按 #2474 施加对 preset-standard 的完整快照覆盖。网页配置页提供同一开关;环境变量优先于文件。在配置文件的 `"dsh"` 段下设 `"allowDshCompaction": true` 效果相同(旧文件里裸写在顶层的 `"allowDshCompaction"` 会在加载时自动迁移到该位置)。 | +| `BILI_ALLOW_DSH_COMPACTION` | 设为 `1` 放行 dsh 内置自动压缩(#2028)。默认由 wire 级守卫(#1729)**在本地拒绝 dsh 原生压缩调用**(422),覆盖 bili 服务的全部线路——openai、anthropic、responses(responses 由 #2360 补上:此前的协议白名单在检查标记之前就把 responses 短路了,而 dsh 桌面端的压缩恰好走这条线路,调用因此静默穿过):dsh 的 `compaction-basic` 应对上下文压力时会重放会话前缀、并把固定摘要指令作为最后一条 user 消息发出,此类调用一旦落地,其 checkpoint 会永久覆盖原始历史——不可逆,且摧毁代理的压缩基底。本开关解除该拒绝,让 dsh 原生压缩真正执行。在守卫拒绝期间,dsh 自己的上下文压力计数(原始账本)只增不减——被拒的 checkpoint 永不落地——而 bili 的折叠保持 wire 体积很小,因此会话在客户端仪表显示「已满」之后仍能继续工作,dsh 会不断重新请求压缩(持续被拒)。自 #3384 起,出站输出额度也不再按该原始账本规划——除非 bili 自己的折叠视图已到达强制的 headroom 目标线,或请求的额度达到原生窗口的 25% 及以上(#2490 保护区间);修复前,增长的账本会逐轮悄悄压掉模型的输出额度,在 onlyReasoning 模型上最终表现为无报错的空回复。作用范围:非 web profile 下随附 bundle patch(`auto: false`)仍抑制**自动**触发,因此那里只有手动 `/compact` 受益;web profile 下任何 patch 层都无法按 id 直寻 preset 嵌套行(#1772),因此放行后自动触发照常工作——除非在你的 profile `cordis.patch.yml` 里按 #2474 施加对 preset-standard 的完整快照覆盖。网页配置页提供同一开关;环境变量优先于文件。在配置文件的 `"dsh"` 段下设 `"allowDshCompaction": true` 效果相同(旧文件里裸写在顶层的 `"allowDshCompaction"` 会在加载时自动迁移到该位置)。 | +| `BILI_ALLOW_DSH_COMPACTION` | 设为 `1` 放行 dsh 内置自动压缩(#2028)。默认由 wire 级守卫(#1729)**在本地拒绝 dsh 原生压缩调用**(422),覆盖 bili 服务的全部线路——openai、anthropic、responses(responses 由 #2360 补上:此前的协议白名单在检查标记之前就把 responses 短路了,而 dsh 桌面端的压缩恰好走这条线路,调用因此静默穿过):dsh 的 `compaction-basic` 应对上下文压力时会重放会话前缀、并把固定摘要指令作为最后一条 user 消息发出,此类调用一旦落地,其 checkpoint 会永久覆盖原始历史——不可逆,且摧毁代理的压缩基底。本开关解除该拒绝,让 dsh 原生压缩真正执行。作用范围:bili 开启时,dsh 原生自动压缩现已在所有平面关闭 —— 宿主平面行经随附 bundle patch(`auto: false`)、web profile 的 preset 实例经 #3354 加载期 overlay —— 因此该开关主要治理非 web profile 的手动 `/compact`(那里的自动触发仍被 `auto: false` 抑制),以及 #3354 overlay 不适用的 dsh 构建上残留的自动触发。网页配置页提供同一开关;环境变量优先于文件。在配置文件的 `"dsh"` 段下设 `"allowDshCompaction": true` 效果相同(旧文件里裸写在顶层的 `"allowDshCompaction"` 会在加载时自动迁移到该位置)。 | | `BILI_NO_CACHE_CONTROL` | 设为 `1` 关闭 bili 在 Anthropic 通道上的 `cache_control` 断点标注(#1637,随 #1639 落地)。默认开启:Anthropic 系上游只缓存被显式打断点的内容(每请求最多 4 个,按 system + tools + 消息块合并计数),因此 bili 会标注 system 块加上至多 3 个累积消息断点——被标注的消息保持标注(前缀字节稳定),断点只随折叠消亡,最近 3 条稳定消息承载推进前沿。客户端自设的任何 `cache_control`(消息块、tools 条目)都会完全抑制 bili 的标注——客户端自管缓存优先。断点随会话持久化。本开关是逃生阀:用于拒绝该字段的上游或有自己断点策略的中继。仅限 plain-proxy Anthropic 通道;OpenAI/Responses 通道隐式缓存,从不标注。配置文件对应项:[`compat.noCacheControl`](#compat) —— 环境变量优先。 | | `BILI_CHAIN_CONTENT` | 设为 `1` 开启 bili→bili 链感知的 ACP 产物 / `` 检查点**正文内容**检测(#1086/#1421):当入站请求的正文携带压缩产物(渲染标签 / 历史 `acp_status`+`search_context` 工具调用)或带摘要的检查点,但既无 `x-bili-hop` 头、本实例也无该会话的压缩状态时,bili 记录一次告警性观察和/或应用「首个处理器优先」透传。**默认关闭**(#1683 后续):默认下只有 `x-bili-hop` 头驱动链识别,因为扫描请求正文可能把 CCR/文件引入的文本和模型回声标签误判为真实标记。仅在中间盒子剥掉 `x-bili-hop`、且你接受该误判风险的狭窄多 bili 中继场景下才启用。配置文件中设 `"chainContentDetection": true` 效果相同;环境变量优先。`x-bili-hop` 信号本身不受此开关影响。 | | `BILI_CHAIN_STAMP` | 设为 `1` 开启**模型可见**的 `` 链完整性检查点载体的出站注入(#1683,默认关闭):开启后,本实例实际处理的每个请求都携带一个带摘要的戳,使下游 bili 即使 `x-bili-hop` 头在传输中被剥离也能应用「首个处理器优先」(first-processor-wins)(#1421)。该载体落在终端模型同样会读取的位置(OpenAI/Responses 上是一条尾部 `user` 消息,Anthropic/Google 上是尾部文本 part),因此模型会把它当作幽灵用户输入并花 token 去评论它——这正是它默认关闭的原因。仅在多 bili 中继、且中间盒子剥掉 `x-bili-hop`、带摘要校验的 body-stamp 是防止双重处理的唯一手段这一狭窄场景下才启用。与 `BILI_CHAIN_CONTENT`(入站正文检测同样默认关闭)及 `x-bili-hop` 透传相互独立(后者无论如何都生效)。配置文件中设 `"chainEgressStamp": true` 效果相同;环境变量优先。 | @@ -1785,7 +1832,7 @@ ACP 原生 agent(当前为 `pi` 扩展)会在每个进程内向代理上报 | `BILI_PERSIST_EPERM_ALERT_THRESHOLD` | 同一会话连续 N 次持久化写失败(`EPERM`/`EBUSY`/`EACCES`)后触发一次性「把该目录加入杀软排除项」告警的阈值(默认 `5`)。仅 Windows。见下文「Windows:把会话目录加入杀软排除项」章节。 | | `BILI_PERSIST_EPERM_ALERT_REPEAT_MS` | persist EPERM 告警的重复窗口(毫秒)。`0`(默认)= 只告警一次后静默;`>0` = 失败持续期间最多每这么久重复告警一次。 | | `BILI_MAX_SESSIONS` | 内存中最多保留的会话数(默认 `256`;LRU 淘汰 —— 磁盘是事实源)。 | -| `BILI_SESSION_WARM_MAX` | 有界启动(#2803):监听前启动时最多预加载的最近保存会话数(默认 `8`,受 `sessions.max` 封顶);更早的状态按需重载。`0` 表示不预热。文件侧对应:`sessions.warmMax`。 | +| `BILI_SESSION_WARM_MAX` | 有界启动(#2803):监听前启动时最多预加载的最近保存会话数(默认 `0` = 不预热,受 `sessions.max` 封顶);更早的状态按需重载。设为如 `8` 可恢复对最近集的热加载。文件侧对应:`sessions.warmMax`。 | | `BILI_SESSION_WARM_MAX_BYTES` | 启动预热集的解码字节预算(#2803;默认 `67108864` = 64 MiB)。计的是**解码后**尺寸而非磁盘压缩字节,zstd 归档无法绕过内存上限。单文件超预算则永不预热。文件侧对应:`sessions.warmMaxBytes`。 | | `BILI_SESSION_RESIDENT_MAX_BYTES` | 常驻会话估算字节的软目标(#2803;默认 `536870912` = 512 MiB)。活跃/in-flight 状态不受此限制(字节预算约束的是空闲缓存而非活跃负载);超出目标时空闲状态随请求完成回落磁盘。文件侧对应:`sessions.residentMaxBytes`。 | | `BILI_PUBLIC_SNAPSHOT_CAP_BYTES` | 为公开 fork API 按插件会话保留的原始 wire 历史快照的大小上限(字节,#2017)。序列化快照超限的插件会话不再可 fork —— `GET /__bili/plugin/snapshot` 与 `POST /__bili/plugin/fork` 以 `409` fail-closed 并注明原因 —— 而不是永久保留无上限的原始历史副本。上限在每次插件模型请求时重新评估:会话缩回上限内(fork 裁剪后或宿主缩短历史)即恢复可 fork。默认 `104857600`(100 MiB);`0` 完全停用留存(所有会话不可 fork,已有快照在下一次请求时丢弃)。文件孪生键:`plugin.snapshotCapBytes`。 | @@ -2104,13 +2151,13 @@ bili plugin remove pi # 撤销(原文件一次性备份为 *.bili-bak) - **运行时证据,按证据层级展示(#2545)**:未宣告的历史改写(#1001)与孤儿块废弃(被摘要的内容从客户端历史中被删掉)与「疑似并存」记入同一台账。严重程度表面按**证据层级**分级:**已确认** = 检测到的客户端原生压缩落点(bili 会 rebase 到其上,#2372/#2432)或非 suspected 的插件发现——只有它会触发「请只保留一个压缩器」的强制告警;**未确认信号** = 未宣告改写 + 孤儿块废弃——观测到变化但原因未确认(「历史发生了变化,原因未知」),绝不呈现为原生自动压缩或第二个压缩器;纯名称匹配的 [suspected] 停留在软性的「先核实」层级(#1736)。指向已验证只读插件的存量台账记录在显示期被中和,而不是持续作为告警残留;跨会话聚合的未确认信号计数永远不会升级成「同一会话双压缩」的结论。 - **dsh 的 `auto: false` 只关闭自动触发**。profile bundle patch(`dsh.bundle.patch.yml`)写入的 `compaction-basic: { auto: false }` 跳过压力/溢出自压缩 —— 手动 `/compact`(以及空闲会话压缩)仍会触发。经 bili 路由的调用会被服务端闸门拒绝(#1729/#2360);未经过 bili 直达上游的调用(桌面端插件接管门无法归因的路径)会落地,bili 在下次重放时检测出来(checkpoint 框架 + 折叠覆盖缺口),一个 turn 内重建自己的压缩状态,而不是让之后每次 compress 永久失败(#2432)。 -#### 在 web profile 下关掉 dsh 原生自动压缩(#1772/#2474) + #### 在 web profile 下关掉 dsh 原生自动压缩(#1772/#3354,#3354 起自动;#2474 配方为回退) -随包发行的 `dsh.bundle.patch.yml` 做不到这件事:在 bundles 含 `@deepseek-ai/dsh-web-app` 的 profile 里,它的 `- id: compaction-basic` 行落在 web-app 的**宿主层行上,而那一行本来就是 `disabled: true`**;真正在运行的实例位于 `preset-standard.config.plugins` 内部。补丁静默生效、什么都没改。 + **#3354 起自动完成。** 在 bundles 含 `@deepseek-ai/dsh-web-app` 的 profile 里,真正运行的 compaction-basic 实例位于 agent preset(`preset-standard.config.plugins`)内部,任何 patch 层都无法按 id 直寻那条嵌套行(#1772)—— 随包的 `- id: compaction-basic` 行落在 web-app 已禁用的宿主平面行上、什么都没改。bili 现在在**加载期**补上这一环:`bili-native` bundle 条目注册一个 cordis `internal/config` overlay(移植自 [deshark42/dsh-compaction-basic-disable](https://github.com/deshark42/dsh-compaction-basic-disable),MIT),把 preset 里的 `compaction-basic` 与 `command-compact` 行标记为 `disabled: true`;`dsh.bundle.patch.yml` / 启动器 overlay 里的 preset 门控行让每个 Web preset 在注册前等待该 overlay。这些 profile 里原生自动压缩无需任何用户操作即关闭;ACP 压缩不受影响。 -**profile 层补丁可以关掉它** —— preset 是 bundle 层插入的普通顶层条目,而 profile 补丁层在其之后应用。不能用「按 id 定位嵌套行」的写法,可行的是**整体覆盖 preset 行的 `config`**。 + overlay 依赖 dsh 的内部钩子,对版本敏感。若在新/旧 dsh 构建上仍见到 dsh 压缩 checkpoint 落地,用下面的回退配方。 -**配方**(已在 dsh `0.2.1-alpha.1`、Windows 11、bundles 含 `@deepseek-ai/dsh-web-app` 的 profile 上实测): + **回退配方(#2474)**(已在 dsh `0.2.1-alpha.1`、Windows 11、bundles 含 `@deepseek-ai/dsh-web-app` 的 profile 上实测)。profile 层补丁可以关掉它 —— preset 是 bundle 层插入的普通顶层条目,而 profile 补丁层在其之后应用。不能用「按 id 定位嵌套行」的写法,可行的是**整体覆盖 preset 行的 `config`**。 ```bash # 1. dump 组合后的 profile,复制 preset-standard 的完整 config 块 diff --git a/PLUGIN.md b/PLUGIN.md index 2fadda12d1..25edffade9 100644 --- a/PLUGIN.md +++ b/PLUGIN.md @@ -53,7 +53,7 @@ Fetch once at plugin startup. } ``` -Register the tools natively with your agent, in whichever wire format your agent speaks — register exactly what `toolNames` lists. `acp_rule` is advertised only when the proxy's config enables it (`compress.rules: true`, opt-in). Calling a tool that is known but disabled answers `200` with an explanatory `result` instead of an error (#1192). If `protocolVersion` is higher than you know, still register the tools — extra fields in schemas are ignored by agents. +Register the tools natively with your agent, in whichever wire format your agent speaks — register exactly what `toolNames` lists. `absorb` appears only when the proxy's config enables it; `acp_rule` is advertised only when the proxy's config enables it (`compress.rules: true`, opt-in). Calling a tool that is known but disabled answers `200` with an explanatory `result` instead of an error (#1192). If `protocolVersion` is higher than you know, still register the tools — extra fields in schemas are ignored by agents. ### 2. Request headers @@ -98,7 +98,7 @@ Notes: - Execution happens **under the session lock**, against the same view the model was shown on the last request (refs match what the model sees). - Optional `expectedRevision` is the opaque `parentRevision` from a snapshot. It requires an explicit `conversationId` and is checked under the same session lock immediately before execution. A stale revision returns `409 PARENT_REVISION_CONFLICT` without executing the tool. This also avoids witness routing to another conversation. Existing callers omitting it retain their routing and execution behavior. - `compress` mutates state. Inline `decompress` marks a block restored for re-folding; range restores queue content for the next request. Successful restores also update the estimated context footprint. `search_context` / `acp_status` are read-only. -- Errors: `400` invalid JSON / missing `conversationId` / unknown tool, `404` unknown conversation (no model request has arrived with that conversation id yet), `500` execution failure. A **known but disabled** opt-in tool (`acp_rule`) is not an error: it answers `200` with `ok: true` and a `result` explaining that the feature is off on this proxy (#1192). +- Errors: `400` invalid JSON / missing `conversationId` / unknown tool, `404` unknown conversation (no model request has arrived with that conversation id yet), `500` execution failure. A **known but disabled** opt-in tool (`absorb` / `acp_rule`) is not an error: it answers `200` with `ok: true` and a `result` explaining that the feature is off on this proxy (#1192). ### 4. `GET /__bili/plugin/status?conversationId=` diff --git a/README.md b/README.md index d0aef5889b..e022fa67b9 100644 --- a/README.md +++ b/README.md @@ -118,7 +118,7 @@ Pick by your client: | **gemini** (Gemini CLI) | `bili gemini` (launcher, `GOOGLE_GEMINI_BASE_URL` `/bili/` rewrite) or `/bili/` prefix — launcher-only (no in-loop tool seam, #1043) | | **iflow** (iFlow CLI) | `bili iflow` (launcher, `IFLOW_BASE_URL` `/bili/` rewrite) or `/bili/` prefix | | **qwen** (Qwen Code) | `bili qwen` (launcher, cert-MITM) or `/bili/` prefix | -| **antigravity** (Antigravity CLI / agy, Google) | `bili antigravity` (launcher, `CLOUD_CODE_URL` `/bili/` rewrite of cloudcode-pa.googleapis.com) or `/bili/` prefix — no plugin seam (closed Go language_server; its user-plugin surface is additive-only, #2115) — details: [CLIENTS.md](CLIENTS.md) | +| **antigravity** (Antigravity CLI / agy, Google) | `bili antigravity` (launcher, `CLOUD_CODE_URL` `/bili/` rewrite of daily-cloudcode-pa.googleapis.com — Antigravity's real default upstream, #3129) or `/bili/` prefix — no plugin seam (closed Go language_server; its user-plugin surface is additive-only, #2115) — details: [CLIENTS.md](CLIENTS.md) | | **mcode** (MiniMax Code) | [`billion-context`](https://github.com/ranxianglei/billion-context) via `bili mcode` (cert-MITM) or `/bili/` prefix — no native mode (event hooks only, no model-request seam, [#1050](https://github.com/ranxianglei/billion-context/issues/1050)) | | **aider** | [`billion-context`](https://github.com/ranxianglei/billion-context) via `bili aider` (cert-MITM) or `/bili/` prefix — no native mode (shell-command-only hooks, no tool-injection seam, [#1048](https://github.com/ranxianglei/billion-context/issues/1048)) | | **copilot** (GitHub Copilot CLI) | `bili copilot` (launcher, cert-MITM) — closed Go binary, no plugin seam (#1049) | @@ -258,7 +258,7 @@ bili claude # launch claude through the proxy bili omp # pi-style, file-free (#535): env + extension registerProvider + compaction cancel, real ~/.omp untouched bili opencode # OpenCode (1.x & 2.x): full guide in the [OpenCode](CLIENTS.md#opencode) section below bili hermes # file-free (#535): hermes proxy env (HTTPS_PROXY + combined CA bundle via SSL_CERT_FILE) — https via CONNECT MITM, http via absolute-form forward proxy; real ~/.hermes untouched -bili dsh # deepseek-harness: full native plugin injected via --patch (#941) — real dsh tools, session-bound /acp + /acp-cache (plugin mode); non-loopback upstreams ride proxy envs, loopback keeps the overlay DSH_HOME rewrite (#535); dsh auto-compaction off outside web profiles, and switchable off there via a profile-level full-snapshot override of preset-standard (#1772/#2474) +bili dsh # deepseek-harness: full native plugin injected via --patch (#941) — real dsh tools, session-bound /acp + /acp-cache (plugin mode); non-loopback upstreams ride proxy envs, loopback keeps the overlay DSH_HOME rewrite (#535); dsh auto-compaction off everywhere — host plane via the bundle patch, web-profile presets at load time (#1772/#2474/#3354) bili codebuddy # Tencent CodeBuddy Code CLI: CODEBUDDY_BASE_URL /bili/ rewrite (OpenAI chat completions wire), budget aligned via CODEBUDDY_AUTO_COMPACT_WINDOW; real ~/.codebuddy untouched bili qoder # qoder: model endpoint is hardcoded https (no /bili/ rewrite possible) — cert-MITM via HTTPS_PROXY + NODE_EXTRA_CA_CERTS, default model hosts whitelisted (#653) bili trae # Trae CLI (ByteDance, closed Go binary, no base-URL override) — cert-MITM via HTTPS_PROXY + SSL_CERT_FILE, model host from TRAE_CLI_API_HOST or the default enterprise gateway (#655) @@ -267,7 +267,7 @@ bili kimi # Kimi Code CLI (Moonshot): standard proxy bili gemini # Gemini CLI (Google): GOOGLE_GEMINI_BASE_URL /bili/ rewrite to generativelanguage.googleapis.com (Google native wire), real ~/.gemini untouched bili iflow # iFlow CLI: IFLOW_BASE_URL /bili/ rewrite to apis.iflow.cn/v1 (OpenAI chat-completions wire), real ~/.iflow untouched bili qwen # Qwen Code (multi-protocol gemini-cli fork, no base-URL hook): cert-MITM via HTTPS_PROXY + NODE_EXTRA_CA_CERTS, default DashScope/Qwen model hosts whitelisted, custom relays via --mitm-domain -bili antigravity # Antigravity CLI (agy, Google): CLOUD_CODE_URL /bili/ rewrite of cloudcode-pa.googleapis.com (undocumented language-server env override, v2.19.1 binary-verified); wire recognized as Google native by path (#2115) +bili antigravity # Antigravity CLI (agy, Google): CLOUD_CODE_URL /bili/ rewrite of daily-cloudcode-pa.googleapis.com — the real default upstream (bare cloudcode-pa 400s on /v1internal:*, #3129); undocumented language-server env override, v2.19.1 binary-verified; wire recognized as Google native by path (#2115) bili mcode # MiniMax Code CLI: same shape as kimi (proxy envs, unconditional loopback bypass, cert-MITM/absolute-form); session bound via X-Mavis-Session-Id (#1050) bili aider # Aider (Python pair programmer): cert-MITM via HTTPS_PROXY + SSL_CERT_FILE/REQUESTS_CA_BUNDLE; endpoint from OPENAI_API_BASE / ANTHROPIC_BASE_URL / --openai-api-base / .aider.conf.yml (#1048) bili copilot # Copilot CLI (GitHub, closed Go binary) — cert-MITM via HTTPS_PROXY + SSL_CERT_FILE, api.githubcopilot.com + per-plan subdomains whitelisted (#1049) diff --git a/README.zh-CN.md b/README.zh-CN.md index 16f524a089..da870da7a6 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -120,7 +120,7 @@ QQ群: | **gemini**(Gemini CLI) | `bili gemini`(启动器,`GOOGLE_GEMINI_BASE_URL` `/bili/` 改写)或 `/bili/` 前缀 —— 仅启动器(无环内工具注入接缝,#1043) | | **iflow**(iFlow CLI) | `bili iflow`(启动器,`IFLOW_BASE_URL` `/bili/` 改写)或 `/bili/` 前缀 | | **qwen**(Qwen Code) | `bili qwen`(启动器,cert-MITM)或 `/bili/` 前缀 | -| **antigravity**(Antigravity CLI / agy,Google) | `bili antigravity`(启动器,`CLOUD_CODE_URL` 对 `cloudcode-pa.googleapis.com` 的 `/bili/` 改写)或 `/bili/` 前缀 —— 无插件接缝(闭源 Go language_server;其用户插件面仅可追加、不能拦截模型流,[#2115](https://github.com/ranxianglei/billion-context/issues/2115))—— 细节见 [CLIENTS.zh-CN.md](CLIENTS.zh-CN.md) | +| **antigravity**(Antigravity CLI / agy,Google) | `bili antigravity`(启动器,`CLOUD_CODE_URL` 对 `daily-cloudcode-pa.googleapis.com` 的 `/bili/` 改写 —— Antigravity 真实默认上游,#3129)或 `/bili/` 前缀 —— 无插件接缝(闭源 Go language_server;其用户插件面仅可追加、不能拦截模型流,[#2115](https://github.com/ranxianglei/billion-context/issues/2115))—— 细节见 [CLIENTS.zh-CN.md](CLIENTS.zh-CN.md) | | **mcode**(MiniMax Code) | [`billion-context`](https://github.com/ranxianglei/billion-context),`bili mcode`(cert-MITM)或 `/bili/` 前缀 —— 无原生模式(纯声明式事件钩子,无模型请求接缝,[#1050](https://github.com/ranxianglei/billion-context/issues/1050)) | | **aider** | [`billion-context`](https://github.com/ranxianglei/billion-context),`bili aider`(cert-MITM)或 `/bili/` 前缀 —— 无原生模式(仅 shell 命令钩子,无工具注入接缝,[#1048](https://github.com/ranxianglei/billion-context/issues/1048)) | | **copilot**(GitHub Copilot CLI) | `bili copilot`(启动器,cert-MITM)—— 闭源 Go 二进制、无插件接缝(#1049) | @@ -204,7 +204,7 @@ bili claude # 拉起 claude bili omp # pi 同款,file-free(#535):环境变量 + 扩展 registerProvider + 压缩取消,真实 ~/.omp 不动 bili opencode # OpenCode(1.x 与 2.x):完整指南见下文 [OpenCode](CLIENTS.zh-CN.md#opencode) 一节 bili hermes # file-free(#535):hermes 代理环境变量(HTTPS_PROXY + SSL_CERT_FILE 组合 CA bundle)—— https 走 CONNECT MITM,http 走绝对形式转发;真实 ~/.hermes 不动 -bili dsh # deepseek-harness:经 --patch 注入完整原生插件(#941) —— 真实 dsh 工具、会话绑定 /acp(plugin 模式);非回环上游走代理 env,回环保留 overlay DSH_HOME 改写(#535);dsh 自动压缩关(web profile 除外——嵌套行不可按 id 直寻,但可经 profile 层完整快照覆盖 preset-standard 关掉,#1772/#2474) +bili dsh # deepseek-harness:经 --patch 注入完整原生插件(#941) —— 真实 dsh 工具、会话绑定 /acp(plugin 模式);非回环上游走代理 env,回环保留 overlay DSH_HOME 改写(#535);dsh 自动压缩处处关闭(宿主平面经 bundle patch,web profile 的 preset 经 #3354 加载期 overlay,#1772/#2474/#3354) bili codebuddy # Tencent CodeBuddy Code CLI:CODEBUDDY_BASE_URL /bili/ 重写(OpenAI chat completions wire),预算对齐走 CODEBUDDY_AUTO_COMPACT_WINDOW;真实 ~/.codebuddy 不动 bili qoder # qoder:模型端点硬编码 https(无法 /bili/ 改写)—— 证书 MITM(HTTPS_PROXY + NODE_EXTRA_CA_CERTS),默认模型主机已加白名单(#653) bili trae # Trae CLI(字节跳动,闭源 Go 二进制,无 base-URL 覆盖)—— 证书 MITM(HTTPS_PROXY + SSL_CERT_FILE),模型主机取 TRAE_CLI_API_HOST 或默认企业网关(#655) @@ -213,7 +213,7 @@ bili kimi # Kimi Code CLI(Moonshot):除无条件回 bili gemini # Gemini CLI(Google):GOOGLE_GEMINI_BASE_URL /bili/ 改写到 generativelanguage.googleapis.com(Google 原生 wire),真实 ~/.gemini 零改动 bili iflow # iFlow CLI:IFLOW_BASE_URL /bili/ 改写到 apis.iflow.cn/v1(OpenAI chat-completions wire),真实 ~/.iflow 零改动 bili qwen # Qwen Code(多协议 gemini-cli fork,无 base-URL 钩子):HTTPS_PROXY + NODE_EXTRA_CA_CERTS 证书 MITM,默认 DashScope/Qwen 模型主机加白,自建中转用 --mitm-domain 追加 -bili antigravity # Google Antigravity CLI(agy,Gemini CLI 继任者):CLOUD_CODE_URL 对 cloudcode-pa.googleapis.com 的 /bili/ 改写,无需证书 MITM(#2115) +bili antigravity # Google Antigravity CLI(agy,Gemini CLI 继任者):CLOUD_CODE_URL 对 daily-cloudcode-pa.googleapis.com 的 /bili/ 改写 —— 真实默认上游(裸 cloudcode-pa 在 /v1internal:* 上 400,#3129),无需证书 MITM(#2115) bili mcode # MiniMax Code CLI:与 kimi 同构(代理环境变量、无条件回环绕过、证书 MITM/绝对形式);会话经 X-Mavis-Session-Id 绑定(#1050) bili aider # Aider(Python pair programmer):HTTPS_PROXY + SSL_CERT_FILE/REQUESTS_CA_BUNDLE 证书 MITM;端点取自 OPENAI_API_BASE / ANTHROPIC_BASE_URL / --openai-api-base / .aider.conf.yml(#1048) bili copilot # Copilot CLI(GitHub,闭源 Go 二进制)—— 证书 MITM(HTTPS_PROXY + SSL_CERT_FILE),api.githubcopilot.com + 各套餐子域加白(#1049) diff --git a/TECHNICAL-NOTES.md b/TECHNICAL-NOTES.md index 7bace5a754..78f7909dc0 100644 --- a/TECHNICAL-NOTES.md +++ b/TECHNICAL-NOTES.md @@ -345,7 +345,7 @@ bypass and side lanes never earn credit): compression reduction. Remedy: **degrade to proxy mode** — the single `pluginMode` flip (`pluginAgent !== undefined && !pluginLaneDegraded(session)`) re-injects the ACP tools wire-side, re-arms the compress rewriter, CCR - stamping, and **keeps the nudge** (the tools are back). When wire + stamping and absorb, and **keeps the nudge** (the tools are back). When wire injection is unavailable (`compress.injectTool=false` → `degradeAvailable` false) the remedy falls back to **suppress-nudge**. Recovery: the live plugin header (`pluginLaneRestore`, hoisted BEFORE the `pluginMode` flip so @@ -403,7 +403,7 @@ functions, exit audits): reasoning-echo strip — the thinking channel is no longer cleaned. 2. **Model prose is manageable.** Render-tag echoes, marker lines, bili-internal text, degenerate open/close residue, and (when - instructed) whole tool-call emissions are stripped from the visible + absorb-instructed) whole tool-call emissions are stripped from the visible text channel only. 3. **Model output bound for files is byte-identical (#1039).** Tool-call arguments never enter any filter, whatever bytes they carry. diff --git a/TECHNICAL-NOTES.zh-CN.md b/TECHNICAL-NOTES.zh-CN.md index bf55388326..7293c350b6 100644 --- a/TECHNICAL-NOTES.zh-CN.md +++ b/TECHNICAL-NOTES.zh-CN.md @@ -102,7 +102,7 @@ bili 永不拥有用户数据:每个被启动的客户端都跑在**真实 home* 三种故障形态共享 `src/session-self-heal.ts` 里的同一族检测器,每请求评估一次(prepare 之后,仅有 nudge 的主车道;旁路/侧请求车道不积累信用): -- **D2 僵尸插件车道** —— 插件已卸载(或 MCP 子进程已死)但已开着的客户端窗口仍带着插件绑定(粘滞 `metadata.pluginAgent`)。信号:连续 5 轮(`SELF_HEAL_WINDOW`)被 nudge、**无** `x-bili-plugin` 头、线上**无** bili 工具声明、**零**压缩缩减。补救:**降级为代理模式** —— 单点 `pluginMode` 翻转(`pluginAgent !== undefined && !pluginLaneDegraded(session)`)线上回注 ACP 工具、重新接管压缩 rewriter/CCR 盖章,并**保留 nudge**(工具回来了)。线上注入不可用时(`compress.injectTool=false` → `degradeAvailable=false`)退为**静默 nudge**。恢复:**活的**插件头(`pluginLaneRestore`,在 `pluginMode` 翻转**之前**调用,首个带头请求即恢复 plugin 模式 —— 恢复信号绝不能用粘滞 `pluginAgent`,否则降级一轮后即被自己清掉)。绑定保留、永不抹除。 +- **D2 僵尸插件车道** —— 插件已卸载(或 MCP 子进程已死)但已开着的客户端窗口仍带着插件绑定(粘滞 `metadata.pluginAgent`)。信号:连续 5 轮(`SELF_HEAL_WINDOW`)被 nudge、**无** `x-bili-plugin` 头、线上**无** bili 工具声明、**零**压缩缩减。补救:**降级为代理模式** —— 单点 `pluginMode` 翻转(`pluginAgent !== undefined && !pluginLaneDegraded(session)`)线上回注 ACP 工具、重新接管压缩 rewriter/CCR 盖章/absorb,并**保留 nudge**(工具回来了)。线上注入不可用时(`compress.injectTool=false` → `degradeAvailable=false`)退为**静默 nudge**。恢复:**活的**插件头(`pluginLaneRestore`,在 `pluginMode` 翻转**之前**调用,首个带头请求即恢复 plugin 模式 —— 恢复信号绝不能用粘滞 `pluginAgent`,否则降级一轮后即被自己清掉)。绑定保留、永不抹除。 - **D1 nudge 空转**(非插件泛化)—— 连续 5 轮被 nudge 却零缩减、零 bili 工具调用。补救:在全部四处 `willInjectNudge` 门加 `&& !nudgeSuppressed(session)`(覆盖 emergency 路径)。恢复:任一次压缩缩减或 bili 工具调用即解除——或者在 `SELF_HEAL_SUPPRESS_CAP_MS`(10 分钟)内两者皆无时,抑制到时自解并重新武装(#3004/#3008:上下文单调增长时永远产生不了恢复证据,纯证据式解除曾把建议通道锁死一个多小时而用量停在 ~75%)。重新武装是有界的:再空转完整一个 `SELF_HEAL_WINDOW`(5 轮)才会再次抑制;决策梯的 fallback 逃逸在抑制期间仍可打出一次性 legacy 建议(`consumeFallback` 重置计数)。 - **D3 网关杀摘要** —— **非流式** preflight 摘要以 HTTP 524/504(Cloudflare 类)死亡时不再原地重试(旧的 transient 重试对着死网关每次烧 ~100 秒);会话在**首次**命中即学习 `metadata.preflightStreamSummary = true` 并改用 SSE 重取摘要 —— 与手动 `compress.streamSummary` 旋钮(#2133)同一学习存储,自动武装。级联双向被尊重:任一级显式 `streamSummary: false`(global → provider → model)即操作员 opt-out —— 两条学习路径都不武装、已学习的旧标志被忽略(`PreflightDeps.streamSummaryOff`)。 @@ -113,7 +113,7 @@ bili 永不拥有用户数据:每个被启动的客户端都跑在**真实 home* 一个 PR 把三条整治腿合并(refs-run 残留 #2023、退化闭名剥离+出口审计网 #2190、emission 整段丢弃+签名思考 verbatim #2066),并统一受四条硬规则约束 —— 覆盖**所有**文本改写面(loop 适配器、插件直通、JSON strip 函数、出口审计): 1. **思考通道逐字节不动。** Anthropic `thinking`、Google `thought`、OpenAI `reasoning_content`/`reasoning`、Responses reasoning summaries 原样透传 —— 哪怕长得像标签回声。理由:reasoning 回放(DeepSeek 式)与签名校验验证的就是这些字节,任何改写都会让它们失步并卡死会话(#1960/KDD#10)。适配器侧走 `createIdentityStreamFilter`,strip 函数(`stripOpenaiChatText`、`stripResponsesText`)同样不动它们(纵深防御)。**注意:这翻转了 #1881 时代的 reasoning 回声剥离 —— 思考通道不再清洗。** -2. **模型的正文输出可以治理。** 渲染标签回声、marker 行、bili 内部文本、退化开/闭名残骸,以及(受指令时)整段工具调用 emission,只在可见文本通道剥离。 +2. **模型的正文输出可以治理。** 渲染标签回声、marker 行、bili 内部文本、退化开/闭名残骸,以及(absorb 受指令时)整段工具调用 emission,只在可见文本通道剥离。 3. **模型写入文件的内容逐字节不变(#1039)。** 工具调用 arguments 无论携带什么字节都不进任何过滤器。 4. **用户发送的消息永不被改写。** 用户自己片段的逐字回声在武装清洗下也保留(#463 豁免,按请求文本匹配,不按形状)。 diff --git a/dsh.bundle.patch.yml b/dsh.bundle.patch.yml index 95305f807c..6fec7ae17f 100644 --- a/dsh.bundle.patch.yml +++ b/dsh.bundle.patch.yml @@ -2,18 +2,32 @@ # Mounted automatically when the profile's package.json lists billion-context # in dsh.profile.bundles (`dsh plugin --profile add billion-context`); # the loader resolves `name` from the profile's node_modules. -# #1772/#2474: the `auto: false` below targets the HOST-PLANE compaction-basic -# row. In profiles bundling @deepseek-ai/dsh-web-app that row is disabled and -# the running instance lives inside the preset-standard entry, which this -# bundle layer cannot address by id — switching off auto-compaction there -# needs a profile-level full-snapshot override of preset-standard (config -# replaces wholesale; see the plugin's boot warning / #2474). +# #1772/#2474/#3354: the `auto: false` below targets the HOST-PLANE +# compaction-basic row. In profiles bundling @deepseek-ai/dsh-web-app that row +# is disabled and the running instance lives inside the agent presets, whose +# nested rows no patch layer can address by id — the preset gate rows below +# make each Web preset wait for the load-time overlay the bili-native entry +# registers (vendored from deshark42/dsh-compaction-basic-disable, MIT, #3354; +# see src/agent/dsh-compaction-disable.ts). A profile-level full-snapshot +# override of preset-standard (#2474) remains the fallback for dsh builds +# whose internal 'internal/config' hook moved. - insert: - id: bili-native name: billion-context - id: compaction-basic config: auto: false +# #3354: keep in sync with writeDshAcpPatch in src/launcher.ts — entry inject +# lists are REPLACED by patches, not concatenated; if a future dsh ships its +# own inject list on these entries, merge manually in BOTH layers. +- id: preset-standard + inject: [biliBasicCompactionPolicy] +- id: preset-ptc + inject: [biliBasicCompactionPolicy] +- id: preset-minimal + inject: [biliBasicCompactionPolicy] +- id: preset-cordis + inject: [biliBasicCompactionPolicy] # #2605: DSH's default retry whitelist omits MALFORMED_RESPONSE, so one # sampling flake that breaks a tool-call argument JSON kills the whole turn # unretried. A patch replaces the targeted row's WHOLE config (no deep diff --git a/kernel/package-lock.json b/kernel/package-lock.json index f57461d1fa..6b1cfc297d 100644 --- a/kernel/package-lock.json +++ b/kernel/package-lock.json @@ -1,12 +1,12 @@ { "name": "acp-kernel", - "version": "0.0.128", + "version": "0.0.129", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "acp-kernel", - "version": "0.0.128", + "version": "0.0.129", "license": "MIT", "devDependencies": { "@types/node": "^22.0.0", diff --git a/kernel/package.json b/kernel/package.json index f927d49acf..a318eb703b 100644 --- a/kernel/package.json +++ b/kernel/package.json @@ -1,6 +1,6 @@ { "name": "acp-kernel", - "version": "0.0.128", + "version": "0.0.130", "description": "Framework-agnostic context-compression engine (model-driven, 3-tier LSM). Pure core: no host dependency.", "license": "MIT", "author": "ranxianglei", diff --git a/kernel/src/absorb.ts b/kernel/src/absorb.ts index 0865425599..b1365b3972 100644 --- a/kernel/src/absorb.ts +++ b/kernel/src/absorb.ts @@ -4,6 +4,7 @@ import { isStoredPlaceholderText } from "./ccr.js"; import { collectLatestProtected, collectProtectedToolCallIds, + hasMediaPayload, isMessageLatestProtected, isMessageProtectedWithPairing, matchToolPattern, @@ -108,6 +109,7 @@ export function isAbsorbCandidate( for (const pattern of cfg.excludeTools) { if (msg.toolName && matchToolPattern(msg.toolName, pattern)) return false; } + if (cfg.mediaOnly && !hasMediaPayload(msg)) return false; return true; } @@ -170,7 +172,10 @@ export function appendAbsorbPrompts( const text = msg.text ?? ""; if (text.includes(ABSORB_PROMPT_MARKER)) return msg; const tokens = countTokens(text); - if (tokens < cfg.minToolTokens) return msg; + // #3402: image bytes are not in msg.text, so countTokens(text) never reaches + // minToolTokens for a media result — exempt it or the feature is dead. + const mediaExempt = cfg.mediaOnly && hasMediaPayload(msg); + if (!mediaExempt && tokens < cfg.minToolTokens) return msg; const ref = refForRaw(state.messageRefs, msg.id); if (!ref || ref === BLOCKED_REF) return msg; promptedCount++; @@ -305,6 +310,15 @@ export function applyAbsorb(input: AbsorbInput): AbsorbOutcome { resultText: `absorb failed: ref ${input.ref} is a ${target.contentType}, not a tool result.`, }; } + // #3371 discipline: screening and execution must share one judgment — a + // mediaOnly session never absorbs a target that isAbsorbCandidate would skip. + if (cfg.mediaOnly && !hasMediaPayload(target)) { + return { + state: input.state, + ok: false, + resultText: `absorb failed: ${input.ref} carries no media payload — absorb is scoped to image-bearing results (compress.absorb.mediaOnly).`, + }; + } if (isAcpOrConfiguredTool(target.toolName, cfg)) { return { state: input.state, diff --git a/kernel/src/nudge-text.ts b/kernel/src/nudge-text.ts index d3caa47b83..7d656b728d 100644 --- a/kernel/src/nudge-text.ts +++ b/kernel/src/nudge-text.ts @@ -114,6 +114,7 @@ export function formatRanges( toolPct: number; textPct: number; dangerous: boolean; + noMergeBefore: boolean; } const refNum = (ref: string): number => { const m = ref.match(/\d+/); @@ -139,6 +140,7 @@ export function formatRanges( protectedCount: 0, protectedTools: [], dangerous: r.dangerous ?? false, + noMergeBefore: r.noMergeBefore ?? false, }); } for (const r of protectedRanges) { @@ -160,6 +162,7 @@ export function formatRanges( protectedCount: r.count, protectedTools: [...r.tools], dangerous: false, + noMergeBefore: r.noMergeBefore ?? false, }); } // Order/merge by POSITION, never ref number: refs can be non-monotonic vs array @@ -170,7 +173,9 @@ export function formatRanges( const merged: Merged[] = []; for (const e of entries) { const last = merged[merged.length - 1]; - if (last && e.startPos <= last.endPos + 1) { + // #3192/#3205: a dead-hole boundary has no index gap, so position adjacency + // alone would bridge across it — honor the flag set by buildCompressibleRanges. + if (last && !e.noMergeBefore && e.startPos <= last.endPos + 1) { last.endRef = e.endRef; last.endNum = Math.max(last.endNum, e.endNum); last.endPos = Math.max(last.endPos, e.endPos); diff --git a/kernel/src/recommend.ts b/kernel/src/recommend.ts index 8517176aeb..3f4030eae6 100644 --- a/kernel/src/recommend.ts +++ b/kernel/src/recommend.ts @@ -193,6 +193,58 @@ export function computeProtectedRefs( // ─── 2. Build Compressible + Protected Ranges ──────────────────────────────── +/** Numeric part of a message ref ("m00005" → 5); null for non-m refs. */ +function refNumber(ref: string): number | null { + const m = /^m(\d+)$/i.exec(ref); + return m ? Number(m[1]) : null; +} + +/** True when a recorded-dead ref sits STRICTLY between two live ref numbers — + * the endpoints themselves are live (a dead endpoint can't be a compressible + * message), so only interior holes matter (#3192/#3205). */ +function hasDeadRefBetween( + deadRefNums: ReadonlySet, + lo: number, + hi: number, +): boolean { + for (const n of deadRefNums) if (n > lo && n < hi) return true; + return false; +} + +interface NoMergeTarget { + startRef: string; + endRef: string; + startIndex?: number; + endIndex?: number; + noMergeBefore?: boolean; +} + +/** #3192/#3205: tag every range that opens right after a dead-ref hole so the + * position-based mergers (mergeRangesToThreshold, formatRanges) cannot bridge + * across it — a dropped message leaves no array slot, so the hole has no index + * gap and position adjacency alone would re-emit one doomed span bracketing the + * dead ref. Runs over compressible + protected together so cross-type + * boundaries are covered too. */ +function markNoMergeAcrossDeadHoles( + ranges: NoMergeTarget[], + deadRefNums: ReadonlySet, +): void { + if (deadRefNums.size === 0 || ranges.length < 2) return; + const startPos = (r: NoMergeTarget): number => + r.startIndex ?? refNumber(r.startRef) ?? 0; + const order = [...ranges].sort((a, b) => startPos(a) - startPos(b)); + for (let i = 1; i < order.length; i++) { + const prevEnd = refNumber(order[i - 1]!.endRef); + const curStart = refNumber(order[i]!.startRef); + if (prevEnd === null || curStart === null) continue; + const lo = Math.min(prevEnd, curStart); + const hi = Math.max(prevEnd, curStart); + if (hi - lo > 1 && hasDeadRefBetween(deadRefNums, lo, hi)) { + order[i]!.noMergeBefore = true; + } + } +} + /** * Build compressible and protected range groups from the message list. * @@ -257,6 +309,15 @@ export function buildCompressibleRanges( let skipSinceCompressible = false; let skipSinceProtected = false; let msgIndex = -1; + // #3192/#3205: numeric set of recorded-dead refs. A dropped message leaves a + // hole in the ref map that array-adjacency segmentation bridges, so we break + // groups at those holes (checked right before each compressible push below). + const deadRefNums = new Set( + (state.deadRefs ?? []) + .map((ref) => refNumber(ref)) + .filter((n): n is number => n !== null), + ); + let lastCompressibleRefNum: number | null = null; for (const msg of messages) { msgIndex++; @@ -315,6 +376,21 @@ export function buildCompressibleRanges( continue; } + // #3192/#3205: break the group at a dead-ref hole between this message and + // the previous compressible one (a dropped message leaves no array slot, so + // no earlier gap flag fires). Same level as covered/pruned/carrier gaps. + const curRefNum = refNumber(ref); + if ( + deadRefNums.size > 0 && + curRefNum !== null && + lastCompressibleRefNum !== null + ) { + const lo = Math.min(curRefNum, lastCompressibleRefNum); + const hi = Math.max(curRefNum, lastCompressibleRefNum); + if (hi - lo > 1 && hasDeadRefBetween(deadRefNums, lo, hi)) { + skipSinceCompressible = true; + } + } compressibleMsgs.push({ id: msg.id, ref, @@ -325,6 +401,7 @@ export function buildCompressibleRanges( isUser: msg.role === "user", index: msgIndex, }); + if (curRefNum !== null) lastCompressibleRefNum = curRefNum; skipSinceCompressible = false; skipSinceProtected = true; } @@ -428,6 +505,8 @@ export function buildCompressibleRanges( } if (pcur) protectedRanges.push(pcur); + markNoMergeAcrossDeadHoles([...compressible, ...protectedRanges], deadRefNums); + return { compressible: compressible.filter((g) => g.tokens > 0), protected: protectedRanges, @@ -460,6 +539,10 @@ function mergeBatch(batch: CompressibleRange[]): CompressibleRange { if (batch.some((r) => r.dangerous === true)) { merged.dangerous = true; } + // #3192/#3205: carry the leading dead-hole boundary forward so downstream + // position-based merging (formatRanges) still sees it after this batch is + // re-emitted as a single range. + if (first.noMergeBefore === true) merged.noMergeBefore = true; return merged; } @@ -502,6 +585,9 @@ export function mergeRangesToThreshold( } }; for (const r of ranges) { + // #3192/#3205: hard boundary at a dead-ref hole — never bridge across it, + // even though the hole carries no array-index gap (see noMergeBefore). + if (r.noMergeBefore) closeBatch(); const prev = batch[batch.length - 1]; // A physical array-index gap between the previous range and this one marks // a block / protected / pruned boundary (buildCompressibleRanges leaves a @@ -522,9 +608,10 @@ export function mergeRangesToThreshold( if (batch.length > 0 && result.length > 0) { const prev = result[result.length - 1]!; const contiguous = - prev.endIndex === undefined || - batch[0]!.startIndex === undefined || - batch[0]!.startIndex <= prev.endIndex + 1; + !batch[0]!.noMergeBefore && + (prev.endIndex === undefined || + batch[0]!.startIndex === undefined || + batch[0]!.startIndex <= prev.endIndex + 1); if (contiguous) { result[result.length - 1] = mergeBatch([prev, ...batch]); } else if (batchChars >= minChars) { diff --git a/kernel/src/types.ts b/kernel/src/types.ts index 00c9479f88..22523eeea7 100644 --- a/kernel/src/types.ts +++ b/kernel/src/types.ts @@ -130,6 +130,8 @@ export interface AbsorbConfig { contextThresholdPct: number; /** Tool-name patterns (glob suffix allowed) never absorbable, independent of protectedTools. */ excludeTools: string[]; + /** #3402: when true, only image-bearing tool results are candidates, and they bypass minToolTokens. */ + mediaOnly?: boolean; } /** Emitted when the session's irreducible floor (system prompt + block @@ -467,6 +469,11 @@ export interface CompressibleRange { startIndex?: number; /** Messages-array index of this range's last message. */ endIndex?: number; + /** #3192/#3205: buildCompressibleRanges sets this when the range opens right + * after a dead-ref hole. A dropped message leaves NO array slot, so the hole + * has no index gap and the index check alone would let mergeRangesToThreshold + * bridge back into one doomed span bracketing the dead ref. */ + noMergeBefore?: boolean; } export interface ProtectedRange { @@ -479,6 +486,10 @@ export interface ProtectedRange { startIndex?: number; /** Messages-array index of last message. */ endIndex?: number; + /** #3192/#3205: see CompressibleRange.noMergeBefore — set when this protected + * range opens right after a dead-ref hole, so formatRanges does not bridge a + * preceding compressible range across the hole into one doomed span. */ + noMergeBefore?: boolean; } export interface ContextRanges { diff --git a/kernel/tests/absorb.test.ts b/kernel/tests/absorb.test.ts index d4d24cb968..caba5271a6 100644 --- a/kernel/tests/absorb.test.ts +++ b/kernel/tests/absorb.test.ts @@ -8,6 +8,7 @@ import { hideAbsorbedMessages, isAbsorbCandidate, parseAbsorbInput, + DEFAULT_ABSORB_CONFIG, ABSORB_PROMPT_MARKER, } from "../src/absorb.js"; import { createCore } from "../src/compress.js"; @@ -66,6 +67,10 @@ function session(): CoreMessage[] { ]; } +// mediaOnly is pinned false here on purpose: these tests exercise the absorb +// MECHANISM (prompting/hiding/stats/skip rules) in isolation over text results. +// The #3402 media-scope gate (default true) is covered by dedicated tests at +// the bottom of this file — mixing it in would make every fixture need a payload. function absorbConfig( overrides: Partial = {}, ): Config["absorb"] { @@ -75,10 +80,17 @@ function absorbConfig( minToolTokens: 1000, contextThresholdPct: 0, excludeTools: [], + mediaOnly: false, ...overrides, }; } +function mediaResult(id: string, callId: string, toolName = "screenshot"): CoreMessage { + return Object.assign(smallResult(id, callId, toolName), { + imageBase64: "iVBORw0KGgoAAAANSUhEUg==", + }); +} + function stateWithRefs(messages: CoreMessage[]): CompressionState { const state = createInitialState(); state.messageRefs = assignRefs(messages, { @@ -408,6 +420,7 @@ test("absorbed records survive applyCompression cloning", () => { tokenCount: 100, }).state; const config = defaultConfig(200000, { + absorb: absorbConfig(), compress: { minCompressRange: 0, maxSummaryLength: 20000, @@ -479,3 +492,66 @@ test("absorb config validation reports range errors", () => { ); assert.ok(errors.some((e) => e.includes("absorb.contextThresholdPct"))); }); + +// --- #3402 media-scope gate (revival of absorb, scoped to image-bearing results) --- + +test("isAbsorbCandidate: mediaOnly gates on hasMediaPayload", () => { + const textRes = bigResult("r1", "call1"); + const mediaRes = mediaResult("r2", "call2"); + const on = defaultConfig(200000, { absorb: absorbConfig({ mediaOnly: true }) }); + const off = defaultConfig(200000, { absorb: absorbConfig({ mediaOnly: false }) }); + assert.equal(isAbsorbCandidate(textRes, on), false, "mediaOnly on + no media → not a candidate"); + assert.equal(isAbsorbCandidate(mediaRes, on), true, "mediaOnly on + media → candidate"); + assert.equal(isAbsorbCandidate(textRes, off), true, "mediaOnly off + no media → candidate (old behavior)"); + assert.equal(isAbsorbCandidate(mediaRes, off), true, "mediaOnly off + media → candidate"); +}); + +test("appendAbsorbPrompts: mediaOnly marks image results (even small) and skips text results", () => { + const build = (): CoreMessage[] => [ + toolCall("c1", "call1", "read"), + bigResult("r1", "call1", "read"), + toolCall("c2", "call2", "screenshot"), + mediaResult("r2", "call2", "screenshot"), + ]; + const onMsgs = build(); + const onState = stateWithRefs(onMsgs); + const onCfg = defaultConfig(200000, { absorb: absorbConfig({ mediaOnly: true }) }); + const onRes = appendAbsorbPrompts(onMsgs, onState, onCfg, 5000, countTokens); + assert.ok(!onRes.messages.find((m) => m.id === "r1")!.text!.includes(ABSORB_PROMPT_MARKER), "text-only result NOT marked under mediaOnly"); + assert.ok(onRes.messages.find((m) => m.id === "r2")!.text!.includes(ABSORB_PROMPT_MARKER), "small image result IS marked (exempt from minToolTokens)"); + assert.equal(onRes.promptedCount, 1); + + const offMsgs = build(); + const offState = stateWithRefs(offMsgs); + const offCfg = defaultConfig(200000, { absorb: absorbConfig({ mediaOnly: false }) }); + const offRes = appendAbsorbPrompts(offMsgs, offState, offCfg, 5000, countTokens); + assert.ok(offRes.messages.find((m) => m.id === "r1")!.text!.includes(ABSORB_PROMPT_MARKER), "old behavior: large text result marked"); + assert.ok(!offRes.messages.find((m) => m.id === "r2")!.text!.includes(ABSORB_PROMPT_MARKER), "old behavior: tiny result below bar stays unmarked"); +}); + +test("applyAbsorb: mediaOnly rejects a non-media target (mirror of screening)", () => { + const messages: CoreMessage[] = [ + toolCall("c1", "call1", "read"), + bigResult("r1", "call1", "read"), + toolCall("c2", "call2", "screenshot"), + mediaResult("r2", "call2", "screenshot"), + ]; + const state = stateWithRefs(messages); + const cfg = defaultConfig(200000, { absorb: absorbConfig({ mediaOnly: true }) }); + const refused = applyAbsorb({ ref: refForRaw(state.messageRefs, "r1")!, summary: "s", messages, state, config: cfg }); + assert.equal(refused.ok, false); + assert.ok(refused.resultText.includes("no media payload")); + const accepted = applyAbsorb({ ref: refForRaw(state.messageRefs, "r2")!, summary: "s", messages, state, config: cfg }); + assert.ok(accepted.ok, accepted.resultText); + assert.equal(accepted.state.absorbed!.length, 1); +}); + +test("kernel ships mediaOnly neutral; the media-scoped default is host policy", () => { + // The engine exposes the capability with a neutral default (unset → legacy + // full-text when enabled) so engine goldens stay untouched; bili's product + // stance (media-scoped by default, #3402) is applied one layer up in + // resolveAbsorbSettings — pinned in tests/absorb.test.ts ("maps settings + // absorb onto kernel AbsorbConfig"). + assert.equal(defaultConfig(100000).absorb?.mediaOnly, undefined); + assert.equal(DEFAULT_ABSORB_CONFIG.mediaOnly, undefined); +}); diff --git a/kernel/tests/dead-ref-split.test.ts b/kernel/tests/dead-ref-split.test.ts new file mode 100644 index 0000000000..667f69490f --- /dev/null +++ b/kernel/tests/dead-ref-split.test.ts @@ -0,0 +1,127 @@ +import { test } from "node:test"; +import assert from "node:assert/strict"; +import { + buildCompressibleRanges, + mergeRangesToThreshold, +} from "../src/recommend.js"; +import { createInitialState } from "../src/state.js"; +import { assignRefs } from "../src/refs.js"; +import type { Config, CoreMessage } from "../src/types.js"; + +// #3192/#3205: a client-dropped message leaves a hole in the ref map that +// array-adjacency segmentation bridges, so buildCompressibleRanges must break +// groups at recorded-dead refs and emit adjacent live sub-spans — and +// mergeRangesToThreshold must not bridge them back across the hole. + +function config(overrides: Partial = {}): Config { + return { + tiers: { enabled: true, tier2Trigger: 5, tier3Trigger: 10 }, + nudge: { + maxContextLimitPct: 0.55, + minContextLimitPct: 0.45, + frequency: 5, + iterationThreshold: 15, + force: "soft", + growthRatio: 0.05, + }, + promotionThreshold: 5, + truncate: { threshold: 1 }, + merge: { maxSummaryLength: 3000, minOldGenBlocks: 3 }, + compress: { minCompressRange: 0, maxSummaryLength: 0, minSummaryLength: 0 }, + protectedTools: [], + preserveRecentMessages: 0, + preserveRecentTokens: 0, + modelContextLimit: 100000, + ...overrides, + }; +} + +function msg(id: string, text: string): CoreMessage { + return { id, role: "assistant", contentType: "text", text }; +} + +/** Mint refs m00001.. over the FULL id list, then hand back the state plus the + * source messages so a test can drop entries from the visible view. */ +function setup(ids: string[]): { messages: CoreMessage[]; state: ReturnType } { + const messages = ids.map((id) => msg(id, "x".repeat(2000))); + const state = createInitialState(); + state.messageRefs = assignRefs(messages, { + existing: state.messageRefs, + nextIndex: 1, + }).map; + return { messages, state }; +} + +function spans(ranges: { startRef: string; endRef: string }[]): [string, string][] { + return ranges.map((r) => [r.startRef, r.endRef]); +} + +test("buildCompressibleRanges splits around an interior dead ref into adjacent live sub-spans", () => { + const { messages, state } = setup(["a", "b", "c", "d", "e"]); // m00001..m00005 + state.deadRefs = ["m00003"]; // 'c' dropped by the client + const visible = [messages[0], messages[1], messages[3], messages[4]]; // a,b,d,e + const { compressible } = buildCompressibleRanges(visible, state, config()); + assert.deepEqual(spans(compressible), [ + ["m00001", "m00002"], + ["m00004", "m00005"], + ]); + assert.equal(compressible[0]!.noMergeBefore, undefined); + assert.equal(compressible[1]!.noMergeBefore, true); +}); + +test("control: a NON-dead ref-map hole stays bridged (deadRefs empty → unchanged behavior)", () => { + const { messages, state } = setup(["a", "b", "c", "d", "e"]); + // Same drop, but 'c' is NOT recorded dead → array adjacency bridges it as before. + const visible = [messages[0], messages[1], messages[3], messages[4]]; + const { compressible } = buildCompressibleRanges(visible, state, config()); + assert.deepEqual(spans(compressible), [["m00001", "m00005"]]); +}); + +test("multiple dead holes → one sub-span per surviving run, none brackets a dead ref", () => { + const { messages, state } = setup(["a", "b", "c", "d", "e", "f", "g"]); // m1..m7 + state.deadRefs = ["m00003", "m00006"]; // drop c and f + const visible = [ + messages[0], + messages[1], + messages[3], + messages[4], + messages[6], + ]; // a,b,d,e,g + const { compressible } = buildCompressibleRanges(visible, state, config()); + assert.deepEqual(spans(compressible), [ + ["m00001", "m00002"], + ["m00004", "m00005"], + ["m00007", "m00007"], + ]); + const dead = new Set([3, 6]); + for (const r of compressible) { + const s = Number(r.startRef.slice(1)); + const e = Number(r.endRef.slice(1)); + const [lo, hi] = s < e ? [s, e] : [e, s]; + for (let n = lo + 1; n < hi; n++) { + assert.ok(!dead.has(n), `span ${r.startRef}–${r.endRef} brackets dead ${n}`); + } + } +}); + +test("trailing dead ref never becomes a span endpoint", () => { + const { messages, state } = setup(["a", "b", "c", "d", "e"]); + state.deadRefs = ["m00005"]; // drop 'e' + const visible = [messages[0], messages[1], messages[2], messages[3]]; // a,b,c,d + const { compressible } = buildCompressibleRanges(visible, state, config()); + assert.deepEqual(spans(compressible), [["m00001", "m00004"]]); +}); + +test("mergeRangesToThreshold does NOT re-bridge across a dead-ref hole", () => { + const { messages, state } = setup(["a", "b", "c", "d", "e"]); + state.deadRefs = ["m00003"]; + const visible = [messages[0], messages[1], messages[3], messages[4]]; + const { compressible } = buildCompressibleRanges(visible, state, config()); + // Each sub-span is ~4000 chars, well above the 1000 gate — without the + // noMergeBefore flag these would merge back into one doomed m00001–m00005. + const merged = mergeRangesToThreshold(compressible, 1000); + assert.deepEqual(spans(merged), [ + ["m00001", "m00002"], + ["m00004", "m00005"], + ]); +}); diff --git a/package-lock.json b/package-lock.json index 29da925464..abc0afabfe 100644 --- a/package-lock.json +++ b/package-lock.json @@ -26,6 +26,7 @@ "billion-context-pi-subagents": "file:./pi-subagents", "esbuild": "^0.27.0", "fzstd": "0.1.1", + "jimp": "1.6.1", "jsonc-parser": "^3.3.1", "node-forge": "^1.4.0", "prettier": "^3.3.0", @@ -45,7 +46,7 @@ }, "kernel": { "name": "acp-kernel", - "version": "0.0.128", + "version": "0.0.130", "dev": true, "license": "MIT", "devDependencies": { @@ -59,6 +60,17 @@ "node": ">=20" } }, + "node_modules/@borewit/text-codec": { + "version": "0.2.2", + "resolved": "https://registry.npmjs.org/@borewit/text-codec/-/text-codec-0.2.2.tgz", + "integrity": "sha512-DDaRehssg1aNrH4+2hnj1B7vnUGEjU6OIlyRdkMd0aUdIUvKXrJfXsy8LVtXAy7DRvYVluWbMspsRhz2lcW0mQ==", + "dev": true, + "license": "MIT", + "funding": { + "type": "github", + "url": "https://github.com/sponsors/Borewit" + } + }, "node_modules/@cypress/request": { "version": "4.0.1", "resolved": "https://registry.npmjs.org/@cypress/request/-/request-4.0.1.tgz", @@ -2410,110 +2422,710 @@ ], "dev": true, "license": "MIT", - "optional": true, - "os": [ - "openbsd" - ], + "optional": true, + "os": [ + "openbsd" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/openharmony-arm64": { + "version": "0.27.7", + "resolved": "https://registry.npmjs.org/@esbuild/openharmony-arm64/-/openharmony-arm64-0.27.7.tgz", + "integrity": "sha512-+KrvYb/C8zA9CU/g0sR6w2RBw7IGc5J2BPnc3dYc5VJxHCSF1yNMxTV5LQ7GuKteQXZtspjFbiuW5/dOj7H4Yw==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "openharmony" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/sunos-x64": { + "version": "0.27.7", + "resolved": "https://registry.npmjs.org/@esbuild/sunos-x64/-/sunos-x64-0.27.7.tgz", + "integrity": "sha512-ikktIhFBzQNt/QDyOL580ti9+5mL/YZeUPKU2ivGtGjdTYoqz6jObj6nOMfhASpS4GU4Q/Clh1QtxWAvcYKamA==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "sunos" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/win32-arm64": { + "version": "0.27.7", + "resolved": "https://registry.npmjs.org/@esbuild/win32-arm64/-/win32-arm64-0.27.7.tgz", + "integrity": "sha512-7yRhbHvPqSpRUV7Q20VuDwbjW5kIMwTHpptuUzV+AA46kiPze5Z7qgt6CLCK3pWFrHeNfDd1VKgyP4O+ng17CA==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/win32-ia32": { + "version": "0.27.7", + "resolved": "https://registry.npmjs.org/@esbuild/win32-ia32/-/win32-ia32-0.27.7.tgz", + "integrity": "sha512-SmwKXe6VHIyZYbBLJrhOoCJRB/Z1tckzmgTLfFYOfpMAx63BJEaL9ExI8x7v0oAO3Zh6D/Oi1gVxEYr5oUCFhw==", + "cpu": [ + "ia32" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@esbuild/win32-x64": { + "version": "0.27.7", + "resolved": "https://registry.npmjs.org/@esbuild/win32-x64/-/win32-x64-0.27.7.tgz", + "integrity": "sha512-56hiAJPhwQ1R4i+21FVF7V8kSD5zZTdHcVuRFMW0hn753vVfQN8xlx4uOPT4xoGH0Z/oVATuR82AiqSTDIpaHg==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">=18" + } + }, + "node_modules/@isaacs/fs-minipass": { + "version": "4.0.1", + "resolved": "https://registry.npmjs.org/@isaacs/fs-minipass/-/fs-minipass-4.0.1.tgz", + "integrity": "sha512-wgm9Ehl2jpeqP3zw/7mo3kRHFp5MEDhqAdwy1fTGkHAwnkGOVsgpvQhL8B5n1qlb01jV3n/bI0ZfZp5lWA1k4w==", + "dev": true, + "license": "ISC", + "dependencies": { + "minipass": "^7.0.4" + }, + "engines": { + "node": ">=18.0.0" + } + }, + "node_modules/@jimp/core": { + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/@jimp/core/-/core-1.6.1.tgz", + "integrity": "sha512-+BoKC5G6hkrSy501zcJ2EpfnllP+avPevcBfRcZe/CW+EwEfY6X1EZ8QWyT7NpDIvEEJb1fdJnMMfUnFkxmw9A==", + "dev": true, + "license": "MIT", + "dependencies": { + "@jimp/file-ops": "1.6.1", + "@jimp/types": "1.6.1", + "@jimp/utils": "1.6.1", + "await-to-js": "^3.0.0", + "exif-parser": "^0.1.12", + "file-type": "^21.3.3", + "mime": "3" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@jimp/diff": { + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/@jimp/diff/-/diff-1.6.1.tgz", + "integrity": "sha512-YkKDPdHjLgo1Api3+Bhc0GLAygldlpt97NfOKoNg1U6IUNXA6X2MgosCjPfSBiSvJvrrz1fsIR+/4cfYXBI/HQ==", + "dev": true, + "license": "MIT", + "dependencies": { + "@jimp/plugin-resize": "1.6.1", + "@jimp/types": "1.6.1", + "@jimp/utils": "1.6.1", + "pixelmatch": "^5.3.0" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@jimp/file-ops": { + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/@jimp/file-ops/-/file-ops-1.6.1.tgz", + "integrity": "sha512-T+gX6osHjprbDRad0/B71Evyre7ZdVY1z/gFGEG9Z8KOtZPKboWvPeP2UjbZYWQLy9UKCPQX1FNAnDiOPkJL7w==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=18" + } + }, + "node_modules/@jimp/js-bmp": { + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/@jimp/js-bmp/-/js-bmp-1.6.1.tgz", + "integrity": "sha512-xzWzNT4/u5zGrTT3Tme9sGU7YzIKxi13+BCQwLqACbt5DXf9SAfdzRkopZQnmDko+6In5nqaT89Gjs43/WdnYQ==", + "dev": true, + "license": "MIT", + "dependencies": { + "@jimp/core": "1.6.1", + "@jimp/types": "1.6.1", + "@jimp/utils": "1.6.1", + "bmp-ts": "^1.0.9" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@jimp/js-gif": { + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/@jimp/js-gif/-/js-gif-1.6.1.tgz", + "integrity": "sha512-YjY2W26rQa05XhanYhRZ7dingCiNN+T2Ymb1JiigIbABY0B28wHE3v3Cf1/HZPWGu0hOg36ylaKgV5KxF2M58w==", + "dev": true, + "license": "MIT", + "dependencies": { + "@jimp/core": "1.6.1", + "@jimp/types": "1.6.1", + "gifwrap": "^0.10.1", + "omggif": "^1.0.10" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@jimp/js-jpeg": { + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/@jimp/js-jpeg/-/js-jpeg-1.6.1.tgz", + "integrity": "sha512-HT9H3yOmlOFzYmdI15IYdfy6ggQhSRIaHeA+OTJSEORXBqEo97sUZu/DsgHIcX5NJ7TkJBTgZ9BZXsV6UbsyMg==", + "dev": true, + "license": "MIT", + "dependencies": { + "@jimp/core": "1.6.1", + "@jimp/types": "1.6.1", + "jpeg-js": "^0.4.4" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@jimp/js-png": { + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/@jimp/js-png/-/js-png-1.6.1.tgz", + "integrity": "sha512-SZ/KVhI5UjcSzzlXsXdIi/LhJ7UShf2NkMOtVrbZQcGzsqNtynAelrOXeoTxcanfVqmNhAoVHg8yR2cYoqrYjA==", + "dev": true, + "license": "MIT", + "dependencies": { + "@jimp/core": "1.6.1", + "@jimp/types": "1.6.1", + "pngjs": "^7.0.0" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@jimp/js-tiff": { + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/@jimp/js-tiff/-/js-tiff-1.6.1.tgz", + "integrity": "sha512-jDG/eJquID1M4MBlKMmDRBmz2TpXMv7TUyu2nIRUxhlUc2ogC82T+VQUkca9GJH1BBJ9dx5sSE5dGkWNjIbZxw==", + "dev": true, + "license": "MIT", + "dependencies": { + "@jimp/core": "1.6.1", + "@jimp/types": "1.6.1", + "utif2": "^4.1.0" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@jimp/plugin-blit": { + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/@jimp/plugin-blit/-/plugin-blit-1.6.1.tgz", + "integrity": "sha512-MwnI7C7K81uWddY9FLw1fCOIy6SsPIUftUz36Spt7jisCn8/40DhQMlSxpxTNelnZb/2SnloFimQfRZAmHLOqQ==", + "dev": true, + "license": "MIT", + "dependencies": { + "@jimp/types": "1.6.1", + "@jimp/utils": "1.6.1", + "zod": "^3.23.8" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@jimp/plugin-blit/node_modules/zod": { + "version": "3.25.76", + "resolved": "https://registry.npmjs.org/zod/-/zod-3.25.76.tgz", + "integrity": "sha512-gzUt/qt81nXsFGKIFcC3YnfEAx5NkunCfnDlvuBSSFS02bcXu4Lmea0AFIUwbLWxWPx3d9p8S5QoaujKcNQxcQ==", + "dev": true, + "license": "MIT", + "funding": { + "url": "https://github.com/sponsors/colinhacks" + } + }, + "node_modules/@jimp/plugin-blur": { + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/@jimp/plugin-blur/-/plugin-blur-1.6.1.tgz", + "integrity": "sha512-lIo7Tzp5jQu30EFFSK/phXANK3citKVEjepDjQ6ljHoIFtuMRrnybnmI2Md24ulvWlDaz+hh3n6qrMb8ydwhZQ==", + "dev": true, + "license": "MIT", + "dependencies": { + "@jimp/core": "1.6.1", + "@jimp/utils": "1.6.1" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@jimp/plugin-circle": { + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/@jimp/plugin-circle/-/plugin-circle-1.6.1.tgz", + "integrity": "sha512-kK1PavY6cKHNNKce37vdV4Tmpc1/zDKngGoeOV3j+EMatoHFZUinV3s6F9aWryPs3A0xhCLZgdJ6Zeea1d5LCQ==", + "dev": true, + "license": "MIT", + "dependencies": { + "@jimp/types": "1.6.1", + "zod": "^3.23.8" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@jimp/plugin-circle/node_modules/zod": { + "version": "3.25.76", + "resolved": "https://registry.npmjs.org/zod/-/zod-3.25.76.tgz", + "integrity": "sha512-gzUt/qt81nXsFGKIFcC3YnfEAx5NkunCfnDlvuBSSFS02bcXu4Lmea0AFIUwbLWxWPx3d9p8S5QoaujKcNQxcQ==", + "dev": true, + "license": "MIT", + "funding": { + "url": "https://github.com/sponsors/colinhacks" + } + }, + "node_modules/@jimp/plugin-color": { + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/@jimp/plugin-color/-/plugin-color-1.6.1.tgz", + "integrity": "sha512-LtUN1vAP+LRlZAtTNVhDRSiXx+26Kbz3zJaG6a5k59gQ95jgT5mknnF8lxkHcqJthM4MEk3/tPxkdJpEybyF/A==", + "dev": true, + "license": "MIT", + "dependencies": { + "@jimp/core": "1.6.1", + "@jimp/types": "1.6.1", + "@jimp/utils": "1.6.1", + "tinycolor2": "^1.6.0", + "zod": "^3.23.8" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@jimp/plugin-color/node_modules/zod": { + "version": "3.25.76", + "resolved": "https://registry.npmjs.org/zod/-/zod-3.25.76.tgz", + "integrity": "sha512-gzUt/qt81nXsFGKIFcC3YnfEAx5NkunCfnDlvuBSSFS02bcXu4Lmea0AFIUwbLWxWPx3d9p8S5QoaujKcNQxcQ==", + "dev": true, + "license": "MIT", + "funding": { + "url": "https://github.com/sponsors/colinhacks" + } + }, + "node_modules/@jimp/plugin-contain": { + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/@jimp/plugin-contain/-/plugin-contain-1.6.1.tgz", + "integrity": "sha512-m0qhrfA8jkTqretGv4w+T/ADFR4GwBpE0sCOC2uJ0dzr44/ddOMsIdrpi89kabqYiPYIrxkgdCVCLm3zn1Vkkg==", + "dev": true, + "license": "MIT", + "dependencies": { + "@jimp/core": "1.6.1", + "@jimp/plugin-blit": "1.6.1", + "@jimp/plugin-resize": "1.6.1", + "@jimp/types": "1.6.1", + "@jimp/utils": "1.6.1", + "zod": "^3.23.8" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@jimp/plugin-contain/node_modules/zod": { + "version": "3.25.76", + "resolved": "https://registry.npmjs.org/zod/-/zod-3.25.76.tgz", + "integrity": "sha512-gzUt/qt81nXsFGKIFcC3YnfEAx5NkunCfnDlvuBSSFS02bcXu4Lmea0AFIUwbLWxWPx3d9p8S5QoaujKcNQxcQ==", + "dev": true, + "license": "MIT", + "funding": { + "url": "https://github.com/sponsors/colinhacks" + } + }, + "node_modules/@jimp/plugin-cover": { + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/@jimp/plugin-cover/-/plugin-cover-1.6.1.tgz", + "integrity": "sha512-hZytnsth0zoll6cPf434BrT+p/v569Wr5tyO6Dp0dH1IDPhzhB5F38sZGMLDo7bzQiN9JFVB3fxkcJ/WYCJ3Mg==", + "dev": true, + "license": "MIT", + "dependencies": { + "@jimp/core": "1.6.1", + "@jimp/plugin-crop": "1.6.1", + "@jimp/plugin-resize": "1.6.1", + "@jimp/types": "1.6.1", + "zod": "^3.23.8" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@jimp/plugin-cover/node_modules/zod": { + "version": "3.25.76", + "resolved": "https://registry.npmjs.org/zod/-/zod-3.25.76.tgz", + "integrity": "sha512-gzUt/qt81nXsFGKIFcC3YnfEAx5NkunCfnDlvuBSSFS02bcXu4Lmea0AFIUwbLWxWPx3d9p8S5QoaujKcNQxcQ==", + "dev": true, + "license": "MIT", + "funding": { + "url": "https://github.com/sponsors/colinhacks" + } + }, + "node_modules/@jimp/plugin-crop": { + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/@jimp/plugin-crop/-/plugin-crop-1.6.1.tgz", + "integrity": "sha512-EerRSLlclXyKDnYc/H9w/1amZW7b7v3OGi/VlerPd2M/pAu5X8TkyYWtfqYCXnNp1Ixtd8oCo9zGfY9zoXT4rg==", + "dev": true, + "license": "MIT", + "dependencies": { + "@jimp/core": "1.6.1", + "@jimp/types": "1.6.1", + "@jimp/utils": "1.6.1", + "zod": "^3.23.8" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@jimp/plugin-crop/node_modules/zod": { + "version": "3.25.76", + "resolved": "https://registry.npmjs.org/zod/-/zod-3.25.76.tgz", + "integrity": "sha512-gzUt/qt81nXsFGKIFcC3YnfEAx5NkunCfnDlvuBSSFS02bcXu4Lmea0AFIUwbLWxWPx3d9p8S5QoaujKcNQxcQ==", + "dev": true, + "license": "MIT", + "funding": { + "url": "https://github.com/sponsors/colinhacks" + } + }, + "node_modules/@jimp/plugin-displace": { + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/@jimp/plugin-displace/-/plugin-displace-1.6.1.tgz", + "integrity": "sha512-K07QVl7xQwIfD6KfxRV/c3E9e7ZBXxUXdWuvoTWcKHL2qV48MOF5Nqbz/aJW4ThnQARIsxvYlZjPFiqkCjlU+g==", + "dev": true, + "license": "MIT", + "dependencies": { + "@jimp/types": "1.6.1", + "@jimp/utils": "1.6.1", + "zod": "^3.23.8" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@jimp/plugin-displace/node_modules/zod": { + "version": "3.25.76", + "resolved": "https://registry.npmjs.org/zod/-/zod-3.25.76.tgz", + "integrity": "sha512-gzUt/qt81nXsFGKIFcC3YnfEAx5NkunCfnDlvuBSSFS02bcXu4Lmea0AFIUwbLWxWPx3d9p8S5QoaujKcNQxcQ==", + "dev": true, + "license": "MIT", + "funding": { + "url": "https://github.com/sponsors/colinhacks" + } + }, + "node_modules/@jimp/plugin-dither": { + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/@jimp/plugin-dither/-/plugin-dither-1.6.1.tgz", + "integrity": "sha512-+2V+GCV2WycMoX1/z977TkZ8Zq/4MVSKElHYatgUqtwXMi2fDK2gKYU2g9V39IqFvTJsTIsK0+58VFz/ROBVew==", + "dev": true, + "license": "MIT", + "dependencies": { + "@jimp/types": "1.6.1" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@jimp/plugin-fisheye": { + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/@jimp/plugin-fisheye/-/plugin-fisheye-1.6.1.tgz", + "integrity": "sha512-XtS5ZyoZ0vxZxJ6gkqI63SivhtI58vX95foMPM+cyzYkRsJXMOYCr8DScxF5bp4Xr003NjYm/P+7+08tibwzHA==", + "dev": true, + "license": "MIT", + "dependencies": { + "@jimp/types": "1.6.1", + "@jimp/utils": "1.6.1", + "zod": "^3.23.8" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@jimp/plugin-fisheye/node_modules/zod": { + "version": "3.25.76", + "resolved": "https://registry.npmjs.org/zod/-/zod-3.25.76.tgz", + "integrity": "sha512-gzUt/qt81nXsFGKIFcC3YnfEAx5NkunCfnDlvuBSSFS02bcXu4Lmea0AFIUwbLWxWPx3d9p8S5QoaujKcNQxcQ==", + "dev": true, + "license": "MIT", + "funding": { + "url": "https://github.com/sponsors/colinhacks" + } + }, + "node_modules/@jimp/plugin-flip": { + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/@jimp/plugin-flip/-/plugin-flip-1.6.1.tgz", + "integrity": "sha512-ws38W/sGj7LobNRayQ83garxiktOyWxM5vO/y4a/2cy9v65SLEUzVkrj+oeAaUSSObdz4HcCEla7XtGlnAGAaA==", + "dev": true, + "license": "MIT", + "dependencies": { + "@jimp/types": "1.6.1", + "zod": "^3.23.8" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@jimp/plugin-flip/node_modules/zod": { + "version": "3.25.76", + "resolved": "https://registry.npmjs.org/zod/-/zod-3.25.76.tgz", + "integrity": "sha512-gzUt/qt81nXsFGKIFcC3YnfEAx5NkunCfnDlvuBSSFS02bcXu4Lmea0AFIUwbLWxWPx3d9p8S5QoaujKcNQxcQ==", + "dev": true, + "license": "MIT", + "funding": { + "url": "https://github.com/sponsors/colinhacks" + } + }, + "node_modules/@jimp/plugin-hash": { + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/@jimp/plugin-hash/-/plugin-hash-1.6.1.tgz", + "integrity": "sha512-sZt6ZcMX6i8vFWb4GYnw0pR/o9++ef0dTVcboTB5B/g7nrxCODIB4wfEkJ/YqZM5wUvol77K1qeS0/rVO6z21A==", + "dev": true, + "license": "MIT", + "dependencies": { + "@jimp/core": "1.6.1", + "@jimp/js-bmp": "1.6.1", + "@jimp/js-jpeg": "1.6.1", + "@jimp/js-png": "1.6.1", + "@jimp/js-tiff": "1.6.1", + "@jimp/plugin-color": "1.6.1", + "@jimp/plugin-resize": "1.6.1", + "@jimp/types": "1.6.1", + "@jimp/utils": "1.6.1", + "any-base": "^1.1.0" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@jimp/plugin-mask": { + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/@jimp/plugin-mask/-/plugin-mask-1.6.1.tgz", + "integrity": "sha512-SIG0/FcmEj3tkwFxc7fAGLO8o4uNzMpSOdQOhbCgxefQKq5wOVMk9BQx/sdMPBwtMLr9WLq0GzLA/rk6t2v20A==", + "dev": true, + "license": "MIT", + "dependencies": { + "@jimp/types": "1.6.1", + "zod": "^3.23.8" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@jimp/plugin-mask/node_modules/zod": { + "version": "3.25.76", + "resolved": "https://registry.npmjs.org/zod/-/zod-3.25.76.tgz", + "integrity": "sha512-gzUt/qt81nXsFGKIFcC3YnfEAx5NkunCfnDlvuBSSFS02bcXu4Lmea0AFIUwbLWxWPx3d9p8S5QoaujKcNQxcQ==", + "dev": true, + "license": "MIT", + "funding": { + "url": "https://github.com/sponsors/colinhacks" + } + }, + "node_modules/@jimp/plugin-print": { + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/@jimp/plugin-print/-/plugin-print-1.6.1.tgz", + "integrity": "sha512-BYVz/X3Xzv8XYilVeDy11NOp0h7BTDjlOtu0BekIFHP1yHVd24AXNzbOy52XlzYZWQ0Dl36HOHEpl/nSNrzc6w==", + "dev": true, + "license": "MIT", + "dependencies": { + "@jimp/core": "1.6.1", + "@jimp/js-jpeg": "1.6.1", + "@jimp/js-png": "1.6.1", + "@jimp/plugin-blit": "1.6.1", + "@jimp/types": "1.6.1", + "parse-bmfont-ascii": "^1.0.6", + "parse-bmfont-binary": "^1.0.6", + "parse-bmfont-xml": "^1.1.6", + "simple-xml-to-json": "^1.2.2", + "zod": "^3.23.8" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@jimp/plugin-print/node_modules/zod": { + "version": "3.25.76", + "resolved": "https://registry.npmjs.org/zod/-/zod-3.25.76.tgz", + "integrity": "sha512-gzUt/qt81nXsFGKIFcC3YnfEAx5NkunCfnDlvuBSSFS02bcXu4Lmea0AFIUwbLWxWPx3d9p8S5QoaujKcNQxcQ==", + "dev": true, + "license": "MIT", + "funding": { + "url": "https://github.com/sponsors/colinhacks" + } + }, + "node_modules/@jimp/plugin-quantize": { + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/@jimp/plugin-quantize/-/plugin-quantize-1.6.1.tgz", + "integrity": "sha512-J2En9PLURfP+vwYDtuZ9T8yBW6BWYZBScydAjRiPBmJfEhTcNQqiiQODrZf7EqbbX/Sy5H6dAeRiqkgoV9N6Ww==", + "dev": true, + "license": "MIT", + "dependencies": { + "image-q": "^4.0.0", + "zod": "^3.23.8" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@jimp/plugin-quantize/node_modules/zod": { + "version": "3.25.76", + "resolved": "https://registry.npmjs.org/zod/-/zod-3.25.76.tgz", + "integrity": "sha512-gzUt/qt81nXsFGKIFcC3YnfEAx5NkunCfnDlvuBSSFS02bcXu4Lmea0AFIUwbLWxWPx3d9p8S5QoaujKcNQxcQ==", + "dev": true, + "license": "MIT", + "funding": { + "url": "https://github.com/sponsors/colinhacks" + } + }, + "node_modules/@jimp/plugin-resize": { + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/@jimp/plugin-resize/-/plugin-resize-1.6.1.tgz", + "integrity": "sha512-CLkrtJoIz2HdWnpYiN6p8KYcPc00rCH/SUu6o+lfZL05Q4uhecJlnvXuj9x+U6mDn3ldPmJj6aZqMHuUJzdVqg==", + "dev": true, + "license": "MIT", + "dependencies": { + "@jimp/core": "1.6.1", + "@jimp/types": "1.6.1", + "zod": "^3.23.8" + }, + "engines": { + "node": ">=18" + } + }, + "node_modules/@jimp/plugin-resize/node_modules/zod": { + "version": "3.25.76", + "resolved": "https://registry.npmjs.org/zod/-/zod-3.25.76.tgz", + "integrity": "sha512-gzUt/qt81nXsFGKIFcC3YnfEAx5NkunCfnDlvuBSSFS02bcXu4Lmea0AFIUwbLWxWPx3d9p8S5QoaujKcNQxcQ==", + "dev": true, + "license": "MIT", + "funding": { + "url": "https://github.com/sponsors/colinhacks" + } + }, + "node_modules/@jimp/plugin-rotate": { + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/@jimp/plugin-rotate/-/plugin-rotate-1.6.1.tgz", + "integrity": "sha512-nOjVjbbj705B02ksysKnh0POAwEBXZtJ9zQ5qC+X7Tavl3JNn+P3BzQovbBxLPSbUSld6XID9z5ijin4PtOAUg==", + "dev": true, + "license": "MIT", + "dependencies": { + "@jimp/core": "1.6.1", + "@jimp/plugin-crop": "1.6.1", + "@jimp/plugin-resize": "1.6.1", + "@jimp/types": "1.6.1", + "@jimp/utils": "1.6.1", + "zod": "^3.23.8" + }, "engines": { "node": ">=18" } }, - "node_modules/@esbuild/openharmony-arm64": { - "version": "0.27.7", - "resolved": "https://registry.npmjs.org/@esbuild/openharmony-arm64/-/openharmony-arm64-0.27.7.tgz", - "integrity": "sha512-+KrvYb/C8zA9CU/g0sR6w2RBw7IGc5J2BPnc3dYc5VJxHCSF1yNMxTV5LQ7GuKteQXZtspjFbiuW5/dOj7H4Yw==", - "cpu": [ - "arm64" - ], + "node_modules/@jimp/plugin-rotate/node_modules/zod": { + "version": "3.25.76", + "resolved": "https://registry.npmjs.org/zod/-/zod-3.25.76.tgz", + "integrity": "sha512-gzUt/qt81nXsFGKIFcC3YnfEAx5NkunCfnDlvuBSSFS02bcXu4Lmea0AFIUwbLWxWPx3d9p8S5QoaujKcNQxcQ==", "dev": true, "license": "MIT", - "optional": true, - "os": [ - "openharmony" - ], - "engines": { - "node": ">=18" + "funding": { + "url": "https://github.com/sponsors/colinhacks" } }, - "node_modules/@esbuild/sunos-x64": { - "version": "0.27.7", - "resolved": "https://registry.npmjs.org/@esbuild/sunos-x64/-/sunos-x64-0.27.7.tgz", - "integrity": "sha512-ikktIhFBzQNt/QDyOL580ti9+5mL/YZeUPKU2ivGtGjdTYoqz6jObj6nOMfhASpS4GU4Q/Clh1QtxWAvcYKamA==", - "cpu": [ - "x64" - ], + "node_modules/@jimp/plugin-threshold": { + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/@jimp/plugin-threshold/-/plugin-threshold-1.6.1.tgz", + "integrity": "sha512-JOKv9F8s6tnVLf4sB/2fF0F339EFnHvgEdFYugO6VhowKLsap0pEZmLyE/DlRnYtIj2RddHZVxVMp/eKJ04l2Q==", "dev": true, "license": "MIT", - "optional": true, - "os": [ - "sunos" - ], + "dependencies": { + "@jimp/core": "1.6.1", + "@jimp/plugin-color": "1.6.1", + "@jimp/plugin-hash": "1.6.1", + "@jimp/types": "1.6.1", + "@jimp/utils": "1.6.1", + "zod": "^3.23.8" + }, "engines": { "node": ">=18" } }, - "node_modules/@esbuild/win32-arm64": { - "version": "0.27.7", - "resolved": "https://registry.npmjs.org/@esbuild/win32-arm64/-/win32-arm64-0.27.7.tgz", - "integrity": "sha512-7yRhbHvPqSpRUV7Q20VuDwbjW5kIMwTHpptuUzV+AA46kiPze5Z7qgt6CLCK3pWFrHeNfDd1VKgyP4O+ng17CA==", - "cpu": [ - "arm64" - ], + "node_modules/@jimp/plugin-threshold/node_modules/zod": { + "version": "3.25.76", + "resolved": "https://registry.npmjs.org/zod/-/zod-3.25.76.tgz", + "integrity": "sha512-gzUt/qt81nXsFGKIFcC3YnfEAx5NkunCfnDlvuBSSFS02bcXu4Lmea0AFIUwbLWxWPx3d9p8S5QoaujKcNQxcQ==", "dev": true, "license": "MIT", - "optional": true, - "os": [ - "win32" - ], - "engines": { - "node": ">=18" + "funding": { + "url": "https://github.com/sponsors/colinhacks" } }, - "node_modules/@esbuild/win32-ia32": { - "version": "0.27.7", - "resolved": "https://registry.npmjs.org/@esbuild/win32-ia32/-/win32-ia32-0.27.7.tgz", - "integrity": "sha512-SmwKXe6VHIyZYbBLJrhOoCJRB/Z1tckzmgTLfFYOfpMAx63BJEaL9ExI8x7v0oAO3Zh6D/Oi1gVxEYr5oUCFhw==", - "cpu": [ - "ia32" - ], + "node_modules/@jimp/types": { + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/@jimp/types/-/types-1.6.1.tgz", + "integrity": "sha512-leI7YbveTNi565m910XgIOwXyuu074H5qazAD1357HImJSv2hqxnWXpwxQbadGWZ7goZRYBDZy5lpqud0p7q5w==", "dev": true, "license": "MIT", - "optional": true, - "os": [ - "win32" - ], + "dependencies": { + "zod": "^3.23.8" + }, "engines": { "node": ">=18" } }, - "node_modules/@esbuild/win32-x64": { - "version": "0.27.7", - "resolved": "https://registry.npmjs.org/@esbuild/win32-x64/-/win32-x64-0.27.7.tgz", - "integrity": "sha512-56hiAJPhwQ1R4i+21FVF7V8kSD5zZTdHcVuRFMW0hn753vVfQN8xlx4uOPT4xoGH0Z/oVATuR82AiqSTDIpaHg==", - "cpu": [ - "x64" - ], + "node_modules/@jimp/types/node_modules/zod": { + "version": "3.25.76", + "resolved": "https://registry.npmjs.org/zod/-/zod-3.25.76.tgz", + "integrity": "sha512-gzUt/qt81nXsFGKIFcC3YnfEAx5NkunCfnDlvuBSSFS02bcXu4Lmea0AFIUwbLWxWPx3d9p8S5QoaujKcNQxcQ==", "dev": true, "license": "MIT", - "optional": true, - "os": [ - "win32" - ], - "engines": { - "node": ">=18" + "funding": { + "url": "https://github.com/sponsors/colinhacks" } }, - "node_modules/@isaacs/fs-minipass": { - "version": "4.0.1", - "resolved": "https://registry.npmjs.org/@isaacs/fs-minipass/-/fs-minipass-4.0.1.tgz", - "integrity": "sha512-wgm9Ehl2jpeqP3zw/7mo3kRHFp5MEDhqAdwy1fTGkHAwnkGOVsgpvQhL8B5n1qlb01jV3n/bI0ZfZp5lWA1k4w==", + "node_modules/@jimp/utils": { + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/@jimp/utils/-/utils-1.6.1.tgz", + "integrity": "sha512-veFPRd93FCnS7AgmCkPgARVGoDRrJ9cm1ujuNyA+UfQ5VKbED2002sm5XfFLFwTsKC8j04heTrwe+tU1dluXOw==", "dev": true, - "license": "ISC", + "license": "MIT", "dependencies": { - "minipass": "^7.0.4" + "@jimp/types": "1.6.1", + "tinycolor2": "^1.6.0" }, "engines": { - "node": ">=18.0.0" + "node": ">=18" } }, "node_modules/@jridgewell/gen-mapping": { @@ -2994,6 +3606,31 @@ "url": "https://github.com/sindresorhus/is?sponsor=1" } }, + "node_modules/@tokenizer/inflate": { + "version": "0.4.1", + "resolved": "https://registry.npmjs.org/@tokenizer/inflate/-/inflate-0.4.1.tgz", + "integrity": "sha512-2mAv+8pkG6GIZiF1kNg1jAjh27IDxEPKwdGul3snfztFerfPGI1LjDezZp3i7BElXompqEtPmoPx6c2wgtWsOA==", + "dev": true, + "license": "MIT", + "dependencies": { + "debug": "^4.4.3", + "token-types": "^6.1.1" + }, + "engines": { + "node": ">=18" + }, + "funding": { + "type": "github", + "url": "https://github.com/sponsors/Borewit" + } + }, + "node_modules/@tokenizer/token": { + "version": "0.3.0", + "resolved": "https://registry.npmjs.org/@tokenizer/token/-/token-0.3.0.tgz", + "integrity": "sha512-OvjF+z51L3ov0OyAU0duzsYuvO01PH7x4t6DJx+guahgTnBHkhJdG7soQeTSFLWN3efnHyibZ4Z8l2EuWwJN3A==", + "dev": true, + "license": "MIT" + }, "node_modules/@types/estree": { "version": "1.0.9", "resolved": "https://registry.npmjs.org/@types/estree/-/estree-1.0.9.tgz", @@ -3483,6 +4120,13 @@ "url": "https://github.com/sponsors/epoberezkin" } }, + "node_modules/any-base": { + "version": "1.1.0", + "resolved": "https://registry.npmjs.org/any-base/-/any-base-1.1.0.tgz", + "integrity": "sha512-uMgjozySS8adZZYePpaWs8cxB9/kdzmpX6SgJZ+wbz1K5eYk5QMYDVJaZKhxyIHUdnnJkfR7SVgStgH7LkGUyg==", + "dev": true, + "license": "MIT" + }, "node_modules/any-promise": { "version": "1.3.0", "resolved": "https://registry.npmjs.org/any-promise/-/any-promise-1.3.0.tgz", @@ -3568,6 +4212,16 @@ "node": ">=8.0.0" } }, + "node_modules/await-to-js": { + "version": "3.0.0", + "resolved": "https://registry.npmjs.org/await-to-js/-/await-to-js-3.0.0.tgz", + "integrity": "sha512-zJAaP9zxTcvTHRlejau3ZOY4V7SRpiByf3/dxx2uyKxxor19tpmpV2QRsTKikckwhaPmr2dVpxxMr7jOCYVp5g==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=6.0.0" + } + }, "node_modules/aws-sign2": { "version": "0.7.0", "resolved": "https://registry.npmjs.org/aws-sign2/-/aws-sign2-0.7.0.tgz", @@ -3737,6 +4391,13 @@ "resolved": "pi-subagents", "link": true }, + "node_modules/bmp-ts": { + "version": "1.0.9", + "resolved": "https://registry.npmjs.org/bmp-ts/-/bmp-ts-1.0.9.tgz", + "integrity": "sha512-cTEHk2jLrPyi+12M3dhpEbnnPOsaZuq7C45ylbbQIiWgDFZq4UVYPEY5mlqjvsj/6gJv9qX5sa+ebDzLXT28Vw==", + "dev": true, + "license": "MIT" + }, "node_modules/body-parser": { "version": "1.20.8", "resolved": "https://registry.npmjs.org/body-parser/-/body-parser-1.20.8.tgz", @@ -4521,6 +5182,12 @@ "bare-events": "^2.7.0" } }, + "node_modules/exif-parser": { + "version": "0.1.12", + "resolved": "https://registry.npmjs.org/exif-parser/-/exif-parser-0.1.12.tgz", + "integrity": "sha512-c2bQfLNbMzLPmzQuOr8fy0csy84WmwnER81W88DzTp9CYNPJ6yzOj2EZAh9pywYpqHnshVLHQJ8WzldAyfY+Iw==", + "dev": true + }, "node_modules/express": { "version": "4.22.3", "resolved": "https://registry.npmjs.org/express/-/express-4.22.3.tgz", @@ -4709,6 +5376,25 @@ "node": "^12.20 || >= 14.13" } }, + "node_modules/file-type": { + "version": "21.3.4", + "resolved": "https://registry.npmjs.org/file-type/-/file-type-21.3.4.tgz", + "integrity": "sha512-Ievi/yy8DS3ygGvT47PjSfdFoX+2isQueoYP1cntFW1JLYAuS4GD7NUPGg4zv2iZfV52uDyk5w5Z0TdpRS6Q1g==", + "dev": true, + "license": "MIT", + "dependencies": { + "@tokenizer/inflate": "^0.4.1", + "strtok3": "^10.3.4", + "token-types": "^6.1.1", + "uint8array-extras": "^1.4.0" + }, + "engines": { + "node": ">=20" + }, + "funding": { + "url": "https://github.com/sindresorhus/file-type?sponsor=1" + } + }, "node_modules/fill-range": { "version": "7.1.1", "resolved": "https://registry.npmjs.org/fill-range/-/fill-range-7.1.1.tgz", @@ -4955,6 +5641,17 @@ "assert-plus": "^1.0.0" } }, + "node_modules/gifwrap": { + "version": "0.10.1", + "resolved": "https://registry.npmjs.org/gifwrap/-/gifwrap-0.10.1.tgz", + "integrity": "sha512-2760b1vpJHNmLzZ/ubTtNnEx5WApN/PYWJvXvgS+tL1egTTthayFYIQQNi136FLEDcN/IyEY2EcGpIITD6eYUw==", + "dev": true, + "license": "MIT", + "dependencies": { + "image-q": "^4.0.0", + "omggif": "^1.0.10" + } + }, "node_modules/glob-parent": { "version": "5.1.2", "resolved": "https://registry.npmjs.org/glob-parent/-/glob-parent-5.1.2.tgz", @@ -5250,6 +5947,23 @@ "node": ">= 4" } }, + "node_modules/image-q": { + "version": "4.0.0", + "resolved": "https://registry.npmjs.org/image-q/-/image-q-4.0.0.tgz", + "integrity": "sha512-PfJGVgIfKQJuq3s0tTDOKtztksibuUEbJQIYT3by6wctQo+Rdlh7ef4evJ5NCdxY4CfMbvFkocEwbl4BF8RlJw==", + "dev": true, + "license": "MIT", + "dependencies": { + "@types/node": "16.9.1" + } + }, + "node_modules/image-q/node_modules/@types/node": { + "version": "16.9.1", + "resolved": "https://registry.npmjs.org/@types/node/-/node-16.9.1.tgz", + "integrity": "sha512-QpLcX9ZSsq3YYUUnD3nFDY8H7wctAhQj/TFKL8Ya8v5fMm3CFXxo8zStsLAl780ltoYoo1WvKUVGBQK+1ifr7g==", + "dev": true, + "license": "MIT" + }, "node_modules/inherits": { "version": "2.0.4", "resolved": "https://registry.npmjs.org/inherits/-/inherits-2.0.4.tgz", @@ -5358,6 +6072,45 @@ "dev": true, "license": "MIT" }, + "node_modules/jimp": { + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/jimp/-/jimp-1.6.1.tgz", + "integrity": "sha512-hNQh6rZtWfSVWSNVmvq87N5BPJsNH7k7I7qyrXf9DOma9xATQk3fsyHazCQe51nCjdkoWdTmh0vD7bjVSLoxxw==", + "dev": true, + "license": "MIT", + "dependencies": { + "@jimp/core": "1.6.1", + "@jimp/diff": "1.6.1", + "@jimp/js-bmp": "1.6.1", + "@jimp/js-gif": "1.6.1", + "@jimp/js-jpeg": "1.6.1", + "@jimp/js-png": "1.6.1", + "@jimp/js-tiff": "1.6.1", + "@jimp/plugin-blit": "1.6.1", + "@jimp/plugin-blur": "1.6.1", + "@jimp/plugin-circle": "1.6.1", + "@jimp/plugin-color": "1.6.1", + "@jimp/plugin-contain": "1.6.1", + "@jimp/plugin-cover": "1.6.1", + "@jimp/plugin-crop": "1.6.1", + "@jimp/plugin-displace": "1.6.1", + "@jimp/plugin-dither": "1.6.1", + "@jimp/plugin-fisheye": "1.6.1", + "@jimp/plugin-flip": "1.6.1", + "@jimp/plugin-hash": "1.6.1", + "@jimp/plugin-mask": "1.6.1", + "@jimp/plugin-print": "1.6.1", + "@jimp/plugin-quantize": "1.6.1", + "@jimp/plugin-resize": "1.6.1", + "@jimp/plugin-rotate": "1.6.1", + "@jimp/plugin-threshold": "1.6.1", + "@jimp/types": "1.6.1", + "@jimp/utils": "1.6.1" + }, + "engines": { + "node": ">=18" + } + }, "node_modules/joycon": { "version": "3.1.1", "resolved": "https://registry.npmjs.org/joycon/-/joycon-3.1.1.tgz", @@ -5368,6 +6121,13 @@ "node": ">=10" } }, + "node_modules/jpeg-js": { + "version": "0.4.4", + "resolved": "https://registry.npmjs.org/jpeg-js/-/jpeg-js-0.4.4.tgz", + "integrity": "sha512-WZzeDOEtTOBK4Mdsar0IqEU5sMr3vSV2RqkAIzUEV2BHnUfKGyswWFPFwK5EeDo93K3FohSHbLAjj0s1Wzd+dg==", + "dev": true, + "license": "BSD-3-Clause" + }, "node_modules/js-yaml": { "version": "5.2.2", "resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-5.2.2.tgz", @@ -6017,6 +6777,13 @@ "url": "https://github.com/sponsors/ljharb" } }, + "node_modules/omggif": { + "version": "1.0.10", + "resolved": "https://registry.npmjs.org/omggif/-/omggif-1.0.10.tgz", + "integrity": "sha512-LMJTtvgc/nugXj0Vcrrs68Mn2D1r0zf630VNtqtpI1FEO7e+O9FP4gqs9AcnBaSEeoHIPm28u6qgPR0oyEpGSw==", + "dev": true, + "license": "MIT" + }, "node_modules/on-exit-leak-free": { "version": "2.1.2", "resolved": "https://registry.npmjs.org/on-exit-leak-free/-/on-exit-leak-free-2.1.2.tgz", @@ -6067,6 +6834,31 @@ "dev": true, "license": "MIT" }, + "node_modules/parse-bmfont-ascii": { + "version": "1.0.6", + "resolved": "https://registry.npmjs.org/parse-bmfont-ascii/-/parse-bmfont-ascii-1.0.6.tgz", + "integrity": "sha512-U4RrVsUFCleIOBsIGYOMKjn9PavsGOXxbvYGtMOEfnId0SVNsgehXh1DxUdVPLoxd5mvcEtvmKs2Mmf0Mpa1ZA==", + "dev": true, + "license": "MIT" + }, + "node_modules/parse-bmfont-binary": { + "version": "1.0.6", + "resolved": "https://registry.npmjs.org/parse-bmfont-binary/-/parse-bmfont-binary-1.0.6.tgz", + "integrity": "sha512-GxmsRea0wdGdYthjuUeWTMWPqm2+FAd4GI8vCvhgJsFnoGhTrLhXDDupwTo7rXVAgaLIGoVHDZS9p/5XbSqeWA==", + "dev": true, + "license": "MIT" + }, + "node_modules/parse-bmfont-xml": { + "version": "1.1.6", + "resolved": "https://registry.npmjs.org/parse-bmfont-xml/-/parse-bmfont-xml-1.1.6.tgz", + "integrity": "sha512-0cEliVMZEhrFDwMh4SxIyVJpqYoOWDJ9P895tFuS+XuNzI5UBmBk5U5O4KuJdTnZpSBI4LFA2+ZiJaiwfSwlMA==", + "dev": true, + "license": "MIT", + "dependencies": { + "xml-parse-from-string": "^1.0.0", + "xml2js": "^0.5.0" + } + }, "node_modules/parseurl": { "version": "1.3.3", "resolved": "https://registry.npmjs.org/parseurl/-/parseurl-1.3.3.tgz", @@ -6265,6 +7057,29 @@ "node": ">= 6" } }, + "node_modules/pixelmatch": { + "version": "5.3.0", + "resolved": "https://registry.npmjs.org/pixelmatch/-/pixelmatch-5.3.0.tgz", + "integrity": "sha512-o8mkY4E/+LNUf6LzX96ht6k6CEDi65k9G2rjMtBe9Oo+VPKSvl+0GKHuH/AlG+GA5LPG/i5hrekkxUc3s2HU+Q==", + "dev": true, + "license": "ISC", + "dependencies": { + "pngjs": "^6.0.0" + }, + "bin": { + "pixelmatch": "bin/pixelmatch" + } + }, + "node_modules/pixelmatch/node_modules/pngjs": { + "version": "6.0.0", + "resolved": "https://registry.npmjs.org/pngjs/-/pngjs-6.0.0.tgz", + "integrity": "sha512-TRzzuFRRmEoSW/p1KVAmiOgPco2Irlah+bGFCeNfJXxxYGwSw7YwAOAcd7X28K/m5bjBWKsC29KyoMfHbypayg==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=12.13.0" + } + }, "node_modules/pkg-types": { "version": "1.3.1", "resolved": "https://registry.npmjs.org/pkg-types/-/pkg-types-1.3.1.tgz", @@ -6277,6 +7092,16 @@ "pathe": "^2.0.1" } }, + "node_modules/pngjs": { + "version": "7.0.0", + "resolved": "https://registry.npmjs.org/pngjs/-/pngjs-7.0.0.tgz", + "integrity": "sha512-LKWqWJRhstyYo9pGvgor/ivk2w94eSjE3RGVuzLGlr3NmD8bf7RcYGze1mNdEHRP6TRP6rMuDHk5t44hnTRyow==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=14.19.0" + } + }, "node_modules/postcss-load-config": { "version": "6.0.1", "resolved": "https://registry.npmjs.org/postcss-load-config/-/postcss-load-config-6.0.1.tgz", @@ -6717,6 +7542,16 @@ "truncate-utf8-bytes": "^1.0.0" } }, + "node_modules/sax": { + "version": "1.6.1", + "resolved": "https://registry.npmjs.org/sax/-/sax-1.6.1.tgz", + "integrity": "sha512-42tBVwLWnaQvW5zc4HbZrTuWccECCZfBi92FDuwtqxasH+JbPB3/FOKb1m222K42R4WxuxzzMsTswfzgtSu64Q==", + "dev": true, + "license": "BlueOak-1.0.0", + "engines": { + "node": ">=11.0.0" + } + }, "node_modules/semver": { "version": "7.8.5", "resolved": "https://registry.npmjs.org/semver/-/semver-7.8.5.tgz", @@ -6891,6 +7726,16 @@ "dev": true, "license": "ISC" }, + "node_modules/simple-xml-to-json": { + "version": "1.2.7", + "resolved": "https://registry.npmjs.org/simple-xml-to-json/-/simple-xml-to-json-1.2.7.tgz", + "integrity": "sha512-mz9VXphOxQWX3eQ/uXCtm6upltoN0DLx8Zb5T4TFC4FHB7S9FDPGre8CfLWqPWQQH/GrQYd2AXhhVM5LDpYx6Q==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=20.12.2" + } + }, "node_modules/slash": { "version": "3.0.0", "resolved": "https://registry.npmjs.org/slash/-/slash-3.0.0.tgz", @@ -7013,6 +7858,23 @@ "dev": true, "license": "MIT" }, + "node_modules/strtok3": { + "version": "10.5.0", + "resolved": "https://registry.npmjs.org/strtok3/-/strtok3-10.5.0.tgz", + "integrity": "sha512-PgAsgIuizbZN/r4+g4UNvbVCTpJWdoIdI71kIWwKmdIH0vJ0q7ylMVGU8PWj2s9O98y4MtFWiDTIrtNukZMMgQ==", + "dev": true, + "license": "MIT", + "dependencies": { + "@tokenizer/token": "^0.3.0" + }, + "engines": { + "node": ">=18" + }, + "funding": { + "type": "github", + "url": "https://github.com/sponsors/Borewit" + } + }, "node_modules/sucrase": { "version": "3.35.1", "resolved": "https://registry.npmjs.org/sucrase/-/sucrase-3.35.1.tgz", @@ -7150,6 +8012,13 @@ "xtend": "~4.0.1" } }, + "node_modules/tinycolor2": { + "version": "1.6.0", + "resolved": "https://registry.npmjs.org/tinycolor2/-/tinycolor2-1.6.0.tgz", + "integrity": "sha512-XPaBkWQJdsf3pLKJV9p4qN/S+fm2Oj8AIPo1BTUhg5oxkvm9+SVEGFdhyOz7tTdUTfvxMiAs4sp6/eZO2Ew+pw==", + "dev": true, + "license": "MIT" + }, "node_modules/tinyexec": { "version": "0.3.2", "resolved": "https://registry.npmjs.org/tinyexec/-/tinyexec-0.3.2.tgz", @@ -7217,6 +8086,25 @@ "node": ">=0.6" } }, + "node_modules/token-types": { + "version": "6.1.2", + "resolved": "https://registry.npmjs.org/token-types/-/token-types-6.1.2.tgz", + "integrity": "sha512-dRXchy+C0IgK8WPC6xvCHFRIWYUbqqdEIKPaKo/AcTUNzwLTK6AH7RjdLWsEZcAN/TBdtfUw3PYEgPr5VPr6ww==", + "dev": true, + "license": "MIT", + "dependencies": { + "@borewit/text-codec": "^0.2.1", + "@tokenizer/token": "^0.3.0", + "ieee754": "^1.2.1" + }, + "engines": { + "node": ">=14.16" + }, + "funding": { + "type": "github", + "url": "https://github.com/sponsors/Borewit" + } + }, "node_modules/tough-cookie": { "version": "5.1.2", "resolved": "https://registry.npmjs.org/tough-cookie/-/tough-cookie-5.1.2.tgz", @@ -7969,6 +8857,23 @@ "dev": true, "license": "(WTFPL OR MIT)" }, + "node_modules/utif2": { + "version": "4.1.0", + "resolved": "https://registry.npmjs.org/utif2/-/utif2-4.1.0.tgz", + "integrity": "sha512-+oknB9FHrJ7oW7A2WZYajOcv4FcDR4CfoGB0dPNfxbi4GO05RRnFmt5oa23+9w32EanrYcSJWspUiJkLMs+37w==", + "dev": true, + "license": "MIT", + "dependencies": { + "pako": "^1.0.11" + } + }, + "node_modules/utif2/node_modules/pako": { + "version": "1.0.11", + "resolved": "https://registry.npmjs.org/pako/-/pako-1.0.11.tgz", + "integrity": "sha512-4hLB8Py4zZce5s4yd9XzopqwVv/yGNhV1Bl8NTmCq1763HeK2+EwVTv+leGeL13Dnh2wfbqowVPXCIO0z4taYw==", + "dev": true, + "license": "(MIT AND Zlib)" + }, "node_modules/util-deprecate": { "version": "1.0.2", "resolved": "https://registry.npmjs.org/util-deprecate/-/util-deprecate-1.0.2.tgz", @@ -8167,6 +9072,37 @@ } } }, + "node_modules/xml-parse-from-string": { + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/xml-parse-from-string/-/xml-parse-from-string-1.0.1.tgz", + "integrity": "sha512-ErcKwJTF54uRzzNMXq2X5sMIy88zJvfN2DmdoQvy7PAFJ+tPRU6ydWuOKNMyfmOjdyBQTFREi60s0Y0SyI0G0g==", + "dev": true, + "license": "MIT" + }, + "node_modules/xml2js": { + "version": "0.5.0", + "resolved": "https://registry.npmjs.org/xml2js/-/xml2js-0.5.0.tgz", + "integrity": "sha512-drPFnkQJik/O+uPKpqSgr22mpuFHqKdbS835iAQrUC73L2F5WkboIRd63ai/2Yg6I1jzifPFKH2NTK+cfglkIA==", + "dev": true, + "license": "MIT", + "dependencies": { + "sax": ">=0.6.0", + "xmlbuilder": "~11.0.0" + }, + "engines": { + "node": ">=4.0.0" + } + }, + "node_modules/xmlbuilder": { + "version": "11.0.1", + "resolved": "https://registry.npmjs.org/xmlbuilder/-/xmlbuilder-11.0.1.tgz", + "integrity": "sha512-fDlsI/kFEx7gLvbecc0/ohLG50fugQp8ryHzMTuW9vSa1GJ0XYWKnhsUx7oie3G98+r56aTQIUB4kht42R3JvA==", + "dev": true, + "license": "MIT", + "engines": { + "node": ">=4.0" + } + }, "node_modules/xtend": { "version": "4.0.2", "resolved": "https://registry.npmjs.org/xtend/-/xtend-4.0.2.tgz", diff --git a/package.json b/package.json index 6d22b2f380..db5fd6170a 100644 --- a/package.json +++ b/package.json @@ -72,6 +72,7 @@ "billion-context-pi-subagents": "file:./pi-subagents", "esbuild": "^0.27.0", "fzstd": "0.1.1", + "jimp": "1.6.1", "jsonc-parser": "^3.3.1", "node-forge": "^1.4.0", "prettier": "^3.3.0", diff --git a/reference/architecture.md b/reference/architecture.md index d7356800f6..f929892475 100644 --- a/reference/architecture.md +++ b/reference/architecture.md @@ -66,7 +66,7 @@ billion-context/ │ ├── image-tokens.ts # Image token cost estimation (pixels/bytes modes, #767) │ ├── image-compress.ts # Image-compression policy stamping per request │ ├── image-note.ts # Image-block placeholder → rich note rendering -│ ├── proxy-tool.ts # isProxyToolFor — client-named tools routed to the bili proxy +│ ├── absorb.ts # absorb tool (kernel ACP_TOOL_NAMES extension) │ ├── compress-loop-responses.ts # Compress loop for Responses API format │ ├── compress-settings.ts # Three-level compress config merge (global→provider→model) │ ├── compress-tool.ts # ACP tool surface — thin re-export from acp-kernel diff --git a/scripts/gate/README.md b/scripts/gate/README.md index 424096b55b..93cf32d24e 100644 --- a/scripts/gate/README.md +++ b/scripts/gate/README.md @@ -33,10 +33,68 @@ actually received — internal logs can be fooled by suppression-class bugs S40 last CODEWORDS markers) - no `unknown tool` loops +## dsh protocol matrix (#3475) + +dsh's model client (pi-ai) speaks all four common protocols, so the dsh lane +expands from one door to four — same headless driver, only the cordis provider +`api` and the baseURL suffix change; the t1→t4 flow and assertions are +unchanged: + +| lane | pi-ai api | bili front door | upstream door required | +|------|-----------|-----------------|------------------------| +| `dsh` (original) | openai-completions | `/v1/chat/completions` | chat completions | +| `dsh-responses` | openai-responses | `/v1/responses` | a real `/v1/responses` (404 / mapping 400 → env-blocked) | +| `dsh-anthropic` | anthropic-messages | `/v1/messages` | `/v1/messages` (cross-checks the claude lane) | +| `dsh-google` | google-generative-ai | `/v1beta/models/:streamGenerateContent` | a `/v1beta` google door | +| `dsh-proxy` | openai-completions (**proxy mode**) | `/bili/…/v1/chat/completions` | chat completions (via the `/bili/` prefix) | + +`dsh-proxy` is the **other activation path** of the same host (owner add-on): +the profile does **NOT** load the billion-context bundle (its fetch intercept +would stamp `x-bili-plugin` and flip the lane back into plugin mode); dsh +drives the manually started proxy's `/bili/` URL directly — the same shape as +codex/claude. Contrasted with the native `dsh` lane within ONE script: +#3177/#3401-style suppression only exists on the plugin path; in proxy mode +nudge/fold must ride the wire with the acp_summary carrier; window truth = +bili.json (like every prefix lane). + +Every door gains a **wire-shape hard assertion**: each recorder POST through +the door must match the protocol's path AND top-level body fields +(chat=`messages` / responses=`input` / anthropic=`system`+blocks / +google=`contents`) — a silent rewire must not read green; the first egress +request is dumped to `//egress-sample.json` as round-trip +evidence. + +Two known facts: + +- **bili does not translate google→chat on egress** — the recorder sees exactly + the google shape dsh sends (`:streamGenerateContent` + contents/parts). The + google lane can only go fully green against an upstream that has a real + google door; requiring "it must come out as chat" would be a new + egress-translation feature for bili — separate issue. +- **blocked demotion rules** (two kinds; the script prints an + `ENV-BLOCKED`/`CLIENT-BLOCKED [lane] ` line at the end): + - **env**: the upstream door is unhealthy — ALL requests on the door are + 4xx/5xx/connection errors with no turn completed (the original + opencode2/#3461 rule), OR degraded mid-flow (some 200s at first, then + rejections, the flow never completes — the classic self-hosted + responses→chat mapping-bug shape, #3461 sibling). Traffic + wire shape + stay hard assertions; turn-dependent records drop out of the gate tally. + - **client**: the client version cannot express the api at all (dsh + ≤0.2.1-rc.1 cordis schema lists only openai-completions / openai-responses / + anthropic-messages; google-generative-ai exists in the pi-ai library but + dsh's config surface does not expose it) — zero upstream traffic + every + turn dies on a ValidationError. The whole lane drops out of the gate; it + auto-promotes when the client ships the protocol — no script change needed. + +Not in the default GATE_AGENTS set; runs only when explicitly listed: +`GATE_AGENTS=dsh,dsh-responses,dsh-anthropic,dsh-google,dsh-proxy`. + ## Lane flow -1. **t1 @ big window**: read the corpus, echo the S40 codeword (fold must NOT - fire in this phase) +1. **t1 @ big window**: read the corpus, echo the S40 + S07 codewords (dual + marker proves the corpus middle was really read — models may chunk-read or + elide the middle, and an elided middle turns the later S07 recall assertions + into false reds; fold must NOT fire in this phase) 2. **window shrink**: rewrite the **host's own** model config (dsh cordis / pi models.json / opencode.json limit / codex config.toml / claude model alias) — the effective window reaches bili via the @@ -66,8 +124,17 @@ node scripts/gate/plugin-lanes.mjs - `GATE_AGENTS` selects a subset of lanes; `GATE_TURN_TIMEOUT_MS` (default 300000), `GATE_BIG/MID/SMALL_WINDOW`, and `PLUGIN_LANES_WORK` (artifact dir, default `./plugin-lanes/`) are tunable +- dsh lanes: `GATE_DSH_VERSION` (each lane self-installs + `@deepseek-ai/dsh@`, default latest — system dsh may be too old and a + global billion-context shadows the candidate bundle), `GATE_OUTPUT_TOKENS` + (output budget: bili.json reservation + dsh cordis maxTokens, default 4096; + raise it when a thinking model burns the whole budget on reasoning with no + visible text), `GATE_DSH_SMALL_WINDOW` (fixed override of the t3 small + window; by default derived from the live t2-end payload — dsh's fixed + scaffolding is ~18k tok and a static 24576 minus the output reservation + cannot fit even an empty session) - per-lane artifacts: `//turn-t{1..4}.txt`, `upstream.jsonl`, - `bili.log` + `bili.log`, plus `//egress-sample.json` for the dsh family - the opencode2 lane is not in the default set: transport websocket passes Responses-WS straight through (HTTP-only upstreams die on the handshake), and transport http depends on upstream `/v1/responses` quality — see diff --git a/scripts/gate/README.zh-CN.md b/scripts/gate/README.zh-CN.md index acd7c4e9de..ac15a9de5d 100644 --- a/scripts/gate/README.zh-CN.md +++ b/scripts/gate/README.zh-CN.md @@ -26,9 +26,56 @@ wire 骗不了: - 折叠前后暗号回忆(corpus 40 段,S07 早段/S40 末段 CODEWORDS 暗号) - 无 `unknown tool` 空转 +## dsh 协议矩阵(#3475) + +dsh 的模型客户端 pi-ai 支持全部四种通用协议,dsh 车道因此从一门扩成四门—— +同一套 headless 驱动,只换 cordis provider 的 `api` 与 baseURL 后缀,t1→t4 +流程和断言不变: + +| 车道 | pi-ai api | bili 前门 | 上游门要求 | +|------|-----------|----------|------------| +| `dsh`(原有) | openai-completions | `/v1/chat/completions` | chat completions | +| `dsh-responses` | openai-responses | `/v1/responses` | 真实 `/v1/responses`(404/映射 400 → env-blocked) | +| `dsh-anthropic` | anthropic-messages | `/v1/messages` | `/v1/messages`(claude 车道的交叉验证) | +| `dsh-google` | google-generative-ai | `/v1beta/models/:streamGenerateContent` | `/v1beta` google 门 | +| `dsh-proxy` | openai-completions(**代理模式**) | `/bili/…/v1/chat/completions` | chat completions(经 `/bili/` 前缀) | + +`dsh-proxy` 是同一宿主的**另一条激活路径**(owner 加项):profile **不装** +billion-context bundle(装了它的 fetch 拦截会盖 `x-bili-plugin` 头把车道翻回 +插件模式),dsh 直连手动起的代理 `/bili/` URL——codex/claude 同款形状。与 +native `dsh` 车道同脚本对照:#3177/#3401 类抑制只存在于插件路径,代理模式下 +nudge/fold 必须以 acp_summary 载体正常上 wire;窗口真值 = bili.json(与所有 +prefix 车道一致)。 + +每门新增 **wire 形状硬断言**:recorder 里该门的每个 POST 必须路径+顶层体字段 +都是该协议形状(chat=`messages` / responses=`input` / anthropic=`system`+blocks / +google=`contents`),防止静默换线假绿;首个出站请求存 +`//egress-sample.json` 作往返证据。 + +两个已知事实: + +- **bili 出站不做 google→chat 翻译**——recorder 看到的正是 dsh 发出的 google + 形状(`:streamGenerateContent` + contents/parts)。google lane 只能在有真 + google 门的上游上满绿;要「必须 chat 出」则是给 bili 加出站翻译功能,单独立项。 +- **blocked 降级规则**(两种 kind,脚本结尾打印 `ENV-BLOCKED`/`CLIENT-BLOCKED + [lane] <原因>` 行): + - **env**:上游门不健康——门上请求**全部** 4xx/5xx/连接错误且无回合完成 + (opencode2/#3461 原规矩),或**中途劣化**(前若干请求 200、后续被拒、流程 + 未走完——自建服务 responses→chat 映射 bug 的典型形态,#3461 同族)。 + traffic + wire 形状仍是硬断言,turn 相关断言不计入 gate 成败。 + - **client**:客户端当前版本根本表达不了该 api(dsh ≤0.2.1-rc.1 的 cordis + schema 只列 openai-completions/openai-responses/anthropic-messages 三门, + google-generative-ai 在 pi-ai 库里有实现但 dsh 配置面没暴露)——零上游流量 + + 每回合死于 ValidationError。整条车道不计入 gate,客户端放出该协议后 + 自动转正,脚本不用改。 + +不进默认 GATE_AGENTS 集,显式加入才跑:`GATE_AGENTS=dsh,dsh-responses,dsh-anthropic,dsh-google,dsh-proxy`。 + ## 车道流程 -1. **t1 @ 大窗**:读 corpus,回显 S40 暗号(此阶段 fold 必须不触发) +1. **t1 @ 大窗**:读 corpus,回显 S40 + S07 双暗号(双标记验证「真的读全了」—— + 模型可能照指令分块读、也可能漏读中段,漏读会让后续 S07 回忆断言变成假红; + 此阶段 fold 必须不触发) 2. **窗口收缩**:改**宿主自己的**模型配置(dsh cordis / pi models.json / opencode.json limit / codex config.toml / claude 模型别名)——有效窗口经 `x-bili-plugin-context-window` 头进 bili,native 代理的 zone 端口是粘滞的, @@ -55,7 +102,14 @@ node scripts/gate/plugin-lanes.mjs - `GATE_AGENTS` 选车道子集;`GATE_TURN_TIMEOUT_MS`(默认 300000)、 `GATE_BIG/MID/SMALL_WINDOW`、`PLUGIN_LANES_WORK`(工件目录,默认 `./plugin-lanes/`)可调 +- dsh 车道:`GATE_DSH_VERSION`(每车道自装 `@deepseek-ai/dsh@`,默认 + latest——系统 dsh 可能太旧且全局 billion-context 会遮蔽候选 bundle)、 + `GATE_OUTPUT_TOKENS`(输出预算:bili.json 预留 + dsh cordis maxTokens, + 默认 4096;思考型模型吃光预算只出 reasoning 不出正文时调大)、 + `GATE_DSH_SMALL_WINDOW`(固定覆盖 t3 小窗;默认从 t2 末尾实际 payload + 推导——dsh 固定脚手架 ~18k tok,静态 24576 扣掉输出预留后连空会话都装不下) - 每车道工件:`//turn-t{1..4}.txt`、`upstream.jsonl`、`bili.log` + (dsh 家族另有 `egress-sample.json`) - opencode2 车道不进默认集:transport websocket 直通 Responses-WS(HTTP-only 上游握手必死)、transport http 依赖上游 /v1/responses 质量——见 #3461; 车道仍断言 tools/claim/流量 diff --git a/scripts/gate/plugin-lanes.mjs b/scripts/gate/plugin-lanes.mjs index dd564bb088..b39ffac7fc 100644 --- a/scripts/gate/plugin-lanes.mjs +++ b/scripts/gate/plugin-lanes.mjs @@ -29,9 +29,26 @@ // t3 @ small window : "DONE" — preflight fold path must fire // t4 : recall S07 again — must survive the fold // +// dsh protocol matrix (#3475): the dsh recipe is parameterized over the four +// provider api types pi-ai supports — openai-completions (the original lane), +// openai-responses, anthropic-messages, google-generative-ai. Same headless +// driver, only the cordis provider `api` + baseURL suffix change. Each variant +// carries a WIRE_EXPECT entry: the recorder must see POSTs whose path AND body +// top-level shape match that wire (hard wire assertion, not log). When the +// upstream lacks the door (404/405/mapping-400 with no completed turn) the +// lane degrades to env-blocked (traffic + wire shape + plugin claim stay +// hard) instead of failing the gate — same rule as opencode2/#3461. +// // Env: TGZ, REPO, MODEL1_BASE_URL, MODEL1_MODEL, MODEL1_API_KEY, -// GATE_AGENTS (default "dsh,pi,codex,opencode,claude"), GATE_SKIP=p, -// GATE_TURN_TIMEOUT_MS (default 300000), PLUGIN_LANES_WORK +// GATE_AGENTS (default "dsh,pi,codex,opencode,claude"; matrix + proxy +// lanes opt in: dsh-responses,dsh-anthropic,dsh-google,dsh-proxy), +// GATE_SKIP=p, GATE_TURN_TIMEOUT_MS (default 300000), PLUGIN_LANES_WORK, +// GATE_DSH_VERSION (dsh lanes install @deepseek-ai/dsh@ per lane, +// default "latest"), GATE_OUTPUT_TOKENS (output budget: bili.json +// reservation + dsh cordis maxTokens, default 4096 — thinking models +// need more or they degenerate to zero-visible-text turns), +// GATE_DSH_SMALL_WINDOW (fixed override for the dsh lanes' phase-3 +// window; by default it derives from the live t2-end payload) import { spawn, spawnSync } from "node:child_process"; import fs from "node:fs"; @@ -58,11 +75,24 @@ const AGENTS = (process.env.GATE_AGENTS || "dsh,pi,codex,opencode,claude") .split(",").map((s) => s.trim().toLowerCase()).filter(Boolean); if (AGENTS.includes("opencode2")) console.log("[opencode2] NOTE env-blocked on some upstreams: transport websocket passes Responses-WS straight through (HTTP-only upstreams fail the WS handshake); transport http needs a real /v1/responses endpoint (some self-hosted servers have a Responses->chat mapping bug or 404). Lane asserts tools/claim/traffic; turn completion may fail against such upstreams — see #3461."); +if (AGENTS.includes("dsh-responses")) + console.log("[dsh-responses] NOTE env-blocked when the upstream has no healthy /v1/responses door (self-hosted servers: 404, or a broken Responses->chat mapping answering 400 — same failure class as opencode2/#3461). bili forwards the responses wire as responses; the door must be real. Wire shape + traffic stay hard-asserted; turn/nudge/fold degrade to env-blocked."); +if (AGENTS.includes("dsh-anthropic")) + console.log("[dsh-anthropic] NOTE env-blocked when the upstream has no /v1/messages door. Cross-validates the claude lane's front door from the dsh side (pi-ai Anthropic SDK)."); +if (AGENTS.includes("dsh-google")) + console.log("[dsh-google] NOTE bili does NOT translate google->chat on egress — the recorder sees the google wire (:streamGenerateContent + contents/parts) exactly as dsh sent it. The lane is env-blocked unless the upstream exposes a /v1beta/models/:generateContent door; the observed egress shape is recorded as the round-trip evidence (#3475 acceptance 2)."); +if (AGENTS.includes("dsh-proxy")) + console.log("[dsh-proxy] NOTE proxy-mode contrast lane (#3475): same dsh host WITHOUT the billion-context bundle, driving the /bili/ prefix door (proxy mode, acp_summary carrier) — contrasts the native plugin lane in one script."); const TMO = Number(process.env.GATE_TURN_TIMEOUT_MS || 300_000); +const DSH_VERSION = process.env.GATE_DSH_VERSION || "latest"; const BIG_WINDOW = Number(process.env.GATE_BIG_WINDOW || 262144); const MID_WINDOW = Number(process.env.GATE_MID_WINDOW || 58368); const SMALL_WINDOW = Number(process.env.GATE_SMALL_WINDOW || 24576); const CORPUS_TOKENS = Number(process.env.GATE_CORPUS_TOKENS || 25000); +// #3475: output budget (bili.json `output` reservation + dsh cordis maxTokens). +// Thinking models can spend the whole budget on reasoning and end the turn with +// zero visible text — raise it per environment when the gate model thinks hard. +const OUTPUT_TOKENS = Number(process.env.GATE_OUTPUT_TOKENS || 4096); if (!TGZ || !REPO || !MODEL1.base || !MODEL1.model) { console.error("plugin-lanes: TGZ, REPO, MODEL1_BASE_URL, MODEL1_MODEL are required"); @@ -70,6 +100,9 @@ if (!TGZ || !REPO || !MODEL1.base || !MODEL1.model) { } const results = []; +// #3475: lanes whose upstream lacks the protocol door — degraded verdict, +// excluded from the failure tally (see runLane's env-blocked rule). +const BLOCKED = new Map(); // lane -> { kind: "env" | "client", reason } function record(lane, name, ok, note) { results.push({ lane, name, ok, note }); console.log(`${ok ? "PASS" : "FAIL"} [${lane}] ${name}${note ? ` — ${note}` : ""}`); @@ -143,6 +176,36 @@ const NUDGE_WIRE_RE = /efficiency nudge to compress early|Context limit reached // must not pass — hence full codeword + TITLE anchor. const MARK_RE = (i) => new RegExp(`TITLE:\\s*GATE-MARK-${S(i)}-${CODEWORDS[i]}\\b`); +// #3475 protocol matrix: what each pi-ai api type MUST look like at the +// bili→upstream hop (the recorder's view). pathRe selects the door requests +// (POST model calls only — GET /models discovery is ignored); bodyCheck pins +// the wire's top-level shape so a silent wire swap (e.g. google arriving as +// chat) fails loudly instead of reading green. +const WIRE_EXPECT = { + "openai-completions": { + door: "/v1/chat/completions", + // SDKs append query params (?beta=true on Anthropic's messages door, + // ?alt=sse on google) — match the path prefix, not a bare $ anchor. + pathRe: /\/chat\/completions(\?[^#]*)?$/, + bodyCheck: (b) => Array.isArray(b.messages), + }, + "openai-responses": { + door: "/v1/responses", + pathRe: /\/responses(\?[^#]*)?$/, + bodyCheck: (b) => b.input !== undefined, + }, + "anthropic-messages": { + door: "/v1/messages", + pathRe: /\/messages(\?[^#]*)?$/, + bodyCheck: (b) => Array.isArray(b.messages) && b.model !== undefined, + }, + "google-generative-ai": { + door: "/v1beta/models/:streamGenerateContent", + pathRe: /\/models\/[^/:?]+:(?:streamGenerateContent|generateContent)/, + bodyCheck: (b) => Array.isArray(b.contents), + }, +}; + function hashStr(s) { let h = 0; for (let i = 0; i < s.length; i++) h = (h * 31 + s.charCodeAt(i)) | 0; @@ -203,7 +266,7 @@ async function runLane(agent, recipe) { providers: { [recUrl]: { models: Object.fromEntries( - [MODEL1.model, ...extraModels].map((m) => [m, { context: w, output: 4096 }]), + [MODEL1.model, ...extraModels].map((m) => [m, { context: w, output: OUTPUT_TOKENS }]), ), }, }, @@ -268,10 +331,15 @@ async function runLane(agent, recipe) { try { await recipe.setup(ctx); const sid = { v: null }; - const t1 = await recipe.turn(ctx, `Read corpus.md in your cwd fully (~1300 lines). Your read tool may truncate or ELIDE the middle of a long file (e.g. claude-code's Read shows head+tail with "... N lines elided ...") — read in sequential chunks (offset 1, 201, 401, …, ~200 lines each, or equivalent line-range calls) until you have seen EVERY line including the middle; do not stop at a truncation or elision. Then reply with ONLY the TITLE line of section S40. Do not run any other tool or command.`, sid, "t1"); + // Dual marker: the t2/t4 S07-recall assertions presume t1 actually READ + // the corpus middle — models sometimes elide it despite the chunk-read + // instruction (observed in proxy mode). Verify the precondition at t1 + // instead of letting downstream recalls fail for fixture reasons. + const t1 = await recipe.turn(ctx, `Read corpus.md in your cwd fully (~1300 lines). Your read tool may truncate or ELIDE the middle of a long file (e.g. claude-code's Read shows head+tail with "... N lines elided ...") — read in sequential chunks (offset 1, 201, 401, …, ~200 lines each, or equivalent line-range calls) until you have seen EVERY line including the middle; do not stop at a truncation or elision. Then reply with ONLY the TITLE lines of sections S40 and S07 (one per line). Do not run any other tool or command.`, sid, "t1"); fs.writeFileSync(path.join(ctx.laneDir, `turn-t1.txt`), t1.out || ""); record(agent, "t1 read @ big window completes", t1.ok, t1.why.slice(0, 160)); record(agent, "S40 marker answered (t1)", MARK_RE(39).test(t1.out), `"${t1.out.trim().slice(0, 80)}"`); + record(agent, "S07 marker answered (t1)", MARK_RE(6).test(t1.out), `"${t1.out.trim().slice(0, 80)}"`); // ---- phase 2: MID window — high-usage band, nudge must ride the wire -- // At ~77% usage with ≥5k pending, the over-limit nudge tier injects and @@ -283,10 +351,13 @@ async function runLane(agent, recipe) { // include. Attempt 1 seeds mid from p1×1.6 (surface guess); after the t2 // request the log carries the REAL payload P ("usage=X% (P/eff)"), so // attempt 2 recomputes mid = P/0.77 + 4096 exactly. Bounded at 2 tries. + // p1 = largest session payload seen during the BIG phase (= t1-end state). + // Do NOT slice at the first [window-change]: source flips (configured→ + // runtime-info, #2283) land mid-t1 and truncate the phase before the + // payload-bearing idle lines, zeroing p1 and mis-seeding mid. const log1 = ctx.readLog(); - const bigPhase = log1.slice(0, log1.indexOf("[window-change]")); - const usageMatches = [...bigPhase.matchAll(/nudge idle: usage=\d+% \((\d+)\//g)]; - const p1idle = usageMatches.length ? Number(usageMatches[usageMatches.length - 1][1]) : 0; + const usageMatches = [...log1.matchAll(/nudge idle: usage=\d+% \((\d+)\//g)]; + const p1idle = usageMatches.length ? Math.max(...usageMatches.map((m) => Number(m[1]))) : 0; // wire seed: the biggest t1 request body ≈ the real live payload // (scaffolding included). Idle log lines lag the final corpus read — // a container run seeded mid from a stale idle line (9.3k) while the @@ -298,6 +369,13 @@ async function runLane(agent, recipe) { let mid = process.env.GATE_MID_WINDOW ? MID_WINDOW : (p1 > 0 ? Math.ceil((p1 * 1.6 / 0.77 + 4096) / 1024) * 1024 : MID_WINDOW); + // A recipe whose small window can grow past the seeded mid (dsh adaptive, + // #3475) would shrink UP in phase 3 — no fold would ever fire. Keep the + // mid strictly above the small-window estimate. + if (recipe.smallWindow && !process.env.GATE_MID_WINDOW) { + const swEst = recipe.smallWindow(ctx, p1); + if (mid <= swEst) mid = Math.ceil((swEst * 1.2) / 1024) * 1024; + } let t2 = { ok: false, why: "never ran", out: "" }; let attempts = 0; @@ -306,10 +384,7 @@ async function runLane(agent, recipe) { ctx.harvestWireModels(); writeWindow(mid); if (recipe.setWindow) await recipe.setWindow(ctx, mid); - if (recipe.type === "prefix") { - if (ctx.proxy) { try { process.kill(-ctx.proxy.pid, "SIGKILL"); } catch {} ctx.proxy = null; } - await startManualProxy(ctx, ctx.proxyArgs ? ctx.proxyArgs() : { port: ctx.proxyPort }); - } + await restartLaneProxy(ctx, recipe); await new Promise((r) => setTimeout(r, 1000)); const mark = ctx.readLog().length; // dsh: a leading cheap tool call guarantees ≥2 LLM requests — the @@ -351,10 +426,7 @@ async function runLane(agent, recipe) { const smallWin = recipe.smallWindow ? recipe.smallWindow(ctx, lastP) : SMALL_WINDOW; writeWindow(smallWin); if (recipe.setWindow) await recipe.setWindow(ctx, smallWin); - if (recipe.type === "prefix") { - if (ctx.proxy) { try { process.kill(-ctx.proxy.pid, "SIGKILL"); } catch {} ctx.proxy = null; } - await startManualProxy(ctx, ctx.proxyArgs ? ctx.proxyArgs() : { port: ctx.proxyPort }); - } + await restartLaneProxy(ctx, recipe); await new Promise((r) => setTimeout(r, 1000)); if (recipe.turnTimeoutMs) ctx.TMO = recipe.turnTimeoutMs(ctx, "t3"); @@ -365,6 +437,36 @@ async function runLane(agent, recipe) { fs.writeFileSync(path.join(ctx.laneDir, `turn-t3.txt`), t3.out || ""); record(agent, "t3 @ small window completes (preflight fold path)", t3.ok, t3.why.slice(0, 160)); + // phase 3b (#3475): hosts with a large fixed scaffolding (dsh 0.2.x ~18k) + // leave a post-fold FLOOR that can sit AT the small-window preflight + // target — t4 then dead-ends ("0 compressible ranges" fail-fast 502, the + // #2847 shape) although the fold itself fired correctly in t3. Measure the + // floor (raw wire estimate from the last preflight line) and widen just + // enough for t4 to land with margin. Adaptive-smallWindow recipes only — + // fixed-SMALL_WINDOW lanes keep byte-identical behavior. + if (recipe.smallWindow && t3.ok) { + // Direct evidence = the last preflight trigger line's raw wire estimate. + // Without one (clean t3, no over-window request ran yet) assume the + // WORST CASE: the fold removed nothing net, floor ≈ the t2-end payload. + const floorM = [...ctx.readLog().matchAll(/local-est=[\d.]+ raw=(\d+)/g)].pop(); + const floor = floorM ? Number(floorM[1]) : lastP; + const headroom = (w) => Math.min(OUTPUT_TOKENS, Math.floor(w / 4)); + if (floor > 0 && floor >= smallWin - headroom(smallWin)) { + const want = Math.ceil((floor * 1.1) / 1024) * 1024; + let w2 = smallWin; + while (w2 - headroom(w2) < want && w2 < BIG_WINDOW) w2 += 1024; + if (w2 !== smallWin && w2 <= mid) { + console.log(`[${agent}] post-fold floor ${floor} tok at the target edge — widening small window ${smallWin} -> ${w2} so t4 lands with margin`); + writeWindow(w2); + if (recipe.setWindow) await recipe.setWindow(ctx, w2); + await restartLaneProxy(ctx, recipe); + await new Promise((r) => setTimeout(r, 1000)); + } else if (w2 > mid) { + console.log(`[${agent}] post-fold floor ${floor} tok needs more than mid ${mid} — leaving ${smallWin}, t4 will report honestly`); + } + } + } + let t4 = { ok: false, why: "never ran", out: "" }; const warm4 = recipe.warmupPrefix || ""; for (let i = 0; i < 2 && !(t4.ok && t4.out.trim()); i++) @@ -376,9 +478,73 @@ async function runLane(agent, recipe) { } // ---- wire + proxy truth assertions ------------------------------------ - const reqs = readJsonl(recOut).filter((r) => r.tag === "req"); + const allLines = readJsonl(recOut); + const reqs = allLines.filter((r) => r.tag === "req"); + const respBySeq = new Map(allLines.filter((r) => r.tag === "resp").map((r) => [r.seq, r])); + const errSeqs = new Set(allLines.filter((r) => r.tag === "error").map((r) => r.seq)); const log = ctx.readLog(); record(agent, "upstream saw traffic", reqs.length > 0, `${reqs.length} requests`); + + // #3475 protocol matrix: hard wire-shape assertion — every door POST at the + // bili→upstream hop must carry this api type's path AND top-level body + // shape. A silent wire swap (google arriving as chat, responses as chat…) + // must fail here, not read green on a generic "turn completed". + if (recipe.wireApi && WIRE_EXPECT[recipe.wireApi]) { + const expect = WIRE_EXPECT[recipe.wireApi]; + const doorReqs = reqs.filter((r) => r.method === "POST" && expect.pathRe.test(r.path)); + const badBody = doorReqs.filter((r) => !expect.bodyCheck(r.body || {})); + record(agent, `wire shape ${recipe.wireApi} at bili->upstream hop`, doorReqs.length > 0 && badBody.length === 0, + doorReqs.length === 0 + ? `no POST matched ${expect.door}; paths seen: ${[...new Set(reqs.map((r) => `${r.method} ${r.path}`))].slice(0, 6).join(" | ") || "(none)"}` + : badBody.length > 0 + ? `${badBody.length}/${doorReqs.length} off-shape body(s); first off-shape keys: [${Object.keys(badBody[0].body || {}).join(",")}]` + : `${doorReqs.length} request(s), all on-shape (path ~${expect.door})`); + // round-trip evidence (#3475 acceptance 2): what the model hop actually + // received — first door request, persisted for the report. + if (doorReqs[0]) { + const b = doorReqs[0].body || {}; + console.log(`[${agent}] egress sample: ${doorReqs[0].path} top-level=[${Object.keys(b).join(",")}]`); + fs.writeFileSync(path.join(ctx.laneDir, "egress-sample.json"), JSON.stringify({ + path: doorReqs[0].path, headers: doorReqs[0].headers, topLevelKeys: Object.keys(b), bodyHead: JSON.stringify(b).slice(0, 2000), + }, null, 2)); + } + // Blocked-lane rules — two kinds, both excluded from the gate tally: + // (env) the UPSTREAM door is unhealthy, so bili never ran against a + // live model: either EVERY door request was rejected (4xx/5xx or + // connection error) and no turn completed (opencode2/#3461 rule), + // or the door DEGRADED MID-FLOW — some requests succeeded, later + // ones were rejected, and the flow never finished. A self-hosted + // door mapping bug (responses→chat validation, #3461 sibling), + // not a bili wire defect. Traffic + wire-shape + tools stay the + // hard verdict; turn-dependent records drop out of the tally. + // (client) the CLIENT cannot drive this protocol at its installed version: + // zero upstream traffic and every turn died on a config-validation + // error (dsh ≤0.2.1-rc.1's cordis schema lists no + // google-generative-ai among provider apis). Nothing ran, so the + // whole lane drops out of the tally; it auto-promotes when the + // client ships the protocol — no script change needed. + const turnRecs = results.filter((r) => r.lane === agent && /completes/.test(r.name)); + const flowComplete = turnRecs.length >= 4 && turnRecs.every((r) => r.ok); + const anyTurnOk = turnRecs.some((r) => r.ok); + const doorStatusOf = (r) => errSeqs.has(r.seq) ? "conn-error" : (respBySeq.get(r.seq) ? `HTTP ${respBySeq.get(r.seq).status}` : "no-response"); + const rejected = doorReqs.filter((r) => errSeqs.has(r.seq) || !respBySeq.get(r.seq) || respBySeq.get(r.seq).status >= 400); + const statuses = [...new Set(rejected.map(doorStatusOf))].join(", "); + const turnStderr = fs.existsSync(path.join(ctx.laneDir, "turn-stderr.log")) ? fs.readFileSync(path.join(ctx.laneDir, "turn-stderr.log"), "utf8") : ""; + let blocked = null; + if (rejected.length === doorReqs.length && doorReqs.length > 0 && !anyTurnOk) { + blocked = { kind: "env", reason: `upstream ${expect.door} unhealthy (${statuses})` }; + } else if (rejected.length > 0 && doorReqs.length - rejected.length > 0 && !flowComplete) { + blocked = { kind: "env", reason: `upstream ${expect.door} degraded mid-flow (${statuses}) after ${doorReqs.length - rejected.length} OK request(s)` }; + } else if (doorReqs.length === 0 && !anyTurnOk && /ValidationError/.test(turnStderr) && /api expected/.test(turnStderr)) { + const m = /\$\.providers\.\w+\.api expected [^\n]+/.exec(turnStderr); + blocked = { kind: "client", reason: `client config surface rejects api ${recipe.wireApi} (${m ? m[0] : "ValidationError"}) — zero upstream traffic` }; + } + if (blocked) { + BLOCKED.set(agent, blocked); + record(agent, "BLOCKED degraded verdict", true, + `${blocked.kind}-blocked: ${blocked.reason} — ${blocked.kind === "env" ? "turn-dependent records below are excluded from the gate tally (opencode2/#3461 rule); rerun against an endpoint exposing a real " + expect.door + " door for the full verdict" : "nothing ran (client cannot express this api yet); the whole lane is excluded from the tally and auto-promotes when the client ships the protocol"}`); + } + } const fold = /acp-compress-obs\] shrink (\d+)%/.exec(log); record(agent, "fold fired (preflight or model-side)", !!fold, fold ? `shrink ${fold[1]}%` : "no acp-compress-obs line"); const t4txt = fs.existsSync(path.join(ctx.laneDir, "turn-t4.txt")) ? fs.readFileSync(path.join(ctx.laneDir, "turn-t4.txt"), "utf8") : ""; @@ -387,7 +553,11 @@ async function runLane(agent, recipe) { const toolsNames = (r) => { const b = r.body || {}; const flat = (arr) => Array.isArray(arr) ? arr.map((t) => t?.function?.name ?? t?.name).filter(Boolean) : []; - return [...flat(b.tools), ...flat(b.input_tools)]; + // google wire declares tools as tools[].functionDeclarations[].name (#3475). + const gdecls = (arr) => Array.isArray(arr) + ? arr.flatMap((t) => (Array.isArray(t?.functionDeclarations) ? t.functionDeclarations.map((d) => d?.name) : [])).filter(Boolean) + : []; + return [...flat(b.tools), ...gdecls(b.tools), ...flat(b.input_tools)]; }; const allTools = [...new Set(reqs.flatMap(toolsNames))]; record(agent, "ACP tools declared on wire", allTools.some((n) => n === "compress" || n === "acp_status"), @@ -446,6 +616,14 @@ async function startManualProxy(ctx, args) { throw new Error(`manual proxy :${args.port} did not become healthy (see ${ctx.laneDir}/proxy.log)`); } +// Prefix lanes re-read bili.json through a fresh proxy per phase (native lanes' +// self-spawned zone proxy picks the new window up per request instead). +async function restartLaneProxy(ctx, recipe) { + if (recipe.type !== "prefix") return; + if (ctx.proxy) { try { process.kill(-ctx.proxy.pid, "SIGKILL"); } catch {} ctx.proxy = null; } + await startManualProxy(ctx, ctx.proxyArgs ? ctx.proxyArgs() : { port: ctx.proxyPort }); +} + // --------------------------------------------------------------------------- // host recipes // --------------------------------------------------------------------------- @@ -464,68 +642,160 @@ function parseEvents(stdout) { return { sid, final: final.trim() }; } -const RECIPES = { - // ---- dsh: native plugin lane (THE #3401 lane) ------------------------- - dsh: { - type: "native", pluginAgent: "dsh", +// ---- dsh family: native plugin lanes (THE #3401 lane) + protocol matrix +// (#3475) + proxy-mode contrast lane (owner add-on) ----------------------- +// One headless driver parameterized over the four pi-ai provider api types. +// Only the cordis provider `api` and the baseURL change per variant; profile +// layout, window rewrite and turn shape are shared. The openai-completions +// NATIVE variant emits a cordis.patch.yml byte-identical to the pre-#3475 dsh +// lane (zero behavior change for the existing lane). +// BaseURL suffixes follow each SDK's URL assembly: +// openai-completions / openai-responses — OpenAI SDK appends /chat/completions | /responses +// anthropic-messages — Anthropic SDK appends /v1/messages +// google-generative-ai — genai SDK appends /models/:streamGenerateContent?alt=sse +// (pi-ai sets apiVersion="" so NO version segment is added): +// the suffix must carry /v1beta itself. +const DSH_WARMUP_PREFIX = "First run the bash tool with command `echo warmup`. Then "; + +async function dshProfileSetup(ctx, { api, baseURL, withBili }) { + // Lane-local dsh install: the system dsh may be too old for the + // headless --json shape, and resolveBundleDir() tries the dsh install + // anchor BEFORE the profile dir — a stale global billion-context on the + // machine would shadow the candidate bundle ("declares no dsh.bundle"). + // A per-lane prefix keeps the lane hermetic on any machine. + const dshPrefix = path.join(ctx.laneDir, "dsh-prefix"); + console.log(`-- lane ${ctx.agent}: installing @deepseek-ai/dsh@${DSH_VERSION}`); + must("npm", ["install", "-g", "--prefix", dshPrefix, "--no-audit", "--no-fund", `@deepseek-ai/dsh@${DSH_VERSION}`], { timeout: 300_000 }); + const dshHome = path.join(ctx.laneDir, "dsh-home"); + const profile = path.join(dshHome, "profiles", "headless"); + fs.mkdirSync(path.join(profile, "node_modules"), { recursive: true }); + fs.mkdirSync(path.join(ctx.laneDir, "tmp"), { recursive: true }); + const bundles = ["@deepseek-ai/dsh-base", "@deepseek-ai/dsh-headless"]; + const deps = {}; + if (withBili) { + // candidate as a REAL copy in the profile (e2e-native-dsh.test.ts shape; + // bundles @deepseek-ai/dsh-base/-headless resolve from the dsh install) + fs.cpSync(ctx.cand, path.join(profile, "node_modules", "billion-context"), { recursive: true }); + deps["billion-context"] = ctx.candV; + bundles.push("billion-context"); + } + const pkg = { + name: withBili ? "dsh-profile-plugin-gate" : "dsh-profile-proxy-gate", private: true, + dependencies: deps, + dsh: { profile: { bundles, patchReload: "startup" } }, + }; + fs.writeFileSync(path.join(profile, "package.json"), JSON.stringify(pkg, null, 2) + "\n"); + fs.writeFileSync(path.join(profile, "cordis.yml"), "[]\n"); + const cordisFor = (window) => + fs.writeFileSync(path.join(profile, "cordis.patch.yml"), [ + "- id: llm-pi-ai", + " config:", + " providers:", + " gate:", + " displayName: Gate Upstream", + ` apiKeyEnv: GATE_UPSTREAM_KEY`, + ` api: ${api}`, + ` baseURL: ${baseURL}`, + " models:", + ...MODEL_IDS.map((id) => [ + ` - id: "${id}"`, + ` contextWindow: ${window}`, + ` maxTokens: ${OUTPUT_TOKENS}`, + ].join("\n")), + "- id: agent-default-model", + " config:", + " provider: gate", + ` model: "${MODEL_MAIN}"`, + ].join("\n") + "\n"); + cordisFor(BIG_WINDOW); + fs.writeFileSync(path.join(profile, "pnpm-workspace.yaml"), "packages:\n - .\nnodeLinker: hoisted\nautoInstallPeers: false\n"); + ctx.dshEnv = { + ...ctx.baseEnv, + DSH_HOME: dshHome, + DSH_TELEMETRY_MODE: "DISABLED", + // spill-local mkdtemps into the lane (host /tmp may be read-only) + TMPDIR: path.join(ctx.laneDir, "tmp"), + PATH: [path.join(dshPrefix, "bin"), process.env.PATH].filter(Boolean).join(path.delimiter), + }; +} + +const dshSetWindow = (ctx, w) => { + // window truth on plugin lanes = the host's own model config (rides + // the x-bili-plugin-context-window header); bili.json is ignored there. + // On the proxy-mode lane the same rewrite keeps the host view consistent + // with the bili.json window that writeWindow() writes. + const profile = path.join(ctx.laneDir, "dsh-home", "profiles", "headless"); + const txt = fs.readFileSync(path.join(profile, "cordis.patch.yml"), "utf8"); + // /g: MODEL_IDS may name several models — every entry carries its own + // contextWindow and all must move together (multi-model cordis). + fs.writeFileSync(path.join(profile, "cordis.patch.yml"), txt.replace(/contextWindow: \d+/g, `contextWindow: ${w}`)); +}; + +const dshTurn = async (ctx, prompt, sid, tag) => { + const args = ["--profile", "headless", "headless"]; + if (sid.v) args.push("--session-id", sid.v); + args.push("--json", prompt); + const r = sh("dsh", args, { cwd: ctx.workDir, env: ctx.dshEnv, timeout: ctx.TMO }); + fs.appendFileSync(path.join(ctx.laneDir, "turn-stderr.log"), `\n=== ${tag} exit=${r.status} ===\n${r.stderr}\n`); + const { sid: s, final } = parseEvents(r.stdout); + if (s) sid.v = s; + return { ok: r.status === 0 && !!final, why: r.stderr.slice(-300) || `exit ${r.status}`, out: final }; +}; + +const dshSmallWindow = (ctx, lastP) => { + // dsh's fixed scaffolding (system prompt + tool defs, ~18k tok on 0.2.x) + // exceeds the static SMALL_WINDOW minus the output reservation — preflight + // fail-fasts forever at 24576. Derive from the LIVE t2-end payload instead: + // deep enough to force a fold, wide enough that the folded session fits. + const base = Number(process.env.GATE_DSH_SMALL_WINDOW || 0); + if (base > 0) return base; + const p = lastP > 0 ? lastP : 30000; + return Math.min(BIG_WINDOW >> 1, Math.max(SMALL_WINDOW, Math.ceil((p * 0.85 + OUTPUT_TOKENS) / 1024) * 1024)); +}; + +function dshRecipe(api) { + const baseSuffix = api === "anthropic-messages" ? "" : api === "google-generative-ai" ? "/v1beta" : "/v1"; + return { + type: "native", pluginAgent: "dsh", wireApi: api, // guarantee ≥2 LLM requests per turn so the model-info retry (see // BILI_MODEL_INFO_RETRY_MS) gets a second request to stamp the window on - warmupPrefix: "First run the bash tool with command `echo warmup`. Then ", + warmupPrefix: DSH_WARMUP_PREFIX, async setup(ctx) { - const dshHome = path.join(ctx.laneDir, "dsh-home"); - const profile = path.join(dshHome, "profiles", "headless"); - fs.mkdirSync(path.join(profile, "node_modules"), { recursive: true }); - // candidate as a REAL copy in the profile (e2e-native-dsh.test.ts shape; - // bundles @deepseek-ai/dsh-base/-headless resolve from the dsh install) - fs.cpSync(ctx.cand, path.join(profile, "node_modules", "billion-context"), { recursive: true }); - const pkg = { - name: "dsh-profile-plugin-gate", private: true, - dependencies: { "billion-context": ctx.candV }, - dsh: { profile: { bundles: ["@deepseek-ai/dsh-base", "@deepseek-ai/dsh-headless", "billion-context"], patchReload: "startup" } }, - }; - fs.writeFileSync(path.join(profile, "package.json"), JSON.stringify(pkg, null, 2) + "\n"); - fs.writeFileSync(path.join(profile, "cordis.yml"), "[]\n"); - const cordisFor = (window) => - fs.writeFileSync(path.join(profile, "cordis.patch.yml"), [ - "- id: llm-pi-ai", - " config:", - " providers:", - " gate:", - " displayName: Gate Upstream", - ` apiKeyEnv: GATE_UPSTREAM_KEY`, - " api: openai-completions", - ` baseURL: ${ctx.recUrl}/v1`, - " models:", - ...MODEL_IDS.map((id) => [ - ` - id: "${id}"`, - ` contextWindow: ${window}`, - " maxTokens: 4096", - ].join("\n")), - "- id: agent-default-model", - " config:", - " provider: gate", - ` model: "${MODEL_MAIN}"`, - ].join("\n") + "\n"); - cordisFor(BIG_WINDOW); - fs.writeFileSync(path.join(profile, "pnpm-workspace.yaml"), "packages:\n - .\nnodeLinker: hoisted\nautoInstallPeers: false\n"); - ctx.env = { ...ctx.baseEnv, DSH_HOME: dshHome, DSH_TELEMETRY_MODE: "DISABLED" }; - }, - setWindow(ctx, w) { - // window truth on plugin lanes = the host's own model config (rides - // the x-bili-plugin-context-window header); bili.json is ignored here - const profile = path.join(ctx.laneDir, "dsh-home", "profiles", "headless"); - const txt = fs.readFileSync(path.join(profile, "cordis.patch.yml"), "utf8"); - fs.writeFileSync(path.join(profile, "cordis.patch.yml"), txt.replace(/contextWindow: \d+/g, `contextWindow: ${w}`)); + await dshProfileSetup(ctx, { api, baseURL: `${ctx.recUrl}${baseSuffix}`, withBili: true }); }, - async turn(ctx, prompt, sid, tag) { - const args = ["--profile", "headless", "headless"]; - if (sid.v) args.push("--session-id", sid.v); - args.push("--json", prompt); - const r = sh("dsh", args, { cwd: ctx.workDir, env: ctx.env, timeout: ctx.TMO }); - const { sid: s, final } = parseEvents(r.stdout); - if (s) sid.v = s; - return { ok: r.status === 0 && !!final, why: r.stderr.slice(-300) || `exit ${r.status}`, out: final }; + setWindow: dshSetWindow, + turn: dshTurn, + smallWindow: dshSmallWindow, + }; +} + +const RECIPES = { + dsh: dshRecipe("openai-completions"), + "dsh-responses": dshRecipe("openai-responses"), + "dsh-anthropic": dshRecipe("anthropic-messages"), + "dsh-google": dshRecipe("google-generative-ai"), + + // ---- dsh-proxy: PROXY-mode contrast lane (#3475 owner add-on) ---------- + // Same host, OTHER activation path: the profile does NOT load the + // billion-context bundle (its fetch intercept would stamp x-bili-plugin and + // flip the lane back into plugin mode) — dsh drives the chat wire straight + // into a manually started proxy via the /bili/ URL, codex/claude style. + // Contrasts the native dsh lane within ONE script: #3177/#3401-style + // suppression only exists on the plugin path; here nudge/fold must ride the + // wire in plain proxy mode (acp_summary carrier). Window truth = bili.json + // (writeWindow), like every other prefix lane. + "dsh-proxy": { + type: "prefix", pluginAgent: "dsh", wireApi: "openai-completions", + warmupPrefix: DSH_WARMUP_PREFIX, + async setup(ctx) { + const port = await freePort(); + await dshProfileSetup(ctx, { api: "openai-completions", baseURL: `http://127.0.0.1:${port}/bili/${ctx.recUrl}/v1`, withBili: false }); + ctx.proxyArgs = () => ({ port }); + await startManualProxy(ctx, { port }); }, + setWindow: dshSetWindow, + turn: dshTurn, + smallWindow: dshSmallWindow, }, // ---- pi: native lane, three wires possible — drive openai-chat --------- @@ -766,8 +1036,21 @@ async function main() { console.log(`\n===== plugin lane: ${agent} (${recipe.type}) =====`); try { await runLane(agent, recipe); } catch (e) { record(agent, "lane", false, String(e).slice(0, 400)); } } - const failed = results.filter((r) => !r.ok); - console.log(`\nplugin-lanes: ${results.length - failed.length}/${results.length} passed`); + // #3475: env-blocked lanes (upstream lacks the protocol door) keep their + // degraded verdict — FAIL records from turn-dependent assertions are honest + // output but excluded from the gate tally, mirroring the opencode2/#3461 rule. + // Blocked lanes drop turn-dependent records from the tally: env-blocked + // lanes keep their observable hard assertions (traffic, wire shape, tools, + // claim), client-blocked lanes ran nothing so the whole lane drops out. + const isTurnDependent = (name) => /completes|marker answered|self-calibrated|recalled|nudge reached|fold fired|model invoked/.test(name); + const counted = results.filter((r) => { + const b = BLOCKED.get(r.lane); + if (!b) return true; + return b.kind === "env" ? !isTurnDependent(r.name) : false; + }); + const failed = counted.filter((r) => !r.ok); + for (const [lane, b] of BLOCKED) console.log(`${b.kind.toUpperCase()}-BLOCKED [${lane}] ${b.reason}`); + console.log(`\nplugin-lanes: ${counted.length - failed.length}/${counted.length} passed${BLOCKED.size ? ` (${[...BLOCKED.keys()].map((l) => `${l}:${BLOCKED.get(l).kind}`).join(", ")})` : ""}`); process.exit(failed.length ? 1 : 0); } main(); diff --git a/scripts/release-gate-local.mjs b/scripts/release-gate-local.mjs index 2f09d8676d..fae7a28299 100644 --- a/scripts/release-gate-local.mjs +++ b/scripts/release-gate-local.mjs @@ -426,7 +426,13 @@ async function main() { const fold = /acp-compress-obs\] shrink (\d+)%/.exec(s2); record("A", "turn 2 completes after window shrink", t2.ok, t2.ok ? "" : t2.why.slice(0, 160)); record("A", "fold fired mid-session", !!fold, fold ? `shrink ${fold[1]}%` : "no acp-compress-obs line"); - record("A", "nudge/preflight engaged", /nudge INJECT|preflight compressing/.test(s2), /nudge INJECT/.test(s2) ? "nudge INJECT" : ""); + // #3416: two INDEPENDENT assertions — the old OR form let a preflight + // line alone green this check while the nudge path was silently dead + // (exactly the #3401 blind spot: suppression-class bugs passed gate). + const nudgeHit = /nudge INJECT/.test(s2); + const preflightHit = /preflight compressing/.test(s2); + record("A", "nudge injected (hard)", nudgeHit, nudgeHit ? "nudge INJECT" : "no nudge INJECT line — preflight must not substitute"); + record("A", "preflight engaged", preflightHit, preflightHit ? "preflight compressing" : "no preflight compressing line"); record("A", "S07 marker recalled from FOLDED context", has(t2.final, CODEWORDS[6]), `final="${t2.final.trim().slice(0, 80)}"`); const hitsA = [...s2.matchAll(/cache hit (\d+)%/g)].map((m) => Number(m[1])); record("A", "cache recovered after fold", hitsA.length > 0 && Math.max(...hitsA) >= 50, hitsA.length ? `hits=${hitsA.join(",")}%` : "no cache report at all"); diff --git a/src/absorb.ts b/src/absorb.ts new file mode 100644 index 0000000000..38dfa50ba7 --- /dev/null +++ b/src/absorb.ts @@ -0,0 +1,126 @@ +import { + ABSORB_TOOL_NAME, + applyAbsorb, + appendAbsorbPrompts, + defaultCountTokens, + hideAbsorbedMessages, + parseAbsorbInput, + type AbsorbConfig, + type AbsorbOutcome, + type CompressionState, + type Config, + type CoreMessage, +} from "acp-kernel"; +import { PROXY_TOOL_NAMES, READ_ORIGINAL_IMAGE_TOOL_NAME, RULE_TOOL_NAME } from "./compress-tool.js"; +import { effectiveRulesEnabled } from "./rules-feature.js"; +import { ccrEnabled, retrieveToolName } from "./store.js"; +import { log as loggerLog } from "./logger.js"; +import type { Session } from "./session.js"; +import { toolFail, toolOk, type ProxyToolResult } from "./proxy-tool-result.js"; + +const EFFECTIVE_ABSORB_KEY = "effectiveAbsorb"; + +export function absorbEnabled(config: Config): boolean { + return config.absorb?.enabled === true; +} + +export function absorbToolName(config: Config): string { + return config.absorb?.toolName ?? ABSORB_TOOL_NAME; +} + +// The kernel's ACP_TOOL_NAMES deliberately excludes absorb (hosts register it +// opt-in), so adjudication must check the resolved config alongside the static +// set. Two config sources exist: wire paths carry the per-request resolved +// Config, while the plugin tool API resolves sessions without a request +// context and falls back to the base kernel Config — prepare* therefore stores +// the last resolved absorb block per session for that path to read. +export function effectiveAbsorbConfig(session: Session | undefined, fallback: Config): AbsorbConfig | undefined { + const meta = session?.metadata[EFFECTIVE_ABSORB_KEY]; + if (meta && typeof meta === "object" && typeof (meta as AbsorbConfig).enabled === "boolean") { + return meta as AbsorbConfig; + } + return fallback.absorb; +} + +export function storeEffectiveAbsorb(session: Session, config: Config): void { + session.metadata[EFFECTIVE_ABSORB_KEY] = config.absorb ?? null; +} + +export function isProxyToolFor(name: string, session: Session | undefined, config: Config): boolean { + if (PROXY_TOOL_NAMES.has(name)) return true; + // #3400: read_original_image executes server-side on EVERY lane — even + // where it is not registered in the manifest (codex) a hallucinated call + // must never reach the client as an unsupported tool. + if (name === READ_ORIGINAL_IMAGE_TOOL_NAME) return true; + const absorb = effectiveAbsorbConfig(session, config); + if (absorb?.enabled === true && name === (absorb.toolName ?? ABSORB_TOOL_NAME)) return true; + if (ccrEnabled(session) && name === retrieveToolName(session)) return true; + return effectiveRulesEnabled(session, config) && name === RULE_TOOL_NAME; +} + +// Per-turn view transform: drop absorbed tool-call/result pairs, then append +// the forced [ACP absorb] instruction to every eligible large tool result. +// Hiding is unconditional so recorded absorptions stay hidden even if the +// feature is disabled mid-session; prompting self-gates inside the kernel on +// config.absorb.enabled (processTurn already injects markers from the same +// config — the append here is an idempotent safety net). Returns a new array +// that shares message refs with the input. +export function applyAbsorbView(messages: CoreMessage[], state: CompressionState, config: Config, tokenCount: number): CoreMessage[] { + const hidden = hideAbsorbedMessages(messages, state); + const prompted = appendAbsorbPrompts(hidden, state, config, tokenCount, defaultCountTokens); + return prompted.messages; +} + +type AbsorbExecCtx = { + config: Config; + messages: CoreMessage[]; + session: Session; + log?: (msg: string) => void; +}; + +// Execute one absorb call against the session (streaming loop, JSON rewrite +// and the plugin tool API share this). Returns model-facing result text; +// failures carry FAILED so visibility markers render ❌ and the loop's +// repeat-failure guard reasons about them uniformly with compress failures. +export function executeAbsorb(args: unknown, callId: string | undefined, absorb: AbsorbConfig, ctx: AbsorbExecCtx): ProxyToolResult { + const log = ctx.log ?? ((msg: string) => loggerLog("info", msg)); + const parsed = parseAbsorbInput(args, callId, (w) => log(`[acp-absorb] ${w}`)); + if (!parsed) { + return toolFail('[absorb FAILED: invalid input — expected { ref: "mNNNNN", summary: "..." }]'); + } + const before = ctx.session.state.absorbed?.length ?? 0; + let outcome: AbsorbOutcome; + try { + outcome = applyAbsorb({ + ref: parsed.ref, + summary: parsed.summary, + absorbCallId: parsed.absorbCallId, + messages: ctx.messages, + state: ctx.session.state, + config: { ...ctx.config, absorb }, + }); + } catch (err) { + log(`[acp-absorb] error: ${String(err)}`); + return toolFail(`[absorb FAILED: ${String(err)}]`); + } + ctx.session.state = outcome.state; + if (!outcome.ok) { + log(`[acp-absorb] ${outcome.resultText}`); + return toolFail(`[absorb FAILED: ${outcome.resultText}]`); + } + // ok=true also covers the no-op re-absorb ("already absorbed"); detect a + // real absorption by the appended record so credits net exactly once. + const records = outcome.state.absorbed ?? []; + const record = records.length > before ? records[records.length - 1] : undefined; + if (record) { + // Same credit netting as compress (#252): the re-request re-sends the + // unfolded history, so usage reports over-report until the next + // prepare hides the pair. + ctx.session.stats.compressCreditTokens = (ctx.session.stats.compressCreditTokens ?? 0) + record.tokensReclaimed; + ctx.session.stats.lastInputTokens = Math.max(0, ctx.session.stats.lastInputTokens - record.tokensReclaimed); + log(`[acp-absorb] ${outcome.resultText} (credit -${record.tokensReclaimed} tok)`); + } else { + log(`[acp-absorb] ${outcome.resultText}`); + } + return toolOk(outcome.resultText); +} diff --git a/src/acp-status.ts b/src/acp-status.ts index d8da280a54..17ed8d0455 100644 --- a/src/acp-status.ts +++ b/src/acp-status.ts @@ -24,6 +24,7 @@ import { getUpdateVisibility } from "./update-notes.js"; import { VERSION, BUILD_COMMIT } from "./version.js"; import { isExternalSummaryBlock } from "./external-summary-marker.js"; import { storageStatusLabel } from "./persist.js"; +import { storageStatusWire } from "./storage/runtime-status.js"; import { toolOk, type ProxyToolResult } from "./proxy-tool-result.js"; interface AcpStatusCtx { @@ -362,6 +363,15 @@ export function handleAcpStatus(args: Record, ctx: AcpStatusCtx // am I on" never requires shell access to the lane log. extra.push(""); extra.push(`STORAGE — ${storageStatusLabel()}`); + // #3385 (owner directive 3, acp_status surface): when the unified store is + // degraded (engine missing, self-download cooldown, ...), surface the SAME + // reason string the web banner and diagnostics carry — on Node < 22.5 it + // contains the explicit Node upgrade lever, so the MODEL can tell the + // user instead of the fact living only in lane logs the user never opens. + const storageWire = storageStatusWire(); + if (storageWire.degraded && storageWire.reason) { + extra.push(` ⚠ degraded — ${storageWire.reason} (relay this to the user)`); + } const archive = preCompactionArchiveOf(ctx.session); if (Object.keys(archive).length > 0) { extra.push(""); diff --git a/src/agent/dsh-compaction-disable.ts b/src/agent/dsh-compaction-disable.ts new file mode 100644 index 0000000000..7b9cd48f90 --- /dev/null +++ b/src/agent/dsh-compaction-disable.ts @@ -0,0 +1,92 @@ +// #3354: stand dsh's native basic compaction down at LOAD TIME — vendored from +// deshark42/dsh-compaction-basic-disable v0.1.1 (MIT © 2026 deshark42, +// https://github.com/deshark42/dsh-compaction-basic-disable), ported into +// bili's own bundle layer at the author's suggestion (their repo issue #1). +// The overlay marks the preset rows disabled instead of copying or overwriting +// the official preset configs, so upstream preset updates keep working without +// re-copying a frozen snapshot — the #2474 full-snapshot recipe stays as the +// fallback for dsh builds whose internal 'internal/config' hook moved. +// +// Why it exists: when bili carries a dsh conversation it owns compression +// (#1729); a dsh compaction checkpoint landing outside bili's knowledge +// durably shadows the raw history and destroys the compression substrate. +// Non-web profiles already stand down the host-plane row via the bundle patch +// (auto:false); web profiles run their instance INSIDE an agent preset whose +// nested rows no patch layer can address by id (#1772/#2360) — this module is +// what reaches them. + +/** The two dsh packages whose backends must not instantiate while bili owns the lane. */ +const DSH_COMPACTION_BASIC_PACKAGE = "@deepseek-ai/dsh-compaction-basic"; +const DSH_COMMAND_COMPACT_PACKAGE = "@deepseek-ai/dsh-command-compact"; +/** The agent-preset entry whose resolved config carries the plugin rows. */ +const DSH_AGENT_PRESET_PACKAGE = "@deepseek-ai/dsh-agent-preset"; +/** Gate service name referenced by the preset `inject:` entries in + * dsh.bundle.patch.yml and the launcher overlay patch. A preset cannot + * register until this service is provided, so the middleware below is + * guaranteed registered before any protected preset resolves its config even + * when sibling imports finish out of order; if this module fails to load the + * presets stay pending (fail-safe: no unprotected start). Coexistence with + * the upstream plugin is safe: its service name differs and both overlays are + * idempotent (disabled:true twice = disabled:true). */ +export const BILI_COMPACTION_DISABLE_SERVICE = "biliBasicCompactionPolicy"; +/** The four Web agent presets the overlay must reach (dsh-web-app roster; + * preset-minimal ships no compaction group today — the row is a no-op there + * and future-proofs it if one lands). */ +const DSH_WEB_PRESET_IDS = ["preset-standard", "preset-ptc", "preset-minimal", "preset-cordis"] as const; + +/** The bundle-patch rows gating each Web preset on the overlay service — the + * single source of truth shared by the launcher overlay string and pinned + * against dsh.bundle.patch.yml by tests (the two layers must stay in sync, + * cf. #2605). Entry inject lists are REPLACED by patches, not concatenated — + * if a future dsh ships its own inject list on these entries, merge manually + * here AND in both patch layers. */ +export function compactionDisablePatchRows(): string { + return DSH_WEB_PRESET_IDS.map((id) => `- id: ${id}\n inject: [${BILI_COMPACTION_DISABLE_SERVICE}]\n`).join(""); +} + +type PresetRow = { name?: unknown; group?: unknown; config?: unknown }; + +/** Mark every compaction row of a preset plugins tree disabled. Pure: returns + * detached copies and never mutates the input rows (the resolved config may + * be shared with other consumers). */ +export function disableBasicCompaction(rows: readonly unknown[]): unknown[] { + return rows.map((row) => { + if (row === null || typeof row !== "object") return row; + const r = row as PresetRow & Record; + if (r.name === DSH_COMPACTION_BASIC_PACKAGE || r.name === DSH_COMMAND_COMPACT_PACKAGE) { + return { ...r, disabled: true }; + } + if (r.group && Array.isArray(r.config)) { + return { ...r, config: disableBasicCompaction(r.config) }; + } + return row; + }); +} + +type CompactionDisableHost = { + on?: (event: string, listener: (raw: unknown, next: () => unknown) => unknown, options?: { global?: boolean }) => void; + reflect?: { provide?: (name: string, value: unknown) => void } | undefined; +}; +type MiddlewareSelf = { entry?: { options?: { name?: unknown } } | undefined }; + +/** Register the load-time overlay + publish the gate. Must run during + * apply() — the provide happens AFTER the middleware registers so the inject + * gate can never be satisfied before the transform is live. Degradation is + * per-layer and fail-safe: no `on` → the host predates the internal/config + * hook, the wire-level refusal guard (#1729) stays the backstop; no + * `reflect.provide` → the presets simply never see the gate and boot as they + * did pre-#3354. */ +export function registerCompactionDisable(ctx: CompactionDisableHost): void { + if (typeof ctx.on === "function") { + // Preset entries must carry the gate: sibling imports may finish out of order. + ctx.on("internal/config", function (this: MiddlewareSelf, _raw: unknown, next: () => unknown): unknown { + const resolved = next(); + if (this?.entry?.options?.name !== DSH_AGENT_PRESET_PACKAGE) return resolved; + const config = (resolved ?? {}) as { plugins?: unknown }; + if (!Array.isArray(config.plugins)) return resolved; + return { ...config, plugins: disableBasicCompaction(config.plugins) }; + }, { global: true }); + } + // Publish only after the middleware is registered; the host owns both lifetimes. + ctx.reflect?.provide?.(BILI_COMPACTION_DISABLE_SERVICE, Object.freeze({ owner: "billion-context", issue: "#3354" })); +} diff --git a/src/agent/dsh-native-client.ts b/src/agent/dsh-native-client.ts index 479afd2acc..f9e6aa845d 100644 --- a/src/agent/dsh-native-client.ts +++ b/src/agent/dsh-native-client.ts @@ -234,8 +234,13 @@ const en: Dict = { }; function readOrigin(): string | undefined { - const g = globalThis as { __BILI__?: { origin?: unknown } }; - const origin = g.__BILI__?.origin; + const g = globalThis as { __BILI__?: { origin?: unknown; relay?: unknown } }; + const snap = g.__BILI__; + // #3373: relay snapshot — the entry serves the admin surface itself, so + // the base is same-origin ("" = entry-relative paths). Prefix gateways + // correct this to an absolute base on the first live probe. + if (snap !== undefined && snap.relay === true) return ""; + const origin = snap?.origin; return typeof origin === "string" && origin.length > 0 ? origin : undefined; } @@ -244,6 +249,38 @@ function openExternal(url: string): void { if (typeof w.open === "function") w.open(url, "_blank", "noopener,noreferrer"); } +/** #3373: relay mode — the probe response carries `relay: true` instead of + * a loopback origin. Derive the UI base from the probe's own response URL: + * whatever origin+prefix carried the probe also serves /__bili/. An empty + * base means same-origin paths (root deployments, test harnesses). */ +function relayBaseOf(res: { url?: unknown }): string { + const url = typeof res.url === "string" ? res.url : ""; + if (url.endsWith(ORIGIN_PATH)) { + const base = url.slice(0, -ORIGIN_PATH.length).replace(/\/+$/, ""); + return base.length > 0 ? base : ""; + } + return ""; +} + +// #3373: gateways that mount dsh web under a path prefix re-map every URL +// the index references, so the absolute probe path misses. Probe relative to +// the page's own base as a second form; the last form that answered wins the +// next tick (direct hosts never leave the primary form). +let winningProbeForm: string | undefined; + +function probeForms(): string[] { + if (winningProbeForm !== undefined) return [winningProbeForm]; + const doc = (globalThis as { document?: { baseURI?: unknown } }).document; + const baseURI = typeof doc?.baseURI === "string" && doc.baseURI.length > 0 ? doc.baseURI : undefined; + if (baseURI === undefined) return [ORIGIN_PATH]; + try { + const fallback = new URL("bili/origin", baseURI).toString(); + return fallback === ORIGIN_PATH ? [ORIGIN_PATH] : [ORIGIN_PATH, fallback]; + } catch { + return [ORIGIN_PATH]; + } +} + /** #1809/#2187/#2288: follow the host's live origin route for the lifetime * of the mount; returns the cancel used as the effect cleanup (no fetch ⇒ * no-op, older hosts without the route simply stay on the snapshot or @@ -260,25 +297,35 @@ function probeOrigin(onOrigin: (origin: string) => void, onTheme?: (theme: "ligh let timer: ReturnType | undefined; let attempts = 0; const poll = async (): Promise => { - try { - const res = await fetch(ORIGIN_PATH); - if (res.ok) { - const data = (await res.json()) as { origin?: unknown }; - if (typeof data.origin === "string" && data.origin.length > 0 && !cancelled) { - onOrigin(data.origin); + // #3373: try the known-good form first — under a path-prefix gateway + // that becomes the relative form after its first hit. + let sawOk = false; + for (const form of probeForms()) { + try { + const res = await fetch(form); + if (!res.ok) continue; + sawOk = true; + const data = (await res.json()) as { origin?: unknown; relay?: unknown }; + let bind: string | undefined; + if (data.relay === true) bind = relayBaseOf(res); + else if (typeof data.origin === "string" && data.origin.length > 0) bind = data.origin; + if (bind !== undefined && !cancelled) { + winningProbeForm = form; + onOrigin(bind); settled = true; // #3100: every resolution is also an embed-probe retry slot — // a snapshot-only origin (older hosts without the route) // never flows through onOrigin again after mount. - if (onTick !== undefined) onTick(data.origin); - } - if (onTheme && !cancelled) { - const theme = detectHostTheme(); - if (theme !== undefined) onTheme(theme); + if (onTick !== undefined) onTick(bind); + break; } + } catch { + // host without the route (older builds) or transient error: try the next form } - } catch { - // host without the route (older builds) or transient error: retry below + } + if (sawOk && onTheme && !cancelled) { + const theme = detectHostTheme(); + if (theme !== undefined) onTheme(theme); } if (cancelled) return; attempts += 1; diff --git a/src/agent/dsh-native.ts b/src/agent/dsh-native.ts index 944d2e8d54..86cff1b3c1 100644 --- a/src/agent/dsh-native.ts +++ b/src/agent/dsh-native.ts @@ -34,11 +34,13 @@ // panel labels a conversation with the host name instead of the // truncated first user message. Both services are optional — older // hosts degrade to the pre-#2604 labelless behavior, never block boot. -// Native auto-compaction is disabled by the INSTALLER/LAUNCHER patch file -// (compaction-basic auto:false override) — not by this module. dsh has no -// compaction event hook to observe a manual /compact, so its boundary is -// left to the kernel's natural ingest diff (#395 gap, acceptable: manual -// /compact is rare and auto mode is off). +// Native auto-compaction stands down on BOTH planes: the INSTALLER/LAUNCHER +// patch file keeps the host-plane row at auto:false, and this module disables +// the web-profile PRESET instances at load time (#3354, vendored from +// deshark42/dsh-compaction-basic-disable — see dsh-compaction-disable.ts). +// dsh has no compaction event hook to observe a manual /compact, so its +// boundary is left to the kernel's natural ingest diff (#395 gap, acceptable: +// manual /compact is rare and auto mode is off). import { readFileSync } from "node:fs"; import { homedir } from "node:os"; @@ -46,12 +48,13 @@ import path from "node:path"; import { logHostEvent } from "./host-log.js"; import { ensureProxyRunning, LAUNCHER_DEFAULT_HOST } from "../launcher.js"; import { APIG_RESIGN_SCHEME, unresolvedRefusals } from "../apig-resign.js"; -import { resolveResignSettings } from "../config.js"; +import { adminRelayEnabled, resolveResignSettings } from "../config.js"; import { markNativeHost, nativeAttachOrigin, nativeBootstrapGate, nativeProxyScriptPath, proxyEnvOrigin, singleFlight } from "./native-bootstrap.js"; import { installNativeFetchIntercept, noteRoutedOrigin, observeRoutedOrigin, type GateVerdict, type NativeInterceptState } from "./native-intercept.js"; import { asciiHeaderValue, fetchManifest, fetchProxyVersion, fetchStatus, fetchStatusLatest, forwardTool, reportRuntimeInfo, waitForProxyVersion, type ManifestTool } from "./shared.js"; import { createForkAdopter, sideShapedBody } from "./fork-adopt.js"; -import { clearIncrementalProbe, INCREMENTAL_TURN_MARKER_HEADER, noteIncrementalRejection, noteIncrementalAck, planIncrementalTurn, probeIncrementalTurns, type IncrementalWireMessage } from "./incremental-client.js"; +import { clearIncrementalProbe, INCREMENTAL_TURN_MARKER_HEADER, INCREMENTAL_TURN_ID_HEADER, newIncrementalTurnId, noteIncrementalRejection, noteIncrementalAck, planIncrementalTurn, probeIncrementalTurns, sweepStale, type IncrementalWireMessage } from "./incremental-client.js"; +import { registerCompactionDisable } from "./dsh-compaction-disable.js"; export { sideShapedBody } from "./fork-adopt.js"; export const name = "bili-native"; @@ -66,6 +69,21 @@ function retryIntervalMs(): number { return Number.isFinite(parsed) && parsed > 0 ? parsed : RETRY_INTERVAL_MS; } +// #3380 P2: a registration-failure streak doubles the back-off per step up to +// a hard cap — the incident retried an identical doomed failure on a flat 10s +// wall 145 times in an hour while the user saw nothing change. The cap (not a +// stop) keeps the loop alive for transient faults: a network blip or a slow +// manifest must still heal without user action. +const REG_RETRY_MAX_MS = 5 * 60 * 1000; +// #3380 P2: after this many consecutive failures, say it once in the user's +// face with the recovery action — silence is how the incident stayed hidden. +const REG_GIVE_UP_NOTICE_STREAK = 5; +/** #3380: capped exponential back-off for the registration-failure streak. */ +function regBackoffMs(failStreak: number): number { + const steps = Math.max(0, Math.min(failStreak - 1, 16)); + return Math.min(retryIntervalMs() * 2 ** steps, REG_RETRY_MAX_MS); +} + type AgentLike = { session?: { id?: unknown; @@ -131,7 +149,11 @@ export type PluginContext = { // cordis ctx.on covers both shapes — webserver/index-inject gathers // per-startup rows for the web index (__BILI__ global for the settings // entry), while session/created and session/event feed the title channel. - on?: (event: string, listener: ((table: Array<{ kind: string; name?: string; value?: unknown }>) => void) | ((session: { id?: unknown }, event: { type?: unknown; data?: { title?: unknown } }) => void)) => void; + on?: (event: string, listener: ((table: Array<{ kind: string; name?: string; value?: unknown }>) => void) | ((session: { id?: unknown }, event: { type?: unknown; data?: { title?: unknown } }) => void) | ((raw: unknown, next: () => unknown) => unknown), options?: { global?: boolean }) => void; + // #3354: cordis service registry — the load-time compaction-disable overlay + // publishes its gate service here so the preset inject entries latch on it. + // Optional: hosts without a registry skip the gate without breaking boot. + reflect?: { provide?: (name: string, value: unknown) => void } | undefined; // #2604: dsh session store + title service (core services on modern // builds, optional so older hosts keep the plugin alive). The folded // snapshot seeds labels for sessions whose `session/title` events all @@ -147,7 +169,7 @@ export type PluginContext = { register: (route: { kind: "exact" | "prefix"; path: string; - handler: (req: unknown, res: { writeHead: (status: number, headers?: Record) => void; end: (body?: string) => void }) => void | Promise; + handler: (req: unknown, res: { writeHead: (status: number, headers?: Record) => void; write?: (chunk: string | Uint8Array) => boolean; end: (body?: string | Uint8Array) => void }) => void | Promise; }) => (() => void) | undefined; }; }; @@ -174,13 +196,31 @@ export function planNativeDsh(env: NodeJS.ProcessEnv): { mode: "off" | "attach" const state: NativeInterceptState = { origin: undefined, ready: Promise.resolve(undefined) }; -// One registration lifecycle per process. toolsReady is the ONLY gate for -// header stamping (pi.ts discipline): stamped headers flip the proxy into +// One registration lifecycle per HOST CONTEXT. toolsReady is the ONLY gate +// for header stamping (pi.ts discipline): stamped headers flip the proxy into // plugin mode, which suppresses wire tool injection — stamping before the // local tools exist would send a tool-less request. -type RegisterState = { base: string | undefined; toolsReady: boolean; dead: boolean; retryAt: number; pending: Promise | undefined }; +// #3380: registrations live in the host process's tool layer and die ONLY +// with their context (the apply() re-arm below is the sole reset). Proxy +// death merely moves the origin — the execute closures read the LIVE base +// (#983) — so proxy-death paths must NOT clear toolsReady: doing so forced a +// doomed re-registration that DSH rejects ("tool X is already registered"), +// locked the latch false forever, and silently stripped every request of its +// identity headers (wire mode → anonymous zero-fold ledgers → 413 on large +// sessions). +type RegisterState = { + base: string | undefined; + toolsReady: boolean; + dead: boolean; + retryAt: number; + pending: Promise | undefined; + // #3380 P2: consecutive-failure bookkeeping — capped exponential back-off + // (regBackoffMs) + the one-time "restart dsh" notice. + failStreak: number; + lastRegistrationError: string | undefined; +}; -const register: RegisterState = { base: undefined, toolsReady: false, dead: false, retryAt: 0, pending: undefined }; +const register: RegisterState = { base: undefined, toolsReady: false, dead: false, retryAt: 0, pending: undefined, failStreak: 0, lastRegistrationError: undefined }; // #1797: attach/recovery chains still in flight after apply() returned — a late // chain would clobber the shared register mid-run of whatever executes next. @@ -199,6 +239,10 @@ function trackChain(p: Promise): Promise { // single time even though apply() may run again after context re-arming. let webProfileWarned = false; +// #3380: once-per-process — the idempotent-adoption convergence notice must +// not repeat per attempt (same discipline as webProfileWarned). +let regAdoptLogged = false; + /** #1590/#1809: origin of the proxy as currently reachable — register.base * once bound (attach synchronously, spawn after bootstrap), else the preset * env origin. Shared by the index-inject row and the live /bili/origin route. */ @@ -207,6 +251,129 @@ function currentOrigin(): string | undefined { return register.base ?? (envOrigin !== undefined && envOrigin.length > 0 ? envOrigin : undefined); } +/** #3373: the /__bili admin relay — dsh web's remote-panel path. With + * `security.adminRelay` on and the proxy bound, forward browser admin calls + * received on dsh's web entry to the loopback proxy, so a remote browser + * talks same-origin through the entry it already reached dsh by. The + * forwarded hop is a local non-browser client (loopback source, loopback + * Host via fetch, no Origin header), so the admin gate (#1538) holds + * unchanged; when the gate is off this stays a plain 404 (fail-closed). */ +const RELAY_REQUEST_DROP = new Set([ + "connection", "keep-alive", "proxy-authenticate", "proxy-authorization", "te", + "trailer", "transfer-encoding", "upgrade", + // fetch re-derives these from the target; the browser's values describe + // the entry origin and would leak host-session state into the proxy. + "host", "origin", "cookie", "referer", "content-length", "accept-encoding", +]); +const RELAY_RESPONSE_DROP = new Set(["connection", "keep-alive", "transfer-encoding", "content-length", "content-encoding"]); +const RELAY_MAX_BODY_BYTES = 8 * 1024 * 1024; + +type RelayRes = { writeHead: (status: number, headers?: Record) => void; write?: (chunk: string | Uint8Array) => boolean; end: (body?: string | Uint8Array) => void }; + +async function relayAdminRequest(req: unknown, res: RelayRes): Promise { + const fail = (status: number, code: string): void => { + try { + res.writeHead(status, { "content-type": "application/json" }); + res.end(JSON.stringify({ error: code })); + } catch { + // socket already gone — nothing left to report + } + }; + // Fail-closed presentation first: while the gate is off the route answers + // 404 regardless of proxy state (no state disclosure). + if (!adminRelayEnabled()) { + fail(404, "admin relay disabled"); + return; + } + const base = register.base; + if (base === undefined) { + fail(503, "bili proxy not running"); + return; + } + const rawMethod = (req as { method?: unknown }).method; + const method = typeof rawMethod === "string" ? rawMethod : "GET"; + const rawUrl = (req as { url?: unknown }).url; + const url = typeof rawUrl === "string" ? rawUrl : "/"; + // Contain the relay to the declared /__bili surface: fetch() normalizes + // dot segments (literal and %2e-encoded alike), so an unchecked + // "/__bili/.." would land on sibling paths of the loopback proxy. + const cut = url.search(/[?#]/); + const pathPart = cut === -1 ? url : url.slice(0, cut); + let decodedPath: string; + try { + decodedPath = decodeURIComponent(pathPart).replaceAll("\\", "/"); + } catch { + fail(400, "bad request path"); + return; + } + if (decodedPath.split("/").includes("..")) { + fail(400, "bad request path"); + return; + } + const headersIn = (req as { headers?: unknown }).headers; + const headers: Record = {}; + if (headersIn !== null && typeof headersIn === "object") { + for (const [key, value] of Object.entries(headersIn as Record)) { + if (RELAY_REQUEST_DROP.has(key)) continue; + if (typeof value === "string") headers[key] = value; + else if (Array.isArray(value)) headers[key] = value.filter((entry): entry is string => typeof entry === "string").join(", "); + } + } + let body: Uint8Array | undefined; + if (method !== "GET" && method !== "HEAD") { + const chunks: Buffer[] = []; + let size = 0; + try { + for await (const chunk of req as AsyncIterable) { + const buf = typeof chunk === "string" ? Buffer.from(chunk) : chunk instanceof Uint8Array ? Buffer.from(chunk.buffer, chunk.byteOffset, chunk.byteLength) : undefined; + if (buf === undefined) break; + size += buf.length; + if (size > RELAY_MAX_BODY_BYTES) { + fail(413, "request body too large"); + return; + } + chunks.push(buf); + } + body = chunks.length > 0 ? Buffer.concat(chunks) : undefined; + } catch { + fail(400, "request body read failed"); + return; + } + } + let upstream: Response; + try { + upstream = await fetch(`${base}${url}`, { + method, + headers, + body, + redirect: "manual", + signal: AbortSignal.timeout(120_000), + }); + } catch { + fail(502, "bili proxy unreachable"); + return; + } + const headersOut: Record = {}; + upstream.headers.forEach((value, key) => { + if (!RELAY_RESPONSE_DROP.has(key)) headersOut[key] = value; + }); + try { + res.writeHead(upstream.status, headersOut); + if (upstream.body !== null) { + if (typeof res.write === "function") { + for await (const chunk of upstream.body) res.write(chunk); + res.end(); + } else { + res.end(Buffer.from(await upstream.arrayBuffer())); + } + } else { + res.end(); + } + } catch { + // client hung up mid-stream — the upstream read dies with it + } +} + // #2604: host-title channel (mirror of pi.ts #2322/#2354): dsh owns its // session names; the proxy web UI labels a conversation from meta.hostTitle // when present. Pushes are fire-and-forget, deduped per (base, sid) — a @@ -520,9 +687,10 @@ async function verifyAttachAndRecover(attachOrigin: string): Promise { persistClientEvent("proxy respawn gave up — model traffic goes direct (uncompressed)"); + // #3380: only the origin dies here — the host-context tool + // registrations survive, so toolsReady stays put. register.base = undefined; - register.toolsReady = false; }; const landed = start().then((origin) => { if (origin === undefined) { if (landingOwnsRegister(attachOrigin)) { register.base = undefined; - register.toolsReady = false; // A back-off armed against the stale base must die with it: // the unfreeze is a one-shot transition to a fresh base-less // world and must not inherit the old base's 10s wall (#1783). @@ -613,8 +781,27 @@ async function registerTools(ctx: PluginContext): Promise { // flight — a re-activation mid-fetch must not register into (or die // on) the deactivated context captured at call time. const target = activeCtx ?? ctx; - for (const tool of tools) target.tools.register(toolDefinition(tool)); + let adopted = 0; + for (const tool of tools) { + try { + target.tools.register(toolDefinition(tool)); + } catch (err) { + // #3380: idempotent adoption — a name this process ALREADY + // holds in the host tool layer throws "already registered"; + // that registration is exactly what we want, so skip it and + // register the rest (also converges a partial registration + // left by an earlier interrupted flight). + if (!errMessage(err).includes("already registered")) throw err; + adopted += 1; + } + } + if (adopted > 0 && !regAdoptLogged) { + regAdoptLogged = true; + persistClientEvent(`tool registration adopted ${adopted} pre-existing registration(s) from this process — continuing`); + } register.toolsReady = true; + register.failStreak = 0; + register.lastRegistrationError = undefined; })() .catch((err: unknown) => { // Cordis deactivates the plugin context while the host tears down @@ -625,12 +812,22 @@ async function registerTools(ctx: PluginContext): Promise { register.dead = true; return; } + // #3380 P2: consecutive failures escalate — capped exponential + // back-off (a flat wall retried the incident's identical doomed + // failure 145×/h) and a one-time loud notice with the recovery + // action. Retries NEVER stop: a transient fault must still heal + // (#983/#2082 — permanent non-recovery was the original bug class). + register.failStreak += 1; + register.lastRegistrationError = errMessage(err); + if (register.failStreak === REG_GIVE_UP_NOTICE_STREAK) { + persistClientEvent(`bili tools still unavailable after ${register.failStreak} failed registrations (last: ${register.lastRegistrationError}) — model requests ride bili WITHOUT identity headers (wire mode: no compression, anonymous sessions). Restart dsh to restore; retries continue in the background.`); + } // Scope the back-off to the base that actually failed: if the // register moved on while this manifest fetch was in flight (the // apply chain unfroze a dead preset, maybeRetry healed elsewhere), // an armed retryAt would gate the NEXT base's first heal behind a // 10s wall — exactly the windows-22 CI deadlock (#1783). - if (register.base === base) register.retryAt = Date.now() + retryIntervalMs(); + if (register.base === base) register.retryAt = Date.now() + regBackoffMs(register.failStreak); persistClientEvent(`manifest registration failed (${errMessage(err)}) — retrying; requests stay in wire mode until it succeeds`); }) .finally(() => { @@ -679,8 +876,12 @@ function armRecoveryTimer(): void { function maybeRetry(ctx: PluginContext): void { activeCtx = ctx; - if (!register.dead && !register.toolsReady) armRecoveryTimer(); - if (register.dead || register.toolsReady) return; + // #3380: the base-less state heals through the RESPAWN branch below even + // when toolsReady is true (registrations survive proxy death) — gating on + // toolsReady alone would strand a given-up register with no driver left. + if (!register.dead && (register.base === undefined || !register.toolsReady)) armRecoveryTimer(); + if (register.dead) return; + if (register.base !== undefined && register.toolsReady) return; if (register.pending !== undefined) return; if (Date.now() < register.retryAt) return; if (register.base === undefined) { @@ -822,12 +1023,24 @@ function latestSessionNote(status: Record, requestedSid: string async function statusOutcome(ctx: PluginContext, invocation?: CommandInvocation): Promise { const base = register.base; if (!base) { + // #3380 P3: toolsReady implies a proxy WAS serving this session — a + // missing origin is then a transient outage with recovery running, + // not a missing install. return { kind: "error", - text: "bili: no proxy detected — install via `bili plugin install dsh` or launch through `bili dsh`.", + text: register.toolsReady + ? "bili: proxy temporarily unreachable — it was serving this session before and recovery is running in the background; compression resumes automatically when it answers." + : "bili: no proxy detected — install via `bili plugin install dsh` or launch through `bili dsh`.", }; } maybeRetry(ctx); + // #3380 P3: wire-mode exposure — the proxy answers but the local tools are + // down, so every model request rides bili WITHOUT identity headers + // (anonymous zero-fold sessions, no compression). Say so in the panel the + // user already looks at instead of letting it stay silent. + const wireNote = !register.toolsReady && !register.dead + ? `⚠️ bili tools NOT registered — model requests ride bili WITHOUT identity headers (wire mode: no compression, anonymous sessions). Last error: ${register.lastRegistrationError ?? "unknown"}. Restart dsh to restore; retries continue in the background.\n\n` + : ""; // #1677: the command executor hands us the invoking session via the invocation — // DSH never establishes the AsyncLocalStorage boundary on the command path, so // currentInitiator() is always empty here and relying on it made /acp silently @@ -844,7 +1057,7 @@ async function statusOutcome(ctx: PluginContext, invocation?: CommandInvocation) if (status && typeof panel === "string" && panel.length > 0) { // A panel reached through the latest-fallback belongs to some OTHER session — // say so instead of presenting foreign data as the caller's own (#1677). - return { kind: "success", text: fellBackToLatest ? `${latestSessionNote(status, sid)}\n\n${panel}` : panel }; + return { kind: "success", text: `${wireNote}${fellBackToLatest ? `${latestSessionNote(status, sid)}\n\n` : ""}${panel}` }; } // #955: pre-first-request view — the proxy answers from the runtime-info // table this plugin populated at bootstrap, so /acp shows the client's @@ -858,14 +1071,14 @@ async function statusOutcome(ctx: PluginContext, invocation?: CommandInvocation) const version = await fetchProxyVersion(base); return { kind: "success", - text: `billion-context${version ? `@${version}` : ""} — proxy connected, compression armed. Runtime info${typeof ri.source === "string" ? ` (${ri.source})` : ""}: ${parts.join(" ")}. No model request yet; send one, then run /acp again for the full panel.`, + text: `${wireNote}billion-context${version ? `@${version}` : ""} — proxy connected, compression armed. Runtime info${typeof ri.source === "string" ? ` (${ri.source})` : ""}: ${parts.join(" ")}. No model request yet; send one, then run /acp again for the full panel.`, }; } const version = await fetchProxyVersion(base); if (version) { return { kind: "success", - text: `billion-context@${version} — proxy connected, compression armed. No model request seen yet; send one, then run /acp again.`, + text: `${wireNote}billion-context@${version} — proxy connected, compression armed. No model request seen yet; send one, then run /acp again.`, }; } return { @@ -937,6 +1150,13 @@ export function apply(ctx: PluginContext): void { const plan = planNativeDsh(process.env); if (plan.mode === "off") return; + // #3354: stand dsh's native basic compaction down at LOAD TIME — reaches + // the web-profile preset instances no patch layer can address by id + // (#1772/#2360); the host plane already stands down via the bundle patch + // (auto:false). Gated on plan.mode: a kill-switched bili must hand the + // lane back to stock dsh behavior, so this runs only when bili is on. + registerCompactionDisable(ctx); + // #2082: a fresh activation invalidates the "host tearing down" verdict. // Without this reset, ONE registerTools rejection carrying "inactive // context" (a cordis deactivate/reactivate cycle — dsh reloading the @@ -960,6 +1180,12 @@ export function apply(ctx: PluginContext): void { register.retryAt = 0; } register.dead = false; + // #3380: a fresh activation starts a clean registration cycle — stale + // failure bookkeeping from the previous context must not carry over + // (back-off would resume mid-ladder, the notice would stay spent). + register.failStreak = 0; + register.lastRegistrationError = undefined; + regAdoptLogged = false; activeCtx = ctx; // #1590: the dsh web-profile settings panel shows a "bili设置" entry @@ -970,7 +1196,11 @@ export function apply(ctx: PluginContext): void { // a stale link. ctx.on?.("webserver/index-inject", (table: Array<{ kind: string; name?: string; value?: unknown }>) => { const origin = currentOrigin(); - if (origin !== undefined) table.push({ kind: "global", name: "__BILI__", value: { origin } }); + if (origin === undefined) return; + // #3373: with the admin relay armed the settings entry must talk + // same-origin (the entry's origin, relayed by the host) — publish the + // relay marker instead of the loopback origin. + table.push({ kind: "global", name: "__BILI__", value: adminRelayEnabled() ? { origin: null, relay: true } : { origin } }); }); // #2604: follow the host session name (dsh-session-title service). Every @@ -982,7 +1212,7 @@ export function apply(ctx: PluginContext): void { ctx.on("session/created", (session: { id?: unknown }) => { seedDshSessionTitle(session); }); - ctx.on("session/event", (session, event) => { + ctx.on("session/event", (session: { id?: unknown }, event: { type?: unknown; data?: { title?: unknown } }) => { if (event?.type !== "session/title") return; const title = event.data?.title; if (typeof title !== "string" || title.length === 0) return; @@ -1002,11 +1232,20 @@ export function apply(ctx: PluginContext): void { kind: "exact" as const, path: "/bili/origin", handler: (_req: unknown, res: { writeHead: (status: number, headers?: Record) => void; end: (body?: string) => void }): void => { - const body = JSON.stringify({ origin: currentOrigin() ?? null }); + const origin = currentOrigin(); + // #3373: relay mode ⇒ the client derives its base from the probe + // response URL itself; the loopback origin is useless (and leaky) + // to a remote browser. + const body = JSON.stringify(origin !== undefined && adminRelayEnabled() ? { origin: null, relay: true } : { origin: origin ?? null }); res.writeHead(200, { "content-type": "application/json" }); res.end(body); }, }; + const adminRelayRoute = { + kind: "prefix" as const, + path: "/__bili", + handler: (req: unknown, res: RelayRes): void | Promise => relayAdminRequest(req, res), + }; if (typeof ctx.inject === "function") { // Cordis inject: fires once the service is available, re-runs on // change; the effect disposes the prior registration on each re-run @@ -1015,11 +1254,25 @@ export function apply(ctx: PluginContext): void { ctx.inject(["webServer"], (sub) => { const ws = sub.webServer; if (ws === undefined) return; - if (typeof sub.effect === "function") sub.effect(() => ws.register(originRoute), "bili: /bili/origin route"); - else ws.register(originRoute); + if (typeof sub.effect === "function") sub.effect(() => { + const offOrigin = ws.register(originRoute); + const offRelay = ws.register(adminRelayRoute); + // The effect's return value is how dsh disposes this + // registration on re-run — combine both route disposers or + // stale duplicates accumulate in the router table. + return () => { + offOrigin?.(); + offRelay?.(); + }; + }, "bili: panel routes (origin + admin relay)"); + else { + ws.register(originRoute); + ws.register(adminRelayRoute); + } }); } else if (ctx.webServer !== undefined) { ctx.webServer.register(originRoute); + ctx.webServer.register(adminRelayRoute); } if (plan.mode === "attach") { @@ -1039,8 +1292,9 @@ export function apply(ctx: PluginContext): void { state.respawn = start; state.onGiveUp = () => { persistClientEvent("proxy respawn gave up — model traffic goes direct (uncompressed)"); + // #3380: only the origin dies here — the host-context tool + // registrations survive, so toolsReady stays put. register.base = undefined; - register.toolsReady = false; }; // #1365: late routed evidence — if model traffic later arrives baked // against a DIFFERENT origin than the one we attached to, rebind the @@ -1066,8 +1320,9 @@ export function apply(ctx: PluginContext): void { state.respawn = start; state.onGiveUp = () => { persistClientEvent("proxy respawn gave up — model traffic goes direct (uncompressed)"); + // #3380: only the origin dies here — the host-context tool + // registrations survive, so toolsReady stays put. register.base = undefined; - register.toolsReady = false; }; state.ready = trackChain(start()); } @@ -1305,11 +1560,12 @@ export function apply(ctx: PluginContext): void { const target = sessionModelTargetOf(ctx) ?? globalModelTarget(modelInfo.services); refreshModelInfo(register.base, target); const headers: Record = { "x-bili-plugin": "dsh", "x-bili-plugin-conversation": sid }; - // #3370: the incremental-turn marker is no longer stamped here — a - // closure-level armed flag shared by every session raced concurrent - // requests across the awaited beforeSend → headersFor boundary. The - // marker now rides init.headers from the conversion itself (see - // maybeIncrementalNative), atomic with the body swap. + // #3370: the incremental-turn marker (and, since #3329, the turn id) + // is no longer stamped here — a closure-level armed flag shared by + // every session raced concurrent requests across the awaited + // beforeSend → headersFor boundary. Both now ride init.headers from + // the conversion itself (see maybeIncrementalNative), atomic with + // the body swap. if (target !== undefined) { const cached = modelInfo.byKey.get(modelInfoKey(target.provider, target.model)); if (cached !== undefined) { @@ -1336,19 +1592,30 @@ export function apply(ctx: PluginContext): void { // #3248 incremental turns (#3370 hardened): reference wiring for the // native fetch lanes; opencode/omp-native follow the same recipe: // beforeSend — parse the anthropic body, plan (probe-gated), replace + // #3248 incremental turns (#3370 hardened, #3329 attribution): reference + // wiring for the native fetch lanes; opencode/omp-native follow the same + // recipe: + // beforeSend — parse the anthropic body, plan (probe-gated), replace // init.body with the delta payload and stamp the marker - // header on the SAME init (atomic: omp pattern) + stash - // the ack context keyed by URL; - // onResponse — read x-bili-applied-seq and record the ack (or fall - // back to full resend on 409/503). + // + turn-id headers on the SAME init (atomic: omp + // pattern) + stash the ack context keyed by the + // per-request turn id (#3329 — keyed by URL, a late + // response on a shared model endpoint could be + // attributed to whichever conversation planned last); + // onResponse — attribute the response via the turn id THIS dispatch + // carried (threaded from the dispatch's outgoing + // headers), read x-bili-applied-seq, record the ack (or + // fall back to full resend on 409/503). // The old arms-a-shared-flag/consume-it-in-headersFor split raced when two - // sessions crossed the awaited hook boundary: the flag could be consumed - // by the WRONG request, shipping a delta body WITHOUT the marker — the - // proxy forwarded it verbatim and the model answered with no history - // (#3370). init.headers is merged by withHeaders and survives headersFor's - // own stamping, so the marker and the body can never part ways again. + // sessions crossed the awaited hook boundary (#3370) — init.headers is + // merged by withHeaders and survives headersFor's own stamping, so the + // marker and the body can never part ways again. The #3370 single-flight + // guard per URL is KEPT on top of turn-id keying (#3340 rebase decision, + // issuecomment-6212900441): while a marker'd request for this URL is + // still awaiting its ack, later requests on the same URL ride the normal + // pipeline — dropping it is a separate follow-up, not this PR. const INC_PENDING_TTL_MS = 30_000; // matches omp-native / opencode-native - const incPendingAcks = new Map(); + const incPendingAcks = new Map(); async function maybeIncrementalNative(url: string, init: RequestInit | undefined): Promise { if (!register.toolsReady || init === undefined) return; const attr = attributionOf(ctx); @@ -1365,6 +1632,7 @@ export function apply(ctx: PluginContext): void { const base = register.base ?? state.origin; if (base === undefined) return; if (!(await probeIncrementalTurns(base))) return; + sweepStale(incPendingAcks, INC_PENDING_TTL_MS); const sid = attr.sid; const plan = planIncrementalTurn({ conversationId: sid, @@ -1374,21 +1642,29 @@ export function apply(ctx: PluginContext): void { request: parsed as Record, }); if (plan.mode === "skip") return; - // #3370 single-flight (omp/opencode pattern): while a marker'd request - // for this URL is still awaiting its ack, later requests ride the - // normal pipeline — their acks could otherwise collide with the - // pending entry keyed by the same shared model endpoint URL. - const inflight = incPendingAcks.get(url); - if (inflight !== undefined && Date.now() - inflight.at < INC_PENDING_TTL_MS) return; + // #3370 single-flight (omp/opencode pattern; kept over #3329's + // removal per the rebase decision): while a marker'd request for + // this URL is still awaiting its ack, later requests ride the normal + // pipeline — pending entries are now keyed by turn id (#3329), so + // the guard scans for a live entry stashed on THIS url. + let inflight = false; + for (const pending of incPendingAcks.values()) { + if (pending.url === url && Date.now() - pending.at < INC_PENDING_TTL_MS) { inflight = true; break; } + } + if (inflight) return; init.body = (plan as { body: string }).body; + const turnId = newIncrementalTurnId(); const headers = new Headers(init.headers); headers.set(INCREMENTAL_TURN_MARKER_HEADER, "1"); + headers.set(INCREMENTAL_TURN_ID_HEADER, turnId); init.headers = headers; - incPendingAcks.set(url, { sid, messages: parsed.messages as IncrementalWireMessage[], system: "system" in parsed ? parsed.system : undefined, base, at: Date.now() }); + incPendingAcks.set(turnId, { sid, url, messages: parsed.messages as IncrementalWireMessage[], system: "system" in parsed ? parsed.system : undefined, base, at: Date.now() }); } - state.onResponse = (url, response) => { - const pending = incPendingAcks.get(url); - if (pending === undefined) return; + state.onResponse = (_url, response, headers) => { + const turnId = headers?.[INCREMENTAL_TURN_ID_HEADER]; + if (turnId === undefined) return; // not an incremental dispatch — nothing to attribute (#3329) + const pending = incPendingAcks.get(turnId); + if (pending === undefined) return; // unknown/stale turn — nothing to attribute const seq = response.headers.get("x-bili-applied-seq"); if (response.status === 200 && (seq === null || !/^\d+$/.test(seq))) { // 200 without an ack header CANNOT be the marker'd response — the @@ -1399,7 +1675,7 @@ export function apply(ctx: PluginContext): void { // flight, or its ack is lost and the next turn 409s into a resync. return; } - incPendingAcks.delete(url); + incPendingAcks.delete(turnId); if (response.status === 200) { noteIncrementalAck(pending.sid, Number(seq), pending.messages, pending.system); return; @@ -1477,18 +1753,17 @@ export function apply(ctx: PluginContext): void { sweepLiveDshTitles(); } - // #1772/#2474: in profiles bundling @deepseek-ai/dsh-web-app the RUNNING + // #1772/#3354: in profiles bundling @deepseek-ai/dsh-web-app the RUNNING // compaction-basic instance sits inside an agent preset (preset-standard's - // config.plugins). No patch layer can address that NESTED row by id — the - // engine indexes top-level rows only — so the bundled `auto: false` lands - // on web-app's already-disabled host-plane row and the preset instance - // keeps auto-compaction ON. #2474 corrected the earlier "unreachable" - // conclusion: preset-standard itself is an ordinary bundle-layer entry, - // and a profile-layer patch (applied after every bundle) can replace its - // config WHOLESALE with a full snapshot carrying auto:false — so the - // warning names that recipe instead of ending in a dead end. Surface it - // once through the durable channel (GUI hosts swallow stderr); ACP - // compression is unaffected. + // config.plugins) where no patch layer can address the NESTED row by id + // (#1772/#2360). Since #3354 bili disables those preset rows at LOAD TIME + // (registerCompactionDisable above, vendored from + // deshark42/dsh-compaction-basic-disable, MIT), so native auto-compaction + // is off in these profiles too. The durable note confirms the stance and + // keeps the #2474 full-snapshot recipe as the fallback for dsh builds + // whose internal 'internal/config' hook moved. Surface it once through the + // durable channel (GUI hosts swallow stderr); ACP compression is + // unaffected. if (!webProfileWarned) { let bundles: readonly string[] | undefined; try { @@ -1504,7 +1779,7 @@ export function apply(ctx: PluginContext): void { } if (bundles !== undefined && bundles.includes("@deepseek-ai/dsh-web-app")) { webProfileWarned = true; - persistClientEvent("profile bundles include @deepseek-ai/dsh-web-app — its agent presets run their own compaction-basic where the bundle patch cannot set auto:false by id (#1772); dsh native auto-compaction stays enabled in this profile (ACP compression unaffected). To switch it off (#2474): append to your profile's cordis.patch.yml a FULL-SNAPSHOT override of the preset-standard entry (- id: preset-standard + the complete config copied from 'dsh --profile --dump-config', plus config.auto: false under its compaction-basic row) — the patch engine replaces a matched row's config WHOLESALE, so a partial snippet silently drops everything you did not restate (the other plugins AND the preset's id/order identity), and - insert: with the same id appends a dead duplicate row instead of merging; the snapshot also freezes the roster until you re-copy it after a dsh preset update"); + persistClientEvent("profile bundles include @deepseek-ai/dsh-web-app — its agent presets run their own compaction-basic in nested rows no patch layer can address by id (#1772/#2360); bili disables those preset rows at load time via a vendored overlay (deshark42/dsh-compaction-basic-disable, MIT, #3354), so dsh native auto-compaction stays OFF in this profile (ACP compression unaffected). If you still see a dsh compaction checkpoint land on a newer/older dsh build — the overlay rides cordis' INTERNAL 'internal/config' hook and is version-sensitive — fall back to the #2474 full-snapshot override of preset-standard in your profile's cordis.patch.yml (a partial snippet silently drops everything not restated; see CONFIGURATION.md)"); } } @@ -1544,6 +1819,9 @@ export function _resetRegisterForTest(base: string | undefined): void { register.dead = false; register.retryAt = 0; register.pending = undefined; + register.failStreak = 0; + register.lastRegistrationError = undefined; + regAdoptLogged = false; modelInfo.byKey.clear(); modelInfo.refreshingKeys.clear(); modelInfo.retryAtByKey.clear(); @@ -1587,7 +1865,7 @@ export function _stateBeforeSendForTest(): ((url: string, init: RequestInit | un return state.beforeSend; } -export function _stateOnResponseForTest(): ((url: string, response: Response) => void) | undefined { +export function _stateOnResponseForTest(): ((url: string, response: Response, headers?: Record) => void) | undefined { return state.onResponse; } @@ -1604,6 +1882,22 @@ export function _stateRespawnForTest(): (() => Promise) | un return state.respawn; } +/** Test hook (#3380): expose the armed give-up seam so a suite can fire the + * exact callback the interceptor's death branch drives. */ +export function _stateOnGiveUpForTest(): (() => void) | undefined { + return state.onGiveUp; +} + +/** Test hook (#3380): read-only snapshot of the registration lifecycle. */ +export function _registerStateForTest(): { base: string | undefined; toolsReady: boolean; dead: boolean; retryAt: number; failStreak: number; lastRegistrationError: string | undefined } { + return { base: register.base, toolsReady: register.toolsReady, dead: register.dead, retryAt: register.retryAt, failStreak: register.failStreak, lastRegistrationError: register.lastRegistrationError }; +} + +/** Test hook (#3380): the capped exponential registration back-off schedule. */ +export function _regBackoffForTest(failStreak: number): number { + return regBackoffMs(failStreak); +} + /** Test hook (#1365): record routed-channel evidence exactly as the fetch * patch would (the patch is not installed under NODE_TEST_CONTEXT). */ export function _noteRoutedForTest(url: string): void { diff --git a/src/agent/incremental-client.ts b/src/agent/incremental-client.ts index 166a05367d..92a53b21f4 100644 --- a/src/agent/incremental-client.ts +++ b/src/agent/incremental-client.ts @@ -23,6 +23,34 @@ import { createHash } from "node:crypto"; * helper must not import server modules. Keep the two in sync. */ export const INCREMENTAL_TURN_MARKER_HEADER = "x-bili-plugin-turn"; +/** #3329: per-request correlation id stamped on each planned incremental + * turn. The ack channel (native-intercept's onResponse) only carries the URL, + * and several conversations share one provider URL — keying the pending-ack + * table by URL mis-attributes a late response to whichever conversation + * planned last on that URL. The id rides the outgoing request and is read + * back from THAT dispatch's own headers when its response lands, so + * attribution is exact per request. It never travels to the proxy for any + * purpose (the lane ignores it) — pure in-process correlation. */ +export const INCREMENTAL_TURN_ID_HEADER = "x-bili-turn-id"; + +let turnIdCounter = 0; + +/** Fresh per-request turn id: time base + process counter + random tail. + * Uniqueness only needs to hold inside this process (the id never crosses a + * wire boundary for correlation purposes). */ +export function newIncrementalTurnId(): string { + return `${Date.now().toString(36)}-${(turnIdCounter++).toString(36)}-${Math.random().toString(36).slice(2, 10)}`; +} + +/** #3329: drop pending entries older than `ttlMs` — a planned turn whose + * response never comes (network-dead request) must not pin its slot forever. + * Memory hygiene only since attribution is exact per request: correctness no + * longer depends on how long an entry survives. */ +export function sweepStale(map: Map, ttlMs: number): void { + const now = Date.now(); + for (const [key, entry] of map) if (now - entry.at >= ttlMs) map.delete(key); +} + /** Same allowlist the lane accepts in `request` (mirror of * REQUEST_PARAM_KEYS in src/incremental-lane.ts — keep in sync). */ const REQUEST_PARAM_KEYS = [ diff --git a/src/agent/native-intercept.ts b/src/agent/native-intercept.ts index ff951aa59b..4b8020b609 100644 --- a/src/agent/native-intercept.ts +++ b/src/agent/native-intercept.ts @@ -68,9 +68,16 @@ export interface NativeInterceptState { * readable without consuming the stream. Native hosts use it to read * x-bili-applied-seq (the ack channel). Always fires with the * HOST-REQUESTED url (the fetch argument) — never a rerouted or retried - * target: consumers key their in-flight plans by the url they saw in - * beforeSend/the hook. Best-effort: never throws into the request path. */ - onResponse?: (url: string, response: Response) => void; + * target. #3329: the optional third argument carries THIS DISPATCH'S + * outgoing headers (lowercase-name record; init.headers wins over a + * Request-object input's own headers, exactly as fetch itself resolves + * them) — consumers correlate a response back to THE REQUEST that + * produced it via their per-request correlation id stamped on send, + * because several conversations share one provider URL and keying by + * url alone mis-attributes a late response to whichever conversation + * happened to plan last on that URL. Undefined for a dispatch with no + * readable headers. Best-effort: never throws into the request path. */ + onResponse?: (url: string, response: Response, headers?: Record) => void; /** #1117: attribution gate — called per model-API request with the * (pre-rewrite) target URL. Returning false means the caller is NOT the * host itself (e.g. a third-party in-process plugin riding the host's @@ -302,6 +309,23 @@ function withHeaders(input: string | URL | Request, init: RequestInit | undefine return { input, init }; } +/** #3329: the outgoing headers of a dispatch exactly as fetch resolves them — + * init.headers wins over a Request-object input's own headers (spec: init + * overrides), normalized to a lowercase-name record so response-time + * consumers can look up their per-request correlation id without re-deriving + * the merge rules. undefined when no headers exist or none can be read. */ +function outgoingHeadersOf(input: string | URL | Request, init?: RequestInit): Record | undefined { + try { + const src = init?.headers !== undefined ? init.headers : input instanceof Request ? input.headers : undefined; + if (src === undefined) return undefined; + const out: Record = {}; + for (const [k, v] of new Headers(src)) out[k.toLowerCase()] = v; + return out; + } catch { + return undefined; + } +} + /** #2592/#2709: client-side mirror of the passthrough lane's cheap ASCII * prefilter — true when the dispatch body contains DSH_COMPACTION_MARKER_SNIFF * (the exact bytes the server sniffs; imported, not re-derived). A hit is NOT @@ -717,7 +741,7 @@ export function installNativeFetchIntercept(state: NativeInterceptState): boolea state.onDispatch?.(target, target === url ? "self" : "retry"); const resp = await send(stamped.input, stamped.init); try { - state.onResponse?.(url, resp); + state.onResponse?.(url, resp, outgoingHeadersOf(stamped.input, stamped.init)); } catch { // observability hook only — never break the response } @@ -757,7 +781,7 @@ export function installNativeFetchIntercept(state: NativeInterceptState): boolea state.onDispatch?.(retried, "retry"); const resp = await send(restamped.input, restamped.init); try { - state.onResponse?.(url, resp); + state.onResponse?.(url, resp, outgoingHeadersOf(restamped.input, restamped.init)); } catch { // observability hook only — never break the response } @@ -914,7 +938,7 @@ export function installNativeFetchIntercept(state: NativeInterceptState): boolea const stamped = withHeaders(first, init, resignExtra !== undefined ? { ...(state.headersFor?.(url) ?? {}), ...resignExtra } : state.headersFor?.(url)); const resp = await send(stamped.input, stamped.init); try { - state.onResponse?.(url, resp); + state.onResponse?.(url, resp, outgoingHeadersOf(stamped.input, stamped.init)); } catch { // observability hook only — never break the response } @@ -935,7 +959,7 @@ export function installNativeFetchIntercept(state: NativeInterceptState): boolea const stamped = withHeaders(retried, init, resignExtra !== undefined ? { ...(state.headersFor?.(url) ?? {}), ...resignExtra } : state.headersFor?.(url)); const resp = await send(stamped.input, stamped.init); try { - state.onResponse?.(url, resp); + state.onResponse?.(url, resp, outgoingHeadersOf(stamped.input, stamped.init)); } catch { // observability hook only — never break the response } diff --git a/src/agent/omp-native.ts b/src/agent/omp-native.ts index 4c445e98ac..2b959fc47d 100644 --- a/src/agent/omp-native.ts +++ b/src/agent/omp-native.ts @@ -25,7 +25,7 @@ import { ensureProxyRunning, LAUNCHER_DEFAULT_HOST } from "../launcher.js"; import { createBiliPlugin } from "./pi.js"; import { applyOmpFirstEventTimeout, markNativeHost, nativeAttachOrigin, nativeBootstrapGate, nativeProxyScriptPath, proxyEnvOrigin, setNativeOriginWaiter, singleFlight } from "./native-bootstrap.js"; import { installNativeFetchIntercept, noteRoutedOrigin, observeRoutedOrigin, readyOrigin, type NativeInterceptState } from "./native-intercept.js"; -import { INCREMENTAL_TURN_MARKER_HEADER, noteIncrementalAck, noteIncrementalRejection, planIncrementalTurn, probeIncrementalTurns, clearIncrementalProbe, wireMessagesLookAnthropic, type IncrementalWireMessage } from "./incremental-client.js"; +import { clearIncrementalProbe, INCREMENTAL_TURN_MARKER_HEADER, INCREMENTAL_TURN_ID_HEADER, newIncrementalTurnId, noteIncrementalAck, noteIncrementalRejection, planIncrementalTurn, probeIncrementalTurns, sweepStale, wireMessagesLookAnthropic, type IncrementalWireMessage } from "./incremental-client.js"; import { fetchProxyVersion, waitForProxyVersion } from "./shared.js"; import { logHostEvent } from "./host-log.js"; @@ -73,24 +73,35 @@ const state: NativeInterceptState = { origin: undefined, ready: Promise.resolve( // `prompt_cache_key` body field the omp extension stamps (pi.ts // stampPromptCacheKey — present exactly when the plugin is active). The // beforeSend hook reads it as the conversation id, plans the incremental -// body, swaps init.body and stamps the plugin headers DIRECTLY ON init -// (per-request, so concurrent sessions on one provider URL cannot cross -// attributions) — plugin mode is safe here: x-bili-plugin-conversation gets -// the same value wire mode already resolves to via -// preferPromptCacheKeyIdentity, so the session id does not fork. One planned -// in-flight request per URL (onResponse acks are URL-keyed); a second -// concurrent request on the same URL falls back to full resend. +// body, swaps init.body and stamps the plugin headers DIRECTLY ON init — +// plugin mode is safe here: x-bili-plugin-conversation gets the same value +// wire mode already resolves to via preferPromptCacheKeyIdentity, so the +// session id does not fork. #3329: each planned turn is keyed by a fresh +// per-request turn id stamped onto init's headers (x-bili-turn-id) — +// several sessions share one provider URL, and keying the pending table by +// URL mis-attributes a late response to whichever session planned last on +// that URL. The #3378 single-flight guard per URL is KEPT on top of turn-id +// keying (#3340 rebase decision): while a marker'd request for this URL is +// still awaiting its ack, later requests on the same URL ride the normal +// pipeline — attribution answers "which ack consumes which pending", the +// guard answers "no second conversion on this URL until the first is acked +// or ages out"; dropping the guard is a separate follow-up. interface IncrementalPendingTurn { sid: string; + url: string; messages: IncrementalWireMessage[]; system: unknown; base: string; at: number; } +/** Keyed by per-request turn id (#3329); each entry remembers the URL it + * was planned on so the #3378 single-flight guard can scan for live + * same-URL entries. */ const incPendingOmp = new Map(); /** Planned turns age out (onResponse only fires on successful sends — a - * network-dead request must not block the URL forever). Mirrors opencode. */ + * network-dead request must not pin its slot forever). Memory hygiene only + * (#3329): attribution is exact per turn id. Mirrors opencode. */ const INC_PENDING_TTL_MS = 30_000; /** Test hook: drop all planned turns (module state across suites). */ @@ -101,9 +112,7 @@ export function __resetOmpIncrementalForTests(): void { export async function ompIncrementalBeforeSend(url: string, init: RequestInit | undefined): Promise { try { if (init === undefined || typeof init.body !== "string") { return; } - const stale = incPendingOmp.get(url); - if (stale !== undefined && Date.now() - stale.at < INC_PENDING_TTL_MS) { return; } - incPendingOmp.delete(url); + sweepStale(incPendingOmp, INC_PENDING_TTL_MS); let parsed: unknown; try { parsed = JSON.parse(init.body); @@ -120,13 +129,26 @@ export async function ompIncrementalBeforeSend(url: string, init: RequestInit | if (!(await probeIncrementalTurns(base))) { return; } const plan = planIncrementalTurn({ conversationId: sid, path: "/v1/messages", messages: b.messages as IncrementalWireMessage[], system: b.system as unknown, request: b as { model?: unknown } & Record }); if (plan.mode === "skip") { return; } + // #3378 single-flight (kept over #3329's removal per the rebase + // decision): while a marker'd request for this URL is still awaiting + // its ack, later requests ride the normal pipeline. Pending entries + // are keyed by turn id (#3329), so the guard scans for a live entry + // stashed on THIS url (sweepStale above already aged out the stale + // ones — an aged entry must not block a fresh conversion). + let inflight = false; + for (const pending of incPendingOmp.values()) { + if (pending.url === url && Date.now() - pending.at < INC_PENDING_TTL_MS) { inflight = true; break; } + } + if (inflight) { return; } init.body = plan.body; const headers = new Headers(init.headers); headers.set("x-bili-plugin", "omp"); headers.set("x-bili-plugin-conversation", sid); headers.set(INCREMENTAL_TURN_MARKER_HEADER, "1"); + const turnId = newIncrementalTurnId(); + headers.set(INCREMENTAL_TURN_ID_HEADER, turnId); init.headers = headers; - incPendingOmp.set(url, { sid, messages: b.messages as IncrementalWireMessage[], system: b.system as unknown, base, at: Date.now() }); + incPendingOmp.set(turnId, { sid, url, messages: b.messages as IncrementalWireMessage[], system: b.system as unknown, base, at: Date.now() }); } catch (e) { // never break the host's request on adapter failure } @@ -134,9 +156,11 @@ export async function ompIncrementalBeforeSend(url: string, init: RequestInit | state.beforeSend = ompIncrementalBeforeSend; -export function ompIncrementalOnResponse(url: string, response: Response): void { - const pending = incPendingOmp.get(url); - if (pending === undefined) return; +export function ompIncrementalOnResponse(_url: string, response: Response, headers?: Record): void { + const turnId = headers?.[INCREMENTAL_TURN_ID_HEADER]; + if (turnId === undefined) return; // not an incremental dispatch — nothing to attribute (#3329) + const pending = incPendingOmp.get(turnId); + if (pending === undefined) return; // unknown/stale turn — nothing to attribute const seq = response.headers.get("x-bili-applied-seq"); if (response.status === 200 && (seq === null || !Number.isFinite(Number(seq)))) { // 200 without an ack header CANNOT be the marker'd response — the @@ -146,7 +170,7 @@ export function ompIncrementalOnResponse(url: string, response: Response): void // keep the pending entry for the marker'd response still in flight. return; } - incPendingOmp.delete(url); + incPendingOmp.delete(turnId); try { if (response.status === 200) { noteIncrementalAck(pending.sid, Number(seq), pending.messages, pending.system); diff --git a/src/agent/opencode-native.ts b/src/agent/opencode-native.ts index 46164f0637..62818d15ee 100644 --- a/src/agent/opencode-native.ts +++ b/src/agent/opencode-native.ts @@ -77,7 +77,7 @@ import { ensureProxyRunning, LAUNCHER_DEFAULT_HOST, unwrapUpstream, wrapUpstream import { createAcpCommandHooks, showAcpText } from "./opencode-acp-command.js"; import { markNativeHost, nativeAttachOrigin, nativeBootstrapGate, nativeProxyScriptPath, proxyEnvOrigin, singleFlight } from "./native-bootstrap.js"; import { createLiveOriginResolver, installNativeFetchIntercept, isModelApiUrl, noteRoutedOrigin, observeRoutedOrigin, readyOrigin, replaceRequestPayload, replaceRequestTarget, routedBiliModelUrl, type LiveOriginResolverDeps, type NativeInterceptState } from "./native-intercept.js"; -import { INCREMENTAL_TURN_MARKER_HEADER, noteIncrementalAck, noteIncrementalRejection, planIncrementalTurn, probeIncrementalTurns, clearIncrementalProbe, wireMessagesLookAnthropic, type IncrementalWireMessage } from "./incremental-client.js"; +import { clearIncrementalProbe, INCREMENTAL_TURN_MARKER_HEADER, INCREMENTAL_TURN_ID_HEADER, newIncrementalTurnId, noteIncrementalAck, noteIncrementalRejection, planIncrementalTurn, probeIncrementalTurns, sweepStale, wireMessagesLookAnthropic, type IncrementalWireMessage } from "./incremental-client.js"; import { createOpencodeV2Setup, type V2HttpRequestEvent, type V2State } from "./opencode-v2.js"; import { asciiHeaderValue, fetchProxyVersion, postIdentityRegister, reportRuntimeInfoOnChange, waitForProxyVersion, type ForwardedToolResult } from "./shared.js"; import { callLegacyAcpConfig, isLegacyAcpSession, loadLegacyAcp, type LegacyAcpModule } from "./opencode-legacy.js"; @@ -832,10 +832,17 @@ export default { id: "billion-context-opencode-native", setup: createOpencodeV2S * http.request hook (routing + header stamping already happened, so the * stamped plugin headers ride the SAME request object we replace). The * conversation id is opencode's own session id (x-bili-plugin-conversation - * was just stamped with it — identity upgrades, never forks). One planned - * in-flight request per URL: concurrent sessions on the same provider fall - * back to full resend (ack attribution must stay unambiguous — onResponse - * is keyed by URL). Exported for the native suites; wired above. */ + * was just stamped with it — identity upgrades, never forks). #3329: each + * planned turn is keyed by a fresh per-request turn id stamped onto the + * outgoing request (x-bili-turn-id) — several sessions share one provider + * URL, and keying by URL mis-attributes a late response to whichever + * session planned last on that URL. The #3378 single-flight guard per URL + * is KEPT on top of turn-id keying (#3340 rebase decision): while a + * marker'd request for this URL is still awaiting its ack, later requests + * on the same URL ride the normal pipeline — attribution answers "which + * ack consumes which pending", the guard answers "no second conversion on + * this URL until the first is acked or ages out"; dropping the guard is a + * separate follow-up. Exported for the native suites; wired above. */ export async function maybeIncrementalOpencode(e: V2HttpRequestEvent, s: V2State): Promise { try { const sid = typeof e.sessionID === "string" ? e.sessionID : ""; @@ -843,9 +850,7 @@ export async function maybeIncrementalOpencode(e: V2HttpRequestEvent, s: V2State const req = e.request as Request | undefined; if (!sid || !base || req === undefined || typeof req.clone !== "function") return; const url = req.url; - const stale = incPendingOpencode.get(url); - if (stale !== undefined && Date.now() - stale.at < INC_PENDING_TTL_MS) return; // single in-flight per URL - incPendingOpencode.delete(url); + sweepStale(incPendingOpencode, INC_PENDING_TTL_MS); const body: unknown = await req.clone().json().catch(() => null); if (body === null || typeof body !== "object") return; const b = body as { messages?: unknown; model?: unknown; system?: unknown }; @@ -853,15 +858,31 @@ export async function maybeIncrementalOpencode(e: V2HttpRequestEvent, s: V2State if (!(await probeIncrementalTurns(base))) return; const plan = planIncrementalTurn({ conversationId: sid, path: "/v1/messages", messages: b.messages as IncrementalWireMessage[], system: b.system as unknown, request: b as { model?: unknown } & Record }); if (plan.mode === "skip") return; - replaceRequestPayload(e, plan.body, { [INCREMENTAL_TURN_MARKER_HEADER]: "1" }); - incPendingOpencode.set(url, { sid, messages: b.messages as IncrementalWireMessage[], system: b.system as unknown, base, at: Date.now() }); + // #3378 single-flight (kept over #3329's removal per the rebase + // decision): while a marker'd request for this URL is still awaiting + // its ack, later requests ride the normal pipeline. Pending entries + // are keyed by turn id (#3329), so the guard scans for a live entry + // stashed on THIS url (sweepStale above already aged out the stale + // ones — an aged entry must not block a fresh conversion). + let inflight = false; + for (const pending of incPendingOpencode.values()) { + if (pending.url === url && Date.now() - pending.at < INC_PENDING_TTL_MS) { inflight = true; break; } + } + if (inflight) return; + const turnId = newIncrementalTurnId(); + replaceRequestPayload(e, plan.body, { [INCREMENTAL_TURN_MARKER_HEADER]: "1", [INCREMENTAL_TURN_ID_HEADER]: turnId }); + incPendingOpencode.set(turnId, { sid, url, messages: b.messages as IncrementalWireMessage[], system: b.system as unknown, base, at: Date.now() }); } catch { // never break the host's request on adapter failure } } interface IncrementalPendingTurn { + /** Conversation (opencode session id) this planned turn belongs to. */ sid: string; + /** Model-endpoint URL this turn was planned on — carried for the #3378 + * single-flight scan; attribution itself is by turn id (#3329). */ + url: string; messages: IncrementalWireMessage[]; system: unknown; base: string; @@ -869,11 +890,14 @@ interface IncrementalPendingTurn { } /** Planned turns age out: onResponse only fires on a SUCCESSFUL send, so a - * request that died network-level would otherwise block incremental for its - * URL forever (single in-flight guard). 30s covers any sane model round-trip - * while keeping concurrent-request attribution unambiguous. */ + * request that died network-level would otherwise pin its slot forever. + * Memory hygiene only (#3329) — attribution is exact per turn id, so a + * lingering entry can no longer mis-attribute another request's ack. */ const INC_PENDING_TTL_MS = 30_000; +/** Keyed by per-request turn id (#3329); each entry remembers the URL it + * was planned on so the #3378 single-flight guard can scan for live + * same-URL entries — see maybeIncrementalOpencode. */ const incPendingOpencode = new Map(); /** Test hook: drop all planned turns (the map is module state across suites). */ @@ -882,11 +906,17 @@ export function __resetOpencodeIncrementalForTests(): void { } /** Ack collector registered on the module's NativeInterceptState: the - * model-API and routed branches both fire onResponse(url, resp) after a - * successful send. Exported for the native suites; wired below. */ -export function opencodeIncrementalOnResponse(url: string, response: Response): void { - const pending = incPendingOpencode.get(url); - if (pending === undefined) return; // non-incremental responses just clear nothing + * model-API and routed branches both fire onResponse(url, resp, headers) + * after a successful send. #3329: attribution is by the per-request turn id + * read back from THIS dispatch's outgoing headers (third argument) — the + * url is only ever shared context, never the key. A response without a + * turn-id header is not an incremental ack at all. Exported for the native + * suites; wired below. */ +export function opencodeIncrementalOnResponse(_url: string, response: Response, headers?: Record): void { + const turnId = headers?.[INCREMENTAL_TURN_ID_HEADER]; + if (turnId === undefined) return; // not an incremental dispatch — nothing to attribute (#3329) + const pending = incPendingOpencode.get(turnId); + if (pending === undefined) return; // unknown/stale turn — nothing to attribute const seq = response.headers.get("x-bili-applied-seq"); if (response.status === 200 && (seq === null || !Number.isFinite(Number(seq)))) { // 200 without an ack header CANNOT be the marker'd response — the @@ -896,7 +926,7 @@ export function opencodeIncrementalOnResponse(url: string, response: Response): // keep the pending entry for the marker'd response still in flight. return; } - incPendingOpencode.delete(url); + incPendingOpencode.delete(turnId); try { if (response.status === 200) { noteIncrementalAck(pending.sid, Number(seq), pending.messages, pending.system); diff --git a/src/agent/opencode-v2.ts b/src/agent/opencode-v2.ts index 6462c3b7ec..c29c882972 100644 --- a/src/agent/opencode-v2.ts +++ b/src/agent/opencode-v2.ts @@ -48,7 +48,7 @@ // openai tool list, src/compress-tool.ts) because reload-based refresh is not // available on all observed surfaces. -import { ACP_TOOLS_OPENAI } from "../compress-tool.js"; +import { ACP_TOOLS_OPENAI, ABSORB_TOOL_OPENAI } from "../compress-tool.js"; import { asciiHeaderValue, fetchManifest, fetchProxyVersion, fetchStatus, fitNoticeDescription, forwardTool, postIdentityRegister, proxyBaseFromEnv, proxyBaseFromUrl, reportCompactionBoundary, reportRuntimeInfoOnChange, V2_SYNTHETIC_TEXT } from "./shared.js"; import { createForkAdopter } from "./fork-adopt.js"; import { logHostEvent } from "./host-log.js"; @@ -140,7 +140,7 @@ const TOOL_DELTA_RETRY_MS = 15000; // Kill switch = fully inert (same semantics as detectProxyBase): gates header stamping, tool forwarding, compaction reporting. const pluginDisabled = (): boolean => process.env.BILLION_CONTEXT_PLUGIN === "0"; -const V2_BILI_TOOLS = ACP_TOOLS_OPENAI.map((t) => ({ +const V2_BILI_TOOLS = [...ACP_TOOLS_OPENAI, ABSORB_TOOL_OPENAI].map((t) => ({ name: t.function.name, description: t.function.description, input: t.function.parameters, diff --git a/src/agent/pi.ts b/src/agent/pi.ts index e3fb403704..293c0c5414 100644 --- a/src/agent/pi.ts +++ b/src/agent/pi.ts @@ -20,6 +20,7 @@ import { createForkAdopter } from "./fork-adopt.js"; import { logHostEvent } from "./host-log.js"; import { plainHttpTrusted } from "../private-host.js"; import { allowPrivateHttpProviders } from "../knobs.js"; +import { hostProfileFor } from "../host-profiles.js"; export type Ctx = { sessionManager?: { getSessionId?: () => string; getHeader?: () => unknown; getBranch?: () => unknown } | undefined; @@ -445,7 +446,12 @@ function noteOmpCoworkBypass(modelId: string): void { // proxy's anonymous prefix-affinity sessions (#309): compression still works, // /acp lookup by session id does not — acceptable vs a guaranteed 400. export function stampPromptCacheKey(event: unknown, ctx: Ctx, agent: string): Record | undefined { - if (agent !== "omp") return undefined; + // #3482 site 3: the identity-stamp CHANNEL is a registry property — omp + // stamps the body pck, pi stamps x-bili-plugin-conversation in + // before_provider_headers (which outranks the body field). No adapter-side + // name gate. + const profile = hostProfileFor(agent); + if (!profile.stampsBodyPromptCacheKey) return undefined; if (!destinationRoutedThroughProxy(ctx.model?.baseUrl)) { // #1579: native omp mode routes at the FETCH layer — the model // baseUrl stays the real upstream, so the launcher-shaped checks @@ -483,7 +489,9 @@ export function stampPromptCacheKey(event: unknown, ctx: Ctx, agent: string): Re // stamp where the transport is proven direct: those requests go // direct anyway (uncompressed, anonymous to the proxy) — soft // identity loss instead of a guaranteed hard failure (#1403). - if (ompCoworkTransportBypassesFetch(ctx.model, process.env)) { + // #3482 site 2: whether that transport is even POSSIBLE is a registry + // property; the env-observation below is the per-request truth. + if (profile.coworkTransportPossible && ompCoworkTransportBypassesFetch(ctx.model, process.env)) { noteOmpCoworkBypass(ctx.model?.id ?? base); return undefined; } diff --git a/src/cli.ts b/src/cli.ts index 9b3ee461fe..88245b6d57 100644 --- a/src/cli.ts +++ b/src/cli.ts @@ -125,7 +125,7 @@ Launcher (bili pi / bili codex / bili claude / bili omp / bili opencode / bili h bili gemini # launch Gemini CLI through the proxy (GOOGLE_GEMINI_BASE_URL /bili/ rewrite; API-key & gateway auth) bili iflow # launch iFlow CLI through the proxy (IFLOW_BASE_URL /bili/ rewrite of apis.iflow.cn/v1) bili qwen # launch Qwen Code through the proxy (cert-MITM; DashScope/Qwen gateways whitelisted by default, custom relays via --mitm-domain) - bili antigravity # launch Antigravity CLI (agy) through the proxy (CLOUD_CODE_URL /bili/ rewrite of cloudcode-pa.googleapis.com) + bili antigravity # launch Antigravity CLI (agy) through the proxy (CLOUD_CODE_URL /bili/ rewrite of daily-cloudcode-pa.googleapis.com — real default upstream, #3129) bili mcode # launch MiniMax Code through the proxy (cert-MITM; provider hosts from ~/.minimax*/config.yaml or the official agent.minimax.* endpoints) bili aider # launch aider through the proxy (cert-MITM; endpoint from OPENAI_API_BASE/--openai-api-base/.aider.conf.yml or api.openai.com+api.anthropic.com by default) bili copilot # launch Copilot CLI through the proxy (cert-MITM; api.githubcopilot.com + plan subdomains whitelisted by default) diff --git a/src/compress-loop-responses.ts b/src/compress-loop-responses.ts index fdfbcf80bd..0d8620886f 100644 --- a/src/compress-loop-responses.ts +++ b/src/compress-loop-responses.ts @@ -6,7 +6,9 @@ import { import { lastCompressSuffix, withSessionLock, type Session } from "./session.js"; import { extractResponsesTextTriggers, PROXY_TOOL_NAMES, MUTATING_PROXY_TOOLS } from "./compress-tool.js"; import { log as loggerLog } from "./logger.js"; -import { drainPendingRetrievals } from "./store.js"; +import { drainPendingRetrievals, imageRescaleToolArmed } from "./store.js"; +import { imageRescaleMaxDim } from "./knobs.js"; +import { applyImageRescaleItems } from "./image-rescale.js"; import { executeProxyToolAsync, buildVisibilityMarker } from "./loop/core.js"; import { hoistTrappedToolItems } from "./tool-pair-order.js"; import { mergeAdjacentConfigurationUpdates } from "./responses-tool-output.js"; @@ -169,7 +171,18 @@ export async function compressLoopResponsesJson( } return current; } - const inputItems: ResponseInputItem[] = Array.isArray(requestBody.input) ? [...requestBody.input] : []; + let inputItems: ResponseInputItem[] = Array.isArray(requestBody.input) ? [...requestBody.input] : []; + // #3400: the loop re-request resends the client's RAW input items (big + // images intact) — run the same egress pass the steady path applies so + // the upstream keeps seeing the identical downscaled bytes (the shared + // encode cache makes this byte-stable across rounds). + inputItems = (await applyImageRescaleItems(inputItems, { + session: ctx.session, + sessionId: ctx.session.id, + log: (lvl, msg) => ctx.log(`[${lvl}] ${msg}`), + maxDim: imageRescaleMaxDim(), + toolArmed: imageRescaleToolArmed(ctx.session), + })) as ResponseInputItem[]; if (extracted.clean.trim()) { inputItems.push({ type: "message", role: "assistant", content: [{ type: "output_text", text: extracted.clean }] }); } diff --git a/src/compress-settings.ts b/src/compress-settings.ts index 06fa06e538..65b74b5cfa 100644 --- a/src/compress-settings.ts +++ b/src/compress-settings.ts @@ -1,4 +1,4 @@ -import { DEFAULT_CCR_CONFIG, defaultPrompts, resolvePrompts, createPackResolver, defaultPackSources, isValidPackName, type Config, type CcrConfig, type PackSurface, type Prompts } from "acp-kernel"; +import { DEFAULT_ABSORB_CONFIG, DEFAULT_CCR_CONFIG, defaultPrompts, resolvePrompts, createPackResolver, defaultPackSources, isValidPackName, type AbsorbConfig, type Config, type CcrConfig, type PackSurface, type Prompts } from "acp-kernel"; import * as path from "node:path"; import { findRoute, type CompressSettings, type NamedProviderRecipe, type ProviderRoutes } from "./config.js"; import { configDir } from "./paths.js"; @@ -52,6 +52,10 @@ export function mergeCompress( // value still beats a global-level new-name value. The merged output // always carries the canonical name only. const rangeOf = (s?: CompressSettings): number | undefined => s?.minCompressRangeChars ?? s?.minCompressRange; + // `absorb` is a second nested-object field, merged sub-field-wise exactly + // like `prompts`: a model-level minToolTokens must not discard a + // provider-level excludeTools. + const absorbLevels = [global?.absorb, provider?.absorb, model?.absorb].filter(Boolean) as NonNullable[]; const ccrLevels = [global?.ccr, provider?.ccr, model?.ccr].filter(Boolean) as NonNullable[]; const searchLevels = [global?.search, provider?.search, model?.search].filter(Boolean) as NonNullable[]; const reasoningLevels = [global?.reasoning, provider?.reasoning, model?.reasoning].filter(Boolean) as NonNullable[]; @@ -59,7 +63,7 @@ export function mergeCompress( const outputSteeringLevels = [global?.outputSteering, provider?.outputSteering, model?.outputSteering].filter(Boolean) as NonNullable[]; const priceProfileLevels = [global?.priceProfile, provider?.priceProfile, model?.priceProfile].filter(Boolean) as NonNullable[]; // `tierNudgeTokens` is another nested-object field merged sub-field-wise - // like `prompts`: a model-level t2 must not discard a provider-level t1. + // like `absorb`: a model-level t2 must not discard a provider-level t1. const tierNudgeLevels = [global?.tierNudgeTokens, provider?.tierNudgeTokens, model?.tierNudgeTokens].filter(Boolean) as NonNullable[]; return { modelContextLimit: pick("modelContextLimit"), @@ -81,6 +85,7 @@ export function mergeCompress( preserveRecentTools: pick("preserveRecentTools"), prompts: promptLevels.length > 0 ? Object.assign({}, ...promptLevels) : undefined, acknowledgePromptsRisk: pick("acknowledgePromptsRisk"), + absorb: absorbLevels.length > 0 ? Object.assign({}, ...absorbLevels) : undefined, ccr: ccrLevels.length > 0 ? Object.assign({}, ...ccrLevels) : undefined, search: searchLevels.length > 0 ? Object.assign({}, ...searchLevels) : undefined, rules: pick("rules"), @@ -88,7 +93,7 @@ export function mergeCompress( visibilityMarkers: pick("visibilityMarkers"), streamSummary: pick("streamSummary"), // `reasoning` is a third nested-object field merged sub-field-wise - // exactly like `prompts`: a model-level `threshold` must not + // exactly like `absorb`/`prompts`: a model-level `threshold` must not // discard a provider-level `drop: false`. reasoning: reasoningLevels.length > 0 ? Object.assign({}, ...reasoningLevels) : undefined, reasoningGuard: reasoningGuardLevels.length > 0 ? Object.assign({}, ...reasoningGuardLevels) : undefined, @@ -237,6 +242,10 @@ export function hasCompressSettings(s: CompressSettings): boolean { * "read"]` is the one-entry #1198/#1277 remedy, no built-in list * restating/freezing. Whole-array replace, deepest level wins; unset * passes `base.preserveRecentTools` through. + * - `absorb` → `absorb` (kernel AbsorbConfig; unset fields inherit the + * kernel DEFAULT_ABSORB_CONFIG, so a partial user block still resolves + * fully). Absent `s.absorb` leaves `base.absorb` untouched — the feature + * stays off unless some level enables it. * - `rules` → `rules = { enabled }` (kernel RuleFeatureConfig; limits stay * at kernel defaults). Absent `s.rules` leaves `base.rules` untouched. * - `ccr` → `ccr` (kernel CcrConfig, acp-kernel >= 0.0.84; the kernel runs @@ -255,6 +264,24 @@ export function hasCompressSettings(s: CompressSettings): boolean { // (validateConfig does not flag unknown keys). export type ResolvedKernelConfig = Config & { preserveRecentTools?: string[]; externalSummary?: ExternalSummarySettings }; +// Single source of truth for raw->resolved absorb: applyCompressSettings and the +// plugin-lane base stamp (#1359) both go through this or they drift apart. +export function resolveAbsorbSettings(s: CompressSettings["absorb"]): AbsorbConfig | undefined { + if (s === undefined) return undefined; + const d = DEFAULT_ABSORB_CONFIG; + return { + enabled: s.enabled === true, + toolName: s.toolName ?? d.toolName, + minToolTokens: s.minToolTokens ?? d.minToolTokens, + contextThresholdPct: s.contextThresholdPct !== undefined ? parsePercent(s.contextThresholdPct) : d.contextThresholdPct, + excludeTools: s.excludeTools ?? [...d.excludeTools], + // #3402: bili's default stance is media-scoped — the revival exists to fix + // image pile-up (#3398) and to kill the old full-text over-compression. The + // kernel ships mediaOnly neutral (unset); this is the single place the + // default-on policy is applied. Explicit false restores pre-revival full-text. + mediaOnly: s.mediaOnly ?? true, + }; +} export function applyCompressSettings(base: Config, limit: number, s: CompressSettings, recipes: Record = {}): ResolvedKernelConfig { const nudge = { ...base.nudge }; @@ -281,6 +308,7 @@ export function applyCompressSettings(base: Config, limit: number, s: CompressSe } const tiers = { ...base.tiers }; if (s.tiers !== undefined) tiers.enabled = s.tiers; + const absorb = resolveAbsorbSettings(s.absorb); // #1207 owner decision: CCR is opt-in on every lane — no default-on // else-branch; an unset `s.ccr` leaves `base.ccr` untouched (off). let ccr: CcrConfig | undefined; @@ -321,6 +349,7 @@ export function applyCompressSettings(base: Config, limit: number, s: CompressSe // HERE, at resolve time — the executor still receives a fully // expanded ExternalSummarySettings and never sees references. externalSummary: s.externalSummary !== undefined ? expandExternalSummaryChainTolerant(s.externalSummary, recipes) : (base as ResolvedKernelConfig).externalSummary, + ...(absorb !== undefined ? { absorb } : {}), ...(ccr !== undefined ? { ccr } : {}), ...(s.rules !== undefined ? { rules: { enabled: s.rules === true } } : {}), }; diff --git a/src/compress-tool.ts b/src/compress-tool.ts index 1b391c53d8..e679fdf047 100644 --- a/src/compress-tool.ts +++ b/src/compress-tool.ts @@ -12,6 +12,9 @@ */ import { parseCompressArgs, + ABSORB_TOOL, + ABSORB_TOOL_GOOGLE, + ABSORB_TOOL_OPENAI, DECOMPRESS_TOOL, DECOMPRESS_TOOL_GOOGLE, DECOMPRESS_TOOL_NAME, @@ -85,9 +88,14 @@ export { buildCompressSystemPrompt, buildCompressTextSystemPrompt, buildCompressHybridSystemPrompt, + ABSORB_TOOL_NAME, + ABSORB_TOOL, + ABSORB_TOOL_OPENAI, + ABSORB_TOOL_GOOGLE, + buildAbsorbSystemPrompt, RULE_TOOL_NAME, } from "acp-kernel"; -export type { ParsedRange } from "acp-kernel"; +export type { ParsedRange, AbsorbConfig } from "acp-kernel"; export { ACP_TOOL_NAMES as PROXY_TOOL_NAMES, ACP_MUTATING_TOOLS as MUTATING_PROXY_TOOLS, ACP_READONLY_TOOLS as READONLY_PROXY_TOOLS } from "acp-kernel"; // #1881: the ACP-TAGS NEVER-echo prohibition must not disappear with injectTool. @@ -328,6 +336,29 @@ export function ptcNoteMode(names: Iterable): AcpNoteMode { return "none"; } +// The kernel ships no Responses-format absorb const (the four ACP tools have +// *_RESPONSES variants; absorb is host-registered opt-in). Synthesize it in +// the same flat shape as SEARCH_CONTEXT_TOOL_RESPONSES. +export const ABSORB_TOOL_RESPONSES = { + type: "function", + name: ABSORB_TOOL_OPENAI.function.name, + description: ABSORB_TOOL_OPENAI.function.description, + parameters: ABSORB_TOOL_OPENAI.function.parameters, +}; + +// #1359: absorb wire shapes parameterized by name so registration (manifest + +// per-request injection) follows the resolved `absorb.toolName` in BOTH lanes — +// a renamed tool must be advertised, injected, and adjudicated under one name. +// At the default name this is byte-identical to the static consts above. +export function absorbToolsFor(name: string) { + return { + anthropic: { name, description: ABSORB_TOOL.description, input_schema: ABSORB_TOOL.input_schema }, + openai: { type: "function" as const, function: { name, description: ABSORB_TOOL_OPENAI.function.description, parameters: ABSORB_TOOL_OPENAI.function.parameters } }, + responses: { type: "function" as const, name, description: ABSORB_TOOL_OPENAI.function.description, parameters: ABSORB_TOOL_OPENAI.function.parameters }, + google: { name, description: ABSORB_TOOL_GOOGLE.description, parameters: ABSORB_TOOL_GOOGLE.parameters }, + }; +} + // The reconciled kernel (acp-kernel#332) ships RULE_TOOL_NAME + the rule state // helpers but no wire tool objects. Synthesize all four wire shapes here so // every injection point (wire helpers, plugin manifest) serves one definition. @@ -345,6 +376,26 @@ export const RULE_TOOL_OPENAI = { type: "function" as const, function: { name: R export const RULE_TOOL_RESPONSES = { type: "function" as const, name: RULE_TOOL_NAME, description: RULE_TOOL_DESCRIPTION, parameters: RULE_PARAM_SCHEMA }; export const RULE_TOOL_GOOGLE = { name: RULE_TOOL_NAME, description: RULE_TOOL_DESCRIPTION, parameters: RULE_PARAM_SCHEMA }; +// #3400: read_original_image — fetch the ORIGINAL full-resolution bytes of an +// image whose egress copy was downscaled by the rescale pass (the [bili] note +// beside the shrunken image quotes the mNNNNN ref). Host-side execution only +// (never forwarded to the client — see isProxyToolFor); registration is +// lane-gated at the manifest sites (proxy-injection lanes only, never codex), +// while the pass itself runs on every lane (codex gets size-only notes). +export const READ_ORIGINAL_IMAGE_TOOL_NAME = "read_original_image"; +const READ_ORIGINAL_IMAGE_TOOL_DESCRIPTION = "Read the ORIGINAL full-resolution image of a message whose picture was downscaled before reaching you. Beside each shrunken image there is a [bili] note quoting its mNNNNN ref and both sizes — pass that ref here. The original bytes are returned as an image attached to this result. The original can be much larger than the downscaled copy, so call this only when you actually need full-resolution detail (small text, colors, pixel alignment); the downscaled version is enough for most work."; +const READ_ORIGINAL_IMAGE_PARAM_SCHEMA = { + type: "object" as const, + properties: { + ref: { type: "string", description: "The mNNNNN message ref quoted in the [bili] downsample note beside the shrunken image." }, + }, + required: ["ref"], +}; +export const READ_ORIGINAL_IMAGE_TOOL = { name: READ_ORIGINAL_IMAGE_TOOL_NAME, description: READ_ORIGINAL_IMAGE_TOOL_DESCRIPTION, input_schema: READ_ORIGINAL_IMAGE_PARAM_SCHEMA }; +export const READ_ORIGINAL_IMAGE_TOOL_OPENAI = { type: "function" as const, function: { name: READ_ORIGINAL_IMAGE_TOOL_NAME, description: READ_ORIGINAL_IMAGE_TOOL_DESCRIPTION, parameters: READ_ORIGINAL_IMAGE_PARAM_SCHEMA } }; +export const READ_ORIGINAL_IMAGE_TOOL_RESPONSES = { type: "function" as const, name: READ_ORIGINAL_IMAGE_TOOL_NAME, description: READ_ORIGINAL_IMAGE_TOOL_DESCRIPTION, parameters: READ_ORIGINAL_IMAGE_PARAM_SCHEMA }; +export const READ_ORIGINAL_IMAGE_TOOL_GOOGLE = { name: READ_ORIGINAL_IMAGE_TOOL_NAME, description: READ_ORIGINAL_IMAGE_TOOL_DESCRIPTION, parameters: READ_ORIGINAL_IMAGE_PARAM_SCHEMA }; + // #1097: acp_retrieve — resolve a stored ID-referenced message back to its full // original. Kernel-owned tool (RETRIEVE_TOOL_NAME); synthesized in all four // wire shapes here so every injection point (wire helpers, plugin manifest) diff --git a/src/config.ts b/src/config.ts index 71af485a8d..a846e378d1 100644 --- a/src/config.ts +++ b/src/config.ts @@ -389,6 +389,36 @@ export type CompressSettings = { * builtin (`default`, `lean`)]. Deepest-wins like every other field; * unknown names fall back to the identity surface. Kernel >= 0.0.66. */ promptPack?: string; + /** Instant tool-result absorption (kernel absorb API, acp-kernel >= 0.0.54). + * When `enabled`, eligible large tool results carry a forced [ACP absorb] + * instruction and the model distills them via the injected `absorb` tool; + * the original output is then hidden from every wire view until the next + * fold round (and its token cost is netted out of usage credits, like + * compress). Maps to kernel `Config.absorb`. Off unless explicitly + * enabled at some level. NOT supported on Responses marker/text-protocol + * routes (no native tool surface there). */ + absorb?: { + /** Enable absorb for this scope. Absent/false = off (kernel semantics). */ + enabled?: boolean; + /** Tool results smaller than this many tokens never get the absorb + * instruction (kernel default 1000). */ + minToolTokens?: number; + /** Only emit instructions once context usage reaches this fraction of + * the model window (kernel `contextThresholdPct`). Accepts a ratio + * (0.5) or percent string ("50%"); 0 = size gate alone (kernel + * default). */ + contextThresholdPct?: number | string; + /** Tool names whose results are never absorbable (glob-suffix + * patterns, e.g. "read"). Kernel default: none. */ + excludeTools?: string[]; + /** Rename the wire tool (default "absorb"). Must stay unique against + * the client's own tool names or the agent will call its own tool. */ + toolName?: string; + /** [#3402] When true (default), only image-bearing tool results are + * absorb candidates and they bypass minToolTokens; false restores the + * old behavior of marking any result ≥ minToolTokens. */ + mediaOnly?: boolean; + }; /** [#1097] Kernel CCR content store (kernel `Config.ccr`, acp-kernel * 0.0.84). When `enabled`, oversized tool results (>= minToolTokens) are * ID-referenced at arrival by the kernel's ccr-store node (prune → @@ -398,7 +428,7 @@ export type CompressSettings = { * Lossless by default — a retrieve not made costs one cheap tool call, * whereas a distilled-away detail is gone for good. ID-reference wins * over absorb (kernel ordering). Off unless explicitly enabled at some - * level. Merged sub-field-wise across the three levels like `prompts`. */ + * level. Merged sub-field-wise across the three levels like `absorb`. */ ccr?: { /** Enable CCR for this scope. Absent/false = off (kernel semantics). */ enabled?: boolean; @@ -424,7 +454,7 @@ export type CompressSettings = { * are all reused unchanged. Disabled or no plan state in context ⇒ * byte-identical output to the plain lexical ranking. Off by default; * measure before enabling in shared configs. Merged sub-field-wise across - * the three levels like `ccr`. See src/decompress-shared.ts. */ + * the three levels like `absorb`/`ccr`. See src/decompress-shared.ts. */ search?: { /** Enable plan-aware re-ranking + steering for search_context. * Absent/false = off (byte-identical output). */ @@ -448,7 +478,7 @@ export type CompressSettings = { * billion-context-pi #336/#339 and opencode-acp #377). Compress turns * are hard-exempt from compression, so their reasoning is otherwise an * unreclaimable context floor. Merged sub-field-wise across the three - * config levels like `prompts`. */ + * config levels like `absorb`. */ reasoning?: { /** Master switch (default true). Set `drop: false` per-provider for * models whose reasoning must round-trip unmodified. */ @@ -463,7 +493,7 @@ export type CompressSettings = { * hits the lattice AND carries an encrypted_content blob, bili buffers the * response, replays its own reasoning plus a continue nudge (up to maxContinue * rounds), and folds to ONE response with true summed usage. Merged sub-field-wise - * across the three levels like `reasoning`; off unless enabled at some + * across the three levels like `absorb`/`reasoning`; off unless enabled at some * level. See src/reasoning-guard.ts. */ reasoningGuard?: ReasoningGuardConfig; /** [#1093] Output-side compression levers — verbosity steering (a conciseness @@ -485,7 +515,7 @@ export type CompressSettings = { * unset profile yields byte-identical reports to pre-#1279 behavior. The * last request's effective value is stamped onto the session and used by * every report face. Merged sub-field-wise across the three levels like - * `prompts`. */ + * `absorb`. */ priceProfile?: { w?: number; r?: number; q?: number }; /** [#1921] Fold-state reconciliation — how the proxy reacts when the * resent history no longer contains a folded message's content-hash id @@ -890,7 +920,7 @@ export interface DshFileSettings { const NAMED_PROVIDER_ROUTING_FIELDS = ["compress", "models", "proxy", "passthrough", "compressProtocol", "protocol", "compat", "imageBilling", "imageTokenCap"] as const; // Once-per-signature dedup so hot-reload / repeated launcher loads don't spam -// the same named-provider warning. +// the same named-provider warning (same pattern as the absorb warnings below). const seenNamedProviderWarnings = new Set(); function warnNamedProviderOnce(signature: string, message: string): void { @@ -1146,6 +1176,45 @@ export function allowDshCompactionState(env: NodeJS.ProcessEnv): { enabled: bool return { enabled: fileAllow, source: fileAllow ? "file" : null }; } +// #1359: provider/model absorb.* overrides apply only to the proxy lane (plugin +// lane follows the base block). Warn once per load so the divergence isn't silent. +const ABSORB_DIVERGENCE_FIELDS = ["enabled", "toolName", "minToolTokens", "contextThresholdPct", "excludeTools", "mediaOnly"] as const; +const seenAbsorbDivergenceWarnings = new Set(); + +function normAbsorbValue(field: string, v: unknown): unknown { + if (field === "contextThresholdPct" && typeof v === "string") { + const m = v.match(/^(-?\d+(?:\.\d+)?)\s*%$/); + if (m) return Number(m[1]) / 100; + } + return v; +} + +export function findAbsorbPluginDivergences(routes: ProviderRoutes, baseAbsorb?: CompressSettings["absorb"]): string[] { + const out: string[] = []; + const check = (scope: string, level?: CompressSettings["absorb"]) => { + if (!level) return; + for (const field of ABSORB_DIVERGENCE_FIELDS) { + if (level[field] === undefined) continue; + if (JSON.stringify(normAbsorbValue(field, level[field])) !== JSON.stringify(normAbsorbValue(field, baseAbsorb?.[field]))) { + out.push(`${scope}.absorb.${field}=${JSON.stringify(level[field])} (base=${JSON.stringify(baseAbsorb?.[field])})`); + } + } + }; + for (const [url, route] of Object.entries(routes)) { + check(url, route.compress?.absorb); + for (const [model, entry] of Object.entries(route.models ?? {})) check(`${url}/${model}`, entry.compress?.absorb); + } + return out; +} + +function warnAbsorbPluginDivergences(routes: ProviderRoutes, baseAbsorb?: CompressSettings["absorb"]): void { + const divs = findAbsorbPluginDivergences(routes, baseAbsorb); + if (divs.length === 0) return; + const sig = divs.join("\u0000"); + if (seenAbsorbDivergenceWarnings.has(sig)) return; + seenAbsorbDivergenceWarnings.add(sig); + loggerLog("warn", `[acp-config] provider/model absorb override diverges from base [${divs.join("; ")}] — plugin-mode sessions follow the base value, proxy-mode sessions honor the override (#1359)`); +} /** [#1345] A provider/model-level `ccr` field that diverges from what plugin @@ -1250,6 +1319,7 @@ export function loadOptions(env: NodeJS.ProcessEnv = process.env): ProxyOptions const upstream = (env.ACP_UPSTREAM ?? fileConfig.upstream ?? "https://api.anthropic.com").replace(/\/$/, ""); const routes = loadRoutes(env); const namedProviders = loadNamedProviders(env); + warnAbsorbPluginDivergences(routes, fileConfig.compress?.absorb); warnCcrPluginDivergences(routes, fileConfig.compress); const passthrough = passthroughState(env); const modelContextLimit = parseInt(env.ACP_MODEL_CONTEXT_LIMIT ?? `${fileConfig.modelContextLimit ?? 200000}`, 10); @@ -1660,6 +1730,23 @@ type FileConfig = { * BILI_ROLLOUT_=0/1 (UPPER_SNAKE name) wins over everything, * including `enabled`. */ rollout?: RolloutFileConfig; + /** Security posture block (#3373) — knobs that WIDEN a trust boundary + * live here so the exposure surface is auditable in one place. + * `adminRelay` (default false, fail-closed): allow an embedding host + * (dsh web today) to relay the ENTIRE /__bili/ admin surface — including + * config editing and credential writes, not just the read-only panel — + * through its own entry origin: the host registers a /__bili prefix + * route forwarding to the loopback proxy, so a browser on another + * machine opens the panel through the same origin it already reached + * the host by. The admin gate itself is unchanged: the relayed hop is a + * local non-browser client (loopback source + loopback Host + no + * Origin header), so #1538 DNS-rebinding protection holds verbatim. + * Enabling it means whoever can reach the host web entry can reach the + * admin surface — protect the entry (auth/reverse proxy) accordingly. + * File-only, no env var (#2030). */ + security?: { + adminRelay?: boolean; + }; /** Native-hook attach behavior. * `attachExternal` (#1335): set `true` to let native hooks attach to * lifecycle-less listeners (a manually started `bili start` daemon — no @@ -1850,6 +1937,8 @@ type FileConfig = { ccrRetrievalTtlMs?: number; /** Large-decompress temp-file cap (#2030) — was BILI_DECOMPRESS_TMP_CAP (default 50). */ decompressTmpCap?: number; + /** Image rescale egress pass on/off (#3400) — env BILI_IMAGE_RESCALE (default: on). */ + imageRescale?: boolean; }; function nonEmpty(value: string | undefined): string | undefined { @@ -1894,11 +1983,11 @@ const KNOWN_TOP_LEVEL_KEYS = new Set([ "rollout", "dsh", "compat", "imageBilling", "imageTokenCap", "claude", "native", "resign", - "pi", "storage", + "pi", "storage", "security", // #2030 subsystem blocks: "network", "persist", "sessions", "plugin", "update", "diagnostics", "fakeCompletion", "codexCompact", "ccrRetrievalTtlMs", "decompressTmpCap", - "autoModeForge", + "autoModeForge", "imageRescale", ]); // Every field parseCompressSettings accepts — hint source for misplaced keys: @@ -1912,13 +2001,13 @@ const COMPRESS_SETTING_FIELDS = new Set([ "outputHeadroomMaxPct", "tiers", "protectedLatestTools", "protectedTools", "neverPreserveRecentTools", "preserveRecentTools", "visibilityMarkers", "rules", "injectTool", "injectNudge", - "acknowledgePromptsRisk", "ccr", "search", + "acknowledgePromptsRisk", "absorb", "ccr", "search", "prompts", "promptPack", "reasoningGuard", "outputSteering", "priceProfile", "reconcile", "streamSummary", ]); // Deduped per unique key set per process (same pattern as -// seenNamedProviderWarnings): loadConfigFile() runs on every config read, +// seenAbsorbDivergenceWarnings): loadConfigFile() runs on every config read, // so without this one startup would log the same warning several times. const seenUnknownTopLevelKeySignatures = new Set(); @@ -1965,6 +2054,7 @@ export function loadConfigFile(): FileConfig { warnUnknownTopLevelKeys(obj); warnInertResignPassthrough(obj); warnInertStripImages(obj); + warnInertRemovedCompressKeys(obj); value = obj as FileConfig; } else { value = {}; @@ -2106,6 +2196,46 @@ function warnInertStripImages(obj: Record): void { } let seenInertStripImagesSignature = ""; +// #3344: compress.imageCompression was REMOVED wholesale (#3266, image +// pre-compression + the image_full restore channel). Old configs that still set +// it get a one-time inert notice instead of a silent no-op: the parse whitelist +// (COMPRESS_SETTING_FIELDS) no longer carries the field, so it is dropped at +// EVERY compress level it can appear (global / provider / model). (`absorb` was +// removed by #3293 but REVIVED by #3402/#3422 — it parses again and is NOT inert.) +// Dedup by dead-key path signature (#1815 style): re-warn when the set changes. +const INERT_REMOVED_COMPRESS_KEYS = ["imageCompression"] as const; +let seenInertRemovedCompressKeysSignature = ""; +function warnInertRemovedCompressKeys(obj: Record): void { + const hits: string[] = []; + const scanBlock = (block: unknown, pathPrefix: string): void => { + if (!block || typeof block !== "object" || Array.isArray(block)) return; + for (const key of INERT_REMOVED_COMPRESS_KEYS) { + if ((block as Record)[key] !== undefined) hits.push(`${pathPrefix}.${key}`); + } + }; + scanBlock(obj.compress, "compress"); + const providers = obj.providers; + if (providers && typeof providers === "object" && !Array.isArray(providers)) { + for (const [url, entry] of Object.entries(providers as Record)) { + if (!entry || typeof entry !== "object" || Array.isArray(entry)) continue; + const p = entry as Record; + scanBlock(p.compress, `providers["${url}"].compress`); + const models = p.models; + if (models && typeof models === "object" && !Array.isArray(models)) { + for (const [model, mEntry] of Object.entries(models as Record)) { + if (!mEntry || typeof mEntry !== "object" || Array.isArray(mEntry)) continue; + scanBlock((mEntry as Record).compress, `providers["${url}"].models["${model}"].compress`); + } + } + } + } + hits.sort(); + const sig = hits.join(","); + if (sig === "" || sig === seenInertRemovedCompressKeysSignature) return; + seenInertRemovedCompressKeysSignature = sig; + loggerLog("warn", `[acp-config] ${hits.join(", ")} is INERT — compress.imageCompression was removed and no longer does anything at any level (image pre-compression + image_full restore channel, #3266). Remove it from the config; images pass through byte-identical.`); +} + function warnInertResignPassthrough(obj: Record): void { const inert: string[] = []; const consider = (map: unknown, providerBlockFor: (key: string) => ResignFileSettings | undefined): void => { @@ -2225,6 +2355,13 @@ export function resolveNativeAttachLanePriority(): string[] { return out; } +/** #3373: the /__bili/ admin relay gate (fail-closed). Read per use — the + * config file cache makes this cheap and hot edits apply without a host + * restart. */ +export function adminRelayEnabled(): boolean { + return loadConfigFile().security?.adminRelay === true; +} + /** #2877: the native-hook discovery policy. "own-lane" (default) rides only a * same-lane daemon and never falls back to a lane-less / user-zone one; * "global-fallback" restores the pre-#2877 fallback. Env @@ -2532,6 +2669,37 @@ export function parseCompressSettings(v: unknown): (CompressSettings & { injectT if (typeof obj.acknowledgePromptsRisk !== "boolean") ok = false; else out.acknowledgePromptsRisk = obj.acknowledgePromptsRisk; } + if ("absorb" in obj && obj.absorb !== undefined) { + const a = obj.absorb; + if (!a || typeof a !== "object" || Array.isArray(a)) { + ok = false; + } else { + const ao = a as Record; + const cleaned: NonNullable = {}; + for (const key of ["enabled", "minToolTokens", "contextThresholdPct", "excludeTools", "toolName", "mediaOnly"] as const) { + if (!(key in ao)) continue; + const v = ao[key]; + if (key === "enabled" || key === "mediaOnly") { + if (typeof v !== "boolean") { ok = false; continue; } + if (key === "enabled") cleaned.enabled = v; + else cleaned.mediaOnly = v; + } else if (key === "minToolTokens") { + if (typeof v !== "number" || !Number.isFinite(v)) { ok = false; continue; } + cleaned.minToolTokens = v; + } else if (key === "contextThresholdPct") { + if (!numberOrPercent(v)) { ok = false; continue; } + cleaned.contextThresholdPct = typeof v === "string" ? v.trim() : v; + } else if (key === "excludeTools") { + if (!Array.isArray(v) || v.some((x) => typeof x !== "string")) { ok = false; continue; } + cleaned.excludeTools = [...v] as string[]; + } else { + if (typeof v !== "string" || v.trim().length === 0) { ok = false; continue; } + cleaned.toolName = v.trim(); + } + } + if (ok) out.absorb = cleaned; + } + } if ("ccr" in obj && obj.ccr !== undefined) { const c = obj.ccr; if (!c || typeof c !== "object" || Array.isArray(c)) { diff --git a/src/fork-adoption.ts b/src/fork-adoption.ts index 87736f0e2a..5de73f8a2b 100644 --- a/src/fork-adoption.ts +++ b/src/fork-adoption.ts @@ -314,6 +314,15 @@ export function maybeAdoptForkBlocks(args: { } } +/** Outcome of an inheritance attempt (#3351): lets the caller tell "blocks + * were adopted" apart from "refs-only / nothing," so the derived-blocks gate + * closes only on real block adoption — the same discipline the boot sweep + * (reconcileDerivedLineage) applies, not on ref-only seeding. */ +interface AdoptionOutcome { + adoptedActive: number; + seededRefs: number; +} + /** #1486: copy-on-resume for an identified client whose resume forked a new * session id (Claude Code --resume). The parent is resolved by the caller * (memory first, then disk) via prefix-affinity's byte-exact full-history @@ -330,19 +339,19 @@ export function maybeAdoptResume(args: { upstreamOrigin: string; blocksEnabled: boolean; log: (level: string, msg: string) => void; -}): void { +}): AdoptionOutcome { const { session, parent, sharedDepth, protocol, parsed, upstreamOrigin, blocksEnabled, log } = args; const incomingMsgs = incomingCoreMessages(protocol, parsed); if (!incomingMsgs) { log("info", `[resume-inheritance] ${session.id}: resumed ${parent.id} (${sharedDepth} shared msgs) but ${protocol} has no adoption support in v1; lineage recorded, refs/blocks not inherited (#1486)`); - return; + return { adoptedActive: 0, seededRefs: 0 }; } const ids = new Set(incomingMsgs.map((m) => m.id)); const plan = planForkAdoption(parent, ids, { seedAllRefs: true, includeBlocks: blocksEnabled }); const seededRefs = Object.keys(plan.refs.byRaw).length; if (seededRefs === 0 && plan.adoptedActive === 0) { log("info", `[resume-inheritance] ${session.id}: resume of ${parent.id} had nothing to inherit (no overlapping refs${blocksEnabled ? " or fully-present blocks" : ""}); starting fresh (#1486)`); - return; + return { adoptedActive: 0, seededRefs: 0 }; } applyForkAdoption(session, plan, parent); log( @@ -356,6 +365,7 @@ export function maybeAdoptResume(args: { const n = Object.keys(storeSlice.byRef).length; log("info", `[resume-inheritance] ${session.id} adopted ${n} CCR content-store entr${n === 1 ? "y" : "ies"} for the inherited/cited refs from parent ${parent.id} (#1341/#1486)`); } + return { adoptedActive: plan.adoptedActive, seededRefs }; } /** #2807: the derived (plugin-register) fork link previously recorded @@ -365,8 +375,12 @@ export function maybeAdoptResume(args: { * history while the parent's fold ledger sat on disk. Adopt the parent's * fully-present blocks by message-id intersection at link time — the same * copy-on-fork seeding maybeAdoptResume runs for the strong path, minus the - * byte-prefix requirement. Runs once per session (derivedBlocksAdopted). - * Gated on resumeInheritance, NOT forkAdoption: this is an identified fork + * byte-prefix requirement. The call lands once per session (handle.ts gates + * it on derivedFromSessionId === undefined); the derivedBlocksAdopted marker + * it sets gates the BOOT reconcile, and since #3351 it is set only when blocks + * were actually adopted (mirroring reconcileDerivedLineage), so a young parent + * with no folds yet keeps the door open for a later retry. Gated on + * resumeInheritance, NOT forkAdoption: this is an identified fork * (the parent came from the plugin register), same class as the strong * path; #1834/#629 reserve forkAdoption for anonymous forks, and answering * to two different switches left the weak path dark while the boot-time @@ -383,8 +397,12 @@ export function maybeAdoptDerivedFork(args: { const { session, parent, protocol, parsed, upstreamOrigin, enabled, log } = args; if (!enabled || session.metadata.derivedBlocksAdopted === true) return; try { - maybeAdoptResume({ session, parent, sharedDepth: 0, protocol, parsed, upstreamOrigin, blocksEnabled: true, log }); - session.metadata.derivedBlocksAdopted = true; + const outcome = maybeAdoptResume({ session, parent, sharedDepth: 0, protocol, parsed, upstreamOrigin, blocksEnabled: true, log }); + if (outcome.adoptedActive > 0) { + session.metadata.derivedBlocksAdopted = true; + } else { + log("info", `[${session.id}] [derived] no blocks adopted from ${parent.id} yet (refs-only or none fully present); leaving the gate open so the boot reconcile can retry once the parent folds (#3351)`); + } } catch (e) { log("warn", `[${session.id}] [derived] block adoption from ${parent.id} failed (${String(e)}); continuing fresh (#2807)`); } diff --git a/src/host-profiles.ts b/src/host-profiles.ts new file mode 100644 index 0000000000..9420e62573 --- /dev/null +++ b/src/host-profiles.ts @@ -0,0 +1,89 @@ +/** + * #3417 layer ① — the per-agent DECLARED-defaults registry, as an IMPORT-FREE + * LEAF module. + * + * Why a separate file: layer ① must be readable from BOTH sides of the proxy + * boundary. Server-side consumers (lane-decisions, the four prepares) AND + * host-side agent bundles (src/agent/pi.ts resolves its own profile for the + * body-stamp channel) need these constants. Lane-decisions itself drags in + * server modules (preflight → kernel); an agent-side import of it would pull + * the whole compression stack into dist/agent/*.js client-process bundles + * (#3482 review: the first cut doubled dist/agent/pi.js and omp.js). Keeping + * the pure table here — zero imports, no payload scanning — lets both sides + * share ONE registry without crossing the boundary. A structural guard in + * tests/lane-decisions-matrix.test.ts pins the import-free property. + */ + +// ① HostProfile registry — the ONLY place lane names are compared (#3417). +// Properties here are per-AGENT constants. Anything that can vary per +// environment/profile/request belongs in the WireObservation layer instead. +export interface HostProfile { + readonly identity: string; + /** + * Declared default: does this host register bili's ACP tools host-scope + * (invisible to the model) by default? dsh does (#3177: 0/3939 preset + * requests carried the tools); pi/omp/opencode expose them visibly. The + * observation layer overrides this when the request itself declares the + * tools (#3401: some dsh profiles DO surface them). + */ + readonly toolsHostScopedByDefault: boolean; + /** + * Declared: this host's upstream requests need the billing block carried + * from the client request (#2189, subscription-OAuth lanes 429 without it). + */ + readonly wantsBillingBlock: boolean; + /** + * Declared: this host's identity stamp rides the request BODY + * (`prompt_cache_key`) — omp has no session header the proxy can bind + * (#1403/#1579); pi stamps `x-bili-plugin-conversation` in + * before_provider_headers instead (which outranks the body field). + * Registry property, not an adapter-side name gate (#3482 site 3). + */ + readonly stampsBodyPromptCacheKey: boolean; + /** + * Declared: this host build can route anthropic-messages traffic over + * its own direct TLS transport, bypassing globalThis.fetch (OMP Cowork, + * #2583) — the fetch-layer interception claim does not hold for such + * hosts, so body stamps must be withheld there. The per-request truth + * stays environmental (ompCoworkTransportBypassesFetch reads + * provider/api/proxy env); this flag owns the "is that even possible + * here" question (#3482 site 2). + */ + readonly coworkTransportPossible: boolean; + /** + * Declared: this client's `instructions` drift means a NEW persona, so + * the compression-session key fingerprints it (inverted allowlist — + * everyone else keys verbatim). Wire-CLIENT entries only: codex (#150, + * task ids reused across tasks) and claude-on-responses (#970, subagents + * share the main session id there). #3482 site 4. + */ + readonly instructionsFingerprintKeyed: boolean; + /** + * Declared: this host compacts histories CLIENT-SIDE and resends the + * compacted form (dsh native compaction, #2432) — the four prepares run + * the dual-signal compact-guard rebase for such hosts. Registry property, + * not a name comparison in each adapter (#3417). + */ + readonly compactsClientSide: boolean; +} + +const PROFILES: Record = { + dsh: { identity: "dsh", toolsHostScopedByDefault: true, wantsBillingBlock: true, stampsBodyPromptCacheKey: false, coworkTransportPossible: false, instructionsFingerprintKeyed: false, compactsClientSide: true }, + pi: { identity: "pi", toolsHostScopedByDefault: false, wantsBillingBlock: false, stampsBodyPromptCacheKey: false, coworkTransportPossible: false, instructionsFingerprintKeyed: false, compactsClientSide: false }, + omp: { identity: "omp", toolsHostScopedByDefault: false, wantsBillingBlock: false, stampsBodyPromptCacheKey: true, coworkTransportPossible: true, instructionsFingerprintKeyed: false, compactsClientSide: false }, + opencode: { identity: "opencode", toolsHostScopedByDefault: false, wantsBillingBlock: false, stampsBodyPromptCacheKey: false, coworkTransportPossible: false, instructionsFingerprintKeyed: false, compactsClientSide: false }, +}; + +/** No declared defaults at all — unknown agents and header-less proxy-lane + * clients resolve here (lane-decisions re-identifies them by name or wire + * signals). Exported because the wire-client table spreads it. */ +export const ANONYMOUS: HostProfile = { identity: "", toolsHostScopedByDefault: false, wantsBillingBlock: false, stampsBodyPromptCacheKey: false, coworkTransportPossible: false, instructionsFingerprintKeyed: false, compactsClientSide: false }; + +/** Resolve the declared profile for a plugin agent header value. Trimmed + * exact match — the historical shape of every pluginAgent comparison. + * Agent-side consumers (the host plugins) resolve their own profile through + * this same registry — name gates never re-appear adapter-side (#3482). */ +export function hostProfileFor(agentName: string | undefined | null): HostProfile { + const key = (agentName ?? "").trim(); + return PROFILES[key] ?? { ...ANONYMOUS, identity: key }; +} diff --git a/src/image-rescale.ts b/src/image-rescale.ts new file mode 100644 index 0000000000..02792f79a0 --- /dev/null +++ b/src/image-rescale.ts @@ -0,0 +1,578 @@ +// #3400: egress image rescale — the simplified re-do of the removed #3295 +// image pipeline. One shared pass over the decoded generic message structure +// (BiliMessage[]) at every egress point AFTER the kernel fold and BEFORE +// protocol encoding: big images are re-encoded SMALLER on the outbound copy +// only; session storage keeps the originals byte-exact (the wire received +// them, the storage holds them — we never rewrite history). A stable +// annotation rides beside each shrunken image telling the model both sizes +// and how to fetch the original (read_original_image, registered only on +// lanes that support injected tools; codex gets sizes only). +// +// Design constraints (issue #3400): +// - ZERO kernel changes: no version bump, no golden ledger, no kernel source. +// - Deterministic: same input bytes → same output bytes AND same annotation +// text, so the outbound body is byte-stable across rounds for the same +// history (the #3266 75%→28% cache-hit lesson). A process-lifetime hash +// cache encodes each unique image once. +// - Structural loop guard: images carried by a read_original_image tool +// result are NEVER re-shrunk (that exemption is what kills the #3266 +// read→resend→re-shrink cycle). Zero state, survives restarts. +// - Pure-JS encoder (jimp, devDependency inlined by tsup): sharp stays out, +// zero native dependencies. png/jpeg/bmp are handled; gif/webp/unknown +// formats pass through untouched (no reliable pure-JS round-trip). +// - No system-prompt injection, no sticky-restore state machine, no shrink +// records, no stats counters (logs are the observability surface). + +import { createHash } from "node:crypto"; +import { readFile } from "node:fs/promises"; +import { Jimp } from "jimp"; +import { parseDataUrl, type BiliMessage } from "acp-kernel/wire"; +import { decodeImageDims } from "./image-tokens.js"; +import { imageRescaleEnabled, imageRescaleMaxDim } from "./knobs.js"; +import { READ_ORIGINAL_IMAGE_TOOL_NAME } from "./compress-tool.js"; +import { toolFail, toolOk, type ProxyToolResult } from "./proxy-tool-result.js"; +import type { Session } from "./session.js"; +import type { Logger } from "./logger.js"; + +/** Trigger gate: longest edge above maxDim OR decoded bytes above this. */ +const RESCALE_TRIGGER_BYTES = 100 * 1024; +/** Keep-smaller rule: adopt the re-encoding only when it saves ≥20%. */ +const KEEP_RATIO = 0.8; +/** Explicit JPEG quality — pinned so the output is deterministic. */ +const JPEG_QUALITY = 80; +/** Process-lifetime encode cache cap (unique images per process). */ +const ENCODE_CACHE_MAX = 512; + +type RescaledMime = "image/png" | "image/jpeg" | "image/bmp"; + +const MIME_TO_JIMP: Record = { + "image/png": "image/png", + "image/jpeg": "image/jpeg", + "image/jpg": "image/jpeg", + "image/bmp": "image/bmp", +}; + +function normalizeMime(mediaType: string | undefined): RescaledMime | undefined { + if (typeof mediaType !== "string") return undefined; + return MIME_TO_JIMP[mediaType.trim().toLowerCase()]; +} + +/** Decoded byte length of a base64 payload (exact for valid base64). */ +function b64Bytes(b64: string): number { + const pad = b64.endsWith("==") ? 2 : b64.endsWith("=") ? 1 : 0; + return Math.max(0, Math.floor(b64.length / 4) * 3 - pad); +} + +interface RescaledImage { + base64: string; + width: number; + height: number; + origWidth: number; + origHeight: number; + origBytes: number; + newBytes: number; + mime: string; + /** True when this result came from an actual encode (cache miss). */ + fresh: boolean; +} + +// hash → pending/finished encoding. A promise map doubles as the in-flight +// de-duplication: concurrent requests for the same image share one encode. +const encodeCache = new Map>(); + +async function doRescale(mime: RescaledMime, b64: string, origBytes: number, maxDim: number): Promise { + try { + const img = await Jimp.read(Buffer.from(b64, "base64")); + const ow = img.width; + const oh = img.height; + // Re-check the gate against the AUTHORITATIVE dims (the header parse + // may have failed for exotic containers). + const scale = Math.min(1, maxDim / Math.max(ow, oh)); + if (scale >= 1 && origBytes <= RESCALE_TRIGGER_BYTES) return null; + const tw = Math.max(1, Math.round(ow * scale)); + const th = Math.max(1, Math.round(oh * scale)); + if (tw !== ow || th !== oh) await img.resize({ w: tw, h: th }); + const out = mime === "image/jpeg" + ? await img.getBuffer("image/jpeg", { quality: JPEG_QUALITY }) + : await img.getBuffer(mime); + if (out.length >= origBytes * KEEP_RATIO) return null; + return { + base64: out.toString("base64"), + width: tw, + height: th, + origWidth: ow, + origHeight: oh, + origBytes, + newBytes: out.length, + mime, + fresh: true, + }; + } catch { + return null; + } +} + +/** Decide + encode ONE image occurrence. Returns null when the original must + * be forwarded untouched (unsupported format, below the gate, low benefit, + * or decode failure). Cached per content hash — each unique image encodes + * once per process. */ +async function rescaleImage(mediaType: string, b64: string, maxDim: number): Promise { + const mime = normalizeMime(mediaType); + if (!mime || typeof b64 !== "string" || b64.length === 0) return null; + const origBytes = b64Bytes(b64); + const dims = decodeImageDims(b64); + const gated = (dims !== undefined && Math.max(dims.w, dims.h) > maxDim) || origBytes > RESCALE_TRIGGER_BYTES; + if (!gated) return null; + const hash = createHash("sha256").update(b64, "utf8").digest("hex"); + let pending = encodeCache.get(hash); + const cacheHit = pending !== undefined; + if (!pending) { + pending = doRescale(mime, b64, origBytes, maxDim); + encodeCache.set(hash, pending); + while (encodeCache.size > ENCODE_CACHE_MAX) { + const oldest = encodeCache.keys().next().value; + if (oldest === undefined || oldest === hash) break; + encodeCache.delete(oldest); + } + } + const result = await pending; + // A cache hit serves the stored encode result again — mark it non-fresh so + // the compression log fires exactly once per unique image (#3400 §8). + if (result !== null && cacheHit) return { ...result, fresh: false }; + return result; +} + +/** Test seam: drop the process-lifetime encode cache between cases. */ +export function clearRescaleEncodeCache(): void { + encodeCache.clear(); +} + +function formatKb(bytes: number): number { + return Math.round(bytes / 1024); +} + +/** Deterministic annotation text beside a shrunken image. Derived ONLY from + * the image dims/bytes and the message ref — no timestamps, no counters — + * so the same history always produces the same outbound bytes. */ +export function buildRescaleNote(opts: { + ref?: string; + toolArmed: boolean; + width: number; + height: number; + kb: number; + origWidth: number; + origHeight: number; + origKb: number; + ratio: number; +}): string { + const head = `[bili] 本图已降采样:当前 ${opts.width}×${opts.height} / ${opts.kb}KB;原图 ${opts.origWidth}×${opts.origHeight} / ${opts.origKb}KB`; + if (opts.ref !== undefined && opts.toolArmed) { + return `${head}(约 ${opts.ratio.toFixed(1)} 倍)。需要原分辨率细节(小字、颜色、对齐)时,调用 read_original_image("${opts.ref}") 读取原图。原图较大,按需读取。`; + } + return `${head}。`; +} + +function noteFor(ctx: RescaleCtx, ref: string | undefined, r: RescaledImage): string { + return buildRescaleNote({ + ref, + toolArmed: ctx.toolArmed, + width: r.width, + height: r.height, + kb: formatKb(r.newBytes), + origWidth: r.origWidth, + origHeight: r.origHeight, + origKb: formatKb(r.origBytes), + ratio: r.newBytes > 0 ? r.origBytes / r.newBytes : 0, + }); +} + +interface RescaleCtx { + session: Session; + sessionId: string; + log: Logger; + /** Longest-edge target; resolved from the knob by the caller. */ + maxDim: number; + /** True when read_original_image is registered for this session's lane — + * the annotation then carries the invocation entry; otherwise (codex, + * plugin mode) it reports sizes only. */ + toolArmed: boolean; +} + +function refFor(session: Session, id: string | undefined): string | undefined { + if (typeof id !== "string" || id.length === 0) return undefined; + const ref = session.state.messageRefs.byRaw[id]; + return typeof ref === "string" && ref.length > 0 ? ref : undefined; +} + +function logCompression(ctx: RescaleCtx, ref: string | undefined, r: RescaledImage): void { + // Spec (#3400 §8): the compression log fires ONCE per unique image — a + // hash-cache hit serves the same bytes without re-logging. + if (!r.fresh) return; + ctx.log("info", `[${ctx.sessionId}] [image-rescale] ${r.mime}${ref !== undefined ? ` ${ref}` : ""} ${r.origWidth}x${r.origHeight}/${formatKb(r.origBytes)}KB -> ${r.width}x${r.height}/${formatKb(r.newBytes)}KB`); +} + +type Part = Record; + +function asPart(v: unknown): Part | undefined { + return typeof v === "object" && v !== null && !Array.isArray(v) ? (v as Part) : undefined; +} + +// --------------------------------------------------------------------------- +// Per-shape writers. Each returns a COPY (or copies) of the message with the +// shrunk bytes + annotation attached; the input array and its objects are +// never mutated (session state and persistence keep the originals). +// --------------------------------------------------------------------------- + +async function rescaleAnthropic(m: BiliMessage, ctx: RescaleCtx): Promise { + const block = asPart(m.rawAnthropicBlock); + if (!block) return undefined; + if (block.type === "image") { + const src = asPart(block.source); + if (!src || src.type !== "base64" || typeof src.data !== "string") return undefined; + const r = await rescaleImage(typeof src.media_type === "string" ? src.media_type : "image/png", src.data, ctx.maxDim); + if (!r) return undefined; + const ref = refFor(ctx.session, m.id); + logCompression(ctx, ref, r); + const copy: BiliMessage = { ...m, rawAnthropicBlock: { ...block, source: { ...src, data: r.base64 } } }; + // The codec drops m.text ("[image]") when rawAnthropicBlock is set, so + // the annotation rides a SEPARATE adjacent core message: same role → + // coreToAnthropic merges both into one wire message [image, text]. + const noteMsg: BiliMessage = { id: `acp_imgnote_${m.id}`, role: m.role, contentType: "text", text: noteFor(ctx, ref, r) }; + return [copy, noteMsg]; + } + if (block.type === "tool_result" && Array.isArray(block.content)) { + const content = block.content as unknown[]; + const changed: unknown[] = []; + let any = false; + const ref = refFor(ctx.session, m.id); + for (const part of content) { + const p = asPart(part); + if (p?.type === "image") { + const src = asPart(p.source); + if (src?.type === "base64" && typeof src.data === "string") { + const r = await rescaleImage(typeof src.media_type === "string" ? src.media_type : "image/png", src.data, ctx.maxDim); + if (r) { + any = true; + logCompression(ctx, ref, r); + changed.push({ ...p, source: { ...src, data: r.base64 } }); + changed.push({ type: "text", text: noteFor(ctx, ref, r) }); + continue; + } + } + } + changed.push(part); + } + if (!any) return undefined; + return [{ ...m, rawAnthropicBlock: { ...block, content: changed } }]; + } + return undefined; +} + +async function rescaleOpenaiParts(m: BiliMessage, ctx: RescaleCtx): Promise { + const ref = refFor(ctx.session, m.id); + if (Array.isArray(m.rawOpenaiContentParts)) { + const parts = m.rawOpenaiContentParts as unknown[]; + const changed: unknown[] = []; + let any = false; + for (const part of parts) { + const p = asPart(part); + if (p?.type === "image_url") { + const url = asPart(p.image_url)?.url; + if (typeof url === "string" && url.startsWith("data:")) { + const d = parseDataUrl(url); + if (d) { + const r = await rescaleImage(d.mediaType, d.base64, ctx.maxDim); + if (r) { + any = true; + logCompression(ctx, ref, r); + changed.push({ ...p, image_url: { ...(asPart(p.image_url) ?? {}), url: `data:${d.mediaType};base64,${r.base64}` } }); + changed.push({ type: "text", text: noteFor(ctx, ref, r) }); + continue; + } + } + } + } + changed.push(part); + } + if (!any) return undefined; + return [{ ...m, rawOpenaiContentParts: changed }]; + } + if (m.rawOpenaiContent !== undefined) { + const p = asPart(m.rawOpenaiContent); + if (!p || p.type !== "image_url") return undefined; + const url = asPart(p.image_url)?.url; + if (typeof url !== "string" || !url.startsWith("data:")) return undefined; + const d = parseDataUrl(url); + if (!d) return undefined; + const r = await rescaleImage(d.mediaType, d.base64, ctx.maxDim); + if (!r) return undefined; + logCompression(ctx, ref, r); + // Convert the singular sidecar to the plural one carrying the note — + // the codec prefers rawOpenaiContentParts when present, so the copy + // drops the singular fields instead of double-rendering. + const copy: BiliMessage = { + ...m, + rawOpenaiContentParts: [ + { ...p, image_url: { ...(asPart(p.image_url) ?? {}), url: `data:${d.mediaType};base64,${r.base64}` } }, + { type: "text", text: noteFor(ctx, ref, r) }, + ], + }; + delete copy.rawOpenaiContent; + delete copy.imageBase64; + delete copy.imageMediaType; + return [copy]; + } + return undefined; +} + +async function rescaleGoogle(m: BiliMessage, ctx: RescaleCtx): Promise { + if (!Array.isArray(m.rawGoogleParts)) return undefined; + const parts = m.rawGoogleParts as unknown[]; + const changed: unknown[] = []; + let any = false; + const notes: string[] = []; + const ref = refFor(ctx.session, m.id); + for (const part of parts) { + const p = asPart(part); + const inline = p ? asPart(p.inlineData) : undefined; + if (inline && typeof inline.data === "string") { + const r = await rescaleImage(typeof inline.mimeType === "string" ? inline.mimeType : "image/png", inline.data, ctx.maxDim); + if (r) { + any = true; + logCompression(ctx, ref, r); + changed.push({ ...p, inlineData: { ...inline, data: r.base64 } }); + notes.push(noteFor(ctx, ref, r)); + continue; + } + } + changed.push(part); + } + if (!any) return undefined; + // The note appends to the core TEXT (never as a raw text part): the + // codec's re-render path drops raw text parts whose joined value diverges + // from m.text, but a changed m.text re-renders cleanly with every + // non-text part kept verbatim. + const text = (m.text ?? "") + ((m.text ?? "").length > 0 ? "\n\n" : "") + notes.join("\n"); + return [{ ...m, rawGoogleParts: changed, text }]; +} + +async function rescaleResponsesItem(m: BiliMessage, ctx: RescaleCtx): Promise { + const item = asPart(m.rawResponsesItem); + if (!item || item.type !== "message" || !Array.isArray(item.content)) return undefined; + const content = item.content as unknown[]; + const changed: unknown[] = []; + let first: RescaledImage | undefined; + const ref = refFor(ctx.session, m.id); + for (const part of content) { + const p = asPart(part); + if (p?.type === "input_image" && typeof p.image_url === "string" && p.image_url.startsWith("data:")) { + const d = parseDataUrl(p.image_url); + if (d) { + const r = await rescaleImage(d.mediaType, d.base64, ctx.maxDim); + if (r) { + if (!first) first = r; + logCompression(ctx, ref, r); + changed.push({ ...p, image_url: `data:${d.mediaType};base64,${r.base64}` }); + // NO note part inside the item: coreToResponses' verbatim + // condition compares the item's display text against + // m.text — an inserted text part breaks the round-trip and + // silently drops the image. The annotation rides a + // separate adjacent message instead. + continue; + } + } + } + changed.push(part); + } + if (!first) return undefined; + const copy: BiliMessage = { ...m, rawResponsesItem: { ...item, content: changed } }; + const noteMsg: BiliMessage = { id: `acp_imgnote_${m.id}`, role: "user", contentType: "text", text: noteFor(ctx, ref, first) }; + return [copy, noteMsg]; +} + +async function rescaleBareSidecar(m: BiliMessage, ctx: RescaleCtx): Promise { + if (typeof m.imageBase64 !== "string" || m.imageBase64.length === 0) return undefined; + const r = await rescaleImage(m.imageMediaType ?? "image/png", m.imageBase64, ctx.maxDim); + if (!r) return undefined; + const ref = refFor(ctx.session, m.id); + logCompression(ctx, ref, r); + const note = noteFor(ctx, ref, r); + const text = (m.text ?? "") + ((m.text ?? "").length > 0 ? "\n\n" : "") + note; + return [{ ...m, imageBase64: r.base64, text }]; +} + +/** The shared egress pass: walks the generic message structure once and + * returns a NEW array (untouched messages keep their object identity). + * No-op (same array) when the feature is disabled or nothing qualifies. */ +export async function applyImageRescalePass(messages: BiliMessage[], ctx: RescaleCtx): Promise { + if (!imageRescaleEnabled()) return messages; + const out: BiliMessage[] = []; + for (const m of messages) { + // STRUCTURAL LOOP GUARD: a read_original_image result carries the + // original bytes by design — re-shrinking it would recreate the #3266 + // read→resend→re-shrink cycle. Zero state: the marker is the tool + // name on the message itself. + if (m.toolName === READ_ORIGINAL_IMAGE_TOOL_NAME) { + out.push(m); + continue; + } + let handled: BiliMessage[] | undefined; + if (m.rawAnthropicBlock !== undefined) handled = await rescaleAnthropic(m, ctx); + else if (Array.isArray(m.rawGoogleParts)) handled = await rescaleGoogle(m, ctx); + else if (m.rawResponsesItem !== undefined) handled = await rescaleResponsesItem(m, ctx); + else if (m.rawOpenaiContentParts !== undefined || m.rawOpenaiContent !== undefined) handled = await rescaleOpenaiParts(m, ctx); + else if (m.imageBase64 !== undefined) handled = await rescaleBareSidecar(m, ctx); + if (handled) out.push(...handled); + else out.push(m); + } + return out; +} + +// --------------------------------------------------------------------------- +// Responses-API item-level walker (steady path only). patchResponsesInput +// rebuilds wire items from the RAW parsed input (input_image parts always come +// from slot.original), so the BiliMessage-level pass cannot reach the steady +// responses egress — this walker applies the same engine to the final +// ResponseInputItem[] instead. Annotations ride an input_text part directly +// after each shrunk input_image part in the same item's content/output array. +// --------------------------------------------------------------------------- + +/** Resolve the mNNNNN ref of an image by matching its bytes against the + * session's incoming-image index (rebuilt from the raw inbound body every + * request). Ambiguous matches (same image under several refs) degrade to a + * size-only note rather than guess. */ +async function refForItemBytes(session: Session, mediaType: string, b64: string): Promise { + const index = session.incomingImageIndex; + if (!index) return undefined; + const want = b64Bytes(b64); + const candidates: Array<{ ref: string; mediaType: string; bytes: number; path: string }> = []; + for (const [ref, entries] of index.entries()) { + for (const e of entries) { + if (e.bytes === want && e.mediaType === mediaType) candidates.push({ ref, mediaType: e.mediaType, bytes: e.bytes, path: e.path }); + } + } + if (candidates.length === 0) return undefined; + if (candidates.length === 1) return candidates[0].ref; + let matched: string | undefined; + for (const c of candidates) { + try { + const buf = await readFile(c.path); + if (buf.toString("base64") === b64) { + if (matched !== undefined && matched !== c.ref) return undefined; + matched = c.ref; + } + } catch { + // evicted spill — not the match + } + } + return matched; +} + +/** Egress pass for the responses steady path (see module header). No-op + * (same array) when disabled or nothing qualifies. */ +export async function applyImageRescaleItems(items: unknown[], ctx: RescaleCtx): Promise { + if (!imageRescaleEnabled()) return items; + // call_id → tool name, for the structural exemption on tool outputs. + const nameByCallId = new Map(); + for (const item of items) { + const i = asPart(item); + if ((i?.type === "function_call" || i?.type === "custom_tool_call") && typeof i.call_id === "string" && typeof i.name === "string") { + nameByCallId.set(i.call_id, i.name); + } + } + const out: unknown[] = []; + for (const item of items) { + const i = asPart(item); + // Structural loop guard, item form: an output answering a + // read_original_image call carries the original image by design. + if (i && (i.type === "function_call_output" || i.type === "custom_tool_call_output") + && typeof i.call_id === "string" + && nameByCallId.get(i.call_id) === READ_ORIGINAL_IMAGE_TOOL_NAME) { + out.push(item); + continue; + } + // Image-bearing shapes on the Responses wire: user/message items + // (content array) and tool outputs (output array — see + // responsesToolImageParts, which models both). + const partsKey = i?.type === "message" ? "content" : i?.type === "function_call_output" || i?.type === "custom_tool_call_output" ? "output" : undefined; + if (partsKey !== undefined && i !== undefined && Array.isArray(i[partsKey])) { + const parts = i[partsKey] as unknown[]; + const changed: unknown[] = []; + let any = false; + for (const part of parts) { + const p = asPart(part); + if (p?.type === "input_image" && typeof p.image_url === "string" && p.image_url.startsWith("data:")) { + const d = parseDataUrl(p.image_url); + if (d) { + const r = await rescaleImage(d.mediaType, d.base64, ctx.maxDim); + if (r) { + any = true; + const ref = await refForItemBytes(ctx.session, d.mediaType, d.base64); + logCompression(ctx, ref, r); + changed.push({ ...p, image_url: `data:${d.mediaType};base64,${r.base64}` }); + changed.push({ type: "input_text", text: noteFor(ctx, ref, r) }); + continue; + } + } + } + changed.push(part); + } + if (any) { + out.push({ ...i, [partsKey]: changed }); + continue; + } + } + out.push(item); + } + return out; +} + +// --------------------------------------------------------------------------- +// read_original_image execution. +// --------------------------------------------------------------------------- + +const REF_PATTERN = /^m\d{5,}$/; + +/** Execute read_original_image: fetch the ORIGINAL bytes for a message ref + * from the local session storage (incoming-image index + spill dir) and + * return them as an image attachment — the tool result itself stays TEXT. + * The caller attaches the bytes to an ephemeral carrier message whose + * toolName marks it structurally exempt from further rescaling. */ +export async function executeReadOriginalImage( + args: Record, + session: Session, + log: Logger, +): Promise { + const ref = typeof args.ref === "string" ? args.ref.trim() : ""; + if (!REF_PATTERN.test(ref)) { + return toolFail(`read_original_image: invalid ref "${ref}" — pass the mNNNNN message ref quoted in the [bili] note beside the shrunken image.`); + } + const entries = session.incomingImageIndex?.get(ref); + if (!entries || entries.length === 0) { + return toolFail(`read_original_image: no original image stored for ${ref} (it may no longer ride the conversation history). Work from the downscaled version you already have.`); + } + const entry = entries[0]; + let buf: Buffer; + try { + buf = await readFile(entry.path); + } catch { + return toolFail(`read_original_image: the original for ${ref} was evicted from local storage; it becomes readable again while the image rides the resent history.`); + } + const counterMeta = session.metadata.imageRescaleReads; + const counter = counterMeta && typeof counterMeta === "object" + ? (counterMeta as { count: number; bytes: number }) + : { count: 0, bytes: 0 }; + counter.count += 1; + counter.bytes += buf.length; + session.metadata.imageRescaleReads = counter; + log("info", `[${session.id}] [image-rescale] read_original_image ${ref}: ${entry.width ?? "?"}x${entry.height ?? "?"}/${formatKb(buf.length)}KB session-cumulative ${counter.count} read(s)/${formatKb(counter.bytes)}KB`); + // The receipt always names the local spill path: on the loop/injection + // lanes the bytes also ride an ephemeral carrier message (see the loop's + // proxyResults handling), but on the plugin lane the string tool protocol + // carries TEXT only — the host agent opens this file with its own tools. + const dims = `${entry.width ?? "?"}x${entry.height ?? "?"}`; + return { + ...toolOk(`Original image for ${ref} restored (${dims}, ${formatKb(buf.length)}KB${entry.mediaType ? `, ${entry.mediaType}` : ""}). Local copy: ${entry.path}. It is larger than the downscaled copy you saw — use it for fine detail (small text, colors, alignment).`), + image: { base64: buf.toString("base64"), mediaType: entry.mediaType }, + }; +} diff --git a/src/incremental-lane.ts b/src/incremental-lane.ts index bcf9fa69fd..6900a24888 100644 --- a/src/incremental-lane.ts +++ b/src/incremental-lane.ts @@ -482,14 +482,21 @@ export async function handleIncrementalTurn( socket: req.socket, }); - // Ack header: inject ahead of whatever status the pipeline writes. + // Ack header: stamped ONLY on 2xx — a non-2xx response means the turn + // did not land, and a host trusting the ack unconditionally would skip + // its resend and lose the user's message (#3344). In-repo clients gate + // on 200 already, so this tightens the contract rather than changing + // any live behavior. const appliedSeq = applied.appliedSeq; const originalWriteHead = res.writeHead.bind(res) as (...args: never[]) => ReturnType; (res as { writeHead?: unknown }).writeHead = (...args: unknown[]) => { - try { - res.setHeader("x-bili-applied-seq", String(appliedSeq)); - } catch { - // header injection is best-effort; the seq is also in the log line + const status = typeof args[0] === "number" ? args[0] : 200; + if (status >= 200 && status < 300) { + try { + res.setHeader("x-bili-applied-seq", String(appliedSeq)); + } catch { + // header injection is best-effort; the seq is also in the log line + } } return (originalWriteHead as (...a: unknown[]) => ReturnType)(...args); }; diff --git a/src/knobs.ts b/src/knobs.ts index 9d60ff52b0..d6df02107d 100644 --- a/src/knobs.ts +++ b/src/knobs.ts @@ -434,6 +434,32 @@ export function decompressTmpCap(): number { return Number.isFinite(v) && v > 0 ? v : 50; } +// --- image rescale (#3400) --- + +/** Default longest-edge target of the egress rescale pass (single source of + * truth; BILI_IMAGE_RESCALE_MAX_DIM overrides). */ +const IMAGE_RESCALE_DEFAULT_MAX_DIM = 1600; + +/** BILI_IMAGE_RESCALE > imageRescale > true. The egress image-rescale pass is + * ON by default (owner decision, #3400): big images are sent downscaled and + * the original stays in local session storage. "0"/"false" disables it + * (every image passes through untouched). Read per request so the switch + * flips without a restart. */ +export function imageRescaleEnabled(): boolean { + const envRaw = process.env.BILI_IMAGE_RESCALE; + if (envRaw !== undefined) return !(envRaw === "0" || envRaw === "false"); + return loadConfigFile().imageRescale !== false; +} + +/** BILI_IMAGE_RESCALE_MAX_DIM > IMAGE_RESCALE_DEFAULT_MAX_DIM. Longest-edge + * target of the rescale pass (fit-inside, never upscale). Env-only by design + * (#3400 spec): the config file carries just the on/off key. Floor 128 keeps + * a typo'd 0/16 from degenerating every image to an icon. */ +export function imageRescaleMaxDim(): number { + const v = tIntLoose("BILI_IMAGE_RESCALE_MAX_DIM", undefined); + return Number.isInteger(v) && v >= 128 ? v : IMAGE_RESCALE_DEFAULT_MAX_DIM; +} + // --- diagnostics & debug surface (was ACP_DUMP_*/BILI_DUMP_*/ACP_RENDER_NONE etc.) --- function fileDiag() { diff --git a/src/launcher.ts b/src/launcher.ts index 41aadf3154..9464d0f398 100644 --- a/src/launcher.ts +++ b/src/launcher.ts @@ -56,6 +56,7 @@ import { import { selfPackageRoot, isBiliPiEntry, ompPluginLoadedFrom, dshNativeInstalled, claudeNativeInstalled, piPackageStrings } from "./plugin-install.js"; import { applyOmpFirstEventTimeout } from "./agent/native-bootstrap.js"; import { logHostEvent } from "./agent/host-log.js"; +import { compactionDisablePatchRows } from "./agent/dsh-compaction-disable.js"; import { log as teeLog } from "./logger.js"; import { VERSION } from "./version.js"; import { winCmdUnsafeToken, winCmdRefusalError } from "./win-cmd.js"; @@ -3084,7 +3085,9 @@ export function prepareCodexHome(opts: { * plus the compaction-basic auto:false override so dsh's native * auto-compaction stands down for the bili proxy (a patch replaces the * target row's whole config, and dsh-base ships compaction-basic with no - * config, so {auto:false} is complete). Lives in the persistent + * config, so {auto:false} is complete), plus the #3354 Web-preset gate rows + * that make the agent presets wait for the load-time compaction-disable + * overlay the injected plugin registers. Lives in the persistent * `-bili` dir, INDEPENDENT of the settings.yaml rewrite — the * plugin is injected even when the user has no custom providers (pure * built-in deepseek route). Returns the patch file path (undefined when it @@ -3170,7 +3173,7 @@ export function writeDshAcpPatch(dshHome: string, entryName?: string): string | writeOverlayFileAtomic( dir, ".bili-acp.patch.yml", - `- insert:\n - id: bili-native\n name: ${pluginUrl}\n- id: compaction-basic\n config:\n auto: false\n# #2605: keep in sync with dsh.bundle.patch.yml — a patch replaces the row's whole config, so every default code is restated\n- id: llm-deepseek\n config:\n retryPolicy:\n mode: normal\n retryableCodes: [EMPTY_RESPONSE, RATE_LIMIT, SERVER, TIMEOUT, TRANSPORT, MALFORMED_RESPONSE]\n- id: llm-deepseek-account\n config:\n retryPolicy:\n mode: normal\n retryableCodes: [EMPTY_RESPONSE, RATE_LIMIT, SERVER, TIMEOUT, TRANSPORT, MALFORMED_RESPONSE]\n`, + `- insert:\n - id: bili-native\n name: ${pluginUrl}\n- id: compaction-basic\n config:\n auto: false\n# #3354: keep in sync with dsh.bundle.patch.yml — web-profile presets run their own compaction-basic in nested rows no patch layer can address by id (#1772); these gates make them wait for the load-time overlay the injected plugin registers (vendored from deshark42/dsh-compaction-basic-disable, MIT)\n${compactionDisablePatchRows()}# #2605: keep in sync with dsh.bundle.patch.yml — a patch replaces the row's whole config, so every default code is restated\n- id: llm-deepseek\n config:\n retryPolicy:\n mode: normal\n retryableCodes: [EMPTY_RESPONSE, RATE_LIMIT, SERVER, TIMEOUT, TRANSPORT, MALFORMED_RESPONSE]\n- id: llm-deepseek-account\n config:\n retryPolicy:\n mode: normal\n retryableCodes: [EMPTY_RESPONSE, RATE_LIMIT, SERVER, TIMEOUT, TRANSPORT, MALFORMED_RESPONSE]\n`, ); const file = path.join(dir, ".bili-acp.patch.yml"); try { diff --git a/src/loop/core.ts b/src/loop/core.ts index afaf08dac1..703aa184ad 100644 --- a/src/loop/core.ts +++ b/src/loop/core.ts @@ -15,12 +15,15 @@ import type { BiliMessage } from "acp-kernel/wire"; import { coreToAnthropic } from "acp-kernel/wire"; import { parseCompressInput, + ABSORB_TOOL_NAME, COMPRESS_TOOL_NAME, + READ_ORIGINAL_IMAGE_TOOL_NAME, RULE_TOOL_NAME, } from "../compress-tool.js"; -import { isProxyToolFor } from "../proxy-tool.js"; +import { effectiveAbsorbConfig, executeAbsorb, isProxyToolFor } from "../absorb.js"; import { effectiveRulesEnabled, executeRule } from "../rules-feature.js"; -import { ccrEnabled, drainPendingRetrievals, executeRetrieve, retrieveToolName } from "../store.js"; +import { ccrEnabled, drainPendingRetrievals, executeRetrieve, imageRescaleToolArmed, retrieveToolName } from "../store.js"; +import { applyImageRescalePass, executeReadOriginalImage } from "../image-rescale.js"; import { recordThreadTail } from "../server/thread-tracking.js"; import { applyRanges } from "../stream.js"; import { applyConfiguredCompression } from "../external-summary-compress.js"; @@ -31,7 +34,7 @@ import { classifyUpstreamFailure, type SkippedProxyInfo, type UpstreamFailureKin import { formatUpstreamError, proxyDispatcher } from "../upstream-proxy.js"; import { warnCacheCollapse } from "../cache-warn.js"; import { dumpRejectedBody } from "../error-dump.js"; -import { bodyDumpEnabled as knobBodyDumpEnabled } from "../knobs.js"; +import { bodyDumpEnabled as knobBodyDumpEnabled, imageRescaleMaxDim } from "../knobs.js"; import { dumpsDir } from "../paths.js"; import { isStrictReasoningEcho, modelIdOf, normalizeStrictEchoBody } from "../strict-echo.js"; import { log as loggerLog } from "../logger.js"; @@ -54,6 +57,7 @@ export function buildVisibilityMarker(toolName: string, result: string): string decompress: "📤", search_context: "🔍", acp_status: "📊", + absorb: "🫧", }; const icon = failed ? "❌" : (icons[toolName] ?? "📦"); @@ -138,6 +142,10 @@ const TRANSPORT_FAILURE_LABELS: Record = { unknown: "network error", }; +function activeAbsorbToolName(session: Session, config: Config): string | undefined { + const a = effectiveAbsorbConfig(session, config); + return a?.enabled === true ? a.toolName ?? ABSORB_TOOL_NAME : undefined; +} // Canonical args for repeat-detection: key-order/whitespace differences must not // defeat an identical-call match, so parse + re-stringify with sorted keys. @@ -328,6 +336,10 @@ export async function executeProxyToolAsync( toolName: string, args: Record, ctx: LoopCtx, callId?: string, rawArguments?: string, signal?: AbortSignal, ): Promise { + // #3400: async because it reads the spilled original from disk. The result + // carries the image bytes; the loop attaches them as an ephemeral carrier + // message (see proxyResults below). + if (toolName === READ_ORIGINAL_IMAGE_TOOL_NAME) return executeReadOriginalImage(args, ctx.session, (lvl, msg) => ctx.log(`[${lvl}] ${msg}`)); if (toolName !== "compress") return executeProxyTool(toolName, args, ctx, callId, rawArguments); const input = typeof rawArguments === "string" && rawArguments.length > 0 ? rawArguments : args; return applyConfiguredCompression(input, ctx, callId, signal); @@ -359,6 +371,10 @@ export function executeProxyTool( if (toolName === "acp_cache") { return handleAcpCache(ctx.session, args); } + const absorb = effectiveAbsorbConfig(ctx.session, ctx.config); + if (absorb?.enabled === true && toolName === (absorb.toolName ?? ABSORB_TOOL_NAME)) { + return executeAbsorb(args, callId, absorb, ctx); + } if (effectiveRulesEnabled(ctx.session, ctx.config) && toolName === RULE_TOOL_NAME) { return executeRule(args, ctx); } @@ -566,24 +582,33 @@ export async function* runCompressLoop( // continue from) — callers fall back to the legacy rebuild for that case // only, which is safe pre-start because the client's target is then // unconsumed. - const buildThreadBody = (extra: CoreMessage[]): Record | undefined => { + // #3400: every loop re-request re-encodes from core messages that may + // still carry full-size originals (the post-compress refreshFolded view is + // rebuilt from prepared.originalMessages), so each encode goes through the + // shared egress rescale pass. Outbound copies only — coreMessages keeps + // holding what it held. Already-shrunken images hit the hash cache and + // pass through byte-identical (deterministic encoder), so repeated calls + // never perturb the upstream prefix cache. + const rescaleForEncode = async (messages: CoreMessage[]): Promise => + applyImageRescalePass(messages as BiliMessage[], { session: ctx.session, sessionId: ctx.session.id, log: (lvl, msg) => ctx.log(`[${lvl}] ${msg}`), maxDim: imageRescaleMaxDim(), toolArmed: imageRescaleToolArmed(ctx.session) }); + const buildThreadBody = async (extra: CoreMessage[]): Promise | undefined> => { const tail = threadCurrentTail(); if (tail === undefined) return undefined; threadInFlightLen = threadDelta.length; threadTailAtSend = tail; - const messages = coreToAnthropic([...threadUnsent(), ...extra] as BiliMessage[], undefined); + const messages = coreToAnthropic(await rescaleForEncode([...threadUnsent(), ...extra]), undefined); return withStrictEchoRepair({ ...requestBody, thread: { ...(requestBody.thread as Record), previous_message_id: tail }, messages, }); }; - const buildLoopRequestBody = (extra: CoreMessage[] = []): Record => { + const buildLoopRequestBody = async (extra: CoreMessage[] = []): Promise> => { if (threadContinue) { - const body = buildThreadBody(extra); + const body = await buildThreadBody(extra); if (body !== undefined) return body; } - return withStrictEchoRepair(adapter.buildRequest([...coreMessages, ...extra], systemPrompt, requestBody)); + return withStrictEchoRepair(adapter.buildRequest(await rescaleForEncode([...coreMessages, ...extra]), systemPrompt, requestBody)); }; // The blind truncation retry re-sends roundBody byte-identical — on a // threads turn that re-continues the body's target. Allowed only while @@ -910,7 +935,7 @@ export async function* runCompressLoop( text: truncationContinuationNudge(tail), }; try { - const retryBody = buildLoopRequestBody([nudge]); + const retryBody = await buildLoopRequestBody([nudge]); const respResult = await fetchUpstream(retryBody); if (!respResult.response.body) { respResult.clearTimer(); @@ -956,7 +981,7 @@ export async function* runCompressLoop( text: DEGENERATE_RETRY_NUDGE, }; try { - const retryBody = buildLoopRequestBody([nudge]); + const retryBody = await buildLoopRequestBody([nudge]); const respResult = await fetchUpstream(retryBody); if (!respResult.response.body) { respResult.clearTimer(); @@ -1020,7 +1045,7 @@ export async function* runCompressLoop( text: DEGENERATE_RETRY_NUDGE, }; try { - const retryBody = buildLoopRequestBody([nudge]); + const retryBody = await buildLoopRequestBody([nudge]); const respResult = await fetchUpstream(retryBody); if (!respResult.response.body) { respResult.clearTimer(); @@ -1071,7 +1096,7 @@ export async function* runCompressLoop( let realCalls = 0; const realToolCalls: ToolCallEmit[] = []; - const proxyResults: { name: string; callId: string; result: string; arguments: string; signature?: string }[] = []; + const proxyResults: { name: string; callId: string; result: string; arguments: string; signature?: string; image?: { base64: string; mediaType: string } }[] = []; for (const call of allCalls) { if (call.name.length === 0) { @@ -1095,7 +1120,7 @@ export async function* runCompressLoop( parsedArgs = {}; } const result = await withSessionLock(ctx.session, () => executeProxyToolAsync(call.name, parsedArgs, ctx, call.callId, rawArgs, signal)); - proxyResults.push({ name: call.name, callId: call.callId, result: result.text, arguments: call.arguments, signature: call.signature }); + proxyResults.push({ name: call.name, callId: call.callId, result: result.text, arguments: call.arguments, signature: call.signature, ...(result.image ? { image: result.image } : {}) }); if (ctx.visibilityMarkers !== false) { const markerKey = `${call.name}\u0000${result.text}`; if (seenMarkers.has(markerKey)) { @@ -1225,6 +1250,28 @@ export async function* runCompressLoop( // ride the server-side thread (the assistant tool_use it // answers already lives there in r_k). if (threadContinue) threadDelta.push(toolResultMsg); + // #3400: the original image rides an ephemeral user + // message right after its ack. toolName marks it so the + // egress rescale pass exempts it structurally (the loop + // guard — without this, reading an original would just + // get it shrunk again, #3266's cycle). Ephemeral like + // every acp_loop_* record: never persisted, never seen + // by the client (#1548). + if (pr.image) { + const dataUrl = `data:${pr.image.mediaType};base64,${pr.image.base64}`; + const carrierMsg: BiliMessage = { + id: `acp_loop_r${round}_img_${pr.callId}`, + role: "user", + contentType: "text", + text: "[bili] 原图已读取,完整分辨率的原始图片如下。", + toolName: pr.name, + imageBase64: pr.image.base64, + imageMediaType: pr.image.mediaType, + rawResponsesItem: { type: "message", role: "user", content: [{ type: "input_text", text: "[bili] 原图已读取,完整分辨率的原始图片如下。" }, { type: "input_image", image_url: dataUrl }] }, + }; + coreMessages.push(carrierMsg); + if (threadContinue) threadDelta.push(carrierMsg); + } } else if (ctx.visibilityMarkers !== false) { coreMessages.push({ id: `acp_loop_r${round}_marker_${pr.callId}`, @@ -1267,9 +1314,10 @@ export async function* runCompressLoop( // its refresh is a harmless identical-view re-fold. Falls back // to the pre-compress view (previous behavior) if the host hook // is absent or throws. + const absorbName = activeAbsorbToolName(ctx.session, ctx.config); if ( ctx.refreshFolded && - proxyResults.some((pr) => (pr.name === "compress") && !pr.result.includes("FAILED")) + proxyResults.some((pr) => (pr.name === "compress" || pr.name === absorbName) && !pr.result.includes("FAILED")) ) { try { const refreshed = await ctx.refreshFolded(coreMessages); @@ -1358,7 +1406,7 @@ export async function* runCompressLoop( // #2701: threads-continue turns build the delta-shaped body (see // buildLoopRequestBody); legacy full-rebuild otherwise. - let newBody = buildLoopRequestBody(); + let newBody = await buildLoopRequestBody(); if (knobBodyDumpEnabled()) { try { const fs = await import("node:fs"); @@ -1398,7 +1446,7 @@ export async function* runCompressLoop( coreMessages.push(...stripped); ctx.log(`[acp-loop] round ${round}: re-request rejected (${e.status}: ${e.body.slice(0, 200)}); retrying without replayed thinking blocks`); loggerLog("warn", `[acp-loop] re-request rejected (${e.status}); retrying without thinking replay: ${e.body.slice(0, 200)}`); - newBody = buildLoopRequestBody(); + newBody = await buildLoopRequestBody(); respResult = await fetchUpstream(newBody); } } catch (e) { @@ -1421,9 +1469,10 @@ export async function* runCompressLoop( if (blocks) committedBlocks += Number(blocks[1]); } } + const absorbed = proxyResults.some((pr) => pr.name === activeAbsorbToolName(ctx.session, ctx.config) && !pr.result.includes("FAILED")); const note = committedTokens > 0 ? `compression committed (~${committedTokens.toLocaleString("en-US")} tokens saved${committedBlocks > 0 ? ` in ${committedBlocks} block(s)` : ""}); ` - : ""; + : absorbed ? "context work committed; " : ""; const msg = `${note}the follow-up request to the model failed (${TRANSPORT_FAILURE_LABELS[kind]}) before any response — your context is intact, resend your last message to continue`; ctx.log(`[acp-loop] round ${round}: re-request transport failure (${kind}); ending turn in-band instead of throwing (#1453)`); loggerLog("error", `[acp-loop] re-request transport failure after ${round} round(s): ${formatUpstreamError(e, requestOptions.url, ctx.proxyUrl, ctx.skippedProxy)}`); diff --git a/src/persist.ts b/src/persist.ts index 2ce5187ade..f2928a4496 100644 --- a/src/persist.ts +++ b/src/persist.ts @@ -108,12 +108,13 @@ const MIGRATION_MARKER = ".bili-migration-286.done"; const STALE_WARN_THROTTLE_MS = 60_000; -/** #3049: slack for the ownership fast path's mtime check — covers this - * process's own write latency (builder → atomic write lands a few ms after - * builtAtMs). Foreign writes landing INSIDE this window are missed for one - * cycle only if their writer also bypasses the shared index (disabled - * sidecar); see ownershipFastPath. */ -const OWN_COMMIT_GRACE_MS = 1_500; +/** #3406: mtime slack for the ownership stamp check — the same physical write + * can surface slightly different mtimeMs across statSync calls on some + * filesystems (FUSE, coarse-grained timestamps); the landed-write stamp is + * taken from the file we just wrote, so a sub-ms drift must not void our own + * ownership. Anything at or below this epsilon is OUR write; anything newer + * with a size change is foreign and forces the full read. */ +const OWN_WRITE_STAMP_EPSILON_MS = 1; interface PersistedSession { version: number; @@ -322,15 +323,23 @@ export class SessionStore { private readonly log: Logger; private readonly staleWarnAt = new Map(); - /** #3049: ownership token for the stale-guard fast path. Set INSIDE the - * guardedBuild closure after the sidecar ingest of the record we are + /** #3049/#3406: ownership token for the stale-guard fast path. Set INSIDE + * the guardedBuild closure after the sidecar ingest of the record we are * about to write (or the disk record we adopted): {counters of that - * commit, wall-clock build time}. The next guard compares this token - * against the unified row + the envelope file's mtime to prove nobody - * else committed since — skipping the full-file read+parse that costs - * ~320ms/turn on >40MiB sessions. Cleared on the direct-write paths - * (restore/rekey) that bypass guardedBuild. */ - private readonly ownCommit = new Map(); + * commit, build time, and — once the write has LANDED — the envelope's + * post-write stat ("written")}. The next guard compares this token + * against the unified row + the envelope file's size/mtime to prove + * nobody else committed since — skipping the full-file read+parse that + * costs ~320ms/turn on >40MiB sessions. An unstamped token (no "written") + * is unproven and never takes the fast path. Cleared on the direct-write + * paths (restore/rekey) that bypass guardedBuild. */ + private readonly ownCommit = new Map(); /** #3049 test seam: how many times the stale guard took the FULL * envelope read (the expensive path). The fast path collapses this to @@ -370,10 +379,19 @@ export class SessionStore { * synchronously on EVERY call; a microtask-batched run per turn keeps * the immediate-ish semantics while coalescing same-tick calls. */ private readonly writerOffPending = new Map void>(); + /** #3406: host-owned debounced envelope saves (id → armed timer / latest builder). + * The debounce lives HERE instead of inside the kernel store so each landed + * write goes through the stamped path (store.writeNow + stampOwnWrite): a + * kernel-internal debounce cannot observe its own landing, which left every + * per-turn debounced save unstamped and killed the ownership fast path. */ + private readonly saveTimers = new Map>(); + private readonly saveBuilders = new Map PersistedSession>(); + private readonly debounceMs: number; constructor(opts?: { dir?: string; debounceMs?: number; enabled?: boolean; log?: Logger }) { const debounceMs = opts?.debounceMs ?? defaultDebounce(); this.enabled = (opts?.enabled ?? true) && debounceMs >= 0; + this.debounceMs = Math.max(0, debounceMs); this.sessionsDir = opts?.dir ?? defaultDir(); const baseLog = opts?.log ?? defaultLogger; this.log = baseLog; @@ -432,7 +450,7 @@ export class SessionStore { this.store = new StateStore({ dir: this.sessionsDir, version: PERSIST_VERSION, - debounceMs: Math.max(0, debounceMs), + debounceMs: this.debounceMs, enabled: this.enabled, codec: this.codec, log: (level, msg) => { @@ -1550,8 +1568,8 @@ export class SessionStore { return null; } - /** #3049 ownership fast path — proves "the disk envelope is still the - * record this process last committed" with three cheap signals, no + /** #3049/#3406 ownership fast path — proves "the disk envelope is still + * the record this process last committed" with three cheap signals, no * payload read: * * 1. Unified row counters equal OUR last commit's counters. Every @@ -1559,22 +1577,31 @@ export class SessionStore { * multi-process topology), and each commit ingests before it writes * the file — so a sibling's newer commit bumps the row first and this * comparison fails, falling back to the exact full read. - * 2. The envelope file's mtime is not newer than our last build (plus - * grace for our own write latency). A writer that does NOT share the - * index (legacy-mode sibling, manual edit) still touches the file, and - * an mtime past our build window forces the full read. + * 2. #3406: the token carries the stat of a write that actually LANDED + * ("written", stamped after the awaited write resolves — see + * stampOwnWrite — including host-debounced saves, whose debounce is + * host-owned precisely so their landing goes through the stamped + * writeNow path), and the envelope file still matches it BYTE-FOR- + * BYTE: same size, mtime not newer beyond sub-ms clock skew. Any + * difference means a writer that does NOT share the index (legacy- + * mode sibling, manual edit) touched the file after our last write, + * so the full read decides. An unstamped token (restore-replica seed, + * crash before stamping) is unproven and never takes this path — the + * old #3286 grace window (builtAtMs + 1.5s) mis-classified exactly + * those foreign writes as ours when they landed inside the window + * (#3406 item 2). * 3. The incoming counters are not behind our own commit — they cannot * be, in-process (same session object, monotonic counters), but the * check keeps this path self-contained; any surprise defers to the * full read. * - * Residual (deliberate, strictly narrower than an unconditional - * size/mtime ownership skip): a sibling whose sidecar is DISABLED (3- - * strike drift) writing a newer envelope within the same grace window - * (~1.5s) as our build is missed for one cycle; its next write wins the - * exact guard again. Crash safety is clean: a process that dies between - * its DB ingest and its file write leaves no token here (fresh map), so - * the next boot always takes the full read. */ + * Residual: a foreign write landing in the microsecond gap between our + * awaited write resolving and the stamp's statSync can be adopted as + * "ours" for one cycle — vs the 1.5s window it replaces. A foreign + * write byte-identical to ours changes nothing observable. Crash safety + * is clean: a process that dies between its DB ingest and its file write + * leaves no stamped token here (fresh map), so the next boot always + * takes the full read. */ private ownershipFastPath(incoming: PersistedSession): boolean { const own = this.ownCommit.get(incoming.id); if (!own) return false; @@ -1592,7 +1619,9 @@ export class SessionStore { if (!row || row.requests !== own.requests || row.nextBlockId !== own.nextBlockId) return false; const stat = this.statEnvelope(incoming.id, incoming.meta); if (!stat) return false; - if (stat.mtimeMs > own.builtAtMs + OWN_COMMIT_GRACE_MS) return false; + if (!own.written) return false; // unproven token (see stampOwnWrite) — full read decides + if (stat.size !== own.written.size) return false; // any byte-count change is a foreign write (bigger OR smaller) + if (stat.mtimeMs > own.written.mtimeMs + OWN_WRITE_STAMP_EPSILON_MS) return false; return true; } @@ -1600,7 +1629,7 @@ export class SessionStore { * kernel-discovered source file first, then the canonical live paths. * Null when no file exists yet (first save: the full read correctly * returns null-fast via loadEnvelope's miss). */ - private statEnvelope(id: string, meta?: { protocol?: string; upstreamOrigin?: string }): { mtimeMs: number } | null { + private statEnvelope(id: string, meta?: { protocol?: string; upstreamOrigin?: string }): { size: number; mtimeMs: number } | null { const candidates: string[] = []; const known = this.sourceFiles.get(id); if (known) candidates.push(known); @@ -1612,7 +1641,7 @@ export class SessionStore { for (const file of candidates) { try { const st = statSync(file); - return { mtimeMs: st.mtimeMs }; + return { size: st.size, mtimeMs: st.mtimeMs }; } catch { // try next } @@ -1678,15 +1707,39 @@ export class SessionStore { * returned — our fresh build, or the disk payload the guard made us * adopt). The unified row was ingested from the same payload in the same * closure, so token counters === row counters until any other writer - * commits. Called from inside the guardedBuild closure only. */ + * commits. Called from inside the guardedBuild closure only. + * #3406: the LANDED-write stamp ("written") and the record's identity + * meta are carried across rebuilds — envelope-writing lanes re-stamp + * after their write lands (stampOwnWrite), and writerOff/index-only + * saves never touch the file, so the previous stamp stays valid. */ private noteOwnCommit(id: string, committed: PersistedSession): void { + const prev = this.ownCommit.get(id); this.ownCommit.set(id, { requests: committed.stats?.requests ?? committed.requests ?? 0, nextBlockId: committed.state?.nextBlockId ?? 0, builtAtMs: Date.now(), + written: prev?.written, + meta: committed.meta + ? { protocol: committed.meta.protocol, upstreamOrigin: committed.meta.upstreamOrigin } + : prev?.meta, }); } + /** #3406: stamp the envelope's post-write stat into the ownership token — + * the proof that OUR bytes are the ones on disk right now, replacing the + * #3286 build-time grace window. Called ONLY after a write has LANDED — + * the explicit writeNow resolved, the host-debounced save fired through + * writeNow, or flushSync reported ok. A missing file (null stat) or a + * write failure leaves the token unstamped, which ownershipFastPath + * treats as unproven → full read. */ + private stampOwnWrite(id: string): void { + const token = this.ownCommit.get(id); + if (!token) return; + const stat = this.statEnvelope(id, token.meta); + if (!stat) return; + token.written = stat; + } + /** #2401: read-side convergence. Replace the live session's persisted * surfaces with the newer disk record, in place (the object identity is * load-bearing — request handlers and the pool map hold references to @@ -1730,7 +1783,13 @@ export class SessionStore { /** Schedule a debounced write for a session. Multiple calls within the * window coalesce; the record is built at WRITE time, so the freshest * session state is persisted. Safe to call on the hot path. No-op if - * disabled. */ + * disabled. + * #3406: the debounce is host-owned (saveTimers/saveBuilders) — timer + * semantics mirror the kernel store's (first call arms, later calls swap + * the builder only, unref'd) — but the firing write lands through + * store.writeNow AND gets stamped (stampOwnWrite), which the kernel's + * internal debounce could not do: every per-turn save then proved its + * bytes on disk and the ownership fast path stayed live. */ scheduleSave(session: Session): void { if (!this.saveContentStore(session)) return; if (this.writerOffActive()) { @@ -1760,7 +1819,24 @@ export class SessionStore { }); return; } - this.store.scheduleSave(session.id, this.guardedBuild(session)); + if (!this.enabled) return; + const id = session.id; + this.saveBuilders.set(id, this.guardedBuild(session)); + const existing = this.saveTimers.get(id); + if (existing) return; + const timer = setTimeout(() => { + this.saveTimers.delete(id); + const build = this.saveBuilders.get(id); + this.saveBuilders.delete(id); + if (!build) return; + // Errors are logged inside the kernel write chain; swallowing here + // keeps a failed timer write from becoming an unhandledRejection. + void this.store.writeNow(id, build).then(() => { + this.stampOwnWrite(id); // #3406: the write LANDED — prove our bytes are on disk + }).catch(() => {}); + }, this.debounceMs); + timer.unref?.(); + this.saveTimers.set(id, timer); } /** Asynchronously persist a session right now (skips the debounce). Throws @@ -1792,6 +1868,7 @@ export class SessionStore { return; } await this.store.writeNow(session.id, this.guardedBuild(session)); + this.stampOwnWrite(session.id); // #3406: the write LANDED — prove our bytes are on disk this.maybeCutForkBase(session); this.foldResidentTail(session); this.dropResidentBlockContents(session); @@ -1878,8 +1955,18 @@ export class SessionStore { this.dropResidentMetadata(session); return true; } + // Mirror the kernel flushSync contract: a sync commit supersedes any + // armed debounce for this id (its builder would just re-write the + // fresher state moments later — redundant IO racing this fence). + const armed = this.saveTimers.get(session.id); + if (armed) { + clearTimeout(armed); + this.saveTimers.delete(session.id); + this.saveBuilders.delete(session.id); + } const ok = this.store.flushSync(session.id, this.guardedBuild(session)); if (ok) { + this.stampOwnWrite(session.id); // #3406: the write LANDED — prove our bytes are on disk this.maybeCutForkBase(session); this.foldResidentTail(session); this.dropResidentBlockContents(session); @@ -1898,10 +1985,34 @@ export class SessionStore { // 3-strike disable for no reason. this.drainWriterOffPending(); for (const session of sessions) this.saveContentStore(session); + await this.flushScheduledSaves(); await this.store.flushAll(); this.shadow?.close(); } + /** #3406: drain the host-owned debounced saves before the kernel store's + * own flushAll — its timers no longer know about them. Mirrors the + * kernel's drain: clear timers, write each pending builder now (stamped), + * log failures; store.flushAll then also drains any still-in-flight chain. */ + private async flushScheduledSaves(): Promise { + const ids = [...this.saveTimers.keys()]; + for (const timer of this.saveTimers.values()) clearTimeout(timer); + this.saveTimers.clear(); + const builders = new Map PersistedSession>(); + for (const id of ids) { + const build = this.saveBuilders.get(id); + if (build) builders.set(id, build); + this.saveBuilders.delete(id); + } + await Promise.all([...builders.entries()].map(([id, build]) => + this.store.writeNow(id, build).then(() => { + this.stampOwnWrite(id); + }).catch((e) => { + this.log("error", `[persist] shutdown flush failed for ${id}: ${String(e)}`); + }), + )); + } + /** Close the unified sidecar without flushing pending writes. Intended for * short-lived stores (CLI export, one-shot loaders): cancels debounced * writes first so nothing builds against a closed handle. Windows cannot @@ -1914,7 +2025,7 @@ export class SessionStore { /** Whether a write is currently pending (debounce timer armed) for a id. */ hasPending(id: string): boolean { - return this.store.hasPending(id) || this.writerOffPending.has(id); + return this.saveTimers.has(id) || this.writerOffPending.has(id); } /** Phase 3 (#2960): the JSON envelope is skipped only while the sidecar @@ -1926,6 +2037,9 @@ export class SessionStore { /** Cancel all pending writes without flushing (e.g. for tests). */ cancelAll(): void { + for (const timer of this.saveTimers.values()) clearTimeout(timer); + this.saveTimers.clear(); + this.saveBuilders.clear(); this.store.cancelAll(); this.writerOffPending.clear(); } diff --git a/src/plugin.ts b/src/plugin.ts index 7b46a9fca8..8b9fa0b10c 100644 --- a/src/plugin.ts +++ b/src/plugin.ts @@ -1,5 +1,5 @@ import { BLOCKED_REF, COMPRESS_TOOL_NAME, type CompressionCore, type Config, type CoreMessage, type NudgeDecision, type RetrievalExport, applyRetrieve, countMessageTokens, parseStoredPlaceholder, prune } from "acp-kernel"; -import { publicSnapshotCapBytes } from "./knobs.js"; +import { imageRescaleEnabled, publicSnapshotCapBytes } from "./knobs.js"; import { buildStatusPanel } from "acp-kernel/panel"; import { fileURLToPath } from "node:url"; import type { ServerResponse } from "node:http"; @@ -10,10 +10,10 @@ import { getStore, storageStatusLabel } from "./persist.js"; import { cloneStoreForRefs } from "./store.js"; import { acquireInFlight, createSession, getSession, publishForkSession, diagnoseSuccessWithoutUsage, effectiveConfig, findSessionByCanonicalId, listSessions, markCompactionBoundary, markDirty, peekSession, releaseInFlight, statusInputBaseline, withSessionLock, type Session } from "./session.js"; import { clientConversationHeader } from "./session-id.js"; -import { BILI_ACP_TOOLS_ANTHROPIC, BILI_ACP_TOOLS_ANTHROPIC_NO_RANGE, BILI_ACP_TOOLS_OPENAI, BILI_ACP_TOOLS_OPENAI_NO_RANGE, BILI_ACP_TOOLS_RESPONSES_NO_RANGE, PROXY_TOOL_NAMES, RETRIEVE_TOOL_NAME, RULE_TOOL, RULE_TOOL_NAME, RULE_TOOL_OPENAI, RULE_TOOL_RESPONSES, SEARCH_CONTEXT_TOOL_NAME, retrieveToolsFor, withAcpNotes, withAcpNotesOpenAI } from "./compress-tool.js"; +import { ABSORB_TOOL_NAME, BILI_ACP_TOOLS_ANTHROPIC, BILI_ACP_TOOLS_ANTHROPIC_NO_RANGE, BILI_ACP_TOOLS_OPENAI, BILI_ACP_TOOLS_OPENAI_NO_RANGE, BILI_ACP_TOOLS_RESPONSES_NO_RANGE, PROXY_TOOL_NAMES, READ_ORIGINAL_IMAGE_TOOL, READ_ORIGINAL_IMAGE_TOOL_NAME, READ_ORIGINAL_IMAGE_TOOL_OPENAI, READ_ORIGINAL_IMAGE_TOOL_RESPONSES, RETRIEVE_TOOL_NAME, RULE_TOOL, RULE_TOOL_NAME, RULE_TOOL_OPENAI, RULE_TOOL_RESPONSES, SEARCH_CONTEXT_TOOL_NAME, absorbToolsFor, retrieveToolsFor, withAcpNotes, withAcpNotesOpenAI } from "./compress-tool.js"; import { externalSummaryEnabled, externalSummaryMaxToolDurationMs, withExternalSummaryTools } from "./external-summary-surface.js"; import { salvageCompressArgs } from "./plugin-compress-salvage.js"; -import { isProxyToolFor } from "./proxy-tool.js"; +import { absorbEnabled, effectiveAbsorbConfig, isProxyToolFor } from "./absorb.js"; import { effectiveRulesEnabled, rulesEnabled } from "./rules-feature.js"; import { executeProxyToolAsync } from "./loop/core.js"; import type { ProxyToolResult } from "./proxy-tool-result.js"; @@ -812,6 +812,9 @@ export function handlePluginManifest(res: import("node:http").ServerResponse, co // enablement stays enforced at execution (isProxyToolFor / executeProxyTool). // #1359: the advertised name is the base-config toolName, matching the // plugin-lane gate (which adjudicates the same base block). + const absorbOn = absorbEnabled(config); + const absorbName = config.absorb?.toolName ?? ABSORB_TOOL_NAME; + const absorbTools = absorbOn ? absorbToolsFor(absorbName) : undefined; const rulesOn = rulesEnabled(config); // [#1271] acp_retrieve is advertised only while the base config enables CCR (same // #1192 conservative rule as absorb/acp_rule). The proxy wires it on the anthropic/ @@ -820,6 +823,13 @@ export function handlePluginManifest(res: import("node:http").ServerResponse, co const ccrOn = config.ccr?.enabled === true; const ccrName = config.ccr?.toolName ?? RETRIEVE_TOOL_NAME; const ccrTools = ccrOn ? retrieveToolsFor(ccrName) : undefined; + // #3400: read_original_image restores the full-resolution original of a + // downscaled image from local session storage. Same conservative #1192 + // rule as acp_retrieve: advertised only while the base knob enables the + // rescale pass (disabled ⇒ no [bili] note ever names it, so a registered + // copy could only be dead weight). Execution stays always-allowed: it is + // an idempotent local read, never a state mutation. + const rescaleOn = imageRescaleEnabled(); // #1712: decompress's startId/endId (range restore) executes only on CCR-armed // sessions, so the manifest advertises them only when the base config enables // CCR (#1345 plugin policy = base block verbatim) — same conservative #1192 @@ -845,11 +855,11 @@ export function handlePluginManifest(res: import("node:http").ServerResponse, co proxy: "billion-context", version: VERSION, commit: BUILD_COMMIT, - toolNames: [...PROXY_TOOL_NAMES, ...(rulesOn ? [RULE_TOOL_NAME] : []), ...(ccrOn ? [ccrName] : [])], + toolNames: [...PROXY_TOOL_NAMES, ...(absorbTools ? [absorbName] : []), ...(rulesOn ? [RULE_TOOL_NAME] : []), ...(ccrOn ? [ccrName] : []), ...(rescaleOn ? [READ_ORIGINAL_IMAGE_TOOL_NAME] : [])], tools: { - anthropic: [...acpAnthropic, ...(rulesOn ? [RULE_TOOL] : []), ...(ccrTools ? [ccrTools.anthropic] : [])], - openai: [...acpOpenai, ...(rulesOn ? [RULE_TOOL_OPENAI] : []), ...(ccrTools ? [ccrTools.openai] : [])], - responses: [...acpResponses, ...(rulesOn ? [RULE_TOOL_RESPONSES] : [])], + anthropic: [...acpAnthropic, ...(absorbTools ? [absorbTools.anthropic] : []), ...(rulesOn ? [RULE_TOOL] : []), ...(ccrTools ? [ccrTools.anthropic] : []), ...(rescaleOn ? [READ_ORIGINAL_IMAGE_TOOL] : [])], + openai: [...acpOpenai, ...(absorbTools ? [absorbTools.openai] : []), ...(rulesOn ? [RULE_TOOL_OPENAI] : []), ...(ccrTools ? [ccrTools.openai] : []), ...(rescaleOn ? [READ_ORIGINAL_IMAGE_TOOL_OPENAI] : [])], + responses: [...acpResponses, ...(absorbTools ? [absorbTools.responses] : []), ...(rulesOn ? [RULE_TOOL_RESPONSES] : []), ...(rescaleOn ? [READ_ORIGINAL_IMAGE_TOOL_RESPONSES] : [])], }, headers: { agent: PLUGIN_AGENT_HEADER, conversation: PLUGIN_CONVERSATION_HEADER, contextWindow: PLUGIN_CONTEXT_WINDOW_HEADER, maxOutput: PLUGIN_MAX_OUTPUT_HEADER, model: PLUGIN_MODEL_HEADER, instructionsMutable: PLUGIN_INSTRUCTIONS_MUTABLE_HEADER }, toolEndpoint: "/__bili/plugin/tool", @@ -1321,6 +1331,8 @@ export async function handlePluginFork(payload: string, res: ServerResponse): Pr * name is not one of the known opt-in tools (a truly unknown tool keeps the * generic 400 with its allowed list). */ function disabledOptionalToolNote(tool: string, session: Session, config: Config): string | undefined { + const absorbName = effectiveAbsorbConfig(session, config)?.toolName ?? ABSORB_TOOL_NAME; + if (tool === absorbName) return `${tool} is not enabled on this bili proxy (compress.absorb.enabled is not true) — nothing was absorbed.`; if (tool === RULE_TOOL_NAME) return `${tool} is not enabled on this bili proxy (compress.rules.enabled is not true) — nothing was recorded.`; if (!ccrEnabled(session) && tool === retrieveToolName(session)) return `${tool} is not enabled on this bili proxy (compress.ccr.enabled is not true) — nothing was retrieved.`; return undefined; @@ -1725,7 +1737,7 @@ export async function handlePluginTool( if (!isProxyToolFor(tool, session, deps.config)) { // #1192: a known opt-in tool disabled by this session's effective config // (registered from a manifest served while it was enabled) answers with - // model-facing text on the same channel executeRule use + // model-facing text on the same channel executeRule/executeAbsorb use // for failures — a hard 400 would surface as an unfixable red error card. const note = disabledOptionalToolNote(tool, session, deps.config); if (note !== undefined) { @@ -1735,6 +1747,8 @@ export async function handlePluginTool( return; } const allowed = [...PROXY_TOOL_NAMES]; + const absorb = effectiveAbsorbConfig(session, deps.config); + if (absorb?.enabled === true) allowed.push(absorb.toolName ?? ABSORB_TOOL_NAME); if (effectiveRulesEnabled(session, deps.config)) allowed.push(RULE_TOOL_NAME); res.writeHead(400, { "content-type": "application/json" }); res.end(JSON.stringify({ ok: false, error: `unknown tool "${tool}" (expected one of: ${allowed.join(", ")})` })); diff --git a/src/preflight.ts b/src/preflight.ts index 715e7c58a9..79f486526c 100644 --- a/src/preflight.ts +++ b/src/preflight.ts @@ -11,6 +11,7 @@ import { } from "acp-kernel"; import { buildCompressSystemPrompt, parseCompressInput } from "./compress-tool.js"; import { IMAGE_PLACEHOLDER, imagePlaceholdersForSummary } from "./image-note.js"; +import { applyAbsorbView } from "./absorb.js"; import { adoptContentStore, ccrLoopConfig, contentStoreOf } from "./store.js"; import { applyRanges, normalizeRangeOrder, type RewriteCtx } from "./stream.js"; import { fetchWithTimeout, isTransientUpstreamError, replayMaxAttempts, replayBackoffMs, sleep, UpstreamHttpError } from "./fetch-util.js"; @@ -263,6 +264,12 @@ interface PreflightFailure { kind: PreflightFailureKind; /** A temporary transport failure, not evidence that this context cannot be compressed. */ retryable?: boolean; + /** Pure fold-math proved NO combination of folds can close the gap regardless of + * upstream health (#3461) — the host answers non-retryable 4xx instead of a + * transient 502. Relay-dependent exhaustion (empty/truncated/incomplete + * summaries, budget draws, pool saturation) deliberately leaves this unset: + * those walks can succeed on retry once the relay recovers. */ + deterministic?: boolean; /** Upstream HTTP status when kind === "upstream" and the failure was an HTTP response. */ status?: number; /** Human-readable cause (safe to surface to the client). */ @@ -1637,7 +1644,7 @@ export async function preflightCompress(deps: PreflightDeps, messages: CoreMessa }); deps.session.state = turn.state; adoptContentStore(deps.session, turn.contentStore); - const view = turn.messages; + const view = applyAbsorbView(turn.messages, turn.state, activeConfig, currentTokens); const est = applyEstimateCalibration(estimateCoreMessages(view) + wireOverhead, kFactor, kOrigin, deps.upstreamOrigin) + imageReserve; result.payloadEstimate = est; // #2952 review: keep the char-count UPPER bound fresh through recovery. @@ -1806,6 +1813,7 @@ export async function preflightCompress(deps: PreflightDeps, messages: CoreMessa adoptContentStore(deps.session, turn.contentStore); // Absorbed pairs are hidden on the wire, so the fit check must see the // same reduced payload prepare* will actually forward. + turn.messages = applyAbsorbView(turn.messages, turn.state, activeConfig, currentTokens); // Floor on the session's measured input baseline: the upstream's // input_tokens also covers the system prompt + tool definitions, which // are not in turn.messages, so the direct estimate can undershoot. @@ -1948,7 +1956,11 @@ export async function preflightCompress(deps: PreflightDeps, messages: CoreMessa deps.log("info", `[preflight] raw-fold pool (~${potential} tok${zoneNote}) below deficit (~${deficit} tok); deferring to bounded tier-distillation recovery (#2952)`); break; } - failure = { kind: "exhausted", detail: `futile round: the maximum possible saving from all foldable content (~${potential} tokens${zoneNote}) is below the required reduction (~${deficit} tokens) — no combination of folds can bring the payload under the target. Raise the model context window or restart the session.` }; + // #3461: structurally doomed only when NO draw ever came back + // unusable and the pool never dropped — an unusable draw means + // the shortfall may be the relay, not the content, and the + // retry/canary machinery (#726/#2832) owns that case. + failure = { kind: "exhausted", deterministic: lastUnusableDetail === undefined && !poolQueueDrop, detail: `futile round: the maximum possible saving from all foldable content (~${potential} tokens${zoneNote}) is below the required reduction (~${deficit} tokens) — no combination of folds can bring the payload under the target. Raise the model context window or restart the session.` }; deps.log("warn", `[preflight] skipping futile round: max possible saving ~${potential} tok < required ~${deficit} tok${zoneNote}; zero summarization calls spent (#1841)`); break; } @@ -1974,7 +1986,11 @@ export async function preflightCompress(deps: PreflightDeps, messages: CoreMessa deps.log("warn", "[preflight] no compressible ranges outside the protected recent zone; relaxing soft protection (preserveRecentMessages/Tokens -> 0) and retrying"); continue; } - failure = { kind: "exhausted", detail: relaxed ? relaxedExhaustedDetail : subMinNoted + // #3461: every subcase here is structural — nothing compressible + // remains (or nothing above minCompressRange, or the whole relaxed + // zone already folded) — so no retry against a healthier relay can + // change the verdict. + failure = { kind: "exhausted", deterministic: true, detail: relaxed ? relaxedExhaustedDetail : subMinNoted ? `no foldable compressible ranges remain: all ${viable.length} viable range(s) are below minCompressRange (${minChars} chars)` : "no compressible ranges remain in the conversation" }; break; @@ -2655,7 +2671,10 @@ export async function preflightCompress(deps: PreflightDeps, messages: CoreMessa deps.log("info", `[preflight] remaining raw-fold pool (~${bailRemaining} tok${zoneNote}) below deficit (~${bailDeficit} tok); deferring to bounded tier-distillation recovery (#2952)`); break; } - failure = { kind: "exhausted", detail: `futile round: after the folds so far, the remaining foldable content (~${bailRemaining} tokens${zoneNote}) cannot close the ~${bailDeficit}-token gap to the target even if every remaining range folded successfully.${skipNote} Raise the model context window or restart the session.` }; + // #3461: doomed regardless of relay health only when no draw came + // back unusable and the pool never dropped — landed folds prove + // the relay worked, so the residual shortfall is real content. + failure = { kind: "exhausted", deterministic: lastUnusableDetail === undefined && !poolQueueDrop, detail: `futile round: after the folds so far, the remaining foldable content (~${bailRemaining} tokens${zoneNote}) cannot close the ~${bailDeficit}-token gap to the target even if every remaining range folded successfully.${skipNote} Raise the model context window or restart the session.` }; break; } if (!failure && !budgetHit && !relaxed && result.payloadEstimate >= limit) { @@ -2699,14 +2718,18 @@ export async function preflightCompress(deps: PreflightDeps, messages: CoreMessa if (budgetHit) { failure = { kind: "exhausted", detail: `the preflight summarization budget (${summaryBudget} calls per protection regime) was exhausted before the payload fit the window${unusableNote}${skipNote}` }; } else if (relaxed && result.compressedRanges > 0) { - failure = { kind: "exhausted", detail: `${relaxedExhaustedDetail}${skipNote}` }; + // #3461: "folded everything compressible" is only a structural + // verdict when nothing was lost to unusable draws or pool drops. + failure = { kind: "exhausted", deterministic: lastUnusableDetail === undefined && !poolQueueDrop, detail: `${relaxedExhaustedDetail}${skipNote}` }; } else if (result.compressedRanges === 0) { const cause = poolQueueDrop ? "the shared summary pool was saturated and no summary call was ever dispatched (scheduler congestion, not a chain failure; auto-fold retries next turn)" : skipReasons.length > 0 ? skipReasons.slice(0, 3).join(" | ") : "each was below minCompressRange, had an unusable summary, or failed to apply"; - failure = { kind: "exhausted", detail: rangesTried === 0 + // #3461: zero dispatches means the walk never touched the relay — + // the shortfall is structural unless the pool itself dropped work. + failure = { kind: "exhausted", deterministic: rangesTried === 0 && !poolQueueDrop && lastUnusableDetail === undefined, detail: rangesTried === 0 ? `no viable range could be compressed: ${cause}${unusableNote}` : `no range could be compressed across ${rangesTried} viable range${rangesTried === 1 ? "" : "s"}: ${cause}${unusableNote}` }; } else { diff --git a/src/proxy-tool-result.ts b/src/proxy-tool-result.ts index 8033b1578d..9cdebc6957 100644 --- a/src/proxy-tool-result.ts +++ b/src/proxy-tool-result.ts @@ -42,6 +42,10 @@ export interface ProxyToolResult { // `parse:missing-content`, `gate:cannot-anchor`) — outcome=refused alone // never says WHY in the [plugin] execution line. reason?: string; + // #3400: original image bytes restored by read_original_image — the loop + // rides them as an ephemeral carrier message; plugin-lane hosts open the + // local path named in the receipt text instead (string protocol). + image?: { base64: string; mediaType: string }; } export function toolOk(text: string): ProxyToolResult { diff --git a/src/proxy-tool.ts b/src/proxy-tool.ts deleted file mode 100644 index 4067e2fefe..0000000000 --- a/src/proxy-tool.ts +++ /dev/null @@ -1,11 +0,0 @@ -import { RULE_TOOL_NAME, PROXY_TOOL_NAMES } from "./compress-tool.js"; -import { ccrEnabled, retrieveToolName } from "./store.js"; -import { effectiveRulesEnabled } from "./rules-feature.js"; -import type { Config } from "acp-kernel"; -import type { Session } from "./session.js"; - -export function isProxyToolFor(name: string, session: Session | undefined, config: Config): boolean { - if (PROXY_TOOL_NAMES.has(name)) return true; - if (ccrEnabled(session) && name === retrieveToolName(session)) return true; - return effectiveRulesEnabled(session, config) && name === RULE_TOOL_NAME; -} diff --git a/src/restart.ts b/src/restart.ts index 9e05d8318a..bfad040bcf 100644 --- a/src/restart.ts +++ b/src/restart.ts @@ -412,18 +412,31 @@ export async function performSelfRestart(deps: SelfRestartDeps): Promise { + const deadline = Date.now() + ms; + while (Date.now() < deadline) { + if (!isPidAlive(pid)) return true; + await sleep(POLL_MS); + } + return !isPidAlive(pid); +} + /** Terminate a failed replacement and verify it actually died (#3044): a * kill that silently no-ops leaves exactly the orphan processes that report - * describes — idle, holding a port nobody routes to, accumulating memory. */ + * describes — idle, holding a port nobody routes to, accumulating memory. + * #3344: escalate SIGTERM → SIGKILL before asking a human — a replacement + * stuck in a SIGTERM-resistant state (blocked syscall / TERM ignored) is + * still force-killable; manual cleanup stays only the last resort. */ async function killAndVerify(child: ChildProcess, pid: number | undefined, verifyMs: number, log: Logger): Promise { try { child.kill(); } catch { /* already gone */ } if (pid === undefined) return; - const deadline = Date.now() + verifyMs; - while (Date.now() < deadline) { - if (!isPidAlive(pid)) return; - await sleep(POLL_MS); - } - log("error", `[restart] replacement pid ${pid} is still alive after SIGTERM — manual cleanup required (kill ${pid}); until it dies it keeps leaking CPU/memory on a port nobody routes to (#3044)`); + if (await waitUntilGone(pid, verifyMs)) return; + // SIGTERM didn't take within the window — escalate before reporting. + try { process.kill(pid, "SIGKILL"); } catch { /* already gone */ } + if (await waitUntilGone(pid, verifyMs)) return; + log("error", `[restart] replacement pid ${pid} is still alive after SIGTERM+SIGKILL — manual cleanup required (kill -9 ${pid}); until it dies it keeps leaking CPU/memory on a port nobody routes to (#3044)`); } function resumeListening(server: http.Server, host: string, port: number, log: Logger): void { diff --git a/src/server.ts b/src/server.ts index a2d4fc103d..229056ebc7 100644 --- a/src/server.ts +++ b/src/server.ts @@ -73,7 +73,8 @@ import { type GoogleSystemInstruction, type GoogleTool, } from "acp-kernel/wire"; -import { COMPRESS_TOOL, BILI_ACP_TOOLS_ANTHROPIC, BILI_ACP_TOOLS_ANTHROPIC_NO_RANGE, BILI_ACP_TOOLS_GOOGLE, BILI_ACP_TOOLS_GOOGLE_NO_RANGE, BILI_ACP_TOOLS_OPENAI, BILI_ACP_TOOLS_OPENAI_NO_RANGE, BILI_ACP_TOOLS_RESPONSES, BILI_ACP_TOOLS_RESPONSES_NO_RANGE, BILI_ACP_READONLY_TOOLS_RESPONSES, BILI_ACP_READONLY_TOOLS_RESPONSES_NO_RANGE, COMPRESS_TOOL_NAME, RULE_TOOL, RULE_TOOL_GOOGLE, RULE_TOOL_OPENAI, RULE_TOOL_RESPONSES, retrieveToolsFor, buildAcpTagsOnlyPrompt } from "./compress-tool.js"; +import { ABSORB_TOOL_NAME, COMPRESS_TOOL, BILI_ACP_TOOLS_ANTHROPIC, BILI_ACP_TOOLS_ANTHROPIC_NO_RANGE, BILI_ACP_TOOLS_GOOGLE, BILI_ACP_TOOLS_GOOGLE_NO_RANGE, BILI_ACP_TOOLS_OPENAI, BILI_ACP_TOOLS_OPENAI_NO_RANGE, BILI_ACP_TOOLS_RESPONSES, BILI_ACP_TOOLS_RESPONSES_NO_RANGE, BILI_ACP_READONLY_TOOLS_RESPONSES, BILI_ACP_READONLY_TOOLS_RESPONSES_NO_RANGE, COMPRESS_TOOL_NAME, RULE_TOOL, RULE_TOOL_GOOGLE, RULE_TOOL_OPENAI, RULE_TOOL_RESPONSES, absorbToolsFor, retrieveToolsFor, buildAcpTagsOnlyPrompt } from "./compress-tool.js"; +import { absorbEnabled, storeEffectiveAbsorb } from "./absorb.js"; import { ccrEnabled, ccrLoopConfig, ccrPluginWireOk, contentStoreOf, executeRetrieve, pruneExpiredRetrievals, reconcileReloadedRetrievals, renderRetrievalNotes, retrieveToolName, snapshotPendingRetrievals, snapshotRetrievalNotes, storeEffectiveCcr, type CcrSettings } from "./store.js"; import { buildIncomingImageIndex, pruneRetrieveImgExports } from "./image-restore.js"; import { rulesEnabled, storeEffectiveRules } from "./rules-feature.js"; @@ -1472,13 +1473,16 @@ export type Prepared = { * ACTUALLY reached the payload, not the pre-gate intent. The model-decided * gate runs asynchronously between diagNudge's label and the rebuild, so * consumers that need truth (self-heal streaks, log labels) read this. - * "injected": an advisory/directive/fallback message landed; "vetoed": - * the decision gate ran and nothing was injected (model declined or the - * answer was unusable); "skipped": armed but nothing renderable (empty - * nudge text / yes-with-no-live-range). Undefined = no arm this round, or - * the arm was gated off before delivery was attempted (suppression / - * breaker / gate off) — consumers only ever test for "injected". */ - nudgeOutcome?: "injected" | "vetoed" | "skipped"; + * "injected": an advisory/directive/fallback message landed; "vetoed": + * the decision gate ran and nothing was injected (model declined or the + * answer was unusable); "skipped": armed but nothing renderable (empty + * nudge text / yes-with-no-live-range); "gated" (#3461): over-limit + * pressure admitted by every standing gate but held by the emergency + * backoff this round — a pressure fact for self-heal credit, not a + * delivery. Undefined = no arm this round, or the arm was gated off + * before delivery was attempted (suppression / breaker / gate off) — + * consumers test for "injected" (+ "gated" where pressure counts). */ + nudgeOutcome?: "injected" | "vetoed" | "skipped" | "gated"; /** Render strategy the prepare used for processTurn ("none" for codex * compaction triggers / ACP_RENDER_NONE). The #422 fold-refresh hook in * forward() re-runs processTurn with the same strategy so the re-request @@ -3448,7 +3452,23 @@ export async function preflightCompressIfNeeded( } // The payload still overflows the window: fail fast with a diagnostic // error instead of forwarding a guaranteed-400 payload (#301). - const status = f?.kind === "upstream" && f.status === 429 ? 503 : 502; + // #3461 problem 2: DETERMINISTIC exhaustion (futile-round math proved no + // combination of folds can close the gap regardless of upstream health) + // can never succeed for this identical payload — answer it 413 like every + // other "can never fit as-is" verdict (#2847 lesson applied to the + // exhaustion path; #2921/#3146 doomed-bytes precedent) instead of 502, + // which status-code-driven headless hosts read as transient and spin on + // forever (field measurement: 5+ doomed rounds). Relay-dependent + // exhaustion (empty/truncated/incomplete summaries, budget draws, pool + // saturation) stays 502: those walks succeed once the relay recovers. + // AND only when bili's own calibrated meter agrees the payload is over the + // window (!payloadFitsWindow): when the calibrated estimate FITS but the + // walk still ran (char-count trigger arm, overflow-armed baseline), the + // doom is a relay-billing artifact bili cannot compute (#496/#767/#857 + // byte-relay lanes) — bili must not declare structural doom for it; those + // lanes keep their 502 fail-fast contract where NOT re-forwarding IS the + // protection, not a client spin. + const status = f?.kind === "upstream" && f.status === 429 ? 503 : f?.kind === "exhausted" && f.deterministic && !payloadFitsWindow ? 413 : 502; // #2189: an explicit retryable=false from preflight (credential-shape // rejection — deterministic for the identical payload) overrides the // status-based heuristic below; every other path keeps today's mapping. diff --git a/src/server/budget.ts b/src/server/budget.ts index 6bb7b6a8eb..68beb94a07 100644 --- a/src/server/budget.ts +++ b/src/server/budget.ts @@ -1,6 +1,8 @@ import { defaultCountTokens, type CoreMessage, type NudgeDecision } from "acp-kernel"; import { googleSystemText, type BiliMessage, type GoogleRequestBody } from "acp-kernel/wire"; import { estimateCoreMessages } from "../preflight.js"; +import { SELF_HEAL_SUPPRESS_CAP_MS, reductionKeyOf } from "../session-self-heal.js"; +import type { Session } from "../session.js"; import { applyEstimateCalibration } from "../util.js"; import { readOutputBudget, writeOutputBudget, type OutputBudgetField } from "./side-request.js"; @@ -15,6 +17,20 @@ const OUTPUT_CLAMP_MIN_MARGIN = 2048; const OUTPUT_CLAMP_FLOOR = 1024; // #3071/#2490: capability floor for the clamped budget — see clampOutputBudget. const OUTPUT_CLAMP_MIN_OUTPUT = 4096; +// #3384: giant-request line for applying the refused-dsh-ledger planning floor +// (#2490). A requested output budget at or above this fraction of the native +// window can push a near-wall raw ledger past the window in ONE turn — the +// #2490 shape (384K requested against a 1M window / ~900K ledger produced the +// 7.5MB un-foldable turn). Below the line no single-turn output can overflow +// the ledger, so applying the floor there only reserves output budget against +// bytes bili never sends (its folds keep the wire small while the client's raw +// ledger grows because compaction is refused) — the #3384 starvation: folded +// wire at 8–11% of a 1M window, budget clamped 50670 -> 5909, every byte eaten +// by thinking on an onlyReasoning model -> stopReason=length -> blank turns. +// Aligned with the #896 default outputHeadroomMaxPct (0.25) and the #2096 note +// below ("when max_tokens > 25% of the window the overflow boundary lies BELOW +// the headroom target"). +const OUTPUT_LEDGER_FLOOR_GIANT_REQUEST_PCT = 0.25; // #453 mitigation: host-side escalation line. The kernel already force-injects // every turn once usage >= nudge.maxContextLimitPct (its pressure branch has no // cadence gate), so the only cadence-silent zone is BELOW that line. Force the @@ -282,7 +298,10 @@ export function clampOutputBudget(requested: number, inputEstimate: number, nati * chain. Lifetime ends with the session (metadata eviction). Known * conservative residue: flipping allowDshCompaction on mid-session lets * checkpoints land and shrink the ledger, but the floor stays at its old - * high until eviction — fail-closed over-tightening, accepted. */ + * high until eviction — fail-closed over-tightening, accepted. + * Note (#3384): recording stays sticky; what changed is APPLICATION — the + * clamp gates where this floor may drive planning (see clampOutgoingOutput), + * so a stale high-water mark can no longer starve a healthy session. */ export function dshLedgerFloorTokens(metadata: Record | undefined | null): number { const bytes = metadata?.["dshCompactionRefusedBytes"]; if (typeof bytes !== "number" || !Number.isFinite(bytes) || bytes <= 0) return 0; @@ -331,6 +350,62 @@ export function emergencyNudge(nudge: NudgeDecision | null | undefined, escalati return !(typeof minBenefit === "number" && Number.isFinite(minBenefit) && minBenefit > 0 && (b.maxPending ?? 0) < minBenefit); } +/** #3461 problem 2: backoff for the EMERGENCY override. The pure check above + * has no cadence of its own — once usage sits over the escalation line it + * re-arms on EVERY request, so a headless client retrying a doomed request + * injected the tier-1 nudge ~7×/second (field measurement, dsh/pi lanes). A + * repeat injection carries no new information unless something changed that + * could make compression newly viable or newly needed, so this wrapper adds a + * per-session gate stamped on every ALLOWED injection. Re-arm grounds: + * - first injection under the current state (no gate yet / corrupt shape); + * - a reduction landed since (reductionKeyOf moved — same evidence self-heal + * D1 watches: committed fold → nextBlockId advanced, or plugin-lane + * compress tool stamped lastCompress): the payload shrank and still needs + * the prompt; + * - input grew by ≥ the flat growth step since (KDD #8 discipline; the step + * is loopConfig.nudge.growthFloor, flat 50K by default — NOT rescaled with + * the window): new content worth advising about; + * - wall-clock safety valve (SELF_HEAL_SUPPRESS_CAP_MS precedent, 10 min): + * one more injection so a permanently stuck session gets periodic reminders + * instead of silence forever. + * Kernel-driven injections (nudge.shouldInject=true) never pass through here — + * call sites OR them in BEFORE the emergency arm, so normal cadence is + * untouched. State lives in session.metadata.emergencyNudgeGate (survives + * restarts with the snapshot, visible on /__bili/sessions like selfHeal). */ +export function emergencyNudgeArmed( + session: Session, + nudge: NudgeDecision | null | undefined, + minCompressRange: number, + tokenCount: number, + growthStep?: number, +): boolean { + if (!emergencyNudge(nudge, undefined, minCompressRange)) return false; + const now = Date.now(); + const key = reductionKeyOf(session); + const raw = session.metadata["emergencyNudgeGate"]; + if (raw === null || typeof raw !== "object") { + session.metadata["emergencyNudgeGate"] = { at: now, tokens: tokenCount, reductionKey: key }; + return true; + } + const g = raw as Record; + const prevAt = typeof g["at"] === "number" ? g["at"] : NaN; + const prevTokens = typeof g["tokens"] === "number" ? g["tokens"] : NaN; + const prevKey = typeof g["reductionKey"] === "string" ? g["reductionKey"] : undefined; + const step = typeof growthStep === "number" && Number.isFinite(growthStep) && growthStep > 0 ? growthStep : 50_000; + let arm = false; + if (!Number.isFinite(prevAt) || !Number.isFinite(prevTokens) || prevKey === undefined) { + arm = true; // corrupt/legacy shape → re-stamp + } else if (prevKey !== key) { + arm = true; // a reduction landed since the last injection + } else if (tokenCount - prevTokens >= step) { + arm = true; // a full flat-growth step of new content + } else if (now - prevAt >= SELF_HEAL_SUPPRESS_CAP_MS) { + arm = true; // wall-clock safety valve + } + if (arm) session.metadata["emergencyNudgeGate"] = { at: now, tokens: tokenCount, reductionKey: key }; + return arm; +} + export function clampOutgoingOutput( rebuilt: Record, field: OutputBudgetField, @@ -341,13 +416,18 @@ export function clampOutgoingOutput( * window the overflow boundary lies BELOW the headroom target, so capping * against it would no-op exactly when post-compression turns need the * guarantee most (#453). */ headroomWindow?: number; - /** #2490: floor for the PLANNING input when the host (dsh native compaction) - * keeps a raw ledger bili's folded view cannot see. The guard's refusals - * record the replayed envelope size on the session; that ledger will be - * re-sent whole after this turn (and after every turn — the compaction - * never lands), so a turn planned only against the folded input invites - * output-sized overhang the next raw replay cannot survive. Raising the - * planning basis only ever tightens the cap (fail-closed direction). */ + /** #2490: floor for the PLANNING input when the host (dsh native compaction) + * keeps a raw ledger bili's folded view cannot see. The guard's refusals + * record the replayed envelope size on the session; that ledger will be + * re-sent whole after this turn (and after every turn — the compaction + * never lands), so a turn planned only against the folded input invites + * output-sized overhang the next raw replay cannot survive. Raising the + * planning basis only ever tightens the cap (fail-closed direction). + * #3384: APPLIED ONLY WHEN GATED — the floor engages iff the folded view + * already sits at the enforced headroom target OR the requested budget is + * giant (>= OUTPUT_LEDGER_FLOOR_GIANT_REQUEST_PCT of the window); below + * both lines it would reserve output budget against bytes never sent and + * starve healthy sessions (see clampOutgoingOutput). */ ledgerFloorTokens?: number; /** #2122/#1933 F1: per-route estimator calibration — scales the local-est * side of estimateInputTokens so the clamp judges the payload on the same @@ -373,7 +453,24 @@ export function clampOutgoingOutput( // internal max() (usage baseline never scales DOWN), so all three calibers stay // conservative in the same direction. const ledgerFloor = typeof ctx.ledgerFloorTokens === "number" && Number.isFinite(ctx.ledgerFloorTokens) ? Math.max(0, Math.floor(ctx.ledgerFloorTokens)) : 0; - const planningInput = Math.max(inputEstimate, ledgerFloor); + // #3384: the floor only applies where it can still defend something — two + // independent grounds, either suffices: + // - viewAtEnforcedTarget: bili's OWN folded view has reached the + // headroom-adjusted target nudge/preflight enforce — the session really + // is at the wall, so planning against the worst-case ledger is right; + // - giantRequestedBudget: the requested output alone can push a near-wall + // ledger past the window in one turn (the #2490 shape). + // Otherwise the floor is pure over-reservation: the client's raw ledger only + // grows because bili refuses compaction, while the wire bili actually sends + // stays small via its folds — reserving output budget against bytes never + // sent starves the session's own replies (#3384: folded wire 8–11% of a 1M + // window, budget clamped to 5909, all eaten by thinking -> blank turns). + // Both pinned protection regimes survive: the #2490 incident request (384K + // >= 25% of 1M) takes giantRequestedBudget; near-wall sessions take + // viewAtEnforcedTarget. + const viewAtEnforcedTarget = typeof ctx.headroomWindow === "number" && Number.isFinite(ctx.headroomWindow) && ctx.headroomWindow > 0 && inputEstimate >= ctx.headroomWindow; + const giantRequestedBudget = ctx.nativeWindow > 0 && raw >= OUTPUT_LEDGER_FLOOR_GIANT_REQUEST_PCT * ctx.nativeWindow; + const planningInput = viewAtEnforcedTarget || giantRequestedBudget ? Math.max(inputEstimate, ledgerFloor) : inputEstimate; const capped = clampOutputBudget(raw, planningInput, ctx.nativeWindow); if (capped !== undefined) { writeOutputBudget(rebuilt, field, capped); diff --git a/src/server/handle.ts b/src/server/handle.ts index 372f447277..de34135e3f 100644 --- a/src/server/handle.ts +++ b/src/server/handle.ts @@ -47,7 +47,7 @@ import { BILI_PASSTHROUGH_HEADER, BILI_PLUGIN_BYPASS_HEADER, isLoopbackAddress, import { checkTunnelDestination, tunnelAllowlistFromEnv } from "../tunnel-guard.js"; import { resolveCompatDropFields } from "../compat-drop.js"; import { DecompressedTooLargeError, decodeRequestBody } from "../content-encoding.js"; -import { autoModeForgeEnabled, limitsDecodedBytes as knobLimitsDecodedBytes, noInjectTool as knobNoInjectTool, rawDumpDir as knobRawDumpDir } from "../knobs.js"; +import { autoModeForgeEnabled, imageRescaleEnabled, limitsDecodedBytes as knobLimitsDecodedBytes, noInjectTool as knobNoInjectTool, rawDumpDir as knobRawDumpDir } from "../knobs.js"; import { buildForgeResponse, classifierForgeVariant, estimateCjkTokens, isAutoModeClassifier, requestText, summarizeAction } from "../auto-mode-forge.js"; import { extractConversationId, isV1BackgroundSideRequest, isV1InternalRequest, unwrapV1Internal } from "../v1internal.js"; import { anthropicBetaContextWindow, BILI_HOP_HEADER, capRegistryWindowByStandard, expandedContextSuffixWindow, launcherContextWindow, launcherMaxOutput, windowSourceLogged } from "./context-window.js"; @@ -64,7 +64,7 @@ import { prepareResponses, prepareResponsesCompact } from "./prepare-responses.j import { bodyDumpEnabled, isModelDiscoveryPath, isV1InternalControlPlanePath, logDumpFailure, logUnrecognizedPath } from "./observability.js"; import { adminGateDenial, handleAdminRoute, isManagementPath } from "./admin.js"; import { handleIncrementalTurn, handleIncrementalProbe, incrementalTurnBodyShape, incrementalTurnsEnabled, INCREMENTAL_TURN_MARKER_HEADER, INCREMENTAL_TURN_PATH } from "../incremental-lane.js"; -import { ccrPluginWireOk, storeEffectiveCcr, type CcrSettings } from "../store.js"; +import { ccrPluginWireOk, storeEffectiveCcr, storeEffectiveImageRescale, type CcrSettings } from "../store.js"; import { DEFAULT_DECIDE_MAX_TOKENS } from "../nudge-decide.js"; import { BodyTooLargeError, forward, forwardUpstreamUrl, googleModelFromPath, googlePathKind, headroomEffectiveLogged, headroomFallbackLogged, headerValue, imageBillingFor, imageReserveFor, imageTokenCapFor, isCountTokensRequest, isPreflightFailFast, logRequestCost, NON_CONVERSATION_RELAY_WARN_CAP, nonConversationRelayWarned, prepareCountTokens, preflightCompressIfNeeded, readBody, resolveKnownOutputCeiling, resolveUpstream, resignSettingsFor, selfAddressedLocalPath, scrubAnthropicPck, scrubCompatDrop, vertexAnthropicModelFromPath, warnRouteMissIfNew, type Prepared } from "../server.js"; import { normalizeVertexModelId } from "../vertex.js"; @@ -1723,6 +1723,16 @@ async function handleRequest( // placeholder is ever emitted; regular compress/decompress folding is unaffected // (the codex lane simply keeps large tool results verbatim until they are folded). storeEffectiveCcr(session, opts.compress.injectTool && storeChannelOk && !(protocol === "responses" && isCodexClient(req.headers)) ? pluginCcrStamp : undefined); + // #3400: read_original_image arms where the model can actually reach + // it — the injection lanes (proxy owns tool execution) or the plugin + // lanes (the agent registers it natively from handlePluginManifest, + // base-config gated like every other opt-in tool, #1192). Never on the + // codex lane: it rejects bili-injected tools ("unsupported call", + // #3270), so there the annotation degrades to size-only. + const imageRescaleArmed = imageRescaleEnabled() + && !(protocol === "responses" && isCodexClient(req.headers)) + && ((opts.compress.injectTool && storeChannelOk) || pluginMode); + storeEffectiveImageRescale(session, imageRescaleArmed || undefined); // [#1336] no channel gating: search_context is already available on // whichever mode served this session and the re-rank is pure output- // side policy on its result — both proxy and plugin lanes apply it. @@ -2347,13 +2357,18 @@ async function handleRequest( // model never received. Actual injection subsumes every factor // the old expression listed: gated-off rounds (suppression / // breaker / gate off / no arm) leave it undefined or yield - // "vetoed"/"skipped" — never "injected". + // "vetoed"/"skipped" — never "injected". #3461: "gated" joins + // "injected" as active — a pressure round held only by the + // emergency backoff still counts, otherwise the rate limit + // starves the streaks (#2155 zombie degrade, D1 suppression) + // that exist precisely for sessions pinned under continuous + // pressure. if (!selfHealRoundDone && prepared.nudge !== undefined) { selfHealRoundDone = true; evaluateSelfHealRound(session, { pluginHeaderPresent: pluginAgentHeader(req.headers) !== undefined, biliToolsDeclared: biliToolsDeclaredOnWire(parsed, protocol), - nudgeActive: prepared.nudgeOutcome === "injected", + nudgeActive: prepared.nudgeOutcome === "injected" || prepared.nudgeOutcome === "gated", biliToolUses: countBiliToolUses(prepared.processedMessages), degradeAvailable: opts.compress.injectTool && !knobNoInjectTool(), }, log); diff --git a/src/server/inject.ts b/src/server/inject.ts index c7316e9ce5..6c342f7102 100644 --- a/src/server/inject.ts +++ b/src/server/inject.ts @@ -3,7 +3,8 @@ import { applyAcpToolOverrides, defaultPrompts } from "acp-kernel"; import type { ProxyOptions } from "../config.js"; import { type AnthropicRequestBody, type GoogleFunctionDeclaration, type GoogleTool, type OpenAITool } from "acp-kernel/wire"; import { appendSystemText } from "../util.js"; -import { BILI_ACP_TOOLS_ANTHROPIC, BILI_ACP_TOOLS_ANTHROPIC_NO_RANGE, BILI_ACP_TOOLS_GOOGLE, BILI_ACP_TOOLS_GOOGLE_NO_RANGE, BILI_ACP_TOOLS_OPENAI, BILI_ACP_TOOLS_OPENAI_NO_RANGE, BILI_ACP_TOOLS_RESPONSES, buildAcpTagsOnlyPrompt, buildCompressSystemPrompt, packOwnsDescription, ptcNoteMode, withAcpNotes, withAcpNotesOpenAI, withMarkerIntegrityNote, withSummaryBudgetNote, withProjectAttribution } from "../compress-tool.js"; +import { absorbEnabled, absorbToolName } from "../absorb.js"; +import { BILI_ACP_TOOLS_ANTHROPIC, BILI_ACP_TOOLS_ANTHROPIC_NO_RANGE, BILI_ACP_TOOLS_GOOGLE, BILI_ACP_TOOLS_GOOGLE_NO_RANGE, BILI_ACP_TOOLS_OPENAI, BILI_ACP_TOOLS_OPENAI_NO_RANGE, BILI_ACP_TOOLS_RESPONSES, buildAbsorbSystemPrompt, buildAcpTagsOnlyPrompt, buildCompressSystemPrompt, packOwnsDescription, ptcNoteMode, withAcpNotes, withAcpNotesOpenAI, withMarkerIntegrityNote, withSummaryBudgetNote, withProjectAttribution } from "../compress-tool.js"; import { externalSummaryEnabled, withExternalSummaryTools } from "../external-summary-surface.js"; import { forceTextProtocol as knobForceTextProtocol, renderNone as knobRenderNone } from "../knobs.js"; @@ -27,6 +28,7 @@ export function injectSystem( const tagsOnly = buildAcpTagsOnlyPrompt("function", prompts, surface?.promptSections); if (tagsOnly) parts.push(tagsOnly); } + if (opts.compress.injectTool && absorbEnabled(config)) parts.push(buildAbsorbSystemPrompt(absorbToolName(config))); if (parts.length === 0) return parsed.system; // #1876: APPEND the prompt as a trailing block (client blocks byte-exact) // instead of merging everything into one block via kernel buildSystem. diff --git a/src/server/lane-decisions.ts b/src/server/lane-decisions.ts index ebd619e01e..c1f5f77710 100644 --- a/src/server/lane-decisions.ts +++ b/src/server/lane-decisions.ts @@ -23,51 +23,17 @@ */ import { biliToolsDeclaredOnWire } from "../session-self-heal.js"; +import { extractBillingAttributionBlock } from "../preflight.js"; +import { ANONYMOUS, hostProfileFor, type HostProfile } from "../host-profiles.js"; type LaneProtocol = "anthropic" | "openai" | "google" | "responses"; -// ① HostProfile registry — the ONLY place lane names are compared (#3417). -// Properties here are per-AGENT constants. Anything that can vary per -// environment/profile/request belongs in ② WireObservation instead. -interface HostProfile { - readonly identity: string; - /** - * Declared default: does this host register bili's ACP tools host-scope - * (invisible to the model) by default? dsh does (#3177: 0/3939 preset - * requests carried the tools); pi/omp/opencode expose them visibly. The - * observation layer overrides this when the request itself declares the - * tools (#3401: some dsh profiles DO surface them). - */ - readonly toolsHostScopedByDefault: boolean; - /** - * Declared: this host's upstream requests need the billing block carried - * from the client request (#2189, subscription-OAuth lanes 429 without it). - */ - readonly wantsBillingBlock: boolean; - /** - * Declared: this host compacts histories CLIENT-SIDE and resends the - * compacted form (dsh native compaction, #2432) — the four prepares run - * the dual-signal compact-guard rebase for such hosts. Registry property, - * not a name comparison in each adapter (#3417). - */ - readonly compactsClientSide: boolean; -} - -const PROFILES: Record = { - dsh: { identity: "dsh", toolsHostScopedByDefault: true, wantsBillingBlock: true, compactsClientSide: true }, - pi: { identity: "pi", toolsHostScopedByDefault: false, wantsBillingBlock: false, compactsClientSide: false }, - omp: { identity: "omp", toolsHostScopedByDefault: false, wantsBillingBlock: false, compactsClientSide: false }, - opencode: { identity: "opencode", toolsHostScopedByDefault: false, wantsBillingBlock: false, compactsClientSide: false }, -}; - -const ANONYMOUS: HostProfile = { identity: "", toolsHostScopedByDefault: false, wantsBillingBlock: false, compactsClientSide: false }; - -/** Resolve the declared profile for a plugin agent header value. Trimmed - * exact match — the historical shape of every pluginAgent comparison. */ -function resolveHostProfile(pluginAgent: string | undefined | null): HostProfile { - const key = (pluginAgent ?? "").trim(); - return PROFILES[key] ?? { ...ANONYMOUS, identity: key }; -} +// ① HostProfile registry lives in ../host-profiles.ts — an import-free leaf +// so host-side agent bundles can read declared defaults without dragging +// this module's server dependencies (preflight → kernel) across the proxy +// boundary (#3482 review). Re-exported for consumers already on this path. +export { hostProfileFor } from "../host-profiles.js"; +export type { HostProfile } from "../host-profiles.js"; // ② WireObservation — facts read off THIS request body. interface WireObservation { @@ -75,6 +41,12 @@ interface WireObservation { * the nudge advertises `compress`, so THAT name decides reachability, * #3403 review follow-up). */ readonly declaredTools: ReadonlySet; + /** #3482: does this request's system carry a billing-attribution block + * (#2189 shape — `x-anthropic-billing-header:` text block)? Observation, + * not name-derivation: any lane's request that actually carries one + * keeps it honored (the same evidence-over-defaults override as + * toolsReachable, #3401). */ + readonly billingBlockObserved: boolean; } function observeDeclaredTools(parsed: unknown, protocol: LaneProtocol): ReadonlySet { @@ -104,6 +76,54 @@ function observeDeclaredTools(parsed: unknown, protocol: LaneProtocol): Readonly return out; } +// ①b WireClientProfile — proxy-lane clients carry no pluginAgent header; +// their identity is OBSERVED from wire signals. Registry discipline as ①: +// the wire-client header-name literals live ONLY here (the old +// variable-mediated comparisons in session-id.ts were un-greppable by +// design — #3482 Step 2 site 4 pins them here). Shared detection +// primitives (isCodexClient, the codex UA definition) stay where they are +// — they feed many decisions and are exempt by the #3417 guard's design +// note; the DECISION table moves here. +interface WireClientSignals { + /** Header NAMES whose value is a non-empty trimmed string. */ + readonly headerNames: ReadonlySet; + /** The conversation-header source that won precedence (name only). */ + readonly conversationSourceName: string | undefined; + /** The shared codex UA/header detection (isCodexClient). */ + readonly codexUserAgent: boolean; +} + +interface WireClientEntry { + readonly profile: HostProfile; + readonly matches: (s: WireClientSignals) => boolean; +} + +const WIRE_CLIENTS: readonly WireClientEntry[] = [ + { + // codex (#150): task-level session ids are REUSED across tasks — + // different task instructions under one id must not merge. Modern + // codex isolates subagent threads via the turn-metadata header + // (keyed verbatim there); the fingerprint still matters for root + // threads and older builds (same evidence as isCodexClient, #645). + profile: { ...ANONYMOUS, identity: "codex", instructionsFingerprintKeyed: true }, + matches: (s) => s.headerNames.has("x-codex-turn-metadata") || s.codexUserAgent, + }, + { + // claude over Responses (#970): subagents SHARE the main agent's + // session id (the anthropic path splits them via claudeSubagentSplit; + // a claude client on the Responses wire only has this fingerprint). + profile: { ...ANONYMOUS, identity: "claude-responses", instructionsFingerprintKeyed: true }, + matches: (s) => s.conversationSourceName === "x-claude-code-session-id", + }, +]; + +/** Resolve the wire-client profile from observed signals (first match wins; + * unknown clients resolve ANONYMOUS — instructions key VERBATIM for them, + * #1106). */ +export function resolveWireClientProfile(signals: WireClientSignals): HostProfile { + for (const entry of WIRE_CLIENTS) if (entry.matches(signals)) return entry.profile; + return ANONYMOUS; +} // ③ LaneDecisions — the frozen decision object every gate reads. interface LaneDecisions { /** The resolved profile (identity + declared defaults). */ @@ -129,6 +149,12 @@ interface LaneDecisions { /** Legacy alias kept for the four prepares' migration: TRUE when the * surface must be OFF (the old `dshCompressSurfaceOff` variable). */ readonly compressSurfaceOff: boolean; + /** #3482 site 1: should this lane's summary side-calls carry the billing + * attribution block extracted from the client system (#2189)? The dsh + * profile declares it (subscription-OAuth lanes 429 without it); any + * request observed carrying the block keeps it honored regardless of + * profile — evidence over defaults, same shape as toolsReachable. */ + readonly billingBlockWanted: boolean; } interface LaneDecisionInput { @@ -139,9 +165,55 @@ interface LaneDecisionInput { /** injectTool config (`shouldInject`) — the config-level master switch. */ readonly shouldInject: boolean; } +// ④ nudgeOn — the ONE nudge-injection gate (#3482 Step 2.5). The four +// prepares used to carry four hand-copied conjunctions that drifted (#3293: +// anthropic lost the injectTool conjunct entirely). The deliberate +// per-lane differences are EXPLICIT inputs, pinned by equivalence tests: +// - `compactionTrigger` — responses only (#2155/#2432 family): a +// compaction-trigger request never carries the nudge; the other three +// wires have no trigger concept and pass `false`. +// - `growthFoldingArmed` — callers arm it with the codex-intercept health +// window (`modelContextLimit × CODEX_COMPACT_HEALTH_RATIO`) when that +// mode is live on the responses lane; all other callers pass the +// windowless variant. +// - `compressSurfaceOn` — the lane-decisions surface (this is where the +// old `shouldInject` conjunct lives for every lane; the standalone +// copies in openai/google were redundant — compressSurfaceOn already +// implies injectTool). +export interface NudgeGateInputs { + /** opts.compress.injectNudge — the config-level master switch. */ + readonly injectNudge: boolean; + /** !!turn.nudge — the kernel produced a nudge plan this turn. */ + readonly hasNudgePlan: boolean; + /** laneDecisions.compressSurfaceOn. */ + readonly compressSurfaceOn: boolean; + /** responses: hasCompactionTrigger(parsed.input); others: false. */ + readonly compactionTrigger: boolean; + /** !nudgeSuppressed(session) || decideFallbackOverride — the decide + * ladder's fallback is the ONE exception that punches through the + * suppression latch (#3004 dead-lock guard). */ + readonly suppressionBypassed: boolean; + /** compressBreakerArmed(session). */ + readonly breakerArmed: boolean; + /** autoFoldEngaged(loopConfig, session). */ + readonly autoFoldEngaged: boolean; + /** growthFoldingArmed(loopConfig, healthWindow?) — precomputed with the + * lane's health window. */ + readonly growthFoldingArmed: boolean; + /** turn.nudge.shouldInject || emergencyNudge(turn.nudge, …) — the model + * actually wants compression (or is past the emergency bar). */ + readonly modelWantsIt: boolean; +} + +export function nudgeOn(i: NudgeGateInputs): boolean { + return i.injectNudge && i.hasNudgePlan && i.compressSurfaceOn && !i.compactionTrigger + && i.suppressionBypassed && !i.breakerArmed + && !(i.autoFoldEngaged && i.growthFoldingArmed) + && i.modelWantsIt; +} export function resolveLaneDecisions(input: LaneDecisionInput): LaneDecisions { - const profile = resolveHostProfile(input.pluginAgent); + const profile = hostProfileFor(input.pluginAgent); const declaredTools = observeDeclaredTools(input.parsed, input.protocol); // Reachability truth: in PROXY mode bili itself injects the tool // definitions into the outbound body, so the surface is structurally on. @@ -153,11 +225,21 @@ export function resolveLaneDecisions(input: LaneDecisionInput): LaneDecisions { || declaredTools.has("compress") || !profile.toolsHostScopedByDefault; const compressSurfaceOn = input.shouldInject && compressReachable; + const billingBlockObserved = extractBillingAttributionBlock(systemOf(input.parsed)) !== undefined; + const billingBlockWanted = profile.wantsBillingBlock || billingBlockObserved; return Object.freeze({ profile, - observed: Object.freeze({ declaredTools }), + observed: Object.freeze({ declaredTools, billingBlockObserved }), toolsReachable: Object.freeze({ compress: compressReachable }), compressSurfaceOn, compressSurfaceOff: !compressSurfaceOn, + billingBlockWanted, }); } + +function systemOf(parsed: unknown): string | Array> | undefined { + if (parsed === null || typeof parsed !== "object") return undefined; + const system = (parsed as Record)["system"]; + if (typeof system === "string" || Array.isArray(system)) return system; + return undefined; +} diff --git a/src/server/prepare-anthropic.ts b/src/server/prepare-anthropic.ts index 451a94bdc2..67d84035b9 100644 --- a/src/server/prepare-anthropic.ts +++ b/src/server/prepare-anthropic.ts @@ -14,8 +14,9 @@ import { nudgeSuppressed } from "../session-self-heal.js"; import { compressBreakerArmed } from "../stream.js"; import { applyCompactionArchive, coveredRealHistoryIds, detectUnannouncedHistoryRewrite, foldCoverage, markCompactionBoundary, markDirty, markNativeCompactionBoundary, reconcileNativeCompactionBoundary, REWRITE_MIN_INCOMING_TOTAL, snapshotMessages, type PendingRetrieval, type Session } from "../session.js"; import { carriesDshLocalCompactionSummary, DSH_LOCAL_COMPACTION_MIN_MISSING } from "./dsh-compaction-guard.js"; -import { RULE_TOOL, retrieveToolsFor, withFirstSightDrain, withMarkerIntegrityNote, withSummaryBudgetNote } from "../compress-tool.js"; -import { adoptContentStore, ccrEnabled, ccrLoopConfig, contentStoreOf, dropRetrievals, pruneExpiredRetrievals, reconcileReloadedRetrievals, renderRetrievalNotes, retrieveToolName, snapshotPendingRetrievals, snapshotRetrievalNotes } from "../store.js"; +import { ABSORB_TOOL_NAME, READ_ORIGINAL_IMAGE_TOOL, RULE_TOOL, absorbToolsFor, retrieveToolsFor, withFirstSightDrain, withMarkerIntegrityNote, withSummaryBudgetNote } from "../compress-tool.js"; +import { applyAbsorbView, storeEffectiveAbsorb } from "../absorb.js"; +import { adoptContentStore, ccrEnabled, ccrLoopConfig, contentStoreOf, dropRetrievals, imageRescaleToolArmed, pruneExpiredRetrievals, reconcileReloadedRetrievals, renderRetrievalNotes, retrieveToolName, snapshotPendingRetrievals, snapshotRetrievalNotes } from "../store.js"; import { rulesEnabled, storeEffectiveRules } from "../rules-feature.js"; import { autoFoldEngaged, externalSummaryEnabled, growthFoldingArmed } from "../external-summary-surface.js"; import { attachSubagentSessions } from "../subagent-sessions.js"; @@ -25,11 +26,13 @@ import { anthropicToolsCarryCacheControl, computeAnthropicMessageMarks, stampAnt import { reconcileSystemAnchor } from "../system-anchor.js"; import { isStrictReasoningEcho, modelIdOf } from "../strict-echo.js"; import { logRawStripCounts, markerArtifactCount, stripRawStage } from "../marker/engine.js"; -import { renderNone as knobRenderNone } from "../knobs.js"; +import { imageRescaleMaxDim, renderNone as knobRenderNone } from "../knobs.js"; +import { applyImageRescalePass } from "../image-rescale.js"; import { isAutoModeClassifier } from "../auto-mode-forge.js"; -import { countSystemAndToolsTokens, emergencyNudge, projectThinkingMass } from "./budget.js"; +import { countSystemAndToolsTokens, emergencyNudge, emergencyNudgeArmed, projectThinkingMass } from "./budget.js"; +import { effectiveAbsorbBlock } from "./prepare-responses.js"; import { injectSystem, injectTool } from "./inject.js"; -import { resolveLaneDecisions } from "./lane-decisions.js"; +import { nudgeOn, resolveLaneDecisions } from "./lane-decisions.js"; export async function prepareAnthropic( parsed: AnthropicRequestBody, @@ -65,7 +68,6 @@ export async function prepareAnthropic( parsed, shouldInject: opts.compress.injectTool, }); - const dshCompressSurfaceOff = laneDecisions.compressSurfaceOff; // #3222: #651's reasoning-drop plan, computed on the INBOUND history before // core.processTurn — see planReasoningDrop for why the post-turn view is too late. let reasoningPlan = new Set(); @@ -168,7 +170,12 @@ export async function prepareAnthropic( // #2189: from clientSystem (pre-anchor) — the anchor buildSystem below may // collapse the block array, and subscription-OAuth upstreams 429 summary // calls whose system lacks this block. - const anthropicBillingBlock = extractBillingAttributionBlock(clientSystem); + // #3482 site 1: the billing-attribution block (#2189) moves with the + // session summary calls — policy in the registry (dsh declares it), + // observation over defaults: a request observed carrying the block keeps + // it honored on every lane. Extraction stays here (anthropic is the only + // wire whose system is a block array); the decision is lane-decisions'. + const anthropicBillingBlock = laneDecisions.billingBlockWanted ? extractBillingAttributionBlock(clientSystem) : undefined; // Plugin-mode agents own their context management and may already apply // their own cache-friendly head handling (#1085 scope: plain-proxy mode // only) — anchoring them would double-process. @@ -219,14 +226,23 @@ export async function prepareAnthropic( // compression trigger fires until overflow. See effectiveTokenCount. const { tokens: tokenCount, source: tokenCountSource } = effectiveTokenCount(session, msgs, inboundImageTokens); const activeBefore = new Set(session.state.blocks.filter((b) => b.active).map((b) => b.blockId)); + // Absorb markers are injected by the kernel's processTurn from + // config.absorb. With no channel to call the tool (injection off), + // strip absorb from the loop config so the REQUIRED instruction never + // reaches the wire. Hiding recorded absorptions is unaffected + // (applyAbsorbView hides regardless of enablement). + const absorbBlock = effectiveAbsorbBlock(pluginMode, config, opts.compress.absorb); + const absorbTools = absorbToolsFor(absorbBlock?.toolName ?? ABSORB_TOOL_NAME); + const absorbActive = absorbBlock?.enabled === true && opts.compress.injectTool; // acp_rule has no processTurn side effect (no markers/instructions are - // ever injected into messages) — only tool availability matters. + // ever injected into messages), so unlike absorb it needs no loop- + // config stripping — only tool availability matters. const rulesActive = rulesEnabled(config) && opts.compress.injectTool; // [#1097] the kernel ccr-store node ID-references oversized tool results // BEFORE absorb (ID-reference wins over distill); armed policy is // stamped per-request — strip `ccr` from the loop config when disarmed // (plugin mode / no tool channel) so placeholders never hit the wire. - const loopConfig = ccrLoopConfig(session, config); + const loopConfig = ccrLoopConfig(session, { ...config, absorb: absorbActive ? absorbBlock : undefined }); // #2432/#2658: dsh desktop's native compaction can LAND without ever // transiting bili — its summarizer calls ctx.llm.stream() directly and // manual /compact / idle-session paths have no ALS attribution, so the @@ -282,7 +298,9 @@ export async function prepareAnthropic( const gap = foldCoverage(foldCoveredBefore, msgs.map((m) => m.id)); if (gap) log("warn", `[${sessionId}] [acp-drift] fold coverage mismatch: ${gap.matched}/${gap.expected} covered message id(s) present in resent history — ${gap.expected - gap.matched} covered id(s) missing from resent history — mutation (content edit invalidates content-hash refs, fold silently lost) or client-side deletion/truncation (benign, message no longer on the wire); observability complement to the #1328 overflow rescue (#1195)`); } + storeEffectiveAbsorb(session, loopConfig); storeEffectiveRules(session, config); + turn.messages = applyAbsorbView(turn.messages, session.state, loopConfig, tokenCount); turn.messages = attachSubagentSessions(turn.messages, session); // Drop sub-viability fragments before any consumer sees them: a tiny // range in the list makes batched compress attempts fail atomically @@ -311,7 +329,25 @@ export async function prepareAnthropic( // consecutive decision failures. const decideTier1Arm = decide !== undefined && !!turn.nudge && turn.nudge.tier === 1 && turn.nudge.breakdown.emergencyOverride !== 1; const decideFallbackOverride = decideTier1Arm && ladderMode(session.metadata) === "fallback"; - const willInjectNudge = opts.compress.injectNudge && !!turn.nudge && !dshCompressSurfaceOff && (!nudgeSuppressed(session) || decideFallbackOverride) && !compressBreakerArmed(session) && !(autoFoldEngaged(loopConfig, session) && growthFoldingArmed(loopConfig)) && (turn.nudge.shouldInject || emergencyNudge(turn.nudge, undefined, loopConfig.compress.minCompressRange)); + const nudgeEligible = opts.compress.injectNudge && !!turn.nudge && laneDecisions.compressSurfaceOn && (!nudgeSuppressed(session) || decideFallbackOverride) && !compressBreakerArmed(session) && !(autoFoldEngaged(loopConfig, session) && growthFoldingArmed(loopConfig)); + // #3461: pressure admitted by every standing gate but held by the backoff + // is still an active nudge round — self-heal credit (#2155/#3004) must + // not starve because we chose not to re-inject this round. + const emergencyPressureHeld = nudgeEligible && !!turn.nudge && !turn.nudge.shouldInject && emergencyNudge(turn.nudge, undefined, loopConfig.compress.minCompressRange); + // #3482 Step 2.5: the ONE nudge gate. This lane has no compaction + // trigger and no intercept health window — those differences are + // explicit inputs on the responses lane. + const willInjectNudge = nudgeOn({ + injectNudge: opts.compress.injectNudge, + hasNudgePlan: !!turn.nudge, + compressSurfaceOn: laneDecisions.compressSurfaceOn, + compactionTrigger: false, + suppressionBypassed: !nudgeSuppressed(session) || decideFallbackOverride, + breakerArmed: compressBreakerArmed(session), + autoFoldEngaged: autoFoldEngaged(loopConfig, session), + growthFoldingArmed: growthFoldingArmed(loopConfig), + modelWantsIt: turn.nudge !== undefined && (turn.nudge.shouldInject || emergencyNudgeArmed(session, turn.nudge, loopConfig.compress.minCompressRange, tokenCount, loopConfig.nudge.growthFloor)), + }); log("info", diagNudge(turn, sessionId, tokenCount, config.modelContextLimit, resolvedModel ?? parsed.model, willInjectNudge, decideTier1Arm && !decideFallbackOverride)); processedMessages = applyReasoningDrop(stripKernelSummaries(turn.messages, turn.state), reasoningPlan, log, sessionId); // #1001: a silent client history rewrite takes the same archive+prune path @@ -359,12 +395,15 @@ export async function prepareAnthropic( // map is the fallback the rebuild applies when bili adds no marks of its // own (a client-managed session), so its trigger-turn breakpoint survives. clientCacheControls = cacheControls; - rebuiltMessages = coreToAnthropic(processedMessages as BiliMessage[], anthropicCacheMarks ?? cacheControls); + // #3400: egress image rescale — outbound copies only; the originals + // stay in session state/persistence untouched. + const egressMessages = await applyImageRescalePass(processedMessages as BiliMessage[], { session, sessionId, log, maxDim: imageRescaleMaxDim(), toolArmed: imageRescaleToolArmed(session) }); + rebuiltMessages = coreToAnthropic(egressMessages, anthropicCacheMarks ?? cacheControls); if (sysNotes.length > 0) { rebuiltMessages = [...rebuiltMessages, ...sysNotes.map((text) => ({ role: "user" as const, content: text }))]; } - systemOut = injectSystem(parsed, opts, prompts, loopConfig, surface, visibilityMarkers, dshCompressSurfaceOff); + systemOut = injectSystem(parsed, opts, prompts, loopConfig, surface, visibilityMarkers, laneDecisions.compressSurfaceOff); // #1637: stamp the steady body's system. NOTE: parsed.system stays the // #1085 frozen CLIENT head (injectSystem's in-place view) — the round-2 // adapter consumes that frozen head and stamps its own rebuild, so NO @@ -372,7 +411,7 @@ export async function prepareAnthropic( // the compress prompt on round-2, the exact F2 seam the matrix pins). systemOut = stampAnthropicSystemCacheControl(systemOut, anthropicCacheMarks !== undefined); if (injectTools) { - toolsOut = injectTool(parsed.tools, [...(rulesActive ? [RULE_TOOL] : []), ...(ccrEnabled(session) ? [retrieveToolsFor(retrieveToolName(session)).anthropic] : [])], surface?.toolPrompts, ccrEnabled(session), externalSummaryEnabled(config)); + toolsOut = injectTool(parsed.tools, [...(absorbActive ? [absorbTools.anthropic] : []), ...(rulesActive ? [RULE_TOOL] : []), ...(ccrEnabled(session) ? [retrieveToolsFor(retrieveToolName(session)).anthropic] : []), ...(imageRescaleToolArmed(session) ? [READ_ORIGINAL_IMAGE_TOOL] : [])], surface?.toolPrompts, ccrEnabled(session), externalSummaryEnabled(config)); } // Nudge as a separate trailing user message (cache-friendly): the // system block stays byte-stable so the prefix cache survives. @@ -437,6 +476,8 @@ export async function prepareAnthropic( nudgeOutcome = "skipped"; } } + } else if (emergencyPressureHeld) { + nudgeOutcome = "gated"; } // [#1343/#1457] surface any earlier undelivered retrieve as an ephemeral // trailing user note (kept last so it never reorders cached messages). diff --git a/src/server/prepare-google.ts b/src/server/prepare-google.ts index d86958793b..4a456efd19 100644 --- a/src/server/prepare-google.ts +++ b/src/server/prepare-google.ts @@ -13,18 +13,21 @@ import { compressBreakerArmed } from "../stream.js"; import { applyCompactionArchive, coveredRealHistoryIds, detectUnannouncedHistoryRewrite, foldCoverage, markCompactionBoundary, markDirty, markNativeCompactionBoundary, reconcileNativeCompactionBoundary, REWRITE_MIN_INCOMING_TOTAL, snapshotMessages, type Session } from "../session.js"; import { recordConflict } from "../conflict-watch.js"; import { carriesDshLocalCompactionSummary, DSH_LOCAL_COMPACTION_MIN_MISSING } from "./dsh-compaction-guard.js"; -import { RULE_TOOL_GOOGLE, buildAcpTagsOnlyPrompt, buildCompressSystemPrompt, retrieveToolsFor, withFirstSightDrain, withMarkerIntegrityNote, withSummaryBudgetNote, withProjectAttribution } from "../compress-tool.js"; -import { adoptContentStore, ccrEnabled, ccrLoopConfig, contentStoreOf, retrieveToolName } from "../store.js"; +import { ABSORB_TOOL_NAME, READ_ORIGINAL_IMAGE_TOOL_GOOGLE, RULE_TOOL_GOOGLE, absorbToolsFor, buildAbsorbSystemPrompt, buildAcpTagsOnlyPrompt, buildCompressSystemPrompt, retrieveToolsFor, withFirstSightDrain, withMarkerIntegrityNote, withSummaryBudgetNote, withProjectAttribution } from "../compress-tool.js"; +import { absorbToolName, applyAbsorbView, storeEffectiveAbsorb } from "../absorb.js"; +import { adoptContentStore, ccrEnabled, ccrLoopConfig, contentStoreOf, imageRescaleToolArmed, retrieveToolName } from "../store.js"; import { rulesEnabled, storeEffectiveRules } from "../rules-feature.js"; import { autoFoldEngaged, externalSummaryEnabled, growthFoldingArmed } from "../external-summary-surface.js"; import { attachSubagentSessions } from "../subagent-sessions.js"; import { reconcileSystemAnchor } from "../system-anchor.js"; import { logRawStripCounts, markerArtifactCount, stripRawStage } from "../marker/engine.js"; -import { renderNone as knobRenderNone } from "../knobs.js"; -import { clampOutgoingOutput, countSystemAndToolsTokens, dshLedgerFloorTokens, emergencyNudge } from "./budget.js"; +import { imageRescaleMaxDim, renderNone as knobRenderNone } from "../knobs.js"; +import { applyImageRescalePass } from "../image-rescale.js"; +import { clampOutgoingOutput, countSystemAndToolsTokens, dshLedgerFloorTokens, emergencyNudge, emergencyNudgeArmed } from "./budget.js"; import { estimateCoreMessages } from "../preflight.js"; +import { effectiveAbsorbBlock } from "./prepare-responses.js"; import { injectGoogleTool, injectTool } from "./inject.js"; -import { resolveLaneDecisions } from "./lane-decisions.js"; +import { nudgeOn, resolveLaneDecisions } from "./lane-decisions.js"; function appendGoogleNudge(contents: GoogleContent[], text: string): GoogleContent[] { const last = contents[contents.length - 1]; @@ -95,7 +98,6 @@ export async function prepareGoogle( // #3417: surface gating lives in lane-decisions (observed reachability // overrides the dsh default, #3401); alias kept for the remaining reads. const laneDecisions = resolveLaneDecisions({ protocol: "google", pluginMode, pluginAgent: typeof session.metadata["pluginAgent"] === "string" ? session.metadata["pluginAgent"] as string : undefined, parsed, shouldInject }); - const dshCompressSurfaceOff = laneDecisions.compressSurfaceOff; // #2483: this lane used to ignore renderNone entirely (hardcoded "text-only" // in processTurn, Prepared and countTokens) — the knob was dead config on /v1beta. const renderStrategy = knobRenderNone() ? "none" : "text-only"; @@ -122,10 +124,14 @@ export async function prepareGoogle( const knownRefsBefore = new Set(Object.keys(session.state.messageRefs.byRaw)); const { tokens: tokenCount, source: tokenCountSource } = effectiveTokenCount(session, msgs, imageReserveFor(session, "google", parsed, opts, upstreamOrigin)); const activeBefore = new Set(session.state.blocks.filter((b) => b.active).map((b) => b.blockId)); + const absorbBlock = effectiveAbsorbBlock(pluginMode, config, opts.compress.absorb); + const absorbTools = absorbToolsFor(absorbBlock?.toolName ?? ABSORB_TOOL_NAME); + const absorbActive = absorbBlock?.enabled === true && shouldInject; // acp_rule has no processTurn side effect (no markers/instructions are - // ever injected into messages) — only tool availability matters. + // ever injected into messages), so unlike absorb it needs no loop- + // config stripping — only tool availability matters. const rulesActive = rulesEnabled(config) && shouldInject; - const loopConfig = ccrLoopConfig(session, config); + const loopConfig = ccrLoopConfig(session, { ...config, absorb: absorbActive ? absorbBlock : undefined }); // #2432/#2658: same post-landing recovery as the openai/responses/ // anthropic lanes — dsh desktop's native compaction can land without // transiting bili (direct-bypass / allowDshCompaction / future gate @@ -169,7 +175,9 @@ export async function prepareGoogle( const gap = foldCoverage(foldCoveredBefore, msgs.map((m) => m.id)); if (gap) log("warn", `[${sessionId}] [acp-drift] fold coverage mismatch: ${gap.matched}/${gap.expected} covered message id(s) present in resent history — ${gap.expected - gap.matched} covered id(s) missing from resent history — mutation (content edit invalidates content-hash refs, fold silently lost) or client-side deletion/truncation (benign, message no longer on the wire); observability complement to the #1328 overflow rescue (#1195)`); } + storeEffectiveAbsorb(session, loopConfig); storeEffectiveRules(session, config); + turn.messages = applyAbsorbView(turn.messages, session.state, loopConfig, tokenCount); turn.messages = attachSubagentSessions(turn.messages, session); // Drop sub-viability fragments before any consumer sees them: a tiny // range in the list makes batched compress attempts fail atomically @@ -196,7 +204,27 @@ export async function prepareGoogle( // advisory punches through per 3 consecutive decision failures. const decideTier1Arm = decide !== undefined && !!turn.nudge && turn.nudge.tier === 1 && turn.nudge.breakdown.emergencyOverride !== 1; const decideFallbackOverride = decideTier1Arm && ladderMode(session.metadata) === "fallback"; - const willInjectNudge = opts.compress.injectNudge && !!turn.nudge && shouldInject && !dshCompressSurfaceOff && (!nudgeSuppressed(session) || decideFallbackOverride) && !compressBreakerArmed(session) && !(autoFoldEngaged(loopConfig, session) && growthFoldingArmed(loopConfig)) && (turn.nudge.shouldInject || emergencyNudge(turn.nudge, undefined, loopConfig.compress.minCompressRange)); + const nudgeEligible = opts.compress.injectNudge && !!turn.nudge && laneDecisions.compressSurfaceOn && (!nudgeSuppressed(session) || decideFallbackOverride) && !compressBreakerArmed(session) && !(autoFoldEngaged(loopConfig, session) && growthFoldingArmed(loopConfig)); + // #3461: pressure admitted by every standing gate but held by the backoff + // is still an active nudge round — self-heal credit (#2155/#3004) must + // not starve because we chose not to re-inject this round. + const emergencyPressureHeld = nudgeEligible && !!turn.nudge && !turn.nudge.shouldInject && emergencyNudge(turn.nudge, undefined, loopConfig.compress.minCompressRange); + // #3482 Step 2.5: the ONE nudge gate (lane-decisions). No compaction trigger + // and no intercept health window on this wire — those differences are + // explicit inputs on the responses lane. The standalone `shouldInject` + // conjunct is subsumed by compressSurfaceOn (=== shouldInject && + // compressReachable). + const willInjectNudge = nudgeOn({ + injectNudge: opts.compress.injectNudge, + hasNudgePlan: !!turn.nudge, + compressSurfaceOn: laneDecisions.compressSurfaceOn, + compactionTrigger: false, + suppressionBypassed: !nudgeSuppressed(session) || decideFallbackOverride, + breakerArmed: compressBreakerArmed(session), + autoFoldEngaged: autoFoldEngaged(loopConfig, session), + growthFoldingArmed: growthFoldingArmed(loopConfig), + modelWantsIt: turn.nudge !== undefined && (turn.nudge.shouldInject || emergencyNudgeArmed(session, turn.nudge, loopConfig.compress.minCompressRange, tokenCount, loopConfig.nudge.growthFloor)), + }); log("info", diagNudge(turn, sessionId, tokenCount, config.modelContextLimit, model, willInjectNudge, decideTier1Arm && !decideFallbackOverride)); processedMessages = stripKernelSummaries(turn.messages, turn.state); // #1001: a silent client history rewrite takes the same archive+prune path @@ -212,7 +240,10 @@ export async function prepareGoogle( } applyCompactionArchive(session, activeBefore, new Set(msgs.map((m) => m.id)), log); reapOrphansLogged(session, msgs, log, sessionId); - rebuiltContents = coreToGoogle(processedMessages as BiliMessage[]); + // #3400: egress image rescale — outbound copies only; the originals + // stay in session state/persistence untouched. + const egressMessages = await applyImageRescalePass(processedMessages as BiliMessage[], { session, sessionId, log, maxDim: imageRescaleMaxDim(), toolArmed: imageRescaleToolArmed(session) }); + rebuiltContents = coreToGoogle(egressMessages); // ONLY the static compress prompt joins the client's system text — the // system instruction is the prefix-cache anchor and must stay @@ -225,12 +256,13 @@ export async function prepareGoogle( // constants, so the system anchor stays identical across normal turns // and round-2 re-requests — skipping them here would fork the prefix // at every fold and collapse the upstream cache hit. - if (shouldInject && !dshCompressSurfaceOff) sysParts.push(withProjectAttribution(withMarkerIntegrityNote(withSummaryBudgetNote(buildCompressSystemPrompt(prompts, surface?.promptSections), externalSummaryEnabled(config)), visibilityMarkers))); + if (laneDecisions.compressSurfaceOn) sysParts.push(withProjectAttribution(withMarkerIntegrityNote(withSummaryBudgetNote(buildCompressSystemPrompt(prompts, surface?.promptSections), externalSummaryEnabled(config)), visibilityMarkers))); else if (!isTitleGen && renderStrategy !== "none") { // #1881: the NEVER-echo prohibition follows the rendered tags, not the tool switch. const tagsOnly = buildAcpTagsOnlyPrompt("function", prompts, surface?.promptSections); if (tagsOnly) sysParts.push(tagsOnly); } + if (absorbActive) sysParts.push(buildAbsorbSystemPrompt(absorbToolName(loopConfig))); googleOutboundSystem = sysParts.join("\n\n"); // Untouched when nothing was added beyond the client's own text: the // original `systemInstruction` object then rides through byte-identical @@ -238,7 +270,7 @@ export async function prepareGoogle( const extraSystemParts = sysParts.slice(googleClientSystem ? 1 : 0); systemInstruction = extraSystemParts.length > 0 ? { parts: sysParts.map((text) => ({ text })) } : parsed.systemInstruction; if (injectTools) { - toolsOut = injectGoogleTool(parsed.tools, [...(rulesActive ? [RULE_TOOL_GOOGLE] : []), ...(ccrEnabled(session) ? [retrieveToolsFor(retrieveToolName(session)).google] : [])], surface?.toolPrompts, ccrEnabled(session), externalSummaryEnabled(config)); + toolsOut = injectGoogleTool(parsed.tools, [...(absorbActive ? [absorbTools.google] : []), ...(rulesActive ? [RULE_TOOL_GOOGLE] : []), ...(ccrEnabled(session) ? [retrieveToolsFor(retrieveToolName(session)).google] : []), ...(imageRescaleToolArmed(session) ? [READ_ORIGINAL_IMAGE_TOOL_GOOGLE] : [])], surface?.toolPrompts, ccrEnabled(session), externalSummaryEnabled(config)); } if (sysNotes.length > 0) { rebuiltContents = appendGoogleNudge(rebuiltContents, sysNotes.join("\n\n---\n\n")); @@ -295,6 +327,8 @@ export async function prepareGoogle( nudgeOutcome = "skipped"; } } + } else if (emergencyPressureHeld) { + nudgeOutcome = "gated"; } } catch (err) { log("warn", `[${sessionId}] kernel transform failed, forwarding unchanged: ${String(err)}`); diff --git a/src/server/prepare-openai.ts b/src/server/prepare-openai.ts index 846eac3802..2fb81e6499 100644 --- a/src/server/prepare-openai.ts +++ b/src/server/prepare-openai.ts @@ -14,8 +14,9 @@ import { nudgeSuppressed } from "../session-self-heal.js"; import { compressBreakerArmed } from "../stream.js"; import { applyCompactionArchive, coveredRealHistoryIds, detectUnannouncedHistoryRewrite, foldCoverage, markCompactionBoundary, markDirty, markNativeCompactionBoundary, reconcileNativeCompactionBoundary, REWRITE_MIN_INCOMING_TOTAL, snapshotMessages, type PendingRetrieval, type Session } from "../session.js"; import { carriesDshLocalCompactionSummary, DSH_LOCAL_COMPACTION_MIN_MISSING } from "./dsh-compaction-guard.js"; -import { RULE_TOOL_OPENAI, buildAcpTagsOnlyPrompt, buildCompressSystemPrompt, retrieveToolsFor, withFirstSightDrain, withMarkerIntegrityNote, withSummaryBudgetNote, withProjectAttribution } from "../compress-tool.js"; -import { adoptContentStore, ccrEnabled, ccrLoopConfig, contentStoreOf, dropRetrievals, pruneExpiredRetrievals, reconcileReloadedRetrievals, renderRetrievalNotes, retrieveToolName, snapshotPendingRetrievals, snapshotRetrievalNotes } from "../store.js"; +import { ABSORB_TOOL_NAME, READ_ORIGINAL_IMAGE_TOOL_OPENAI, RULE_TOOL_OPENAI, absorbToolsFor, buildAbsorbSystemPrompt, buildAcpTagsOnlyPrompt, buildCompressSystemPrompt, retrieveToolsFor, withFirstSightDrain, withMarkerIntegrityNote, withSummaryBudgetNote, withProjectAttribution } from "../compress-tool.js"; +import { absorbToolName, applyAbsorbView, storeEffectiveAbsorb } from "../absorb.js"; +import { adoptContentStore, ccrEnabled, ccrLoopConfig, contentStoreOf, dropRetrievals, imageRescaleToolArmed, pruneExpiredRetrievals, reconcileReloadedRetrievals, renderRetrievalNotes, retrieveToolName, snapshotPendingRetrievals, snapshotRetrievalNotes } from "../store.js"; import { rulesEnabled, storeEffectiveRules } from "../rules-feature.js"; import { autoFoldEngaged, externalSummaryEnabled, growthFoldingArmed } from "../external-summary-surface.js"; import { attachSubagentSessions } from "../subagent-sessions.js"; @@ -24,10 +25,12 @@ import { dshNativeCompactionWitness, recordConflict } from "../conflict-watch.js import { isStrictReasoningEcho, modelIdOf, normalizeStrictEchoReasoning } from "../strict-echo.js"; import { reconcileSystemAnchor } from "../system-anchor.js"; import { logRawStripCounts, markerArtifactCount, stripRawStage } from "../marker/engine.js"; -import { renderNone as knobRenderNone } from "../knobs.js"; -import { clampOutgoingOutput, countSystemAndToolsTokens, dshLedgerFloorTokens, emergencyNudge } from "./budget.js"; +import { imageRescaleMaxDim, renderNone as knobRenderNone } from "../knobs.js"; +import { applyImageRescalePass } from "../image-rescale.js"; +import { clampOutgoingOutput, countSystemAndToolsTokens, dshLedgerFloorTokens, emergencyNudge, emergencyNudgeArmed } from "./budget.js"; +import { effectiveAbsorbBlock } from "./prepare-responses.js"; import { injectOpenaiTool, injectTool } from "./inject.js"; -import { resolveLaneDecisions } from "./lane-decisions.js"; +import { nudgeOn, resolveLaneDecisions } from "./lane-decisions.js"; export async function prepareOpenai( parsed: OpenAIRequestBody, @@ -89,7 +92,6 @@ export async function prepareOpenai( // #3417: that judgement now lives in lane-decisions (observed reachability // overrides the dsh default — #3401); alias kept for the remaining reads. const laneDecisions = resolveLaneDecisions({ protocol: "openai", pluginMode, pluginAgent: typeof session.metadata["pluginAgent"] === "string" ? session.metadata["pluginAgent"] as string : undefined, parsed, shouldInject }); - const dshCompressSurfaceOff = laneDecisions.compressSurfaceOff; const markerCounts = stripRawStage(parsed, "openai"); logRawStripCounts(markerCounts, log, sessionId); @@ -119,8 +121,14 @@ export async function prepareOpenai( const { tokens: tokenCount, source: tokenCountSource } = effectiveTokenCount(session, msgs, imageReserveFor(session, "openai", parsed, opts, billingUpstream ?? upstreamOrigin)); const activeBefore = new Set(session.state.blocks.filter((b) => b.active).map((b) => b.blockId)); + // Absorb markers ride in the kernel's processTurn output (gated by + // config.absorb). Title-gen requests skip ALL injection for + // prefix-cache stability, so strip absorb from the loop config there. + const absorbBlock = effectiveAbsorbBlock(pluginMode, config, opts.compress.absorb); + const absorbTools = absorbToolsFor(absorbBlock?.toolName ?? ABSORB_TOOL_NAME); + const absorbActive = absorbBlock?.enabled === true && shouldInject; const rulesActive = rulesEnabled(config) && shouldInject; - const loopConfig = ccrLoopConfig(session, config); + const loopConfig = ccrLoopConfig(session, { ...config, absorb: absorbActive ? absorbBlock : undefined }); // #2432: dsh desktop's native compaction can LAND without ever // transiting bili — its summarizer calls ctx.llm.stream() directly, // and manual /compact / idle-session paths have no ALS attribution so @@ -180,7 +188,9 @@ export async function prepareOpenai( const gap = foldCoverage(foldCoveredBefore, msgs.map((m) => m.id)); if (gap) log("warn", `[${sessionId}] [acp-drift] fold coverage mismatch: ${gap.matched}/${gap.expected} covered message id(s) present in resent history — ${gap.expected - gap.matched} covered id(s) missing from resent history — mutation (content edit invalidates content-hash refs, fold silently lost) or client-side deletion/truncation (benign, message no longer on the wire); observability complement to the #1328 overflow rescue (#1195)`); } + storeEffectiveAbsorb(session, loopConfig); storeEffectiveRules(session, config); + turn.messages = applyAbsorbView(turn.messages, session.state, loopConfig, tokenCount); turn.messages = attachSubagentSessions(turn.messages, session); // Drop sub-viability fragments before any consumer sees them: a tiny // range in the list makes batched compress attempts fail atomically @@ -211,7 +221,27 @@ export async function prepareOpenai( // counter on the same round, so at most ONE legacy advisory punches // through per DECIDE_FALLBACK_THRESHOLD consecutive decision failures. const decideFallbackOverride = decideTier1Arm && ladderMode(session.metadata) === "fallback"; - const willInjectNudge = opts.compress.injectNudge && !!turn.nudge && shouldInject && !dshCompressSurfaceOff && (!nudgeSuppressed(session) || decideFallbackOverride) && !compressBreakerArmed(session) && !(autoFoldEngaged(loopConfig, session) && growthFoldingArmed(loopConfig)) && (turn.nudge.shouldInject || emergencyNudge(turn.nudge, undefined, loopConfig.compress.minCompressRange)); + const nudgeEligible = opts.compress.injectNudge && !!turn.nudge && laneDecisions.compressSurfaceOn && (!nudgeSuppressed(session) || decideFallbackOverride) && !compressBreakerArmed(session) && !(autoFoldEngaged(loopConfig, session) && growthFoldingArmed(loopConfig)); + // #3461: pressure admitted by every standing gate but held by the backoff + // is still an active nudge round — self-heal credit (#2155/#3004) must + // not starve because we chose not to re-inject this round. + const emergencyPressureHeld = nudgeEligible && !!turn.nudge && !turn.nudge.shouldInject && emergencyNudge(turn.nudge, undefined, loopConfig.compress.minCompressRange); + // #3482 Step 2.5: the ONE nudge gate (lane-decisions). No compaction trigger + // and no intercept health window on this wire — those differences are + // explicit inputs on the responses lane. The standalone `shouldInject` + // conjunct is subsumed by compressSurfaceOn (=== shouldInject && + // compressReachable). + const willInjectNudge = nudgeOn({ + injectNudge: opts.compress.injectNudge, + hasNudgePlan: !!turn.nudge, + compressSurfaceOn: laneDecisions.compressSurfaceOn, + compactionTrigger: false, + suppressionBypassed: !nudgeSuppressed(session) || decideFallbackOverride, + breakerArmed: compressBreakerArmed(session), + autoFoldEngaged: autoFoldEngaged(loopConfig, session), + growthFoldingArmed: growthFoldingArmed(loopConfig), + modelWantsIt: turn.nudge !== undefined && (turn.nudge.shouldInject || emergencyNudgeArmed(session, turn.nudge, loopConfig.compress.minCompressRange, tokenCount, loopConfig.nudge.growthFloor)), + }); log("info", diagNudge(turn, sessionId, tokenCount, config.modelContextLimit, parsed.model, willInjectNudge, decideTier1Arm && !decideFallbackOverride)); processedMessages = applyReasoningDrop(stripKernelSummaries(turn.messages, turn.state), reasoningPlan, log, sessionId); // #1001: a silent client history rewrite takes the same archive+prune path @@ -246,7 +276,10 @@ export async function prepareOpenai( attachedRetrievals = snapshotPendingRetrievals(session); if (attachedRetrievals.length > 0) processedMessages = [...processedMessages, ...attachedRetrievals.map((i) => i.injection)]; } - rebuiltMessages = systemToUser(hardenOpenaiAssistantContent(coreToOpenai(processedMessages as BiliMessage[]))); + // #3400: egress image rescale — outbound copies only; the originals + // stay in session state/persistence untouched. + const egressMessages = await applyImageRescalePass(processedMessages as BiliMessage[], { session, sessionId, log, maxDim: imageRescaleMaxDim(), toolArmed: imageRescaleToolArmed(session) }); + rebuiltMessages = systemToUser(hardenOpenaiAssistantContent(coreToOpenai(egressMessages))); // ONLY the static compress prompt goes into the system message — the // system prompt is the prefix-cache anchor and must be byte-stable @@ -256,12 +289,13 @@ export async function prepareOpenai( // would invalidate the cache every turn. const sysParts: string[] = []; if (openaiSystemText) sysParts.push(openaiSystemText); - if (shouldInject && !dshCompressSurfaceOff) sysParts.push(withProjectAttribution(withMarkerIntegrityNote(withSummaryBudgetNote(buildCompressSystemPrompt(prompts, surface?.promptSections), externalSummaryEnabled(config)), visibilityMarkers))); + if (laneDecisions.compressSurfaceOn) sysParts.push(withProjectAttribution(withMarkerIntegrityNote(withSummaryBudgetNote(buildCompressSystemPrompt(prompts, surface?.promptSections), externalSummaryEnabled(config)), visibilityMarkers))); else if (!isTitleGen && !knobRenderNone()) { // #1881: the NEVER-echo prohibition follows the rendered tags, not the tool switch. const tagsOnly = buildAcpTagsOnlyPrompt("function", prompts, surface?.promptSections); if (tagsOnly) sysParts.push(tagsOnly); } + if (absorbActive) sysParts.push(buildAbsorbSystemPrompt(absorbToolName(loopConfig))); rebuiltMessages = injectOpenaiSystem(rebuiltMessages, sysParts); if (sysNotes.length > 0) { rebuiltMessages = [...rebuiltMessages, ...sysNotes.map((text) => ({ role: "user" as const, content: text }))]; @@ -272,7 +306,7 @@ export async function prepareOpenai( // avoids double-counting it. openaiOutboundSystem = sysParts.join("\n\n"); if (injectTools) { - toolsOut = injectOpenaiTool(parsed.tools, [...(rulesActive ? [RULE_TOOL_OPENAI] : []), ...(ccrEnabled(session) ? [retrieveToolsFor(retrieveToolName(session)).openai] : [])], surface?.toolPrompts, ccrEnabled(session), externalSummaryEnabled(config)); + toolsOut = injectOpenaiTool(parsed.tools, [...(absorbActive ? [absorbTools.openai] : []), ...(rulesActive ? [RULE_TOOL_OPENAI] : []), ...(ccrEnabled(session) ? [retrieveToolsFor(retrieveToolName(session)).openai] : []), ...(imageRescaleToolArmed(session) ? [READ_ORIGINAL_IMAGE_TOOL_OPENAI] : [])], surface?.toolPrompts, ccrEnabled(session), externalSummaryEnabled(config)); } // Nudge as a separate trailing user message (cache-friendly). Injected // in BOTH modes (#451): plugin agents supply the ACP tools but have no @@ -339,6 +373,8 @@ export async function prepareOpenai( nudgeOutcome = "skipped"; } } + } else if (emergencyPressureHeld) { + nudgeOutcome = "gated"; } // [#1343/#1457] surface any earlier undelivered retrieve as an ephemeral // trailing user note (kept last so it never reorders cached messages). diff --git a/src/server/prepare-responses.ts b/src/server/prepare-responses.ts index db0629deab..b8ec042f3e 100644 --- a/src/server/prepare-responses.ts +++ b/src/server/prepare-responses.ts @@ -1,8 +1,9 @@ import http from "node:http"; -import type { CompressionCore, Config, CoreMessage, NudgeDecision, PackSurface, Prompts } from "acp-kernel"; +import type { AbsorbConfig, CompressionCore, Config, CoreMessage, NudgeDecision, PackSurface, Prompts } from "acp-kernel"; import { defaultCountTokens, renderNudgeText, viableRanges } from "acp-kernel"; import { injectResponsesDeveloperMessage, type BiliMessage, type ResponseInputItem, type ResponsesProjection, type ResponsesRequestBody } from "acp-kernel/wire"; import type { CompressReasoningConfig } from "../reasoning-drop.js"; +import { resolveAbsorbSettings } from "../compress-settings.js"; import { resolveCompressProtocol, type CompressSettings, type ProxyOptions } from "../config.js"; import { currentCalibrationFactor, strippedResponseIdWarning } from "../util.js"; import { applyReasoningDrop, diagNudge, diagTagSummary, deriveTitle, effectiveTokenCount, imageBillingFor, imageReserveFor, imageTokenCapFor, isAutoInjectedNotification, planReasoningDrop, repairResponsesAssistantOrdering, reapOrphansLogged, runNudgeDecision, stripKernelSummaries, usageGradeInputBaseline, warnResponsesReasoningPairs, type Prepared } from "../server.js"; @@ -16,8 +17,9 @@ import { compressBreakerArmed } from "../stream.js"; import { coveredRealHistoryIds, foldCoverage, markDirty, markNativeCompactionBoundary, reconcileNativeCompactionBoundary, REWRITE_MIN_INCOMING_TOTAL, snapshotMessages, type Session } from "../session.js"; import { carriesDshLocalCompactionSummary, DSH_LOCAL_COMPACTION_MIN_MISSING } from "./dsh-compaction-guard.js"; import { recordConflict } from "../conflict-watch.js"; -import { BILI_ACP_READONLY_TOOLS_RESPONSES, BILI_ACP_READONLY_TOOLS_RESPONSES_NO_RANGE, BILI_ACP_TOOLS_RESPONSES, BILI_ACP_TOOLS_RESPONSES_NO_RANGE, RULE_TOOL_RESPONSES, buildAcpTagsOnlyPrompt, buildCompressHybridSystemPrompt, buildCompressSystemPrompt, retrieveToolsFor, withFirstSightDrain, withMarkerIntegrityNote, withSummaryBudgetNote, withProjectAttribution } from "../compress-tool.js"; -import { adoptContentStore, ccrEnabled, ccrLoopConfig, contentStoreOf, retrieveToolName } from "../store.js"; +import { ABSORB_TOOL_NAME, BILI_ACP_READONLY_TOOLS_RESPONSES, BILI_ACP_READONLY_TOOLS_RESPONSES_NO_RANGE, BILI_ACP_TOOLS_RESPONSES, BILI_ACP_TOOLS_RESPONSES_NO_RANGE, READ_ORIGINAL_IMAGE_TOOL_RESPONSES, RULE_TOOL_RESPONSES, absorbToolsFor, buildAbsorbSystemPrompt, buildAcpTagsOnlyPrompt, buildCompressHybridSystemPrompt, buildCompressSystemPrompt, retrieveToolsFor, withFirstSightDrain, withMarkerIntegrityNote, withSummaryBudgetNote, withProjectAttribution } from "../compress-tool.js"; +import { absorbToolName, applyAbsorbView, storeEffectiveAbsorb } from "../absorb.js"; +import { adoptContentStore, ccrEnabled, ccrLoopConfig, contentStoreOf, imageRescaleToolArmed, retrieveToolName } from "../store.js"; import { rulesEnabled, storeEffectiveRules } from "../rules-feature.js"; import { autoFoldEngaged, externalSummaryEnabled, growthFoldingArmed } from "../external-summary-surface.js"; import { attachSubagentSessions } from "../subagent-sessions.js"; @@ -28,10 +30,11 @@ import { reconcileSystemAnchor } from "../system-anchor.js"; import { buildPromptForgeBody, buildTriggerForgeBody, carriesCodexLocalCompactionSummary, codexCompactGate, codexCompactGatePre, codexCompactMode, hasCompactionPrompt, hasCompactionTrigger, isCodexClient, mergeForgedSummaries, replaceBiliCompactionItems, stripBiliCompactionItems, CODEX_COMPACT_HEALTH_RATIO, CODEX_LOCAL_COMPACTION_MIN_MISSING } from "../codex-compact.js"; import type { ConversationIdentity } from "../session-id.js"; import { logRawStripCounts, markerArtifactCount, stripRawStage } from "../marker/engine.js"; -import { keepResponseId as knobKeepResponseId, noCompressPrompt as knobNoCompressPrompt, noInjectTool as knobNoInjectTool, renderNone as knobRenderNone } from "../knobs.js"; -import { clampOutgoingOutput, countLoadedToolTokens, countSystemAndToolsTokens, dshLedgerFloorTokens, emergencyNudge, modelVisibleTools } from "./budget.js"; +import { imageRescaleMaxDim, keepResponseId as knobKeepResponseId, noCompressPrompt as knobNoCompressPrompt, noInjectTool as knobNoInjectTool, renderNone as knobRenderNone } from "../knobs.js"; +import { applyImageRescaleItems } from "../image-rescale.js"; +import { clampOutgoingOutput, countLoadedToolTokens, countSystemAndToolsTokens, dshLedgerFloorTokens, emergencyNudge, emergencyNudgeArmed, modelVisibleTools } from "./budget.js"; import { FORCE_TEXT_PROTOCOL, injectResponsesTool, injectTool } from "./inject.js"; -import { resolveLaneDecisions } from "./lane-decisions.js"; +import { nudgeOn, resolveLaneDecisions } from "./lane-decisions.js"; export async function prepareResponses( parsed: ResponsesRequestBody, @@ -209,8 +212,14 @@ export async function prepareResponses( log("info", `[${sessionId}] input items: ${Array.isArray(parsed.input) ? parsed.input.map((i: ResponseInputItem) => i.type).join(",") : "(string)"}`); } const { tokens: tokenCount, source: tokenCountSource } = effectiveTokenCount(session, msgs, imageReserveFor(session, "responses", parsed, opts, billingUpstream ?? upstreamOrigin)); + // Absorb markers ride in the kernel's processTurn output (gated by + // config.absorb). The marker/text protocol has no native tool channel, + // so strip absorb from the loop config there (both modes). + const absorbBlock = effectiveAbsorbBlock(pluginMode, config, opts.compress.absorb); + const absorbTools = absorbToolsFor(absorbBlock?.toolName ?? ABSORB_TOOL_NAME); + const absorbActive = absorbBlock?.enabled === true && shouldInject && !isCompactionTrigger && !responsesTextProtocol; const rulesActive = rulesEnabled(config) && shouldInject && !isCompactionTrigger && !responsesTextProtocol; - const loopConfig = ccrLoopConfig(session, config); + const loopConfig = ccrLoopConfig(session, { ...config, absorb: absorbActive ? absorbBlock : undefined }); // #2372/#2432: LOCAL auto-compaction — the client compacts on its own // and the call never transits the proxy; the first the proxy hears of // it is this request replaying [compaction summary, retained tail…]. @@ -284,7 +293,9 @@ export async function prepareResponses( const gap = foldCoverage(foldCoveredBefore, msgs.map((m) => m.id)); if (gap) log("warn", `[${sessionId}] [acp-drift] fold coverage mismatch: ${gap.matched}/${gap.expected} covered message id(s) present in resent history — ${gap.expected - gap.matched} covered id(s) missing from resent history — mutation (content edit invalidates content-hash refs, fold silently lost) or client-side deletion/truncation (benign, message no longer on the wire); observability complement to the #1328 overflow rescue (#1195)`); } + storeEffectiveAbsorb(session, loopConfig); storeEffectiveRules(session, config); + turn.messages = applyAbsorbView(turn.messages, session.state, loopConfig, tokenCount); turn.messages = attachSubagentSessions(turn.messages, session); // Drop sub-viability fragments before any consumer sees them: a tiny // range in the list makes batched compress attempts fail atomically @@ -311,12 +322,35 @@ export async function prepareResponses( // advisory punches through per 3 consecutive decision failures. const decideTier1Arm = decide !== undefined && !!turn.nudge && turn.nudge.tier === 1 && turn.nudge.breakdown.emergencyOverride !== 1; const decideFallbackOverride = decideTier1Arm && ladderMode(session.metadata) === "fallback"; - const willInjectNudge = opts.compress.injectNudge && !!turn.nudge && compressSurfaceOn && !isCompactionTrigger && (!nudgeSuppressed(session) || decideFallbackOverride) && !compressBreakerArmed(session) && !(autoFoldEngaged(loopConfig, session) && growthFoldingArmed(loopConfig, codexLane && codexCompactMode() === "intercept" ? loopConfig.modelContextLimit * CODEX_COMPACT_HEALTH_RATIO : undefined)) && (turn.nudge.shouldInject || emergencyNudge(turn.nudge, undefined, loopConfig.compress.minCompressRange)); + const nudgeEligible = opts.compress.injectNudge && !!turn.nudge && compressSurfaceOn && !isCompactionTrigger && (!nudgeSuppressed(session) || decideFallbackOverride) && !compressBreakerArmed(session) && !(autoFoldEngaged(loopConfig, session) && growthFoldingArmed(loopConfig, codexLane && codexCompactMode() === "intercept" ? loopConfig.modelContextLimit * CODEX_COMPACT_HEALTH_RATIO : undefined)); + // #3461: pressure admitted by every standing gate but held by the backoff + // is still an active nudge round — self-heal credit (#2155/#3004) must + // not starve because we chose not to re-inject this round. + const emergencyPressureHeld = nudgeEligible && !!turn.nudge && !turn.nudge.shouldInject && emergencyNudge(turn.nudge, undefined, loopConfig.compress.minCompressRange); + // #3482 Step 2.5: the ONE nudge gate (lane-decisions). The responses lane's + // two DELIBERATE differences are explicit inputs: the compaction + // trigger (#2155/#2432 — a trigger request never carries the nudge) + // and the codex-intercept health window (#2774). + const willInjectNudge = nudgeOn({ + injectNudge: opts.compress.injectNudge, + hasNudgePlan: !!turn.nudge, + compressSurfaceOn: laneDecisions.compressSurfaceOn, + compactionTrigger: isCompactionTrigger, + suppressionBypassed: !nudgeSuppressed(session) || decideFallbackOverride, + breakerArmed: compressBreakerArmed(session), + autoFoldEngaged: autoFoldEngaged(loopConfig, session), + growthFoldingArmed: growthFoldingArmed(loopConfig, codexLane && codexCompactMode() === "intercept" ? loopConfig.modelContextLimit * CODEX_COMPACT_HEALTH_RATIO : undefined), + modelWantsIt: turn.nudge !== undefined && (turn.nudge.shouldInject || emergencyNudgeArmed(session, turn.nudge, loopConfig.compress.minCompressRange, tokenCount, loopConfig.nudge.growthFloor)), + }); log("info", diagNudge(turn, sessionId, tokenCount, config.modelContextLimit, parsed.model, willInjectNudge, decideTier1Arm && !decideFallbackOverride)); processedMessages = repairResponsesAssistantOrdering(applyReasoningDrop(stripKernelSummaries(turn.messages, turn.state), reasoningPlan, log, sessionId), originalMessages); reapOrphansLogged(session, msgs, log, sessionId); rebuiltInput = patchResponsesInput(projection, processedMessages); if (Array.isArray(rebuiltInput)) rebuiltInput = mergeAdjacentConfigurationUpdates(hoistTrappedToolItems(rebuiltInput)); + // #3400: the Responses steady path rebuilds items from the RAW parsed + // input (patchOriginalItem), so BiliMessage-level modifications never + // reach it — rescale at item level instead. Outbound copies only. + if (Array.isArray(rebuiltInput)) rebuiltInput = (await applyImageRescaleItems(rebuiltInput as unknown[], { session, sessionId, log, maxDim: imageRescaleMaxDim(), toolArmed: imageRescaleToolArmed(session) })) as ResponseInputItem[]; // Fallback path: when the echo did NOT come back this turn (client // dropped it / restarted), the history-borne handoff is absent and the // forge-time captured summaries are re-injected into the developer @@ -333,6 +367,7 @@ export async function prepareResponses( if (compressSurfaceOn && !isCompactionTrigger && !knobNoCompressPrompt()) { const prompt = withProjectAttribution(withMarkerIntegrityNote(withSummaryBudgetNote(responsesTextProtocol ? buildCompressHybridSystemPrompt(prompts, surface?.promptSections) : buildCompressSystemPrompt(prompts, surface?.promptSections), externalSummaryEnabled(config)), visibilityMarkers)); const devParts = [...projection.systemParts, ...forgedSummaries, prompt]; + if (absorbActive) devParts.push(buildAbsorbSystemPrompt(absorbToolName(loopConfig))); const devContent = devParts.join("\n\n---\n\n"); responsesDevContent = devContent; if (forgedSummaries.length > 0) log("debug", `[${sessionId}] [inject] ${forgedSummaries.length} captured summary block(s) re-injected into developer message`); @@ -341,7 +376,7 @@ export async function prepareResponses( // #1712: decompress's startId/endId execute only on CCR-armed // sessions (#1179) — serve the no-range schema otherwise. const ccrOn = ccrEnabled(session); - const respExtra = [...(rulesActive ? [RULE_TOOL_RESPONSES] : []), ...(ccrOn ? [retrieveToolsFor(retrieveToolName(session)).responses] : [])]; + const respExtra = [...(absorbActive ? [absorbTools.responses] : []), ...(rulesActive ? [RULE_TOOL_RESPONSES] : []), ...(ccrOn ? [retrieveToolsFor(retrieveToolName(session)).responses] : []), ...(imageRescaleToolArmed(session) ? [READ_ORIGINAL_IMAGE_TOOL_RESPONSES] : [])]; toolsOut = responsesTextProtocol ? injectResponsesTool(parsed.tools, ccrOn ? BILI_ACP_READONLY_TOOLS_RESPONSES : BILI_ACP_READONLY_TOOLS_RESPONSES_NO_RANGE, surface?.toolPrompts, externalSummaryEnabled(config)) : injectResponsesTool(parsed.tools, respExtra.length > 0 ? [...(ccrOn ? BILI_ACP_TOOLS_RESPONSES : BILI_ACP_TOOLS_RESPONSES_NO_RANGE), ...respExtra] : (ccrOn ? BILI_ACP_TOOLS_RESPONSES : BILI_ACP_TOOLS_RESPONSES_NO_RANGE), surface?.toolPrompts, externalSummaryEnabled(config)); @@ -440,6 +475,8 @@ export async function prepareResponses( nudgeOutcome = "skipped"; } } + } else if (emergencyPressureHeld) { + nudgeOutcome = "gated"; } transformOk = true; } catch (err) { @@ -648,7 +685,10 @@ export function prepareResponsesCompact( let transformOk = false; try { const projection = responsesToCore(forgeBody); - const compactConfig = ccrLoopConfig(session, config); + // The forged handoff is one-shot with no tool channel: strip absorb so + // no [ACP absorb] instruction bakes into the forged history, and run + // the absorb view so absorbed pairs stay hidden in it (wire parity). + const compactConfig = ccrLoopConfig(session, { ...config, absorb: undefined }); const turn = core.processTurn({ messages: projection.msgs, state: session.state, config: compactConfig, tokenCount: usageGradeInputBaseline(session), renderTags: knobRenderNone() ? "none" : "text-only", contentStore: contentStoreOf(session) }); session.state = turn.state; adoptContentStore(session, turn.contentStore); @@ -659,7 +699,7 @@ export function prepareResponsesCompact( session.contentStoreDirty = prevStoreDirty; return base; } - const viewed = turn.messages; + const viewed = applyAbsorbView(turn.messages, turn.state, compactConfig, usageGradeInputBaseline(session)); const processed = repairResponsesAssistantOrdering(stripKernelSummaries(viewed, turn.state), projection.msgs); let output = patchResponsesInput(projection, processed); if (typeof output === "string") { @@ -726,6 +766,9 @@ export function resolvePromptCacheKey( // per-request merged block (provider/model overrides apply); plugin lane uses // the base block so the manifest's advertised name and the gate's adjudicated // name always agree — provider/model absorb.* overrides are proxy-lane-only. +export function effectiveAbsorbBlock(pluginMode: boolean, config: Config, baseAbsorb?: CompressSettings["absorb"]): AbsorbConfig | undefined { + return pluginMode ? resolveAbsorbSettings(baseAbsorb) : config.absorb; +} // #2867: canonical incoming system/developer view of a Responses body — the // top-level instructions first (mirroring the kernel's responsesToCore diff --git a/src/server/relay.ts b/src/server/relay.ts index fe91a51b96..0297f5b5f2 100644 --- a/src/server/relay.ts +++ b/src/server/relay.ts @@ -6,6 +6,7 @@ import fs from "node:fs"; import path from "node:path"; import { type CompressionCore, type Config, type CoreMessage, defaultCountTokens, defaultPrompts, resolveOutputSteeringConfig } from "acp-kernel"; import { type BiliMessage } from "acp-kernel/wire"; +import { absorbToolName, applyAbsorbView } from "../absorb.js"; import { APIG_RESIGN_SCHEME, resignApig } from "../apig-resign.js"; import { credentialFingerprint, noteClientAbort, noteForwardedBody, noteForwardedImageFacts, noteInboundMessages, settleUsageReport } from "../cache-ledger.js"; import { warnCacheCollapse } from "../cache-warn.js"; @@ -14,7 +15,7 @@ import { applyCompatDropFields, dropCompatFieldsJson, resolveCompatDropFields } import { applyCompatRoles, applyCompatRolesJson, detectRoleRejection, detectSystemPlacementError, hasOffHeadSystem, resolveCompatRoles, type CompatRoles } from "../compat-roles.js"; import { compressLoopResponsesJson } from "../compress-loop-responses.js"; import { resolveCompress } from "../compress-settings.js"; -import { buildCompressHybridSystemPrompt, buildCompressSystemPrompt, withMarkerIntegrityNote, withProjectAttribution, withSummaryBudgetNote } from "../compress-tool.js"; +import { buildAbsorbSystemPrompt, buildCompressHybridSystemPrompt, buildCompressSystemPrompt, withMarkerIntegrityNote, withProjectAttribution, withSummaryBudgetNote } from "../compress-tool.js"; import { findRoute, resolveDeclaredProtocol, type ProxyOptions } from "../config.js"; import { makeContinuationRefetch } from "../degenerate-retry.js"; import { dumpRejectedBody } from "../error-dump.js"; @@ -52,6 +53,7 @@ import { applyOutputSteering, applyOutputSteeringJson } from "../output-steering import { estimateWireOverhead } from "./budget.js"; import { buildForwardHeaders, connectionNamedHeaders, RESPONSE_ONLY_STRIP_HEADERS, safeSessionId, UPSTREAM_HOP_HEADERS } from "./headers.js"; import { bodyDumpEnabled, logDumpFailure } from "./observability.js"; +import { effectiveAbsorbBlock } from "./prepare-responses.js"; import { awaitDrain, bufferToStream, dumpStreamToFile, pipeThrough, readStreamToBuffer } from "./stream-io.js"; import { buildForwardTarget, outboundPayloadBreakdown, registerRequestAbort, beginStreamKeepalive, effectiveTokenCount, googlePathKind, imageBillingFor, imageTokenCapFor, inboundResignCredential, outboundContextEstimates, repairResponsesAssistantOrdering, resolveUpstream, stripKernelSummaries, usageGradeInputBaseline, type Prepared } from "../server.js"; @@ -166,6 +168,63 @@ function armFailureShrink(prepared: Prepared, log: (level: string, msg: string) } } +// #3434: upstream accepted the request then sent no bytes — the log shows +// nothing between `forward …` and the outcome (client disconnect / idle-watchdog +// abort), so an upstream stall reads as proxy-side. Arm a one-shot first-byte +// watchdog around the initial fetch: headers not arrived by the threshold → ONE +// warn naming target+method+lane. Additive logging only — deliberately NO +// fast-fail (would truncate long prefills, cf. #1706/#1714) and NO new config +// surface (code constant + test seam). +const UPSTREAM_FIRST_BYTE_WARN_MS = 30_000; +let firstByteWarnMs = UPSTREAM_FIRST_BYTE_WARN_MS; +export function _setFirstByteWarnMsForTest(ms?: number): void { + firstByteWarnMs = ms ?? UPSTREAM_FIRST_BYTE_WARN_MS; +} +// Dedupe once per origin per process (windowSourceLogged pattern) so a +// long-prefill local model logs at most one line, not one per slow request. +// Claimed at FIRE time (not arm time) so a fast request never suppresses a +// later genuine stall on the same origin. +const firstByteWarnedOrigins = new Set(); +export function _resetFirstByteWarnedOriginsForTest(): void { + firstByteWarnedOrigins.clear(); +} +/** + * #3434: arm a one-shot watchdog logging ONE warn if the fetch behind `signal` + * is still pending after `firstByteWarnMs`. Returns the timer handle (the caller + * clears it when the fetch settles) or undefined when disarmed (threshold + * disabled, origin already warned, or signal already aborted). + */ +export function armFirstByteWatchdog(o: { + url: string; + method: string; + lane: string; + sessionId: string; + log: (level: string, msg: string) => void; + signal: AbortSignal; +}): ReturnType | undefined { + const threshold = firstByteWarnMs; + if (!Number.isFinite(threshold) || threshold <= 0) return undefined; + let origin: string; + try { + origin = new URL(o.url).origin; + } catch { + origin = o.url; + } + if (firstByteWarnedOrigins.has(origin)) return undefined; + if (o.signal.aborted) return undefined; + const seconds = Math.round(threshold / 1000); + const fire = (): void => { + // First-to-fire claims the origin; concurrent same-origin timers stay silent. + if (firstByteWarnedOrigins.has(origin)) return; + firstByteWarnedOrigins.add(origin); + o.log("warn", `[${o.sessionId}] upstream awaiting first byte ≥${seconds}s: ${o.method} ${maskUrlForLog(o.url)} (lane=${o.lane})`); + }; + const timer = setTimeout(fire, threshold); + timer.unref(); + o.signal.addEventListener("abort", () => clearTimeout(timer), { once: true }); + return timer; +} + export async function forward( req: http.IncomingMessage, res: http.ServerResponse, @@ -598,9 +657,16 @@ export async function forward( } let upstreamResult: Awaited>; try { - upstreamResult = await fetchWithTransportRetry(upstreamUrl, init, undefined, clientAbort.signal, (info) => { + const lane = prepared?.sidePassthrough ? "side" : prepared ? "main" : "passthrough"; + const fetchPromise = fetchWithTransportRetry(upstreamUrl, init, undefined, clientAbort.signal, (info) => { log("warn", `[${prepared?.session.id ?? "unknown"}] [acp-proxy] upstream ${info.detail}; retrying in ${info.delayMs}ms (attempt ${info.attempt}/${info.maxAttempts})`); }); + const firstByteTimer = armFirstByteWatchdog({ url: upstreamUrl, method: req.method ?? "UNKNOWN", lane, sessionId: prepared?.session.id ?? "passthrough", log, signal: clientAbort.signal }); + try { + upstreamResult = await fetchPromise; + } finally { + clearTimeout(firstByteTimer); + } recordUpstreamConnection(upstreamUrl, proxyUrl); // #1682: any resolved response proves the host is reachable again. clearUpstreamAlertsForHost(upstreamUrl); @@ -1428,13 +1494,30 @@ export async function forward( // Same absorb gate as prepare*: the section only exists where the // tool is callable, keeping loop re-requests byte-consistent with // the first request (prefix-cache anchor). - const loopConfig = ccrLoopConfig(prepared.session, config); + const absorbBlock = effectiveAbsorbBlock(prepared.pluginMode === true, config, opts.compress.absorb); + const absorbActive = absorbBlock?.enabled === true && opts.compress.injectTool && !textProtocol; + const loopConfig = ccrLoopConfig(prepared.session, { ...config, absorb: absorbActive ? absorbBlock : undefined }); + // Cache-seam: the compress<->absorb join must be byte-identical to + // the steady prepare* paths. Every steady wire joins the absorb + // section with a plain "\n\n" (sysParts.join / injectSystem parts + // join); the historical "---" divider here existed ONLY on round-2 + // and broke the system-element prefix on every fold while absorb + // was armed (probe: cache-seam-probes P1+P2, chat wire diverged at + // the absorb boundary char). + const absorbSection = absorbActive + ? `\n\n${buildAbsorbSystemPrompt(absorbToolName(loopConfig))}` + : ""; const visibilityMarkers = resolveCompress(opts.routes, route?.rewrittenUrl, (parsedReq as { model?: string }).model ?? prepared.model, opts.compress).visibilityMarkers ?? true; // #3298 cache-seam parity: every steady prepare* path appends the // project-attribution outermost, so the round-2 rebuild must too or // the head bytes drift at every fold. - const systemPrompt = withProjectAttribution(withMarkerIntegrityNote(withSummaryBudgetNote(textProtocol ? buildCompressHybridSystemPrompt(prepared.prompts ?? defaultPrompts, prepared.surface?.promptSections) : buildCompressSystemPrompt(prepared.prompts ?? defaultPrompts, prepared.surface?.promptSections), externalSummaryEnabled(config)), visibilityMarkers)); - const adapter = pickAdapter(prepared.protocol, parsedReq, textProtocol, prepared.responsesProjection, prepared.anthropicSystem, prepared.openaiSystemText, undefined, prepared.google, prepared.systemNotes, opts.streamErrorShape, prepared.anthropicCacheMarks, prepared.anthropicClientCacheControls, false); + // absorbSection sits OUTSIDE withProjectAttribution: every steady + // prepare* path appends the attribution to the compress prompt and + // only THEN pushes the absorb section (sysParts order). Nesting it + // inside flipped the two sections on round-2 and broke the + // system-element prefix at every fold (cache-seam-probes P1+P2/P3). + const systemPrompt = withProjectAttribution(withMarkerIntegrityNote(withSummaryBudgetNote(textProtocol ? buildCompressHybridSystemPrompt(prepared.prompts ?? defaultPrompts, prepared.surface?.promptSections) : buildCompressSystemPrompt(prepared.prompts ?? defaultPrompts, prepared.surface?.promptSections), externalSummaryEnabled(config)), visibilityMarkers)) + absorbSection; + const adapter = pickAdapter(prepared.protocol, parsedReq, textProtocol, prepared.responsesProjection, prepared.anthropicSystem, prepared.openaiSystemText, absorbActive ? absorbToolName(loopConfig) : undefined, prepared.google, prepared.systemNotes, opts.streamErrorShape, prepared.anthropicCacheMarks, prepared.anthropicClientCacheControls, absorbActive); const refreshFolded = async (current: CoreMessage[]): Promise => { return withSessionLock(prepared.session, async () => { // #422: mirror the prepare's fold with the post-compress state so @@ -1469,7 +1552,7 @@ export async function forward( // crossings — absorb prompts appeared/disappeared mid-history // and broke the prefix cache on every fold while the // threshold was being straddled (probe: cache-seam-probes). - const viewed = turn.messages; + const viewed = applyAbsorbView(turn.messages, turn.state, loopConfig, effectiveTokenCount(prepared.session, turn.messages).tokens); const records = current.filter((m) => typeof m.id === "string" && m.id.startsWith("acp_loop_")); // #1548: strip only when the compress call rides INBOUND history (client persists // it). Ephemeral acp_loop_* pairs are never re-sent by proxy-mode clients; stripping diff --git a/src/server/side-request.ts b/src/server/side-request.ts index 9c25006976..0433e36494 100644 --- a/src/server/side-request.ts +++ b/src/server/side-request.ts @@ -3,7 +3,8 @@ import { estimateRawBodyTokens } from "../preflight.js"; import { imageTokensInParsedBody } from "../image-tokens.js"; import { reserveOutputHeadroom, shouldReserveOutputHeadroom, systemCarrierTexts } from "../util.js"; import { type ResolvedImageBilling } from "../image-tokens.js"; -import { ACP_TOOL_NAMES, RETRIEVE_TOOL_NAME, RULE_TOOL_NAME } from "acp-kernel"; +import { ACP_TOOL_NAMES, ABSORB_TOOL_NAME, RETRIEVE_TOOL_NAME, RULE_TOOL_NAME } from "acp-kernel"; +import { READ_ORIGINAL_IMAGE_TOOL_NAME } from "../compress-tool.js"; // #388: side requests (title-gen etc.) share the main session key but must not // touch kernel state. Identified by a tiny output budget (same heuristic as @@ -185,8 +186,10 @@ export function resolveSideLane(s: SideLaneSignals): SideLaneDecision { // guaranteed bili-owned, and any host tool vetoes the demotion. export const BILI_TOOL_NAMES: ReadonlySet = new Set([ ...ACP_TOOL_NAMES, + ABSORB_TOOL_NAME, RULE_TOOL_NAME, RETRIEVE_TOOL_NAME, + READ_ORIGINAL_IMAGE_TOOL_NAME, ]); /** Tool names declared by `parsed.tools`, proto-agnostically. null when the @@ -302,6 +305,13 @@ export function writeOutputBudget(parsed: Record, field: Output // this minimum so a ceiling-less model can never truncate every reply below the // point where the model can even emit a compress call or short visible text. It is // a LOWER bound only — a healthy client value above it passes through untouched. +// #3406: the floor applies ONLY when the session has no FUNCTIONAL bound of +// its own (born-starved / fork-inherited rattle, or a high-water below +// OUTPUT_HIGH_WATER_TRUST_MIN). A high-water at/above the trust line is the +// session's own proven budget — the healthy branch forwarded it verbatim and the +// provider accepted it at full value — so it is an explicit smaller legal bound: +// raising it to this floor trades a degraded-but-working session for a hard 400 +// on any provider whose maxOutputTokens cap sits below 32K. // Magnitude is 32K, not 8K (#3071 follow-up, owner decision 2026-10-15): a ceiling-less // model generally sustains >=32K of output, while 8K is consumed by a single modest // thinking block before any visible text survives, so 8K left the problem unfixed. @@ -311,6 +321,17 @@ export function writeOutputBudget(parsed: Record, field: Output // regardless of this floor. export const OUTPUT_RESTORE_ABSOLUTE_FLOOR = 32768; +// #3406: a recorded high-water is trusted as the restore target (never raised +// to the absolute floor) only when it is a FUNCTIONAL budget — at or above the +// same capability line the #453 downstream clamp already enforces +// (OUTPUT_CLAMP_MIN_OUTPUT, src/server/budget.ts: "capability floor for the +// clamped budget", #3071/#2490) and the same magnitude a real host ships as a +// deliberate preset (dsh bundle 4096, "a ceiling against runaway output"). A +// high-water below it is the observed #3071 death-rattle zone (decay chains +// …, 2061, 252): too small to emit a compress call or visible text, so it is +// NOT treated as a legal bound and the floor still overrides it. +const OUTPUT_HIGH_WATER_TRUST_MIN = 4096; + // #1840/#3071: one-shot "restore has no ceiling" warnings, keyed by SESSION+MODEL. // #1840 keyed them by MODEL alone, but the degradation is SESSION-scoped: the // session actually being starved got no warn because another session hit the same @@ -366,13 +387,17 @@ function clearNoCeilingDegraded(session: { metadata: Record }, * still bounds the result by real window headroom. A ceiling at/below the * side-request threshold is not usable; a healthy water mark above the * ceiling keeps winning (it is a capability floor, not an instruction cap). - * #3071: when NO source knows a ceiling, a STARVED / born-starved / - * fork-inherited value must never leave the proxy below - * OUTPUT_RESTORE_ABSOLUTE_FLOOR — the restore target is floored there so a - * ceiling-less model can never pin the session at a death rattle (the client's - * own decaying high-water is itself the poisoned quantity). A non-starved value - * is always trusted and recorded verbatim as the high-water. The no-ceiling - * condition warns once per SESSION+model (the degradation is session-scoped). */ + * #3071/#3406: when NO source knows a ceiling, a starved request is floored + * at OUTPUT_RESTORE_ABSOLUTE_FLOOR unless the session's recorded high-water is + * a functional budget (>= OUTPUT_HIGH_WATER_TRUST_MIN) — a fresh/forked + * session with no high-water and a decayed death-rattle high-water both keep + * the floor (#3071), while a proven working high-water is restored to AS-IS + * and never raised above (#3406): it is the budget the provider already + * accepted at full value, and raising it to the 32K floor hard-rejects on + * providers that cap maxOutputTokens below 32K (availability over ideology, + * #3128). A non-starved value is always trusted and recorded verbatim as the + * high-water. The no-ceiling condition warns once per SESSION+model (the + * degradation is session-scoped). */ export function restoreOutputBudget( parsed: unknown, session: { id: string; metadata: Record }, @@ -405,17 +430,29 @@ export function restoreOutputBudget( if (ceiling !== undefined && (target === undefined || ceiling > target)) { target = ceiling; } else if (ceiling === undefined) { - // #3071 path 1 + path 2b: a starved budget must never leave the proxy - // verbatim. A fresh fork child inherits the parent's rattle with no - // high-water of its own, so this branch guarantees max_tokens is always - // raised to at least the capability floor when no ceiling is known. - if (target === undefined || OUTPUT_RESTORE_ABSOLUTE_FLOOR > target) target = OUTPUT_RESTORE_ABSOLUTE_FLOOR; + // #3071 path 1 + path 2b: a starved budget with NO usable bound of its + // own must never leave the proxy verbatim — a fresh fork child inherits + // the parent's rattle with no high-water, and a death-rattle high-water + // (below OUTPUT_HIGH_WATER_TRUST_MIN) is itself the poisoned quantity, + // so both fall to the capability floor. + // #3406: a high-water AT/ABOVE the trust line is the session's own + // proven budget — the healthy branch forwarded it verbatim and the + // provider accepted it at full value — so it is an explicit smaller + // legal bound. Never raise above it: a provider hard-capping + // maxOutputTokens below this floor rejects the raised value outright + // (400), trading a degraded-but-working session for a failing one + // (availability over ideology, #3128). + if (target === undefined || target < OUTPUT_HIGH_WATER_TRUST_MIN) target = OUTPUT_RESTORE_ABSOLUTE_FLOOR; } if (typeof target === "number") { writeOutputBudget(p, field, target); const note = ceiling !== undefined && target === ceiling ? (highWater === undefined ? "; no healthy high-water yet — using known output ceiling (#1665/#1840)" : `; high-water ${highWater} below known output ceiling — floored (#1665/#1840)`) - : (ceiling === undefined ? "; no known output ceiling — floored to the absolute capability minimum (#3071)" : ""); + : (ceiling === undefined + ? (target === highWater + ? "; no known output ceiling — restored to the session's own recorded budget, never raised above it (#3406)" + : "; no known output ceiling — floored to the absolute capability minimum (#3071)") + : ""); log("info", `[${session.id}] output budget restored ${value} -> ${target} (#546: client shrank it from its raw-history estimate${note})`); } // #2904: persist the no-ceiling degraded state for the user-visible surfaces. diff --git a/src/session-id.ts b/src/session-id.ts index 51e9f86ef4..cd993ad54d 100644 --- a/src/session-id.ts +++ b/src/session-id.ts @@ -1,5 +1,6 @@ import { isCodexClient } from "./codex-compact.js"; import { isLoopbackAddress } from "./util.js"; +import { resolveWireClientProfile } from "./server/lane-decisions.js"; export type ConversationIdentity = { value: string; @@ -120,17 +121,20 @@ export function conversationHeaderSource(headers: Record): boolean { - const turnMeta = headers["x-codex-turn-metadata"]; - if (typeof turnMeta === "string" && turnMeta.trim().length > 0) return true; - // One definition of "codex traffic" process-wide (every other codex path in - // server.ts uses isCodexClient, #645). Its lenient fallback — a UA - // component starting with lowercase "codex" (#1641) — stays CASE-SENSITIVE - // precisely because a case-insensitive match would pull non-codex relays - // with "Codex"-shaped UAs back into the fingerprint and re-fork them - // mid-conversation (#1106); only known codex prefixes (incl. Codex - // Desktop, #1169) match case-insensitively. - if (isCodexClient(headers)) return true; - return conversationHeaderSource(headers)?.name === "x-claude-code-session-id"; + // Detection only — the client-identity DECISION (who keys by + // instructions) lives in the lane-decisions wire-client registry + // (#3482 site 4): the header-name literals are registry data, not + // scattered comparisons. Signals are name-free observations here. + const headerNames = new Set(); + for (const [name, value] of Object.entries(headers ?? {})) { + const first = Array.isArray(value) ? value[0] : value; + if (typeof first === "string" && first.trim().length > 0) headerNames.add(name); + } + return resolveWireClientProfile({ + headerNames, + conversationSourceName: conversationHeaderSource(headers)?.name, + codexUserAgent: isCodexClient(headers), + }).instructionsFingerprintKeyed; } /** #1916/#1307/#1314: whether the persona fingerprint (session id + system diff --git a/src/session-self-heal.ts b/src/session-self-heal.ts index 45082f5ca1..b58e13c756 100644 --- a/src/session-self-heal.ts +++ b/src/session-self-heal.ts @@ -120,7 +120,11 @@ function armSelfHeal(session: Session, detected: "plugin-lane-gone" | "nudge-idl session.metadata["selfHeal"] = { detected, action, since: Date.now() }; } -function reductionKey(session: Session): string { +/** Fingerprint of "a reduction landed": a committed fold advances + * nextBlockId (or the plugin-lane compress tool stamps lastCompress). Exported + * so other nudge machinery (#3461 emergency backoff) watches the SAME evidence + * instead of inventing a parallel signal. */ +export function reductionKeyOf(session: Session): string { return `${session.state.nextBlockId}:${session.lastCompress?.at ?? 0}`; } @@ -142,7 +146,8 @@ type SelfHealRoundInputs = { pluginHeaderPresent: boolean; /** the request's own tools array declares bili tools (live MCP lane) */ biliToolsDeclared: boolean; - /** a nudge was actually injected into this round's payload */ + /** a nudge was injected into this round's payload, or the round carried + * admitted over-limit pressure held only by the #3461 backoff ("gated") */ nudgeActive: boolean; /** bili tool_use entries in the re-sent history (plugin-mode evidence the agent is still compressing) */ biliToolUses: number; @@ -159,7 +164,7 @@ type LogFn = (level: "info" | "warn", message: string) => void; export function evaluateSelfHealRound(session: Session, inputs: SelfHealRoundInputs, log: LogFn): void { const st = counters.get(session) ?? { lastReductionKey: undefined, zombieStreak: 0, idleStreak: 0 }; counters.set(session, st); - const key = reductionKey(session); + const key = reductionKeyOf(session); const reduction = st.lastReductionKey !== undefined && key !== st.lastReductionKey; st.lastReductionKey = key; diff --git a/src/storage/engine-bootstrap.ts b/src/storage/engine-bootstrap.ts index 80afec8336..b95795545a 100644 --- a/src/storage/engine-bootstrap.ts +++ b/src/storage/engine-bootstrap.ts @@ -58,7 +58,11 @@ const SUPPORTED_TARGETS = [ "win32-x64", "win32-arm64", ]; -const BOOTSTRAP_TIMEOUT_MS = 10_000; +// Whole-download budget for the ~11 MB engine tarball. Must cover a multi-MB +// registry fetch over a non-trivial network (corporate proxy / slow link); +// 10 s locked Node<=21 machines into a 6 h legacy-only cooldown after one slow +// boot (#3338). Aligned with the updater's own tarball budget (src/update.ts). +export const BOOTSTRAP_TIMEOUT_MS = 60_000; export const COOLDOWN_MS = 6 * 60 * 60 * 1000; type BootstrapStatus = @@ -94,6 +98,13 @@ export function resetEngineBootstrapForTests(): void { inFlight = null; } +let bootstrapTimeoutOverrideMs: number | null = null; + +/** Test seam: override the whole-download timeout (null = production value). */ +export function setBootstrapTimeoutMsForTests(ms: number | null): void { + bootstrapTimeoutOverrideMs = ms; +} + function activeSpec(): EngineBootstrapSpec { return specOverride ?? DEFAULT_ENGINE_SPEC; } @@ -226,7 +237,7 @@ async function runBootstrap(opts: EnsureSqliteEngineOptions): Promise= 20 is in-contract, but node:sqlite ships at >= 22.5, so + * Node 20 users see "no SQLite engine" unless the tarball download succeeds. + * Pure (version in → text out) so tests pin the threshold logic; empty string + * on Nodes that already have the builtin. + */ +export function nodeSqliteUpgradeHint(nodeVersion: string = process.versions.node): string { + const parts = nodeVersion.split(".").map((p) => Number.parseInt(p, 10)); + const major = parts[0]; + const minor = parts[1]; + const hasBuiltin = typeof major === "number" && typeof minor === "number" && !Number.isNaN(major) && !Number.isNaN(minor) && (major > 22 || (major === 22 && minor >= 5)); + if (hasBuiltin) return ""; + return " — node:sqlite needs Node.js >= 22.5; upgrade Node.js for the builtin engine (engine tarball mirrors tried in order: npm + npmmirror; set BILI_UPDATE_REGISTRY or update.registry to pin one)"; } function tryRegisterFromCache(dir: string): boolean { @@ -277,41 +327,32 @@ async function pruneSiblingVersions(keepDir: string): Promise { // --------------------------------------------------------------------------- async function downloadAndInstall(spec: EngineBootstrapSpec, target: string, resolveProxy?: (url: string) => string | undefined): Promise { - const base = registryBaseFor(spec); - const url = `${base}/better-sqlite3/-/better-sqlite3-${spec.version}.tgz`; const root = cacheRoot(); await fsp.mkdir(root, { recursive: true }); const tmpTgz = path.join(root, `.dl-${randomBytes(6).toString("hex")}.tgz`); const tmpDir = path.join(root, `.wip-${randomBytes(6).toString("hex")}`); try { - const dispatcher = proxyDispatcher(resolveProxy?.(url)); - // @types/node types RequestInit.dispatcher as its internal Dispatcher; - // cast once here (no `as any`), same pattern as src/update.ts fetchWithEgress. - const res = await fetch(url, { - signal: AbortSignal.timeout(BOOTSTRAP_TIMEOUT_MS), - ...(dispatcher ? { dispatcher } : {}), - } as RequestInit); - if (!res.ok || !res.body) throw new Error(`registry returned HTTP ${res.status} for ${url}`); - const hash = createHash("sha512"); - const out = await fsp.open(tmpTgz, "w"); - try { - const reader = res.body.getReader(); - for (;;) { - const { done, value } = await reader.read(); - if (done) break; - if (value) { - hash.update(value); - await out.write(value); - } + // #3338 directive 2: walk the mirror list; the FIRST mirror that + // yields a sha512-verified tarball wins. Fetch + integrity are + // per-mirror; extraction/publishing happens once, mirror-independent. + let tgz: Buffer | null = null; + const attempted: string[] = []; + let lastErr: unknown = null; + for (const base of engineMirrors()) { + const url = `${base}/better-sqlite3/-/better-sqlite3-${spec.version}.tgz`; + attempted.push(base); + try { + tgz = await fetchVerifiedTarball(url, spec, tmpTgz, resolveProxy); + break; + } catch (err) { + lastErr = err; + loggerLog("warn", `[storage] sqlite engine self-bootstrap: mirror ${base} failed (${err instanceof Error ? err.message : String(err)}) — trying next mirror`); } - } finally { - await out.close(); } - const digest = hash.digest("base64"); - if (digest !== spec.sha512Base64) { - throw new Error(`sha512 mismatch (got ${digest.slice(0, 16)}..., pinned ${spec.sha512Base64.slice(0, 16)}...) — quarantining download`); + if (!tgz) { + const reason = lastErr instanceof Error ? lastErr.message : String(lastErr); + throw new Error(`${reason} (tried mirrors: ${attempted.join(", ")})`); } - const tgz = await fsp.readFile(tmpTgz); await extractTgzToDir(tgz, tmpDir, target); const dest = finalDir(spec.version); if (existsSync(dest)) { @@ -329,6 +370,40 @@ async function downloadAndInstall(spec: EngineBootstrapSpec, target: string, res } } +/** One mirror attempt: stream the tarball to `tmpTgz` while hashing; return + * the bytes only when the pinned sha512 matches. Throws on timeout / HTTP + * error / integrity mismatch so the caller can advance to the next mirror. */ +async function fetchVerifiedTarball(url: string, spec: EngineBootstrapSpec, tmpTgz: string, resolveProxy?: (url: string) => string | undefined): Promise { + const dispatcher = proxyDispatcher(resolveProxy?.(url)); + // @types/node types RequestInit.dispatcher as its internal Dispatcher; + // cast once here (no `as any`), same pattern as src/update.ts fetchWithEgress. + const res = await fetch(url, { + signal: AbortSignal.timeout(bootstrapTimeoutOverrideMs ?? BOOTSTRAP_TIMEOUT_MS), + ...(dispatcher ? { dispatcher } : {}), + } as RequestInit); + if (!res.ok || !res.body) throw new Error(`registry returned HTTP ${res.status} for ${url}`); + const hash = createHash("sha512"); + const out = await fsp.open(tmpTgz, "w"); + try { + const reader = res.body.getReader(); + for (;;) { + const { done, value } = await reader.read(); + if (done) break; + if (value) { + hash.update(value); + await out.write(value); + } + } + } finally { + await out.close(); + } + const digest = hash.digest("base64"); + if (digest !== spec.sha512Base64) { + throw new Error(`sha512 mismatch (got ${digest.slice(0, 16)}..., pinned ${spec.sha512Base64.slice(0, 16)}...) — quarantining download`); + } + return await fsp.readFile(tmpTgz); +} + // --------------------------------------------------------------------------- // Minimal tgz extractor (devDeps are unavailable at runtime — bili ships a // self-contained bundle). Supports exactly what npm tarballs of the pinned diff --git a/src/storage/runtime-status.ts b/src/storage/runtime-status.ts index e33846db11..9f71260242 100644 --- a/src/storage/runtime-status.ts +++ b/src/storage/runtime-status.ts @@ -24,7 +24,7 @@ */ import type { BootstrapOutcome, CooldownMarker } from "./engine-bootstrap.js"; -import { COOLDOWN_MS, readBootstrapFailureMarker } from "./engine-bootstrap.js"; +import { COOLDOWN_MS, nodeSqliteUpgradeHint, readBootstrapFailureMarker } from "./engine-bootstrap.js"; import { fileStorageAutoDownload, fileStorageJsonWriter, fileStorageUnified, fileStorageWalJournalSizeLimit, resolveStorageMode } from "../knobs.js"; import { MAX_CONSECUTIVE_FAILURES, type UnifiedShadow } from "./shadow.js"; import type { StorageMode } from "./shadow.js"; @@ -85,7 +85,7 @@ function degradeReason(): string { // The driver's no-engine error surfaces through the same open path — // say what it actually is instead of "index database failed to open". if (/no SQLite engine available/i.test(e)) { - return "no SQLite engine available (node:sqlite missing, better-sqlite3 not installed)"; + return `no SQLite engine available (node:sqlite missing, better-sqlite3 not installed)${nodeSqliteUpgradeHint()}`; } return `index database failed to open (${e})`; } @@ -104,7 +104,7 @@ function degradeReason(): string { break; } } - return "no SQLite engine available (node:sqlite missing, better-sqlite3 not installed)"; + return `no SQLite engine available (node:sqlite missing, better-sqlite3 not installed)${nodeSqliteUpgradeHint()}`; } const COOLDOWN_CACHE_MS = 30_000; diff --git a/src/store.ts b/src/store.ts index 2527fde18e..ea0862337e 100644 --- a/src/store.ts +++ b/src/store.ts @@ -58,6 +58,27 @@ export function ccrEnabled(session: Session | undefined): boolean { return effectiveCcr(session)?.enabled === true; } +// #3400: per-request stamp of the read_original_image TOOL registration gate. +// The rescale PASS itself runs on every lane (governed by the BILI_IMAGE_RESCALE +// knob); only the manifest registration and the annotation's invocation entry +// follow this stamp — codex/plugin lanes get size-only notes. +const EFFECTIVE_IMAGE_RESCALE_KEY = "effectiveImageRescale"; + +/** Stamp whether read_original_image may be registered for this session's + * lane (per-request; mirrors the storeEffectiveCcr site). */ +export function storeEffectiveImageRescale(session: Session, armed: boolean | undefined): void { + // Absent == not armed (imageRescaleToolArmed reads === true). Writing only the + // armed state keeps non-armed lanes' metadata byte-identical to pre-#3400 + // (storage-render digests pin that shape). + if (armed === true) session.metadata[EFFECTIVE_IMAGE_RESCALE_KEY] = true; + else delete session.metadata[EFFECTIVE_IMAGE_RESCALE_KEY]; +} + +/** Read back the #3400 tool-registration stamp. */ +export function imageRescaleToolArmed(session: Session | undefined): boolean { + return session?.metadata[EFFECTIVE_IMAGE_RESCALE_KEY] === true; +} + /** Model-facing retrieve tool name for this session: the config `ccr.toolName` * override when set, else the kernel default. */ export function retrieveToolName(session: Session | undefined): string { diff --git a/src/stream-google.ts b/src/stream-google.ts index 723493f4ce..54dc5dc18a 100644 --- a/src/stream-google.ts +++ b/src/stream-google.ts @@ -1,7 +1,7 @@ import type { CompressionCore, Config, CoreMessage } from "acp-kernel"; import type { GooglePart } from "acp-kernel/wire"; import type { Session } from "./session.js"; -import { isProxyToolFor } from "./proxy-tool.js"; +import { effectiveAbsorbConfig, isProxyToolFor } from "./absorb.js"; import { executeProxyToolAsync } from "./loop/core.js"; import { drainPendingRetrievals } from "./store.js"; import { runJsonRewriteAsync, type JsonToolCall, type RewriteCtx } from "./stream.js"; @@ -26,7 +26,7 @@ function* rewriteGoogleJsonSteps(body: unknown, ctx: RewriteCtx): Generator injection. Direct + // serving (/__bili/) keeps the byte-identical root "/__bili". + let API_ROOT = "/__bili"; + try { + if (typeof location !== "undefined" && location.pathname) { + const m = /^(.*\\/)__bili(\\/|$)/.exec(location.pathname); + if (m && m[1] !== undefined) API_ROOT = m[1] + "__bili"; + } + } catch (e) {} function t(key, vars) { let text = MESSAGES[locale][key]; if (text === undefined) text = MESSAGES["zh-CN"][key]; @@ -273,7 +285,7 @@ export const WEB_CLIENT = `(function () { async function putCfg(btn, payload) { busy(btn, true); try { - await json("/__bili/config", { method: "PUT", headers: { "content-type": "application/json" }, body: JSON.stringify(payload) }); + await json(API_ROOT + "/config", { method: "PUT", headers: { "content-type": "application/json" }, body: JSON.stringify(payload) }); toast(t("cfg.saved"), "ok"); loadConfig(); } catch (e) { @@ -480,7 +492,7 @@ export const WEB_CLIENT = `(function () { async function loadOverview(silent) { try { - const d = await json("/__bili/overview"); + const d = await json(API_ROOT + "/overview"); const o = d.overview || {}; // #1426: total splits live vs historical (disk-restored pool entries are history); // counters without usage samples render "—" instead of a misleading 0; the saved @@ -632,7 +644,7 @@ export const WEB_CLIENT = `(function () { btn.addEventListener("click", async () => { busy(btn, true); try { - await json("/__bili/conflicts/clear", { method: "POST" }); + await json(API_ROOT + "/conflicts/clear", { method: "POST" }); toast(t("conflict.cleared"), "ok"); await loadOverview(true); } catch (e) { @@ -704,7 +716,7 @@ export const WEB_CLIENT = `(function () { btn.addEventListener("click", async () => { busy(btn, true); try { - await json("/__bili/host-plugins/dismiss", { method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify({ id: btn.getAttribute("data-hpid") || "" }) }); + await json(API_ROOT + "/host-plugins/dismiss", { method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify({ id: btn.getAttribute("data-hpid") || "" }) }); toast(t("hostplug.dismissed"), "ok"); await loadOverview(true); } catch (e) { @@ -832,7 +844,7 @@ export const WEB_CLIENT = `(function () { function sessionListUrl(page) { const input = $("ses-search"); const q = ((input && input.value) || "").trim(); - return "/__bili/sessions?page=" + page + "&pageSize=" + SES_PAGE_SIZE + (q ? "&q=" + encodeURIComponent(q) : ""); + return API_ROOT + "/sessions?page=" + page + "&pageSize=" + SES_PAGE_SIZE + (q ? "&q=" + encodeURIComponent(q) : ""); } async function loadMoreSessions() { const btn = $("ses-loadmore"); @@ -1703,7 +1715,7 @@ export const WEB_CLIENT = `(function () { host.innerHTML = '
' + t("common.loading") + "
"; let d = null; try { - d = await json("/__bili/sessions/" + encodeURIComponent(id) + "/detail"); + d = await json(API_ROOT + "/sessions/" + encodeURIComponent(id) + "/detail"); } catch (e) {} if (!d) { host.innerHTML = '' + t("common.back") + "" @@ -1722,7 +1734,7 @@ export const WEB_CLIENT = `(function () { cc.addEventListener("click", async () => { busy(cc, true); try { - await json("/__bili/conflicts/clear?session=" + encodeURIComponent(id), { method: "POST" }); + await json(API_ROOT + "/conflicts/clear?session=" + encodeURIComponent(id), { method: "POST" }); toast(t("det.conflicts_cleared"), "ok"); await loadDetail(id); } catch (e) { @@ -1793,7 +1805,7 @@ export const WEB_CLIENT = `(function () { if (hit && Date.now() - hit.at < DIAG_CACHE_MS) return bindReport(hit.d, copyBtn, issueLink); busy(copyBtn, true); try { - const d = await json("/__bili/diagnostics?topic=" + encodeURIComponent(topic)); + const d = await json(API_ROOT + "/diagnostics?topic=" + encodeURIComponent(topic)); diagCache[topic] = { at: Date.now(), d }; bindReport(d, copyBtn, issueLink); } catch (e) { @@ -1863,7 +1875,7 @@ export const WEB_CLIENT = `(function () { if (mdBtn) mdBtn.addEventListener("click", async function () { busy(mdBtn, true); try { - var r = await json("/__bili/session-report?format=md&" + q); + var r = await json(API_ROOT + "/session-report?format=md&" + q); if (!r || typeof r.md !== "string") throw new Error("no report"); downloadUrl(URL.createObjectURL(new Blob([r.md], { type: "text/markdown;charset=utf-8" })), base + ".md"); } catch (e) { toast(t("toast.failed", { msg: e.message }), "err"); } finally { busy(mdBtn, false); } @@ -1871,7 +1883,7 @@ export const WEB_CLIENT = `(function () { if (pngBtn) pngBtn.addEventListener("click", async function () { busy(pngBtn, true); try { - var r = await json("/__bili/session-report?format=svg&" + q); + var r = await json(API_ROOT + "/session-report?format=svg&" + q); if (!r || !r.svg) throw new Error("no report"); var blob = await svgToPngBlob(r.svg, r.w || 920, r.h || 800); downloadUrl(URL.createObjectURL(blob), base + ".png"); @@ -1906,7 +1918,7 @@ export const WEB_CLIENT = `(function () { const dlB = $("cacherpt-dl"); if (!copyB || !dlB) return; const grab = async () => { - const r = await json("/__bili/cache-report?session=" + encodeURIComponent(d.id)); + const r = await json(API_ROOT + "/cache-report?session=" + encodeURIComponent(d.id)); const rep = r && Array.isArray(r.reports) && r.reports[0] ? r.reports[0].report : null; if (!rep) throw new Error("no cache report for this session yet"); return rep; @@ -2009,7 +2021,7 @@ export const WEB_CLIENT = `(function () { async function loadConfig() { try { - const cfg = await json("/__bili/config"); + const cfg = await json(API_ROOT + "/config"); const cfgPathEl = $("cfg-path"); if (cfgPathEl) cfgPathEl.textContent = cfg.path || t("common.empty"); const fileBtn = $("copy-cfg-file"); @@ -2075,7 +2087,7 @@ export const WEB_CLIENT = `(function () { if (!box) return; let data; try { - data = await json("/__bili/resign"); + data = await json(API_ROOT + "/resign"); } catch (e) { const el = document.createElement("div"); el.className = "dim small"; @@ -2573,7 +2585,7 @@ export const WEB_CLIENT = `(function () { async function saveKey(btn, value) { busy(btn, true); try { - const result = await json("/__bili/external-summary/credential", { method: "PUT", headers: { "content-type": "application/json" }, body: JSON.stringify({ name: recipe.credentialRef, key: value }) }); + const result = await json(API_ROOT + "/external-summary/credential", { method: "PUT", headers: { "content-type": "application/json" }, body: JSON.stringify({ name: recipe.credentialRef, key: value }) }); key.value = ""; status[ref] = result.configured; render(); toast(t("summary.key_saved"), "ok"); @@ -2602,7 +2614,7 @@ export const WEB_CLIENT = `(function () { } async function loadUpstream(cfg) { let up = null; - try { up = await json("/__bili/upstream"); } catch (e) {} + try { up = await json(API_ROOT + "/upstream"); } catch (e) {} // #1426: mode/proxy are editable form fields again, not read-only labels const mode = (up && up.mode) || cfg.upstreamProxyMode || "auto"; document.querySelectorAll('input[name="proxy-mode"]').forEach((el) => { el.checked = el.value === mode; }); @@ -2634,7 +2646,7 @@ export const WEB_CLIENT = `(function () { const q = (qEl.value || "").trim(); try { const linesSel = $("log-lines"); - const d = await json("/__bili/logs?q=" + encodeURIComponent(q) + "&lines=" + (linesSel ? linesSel.value : "500") + logViewExtras()); + const d = await json(API_ROOT + "/logs?q=" + encodeURIComponent(q) + "&lines=" + (linesSel ? linesSel.value : "500") + logViewExtras()); $("log-path").textContent = d.path || t("logs.empty"); $("copy-log-path").dataset.copy = d.path || ""; $("log-count").textContent = d.total > 0 @@ -2728,7 +2740,7 @@ export const WEB_CLIENT = `(function () { busy(ldl, true); try { const q = ($("log-search").value || "").trim(); - const r = await fetch("/__bili/logs?raw=1&q=" + encodeURIComponent(q) + "&lines=2000" + logViewExtras()); + const r = await fetch(API_ROOT + "/logs?raw=1&q=" + encodeURIComponent(q) + "&lines=2000" + logViewExtras()); const blob = await r.blob(); const a = document.createElement("a"); a.href = URL.createObjectURL(blob); @@ -2749,7 +2761,7 @@ export const WEB_CLIENT = `(function () { try { // Full unfiltered log (rotated .old + current) regardless of the // search box — the honest "download everything" path. - const r = await fetch("/__bili/logs?raw=1&all=1"); + const r = await fetch(API_ROOT + "/logs?raw=1&all=1"); const blob = await r.blob(); const a = document.createElement("a"); a.href = URL.createObjectURL(blob); @@ -2768,7 +2780,7 @@ export const WEB_CLIENT = `(function () { if (testBtn) testBtn.addEventListener("click", async () => { busy(testBtn, true); try { - const r = await json("/__bili/upstream/test", { method: "POST" }); + const r = await json(API_ROOT + "/upstream/test", { method: "POST" }); // #1426: an HTTP >= 400 answer still proves the network path works — auth is the // client's job, so report reachability instead of a flat failure const st = $("up-state"); @@ -2820,7 +2832,7 @@ export const WEB_CLIENT = `(function () { if (cp) cp.addEventListener("click", async () => { busy(cp, true); try { - await json("/__bili/config", { method: "PUT", headers: { "content-type": "application/json" }, body: JSON.stringify({ passthrough: null }) }); + await json(API_ROOT + "/config", { method: "PUT", headers: { "content-type": "application/json" }, body: JSON.stringify({ passthrough: null }) }); toast(t("toast.passthrough_cleared"), "ok"); loadConfig(); } catch (e) { diff --git a/src/web/i18n.ts b/src/web/i18n.ts index b49c5d8ee7..dee2ee9f16 100644 --- a/src/web/i18n.ts +++ b/src/web/i18n.ts @@ -438,7 +438,7 @@ export const MESSAGES: Record> = { "con.note_gemini": "Gemini CLI(Google):GOOGLE_GEMINI_BASE_URL /bili/ 改写到 generativelanguage.googleapis.com(Google 原生 wire);真实 ~/.gemini 零改动", "con.note_iflow": "iFlow CLI:IFLOW_BASE_URL /bili/ 改写到 apis.iflow.cn/v1(OpenAI chat-completions wire);真实 ~/.iflow 零改动", "con.note_qwen": "Qwen Code(多协议 gemini-cli fork,无 base-URL 钩子):HTTPS_PROXY + NODE_EXTRA_CA_CERTS 证书 MITM;DashScope/Qwen 默认模型主机加白,自建中转用 --mitm-domain 追加", - "con.note_antigravity": "Antigravity CLI(agy,Google,Gemini CLI 继任者):CLOUD_CODE_URL /bili/ 改写到 cloudcode-pa.googleapis.com(语言服务器内未公开的环境变量覆盖,v2.19.1 二进制验证;wire 按路径识别为 Google 原生)(#2115)", + "con.note_antigravity": "Antigravity CLI(agy,Google,Gemini CLI 继任者):CLOUD_CODE_URL /bili/ 改写到 daily-cloudcode-pa.googleapis.com —— 真实默认上游(裸 cloudcode-pa 在 /v1internal:* 上 400,#3129;语言服务器内未公开的环境变量覆盖,v2.19.1 二进制验证;wire 按路径识别为 Google 原生)(#2115)", "con.note_mcode": "MiniMax Code CLI:除无条件回环绕过外遵循标准代理环境变量——非回环 https 证书 MITM、http 绝对形式转发;主机取 ~/.minimax*/config.yaml;会话经 X-Mavis-Session-Id 头绑定(#1050)", "con.note_aider": "Aider(Python pair programmer):HTTPS_PROXY + SSL_CERT_FILE/REQUESTS_CA_BUNDLE 证书 MITM;端点取自继承 env、--openai-api-base 或 .aider.conf.yml;回环端点经 NO_PROXY 直连(#1048)", "con.note_copilot": "Copilot CLI(GitHub,闭源 Go 二进制)——证书 MITM,api.githubcopilot.com + 各套餐子域加白(#1049)", @@ -914,7 +914,7 @@ export const MESSAGES: Record> = { "con.note_gemini": "Gemini CLI (Google): GOOGLE_GEMINI_BASE_URL /bili/ rewritten to generativelanguage.googleapis.com (Google native wire); real ~/.gemini zero-touch", "con.note_iflow": "iFlow CLI: IFLOW_BASE_URL /bili/ rewritten to apis.iflow.cn/v1 (OpenAI chat-completions wire); real ~/.iflow zero-touch", "con.note_qwen": "Qwen Code (multi-protocol gemini-cli fork, no base-URL hook): certificate MITM via HTTPS_PROXY + NODE_EXTRA_CA_CERTS; DashScope/Qwen default model hosts whitelisted; self-hosted relays appended via --mitm-domain", - "con.note_antigravity": "Antigravity CLI (agy, Google, Gemini CLI successor): CLOUD_CODE_URL /bili/ rewritten to cloudcode-pa.googleapis.com (undocumented language-server env override, verified in the v2.19.1 binary; wire recognized as Google native by path) (#2115)", + "con.note_antigravity": "Antigravity CLI (agy, Google, Gemini CLI successor): CLOUD_CODE_URL /bili/ rewritten to daily-cloudcode-pa.googleapis.com — the real default upstream (bare cloudcode-pa 400s on /v1internal:*, #3129; undocumented language-server env override, verified in the v2.19.1 binary; wire recognized as Google native by path) (#2115)", "con.note_mcode": "MiniMax Code CLI: aside from the unconditional loopback bypass, standard proxy env — non-loopback https certificate MITM, http absolute-form forwarding; hosts from ~/.minimax*/config.yaml; sessions bound via X-Mavis-Session-Id header (#1050)", "con.note_aider": "Aider (Python pair programmer): certificate MITM via HTTPS_PROXY + SSL_CERT_FILE/REQUESTS_CA_BUNDLE; endpoint from inherited env, --openai-api-base or .aider.conf.yml; loopback endpoints direct via NO_PROXY (#1048)", "con.note_copilot": "Copilot CLI (GitHub, closed-source Go binary) — certificate MITM; api.githubcopilot.com plus tier subdomains whitelisted (#1049)", diff --git a/tests/absorb-plugin.test.ts b/tests/absorb-plugin.test.ts new file mode 100644 index 0000000000..519cbf8cbf --- /dev/null +++ b/tests/absorb-plugin.test.ts @@ -0,0 +1,283 @@ +// #1359: absorb plugin-lane tool-name divergence (same class as #1345/#1360 CCR). +// The manifest must advertise the SAME name the per-session gate adjudicates. +// Plugin lane governs the whole absorb block by the BASE config (provider/model +// overrides are proxy-lane-only); proxy lane follows the per-request merged block. +// Regression pinned here: a base-level rename advertised the static name while the +// gate expected the renamed one, so the host-registered tool 400'd on the plugin lane. + +import test from "node:test"; +import assert from "node:assert/strict"; +import http from "node:http"; +import { once } from "node:events"; +import type { AddressInfo } from "node:net"; + +process.env.NODE_ENV = "test"; +process.env.BILI_PERSIST = "0"; + +import { defaultConfig, DEFAULT_ABSORB_CONFIG, type Config } from "acp-kernel"; +import { startServer } from "../src/server.ts"; +import { findAbsorbPluginDivergences, type CompressSettings, type ProxyOptions } from "../src/config.ts"; +import { SessionStore, _setStoreForTest } from "../src/persist.ts"; +import { _setForTest as setRegistryForTest } from "../src/registry.ts"; +import { listSessions, getSession } from "../src/session.ts"; +import { handlePluginManifest } from "../src/plugin.ts"; +import { effectiveAbsorbConfig, storeEffectiveAbsorb, isProxyToolFor } from "../src/absorb.ts"; +import { applyCompressSettings } from "../src/compress-settings.ts"; +import { effectiveAbsorbBlock } from "../src/server/prepare-responses.ts"; + +const MODEL = "test-model"; + +function readManifest(config: Parameters[1]): { toolNames: string[]; tools: Record } { + let body = ""; + const res = { writeHead: () => {}, end: (b: string) => { body = b; } } as unknown as Parameters[0]; + handlePluginManifest(res, config); + return JSON.parse(body) as { toolNames: string[]; tools: Record }; +} + +function namesOnWire(wire: unknown[]): string[] { + return wire.map((t) => { + const o = t as Record; + return typeof o.name === "string" ? o.name : String((o.function as { name?: unknown } | undefined)?.name ?? ""); + }); +} + +// — manifest advertising follows base absorb.toolName (the #1359 core) — + +test("handlePluginManifest: absorb NOT advertised when disabled", () => { + const m = readManifest(defaultConfig(200_000)); + assert.ok(!m.toolNames.includes("absorb"), "disabled by default → not advertised"); +}); + +test("handlePluginManifest: enabled absorb with default name advertises 'absorb'", () => { + const m = readManifest({ ...defaultConfig(200_000), absorb: { ...DEFAULT_ABSORB_CONFIG, enabled: true } }); + assert.ok(m.toolNames.includes("absorb"), "default name advertised"); + for (const wire of ["anthropic", "openai", "responses"]) { + assert.ok(namesOnWire(m.tools[wire] ?? []).includes("absorb"), `${wire} carries 'absorb'`); + } +}); + +test("handlePluginManifest: base absorb.toolName rename is advertised, not the static name", () => { + const m = readManifest({ ...defaultConfig(200_000), absorb: { ...DEFAULT_ABSORB_CONFIG, enabled: true, toolName: "my_absorb" } }); + assert.ok(m.toolNames.includes("my_absorb"), "renamed tool advertised"); + assert.ok(!m.toolNames.includes("absorb"), "static name no longer advertised after rename"); + for (const wire of ["anthropic", "openai", "responses"]) { + const names = namesOnWire(m.tools[wire] ?? []); + assert.ok(names.includes("my_absorb"), `${wire} carries the renamed tool`); + assert.ok(!names.includes("absorb"), `${wire} must not carry the static name`); + } +}); + +// — per-session gate: the advertised (renamed) name is the only accepted one — + +test("gate: plugin-lane stamp makes the renamed name the sole accepted absorb tool", () => { + const base = defaultConfig(200_000); + const renamed: Config = { ...base, absorb: { ...DEFAULT_ABSORB_CONFIG, enabled: true, toolName: "my_absorb", minToolTokens: 1 } }; + const s = getSession(`t-absorb-plugin-${Math.random().toString(36).slice(2)}`); + storeEffectiveAbsorb(s, renamed); + assert.equal(isProxyToolFor("my_absorb", s, base), true, "renamed (advertised) name accepted"); + assert.equal(isProxyToolFor("absorb", s, base), false, "static name rejected after rename"); +}); + +// #3293 guard: the plugin-lane prompt/mark gate and the manifest advertisement must +// key off the SAME base absorb block, else a turn prompts for a tool the host never +// registered (the unknown-tool spin that got absorb deleted in #3309). Drift re-trips #3293. +test("#3293: plugin-lane absorb prompt gate ⟺ manifest advertisement (same base block)", () => { + const cases: Array<{ label: string; absorb?: CompressSettings["absorb"] }> = [ + { label: "unset (default-off)" }, + { label: "explicitly disabled", absorb: { enabled: false } }, + { label: "enabled default-name", absorb: { enabled: true } }, + { label: "enabled renamed", absorb: { enabled: true, toolName: "my_absorb" } }, + ]; + for (const c of cases) { + const compress = { injectTool: true, ...(c.absorb !== undefined ? { absorb: c.absorb } : {}) }; + // Mirror the two production sources that must stay in sync: admin.ts:336 (manifest) + // and prepare-responses.ts:741 (prompt gate). + const resolved = applyCompressSettings(defaultConfig(200_000), 200_000, compress as Parameters[2]); + const name = resolved.absorb?.toolName ?? DEFAULT_ABSORB_CONFIG.toolName; + const advertised = readManifest(resolved).toolNames.includes(name); + const promptGate = effectiveAbsorbBlock(true, resolved, c.absorb)?.enabled === true; + assert.equal(promptGate, advertised, `${c.label}: prompt gate (${promptGate}) must equal manifest advertisement (${advertised})`); + } +}); + +// — load-time per-field divergence diagnostic — + +test("findAbsorbPluginDivergences: aligned / unset levels report nothing", () => { + const u = "http://u"; + assert.deepEqual(findAbsorbPluginDivergences({ [u]: {} }, { enabled: true, minToolTokens: 100 }), []); + assert.deepEqual(findAbsorbPluginDivergences({ [u]: { compress: { absorb: undefined } } }, { enabled: true }), []); + assert.deepEqual(findAbsorbPluginDivergences({ [u]: { compress: { absorb: { enabled: true } } } }, { enabled: true }), []); +}); + +test("findAbsorbPluginDivergences: route-level rename diverges from base", () => { + const u = "http://u"; + const divs = findAbsorbPluginDivergences( + { [u]: { compress: { absorb: { enabled: true, toolName: "route_absorb" } } } }, + { enabled: true }, + ); + assert.equal(divs.length, 1, `expected exactly the toolName field, got ${JSON.stringify(divs)}`); + assert.match(divs[0]!, /toolName=\s*"route_absorb"/); +}); + +test("findAbsorbPluginDivergences: model-level field diverges; only fields the level SETS are reported", () => { + const u = "http://u"; + const divs = findAbsorbPluginDivergences( + { [u]: { models: { m1: { compress: { absorb: { minToolTokens: 999 } } } } } }, + { minToolTokens: 100, contextThresholdPct: 0.5, excludeTools: ["bash"] }, + ); + assert.equal(divs.length, 1, `only the set field (minToolTokens) diverges, got ${JSON.stringify(divs)}`); + assert.match(divs[0]!, /m1\.absorb\.minToolTokens=\s*999/); + assert.ok(!divs.join("").includes("contextThresholdPct"), "unset-at-level field not reported"); + assert.ok(!divs.join("").includes("excludeTools"), "unset-at-level field not reported"); +}); + +test("findAbsorbPluginDivergences: percent form normalizes to the fraction form", () => { + const u = "http://u"; + assert.deepEqual( + findAbsorbPluginDivergences({ [u]: { compress: { absorb: { contextThresholdPct: "80%" } } } }, { contextThresholdPct: 0.8 }), + [], + "'80%' == 0.8 → no divergence", + ); + const divs = findAbsorbPluginDivergences({ [u]: { compress: { absorb: { contextThresholdPct: "90%" } } } }, { contextThresholdPct: 0.8 }); + assert.equal(divs.length, 1, "'90%' != 0.8 → divergence"); +}); + +// — e2e lane governance (real server through the proxy) — + +type Rig = { proxyPort: number; upstreamPort: number; forwards: string[]; proxy: http.Server; upstream: http.Server }; + +function okJson(): string { + return JSON.stringify({ + id: "chatcmpl-1", + object: "chat.completion", + choices: [{ index: 0, message: { role: "assistant", content: "ok" }, finish_reason: "stop" }], + usage: { prompt_tokens: 10, completion_tokens: 3, total_tokens: 13 }, + }); +} + +async function startRig(baseCompress: Record, routeCompress?: Record): Promise { + const forwards: string[] = []; + const upstream = http.createServer((req, res) => { + let b = ""; + req.on("data", (c) => (b += c)); + req.on("end", () => { + forwards.push(b); + res.writeHead(200, { "content-type": "application/json" }); + res.end(okJson()); + }); + }); + upstream.listen(0, "127.0.0.1"); + await once(upstream, "listening"); + const upstreamPort = (upstream.address() as AddressInfo).port; + + _setStoreForTest(new SessionStore({ enabled: false })); + setRegistryForTest({}); + const url = `http://127.0.0.1:${upstreamPort}`; + const routes = routeCompress ? { [url]: { compress: routeCompress } } : { [url]: {} }; + const proxy = await startServer({ + port: 0, + host: "127.0.0.1", + upstream: "http://127.0.0.1", + routes, + modelContextLimit: 200_000, + kernelConfig: defaultConfig(200_000), + compress: baseCompress, + promptCache: { routing: "auto" }, + sessionHeader: "x-acp-session", + log: false, + debug: false, + passthrough: false, + autoUpdate: false, + compat: { roles: {} }, + streamErrorShape: "protocol", + passthroughSource: null, + autoRestartOnUpdate: false, + updateTag: "latest", + advisoryCheck: false, + releaseNotesCheck: false, + mitm: { enabled: false, domains: [] }, + } as unknown as ProxyOptions); + await once(proxy, "listening"); + return { proxyPort: (proxy.address() as AddressInfo).port, upstreamPort, forwards, proxy, upstream }; +} + +async function closeRig(rig: Rig): Promise { + rig.proxy.close(); + rig.proxy.closeIdleConnections?.(); + await once(rig.proxy, "close"); + rig.upstream.close(); + rig.upstream.closeIdleConnections?.(); + await once(rig.upstream, "close"); +} + +async function post(rig: Rig, convId: string, pluginMode: boolean, messages: unknown[]): Promise { + const headers: Record = { "content-type": "application/json", "x-acp-session": convId }; + if (pluginMode) headers["x-bili-plugin"] = "test-agent"; + const res = await fetch(`http://127.0.0.1:${rig.proxyPort}/bili/http://127.0.0.1:${rig.upstreamPort}/v1/chat/completions`, { + method: "POST", + headers, + body: JSON.stringify({ model: MODEL, max_tokens: 64_000, messages }), + }); + const txt = await res.text(); + if (res.status !== 200) throw new Error(`proxy returned ${res.status}: ${txt}`); +} + +const SIMPLE_MSGS = (): unknown[] => [ + { role: "user", content: "hello" }, + { role: "assistant", content: "hi there" }, +]; + +test("e2e: base rename → plugin lane advertises, stamps, and executes the renamed tool", async () => { + const baseCompress = { injectTool: true, injectNudge: false, absorb: { enabled: true, toolName: "my_absorb", minToolTokens: 1 } }; + const rig = await startRig(baseCompress); + try { + // Manifest (built from the base config, as server.ts does) advertises the rename. + const m = readManifest(applyCompressSettings(defaultConfig(200_000), 200_000, baseCompress as Parameters[2])); + assert.ok(m.toolNames.includes("my_absorb"), "manifest advertises the base-renamed tool"); + + // A plugin-mode turn stamps the session with the base (renamed) name. + await post(rig, "abs-e2e-a", true, SIMPLE_MSGS()); + const sess = listSessions().find((s) => effectiveAbsorbConfig(s, defaultConfig(200_000))?.enabled === true); + assert.ok(sess, "a session armed absorb"); + assert.equal(effectiveAbsorbConfig(sess!, defaultConfig(200_000))?.toolName, "my_absorb", "plugin lane stamps the base (renamed) name"); + + // THE core repro: the advertised tool runs through the REAL plugin endpoint + // (pre-fix it 400'd because the gate expected the renamed name while the + // manifest advertised the static one). + const tr = await fetch(`http://127.0.0.1:${rig.proxyPort}/__bili/plugin/tool`, { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify({ conversationId: "abs-e2e-a", tool: "my_absorb", args: { ref: "m00001", summary: "x" } }), + }); + const tj = JSON.parse(await tr.text()) as { ok: boolean; error?: string }; + assert.equal(tr.status, 200, `advertised tool must not 400; got ${tr.status}: ${JSON.stringify(tj)}`); + assert.equal(tj.ok, true, "gate accepts the advertised (renamed) absorb tool"); + } finally { + await closeRig(rig); + } +}); + +test("e2e: route rename → plugin lane stays on base, proxy lane honors the override", async () => { + const baseCompress = { injectTool: true, injectNudge: false, absorb: { enabled: true, minToolTokens: 1 } }; + const routeCompress = { absorb: { enabled: true, toolName: "route_absorb", minToolTokens: 1 } }; + const rig = await startRig(baseCompress, routeCompress); + try { + // Plugin lane: the route override must NOT leak into the base-governed stamp. + const beforeP = new Set(listSessions().map((s) => s.id)); + await post(rig, "abs-e2e-b1", true, SIMPLE_MSGS()); + const pSess = listSessions().find((s) => !beforeP.has(s.id)); + assert.ok(pSess, "plugin turn created a session"); + assert.equal(effectiveAbsorbConfig(pSess!, defaultConfig(200_000))?.toolName, "absorb", "plugin lane follows the BASE name, ignoring the route override"); + + // Proxy lane: the merged (route) name governs, and the injected schema carries it. + const beforeX = new Set(listSessions().map((s) => s.id)); + await post(rig, "abs-e2e-b2", false, SIMPLE_MSGS()); + const xSess = listSessions().find((s) => !beforeX.has(s.id)); + assert.ok(xSess, "proxy turn created a session"); + assert.equal(effectiveAbsorbConfig(xSess!, defaultConfig(200_000))?.toolName, "route_absorb", "proxy lane honors the route override"); + const lastFwd = rig.forwards[rig.forwards.length - 1]!; + assert.ok(lastFwd.includes('"route_absorb"'), "proxy wire injects the renamed absorb tool"); + } finally { + await closeRig(rig); + } +}); diff --git a/tests/absorb.test.ts b/tests/absorb.test.ts new file mode 100644 index 0000000000..5ff7be1c0d --- /dev/null +++ b/tests/absorb.test.ts @@ -0,0 +1,273 @@ +import test from "node:test"; +import assert from "node:assert/strict"; +import { mkdtempSync } from "node:fs"; +import { tmpdir } from "node:os"; +import path from "node:path"; +import { + createCore, + createInitialState, + defaultConfig, + DEFAULT_ABSORB_CONFIG, + type AbsorbConfig, + type AbsorbRecord, + type Config, + type CoreMessage, +} from "acp-kernel"; +import { + applyAbsorbView, + absorbEnabled, + absorbToolName, + effectiveAbsorbConfig, + executeAbsorb, + isProxyToolFor, + storeEffectiveAbsorb, +} from "../src/absorb.ts"; +import { mergeCompress, applyCompressSettings } from "../src/compress-settings.ts"; +import { parseCompressSettings } from "../src/config.ts"; +import { handlePluginManifest } from "../src/plugin.ts"; +import { getSession, type Session } from "../src/session.ts"; +import { SessionStore } from "../src/persist.ts"; + +const BIG_TEXT = "line of build output ".repeat(700); // ~3.7K tokens + +function freshMsgs(): CoreMessage[] { + return [ + { id: "u1", role: "user", contentType: "text", text: "run a big build" }, + { id: "a-tc", role: "assistant", contentType: "tool-call", toolName: "bash", toolCallId: "call_1", text: JSON.stringify({ command: "npm test" }) }, + { id: "t-res", role: "tool", contentType: "tool-result", toolCallId: "call_1", text: BIG_TEXT }, + ]; +} + +function makeTurn() { + const core = createCore(); + const msgs = freshMsgs(); + // mediaOnly off: these host-plumbing tests exercise the credit/view + // mechanism over a TEXT result; the #3402 media-scope gate is covered separately. + const config: Config = { ...defaultConfig(200000), absorb: { ...DEFAULT_ABSORB_CONFIG, enabled: true, minToolTokens: 100, mediaOnly: false } }; + const turn = core.processTurn({ messages: msgs, state: createInitialState(), config, tokenCount: 0, renderTags: "text-only" }); + const byRef = Object.entries(turn.state.messageRefs.byRef ?? {}); + const refFor = (rawId: string): string => byRef.find(([, raw]) => raw === rawId)?.[0] ?? ""; + return { core, turn, config, refFor }; +} + +function makeSession(): Session { + const session = getSession(`t-absorb-${Math.random().toString(36).slice(2)}`); + session.stats.lastInputTokens = 100_000; + return session; +} + +test("absorbEnabled / absorbToolName respect config", () => { + assert.equal(absorbEnabled(defaultConfig(200000)), false); + assert.equal(absorbEnabled({ ...defaultConfig(200000), absorb: { ...DEFAULT_ABSORB_CONFIG, enabled: true } }), true); + assert.equal(absorbToolName({ ...defaultConfig(200000), absorb: { ...DEFAULT_ABSORB_CONFIG, enabled: true, toolName: "acp_absorb" } }), "acp_absorb"); + assert.equal(absorbToolName(defaultConfig(200000)), "absorb"); +}); + +test("isProxyToolFor: static ACP names always proxy; absorb only when effectively enabled", () => { + const base = defaultConfig(200000); + assert.equal(isProxyToolFor("compress", undefined, base), true); + assert.equal(isProxyToolFor("absorb", undefined, base), false); + assert.equal(isProxyToolFor("bash", undefined, base), false); + + const on: Config = { ...base, absorb: { ...DEFAULT_ABSORB_CONFIG, enabled: true } }; + assert.equal(isProxyToolFor("absorb", undefined, on), true); + + const session = getSession(`t-absorb-adjudicate-${Math.random().toString(36).slice(2)}`); + storeEffectiveAbsorb(session, on); + // Plugin tool API reads the per-session stored block even when the fallback + // (base kernel config) has the feature off. + assert.equal(isProxyToolFor("absorb", session, base), true); + assert.equal(isProxyToolFor("compress", session, base), true); + + storeEffectiveAbsorb(session, base); + assert.equal(isProxyToolFor("absorb", session, on), false); + + const renamed: Config = { ...base, absorb: { ...DEFAULT_ABSORB_CONFIG, enabled: true, toolName: "acp_absorb" } }; + assert.equal(isProxyToolFor("acp_absorb", undefined, renamed), true); + assert.equal(isProxyToolFor("absorb", undefined, renamed), false); +}); + +test("effectiveAbsorbConfig: session metadata wins over fallback", () => { + const on: AbsorbConfig = { ...DEFAULT_ABSORB_CONFIG, enabled: true, minToolTokens: 555 }; + const session = getSession(`t-absorb-meta-${Math.random().toString(36).slice(2)}`); + storeEffectiveAbsorb(session, { ...defaultConfig(200000), absorb: on }); + assert.deepEqual(effectiveAbsorbConfig(session, defaultConfig(200000)), on); + // defaultConfig() ships the kernel-default (disabled) block, so storing a + // plain base config records "absorb off" for this session and shadows a + // more permissive fallback. + storeEffectiveAbsorb(session, defaultConfig(200000)); + const eff = effectiveAbsorbConfig(session, { ...defaultConfig(200000), absorb: on }); + assert.equal(eff?.enabled, false); +}); + +test("executeAbsorb: success records absorption, credits net exactly once", () => { + const { turn, config, refFor } = makeTurn(); + const ref = refFor("t-res"); + assert.ok(ref, "tool result must have a ref"); + const session = makeSession(); + session.state = turn.state; + const before = session.stats.compressCreditTokens ?? 0; + const result = executeAbsorb({ ref, summary: "Build passed; 42 tests ok." }, "call_x", config.absorb!, { + config, + messages: turn.messages, + session, + }).text; + assert.ok(!result.includes("FAILED"), result); + assert.match(result, /^absorbed m\d+/); + assert.equal(session.state.absorbed?.length, 1); + const record = session.state.absorbed![0]; + assert.ok(record.tokensReclaimed > 0); + assert.equal(session.state.stats.absorbedTokens, record.tokensReclaimed); + assert.equal(session.stats.compressCreditTokens, before + record.tokensReclaimed); + assert.equal(session.stats.lastInputTokens, 100_000 - record.tokensReclaimed); + + // Re-absorbing the same ref is an ok=true no-op and must NOT credit again. + const again = executeAbsorb({ ref, summary: "retry" }, "call_y", config.absorb!, { config, messages: turn.messages, session }).text; + assert.ok(!again.includes("FAILED"), again); + assert.match(again, /already absorbed/); + assert.equal(session.state.absorbed?.length, 1); + assert.equal(session.stats.compressCreditTokens, before + record.tokensReclaimed); +}); + +test("executeAbsorb: bad ref and invalid input fail with FAILED marker", () => { + const { turn, config } = makeTurn(); + const session = makeSession(); + session.state = turn.state; + const badRef = executeAbsorb({ ref: "m99999", summary: "x" }, undefined, config.absorb!, { config, messages: turn.messages, session }).text; + assert.match(badRef, /\[absorb FAILED: absorb failed: ref m99999/); + const badInput = executeAbsorb({ summary: "no ref" }, undefined, config.absorb!, { config, messages: turn.messages, session }).text; + assert.match(badInput, /\[absorb FAILED: invalid input/); + assert.equal((session.state.absorbed ?? []).length, 0); +}); + +test("kernel gating: prompts honor minToolTokens/contextThresholdPct at processTurn time", () => { + // Gates are evaluated when processTurn renders the turn, so each scenario + // needs its own turn computed under that config (reusing one rendered view + // across configs would see markers already baked in). + const run = (absorb: AbsorbConfig, tokenCount = 100_000): boolean => { + const core = createCore(); + const cfg: Config = { ...defaultConfig(200000), absorb }; + const turn = core.processTurn({ messages: freshMsgs(), state: createInitialState(), config: cfg, tokenCount, renderTags: "text-only" }); + return turn.messages.some((m) => (m.text ?? "").includes("[ACP absorb]")); + }; + // mediaOnly pinned off: these scenarios isolate the token/context gates; + // the #3402 media-scope gate is covered separately in the kernel suite. + const A = { ...DEFAULT_ABSORB_CONFIG, mediaOnly: false }; + assert.equal(run({ ...A, enabled: true, minToolTokens: 100 }), true, "large eligible result is prompted"); + assert.equal(run({ ...A, enabled: true, minToolTokens: 100_000 }), false, "minToolTokens gate"); + assert.equal(run({ ...A, enabled: true, minToolTokens: 100, contextThresholdPct: 0.8 }), false, "below context threshold (50% usage)"); + assert.equal(run({ ...A, enabled: true, minToolTokens: 100, contextThresholdPct: 0.8 }, 180_000), true, "above context threshold (90% usage)"); + // excludeTools is a known no-op on tool RESULTS: the wire projections build + // result CoreMessages without toolName, so isAbsorbCandidate's name guard + // never fires (ranxianglei/acp-kernel#213). Asserting current behavior so a + // kernel fix flips this line red instead of silently changing semantics. + assert.equal(run({ ...A, enabled: true, minToolTokens: 100, excludeTools: ["bash"] }), true, "excluded tool name (kernel limitation)"); + assert.equal(run({ ...A, enabled: false, minToolTokens: 100 }), false, "feature disabled"); +}); + +test("applyAbsorbView: hides recorded pairs even after the feature is disabled", () => { + const { turn, config, refFor } = makeTurn(); + const session = makeSession(); + session.state = turn.state; + const ref = refFor("t-res"); + const result = executeAbsorb({ ref, summary: "short" }, undefined, config.absorb!, { config, messages: turn.messages, session }).text; + assert.ok(!result.includes("FAILED"), result); + const hidden = applyAbsorbView(turn.messages, session.state, defaultConfig(200000), 100_000); + assert.deepEqual(hidden.map((m) => m.id), ["u1"], "both halves of the absorbed pair are hidden"); + assert.ok(!hidden.some((m) => (m.text ?? "").includes("[ACP absorb]"))); +}); + +test("mergeCompress: absorb sub-fields merge deepest-wins like other nested fields", () => { + const merged = mergeCompress( + { absorb: { enabled: true, minToolTokens: 500 } }, + { absorb: { contextThresholdPct: "75%" } }, + { absorb: { excludeTools: ["bash"] } }, + ); + assert.deepEqual(merged.absorb, { enabled: true, minToolTokens: 500, contextThresholdPct: "75%", excludeTools: ["bash"] }); + const partial = mergeCompress({ absorb: { enabled: true } }, undefined, { absorb: { toolName: "x" } }); + assert.deepEqual(partial.absorb, { enabled: true, toolName: "x" }); + assert.equal(mergeCompress(undefined, undefined, undefined).absorb, undefined); +}); + +test("applyCompressSettings: maps settings absorb onto kernel AbsorbConfig with defaults + percent parse", () => { + const base = defaultConfig(200000); + const out = applyCompressSettings(base, 200_000, { absorb: { enabled: true, contextThresholdPct: "80%" } }); + assert.deepEqual(out.absorb, { + enabled: true, + toolName: "absorb", + minToolTokens: 4000, + contextThresholdPct: 0.8, + excludeTools: [], + mediaOnly: true, + }); + const numeric = applyCompressSettings(base, 200_000, { absorb: { enabled: false, minToolTokens: 250, contextThresholdPct: 0.5 } }); + assert.equal(numeric.absorb?.enabled, false); + assert.equal(numeric.absorb?.minToolTokens, 250); + assert.equal(numeric.absorb?.contextThresholdPct, 0.5); + const absent = applyCompressSettings(base, 200_000, {}); + assert.deepEqual(absent.absorb, base.absorb, "absent settings leave the base absorb block untouched"); +}); + +test("parseCompressSettings: validates absorb sub-object field-by-field, rejects whole block on any bad field", () => { + const good = parseCompressSettings({ absorb: { enabled: true, minToolTokens: 100, contextThresholdPct: "75%", excludeTools: ["bash"], toolName: "abs" } }); + assert.deepEqual(good?.absorb, { enabled: true, minToolTokens: 100, contextThresholdPct: "75%", excludeTools: ["bash"], toolName: "abs" }); + assert.equal(parseCompressSettings({ absorb: { enabled: "yes" } }), undefined); + assert.equal(parseCompressSettings({ absorb: { minToolTokens: "big" } }), undefined); + assert.equal(parseCompressSettings({ absorb: { contextThresholdPct: "abc" } }), undefined); + assert.equal(parseCompressSettings({ absorb: { excludeTools: "bash" } }), undefined); + assert.equal(parseCompressSettings({ absorb: { toolName: "" } }), undefined); + assert.equal(parseCompressSettings({ absorb: { mediaOnly: false } })?.absorb?.mediaOnly, false); + assert.equal(parseCompressSettings({ absorb: { mediaOnly: "yes" } }), undefined); + assert.equal(parseCompressSettings({ absorb: [] }), undefined); + // A bad absorb field rejects sibling fields too (whole-block convention). + assert.equal(parseCompressSettings({ tiers: false, absorb: { enabled: 1 } }), undefined); +}); + +// #1192: hosts register manifest tools verbatim, so a disabled absorb must not +// be advertised at all — only an absorb-enabled config may list it. +test("handlePluginManifest: absorb advertised on all three wires only when enabled", () => { + let body = ""; + const res = { writeHead: () => {}, end: (b: string) => { body = b; } } as unknown as Parameters[0]; + const data = (): { + toolNames: string[]; + tools: { anthropic: { name: string; input_schema?: unknown }[]; openai: { name?: string; function?: { name: string } }[]; responses: { name?: string; type?: string }[] }; + } => JSON.parse(body); + + handlePluginManifest(res, defaultConfig(200_000)); + let d = data(); + assert.ok(!d.toolNames.includes("absorb"), "disabled by default → not advertised"); + assert.ok(!d.tools.anthropic.some((t) => t.name === "absorb")); + assert.ok(!d.tools.openai.some((t) => t.function?.name === "absorb")); + assert.ok(!d.tools.responses.some((t) => t.name === "absorb")); + + handlePluginManifest(res, { ...defaultConfig(200_000), absorb: { ...DEFAULT_ABSORB_CONFIG, enabled: true } }); + d = data(); + assert.ok(d.toolNames.includes("absorb")); + const anthro = d.tools.anthropic.find((t) => t.name === "absorb"); + assert.ok(anthro, "anthropic schema present"); + const schema = anthro!.input_schema as { required?: string[] }; + assert.deepEqual(schema.required, ["ref", "summary"]); + assert.ok(d.tools.openai.some((t) => t.function?.name === "absorb")); + assert.ok(d.tools.responses.some((t) => t.name === "absorb")); +}); + +test("persist round-trip: absorbed records and absorbedTokens survive save/load", () => { + const dir = mkdtempSync(path.join(tmpdir(), "bili-absorb-")); + const store = new SessionStore({ dir, debounceMs: 0 }); + const session = getSession(`t-absorb-persist-${Math.random().toString(36).slice(2)}`, { protocol: "anthropic" }); + const record: AbsorbRecord = { + toolCallId: "call_1", + callMessageId: "a-tc", + resultMessageId: "t-res", + summary: "Build passed.", + tokensReclaimed: 4321, + createdAt: Date.now(), + }; + session.state.absorbed = [record]; + session.state.stats.absorbedTokens = 4321; + assert.equal(store.flushSync(session), true); + const loaded = store.loadSync(session.id, { protocol: "anthropic" }); + assert.ok(loaded, "session must reload"); + assert.deepEqual(loaded!.state.absorbed, [record], "absorbed records must survive restart"); + assert.equal(loaded!.state.stats.absorbedTokens, 4321); +}); diff --git a/tests/acp-status-storage-degraded.test.ts b/tests/acp-status-storage-degraded.test.ts new file mode 100644 index 0000000000..eda94fd3b5 --- /dev/null +++ b/tests/acp-status-storage-degraded.test.ts @@ -0,0 +1,113 @@ +import { test } from "node:test"; +import assert from "node:assert/strict"; +import { mkdtempSync } from "node:fs"; +import { tmpdir } from "node:os"; +import * as path from "node:path"; +import { handleAcpStatus } from "../src/acp-status.ts"; +import { SessionStore, _setStoreForTest } from "../src/persist.ts"; +import { setSqliteEnginesUnavailableForTests } from "../src/storage/driver.ts"; +import { _resetStorageRuntimeStatusForTest, recordEngineBootstrap } from "../src/storage/runtime-status.ts"; +import { createCore, createInitialState, defaultConfig, type CompressionCore, type Config, type CoreMessage } from "acp-kernel"; +import type { Session } from "../src/session.ts"; +import { rmrf } from "./tmp-rm.ts"; + +/** + * #3385 (owner directive 3, acp_status surface): the STORAGE section of the + * acp_status tool must surface the runtime degrade reason verbatim — on + * Node < 22.5 with no engine it carries the explicit "upgrade Node.js" + * lever (see nodeSqliteUpgradeHint in src/storage/engine-bootstrap.ts), so + * the MODEL can relay the fix to the user instead of the fact living only + * in lane logs the user never opens. Healthy storage stays quiet and an + * intentional legacy opt-out is never nagged. + * + * The store is built for real (engines forced unavailable) so the recording + * seams fire exactly as they do in production; _setStoreForTest then keeps + * handleAcpStatus's ccrLoopConfig from rebuilding a healthy store mid-call + * and clobbering the degraded state (that clobber is also why the store is + * constructed BEFORE handleAcpStatus here). + */ + +function makeCtx(): { core: CompressionCore; config: Config; messages: CoreMessage[]; session: Session } { + const session = { id: "acp-status-3385", meta: {}, metadata: {}, stats: { requests: 0 }, state: createInitialState() } as unknown as Session; + return { core: createCore(), config: defaultConfig(200000), messages: [], session }; +} + +const ENV_KEYS = ["BILI_SESSIONS_DIR", "BILI_CONFIG_FILE", "XDG_CACHE_HOME", "XDG_STATE_HOME", "BILI_STORAGE_UNIFIED", "BILI_ENCRYPTION_KEY"] as const; + +function isolate(base: string): Record { + const snap: Record = {}; + for (const k of ENV_KEYS) snap[k] = process.env[k]; + const sessions = path.join(base, "sessions"); + mkdtempSync(sessions); // replace mkdirSync recursive — sessions parent exists + process.env.BILI_SESSIONS_DIR = sessions; + process.env.BILI_CONFIG_FILE = path.join(base, "absent-config.json"); + process.env.XDG_CACHE_HOME = path.join(base, "cache"); + process.env.XDG_STATE_HOME = path.join(base, "state"); + delete process.env.BILI_STORAGE_UNIFIED; + delete process.env.BILI_ENCRYPTION_KEY; + return snap; +} + +function restore(snap: Record): void { + for (const k of ENV_KEYS) { + if (snap[k] === undefined) delete process.env[k]; + else process.env[k] = snap[k]; + } +} + +test("#3385: acp_status STORAGE section surfaces the degrade reason so the model can relay the lever", () => { + const base = mkdtempSync(path.join(tmpdir(), "acp-status-3385-")); + const snap = isolate(base); + let store: SessionStore | null = null; + try { + setSqliteEnginesUnavailableForTests(true); + _resetStorageRuntimeStatusForTest(); + recordEngineBootstrap({ status: "failed", detail: "The operation was aborted due to timeout" }); + store = new SessionStore({ dir: process.env.BILI_SESSIONS_DIR, enabled: true }); + _setStoreForTest(store); + + const out = handleAcpStatus({}, makeCtx()); + const text = out.text ?? ""; + assert.ok(text.includes("STORAGE —"), "the storage line is present"); + assert.ok(text.includes("⚠ degraded —"), `the degrade warning line is present (got: ${text.slice(-400)})`); + assert.ok(text.includes("no SQLite engine available"), "the reason names the missing engine"); + assert.ok(text.includes("(relay this to the user)"), "the model is told to relay it"); + // The Node >= 22.5 lever is part of the reason string ONLY on Nodes + // that lack the builtin (nodeSqliteUpgradeHint is empty on >= 22.5 — + // pinned separately in tests/storage-engine-bootstrap.test.ts). + if (Number(process.versions.node.split(".")[0]) < 22 || (Number(process.versions.node.split(".")[0]) === 22 && Number(process.versions.node.split(".")[1]) < 5)) { + assert.ok(/Node\.js >= 22\.5/.test(text), "this Node lacks the builtin — the reason must carry the upgrade lever"); + } + } finally { + store?.closeQuietly(); + _setStoreForTest(null as unknown as SessionStore); + setSqliteEnginesUnavailableForTests(false); + _resetStorageRuntimeStatusForTest(); + restore(snap); + rmrf(base); + } +}); + +test("#3385: healthy storage stays quiet — no degrade line", () => { + // No engine forcing: on any runner the builtin/host/cached ladder serves + // a real engine, the sidecar opens, and the STORAGE line must NOT be + // followed by a degrade warning. + const base = mkdtempSync(path.join(tmpdir(), "acp-status-3385-ok-")); + const snap = isolate(base); + let store: SessionStore | null = null; + try { + _resetStorageRuntimeStatusForTest(); + store = new SessionStore({ dir: process.env.BILI_SESSIONS_DIR, enabled: true }); + _setStoreForTest(store); + const out = handleAcpStatus({}, makeCtx()); + const text = out.text ?? ""; + assert.ok(text.includes("STORAGE —"), "the storage line is present"); + assert.ok(!text.includes("⚠ degraded"), `healthy storage shows no degrade warning (got: ${text.slice(-400)})`); + } finally { + store?.closeQuietly(); + _setStoreForTest(null as unknown as SessionStore); + _resetStorageRuntimeStatusForTest(); + restore(snap); + rmrf(base); + } +}); diff --git a/tests/auto-restart.test.ts b/tests/auto-restart.test.ts index bfb2c34ac7..d40fabf149 100644 --- a/tests/auto-restart.test.ts +++ b/tests/auto-restart.test.ts @@ -674,7 +674,7 @@ test("performSelfRestart: port taken over by another bili instance → parent ex } }); -test("performSelfRestart: a replacement that survives SIGTERM is escalated loudly (#3044)", { timeout: 30_000, skip: process.platform === "win32" ? "POSIX-only: Windows terminates on SIGTERM uncatchably, so a SIGTERM-surviving stub cannot be modeled (was a w24 CI red, #3182 family)" : false }, async () => { +test("performSelfRestart: a replacement that ignores SIGTERM is reaped by the SIGKILL escalation (#3044/#3344)", { timeout: 30_000, skip: process.platform === "win32" ? "POSIX-only: Windows terminates on SIGTERM uncatchably, so a SIGTERM-surviving stub cannot be modeled (was a w24 CI red, #3182 family)" : false }, async () => { // #3182: same stolen-port retry as above (stub exit 91 = infra TOCTOU). const stubborn = path.join(root, "stub-stubborn.mjs"); for (let attempt = 0; ; attempt++) { @@ -683,8 +683,9 @@ test("performSelfRestart: a replacement that survives SIGTERM is escalated loudl const record: SpawnRecord = {}; writeFileSync(stubborn, [ // No listener by design: the stub models a replacement that never - // reaches readiness. It only ignores SIGTERM to force the loud - // escalation path (#3044). + // reaches readiness. It ignores SIGTERM to force the escalation path; + // #3344 escalates to SIGKILL, which it cannot ignore — so the orphan + // is reaped instead of left for manual cleanup (#3044). "process.on('SIGTERM', () => {});", "setInterval(() => {}, 1000);", ].join("\n")); @@ -700,9 +701,13 @@ test("performSelfRestart: a replacement that survives SIGTERM is escalated loudl }); assert.equal(result.ok, false, JSON.stringify(result)); assert.equal(result.error, "replacement-not-ready"); - // An unverified kill was how the orphans of #3044 survived their - // parent's timeout — the loud escalation is the point of this test. - assert.match(entries.join("\n"), /still alive after SIGTERM — manual cleanup required \(kill \d+\)/s); + // #3344: the SIGTERM-ignoring replacement is reaped by the SIGKILL + // escalation — no orphan survives its parent's timeout, so no + // manual-cleanup error is logged. (Pre-#3344 this stub stayed alive + // and the assertion matched the loud manual-cleanup message.) + assert.ok(record.child?.pid, "stub pid for the liveness check"); + assert.equal(isPidAlive(record.child!.pid), false, "the SIGTERM-stubborn replacement was reaped by SIGKILL"); + assert.doesNotMatch(entries.join("\n"), /manual cleanup required/s); assert.equal(await waitForListening(server), true, "service resumes regardless"); break; } catch (e) { diff --git a/tests/cache-proof.test.ts b/tests/cache-proof.test.ts index c69dd44f97..b2b74343f6 100644 --- a/tests/cache-proof.test.ts +++ b/tests/cache-proof.test.ts @@ -1015,6 +1015,12 @@ function runProof(wire: Wire, bodies: string[], urls: string[], ev: ProofEvents) } +// #3400: pin legacy wire economics - the default-on read_original_image tool +// adds ~200 tokens per request, shifting this scenario's nudge timing so the +// fold/rearm arithmetic moves by design. Fidelity here is compaction behavior, +// not egress policy (tests/issue3400-image-rescale.test.ts covers that). +process.env.BILI_IMAGE_RESCALE = "0"; + test("cache proof (openai chat wire): byte-stable growth, failure paths, param drift, folds", { timeout: 120_000 }, async () => { const { bodies, urls, ev } = await driveProof("chat", "proof-chat", 200_000); assert.ok(ev.retry.length === 1, "transport-retry scenario must have produced a resend pair"); diff --git a/tests/cache-seam-fuzz.test.ts b/tests/cache-seam-fuzz.test.ts index 849ebe16c7..25058fc6f1 100644 --- a/tests/cache-seam-fuzz.test.ts +++ b/tests/cache-seam-fuzz.test.ts @@ -41,6 +41,7 @@ interface Scenario { thresholdBytes: number; plan: Array; gapless: boolean; + absorbPct: number; ccr: boolean; toolsOnClient: boolean; } @@ -69,6 +70,7 @@ function genScenario(rnd: () => number, seed: number): Scenario { thresholdBytes: (34 + Math.floor(rnd() * 40)) * 1024, plan, gapless: rnd() < 0.5, + absorbPct: rnd() < 0.5 ? 0.05 : 0, ccr: rnd() < 0.5, toolsOnClient: rnd() < 0.6, }; @@ -179,6 +181,7 @@ async function drive(sc: Scenario, sessionId: string): Promise { compress: { injectTool: true, injectNudge: true, + absorb: { enabled: true, contextThresholdPct: sc.absorbPct }, ...(sc.ccr ? { ccr: { enabled: true, minToolTokens: 50 } } : {}), }, promptCache: { routing: "auto" }, diff --git a/tests/cache-seam-probes.test.ts b/tests/cache-seam-probes.test.ts index 6e9f902445..446fe23c43 100644 --- a/tests/cache-seam-probes.test.ts +++ b/tests/cache-seam-probes.test.ts @@ -7,6 +7,11 @@ // P1 tools array — the tools list is hashed into the provider prefix; any // per-request tool-list mutation breaks the cache at // element 0. (Also pinned matrix-wide via normOf.) +// P2 absorb view — with absorb.contextThresholdPct > 0 the absorb-prompt +// gate is token-count-fed; round-2 must use the SAME +// source as the steady paths or prompts flicker +// mid-history across every fold while the session +// straddles the threshold (#1592 family). // P3 CCR armed — with compress.ccr.enabled the oversized tool result // rides the wire as a placeholder; the placeholder element // must stay byte-identical across folds (round-2 and the @@ -29,6 +34,7 @@ type Item = Record; type Msg = Record; const THRESHOLD = 60 * 1024; +const ABSORB_PROMPT_FRAGMENT = "[ACP absorb]"; const CCR_PLACEHOLDER_FRAGMENT = "[acp-stored"; const FILLER = (seed: number, kb: number): string => { @@ -108,6 +114,7 @@ function startUpstream(captured: string[], plan: Array): htt } interface RigOpts { + absorbThresholdPct?: number; ccr?: boolean; turns?: number; bigToolFrom?: number; @@ -136,6 +143,7 @@ async function driveChat(sessionId: string, opts: RigOpts): Promise { compress: { injectTool: true, injectNudge: true, + absorb: { enabled: true, contextThresholdPct: opts.absorbThresholdPct ?? 0 }, ...(opts.ccr === true ? { ccr: { enabled: true, minToolTokens: 50 } } : {}), }, promptCache: { routing: "auto" }, @@ -228,12 +236,12 @@ function assertPairSeamStable(prev: string, cur: string, label: string): void { const emergencyTrunc = JSON.stringify(mb[k] ?? "").includes("[truncated for context space]"); assert.ok( k >= solidA || summaryNew || emergencyTrunc, - `${label}: mid-history divergence at message[${k}] of ${ma.length} (round2=${isRound2(b)}) — CCR view flipped between adjacent requests`, + `${label}: mid-history divergence at message[${k}] of ${ma.length} (round2=${isRound2(b)}) — absorb/CCR view flipped between adjacent requests`, ); } -test("probe P1: tools array byte-stable across folds (estimate-grade session)", { timeout: 180_000 }, async () => { - const bodies = await driveChat("seam-p2", {}); +test("probe P1+P2: tools stable and absorb prompts do not flicker across folds (estimate-grade session, threshold 5%)", { timeout: 180_000 }, async () => { + const bodies = await driveChat("seam-p2", { absorbThresholdPct: 0.05 }); assert.ok(bodies.length >= 10, `expected >= 10 requests, got ${bodies.length}`); let folds = 0; for (let i = 1; i < bodies.length; i++) { diff --git a/tests/ccr-v2.test.ts b/tests/ccr-v2.test.ts index 37bab7da07..3785542780 100644 --- a/tests/ccr-v2.test.ts +++ b/tests/ccr-v2.test.ts @@ -64,7 +64,7 @@ function fold(opts: { ccr: boolean; preStore?: MessageContentStore }): FoldResul const config = applyCompressSettings( defaultConfig(200_000), 200_000, - { ccr: opts.ccr ? { enabled: true, minToolTokens: 50 } : { enabled: false, minToolTokens: 50 } }, + { absorb: { enabled: true, minToolTokens: 50 }, ccr: opts.ccr ? { enabled: true, minToolTokens: 50 } : { enabled: false, minToolTokens: 50 } }, ) as Config; const session = getSession(`ccrv2-${Math.random().toString(36).slice(2)}`); if (opts.ccr) storeEffectiveCcr(session, { enabled: true, minToolTokens: 50 }); diff --git a/tests/codex-official.test.ts b/tests/codex-official.test.ts index eff093cc67..86542b618a 100644 --- a/tests/codex-official.test.ts +++ b/tests/codex-official.test.ts @@ -130,7 +130,7 @@ test("Codex official transport preserves OAuth headers, decodes bodies, and reba // the injected tool set. assert.deepEqual( forwarded.tools.map((t: unknown) => (t as { name: string }).name), - ["shell", "compress", "decompress", "search_context", "acp_status", "acp_cache", "acp_rule", "acp_retrieve"], + ["shell", "compress", "decompress", "search_context", "acp_status", "acp_cache", "acp_rule", "acp_retrieve", "read_original_image"], ); const session = listSessions().find((candidate) => candidate.meta.label === sessionId); diff --git a/tests/compress-recovery-ranges.test.ts b/tests/compress-recovery-ranges.test.ts index 6ae147b928..f934138b83 100644 --- a/tests/compress-recovery-ranges.test.ts +++ b/tests/compress-recovery-ranges.test.ts @@ -183,11 +183,24 @@ test("#3192: a dropped message's DEAD ref is never re-advertised by the receipt const rejected = request(ctx, "m00001", "m00005"); assert.equal(rejected.outcome, "refused"); assert.match(rejected.text, /cannot be anchored/); - // No advertised compressible span may cross the dead ref m00005. - assert.doesNotMatch(rejected.text, /m0000[1-4]\s*[–-]\s*m0000[5-9]/); - // The only live span bracketed the dead ref -> stop instruction, not a doomed retry span. - assert.match(rejected.text, /continue the task without compressing/); - assert.doesNotMatch(rejected.text, /use these refs exactly as listed/); + // No advertised compressible span may cross the dead ref m00005 — the red gate + // on the unfixed kernel (it advertised one span m00001–m00007 crossing m00005). + const advertised = advertisedRanges(rejected.text); + assert.ok(advertised.length > 0, `receipt must advertise the surviving sub-spans: ${rejected.text}`); + for (const [start, end] of advertised) { + assert.ok(!spanContains(start, end, 5), `${start}..${end} must not span the dead ref m00005`); + } + // The kernel SPLIT around the hole into adjacent live sub-spans (not a whole-span + // drop): the run before the hole ends at/before m00004, the run after starts at/ + // after m00006. Both present proves the split rather than a silent drop. + assert.ok( + advertised.some(([start, end]) => Number(start.slice(1)) <= 4 && Number(end.slice(1)) <= 4), + `pre-hole sub-span must be advertised: ${JSON.stringify(advertised)}`, + ); + assert.ok( + advertised.some(([start, end]) => Number(start.slice(1)) >= 6 && Number(end.slice(1)) >= 6), + `post-hole sub-span must be advertised: ${JSON.stringify(advertised)}`, + ); // Receipt and acp_status share ONE eligibility source (liveCompressionRanges); // pin that source so the two surfaces cannot drift apart again (#2778 promise). diff --git a/tests/derived-fork-block-inheritance.test.ts b/tests/derived-fork-block-inheritance.test.ts index 21583aa67a..0bcdaa0bf0 100644 --- a/tests/derived-fork-block-inheritance.test.ts +++ b/tests/derived-fork-block-inheritance.test.ts @@ -8,7 +8,7 @@ import path from "node:path"; import { defaultConfig } from "acp-kernel"; import { startServer } from "../src/server.ts"; import { SessionStore, _setStoreForTest } from "../src/persist.ts"; -import { maybeAdoptResume } from "../src/fork-adoption.ts"; +import { maybeAdoptResume, maybeAdoptDerivedFork, incomingCoreMessages } from "../src/fork-adoption.ts"; import type { Session } from "../src/session.ts"; import { listSessions } from "../src/session.ts"; import type { ProxyOptions } from "../src/config.ts"; @@ -364,3 +364,92 @@ test("derived fork inherits parent refs and blocks via message-id intersection", // Parent session must stay untouched (copy-on-fork). assert.ok(r.parentUntouched, "parent session must stay untouched"); }); + +/** + * #3351: the weak-link gate must mirror the boot sweep's discipline — close + * ONLY when blocks were actually adopted. Pre-fix, maybeAdoptDerivedFork set + * derivedBlocksAdopted=true unconditionally right after maybeAdoptResume, so a + * multi-hop grandchild whose registered parent was still young (zero folds) at + * first contact permanently lost eligibility: the link lands exactly once + * (handle.ts gates it on derivedFromSessionId === undefined) and the + * prematurely-persisted marker made every later boot sweep skip it. These pin + * the gate discipline directly at the changed seam; the restart convergence is + * covered by the lineage-reconcile-boot suite (AC9/AC12). + */ + +function sharedWireBody(): { parsed: unknown; ids: string[] } { + const parsed = { + messages: [ + { role: "user", content: "shared turn one" }, + { role: "assistant", content: "shared reply one" }, + ], + }; + const msgs = incomingCoreMessages("openai", parsed); + assert.ok(msgs && msgs.length === 2, "wire body must yield two core messages"); + return { parsed, ids: msgs!.map((m) => m.id) }; +} + +function blankSession(id: string): Session { + return { + id, + metadata: {}, + // A real session always carries a content store; supply an empty one so + // contentStoreOf short-circuits instead of probing the (disabled) store. + contentStore: { version: 1, byHash: {}, byRef: {} }, + state: { blocks: [], messageRefs: { byRaw: {}, byRef: {} }, tokenSnapshot: {}, nextBlockId: 1, nextRunId: 1 }, + blockContents: new Map(), + } as unknown as Session; +} + +function seedParentRefs(parent: Session, ids: string[]): void { + parent.state.messageRefs.byRaw = { [ids[0]!]: "m00001", [ids[1]!]: "m00002" }; + parent.state.messageRefs.byRef = { m00001: ids[0]!, m00002: ids[1]! }; +} + +test("#3351: weak-link gate stays OPEN when the registered parent has no blocks at first contact", () => { + _setStoreForTest(new SessionStore({ enabled: false })); + const { parsed, ids } = sharedWireBody(); + const [idA] = [ids[0]!, ids[1]!]; + assert.ok(ids[0] && ids[1] && ids[0] !== ids[1], "core message ids must be distinct"); + + const parent = blankSession("P"); + seedParentRefs(parent, ids); + assert.equal(parent.state.blocks.length, 0, "precondition: parent is young (no folds yet)"); + + const child = blankSession("G"); + child.metadata.derivedFromSessionId = "P"; + maybeAdoptDerivedFork({ session: child, parent, protocol: "openai", parsed, upstreamOrigin: "http://127.0.0.1", enabled: true, log: () => {} }); + + assert.ok(idA in child.state.messageRefs.byRaw, "refs-only inheritance still happens against a young parent"); + assert.equal(child.state.blocks.length, 0, "no block adopted from a block-less parent"); + assert.notEqual(child.metadata.derivedBlocksAdopted, true, "#3351: gate must stay OPEN so a later boot can adopt once the parent folds"); +}); + +test("#3351: weak-link gate CLOSES when the registered parent has fully-present blocks", () => { + _setStoreForTest(new SessionStore({ enabled: false })); + const { parsed, ids } = sharedWireBody(); + + const parent = blankSession("P"); + seedParentRefs(parent, ids); + parent.state.blocks.push({ + blockId: "b1", + runId: 1, + tier: 1, + active: true, + summary: "summary over the shared prefix", + directMessageIds: [ids[0]!], + effectiveMessageIds: [...ids], + directBlockIds: [], + createdAt: 0, + compressedTokens: 900, + } as unknown as (typeof parent.state.blocks)[number]); + parent.blockContents.set("b1", { full: { text: "summary over the shared prefix", count: 1 }, one: null } as never); + + const child = blankSession("G"); + child.metadata.derivedFromSessionId = "P"; + maybeAdoptDerivedFork({ session: child, parent, protocol: "openai", parsed, upstreamOrigin: "http://127.0.0.1", enabled: true, log: () => {} }); + + assert.equal(child.state.blocks.length, 1, "block adopted from a folded parent"); + assert.equal(child.metadata.derivedBlocksAdopted, true, "gate closes on real block adoption"); + assert.ok(child.blockContents.has("b1"), "adopted block carries its stored content"); +}); diff --git a/tests/dsh-admin-relay.test.ts b/tests/dsh-admin-relay.test.ts new file mode 100644 index 0000000000..367b26bc73 --- /dev/null +++ b/tests/dsh-admin-relay.test.ts @@ -0,0 +1,287 @@ +import test from "node:test"; +import assert from "node:assert/strict"; +import fs from "node:fs"; +import os from "node:os"; +import path from "node:path"; +import http from "node:http"; +import { apply, _resetRegisterForTest } from "../src/agent/dsh-native.ts"; +import { rmrf } from "./tmp-rm.ts"; + +// The node:test runner sets NODE_TEST_CONTEXT itself (see tests/e2e/README.md); +// set it defensively so a direct single-file run also stands down the spawn +// bootstrap and the global fetch intercept (the relay handler must exercise +// the REAL fetch — that is the hop the admin gate sees). +process.env.NODE_TEST_CONTEXT = process.env.NODE_TEST_CONTEXT ?? "1"; + +async function withEnv(env: Record, fn: () => Promise | T): Promise { + const prev: Record = {}; + for (const [key, value] of Object.entries(env)) { + prev[key] = process.env[key]; + if (value === undefined) delete process.env[key]; + else process.env[key] = value; + } + try { + return await fn(); + } finally { + for (const [key, value] of Object.entries(prev)) { + if (value === undefined) delete process.env[key]; + else process.env[key] = value; + } + } +} + +type RouteRow = { kind: string; path: string; handler: (req: unknown, res: RelayRes) => void | Promise }; + +class RelayRes { + status?: number; + headers?: Record; + chunks: string[] = []; + writeHead(status: number, headers?: Record): void { + this.status = status; + this.headers = headers; + } + write(chunk: string | Uint8Array): boolean { + this.chunks.push(typeof chunk === "string" ? chunk : Buffer.from(chunk).toString("utf8")); + return true; + } + end(body?: string | Uint8Array): void { + if (body !== undefined) this.chunks.push(typeof body === "string" ? body : Buffer.from(body).toString("utf8")); + } + get body(): string { + return this.chunks.join(""); + } +} + +function fakeReq(method: string, url: string, headers: Record, bodyChunks: string[] = []): unknown { + return { + method, + url, + headers, + async *[Symbol.asyncIterator]() { + for (const chunk of bodyChunks) yield chunk; + }, + }; +} + +/** A browser admin call as it arrives on the dsh entry: the browser's Origin + * and cookies describe the ENTRY, not the proxy — the relay must drop both + * so the forwarded hop reads as a local non-browser client (#1538 gate). */ +const browserHeaders = (): Record => ({ + "content-type": "application/json", + host: "dsh.example:443", + origin: "https://dsh.example", + cookie: "dsh-session=secret", + referer: "https://dsh.example/settings", + "user-agent": "relay-test", +}); + +type Seen = { method: string; url: string; headers: http.IncomingHttpHeaders; body: string }; + +function startAdminStub(): Promise<{ origin: string; seen: Seen[]; close: () => void }> { + const seen: Seen[] = []; + const server = http.createServer((req, res) => { + let body = ""; + req.on("data", (c: Buffer) => { + body += c.toString("utf8"); + }); + req.on("end", () => { + seen.push({ method: req.method ?? "", url: req.url ?? "", headers: req.headers, body }); + if ((req.url ?? "").startsWith("/__bili/config")) { + res.writeHead(200, { "content-type": "application/json", "x-bili-relay": "yes" }); + res.end(JSON.stringify({ ok: true, echoed: body })); + return; + } + res.writeHead(404, { "content-type": "application/json" }); + res.end("{}"); + }); + }); + return new Promise((resolve) => { + server.listen(0, "127.0.0.1", () => { + const port = (server.address() as { port: number }).port; + resolve({ origin: `http://127.0.0.1:${port}`, seen, close: () => server.close() }); + }); + }); +} + +function collectRoutes(): RouteRow[] { + const routes: RouteRow[] = []; + const ctx = { + tools: { register: (_t: unknown) => {} }, + commands: { register: (_c: unknown) => {} }, + agents: { currentInitiator: () => ({ session: { id: "s-3373" } }) }, + inject: (_deps: readonly string[], cb: (sub: unknown) => void) => { + cb({ webServer: { register: (r: RouteRow) => routes.push(r) } }); + }, + on: (_event: string, _listener: unknown) => {}, + }; + apply(ctx as Parameters[0]); + return routes; +} + +function relayRoute(routes: RouteRow[]): RouteRow { + const route = routes.find((r) => r.path === "/__bili"); + assert.ok(route !== undefined, "the /__bili prefix route must be registered"); + assert.equal(route.kind, "prefix"); + return route; +} + +function writeConfig(home: string, body: string): string { + const file = path.join(home, "bili-config.json"); + fs.writeFileSync(file, body); + return file; +} + +test("#3373: the admin relay stays fail-closed while security.adminRelay is off — even with a bound proxy and the route mounted", async () => { + const home = fs.mkdtempSync(path.join(os.tmpdir(), "bili-relay-")); + try { + // register.base points at a port with nothing listening: if the relay + // ever left the fail-closed branch it would 502, not 404. + await withEnv({ DSH_HOME: home, BILLION_CONTEXT_PROXY: undefined, BILI_CONFIG_FILE: writeConfig(home, "{}") }, async () => { + _resetRegisterForTest("http://127.0.0.1:1"); + const route = relayRoute(collectRoutes()); + const res = new RelayRes(); + await route.handler(fakeReq("GET", "/__bili/config", browserHeaders()), res); + assert.equal(res.status, 404); + assert.deepEqual(JSON.parse(res.body), { error: "admin relay disabled" }); + }); + } finally { + rmrf(home); + } +}); + +test("#3373: with the gate on but no proxy bound, the relay answers 503 instead of pretending", async () => { + const home = fs.mkdtempSync(path.join(os.tmpdir(), "bili-relay-")); + try { + await withEnv({ DSH_HOME: home, BILLION_CONTEXT_PROXY: undefined, BILI_CONFIG_FILE: writeConfig(home, JSON.stringify({ security: { adminRelay: true } })) }, async () => { + _resetRegisterForTest(undefined); + const route = relayRoute(collectRoutes()); + const res = new RelayRes(); + await route.handler(fakeReq("GET", "/__bili/config", browserHeaders()), res); + assert.equal(res.status, 503); + assert.deepEqual(JSON.parse(res.body), { error: "bili proxy not running" }); + }); + } finally { + rmrf(home); + } +}); + +test("#3373: gate on + proxy bound — the relay forwards to loopback as a local non-browser client (no Origin/Cookie/Referer, fetch-derived loopback Host), passes method/url/body/status/headers through, and streams the response", async () => { + const stub = await startAdminStub(); + const home = fs.mkdtempSync(path.join(os.tmpdir(), "bili-relay-")); + try { + await withEnv({ DSH_HOME: home, BILLION_CONTEXT_PROXY: undefined, BILI_CONFIG_FILE: writeConfig(home, JSON.stringify({ security: { adminRelay: true } })) }, async () => { + _resetRegisterForTest(stub.origin); + const route = relayRoute(collectRoutes()); + + const get = new RelayRes(); + await route.handler(fakeReq("GET", "/__bili/config", browserHeaders()), get); + assert.equal(get.status, 200); + assert.equal(get.headers?.["content-type"], "application/json"); + assert.equal(get.headers?.["x-bili-relay"], "yes"); + assert.equal(JSON.parse(get.body).ok, true); + + const post = new RelayRes(); + await route.handler(fakeReq("POST", "/__bili/config", browserHeaders(), ["{", '"a":1}']), post); + assert.equal(post.status, 200); + assert.equal(JSON.parse(post.body).echoed, '{"a":1}', "request body rides through byte-exact"); + + // apply() also probes the bound proxy (manifest/liveness) — count + // only the relayed admin calls. + const admin = stub.seen.filter((h) => h.url.startsWith("/__bili/config")); + assert.equal(admin.length, 2); + const first = admin[0]; + assert.equal(first.method, "GET"); + assert.equal(first.url, "/__bili/config", "the full /__bili path is preserved"); + const second = admin[1]; + assert.equal(second.method, "POST"); + assert.equal(second.body, '{"a":1}'); + for (const hit of [first, second]) { + assert.equal(hit.headers.host, new URL(stub.origin).host, "Host is the fetch-derived loopback host, not the browser's entry host"); + assert.equal(hit.headers.origin, undefined, "the browser's Origin header must not reach the admin gate"); + assert.equal(hit.headers.cookie, undefined, "entry cookies must not leak into the proxy"); + assert.equal(hit.headers.referer, undefined); + assert.equal(hit.headers["user-agent"], "relay-test", "ordinary headers ride through"); + } + }); + } finally { + stub.close(); + rmrf(home); + } +}); + +test("#3373: an unreachable proxy behind the gate surfaces as 502 JSON, not a crash", async () => { + const home = fs.mkdtempSync(path.join(os.tmpdir(), "bili-relay-")); + try { + await withEnv({ DSH_HOME: home, BILLION_CONTEXT_PROXY: undefined, BILI_CONFIG_FILE: writeConfig(home, JSON.stringify({ security: { adminRelay: true } })) }, async () => { + _resetRegisterForTest("http://127.0.0.1:1"); + const route = relayRoute(collectRoutes()); + const res = new RelayRes(); + await route.handler(fakeReq("GET", "/__bili/config", browserHeaders()), res); + assert.equal(res.status, 502); + assert.deepEqual(JSON.parse(res.body), { error: "bili proxy unreachable" }); + }); + } finally { + rmrf(home); + } +}); + +test("#3373 review: dot-segment traversal is contained — literal and %2e-encoded \"..\" are rejected with 400 and never reach the proxy", async () => { + const stub = await startAdminStub(); + const home = fs.mkdtempSync(path.join(os.tmpdir(), "bili-relay-")); + try { + await withEnv({ DSH_HOME: home, BILLION_CONTEXT_PROXY: undefined, BILI_CONFIG_FILE: writeConfig(home, JSON.stringify({ security: { adminRelay: true } })) }, async () => { + _resetRegisterForTest(stub.origin); + const route = relayRoute(collectRoutes()); + // apply() itself probes the bound proxy — baseline after mount so + // only relay hops are counted. + const baseline = stub.seen.length; + for (const url of ["/__bili/../__acp/sessions", "/__bili/%2e%2e/config", "/__bili/a/../../status"]) { + const res = new RelayRes(); + await route.handler(fakeReq("GET", url, browserHeaders()), res); + assert.equal(res.status, 400, url); + assert.deepEqual(JSON.parse(res.body), { error: "bad request path" }); + } + assert.equal(stub.seen.length, baseline, "rejected paths must not produce any upstream hop"); + }); + } finally { + stub.close(); + rmrf(home); + } +}); + +test("#3373 review: gate off + proxy unbound still presents 404 (fail-closed beats state disclosure)", async () => { + const home = fs.mkdtempSync(path.join(os.tmpdir(), "bili-relay-")); + try { + await withEnv({ DSH_HOME: home, BILLION_CONTEXT_PROXY: undefined, BILI_CONFIG_FILE: writeConfig(home, "{}") }, async () => { + _resetRegisterForTest(undefined); + const route = relayRoute(collectRoutes()); + const res = new RelayRes(); + await route.handler(fakeReq("GET", "/__bili/config", browserHeaders()), res); + assert.equal(res.status, 404); + assert.deepEqual(JSON.parse(res.body), { error: "admin relay disabled" }); + }); + } finally { + rmrf(home); + } +}); + +test("#3373 review: oversized bodies are rejected with 413 before any upstream hop", async () => { + const stub = await startAdminStub(); + const home = fs.mkdtempSync(path.join(os.tmpdir(), "bili-relay-")); + try { + await withEnv({ DSH_HOME: home, BILLION_CONTEXT_PROXY: undefined, BILI_CONFIG_FILE: writeConfig(home, JSON.stringify({ security: { adminRelay: true } })) }, async () => { + _resetRegisterForTest(stub.origin); + const route = relayRoute(collectRoutes()); + const baseline = stub.seen.length; + const res = new RelayRes(); + // 8 MiB cap + 1 byte (RELAY_MAX_BODY_BYTES in dsh-native.ts) + await route.handler(fakeReq("POST", "/__bili/config", browserHeaders(), ["x".repeat(8 * 1024 * 1024 + 1)]), res); + assert.equal(res.status, 413); + assert.deepEqual(JSON.parse(res.body), { error: "request body too large" }); + assert.equal(stub.seen.length, baseline, "an oversized body must not be forwarded"); + }); + } finally { + stub.close(); + rmrf(home); + } +}); diff --git a/tests/dsh-native.test.ts b/tests/dsh-native.test.ts index 9a23653bbf..293b6af152 100644 --- a/tests/dsh-native.test.ts +++ b/tests/dsh-native.test.ts @@ -7,6 +7,7 @@ import http from "node:http"; import { pathToFileURL } from "node:url"; import { apply, planNativeDsh, shouldBootstrapNativeDsh, persistClientEvent, _resetRegisterForTest, _setModelInfoForTest, _setSpawnForTest, _setThrewRetryForTest, _settleNativeForTest, _stateHeadersForTest, _stateRespawnForTest, _stateTakeoverGateForTest, _stateToolsReadyForTest, _stateBeforeSendForTest, _stateOnResponseForTest, _noteRoutedForTest, _resetRoutedForTest, _resetWebProfileWarningForTest, _resetTitlesForTest, type PluginContext } from "../src/agent/dsh-native.ts"; import { __resetIncrementalClientForTests } from "../src/agent/incremental-client.ts"; +import { BILI_COMPACTION_DISABLE_SERVICE } from "../src/agent/dsh-compaction-disable.ts"; // #3370: the incremental-turn conversion must stamp the marker header on the // SAME init as the body swap (atomic, omp pattern) — the old arms-a-flag / @@ -40,6 +41,8 @@ test("#3370: dsh incremental conversion is atomic (marker + body on the same ini assert.equal(body1.conversationId, "dsh-inc-1", "seed body swapped in"); assert.equal(body1.resync, true); assert.equal(new Headers(init1.headers).get("x-bili-plugin-turn"), "1", "#3370: marker stamped on the same init as the body"); + const turnId1 = new Headers(init1.headers).get("x-bili-turn-id"); + assert.ok(turnId1, "#3329: per-request turn id stamped alongside the marker"); const extra = headersFor(url) ?? {}; assert.equal(extra["x-bili-plugin-turn"], undefined, "headersFor no longer stamps/consumes the marker"); assert.equal(extra["x-bili-plugin"], "dsh", "plugin identity headers still stamped"); @@ -56,11 +59,18 @@ test("#3370: dsh incremental conversion is atomic (marker + body on the same ini // 2b. that skipped request's plain 200 (no ack header) arrives // BEFORE the marker'd response — it must not consume the // pending entry, or the seed's ack below is lost and turn 3 - // replans a resync instead of a delta (#3378 review). + // replans a resync instead of a delta (#3378 review). Under + // turn-id keying (#3329) the 2-arg call carries no outgoing + // headers at all — a response with no turn-id header is not + // an incremental ack and consumes nothing, which is exactly + // the no-op this step pins. onResponse(url, new Response("{}", { status: 200 })); - // 3. ack the seed → the next turn plans a DELTA (sinceSeq advanced) - onResponse(url, new Response("{}", { status: 200, headers: { "x-bili-applied-seq": String(msgs1.length) } })); + // 3. ack the seed → the next turn plans a DELTA (sinceSeq advanced). + // #3329: the ack is attributed via the seed's OWN turn id, read + // back from its outgoing headers (what native-intercept threads + // into onResponse in production). + onResponse(url, new Response("{}", { status: 200, headers: { "x-bili-applied-seq": String(msgs1.length) } }), { "x-bili-turn-id": turnId1 }); const init3: RequestInit = { method: "POST", body: original2 }; await beforeSend(url, init3); const body3 = JSON.parse(init3.body as string) as { sinceSeq?: number; messages?: unknown[]; resync?: boolean }; @@ -539,6 +549,11 @@ function mockCtx() { type HostCtx = Parameters[0]; type HostListener = Parameters>[1]; const listeners: Record = {}; + // #3354: capture the load-time compaction-disable overlay — event order, + // provided services, and the internal/config middleware for simulation. + const order: string[] = []; + const providedServices: Array<{ name: string; value: unknown }> = []; + let compactionMiddleware: ((raw: unknown, next: () => unknown) => unknown) | undefined; // Stable service identities (like real cordis services: one facade per // process whose state mutates underneath); getters gate exposure on the // backing fakes so T4's no-services degradation still reads absent. @@ -562,7 +577,11 @@ function mockCtx() { setInitiator: (i: { session?: { id?: unknown; requestHeader?: () => { config?: { provider?: string; model?: string } } | undefined } } | undefined) => (initiator = i), registeredTools: tools, registeredCommands: commands, - on: (event: string, listener: HostListener) => { (listeners[event] ??= []).push(listener); }, + on: (event: string, listener: HostListener, _options?: { global?: boolean }) => { + order.push(`on:${event}`); + if (event === "internal/config") compactionMiddleware = listener as (raw: unknown, next: () => unknown) => unknown; + (listeners[event] ??= []).push(listener); + }, emit: (event: string, ...args: unknown[]) => { for (const l of listeners[event] ?? []) (l as (...a: unknown[]) => void)(...args); }, get sessions(): PluginContext["sessions"] { return dshSessions !== undefined ? sessionsService : undefined; }, get sessionTitle(): PluginContext["sessionTitle"] { return dshTitleFold !== undefined ? titleService : undefined; }, @@ -591,6 +610,10 @@ function mockCtx() { agentDefaultModel = a; }, setProfileContext: (pc: { startedBundles?: readonly string[] } | undefined) => (profileContext = pc), + reflect: { provide: (name: string, value: unknown) => { order.push(`provide:${name}`); providedServices.push({ name, value }); } }, + providedServices, + registrationOrder: () => [...order], + compactionMiddleware: () => compactionMiddleware, }; } @@ -906,6 +929,10 @@ test("apply() is a no-op under the kill switches", async () => { apply(ctx); assert.equal(ctx.registeredTools.length, 0); assert.equal(ctx.registeredCommands.length, 0); + // #3354: a kill-switched bili must not register the load-time + // overlay either — the lane goes back to stock dsh behavior. + assert.ok(!ctx.registrationOrder().some((e) => e.startsWith("on:internal/config"))); + assert.equal(ctx.providedServices.length, 0); }); } finally { rmrf(home); @@ -933,13 +960,14 @@ test("#1772 apply(): web-profile compaction caveat warns once in the durable log apply(ctx); assert.equal(warnLines().length, 1); assert.match(warnLines()[0], /#1772/); - // #2474: the caveat must point at the working exit (full-snapshot - // override of preset-standard via cordis.patch.yml), not end in a - // dead end — pin the pointer so the correction can't regress. + // #3354: the note now states bili disables the preset rows at load + // time (vendored overlay, attribution preserved) and keeps the + // #2474 full-snapshot recipe as the documented fallback — pin both + // pointers so either regression is caught. + assert.match(warnLines()[0], /#3354/); + assert.match(warnLines()[0], /deshark42\/dsh-compaction-basic-disable/); assert.match(warnLines()[0], /#2474/); assert.match(warnLines()[0], /cordis\.patch\.yml/); - assert.match(warnLines()[0], /WHOLESALE/); - assert.match(warnLines()[0], /insert:/); // #1791: settle each apply's attach chain while the mock is still // open — a deferred probe settling against a closed mock runs the // spawn fallback mid-way through a LATER test and clobbers the @@ -987,6 +1015,57 @@ test("#1772 apply(): web-profile compaction caveat warns once in the durable log } }); +test("#3354 apply(): registers the load-time compaction-disable overlay and publishes its gate", async () => { + const proxy = await startMockProxy([]); + const home = fs.mkdtempSync(path.join(os.tmpdir(), "bili-dsh-cd-")); + try { + await withEnv({ DSH_HOME: home, BILLION_CONTEXT_PROXY: proxy.origin }, async () => { + _resetRegisterForTest(proxy.origin); + const ctx = mockCtx(); + apply(ctx); + // the gate service publishes AFTER the middleware registers — the + // preset inject latch can never be satisfied before the transform + // is live + const iOn = ctx.registrationOrder().indexOf("on:internal/config"); + const iProvide = ctx.registrationOrder().indexOf(`provide:${BILI_COMPACTION_DISABLE_SERVICE}`); + assert.ok(iOn >= 0, "internal/config middleware must be registered"); + assert.ok(iProvide > iOn, "gate service must publish after the middleware registers"); + assert.deepEqual(ctx.providedServices.map((s) => s.name), [BILI_COMPACTION_DISABLE_SERVICE]); + + const mw = ctx.compactionMiddleware(); + assert.equal(typeof mw, "function"); + const run = (self: unknown, cfg: Record) => + (mw as (raw: unknown, next: () => unknown) => unknown).call(self, null, () => cfg); + // agent-preset entry: both compaction rows disabled, everything else untouched + const plugins: Array> = [ + { name: "@deepseek-ai/dsh-compaction-basic", config: { auto: true } }, + { name: "@deepseek-ai/dsh-command-compact" }, + { name: "@deepseek-ai/dsh-other" }, + ]; + const presetCfg: Record = { plugins, x: 1 }; + const out = run({ entry: { options: { name: "@deepseek-ai/dsh-agent-preset" } } }, presetCfg) as { plugins?: Array>; x?: number }; + assert.equal(out.x, 1); + assert.deepEqual(out.plugins?.[0]?.config, { auto: true }); + assert.equal(out.plugins?.[0]?.disabled, true); + assert.equal(out.plugins?.[1]?.disabled, true); + assert.equal(out.plugins?.[2]?.disabled, undefined); + // the original input is not mutated — detached copies only + assert.equal(plugins[0].disabled, undefined); + // non-preset entries pass through verbatim + const otherCfg: Record = { plugins }; + assert.equal(run({ entry: { options: { name: "something-else" } } }, otherCfg), otherCfg); + // a preset config without a plugins key passes through verbatim + const emptyCfg: Record = {}; + assert.equal(run({ entry: { options: { name: "@deepseek-ai/dsh-agent-preset" } } }, emptyCfg), emptyCfg); + await _stateToolsReadyForTest(); + }); + } finally { + proxy.close(); + rmrf(home); + _resetRegisterForTest(undefined); + } +}); + test("apply() runtime-info (#955): model services stamp model/window/max-output headers", async () => { const proxy = await startMockProxy([]); const home = fs.mkdtempSync(path.join(os.tmpdir(), "bili-dsh-ri-")); diff --git a/tests/dsh-settings-ui.test.ts b/tests/dsh-settings-ui.test.ts index 13c7c89392..01598ecb35 100644 --- a/tests/dsh-settings-ui.test.ts +++ b/tests/dsh-settings-ui.test.ts @@ -48,6 +48,15 @@ function startManifestProxy(): Promise<{ origin: string; close: () => void }> { }); } +// #3373: hermetic config isolation — the relay gate reads the real config +// file unless pointed elsewhere; a machine with security.adminRelay on must +// not flip these tests. +function emptyConfigFile(home: string): string { + const file = path.join(home, "bili-config.json"); + fs.writeFileSync(file, "{}"); + return file; +} + type InjectRow = { kind: string; name?: string; value?: unknown }; type InjectListener = (table: InjectRow[]) => void; @@ -67,7 +76,7 @@ test("#1590: webserver/index-inject publishes __BILI__ while an origin is known const proxy = await startManifestProxy(); const home = fs.mkdtempSync(path.join(os.tmpdir(), "bili-dsh-ui-")); try { - await withEnv({ DSH_HOME: home, BILLION_CONTEXT_PROXY: proxy.origin }, async () => { + await withEnv({ DSH_HOME: home, BILLION_CONTEXT_PROXY: proxy.origin, BILI_CONFIG_FILE: emptyConfigFile(home) }, async () => { _resetRegisterForTest(proxy.origin); const listeners: InjectListener[] = []; apply(uiCtx(listeners) as Parameters[0]); @@ -87,7 +96,7 @@ test("#1590: webserver/index-inject publishes __BILI__ while an origin is known test("#1590: index-inject stays silent when no origin is known yet (spawn mode)", async () => { const home = fs.mkdtempSync(path.join(os.tmpdir(), "bili-dsh-ui-")); try { - await withEnv({ DSH_HOME: home, BILLION_CONTEXT_PROXY: undefined }, async () => { + await withEnv({ DSH_HOME: home, BILLION_CONTEXT_PROXY: undefined, BILI_CONFIG_FILE: emptyConfigFile(home) }, async () => { _resetRegisterForTest(undefined); const listeners: InjectListener[] = []; apply(uiCtx(listeners) as Parameters[0]); @@ -133,15 +142,19 @@ function routeCtx(routes: RouteRow[], listeners: InjectListener[]) { test("#1809: live /bili/origin route reflects the bound origin", async () => { const home = fs.mkdtempSync(path.join(os.tmpdir(), "bili-dsh-ui-")); try { - await withEnv({ DSH_HOME: home, BILLION_CONTEXT_PROXY: undefined }, async () => { + await withEnv({ DSH_HOME: home, BILLION_CONTEXT_PROXY: undefined, BILI_CONFIG_FILE: emptyConfigFile(home) }, async () => { _resetRegisterForTest("http://127.0.0.1:8787"); const routes: RouteRow[] = []; apply(routeCtx(routes, []) as Parameters[0]); - assert.equal(routes.length, 1); - assert.equal(routes[0].kind, "exact"); - assert.equal(routes[0].path, "/bili/origin"); + // #3373: the panel now registers two routes — the origin probe and + // the /__bili admin relay (fail-closed while the gate is off). + assert.equal(routes.length, 2); + const originRoute = routes.find((r) => r.path === "/bili/origin")!; + const relayRoute = routes.find((r) => r.path === "/__bili")!; + assert.equal(originRoute.kind, "exact"); + assert.equal(relayRoute.kind, "prefix"); const res = new FakeRes(); - await routes[0].handler({}, res); + await originRoute.handler({}, res); assert.equal(res.status, 200); assert.equal(res.headers?.["content-type"], "application/json"); assert.deepEqual(JSON.parse(res.body), { origin: "http://127.0.0.1:8787" }); @@ -154,13 +167,14 @@ test("#1809: live /bili/origin route reflects the bound origin", async () => { test("#1809: /bili/origin answers null before binding", async () => { const home = fs.mkdtempSync(path.join(os.tmpdir(), "bili-dsh-ui-")); try { - await withEnv({ DSH_HOME: home, BILLION_CONTEXT_PROXY: undefined }, async () => { + await withEnv({ DSH_HOME: home, BILLION_CONTEXT_PROXY: undefined, BILI_CONFIG_FILE: emptyConfigFile(home) }, async () => { _resetRegisterForTest(undefined); const routes: RouteRow[] = []; apply(routeCtx(routes, []) as Parameters[0]); - assert.equal(routes.length, 1); + assert.equal(routes.length, 2); + const originRoute = routes.find((r) => r.path === "/bili/origin")!; const res = new FakeRes(); - await routes[0].handler({}, res); + await originRoute.handler({}, res); assert.deepEqual(JSON.parse(res.body), { origin: null }); }); } finally { @@ -171,7 +185,7 @@ test("#1809: /bili/origin answers null before binding", async () => { test("#1809: route registration rides the injected context's effect lifecycle", async () => { const home = fs.mkdtempSync(path.join(os.tmpdir(), "bili-dsh-ui-")); try { - await withEnv({ DSH_HOME: home, BILLION_CONTEXT_PROXY: undefined }, async () => { + await withEnv({ DSH_HOME: home, BILLION_CONTEXT_PROXY: undefined, BILI_CONFIG_FILE: emptyConfigFile(home) }, async () => { _resetRegisterForTest("http://127.0.0.1:8787"); const routes: RouteRow[] = []; let disposed = 0; @@ -197,12 +211,15 @@ test("#1809: route registration rides the injected context's effect lifecycle", }; apply(ctx as Parameters[0]); assert.equal(effects.length, 1, "registration must be wrapped in the context effect"); - assert.equal(effects[0].label, "bili: /bili/origin route"); + assert.equal(effects[0].label, "bili: panel routes (origin + admin relay)"); const cleanup = effects[0].fn(); - assert.equal(routes.length, 1); - assert.equal(typeof cleanup, "function", "the disposer doubles as the effect cleanup"); + assert.equal(routes.length, 2); + // #3373 review: the effect fn must hand back a combined disposer — + // dsh calls it on re-run to un-register BOTH routes; returning + // undefined lets stale duplicates accumulate in the router table. + assert.equal(typeof cleanup, "function", "the combined disposer doubles as the effect cleanup"); (cleanup as () => void)(); - assert.equal(disposed, 1); + assert.equal(disposed, 2); }); } finally { rmrf(home); @@ -546,6 +563,94 @@ test("#1590/#2125: client bundle registers the settings.section entry bili AND t } }); +test("#3373: relay mode — probe {origin:null,relay:true} binds same-origin; prefix gateways resolve through the base-relative fallback form", async () => { + { + // Root relay deployment: no response URL ⇒ base "" (same-origin paths). + const opened: string[] = []; + const m = await mountClient({ + open: (url: string) => opened.push(url), + fetch: async (url: string) => { + if (String(url).endsWith("/__bili/?embed=1")) return htmlOk(); + return { ok: true, json: async () => ({ origin: null, relay: true }) }; + }, + setTimeout, + clearTimeout, + }); + m.resetHooks(); + const first = m.component() as ElementNode; + // No snapshot here — first paint is the degraded hint until the first + // poll resolves (the snapshot path is covered below). + assert.equal(findTag(first, "iframe"), undefined, "#3100: the frame still waits for the pre-flight verdict"); + await tick(); + m.component(); + await tick(); + const second = m.component() as ElementNode; + const ft: string[] = []; + collectText(second, ft); + assert.ok(!ft.some((x) => x.includes("/acp")), "relay mode is bound, not degraded"); + const frame = findTag(second, "iframe"); + assert.ok(frame !== undefined, "relay mode renders the frame"); + assert.equal(frame.props?.src, "/__bili/?embed=1&lang=zh#/overview", "same-origin frame under the entry relay"); + const buttons = collectButtons(second); + (buttons[buttons.length - 1].props!.onClick as () => void)(); + assert.deepEqual(opened, ["/__bili/"]); + m.runCleanups(); + } + { + // Prefix gateway (fnOS-style /app/dsh/): the absolute probe path misses; + // the base-relative fallback form carries the prefix and its response + // URL becomes the absolute UI base. + const m = await mountClient({ + document: { baseURI: "https://dsh.example/app/dsh/" }, + URL, + open: (_url: string) => {}, + fetch: async (url: string) => { + const u = String(url); + if (u === "/bili/origin") return { ok: false }; + if (u.endsWith("/bili/origin")) return { ok: true, url: u, json: async () => ({ origin: null, relay: true }) }; + if (u.endsWith("/__bili/?embed=1")) return htmlOk(); + throw new Error(`unexpected fetch: ${u}`); + }, + setTimeout, + clearTimeout, + }); + m.resetHooks(); + m.component(); + await tick(); + m.component(); + await tick(); + const tree = m.component() as ElementNode; + const frame = findTag(tree, "iframe"); + assert.ok(frame !== undefined, "prefix gateway resolves through the fallback probe form"); + assert.equal(frame.props?.src, "https://dsh.example/app/dsh/__bili/?embed=1&lang=zh#/overview"); + m.runCleanups(); + } + { + // Relay snapshot (__BILI__ carries relay:true): first paint is bound. + const m = await mountClient({ + __BILI__: { origin: null, relay: true }, + open: (_url: string) => {}, + fetch: async (url: string) => { + if (String(url).endsWith("/__bili/?embed=1")) return htmlOk(); + return { ok: true, json: async () => ({ origin: null, relay: true }) }; + }, + setTimeout, + clearTimeout, + }); + m.resetHooks(); + const first = m.component() as ElementNode; + const ft: string[] = []; + collectText(first, ft); + assert.ok(!ft.some((x) => x.includes("/acp")), "relay snapshot is bound from first paint"); + await tick(); + m.component(); + await tick(); + const second = m.component() as ElementNode; + assert.ok(findTag(second, "iframe") !== undefined, "relay snapshot + live route renders the frame"); + m.runCleanups(); + } +}); + test("#1809/#2125/#2187: client polls /bili/origin while unresolved — upgrades on success, stays degraded and cancels when absent (both slots); slow phase continues past the fast attempts; #2288 corrects a stale boot snapshot and follows mid-session re-binds", async () => { // #3100: every fetch stub below dispatches on URL — the host origin route // answers JSON, the proxy embed pre-flight answers HTML (htmlOk above). diff --git a/tests/fix-1897-leaked-tools-side.test.ts b/tests/fix-1897-leaked-tools-side.test.ts index 93a575c639..9d98a01a2a 100644 --- a/tests/fix-1897-leaked-tools-side.test.ts +++ b/tests/fix-1897-leaked-tools-side.test.ts @@ -5,7 +5,7 @@ import { once } from "node:events"; import { defaultConfig, createInitialState } from "acp-kernel"; import { startServer, stripLeakedBiliTools, BILI_TOOL_NAMES } from "../src/server.ts"; import type { ProxyOptions } from "../src/config.ts"; -import { PROXY_TOOL_NAMES, RETRIEVE_TOOL_NAME, RULE_TOOL_NAME } from "../src/compress-tool.ts"; +import { PROXY_TOOL_NAMES, ABSORB_TOOL_NAME, RETRIEVE_TOOL_NAME, RULE_TOOL_NAME } from "../src/compress-tool.ts"; import { SessionStore, _setStoreForTest } from "../src/persist.ts"; import { _setForTest as setRegistryForTest } from "../src/registry.ts"; import { getSession, _resetSessionsForTest } from "../src/session.ts"; @@ -59,9 +59,9 @@ test("stripLeakedBiliTools: google functionDeclarations (one entry may declare m }); test("stripLeakedBiliTools: opt-in bili extras count as bili-owned", () => { - const body = { model: "m", tools: [openAiTool(RULE_TOOL_NAME), openAiTool(RETRIEVE_TOOL_NAME)] }; + const body = { model: "m", tools: [openAiTool(ABSORB_TOOL_NAME), openAiTool(RULE_TOOL_NAME), openAiTool(RETRIEVE_TOOL_NAME)] }; assert.equal(stripLeakedBiliTools(body), true); - for (const n of [RULE_TOOL_NAME, RETRIEVE_TOOL_NAME]) { + for (const n of [ABSORB_TOOL_NAME, RULE_TOOL_NAME, RETRIEVE_TOOL_NAME]) { assert.ok(BILI_TOOL_NAMES.has(n), `${n} in the bili set`); } }); diff --git a/tests/fix-decompress-tofile.test.ts b/tests/fix-decompress-tofile.test.ts index 0441c30860..bbdf6bdcfa 100644 --- a/tests/fix-decompress-tofile.test.ts +++ b/tests/fix-decompress-tofile.test.ts @@ -134,7 +134,7 @@ function foldCcr() { const config = applyCompressSettings( defaultConfig(200_000), 200_000, - { ccr: { enabled: true, minToolTokens: 50 } }, + { absorb: { enabled: true, minToolTokens: 50 }, ccr: { enabled: true, minToolTokens: 50 } }, ) as Config; const session = getSession(`tfrange-${Math.random().toString(36).slice(2)}`); storeEffectiveCcr(session, { enabled: true, minToolTokens: 50 }); diff --git a/tests/fork-ccr-adoption.test.ts b/tests/fork-ccr-adoption.test.ts index b40133efac..66617304f8 100644 --- a/tests/fork-ccr-adoption.test.ts +++ b/tests/fork-ccr-adoption.test.ts @@ -54,7 +54,7 @@ function ccrConfig(on: boolean): Config { // whole fold range nondeterministically. defaultConfig(LIMIT, { preserveRecentMessages: 0, preserveRecentTokens: 0, compress: { minCompressRange: 10, minSummaryLength: 20, maxSummaryLength: 5000 } }), LIMIT, - { ccr: { enabled: on, minToolTokens: 50 } }, + { absorb: { enabled: true, minToolTokens: 50 }, ccr: { enabled: on, minToolTokens: 50 } }, ) as Config; } diff --git a/tests/golden-storage-render.test.ts b/tests/golden-storage-render.test.ts index 180347458f..6e02999dde 100644 --- a/tests/golden-storage-render.test.ts +++ b/tests/golden-storage-render.test.ts @@ -17,6 +17,11 @@ import path from "node:path"; import { fileURLToPath } from "node:url"; import { driveWire, WIRES } from "./golden-storage-drive.ts"; +// #3400: pin the legacy wire shape — the egress image rescale changes +// these digests by design (its own suite covers it); this suite proves +// render/storage fidelity against committed references. +process.env.BILI_IMAGE_RESCALE = "0"; + const GOLDEN_DIR = path.join(path.dirname(fileURLToPath(import.meta.url)), "golden", "storage-render"); test("storage-render goldens: no orphaned snapshot files", () => { diff --git a/tests/golden/storage-render/anthropic.json b/tests/golden/storage-render/anthropic.json index 699bd64cba..258b7a7879 100644 --- a/tests/golden/storage-render/anthropic.json +++ b/tests/golden/storage-render/anthropic.json @@ -80,12 +80,12 @@ "foldAt": 9, "editedAt": 12, "summaryFrom": 9, - "stateDigest": "7975db3b6782fc36a06cabe7f70d6c0813a26254ea16b1bfdfa712482cf1a18b", + "stateDigest": "00a49d95b7ffd5e5c76e9f17d1d5405293aa1a7d08e4ed7fa778b77b5e8cb788", "sectionDigests": { "state": "fae14b25a41f5844", "blockContents": "dc7d13d250c94bb4", "stats": "cff3c2546898aeb3", - "metadata": "b04f2fb422a5529b" + "metadata": "5413c879755acf18" }, "stateKeyDigests": { "blocks": "1cbe172309e4490d", diff --git a/tests/golden/storage-render/chat.json b/tests/golden/storage-render/chat.json index 8c467cd788..3af63534ec 100644 --- a/tests/golden/storage-render/chat.json +++ b/tests/golden/storage-render/chat.json @@ -80,12 +80,12 @@ "foldAt": 9, "editedAt": 12, "summaryFrom": 9, - "stateDigest": "b9621b1ad7556db259a9f03d3c37df81e77668fcc0c2be78513f066ea06dc750", + "stateDigest": "7453eabb99ead81805a1bf2d6630232df2e02ab10772b7b03dd88495cfc90d17", "sectionDigests": { "state": "d5bc5306f840723d", "blockContents": "94a58f96d7c5cfc3", "stats": "59842ce5415d3a4d", - "metadata": "b323123d4fc029f7" + "metadata": "d05f9d95a0dd2fce" }, "stateKeyDigests": { "blocks": "9aa73e5fb9f93d6c", diff --git a/tests/golden/storage-render/google.json b/tests/golden/storage-render/google.json index ec0df6c804..d76d38a238 100644 --- a/tests/golden/storage-render/google.json +++ b/tests/golden/storage-render/google.json @@ -80,12 +80,12 @@ "foldAt": 9, "editedAt": 12, "summaryFrom": 9, - "stateDigest": "842b1818adac8210db66fd5cc923a4e0e8788e1209d8457cd4d167b1c45ae8e1", + "stateDigest": "48c0fbd96f53485e9b8109ae7e24cc1cf7d0b0a265591f7f4e0b04689dbbdb86", "sectionDigests": { "state": "2a2af1c92fcb4f88", "blockContents": "473264d9df8ba338", "stats": "660f1797bf2044c4", - "metadata": "a97a1e2bf9f13c94" + "metadata": "4d073f1e812dcb89" }, "stateKeyDigests": { "blocks": "8e14f0879bf7ebd7", diff --git a/tests/golden/storage-render/responses.json b/tests/golden/storage-render/responses.json index 55ba150bf4..a7cd175ed7 100644 --- a/tests/golden/storage-render/responses.json +++ b/tests/golden/storage-render/responses.json @@ -80,12 +80,12 @@ "foldAt": 9, "editedAt": 12, "summaryFrom": 9, - "stateDigest": "918b8cef2fc95712c66910e344b5e7e132f2500906c1151a7bdea1d84c2602e9", + "stateDigest": "05efdc2df5d4ef3a4d9f8f3c98de81668dc33eb81d2e3300b18dde92d9bdb54d", "sectionDigests": { "state": "6546d2e5f2826284", "blockContents": "8460729b2654a816", "stats": "e3a3aeb598c183ff", - "metadata": "15c1da97b893af8c" + "metadata": "44e07f67bbeb366b" }, "stateKeyDigests": { "blocks": "7fff1dfb54a617cf", diff --git a/tests/golden/wire-contract/bili.json b/tests/golden/wire-contract/bili.json index eef6e13d04..95b5629723 100644 --- a/tests/golden/wire-contract/bili.json +++ b/tests/golden/wire-contract/bili.json @@ -1057,6 +1057,28 @@ } }, "synthesized": { + "absorbResponses": { + "type": "function", + "name": "absorb", + "description": "Distill a tool result into a compact summary you write. REQUIRED immediately after a tool result ends with an [ACP absorb] instruction: pass its ref and the distilled essentials (outcome, key values, paths:lines, errors, decisions). The original output is then removed from context; your summary is the durable record.", + "parameters": { + "type": "object", + "properties": { + "ref": { + "type": "string", + "description": "mNNNNN ref of the tool result to absorb (from the [ACP absorb] instruction)" + }, + "summary": { + "type": "string", + "description": "Distilled essentials of the tool result — this replaces the original output in context" + } + }, + "required": [ + "ref", + "summary" + ] + } + }, "ruleAnthropic": { "name": "acp_rule", "description": "Record a short, principle-level reminder so it survives context compression — the call and its result are protected and stay in context. Record when: the user calls out or repeatedly emphasizes a lesson; the user asks you to remember or follow a behavior; you personally hit a major pitfall worth remembering long-term. Keep each rule to one short line. Omit the rule argument to list recorded rules. To remove a recorded rule pass delete with its id (e.g. \"rule3\"); to remove every recorded rule pass clear: true. delete and clear are mutually exclusive with each other and with rule — use one operation per call.", diff --git a/tests/golden/wire-contract/manifest-default.json b/tests/golden/wire-contract/manifest-default.json index dfb64e1c5f..ff56c1659e 100644 --- a/tests/golden/wire-contract/manifest-default.json +++ b/tests/golden/wire-contract/manifest-default.json @@ -9,7 +9,8 @@ "decompress", "search_context", "acp_status", - "acp_cache" + "acp_cache", + "read_original_image" ], "tools": { "anthropic": [ @@ -203,6 +204,22 @@ } } } + }, + { + "name": "read_original_image", + "description": "Read the ORIGINAL full-resolution image of a message whose picture was downscaled before reaching you. Beside each shrunken image there is a [bili] note quoting its mNNNNN ref and both sizes — pass that ref here. The original bytes are returned as an image attached to this result. The original can be much larger than the downscaled copy, so call this only when you actually need full-resolution detail (small text, colors, pixel alignment); the downscaled version is enough for most work.", + "input_schema": { + "type": "object", + "properties": { + "ref": { + "type": "string", + "description": "The mNNNNN message ref quoted in the [bili] downsample note beside the shrunken image." + } + }, + "required": [ + "ref" + ] + } } ], "openai": [ @@ -411,6 +428,25 @@ } } } + }, + { + "type": "function", + "function": { + "name": "read_original_image", + "description": "Read the ORIGINAL full-resolution image of a message whose picture was downscaled before reaching you. Beside each shrunken image there is a [bili] note quoting its mNNNNN ref and both sizes — pass that ref here. The original bytes are returned as an image attached to this result. The original can be much larger than the downscaled copy, so call this only when you actually need full-resolution detail (small text, colors, pixel alignment); the downscaled version is enough for most work.", + "parameters": { + "type": "object", + "properties": { + "ref": { + "type": "string", + "description": "The mNNNNN message ref quoted in the [bili] downsample note beside the shrunken image." + } + }, + "required": [ + "ref" + ] + } + } } ], "responses": [ @@ -609,6 +645,23 @@ } } } + }, + { + "type": "function", + "name": "read_original_image", + "description": "Read the ORIGINAL full-resolution image of a message whose picture was downscaled before reaching you. Beside each shrunken image there is a [bili] note quoting its mNNNNN ref and both sizes — pass that ref here. The original bytes are returned as an image attached to this result. The original can be much larger than the downscaled copy, so call this only when you actually need full-resolution detail (small text, colors, pixel alignment); the downscaled version is enough for most work.", + "parameters": { + "type": "object", + "properties": { + "ref": { + "type": "string", + "description": "The mNNNNN message ref quoted in the [bili] downsample note beside the shrunken image." + } + }, + "required": [ + "ref" + ] + } } ] }, diff --git a/tests/golden/wire-contract/manifest-optin.json b/tests/golden/wire-contract/manifest-optin.json index e10877f31b..ebdb07b6b7 100644 --- a/tests/golden/wire-contract/manifest-optin.json +++ b/tests/golden/wire-contract/manifest-optin.json @@ -10,8 +10,10 @@ "search_context", "acp_status", "acp_cache", + "absorb", "acp_rule", - "acp_retrieve" + "acp_retrieve", + "read_original_image" ], "tools": { "anthropic": [ @@ -214,6 +216,27 @@ } } }, + { + "name": "absorb", + "description": "Distill a tool result into a compact summary you write. REQUIRED immediately after a tool result ends with an [ACP absorb] instruction: pass its ref and the distilled essentials (outcome, key values, paths:lines, errors, decisions). The original output is then removed from context; your summary is the durable record.", + "input_schema": { + "type": "object", + "properties": { + "ref": { + "type": "string", + "description": "mNNNNN ref of the tool result to absorb (from the [ACP absorb] instruction)" + }, + "summary": { + "type": "string", + "description": "Distilled essentials of the tool result — this replaces the original output in context" + } + }, + "required": [ + "ref", + "summary" + ] + } + }, { "name": "acp_rule", "description": "Record a short, principle-level reminder so it survives context compression — the call and its result are protected and stay in context. Record when: the user calls out or repeatedly emphasizes a lesson; the user asks you to remember or follow a behavior; you personally hit a major pitfall worth remembering long-term. Keep each rule to one short line. Omit the rule argument to list recorded rules. To remove a recorded rule pass delete with its id (e.g. \"rule3\"); to remove every recorded rule pass clear: true. delete and clear are mutually exclusive with each other and with rule — use one operation per call.", @@ -250,6 +273,22 @@ "ref" ] } + }, + { + "name": "read_original_image", + "description": "Read the ORIGINAL full-resolution image of a message whose picture was downscaled before reaching you. Beside each shrunken image there is a [bili] note quoting its mNNNNN ref and both sizes — pass that ref here. The original bytes are returned as an image attached to this result. The original can be much larger than the downscaled copy, so call this only when you actually need full-resolution detail (small text, colors, pixel alignment); the downscaled version is enough for most work.", + "input_schema": { + "type": "object", + "properties": { + "ref": { + "type": "string", + "description": "The mNNNNN message ref quoted in the [bili] downsample note beside the shrunken image." + } + }, + "required": [ + "ref" + ] + } } ], "openai": [ @@ -467,6 +506,30 @@ } } }, + { + "type": "function", + "function": { + "name": "absorb", + "description": "Distill a tool result into a compact summary you write. REQUIRED immediately after a tool result ends with an [ACP absorb] instruction: pass its ref and the distilled essentials (outcome, key values, paths:lines, errors, decisions). The original output is then removed from context; your summary is the durable record.", + "parameters": { + "type": "object", + "properties": { + "ref": { + "type": "string", + "description": "mNNNNN ref of the tool result to absorb (from the [ACP absorb] instruction)" + }, + "summary": { + "type": "string", + "description": "Distilled essentials of the tool result — this replaces the original output in context" + } + }, + "required": [ + "ref", + "summary" + ] + } + } + }, { "type": "function", "function": { @@ -509,6 +572,25 @@ ] } } + }, + { + "type": "function", + "function": { + "name": "read_original_image", + "description": "Read the ORIGINAL full-resolution image of a message whose picture was downscaled before reaching you. Beside each shrunken image there is a [bili] note quoting its mNNNNN ref and both sizes — pass that ref here. The original bytes are returned as an image attached to this result. The original can be much larger than the downscaled copy, so call this only when you actually need full-resolution detail (small text, colors, pixel alignment); the downscaled version is enough for most work.", + "parameters": { + "type": "object", + "properties": { + "ref": { + "type": "string", + "description": "The mNNNNN message ref quoted in the [bili] downsample note beside the shrunken image." + } + }, + "required": [ + "ref" + ] + } + } } ], "responses": [ @@ -708,6 +790,28 @@ } } }, + { + "type": "function", + "name": "absorb", + "description": "Distill a tool result into a compact summary you write. REQUIRED immediately after a tool result ends with an [ACP absorb] instruction: pass its ref and the distilled essentials (outcome, key values, paths:lines, errors, decisions). The original output is then removed from context; your summary is the durable record.", + "parameters": { + "type": "object", + "properties": { + "ref": { + "type": "string", + "description": "mNNNNN ref of the tool result to absorb (from the [ACP absorb] instruction)" + }, + "summary": { + "type": "string", + "description": "Distilled essentials of the tool result — this replaces the original output in context" + } + }, + "required": [ + "ref", + "summary" + ] + } + }, { "type": "function", "name": "acp_rule", @@ -729,6 +833,23 @@ } } } + }, + { + "type": "function", + "name": "read_original_image", + "description": "Read the ORIGINAL full-resolution image of a message whose picture was downscaled before reaching you. Beside each shrunken image there is a [bili] note quoting its mNNNNN ref and both sizes — pass that ref here. The original bytes are returned as an image attached to this result. The original can be much larger than the downscaled copy, so call this only when you actually need full-resolution detail (small text, colors, pixel alignment); the downscaled version is enough for most work.", + "parameters": { + "type": "object", + "properties": { + "ref": { + "type": "string", + "description": "The mNNNNN message ref quoted in the [bili] downsample note beside the shrunken image." + } + }, + "required": [ + "ref" + ] + } } ] }, diff --git a/tests/incremental-lane.test.ts b/tests/incremental-lane.test.ts index 02ce49aca0..cf71b9edf2 100644 --- a/tests/incremental-lane.test.ts +++ b/tests/incremental-lane.test.ts @@ -39,7 +39,7 @@ interface Harness { close: () => Promise; } -async function harness(storageMode: "full" | "legacy", opts: { upstreamDelayMs?: number } = {}): Promise { +async function harness(storageMode: "full" | "legacy", opts: { upstreamDelayMs?: number; upstreamStatus?: number } = {}): Promise { const dir = mkdtempSync(join(testRoot, "run-")); for (const key of ["XDG_STATE_HOME", "XDG_DATA_HOME", "XDG_CACHE_HOME", "XDG_CONFIG_HOME"]) process.env[key] = dir; process.env.BILI_STORAGE_UNIFIED = storageMode === "full" ? "full" : "legacy"; @@ -57,7 +57,7 @@ async function harness(storageMode: "full" | "legacy", opts: { upstreamDelayMs?: req.on("end", () => { forwarded.push(JSON.parse(Buffer.concat(chunks).toString("utf8")) as Record); const reply = (): void => { - res.writeHead(200, { "content-type": "application/json" }); + res.writeHead(opts.upstreamStatus ?? 200, { "content-type": "application/json" }); res.end(JSON.stringify({ id: "msg_test", role: "assistant", content: [{ type: "text", text: "answer" }], usage: { input_tokens: 10000, output_tokens: 10 } })); }; if (opts.upstreamDelayMs) setTimeout(reply, opts.upstreamDelayMs); @@ -134,6 +134,19 @@ test("incremental lane: seed → delta → upstream receives the assembled full } }); +test("incremental lane: a failed turn (non-2xx) carries NO x-bili-applied-seq (#3344)", async () => { + const h = await harness("full", { upstreamStatus: 500 }); + try { + const failed = await h.turn({ sinceSeq: 0, messages: [userTurn(1)] }); + assert.ok(failed.status >= 400, `expected a non-2xx failure from the failing upstream, got ${failed.status}`); + // The ack must not ride a failure: a host trusting it unconditionally + // would skip its resend and lose the user's message (#3344). + assert.equal(failed.headers.get("x-bili-applied-seq"), null); + } finally { + await h.close(); + } +}); + test("incremental lane: seq mismatch answers 409 with the server's seq; resync recovers", async () => { const h = await harness("full"); try { diff --git a/tests/issue1192-disabled-tools.test.ts b/tests/issue1192-disabled-tools.test.ts index e7c38afe3c..d1c2057c01 100644 --- a/tests/issue1192-disabled-tools.test.ts +++ b/tests/issue1192-disabled-tools.test.ts @@ -1,7 +1,7 @@ // #1192: dsh registered the manifest's tools verbatim, so it called acp_rule // even though compress.rules was disabled — every call died with a hard // 400 "unknown tool". The manifest side (only advertise enabled opt-in tools) -// is covered by the plugin-protocol / rules suites; this pins the +// is covered by the plugin-protocol / rules / absorb suites; this pins the // execution side: a known-but-disabled opt-in tool answers with a model-facing // explanation (ok:true), while a genuinely unknown tool still 400s. Since // #1399 the feature is on by default, so "disabled" here means an explicit @@ -56,6 +56,13 @@ describe("#1192: known-but-disabled opt-in tools get an explanation, not a hard assert.match(String(r.json.result), /acp_rule is not enabled on this bili proxy \(compress\.rules\.enabled is not true\)/); }); + it("absorb disabled → ok:true explanation", async () => { + const r = await call("absorb", { ref: "m00001", summary: "x" }); + assert.equal(r.status, 200); + assert.equal(r.json.ok, true); + assert.match(String(r.json.result), /absorb is not enabled on this bili proxy \(compress\.absorb\.enabled is not true\)/); + }); + it("truly unknown tool keeps the 400 with the allowed list", async () => { const r = await call("definitely_not_a_tool"); assert.equal(r.status, 400); diff --git a/tests/issue1841-preflight-futility.test.ts b/tests/issue1841-preflight-futility.test.ts index f0207c9e9b..0e5536c154 100644 --- a/tests/issue1841-preflight-futility.test.ts +++ b/tests/issue1841-preflight-futility.test.ts @@ -167,7 +167,7 @@ test("#1841 A: a payload whose foldable pool cannot close the gap skips the roun ]; const resp = await post(p, "issue1841-futility-skip", messages); const body = await resp.text(); - assert.equal(resp.status, 502, `a futile round must fail fast: HTTP ${resp.status} ${body.slice(0, 240)}`); + assert.equal(resp.status, 413, `a futile round must fail fast non-retryably (#3461): HTTP ${resp.status} ${body.slice(0, 240)}`); const json = JSON.parse(body) as { error?: { code?: string; message?: string } }; assert.equal(json.error?.code, "preflight_compress_failed"); assert.match(json.error?.message ?? "", /futile round/, `names the futility verdict (got: ${json.error?.message})`); @@ -246,6 +246,9 @@ test("#1841 B: mid-walk bail stops before summarizing ranges that cannot change ]; const resp = await post(p, "issue1841-futility-midbail", messages); const body = await resp.text(); + // Relay-dependent by fixture design (every summary comes back empty): + // a healthy relay could still fold the drawn range, so this verdict + // stays the retryable 502 — unlike case A's zero-call structural 413. assert.equal(resp.status, 502, `a bailed walk must fail fast: HTTP ${resp.status} ${body.slice(0, 240)}`); const json = JSON.parse(body) as { error?: { code?: string; message?: string } }; assert.equal(json.error?.code, "preflight_compress_failed"); diff --git a/tests/issue2155-self-heal.test.ts b/tests/issue2155-self-heal.test.ts index b1e4bb3a9a..f7e09b3ece 100644 --- a/tests/issue2155-self-heal.test.ts +++ b/tests/issue2155-self-heal.test.ts @@ -299,20 +299,20 @@ test("integration: nudge-idle loop (non-plugin) suppresses the nudge after the w }); try { const body = { model: "m", max_tokens: 20_000, messages: turn2Compressible() }; + const len = () => ((received[received.length - 1]!.messages as unknown[]) ?? []).length; await send({}, body); // r1 baseline — usage lands only after its own response (no nudge yet, no idle credit) - for (let i = 0; i < SELF_HEAL_WINDOW - 2; i++) await send({}, body); // r2..r4 — nudged, idle credit 3 - await send({}, body); // r5 — nudged, idle credit 4 - const before = ((received[received.length - 1]!.messages as unknown[]) ?? []).length; // nudged count + const base = len(); + await send({}, body); // r2 — first pressure round: the backoff gate opens once, nudged (idle credit 1) + assert.equal(len(), base + 1, "the first over-limit round still carries the nudge"); + for (let i = 0; i < SELF_HEAL_WINDOW - 2; i++) await send({}, body); // r3..r5 — #3461 backoff holds: no repeat injection, but the gated pressure rounds still earn idle credit (2..4) + assert.equal(len(), base, "#3461: no storm — the held rounds carry no nudge"); const sh = listSessions().find((x) => x.id === sessionId)!.metadata.selfHeal; - assert.equal(sh, undefined, "not yet armed at window-1 idle rounds"); - await send({}, body); // r6 — the 5th nudged idle round arms AFTER its own nudge went out + assert.equal(sh, undefined, "not yet armed at window-1 idle credits"); + await send({}, body); // r6 — the 5th credited pressure round arms suppression const armed = listSessions().find((x) => x.id === sessionId)!.metadata.selfHeal as { action: string }; assert.equal(armed.action, "suppress-nudge"); await send({}, body); // r7 runs suppressed - const suppressed = ((received[received.length - 1]!.messages as unknown[]) ?? []).length; - const nudged = ((received[received.length - 2]!.messages as unknown[]) ?? []).length; - assert.ok(nudged === before + 1 - 1, "the arming round was still nudged (baseline round carried no nudge)"); - assert.ok(suppressed === before - 1, "nudge message gone once suppression armed (back to the un-nudged count)"); + assert.equal(len(), base, "suppression keeps the nudge gone"); } finally { proxy.close(); proxy.closeIdleConnections?.(); diff --git a/tests/issue2204-decompress-to-file-outcome.test.ts b/tests/issue2204-decompress-to-file-outcome.test.ts index 5ea6f2c8e8..f9fdd879cd 100644 --- a/tests/issue2204-decompress-to-file-outcome.test.ts +++ b/tests/issue2204-decompress-to-file-outcome.test.ts @@ -68,7 +68,7 @@ function foldCcr() { const config = applyCompressSettings( defaultConfig(200_000), 200_000, - { ccr: { enabled: true, minToolTokens: 50 } }, + { absorb: { enabled: true, minToolTokens: 50 }, ccr: { enabled: true, minToolTokens: 50 } }, ) as Config; const session = getSession(`tfr2204-${Math.random().toString(36).slice(2)}`); storeEffectiveCcr(session, { enabled: true, minToolTokens: 50 }); diff --git a/tests/issue2861-host-plugin-notice.test.ts b/tests/issue2861-host-plugin-notice.test.ts index 6173b429b2..10eda6ff78 100644 --- a/tests/issue2861-host-plugin-notice.test.ts +++ b/tests/issue2861-host-plugin-notice.test.ts @@ -214,7 +214,7 @@ test("#2861: wiring drift guards (seam export, banner slot, i18n refs, payload f for (const k of ['t("hostplug.on")', 't("hostplug.desc")', 't("hostplug.hint")', 't("hostplug.dismiss_btn")', 't("hostplug.dismissed")']) { assert.ok(WEB_CLIENT.includes(k), `i18n key ${k} referenced literally (dead-key lint)`); } - assert.ok(WEB_CLIENT.includes("/__bili/host-plugins/dismiss"), "dismiss POST target present"); + assert.ok(WEB_CLIENT.includes('API_ROOT + "/host-plugins/dismiss"'), "dismiss POST target present"); const here = dirname(fileURLToPath(import.meta.url)); const adminSrc = fs.readFileSync(join(here, "..", "src", "server", "admin.ts"), "utf8"); assert.ok(adminSrc.includes("hostPluginNotices: visibleHostPluginNotices(process.env)"), "overview payload carries the field"); diff --git a/tests/issue3148-upstream-circuit.test.ts b/tests/issue3148-upstream-circuit.test.ts index 9fd79d8869..edc145beef 100644 --- a/tests/issue3148-upstream-circuit.test.ts +++ b/tests/issue3148-upstream-circuit.test.ts @@ -124,15 +124,14 @@ test("#3148 review: an UNCOUNTED probe failure (timeout/client abort) must not p }); test("#3148: clampOutgoingOutput records the exact clamp event without changing the wire decision", () => { - const rebuilt: Record = { max_tokens: 65536 }; + const rebuilt: Record = { max_tokens: 300_000 }; let seen: { raw: number; capped: number; window: number; planningInput: number; ledgerFloor: number } | undefined; - // Reproduces the reporter's exact numbers: floor 997348, window 1048576 → - // margin ceil(997348*0.05)=49868, cap floor(1048576-997348-49868)=1360 — - // then #3071's capability floor lifts the cap to max(1360, 4096)=4096 - // (bounded by raw headroom floor(1048576-997348)=51228, so input+output - // <= window still holds). Pinned post-#3071; this expectation went stale - // when the two PRs landed in the same window — master push CI skips the - // test matrix, so the mismatch surfaced only on the next PR. + // The reporter's numbers (floor 997348, window 1048576 -> margin + // ceil(997348*0.05)=49868, cap floor(1048576-997348-49868)=1360, lifted to + // 4096 by #3071's capability floor bounded by raw headroom 51228) are kept; + // #3384 re-gated the floor, so the request is raised to 300K >= + // 0.25*1048576=262144 to take the giant-request ground and keep the hook + // path covered. clampOutgoingOutput( rebuilt, "max_tokens", @@ -150,9 +149,35 @@ test("#3148: clampOutgoingOutput records the exact clamp event without changing () => {}, (info) => { seen = info; }, ); - assert.deepEqual(seen, { raw: 65536, capped: 4096, window: 1048576, planningInput: 997348, ledgerFloor: 997348 }, "exact clamp event captured"); + assert.deepEqual(seen, { raw: 300_000, capped: 4096, window: 1048576, planningInput: 997348, ledgerFloor: 997348 }, "exact clamp event captured"); assert.equal(rebuilt.max_tokens, 4096, "wire decision unchanged (same cap as before the hook)"); + // #3384: the OLD expectation here (modest 65536 request against this same + // floor clamps to 4096) IS the starvation regression — below both gate + // lines the floor must not drive planning, or a healthy session (folded + // wire ~9% of the window) loses its output budget to bytes never sent. + const modest: Record = { max_tokens: 65_536 }; + let modestCalls = 0; + clampOutgoingOutput( + modest, + "max_tokens", + { + systemText: "", + tools: [], + processedMessages: [], + lastInputTokens: 97730, + lastInputTokensSource: "usage", + nativeWindow: 1048576, + imageTokens: 0, + ledgerFloorTokens: 997348, + }, + "t", + () => {}, + () => { modestCalls++; }, + ); + assert.equal(modestCalls, 0, "#3384: no clamp event — the floor is gated off for a modest request"); + assert.equal(modest.max_tokens, 65_536, "#3384: budget untouched while the folded view is healthy"); + const healthy: Record = { max_tokens: 65536 }; let healthyCalls = 0; clampOutgoingOutput( diff --git a/tests/issue3354-dsh-compaction-disable.test.ts b/tests/issue3354-dsh-compaction-disable.test.ts new file mode 100644 index 0000000000..db1e783886 --- /dev/null +++ b/tests/issue3354-dsh-compaction-disable.test.ts @@ -0,0 +1,137 @@ +import test from "node:test"; +import assert from "node:assert/strict"; +import fs from "node:fs"; +import os from "node:os"; +import path from "node:path"; +import { fileURLToPath } from "node:url"; +import { BILI_COMPACTION_DISABLE_SERVICE, compactionDisablePatchRows, disableBasicCompaction, registerCompactionDisable } from "../src/agent/dsh-compaction-disable.ts"; +import { writeDshAcpPatch } from "../src/launcher.ts"; + +// #3354: the vendored dsh basic-compaction disable overlay (deshark42/ +// dsh-compaction-basic-disable, MIT) — pure-function behavior, registration +// degradation, and the two patch carriers' agreement on the preset gate rows. +// The apply-level wiring is covered in tests/dsh-native.test.ts. + +const TESTS_DIR = path.dirname(fileURLToPath(import.meta.url)); +const PRESET_ROWS = compactionDisablePatchRows(); + +test("#3354 disableBasicCompaction: marks both compaction rows at any depth", () => { + const rows = [ + { name: "@deepseek-ai/dsh-compaction-basic", config: { auto: true } }, + { name: "@deepseek-ai/dsh-command-compact" }, + { name: "@deepseek-ai/dsh-other" }, + { + id: "compaction", + name: "cordis:group", + group: true, + config: [ + { name: "@deepseek-ai/dsh-compaction-basic" }, + { name: "@deepseek-ai/dsh-command-compact" }, + { group: true, config: [{ name: "@deepseek-ai/dsh-compaction-basic" }] }, + ], + }, + ]; + const out = disableBasicCompaction(rows) as Array>; + assert.equal(out[0].disabled, true); + assert.equal(out[1].disabled, true); + assert.equal(out[2].disabled, undefined); + const group = out[3] as Record; + assert.equal(group.group, true); + const inner = group.config as Array>; + assert.equal(inner[0].disabled, true); + assert.equal(inner[1].disabled, true); + assert.equal((inner[2].config as Array>)[0].disabled, true); +}); + +test("#3354 disableBasicCompaction: pure — input untouched, output detached", () => { + const row: Record = { name: "@deepseek-ai/dsh-compaction-basic", config: { auto: true } }; + const group = { group: true, config: [{ name: "@deepseek-ai/dsh-command-compact" }] }; + const input = [row, group]; + const snapshot = JSON.stringify(input); + const out = disableBasicCompaction(input) as Array>; + assert.equal(JSON.stringify(input), snapshot); + assert.notEqual(out, input); + assert.notEqual(out[0], row); + assert.notEqual(out[1], group); + assert.deepEqual((out[0] as Record).config, { auto: true }); + // mutating the output must not leak back into the input + (out[0] as Record).x = 1; + assert.equal(row.x, undefined); +}); + +test("#3354 disableBasicCompaction: degenerate inputs pass through", () => { + assert.deepEqual(disableBasicCompaction([]), []); + const mixed: unknown[] = [null, 42, "s", { name: "other" }, { group: true, config: { not: "an array" } }]; + const out = disableBasicCompaction(mixed) as Array; + assert.deepEqual(out, mixed); + // an already-disabled row stays disabled (idempotent overlay) + const done = disableBasicCompaction([{ name: "@deepseek-ai/dsh-compaction-basic", disabled: true }]) as Array>; + assert.equal(done[0].disabled, true); +}); + +test("#3354 registerCompactionDisable: degrades per layer without throwing", () => { + // no host surface at all — pre-hook builds / stripped hosts + registerCompactionDisable({}); + // registry only, no event bus + const provided: string[] = []; + registerCompactionDisable({ reflect: { provide: (name) => provided.push(name) } }); + assert.deepEqual(provided, [BILI_COMPACTION_DISABLE_SERVICE]); + // bus only, no registry — the middleware still registers + let seen: unknown[] = []; + registerCompactionDisable({ on: (_event, listener) => { seen.push(listener); } }); + assert.equal(seen.length, 1); +}); + +test("#3354 registerCompactionDisable: transform honors the entry predicate and preserves siblings", () => { + type Mw = (raw: unknown, next: () => unknown) => unknown; + let mw: Mw | undefined; + let options: unknown; + registerCompactionDisable({ + on: (_event, listener, opts) => { mw = listener as Mw; options = opts; }, + reflect: {}, + }); + assert.equal(typeof mw, "function"); + assert.deepEqual(options, { global: true }); + const plugins = [ + { name: "@deepseek-ai/dsh-compaction-basic", config: { auto: true } }, + { name: "@deepseek-ai/dsh-command-compact" }, + { name: "@deepseek-ai/dsh-tool-result-pruner" }, + ]; + const cfg: Record = { plugins, isolate: { compaction: true } }; + const self = { entry: { options: { name: "@deepseek-ai/dsh-agent-preset" } } }; + const out = mw?.call(self, null, () => cfg) as { plugins?: Array>; isolate?: Record }; + assert.deepEqual(out.isolate, { compaction: true }); + assert.equal(out.plugins?.[0]?.disabled, true); + assert.equal(out.plugins?.[1]?.disabled, true); + assert.equal(out.plugins?.[2]?.disabled, undefined); + // a different entry (or a host that passes none) must not be transformed + const other: Record = { plugins }; + assert.equal(mw?.call({ entry: { options: { name: "other-entry" } } }, null, () => other), other); + assert.equal(mw?.call(undefined, null, () => other), other); + // next() returning nothing → verbatim passthrough + assert.equal(mw?.call(self, null, () => undefined), undefined); +}); + +test("#3354 patch carriers agree: root bundle yml and launcher overlay carry the identical preset gate rows", () => { + // #2631: normalize CRLF checkouts before the byte-exact includes + const bundle = fs.readFileSync(path.join(TESTS_DIR, "..", "dsh.bundle.patch.yml"), "utf8").replaceAll("\r\n", "\n"); + for (const id of ["preset-standard", "preset-ptc", "preset-minimal", "preset-cordis"]) { + const row = `- id: ${id}\n inject: [${BILI_COMPACTION_DISABLE_SERVICE}]\n`; + assert.ok(bundle.includes(row), `bundle yml must carry the ${id} gate row`); + } + assert.ok(bundle.includes(PRESET_ROWS), "bundle yml must carry the gate rows contiguously"); + + const home = fs.mkdtempSync(path.join(os.tmpdir(), "bili-dsh-cd-patch-")); + try { + const file = writeDshAcpPatch(home, "billion-context"); + assert.ok(file, "overlay patch must be written"); + const overlay = fs.readFileSync(file!, "utf8").replaceAll("\r\n", "\n"); + assert.ok(overlay.includes(PRESET_ROWS), "launcher overlay must carry the same gate rows"); + // the overlay still stands down the host plane and keeps the #2605 retries + assert.ok(overlay.includes("- id: compaction-basic\n config:\n auto: false\n")); + assert.match(overlay, /retryableCodes: \[EMPTY_RESPONSE, RATE_LIMIT, SERVER, TIMEOUT, TRANSPORT, MALFORMED_RESPONSE\]/); + } finally { + fs.rmSync(home, { recursive: true, force: true }); + fs.rmSync(`${home}-bili`, { recursive: true, force: true }); + } +}); diff --git a/tests/issue3380-dsh-rereg-selflock.test.ts b/tests/issue3380-dsh-rereg-selflock.test.ts new file mode 100644 index 0000000000..4f956e3d5b --- /dev/null +++ b/tests/issue3380-dsh-rereg-selflock.test.ts @@ -0,0 +1,395 @@ +// #3380: plugin tool registration state self-lock. +// +// Incident (bili v0.1.197, DSH desktop): a healthy session lost its proxy at +// runtime; the respawn gave up once (onGiveUp), which cleared BOTH +// register.base AND register.toolsReady. Tools had been registered into the +// host process's tool layer hours earlier and survived the proxy death — so +// every later re-registration attempt threw DSH's duplicate error +// ('tool "compress" is already registered'), the retry loop burned the same +// doomed failure 145×/h on a flat 10s wall, toolsReady stayed false forever, +// headersFor stopped stamping x-bili-plugin*, and every model request rode +// bili anonymously (wire mode → zero-fold pfa-* ledgers → 413 +// preflight_compress_failed on large sessions) until the host restarted. +// +// Root cause: conflation of two lifecycles — tool registrations belong to the +// HOST CONTEXT (they die only with their context; execute closures read the +// live base, #983), while the proxy-death paths treated them as proxy-owned. +// Fix under test: +// 1. proxy-death paths no longer clear toolsReady (identity survives a +// respawn; NO doomed re-registration ever runs again); +// 2. registration is idempotent — a name this process already holds is +// adopted, not thrown on (converges partial/dirty states); +// 3. consecutive failures escalate: capped exponential back-off + one loud +// notice with the recovery action (retries never stop — #983/#2082); +// 4. /acp exposes wire-mode identity loss instead of staying silent. + +import test, { afterEach } from "node:test"; +import assert from "node:assert/strict"; +import fs from "node:fs"; +import os from "node:os"; +import path from "node:path"; +import http from "node:http"; +import { + apply, + maybeRetryForTest, + _resetRegisterForTest, + _setSpawnForTest, + _settleNativeForTest, + _stateHeadersForTest, + _stateOnGiveUpForTest, + _registerStateForTest, + _regBackoffForTest, + type PluginContext, +} from "../src/agent/dsh-native.ts"; +import { rmrf } from "./tmp-rm.ts"; + +// The node:test runner sets NODE_TEST_CONTEXT itself; set defensively so a +// standalone file run behaves identically (dsh-native reads it lazily). +process.env.NODE_TEST_CONTEXT = process.env.NODE_TEST_CONTEXT ?? "1"; +process.env.BILI_PERSIST = "0"; + +afterEach(() => { void _settleNativeForTest(); }); + +type RegisteredTool = { + name: string; + description?: string; + parameters: unknown; + output: { schema: unknown; render: (args: unknown, value: unknown) => Array<{ type: string; text: string }> }; + execute: (args: Record, exec: { agent?: { session?: { id?: unknown } }; signal?: AbortSignal }) => Promise; +}; +type CommandSpec = { + name: string; + description?: string; + handler: (invocation?: { agent?: { session?: { id?: unknown } } }) => Promise<{ kind: string; text: string }>; +}; + +// DSH's root tool layer: ONE registry per host process. Re-registering a name +// this process already holds throws the verbatim incident error — this is +// exactly what turned #3380's post-respawn re-registration into a self-lock. +const rootLayer = new Map(); +const DSH_DUP_ERROR = (name: string): Error => + new Error(`tool "${name}" is already registered (for a per-agent variant, register through that agent's \`agent.ctx\` instead)`); + +function resetRootLayer(): void { + rootLayer.clear(); +} + +async function startMockProxy(toolNames: string[]): Promise<{ origin: string; close: () => void; manifestRequests: () => number }> { + const counters = { manifestRequests: 0 }; + const server = http.createServer((req, res) => { + const url = req.url ?? ""; + if (url === "/__bili/plugin/manifest") { + counters.manifestRequests += 1; + res.writeHead(200, { "content-type": "application/json" }); + res.end(JSON.stringify({ + version: "0.1.119", + tools: { anthropic: toolNames.map((name) => ({ name, description: `${name} (test)`, input_schema: { type: "object", properties: {} } })) }, + })); + return; + } + res.writeHead(404); + res.end("{}"); + }); + await new Promise((resolve) => server.listen(0, "127.0.0.1", () => resolve())); + const addr = server.address() as { port: number }; + return { + origin: `http://127.0.0.1:${addr.port}`, + close: () => { server.close(); server.closeAllConnections?.(); }, + manifestRequests: () => counters.manifestRequests, + }; +} + +type HostCtx = Parameters[0]; + +function mockCtx(opts: { registerError?: () => Error | undefined } = {}) { + const tools: RegisteredTool[] = []; + const commands: CommandSpec[] = []; + let initiator: { session?: { id?: unknown } } | undefined = undefined; + return { + tools: { + register: (t: RegisteredTool) => { + const err = opts.registerError?.(); + if (err !== undefined) throw err; + if (rootLayer.has(t.name)) throw DSH_DUP_ERROR(t.name); + rootLayer.set(t.name, true); + tools.push(t); + }, + }, + commands: { register: (c: CommandSpec) => commands.push(c) }, + agents: { currentInitiator: () => initiator }, + setInitiator: (i: { session?: { id?: unknown } } | undefined) => (initiator = i), + registeredTools: tools, + registeredCommands: commands, + inject: (_deps: readonly string[], _cb: (sub: HostCtx) => void) => {}, + }; +} + +function mockCtxTyped(ctx: ReturnType): PluginContext { + return ctx as unknown as PluginContext; +} + +async function withEnv(env: Record, fn: () => Promise | T): Promise { + const saved: Record = {}; + for (const k of Object.keys(env)) { + saved[k] = process.env[k]; + const v = env[k]; + if (v === undefined) delete process.env[k]; + else process.env[k] = v; + } + try { + return await fn(); + } finally { + for (const [k, v] of Object.entries(saved)) { + if (v === undefined) delete process.env[k]; + else process.env[k] = v; + } + } +} + +async function waitFor(cond: () => boolean, what: string): Promise { + const deadline = Date.now() + 15000; + while (!cond()) { + if (Date.now() > deadline) throw new Error(`timeout waiting for ${what}`); + await new Promise((r) => setTimeout(r, 10)); + } +} + +function logFileOf(home: string): string { + return path.join(home, "billion-context", "bili.log"); +} + +test("#3380 incident repro: identity survives proxy death + respawn — no doomed re-registration ever runs", async () => { + const proxyA = await startMockProxy(["compress", "decompress"]); + const proxyB = await startMockProxy(["compress", "decompress"]); + const home = fs.mkdtempSync(path.join(os.tmpdir(), "bili-dsh-3380-incident-")); + const stateHome = fs.mkdtempSync(path.join(os.tmpdir(), "bili-dsh-3380-incident-state-")); + try { + // grace window shrunk so the dead-attach probe falls through to the + // spawn seam quickly instead of waiting the production evidence grace + await withEnv({ DSH_HOME: home, BILLION_CONTEXT_PROXY: proxyA.origin, BILI_ATTACH_EVIDENCE_GRACE_MS: "50", XDG_STATE_HOME: stateHome }, async () => { + _resetRegisterForTest(undefined); + resetRootLayer(); + const raw = mockCtx(); + const ctx = mockCtxTyped(raw); + raw.setInitiator({ session: { id: "i3380" } }); + apply(ctx); + await waitFor(() => rootLayer.size === 2, "initial registration"); + const h1 = _stateHeadersForTest()?.("http://example.test/v1/messages"); + assert.equal(h1?.["x-bili-plugin"], "dsh", "phase 1: identity stamped"); + assert.equal(h1?.["x-bili-plugin-conversation"], "i3380"); + + // Phase 2: the proxy dies and its respawn gives up. Old code + // cleared toolsReady here — the self-lock's ignition. + proxyA.close(); + const giveUp = _stateOnGiveUpForTest(); + assert.ok(giveUp !== undefined, "give-up seam armed by the attach plan"); + giveUp!(); + await _settleNativeForTest(); + assert.equal(_registerStateForTest().base, undefined, "the origin is gone"); + assert.equal(_registerStateForTest().toolsReady, true, "the latch SURVIVES proxy death (registrations belong to the host context)"); + + // Phase 3: a NEW origin lands through the respawn seam. + _setSpawnForTest(async () => proxyB.origin); + try { + maybeRetryForTest(ctx); + await waitFor(() => _registerStateForTest().base === proxyB.origin, "respawn landing on the new origin"); + assert.equal(proxyB.manifestRequests(), 0, "NO doomed re-registration flight ran against the new proxy"); + assert.equal(rootLayer.size, 2, "no duplicate registration reached the host tool layer"); + const h2 = _stateHeadersForTest()?.("http://example.test/v1/messages"); + assert.equal(h2?.["x-bili-plugin"], "dsh", "identity restored on the new origin without re-registration"); + assert.equal(h2?.["x-bili-plugin-conversation"], "i3380", "same conversation identity carried across the respawn"); + } finally { + _setSpawnForTest(undefined); + } + }); + } finally { + proxyA.close(); + proxyB.close(); + rmrf(home); + rmrf(stateHome); + _resetRegisterForTest(undefined); + resetRootLayer(); + } +}); + +test("#3380 idempotent adoption: a pre-existing registration is adopted, the rest still lands", async () => { + const proxy = await startMockProxy(["compress", "decompress"]); + const home = fs.mkdtempSync(path.join(os.tmpdir(), "bili-dsh-3380-adopt-")); + const stateHome = fs.mkdtempSync(path.join(os.tmpdir(), "bili-dsh-3380-adopt-state-")); + try { + await withEnv({ DSH_HOME: home, BILLION_CONTEXT_PROXY: proxy.origin, XDG_STATE_HOME: stateHome }, async () => { + _resetRegisterForTest(undefined); + resetRootLayer(); + // Dirty state: an earlier interrupted flight left "compress" held + // in the process's tool layer. + rootLayer.set("compress", true); + const raw = mockCtx(); + const ctx = mockCtxTyped(raw); + raw.setInitiator({ session: { id: "i3380-adopt" } }); + apply(ctx); + await waitFor(() => _registerStateForTest().toolsReady, "registration converges despite the held name"); + assert.ok(rootLayer.has("compress") && rootLayer.has("decompress"), "both tools present in the host layer"); + assert.equal(raw.registeredTools.length, 1, "only the missing tool was registered fresh"); + const h = _stateHeadersForTest()?.("http://example.test/v1/messages"); + assert.equal(h?.["x-bili-plugin"], "dsh", "plugin mode restored after adoption"); + await waitFor(() => fs.existsSync(logFileOf(stateHome)) && fs.readFileSync(logFileOf(stateHome), "utf8").includes("adopted 1 pre-existing registration"), "adoption convergence logged once"); + }); + } finally { + proxy.close(); + rmrf(home); + rmrf(stateHome); + _resetRegisterForTest(undefined); + resetRootLayer(); + } +}); + +test("#3380 P2 back-off schedule: capped exponential, streak resets on success", async () => { + const saved = process.env.BILI_DSH_RETRY_INTERVAL_MS; + process.env.BILI_DSH_RETRY_INTERVAL_MS = "20"; + try { + assert.equal(_regBackoffForTest(1), 20, "first failure: base interval"); + assert.equal(_regBackoffForTest(2), 40, "second failure: doubled"); + assert.equal(_regBackoffForTest(3), 80, "third failure: doubled again"); + const plateau = _regBackoffForTest(100); + assert.equal(plateau, 5 * 60 * 1000, "hard cap at 5 minutes"); + assert.equal(_regBackoffForTest(101), plateau, "stays capped"); + } finally { + if (saved === undefined) delete process.env.BILI_DSH_RETRY_INTERVAL_MS; + else process.env.BILI_DSH_RETRY_INTERVAL_MS = saved; + } +}); + +test("#3380 P2 integration: failures escalate on the ladder, then heal clean", async () => { + const proxy = await startMockProxy(["compress"]); + const home = fs.mkdtempSync(path.join(os.tmpdir(), "bili-dsh-3380-backoff-")); + const stateHome = fs.mkdtempSync(path.join(os.tmpdir(), "bili-dsh-3380-backoff-state-")); + const savedInterval = process.env.BILI_DSH_RECOVERY_INTERVAL_MS; + const savedRetry = process.env.BILI_DSH_RETRY_INTERVAL_MS; + process.env.BILI_DSH_RECOVERY_INTERVAL_MS = "20"; + process.env.BILI_DSH_RETRY_INTERVAL_MS = "20"; + try { + await withEnv({ DSH_HOME: home, BILLION_CONTEXT_PROXY: proxy.origin, XDG_STATE_HOME: stateHome }, async () => { + _resetRegisterForTest(undefined); + resetRootLayer(); + let failing = true; + const raw = mockCtx({ registerError: () => (failing ? new Error("manifest down (transient)") : undefined) }); + const ctx = mockCtxTyped(raw); + apply(ctx); + // Third consecutive failure arms the third ladder step (4×base). + await waitFor(() => _registerStateForTest().failStreak >= 3, "three consecutive failures"); + const snap = _registerStateForTest(); + assert.equal(snap.lastRegistrationError, "manifest down (transient)", "last error recorded for status surfaces"); + const delta = snap.retryAt - Date.now(); + assert.ok(delta >= 30 && delta <= 95, `retryAt sits on the 80ms ladder step (got ${delta}ms)`); + // Heal: the transient fault clears, the next attempt succeeds, and + // the bookkeeping resets. + failing = false; + await waitFor(() => _registerStateForTest().toolsReady, "heal after the fault clears"); + const healed = _registerStateForTest(); + assert.equal(healed.failStreak, 0, "streak resets on success"); + assert.equal(healed.lastRegistrationError, undefined, "stale error cleared"); + }); + } finally { + for (const [k, v] of [["BILI_DSH_RECOVERY_INTERVAL_MS", savedInterval], ["BILI_DSH_RETRY_INTERVAL_MS", savedRetry]] as const) { + if (v === undefined) delete process.env[k]; + else process.env[k] = v; + } + proxy.close(); + rmrf(home); + rmrf(stateHome); + _resetRegisterForTest(undefined); + resetRootLayer(); + } +}); + +test("#3380 P2 notice: exactly one loud restart hint across a long failure streak", async () => { + const proxy = await startMockProxy(["compress"]); + const home = fs.mkdtempSync(path.join(os.tmpdir(), "bili-dsh-3380-notice-")); + const stateHome = fs.mkdtempSync(path.join(os.tmpdir(), "bili-dsh-3380-notice-state-")); + const savedInterval = process.env.BILI_DSH_RECOVERY_INTERVAL_MS; + const savedRetry = process.env.BILI_DSH_RETRY_INTERVAL_MS; + process.env.BILI_DSH_RECOVERY_INTERVAL_MS = "20"; + process.env.BILI_DSH_RETRY_INTERVAL_MS = "20"; + try { + await withEnv({ DSH_HOME: home, BILLION_CONTEXT_PROXY: proxy.origin, XDG_STATE_HOME: stateHome }, async () => { + _resetRegisterForTest(undefined); + resetRootLayer(); + const raw = mockCtx({ registerError: () => new Error("manifest down (persistent)") }); + apply(mockCtxTyped(raw)); + const MARKER = "bili tools still unavailable after"; + await waitFor(() => { + if (_registerStateForTest().failStreak < 7) return false; + const log = fs.existsSync(logFileOf(stateHome)) ? fs.readFileSync(logFileOf(stateHome), "utf8") : ""; + return log.split(MARKER).length - 1 >= 1; + }, "streak past the notice threshold with the notice logged"); + const log = fs.readFileSync(logFileOf(stateHome), "utf8"); + assert.equal(log.split(MARKER).length - 1, 1, "the notice fires exactly once, not per attempt"); + assert.ok(log.includes("Restart dsh to restore"), "carries the recovery action"); + assert.ok(log.includes("manifest down (persistent)"), "carries the last error"); + }); + } finally { + for (const [k, v] of [["BILI_DSH_RECOVERY_INTERVAL_MS", savedInterval], ["BILI_DSH_RETRY_INTERVAL_MS", savedRetry]] as const) { + if (v === undefined) delete process.env[k]; + else process.env[k] = v; + } + proxy.close(); + rmrf(home); + rmrf(stateHome); + _resetRegisterForTest(undefined); + resetRootLayer(); + } +}); + +test("#3380 P3 /acp: wire-mode identity loss is exposed, not silent", async () => { + const proxy = await startMockProxy(["compress"]); + const home = fs.mkdtempSync(path.join(os.tmpdir(), "bili-dsh-3380-acp-")); + const stateHome = fs.mkdtempSync(path.join(os.tmpdir(), "bili-dsh-3380-acp-state-")); + const savedInterval = process.env.BILI_DSH_RECOVERY_INTERVAL_MS; + const savedRetry = process.env.BILI_DSH_RETRY_INTERVAL_MS; + process.env.BILI_DSH_RECOVERY_INTERVAL_MS = "20"; + process.env.BILI_DSH_RETRY_INTERVAL_MS = "20"; + try { + await withEnv({ DSH_HOME: home, BILLION_CONTEXT_PROXY: proxy.origin, XDG_STATE_HOME: stateHome }, async () => { + _resetRegisterForTest(undefined); + resetRootLayer(); + let failing = true; + const raw = mockCtx({ registerError: () => (failing ? new Error("manifest down (transient)") : undefined) }); + const ctx = mockCtxTyped(raw); + apply(ctx); + const acp = raw.registeredCommands.find((c) => c.name === "acp"); + assert.ok(acp !== undefined, "acp command registered"); + const invocation = { agent: { session: { id: "i3380-acp" } } }; + + // (a) proxy answers but tools are down → wire-mode warning. + await waitFor(() => _registerStateForTest().failStreak >= 1, "first registration failure"); + const wire = await acp!.handler(invocation); + assert.equal(wire.kind, "success"); + assert.ok(wire.text.includes("WITHOUT identity headers (wire mode"), "wire-mode loss named explicitly"); + assert.ok(wire.text.includes("manifest down (transient)"), "last error surfaced"); + assert.ok(wire.text.includes("Restart dsh to restore"), "recovery action surfaced"); + + // (b) heal, then lose the proxy entirely → recovering, not "install". + failing = false; + await waitFor(() => _registerStateForTest().toolsReady, "heal"); + proxy.close(); + const giveUp = _stateOnGiveUpForTest(); + assert.ok(giveUp !== undefined); + giveUp!(); + await _settleNativeForTest(); + const out = await acp!.handler(invocation); + assert.equal(out.kind, "error"); + assert.ok(out.text.includes("proxy temporarily unreachable"), "a previously-serving proxy reads as an outage with recovery running"); + assert.ok(!out.text.includes("no proxy detected — install"), "not misread as a missing install"); + }); + } finally { + for (const [k, v] of [["BILI_DSH_RECOVERY_INTERVAL_MS", savedInterval], ["BILI_DSH_RETRY_INTERVAL_MS", savedRetry]] as const) { + if (v === undefined) delete process.env[k]; + else process.env[k] = v; + } + proxy.close(); + rmrf(home); + rmrf(stateHome); + _resetRegisterForTest(undefined); + resetRootLayer(); + } +}); diff --git a/tests/issue3384-ledger-floor-starvation.test.ts b/tests/issue3384-ledger-floor-starvation.test.ts new file mode 100644 index 0000000000..e8bd7ecf4c --- /dev/null +++ b/tests/issue3384-ledger-floor-starvation.test.ts @@ -0,0 +1,275 @@ +// #3384: the #2490 refused-ledger planning floor was applied UNCONDITIONALLY — +// bili planned every outgoing output budget against the client's raw ledger +// even though its own folds kept the wire small. In the reported session +// (window 1M, DSH profile, default config) the ledger climbed 832K -> 947K +// while the folded wire sat at 8–11%, so the clamp shrank the requested +// budget 128000 -> 5909 over four turns; the model was onlyReasoning / +// canDisableThinking:false, so each clamped budget was eaten entirely by +// thinking -> upstream stopReason=length -> no text, no tool-call, blank GUI +// turns, and DSH (which does not auto-continue on max-tokens) just replayed +// the same dead loop on every "continue". Fix: the floor now engages only on +// one of two independent grounds — the folded view already sits at the +// enforced headroom target, or the requested budget is giant (>= 25% of the +// native window, the #2490 monster-request line). Both protection regimes +// stay pinned here AND in tests/issue2490-ledger-floor-doomed-probe.test.ts +// (giant request) — this file pins the starvation side they left open. +import assert from "node:assert/strict"; +import http from "node:http"; +import { once } from "node:events"; +import test from "node:test"; + +process.env.NODE_ENV = "test"; + +import { defaultConfig } from "acp-kernel"; +import { startServer } from "../src/server.ts"; +import type { ProxyOptions } from "../src/config.ts"; +import { SessionStore, _setStoreForTest } from "../src/persist.ts"; +import { _setForTest as setRegistryForTest } from "../src/registry.ts"; +import { getSession } from "../src/session.ts"; +import { clampOutgoingOutput } from "../src/server/budget.ts"; +import { DSH_COMPACTION_INSTRUCTION_PREFIX } from "../src/server/dsh-compaction-guard.ts"; +import { rmrf } from "./tmp-rm.ts"; +import { mkdtempSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import path from "node:path"; + +const WINDOW = 1_000_000; // the reported deployment's declared window +const FLOOR = 946_753; // the incident's final planning floor (refused replay bytes / 4) + +type CtxArgs = Parameters[2]; + +// Mirrors the incident log line "output budget clamped 50670 -> 5909 +// (input~89699.41..., planning floor 946753 ..., window=1000000)": folded +// wire ~89.7K (9% of the window), ledger floor 946.7K, modest request. +const incidentCtx = (over: Partial): CtxArgs => ({ + systemText: "", + tools: undefined, + processedMessages: [{ role: "user", text: "hello" } as never], + lastInputTokens: 89_699, + lastInputTokensSource: "usage", + nativeWindow: WINDOW, + headroomWindow: WINDOW, // real dsh deployments run headroom ~= declared window + imageTokens: 0, + ...over, +}) as CtxArgs; + +const clampLog = (lines: string[]): ((level: string, msg: string) => void) => + (level, msg) => { if (level === "info" || level === "warn") lines.push(msg); }; + +test("#3384: the incident shape keeps its full budget — modest request, healthy folded view, huge refused ledger", () => { + const lines: string[] = []; + const rebuilt: Record = { max_tokens: 50_670 }; + clampOutgoingOutput(rebuilt, "max_tokens", incidentCtx({ ledgerFloorTokens: FLOOR }), "s-3384-a", clampLog(lines)); + // Pre-fix this clamped to 5909 (margin ceil(946753*0.05)=47338, cap + // floor(1000000-946753-47338)=5909) — the starvation the session died in. + assert.equal(rebuilt.max_tokens, 50_670, "budget untouched below both gate lines"); + assert.equal(lines.length, 0, "no clamp, no log"); +}); + +test("#3384: the giant-request ground still plans against the ledger (#2490 regime preserved)", () => { + const lines: string[] = []; + const rebuilt: Record = { max_tokens: 384_000 }; + // 384K >= 0.25 * 1M — the #2490 incident's own request size. + clampOutgoingOutput(rebuilt, "max_tokens", incidentCtx({ ledgerFloorTokens: FLOOR }), "s-3384-b", clampLog(lines)); + assert.equal(rebuilt.max_tokens, 5_909, "planned against the refused ledger: cap floor(1M-946753-47338)=5909, above the 4096 capability floor"); + assert.match(lines[0] ?? "", /planning floor 946753 from the refused dsh compaction replay \(#2490\)/); +}); + +test("#3384: the boundary is inclusive — exactly 25% of the window takes the giant ground", () => { + const lines: string[] = []; + const rebuilt: Record = { max_tokens: 250_000 }; + clampOutgoingOutput(rebuilt, "max_tokens", incidentCtx({ ledgerFloorTokens: FLOOR }), "s-3384-c", clampLog(lines)); + assert.equal(rebuilt.max_tokens, 5_909, "raw == 0.25*window engages the floor (fail-closed side of the line)"); + assert.ok(lines.length >= 1, "the clamp is logged"); +}); + +test("#3384: one token under the giant line stays unclamped while the view is healthy", () => { + const lines: string[] = []; + const rebuilt: Record = { max_tokens: 249_999 }; + clampOutgoingOutput(rebuilt, "max_tokens", incidentCtx({ ledgerFloorTokens: FLOOR }), "s-3384-d", clampLog(lines)); + assert.equal(rebuilt.max_tokens, 249_999, "below the line the ledger must not drive planning"); + assert.equal(lines.length, 0); +}); + +test("#3384: the view-at-enforced-target ground engages when bili's OWN view reaches the wall", () => { + const lines: string[] = []; + const rebuilt: Record = { max_tokens: 50_670 }; + // Folded view 110K already past an enforced 100K headroom target — the + // session really is at the wall, so worst-case-ledger planning applies + // even for a modest request. + clampOutgoingOutput(rebuilt, "max_tokens", incidentCtx({ headroomWindow: 100_000, lastInputTokens: 110_000, ledgerFloorTokens: FLOOR }), "s-3384-e", clampLog(lines)); + assert.equal(rebuilt.max_tokens, 5_909, "view at the enforced target re-engages the floor"); + assert.match(lines[0] ?? "", /planning floor 946753/); +}); + +test("#3384: legacy paths stay byte-identical — no refusal recorded, or floor below the folded view", () => { + const legacy: Record = { max_tokens: 50_670 }; + clampOutgoingOutput(legacy, "max_tokens", incidentCtx({}), "s-3384-f", clampLog([])); + assert.equal(legacy.max_tokens, 50_670, "no floor recorded -> unchanged legacy behavior"); + + const small: Record = { max_tokens: 50_670 }; + clampOutgoingOutput(small, "max_tokens", incidentCtx({ ledgerFloorTokens: 10_000 }), "s-3384-g", clampLog([])); + assert.equal(small.max_tokens, 50_670, "a floor smaller than the folded view changes nothing"); +}); + +// --------------------------------------------------------------------------- +// e2e: refuse a compaction envelope (records the ledger high-water), then a +// normal turn with a modest budget must go out UNCLAMPED. Pre-fix the second +// turn's max_tokens came back ~8056 instead of 20000 (floor ~87.5K against a +// 100K window), which is exactly how the reported session starved. +// --------------------------------------------------------------------------- + +const E2E_WINDOW = 100_000; + +function chatSse(text: string, usage?: { prompt_tokens: number; completion_tokens: number }): string { + const chunk = (delta: Record, finish: string | null, extra?: Record): string => + `data: ${JSON.stringify({ id: "chatcmpl-1", object: "chat.completion.chunk", created: 1, model: "gpt-test", choices: [{ index: 0, delta, finish_reason: finish }], ...extra })}\n\n`; + const u = usage ? { usage } : {}; + return chunk({ role: "assistant" }, null) + chunk({ content: text }, null) + chunk({}, "stop", u) + "data: [DONE]\n\n"; +} + +type Call = { maxTokens: number | null; contentChars: number }; + +function makeUpstream(): { server: http.Server; calls: Call[] } { + const calls: Call[] = []; + const server = http.createServer((req, res) => { + const chunks: Buffer[] = []; + req.on("data", (c: Buffer) => chunks.push(c)); + req.on("end", () => { + const raw = Buffer.concat(chunks).toString("utf8"); + try { + const parsed = JSON.parse(raw) as { messages?: Array<{ content?: unknown }>; max_tokens?: number }; + const chars = (parsed.messages ?? []).reduce((n, m) => n + (typeof m.content === "string" ? m.content.length : 0), 0); + calls.push({ maxTokens: typeof parsed.max_tokens === "number" ? parsed.max_tokens : null, contentChars: chars }); + res.writeHead(200, { "content-type": "text/event-stream" }); + res.end(chatSse("forwarded answer", { prompt_tokens: Math.max(1, Math.round(chars / 40)), completion_tokens: 3 })); + } catch { + res.writeHead(400, { "content-type": "application/json" }); + res.end('{"error":{"message":"unparseable"}}'); + } + }); + }); + return { server, calls }; +} + +function proxyOptions(routes: Record): ProxyOptions { + return { + port: 0, + host: "127.0.0.1", + upstream: "http://127.0.0.1", + routes: routes as ProxyOptions["routes"], + modelContextLimit: E2E_WINDOW, + kernelConfig: defaultConfig(E2E_WINDOW, { preserveRecentMessages: 2, preserveRecentTokens: 2000 }), + compress: { injectTool: true, injectNudge: true }, + sessionHeader: "x-acp-session", + log: false, + debug: false, + passthrough: false, + autoUpdate: false, + compat: { roles: {} }, + streamErrorShape: "protocol", + passthroughSource: null, + autoRestartOnUpdate: false, + updateTag: "latest", + advisoryCheck: false, + releaseNotesCheck: false, + promptCache: { routing: "auto" }, + mitm: { enabled: false, domains: [] }, + } as ProxyOptions; +} + +type Ctx = { url: string; headers: Record }; + +async function setup(): Promise<{ ctx: Ctx; calls: Call[]; close: () => Promise }> { + _setStoreForTest(new SessionStore({ enabled: false })); + setRegistryForTest({}); + const { server: upstream, calls } = makeUpstream(); + upstream.listen(0, "127.0.0.1"); + await once(upstream, "listening"); + const upstreamPort = (upstream.address() as { port: number }).port; + const proxy = await startServer(proxyOptions({ [`http://127.0.0.1:${upstreamPort}`]: { models: { "gpt-test": { context: E2E_WINDOW } } } })); + await once(proxy, "listening"); + const proxyPort = (proxy.address() as { port: number }).port; + return { + ctx: { + url: `http://127.0.0.1:${proxyPort}/bili/http://127.0.0.1:${upstreamPort}/chat/completions`, + headers: { "content-type": "application/json", "x-acp-session": "" }, + }, + calls, + close: async () => { + proxy.close(); + proxy.closeIdleConnections?.(); + upstream.close(); + upstream.closeIdleConnections?.(); + await Promise.allSettled([once(proxy, "close"), once(upstream, "close")]); + }, + }; +} + +test("e2e #3384: after a refused compaction envelope, a modest-budget turn goes out unclamped", async () => { + const root = mkdtempSync(path.join(tmpdir(), "bili-3384-")); + const cfgFile = path.join(root, "billion-context.json"); + writeFileSync(cfgFile, '{"providers":{}}\n', "utf8"); + const prevConfig = process.env.BILI_CONFIG_FILE; + process.env.BILI_CONFIG_FILE = cfgFile; + const { ctx, calls, close } = await setup(); + try { + ctx.headers["x-acp-session"] = "s-3384-e2e"; + const postRaw = (body: string) => fetch(ctx.url, { method: "POST", headers: { ...ctx.headers }, body }); + + // Turn 1: seed the session with a small exchange. + let r = await postRaw(JSON.stringify({ model: "gpt-test", stream: true, messages: [{ role: "user", content: "hello" }] })); + assert.equal(r.status, 200, "seed turn accepted"); + await r.text(); + + // The dsh compaction envelope (~350KB of core text -> ledger floor + // ~87.5K tokens): the guard refuses it (default posture) and records + // the high-water byte count. The size is chosen so the pre-fix cap + // floor(100000 - 87.5K - ceil(87.5K*0.05)) = ~8056 sits BELOW the 20K + // request — this assertion only discriminates in that band. + const envelope = JSON.stringify({ + model: "gpt-test", + stream: true, + messages: [ + { role: "system", content: "sys" }, + { role: "user", content: `${DSH_COMPACTION_INSTRUCTION_PREFIX} ${"x".repeat(350_000)}` }, + ], + }); + r = await postRaw(envelope); + assert.equal(r.status, 422, "guard refuses the compaction call (default)"); + await r.text(); + const s = getSession("s-3384-e2e"); + assert.ok(s, "session exists"); + assert.equal(s.metadata.dshCompactionRefused, true, "refusal flag set"); + const floor = Math.ceil(Buffer.byteLength(envelope) / 4); + assert.equal(s.metadata.dshCompactionRefusedBytes, Buffer.byteLength(envelope), "high-water bytes recorded"); + assert.ok(floor > E2E_WINDOW * 0.25 && floor < E2E_WINDOW, `floor ${floor} sits in the discriminating band (25%, window) of a ${E2E_WINDOW} window`); + + // Turn 2: a normal short turn with a MODERATE budget (20K < 25K = the + // giant line). Pre-fix the planning basis became the ~87.5K floor and + // this went out clamped to ~8056 — the reported session's death spiral + // in miniature. Post-fix the wire is what it is (~a few hundred + // tokens), so the full 20K rides out. + r = await postRaw(JSON.stringify({ + model: "gpt-test", + stream: true, + max_tokens: 20_000, + messages: [ + { role: "user", content: "hello" }, + { role: "assistant", content: "forwarded answer" }, + { role: "user", content: "and now?" }, + ], + })); + assert.equal(r.status, 200, "normal turn accepted"); + await r.text(); + + const forwarded = calls[calls.length - 1]; + assert.ok(forwarded, "the normal turn reached the upstream"); + assert.equal(forwarded.maxTokens, 20_000, "#3384: the refused-ledger floor did not starve a modest budget on a healthy wire"); + } finally { + if (prevConfig === undefined) delete process.env.BILI_CONFIG_FILE; + else process.env.BILI_CONFIG_FILE = prevConfig; + await close(); + rmrf(root); + } +}); diff --git a/tests/issue3400-image-rescale.test.ts b/tests/issue3400-image-rescale.test.ts new file mode 100644 index 0000000000..e26528e0e3 --- /dev/null +++ b/tests/issue3400-image-rescale.test.ts @@ -0,0 +1,377 @@ +import { test } from "node:test"; +import assert from "node:assert/strict"; +import { mkdtempSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import path from "node:path"; +import { once } from "node:events"; +import { Jimp } from "jimp"; +import { type BiliMessage } from "acp-kernel/wire"; +import type { ProxyOptions } from "../src/config.js"; + +// #3400: egress image rescale (simplified re-do of #3295's removal). Contract: +// big images (>1600 longest edge or >100KB) are downscaled on the OUTBOUND +// copy only; originals stay in session storage; the model is told both sizes +// beside the image; read_original_image returns the exact stored bytes and +// its results are structurally exempt from re-shrinking (the #3266 loop guard). +process.env.BILI_CONFIG_FILE = "/nonexistent/bili-issue3400-config.json"; + +const { createSession, peekSession } = await import("../src/session.js"); +const { + applyImageRescalePass, applyImageRescaleItems, executeReadOriginalImage, + buildRescaleNote, clearRescaleEncodeCache, +} = await import("../src/image-rescale.js"); +const { storeEffectiveImageRescale, imageRescaleToolArmed } = await import("../src/store.js"); +const { defaultConfig } = await import("acp-kernel"); +const { startServer } = await import("../src/server.js"); +const { SessionStore, _setStoreForTest } = await import("../src/persist.js"); +const { _setForTest: setRegistryForTest } = await import("../src/registry.js"); +const { startFakeUpstream } = await import("./wire-contract-fakes.js"); + +const root = mkdtempSync(path.join(tmpdir(), "issue3400-rescale-")); +const logLines: string[] = []; +const log = (level: string, msg: string): void => { logLines.push(`${level} ${msg}`); }; + +function ctxFor(sessionId: string, toolArmed = true) { + const session = createSession(sessionId); + return { session, ctx: { session, sessionId, log, maxDim: 1600, toolArmed } }; +} + +async function makePng(width: number, height: number): Promise<{ b64: string; bytes: number }> { + const img = new Jimp({ width, height }); + for (let y = 0; y < height; y++) { + for (let x = 0; x < width; x++) { + const c = (((x % 256) << 16) | ((y % 256) << 8) | ((x + y) % 256)) >>> 0; + img.setPixelColor((0xff000000 | c) >>> 0, x, y); + } + } + const buf = await img.getBuffer("image/png"); + return { b64: buf.toString("base64"), bytes: buf.length }; +} + +async function makeBmp(width: number, height: number): Promise<{ b64: string; bytes: number }> { + const seed = new Jimp({ width: 1, height: 1 }); + seed.setPixelColor(0xff8899aa, 0, 0); + const img = await Jimp.read(await seed.getBuffer("image/bmp")); + await img.resize({ w: width, h: height }); + const buf = await img.getBuffer("image/bmp"); + return { b64: buf.toString("base64"), bytes: buf.length }; +} + +test.beforeEach(() => { + clearRescaleEncodeCache(); + logLines.length = 0; + delete process.env.BILI_IMAGE_RESCALE; +}); + +const BIG = await makePng(2400, 1800); +const SMALL = await makePng(400, 300); +const BMP = await makeBmp(1601, 400); + +function anthropicImageMsg(id: string, b64: string, mediaType = "image/png"): BiliMessage { + return { id, role: "user", contentType: "text", text: "[image]", rawAnthropicBlock: { type: "image", source: { type: "base64", media_type: mediaType, data: b64 } } }; +} + +test("big image shrunk: anthropic standalone gets copied block + adjacent note message", async () => { + const { session, ctx } = ctxFor("s-anth"); + session.state.messageRefs.byRaw["msg_img_1"] = "m12345"; + const input = [anthropicImageMsg("msg_img_1", BIG.b64)]; + const out = await applyImageRescalePass(input, ctx); + assert.equal(out.length, 2); + const block = out[0].rawAnthropicBlock as { source: { data: string } }; + assert.notEqual(block.source.data, BIG.b64); + const decoded = await Jimp.read(Buffer.from(block.source.data, "base64")); + assert.ok(decoded.width <= 1600 && decoded.height <= 1600, `shrunk dims ${decoded.width}x${decoded.height}`); + assert.notEqual(out[0], input[0], "input object not mutated in place"); + assert.equal(out[1].id, "acp_imgnote_msg_img_1"); + assert.equal(out[1].role, "user"); + assert.match(String(out[1].text), /^\[bili\] 本图已降采样:当前 \d+×\d+ \/ \d+KB;原图 2400×1800 \/ \d+KB\(约 [\d.]+ 倍\)。需要原分辨率细节\(小字、颜色、对齐\)时,调用 read_original_image\("m12345"\) 读取原图。原图较大,按需读取。$/); + // original sidecar untouched in the input array + assert.equal((input[0].rawAnthropicBlock as { source: { data: string } }).source.data, BIG.b64); +}); + +test("small image passes untouched (object identity preserved)", async () => { + const { session, ctx } = ctxFor("s-small"); + const input = [anthropicImageMsg("msg_small", SMALL.b64)]; + const out = await applyImageRescalePass(input, ctx); + assert.equal(out.length, 1); + assert.equal(out[0], input[0]); +}); + +test("low-benefit image kept (re-encode saves <20% → keep-smaller)", async () => { + const { session, ctx } = ctxFor("s-bmp"); + // 1601x400 flat BMP: gated by width, but BMP bytes scale ~linearly with + // pixel count — the 1600x400 re-encode is ~0.999x of the original, above + // the 0.8 keep-smaller threshold, so it must pass through byte-identical. + const input = [anthropicImageMsg("msg_bmp", BMP.b64, "image/bmp")]; + const out = await applyImageRescalePass(input, ctx); + assert.equal(out.length, 1); + assert.equal(out[0], input[0]); +}); + +test("hash cache: same content encodes once, cache hits log nothing", async () => { + const { session, ctx } = ctxFor("s-cache"); + const h1 = [anthropicImageMsg("msg_a", BIG.b64)]; + await applyImageRescalePass(h1, ctx); + const firstLogs = logLines.filter((l) => l.includes("[image-rescale]")).length; + assert.equal(firstLogs, 1); + const h2 = [anthropicImageMsg("msg_b", BIG.b64)]; + await applyImageRescalePass(h2, ctx); + assert.equal(logLines.filter((l) => l.includes("[image-rescale]")).length, 1, "second identical image must hit the cache without re-encoding or re-logging"); + const d1 = (h1[0].rawAnthropicBlock as { source: { data: string } }).source.data; + const d2 = (h2[0].rawAnthropicBlock as { source: { data: string } }).source.data; + assert.equal(d2, d1); +}); + +test("byte-stable: same history → identical outbound bytes across cache states", async () => { + const build = (): BiliMessage[] => [anthropicImageMsg("msg_x", BIG.b64), { id: "msg_t", role: "user", contentType: "text", text: "hi" }]; + const s1 = createSession("s-stab"); + const c1 = { session: s1, sessionId: "s-stab", log, maxDim: 1600, toolArmed: true }; + const out1 = JSON.stringify(await applyImageRescalePass(build(), c1)); + clearRescaleEncodeCache(); + const s2 = createSession("s-stab"); + const c2 = { session: s2, sessionId: "s-stab", log, maxDim: 1600, toolArmed: true }; + const out2 = JSON.stringify(await applyImageRescalePass(build(), c2)); + assert.equal(out2, out1, "two independent encodes of the same history must be byte-identical (upstream prefix-cache protection)"); +}); + +test("openai singular image_url converts to plural parts with inline note", async () => { + const { session, ctx } = ctxFor("s-openai"); + session.state.messageRefs.byRaw["msg_o"] = "m20001"; + const input: BiliMessage[] = [{ + id: "msg_o", role: "user", contentType: "text", text: "look", + rawOpenaiContent: { type: "image_url", image_url: { url: `data:image/png;base64,${BIG.b64}` } }, + imageBase64: BIG.b64, imageMediaType: "image/png", + }]; + const out = await applyImageRescalePass(input, ctx); + assert.equal(out.length, 1); + const m = out[0]; + assert.equal(m.rawOpenaiContent, undefined); + assert.equal(m.imageBase64, undefined); + assert.ok(Array.isArray(m.rawOpenaiContentParts)); + const parts = m.rawOpenaiContentParts as Array>; + assert.equal(parts.length, 2); + const url = (parts[0].image_url as { url: string }).url; + assert.match(url, /^data:image\/png;base64,/); + assert.notEqual(url, `data:image/png;base64,${BIG.b64}`); + assert.equal(parts[1].type, "text"); + assert.match(String(parts[1].text), /read_original_image\("m20001"\)/); +}); + +test("google inlineData shrunk, note appended to core text (never a raw text part)", async () => { + const { session, ctx } = ctxFor("s-google"); + session.state.messageRefs.byRaw["msg_g"] = "m30001"; + const input: BiliMessage[] = [{ + id: "msg_g", role: "user", contentType: "text", text: "hello", + rawGoogleParts: [{ inlineData: { mimeType: "image/png", data: BIG.b64 } }], + }]; + const out = await applyImageRescalePass(input, ctx); + assert.equal(out.length, 1); + const parts = out[0].rawGoogleParts as Array<{ inlineData?: { data?: string } }>; + assert.equal(parts.length, 1); + assert.notEqual(parts[0].inlineData?.data, BIG.b64); + assert.match(String(out[0].text), /^hello\n\n\[bili\] 本图已降采样:.*read_original_image\("m30001"\)/); +}); + +test("responses item: image url replaced, NO text part inside item, note rides separate message", async () => { + const { session, ctx } = ctxFor("s-resp"); + session.state.messageRefs.byRaw["msg_r"] = "m40001"; + const input: BiliMessage[] = [{ + id: "msg_r", role: "user", contentType: "text", text: "[image]", + rawResponsesItem: { + type: "message", role: "user", + content: [{ type: "input_text", text: "see" }, { type: "input_image", image_url: `data:image/png;base64,${BIG.b64}` }], + }, + }]; + const out = await applyImageRescalePass(input, ctx); + assert.equal(out.length, 2); + const content = (out[0].rawResponsesItem as { content: Array> }).content; + assert.equal(content.length, 2, "no note part may be inserted inside the item (verbatim round-trip condition)"); + assert.match(String(content[1].image_url), /^data:image\/png;base64,/); + assert.notEqual(content[1].image_url, `data:image/png;base64,${BIG.b64}`); + assert.equal(out[1].id, "acp_imgnote_msg_r"); + assert.match(String(out[1].text), /read_original_image\("m40001"\)/); +}); + +test("items walker (responses steady path): shrinks + inserts input_text note after the image", async () => { + const { session, ctx } = ctxFor("s-items"); + const items = [ + { type: "message", role: "user", content: [{ type: "input_text", text: "see" }, { type: "input_image", image_url: `data:image/png;base64,${BIG.b64}` }] }, + ]; + const out = (await applyImageRescaleItems(items, ctx)) as Array<{ content: Array> }>; + assert.equal(out.length, 1); + assert.equal(out[0].content.length, 3); + assert.equal(out[0].content[2].type, "input_text"); + assert.match(String(out[0].content[2].text), /^\[bili\] 本图已降采样:/); +}); + +test("items walker exemption: output answering read_original_image passes verbatim", async () => { + const { session, ctx } = ctxFor("s-items-exempt"); + const items = [ + { type: "function_call", call_id: "call_1", name: "read_original_image", arguments: '{"ref":"m12345"}' }, + { type: "function_call_output", call_id: "call_1", output: [{ type: "input_image", image_url: `data:image/png;base64,${BIG.b64}` }] }, + ]; + const out = (await applyImageRescaleItems(items, ctx)) as unknown[]; + assert.equal(out.length, 2); + assert.equal(out[1], items[1], "exempt tool output must be the SAME object (untouched)"); +}); + +test("structural loop guard: carrier message with toolName read_original_image is never re-shrunk", async () => { + const { session, ctx } = ctxFor("s-guard"); + const input: BiliMessage[] = [{ + id: "acp_loop_r2_img_call_1", role: "user", contentType: "text", + text: "[bili] 原图已读取,完整分辨率的原始图片如下。", + toolName: "read_original_image", imageBase64: BIG.b64, imageMediaType: "image/png", + }]; + const out = await applyImageRescalePass(input, ctx); + assert.equal(out.length, 1); + assert.equal(out[0], input[0], "the returned ORIGINAL must survive subsequent rounds byte-identical (#3266 regression pin)"); +}); + +test("knob off (BILI_IMAGE_RESCALE=0): pass returns the same array untouched", async () => { + process.env.BILI_IMAGE_RESCALE = "0"; + try { + const { session, ctx } = ctxFor("s-off"); + const input = [anthropicImageMsg("msg_off", BIG.b64)]; + const out = await applyImageRescalePass(input, ctx); + assert.equal(out, input); + } finally { + delete process.env.BILI_IMAGE_RESCALE; + } +}); + +test("size-only note when the tool lane is not armed (codex/plugin lanes)", async () => { + const { session, ctx } = ctxFor("s-codex", false); + session.state.messageRefs.byRaw["msg_c"] = "m50001"; + const input = [anthropicImageMsg("msg_c", BIG.b64)]; + const out = await applyImageRescalePass(input, ctx); + assert.equal(out.length, 2); + assert.match(String(out[1].text), /^\[bili\] 本图已降采样:当前 \d+×\d+ \/ \d+KB;原图 2400×1800 \/ \d+KB。$/); + assert.doesNotMatch(String(out[1].text), /read_original_image/); +}); + +test("buildRescaleNote: deterministic templates (full + size-only)", () => { + const full = buildRescaleNote({ ref: "m12345", toolArmed: true, width: 1600, height: 1173, kb: 73, origWidth: 2140, origHeight: 1560, origKb: 317, ratio: 4.3 }); + assert.equal(full, "[bili] 本图已降采样:当前 1600×1173 / 73KB;原图 2140×1560 / 317KB(约 4.3 倍)。需要原分辨率细节(小字、颜色、对齐)时,调用 read_original_image(\"m12345\") 读取原图。原图较大,按需读取。"); + assert.equal(full, buildRescaleNote({ ref: "m12345", toolArmed: true, width: 1600, height: 1173, kb: 73, origWidth: 2140, origHeight: 1560, origKb: 317, ratio: 4.3 })); + const sizeOnly = buildRescaleNote({ ref: "m12345", toolArmed: false, width: 1600, height: 1173, kb: 73, origWidth: 2140, origHeight: 1560, origKb: 317, ratio: 4.3 }); + assert.equal(sizeOnly, "[bili] 本图已降采样:当前 1600×1173 / 73KB;原图 2140×1560 / 317KB。"); + const noRef = buildRescaleNote({ ref: undefined, toolArmed: true, width: 1600, height: 1173, kb: 73, origWidth: 2140, origHeight: 1560, origKb: 317, ratio: 4.3 }); + assert.equal(noRef, sizeOnly); +}); + +test("gate stamping: storeEffectiveImageRescale/imageRescaleToolArmed", () => { + const s = createSession("s-gate"); + assert.equal(imageRescaleToolArmed(s), false); + storeEffectiveImageRescale(s, true); + assert.equal(imageRescaleToolArmed(s), true); + storeEffectiveImageRescale(s, undefined); + assert.equal(imageRescaleToolArmed(s), false); +}); + +test("executeReadOriginalImage: returns EXACT stored bytes + cumulative counter", async () => { + const session = createSession("s-read"); + const imgPath = path.join(root, "original.png"); + const buf = Buffer.from(BIG.b64, "base64"); + writeFileSync(imgPath, buf); + session.incomingImageIndex = new Map([["m12345", [{ mediaType: "image/png", bytes: buf.length, width: 2400, height: 1800, path: imgPath }]]]); + const r1 = await executeReadOriginalImage({ ref: "m12345" }, session, log); + assert.equal(r1.outcome, "success"); + assert.equal(r1.image?.base64, BIG.b64, "returned bytes must equal the stored original exactly"); + assert.equal(r1.image?.mediaType, "image/png"); + assert.match(r1.text, new RegExp(imgPath.replace(/[.*+?^${}()|[\]\\]/g, "\\$&"))); + const counter = session.metadata.imageRescaleReads as { count: number; bytes: number }; + assert.deepEqual(counter, { count: 1, bytes: buf.length }); + await executeReadOriginalImage({ ref: "m12345" }, session, log); + assert.deepEqual(session.metadata.imageRescaleReads, { count: 2, bytes: buf.length * 2 }); + assert.match(logLines.at(-1) ?? "", /session-cumulative 2 read\(s\)/); +}); + +test("executeReadOriginalImage: invalid / unknown / evicted refs fail cleanly", async () => { + const session = createSession("s-read-fail"); + const bad = await executeReadOriginalImage({ ref: "not-a-ref" }, session, log); + assert.equal(bad.outcome, "failure"); + const missing = await executeReadOriginalImage({ ref: "m99999" }, session, log); + assert.equal(missing.outcome, "failure"); + assert.match(missing.text, /no original image stored/); + session.incomingImageIndex = new Map([["m12345", [{ mediaType: "image/png", bytes: 10, width: 10, height: 10, path: path.join(root, "gone.png") }]]]); + const evicted = await executeReadOriginalImage({ ref: "m12345" }, session, log); + assert.equal(evicted.outcome, "failure"); + assert.match(evicted.text, /evicted/); +}); + +// #3400 spec goal 2 pin: the shrink must happen on the OUTBOUND COPY ONLY. +// Regression pinned: prepare-* once reassigned its local processedMessages to +// the rescaled array, and Prepared returned that local — leaking shrunken bytes +// plus synthetic acp_imgnote_* messages into rememberPluginMessages() → +// session.transient.pluginMessages (the tool-API compress anchor view), i.e. +// the session storage no longer kept what the wire received. This test drives +// one real request through startServer and checks BOTH halves: upstream saw the +// shrunken image, while the remembered session view keeps the original bytes +// byte-exact and contains no synthetic note messages. +test("persistence invariant: remembered session view keeps ORIGINAL image bytes while egress sends shrunken (#3400)", async () => { + const fake = await startFakeUpstream("anthropic"); + _setStoreForTest(new SessionStore({ enabled: false })); + setRegistryForTest({}); + const opts: ProxyOptions = { + port: 0, + host: "127.0.0.1", + upstream: fake.url, + routes: { [fake.url]: { models: { "claude-test": { context: 100_000 } } } }, + modelContextLimit: 100_000, + kernelConfig: defaultConfig(100_000), + compress: { injectTool: true, injectNudge: true }, + promptCache: { routing: "auto" }, + log: false, + sessionHeader: "x-acp-session", + debug: false, + passthrough: false, + autoUpdate: false, + compat: { roles: {} }, + passthroughSource: null, + autoRestartOnUpdate: false, + updateTag: "latest", + streamErrorShape: "protocol", + advisoryCheck: false, + releaseNotesCheck: false, + mitm: { enabled: false, domains: [] }, + }; + const proxy = await startServer(opts); + try { + await once(proxy, "listening"); + const port = (proxy.address() as { port: number }).port; + const res = await fetch(`http://127.0.0.1:${port}/bili/${fake.url}/v1/messages`, { + method: "POST", + headers: { "content-type": "application/json", "x-api-key": "test", "x-acp-session": "wc-persist-pin" }, + body: JSON.stringify({ + model: "claude-test", max_tokens: 1024, stream: true, + messages: [{ role: "user", content: [ + { type: "text", text: "look at this" }, + { type: "image", source: { type: "base64", media_type: "image/png", data: BIG.b64 } }, + ]}], + }), + }); + assert.equal(res.status, 200); + await res.text(); + + // Egress half: the forwarded body carried the SHRUNKEN image. + const fwdBody = fake.requests[0]?.body as { messages?: Array<{ content?: Array<{ type?: string; source?: { data?: string } }> }> } | undefined; + const fwdImage = fwdBody?.messages?.flatMap((m) => m.content ?? []).find((c) => c.type === "image"); + assert.ok(fwdImage, "forwarded body carries the image block"); + assert.notEqual(fwdImage.source?.data, BIG.b64, "egress copy must be downscaled"); + + // Storage half: the remembered view keeps the ORIGINAL bytes byte-exact. + const session = peekSession("wc-persist-pin"); + assert.ok(session, "session exists after the turn"); + const stored = session.transient?.pluginMessages; + assert.ok(stored, "rememberPluginMessages must have run for the forwarded turn"); + const originalKept = stored.processed.some((m) => { + const block = (m as BiliMessage).rawAnthropicBlock as { type?: string; source?: { data?: string } } | undefined; + return block?.type === "image" && block?.source?.data === BIG.b64; + }); + assert.ok(originalKept, "remembered processed view must keep the ORIGINAL big-image bytes"); + assert.ok(!stored.processed.some((m) => String(m.id).startsWith("acp_imgnote_")), "synthetic note messages must not enter the stored view"); + } finally { + await new Promise((resolve, reject) => proxy.close((e) => (e ? reject(e) : resolve()))); + await fake.close(); + } +}); diff --git a/tests/issue3434-first-byte-watchdog.test.ts b/tests/issue3434-first-byte-watchdog.test.ts new file mode 100644 index 0000000000..9dbe44e3ab --- /dev/null +++ b/tests/issue3434-first-byte-watchdog.test.ts @@ -0,0 +1,122 @@ +import assert from "node:assert/strict"; +import { beforeEach, test } from "node:test"; +import { + _resetFirstByteWarnedOriginsForTest, + _setFirstByteWarnMsForTest, + armFirstByteWatchdog, +} from "../src/server/relay.ts"; + +process.env.NODE_ENV = "test"; + +const THRESHOLD_MS = 30; +const WAIT_MS = 150; +const sleep = (ms: number): Promise => new Promise((r) => setTimeout(r, ms)); + +type LogCall = { level: string; msg: string }; +function makeLog(): { log: (level: string, msg: string) => void; calls: LogCall[] } { + const calls: LogCall[] = []; + return { log: (level, msg): void => { calls.push({ level, msg }); }, calls }; +} + +beforeEach(() => { + _resetFirstByteWarnedOriginsForTest(); + _setFirstByteWarnMsForTest(THRESHOLD_MS); +}); + +test("warns exactly once when the fetch is still pending past the threshold", async () => { + const { log, calls } = makeLog(); + const ac = new AbortController(); + const timer = armFirstByteWatchdog({ url: "https://api.example.com/v1/messages", method: "POST", lane: "side", sessionId: "sess-1", log, signal: ac.signal }); + assert.notEqual(timer, undefined); + await sleep(WAIT_MS); + if (timer) clearTimeout(timer); + assert.equal(calls.length, 1); + assert.equal(calls[0].level, "warn"); + assert.ok(calls[0].msg.startsWith("[sess-1] upstream awaiting first byte ≥"), calls[0].msg); + // The target goes through maskUrlForLog — a non-allowlisted host is reduced + // to , so the warn line must never leak the real upstream host. + assert.ok(calls[0].msg.includes(": POST https:///v1/messages"), calls[0].msg); + assert.ok(calls[0].msg.endsWith("(lane=side)"), calls[0].msg); +}); + +test("does not warn when the fetch settles before the threshold (caller clears)", async () => { + const { log, calls } = makeLog(); + const ac = new AbortController(); + const timer = armFirstByteWatchdog({ url: "https://fast.example.com/v1/messages", method: "POST", lane: "main", sessionId: "s", log, signal: ac.signal }); + assert.notEqual(timer, undefined); + clearTimeout(timer); + await sleep(WAIT_MS); + assert.equal(calls.length, 0); +}); + +test("dedupes per origin: a later pending request to the same origin stays silent", async () => { + const { log, calls } = makeLog(); + const acA = new AbortController(); + const tA = armFirstByteWatchdog({ url: "https://dup.example.com/x", method: "POST", lane: "main", sessionId: "a", log, signal: acA.signal }); + await sleep(WAIT_MS); + if (tA) clearTimeout(tA); + assert.equal(calls.length, 1); + // Same origin (scheme+host+port), different path — disarmed at arm time. + const acB = new AbortController(); + const tB = armFirstByteWatchdog({ url: "https://dup.example.com/y", method: "POST", lane: "main", sessionId: "b", log, signal: acB.signal }); + assert.equal(tB, undefined); + await sleep(WAIT_MS); + assert.equal(calls.length, 1); +}); + +test("different origins each warn once (dedupe is keyed by origin, not global)", async () => { + const { log, calls } = makeLog(); + const acA = new AbortController(); + const tA = armFirstByteWatchdog({ url: "https://a.example.com/", method: "GET", lane: "main", sessionId: "a", log, signal: acA.signal }); + const acB = new AbortController(); + const tB = armFirstByteWatchdog({ url: "https://b.example.com/", method: "GET", lane: "main", sessionId: "b", log, signal: acB.signal }); + await sleep(WAIT_MS); + if (tA) clearTimeout(tA); + if (tB) clearTimeout(tB); + assert.equal(calls.length, 2); +}); + +test("does not warn when the client aborts before the threshold", async () => { + const { log, calls } = makeLog(); + const ac = new AbortController(); + const timer = armFirstByteWatchdog({ url: "https://ab.example.com/", method: "POST", lane: "main", sessionId: "a", log, signal: ac.signal }); + assert.notEqual(timer, undefined); + ac.abort(); + await sleep(WAIT_MS); + assert.equal(calls.length, 0); +}); + +test("is disarmed when the signal was already aborted before arming", () => { + const { log, calls } = makeLog(); + const ac = new AbortController(); + ac.abort(); + const timer = armFirstByteWatchdog({ url: "https://ab2.example.com/", method: "POST", lane: "main", sessionId: "a", log, signal: ac.signal }); + assert.equal(timer, undefined); + assert.equal(calls.length, 0); +}); + +test("is disarmed when the threshold is disabled (<=0 or non-finite)", () => { + const { log, calls } = makeLog(); + const ac = new AbortController(); + _setFirstByteWarnMsForTest(0); + assert.equal(armFirstByteWatchdog({ url: "https://d.example.com/", method: "POST", lane: "main", sessionId: "a", log, signal: ac.signal }), undefined); + _setFirstByteWarnMsForTest(Number.NaN); + assert.equal(armFirstByteWatchdog({ url: "https://d.example.com/", method: "POST", lane: "main", sessionId: "a", log, signal: ac.signal }), undefined); + assert.equal(calls.length, 0); +}); + +test("_resetFirstByteWarnedOriginsForTest lets a previously-warned origin warn again", async () => { + const { log, calls } = makeLog(); + const ac1 = new AbortController(); + const t1 = armFirstByteWatchdog({ url: "https://rs.example.com/", method: "POST", lane: "main", sessionId: "a", log, signal: ac1.signal }); + await sleep(WAIT_MS); + if (t1) clearTimeout(t1); + assert.equal(calls.length, 1); + _resetFirstByteWarnedOriginsForTest(); + const ac2 = new AbortController(); + const t2 = armFirstByteWatchdog({ url: "https://rs.example.com/", method: "POST", lane: "main", sessionId: "b", log, signal: ac2.signal }); + assert.notEqual(t2, undefined); + await sleep(WAIT_MS); + if (t2) clearTimeout(t2); + assert.equal(calls.length, 2); +}); diff --git a/tests/issue3461-overlimit-exits.test.ts b/tests/issue3461-overlimit-exits.test.ts new file mode 100644 index 0000000000..a3fdbc4bda --- /dev/null +++ b/tests/issue3461-overlimit-exits.test.ts @@ -0,0 +1,216 @@ +import assert from "node:assert/strict"; +import http from "node:http"; +import { once } from "node:events"; +import test from "node:test"; + +process.env.NODE_ENV = "test"; + +// #3461 problem 2 (real-model stress test, dsh/pi lanes, qwen3.8-27b @ sglang): +// a session pinned over its window with less foldable mass than the deficit had +// NO exit — preflight answered 502 (read as transient by status-code-driven +// headless hosts, which spun through 5+ doomed rounds) while the EMERGENCY +// tier-1 nudge override re-armed on EVERY request (measured 7 injections in +// 1 second). Pinned here: +// 1. unit — emergencyNudgeArmed: the override injects once per state; it +// re-arms only on a landed reduction, a full flat-growth step of new +// content, or the 10-minute wall-clock safety valve. Kernel-driven +// injections (shouldInject=true) never pass through the gate. +// 2. e2e — a payload that can never fit (oversized system prompt, nothing +// foldable left) fails fast 413 non-retryable instead of 502, and the +// dead-end cooldown replays the same verdict without touching the +// upstream again. + +import { defaultConfig, type NudgeDecision } from "acp-kernel"; +import { startServer } from "../src/server.ts"; +import type { ProxyOptions } from "../src/config.ts"; +import { SessionStore, _setStoreForTest } from "../src/persist.ts"; +import { _setForTest as setRegistryForTest } from "../src/registry.ts"; +import { listSessions, _resetSessionsForTest, type Session } from "../src/session.ts"; +import { emergencyNudgeArmed } from "../src/server/budget.ts"; +import { SELF_HEAL_SUPPRESS_CAP_MS } from "../src/session-self-heal.ts"; + +// ---- Part A: unit — the emergency override backoff gate --------------------- + +function mkSession(): Session { + return { + id: "s3461", + metadata: {}, + state: { nextBlockId: 1 }, + lastCompress: undefined, + } as unknown as Session; +} + +/** Over-limit (>= 0.7 escalation line), kernel declined (cadence), one + * executable range — the exact shape that made the pure check fire every + * request in the field report. */ +function overLimitDeclined(usage = 0.85): NudgeDecision { + return { shouldInject: false, contextUsage: usage, compressibleRanges: [{ chars: 20_000 }] } as unknown as NudgeDecision; +} + +test("#3461 unit: seven identical doomed retries inject the emergency nudge exactly once", () => { + const s = mkSession(); + const results = Array.from({ length: 7 }, () => emergencyNudgeArmed(s, overLimitDeclined(), 0, 90_000)); + assert.deepEqual(results, [true, false, false, false, false, false, false], "first retry arms, the rest stay quiet"); + const gate = s.metadata["emergencyNudgeGate"] as Record; + assert.equal(gate["tokens"], 90_000, "gate stamped with the injecting token count"); + assert.equal(gate["reductionKey"], "1:0", "gate stamps the reduction fingerprint"); + assert.equal(typeof gate["at"], "number"); +}); + +test("#3461 unit: a landed reduction re-arms the emergency nudge", () => { + const s = mkSession(); + assert.equal(emergencyNudgeArmed(s, overLimitDeclined(), 0, 90_000), true); + assert.equal(emergencyNudgeArmed(s, overLimitDeclined(), 0, 90_000), false); + s.state.nextBlockId = 2; // committed fold advanced the block cursor + assert.equal(emergencyNudgeArmed(s, overLimitDeclined(), 0, 90_000), true, "shrunken payload still needs the prompt"); + assert.equal(emergencyNudgeArmed(s, overLimitDeclined(), 0, 90_000), false, "and stays quiet again until the next change"); +}); + +test("#3461 unit: only a full flat-growth step re-arms (no window rescaling)", () => { + const s = mkSession(); + assert.equal(emergencyNudgeArmed(s, overLimitDeclined(), 0, 90_000), true); + assert.equal(emergencyNudgeArmed(s, overLimitDeclined(), 0, 90_000 + 49_999), false, "sub-step growth is noise"); + assert.equal(emergencyNudgeArmed(s, overLimitDeclined(), 0, 90_000 + 50_000), true, "a full 50K step is new content worth advising about"); + const s2 = mkSession(); + assert.equal(emergencyNudgeArmed(s2, overLimitDeclined(), 0, 1_000, 1_000), true); + assert.equal(emergencyNudgeArmed(s2, overLimitDeclined(), 0, 1_999, 1_000), false); + assert.equal(emergencyNudgeArmed(s2, overLimitDeclined(), 0, 2_000, 1_000), true, "honors the caller-supplied growth step"); +}); + +test("#3461 unit: wall-clock safety valve re-arms once after the cap", () => { + const realNow = Date.now; + let fake = 1_000_000; + Date.now = () => fake; + try { + const s = mkSession(); + assert.equal(emergencyNudgeArmed(s, overLimitDeclined(), 0, 90_000), true); + fake += 9 * 60_000; + assert.equal(emergencyNudgeArmed(s, overLimitDeclined(), 0, 90_000), false, "under the cap: quiet"); + fake += 2 * 60_000; // total >= SELF_HEAL_SUPPRESS_CAP_MS (10 min) + assert.ok(fake - 1_000_000 >= SELF_HEAL_SUPPRESS_CAP_MS, "test clock crossed the cap"); + assert.equal(emergencyNudgeArmed(s, overLimitDeclined(), 0, 90_000), true, "permanently stuck sessions still get periodic reminders"); + assert.equal(emergencyNudgeArmed(s, overLimitDeclined(), 0, 90_000), false, "the valve fires once, then the gate holds again"); + } finally { + Date.now = realNow; + } +}); + +test("#3461 unit: below the escalation line the gate never arms or stamps", () => { + const s = mkSession(); + assert.equal(emergencyNudgeArmed(s, overLimitDeclined(0.5), 0, 90_000), false); + assert.equal(emergencyNudgeArmed(s, overLimitDeclined(0.69), 0, 90_000), false); + assert.equal(s.metadata["emergencyNudgeGate"], undefined, "no state written for a declined pure check"); +}); + +test("#3461 unit: kernel-driven injections never pass through the gate", () => { + const s = mkSession(); + const kernelDriven = { shouldInject: true, contextUsage: 0.9, compressibleRanges: [{ chars: 20_000 }] } as unknown as NudgeDecision; + assert.equal(emergencyNudgeArmed(s, kernelDriven, 0, 90_000), false, "call sites OR shouldInject in BEFORE the emergency arm"); + assert.equal(s.metadata["emergencyNudgeGate"], undefined, "and must not consume the emergency gate"); +}); + +test("#3461 unit: no executable range keeps the pure-check refusal", () => { + const s = mkSession(); + const none = { shouldInject: false, contextUsage: 0.9, compressibleRanges: [] } as unknown as NudgeDecision; + assert.equal(emergencyNudgeArmed(s, none, 0, 90_000), false); + assert.equal(s.metadata["emergencyNudgeGate"], undefined); +}); + +// ---- Part B: e2e — deterministic exhaustion answers 413, not 502 ------------ + +// 8k keeps the fixed compress surface (system section + tool schemas, ~4.6k +// tokens measured — see issue #3461 problem 3) inside the window so turn 1 +// fits, while the 60k-char system swap on turn 2 still overshoots by ~2.5x. +const WINDOW = 8_000; + +function okSse(inputTokens: number): string { + const start: Record = { type: "message_start", message: { id: "m1", role: "assistant", usage: { input_tokens: inputTokens } } }; + const delta: Record = { type: "message_delta", delta: { stop_reason: "end_turn", stop_sequence: null }, usage: { output_tokens: 3 } }; + return ( + `event: message_start\ndata: ${JSON.stringify(start)}\n\n` + + `event: content_block_start\ndata: ${JSON.stringify({ type: "content_block_start", index: 0, content_block: { type: "text", text: "" } })}\n\n` + + `event: content_block_delta\ndata: ${JSON.stringify({ type: "content_block_delta", index: 0, delta: { type: "text_delta", text: "ok" } })}\n\n` + + `event: content_block_stop\ndata: ${JSON.stringify({ type: "content_block_stop", index: 0 })}\n\n` + + `event: message_delta\ndata: ${JSON.stringify(delta)}\n\n` + + `event: message_stop\ndata: ${JSON.stringify({ type: "message_stop" })}\n\n` + ); +} + +type Msg = { role: string; content: unknown }; + +test("#3461 e2e: a payload that can never fit fails fast 413 non-retryable, cooldown replays without upstream calls", async () => { + _resetSessionsForTest(); + _setStoreForTest(new SessionStore({ enabled: false })); + setRegistryForTest({}); + let upstreamCalls = 0; + const server = http.createServer((_req, res) => { + upstreamCalls += 1; + res.writeHead(200, { "content-type": "text/event-stream" }); + res.end(okSse(500)); + }); + server.listen(0, "127.0.0.1"); + await once(server, "listening"); + const port = (server.address() as { port: number }).port; + const proxy = await startServer({ + port: 0, + host: "127.0.0.1", + upstream: "http://127.0.0.1", + routes: { [`http://127.0.0.1:${port}`]: { models: { "claude-relay": { context: WINDOW } } } }, + modelContextLimit: WINDOW, + kernelConfig: defaultConfig(WINDOW), + compress: { injectTool: true, injectNudge: true }, + promptCache: { routing: "auto" }, + sessionHeader: "x-acp-session", + log: false, + debug: false, + passthrough: false, + compat: { roles: {} }, streamErrorShape: "protocol", passthroughSource: null, autoRestartOnUpdate: false, updateTag: "latest", advisoryCheck: false, releaseNotesCheck: false, + autoUpdate: false, + mitm: { enabled: false, domains: [] }, + } as ProxyOptions); + await once(proxy, "listening"); + const id = "sess-3461-exhausted"; + const url = `http://127.0.0.1:${(proxy.address() as { port: number }).port}/bili/http://127.0.0.1:${port}/v1/messages`; + const headers = { "content-type": "application/json", "x-acp-session": id }; + const post = (system: string, messages: Msg[]) => fetch(url, { + method: "POST", + headers, + body: JSON.stringify({ model: "claude-relay", max_tokens: 1024, stream: true, system, messages }), + }); + try { + // Turn 1: normal-sized payload, succeeds, arms the usage-grade baseline + // and route evidence (so turn 2 takes the sizing path, not the + // fresh-session forward-once arbitration). + const smallSystem = "You are a helpful assistant."; + const r1 = await post(smallSystem, [{ role: "user", content: "go?" }]); + assert.equal(r1.status, 200, "turn 1 succeeds"); + await r1.text(); + + // Turn 2: the client swaps in an oversized system prompt (~60k chars ≈ + // well over the 4k window) on top of the short history. The system + // prompt is never foldable and the three history messages sit inside + // the protected recent zone — no combination of folds closes the gap, + // so the walk exhausts deterministically. + const hugeSystem = "S".repeat(60_000); + const r2 = await post(hugeSystem, [{ role: "user", content: "go?" }, { role: "assistant", content: "ok" }, { role: "user", content: "again?" }]); + const j2 = (await r2.json()) as Record; + assert.equal(r2.status, 413, `deterministic exhaustion must answer 413 (pre-#3461: 502), got ${r2.status}: ${JSON.stringify(j2)}`); + const err = j2["error"] as Record; + assert.match(String(err?.["message"]), /NOT forwarded/, "the diagnostic says the payload was not forwarded"); + assert.match(String(err?.["message"]), /exhaust|window|session/i, "the diagnostic carries the exhaustion reason"); + + // Turn 3: identical payload hits the dead-end cooldown — same verdict, + // zero upstream traffic. + const before = upstreamCalls; + const r3 = await post(hugeSystem, [{ role: "user", content: "go?" }, { role: "assistant", content: "ok" }, { role: "user", content: "again?" }]); + assert.equal(r3.status, 413, "cooldown replays the 413 verdict"); + await r3.text(); + assert.equal(upstreamCalls, before, "the cooldown round must not touch the upstream"); + const s = listSessions().find((x) => x.id === id); + assert.ok(s, "session exists"); + assert.equal((s!.metadata.preflightDeadEnd as Record)["status"], 413, "the dead-end marker carries the 413 status"); + } finally { + await new Promise((r) => proxy.close(() => r())); + await new Promise((r) => server.close(() => r())); + } +}); diff --git a/tests/issue496-forward-once-learn.test.ts b/tests/issue496-forward-once-learn.test.ts index cdf3d3ca30..f23c20e29c 100644 --- a/tests/issue496-forward-once-learn.test.ts +++ b/tests/issue496-forward-once-learn.test.ts @@ -128,14 +128,17 @@ mitm: { enabled: false, domains: [] }, // --- Request 2: overflow EVIDENCE now exists → NO forward-once → fail-fast --- const r2 = await fetch(url, { method: "POST", headers, body: screenshotPayload() }); - assert.equal(r2.status, 502, "second request fails fast instead of forwarding again"); + // #3461: the image payload is over-window per bili's own meter and no + // fold can close the gap — a structurally doomed verdict answers 413 + // (non-retryable) instead of the legacy 502. + assert.equal(r2.status, 413, "second request fails fast instead of forwarding again"); const err2 = JSON.parse(await r2.text()) as { error?: { code?: string } }; assert.equal(err2.error?.code, "preflight_compress_failed"); assert.deepEqual(streamForwards, [true], "request 2 was NEVER forwarded (loop broken after one rejection)"); // --- Request 3: stable — still fail-fast, still exactly one forward total --- const r3 = await fetch(url, { method: "POST", headers, body: screenshotPayload() }); - assert.equal(r3.status, 502, "third request also fails fast"); + assert.equal(r3.status, 413, "third request also fails fast (cooldown replays the same verdict, #3461)"); await r3.text(); assert.deepEqual(streamForwards, [true], "still exactly one forward total (no re-forwarding loop)"); } finally { diff --git a/tests/issue767-pixel-billing.test.ts b/tests/issue767-pixel-billing.test.ts index f8711ee09d..6cf2958767 100644 --- a/tests/issue767-pixel-billing.test.ts +++ b/tests/issue767-pixel-billing.test.ts @@ -357,7 +357,8 @@ test("e2e #767 control: explicit bytes route still fails fast — relay protecti s!.stats.lastInputTokens = STALE_BASELINE; const r2 = await fetch(url, { method: "POST", headers, body: imageTurn("img-byt-sess") }); - assert.equal(r2.status, 502, "byte-billed over-window images stay blocked locally"); + // #3461: structurally doomed (byte-billed images can't fold enough) → 413. + assert.equal(r2.status, 413, "byte-billed over-window images stay blocked locally"); const err2 = JSON.parse(await r2.text()) as { error?: { code?: string; message?: string } }; assert.equal(err2.error?.code, "preflight_compress_failed"); assert.ok(err2.error?.message?.includes("Images alone account for"), "fail-fast carries the image remedy wording"); @@ -439,7 +440,8 @@ test("e2e #767: learned-limit-only variant also closes forward-once (explicit by (s!.stats as { lastInputTokensSource: string }).lastInputTokensSource = "usage"; const r2 = await fetch(url, { method: "POST", headers, body: imageTurn("img-lrn-sess") }); - assert.equal(r2.status, 502, "overflow evidence (learned limit) closes #496 forward-once even with a fresh baseline"); + // #3461: structurally doomed (byte-billed images can't fold enough) → 413. + assert.equal(r2.status, 413, "overflow evidence (learned limit) closes #496 forward-once even with a fresh baseline"); const err2 = JSON.parse(await r2.text()) as { error?: { code?: string } }; assert.equal(err2.error?.code, "preflight_compress_failed"); assert.equal(stats.streamingForwards, 1, "image-bearing turn was never forwarded"); @@ -480,7 +482,8 @@ test("e2e #767: env BILI_IMAGE_BILLING beats per-provider route config", async ( } finally { delete process.env.BILI_IMAGE_BILLING; } - assert.equal(r2.status, 502, "env override downgrades the pixels route back to conservative byte billing"); + // #3461: structurally doomed (byte-billed images can't fold enough) → 413. + assert.equal(r2.status, 413, "env override downgrades the pixels route back to conservative byte billing"); const err2 = JSON.parse(await r2.text()) as { error?: { code?: string } }; assert.equal(err2.error?.code, "preflight_compress_failed"); assert.equal(stats.streamingForwards, 1); diff --git a/tests/issue857-image-baseline.test.ts b/tests/issue857-image-baseline.test.ts index 1c59430fd6..013712c69b 100644 --- a/tests/issue857-image-baseline.test.ts +++ b/tests/issue857-image-baseline.test.ts @@ -208,7 +208,8 @@ test("#857 B: genuine overflow evidence (learned limit == configured window) sti s.stats.lastInputTokensSource = "usage"; const r2 = await fetch(url, { method: "POST", headers, body: body(MODEL, "s857b", imageInput()) }); - assert.equal(r2.status, 502, "no blind forward when the upstream proved the window"); + // #3461: over-window per bili's own meter with no foldable mass left → 413. + assert.equal(r2.status, 413, "no blind forward when the upstream proved the window"); await r2.text(); assert.equal(stats.streamingForwards, 1, "image turn was not forwarded"); assert.equal(stats.imagesSeen, 0); @@ -246,7 +247,8 @@ test("#857 C: usage-grounded over-window baseline — hatch stays closed AND the // #1841 removes (old code walked both ranges, burned the calls, and // still failed here). const r2 = await fetch(url, { method: "POST", headers, body: body(FALLBACK_MODEL, "s857c", [...fillerHistory(16), ...imageInput()]) }); - assert.equal(r2.status, 502, "usage evidence keeps the hatch closed"); + // #3461: over-window per bili's own meter with no foldable mass left → 413. + assert.equal(r2.status, 413, "usage evidence keeps the hatch closed"); const err2 = JSON.parse(await r2.text()) as { error?: { code?: string; message?: string } }; assert.equal(err2.error?.code, "preflight_compress_failed"); assert.match(err2.error?.message ?? "", /futile round/, `the skipped round names its futility verdict (got: ${err2.error?.message})`); diff --git a/tests/lane-decisions-b-class.test.ts b/tests/lane-decisions-b-class.test.ts new file mode 100644 index 0000000000..2852357089 --- /dev/null +++ b/tests/lane-decisions-b-class.test.ts @@ -0,0 +1,127 @@ +import assert from "node:assert/strict"; +import { test } from "node:test"; +import { nudgeOn, resolveLaneDecisions, resolveWireClientProfile, type NudgeGateInputs } from "../src/server/lane-decisions.js"; +import { instructionsFingerprintApplies } from "../src/session-id.js"; + +// #3482 Step 2.5 equivalence proof: the single nudgeOn() gate must reproduce +// the four legacy hand-copied conjunctions BIT-FOR-BIT over the full boolean +// cube of their raw primitives. The three deliberate per-lane differences are +// explicit inputs: +// - compactionTrigger: responses only (hasCompactionTrigger(parsed.input)) +// - growth health window: responses codex-intercept only +// - the standalone `shouldInject` conjunct of openai/google is subsumed by +// compressSurfaceOn (=== shouldInject && compressReachable) +// anthropic's #3293 drift (missing injectTool conjunct) was ALREADY fixed by +// #3427 — the cube asserts the fixed shape, so a regression to the drifted +// formula fails here too. +interface Primitives { + injectNudge: boolean; hasNudgePlan: boolean; shouldInject: boolean; compressReachable: boolean; + compactionTrigger: boolean; suppressionBypassed: boolean; breakerArmed: boolean; + autoFoldEngaged: boolean; growthFoldingArmed: boolean; modelWantsIt: boolean; +} +function laneInputs(p: Primitives): NudgeGateInputs { + const compressSurfaceOn = p.shouldInject && p.compressReachable; + return { + injectNudge: p.injectNudge, + hasNudgePlan: p.hasNudgePlan, + compressSurfaceOn, + compactionTrigger: p.compactionTrigger, + suppressionBypassed: p.suppressionBypassed, + breakerArmed: p.breakerArmed, + autoFoldEngaged: p.autoFoldEngaged, + growthFoldingArmed: p.growthFoldingArmed, + modelWantsIt: p.modelWantsIt, + }; +} +// The legacy formulas, transcribed verbatim from the pre-#3482 prepares. +// dshCompressSurfaceOff === !compressSurfaceOn held on every lane (the alias +// was assigned laneDecisions.compressSurfaceOff). +function legacyAnthropic(p: Primitives): boolean { + const compressSurfaceOn = p.shouldInject && p.compressReachable; + return p.injectNudge && p.hasNudgePlan && compressSurfaceOn && p.suppressionBypassed && !p.breakerArmed && !(p.autoFoldEngaged && p.growthFoldingArmed) && p.modelWantsIt; +} +function legacyOpenaiGoogle(p: Primitives): boolean { + const compressSurfaceOn = p.shouldInject && p.compressReachable; + return p.injectNudge && p.hasNudgePlan && p.shouldInject && compressSurfaceOn && p.suppressionBypassed && !p.breakerArmed && !(p.autoFoldEngaged && p.growthFoldingArmed) && p.modelWantsIt; +} +function legacyResponses(p: Primitives): boolean { + const compressSurfaceOn = p.shouldInject && p.compressReachable; + return p.injectNudge && p.hasNudgePlan && compressSurfaceOn && !p.compactionTrigger && p.suppressionBypassed && !p.breakerArmed && !(p.autoFoldEngaged && p.growthFoldingArmed) && p.modelWantsIt; +} +test("nudgeOn: full boolean cube equals all four legacy formulas (#3482 Step 2.5)", () => { + const KEYS = Object.keys({ injectNudge: 0, hasNudgePlan: 0, shouldInject: 0, compressReachable: 0, compactionTrigger: 0, suppressionBypassed: 0, breakerArmed: 0, autoFoldEngaged: 0, growthFoldingArmed: 0, modelWantsIt: 0 }) as Array; + let checked = 0; + for (let mask = 0; mask < (1 << KEYS.length); mask++) { + const p = {} as Primitives; + for (let i = 0; i < KEYS.length; i++) (p as unknown as Record)[KEYS[i]] = ((mask >> i) & 1) === 1; + const got = nudgeOn(laneInputs(p)); + // anthropic/openai/google have NO compaction-trigger concept on the + // wire — that input is unrepresentable there, so their formulas are + // only comparable on the sub-cube where it is false (the responses + // lane owns the full cube). + if (!p.compactionTrigger) { + assert.equal(got, legacyAnthropic(p), `anthropic cube ${mask}`); + assert.equal(got, legacyOpenaiGoogle(p), `openai/google cube ${mask}`); + } + // responses carries the trigger difference — its formula, its input. + assert.equal(got, legacyResponses(p), `responses cube ${mask}`); + checked++; + } + assert.equal(checked, 1024); +}); +// The three DELIBERATE differences must stay visible as input differences, +// not vanish into the gate: pin the discriminating combos. +test("nudgeOn: deliberate per-lane differences stay explicit (#3482)", () => { + const base: Primitives = { injectNudge: true, hasNudgePlan: true, shouldInject: true, compressReachable: true, compactionTrigger: false, suppressionBypassed: true, breakerArmed: false, autoFoldEngaged: false, growthFoldingArmed: false, modelWantsIt: true }; + // 1) responses: a compaction-trigger request never carries the nudge. + assert.equal(nudgeOn(laneInputs({ ...base, compactionTrigger: true })), false); + // other lanes never pass the flag (source shape pinned by the guard + // reading the prepares below). + // 2) injectTool subsumed: compressSurfaceOn false ⇒ no nudge, which is + // exactly the old standalone `shouldInject &&` conjunct. + assert.equal(nudgeOn(laneInputs({ ...base, shouldInject: false })), false); + // 3) codex-intercept health window feeds growthFoldingArmed precomputed — + // the gate only sees the boolean. + assert.equal(nudgeOn(laneInputs({ ...base, autoFoldEngaged: true, growthFoldingArmed: true })), false); + assert.equal(nudgeOn(laneInputs({ ...base, autoFoldEngaged: true, growthFoldingArmed: false })), true); +}); + +// #3482 site 1: billing-block policy in the registry, observation over +// defaults (any request observed carrying the #2189 block keeps it honored). +test("resolveLaneDecisions: billingBlockWanted = profile-declared OR wire-observed (#3482 site 1)", () => { + const billingSystem = [{ type: "text", text: "x-anthropic-billing-header: acct-123" }, { type: "text", text: "You are helpful." }]; + const plainSystem = [{ type: "text", text: "You are helpful." }]; + const input = (system: unknown, pluginAgent: string | undefined) => ({ protocol: "anthropic" as const, pluginMode: true, pluginAgent, parsed: { system }, shouldInject: true }); + // dsh declares it — even with no block on this wire. + const dsh = resolveLaneDecisions(input(plainSystem, "dsh")); + assert.equal(dsh.billingBlockWanted, true); + assert.equal(dsh.observed.billingBlockObserved, false); + // anonymous request observed carrying the block — honored regardless of + // profile (evidence over defaults, same shape as toolsReachable). + const anon = resolveLaneDecisions(input(billingSystem, undefined)); + assert.equal(anon.billingBlockWanted, true); + assert.equal(anon.observed.billingBlockObserved, true); + // string system can never carry the block (extraction needs the array). + const strSys = resolveLaneDecisions(input("You are helpful.", "dsh")); + assert.equal(strSys.observed.billingBlockObserved, false); + assert.equal(strSys.billingBlockWanted, true); // still dsh-declared + // pi with nothing on the wire — off. + const pi = resolveLaneDecisions(input(plainSystem, "pi")); + assert.equal(pi.billingBlockWanted, false); +}); + +// #3482 site 4: the wire-client registry decides who keys by instructions; +// session-id.ts only observes. +test("wire-client registry: instructionsFingerprintKeyed by observed signals (#3482 site 4)", () => { + assert.equal(resolveWireClientProfile({ headerNames: new Set(["x-codex-turn-metadata"]), conversationSourceName: undefined, codexUserAgent: false }).instructionsFingerprintKeyed, true); + assert.equal(resolveWireClientProfile({ headerNames: new Set(), conversationSourceName: undefined, codexUserAgent: true }).instructionsFingerprintKeyed, true); + assert.equal(resolveWireClientProfile({ headerNames: new Set(), conversationSourceName: "x-claude-code-session-id", codexUserAgent: false }).instructionsFingerprintKeyed, true); + assert.equal(resolveWireClientProfile({ headerNames: new Set(["x-acp-session"]), conversationSourceName: "x-acp-session", codexUserAgent: false }).identity, ""); // ANONYMOUS + assert.equal(resolveWireClientProfile({ headerNames: new Set(["x-acp-session"]), conversationSourceName: "x-acp-session", codexUserAgent: false }).instructionsFingerprintKeyed, false); + // through the session-id wrapper (observation → registry decision) + assert.equal(instructionsFingerprintApplies({ "x-codex-turn-metadata": "t=1" }), true); + assert.equal(instructionsFingerprintApplies({ "user-agent": "codex_cli_rs/0.1 (Linux; x86_64) Rust" }), true); + assert.equal(instructionsFingerprintApplies({ "x-claude-code-session-id": "9ae0…" }), true); + assert.equal(instructionsFingerprintApplies({ "x-claude-code-session-id": " " }), false); // blank ≠ signal + assert.equal(instructionsFingerprintApplies({ "x-acp-session": "abc" }), false); +}); diff --git a/tests/lane-decisions-matrix.test.ts b/tests/lane-decisions-matrix.test.ts index 4d6f07202e..33c531bfa8 100644 --- a/tests/lane-decisions-matrix.test.ts +++ b/tests/lane-decisions-matrix.test.ts @@ -69,6 +69,26 @@ function historyTexts(): string[] { } async function harness(wire: Wire, lane: Lane, toolNames: string[], injectTool = true) { + // Windows CI flake class (seen on runners, not on dev VMs): listen(0) + // occasionally lands in an OS-reserved/excluded port range and emits + // "error" (EACCES/EADDRINUSE) instead of "listening". events.once( + // server, "listening") REJECTS on error, so surface that as a bounded + // retry instead of letting a failed listen strand the test promise + // ("Promise resolution is still pending but the event loop has already + // resolved" — #3490 windows shards). + const listenWithRetry = async (server: http.Server) => { + for (let attempt = 1; ; attempt++) { + try { + server.listen(0, "127.0.0.1"); + await once(server, "listening"); + return; + } catch (err) { + const code = (err as NodeJS.ErrnoException).code; + if (attempt >= 5 || (code !== "EACCES" && code !== "EADDRINUSE")) throw err; + await new Promise((r) => setTimeout(r, 25 * attempt)); + } + } + }; _resetSessionsForTest(); _resetPluginStateForTest(); resetToolRingForTest(); @@ -92,7 +112,7 @@ async function harness(wire: Wire, lane: Lane, toolNames: string[], injectTool = }); }); upstream.listen(0, "127.0.0.1"); - await once(upstream, "listening"); + await listenWithRetry(upstream); const addr = upstream.address(); assert(addr && typeof addr === "object"); const upstreamUrl = `http://127.0.0.1:${addr.port}`; @@ -146,8 +166,11 @@ async function harness(wire: Wire, lane: Lane, toolNames: string[], injectTool = }; const close = async () => { store.cancelAll(); - proxy.close(); proxy.closeIdleConnections?.(); - upstream.close(); upstream.closeIdleConnections?.(); + // closeAllConnections destroys keep-alive sockets that close() alone + // would wait on forever (server.close only completes once every + // connection ends) — belt-and-suspenders against teardown hangs. + proxy.close(); proxy.closeIdleConnections?.(); proxy.closeAllConnections?.(); + upstream.close(); upstream.closeIdleConnections?.(); upstream.closeAllConnections?.(); await Promise.all([once(proxy, "close"), once(upstream, "close")]); }; return { send, forwarded, close }; @@ -211,7 +234,7 @@ const ROWS: { name: string; lane: Lane; tools: string[]; injectTool?: boolean; e for (const wire of ["anthropic", "openai", "responses", "google"] as const) { for (const row of ROWS) { - test(`lane matrix [${wire}] ${row.name}`, async () => { + test(`lane matrix [${wire}] ${row.name}`, { timeout: 60_000 }, async () => { const h = await harness(wire, row.lane, row.tools, row.injectTool); try { await h.send(["hello"]); // teach usage 50k / 64k = 78% @@ -231,7 +254,7 @@ for (const wire of ["anthropic", "openai", "responses", "google"] as const) { // pluginAgent against lane names — gating flows through lane-decisions // (resolveLaneDecisions). Header-evidence helpers (isCodexClient: codex sends // no x-bili-plugin header) are exempt by design. -test("lane matrix: prepares carry no raw pluginAgent name comparisons (#3417)", async () => { +test("lane matrix: prepares carry no raw pluginAgent name comparisons (#3417)", { timeout: 60_000 }, async () => { const root = new URL("../src/server/", import.meta.url); for (const f of ["prepare-anthropic.ts", "prepare-openai.ts", "prepare-google.ts", "prepare-responses.ts"]) { const src = await readFile(new URL(f, root), "utf8"); @@ -241,3 +264,40 @@ test("lane matrix: prepares carry no raw pluginAgent name comparisons (#3417)", assert.ok(!/(pluginAgent|pluginAgentHeader\([^)]*\))\s*===\s*"(dsh|pi|omp|opencode|codex|kimi)"/.test(src), `${f}: raw lane-name comparison — route it through resolveLaneDecisions (#3417)`); } }); + +// #3482 Step 2 structural guard: the four migrated B-class sites must stay +// name-gate-free — identity policy lives in the lane-decisions registries. +test("lane matrix: B-class sites carry no raw lane-name gates (#3482)", { timeout: 60_000 }, async () => { + const srcRoot = new URL("../src/", import.meta.url); + // Site 3 (omp body stamp): the stamp channel is a registry property. + // Brace-balanced slice (helpers between exports make anchor-based bounds + // under-slice/over-slice). + const piSrc = await readFile(new URL("agent/pi.ts", srcRoot), "utf8"); + const stampStart = piSrc.indexOf("export function stampPromptCacheKey"); + let depth = 0, stampEnd = -1; + for (let i = stampStart; i < piSrc.length; i++) { + if (piSrc[i] === "{") depth++; + else if (piSrc[i] === "}") { depth--; if (depth === 0) { stampEnd = i + 1; break; } } + } + const stampFn = piSrc.slice(stampStart, stampEnd < 0 ? undefined : stampEnd); + assert.ok(!/agent\s*[!=]==\s*"(omp|pi|dsh)"/.test(stampFn), "agent/pi.ts stampPromptCacheKey: raw agent-name gate — use hostProfileFor(agent).stampsBodyPromptCacheKey (#3482 site 3)"); + // Site 4 (instructions fingerprint): the wire-client header literals live + // ONLY in the registry — session-id.ts observes, never decides. + const sidSrc = await readFile(new URL("session-id.ts", srcRoot), "utf8"); + const fpFn = sidSrc.slice(sidSrc.indexOf("export function instructionsFingerprintApplies"), sidSrc.indexOf("}", sidSrc.indexOf("instructionsFingerprintKeyed"))); + assert.ok(!/x-codex-turn-metadata|x-claude-code-session-id/.test(fpFn), "session-id.ts instructionsFingerprintApplies: raw wire-client header literal — the decision table lives in WIRE_CLIENTS (#3482 site 4)"); + // Registry hygiene: every wire-client entry's matches() must reference the + // signals it matches on (guards against an entry rotting into a copy-paste + // of another's condition). + const laneSrc = await readFile(new URL("server/lane-decisions.ts", srcRoot), "utf8"); + const registryBlock = laneSrc.slice(laneSrc.indexOf("const WIRE_CLIENTS"), laneSrc.indexOf("resolveWireClientProfile")); + const entryCount = (registryBlock.match(/profile:\s*\{/g) ?? []).length; + const matchCount = (registryBlock.match(/matches:\s*\(/g) ?? []).length; + assert.equal(entryCount, matchCount, "lane-decisions.ts WIRE_CLIENTS: every entry needs a matches() (#3482)"); + // #3482 review: the profile registry must stay an IMPORT-FREE leaf — + // agent-side bundles (dist/agent/pi.js, omp.js) import it directly; any + // import here would drag server modules (preflight → kernel) into the + // client-process bundles (first cut of this PR doubled them). + const hpSrc = await readFile(new URL("host-profiles.ts", srcRoot), "utf8"); + assert.ok(!/^import /m.test(hpSrc), "src/host-profiles.ts must stay import-free (#3482 review)"); +}); diff --git a/tests/launcher-plugin-mode.test.ts b/tests/launcher-plugin-mode.test.ts index 31447b2fc9..4e98c07ee3 100644 --- a/tests/launcher-plugin-mode.test.ts +++ b/tests/launcher-plugin-mode.test.ts @@ -338,7 +338,7 @@ test("mcp stdio shell: manifest → tools/list → tools/call forwards to the pl const init = byId(1) as { result?: { serverInfo?: { name?: string } } }; assert.equal(init.result?.serverInfo?.name, "bili"); const tools = byId(2) as { result?: { tools?: { name: string }[] } }; - assert.deepEqual(tools.result?.tools?.map((t) => t.name).sort(), ["acp_cache", "acp_status", "compress", "decompress", "search_context"]); + assert.deepEqual(tools.result?.tools?.map((t) => t.name).sort(), ["acp_cache", "acp_status", "compress", "decompress", "read_original_image", "search_context"]); const call = byId(3) as { result?: { content?: { text?: string }[]; isError?: boolean } }; assert.equal(call.result?.isError, false); assert.match(call.result?.content?.[0]?.text ?? "", /CONTEXT BREAKDOWN/, "acp_status result forwarded verbatim"); @@ -409,7 +409,7 @@ test("mcp stdio shell (codex style): BILI_CONVERSATION_ID self-register → head const byId = (n: number): Record => JSON.parse(lines.find((l) => (JSON.parse(l) as { id?: number }).id === n) ?? "{}"); const tools = byId(2) as { result?: { tools?: { name: string }[] } }; - assert.deepEqual(tools.result?.tools?.map((t) => t.name).sort(), ["acp_cache", "acp_status", "compress", "decompress", "search_context"], "tools listed"); + assert.deepEqual(tools.result?.tools?.map((t) => t.name).sort(), ["acp_cache", "acp_status", "compress", "decompress", "read_original_image", "search_context"], "tools listed"); const call = byId(3) as { result?: { content?: { text?: string }[]; isError?: boolean } }; assert.equal(call.result?.isError, false, `tools/call succeeded (self-registered conversation bound)${call.result?.isError ? ": " + (call.result?.content?.[0]?.text ?? "") : ""}`); assert.match(call.result?.content?.[0]?.text ?? "", /CONTEXT BREAKDOWN/); diff --git a/tests/lineage-reconcile-boot.test.ts b/tests/lineage-reconcile-boot.test.ts index 6262f58107..7c5a1d637f 100644 --- a/tests/lineage-reconcile-boot.test.ts +++ b/tests/lineage-reconcile-boot.test.ts @@ -458,6 +458,18 @@ test("AC12: a multi-hop grandchild stays eligible when its intermediate parent h // blocks and legally adopt the grandchild, closing its gate. assert.ok(getStore().flushSync(s), `post-strip persist of ${id} must land`); } + // The scenario under test needs the sweep to reach the grandchild + // BEFORE the child adopts. listSessions() orders by lastSeen desc, and + // the two turns' timestamps are ambient wall-clock: a coarse clock or + // a reload path restamping lastSeen can tie or flip them (observed + // deterministically on the windows-latest CI lanes). Once the child is + // visited first it legitimately adopts from the root, and the + // grandchild's adoption from a now-block-bearing parent is CORRECT + // behavior (#2807 contract) — failing this assertion for the wrong + // reason. Pin the order instead of leaning on clock luck. + const pinnedChild = listSessions().find((x) => x.id === childId)!; + const pinnedGrandchild = listSessions().find((x) => x.id === grandchildId)!; + pinnedGrandchild.lastSeen = Math.max(pinnedGrandchild.lastSeen, pinnedChild.lastSeen + 60_000); } finally { await close(proxy); await close(relay); diff --git a/tests/native-intercept.test.ts b/tests/native-intercept.test.ts index 511ff6090e..4ec5be2c08 100644 --- a/tests/native-intercept.test.ts +++ b/tests/native-intercept.test.ts @@ -408,6 +408,44 @@ test("install: TypeError triggers one respawn + retry", async () => { } }); +test("install: onResponse carries THIS dispatch's outgoing headers, per request (#3329)", async () => { + const seen: Array<{ url: string; headers?: Record }> = []; + let n = 0; + const state: NativeInterceptState = { + origin: "http://127.0.0.1:40001", + ready: Promise.resolve("http://127.0.0.1:40001"), + beforeSend: async (_url, init) => { + // Stamp a per-request correlation id, exactly like the + // incremental adapters do. + n += 1; + if (init !== undefined) { + const h = new Headers(init.headers); + h.set("x-bili-turn-id", `t-${n}`); + init.headers = h; + } + }, + onResponse: (url, _resp, headers) => { seen.push({ url, headers }); }, + }; + await withPatchRecording(state, async (fetch) => { + // Two dispatches on the SAME provider URL (string + init shape — the + // dsh/omp lanes): each must get back its OWN stamped headers. + await fetch("http://127.0.0.1:8199/v1/messages", { method: "POST", headers: { "content-type": "application/json" }, body: "{}" }); + await fetch("http://127.0.0.1:8199/v1/messages", { method: "POST", headers: { "content-type": "application/json", "x-extra": "second" }, body: "{}" }); + // Request-object shape (the opencode lane): the host stamps the + // correlation id onto the Request itself before fetch. + const req = new Request("http://127.0.0.1:8199/v1/messages", { method: "POST", headers: { "x-req": "r", "x-bili-turn-id": "t-pre" }, body: "{}" }); + await fetch(req); + }); + assert.equal(seen.length, 3, "onResponse fired once per dispatch"); + for (const s of seen) assert.equal(s.url, "http://127.0.0.1:8199/v1/messages", "host-requested url stays the key context"); + assert.equal(seen[0].headers?.["x-bili-turn-id"], "t-1"); + assert.equal(seen[0].headers?.["content-type"], "application/json", "original outgoing headers ride along"); + assert.equal(seen[1].headers?.["x-bili-turn-id"], "t-2", "the second same-URL response carries the second dispatch's id, not the first's"); + assert.equal(seen[1].headers?.["x-extra"], "second"); + assert.equal(seen[2].headers?.["x-bili-turn-id"], "t-pre", "Request-object inputs carry their own pre-stamped id through the rewrite"); + assert.equal(seen[2].headers?.["x-req"], "r"); +}); + test("install: second install is a no-op while active", async () => { const state: NativeInterceptState = { origin: "http://127.0.0.1:40001", ready: Promise.resolve("http://127.0.0.1:40001") }; const saved = globalThis.fetch; diff --git a/tests/omp-native-incremental.test.ts b/tests/omp-native-incremental.test.ts index 1a5fa8d4dd..fb39afa649 100644 --- a/tests/omp-native-incremental.test.ts +++ b/tests/omp-native-incremental.test.ts @@ -2,7 +2,10 @@ // events, so the ONLY identity signal at the fetch layer is prompt_cache_key // (stamped into the body by the omp extension). These suites drive the two // exported seams (ompIncrementalBeforeSend / ompIncrementalOnResponse) with a -// fake lane origin pinned via _setOmpOriginForTest. +// fake lane origin pinned via _setOmpOriginForTest. #3329: ack calls pass the +// OUTGOING headers of the very request being acknowledged (the init.headers +// the plan stamped, exactly as native-intercept threads them into onResponse) +// — attribution is by the per-request turn id, never by URL. import { after, test } from "node:test"; import assert from "node:assert/strict"; import { createServer } from "node:http"; @@ -33,6 +36,14 @@ function initOf(body: unknown): RequestInit { return { method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify(body) }; } +/** The outgoing headers of a planned init, normalized exactly like + * native-intercept's outgoingHeadersOf does (init.headers wins). */ +function headersOf(h: Headers): Record { + const out: Record = {}; + for (const [k, v] of h) out[k.toLowerCase()] = v; + return out; +} + const lanes: { close: () => Promise }[] = []; after(async () => { for (const l of lanes) await l.close(); @@ -59,8 +70,9 @@ test("omp incremental: prompt_cache_key becomes the conversation id; plugin head assert.equal(headers.get("x-bili-plugin"), "omp"); assert.equal(headers.get("x-bili-plugin-conversation"), "omp-inc-1"); assert.equal(headers.get("x-bili-plugin-turn"), "1"); + assert.notEqual(headers.get("x-bili-turn-id"), null, "turn id stamped on the outgoing init (#3329)"); - ompIncrementalOnResponse(MODEL_URL, new Response("{}", { status: 200, headers: { "x-bili-applied-seq": "1" } })); + ompIncrementalOnResponse(MODEL_URL, new Response("{}", { status: 200, headers: { "x-bili-applied-seq": "1" } }), headersOf(headers)); const msgs2 = [...msgs1, { role: "assistant", content: "hi" }, { role: "user", content: "next" }]; const init2 = initOf({ model: "claude-test", max_tokens: 512, prompt_cache_key: "omp-inc-1", messages: msgs2 }); @@ -94,3 +106,87 @@ test("omp incremental: probe disabled leaves the body untouched", async () => { await ompIncrementalBeforeSend(MODEL_URL, init); assert.deepEqual(JSON.parse(String(init.body)), original); }); + +test("omp incremental: late response crossing the TTL attributes by turn id, never by URL (#3329)", async () => { + __resetIncrementalClientForTests(); + __resetOmpIncrementalForTests(); + const lane = await fakeLane(true); + lanes.push(lane); + _setOmpOriginForTest(lane.base); + // Two conversations sharing ONE provider URL and the SAME host system + // prompt — the shape #3329 reports in the field. Sequenced for the #3378 + // single-flight guard (kept over #3329's removal per the #3340 rebase + // decision): while a marker'd request for this URL is un-acked, a second + // conversion on the same URL rides full resend — so B's post-ack turn + // can only be exercised after B's OWN ack, and A's reseed after B's + // pending is consumed. Every original assertion intent survives the + // reordering (documented inline). + const system = "be terse"; + const a1msgs = [{ role: "user", content: "a1" }]; + const b1msgs = [{ role: "user", content: "b1" }]; + + const realNow = Date.now; + let t = realNow(); + Date.now = () => t; + try { + const initA1 = initOf({ model: "claude-test", max_tokens: 512, system, prompt_cache_key: "omp-late-a", messages: a1msgs }); + await ompIncrementalBeforeSend(MODEL_URL, initA1); + const aHeaders = headersOf(initA1.headers as Headers); + assert.notEqual(aHeaders["x-bili-turn-id"], undefined, "turn id stamped on A's outgoing init"); + + // A's round-trip outlives the pending TTL (slow non-streaming request) + // — the sweep frees the URL slot, so B converts under the guard. + t += 31_000; + const initB1 = initOf({ model: "claude-test", max_tokens: 512, system, prompt_cache_key: "omp-late-b", messages: b1msgs }); + await ompIncrementalBeforeSend(MODEL_URL, initB1); + const bHeaders = headersOf(initB1.headers as Headers); + assert.notEqual(bHeaders["x-bili-turn-id"], undefined, "turn id stamped on B's outgoing init"); + assert.notEqual(bHeaders["x-bili-turn-id"], aHeaders["x-bili-turn-id"], "each dispatch carries its own correlation id"); + const b1Body = JSON.parse(String(initB1.body)) as Record; + assert.equal(b1Body.sinceSeq, 0, "B's first turn seeds (not credited with anything yet)"); + assert.equal(b1Body.resync, true, "B's first turn seeds"); + assert.equal(b1Body.system, system, "seed carries B's own system — no taint from A"); + + // A's LATE response lands after B planned on the same URL. The old + // URL-keyed table credited this ack to B; now A's entry was swept and + // the response carries no live turn id — it cannot touch either state + // machine except through its own (expired) id. + ompIncrementalOnResponse(MODEL_URL, new Response("{}", { status: 200, headers: { "x-bili-applied-seq": "42" } }), aHeaders); + + // B's OWN response then credits exactly B — and the delta it plans + // rides watermark 7, never 42: if A's late ack had been attributed to + // B (the URL-keyed bug), this sinceSeq would read 42. + ompIncrementalOnResponse(MODEL_URL, new Response("{}", { status: 200, headers: { "x-bili-applied-seq": "7" } }), bHeaders); + const b2msgs = [...b1msgs, { role: "assistant", content: "b1r" }, { role: "user", content: "b2" }]; + const initB2 = initOf({ model: "claude-test", max_tokens: 512, system, prompt_cache_key: "omp-late-b", messages: b2msgs }); + await ompIncrementalBeforeSend(MODEL_URL, initB2); + const b2Body = JSON.parse(String(initB2.body)) as Record; + assert.equal(b2Body.sinceSeq, 7, "B was never credited with A's ack (42) — it rides its own (7)"); + assert.equal((b2Body.messages as unknown[]).length, 2, "delta only — B's own watermark carries"); + + // B's chain keeps riding its own acks: ack b2 FIRST — the #3378 + // single-flight guard holds the URL slot until b2's ack lands. + ompIncrementalOnResponse(MODEL_URL, new Response("{}", { status: 200, headers: { "x-bili-applied-seq": "9" } }), headersOf(initB2.headers as Headers)); + + // A's expired turn is an honest loss: A reseeds instead of riding a + // watermark it never actually received (b2's pending was consumed by + // its own ack above, so the URL slot is free again). + const a2msgs = [...a1msgs, { role: "assistant", content: "a1r" }, { role: "user", content: "a2" }]; + const initA2 = initOf({ model: "claude-test", max_tokens: 512, system, prompt_cache_key: "omp-late-a", messages: a2msgs }); + await ompIncrementalBeforeSend(MODEL_URL, initA2); + const a2Body = JSON.parse(String(initA2.body)) as Record; + assert.equal(a2Body.sinceSeq, 0, "A's expired entry was swept — honest reseed"); + assert.equal(a2Body.resync, true, "A reseeds wholesale, not on a watermark it never saw"); + + // A's reseed acks on ITS OWN turn id (a1's expired id stays dead); + // B's next delta is then free to convert under the guard. + ompIncrementalOnResponse(MODEL_URL, new Response("{}", { status: 200, headers: { "x-bili-applied-seq": "11" } }), headersOf(initA2.headers as Headers)); + const b3msgs = [...b2msgs, { role: "assistant", content: "b2r" }, { role: "user", content: "b3" }]; + const initB3 = initOf({ model: "claude-test", max_tokens: 512, system, prompt_cache_key: "omp-late-b", messages: b3msgs }); + await ompIncrementalBeforeSend(MODEL_URL, initB3); + const b3Body = JSON.parse(String(initB3.body)) as { sinceSeq: number }; + assert.equal(b3Body.sinceSeq, 9, "B rides its own watermark"); + } finally { + Date.now = realNow; + } +}); diff --git a/tests/opencode-incremental.test.ts b/tests/opencode-incremental.test.ts index a3f475a521..c9d20a9112 100644 --- a/tests/opencode-incremental.test.ts +++ b/tests/opencode-incremental.test.ts @@ -2,8 +2,11 @@ // seams directly (maybeIncrementalOpencode / opencodeIncrementalOnResponse) // against a fake lane origin: the probe endpoint decides enablement, the // body/marker assertions pin the divert contract, and the ack→delta pair -// pins the state machine ride-along. The full e2e (marker divert through -// the real server) lives in incremental-lane.test.ts. +// pins the state machine ride-along. #3329: ack calls pass the OUTGOING +// headers of the very request being acknowledged (read back off the swapped +// Request, exactly as native-intercept threads them into onResponse) — +// attribution is by the per-request turn id, never by URL. The full e2e +// (marker divert through the real server) lives in incremental-lane.test.ts. import { after, test } from "node:test"; import assert from "node:assert/strict"; import { createServer } from "node:http"; @@ -45,6 +48,14 @@ function state(base: string): V2State { return { proxyBase: base } as unknown as V2State; } +/** The outgoing headers of a swapped Request, normalized exactly like + * native-intercept's outgoingHeadersOf does for a Request-object input. */ +function headersOf(req: Request): Record { + const out: Record = {}; + for (const [k, v] of req.headers) out[k.toLowerCase()] = v; + return out; +} + const lanes: { close: () => Promise }[] = []; after(async () => { for (const l of lanes) await l.close(); @@ -72,8 +83,9 @@ test("opencode incremental: seed swap (marker + shaped body, headers preserved), assert.equal(swapped.headers.get("x-bili-plugin-turn"), "1", "marker stamped on the replaced request"); assert.equal(swapped.headers.get("authorization"), "Bearer sk-test", "original headers survive the swap"); - // ack advances the helper state machine - opencodeIncrementalOnResponse(url, new Response("{}", { status: 200, headers: { "x-bili-applied-seq": "1" } })); + // ack advances the helper state machine (attributed by this request's own turn id) + opencodeIncrementalOnResponse(url, new Response("{}", { status: 200, headers: { "x-bili-applied-seq": "1" } }), headersOf(swapped)); + assert.notEqual(headersOf(swapped)["x-bili-turn-id"], undefined, "turn id stamped on the outgoing request"); const msgs2 = [...msgs1, { role: "assistant", content: "hi" }, { role: "user", content: "next" }]; const t2 = fakeEvent({ model: "claude-test", max_tokens: 512, system: "be terse", messages: msgs2 }, "oc-inc-1"); @@ -119,14 +131,14 @@ test("opencode incremental: #3369 — one 409 re-seeds the next turn; three cons const msgs1 = [{ role: "user", content: "hello" }]; const t1 = fakeEvent({ model: "claude-test", max_tokens: 512, messages: msgs1 }, "oc-inc-dead"); await maybeIncrementalOpencode(t1.e, s); - opencodeIncrementalOnResponse(t1.url, new Response("{}", { status: 200, headers: { "x-bili-applied-seq": "1" } })); + opencodeIncrementalOnResponse(t1.url, new Response("{}", { status: 200, headers: { "x-bili-applied-seq": "1" } }), headersOf(t1.e.request as Request)); const msgs2 = [...msgs1, { role: "assistant", content: "hi" }, { role: "user", content: "again" }]; const t2 = fakeEvent({ model: "claude-test", max_tokens: 512, messages: msgs2 }, "oc-inc-dead"); await maybeIncrementalOpencode(t2.e, s); const t2Body = JSON.parse(await (t2.e.request as Request).clone().json().then((j) => JSON.stringify(j))) as { sinceSeq: number }; assert.equal(t2Body.sinceSeq, 1, "delta planned"); // the delta was parsed from the CLONED request — ack with the ORIGINAL t2 url - opencodeIncrementalOnResponse(t2.url, new Response("{}", { status: 409, headers: { "x-bili-server-seq": "1" } })); + opencodeIncrementalOnResponse(t2.url, new Response("{}", { status: 409, headers: { "x-bili-server-seq": "1" } }), headersOf(t2.e.request as Request)); const original = { model: "claude-test", max_tokens: 512, messages: [...msgs2, { role: "user", content: "more" }] }; const t3 = fakeEvent(original, "oc-inc-dead"); await maybeIncrementalOpencode(t3.e, s); @@ -137,11 +149,11 @@ test("opencode incremental: #3369 — one 409 re-seeds the next turn; three cons // the re-seed landed and was acked (circuit cleared) — now trip it for // good: three consecutive 409s with no ack between → sticky fallback. - opencodeIncrementalOnResponse(t3.url, new Response("{}", { status: 200, headers: { "x-bili-applied-seq": String(original.messages.length) } })); + opencodeIncrementalOnResponse(t3.url, new Response("{}", { status: 200, headers: { "x-bili-applied-seq": String(original.messages.length) } }), headersOf(t3.e.request as Request)); for (let i = 0; i < 3; i++) { const t = fakeEvent({ model: "claude-test", max_tokens: 512, messages: [...original.messages, { role: "user", content: `x${i}` }] }, "oc-inc-dead"); await maybeIncrementalOpencode(t.e, s); - opencodeIncrementalOnResponse(t.url, new Response("{}", { status: 409, headers: { "x-bili-server-seq": "1" } })); + opencodeIncrementalOnResponse(t.url, new Response("{}", { status: 409, headers: { "x-bili-server-seq": "1" } }), headersOf(t.e.request as Request)); } const last = { model: "claude-test", max_tokens: 512, messages: [...original.messages, { role: "user", content: "final" }] }; const t4 = fakeEvent(last, "oc-inc-dead"); @@ -161,11 +173,14 @@ test("opencode incremental: a 200 without an ack header does not steal the pendi await maybeIncrementalOpencode(t1.e, s); // seed — pending entry set for the model URL // a concurrent full-resend request on the same URL answers first WITHOUT - // the ack header — it must not consume the marker'd turn's pending slot + // the ack header — it must not consume the marker'd turn's pending slot. + // #3329: the 2-arg call carries no outgoing headers at all — a response + // with no turn-id header is not an incremental ack and consumes nothing. opencodeIncrementalOnResponse(t1.url, new Response("{}", { status: 200 })); - // the marker'd turn's own 200 still records its ack - opencodeIncrementalOnResponse(t1.url, new Response("{}", { status: 200, headers: { "x-bili-applied-seq": "1" } })); + // the marker'd turn's own 200 still records its ack (attributed by its + // own turn id, threaded from its outgoing headers) + opencodeIncrementalOnResponse(t1.url, new Response("{}", { status: 200, headers: { "x-bili-applied-seq": "1" } }), headersOf(t1.e.request as Request)); // ...so the next turn rides the delta instead of re-seeding const msgs2 = [...msgs1, { role: "assistant", content: "hi" }, { role: "user", content: "next" }]; @@ -175,3 +190,87 @@ test("opencode incremental: a 200 without an ack header does not steal the pendi assert.equal(body2.sinceSeq, 1, "ack survived the interleaved headerless 200"); assert.equal((body2.messages as unknown[]).length, 2, "delta rides — no re-seed after the interleaving"); }); + +test("opencode incremental: late response crossing the TTL attributes by turn id, never by URL (#3329)", async () => { + __resetIncrementalClientForTests(); + __resetOpencodeIncrementalForTests(); + const lane = await fakeLane(true); + lanes.push(lane); + const s = state(lane.base); + // Two conversations sharing ONE provider URL and the SAME host system + // prompt — the shape #3329 reports in the field. Sequenced for the #3378 + // single-flight guard (kept over #3329's removal per the #3340 rebase + // decision): while a marker'd request for this URL is un-acked, a second + // conversion on the same URL rides full resend — so B's post-ack turn + // can only be exercised after B's OWN ack, and A's reseed after B's + // pending is consumed. Every original assertion intent survives the + // reordering (documented inline). + const system = "be terse"; + const a1msgs = [{ role: "user", content: "a1" }]; + const b1msgs = [{ role: "user", content: "b1" }]; + + const realNow = Date.now; + let t = realNow(); + Date.now = () => t; + try { + const a1 = fakeEvent({ model: "claude-test", max_tokens: 512, system, messages: a1msgs }, "oc-late-a"); + await maybeIncrementalOpencode(a1.e, s); + const aHeaders = headersOf(a1.e.request as Request); + assert.notEqual(aHeaders["x-bili-turn-id"], undefined, "turn id stamped on A's outgoing request"); + + // A's round-trip outlives the pending TTL (slow non-streaming request) + // — the sweep frees the URL slot, so B converts under the guard. + t += 31_000; + const b1 = fakeEvent({ model: "claude-test", max_tokens: 512, system, messages: b1msgs }, "oc-late-b"); + await maybeIncrementalOpencode(b1.e, s); + const bHeaders = headersOf(b1.e.request as Request); + assert.notEqual(bHeaders["x-bili-turn-id"], undefined, "turn id stamped on B's outgoing request"); + assert.notEqual(bHeaders["x-bili-turn-id"], aHeaders["x-bili-turn-id"], "each dispatch carries its own correlation id"); + const b1Body = JSON.parse(await (b1.e.request as Request).clone().json().then((j) => JSON.stringify(j))) as Record; + assert.equal(b1Body.sinceSeq, 0, "B's first turn seeds (not credited with anything yet)"); + assert.equal(b1Body.resync, true, "B's first turn seeds"); + assert.equal(b1Body.system, system, "seed carries B's own system — no taint from A"); + + // A's LATE response lands after B planned on the same URL. The old + // URL-keyed table credited this ack to B; now A's entry was swept and + // the response carries no live turn id — it cannot touch either state + // machine except through its own (expired) id. + opencodeIncrementalOnResponse(MODEL_URL, new Response("{}", { status: 200, headers: { "x-bili-applied-seq": "42" } }), aHeaders); + + // B's OWN response then credits exactly B — and the delta it plans + // rides watermark 7, never 42: if A's late ack had been attributed to + // B (the URL-keyed bug), this sinceSeq would read 42. + opencodeIncrementalOnResponse(MODEL_URL, new Response("{}", { status: 200, headers: { "x-bili-applied-seq": "7" } }), bHeaders); + const b2msgs = [...b1msgs, { role: "assistant", content: "b1r" }, { role: "user", content: "b2" }]; + const b2 = fakeEvent({ model: "claude-test", max_tokens: 512, system, messages: b2msgs }, "oc-late-b"); + await maybeIncrementalOpencode(b2.e, s); + const b2Body = JSON.parse(await (b2.e.request as Request).clone().json().then((j) => JSON.stringify(j))) as Record; + assert.equal(b2Body.sinceSeq, 7, "B was never credited with A's ack (42) — it rides its own (7)"); + assert.equal((b2Body.messages as unknown[]).length, 2, "delta only — B's own watermark carries"); + + // B's chain keeps riding its own acks: ack b2 FIRST — the #3378 + // single-flight guard holds the URL slot until b2's ack lands. + opencodeIncrementalOnResponse(MODEL_URL, new Response("{}", { status: 200, headers: { "x-bili-applied-seq": "9" } }), headersOf(b2.e.request as Request)); + + // A's expired turn is an honest loss: A reseeds instead of riding a + // watermark it never actually received (b2's pending was consumed by + // its own ack above, so the URL slot is free again). + const a2msgs = [...a1msgs, { role: "assistant", content: "a1r" }, { role: "user", content: "a2" }]; + const a2 = fakeEvent({ model: "claude-test", max_tokens: 512, system, messages: a2msgs }, "oc-late-a"); + await maybeIncrementalOpencode(a2.e, s); + const a2Body = JSON.parse(await (a2.e.request as Request).clone().json().then((j) => JSON.stringify(j))) as Record; + assert.equal(a2Body.sinceSeq, 0, "A's expired entry was swept — honest reseed"); + assert.equal(a2Body.resync, true, "A reseeds wholesale, not on a watermark it never saw"); + + // A's reseed acks on ITS OWN turn id (a1's expired id stays dead); + // B's next delta is then free to convert under the guard. + opencodeIncrementalOnResponse(MODEL_URL, new Response("{}", { status: 200, headers: { "x-bili-applied-seq": "11" } }), headersOf(a2.e.request as Request)); + const b3msgs = [...b2msgs, { role: "assistant", content: "b2r" }, { role: "user", content: "b3" }]; + const b3 = fakeEvent({ model: "claude-test", max_tokens: 512, system, messages: b3msgs }, "oc-late-b"); + await maybeIncrementalOpencode(b3.e, s); + const b3Body = JSON.parse(await (b3.e.request as Request).clone().json().then((j) => JSON.stringify(j))) as { sinceSeq: number }; + assert.equal(b3Body.sinceSeq, 9, "B rides its own watermark"); + } finally { + Date.now = realNow; + } +}); diff --git a/tests/opencode-native.test.ts b/tests/opencode-native.test.ts index c69c2aa4c6..23f51b4647 100644 --- a/tests/opencode-native.test.ts +++ b/tests/opencode-native.test.ts @@ -29,14 +29,14 @@ after(() => { import type { NativeInterceptState } from "../src/agent/native-intercept.ts"; import type { V2HttpRequestEvent, V2PluginContext, V2State } from "../src/agent/opencode-v2.ts"; -import { ACP_TOOLS_OPENAI } from "../src/compress-tool.ts"; +import { ACP_TOOLS_OPENAI, ABSORB_TOOL_OPENAI } from "../src/compress-tool.ts"; const { shouldBootstrapNativeOpencode, createNativeRoute, planNativeOpencode, armNativeOpencode, verifyAttachAndRecover, _resetNativeStateForTest, _setSpawnForTest, _stateRespawnForTest, _noteRoutedForTest } = await import("../src/agent/opencode-native.ts"); const { createOpencodeV2Setup } = await import("../src/agent/opencode-v2.ts"); const { markNativeHost, nativeAttachOrigin } = await import("../src/agent/native-bootstrap.ts"); const nativeDefault = (await import("../src/agent/opencode-native.ts")).default; -const EXPECTED_TOOLS = [...ACP_TOOLS_OPENAI.map((t) => t.function.name)]; +const EXPECTED_TOOLS = [...ACP_TOOLS_OPENAI.map((t) => t.function.name), ABSORB_TOOL_OPENAI.function.name]; const MODEL_URL = "https://api.anthropic.com/v1/messages"; const sleep = (ms: number): Promise => new Promise((r) => setTimeout(r, ms)); diff --git a/tests/opencode-v2.test.ts b/tests/opencode-v2.test.ts index a512f55d4e..798a80c771 100644 --- a/tests/opencode-v2.test.ts +++ b/tests/opencode-v2.test.ts @@ -9,10 +9,10 @@ import test from "node:test"; process.env.NODE_ENV = "test"; import biliOpencodePlugin from "../src/agent/opencode.ts"; -import { ACP_TOOLS_OPENAI, RETRIEVE_TOOL_NAME, retrieveToolsFor } from "../src/compress-tool.ts"; +import { ACP_TOOLS_OPENAI, ABSORB_TOOL_OPENAI, RETRIEVE_TOOL_NAME, retrieveToolsFor } from "../src/compress-tool.ts"; import { fetchManifest, fitNoticeDescription } from "../src/agent/shared.ts"; -const EXPECTED_TOOLS = [...ACP_TOOLS_OPENAI.map((t) => t.function.name)]; +const EXPECTED_TOOLS = [...ACP_TOOLS_OPENAI.map((t) => t.function.name), ABSORB_TOOL_OPENAI.function.name]; const LONG_PANEL = Array.from({ length: 30 }, (_, i) => `panel line ${String(i + 1).padStart(2, "0")}: ${"x".repeat(38)}`).join("\n"); const LONG_REPORT = Array.from({ length: 200 }, (_, i) => `report line ${String(i + 1).padStart(3, "0")}: ${"y".repeat(38)}`).join("\n"); @@ -230,7 +230,7 @@ test("v2 setup: registers the bundled bili tools synchronously with exact schema assert.equal(t.options?.codemode, false); assert.equal(t.options?.permission, "allow"); } - const compressSrc = [...ACP_TOOLS_OPENAI].find((t) => t.function.name === "compress")!; + const compressSrc = [...ACP_TOOLS_OPENAI, ABSORB_TOOL_OPENAI].find((t) => t.function.name === "compress")!; assert.deepEqual(fake.addedTools.find((t) => t.name === "compress")!.input, compressSrc.function.parameters); } finally { cleanup(); diff --git a/tests/persist-guard-ownership.test.ts b/tests/persist-guard-ownership.test.ts index 3f47ee2035..0443bad178 100644 --- a/tests/persist-guard-ownership.test.ts +++ b/tests/persist-guard-ownership.test.ts @@ -8,18 +8,16 @@ import type { Session } from "../src/session.ts"; import { createInitialState } from "acp-kernel"; import { rmrf } from "./tmp-rm.ts"; -function sleepSync(ms: number): void { - Atomics.wait(new Int32Array(new SharedArrayBuffer(4)), 0, 0, ms); -} - -// #3049: the stale-disk guard's ownership fast path. The guard protects the -// shared sessions dir from counter rollback (#405/#2401); it used to re-read +// #3049/#3406: the stale-disk guard's ownership fast path. The guard protects +// the shared sessions dir from counter rollback (#405/#2401); it used to re-read // and re-parse the ENTIRE envelope on every save for two counters. In // shadow/full mode the fast path proves "no other writer committed since our -// last build" with (unified-row counters === our token) + (envelope mtime not -// newer than our build) and skips the read. These pins cover the collapse AND -// the three escape hatches (cross-instance commit, foreign file write, legacy -// exactness). +// last landed write" with (unified-row counters === our token) + (envelope +// still byte-identical to the post-write stamp: same size, mtime not newer — +// #3406 replaced the old builtAtMs+1.5s grace window, which missed foreign +// writes landing inside it) and skips the read. These pins cover the collapse +// AND the escape hatches (cross-instance commit, foreign file write, the +// #3406 in-window race, legacy exactness). function makeSession(id: string): Session { return { @@ -52,6 +50,8 @@ function withMode(mode: "shadow" | "legacy", name: string, fn: (dir: string) => }) as unknown as Promise; } +const sleep = (ms: number): Promise => new Promise((r) => setTimeout(r, ms)); + function envelopeFileOf(dir: string, id: string): string { for (const d of readdirSync(dir, { withFileTypes: true }).filter((e) => e.isDirectory() && e.name !== "index.db")) { for (const f of readdirSync(join(dir, d.name))) { @@ -124,7 +124,7 @@ await withMode("shadow", "guard ownership: a sibling instance's commit is detect storeB.closeQuietly(); }); -await withMode("shadow", "guard ownership: a foreign file write (legacy sibling / manual edit) trips the mtime check", async (dir) => { +await withMode("shadow", "guard ownership: a foreign file write (legacy sibling / manual edit) trips the landed-write stamp check", async (dir) => { const warnings: string[] = []; const log = (level: string, msg: string): void => { if (level === "warn") warnings.push(msg); @@ -134,9 +134,9 @@ await withMode("shadow", "guard ownership: a foreign file write (legacy sibling s.stats.requests = 3; await store.writeNow(s); - // Push the foreign write outside the ownership grace window so the mtime - // check is deterministic in-test (1.5s slack for our own write latency). - sleepSync(1700); + // No sleep needed: the stamp ("written") records the size+mtime of OUR + // landed write, so ANY foreign bytes — regardless of when they land — + // mismatch the stamp and force the full read. const file = envelopeFileOf(dir, "foreign"); const parsed = JSON.parse(readFileSync(file, "utf8")) as { payload: { stats: { requests: number }; metadata: Record } }; parsed.payload.stats.requests = 7; // a legacy sibling that never touches index.db @@ -154,6 +154,94 @@ await withMode("shadow", "guard ownership: a foreign file write (legacy sibling store.closeQuietly(); }); +await withMode("shadow", "guard ownership: #3406 — a foreign write INSIDE the old grace window no longer clobbers the envelope", async (dir) => { + const warnings: string[] = []; + const log = (level: string, msg: string): void => { + if (level === "warn") warnings.push(msg); + }; + const store = new SessionStore({ dir, debounceMs: 1, enabled: true, log }); + const s = makeSession("in-window"); + s.stats.requests = 3; + await store.writeNow(s); + const fullReadsBefore = store.__staleGuardFullReadsForTests(); + + // The #3406 item-2 race: a legacy-mode sibling (sidecar disabled, never + // touches index.db) writes a NEWER envelope milliseconds after ours — + // inside the 1.5s grace window the old #3286 fast path trusted blindly + // (mtime was only compared against BUILD time, and no counter moved), so + // the next save skipped the full read and clobbered the sibling's file. + const file = envelopeFileOf(dir, "in-window"); + const parsed = JSON.parse(readFileSync(file, "utf8")) as { payload: { stats: { requests: number }; metadata: Record } }; + parsed.payload.stats.requests = 9; + parsed.payload.metadata.lineage = "sibling-newer"; + writeFileSync(file, JSON.stringify(parsed)); + + // Same in-memory counters — only the landed-write stamp can catch it. + s.stats.requests = 3; + await store.writeNow(s); + + assert.equal(store.__staleGuardFullReadsForTests(), fullReadsBefore + 1, "size mismatch vs the landed-write stamp forces the full read (fast path must NOT fire)"); + const after = (JSON.parse(readFileSync(envelopeFileOf(dir, "in-window"), "utf8")) as { payload: { stats: { requests: number }; metadata: { lineage: string } } }).payload; + assert.equal(after.stats.requests, 9, "the sibling's fresher envelope survives — no clobber"); + assert.equal(after.metadata.lineage, "sibling-newer"); + assert.ok(warnings.some((w) => w.includes("rejected stale snapshot")), "the in-window foreign write still warns"); + store.closeQuietly(); +}); + +await withMode("shadow", "guard ownership: #3406 — DEBOUNCED saves stay stamped (host-owned debounce keeps the fast path live)", async (dir) => { + const store = new SessionStore({ dir, debounceMs: 25, enabled: true, log: () => {} }); + const s = makeSession("deb-stamp"); + const SAVES = 5; + for (let i = 1; i <= SAVES; i++) { + s.stats.requests = i; + store.scheduleSave(s); + await sleep(150); // debounce fires, the write lands AND gets stamped + } + const file = envelopeFileOf(dir, "deb-stamp"); + const payload = (JSON.parse(readFileSync(file, "utf8")) as { payload: { stats: { requests: number } } }).payload; + assert.equal(payload.stats.requests, SAVES, "coalesced builds still persist the freshest state"); + // Before the host-owned debounce, every debounced save landed through the + // kernel-internal write path WITHOUT a stamp → the fast path died on every + // per-turn save (full re-read of the envelope each time). The first save + // has no token yet and must full-read; every later one must collapse. + const fullReads = store.__staleGuardFullReadsForTests(); + assert.ok(fullReads <= 2, `debounced saves must collapse to the first full read (got ${fullReads} for ${SAVES} saves)`); + store.closeQuietly(); +}); + +await withMode("shadow", "guard ownership: #3406 — a foreign write between DEBOUNCED saves no longer clobbers the envelope", async (dir) => { + const warnings: string[] = []; + const log = (level: string, msg: string): void => { + if (level === "warn") warnings.push(msg); + }; + const store = new SessionStore({ dir, debounceMs: 25, enabled: true, log }); + const s = makeSession("deb-foreign"); + s.stats.requests = 3; + store.scheduleSave(s); + await sleep(150); // first save lands AND gets stamped + const fullReadsBefore = store.__staleGuardFullReadsForTests(); + + // The #3406 item-2 race on the production per-turn lane: a legacy-mode + // sibling (never touches index.db) writes a NEWER envelope after ours. + const file = envelopeFileOf(dir, "deb-foreign"); + const parsed = JSON.parse(readFileSync(file, "utf8")) as { payload: { stats: { requests: number }; metadata: Record } }; + parsed.payload.stats.requests = 9; + parsed.payload.metadata.lineage = "sibling-newer"; + writeFileSync(file, JSON.stringify(parsed)); + + // Same in-memory counters — only the landed-write stamp can catch it. + s.stats.requests = 3; + store.scheduleSave(s); + await sleep(150); + + assert.equal(store.__staleGuardFullReadsForTests(), fullReadsBefore + 1, "size mismatch vs the landed-write stamp forces the full read (fast path must NOT fire)"); + const after = (JSON.parse(readFileSync(envelopeFileOf(dir, "deb-foreign"), "utf8")) as { payload: { stats: { requests: number }; metadata: { lineage: string } } }).payload; + assert.equal(after.stats.requests, 9, "the sibling's fresher envelope survives — no clobber"); + assert.equal(after.metadata.lineage, "sibling-newer"); + assert.ok(warnings.some((w) => w.includes("rejected stale snapshot")), "the debounced-lane foreign write still warns"); + store.closeQuietly(); +}); + await withMode("shadow", "guard ownership: interleaved storm stays exact with the sidecar shared", async (dir) => { const warnings: string[] = []; const log = (level: string, msg: string): void => { diff --git a/tests/plugin-fork.test.ts b/tests/plugin-fork.test.ts index ab06cfc241..852ae4ef66 100644 --- a/tests/plugin-fork.test.ts +++ b/tests/plugin-fork.test.ts @@ -163,7 +163,7 @@ test("HTTP explicit main with only public ACP tools captures initial history, us const h = await harness(false, false); try { const tools = (await h.request("/__bili/plugin/manifest")).body.tools.openai; - assert.deepEqual(tools.map((tool) => tool.function.name), ["compress", "decompress", "search_context", "acp_status", "acp_cache"]); + assert.deepEqual(tools.map((tool) => tool.function.name), ["compress", "decompress", "search_context", "acp_status", "acp_cache", "read_original_image"]); await sendIntentModel(h, "parent", h.messages, tools, "main"); assert.deepEqual(h.forwarded.at(-1)!.tools, tools, "main tools survive unchanged"); assert.equal(h.forwarded.at(-1)!.max_tokens, 256); diff --git a/tests/plugin-protocol.test.ts b/tests/plugin-protocol.test.ts index a8ffb58777..2ec142a2c2 100644 --- a/tests/plugin-protocol.test.ts +++ b/tests/plugin-protocol.test.ts @@ -88,7 +88,7 @@ async function startHarness( scripts: string[][], kernelConfig?: Config, // #1278: the user-facing `compress` block (loadOptions → opts.compress), distinct - // from kernelConfig. Tests pass rules here to exercise the REAL path the + // from kernelConfig. Tests pass absorb/rules here to exercise the REAL path the // file config takes, not the kernelConfig shortcut the pre-fix manifest read. compress?: Partial, ): Promise { @@ -239,14 +239,14 @@ test("plugin manifest serves the exact wire tool schemas, headers and version", assert.equal(manifest.ok, true); assert.equal(manifest.protocolVersion, 1); assert.ok(/^\d+\.\d+\.\d+/.test(manifest.version), `version looks wrong: ${manifest.version}`); - // #1192: every extra tool (acp_rule) is opt-in and only advertised + // #1192: every extra tool (absorb, acp_rule) is opt-in and only advertised // when enabled — hosts register manifest tools verbatim, so a disabled // tool must not be listed. acp_cache stays in the base toolset. - assert.deepEqual([...manifest.toolNames].sort(), ["acp_cache", "acp_status", "compress", "decompress", "search_context"]); + assert.deepEqual([...manifest.toolNames].sort(), ["acp_cache", "acp_status", "compress", "decompress", "read_original_image", "search_context"]); const names = manifest.tools.anthropic!.map((t) => String(t.name)).sort(); - assert.deepEqual(names, ["acp_cache", "acp_status", "compress", "decompress", "search_context"]); - assert.equal(manifest.tools.openai!.length, 5); - assert.equal(manifest.tools.responses!.length, 5); + assert.deepEqual(names, ["acp_cache", "acp_status", "compress", "decompress", "read_original_image", "search_context"]); + assert.equal(manifest.tools.openai!.length, 6); + assert.equal(manifest.tools.responses!.length, 6); // #841: search_context's conversation_id doubles as a cross-session // read-only search target — its param description must carry the // historical-search wording in every wire shape. @@ -278,21 +278,82 @@ test("plugin manifest: acp_rule advertised when rules enabled in config", async const resp = await fetch(`http://127.0.0.1:${h.proxyPort}/__bili/plugin/manifest`); assert.equal(resp.status, 200); const manifest = (await resp.json()) as { toolNames: string[]; tools: Record>> }; - assert.deepEqual([...manifest.toolNames].sort(), ["acp_cache", "acp_rule", "acp_status", "compress", "decompress", "search_context"]); + assert.deepEqual([...manifest.toolNames].sort(), ["acp_cache", "acp_rule", "acp_status", "compress", "decompress", "read_original_image", "search_context"]); for (const shape of ["anthropic", "openai", "responses"] as const) { const listed = manifest.tools[shape]!.map((t) => { const fn = t.function as { name?: unknown } | undefined; return typeof t.name === "string" ? t.name : (typeof fn?.name === "string" ? fn.name : ""); }); assert.ok(listed.includes("acp_rule"), `${shape} has acp_rule`); - assert.equal(listed.length, 6); + assert.equal(listed.length, 7); } } finally { await h.close(); } }); +// #1192 (regression): with absorb + rules enabled in the base config the +// manifest advertises all eight tools again — enabling the features must not +// shrink the toolset dsh/pi/omp/MCP shims register. +test("plugin manifest: absorb and acp_rule advertised when enabled in config", async () => { + const h = await startHarness([textScript()], { ...defaultConfig(400_000), absorb: { enabled: true, toolName: "absorb", minToolTokens: 4000, contextThresholdPct: 0, excludeTools: [] }, rules: { enabled: true } }); + try { + const resp = await fetch(`http://127.0.0.1:${h.proxyPort}/__bili/plugin/manifest`); + assert.equal(resp.status, 200); + const manifest = (await resp.json()) as { toolNames: string[]; tools: Record>> }; + assert.deepEqual([...manifest.toolNames].sort(), ["absorb", "acp_cache", "acp_rule", "acp_status", "compress", "decompress", "read_original_image", "search_context"]); + for (const shape of ["anthropic", "openai", "responses"] as const) { + const listed = manifest.tools[shape]!.map((t) => { + const fn = t.function as { name?: unknown } | undefined; + return typeof t.name === "string" ? t.name : (typeof fn?.name === "string" ? fn.name : ""); + }); + assert.ok(listed.includes("absorb"), `${shape} has absorb`); + assert.ok(listed.includes("acp_rule"), `${shape} has acp_rule`); + assert.equal(listed.length, 8); + } + } finally { + await h.close(); + } +}); + +// #1278 (regression): the pre-fix manifest read opts.kernelConfig (kernel defaults), +// so an absorb enabled in the user's GLOBAL compress block — where loadOptions puts +// the file config — was never advertised. Pi then registered no callable tool for the +// [ACP absorb] prompts the wire emits. Enabling via the global compress block (the real +// path) must now advertise absorb on every wire shape. +test("plugin manifest: absorb advertised when enabled in the global compress block", async () => { + const h = await startHarness([textScript()], undefined, { absorb: { enabled: true } }); + try { + const resp = await fetch(`http://127.0.0.1:${h.proxyPort}/__bili/plugin/manifest`); + assert.equal(resp.status, 200); + const manifest = (await resp.json()) as { toolNames: string[]; tools: Record>> }; + assert.deepEqual([...manifest.toolNames].sort(), ["absorb", "acp_cache", "acp_status", "compress", "decompress", "read_original_image", "search_context"]); + for (const shape of ["anthropic", "openai", "responses"] as const) { + const listed = manifest.tools[shape]!.map((t) => { + const fn = t.function as { name?: unknown } | undefined; + return typeof t.name === "string" ? t.name : (typeof fn?.name === "string" ? fn.name : ""); + }); + assert.ok(listed.includes("absorb"), `${shape} has absorb`); + assert.equal(listed.length, 7); + } + } finally { + await h.close(); + } +}); +// #1278: an explicit global disable keeps absorb out of the manifest (guards against +// flipping the logic to "advertise whenever an absorb key is present"). +test("plugin manifest: absorb omitted when the global compress block disables it", async () => { + const h = await startHarness([textScript()], undefined, { absorb: { enabled: false } }); + try { + const resp = await fetch(`http://127.0.0.1:${h.proxyPort}/__bili/plugin/manifest`); + assert.equal(resp.status, 200); + const manifest = (await resp.json()) as { toolNames: string[] }; + assert.deepEqual([...manifest.toolNames].sort(), ["acp_cache", "acp_status", "compress", "decompress", "read_original_image", "search_context"]); + } finally { + await h.close(); + } +}); test("plugin mode: no wire tool injection, philosophy still injected, native compress tool_use passes through untouched", async () => { const h = await startHarness([compressToolScript()]); diff --git a/tests/preflight-fail-fast.test.ts b/tests/preflight-fail-fast.test.ts index 74c66b74f5..d516943ca5 100644 --- a/tests/preflight-fail-fast.test.ts +++ b/tests/preflight-fail-fast.test.ts @@ -225,7 +225,7 @@ test("e2e #301: payload fits the window + preflight 429 → request still forwar } }); -test("e2e #301: over-window payload with nothing compressible → structured 502, no summary call, no forward", async () => { +test("e2e #301: over-window payload with nothing compressible → structured 413 (#3461), no summary call, no forward", async () => { const calls: Call[] = []; const upstream = makeUpstream429(calls); upstream.listen(0, "127.0.0.1"); @@ -256,7 +256,7 @@ test("e2e #301: over-window payload with nothing compressible → structured 502 ], }), }); - assert.equal(r.status, 502, "fail-fast 502 when nothing is compressible"); + assert.equal(r.status, 413, "fail-fast 413 when nothing is compressible (#3461: deterministic exhaustion is non-retryable)"); const json = JSON.parse(await r.text()) as { error?: { code?: string; message?: string; retryable?: boolean } }; assert.equal(json.error?.code, "preflight_compress_failed"); assert.equal(json.error?.retryable, false); @@ -404,7 +404,7 @@ test("e2e #736: operator-shrunk window (compress.modelContextLimit below the mod ], }), }); - assert.equal(r.status, 502, "fail-fast 502 when nothing is compressible under the shrunken window"); + assert.equal(r.status, 413, "fail-fast 413 when nothing is compressible under the shrunken window (#3461)"); const json = JSON.parse(await r.text()) as { error?: { code?: string; message?: string } }; assert.equal(json.error?.code, "preflight_compress_failed"); const msg = json.error?.message ?? ""; @@ -433,7 +433,7 @@ test("e2e #736: operator-shrunk window (compress.modelContextLimit below the mod ], }), }); - assert.equal(r2.status, 502); + assert.equal(r2.status, 413); const msg2 = (JSON.parse(await r2.text()) as { error?: { message?: string } }).error?.message ?? ""; assert.ok(!msg2.includes("full window"), `no shrink note when the window was not operator-shrunk (got: ${msg2})`); proxyPlain.close(); @@ -487,7 +487,7 @@ test("e2e #737: output-headroom-shrunk window (non-Anthropic, no operator overri ], }), }); - assert.equal(r.status, 502, "fail-fast 502 when nothing is compressible under the headroom-shrunk window"); + assert.equal(r.status, 413, "fail-fast 413 when nothing is compressible under the headroom-shrunk window (#3461)"); const json = JSON.parse(await r.text()) as { error?: { code?: string; message?: string } }; assert.equal(json.error?.code, "preflight_compress_failed"); const msg = json.error?.message ?? ""; diff --git a/tests/removed-compress-keys-inert.test.ts b/tests/removed-compress-keys-inert.test.ts new file mode 100644 index 0000000000..111be4b0a8 --- /dev/null +++ b/tests/removed-compress-keys-inert.test.ts @@ -0,0 +1,83 @@ +import { test } from "node:test"; +import assert from "node:assert/strict"; +import fs from "node:fs"; +import os from "node:os"; +import path from "node:path"; +import { loadConfigFile, parseCompressSettings } from "../src/config.ts"; +import { setLogCapture } from "../src/logger.ts"; +import { rmrf } from "./tmp-rm.ts"; + +type Captured = { level: string; msg: string }; + +function captureLogs(): { captured: Captured[]; stop: () => void } { + const captured: Captured[] = []; + setLogCapture((level, msg) => captured.push({ level, msg })); + return { captured, stop: () => setLogCapture(null) }; +} + +function withConfigFile(body: string, fn: () => void): void { + const dir = fs.mkdtempSync(path.join(os.tmpdir(), "bili-removed-keys-")); + const file = path.join(dir, "billion-context.json"); + fs.writeFileSync(file, body); + const prev = process.env.BILI_CONFIG_FILE; + process.env.BILI_CONFIG_FILE = file; + try { + fn(); + } finally { + if (prev === undefined) delete process.env.BILI_CONFIG_FILE; + else process.env.BILI_CONFIG_FILE = prev; + rmrf(dir); + } +} + +// Warnings for the removed-key family only (other INERT emitters exist too). +const familyWarns = (captured: Captured[]) => + captured.filter((e) => e.level === "warn" && e.msg.includes("is INERT") && /absorb|imageCompression/.test(e.msg)); + +// `absorb` was removed by #3293 but REVIVED by #3402/#3422 — it parses again +// and is NOT inert; only imageCompression (#3266) stays in the removed family. +test("#3344: removed compress.imageCompression warns INERT once (root level) and is ignored; revived absorb parses", () => { + const { captured, stop } = captureLogs(); + withConfigFile( + `{"compress":{"absorb":{"enabled":true,"minToolTokens":500},"imageCompression":{"enabled":true,"quality":60}}}`, + () => { + const cfg = loadConfigFile(); + const cs = parseCompressSettings(cfg.compress ?? {}); + assert.deepEqual((cs as Record).absorb, { enabled: true, minToolTokens: 500 }); + assert.equal((cs as Record).imageCompression, undefined); + const warns = familyWarns(captured); + assert.equal(warns.length, 1, `expected exactly one family INERT warning, got: ${warns.map((w) => w.msg).join(" | ")}`); + assert.doesNotMatch(warns[0]!.msg, /compress\.absorb/); + assert.match(warns[0]!.msg, /compress\.imageCompression/); + assert.match(warns[0]!.msg, /was removed/); + // Second load of the SAME content: deduped (one notice per process). + loadConfigFile(); + assert.equal(familyWarns(captured).length, 1); + }, + ); + stop(); +}); + +test("#3344: removed key at provider and model levels is named by full path; provider-level absorb does not warn", () => { + const { captured, stop } = captureLogs(); + withConfigFile( + `{"providers":{"https://u.example":{"compress":{"absorb":{"enabled":true}},"models":{"gpt-x":{"compress":{"imageCompression":{"enabled":true}}}}}}}`, + () => { + loadConfigFile(); + const warns = familyWarns(captured); + assert.equal(warns.length, 1, `expected exactly one family INERT warning, got: ${warns.map((w) => w.msg).join(" | ")}`); + assert.doesNotMatch(warns[0]!.msg, /compress\.absorb/); + assert.match(warns[0]!.msg, /providers\["https:\/\/u\.example"\]\.models\["gpt-x"\]\.compress\.imageCompression/); + }, + ); + stop(); +}); + +test("#3344: configs without the removed keys stay silent", () => { + const { captured, stop } = captureLogs(); + withConfigFile(`{"compress":{"ccr":{"enabled":true}}}`, () => { + loadConfigFile(); + assert.equal(familyWarns(captured).length, 0); + }); + stop(); +}); diff --git a/tests/rules.test.ts b/tests/rules.test.ts index 45eb51f888..6e1d8c0fd5 100644 --- a/tests/rules.test.ts +++ b/tests/rules.test.ts @@ -9,7 +9,7 @@ import { RULE_TOOL_NAME, type Config, } from "acp-kernel"; -import { isProxyToolFor } from "../src/proxy-tool.js"; +import { isProxyToolFor } from "../src/absorb.ts"; import { mergeCompress, applyCompressSettings } from "../src/compress-settings.ts"; import { parseCompressSettings } from "../src/config.ts"; import { RULE_TOOL, RULE_TOOL_GOOGLE, RULE_TOOL_OPENAI, RULE_TOOL_RESPONSES } from "../src/compress-tool.ts"; diff --git a/tests/search-plan-aware.test.ts b/tests/search-plan-aware.test.ts index 11e335b974..ea5bac1e30 100644 --- a/tests/search-plan-aware.test.ts +++ b/tests/search-plan-aware.test.ts @@ -55,7 +55,7 @@ type Folded = { core: ReturnType; config: Config; session: Re function foldBlocks(ccr: boolean): Folded { const core = createCore(); - const config = applyCompressSettings(defaultConfig(200_000), 200_000, { ccr: { enabled: ccr, minToolTokens: 50 } }) as Config; + const config = applyCompressSettings(defaultConfig(200_000), 200_000, { absorb: { enabled: true, minToolTokens: 50 }, ccr: { enabled: ccr, minToolTokens: 50 } }) as Config; const session = getSession(`sp-${Math.random().toString(36).slice(2)}`); if (ccr) storeEffectiveCcr(session, { enabled: true, minToolTokens: 50 }); const turn = core.processTurn({ messages: makeMsgs(40), state: session.state, config, tokenCount: 9999, renderTags: "text-only" }); diff --git a/tests/side-lane-invariants.test.ts b/tests/side-lane-invariants.test.ts index 394ec5a1ca..b10caedb05 100644 --- a/tests/side-lane-invariants.test.ts +++ b/tests/side-lane-invariants.test.ts @@ -8,6 +8,7 @@ import { join } from "node:path"; import { fileURLToPath } from "node:url"; import { defaultConfig, ABSORB_TOOL_NAME, RULE_TOOL_NAME, RETRIEVE_TOOL_NAME } from "acp-kernel"; import { startServer, BILI_TOOL_NAMES } from "../src/server.ts"; +import { READ_ORIGINAL_IMAGE_TOOL_NAME } from "../src/compress-tool.ts"; import type { ProxyOptions } from "../src/config.ts"; import { SessionStore, _setStoreForTest } from "../src/persist.ts"; import { _resetPluginStateForTest, resolveConversation } from "../src/plugin.ts"; @@ -343,13 +344,22 @@ test("(H) extracted relay module keeps the side-lane boundary (#1440 P2 cut 4)", ); }); -// (#3293): absorb was removed from bili entirely — the kernel's absorb tool -// name must no longer be bili-owned: any "absorb" tool in a request is -// host/client-originated and vetoes side-lane demotion like any other foreign -// tool. Lands alongside the src/server/side-request.ts edit per #2170 -// (lane logic moves ⇔ a dedicated side-lane suite moves with it). -test("(I) post-#3293: the removed absorb tool name is no longer bili-owned", () => { - assert.ok(!BILI_TOOL_NAMES.has(ABSORB_TOOL_NAME), "absorb is out of the bili-owned set"); +// (#3293 removed absorb; #3402/#3422 revive it): the kernel's absorb tool name +// is bili-owned again — a session-sidecar request carrying it is demoted by +// #1897 like any other bili-owned tool. Lands alongside the +// src/server/side-request.ts edit per #2170 (lane logic moves ⇔ a dedicated +// side-lane suite moves with it). +test("(I) revival (#3402): the absorb tool name is bili-owned again", () => { + assert.ok(BILI_TOOL_NAMES.has(ABSORB_TOOL_NAME), "absorb is back in the bili-owned set"); assert.ok(BILI_TOOL_NAMES.has(RULE_TOOL_NAME), "rules tool still bili-owned"); assert.ok(BILI_TOOL_NAMES.has(RETRIEVE_TOOL_NAME), "retrieve tool still bili-owned"); }); + +// (#3400): read_original_image is bili's own context-management tool. A +// session-sidecar request that leaks the armed tool set must still be demoted +// by #1897, so the name joins the bili-owned set in side-request.ts — and per +// #2170 the lane-logic move lands with its dedicated suite pinning the +// membership (same discipline as (I) above for the absorb removal). +test("(J) post-#3400: read_original_image is bili-owned for #1897 demotion", () => { + assert.ok(BILI_TOOL_NAMES.has(READ_ORIGINAL_IMAGE_TOOL_NAME), "read_original_image is in the bili-owned set"); +}); diff --git a/tests/side-request-isolation.test.ts b/tests/side-request-isolation.test.ts index 2ff3ea4da6..2cf50c163a 100644 --- a/tests/side-request-isolation.test.ts +++ b/tests/side-request-isolation.test.ts @@ -110,7 +110,7 @@ test("restoreOutputBudget: high-water learning + starved-budget restore (#546)", assert.equal(s.metadata.outputBudgetHighWater, 8000, "non-starved value recorded verbatim as high-water"); const starved2 = { max_tokens: 200, tools: [{ name: "t" }] } as { max_tokens: number }; restoreOutputBudget(starved2, s, noopLog); - assert.equal(starved2.max_tokens, OUTPUT_RESTORE_ABSOLUTE_FLOOR, "boundary 200 starve floored to the absolute minimum (#3071)"); + assert.equal(starved2.max_tokens, 8000, "boundary 200 starve restored to the session's own high-water, never raised to the floor above it (#3406)"); restoreOutputBudget({ max_tokens: 32768 }, s, noopLog); assert.equal(s.metadata.outputBudgetHighWater, 32768, "a later healthy value wins as high-water"); // Field-name fidelity: restore writes the SAME field the client used. @@ -155,13 +155,53 @@ test("restoreOutputBudget: configured-output-limit floor for poisoned or missing restoreOutputBudget({ max_tokens: 234 }, s5, noopLog); const tinyFloor = { max_tokens: 1, tools: [{ name: "t" }] } as { max_tokens: number }; restoreOutputBudget(tinyFloor, s5, noopLog, 200); - assert.equal(tinyFloor.max_tokens, OUTPUT_RESTORE_ABSOLUTE_FLOOR, "limit <= 200 is not a usable ceiling — absolute floor applies"); + assert.equal(tinyFloor.max_tokens, OUTPUT_RESTORE_ABSOLUTE_FLOOR, "limit <= 200 is not a usable ceiling; rattle high-water 234 is below the trust line — absolute floor applies (#3071/#3406)"); // Side requests (no tools) never receive the floor. const side = { max_tokens: 100 } as { max_tokens: number }; restoreOutputBudget(side, s3, noopLog, 32768); assert.equal(side.max_tokens, 100, "side request untouched even with a floor available"); }); +test("restoreOutputBudget: no-ceiling restore never exceeds the session's proven high-water (#3406 hard-cap providers)", () => { + // A provider hard-caps maxOutputTokens at 8192 — below the 32K absolute + // floor — and bili has NO ceiling source for it (no runtime-info, no + // launcher, no config, no registry row). The session ran HEALTHY at 8000 + // (the provider accepted that exact value at full size), then the client's + // budget starved. Raising to the 32K floor would hard-reject the request; + // the restore must return the session's own proven budget instead. + const s = metaSession("hardcap"); + restoreOutputBudget({ max_tokens: 8000 }, s, noopLog); + const starved = { max_tokens: 1, tools: [{ name: "t" }] } as { max_tokens: number }; + restoreOutputBudget(starved, s, noopLog); + assert.equal(starved.max_tokens, 8000, "restored to the proven high-water, never raised above it (#3406)"); + // A high-water ABOVE the floor keeps passing untouched (unchanged). + const s2 = metaSession("hardcap2"); + restoreOutputBudget({ max_tokens: 49152 }, s2, noopLog); + const starved2 = { max_tokens: 1, tools: [{ name: "t" }] } as { max_tokens: number }; + restoreOutputBudget(starved2, s2, noopLog); + assert.equal(starved2.max_tokens, 49152, "high-water above the floor is still restored as-is"); + // Without ANY high-water the #3071 floor still guarantees a lower bound + // (born-starved / fork-inherited rattle — nothing was ever proven legal). + const s3 = metaSession("hardcap3"); + const born = { max_tokens: 1, tools: [{ name: "t" }] } as { max_tokens: number }; + restoreOutputBudget(born, s3, noopLog); + assert.equal(born.max_tokens, OUTPUT_RESTORE_ABSOLUTE_FLOOR, "born-starved keeps the absolute floor (#3071)"); + // A rattle high-water BELOW the trust line still gets the floor — the + // #3071 death spiral must stay fixed (decay chains end in exactly this + // state, and a few hundred tokens cannot emit a compress call). + const s4 = metaSession("hardcap4"); + restoreOutputBudget({ max_tokens: 2061 }, s4, noopLog); + const rattle = { max_tokens: 1, tools: [{ name: "t" }] } as { max_tokens: number }; + restoreOutputBudget(rattle, s4, noopLog); + assert.equal(rattle.max_tokens, OUTPUT_RESTORE_ABSOLUTE_FLOOR, "high-water below the trust line (2061) is overridden by the floor (#3071)"); + // Boundary: exactly at the trust line the high-water is a proven bound. + const s5 = metaSession("hardcap5"); + restoreOutputBudget({ max_tokens: 4096 }, s5, noopLog); + const atLine = { max_tokens: 1, tools: [{ name: "t" }] } as { max_tokens: number }; + restoreOutputBudget(atLine, s5, noopLog); + assert.equal(atLine.max_tokens, 4096, "high-water at the trust line (4096) is restored as-is, not raised (#3406)"); +}); + test("restoreOutputBudget: warns once per SESSION+model when no output ceiling backs the restore (#1840/#3071)", () => { _resetNoOutputCeilingWarningsForTest(); const lines: string[] = []; @@ -176,7 +216,7 @@ test("restoreOutputBudget: warns once per SESSION+model when no output ceiling b assert.equal(s.metadata.outputBudgetHighWater, 583, "non-starved learning is silent and tracked verbatim"); const starved = { max_tokens: 1, tools, model: A } as { max_tokens: number }; restoreOutputBudget(starved, s, log); - assert.equal(starved.max_tokens, OUTPUT_RESTORE_ABSOLUTE_FLOOR, "ceiling-less starve floored to the absolute minimum (#3071)"); + assert.equal(starved.max_tokens, OUTPUT_RESTORE_ABSOLUTE_FLOOR, "ceiling-less starve with a rattle high-water (583 < trust line) floored to the absolute minimum (#3071)"); let warns = lines.filter((l) => l.includes("NO known output ceiling")); assert.equal(warns.length, 1, "one warn per session+model (on the starve, not the silent learning)"); assert.match(warns[0] ?? "", /model=stepfun\/step-5-preview/, "names the model"); diff --git a/tests/storage-engine-bootstrap.test.ts b/tests/storage-engine-bootstrap.test.ts index 6105f7e34d..d0f5a3c6e9 100644 --- a/tests/storage-engine-bootstrap.test.ts +++ b/tests/storage-engine-bootstrap.test.ts @@ -41,11 +41,15 @@ import { setSqliteProbeOverridesForTests, } from "../src/storage/driver.ts"; import { + BOOTSTRAP_TIMEOUT_MS, ensureSqliteEngine, extractTgzToDir, + nodeSqliteUpgradeHint, platformTargetKey, resetEngineBootstrapForTests, + setBootstrapTimeoutMsForTests, setEngineBootstrapSpecForTests, + setEngineMirrorsForTests, setPlatformKeyForTests, type BootstrapOutcome, } from "../src/storage/engine-bootstrap.ts"; @@ -142,7 +146,7 @@ interface MockRegistry { close(): Promise; } -async function startMockRegistry(body: Buffer, status = 200, version = "13.0.3"): Promise { +async function startMockRegistry(body: Buffer, status = 200, version = "13.0.3", delayMs = 0): Promise { let count = 0; const server = http.createServer((req, res) => { count++; @@ -151,15 +155,27 @@ async function startMockRegistry(body: Buffer, status = 200, version = "13.0.3") res.end(); return; } - res.writeHead(status, { "content-type": "application/octet-stream" }); - res.end(body); + const send = (): void => { + if (res.writableEnded || res.destroyed) return; + try { + res.writeHead(status, { "content-type": "application/octet-stream" }); + res.end(body); + } catch { + // client aborted the download before we got here — nothing to serve + } + }; + if (delayMs > 0) setTimeout(send, delayMs); + else send(); }); await new Promise((r) => server.listen(0, "127.0.0.1", r)); const addr = server.address() as AddressInfo; return { base: `http://127.0.0.1:${addr.port}`, requests: () => count, - close: () => new Promise((r) => server.close(() => r())), + close: () => new Promise((r) => { + (server as unknown as { closeAllConnections?: () => void }).closeAllConnections?.(); + server.close(() => r()); + }), }; } @@ -187,20 +203,28 @@ function resetAllSeams(): void { setSqliteProbeOverridesForTests(null); clearSelfDownloadedEngineForTests(); setEngineBootstrapSpecForTests(null); + setEngineMirrorsForTests(null); setPlatformKeyForTests(null); + setBootstrapTimeoutMsForTests(null); resetEngineBootstrapForTests(); } const UNIFIED_ENV = { BILI_STORAGE_UNIFIED: "full" }; async function withRegistry(base: string, fn: () => Promise): Promise { + // #3338: the engine download now walks a mirror list (configured first, + // then builtin npm + npmmirror). Pin the list to this one mock so the + // pre-mirror-era tests stay hermetic — a failing configured mock must + // fail the bootstrap instead of falling through to the real registries. const prev = process.env.BILI_UPDATE_REGISTRY; process.env.BILI_UPDATE_REGISTRY = base; + setEngineMirrorsForTests([base]); try { await fn(); } finally { if (prev === undefined) delete process.env.BILI_UPDATE_REGISTRY; else process.env.BILI_UPDATE_REGISTRY = prev; + setEngineMirrorsForTests(null); } } @@ -431,6 +455,61 @@ test("bootstrap: stale unusable cache dir is replaced instead of looping the coo } }); +test("bootstrap: production download budget comfortably covers an ~11 MB registry fetch (#3338)", () => { + assert.ok(BOOTSTRAP_TIMEOUT_MS >= 30_000, + `BOOTSTRAP_TIMEOUT_MS=${BOOTSTRAP_TIMEOUT_MS} too tight for an ~11 MB download over a non-trivial network (was 10 s, see #3338)`); +}); + +test("bootstrap: a download that outlasts its timeout budget aborts fail-safe and arms the cooldown (#3338)", async () => { + const cachePrev = isolateCache(); + let reg: MockRegistry | null = null; + try { + resetAllSeams(); + const tgz = fakePackageTgz(); + setEngineBootstrapSpecForTests({ version: "13.0.3", sha512Base64: sha512B64(tgz) }); + setSqliteProbeOverridesForTests({ builtin: false, host: false }); + const delayMs = 800; + setBootstrapTimeoutMsForTests(Math.floor(delayMs / 5)); + reg = await startMockRegistry(tgz, 200, "13.0.3", delayMs); + await withRegistry(reg.base, async () => { + const out = await ensureSqliteEngine({ env: { ...UNIFIED_ENV } }); + assert.equal(out.status, "failed", `expected timeout failure, got ${JSON.stringify(out)}`); + assert.match(out.detail ?? "", /abort|time/i, "failure reason names the abort/timeout"); + assert.ok(!fs.existsSync(path.join(cachePrev, "billion-context", "sqlite", "better-sqlite3-13.0.3")), "aborted download must not publish a cache dir"); + assert.ok(fs.existsSync(path.join(cachePrev, "billion-context", "sqlite", ".bootstrap-failed.json")), "aborted download arms the cooldown"); + }); + } finally { + await reg?.close(); + resetAllSeams(); + restoreCache(cachePrev); + rmrf(cachePrev); + } +}); + +test("bootstrap: a slow-but-completing download succeeds once the budget covers it (#3338)", async () => { + const cachePrev = isolateCache(); + let reg: MockRegistry | null = null; + try { + resetAllSeams(); + const tgz = fakePackageTgz(); + setEngineBootstrapSpecForTests({ version: "13.0.3", sha512Base64: sha512B64(tgz) }); + setSqliteProbeOverridesForTests({ builtin: false, host: false }); + const delayMs = 400; + setBootstrapTimeoutMsForTests(delayMs * 10); + reg = await startMockRegistry(tgz, 200, "13.0.3", delayMs); + await withRegistry(reg.base, async () => { + const out = await ensureSqliteEngine({ env: { ...UNIFIED_ENV } }); + assert.equal(out.status, "downloaded", `expected slow download to succeed, got ${JSON.stringify(out)}`); + assert.ok(fs.existsSync(path.join(cachePrev, "billion-context", "sqlite", "better-sqlite3-13.0.3", "package.json")), "slow download materialized"); + }); + } finally { + await reg?.close(); + resetAllSeams(); + restoreCache(cachePrev); + rmrf(cachePrev); + } +}); + test("bootstrap: offline (connection refused) degrades fail-safe with cooldown, never throws", async () => { const cachePrev = isolateCache(); try { @@ -690,3 +769,107 @@ test("platformTargetKey: musl detection mirrors binding.js (glibc presence decid assert.equal(platformTargetKey({ platform: "darwin", arch: "arm64" }), "darwin-arm64"); assert.equal(platformTargetKey({ platform: "win32", arch: "x64" }), "win32-x64"); }); + +// --------------------------------------------------------------------------- +// Multi-mirror engine download (#3338 directive 2) + Node upgrade hint (directive 3) +// --------------------------------------------------------------------------- + +test("mirror fallback: a failing first mirror falls to the next and the bootstrap succeeds (#3338)", async () => { + const cachePrev = isolateCache(); + let bad: MockRegistry | null = null; + let good: MockRegistry | null = null; + try { + resetAllSeams(); + const tgz = fakePackageTgz(); + setEngineBootstrapSpecForTests({ version: "13.0.3", sha512Base64: sha512B64(tgz) }); + setSqliteProbeOverridesForTests({ builtin: false, host: false }); + bad = await startMockRegistry(tgz, 500); // registry up, tarball broken at the HTTP layer + good = await startMockRegistry(tgz); + setEngineMirrorsForTests([bad.base, good.base]); + const out = await ensureSqliteEngine({ env: { ...UNIFIED_ENV } }); + assert.equal(out.status, "downloaded", `expected fallback success, got ${JSON.stringify(out)}`); + assert.ok(bad.requests() >= 1, "the failing mirror was attempted first"); + assert.ok(good.requests() >= 1, "the fallback mirror served the tarball"); + assert.ok(fs.existsSync(path.join(cachePrev, "billion-context", "sqlite", "better-sqlite3-13.0.3", "package.json")), "fallback download materialized"); + } finally { + await bad?.close(); + await good?.close(); + resetAllSeams(); + restoreCache(cachePrev); + rmrf(cachePrev); + } +}); + +test("mirror exhaustion: the cooldown reason lists every mirror attempted (#3338)", async () => { + const cachePrev = isolateCache(); + let bad1: MockRegistry | null = null; + let bad2: MockRegistry | null = null; + try { + resetAllSeams(); + const tgz = fakePackageTgz(); + setEngineBootstrapSpecForTests({ version: "13.0.3", sha512Base64: sha512B64(tgz) }); + setSqliteProbeOverridesForTests({ builtin: false, host: false }); + bad1 = await startMockRegistry(tgz, 500); + bad2 = await startMockRegistry(tgz, 503); + setEngineMirrorsForTests([bad1.base, bad2.base]); + const out = await ensureSqliteEngine({ env: { ...UNIFIED_ENV } }); + assert.equal(out.status, "failed", `expected exhaustion failure, got ${JSON.stringify(out)}`); + const detail = out.detail ?? ""; + assert.ok(detail.includes(bad1.base), `reason must name the first mirror: ${detail}`); + assert.ok(detail.includes(bad2.base), `reason must name the second mirror: ${detail}`); + assert.match(detail, /tried mirrors: /, "reason enumerates the mirror list"); + const marker = path.join(cachePrev, "billion-context", "sqlite", ".bootstrap-failed.json"); + assert.ok(fs.existsSync(marker), "exhaustion arms the cooldown"); + } finally { + await bad1?.close(); + await bad2?.close(); + resetAllSeams(); + restoreCache(cachePrev); + rmrf(cachePrev); + } +}); + +test("mirror list: a configured registry is attempted before the builtins (#3338)", async () => { + const cachePrev = isolateCache(); + let reg: MockRegistry | null = null; + try { + resetAllSeams(); + const tgz = fakePackageTgz(); + setEngineBootstrapSpecForTests({ version: "13.0.3", sha512Base64: sha512B64(tgz) }); + setSqliteProbeOverridesForTests({ builtin: false, host: false }); + reg = await startMockRegistry(tgz); + // Deliberately NO mirrors override: the production list must put the + // configured registry first. The mock SUCCEEDS, so the loop breaks + // before ever reaching the builtin production registries (no network). + // If the ordering regressed (builtins first), CI's real egress would + // serve the tarball and this mock would record zero requests. + const prev = process.env.BILI_UPDATE_REGISTRY; + process.env.BILI_UPDATE_REGISTRY = reg.base; + try { + const out = await ensureSqliteEngine({ env: { ...UNIFIED_ENV } }); + assert.equal(out.status, "downloaded", `expected configured-first success, got ${JSON.stringify(out)}`); + assert.equal(reg!.requests(), 1, "the configured registry was the one (and only) mirror hit"); + } finally { + if (prev === undefined) delete process.env.BILI_UPDATE_REGISTRY; + else process.env.BILI_UPDATE_REGISTRY = prev; + } + } finally { + await reg?.close(); + resetAllSeams(); + restoreCache(cachePrev); + rmrf(cachePrev); + } +}); + +test("nodeSqliteUpgradeHint: pins the Node >= 22.5 threshold and the levers (#3338 directive 3)", () => { + assert.equal(nodeSqliteUpgradeHint("22.5.0"), "", "22.5.0 ships node:sqlite — no hint"); + assert.equal(nodeSqliteUpgradeHint("23.0.0"), "", "23.x ships node:sqlite — no hint"); + assert.equal(nodeSqliteUpgradeHint("24.11.0"), "", "24.x ships node:sqlite — no hint"); + const old = nodeSqliteUpgradeHint("20.20.2"); + assert.match(old, /Node\.js >= 22\.5/, "the #3385 node version gets the upgrade lever"); + assert.match(old, /npmmirror/, "the hint names the builtin mirrors"); + assert.match(old, /BILI_UPDATE_REGISTRY/, "the hint names the registry override"); + assert.match(nodeSqliteUpgradeHint("22.4.9"), /Node\.js >= 22\.5/, "22.4.x predates node:sqlite"); + assert.match(nodeSqliteUpgradeHint("19.9.9"), /Node\.js >= 22\.5/, "19.x predates node:sqlite"); + assert.equal(nodeSqliteUpgradeHint("22.5.1"), "", "22.5.1 ships node:sqlite — no hint"); +}); diff --git a/tests/storage-engine-proofs.test.ts b/tests/storage-engine-proofs.test.ts index 07027a91fa..f323110c97 100644 --- a/tests/storage-engine-proofs.test.ts +++ b/tests/storage-engine-proofs.test.ts @@ -47,6 +47,11 @@ import { driveWire, WIRES, type Wire } from "./golden-storage-drive.ts"; import { rmrf } from "./tmp-rm.ts"; process.env.BILI_PERSIST_ZSTD = "0"; +// #3400: pin the legacy wire shape. This suite proves STORAGE restart +// invisibility against committed reference digests; the new default egress +// image rescale (#3400) changes those digests by design and is covered by its +// own suite (tests/issue3400-image-rescale.test.ts) — keep the two concerns apart. +process.env.BILI_IMAGE_RESCALE = "0"; // Same hermeticity contract as the corpus suites: a developer machine with // storage.unified in its real config must not leak a tier into these tests. diff --git a/tests/store.test.ts b/tests/store.test.ts index e80d89e806..339a886588 100644 --- a/tests/store.test.ts +++ b/tests/store.test.ts @@ -38,6 +38,7 @@ function toolResult(): CoreMessage[] { function ccrConfig() { return applyCompressSettings(defaultConfig(200000), 200_000, { + absorb: { enabled: true, minToolTokens: 50 }, ccr: { enabled: true, minToolTokens: 50 }, }); } @@ -110,7 +111,7 @@ test("applyCompressSettings: maps ccr onto kernel CcrConfig with DEFAULT_CCR_CON assert.equal(off.ccr?.enabled, false); }); -test("integration: kernel processTurn ID-references the oversized tool result (ccr enabled)", () => { +test("integration: kernel processTurn ID-references the oversized tool result (ccr+absorb enabled)", () => { const cfg = ccrConfig(); const turn = turnWith(cfg); const res = turn.messages.find((m) => m.role === "tool" && m.toolCallId === "call_1")!; diff --git a/tests/web-storage-banner.test.ts b/tests/web-storage-banner.test.ts index 1b8e31f473..f6a65a2653 100644 --- a/tests/web-storage-banner.test.ts +++ b/tests/web-storage-banner.test.ts @@ -195,7 +195,7 @@ test("#3040 forced none-engine: status+overview report degraded; diagnostics blo assert.ok(WEB_CLIENT.includes('t("sys.storage_degraded"'), "client composes the localized sentence with {reason}"); assert.ok(WEB_CLIENT.includes('t("sys.report_copy")'), "client offers the copy button"); assert.ok(WEB_CLIENT.includes('t("sys.report_issue")'), "client offers the GitHub button"); - assert.ok(WEB_CLIENT.includes("/__bili/diagnostics?topic="), "buttons wire to the generic diagnostics endpoint"); + assert.ok(WEB_CLIENT.includes('API_ROOT + "/diagnostics?topic="'), "buttons wire to the generic diagnostics endpoint (through the #3373 API root)"); // the overview poll re-renders the banner every 5s while degraded: the // diagnostics fetch must be cached per topic, and a failed fetch must // toast once — otherwise every poll re-hits the endpoint (full blob diff --git a/tests/wire-contract-inventory.ts b/tests/wire-contract-inventory.ts index dfa618667a..be25476a55 100644 --- a/tests/wire-contract-inventory.ts +++ b/tests/wire-contract-inventory.ts @@ -10,6 +10,7 @@ import * as K from "acp-kernel"; import { defaultConfig } from "acp-kernel"; import { handlePluginManifest } from "../src/plugin.ts"; import { + ABSORB_TOOL_RESPONSES, BILI_ACP_READONLY_TOOLS_RESPONSES, BILI_ACP_TOOLS_ANTHROPIC, BILI_ACP_TOOLS_GOOGLE, @@ -124,6 +125,7 @@ function buildBiliGolden(): Record { decompressGoogle: BILI_DECOMPRESS_TOOL_GOOGLE, }, synthesized: { + absorbResponses: ABSORB_TOOL_RESPONSES, ruleAnthropic: RULE_TOOL, ruleOpenAi: RULE_TOOL_OPENAI, ruleResponses: RULE_TOOL_RESPONSES, @@ -154,6 +156,6 @@ export function buildGoldens(): GoldenSpec[] { { name: "kernel.json", layer: "acp-kernel exports (pinned)", build: buildKernelGolden }, { name: "bili.json", layer: "bili-built surfaces (compress-tool.ts)", build: buildBiliGolden }, { name: "manifest-default.json", layer: "plugin manifest, default config", build: () => captureManifest(defaultConfig(100_000)) }, - { name: "manifest-optin.json", layer: "plugin manifest, rules+ccr enabled", build: () => captureManifest({ ...defaultConfig(100_000), rules: { enabled: true }, ccr: { ...K.DEFAULT_CCR_CONFIG, enabled: true } }) }, + { name: "manifest-optin.json", layer: "plugin manifest, absorb+rules+ccr enabled", build: () => captureManifest({ ...defaultConfig(100_000), absorb: { ...K.DEFAULT_ABSORB_CONFIG, enabled: true }, rules: { enabled: true }, ccr: { ...K.DEFAULT_CCR_CONFIG, enabled: true } }) }, ]; } diff --git a/tests/wire-contract.test.ts b/tests/wire-contract.test.ts index 468596a70c..cd151d4fd9 100644 --- a/tests/wire-contract.test.ts +++ b/tests/wire-contract.test.ts @@ -45,6 +45,8 @@ import { BILI_DECOMPRESS_TOOL_OPENAI, BILI_DECOMPRESS_TOOL_RESPONSES, BILI_DECOMPRESS_TOOL_GOOGLE, + ABSORB_TOOL_RESPONSES, + READ_ORIGINAL_IMAGE_TOOL_NAME, RULE_TOOL, RULE_TOOL_OPENAI, RULE_TOOL_RESPONSES, @@ -291,6 +293,7 @@ const BILI_ARRAYS: Array<[string, Wire]> = [ const BILI_INDIVIDUALS: Array<[string, Wire]> = [ ["BILI_SEARCH_CONTEXT_TOOL", "anthropic"], ["BILI_SEARCH_CONTEXT_TOOL_OPENAI", "openai-chat"], ["BILI_SEARCH_CONTEXT_TOOL_RESPONSES", "responses"], ["BILI_SEARCH_CONTEXT_TOOL_GOOGLE", "google"], ["BILI_DECOMPRESS_TOOL", "anthropic"], ["BILI_DECOMPRESS_TOOL_OPENAI", "openai-chat"], ["BILI_DECOMPRESS_TOOL_RESPONSES", "responses"], ["BILI_DECOMPRESS_TOOL_GOOGLE", "google"], + ["ABSORB_TOOL_RESPONSES", "responses"], ["RULE_TOOL", "anthropic"], ["RULE_TOOL_OPENAI", "openai-chat"], ["RULE_TOOL_RESPONSES", "responses"], ["RULE_TOOL_GOOGLE", "google"], ]; @@ -298,7 +301,7 @@ const BILI_CONSTANTS: Record = { BILI_ACP_TOOLS_ANTHROPIC, BILI_ACP_TOOLS_OPENAI, BILI_ACP_TOOLS_RESPONSES, BILI_ACP_TOOLS_GOOGLE, BILI_ACP_READONLY_TOOLS_RESPONSES, BILI_SEARCH_CONTEXT_TOOL, BILI_SEARCH_CONTEXT_TOOL_OPENAI, BILI_SEARCH_CONTEXT_TOOL_RESPONSES, BILI_SEARCH_CONTEXT_TOOL_GOOGLE, BILI_DECOMPRESS_TOOL, BILI_DECOMPRESS_TOOL_OPENAI, BILI_DECOMPRESS_TOOL_RESPONSES, BILI_DECOMPRESS_TOOL_GOOGLE, - RULE_TOOL, RULE_TOOL_OPENAI, RULE_TOOL_RESPONSES, RULE_TOOL_GOOGLE, + ABSORB_TOOL_RESPONSES, RULE_TOOL, RULE_TOOL_OPENAI, RULE_TOOL_RESPONSES, RULE_TOOL_GOOGLE, RETRIEVE_TOOLS: retrieveToolsFor("acp_retrieve"), }; @@ -364,25 +367,25 @@ test("wire-contract C: default manifest serves only legal base tools", () => { assertManifestLegal(m, "default"); const anthNames = new Set(m.tools.anthropic.map((t) => t.name)); for (const n of BILI_ACP_TOOLS_ANTHROPIC.map((t) => t.name)) assert.ok(anthNames.has(n), `default manifest: base tool ${n} present`); - for (const optIn of ["acp_retrieve"]) assert.ok(!anthNames.has(optIn), `default manifest: ${optIn} absent when disabled (#1192)`); - assert.ok(!anthNames.has("acp_rule"), "default manifest: acp_rule absent — opt-in extra (#1192 discipline)"); + for (const optIn of ["absorb", "acp_retrieve"]) assert.ok(!anthNames.has(optIn), `default manifest: ${optIn} absent when disabled (#1192)`); + assert.ok(!anthNames.has("acp_rule"), "default manifest: acp_rule absent — opt-in like absorb (#1192 discipline)"); const offM = captureManifest({ ...defaultConfig(100_000), rules: { enabled: false } }); assert.ok(!new Set(offM.tools.anthropic.map((t) => t.name)).has("acp_rule"), "disabled manifest: acp_rule absent (#1192)"); assertCompressFormsKept(m.tools.anthropic.find((t) => t.name === "compress") as ToolShape, "manifest-default"); }); -test("wire-contract C: opt-in manifest advertises acp_rule/acp_retrieve legally", () => { - const config = { ...defaultConfig(100_000), rules: { enabled: true }, ccr: { ...K.DEFAULT_CCR_CONFIG, enabled: true } }; +test("wire-contract C: opt-in manifest advertises absorb/acp_rule/acp_retrieve legally", () => { + const config = { ...defaultConfig(100_000), absorb: { ...K.DEFAULT_ABSORB_CONFIG, enabled: true }, rules: { enabled: true }, ccr: { ...K.DEFAULT_CCR_CONFIG, enabled: true } }; const m = captureManifest(config); assertManifestLegal(m, "optin"); const anthNames = new Set(m.tools.anthropic.map((t) => t.name)); const openAiNames = new Set(m.tools.openai.map((t) => toolNameOf("openai-chat", t))); const respNames = new Set(m.tools.responses.map((t) => t.name)); - for (const n of ["acp_rule", "acp_retrieve"]) { + for (const n of ["absorb", "acp_rule", "acp_retrieve"]) { assert.ok(anthNames.has(n), `optin manifest: anthropic advertises ${n}`); assert.ok(openAiNames.has(n), `optin manifest: openai advertises ${n}`); } - assert.ok(respNames.has("acp_rule"), "optin manifest: responses advertises acp_rule"); + for (const n of ["absorb", "acp_rule"]) assert.ok(respNames.has(n), `optin manifest: responses advertises ${n}`); assert.ok(!respNames.has("acp_retrieve"), "optin manifest: responses does NOT advertise acp_retrieve (#1271)"); assertCompressFormsKept(m.tools.anthropic.find((t) => t.name === "compress") as ToolShape, "manifest-optin"); }); @@ -474,11 +477,14 @@ const LANES: Lane[] = [ // #1399: acp_rule is part of the default injected surface on every wire, so // it joins the base expectation here (opt-in lanes below no longer add it). +// #3400: read_original_image likewise — the egress rescale pass defaults ON, +// so proxy-injection lanes forward it on every wire (tool-rejecting lanes +// like codex degrade to size-only notes and never run these rigs). const BASE_EXPECTED: Record = { - anthropic: BILI_ACP_TOOLS_ANTHROPIC.map((t) => t.name), - "openai-chat": BILI_ACP_TOOLS_OPENAI.map((t) => t.function.name), - responses: BILI_ACP_TOOLS_RESPONSES.map((t) => t.name), - google: BILI_ACP_TOOLS_GOOGLE.map((t) => t.name), + anthropic: [...BILI_ACP_TOOLS_ANTHROPIC.map((t) => t.name), READ_ORIGINAL_IMAGE_TOOL_NAME], + "openai-chat": [...BILI_ACP_TOOLS_OPENAI.map((t) => t.function.name), READ_ORIGINAL_IMAGE_TOOL_NAME], + responses: [...BILI_ACP_TOOLS_RESPONSES.map((t) => t.name), READ_ORIGINAL_IMAGE_TOOL_NAME], + google: [...BILI_ACP_TOOLS_GOOGLE.map((t) => t.name), READ_ORIGINAL_IMAGE_TOOL_NAME], }; function extractForwardedNames(wire: Wire, body: unknown): string[] { @@ -576,12 +582,12 @@ for (const lane of LANES) { }); } -test("wire-contract D: opt-in lane (rules+ccr) forwards the extended surface legally", async () => { +test("wire-contract D: opt-in lane (absorb+rules+ccr) forwards the extended surface legally", async () => { const lane = LANES[0]; const fake = await startFakeUpstream(lane.wire); let rig: Rig | undefined; try { - rig = await startRig(fake.url, lane.model, { rules: true, ccr: { enabled: true } }); + rig = await startRig(fake.url, lane.model, { absorb: { enabled: true }, rules: true, ccr: { enabled: true } }); const res = await fetch(`${rig.proxyUrl}${lane.path}`, { method: "POST", headers: { "content-type": "application/json", "x-api-key": "test", "x-acp-session": "wc-anth-optin" }, @@ -591,8 +597,8 @@ test("wire-contract D: opt-in lane (rules+ccr) forwards the extended surface leg await res.text(); assert.equal(fake.violations.length, 0, `opt-in fake violations:\n${fake.violations.join("\n")}`); const fwdNames = extractForwardedNames(lane.wire, fake.requests[0].body); - const expected = [...BASE_EXPECTED[lane.wire], ...lane.clientToolNames, "acp_rule", "acp_retrieve"].sort(); - assert.deepEqual(fwdNames.sort(), expected, "opt-in lane forwards base + acp_rule (rules:true) + acp_retrieve + client tools"); + const expected = [...BASE_EXPECTED[lane.wire], ...lane.clientToolNames, "acp_rule", "absorb", "acp_retrieve"].sort(); + assert.deepEqual(fwdNames.sort(), expected, "opt-in lane forwards base + acp_rule (rules:true) + absorb/acp_retrieve + client tools"); const fwdTools = (fake.requests[0].body as ToolShape).tools; const compress = (Array.isArray(fwdTools) ? fwdTools : []).find((t) => (t as ToolShape).name === "compress") as ToolShape; assert.ok(compress, "opt-in forwarded body carries compress"); diff --git a/tsup.config.ts b/tsup.config.ts index e25f2505f5..6867e1b58b 100644 --- a/tsup.config.ts +++ b/tsup.config.ts @@ -43,15 +43,9 @@ defineConfig({ // billion-context-pi-subagents (#2384): vendored under pi-subagents/, // consumed via file:, and its own build already inlines its typebox peer, // so its dist is self-contained too. - noExternal: ["acp-kernel", "billion-context-pi-subagents", "fzstd", "node-forge", "semver", "tar", "undici", "jsonc-parser", "ws"], - // sharp is an OPTIONAL runtime dependency (native module): it must stay - // EXTERNAL so dist keeps a real lazy `import("sharp")` that Node resolves - // at runtime from node_modules — missing ⇒ clean pass-through, and the - // bundle stays independent of which sharp version npm resolved (the caret - // range in optionalDependencies would otherwise bake that version into - // dist and break reproducible builds). tsup does not auto-externalize - // optionalDependencies, hence the explicit entry. - external: ["sharp"], + // jimp (#3400): pure-JS image encoder, devDependency inlined like every + // other bundled dep — dist stays self-contained, zero native modules. + noExternal: ["acp-kernel", "billion-context-pi-subagents", "fzstd", "jimp", "node-forge", "semver", "tar", "undici", "jsonc-parser", "ws"], banner: { // node-forge is a CommonJS dependency that calls require("crypto") etc. // inlined into our ESM output, esbuild's __require shim throws in an diff --git a/website/config-reference/compress.yaml b/website/config-reference/compress.yaml index 894aaafb52..87b4db99c0 100644 --- a/website/config-reference/compress.yaml +++ b/website/config-reference/compress.yaml @@ -219,10 +219,10 @@ entries: group: compress id: ct.rules render: compress-row - desc_en: "Persistent model reminders delivered through an injected acp_rule tool, hard-protected from folds; pi/omp expose an /acp-rule command." - desc_zh: "通过注入的 acp_rule 工具提供持久模型提醒——对折叠硬保护;pi/omp 提供 /acp-rule 命令。" - site_en: "Off by default: persistent model reminders delivered through an injected acp_rule tool, hard-protected from folds. pi/omp expose an /acp-rule command." - site_zh: "默认关:通过注入的 acp_rule 工具提供持久模型提醒——对折叠硬保护。pi/omp 提供 /acp-rule 命令。" + desc_en: "Persistent model reminders delivered through an injected acp_rule tool, hard-protected from folds; pi/omp expose an /acp-rule command. Proxy-mode only — inert in plugin mode, where the host owns the tool surface." + desc_zh: "通过注入的 acp_rule 工具提供持久模型提醒——对折叠硬保护;pi/omp 提供 /acp-rule 命令。仅代理模式生效——插件模式下宿主自管工具面,此项无效。" + site_en: "Off by default: persistent model reminders delivered through an injected acp_rule tool, hard-protected from folds. pi/omp expose an /acp-rule command. Proxy-mode only — inert in plugin mode." + site_zh: "默认关:通过注入的 acp_rule 工具提供持久模型提醒——对折叠硬保护。pi/omp 提供 /acp-rule 命令。仅代理模式生效——插件模式下无效。" - key: compress.injectTool type: boolean default: "true" @@ -255,6 +255,16 @@ entries: desc_zh: "丢弃超过 2048 字符的已结束轮次推理块(drop 默认 true);严格推理上游需设 drop:false。" site_en: "{ drop?, threshold? } — drop finished-round reasoning runs longer than 2048 chars (drop: true by default); strict-reasoning upstreams need drop:false." site_zh: "{ drop?, threshold? }——丢弃超过 2048 字符的已结束轮次推理块(drop 默认 true);严格推理上游需设 drop:false。" + - key: compress.absorb + type: object + default: "opt-in (disabled)" + group: compress + id: co.absorb + render: obj-item + desc_en: "Opt-in instant distillation of tool results into short abstracts; originals go to the content store. By default only image-bearing results qualify." + desc_zh: "可选即时蒸馏:把工具结果蒸馏成短摘要,原文进内容库。默认仅图片类结果参与。" + site_en: "absorb — opt-in instant distillation of tool results into short abstracts; originals go to the content store; by default only image-bearing results qualify (enabled, minToolTokens (default 1000), contextThresholdPct, excludeTools, mediaOnly (default true), toolName \"absorb\")." + site_zh: "absorb —— 可选即时蒸馏:把工具结果蒸馏成短摘要,原文进内容库;默认仅图片类结果参与(enabled, minToolTokens(默认 1000), contextThresholdPct, excludeTools, mediaOnly(默认 true), toolName \"absorb\")。" - key: compress.ccr type: object default: "enabled in proxy mode since v2" @@ -335,6 +345,42 @@ entries: desc_zh: "必须先置 true,自定义 prompts 才会生效。" site_en: "acknowledgePromptsRisk — must be true before custom prompts take effect." site_zh: "acknowledgePromptsRisk —— 必须先置 true,自定义 prompts 才会生效。" + - key: compress.absorb.enabled + type: boolean + default: "false" + group: compress + desc_en: "Enable the absorb distillation block." + desc_zh: "启用 absorb 蒸馏块。" + - key: compress.absorb.minToolTokens + type: number + default: "1000" + group: compress + desc_en: "Minimum estimated token size of a tool result eligible for absorption." + desc_zh: "可被吸收的工具结果的最小估算 token 大小。" + - key: compress.absorb.contextThresholdPct + type: "number | %" + default: "unset" + group: compress + desc_en: "Only absorb when context usage passes this share of the window." + desc_zh: "仅当上下文使用率超过窗口该占比时才吸收。" + - key: compress.absorb.excludeTools + type: "string[]" + default: "[]" + group: compress + desc_en: "Tool names excluded from absorption." + desc_zh: "排除出吸收范围的工具名。" + - key: compress.absorb.toolName + type: string + default: "\"absorb\"" + group: compress + desc_en: "Registered name of the absorb tool." + desc_zh: "absorb 工具的注册名。" + - key: compress.absorb.mediaOnly + type: boolean + default: "true" + group: compress + desc_en: "Restrict absorption to tool results that carry media (images). Image bytes are exempt from minToolTokens since they are not counted in text. Set false to restore legacy full-text absorption of any large result." + desc_zh: "仅吸收携带媒体(图片)的工具结果;图片字节不计入文本 token,故豁免 minToolTokens 门槛。置 false 恢复对任意大结果的全文吸收(旧行为)。" - key: compress.ccr.enabled type: boolean default: "true (proxy mode)" diff --git a/website/config-reference/env-only.yaml b/website/config-reference/env-only.yaml index 97857b2964..f1db743913 100644 --- a/website/config-reference/env-only.yaml +++ b/website/config-reference/env-only.yaml @@ -418,3 +418,11 @@ entries: group: envonly desc_en: "Spawn channel: marks a plugin-agent process." desc_zh: "派生通道:标记插件 agent 进程。" + - key: BILI_IMAGE_RESCALE_MAX_DIM + type: integer + default: "1600" + kind: env-only + audience: user + group: envonly + desc_en: "Longest-edge trigger threshold (px) for the egress image downscale pass (#3400); non-numeric or below 128 ignored (default applies). Env-only by design." + desc_zh: "出站图片降采样 pass 的最长边触发阈值(像素,#3400);非数字或低于 128 的取值按默认处理。设计上仅环境变量。" diff --git a/website/config-reference/process.yaml b/website/config-reference/process.yaml index 4df4cb4870..f12948a09c 100644 --- a/website/config-reference/process.yaml +++ b/website/config-reference/process.yaml @@ -243,8 +243,8 @@ entries: default: "true" env: BILI_SQLITE_AUTO_DOWNLOAD group: process - desc_en: "Gate for the one-time SQLite engine self-download (#2833): when no engine is available in-process (no built-in node:sqlite, no host-installed better-sqlite3) and the unified sidecar is wanted on this platform, bili downloads the pinned better-sqlite3 prebuilt tarball from the npm registry once (~11 MB, sha512-verified) into the cache dir and reuses it on every later start. Engine resolution order: built-in node:sqlite (Node >= 22.5) -> host-provided better-sqlite3 -> cached self-download -> self-download (gated by this switch) -> none (legacy-only, the #2825 degrade contract). The switch gates only the network fetch - a cached copy is used even when it is false; unsupported platforms never download." - desc_zh: "SQLite 引擎一次性自下载的闸门(#2833):进程内没有可用引擎(无内置 node:sqlite、宿主未装 better-sqlite3)且本平台希望启用统一边车时,bili 从 npm registry 一次性下载钉死的 better-sqlite3 预编译 tarball(约 11 MB,sha512 校验)到缓存目录,之后每次启动复用。引擎解析顺序:内置 node:sqlite(Node >= 22.5)-> 宿主机已装的 better-sqlite3 -> 缓存的自下载副本 -> 自下载(本开关控制)-> 无引擎(仅 legacy,即 #2825 降级契约)。开关只控制网络下载 —— 为 false 时缓存副本仍会使用;不支持的平台永不下载。" + desc_en: "Gate for the one-time SQLite engine self-download (#2833): when no engine is available in-process (no built-in node:sqlite, no host-installed better-sqlite3) and the unified sidecar is wanted on this platform, bili downloads the pinned better-sqlite3 prebuilt tarball from the npm registry once (~11 MB, sha512-verified) into the cache dir and reuses it on every later start. Engine resolution order: built-in node:sqlite (Node >= 22.5) -> host-provided better-sqlite3 -> cached self-download -> self-download (gated by this switch) -> none (legacy-only, the #2825 degrade contract). The switch gates only the network fetch - a cached copy is used even when it is false; unsupported platforms never download. The tarball download walks a builtin mirror list (#3338): the configured registry (BILI_UPDATE_REGISTRY / update.registry) is tried first, then https://registry.npmjs.org, then https://registry.npmmirror.com - the first mirror that yields a sha512-verified tarball wins; a Node < 22.5 with no engine gets an explicit Node upgrade hint in the degrade reason (#3385)." + desc_zh: "SQLite 引擎一次性自下载的闸门(#2833):进程内没有可用引擎(无内置 node:sqlite、宿主未装 better-sqlite3)且本平台希望启用统一边车时,bili 从 npm registry 一次性下载钉死的 better-sqlite3 预编译 tarball(约 11 MB,sha512 校验)到缓存目录,之后每次启动复用。引擎解析顺序:内置 node:sqlite(Node >= 22.5)-> 宿主机已装的 better-sqlite3 -> 缓存的自下载副本 -> 自下载(本开关控制)-> 无引擎(仅 legacy,即 #2825 降级契约)。开关只控制网络下载 —— 为 false 时缓存副本仍会使用;不支持的平台永不下载。 tarball 下载内置镜像列表兜底(#3338):先试配置的 registry(BILI_UPDATE_REGISTRY / update.registry),再试 https://registry.npmjs.org、https://registry.npmmirror.com —— 第一个通过 sha512 校验的镜像生效;Node < 22.5 且无引擎时,降级原因会给出明确的 Node 升级提示(#3385)。" - key: storage.jsonWriter type: "on|off" default: "on" diff --git a/website/config-reference/server.yaml b/website/config-reference/server.yaml index 9c46aba1d8..6d5e12b7d5 100644 --- a/website/config-reference/server.yaml +++ b/website/config-reference/server.yaml @@ -179,6 +179,17 @@ entries: desc_zh: "全局单图 token 上限,叠加在计费口径之上生效;逐请求实时读取。" site_en: "imageTokenCap — global per-image token cap; unset = uncapped (BILI_IMAGE_TOKEN_CAP wins, read live)." site_zh: "imageTokenCap —— 全局单图 token 上限;未设置 = 不限(BILI_IMAGE_TOKEN_CAP 优先且实时读取)。" + - key: imageRescale + type: boolean + default: "true" + env: BILI_IMAGE_RESCALE + group: upstream + id: tt.ir2 + site_group: up + desc_en: "Egress image downscale master switch (#3295/#3400): large outbound images are sent shrunken while originals stay untouched in session storage. Live-read per request." + desc_zh: "出站图片降采样总开关(#3295/#3400):大图以缩小形态发给模型,原图在会话存储中保持字节级原样。逐请求实时读取。" + site_en: "imageRescale — egress downscale switch; large outbound images sent shrunken, originals kept byte-exact in session storage." + site_zh: "imageRescale —— 出站图片降采样总开关;大图以缩小形态出站,原图在会话存储中保持字节级原样。" - key: maskHosts type: boolean default: "true" @@ -190,6 +201,16 @@ entries: desc_zh: "日志中把非公开目标主机遮成 ;凭据头无论此开关如何总是遮盖。" site_en: "maskHosts — mask non-public host names in logs as <private-host> (on by default; credentials are always masked)." site_zh: "maskHosts —— 日志中把非公开主机名遮成 <private-host>(默认开;凭据总是遮盖)。" + - key: security.adminRelay + type: boolean + default: "false" + group: behavior + id: tt.adminrelay + site_group: beh + desc_en: "Let an embedding host (dsh web today) relay the ENTIRE /__bili/ admin surface through its own entry origin so a remote browser can open the panel (#3373). Fail-closed, file-only. The relayed hop reaches the proxy as a local non-browser client (loopback source + loopback Host, no Origin), so the admin gate stays unchanged; enabling it exposes config editing and credential writes to whoever can reach the host web entry — protect that entry (auth / reverse proxy) first." + desc_zh: "允许宿主(当前为 dsh web)把它自己入口域名下的整个 /__bili/ 管理面中继到本机代理,使远程浏览器能打开面板(#3373)。默认关闭、仅配置文件。中继跳到达代理时是本机非浏览器客户端(回环源地址 + 回环 Host、无 Origin 头),管理端门槛不变;开启后,凡能访问宿主 web 入口的人都能改配置、写凭据——请先给入口加鉴权/反代。" + site_en: "security.adminRelay — relay the /__bili/ admin surface through the embedding host's web entry (off by default; fail-closed)." + site_zh: "security.adminRelay —— 将 /__bili/ 管理面经宿主 web 入口中继(默认关;fail-closed)。" - key: subagentSplit type: boolean default: "true" diff --git a/website/docs/index.html b/website/docs/index.html index 24e9b309f9..5e50a4492a 100644 --- a/website/docs/index.html +++ b/website/docs/index.html @@ -199,7 +199,7 @@

compress — every knob

neverPreserveRecentToolsstring[] ([] valid)Excluded from the recent protection zone (immediately compressible); default ["decompress", "search_context", "read", "bash"]. An empty array is valid — it excludes nothing (maximum protection). preserveRecentToolsstring[]Subtraction form: recent-zone tools minus this list get full protection. An empty array here is rejected as a typo. visibilityMarkersbooleanAppend 📦/❌ visibility markers after compress/decompress/search_context/acp_status results (on by default; off stops imitated narration). - rulesbooleanOff by default: persistent model reminders delivered through an injected acp_rule tool, hard-protected from folds. pi/omp expose an /acp-rule command. + rulesbooleanOff by default: persistent model reminders delivered through an injected acp_rule tool, hard-protected from folds. pi/omp expose an /acp-rule command. Proxy-mode only — inert in plugin mode. injectToolbooleanRegister the acp_compress tool for clients (on by default; global level only). injectNudgebooleanSend growth nudges as the window fills (on by default; global level only). reasoning{ drop?, threshold? }{ drop?, threshold? } — drop finished-round reasoning runs longer than 2048 chars (drop: true by default); strict-reasoning upstreams need drop:false. @@ -208,6 +208,7 @@

compress — every knob

    +
  • absorb — opt-in instant distillation of tool results into short abstracts; originals go to the content store; by default only image-bearing results qualify (enabled, minToolTokens (default 1000), contextThresholdPct, excludeTools, mediaOnly (default true), toolName "absorb").
  • ccr — content cache & retrieve: large outputs are stored losslessly beside the session and replaced by a head excerpt plus pointer; the model fetches originals back via acp_retrieve. No cap or eviction; on by default in proxy mode since v2 (enabled, minToolTokens (~4000), excludeTools, toolName "acp_retrieve", maxHeadChars).
  • search — search-behavior settings for search_context; currently holds planAware.
  • search.planAware — off by default: when on, search_context candidates are re-ranked against current plan state (latest-wins todos + last user turn); off keeps byte-identical lexical results.
  • @@ -261,9 +262,11 @@

    Other top-level fields

  • modelContextLimit — legacy global window ceiling (default 200000); prefer compress.modelContextLimit.
  • imageBilling — global image-token estimation: "auto" (default → pixels), "pixels" or "bytes" (explicit opt-in for byte relays).
  • imageTokenCap — global per-image token cap; unset = uncapped (BILI_IMAGE_TOKEN_CAP wins, read live).
  • +
  • imageRescale — egress downscale switch; large outbound images sent shrunken, originals kept byte-exact in session storage.
  • Behavior toggles
  • rollout — canary gating: features enable for a percent of installs, bucketed on a stable per-machine UUID; nested base rolls B out only inside A's cohort.
  • maskHosts — mask non-public host names in logs as <private-host> (on by default; credentials are always masked).
  • +
  • security.adminRelay — relay the /__bili/ admin surface through the embedding host's web entry (off by default; fail-closed).
  • subagentSplit — Claude subagents get their own session namespace (on by default).
  • forkAdoption — adopt anonymous forks into their parent session (off by default).
  • autoModeForge.enabled — answer Claude Code auto-mode classifier calls locally with a forged allow verdict (on by default; BILI_AUTO_MODE_FORGE=0 = pure passthrough).
  • diff --git a/website/main.js b/website/main.js index 35a9f413ce..bf7b1cc7c8 100644 --- a/website/main.js +++ b/website/main.js @@ -82,10 +82,11 @@ "ct.nprt": "Excluded from the recent protection zone (immediately compressible); default [\"decompress\", \"search_context\", \"read\", \"bash\"]. An empty array is valid — it excludes nothing (maximum protection).", "ct.prt2": "Subtraction form: recent-zone tools minus this list get full protection. An empty array here is rejected as a typo.", "ct.vm": "Append 📦/❌ visibility markers after compress/decompress/search_context/acp_status results (on by default; off stops imitated narration).", - "ct.rules": "Off by default: persistent model reminders delivered through an injected acp_rule tool, hard-protected from folds. pi/omp expose an /acp-rule command.", + "ct.rules": "Off by default: persistent model reminders delivered through an injected acp_rule tool, hard-protected from folds. pi/omp expose an /acp-rule command. Proxy-mode only — inert in plugin mode.", "ct.it": "Register the acp_compress tool for clients (on by default; global level only).", "ct.in": "Send growth nudges as the window fills (on by default; global level only).", "ct.rz": "{ drop?, threshold? } — drop finished-round reasoning runs longer than 2048 chars (drop: true by default); strict-reasoning upstreams need drop:false.", + "co.absorb": "absorb — opt-in instant distillation of tool results into short abstracts; originals go to the content store; by default only image-bearing results qualify (enabled, minToolTokens (default 1000), contextThresholdPct, excludeTools, mediaOnly (default true), toolName \"absorb\").", "co.ccr": "ccr — content cache & retrieve: large outputs are stored losslessly beside the session and replaced by a head excerpt plus pointer; the model fetches originals back via acp_retrieve. No cap or eviction; on by default in proxy mode since v2 (enabled, minToolTokens (~4000), excludeTools, toolName \"acp_retrieve\", maxHeadChars).", "co.searchRoot": "search — search-behavior settings for search_context; currently holds planAware.", "co.search": "search.planAware — off by default: when on, search_context candidates are re-ranked against current plan state (latest-wins todos + last user turn); off keeps byte-identical lexical results.", @@ -143,7 +144,9 @@ "tt.mcl2": "modelContextLimit — legacy global window ceiling (default 200000); prefer compress.modelContextLimit.", "tt.ib2": "imageBilling — global image-token estimation: \"auto\" (default → pixels), \"pixels\" or \"bytes\" (explicit opt-in for byte relays).", "tt.itc2": "imageTokenCap — global per-image token cap; unset = uncapped (BILI_IMAGE_TOKEN_CAP wins, read live).", + "tt.ir2": "imageRescale — egress downscale switch; large outbound images sent shrunken, originals kept byte-exact in session storage.", "tt.mask": "maskHosts — mask non-public host names in logs as <private-host> (on by default; credentials are always masked).", + "tt.adminrelay": "security.adminRelay — relay the /__bili/ admin surface through the embedding host's web entry (off by default; fail-closed).", "tt.subagent": "subagentSplit — Claude subagents get their own session namespace (on by default).", "tt.fork": "forkAdoption — adopt anonymous forks into their parent session (off by default).", "tt.autoforge": "autoModeForge.enabled — answer Claude Code auto-mode classifier calls locally with a forged allow verdict (on by default; BILI_AUTO_MODE_FORGE=0 = pure passthrough).", @@ -321,10 +324,11 @@ "ct.nprt": "从近期保护区排除(立即可压);默认 [\"decompress\", \"search_context\", \"read\", \"bash\"]。空数组合法=不排除任何工具(最大保护)。", "ct.prt2": "减法形式:近期区工具减去本列表得到完全保护;此处空数组按笔误拒绝。", "ct.vm": "在 compress/decompress/search_context/acp_status 结果后追加 📦/❌ 可见性标记(默认开;关闭可抑制模型模仿叙述)。", - "ct.rules": "默认关:通过注入的 acp_rule 工具提供持久模型提醒——对折叠硬保护。pi/omp 提供 /acp-rule 命令。", + "ct.rules": "默认关:通过注入的 acp_rule 工具提供持久模型提醒——对折叠硬保护。pi/omp 提供 /acp-rule 命令。仅代理模式生效——插件模式下无效。", "ct.it": "向客户端注册 acp_compress 工具(默认开;仅全局层级生效)。", "ct.in": "窗口填满过程中发送增长提醒(默认开;仅全局层级生效)。", "ct.rz": "{ drop?, threshold? }——丢弃超过 2048 字符的已结束轮次推理块(drop 默认 true);严格推理上游需设 drop:false。", + "co.absorb": "absorb —— 可选即时蒸馏:把工具结果蒸馏成短摘要,原文进内容库;默认仅图片类结果参与(enabled, minToolTokens(默认 1000), contextThresholdPct, excludeTools, mediaOnly(默认 true), toolName \"absorb\")。", "co.ccr": "ccr —— 内容缓存与回取:大输出无损存到会话旁、替换为首段摘录+指针,模型用 acp_retrieve 取回原文;无上限、永不清除;自 v2 起代理模式默认开(enabled, minToolTokens(~4000), excludeTools, toolName \"acp_retrieve\", maxHeadChars)。", "co.searchRoot": "search —— search_context 的检索行为设置;当前含 planAware。", "co.search": "search.planAware —— 默认关:开启后 search_context 候选按当前计划状态重排(todo 最新优先 + 上一轮用户消息);关闭时结果逐字节不变。", @@ -382,7 +386,9 @@ "tt.mcl2": "modelContextLimit —— 旧版全局窗口上限(默认 200000);建议改用 compress.modelContextLimit。", "tt.ib2": "imageBilling —— 全局图片 token 估算口径:\"auto\"(默认→pixels)、\"pixels\" 或 \"bytes\"(字节中继需显式指定)。", "tt.itc2": "imageTokenCap —— 全局单图 token 上限;未设置 = 不限(BILI_IMAGE_TOKEN_CAP 优先且实时读取)。", + "tt.ir2": "imageRescale —— 出站图片降采样总开关;大图以缩小形态出站,原图在会话存储中保持字节级原样。", "tt.mask": "maskHosts —— 日志中把非公开主机名遮成 <private-host>(默认开;凭据总是遮盖)。", + "tt.adminrelay": "security.adminRelay —— 将 /__bili/ 管理面经宿主 web 入口中继(默认关;fail-closed)。", "tt.subagent": "subagentSplit —— Claude 子代理拥有独立会话命名空间(默认开)。", "tt.fork": "forkAdoption —— 匿名 fork 会话并入父会话(默认关)。", "tt.autoforge": "autoModeForge.enabled —— Claude Code auto mode 分类器请求本地代答放行判定(默认开;BILI_AUTO_MODE_FORGE=0 纯透传)。",