Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,15 @@ Versions follow the merge of a `*_release-v*` branch; CI publishes to npm on tag

## [Unreleased]

### 存储收尾九项一车(#3901,修复 #3463/#3762/#3841/#3798/#3633;#3723 落地)

- **GC 与归档(#3463)**:新 `storage.archiveAfterDays`(环境变量 `BILI_STORAGE_ARCHIVE_AFTER_DAYS`,缺省 365,0=关)——闲置超期的会话归档(完整导出到 sessions 目录下 `archive/` + 墓碑行保标签 + 释放全部内容/媒体引用),`POST /__bili/storage/gc`(可带 `vacuum`)手动触发,`POST /__bili/storage/restore` 恢复;`auto_vacuum=INCREMENTAL` 常开(存量库首次需 `?vacuum=full` 收缩)。
- **reap 台账(#3841)**:schema v6 `sessions.orphans_json`——收割形状(state.blocks 失块、blockContents 留文)现在定向记台账:被收割块行删除(引用计数释放),孤儿内容并入视图但不复活 state.blocks;重启不再复活已收割块、不再污染信封。
- **媒体 CAS(#3762)**:schema v7 `message_media`——图片 base64 以内容哈希进 CAS、消息行只存 `{"$media":hash}` 引用(读回逐字节水合),同图多会话只存一份;删除/归档时引用计数释放。
- **增量指针(#3798)**:`inc:canonical-ptrs` 逐行指针替代 base-seq 计数切片——resync 共享前缀的行复用不再丢历史;resync 时点先失效旧指针(响应前持久化),崩溃窗口退回旧切片语义(永不供错分支);turn 锁重排:响应阶段 409 turn-in-flight 照旧、ack 之后的回写链被下一轮 await(ack 永不超前行落盘)。
- **dsh 代压缩默认开(#3723/#3733 落地)**:`dsh.authoredCompaction` 缺省 true(环境变量 `BILI_DSH_AUTHORED_COMPACTION=0` 或文件 false 恢复 #1729 拒绝路径)——0.1.203 狗粮窗口守住,失败模式只是 checkpoint 不缩,不丢数据。
- **多实例姿态文档(#3633)**:README 新章节「Running multiple instances」:一会话一实例、L7 按会话头哈希才是正确的 sticky 路由(ClientIP affinity 是故障转移反模式)、RWO 一实例一卷(NFS 上 WAL=静默损坏)、/__bili/replicas 是 checkpoint 不是 pod 副本。

### 快照渲染影子对账(#3756 后续,默认开启)

- 新增影子车道:灰度仍为 0% 的出站快照引擎在每轮已折叠会话上现算视图,与 legacy 实际出站比对后丢弃 —— 出站字节零变化,纯观测。
Expand Down
7 changes: 6 additions & 1 deletion CONFIGURATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,7 @@ This index is generated from `website/config-reference/*.yaml` — edit the seed
| `compat.noCacheControl` | boolean | false | BILI_NO_CACHE_CONTROL | Stop stamping Anthropic-lane cache_control breakpoints entirely (escape hatch for upstreams/relays with their own breakpoint policy). |
| `compat.keepResponseId` | boolean | false | ACP_KEEP_RESPONSE_ID | Preserve previous_response_id on kernel-rebuilt Responses requests (default strips it so rebuilt bodies never reference ids the upstream never issued). |
| `dsh.allowDshCompaction` | boolean | false | BILI_ALLOW_DSH_COMPACTION | Lift bili's local 422 refusal of dsh native compaction calls on EVERY wire lane bili serves — openai, anthropic, and responses (#1729/#2028; responses added by #2360, the lane dsh desktop's compaction actually rides): when true, dsh's compaction-basic may execute through bili; its checkpoint durably shadows raw history, so this is an explicit irreversible opt-in. |
| `dsh.authoredCompaction` | boolean | true (on, #3901 flip) | BILI_DSH_AUTHORED_COMPACTION | bili answers dsh compaction summarize calls locally with the folded snapshot instead of refusing (#3723/#3733). ON by default since the 0.1.203 dogfood window held; false (or env BILI_DSH_AUTHORED_COMPACTION=0) restores the #1729 refusal path. |
| `resign` | scheme → { enabled?, passthrough?, credentialRef? } | {} (armed; built-in scheme sdk-hmac-sha256) | BILI_RESIGN, BILI_RESIGN_PASSTHROUGH, BILI_CODEARTS_REF, BILI_RESIGN_BENEFIT | Re-sign arm keyed by Authorization scheme: signed model requests tunnel with every egress body re-signed; unresignable requests are refused locally 403 unless passthrough opts that scheme into verbatim forwarding. |
| `promptCache.routing` | "auto" \| "enabled" \| "disabled" | auto | ACP_PROMPT_CACHE_ROUTING | Prompt-cache routing posture for cache-aware lane selection. |
| `native.attachExternal` | boolean | false | BILI_NATIVE_ATTACH_EXTERNAL | Let launcher-less native plugins attach to an external (lane'd, unarmed) daemon instead of spawning their own. |
Expand Down Expand Up @@ -177,6 +178,7 @@ This index is generated from `website/config-reference/*.yaml` — edit the seed
| `storage.sqliteAutoDownload` | boolean | true | BILI_SQLITE_AUTO_DOWNLOAD | Gate for the one-time SQLite engine self-download (#2833): when no engine is available in-process (no built-in node:sqlite, no host-installed better-sqlite3) and the unified sidecar is wanted on this platform, bili downloads the pinned better-sqlite3 prebuilt tarball from the npm registry once (~11 MB, sha512-verified) into the cache dir and reuses it on every later start. Engine resolution order: built-in node:sqlite (Node >= 22.5) -> host-provided better-sqlite3 -> cached self-download -> self-download (gated by this switch) -> none (legacy-only, the #2825 degrade contract). The switch gates only the network fetch - a cached copy is used even when it is false; unsupported platforms never download. The tarball download walks a builtin mirror list (#3338): the configured registry (BILI_UPDATE_REGISTRY / update.registry) is tried first, then https://registry.npmjs.org, then https://registry.npmmirror.com - the first mirror that yields a sha512-verified tarball wins; a Node < 22.5 with no engine gets an explicit Node upgrade hint in the degrade reason (#3385). A failed download arms a 6-hour retry cooldown persisted as a marker file under the cache dir (billion-context/sqlite/.bootstrap-failed.json); the storage diagnostics blob shows the marker's absolute expiry time and path - delete that file and restart bili to force an immediate retry instead of waiting out the whole window (#3788). |
| `storage.jsonWriter` | on\|off | on | BILI_STORAGE_JSON_WRITER | Phase 3 envelope switch (#2960): off stops writing the legacy JSON session envelope while storage.unified is full - the SQLite index becomes the only persistent copy (reads are already unified-first; boot enumerates the index union, so index-only sessions load normally). Only honored in effective-full mode: under shadow/legacy (or with encryption active) the request is ignored with a one-time warning because the JSON file remains the sole source of truth. Env tier 0/false/off = off and 1/true/on = on wins over the strict file spellings. Fail-safe: if the sidecar disables itself mid-run, envelope writes resume on the next save. Rollback: unset the knob - writing resumes, index-only sessions keep loading via the boot union, and replicas/restore re-materialize portable JSON envelopes. Downgrading the binary (or losing the SQLite engine) makes index-only sessions invisible to v0.1.196 and older until a >=v0.1.197 binary runs again - the data stays intact in index.db, it is just not read by the older code. The per-session content-store sidecar keeps being written either way. A jsonWriter=off lane must be the SOLE writer of its sessions directory: an off lane reads unified-first and never falls back to the file, so any peer instance running legacy/shadow against the same directory diverges silently (its JSON copy stops receiving updates, and the off lane never sees the peer's writes). |
| `storage.walJournalSizeLimitBytes` | number | 8388608 | BILI_STORAGE_WAL_JOURNAL_SIZE_LIMIT_BYTES | WAL reclamation cap for index.db-wal (#3250), in bytes: every completed checkpoint truncates the WAL file back to this size, bounding disk use and crash-recovery replay in long-lived processes. NOT a per-session data quota - data lands in index.db as it is written and the WAL only stages frames between checkpoints, so session count/size does not scale this; a large burst can overshoot temporarily until the next completed checkpoint truncates back. Higher = rarer truncate IO but more steady disk use and a longer crash-replay tail. Env tier (positive integer) wins over the file tier; a garbage or non-positive env value falls through to the file/default tier instead of failing boot. Applied when the store opens at lane boot - hot edits need a restart. The effective value appears on the unified-enable log line and in the storage diagnostics blob. |
| `storage.archiveAfterDays` | integer | 365 | BILI_STORAGE_ARCHIVE_AFTER_DAYS | GC age sweep window in days (#3463, #3901): sessions idle longer than this are archived - full export to the archive dir under the sessions directory, a tombstone row keeps the label and identity, content/media refs are released, and restore is one POST away. 0 disables the age sweep; POST /__bili/storage/gc always runs and takes ?vacuum=full to shrink an existing index.db (auto_vacuum=INCREMENTAL is always on for new writes). Env wins over the file tier. |
| `sessions` | object | {} (all defaults) | — | Session-table size cap and opt-in garbage collection. |
| `sessions.max` | number | 256 | BILI_MAX_SESSIONS | In-memory session table cap (LRU eviction of idle sessions). |
| `sessions.warmMax` | number | 0 | BILI_SESSION_WARM_MAX | Bounded startup (#2803/#3050): max most-recently-saved sessions hydrated at boot before listening; older state reloads on demand. Default 0 (no eager warm set — first touch pays one on-demand load; set 8 to restore the eager behavior). Capped by sessions.max. |
Expand Down Expand Up @@ -330,7 +332,8 @@ This index is generated from `website/config-reference/*.yaml` — edit the seed

| Key | Type | Default | Env | Description |
|-----|------|---------|-----|-------------|
| `BILI_DSH_AUTHORED_COMPACTION` | boolean | false | — | Let bili ANSWER dsh compaction summarize calls locally with the folded snapshot instead of refusing (#3733): identity header + kernel-rendered block summaries, zero egress; replay recognized via the #2432 rebase. |
| `BILI_DSH_AUTHORED_COMPACTION` | boolean | true (on, #3901 flip) | — | Let bili ANSWER dsh compaction summarize calls locally with the folded snapshot instead of refusing (#3733): identity header + kernel-rendered block summaries, zero egress; replay recognized via the #2432 rebase. |
| `BILI_STORAGE_ARCHIVE_AFTER_DAYS` | integer | 365 | — | GC age sweep window in days (#3463): sessions idle longer than this are archived (full export to the archive dir + tombstone row + content/media refs released). 0 disables the age sweep; the manual entry POST /__bili/storage/gc always runs. Config-file key storage.archiveAfterDays is the persistent form. |
| `BILI_SNAPSHOT_SHADOW` | boolean | true (on) | — | Snapshot-render shadow observability (#3756 follow-up): compute the outbound snapshot view on every armed turn WITHOUT using it, classify the diff against the legacy outbound, and surface per-session counters in the web panel + the snapshot-shadow diagnostics topic. 0/false/off disables; 1/true/on enables; otherwise the config-file snapshot.shadow boolean applies (default on). The render lane itself stays rollout-gated (BILI_ROLLOUT_SNAPSHOT_RENDER), so the shadow lane never changes outbound bytes - it is the dry-run evidence that flips the rollout later. |
| `BILI_CONFIG_FILE` | string | unset (XDG config path) | — | Override the config file path (path relocation). |
| `BILI_SESSIONS_DIR` | string | unset (XDG state path) | — | Where session records are stored (path relocation). |
Expand Down Expand Up @@ -1717,6 +1720,7 @@ File keys resolve only when the matching env var is unset. Defaults in parenthes
| `BILI_CODEX_COMPACT` | `codexCompact` | "intercept" |
| `BILI_DECODED_BODY_BYTES` | `network.decodedBodyBytes` | 10737418240 |
| `BILI_DECOMPRESS_TMP_CAP` | `decompressTmpCap` | 50 |
| `BILI_DSH_AUTHORED_COMPACTION` | `dsh.authoredCompaction` | true (on, #3901 flip) |
| `BILI_DUMP_4XX` | `diagnostics.dump4xx` | false |
| `BILI_DUMP_4XX_MAX_BYTES` | `diagnostics.dump4xxMaxBytes` | 2097152 (floor 1024) |
| `BILI_DUMP_MAX_AGE_DAYS` | `diagnostics.maxAgeDays` | unset (off) |
Expand Down Expand Up @@ -1778,6 +1782,7 @@ File keys resolve only when the matching env var is unset. Defaults in parenthes
| `BILI_SESSION_WARM_MAX_BYTES` | `sessions.warmMaxBytes` | 67108864 |
| `BILI_SQLITE_AUTO_DOWNLOAD` | `storage.sqliteAutoDownload` | true |
| `BILI_STABLE_SYSTEM_ANCHOR` | `stableSystemAnchor` | false |
| `BILI_STORAGE_ARCHIVE_AFTER_DAYS` | `storage.archiveAfterDays` | 365 |
| `BILI_STORAGE_JSON_WRITER` | `storage.jsonWriter` | on |
| `BILI_STORAGE_UNIFIED` | `storage.unified` | unset (full) |
| `BILI_STORAGE_WAL_JOURNAL_SIZE_LIMIT_BYTES` | `storage.walJournalSizeLimitBytes` | 8388608 |
Expand Down
Loading
Loading