Skip to content

feat(web): configurable dsh native compaction switch (allowDshCompaction) - #2034

Merged
ranxianglei merged 7 commits into
masterfrom
2026-10-04_dsh-compaction-toggle
Oct 5, 2026
Merged

ranxianglei merged 7 commits into
masterfrom
2026-10-04_dsh-compaction-toggle

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Closes #2028 (context: #1772, mechanism: #1729)

What

The owner-requested web-config toggle for dsh's built-in auto-compaction: a checkbox on the bili config page that lifts/restores the #1729 wire-level refusal of dsh native compaction envelopes, hot, per proxy instance.

Default OFF = today's behavior (guard active). Enabled = dsh native compaction calls are forwarded and may land.

Config surface (owner-gated discipline)

Explicit owner sign-off in the issue threads: #2028 OP and #2001 floor 10 ("我要的是 一个在bili配置界面允许点击关闭dsh压缩的功能").

New surface:

kind name default semantics
config field allowDshCompaction (top-level boolean) absent = false true lifts the #1729 refusal
env var BILI_ALLOW_DSH_COMPACTION unset "0" forces off, any other value forces on; wins over the file
web UI quick-settings checkbox unchecked mirrors the field; disabled + note when env-forced

Why the existing surface cannot express it: the refusal lives in bili's request pipeline (src/server/dsh-compaction-guard.ts, wired unconditionally in src/server.ts) and no existing switch touches it. The dsh-side alternative (restating the whole preset-standard plugins block in the profile cordis.patch.yml with auto:false, ~140 lines, #1772) is a drifting snapshot, needs a dsh restart, covers one profile at a time, and still would not lift the bili-side refusal — it cannot serve as this control. Considered and rejected: reusing passthrough (whole-request scope, different failure modes); a providers-table entry (compaction-basic is not a provider).

Compat/migration: purely additive, opt-in. Existing configs behave identically (default OFF = current refusal). GET /__bili/config gains allowDshCompaction: {enabled, source}; PUT accepts partial {allowDshCompaction: true|false|null} or whole-file saves. Hot-reload through the existing onChanged path (no restart); flips log warn/info.

Behavior changes (old -> new)

  1. None by default: without the new field/env, dsh compaction envelopes are still refused 403 exactly as before.
  2. Refusal message text changed (it rides into dsh's own warn line): old final sentence Disable dsh auto-compaction (bili preset 'standard-bili-auto-off') or remove the co-resident dsh compaction plugin. -> new To run dsh native compaction on this lane anyway, opt in explicitly: "allowDshCompaction": true in the bili config (web UI) or env BILI_ALLOW_DSH_COMPACTION=1 (#2028). Reason: the referenced preset does not exist anywhere in the codebase — stale reference found while wiring the real control.
  3. PUT contradiction handling is deliberately stricter than the passthrough precedent: a write contradicted by the env-forced state returns 409 in BOTH directions (the passthrough check only covers one). Rationale: silently accepting a contradicted write would make the saved state lie about the effective mode.
  4. Scope nuance (documented in the UI warning note): when enabled on NON-web profiles the shipped bundle patch's auto:false still suppresses AUTO-triggering — only manual /compact benefits there; on web profiles auto-triggering works as-is (no patch layer reaches the preset-nested instance, dsh.bundle.patch.yml 禁用自动压缩在 web profile 下静默失效 #1772). A landed checkpoint durably shadows the raw history (irreversible) and destroys bili's compression substrate for that session — hence opt-in with a warning, not a silent flip.

Passing fix (separate labeled commit)

KNOWN_TOP_LEVEL_KEYS was missing three real FileConfig fields — releaseNotesCheck/releaseNotesUrl (#1870) and imageTokenCap (#1843) — so config files carrying them logged a false "ignoring unknown top-level config key(s)" warning. Added to the mirror set and to the documented-surface test.

Tests

  • tests/issue2028-dsh-compaction-toggle.test.ts (new): resolution matrix (file/env/default OFF), state-source triad, refusal-message pin, and a full integration flip through a real server + mock upstream — default 403 -> live PUT enable -> forwarded byte-exact (200, observed at the upstream) -> clear -> 403 again — plus 400 validation and env-forced 409s in both directions.
  • tests/issue1815-unknown-top-level-keys.test.ts: documented surface extended with the four keys.
  • Local targeted runs green: issue2028 4/4, issue1815 4/4, plus dsh-compaction-guard / web-routing / named-provider-bind / web-i18n (39/39 across those five files). Full regression left to CI.

@github-actions

github-actions Bot commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-04_dsh-compaction-toggle (10a1c46)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.2198-pr.2034.b03c3d8

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr2034.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 Reviewing this PR now — pulling the branch, walking the diff against current master, running targeted local tests, and checking CI. Full structured verdict (七问) in a follow-up comment.

ranxianglei pushed a commit that referenced this pull request Oct 4, 2026
The #2028 change referenced bare `cfg` inside hydrateQuickConfig(), but
cfg is local to loadConfig() — a ReferenceError on every config-page load,
swallowed by loadConfig's catch into the failure toast, leaving the
quick-settings card half-built (dsh checkbox and all later rows never
rendered) and skipping the passthrough badge + upstream-state updates.
Pass cfg as a parameter, same pattern as loadUpstream(cfg). Invisible to
tsc because the browser bundle is a template-literal string.
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 Review of PR #2034 complete — found and fixed one blocking UI regression in the process. Verdict at the bottom.

Review work performed

  1. Full diff read (10 files, +364/−9) against the 在bili网页配置界面支持关闭和开启dsh的自动压缩功能 #2028/nudge 死锁无检测:同一 session 交替收到两种结构互斥的请求流(巨型单消息 ↔ 多轮)时不告警 #1729/dsh.bundle.patch.yml 禁用自动压缩在 web profile 下静默失效 #1772 context.
  2. Branch is 108 commits behind master; audited master-side churn on the same files (src/server.ts +320, src/config.ts +71, src/web/api.ts +10) hunk-by-hunk for semantic overlap: [feature] configurable wire-path table (per-provider custom paths) - Claude Desktop's claude.ai private protocol as motivating case #1909 declaredProtocol feeds the guard's protocol !== null precondition, but dsh envelopes already resolve openai via the suffix heuristic (no behavior delta); feat: support wss:// (WebSocket) traffic — opt-in transparent passthrough now, protocol-aware compression later #1467 WS-lane veto and the api.ts parseRouteEntry try/catch are independent; the config.ts hunk is adjacent, non-overlapping. No semantic conflicts.
  3. Rebased the head onto current master (e6b28571) locally: clean, zero conflicts.
  4. Targeted suites (full regression stays with CI): issue2028-dsh-compaction-toggle + issue1815-unknown-top-level-keys + dsh-compaction-guard + web-routing + web-i18n + named-provider-bind = 43/43 green on both the PR head and the rebased head; typecheck clean on both.
  5. Verified the stale-preset claim: standard-bili-auto-off exists nowhere in the repo except the new test's negative assertion — the old refusal text genuinely pointed at a non-existent preset.

Blocking defect found while reviewing (fixed, pushed as commit 94682da1)

The new code in src/web/client.ts references bare cfg inside hydrateQuickConfig() (line 1250), but cfg is local to loadConfig() (line 1088). Consequence: ReferenceError: cfg is not defined on every config-page load, swallowed by loadConfig's catch into the failure toast — the quick-settings card left half-built (the new dsh checkbox and every later row: pack/nudge/prm/never/mitm never render), and the passthrough badge + upstream-state updates skipped. tsc cannot see it because the browser bundle is a template-literal string, which is why typecheck + tests stayed green.

Reproduced at runtime with a DOM-stub harness that extracts loadConfig+hydrateQuickConfig out of the bundle (kept in this issue workspace: .tmp/repro-cfg-scope.mjs, .tmp/repro-cfg-envforced.mjs): pre-fix → failure toast + only 3/14 top-level rows rendered; post-fix → clean, 14/14 rows, and under env force the checkbox renders disabled with the env note, as designed.

The fix is 2 lines, following the existing loadUpstream(cfg) parameter-passing pattern: call site hydrateQuickConfig(cfg) + signature. Pushed as an additive labeled commit to the PR branch (fast-forward, no history rewrite).

Coverage-gap note: the browser bundle has zero DOM-execution coverage in the repo (pre-existing — no quick-config row is tested), which is how this slipped past local runs. I did not commit a DOM harness as part of this PR (new infra, out of scope); flagging it as a known gap instead of silently dropping it.

结构化七问

  1. 是什么问题:在bili网页配置界面支持关闭和开启dsh的自动压缩功能 #2028 功能 —— 让 nudge 死锁无检测:同一 session 交替收到两种结构互斥的请求流(巨型单消息 ↔ 多轮)时不告警 #1729 wire 守卫对 dsh 原生压缩调用的无条件 403 拒绝变为可配置(allowDshCompaction),默认保持拒绝。
  2. 什么原因导致的:dsh compaction-basic 应对上下文压力时重放会话前缀 + 固定摘要指令,此类调用一旦落地其 checkpoint 永久覆盖原始历史(不可逆)并摧毁 bili 压缩基底,故 nudge 死锁无检测:同一 session 交替收到两种结构互斥的请求流(巨型单消息 ↔ 多轮)时不告警 #1729 一律拒绝;想使用 dsh 原生压缩的用户没有出口,且旧拒绝文案指向一个不存在的 preset。
  3. 怎么修复的:按既有 env+file+web 三件套模式加开关(BILI_ALLOW_DSH_COMPACTION / 顶层文件键 / quick-settings 复选框);allowDshCompactionState() 作为 loadOptions 与 web API 的单一事实源;守卫条件只加一个合取项(位置/顺序/前置 lane 不变);热加载走既有 onChanged 路径并在翻转时记 warn/info;web PUT 沿用 passthrough 的读/清契约,env 强制时矛盾写入双向 409。附带修复(作者独立提交 31c5f303):KNOWN_TOP_LEVEL_KEYS 补齐 releaseNotesCheck/releaseNotesUrl/imageTokenCap(发版可见性:分级标签的 release 元数据(模型撰写摘要)+ 用户版跨 span 更新说明 + acp_status //acp 面板送达「更新已就绪待重启」(#1577 延伸) #1870/Bill images and text separately — the image estimate's ±15× error bar poisons every window decision (dual-channel accounting) #1843),消除对真实文档化键的误报。审查中发现本 PR 引入的 cfg 作用域缺陷并已直接修掉(见上,commit 94682da1)。
  4. 是否有回归问题:既有行为逐项审过——默认守卫行为不变(集成测试钉住默认 403 且被拒请求不达上游)、守卫触发位置与顺序不变、拒绝体 shape/status/error.code 不变(仅文案末句变化,属已披露变更且旧引用确属失效)、server 热加载块纯增量、web GET/PUT 契约纯增量、[config] misplaced/unknown top-level keys are silently ignored — root-level promptPack does nothing without any warning #1815 未知键告警面仅对 3 个真实存在的键收窄、i18n 双语 parity 有测试钉住。本 PR 原 head 含一处真实回归(web 配置页 hydration 每次加载崩溃),已修复并用运行时复现验证;master 侧 108 个提交的语义交叉已逐 hunk 排除。
  5. 是否破坏缓存:不涉及——prefix-cache 标注、持久化格式、ref/id 逻辑、throttle/重试/去重键均未触碰;config schema 为纯增量扩展,无版本迁移。
  6. 是否改了设计方向:不涉及——沿用既有开关模式,无架构转向;状态单一事实源比各层各自推导更稳。
  7. 是否引入了新的配置:是——allowDshCompaction(顶层布尔,缺省 = false)、env BILI_ALLOW_DSH_COMPACTION("0" 强制关,其他值强制开,优先于文件)、web 复选框(env 强制时禁用并显示说明)。默认值完全保持现状,无需迁移。

测试有效性:新测试是真行为测试——真 server + mock upstream 的 refuse→forward→refuse 全程热翻转、转发字节级断言、拒绝文案钉住、解析矩阵、双向 env-forced 409,机制回归时抓得住。

判定:质量关已过,可以合并。feat 类按 §7.4 归 human merge;分支仍 behind master(本地已验证 rebase 干净、rebase 后定向测试全绿),直接 merge 或先 rebase 均可。CI 在新 head 94682da1 上运行中(e2e/version-guard 已绿,test 矩阵在跑),全绿后即可合。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 CI update — the one red cell is a pre-existing flake, now resolved

The test (windows-latest, 22) cell failed once with:

not ok 3142 - Responses WS: a title-persona side request rides the lane transport instead of 502ing
  error: The input did not match the regular expression /side request \(agent=title\)/. Input: ''

I re-ran only that job — it passed on the same commit, so CI is now fully green on 94682da1 (all four matrix cells + version-guard; every e2e workflow was already green).

Root cause of the flake (verified in code, unrelated to this PR's surface): the title-persona test in tests/responses-websocket.test.ts reads the proxy log with a raw synchronous fs.readFileSync immediately after the WS turn completes — a race against the tee-logger's file flush on loaded runners. That exact race class is already known (#1968) and was fixed for the rest of this file by 97ff781, which introduced the readLogUntil() polling helper; this test postdates that sweep and is the last raw reader left. Evidence it's not your change: the test passed on the other three matrix cells of the same commit, and the PR doesn't touch the WS lane or side-request routing.

Since the fix belongs on master (the test file doesn't exist on this branch — adding it here would dump ~760 unrelated lines into this diff), I filed #2040 with the full analysis and opened one-line follow-up PR #2041 (fs.readFileSync → await readLogUntil(f.logPath, [/side request \(agent=title\)/]), both original assertions kept). Verified locally: that file 36/36 + typecheck clean.

Nothing blocks merging this PR right now — #2041 just prevents the recurrence. Verdict from my previous comment stands: mergeable once you're ready.

ranxianglei pushed a commit to yingzhongshuoqi/billion-context that referenced this pull request Oct 4, 2026
… flush (ranxianglei#2040)

The title-persona test was the last reader in this file still using a raw
synchronous fs.readFileSync right after turn completion. On loaded runners
the tee-logger write lags the wire completion, so the snapshot misses a
line that was logically already emitted — observed as a red
windows-latest/Node 22 cell in PR ranxianglei#2034's CI (run 37179443315), green on
re-run, green on the other three matrix cells.

Use the established readLogUntil() polling helper (added by 97ff781 for
exactly this race class, ranxianglei#1968). Both original assertions are unchanged
and now vet the settled snapshot.
@ranxianglei

Copy link
Copy Markdown
Owner Author

解决冲突

@ranxianglei
ranxianglei force-pushed the 2026-10-04_dsh-compaction-toggle branch from 531a143 to 7073280 Compare October 5, 2026 05:38
ranxianglei pushed a commit that referenced this pull request Oct 5, 2026
The #2028 change referenced bare `cfg` inside hydrateQuickConfig(), but
cfg is local to loadConfig() — a ReferenceError on every config-page load,
swallowed by loadConfig's catch into the failure toast, leaving the
quick-settings card half-built (dsh checkbox and all later rows never
rendered) and skipping the passthrough badge + upstream-state updates.
Pass cfg as a parameter, same pattern as loadUpstream(cfg). Invisible to
tsc because the browser bundle is a template-literal string.
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 冲突已解决,分支已 rebase 到当前 master(1d592621)并强推:新 head 70732806,GitHub 已显示 mergeable(无冲突,等 CI)。

冲突点与解法(共两轮,master 在期间又进了 #2135/#2133):

验证(本地定向,全量交 CI):typecheck 干净;定向套件 64/64 通过——issue2028、issue1815、dsh-compaction-guard、web-routing、web-i18n、named-provider-bind、config-file-cache(#2078 缓存行为)、knobs-config-precedence、preflight-stream-summary-config(#2133,确认 rebase 未伤及)。对 master 的最终 diff 恰好是本 PR 自己的 11 个文件(+454/−17),无夹带。

CI 跑完前结论不变:可合并。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 Rebase 后首轮 CI 里新发现一个失败:docs-coverage 门禁(master 近期 #2107「config 文档单一数据源」引入的 tools/gen-config-docs.mjs check-coverage)。

原因:该门禁不读 markdown,而是把 src/ 实际读取的 env 变量名和 KNOWN_TOP_LEVEL_KEYS 逐个对照 website/config-reference/*.yaml 种子。本 PR 早于 #2107,文档行是手写进 CONFIGURATION.md 表格的——门禁根本不看那里,所以报 BILI_ALLOW_DSH_COMPACTION(env)和 dsh(顶层键)UNDOCUMENTED。这是 rebase 到新 master 后才暴露的交互,不是原分支的问题。

修复(已推,新 head 46927907,单独一个 commit):在 server.yaml 种子加一条 dsh.allowDshCompaction(type boolean / default false / env BILI_ALLOW_DSH_COMPACTION / group behavior),然后跑 node tools/gen-config-docs.mjs generate 重新生成 10 个区块。本地 check-coverage 通过("139 env names + 49 top-level keys all documented"),i18n 校验也过。生成的两张表各多一行;legacy 详述表里的手写长描述保留不动(与 #2107 后其他条目的双表并存模式一致)。diff 仅 3 文件 +11 行。

等这轮 CI 结果。

ework-agent and others added 7 commits October 5, 2026 13:54
…ion) (#2028)

The #1729 wire guard refuses dsh native compaction envelopes unconditionally;
users who deliberately want dsh's built-in auto-compaction to run through bili
had no control over that refusal (and on web profiles no patch layer reaches
the preset-nested compaction-basic instance at all, #1772).

Add the owner-requested web-config toggle:
- config field allowDshCompaction (default false = guard stays active,
  current behavior unchanged) + env BILI_ALLOW_DSH_COMPACTION (env wins);
- web UI quick-settings checkbox with an irreversibility warning note and an
  env-forced disabled state (mirrors the passthrough row);
- GET /__bili/config exposes {enabled, source}; PUT accepts partial
  {allowDshCompaction: true|false|null} and whole-file saves, with a 409 when
  the write contradicts the env-forced value (symmetric, stricter than the
  passthrough precedent); hot-reload via the existing onChanged path logs the
  flip;
- refusal message now points at the real opt-in instead of the nonexistent
  'standard-bili-auto-off' preset (stale reference fixed in passing);
- docs: CONFIGURATION.md en/zh; regression suite pins the resolution matrix,
  the API contract, and the live refuse->forward->refuse flip.
The #2028 change referenced bare `cfg` inside hydrateQuickConfig(), but
cfg is local to loadConfig() — a ReferenceError on every config-page load,
swallowed by loadConfig's catch into the failure toast, leaving the
quick-settings card half-built (dsh checkbox and all later rows never
rendered) and skipping the passthrough badge + upstream-state updates.
Pass cfg as a parameter, same pattern as loadUpstream(cfg). Invisible to
tsc because the browser bundle is a template-literal string.
…ion (#2028)

The switch field never shipped in any release, so both surfaces move cleanly:
- file key: top-level "allowDshCompaction" -> "dsh": { "allowDshCompaction": true }
  (DshFileSettings section for future dsh keys); bare legacy values are
  auto-relocated on load AND on whole-file save instead of being dropped
  silently (normalizeLegacyAllowDshCompaction); non-boolean values stay in
  place so the unknown-top-level-key warning still names them.
- structured PUT now takes { dsh: { allowDshCompaction: true|null } }; the
  GET resolved view ({ enabled, source }) and env BILI_ALLOW_DSH_COMPACTION
  semantics are unchanged.
@ranxianglei
ranxianglei force-pushed the 2026-10-04_dsh-compaction-toggle branch from 4692790 to 10a1c46 Compare October 5, 2026 05:55
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 收工:CI 全绿,PR 已可合并。

  • 最终 head 10a1c46d(docs 种子 commit 46927907 rebase 后的新号),20/20 check runs 全部通过——含之前红的 docs-coverage、四格 test matrix(windows-22 这次也绿)和全部 e2e lane。
  • 期间 master 又前进了 3 个 commit(test: code prover proving cache-byte stability across failure paths (#2138) #2144,纯新增 tests/cache-proof.test.ts +1068 行,与本 PR 零文件交集),已顺手 rebase 上去(无冲突),本地冒烟 issue2028/issue1815/dsh-compaction-guard 18/18。
  • GitHub 状态:mergeable: true, mergeable_state: clean,base = 当前 master bb41405c。

合并动作留给你(规则上我不能合 PR)。之前 review 结论不变:功能实现正确、测试有牙、行为变更已在 PR 描述里披露,可以合并。

@ranxianglei
ranxianglei merged commit 25d358c into master Oct 5, 2026
20 checks passed
@github-actions

github-actions Bot commented Oct 5, 2026

Copy link
Copy Markdown
Contributor

📦 Merged Build Published

This merge to master was published as 0.0.38-master.25d358c (rolling master tag).

npm install -g billion-context@0.0.38-master.25d358c  # this exact build
npm install -g billion-context@master  # always the newest merged build

npm view billion-context@master prints the current version at any time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

在bili网页配置界面支持关闭和开启dsh的自动压缩功能

1 participant