Skip to content

feat(dsh-lane): converge all three ledgers on in-place self-heal + bili doctor --fix (#3496) - #3509

Merged
ranxianglei merged 3 commits into
masterfrom
2026-10-17_dsh-ledger-convergence
Oct 17, 2026
Merged

ranxianglei merged 3 commits into
masterfrom
2026-10-17_dsh-ledger-convergence

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

(ai: vllm-qwen/qwen3.8-27b)

Closes #3496

What this does

The dsh desktop lane is the one copy bili overwrites in place (self-heal swaps node_modules/billion-context from a verified registry tarball, #1575/#2894). Until now that writer moved only ONE of the three ledgers — the manifest spec (#2894) — while deliberately leaving the lockfile record naming the OLD version. That manufactured exactly the wedge class #3224 detects and #3160 bit users on: after an auto-update, the profile's pnpm bookkeeping still names the old version, so dsh's in-app plugin manager can revert the copy on the next "install" click, or a hand-edited manifest (^1.2.9 phantom) makes the next pnpm install 404.

This PR closes the loop per the issue's three requirements:

  1. Root cause — self-heal converges ALL THREE LEDGERS in one transaction. New src/dsh-ledger.ts: convergeDshLedger(profileDir, targetVersion) plans every change first (all-or-nothing), then writes atomically (tmp+fsync+rename, the [dsh] #2894/#2891 dsh lane 状态同步与检测:四个子缺陷(清单原子写 / lockfile 一致性 / Windows 大小写 / doctor 告警收敛窗口) #2972-M1/[bug] 会话状态写入缺 fsync:内核崩溃后留下 size 正常但 VDL=0 的全零状态文件,loadSync 永久抛错 → 502 acp-proxy failure 且不自愈(含 7 处 tmp+rename 写路径盘点) #3157 discipline) — manifest spec (via split-out planDshProfileManifestSpec, byte-identical rules) + pnpm-lock.yaml record (nested v6+/v9 importers, flat ≤5.4 specifiers:/dependencies:, v6–v8 top-level sections) + package-lock.json fallback (npm-materialized profiles). The installViaTarball success hook (src/update.ts) now calls it instead of bare manifest sync — desktop-gated, so the global self-update and advisory call sites are untouched.
  2. Explicit repair for pre-existing drift — bili doctor --fix. Walks every dsh profile (desktop included — detection excludes it by design, but repair must reach the very profile where Profile 副本自愈可被本地高版本号永久卡死,且 out-of-band 副本无来源校验(#1196 变体) #3160 happened), converges drifted profiles to their on-disk copy version, prints every before→after change, re-audits for residuals, exits non-zero when anything remains. Healthy profiles get ZERO writes (byte-identical, tested).
  3. Detection preserved. This PR is self-contained on master; open PR feat: detect manifest/lockfile/copy three-way disagreement in dsh profiles (#3224) #3231 (provenance detection: startup audit + doctor warnings) stays complementary as the last defense against NEW third-party writers. Its lockfile readers duplicate mine — proposed follow-up (separate issue after both merge): dedup so one module owns the readers.

config surface

New CLI flag: bili doctor --fix (no env vars, no config-file fields; --json output gains an optional ledgerFixes array). Shape proposed by the owner in the issue body ("如 bili doctor --fix 形态") — recorded here as the sign-off per AGENTS.md Configuration Surface Discipline. Justification: the existing surface cannot express "repair" at all — doctor was read-only by contract, and a finding's remediation text pointing at re-running install through the host channel does not work for the DESKTOP profile (the CLI refuses --profile desktop, #1575) and would not touch the lockfile record anyway.

§5 declaration — src/update.ts touched

The change touches src/update.ts (the installViaTarball success hook, inside the existing desktop-only gate). Per §5 this requires a NO-OP validation release FIRST to prove the existing upgrade path is healthy end-to-end before this ships. That release needs your go — approve in-thread and I'll cut it (or fold it into the release that carries this PR).

Behavior changes (old → new)

# Change Old New Why
1 Desktop post-self-heal state manifest synced to new version, pnpm-lock record deliberately left naming the OLD version ("host's pnpm bookkeeping still names the OLD version") all three ledgers synced to the installed version in one transaction the stale record was the wedge producer (#3160 class); #3224 decision-A's permanent-noise rationale retires going forward — the detection skip for desktop stays correct either way
2 Self-heal log line always info ... manifest spec now declares X silent when nothing changed; info lists each before→after when something changed; warn + bili doctor --fix hint when convergence failed less noise on the healthy path (the common case); failures become actionable
3 npm package-lock.json convergence n/a when the recorded version moves, the stale integrity hash is DROPPED alongside the version/resolved rewrite keeping the old SHA against the new version would hard-fail npm ci; dropping it lets npm re-fetch and verify
4 Tilde / non-authority specs silently untouched by the #2894 sync REFUSED by converge with an explicit "rewrite authority" reason (zero writes); the finding stays visible writing a lockfile under a spec that cannot admit the copy would manufacture a wedge wearing a green badge; respect stays, silence becomes explanation
5 bili doctor strictly read-only read-only audit unchanged; --fix is the sole write path — explicit trigger, prints every change, re-verifies issue requirement 2 + boundary "修复动作不得静默写用户没授权的文件", honored by construction

Acceptance mapping (issue #3496)

@github-actions

github-actions Bot commented Oct 17, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-10-17_dsh-ledger-convergence (bc1b3c8)

Option A — Install this exact build (recommended)

npm install -g billion-context@0.0.4224-pr.3509.b154279

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Option B — Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr3509.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 (ai: vllm-qwen/qwen3.8-27b)

查重:不重复。 本 PR 是 #3496 的唯一实现 PR(正文 Closes #3496),与在途的 #3231(provenance 检测半边)互补不重叠——检测归 #3231、修复/收敛归本 PR;近期 open/closed issue 与 PR 中无同签名(manifest+lockfile+副本三账收敛 / doctor --fix)的并行工作项。#3160/#3164/#3224 均为背景引用,非重复立项。

开始独立审阅(PR Review 规范全套:方向关→回归关→代码关→测试关→diff 整洁度)。已注意分支 base 94e0fe04a 落后当前 master 19 个提交,会先 rebase 到最新 master 再下结论。完整八问结论随后回复。

@ranxianglei
ranxianglei force-pushed the 2026-10-17_dsh-ledger-convergence branch from a74a7e5 to 9e8e331 Compare October 17, 2026 09:29
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

Review 结论(独立重审:原分支落后 master 19 个提交,已 rebase 到当前 master 3f513c234——那 19 个提交没有一个触碰本 PR 修改的文件,rebase 零冲突;随后全量读 diff、逐项核对既有行为清单。新 head:d1248ad48)

八问:

  1. 是什么问题:desktop profile 的自更新(in-place 覆盖 node_modules)之后,只有 manifest 的 spec 被同步,lockfile 记录仍指向旧版本——三个账本(manifest / lockfile / 磁盘副本)永久漂移,用户下次 pnpm install 会 404 或静默回退(Profile 副本自愈可被本地高版本号永久卡死,且 out-of-band 副本无来源校验(#1196 变体) #3160/[Detection] No signal when dsh profile manifest/lockfile/on-disk copy are in three-way conflict (follow-up to #3160, #3169 closed without merge) | [Detection] dsh profile 的 manifest/lockfile/盘上副本三方矛盾时无任何信号(#3160 后续,#3169 已关闭未合并) #3224 类)。
  2. 什么原因导致的:缺陷的生产者就是这个 writer——installViaTarball 的成功钩子原来只调 syncDshProfileManifestSpec(只同步 manifest,lockfile 故意留旧版本);手工编辑 manifest(Profile 副本自愈可被本地高版本号永久卡死,且 out-of-band 副本无来源校验(#1196 变体) #3160)是第二个来源。本 PR 在生产者层面修:同一次操作内收敛所有可写账本(根治)+ 显式 bili doctor --fix 修存量漂移(补修)+ 检测留给 feat: detect manifest/lockfile/copy three-way disagreement in dsh profiles (#3224) #3231(最后防线)。方向核实:改的是根因,不是症状。
  3. 影响范围与严重程度:仅 dsh desktop lane(in-place 自更新路径有 isDshDesktopBiliCopy 门控);全局安装、reference lane、其他 host world 不受影响。修复前:每次自更新确定性产生 lockfile 漂移(功能性退化,下次 install 404/静默回退);修复后:不再产生,存量可 --fix 修复。
  4. 怎么修复的:新模块 src/dsh-ledger.ts,convergeDshLedger 先规划全部变更(all-or-nothing planning,任一步失败则零写入),再原子写(tmp+fsync+rename);覆盖 pnpm lockfile 三种形态(v6+/v9 nested、≤5.4 flat、v6–v8 top-level)与 npm package-lock fallback;重写权限严格沿用 插件自动更新后可能被dsh退回 #2894 规则(只动 ^x.y.z/精确版本);tilde/dist-tag/range spec 在不承认副本版本时拒绝并说明原因(零写入)。自更新成功钩子改调它(desktop gate 不变);doctor --fix 遍历含 desktop 的全部 profile、向磁盘副本版本收敛、打印 before→after、事后复审、残留时非零退出。
  5. 是否有回归问题:无。逐项审过的既有行为:① syncDshProfileManifestSpec 契约逐字节保持(skip reason 字符串、序列化格式、原子写纪律、错误形状均不变,拆成纯规划半部+薄包装,既有 20 个测试含全部 插件自动更新后可能被dsh退回 #2894 用例全绿);② 自更新钩子仍在原 desktop gate 内,全局自更新与 advisory 调用点未动;③ plain bili doctor 输出路径逐字节不变,退出码语义不变(--json 仅在 --fix 下多一个可选字段);④ --fix 解析位置与既有 --json 等全局 flag 惯例一致;⑤ 完全不触碰请求管线/wire 字节,两种压缩模式均不受影响。有意变更(均在 PR 表格披露,我逐条对 diff 复核过):日志行 always-info → 静默/info/warn;npm lock 的 stale integrity 随版本移动被丢弃(保留会让 npm ci 硬失败);tilde 类 spec 从「静默不动」变为「显式拒绝并提示」(零写入,finding 仍可见)。
  6. 是否破坏缓存:不涉及——不触碰 wire/prefix cache/重试/游标/去重键,只移动磁盘上的账本文件。
  7. 是否改了设计方向:无转向。把 Install-Lane Contract 第 6 条(in-place writer 必须三账本齐步走)从隐含期望变成强制代码,AGENTS.md 补记为规范表述;单 writer 原则(native模式下自动更新问题需要解决 #991/dsh plugin-market installs never auto-update: the refresh chain's only trigger is a global bili self-update #1196)未变。
  8. 是否引入了新的配置:一个 CLI flag bili doctor --fix(issue 正文里 owner 提议的形状,视为已签核;README en/zh 已记档);无新 env var、无 config 文件字段;--json 输出在 --fix 下新增可选 ledgerFixes 数组。

Review 中发现的 3 个问题已直接在分支上修掉(独立 commit d1248ad48,标注 fix(#3496 review)):

  • yamlScalar 裸标量正则允许以 * 开头(YAML alias 指示符)——手写的 * 类 spec 会被裸写入 pnpm-lock.yaml 并被解析成别名 token,直接坏掉 lockfile;现改为 JSON 引号包裹(与 pnpm 自身 writer 一致),补了回归测试;
  • PR 描述称健康 profile 的零写入断言「content+mtime」,实际只断言了 content——已让全部 6 处调用点真正断言 mtime;
  • --fix flag 没有解析测试——已在 cli-parseargs 补默认关/开启两条断言。

验证证据:本地 typecheck+build 绿;定向套件 105/105(修复前,9 个文件)+ 62/62(修复后,4 个文件);远程 CI 在 d1248ad48 全绿(42/42 check runs,含 ubuntu/windows × node 22/24 分片)。

合并判定:代码就绪,但按仓库规则还有两道人工闸门:

  1. §7.4 —— 触碰 src/update.ts,无论测试多绿都必须人工合并;
  2. §5 —— 改动涉及 updater 自身,需先发一个 NO-OP 验证 release(纯版本号 bump)证明现有升级路径端到端健康,且发版需要你明确批准。你点头后我先切 no-op release PR,合入后再合本 PR。

后续项(不扩本 PR 范围):本 PR 与 #3231 都合入后,把 provenance.ts 与 dsh-ledger.ts 之间重复的 lockfile reader 收敛为一份,届时单独立项跟踪(PR 正文已记)。

@ranxianglei 等你拍板发版顺序(no-op release 先行 → 再合本 PR)。

@ranxianglei

Copy link
Copy Markdown
Owner Author

ai: zhipuai-lb/glm-5.3

Live end-to-end verification (real proxy driving the desktop-lane in-place update cycle against a local static registry, three pnpm major versions, plus bili doctor --fix and a real dsh plugin add/remove cycle). All local — zero tokens.

Matrix (all with the PR build as both the old and new copy):

scenario result
in-place self-heal 0.1.199-test.0 → test.2 (desktop profile, pnpm 9.15.9 / 11.7.0 / 12.4.2 layouts, exact-pin and caret styles) ✅ one transaction converges all three ledgers — log: dsh desktop profile ledgers converged to 0.1.199-test.2: package.json … → …; pnpm-lock.yaml specifier … → …; pnpm-lock.yaml version … → …
next pnpm install after converge — pnpm 11.7.0 (desktop runtime) ✅ heals: downloaded 1, added 1, store entry materialized, working symlink, lock packages/snapshots completed
next pnpm install after converge — pnpm 9.15.9 (old CLI) ✅ heals identically
next pnpm install after converge — pnpm 12.4.2 ⚠️ edge: importers-only patch + pnpm 12's "resolution step is skipped" → it swaps the real dir for a symlink to a store path that was never fetched (broken link), then reports Already up to date on every later run — including pnpm add billion-context@<same version>. No self-heal.
bili doctor --fix on a drifted profile (lock+manifest rewound to old, copy new) ✅ converges, prints every before→after + the #3160/#3164 provenance note
normal-lane uninstall/reinstall with real dsh plugin --profile web remove/add ✅ clean — remove wipes all three ledgers, reinstall re-aligns; bili side stays zero-write
PR test files locally ✅ 41/41 (issue3496 + update-desktop-inplace + cli-parseargs)

Assessment of the pnpm ≥12 edge: dsh's own runtimes embed pnpm 9/11 (op logs: using pnpm v11.7.0 desktop, v9.15.9 old CLI), and both heal the importers-only lockfile correctly. pnpm ≥12 only enters the picture when a user manually runs their own global pnpm inside a dsh profile (I reproduced it with host pnpm 12.4.2). So: not a merge blocker for the populations dsh actually ships, but worth a follow-up.

Suggested follow-up (not blocking): converge could also inject the packages:/snapshots: entries for the new version — during an in-place install bili already holds the registry doc's integrity and the tarball's own package.json (deps: zod@4.1.8, optional: jimp), so a complete lock entry is fabricable without a pnpm spawn. That would make even headless pnpm ≥12 fetch-and-materialize instead of dangling.

Reproduction artifacts kept locally (~/tmp/t3496-verify/ + /tmp/t3496/): static registry registry.mjs, three DSH_HOMEs, proxy logs.

Verdict: verified, recommend merge — the #3160/#3496 failure mode (update leaves ledgers diverged; next install silently reverts or, worst case, breaks) is closed on every pnpm version dsh actually drives.

awork added 3 commits October 17, 2026 20:34
…li doctor --fix (#3496)

The dsh desktop lane is the one copy bili overwrites in place (#1575/#2894).
Its self-heal previously synced only the manifest spec and deliberately left
the lockfile record naming the old version — manufacturing exactly the
wedge class #3224 detects and #3160 bit users on (stale lock record /
unresolvable manifest spec -> next pnpm install 404s or reverts the update).

- src/dsh-ledger.ts (new): ledger readers (pnpm nested v6+/v9, flat <=5.4,
  v6-v8 top-level; npm package-lock.json fallback), #3231's three-rung
  findings semantics ported verbatim, and convergeDshLedger() — plan everything
  then write atomically (tmp+fsync+rename), healthy profiles get zero writes.
- src/update.ts: the installViaTarball success hook (desktop-gated) now calls
  convergeDshLedger instead of bare manifest sync — all three ledgers move in
  one transaction, local pins respected, failure logs but never fails install.
- src/dsh-channel.ts: syncDshProfileManifestSpec split into a pure planner +
  atomic writer; the wrapper keeps byte-identical behavior/reasons.
- bili doctor --fix: explicit repair pass over every dsh profile (desktop
  included — detection excludes it by design, repair must reach it), converges
  to the on-disk copy version, prints every before->after change, re-audits
  for residuals, non-zero exit when anything remains.
- AGENTS.md: Install-Lane contract item 6 — an in-place writer keeps all
  three ledgers in step.

Tests: tests/issue3496-dsh-ledger-convergence.test.ts (23, fixture matrix
incl. the #3160 ^1.2.9 shape, healthy zero-writes, refusal cases) + one
integration test in tests/update-desktop-inplace.test.ts.
… pin zero-writes by mtime; cover doctor --fix flag parse
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

收到实机端到端验证矩阵——和我这边的独立审核结论互相印证(#3160/#3496 失败模式在 dsh 实际驱动的所有 pnpm 版本上已闭环),同意「verified, recommend merge」的判定。两件事已处理:

1. pnpm ≥12 边缘已单独立项跟踪:#3570
按你「not blocking」的定位没有塞进本 PR。issue 里记了完整症状(用户自己跑全局 pnpm ≥12 时 importers-only 收敛后的 lockfile 被跳过 resolution、换成指向未拉取 store 路径的断链、此后永久 Already up to date)、影响面(仅手动场景,dsh 自带 pnpm 9/11 运行时不受影响)、复现摘要和你提出的修法方向(converge 顺带注入 packages:/snapshots: 条目,bili 手里已有 integrity + tarball package.json,无需 spawn pnpm)。标注了它要改变「store/snapshots 段不动」的现有不变量,#3509 合入后做。查重:无。

2. 分支又往前挪了一步:我审核期间 master 合入了 v0.1.198 release(#3534)等 15 个提交,与本 PR 零文件重叠,已再次 rebase 到当前 master。新 head:bc1b3c814,CI 正在重跑,绿了我再回一条确认。

等你拍板的一件事 — no-op 验证 release(§5 流程闸门):本 PR 改了 src/update.ts(updater 自身),按仓库规则必须先发一个纯版本号 bump 的 release,证明现有升级路径端到端健康,之后才能合本 PR。建议版本 v0.1.199(当前最新 tag v0.1.198)。

关于这个 release 提议本身:

  1. 是什么问题:非缺陷——§5 流程要求(updater 改动前先验证现有升级链)。
  2. 什么原因导致的:仓库规则,坏 updater 会 brick 后续所有升级;无技术根因。
  3. 影响范围与严重程度:diff 只有 package.json + lockfile 的版本字段,零代码变化;所有用户的下次自动更新检查拿到的是与 0.1.198 完全相同的代码换个号——纯金丝雀,零行为变化。
  4. 怎么修复的:不适用——发版动作,分支 2026-10-17_release-v0.1.199,单个 commit 只改 version。
  5. 是否有回归问题:无——两行版本字段,version-guard/lock-sync 门禁把关。
  6. 是否破坏缓存:不涉及。
  7. 是否改了设计方向:无。
  8. 是否引入了新的配置:无。

你回复「批准发布 v0.1.199」我就切分支开 release PR;它合并发布后,本 PR 即可合并(§7.4 人工合并)。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

CI 确认(rebased head bc1b3c814,基于含 v0.1.198 的当前 master):全部真实门禁绿——主 CI(ubuntu/windows × node 22/24 分片 + static gates)、PR Build Artifact、以及 e2e-local-registry / native-dsh / native-opencode / pi-codex-ws / fake-codex / codex-persona / native-pi / dsh-plugin-lane 七个 e2e lane 全过。

清单里唯一的一个红是 Gate Plugins (manual)(ci-gate-plugins.yml),与代码无关:该工作流是纯手动触发(#3416,只有 workflow_dispatch,无任何 push/PR 触发器)。rebase 把这个文件第一次带进分支树,GitHub 对「push 引入新工作流文件」会生成一个零 job 的幽灵 run 并直接标 failure——它没有跑任何 job、不产生任何 check、不挡合并。本 PR 的 diff 完全不触碰 .github/workflows/(已核对文件清单)。从当前 master 切出的其他分支不受此影响。

状态不变:等你拍板 no-op 验证 release v0.1.199(§5,updater 改动的前置闸门)。你回复「批准发布 v0.1.199」我就切 2026-10-17_release-v0.1.199 开 release PR;它合并发布后,本 PR 即可人工合并。pnpm ≥12 边缘已立项 #3570 跟踪,不阻塞本 PR。

@ranxianglei
ranxianglei merged commit 5bea7ee into master Oct 17, 2026
69 of 77 checks passed
@github-actions

Copy link
Copy Markdown
Contributor

📦 Merged Build Published

This merge to master was published as 0.0.539-master.5bea7ee (rolling master tag).

npm install -g billion-context@0.0.539-master.5bea7ee  # this exact build
npm install -g billion-context@master  # always the newest merged build

npm view billion-context@master prints the current version at any time.

Louie0806 pushed a commit to Louie0806/billion-context that referenced this pull request Nov 12, 2026
…ions and keep it when rewriting

pnpm appends a peer-resolution suffix to an importer's resolved version
when the package or one of its dependencies has a satisfied peer:
`version: 0.1.213(@types/node@16.9.1)`. The ledger code took the whole
scalar for the version. A healthy profile was reported as copy-vs-lock,
and convergence (`bili doctor --fix`, the pass after an in-place update)
rewrote the line to the bare version: a write where nothing had drifted,
leaving the importer without the suffix its snapshots key carries.

- splitPeerSuffix(): split the scalar at the first "(".
- readPnpmLockResolution returns the version without the suffix, so the
  state and the findings carry a plain version.
- patchNestedEntries compares the version part only; when the version
  really moves it swaps that part and keeps the suffix and the quoting
  the line already had.
- tests/issue3496-dsh-ledger-convergence.test.ts: v9 fixture with a peer
  suffix; healthy triple gives no finding and zero writes; real drift
  changes only the two importer lines and keeps the suffix; multi-peer,
  quoted and v6-v8 nested shapes.

packages:/snapshots: stay untouched as before (ranxianglei#3509, ranxianglei#3570).
(cherry picked from commit 9ae09b3)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

dsh 车道自动更新改全:自愈同步三本账(manifest/lockfile/副本)+ 存量漂移显式修复,自动更新后必须还能用

1 participant