Skip to content

[PR] fix(dsh-ledger): move the pnpm store sections with the importer on in-place update - #7137

Open
ranxianglei wants to merge 3 commits into
masterfrom
2026-11-18_dsh-ledger-store-sync
Open

ranxianglei wants to merge 3 commits into
masterfrom
2026-11-18_dsh-ledger-store-sync

Conversation

@ranxianglei

Copy link
Copy Markdown
Owner

Closes #7125

Problem

After billion-context auto-updates in place (e.g. 0.1.215 → 0.1.216) in a DSH desktop profile, the pnpm lockfile's importers record moves to the new version but packages: / snapshots: still carry only the old version. The lockfile is internally inconsistent — the importer points at a version whose resolution data is absent.

Consequences:

The pre-existing peer-suffix fix (#6314) keeps packages:/snapshots: untouched; this report covers the additional failure path through the bundled pnpm 11 remove command.

Change

patchPnpmLock now syncs the store sections to the new version when the importer record moves (opt-in via a new opts argument; the 3-arg call keeps the old behavior):

  • A present entry is left alone (zero writes).
  • A missing entry is cloned from the closest existing billion-context entry — same peer suffix first, then nearest major.minor.patch.
  • The clone's integrity is the new version's SRI hash, sourced from the registry doc. The source's hash is never carried to the new key (a wrong hash makes pnpm hard-fail verification). When the source carries an integrity and the new one is unknown, the patch fails soft with zero writes rather than writing a broken lockfile.
  • The clone's dep map is cross-checked against the on-disk copy's declaration: declared deps missing from the map are resolved from the store (exactly one candidate — ambiguity fails soft); names no longer declared are dropped. A store entry that has no dep map at all (v9 packages:) does not grow one.

Call sites:

  • convergeDshLedger(profileDir, targetVersion, { integrity }) — the self-heal hook in src/update.ts passes the tarball's integrity; the pnpm path now carries { integrity, deps } into patchPnpmLock.
  • doctor --fix (fixDshLedgers) sources the SRI hash from the registry doc — one lightweight GET per distinct drifted pnpm profile; a fetch failure makes the converge fail soft with zero writes (strictly better than a broken lockfile write).

Detection: a new lockfile-internal finding kind makes the broken state visible to bili doctor (the three-rung audit — importer, copy, manifest — all agree, so the inconsistency was previously invisible).

Behavior change (disclosure)

  • New: on an in-place pnpm update, the store sections (packages:/snapshots:) now move with the importer. Old behavior: importer moved, store sections left at the old version (broken lockfile). New behavior: store sections carry the new version's entries (cloned + integrity + dep-map cross-check).
  • New: a version move that cannot be completed safely (new integrity unknown while the old entry carries one) now fails soft with zero writes instead of writing a broken lockfile.
  • New: bili doctor reports a lockfile-internal finding for the broken state.
  • New: doctor --fix performs one registry-doc GET per distinct drifted pnpm profile (to source the SRI hash).

Coordination

This PR and #7133 both touch src/dsh-ledger.ts and tests/issue3496-dsh-ledger-convergence.test.ts. #7133 changes the specifiers (top-level dependency maps) branch of patchPnpmLock; this PR changes the importers (v9+) and v6 branches. No overlap in the source change; the test additions are in different regions. Whichever merges first, the other may need a rebase.

Validation

@ranxianglei ranxianglei added bug Something isn't working dsh p1 scope/storage labels Nov 18, 2026
@github-actions

github-actions Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

📦 Built Package Artifact

Branch: 2026-11-18_dsh-ledger-store-sync (7ea6077)

Latest build — install this exact version

npm install -g billion-context@0.0.10880-pr.7137.fea5662

This exact version is immutable and reproducible. The rolling pr tag tracks the newest test build across all PRs.

Recent builds (commit → version)

commit version build
7ea6077 billion-context@0.0.10880-pr.7137.fea5662 build
bfc2391 billion-context@0.0.10794-pr.7137.ab75956 build
9c63132 billion-context@0.0.10784-pr.7137.5e9f421 build
9d4a85c billion-context@0.0.10782-pr.7137.2b07a84 build

Download artifact

  1. Download the artifact from the Actions run
  2. Extract the tarball and install:
tar xzf billion-context-pr7137.tgz
npm install -g package

Install instructions are refreshed on each push.

@ranxianglei ranxianglei added p2 and removed p1 labels Nov 18, 2026
@ranxianglei ranxianglei changed the title fix(dsh-ledger): move the pnpm store sections with the importer on in-place update [PR] fix(dsh-ledger): move the pnpm store sections with the importer on in-place update Nov 18, 2026
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷

(ai: vllm-qwen/qwen3.8-27b)

查重: 不重复 — this is the PR mirror for #7125 (dsh-ledger store-section sync); no other open thread targets this fix.

标签: bug, dsh, p2, scope/storage (applied to the mirror; the PR already carried the same set).

Status (verified live): PR #7137 open, mergeable (no conflicts), but CI was red on the unused-exports static gate — the new PnpmLockPatchOpts type was exported from src/dsh-ledger.ts yet never imported anywhere (all consumers — patchPnpmLock and the four internal helpers — live in the same file). The test lanes were skipped/cancelled behind that gate, so no full-suite verdict had run yet.

Fix (pushed to the PR branch, commit 9c63132591):

  • src/dsh-ledger.ts:440 — dropped the export from interface PnpmLockPatchOpts, the gate's own suggested remedy for INTERNAL_ONLY (no waiver entry needed).
  • Verified locally: node scripts/check-unused-exports.mjs clean; npm run typecheck clean; targeted suites tests/issue3496-dsh-ledger-convergence.test.ts + tests/doctor.test.ts = 52/52 pass. Full-suite verdict is left to remote CI on the new head.

Waiting on: CI re-run on 9c63132591 (test lanes unblocked once the static gates pass).

八问:

  1. 是什么问题:PR 本身无缺陷;CI 唯一红灯是 unused-exports 静态门禁——新增的 PnpmLockPatchOpts 类型被导出但无人 import。
  2. 什么原因导致的:PR 新增该类型时标了 export,而全部使用点(patchPnpmLock 及内部 helper)都在同文件内,没有跨模块消费者。
  3. 影响范围与严重程度:仅 CI——test 车道被门禁挡住跳过,无运行时影响。
  4. 怎么修复的:去掉 export 关键字(门禁对 INTERNAL_ONLY 的推荐修法);不加 waiver,类型结构与导出函数签名均不变(导出函数签名引用模块内类型在 TS 下合法,本项目不生成 dts)。
  5. 是否有回归问题:无——单行可见性变更,typecheck + 52 个定向测试全绿;importer/copy/manifest 三档审计与 store 同步逻辑未触碰。
  6. 是否破坏缓存:不涉及。
  7. 是否改了设计方向:不涉及。
  8. 是否引入了新的配置:不涉及。

结论:PR 代码就绪,等 CI 在新 head 上转绿后即可由人合并(合并权在人)。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷

(ai: vllm-qwen/qwen3.8-27b)

CI now green on 9c63132591 (verified live): all 50 check runs completed — 37 success, 13 skipped (gated lanes), 0 failure, 0 cancelled. The unused-exports gate passes and the full test matrix (ubuntu/windows × node 22/24 × shards) ran clean.

PR state: open, mergeable (no conflicts); mergeable_state: blocked is branch-protection review gating, not CI. Ready for human merge whenever you are: #7137

@Amzp

Amzp commented Nov 18, 2026 •

Copy link
Copy Markdown

(ai: openai/gpt-6.2-sol)

Independent verification on 9c63132591 with the actual #7125 lockfile and published npm tarballs: the original 0.1.215 → 0.1.216 update and doctor --fix both pass. The target SRI and peer-suffixed snapshot agree with the published release.

I found two related cases worth guarding before this ships:

  1. Dependency range changes without a name/bucket change. DSH’s pnpm 11.28.5 resolves zod to 4.1.8 for billion-context@0.1.209 and to 4.6.7 for 0.1.215. Updating the earlier lock with this head returns ok: true but stores zod: 4.1.8 in the 0.1.215 snapshot, even though the tarball requires exact 4.6.7. I also verified that a frozen install can exit 0 with this transplanted snapshot and actually install the wrong zod. The new SRI verifies the package, not the cloned dependency graph.
  2. Fail-soft lock planning needs to preserve the installed copy too. With a verified legacy SHA-1 tarball but no target SRI, the v9 lock patch correctly refuses to write, but installViaTarball discards the old pnpm link/backup first and still returns ok: true. The copy and the two ledgers then disagree.

A local supplement passes the three real fixtures and 67 scoped tests: it checks snapshot resolutions against the target manifest, declines ambiguous v9 in-place updates before swapping, and keeps the previous copy available until convergence succeeds. On the real 0.1.209 → 0.1.215 case it refuses the transplant with the old link and both ledger files byte-identical. The dependency-changing update itself still needs a real pnpm resolution. Since this touches src/update.ts, the repo’s human-review/no-op-release gate applies.

Original reproduction with more detail: #7125.

Tested candidate supplement

Based on 9c63132591. Local typecheck/build and unused-export check passed; 67 targeted tests and 12 hermetic registry/advisory E2E tests passed. The injected lockfile rename failure runs after the manifest write and verifies the original manifest/lockfile bytes and pnpm link are restored. This is a conservative guard: dependency-changing upgrades still need pnpm resolution.

Applyable diff against the current PR head
diff --git a/src/dsh-ledger.ts b/src/dsh-ledger.ts
index df8f1ce8b885231b8634c73e552c97e00776ca1d..247780d559ff648d3b0b0b373e2b61f758e03c20 100644
--- a/src/dsh-ledger.ts
+++ b/src/dsh-ledger.ts
@@ -22,6 +22,8 @@
 
 import fs from "node:fs";
 import path from "node:path";
+import { isDeepStrictEqual } from "node:util";
+import semver from "semver";
 import { DSH_DESKTOP_PROFILE, DSH_PACKAGE, dshProfileDepSpec, isRegistryDepSpec, planDshProfileManifestSpec } from "./dsh-channel.js";
 
 type LedgerLockKind = "pnpm" | "npm" | "none";
@@ -159,6 +161,43 @@ function readDepMaps(doc: { dependencies?: unknown; optionalDependencies?: unkno
     return out.dependencies !== undefined || out.optionalDependencies !== undefined ? out : undefined;
 }
 
+function isPnpmV9Lock(text: string): boolean {
+    return /^\uFEFF?lockfileVersion:[ \t]*(?:9(?:\.\d+)?|'9(?:\.\d+)?'|"9(?:\.\d+)?")[ \t\r]*(?:#.*)?$/m.test(text);
+}
+
+export function preflightDshPnpmV9Update(
+    profileDir: string, installDir: string, stagingDir: string, integrity: string | undefined,
+): { kind: "skip" | "ready" } | { kind: "fail"; reason: string } {
+    let lock: string;
+    try {
+        lock = fs.readFileSync(path.join(profileDir, "pnpm-lock.yaml"), "utf8");
+    } catch (error) {
+        return (error as NodeJS.ErrnoException).code === "ENOENT"
+            ? { kind: "skip" }
+            : { kind: "fail", reason: "unable to read the desktop pnpm lockfile" };
+    }
+    if (!isPnpmV9Lock(lock)) return { kind: "skip" };
+    const spec = dshProfileDepSpec(profileDir);
+    if (spec === undefined || !isRegistryDepSpec(spec)) {
+        return { kind: "fail", reason: "the desktop profile must declare a registry dependency to update in place" };
+    }
+    if (!integrity) return { kind: "fail", reason: "a pnpm v9 update requires the target registry integrity" };
+    const readManifest = (dir: string) => readJsonFile(path.join(dir, "package.json")) as Record<string, unknown> | undefined;
+    const current = readManifest(installDir);
+    const staged = readManifest(stagingDir);
+    if (!current || !staged || typeof current.version !== "string" || typeof staged.version !== "string") {
+        return { kind: "fail", reason: "unable to compare the desktop package dependency declarations" };
+    }
+    for (const field of ["dependencies", "optionalDependencies", "peerDependencies", "peerDependenciesMeta", "bundleDependencies", "bundledDependencies"]) {
+        if (!isDeepStrictEqual(current[field] ?? {}, staged[field] ?? {})) {
+            return { kind: "fail", reason: "the package dependency declarations changed; update through the DSH plugin manager" };
+        }
+    }
+    const patch = patchPnpmLock(lock, spec, staged.version, { integrity, deps: readDepMaps(staged) });
+    if (patch.kind === "fail") return patch;
+    return { kind: "ready" };
+}
+
 interface LockResolution {
     specifier?: string;
     version?: string;
@@ -596,11 +635,12 @@ function resolveDepFromStore(lines: string[], section: "packages" | "snapshots",
  *  ambiguity fails soft); names no longer declared are dropped. */
 function crossCheckDepMaps(lines: string[], block: string[], section: "packages" | "snapshots", deps: PnpmLockPatchOpts["deps"]): { kind: "ok"; lines: string[] } | { kind: "fail"; reason: string } {
     if (deps === undefined) return { kind: "ok", lines: block };
-    const declared: Record<string, "dependencies" | "optionalDependencies"> = {};
-    for (const name of Object.keys(deps.dependencies ?? {})) declared[name] = "dependencies";
-    for (const name of Object.keys(deps.optionalDependencies ?? {})) declared[name] = "optionalDependencies";
-    const headerIdx: Record<string, number> = {};
-    const entryIdx: Record<string, Record<string, number>> = {};
+    const declared = new Map<string, { bucket: string; spec: string }>();
+    for (const bucket of ["dependencies", "optionalDependencies"] as const) {
+        for (const [name, spec] of Object.entries(deps[bucket] ?? {})) declared.set(name, { bucket, spec });
+    }
+    const matchesSpec = (resolved: string, spec: string): boolean => semver.satisfies(splitPeerSuffix(unquote(resolved))[0], spec);
+    const entries: { name: string; bucket: string; idx: number; version: string }[] = [];
     let current: string | undefined;
     let headerIndent = 0;
     for (let i = 0; i < block.length; i++) {
@@ -611,36 +651,48 @@ function crossCheckDepMaps(lines: string[], block: string[], section: "packages"
         if (hm !== null) {
             current = hm[2];
             headerIndent = hm[1].length;
-            headerIdx[current] = i;
-            entryIdx[current] = entryIdx[current] ?? {};
             continue;
         }
         if (current === undefined) continue;
         if (ind <= headerIndent) { current = undefined; continue; }
-        const em = l.match(/^([ \t]+)([^\s:][^:]*):[ \t]*\S.*$/);
-        if (em !== null) entryIdx[current][em[2]] = i;
+        const em = l.match(/^([ \t]+)([^\s:][^:]*):[ \t]*(\S.*)$/);
+        if (em !== null) {
+            const name = unquote(em[2]);
+            entries.push({ name, bucket: current, idx: i, version: em[3].trim() });
+        }
     }
     const out = block.slice();
-    const surviving: Record<string, string> = {};
+    const surviving = new Set<string>();
     const drop: number[] = [];
-    for (const bucket of Object.keys(headerIdx)) {
-        for (const [name, idx] of Object.entries(entryIdx[bucket]!)) {
-            if (declared[name] === undefined || declared[name] !== bucket) drop.push(idx);
-            else surviving[name] = bucket;
+    for (const { name, bucket, idx, version } of entries) {
+        const target = declared.get(name);
+        if (target === undefined || target.bucket !== bucket) {
+            drop.push(idx);
+            continue;
+        }
+        if (section === "snapshots" && !matchesSpec(version, target.spec)) {
+            return { kind: "fail", reason: 'the pnpm snapshot resolution for ' + name + ' does not satisfy ' + target.spec };
         }
+        surviving.add(name);
     }
     for (const idx of drop.sort((a, b) => b - a)) out.splice(idx, 1);
-    for (const [name, bucket] of Object.entries(declared)) {
-        if (surviving[name] !== undefined) continue;
+    for (const [name, { bucket, spec }] of declared) {
+        if (surviving.has(name)) continue;
         // Only fill a bucket the entry already carries — a store entry that
         // has no dep map at all (v9 packages:) must not grow one; a dep a
         // missing bucket would hold is left to pnpm's install-time repair.
         const hIdx = out.findIndex((l) => new RegExp(`^[ \\t]+${bucket}:[ \\t]*$`).test(l));
-        if (hIdx === -1) continue;
+        if (hIdx === -1) {
+            if (section === "snapshots") return { kind: "fail", reason: "the pnpm snapshot is missing a dependency group for " + name };
+            continue;
+        }
         const value = resolveDepFromStore(lines, section, name);
         if (value === undefined) {
             return { kind: "fail", reason: `cannot resolve '${name}' for the cloned store entry — the lockfile's ${section} section has no unique entry for it; let pnpm repair the lockfile (install --fix-lockfile) and rerun` };
         }
+        if (section === "snapshots" && !matchesSpec(value, spec)) {
+            return { kind: "fail", reason: "the pnpm snapshot resolution for " + name + " does not satisfy " + spec };
+        }
         const hInd = out[hIdx]!.length - out[hIdx]!.trimStart().length;
         out.splice(hIdx + 1, 0, `${" ".repeat(hInd + 2)}${name}: ${value}`);
     }
@@ -691,6 +743,9 @@ function syncStoreSection(lines: string[], section: "packages" | "snapshots", ta
     }
     let newLines: string[];
     if (source.inline !== undefined) {
+        if (section === "snapshots" && (Object.keys(opts?.deps?.dependencies ?? {}).length || Object.keys(opts?.deps?.optionalDependencies ?? {}).length)) {
+            return { kind: "fail", reason: "an empty pnpm snapshot cannot resolve the target package dependencies" };
+        }
         // An inline source (`key: {}`) has no children to rewrite — clone it
         // verbatim under the new key.
         newLines = [`  ${targetKey}: ${source.inline}`];
@@ -907,6 +962,7 @@ export function convergeDshLedger(profileDir: string, targetVersion: string, opt
     // Lockfile half — planned in memory before any disk touch.
     let lockNext: string | undefined;
     let lockDetails: LedgerChangeDetail[] = [];
+    let restoreManifestOnFailure = false;
     if (state.lockKind === "none") {
         notes.push("no lockfile present — manifest/copy agreement is the whole story");
     } else {
@@ -927,14 +983,24 @@ export function convergeDshLedger(profileDir: string, targetVersion: string, opt
         if (patch.details.length > 0) {
             lockNext = patch.next;
             lockDetails = patch.details;
+            restoreManifestOnFailure = state.lockKind === "pnpm" && isPnpmV9Lock(raw);
         }
     }
 
     // Execute — both plans validated; each write is atomic on its own.
     const changed: string[] = [];
     const details: LedgerChangeDetail[] = [];
+    const manifestFile = path.join(profileDir, "package.json");
+    let manifestBefore: string | undefined;
     if (plan.action === "rewrite") {
-        const err = atomicWriteText(path.join(profileDir, "package.json"), plan.nextRaw ?? "");
+        if (restoreManifestOnFailure) {
+            try {
+                manifestBefore = fs.readFileSync(manifestFile, "utf8");
+            } catch {
+                return fail("unable to retain the desktop profile manifest before writing the ledger");
+            }
+        }
+        const err = atomicWriteText(manifestFile, plan.nextRaw ?? "");
         if (err !== undefined) return { changed, details, notes, error: `manifest write failed: ${err}` };
         changed.push("package.json");
         details.push({ file: "package.json", field: "dependencies[billion-context]", before: spec, after: finalSpec });
@@ -942,6 +1008,11 @@ export function convergeDshLedger(profileDir: string, targetVersion: string, opt
     if (lockNext !== undefined) {
         const lockFile = state.lockKind === "pnpm" ? "pnpm-lock.yaml" : "package-lock.json";
         const err = atomicWriteText(path.join(profileDir, lockFile), lockNext);
+        if (err !== undefined && manifestBefore !== undefined) {
+            const rollbackError = atomicWriteText(manifestFile, manifestBefore);
+            if (rollbackError === undefined) return fail("lockfile write failed (restored previous manifest): " + err);
+            return { changed, details, notes, error: "lockfile write failed: " + err + "; manifest rollback failed: " + rollbackError };
+        }
         if (err !== undefined) return { changed, details, notes, error: `lockfile write failed: ${err}${changed.length > 0 ? ` (package.json already updated — rerun to finish)` : ""}` };
         changed.push(lockFile);
         details.push(...lockDetails);
diff --git a/src/update.ts b/src/update.ts
index c3f1f0332a9dc46bd2ce4eecfc8714e7189258de..4703397a12cbc8a109541586afc0e8db3904da10 100644
--- a/src/update.ts
+++ b/src/update.ts
@@ -32,7 +32,7 @@ import { safeSuffix } from "./text-safe.js";
 import { cacheDir } from "./paths.js";
 import { log as loggerLog, type Logger } from "./logger.js";
 import { refreshDshProfileBundles, isDshProfileCopy, dshProfileDirs, dshProfileDependsOnBili, dshProfileDepSpec, isRegistryDepSpec, DSH_PACKAGE, DSH_DESKTOP_PROFILE } from "./dsh-channel.js";
-import { convergeDshLedger } from "./dsh-ledger.js";
+import { convergeDshLedger, preflightDshPnpmV9Update } from "./dsh-ledger.js";
 import { isPiNpmCopy, piNpmEntrySpec, runPiAsync, PI_NPM_SPEC } from "./pi-channel.js";
 import { resolveDshHome, resolveKimiHome, resolveOmpHome, resolvePiHome } from "./client-config.js";
 import { proxyDispatcher } from "./upstream-proxy.js";
@@ -2044,6 +2044,15 @@ export async function installViaTarball(
         await rm(tmpFile, { force: true });
     }
 
+    const desktopProfile = isDshDesktopBiliCopy(installDir, real, env)
+        ? path.join(resolveDshHome(env), "profiles", DSH_DESKTOP_PROFILE)
+        : undefined;
+    const ledger = desktopProfile === undefined ? undefined : preflightDshPnpmV9Update(desktopProfile, installDir, stagingDir, integrity);
+    if (ledger?.kind === "fail") {
+        await rm(workDir, { recursive: true, force: true });
+        return { ok: false, error: ledger.reason };
+    }
+
     // pnpm virtual-store copy (#1575 desktop lane): the flat node_modules
     // entry is a directory symlink/junction into <profile>/.pnpm/. fs.cp would
     // FOLLOW the link and rewrite shared store content (hardlinked,
@@ -2172,6 +2181,22 @@ export async function installViaTarball(
         }
     }
 
+    if (desktopProfile !== undefined) {
+        const conv = convergeDshLedger(desktopProfile, version, { integrity });
+        if (conv.error !== undefined) {
+            if (ledger?.kind === "ready") {
+                const rollbackError = await restoreFromBackup();
+                if (rollbackError === null) await rm(workDir, { recursive: true, force: true });
+                return { ok: false, error: rollbackError === null
+                    ? "dsh desktop pnpm ledger commit failed (restored previous copy): " + conv.error
+                    : "dsh desktop pnpm ledger commit failed: " + conv.error + "; " + rollbackError };
+            }
+            loggerLog("warn", `[update] dsh ${DSH_DESKTOP_PROFILE} profile copy replaced at ${version} but its ledgers were not converged (${conv.error}) — dsh may revert the copy on its next plugin install; rerun 'bili doctor --fix'`);
+        } else if (conv.changed.length > 0) {
+            loggerLog("info", `[update] dsh ${DSH_DESKTOP_PROFILE} profile ledgers converged to ${version}: ${conv.details.map((d) => `${d.file} ${d.field} ${d.before} → ${d.after}`).join("; ")}`);
+        }
+    }
+
     // Success: the backup is no longer needed, and neither is the displaced
     // pnpm link (the store copy underneath it was left byte-identical).
     if (!pnpmOldLink) await rm(backupDir, { recursive: true, force: true });
@@ -2184,26 +2209,6 @@ export async function installViaTarball(
     }
     await rm(workDir, { recursive: true, force: true });
 
-    // #2894/#3496: the dsh desktop lane is the only one bili overwrites in
-    // place while the host's pnpm bookkeeping lags behind. Converting the copy
-    // WITHOUT converting the ledgers leaves a latent wedge: dsh's in-app
-    // plugin manager reconciles node_modules against the stale records on the
-    // user's next "install" click (silently reverting the update), or a
-    // hand-edited manifest (#3160) makes the next `pnpm install` 404. So every
-    // in-place write converges ALL writable ledgers — manifest spec AND
-    // lockfile record — to the version just installed, as one transaction.
-    // Local pins stay respected; a ledger-sync failure logs but never fails
-    // the install (the copy itself is verified and standing).
-    if (isDshDesktopBiliCopy(installDir, real, env)) {
-        const profileDir = path.join(resolveDshHome(env), "profiles", DSH_DESKTOP_PROFILE);
-        const conv = convergeDshLedger(profileDir, version, { integrity });
-        if (conv.error !== undefined) {
-            loggerLog("warn", `[update] dsh ${DSH_DESKTOP_PROFILE} profile copy replaced at ${version} but its ledgers were not converged (${conv.error}) — dsh may revert the copy on its next plugin install; rerun 'bili doctor --fix'`);
-        } else if (conv.changed.length > 0) {
-            loggerLog("info", `[update] dsh ${DSH_DESKTOP_PROFILE} profile ledgers converged to ${version}: ${conv.details.map((d) => `${d.file} ${d.field} ${d.before} → ${d.after}`).join("; ")}`);
-        }
-    }
-
     return { ok: true };
 }
 
diff --git a/tests/issue3496-dsh-ledger-convergence.test.ts b/tests/issue3496-dsh-ledger-convergence.test.ts
index 39a346c82d8f4af14240a972ad9b5621a03a864c..5f8a4a3e33379d213239961de2e29579b949a8d9 100644
--- a/tests/issue3496-dsh-ledger-convergence.test.ts
+++ b/tests/issue3496-dsh-ledger-convergence.test.ts
@@ -774,6 +774,73 @@ test("#7125: dep-map cross-check — declared dep resolved from the store, undec
     }
 });
 
+for (const { title, declaration, acceptsLockedZod } of [
+    { title: "rejects locked zod 4.1.8 for exact 4.6.7", declaration: "4.6.7", acceptsLockedZod: false },
+    { title: "keeps locked zod 4.1.8 for range ^4.1.0", declaration: "^4.1.0", acceptsLockedZod: true },
+]) {
+    test("#7125: v9 snapshot " + title, () => {
+        const root = mkdtempSync(path.join(tmpdir(), "bc-i7125-zod-"));
+        try {
+            const lock = [
+                "lockfileVersion: '9.0'",
+                "",
+                "importers:",
+                "",
+                "  .:",
+                "    dependencies:",
+                "      billion-context:",
+                "        specifier: ^0.1.209",
+                "        version: 0.1.209",
+                "      other-dep:",
+                "        specifier: ^1.2.3",
+                "        version: 1.2.3",
+                "",
+                "packages:",
+                "",
+                "  billion-context@0.1.209:",
+                "    resolution: {integrity: sha512-fake==}",
+                "",
+                "  other-dep@1.2.3:",
+                "    resolution: {integrity: sha512-other==}",
+                "",
+                "  zod@4.1.8:",
+                "    resolution: {integrity: sha512-zod==}",
+                "",
+                "snapshots:",
+                "",
+                "  billion-context@0.1.209:",
+                "    dependencies:",
+                "      zod: 4.1.8",
+                "",
+                "  other-dep@1.2.3: {}",
+                "",
+                "  zod@4.1.8: {}",
+                "",
+            ].join("\n");
+            const dir = makeProfile(root, "zod", {
+                spec: "^0.1.209",
+                copyVersion: "0.1.215",
+                pnpmLock: lock,
+                copyDeps: { dependencies: { zod: declaration } },
+            });
+            const before = profileFiles(dir);
+            const beforeMt = profileMtimes(dir);
+            const res = convergeDshLedger(dir, "0.1.215", { integrity: "sha512-new==" });
+            if (acceptsLockedZod) {
+                assert.equal(res.error, undefined);
+                assert.deepEqual(res.changed, ["package.json", "pnpm-lock.yaml"]);
+                assert.match(readFileSync(path.join(dir, "pnpm-lock.yaml"), "utf8"), /billion-context@0\.1\.215:\n    dependencies:\n      zod: 4\.1\.8\n/);
+            } else {
+                assert.match(res.error ?? "", /\bzod\b/, "convergence must report the unsatisfied zod declaration");
+                assert.deepEqual(res.changed, []);
+                assertZeroWrites(dir, before, beforeMt);
+            }
+        } finally {
+            rmrf(root);
+        }
+    });
+}
+
 test("#7125: dep-map cross-check — an ambiguous store entry fails soft, zero writes", () => {
     const root = mkdtempSync(path.join(tmpdir(), "bc-i7125-d-"));
     try {
diff --git a/tests/update-desktop-inplace.test.ts b/tests/update-desktop-inplace.test.ts
index a169dc32faf0c74958d3c42d2167917faf5fa67d..cefd096c9bc7e8807a3194c64ebdd5dda2fbcb11 100644
--- a/tests/update-desktop-inplace.test.ts
+++ b/tests/update-desktop-inplace.test.ts
@@ -9,7 +9,7 @@
 import { test } from "node:test";
 import assert from "node:assert/strict";
 import crypto from "node:crypto";
-import { mkdtempSync, mkdirSync, writeFileSync, readFileSync, symlinkSync, lstatSync, readdirSync } from "node:fs";
+import fs, { mkdtempSync, mkdirSync, writeFileSync, readFileSync, readlinkSync, symlinkSync, lstatSync, readdirSync } from "node:fs";
 import { tmpdir } from "node:os";
 import path from "node:path";
 import * as tar from "tar";
@@ -28,11 +28,11 @@ function integrityField(buf: Buffer, alg = "sha512"): string {
     return `${alg}-${crypto.createHash(alg).update(buf).digest("base64")}`;
 }
 
-function writePkg(dir: string, version: string): void {
+function writePkg(dir: string, version: string, dependencies?: Record<string, string>): void {
     mkdirSync(path.join(dir, "dist"), { recursive: true });
     writeFileSync(
         path.join(dir, "package.json"),
-        JSON.stringify({ name: "billion-context", version, type: "module", main: "dist/index.js", bin: { bili: "./dist/index.js" } }),
+        JSON.stringify({ name: "billion-context", version, type: "module", main: "dist/index.js", bin: { bili: "./dist/index.js" }, dependencies }),
     );
     writeFileSync(path.join(dir, "dist", "index.js"), `export const loaded = '${version}';\n`);
 }
@@ -108,16 +108,21 @@ test("desktop profile copy: link displaced, real dir laid down, .pnpm store unto
 
 interface DesktopFixture { root: string; dshHome: string; storeCopy: string; flat: string; tgz: Buffer; }
 
-function buildDesktopFixture(root: string, storeVersion: string, tarballVersion: string): DesktopFixture {
+function buildDesktopFixture(
+    root: string,
+    storeVersion: string,
+    tarballVersion: string,
+    deps?: { store: Record<string, string>; tarball: Record<string, string> },
+): DesktopFixture {
     const dshHome = path.join(root, "dsh-home");
     const profileDir = path.join(dshHome, "profiles", "desktop");
     const storeCopy = path.join(profileDir, ".pnpm", `billion-context@${storeVersion}`, "node_modules", "billion-context");
     const flat = path.join(profileDir, "node_modules", "billion-context");
-    writePkg(storeCopy, storeVersion);
+    writePkg(storeCopy, storeVersion, deps?.store);
     mkdirSync(path.join(profileDir, "node_modules"), { recursive: true });
     makePnpmLink(storeCopy, flat);
     const src = path.join(root, "pkg", "package");
-    writePkg(src, tarballVersion);
+    writePkg(src, tarballVersion, deps?.tarball);
     const tgzPath = path.join(root, "pkg.tgz");
     tar.c({ cwd: path.join(root, "pkg"), file: tgzPath, gzip: true, sync: true }, ["package"]);
     return { root, dshHome, storeCopy, flat, tgz: readFileSync(tgzPath) };
@@ -364,6 +369,95 @@ test("refreshDshDesktopCopy converges the lockfile record along with the manifes
     }
 });
 
+for (const { reason, targetZod, hasIntegrity, expectedError, failLockCommit } of [
+    { reason: "declared zod changes from 4.1.8 to 4.6.7", targetZod: "4.6.7", hasIntegrity: true, expectedError: /dependenc/i },
+    { reason: "only a valid legacy SHA-1 shasum is supplied", targetZod: "4.1.8", hasIntegrity: false, expectedError: /integrity|SRI/i },
+    { reason: "lockfile rename fails after the manifest write", targetZod: "4.1.8", hasIntegrity: true, expectedError: /restored previous copy.*restored previous manifest/i, failLockCommit: true },
+]) {
+    test("#7125: installViaTarball preserves the desktop v9 install when " + reason, { timeout: 30_000 }, async (t) => {
+        const root = mkdtempSync(path.join(tmpdir(), "bc-desktop-v9-reject-"));
+        t.after(() => rmrf(root));
+        const fx = buildDesktopFixture(root, "0.1.209", "0.1.215", {
+            store: { zod: "4.1.8" },
+            tarball: { zod: targetZod },
+        });
+        const profileDir = path.join(fx.dshHome, "profiles", "desktop");
+        const manifestPath = path.join(profileDir, "package.json");
+        const lockPath = path.join(profileDir, "pnpm-lock.yaml");
+        const manifestBefore = Buffer.from(JSON.stringify({
+            name: "@dsh/profile-desktop",
+            dependencies: { "billion-context": "^0.1.209" },
+        }, null, 2) + "\n");
+        const lockBefore = Buffer.from([
+            "lockfileVersion: '9.0'",
+            "",
+            "importers:",
+            "",
+            "  .:",
+            "    dependencies:",
+            "      billion-context:",
+            "        specifier: ^0.1.209",
+            "        version: 0.1.209",
+            "",
+            "packages:",
+            "",
+            "  billion-context@0.1.209:",
+            "    resolution: {integrity: sha512-fake==}",
+            "",
+            "  zod@4.1.8:",
+            "    resolution: {integrity: sha512-zod==}",
+            "",
+            "snapshots:",
+            "",
+            "  billion-context@0.1.209:",
+            "    dependencies:",
+            "      zod: 4.1.8",
+            "",
+            "  zod@4.1.8: {}",
+            "",
+        ].join("\n"));
+        writeFileSync(manifestPath, manifestBefore);
+        writeFileSync(lockPath, lockBefore);
+        const linkBefore = readlinkSync(fx.flat);
+        let lockCommitAttempts = 0;
+        if (failLockCommit) {
+            const renameSync = fs.renameSync;
+            t.mock.method(fs, "renameSync", (...args: Parameters<typeof fs.renameSync>) => {
+                if (args[1] === lockPath) {
+                    lockCommitAttempts++;
+                    assert.match(readFileSync(manifestPath, "utf8"), /\^0\.1\.215/);
+                    throw new Error("simulated pnpm lockfile rename failure");
+                }
+                renameSync(...args);
+            });
+        }
+        const tarballUrl = "https://registry.test/bc-0.1.215.tgz";
+        t.mock.method(globalThis, "fetch", (url: unknown) => {
+            assert.equal(String(url), tarballUrl);
+            return Promise.resolve(new Response(new Uint8Array(fx.tgz)));
+        });
+
+        const result = await installViaTarball(
+            "0.1.215",
+            tarballUrl,
+            fx.flat,
+            hasIntegrity ? integrityField(fx.tgz) : undefined,
+            crypto.createHash("sha1").update(fx.tgz).digest("hex"),
+            undefined,
+            { ...process.env, DSH_HOME: fx.dshHome },
+        );
+
+        assert.equal(result.ok, false, "desktop v9 update must refuse to commit with unresolved ledger changes");
+        assert.match(result.error ?? "", expectedError);
+        assert.equal(lockCommitAttempts, failLockCommit ? 1 : 0);
+        assert.equal(await readDiskVersion(fx.flat), "0.1.209");
+        assert.equal(lstatSync(fx.flat).isSymbolicLink(), true);
+        assert.equal(readlinkSync(fx.flat), linkBefore);
+        assert.deepEqual(readFileSync(manifestPath), manifestBefore);
+        assert.deepEqual(readFileSync(lockPath), lockBefore);
+    });
+}
+
 test("syncDshProfileManifestSpec: style-preserving convergence, pins and odd specs respected (#2894)", () => {
     const root = mkdtempSync(path.join(tmpdir(), "bc-desktop-spec-"));
     try {

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

Both findings independently verified — both are real, and both are now fixed in this PR (commit bfc239176a, pushed to the branch).

Claim 1 — dependency range changes without a name/bucket change: confirmed.
crossCheckDepMaps (src/dsh-ledger.ts) only handled name-level changes: declared-but-missing → resolved from the store, undeclared → dropped. Surviving names (declared AND present) kept the source version's resolved version verbatim — no check against the new manifest's declared spec. So 0.1.209 → 0.1.215 (zod 4.1.8 → 4.6.7) clones a snapshot recording zod: 4.1.8, and a frozen install faithfully installs the wrong zod. The new SRI indeed verifies the package tarball, not the cloned dependency graph.

Fix: every surviving entry's resolved version is now checked against the on-disk copy's declared spec via semver.satisfies (new depResolutionAdmits helper). A proven contradiction → fail soft with zero writes; the reason names the dep, the cloned version, and the declared spec, and says to let pnpm record the resolution (install) and rerun. Unjudgeable inputs (non-semver resolved value, or a dist-tag-like spec such as latest) stay silent — the same "err on fewer false positives" discipline as specAdmits. semver is already inlined into the bundle (tsup noExternal) and already imported in src/update-notes.ts / src/advisory.ts, so no new runtime dependency.

Claim 2 — fail-soft lock planning must preserve the installed copy: confirmed.
installViaTarball (src/update.ts) renamed the pnpm link aside, deleted the backup and the displaced link, and only then ran convergeDshLedger — so a fail-soft refusal (verified legacy SHA-1 tarball, no target SRI) left the copy at the new version with both ledgers at the old version, the only rollback source already destroyed.

Fix: LedgerConvergenceResult gains a divergent flag, set on error paths where at least one ledger exists (manifest spec and/or lockfile record) — i.e. "the ledgers still record the old version after the copy moved." The displaced pnpm link is now removed only after convergence has run and is NOT divergent. A divergent convergence keeps the link and names it in the warn log as the rollback source. The "nothing to converge" case (profile manifest without a billion-context dependency) and the non-dsh-desktop case behave exactly as before — the link is unlinked.

Behavior changes (old → new):

  1. Store-section sync now fails soft (zero writes) when a surviving dep's cloned resolved version contradicts the on-disk copy's declared spec. Old: cloned verbatim — a dep-spec change between versions silently recorded the old resolution and a frozen install would install the wrong dep. New: no write, actionable reason.
  2. On convergence failure with ledgers present, the displaced pnpm link is now KEPT as the named rollback source instead of unlinked immediately. Old: destroyed before convergence ran — no way back. Non-divergent / no-ledger cases unchanged.

Tests: 5 new regression tests — 3 in tests/issue3496-dsh-ledger-convergence.test.ts (contradiction fails soft with zero writes; a satisfying resolution clones through; an unjudgeable dist-tag spec stays silent) and 2 in tests/update-desktop-inplace.test.ts (divergent convergence keeps exactly one displaced link and names it in the warn; the converged control unlinks as before and converges both ledgers). Targeted suites: 67/67 green, typecheck clean, unused-exports gate clean.

Note: this PR now touches src/update.ts — per the repo rules it needs a NO-OP validation release before shipping (proves the existing upgrade path is healthy before the updater change ships). That is a release-time step for the owner.

CI is running on bfc239176a (previous commit 9c63132591 was fully green: 37 success / 13 skipped / 0 failure). I'll confirm once the new run finishes.


八问(中文,供人审)

  1. 是什么问题:[Bug][dsh] In-place self-update leaves missing pnpm lockfile entries and blocks plugin removal #7125 的 store 段同步有两条 fail-soft 缝隙——①依赖声明版本变了但名字/桶没变时,克隆的快照带着旧解析结果,冻结安装会装错依赖;②锁规划 fail-soft 拒写时,被换开的 pnpm 链接已被删除,失败后没有回滚源。
  2. 什么原因导致的:①crossCheckDepMaps 只处理名字级变化(新增/删除),存活依赖的解析版本逐字照抄源版本,从不与目标清单声明的 spec 做满足性检查;②installViaTarball 收尾顺序是"先删备份和旧链接、后跑收敛",而收敛失败按设计只告警不失败——两者叠加,失败路径上唯一的回滚源先于失败判定被销毁。
  3. 影响范围与严重程度:仅 dsh 桌面 profile 的原位更新路径,且触发条件苛刻——①需要版本间依赖 spec 变化 + store 段克隆(如 zod 4.1.8→4.6.7);②需要目标版本无 SRI(仅 SHA-1 的旧 registry 文档)+ 锁规划拒写。严重度:特定条件下有问题(升级后依赖与清单矛盾 / 失败后无回滚源),非常态路径。
  4. 怎么修复的:①对每个存活依赖,用 semver.satisfies 把克隆的解析版本与磁盘上清单声明的 spec 比对,矛盾即 fail-soft 零写入并给出可操作原因;不可判定(非 semver 版本 / dist-tag spec)保持沉默,与 specAdmits 同一纪律。②LedgerConvergenceResult 增加 divergent 标志(有账本且收敛失败时置位),旧链接改为收敛之后、且仅非 divergent 时才删除;divergent 时保留并在告警日志中点名作为回滚源。
  5. 是否有回归问题:审过的既有行为——无 manifest 的桌面 fixture(原有测试"无残留 pnpm 链接")不变:convergeDshLedger 报 "nothing to converge" 时 divergent 不置位,链接照删;非 dsh 桌面路径(convergence 不跑)照删;收敛成功路径照删。fail-soft 语义只收紧(多一种拒写情形),不放宽;"账本失败不失败安装"的既有设计不变。
  6. 是否破坏缓存:不涉及——不触碰缓存/重试/游标/去重键;store 克隆逻辑只增加一个拒写门,成功路径写入内容不变。
  7. 是否改了设计方向:不——仍是 [Bug][dsh] In-place self-update leaves missing pnpm lockfile entries and blocks plugin removal #7125 的同一机制(store 段随 importer 迁移 + fail-soft 零写入),把两条漏网路径补进同一机制;回滚源保留是 fail-soft 的配套,不引入新子系统。
  8. 是否引入了新的配置:不涉及——无新 env/config/默认值。

判定:两条发现均已在本 PR 内修复(commit bfc239176a),等 CI 绿后交人工合并;src/update.ts 变更按仓库规则需先发 no-op 验证版(release 时 owner 职责)。

@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

CI is fully green on bfc239176a: 17 success / 13 skipped / 0 failure (verified live on the commit's check runs). The PR is ready for human merge — mergeable_state: blocked is branch-protection review, not CI.

Reminder of the one release-time step this now requires: the PR touches src/update.ts, so per the repo rules a NO-OP validation release should go out before this ships (proves the existing upgrade path is healthy before the updater change lands).

ci-check added 3 commits November 19, 2026 03:35
…-place update

but left packages:/snapshots: at the old version — an internally inconsistent
lockfile that hard-fails pnpm 11 remove (and dangles a symlink on pnpm >=12,
entry is left alone, a missing one is cloned from the closest existing
billion-context entry (same peer suffix first, then nearest semver), with the
new version's SRI integrity taken from the registry doc (the source's hash is
never carried). A version move that cannot be completed safely (new hash
unknown while the old entry carries one) fails soft with zero writes.

Also: the dep map of a cloned entry is cross-checked against the on-disk
copy's declaration (declared deps missing from the map are resolved from the
store; names no longer declared are dropped); a new lockfile-internal finding
kind makes the broken state visible to bili doctor; doctor --fix sources the
SRI hash from the registry doc (one lightweight GET per distinct drifted pnpm
profile).

Closes #7125
The store-section sync cloned the SOURCE version's resolution data
verbatim: when a dependency's declared spec changed between versions
(zod 4.1.8 -> exact 4.6.7) without the name or bucket moving, the clone
recorded a resolution the new manifest contradicts, and a frozen install
would faithfully install the wrong dep. crossCheckDepMaps now checks
every surviving entry's resolved version against the on-disk copy's
declared spec (semver; unjudgeable specs stay silent, same discipline as
specAdmits) and fails soft on a proven contradiction.

And when that fail-soft (or any convergence failure) leaves the ledgers
on the old version while the copy stands at the new one, installViaTarball
had already destroyed the displaced pnpm link - the copy's only rollback
source. The link is now removed only after a non-divergent convergence;
a divergent one keeps it and names it in the warn log.
LedgerConvergenceResult gains a 'divergent' flag (set on error paths
where at least one ledger exists) so the caller can tell 'nothing to
converge' (link removed as before) from 'ledgers left behind' (link kept).
@ranxianglei
ranxianglei force-pushed the 2026-11-18_dsh-ledger-store-sync branch from bfc2391 to 7ea6077 Compare November 18, 2026 19:37
@ranxianglei

Copy link
Copy Markdown
Owner Author

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

Conflict resolved, branch rebased onto current master and pushed.

State before (verified live this run): PR open, merge state dirty (conflict with master); last CI run on bfc239176a was green (37 success / 13 skipped / 0 failure).

What I did:

  1. Rebased 2026-11-18_dsh-ledger-store-sync onto current master (a446a3c1c8).
  2. One conflict, in the import block of src/doctor.ts: master added the opencode-claude agent imports to the plugin-install line while this PR added fetchVersionDoc to the update line. Resolved as the union — master's superset of plugin-install imports plus the PR's fetchVersionDoc (confirmed present in src/update.ts:2239). No other conflicts; commits 2/3 applied cleanly.
  3. Verified locally on the rebased head: npm run typecheck clean; targeted suites tests/issue3496-dsh-ledger-convergence.test.ts + tests/doctor.test.ts + tests/update-desktop-inplace.test.ts = 72/72 pass. Diff scope unchanged from the original PR (same 6 files). Full-suite verdict is the single CI pass now running on the new head.
  4. Force-pushed (the authorized conflict rebase): bfc239176a → 7ea607712e. This is the branch's one full CI pass — no [ework:ci-held] marker on the head.

Reminder carried over from the review thread: the PR touches src/update.ts, so a NO-OP validation release must go out before this ships (release-time step, owner's call).

八问(中文,供人审)

  1. 是什么问题:PR 分支落后 master 产生合并冲突(dirty),无法合并。
  2. 什么原因导致的:master 期间前进了多个合并(含 batch11-wave11),双方都改了 src/doctor.ts 的 import 块——master 加了 opencode-claude 相关 import,本 PR 加了 fetchVersionDoc import,行级冲突。
  3. 影响范围与严重程度:仅合并可用性——PR 处于不可合并状态;无运行时影响。
  4. 怎么修复的:rebase 到当前 master;唯一冲突按 import 并集解决(master 超集 + PR 的 fetchVersionDoc),无语义改动;本地 typecheck + 72 个定向测试验证后 force-push。
  5. 是否有回归问题:无——冲突解法只是 import 行并集,两侧 import 全部保留且各自有使用点;diff 文件集与原 PR 完全一致(6 个文件),未引入任何新改动。
  6. 是否破坏缓存:不涉及。
  7. 是否改了设计方向:不涉及。
  8. 是否引入了新的配置:不涉及。

判定:冲突已解、分支已推(head 7ea607712e),等本次 CI 转绿后交人工合并(合并权在人);合入前仍需先发 no-op 验证版(src/update.ts 变更的仓库规则)。

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug][dsh] In-place self-update leaves missing pnpm lockfile entries and blocks plugin removal

2 participants