pm: garbage-collect the virtual store and extracted-tree tiers

[colinhacks]

store prune only walked the CAS. Nothing collected the global virtual store or <store>/v1/trees/. Both key on the dep path plus its graph hash, and calc_deps_hash folds each child's hash into every ancestor, so one dep bump strands that package and every ancestor reaching it. One dev machine held 5,999 and 26,143 entries, 6.8%/11.7% of them churn duplicates.

Installs now record the project at <virtual-store>/.projects/<hash>; prune marks what registered projects reach and sweeps the rest, as pnpm's pruneGlobalVirtualStore does. An empty registry prunes nothing. Also splits out the CAS sweep, which returned early when the CAS root was absent and skipped the tiers.

6 unit tests, each proven to fail when its guard is removed, plus 11 e2e cases in tests/store-prune/.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
nub	Ready	Preview	Aug 12, 2026 5:03pm

[pullfrog]

Important

store prune can delete a global-virtual-store entry that a concurrently running install has already materialized but not yet symlinked. The install still exits 0.

Reviewed changes — the full diff at 5dc26e8 (6 files, 1 commit), plus the linker's two-phase GVS write path, the registration call site's gating, dep_path_to_filename's output space, the CI wiring for vendor/aube tests, and pnpm's actual pruneGlobalVirtualStore/projectRegistry sources.

• Project registry inside the virtual store — PROJECTS_SUBDIR/projects_dir()/register_project()/registered_projects() in aube-store, each record a plain file named blake3(path)[..16] holding the absolute project path, published write-then-rename, with records for deleted projects swept on read.
• Registration on install — run_link_phase records cwd when the linker uses the global virtual store, best-effort so a registry failure can't fail an install.
• prune split into prune_cas + prune_virtual_store — the old return Ok(()) on an absent CAS root no longer skips the tier sweep, which is the bug that let an e2e case pass without ever reaching the guard it tested.
• Mark-and-sweep over the virtual store and <store>/v1/trees/ — walk each registered project's node_modules, follow symlinks landing in the store, recurse transitively; sweep unmarked, non-dot children. An empty registry prunes nothing.
• Tests and docs — 6 unit tests (which do run in CI via aube-parity.yml's cd vendor/aube && cargo test --workspace), a manual bash e2e harness under tests/store-prune/, and a "Reclaiming disk space" section in the virtual-store docs.

I verified the pnpm parity claim against pnpm's source rather than its docs: the algorithm, the <store>/projects/ location, and the empty-registry bail-out all match. Two differences worth knowing — pnpm's registry records are symlinks (nub's plain-file choice is deliberate and documented), and pnpm's walk skips every dot directory where this one skips only .git.

ℹ️ A store that predates the registry loses entries on the first prune after another project registers

The empty-registry guard only protects a store where nothing is registered. As soon as one project reinstalls, every project that hasn't is invisible to the mark phase and its entries are swept — the docs' <Callout> says as much, and pnpm has the identical exposure, so this reads as an accepted design rather than a defect. The gap is that the operator gets no signal at the moment it matters: the output names the count only after the fact, in the "fully referenced by N registered project(s)" branch, which is exactly the branch that does not fire when entries are being deleted.

Technical details

# Surface the reachability basis before the sweep, not after

## Affected sites
- `vendor/aube/crates/aube/src/commands/store.rs:350-385` — `projects.len()` is only
  printed on the "nothing was removed" path; the path that deletes entries prints
  counts but never says how many projects the marking was based on.

## Required outcome
- A user running the first prune after upgrading can tell, from the output alone,
  that the sweep was computed against a suspiciously small set of projects — before
  they discover it by way of `Cannot find module` in an unrelated checkout.

## Suggested approach (optional)
- Print the registered-project count on the sweep path too, e.g.
  `Marking against {n} registered project(s)` before `sweep_unreachable` runs.

ℹ️ Nitpicks

• the_project_scan_finds_workspace_node_modules_but_does_not_descend creates .git/objects, but nothing under .git would appear in found even if the .git arm were removed — that arm is not actually pinned by the assertion.
• find_node_modules_dirs uses entry.file_type(), which does not follow symlinks, so a symlinked node_modules or symlinked workspace member is never entered. That is an under-marking case, which sits awkwardly with the "every heuristic here fails toward over-marking" claim in prune_virtual_store's doc comment. pnpm behaves the same way, so this is a comment-accuracy point rather than a behavior ask.

  ^{｜ Fix all ➔ ｜ Fix 👍s ➔ ｜ View workflow run ｜ Using Claude Opus ｜ 𝕏}

[pullfrog]

Important

The docs' own remedy for an unregistered project — reinstall, then prune — is a no-op on the install warm path, so the projects most at risk never register. The new sweep lock is also taken after the fetch phase has already published entries under their final names.

Reviewed changes — the delta from 5dc26e8 to fb9a3b1, which reworks registration and adds a store-level advisory lock in response to the prior review. I also traced the install pipeline's ordering (install/mod.rs, install/startup.rs, install/materialize.rs) and the linker's publish path (aube-linker/src/materialize.rs) to check what the new lock actually covers.

• Added a .gc.lock advisory lock — gc_lock_file / lock_for_link (blocking lock_shared) / try_lock_for_sweep (non-blocking try_lock) at <store>/v1/.gc.lock, with prune_virtual_store declining rather than racing and the link phase holding it shared.
• Moved registration after the link and made it unconditional — store.register_project(cwd) now runs once link_all/link_workspace has succeeded, on every linker mode rather than only the shared store, so a project-local install's un-hashed tree entries are protected by its own registration.
• Widened the project scan to skip every dot directory — matching pnpm's findAllNodeModulesDirs, with the doc comment explaining why the project's own .aube/ is still reached.
• Tightened two tests — the registry test now reads the registry directory instead of re-calling the accessor, and the e2e sweep compares realpaths so macOS's /tmp → /private/tmp no longer reports a false mismatch.
• Added a lock unit test and e2e CASE 7 — a_held_link_lock_stops_the_sweep with a drop-the-guard positive control, and a hoisted-project registration case.

I checked the std API against the toolchain: CI pins Rust 1.93/1.95/stable and File::try_lock() returns Result<(), TryLockError> (stabilized 1.89), so try_lock().ok()? correctly maps contention to None. Per flock(2), two file descriptions on the same path do conflict within one process, so a_held_link_lock_stops_the_sweep pins the lock rather than passing vacuously.

I left the TOCTOU thread from the previous review open — the lock closes the link-phase half of that window, not the fetch-phase half.

ℹ️ Nitpicks

• prune_virtual_store's contended-lock message reads An install is using this store, but the exclusive lock is also what a second concurrent store prune holds, so the message names the wrong holder in that case.
• prune_cas never takes the lock, so half of the same prune() command still runs unsynchronized. referenced_hashes reads only <store>/v1/index/*.json, and import_verified_tarball writes CAS content (lifecycle.rs:921) before the index naming it (:948); a freshly imported file has nlink == 1 so the existing nlink > 1 guard does not cover it. Pre-existing rather than introduced here, and it fails loudly as MissingStoreFile, but it sits directly inside this commit's stated goal.

  ^{｜ Fix all ➔ ｜ Fix 👍s ➔ ｜ View workflow run ｜ Using Claude Opus ｜ 𝕏}

[pullfrog]

ℹ️ Both blocking concerns from the last review are closed. Minor suggestions inline.

Reviewed changes — the delta from fb9a3b1 to aeadb33, which moves the sweep lock and adds warm-path registration in response to the prior review. I re-traced the install pipeline to check what the relocated guard now covers, and audited every command that can reach the linker's store-write paths.

• Moved the sweep lock to run_inner, ahead of the fetch phase — let _sweep_guard = store.lock_for_link() now sits at install/mod.rs:1356, before select_lockfile_result, both spawn_gvs_prewarm sites and run_link_phase, and is held to the end of the install. link.rs keeps a comment where the acquisition used to be.
• Registered the project on the install warm path — try_install_fast_path's return at install/mod.rs:789 now calls store.register_project(&cwd) before control::complete, so the docs' repair advice ("run an install, then prune again") actually registers the projects it was written for.
• Corrected prune_virtual_store's doc comment — it names the dot-directory skip and the lock decline instead of the stale .git-only wording.
• Added e2e CASE 8 — empties the registry, reinstalls, and asserts the record comes back plus that projA still resolves after a follow-up prune.

I verified the relocated guard actually covers everything: run_link_phase has exactly one caller, and add / remove / update / dedupe / ci / dlx / patch / deploy / auto_install all funnel through install::run → run_scoped → run_inner, so they inherit it. import and rebuild never write either tier, and aube-linker/src/quarantine.rs only strips xattrs off entries the linker already materialized. Both prewarm handles are joined on the ordinary success and error paths. The warm-path registration also cannot arm the sweep with a project that marks nothing — compatibility_metadata_is_current requires .modules.yaml currency, and that file lives inside node_modules.

ℹ️ Nitpicks

• Store::lock_for_link's doc at vendor/aube/crates/aube-store/src/lib.rs:384 still says "for the duration of a link phase". It now spans the whole install, which is the fact a future reader most needs; the method name says "link" too.
• The contention window widened from the link phase to the entire install, lifecycle scripts included, and try_lock_for_sweep never waits. So nub store prune now skips the tier sweep — and still exits 0 — for as long as any install is running. A bounded blocking acquire, or a non-zero exit when the sweep was skipped, would keep the command from looking like it collected nothing to collect.
• materialize_handle (install/mod.rs:2514) is not joined or aborted when the sibling fetch_handle task panics: the ? at :2525 fires on the outer JoinError and returns before the combine_install_pipeline_errors join at :2533, and MaterializeJoinHandle is a bare JoinHandle rather than an abort-on-drop JoinSet. A detached prewarm writer can then outlive _sweep_guard. Pre-existing and panic-only, but it bounds the guarantee the new comment states.
• The warm path's super::open_store(&cwd) costs four with_settings_ctx loads by default — resolved_store_dir, resolved_cache_dir, global_virtual_store_dir, and that function's own resolved_cache_dir fallback — and FileSources::load is not memoized. That is paid on every "Already up to date" install to rewrite a record that is almost always already there.

  ^{｜ Fix all ➔ ｜ Fix 👍s ➔ ｜ View workflow run ｜ Using Claude Opus ｜ 𝕏}

[pullfrog]

ℹ️ The warm-path pin lands correctly. Two minor points — one factual, one about the pin's durability.

Reviewed changes — the delta from aeadb33 to 9487050, which is test-harness only; no Rust changed. I traced the RUST_LOG plumbing to confirm the new probe can actually fire, and checked the README's CI claim against aube-parity.yml.

• Pinned the warm-path precondition in e2e CASE 8 — the install now runs under RUST_LOG=debug and an ok asserts phase:link is absent, so a fall-through to the slow path fails the case instead of passing green via the link-phase registration.
• Documented the coverage situation in the harness README — a "Known gaps" paragraph and a table row for the warm-install case.

I verified the probe is live rather than assuming it: crates/nub-cli/src/main.rs:49 calls pm_engine::log::init() on every invocation, log.rs:81-92 hands RUST_LOG ownership of the EnvFilter, and RewriteLayer::on_event eprintln!s each enabled event as LEVEL <message>, so a slow install prints DEBUG phase:link 1.2ms (N files) and present::rewrite leaves that text alone. The trailing space is load-bearing exactly as the comment says — link.rs:542 emits phase:link_bins.

ℹ️ A command that deletes directories from the user's global store has no automated end-to-end coverage

The Rust unit tests exercise mark_from / sweep_unreachable / the registry as pure functions, but nothing automated drives nub store prune against a real store built by a real install — which is where the sweep's actual failure mode lives (deleting a directory a live project is symlinked at, then exiting 0). The new README paragraph accepts that as a known gap; whether manual-only is the right long-term posture for a destructive command is a call for the human, not something the diff can settle.

Technical details

# The GC's only end-to-end coverage is a manual script

## Affected sites
- `tests/store-prune/run-prune-sweep.sh` — 11 assertions over 8 cases, invoked by hand only.
- `tests/store-prune/README.md:34` — records the gap as accepted.
- `.github/workflows/aube-parity.yml` — already path-filtered to `vendor/aube/**` and already
  provisions a Rust toolchain, so it is the closest existing home if this becomes a CI job.

## Required outcome
- A decision on record: either the harness gets a CI trigger, or the manual-only posture is
  deliberate and the README says so as a choice rather than as a limitation.

## Open questions for the human
- Is `tests/store-prune/` intended to stay manual indefinitely, in the shape of
  `tests/pnp/`, or is wiring it up follow-up work worth an issue?
- The script needs network (`nub install` of `debug` / `ms`) and reassigns `HOME`; the
  registry-dependent cases would also need `CI` unset, which a CI job makes awkward. Is that
  what makes it manual, or just sequencing?

  ^{｜ Fix all ➔ ｜ Fix 👍s ➔ ｜ View workflow run ｜ Using Claude Opus ｜ 𝕏}

[pullfrog]

The unit tests do run in nub CI: aube-parity.yml runs cargo test --workspace inside vendor/aube on ubuntu and windows-latest, both gating pull_request into main, path-filtered on vendor/aube/** — which this PR touches. The --all-targets parenthetical is true of the root-workspace gates only, so as written the paragraph tells a future maintainer these tests are unprotected when only the bash script is.

Suggested change

**This harness is manual, and it is the only coverage the warm-path registration has.** The Rust unit tests reach the mark, sweep, and registry functions but not the install pipeline, and neither this script nor those tests run in nub CI — the unit tests live under `vendor/aube/**`, which `--all-targets` never builds from the root (the aube-workspace gate). Run it by hand when touching `store.rs`, `aube-store`, or the install fast path.

**This harness is manual, and it is the only coverage the warm-path registration has.** The Rust unit tests reach the mark, sweep, and registry functions but not the install pipeline. They do run in CI — `aube-parity.yml` runs `cargo test --workspace` inside `vendor/aube` on ubuntu and windows for any PR touching `vendor/aube/**` — but this script runs nowhere. Run it by hand when touching `store.rs`, `aube-store`, or the install fast path.

[pullfrog]

This is an absence assertion with nothing proving the string can appear, so if the label moves, the RUST_LOG plumbing changes, or the debug channel goes quiet, the grep stops matching, the case reports warm unconditionally, and a slow-path fall-through passes green again — the exact failure the pin exists to catch. A total install failure is still caught by the next assertion (registry count 0 ≠ 1); only the broken-instrument case slips through.

Technical details

# Give the `phase:link ` probe a positive control

## Affected sites
- `tests/store-prune/run-prune-sweep.sh:98-99` — asserts `phase:link ` is absent with no
  companion assertion that it is ever present.

## Required outcome
- A broken or renamed probe fails a case instead of making CASE 8 pass vacuously.

## Suggested approach (optional)
- CASE 7's `flat` install (`:78`) is a guaranteed-cold install that must reach
  `run_link_phase`. Run it under `RUST_LOG=debug` and assert `phase:link ` IS present there;
  two lines, and it validates both the pattern and the trailing-space distinction against
  `phase:link_bins`.
- Scoping the filter to `RUST_LOG=aube=debug` would also cut the log to the engine's own
  events while still matching, since `link.rs` lives in the `aube` crate.

[pullfrog]

Important

The new rule that an unresolvable project stops the sweep collides with nubx: nub dlx / nubx / nub x / nub create register a scratch project they delete on exit, so a fresh permanently-unresolvable record is minted on every invocation and the tier sweep this PR exists to add never runs.

Reviewed changes — the delta from 9487050 to 9eb810f, the first Rust change since aeadb33. I traced every caller that reaches Store::register_project, audited whether the new .gc-state file can be misread by any other consumer of the store roots, and derived pluralizer's actual output from its 0.5.0 rule tables rather than assuming it.

• Two-phase removal on a 30-day clock — GRACE, SweepOutcome, and a .gc-state file per tier recording when each entry was first seen unreferenced; sweep_unreachable defers on first sighting, clears the record when an entry becomes reachable again, and drops records for entries that have vanished.
• Reading the registry no longer destroys records — registered_projects returns RegisteredProject { record, dir, exists } and removal moved to an explicit Store::forget_project, with prune_virtual_store aging stale records out on the same clock.
• The sweep declines while any registered project is unresolvable — an unmounted disk and a deletion are indistinguishable, so it returns rather than marking against partial knowledge.
• Fixed two ways a live project read as dead — registered_projects skips every dot-prefixed name (its own state file was being parsed back as a registration) and strips only \r/\n instead of trim()ing a legitimate trailing space; find_node_modules_dirs now takes a symlinked node_modules.
• Announced the blocking acquire — lock_for_link_with(on_wait) prints Waiting for a store prune to finish... on the slow install path, prune takes the lock once for both sweeps, and store add takes it too.
• Extended both test layers — four unit tests (grace deferral with a positive control, clock reset on revival, state-file bounding, symlinked node_modules, trailing-space path) plus e2e CASE 9 for the upgrade sequence.

I confirmed the new .gc-state file and the .projects directory cannot be misread elsewhere: only sweep_unreachable and registered_projects enumerate the virtual-store and trees roots, both with an explicit dot-skip, and aube-linker/src/sweep.rs:16-43 matches only .tmp-<pid>-. Every other read_dir in the engine targets a project-local .aube, a project node_modules, or the CAS.

ℹ️ The 30-day hold has no override, so store prune cannot reclaim disk today

The hold is the right default and the reasoning in GRACE's doc comment is convincing. What is missing is the exit: a user who runs nub store prune because they are out of disk space now gets Holding N entries... and zero bytes back, with no flag to say "I know what I am doing" and no way to shorten the window. pnpm's store prune deletes immediately, so this is a deliberate divergence in the command's headline purpose rather than an oversight — worth being a decision on record.

Technical details

# `store prune` has no way to act on the first pass

## Affected sites
- `vendor/aube/crates/aube/src/commands/store.rs:588` — `GRACE` is a hard-coded constant with no
  setting, env var, or flag reaching it.
- `vendor/aube/crates/aube/src/commands/store.rs:451-464` — the deferral message tells the user
  how to KEEP entries, never how to remove them now.
- `site/content/docs/install/virtual-store.mdx:142-149` — documents the delay as unconditional.

## Required outcome
- A user who understands the tradeoff can reclaim the space in one command, or the docs state
  plainly that they cannot and why.

## Open questions for the human
- Is a `--force` / `--min-age=<duration>` on `store prune` in scope for this PR, or deliberately
  deferred? A duration knob generalizes better than a boolean and matches the `NUB_PRIMER_TTL`
  precedent in AGENTS.md.
- Should the deferral message name the remedy for the other direction ("run again after N days,
  or `--force` to remove now")?

ℹ️ Nitpicks

• pluralizer::pluralize("registered project is", n, true) does not produce English. Derived from pluralizer 0.5.0's constants.rs: at n == 1 to_singular falls through to the generic ("(?i)s$", "") rule and prints 1 registered project i not reachable right now; at n >= 2 to_plural hits ("(?i)s?$", "s"), which consumes the trailing s and re-emits it, printing 2 registered project is not reachable right now. The verb needs to sit outside the pluralized noun. The e2e case greps only not reachable right now, so it passes either way.
• unix_now()'s unwrap_or(0) (store.rs:603-608) means "already expired": a prune that runs while the clock is unreadable stamps every unreferenced entry 0, and the next prune with a healthy clock deletes all of them at once, bypassing the window entirely — the harness's own expire_all uses exactly 0 for that effect. It needs a pre-1970 clock to trigger, so this is cheap insurance rather than a live bug: skip recording instead of recording 0.
• lock_for_link's doc (aube-store/src/lib.rs:423-424) says it is "for callers whose critical section is a single file write, where a wait is imperceptible", but neither caller fits. store add holds it across the whole fetch loop, and the warm install path can stall for the length of a prune — the same 24 s of silence lock_for_link_with was added to fix, on the most common install of all. What the user feels is the wait, not the critical section.
• e2e CASE 9's rescue assertions (run-prune-sweep.sh:147-150) only check that two and three still resolve, which is also true if the second prune swept nothing at all. Asserting the entry count dropped, or that u-prune2.log reports a removal, would keep the case from passing vacuously.

  ^{｜ Fix all ➔ ｜ Fix 👍s ➔ ｜ View workflow run ｜ Using Claude Opus ｜ 𝕏}

[pullfrog]

nubx / nub dlx / nub x / nub create install into a tempfile scratch project and delete it on exit (dlx.rs:200-205, :239, :329; create.rs:58), and that install always reaches run_link_phase's registration — dlx_install_options uses FrozenMode::No, so the warm fast path is never eligible (install/startup.rs:75). Every invocation therefore mints a fresh record naming a directory that will never come back, so on any machine that uses nubx there is always one inside its 30-day window and this branch declines the tier sweep on every run. The CAS sweep still works; the virtual-store and trees sweep this PR exists to add does not.

Technical details

# An ephemeral project's registration blocks the sweep indefinitely

## Affected sites
- `vendor/aube/crates/aube/src/commands/store.rs:397-404` — one unresolvable record aborts the
  whole tier sweep, for every project, not just the entries that record would have marked.
- `vendor/aube/crates/aube/src/commands/install/link.rs:470` — registers unconditionally after a
  successful link, with no notion of a throwaway project.
- `vendor/aube/crates/aube/src/commands/dlx.rs:200-205, 239, 245, 329` — scratch dir created,
  pinned as `opts.project_dir`, installed into, then deleted on exit.
- `vendor/aube/crates/aube/src/commands/create.rs:58` — `nub create` routes through `dlx::run`.
- `crates/nub-cli/src/pm_engine/install_family.rs:693, 742, 797` — the nub-side entrypoints.

## Required outcome
- A store on a machine that uses `nubx` still has its virtual-store and extracted-tree tiers
  collected. A project that is genuinely throwaway must not leave evidence that looks like an
  unmounted disk.

## Suggested approach (optional)
- Root-cause fix: give `InstallOptions` a flag that suppresses registration and set it in
  `dlx_install_options`, or have `dlx` call `store.forget_project` for its scratch dir just
  before `drop(tmp)`. dlx's GVS entries are genuine garbage the moment it exits — exactly what
  this sweep should be free to collect.
- Independently, consider narrowing the blast radius of the decline itself: an unresolvable
  project could disqualify only its own contribution to `reachable` while the rest of the sweep
  proceeds, rather than aborting the command.

## Open questions for the human
- `add -g` / `remove -g` (`add/global.rs:89-113, 320-338`, `global.rs:508-521`) and `deploy`
  into a discarded Docker stage (`deploy/mod.rs:319, 351`) are narrower versions of the same
  shape — each self-heals after 30 days but blocks everything meanwhile. Is a 30-day stall
  after an ordinary `nub remove -g` acceptable, or should deletion of a known-owned directory
  call `forget_project` at the point of deletion?

[pullfrog]

try_lock_for_sweep is file.try_lock().ok()? (aube-store/src/lib.rs:434-438), which maps a genuine lock error the same way as contention — and because this commit takes one acquire for the whole command, that now aborts the CAS sweep too, which previously ran unconditionally. On a store where flock is unsupported (some FUSE and NFS mounts) nub store prune becomes a permanent no-op that exits 0 while blaming an install that does not exist. lock_for_link_with distinguishes WouldBlock from Error(_) two screens up; the sweep side should make the same distinction and fall back to lock_for_link's documented posture of proceeding unsynchronized.

[pullfrog]

This sentence no longer describes the behavior. Deleting a project directory makes prune_virtual_store decline the entire tier sweep (store.rs:397-404) for 30 days, then the record ages out, then the entries need their own 30-day window — so a deletion pushes collection out by up to 60 days and stops every other project's garbage being collected in the meantime. The ... not reachable right now message, which is the one a user is most likely to hit, appears nowhere on this page.