Repository navigation
Closed as not planned
Closed as not planned
Copy link
Labels
area: clipackages/stim-cli, including its README and guidepackages/stim-cli, including its README and guidepriority: P1Next up: user-facing defect or unblocks other workNext up: user-facing defect or unblocks other worktype: bugBreaks documented or expected behaviorBreaks documented or expected behavior
Milestone
Description
Report
With discovery (#5519) live, the daemon's maintenance lane spends much more time in the dormant dependency step, and stim gc --dry-run takes several minutes on a machine with many checkouts.
Read-only measurements on one MacBook, 2026-11-11 23:57 to 2026-11-12 00:26, discovery build 16a2751ef, under heavy load from other work (so partly load, not only code):
- The dependency phase took 331 s, 251 s and 277 s in 31 minutes: about 46% of the maintenance lane's time, against 2.7% in the six hours before. The lane runs one thing at a time, so a busy lane cannot start the pass that would run the size scan.
- That step reaches discovery through
stimd/passes/artifacts.ts(artifactWorktreesAsync:discoverCheckouts, then a source-checkout resolve and walks per candidate) anddaemon/stages.ts(discover: true). discoverCheckoutsis memoised per operation only, so every pass recomputes it. Its listing is cheap (git worktree listover 41 roots took 1.3 s in total); the cost is the per-checkout evaluation: 41 registered plus about 112 discovered checkouts, each checked for merged, idle, dirty, unpushed and ignored files.recognises()andisProject()inworkspace/discovery.tsuse synchronous directory reads on the event loop; event-loop max delay in lane passes was 6.5 s and 11 s after the discovery build, 3.2 s before.- The dependency step is gated to once per hour (
daemon/dependencies.ts,dormantCheckedAt) yet ran three times in 30 minutes in one daemon run; the cause was not found (possiblystim gc --dry-runforcing it through the daemon). stim gc --dry-runsat for minutes at "checking 153 projects for merged or idle worktrees".
Expected
- One discovery result reused across ticks for a bounded time.
- Discovered-checkout evaluation under a time budget per pass, resuming where it stopped, so it cannot occupy the lane for minutes.
- No synchronous filesystem walks on the daemon's event loop.
- The reason the hourly gate did not hold, found and fixed.
- A dry run that reports progress per checkout or finishes in seconds on 150 checkouts.
Parent: #5512.
Metadata
Metadata
Assignees
Labels
area: clipackages/stim-cli, including its README and guidepackages/stim-cli, including its README and guidepriority: P1Next up: user-facing defect or unblocks other workNext up: user-facing defect or unblocks other worktype: bugBreaks documented or expected behaviorBreaks documented or expected behavior
Activity
janicduplessis commentedon Nov 14, 2026
Current-main triage at
7f822024d: the original numbers came from a loaded Mac, and the report distinguishes the cheap checkout listing from the slower per-checkout evaluation. Keep this open for a matched dry-run plus maintenance tick against the same read-only inventory. Record discovery, per-checkout evaluation, and event-loop delay separately, and distinguish scheduled dependency runs from manual gc requests before changing their cadence.#5163 is related on the serving side. Reusing a fresh preview can avoid repeat traversal by Desktop, while this issue still owns the cost and scheduling of producing that preview. Preserve the per-checkout evidence and pre-deletion rechecks described by LLP 0004.
janicduplessis commentedon Nov 14, 2026
Found a path that resets the hourly dependency check without restarting stimd. On main
df9bfb200,dormantCheckedAtbelongs to the module loaded in thepassesworker. The worker exits after two minutes without work, so its replacement starts with-Infinity. The same code was present in the reported16a2751efbuild. Hourly gate · Worker lifetimeA read-only probe called the built dependency pass with an empty inventory and a fixed clock in real Node workers. The same worker inspected the inventory at
+0s, skipped+60s, ran the explicit force at+61s, and skipped+62sand+180s. A replacement worker inspected the inventory at that same+180s. The original log has three matching gaps of 136 s, 327 s and 151 s between the end of the last pass and the estimated start of the next expensive pass, all in one daemon run. The log does not carry a worker ID, so those older worker retirements remain an inference. Manual GC, including dry-run, intentionally forces a separate pass and does not emitdaemon_maintenancefor it. Manual pathCurrent dry-run worktree classification already overlaps some Git reads when child claims permit it and reports an aggregate project count. The dormant pass's candidate loop remains sequential, discovery is cached only within one operation, and folder recognition still uses synchronous reads. The scheduled dependency pass runs in its own worker, while other foreground idle-stage paths also call discovery. This isolates the cadence defect from the remaining traversal, timing and progress work. Dry-run classification · Dormant candidates · Discovery memo
janicduplessis commentedon Nov 14, 2026
Claimed: fix/5531-dormant-dependency-cadence
janicduplessis commentedon Nov 14, 2026
#5745 landed as
715d8588a. The dependency schedule now survives retirement of an idle passes worker. The focused test rejects the first inspection, replaces its worker, skips the premature automatic pass, runs a manual forced scan, and permits a scheduled scan at the hour boundary. Independent review, CI and the merge gate passed.This leaves the cost and reporting of individual discovery scans open. The next measurement should distinguish scheduled scan frequency from the end-to-end latency of one large scan or manual dry run.
janicduplessis commentedon Nov 15, 2026
The reproduced hourly scheduling defect was fixed in #5745. Closing the broader resumable-scan and shared-cache redesign. The original timings predate that fix and were collected under heavy host load. Reopen a focused performance report with current large-checkout timings. The separately reproduced configured-root issue remains tracked in #5757.
janicduplessis commentedon Nov 15, 2026
Measurement for this issue, from the work on #5762 (pull request #5764), read-only on one MacBook at high load, 2026-11-15:
collectWorktreeSweeptook 23 s for 162 candidates, about 0.14 s per discovered worktree (git status, activity, HEAD, unpushed).~/.codex/worktreesand a home folder scan (17 more on that machine, an estimated 2 to 3 s). Its 40-checkout window bounds the reduction passes only. The finished-worktree check, in the background pass and in the pressure ladder, still inspects every discovered worktree.