Repository navigation
feat: record timing of each ladder rung and evidence loop - #5768
Conversation
…ime in disk-usage entries
janicduplessis
left a comment
There was a problem hiding this comment.
AI review: I read the whole diff at 68ac716 and the surrounding functions. No behaviour change found in the wrapped calls. One defect in the new record itself.
Findings
-
BLOCKING,
daemon/ladder.ts:1002with the three daemon log writers (stimd/main.ts:775,daemon/passes.ts:317-320,:438,:518). The rung record passesfields: { rung, level: context.level }, and every writer builds{ src, level, event, msg, ...fields }, so the spread overwrites the record's ownlevel. Checked with the same expression:{"src":"stimd","level":"high","event":"daemon_step","msg":"x","step":"rung","rung":2}. Every rung record indaemon.ndjsontherefore haslevel: "high"or"critical"instead ofdebug. Neither is inLEVELS(core/state/ndjson.ts:14);levelRankmaps an unknown level to 0, sologs --leveland the block filter inlog-reader.ts:365treat it as debug by accident, andlogs --jsonshows an invalid level. The guide text ("one debug record ... for a rung: rung and level") describes a record the daemon cannot write. The ladder test misses it becauselogis avi.fnthat receives level and fields as separate arguments and never goes through a writer. Rename the field (for examplepressureLevel) and update the guide, the website text and the test, or assert throughcreateNdjsonWriter. -
NON-BLOCKING,
daemon/step-timing.ts:42. In thecatch,String(error)can throw for a non-Error thrown value (a null-prototype object, a throwingtoString). That TypeError would replace the original error, which is the one thing the wrapper promises not to do. Errors in these call sites areErrors, so this is unlikely. Move the message computation into thefinallytry, or wrap it. -
NON-BLOCKING,
__tests__/daemon-step-timing.test.ts. The throw test asserts onlyrejects.toBe(failure)andtoHaveBeenCalledTimes(1), notoutcome: 'threw'orerror. There is no unit test foraborted(only the ladder test covers it, and it does not check error identity). No test covers a changed return value at the dependency or stage call sites. The PR description lists both under "Test plan". -
NON-BLOCKING,
core/state/status-measures.ts:215.readDiskUsageWalkMshas no production caller, only the test. CLAUDE.md says not to export an internal solely to test it. Either drop it and read the file in the test, or keep it as the reader for the follow-up, and say so. -
NON-BLOCKING,
disk-usage.ts:~199.walkMsisnow() - startedAt, wherestartedAtis taken beforetreeSignatureandoptions.targets(), so it includes those. The measure is the same bytes andmeasuredAtas before. It also adds onenow()call on the measured path; fine with a pure clock.
Transparency (checked by reading old and new code)
Every wrapped call is the same function with the same arguments, awaited at the same point, result used the same way. timedStep is async, so the wrapped call adds only microtask ticks. idleWorkspaces was async and is now a non-async wrapper that returns timedStep(...), so a synchronous throw still becomes a rejection. readIdleWorkspaces iterates the same array built in the same order as before (checkouts is the old inline expression), after the same discoverCheckouts await. runDormantDependencies keeps the cadence block outside the wrapper and moves only the actions body into a closure with the same two conditionals. runRung keeps the same catch (claim-unavailable rethrow, throwIfAborted, pressure_rung_failed, failed action push); before is read outside try.
Changed signatures and callers
rankDependencies(idle, protectedRoot, operation?, source = false, log?): callersladder.ts:586anddependencies.ts:522, both(idle, protectedRoot, operation, false, log).falseis the old default. No other callers (tests included).rankSourceDependencies(idle, keeps, protectedRoot, rule, operation?, log?): callersladder.ts:768anddependencies.ts:474, both the same first five arguments as before, thenlog. No other callers.idleWorkspaces(step, config, reading, ...): private; step is prepended and all three callers (idleStageCandidates,stoppedOutputCandidates,dormantCandidates) pass their original arguments unchanged. The exported function signatures are unchanged.DeviceReadinggains optionallog. It is already present on everyRungContextandIdlePassInputspread into it, so the loops now log wherever those objects reach them.collectArtifactsAsyncandtakeIdleDependencies: signatures unchanged.
Other checks
- Reads and writes:
readDiskUsage,trustedMeasureandfreshMeasureare unchanged.readDiskUsagepicks named fields, so an older Stim ignoreswalkMs, and this one reads an entry without it (the test covers that). Nothing readswalkMsor any step timing to decide anything. - Logging:
logis a synchronouswriteSyncappend per record on the calling thread, inside afinallyguarded by try/catch. A throwing logger is swallowed (tested).debugreachesdaemon.ndjsonby default: the writers apply no level filter, the same path asdaemon_maintenance. - Types: the daemon
logtype now allowsdebug(IdlePassInput['log']inidle.ts). It is internal; nothing inapi.api.txtor the RPC schema mentions it. - Volume: from the code I count about 20 at critical, 16 at high, and 7 per hour for the dormant dependency step. This is consistent with the stated bounds.
- Comments follow the policy; no invented quotes.
Not checked: I ran no tests or builds (read-only). I did not exercise a real daemon.
… unused walkMs reader
janicduplessis
left a comment
There was a problem hiding this comment.
AI review: clear at fe90b55. I read the whole diff against main and the surrounding functions, with "no behaviour change" as the claim to check. I found no blocking defect. I ran no tests or builds (read-only).
Signatures and callers (from the code at this head)
| Function | Change | Callers (all of them) |
|---|---|---|
rankDependencies(idle, protectedRoot, operation?, source = false, log?) |
log appended |
ladder.ts:586, dependencies.ts:522; both pass (…, operation, false, log), false being the old default. No other caller in src or tests. |
rankSourceDependencies(idle, keeps, protectedRoot, rule, operation?, log?) |
log appended |
ladder.ts:768, dependencies.ts:474; same first five arguments as before, then log. No other caller. |
idleWorkspaces(step, config, reading, thresholdMs, setting, worthIt, selection) (private) |
step prepended; body moved to readIdleWorkspaces(tally, …) |
idleStageCandidates, stoppedOutputCandidates, dormantCandidates (stages.ts:337, 349, 497); each passes its old arguments after the step name. The three exported signatures are unchanged. |
DeviceReading.log? |
new optional field | Not a parameter; absent means no record. Set only where a RungContext / IdlePassInput is spread into it. |
IdlePassInput.log |
type now DaemonLog (adds 'debug') |
Internal type only; api.api.txt is untouched by the diff. |
refreshMeasure's inner record(…, walkMs) |
fifth parameter | Three call sites inside refreshMeasure (absent path passes null, measured path passes the elapsed time); the early-failure and fresh paths do not call it. |
Wrapped calls: each is the same function with the same arguments in the same order, awaited at the same point, its result used as before (collectArtifactsAsync x2, takeIdleDependencies/rankSourceDependencies/rankDependencies, planRung6, planWorktrees, planHistory, RUNG_ACTS[rung]). runDormantDependencies keeps the cadence gate outside the wrapper and moves only the two conditionals into the closure. runRung reads before outside the try, and its catch is unchanged. readIdleWorkspaces iterates the same array as the old inline expression, after the same discoverCheckouts await.
timedStep
- Returns
result; with nologit returnsrun(tally)directly (a synchronous throw inside the async function is still a rejection). - The error object is never replaced:
describehas its own try/catch (the earlier non-ErrorString()hazard is fixed and tested), and everything infinally(clock reads,toISOString,items, the log call) is inside one try/catch. The only unguarded expression in the catch iserror.nameon anError, which cannot realistically throw. - Reserved names (
src,level,event,msg,ts) are filtered out of caller fields. The only caller fields in the diff arerungandpressureLevel. The record's own keys (step,startedAt,durationMs,items,outcome,error) collide with nothing a writer sets (stimd/main.ts:775,passes.ts:317-320, 438, 518all spread...fieldsaftersrc/level/event/msg). The writer-shape test showslevel: 'debug'survives andpressureLevelis present. debugis inLEVELS(core/state/ndjson.ts:14).
walkMs: written only by refreshMeasure. git grep finds no reader in production code and no readDiskUsageWalkMs remnant, so knip has nothing to flag. readDiskUsage, trustedMeasure and freshMeasure are unchanged; readDiskUsage picks named fields, so entries with and without walkMs read identically (the test checks both, and that the legacy entry still counts as fresh). The guide's span ("just before the install signature and target reads to the end of the walk") matches startedAt at disk-usage.ts:174.
Status JSON and existing records: no existing record's fields changed; no status payload code is touched.
Tests (against the CLAUDE.md rules): throw asserts the same object rethrown plus outcome: 'threw' and error; aborted asserts the outcome and no error field; the writer-shape test goes through the spread the real writers use; dormantCandidates returns an equal result with and without a logger; performance.now and the walk clock are faked. The ladder test asserts the ladder continues after a throwing rung and that an abort still reaches the caller.
Guide and website: step names, pressureLevel, and what items counts match the code for each step (candidates+skips for collect_artifacts; ranked (+kept) for the rank steps; actions/worktrees for the plans; checkouts visited for the three evidence loops; rung actions for rung). I did not re-derive the "up to 16 at high / 22 at critical" counts by enumerating every path.
NON-BLOCKING nit, no change needed to merge: guide/logs.ts:381 says "each disk-usage cache entry also records walkMs". Entries for an absent dependency path (record(…, null)) carry none, and the "no targets" case records a near-zero value. Say "each entry written after a walk".
Labels: adding review: ai-approved. Not adding review: needs-human: the change adds a debug log record and a cache field, no command, flag, setting or default, and no user-visible cleanup behaviour; the record is observability the issue's design lists as item 1, and diagnosing the hours-long ticks is the stated purpose. The one thing a maintainer might want to know is log volume (up to 22 small records per tick at critical, in a file rotated at 8 MiB), which is documented in the guide.
janicduplessis
left a comment
There was a problem hiding this comment.
AI review: clear at 02bde7a. No defect found. Read-only: I ran no tests or builds.
timedStep in step-timing.ts
- Success returns
result; the throw path rethrowserror, the same object, unconditionally. With no logger it returnsrun(tally)directly. - The catch holds only
classify(...)and a destructuring assignment of a plain object, so it cannot throw.classifywraps every read of the thrown value (instanceof,.name,.message,String()) and ofsignal(.aborted) in one try, and its fallback is{ outcome: 'threw' }, so a failed classification recordsthrewwith noerrorfield. The finally is a single try/catch around the clock reads,items, and the log call. - Reserved names (
src,level,event,msg,ts) are still filtered from caller fields. - Aborted precedence is unchanged:
signal.abortedor anAbortErrorname givesaborted, otherwisethrew.
Scope of 02bde7a: git diff fe90b55cc 02bde7a61 --stat shows only step-timing.ts, its test and guide/logs.ts, so the call sites tabulated in the earlier review are unchanged. I checked ladder.ts (rank_dependencies, plan_rung6, plan_worktrees, rank_source_dependencies, plan_history, rung) against origin/main. Each wraps the same call with the same arguments, plus the trailing context.log where the signature gained it.
Regression test: an Error whose name getter throws. It asserts rejects.toBe(odd) (the same object) and that the record has outcome: 'threw'. The log mock is reset in beforeEach, so calls[0] is this test's record.
Guide: "written after a walk ... from just before the install signature and target reads to the end of the folder walk" matches disk-usage.ts (startedAt before treeSignature, walkMs = now() - startedAt, omitted when null).
Label: review: needs-human does not apply. The change adds a debug log record and a cache field, and no command, flag, setting, default or cleanup behaviour. I am adding review: ai-approved and marking the PR ready.
Description
On 2026-11-14 one daemon tick spent 6,699 s in the pressure ladder, and the next maintenance lane spent 4,137 s in the dormant dependency step, while a disk went from 34 GB to 0.2 GB free. The log cannot say where that time went:
daemon_tickanddaemon_maintenancecarry phase totals, rungs are not timed, andpressure_actionrecords only mark when an action finished. Between two of them there are 4,587 s with no record at all. The design on #5759 concludes that these were serial per-checkout evidence loops, not size walks, and that every later step depends on knowing which loop.This change adds the records and decides nothing.
Solution
One new record in
daemon.ndjson,daemon_step, written when a step ends: its name, when it started, how long it took, how many items it counted, and whether it finished, threw or was aborted. It is written for every ladder rung (with the rung and the pressure level, aspressureLevel) and for the evidence loops the design names:idleWorkspaceswith its candidate loops,collectArtifactsAsync, the two dependency rankings,planRung6,planWorktrees,planHistoryandrunDormantDependencies.The wrapper,
timedStep, takes two timestamps around the call it is given, returns its value and rethrows the same error object. The record is written in afinally, a failing logger is ignored, describing an error can never replace it, and with no logger it only calls the function. A caller cannot pass a field named like one the log writers set themselves (src,level,event,msg,ts): an earlier head named the pressure levellevel, which overwrote the record's severity. A loop is recorded once with a count, never per item.itemsmeans the actions recorded for a rung, the checkouts visited for the idle loops, and what the step returned for the others; the guide says which.refreshMeasurealso stores how long each measurement took, aswalkMsin the disk-usage cache entry, from just before the install-signature and target reads to the end of the walk. Nothing reads it yet; entries without it stay valid, andDiskMeasureand the status JSON are unchanged.Volume: at most 1 extra record a tick at normal, 16 at high and 22 at critical, plus about 7 an hour from the dormant dependency step. At critical that is roughly 460 KB an hour against an 8 MiB rotation with one older file kept.
Not wrapped: the activity pass
runIdlePass.What a reviewer should check at each call site in
daemon/ladder.ts,daemon/stages.tsanddaemon/dependencies.ts: the diff is the wrapper around the same call with the same arguments. Where a function gained a parameter to carry the logger down, the value passed for any parameter before it is the default it had.Test plan
timedStep: a step that finishes, one that throws (the caller receives the same error object, also when describing or classifying it throws) and one that is aborted each leave one record with the right outcome; no logger means no record and the same result; a record built the way the log writers build it keeps severitydebugand carriespressureLevel; reserved field names are dropped.walkMsis written by a walk and an entry without it still reads.Part of #5759: pull request 1 of the design posted there.