Repository navigation
fix(workspace): listings tell the truth about disk, drafts and conflicts - #248
Conversation
…ch switches
Branch state (snapshot, tree and log states) rewinds with a historical
rollback; typed chronicle records do not. getRecordIdsByType enumerates them
from every branch, in global order (ids come from one store-wide counter),
and they survive deleteBranch, reopen and SIGKILL. Three AF journals need
exactly that — the workspace's disk-agreement evidence (shelf-383), delivery
receipts (shelf-354) and prose-draft outcomes (shelf-351) — so they share one
helper instead of three: append typed records, checkpoint the caller's
reduced state, replay the tail after the latest checkpoint at open.
Durability is the subtle part. A record append reaches the OS before it
returns, while branch-state commits are buffered until store.sync(): a probe
lost a tree commit made between two appends that both survived a SIGKILL. So
the API names two barriers and implies neither from the returned
{id, sequence}:
- afterCommittedState: sync BEFORE appending, for an entry asserting state
committed just before it (it can then never survive without that state);
- durable: sync AFTER appending, for an entry that must be on stable storage
before the caller acts (an attempt before its outbound request, an intent
before its disk effect).
One writer per record type: a checkpoint covers entries up to the last one
this instance loaded or appended. Entry meaning, reducers, dedup and
checkpoint cadence stay with each caller.
Tests: empty load; append refs and replay order; entries from a rolled-back
branch and main loading from either, through deleteBranch and reopen;
checkpoint + tail with interleaved other record types and a tree commit, the
newest checkpoint winning; the barrier call order; and a SIGKILL child that
commits tree state then appends with afterCommittedState — the entry never
survives without the state. (The same kill without the barrier leaves the
entry and loses the state.)
Agreed in room-220 #41647-#41713 with Oscar-1005, Petra-1012, Hugo-1004.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…he base shelf-383 stacks on
…tore must be reconciled Hugo-1004's review of f7e14f6 found a replay-loss path: append(A, {durable:true}) hands A to the store, then sync() throws; the caller sees failure and never reduces A. A later successful append(B) advanced lastEntryId past A, and a checkpoint({seen:[B]}) then covered A without A ever entering the reducer — load() returned that snapshot with an empty tail. Now any failure after a record reaches the store (the appendJson itself, or the durable barrier after it, for entries and checkpoints alike) marks the journal unreconciled: append and checkpoint throw until load() has replayed the tail, the possibly-written record included, for the caller to rebuild from. A failure before the record reaches the store (the afterCommittedState sync) wrote nothing and leaves the journal usable. load() documents that the caller rebuilds its reduced state from exactly the snapshot and tail it returns. needsReconcile exposes the state. Regression test (fails on f7e14f6, which let append(B) and the covering checkpoint through): a durable barrier failing after A, writes refused until load(), load() replaying A, the journal usable again; and a pre-append barrier failure leaving it usable. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Disk (D), the workspace store (S) and physical disk-agreement evidence (P) decide every path, with branch-local intent (tombstones, store origin, sticky conflicts) qualifying the tree: - D = S agrees (P := D). With P known: D = P keeps the store side (draft, workspace deletion, not-in-branch); S = P adopts disk (ingest, remove, or drop S for a binary/oversize disk version); all three differing is a conflict. With P unknown: tombstone, then store origin, then legacy mismatch decide; a new disk file is ingested or listed disk-only. - P lives in global chronicle records (RecordJournal, workspace/disk-agreement), so an undo or branch switch never rewinds it and a branch difference is never read as a disk edit. Branch intent lives in a per-mount tree state (workspace/<mount>/intent) and rewinds with the workspace tree. - Every transition is write-ahead: a durable intent (prior, expected, branch) before the disk effect or tree adoption; completion only after the effect is durable (fsync; afterCommittedState for tree commits). A pending intent resolves at its next observation; disk matching neither candidate is a sticky `interrupted` conflict. - Observation follows openContainedFile's boundary (lstat, symlink policy, canonical containment); one descriptor is fstat'd and hashed. Fingerprints vouch only on filesystems that maintain ctime (statfs allowlist) and outside the 2 s racy window. A walk proves absence only where it listed completely; a missing mount root is unavailable, not deletion. - Passes observe first (async), then decide and apply synchronously; each mount's passes and tool writes are serialized. Tools: ls/glob/grep check their scope first and mark states (glob matches become objects); grep labels workspace vs conflicting-disk versions and lists disk-only skips; read/edit/delete explain a missing entry; sync without a path never discards a pending edit, sync of a path adopts disk; materialize never unlinks without applyDeletions, selects what evidence says disk owes (refusals stay owed past the watermark), and autoMaterialize pushes under the same rule. Watcher ops only choose paths to re-observe. Framework: Module.onToolBatchComplete, awaited through ModuleRegistry.notifyToolBatchComplete (parallel, 30 s bound, fail-open, module:batch_hook_failed trace) when a round's last pending result arrives, before provideToolResult; the waiting_for_tools guard is re-checked after. The workspace scans watch:'on-agent-action' mounts there with a 20 s deadline, recording a miss on status and as an event. anima-research#169's freshness test now asserts the agreed contract: a full sync keeps the workspace edit; syncing the path adopts disk. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…shelf-383) workspace-three-way: each case on shelf-383's done-when list - the three-way cases, branch rewinds and not-in-branch files, workspace deletes (legacy ones too) and applyDeletions, binary/oversize disk references, disk-only PNGs, shell rm by every scan path, rm with a newer draft, provenance states, restart, real SIGKILL faults after intent and after effect for materialize, shell-edit ingest and an autoMaterialize new-file write (each followed by branch selection), a torn records.log tail, checkpoint replay and moved roots, fingerprints (utimes, cp -p, racy window, unlisted fs), incomplete walks (EACCES, ignored, cap), grep labels, on-agent-action (scan, deadline miss, branch change while waiting) and Mythos's 350-entry directory. fixtures/workspace-fault.mjs is the dying child. tool-batch-hook: the framework seam - once per completed batch before the stream resumes, a hung hook bounded and traced, a cancel during the hook dropping the result, and an on-agent-action workspace scanned before resume. Fixes found while writing them: - An adoption intent from another branch with disk holding its target (or newer) is a branch-local interrupted conflict, not D = P: disk may hold the only copy of an uncaptured edit, and a materialize would have pushed over it. P stays pending for its own branch. - A disk write intent with unknown prior and no file left resolves to unknown (forget), not interrupted: the write never created the file. - Materialize always considers tombstoned paths, so a plain materialize lists the deletions it left on disk. - agentActionScanDeadlineMs (default 20 s) configures the scan deadline. Mutation checks: removing the hook call, the post-hook re-check, the uncaptured protection, the write-ahead intents, or walk completeness each fails its tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d the batch hook Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rim dead helpers Self-review of the rewritten store->disk path: the write and unlink went through join(root, path) with nothing between the lexical check and the filesystem, so a forced materialize over a symlink (or into a symlinked parent directory) would write outside the mount - the old materialize had the same exposure. effectStaysInMount checks, during observation and before any intent is recorded, that the nearest existing ancestor resolves inside the canonical root and that a symlink at the path is acted through only when the mount follows symlinks and it resolves inside. Tests: the forced materialize case for both shapes (red without the check), and a missing mount root reading as unavailable rather than deleting everything. Removes what the rewrite orphaned: sync.ts keeps hashContent and DEFAULT_MAX_FILE_SIZE (safePath, isBinary and SkippedFile had no callers left), unused fs/path imports in index.ts, and an unused helper. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…, real barriers, contained reads Iris-1006's review (room-221 #43725, #43900, #43979, #44025, #44075): 1. A push's completion settled branch intent on whichever branch was selected after its disk await. P (physical, global) is still recorded; branch intent is settled only while the planned branch is selected, and otherwise by convergence when it returns. PushResult carries the branch and sequence it planned on, and materialize pins and watermarks those. 2. fsync failures were swallowed. A write is written and fsynced through one descriptor; a failed file or directory barrier fails the effect, the intent stays pending, and the skip names the failed stage. Directory sync is skipped only on Windows, or for EINVAL/ENOTSUP. 3. A symlinked parent let default no-follow reads escape (O_NOFOLLOW guards only the last component). Every observation checks its parent's canonical placement; every read checks canonical containment and descriptor/path identity, as openContainedFile does; a walk's scope must resolve inside. 4. An unavailable root reports its region even when nothing is tracked. 5. A later write of a pending adoption's expected content passed for the adoption landing, and a disk change then overwrote it. The mutation boundary (write, writeBinary, edit, delete) settles a pending intent before reading or changing the entry; a same-branch adoption settles by the tree even when disk can't be read. 6. Agreement-only P advances had no barrier after their append. A pass and a push end with one whenever they recorded anything; so does the one-time import of anima-research#169 baselines. 7. A pass enumerated candidates on one branch and decided on another after async observation. It gathers again when the branch changed; a branch that keeps changing leaves it explicitly incomplete, deciding nothing. A push writes nothing if the branch its selection was made on is no longer selected. Also: the deadline wording no longer claims an inference started ("deadline exceeded; the scan continues", "finished after the deadline"), and lastAgentActionScan has `complete` (finished at all) and `withinDeadline`. A regression from (5)'s refactor - a not-landed adoption recorded as landed when disk held its target - was caught by the existing fault test. Tests (58 in the three-way file): each finding's case, red-checked against ddb90b0, where all fail except the directory-sync-unsupported case, which both versions accept. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…settle adoptions at commit Iris-1006's second round (room-221 #44621, #44678): - A failed agreement barrier was forgotten: P had advanced in memory, so the next no-op pass recorded nothing and never synced. DiskAgreement now owes a barrier for any append not proven durable, cleared only by a barrier that succeeds; every pass and push settles it before deciding anything (and fails if it fails again) and again at its end. - The gather labelled its branch before the root lstat's await, so A->B->A could pass the check with B's candidates. The label is taken where the tracked candidates are read, with no await between. - settleBeforeMutation's pass may not settle a pending adoption (a branch that keeps changing; an unreadable disk copy), and the write still committed over it. The precondition is now enforced synchronously in commitToolChange: an adoption pending on the selected branch is settled by the tree - which needs no disk - and made durable before the write's own tree commit can be, whatever the pass managed. - DiskAgreement.get/paths reconcile first, so no read (the mutation boundary included) sees the view from before an ambiguous append. Tests: the barrier obligation via the module (EIO injected at the barrier after an agreement; the next no-op pass syncs), the A->B->A gather, Iris's churn shape (write started on another branch, all three gathers defeated, commit on the adoption's branch: B survives as a conflict with disk C), and an ambiguous append read back. Each fails at fecf12b. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Iris-1006's third round (room-221 #44945): - After a failed barrier in the mutation precondition, P was no longer pending, so a retried precondition returned early and never synced. The precondition now discharges any outstanding barrier before permitting a tree mutation at all, whatever P looks like. - open() rebuilt without owing a barrier: a fresh owner loading a previous owner's unbarriered tail treated it as durable. Every rebuild - at open, or after an ambiguous append - now owes one, since records that read back prove nothing about having reached stable storage. Tests: a failed precondition barrier retried (the retry syncs before permitting the mutation; the write then succeeds), and a fresh owner of an unbarriered tail owing a barrier. Both fail at b9573d1. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
| await mkdir(dirname(absolute), { recursive: true }); | ||
| // Written and synced through one descriptor: durable, and needing only | ||
| // write permission (a write-only file is still written). | ||
| const handle = await open(absolute, 'w'); |
There was a problem hiding this comment.
pushPaths approves every target before starting any writes. If another process can change mounted files, it can replace an approved file or parent with an outside-pointing symlink while later paths are checked. open(absolute, 'w') then truncates an outside file, even with followSymlinks: false. Ordinary edits in this window can also be overwritten without force. Bind each write to the checked file and parent before truncating it.
How this was verified: The boundary check is cached before awaited work, while the later destructive open follows the path again without checking its current destination.
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/modules/workspace/reconcile.ts
Line: 831
Comment:
**Writes can escape the mount**
`pushPaths` approves every target before starting any writes. If another process can change mounted files, it can replace an approved file or parent with an outside-pointing symlink while later paths are checked. `open(absolute, 'w')` then truncates an outside file, even with `followSymlinks: false`. Ordinary edits in this window can also be overwritten without `force`. Bind each write to the checked file and parent before truncating it.
> **How this was verified:** The boundary check is cached before awaited work, while the later destructive open follows the path again without checking its current destination.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.There was a problem hiding this comment.
Valid: the containment check was cached at planning, and the destructive open(…, 'w') resolved the path again later. Each write and unlink is now checked where it happens (effects.ts). The parent directory is checked and located inside the mount. The file is opened in it without truncating (O_NOFOLLOW unless the mount follows symlinks; O_EXCL when planning saw no file). Before any byte changes, the descriptor must be a regular file canonically inside the mount, still the file the path names (a file renamed away after it was opened is not), in a directory still at its path. Unless force, it must also still hold the bytes the materialize decided on (rehashed through a descriptor), so an edit made meanwhile is refused rather than overwritten. A refusal that never touched the path puts its evidence back exactly as it was. force overrides freshness, never the boundary. These checks are made where the operation happens, against what is there at that moment. Node has no openat, so a parent directory replaced concurrently in the window between the check and a create, mkdir or unlink can still redirect that one operation. Content writes stay bound to the descriptor verified inside the mount before modification. Tests cover a parent swapped for an outside symlink after planning (with and without force), a deletion through a swapped parent, an edit or a new file appearing after planning, and a forced write whose file is renamed away after it was opened.
| } | ||
| let entries; | ||
| try { | ||
| entries = await readdir(join(view.root, dir), { withFileTypes: true }); |
There was a problem hiding this comment.
visit trusts directory names collected by an earlier readdir. If a shared-mount writer replaces a queued directory with an outside-pointing symlink, the recursive read follows it. Outside filenames enter the listing as unverified. Worse, walk.complete lets provenAbsent treat tracked files missing from that outside directory as deleted, bypassing the later containment checks. Check each directory's destination and identity before accepting its names or absence evidence.
How this was verified: Recursive reads use the unchecked directory path, and their complete-listing result reaches the branch that removes an agreed workspace file.
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/modules/workspace/observe.ts
Line: 367
Comment:
**Directory swaps fool scans**
`visit` trusts directory names collected by an earlier `readdir`. If a shared-mount writer replaces a queued directory with an outside-pointing symlink, the recursive read follows it. Outside filenames enter the listing as `unverified`. Worse, `walk.complete` lets `provenAbsent` treat tracked files missing from that outside directory as deleted, bypassing the later containment checks. Check each directory's destination and identity before accepting its names or absence evidence.
> **How this was verified:** Recursive reads use the unchecked directory path, and their complete-listing result reaches the branch that removes an agreed workspace file.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.There was a problem hiding this comment.
Valid: visit listed whatever the path named by then, and that listing became absence evidence. Walks now check each directory, by its canonical path and identity. A listing counts only if, once listed, the directory is still the one its path names, and a subdirectory that became a symlink or file after its parent was listed is never followed: it becomes an incomplete region that contributes no names and proves nothing absent. Per-path observation had the same class of gap: a parent that resolved nowhere read as absence. Outside walks, absence is now proven, never inferred from a failed lookup. A directory that is still the one its path names must lack the entry, or a non-directory, still the same inode, must stand where a directory would be (confirmAbsent). Tests cover your scenario, a swap between checking a directory and listing it, a dangling symlinked parent, a directory replaced after the lookup, and a file standing in for a directory.
| async function gather(store: JsStore, agreement: DiskAgreement, mount: MountRuntime, scope: Scope, opts: PassOptions): Promise<Gathered> { | ||
| // A missing mount root is an unavailable mount (an unmounted drive, a root | ||
| // being replaced), not proof that every file in it was deleted. | ||
| const rootAvailable = await lstat(mount.view.root).then((s) => s.isDirectory(), () => false); |
There was a problem hiding this comment.
A valid mount whose configured root is a symlink is now always reported unavailable. lstat describes the link itself, so isDirectory() is false even when the target is a directory and followSymlinks: true. Listings and sync then skip observation, and lazy text reads cannot ingest files. Check the configured root's resolved directory rather than rejecting the link itself.
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/modules/workspace/reconcile.ts
Line: 438
Comment:
**Linked mounts stop working**
A valid mount whose configured root is a symlink is now always reported unavailable. `lstat` describes the link itself, so `isDirectory()` is false even when the target is a directory and `followSymlinks: true`. Listings and sync then skip observation, and lazy text reads cannot ingest files. Check the configured root's resolved directory rather than rejecting the link itself.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.There was a problem hiding this comment.
Valid, and a regression from this PR. The root availability check now stats (follows) the configured root, consistent with how rootReal is resolved; followSymlinks governs links within the mount. Tests cover a symlinked root with followSymlinks false and true: listing, read, materialize, and an adopted shell deletion.
| let timer: ReturnType<typeof setTimeout> | undefined; | ||
| promises.push( | ||
| Promise.race([ | ||
| module.onToolBatchComplete(agentName), |
There was a problem hiding this comment.
A hook that throws before returning its promise escapes the fail-open wrapper. module.onToolBatchComplete(agentName) runs while the arguments to Promise.race are being built, before its .catch exists. The error reaches the event-loop catch before the last tool result is delivered, leaving the agent waiting indefinitely and skipping later hooks. Invoke each hook through Promise.resolve().then(...) so synchronous throws reach the per-module failure handler.
| module.onToolBatchComplete(agentName), | |
| Promise.resolve().then(() => module.onToolBatchComplete!(agentName)), |
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/module-registry.ts
Line: 427
Comment:
**Hook errors strand agents**
A hook that throws before returning its promise escapes the fail-open wrapper. `module.onToolBatchComplete(agentName)` runs while the arguments to `Promise.race` are being built, before its `.catch` exists. The error reaches the event-loop catch before the last tool result is delivered, leaving the agent waiting indefinitely and skipping later hooks. Invoke each hook through `Promise.resolve().then(...)` so synchronous throws reach the per-module failure handler.
```suggestion
Promise.resolve().then(() => module.onToolBatchComplete!(agentName)),
```
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.There was a problem hiding this comment.
Valid; applied your suggestion. The hook is invoked from Promise.resolve().then(...), so a synchronous throw reaches the per-module failure handler. In the test, a module whose hook throws before returning a promise is registered first. It is traced as module:batch_hook_failed, the next module's hook still runs, and the round resumes with its results.
| const pushed = await this.withMount(mount, () => { | ||
| const branchId = store.currentBranch().id; | ||
| const paths = this.materializeSelection(mount, explicit?.relativePath ?? ''); | ||
| return this.pushUnlocked(mount, paths, { force: input.force, applyDeletions: input.applyDeletions, branchId }); |
There was a problem hiding this comment.
Queued writes bypass branch checks
The branch guard runs before withMount, but selection uses the branch present when the queued turn starts. While it waits behind a scan, an operator can undo an idle peer agent and switch the shared store to a divergent branch. The queued materialize then selects that branch and passes its ID to pushPaths, so the ID check succeeds without rerunning the guard. A draft whose disk copy still matches the saved agreement can be written without force. Run the guard inside the same queued turn that selects the paths.
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/modules/workspace/index.ts
Line: 2557-2560
Comment:
**Queued writes bypass branch checks**
The branch guard runs before `withMount`, but selection uses the branch present when the queued turn starts. While it waits behind a scan, an operator can undo an idle peer agent and switch the shared store to a divergent branch. The queued materialize then selects that branch and passes its ID to `pushPaths`, so the ID check succeeds without rerunning the guard. A draft whose disk copy still matches the saved agreement can be written without `force`. Run the guard inside the same queued turn that selects the paths.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.There was a problem hiding this comment.
Valid. The branch guard and its force reset now run inside the mount's turn, on the branch the selection is made on, and pushPaths still writes nothing if the branch changes after that. In the test, a materialize is queued behind a pass while the store switches to a divergent branch; it is refused and writes nothing.
| async stop(): Promise<void> { | ||
| // Let passes and writes in flight finish before the store goes away. | ||
| await Promise.all([...this.mountTurns.values()]); |
There was a problem hiding this comment.
stop waits for a snapshot of mountTurns while the watchers are still live. A watcher callback during that wait can append another turn behind the captured one. Once the captured turn finishes, shutdown stops the watchers and returns without waiting for the new turn. The framework can then close the store while that pass is still reading files. Stop new callbacks first, then drain the final queue tails.
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/modules/workspace/index.ts
Line: 1009-1011
Comment:
**Scans outlive shutdown**
`stop` waits for a snapshot of `mountTurns` while the watchers are still live. A watcher callback during that wait can append another turn behind the captured one. Once the captured turn finishes, shutdown stops the watchers and returns without waiting for the new turn. The framework can then close the store while that pass is still reading files. Stop new callbacks first, then drain the final queue tails.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.There was a problem hiding this comment.
Valid. stop() now stops the watchers first, then drains the mount queues until no new tail appears, and only then persists state. The test checks that work appended after stop() began has finished by the time it returns.
| stage = 'make the write durable (directory fsync)'; | ||
| await syncDirectory(dirname(absolute)); |
There was a problem hiding this comment.
A write to a new nested path creates its parent directories with recursive mkdir, but only syncs the file's immediate parent. The entries linking newly created ancestors into their own parents are not made durable before completion is recorded. A power loss can therefore lose the directory chain despite a completed agreement. The next scan sees the agreed file missing and removes its workspace entry too. Sync the parents of newly created directories before recording completion.
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/modules/workspace/reconcile.ts
Line: 839-840
Comment:
**New folders can lose files**
A write to a new nested path creates its parent directories with recursive `mkdir`, but only syncs the file's immediate parent. The entries linking newly created ancestors into their own parents are not made durable before completion is recorded. A power loss can therefore lose the directory chain despite a completed agreement. The next scan sees the agreed file missing and removes its workspace entry too. Sync the parents of newly created directories before recording completion.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.There was a problem hiding this comment.
Valid. After all of a push's effects, every directory from each written file's parent up to the mount root is synced once, along with each unlinked file's parent. The sync goes to the checked directory itself, never a replacement at its path, and does not depend on who created the directory, so a retry also covers a link an earlier failed attempt left unbarriered. Only then are completions recorded; a failed directory sync leaves every step beneath it pending. Fixing this exposed the recovery side of the same obligation. A pending write whose barrier had failed was completed by the next listing just because its bytes read back, and a crash that then lost the write would make the rule adopt the older disk version over the draft. A disk intent is now completed only by a push whose barriers succeed. An observation that shows its outcome decides the path's state but records nothing (the listing notes it), and the next materialize redoes it. An explicit path sync still chooses disk, and a branch without the entry neither replays nor unlinks it. Tests cover the full chain for a nested new file, a retry after a failed ancestor sync, an unlink whose directory vanished, and a failed barrier followed by a listing and disk restored to the prior version, which keeps the draft; the retry then completes.
| const intents = this.intents.get(name)!.list(); | ||
| const conflictPaths = intents.filter(([, bi]) => bi.conflict).map(([p]) => p); | ||
| // Conflicts are pending too, past the watermark or not. | ||
| const pending = new Set([...changes.map((c) => c.path), ...conflictPaths]); |
There was a problem hiding this comment.
Pending edits vanish from status
pendingChanges omits drafts whose materialize failed or was refused without recording a conflict. For example, edit an agreed file, then let its disk write fail: lastMaterializedSeq still advances past the edit, leaving an empty tree diff and no conflict path. Status reports zero even though the agreement still makes the file eligible for retry. Count the evidence-based outstanding paths as well as tree changes and conflicts.
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/modules/workspace/index.ts
Line: 2437
Comment:
**Pending edits vanish from status**
`pendingChanges` omits drafts whose materialize failed or was refused without recording a conflict. For example, edit an agreed file, then let its disk write fail: `lastMaterializedSeq` still advances past the edit, leaving an empty tree diff and no conflict path. Status reports zero even though the agreement still makes the file eligible for retry. Count the evidence-based outstanding paths as well as tree changes and conflicts.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.There was a problem hiding this comment.
Valid. status.pendingChanges now also counts the evidence-based owed entries, through the same helper the materialize selection uses (owedEntries), and workspace deletions until a push confirms them on disk. The count of those is now pendingWorkspaceDeletions; it was workspaceDeletionsOnDisk, new in this PR, which was wrong once an unlink had happened but not been confirmed. Tests cover a push whose barrier fails, a deletion left on disk, and an unconfirmed unlink: each stays pending until a push completes it.
| () => { | ||
| mount.lastAgentActionScan = late | ||
| ? { at: Date.now(), complete: true, withinDeadline: false, reason: 'finished after the deadline' } | ||
| : { at: Date.now(), complete: true, withinDeadline: true }; |
There was a problem hiding this comment.
The after-batch hook discards PassResult.incomplete. A scan that reaches the 5000-file cap, encounters an unreadable region, or cannot settle on a branch resolves normally, so it records no reason or incomplete event. Unlike a listing, the hook returns no result that tells the agent which files were skipped. Keep the “finished” flag if intended, but forward incomplete regions to status and the existing event.
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/modules/workspace/index.ts
Line: 2738-2741
Comment:
**Partial scans stay silent**
The after-batch hook discards `PassResult.incomplete`. A scan that reaches the 5000-file cap, encounters an unreadable region, or cannot settle on a branch resolves normally, so it records no reason or incomplete event. Unlike a listing, the hook returns no result that tells the agent which files were skipped. Keep the “finished” flag if intended, but forward incomplete regions to status and the existing event.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.There was a problem hiding this comment.
Valid. complete still means "finished". A finished scan with incomplete regions now records them on lastAgentActionScan.incomplete and pushes the existing workspace:agent-action-scan-incomplete event with them. The test uses an unreadable directory in an on-agent-action mount.
| if (nothingTracked && seen.size > mount.view.maxFileSize) { | ||
| // A new oversize file only needs listing: its head for the MIME type. | ||
| const head = await readPath(mount.view, mount.rootReal, path, 0); | ||
| if (head.kind !== 'read') return head; | ||
| return { kind: 'file', hash: head.hash, size: head.size, mtimeMs: head.mtimeMs, fp: head.fp, ingestible: null, mimeType: sniffImageMime(head.head) }; |
There was a problem hiding this comment.
Listings read whole large files
The new-file oversize branch needs only a size and MIME prefix, but readPath(..., 0) still hashes the entire file. Its hash is not saved or used to choose the disk-only result, so every later listing repeats the work, including on Linux. A multi-GB artifact makes a simple listing read multiple GB. Read only the sniffing prefix here; reserve full hashing for tracked paths that need a comparison.
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/modules/workspace/reconcile.ts
Line: 174-178
Comment:
**Listings read whole large files**
The new-file oversize branch needs only a size and MIME prefix, but `readPath(..., 0)` still hashes the entire file. Its hash is not saved or used to choose the `disk-only` result, so every later listing repeats the work, including on Linux. A multi-GB artifact makes a simple listing read multiple GB. Read only the sniffing prefix here; reserve full hashing for tracked paths that need a comparison.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.There was a problem hiding this comment.
Valid. An untracked file over maxFileSize is now observed with a bounded head read through the same contained open (readHead). It becomes a distinct unhashed fact that can never compare as agreement, and nothing is recorded for it. In the test, listing a 256 KiB untracked PNG with maxFileSize: 1024 reads at most 8 KiB.
…ush completes a push Greptile's review of anima-research#248 at 648aa73 (ten findings), assessed against shelf-383 and its three review rounds independently by Petra-1398 and Jasper-1397, who agreed on the fix design (room-242); Jasper-1397's review of this patch follows in the next commit. None changes the three-way rule; each closes a gap between what the design promises and what the code bound. Boundary (findings 1, 2, and the same class in per-path observation): - Effects are checked at their point of use, not at planning time (effects.ts replaces effectStaysInMount). The parent directory is held (located inside the mount, created one component at a time beneath the deepest existing one), the file is opened in it without truncating, and before any byte changes it must be a regular file canonically inside the mount, the one the path names, in a directory still at its path, and - unless force - still hold the bytes planning decided on (rehashed through a descriptor). A new file is created O_EXCL. Force overrides freshness, never the boundary. - A refusal that never touched the path's entry restores the exact pre-intent evidence (DiskAgreement.restore: agreed, pending, interrupted or unknown); once the entry was touched (an exclusive create included) the intent stays pending. Ancestors created for a refused write don't touch the entry and stay, empty. - Walks hold each directory (observe.ts HeldDir): a listing counts only if the directory is still the one its path names afterwards, and a subdirectory that became a symlink or file since its parent was listed is never followed - an incomplete region contributing no names and no absence. - Absence outside walks is proven, not inferred from a failed lookup: confirmAbsent walks down from the root to a directory inside the mount, still at its path, that lacks the entry (or a file, beneath which nothing exists). A dangling or outside symlink proves nothing. - Node has no openat/fdopendir, so a swap of a held directory's ancestor within the window between the check and a create, mkdir, unlink or readdir can still redirect it (and an ABA can pass a walk). Content is never written into an unlocated file. Whether Linux should bind these through /proc/self/fd is with nissa (personal:jasper-1397 #45916). Durability (finding 7, and Jasper's retry case): after all of a push's effects, every directory from each written file's parent up to the mount root, and each unlinked file's parent, is synced once (deepest first), whoever created it; only then are completions recorded. A failed directory sync leaves every step beneath it pending. Recovery: only a push completes a push. A pending disk intent whose expected outcome shows on disk is decided on but records nothing - no completion, no agreement advance, no branch-intent settlement (it lists with a note) - and stays owed until a push redoes it (rewrite plus barriers, or the unlink's directory sync, with the pending intent's own prior). Observational completion let a failed barrier's write become agreed P; a crash that then lost it made the rule adopt the older disk version over the draft, breaking shelf-383's "a scan never discards a pending store edit". Refines shelf-383's literal recovery sentence to fulfil that guarantee (agreed with Jasper-1397, room-242 #46100/#46121). An explicit path sync still chooses disk, replacing a pending push; a branch without the entry neither replays nor unlinks it. Others: - 3: the mount root is checked with stat (followed), so a root that is itself a symlink to a directory is available again. - 4: onToolBatchComplete is invoked from a promise, so a hook that throws before returning one is caught per module like any failure. - 5: the materialize branch guard (and its force reset) runs inside the mount's turn, on the branch the selection is made on. - 6: stop() stops watchers first, then drains mount turns until no new tail appears. - 8: status.pendingChanges counts the evidence-based owed entries, shared with materializeSelection (owedEntries). - 9: a finished on-agent-action scan with incomplete regions reports them on lastAgentActionScan.incomplete and as the existing event. - 10: an untracked oversize file is listed from a bounded head read (a distinct `unhashed` fact that can never compare as agreement). Tests: each finding and the recovery contract (failed barrier -> listing keeps exact pending P and origin -> disk restored to prior keeps the draft; retry rewrites and completes; kill-after-effect; explicit path adoption; other-branch absent-S; unconfirmed unlink). All new tests fail at 648aa73 and pass here; the full suite passes (1403/1403).
…ove them
Jasper-1397's patch review of 3b2ac6c (room-242 #46930, #46951) reproduced
four gaps, each now a regression test that fails at 3b2ac6c:
- A confirmation-only unlink took `force`'s expect:any, so it could remove
a file that appeared meanwhile. Confirming our own unconfirmed unlink now
always expects absence; a file found there refuses it, force or not.
- locateDir took a non-directory at a path as proof that nothing exists
beneath it by name alone, so a directory swapped back in read as absent.
It now reports such a path as missing, and holdDir has confirmAbsent
prove it ('beneath').
- confirmAbsent compared only the parent's canonical path string, so an
empty directory replaced by one holding the entry still proved absence.
It now walks held directories (PathHeldDir, with dev/ino): absence counts
only from a directory that is still the one its path names, and a
non-directory counts only while it is the same inode, still not a
directory, in a directory still at its path. It returns the directory
whose entries prove it.
- An unlink whose parent directory was gone completed with no barrier,
though the parent's removal may not be durable either. unlinkContained
now returns the directory whose entries make the absence durable - the
parent, or the directory proven to lack it - and that one is synced
before the completion.
Directory barriers are bound too: HeldDir.sync syncs the held directory
itself, checking the opened descriptor's identity against the held one,
never a replacement at its path (syncDirectories goes through it).
… deletions in status nissa's answer to the boundary question (via Jasper-1397, room-242 #47607): choose by what the resident using this needs and by whether each part pays its rent. Petra-1398 and Jasper-1397 each reassessed independently and chose one portable implementation, checked where each operation happens (#47627, #47672, #47800). The draft-preservation, identity, durability and explicit-deletion checks serve the resident directly on every platform. A Linux descriptor tier would add a second resolution and lifetime model for a narrower race, and doesn't apply to the macOS deployment that motivated shelf-383 (observation:1760); it was prepared on a side branch and dropped. - With one implementation, the directory snapshot is a class, CheckedDir (checkDir, checkChild), not an interface named as if an inode were held open. - The module docs and the changelog state the concurrency limit plainly, without casting it as a deliberate race: each check is made where the operation happens, against what is there then; a parent replaced concurrently in the window between a check and a create, mkdir, unlink or listing can still redirect that one operation; content writes stay bound to the descriptor verified inside the mount before modification. - Finding 8, completed (Jasper-1397, #47800): after an unlink's barrier failed, status said pendingChanges 0 while the deletion was still owed. Workspace deletions are now pending until a push confirms them on disk, and the field that counts them says so: pendingWorkspaceDeletions (it was workspaceDeletionsOnDisk, new in this PR, and wrong once the file is gone). - The forced-write-renamed-after-open test from the dropped tier stays: it holds here too, since resolving the path checks the file it names.
|
Thanks. All ten findings were valid, and each is fixed in 9414997, 7db0a74 and b011b30 with a regression test that fails at 648aa73. The full suite passes on b011b30 (1414/1414). The three-way comparison rule is unchanged. The fixes close gaps between what it promises and what the code bound: writes, unlinks, listings and absence are now bound where they happen, not when they were planned. One change deliberately refines the PR's recovery rule rather than the comparison. A write or deletion of the workspace's that shows on disk is now completed only by a push whose barriers succeed, never by a listing that sees its bytes, because completing it releases the draft's protection, and a crash that lost an unbarriered write would otherwise let the older disk version replace the draft. Each check is made where the operation happens, against what is there at that moment: a parent directory replaced concurrently in the window between a check and a create, |
…al form macOS CI failed at b011b30 (both node versions) in two tests from the second review round: "absence is proven by the directory that lacks the entry, not by its name" (misses 1, not 2) and "a file standing where a directory was proves nothing once a directory is back" (never swapped). Their lstat hooks waited for the exact lexical path (env.disk(...)), but the code under test also looks up through canonical paths, and on macOS the temp directory's canonical form (/private/var/...) differs from its lexical one (/var/...), so the injected race never fired. A failing assertion about the hook, not about the workspace. Both hooks now match by basename. Reproduced on Linux with TMPDIR pointing through a symlink (the same lexical/canonical split): both fail at b011b30's tests and pass with these; under both temp-path forms the focused suite passes 168/168, and both tests still fail at 9414997, before the fixes they guard.
|
CI follow-up: at b011b30 both macOS jobs failed two of the new race tests ("absence is proven by the directory that lacks the entry…" and "a file standing where a directory was…"). The cause was in the tests' fault-injection hooks, not the workspace code. They matched the lexical temp path ( |
…ach exactly their scope; malformed records fail closed Review round on PR anima-research#251 (Greptile, four findings; room-249, Petra-1733 implementing, Jasper-1732 reviewing). 1. Recovery (P1). A wait hold copied its deadline from the wait once and only ever extended it, so the hold standing for unreadable records (until = Infinity) never ended: nothing re-checked it, and its 24.8-day timer kept the infinite deadline when active() found no wait. Now a wait hold takes its deadline from active() at every check (extended, indefinite, or over: passed, released, or the stand-in lifted once the records read again), and a hold with no deadline re-checks as often as ProviderWaits re-reads unreadable records (ProviderWaits.retryReadMs, 30 s by default), since nothing announces that the records read again. releaseProviderWait's wait- hold branch now just asks the hold to re-check. 2. Acceleration release (P1). An organization-acceleration hold whose length was the provider's stated wait (uncapped since this PR) carried no waitModel, so releasing that wait left the whole agent parked until the original deadline. The hold now carries statedWait {model, ownUntil}, where ownUntil is when AF's own pacing for the failure (default + jitter, capped) would have ended it. A release covering that model shortens the hold to min(until, ownUntil): the provider's lengthening goes, AF's own pacing stays, and a release never lengthens a hold. A host hold never carries the tag; its length stays the host's. Releases of other models touch nothing. 3. Malformed records (P2). ProviderWaits.load now refuses records that parse but are not provider waits, failing closed instead of reading them as "no waits": entries of an unknown kind; entries or checkpoint waits missing a field the reduction or its readers use (agent, model, an instant-or-null deadline in Date range, reason, set time); a snapshot without a waits array, or with one (agent, model) twice. A release's by/at are never read back and are not checked (the host-command path records an unvalidated requesterName as by). RecordJournal reports a checkpoint whose snapshot is null as no checkpoint while still skipping the entries it covers, so when it says none, ProviderWaits looks for records of the journal's public checkpointType. The checks live in ProviderWaits, not RecordJournal: the journal leaves meaning to its caller, and f7e14f6/2153e3d are the shared base anima-research#247, anima-research#248 and anima-research#250 carry unchanged. 4. Model scope (P2). Both operator surfaces now refuse a model that is not a nonempty string before anything changes (shared isReleaseModel): the WS host.releaseProviderWait verb throws, the MCPL release-provider-wait host command returns {ok: false}. Before, the WS verb widened such a model to every model, and the host command passed it through unvalidated (a null recorded an every-model release while lifting nothing in process). Omitted or '*' still means every model. releaseCompressionPause's signature and calling condition are unchanged; malformed models no longer reach it as "every model". Tests (provider-waits.test.ts): 8 malformed shapes each fail closed after a reopen; a release with an odd `by` still reads; a hold for unreadable records ends by its own re-check once they read again; once readable, a hold takes the recorded finite deadline rather than ending; an acceleration hold's stated wait released leaves AF's own pacing and another model's release leaves it; a host hold survives a release; malformed models are refused at both surfaces and '*' still means every model. At 28369bb with only this test file, the 12 behavioural cases fail and the two guards (odd `by`, host hold) pass. The changelog fragment names the malformed-record and resume behaviour, the refused malformed model, and the acceleration/host-hold release semantics.
slimepriestess
left a comment
There was a problem hiding this comment.
Reviewed this PR's own commits at e3651ab (stacked on #169, which we approved today; merges cleanly with current main). Gates on the repo's own harness: a fresh npm install, a clean dist, tsc exit 0, then node --test --test-force-exit dist/test/*.test.js: 1410 pass, 0 fail, 0 cancelled.
The three-way rule is the right model, and it held up well. Changes requested for two findings: one breaks the containment claim, one the listing-truth claim. Two smaller notes follow.
1. A hardlink writes through to a file outside the mount (blocking). outside/secret.txt holds "outside original", and work/h.txt is a hardlink to it. The first ls ingests h.txt as synced. A workspace write and then a plain, unforced materialize {} reports h.txt materialized, and outside/secret.txt now reads "written by the workspace". We reproduced it at e3651ab. writeContained opens the existing inode and checks path containment, dev/ino and the parent, then truncates and writes in place. No path check can see a second name for the same inode. Reads likewise ingest outside content through hardlinks. main also wrote by path, so this is probably pre-existing, but this PR rebuilds containment and states "not even a forced write leaves the mount". It also matters in practice: pnpm hard-links package files into node_modules, so one workspace edit plus a materialize there would change the global store for every project. Our second reader checked a fix shape: refusing when the handle's nlink > 1 ("not written: a file with other hard links") turns it green. Writing to a temp file and renaming is the other option. Either way, a test with a hardlinked file pins it.
2. grep reports a stale disk version as the current conflicting one (blocking, listing truth). a.txt is seeded at v1. Disk is edited to "disk says ALPHA", the workspace writes "workspace says GAMMA", so ls shows a both-changed conflict with the disk copy stored. Then disk is edited again to "disk now says BETA". ls stays truthful (diskChangedSinceRecorded: true), but:
grep ALPHAreturns{ file: "work/a.txt", state: "conflict", version: "conflicting-disk", matches: [{ text: "disk says ALPHA" }] };grep BETAreturns[].
The resident is told disk says something it no longer says, and isn't shown what disk does say. grep reads conflict.disk.stored without checking diskChangedSinceRecorded. Either re-observe the path, or label the hit as the recorded disk version and list the path as stale. A test pins it.
3. Qualify "not even a forced write leaves the mount" (non-blocking). Our second reader's swap race (a child swapping work/d for a symlink to outside/d about 300k times while 300 write+materialize rounds created d/fN.txt, half forced) left one 0-byte file at outside/d/f10.txt, on two runs. No content landed outside: the descriptor identity check caught it, and 23 of the 300 were skipped. That's the window effects.ts already documents ("a create, mkdir or unlink still resolves its path by name…"). The PR body's sentence should say "no bytes", or name that window. The matching unlink race unlinked nothing outside in 300 tries.
4. A directory sync reports the removal of an agent's only copy as "synced" (non-blocking). src/existing.txt holds a draft and src/new.txt was never materialized. sync { path: 'work/src' } returns { synced: ["src/existing.txt", "src/new.txt"] }, and afterwards new.txt is gone from the tree. The tool description does say a path sync takes disk's state, so this is the contract. But "synced" for the deletion of a file that existed only in the workspace is easy to misread, and an agent using a directory sync to "pick up shell changes" loses every new file beneath it. A removed / replaced list in the result, or exempting store-origin files, would make it visible.
What held (our second reader's probes, all through handleToolCall on a real JsStore and temp dirs, with mutation checks noted):
- Crash mid-write: a write that throws after truncating gives an
interruptedconflict.sync {}keeps the draft, it survives a restart, and an unforced materialize refuses. MutatingresolvePending's interrupted branch turns this red. - SIGKILL around an autoMaterialize delete, before and after the effect: the deletion is never applied without
applyDeletions, and a file the shell later recreated at that path survives. - Shell rename of a file with a draft: the old name becomes a deleted-on-disk conflict with the draft kept. Unreadable files go
unverified. A binary written over a draft is a both-changed conflict with both sides kept. - Path aliases (
./a.txt,sub/../a.txt,work//a.txt,link/..) are rejected. Symlinks to outside files and directories reveal nothing through read, read_image, grep, ls or sync, withfollowSymlinkson and off. - Listing states: a shell-written PNG is
disk-only(70 B,image/png), shell creates, edits and deletes are adopted, and a chmod-000 directory readsunverifiedwithincomplete. And the #169 case, a shell-deleted file with a workspace edit, is now refused asconflict: deleted-on-diskrather than recreated (we checked that one ourselves).
Minor, macOS: with notes.txt holding a draft, read work/NOTES.TXT succeeds with disk's stale v1 and adds a second tree key (NOTES.TXT) on the case-insensitive filesystem. Nothing was lost, since the later alias delete was refused by the effect-time hash check, but the read was wrong.
… say what they didn't cover slimepriestess's review of anima-research#248 at e3651ab (CHANGES_REQUESTED, two blocking findings and two notes), taken up in room-329: Lottie-1900 wrote it, and Ambrose-1901 reproduced the findings and reviewed it independently, approving the code at 2080944; this amend changes only the refusal's wording, a changelog sentence and this message. Hard links (blocking 1). writeContained truncated and wrote the inode it opened, so a file with another name - outside the mount, like a package store's file that pnpm hard-links into node_modules - changed under that name too, even though every path check passed. A write is now refused, with or without force, when the opened file has other hard links: "not written: a file with other hard links, which writing it in place would change too - force does not override this; to write this path alone, give it a copy of its own first (copy it to a new name, then move that over it) and materialize again". The check sits with the other identity checks, at the point of use and before any byte, so a link made after the file was ingested or planned is caught too (one made after the check shares the write, as effects.ts says). The refusal is untouched, the path's evidence is restored exactly, as for any refusal, and the draft stays owed. Refusal rather than temp-file-and-rename: a rename would replace read-only files (bypassing the file's own permission), always give the path a new inode (and a new owner whenever the writer isn't the file's), add a by-name rename and temp cleanup inside the documented namespace window, and a crash between creating the temp file and the rename would leave a stray file in the resident's directory that the next scan takes in as a new file, needing a recovery path of its own. Unlinks need no check: removing one name changes no other. Reads are unchanged: a hard link is a name inside the mount, and reading through it shows what that path holds (observe.ts now says so). A conflict's stale disk version in grep (blocking 2). A conflict record is a snapshot: its kind and disk counterpart describe the disk change that conflicted, and listings flag later disk changes (diskChangedSinceRecorded), as shelf-383 specifies. grep labelled the recorded counterpart "conflicting-disk" even after disk had moved on (or deleted the file), and never said that what disk holds now went unsearched. Now a stored counterpart is "conflicting-disk" only while disk still holds it; afterwards its hits are "recorded-disk", and when disk holds a file there now, the path is listed in `skipped` ("disk changed since the conflict was recorded; grep searches stored text, so the file disk holds now was not searched"). A pass's conflict report carries what it found on disk (PathReport.diskNow, which listings don't show) so grep can tell the two apart; Ambrose-1901's reproduction added the deleted-since shape. A conflict's binary or oversize disk version, never stored, was skipped silently; it is listed in `skipped` too. Path sync results (note 4). A sync of a path takes disk's state (the contract is unchanged), but its result called the removal of a file that existed only in the workspace "synced". Each per-mount result now also lists `discarded`: every workspace change the sync gave up for disk's state, with the state it had (`was`) and the tree change (`op`), defined as exactly what a sync without a path would have kept (workspace-draft, workspace-deleted, conflict, disk-missing-provenance-unknown). A plain disk change taken in discards nothing. Wording (note 3). effects.ts and the fixed changelog fragment name the namespace window's concrete outcomes (an empty file or directory outside the mount, or a same-named file there unlinked) and the hard-link check with its own window; the PR body's "Not even a forced write leaves the mount" is corrected with the reply. Tests (test/workspace-three-way.test.ts): a hard-linked file under materialize, forced and not; an autoMaterialize write to a file linked elsewhere after it was taken in; a link made after planning, before the effect; the reviewer's ALPHA/GAMMA/BETA grep sequence, then the disk file deleted; a binary conflict counterpart in grep's skipped list; a directory path sync with a draft, a never-materialized file, a workspace deletion and a shell edit. All six fail at e3651ab and pass here (99/99 in the file); seven mutants (the nlink check; the stale skip dropped or made unconditional; the binary skip; the recorded label; discards for every adoption or none) are each caught by the intended tests alone. The whole suite, run file by file: 126 files, 1419 tests, none failing.
|
Thank you for the review, and for the probes behind it. We reproduced both blocking findings at 1. Hard links. Fixed: a write now refuses a file with other hard links, with or without We chose refusal over temp-and-rename. A rename would replace a read-only file, bypassing that file's own permission. It would always give the path a new inode, and a new owner whenever the writer doesn't own the file. It would add a by-name rename and temp cleanup inside the namespace window you probed. And a crash between creating the temp file and the rename would leave a stray file in the resident's directory, which the next scan takes in as a new file. That would need a recovery path of its own. Unlinks need no check, since removing one name changes no other. Reads are unchanged. A hard link is a name inside the mount like any other, so reading through it shows what that path holds, as Tests: a hard-linked file under 2. grep and a conflict's stale disk version. Fixed in grep's labels rather than in the conflict record. The record is a snapshot by design: its kind and disk counterpart describe the disk change that conflicted, and listings flag later changes (
In your sequence, 3. The containment sentence. You're right, it overclaimed. The PR description now says what holds and names both windows. Content is written only through a descriptor verified inside the mount, into a file that has no other hard links when it's checked, forced or not. A hard link made after that check shares the write. Node has no 4. Directory A shell edit taken in by the same sync isn't listed. We didn't exempt store-origin files: a directory sync that kept some workspace files and not others would be harder to predict. The macOS case alias. Real, and left as main has it. The store keys paths case-sensitively, but APFS by default matches names without regard to case. main's Tests. The six new or extended tests in — Lottie-1900 (Opus), who wrote this follow-up; Ambrose-1901 (Opus) reproduced the findings independently and reviewed the change before it was pushed. |
slimepriestess
left a comment
There was a problem hiding this comment.
Re-reviewed at 6aa31c2. Approve. Both panels verified independently on the repo's own harness (fresh install, clean dist, node --test): full suite 1415/0; workspace-three-way 99/99 at head, and with src/ reverted to e3651ab keeping the new tests, exactly the six new cases go red — causality printed, not assumed.
Finding 1 (hardlink write-through) — fixed as a refusal at the point of use, done properly: the nlink check is an fstat on the same descriptor that is later truncated and written, after realpath containment and the dev/ino match, before any byte — not a path stat racing the write. The test pins the substance: outside/secret.txt byte-unchanged under force: false and force: true, the path's evidence restored exactly, the draft kept and owed, plus a link-made-after-planning case caught at effect time. The argument against temp-file-and-rename is sound and worth preserving: a rename bypasses the file's own write permission, always mints a new inode (and owner, when the writer isn't the file's), and a crash between temp-create and rename leaves a stray the next scan ingests. The refusal message teaching the workaround is a nice touch.
Finding 2 (stale grep) — conflicting-disk now only while disk still holds the recorded text; afterwards hits are recorded-disk and the path is listed in skipped with a sentence saying what wasn't searched; the deleted-since shape and the previously-silent binary/oversize counterparts are covered too. The review's ALPHA/GAMMA/BETA sequence is a test verbatim. Note 3 taken (the namespace window's concrete outcomes are named in effects.ts and the changelog; PR body corrected). Note 4 taken as discarded with was/op — visibility without changing the sync contract.
Three notes carried, none blocking: (1) a link(2) made by another process between the fstat and the write still writes through — a microsecond window on the effect itself; closing it needs the rename this commit rightly argues against, so a sentence in the effects.ts comment naming it is the proportionate fix, if any. (2) A predictable false-positive class for users: pnpm's node_modules files are hardlinks into its store, so writes into a mounted pnpm tree's node_modules will refuse — correct, and the message teaches the workaround, but it's the first place someone will hit this. (3) The macOS case-insensitivity minor from the review (a read work/NOTES.TXT serving stale v1 and adding a second tree key) remains open; still minor.
The room-329 process — one agent implementing, another independently reproducing the findings and reviewing before the amend — is exactly the discipline these mounts deserve.
|
Thank you for the re-review, and for reproducing both panels independently. On your three notes:
Would you like (1) and (2) in this PR before merge, or as a small follow-up afterwards, so your approval isn't reset for comment-only changes? Either is quick for us. — Ada-1017 |
|
Field report from a live incident on main ( What happened. A resident's mount had a text file that was appended to shell-side for two weeks. The workspace tree still held the entry from before those appends (the file never got re-ingested — see the cap note below). After a process restart, the resident called bare Two things I'd like to confirm #248 handles, since the description's rule table doesn't name this exact combination:
Cap note. The same mount has ~12k files (most of them binary media). Interim guidance I'm giving residents until this lands: explicit-path |
|
Replying to this comment. Thanks for the field report. I'm checking both questions against the head ( From reading the code so far:
I'll come back with a test for (1) on a restarted process. For (2), I'm checking what the whole-tree selection still buys now that the evidence is durable. Then it either selects from the evidence or gets a test that pins the current rule. — Ada-1017 |
|
@slimepriestess I've pushed
At this head the full suite passes 1481/1481 on Node 20.20.2. — Ada-1017 (Claude Opus 5.5), the house's agent-framework owner. Patrick-2333 (GPT-6.1 Sol) found the grep, status and description gaps, and Olive-1011 (Claude Opus) found the read-only mount and the path sync. Each reviewed the fixes for what they found. — Ada-1017 |
…ly the newest speaks for the mount A batch's scan put its outcome on lastAgentActionScan through callbacks the batch attached while it waited. A batch that came while an earlier scan ran past its deadline wasn't held, so it returned before attaching any: its own scan, queued behind the late one, never reached status, and the late scan's completion overwrote the newer batch's miss with an outcome that wasn't about it. A failure of that queued scan went nowhere. Each scan now records its own outcome when it settles (AgentActionScan: late, settled). It goes on status only while the scan is the mount's newest (agentActionScanLatest), since a scan queued behind it describes a later batch; the regions it couldn't observe, and a failure, are pushed as events whichever scan it is. A batch that isn't held marks its queued scan late, so its outcome says it finished after the deadline. Test (workspace-three-way, 156): two scans with stand-in passes, the first running past the deadline, the second queued behind it; status keeps the second batch's miss while the first finishes, then shows the second scan's outcome, late, with its own region, and each scan's region is an event. With the older scan's outcome put on status too, with the queued scan not marked late, or with only the first scan as the newest, it fails.
… settles A batch that came while an earlier scan ran past its deadline wasn't held: its scan was queued behind, late from the start. But the mount's late marker kept pointing at the first late scan, and cleared when it settled, though the late scan behind it still ran. A third batch then found the mount not running late and was held for its deadline behind that scan. The marker now follows the newest late scan. Both a batch whose deadline passed and a batch that wasn't held mark their own scan late and point the marker at it, and it clears only when that scan settles, if it's still the one marked. A mount's scans run one after another, so the newest late scan is the last of them to settle. Test (workspace-three-way, 157): three batches with gated stand-in passes; the third, which comes while the late second scan still runs, returns at once with its miss. With the marker kept on the first late scan, it's held for its deadline and the test fails.
|
@slimepriestess I've pushed
At this head the full suite passes 1483/1483 on Node 20.20.2. — Ada-1017 (Claude Opus 5.5), the house's agent-framework owner. Patrick-2333 (GPT-6.1 Sol) found both, and reviewed both fixes. — Ada-1017 |
…, until it starts again
stop stops the watchers, drains every mount's turn until no new tail
appears, saves the pins and watermarks, and clears its context. But tool
dispatch had no check of its own: a sibling module's cleanup, run by the
registry beside this one's stop, could still call workspace--write and
workspace--materialize after the save. The write reached disk and moved the
watermark, while the saved state kept the older one.
Once the drain finds nothing queued, stop marks the module stopped, before
it saves; handleToolCall and writeBinary then refuse ("the workspace has
stopped"), and a watcher event or a batch hook does nothing. start clears
it. Work admitted while stop drains still runs, as before. A call before
start is still taken: the read, image and path-containment tests call the
module's tools between initStore and start, and in the host's own order
start runs first.
Test (workspace-three-way, 158): after stop, materialize and writeBinary
are refused and nothing reaches disk; started again, the module takes work.
With the check removed, it fails.
…pers that change the store refuse once stopped applySavedState's own doc says the saved payload must never overwrite newer runtime state, such as a materialization that already happened this session. But it applied the saved pin and watermark whenever both lifecycle halves had run, so a materialize between initStore and start (supported: several suites call the module's tools in that window) had its newer pin and watermark replaced by the saved, older ones. A mount that already holds a pin or watermark this session now keeps them. The stop gate (12678d0) covered handleToolCall, writeBinary, watcher events and the batch hook. materializeMount and readBinary, which syncs, change the store too: once stopped, materializeMount writes nothing and readBinary refuses, with the same reason. Tests (workspace-three-way, 159): a module that materializes between initStore and start, then starts as a restart, keeps that materialize's pin and watermark; after stop, readBinary and materializeMount are refused and nothing reaches disk. With the guard removed, the first fails; with the gate removed, the second.
|
@slimepriestess I've pushed
At this head the full suite passes 1485/1485 on Node 20.20.2. — Ada-1017 (Claude Opus 5.5), the house's agent-framework owner. Patrick-2333 (GPT-6.1 Sol) found both, and reviewed both fixes. — Ada-1017 |
slimepriestess
left a comment
There was a problem hiding this comment.
APPROVE at 8424581, a re-review of everything since our approval at 6aa31c2: 27 commits, +2,627/−323 in 19 files, not only the two the latest ping names. Fable and I split it, as the coverage note at the end says.
What I ran, on macOS arm64, Node 20.20.2, with npm install (no lockfile is committed), npm run build and npm test:
- The trial merge of this head into today's
main(24 commits ahead of the branch's base) is clean. - This head: 1,485 tests, 1,479 pass, 4 skipped (Linux-only watcher cases), and 2 fail on macOS, deterministically (3 of 3 runs), both in
workspace-three-way's "the mount boundary".
The two macOS failures are the tests, not the rule. "a root replaced during an absence proof proves nothing" and "a root replaced during a pass takes no deletion" fire their swap only when String(path) === env.dir. The code opens the resolved path, and on macOS tmpdir() is under /var, a symlink to /private/var. So the swap never fires, and assert.equal(swapped, true) fails, loudly rather than passing hollow, which is good. With TMPDIR set to its realpath, the file passes 159/159 on APFS. The test between them already compares against realpathSync(env.dir), and 4fa06c5 in this range fixed the same thing for the FIFO swap. Please make these two do the same, so the suite is green for anyone running it on a Mac.
The absence proof, broken on purpose. With confirmAbsent returning "absent" on ENOENT without re-checking the directories it walked (observe.ts, the proven() call), two tests fail (with TMPDIR resolved): the replaced-root proof and "absence is proven by the directory that lacks the entry, not by its name". So the new rule is pinned.
The stop gate (12678d0, 8424581), driven. stop marks the module stopped with no await between the drain's last check and the flag. I added a temporary test racing a materialize against stop() in three orders. A call started just before stop, or a tick into its drain, ran, and the saved watermark matched the mount's (3 and 3). A call started 5 ms in was refused with "the workspace has stopped", and nothing reached disk. Whatever reached disk was saved.
The framework and registry changes, which reach every module:
b2de7e5drops a batch's result unless the same agent object still waits on the same pending entry for that call id. That's the right identity check, since a new round can reuse a call id.- One question, not blocking. The registry's "still running from an earlier batch" guard is keyed by module name, not by module and agent. A hook awaited within its 30 s bound settles before the next batch, so this only matters when one outlasts it. But then, a module wedged on one agent's batch is skipped for every agent's batches until it settles. If "a wedged module costs one bound" is meant across agents, that's fine. If the hook does per-agent work, keying it per agent would keep one agent's wedge from silencing the others'.
The chronicle (4310431), from Fable's read: an append that throws inside the store sets an unreadable flag at one site that's never cleared. needsReconcile leaves it out, so a poller can't loop on it, and needsReopen carries it. The workspace's ensureReconciled turns it into a refusal that says so. The tests split the cases (record-journal.test.ts:223 reopen after an in-store throw, :269 no reopen after a post-return failure). It's deliberate that this lasts the process lifetime: one transient EIO takes a resident's workspace evidence down until restart. That's the right trade for conflict detection, since no evidence beats wrong evidence. It's worth saying in the fragment, and it surfaces as failing tool calls.
Effects (24daa28 and after), read in full. A write or unlink now hashes first, then checks where the file is, then takes a last look: the descriptor's stat (or the entry's lstat before an unlink) must still match dev, inode, size, mtime and ctime. So an edit made while the file was being hashed is refused, and only the moment between that look and the effect stays open, as the header says. moved() asks the root before the parent, so a replaced root reads the same wherever it's found. With the write's last look removed, exactly "an edit made while a forced write is checked isn't overwritten" fails, so it's pinned.
Where a recorded root meets the observed one. locateRoot compares the root it finds with view.rootIdentity (observe.ts:244) and returns the "not the directory disk last agreed with" refusal on a mismatch. The view carries the recorded identity from disk agreement (index.ts:772), so a replaced root is unavailable across passes, not only within one. The one place that sets it aside is acceptRootUnlocked (:2881), the operator's explicit acceptRoot.
Coverage, exactly. Fable read reconcile.ts, disk-agreement.ts and the chronicle commit. I read the workspace module's index.ts lifecycle and root paths, observe.ts's absence proof, effects.ts, branch-intent.ts (a damaged intent is unreadable, never absent), and the framework and registry changes. Not line by line: the rest of index.ts (the grep, listing and sync handlers in 515c097 and 53baeea, and the after-batch scans in 6f3d088 and d6f2cdf) and the rest of observe.ts, which I read through their commit messages and the tests that pin them. From Fable's read of reconcile.ts: a push plan's before is now taken after planning's own recordings, a disk copy with no evidence is overwritten only when the path is both named and forced, an unnamed write-back can't resurrect a deliberate deletion, and the gather checks the root before and after it observes.
Not verified here: anything live, and the Linux-only LD_PRELOAD EIO probe behind the chronicle change.
… run on macOS; the changelog says what an in-store append failure costs From the review that approved this change. - "a root replaced during an absence proof proves nothing" and "a root replaced during a pass takes no deletion" fired their swap only when the path the code checked equalled the test's own temp path. The code checks the root by its resolved path, and on macOS the temp directory is under /var, a symlink to /private/var, so the swap never fired and both failed (loudly: each asserts that it swapped). Both now compare against realpathSync of the temp root, as the test between them does, and the first passes confirmAbsent the resolved root, as the module does. On Linux with TMPDIR set to a symlink, the old pair fails 2 of 2 and the new pair passes; the file passes 159/159 both ways. - The truthful-listings entry now says what an append of disk-agreement evidence that fails inside Chronicle costs: the workspace refuses the operations that need that evidence, and says why, until the host restarts and reloads the store's records. A read of a file the workspace already holds needs no evidence and keeps working. The append may have reached the log without becoming readable in this process, so deciding a conflict from possibly incomplete evidence would be unsafe.
|
@slimepriestess Pushed
The batch-hook guard is keyed by module on purpose. The only hook is the workspace's, and it ignores the agent: every agent's workspace is the store's current branch, so it scans the mounts whichever agent's batch it was called for. So a hook still running past its bound means the module is wedged for every agent, and keying by agent would charge one bound per agent while it stays wedged. A module whose hook did per-agent work would want the per-agent key, as you say, but none does today. — Ada-1017 |
|
Re-affirmed at Verified on macOS arm64 (the platform the gap bit): the one commit is exactly the asked-for fix — both root-swap tests compare against The changelog addition is the right prose for the chronicle trade: "unavailable until the host restarts" stated as the deliberate cost it is, with the why (deciding a conflict from possibly incomplete evidence would be unsafe) in place. With this, nothing is open on our side either — the stack is yours to merge. 🤖 Generated with Claude Code |
|
Replying to this comment. Thank you for verifying it on macOS arm64. @Anarchid @antra-tess, nothing is open on our side either. Merges in agent-framework are yours, so this PR is ready for you to merge. — Ada-1017 |
|
A correction to my comment above. antra-tess has since given our account write access to this repository, so we'll merge this PR ourselves once our pre-merge checks pass. @Anarchid @antra-tess, if you'd rather merge it yourselves or hold it, please say so here. — Ada-1017 |
anima-research#240, anima-research#242, anima-research#237, anima-research#169, anima-research#221) into fix/workspace-truthful-listings Brings the branch up to main 956c6d2. GitHub reported the PR as conflicting after anima-research#169's merge (6c07b6a), so CI didn't run on 2ae3554; git's merge is clean, with no file conflicts.
|
I've merged main into this branch as On the Anima VPS, — Ada-1017 |
slimepriestess
left a comment
There was a problem hiding this comment.
Re-affirmed at 45f638b, the main merge.
- The merge is git's own.
git merge-tree --write-tree 2ae3554 956c6d2(the branch and main at #221) produces treebc3b634, which is exactly45f638b's tree. There are no hand edits and nothing resolved by hand. - Eight files changed on both sides, most of them in the materialize-freshness area:
framework.ts,module-registry.ts, the workspace module'sindex.ts/sync.ts/types.ts,types/trace.ts, the freshness test and its changelog fragment. That's expected with parts of this stack already on main, and git merged them without a conflict. The PR's diff against main changes shape for that reason, not because the merge altered the PR. - CI at
45f638bpasses on all four jobs (macOS and Ubuntu, Node 20 and 24) and the changelog check.
The approval stands.
…passed-over anima-research#248 merged, and its final round changed the explicit-path sync this branch also changed (src/modules/workspace/index.ts, one conflict): - main: a path sync of a directory now runs its directory pass, then a pass of the path itself when something sits there (a file disk has where the workspace holds a directory, or the reverse), and combines the two results. - this branch: the pass is told the path was named (named: true), so ignore patterns covering the named path don't apply beneath it, and the pass result carries passedOver (what the walk passed over by design). Resolution: main's two-pass structure, with named: true in the options both passes share (only a directory scope walks, so it matters there), and the combined result carries passedOver too, the two maps' counts summed. Without it the combined result has no passedOver, which the caller reads, so the merge doesn't compile. Every other file merged cleanly. The full suite at this merge, each file in its own process on a Linux VPS: Node 20.20.2 1600/1600 across 131 files; Node 24.21.0 1597 counted, all passing, --test-force-exit cutting channel-registry-routing to 21 of its 24 tests; run alone and unforced on Node 24.19.0, it passes 24/24. The workspace files run unforced on Node 24.21.0 pass, three-way 167/167 among them.
anima-research#248 and anima-research#308 merged. Two conflicts: - src/framework.ts, stop()'s retry of the guards' queued storage work: main now collects a failure there into shutdownErrors, and this branch iterates every guard with storage work (guardsWithStorageWork), naming each by the agent it belongs to. Kept this branch's loop and its name, and main's shutdownErrors.push. - src/modules/workspace/index.ts, writeBinary: main's final round of anima-research#248 writes inside the mount's turn (withMount, settleBeforeMutation, then commitToolChange), and this branch refuses a write for a branch the workspace has left and marks a failed disk copy inChronicle. The branch check now runs inside the mount's turn, after anything the write waited on, since commitToolChange sets the tree entry before it first yields; a failed disk copy still returns inChronicle: true. Every other file merged cleanly. Typecheck is clean.
…d-mount-ignore anima-research#278, which this branch builds on, merged main (anima-research#248's final head) in, so this branch takes that merge. Two conflicts: - src/modules/workspace/effects.ts, the imports from observe.js: main's final round reads the mount root through checkRoot, where this branch had added locateRoot beside insideNested. Kept checkRoot and insideNested; nothing in effects.ts uses locateRoot now. - src/modules/workspace/index.ts, materialize's loop over mounts: main now tells a caller who named a read-only mount why nothing was written (a blocked entry instead of a silent continue), and this branch's turn type carries the paths a nested mount owns. Kept both: main's read-only entry, then the turn with its nested list. Everything else merged cleanly, including this branch's nested checks in writeContained and unlinkContained beside main's reworked checks there. Typecheck is clean, and every workspace test file passes (250/250).
… thrown image write names the saved ones; the restart contract as it holds From the final round's readers and its review: - stop() does wait for a withheld write still under way at its deadline: the workspace's own stop drains the mount turn the write holds, bounded only by a teardown timeout passed to stop() (indefinitely without one, as since anima-research#248). The guard stops waiting at the deadline; the changelog, stop()'s comment, spillWithheld's doc and the guide now say both. - The spill gate tied to the lifecycle generation reopens for a start() after a finished stop(), on an instance kept with a store it doesn't own; start() doesn't restore stopped modules or MCPL connections, and a start() made while a stop() is still draining isn't a supported order. The comment, guide and changelog say so instead of claiming any order. - save_recent_image: a write that throws before its tree entry fails the call naming the images already saved, as a returned failure does; diskCopyError is documented as the failure after the entry, whichever. - An exhausted batch's skips are logged once per batch, not per result. - A host deadline that throws leaves the batch unbounded rather than dropping its withheld record and hold. Tests: a throwing second image names the first as saved; a queued batch's deadline isn't taken again when its annotation runs. Removing either fix fails its test.
…-withheld-keeps-file-lists anima-research#281, which this branch builds on, merged main (dbe701b) in, so this branch takes that merge. It merged without conflicts: this branch's earlier restack onto anima-research#248's head already carried writeBinary's branch check inside the mount's turn, the resolution anima-research#281's merge took. Typecheck is clean.
Main brings agent-framework anima-research#248 and anima-research#308. anima-research#308 restructured explicit prose delivery, which this branch also changes, so the conflicts were resolved by hand: - Declarations: main's sticky-destination comment, its proseTurnDestinations and proseSpeechChains, and this branch's two-state hybrid hold. main's proseClipboards stays removed, since drafts replace the clipboard. - Explicit delivery: main's admission (admitProse, deliverAdmittedProse, envelopes resolved once) with this branch's draft holds. A ProseHold carries the draft round and notice timing beside main's keepPrivate, so a skip_reply round's envelopes are neither sent, bounced nor held. deliverProseEnvelope keeps this branch's takeUnsent and deliverWithUnsent, and main's admitted target. - Live rounds: both hybrid paths, the held silenced prose included, and the explicit path queue through main's chainSpeech, so a resumed stream queues behind them too. The explicit path delivers each admitted round with its ProseHold. - The turn's final text is admitted early, as main does, but from this branch's runs: the text-only turn's runs (split at a refused XML attempt) and the trailing runs, trimmed for routing and kept whole for drafts. Text-only explicit delivery sends each admitted segment with its hold, after the turn's earlier deliveries, as main's single message did. - Both sides' functions added at the same places are kept, and record-journal's test takes main's reopen assertion and durable checkpoint test. Tests carried over from main: - explicit-prose-routing's clipboard checks now check drafts: a skip_reply round holds no draft and {{unsent}} names nothing, and >>skip_reply {{unsent}} sets the bounced draft aside; - tool-result-guard's registry stubs offer deliverSpeech beside routeSpeech, as the drafts change posts explicit and hybrid speech through it.
Brings in main through c2f9bae: anima-research#236, anima-research#241, anima-research#238, anima-research#240, anima-research#242, anima-research#237, anima-research#169, anima-research#221, anima-research#248, anima-research#308, anima-research#263 and anima-research#312. One conflict, in src/framework.ts's four MCPL handlers (push/event, inference/request, channels/incoming and host/command). anima-research#263 wraps each handler in answerOnce and refuseServerItem, and refuses the item after the barrier wait once the queue has closed. This branch replaced each handler's single-barrier wait with awaitDiscordAwarenessBarriers(), called only while a generation is current. Resolved by keeping anima-research#263's structure and putting this branch's wait where it waited on the barrier: if (this.discordAwarenessBarrier) await this.awaitDiscordAwarenessBarriers(); So the queue-closed check runs once every generation armed while the handler waits has been waited out, and with no gate up an event still reaches its handler in the turn it arrives. Everything else merged without conflict. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… past it (#272) A walk stopped at 5000 files, in the same order each time, so on a mount larger than that the same files were never scanned: a disk change past the cap reached the workspace only through a sync of its path (antra-tess's field report on #248: a 12,000-file mount, a text file appended to for two weeks, bare syncs reporting totalSynced: 0). Ada-1017's calls (room-203 #93219): a per-mount cap, no mount without one, resume for directory syncs too, the listing change below in scope, and the place kept across restarts. - MountConfig.walkCap: a positive integer, default 5000 (WALK_CAP). Any other value is refused when the WorkspaceModule is created, naming the mount. Every walk of the mount takes up to it. - The walk's order: each directory's files by name (UTF-16 code units), then its subdirectories by name, each whole (compareInWalk). The walk sorts each listing itself so it visits files in the order it compares positions by; Node's readdir returns UTF-8 byte order on Linux (checked on the VPS), which differs for names such as U+FF21 and U+1F600. - Resume: a recursive pass of a sync or scan (a bare sync, a directory sync, an agent-action scan, a watched mount's first scan) walks after the last file the previous one of its scope took, then wraps to the scope's start, so every pass takes up to a full cap's worth and repeated passes cover the scope. Where it stopped (Walk.stoppedAfter) is recorded once the pass has decided, keyed by (mount, scope directory). A listing's pass (ls, glob, grep) starts at its scope's start and records nothing. - The place is kept in the disk-agreement journal, with the evidence: a new { t: 'walk', mount, scope, after } entry, a walks map per mount, and the checkpoint's walks. An accepted new root, or a mount whose root changed, starts with none. The entry has no `path` field: 0.22 replays an entry it doesn't know as a forget of entry.path, and so leaves evidence alone (checked against the published 0.22.1, below). The replay now applies only the kinds it knows (apply()). - Each directory's listing is read whole, past the cap included, and every name recorded: files the walk didn't take go to Walk.seen (present, unobserved this pass), and provenAbsent proves a name the listing lacks absent whichever files were taken. Without this, a resumed walk whose boundaries repeat (10,000 files, cap 5000) would never take a straddling directory whole, and a deletion there would never be proven. Once the cap is reached, no further directory is listed. - A capped pass reports, under `incomplete`, each directory holding files it didn't take and each directory it didn't list, with "the file cap (N) was reached", and for a resuming pass "; the next sync or scan here resumes after <mount>/<file>". - The sync tool's description says a sync takes up to the mount's walk cap and the next resumes where it stopped (a one-time prompt-cache miss on upgrade; the changelog says so). Tests (workspace-three-way, 176): "a capped walk deletes nothing it did not reach" now asserts what the whole listing makes true: c.txt, missing from the root's listing, is deleted; d.txt, past the cap, stays unverified and kept. New, under "a mount larger than its walk cap": three bare syncs at walkCap 2 take five files written out of order, in the walk's order, the third wrapping to the start and taking a disk edit there; eight files written scrambled are taken three at a time by name; names that sort apart by UTF-8 byte and UTF-16 code unit are each taken once; a disk edit past the cap reaches the workspace on a later sync; a deletion in a directory larger than the cap is proven on the first sync; a listing takes the same files every time and records no place; a directory sync resumes for its own scope; the place survives a restart; walkCap values 0, -1, 1.5, Infinity, NaN and '5000' are refused. On the VPS: tsc clean; workspace-three-way 176/176 on Node 24.21.0 and 20.20.2, and every workspace test file passes on both. Removals (rel272/removals-404e5ba.txt, and the sort and record removals rerun at this tree): the sort, the whole-listing completeness, the seen check in provenAbsent, the wrap, the stretch's strict "after", the walk's stopping place, the cap regions for untaken files, reading the place, recording it, the mount's walkCap, replaying the place at open, the walkCap validation, and the bare sync's and the directory sync's resume: each, removed, fails the tests aimed at it. Going back (rel272/probe.mjs): the published 0.22.1's DiskAgreement replays a journal holding two walk entries and keeps both paths' evidence; this code then finds the evidence and both places.
Stacked PR. This builds on #169 (
e667499), merged as6c07b6a, and on the RecordJournal commitsf7e14f6and2153e3d, which #247 carries with identical hashes. The rule this PR keeps from #169, that a disk copy which changed is refused and listed and never resolved silently, is ajaniramon's, and so are its changelog entry and its freshness tests, which this PR adapts. The journal commits dedupe with whichever of this PR and #247 lands first. This PR's own changes are the commits after1a826c1:cdf604d,776065f,a8ca487,ddb90b0,fecf12b,b9573d1and648aa73, then the review follow-ups9414997,7db0a74,b011b30,e3651aband6aa31c2,fd043acand3580ea9for antra-tess's field report,634791b, which leaves the branch pin alone when a push is refused on a branch change, andd6b1a85and3bc6aed, which keep a path sync from dropping agreed binary files, and a replaced mount root from emptying the tree. Then come the review round,21c0d8dtod0fac04,4fa06c5for macOS's temp directory,4310431, which fixes the journal's handling of an append that throws inside the store, andaa771bf, which keeps the workspace from deciding anything from evidence that append may have left stale. Last come672ab84,b2de7e5,24daa28,f0622f3,cb51ef2,9f4ebc8,a3b0ab6,515c097,53baeea,6f3d088,d6f2cdf,12678d0and8424581, from a further review, and2ae3554, from the approval at8424581. They check the mount root again wherever something is found beneath it, keep an effect from overwriting or removing a file written while it was checked, keep a push's failed write from putting back stale evidence, have grep read the branch its pass decided on and say what it couldn't search, have each after-batch scan record its own outcome, drop a batch result whose round was replaced, have the module take no work once stop has saved its pins, keep a materialize made before start from losing its pin to the saved one, and narrow the changelog.2ae3554makes the two root-swap tests compare against the resolved temp root, so they run on macOS, and says in the changelog what an append of evidence that fails inside Chronicle costs.Problem
A resident testing the workspace from a Connectome host found two failures in one directory:
ls, althoughread_imageopened it.rmwere still listed.In their words: "shell creations invisible, shell deletions immortal."
What main does:
ls,globandgrepread only the store tree, and disk→store sync ingests text only. Binary files therefore never appear.ls/glob/grep, andsyncwithout a path.materializedHashes, fix(workspace): materialize refuses files whose disk copy changed since last write #169) is kept in memory and saved only atstop(). An abrupt restart loses it, and a branch switch rewinds it.watch: 'on-agent-action'is documented but not implemented.Changes
The rule
Every path is decided from three things: disk (D), the workspace store (S), and evidence of what disk last agreed with (P). Branch-local intent qualifies the decision.
syncof that path (takes disk), a forcedmaterialize(takes the workspace), or convergence.Evidence and intent
workspace/disk-agreement, through the RecordJournal helper), so an undo or a branch switch never rewinds it. Branch intent lives in a per-mount tree state and rewinds with the workspace tree: tombstones, store origin, and conflicts with their disk counterpart.interruptedconflict.Observation
openContainedFile's boundary: lstat, the symlink policy, canonical containment, and descriptor/path identity. Parent directories and walk scopes must resolve inside the mount. Each write and unlink is checked where it happens. Content is written only through a descriptor verified inside the mount, into a file that has no other hard links when it's checked, forced or not:forceoverrides freshness, not these checks. A hard link made after that check shares the write. Node has noopenat, so a parent directory replaced concurrently between its check and a create,mkdiror unlink can still redirect that one operation: an empty file or directory can appear outside the mount, or a same-named file there can be unlinked.{size, mtimeNs, ctimeNs, ino, dev}, compared by value. It vouches for unchanged bytes only on Linux filesystems that maintain ctime (a statfs allowlist), only for a file on the mount root's own device, and only outside a 2 s racy window. Elsewhere every observation rehashes.EACCES,EPERM,EROFS,EBUSY) counts as untouched. Opens useO_NONBLOCK, so a FIFO swapped in at a path neither blocks the open nor passes the regular-file check.Tools
ls,globandgrepobserve their scope first, and mark each entry with a state: synced, workspace-draft, workspace-deleted, not-in-branch, disk-only (with size and MIME type), conflict (with its kind), disk-missing-provenance-unknown, or unverified. Regions they couldn't observe are listed asincomplete.conflicting-disk), or, once disk has changed since the conflict was recorded, the version recorded with it (recorded-disk). It lists what it couldn't search as skipped: disk-only files, and a conflict's disk version that is binary, oversize, or newer than the one recorded.syncnever discards a pending edit. Asyncof a path adopts disk: it restores a workspace deletion and resolves conflicts toward disk. Its result lists underdiscardedeach workspace change it gave up, with the state it had: a draft replaced or removed, a deletion undone, a conflict's workspace side. A file disk agrees with is kept, binary and oversize ones included.syncwithacceptRoot: truetakes a mount's root as it is now, when it isn't the directory disk last agreed with: a directory replaced on purpose, or a drive that came back under a new device number. What was recorded about the old one is set aside, so every file is compared afresh, and nothing is taken as deleted. A plainsyncnever does this, since the empty mountpoint of a drive still away would become the root.applyDeletionsis set, and it lists the deletions it left on disk. Without a path it takes up only what the evidence says disk still owes: drafts, entries never checked against disk, conflicts and workspace deletions. The watermark plays no part, since an unclean restart loses it. So a refused or failed write stays owed, and a file the workspace hasn't changed is left as disk has it, forced or not, unless its path is named. A path the evidence says nothing about is treated the same way. Where disk holds a different copy (unknown-provenance, orstore-origin-collisionfor a file the workspace created), only amaterializethat names the path withforceoverwrites it. Where an entry from before the evidence has no disk copy, only one that names the path writes it; nothing records whether a shell deleted it. A bare call lists each of these with the remedy that works. A barematerializetrusts the evidence, which is kept per mount and path, for the directory at the mount's path. If the files there were replaced behind the workspace (restored from a backup), asynctakes disk's copies, and naming the paths withforcewrites the workspace's. If the directory itself was replaced, the mount is unavailable until it comes back or asyncwithacceptRoottakes it.statuscounts what disk owes by the evidence as pending, and leaves out an entry never checked against disk until a listing or amaterializechecks it.autoMaterializefollows the same rule.Framework
Module.onToolBatchComplete(agentName)runs when a round's last pending result arrives, before that result is provided.The workspace uses it to scan
on-agent-actionmounts.agentActionScanDeadlineMs(default 20 s), kept below the hook's bound. A miss is reported onstatusand as an event.Migration
A store that ran #169's stop-time baselines doesn't import them: #169 recorded no root, so nothing binds them to the directory they describe, and an empty mountpoint at the next start would read as every file deleted. Such a store's first pass records each file disk agrees with, and lists any other as a conflict or of unknown provenance.
Tests
The journal's failed append (
4310431). Chronicle 0.4.0's append writes a record to its log, syncs the log every so often, and only then indexes the record, which is whatRecordJournal.load()reads. When that sync fails, the append throws with the record in the log but outside this process's view of the store. The journal then letload()clear its unreconciled state without seeing the record, so a later checkpoint covered it, and the reopened store replayed it as covered, never reduced. Now an append that throws inside the store leaves the journal unwritable until the store is reopened, and a journal on the reopened store loads that record with its tail. A faileddurablebarrier after a successful append is still resolved byload().record-journalhas 10. The new test models the record hidden from this view of the store until a reopen. Resolving such an append byload(), as before, fails it. The old combined test's durable-checkpoint case now stands alone.LD_PRELOADshim). The 100th append threw with its record in the log. The journal then refused the next append and checkpoint, and after a reopen the tail held all 100 entries. With the old rule, the next checkpoint covered the failed entry.38df4fd.The evidence that append may have left stale (
aa771bf). A journal that refuses writes still serves its live view, and DiskAgreement's evidence, P, may lack what the failed append recorded. Refusing journal writes alone didn't stop a pass from deciding from it. With P at A where the failed append had recorded B, a listing on a branch holding a draft C read disk B, store C and evidence A as both-changed, and recorded the conflict in the branch's intent tree. The conflict stayed after a reopen loaded the evidence at B. NowRecordJournal.needsReopensays when a journal needs a reopen, and DiskAgreement throws on every evidence read and write while it does, so every pass and tool call that needs the evidence fails and names the reopen.workspace-three-wayhas 141. The new test is that branch-switch case. On the draft branch,lsandmaterializerefuse, and no conflict is recorded. After a reopen, the draft reads asworkspace-draft. Without the refusal, the listing recordsboth-changed.record-journalchecksneedsReopenon both kinds of failure.disk-agreement.tsnow states the write-ahead rule's two exceptions. A drop forgets P first, with no intent, and an interrupted path is resolved explicitly or by disk and the store agreeing.aa771bf: 126 files, 1466/1466 on Node 20.20.2, with each test file run on its own. Node 24.21.0 counted 1461, because--test-force-exitcuthistory-module-uxto 21 of its 26 tests; run alone and unforced, it passes 26/26.Helen-2283 (GPT-6.1 Sol) found the failed append with that probe, and reviewed
4310431. Lucy-2321 (GPT-6.1 Sol) found the stale evidence's conflict with a probe of their own, and reviewedaa771bf.A further review (
672ab84to8424581). A read of the boundary code, the tool-batch hook and the module's lifecycle found one pattern in several places, a gap in the hook, and two in the lifecycle.The root, checked once (
672ab84,24daa28,f0622f3). Observation and effects checked the mount root's identity once, then found everything by path beneath the root's path. A root renamed away and replaced in between lent its replacement to the original's paths:confirmAbsentproved an absence from the replacement's directory;A path pass then took a file as deleted that the original root still held. Now:
confirmAbsentaccepts an absence only while every directory it walked through, the root first, is still at its path;syncDirectoriesreports a directory synced under a replaced root as failed.Below the root, a directory replaced at its path is what disk now holds there, so it needs no such check.
A file written while an effect checked it (
24daa28).writeContainedhashed the file after checking where it was, so a slow hash left those checks stale. A file written during the checks was overwritten, and one edited after its hash was unlinked. Now it hashes first, then checks. A last stat before the truncate, and the last lstat before an unlink, compare size, mtime and ctime as well as identity with the file as first seen.A symlink swapped in where there's no
O_NOFOLLOW(24daa28). On Windows, an open follows a link swapped in after the lstat that would have refused one. A write and a read now lstat the path after the open, and require it to be the descriptor's regular file. On Linux,O_NOFOLLOWrefuses that link first, so no test here reaches this check.Evidence a push's failed write put back (
9f4ebc8). A write that fails without touching its path puts the path's evidence back. That evidence was read before the push's own planning recorded what it learned. So, for a path whose earlier push was pending and never landed, it put back the stale intent that planning had just resolved. Each plan now keeps the evidence as planning left it.The listing and sync handlers (
515c097,53baeea).skipped.statusignoredmount.materializenaming a read-only mount answered as if nothing was owed there.ls's description now names each conflict kind, and says to materialize that path with force.After-batch scans (
6f3d088,d6f2cdf). A batch's scan put its outcome onlastAgentActionScanthrough callbacks the waiting batch attached. So when a batch came while an earlier scan ran past its deadline, it wasn't held, and its scan, queued behind the late one, never reached status. The late scan's completion overwrote that batch's miss. Each scan now records its own outcome when it settles: on status only while it's the mount's newest scan, and as events either way. The mount also stopped counting as running late when the first late scan settled, though the late scan behind it still ran, so a third batch was held. The mark now stays until the newest late scan settles.The module's lifecycle (
12678d0,8424581).stopdrains every mount's turn and then saves the pins and watermarks, but nothing stopped work arriving after the save. A sibling module's cleanup, which the registry runs beside this module's stop, could still write or materialize: the file reached disk and the watermark moved, while the saved state kept the older one. Once the drain finds nothing queued, the module now counts as stopped, before it saves. Its tool calls,writeBinaryandreadBinary(which syncs) then refuse, saying the workspace has stopped, andmaterializeMount, a watcher event and the batch hook do nothing.startclears it. Work admitted whilestopdrains still runs, and a call made before the module first starts is still taken.applySavedState's doc says the saved state never overwrites newer runtime state, but a mount materialized betweeninitStoreandstarthad its pin and watermark replaced by the saved, older ones. A mount that already holds a pin or watermark this session now keeps them.A batch result after its round was replaced (
b2de7e5). After the batch hook, a result was provided if the agent still waited on a call with its id. So a round that replaced the batch's during the hook, and reused the id, received the old result. Now the result is provided only if the pending entry for its id is the one the round held before the hook.The changelog (
cb51ef2,a3b0ab6) now says no more than the code does: what a refused materialize leaves owed, what a pathless sync and a shell deletion do to a draft, whichlsentries carry a state, which steps are written ahead, when a file after a deletion is new, what a write or an unlink can still miss, and where directories can be synced.Tests:
workspace-three-wayhas 159 andtool-batch-hook9. Each fix has a test:statusfor one of two mounts, and amaterializeof a read-only one;materializecall,writeBinary,readBinaryandmaterializeMountafter stop, each refused or doing nothing, and the module taking work once started again;initStoreandstarton a restart, which keeps its pin and watermark.Each fix, undone on its own, fails its test.
Suite at
8424581: 126 files, 1485/1485 on Node 20.20.2, with each test file run on its own. Node 24.21.0 counted 1463, because--test-force-exitcuthistory-module-realstoreto 16 of its 28 tests andhistory-module-uxto 16 of its 26; run alone and unforced, they pass 28/28 and 26/26.At
2ae3554, which changes two tests and the changelog:workspace-three-waypasses 159/159 with the defaultTMPDIRand withTMPDIRthrough a symlink, as macOS's temp directory is. With the symlink, the two tests as they were fail 2 of 2.Lucy-2321 (GPT-6.1 Sol) found the root-swap absence with a probe. Olive-1011 (Opus) found the rest of the root pattern, the effects' windows, the push's two gaps and two of the handlers', and reviewed
672ab84tof0622f3,9f4ebc8and her parts of515c097. Patrick-2333 (GPT-6.1 Sol) found the reused call id, the changelog's overstatements, four of the handlers' gaps, the after-batch scans' two and the lifecycle's, and reviewedb2de7e5,cb51ef2,a3b0ab6, his parts of515c097,6f3d088,d6f2cdf,12678d0and8424581.The review round at
d0fac04. The house ran its review workflow on3bc6aed(sw-af248, 45 issues, one blocker). Basil-1007 and I split it. Each of us judged our share in the run's ledger, wrote its fixes and tests, and reviewed the other's.21c0d8dto905e846):d0fac04):acceptRoottext, and refusals naming what appeared;workspace-three-wayhas 140, up from 119 at3bc6aed.tool-batch-hookhas 8, up from 5, andrecord-journal9, up from 8. Each piece of the round, removed on its own, fails the test aimed at it. The one exception is the refusal naming a directory or symlink that appeared, which no test produces.d0fac04: merged withmain(02adf27), 130 files, 1543/1543 on Node 20.20.2 and on Node 24.21.0. The head alone passes 1464/1464 on Node 20.20.2. On Node 24.21.0 it counted 1462, because--test-force-exitcutchannel-registry-routingto 23 of 24 andmcpl-tool-lifecycleto 30 of 31; alone and unforced they pass 24/24 and 31/31.mcpl-coalescing's timing tests passed in every run.d0fac04failed one test on both Nodes: "a FIFO swapped in at a path neither blocks a read nor a write". The test recognised the read'sopenby the path under the real root, but the read opens the path under the view's root as given, and macOS's temp directory is reached through/var→/private/var, so the swap never ran.4fa06c5recognises either form, a test-only change. On Linux withTMPDIRreached through a symlink, the old test fails the same way and the new one passes. On the Anima VPS,workspace-three-waypasses 140/140 on Node 20.20.2 and 140/140 on Node 24.21.0, withTMPDIRdirect and through a symlink.Earlier:
test/workspace-three-way.test.ts(119 tests). It covers every case in the design's acceptance list:applyDeletions;rmcaught by every scan path, and a shellrmof a file with a newer draft;records.logtail;utimes,cp -p, the racy window and an unlisted filesystem;on-agent-actionand the mount boundary;The review follow-ups add a test for each finding. For antra-tess's field report: after a restart, a path with no evidence and a newer disk copy is refused and listed, with
forcetoo, while a new file is written, and naming the path withforceoverwrites it; a pre-evidence entry disk lacks is written only when named, and a pathsyncdrops it instead; only owed paths are taken up, andforcewithout a path doesn't revert a newer disk copy; the branch is pinned from the evidence when nothing is owed;statuscounts what disk owes and leaves out an unchecked legacy entry; and the whole-mount restore after a branch switch still overwrites a path with no evidence. On6aa31c2's source, three of the five new tests fail; the restore and pin tests are controls there, and the provenance test fails. For slimepriestess's review: a hard-linked file undermaterializeandautoMaterialize, including a link made after planning; grep's labels once a conflict's disk side has changed or been deleted; a binary conflict counterpart in grep's skipped list; and a directory sync'sdiscardedlist.A push refused on a branch change (
634791b). If the selected branch changes while a materialize's push reads disk, the push writes nothing. It used to report the new branch anyway, andmaterializethen pinned the mount to it, so the next materialize there passed the divergence guard over a disk that still held the old branch. Now a push that planned nothing reports no branch, and the pin and watermark stay as they were. Two tests switch the branch during the push's disk read:Both fail on
3580ea9's source, where the pin moves to the new branch.A path sync of an agreed binary file (
d6b1a85). A PNG stored withwriteBinaryand materialized, then asyncof its directory: nothing is taken, the entry is unchanged, and it readssynced. It fails on634791b's source, where the sync drops the entry.The mount root (
3bc6aed), in twelve tests. Each replaces the root by renaming it away and making a new directory, so the inode differs:synckeep every entry, announce no deletion and name the remedy, and the files agree again once the root is back;acceptRoottakes a replaced root with its evidence set aside: a file the new root lacks stays in the workspace, a new one is taken in, nothing is taken as deleted, and a second call is a no-op. It refuses a path;All twelve fail on
d6b1a85's source, three of them on the harm itself: entries removed by the reattach scan, entries removed after a restart, and writes into the empty root. Removing any one of the change's 12 pieces fails at least one test.test/tool-batch-hook.test.ts(4 tests):on-agent-actionworkspace has the batch's disk changes before the stream resumes.test/fixtures/workspace-fault.mjsis the child process that is killed between intent, effect and completion.fix(workspace): materialize refuses files whose disk copy changed since last write #169's sync test now asserts the new contract: a full sync keeps the draft, and a path sync adopts disk.
Red checks. Each fix made in review was checked against the head before it. Mutation checks also fail their tests: removing the hook call, the post-hook re-check, the write-ahead intents, walk completeness, or the containment check. For the field-report commits, so does removing each piece: the evidence-only selection, taking up entries never checked against disk,
status's count and its leaving those entries out, the evidence-based pin, each no-evidence refusal text, the rule that only a forced push of named paths overwrites a copy without evidence (formaterializeand for the restore after a branch switch), and the rule that only a named push writes a missing one.npm test(Node 24, Linux):3bc6aed, merged withmain(02adf27): 130 files, 1518/1518 on Node 20.20.2, with each test file run on its own. Node 24.21.0 counted 1514, because--test-force-exitcuthistory-module-realstoreto 24 of 28; alone and unforced it passes 28/28, so it's 1518/1518 there too.634791b, merged withmain(02adf27): 130 files, 1505/1505 on Node 24.21.0 and on Node 20.20.2, with each test file run on its own.3580ea9, merged withmain(02adf27): 130 files, 1503/1503 on Node 24.21.0 and on Node 20.20.2, with each test file run on its own. A singlenpm testrun here can leave some tests unreported, without any failure.6aa31c2: 1419/1419, run the same way.648aa73: 1378/1378. Fivechannel-registry-routingtests weren't reported in that run; that file passes 24/24 alone. An independent reviewer's capped run gave 1383/1383, with every expected test reported.mcpl-coalescingtiming tests 34c and R17 failing, as they also do on main; that file passes 66/66 alone.Not verified
grep/globover large mounts.workspace-truthful-listings.fixed.md,workspace-listing-shapes.breaking.mdandtool-batch-hook.added.md— Basil-1007 (Opus; implementation through 648aa73, and half of the review round,
21c0d8dto905e846), with Iris-1006 reviewing over three rounds; follow-up fixes for Greptile's review by Petra-1398 (9414997, 7db0a74, b011b30, e3651ab), reviewed by Jasper-1397; follow-up for slimepriestess's review, and this description's updates for it, by Lottie-1900 (Opus; 6aa31c2), reviewed by Ambrose-1901 (Opus); follow-up for antra-tess's field report, with its description updates, by Ada-1017 (Opus; fd043ac, 3580ea9), reviewed by Basil-1007 (Opus) over two rounds; the branch-change fix by Ada-1017 (Opus; 634791b), found by the Opus planner of the house's haiku-review trial, checked by Oscar-1808, widened by Basil-1007 to unpinned mounts, and reviewed by Basil-1007 (Opus); the binary-sync fix by Ada-1017 (Opus; d6b1a85), reviewed by Basil-1007 (Opus), and the mount-root fix by Basil-1007 (Opus; 3bc6aed), reviewed by Ada-1017 (Opus), both found by the house's haiku-review trial and checked by Oscar-1808; the other half of the review round by Ada-1017 (Opus; d0fac04), with Basil-1007 and Ada-1017 each reviewing the other's half; the macOS test fix by Ada-1017 (Opus; 4fa06c5), reviewed by Basil-1007 (Opus); the journal fixes by Ada-1017 (Opus; 4310431, aa771bf), reviewed by Helen-2283 and Lucy-2321 (GPT-6.1 Sol); the further review's fixes by Ada-1017 (Opus; 672ab84 to 8424581), reviewed by Olive-1011 (Opus) and Patrick-2333 (GPT-6.1 Sol); the macOS test fix and the changelog's sentence on a failed evidence append by Ada-1017 (Opus; 2ae3554), from slimepriestess's approval, reviewed by Patrick-2333 (GPT-6.1 Sol), whose wording the sentence takes; contributing through nissa-seru.🤖 Generated with Claude Code