diff --git a/changelog.d/materialize-freshness-guard.fixed.md b/changelog.d/materialize-freshness-guard.fixed.md index 9dcc6620a..00974d292 100644 --- a/changelog.d/materialize-freshness-guard.fixed.md +++ b/changelog.d/materialize-freshness-guard.fixed.md @@ -1 +1 @@ -- **`materialize` no longer silently overwrites disk files changed by another writer** (#109). Before writing, each file's disk hash is compared against what the workspace layer last materialized; a mismatch means a shell/tool edit happened since, and the path is refused and listed in the result (`skipped`, with the remedy) instead of silently reverted. A disk copy that exists but cannot be read (e.g. a write-only file) is refused the same way, with the filesystem error in the reason, rather than treated as absent. `force: true` remains the deliberate override. Refused paths stay owed to every later materialize (and count in `status.pendingChanges`) without holding back the sequence the branch guard reads. `sync` re-pins the baseline when it adopts a disk copy, so the agent's next edit to that file is not refused. Baselines and refused paths persist across restarts, and a first materialize that finds every file already in place still pins the branch. Files whose disk copy already matches the target bytes are left untouched (mtime preserved). The post-undo/redo `_config` restore keeps its overwrite semantics. +- **`materialize` no longer silently overwrites disk files changed by another writer** (#109). A file disk changed since it last agreed with the workspace is refused and listed in the result (`skipped`, with the remedy) instead of silently reverted. So is a disk copy that exists but can't be read (e.g. a write-only file), with the filesystem error in the reason, rather than treated as absent. `force: true` remains the deliberate override. A refused workspace edit stays owed: every later materialize takes it up again, and it counts in `status.pendingChanges`. A file the workspace hasn't changed, whose disk copy changed, isn't owed: a materialize naming it refuses it, and one without a path leaves it, for a `sync` of the path (take disk) or a `materialize` naming it with `force` (take the workspace). What disk last agreed with is kept as evidence that survives restarts and branch switches (see the truthful-listings entry), and a `sync` that takes a disk copy records that agreement, so the agent's next edit to that file isn't refused. A first materialize that finds every file already in place still pins the branch, and a file whose disk copy already matches the target bytes is left untouched (mtime preserved). The post-undo/redo `_config` restore keeps its overwrite semantics. diff --git a/changelog.d/tool-batch-hook.added.md b/changelog.d/tool-batch-hook.added.md new file mode 100644 index 000000000..8ffcd7e4d --- /dev/null +++ b/changelog.d/tool-batch-hook.added.md @@ -0,0 +1,15 @@ +- **`Module.onToolBatchComplete(agentName)`**: called once an agent's tool batch completes, + when the round's last pending result arrives and before it is provided. It therefore runs + before anything that continues the turn: re-inference, stream resume or the end of the turn. + The framework awaits every module's hook in parallel, each bounded at 30 s and fail-open; a + hook that throws (before returning its promise or after) or times out is traced as + `module:batch_hook_failed` and the round goes on. + The agent's state is checked again afterwards, so a result whose turn was cancelled or + replaced during the hook is dropped (traced as `tool:result_dropped`). A hook still running + from an earlier batch, past its bound, is skipped rather than called again. The hook is + awaited in the framework's event loop, so a slow one holds every agent's event processing, + not only the batch's agent. Script-inner tool calls don't trigger it, nor does a batch whose + error results the framework fills in after its stream ended before its tools answered. +- Workspace: `materialize` accepts `applyDeletions`, `WorkspaceConfig` accepts + `agentActionScanDeadlineMs` (default 20000, kept below the hook's 30 s bound), and + `workspace:deleted` events carry `conflicts` when a disk deletion met a newer workspace edit. diff --git a/changelog.d/workspace-listing-shapes.breaking.md b/changelog.d/workspace-listing-shapes.breaking.md new file mode 100644 index 000000000..55339ab6e --- /dev/null +++ b/changelog.d/workspace-listing-shapes.breaking.md @@ -0,0 +1,18 @@ +- **Workspace tool-output consumers:** `glob` returns `matches` as objects, + `{ path, state, size?, mimeType?, conflict?, note? }`, instead of path strings. `sync` reports + `conflicts` as `{ path, kind, diskCopy }`. `sync` without a path no longer replaces a workspace + draft with a differing disk copy: the draft stays, and where disk changed too, the path is a + conflict until a `sync` of that path or a forced `materialize` resolves it, or the two converge. + For an `unknown-provenance` or `store-origin-collision` conflict, the forced `materialize` must + name the path: one without a path leaves it. `ls`, `grep` and `sync` otherwise only add fields + (each `ls` file entry's `state`; grep's `state`, `version`, `skipped` and `incomplete`; a path + sync's `discarded`). +- **Hosts and module authors reading workspace internals:** `MountState` no longer has + `materializedHashes` or `refusedPaths`. Disk-agreement evidence lives in `workspace/disk-agreement` + chronicle records, and branch-local intent in each mount's `workspace//intent` tree state. + The module's saved state carries neither field, so a build from #169's branch started on a + store this release wrote begins with no freshness baselines. +- **Prompt cache:** the descriptions of the workspace's `ls`, `glob`, `grep`, `sync`, + `materialize` and `delete` tools change, and `sync` and `materialize` take new parameters, so + the first request after the upgrade that carries those tools' definitions misses the cache from + its tool block on. diff --git a/changelog.d/workspace-truthful-listings.fixed.md b/changelog.d/workspace-truthful-listings.fixed.md new file mode 100644 index 000000000..a64e21e57 --- /dev/null +++ b/changelog.d/workspace-truthful-listings.fixed.md @@ -0,0 +1,106 @@ +- **Workspace listings tell the truth about disk, and no scan discards a workspace edit.** `ls`, + `glob` and `grep` check their scope against disk before answering, under one rule over disk, the + workspace store and what disk last agreed with. Files created by a shell show up, including + binary and oversize ones, listed as `disk-only` with their size and, for images, their type. + A file deleted by a shell leaves the workspace on every scan path (a watcher event, `sync`, a + listing, the scan after a restart, and a watcher reattach) where the workspace held what disk + last agreed with; a workspace edit of it stays, as a `deleted-on-disk` conflict. Each file entry + carries a state: + `synced`, `workspace-draft`, `workspace-deleted`, `not-in-branch`, `disk-only`, `conflict` (with + its kind) or `disk-missing-provenance-unknown`. Where disk couldn't be checked, the state is + `unverified`, and `incomplete` names the unreadable, ignored or capped regions. A pending + workspace edit is never overwritten by a scan, a watcher event or a lazy read. When disk and + the workspace both changed, the path becomes a conflict that stays until a `sync` of that path + (take disk), a forced `materialize` (take the workspace) or convergence. A text disk version is + kept in the store; a binary or oversize one is referenced, and the listing says if it changed + since. `grep` labels a kept disk version `recorded-disk` once disk has changed since, and lists + a conflict's disk side it couldn't search as skipped. A `sync` of a path lists under + `discarded` each workspace change it gave up for disk's state. +- **A mount root that isn't the directory disk last agreed with is unavailable, not empty.** The + workspace records the root's identity (its device and inode) with the first evidence it + gathers there. When the drive under a read-write mount is unmounted, the watcher recreates its + mountpoint as an empty directory; that root is listed as unavailable (`incomplete`, saying what + to do), so no scan reads its emptiness as every file deleted, and no `materialize`, forced or + not, writes into it. When the drive comes back at the same path, the mount is available again; + one that comes back elsewhere leaves it unavailable until it is back at that path. If the + directory was replaced on purpose, or a drive came back under a new device number, `sync` with + `acceptRoot: true` takes the root as it is now: what was recorded about the old one is set + aside, and every file is compared afresh. +- **A workspace deletion stays deleted, and materialize never deletes from disk unasked.** Without + `autoMaterialize`, a deleted file's disk copy, unchanged since disk last agreed with the + workspace, is listed as `workspace-deleted`, and no scan or lazy read brings that copy back; a + copy changed on disk since is a conflict. `materialize` lists the deletions it left on disk; + `applyDeletions: true` removes those whose disk copy is still one the workspace held, and + `force` too removes those changed since. A `sync` of the path takes disk's copy back into the + workspace when it's one the workspace can hold (text within the size limit). Once the deletion + is confirmed on disk, a file that later appears at the path is a new file, adopted like any + other. + `autoMaterialize` writes and deletes follow materialize's freshness rule: a disk copy changed + by another writer since it last agreed is refused, not overwritten or unlinked. +- **A `materialize` without a path takes up only what disk still owes:** workspace edits, + entries never checked against disk, conflicts and workspace deletions. It no longer depends on + the sequence the mount was last materialized at (`status.lastMaterializedSeq`), which is saved + only at a clean stop: after a restart that lost it, a bare `materialize` took up the whole tree. + A file the workspace hasn't changed is left as disk has it, with or without `force`, unless its + path is given. A path the evidence says nothing about is treated the same way. Where disk holds + a different copy (`unknown-provenance`, or `store-origin-collision` for a file the workspace + created), only a `materialize` that names the path with `force` overwrites it. Where an entry + from before the evidence has no disk copy, only one that names the path writes it, and a `sync` + of the path drops it from the workspace instead. A bare `materialize` trusts the evidence: if + disk was replaced behind the workspace, a `sync` takes disk's copies, and naming the paths with + `force` writes the workspace's. `status.pendingChanges` counts what disk owes by the evidence; + an entry never checked against disk isn't counted until a listing or a `materialize` checks it. +- **Undo and branch switches are not mistaken for disk edits.** What disk last agreed with is kept + in global chronicle records rather than branch state, so selecting an earlier state keeps that + state as a draft over the later disk version, and a file present only in another branch's + history is listed `not-in-branch`, never ingested or unlinked. Each write or unlink a + materialize makes, and each adoption that ingests a disk change or removes a file, is recorded + as an intent ahead of it and completed once it is durable, so a crash or a torn record tail + never costs an edit. If an append of disk-agreement evidence fails inside Chronicle, the + workspace refuses operations that need that evidence and reports why. Those operations remain + unavailable until the host restarts and reloads the store's records. The append may have + reached the log without becoming readable in this process, so deciding a conflict from + possibly incomplete evidence would be unsafe. A binary or oversize disk version, for which the workspace drops its copy, + forgets the evidence first instead. After a crash, an interrupted write or unlink is decided by what disk shows (a copy + matching neither the one before nor the one intended is an `interrupted` conflict), and one + that shows on disk without proof it is durable is completed only by the next `materialize`, + which redoes it; an interrupted adoption is settled by its own branch's tree. This replaces + #109's stop-time baselines and refused paths, + which an abrupt exit or a branch switch could lose or rewind. A store that saved them doesn't + import them, since they name no root: its first pass records each file disk agrees with, and + lists any other as a conflict or of unknown provenance. +- **`watch: 'on-agent-action'` works as documented:** after each completed tool batch, before + anything continues the agent's turn, the mount is scanned. A scan that outlasts + `agentActionScanDeadlineMs` stops holding the round and continues in the background; the miss + is reported on `status` (`lastAgentActionScan`) and as a `workspace:agent-action-scan-incomplete` + event, as are the regions a finished scan couldn't observe (`incomplete`). Batches that come + while a scan waits share it, and while a scan that outlasted its deadline still runs, a later + batch isn't held: its miss is recorded at once, and its scan comes after. The hold is in the + framework's event loop, so while it lasts every agent's events wait. +- Disk text is stored as its raw bytes rather than through a UTF-8 round trip, so a file in + another encoding no longer differs from its own stored copy. +- `status` with `mount` reports that mount alone, and refuses one that doesn't exist. A + `materialize` that names a read-only mount lists it under `skipped`, rather than answering as + if nothing was owed there. +- `materialize` and `autoMaterialize` check each write and unlink where it happens, not only when + it is planned. A file is opened in its directory without truncating and located before any + byte changes: never through a symlink the mount doesn't follow, or a directory that resolves + outside the mount, and never into a file with other hard links, whose other names (a package + store's, say) a write in place would change too, even with `force`. Unless `force`, it must + also still hold what the materialize decided on, so a disk edit or a new file that appears + meanwhile is refused, not overwritten. Refused paths are listed as skipped with the reason. + Where the platform can sync a directory (not Windows, nor a filesystem that refuses it), every + directory from a written file up to the mount root is synced before the write counts as done, + directories it had to create included. Listings likewise accept a directory's contents only + while it is still the directory at its path, and a scan, write or unlink accepts what it found + only while the mount root is still the one it began under: a scan that finds the root replaced + when it looks again treats what it observed as unverified, and reports the root. Each check is made where the operation happens, + against what is there at that moment. Node has no `openat`, so a parent directory replaced + concurrently in the window between a check and a create, `mkdir`, unlink or listing can still + redirect that one operation: an empty file or directory can appear outside the mount, or a + same-named file there can be unlinked. Nor can a write or an unlink be bound to the moment of + its last check: an edit to the open file between the last look and the truncate can be + overwritten, and a file put at the path, or an edit made to it, between the last look and the + unlink can be removed. Content writes stay bound to the + descriptor verified, inside the mount and with no other hard links, before modification; a + hard link made to the file after that check shares the write. diff --git a/src/framework.ts b/src/framework.ts index f82e3e8df..174b9eb80 100644 --- a/src/framework.ts +++ b/src/framework.ts @@ -6694,6 +6694,44 @@ export class AgentFramework { result: event.result, }); } + // The round's last pending result: let modules catch up with what the + // batch's tools did (the workspace scans `on-agent-action` mounts) before + // anything continues the turn. Bounded and fail-open; the agent may be + // reset or cancelled meanwhile, so its state is checked again after. + if ( + agent && agent.state.status === 'waiting_for_tools' + && agent.state.pending.size === 1 && agent.state.pending.has(event.callId) + ) { + // The call this result answers, as this round holds it: a new round + // that reuses the call's id holds a new entry for it. + const owed = agent.state.pending.get(event.callId); + const failures = await this.moduleRegistry.notifyToolBatchComplete(event.agentName); + for (const failure of failures) { + this.emitTrace({ type: 'module:batch_hook_failed', agentName: event.agentName, module: failure.module, error: failure.error }); + } + // The turn may have been reset, cancelled or replaced while the hook + // ran, even back to waiting on a new batch, whose calls may reuse this + // one's id: this result is still owed only if this agent still waits + // on this round's call. + const after = agent.state as AgentState; + if ( + this.agents.get(event.agentName) !== agent + || after.status !== 'waiting_for_tools' || after.pending.get(event.callId) !== owed + ) { + console.warn( + `[framework] Dropping tool-result for agent '${event.agentName}' (callId=${event.callId}): ` + + `its turn changed while modules caught up with the batch (status '${after.status}').`, + ); + this.emitTrace({ + type: 'tool:result_dropped', + agentName: event.agentName, + callId: event.callId, + agentStatus: (agent.state as AgentState).status, + result: event.result, + }); + return; + } + } if (agent && agent.state.status === 'waiting_for_tools') { agent.provideToolResult(event.callId, event.result); diff --git a/src/module-registry.ts b/src/module-registry.ts index 8c56934e4..061107ed0 100644 --- a/src/module-registry.ts +++ b/src/module-registry.ts @@ -47,6 +47,15 @@ export function isStateExistsError(err: unknown): boolean { */ const MAX_CONTEXT_TIMEOUT_MS = 30_000; +/** + * Bound on each module's onToolBatchComplete. The hook runs inside the + * framework's sequential event loop, so a wedged module would stall every + * agent's events, not just its own round; past this the round goes on without + * it. A module with its own semantic deadline (the workspace scan's) sets it + * below this one, so it can record the miss itself. + */ +export const TOOL_BATCH_HOOK_TIMEOUT_MS = 30_000; + /** * Registered speech handler. */ @@ -61,6 +70,9 @@ interface SpeechHandler { * Registry for managing modules. */ export class ModuleRegistry { + /** Each module's tool-batch hook still running, past its bound or not. */ + private batchHooksRunning = new Map>(); + private modules: Map = new Map(); private stopping = new Map>(); private moduleContexts: Map = new Map(); @@ -415,6 +427,57 @@ export class ModuleRegistry { return injections; } + /** + * Tell modules an agent's tool batch completed (see + * Module.onToolBatchComplete). Hooks run in parallel, each awaited up to + * `timeoutMs`; a hook that throws or times out is logged and skipped, so a + * wedged module can't hold the round. Returns the modules that didn't + * finish, for the caller's trace. + */ + async notifyToolBatchComplete( + agentName: string, + timeoutMs = TOOL_BATCH_HOOK_TIMEOUT_MS, + ): Promise> { + const failures: Array<{ module: string; error: string }> = []; + const promises: Promise[] = []; + for (const module of this.modules.values()) { + if (!module.onToolBatchComplete) continue; + // A hook still running from an earlier batch (it outlasted its bound) + // isn't called again or waited for: a wedged module costs one bound, + // not one per batch. + if (this.batchHooksRunning.has(module.name)) { + failures.push({ module: module.name, error: 'still running from an earlier batch' }); + continue; + } + // Called from a promise, so a hook that throws synchronously, before + // returning its promise, is caught below like any other failure + // instead of escaping the round (the result would never be provided). + const call = Promise.resolve().then(() => module.onToolBatchComplete!(agentName)); + this.batchHooksRunning.set(module.name, call); + const settled = (): void => { + if (this.batchHooksRunning.get(module.name) === call) this.batchHooksRunning.delete(module.name); + }; + call.then(settled, settled); + let timer: ReturnType | undefined; + promises.push( + Promise.race([ + call, + new Promise((_, rej) => { + timer = setTimeout(() => rej(new Error(`onToolBatchComplete timed out after ${timeoutMs}ms`)), timeoutMs); + }), + ]) + .catch((err: unknown) => { + const error = err instanceof Error ? err.message : String(err); + failures.push({ module: module.name, error }); + console.error(`[${module.name}] onToolBatchComplete:`, err); + }) + .finally(() => clearTimeout(timer)), + ); + } + await Promise.all(promises); + return failures; + } + /** * Dispatch speech to registered handlers. */ diff --git a/src/modules/workspace/branch-intent.ts b/src/modules/workspace/branch-intent.ts new file mode 100644 index 000000000..b2ee77b48 --- /dev/null +++ b/src/modules/workspace/branch-intent.ts @@ -0,0 +1,123 @@ +/** + * Branch-local intent about workspace paths: what the workspace on THIS branch + * deliberately did or still owes, kept in a per-mount Chronicle tree state so + * it rewinds with the workspace tree it qualifies (a branch switch that brings + * back a deleted file brings back its missing tombstone too). + * + * - tombstone: the workspace deleted the path while disk kept a copy (no + * autoMaterialize, or a refused unlink). The hash is the deleted entry's, + * so even a legacy entry with no disk evidence keeps a durable intent. + * - origin 'store': the entry was created through workspace tools and disk + * has never agreed with it. + * - conflict: disk and workspace diverged in a way no rule may resolve + * silently; kept until a path sync, a forced materialize, or convergence. + * + * Each record is a small JSON blob addressed from the tree, so reading one is + * treeGet + getBlob and writing one is storeBlob + treeSet. + */ + +import type { JsStore } from '@animalabs/chronicle'; + +export type ConflictKind = + /** Disk, workspace and the last agreement all differ. */ + | 'both-changed' + /** Disk deleted a file whose workspace copy differs from the last agreement. */ + | 'deleted-on-disk' + /** Disk changed a file the workspace deleted. */ + | 'changed-after-workspace-delete' + /** Disk holds a file at the path of a workspace-created entry, with other content. */ + | 'store-origin-collision' + /** A legacy entry whose disk copy differs, with no evidence of which side changed. */ + | 'unknown-provenance' + /** A module write or adoption was interrupted, and disk holds neither outcome. */ + | 'interrupted'; + +/** The disk side of a conflict, as observed when it was recorded. */ +export interface DiskCounterpart { + hash: string; + size: number; + mtimeMs: number; + /** Blob hash of the stored copy (text within limits); absent when only referenced. */ + stored?: string; +} + +export interface ConflictRecord { + kind: ConflictKind; + /** When the conflict was recorded (ms). */ + at: number; + /** null when disk had no file. */ + disk: DiskCounterpart | null; +} + +export interface BranchIntent { + tombstone?: { hash: string }; + origin?: 'store'; + conflict?: ConflictRecord; + /** + * Why a recorded intent can't be read (its blob missing from the store, or + * not an intent): a tombstone or conflict it held is unknown, never absent. + * Never written; a put replaces the damaged record. + */ + unreadable?: string; +} + +function isEmpty(intent: BranchIntent): boolean { + return intent.tombstone === undefined && intent.origin === undefined && intent.conflict === undefined && + intent.unreadable === undefined; +} + +/** A recorded intent's blob, read; a damaged one says why it can't be. */ +function readIntent(store: JsStore, blobHash: string): BranchIntent { + const blob = store.getBlob(blobHash); + if (!blob) return { unreadable: "its record's blob is missing from the store" }; + try { + const parsed = JSON.parse(blob.toString('utf-8')) as unknown; + if (parsed && typeof parsed === 'object' && !Array.isArray(parsed)) return parsed as BranchIntent; + } catch { + // reported below + } + return { unreadable: 'its record is not a branch intent' }; +} + +export class BranchIntents { + constructor(private readonly store: JsStore, readonly treeStateId: string) {} + + get(path: string): BranchIntent | null { + const entry = this.store.treeGet(this.treeStateId, path); + if (!entry) return null; + return readIntent(this.store, entry.blobHash); + } + + /** Replace a path's intent; an empty intent removes the record. */ + put(path: string, intent: BranchIntent | null): void { + if (intent?.unreadable !== undefined) { + const { unreadable: _damaged, ...rest } = intent; + intent = rest; + } + if (!intent || isEmpty(intent)) { + if (this.store.treeGet(this.treeStateId, path)) this.store.treeRemove(this.treeStateId, path); + return; + } + const bytes = Buffer.from(JSON.stringify(intent), 'utf-8'); + const blobHash = this.store.storeBlob(bytes, 'application/json'); + this.store.treeSet(this.treeStateId, path, { blobHash, size: bytes.byteLength, mode: 0 }); + } + + /** Update one path's intent in place. */ + update(path: string, change: (current: BranchIntent) => BranchIntent): void { + this.put(path, change({ ...(this.get(path) ?? {}) })); + } + + /** Every recorded intent under a prefix ('' for the whole mount). */ + list(prefix = ''): Array<[string, BranchIntent]> { + const out: Array<[string, BranchIntent]> = []; + for (const entry of this.store.treeList(this.treeStateId, prefix ? prefix + '/' : undefined)) { + out.push([entry.path, readIntent(this.store, entry.blobHash)]); + } + if (prefix) { + const exact = this.get(prefix); + if (exact) out.push([prefix, exact]); + } + return out; + } +} diff --git a/src/modules/workspace/disk-agreement.ts b/src/modules/workspace/disk-agreement.ts new file mode 100644 index 000000000..ea9c7aceb --- /dev/null +++ b/src/modules/workspace/disk-agreement.ts @@ -0,0 +1,325 @@ +/** + * Physical disk-agreement evidence (P): for each mount and path, what disk held + * when it last agreed with the workspace store, or what the module last wrote + * there or adopted from it. + * + * P describes disk, and disk does not rewind with a branch switch, so P lives + * in global chronicle records (RecordJournal) rather than branch state: after + * an undo selects an older tree, P still says what disk holds, and a + * difference reads as a store-side change, never as a disk edit. Branch-local + * intent — tombstones, store origin, conflicts — lives in BranchIntents and + * rewinds with the tree. + * + * P is evidence about one directory, so each mount also records the identity + * (device and inode) of the root that evidence was gathered under. A root + * that isn't that directory any more (an unmounted drive whose mountpoint is + * now empty, a replaced directory) is an unavailable mount, never a disk on + * which every file was deleted; only an explicit acceptance takes it, and + * that starts its evidence afresh. + * + * Every transition is write-ahead (see reconcile.ts): an `intent` entry + * (`pending`, with the prior value and the expected one) is durable before the + * disk effect or tree adoption it announces, and the completion `set` is + * appended only after that effect is durable. One tree adoption has no + * intent: a drop, which removes the store's copy of a disk version the + * workspace can't hold, forgets P first, so a crash before the removal leaves + * P unknown, never a wrong adoption. A pending adoption resolves at its next + * observation, by the tree. A pending disk effect resolves there only when + * disk shows it didn't happen (the prior) or can't tell (`interrupted`, a + * sticky conflict resolved explicitly, or by disk and the store coming to + * agree); one that shows on disk is completed only by a push whose barriers + * succeed, since its completion releases a draft's protection. + */ + +import { resolve } from 'node:path'; +import type { JsStore } from '@animalabs/chronicle'; +import { RecordJournal } from '../../record-journal.js'; + +/** Everything `stat` says about a file that can vouch for unchanged bytes. */ +export interface Fingerprint { + size: number; + mtimeNs: string; + ctimeNs: string; + ino: string; + dev: string; + /** When the bytes were hashed (ms). Timestamps within 2 s of it can't vouch. */ + hashedAt: number; +} + +export type Agreed = + | { kind: 'absent' } + | { kind: 'content'; hash: string; size: number; fp?: Fingerprint }; + +export type Candidate = { kind: 'absent' } | { kind: 'content'; hash: string }; + +export interface Pending { + kind: 'pending'; + /** What the module is about to do: change disk, or adopt disk into the tree. */ + effect: 'disk' | 'adopt'; + /** P before the intent (null: unknown). */ + prior: Agreed | null; + expect: Candidate; + /** Branch the intent was recorded on (an adoption's meaning is branch-relative). */ + branchId: string; +} + +export interface Interrupted { + kind: 'interrupted'; + /** The values disk could have held after the interrupted effect. */ + candidates: Candidate[]; +} + +export type Physical = Agreed | Pending | Interrupted; + +/** Which directory a mount root is: its device and inode, as decimal strings. */ +export interface RootIdentity { + dev: string; + ino: string; +} + +/** Whether two configured roots name the same path, whatever their spelling + * (a trailing slash, a relative form). A symlinked alias is another path. */ +function sameRootPath(a: string, b: string): boolean { + return resolve(a) === resolve(b); +} + +export function sameRootIdentity(a: RootIdentity, b: RootIdentity): boolean { + return a.dev === b.dev && a.ino === b.ino; +} + +type JournalEntry = + | { t: 'mount'; mount: string; root: string } + /** The root's identity; with `reset`, an accepted new root, whose evidence starts empty. */ + | { t: 'root'; mount: string; identity: RootIdentity; reset?: true } + | { t: 'set'; mount: string; path: string; value: Physical } + | { t: 'forget'; mount: string; path: string }; + +interface MountEvidence { + root: string; + identity?: RootIdentity; + paths: Map; +} + +interface Snapshot { + mounts: Record }>; +} + +export const DISK_AGREEMENT_RECORD_TYPE = 'workspace/disk-agreement'; + +/** Checkpoint once the tail outgrows this many entries, or the live paths, whichever is larger. */ +const MIN_CHECKPOINT_TAIL = 1000; + +export function candidateOf(value: Agreed): Candidate { + return value.kind === 'absent' ? { kind: 'absent' } : { kind: 'content', hash: value.hash }; +} + +export function sameCandidate(a: Candidate, b: Candidate): boolean { + return a.kind === b.kind && (a.kind === 'absent' || a.hash === (b as { hash: string }).hash); +} + +export class DiskAgreement { + private readonly store: JsStore; + private readonly journal: RecordJournal; + private readonly mounts = new Map(); + private configured: Array<{ name: string; root: string }> = []; + /** + * An append (or a rebuild) not yet made durable. Cleared only by a + * barrier that succeeds, so a failed barrier stays owed: a later pass with + * nothing new to record still settles it before deciding anything. + */ + private unsynced = false; + + constructor(store: JsStore) { + this.store = store; + this.journal = new RecordJournal(store, { type: DISK_AGREEMENT_RECORD_TYPE }); + } + + /** + * Rebuild P from the journal for the configured mounts. A mount whose + * recorded root differs from its configured one starts empty: evidence + * about another directory says nothing about this one. + */ + open(mounts: Array<{ name: string; root: string }>): void { + this.configured = mounts; + this.rebuild(); + } + + /** + * Re-read the journal when an earlier write left it unreconciled, and throw + * once an append failed inside the store. P may then lack what that append + * recorded (the store may hold it where this process can't read it back), + * and a decision from stale P can record what reopening contradicts, such + * as a conflict that stays after the right evidence is back. So until the + * store is reopened, no evidence is read, written or checkpointed, and + * every pass and tool call that needs it fails. + */ + ensureReconciled(): void { + if (this.journal.needsReopen) { + throw new Error( + 'Disk-agreement evidence is unavailable until the store is reopened, as at the next start: ' + + 'an append to its journal failed inside the store, and the evidence may lack what it recorded', + ); + } + if (this.journal.needsReconcile) this.rebuild(); + } + + /** Whether evidence recorded so far still awaits a barrier. */ + get needsBarrier(): boolean { + return this.unsynced; + } + + /** + * Replay the journal into memory. Records that read back are not proven + * durable — a previous owner may have appended them and died before any + * barrier — so evidence rebuilt here owes one, whether at open or after an + * ambiguous append. + */ + private rebuild(): void { + this.unsynced = true; + const { snapshot, entries } = this.journal.load(); + const replayed = new Map(); + for (const [name, mount] of Object.entries(snapshot?.mounts ?? {})) { + replayed.set(name, { + root: mount.root, + ...(mount.identity ? { identity: mount.identity } : {}), + paths: new Map(Object.entries(mount.paths)), + }); + } + for (const { entry } of entries) { + if (entry.t === 'mount') { + const existing = replayed.get(entry.mount); + if (!existing || !sameRootPath(existing.root, entry.root)) replayed.set(entry.mount, { root: entry.root, paths: new Map() }); + continue; + } + const mount = replayed.get(entry.mount); + if (!mount) continue; + if (entry.t === 'root') applyRoot(mount, entry); + else if (entry.t === 'set') mount.paths.set(entry.path, entry.value); + else mount.paths.delete(entry.path); + } + this.mounts.clear(); + for (const { name, root } of this.configured) { + const found = replayed.get(name); + if (found && sameRootPath(found.root, root)) { + this.mounts.set(name, found); + } else { + this.mounts.set(name, { root, paths: new Map() }); + this.journal.append({ t: 'mount', mount: name, root }); + } + } + } + + /** The identity of the root this mount's evidence was gathered under, once any was. */ + rootIdentity(mount: string): RootIdentity | undefined { + this.ensureReconciled(); + return this.mounts.get(mount)?.identity; + } + + /** + * Record the root's identity just before the first evidence a mount gains: + * the caller saw it as the root while gathering that evidence. A no-op once + * one is recorded; changing it is acceptRoot's alone. + */ + recordRootIdentity(mount: string, identity: RootIdentity): void { + if (this.rootIdentity(mount) !== undefined) return; + this.write({ t: 'root', mount, identity }, {}); + } + + /** + * Take the root as it is now, at the operator's word: record its identity + * and forget every path's evidence, which described another directory. + */ + acceptRoot(mount: string, identity: RootIdentity): void { + this.write({ t: 'root', mount, identity, reset: true }, {}); + } + + get(mount: string, path: string): Physical | undefined { + this.ensureReconciled(); // an ambiguous append may have landed: read it back first + return this.mounts.get(mount)?.paths.get(path); + } + + /** Every path with evidence under a prefix ('' for the whole mount). */ + paths(mount: string, prefix = ''): Array<[string, Physical]> { + this.ensureReconciled(); + const paths = this.mounts.get(mount)?.paths; + if (!paths) return []; + const out: Array<[string, Physical]> = []; + for (const [path, value] of paths) { + if (prefix === '' || path === prefix || path.startsWith(prefix + '/')) out.push([path, value]); + } + return out; + } + + /** + * Record an intent. `durable` makes it (and every append before it) stable + * before the caller performs the effect; batches pass it on their last one. + */ + intend(mount: string, path: string, intent: Omit, opts: { durable?: boolean } = {}): void { + this.write({ t: 'set', mount, path, value: { kind: 'pending', ...intent } }, { durable: opts.durable }); + } + + /** + * Record agreed evidence. `afterCommittedState` when it asserts a tree + * commit made just before (the commit is synced first). + */ + set(mount: string, path: string, value: Agreed | Interrupted, opts: { afterCommittedState?: boolean } = {}): void { + this.write({ t: 'set', mount, path, value }, { afterCommittedState: opts.afterCommittedState }); + } + + /** Forget a path: P becomes unknown, the most conservative evidence. */ + forget(mount: string, path: string): void { + if (this.get(mount, path) === undefined) return; + this.write({ t: 'forget', mount, path }, {}); + } + + /** + * Put back exactly the evidence a path had before an intent whose effect + * never touched it — whatever kind it was (agreed, pending, interrupted, or + * unknown) — so the intent leaves no trace in what P says. + */ + restore(mount: string, path: string, value: Physical | undefined): void { + if (value === undefined) this.forget(mount, path); + else this.write({ t: 'set', mount, path, value }, {}); + } + + private write(entry: JournalEntry, opts: { durable?: boolean; afterCommittedState?: boolean }): void { + this.ensureReconciled(); + this.unsynced = true; // until proven durable: a failed append may still have landed + this.journal.append(entry, opts); + if (opts.durable) this.unsynced = false; // synced after this append, and so everything before it + const mount = this.mounts.get((entry as { mount: string }).mount); + if (!mount) return; + if (entry.t === 'root') applyRoot(mount, entry); + else if (entry.t === 'set') mount.paths.set(entry.path, entry.value); + else if (entry.t === 'forget') mount.paths.delete(entry.path); + } + + /** Checkpoint when the tail has outgrown the live evidence. */ + maybeCheckpoint(): void { + // Like every entry point: an ambiguous write is read back first. + this.ensureReconciled(); + let live = 0; + for (const mount of this.mounts.values()) live += mount.paths.size; + if (this.journal.entriesSinceCheckpoint < Math.max(MIN_CHECKPOINT_TAIL, live)) return; + const snapshot: Snapshot = { mounts: {} }; + for (const [name, mount] of this.mounts) { + snapshot.mounts[name] = { + root: mount.root, + ...(mount.identity ? { identity: mount.identity } : {}), + paths: Object.fromEntries(mount.paths), + }; + } + this.journal.checkpoint(snapshot); + } + + /** Make every append so far stable. Throws, and stays owed, if the sync fails. */ + barrier(): void { + this.store.sync(); + this.unsynced = false; + } +} + +function applyRoot(mount: MountEvidence, entry: Extract): void { + mount.identity = entry.identity; + if (entry.reset) mount.paths.clear(); +} diff --git a/src/modules/workspace/effects.ts b/src/modules/workspace/effects.ts new file mode 100644 index 000000000..5e280fe5c --- /dev/null +++ b/src/modules/workspace/effects.ts @@ -0,0 +1,362 @@ +/** + * The workspace's disk effects — a materialize's writes and unlinks, and the + * barriers that make them durable — under the mount boundary. + * + * Each effect is bound at its point of use, not at planning time: the parent + * directory is checked (located inside the mount, see observe.ts), the file is + * reached through it, and before anything is changed the effect checks that + * it acts on what was decided on — a regular file inside the mount, at this + * path, still holding the bytes planning saw (unless `force`, which overrides + * that freshness but never the boundary). Content is written only through a + * descriptor whose own location was confirmed inside the mount first, and + * only into a file with no other hard links: a write in place changes the + * file under every name it has, and the others may lie outside the mount (a + * package store's, say), so such a file is refused. Unlinking one name + * changes no other, so an unlink needs no such check. + * + * What an effect reports matters to the caller's evidence: a failure that + * never touched the path's entry leaves disk as planning saw it, so the + * path's evidence can be put back as it was; once the entry was touched + * (created, truncated, written or unlinked) the outcome is for the next + * observation to resolve. Directories created for a write don't touch the + * path's entry; they stay, empty, if the write is refused. + * + * Node has no openat, so a create, mkdir or unlink still resolves its path by + * name after the parent was checked: a parent replaced concurrently in that + * window can redirect that one operation, leaving an empty file or directory + * outside the mount or unlinking a same-named file there. Nor is there an + * unlink bound to an inode: an unlink checks the entry's identity again just + * before it, and a file replaced at its path after that check is the one + * unlinked. Content writes stay bound to the descriptor verified before + * modification: inside the mount, with no other hard links (a link another + * process makes to it after that check shares the write). The hash, which + * reads the whole file, comes before the checks of where the file is, and + * the root is checked again beside the parent, since the parent is found by + * path beneath the root's. A last look at the file (its descriptor's stat + * before a write, its entry's lstat before an unlink) refuses one written + * since it was first seen, so only an edit made in the moment between that + * look and the effect is overwritten or removed. + */ + +import { createHash } from 'node:crypto'; +import { constants as fsConstants } from 'node:fs'; +import { lstat, open, realpath, stat, unlink } from 'node:fs/promises'; +import type { FileHandle } from 'node:fs/promises'; +import { + type CheckedDir, + type MountView, + NONBLOCK, + NOT_FOLLOWED, + OUTSIDE_PARENT, + ROOT_REPLACED, + contained, + errno, + checkDir, + checkRoot, + nameOf, + parentOf, +} from './observe.js'; + +/** What a write or unlink may find at its path when it acts. */ +export type Expect = + | { kind: 'absent' } + /** The bytes planning decided on, by hash. */ + | { kind: 'content'; hash: string } + /** Anything (force): freshness is overridden, the boundary is not. */ + | { kind: 'any' }; + +/** Errors with which an unlink refuses outright, leaving the entry as it was. */ +const UNLINK_REFUSALS = new Set(['EACCES', 'EPERM', 'EROFS', 'EBUSY']); + +export class EffectFailed extends Error { + /** Whether the path's entry was touched before the failure. */ + readonly touched: boolean; + + constructor(message: string, touched: boolean) { + super(message); + this.touched = touched; + } +} + +const CHANGED = 'disk changed since it was checked'; +const MOVED = 'the directory changed since it was checked'; +const HARD_LINKED = 'a file with other hard links, which writing it in place would change too — force does not override ' + + 'this; to write this path alone, give it a copy of its own first (copy it to a new name, then move that over it) ' + + 'and materialize again'; + +function message(err: unknown): string { + return err instanceof Error ? err.message : String(err); +} + +function noFollowFlag(view: MountView): number { + return !view.followSymlinks && typeof fsConstants.O_NOFOLLOW === 'number' ? fsConstants.O_NOFOLLOW : 0; +} + +/** + * The parent of a path, checked for an effect; proven absent; or why neither; + * with the root. The root comes first: still a directory, and still the one + * disk last agreed under, as any observation of it checks. An effect under a + * replaced root acts on nothing, whatever its caller checked before, and the + * parent is found by path beneath the root's path, so whoever acts checks the + * root again with the parent (`moved`): a root replaced after it was + * checked would otherwise lend the effect its replacement's directory. + */ +async function parentFor(view: MountView, rootReal: string, rel: string, create: boolean) { + const root = await checkRoot(view, rootReal); + if ('reason' in root) throw new EffectFailed(root.reason, false); + const checked = await checkDir(view, rootReal, parentOf(rel), { create }); + if (checked.kind === 'unobserved') throw new EffectFailed(checked.outside ? OUTSIDE_PARENT : checked.reason, false); + return { ...checked, root: root.dir }; +} + +/** + * Why the parent, or the root it was found beneath, is no longer the + * directory its path names; null while each still is. The root is asked + * first, so a replaced root reads the same wherever it's found. + */ +async function moved(root: CheckedDir, parent: CheckedDir): Promise { + if (!(await root.stillHere())) return ROOT_REPLACED; + return (await parent.stillHere()) ? null : MOVED; +} + +/** A file as a stat describes it: which file, and the size and stamps a write changes. */ +type Stamped = { dev: bigint; ino: bigint; size: bigint; mtimeNs: bigint; ctimeNs: bigint }; + +/** + * Whether the file at `target` is still the one `before` described: the same + * file, with the same size and stamps, so nothing wrote it since. Read just + * before an effect, after the slower checks, so an edit made while the file + * was hashed is never overwritten or deleted unseen; what remains is the + * moment between this and the effect, which Node can't close (see the header). + */ +function unchanged(before: Stamped, now: Stamped | null): boolean { + return now !== null && now.dev === before.dev && now.ino === before.ino && now.size === before.size + && now.mtimeNs === before.mtimeNs && now.ctimeNs === before.ctimeNs; +} + +/** The hash of what `target` holds, read through a descriptor that must be the file `id` names. */ +async function hashAt(target: string, id: { dev: bigint; ino: bigint }, flags: number): Promise { + let handle: FileHandle; + try { + handle = await open(target, fsConstants.O_RDONLY | NONBLOCK | flags); + } catch (err) { + throw new EffectFailed(`cannot verify the disk copy (${errno(err) ?? message(err)})`, false); + } + try { + const info = await handle.stat({ bigint: true }); + if (info.dev !== id.dev || info.ino !== id.ino) throw new EffectFailed(CHANGED, false); + const hash = createHash('sha256'); + const chunk = Buffer.alloc(1 << 20); + for (let offset = 0; ;) { + const { bytesRead } = await handle.read(chunk, 0, chunk.length, offset); + if (bytesRead === 0) break; + hash.update(chunk.subarray(0, bytesRead)); + offset += bytesRead; + } + return hash.digest('hex'); + } finally { + await handle.close(); + } +} + +/** + * Write `bytes` at a mount-relative path, creating missing parent + * directories, and make the file durable (its directories are synced by the + * caller, after all of a push's effects: syncDirectories). + */ +export async function writeContained(view: MountView, rootReal: string, rel: string, bytes: Buffer, expect: Expect): Promise { + const checked = await parentFor(view, rootReal, rel, true); + if (checked.kind !== 'checked') throw new EffectFailed('a parent path is not a directory', false); + const parent = checked.dir; + const root = checked.root; + const target = parent.at(nameOf(rel)); + const noFollow = noFollowFlag(view); + if (!view.followSymlinks && !noFollow) { + // No O_NOFOLLOW (Windows): refuse a symlink by lstat before opening. + if ((await lstat(target).catch(() => null))?.isSymbolicLink()) throw new EffectFailed(NOT_FOLLOWED, false); + } + + let touched = false; + const create = async (): Promise => { + try { + // Exclusive: never follows a symlink, never takes over a file that + // appeared since planning saw none. + const handle = await open(target, fsConstants.O_WRONLY | fsConstants.O_CREAT | fsConstants.O_EXCL, 0o644); + touched = true; + return handle; + } catch (err) { + // O_EXCL refuses any entry: a file, a directory or a link. + if (errno(err) === 'EEXIST') throw new EffectFailed('something appeared at this path since it was checked', false); + throw new EffectFailed(`cannot create the file (${errno(err) ?? message(err)})`, false); + } + }; + let handle: FileHandle; + if (expect.kind === 'absent') { + handle = await create(); + } else { + try { + // Opened without truncating: nothing changes until it is located. + handle = await open(target, fsConstants.O_WRONLY | NONBLOCK | noFollow); + } catch (err) { + const code = errno(err); + if (code === 'ENOENT' && expect.kind === 'any') handle = await create(); + else if (code === 'ENOENT') throw new EffectFailed('the file was removed since it was checked', false); + else if (code === 'ELOOP' && !view.followSymlinks) throw new EffectFailed(NOT_FOLLOWED, false); + else if (code === 'EISDIR') throw new EffectFailed('a directory', false); + else if (code === 'ENXIO') throw new EffectFailed('not a regular file', false); // a FIFO with no reader + else throw new EffectFailed(`cannot open the file (${code ?? message(err)})`, false); + } + } + + let stage = 'write the file'; + try { + // Before any byte: a regular file, and — unless forced — still holding + // what was decided on. The hash reads the whole file, so it comes first, + // and the checks of where the file is follow it. + const info = await handle.stat({ bigint: true }); + if (!info.isFile()) throw new EffectFailed('not a regular file', touched); + if (expect.kind === 'content' && (await hashAt(target, info, noFollow)) !== expect.hash) throw new EffectFailed(CHANGED, touched); + // Then: canonically inside the mount, the one this path names (and not a + // symlink, where the mount follows none and the open couldn't refuse + // one), with no other names, in the directory still at its path beneath + // the root still at its own. + const real = await realpath(target).catch(() => null); + if (real === null) throw new EffectFailed(CHANGED, touched); + if (!contained(rootReal, real)) throw new EffectFailed(view.followSymlinks ? 'a symlink that leaves the mount' : OUTSIDE_PARENT, touched); + const named = await stat(real, { bigint: true }).catch(() => null); + if (named === null || named.dev !== info.dev || named.ino !== info.ino) throw new EffectFailed(CHANGED, touched); + if (!view.followSymlinks) { + const entry = await lstat(target, { bigint: true }).catch(() => null); + if (entry?.isSymbolicLink()) throw new EffectFailed(NOT_FOLLOWED, touched); + if (entry === null || !entry.isFile() || entry.dev !== info.dev || entry.ino !== info.ino) throw new EffectFailed(CHANGED, touched); + } + if (info.nlink > 1n) throw new EffectFailed(HARD_LINKED, touched); + const gone = await moved(root, parent); + if (gone !== null) throw new EffectFailed(gone, touched); + // Nothing wrote it while those ran. + if (!unchanged(info, await handle.stat({ bigint: true }).catch(() => null))) throw new EffectFailed(CHANGED, touched); + + touched = true; + await handle.truncate(0); + await handle.writeFile(bytes); + stage = 'make the write durable (file fsync)'; + await handle.sync(); + } catch (err) { + if (err instanceof EffectFailed) throw err; + throw new EffectFailed(`could not ${stage}: ${message(err)}`, touched); + } finally { + await handle.close(); + } +} + +/** + * Unlink a mount-relative file, or confirm it gone (`absent` expected: a + * file found there is never removed). Returns the mount directory whose + * entries make the absence durable, for the caller's barrier: the file's + * parent, or — when the parent itself is gone — the directory proven to lack + * it, since that removal may not be durable either. + */ +export async function unlinkContained(view: MountView, rootReal: string, rel: string, expect: Expect): Promise { + const checked = await parentFor(view, rootReal, rel, false); + if (checked.kind === 'absent') { + // No directory, so no file: proven beneath a root still at its path. + if (!(await checked.root.stillHere())) throw new EffectFailed(ROOT_REPLACED, false); + return checked.by; + } + const parent = checked.dir; + const root = checked.root; + const target = parent.at(nameOf(rel)); + let info; + try { + info = await lstat(target, { bigint: true }); + } catch (err) { + if (errno(err) === 'ENOENT') { + const gone = await moved(root, parent); + if (gone !== null) throw new EffectFailed(gone, false); + return parent.rel; + } + throw new EffectFailed(`cannot stat (${errno(err) ?? message(err)})`, false); + } + // Confirming an unlink that already happened never removes a file that + // appeared since. + if (expect.kind === 'absent') { + const kind = info.isDirectory() ? 'a directory' : info.isSymbolicLink() ? 'a symlink' : 'a file'; + throw new EffectFailed(`${kind} appeared at this path since it was checked`, false); + } + let file = info; + if (info.isSymbolicLink()) { + // The link itself is what is removed; a followed one must still lie inside. + if (!view.followSymlinks) throw new EffectFailed(NOT_FOLLOWED, false); + const real = await realpath(target).catch(() => null); + if (real === null || !contained(rootReal, real)) throw new EffectFailed('a symlink that leaves the mount', false); + const through = await stat(real, { bigint: true }).catch(() => null); + if (through === null) throw new EffectFailed(CHANGED, false); + file = through; + } else if (info.isDirectory()) { + throw new EffectFailed('a directory', false); + } + if (expect.kind === 'content' && (await hashAt(target, file, noFollowFlag(view))) !== expect.hash) throw new EffectFailed(CHANGED, false); + const gone = await moved(root, parent); + if (gone !== null) throw new EffectFailed(gone, false); + // The entry checked is still the one at the path, unwritten since it was + // hashed, as close to the unlink as Node allows (see the header): a file + // replaced or edited while it was hashed isn't removed. + const again = await lstat(target, { bigint: true }).catch(() => null); + const named = again?.isSymbolicLink() ? await stat(target, { bigint: true }).catch(() => null) : again; + if (!unchanged(file, named)) throw new EffectFailed(CHANGED, false); + try { + await unlink(target); + } catch (err) { + const code = errno(err); + // An unlink is all or nothing: one refused outright changed nothing, so + // the path's evidence can go back as it was. Anything else may have + // landed (a network filesystem's EIO, say): the next observation resolves it. + if (code !== 'ENOENT') throw new EffectFailed(`could not unlink the file: ${message(err)}`, !UNLINK_REFUSALS.has(code ?? '')); + } + return parent.rel; +} + +/** + * Sync each mount directory (mount-relative), deepest first, so the entries + * a push created or removed in them are durable. Each is checked first and the + * directory it holds is the one synced — never a replacement at its path. + * Returns the ones that failed, with why. Where directories can't be synced + * at all (Windows, or a filesystem answering EINVAL/ENOTSUP), there is + * nothing to wait for. + */ +export async function syncDirectories(view: MountView, rootReal: string, rels: Iterable): Promise> { + const failed = new Map(); + const depth = (rel: string): number => (rel === '' ? 0 : rel.split('/').length); + const sorted = [...new Set(rels)].sort((a, b) => depth(b) - depth(a)); + // Each directory is found by path beneath the root's path: under a root + // that isn't the one disk agreed under, or that is replaced while they're + // synced, none of them is the mount's. + const root = await checkRoot(view, rootReal); + if ('reason' in root) { + for (const rel of sorted) failed.set(rel, root.reason); + return failed; + } + for (const rel of sorted) { + const checked = await checkDir(view, rootReal, rel); + if (checked.kind !== 'checked') { + failed.set(rel, checked.kind === 'absent' ? 'the directory is gone' : checked.reason); + continue; + } + try { + await checked.dir.sync(); + if (!(await root.dir.stillHere())) failed.set(rel, ROOT_REPLACED); + } catch (err) { + failed.set(rel, message(err)); + } + } + return failed; +} + +/** The directories whose entries make a written file reachable: its parent up to the mount root. */ +export function chainOf(rel: string): string[] { + const chain: string[] = []; + for (let dir = parentOf(rel); ; dir = parentOf(dir)) { + chain.push(dir); + if (dir === '') return chain; + } +} diff --git a/src/modules/workspace/index.ts b/src/modules/workspace/index.ts index c6b7d8564..192dd990c 100644 --- a/src/modules/workspace/index.ts +++ b/src/modules/workspace/index.ts @@ -7,12 +7,13 @@ import { constants as fsConstants } from 'node:fs'; import type { Stats } from 'node:fs'; -import { open, readFile, stat, access, writeFile, unlink, mkdir, lstat, realpath } from 'node:fs/promises'; -import { join, resolve, relative, dirname, sep } from 'node:path'; +import { open, readFile, stat, lstat, realpath } from 'node:fs/promises'; +import { join, resolve, relative, sep } from 'node:path'; import type { JsStore } from '@animalabs/chronicle'; import type { Module, ModuleContext, ProcessState, EventResponse } from '../../types/module.js'; import type { ProcessEvent, ToolDefinition, ToolCall, ToolResult } from '../../types/events.js'; import type { + AgentActionScan, WorkspaceConfig, MountConfig, MountState, @@ -35,7 +36,31 @@ import type { } from './types.js'; import { WORKSPACE_FS_EVENT_TYPES, opToEventType } from './types.js'; import { MountWatcher, type FsChange } from './watcher.js'; -import { syncFromFs, materializeToFs, hashContent, isBinary, DEFAULT_MAX_FILE_SIZE, type ConflictInfo } from './sync.js'; +import { hashContent, DEFAULT_MAX_FILE_SIZE } from './sync.js'; +import { DiskAgreement, sameRootIdentity } from './disk-agreement.js'; +import { BranchIntents } from './branch-intent.js'; +import { filesystemTrustsCtime, locateRoot } from './observe.js'; +import { TOOL_BATCH_HOOK_TIMEOUT_MS } from '../../module-registry.js'; +import { + type MountRuntime, + type PassOptions, + type PassResult, + type PathReport, + type PushOptions, + type PushResult, + type Scope, + type TreeOp, + pushPaths, + reconcilePass, + settleAdoptionBeforeMutation, +} from './reconcile.js'; + +/** Default for how long the after-batch scan may hold the agent's next inference. */ +const AGENT_ACTION_SCAN_DEADLINE_MS = 20_000; +/** Why a call is refused once the module has stopped. */ +const NOT_RUNNING = 'The workspace has stopped: it takes no more work until it starts again'; +/** The workspace's own deadline must end before the framework's hook bound. */ +const AGENT_ACTION_SCAN_DEADLINE_MAX_MS = TOOL_BATCH_HOOK_TIMEOUT_MS - 1_000; export type { WorkspaceConfig, @@ -603,9 +628,22 @@ export class WorkspaceModule implements Module { private ctx: ModuleContext | null = null; private store: JsStore | null = null; + /** Set once stop has drained and saved, cleared by start (running()). */ + private stopped = false; private config: WorkspaceConfig; private mounts = new Map(); private watchers = new Map(); + /** What disk last agreed with (global records); created with the store. */ + private agreement: DiskAgreement | null = null; + /** Branch-local intent per mount. */ + private intents = new Map(); + /** Each mount's turn: the tail of its serialized passes and writes. */ + private mountTurns = new Map>(); + /** Whether an over-bound agentActionScanDeadlineMs has been warned about. */ + private warnedScanDeadline = false; + /** Whether a mount root's filesystem maintains ctime, by canonical root. */ + /** Per canonical root: the device of its filesystem when that maintains ctime, else null. */ + private ctimeTrust = new Map(); /** Persisted state decoded in start(), held until mounts exist to apply it * to — in the Host ordering start() runs before initStore() creates the * mounts, so restoration must be second-callback-safe like watcher @@ -651,41 +689,119 @@ export class WorkspaceModule implements Module { initStore(store: JsStore): void { this.store = store; - // Register tree states for each mount + // Register tree states for each mount: the workspace tree, and the + // branch-local intent that qualifies it. for (const mount of this.config.mounts) { const treeStateId = `workspace/${mount.name}/tree`; - try { - store.registerState({ - id: treeStateId, - strategy: 'tree', - deltaSnapshotEvery: this.config.deltaSnapshotEvery ?? 50, - fullSnapshotEvery: this.config.fullSnapshotEvery ?? 10, - }); - } catch { - // State already registered (restart scenario) + const intentTreeStateId = `workspace/${mount.name}/intent`; + for (const id of [treeStateId, intentTreeStateId]) { + try { + store.registerState({ + id, + strategy: 'tree', + deltaSnapshotEvery: this.config.deltaSnapshotEvery ?? 50, + fullSnapshotEvery: this.config.fullSnapshotEvery ?? 10, + }); + } catch { + // State already registered (restart scenario) + } } const mountState: MountState = { config: mount, treeStateId, + intentTreeStateId, lastMaterializedSeq: 0, suppressedPaths: new Set(), initialSyncDone: false, lastMaterializedBranchId: null, - materializedHashes: new Map(), - refusedPaths: new Set(), watcherReadyAt: null, watcherError: null, }; this.mounts.set(mount.name, mountState); + this.intents.set(mount.name, new BranchIntents(store, intentTreeStateId)); } + // What disk last agreed with, per mount and path: global records, so a + // branch switch never rewinds it. + this.agreement = new DiskAgreement(store); + this.agreement.open(this.config.mounts.map((m) => ({ name: m.name, root: resolve(m.path) }))); + this.applySavedState(); this.ensureRunning(); } + // ========================================================================== + // Reconciliation plumbing + // ========================================================================== + + /** + * Run `fn` with the mount to itself: observation, decision and effect never + * interleave with another pass or a tool write on the same mount (a disk + * observation taken before an autoMaterialize write and applied after it + * would ingest old bytes over the new entry). Not re-entrant: code holding + * a mount's turn calls the unlocked helpers. + */ + private withMount(mount: MountState, fn: () => Promise): Promise { + const name = mount.config.name; + const previous = this.mountTurns.get(name) ?? Promise.resolve(); + const run = previous.then(fn, fn); + this.mountTurns.set(name, run.then(() => undefined, () => undefined)); + return run; + } + + private agreementOrThrow(): DiskAgreement { + if (!this.agreement) throw new Error('Workspace store not initialized'); + return this.agreement; + } + + /** The view reconciliation needs of a mount, refreshed per pass. */ + private async runtime(mount: MountState): Promise { + const root = resolve(mount.config.path); + const rootReal = await realpath(root).catch(() => root); + let trustedDev = this.ctimeTrust.get(rootReal); + if (trustedDev === undefined) { + const dev = (await stat(rootReal, { bigint: true }).catch(() => null))?.dev; + trustedDev = dev !== undefined && (await filesystemTrustsCtime(rootReal)) ? dev.toString() : null; + this.ctimeTrust.set(rootReal, trustedDev); + } + return { + name: mount.config.name, + view: { + root, + rootIdentity: this.agreementOrThrow().rootIdentity(mount.config.name), + followSymlinks: mount.config.followSymlinks === true, + maxFileSize: mount.config.maxFileSize ?? DEFAULT_MAX_FILE_SIZE, + ignore: mount.config.ignore ?? [], + }, + rootReal, + ctimeTrustedDev: trustedDev, + treeStateId: mount.treeStateId, + intents: this.intents.get(mount.config.name)!, + readOnly: mount.config.mode === 'read-only', + }; + } + + /** One disk→store pass; the caller holds the mount's turn. Emits its events. */ + private async passUnlocked(mount: MountState, scope: Scope, opts: PassOptions = {}): Promise { + const result = await reconcilePass(this.getStore(), this.agreementOrThrow(), await this.runtime(mount), scope, opts); + if (scope.kind === 'dir' && scope.dir === '' && scope.recursive) mount.initialSyncDone = true; + this.emitFsEvents(mount.config.name, result.ops, result.newConflicts); + return result; + } + + /** One store→disk push; the caller holds the mount's turn. */ + private async pushUnlocked(mount: MountState, paths: string[], opts: PushOptions = {}): Promise { + const watcher = this.watchers.get(mount.config.name); + return pushPaths(this.getStore(), this.agreementOrThrow(), await this.runtime(mount), paths, { + ...opts, + beforeEffect: (p) => watcher?.suppress(p), + }); + } + async start(ctx: ModuleContext): Promise { this.ctx = ctx; + this.stopped = false; // Decode persisted state if restarting; application waits until mounts // exist (see applySavedState) — in the Host ordering they don't yet. @@ -710,15 +826,18 @@ export class WorkspaceModule implements Module { for (const [name, meta] of Object.entries(this.savedState.mounts)) { const mount = this.mounts.get(name); - if (mount) { + // A mount materialized this session already (a call between initStore + // and start) has a pin and watermark newer than anything saved. + if (mount && mount.lastMaterializedBranchId === null && mount.lastMaterializedSeq === 0) { mount.lastMaterializedSeq = meta.lastMaterializedSeq; mount.lastMaterializedBranchId = meta.lastMaterializedBranchId ?? null; - // Freshness-guard baselines: without them the first materialize after - // a restart can't tell a shell edit from our own last write (#109). - for (const [path, hash] of Object.entries(meta.materializedHashes ?? {})) { - if (!mount.materializedHashes.has(path)) mount.materializedHashes.set(path, hash); - } - for (const path of meta.refusedPaths ?? []) mount.refusedPaths.add(path); + // A store that ran #169's stop-time baselines (#109) saved them here + // too. They aren't carried over as evidence: #169 recorded no root, + // so nothing binds them to the directory they describe, and an empty + // mountpoint at the next start would read as every file deleted. + // Such a store gains its evidence from its first pass instead: a + // file disk agrees with is recorded, and anything else is a conflict + // or unknown provenance, never an adoption. // watcherReadyAt intentionally not restored — each session must // observe its own watcher attach, otherwise a stale timestamp // would hide a new-session attach failure. @@ -862,12 +981,10 @@ export class WorkspaceModule implements Module { watcher.start(); this.watchers.set(name, watcher); - // Chokidar is started with ignoreInitial:true, so files already on - // disk at session start would be invisible. Trigger one syncFromFs - // pass — syncFromFs diffs disk against the tree state, so only files - // new-to-this-session's-tree fire workspace:created events. Fresh - // sessions see the existing catalog; restarts only see what appeared - // while the session was down. + // Chokidar is started with ignoreInitial:true, so changes made while + // the session was down would be invisible. One full pass catches up + // under the three-way rule: only real differences produce tree + // changes and events. void this.initialScan(name); } @@ -885,9 +1002,9 @@ export class WorkspaceModule implements Module { if (!store || !mount) return; try { - const result = await syncFromFs(store, mount); - mount.initialSyncDone = true; - this.emitFsEvents(mountName, result.synced, result.conflicts); + // A full pass under the three-way rule: deletions on disk are confirmed + // where nothing newer is pending, and nothing unvisited is touched. + await this.withMount(mount, () => this.passUnlocked(mount, { kind: 'dir', dir: '', recursive: true })); } catch (err) { const msg = err instanceof Error ? err.message : String(err); this.ctx?.pushEvent({ @@ -899,6 +1016,24 @@ export class WorkspaceModule implements Module { } async stop(): Promise { + // No new watcher-driven passes, then let every pass and write finish + // before the store goes away — including any queued behind the ones in + // flight (a tool call, a scan continuing past its deadline): drained + // until no new tail appears. + for (const watcher of this.watchers.values()) { + await watcher.stop(); + } + this.watchers.clear(); + for (;;) { + const tails = [...this.mountTurns.values()]; + await Promise.all(tails); + const now = [...this.mountTurns.values()]; + if (now.length === tails.length && now.every((tail, i) => tail === tails[i])) break; + } + // Nothing is queued now, and nothing more is taken: what follows saves + // the pins and watermarks as the last work left them. + this.stopped = true; + // Persist state if (this.ctx) { const activeBranchId = this.store ? this.store.currentBranch().id : undefined; @@ -907,20 +1042,12 @@ export class WorkspaceModule implements Module { state.mounts[name] = { lastMaterializedSeq: mount.lastMaterializedSeq, lastMaterializedBranchId: mount.lastMaterializedBranchId ?? undefined, - materializedHashes: Object.fromEntries(mount.materializedHashes), - refusedPaths: [...mount.refusedPaths], watcherReadyAt: mount.watcherReadyAt, watcherError: mount.watcherError, }; } this.ctx.setState(state); } - - // Stop watchers - for (const watcher of this.watchers.values()) { - await watcher.stop(); - } - this.watchers.clear(); this.ctx = null; } @@ -997,7 +1124,9 @@ export class WorkspaceModule implements Module { }, { name: 'delete', - description: 'Delete a file from the workspace.', + description: 'Delete a file from the workspace. On a mount without autoMaterialize the disk copy stays, ' + + 'listed as workspace-deleted (no scan brings it back), until materialize with applyDeletions removes it ' + + 'or a sync of the path restores it.', inputSchema: { type: 'object' as const, properties: { @@ -1008,7 +1137,14 @@ export class WorkspaceModule implements Module { }, { name: 'ls', - description: 'List directory contents from the workspace tree.', + description: 'List a directory, checked against disk first. Each file has a state: synced; ' + + 'workspace-draft (the workspace has changes disk lacks); workspace-deleted (deleted in the workspace, ' + + 'still on disk); not-in-branch (on disk, absent from this branch\'s workspace); disk-only (binary or over ' + + 'the size limit, never stored — shown with size and image type); conflict, with its kind (disk and the ' + + 'workspace both changed; disk holds a different copy and nothing records which is newer; or a write was ' + + 'interrupted) — sync the path to take disk, or materialize that path with force to take the workspace; ' + + 'disk-missing-provenance-unknown; unverified (disk could not be checked). `incomplete` names regions ' + + 'the listing could not see.', inputSchema: { type: 'object' as const, properties: { @@ -1019,7 +1155,7 @@ export class WorkspaceModule implements Module { }, { name: 'glob', - description: 'Find files matching a glob pattern in the workspace.', + description: 'Find files matching a glob pattern, checked against disk first; each match carries its state, as ls describes.', inputSchema: { type: 'object' as const, properties: { @@ -1031,7 +1167,11 @@ export class WorkspaceModule implements Module { }, { name: 'grep', - description: 'Search file contents with a regex pattern.', + description: 'Search stored file text with a regex pattern, checked against disk first. Each result says ' + + 'which version matched: the workspace\'s (with its state); the disk version kept with a conflict ' + + '(conflicting-disk); or, once disk has changed since the conflict was recorded, that recorded version ' + + '(recorded-disk). Disk versions it could not search are listed as skipped: disk-only files (binary or ' + + 'over the size limit), and a conflict\'s disk version that is binary, oversize, or newer than the one recorded.', inputSchema: { type: 'object' as const, properties: { @@ -1056,24 +1196,45 @@ export class WorkspaceModule implements Module { }, { name: 'materialize', - description: 'Write workspace files to the real filesystem. Use after writing/editing to push changes to disk.', + description: 'Write workspace files to the real filesystem. Use after writing/editing to push changes to disk. ' + + 'Never deletes disk files unless applyDeletions is given; lists workspace deletions it left on disk.', inputSchema: { type: 'object' as const, properties: { - path: { type: 'string', description: 'Specific path to materialize (optional — defaults to all changed)' }, + path: { type: 'string', description: 'Specific path to materialize (optional — defaults to what disk still owes: workspace edits, conflicts, workspace deletions, and files never checked against disk)' }, mount: { type: 'string', description: 'Specific mount (optional)' }, - force: { type: 'boolean', description: 'Overwrite files whose disk copy changed since the last materialize (another writer), and materialize even if the current branch has diverged from the branch last written to disk (default false). Without it, divergent files are skipped and listed.' }, + force: { type: 'boolean', description: 'Overwrite files whose disk copy changed since it last agreed with the workspace (another writer, or a conflict), and materialize even if the current branch has diverged from the branch last written to disk (default false). Without it, divergent files are skipped and listed. A file the workspace has not changed is left as disk has it unless you name its path.' }, + applyDeletions: { type: 'boolean', description: 'Also delete from disk the files deleted in the workspace whose disk copy is one the workspace holds (default false). With force, also those whose disk copy changed since.' }, }, }, }, { name: 'sync', - description: 'Pull filesystem state into the workspace. Detects user changes on disk.', + description: 'Bring disk changes into the workspace. Without a path, every mount (or the given one) is ' + + 'rechecked in full and nothing pending in the workspace is discarded. With a path (file or directory), ' + + 'the workspace takes disk\'s state there explicitly: it restores a workspace-deleted file, resolves ' + + 'conflicts toward disk, replaces a workspace draft, and removes a workspace file disk does not have. ' + + 'Each workspace change it gives up is listed under `discarded`, with the state it had and, as `op`, ' + + 'what that did to the workspace copy (absent when it changed none: a deletion or conflict given up for ' + + 'a disk copy the workspace can\'t hold). ' + + 'A mount whose root is not the directory disk last agreed with (an unmounted drive, a replaced ' + + 'directory) is unavailable and listed under `incomplete`. Reconnect the drive; acceptRoot is for a ' + + 'directory replaced on purpose, or a drive back under a new device number, never for one still ' + + 'unplugged, whose empty mountpoint would become the root.', inputSchema: { type: 'object' as const, properties: { - path: { type: 'string', description: 'Specific path to sync (optional — defaults to all)' }, + path: { type: 'string', description: 'Specific path to take from disk (optional — defaults to a full recheck)' }, mount: { type: 'string', description: 'Specific mount (optional)' }, + acceptRoot: { + type: 'boolean', + description: 'Take each mount\'s root as it is now when it is not the directory disk last agreed with: ' + + 'a directory replaced on purpose, or a drive that came back under a new device number (default false). ' + + 'What was recorded about the old one is set aside, so every file is compared afresh: a file disk ' + + 'and the workspace agree on agrees again, a differing one is a conflict, and a workspace file the ' + + 'root lacks stays in the workspace. Not with path. Leave it off while a drive is merely unplugged: ' + + 'its empty mountpoint would become the root.', + }, }, }, }, @@ -1117,6 +1278,7 @@ export class WorkspaceModule implements Module { } async handleToolCall(call: ToolCall): Promise { + if (!this.running()) return { success: false, isError: true, error: NOT_RUNNING }; try { const input = call.input as Record; const generated = typeof input?.path === 'string' ? this.findGeneratedTextFile(input.path) : undefined; @@ -1251,6 +1413,7 @@ export class WorkspaceModule implements Module { data: Buffer, mimeType: string, ): Promise { + if (!this.running()) return { success: false, isError: true, error: NOT_RUNNING }; let mount: MountState; let relativePath: string; try { @@ -1270,24 +1433,22 @@ export class WorkspaceModule implements Module { return { success: false, error: `Content exceeds max file size (${maxSize} bytes)`, isError: true }; } const store = this.getStore(); - const blobHash = store.storeBlob(data, mimeType); - store.treeSet(mount.treeStateId, relativePath, { - blobHash, - size: data.byteLength, - mode: 0o644, - }); - const materializeError = await this.autoMaterialize(mount, relativePath, 'write', data); - if (materializeError) { + return this.withMount(mount, async () => { + await this.settleBeforeMutation(mount, relativePath); + const blobHash = store.storeBlob(data, mimeType); + const materializeError = await this.commitToolChange(mount, relativePath, { kind: 'set', blobHash, size: data.byteLength }); + if (materializeError) { + return { + success: false, + error: `Wrote to Chronicle but failed to materialize "${mountPrefixedPath}" to disk: ${materializeError}.`, + isError: true, + }; + } return { - success: false, - error: `Wrote to Chronicle but failed to materialize "${mountPrefixedPath}" to disk: ${materializeError}.`, - isError: true, + success: true, + data: { path: mountPrefixedPath, size: data.byteLength, mimeType }, }; - } - return { - success: true, - data: { path: mountPrefixedPath, size: data.byteLength, mimeType }, - }; + }); } /** @@ -1298,6 +1459,7 @@ export class WorkspaceModule implements Module { async readBinary( mountPrefixedPath: string, ): Promise<{ data: Buffer } | { error: string }> { + if (!this.running()) return { error: NOT_RUNNING }; let mount: MountState; let relativePath: string; try { @@ -1306,12 +1468,14 @@ export class WorkspaceModule implements Module { return { error: error instanceof Error ? error.message : String(error) }; } const store = this.getStore(); - await this.ensureSynced(mount, relativePath); - const entry = store.treeGet(mount.treeStateId, relativePath); - if (!entry) return { error: `File not found: ${mountPrefixedPath}` }; - const blob = store.getBlob(entry.blobHash); - if (!blob) return { error: `Blob not found for: ${mountPrefixedPath}` }; - return { data: blob }; + return this.withMount(mount, async () => { + await this.ensureSynced(mount, relativePath); + const entry = store.treeGet(mount.treeStateId, relativePath); + if (!entry) return { error: `File not found: ${mountPrefixedPath}` }; + const blob = store.getBlob(entry.blobHash); + if (!blob) return { error: `Blob not found for: ${mountPrefixedPath}` }; + return { data: blob }; + }); } /** @@ -1380,6 +1544,18 @@ export class WorkspaceModule implements Module { return { mount, relativePath }; } + /** + * Whether the module takes work: stop hasn't finished. Work admitted while + * stop drains still runs; once stop has saved the pins and watermarks, + * nothing more changes them, so it takes nothing until it starts again. A + * call before start is taken, as it always was (a generated file's read + * needs no store at all): in the host's order, start runs first, and + * initStore applies the saved pins. + */ + private running(): boolean { + return !this.stopped; + } + private getStore(): JsStore { if (!this.store) throw new Error('WorkspaceModule: store not initialized. Call initStore() first.'); return this.store; @@ -1701,55 +1877,64 @@ export class WorkspaceModule implements Module { } /** - * Persist a single write/edit/delete to disk when the mount opts in via - * `autoMaterialize`. Required for cross-agent pipelines — another agent's - * chokidar watcher on the same directory only sees real filesystem events. - * The local watcher's `suppress()` absorbs the echo so we don't self-wake. + * Commit a tool's write or delete to the workspace tree, with the branch + * intent it implies (store origin for a path disk never agreed with; a + * tombstone for a deletion), and push it to disk when the mount opts in via + * `autoMaterialize` — required for cross-agent pipelines, since another + * agent's watcher on the same directory only sees real filesystem events. + * The push follows materialize's rule: a disk copy that changed since it + * last agreed is not overwritten or unlinked, and the refusal says why. * - * Returns an error string on failure so callers can surface it in the tool - * result. The autoMaterialize contract is specifically "disk is the source - * of truth for downstream agents", so a silent disk-write failure with a - * Chronicle commit would defeat the whole purpose. No-op cases (flag off, - * read-only mount) return null (success). + * Returns the reason when the change reached the workspace but not disk, so + * callers surface it in the tool result: the autoMaterialize contract is + * that disk is the source of truth for downstream agents. The caller holds + * the mount's turn and has settled the path first (settleBeforeMutation). */ - private async autoMaterialize( + private async commitToolChange( mount: MountState, relativePath: string, - op: 'write' | 'delete', - content?: Buffer, + change: { kind: 'set'; blobHash: string; size: number; mode?: number } | { kind: 'remove'; previousHash: string }, ): Promise { - if (!mount.config.autoMaterialize) return null; - if (mount.config.mode === 'read-only') return null; - - const absolutePath = join(mount.config.path, relativePath); - const watcher = this.watchers.get(mount.config.name); - watcher?.suppress(relativePath); - - try { - if (op === 'write' && content) { - await mkdir(dirname(absolutePath), { recursive: true }); - await writeFile(absolutePath, content); - mount.materializedHashes.set(relativePath, hashContent(content)); - } else if (op === 'delete') { - try { - await unlink(absolutePath); - } catch (err) { - if ((err as NodeJS.ErrnoException).code !== 'ENOENT') throw err; - } - mount.materializedHashes.delete(relativePath); - } - return null; - } catch (err) { - const msg = err instanceof Error ? err.message : String(err); - this.ctx?.pushEvent({ - type: 'workspace:materialize-failed', - mount: mount.config.name, - path: relativePath, - op, - error: msg, - } as ProcessEvent); - return msg; + const store = this.getStore(); + const intents = this.intents.get(mount.config.name)!; + // The mutation's precondition, enforced here, synchronously with the + // commit: whatever settleBeforeMutation's pass could or couldn't settle. + settleAdoptionBeforeMutation(store, this.agreementOrThrow(), { name: mount.config.name, treeStateId: mount.treeStateId }, relativePath); + const known = this.agreementOrThrow().get(mount.config.name, relativePath) !== undefined; + if (change.kind === 'set') { + store.treeSet(mount.treeStateId, relativePath, { blobHash: change.blobHash, size: change.size, mode: change.mode ?? 0o644 }); + intents.update(relativePath, (cur) => { + const next = { ...cur }; + delete next.tombstone; + // Disk has never agreed with this path: a missing disk copy is a draft + // not yet materialized, not a deletion. + if (!known) next.origin = 'store'; + return next; + }); + } else { + store.treeRemove(mount.treeStateId, relativePath); + intents.update(relativePath, (cur) => { + const next = { ...cur, tombstone: { hash: change.previousHash } }; + delete next.origin; + return next; + }); } + if (!mount.config.autoMaterialize || mount.config.mode === 'read-only') return null; + + const pushed = await this.pushUnlocked(mount, [relativePath], { + applyDeletions: change.kind === 'remove', + branchId: store.currentBranch().id, + }); + const refused = pushed.skipped.find((s) => s.path === relativePath); + if (!refused) return null; + this.ctx?.pushEvent({ + type: 'workspace:materialize-failed', + mount: mount.config.name, + path: relativePath, + op: change.kind === 'set' ? 'write' : 'delete', + error: refused.reason, + } as ProcessEvent); + return refused.reason; } // ========================================================================== @@ -1757,36 +1942,49 @@ export class WorkspaceModule implements Module { // ========================================================================== /** - * Ensure a file is synced from filesystem if not yet in tree (lazy sync). + * Before a tool reads or changes a path's workspace entry, a pending intent + * on that path is settled by a pass over it, so the tool works from what + * disk now holds (an edit sees a disk change the settled adoption took in). + * The pass may not settle it (a branch that keeps changing, an unreadable + * disk copy); the precondition that matters is enforced again, + * synchronously, where the change commits (commitToolChange). The caller + * holds the mount's turn. */ - private async ensureSynced(mount: MountState, relativePath: string): Promise { - const store = this.getStore(); - const existing = store.treeGet(mount.treeStateId, relativePath); - if (existing) return; // Already in tree + private async settleBeforeMutation(mount: MountState, relativePath: string): Promise { + if (this.agreementOrThrow().get(mount.config.name, relativePath)?.kind !== 'pending') return; + await this.passUnlocked(mount, { kind: 'paths', paths: [relativePath] }); + } - // Try to read from filesystem - const absolutePath = join(mount.config.path, relativePath); - try { - const maxSize = mount.config.maxFileSize ?? DEFAULT_MAX_FILE_SIZE; - const fileStat = await stat(absolutePath); - if (!fileStat.isFile() || fileStat.size > maxSize) return; - - const buffer = await readFile(absolutePath); - // Binaries are never synced into the tree (mirrors syncFromFs): the old - // utf-8 round-trip here silently replaced non-UTF-8 bytes with U+FFFD, - // permanently mangling images in the blob store. Binary reads are served - // straight from disk (readImageFromFilesystem / read_image fallback). - if (isBinary(buffer)) return; - const content = buffer.toString('utf-8'); - const blobHash = store.storeBlob(Buffer.from(content, 'utf-8'), 'text/plain'); - - store.treeSet(mount.treeStateId, relativePath, { - blobHash, - size: buffer.length, - mode: 0o644, - }); - } catch { - // File doesn't exist on disk — that's fine + /** + * Before a read: a path the workspace doesn't hold is observed under the + * three-way rule, which ingests a new disk file but never brings back a + * workspace deletion or a file absent from this branch. The caller holds + * the mount's turn. + */ + private async ensureSynced(mount: MountState, relativePath: string): Promise { + const store = this.getStore(); + if (store.treeGet(mount.treeStateId, relativePath)) return undefined; + const pass = await this.passUnlocked(mount, { kind: 'paths', paths: [relativePath] }); + return pass.reports.get(relativePath); + } + + /** Why the workspace holds no entry at a path, when disk explains it. */ + private notInWorkspace(path: string, report: PathReport | undefined): string { + switch (report?.state) { + case 'workspace-deleted': + return `${path} was deleted in the workspace; disk still has a copy. Sync this path to restore it.`; + case 'not-in-branch': + return `${path} is not in this branch's workspace; disk has a copy. Sync this path to add it.`; + case 'disk-only': + return `${path} is on disk but not stored in the workspace (binary or over the size limit)` + + (report.mimeType?.startsWith('image/') ? '; read it with read_image.' : '.'); + case 'conflict': + return `${path} is in conflict: the workspace deleted it and disk changed it since. ` + + 'Sync this path to take the disk version.'; + case 'unverified': + return `${path} is not in the workspace, and disk could not be checked (${report.note ?? 'not observed'}).`; + default: + return `File not found: ${path}`; } } @@ -1810,22 +2008,18 @@ export class WorkspaceModule implements Module { } let content = generatedContent; if (content === undefined) { - const { mount, relativePath } = this.parsePath(input.path); - const store = this.getStore(); - - await this.ensureSynced(mount, relativePath); - - const entry = store.treeGet(mount.treeStateId, relativePath); - if (!entry) { - return { success: false, error: `File not found: ${input.path}`, isError: true }; - } - - const blob = store.getBlob(entry.blobHash); - if (!blob) { - return { success: false, error: `Blob not found for: ${input.path}`, isError: true }; - } - - content = blob.toString('utf-8'); + const { mount, relativePath } = this.parsePath(input.path); + const store = this.getStore(); + const found = await this.withMount(mount, async (): Promise<{ text: string } | { error: string }> => { + const report = await this.ensureSynced(mount, relativePath); + const entry = store.treeGet(mount.treeStateId, relativePath); + if (!entry) return { error: this.notInWorkspace(input.path, report) }; + const blob = store.getBlob(entry.blobHash); + if (!blob) return { error: `Blob not found for: ${input.path}` }; + return { text: blob.toString('utf-8') }; + }); + if ('error' in found) return { success: false, error: found.error, isError: true }; + content = found.text; } if (characterPaging) { const start = Math.min(offsetChars, content.length); @@ -1967,13 +2161,11 @@ export class WorkspaceModule implements Module { } const buffer = Buffer.from(input.content, 'utf-8'); - const blobHash = store.storeBlob(buffer, 'text/plain'); - store.treeSet(mount.treeStateId, relativePath, { - blobHash, - size: Buffer.byteLength(input.content), - mode: 0o644, + const materializeError = await this.withMount(mount, async () => { + await this.settleBeforeMutation(mount, relativePath); + const blobHash = store.storeBlob(buffer, 'text/plain'); + return this.commitToolChange(mount, relativePath, { kind: 'set', blobHash, size: buffer.byteLength }); }); - const materializeError = await this.autoMaterialize(mount, relativePath, 'write', buffer); if (materializeError) { return { success: false, @@ -1999,62 +2191,65 @@ export class WorkspaceModule implements Module { } const store = this.getStore(); - await this.ensureSynced(mount, relativePath); + return this.withMount(mount, async (): Promise => { + const report = await this.ensureSynced(mount, relativePath); + await this.settleBeforeMutation(mount, relativePath); // before the edit reads the entry - const entry = store.treeGet(mount.treeStateId, relativePath); - if (!entry) { - return { success: false, error: `File not found: ${input.path}`, isError: true }; - } + const entry = store.treeGet(mount.treeStateId, relativePath); + if (!entry) { + return { success: false, error: this.notInWorkspace(input.path, report), isError: true }; + } - const blob = store.getBlob(entry.blobHash); - if (!blob) { - return { success: false, error: `Blob not found for: ${input.path}`, isError: true }; - } + const blob = store.getBlob(entry.blobHash); + if (!blob) { + return { success: false, error: `Blob not found for: ${input.path}`, isError: true }; + } - let content = blob.toString('utf-8'); + let content = blob.toString('utf-8'); - // Validate uniqueness - if (!input.replaceAll) { - const count = content.split(input.oldString).length - 1; - if (count === 0) { - return { success: false, error: `String not found in ${input.path}`, isError: true }; + // Validate uniqueness + if (!input.replaceAll) { + const count = content.split(input.oldString).length - 1; + if (count === 0) { + return { success: false, error: `String not found in ${input.path}`, isError: true }; + } + if (count > 1) { + return { + success: false, + error: `String found ${count} times in ${input.path}. Use replaceAll: true or provide more context.`, + isError: true, + }; + } } - if (count > 1) { + + content = input.replaceAll + ? content.replaceAll(input.oldString, input.newString) + : content.replace(input.oldString, input.newString); + + const newBuffer = Buffer.from(content, 'utf-8'); + const newBlobHash = store.storeBlob(newBuffer, 'text/plain'); + const materializeError = await this.commitToolChange(mount, relativePath, { + kind: 'set', + blobHash: newBlobHash, + size: newBuffer.byteLength, + mode: entry.mode, + }); + if (materializeError) { return { success: false, - error: `String found ${count} times in ${input.path}. Use replaceAll: true or provide more context.`, + error: `Edited Chronicle but failed to materialize "${input.path}" to disk: ${materializeError}. Downstream agents will not see this change until the next materialize call.`, isError: true, }; } - } - - content = input.replaceAll - ? content.replaceAll(input.oldString, input.newString) - : content.replace(input.oldString, input.newString); - const newBuffer = Buffer.from(content, 'utf-8'); - const newBlobHash = store.storeBlob(newBuffer, 'text/plain'); - store.treeSet(mount.treeStateId, relativePath, { - blobHash: newBlobHash, - size: Buffer.byteLength(content), - mode: entry.mode, - }); - const materializeError = await this.autoMaterialize(mount, relativePath, 'write', newBuffer); - if (materializeError) { return { - success: false, - error: `Edited Chronicle but failed to materialize "${input.path}" to disk: ${materializeError}. Downstream agents will not see this change until the next materialize call.`, - isError: true, + success: true, + data: { + path: input.path, + size: newBuffer.byteLength, + }, }; - } - - return { - success: true, - data: { - path: input.path, - size: Buffer.byteLength(content), - }, - }; + }); } private async handleDelete(input: DeleteInput): Promise { @@ -2064,22 +2259,26 @@ export class WorkspaceModule implements Module { } const store = this.getStore(); - const entry = store.treeGet(mount.treeStateId, relativePath); - if (!entry) { - return { success: false, error: `File not found: ${input.path}`, isError: true }; - } + return this.withMount(mount, async (): Promise => { + const report = await this.ensureSynced(mount, relativePath); + await this.settleBeforeMutation(mount, relativePath); + const entry = store.treeGet(mount.treeStateId, relativePath); + if (!entry) { + return { success: false, error: this.notInWorkspace(input.path, report), isError: true }; + } - store.treeRemove(mount.treeStateId, relativePath); - const materializeError = await this.autoMaterialize(mount, relativePath, 'delete'); - if (materializeError) { - return { - success: false, - error: `Removed from Chronicle but failed to unlink "${input.path}" from disk: ${materializeError}. Downstream agents will still see the stale file until manual cleanup.`, - isError: true, - }; - } + const materializeError = await this.commitToolChange(mount, relativePath, { kind: 'remove', previousHash: entry.blobHash }); + if (materializeError) { + return { + success: false, + error: `Removed from Chronicle but did not unlink "${input.path}" from disk: ${materializeError}. ` + + 'The disk copy stays (ls shows its state) until materialize with applyDeletions removes it.', + isError: true, + }; + } - return { success: true, data: { path: input.path, deleted: true } }; + return { success: true, data: { path: input.path, deleted: true } }; + }); } private async handleLs(input: LsInput): Promise { @@ -2096,63 +2295,55 @@ export class WorkspaceModule implements Module { } const { mount, relativePath } = this.parsePath(input.path); + const recursive = input.recursive === true; - // Ensure initial sync — always sync on first access regardless of watch mode, - // so that ls/glob/grep see filesystem contents even for unwatched mounts - if (!mount.initialSyncDone) { - await syncFromFs(store, mount); - mount.initialSyncDone = true; - } + // Every listing checks the paths it shows against disk first, under the + // three-way rule, whatever the mount's watch mode. + return this.withMount(mount, async (): Promise => { + const pass = await this.passUnlocked(mount, { kind: 'dir', dir: relativePath, recursive }); + const reports = [...pass.reports.values()].sort((a, b) => a.path.localeCompare(b.path)); + const incomplete = pass.incomplete.length > 0 ? { incomplete: pass.incomplete } : {}; - const prefix = relativePath ? relativePath + '/' : ''; - const entries = store.treeList(mount.treeStateId, prefix || undefined); + if (recursive) { + return { + success: true, + data: { + path: input.path, + entries: reports.map((r) => ({ path: r.path, ...stateView(r) })), + count: reports.length, + ...incomplete, + }, + }; + } - if (input.recursive) { + // Non-recursive: the scope's files, and the directories beneath it that + // disk or the workspace has. + const prefix = relativePath ? relativePath + '/' : ''; + const dirNames = new Set(pass.dirs.map((d) => d.slice(prefix.length))); + for (const entry of store.treeList(mount.treeStateId, prefix || undefined)) { + const slash = entry.path.indexOf('/', prefix.length); + if (slash >= 0) dirNames.add(entry.path.slice(prefix.length, slash)); + } + const children: Array> = [ + ...[...dirNames].sort().map((name) => ({ name, type: 'directory' })), + ...reports.map((r) => ({ name: r.path.slice(prefix.length), type: 'file', ...stateView(r) })), + ]; return { success: true, data: { path: input.path, - entries: entries.map(e => ({ - path: e.path, - size: e.size, - })), - count: entries.length, + entries: children, + count: children.length, + ...incomplete, }, }; - } - - // Non-recursive: deduplicate to show immediate children only - const seen = new Set(); - const children: Array<{ name: string; type: 'file' | 'directory' }> = []; - - for (const entry of entries) { - const rest = entry.path.slice(prefix.length); - const slashIdx = rest.indexOf('/'); - if (slashIdx >= 0) { - const dirName = rest.slice(0, slashIdx); - if (!seen.has(dirName)) { - seen.add(dirName); - children.push({ name: dirName, type: 'directory' }); - } - } else { - children.push({ name: rest, type: 'file' }); - } - } - - return { - success: true, - data: { - path: input.path, - entries: children, - count: children.length, - }, - }; + }); } private async handleGlob(input: GlobInput): Promise { - const store = this.getStore(); const regex = globToRegex(input.pattern); - const matches: string[] = []; + const matches: Array> = []; + const incomplete: Array<{ path: string; reason: string }> = []; // Search across mounts const mountsToSearch = input.path @@ -2160,20 +2351,17 @@ export class WorkspaceModule implements Module { : [...this.mounts.values()].map(m => ({ mount: m, relativePath: '' })); for (const { mount, relativePath } of mountsToSearch) { - if (!mount.initialSyncDone) { - await syncFromFs(store, mount); - mount.initialSyncDone = true; - } - - const prefix = relativePath ? relativePath + '/' : undefined; - const entries = store.treeList(mount.treeStateId, prefix); - - for (const entry of entries) { - const testPath = relativePath ? entry.path.slice(relativePath.length + 1) : entry.path; + const pass = await this.withMount(mount, () => + this.passUnlocked(mount, { kind: 'dir', dir: relativePath, recursive: true })); + for (const report of [...pass.reports.values()].sort((a, b) => a.path.localeCompare(b.path))) { + const testPath = relativePath ? report.path.slice(relativePath.length + 1) : report.path; if (regex.test(testPath)) { - matches.push(`${mount.config.name}/${entry.path}`); + matches.push({ path: `${mount.config.name}/${report.path}`, ...stateView(report) }); } } + for (const region of pass.incomplete) { + incomplete.push({ path: `${mount.config.name}/${region.path}`, reason: region.reason }); + } } return { @@ -2182,6 +2370,7 @@ export class WorkspaceModule implements Module { pattern: input.pattern, matches, count: matches.length, + ...(incomplete.length > 0 ? { incomplete } : {}), }, }; } @@ -2203,61 +2392,107 @@ export class WorkspaceModule implements Module { ? [this.parsePath(input.path)] : [...this.mounts.values()].map(m => ({ mount: m, relativePath: '' })); - const results: Array<{ file: string; matches: Array<{ line: number; text: string; context?: string[] }> }> = []; - - for (const { mount, relativePath } of mountsToSearch) { - if (!mount.initialSyncDone) { - await syncFromFs(store, mount); - mount.initialSyncDone = true; - } - - // If `path` points at a single FILE, grep just that file. Otherwise treat - // `path` as a directory prefix (the original behaviour). Previously a file - // path produced an empty prefix like "notes.md/" and matched nothing, so - // grepping a specific file silently returned zero results. - let entries: Array<{ path: string; blobHash: string }>; - const fileNode = relativePath - ? (store.treeGet(mount.treeStateId, relativePath) as { blobHash?: string } | null) - : null; - if (fileNode && fileNode.blobHash) { - entries = [{ path: relativePath, blobHash: fileNode.blobHash }]; - } else { - const prefix = relativePath ? relativePath + '/' : undefined; - entries = store.treeList(mount.treeStateId, prefix); + type GrepMatch = { line: number; text: string; context?: string[] }; + const results: Array<{ file: string; state: string; version: 'workspace' | 'conflicting-disk' | 'recorded-disk'; matches: GrepMatch[] }> = []; + const skipped: Array<{ file: string; reason: string; size?: number; mimeType?: string }> = []; + const incomplete: Array<{ path: string; reason: string }> = []; + + const search = (content: string): GrepMatch[] => { + const lines = content.split('\n'); + const found: GrepMatch[] = []; + for (let i = 0; i < lines.length; i++) { + if (regex.test(lines[i])) { + const match: GrepMatch = { line: i + 1, text: lines[i] }; + if (contextBefore > 0 || contextAfter > 0) { + const start = Math.max(0, i - contextBefore); + const end = Math.min(lines.length, i + contextAfter + 1); + match.context = lines.slice(start, end); + } + found.push(match); + } } + return found; + }; - for (const entry of entries) { - if (fileGlob && !fileGlob.test(entry.path)) continue; + for (const { mount, relativePath } of mountsToSearch) { + await this.withMount(mount, async () => { + // A `path` naming a single file greps just that file; otherwise it is a + // directory prefix. (A file path once produced the prefix "notes.md/" + // and silently matched nothing.) + const fileScope = relativePath !== '' && ( + store.treeGet(mount.treeStateId, relativePath) !== null || + await lstat(join(mount.config.path, relativePath)).then((s) => !s.isDirectory(), () => false) + ); + // What follows reads the tree and branch intent to go with the pass's + // reports, synchronously. They must be the branch the pass decided + // on: one selected in between would label its content with the other + // branch's states. So a pass whose branch is no longer selected runs + // again, and a branch that keeps changing leaves the scope unsearched. + let pass: PassResult | null = null; + for (let attempt = 0; attempt < 3 && pass === null; attempt++) { + const next = await this.passUnlocked(mount, fileScope + ? { kind: 'paths', paths: [relativePath] } + : { kind: 'dir', dir: relativePath, recursive: true }); + if (next.branchId === undefined || next.branchId === store.currentBranch().id) pass = next; + } + if (pass === null) { + incomplete.push({ + path: `${mount.config.name}/${relativePath}`, + reason: 'the selected branch kept changing while grep ran; nothing here was searched — grep again', + }); + return; + } + for (const region of pass.incomplete) { + incomplete.push({ path: `${mount.config.name}/${region.path}`, reason: region.reason }); + } - const blob = store.getBlob(entry.blobHash); - if (!blob) continue; - - const content = blob.toString('utf-8'); - const lines = content.split('\n'); - const fileMatches: Array<{ line: number; text: string; context?: string[] }> = []; - - for (let i = 0; i < lines.length; i++) { - if (regex.test(lines[i])) { - const match: { line: number; text: string; context?: string[] } = { - line: i + 1, - text: lines[i], - }; - if (contextBefore > 0 || contextAfter > 0) { - const start = Math.max(0, i - contextBefore); - const end = Math.min(lines.length, i + contextAfter + 1); - match.context = lines.slice(start, end); + const intents = this.intents.get(mount.config.name)!; + for (const report of [...pass.reports.values()].sort((a, b) => a.path.localeCompare(b.path))) { + if (fileGlob && !fileGlob.test(report.path)) continue; + const file = `${mount.config.name}/${report.path}`; + if (report.state === 'disk-only') { + skipped.push({ + file, + reason: 'disk-only: binary or over the size limit, never stored', + ...(report.size !== undefined ? { size: report.size } : {}), + ...(report.mimeType ? { mimeType: report.mimeType } : {}), + }); + continue; + } + // The workspace's version, labelled with the path's state. + const entry = store.treeGet(mount.treeStateId, report.path); + const blob = entry ? store.getBlob(entry.blobHash) : null; + if (blob) { + const found = search(blob.toString('utf-8')); + if (found.length > 0) results.push({ file, state: report.state, version: 'workspace', matches: found }); + } else if (entry) { + skipped.push({ file, reason: 'the workspace copy is missing from the store, so it was not searched' }); + } + if (report.state !== 'conflict') continue; + // A conflict's disk side. The version recorded with the conflict is + // disk's only while disk still holds it; once disk has changed + // since, it is labelled as recorded, and a file disk holds now + // (never stored) is listed as not searched. A binary or oversize + // disk version was never stored either. + const disk = intents.get(report.path)?.conflict?.disk; + const changed = report.conflict?.diskChangedSinceRecorded === true; + const diskBlob = disk?.stored ? store.getBlob(disk.stored) : null; + if (diskBlob) { + const found = search(diskBlob.toString('utf-8')); + if (found.length > 0) results.push({ file, state: report.state, version: changed ? 'recorded-disk' : 'conflicting-disk', matches: found }); + } + if (disk?.stored && !diskBlob) { + skipped.push({ file, reason: 'conflict: the disk version recorded with the conflict is missing from the store, so it was not searched' }); + } + if (changed) { + if (report.diskNow === 'file') { + skipped.push({ file, reason: 'conflict: disk changed since the conflict was recorded; grep searches stored text, so the file disk holds now was not searched' }); } - fileMatches.push(match); + } else if (disk && !disk.stored) { + skipped.push({ file, reason: 'conflict: the disk version is binary or over the size limit, never stored', size: disk.size }); } } - - if (fileMatches.length > 0) { - results.push({ - file: `${mount.config.name}/${entry.path}`, - matches: fileMatches, - }); - } - } + }); } return { @@ -2266,22 +2501,33 @@ export class WorkspaceModule implements Module { pattern: input.pattern, results, totalMatches: results.reduce((sum, r) => sum + r.matches.length, 0), + ...(skipped.length > 0 ? { skipped } : {}), + ...(incomplete.length > 0 ? { incomplete } : {}), }, }; } - private async handleStatus(_input: StatusInput): Promise { + private async handleStatus(input: StatusInput): Promise { const store = this.getStore(); const status: Record = {}; + if (input.mount !== undefined && !this.mounts.has(input.mount)) { + return { success: false, error: `Unknown mount: ${input.mount}`, isError: true }; + } for (const [name, mount] of this.mounts) { + if (input.mount !== undefined && name !== input.mount) continue; const entries = store.treeList(mount.treeStateId); const currentSeq = store.currentSequence(); - const changes = mount.lastMaterializedSeq > 0 - ? store.treeDiff(mount.treeStateId, mount.lastMaterializedSeq, currentSeq) - : []; - // Refused paths (#109) are pending too, past the watermark or not. - const pending = new Set([...changes.map((c) => c.path), ...mount.refusedPaths]); + const intents = this.intents.get(name)!.list(); + const conflictPaths = intents.filter(([, bi]) => bi.conflict).map(([p]) => p); + // A workspace deletion is pending until a push confirms it on disk: + // left there without applyDeletions, or unlinked but unconfirmed. + const deletionPaths = intents.filter(([, bi]) => bi.tombstone).map(([p]) => p); + // Pending is what disk owes by the evidence: every entry it doesn't + // show on disk (a refused or failed push stays owed), conflicts and + // deletions. An entry from before the evidence isn't counted until a + // listing or a materialize checks it: unchecked isn't unpushed. + const pending = new Set([...conflictPaths, ...deletionPaths, ...this.entriesByEvidence(mount).owed]); const currentBranch = store.currentBranch(); status[name] = { @@ -2292,23 +2538,89 @@ export class WorkspaceModule implements Module { lastMaterializedSeq: mount.lastMaterializedSeq, currentSeq, pendingChanges: pending.size, + conflicts: conflictPaths.length, + pendingWorkspaceDeletions: deletionPaths.length, initialSyncDone: mount.initialSyncDone, currentBranch: currentBranch.name, lastMaterializedBranch: mount.lastMaterializedBranchId, canMaterialize: this.mountMaterializeBlockReason(store, mount) === null, + ...(mount.lastAgentActionScan ? { lastAgentActionScan: mount.lastAgentActionScan } : {}), }; } return { success: true, data: status }; } + /** + * What a materialize of `mount` should consider: a named file, or every + * tracked path beneath a named directory; otherwise what the evidence says + * disk still owes, the entries never checked against disk, recorded + * conflicts, and workspace deletions still on disk (applied with + * `applyDeletions`, listed without it). The watermark plays no part: it is + * lost at an unclean restart, and an entry the evidence shows on disk has + * nothing to push. If disk changed since, a sync adopts that; a bare + * materialize, forced or not, never reverts it (naming the path with force + * does). `inPlace` says whether any tree entry was left out for that reason. + */ + private materializeSelection(mount: MountState, explicitPath: string): { paths: string[]; inPlace: boolean } { + const store = this.getStore(); + const intents = new Map(this.intents.get(mount.config.name)!.list(explicitPath)); + const selected = new Set(); + + if (explicitPath !== '') { + if (store.treeGet(mount.treeStateId, explicitPath)) selected.add(explicitPath); + for (const e of store.treeList(mount.treeStateId, explicitPath + '/')) selected.add(e.path); + for (const [p, bi] of intents) if (bi.tombstone || bi.conflict) selected.add(p); + return { paths: [...selected], inPlace: false }; + } + + const { owed, unchecked, agreed } = this.entriesByEvidence(mount); + for (const path of [...owed, ...unchecked]) selected.add(path); + for (const [p, bi] of intents) if (bi.conflict || bi.tombstone) selected.add(p); + return { paths: [...selected], inPlace: agreed > 0 }; + } + + /** + * The workspace entries by what the evidence says of disk. `owed`: P + * differs from the entry (a draft, or a push still pending), or there is no + * P and the entry came from the workspace (a draft disk never had). + * `unchecked`: no P and no workspace origin, an entry from before the + * evidence; a push agrees it, writes it where disk lacks it, or refuses it + * as a conflict. `agreed`: how many have P = S, so disk holds them. Status + * counts the owed as pending; a materialize without a path takes up the + * owed and the unchecked. + */ + private entriesByEvidence(mount: MountState): { owed: string[]; unchecked: string[]; agreed: number } { + const store = this.getStore(); + const agreement = this.agreementOrThrow(); + const name = mount.config.name; + const intents = this.intents.get(name)!; + const owed: string[] = []; + const unchecked: string[] = []; + let agreed = 0; + for (const e of store.treeList(mount.treeStateId)) { + const p = agreement.get(name, e.path); + if (p === undefined) (intents.get(e.path)?.origin === 'store' ? owed : unchecked).push(e.path); + else if (p.kind !== 'content' || p.hash !== e.blobHash) owed.push(e.path); + else agreed++; + } + return { owed, unchecked, agreed }; + } + private async handleMaterialize(input: MaterializeInput): Promise { const store = this.getStore(); const allWritten: Array<{ mount: string; path: string }> = []; + const allDeleted: Array<{ mount: string; path: string }> = []; + const pendingDeletions: Array<{ mount: string; path: string }> = []; + + let explicit: { mount: MountState; relativePath: string } | null = null; + if (input.path) explicit = this.parsePath(input.path); let mountsToMaterialize: Array<{ name: string; mount: MountState }>; - if (input.mount) { + if (explicit) { + mountsToMaterialize = [{ name: explicit.mount.config.name, mount: explicit.mount }]; + } else if (input.mount) { const m = this.mounts.get(input.mount); if (!m) { return { success: false, error: `Unknown mount: ${input.mount}`, isError: true }; @@ -2323,71 +2635,94 @@ export class WorkspaceModule implements Module { // Branch guard, scoped to the mounts actually being materialized: a // linear continuation (current branch descends from the pinned branch at // or after the pinned seq) passes; genuine divergence refuses unless - // force. One mount's stale pin must never block another mount. + // force. One mount's stale pin must never block another mount. It is + // judged inside the mount's turn, on the same branch the selection is + // made on: a materialize queued behind a pass would otherwise select on + // whatever branch is current when its turn comes, unguarded. The push + // then writes nothing if the branch changes after that. const blocked: Array<{ mount: string; reason: string }> = []; - for (const { name, mount } of mountsToMaterialize) { - const reason = this.mountMaterializeBlockReason(store, mount); - if (!reason) continue; - if (input.force) { - // Disk reflects another line of history, so an incremental diff from - // the pinned seq is meaningless — reset tracking and re-materialize - // the full tree, exactly like materializeMount() after a deliberate - // branch switch. - mount.lastMaterializedSeq = 0; - mount.lastMaterializedBranchId = null; - } else { - blocked.push({ mount: name, reason }); - } - } - mountsToMaterialize = mountsToMaterialize.filter( - ({ name }) => !blocked.some((b) => b.mount === name), - ); - if (mountsToMaterialize.length === 0 && blocked.length > 0) { - return { - success: false, - error: `Cannot materialize: ${blocked.map((b) => `[${b.mount}] ${b.reason}`).join('; ')}`, - isError: true, - }; - } + const guarded: string[] = []; + let proceeded = 0; for (const { name, mount } of mountsToMaterialize) { - - let paths: string[] | undefined; - if (input.path) { - const { relativePath } = this.parsePath(input.path); - paths = relativePath ? [relativePath] : undefined; + if (mount.config.mode === 'read-only') { + // Only a mount the caller named gets here read-only (a bare + // materialize takes the read-write ones): say why nothing was written. + blocked.push({ mount: name, reason: 'a read-only mount: materialize writes nothing to it' }); + continue; } - - // Suppress watcher for paths we're about to write - const watcher = this.watchers.get(name); - const { written, unchanged, skipped } = await materializeToFs(store, mount, paths, { force: input.force }); - - for (const p of written) { - watcher?.suppress(p); - allWritten.push({ mount: name, path: p }); + const turn = await this.withMount(mount, async (): Promise<{ blocked: string } | { pushed: PushResult; inPlace: boolean }> => { + const branchId = store.currentBranch().id; + const reason = this.mountMaterializeBlockReason(store, mount); + // Force passes a divergence. The selection follows the evidence, which + // is per path and kept across branches, so nothing needs resetting: + // the push re-pins to this branch. + if (reason && !input.force) return { blocked: reason }; + const { paths, inPlace } = this.materializeSelection(mount, explicit?.relativePath ?? ''); + return { + inPlace, + pushed: await this.pushUnlocked(mount, paths, { + force: input.force, named: explicit !== null, applyDeletions: input.applyDeletions, branchId, + }), + }; + }); + if ('blocked' in turn) { + blocked.push({ mount: name, reason: turn.blocked }); + guarded.push(name); + continue; } - // Freshness-guard refusals (#109) ride the same skipped list as branch - // blocks: divergence must be visible, not resolved silently either way. - for (const s of skipped) { + proceeded++; + const pushed = turn.pushed; + + for (const p of pushed.written) allWritten.push({ mount: name, path: p }); + for (const p of pushed.deleted) allDeleted.push({ mount: name, path: p }); + for (const p of pushed.pendingDeletions) pendingDeletions.push({ mount: name, path: p }); + // Refusals ride the same skipped list as branch blocks: divergence must + // be visible, not resolved silently either way. + for (const s of pushed.skipped) { blocked.push({ mount: name, reason: `${s.path}: ${s.reason}` }); } + // What disk still owes is kept by its evidence, so the watermark and + // the pin only feed the branch guard. Both are the push's own: what it + // planned from is what disk now reflects, whatever branch was selected + // while it wrote. A push that planned nothing (the selected branch + // changed after its paths were chosen) left disk as it was, so it + // leaves both as they were too. + if (pushed.branchId === undefined || pushed.sequence === undefined) continue; + mount.lastMaterializedSeq = pushed.sequence; // Track which branch we materialized on. Re-pin on a clean empty // materialize too (previously-pinned mount, nothing pending): disk // already reflects the current branch's tree, and leaving the old pin // would keep force required forever after a cross-branch materialize - // that happened to write nothing. A first materialize that found every - // file already in place pins too: disk holds this branch's tree. - if (written.length > 0 || unchanged.length > 0 || mount.lastMaterializedBranchId !== null) { - mount.lastMaterializedBranchId = store.currentBranch().id; + // that happened to write nothing. A first materialize pins once files + // of this branch are in place, whether it found them on disk or the + // evidence shows them there: disk holds this branch's tree. + if (pushed.written.length > 0 || pushed.unchanged.length > 0 || turn.inPlace || mount.lastMaterializedBranchId !== null) { + mount.lastMaterializedBranchId = pushed.branchId; } } + if (proceeded === 0 && guarded.length > 0 && guarded.length === mountsToMaterialize.length) { + return { + success: false, + error: `Cannot materialize: ${blocked.map((b) => `[${b.mount}] ${b.reason}`).join('; ')}`, + isError: true, + }; + } + return { success: true, data: { materialized: allWritten, count: allWritten.length, + ...(allDeleted.length > 0 ? { deleted: allDeleted } : {}), + ...(pendingDeletions.length > 0 + ? { + workspaceDeletionsLeftOnDisk: pendingDeletions, + note: 'Files deleted in the workspace were left on disk; materialize with applyDeletions to remove them.', + } + : {}), ...(blocked.length > 0 ? { skipped: blocked } : {}), }, }; @@ -2399,6 +2734,7 @@ export class WorkspaceModule implements Module { * Resets branch tracking to allow cross-branch materialization. */ async materializeMount(mountName: string): Promise { + if (!this.running()) return []; const store = this.getStore(); const mount = this.mounts.get(mountName); if (!mount || mount.config.mode === 'read-only') return []; @@ -2407,61 +2743,119 @@ export class WorkspaceModule implements Module { mount.lastMaterializedBranchId = null; mount.lastMaterializedSeq = 0; - const watcher = this.watchers.get(mountName); // force: this path only runs after a deliberate undo/redo/branch switch // on the framework's own _config mount — restoring disk to the branch - // state IS the operator intent, so the freshness guard yields. - const { written, unchanged } = await materializeToFs(store, mount, undefined, { force: true }); - for (const p of written) { - watcher?.suppress(p); - } - if (written.length > 0 || unchanged.length > 0) { - mount.lastMaterializedBranchId = store.currentBranch().id; + // state IS the operator intent, so the freshness guard yields. named: + // the whole tree is that deliberate scope, entries without evidence too. + const pushed = await this.withMount(mount, () => { + const branchId = store.currentBranch().id; + return this.pushUnlocked(mount, store.treeList(mount.treeStateId).map((e) => e.path), { force: true, named: true, branchId }); + }); + // A push that planned nothing leaves the reset tracking as it is. + if (pushed.branchId !== undefined && pushed.sequence !== undefined) { + mount.lastMaterializedSeq = pushed.sequence; + if (pushed.written.length > 0 || pushed.unchanged.length > 0) { + mount.lastMaterializedBranchId = pushed.branchId; + } } - return written; + return pushed.written; } private async handleSync(input: SyncInput): Promise { const store = this.getStore(); - const allResults: Array<{ mount: string; synced: string[]; conflicts: ConflictInfo[] }> = []; + // A path sync adopts on the branch it was asked on, read before any await. + const askedOn = store.currentBranch().id; + const allResults: Array<{ + mount: string; + synced: string[]; + conflicts: Array<{ path: string; kind: string; diskCopy: string }>; + /** A path sync's workspace changes given up for disk's state, each with the state it had, + * and what that did to the workspace copy (absent when nothing). */ + discarded?: Array<{ path: string; was: string; op?: string }>; + }> = []; const allSkipped: Array<{ mount: string; path: string; reason: string }> = []; - - let mountsToSync: Array<{ name: string; mount: MountState }>; - if (input.mount) { + const allIncomplete: Array<{ mount: string; path: string; reason: string }> = []; + const rootsAccepted: string[] = []; + if (input.acceptRoot && input.path) { + return { success: false, error: 'acceptRoot takes a whole mount\'s root: give mount, or nothing, not path', isError: true }; + } + + // A path names its own mount; otherwise the given mount, or every mount. + let targets: Array<{ name: string; mount: MountState; relativePath: string }>; + if (input.path) { + const { mount, relativePath } = this.parsePath(input.path); + targets = [{ name: mount.config.name, mount, relativePath }]; + } else if (input.mount) { const m = this.mounts.get(input.mount); if (!m) { return { success: false, error: `Unknown mount: ${input.mount}`, isError: true }; } - mountsToSync = [{ name: input.mount, mount: m }]; + targets = [{ name: input.mount, mount: m, relativePath: '' }]; } else { - mountsToSync = [...this.mounts.entries()].map(([name, mount]) => ({ name, mount })); + targets = [...this.mounts.entries()].map(([name, mount]) => ({ name, mount, relativePath: '' })); } - for (const { name, mount } of mountsToSync) { - - let paths: string[] | undefined; - if (input.path) { - const { relativePath } = this.parsePath(input.path); - // Empty relativePath means mount root — sync entire mount, not a single path - paths = relativePath ? [relativePath] : undefined; - } - - const result = await syncFromFs(store, mount, paths); - mount.initialSyncDone = true; + for (const { name, mount, relativePath } of targets) { + const pass = await this.withMount(mount, async () => { + if (input.acceptRoot && await this.acceptRootUnlocked(mount)) rootsAccepted.push(name); + if (!relativePath) { + // A full recheck: every file rehashed, nothing pending discarded. + return this.passUnlocked(mount, { kind: 'dir', dir: '', recursive: true }, { rehash: true }); + } + // An explicit path: the workspace takes disk's state there. + const prefix = relativePath + '/'; + const isDir = + await lstat(join(mount.config.path, relativePath)).then((s) => s.isDirectory(), () => false) || + store.treeList(mount.treeStateId, prefix).length > 0 || + this.agreementOrThrow().paths(name, relativePath).some(([p]) => p.startsWith(prefix)) || + this.intents.get(name)!.list(relativePath).some(([p]) => p.startsWith(prefix)); + const opts: PassOptions = { rehash: true, adopt: () => true, branchId: askedOn }; + if (!isDir) return this.passUnlocked(mount, { kind: 'paths', paths: [relativePath] }, opts); + const beneath = await this.passUnlocked(mount, { kind: 'dir', dir: relativePath, recursive: true }, opts); + // Something at the path itself is no part of a directory scope: a file + // disk has where the workspace holds a directory, or a workspace file + // where disk has a directory. Its own pass takes disk's state there + // too, after what lies beneath, so a file never sits over entries it + // replaced. + const onDisk = await lstat(join(mount.config.path, relativePath)).catch(() => null); + const atPath = (onDisk !== null && !onDisk.isDirectory()) + || store.treeGet(mount.treeStateId, relativePath) !== null + || this.agreementOrThrow().get(name, relativePath) !== undefined + || this.intents.get(name)!.get(relativePath) !== null; + if (!atPath) return beneath; + const itself = await this.passUnlocked(mount, { kind: 'paths', paths: [relativePath] }, opts); + return { + ops: [...beneath.ops, ...itself.ops], + discarded: [...beneath.discarded, ...itself.discarded], + newConflicts: [...beneath.newConflicts, ...itself.newConflicts], + reports: new Map([...beneath.reports, ...itself.reports]), + incomplete: [...beneath.incomplete, ...itself.incomplete], + dirs: beneath.dirs, + branchId: itself.branchId ?? beneath.branchId, + }; + }); - if (result.synced.length > 0 || result.conflicts.length > 0) { + const conflicts = [...pass.reports.values()] + .filter((r) => r.state === 'conflict') + .map((r) => ({ path: r.path, kind: r.conflict?.kind ?? 'both-changed', diskCopy: r.conflict?.diskCopy ?? 'referenced' })); + if (pass.ops.length > 0 || conflicts.length > 0 || pass.discarded.length > 0) { allResults.push({ mount: name, - synced: result.synced.map(s => s.path), - conflicts: result.conflicts, + synced: pass.ops.map((o) => o.path), + conflicts, + ...(pass.discarded.length > 0 ? { discarded: pass.discarded } : {}), }); - this.emitFsEvents(name, result.synced, result.conflicts); } - // Skips were previously computed and dropped, so "nothing synced" and - // "your file was refused" looked identical from the outside. Say which. - for (const s of result.skipped) { - allSkipped.push({ mount: name, path: s.path, reason: s.reason }); + // Say why a path wasn't taken in, so "nothing synced" and "your file was + // refused" don't look identical from the outside. + for (const r of pass.reports.values()) { + if (r.state === 'unverified') { + allSkipped.push({ mount: name, path: r.path, reason: `disk could not be checked: ${r.note ?? 'not observed'}` }); + } else if (r.state === 'disk-only') { + allSkipped.push({ mount: name, path: r.path, reason: 'disk-only: binary or over the size limit, never stored' }); + } } + for (const region of pass.incomplete) allIncomplete.push({ mount: name, ...region }); } return { @@ -2471,10 +2865,30 @@ export class WorkspaceModule implements Module { totalSynced: allResults.reduce((sum, r) => sum + r.synced.length, 0), totalConflicts: allResults.reduce((sum, r) => sum + r.conflicts.length, 0), ...(allSkipped.length > 0 ? { skipped: allSkipped } : {}), + ...(allIncomplete.length > 0 ? { incomplete: allIncomplete } : {}), + ...(rootsAccepted.length > 0 ? { rootsAccepted } : {}), }, }; } + /** + * Take the mount's root as it is now (acceptRoot), when it is a directory + * other than the one disk last agreed under. Its evidence is set aside: + * it described another directory. Returns whether a root was accepted. + */ + private async acceptRootUnlocked(mount: MountState): Promise { + const runtime = await this.runtime(mount); + const found = await locateRoot({ ...runtime.view, rootIdentity: undefined }, runtime.rootReal); + if ('reason' in found) return false; // still missing: the pass says so + const agreement = this.agreementOrThrow(); + const recorded = agreement.rootIdentity(mount.config.name); + if (recorded && sameRootIdentity(recorded, found.identity)) return false; + if (recorded) agreement.acceptRoot(mount.config.name, found.identity); + else agreement.recordRootIdentity(mount.config.name, found.identity); + agreement.barrier(); + return true; + } + // ========================================================================== // Internal: Filesystem Change Handling // ========================================================================== @@ -2482,74 +2896,163 @@ export class WorkspaceModule implements Module { /** * Handle filesystem changes detected by watcher (watch: 'always' mode). * - * The watcher carries the op per path. For created/modified we call - * syncFromFs to pull content into the tree; for deleted we strip the tree - * entry directly (the file is gone, nothing to read). We emit one event per - * op type. + * The watcher's ops only choose which paths to look at: each path is + * observed again and decided under the three-way rule, so an event that + * arrives late or out of order can't delete or revert anything by itself. */ private async handleFsChanges(mountName: string, changes: FsChange[]): Promise { - const store = this.store; const mount = this.mounts.get(mountName); - if (!store || !mount) return; + if (!this.store || !this.running() || !mount) return; + const paths = [...new Set(changes.map((c) => c.path))]; + if (paths.length === 0) return; + await this.withMount(mount, () => this.passUnlocked(mount, { kind: 'paths', paths })); + } - const touchedPaths = changes.filter(c => c.op !== 'deleted').map(c => c.path); - const deletedPaths = changes.filter(c => c.op === 'deleted').map(c => c.path); + /** + * After an agent's completed tool batch, before anything continues its turn: + * scan every `watch: 'on-agent-action'` mount, so what the batch's tools did + * on disk (a shell command's files, a deletion) is in the workspace when the + * agent next looks. A scan that outlasts the deadline stops holding the + * round: the miss is recorded on the mount's status and pushed as a + * `workspace:agent-action-scan-incomplete` event, and the scan continues, + * deciding on the branch selected when it applies. + */ + async onToolBatchComplete(_agentName?: string): Promise { + // Every agent's workspace is the store's current branch, so the agent + // that ran the batch doesn't choose what is scanned. + const mounts = [...this.mounts.values()].filter((m) => m.config.watch === 'on-agent-action'); + if (mounts.length === 0 || !this.store || !this.running()) return; + const deadlineMs = this.agentActionScanDeadlineMs(); + await Promise.all(mounts.map(async (mount) => { + const runningLate = mount.agentActionScanLate !== undefined; + const scan = this.agentActionScan(mount); + if (runningLate) { + // An earlier batch's scan outlasted its deadline and still runs: this + // round isn't held as well. Its own scan is queued after that one, + // and records its outcome when it finishes. It is late from the + // start, so a batch that comes while it runs isn't held either. + this.agentActionScanLateUntil(mount, scan); + this.agentActionScanMissed(mount, 'an earlier scan is still running past its deadline; this batch is scanned after it'); + return; + } + let timer: ReturnType | undefined; + const deadline = new Promise<'late'>((resolveDeadline) => { + timer = setTimeout(() => resolveDeadline('late'), deadlineMs); + }); + const outcome = await Promise.race([scan.settled.then(() => 'done' as const), deadline]); + clearTimeout(timer); + if (outcome === 'late') { + this.agentActionScanLateUntil(mount, scan); + this.agentActionScanMissed(mount, `deadline exceeded (${deadlineMs} ms); the scan continues`); + } + })); + } + + /** + * Mark a scan late, and the mount as running late until it settles. The + * marker follows the newest late scan: a mount's scans run one after + * another, so the newest is the last to settle, and a batch that comes + * while any late scan of the chain still runs isn't held. + */ + private agentActionScanLateUntil(mount: MountState, scan: AgentActionScan): void { + scan.late = true; + const settled = scan.settled; + mount.agentActionScanLate = settled; + void settled.then(() => { if (mount.agentActionScanLate === settled) mount.agentActionScanLate = undefined; }); + } - const syncResult = touchedPaths.length > 0 - ? await syncFromFs(store, mount, touchedPaths) - : { synced: [], conflicts: [], skipped: [] }; + /** A batch's scan missed its deadline, or failed: said on status and as an event. */ + private agentActionScanMissed(mount: MountState, reason: string): void { + mount.lastAgentActionScan = { at: Date.now(), complete: false, withinDeadline: false, reason }; + this.ctx?.pushEvent({ type: 'workspace:agent-action-scan-incomplete', mount: mount.config.name, reason } as ProcessEvent); + } - // Strip deleted entries from the tree so downstream consumers see a - // coherent state. syncFromFs would also do this if we passed the paths in, - // but the op from the watcher is authoritative — skip the access() probe. - for (const p of deletedPaths) { - const existing = store.treeGet(mount.treeStateId, p); - if (existing) { - store.treeRemove(mount.treeStateId, p); - syncResult.synced.push({ path: p, op: 'deleted' }); - } + /** + * The mount's after-batch scan: one may run and one wait behind it. A batch + * that comes while one waits shares it, since it starts after the batch's + * tools ran. Each scan records its own outcome when it settles: on status + * while it is the mount's newest scan (a scan queued behind it describes a + * later batch), and as an event for regions it couldn't observe (the file + * cap, an unreadable or changing region, a branch that kept changing) or a + * failure, whichever scan it is. + */ + private agentActionScan(mount: MountState): AgentActionScan { + if (mount.agentActionScanWaiting) return mount.agentActionScanWaiting; + const scan: AgentActionScan = { late: false, settled: Promise.resolve() }; + const pass = this.withMount(mount, () => { + if (mount.agentActionScanWaiting === scan) mount.agentActionScanWaiting = undefined; + return this.passUnlocked(mount, { kind: 'dir', dir: '', recursive: true }); + }); + scan.settled = pass.then( + (result) => { + if (result.incomplete.length > 0) { + this.ctx?.pushEvent({ + type: 'workspace:agent-action-scan-incomplete', + mount: mount.config.name, + reason: 'the scan finished, but some regions could not be observed', + incomplete: result.incomplete, + } as ProcessEvent); + } + if (mount.agentActionScanLatest !== scan) return; + const incomplete = result.incomplete.length > 0 ? { incomplete: result.incomplete } : {}; + mount.lastAgentActionScan = scan.late + ? { at: Date.now(), complete: true, withinDeadline: false, reason: 'finished after the deadline', ...incomplete } + : { at: Date.now(), complete: true, withinDeadline: true, ...incomplete }; + }, + (err: unknown) => { + const reason = `scan failed: ${err instanceof Error ? err.message : String(err)}`; + if (mount.agentActionScanLatest === scan) this.agentActionScanMissed(mount, reason); + else this.ctx?.pushEvent({ type: 'workspace:agent-action-scan-incomplete', mount: mount.config.name, reason } as ProcessEvent); + }, + ); + mount.agentActionScanWaiting = scan; + mount.agentActionScanLatest = scan; + return scan; + } + + /** The scan deadline, kept below the framework's hook bound (warned once). */ + private agentActionScanDeadlineMs(): number { + const configured = this.config.agentActionScanDeadlineMs ?? AGENT_ACTION_SCAN_DEADLINE_MS; + if (configured <= AGENT_ACTION_SCAN_DEADLINE_MAX_MS) return configured; + if (!this.warnedScanDeadline) { + this.warnedScanDeadline = true; + console.warn( + `[workspace] agentActionScanDeadlineMs ${configured} leaves less than 1 s before the framework's ` + + `${TOOL_BATCH_HOOK_TIMEOUT_MS} ms bound on a tool-batch hook; using ${AGENT_ACTION_SCAN_DEADLINE_MAX_MS} ms`, + ); } - - this.emitFsEvents(mountName, syncResult.synced, syncResult.conflicts); + return AGENT_ACTION_SCAN_DEADLINE_MAX_MS; } /** - * Group synced paths by op and push one ProcessEvent per non-empty op. + * Push one ProcessEvent per op for a pass's tree changes and new conflicts. + * A conflict rides the event for the disk change behind it (modified, or + * deleted for a disk deletion of a file with newer workspace changes). */ private emitFsEvents( mountName: string, - synced: Array<{ path: string; op: WorkspaceFsOp }>, - conflicts: ConflictInfo[], + ops: Array<{ path: string; op: TreeOp }>, + newConflicts: Array<{ path: string; op: TreeOp }> = [], ): void { - if (!this.ctx || synced.length === 0) return; - - const byOp = new Map(); - for (const { path, op } of synced) { - const list = byOp.get(op) ?? []; - list.push(`${mountName}/${path}`); - byOp.set(op, list); - } - - // Conflicts by definition require a prior tree entry (sync detected that - // the agent-side version diverged from the filesystem baseline), which - // means the path is being re-synced as a modification. Attach conflicts - // only to the workspace:modified event, intersected with its paths — a - // created/deleted batch can't carry a meaningful conflict. - const conflictsByPath = new Map(); - for (const c of conflicts) conflictsByPath.set(c.path, c); - - for (const [op, paths] of byOp) { - const type = opToEventType(op); - let eventConflicts: string[] | undefined; - if (op === 'modified' && conflictsByPath.size > 0) { - const matched = paths.filter(p => conflictsByPath.has(p.slice(mountName.length + 1))); - if (matched.length > 0) eventConflicts = matched; - } + if (!this.ctx || (ops.length === 0 && newConflicts.length === 0)) return; + + const byOp = new Map(); + const add = (op: TreeOp, path: string, conflict: boolean): void => { + const group = byOp.get(op) ?? { paths: [], conflicts: [] }; + const full = `${mountName}/${path}`; + if (!group.paths.includes(full)) group.paths.push(full); + if (conflict) group.conflicts.push(full); + byOp.set(op, group); + }; + for (const { path, op } of ops) add(op, path, false); + for (const { path, op } of newConflicts) add(op, path, true); + + for (const [op, { paths, conflicts }] of byOp) { const event = { - type, + type: opToEventType(op), paths, mount: mountName, - ...(eventConflicts ? { conflicts: eventConflicts } : {}), + ...(conflicts.length > 0 ? { conflicts } : {}), } as WorkspaceCreatedEvent | WorkspaceModifiedEvent | WorkspaceDeletedEvent; this.ctx.pushEvent(event as ProcessEvent); } @@ -2557,6 +3060,17 @@ export class WorkspaceModule implements Module { } +/** What a listing shows of a path beyond its name. */ +function stateView(report: PathReport): Record { + return { + state: report.state, + ...(report.size !== undefined ? { size: report.size } : {}), + ...(report.mimeType ? { mimeType: report.mimeType } : {}), + ...(report.conflict ? { conflict: report.conflict } : {}), + ...(report.note ? { note: report.note } : {}), + }; +} + // ========================================================================== // Utilities // ========================================================================== diff --git a/src/modules/workspace/observe.ts b/src/modules/workspace/observe.ts new file mode 100644 index 000000000..3765d4cbd --- /dev/null +++ b/src/modules/workspace/observe.ts @@ -0,0 +1,839 @@ +/** + * How the workspace observes disk for reconciliation, under the mount's + * boundary. + * + * Observation honors the boundary the way openContainedFile does: the lexical + * path is lstat'd first, a symlink is refused unless the mount follows + * symlinks, and a followed one must resolve inside the canonical mount root. + * Anything that isn't a regular file inside the mount is `unobserved` — it + * proves nothing either way, and reconciliation leaves the path alone. + * + * Reads go through one descriptor that is fstat'd, located inside the mount + * and then read, so what is read and the fingerprint recorded with its hash + * describe the same file, inside the mount. A hard link is a name inside the + * mount like any other, and reading through it shows what that path holds; + * only a write in place reaches a file's other names, so effects.ts refuses + * that instead. + * + * Directories are checked (CheckedDir): located inside the mount, with their + * identity, and what they showed — a listing, an absence — is accepted only + * if each is still the directory its logical path names afterwards. A walk records which + * directories it listed completely, and only those prove a file absent: an + * unreadable directory, an ignored subtree, the file cap, a symlink the walk + * doesn't descend, or a directory that changed while it was listed leave + * everything beneath them unproven. Outside a walk, absence is proven the + * same way: by a directory inside the mount, still at its path, that lacks + * the entry (confirmAbsent). A missing parent alone proves nothing, since a + * dangling symlink or a swapped directory looks just like one. + * + * Each check is made where the operation happens, against what is there at + * that moment, so the boundary holds against what is in the mount, including + * what appears while an operation runs. Node has no openat or fdopendir, so + * a directory is checked by its canonical path and identity, not held open: + * a parent replaced concurrently in the window between a check and its use + * (or replaced and restored around it) can still redirect a listing or + * lookup there. File contents are bound regardless: a file is read only + * through a descriptor bound to its path first. + */ + +import { createHash } from 'node:crypto'; +import { constants as fsConstants } from 'node:fs'; +import type { BigIntStats } from 'node:fs'; +import { lstat, mkdir, open, readdir, realpath, stat, statfs } from 'node:fs/promises'; +import type { FileHandle } from 'node:fs/promises'; +import { dirname, join, sep } from 'node:path'; +import { sameRootIdentity, type Fingerprint, type RootIdentity } from './disk-agreement.js'; + +export interface MountView { + /** Mount root as configured. */ + root: string; + /** + * The root directory disk last agreed under, once the mount has evidence. + * A root that resolves to any other directory is unavailable. + */ + rootIdentity?: RootIdentity; + followSymlinks: boolean; + /** Text above this size is not ingested. */ + maxFileSize: number; + ignore: string[]; +} + +export type StatFingerprint = Omit; + +/** + * Opened with every read or write of a workspace path, where the platform has + * it: for a regular file it changes nothing, and a FIFO or other special file + * swapped in at the path neither blocks the open nor passes the regular-file + * check that follows. + */ +export const NONBLOCK = typeof fsConstants.O_NONBLOCK === 'number' ? fsConstants.O_NONBLOCK : 0; + +export type Observation = + | { kind: 'absent' } + | { kind: 'file'; size: number; mtimeMs: number; fp: StatFingerprint } + | { kind: 'unobserved'; reason: string }; + +export interface ReadResult { + kind: 'read'; + size: number; + mtimeMs: number; + fp: Fingerprint; + hash: string; + /** The whole file, when it is at most the requested limit. */ + bytes: Buffer | null; + /** First bytes, for binary detection and MIME sniffing. */ + head: Buffer; +} + +/** The first bytes of a file and its size: nothing else was read or hashed. */ +export interface HeadResult { + kind: 'head'; + size: number; + mtimeMs: number; + head: Buffer; +} + +/** Bytes hashed per read when streaming a file larger than the read limit. */ +const HASH_CHUNK = 1 << 20; +/** Bytes kept for binary detection and MIME sniffing. */ +export const HEAD_BYTES = 8192; +/** Timestamps this close to the hashing time can't vouch for the bytes. */ +export const RACY_WINDOW_MS = 2000; + +export const OUTSIDE_PARENT = 'a parent directory resolves outside the mount'; +export const NOT_FOLLOWED = 'a symlink, which this mount does not follow'; +export const ROOT_UNAVAILABLE = 'the mount root is unavailable'; +export const ROOT_REPLACED = + 'the mount root is not the directory disk last agreed with (an unmounted drive, or a replaced directory): ' + + 'reconnect it; if it was replaced on purpose, sync the mount with acceptRoot to take it as it is now'; +const CHANGED = 'it changed while it was observed'; +const CHANGED_WALK = 'the directory changed while the walk ran'; +const BENEATH_SYMLINK = 'beneath a symlink that does not lead to a directory inside the mount'; + +export function errno(err: unknown): string | undefined { + return (err as { code?: string } | null)?.code; +} + +export function contained(rootReal: string, candidate: string): boolean { + return candidate === rootReal || candidate.startsWith(rootReal.endsWith(sep) ? rootReal : rootReal + sep); +} + +/** The mount-relative parent of a path ('' for the root). */ +export function parentOf(relativePath: string): string { + const slash = relativePath.lastIndexOf('/'); + return slash < 0 ? '' : relativePath.slice(0, slash); +} + +/** The last component of a mount-relative path. */ +export function nameOf(relativePath: string): string { + return relativePath.slice(relativePath.lastIndexOf('/') + 1); +} + +function lexicalOf(view: MountView, rel: string): string { + return rel === '' ? view.root : join(view.root, rel); +} + +// =========================================================================== +// Checked directories +// =========================================================================== + +export type DirCheck = + | { kind: 'checked'; dir: CheckedDir } + /** + * No directory there, proven: nothing can exist beneath the path. `by` is + * the directory whose entries prove it — the one that lacks it, or that + * holds the non-directory in its place. + */ + | { kind: 'absent'; by: string } + /** Unproven either way; `outside` when it resolves outside the mount. */ + | { kind: 'unobserved'; reason: string; outside?: true }; + +/** Errors meaning a directory can't be synced on this filesystem, not that syncing it failed. */ +const DIRECTORY_SYNC_UNSUPPORTED = new Set(['EINVAL', 'ENOTSUP']); + +/** + * A directory inside the mount, as it was checked: its canonical path and + * its identity. Its entries are named through that path, and what it showed + * is accepted only while it is still the directory its logical path names + * (see the module doc for what that does and doesn't bind). + */ +export class CheckedDir { + constructor( + /** Mount-relative path ('' for the root). */ + readonly rel: string, + /** Its canonical location when checked. */ + readonly real: string, + private readonly lexical: string, + private readonly dev: bigint, + private readonly ino: bigint, + ) {} + + /** Which directory it is. */ + identity(): RootIdentity { + return { dev: String(this.dev), ino: String(this.ino) }; + } + + /** A path that reaches `name` in this directory. */ + at(name: string): string { + return join(this.real, name); + } + + /** Whether it is still the directory its logical path names: same location, same directory. */ + async stillHere(): Promise { + if ((await realpath(this.lexical).catch(() => null)) !== this.real) return false; + const info = await lstat(this.real, { bigint: true }).catch(() => null); + return info !== null && info.isDirectory() && info.dev === this.dev && info.ino === this.ino; + } + + /** + * Make its entries durable: this very directory, never one that replaced + * it at its path. Throws if it can't; a filesystem that can't sync + * directories (or Windows, where a directory can't be opened for it) is + * not a failure. + */ + async sync(): Promise { + if (process.platform === 'win32') return; + const handle = await open(this.real, 'r'); + try { + const info = await handle.stat({ bigint: true }); + if (!info.isDirectory() || info.dev !== this.dev || info.ino !== this.ino) throw new Error(CHANGED); + try { + await handle.sync(); + } catch (err) { + if (!DIRECTORY_SYNC_UNSUPPORTED.has(errno(err) ?? '')) throw err; + } + } finally { + await handle.close(); + } + } +} + +/** Check the directory at a canonical location, if one is there. */ +async function checkedAt(view: MountView, rel: string, real: string): Promise { + const info = await lstat(real, { bigint: true }).catch(() => null); + if (!info?.isDirectory()) return null; + return new CheckedDir(rel, real, lexicalOf(view, rel), info.dev, info.ino); +} + +/** + * Where a mount directory is now: checked; missing — nothing there, or not a + * directory, either way for confirmAbsent to prove; or not inside the mount. + */ +async function locateDir(view: MountView, rootReal: string, rel: string): Promise { + const lexical = lexicalOf(view, rel); + let real: string; + try { + real = await realpath(lexical); + } catch (err) { + const code = errno(err); + if (code === 'ENOENT' || code === 'ENOTDIR') return rel === '' ? { kind: 'unobserved', reason: ROOT_UNAVAILABLE } : { kind: 'missing' }; + return { kind: 'unobserved', reason: `cannot resolve the directory (${code ?? String(err)})` }; + } + if (rel === '') { + // The root is whatever the configured path names, and it must still be + // the root the pass was given. + if (real !== rootReal) return { kind: 'unobserved', reason: ROOT_UNAVAILABLE }; + } else if (!contained(rootReal, real)) { + return { kind: 'unobserved', reason: OUTSIDE_PARENT, outside: true }; + } + const checked = await checkedAt(view, rel, real); + if (checked) { + // A root that isn't the directory disk last agreed under proves nothing + // about the files that agreed: an empty mountpoint where a drive was + // would otherwise read as every file deleted. + if (rel === '' && view.rootIdentity && !sameRootIdentity(checked.identity(), view.rootIdentity)) { + return { kind: 'unobserved', reason: ROOT_REPLACED }; + } + return { kind: 'checked', dir: checked }; + } + return rel === '' ? { kind: 'unobserved', reason: ROOT_UNAVAILABLE } : { kind: 'missing' }; +} + +/** + * The mount root as it is now: its identity if it is available (a directory, + * the one the pass was given, and the one disk last agreed under), else why not. + */ +export async function locateRoot(view: MountView, rootReal: string): Promise<{ identity: RootIdentity } | { reason: string }> { + const found = await checkRoot(view, rootReal); + return 'dir' in found ? { identity: found.dir.identity() } : found; +} + +/** The mount root as locateRoot finds it, checked, so a caller can ask later whether it's still there. */ +export async function checkRoot(view: MountView, rootReal: string): Promise<{ dir: CheckedDir } | { reason: string }> { + const found = await locateDir(view, rootReal, ''); + if (found.kind === 'checked') return { dir: found.dir }; + return { reason: found.kind === 'unobserved' ? found.reason : ROOT_UNAVAILABLE }; +} + +/** + * Check the directory at a mount-relative path. With `create`, missing + * directories are made one at a time beneath the deepest one that exists; + * the mount root itself is never created (a missing root is an unavailable + * mount). Without it, a missing directory is `absent` only where that can be + * proven (confirmAbsent). + */ +export async function checkDir(view: MountView, rootReal: string, rel: string, opts: { create?: boolean } = {}): Promise { + const found = await locateDir(view, rootReal, rel); + if (found.kind !== 'missing') return found; + if (opts.create) return createDir(view, rootReal, rel); + const proof = await confirmAbsent(view, rootReal, rel, 'beneath'); + return typeof proof === 'string' ? { kind: 'unobserved', reason: proof } : { kind: 'absent', by: proof.by }; +} + +async function createDir(view: MountView, rootReal: string, rel: string): Promise { + const parts = rel.split('/'); + let base: CheckedDir | null = null; + let depth = parts.length - 1; + for (; depth >= 0; depth--) { + const found = await locateDir(view, rootReal, parts.slice(0, depth).join('/')); + if (found.kind === 'checked') { + base = found.dir; + break; + } + if (found.kind !== 'missing') return found; + } + if (base === null) return { kind: 'unobserved', reason: ROOT_UNAVAILABLE }; + for (let i = depth; i < parts.length; i++) { + const childRel = parts.slice(0, i + 1).join('/'); + try { + await mkdir(base.at(parts[i]!)); + } catch (err) { + const code = errno(err); + if (code !== 'EEXIST') return { kind: 'unobserved', reason: `cannot create the directory ${childRel} (${code ?? String(err)})` }; + } + // Made, or there already (made meanwhile, or not a directory): it must + // be a real directory now, never a symlink or file in its place. + const checked = await checkedAt(view, childRel, base.at(parts[i]!)); + if (!checked) return { kind: 'unobserved', reason: 'a parent path is not a directory' }; + base = checked; + } + return { kind: 'checked', dir: base }; +} + +/** + * Prove that nothing exists at a mount-relative path (`entry`), or that no + * directory does, so nothing can exist beneath it (`beneath`). Walking down + * from the root through checked directories: some directory lacks the next + * component; or a component is a non-directory, still the same one, beneath + * which nothing can exist. Either proves it only while every directory the + * walk went through, the root included, is still the one its path names + * afterwards: a root or parent replaced during the walk would otherwise lend + * its replacement's absence to the original's path. Symlinked directories + * inside the mount are followed as parents; any other symlink proves + * nothing. Returns the directory whose entries prove it, or why absence + * couldn't be shown. + */ +export async function confirmAbsent(view: MountView, rootReal: string, rel: string, target: 'entry' | 'beneath' = 'entry'): Promise<{ by: string } | string> { + const root = await locateDir(view, rootReal, ''); + if (root.kind !== 'checked') return ROOT_UNAVAILABLE; + let cur: CheckedDir = root.dir; + /** Every directory the walk went through, the root first. */ + const walked: CheckedDir[] = [cur]; + const proven = async (): Promise<{ by: string } | string> => { + for (const dir of walked) if (!(await dir.stillHere())) return CHANGED; + return { by: cur.rel }; + }; + const parts = rel.split('/'); + for (let i = 0; i < parts.length; i++) { + const name = parts[i]!; + const last = i === parts.length - 1; + const childRel = parts.slice(0, i + 1).join('/'); + let info; + try { + info = await lstat(cur.at(name), { bigint: true }); + } catch (err) { + const code = errno(err); + if (code === 'ENOTDIR') return CHANGED; + if (code !== 'ENOENT') return `cannot stat (${code ?? String(err)})`; + // Absent from a directory that, like each above it, is still the one its path names. + return proven(); + } + if (info.isDirectory()) { + if (last) return CHANGED; // present after all + cur = new CheckedDir(childRel, cur.at(name), lexicalOf(view, childRel), info.dev, info.ino); + walked.push(cur); + continue; + } + if (info.isSymbolicLink()) { + if (last && target === 'entry') return CHANGED; // present: the link itself + const located = await locateDir(view, rootReal, childRel); + if (located.kind !== 'checked') return BENEATH_SYMLINK; + if (last) return CHANGED; // a directory there after all + cur = located.dir; + walked.push(cur); + continue; + } + if (last && target === 'entry') return CHANGED; // present after all + // A non-directory: nothing can exist beneath it, while it is still this + // same one, in the directory still at its path. + const again = await lstat(cur.at(name), { bigint: true }).catch(() => null); + if (again === null || again.isDirectory() || again.isSymbolicLink() || again.dev !== info.dev || again.ino !== info.ino) return CHANGED; + return proven(); + } + return CHANGED; +} + +async function absence(view: MountView, rootReal: string, rel: string): Promise> { + const proof = await confirmAbsent(view, rootReal, rel); + return typeof proof === 'string' ? { kind: 'unobserved', reason: proof } : { kind: 'absent' }; +} + +// =========================================================================== +// Fingerprints +// =========================================================================== + +function statFingerprint(st: { size: bigint; mtimeNs: bigint; ctimeNs: bigint; ino: bigint; dev: bigint }): StatFingerprint { + return { + size: Number(st.size), + mtimeNs: st.mtimeNs.toString(), + ctimeNs: st.ctimeNs.toString(), + ino: st.ino.toString(), + dev: st.dev.toString(), + }; +} + +/** Same file, same bytes, as far as stat can tell. */ +export function sameFingerprint(a: StatFingerprint, b: StatFingerprint): boolean { + return a.size === b.size && a.mtimeNs === b.mtimeNs && a.ctimeNs === b.ctimeNs && a.ino === b.ino && a.dev === b.dev; +} + +/** + * Whether a recorded fingerprint vouches that the current file still holds + * the bytes hashed with it: the filesystem maintains ctime (so no user-space + * tool can restore the old stamps after a write), every field matches, and + * the file's timestamps were already more than the racy window old when it + * was hashed (a write in the same timestamp tick can't hide behind them). + * Device and inode are part of it, so a stat that reached another file — a + * path whose parent was swapped, say — can never vouch for P's bytes. + */ +export function fingerprintVouches(recorded: Fingerprint | undefined, current: StatFingerprint, trusted: boolean): boolean { + if (!trusted || !recorded || !sameFingerprint(recorded, current)) return false; + const newest = Math.max(Number(BigInt(current.mtimeNs) / 1_000_000n), Number(BigInt(current.ctimeNs) / 1_000_000n)); + return recorded.hashedAt - newest > RACY_WINDOW_MS; +} + +/** + * Linux filesystems known to maintain ctime on every content write (statfs + * f_type). Elsewhere a fingerprint can't vouch, and every observation rehashes. + */ +const CTIME_TRUSTED_FS_TYPES = new Set([ + 0xef53, // ext2/ext3/ext4 + 0x58465342, // xfs + 0x9123683e, // btrfs + 0x01021994, // tmpfs + 0xf2f52010, // f2fs + 0x2fc12fc1, // zfs + 0x794c7630, // overlayfs +]); + +/** Whether a statfs f_type is one of those filesystems. */ +export function fsTypeTrustsCtime(type: number | bigint): boolean { + return CTIME_TRUSTED_FS_TYPES.has(Number(type) >>> 0); +} + +export async function filesystemTrustsCtime(root: string): Promise { + if (process.platform !== 'linux') return false; + try { + return fsTypeTrustsCtime((await statfs(root)).type); + } catch { + return false; + } +} + +// =========================================================================== +// Paths and files +// =========================================================================== + +/** + * Where a path's parent directory canonically lies: inside the mount, gone + * (absence is then proven separately), or somewhere a no-follow open would + * still reach through — O_NOFOLLOW guards only the final component. + */ +async function parentPlacement(rootReal: string, lexical: string): Promise<'inside' | 'gone' | { reason: string }> { + try { + const parent = await realpath(dirname(lexical)); + return contained(rootReal, parent) ? 'inside' : { reason: OUTSIDE_PARENT }; + } catch (err) { + const code = errno(err); + if (code === 'ENOENT' || code === 'ENOTDIR') return 'gone'; + return { reason: `cannot resolve the parent directory (${code ?? String(err)})` }; + } +} + +/** + * Observe a mount-relative path without reading it. A `file` here is only a + * stat — anything decided on its bytes is read through a located descriptor, + * or vouched for by a fingerprint of the very inode P was recorded from — so + * only `absent` needs proof beyond the path lookup, and gets it. + */ +export async function observePath(view: MountView, rootReal: string, relativePath: string): Promise { + const lexical = join(view.root, relativePath); + const placement = await parentPlacement(rootReal, lexical); + if (placement === 'gone') return absence(view, rootReal, relativePath); + if (placement !== 'inside') return { kind: 'unobserved', reason: placement.reason }; + let info; + try { + info = await lstat(lexical, { bigint: true }); + } catch (err) { + const code = errno(err); + if (code === 'ENOENT' || code === 'ENOTDIR') return absence(view, rootReal, relativePath); + return { kind: 'unobserved', reason: `cannot stat (${code ?? String(err)})` }; + } + if (info.isSymbolicLink()) { + if (!view.followSymlinks) return { kind: 'unobserved', reason: NOT_FOLLOWED }; + let target; + try { + target = await stat(lexical, { bigint: true }); + } catch (err) { + return { kind: 'unobserved', reason: `a symlink whose target cannot be read (${errno(err) ?? String(err)})` }; + } + let real: string; + try { + real = await realpath(lexical); + } catch (err) { + return { kind: 'unobserved', reason: `a symlink that cannot be resolved (${errno(err) ?? String(err)})` }; + } + if (!contained(rootReal, real)) return { kind: 'unobserved', reason: 'a symlink that leaves the mount' }; + if (!target.isFile()) return { kind: 'unobserved', reason: 'not a regular file' }; + info = target; + } else if (info.isDirectory()) { + return { kind: 'unobserved', reason: 'a directory' }; + } else if (!info.isFile()) { + return { kind: 'unobserved', reason: 'not a regular file' }; + } + return { kind: 'file', size: Number(info.size), mtimeMs: Number(info.mtimeMs), fp: statFingerprint(info) }; +} + +/** + * Open a mount-relative file for reading, located: a regular file that + * canonically lies inside the mount — whatever the symlink policy, since a + * symlinked parent escapes O_NOFOLLOW — and is the file the canonical path + * names now. The handle is the caller's to close. + */ +async function openContained( + view: MountView, + rootReal: string, + relativePath: string, +): Promise<{ handle: FileHandle; info: BigIntStats } | Exclude> { + const lexical = join(view.root, relativePath); + const noFollow = !view.followSymlinks && typeof fsConstants.O_NOFOLLOW === 'number'; + if (!view.followSymlinks && !noFollow) { + // No O_NOFOLLOW (Windows): refuse a symlink by lstat before opening. + const seen = await observePath(view, rootReal, relativePath); + if (seen.kind !== 'file') return seen; + } + let handle; + try { + // O_NONBLOCK: a FIFO swapped in since any lstat opens at once instead of + // waiting for a writer, and the regular-file check below refuses it. + handle = await open(lexical, fsConstants.O_RDONLY | NONBLOCK | (noFollow ? fsConstants.O_NOFOLLOW : 0)); + } catch (err) { + const code = errno(err); + if (code === 'ENOENT' || code === 'ENOTDIR') return absence(view, rootReal, relativePath); + if (code === 'ELOOP' && !view.followSymlinks) return { kind: 'unobserved', reason: NOT_FOLLOWED }; + if (code === 'EISDIR') return { kind: 'unobserved', reason: 'a directory' }; + return { kind: 'unobserved', reason: `cannot open (${code ?? String(err)})` }; + } + let keep = false; + try { + const info = await handle.stat({ bigint: true }); + if (!info.isFile()) return { kind: 'unobserved', reason: info.isDirectory() ? 'a directory' : 'not a regular file' }; + const real = await realpath(lexical).catch(() => null); + if (real === null) return { kind: 'unobserved', reason: 'the file changed while it was read' }; + if (!contained(rootReal, real)) { + return { kind: 'unobserved', reason: view.followSymlinks ? 'a symlink that leaves the mount' : OUTSIDE_PARENT }; + } + const named = await stat(real, { bigint: true }).catch(() => null); + if (named === null || named.dev !== info.dev || named.ino !== info.ino) { + return { kind: 'unobserved', reason: 'the file changed while it was read' }; + } + if (!view.followSymlinks) { + // Where no symlink is followed, the path itself must be the file the + // descriptor holds: without O_NOFOLLOW, a link swapped in after the + // lstat above is followed by the open. + const entry = await lstat(lexical, { bigint: true }).catch(() => null); + if (entry?.isSymbolicLink()) return { kind: 'unobserved', reason: NOT_FOLLOWED }; + if (entry === null || !entry.isFile() || entry.dev !== info.dev || entry.ino !== info.ino) { + return { kind: 'unobserved', reason: 'the file changed while it was read' }; + } + } + keep = true; + return { handle, info }; + } catch (err) { + const code = errno(err); + if (code === 'ENOENT') return absence(view, rootReal, relativePath); + return { kind: 'unobserved', reason: `cannot read (${code ?? String(err)})` }; + } finally { + if (!keep) await handle.close(); + } +} + +/** + * Read and hash a mount-relative file through one located descriptor. The + * whole file is returned when it is at most `limit` bytes; larger files are + * hashed in chunks and only their head is kept. + */ +export async function readPath( + view: MountView, + rootReal: string, + relativePath: string, + limit: number, +): Promise> { + const opened = await openContained(view, rootReal, relativePath); + if (!('handle' in opened)) return opened; + const { handle, info } = opened; + try { + const hashedAt = Date.now(); + const size = Number(info.size); + const fp: Fingerprint = { ...statFingerprint(info), hashedAt }; + if (size <= limit) { + const bytes = await handle.readFile(); + return { + kind: 'read', + size: bytes.byteLength, + mtimeMs: Number(info.mtimeMs), + fp: { ...fp, size: bytes.byteLength }, + hash: createHash('sha256').update(bytes).digest('hex'), + bytes, + head: bytes.subarray(0, HEAD_BYTES), + }; + } + const hash = createHash('sha256'); + const chunk = Buffer.alloc(HASH_CHUNK); + let head: Buffer | null = null; + let offset = 0; + for (;;) { + const { bytesRead } = await handle.read(chunk, 0, HASH_CHUNK, offset); + if (bytesRead === 0) break; + if (head === null) head = Buffer.from(chunk.subarray(0, Math.min(bytesRead, HEAD_BYTES))); + hash.update(chunk.subarray(0, bytesRead)); + offset += bytesRead; + } + return { + kind: 'read', + size: offset, + mtimeMs: Number(info.mtimeMs), + fp: { ...fp, size: offset }, + hash: hash.digest('hex'), + bytes: null, + head: head ?? Buffer.alloc(0), + }; + } catch (err) { + const code = errno(err); + if (code === 'ENOENT') return absence(view, rootReal, relativePath); + return { kind: 'unobserved', reason: `cannot read (${code ?? String(err)})` }; + } finally { + await handle.close(); + } +} + +/** + * The first bytes of a mount-relative file and its size, through the same + * located descriptor as readPath — for listing a file nothing tracks, which + * needs a MIME type, not a hash. + */ +export async function readHead( + view: MountView, + rootReal: string, + relativePath: string, +): Promise> { + const opened = await openContained(view, rootReal, relativePath); + if (!('handle' in opened)) return opened; + const { handle, info } = opened; + try { + const head = Buffer.alloc(HEAD_BYTES); + const { bytesRead } = await handle.read(head, 0, HEAD_BYTES, 0); + return { kind: 'head', size: Number(info.size), mtimeMs: Number(info.mtimeMs), head: head.subarray(0, bytesRead) }; + } catch (err) { + return { kind: 'unobserved', reason: `cannot read (${errno(err) ?? String(err)})` }; + } finally { + await handle.close(); + } +} + +/** Null bytes in the first 8 KB: the store holds text only from disk. */ +export function looksBinary(head: Buffer): boolean { + return head.subarray(0, HEAD_BYTES).includes(0); +} + +/** Image MIME type from magic bytes, for listing disk-only files. */ +export function sniffImageMime(head: Buffer): string | undefined { + if (head.length >= 8 && head.subarray(0, 8).equals(Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]))) return 'image/png'; + if (head.length >= 3 && head[0] === 0xff && head[1] === 0xd8 && head[2] === 0xff) return 'image/jpeg'; + if (head.length >= 6 && (head.subarray(0, 6).toString('ascii') === 'GIF87a' || head.subarray(0, 6).toString('ascii') === 'GIF89a')) return 'image/gif'; + if (head.length >= 12 && head.subarray(0, 4).toString('ascii') === 'RIFF' && head.subarray(8, 12).toString('ascii') === 'WEBP') return 'image/webp'; + return undefined; +} + +// =========================================================================== +// Walks +// =========================================================================== + +export interface Walk { + /** Regular files seen. */ + files: Set; + /** Directories seen. */ + dirs: Set; + /** Names seen that are neither (symlinks, sockets…), or ignored: present, unproven. */ + others: Map; + /** Directories listed completely. */ + complete: Set; + /** Directories that don't exist (everything beneath them is absent). */ + missing: Set; + /** + * Where the walk didn't reach, and why. `error` and `cap` regions may hide + * files the listing would show; `ignored` and `symlink` ones matter only + * where tracked entries lie beneath them. + */ + incomplete: Array<{ path: string; reason: string; kind: 'error' | 'cap' | 'ignored' | 'symlink' }>; +} + +/** Simple ignore patterns, as the watcher and the old walk read them. */ +export function isIgnored(relativePath: string, name: string, patterns: string[]): boolean { + for (const pattern of patterns) { + if (pattern === name) return true; + if (pattern.endsWith('/**')) { + const prefix = pattern.slice(0, -3); + if (relativePath === prefix || relativePath.startsWith(prefix + '/')) return true; + } + if (!pattern.includes('/') && relativePath === pattern) return true; + if (pattern.startsWith('*.') && name.endsWith(pattern.slice(1))) return true; + } + return false; +} + +/** + * A subdirectory a checked directory's listing showed, checked in turn: still a + * real directory beneath that same directory. One that became a symlink or a + * file since is never followed, and proves nothing. + */ +async function checkChild(view: MountView, rootReal: string, parent: CheckedDir, rel: string): Promise { + const real = parent.at(nameOf(rel)); + let info; + try { + info = await lstat(real, { bigint: true }); + } catch (err) { + if (errno(err) !== 'ENOENT') return { kind: 'unobserved', reason: `cannot stat (${errno(err) ?? String(err)})` }; + const proof = await confirmAbsent(view, rootReal, rel, 'beneath'); + return typeof proof === 'string' ? { kind: 'unobserved', reason: proof } : { kind: 'absent', by: proof.by }; + } + if (!info.isDirectory()) return { kind: 'unobserved', reason: CHANGED_WALK }; + return { kind: 'checked', dir: new CheckedDir(rel, real, lexicalOf(view, rel), info.dev, info.ino) }; +} + +/** + * List a directory scope ('' for the mount root), descending when recursive, + * up to `cap` files in total. A scope that resolves outside the mount is not + * listed at all; beneath it, the walk only descends real directories, never + * symlinks. Each directory's listing counts only if, once listed, it is still + * the directory at its path: a directory moved or replaced meanwhile is an + * incomplete region, contributing no names and proving nothing absent. + */ +export async function walkScope(view: MountView, rootReal: string, scope: string, recursive: boolean, cap: number): Promise { + const walk: Walk = { files: new Set(), dirs: new Set(), others: new Map(), complete: new Set(), missing: new Set(), incomplete: [] }; + const top = await checkDir(view, rootReal, scope); + if (top.kind === 'absent') { + walk.missing.add(scope); + return walk; + } + if (top.kind === 'unobserved') { + walk.incomplete.push({ path: scope, reason: top.outside ? 'the directory resolves outside the mount' : top.reason, kind: 'error' }); + return walk; + } + let capped = false; + + const visit = async (dir: CheckedDir): Promise => { + const rel = dir.rel; + if (capped) { + walk.incomplete.push({ path: rel, reason: `not visited: the file cap (${cap}) was reached`, kind: 'cap' }); + return; + } + let entries; + try { + entries = await readdir(dir.real, { withFileTypes: true }); + } catch (err) { + const code = errno(err); + if (code === 'ENOENT' || code === 'ENOTDIR') { + // Gone since it was checked: absent beneath only if that can be shown. + const proof = rel === '' ? ROOT_UNAVAILABLE : await confirmAbsent(view, rootReal, rel, 'beneath'); + if (typeof proof === 'string') walk.incomplete.push({ path: rel, reason: proof, kind: 'error' }); + else walk.missing.add(rel); + return; + } + walk.incomplete.push({ path: rel, reason: `cannot list (${code ?? String(err)})`, kind: 'error' }); + return; + } + const files: string[] = []; + const subdirs: string[] = []; + const others: Array<[string, string]> = []; + const regions: Walk['incomplete'] = []; + let reachedCap = false; + for (const entry of entries) { + const childRel = rel ? `${rel}/${entry.name}` : entry.name; + if (isIgnored(childRel, entry.name, view.ignore)) { + others.push([childRel, 'ignored by the mount']); + regions.push({ path: childRel, reason: 'ignored by the mount', kind: 'ignored' }); + continue; + } + if (entry.isDirectory()) { + subdirs.push(childRel); + } else if (entry.isFile()) { + if (walk.files.size + files.length >= cap) { + capped = true; + reachedCap = true; + regions.push({ path: rel, reason: `the file cap (${cap}) was reached`, kind: 'cap' }); + break; + } + files.push(childRel); + } else { + const symlink = entry.isSymbolicLink(); + others.push([childRel, symlink ? 'a symlink' : 'not a regular file']); + if (symlink) regions.push({ path: childRel, reason: 'a symlink, not walked', kind: 'symlink' }); + } + } + // A listing of a directory that was moved or replaced meanwhile says + // nothing about this path: neither its names nor its absences count. + if (!(await dir.stillHere())) { + walk.incomplete.push({ path: rel, reason: CHANGED_WALK, kind: 'error' }); + return; + } + for (const f of files) walk.files.add(f); + for (const d of subdirs) walk.dirs.add(d); + for (const [p, why] of others) walk.others.set(p, why); + walk.incomplete.push(...regions); + if (reachedCap) return; + walk.complete.add(rel); + if (!recursive) return; + for (const sub of subdirs) { + const child = await checkChild(view, rootReal, dir, sub); + if (child.kind === 'checked') await visit(child.dir); + else if (child.kind === 'absent') walk.missing.add(sub); + else walk.incomplete.push({ path: sub, reason: child.reason, kind: 'error' }); + } + }; + + await visit(top.dir); + return walk; +} + +/** + * Whether the walk proves nothing exists at `path`: some ancestor directory + * is missing, or a completely listed ancestor holds no entry that could lead + * to it. Ancestors outside the walked scope are passed through, so a walk of + * `a/b` decides paths beneath `a/b` and nothing else. + */ +export function provenAbsent(walk: Walk, path: string): boolean { + const parts = path.split('/'); + let dir = ''; + for (let i = 0; i < parts.length; i++) { + if (walk.missing.has(dir)) return true; + const child = dir ? `${dir}/${parts[i]}` : parts[i]!; + if (walk.complete.has(dir)) { + if (walk.others.has(child)) return false; // present, but unproven + if (i === parts.length - 1) return !walk.files.has(child) && !walk.dirs.has(child); + if (!walk.dirs.has(child)) return true; // no directory there: nothing beneath can exist + } + dir = child; + } + return false; +} diff --git a/src/modules/workspace/reconcile.ts b/src/modules/workspace/reconcile.ts new file mode 100644 index 000000000..f14dd35ff --- /dev/null +++ b/src/modules/workspace/reconcile.ts @@ -0,0 +1,1136 @@ +/** + * The three-way rule over disk (D), the workspace store (S) and physical + * evidence (P), with branch-local intent (tombstones, store origin, sticky + * conflicts). Under it a scan or read never discards a pending store edit, + * and a branch switch is never mistaken for a disk edit: + * + * - D = S: agreement; P := D. + * - P known: D = P keeps the store side (a draft, a workspace deletion, or a + * file this branch doesn't have); S = P adopts disk (ingest, or remove S, + * or — for a binary/oversize disk version — drop S so the path reads + * disk-only); all three differing is a conflict. + * - P unknown: a tombstone, then store origin, then legacy entries decide; + * otherwise a new disk file is ingested or listed disk-only. + * - P pending (an intent whose effect may or may not have landed) resolves + * first; P interrupted is a sticky conflict until it's resolved explicitly + * or disk and the store agree. Another branch's pending adoption is left + * for its own branch to resolve, even where this branch agrees with disk. + * + * Only a push completes a push. A pending disk intent is the module's own + * write or unlink, and its completion releases the protection a draft has + * (P := what was written), so it is recorded only by a push whose barriers + * succeeded. An observation that finds the intended outcome on disk decides + * the path's state from it but records nothing for it — no completion, no + * agreement, no branch-intent change — and the path stays owed until a push + * redoes it. Otherwise a crash that loses an unbarriered write would leave + * P saying the draft reached disk, and the rule would adopt the older disk + * version over it. (An explicit path sync still chooses disk, replacing + * whatever the path's evidence was; that adopts disk, it doesn't certify the + * push.) + * + * Each pass observes disk first (the only async step), then reads S, P and + * intent and applies every decision in one synchronous step, in the agreed + * order: intents (durable) → tree commits and intent records → completions + * (after `store.sync()`). The caller serializes passes per mount. A drop (S + * removed for a disk copy the workspace can't hold) is the one tree adoption + * without an intent: it forgets P first, which is conservative, since a crash + * before the removal leaves P unknown with S present, a conflict, and never + * a wrong adoption. + */ + +import type { JsStore } from '@animalabs/chronicle'; +import { + type Agreed, + type Candidate, + type DiskAgreement, + type Fingerprint, + type Interrupted, + type Pending, + type Physical, + type RootIdentity, + candidateOf, + sameCandidate, + sameRootIdentity, +} from './disk-agreement.js'; +import type { BranchIntent, BranchIntents, ConflictKind, ConflictRecord, DiskCounterpart } from './branch-intent.js'; +import { + type MountView, + ROOT_REPLACED, + type Walk, + fingerprintVouches, + locateRoot, + looksBinary, + observePath, + provenAbsent, + readHead, + readPath, + sniffImageMime, + walkScope, +} from './observe.js'; +import { EffectFailed, type Expect, chainOf, syncDirectories, unlinkContained, writeContained } from './effects.js'; + +export type EntryState = + | 'synced' + | 'workspace-draft' + | 'workspace-deleted' + | 'not-in-branch' + | 'disk-only' + | 'conflict' + | 'disk-missing-provenance-unknown' + /** Tracked, but disk wasn't observed this pass (unreadable, ignored, unwalked). */ + | 'unverified'; + +export interface ConflictReport { + kind: ConflictKind; + /** Where the disk side is kept: in the store, by reference to the disk file, or absent. */ + diskCopy: 'stored' | 'referenced' | 'absent'; + /** The disk file no longer matches what was recorded with the conflict. */ + diskChangedSinceRecorded: boolean; +} + +export interface PathReport { + path: string; + state: EntryState; + /** Store size for entries in the store; disk size otherwise. */ + size?: number; + mimeType?: string; + conflict?: ConflictReport; + note?: string; + /** + * For a conflict, what this pass found on disk at the path: grep's account + * of a conflict's disk side needs it. Listings don't show it. + */ + diskNow?: 'file' | 'absent'; +} + +export type TreeOp = 'created' | 'modified' | 'deleted'; + +export interface PassResult { + /** Tree changes adopted from disk. */ + ops: Array<{ path: string; op: TreeOp }>; + /** What a path sync gave up of the workspace's own; `op` is what it did to the + * workspace copy, absent when it changed none (a deletion or conflict given + * up for a disk copy the workspace can't hold). */ + discarded: Array<{ path: string; was: EntryState; op?: TreeOp }>; + /** Conflicts recorded by this pass, with the disk change behind each. */ + newConflicts: Array<{ path: string; op: TreeOp }>; + /** Every path the pass decided, by path. */ + reports: Map; + /** Regions the pass couldn't observe that matter to its scope. */ + incomplete: Array<{ path: string; reason: string }>; + /** For a directory scope: the subdirectories disk shows directly beneath it. */ + dirs: string[]; + /** + * The branch the pass decided on, absent when it decided nothing. A caller + * that reads the tree or branch intent to go with the reports checks it is + * still the selected branch, synchronously, before it reads. + */ + branchId?: string; +} + +export interface MountRuntime { + name: string; + view: MountView; + /** Canonical mount root, for symlink containment. */ + rootReal: string; + /** + * The device of the mount root's filesystem when that filesystem maintains + * ctime, else null. A fingerprint vouches, and is recorded, only for a file + * on that device: a file on another filesystem mounted beneath the root + * (which wasn't checked) is rehashed at every observation. + */ + ctimeTrustedDev: string | null; + treeStateId: string; + intents: BranchIntents; + readOnly: boolean; +} + +export type Scope = + | { kind: 'paths'; paths: string[] } + | { kind: 'dir'; dir: string; recursive: boolean }; + +export interface PassOptions { + /** Rehash every file instead of trusting fingerprints (an explicit sync). */ + rehash?: boolean; + /** Paths whose disk state is adopted explicitly (a path sync). */ + adopt?: (path: string) => boolean; + /** Walk cap for directory scopes. */ + cap?: number; + /** + * The branch the caller asked on (a path sync). If another is selected when + * the pass gathers, it decides nothing: an adoption asked for on one branch + * would give up another branch's drafts. + */ + branchId?: string; +} + +const ABSENT: Candidate = { kind: 'absent' }; +export const WALK_CAP = 5000; + +/** What a pass learned about D for one path. */ +type DiskFact = + | { kind: 'absent' } + | { kind: 'unobserved'; reason: string } + | { + kind: 'file'; + hash: string; + size: number; + mtimeMs: number; + fp: Fingerprint | null; + /** Text within limits, with its bytes: what disk→store sync may store. */ + ingestible: Buffer | null; + mimeType?: string; + } + /** + * A file nothing tracks, too large to ingest: listed from its size and head + * alone. Its bytes were never hashed, so it can't be compared with anything + * — it is never evidence of agreement, only of a disk-only file. + */ + | { kind: 'unhashed'; size: number; mtimeMs: number; mimeType?: string }; + +function diskCandidate(d: DiskFact): Candidate | null { + if (d.kind === 'absent') return ABSENT; + if (d.kind === 'file') return { kind: 'content', hash: d.hash }; + return null; +} + +function agreedFromDisk(d: DiskFact): Agreed { + if (d.kind !== 'file') return { kind: 'absent' }; + return { kind: 'content', hash: d.hash, size: d.size, ...(d.fp ? { fp: d.fp } : {}) }; +} + +function inScope(path: string, scope: Scope): boolean { + if (scope.kind === 'paths') return scope.paths.includes(path); + const dir = scope.dir; + if (dir === '') return scope.recursive || !path.includes('/'); + if (!path.startsWith(dir + '/')) return false; + return scope.recursive || !path.slice(dir.length + 1).includes('/'); +} + +/** + * Learn D for a path. Content is read only when a decision needs it; a + * recorded fingerprint can vouch for unchanged bytes where the filesystem + * maintains ctime. + */ +async function learnDisk( + mount: MountRuntime, + path: string, + tracked: { p: Physical | undefined; hasStore: boolean; hasIntent: boolean }, + opts: PassOptions, +): Promise { + const seen = await observePath(mount.view, mount.rootReal, path); + if (seen.kind !== 'file') return seen; + const p = tracked.p; + const trusted = (fp: { dev: string }) => mount.ctimeTrustedDev !== null && fp.dev === mount.ctimeTrustedDev; + if (!opts.rehash && p?.kind === 'content' && fingerprintVouches(p.fp, seen.fp, trusted(seen.fp))) { + return { kind: 'file', hash: p.hash, size: seen.size, mtimeMs: seen.mtimeMs, fp: p.fp ?? null, ingestible: null }; + } + const nothingTracked = p === undefined && !tracked.hasStore && !tracked.hasIntent; + if (nothingTracked && seen.size > mount.view.maxFileSize) { + // A new oversize file only needs listing: its size, and its head for the + // MIME type. Hashing it would read all of it, on every listing. + const head = await readHead(mount.view, mount.rootReal, path); + if (head.kind !== 'head') return head; + return { kind: 'unhashed', size: head.size, mtimeMs: head.mtimeMs, mimeType: sniffImageMime(head.head) }; + } + const read = await readPath(mount.view, mount.rootReal, path, mount.view.maxFileSize); + if (read.kind !== 'read') return read; + const ingestible = read.bytes !== null && !looksBinary(read.head) ? read.bytes : null; + return { + kind: 'file', + hash: read.hash, + size: read.size, + mtimeMs: read.mtimeMs, + // A fingerprint is recorded only where it can later vouch for the bytes. + fp: trusted(read.fp) ? read.fp : null, + ingestible, + ...(ingestible ? {} : { mimeType: sniffImageMime(read.head) }), + }; +} + +/** The rule's verdict for one path, before effects. */ +interface Verdict { + state: EntryState; + /** Adopt disk into the tree: set S to D's bytes, or remove S. */ + adopt?: 'ingest' | 'remove' | 'drop'; + /** New P (undefined: unchanged; 'forget': unknown). */ + p?: Agreed | Interrupted | 'forget'; + /** New branch intent (undefined: unchanged). */ + intent?: BranchIntent; + /** A conflict to record now. */ + conflict?: ConflictKind; + op?: TreeOp; + note?: string; + /** + * Disk shows what a pending push of ours intended, unconfirmed: the state + * is decided from it, and nothing at all is recorded (see module doc). + */ + unconfirmed?: true; + /** A path sync's adoption gives up this workspace state (see decide). */ + discards?: EntryState; + /** + * The last confirmed evidence decide resolved for the path, which an + * adoption's intent carries as its prior: a settled same-branch adoption's + * outcome, where the recorded P is still that adoption's pending record. + */ + prior?: Agreed; +} + +/** States in which the rule keeps a change of the workspace's own over disk. */ +const WORKSPACE_KEPT: ReadonlySet = new Set([ + 'workspace-draft', + 'workspace-deleted', + 'conflict', + 'disk-missing-provenance-unknown', +]); + +const UNCONFIRMED_NOTE = 'disk holds what a materialize that has not completed put here; materialize again to complete it'; + +function keepStoreSide(hasStore: boolean, bi: BranchIntent | null): EntryState { + if (hasStore) return 'workspace-draft'; + return bi?.tombstone ? 'workspace-deleted' : 'not-in-branch'; +} + +/** + * Settle an adoption intent on the branch it was recorded on, by the tree + * alone: S at the adopted content means the commit landed and only its + * completion was lost; anything else means it didn't, and the rule runs again + * from the prior evidence (redoing the adoption if disk still holds it). + * + * Content is proof only while nothing else has written the path since, so + * every workspace mutation settles a pending intent first (the module's + * mutation boundary): a later write of the same content can't pass for the + * old adoption. + */ +function settleAdoption(p: Pending, s: { hash: string; size: number } | null): Agreed | 'forget' { + const sc: Candidate = s ? { kind: 'content', hash: s.hash } : ABSENT; + if (sameCandidate(sc, p.expect)) { + return p.expect.kind === 'absent' ? { kind: 'absent' } : { kind: 'content', hash: p.expect.hash, size: s!.size }; + } + return p.prior ?? 'forget'; +} + +/** + * The synchronous precondition of every workspace mutation: evidence owed a + * barrier is made durable first, and an adoption left pending on the + * selected branch is settled by the tree before the mutation changes it, and + * durably, so the settlement can't be lost while the new write survives. + * Afterwards the write can never pass for the adoption's outcome. Other + * pending intents never consult the tree, so a write can't confuse them. + * Holds whether or not an earlier observation could settle it. + */ +export function settleAdoptionBeforeMutation( + store: JsStore, + agreement: DiskAgreement, + mount: { name: string; treeStateId: string }, + path: string, +): void { + // Evidence still owed a barrier (from a failed one, or read back at open) + // is made durable before any new tree mutation is permitted at all. + if (agreement.needsBarrier) agreement.barrier(); + const p = agreement.get(mount.name, path); + if (p?.kind !== 'pending' || p.effect !== 'adopt' || p.branchId !== store.currentBranch().id) return; + const entry = store.treeGet(mount.treeStateId, path); + const settled = settleAdoption(p, entry ? { hash: entry.blobHash, size: entry.size } : null); + if (settled === 'forget') agreement.forget(mount.name, path); + else agreement.set(mount.name, path, settled, { afterCommittedState: true }); + agreement.barrier(); +} + +/** + * Resolve a pending intent against what was observed. Returns the evidence + * the rule should use, and whether that resolution should be recorded. + */ +function resolvePending( + p: Pending, + dc: Candidate, + s: { hash: string; size: number } | null, + d: DiskFact, + branchId: string, +): { p: Physical | undefined; record?: Agreed | Interrupted | 'forget'; uncaptured?: true; unconfirmed?: true } { + const prior = p.prior; + const priorCandidate = prior ? candidateOf(prior) : null; + if (p.effect === 'disk') { + if (sameCandidate(dc, p.expect)) { + // Our own effect shows on disk, but nothing proves it durable: decided + // on, never recorded. Only a push whose barriers succeed completes it. + const shown = p.expect.kind === 'absent' ? ({ kind: 'absent' } as Agreed) : agreedFromDisk(d); + return { p: shown, unconfirmed: true }; + } + if (priorCandidate && sameCandidate(dc, priorCandidate)) return { p: prior!, record: prior! }; + // A write whose prior was unknown left no file: it never created one (a + // partial write leaves a file behind), so P is as unknown as before. + if (!prior && dc.kind === 'absent' && p.expect.kind === 'content') return { p: undefined, record: 'forget' }; + const interrupted: Interrupted = { kind: 'interrupted', candidates: [...(priorCandidate ? [priorCandidate] : []), p.expect] }; + return { p: interrupted, record: interrupted }; + } + // An adoption: its meaning depends on the branch it was recorded on. + if (p.branchId === branchId) { + const landed = sameCandidate(s ? { kind: 'content', hash: s.hash } : ABSENT, p.expect); + // Landed, with disk at the adopted content too: its fingerprint joins the evidence. + if (landed && p.expect.kind === 'content' && sameCandidate(dc, p.expect)) { + const agreed = agreedFromDisk(d); + return { p: agreed, record: agreed }; + } + const settled = settleAdoption(p, s); + return settled === 'forget' ? { p: undefined, record: 'forget' } : { p: settled, record: settled }; + } + // Another branch, whose S says nothing of whether the adoption happened + // over there. Disk still holding the prior is D = P: the store has that + // content. Disk holding the adopted content, or anything newer, may hold + // the only copy of it, so this branch lists it as a conflict and nothing + // pushes over it; P stays pending for the intent's own branch to resolve. + if (priorCandidate && sameCandidate(dc, priorCandidate)) return { p }; + return { p, uncaptured: true }; +} + +/** + * An explicit path sync: the store takes disk's state, whatever the evidence + * — a pending push's included. That chooses disk; it doesn't certify the push. + */ +function adoptDisk(d: Extract, s: { hash: string; size: number } | null): Verdict { + if (d.kind === 'absent') { + return { state: 'synced', ...(s ? { adopt: 'remove' as const, op: 'deleted' as const } : {}), p: { kind: 'absent' }, intent: {} }; + } + // Disk agreeing with the store is disk's state already, whatever the file + // is: a binary or oversize file the workspace holds stays held. + if (s && s.hash === d.hash) return { state: 'synced', p: agreedFromDisk(d), intent: {} }; + if (d.ingestible) return { state: 'synced', adopt: 'ingest', op: s ? 'modified' : 'created', p: agreedFromDisk(d), intent: {} }; + return { state: 'disk-only', ...(s ? { adopt: 'drop' as const, op: 'deleted' as const } : {}), p: 'forget', intent: {} }; +} + +/** The rule (see module doc). Pure. */ +function decide( + d: DiskFact, + s: { hash: string; size: number } | null, + pIn: Physical | undefined, + bi: BranchIntent | null, + branchId: string, + adopt: boolean, +): Verdict { + // A branch intent that can't be read may have held a tombstone or a + // conflict: nothing is decided over it, until a path sync replaces it. + if (bi?.unreadable !== undefined && !adopt) return { state: 'unverified', note: `its branch intent can't be read: ${bi.unreadable}` }; + if (d.kind === 'unobserved') { + // Disk proves nothing, but an adoption on this branch settles by the tree. + if (pIn?.kind === 'pending' && pIn.effect === 'adopt' && pIn.branchId === branchId && !adopt) { + return { state: 'unverified', note: d.reason, p: settleAdoption(pIn, s) }; + } + return { state: 'unverified', note: d.reason }; + } + if (d.kind === 'unhashed') { + // Observed this way only when nothing tracks the path, which is then a + // disk-only file whatever else holds. Were anything to track it, its + // unhashed bytes could prove nothing about it. + if (pIn === undefined && s === null && bi === null) return { state: 'disk-only' }; + return { state: 'unverified', note: 'not hashed' }; + } + const dc = diskCandidate(d)!; + const sc: Candidate = s ? { kind: 'content', hash: s.hash } : ABSENT; + + if (adopt) { + // What a path sync gives up is exactly what a sync without a path would + // have kept of the workspace's own (a draft, a deletion, a conflict's + // workspace side), and the verdict says which. + const taken = adoptDisk(d, s); + // A give-up is an adoption, or the intent it clears: a deletion or a + // conflict given up for a disk copy the workspace can't hold changes no + // tree entry, and is listed all the same. + if (taken.adopt || (bi && taken.intent !== undefined)) { + const kept = decide(d, s, pIn, bi, branchId, false); + if (WORKSPACE_KEPT.has(kept.state)) taken.discards = kept.state; + if (taken.adopt && kept.prior) taken.prior = kept.prior; + } + return taken; + } + + // A pending intent resolves first; its resolution is recorded unless the + // verdict below records newer evidence. + let p = pIn; + let resolution: Agreed | Interrupted | 'forget' | undefined; + let uncaptured = false; + if (p?.kind === 'pending') { + const resolved = resolvePending(p, dc, s, d, branchId); + if (resolved.unconfirmed) { + // Disk shows our pending push's outcome (P as it intended): the state + // follows from that, and nothing is recorded — no completion, no + // agreement, no branch-intent change — until a push confirms it. + const state: EntryState = sameCandidate(dc, sc) ? 'synced' : bi?.conflict ? 'conflict' : keepStoreSide(s !== null, bi); + return { state, note: UNCONFIRMED_NOTE, unconfirmed: true }; + } + p = resolved.p; + resolution = resolved.record; + uncaptured = resolved.uncaptured === true; + } + const verdict = (v: Verdict): Verdict => (resolution && v.p === undefined ? { ...v, p: resolution } : v); + + // Agreement settles everything, a recorded conflict included. + if (sameCandidate(dc, sc)) { + const settles = bi?.tombstone || bi?.conflict || bi?.origin ? { intent: {} } : {}; + // Another branch's pending adoption is still pending here (resolvePending + // leaves it): that branch resolves it, by its own tree. Recording this + // branch's agreement over it would lose the adoption there. + if (p?.kind === 'pending') return verdict({ state: 'synced', ...settles }); + if (dc.kind === 'absent') { + return verdict({ state: 'synced', ...(p && p.kind !== 'absent' ? { p: { kind: 'absent' } as Agreed } : {}), ...settles }); + } + const agreed = agreedFromDisk(d) as Extract; + const recorded = p?.kind === 'content' ? p : null; + // Re-record only when P says something else, or a fresh fingerprint can + // replace one that no longer vouches. A fresh one exists only where it's + // trusted, so both are judged as trusted here. + const stale = !recorded || recorded.hash !== agreed.hash || recorded.size !== agreed.size || + (agreed.fp !== undefined && fingerprintVouches(agreed.fp, agreed.fp, true) && + !fingerprintVouches(recorded.fp, agreed.fp, true)); + return verdict({ state: 'synced', ...(stale ? { p: agreed } : {}), ...settles }); + } + + // A recorded conflict stays until it is resolved explicitly or disk and store converge. + if (bi?.conflict) return verdict({ state: 'conflict' }); + + if (p?.kind === 'interrupted' || uncaptured) return verdict({ state: 'conflict', conflict: 'interrupted' }); + + if (p?.kind === 'pending') { + // Unresolved from another branch, with disk still holding its prior. + return verdict({ state: keepStoreSide(s !== null, bi) }); + } + + if (p !== undefined) { + const pc = candidateOf(p); + if (sameCandidate(dc, pc)) return verdict({ state: keepStoreSide(s !== null, bi) }); + if (sameCandidate(sc, pc)) { + // Only disk changed since it last agreed: adopt it. P is global, so + // S = P means the workspace holds what disk last agreed with, whichever + // branch that agreement came from, and a branch's intent doesn't + // outrank it. A known P of absent means disk did lose the file, which + // settles a tombstone here: a file disk shows now is a new one, and + // adopting it gives up nothing of the workspace's own. A tombstone + // guards a deletion only while P is unknown (below). + const settles = bi ? { intent: {} } : {}; + // The adoption's intent carries this resolved evidence as its prior. + const prior = p.kind === 'absent' || p.kind === 'content' ? { prior: p } : {}; + if (d.kind === 'absent') return { state: 'synced', adopt: 'remove', op: 'deleted', p: { kind: 'absent' }, ...prior, ...settles }; + if (d.ingestible) { + return { state: 'synced', adopt: 'ingest', op: s ? 'modified' : 'created', p: agreedFromDisk(d), ...prior, ...settles }; + } + return { state: 'disk-only', ...(s ? { adopt: 'drop' as const, op: 'deleted' as const } : {}), p: 'forget', ...settles }; + } + const kind: ConflictKind = d.kind === 'absent' + ? 'deleted-on-disk' + : !s && bi?.tombstone ? 'changed-after-workspace-delete' : 'both-changed'; + return verdict({ state: 'conflict', conflict: kind }); + } + + // No evidence: intent and provenance decide. + if (!s && bi?.tombstone) { + if (d.kind === 'file' && d.hash === bi.tombstone.hash) return { state: 'workspace-deleted' }; + return { state: 'conflict', conflict: 'changed-after-workspace-delete' }; + } + if (s) { + if (d.kind === 'absent') return { state: bi?.origin === 'store' ? 'workspace-draft' : 'disk-missing-provenance-unknown' }; + return { state: 'conflict', conflict: bi?.origin === 'store' ? 'store-origin-collision' : 'unknown-provenance' }; + } + // A new disk file. + if (d.kind === 'file' && d.ingestible) return { state: 'synced', adopt: 'ingest', op: 'created', p: agreedFromDisk(d) }; + return { state: 'disk-only' }; +} + +function counterpartOf(store: JsStore, d: DiskFact): DiskCounterpart | null { + if (d.kind !== 'file') return null; + const stored = d.ingestible ? store.storeBlob(d.ingestible, 'text/plain') : undefined; + return { hash: d.hash, size: d.size, mtimeMs: d.mtimeMs, ...(stored ? { stored } : {}) }; +} + +function conflictReport(record: ConflictRecord, d: DiskFact): ConflictReport { + const current = diskCandidate(d); + const recorded: Candidate = record.disk ? { kind: 'content', hash: record.disk.hash } : ABSENT; + return { + kind: record.kind, + diskCopy: record.disk === null ? 'absent' : record.disk.stored ? 'stored' : 'referenced', + diskChangedSinceRecorded: current !== null && !sameCandidate(current, recorded), + }; +} + +/** A pass's view of its scope: candidates enumerated on one branch, and what disk showed for each. */ +interface Gathered { + branchId: string; + /** The root the facts were gathered under, when it was available. */ + rootIdentity: RootIdentity | null; + walk: Walk | null; + facts: Map; + dirs: string[]; + incomplete: Array<{ path: string; reason: string }>; +} + +/** How often a pass gathers again when the branch changes under it. */ +const GATHER_ATTEMPTS = 3; + +async function gather(store: JsStore, agreement: DiskAgreement, mount: MountRuntime, scope: Scope, opts: PassOptions): Promise { + // A missing mount root is an unavailable mount (an unmounted drive, a root + // being replaced), not proof that every file in it was deleted, and so is a + // root that isn't the directory disk last agreed under: the empty + // directory a watcher makes where a drive's mountpoint was. The root is + // the operator's choice and may itself be a symlink to a directory, so it + // is followed, as `rootReal` is: `followSymlinks` governs links within it. + const root = await locateRoot(mount.view, mount.rootReal); + const rootUnavailable = 'reason' in root ? root.reason : null; + + // The branch is labelled where the tracked candidates are read, with no + // await between: the decision is checked against this label. + const branchId = store.currentBranch().id; + const gathered: Gathered = { + branchId, + rootIdentity: 'identity' in root ? root.identity : null, + walk: null, + facts: new Map(), + dirs: [], + incomplete: [], + }; + if (rootUnavailable !== null) { + // Said whether or not anything is tracked: an unavailable mount must not + // read as a verified empty directory. + gathered.incomplete.push({ path: scope.kind === 'dir' ? scope.dir : '', reason: rootUnavailable }); + } + + // Which paths to decide: what disk shows, and what the store, P and intent + // track. The tracked ones are read synchronously, on this branch. + const candidates = new Set(); + let walk: Walk | null = null; + if (scope.kind === 'paths') { + for (const p of scope.paths) candidates.add(p); + } else { + const prefix = scope.dir ? scope.dir + '/' : undefined; + for (const e of store.treeList(mount.treeStateId, prefix)) if (inScope(e.path, scope)) candidates.add(e.path); + for (const [p] of agreement.paths(mount.name, scope.dir)) if (inScope(p, scope)) candidates.add(p); + for (const [p] of mount.intents.list(scope.dir)) if (inScope(p, scope)) candidates.add(p); + if (rootUnavailable === null) { + walk = await walkScope(mount.view, mount.rootReal, scope.dir, scope.recursive, opts.cap ?? WALK_CAP); + for (const f of walk.files) if (inScope(f, scope)) candidates.add(f); + // Ignored names are listed only when something tracks them (above). + for (const [p, why] of walk.others) if (why !== 'ignored by the mount' && inScope(p, scope)) candidates.add(p); + for (const d of walk.dirs) if (inScope(d, { kind: 'dir', dir: scope.dir, recursive: false })) gathered.dirs.push(d); + } + } + gathered.walk = walk; + + // Observe. Tracked facts are read again, synchronously, when deciding. + for (const path of candidates) { + if (rootUnavailable !== null) { + gathered.facts.set(path, { kind: 'unobserved', reason: rootUnavailable }); + continue; + } + if (walk && provenAbsent(walk, path)) { + gathered.facts.set(path, { kind: 'absent' }); + continue; + } + if (walk && !walk.files.has(path) && !walk.others.has(path)) { + // Beneath a region the walk didn't reach: unproven either way. + const region = walk.incomplete.find((r) => r.path === '' || path === r.path || path.startsWith(r.path + '/')); + gathered.facts.set(path, { kind: 'unobserved', reason: region ? region.reason : 'not reached by the walk' }); + continue; + } + if (walk?.others.get(path) === 'ignored by the mount') { + gathered.facts.set(path, { kind: 'unobserved', reason: 'ignored by the mount' }); + continue; + } + const p = agreement.get(mount.name, path); + const hasStore = store.treeGet(mount.treeStateId, path) !== null; + const hasIntent = mount.intents.get(path) !== null; + gathered.facts.set(path, await learnDisk(mount, path, { p, hasStore, hasIntent }, opts)); + } + + // Everything above was observed by path beneath the root's path. A root + // replaced meanwhile lends its replacement's files, and their absence, to + // the original's paths, so if the root isn't the one the pass began under + // now, nothing observed counts and nothing is decided from it. + if (gathered.rootIdentity !== null) { + const after = await locateRoot(mount.view, mount.rootReal); + if (!('identity' in after) || !sameRootIdentity(after.identity, gathered.rootIdentity)) { + const reason = 'reason' in after ? after.reason : ROOT_REPLACED; + for (const path of gathered.facts.keys()) gathered.facts.set(path, { kind: 'unobserved', reason }); + gathered.rootIdentity = null; + gathered.walk = null; + gathered.dirs = []; + gathered.incomplete.push({ path: scope.kind === 'dir' ? scope.dir : '', reason }); + } + } + return gathered; +} + +/** + * One disk→store pass over a scope: the watcher's paths, a lazy read, a + * listing's directory, a full scan, or a path sync (opts.adopt). + */ +export async function reconcilePass( + store: JsStore, + agreement: DiskAgreement, + mount: MountRuntime, + scope: Scope, + opts: PassOptions = {}, +): Promise { + agreement.ensureReconciled(); + // Evidence a failed barrier left unsynced is settled before anything is + // decided from it; if the barrier fails again, so does the pass. + if (agreement.needsBarrier) agreement.barrier(); + const result: PassResult = { ops: [], discarded: [], newConflicts: [], reports: new Map(), incomplete: [], dirs: [] }; + + // Enumerate on one branch and observe disk (async). The decision below must + // describe the same branch the candidates came from: a branch switch during + // observation would otherwise yield a "complete" listing that omits the new + // branch's drafts. So a changed branch gathers again; a branch that keeps + // changing leaves the pass explicitly incomplete, deciding nothing. + let gathered: Gathered | null = null; + for (let attempt = 0; attempt < GATHER_ATTEMPTS; attempt++) { + const next = await gather(store, agreement, mount, scope, opts); + if (store.currentBranch().id === next.branchId) { + gathered = next; + break; + } + } + if (gathered === null) { + result.incomplete.push({ path: scope.kind === 'dir' ? scope.dir : '', reason: 'the selected branch kept changing while the scan ran' }); + return result; + } + if (opts.branchId !== undefined && gathered.branchId !== opts.branchId) { + result.incomplete.push({ + path: scope.kind === 'dir' ? scope.dir : scope.paths.join(', '), + reason: 'the selected branch changed before this sync ran; nothing was taken from disk — sync again on this branch to take it', + }); + return result; + } + const { walk, facts } = gathered; + result.dirs.push(...gathered.dirs); + result.incomplete.push(...gathered.incomplete); + + // Decide and apply — synchronously, so nothing interleaves. + const branchId = store.currentBranch().id; + result.branchId = branchId; + type Planned = { path: string; verdict: Verdict; d: DiskFact; s: { hash: string; size: number } | null; p: Physical | undefined; bi: BranchIntent | null }; + const planned: Planned[] = []; + for (const [path, d] of facts) { + const entry = store.treeGet(mount.treeStateId, path); + const s = entry ? { hash: entry.blobHash, size: entry.size } : null; + const p = agreement.get(mount.name, path); + const bi = mount.intents.get(path); + const verdict = decide(d, s, p, bi, branchId, opts.adopt?.(path) === true); + planned.push({ path, verdict, d, s, p, bi }); + } + + // The evidence about to be recorded describes the root it was gathered + // under, which is recorded first (once per mount), so no crash can leave + // evidence without its root. + if (gathered.rootIdentity && planned.some((x) => x.verdict.adopt !== undefined || x.verdict.p !== undefined)) { + agreement.recordRootIdentity(mount.name, gathered.rootIdentity); + } + + // 1. Intents for adoptions, durable before any tree commit. + const adoptions = planned.filter((x) => x.verdict.adopt === 'ingest' || x.verdict.adopt === 'remove'); + adoptions.forEach((x, i) => { + const prior = x.verdict.prior ?? (x.p && (x.p.kind === 'absent' || x.p.kind === 'content') ? x.p : null); + agreement.intend(mount.name, x.path, { + effect: 'adopt', + prior, + expect: x.verdict.adopt === 'remove' ? ABSENT : { kind: 'content', hash: (x.d as { hash: string }).hash }, + branchId, + }, { durable: i === adoptions.length - 1 }); + }); + + // 2. Forgetting P for a dropped entry is conservative, so it goes first. + for (const x of planned) if (x.verdict.adopt === 'drop') agreement.forget(mount.name, x.path); + + // 3. Tree commits and branch intent. + let committed = false; + for (const x of planned) { + const v = x.verdict; + if (v.adopt === 'ingest' && x.d.kind === 'file' && x.d.ingestible) { + const blobHash = store.storeBlob(x.d.ingestible, 'text/plain'); + store.treeSet(mount.treeStateId, x.path, { blobHash, size: x.d.ingestible.byteLength, mode: 0o644 }); + committed = true; + } else if ((v.adopt === 'remove' || v.adopt === 'drop') && x.s) { + store.treeRemove(mount.treeStateId, x.path); + committed = true; + } + if (v.conflict) { + const record: ConflictRecord = { kind: v.conflict, at: Date.now(), disk: counterpartOf(store, x.d) }; + mount.intents.update(x.path, (cur) => ({ ...cur, conflict: record })); + result.newConflicts.push({ path: x.path, op: x.d.kind === 'absent' ? 'deleted' : 'modified' }); + committed = true; + } else if (v.intent !== undefined) { + mount.intents.put(x.path, v.intent); + committed = true; + } + if (v.op && v.adopt) result.ops.push({ path: x.path, op: v.op }); + if (v.discards) result.discarded.push({ path: x.path, was: v.discards, ...(v.op && v.adopt ? { op: v.op } : {}) }); + } + + // 4. Completions and agreements, after the commits they assert are synced — + // this pass's, and any earlier tool write's an agreement rests on. + let first = true; + let recorded = false; + for (const x of planned) { + const next = x.verdict.p; + if (x.verdict.adopt === 'drop') recorded = true; // its forget, in step 2 + if (next === undefined) continue; + recorded = true; + if (next === 'forget') { + agreement.forget(mount.name, x.path); // no-op when step 2 already did + continue; + } + agreement.set(mount.name, x.path, next, { afterCommittedState: first }); + first = false; + } + // 5. Durable before returning. An agreement advance (D = S) has no pending + // intent to recover from, so losing its append would leave an older P that + // another branch would misread as a disk edit. + if (recorded || committed || agreement.needsBarrier) agreement.barrier(); + agreement.maybeCheckpoint(); + + // Reports. + for (const x of planned) { + const after = mount.intents.get(x.path); + const entry = store.treeGet(mount.treeStateId, x.path); + const report: PathReport = { path: x.path, state: x.verdict.state }; + if (entry) report.size = entry.size; + else if (x.d.kind === 'file' || x.d.kind === 'unhashed') report.size = x.d.size; + if (x.verdict.state === 'disk-only' && (x.d.kind === 'file' || x.d.kind === 'unhashed') && x.d.mimeType) report.mimeType = x.d.mimeType; + if (x.verdict.state === 'conflict' && after?.conflict) { + report.conflict = conflictReport(after.conflict, x.d); + if (x.d.kind === 'file' || x.d.kind === 'absent') report.diskNow = x.d.kind; + } + if (x.verdict.note) report.note = x.verdict.note; + if (!(x.verdict.state === 'synced' && !entry)) result.reports.set(x.path, report); + } + if (walk) { + const tracked = [...result.reports.values()].filter((r) => r.state === 'unverified').map((r) => r.path); + for (const region of walk.incomplete) { + const hides = tracked.some((p) => p === region.path || p.startsWith(region.path + '/')); + if (region.kind === 'error' || region.kind === 'cap' || hides) result.incomplete.push({ path: region.path, reason: region.reason }); + } + } + return result; +} + +// =========================================================================== +// Store → disk +// =========================================================================== + +export interface PushOptions { + /** + * Overwrite (or unlink) a disk copy in conflict, or one that can't be + * verified. A disk copy nothing has evidence about is overwritten only when + * `named` too: a bare push can't tell it from a newer copy. + */ + force?: boolean; + /** + * The caller named these paths (a path given to materialize), rather than + * the evidence selecting them. Only such a push writes a path nothing has + * evidence about: over a differing disk copy with `force`, or where disk + * has no copy at all. + */ + named?: boolean; + /** Unlink workspace-deleted files whose disk copy is one the store holds. */ + applyDeletions?: boolean; + /** Called before each disk write/unlink, so the caller can suppress the watcher echo. */ + beforeEffect?: (path: string) => void; + /** + * The branch the caller chose `paths` on. If another is selected by the + * time the push decides, it writes nothing: a selection made on one branch + * pushed with another branch's content would be neither branch's materialize. + */ + branchId?: string; +} + +export interface PushResult { + written: string[]; + unchanged: string[]; + deleted: string[]; + /** Refused paths, each with its reason. */ + skipped: Array<{ path: string; reason: string }>; + /** Workspace deletions left on disk because applyDeletions wasn't given. */ + pendingDeletions: string[]; + /** The branch and sequence the push planned on: what disk now reflects. + * Absent when it planned nothing (a read-only mount, or the selected + * branch changed after its paths were chosen): disk then still reflects + * whatever it did before. */ + branchId?: string; + sequence?: number; +} + +/** + * Make disk hold the store's version of each path, under the same rule: + * drafts are written, agreement is left alone, conflicts and unverifiable + * disk copies are refused unless `force`, and a workspace deletion reaches + * disk only with `applyDeletions` (with `force` too when disk changed since). + * A path nothing has evidence about is written only when `named`. + * A path whose earlier push shows on disk unconfirmed is pushed again, so its + * completion rests on barriers that succeeded. + * + * Every write or unlink is announced by a durable intent first, and checked + * at the point of effect (effects.ts): the boundary always, and — unless + * `force` — that disk still holds what was decided on. Then every directory + * whose entries the push created or removed is synced, from each written + * file's parent up to the mount root and each unlinked file's parent, once + * per push and after all of its effects, and only then are completions + * recorded. An effect that failed without touching its path puts the path's + * evidence back exactly as it was before the intent; one that touched it, or + * whose barrier failed, leaves the intent pending. + */ +export async function pushPaths( + store: JsStore, + agreement: DiskAgreement, + mount: MountRuntime, + paths: string[], + opts: PushOptions = {}, +): Promise { + agreement.ensureReconciled(); + if (agreement.needsBarrier) agreement.barrier(); // as a pass does: nothing decided from unsynced evidence + const result: PushResult = { written: [], unchanged: [], deleted: [], skipped: [], pendingDeletions: [] }; + if (mount.readOnly) return result; + // A root that isn't available takes no writes, forced or not: restoring the + // store's files into an empty mountpoint would fill the disk the drive was + // unmounted from. The push plans nothing, so it reports no branch. + const root = await locateRoot(mount.view, mount.rootReal); + if ('reason' in root) { + for (const path of paths) result.skipped.push({ path, reason: root.reason }); + return result; + } + + type Plan = { + path: string; + /** + * The path's evidence just before this push's intent, after what planning + * itself recorded (an agreement, a resolved intent), put back if the + * effect never touches the path. + */ + before: Physical | undefined; + prior: Agreed | null; + /** What the effect may find at the path. */ + expect: Expect; + } & ({ kind: 'write'; blob: Buffer; hash: string } | { kind: 'unlink' }); + const plans: Plan[] = []; + + // Observe disk. + const facts = new Map(); + for (const path of paths) { + const p = agreement.get(mount.name, path); + facts.set(path, await learnDisk(mount, path, { p, hasStore: true, hasIntent: true }, {})); + } + // Each path was observed by path beneath the root's path. Under a root + // replaced meanwhile they describe the replacement, so, as a pass does, + // nothing is decided or recorded from them. + const after = await locateRoot(mount.view, mount.rootReal); + if (!('identity' in after) || !sameRootIdentity(after.identity, root.identity)) { + const reason = 'reason' in after ? after.reason : ROOT_REPLACED; + for (const path of paths) result.skipped.push({ path, reason }); + return result; + } + + // Decide synchronously what to push. + const branchId = store.currentBranch().id; + if (opts.branchId !== undefined && opts.branchId !== branchId) { + for (const path of paths) { + result.skipped.push({ path, reason: 'the selected branch changed after these paths were chosen; nothing was written — materialize again' }); + } + return result; + } + result.branchId = branchId; + result.sequence = store.currentSequence(); + let recorded = false; + for (const [path, d] of facts) { + const entry = store.treeGet(mount.treeStateId, path); + const s = entry ? { hash: entry.blobHash, size: entry.size } : null; + const p = agreement.get(mount.name, path); + const bi = mount.intents.get(path); + const v = decide(d, s, p, bi, branchId, false); + // What the effect may find: what this observation saw, unless forced. + const expect: Expect = opts.force || d.kind === 'unobserved' || d.kind === 'unhashed' + ? { kind: 'any' } + : d.kind === 'absent' ? { kind: 'absent' } : { kind: 'content', hash: d.hash }; + if (v.adopt) { + // Disk changed since it last agreed and the workspace copy didn't. Pushing + // would silently revert the disk change; its evidence stays as it is, so + // a later sync adopts it. + if (!s) continue; // nothing in the workspace to push + if (!opts.force) { + result.skipped.push({ + path, + reason: 'stale copy: disk changed since it last agreed with the workspace, which did not — ' + + 'sync this path to adopt the disk version, or pass force to overwrite it', + }); + continue; + } + const priorP = p && (p.kind === 'absent' || p.kind === 'content') ? p : null; + const blob = store.getBlob(s.hash); + if (blob) plans.push({ path, kind: 'write', blob, hash: s.hash, prior: priorP, before: agreement.get(mount.name, path), expect }); + else result.skipped.push({ path, reason: 'the workspace copy is missing from the store' }); + continue; + } + // Whatever the verdict learned about disk (an agreement, a resolved + // intent) is recorded before anything is pushed over it, and the root it + // was learned under before that (once per mount), as a pass does. + if (v.p !== undefined) agreement.recordRootIdentity(mount.name, root.identity); + if (v.p === 'forget') agreement.forget(mount.name, path); + else if (v.p !== undefined) agreement.set(mount.name, path, v.p, { afterCommittedState: true }); + if (v.p !== undefined) recorded = true; + // The prior a new intent carries is the last confirmed evidence: for a + // path whose earlier push is unconfirmed, that push's own prior, never + // the bytes it wrote. + const known = v.unconfirmed ? (p as Pending).prior ?? undefined : v.p === 'forget' ? undefined : v.p !== undefined ? v.p : p; + const prior = known && (known.kind === 'absent' || known.kind === 'content') ? known : null; + if (v.state === 'synced' && !v.unconfirmed) { + if (v.intent !== undefined) { + mount.intents.put(path, v.intent); + recorded = true; + } + if (s) result.unchanged.push(path); + continue; + } + if (v.state === 'unverified') { + if (!opts.force) { + result.skipped.push({ path, reason: `cannot verify disk copy (${v.note ?? 'not observed'}) — fix it and materialize again, or pass force to overwrite it` }); + continue; + } + } + // A file merely absent from this branch is left alone, an unconfirmed + // push of another branch's bytes included: nothing replays or unlinks it. + if (v.state === 'not-in-branch' || v.state === 'disk-only') continue; + const kind = bi?.conflict?.kind ?? v.conflict ?? 'both-changed'; + // Without evidence there was no agreement for disk to change since, and + // nothing tells a newer disk copy from an older one: only a push of the + // named path overwrites it, forced. + const noEvidence = kind === 'unknown-provenance' || kind === 'store-origin-collision'; + if (v.state === 'conflict' && (!opts.force || (noEvidence && !opts.named))) { + if (v.conflict) { + mount.intents.update(path, (cur) => ({ ...cur, conflict: { kind: v.conflict!, at: Date.now(), disk: counterpartOf(store, d) } })); + recorded = true; + } + result.skipped.push({ + path, + reason: noEvidence + ? `disk holds a different copy, and nothing records which is newer (conflict: ${kind}) — ` + + 'sync this path to adopt the disk version, or materialize this path with force to overwrite it' + : kind === 'interrupted' + ? 'disk holds bytes that match neither what was being written there nor what disk last agreed with ' + + '(conflict: interrupted), as a write that failed or was cut off partway leaves — check the file, then ' + + 'pass force to write the workspace copy over it, or sync this path to adopt the disk copy as it is' + : `stale copy: disk changed since it last agreed with the workspace (conflict: ${kind}) — ` + + 'sync this path to adopt the disk version, or pass force to overwrite it', + }); + continue; + } + // An entry from before the evidence that disk lacks: nothing records + // whether disk deleted it or never had it, and most such entries came + // from disk. Writing it back could undo a deliberate deletion, so only a + // push of the named path writes it. + if (v.state === 'disk-missing-provenance-unknown' && !opts.named) { + result.skipped.push({ + path, + reason: 'disk has no copy, and nothing records whether it was deleted there or never written — ' + + 'sync this path to drop it from the workspace, or materialize this path to write it', + }); + continue; + } + if (!s) { + // Only a deliberate workspace deletion ever unlinks; a file that is + // merely absent from this branch is never deleted from disk. + if (!bi?.tombstone) continue; + if (v.unconfirmed && d.kind === 'absent') { + // Our own unlink already reached disk, unconfirmed: confirm it — its + // directory synced, nothing removed — applyDeletions or not. Only + // ever a confirmation, force or not: a file that appears meanwhile + // refuses it. + plans.push({ path, kind: 'unlink', prior, before: agreement.get(mount.name, path), expect: { kind: 'absent' } }); + continue; + } + if (!opts.applyDeletions) { + result.pendingDeletions.push(path); + continue; + } + if (d.kind === 'absent') continue; + plans.push({ path, kind: 'unlink', prior, before: agreement.get(mount.name, path), expect }); + continue; + } + const blob = store.getBlob(s.hash); + if (!blob) { + result.skipped.push({ path, reason: 'the workspace copy is missing from the store' }); + continue; + } + plans.push({ path, kind: 'write', blob, hash: s.hash, prior, before: agreement.get(mount.name, path), expect }); + } + + // 1. Durable intents before any disk effect, the root they write under first. + if (plans.length > 0) agreement.recordRootIdentity(mount.name, root.identity); + plans.forEach((step, i) => { + agreement.intend(mount.name, step.path, { + effect: 'disk', + prior: step.prior, + expect: step.kind === 'write' ? { kind: 'content', hash: step.hash } : ABSENT, + branchId, + }, { durable: i === plans.length - 1 }); + }); + + // 2. Effects, each bound at its point of use. + const effected: Array<{ step: Plan; dirs: string[] }> = []; + for (const step of plans) { + opts.beforeEffect?.(step.path); + try { + if (step.kind === 'write') { + await writeContained(mount.view, mount.rootReal, step.path, step.blob, step.expect); + effected.push({ step, dirs: chainOf(step.path) }); + } else { + // Its directory is synced whether this unlink removed the entry or + // found it gone: either way the completion claims a durable absence. + effected.push({ step, dirs: [await unlinkContained(mount.view, mount.rootReal, step.path, step.expect)] }); + } + } catch (err) { + const failure = err instanceof EffectFailed ? err : new EffectFailed(err instanceof Error ? err.message : String(err), true); + if (!failure.touched) { + // Disk at this path is as planning saw it: its evidence goes back to + // exactly what it was, whatever kind that was. + agreement.restore(mount.name, step.path, step.before); + recorded = true; + result.skipped.push({ path: step.path, reason: `not ${step.kind === 'write' ? 'written' : 'unlinked'}: ${failure.message}` }); + } else { + // No completion: the intent stays pending, and the next observation + // resolves it from whatever disk then holds. + result.skipped.push({ path: step.path, reason: `${failure.message}; the outcome is checked at the next observation` }); + } + } + } + + // 3. Barriers: every directory whose entries the push changed, synced + // once, after all of its effects. A directory created by an earlier, + // failed attempt is synced too: the whole chain, whoever made it. + const unsynced = await syncDirectories(mount.view, mount.rootReal, effected.flatMap((x) => x.dirs)); + + // 4. Completions. P is physical and global. Branch intent belongs to the + // branch this push planned on: it is settled only while that branch is + // still selected, and otherwise by convergence when the branch returns. + for (const { step, dirs } of effected) { + const failed = dirs.find((dir) => unsynced.has(dir)); + if (failed !== undefined) { + result.skipped.push({ + path: step.path, + reason: `could not make the ${step.kind === 'write' ? 'write' : 'unlink'} durable (directory fsync of ${failed === '' ? 'the mount root' : failed}): ` + + `${unsynced.get(failed)}; the outcome is checked at the next observation`, + }); + continue; + } + const value: Agreed = step.kind === 'write' ? { kind: 'content', hash: step.hash, size: step.blob.byteLength } : { kind: 'absent' }; + agreement.set(mount.name, step.path, value); + recorded = true; + const planned = store.currentBranch().id === branchId; + if (step.kind === 'write') { + if (planned) mount.intents.update(step.path, (cur) => { const next = { ...cur }; delete next.conflict; delete next.origin; return next; }); + result.written.push(step.path); + } else { + if (planned) mount.intents.put(step.path, null); + result.deleted.push(step.path); + } + } + // Durable before returning, like a pass: agreements recorded here have no + // pending intent behind them, and completions shouldn't wait for one. + if (recorded || agreement.needsBarrier) agreement.barrier(); + agreement.maybeCheckpoint(); + return result; +} diff --git a/src/modules/workspace/sync.ts b/src/modules/workspace/sync.ts index b3d14d43d..189491ae3 100644 --- a/src/modules/workspace/sync.ts +++ b/src/modules/workspace/sync.ts @@ -1,445 +1,20 @@ /** - * Sync logic between filesystem and Chronicle tree state. - * - * Two directions: - * - syncFromFs: filesystem → Chronicle (user changes) - * - materializeToFs: Chronicle → filesystem (agent changes) + * Shared helpers for moving content between the filesystem and the Chronicle + * tree. The rules for when content moves — the three-way rule over disk, the + * store and disk-agreement evidence — live in reconcile.ts, and how disk is + * observed (containment, symlinks, walks) in observe.ts. */ import { createHash } from 'node:crypto'; -import { readFile, writeFile, mkdir, readdir, stat, access } from 'node:fs/promises'; -import { join, dirname, relative, resolve, sep } from 'node:path'; -import type { JsStore, JsTreeEntry } from '@animalabs/chronicle'; -import type { MountState } from './types.js'; export const DEFAULT_MAX_FILE_SIZE = 5 * 1024 * 1024; // 5MB /** - * Resolve a relative path within a mount and verify it doesn't escape. - * Returns the absolute path, or null if the path is outside the mount. - */ -function safePath(mountPath: string, relativePath: string): string | null { - // Compare with the PLATFORM separator: resolve() emits backslashes on - // Windows, so a '/'-suffixed root never prefix-matches there and every - // in-mount path was reported as outside the mount. - const resolved = resolve(mountPath, relativePath); - const root = mountPath.endsWith(sep) ? mountPath : mountPath + sep; - if (resolved !== mountPath && !resolved.startsWith(root)) { - return null; - } - return resolved; -} - -/** - * Hash file content to a full SHA-256 hex string. - * Must match Chronicle's storeBlob() hash format (64-char hex). + * Hash content to a full SHA-256 hex string. + * Matches Chronicle's storeBlob() hash format (64-char hex). */ export function hashContent(content: string | Buffer): string { const hash = createHash('sha256'); hash.update(content); return hash.digest('hex'); } - -/** - * Check if content appears to be binary. - */ -export interface SkippedFile { - /** Relative path of the file that was not synced */ - path: string; - /** Why, in words the resident can act on */ - reason: string; -} - -export function isBinary(buffer: Buffer): boolean { - // Check for null bytes in first 8KB - const check = buffer.subarray(0, 8192); - for (let i = 0; i < check.length; i++) { - if (check[i] === 0) return true; - } - return false; -} - -export interface ConflictInfo { - /** Relative path of the conflicted file */ - path: string; - /** Blob hash of the agent's version (retrievable via store.getBlob()) */ - agentBlobHash: string; -} - -export type SyncOp = 'created' | 'modified' | 'deleted'; - -export interface SyncedPath { - path: string; - op: SyncOp; -} - -export interface SyncResult { - /** Paths that were synced, tagged with the op detected against tree state */ - synced: SyncedPath[]; - /** Paths that conflicted (both agent and user modified) — filesystem wins */ - conflicts: ConflictInfo[]; - /** Paths that were skipped, each with the reason — a silently skipped file - * is indistinguishable from one that synced cleanly, which is how a - * resident ends up believing a file is in their workspace when it never - * arrived. */ - skipped: SkippedFile[]; -} - -/** - * Sync filesystem changes into Chronicle tree state. - * - * @param store Chronicle store - * @param mount Mount state - * @param paths Specific paths to sync (relative to mount). If empty, walks the directory. - * @returns Sync result with synced/conflicted/skipped paths - */ -export async function syncFromFs( - store: JsStore, - mount: MountState, - paths?: string[], -): Promise { - const result: SyncResult = { synced: [], conflicts: [], skipped: [] }; - const maxSize = mount.config.maxFileSize ?? DEFAULT_MAX_FILE_SIZE; - - const filesToSync = paths ?? await walkDirectory(mount.config.path, mount.config.ignore ?? []); - - for (const relativePath of filesToSync) { - const absolutePath = safePath(mount.config.path, relativePath); - if (!absolutePath) { - result.skipped.push({ path: relativePath, reason: 'path resolves outside the mount' }); - continue; - } - - try { - await access(absolutePath); - } catch { - // File was deleted on disk — remove from tree - const existing = store.treeGet(mount.treeStateId, relativePath); - if (existing) { - store.treeRemove(mount.treeStateId, relativePath); - result.synced.push({ path: relativePath, op: 'deleted' }); - } - mount.materializedHashes.delete(relativePath); - continue; - } - - try { - const fileStat = await stat(absolutePath); - if (!fileStat.isFile()) continue; - if (fileStat.size > maxSize) { - result.skipped.push({ - path: relativePath, - reason: `too large: ${fileStat.size} bytes > maxFileSize ${maxSize}`, - }); - continue; - } - - const buffer = await readFile(absolutePath); - if (isBinary(buffer)) { - // Disk -> store sync is text-only, at ANY size: the tree stores text - // blobs. Binary that must live in the store has to be written through - // writeBinary; binary that is only passing through should be read with - // readBinaryFromDisk instead of synced at all. - result.skipped.push({ - path: relativePath, - reason: 'binary file — disk sync stores text only (read it from disk, or write it via a binary-aware tool)', - }); - continue; - } - - const content = buffer.toString('utf-8'); - const hash = hashContent(content); - - // Check current tree state - const existing = store.treeGet(mount.treeStateId, relativePath); - - if (existing && existing.blobHash === hash) { - // No change. Disk and tree agree, so this is the baseline now. - mount.materializedHashes.set(relativePath, hashContent(buffer)); - continue; - } - - // Store blob and update tree - const blobHash = store.storeBlob(Buffer.from(content, 'utf-8'), 'text/plain'); - const entry: JsTreeEntry = { - blobHash, - size: buffer.length, - mode: 0o644, - }; - - // Conflict detection: if the file existed in the tree AND the agent has - // modified it since last materialization, this is a genuine conflict - // (both agent and user changed the same file). - if (existing) { - const baselineHash = mount.materializedHashes.get(relativePath); - if (baselineHash && existing.blobHash !== baselineHash) { - // Agent changed the tree entry since we last materialized — conflict. - // Filesystem still wins, but agent's version is recoverable via agentBlobHash. - result.conflicts.push({ - path: relativePath, - agentBlobHash: existing.blobHash, - }); - } - } - - store.treeSet(mount.treeStateId, relativePath, entry); - // The tree adopted the disk copy, so disk and tree agree: re-pin the - // baseline. Left at the old hash, materialize would read the adopted - // edit as a fresh external change and refuse every later agent edit - // to this file, with no sync able to clear it. Raw disk bytes, which is - // what materializeToFs compares against. - mount.materializedHashes.set(relativePath, hashContent(buffer)); - result.synced.push({ path: relativePath, op: existing ? 'modified' : 'created' }); - } catch (error) { - result.skipped.push({ - path: relativePath, - reason: error instanceof Error ? error.message : String(error), - }); - } - } - - return result; -} - -export interface MaterializeResult { - /** Paths actually written to disk */ - written: string[]; - /** Paths whose disk copy already held exactly the tree's bytes: nothing - * written, but disk reflects the current tree for them all the same. */ - unchanged: string[]; - /** Paths refused by the freshness guard, each with the reason — a silently - * overwritten shell edit is unrecoverable, so divergence must surface - * instead of being resolved toward the workspace copy (issue #109: a bulk - * materialize reverted 217 shell-edited files to a stale snapshot). */ - skipped: SkippedFile[]; -} - -/** - * Materialize Chronicle tree state to filesystem. - * - * Freshness guard (#109): files get modified by two hands — this layer AND - * direct shell/tool edits. Before overwriting, the disk content's hash is - * compared against `materializedHashes` (the hash this layer last wrote): a - * mismatch means another writer changed the file since, and the path is - * refused loudly instead of silently reverted, unless `force`. Symmetric - * with syncFromFs's conflict detection, and with the same blind spot: no - * baseline (never materialized, or synced) means no guard. Baselines and - * refused paths persist across restarts with the module state. - * - * Refused paths are kept in `mount.refusedPaths` and retried by every later - * materialize, so the diff watermark can still advance: holding it back - * would understate what disk holds to the branch guard, which reads the - * same sequence. - * - * @param store Chronicle store - * @param mount Mount state - * @param paths Specific paths to materialize. If undefined, materializes all changed since last. - * @param opts force: overwrite even where the disk copy changed under us. - * @returns Paths written, and paths the freshness guard refused - */ -export async function materializeToFs( - store: JsStore, - mount: MountState, - paths?: string[], - opts?: { force?: boolean }, -): Promise { - if (mount.config.mode === 'read-only') { - return { written: [], unchanged: [], skipped: [] }; - } - - const written: string[] = []; - const unchanged: string[] = []; - const skipped: SkippedFile[] = []; - - // Get changed files since last materialization - const currentSeq = store.currentSequence(); - let filesToMaterialize: Array<{ path: string; blobHash: string }>; - - if (paths) { - // Materialize specific paths - filesToMaterialize = []; - for (const p of paths) { - const entry = store.treeGet(mount.treeStateId, p); - if (entry) { - filesToMaterialize.push({ path: p, blobHash: entry.blobHash }); - } - } - } else if (mount.lastMaterializedSeq > 0) { - // Diff since last materialization - const changes = store.treeDiff( - mount.treeStateId, - mount.lastMaterializedSeq, - currentSeq, - ); - filesToMaterialize = changes - .filter(c => c.changeType === 'added' || c.changeType === 'modified') - .map(c => ({ - path: c.path, - blobHash: c.newEntry!.blobHash, - })); - - // Handle removals - for (const change of changes) { - if (change.changeType === 'removed') { - // Don't delete from filesystem — just skip. - // Agent removing from tree doesn't mean delete user's file. - } - } - } else { - // First materialization — materialize everything - const entries = store.treeList(mount.treeStateId); - filesToMaterialize = entries.map(e => ({ - path: e.path, - blobHash: e.blobHash, - })); - } - - // Paths an earlier materialize refused are still owed. A path the tree no - // longer has is owed nothing. - if (!paths) { - const queued = new Set(filesToMaterialize.map((f) => f.path)); - for (const p of mount.refusedPaths) { - if (queued.has(p)) continue; - const entry = store.treeGet(mount.treeStateId, p); - if (entry) filesToMaterialize.push({ path: p, blobHash: entry.blobHash }); - else mount.refusedPaths.delete(p); - } - } - - for (const { path: relativePath, blobHash } of filesToMaterialize) { - const absolutePath = safePath(mount.config.path, relativePath); - if (!absolutePath) continue; // Path traversal — skip silently - - const blob = store.getBlob(blobHash); - if (!blob) continue; - - let diskBuffer: Buffer | null = null; - try { - diskBuffer = await readFile(absolutePath); - } catch (err) { - // Only absence proves there is nothing to overwrite. Any other read - // failure (EACCES on a write-only file, EISDIR, EIO...) means freshness - // cannot be verified, and writeFile may still succeed — so refuse - // rather than treat "unreadable" as "not there". - const code = (err as NodeJS.ErrnoException).code; - if (code !== 'ENOENT' && !opts?.force) { - skipped.push({ - path: relativePath, - reason: - `cannot verify disk copy (${code ?? (err as Error).message}) — ` + - 'fix its permissions and materialize again, or pass force to overwrite it', - }); - mount.refusedPaths.add(relativePath); - continue; - } - } - if (diskBuffer) { - const diskHash = hashContent(diskBuffer); - if (diskHash === blobHash) { - // Disk already holds exactly these bytes: re-pin the baseline and - // leave the file (and its mtime) alone. - mount.materializedHashes.set(relativePath, blobHash); - mount.refusedPaths.delete(relativePath); - unchanged.push(relativePath); - continue; - } - const baselineHash = mount.materializedHashes.get(relativePath); - if (!opts?.force && baselineHash !== undefined && diskHash !== baselineHash) { - skipped.push({ - path: relativePath, - reason: - 'stale copy: disk changed since last materialize (another writer) — ' + - 'sync first to adopt the disk version, or pass force to overwrite it', - }); - mount.refusedPaths.add(relativePath); - continue; - } - } - - // Create parent directories - await mkdir(dirname(absolutePath), { recursive: true }); - - // Write file - await writeFile(absolutePath, blob); - written.push(relativePath); - - // Record blob hash at materialization time for conflict detection - mount.materializedHashes.set(relativePath, blobHash); - mount.refusedPaths.delete(relativePath); - } - - mount.lastMaterializedSeq = currentSeq; - - return { written, unchanged, skipped }; -} - -/** - * Walk a directory recursively, respecting ignore patterns. - */ -async function walkDirectory( - basePath: string, - ignorePatterns: string[], -): Promise { - const results: string[] = []; - const maxFiles = 5000; // Safety limit - - async function walk(dir: string) { - if (results.length >= maxFiles) return; - - let entries; - try { - entries = await readdir(dir, { withFileTypes: true }); - } catch { - return; - } - - for (const entry of entries) { - if (results.length >= maxFiles) break; - - const fullPath = join(dir, entry.name); - // Logical workspace paths are '/'-separated everywhere (mount-prefixed - // paths, chronicle tree paths); node's relative() emits backslashes on - // Windows, so normalize. No-op on POSIX (sep === '/'). - const relativePath = relative(basePath, fullPath).split(sep).join('/'); - - // Check ignore patterns (simple glob matching) - if (shouldIgnore(relativePath, entry.name, ignorePatterns)) continue; - - if (entry.isFile()) { - results.push(relativePath); - } else if (entry.isDirectory()) { - await walk(fullPath); - } - } - } - - await walk(basePath); - return results; -} - -/** - * Simple ignore pattern matching. - */ -function shouldIgnore( - relativePath: string, - name: string, - patterns: string[], -): boolean { - for (const pattern of patterns) { - // Exact name match (e.g., ".git", "node_modules") - if (pattern === name) return true; - - // Simple ** glob: "node_modules/**" matches anything under node_modules - if (pattern.endsWith('/**')) { - const prefix = pattern.slice(0, -3); - if (relativePath.startsWith(prefix + '/') || relativePath === prefix) return true; - } - - // Extension glob: "*.pyc" matches any .pyc file - if (pattern.startsWith('*.')) { - const ext = pattern.slice(1); - if (name.endsWith(ext)) return true; - } - } - return false; -} diff --git a/src/modules/workspace/types.ts b/src/modules/workspace/types.ts index 0ea110d3b..3517a8b96 100644 --- a/src/modules/workspace/types.ts +++ b/src/modules/workspace/types.ts @@ -18,9 +18,13 @@ export interface MountConfig { mode: 'read-write' | 'read-only'; /** * Watch mode for filesystem changes: - * - 'always': chokidar watches continuously, syncs on debounce - * - 'on-agent-action': sync from filesystem after each agent tool call - * - 'never': fully virtual, no automatic filesystem reads + * - 'always': chokidar watches continuously; each event re-observes its paths + * - 'on-agent-action': after each completed agent tool batch, the mount is + * scanned, and the agent's next inference waits for that scan up to + * `agentActionScanDeadlineMs`; a scan that takes longer finishes while + * the round goes on + * - 'never': no background reads; listings and lazy reads still observe + * the paths they show */ watch?: 'always' | 'on-agent-action' | 'never'; /** Debounce window in ms for watch: 'always' mode (default: 300) */ @@ -72,6 +76,19 @@ export interface WorkspaceConfig { deltaSnapshotEvery?: number; /** Full snapshot frequency for tree states (default: 10) */ fullSnapshotEvery?: number; + /** + * How long the scan of `watch: 'on-agent-action'` mounts after a tool batch + * may hold the agent's next inference, in ms (default: 20000). A scan that + * takes longer finishes in the background; the miss is recorded on the + * mount's status and pushed as a `workspace:agent-action-scan-incomplete` + * event. So is a scan that finished without observing everything, with the + * regions it couldn't (`incomplete`). While a scan that outlasted its + * deadline still runs, later batches aren't held: their scan is queued + * after it, and the miss is recorded at once. The framework bounds every + * module's batch hook at 30 s, so a deadline above 29 s is lowered to 29 s, + * with a warning, for the workspace's own miss to come first. + */ + agentActionScanDeadlineMs?: number; } // ============================================================================ @@ -79,26 +96,52 @@ export interface WorkspaceConfig { // ============================================================================ /** - * Per-mount runtime state (not persisted — rebuilt on start). + * Per-mount runtime state (not persisted — rebuilt on start). What disk last + * agreed with lives in the module's disk-agreement journal (global chronicle + * records), and branch-local intent in `intentTreeStateId`. */ export interface MountState { /** The mount config */ config: MountConfig; /** Tree state ID in Chronicle */ treeStateId: string; + /** Branch-local intent (tombstones, store origin, conflicts) for this mount's paths */ + intentTreeStateId: string; /** Sequence number of last materialization */ lastMaterializedSeq: number; /** Paths currently suppressed from watcher (recently materialized) */ suppressedPaths: Set; - /** Whether initial lazy sync has been completed */ + /** Whether a full scan of the mount has completed this session */ initialSyncDone: boolean; /** Branch ID that was active when this mount last materialized */ lastMaterializedBranchId: string | null; - /** Per-file hash at time of last materialization — baseline for conflict detection */ - materializedHashes: Map; - /** Paths the materialize freshness guard refused (#109), owed to every - * later materialize until one writes them or the tree drops them. */ - refusedPaths: Set; + /** + * The last on-agent-action scan, as of `at`: `complete` when it has + * finished at all, `withinDeadline` when it finished before the deadline + * released the round, `reason` for a miss or a failure, and `incomplete` + * for the regions a finished scan couldn't observe (each also pushed as a + * `workspace:agent-action-scan-incomplete` event). + */ + lastAgentActionScan?: { + at: number; + complete: boolean; + withinDeadline: boolean; + reason?: string; + incomplete?: Array<{ path: string; reason: string }>; + }; + /** + * The after-batch scan queued behind the mount's running work, not yet + * started: later batches share it, since it starts after their tools ran. + */ + agentActionScanWaiting?: AgentActionScan; + /** + * The mount's newest after-batch scan. Only its outcome is put on + * `lastAgentActionScan`: one queued behind an older scan describes a later + * batch, which the older scan's outcome mustn't speak for. + */ + agentActionScanLatest?: AgentActionScan; + /** A scan that outlasted a batch's deadline, while it still runs. */ + agentActionScanLate?: Promise; /** * Wall-clock time chokidar emitted `ready` for this mount, or null if the * watcher hasn't finished its initial scan. null after session start = @@ -118,9 +161,6 @@ export interface WorkspaceModuleState { mounts: Record; - refusedPaths?: string[]; watcherReadyAt?: number | null; watcherError?: string | null; }>; @@ -201,6 +241,14 @@ export interface GrepInput { contextAfter?: number; } +/** One after-batch scan of a mount, shared by the batches that wait for it. */ +export interface AgentActionScan { + /** Whether some batch stopped waiting for it (its deadline passed, or an earlier scan still ran late). */ + late: boolean; + /** Settles, never rejects, once its outcome is recorded: on status if it is the newest, as events either way. */ + settled: Promise; +} + export interface StatusInput { /** Specific mount to check (optional — defaults to all) */ mount?: string; @@ -213,11 +261,18 @@ export interface MaterializeInput { mount?: string; /** * Materialize even when the current branch has genuinely diverged from the - * branch last materialized to disk (default: false). Not needed for linear - * continuations (child branches forked at or after the last materialized - * point) — those pass the guard automatically. + * branch last materialized to disk, and overwrite a disk copy in conflict + * (default: false). Not needed for linear continuations (child branches + * forked at or after the last materialized point) — those pass the guard + * automatically. */ force?: boolean; + /** + * Also delete from disk the files deleted in the workspace whose disk copy + * is still one the workspace holds (default: false). With `force`, also + * those whose disk copy changed since. + */ + applyDeletions?: boolean; } export interface SyncInput { @@ -225,6 +280,8 @@ export interface SyncInput { path?: string; /** Specific mount (optional — defaults to all) */ mount?: string; + /** Take each mount's root as it is now, when it isn't the directory disk last agreed with. */ + acceptRoot?: boolean; } // ============================================================================ @@ -258,6 +315,8 @@ export interface WorkspaceDeletedEvent { type: 'workspace:deleted'; paths: string[]; mount: string; + /** Paths deleted on disk whose newer workspace version was kept as a conflict. */ + conflicts?: string[]; [key: string]: unknown; } diff --git a/src/record-journal.ts b/src/record-journal.ts new file mode 100644 index 000000000..179c4f4b6 --- /dev/null +++ b/src/record-journal.ts @@ -0,0 +1,246 @@ +/** + * An append-only journal of typed Chronicle records that does not follow + * branch switches. + * + * Chronicle's branch state — snapshot, tree and log states — rewinds with a + * historical rollback (createBranchAt + switchBranch). Typed records do not: + * `getRecordIdsByType` enumerates them from every branch, in global order, and + * they survive `deleteBranch`, reopen and a hard kill. A journal is for state + * that must not rewind with the conversation but should live and die with its + * store: what a workspace's disk last agreed with, what reached a recipient, + * what became of a suppressed draft. + * + * Durability. A record append reaches the OS before `append` returns, so it + * survives a killed process, while branch-state commits are buffered until + * `store.sync()`: a record appended after a state commit can outlive that + * commit. Two barriers, each named where it's needed and neither implied by + * the `{id, sequence}` an append returns: + * - `afterCommittedState: true` syncs BEFORE appending — for an entry that + * asserts state committed just before it, so the entry can never survive + * without that state; + * - `durable: true` syncs AFTER appending — for an entry that must be on + * stable storage before the caller acts on it (an attempt before its + * outbound request, an intent before its disk effect). + * + * Checkpoints bound the replay at open: `checkpoint(snapshot)` records the + * caller's reduced state together with the last entry it covers, and `load()` + * returns the latest checkpoint's snapshot plus only the entries appended + * after it. A journal that never checkpoints replays everything. What entries + * mean, how they reduce, and when to checkpoint stay with the caller. + * + * One writer per record type: a checkpoint covers every entry up to the last + * one THIS journal loaded or appended, so entries another instance appended + * to the same type in between would be covered without being reduced. + * + * An ambiguous write must be resolved before the journal writes again. Once + * `append` or `checkpoint` has handed a record to the store, a failure leaves + * a record the caller didn't reduce, or may leave one, and a later checkpoint + * would cover it without its ever entering the reducer. There are two kinds: + * - The store's append itself throws. Its record may still be in the store's + * log, outside this process's view of the store: the store writes a + * record to its log, syncs the log every so often, and only then indexes + * the record, which is what `getRecordIdsByType` reads. So a failed sync + * leaves a record that reopening the store will find and no `load()` on + * this store can. Nothing but a reopen rereads the log, so from then on + * every `append` and `checkpoint` throws, for the life of the journal. A + * journal on the store reopened, as at the next start, loads that record + * with its tail for the caller to reduce. + * - A failure after the append returned: a `durable` barrier, or reading + * the id it returned. The record is in this process's view, so every + * further `append` and `checkpoint` throws until `load()` has replayed the + * tail, that record included, for the caller to reduce. + * A failure BEFORE the record reaches the store (the `afterCommittedState` + * sync) wrote nothing and leaves the journal usable. + */ + +import type { JsStore } from '@animalabs/chronicle'; + +/** Where an appended record landed. Not a durability receipt — see the barriers. */ +export interface JournalRecordRef { + /** Chronicle record id: global, increasing in append order. */ + id: string; + /** The record's sequence on the branch current when it was appended. */ + sequence: number; +} + +export interface JournalEntry extends JournalRecordRef { + entry: Entry; +} + +export interface JournalLoad { + /** The latest checkpoint's snapshot, or null when none was written. */ + snapshot: Snapshot | null; + /** Every entry appended after that checkpoint, in append order. */ + entries: Array>; +} + +export interface AppendOptions { + /** `store.sync()` before appending: the entry asserts state committed just before it. */ + afterCommittedState?: boolean; + /** `store.sync()` after appending: the entry must be on stable storage before the caller acts. */ + durable?: boolean; +} + +interface CheckpointPayload { + /** Id of the last entry the snapshot covers ("0" when it covers none). */ + through: string; + snapshot: Snapshot; +} + +const asError = (err: unknown): Error => (err instanceof Error ? err : new Error(String(err))); + +/** Record ids are decimal strings from one store-wide counter. */ +function idValue(id: string): bigint { + if (!/^\d+$/.test(id)) throw new Error(`RecordJournal: unexpected chronicle record id "${id}"`); + return BigInt(id); +} + +function decode(store: JsStore, id: string): { value: T; sequence: number } { + const record = store.getRecord(id); + if (!record) throw new Error(`RecordJournal: record ${id} is listed but cannot be read`); + return { value: JSON.parse(Buffer.from(record.payload).toString('utf-8')) as T, sequence: record.sequence }; +} + +export class RecordJournal { + private readonly store: JsStore; + /** Record type of the entries. */ + readonly type: string; + /** Record type of the checkpoints. */ + readonly checkpointType: string; + /** Highest entry id this journal has loaded or appended (0 when none). */ + private lastEntryId = 0n; + private sinceCheckpoint = 0; + /** Set by a failure after the store's append returned; cleared by load(). */ + private unreconciled: Error | null = null; + /** Set when the store's append itself threw: its record may be in the log + * but not in this process's view of the store. Never cleared. */ + private unreadable: Error | null = null; + + constructor(store: JsStore, opts: { type: string; checkpointType?: string }) { + if (!opts.type) throw new Error('RecordJournal: a record type is required'); + this.store = store; + this.type = opts.type; + this.checkpointType = opts.checkpointType ?? `${opts.type}/checkpoint`; + if (this.checkpointType === this.type) { + throw new Error('RecordJournal: entries and checkpoints need distinct record types'); + } + } + + /** Entries appended since the last checkpoint (or since load, when there was none). */ + get entriesSinceCheckpoint(): number { + return this.sinceCheckpoint; + } + + /** + * The latest checkpoint and every entry after it, whichever branch each was + * appended on. The caller rebuilds its reduced state from exactly these — + * at open, and to reconcile after an ambiguous write. Also positions the + * journal, so later checkpoints cover what was loaded. + */ + load(): JournalLoad { + let snapshot: Snapshot | null = null; + let through = 0n; + let latestCheckpoint: bigint | null = null; + let latestCheckpointId: string | null = null; + for (const id of this.store.getRecordIdsByType(this.checkpointType)) { + const value = idValue(id); + if (latestCheckpoint === null || value > latestCheckpoint) { + latestCheckpoint = value; + latestCheckpointId = id; + } + } + if (latestCheckpointId !== null) { + const payload = decode>(this.store, latestCheckpointId).value; + snapshot = payload.snapshot; + through = idValue(payload.through); + } + + const tail = this.store.getRecordIdsByType(this.type) + .map((id) => ({ id, value: idValue(id) })) + .filter(({ value }) => value > through) + .sort((a, b) => (a.value < b.value ? -1 : a.value > b.value ? 1 : 0)); + const entries = tail.map(({ id }) => { + const { value, sequence } = decode(this.store, id); + return { id, sequence, entry: value }; + }); + + this.lastEntryId = tail.length > 0 ? tail[tail.length - 1]!.value : through; + this.sinceCheckpoint = entries.length; + this.unreconciled = null; + return { snapshot, entries }; + } + + /** True after a failure that load() can resolve, until it has replayed the + * tail. An append that threw inside the store is never resolved here: the + * journal stays unwritable until a journal on the reopened store loads it. */ + get needsReconcile(): boolean { + return this.unreconciled !== null; + } + + /** True once an append threw inside the store, for the rest of the + * journal's life. What `load()` returns may then lack that append's record, + * so a caller that decides from the journal's contents can't trust them + * until a journal on the reopened store has loaded it. */ + get needsReopen(): boolean { + return this.unreadable !== null; + } + + /** Append one entry. The returned ref says where it landed, not that it's durable. */ + append(entry: Entry, opts: AppendOptions = {}): JournalRecordRef { + this.assertReconciled(); + if (opts.afterCommittedState) this.store.sync(); + return this.write(this.type, entry, opts.durable === true, (ref) => { + const value = idValue(ref.id); + if (value > this.lastEntryId) this.lastEntryId = value; + this.sinceCheckpoint++; + }); + } + + /** + * Record the caller's reduced state as covering every entry this journal has + * loaded or appended. Like an entry, it's durable only with `durable`. + */ + checkpoint(snapshot: Snapshot, opts: { durable?: boolean } = {}): JournalRecordRef { + this.assertReconciled(); + const payload: CheckpointPayload = { through: this.lastEntryId.toString(), snapshot }; + return this.write(this.checkpointType, payload, opts.durable === true, () => { + this.sinceCheckpoint = 0; + }); + } + + /** Hand one record to the store; any failure from here on is ambiguous. */ + private write(type: string, payload: unknown, durable: boolean, onWritten: (ref: JournalRecordRef) => void): JournalRecordRef { + let record: { id: string; sequence: number }; + try { + record = this.store.appendJson(type, payload); + } catch (err) { + this.unreadable = asError(err); + throw err; + } + try { + const ref = { id: record.id, sequence: record.sequence }; + onWritten(ref); + if (durable) this.store.sync(); + return ref; + } catch (err) { + this.unreconciled = asError(err); + throw err; + } + } + + private assertReconciled(): void { + if (this.unreadable) { + throw new Error( + `RecordJournal(${this.type}): an earlier append failed inside the store ` + + `(${this.unreadable.message}), and its record may be in the store's log where this process can't ` + + `read it back; nothing more is written until the store is reopened, as at the next start`, + ); + } + if (this.unreconciled) { + throw new Error( + `RecordJournal(${this.type}): an earlier write failed after reaching the store ` + + `(${this.unreconciled.message}); load() and reduce its tail before writing again`, + ); + } + } +} diff --git a/src/types/module.ts b/src/types/module.ts index 724abaae0..0fbec386a 100644 --- a/src/types/module.ts +++ b/src/types/module.ts @@ -109,6 +109,21 @@ export interface Module { */ gatherContext?(agentName: string): Promise; + /** + * Called once an agent's tool batch has completed — when the round's last + * pending result arrives, before it is provided to the agent and before + * anything that continues the turn (re-inference, stream resume). The + * framework awaits it, bounded at 30 s and fail-open, so a module can + * bring its state up to date with what the batch's tools did (the + * workspace's `watch: 'on-agent-action'` mounts scan disk here) before the + * agent's next inference. It's awaited in the framework's event loop, so + * a slow hook holds every agent's event processing, not only this one's. + * A hook still running from an earlier batch is skipped, not called again. + * Not called for script-inner tool calls, nor when the framework fills in + * error results for a batch whose stream ended before its tools answered. + */ + onToolBatchComplete?(agentName: string): Promise; + /** * Declare extra settings for the synthesized `agent_settings` tool. * Modules owning hot-tunable runtime state (e.g. a host settings module diff --git a/src/types/trace.ts b/src/types/trace.ts index de8f41ea8..67b28a02b 100644 --- a/src/types/trace.ts +++ b/src/types/trace.ts @@ -171,6 +171,16 @@ export type TraceEvent = agentStatus: string; result: unknown; }) + | (TraceEventBase & { + /** + * A module's onToolBatchComplete threw or outlasted its bound; the + * round went on without it. + */ + type: 'module:batch_hook_failed'; + agentName: string; + module: string; + error: string; + }) | (TraceEventBase & { /** * A writable workspace existed but the oversized-result spill write diff --git a/test/fixtures/record-journal-kill.mjs b/test/fixtures/record-journal-kill.mjs new file mode 100644 index 000000000..e0c04d802 --- /dev/null +++ b/test/fixtures/record-journal-kill.mjs @@ -0,0 +1,18 @@ +// Child process for record-journal.test.ts: commit branch state, append a +// journal entry that asserts it, then die without sync or close. +// node record-journal-kill.mjs +// mode "asserting": append with afterCommittedState (the barrier under test). +import { JsStore } from '@animalabs/chronicle'; +import { RecordJournal } from '../../src/record-journal.js'; + +const [storePath, mode] = process.argv.slice(2); +const store = JsStore.openOrCreate({ path: storePath }); +try { + store.registerState({ id: 'journal-test/tree', strategy: 'tree' }); +} catch { + // registered by the parent's setup +} +const journal = new RecordJournal(store, { type: 'journal-test/entry' }); +store.treeSet('journal-test/tree', 'committed.txt', { blobHash: 'c'.repeat(64), size: 1, mode: 0o644 }); +journal.append({ asserts: 'committed.txt' }, { afterCommittedState: mode === 'asserting' }); +process.kill(process.pid, 'SIGKILL'); diff --git a/test/fixtures/workspace-fault.mjs b/test/fixtures/workspace-fault.mjs new file mode 100644 index 000000000..14eaee04b --- /dev/null +++ b/test/fixtures/workspace-fault.mjs @@ -0,0 +1,48 @@ +// Child process for workspace-three-way.test.ts: run one workspace operation +// and die by SIGKILL at a chosen point of its write-ahead order. +// node workspace-fault.mjs +// op: +// "materialize": materialize the mount (a store→disk write of a draft) +// "sync": a full sync (a disk→store adoption of a shell edit) +// "write": an autoMaterialize write of work/new.txt +// killAt: +// "after-intent": the intent is durable; the effect never happens +// "after-effect": the effect is durable; its completion is never appended +import { JsStore } from '@animalabs/chronicle'; +import { WorkspaceModule } from '../../src/modules/workspace/index.js'; +import { DiskAgreement } from '../../src/modules/workspace/disk-agreement.js'; + +const [storePath, mountDir, op, killAt] = process.argv.slice(2); +const store = JsStore.openOrCreate({ path: storePath }); +const module = new WorkspaceModule({ + mounts: [{ name: 'work', path: mountDir, mode: 'read-write', watch: 'never', autoMaterialize: op === 'write' }], +}); +module.initStore(store); +await module.start({ isRestart: false, getState: () => null, setState: () => {}, pushEvent: () => {} }); + +const die = () => process.kill(process.pid, 'SIGKILL'); +const intend = DiskAgreement.prototype.intend; +const set = DiskAgreement.prototype.set; +let intended = false; +DiskAgreement.prototype.intend = function (mount, path, intent, opts) { + intended = true; + intend.call(this, mount, path, intent, opts); + if (killAt === 'after-intent' && opts?.durable) die(); +}; +DiskAgreement.prototype.set = function (...args) { + if (killAt === 'after-effect' && intended) { + // Whatever the effect was — a fsynced disk write, or a tree commit — it is + // durable now; the completion that would follow is never appended. + store.sync(); + die(); + } + return set.apply(this, args); +}; + +const call = (name, input) => module.handleToolCall({ id: 'fault', name, input }); +if (op === 'materialize') await call('materialize', {}); +else if (op === 'sync') await call('sync', {}); +else if (op === 'write') await call('write', { path: 'work/new.txt', content: 'new file' }); +// Reaching here means the kill point never came: the parent sees exit 0. +store.close(); +process.exit(0); diff --git a/test/record-journal.test.ts b/test/record-journal.test.ts new file mode 100644 index 000000000..977751457 --- /dev/null +++ b/test/record-journal.test.ts @@ -0,0 +1,281 @@ +/** + * RecordJournal — typed chronicle records that don't follow branch switches, + * replayed from the latest checkpoint, with the two durability barriers named. + */ + +import { describe, it } from 'node:test'; +import assert from 'node:assert'; +import { spawnSync } from 'node:child_process'; +import { mkdtempSync, rmSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { JsStore } from '@animalabs/chronicle'; +import { RecordJournal } from '../src/record-journal.js'; + +const KILL_FIXTURE = join(import.meta.dirname, 'fixtures/record-journal-kill.mjs'); + +function withStore(fn: (store: JsStore, path: string) => T): T { + const dir = mkdtempSync(join(tmpdir(), 'record-journal-')); + const path = join(dir, 'store'); + const store = JsStore.openOrCreate({ path }); + try { + return fn(store, path); + } finally { + if (!store.isClosed()) store.close(); + rmSync(dir, { recursive: true, force: true }); + } +} + +type Entry = { n: string }; +const entriesOf = (journal: RecordJournal): string[] => journal.load().entries.map((e) => e.entry.n); + +describe('RecordJournal', () => { + it('an empty journal loads nothing', () => { + withStore((store) => { + const journal = new RecordJournal(store, { type: 'journal-test/entry' }); + assert.deepStrictEqual(journal.load(), { snapshot: null, entries: [] }); + assert.strictEqual(journal.entriesSinceCheckpoint, 0); + }); + }); + + it('append returns where the record landed; a fresh load replays entries in append order', () => { + withStore((store) => { + const journal = new RecordJournal(store, { type: 'journal-test/entry' }); + const a = journal.append({ n: 'a' }); + const b = journal.append({ n: 'b' }); + assert.match(a.id, /^\d+$/); + assert.ok(BigInt(b.id) > BigInt(a.id), 'ids increase in append order'); + assert.strictEqual(typeof a.sequence, 'number'); + const loaded = new RecordJournal(store, { type: 'journal-test/entry' }).load(); + assert.deepStrictEqual(loaded.entries, [ + { id: a.id, sequence: a.sequence, entry: { n: 'a' } }, + { id: b.id, sequence: b.sequence, entry: { n: 'b' } }, + ]); + }); + }); + + it('entries from every branch load from any branch, in global order, through deleteBranch and reopen', () => { + withStore((store, path) => { + const journal = new RecordJournal(store, { type: 'journal-test/entry' }); + journal.append({ n: 'main-0' }); + const fork = store.currentSequence(); + journal.append({ n: 'main-1' }); + const main = store.currentBranch().name; + store.createBranchAt('rollback', main, fork); + store.switchBranch('rollback'); + journal.append({ n: 'rollback-2' }); + assert.deepStrictEqual(entriesOf(journal), ['main-0', 'main-1', 'rollback-2'], + 'the rolled-back branch still sees what main appended after the fork'); + store.switchBranch(main); + journal.append({ n: 'main-3' }); + store.deleteBranch('rollback'); + assert.deepStrictEqual(entriesOf(journal), ['main-0', 'main-1', 'rollback-2', 'main-3']); + store.close(); + const reopened = JsStore.openOrCreate({ path }); + try { + assert.deepStrictEqual(entriesOf(new RecordJournal(reopened, { type: 'journal-test/entry' })), + ['main-0', 'main-1', 'rollback-2', 'main-3']); + } finally { + reopened.close(); + } + }); + }); + + it('load returns the latest checkpoint and only the entries after it; other records do not disturb it', () => { + withStore((store) => { + store.registerState({ id: 'journal-test/tree', strategy: 'tree' }); + const journal = new RecordJournal(store, { type: 'journal-test/entry' }); + const other = new RecordJournal(store, { type: 'journal-test/other' }); + journal.append({ n: 'a' }); + other.append({ n: 'x' }); + journal.append({ n: 'b' }); + journal.checkpoint({ seen: ['a', 'b'] }); + assert.strictEqual(journal.entriesSinceCheckpoint, 0); + store.treeSet('journal-test/tree', 'f.txt', { blobHash: 'f'.repeat(64), size: 1, mode: 0o644 }); + journal.append({ n: 'c' }); + other.append({ n: 'y' }); + assert.strictEqual(journal.entriesSinceCheckpoint, 1); + + const reader = new RecordJournal(store, { type: 'journal-test/entry' }); + const first = reader.load(); + assert.deepStrictEqual(first.snapshot, { seen: ['a', 'b'] }); + assert.deepStrictEqual(first.entries.map((e) => e.entry.n), ['c']); + assert.strictEqual(reader.entriesSinceCheckpoint, 1); + + // A checkpoint after load covers what was loaded; the newest one wins. + reader.checkpoint({ seen: ['a', 'b', 'c'] }); + const second = new RecordJournal(store, { type: 'journal-test/entry' }).load(); + assert.deepStrictEqual(second, { snapshot: { seen: ['a', 'b', 'c'] }, entries: [] }); + assert.deepStrictEqual(entriesOf(new RecordJournal(store, { type: 'journal-test/other' })), ['x', 'y']); + }); + }); + + it('names its two barriers: afterCommittedState syncs before the append, durable after it, neither by default', () => { + withStore((store) => { + const calls: string[] = []; + const recording = { + getRecordIdsByType: (type: string) => store.getRecordIdsByType(type), + getRecord: (id: string) => store.getRecord(id), + appendJson: (type: string, data: unknown) => { calls.push('append'); return store.appendJson(type, data); }, + sync: () => { calls.push('sync'); store.sync(); }, + } as unknown as JsStore; + const journal = new RecordJournal(recording, { type: 'journal-test/entry' }); + journal.append({ n: 'plain' }); + assert.deepStrictEqual(calls.splice(0), ['append'], 'a plain append does not sync'); + journal.append({ n: 'asserting' }, { afterCommittedState: true }); + assert.deepStrictEqual(calls.splice(0), ['sync', 'append']); + journal.append({ n: 'durable' }, { durable: true }); + assert.deepStrictEqual(calls.splice(0), ['append', 'sync']); + journal.append({ n: 'both' }, { afterCommittedState: true, durable: true }); + assert.deepStrictEqual(calls.splice(0), ['sync', 'append', 'sync']); + journal.checkpoint({ n: 1 }, { durable: true }); + assert.deepStrictEqual(calls.splice(0), ['append', 'sync']); + }); + }); + + it('after a hard kill, an afterCommittedState entry never survives without the state it asserts', () => { + withStore((store, path) => { + store.registerState({ id: 'journal-test/tree', strategy: 'tree' }); + store.sync(); + store.close(); + const child = spawnSync(process.execPath, [KILL_FIXTURE, path, 'asserting'], { encoding: 'utf-8' }); + assert.strictEqual(child.signal, 'SIGKILL', `the child died by SIGKILL (stderr: ${child.stderr})`); + const reopened = JsStore.openOrCreate({ path }); + try { + try { reopened.registerState({ id: 'journal-test/tree', strategy: 'tree' }); } catch { /* exists */ } + const entries = new RecordJournal<{ asserts: string }, unknown>(reopened, { type: 'journal-test/entry' }).load().entries; + assert.strictEqual(entries.length, 1, 'the appended entry survived the kill'); + assert.notStrictEqual(reopened.treeGet('journal-test/tree', 'committed.txt'), null, + 'the state it asserts survived with it'); + } finally { + reopened.close(); + } + }); + }); + + it('a write that fails after reaching the store blocks further writes until load() replays it', () => { + withStore((store) => { + let failNextSync = false; + const flaky = { + getRecordIdsByType: (type: string) => store.getRecordIdsByType(type), + getRecord: (id: string) => store.getRecord(id), + appendJson: (type: string, data: unknown) => store.appendJson(type, data), + sync: () => { + if (failNextSync) { failNextSync = false; throw new Error('disk full'); } + store.sync(); + }, + } as unknown as JsStore; + const journal = new RecordJournal(flaky, { type: 'journal-test/entry' }); + + // The durable barrier fails after A reached the store: the caller never reduces A. + failNextSync = true; + assert.throws(() => journal.append({ n: 'A' }, { durable: true }), /disk full/); + assert.strictEqual(journal.needsReconcile, true); + // Without reconciliation, a later write and checkpoint would cover A unreduced. + assert.throws(() => journal.append({ n: 'B' }), /load\(\) and reduce its tail before writing again/); + assert.throws(() => journal.checkpoint({ seen: ['B'] }), /load\(\) and reduce its tail/); + + const reconciled = journal.load(); + assert.deepStrictEqual(reconciled.entries.map((e) => e.entry.n), ['A'], 'load replays the ambiguous record'); + assert.strictEqual(journal.needsReconcile, false); + journal.append({ n: 'B' }); + journal.checkpoint({ seen: ['A', 'B'] }); + assert.deepStrictEqual(new RecordJournal(store, { type: 'journal-test/entry' }).load(), + { snapshot: { seen: ['A', 'B'] }, entries: [] }); + + // A failure BEFORE the record reaches the store wrote nothing and leaves the journal usable. + failNextSync = true; + assert.throws(() => journal.append({ n: 'C' }, { afterCommittedState: true }), /disk full/); + assert.strictEqual(journal.needsReconcile, false); + journal.append({ n: 'D' }); + assert.deepStrictEqual( + new RecordJournal(store, { type: 'journal-test/entry' }).load().entries.map((e) => e.entry.n), + ['D'], + ); + }); + }); + + it('an append that throws inside the store blocks writes for the life of the journal; a journal on the reopened store loads its record', () => { + withStore((store) => { + // As Chronicle's append can fail: the record is written to the log, the + // periodic sync then fails, and the record never reaches the index this + // process reads. Reopening the store indexes it from the log. `hidden` + // is this process's view of the store. + let failAppend: 'after-writing' | 'before-writing' | null = null; + const hidden = new Set(); + const live = { + getRecordIdsByType: (type: string) => store.getRecordIdsByType(type).filter((id) => !hidden.has(id)), + getRecord: (id: string) => (hidden.has(id) ? null : store.getRecord(id)), + appendJson: (type: string, data: unknown) => { + if (failAppend === 'before-writing') { failAppend = null; throw new Error('EIO: write failed'); } + const record = store.appendJson(type, data); + if (failAppend === 'after-writing') { failAppend = null; hidden.add(record.id); throw new Error('EIO: fsync failed'); } + return record; + }, + sync: () => store.sync(), + } as unknown as JsStore; + const journal = new RecordJournal(live, { type: 'journal-test/entry' }); + journal.append({ n: 'A' }); + + failAppend = 'after-writing'; + assert.throws(() => journal.append({ n: 'B' }), /EIO: fsync failed/); + assert.strictEqual(journal.needsReconcile, false, 'no load() on this store can read B back'); + assert.strictEqual(journal.needsReopen, true, 'what this view holds may lack B until a reopen'); + assert.deepStrictEqual(journal.load().entries.map((e) => e.entry.n), ['A'], 'B is outside this view of the store'); + // A checkpoint now would cover B, unreduced, so nothing more is written. + const blocked = /append failed inside the store \(EIO: fsync failed\).*until the store is reopened/; + assert.throws(() => journal.append({ n: 'C' }), blocked); + assert.throws(() => journal.checkpoint({ seen: ['A'] }), blocked); + assert.throws(() => journal.append({ n: 'C' }), blocked, 'load() does not lift it'); + + // A journal on the reopened store loads B with its tail, and writes again. + const reopened = new RecordJournal(store, { type: 'journal-test/entry' }); + assert.deepStrictEqual(reopened.load().entries.map((e) => e.entry.n), ['A', 'B']); + reopened.append({ n: 'C' }); + reopened.checkpoint({ seen: ['A', 'B', 'C'] }); + assert.deepStrictEqual(new RecordJournal(store, { type: 'journal-test/entry' }).load(), + { snapshot: { seen: ['A', 'B', 'C'] }, entries: [] }); + + // An append that fails before writing anything looks the same from + // here, so it blocks the same way; the reopened store has nothing new. + const second = new RecordJournal(live, { type: 'journal-test/entry' }); + second.load(); + failAppend = 'before-writing'; + assert.throws(() => second.append({ n: 'X' }), /EIO: write failed/); + assert.throws(() => second.append({ n: 'D' }), /until the store is reopened/); + assert.deepStrictEqual(new RecordJournal(store, { type: 'journal-test/entry' }).load(), + { snapshot: { seen: ['A', 'B', 'C'] }, entries: [] }); + }); + }); + + it('a durable checkpoint whose barrier throws blocks writes until load() replays it', () => { + withStore((store) => { + let failNextSync = false; + const flaky = { + getRecordIdsByType: (type: string) => store.getRecordIdsByType(type), + getRecord: (id: string) => store.getRecord(id), + appendJson: (type: string, data: unknown) => store.appendJson(type, data), + sync: () => { + if (failNextSync) { failNextSync = false; throw new Error('disk full'); } + store.sync(); + }, + } as unknown as JsStore; + const journal = new RecordJournal(flaky, { type: 'journal-test/entry' }); + journal.append({ n: 'A' }); + + failNextSync = true; + assert.throws(() => journal.checkpoint({ seen: ['A'] }, { durable: true }), /disk full/); + assert.strictEqual(journal.needsReconcile, true); + assert.strictEqual(journal.needsReopen, false, 'the checkpoint is in this view of the store'); + assert.throws(() => journal.append({ n: 'C' }), /load\(\) and reduce its tail/); + assert.deepStrictEqual(journal.load(), { snapshot: { seen: ['A'] }, entries: [] }, 'the checkpoint landed'); + journal.append({ n: 'C' }); + }); + }); + + it('refuses a checkpoint type equal to the entry type', () => { + withStore((store) => { + assert.throws(() => new RecordJournal(store, { type: 't', checkpointType: 't' }), /distinct record types/); + }); + }); +}); diff --git a/test/tool-batch-hook.test.ts b/test/tool-batch-hook.test.ts new file mode 100644 index 000000000..b2deb73ab --- /dev/null +++ b/test/tool-batch-hook.test.ts @@ -0,0 +1,316 @@ +/** + * Module.onToolBatchComplete (shelf-383): awaited when an agent's tool round + * completes — its last pending result arrives — before that result is + * provided, and so before anything continues the turn. Bounded and fail-open; + * the agent's state is checked again after it. + */ + +import { describe, it, beforeEach, afterEach } from 'node:test'; +import assert from 'node:assert/strict'; +import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import type { ContentBlock } from '@animalabs/membrane'; +import type { + EventResponse, + Module, + ModuleContext, + ProcessEvent, + ProcessState, + ToolCall, + ToolDefinition, + ToolResult, +} from '../src/index.js'; +import { AgentFramework } from '../src/index.js'; +import type { TraceEvent } from '../src/types/trace.js'; +import { WorkspaceModule } from '../src/modules/workspace/index.js'; +import { MockMembrane, MockYieldingStream, createMockResponse } from './helpers/mock-membrane.js'; + +class ToolModule implements Module { + readonly name = 'robot'; + membrane: MockMembrane; + hookCalls: Array<{ agentName: string; resultsProvidedBefore: number }> = []; + onHook: ((agentName: string) => Promise) | null = null; + onTool: ((call: ToolCall) => void) | null = null; + + constructor(membrane: MockMembrane) { + this.membrane = membrane; + } + + async start(_ctx: ModuleContext): Promise {} + async stop(): Promise {} + + getTools(): ToolDefinition[] { + return [{ name: 'act', description: 'Act', inputSchema: { type: 'object', properties: { n: { type: 'number' } } } }]; + } + + async handleToolCall(call: ToolCall): Promise { + this.onTool?.(call); + return { success: true, data: { ok: true } }; + } + + async onProcess(event: ProcessEvent, _state: ProcessState): Promise { + if (event.type !== 'external-message') return {}; + return { + addMessages: [{ participant: 'User', content: [{ type: 'text', text: String((event as { content?: unknown }).content) }] }], + requestInference: true, + }; + } + + async onToolBatchComplete(agentName: string): Promise { + this.hookCalls.push({ agentName, resultsProvidedBefore: this.membrane.lastStream?.receivedToolResults.length ?? 0 }); + await this.onHook?.(agentName); + } +} + +describe('Module.onToolBatchComplete', () => { + let tempDir: string; + let membrane: MockMembrane; + let tools: ToolModule; + + beforeEach(() => { + tempDir = mkdtempSync(join(tmpdir(), 'batch-hook-')); + membrane = new MockMembrane(); + tools = new ToolModule(membrane); + }); + + afterEach(() => { + rmSync(tempDir, { recursive: true, force: true }); + }); + + async function createFramework(extra: Module[] = []): Promise { + return AgentFramework.create({ + storePath: join(tempDir, 'test.chronicle'), + membrane: membrane.asMembrane(), + agents: [{ name: 'assistant', model: 'test-model', systemPrompt: 'You act.' }], + modules: [tools, ...extra], + }); + } + + /** One round of two tool calls, then a closing text response. */ + function scriptTwoCallRound(): void { + membrane.pushResponse(createMockResponse([ + { type: 'tool_use', id: 'c1', name: 'robot--act', input: { n: 1 } }, + { type: 'tool_use', id: 'c2', name: 'robot--act', input: { n: 2 } }, + ] as ContentBlock[], 'tool_use')); + membrane.pushResponse(createMockResponse([{ type: 'text', text: 'done' }] as ContentBlock[])); + } + + function trigger(framework: AgentFramework): void { + framework.pushEvent({ type: 'external-message', source: 'test', content: 'go', metadata: {} } as unknown as ProcessEvent); + } + + it('runs once per completed batch, before its results reach the stream', async () => { + scriptTwoCallRound(); + const framework = await createFramework(); + trigger(framework); + await framework.runUntilIdle(); + + assert.deepEqual(tools.hookCalls, [{ agentName: 'assistant', resultsProvidedBefore: 0 }], + 'one call for the two-call round, before the round was resumed'); + assert.equal(membrane.lastStream!.receivedToolResults.length, 1, 'the round then resumed with its results'); + await framework.stop(); + }); + + it('a hook that outlasts its bound is traced and skipped, and the round goes on', async () => { + scriptTwoCallRound(); + const framework = await createFramework(); + const registry = (framework as unknown as { moduleRegistry: { notifyToolBatchComplete: (a: string, t?: number) => Promise } }).moduleRegistry; + const notify = registry.notifyToolBatchComplete.bind(registry); + registry.notifyToolBatchComplete = (agentName: string) => notify(agentName, 50); + const traces: TraceEvent[] = []; + framework.onTrace((e) => traces.push(e)); + tools.onHook = () => new Promise(() => {}); // never settles + + trigger(framework); + await framework.runUntilIdle(); + + const failed = traces.find((e) => e.type === 'module:batch_hook_failed') as + | { module: string; error: string; agentName: string } | undefined; + assert.equal(failed?.module, 'robot'); + assert.equal(failed?.agentName, 'assistant'); + assert.match(failed!.error, /timed out after 50ms/); + assert.equal(membrane.lastStream!.receivedToolResults.length, 1, 'the round went on without the hook'); + await framework.stop(); + }); + + it('a hook that throws before returning its promise is traced and skipped, and later hooks still run', async () => { + // Registered before the robot module, so its synchronous throw would have + // escaped while the robot's hook was still to be called. + const thrower: Module = { + name: 'thrower', + async start() {}, + async stop() {}, + getTools: () => [], + handleToolCall: async () => ({ success: true }), + onProcess: async () => ({}), + onToolBatchComplete(): Promise { + throw new Error('thrown before any promise'); + }, + }; + scriptTwoCallRound(); + const framework = await AgentFramework.create({ + storePath: join(tempDir, 'test.chronicle'), + membrane: membrane.asMembrane(), + agents: [{ name: 'assistant', model: 'test-model', systemPrompt: 'You act.' }], + modules: [thrower, tools], + }); + const traces: TraceEvent[] = []; + framework.onTrace((e) => traces.push(e)); + + trigger(framework); + await framework.runUntilIdle(); + + const failed = traces.find((e) => e.type === 'module:batch_hook_failed') as + | { module: string; error: string } | undefined; + assert.equal(failed?.module, 'thrower'); + assert.match(failed!.error, /thrown before any promise/); + assert.equal(tools.hookCalls.length, 1, "the other module's hook still ran"); + assert.equal(membrane.lastStream!.receivedToolResults.length, 1, 'the round went on with its results'); + await framework.stop(); + }); + + it('an agent cancelled during the hook has the result dropped, not provided', async () => { + scriptTwoCallRound(); + const framework = await createFramework(); + const traces: TraceEvent[] = []; + framework.onTrace((e) => traces.push(e)); + tools.onHook = async (agentName) => { + framework.abortInference(agentName, 'cancelled at the batch boundary'); + }; + + trigger(framework); + await framework.runUntilIdle(); + + assert.equal(membrane.lastStream!.receivedToolResults.length, 0, 'nothing was provided to the cancelled round'); + assert.ok(traces.some((e) => e.type === 'tool:result_dropped'), 'the dropped result is traced'); + await framework.stop(); + }); + + it('a result whose call the turn no longer waits on after the hook is dropped and traced, not thrown', async () => { + scriptTwoCallRound(); + const framework = await createFramework(); + const traces: TraceEvent[] = []; + framework.onTrace((e) => traces.push(e)); + const errors: string[] = []; + const originalError = console.error; + console.error = (...args: unknown[]) => { errors.push(args.map(String).join(' ')); }; + tools.onHook = async (agentName) => { + // The round is replaced while modules catch up: the agent waits again, + // now on a call of a new batch. + const agent = framework.getAgent(agentName) as unknown as { _state: { status: string; pending: Map } }; + agent._state = { + ...agent._state, + status: 'waiting_for_tools', + pending: new Map([['c-new', { id: 'c-new', name: 'robot--act', input: {}, startedAt: Date.now() }]]), + }; + setTimeout(() => framework.abortInference(agentName, 'end the replaced round'), 50); + }; + try { + trigger(framework); + await framework.runUntilIdle(); + } finally { + console.error = originalError; + } + const dropped = traces.find((e) => e.type === 'tool:result_dropped') as { callId: string } | undefined; + assert.ok(dropped, 'the stale result is traced as dropped'); + assert.ok(!errors.some((line) => line.includes('Unknown tool call')), errors.join('\n')); + assert.equal(membrane.lastStream!.receivedToolResults.length, 0, 'nothing was provided to the replaced round'); + await framework.stop(); + }); + + it('a result is dropped when the round is replaced by one that reuses its call id', async () => { + scriptTwoCallRound(); + const framework = await createFramework(); + const traces: TraceEvent[] = []; + framework.onTrace((e) => traces.push(e)); + tools.onHook = async (agentName) => { + // The round is replaced while modules catch up by one waiting on a call + // with the same id: a new entry for it, not the one this result answers. + const agent = framework.getAgent(agentName) as unknown as { _state: { status: string; pending: Map } }; + const [id] = [...agent._state.pending.keys()]; + agent._state = { + ...agent._state, + status: 'waiting_for_tools', + pending: new Map([[id!, { id: id!, name: 'robot--act', input: { generation: 2 }, startedAt: Date.now() } as unknown as { id: string }]]), + }; + setTimeout(() => framework.abortInference(agentName, 'end the replaced round'), 50); + }; + trigger(framework); + await framework.runUntilIdle(); + assert.ok(traces.some((e) => e.type === 'tool:result_dropped'), 'the stale result is traced as dropped'); + assert.equal(membrane.lastStream!.receivedToolResults.length, 0, 'nothing was provided to the replacing round'); + await framework.stop(); + }); + + it("a result for a call that isn't pending doesn't run the hook early", async () => { + // One call: while it runs, a stray result for another call arrives. + membrane.pushResponse(createMockResponse([ + { type: 'tool_use', id: 'c1', name: 'robot--act', input: { n: 1 } }, + ] as ContentBlock[], 'tool_use')); + membrane.pushResponse(createMockResponse([{ type: 'text', text: 'done' }] as ContentBlock[])); + const framework = await createFramework(); + const originalError = console.error; + console.error = () => {}; + tools.onTool = () => { + framework.pushEvent({ type: 'tool-result', callId: 'stray', agentName: 'assistant', moduleName: 'robot', result: { success: true } } as unknown as ProcessEvent); + }; + try { + trigger(framework); + // The stray result throws where it's provided, as on main; the round + // then goes on with its own result. + await assert.rejects(() => framework.runUntilIdle(), /Unknown tool call: stray/); + await framework.runUntilIdle(); + } finally { + console.error = originalError; + } + assert.equal(tools.hookCalls.length, 1, 'only the round\'s own last result runs the hook'); + assert.equal(membrane.lastStream!.receivedToolResults.length, 1); + await framework.stop(); + }); + + it('a hook still running from an earlier batch is skipped, not called and waited for again', async () => { + const framework = await createFramework(); + const registry = (framework as unknown as { moduleRegistry: { notifyToolBatchComplete: (a: string, t?: number) => Promise> } }).moduleRegistry; + let calls = 0; + tools.onHook = () => { calls++; return new Promise(() => {}); }; // never settles + const first = await registry.notifyToolBatchComplete('assistant', 50); + assert.match(first[0]!.error, /timed out after 50ms/); + const started = Date.now(); + const second = await registry.notifyToolBatchComplete('assistant', 5_000); + assert.ok(Date.now() - started < 1_000, 'the second batch was not held'); + assert.deepEqual(second, [{ module: 'robot', error: 'still running from an earlier batch' }]); + assert.equal(calls, 1, 'the wedged hook was called once'); + await framework.stop(); + }); + + it("an on-agent-action workspace has the batch's disk changes before the stream resumes", async () => { + const dir = join(tempDir, 'files'); + mkdirSync(dir, { recursive: true }); + const workspace = new WorkspaceModule({ + mounts: [{ name: 'files', path: dir, mode: 'read-write', watch: 'on-agent-action' }], + }); + // A tool that changes disk directly, as a shell command would. + tools.onTool = (call) => writeFileSync(join(dir, `made-by-${call.id}.txt`), 'from a tool'); + scriptTwoCallRound(); + const framework = await createFramework([workspace]); + workspace.initStore(framework.getStore()); // as hosts do + + const seenAtResume: Array = []; + const provide = MockYieldingStream.prototype.provideToolResults; + MockYieldingStream.prototype.provideToolResults = function (this: MockYieldingStream, ...args) { + const store = (workspace as unknown as { store: { treeGet: (s: string, p: string) => { blobHash: string } | null } }).store; + for (const id of ['c1', 'c2']) seenAtResume.push(store.treeGet('workspace/files/tree', `made-by-${id}.txt`) ? id : null); + return provide.apply(this, args); + }; + try { + trigger(framework); + await framework.runUntilIdle(); + } finally { + MockYieldingStream.prototype.provideToolResults = provide; + } + + assert.deepEqual(seenAtResume, ['c1', 'c2'], 'both files were in the workspace when the round resumed'); + await framework.stop(); + }); +}); diff --git a/test/workspace-materialize-freshness.test.ts b/test/workspace-materialize-freshness.test.ts index f0e7d748c..6f6256f03 100644 --- a/test/workspace-materialize-freshness.test.ts +++ b/test/workspace-materialize-freshness.test.ts @@ -6,15 +6,16 @@ * single-file occurrence in the issue thread, stopped only by an incidental * branch-guard refusal). * - * The guard compares the disk content's hash against `materializedHashes` - * (what the layer last wrote): a mismatch means another writer changed the - * file, and the path is refused LOUDLY — listed in the result, not resolved - * silently in either direction — unless `force: true`. + * The guard compares disk with the evidence of what disk last agreed with + * (workspace/disk-agreement, recorded as the layer writes and syncs): a + * mismatch means another writer changed the file, and the path is refused + * LOUDLY — listed in the result, not resolved silently in either direction — + * unless `force: true`. */ import test, { type TestContext } from 'node:test'; import assert from 'node:assert/strict'; -import { chmodSync, mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs'; +import { chmodSync, mkdirSync, mkdtempSync, readFileSync, rmSync, statSync, writeFileSync } from 'node:fs'; import { tmpdir } from 'node:os'; import { join } from 'node:path'; import { JsStore } from '@animalabs/chronicle'; @@ -108,6 +109,8 @@ test('a refused path stays pending: the next materialize reports it again instea const first = await call(module, 'materialize', {}); assert.ok(((first.data as MaterializeData).skipped ?? []).length > 0); + const pendingAfter = ((await call(module, 'status', {})).data as Record).work.pendingChanges; + assert.ok(pendingAfter >= 1, `the refused path still counts as pending, got ${pendingAfter}`); // If the watermark advanced past the refusal, this second call would diff // to nothing and the divergence would vanish from view — a loud refusal @@ -118,7 +121,7 @@ test('a refused path stays pending: the next materialize reports it again instea assert.equal(readFileSync(join(dir, 'script.sh'), 'utf8'), 'echo edited-by-shell'); }); -test('sync-then-materialize resolves the divergence in the disk direction', async (t) => { +test('syncing the path resolves the divergence in the disk direction; a full sync keeps the workspace edit', async (t) => { const { dir, module } = setup(t); await module.start(makeCtx()); await seedBaseline(module, dir); @@ -127,11 +130,20 @@ test('sync-then-materialize resolves the divergence in the disk direction', asyn await call(module, 'write', { path: 'work/script.sh', content: 'echo v2' }); await call(module, 'materialize', {}); // refused, listed - // The remedy the skip reason points at: adopt the disk version. + // A full sync rechecks everything but never discards a pending workspace + // edit (shelf-383): the divergence is still there to resolve deliberately. await call(module, 'sync', {}); + const still = await call(module, 'materialize', {}); + assert.ok(((still.data as MaterializeData).skipped ?? []).some((s) => s.reason.includes('script.sh')), + 'a full sync leaves the conflict for an explicit resolution'); + const read = await call(module, 'read', { path: 'work/script.sh' }); + assert.match(String((read.data as { content: string }).content), /echo v2/, 'the workspace edit survives'); + + // The remedy the skip reason points at: sync this path to adopt the disk version. + await call(module, 'sync', { path: 'work/script.sh' }); const res = await call(module, 'materialize', {}); assert.equal(((res.data as MaterializeData).skipped ?? []).length, 0, - 'after sync the tree matches disk — nothing left to refuse'); + 'after syncing the path the tree matches disk — nothing left to refuse'); assert.equal(readFileSync(join(dir, 'script.sh'), 'utf8'), 'echo edited-by-shell'); }); @@ -240,9 +252,12 @@ test('a first materialize that finds every file in place still pins the branch', // Disk already holds the same bytes (e.g. restored from a backup). writeFileSync(join(dir, 'a.txt'), 'a'); writeFileSync(join(dir, 'b.txt'), 'b'); + const untouched = (name: string) => { const st = statSync(join(dir, name)); return { ino: st.ino, mtimeMs: st.mtimeMs }; }; + const before = { a: untouched('a.txt'), b: untouched('b.txt') }; const first = await call(module, 'materialize', {}); assert.equal((first.data as MaterializeData).materialized.length, 0, 'nothing needed writing'); + assert.deepEqual({ a: untouched('a.txt'), b: untouched('b.txt') }, before, 'neither file was rewritten'); const status = (await call(module, 'status', {})).data as Record; assert.equal(status.work.lastMaterializedBranch, store.currentBranch().id, 'the branch is pinned'); diff --git a/test/workspace-three-way.test.ts b/test/workspace-three-way.test.ts new file mode 100644 index 000000000..fb6f033fe --- /dev/null +++ b/test/workspace-three-way.test.ts @@ -0,0 +1,3348 @@ +/** + * Truthful workspace listings under the three-way rule (shelf-383): disk (D), + * the workspace store (S) and physical disk-agreement evidence (P), with + * branch-local intent (tombstones, store origin, sticky conflicts). + * + * Each describe block covers one entry of the shelf's "done when" list. Faults + * are real: a child process (fixtures/workspace-fault.mjs) runs an operation + * and dies by SIGKILL between intent, effect and completion, and a torn tail + * is a truncated records.log. + */ + +import { describe, test, type TestContext } from 'node:test'; +import { TOOL_BATCH_HOOK_TIMEOUT_MS } from '../src/module-registry.js'; +import assert from 'node:assert/strict'; +import { spawnSync } from 'node:child_process'; +import { chmodSync, constants as fsConstants, existsSync, linkSync, mkdirSync, mkdtempSync, readFileSync, realpathSync, renameSync, rmSync, statSync, symlinkSync, truncateSync, unlinkSync, utimesSync, writeFileSync } from 'node:fs'; +import { open, statfs } from 'node:fs/promises'; +import { createHash } from 'node:crypto'; +import { createRequire, syncBuiltinESMExports } from 'node:module'; +import { tmpdir } from 'node:os'; +import { basename, dirname, join } from 'node:path'; +import { JsStore } from '@animalabs/chronicle'; +import { WorkspaceModule } from '../src/modules/workspace/index.js'; +import { DiskAgreement } from '../src/modules/workspace/disk-agreement.js'; +import { confirmAbsent, fingerprintVouches, filesystemTrustsCtime, fsTypeTrustsCtime, observePath, readPath, type MountView } from '../src/modules/workspace/observe.js'; +import { EffectFailed, syncDirectories, unlinkContained, writeContained } from '../src/modules/workspace/effects.js'; +import { pushPaths, settleAdoptionBeforeMutation } from '../src/modules/workspace/reconcile.js'; +import type { MountConfig, WorkspaceConfig } from '../src/modules/workspace/types.js'; +import type { ModuleContext } from '../src/types/module.js'; +import type { ProcessEvent } from '../src/types/events.js'; + +const FAULT_FIXTURE = join(import.meta.dirname, 'fixtures/workspace-fault.mjs'); +const TREE = 'workspace/work/tree'; +const ONE_PX_PNG = Buffer.from( + 'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNkYPhfDwAChwGA60e6kgAAAABJRU5ErkJggg==', + 'base64', +); +const IS_ROOT = typeof process.getuid === 'function' && process.getuid() === 0; + +type Entry = { + path: string; + state: string; + size?: number; + mimeType?: string; + conflict?: { kind: string; diskCopy: string; diskChangedSinceRecorded: boolean }; + note?: string; +}; + +class Env { + readonly root = mkdtempSync(join(tmpdir(), 'af-ws-3way-')); + readonly dir = join(this.root, 'work'); + readonly storePath = join(this.root, 'ws.chronicle'); + store: JsStore; + events: ProcessEvent[] = []; + private live: WorkspaceModule[] = []; + + constructor(t: TestContext) { + mkdirSync(this.dir, { recursive: true }); + this.store = JsStore.openOrCreate({ path: this.storePath }); + t.after(async () => { + for (const m of this.live) await m.stop().catch(() => {}); + if (!this.store.isClosed()) this.store.close(); + rmSync(this.root, { recursive: true, force: true }); + }); + } + + async open(mount: Partial = {}, config: Partial = {}): Promise { + const m = new WorkspaceModule({ + mounts: [{ name: 'work', path: this.dir, mode: 'read-write', watch: 'never', ...mount }], + ...config, + }); + m.initStore(this.store); + await m.start({ + isRestart: false, + getState: () => null, + setState: () => {}, + pushEvent: (e: ProcessEvent) => this.events.push(e), + } as unknown as ModuleContext); + this.live.push(m); + return m; + } + + /** Start a module as a restart that finds `state` saved as its module state. */ + async openRestarted(state: unknown, mount: Partial = {}): Promise { + const m = new WorkspaceModule({ mounts: [{ name: 'work', path: this.dir, mode: 'read-write', watch: 'never', ...mount }] }); + m.initStore(this.store); + await m.start({ + isRestart: true, + getState: () => state, + setState: () => {}, + pushEvent: (e: ProcessEvent) => this.events.push(e), + } as unknown as ModuleContext); + this.live.push(m); + return m; + } + + /** Stop a module and close the store, as a process exit would. */ + async shutdown(m: WorkspaceModule): Promise { + await m.stop(); + this.live = this.live.filter((x) => x !== m); + this.store.close(); + } + + /** A fresh process: reopen the store and start a new module on it. */ + async restart(m: WorkspaceModule, mount: Partial = {}): Promise { + await this.shutdown(m); + this.store = JsStore.openOrCreate({ path: this.storePath }); + return this.open(mount); + } + + /** Run one operation in a child that dies by SIGKILL at `killAt`, then reopen. */ + async fault(m: WorkspaceModule, op: 'materialize' | 'sync' | 'write', killAt: 'after-intent' | 'after-effect'): Promise { + await this.shutdown(m); + const child = spawnSync(process.execPath, [FAULT_FIXTURE, this.storePath, this.dir, op, killAt], { encoding: 'utf-8' }); + assert.equal(child.signal, 'SIGKILL', `the child died at ${killAt} (status ${child.status}, stderr: ${child.stderr})`); + this.store = JsStore.openOrCreate({ path: this.storePath }); + return this.open(); + } + + disk(path: string): string { + return join(this.dir, path); + } + + readDisk(path: string): string { + return readFileSync(this.disk(path), 'utf8'); + } + + writeDisk(path: string, content: string | Buffer): void { + mkdirSync(dirname(this.disk(path)), { recursive: true }); + writeFileSync(this.disk(path), content); + } + + rmDisk(path: string): void { + rmSync(this.disk(path)); + } + + /** A workspace entry written straight into the tree: no evidence, no intent (a pre-383 store). */ + legacyEntry(path: string, content: string): void { + const bytes = Buffer.from(content); + const blobHash = this.store.storeBlob(bytes, 'text/plain'); + this.store.treeSet(TREE, path, { blobHash, size: bytes.byteLength, mode: 0o644 }); + } + + eventsOf(type: string): Array<{ paths: string[]; conflicts?: string[] }> { + return this.events.filter((e) => e.type === type) as unknown as Array<{ paths: string[]; conflicts?: string[] }>; + } +} + +async function call(m: WorkspaceModule, name: string, input: Record): Promise { + const res = await m.handleToolCall({ id: 't', name, input }); + assert.equal(res.success, true, `${name} ${JSON.stringify(input)} failed: ${res.error}`); + return res.data; +} + +async function refused(m: WorkspaceModule, name: string, input: Record): Promise { + const res = await m.handleToolCall({ id: 't', name, input }); + assert.equal(res.success, false, `${name} ${JSON.stringify(input)} unexpectedly succeeded`); + return String(res.error); +} + +async function listing(m: WorkspaceModule, path = 'work'): Promise> { + const data = await call(m, 'ls', { path, recursive: true }); + return new Map((data.entries as Entry[]).map((e) => [e.path, e])); +} + +async function entryOf(m: WorkspaceModule, path: string): Promise { + return (await listing(m)).get(path); +} + +async function stateOf(m: WorkspaceModule, path: string): Promise { + return (await entryOf(m, path))?.state ?? 'not listed'; +} + +/** The workspace's text for a path, unformatted. */ +async function contentOf(m: WorkspaceModule, path: string): Promise { + const data = await call(m, 'read', { path: `work/${path}`, offsetChars: 0, limitChars: 1_000_000 }); + return data.content as string; +} + +/** Written through the workspace and materialized: S = D = P. */ +async function seedSynced(env: Env, m: WorkspaceModule, path: string, content: string): Promise { + await call(m, 'write', { path: `work/${path}`, content }); + await call(m, 'materialize', { path: `work/${path}` }); + assert.equal(env.readDisk(path), content); +} + +const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms)); + +/** The hash evidence and disk facts carry for this text. */ +const sha = (text: string) => createHash('sha256').update(text).digest('hex'); + +/** Run `fn` with FileHandle.readFile switching branches as `onRead` says. */ +async function withReadHook(onRead: (calls: number) => void, fn: () => Promise): Promise { + const probe = await open(join(tmpdir(), `read-hook-${process.pid}`), 'w'); + const proto = Object.getPrototypeOf(probe) as { readFile: (...args: unknown[]) => Promise }; + await probe.close(); + const readFile = proto.readFile; + let calls = 0; + proto.readFile = async function (this: unknown, ...args: unknown[]) { + onRead(++calls); + return readFile.apply(this, args); + }; + try { + return await fn(); + } finally { + proto.readFile = readFile; + } +} + +/** Run `fn` with node:fs/promises' `name` replaced by `make(original)`, as the code under test sees it. */ +async function withFsPromises(name: 'readdir' | 'lstat' | 'open' | 'realpath', make: (original: (...args: any[]) => Promise) => (...args: any[]) => Promise, fn: () => Promise): Promise { + const fsp = createRequire(import.meta.url)('node:fs/promises') as Record Promise>; + const original = fsp[name]!; + fsp[name] = make(original); + syncBuiltinESMExports(); + try { + return await fn(); + } finally { + fsp[name] = original; + syncBuiltinESMExports(); + } +} + +/** A file or directory's identity, as a FileHandle's stat reports it. */ +function idOf(path: string): string { + const s = statSync(path); + return `${s.dev}:${s.ino}`; +} + +/** + * Run `fn` recording what each FileHandle sync reaches (by identity), failing + * those `fail` picks with an injected EIO. + */ +async function withSyncs(env: Env, fail: (id: string, isDirectory: boolean) => boolean, fn: () => Promise): Promise<{ result: T; synced: string[] }> { + const probe = await open(join(env.root, 'sync-probe'), 'w'); + const proto = Object.getPrototypeOf(probe) as { sync: (this: { stat: () => Promise<{ dev: number; ino: number; isDirectory: () => boolean }> }) => Promise }; + await probe.close(); + const sync = proto.sync; + const synced: string[] = []; + proto.sync = async function () { + const info = await this.stat(); + const id = `${info.dev}:${info.ino}`; + if (fail(id, info.isDirectory())) throw Object.assign(new Error('injected EIO'), { code: 'EIO' }); + synced.push(id); + return sync.call(this); + }; + try { + return { result: await fn(), synced }; + } finally { + proto.sync = sync; + } +} + +/** Swap a mount directory for a symlink to `target`, keeping the original as `saved`. */ +function swapForSymlink(env: Env, dir: string, target: string, saved = 'saved'): void { + renameSync(env.disk(dir), env.disk(saved)); + symlinkSync(target, env.disk(dir)); +} + + +// --------------------------------------------------------------------------- + +describe('each three-way case', () => { + test('a disk-only change is ingested', async (t) => { + const env = new Env(t); + const m = await env.open(); + await seedSynced(env, m, 'a.txt', 'v1'); + env.writeDisk('a.txt', 'v1, edited in the shell'); + assert.equal(await stateOf(m, 'a.txt'), 'synced'); + assert.equal(await contentOf(m, 'a.txt'), 'v1, edited in the shell'); + }); + + test('a store-only draft is kept through scans, watcher events, restarts and reattach', async (t) => { + const env = new Env(t); + let m = await env.open(); + await seedSynced(env, m, 'a.txt', 'v1'); + await call(m, 'write', { path: 'work/a.txt', content: 'v2, a draft' }); + const kept = async (when: string): Promise => { + assert.equal(await stateOf(m, 'a.txt'), 'workspace-draft', when); + assert.equal(await contentOf(m, 'a.txt'), 'v2, a draft', `${when}: the draft survives`); + assert.equal(env.readDisk('a.txt'), 'v1', `${when}: disk is untouched`); + }; + await kept('after ls'); + await call(m, 'sync', {}); + await kept('after a full sync'); + await call(m, 'glob', { pattern: '**' }); + await call(m, 'grep', { pattern: 'v' }); + await kept('after glob and grep'); + await (m as any).handleFsChanges('work', [{ path: 'a.txt', op: 'modified' }]); + await kept('after a watcher event'); + m = await env.restart(m); + await kept('after a restart'); + await (m as any).initialScan('work'); + await kept('after a reattach scan'); + }); + + test('agreement advances the baseline', async (t) => { + const env = new Env(t); + const m = await env.open(); + await seedSynced(env, m, 'a.txt', 'v1'); + await call(m, 'write', { path: 'work/a.txt', content: 'v2' }); + env.writeDisk('a.txt', 'v2'); // disk reaches the draft by another route + assert.equal(await stateOf(m, 'a.txt'), 'synced'); + // Had P stayed at v1, this edit would be "all three differ" — a conflict. + env.writeDisk('a.txt', 'v3'); + assert.equal(await stateOf(m, 'a.txt'), 'synced', 'disk changed since it agreed at v2: adopted'); + assert.equal(await contentOf(m, 'a.txt'), 'v3'); + }); + + test('both changed is a conflict, kept, and resolvable either way', async (t) => { + const env = new Env(t); + const m = await env.open(); + for (const f of ['take-disk.txt', 'take-workspace.txt']) { + await seedSynced(env, m, f, 'v1'); + env.writeDisk(f, 'v3, from the shell'); + await call(m, 'write', { path: `work/${f}`, content: 'v2, from the agent' }); + } + const entries = await listing(m); + for (const f of ['take-disk.txt', 'take-workspace.txt']) { + const e = entries.get(f)!; + assert.equal(e.state, 'conflict', f); + assert.equal(e.conflict?.kind, 'both-changed', f); + assert.equal(e.conflict?.diskCopy, 'stored', `${f}: the text disk version is kept in the store`); + } + await call(m, 'sync', {}); + const bulk = await call(m, 'materialize', {}); + assert.equal(bulk.materialized.length, 0, 'neither side overwrites the other unasked'); + assert.equal(env.readDisk('take-disk.txt'), 'v3, from the shell'); + assert.equal(await contentOf(m, 'take-workspace.txt'), 'v2, from the agent'); + + await call(m, 'sync', { path: 'work/take-disk.txt' }); + assert.equal(await stateOf(m, 'take-disk.txt'), 'synced', 'a path sync resolves toward disk'); + assert.equal(await contentOf(m, 'take-disk.txt'), 'v3, from the shell'); + + await call(m, 'materialize', { path: 'work/take-workspace.txt', force: true }); + assert.equal(await stateOf(m, 'take-workspace.txt'), 'synced', 'a forced materialize resolves toward the workspace'); + assert.equal(env.readDisk('take-workspace.txt'), 'v2, from the agent'); + }); +}); + +describe('branches', () => { + test('a switch to an earlier state while disk holds a later agreed version keeps the selected state as a draft', async (t) => { + const env = new Env(t); + const m = await env.open(); + await seedSynced(env, m, 'a.txt', 'v1'); + const main = env.store.currentBranch().name; + env.store.createBranch('earlier', main); + await call(m, 'write', { path: 'work/a.txt', content: 'v2' }); + await call(m, 'materialize', {}); + assert.equal(env.readDisk('a.txt'), 'v2'); + + env.store.switchBranch('earlier'); + assert.equal(await stateOf(m, 'a.txt'), 'workspace-draft', 'a rewind is a store-side difference, not a disk edit'); + await call(m, 'sync', {}); + assert.equal(await contentOf(m, 'a.txt'), 'v1', 'no scan ingests the later version into the earlier state'); + assert.equal(env.readDisk('a.txt'), 'v2', 'and nothing rewrote disk'); + + env.store.switchBranch(main); + assert.equal(await stateOf(m, 'a.txt'), 'synced'); + }); + + test("a file only in another branch's history is listed, neither ingested nor unlinked, and a path sync adopts it", async (t) => { + const env = new Env(t); + const m = await env.open(); + await seedSynced(env, m, 'a.txt', 'shared'); + const main = env.store.currentBranch().name; + env.store.createBranch('without-b', main); + await seedSynced(env, m, 'b.txt', 'only on main'); + + env.store.switchBranch('without-b'); + assert.equal(await stateOf(m, 'b.txt'), 'not-in-branch'); + assert.match(await refused(m, 'read', { path: 'work/b.txt' }), /not in this branch/); + await call(m, 'sync', {}); + assert.equal(await stateOf(m, 'b.txt'), 'not-in-branch', 'a full sync does not ingest it'); + await call(m, 'materialize', { force: true, applyDeletions: true }); + assert.equal(env.readDisk('b.txt'), 'only on main', 'not even force with applyDeletions unlinks it'); + + await call(m, 'sync', { path: 'work/b.txt' }); + assert.equal(await stateOf(m, 'b.txt'), 'synced'); + assert.equal(await contentOf(m, 'b.txt'), 'only on main'); + }); +}); + +describe('a deletion another branch put on disk', () => { + for (const how of ['a sync', 'a path sync'] as const) { + test(`settles this branch's tombstone, and ${how} adopts a new disk file there as new, giving up nothing`, async (t) => { + // P is global: this branch's S (absent) equals P (absent, from the other + // branch's unlink), so the new file is only a disk change. The rule + // decides by P whenever it's known; a tombstone guards only without it. + const env = new Env(t); + const m = await env.open(); + await seedSynced(env, m, 'a.txt', 'v1'); + const main = env.store.currentBranch().name; + env.store.createBranch('other', main); + await call(m, 'delete', { path: 'work/a.txt' }); // main's own deletion, never pushed + env.store.switchBranch('other'); + await call(m, 'delete', { path: 'work/a.txt' }); + await call(m, 'materialize', { applyDeletions: true }); // other removes it from disk: P is absent + assert.equal(existsSync(env.disk('a.txt')), false); + env.writeDisk('a.txt', 'new, from the shell'); + env.store.switchBranch(main); + + const data = await call(m, 'sync', how === 'a sync' ? {} : { path: 'work/a.txt' }); + assert.deepEqual(data.results, [{ mount: 'work', synced: ['a.txt'], conflicts: [] }], 'taken in, with nothing discarded'); + assert.equal(await stateOf(m, 'a.txt'), 'synced'); + assert.equal(await contentOf(m, 'a.txt'), 'new, from the shell'); + }); + } +}); + +describe('workspace deletes', () => { + test('a delete without autoMaterialize is never resurrected, reaches disk only with applyDeletions', async (t) => { + const env = new Env(t); + let m = await env.open(); + await seedSynced(env, m, 'a.txt', 'v1'); + await call(m, 'delete', { path: 'work/a.txt' }); + assert.equal(env.readDisk('a.txt'), 'v1', 'disk keeps its copy'); + + const stillDeleted = async (when: string): Promise => { + assert.equal(await stateOf(m, 'a.txt'), 'workspace-deleted', when); + assert.match(await refused(m, 'read', { path: 'work/a.txt' }), /deleted in the workspace/, `${when}: a lazy read does not restore it`); + }; + await stillDeleted('after ls'); + await call(m, 'sync', {}); + await stillDeleted('after a full sync'); + await (m as any).handleFsChanges('work', [{ path: 'a.txt', op: 'modified' }]); + await stillDeleted('after a watcher event'); + m = await env.restart(m); + await (m as any).initialScan('work'); + await stillDeleted('after a restart and its scan'); + + const plain = await call(m, 'materialize', {}); + assert.deepEqual(plain.workspaceDeletionsLeftOnDisk, [{ mount: 'work', path: 'a.txt' }]); + assert.equal(env.readDisk('a.txt'), 'v1', 'materialize never deletes by default'); + + const applied = await call(m, 'materialize', { applyDeletions: true }); + assert.deepEqual(applied.deleted, [{ mount: 'work', path: 'a.txt' }]); + assert.throws(() => env.readDisk('a.txt'), /ENOENT/); + assert.equal(await stateOf(m, 'a.txt'), 'not listed'); + }); + + test('a path sync cancels a workspace delete', async (t) => { + const env = new Env(t); + const m = await env.open(); + await seedSynced(env, m, 'a.txt', 'v1'); + await call(m, 'delete', { path: 'work/a.txt' }); + await call(m, 'sync', { path: 'work/a.txt' }); + assert.equal(await stateOf(m, 'a.txt'), 'synced'); + assert.equal(await contentOf(m, 'a.txt'), 'v1'); + }); + + test("a deletion whose record can't be read is unverified, never resurrected, until a path sync replaces it", async (t) => { + const env = new Env(t); + let m = await env.open(); + env.legacyEntry('legacy.txt', 'old'); + env.writeDisk('legacy.txt', 'old'); + await call(m, 'delete', { path: 'work/legacy.txt' }); // the tombstone is all that keeps it deleted: no evidence + const record = env.store.treeGet('workspace/work/intent', 'legacy.txt'); + assert.ok(record, 'the tombstone is recorded'); + await env.shutdown(m); + rmSync(join(env.storePath, 'blobs', record.blobHash.slice(0, 2), record.blobHash)); // and its blob is lost + env.store = JsStore.openOrCreate({ path: env.storePath }); + m = await env.open(); + + const e = await entryOf(m, 'legacy.txt'); + assert.equal(e?.state, 'unverified'); + assert.match(String(e?.note), /branch intent can't be read/); + await call(m, 'sync', {}); + assert.equal(env.store.treeGet(TREE, 'legacy.txt'), null, 'not taken back in from disk'); + + await call(m, 'sync', { path: 'work/legacy.txt' }); // the operator's word: take disk's state + assert.equal(await stateOf(m, 'legacy.txt'), 'synced'); + assert.equal(await contentOf(m, 'legacy.txt'), 'old'); + }); + + test('a legacy entry with no baseline, deleted in the workspace, stays deleted', async (t) => { + const env = new Env(t); + const m = await env.open(); + env.writeDisk('legacy.txt', 'old'); + env.legacyEntry('legacy.txt', 'old'); + await call(m, 'delete', { path: 'work/legacy.txt' }); + assert.equal(await stateOf(m, 'legacy.txt'), 'workspace-deleted', 'the tombstone carries the deleted hash'); + await call(m, 'sync', {}); + await (m as any).initialScan('work'); + assert.equal(await stateOf(m, 'legacy.txt'), 'workspace-deleted'); + const applied = await call(m, 'materialize', { applyDeletions: true }); + assert.deepEqual(applied.deleted, [{ mount: 'work', path: 'legacy.txt' }]); + }); + + test('autoMaterialize unlinks under an intent, and refuses when disk changed since', async (t) => { + const env = new Env(t); + const m = await env.open({ autoMaterialize: true }); + await call(m, 'write', { path: 'work/a.txt', content: 'v1' }); + await call(m, 'write', { path: 'work/b.txt', content: 'v1' }); + assert.equal(env.readDisk('a.txt'), 'v1'); + await call(m, 'delete', { path: 'work/a.txt' }); + assert.throws(() => env.readDisk('a.txt'), /ENOENT/, 'unlinked'); + + env.writeDisk('b.txt', 'edited in the shell'); + const error = await refused(m, 'delete', { path: 'work/b.txt' }); + assert.match(error, /did not unlink/); + assert.equal(env.readDisk('b.txt'), 'edited in the shell', 'the shell edit survives'); + }); +}); + +describe('a path sync', () => { + test('takes disk\'s state for a whole directory, and lists each workspace change it discarded', async (t) => { + const env = new Env(t); + const m = await env.open(); + await seedSynced(env, m, 'src/existing.txt', 'v1'); + await call(m, 'write', { path: 'work/src/existing.txt', content: 'a draft' }); + await call(m, 'write', { path: 'work/src/new.txt', content: 'never materialized' }); + await seedSynced(env, m, 'src/gone.txt', 'v1'); + await call(m, 'delete', { path: 'work/src/gone.txt' }); + await seedSynced(env, m, 'src/shell.txt', 'v1'); + env.writeDisk('src/shell.txt', 'edited in the shell'); + + const data = await call(m, 'sync', { path: 'work/src' }); + const result = (data.results as Array<{ mount: string; synced: string[]; discarded?: Array<{ path: string; was: string; op: string }> }>)[0]!; + assert.deepEqual([...result.synced].sort(), ['src/existing.txt', 'src/gone.txt', 'src/new.txt', 'src/shell.txt']); + assert.deepEqual([...(result.discarded ?? [])].sort((a, b) => a.path.localeCompare(b.path)), [ + { path: 'src/existing.txt', was: 'workspace-draft', op: 'modified' }, + { path: 'src/gone.txt', was: 'workspace-deleted', op: 'created' }, + { path: 'src/new.txt', was: 'workspace-draft', op: 'deleted' }, + ], 'a shell edit taken in discards nothing; the rest were the workspace\'s own changes'); + assert.equal(await contentOf(m, 'src/existing.txt'), 'v1'); + assert.equal(await stateOf(m, 'src/new.txt'), 'not listed', 'disk has no such file, so neither does the workspace'); + }); + + test('asked on one branch, adopts nothing on another selected before its turn', async (t) => { + const env = new Env(t); + const m = await env.open(); + await seedSynced(env, m, 'a.txt', 'v1'); + const main = env.store.currentBranch().name; + env.store.createBranch('other', main); + env.store.switchBranch('other'); + await call(m, 'write', { path: 'work/a.txt', content: "other's draft" }); + env.store.switchBranch(main); + env.writeDisk('a.txt', 'from the shell'); + + // Asked on main, queued behind a turn the mount is busy with. + const mount = (m as any).mounts.get('work'); + let release!: () => void; + let busy!: Promise; + await new Promise((started) => { + busy = (m as any).withMount(mount, () => new Promise((r) => { release = r; started(); })); + }); + const syncing = call(m, 'sync', { path: 'work/a.txt' }); + env.store.switchBranch('other'); + release(); + await busy; + const data = await syncing; + assert.match(JSON.stringify(data.incomplete), /the selected branch changed before this sync ran/); + assert.deepEqual(data.results, []); + assert.equal(await contentOf(m, 'a.txt'), "other's draft", "other's draft is untouched"); + env.store.switchBranch(main); + assert.equal(await contentOf(m, 'a.txt'), 'v1', 'and main took nothing either'); + }); + + test("lists a deletion it gives up for a disk copy the workspace can't hold, though no workspace entry changes", async (t) => { + const env = new Env(t); + const m = await env.open(); + await seedSynced(env, m, 'a.txt', 'text'); + await call(m, 'delete', { path: 'work/a.txt' }); + env.writeDisk('a.txt', ONE_PX_PNG); // binary: never stored + const data = await call(m, 'sync', { path: 'work/a.txt' }); + assert.deepEqual(data.results, [{ mount: 'work', synced: [], conflicts: [], discarded: [{ path: 'a.txt', was: 'conflict' }] }], + 'the workspace deletion (a conflict once disk changed) is given up, with no op: the workspace copy was already gone'); + assert.equal(await stateOf(m, 'a.txt'), 'disk-only'); + }); +}); + +describe('disk counterparts that are not stored', () => { + test('a binary counterpart keeps the draft and a disk reference that reports a later change', async (t) => { + const env = new Env(t); + const m = await env.open(); + await seedSynced(env, m, 'a.dat', 'text v1'); + env.writeDisk('a.dat', Buffer.from([1, 0, 2, 0, 3])); + await call(m, 'write', { path: 'work/a.dat', content: 'text v2' }); + + let e = await entryOf(m, 'a.dat'); + assert.equal(e?.state, 'conflict'); + assert.equal(e?.conflict?.diskCopy, 'referenced', 'binary: referenced, not stored'); + assert.equal(e?.conflict?.diskChangedSinceRecorded, false); + + env.writeDisk('a.dat', Buffer.from([9, 0, 9, 0])); + e = await entryOf(m, 'a.dat'); + assert.equal(e?.state, 'conflict'); + assert.equal(e?.conflict?.diskChangedSinceRecorded, true, 'the reference says disk changed since'); + assert.equal(await contentOf(m, 'a.dat'), 'text v2', 'the draft is kept'); + await call(m, 'materialize', {}); + assert.deepEqual([...readFileSync(env.disk('a.dat'))], [9, 0, 9, 0], 'and nothing overwrote the disk file'); + }); + + test('an oversize counterpart is referenced too', async (t) => { + const env = new Env(t); + const m = await env.open({ maxFileSize: 64 }); + await seedSynced(env, m, 'a.txt', 'small'); + env.writeDisk('a.txt', 'x'.repeat(200)); + await call(m, 'write', { path: 'work/a.txt', content: 'small v2' }); + const e = await entryOf(m, 'a.txt'); + assert.equal(e?.state, 'conflict'); + assert.equal(e?.conflict?.diskCopy, 'referenced'); + }); +}); + +describe('disk-only files', () => { + test('a shell-written PNG is listed disk-only with size and type, and read_image opens it', async (t) => { + const env = new Env(t); + const m = await env.open(); + env.writeDisk('shot.png', ONE_PX_PNG); + const data = await call(m, 'ls', { path: 'work' }); + const png = (data.entries as Array>).find((e) => e.name === 'shot.png'); + assert.deepEqual(png, { name: 'shot.png', type: 'file', state: 'disk-only', size: ONE_PX_PNG.length, mimeType: 'image/png' }); + const glob = await call(m, 'glob', { pattern: '*.png' }); + assert.deepEqual(glob.matches, [{ path: 'work/shot.png', state: 'disk-only', size: ONE_PX_PNG.length, mimeType: 'image/png' }]); + const res = await m.handleToolCall({ id: 't', name: 'read_image', input: { path: 'work/shot.png' } }); + assert.equal(res.success, true, `read_image: ${res.error}`); + }); + + test('an untracked file over the size limit is listed from its head, never read whole or taken as evidence', async (t) => { + const env = new Env(t); + const m = await env.open({ maxFileSize: 1024 }); + const big = Buffer.concat([ONE_PX_PNG, Buffer.alloc(256 * 1024, 1)]); + env.writeDisk('big.png', big); + + const probe = await open(join(env.root, 'read-probe'), 'w'); + const proto = Object.getPrototypeOf(probe) as { read: (...args: unknown[]) => Promise<{ bytesRead: number }> }; + await probe.close(); + const read = proto.read; + let bytes = 0; + proto.read = async function (this: unknown, ...args: unknown[]) { + const result = await read.apply(this, args); + bytes += result.bytesRead; + return result; + }; + let entry; + try { + entry = (await call(m, 'ls', { path: 'work' })).entries.find((e: { name: string }) => e.name === 'big.png'); + } finally { + proto.read = read; + } + assert.deepEqual(entry, { name: 'big.png', type: 'file', state: 'disk-only', size: big.length, mimeType: 'image/png' }); + assert.ok(bytes <= 8192, `the listing read ${bytes} bytes of a ${big.length}-byte file`); + assert.equal((m as any).agreement.get('work', 'big.png'), undefined, 'no evidence recorded'); + }); +}); + +describe('shell deletions', () => { + const mechanisms: Array<[string, (env: Env, m: WorkspaceModule) => Promise]> = [ + ['a watcher event', async (_env, m) => { await (m as any).handleFsChanges('work', [{ path: 'gone.txt', op: 'deleted' }]); return m; }], + ['a full sync', async (_env, m) => { await call(m, 'sync', {}); return m; }], + ['sync without a path, on the mount', async (_env, m) => { await call(m, 'sync', { mount: 'work' }); return m; }], + ['a listing', async (_env, m) => { await call(m, 'ls', { path: 'work' }); return m; }], + ['the post-restart scan', async (env, m) => { const r = await env.restart(m); await (r as any).initialScan('work'); return r; }], + ['a reattach', async (_env, m) => { await (m as any).initialScan('work'); return m; }], + ]; + for (const [name, run] of mechanisms) { + test(`a shell rm of a synced file is removed by ${name}`, async (t) => { + const env = new Env(t); + let m = await env.open(); + await seedSynced(env, m, 'gone.txt', 'v1'); + env.rmDisk('gone.txt'); + m = await run(env, m); + assert.equal(env.store.treeGet(TREE, 'gone.txt'), null, 'the tree entry is removed'); + assert.match(await refused(m, 'read', { path: 'work/gone.txt' }), /File not found/); + assert.ok(env.eventsOf('workspace:deleted').some((e) => e.paths.includes('work/gone.txt')), 'a deleted event was pushed'); + }); + } + + test('a shell rm of a file with a newer unmaterialized edit is kept as a conflict', async (t) => { + const env = new Env(t); + const m = await env.open(); + await seedSynced(env, m, 'a.txt', 'v1'); + await call(m, 'write', { path: 'work/a.txt', content: 'v2, not materialized' }); + env.rmDisk('a.txt'); + await (m as any).handleFsChanges('work', [{ path: 'a.txt', op: 'deleted' }]); + const deleted = env.eventsOf('workspace:deleted').at(-1); + assert.deepEqual(deleted?.conflicts, ['work/a.txt'], 'the event names the conflict'); + + const e = await entryOf(m, 'a.txt'); + assert.equal(e?.state, 'conflict'); + assert.equal(e?.conflict?.kind, 'deleted-on-disk'); + assert.equal(e?.conflict?.diskCopy, 'absent'); + await call(m, 'sync', {}); + assert.equal(await contentOf(m, 'a.txt'), 'v2, not materialized', 'the edit survives a full sync'); + assert.equal((await call(m, 'materialize', {})).materialized.length, 0, 'refused without force'); + await call(m, 'materialize', { force: true }); + assert.equal(env.readDisk('a.txt'), 'v2, not materialized'); + assert.equal(await stateOf(m, 'a.txt'), 'synced'); + }); +}); + +describe('provenance', () => { + test('store-origin drafts are distinguished from legacy unknowns', async (t) => { + const env = new Env(t); + const m = await env.open(); + await call(m, 'write', { path: 'work/draft.txt', content: 'never materialized' }); + env.legacyEntry('legacy-missing.txt', 'store only'); + env.legacyEntry('legacy-same.txt', 'same'); + env.writeDisk('legacy-same.txt', 'same'); + env.legacyEntry('legacy-diff.txt', 'store side'); + env.writeDisk('legacy-diff.txt', 'disk side'); + await call(m, 'write', { path: 'work/collide.txt', content: 'from the agent' }); + env.writeDisk('collide.txt', 'from the shell'); + + const entries = await listing(m); + assert.equal(entries.get('draft.txt')?.state, 'workspace-draft'); + assert.equal(entries.get('legacy-missing.txt')?.state, 'disk-missing-provenance-unknown'); + assert.equal(entries.get('legacy-same.txt')?.state, 'synced'); + assert.equal(entries.get('legacy-diff.txt')?.state, 'conflict'); + assert.equal(entries.get('legacy-diff.txt')?.conflict?.kind, 'unknown-provenance'); + assert.equal(entries.get('collide.txt')?.state, 'conflict'); + assert.equal(entries.get('collide.txt')?.conflict?.kind, 'store-origin-collision'); + assert.equal(await contentOf(m, 'legacy-diff.txt'), 'store side', 'a legacy mismatch keeps the store side'); + + const materialized = await call(m, 'materialize', {}); + const written = (materialized.materialized as Array<{ path: string }>).map((w) => w.path).sort(); + assert.deepEqual(written, ['draft.txt'], 'drafts are written; what has no evidence is left for a named call'); + assert.equal(existsSync(env.disk('legacy-missing.txt')), false, 'a shell rm of a pre-evidence file is not undone'); + // With no evidence there was no agreement for disk to have changed since, + // and nothing says whether a missing copy was deleted or never written. + const reasons = (materialized.skipped as Array<{ reason: string }>).map((s) => s.reason).sort(); + assert.deepEqual(reasons, [ + 'collide.txt: disk holds a different copy, and nothing records which is newer (conflict: store-origin-collision) — ' + + 'sync this path to adopt the disk version, or materialize this path with force to overwrite it', + 'legacy-diff.txt: disk holds a different copy, and nothing records which is newer (conflict: unknown-provenance) — ' + + 'sync this path to adopt the disk version, or materialize this path with force to overwrite it', + 'legacy-missing.txt: disk has no copy, and nothing records whether it was deleted there or never written — ' + + 'sync this path to drop it from the workspace, or materialize this path to write it', + ]); + + // Both remedies do what they say. + const named = await call(m, 'materialize', { path: 'work/legacy-missing.txt' }); + assert.deepEqual(named.materialized.map((w: { path: string }) => w.path), ['legacy-missing.txt']); + assert.equal(env.readDisk('legacy-missing.txt'), 'store only'); + env.legacyEntry('legacy-gone.txt', 'deleted in a shell'); + const synced = await call(m, 'sync', { path: 'work/legacy-gone.txt' }); + assert.deepEqual(synced.results[0].discarded, + [{ path: 'legacy-gone.txt', was: 'disk-missing-provenance-unknown', op: 'deleted' }]); + assert.equal(env.store.treeGet(TREE, 'legacy-gone.txt'), null, 'the path sync dropped it from the workspace'); + }); +}); + +describe('restarts and faults', () => { + test('baselines survive a restart', async (t) => { + const env = new Env(t); + let m = await env.open(); + await seedSynced(env, m, 'a.txt', 'v1'); + m = await env.restart(m); + env.writeDisk('a.txt', 'v3, edited after the restart'); + assert.equal(await stateOf(m, 'a.txt'), 'synced', 'P survived: a disk-only change, not unknown provenance'); + assert.equal(await contentOf(m, 'a.txt'), 'v3, edited after the restart'); + }); + + for (const killAt of ['after-intent', 'after-effect'] as const) { + test(`a materialize killed ${killAt}, then branch selection`, async (t) => { + const env = new Env(t); + let m = await env.open(); + await seedSynced(env, m, 'a.txt', 'v1'); + const main = env.store.currentBranch().name; + env.store.createBranch('before-v2', main); + await call(m, 'write', { path: 'work/a.txt', content: 'v2' }); + m = await env.fault(m, 'materialize', killAt); + assert.equal(env.readDisk('a.txt'), killAt === 'after-intent' ? 'v1' : 'v2'); + + env.store.switchBranch('before-v2'); + assert.equal(await stateOf(m, 'a.txt'), killAt === 'after-intent' ? 'synced' : 'workspace-draft', + 'the earlier branch reads disk by the resolved intent, never as a disk edit'); + env.store.switchBranch(main); + if (killAt === 'after-intent') { + assert.equal(await stateOf(m, 'a.txt'), 'workspace-draft', 'the write never happened: still owed'); + await call(m, 'materialize', {}); + assert.equal(env.readDisk('a.txt'), 'v2'); + } + assert.equal(await stateOf(m, 'a.txt'), 'synced'); + if (killAt === 'after-effect') { + // Disk shows the write, but nothing proves the dead process made it + // durable: listings record nothing, and the next push completes it. + assert.equal((m as any).agreement.get('work', 'a.txt').kind, 'pending'); + await call(m, 'materialize', {}); + assert.equal((m as any).agreement.get('work', 'a.txt').kind, 'content'); + } + }); + + test(`a shell-edit ingest killed ${killAt}, then branch selection`, async (t) => { + const env = new Env(t); + let m = await env.open(); + await seedSynced(env, m, 'a.txt', 'v1'); + const main = env.store.currentBranch().name; + env.store.createBranch('before-edit', main); + env.writeDisk('a.txt', 'v3, from the shell'); + m = await env.fault(m, 'sync', killAt); + + env.store.switchBranch('before-edit'); + const e = await entryOf(m, 'a.txt'); + assert.equal(e?.state, 'conflict', 'disk may hold the only copy of the edit: listed, not pushed over'); + assert.equal(e?.conflict?.kind, 'interrupted'); + await call(m, 'materialize', { force: false }); + assert.equal(env.readDisk('a.txt'), 'v3, from the shell'); + + env.store.switchBranch(main); + assert.equal(await stateOf(m, 'a.txt'), 'synced', 'on its own branch the adoption completes or is redone'); + assert.equal(await contentOf(m, 'a.txt'), 'v3, from the shell'); + }); + + test(`an autoMaterialize write of a new file killed ${killAt}, then a branch without it`, async (t) => { + const env = new Env(t); + let m = await env.open(); + await seedSynced(env, m, 'other.txt', 'x'); + const main = env.store.currentBranch().name; + env.store.createBranch('without-new', main); + m = await env.fault(m, 'write', killAt); + + env.store.switchBranch('without-new'); + assert.equal(await stateOf(m, 'new.txt'), killAt === 'after-intent' ? 'not listed' : 'not-in-branch', + 'never ingested into a branch that lacks it'); + await call(m, 'sync', {}); + assert.equal(env.store.treeGet(TREE, 'new.txt'), null); + + env.store.switchBranch(main); + assert.equal(await stateOf(m, 'new.txt'), killAt === 'after-intent' ? 'workspace-draft' : 'synced'); + assert.equal(await contentOf(m, 'new.txt'), 'new file'); + }); + } + + /** S = P = D = A, with an adoption of the shell's B left pending (killed after its intent). */ + async function pendingAdoption(env: Env): Promise { + let m = await env.open(); + await seedSynced(env, m, 'a.txt', 'A'); + env.writeDisk('a.txt', 'B'); + m = await env.fault(m, 'sync', 'after-intent'); + assert.equal((m as any).agreement.get('work', 'a.txt').kind, 'pending'); + return m; + } + + test("this branch agreeing with disk leaves another branch's pending adoption for that branch to resolve", async (t) => { + const env = new Env(t); + let m = await env.open(); + await seedSynced(env, m, 'a.txt', 'A'); + const main = env.store.currentBranch().name; + env.store.createBranch('other', main); + env.writeDisk('a.txt', 'B'); + m = await env.fault(m, 'sync', 'after-intent'); // main's adoption of B, pending + assert.equal((m as any).agreement.get('work', 'a.txt').kind, 'pending'); + + env.store.switchBranch('other'); + await call(m, 'write', { path: 'work/a.txt', content: 'B' }); + assert.equal(await stateOf(m, 'a.txt'), 'synced', 'other agrees with disk'); + assert.equal((m as any).agreement.get('work', 'a.txt').kind, 'pending', "and leaves main's adoption pending"); + + env.store.switchBranch(main); + assert.equal(await stateOf(m, 'a.txt'), 'synced', 'main resolves its own adoption'); + assert.equal(await contentOf(m, 'a.txt'), 'B'); + }); + + test('an adoption intended over a pending one carries the evidence that settled it, so a crash before it lands redoes it', async (t) => { + const env = new Env(t); + let m = await pendingAdoption(env); // S = A, an adoption of B pending + env.writeDisk('a.txt', 'C'); + m = await env.fault(m, 'sync', 'after-intent'); // settles B's (S is A) and intends C + const p = (m as any).agreement.get('work', 'a.txt'); + assert.equal(p.kind, 'pending'); + assert.deepEqual(p.prior && { kind: p.prior.kind, hash: p.prior.hash }, { kind: 'content', hash: sha('A') }, + "its prior is A, as the settled adoption left it, not the pending record's"); + assert.equal(await stateOf(m, 'a.txt'), 'synced', 'from A, the adoption of C is redone'); + assert.equal(await contentOf(m, 'a.txt'), 'C'); + }); + + test('a write of the pending adoption\'s content is not taken for the old adoption landing', async (t) => { + const env = new Env(t); + const m = await pendingAdoption(env); + env.writeDisk('a.txt', 'C'); // disk moves on + await call(m, 'write', { path: 'work/a.txt', content: 'B' }); + assert.equal(await stateOf(m, 'a.txt'), 'workspace-draft', 'the write is a draft over disk C'); + assert.equal(await contentOf(m, 'a.txt'), 'B', 'and it survives the scan'); + assert.equal(env.readDisk('a.txt'), 'C'); + }); + + test('a path sync keeps a stored binary file that disk agrees with', async (t) => { + const env = new Env(t); + const m = await env.open(); + const res = await m.writeBinary('work/img/shot.png', ONE_PX_PNG, 'image/png'); + assert.equal(res.success, true, String(res.error)); + await call(m, 'materialize', { path: 'work/img/shot.png' }); + assert.deepEqual(readFileSync(env.disk('img/shot.png')), ONE_PX_PNG); + const stored = env.store.treeGet(TREE, 'img/shot.png'); + assert.notEqual(stored, null); + + const synced = await call(m, 'sync', { path: 'work/img' }); + assert.deepEqual(synced.results ?? [], [], 'nothing to take from disk'); + assert.deepEqual(env.store.treeGet(TREE, 'img/shot.png'), stored, 'still held, unchanged'); + assert.equal(await stateOf(m, 'img/shot.png'), 'synced'); + }); + + test('writeBinary settles the pending adoption first too', async (t) => { + const env = new Env(t); + const m = await pendingAdoption(env); + env.writeDisk('a.txt', 'C'); + const res = await m.writeBinary('work/a.txt', Buffer.from('B'), 'text/plain'); + assert.equal(res.success, true, String(res.error)); + assert.equal(await stateOf(m, 'a.txt'), 'workspace-draft'); + assert.equal(await contentOf(m, 'a.txt'), 'B'); + }); + + test('an edit sees the disk version the settled adoption takes in, not stale text', async (t) => { + const env = new Env(t); + const m = await pendingAdoption(env); + env.writeDisk('a.txt', 'C'); + const error = await refused(m, 'edit', { path: 'work/a.txt', oldString: 'A', newString: 'B' }); + assert.match(error, /String not found/, 'the entry is disk\'s C by then, so an edit of A has nothing to replace'); + assert.equal(await contentOf(m, 'a.txt'), 'C'); + assert.equal(await stateOf(m, 'a.txt'), 'synced'); + }); + + test('a delete settles the pending adoption first, and deletes what the workspace then holds', async (t) => { + const env = new Env(t); + const m = await pendingAdoption(env); + env.writeDisk('a.txt', 'C'); + await call(m, 'delete', { path: 'work/a.txt' }); + assert.equal(await stateOf(m, 'a.txt'), 'workspace-deleted'); + assert.equal(env.readDisk('a.txt'), 'C'); + }); + + test('a write commits only once a pending adoption on its branch is settled, even if the scan could not settle it', async (t) => { + const env = new Env(t); + const m = await pendingAdoption(env); // pending on main + const main = env.store.currentBranch().name; + env.store.createBranch('other', main); + env.writeDisk('a.txt', 'C'); + env.store.switchBranch('other'); // the write starts on another branch + // Each of the settle pass's three gathers sees the branch change under it, + // so it decides nothing and returns incomplete; the write commits on main. + await withReadHook((n) => env.store.switchBranch(n % 2 === 1 ? main : 'other'), + () => call(m, 'write', { path: 'work/a.txt', content: 'B' })); + assert.equal(env.store.currentBranch().name, main); + assert.notEqual((m as any).agreement.get('work', 'a.txt').kind, 'pending', 'settled at the commit'); + const e = await entryOf(m, 'a.txt'); + assert.equal(e?.state, 'conflict', 'B (the write) and C (disk) both changed since A'); + assert.equal(await contentOf(m, 'a.txt'), 'B', 'the write survives'); + }); + + test('a failed barrier at the mutation precondition stays owed, and the next mutation discharges it first', async (t) => { + const env = new Env(t); + const m = await pendingAdoption(env); + const agreement = (m as any).agreement as DiskAgreement; + const mount = { name: 'work', treeStateId: TREE }; + const set = DiskAgreement.prototype.set; + const storeSync = env.store.sync; + let armed = false; + let syncs = 0; + DiskAgreement.prototype.set = function (this: DiskAgreement, ...args: Parameters) { + set.apply(this, args); + armed = true; // the next sync is the barrier after the settlement + }; + (env.store as unknown as { sync: () => void }).sync = function () { + syncs++; + if (armed) { + armed = false; + throw Object.assign(new Error('injected EIO at the precondition barrier'), { code: 'EIO' }); + } + storeSync.call(env.store); + }; + try { + assert.throws(() => settleAdoptionBeforeMutation(env.store, agreement, mount, 'a.txt'), /precondition barrier/); + DiskAgreement.prototype.set = set; + assert.equal(agreement.get('work', 'a.txt')?.kind, 'content', 'settled in memory'); + assert.equal(agreement.needsBarrier, true, 'and still owed a barrier'); + syncs = 0; + settleAdoptionBeforeMutation(env.store, agreement, mount, 'a.txt'); // P no longer looks pending + assert.ok(syncs >= 1, 'the retry discharges the barrier before permitting the mutation'); + assert.equal(agreement.needsBarrier, false); + } finally { + DiskAgreement.prototype.set = set; + (env.store as unknown as { sync: () => void }).sync = storeSync; + } + await call(m, 'write', { path: 'work/a.txt', content: 'B' }); + }); + + test('evidence loaded at open owes a barrier: reading records back proves nothing about their durability', (t) => { + const env = new Env(t); + const writer = new DiskAgreement(env.store); + writer.open([{ name: 'work', root: '/srv/work' }]); + writer.set('work', 'a.txt', { kind: 'content', hash: 'h', size: 1 }); // no barrier after it + const reader = new DiskAgreement(env.store); + reader.open([{ name: 'work', root: '/srv/work' }]); + assert.deepEqual(reader.get('work', 'a.txt'), { kind: 'content', hash: 'h', size: 1 }); + assert.equal(reader.needsBarrier, true); + reader.barrier(); + assert.equal(reader.needsBarrier, false); + }); + + test('evidence is read only after an ambiguous append is reconciled', (t) => { + const env = new Env(t); + let failSync = false; + const flaky = new Proxy(env.store, { + get(target, prop) { + if (prop === 'sync') { + return () => { + if (failSync) { + failSync = false; + throw Object.assign(new Error('injected EIO after the append'), { code: 'EIO' }); + } + target.sync(); + }; + } + const value = (target as unknown as Record)[prop]; + return typeof value === 'function' ? (value as (...a: unknown[]) => unknown).bind(target) : value; + }, + }); + const agreement = new DiskAgreement(flaky as JsStore); + agreement.open([{ name: 'work', root: '/srv/work' }]); + failSync = true; + assert.throws(() => agreement.intend('work', 'a.txt', { + effect: 'adopt', prior: null, expect: { kind: 'content', hash: 'h' }, branchId: 'b', + }, { durable: true }), /injected EIO/); + assert.equal(agreement.get('work', 'a.txt')?.kind, 'pending', 'the append that landed is read back, not the stale view'); + assert.equal(agreement.needsBarrier, true); + }); + + test('an append that fails inside the store leaves no decision to stale evidence: passes refuse until a reopen, which finds the evidence', async (t) => { + const env = new Env(t); + let m = await env.open(); + await call(m, 'write', { path: 'work/a.txt', content: 'A' }); + await call(m, 'materialize', { path: 'work/a.txt' }); + const main = env.store.currentBranch().name; + env.store.createBranch('draft-c', main); + env.store.switchBranch('draft-c'); + await call(m, 'write', { path: 'work/a.txt', content: 'C' }); + env.store.switchBranch(main); + await call(m, 'write', { path: 'work/a.txt', content: 'B' }); + env.writeDisk('a.txt', 'B'); + + // The pass that agrees at B appends that evidence, and the append throws + // after it landed, as Chronicle's does when the log's periodic sync fails. + const sha = (text: string) => createHash('sha256').update(text).digest('hex'); + const store = env.store as unknown as { appendJson: (type: string, payload: any) => unknown }; + const append = store.appendJson; + let fired = false; + store.appendJson = function (this: unknown, type: string, payload: any) { + const record = append.call(this, type, payload); + if (!fired && type === 'workspace/disk-agreement' && payload.t === 'set' && payload.value?.hash === sha('B')) { + fired = true; + throw new Error('injected: the append landed, then the store failed'); + } + return record; + }; + try { + assert.match(await refused(m, 'ls', { path: 'work', recursive: true }), /injected: the append landed/); + } finally { + store.appendJson = append; + } + assert.equal(fired, true); + + // Its evidence would still say A, so the draft branch's listing would read + // C against A on disk B as both-changed and record a sticky conflict. It + // refuses instead, and records nothing. + env.store.switchBranch('draft-c'); + assert.match(await refused(m, 'ls', { path: 'work', recursive: true }), /evidence is unavailable until the store is reopened/); + assert.match(await refused(m, 'materialize', { path: 'work/a.txt' }), /evidence is unavailable until the store is reopened/); + const intents = (m as unknown as { intents: Map }).intents; + assert.equal(intents.get('work')!.get('a.txt')?.conflict, undefined, 'no conflict was recorded'); + + // Reopened, the store has the evidence at B, and the draft is a draft. + m = await env.restart(m); + env.store.switchBranch('draft-c'); + const entry = (await call(m, 'ls', { path: 'work', recursive: true })).entries.find((e: { path: string }) => e.path === 'a.txt'); + assert.equal(entry.state, 'workspace-draft'); + assert.equal(readFileSync(join(env.dir, 'a.txt'), 'utf8'), 'B', 'disk is untouched'); + }); + + test('a checkpoint check after an ambiguous append reconciles first, like every entry point', (t) => { + const env = new Env(t); + let failSync = false; + const flaky = new Proxy(env.store, { + get(target, prop) { + if (prop === 'sync') { + return () => { + if (failSync) { + failSync = false; + throw Object.assign(new Error('injected EIO after the append'), { code: 'EIO' }); + } + target.sync(); + }; + } + const value = (target as unknown as Record)[prop]; + return typeof value === 'function' ? (value as (...a: unknown[]) => unknown).bind(target) : value; + }, + }); + const agreement = new DiskAgreement(flaky as JsStore); + agreement.open([{ name: 'work', root: '/srv/work' }]); + failSync = true; + assert.throws(() => agreement.intend('work', 'a.txt', { + effect: 'adopt', prior: null, expect: { kind: 'content', hash: 'h' }, branchId: 'b', + }, { durable: true }), /injected EIO/); + const journal = (agreement as unknown as { journal: { needsReconcile: boolean } }).journal; + assert.equal(journal.needsReconcile, true); + agreement.maybeCheckpoint(); + assert.equal(journal.needsReconcile, false, 'read back before deciding whether to checkpoint'); + }); + + test('with disk unreadable at the write, the pending adoption settles by the tree', { skip: IS_ROOT ? 'root reads every file' : false }, async (t) => { + const env = new Env(t); + const m = await pendingAdoption(env); + env.writeDisk('a.txt', 'C'); + chmodSync(env.disk('a.txt'), 0o000); + try { + await call(m, 'write', { path: 'work/a.txt', content: 'B' }); + } finally { + chmodSync(env.disk('a.txt'), 0o644); + } + const e = await entryOf(m, 'a.txt'); + assert.equal(e?.state, 'conflict', 'disk C and the write B both changed since A: kept apart'); + assert.equal(await contentOf(m, 'a.txt'), 'B'); + }); + + test('a failed agreement barrier stays owed: the next pass settles it before deciding', async (t) => { + const env = new Env(t); + const m = await env.open(); + await seedSynced(env, m, 'a.txt', 'A'); + await call(m, 'write', { path: 'work/a.txt', content: 'B' }); + env.writeDisk('a.txt', 'B'); // D = S: an agreement, with no intent behind it + const set = DiskAgreement.prototype.set; + const storeSync = env.store.sync; + let armed = false; + let syncs = 0; + DiskAgreement.prototype.set = function (this: DiskAgreement, ...args: Parameters) { + set.apply(this, args); + armed = true; // the next sync is the barrier that should make it durable + }; + (env.store as unknown as { sync: () => void }).sync = function () { + syncs++; + if (armed) { + armed = false; + throw Object.assign(new Error('injected EIO at the barrier'), { code: 'EIO' }); + } + storeSync.call(env.store); + }; + try { + assert.match(await refused(m, 'ls', { path: 'work' }), /injected EIO at the barrier/); + assert.equal((m as any).agreement.needsBarrier, true, 'the obligation survives the failed pass'); + DiskAgreement.prototype.set = set; + syncs = 0; + await call(m, 'ls', { path: 'work' }); // nothing new to record + assert.ok(syncs >= 1, 'the next pass still syncs'); + assert.equal((m as any).agreement.needsBarrier, false); + } finally { + DiskAgreement.prototype.set = set; + (env.store as unknown as { sync: () => void }).sync = storeSync; + } + }); + + for (const trigger of ['a scan', 'a materialize'] as const) { + test(`an agreement advance is durable before ${trigger} returns`, async (t) => { + const env = new Env(t); + const m = await env.open(); + await seedSynced(env, m, 'a.txt', 'A'); + await call(m, 'write', { path: 'work/a.txt', content: 'B' }); + env.writeDisk('a.txt', 'B'); // D = S: agreement, with no intent behind it + const events: string[] = []; + const set = DiskAgreement.prototype.set; + const storeSync = env.store.sync; + DiskAgreement.prototype.set = function (this: DiskAgreement, ...args: Parameters) { + set.apply(this, args); + events.push(`set ${args[1]}`); + }; + (env.store as unknown as { sync: () => void }).sync = function () { + events.push('sync'); + storeSync.call(env.store); + }; + try { + if (trigger === 'a scan') await call(m, 'ls', { path: 'work' }); + else await call(m, 'materialize', {}); + } finally { + DiskAgreement.prototype.set = set; + (env.store as unknown as { sync: () => void }).sync = storeSync; + } + const last = events.lastIndexOf('set a.txt'); + assert.ok(last >= 0, `the agreement was recorded (${events.join(', ')})`); + assert.ok(events.indexOf('sync', last + 1) > last, `a barrier follows it (${events.join(', ')})`); + }); + } + + test('a push completed after a branch switch settles only the branch it planned on', async (t) => { + const env = new Env(t); + const m = await env.open(); + await call(m, 'write', { path: 'work/a.txt', content: 'A' }); // a draft disk never had + const main = env.store.currentBranch().name; + env.store.createBranch('other', main); + env.store.switchBranch('other'); + await call(m, 'write', { path: 'work/a.txt', content: 'B' }); + env.writeDisk('a.txt', 'from the shell'); + assert.equal(await stateOf(m, 'a.txt'), 'conflict'); // other's own intent: a recorded conflict + env.rmDisk('a.txt'); + const intents = (m as any).intents.get('work'); + const otherBefore = intents.get('a.txt'); + env.store.switchBranch(main); + + const mount = (m as any).mounts.get('work'); + const runtime = await (m as any).runtime(mount); + const pushed = await pushPaths(env.store, (m as any).agreement, runtime, ['a.txt'], { + beforeEffect: () => queueMicrotask(() => env.store.switchBranch('other')), + }); + assert.equal(env.store.currentBranch().name, 'other', 'the switch landed while the push wrote'); + assert.deepEqual(pushed.written, ['a.txt']); + assert.equal(env.readDisk('a.txt'), 'A'); + assert.deepEqual(intents.get('a.txt'), otherBefore, "the other branch's intent is untouched"); + assert.equal((m as any).agreement.get('work', 'a.txt').kind, 'content', 'the physical evidence is updated'); + + env.store.switchBranch(main); + assert.equal(await stateOf(m, 'a.txt'), 'synced', 'its own branch settles by convergence'); + assert.equal(intents.get('a.txt'), null); + }); + + test("a push whose root is replaced while it observes records nothing it saw there", async (t) => { + const env = new Env(t); + const m = await env.open(); + await call(m, 'write', { path: 'work/a.txt', content: 'A' }); + await call(m, 'materialize', { path: 'work/a.txt' }); + await call(m, 'write', { path: 'work/a.txt', content: 'C' }); + const agreement = (m as any).agreement; + const agreed = agreement.get('work', 'a.txt'); + const rootReal = realpathSync(env.dir); + const away = `${env.dir}-away`; + const mount = (m as any).mounts.get('work'); + const runtime = await (m as any).runtime(mount); + let replaced = false; + // Once the push has checked the root, it's replaced by one holding the + // workspace's draft, which the push then observes. + const pushed = await withFsPromises('lstat', (lstat) => async (path: unknown, ...rest: unknown[]) => { + if (!replaced && (String(path) === join(env.dir, 'a.txt') || String(path) === join(rootReal, 'a.txt'))) { + replaced = true; + renameSync(env.dir, away); + mkdirSync(env.dir); + writeFileSync(join(env.dir, 'a.txt'), 'C'); + } + return lstat(path, ...rest); + }, () => pushPaths(env.store, agreement, runtime, ['a.txt'], {})); + assert.equal(replaced, true); + assert.deepEqual(pushed.written, []); + assert.match(JSON.stringify(pushed.skipped), /not the directory disk last agreed with/); + assert.deepEqual(agreement.get('work', 'a.txt'), agreed, "the replacement's copy isn't recorded as agreed"); + assert.equal(readFileSync(join(away, 'a.txt'), 'utf8'), 'A'); + }); + + test('a push whose write is refused untouched puts back the evidence its own planning settled, not the stale intent', async (t) => { + if (IS_ROOT) return t.skip('root writes through file permissions'); + const env = new Env(t); + const m = await env.open(); + await call(m, 'write', { path: 'work/a.txt', content: 'A' }); + await call(m, 'materialize', { path: 'work/a.txt' }); + await call(m, 'write', { path: 'work/a.txt', content: 'C' }); + const agreement = (m as any).agreement; + const agreed = agreement.get('work', 'a.txt'); + // An earlier push of C announced its write and stopped before the effect: + // disk still holds A, which this push's planning resolves the intent to. + agreement.intend('work', 'a.txt', { + effect: 'disk', prior: agreed, expect: { kind: 'content', hash: sha('C') }, branchId: env.store.currentBranch().id, + }, { durable: true }); + assert.equal(agreement.get('work', 'a.txt').kind, 'pending'); + const mount = (m as any).mounts.get('work'); + const runtime = await (m as any).runtime(mount); + let pushed; + try { + pushed = await pushPaths(env.store, agreement, runtime, ['a.txt'], { + beforeEffect: () => chmodSync(env.disk('a.txt'), 0o444), + }); + } finally { + chmodSync(env.disk('a.txt'), 0o644); + } + assert.deepEqual(pushed.written, []); + assert.match(JSON.stringify(pushed.skipped), /not written/); + const now = agreement.get('work', 'a.txt'); + assert.equal(now.kind, 'content', `the resolved evidence, not the stale intent: ${JSON.stringify(now)}`); + assert.equal(now.hash, agreed.hash); + assert.equal(env.readDisk('a.txt'), 'A'); + }); + + test('a failed durability barrier is reported, and its intent stays pending', async (t) => { + const env = new Env(t); + const m = await env.open(); + await call(m, 'write', { path: 'work/a.txt', content: 'v1' }); + const probe = await open(join(env.root, 'probe'), 'w'); + const proto = Object.getPrototypeOf(probe) as { sync: () => Promise }; + await probe.close(); + const sync = proto.sync; + let calls = 0; + proto.sync = async function () { + calls++; + throw Object.assign(new Error('injected EIO'), { code: 'EIO' }); + }; + let res; + try { + res = await call(m, 'materialize', {}); + } finally { + proto.sync = sync; + } + assert.equal(calls, 1, 'the file barrier failed; nothing after it ran'); + assert.deepEqual(res.materialized, []); + assert.match(res.skipped[0].reason, /a\.txt: could not make the write durable \(file fsync\): injected EIO/); + assert.equal((m as any).agreement.get('work', 'a.txt').kind, 'pending', 'no completion was recorded'); + const entry = await entryOf(m, 'a.txt'); + assert.equal(entry?.state, 'synced', 'the next observation decides from disk'); + assert.match(entry?.note ?? '', /materialize that has not completed/); + assert.equal((m as any).agreement.get('work', 'a.txt').kind, 'pending', 'and records nothing: only a push completes it'); + }); + + test('a write into new directories syncs every directory up to the mount root before it completes', async (t) => { + const env = new Env(t); + const m = await env.open(); + await call(m, 'write', { path: 'work/a/b/c.txt', content: 'deep' }); + const { result, synced } = await withSyncs(env, () => false, () => call(m, 'materialize', {})); + assert.deepEqual(result.materialized, [{ mount: 'work', path: 'a/b/c.txt' }]); + for (const dir of ['a/b', 'a', '']) assert.ok(synced.includes(idOf(env.disk(dir))), `${dir || 'the mount root'} was synced`); + assert.ok(synced.includes(idOf(env.disk('a/b/c.txt'))), 'and the file'); + }); + + test('a retry after a failed directory barrier syncs the whole chain, including directories the failed attempt made', async (t) => { + const env = new Env(t); + const m = await env.open(); + await call(m, 'write', { path: 'work/a/b/c.txt', content: 'deep' }); + const failing = await withSyncs(env, (id) => existsSync(env.disk('a')) && id === idOf(env.disk('a')), () => call(m, 'materialize', {})); + assert.deepEqual(failing.result.materialized, []); + assert.match(failing.result.skipped[0].reason, /a\/b\/c\.txt: could not make the write durable \(directory fsync of a\): injected EIO/); + assert.equal((m as any).agreement.get('work', 'a/b/c.txt').kind, 'pending'); + + const retry = await withSyncs(env, () => false, () => call(m, 'materialize', {})); + assert.deepEqual(retry.result.materialized, [{ mount: 'work', path: 'a/b/c.txt' }]); + for (const dir of ['a/b', 'a', '']) assert.ok(retry.synced.includes(idOf(env.disk(dir))), `${dir || 'the mount root'} was synced again`); + assert.equal((m as any).agreement.get('work', 'a/b/c.txt').kind, 'content', 'completed by the push that confirmed it'); + }); + + describe('only a push completes a push', () => { + /** A push of a.txt whose directory barrier failed: the file holds v2, its intent pending. */ + async function unconfirmedPush(env: Env, m: WorkspaceModule, path: string): Promise { + const res = await withSyncs(env, (_id, isDirectory) => isDirectory, () => call(m, 'materialize', {})); + assert.match(res.result.skipped[0].reason, /could not make the write durable \(directory fsync of the mount root\)/); + const pending = (m as any).agreement.get('work', path); + assert.equal(pending.kind, 'pending'); + return pending; + } + + test('a listing does not complete it, so losing its unbarriered write keeps the draft', async (t) => { + const env = new Env(t); + const m = await env.open(); + await seedSynced(env, m, 'a.txt', 'v1'); + await call(m, 'write', { path: 'work/a.txt', content: 'v2' }); + const pending = await unconfirmedPush(env, m, 'a.txt'); + + const entry = await entryOf(m, 'a.txt'); + assert.equal(entry?.state, 'synced'); + assert.match(entry?.note ?? '', /materialize that has not completed/); + assert.deepEqual((m as any).agreement.get('work', 'a.txt'), pending, 'exactly as the push left it'); + + env.writeDisk('a.txt', 'v1'); // the write never reached stable storage + assert.equal(await stateOf(m, 'a.txt'), 'workspace-draft'); + assert.equal(await contentOf(m, 'a.txt'), 'v2', 'the draft is kept, not replaced by the older disk version'); + }); + + test("a new file's store origin survives a listing, so losing the file leaves a draft", async (t) => { + const env = new Env(t); + const m = await env.open(); + await call(m, 'write', { path: 'work/new.txt', content: 'new' }); + const pending = await unconfirmedPush(env, m, 'new.txt'); + assert.equal(await stateOf(m, 'new.txt'), 'synced'); + assert.deepEqual((m as any).agreement.get('work', 'new.txt'), pending); + assert.equal((m as any).intents.get('work').get('new.txt')?.origin, 'store'); + + env.rmDisk('new.txt'); + assert.equal(await stateOf(m, 'new.txt'), 'workspace-draft'); + assert.equal(await contentOf(m, 'new.txt'), 'new'); + }); + + test('the next materialize redoes it, barriers and all, and completes it', async (t) => { + const env = new Env(t); + const m = await env.open(); + await seedSynced(env, m, 'a.txt', 'v1'); + await call(m, 'write', { path: 'work/a.txt', content: 'v2' }); + await unconfirmedPush(env, m, 'a.txt'); + + const status = await call(m, 'status', {}); + assert.equal(status.work.pendingChanges, 1, 'still owed'); + const retry = await withSyncs(env, () => false, () => call(m, 'materialize', {})); + assert.deepEqual(retry.result.materialized, [{ mount: 'work', path: 'a.txt' }]); + assert.ok(retry.synced.includes(idOf(env.disk('a.txt'))), 'the file was written and synced again'); + assert.ok(retry.synced.includes(idOf(env.dir)), 'and its directory'); + assert.equal((m as any).agreement.get('work', 'a.txt').kind, 'content'); + assert.equal((await entryOf(m, 'a.txt'))?.note, undefined); + assert.equal((await call(m, 'status', {})).work.pendingChanges, 0); + }); + + test('a sync of the path chooses disk instead, replacing the pending push', async (t) => { + const env = new Env(t); + const m = await env.open(); + await seedSynced(env, m, 'a.txt', 'v1'); + await call(m, 'write', { path: 'work/a.txt', content: 'v2' }); + await unconfirmedPush(env, m, 'a.txt'); + env.writeDisk('a.txt', 'v3, from the shell'); + + await call(m, 'sync', { path: 'work/a.txt' }); + const p = (m as any).agreement.get('work', 'a.txt'); + assert.equal(p.kind, 'content', 'an adoption, not a completion'); + assert.equal(await contentOf(m, 'a.txt'), 'v3, from the shell'); + assert.equal(await stateOf(m, 'a.txt'), 'synced'); + }); + + test('a branch without the file neither replays its bytes nor unlinks it, and keeps it pending', async (t) => { + const env = new Env(t); + const m = await env.open(); + const main = env.store.currentBranch().name; + env.store.createBranch('before-new', main); + await call(m, 'write', { path: 'work/new.txt', content: 'new' }); + const pending = await unconfirmedPush(env, m, 'new.txt'); + + env.store.switchBranch('before-new'); + assert.equal(await stateOf(m, 'new.txt'), 'not-in-branch'); + for (const input of [{}, { applyDeletions: true }, { force: true }]) { + const res = await call(m, 'materialize', input); + assert.deepEqual(res.materialized, []); + assert.equal(res.deleted, undefined); + } + assert.equal(env.readDisk('new.txt'), 'new'); + assert.deepEqual((m as any).agreement.get('work', 'new.txt'), pending); + + env.store.switchBranch(main); + const res = await call(m, 'materialize', {}); + assert.deepEqual(res.materialized, [{ mount: 'work', path: 'new.txt' }]); + assert.equal((m as any).agreement.get('work', 'new.txt').kind, 'content'); + }); + + test('confirming an unlink never removes a file that appeared since, force or not', async (t) => { + const env = new Env(t); + const m = await env.open(); + await seedSynced(env, m, 'gone.txt', 'v1'); + await call(m, 'delete', { path: 'work/gone.txt' }); + await withSyncs(env, (_id, isDirectory) => isDirectory, () => call(m, 'materialize', { applyDeletions: true })); + const pending = (m as any).agreement.get('work', 'gone.txt'); + assert.equal(pending.kind, 'pending'); + + const mount = (m as any).mounts.get('work'); + const pushed = await pushPaths(env.store, (m as any).agreement, await (m as any).runtime(mount), ['gone.txt'], { + force: true, + beforeEffect: () => writeFileSync(env.disk('gone.txt'), 'a new file from the shell'), + }); + assert.deepEqual(pushed.deleted, []); + assert.match(pushed.skipped[0]!.reason, /^not unlinked: a file appeared at this path since it was checked$/); + assert.equal(env.readDisk('gone.txt'), 'a new file from the shell'); + assert.deepEqual((m as any).agreement.get('work', 'gone.txt'), pending, 'still owed, as before'); + }); + + test('an unlink whose directory is gone is confirmed by syncing the directory that lacks it', async (t) => { + const env = new Env(t); + const m = await env.open(); + await seedSynced(env, m, 'sub/gone.txt', 'v1'); + await call(m, 'delete', { path: 'work/sub/gone.txt' }); + await withSyncs(env, (_id, isDirectory) => isDirectory, () => call(m, 'materialize', { applyDeletions: true })); + assert.equal((m as any).agreement.get('work', 'sub/gone.txt').kind, 'pending'); + rmSync(env.disk('sub'), { recursive: true }); + + const retry = await withSyncs(env, () => false, () => call(m, 'materialize', {})); + assert.deepEqual(retry.result.deleted, [{ mount: 'work', path: 'sub/gone.txt' }]); + assert.ok(retry.synced.includes(idOf(env.dir)), "the root, where the directory's removal is recorded, was synced"); + assert.deepEqual((m as any).agreement.get('work', 'sub/gone.txt'), { kind: 'absent' }); + }); + + test('an unlink whose barrier failed is confirmed by the next materialize, not by a listing', async (t) => { + const env = new Env(t); + const m = await env.open(); + await seedSynced(env, m, 'gone.txt', 'v1'); + await call(m, 'delete', { path: 'work/gone.txt' }); + const res = await withSyncs(env, (_id, isDirectory) => isDirectory, () => call(m, 'materialize', { applyDeletions: true })); + assert.match(res.result.skipped[0].reason, /could not make the unlink durable \(directory fsync of the mount root\)/); + assert.throws(() => env.readDisk('gone.txt'), /ENOENT/); + assert.equal((m as any).agreement.get('work', 'gone.txt').kind, 'pending'); + await listing(m); + assert.equal((m as any).agreement.get('work', 'gone.txt').kind, 'pending', 'a listing records nothing'); + let status = (await call(m, 'status', {})).work; + assert.deepEqual([status.pendingChanges, status.pendingWorkspaceDeletions], [1, 1], 'still owed, and said so'); + + const retry = await withSyncs(env, () => false, () => call(m, 'materialize', {})); + assert.deepEqual(retry.result.deleted, [{ mount: 'work', path: 'gone.txt' }], 'confirmed without applyDeletions: it already reached disk'); + assert.ok(retry.synced.includes(idOf(env.dir))); + assert.deepEqual((m as any).agreement.get('work', 'gone.txt'), { kind: 'absent' }); + status = (await call(m, 'status', {})).work; + assert.deepEqual([status.pendingChanges, status.pendingWorkspaceDeletions], [0, 0]); + }); + }); + + test('a directory that cannot be synced on its filesystem does not fail the write', async (t) => { + const env = new Env(t); + const m = await env.open(); + await call(m, 'write', { path: 'work/a.txt', content: 'v1' }); + const probe = await open(join(env.root, 'probe'), 'w'); + const proto = Object.getPrototypeOf(probe) as { sync: () => Promise }; + await probe.close(); + const sync = proto.sync; + let calls = 0; + proto.sync = async function (this: unknown) { + calls++; + if (calls === 2) throw Object.assign(new Error('fsync of a directory: EINVAL'), { code: 'EINVAL' }); + return sync.call(this); + }; + let res; + try { + res = await call(m, 'materialize', {}); + } finally { + proto.sync = sync; + } + assert.deepEqual(res.materialized, [{ mount: 'work', path: 'a.txt' }]); + assert.equal((m as any).agreement.get('work', 'a.txt').kind, 'content'); + }); + + test('a torn record tail after a materialize, then branch selection', async (t) => { + const env = new Env(t); + let m = await env.open(); + await seedSynced(env, m, 'a.txt', 'v1'); + const main = env.store.currentBranch().name; + env.store.createBranch('before-v2', main); + await call(m, 'write', { path: 'work/a.txt', content: 'v2' }); + await call(m, 'materialize', {}); + await env.shutdown(m); + const log = join(env.storePath, 'records.log'); + truncateSync(log, statSync(log).size - 5); // tears the completion + env.store = JsStore.openOrCreate({ path: env.storePath }); + m = await env.open(); + + env.store.switchBranch('before-v2'); + assert.equal(await stateOf(m, 'a.txt'), 'workspace-draft', 'the intent is decided by disk on any branch'); + env.store.switchBranch(main); + assert.equal(await stateOf(m, 'a.txt'), 'synced'); + assert.equal((m as any).agreement.get('work', 'a.txt').kind, 'pending', 'and completed only by a push'); + await call(m, 'materialize', {}); + assert.equal((m as any).agreement.get('work', 'a.txt').kind, 'content'); + }); + + test("#169's saved baselines aren't evidence: an empty mountpoint at the next start deletes nothing, and agreement records what disk still holds", async (t) => { + // A store that ran #169: entries with no disk-agreement evidence, and the + // stop-time baselines it saved in module state, which name no root. + const env = new Env(t); + await env.shutdown(await env.open()); // the mount's tree state exists, as #169 left it + env.store = JsStore.openOrCreate({ path: env.storePath }); + env.legacyEntry('a.txt', 'alpha'); + env.legacyEntry('b.txt', 'beta'); + const saved = { mounts: { work: { lastMaterializedSeq: 1, materializedHashes: { 'a.txt': sha('alpha'), 'b.txt': sha('beta') }, refusedPaths: [] } } }; + + // The drive isn't mounted yet: its mountpoint is an empty directory. + const m = await env.openRestarted(saved); + const listed = await listing(m); + for (const path of ['a.txt', 'b.txt']) { + assert.equal(listed.get(path)?.state, 'disk-missing-provenance-unknown', `${path} is kept, not adopted as a deletion`); + } + assert.equal(await contentOf(m, 'a.txt'), 'alpha'); + assert.equal(await contentOf(m, 'b.txt'), 'beta'); + + // The drive comes back: what disk agrees with is recorded, root first. + env.writeDisk('a.txt', 'alpha'); + env.writeDisk('b.txt', 'beta, edited in the shell'); + assert.equal(await stateOf(m, 'a.txt'), 'synced'); + assert.equal((await entryOf(m, 'b.txt'))?.conflict?.kind, 'unknown-provenance', 'a differing copy is a conflict, never adopted from a baseline'); + assert.equal(await contentOf(m, 'b.txt'), 'beta'); + }); + + test('evidence checkpoints and replays; a moved mount root discards it', (t) => { + const env = new Env(t); + const agreement = new DiskAgreement(env.store); + agreement.open([{ name: 'work', root: '/srv/work' }]); + for (let i = 0; i < 1100; i++) agreement.set('work', `f${i}.txt`, { kind: 'content', hash: `h${i}`, size: i }); + agreement.maybeCheckpoint(); + agreement.set('work', 'after.txt', { kind: 'absent' }); + + const replayed = new DiskAgreement(env.store); + replayed.open([{ name: 'work', root: '/srv/work' }]); + assert.deepEqual(replayed.get('work', 'f1099.txt'), { kind: 'content', hash: 'h1099', size: 1099 }); + assert.deepEqual(replayed.get('work', 'after.txt'), { kind: 'absent' }); + + const respelled = new DiskAgreement(env.store); + respelled.open([{ name: 'work', root: '/srv/work/' }]); + assert.deepEqual(respelled.get('work', 'f1.txt'), { kind: 'content', hash: 'h1', size: 1 }, 'the same path spelled with a trailing slash keeps its evidence'); + + const moved = new DiskAgreement(env.store); + moved.open([{ name: 'work', root: '/srv/elsewhere' }]); + assert.equal(moved.get('work', 'f1.txt'), undefined, 'evidence about another directory says nothing here'); + }); +}); + +describe('fingerprints', () => { + test('a recorded fingerprint vouches only for an unchanged, non-racy file on a ctime-keeping filesystem', () => { + const fp = { size: 4, mtimeNs: '1000000000000', ctimeNs: '1000000000000', ino: '7', dev: '9' }; + const old = { ...fp, hashedAt: 1_000_000 + 5_000 }; + assert.equal(fingerprintVouches(old, fp, true), true); + assert.equal(fingerprintVouches(old, fp, false), false, 'an unlisted filesystem type always rehashes'); + assert.equal(fingerprintVouches({ ...fp, hashedAt: 1_000_000 + 1_000 }, fp, true), false, 'hashed inside the racy window'); + assert.equal(fingerprintVouches(old, { ...fp, ctimeNs: '1000000000001' }, true), false, 'any field differing'); + assert.equal(fingerprintVouches(undefined, fp, true), false); + }); + + test('the ctime trust gate takes exactly the listed filesystems, and only on Linux', async (t) => { + for (const trusted of [0xef53, 0x58465342, 0x9123683e, 0x01021994, 0xf2f52010, 0x2fc12fc1, 0x794c7630]) { + assert.equal(fsTypeTrustsCtime(trusted), true, trusted.toString(16)); + } + // NFS, FUSE, SMB2 and 9p maintain no ctime it can trust. + for (const untrusted of [0x6969, 0x65735546, 0xfe534d42, 0x01021997]) { + assert.equal(fsTypeTrustsCtime(untrusted), false, untrusted.toString(16)); + } + assert.equal(fsTypeTrustsCtime(BigInt(0xef53)), true, 'a bigint f_type reads the same'); + const env = new Env(t); + const expected = process.platform === 'linux' && fsTypeTrustsCtime((await statfs(env.dir)).type); + assert.equal(await filesystemTrustsCtime(env.dir), expected); + }); + + // The racy window itself is pinned by fingerprintVouches' unit assertions + // above: reaching it here would take a write that leaves mtime and ctime + // unchanged, which no restore can do, since ctime can't be set. + test('no restored timestamp hides a change: utimes, cp -p, and a write right after a hash', async (t) => { + const env = new Env(t); + const m = await env.open(); + await seedSynced(env, m, 'f.txt', 'aaaa'); + await sleep(2100); + await call(m, 'ls', { path: 'work' }); // records a fingerprint old enough to vouch + const before = statSync(env.disk('f.txt')); + + env.writeDisk('f.txt', 'bbbb'); // same size + utimesSync(env.disk('f.txt'), before.atime, before.mtime); + assert.equal(await contentOf(m, 'f.txt'), 'aaaa', 'the read serves the store'); + assert.equal(await stateOf(m, 'f.txt'), 'synced'); + assert.equal(await contentOf(m, 'f.txt'), 'bbbb', 'utimes restoring mtime: rehashed and adopted'); + + await sleep(2100); + await call(m, 'ls', { path: 'work' }); + env.writeDisk(join('..', 'src.txt'), 'cccc'); + utimesSync(join(env.root, 'src.txt'), before.atime, before.mtime); + const cp = spawnSync('cp', ['-p', join(env.root, 'src.txt'), env.disk('f.txt')]); + assert.equal(cp.status, 0, `cp -p: ${cp.stderr}`); + assert.equal(await stateOf(m, 'f.txt'), 'synced'); + assert.equal(await contentOf(m, 'f.txt'), 'cccc', 'cp -p: rehashed and adopted'); + + env.writeDisk('f.txt', 'dddd'); // just after the last hash + assert.equal(await stateOf(m, 'f.txt'), 'synced'); + assert.equal(await contentOf(m, 'f.txt'), 'dddd'); + }); +}); + +describe('trust and identity at the point of use', () => { + /** A view of `dir` as the effects and reads take one. */ + const viewOf = (dir: string, extra: Partial = {}): MountView => + ({ root: dir, followSymlinks: false, maxFileSize: 1 << 20, ignore: [], ...extra }) as MountView; + + test('a fingerprint vouches only for a file on the filesystem whose ctime was checked', async (t) => { + const env = new Env(t); + const m = await env.open(); + await seedSynced(env, m, 'f.txt', 'aaaa'); + env.writeDisk('f.txt', 'bbbb'); // same size: only a read tells + const st = statSync(env.disk('f.txt'), { bigint: true }); + const fp = { size: 4, mtimeNs: st.mtimeNs.toString(), ctimeNs: st.ctimeNs.toString(), ino: st.ino.toString(), dev: st.dev.toString() }; + const newest = Number((st.mtimeNs > st.ctimeNs ? st.mtimeNs : st.ctimeNs) / 1_000_000n); + // Evidence whose fingerprint matches the file as it is now, with the old + // bytes' hash: only a fingerprint that vouches would keep it. + (m as any).agreement.set('work', 'f.txt', { kind: 'content', hash: sha('aaaa'), size: 4, fp: { ...fp, hashedAt: newest + 5_000 } }); + const rootReal = realpathSync(env.dir); + (m as any).ctimeTrust.set(rootReal, st.dev.toString()); // the file's own filesystem was checked + assert.equal(await stateOf(m, 'f.txt'), 'synced'); + assert.equal(await contentOf(m, 'f.txt'), 'aaaa', 'vouched for, so not read'); + (m as any).ctimeTrust.set(rootReal, (st.dev + 1n).toString()); // another device's filesystem was + assert.equal(await stateOf(m, 'f.txt'), 'synced'); + assert.equal(await contentOf(m, 'f.txt'), 'bbbb', 'a file on a filesystem never checked is read, and the change adopted'); + }); + + test('a recheck of an unchanged file records nothing', async (t) => { + const env = new Env(t); + const m = await env.open(); + await seedSynced(env, m, 'f.txt', 'aaaa'); + (m as any).ctimeTrust.set(realpathSync(env.dir), statSync(env.disk('f.txt'), { bigint: true }).dev.toString()); + const old = new Date(Date.now() - 60_000); + utimesSync(env.disk('f.txt'), old, old); + await sleep(2100); // past the racy window of the ctime that utimes set + await call(m, 'sync', {}); // a full recheck rehashes, and records the fingerprint the stamps changed + const agreement = (m as any).agreement; + const set = agreement.set.bind(agreement); + let appended = 0; + agreement.set = (...args: unknown[]) => { appended++; return set(...args); }; + try { + await call(m, 'sync', {}); + await call(m, 'sync', {}); + } finally { + agreement.set = set; + } + assert.equal(appended, 0, 'the same file, the same stamps: nothing to re-record'); + }); + + test('an unlink never removes a file put at its path after the one it hashed', async (t) => { + const env = new Env(t); + env.writeDisk('a.txt', 'old'); + const target = env.disk('a.txt'); + const rootReal = realpathSync(env.dir); + let replaced = false; + const err = await withFsPromises('open', (original) => async (...args: any[]) => { + const handle = await original(...args); + if (!replaced && args[0] === join(rootReal, 'a.txt')) { + replaced = true; // after the hash's descriptor opened: a new file takes the name + writeFileSync(join(env.root, 'swap'), 'new'); + renameSync(join(env.root, 'swap'), target); + } + return handle; + }, () => unlinkContained(viewOf(env.dir), rootReal, 'a.txt', { kind: 'content', hash: sha('old') }).then(() => null, (e) => e)); + assert.ok(replaced, 'the swap ran'); + assert.ok(err instanceof EffectFailed, `refused: ${err}`); + assert.equal(readFileSync(target, 'utf8'), 'new', 'the file now at the path is untouched'); + }); + + test('an unlink refused outright touched nothing, so its evidence goes back at once', { skip: IS_ROOT ? 'root unlinks anywhere' : false }, async (t) => { + const env = new Env(t); + const m = await env.open(); + await seedSynced(env, m, 'locked/a.txt', 'v1'); + await call(m, 'delete', { path: 'work/locked/a.txt' }); + chmodSync(env.disk('locked'), 0o555); + let data; + try { + data = await call(m, 'materialize', { applyDeletions: true }); + } finally { + chmodSync(env.disk('locked'), 0o755); + } + assert.match(JSON.stringify(data), /not unlinked: could not unlink the file/, `reported as untouched: ${JSON.stringify(data)}`); + assert.equal((m as any).agreement.get('work', 'locked/a.txt').kind, 'content', 'the evidence is as it was, not pending'); + assert.equal(env.readDisk('locked/a.txt'), 'v1'); + }); + + test('a FIFO swapped in at a path neither blocks a read nor a write', { skip: process.platform === 'win32' ? 'no FIFOs' : false }, async (t) => { + const env = new Env(t); + const rootReal = realpathSync(env.dir); + const swapIn = (path: string) => { + rmSync(path); + assert.equal(spawnSync('mkfifo', [path]).status, 0, 'mkfifo'); + }; + const bounded = (what: string, run: Promise) => Promise.race([ + run, + new Promise((_, reject) => setTimeout(() => reject(new Error(`${what} blocked on the FIFO`)), 3000)), + ]); + for (const how of ['read', 'write'] as const) { + env.writeDisk(`${how}.txt`, 'regular'); + const path = join(rootReal, `${how}.txt`); + // The read opens the path under the view's root as given; where the + // temp directory is reached through a symlink (macOS's /var), that + // isn't the real path. + const named = join(env.dir, `${how}.txt`); + let swapped = false; + const outcome = await withFsPromises('open', (original) => async (...args: any[]) => { + if (!swapped && (args[0] === path || args[0] === named)) { + swapped = true; // between any lstat and this open + swapIn(path); + } + return original(...args); + }, () => bounded(how, how === 'read' + ? readPath(viewOf(env.dir), rootReal, 'read.txt', 1 << 20) + : writeContained(viewOf(env.dir), rootReal, 'write.txt', Buffer.from('x'), { kind: 'any' }).then(() => 'written', (e) => e))); + assert.ok(swapped, `${how}: the swap ran`); + if (how === 'read') assert.deepEqual(outcome, { kind: 'unobserved', reason: 'not a regular file' }); + else assert.ok(outcome instanceof EffectFailed && /not a regular file/.test(outcome.message), `write refused: ${outcome}`); + } + }); + + /** A root identified as it is now, and a way to replace it: renamed away, with an empty `sub` in its place. */ + const replaceable = (env: Env) => { + mkdirSync(env.disk('sub'), { recursive: true }); + const st = statSync(env.dir, { bigint: true }); + const view = viewOf(env.dir, { rootIdentity: { dev: st.dev.toString(), ino: st.ino.toString() } }); + const away = `${env.dir}-away`; + const replace = () => { + renameSync(env.dir, away); + mkdirSync(join(env.dir, 'sub'), { recursive: true }); + }; + return { view, rootReal: realpathSync(env.dir), away, replace }; + }; + const isSub = (env: Env, rootReal: string, path: unknown) => + String(path) === join(env.dir, 'sub') || String(path) === join(rootReal, 'sub'); + + test('a write whose root is replaced after it was checked, before its parent is found, writes nothing there', async (t) => { + const env = new Env(t); + const { view, rootReal, replace } = replaceable(env); + let replaced = false; + const err = await withFsPromises('realpath', (realpath) => async (path: unknown, ...rest: unknown[]) => { + if (!replaced && isSub(env, rootReal, path)) { + replaced = true; + replace(); + } + return realpath(path, ...rest); + }, () => writeContained(view, rootReal, 'sub/a.txt', Buffer.from('content'), { kind: 'any' }).then(() => null, (e) => e)); + assert.equal(replaced, true); + assert.ok(err instanceof EffectFailed && /not the directory disk last agreed with/.test(err.message), `refused, naming the root: ${err}`); + const there = join(env.dir, 'sub/a.txt'); + assert.equal(existsSync(there) ? readFileSync(there, 'utf8') : '', '', 'no content reached the replacement'); + }); + + test("an edit made while a forced write is checked isn't overwritten", async (t) => { + const env = new Env(t); + env.writeDisk('a.txt', 'before'); + const rootReal = realpathSync(env.dir); + const target = join(rootReal, 'a.txt'); + let edited = false; + const err = await withFsPromises('realpath', (realpath) => async (path: unknown, ...rest: unknown[]) => { + if (!edited && String(path) === target) { + edited = true; + writeFileSync(target, 'before, and an edit'); + } + return realpath(path, ...rest); + }, () => writeContained(viewOf(env.dir), rootReal, 'a.txt', Buffer.from('overwrite'), { kind: 'any' }).then(() => null, (e) => e)); + assert.equal(edited, true); + assert.ok(err instanceof EffectFailed && !err.touched, `refused untouched: ${err}`); + assert.equal(readFileSync(target, 'utf8'), 'before, and an edit'); + }); + + test("an edit made after a file was hashed for its unlink isn't deleted", async (t) => { + const env = new Env(t); + mkdirSync(env.disk('sub'), { recursive: true }); + env.writeDisk('sub/a.txt', 'old'); + const rootReal = realpathSync(env.dir); + const target = join(rootReal, 'sub/a.txt'); + let subChecks = 0; + let edited = false; + // The parent is resolved once to find it and again to check it is still + // there, which comes after the hash and before the unlink: the edit lands + // in between. + const err = await withFsPromises('realpath', (realpath) => async (path: unknown, ...rest: unknown[]) => { + if (isSub(env, rootReal, path) && ++subChecks === 2) { + edited = true; + writeFileSync(target, 'old, edited'); + } + return realpath(path, ...rest); + }, () => unlinkContained(viewOf(env.dir), rootReal, 'sub/a.txt', { kind: 'content', hash: createHash('sha256').update('old').digest('hex') }) + .then(() => null, (e) => e)); + assert.equal(edited, true); + assert.ok(err instanceof EffectFailed && !err.touched, `refused untouched: ${err}`); + assert.equal(readFileSync(target, 'utf8'), 'old, edited'); + }); + + test('a directory sync under a root replaced before the directory was found reports it, not a success', async (t) => { + const env = new Env(t); + const { view, rootReal, replace } = replaceable(env); + let replaced = false; + const failed = await withFsPromises('realpath', (realpath) => async (path: unknown, ...rest: unknown[]) => { + if (!replaced && isSub(env, rootReal, path)) { + replaced = true; + replace(); + } + return realpath(path, ...rest); + }, () => syncDirectories(view, rootReal, ['sub'])); + assert.equal(replaced, true); + assert.match(failed.get('sub') ?? '', /not the directory disk last agreed with/); + }); + + test("an effect under a replaced root acts on nothing, whatever its caller checked", async (t) => { + const env = new Env(t); + const rootReal = realpathSync(env.dir); + const st = statSync(env.dir, { bigint: true }); + const view = viewOf(env.dir, { rootIdentity: { dev: st.dev.toString(), ino: st.ino.toString() } }); + renameSync(env.dir, join(env.root, 'unmounted')); + mkdirSync(join(env.dir, 'sub'), { recursive: true }); // another directory where the drive was, with a same-named subdirectory + const err = await writeContained(view, rootReal, 'sub/a.txt', Buffer.from('x'), { kind: 'absent' }).then(() => null, (e) => e); + assert.ok(err instanceof EffectFailed, `refused: ${err}`); + assert.equal(existsSync(join(env.dir, 'sub', 'a.txt')), false, 'nothing written in the replacement'); + }); +}); + +describe('incomplete walks', () => { + test('a permission-denied subtree deletes nothing beneath it', { skip: IS_ROOT ? 'root reads every directory' : false }, async (t) => { + const env = new Env(t); + const m = await env.open(); + await seedSynced(env, m, 'locked/a.txt', 'v1'); + chmodSync(env.disk('locked'), 0o000); + try { + const data = await call(m, 'ls', { path: 'work', recursive: true }); + const entry = (data.entries as Entry[]).find((e) => e.path === 'locked/a.txt'); + assert.equal(entry?.state, 'unverified'); + assert.ok((data.incomplete as Array<{ path: string; reason: string }>).some((r) => r.path === 'locked' && /cannot list/.test(r.reason))); + await call(m, 'sync', {}); + assert.notEqual(env.store.treeGet(TREE, 'locked/a.txt'), null, 'nothing unvisited is deleted'); + } finally { + chmodSync(env.disk('locked'), 0o755); + } + }); + + test('an ignored subtree deletes nothing beneath it, and untracked ignored files are not listed', async (t) => { + const env = new Env(t); + const m = await env.open({ ignore: ['cache'] }); + await seedSynced(env, m, 'cache/x.txt', 'tracked'); + env.rmDisk('cache/x.txt'); + env.writeDisk('cache/y.txt', 'untracked'); + const entries = await listing(m); + assert.equal(entries.get('cache/x.txt')?.state, 'unverified'); + assert.equal(entries.has('cache/y.txt'), false); + await call(m, 'sync', {}); + assert.notEqual(env.store.treeGet(TREE, 'cache/x.txt'), null); + }); + + test('a capped walk deletes nothing it did not reach', async (t) => { + const env = new Env(t); + const m = await env.open(); + for (const f of ['a.txt', 'b.txt', 'c.txt', 'd.txt']) await seedSynced(env, m, f, f); + env.rmDisk('c.txt'); + const mount = (m as any).mounts.get('work'); + const pass = await (m as any).withMount(mount, () => + (m as any).passUnlocked(mount, { kind: 'dir', dir: '', recursive: true }, { cap: 2 })); + assert.equal(pass.reports.get('c.txt')?.state, 'unverified'); + assert.ok(pass.incomplete.some((r: { reason: string }) => /file cap/.test(r.reason))); + assert.notEqual(env.store.treeGet(TREE, 'c.txt'), null); + }); +}); + +describe('the mount boundary', () => { + test('a root replaced during an absence proof proves nothing: the replacement lacks what the original still holds', async (t) => { + const env = new Env(t); + mkdirSync(env.disk('sub'), { recursive: true }); + env.writeDisk('sub/a.txt', 'A'); + const st = statSync(env.dir, { bigint: true }); + const view = { root: env.dir, rootIdentity: { dev: String(st.dev), ino: String(st.ino) }, followSymlinks: false, maxFileSize: 1e6, ignore: [] } as unknown as MountView; + const away = `${env.dir}-away`; + // The code checks the root by its resolved path (on macOS the temp + // directory is under /var, a symlink to /private/var). + const rootReal = realpathSync(env.dir); + let swapped = false; + // Once the root has been checked, it's renamed away and an empty one put + // in its place, with the same subdirectory. + const proof = await withFsPromises('lstat', (lstat) => async (path: string, ...rest: unknown[]) => { + const info = await lstat(path, ...rest); + if (!swapped && String(path) === rootReal) { + swapped = true; + renameSync(env.dir, away); + mkdirSync(join(env.dir, 'sub'), { recursive: true }); + } + return info; + }, () => confirmAbsent(view, rootReal, 'sub/a.txt')); + assert.equal(swapped, true); + assert.equal(typeof proof, 'string', `no absence is proven: ${JSON.stringify(proof)}`); + assert.equal(readFileSync(join(away, 'sub/a.txt'), 'utf8'), 'A', 'the original still holds the file'); + }); + + test('a root replaced after a walk listed it decides nothing: no deletion is taken from the replacement', async (t) => { + const env = new Env(t); + const m = await env.open(); + await call(m, 'write', { path: 'work/a/f.txt', content: 'F' }); + await call(m, 'materialize', { path: 'work/a/f.txt' }); + const rootReal = realpathSync(env.dir); + const away = `${env.dir}-away`; + let replaced = false; + // Once the walk has listed the root, the root is replaced by one whose + // `a` is empty: the walk then lists the replacement's `a`. + const result = await withFsPromises('readdir', (readdir) => async (path: unknown, ...rest: unknown[]) => { + const entries = await readdir(path, ...rest); + if (!replaced && String(path) === rootReal) { + replaced = true; + renameSync(env.dir, away); + mkdirSync(join(env.dir, 'a'), { recursive: true }); + } + return entries; + }, () => call(m, 'sync', {})); + assert.equal(replaced, true); + assert.notEqual(env.store.treeGet('workspace/work/tree', 'a/f.txt'), null, 'the workspace keeps the file'); + assert.notEqual((m as unknown as { agreement: DiskAgreement }).agreement.get('work', 'a/f.txt')?.kind, 'absent'); + assert.match(JSON.stringify(result), /not the directory disk last agreed with/); + assert.equal(readFileSync(join(away, 'a/f.txt'), 'utf8'), 'F'); + }); + + test('a root replaced during a pass takes no deletion: the entry and its evidence stay', async (t) => { + const env = new Env(t); + const m = await env.open(); + await call(m, 'write', { path: 'work/sub/a.txt', content: 'A' }); + await call(m, 'materialize', { path: 'work/sub/a.txt' }); + const file = env.disk('sub/a.txt'); + const away = `${env.dir}-away`; + // The pass observes paths under the root's resolved path (on macOS the + // temp directory is under /var, a symlink to /private/var). + const rootReal = realpathSync(env.dir); + const fileReal = join(rootReal, 'sub/a.txt'); + unlinkSync(file); + let restored = false; + let swapped = false; + // The pass finds the file gone, then it's put back, and the root is + // replaced by one whose subdirectory lacks it before absence is proven. + await withFsPromises('lstat', (lstat) => async (path: string, ...rest: unknown[]) => { + let info; + try { + info = await lstat(path, ...rest); + } catch (err) { + if (!restored && String(path) === fileReal && (err as NodeJS.ErrnoException).code === 'ENOENT') { + writeFileSync(file, 'A'); + restored = true; + } + throw err; + } + if (restored && !swapped && String(path) === rootReal) { + renameSync(env.dir, away); + mkdirSync(join(env.dir, 'sub'), { recursive: true }); + swapped = true; + } + return info; + }, () => { + // The pass a watcher's change to that path runs. + const internals = m as unknown as { mounts: Map; passUnlocked(mount: unknown, scope: unknown): Promise }; + return internals.passUnlocked(internals.mounts.get('work'), { kind: 'paths', paths: ['sub/a.txt'] }); + }); + assert.equal(restored, true); + assert.equal(swapped, true); + assert.notEqual(env.store.treeGet('workspace/work/tree', 'sub/a.txt'), null, 'the workspace keeps the file'); + assert.notEqual((m as unknown as { agreement: DiskAgreement }).agreement.get('work', 'sub/a.txt')?.kind, 'absent'); + assert.equal(readFileSync(join(away, 'sub/a.txt'), 'utf8'), 'A'); + }); + + test('not even a forced materialize writes through a symlink or a symlinked directory out of the mount', async (t) => { + const env = new Env(t); + const m = await env.open(); + const outside = join(env.root, 'outside.txt'); + writeFileSync(outside, 'outside, untouched'); + mkdirSync(join(env.root, 'outdir')); + await call(m, 'write', { path: 'work/link.txt', content: 'from the workspace' }); + await call(m, 'write', { path: 'work/linkdir/inner.txt', content: 'from the workspace' }); + symlinkSync(outside, env.disk('link.txt')); + symlinkSync(join(env.root, 'outdir'), env.disk('linkdir')); + + const res = await call(m, 'materialize', { force: true }); + const reasons = ((res.skipped ?? []) as Array<{ reason: string }>).map((s) => s.reason).join('\n'); + assert.match(reasons, /link\.txt: not written: a symlink, which this mount does not follow/); + assert.match(reasons, /linkdir\/inner\.txt: not written: a parent directory resolves outside the mount/); + assert.equal(readFileSync(outside, 'utf8'), 'outside, untouched'); + assert.throws(() => readFileSync(join(env.root, 'outdir', 'inner.txt')), /ENOENT/); + }); + + test('a file with other hard links is never written in place, forced or not: its other names keep their content', async (t) => { + const env = new Env(t); + const m = await env.open(); + const outside = join(env.root, 'outside'); + mkdirSync(outside); + writeFileSync(join(outside, 'secret.txt'), 'outside original'); + linkSync(join(outside, 'secret.txt'), env.disk('h.txt')); + assert.equal(await stateOf(m, 'h.txt'), 'synced', 'a listing takes it in: reading through the name in the mount'); + await call(m, 'write', { path: 'work/h.txt', content: 'written by the workspace' }); + const before = (m as any).agreement.get('work', 'h.txt'); + + for (const force of [false, true]) { + const res = await call(m, 'materialize', { force }); + assert.deepEqual(res.materialized, [], `force: ${force}`); + const reasons = ((res.skipped ?? []) as Array<{ reason: string }>).map((s) => s.reason).join('\n'); + assert.match(reasons, /h\.txt: not written: a file with other hard links/, `force: ${force}`); + assert.equal(readFileSync(join(outside, 'secret.txt'), 'utf8'), 'outside original', `force: ${force}`); + assert.deepEqual((m as any).agreement.get('work', 'h.txt'), before, 'its evidence is exactly as before the intent'); + } + assert.equal(await stateOf(m, 'h.txt'), 'workspace-draft', 'the draft is kept, still owed'); + assert.equal(await contentOf(m, 'h.txt'), 'written by the workspace'); + }); + + test('an autoMaterialize write to a file linked elsewhere after it was taken in is refused, and the write says why', async (t) => { + const env = new Env(t); + const m = await env.open({ autoMaterialize: true }); + env.writeDisk('h.txt', 'shared original'); + assert.equal(await stateOf(m, 'h.txt'), 'synced'); + const outside = join(env.root, 'outside.txt'); + linkSync(env.disk('h.txt'), outside); + + const error = await refused(m, 'write', { path: 'work/h.txt', content: 'written by the workspace' }); + assert.match(error, /not written: a file with other hard links/); + assert.equal(readFileSync(outside, 'utf8'), 'shared original'); + assert.equal(await contentOf(m, 'h.txt'), 'written by the workspace', 'the workspace keeps the write'); + }); + + /** A push of `paths` whose `beforeEffect` runs between planning and the effect. */ + async function pushWith(env: Env, m: WorkspaceModule, paths: string[], opts: Parameters[4]) { + const mount = (m as any).mounts.get('work'); + return pushPaths(env.store, (m as any).agreement, await (m as any).runtime(mount), paths, opts); + } + + for (const force of [false, true]) { + test(`a parent swapped out of the mount after planning gets nothing written outside it (force: ${force})`, async (t) => { + const env = new Env(t); + const m = await env.open(); + await seedSynced(env, m, 'sub/a.txt', 'original'); + await call(m, 'write', { path: 'work/sub/a.txt', content: 'draft' }); + const outside = join(env.root, 'outside'); + mkdirSync(outside); + writeFileSync(join(outside, 'a.txt'), 'outside, untouched'); + const before = (m as any).agreement.get('work', 'sub/a.txt'); + + const pushed = await pushWith(env, m, ['sub/a.txt'], { force, beforeEffect: () => swapForSymlink(env, 'sub', outside) }); + assert.equal(readFileSync(join(outside, 'a.txt'), 'utf8'), 'outside, untouched'); + assert.deepEqual(pushed.written, []); + assert.match(pushed.skipped[0]!.reason, /^not written: a parent directory resolves outside the mount$/); + assert.deepEqual((m as any).agreement.get('work', 'sub/a.txt'), before, 'its evidence is exactly as before the intent'); + assert.equal(env.readDisk('saved/a.txt'), 'original'); + }); + } + + test('a workspace deletion is never applied through a parent swapped out of the mount after planning', async (t) => { + const env = new Env(t); + const m = await env.open(); + await seedSynced(env, m, 'sub/gone.txt', 'v1'); + await call(m, 'delete', { path: 'work/sub/gone.txt' }); + const outside = join(env.root, 'outside'); + mkdirSync(outside); + writeFileSync(join(outside, 'gone.txt'), 'outside, untouched'); + + const pushed = await pushWith(env, m, ['sub/gone.txt'], { applyDeletions: true, beforeEffect: () => swapForSymlink(env, 'sub', outside) }); + assert.equal(readFileSync(join(outside, 'gone.txt'), 'utf8'), 'outside, untouched'); + assert.deepEqual(pushed.deleted, []); + assert.match(pushed.skipped[0]!.reason, /^not unlinked: a parent directory resolves outside the mount$/); + }); + + test('a disk edit made after planning is not overwritten without force, and reads as a conflict', async (t) => { + const env = new Env(t); + const m = await env.open(); + await seedSynced(env, m, 'a.txt', 'v1'); + await call(m, 'write', { path: 'work/a.txt', content: 'v2' }); + const before = (m as any).agreement.get('work', 'a.txt'); + + const pushed = await pushWith(env, m, ['a.txt'], { beforeEffect: () => writeFileSync(env.disk('a.txt'), 'edited meanwhile') }); + assert.equal(env.readDisk('a.txt'), 'edited meanwhile'); + assert.deepEqual(pushed.written, []); + assert.match(pushed.skipped[0]!.reason, /^not written: disk changed since it was checked$/); + assert.deepEqual((m as any).agreement.get('work', 'a.txt'), before); + const entry = await entryOf(m, 'a.txt'); + assert.equal(entry?.state, 'conflict'); + assert.equal(entry?.conflict?.kind, 'both-changed', 'both sides changed since they last agreed'); + + const forced = await pushWith(env, m, ['a.txt'], { force: true, beforeEffect: () => writeFileSync(env.disk('a.txt'), 'edited again') }); + assert.deepEqual(forced.written, ['a.txt'], 'force overrides freshness'); + assert.equal(env.readDisk('a.txt'), 'v2'); + }); + + test('a file that appears after planning saw none is not overwritten', async (t) => { + const env = new Env(t); + const m = await env.open(); + await call(m, 'write', { path: 'work/new.txt', content: 'from the workspace' }); + + const pushed = await pushWith(env, m, ['new.txt'], { beforeEffect: () => writeFileSync(env.disk('new.txt'), 'from the shell') }); + assert.equal(env.readDisk('new.txt'), 'from the shell'); + assert.match(pushed.skipped[0]!.reason, /^not written: something appeared at this path since it was checked$/); + assert.equal((m as any).agreement.get('work', 'new.txt'), undefined, 'still unknown, as before the intent'); + assert.equal(await stateOf(m, 'new.txt'), 'conflict'); + }); + + test('a hard link made after planning is caught where the write happens', async (t) => { + const env = new Env(t); + const m = await env.open(); + await seedSynced(env, m, 'a.txt', 'v1'); + await call(m, 'write', { path: 'work/a.txt', content: 'the workspace draft' }); + const outside = join(env.root, 'outside.txt'); + const before = (m as any).agreement.get('work', 'a.txt'); + + const pushed = await pushWith(env, m, ['a.txt'], { force: true, beforeEffect: () => linkSync(env.disk('a.txt'), outside) }); + assert.deepEqual(pushed.written, []); + assert.match(pushed.skipped[0]!.reason, /^not written: a file with other hard links/); + assert.equal(readFileSync(outside, 'utf8'), 'v1', 'the other name keeps its content'); + assert.deepEqual((m as any).agreement.get('work', 'a.txt'), before); + }); + + test('even a forced write never lands in a file renamed away after it was opened', async (t) => { + const env = new Env(t); + const m = await env.open(); + await seedSynced(env, m, 'a.txt', 'v1'); + await call(m, 'write', { path: 'work/a.txt', content: 'the workspace draft' }); + let swapped = false; + const pushed = await withFsPromises('open', (open) => async (path, flags, ...rest) => { + const handle = await open(path, flags, ...rest); + if (!swapped && basename(String(path)) === 'a.txt' && typeof flags === 'number' && (flags & fsConstants.O_WRONLY) !== 0) { + swapped = true; // renamed away once opened, and a new file put at its path + renameSync(env.disk('a.txt'), env.disk('backup.txt')); + writeFileSync(env.disk('a.txt'), 'a new disk edit'); + } + return handle; + }, () => pushWith(env, m, ['a.txt'], { force: true })); + assert.equal(swapped, true); + assert.deepEqual(pushed.written, []); + assert.match(pushed.skipped[0]!.reason, /disk changed since it was checked/); + assert.equal(env.readDisk('backup.txt'), 'v1', 'the renamed file is untouched'); + assert.equal(env.readDisk('a.txt'), 'a new disk edit', 'and so is the file now at the path'); + }); + + test("a forced push of a stale path whose workspace copy is missing from the store says so", async (t) => { + const env = new Env(t); + let m = await env.open(); + await seedSynced(env, m, 'a.txt', 'v1'); + env.writeDisk('a.txt', 'v2, from the shell'); // S = P, D changed: a forced push would overwrite it + await env.shutdown(m); + rmSync(join(env.storePath, 'blobs', sha('v1').slice(0, 2), sha('v1'))); // the store lost the workspace copy + env.store = JsStore.openOrCreate({ path: env.storePath }); + m = await env.open(); + const data = await call(m, 'materialize', { path: 'work/a.txt', force: true }); + assert.match(JSON.stringify(data), /the workspace copy is missing from the store/, `reported, not dropped: ${JSON.stringify(data)}`); + assert.equal(env.readDisk('a.txt'), 'v2, from the shell'); + }); + + test("an interrupted path's refusal says what disk may hold, and that force writes the workspace copy", async (t) => { + const env = new Env(t); + let m = await env.open(); + await seedSynced(env, m, 'a.txt', 'v1'); + await call(m, 'write', { path: 'work/a.txt', content: 'v2' }); + m = await env.fault(m, 'materialize', 'after-intent'); + env.writeDisk('a.txt', 'v2, cut off'); // neither what was being written nor what disk agreed with + assert.equal((await entryOf(m, 'a.txt'))?.conflict?.kind, 'interrupted'); + const data = await call(m, 'materialize', {}); + const skip = JSON.stringify(data); + assert.match(skip, /match neither what was being written there nor what disk last agreed with/); + assert.match(skip, /pass force to write the workspace copy over it/); + assert.equal(env.readDisk('a.txt'), 'v2, cut off'); + await call(m, 'materialize', { force: true }); + assert.equal(env.readDisk('a.txt'), 'v2'); + }); + + test('a refused write puts back whatever evidence the path had, an interrupted one included', async (t) => { + const env = new Env(t); + const m = await env.open(); + await seedSynced(env, m, 'sub/a.txt', 'original'); + await call(m, 'write', { path: 'work/sub/a.txt', content: 'draft' }); + const interrupted = { kind: 'interrupted', candidates: [{ kind: 'absent' }, { kind: 'content', hash: 'h' }] }; + (m as any).agreement.set('work', 'sub/a.txt', interrupted); + mkdirSync(join(env.root, 'outside')); + + await pushWith(env, m, ['sub/a.txt'], { force: true, beforeEffect: () => swapForSymlink(env, 'sub', join(env.root, 'outside')) }); + assert.deepEqual((m as any).agreement.get('work', 'sub/a.txt'), interrupted); + }); + + test('a directory swapped out of the mount after its parent was listed is never walked', async (t) => { + const env = new Env(t); + const m = await env.open(); + env.writeDisk('sub/a.txt', 'original'); + await call(m, 'sync', {}); + const outside = join(env.root, 'outside'); + mkdirSync(outside); + writeFileSync(join(outside, 'secret.txt'), 'outside'); + + let swapped = false; + const data = await withFsPromises('readdir', (readdir) => async (...args) => { + const listed = await readdir(...args); + if (!swapped) { + swapped = true; // after the root's listing, before its subdirectory's + swapForSymlink(env, 'sub', outside); + } + return listed; + }, () => call(m, 'ls', { path: 'work', recursive: true })); + const states = new Map((data.entries as Entry[]).map((e) => [e.path, e.state])); + assert.equal(states.has('sub/secret.txt'), false, 'no outside name is listed'); + assert.equal(states.get('sub/a.txt'), 'unverified'); + assert.deepEqual(data.incomplete, [{ path: 'sub', reason: 'the directory changed while the walk ran' }]); + assert.notEqual(env.store.treeGet(TREE, 'sub/a.txt'), null, 'the agreed entry stays'); + }); + + test("a directory that changes between being held and being listed contributes nothing", async (t) => { + const env = new Env(t); + const m = await env.open(); + env.writeDisk('sub/a.txt', 'original'); + await call(m, 'sync', {}); + const outside = join(env.root, 'outside'); + mkdirSync(outside); + writeFileSync(join(outside, 'secret.txt'), 'outside'); + + let swapped = false; + const data = await withFsPromises('readdir', (readdir) => async (path, ...rest) => { + if (!swapped && basename(String(path)) === 'sub') { + swapped = true; // the subdirectory's own listing reaches outside + swapForSymlink(env, 'sub', outside); + } + return readdir(path, ...rest); + }, () => call(m, 'ls', { path: 'work', recursive: true })); + const states = new Map((data.entries as Entry[]).map((e) => [e.path, e.state])); + assert.equal(swapped, true); + assert.equal(states.has('sub/secret.txt'), false, 'no outside name is listed'); + assert.equal(states.get('sub/a.txt'), 'unverified'); + assert.notEqual(env.store.treeGet(TREE, 'sub/a.txt'), null, 'nothing is removed on its absence'); + }); + + test('absence is proven by the directory that lacks the entry, not by its name', async (t) => { + const env = new Env(t); + const m = await env.open(); + mkdirSync(env.disk('sub')); + const mount = (m as any).mounts.get('work'); + const runtime = await (m as any).runtime(mount); + env.writeDisk('replacement/a.txt', 'present in the replacement'); + let misses = 0; + const seen = await withFsPromises('lstat', (lstat) => async (path, ...rest) => { + try { + return await lstat(path, ...rest); + } catch (err) { + // By name: the lookups may go through the lexical or the canonical path. + if (basename(String(path)) === 'a.txt' && ++misses === 2) { + // The empty directory that lacked it is replaced by one that has it. + renameSync(env.disk('sub'), env.disk('old')); + renameSync(env.disk('replacement'), env.disk('sub')); + } + throw err; + } + }, () => observePath(runtime.view, runtime.rootReal, 'sub/a.txt')); + assert.equal(misses, 2); + assert.equal(seen.kind, 'unobserved', 'not absent: the directory that lacked it is gone'); + assert.equal(env.readDisk('sub/a.txt'), 'present in the replacement'); + }); + + test('a file standing where a directory was proves nothing once a directory is back', async (t) => { + const env = new Env(t); + const m = await env.open(); + env.writeDisk('sub/a.txt', 'A'); + await call(m, 'sync', {}); + renameSync(env.disk('sub'), env.disk('saved')); + writeFileSync(env.disk('sub'), 'a file, for a moment'); + let swapped = false; + const data = await withFsPromises('lstat', (lstat) => async (path, ...rest) => { + const info = await lstat(path, ...rest); + // By name: the lookups may go through the lexical or the canonical path. + if (!swapped && basename(String(path)) === 'sub' && info.isFile()) { + swapped = true; + rmSync(env.disk('sub')); + renameSync(env.disk('saved'), env.disk('sub')); + } + return info; + }, () => call(m, 'ls', { path: 'work/sub', recursive: true })); + assert.equal(swapped, true); + assert.equal((data.entries as Entry[]).find((e) => e.path === 'sub/a.txt')?.state, 'unverified'); + assert.notEqual(env.store.treeGet(TREE, 'sub/a.txt'), null, 'nothing removed while it was present'); + }); + + test('a path beneath a dangling symlink is not proven absent', async (t) => { + const env = new Env(t); + const m = await env.open(); + env.writeDisk('sub/a.txt', 'original'); + await call(m, 'sync', {}); + swapForSymlink(env, 'sub', join(env.root, 'nowhere')); + + await (m as any).handleFsChanges('work', [{ path: 'sub/a.txt', op: 'deleted' }]); + assert.notEqual(env.store.treeGet(TREE, 'sub/a.txt'), null, 'a watcher event deleted nothing'); + renameSync(env.disk('saved'), env.disk('elsewhere')); + rmSync(env.disk('sub')); + await (m as any).handleFsChanges('work', [{ path: 'sub/a.txt', op: 'deleted' }]); + assert.equal(env.store.treeGet(TREE, 'sub/a.txt'), null, 'a real deletion is still adopted'); + }); + + for (const followSymlinks of [false, true]) { + test(`a symlinked parent never lets a read, listing or grep reach outside the mount (followSymlinks: ${followSymlinks})`, async (t) => { + const env = new Env(t); + const m = await env.open({ followSymlinks }); + const outside = join(env.root, 'outside'); + mkdirSync(outside); + writeFileSync(join(outside, 'secret.txt'), 'OUTSIDE_SENTINEL'); + symlinkSync(outside, env.disk('escape')); + + const error = await refused(m, 'read', { path: 'work/escape/secret.txt' }); + assert.match(error, /outside the mount/); + assert.doesNotMatch(error, /OUTSIDE_SENTINEL/); + const scoped = await call(m, 'ls', { path: 'work/escape', recursive: true }); + assert.deepEqual(scoped.entries, []); + assert.deepEqual(scoped.incomplete, [{ path: 'escape', reason: 'the directory resolves outside the mount' }]); + const whole = await listing(m); + assert.equal([...whole.keys()].some((p) => p.startsWith('escape/')), false); + assert.deepEqual((await call(m, 'grep', { pattern: 'OUTSIDE_SENTINEL' })).results, []); + assert.deepEqual((await call(m, 'grep', { pattern: 'OUTSIDE_SENTINEL', path: 'work/escape/secret.txt' })).results, []); + assert.equal(env.store.treeGet(TREE, 'escape/secret.txt'), null, 'nothing from outside was ingested'); + }); + } + + test('an unavailable root with nothing tracked says so instead of listing a verified empty directory', async (t) => { + const env = new Env(t); + const m = await env.open({ path: join(env.root, 'never-created'), mode: 'read-only' }); + const data = await call(m, 'ls', { path: 'work' }); + assert.deepEqual(data.entries, []); + assert.deepEqual(data.incomplete, [{ path: '', reason: 'the mount root is unavailable' }]); + }); + + for (const followSymlinks of [false, true]) { + test(`a mount whose configured root is a symlink to a directory works (followSymlinks: ${followSymlinks})`, async (t) => { + const env = new Env(t); + const linkedRoot = join(env.root, 'linked-root'); + symlinkSync(env.dir, linkedRoot); + env.writeDisk('from-disk.txt', 'written through the real directory'); + const m = await env.open({ path: linkedRoot, followSymlinks }); + + const data = await call(m, 'ls', { path: 'work', recursive: true }); + assert.equal(data.incomplete, undefined, 'the root is available'); + assert.deepEqual((data.entries as Entry[]).map((e) => [e.path, e.state]), [['from-disk.txt', 'synced']]); + assert.equal(await contentOf(m, 'from-disk.txt'), 'written through the real directory'); + await seedSynced(env, m, 'from-workspace.txt', 'materialized through the link'); + env.rmDisk('from-disk.txt'); + await call(m, 'sync', {}); + assert.equal(env.store.treeGet(TREE, 'from-disk.txt'), null, 'a shell deletion is still adopted'); + }); + } + + test('a missing mount root is unavailable, not a deletion of everything in it', async (t) => { + const env = new Env(t); + const m = await env.open(); + await seedSynced(env, m, 'a.txt', 'v1'); + renameSync(env.dir, `${env.dir}.away`); + try { + assert.equal(await stateOf(m, 'a.txt'), 'unverified'); + await call(m, 'sync', {}); + assert.notEqual(env.store.treeGet(TREE, 'a.txt'), null, 'nothing was removed'); + } finally { + renameSync(`${env.dir}.away`, env.dir); + } + assert.equal(await stateOf(m, 'a.txt'), 'synced'); + }); +}); + +describe('grep', () => { + test('hits are labelled by version, and disk-only files are reported as skipped', async (t) => { + const env = new Env(t); + const m = await env.open(); + await seedSynced(env, m, 'synced.txt', 'needle one'); + await seedSynced(env, m, 'draft.txt', 'nothing'); + await call(m, 'write', { path: 'work/draft.txt', content: 'needle two' }); + await seedSynced(env, m, 'conflict.txt', 'nothing'); + env.writeDisk('conflict.txt', 'needle on disk'); + await call(m, 'write', { path: 'work/conflict.txt', content: 'needle in the workspace' }); + await seedSynced(env, m, 'binary-conflict.dat', 'nothing'); + env.writeDisk('binary-conflict.dat', Buffer.from([0x6e, 0, 0x65, 0])); + await call(m, 'write', { path: 'work/binary-conflict.dat', content: 'needle beside a binary disk version' }); + env.writeDisk('shot.png', ONE_PX_PNG); + + const data = await call(m, 'grep', { pattern: 'needle' }); + const hits = (data.results as Array<{ file: string; state: string; version: string; matches: Array<{ text: string }> }>) + .map((r) => `${r.file} ${r.state} ${r.version}: ${r.matches[0]!.text}`); + assert.deepEqual(hits, [ + 'work/binary-conflict.dat conflict workspace: needle beside a binary disk version', + 'work/conflict.txt conflict workspace: needle in the workspace', + 'work/conflict.txt conflict conflicting-disk: needle on disk', + 'work/draft.txt workspace-draft workspace: needle two', + 'work/synced.txt synced workspace: needle one', + ]); + const skipped = data.skipped as Array<{ file: string; reason: string }>; + assert.deepEqual(skipped.map((s) => s.file), ['work/binary-conflict.dat', 'work/shot.png'], + "a conflict's binary disk version is no more searched than a disk-only file, and says so"); + assert.match(skipped[0]!.reason, /^conflict: the disk version is binary or over the size limit/); + + const one = await call(m, 'grep', { pattern: 'needle', path: 'work/draft.txt' }); + assert.deepEqual((one.results as Array<{ file: string }>).map((r) => r.file), ['work/draft.txt']); + }); + + test("once disk changes again, a conflict's stored disk version is labelled as recorded, and what disk holds now as not searched", async (t) => { + const env = new Env(t); + const m = await env.open(); + await seedSynced(env, m, 'a.txt', 'v1'); + env.writeDisk('a.txt', 'disk says ALPHA'); + await call(m, 'write', { path: 'work/a.txt', content: 'workspace says GAMMA' }); + const recorded = await entryOf(m, 'a.txt'); + assert.equal(recorded?.conflict?.kind, 'both-changed'); + assert.equal(recorded?.conflict?.diskCopy, 'stored'); + type Grep = { results: Array<{ file: string; state: string; version: string; matches: Array<{ text: string }> }>; skipped?: Array<{ file: string; reason: string }> }; + const hitsOf = (data: Grep) => data.results.map((r) => `${r.file} ${r.state} ${r.version}: ${r.matches[0]!.text}`); + + const current: Grep = await call(m, 'grep', { pattern: 'ALPHA' }); + assert.deepEqual(hitsOf(current), ['work/a.txt conflict conflicting-disk: disk says ALPHA'], 'while disk still holds it'); + assert.equal(current.skipped, undefined); + + env.writeDisk('a.txt', 'disk now says BETA'); + assert.equal((await entryOf(m, 'a.txt'))?.conflict?.diskChangedSinceRecorded, true); + const alpha: Grep = await call(m, 'grep', { pattern: 'ALPHA' }); + assert.deepEqual(hitsOf(alpha), ['work/a.txt conflict recorded-disk: disk says ALPHA'], 'no longer what disk says'); + const beta: Grep = await call(m, 'grep', { pattern: 'BETA' }); + assert.deepEqual(hitsOf(beta), []); + for (const data of [alpha, beta]) { + assert.deepEqual(data.skipped?.map((s) => s.file), ['work/a.txt'], 'what disk holds now is listed as not searched'); + assert.match(data.skipped![0]!.reason, /^conflict: disk changed since the conflict was recorded/); + } + + env.rmDisk('a.txt'); + const gone: Grep = await call(m, 'grep', { pattern: 'ALPHA' }); + assert.deepEqual(hitsOf(gone), ['work/a.txt conflict recorded-disk: disk says ALPHA'], 'disk now has no file at all'); + assert.equal(gone.skipped, undefined, 'and no disk file went unsearched'); + assert.equal(await contentOf(m, 'a.txt'), 'workspace says GAMMA', 'the draft is kept'); + }); +}); + +describe('grep, when the branch or the store changes under it', () => { + test("a pass whose branch is no longer selected isn't read with another branch's content: grep runs it again, then gives up", async (t) => { + const env = new Env(t); + const m = await env.open(); + await seedSynced(env, m, 'a.txt', 'needle on main'); + const main = env.store.currentBranch().name; + env.store.createBranch('other', main); + env.store.switchBranch('other'); + await call(m, 'write', { path: 'work/a.txt', content: 'needle on other' }); + env.store.switchBranch(main); + // Every pass is followed by a switch to the other branch before grep + // reads the tree that goes with it. + const internals = m as unknown as { passUnlocked: (...args: unknown[]) => Promise }; + const real = internals.passUnlocked.bind(m); + internals.passUnlocked = async (...args: unknown[]) => { + const result = await real(...args); + env.store.switchBranch(env.store.currentBranch().name === main ? 'other' : main); + return result; + }; + const data = await call(m, 'grep', { pattern: 'needle' }); + assert.deepEqual(data.results, [], JSON.stringify(data.results)); + assert.match(JSON.stringify(data.incomplete), /the selected branch kept changing while grep ran/); + }); + + test("a version missing from the store is listed as not searched, not taken for no match", async (t) => { + const env = new Env(t); + const m = await env.open(); + await seedSynced(env, m, 'conflict.txt', 'nothing'); + env.writeDisk('conflict.txt', 'needle on disk'); + await call(m, 'write', { path: 'work/conflict.txt', content: 'needle in the workspace' }); + const listed = (await call(m, 'ls', { path: 'work' })).entries.find((e: { name: string }) => e.name === 'conflict.txt'); + assert.equal(listed?.conflict?.diskCopy, 'stored'); + // Both stored versions go missing. + const store = env.store as unknown as { getBlob: (hash: string) => Buffer | null }; + const getBlob = store.getBlob.bind(env.store); + const gone = new Set([sha('needle on disk'), sha('needle in the workspace')]); + store.getBlob = (hash: string) => (gone.has(hash) ? null : getBlob(hash)); + const data = await call(m, 'grep', { pattern: 'needle' }); + assert.deepEqual(data.results, []); + const reasons = (data.skipped ?? []).map((x: { reason: string }) => x.reason).join('\n'); + assert.match(reasons, /the workspace copy is missing from the store/); + assert.match(reasons, /the disk version recorded with the conflict is missing from the store/); + }); +}); + +describe('on-agent-action', () => { + test('a completed tool batch brings disk changes in before the next inference', async (t) => { + const env = new Env(t); + const m = await env.open({ watch: 'on-agent-action' }); + await seedSynced(env, m, 'edited.txt', 'v1'); + await seedSynced(env, m, 'removed.txt', 'v1'); + env.writeDisk('edited.txt', 'v2, by a shell command'); + env.rmDisk('removed.txt'); + env.writeDisk('created.txt', 'new'); + + await m.onToolBatchComplete('agent'); + const blob = (p: string) => { + const e = env.store.treeGet(TREE, p); + return e ? env.store.getBlob(e.blobHash)?.toString('utf8') : null; + }; + assert.equal(blob('edited.txt'), 'v2, by a shell command'); + assert.equal(blob('removed.txt'), null); + assert.equal(blob('created.txt'), 'new'); + const status = await call(m, 'status', {}); + assert.equal(status.work.lastAgentActionScan.complete, true); + }); + + test('a scan that finishes without seeing everything says where, on status and as an event', { skip: IS_ROOT ? 'root reads every directory' : false }, async (t) => { + const env = new Env(t); + const m = await env.open({ watch: 'on-agent-action' }); + await seedSynced(env, m, 'locked/a.txt', 'v1'); + chmodSync(env.disk('locked'), 0o000); + try { + await m.onToolBatchComplete('agent'); + } finally { + chmodSync(env.disk('locked'), 0o755); + } + const scan = (await call(m, 'status', {})).work.lastAgentActionScan; + assert.equal(scan.complete, true, 'it finished'); + assert.equal(scan.withinDeadline, true); + assert.deepEqual(scan.incomplete, [{ path: 'locked', reason: 'cannot list (EACCES)' }]); + const event = env.events.find((e) => e.type === 'workspace:agent-action-scan-incomplete') as unknown as + | { mount: string; incomplete: Array<{ path: string; reason: string }> } | undefined; + assert.equal(event?.mount, 'work'); + assert.deepEqual(event?.incomplete, [{ path: 'locked', reason: 'cannot list (EACCES)' }]); + }); + + test('a scan that outlasts its deadline is reported incomplete, and still finishes', async (t) => { + const env = new Env(t); + const m = await env.open({ watch: 'on-agent-action' }, { agentActionScanDeadlineMs: 50 }); + env.writeDisk('late.txt', 'arrives late'); + const mount = (m as any).mounts.get('work'); + const held = (m as any).withMount(mount, () => sleep(400)); // another pass holds the mount + + await m.onToolBatchComplete('agent'); + let status = await call(m, 'status', {}); + assert.equal(status.work.lastAgentActionScan.complete, false); + assert.equal(status.work.lastAgentActionScan.withinDeadline, false); + assert.match(status.work.lastAgentActionScan.reason, /deadline exceeded \(50 ms\); the scan continues/); + assert.ok(env.events.some((e) => e.type === 'workspace:agent-action-scan-incomplete')); + assert.equal(env.store.treeGet(TREE, 'late.txt'), null, 'nothing was claimed before the scan ran'); + + await held; + await (m as any).mountTurns.get('work'); + assert.notEqual(env.store.treeGet(TREE, 'late.txt'), null, 'the scan finished afterwards'); + status = await call(m, 'status', {}); + assert.deepEqual( + { complete: status.work.lastAgentActionScan.complete, withinDeadline: status.work.lastAgentActionScan.withinDeadline, reason: status.work.lastAgentActionScan.reason }, + { complete: true, withinDeadline: false, reason: 'finished after the deadline' }, + ); + }); + + test('batches that come while a scan waits share it: one runs, at most one waits', async (t) => { + const env = new Env(t); + const m = await env.open({ watch: 'on-agent-action' }, { agentActionScanDeadlineMs: 5_000 }); + const mod = m as any; + const mount = mod.mounts.get('work'); + let passes = 0; + const pass = mod.passUnlocked.bind(mod); + mod.passUnlocked = (...args: unknown[]) => { passes++; return pass(...args); }; + const held = mod.withMount(mount, () => sleep(200)); // another pass holds the mount + env.writeDisk('a.txt', 'from the batches'); + await Promise.all([m.onToolBatchComplete('agent'), m.onToolBatchComplete('agent'), m.onToolBatchComplete('agent')]); + await held; + assert.equal(passes, 1, 'three batches, one scan'); + assert.notEqual(env.store.treeGet(TREE, 'a.txt'), null); + assert.equal((await call(m, 'status', {})).work.lastAgentActionScan.withinDeadline, true); + }); + + test("while a scan runs past its deadline, a later batch isn't held, and its scan comes after", async (t) => { + const env = new Env(t); + const m = await env.open({ watch: 'on-agent-action' }, { agentActionScanDeadlineMs: 50 }); + const mod = m as any; + const mount = mod.mounts.get('work'); + let passes = 0; + const pass = mod.passUnlocked.bind(mod); + mod.passUnlocked = (...args: unknown[]) => { passes++; return pass(...args); }; + const held = mod.withMount(mount, () => sleep(400)); + + await m.onToolBatchComplete('agent'); // held to its deadline + env.writeDisk('later.txt', 'from the second batch'); + const started = Date.now(); + await m.onToolBatchComplete('agent'); + assert.ok(Date.now() - started < 40, 'the second batch was not held'); + let scan = (await call(m, 'status', {})).work.lastAgentActionScan; + assert.match(scan.reason, /an earlier scan is still running past its deadline/); + + await held; + await mod.mountTurns.get('work'); + assert.equal(passes, 1, 'the second batch shared the scan still waiting'); + assert.notEqual(env.store.treeGet(TREE, 'later.txt'), null, "the second batch's change was scanned"); + // The late scan is over (its bookkeeping settles a few ticks after the + // mount's turn), so the next batch is held for its own scan again, which + // may itself miss 50 ms on a loaded machine: that's its own miss. + await new Promise((resolve) => setImmediate(resolve)); + await m.onToolBatchComplete('agent'); + scan = (await call(m, 'status', {})).work.lastAgentActionScan; + assert.doesNotMatch(scan.reason ?? '', /an earlier scan is still running/); + }); + + test("a scan queued behind a late one puts its own outcome on status, which the late one's doesn't overwrite", async (t) => { + const env = new Env(t); + const m = await env.open({ watch: 'on-agent-action' }, { agentActionScanDeadlineMs: 50 }); + const mod = m as any; + // Stand-ins for the two scans' passes: the first runs, then waits to be + // released, and each says a different region couldn't be observed. + const pass = mod.passUnlocked.bind(mod); + const gates = [0, 1].map(() => { let open!: () => void; const shut = new Promise((resolve) => { open = resolve; }); return { open, shut }; }); + let passes = 0; + mod.passUnlocked = async (...args: unknown[]) => { + const n = ++passes; + const result = await pass(...args); + await gates[n - 1]!.shut; + return { ...result, incomplete: [{ path: n === 1 ? 'FIRST' : 'SECOND', reason: 'stand-in' }] }; + }; + const events: Array<{ type: string; incomplete?: Array<{ path: string }> }> = []; + mod.ctx = { ...mod.ctx, pushEvent: (e: { type: string }) => { events.push(e); } }; + // A failed assertion mustn't leave a scan waiting on its gate: the + // module's stop at cleanup would wait for it. + try { + await m.onToolBatchComplete('agent'); // its scan starts, and runs past the deadline + await m.onToolBatchComplete('agent'); // not held: its own scan is queued behind the first + const status = async () => (await call(m, 'status', {})).work.lastAgentActionScan; + assert.match((await status()).reason, /an earlier scan is still running past its deadline/); + gates[0]!.open(); + for (let i = 0; i < 100 && passes < 2; i++) await sleep(10); + await new Promise((resolve) => setImmediate(resolve)); + // The first scan is done and the second runs: the first's outcome is not + // the second batch's. + assert.match((await status()).reason, /an earlier scan is still running past its deadline/, "the older scan doesn't speak for the newer batch"); + gates[1]!.open(); + await mod.mountTurns.get('work'); + await new Promise((resolve) => setImmediate(resolve)); + assert.equal(passes, 2, 'the second batch had a scan of its own'); + const last = await status(); + assert.equal(last.complete, true); + assert.equal(last.withinDeadline, false, 'its batch was never held for it'); + assert.deepEqual(last.incomplete?.map((r: { path: string }) => r.path), ['SECOND'], JSON.stringify(last)); + const regions = events.flatMap((e) => (e.incomplete ?? []).map((r) => r.path)); + assert.deepEqual(regions.sort(), ['FIRST', 'SECOND'], 'each scan still reports its regions as an event'); + } finally { + for (const gate of gates) gate.open(); + } + }); + + test("a batch that comes while a late scan's successor still runs isn't held either", async (t) => { + const env = new Env(t); + const m = await env.open({ watch: 'on-agent-action' }, { agentActionScanDeadlineMs: 300 }); + const mod = m as any; + // Each scan's pass runs, then waits to be released. + const pass = mod.passUnlocked.bind(mod); + const gates = [0, 1, 2].map(() => { let open!: () => void; const shut = new Promise((resolve) => { open = resolve; }); return { open, shut }; }); + let passes = 0; + mod.passUnlocked = async (...args: unknown[]) => { + const n = ++passes; + const result = await pass(...args); + await gates[n - 1]!.shut; + return result; + }; + try { + await m.onToolBatchComplete('agent'); // its scan runs past the deadline + await m.onToolBatchComplete('agent'); // not held: its scan is queued behind the first, late from the start + gates[0]!.open(); // the first scan finishes, and the second starts + for (let i = 0; i < 100 && passes < 2; i++) await sleep(10); + await new Promise((resolve) => setImmediate(resolve)); + assert.equal(passes, 2, 'the second scan runs'); + const started = Date.now(); + await m.onToolBatchComplete('agent'); // while the late second scan still runs + const held = Date.now() - started; + assert.ok(held < 150, `the third batch was not held (${held} ms, deadline 300 ms)`); + assert.match((await call(m, 'status', {})).work.lastAgentActionScan.reason, /an earlier scan is still running past its deadline/); + } finally { + for (const gate of gates) gate.open(); + } + }); + + test("a scan deadline within 1 s of the framework's hook bound is lowered below it, with a warning", async (t) => { + const env = new Env(t); + const m = await env.open({ watch: 'on-agent-action' }, { agentActionScanDeadlineMs: 60_000 }); + const warnings: string[] = []; + const original = console.warn; + console.warn = (...args: unknown[]) => { warnings.push(args.map(String).join(' ')); }; + try { + assert.equal((m as any).agentActionScanDeadlineMs(), TOOL_BATCH_HOOK_TIMEOUT_MS - 1_000); + assert.equal((m as any).agentActionScanDeadlineMs(), TOOL_BATCH_HOOK_TIMEOUT_MS - 1_000); + } finally { + console.warn = original; + } + assert.equal(warnings.filter((w) => w.includes('agentActionScanDeadlineMs 60000')).length, 1, 'warned once'); + }); + + test('a branch switch while a pass reads disk makes it gather again, on the branch it then decides on', async (t) => { + const env = new Env(t); + const m = await env.open(); + await seedSynced(env, m, 'a.txt', 'A'); + const main = env.store.currentBranch().name; + env.store.createBranch('other', main); + env.store.switchBranch('other'); + await call(m, 'write', { path: 'work/only-other.txt', content: 'a draft on other' }); + env.store.switchBranch(main); + + // The listing that lived through the switch is the one judged: a.txt's + // fingerprint is too fresh to vouch, so the pass reads it. + const data = await withReadHook((n) => { if (n === 1) env.store.switchBranch('other'); }, + () => call(m, 'ls', { path: 'work', recursive: true })); + assert.equal(env.store.currentBranch().name, 'other'); + assert.equal(data.incomplete, undefined); + const entries = new Map((data.entries as Entry[]).map((e) => [e.path, e.state])); + assert.equal(entries.get('only-other.txt'), 'workspace-draft', "the new branch's draft is decided, not omitted"); + assert.equal(entries.get('a.txt'), 'synced'); + }); + + test('a branch that keeps changing under a pass leaves it explicitly incomplete', async (t) => { + const env = new Env(t); + const m = await env.open(); + await seedSynced(env, m, 'a.txt', 'A'); + const main = env.store.currentBranch().name; + env.store.createBranch('other', main); + const data = await withReadHook((n) => env.store.switchBranch(n % 2 === 1 ? 'other' : main), + () => call(m, 'ls', { path: 'work', recursive: true })); + assert.deepEqual(data.entries, []); + assert.deepEqual(data.incomplete, [{ path: '', reason: 'the selected branch kept changing while the scan ran' }]); + }); + + test('a branch switched away during the root check and back during a read is decided on the branch enumerated', async (t) => { + const env = new Env(t); + const m = await env.open(); + await seedSynced(env, m, 'a.txt', 'A'); + const main = env.store.currentBranch().name; + env.store.createBranch('other', main); + await call(m, 'write', { path: 'work/only-main.txt', content: 'a draft on main' }); + + // The root check locates the root and reads its identity (lstat of its real path). + const fsp = createRequire(import.meta.url)('node:fs/promises') as { lstat: (...args: unknown[]) => Promise }; + const lstat = fsp.lstat; + const rootReal = realpathSync(env.dir); + let first = true; + fsp.lstat = async function (path: unknown, ...rest: unknown[]) { + const result = await lstat(path, ...rest); + if (first && path === rootReal) { + first = false; + env.store.switchBranch('other'); // A -> B before the candidates are read + } + return result; + }; + syncBuiltinESMExports(); + try { + const data = await withReadHook((n) => { if (n === 1) env.store.switchBranch(main); }, // B -> A during a read + () => call(m, 'ls', { path: 'work', recursive: true })); + assert.equal(first, false, 'the branch switched during the root check'); + assert.equal(env.store.currentBranch().name, main); + const states = new Map((data.entries as Entry[]).map((e) => [e.path, e.state])); + assert.equal(states.get('only-main.txt'), 'workspace-draft', "main's draft is in main's listing"); + } finally { + fsp.lstat = lstat; + syncBuiltinESMExports(); + } + }); + + test('a push whose selection was made on another branch writes nothing', async (t) => { + const env = new Env(t); + const m = await env.open(); + await call(m, 'write', { path: 'work/a.txt', content: 'A' }); + const mount = (m as any).mounts.get('work'); + const pushed = await pushPaths(env.store, (m as any).agreement, await (m as any).runtime(mount), ['a.txt'], { branchId: 'not-this-branch' }); + assert.deepEqual(pushed.written, []); + assert.match(pushed.skipped[0]!.reason, /selected branch changed/); + assert.throws(() => env.readDisk('a.txt'), /ENOENT/); + }); + + test('a branch change while the scan waits applies to the branch current when it decides', async (t) => { + const env = new Env(t); + const m = await env.open({ watch: 'on-agent-action' }); + await seedSynced(env, m, 'a.txt', 'v1'); + const main = env.store.currentBranch().name; + env.store.createBranch('other', main); + env.writeDisk('new.txt', 'made during the batch'); + const mount = (m as any).mounts.get('work'); + const held = (m as any).withMount(mount, () => sleep(100)); + const scan = m.onToolBatchComplete('agent'); + env.store.switchBranch('other'); // the selection changes before the scan decides + await held; + await scan; + + assert.notEqual(env.store.treeGet(TREE, 'new.txt'), null, 'adopted on the branch current at decision time'); + env.store.switchBranch(main); + assert.equal(env.store.treeGet(TREE, 'new.txt'), null, 'the old selection was not written with the observation'); + assert.equal(await stateOf(m, 'new.txt'), 'not-in-branch'); + }); +}); + +describe('materialize, status and shutdown', () => { + test("a materialize between initStore and start keeps its pin: the saved one, older, isn't applied over it", async (t) => { + const env = new Env(t); + const first = await env.open(); + await call(first, 'write', { path: 'work/a.txt', content: 'A' }); + await call(first, 'materialize', { path: 'work/a.txt' }); + let saved: unknown; + (first as any).ctx.setState = (state: unknown) => { saved = state; }; + await env.shutdown(first); + env.store = JsStore.openOrCreate({ path: env.storePath }); + // The next process materializes before its start, then starts as a restart. + const m = new WorkspaceModule({ mounts: [{ name: 'work', path: env.dir, mode: 'read-write', watch: 'never' }] }); + m.initStore(env.store); + await call(m, 'write', { path: 'work/b.txt', content: 'B' }); + await call(m, 'materialize', { path: 'work/b.txt' }); + const mount = (m as any).mounts.get('work'); + const pinned = { seq: mount.lastMaterializedSeq, branch: mount.lastMaterializedBranchId }; + assert.notEqual(pinned.seq, (saved as { mounts: { work: { lastMaterializedSeq: number } } }).mounts.work.lastMaterializedSeq); + await m.start({ isRestart: true, getState: () => saved, setState: () => {}, pushEvent: () => {} } as unknown as ModuleContext); + (env as any).live.push(m); + assert.deepEqual({ seq: mount.lastMaterializedSeq, branch: mount.lastMaterializedBranchId }, pinned); + }); + + test('the module takes no work after it has stopped and saved its pins, until it starts again', async (t) => { + const env = new Env(t); + const m = await env.open(); + await call(m, 'write', { path: 'work/a.txt', content: 'A' }); + let saved: { mounts: Record } | undefined; + (m as any).ctx.setState = (state: typeof saved) => { saved = state; }; + await m.stop(); + // After stop has saved the pins and watermarks, nothing more changes them. + const after = await m.handleToolCall({ id: 't', name: 'materialize', input: { path: 'work/a.txt' } }); + assert.equal(after.success, false); + assert.match(String(after.error), /has stopped/); + assert.equal(existsSync(env.disk('a.txt')), false, 'nothing was written after the stop'); + assert.equal((await m.writeBinary('work/late.bin', Buffer.from('x'), 'application/octet-stream')).success, false); + assert.deepEqual(await m.readBinary('work/a.txt'), { error: 'The workspace has stopped: it takes no more work until it starts again' }); + assert.deepEqual(await m.materializeMount('work'), []); + assert.equal(existsSync(env.disk('a.txt')), false); + assert.ok(saved, 'stop saved its state'); + // Started again, it takes work again. + await m.start({ isRestart: true, getState: () => saved, setState: () => {}, pushEvent: () => {} } as unknown as ModuleContext); + await call(m, 'materialize', { path: 'work/a.txt' }); + assert.equal(env.readDisk('a.txt'), 'A'); + }); + + test('a materialize queued behind a pass is guarded on the branch it then selects on', async (t) => { + const env = new Env(t); + const m = await env.open(); + await seedSynced(env, m, 'base.txt', 'base'); + const forkAt = env.store.currentSequence(); + await seedSynced(env, m, 'newer.txt', 'newer on main'); // pinned past the fork + const main = env.store.currentBranch().name; + env.store.createBranchAt('divergent', main, forkAt); + env.store.switchBranch('divergent'); + await call(m, 'write', { path: 'work/div.txt', content: 'divergent' }); + env.store.switchBranch(main); + + const mount = (m as any).mounts.get('work'); + const held = (m as any).withMount(mount, () => sleep(100)); + const queued = m.handleToolCall({ id: 't', name: 'materialize', input: {} }); + env.store.switchBranch('divergent'); // while it waits for its turn + await held; + const res = await queued; + assert.equal(res.success, false, 'refused on the branch it would have written'); + assert.match(String(res.error), /diverged/); + assert.equal(existsSync(env.disk('div.txt')), false, 'nothing written'); + }); + + test('a materialize whose push is refused on a branch change leaves the pin, so the next one there is still guarded', async (t) => { + const env = new Env(t); + const m = await env.open(); + await seedSynced(env, m, 'base.txt', 'base'); + const forkAt = env.store.currentSequence(); + await seedSynced(env, m, 'newer.txt', 'newer on main'); // pinned past the fork + const main = env.store.currentBranch(); + env.store.createBranchAt('divergent', main.name, forkAt); + env.store.switchBranch('divergent'); + await call(m, 'write', { path: 'work/div.txt', content: 'divergent' }); + env.store.switchBranch(main.name); + await call(m, 'write', { path: 'work/base.txt', content: 'base v2' }); // owed on main + const mount = (m as any).mounts.get('work'); + const before = { pin: mount.lastMaterializedBranchId, seq: mount.lastMaterializedSeq }; + assert.equal(before.pin, main.id); + + // The selection is made on main; the branch changes while the push reads disk. + let switched = false; + const data = await withFsPromises('lstat', (lstat) => async (...args: any[]) => { + if (!switched && String(args[0]).endsWith('base.txt')) { + switched = true; + env.store.switchBranch('divergent'); + } + return lstat(...args); + }, () => call(m, 'materialize', {})); + assert.equal(switched, true, 'the branch changed while the push read disk'); + assert.deepEqual(data.materialized, []); + assert.match(JSON.stringify(data.skipped), /selected branch changed/); + assert.deepEqual({ pin: mount.lastMaterializedBranchId, seq: mount.lastMaterializedSeq }, before, 'pin and watermark as they were'); + assert.equal(env.readDisk('base.txt'), 'base'); + + // So the next materialize, on the divergent branch, is still refused. + assert.match(await refused(m, 'materialize', {}), /diverged/); + assert.equal(existsSync(env.disk('div.txt')), false, 'nothing of the divergent branch written'); + }); + + test('a first materialize refused on a branch change leaves an unpinned mount unpinned, though entries were in place', async (t) => { + const env = new Env(t); + const m = await env.open({ watch: 'on-agent-action' }); + env.writeDisk('a.txt', 'A'); + await m.onToolBatchComplete('agent'); // adopted: in place on main, never materialized + const mount = (m as any).mounts.get('work'); + assert.equal(mount.lastMaterializedBranchId, null, 'never pinned'); + const main = env.store.currentBranch().name; + env.store.createBranch('other', main); + await call(m, 'write', { path: 'work/b.txt', content: 'B' }); // owed on main + const seq = mount.lastMaterializedSeq; + + let switched = false; + const data = await withFsPromises('lstat', (lstat) => async (...args: any[]) => { + if (!switched && String(args[0]).endsWith('b.txt')) { + switched = true; + env.store.switchBranch('other'); + } + return lstat(...args); + }, () => call(m, 'materialize', {})); + assert.equal(switched, true, 'the branch changed while the push read disk'); + assert.deepEqual(data.materialized, []); + assert.equal(mount.lastMaterializedBranchId, null, 'still unpinned'); + assert.equal(mount.lastMaterializedSeq, seq, 'watermark as it was'); + }); + + test('status counts a workspace deletion as pending until a push applies it', async (t) => { + const env = new Env(t); + const m = await env.open(); + await seedSynced(env, m, 'gone.txt', 'v1'); + await call(m, 'delete', { path: 'work/gone.txt' }); + await call(m, 'materialize', {}); // left on disk: applyDeletions wasn't given + let status = (await call(m, 'status', {})).work; + assert.deepEqual([status.pendingChanges, status.pendingWorkspaceDeletions], [1, 1]); + await call(m, 'materialize', { applyDeletions: true }); + status = (await call(m, 'status', {})).work; + assert.deepEqual([status.pendingChanges, status.pendingWorkspaceDeletions], [0, 0]); + }); + + test("status counts a push that didn't reach disk as pending, past the watermark", async (t) => { + const env = new Env(t); + const m = await env.open(); + await seedSynced(env, m, 'a.txt', 'v1'); + await call(m, 'write', { path: 'work/a.txt', content: 'v2' }); + const failed = await withSyncs(env, () => true, () => call(m, 'materialize', {})); + assert.deepEqual(failed.result.materialized, []); + assert.equal((await call(m, 'status', {})).work.pendingChanges, 1, 'the edit is still owed'); + await call(m, 'materialize', {}); + assert.equal((await call(m, 'status', {})).work.pendingChanges, 0); + }); + + test('stop waits for work queued behind what was in flight when it began', async (t) => { + const env = new Env(t); + const m = await env.open(); + const mount = (m as any).mounts.get('work'); + void (m as any).withMount(mount, () => sleep(100)); + const stopping = m.stop(); + await sleep(10); + let finished = false; + // Appended after stop began, as a watcher callback or a late scan would be. + void (m as any).withMount(mount, async () => { + await sleep(100); + finished = true; + }); + await stopping; + assert.equal(finished, true, 'nothing still runs once stop returns'); + }); +}); + +describe('a named mount, and a path the workspace and disk hold differently', () => { + test("status reports the mount it's asked about, and a materialize of a read-only mount says it wrote nothing there", async (t) => { + const env = new Env(t); + const second = join(env.root, 'second'); + mkdirSync(second); + const m = await env.open({}, { mounts: [ + { name: 'work', path: env.dir, mode: 'read-write', watch: 'never' }, + { name: 'second', path: second, mode: 'read-only', watch: 'never' }, + ] } as Partial); + const status = await call(m, 'status', { mount: 'work' }); + assert.deepEqual(Object.keys(status.mounts ?? status), ['work'], JSON.stringify(status)); + assert.match(await refused(m, 'status', { mount: 'nowhere' }), /Unknown mount: nowhere/); + const done = await call(m, 'materialize', { mount: 'second' }); + assert.equal(done.count, 0); + assert.match(JSON.stringify(done.skipped), /a read-only mount: materialize writes nothing to it/); + }); + + test('a path sync takes in a file disk has where the workspace holds a directory, and drops what was beneath', async (t) => { + const env = new Env(t); + const m = await env.open(); + await seedSynced(env, m, 'd/x.txt', 'x'); + rmSync(env.disk('d'), { recursive: true }); + env.writeDisk('d', 'now a file'); + const data = await call(m, 'sync', { path: 'work/d' }); + assert.equal(env.store.treeGet('workspace/work/tree', 'd/x.txt'), null, 'what was beneath is gone'); + const entry = env.store.treeGet('workspace/work/tree', 'd'); + assert.ok(entry, `the file is taken in: ${JSON.stringify(data)}`); + assert.equal(env.store.getBlob(entry!.blobHash)?.toString('utf8'), 'now a file'); + }); + + test("a path sync where disk has a directory over a workspace file takes in what's beneath, and says the file wasn't checked", async (t) => { + const env = new Env(t); + const m = await env.open(); + await seedSynced(env, m, 'f', 'a file'); + rmSync(env.disk('f')); + mkdirSync(env.disk('f')); + env.writeDisk('f/y.txt', 'y'); + const data = await call(m, 'sync', { path: 'work/f' }); + assert.ok(env.store.treeGet('workspace/work/tree', 'f/y.txt'), 'what disk has beneath is taken in'); + assert.match(JSON.stringify(data.skipped ?? []), /"path":"f".*disk could not be checked: a directory/); + }); +}); + +describe("Mythos's shape", () => { + test('one new PNG and three deleted logs in a 350-entry directory', async (t) => { + const env = new Env(t); + const m = await env.open(); + const name = (i: number) => `files/tmp/log-${String(i).padStart(3, '0')}.txt`; + for (let i = 0; i < 350; i++) env.writeDisk(name(i), `log ${i}`); + await call(m, 'sync', {}); + + env.writeDisk('files/tmp/screenshot.png', ONE_PX_PNG); + for (const i of [7, 123, 300]) env.rmDisk(name(i)); + + const data = await call(m, 'ls', { path: 'work/files/tmp' }); + const entries = data.entries as Array<{ name: string; state: string; mimeType?: string }>; + assert.equal(entries.length, 348); + assert.deepEqual(entries.find((e) => e.name === 'screenshot.png'), + { name: 'screenshot.png', type: 'file', state: 'disk-only', size: ONE_PX_PNG.length, mimeType: 'image/png' }); + for (const i of [7, 123, 300]) { + assert.equal(entries.some((e) => e.name === `log-${String(i).padStart(3, '0')}.txt`), false, `log ${i} is gone`); + } + assert.equal(entries.filter((e) => e.state === 'synced').length, 347); + }); +}); + +// A field report from a host on main: after a restart lost the watermark, a +// bare `materialize` pushing one new file wrote the whole tree, a stale entry +// over its newer disk copy included. +describe('a bare materialize after a restart', () => { + /** The paths each push of `m` was asked to take up, sorted. */ + function recordPushes(m: WorkspaceModule): string[][] { + const seen: string[][] = []; + const push = (m as any).pushUnlocked.bind(m); + (m as any).pushUnlocked = (mount: unknown, paths: string[], opts: unknown) => { + seen.push([...paths].sort()); + return push(mount, paths, opts); + }; + return seen; + } + + test('refuses a path with no evidence whose disk copy differs, and writes a new file', async (t) => { + const env = new Env(t); + let m = await env.open(); + await seedSynced(env, m, 'old.txt', 'materialized before the restart'); + await call(m, 'materialize', {}); + // A tree entry from before the evidence (no P, no #169 baseline), stale: + // its disk copy was appended in a shell, and no pass has observed it. + env.legacyEntry('notes.txt', 'line 1\n'); + env.writeDisk('notes.txt', 'line 1\nline 2, appended in a shell\n'); + m = await env.restart(m); + assert.equal((await call(m, 'status', {})).work.lastMaterializedSeq, 0, 'the restart lost the watermark'); + + await call(m, 'write', { path: 'work/new.txt', content: 'one small new file' }); + const refusal = { + mount: 'work', + reason: 'notes.txt: disk holds a different copy, and nothing records which is newer (conflict: unknown-provenance) — ' + + 'sync this path to adopt the disk version, or materialize this path with force to overwrite it', + }; + const res = await call(m, 'materialize', {}); + assert.deepEqual(res.materialized.map((w: { path: string }) => w.path), ['new.txt']); + assert.equal(env.readDisk('new.txt'), 'one small new file'); + assert.equal(env.readDisk('notes.txt'), 'line 1\nline 2, appended in a shell\n', 'the disk copy is untouched'); + assert.deepEqual(res.skipped, [refusal]); + assert.equal((await entryOf(m, 'notes.txt'))?.conflict?.kind, 'unknown-provenance'); + + // Nothing tells that disk copy from an older one, so force alone doesn't overwrite it either. + const forced = await call(m, 'materialize', { force: true }); + assert.equal(forced.count, 0); + assert.deepEqual(forced.skipped, [refusal]); + assert.equal(env.readDisk('notes.txt'), 'line 1\nline 2, appended in a shell\n'); + await call(m, 'materialize', { path: 'work/notes.txt', force: true }); + assert.equal(env.readDisk('notes.txt'), 'line 1\n', 'naming the path with force does'); + }); + + test('a whole-mount restore after a branch switch still overwrites a path with no evidence', async (t) => { + const env = new Env(t); + const m = await env.open(); + env.legacyEntry('settings.json', '{"restored": true}'); + env.writeDisk('settings.json', '{"restored": false}'); + // materializeMount is the deliberate restore that follows an undo or branch switch: its scope is named. + assert.deepEqual(await m.materializeMount('work'), ['settings.json']); + assert.equal(env.readDisk('settings.json'), '{"restored": true}'); + }); + + test('takes up only what the evidence says disk owes, and never reverts a newer disk copy', async (t) => { + const env = new Env(t); + let m = await env.open(); + for (const name of ['a.txt', 'b.txt', 'c.txt']) await seedSynced(env, m, name, `${name} v1`); + await call(m, 'materialize', {}); + // Disk moves on after it agreed; the tree hasn't seen it. + env.writeDisk('b.txt', 'b.txt v1, appended in a shell'); + m = await env.restart(m); + + await call(m, 'write', { path: 'work/new.txt', content: 'new' }); + const pushes = recordPushes(m); + const forced = await call(m, 'materialize', { force: true }); + assert.deepEqual(pushes, [['new.txt']], 'only the new file is taken up, even with force'); + assert.deepEqual(forced.materialized.map((w: { path: string }) => w.path), ['new.txt']); + assert.equal(env.readDisk('b.txt'), 'b.txt v1, appended in a shell', 'force without a path does not revert it'); + + const bare = await call(m, 'materialize', {}); + assert.deepEqual(pushes[1], [], 'nothing is owed'); + assert.equal(bare.count, 0); + assert.equal(bare.skipped, undefined); + + await call(m, 'materialize', { path: 'work/b.txt', force: true }); + assert.equal(env.readDisk('b.txt'), 'b.txt v1', 'naming the path with force does'); + }); + + test('pins the branch from the evidence when nothing is owed', async (t) => { + const env = new Env(t); + let m = await env.open(); + await seedSynced(env, m, 'a.txt', 'a'); + const forkAt = env.store.currentSequence(); + await seedSynced(env, m, 'b.txt', 'b'); + m = await env.restart(m); + assert.equal((await call(m, 'status', {})).work.lastMaterializedBranch, null, 'the restart lost the pin'); + + const first = await call(m, 'materialize', {}); + assert.equal(first.count, 0, 'nothing is owed'); + const main = env.store.currentBranch(); + assert.equal((await call(m, 'status', {})).work.lastMaterializedBranch, main.id, 'disk holds this branch: pinned'); + + // A branch that diverged before b.txt is refused, not let through. + env.store.createBranchAt('child', main.name, forkAt); + env.store.switchBranch('child'); + await call(m, 'write', { path: 'work/a.txt', content: 'a from child' }); + assert.match(await refused(m, 'materialize', {}), /diverged/); + assert.equal(env.readDisk('a.txt'), 'a'); + }); + + test('status counts what disk owes by the evidence, not every tree change since the watermark', async (t) => { + const env = new Env(t); + const m = await env.open(); + await seedSynced(env, m, 'a.txt', 'v1'); + await call(m, 'materialize', {}); + env.writeDisk('a.txt', 'v2 from a shell'); + await call(m, 'sync', {}); + assert.equal(await contentOf(m, 'a.txt'), 'v2 from a shell', 'disk adopted: a tree change disk already holds'); + // An entry from before the evidence is unchecked, not unpushed. + env.legacyEntry('legacy.txt', 'never checked'); + assert.equal((await call(m, 'status', {})).work.pendingChanges, 0); + + await call(m, 'write', { path: 'work/a.txt', content: 'v3' }); + assert.equal((await call(m, 'status', {})).work.pendingChanges, 1); + }); +}); + +describe('the mount root', () => { + /** What a drive's unmount looks like from the mount: the root goes, and the watcher makes an empty one. */ + function unmount(env: Env): string { + const away = `${env.dir}-away`; + renameSync(env.dir, away); // kept, so the new root can't reuse its inode + mkdirSync(env.dir); + return away; + } + function remount(env: Env, away: string): void { + rmSync(env.dir, { recursive: true }); + renameSync(away, env.dir); + } + const identityOf = (path: string) => { + const s = statSync(path, { bigint: true }); + return { dev: String(s.dev), ino: String(s.ino) }; + }; + + test('the root disk agreed under is recorded with the first evidence', async (t) => { + const env = new Env(t); + const m = await env.open(); + assert.equal((m as any).agreement.rootIdentity('work'), undefined, 'nothing recorded before any evidence'); + await seedSynced(env, m, 'a.txt', 'A'); + assert.deepEqual((m as any).agreement.rootIdentity('work'), identityOf(env.dir)); + await seedSynced(env, m, 'b.txt', 'B'); + await call(m, 'sync', {}); + const rootEntries = (m as any).agreement.journal.load().entries.filter((e: any) => e.entry.t === 'root'); + assert.equal(rootEntries.length, 1, 'recorded once, not with every piece of evidence'); + }); + + test('a root replaced by an empty directory is unavailable, and the tree keeps every entry', async (t) => { + const env = new Env(t); + const m = await env.open(); + await seedSynced(env, m, 'a.txt', 'A'); + await seedSynced(env, m, 'sub/b.txt', 'B'); + const away = unmount(env); + + await (m as any).initialScan('work'); // the reattach + assert.notEqual(env.store.treeGet(TREE, 'a.txt'), null, 'a.txt is still in the tree'); + assert.notEqual(env.store.treeGet(TREE, 'sub/b.txt'), null, 'sub/b.txt is still in the tree'); + assert.equal(env.eventsOf('workspace:deleted').length, 0, 'no deletion was announced'); + const synced = await call(m, 'sync', {}); + assert.equal(synced.totalSynced, 0, 'a plain sync adopts nothing either'); + assert.match(synced.incomplete?.[0]?.reason ?? '', /not the directory disk last agreed with/); + assert.match(synced.incomplete[0].reason, /acceptRoot/, 'the reason names the remedy'); + assert.notEqual(env.store.treeGet(TREE, 'a.txt'), null, 'still in the tree after the sync'); + + remount(env, away); + await (m as any).initialScan('work'); + assert.equal(await stateOf(m, 'a.txt'), 'synced', 'back, and agreed as before'); + assert.equal(await stateOf(m, 'sub/b.txt'), 'synced'); + }); + + test('the recorded root survives a restart', async (t) => { + const env = new Env(t); + let m = await env.open(); + await seedSynced(env, m, 'a.txt', 'A'); + await env.shutdown(m); + unmount(env); // a host that starts while the drive is away + env.store = JsStore.openOrCreate({ path: env.storePath }); + m = await env.open(); + await (m as any).initialScan('work'); + assert.notEqual(env.store.treeGet(TREE, 'a.txt'), null, 'still in the tree'); + assert.equal(env.eventsOf('workspace:deleted').length, 0, 'no deletion was announced'); + }); + + test('a replaced root takes no writes, forced or not', async (t) => { + const env = new Env(t); + const m = await env.open(); + await seedSynced(env, m, 'a.txt', 'A'); + await call(m, 'write', { path: 'work/c.txt', content: 'C, a draft' }); + await call(m, 'write', { path: 'work/sub/d.txt', content: 'D, a draft' }); + unmount(env); + mkdirSync(env.disk('sub')); // a mountpoint with leftovers of its own: a directory a write could reach + + for (const input of [{}, { force: true }, { path: 'work/c.txt', force: true }, { path: 'work/sub/d.txt', force: true }]) { + const res = await m.handleToolCall({ id: 't', name: 'materialize', input }); + assert.equal(existsSync(env.disk('c.txt')), false, `${JSON.stringify(input)} wrote nothing`); + assert.equal(existsSync(env.disk('sub/d.txt')), false, `${JSON.stringify(input)} wrote nothing beneath the root`); + assert.equal(existsSync(env.disk('a.txt')), false, `${JSON.stringify(input)} restored nothing`); + assert.match(JSON.stringify(res), /not the directory disk last agreed with/, `${JSON.stringify(input)} says why`); + } + }); + + for (const op of ['materialize', 'sync'] as const) { + test(`the root is recorded before the first evidence, so a ${op} killed after its intent leaves both`, async (t) => { + const env = new Env(t); + if (op === 'sync') env.writeDisk('a.txt', 'A, a shell file to adopt'); + let m = await env.open(); + if (op === 'materialize') await call(m, 'write', { path: 'work/a.txt', content: 'A, a draft' }); + assert.equal((m as any).agreement.paths('work').length, 0, 'no evidence before the operation'); + m = await env.fault(m, op, 'after-intent'); + assert.equal((m as any).agreement.get('work', 'a.txt')?.kind, 'pending', 'the intent is durable'); + assert.deepEqual((m as any).agreement.rootIdentity('work'), identityOf(env.dir), 'and so is the root it was made under'); + }); + } + + test('a push that only learns an agreement records the root too', async (t) => { + const env = new Env(t); + const m = await env.open(); + env.legacyEntry('a.txt', 'A'); // in the tree, and the same on disk, with no evidence yet + env.writeDisk('a.txt', 'A'); + await call(m, 'materialize', { path: 'work/a.txt' }); + assert.equal((m as any).agreement.get('work', 'a.txt')?.kind, 'content', 'the push learned the agreement'); + assert.equal(statSync(env.disk('a.txt')).size, 1); + assert.deepEqual((m as any).agreement.rootIdentity('work'), identityOf(env.dir), 'and recorded the root it was learned under'); + }); + + test("a pass records the root with the evidence it gathers", async (t) => { + const env = new Env(t); + env.writeDisk('a.txt', 'A'); // on disk before the workspace knows anything + const m = await env.open(); + await call(m, 'sync', {}); + assert.deepEqual((m as any).agreement.rootIdentity('work'), identityOf(env.dir)); + unmount(env); + await (m as any).initialScan('work'); + assert.notEqual(env.store.treeGet(TREE, 'a.txt'), null, 'still in the tree'); + }); + + test('a checkpoint keeps the recorded root', async (t) => { + const env = new Env(t); + const agreement = new DiskAgreement(env.store); + agreement.open([{ name: 'work', root: env.dir }]); + agreement.recordRootIdentity('work', { dev: '1', ino: '2' }); + for (let i = 0; i < 1100; i++) agreement.set('work', `f${i}.txt`, { kind: 'absent' }); + agreement.maybeCheckpoint(); + agreement.barrier(); + const reopened = new DiskAgreement(env.store); + reopened.open([{ name: 'work', root: env.dir }]); + assert.deepEqual(reopened.rootIdentity('work'), { dev: '1', ino: '2' }); + assert.equal(reopened.paths('work').length, 1100); + }); + + test('a mount configured at another path starts with no recorded root', async (t) => { + const env = new Env(t); + let m = await env.open(); + await seedSynced(env, m, 'a.txt', 'A'); + const elsewhere = join(env.root, 'elsewhere'); + mkdirSync(elsewhere); + m = await env.restart(m, { path: elsewhere }); + assert.equal((m as any).agreement.rootIdentity('work'), undefined, 'evidence about another directory is not carried over'); + }); + + test('acceptRoot takes a replaced root, with its evidence set aside', async (t) => { + const env = new Env(t); + const m = await env.open(); + await seedSynced(env, m, 'a.txt', 'A'); + await seedSynced(env, m, 'sub/b.txt', 'B'); + unmount(env); + env.writeDisk('a.txt', 'A'); // the new root holds the same a.txt, no sub/b.txt, and a new c.txt + env.writeDisk('c.txt', 'C'); + + const synced = await call(m, 'sync', { mount: 'work', acceptRoot: true }); + assert.deepEqual(synced.rootsAccepted, ['work']); + assert.deepEqual((m as any).agreement.rootIdentity('work'), identityOf(env.dir), 'the new root is recorded'); + assert.equal(await stateOf(m, 'a.txt'), 'synced', 'agreed again'); + assert.equal(await contentOf(m, 'c.txt'), 'C', "the new root's own file is taken in"); + assert.notEqual(env.store.treeGet(TREE, 'sub/b.txt'), null, 'a file the new root lacks stays in the workspace'); + assert.equal(env.eventsOf('workspace:deleted').length, 0, 'nothing was taken as deleted'); + + const again = await call(m, 'sync', { mount: 'work', acceptRoot: true }); + assert.equal(again.rootsAccepted, undefined, 'the recorded root is not accepted twice'); + }); + + test('acceptRoot takes a whole mount, not a path', async (t) => { + const env = new Env(t); + const m = await env.open(); + assert.match(await refused(m, 'sync', { path: 'work/a.txt', acceptRoot: true }), /whole mount/); + }); +});