diff --git a/changelog.d/workspace-watcher-agrees-with-scans.fixed.md b/changelog.d/workspace-watcher-agrees-with-scans.fixed.md new file mode 100644 index 000000000..9a1619275 --- /dev/null +++ b/changelog.d/workspace-watcher-agrees-with-scans.fixed.md @@ -0,0 +1,5 @@ +- A watched mount passes over what its `ignore` list covers, as a sync without a path does. The watcher handed chokidar the list as strings, which chokidar 4 compares with whole absolute paths, so no pattern matched, and every covered file the watcher saw change was taken in. A copy taken in that way stays as it was stored to every pass that names no path: a later change on disk isn't taken in, and its deletion is taken in where a walk lists its directory, as for anything a complete listing lacks. A copy beneath a directory the list covers stays, deleted on disk or not, the directory itself included, and so does one an `ls`, `glob` or `grep` reaches by its own path, or a sync naming a directory above it reaches by its walk. +- A mount nested inside another is passed over by the outer one as another mount's directory, in every pass, named or not, and by its watcher. The outer mount's ignore list gained the nested mount's path as written, which matched nothing for a mount more than one level down (`a/b`), so the outer mount took its files in as its own, and which matched the name at any depth for one directly beneath (`c`), so the outer mount also passed over unrelated entries with that name, such as `sub/c`. Now the outer mount's `ignore` list is left as configured, and the nested mount's directory is passed over as itself, counted under `passedOver` as "another mount's directory": the outer mount stops taking the nested mount's files, and takes files that only share its name. A sync that names that directory, or a path beneath it, with `/` between its segments, takes nothing in, and says so under `ignored`, where it took the nested mount's files in, since a named sync sets aside the `ignore` entries covering what it names (#276). That holds where an `ignore` entry covers a directory above the nested mount too. Files an outer mount stored from a nested one before, as `main`'s `a/b` mistake let it, stay in its branch's history and are listed with whose they are, but are passed over by its syncs, counted under `passedOver` and not under `incomplete`, and left out of its materialize and of `status`'s pending changes, conflicts and workspace deletions (its `fileCount` counts the tree as it stands). +- **The outer mount now refuses a nested mount's paths, reads included.** `read`, `read_image`, `write`, `edit`, `delete`, `readBinary`, `writeBinary`, `readBinaryFromDisk` and `readFileFromDisk` refuse a path in another mount's directory nested in the mount they name, judged where the path lands once its `.` and `..` segments and repeated separators are resolved, and the message names the mount that holds it and the path to use there; `readFileFromDisk`'s error code for it is `other_mount`, and `resolveAbsolutePath` returns null for such a path. A generated file the host registers at such a path is the same file whichever mount names it, and either reads it from its generator. `grep` returns none of the outer mount's copy of such a path; `ls` and `glob` list it with a note naming the mount that holds it and the path to use; a listing or `grep` naming a path there names them in its region, and a sync naming one in its note under `ignored`. A push of the outer mount, forced or not, skips such a path; a `materialize` that names one lists it under `skipped`, naming the mount that holds it, as a sync that names it does under `ignored`. A mount configured with a trailing separator no longer refuses its own root as traversal, and one configured with a relative path no longer refuses every path, generated files included, so the path the message names works. Before, a write through the outer mount, and any copy its store held there, went to disk at its next materialize over the nested mount's file wherever disk still agreed with the outer mount's evidence, and a forced one, as an undo or a branch switch makes, whatever disk held; and a read through the outer mount returned its own copy. A mount whose name only begins with two dots, such as `..cache`, now counts as nested. +- Three families of editors' swap and backup file names are passed over by a sync without a path, a listing and the watcher: `.name.swp` and `.name.swx` as the whole name (vim), and names ending in `~` (backups) or in `.subl…tmp` (Sublime Text's saves in progress), matched by `^\..+\.sw[px]$`, `~$` and `\.subl.*\.tmp$`. A directory with such a name is passed over with what lies beneath it. A sync counts one it doesn't track under `passedOver` as "an editor's swap or backup file", and reports one it tracks under `incomplete`. Naming one in a sync still syncs it, and lists it under `ignored`. Before, the watcher hid these names through chokidar's `atomic` option while a sync took them in, and that option tested the whole path, unanchored, so it also hid names such as `report.v2.swp`, and under a directory with a dot in its name, such as `~/.local/…`, every name ending in `.swp` or `.swx`, `notes.swp` included. The watcher no longer uses that option. An editor's save by rename reaches the workspace as a change to the file when the new copy is on disk by the time the watcher's batch is taken in, after its debounce. A copy still being written then is taken in as it stands and again whole once written, and one saved after a longer pause than the debounce is taken in as a deletion and then a creation. A copy already stored stays as it was to every pass that names no path, until its deletion is taken in, and a materialize treats it as any file the workspace holds, as it does a path the `ignore` list covers: it writes the workspace's copy while disk has what they last agreed on, as after a branch switch to history that holds it, and leaves a change disk made alone, as a conflict, unless forced. +- An `ls`, `glob` or `grep` that names a directory the mount passes over, such as one a bare-name `ignore` entry covers (`node_modules`) or one with an editor's name (`drafts~`), or a path beneath one, now takes nothing in there, and names the region under `incomplete`, adding that a sync that names the path takes in what's there, editors' files aside, as it still does. A copy the workspace stored there is listed, unobserved. Before, it listed such a directory and started tracking what it found; one a path entry such as `cache/**` covers listed nothing, and named nothing. A file tool names the path it acts on, as a sync naming it does: `read`, `edit`, `delete` and `readBinary` take in a file there that the workspace doesn't hold, as before, and the pass `write`, `edit`, `delete` and `writeBinary` make first, to settle a pending change on their path, observes it too. diff --git a/src/modules/workspace/index.ts b/src/modules/workspace/index.ts index 64880421d..9fc3ffab9 100644 --- a/src/modules/workspace/index.ts +++ b/src/modules/workspace/index.ts @@ -9,7 +9,7 @@ import { isUtf8 } from 'node:buffer'; import { constants as fsConstants } from 'node:fs'; import type { Stats } from 'node:fs'; import { open, readFile, stat, lstat, realpath } from 'node:fs/promises'; -import { join, resolve, relative, sep } from 'node:path'; +import { isAbsolute, join, resolve, relative, sep } from 'node:path'; import type { JsStore } from '@animalabs/chronicle'; import type { Module, ModuleContext, ProcessState, EventResponse } from '../../types/module.js'; import type { ProcessEvent, ToolDefinition, ToolCall, ToolResult } from '../../types/events.js'; @@ -40,7 +40,7 @@ import { MountWatcher, type FsChange } from './watcher.js'; import { hashContent, DEFAULT_MAX_FILE_SIZE } from './sync.js'; import { DiskAgreement, sameRootIdentity } from './disk-agreement.js'; import { BranchIntents } from './branch-intent.js'; -import { coveredByIgnore, filesystemTrustsCtime, locateRoot, readPrefix } from './observe.js'; +import { IGNORED, OTHER_MOUNT, filesystemTrustsCtime, inOtherMount, locateRoot, passedOverAt, readPrefix } from './observe.js'; import { TOOL_BATCH_HOOK_TIMEOUT_MS } from '../../module-registry.js'; import { type MountRuntime, @@ -51,6 +51,7 @@ import { type PushResult, type Scope, type TreeOp, + NOT_PUSHED_IN_OTHER_MOUNT, WALK_CAP, pushPaths, reconcilePass, @@ -113,11 +114,14 @@ class WorkspaceImageReadError extends Error { * (`mount_unavailable`), an ordinary missing file (`not_found`), a directory * (`directory`), a symlink refused by the mount's `followSymlinks` policy * (`symlink`), a symlink whose canonical target lands outside the canonical - * mount root (`escape`), and a file swapped underneath the read (`changed`). + * mount root (`escape`), a file swapped underneath the read (`changed`), and a + * path in another mount's directory nested in the one named (`other_mount`), + * whose message names the mount that holds it and the path to use there. */ export type WorkspaceReadErrorCode = | 'unknown_mount' | 'traversal' + | 'other_mount' | 'mount_unavailable' | 'not_found' | 'directory' @@ -137,6 +141,17 @@ export type WorkspaceReadStage = | 'stat' | 'read'; +/** + * A file operation's path lies in another mount's directory nested in the + * mount it names. The message names the mount that holds it and the path to + * use there. + */ +class NestedMountError extends Error { + constructor(path: string, mount: string, holder: string, use: string) { + super(`"${path}" lies in mount "${holder}", nested in mount "${mount}", which neither reads nor writes it: use "${use}".`); + } +} + /** * Thrown by `WorkspaceModule.readFileFromDisk()`. Messages carry the * mount-prefixed path only — never absolute filesystem paths — so they can be @@ -666,6 +681,11 @@ export class WorkspaceModule implements Module { /** Whether a mount root's filesystem maintains ctime, by canonical root. */ /** Per canonical root: the device of its filesystem when that maintains ctime, else null. */ private ctimeTrust = new Map(); + /** + * Each mount's directories that are other mounts nested inside it: + * mount-relative, `/`-separated, with each mount's name and resolved root. + */ + private otherMounts = new Map>(); /** Persisted state decoded in start(), held until mounts exist to apply it * to — in the Host ordering start() runs before initStore() creates the * mounts, so restoration must be second-callback-safe like watcher @@ -683,26 +703,31 @@ export class WorkspaceModule implements Module { ); } } - // Detect overlapping mount paths: if mount A contains mount B, - // auto-add an ignore rule on A for B's path to prevent syncing - // the sub-mount's directory through the super-mount. + // Detect overlapping mount paths: a mount nested inside another is that + // mount's directory, which the outer mount passes over in every pass, + // named or not, and in its watcher (OTHER_MOUNT). It's kept apart from + // the outer mount's ignore list, which a sync that names a path sets + // aside for what it names (#276), so no sync of the outer mount takes in + // the inner one's files. for (const outer of config.mounts) { for (const inner of config.mounts) { if (outer === inner) continue; const outerPath = resolve(outer.path); const innerPath = resolve(inner.path); const rel = relative(outerPath, innerPath); - if (rel && !rel.startsWith('..') && !rel.startsWith('/')) { - // inner is nested under outer — add ignore rule - outer.ignore = outer.ignore ?? []; - const pattern = rel + '/**'; - if (!outer.ignore.includes(pattern) && !outer.ignore.includes(rel)) { - outer.ignore.push(rel); - console.warn( - `[workspace] Mount "${outer.name}" contains mount "${inner.name}" ` + - `(${rel}/) — auto-ignoring to prevent overlap`, - ); - } + // Outside the outer mount, `relative` gives a path that climbs with a + // `..` segment, or an absolute one across Windows drives. A name that + // only begins with two dots, such as `..cache`, is inside it. + if (rel && rel !== '..' && !rel.startsWith('..' + sep) && !isAbsolute(rel)) { + // inner is nested under outer: passed over as exactly that + // directory, read against `/`-separated paths, whatever the + // platform's. + const relPath = rel.split(sep).join('/'); + this.otherMounts.set(outer.name, [...(this.otherMounts.get(outer.name) ?? []), { dir: relPath, name: inner.name, root: innerPath }]); + console.warn( + `[workspace] Mount "${outer.name}" contains mount "${inner.name}" ` + + `at ${relPath}/ — passing over it as another mount's directory`, + ); } } } @@ -805,6 +830,7 @@ export class WorkspaceModule implements Module { followSymlinks: mount.config.followSymlinks === true, maxFileSize: mount.config.maxFileSize ?? DEFAULT_MAX_FILE_SIZE, ignore: mount.config.ignore ?? [], + others: this.othersOf(mount.config.name), }, rootReal, ctimeTrustedDev: trustedDev, @@ -820,6 +846,17 @@ export class WorkspaceModule implements Module { private async passUnlocked(mount: MountState, scope: Scope, opts: PassOptions = {}): Promise { const result = await reconcilePass(this.getStore(), this.agreementOrThrow(), await this.runtime(mount), scope, opts); if (scope.kind === 'dir' && scope.dir === '' && scope.recursive) mount.initialSyncDone = true; + // What a pass says is another mount's directory names that mount and + // the path to use there, as the file operations' refusal does: a copy's + // note, and the region of a scope that lies there. The pass's counts + // keep the bare reason. + const name = mount.config.name; + for (const report of result.reports.values()) { + if (report.note === OTHER_MOUNT) report.note = this.otherMountNote(name, report.path, 'list'); + } + for (const region of result.incomplete) { + if (region.reason === OTHER_MOUNT) region.reason = this.otherMountNote(name, region.path, 'list'); + } this.emitFsEvents(mount.config.name, result.ops, result.newConflicts); return result; } @@ -1011,6 +1048,7 @@ export class WorkspaceModule implements Module { } as ProcessEvent); }, }, + this.othersOf(name), ); watcher.start(); this.watchers.set(name, watcher); @@ -1271,13 +1309,18 @@ export class WorkspaceModule implements Module { + 'directory) is unavailable and listed under `incomplete`. Reconnect the drive; acceptRoot is for a ' + 'directory replaced on purpose, or a drive back under a new device number, never for one still ' + 'unplugged, whose empty mountpoint would become the root. What the scan passed over by design and ' - + 'reported nowhere else is counted under `passedOver`, by mount and reason: entries the mount\'s ' - + 'ignore list covers that hold nothing the workspace tracks (an ignored directory counts once). So a ' + + 'reported nowhere else is counted under `passedOver`, by mount and reason: another mount\'s ' + + 'directory nested in this one, entries the mount\'s ignore list covers, and entries with an ' + + 'editor\'s swap or backup file name (`.name.swp` and `.name.swx` as the whole name, names ending ' + + 'in `~` or in `.subl…tmp`), that hold nothing the workspace tracks (such a directory counts once, ' + + 'with what lies beneath it). So a ' + 'total of 0, with no conflicts and nothing under `passedOver`, `incomplete` or `skipped`, means the ' - + 'sync looked everywhere and nothing had changed. A path the mount ignores is synced when you name ' - + 'it, and listed under `ignored`: a sync without a path passes over it. Naming an ignored directory ' - + 'sets aside the patterns that cover it or a directory above it; one that covers only something ' - + 'inside it still applies, and what it covers is passed over.', + + 'sync looked everywhere and nothing had changed. A path the mount ignores, or an editor\'s file, is ' + + 'synced when you name it, and listed under `ignored`: a sync without a path passes over it. A path ' + + 'in another mount\'s directory isn\'t synced even when you name it: use the mount that holds it. Naming ' + + 'an ignored directory sets aside the patterns that cover it or a directory above it; one that ' + + 'covers only something inside it still applies, and what it covers is passed over, as are editors\' ' + + 'files inside it.', inputSchema: { type: 'object' as const, properties: { @@ -1310,8 +1353,10 @@ export class WorkspaceModule implements Module { const mountName = slash < 0 ? path : path.slice(0, slash); const mount = this.config.mounts.find(m => m.name === mountName); if (!mount) throw new Error(`Unknown mount: "${mountName}"`); - const resolved = resolve(mount.path, slash < 0 ? '' : path.slice(slash + 1)); - if (!isContainedPath(mount.path, resolved)) throw new Error(`Path traversal detected: "${path}"`); + // Against the resolved root, as parsePath judges it. + const root = resolve(mount.path); + const resolved = resolve(root, slash < 0 ? '' : path.slice(slash + 1)); + if (!isContainedPath(root, resolved)) throw new Error(`Path traversal detected: "${path}"`); return resolved; } @@ -1459,7 +1504,7 @@ export class WorkspaceModule implements Module { */ resolveAbsolutePath(mountPrefixedPath: string): string | null { try { - const { mount, relativePath } = this.parsePath(mountPrefixedPath); + const { mount, relativePath } = this.parseFilePath(mountPrefixedPath); return resolve(mount.config.path, relativePath); } catch { return null; @@ -1497,7 +1542,7 @@ export class WorkspaceModule implements Module { let mount: MountState; let relativePath: string; try { - ({ mount, relativePath } = this.parsePath(mountPrefixedPath)); + ({ mount, relativePath } = this.parseFilePath(mountPrefixedPath)); } catch (error) { return { success: false, @@ -1557,7 +1602,7 @@ export class WorkspaceModule implements Module { let mount: MountState; let relativePath: string; try { - ({ mount, relativePath } = this.parsePath(mountPrefixedPath)); + ({ mount, relativePath } = this.parseFilePath(mountPrefixedPath)); } catch (error) { return { error: error instanceof Error ? error.message : String(error) }; } @@ -1594,7 +1639,7 @@ export class WorkspaceModule implements Module { let mount: MountState; let relativePath: string; try { - ({ mount, relativePath } = this.parsePath(mountPrefixedPath)); + ({ mount, relativePath } = this.parseFilePath(mountPrefixedPath)); } catch (error) { return { error: error instanceof Error ? error.message : String(error) }; } @@ -1613,6 +1658,57 @@ export class WorkspaceModule implements Module { } } + /** The directories of the mounts nested inside a mount, mount-relative. */ + private othersOf(mountName: string): string[] { + return (this.otherMounts.get(mountName) ?? []).map((o) => o.dir); + } + + /** + * The innermost mount nested in `mountName` whose directory holds a path + * of that mount, judged lexically where the path lands (`.`, `..` and + * repeated separators resolved, as for the traversal guard; a link of the + * outer mount isn't followed), with the path to use there; undefined when + * no nested mount's directory holds it. + */ + private holderOf(mountName: string, relativePath: string): { name: string; use: string } | undefined { + const mount = this.mounts.get(mountName); + if (!mount) return undefined; + const resolved = resolve(mount.config.path, relativePath); + const holder = (this.otherMounts.get(mountName) ?? []) + .filter((o) => isContainedPath(o.root, resolved)) + .sort((a, b) => b.root.length - a.root.length)[0]; + if (!holder) return undefined; + const rest = relative(holder.root, resolved).split(sep).join('/'); + return { name: holder.name, use: rest ? `${holder.name}/${rest}` : holder.name }; + } + + /** + * Why a mount leaves a path in a nested mount's directory, naming that + * mount and the path to use: for a sync or a push, which of them leaves + * it; for a listing, whose the path is. + */ + private otherMountNote(mountName: string, relativePath: string, by: 'sync' | 'push' | 'list'): string { + const holder = this.holderOf(mountName, relativePath); + const what = by === 'sync' ? ', which no sync of this mount takes in' : by === 'push' ? ', which no push of this mount writes' : ''; + return holder + ? `it lies in mount "${holder.name}", nested in this one${what}: use "${holder.use}"` + : `it lies in another mount's directory${what}`; + } + + /** + * parsePath for the file operations (read, write, edit, delete, and the + * module's binary and disk reads and writes). A path in another mount's + * directory nested in this one is that mount's: this mount's passes don't + * observe it and its pushes don't write it, so its file operations refuse + * it, naming the innermost mount that holds it and the path to use there. + */ + private parseFilePath(path: string): { mount: MountState; relativePath: string } { + const parsed = this.parsePath(path); + const holder = this.holderOf(parsed.mount.config.name, parsed.relativePath); + if (holder) throw new NestedMountError(path, parsed.mount.config.name, holder.name, holder.use); + return parsed; + } + /** * Parse a mount-prefixed path into (mountName, relativePath). */ @@ -1635,8 +1731,11 @@ export class WorkspaceModule implements Module { // Containment must compare with the PLATFORM separator (isContainedPath): // resolve() emits backslashes on Windows, so a '/'-suffixed root never // prefix-matches there and every in-mount path was rejected as traversal. - const resolved = resolve(mount.config.path, relativePath); - if (!isContainedPath(mount.config.path, resolved)) { + // Against the resolved root, so a root configured with a trailing + // separator, or relative, contains its own paths, the root included. + const root = resolve(mount.config.path); + const resolved = resolve(root, relativePath); + if (!isContainedPath(root, resolved)) { throw new Error(`Path traversal detected: "${path}" resolves outside mount "${mountName}"`); } @@ -1851,8 +1950,10 @@ export class WorkspaceModule implements Module { * read. * * Throws {@link WorkspaceReadError} with a `code` distinguishing unknown - * mount, lexical traversal, unavailable mount, missing file, directory, - * policy-denied symlink, outside-mount target, and file-changed-during-read. + * mount, lexical traversal, a path in another mount's directory nested in + * the one named (`other_mount`), unavailable mount, missing file, + * directory, policy-denied symlink, outside-mount target, and + * file-changed-during-read. */ async readFileFromDisk( mountPrefixedPath: string, @@ -1864,10 +1965,11 @@ export class WorkspaceModule implements Module { let mount: MountState; let relativePath: string; try { - ({ mount, relativePath } = this.parsePath(mountPrefixedPath)); + ({ mount, relativePath } = this.parseFilePath(mountPrefixedPath)); } catch (err) { const message = err instanceof Error ? err.message : String(err); - const code: WorkspaceReadErrorCode = message.startsWith('Unknown mount') ? 'unknown_mount' : 'traversal'; + const code: WorkspaceReadErrorCode = err instanceof NestedMountError ? 'other_mount' + : message.startsWith('Unknown mount') ? 'unknown_mount' : 'traversal'; throw new WorkspaceReadError(code, message, 'parse'); } if (!relativePath) { @@ -2052,24 +2154,29 @@ export class WorkspaceModule implements Module { * disk now holds (an edit sees a disk change the settled adoption took in). * The pass may not settle it (a branch that keeps changing, an unreadable * disk copy); the precondition that matters is enforced again, - * synchronously, where the change commits (commitToolChange). The caller - * holds the mount's turn. + * synchronously, where the change commits (commitToolChange). The pass + * names the path, as a file tool names the one it acts on, so a path the + * mount passes over by design is observed there, as a sync naming it would + * observe it (#276). The caller holds the mount's turn. */ private async settleBeforeMutation(mount: MountState, relativePath: string): Promise { if (this.agreementOrThrow().get(mount.config.name, relativePath)?.kind !== 'pending') return; - await this.passUnlocked(mount, { kind: 'paths', paths: [relativePath] }); + await this.passUnlocked(mount, { kind: 'paths', paths: [relativePath] }, { named: true }); } /** * Before a read: a path the workspace doesn't hold is observed under the * three-way rule, which ingests a new disk file but never brings back a - * workspace deletion or a file absent from this branch. The caller holds - * the mount's turn. + * workspace deletion or a file absent from this branch. The pass names the + * path, as a file tool names the one it acts on: a path the mount passes + * over by design is observed, as a sync naming it would observe it (#276), + * while a path in another mount's directory is refused before this, and + * passed over by any pass. The caller holds the mount's turn. */ private async ensureSynced(mount: MountState, relativePath: string): Promise { const store = this.getStore(); if (store.treeGet(mount.treeStateId, relativePath)) return undefined; - const pass = await this.passUnlocked(mount, { kind: 'paths', paths: [relativePath] }); + const pass = await this.passUnlocked(mount, { kind: 'paths', paths: [relativePath] }, { named: true }); return pass.reports.get(relativePath); } @@ -2108,7 +2215,7 @@ export class WorkspaceModule implements Module { } let content = generatedContent; if (content === undefined) { - const { mount, relativePath } = this.parsePath(input.path); + const { mount, relativePath } = this.parseFilePath(input.path); const store = this.getStore(); const found = await this.withMount(mount, async (): Promise<{ text: string } | { error: string }> => { const report = await this.ensureSynced(mount, relativePath); @@ -2196,7 +2303,7 @@ export class WorkspaceModule implements Module { let mount: MountState; let relativePath: string; try { - ({ mount, relativePath } = this.parsePath(input.path)); + ({ mount, relativePath } = this.parseFilePath(input.path)); } catch (err) { return { success: false, @@ -2268,7 +2375,7 @@ export class WorkspaceModule implements Module { if (LONE_SURROGATE.test(input.content)) { return { success: false, isError: true, error: 'The write did nothing: "content" holds a lone UTF-16 surrogate, which UTF-8 can\'t store.' }; } - const { mount, relativePath } = this.parsePath(input.path); + const { mount, relativePath } = this.parseFilePath(input.path); if (mount.config.mode === 'read-only') { return { success: false, error: `Mount "${mount.config.name}" is read-only`, isError: true }; } @@ -2310,7 +2417,7 @@ export class WorkspaceModule implements Module { return { success: false, isError: true, error: `The edit did nothing: "${field}" holds a lone UTF-16 surrogate, which UTF-8 can't store.` }; } } - const { mount, relativePath } = this.parsePath(input.path); + const { mount, relativePath } = this.parseFilePath(input.path); if (mount.config.mode === 'read-only') { return { success: false, error: `Mount "${mount.config.name}" is read-only`, isError: true }; } @@ -2325,8 +2432,10 @@ export class WorkspaceModule implements Module { // listing uses. The pass settles a pending intent on the path too, as // settleBeforeMutation's would, and the commit enforces the rest // (commitToolChange). A write passes over its path only to settle a - // pending intent, since it replaces the whole file. - const pass = await this.passUnlocked(mount, { kind: 'paths', paths: [relativePath] }); + // pending intent, since it replaces the whole file. As every file + // tool's pass does, it names the path it acts on, so a file the + // mount passes over by design is observed (see PassOptions.named). + const pass = await this.passUnlocked(mount, { kind: 'paths', paths: [relativePath] }, { named: true }); const report = pass.reports.get(relativePath); const took = (op: TreeOp) => pass.ops.some((o) => o.path === relativePath && o.op === op); // Which copy the edit used, when it isn't the one the workspace last @@ -2441,7 +2550,7 @@ export class WorkspaceModule implements Module { } private async handleDelete(input: DeleteInput): Promise { - const { mount, relativePath } = this.parsePath(input.path); + const { mount, relativePath } = this.parseFilePath(input.path); if (mount.config.mode === 'read-only') { return { success: false, error: `Mount "${mount.config.name}" is read-only`, isError: true }; } @@ -2606,6 +2715,13 @@ export class WorkspaceModule implements Module { }; for (const { mount, relativePath } of mountsToSearch) { + // A path named in another mount's directory, as holderOf resolves it, + // is that mount's: grep searches nothing of this mount's there, and + // says whose it is. + if (this.holderOf(mount.config.name, relativePath)) { + incomplete.push({ path: `${mount.config.name}/${relativePath}`, reason: this.otherMountNote(mount.config.name, relativePath, 'list') }); + continue; + } await this.withMount(mount, async () => { // A `path` naming a single file greps just that file; otherwise it is a // directory prefix. (A file path once produced the prefix "notes.md/" @@ -2650,6 +2766,10 @@ export class WorkspaceModule implements Module { }); continue; } + // A path in another mount's directory is that mount's: grep gives + // none of this mount's copy of it, as read refuses it. Beneath a + // scope it's passed over by design, like the directory. + if (inOtherMount(report.path, this.othersOf(mount.config.name))) continue; // The workspace's version, labelled with the path's state. const entry = store.treeGet(mount.treeStateId, report.path); const blob = entry ? store.getBlob(entry.blobHash) : null; @@ -2709,7 +2829,8 @@ export class WorkspaceModule implements Module { if (input.mount !== undefined && name !== input.mount) continue; const entries = store.treeList(mount.treeStateId); const currentSeq = store.currentSequence(); - const intents = this.intents.get(name)!.list(); + const own = this.ownPaths(mount); + const intents = this.intents.get(name)!.list().filter(([p]) => own(p)); const conflictPaths = intents.filter(([, bi]) => bi.conflict).map(([p]) => p); // A workspace deletion is pending until a push confirms it on disk: // left there without applyDeletions, or unlinked but unconfirmed. @@ -2755,12 +2876,13 @@ export class WorkspaceModule implements Module { */ private materializeSelection(mount: MountState, explicitPath: string): { paths: string[]; inPlace: boolean } { const store = this.getStore(); - const intents = new Map(this.intents.get(mount.config.name)!.list(explicitPath)); + const own = this.ownPaths(mount); + const intents = new Map(this.intents.get(mount.config.name)!.list(explicitPath).filter(([p]) => own(p))); const selected = new Set(); if (explicitPath !== '') { - if (store.treeGet(mount.treeStateId, explicitPath)) selected.add(explicitPath); - for (const e of store.treeList(mount.treeStateId, explicitPath + '/')) selected.add(e.path); + if (store.treeGet(mount.treeStateId, explicitPath) && own(explicitPath)) selected.add(explicitPath); + for (const e of store.treeList(mount.treeStateId, explicitPath + '/')) if (own(e.path)) selected.add(e.path); for (const [p, bi] of intents) if (bi.tombstone || bi.conflict) selected.add(p); return { paths: [...selected], inPlace: false }; } @@ -2771,6 +2893,19 @@ export class WorkspaceModule implements Module { return { paths: [...selected], inPlace: agreed > 0 }; } + /** + * Whether a path of the mount is its own to push and to count as owed: + * not in another mount's directory nested in it. Copies the mount's tree + * holds there (stored before the nesting) stay in its history and its + * listings (and in status's fileCount, which counts the tree), passed over + * by its syncs, and left out of what it pushes and of status's pending + * changes, conflicts and workspace deletions. + */ + private ownPaths(mount: MountState): (path: string) => boolean { + const others = this.othersOf(mount.config.name); + return (path) => !inOtherMount(path, others); + } + /** * The workspace entries by what the evidence says of disk. `owed`: P * differs from the entry (a draft, or a push still pending), or there is no @@ -2789,7 +2924,9 @@ export class WorkspaceModule implements Module { const owed: string[] = []; const unchecked: string[] = []; let agreed = 0; + const own = this.ownPaths(mount); for (const e of store.treeList(mount.treeStateId)) { + if (!own(e.path)) continue; const p = agreement.get(name, e.path); if (p === undefined) (intents.get(e.path)?.origin === 'store' ? owed : unchecked).push(e.path); else if (p.kind !== 'content' || p.hash !== e.blobHash) owed.push(e.path); @@ -2842,6 +2979,13 @@ export class WorkspaceModule implements Module { blocked.push({ mount: name, reason: 'a read-only mount: materialize writes nothing to it' }); continue; } + // A path named in another mount's directory, as holderOf resolves it, + // is that mount's: nothing of this mount's is pushed there, and the + // caller is told whose it is. + if (explicit && this.holderOf(name, explicit.relativePath)) { + blocked.push({ mount: name, reason: `${explicit.relativePath}: ${this.otherMountNote(name, explicit.relativePath, 'push')}` }); + continue; + } const turn = await this.withMount(mount, async (): Promise<{ blocked: string } | { pushed: PushResult; inPlace: boolean }> => { const branchId = store.currentBranch().id; const reason = this.mountMaterializeBlockReason(store, mount); @@ -2871,7 +3015,8 @@ export class WorkspaceModule implements Module { // Refusals ride the same skipped list as branch blocks: divergence must // be visible, not resolved silently either way. for (const s of pushed.skipped) { - blocked.push({ mount: name, reason: `${s.path}: ${s.reason}` }); + const reason = s.reason === NOT_PUSHED_IN_OTHER_MOUNT ? this.otherMountNote(name, s.path, 'push') : s.reason; + blocked.push({ mount: name, reason: `${s.path}: ${reason}` }); } // What disk still owes is kept by its evidence, so the watermark and @@ -3061,13 +3206,20 @@ export class WorkspaceModule implements Module { for (const region of pass.incomplete) allIncomplete.push({ mount: name, ...region }); for (const [reason, count] of pass.passedOver) allPassedOver.push({ mount: name, reason, count }); // A path named explicitly is synced even where the mount's ignore list - // covers it, and the result says so: a sync without a path passes - // over it, so nothing else keeps it current (agent-framework #276). - if (coveredByIgnore(relativePath, mount.config.ignore ?? [])) { + // or an editor's file name covers it, and the result says so: a sync + // without a path passes over it, so nothing else keeps it current + // (agent-framework #276). A path in another mount's directory isn't + // synced at all, and the note says so. + const passedOver = passedOverAt(relativePath, mount.config.ignore ?? [], this.othersOf(name)); + if (passedOver !== undefined) { allIgnored.push({ mount: name, path: relativePath, - note: 'this mount\'s ignore list covers it, so a sync without a path won\'t maintain it', + note: passedOver === OTHER_MOUNT + ? this.otherMountNote(name, relativePath, 'sync') + : passedOver === IGNORED + ? 'this mount\'s ignore list covers it, so a sync without a path won\'t maintain it' + : 'it has an editor\'s swap or backup file name, or lies under one, so a sync without a path won\'t maintain it', }); } } diff --git a/src/modules/workspace/observe.ts b/src/modules/workspace/observe.ts index ca92c511c..b3d384182 100644 --- a/src/modules/workspace/observe.ts +++ b/src/modules/workspace/observe.ts @@ -41,7 +41,7 @@ import { constants as fsConstants } from 'node:fs'; import type { BigIntStats } from 'node:fs'; import { lstat, mkdir, open, readdir, realpath, stat, statfs } from 'node:fs/promises'; import type { FileHandle } from 'node:fs/promises'; -import { dirname, join, sep } from 'node:path'; +import { dirname, join, posix, sep } from 'node:path'; import { sameRootIdentity, type Fingerprint, type RootIdentity } from './disk-agreement.js'; export interface MountView { @@ -56,6 +56,11 @@ export interface MountView { /** Text above this size is not ingested. */ maxFileSize: number; ignore: string[]; + /** + * Other mounts' directories nested inside this one, mount-relative. Every + * pass passes over them, named or not, as another mount's directory. + */ + others: readonly string[]; } export type StatFingerprint = Omit; @@ -718,14 +723,18 @@ export interface Walk { */ seen: Set; /** - * The names in `others` that aren't ignored and that the walk took, as it + * The names in `others` that it doesn't pass over by design and that the walk took, as it * takes files, against its cap: the pass observes these by path. The rest * are present, and not observed by this pass. */ othersTaken: Set; /** Directories seen. */ dirs: Set; - /** Names seen that are neither (symlinks, sockets…), or ignored: present, unproven. */ + /** + * Names seen that are neither (symlinks, sockets…), or that the walk passes + * over by design (another mount's directory, the ignore list, an editor's + * file name): present, unproven. + */ others: Map; /** * Directories whose listing the walk read whole, and that were still the @@ -738,9 +747,10 @@ export interface Walk { /** * Where the walk didn't reach, and why. `error` and `cap` regions may hide * files the listing would show. An `ignored` region is passed over by - * design: one that hides a tracked entry is reported as incomplete, and - * one that hides nothing is counted as passed over. A `symlink` region is - * never walked; the pass reports the link itself by path. + * design: one that hides a tracked entry, or is the pass's whole scope, is + * reported as incomplete, and any other is counted as passed over. A + * `symlink` region is never walked; the pass reports the link itself by + * path. */ incomplete: Array<{ path: string; reason: string; kind: 'error' | 'cap' | 'ignored' | 'symlink' }>; /** @@ -811,6 +821,101 @@ export function coveredByIgnore(relativePath: string, patterns: string[]): boole return false; } +/** + * The ignore patterns a walk that names `scope` reads beneath it: a pattern + * covering the scope itself, or a directory above it, doesn't apply beneath + * it (#276); one covering only something inside it does. + */ +export function ignoreBeneath(scope: string, patterns: string[]): string[] { + return patterns.filter((pattern) => !coveredByIgnore(scope, [pattern])); +} + +/** + * Names editors give the files they keep beside the one being edited: vim's + * swap files (`.name.swp`, and `.name.swx` while it checks it can make one), + * backups ending in `~`, and Sublime Text's save-in-progress files, ending + * in `.subl…tmp`. A walk passes over an entry with such a name as it passes + * over what the ignore list covers, a directory with what lies beneath it, + * and the watcher ignores it; a sync that names one takes it in. They are + * the families chokidar's `atomic` option hid from the watcher, but that + * tested them against the whole path, unanchored, so it also hid + * `report.v2.swp`, and under a directory with a dot in its name, a plain + * `notes.swp`. Here each is matched against an entry's name, to its end, + * and vim's from its start. + */ +const EDITOR_FILE = /^\..+\.sw[px]$|~$|\.subl.*\.tmp$/; + +/** The walk's reason for an entry the mount's ignore list covers. */ +export const IGNORED = 'ignored by the mount'; +/** The walk's reason for an entry `EDITOR_FILE` names. */ +const EDITOR_FILE_REASON = "an editor's swap or backup file"; + +/** Whether a reason the walk gave an entry is one it passes over by design. */ +export function passedOverByDesign(why: string | undefined): boolean { + return why === IGNORED || why === EDITOR_FILE_REASON || why === OTHER_MOUNT; +} + +/** Why a pass passes over a directory that another mount nested in this one holds, and what lies beneath it. */ +export const OTHER_MOUNT = "another mount's directory"; + +/** + * Whether a mount-relative path is another mount's directory, or lies + * beneath one, read as a join reads it: `.` and `..` segments within the + * path and repeated or trailing separators resolved, links not followed. A + * stored key such as `a//b/x`, which chronicle accepts, is read as `a/b/x`. + * A leading `..` that climbs above the mount's root is left as written. + * Every nested-mount check (a pass's, a walk's, a push's, grep's) asks this. + */ +export function inOtherMount(relativePath: string, others: readonly string[]): boolean { + const lexical = posix.normalize(relativePath).replace(/\/+$/, ''); + return others.some((dir) => lexical === dir || lexical.startsWith(dir + '/')); +} + +/** + * Why a walk passes over an entry by design, by its mount-relative path and + * its name: another mount's directory, the ignore list, or an editor's file + * name. Undefined when none covers it. + */ +function passedOverBecause(relativePath: string, name: string, patterns: string[], others: readonly string[]): string | undefined { + if (inOtherMount(relativePath, others)) return OTHER_MOUNT; + if (isIgnored(relativePath, name, patterns)) return IGNORED; + if (EDITOR_FILE.test(name)) return EDITOR_FILE_REASON; + return undefined; +} + +/** + * Why a sync without a path, and the watcher, pass over a mount-relative + * path: the path itself or a directory above it is passed over by design, as + * the walk reads each entry by its path and name. A path in another mount's + * directory is that mount's, whatever else covers a directory above it. + * Undefined when nothing covers it. + */ +export function passedOverAt(relativePath: string, patterns: string[], others: readonly string[]): string | undefined { + if (inOtherMount(relativePath, others)) return OTHER_MOUNT; + const segments = relativePath.split('/').filter(Boolean); + for (let i = 0; i < segments.length; i++) { + const why = passedOverBecause(segments.slice(0, i + 1).join('/'), segments[i]!, patterns, others); + if (why !== undefined) return why; + } + return undefined; +} + +/** + * Why a walk of `scope` passes over a directory between it and a path beneath + * it: the first directory strictly beneath the scope and above the path that + * the mount passes over by design, by its path and name, with the patterns + * the walk reads there. Undefined when none is. + */ +export function passedOverBetween(scope: string, relativePath: string, patterns: string[], others: readonly string[]): string | undefined { + const segments = relativePath.split('/').filter(Boolean); + const from = scope === '' ? 0 : scope.split('/').filter(Boolean).length; + for (let i = from; i < segments.length - 1; i++) { + const why = passedOverBecause(segments.slice(0, i + 1).join('/'), segments[i]!, patterns, others); + if (why !== undefined) return why; + } + return undefined; +} + /** Simple ignore patterns, as the watcher and the old walk read them. */ export function isIgnored(relativePath: string, name: string, patterns: string[]): boolean { for (const pattern of patterns) { @@ -872,6 +977,29 @@ export async function walkScope( files: new Set(), seen: new Set(), othersTaken: new Set(), dirs: new Set(), others: new Map(), complete: new Set(), missing: new Set(), incomplete: [], }; + // A walk that isn't named passes over a scope the mount passes over by + // design, at it or a directory above it, as its parent's listing would pass + // over its entry: a name rule (a bare-name ignore entry, an editor's name) + // doesn't match what lies beneath, as a path rule (`cache/**`) does. So a + // listing, glob or grep there never starts tracking what the mount leaves + // out, and a stored copy beneath it is unobserved, as beneath any region + // passed over by design. Its region says how to take what's there in: a + // sync that names it, whose scope is walked (#276), below. + // Another mount's directory is passed over by every walk, named or not, + // and no sync of this mount takes it in, so its region gives no advice. + if (scope !== '' && inOtherMount(scope, view.others)) { + walk.others.set(scope, OTHER_MOUNT); + walk.incomplete.push({ path: scope, reason: OTHER_MOUNT, kind: 'ignored' }); + return walk; + } + if (!opts.named && scope !== '') { + const why = passedOverAt(scope, view.ignore, view.others); + if (why !== undefined) { + walk.others.set(scope, why); + walk.incomplete.push({ path: scope, reason: `${why}: a sync that names this path takes in what's there, editors' files aside`, kind: 'ignored' }); + return walk; + } + } const top = await checkDir(view, rootReal, scope); if (top.kind === 'absent') { walk.missing.add(scope); @@ -884,9 +1012,11 @@ export async function walkScope( // A scope a path sync names is walked even where the ignore list covers // it: a pattern covering the scope itself, or a directory above it, doesn't // apply beneath it. So a named directory's contents are synced, as a named - // file is (agent-framework #276). A listing keeps the ignore: its pass - // would otherwise start tracking what it lists. - const ignore = opts.named ? view.ignore.filter((pattern) => !coveredByIgnore(scope, [pattern])) : view.ignore; + // file is (agent-framework #276), apart from what a pattern covering only + // something inside it covers, and editors' files, which are passed over + // there as anywhere. A listing keeps the ignore: its pass would otherwise + // start tracking what it lists. + const ignore = opts.named ? ignoreBeneath(scope, view.ignore) : view.ignore; // A position outside the scope, or for a walk that doesn't descend, is no // position in this walk's order. const after = recursive && opts.resumeAfter !== undefined && (scope === '' || opts.resumeAfter.startsWith(scope + '/')) @@ -987,9 +1117,10 @@ export async function walkScope( const regions: Walk['incomplete'] = []; for (const entry of entries) { const childRel = rel ? `${rel}/${entry.name}` : entry.name; - if (isIgnored(childRel, entry.name, ignore)) { - others.push([childRel, 'ignored by the mount']); - regions.push({ path: childRel, reason: 'ignored by the mount', kind: 'ignored' }); + const passedOver = passedOverBecause(childRel, entry.name, ignore, view.others); + if (passedOver !== undefined) { + others.push([childRel, passedOver]); + regions.push({ path: childRel, reason: passedOver, kind: 'ignored' }); continue; } if (entry.isDirectory()) { @@ -1083,7 +1214,7 @@ export async function walkScope( const capped = `the walk cap (${cap}) was reached`; const holdingUntaken = new Set(); for (const f of walk.seen) holdingUntaken.add(parentOf(f)); - for (const [p, why] of walk.others) if (why !== 'ignored by the mount' && !walk.othersTaken.has(p)) holdingUntaken.add(parentOf(p)); + for (const [p, why] of walk.others) if (!passedOverByDesign(why) && !walk.othersTaken.has(p)) holdingUntaken.add(parentOf(p)); for (const d of holdingUntaken) { walk.incomplete.push({ path: d, reason: `${capped} before everything here was taken`, kind: 'cap' }); } @@ -1098,7 +1229,8 @@ export async function walkScope( // The cap stopped the walk only if it left something: a walk whose last // file used up the cap covered its scope. A directory the walk didn't list // counts as left, even one that holds nothing it would take (empty, or - // only ignored names): telling them apart would take another read. Such a + // only names it passes over by design): telling them apart would take + // another read. Such a // walk keeps its place and reports the directory as not visited, and the // next one lists it on its way to the wrap. if (left === 0 && lastTaken !== undefined && (holdingUntaken.size > 0 || unlisted > 0)) walk.stoppedAfter = lastTaken; diff --git a/src/modules/workspace/reconcile.ts b/src/modules/workspace/reconcile.ts index 8f2ab7d79..12487d91c 100644 --- a/src/modules/workspace/reconcile.ts +++ b/src/modules/workspace/reconcile.ts @@ -54,13 +54,19 @@ import { } from './disk-agreement.js'; import type { BranchIntent, BranchIntents, ConflictKind, ConflictRecord, DiskCounterpart } from './branch-intent.js'; import { + IGNORED, type MountView, + OTHER_MOUNT, ROOT_REPLACED, type Walk, fingerprintVouches, + ignoreBeneath, + inOtherMount, locateRoot, looksBinary, observePath, + passedOverAt, + passedOverBetween, provenAbsent, readHead, readPath, @@ -120,9 +126,11 @@ export interface PassResult { incomplete: Array<{ path: string; reason: string }>; /** * What the walk passed over by design and reported nowhere else, counted by - * reason: entries the mount's ignore list covers, when nothing beneath them - * is tracked (a tracked one is reported under `incomplete`). An ignored - * directory counts once, everything in it included. Only listings the walk + * reason: another mount's directory nested in this one, entries the mount's + * ignore list covers, and entries with an editor's swap or backup file + * name, when nothing beneath them is tracked (a tracked one is reported + * under `incomplete`). Such a directory counts + * once, everything in it included. Only listings the walk * kept count: a directory changed during the walk, or one past the file * cap, is reported under `incomplete` instead. */ @@ -182,8 +190,10 @@ export interface PassOptions { */ resume?: boolean; /** - * The scope was named by a path sync: ignore patterns that cover it don't - * apply beneath it (see walkScope). A listing's scope keeps them. + * The scope was named, by a path sync or by a file tool's pass on the path + * it acts on: what the mount passes over by design that covers the scope, + * at it or above it, doesn't apply there (see walkScope). A listing's scope + * keeps it. */ named?: boolean; /** @@ -696,12 +706,41 @@ async function gather(store: JsStore, agreement: DiskAgreement, mount: MountRunt gathered.facts.set(path, { kind: 'unobserved', reason: rootUnavailable }); continue; } + // What the mount passes over by design stays unobserved by a pass that + // names no path, however it reaches it, so no listing's absence proves a + // copy there gone. In another mount's directory, no pass of this mount + // observes anything, named or not. + // Beneath a directory a by-design rule covers, a pass that isn't named + // observes nothing. A path a rule covers by its own name is observed as a + // walk observes it: passed over where the walk lists it, and absent where + // the walk lists its directory without it; a pass that reaches it by its + // path alone, as a listing, glob or grep naming it does, passes over it. + // A pass that names its scope, as a sync naming a path and a file tool's + // pass on the path it acts on do, sets aside only what covers the scope, + // at it or above it, as its walk does (#276): a directory passed over by + // design beneath a named directory stays passed over, and so does what + // lies beneath it. + let byDesign: string | undefined; + if (inOtherMount(path, mount.view.others)) { + byDesign = OTHER_MOUNT; + } else if (!opts.named) { + const slash = path.lastIndexOf('/'); + const above = slash < 0 ? undefined : passedOverAt(path.slice(0, slash), mount.view.ignore, mount.view.others); + const own = above === undefined && !walk ? passedOverAt(path, mount.view.ignore, mount.view.others) : undefined; + byDesign = above ?? own; + } else if (scope.kind === 'dir') { + byDesign = passedOverBetween(scope.dir, path, ignoreBeneath(scope.dir, mount.view.ignore), mount.view.others); + } + if (byDesign !== undefined) { + gathered.facts.set(path, { kind: 'unobserved', reason: byDesign }); + continue; + } if (walk && provenAbsent(walk, path)) { gathered.facts.set(path, { kind: 'absent' }); continue; } - if (walk?.others.get(path) === 'ignored by the mount') { - gathered.facts.set(path, { kind: 'unobserved', reason: 'ignored by the mount' }); + if (walk?.others.get(path) === IGNORED) { + gathered.facts.set(path, { kind: 'unobserved', reason: IGNORED }); continue; } if (walk && !walk.files.has(path) && !walk.othersTaken.has(path)) { @@ -915,8 +954,16 @@ export async function reconcilePass( // A link's own region hides only what lies beneath it: the link itself // is reported by path, with its own reason. const hides = tracked.some((p) => (region.kind !== 'symlink' && p === region.path) || p.startsWith(region.path + '/')); - if (region.kind === 'error' || region.kind === 'cap' || hides) result.incomplete.push({ path: region.path, reason: region.reason }); - // An ignored name that hides nothing tracked is reported nowhere else: + // A scope passed over whole is the answer to the pass that asked for + // it, so it's always named. Another mount's directory met beneath the + // scope is passed over by design even where this mount's tree holds + // copies there: no tool of this mount syncs, pushes or clears them + // (listings show them, naming their holder), so naming the directory + // as unreached would never clear. + const wholeScope = scope.kind === 'dir' && scope.dir !== '' && region.path === scope.dir; + const unreached = hides && region.reason !== OTHER_MOUNT; + if (region.kind === 'error' || region.kind === 'cap' || unreached || wholeScope) result.incomplete.push({ path: region.path, reason: region.reason }); + // A name passed over by design that hides nothing tracked is reported nowhere else: // counted, so a sync that says 0 also says what it didn't look at // (agent-framework #276). A symlink region isn't counted: a link the // walk took is a candidate the pass reports by path, taken through the @@ -975,6 +1022,9 @@ export interface PushResult { sequence?: number; } +/** Why a push skips a path in another mount's directory nested in its own. */ +export const NOT_PUSHED_IN_OTHER_MOUNT = "it lies in another mount's directory, which no push of this mount writes"; + /** * Make disk hold the store's version of each path, under the same rule: * drafts are written, agreement is left alone, conflicts and unverifiable @@ -1013,6 +1063,18 @@ export async function pushPaths( for (const path of paths) result.skipped.push({ path, reason: root.reason }); return result; } + // Another mount's directory nested in this one is that mount's: no push + // of this mount writes or deletes a path in it, forced or not. (On a mount + // that follows links, a link of this mount's that leads there is read as + // this mount's own path; on one that doesn't, the link is refused.) + if (mount.view.others.length > 0) { + const own: string[] = []; + for (const path of paths) { + if (inOtherMount(path, mount.view.others)) result.skipped.push({ path, reason: NOT_PUSHED_IN_OTHER_MOUNT }); + else own.push(path); + } + paths = own; + } type Plan = { path: string; diff --git a/src/modules/workspace/types.ts b/src/modules/workspace/types.ts index 0ee943f53..f43c9b13e 100644 --- a/src/modules/workspace/types.ts +++ b/src/modules/workspace/types.ts @@ -38,6 +38,11 @@ export interface MountConfig { * - Exact name match: ".git", "node_modules" * - Directory glob: "node_modules/**" (anything under node_modules) * - Extension glob: "*.pyc" (any file ending in .pyc) + * A sync without a path and the watcher pass over what the list covers, + * and over entries with an editor's swap or backup file name whatever it + * says, a directory with what lies beneath it: `.name.swp` and `.name.swx` + * (the whole name), and names ending in `~` or in `.subl…tmp`, matched by + * `^\..+\.sw[px]$`, `~$` and `\.subl.*\.tmp$`. */ ignore?: string[]; /** Whether to follow symlinks (default: false) */ diff --git a/src/modules/workspace/watcher.ts b/src/modules/workspace/watcher.ts index f853b0365..118a31562 100644 --- a/src/modules/workspace/watcher.ts +++ b/src/modules/workspace/watcher.ts @@ -8,6 +8,7 @@ import { watch, type FSWatcher } from 'chokidar'; import { mkdirSync, statSync } from 'node:fs'; import { basename, sep } from 'node:path'; +import { passedOverAt } from './observe.js'; import { type MountConfig } from './types.js'; export type FsOp = 'created' | 'modified' | 'deleted'; @@ -48,7 +49,8 @@ interface RootIdentity { export class MountWatcher { private watcher: FSWatcher | null = null; // Most recent op seen per path within the current debounce window. - // create -> modify collapses to 'created' (single batch); delete always wins. + // create -> modify collapses to 'created' (single batch); a trailing delete + // wins, and delete -> create reads as 'modified' (see mergeOp). private pendingChanges = new Map(); private debounceTimer: ReturnType | null = null; private rootPollTimer: ReturnType | null = null; @@ -67,6 +69,8 @@ export class MountWatcher { config: MountConfig, onChange: (changes: FsChange[]) => void, lifecycle: WatcherLifecycle = {}, + /** Other mounts' directories nested in this one, mount-relative: passed over as a walk passes over them. */ + private readonly others: readonly string[] = [], ) { this.config = config; this.debounceMs = config.watchDebounceMs ?? 300; @@ -101,10 +105,28 @@ export class MountWatcher { } } - const ignored = this.config.ignore ?? []; + const ignore = this.config.ignore ?? []; this.watcher = watch(this.config.path, { - ignored: ignored.length > 0 ? ignored : undefined, + // What a walk passes over by design, by the path relative to the mount: + // another mount's directory, the mount's ignore list and editors' swap + // and backup files. chokidar 4 + // compares a string given here with the whole absolute path, so the + // list itself would match nothing. + ignored: (path: string) => { + const relative = this.toRelative(path); + return relative !== null && passedOverAt(relative, ignore, this.others) !== undefined; + }, + // Its filter for editors' files tested the whole path, so a directory + // with a dot in its name hid every `*.swp` beneath it, and the walk + // didn't agree. An editor's save by rename then arrives as an unlink at + // once, and an add once `awaitWriteFinish` finds the new copy settled. + // A pass reads disk, not the ops, after the debounce's quiet spell + // following the last event: a copy on disk by then is taken in as a + // change, one still being written is read as it stands and taken in + // again when its add comes, and a re-add after a longer spell is a + // deletion and then a creation. + atomic: false, persistent: true, ignoreInitial: true, followSymlinks: this.config.followSymlinks ?? false, diff --git a/test/workspace-three-way.test.ts b/test/workspace-three-way.test.ts index a41f4c216..fdfa9dd75 100644 --- a/test/workspace-three-way.test.ts +++ b/test/workspace-three-way.test.ts @@ -18,12 +18,12 @@ import { open, statfs } from 'node:fs/promises'; import { createHash } from 'node:crypto'; import { createRequire, syncBuiltinESMExports } from 'node:module'; import { tmpdir } from 'node:os'; -import { basename, dirname, join } from 'node:path'; +import { basename, dirname, isAbsolute, join, relative } from 'node:path'; import { JsStore } from '@animalabs/chronicle'; import { WorkspaceModule } from '../src/modules/workspace/index.js'; import { DISK_AGREEMENT_RECORD_TYPE, DiskAgreement } from '../src/modules/workspace/disk-agreement.js'; import { RecordJournal } from '../src/record-journal.js'; -import { confirmAbsent, fingerprintVouches, filesystemTrustsCtime, fsTypeTrustsCtime, observePath, provenAbsent, readPath, walkScope, type MountView } from '../src/modules/workspace/observe.js'; +import { confirmAbsent, fingerprintVouches, filesystemTrustsCtime, fsTypeTrustsCtime, inOtherMount, observePath, provenAbsent, readPath, walkScope, type MountView } from '../src/modules/workspace/observe.js'; import { EffectFailed, syncDirectories, unlinkContained, writeContained } from '../src/modules/workspace/effects.js'; import { pushPaths, settleAdoptionBeforeMutation } from '../src/modules/workspace/reconcile.js'; import type { MountConfig, WorkspaceConfig } from '../src/modules/workspace/types.js'; @@ -721,6 +721,658 @@ describe('a sync says what it passed over (agent-framework #276)', () => { }); }); +/** The walk's reason for an editor's swap or backup file. */ +const EDITOR = "an editor's swap or backup file"; + +/** Waits, polling, until `ok` holds. */ +async function until(ok: () => boolean, what: string, ms = 5000): Promise { + for (const end = Date.now() + ms; Date.now() < end; await new Promise((r) => setTimeout(r, 25))) if (ok()) return; + assert.fail(what); +} + +describe("editors' swap and backup files are passed over by name (connectome-host #278)", () => { + + test('a sync without a path passes over them by their own whole name, wherever they are, and counts them', async (t) => { + const env = new Env(t); + const m = await env.open(); + // A directory with such a name is passed over with what lies beneath it. + for (const f of ['.gate.json.swp', '.gate.json.swx', 'gate.json~', '.x.subl3f9.tmp', 'deep/dir/.notes.md.swp', 'drafts~/x.txt']) env.writeDisk(f, 'editor state'); + // Not editors' names: one dot before `swp`, no leading dot, another + // backup suffix, a tilde mid-name, Sublime's form not the whole name, and + // a plain name under a directory with a dot in its name. + const taken = ['gate.json', 'notes.swp', 'report.v2.swp', 'backup.txt.bak', 'notes~draft.md', 'notes.sublime.tmpl', 'a.subl.tmp.csv', '.cfg/notes.swp']; + for (const f of taken) env.writeDisk(f, 'taken in'); + + const data = await call(m, 'sync', {}); + assert.equal(data.totalSynced, taken.length); + assert.deepEqual(env.store.treeList(TREE).map((e: { path: string }) => e.path).sort(), [...taken].sort()); + assert.deepEqual(data.passedOver, [{ mount: 'work', reason: EDITOR, count: 6 }]); + assert.equal((await listing(m)).has('gate.json~'), false, 'a listing passes over them too'); + + // Naming a path beneath such a directory syncs it, and says why a sync + // without one won't. + const named = await call(m, 'sync', { path: 'work/drafts~/x.txt' }); + assert.equal(named.totalSynced, 1); + assert.deepEqual(named.ignored, [{ mount: 'work', path: 'drafts~/x.txt', + note: "it has an editor's swap or backup file name, or lies under one, so a sync without a path won't maintain it" }]); + }); + + test('a stored copy is treated as one the ignore list covers: a change on disk is passed over, a deletion is taken', async (t) => { + const env = new Env(t); + const m = await env.open({ ignore: ['*.md'] }); + env.writeDisk('LOG.md', 'v1'); + env.writeDisk('gate.json~', 'v1'); + + // Naming a path syncs it, and the result says a sync without one won't. + const named = await call(m, 'sync', { path: 'work/gate.json~' }); + assert.equal(named.totalSynced, 1); + assert.deepEqual(named.ignored, [{ mount: 'work', path: 'gate.json~', + note: "it has an editor's swap or backup file name, or lies under one, so a sync without a path won't maintain it" }]); + await call(m, 'sync', { path: 'work/LOG.md' }); + const stored = (f: string) => env.store.treeGet(TREE, f)?.blobHash; + const v1 = { 'LOG.md': stored('LOG.md'), 'gate.json~': stored('gate.json~') }; + assert.ok(v1['LOG.md'] && v1['gate.json~'], 'both are stored'); + + env.writeDisk('LOG.md', 'v2'); + env.writeDisk('gate.json~', 'v2'); + const bare = await call(m, 'sync', {}); + assert.equal(bare.totalSynced, 0); + for (const f of ['LOG.md', 'gate.json~'] as const) { + assert.equal(stored(f), v1[f], `${f}: the stored copy stays as it was`); + assert.equal((await listing(m)).get(f)?.state, 'unverified', f); + // A grep that reaches it by its own path passes over it too. + await call(m, 'grep', { pattern: 'v', path: `work/${f}` }); + assert.equal(stored(f), v1[f], `${f}: nor by a grep naming it`); + } + assert.deepEqual((bare.incomplete as Array<{ path: string; reason: string }>).map(({ path, reason }) => [path, reason]).sort(), + [['LOG.md', 'ignored by the mount'], ['gate.json~', EDITOR]]); + + // A complete listing of the directory lacks them: absent, as anything it lacks. + env.rmDisk('LOG.md'); + env.rmDisk('gate.json~'); + await call(m, 'sync', {}); + for (const f of ['LOG.md', 'gate.json~']) assert.equal(env.store.treeGet(TREE, f), null, `${f}: its deletion is taken`); + }); + + test('a materialize treats a stored copy as any file the workspace holds, as it treats one the ignore list covers', async (t) => { + const env = new Env(t); + const m = await env.open({ ignore: ['*.md'] }); + const files = ['LOG.md', 'gate.json~', 'plain.txt']; + const disk = (f: string) => (existsSync(env.disk(f)) ? readFileSync(env.disk(f), 'utf8') : null); + for (const f of files) { + env.writeDisk(f, 'v1'); + await call(m, 'sync', { path: `work/${f}` }); + } + const main = env.store.currentBranch().name; + env.store.createBranch('earlier', main); + // Deleted on disk, and the deletion taken in by a sync without a path. + for (const f of files) env.rmDisk(f); + await call(m, 'sync', {}); + for (const f of files) assert.equal(env.store.treeGet(TREE, f), null, f); + + // A branch whose history holds them writes them back where disk lacks them. + env.store.switchBranch('earlier'); + await m.materializeMount('work'); + for (const f of files) assert.equal(disk(f), 'v1', `${f} is written back`); + + // A change the workspace owes goes to disk while disk has the copy they agreed on. + for (const f of files) await call(m, 'write', { path: `work/${f}`, content: 'v2' }); + const owed = await call(m, 'materialize', { path: 'work' }); + assert.deepEqual((owed.materialized as Array<{ path: string }>).map((w) => w.path).sort(), [...files].sort()); + for (const f of files) assert.equal(disk(f), 'v2', f); + + // A change disk made is left alone, as a conflict, unless forced. + for (const f of files) { + await call(m, 'write', { path: `work/${f}`, content: 'v3' }); + env.writeDisk(f, 'edited on disk'); + } + const left = await call(m, 'materialize', { path: 'work' }); + assert.deepEqual((left.skipped as Array<{ reason: string }>).map((s) => s.reason.split(':')[0]).sort(), [...files].sort()); + for (const s of left.skipped as Array<{ reason: string }>) assert.match(s.reason, /\(conflict: both-changed\)/); + for (const f of files) assert.equal(disk(f), 'edited on disk', f); + await call(m, 'materialize', { path: 'work', force: true }); + for (const f of files) assert.equal(disk(f), 'v3', `${f}: forced`); + }); +}); + +describe('a listing, glob or grep inside a directory passed over by design', () => { + for (const [kind, ignore, dir, reason] of [ + ["an editor's name", [], 'drafts~', EDITOR], + ['a bare-name ignore entry', ['node_modules'], 'node_modules', 'ignored by the mount'], + ['a path ignore entry', ['cache/**'], 'cache', 'ignored by the mount'], + ] as const) { + test(`takes nothing in there and names the region, so it tracks nothing (${kind})`, async (t) => { + const env = new Env(t); + env.writeDisk(`${dir}/a.txt`, 'x'); + env.writeDisk(`${dir}/pkg/index.js`, 'x'); + const m = await env.open({ ignore: [...ignore] }); + // The region says why, and how to take what's there in on purpose. + const region = (path: string) => ({ path, reason: `${reason}: a sync that names this path takes in what's there, editors' files aside` }); + // At the directory, and at a scope beneath it, which a directory above + // passes over too. + for (const scope of [dir, `${dir}/pkg`]) { + for (const recursive of [false, true]) { + const ls = await call(m, 'ls', { path: `work/${scope}`, recursive }); + assert.deepEqual(ls.entries, [], `ls ${scope} (recursive ${recursive})`); + assert.deepEqual(ls.incomplete, [region(scope)], `ls ${scope} (recursive ${recursive}) names the region`); + } + const globbed = await call(m, 'glob', { pattern: '**', path: `work/${scope}` }); + assert.deepEqual(globbed.matches, [], `glob ${scope}`); + assert.deepEqual(globbed.incomplete, [region(`work/${scope}`)], `glob ${scope}`); + const grepped = await call(m, 'grep', { pattern: 'x', path: `work/${scope}` }); + assert.deepEqual(grepped.results, [], `grep ${scope}`); + assert.deepEqual(grepped.incomplete, [region(`work/${scope}`)], `grep ${scope}`); + } + assert.deepEqual(env.store.treeList(TREE), [], 'nothing beneath it is tracked'); + // A sync that names it still syncs what's there (#276). + const named = await call(m, 'sync', { path: `work/${dir}` }); + assert.equal(named.totalSynced, 2); + const stored = () => env.store.treeList(TREE).map((e: { path: string; blobHash: string }) => `${e.path} ${e.blobHash}`).sort(); + const before = stored(); + assert.deepEqual(before.map((e) => e.split(' ')[0]), [`${dir}/a.txt`, `${dir}/pkg/index.js`], 'the named sync stored both'); + // The copies it stored are listed, unobserved, and a change on disk + // isn't taken in by a listing, glob or grep there. + env.writeDisk(`${dir}/a.txt`, 'changed on disk'); + const listed = await listing(m, `work/${dir}`); + assert.deepEqual([...listed.keys()].sort(), [`${dir}/a.txt`, `${dir}/pkg/index.js`]); + for (const e of listed.values()) assert.equal(e.state, 'unverified', `${e.path} is unobserved`); + await call(m, 'glob', { pattern: '**', path: `work/${dir}` }); + await call(m, 'grep', { pattern: 'x', path: `work/${dir}` }); + assert.deepEqual(stored(), before, 'nothing was taken in'); + }); + } +}); + +describe('a file tool naming a path passed over by design', () => { + // The last column: what a sync without a path then reports under skipped. + // Beneath a directory passed over by design, no walk can show the deleted + // file gone either, so its evidence is reported too, as on main; a file + // passed over by its own name is shown absent by the walk of its directory. + for (const [kind, ignore, read, edit, gone, skipped] of [ + ['beneath a directory an ignore entry covers', ['node_modules'], 'node_modules/pkg/index.d.ts', 'node_modules/pkg/edit.d.ts', 'node_modules/pkg/gone.d.ts', + ['node_modules/pkg/edit.d.ts', 'node_modules/pkg/gone.d.ts', 'node_modules/pkg/index.d.ts']], + ['that an ignore entry covers by its own name', ['*.log'], 'build.log', 'edit.log', 'gone.log', ['build.log', 'edit.log']], + ["that is an editor's backup file", [], 'draft.md~', 'edit.md~', 'gone.md~', ['draft.md~', 'edit.md~']], + ] as const) { + test(`reads, edits and deletes a file ${kind} that the store doesn't hold, as a sync naming it takes it in (#276)`, async (t) => { + const env = new Env(t); + for (const f of [read, edit, gone]) env.writeDisk(f, `${f} v1\n`); + const m = await env.open({ ignore: [...ignore], autoMaterialize: true }); + await call(m, 'sync', {}); + assert.deepEqual(env.store.treeList(TREE), [], 'a sync without a path passes over all three'); + assert.equal((await call(m, 'read', { path: `work/${read}` })).content, ` 1\t${read} v1`); + await call(m, 'edit', { path: `work/${edit}`, oldString: 'v1', newString: 'v2' }); + assert.equal(env.readDisk(edit), `${edit} v2\n`, 'the edit reached disk'); + assert.equal((await call(m, 'delete', { path: `work/${gone}` })).deleted, true); + assert.equal(existsSync(env.disk(gone)), false, 'the deletion reached disk'); + const paths = () => env.store.treeList(TREE).map((e: { path: string }) => e.path).sort(); + assert.deepEqual(paths(), [edit, read].sort(), 'the read and the edit took their files in'); + // What they took in stays passed over by a sync without a path, which says so. + const bare = await call(m, 'sync', {}); + assert.deepEqual((bare.skipped as Array<{ path: string }>).map((s) => s.path).sort(), [...skipped]); + assert.deepEqual(paths(), [edit, read].sort()); + }); + } + + test("the nested mount's refusal comes first, and the lazy pass takes nothing in from another mount's directory", async (t) => { + const env = new Env(t); + env.writeDisk('node_modules/pkg/inner.txt', 'inner'); + const m = await env.open({}, { mounts: [ + { name: 'work', path: env.dir, mode: 'read-write', watch: 'never', ignore: ['node_modules'] }, + { name: 'deep', path: env.disk('node_modules/pkg'), mode: 'read-write', watch: 'never' }, + ] }); + for (const [tool, input] of [ + ['read', { path: 'work/node_modules/pkg/inner.txt' }], + ['edit', { path: 'work/node_modules/pkg/inner.txt', oldString: 'inner', newString: 'outer' }], + ['delete', { path: 'work/node_modules/pkg/inner.txt' }], + ] as const) { + const said = await refused(m, tool, input); + assert.ok(said.includes('mount "deep"') && said.includes('"deep/inner.txt"'), `${tool}: ${said}`); + } + // The lazy pass itself, reached past the refusal, observes nothing there. + const mount = (m as any).mounts.get('work'); + await (m as any).withMount(mount, () => (m as any).ensureSynced(mount, 'node_modules/pkg/inner.txt')); + assert.deepEqual(env.store.treeList(TREE), [], 'the outer store gains nothing'); + assert.equal(env.readDisk('node_modules/pkg/inner.txt'), 'inner'); + }); +}); + +describe('a mount nested inside another', () => { + test('is passed over by the outer mount as the directory it is, and nothing else is', async (t) => { + const env = new Env(t); + for (const f of ['top.txt', 'cat.txt', 'a/b/inner.txt', 'c/near.txt', 'sub/c', 'sub/x/c/f.txt', '..cache/f.txt']) env.writeDisk(f, f); + const m = await env.open({}, { mounts: [ + // `a/b` listed by hand matches nothing, so it doesn't stand in for the + // nested mount's entry. + { name: 'work', path: env.dir, mode: 'read-write', watch: 'never', ignore: ['a/b'] }, + { name: 'deep', path: env.disk('a/b'), mode: 'read-write', watch: 'never' }, + { name: 'near', path: env.disk('c'), mode: 'read-write', watch: 'never' }, + // A name that only begins with two dots is inside the outer mount. + { name: 'dots', path: env.disk('..cache'), mode: 'read-write', watch: 'never' }, + ] }); + const OTHER = "another mount's directory"; + const bare = await call(m, 'sync', { mount: 'work' }); + // None of the nested mounts' files, and the entries that only share a + // nested mount's name. + const tree = () => env.store.treeList(TREE).map((e: { path: string }) => e.path).sort(); + // cat.txt only begins with the nested mount c's name. + assert.deepEqual(tree(), ['cat.txt', 'sub/c', 'sub/x/c/f.txt', 'top.txt']); + assert.deepEqual((bare.passedOver as Array<{ reason: string; count: number }>).filter((p) => p.reason === OTHER).map((p) => p.count), [3], + 'it passes over the three nested mounts, each as its directory'); + // A sync that names a nested mount's directory, or a path beneath it, + // takes nothing in, and its region gives no advice: no sync of this + // mount does. + for (const [path, use] of [['a/b', 'deep'], ['a/b/inner.txt', 'deep/inner.txt'], ['c', 'near']]) { + const named = await call(m, 'sync', { path: `work/${path}` }); + assert.equal(named.totalSynced, 0, path); + assert.deepEqual((named.ignored as Array<{ note: string }>).map((i) => i.note), + [`it lies in mount "${use.split('/')[0]}", nested in this one, which no sync of this mount takes in: use "${use}"`], path); + // A named file is a paths pass, which makes no region; a directory's + // walk names it, with the mount that holds it. + assert.deepEqual(((named.incomplete ?? []) as Array<{ path: string; reason: string }>).map((r) => [r.path, r.reason]), + path === 'a/b/inner.txt' ? [] : [[path, `it lies in mount "${use}", nested in this one: use "${use}"`]], path); + } + // A listing, glob or grep of it lists and searches nothing beneath it, + // takes nothing in, and names the region under `incomplete`. + for (const [tool, input, list, region] of [ + ['ls', { path: 'work/a/b' }, 'entries', 'a/b'], + ['glob', { path: 'work/a/b', pattern: '*' }, 'matches', 'work/a/b'], + ['grep', { path: 'work/a/b', pattern: 'inner' }, 'results', 'work/a/b'], + ] as const) { + const listed = await call(m, tool, input as Record); + assert.deepEqual(listed[list], [], `${tool} lists nothing beneath it`); + assert.deepEqual((listed.incomplete as Array<{ path: string; reason: string }>).map((r) => [r.path, r.reason]), + [[region, 'it lies in mount "deep", nested in this one: use "deep"']], tool); + } + assert.deepEqual(tree(), ['cat.txt', 'sub/c', 'sub/x/c/f.txt', 'top.txt'], 'nothing of theirs was taken in'); + // A file that only shares a nested mount's name is this mount's, named or not. + env.writeDisk('sub/c', 'changed'); + const subc = await call(m, 'sync', { path: 'work/sub/c' }); + assert.equal(subc.totalSynced, 1); + }); + + test("never reaches the outer mount's tree while both mounts' watchers run", async (t) => { + const env = new Env(t); + env.writeDisk('top.txt', 'top'); + env.writeDisk('a/b/inner.txt', 'inner'); + const m = await env.open({}, { mounts: [ + { name: 'work', path: env.dir, mode: 'read-write', watch: 'always', watchDebounceMs: 50 }, + { name: 'deep', path: env.disk('a/b'), mode: 'read-write', watch: 'always', watchDebounceMs: 50 }, + ] }); + const mount = (m as any).mounts.get('work'); + const deep = (m as any).mounts.get('deep'); + const tree = () => env.store.treeList(TREE).map((e: { path: string }) => e.path).sort(); + const deepTree = () => env.store.treeList('workspace/deep/tree').map((e: { path: string }) => e.path).sort(); + await until(() => !!mount.watcherReadyAt && !!deep.watcherReadyAt && tree().includes('top.txt') && deepTree().includes('inner.txt'), + 'both watchers are ready and their catch-up scans ran'); + // The inner mount takes its new file in, so the change has reached the + // watchers; the outer mount's own new file, written after it, is taken + // in after that. (MountWatcher's own test pins the outer watcher's filter.) + env.writeDisk('a/b/new.txt', 'nested'); + await until(() => deepTree().includes('new.txt'), 'the inner mount takes its new file in'); + env.writeDisk('later.txt', 'outer'); + await until(() => tree().includes('later.txt'), 'the outer mount takes its own new file in'); + assert.deepEqual(tree(), ['later.txt', 'top.txt']); + }); +}); + +describe("another mount's directory, to the outer mount's file tools and pushes", () => { + const nested = (env: Env): Partial => ({ mounts: [ + { name: 'work', path: env.dir, mode: 'read-write', watch: 'never' }, + { name: 'deep', path: env.disk('a/b'), mode: 'read-write', watch: 'never' }, + ] }); + + test('every file tool of the outer mount refuses a path there, naming the mount that holds it and the path to use', async (t) => { + const env = new Env(t); + env.writeDisk('top.txt', 'top'); + env.writeDisk('a/b/inner.txt', 'inner'); + const m = await env.open({}, nested(env)); + for (const [tool, input, use] of [ + ['read', { path: 'work/a/b/inner.txt' }, 'deep/inner.txt'], + ['write', { path: 'work/a/b/inner.txt', content: 'outer' }, 'deep/inner.txt'], + ['edit', { path: 'work/a/b/inner.txt', oldString: 'inner', newString: 'outer' }, 'deep/inner.txt'], + ['delete', { path: 'work/a/b/inner.txt' }, 'deep/inner.txt'], + ['write', { path: 'work/a/b/sub/new.txt', content: 'outer' }, 'deep/sub/new.txt'], + ['read', { path: 'work/a/b' }, 'deep'], + ] as const) { + const said = await refused(m, tool, input as Record); + assert.ok(said.includes('mount "deep"') && said.includes(`"${use}"`), `${tool} ${input.path}: ${said}`); + } + assert.equal(env.readDisk('a/b/inner.txt'), 'inner'); + assert.equal(existsSync(env.disk('a/b/sub/new.txt')), false); + assert.equal(env.store.treeList(TREE).some((e: { path: string }) => e.path.startsWith('a/b/')), false, 'the outer store gains nothing there'); + await assert.rejects(m.readFileFromDisk('work/a/b/inner.txt'), (err: unknown) => (err as { code?: string }).code === 'other_mount'); + assert.equal(m.resolveAbsolutePath('work/a/b/inner.txt'), null); + const binary = await m.readBinary('work/a/b/inner.txt'); + assert.ok('error' in binary && binary.error.includes('"deep/inner.txt"'), JSON.stringify(binary)); + const fromDisk = await m.readBinaryFromDisk('work/a/b/inner.txt'); + assert.ok('error' in fromDisk && fromDisk.error.includes('"deep/inner.txt"'), JSON.stringify(fromDisk)); + const written = await m.writeBinary('work/a/b/img.png', Buffer.from('png'), 'image/png'); + assert.ok(!written.success && String(written.error).includes('"deep/img.png"'), JSON.stringify(written)); + const image = await refused(m, 'read_image', { path: 'work/a/b/img.png' }); + assert.ok(image.includes('"deep/img.png"'), image); + // The path it names works, and the outer mount's own files stay its own. + assert.match((await call(m, 'read', { path: 'deep/inner.txt' })).content, /inner/); + assert.match((await call(m, 'read', { path: 'work/top.txt' })).content, /top/); + }); + + test('a spelling that lands in its directory is refused as the path it lands on, and one that lands beside it is not', async (t) => { + const env = new Env(t); + env.writeDisk('top.txt', 'top'); + env.writeDisk('a/b/inner.txt', 'inner'); + const m = await env.open({}, nested(env)); + for (const path of ['work/./a/b/inner.txt', 'work/a/c/../b/inner.txt', 'work/a//b/inner.txt']) { + const said = await refused(m, 'read', { path }); + assert.ok(said.includes('mount "deep"') && said.includes('"deep/inner.txt"'), `${path}: ${said}`); + await assert.rejects(m.readFileFromDisk(path), (err: unknown) => (err as { code?: string }).code === 'other_mount', path); + assert.equal(m.resolveAbsolutePath(path), null, path); + const fromDisk = await m.readBinaryFromDisk(path); + assert.ok('error' in fromDisk && fromDisk.error.includes('"deep/inner.txt"'), `${path}: ${JSON.stringify(fromDisk)}`); + } + // A spelling that climbs back out of it is the outer mount's own path, + // judged where it lands: read through what doesn't touch the store. + env.writeDisk('a/x.txt', 'x'); + for (const [path, content] of [['work/a/b/../x.txt', 'x'], ['work/a/b/../../top.txt', 'top']]) { + assert.equal((await m.readFileFromDisk(path)).bytes.toString(), content, path); + assert.notEqual(m.resolveAbsolutePath(path), null, path); + } + }); + + test("grep of the outer mount returns nothing of its copy of a path there, and names the region", async (t) => { + const env = new Env(t); + env.writeDisk('top.txt', 'top'); + env.writeDisk('a/b/inner.txt', 'inner'); + let m = await env.open(); + await call(m, 'sync', {}); + await env.shutdown(m); + env.store = JsStore.openOrCreate({ path: env.storePath }); + m = await env.open({}, nested(env)); + const bytes = Buffer.from('stale outer copy'); + env.store.treeSet(TREE, 'a/b/inner.txt', { blobHash: env.store.storeBlob(bytes, 'text/plain'), size: bytes.byteLength, mode: 0o644 }); + const regions = (found: { incomplete?: Array<{ path: string; reason: string }> }) => (found.incomplete ?? []).map((r) => [r.path, r.reason]); + // Beneath a scope, the nested directory is passed over by design. + const whole = await call(m, 'grep', { path: 'work', pattern: 'stale' }); + assert.deepEqual(whole.results, []); + assert.deepEqual(regions(whole), []); + // Named, it's the answer to what was asked: its region names the holder. + for (const [path, use] of [['work/a/b', 'deep'], ['work/a/b/inner.txt', 'deep/inner.txt'], ['work/a//b/inner.txt', 'deep/inner.txt']]) { + const found = await call(m, 'grep', { path, pattern: 'stale' }); + assert.deepEqual(found.results, [], path); + assert.deepEqual(regions(found), [[path, `it lies in mount "deep", nested in this one: use "${use}"`]], path); + } + // A file there the outer store doesn't hold is named too. + env.writeDisk('a/b/other.txt', 'stale words'); + const unheld = await call(m, 'grep', { path: 'work/a/b/other.txt', pattern: 'stale' }); + assert.deepEqual(unheld.results, []); + assert.deepEqual(regions(unheld), [['work/a/b/other.txt', 'it lies in mount "deep", nested in this one: use "deep/other.txt"']]); + }); + + test('every nested-mount check reads a path as disk access lands it: doubled and trailing separators and dot segments included', () => { + for (const [path, inside] of [ + ['a/b', true], ['a/b/x', true], ['a//b/x', true], ['a/b/', true], ['./a/b/x', true], ['a/c/../b/x', true], + ['a/b/../x', false], ['a/bc/x', false], ['a', false], ['', false], + ] as const) assert.equal(inOtherMount(path, ['a/b']), inside, path); + }); + + test('a doubled or trailing separator is the nested directory too: a sync or a listing of it takes nothing in, and no push of the outer mount writes there', async (t) => { + const env = new Env(t); + env.writeDisk('top.txt', 'top'); + env.writeDisk('a/b/inner.txt', 'inner'); + const m = await env.open({}, nested(env)); + const nestedKeys = () => env.store.treeList(TREE).map((e: { path: string }) => e.path).filter((p: string) => inOtherMount(p, ['a/b'])); + for (const path of ['work/a//b', 'work/a/b/', 'work/a//b/inner.txt']) { + const synced = await call(m, 'sync', { path }); + assert.equal(synced.totalSynced, 0, path); + assert.deepEqual(nestedKeys(), [], `${path}: nothing of theirs taken in`); + const listed = await call(m, 'ls', { path }); + assert.deepEqual(listed.entries, [], path); + } + // An outer copy spelled that way, as chronicle keeps it. + const bytes = Buffer.from('outer copy'); + env.store.treeSet(TREE, 'a//b/inner.txt', { blobHash: env.store.storeBlob(bytes, 'text/plain'), size: bytes.byteLength, mode: 0o644 }); + await call(m, 'materialize', {}); + await m.materializeMount('work'); + assert.equal(env.readDisk('a/b/inner.txt'), 'inner', "the nested mount's file is as it was"); + }); + + test("an outer copy in a nested mount's directory is passed over by design: no sync reports it incomplete, no count or bare materialize takes it up, and listings name its holder", async (t) => { + const env = new Env(t); + env.writeDisk('top.txt', 'top'); + env.writeDisk('a/b/inner.txt', 'inner'); + let m = await env.open(); + await call(m, 'sync', {}); + await env.shutdown(m); + env.store = JsStore.openOrCreate({ path: env.storePath }); + m = await env.open({}, nested(env)); + const bytes = Buffer.from('stale outer copy'); + env.store.treeSet(TREE, 'a/b/inner.txt', { blobHash: env.store.storeBlob(bytes, 'text/plain'), size: bytes.byteLength, mode: 0o644 }); + // A conflict and a workspace deletion recorded there, as main's bug could leave them. + const intents = (m as any).intents.get('work'); + intents.put('a/b/inner.txt', { conflict: { kind: 'both-changed', at: Date.now(), disk: null } }); + intents.put('a/b/gone.txt', { tombstone: { hash: 'f'.repeat(64) } }); + const bare = await call(m, 'sync', { mount: 'work' }); + assert.deepEqual(bare.incomplete ?? [], [], 'nothing incomplete: the sync looked everywhere it should'); + assert.deepEqual((bare.passedOver as Array<{ reason: string; count: number }>).filter((p) => p.reason === "another mount's directory").map((p) => p.count), [1]); + const status = (await call(m, 'status', {})).work; + assert.deepEqual([status.pendingChanges, status.conflicts, status.pendingWorkspaceDeletions], [0, 0, 0]); + const pushed = await call(m, 'materialize', {}); + assert.equal(JSON.stringify(pushed).includes('a/b/'), false, JSON.stringify(pushed)); + assert.equal(env.readDisk('a/b/inner.txt'), 'inner'); + // A materialize naming an ancestor, forced, leaves it out too. + const ancestor = await call(m, 'materialize', { path: 'work/a', force: true }); + assert.equal(JSON.stringify(ancestor).includes('a/b/'), false, JSON.stringify(ancestor)); + assert.equal(env.readDisk('a/b/inner.txt'), 'inner'); + // The copy stays in the outer branch's tree, and a listing shows it with its holder. + assert.ok(env.store.treeGet(TREE, 'a/b/inner.txt')); + const listed = await call(m, 'ls', { path: 'work', recursive: true }); + const entry = (listed.entries as Array<{ path: string; note?: string }>).find((e) => e.path === 'a/b/inner.txt'); + assert.equal(entry?.note, 'it lies in mount "deep", nested in this one: use "deep/inner.txt"'); + const globbed = await call(m, 'glob', { path: 'work', pattern: '**/*.txt' }); + const match = (globbed.matches as Array<{ path: string; note?: string }>).find((e) => e.path === 'work/a/b/inner.txt'); + assert.equal(match?.note, 'it lies in mount "deep", nested in this one: use "deep/inner.txt"'); + }); + + test("a materialize naming a path in a nested mount's directory, however spelled, pushes nothing and names the holder", async (t) => { + const env = new Env(t); + env.writeDisk('a/b/inner.txt', 'inner'); + let m = await env.open(); + await call(m, 'sync', {}); + await env.shutdown(m); + env.store = JsStore.openOrCreate({ path: env.storePath }); + m = await env.open({}, nested(env)); + const bytes = Buffer.from('outer copy'); + env.store.treeSet(TREE, 'a/b/inner.txt', { blobHash: env.store.storeBlob(bytes, 'text/plain'), size: bytes.byteLength, mode: 0o644 }); + for (const [path, use] of [['work/a/b', 'deep'], ['work/a/b/', 'deep'], ['work/a//b/inner.txt', 'deep/inner.txt'], ['work/a/b/inner.txt', 'deep/inner.txt']]) { + const pushed = await call(m, 'materialize', { path, force: true }); + assert.deepEqual(pushed.materialized ?? [], [], path); + assert.ok(JSON.stringify(pushed.skipped ?? []).includes(`use \\"${use}\\"`), `${path}: ${JSON.stringify(pushed)}`); + } + assert.equal(env.readDisk('a/b/inner.txt'), 'inner'); + }); + + test('a mount configured with a relative path contains its own paths and its root, generated files included', async (t) => { + const env = new Env(t); + env.writeDisk('a/b/inner.txt', 'inner'); + env.writeDisk('top.txt', 'top'); + const rel = relative(process.cwd(), env.dir); + assert.ok(!isAbsolute(rel), 'the test runs from somewhere the mount can be named relative to'); + const m = await env.open({}, { mounts: [ + { name: 'work', path: rel, mode: 'read-write', watch: 'never' }, + { name: 'deep', path: relative(process.cwd(), env.disk('a/b')), mode: 'read-write', watch: 'never' }, + ] }); + assert.match((await call(m, 'read', { path: 'work/top.txt' })).content, /top/); + assert.ok((await call(m, 'ls', { path: 'work' })).entries.some((e: { name: string }) => e.name === 'top.txt'), 'the root lists'); + assert.deepEqual((await call(m, 'ls', { path: 'deep' })).entries.map((e: { name: string }) => e.name), ['inner.txt']); + const said = await refused(m, 'read', { path: 'work/a/b/inner.txt' }); + assert.ok(said.includes('use "deep/inner.txt"'), said); + m.registerGeneratedTextFile('work/generated.md', () => 'generated', 'resident'); + const generated = await m.handleToolCall({ id: 't', name: 'read', callerAgentName: 'resident', input: { path: 'work/generated.md' } }); + assert.equal(generated.success, true, generated.error); + assert.match(JSON.stringify(generated.data), /generated/); + }); + + test('the path a refusal names works on a mount whose root is configured with a trailing separator', async (t) => { + const env = new Env(t); + env.writeDisk('a/b/inner.txt', 'inner'); + const m = await env.open({}, { mounts: [ + { name: 'work', path: env.dir + '/', mode: 'read-write', watch: 'never' }, + { name: 'deep', path: env.disk('a/b') + '/', mode: 'read-write', watch: 'never' }, + ] }); + const said = await refused(m, 'read', { path: 'work/a/b' }); + assert.ok(said.includes('use "deep"'), said); + const listed = await call(m, 'ls', { path: 'deep' }); + assert.deepEqual((listed.entries as Array<{ name: string }>).map((e) => e.name), ['inner.txt']); + assert.match((await call(m, 'read', { path: 'deep/inner.txt' })).content, /inner/); + }); + + test('a path in a mount nested two deep is named by its innermost mount', async (t) => { + const env = new Env(t); + env.writeDisk('a/b/c/x.txt', 'x'); + const m = await env.open({}, { mounts: [ + { name: 'work', path: env.dir, mode: 'read-write', watch: 'never' }, + { name: 'deep', path: env.disk('a/b'), mode: 'read-write', watch: 'never' }, + { name: 'deeper', path: env.disk('a/b/c'), mode: 'read-write', watch: 'never' }, + ] }); + for (const path of ['work/a/b/c/x.txt', 'deep/c/x.txt']) { + const said = await refused(m, 'read', { path }); + assert.ok(said.includes('mount "deeper"') && said.includes('"deeper/x.txt"'), `${path}: ${said}`); + } + }); + + test("no push of the outer mount writes a path there that its store holds: a bare materialize leaves it out, and a named one says why, forced or not", async (t) => { + const env = new Env(t); + env.writeDisk('top.txt', 'top'); + env.writeDisk('a/b/inner.txt', 'inner'); + // The outer mount stored it before the inner one was configured. + let m = await env.open(); + await call(m, 'sync', {}); + assert.ok(env.store.treeGet(TREE, 'a/b/inner.txt')); + await env.shutdown(m); + env.store = JsStore.openOrCreate({ path: env.storePath }); + m = await env.open({}, nested(env)); + // Its copy moves on, as a write through the outer mount once left it. + const bytes = Buffer.from('outer copy'); + env.store.treeSet(TREE, 'a/b/inner.txt', { blobHash: env.store.storeBlob(bytes, 'text/plain'), size: bytes.byteLength, mode: 0o644 }); + const SKIPPED = 'a/b/inner.txt: it lies in mount "deep", nested in this one, which no push of this mount writes: use "deep/inner.txt"'; + const reasons = (pushed: { skipped?: Array<{ reason: string }> }) => + (pushed.skipped ?? []).map((x) => x.reason).filter((r) => r.startsWith('a/b/inner.txt')); + // A bare materialize leaves it out: it isn't this mount's to push. + const bare = await call(m, 'materialize', { mount: 'work' }); + assert.deepEqual(reasons(bare), []); + // Named, the caller is told whose it is, forced or not. + const forced = await call(m, 'materialize', { path: 'work/a/b/inner.txt', force: true }); + assert.deepEqual(reasons(forced), [SKIPPED]); + // An undo's or a branch switch's push, forced over every tree path. + const written = await m.materializeMount('work'); + assert.equal(written.some((p) => p.endsWith('a/b/inner.txt')), false); + assert.equal(env.readDisk('a/b/inner.txt'), 'inner', "the inner mount's file is as it was"); + }); + + test('a nested mount beneath a directory the ignore list covers is still another mount\'s directory to a sync that names a path', async (t) => { + const env = new Env(t); + env.writeDisk('vendor/own.txt', 'own'); + env.writeDisk('vendor/lib/x.txt', 'x'); + const m = await env.open({}, { mounts: [ + { name: 'work', path: env.dir, mode: 'read-write', watch: 'never', ignore: ['vendor'] }, + { name: 'lib', path: env.disk('vendor/lib'), mode: 'read-write', watch: 'never' }, + ] }); + const OTHER = 'it lies in mount "lib", nested in this one, which no sync of this mount takes in: use "lib/x.txt"'; + const file = await call(m, 'sync', { path: 'work/vendor/lib/x.txt' }); + assert.equal(file.totalSynced, 0); + assert.deepEqual((file.ignored as Array<{ note: string }>).map((i) => i.note), [OTHER]); + // Naming the ignored directory takes in its own files (#276), and none of the nested mount's. + await call(m, 'sync', { path: 'work/vendor' }); + const tree = env.store.treeList(TREE).map((e: { path: string }) => e.path).sort(); + assert.deepEqual(tree, ['vendor/own.txt']); + }); +}); + +describe('a named sync of a directory, beneath which a directory is passed over by design', () => { + test('keeps a copy stored beneath the passed-over directory, whether that directory is there or gone', async (t) => { + const env = new Env(t); + env.writeDisk('src/main.txt', 'main'); + env.writeDisk('src/cache/x', 'v1'); + const m = await env.open({ ignore: ['cache'] }); + await call(m, 'sync', { path: 'work/src/cache/x' }); + const v1 = env.store.treeGet(TREE, 'src/cache/x')?.blobHash; + assert.ok(v1, 'a sync naming the file stores it (#276)'); + env.writeDisk('src/cache/x', 'v2'); + await call(m, 'sync', { path: 'work/src' }); + assert.equal(env.store.treeGet(TREE, 'src/cache/x')?.blobHash, v1, 'present and changed: unobserved'); + rmSync(env.disk('src/cache'), { recursive: true, force: true }); + await call(m, 'sync', { path: 'work/src' }); + assert.equal(env.store.treeGet(TREE, 'src/cache/x')?.blobHash, v1, 'gone: still unobserved, not taken as deleted'); + // Naming the passed-over directory itself takes in what's there, as #276 has it. + await call(m, 'sync', { path: 'work/src/cache' }); + assert.equal(env.store.treeGet(TREE, 'src/cache/x'), null, 'a sync naming it takes its deletion'); + }); +}); + +describe('a copy stored beneath a directory passed over by design', () => { + for (const [kind, ignore, dir, reason] of [ + ['the ignore list covers', ['cache/**'], 'cache', 'ignored by the mount'], + ["with an editor's name", [], 'drafts~', EDITOR], + ] as const) { + test(`is unobserved there, and stays when it is deleted on disk, since no walk lists that directory (${kind})`, async (t) => { + const env = new Env(t); + env.writeDisk(`${dir}/x`, 'v1'); + const m = await env.open({ ignore: [...ignore] }); + await call(m, 'sync', { path: `work/${dir}/x` }); + const v1 = env.store.treeGet(TREE, `${dir}/x`)?.blobHash; + assert.ok(v1, 'a named sync stores it'); + env.writeDisk(`${dir}/x`, 'v2'); + const bare = await call(m, 'sync', {}); + assert.deepEqual((bare.incomplete as Array<{ path: string; reason: string }>).map(({ path, reason }) => [path, reason]), [[dir, reason]]); + assert.equal((await listing(m)).get(`${dir}/x`)?.state, 'unverified'); + assert.equal(env.store.treeGet(TREE, `${dir}/x`)?.blobHash, v1, 'a change on disk is passed over'); + // A grep that names the file reaches it by its path, and still takes + // nothing in: what the mount passes over by design stays unobserved, + // however a pass reaches it. + await call(m, 'grep', { pattern: 'v', path: `work/${dir}/x` }); + assert.equal(env.store.treeGet(TREE, `${dir}/x`)?.blobHash, v1, 'nor by a grep naming it'); + env.rmDisk(`${dir}/x`); + await call(m, 'sync', {}); + assert.ok(env.store.treeGet(TREE, `${dir}/x`), 'a sync without a path leaves it as it was'); + // With the directory itself gone, the root's listing would prove the + // copy absent, but the mount passes over that directory by design. + rmSync(env.disk(dir), { recursive: true, force: true }); + await call(m, 'sync', {}); + assert.ok(env.store.treeGet(TREE, `${dir}/x`), 'and so does one after the directory is gone'); + }); + } +}); + +describe("an editor's save by rename, with the watcher running", () => { + test('reaches a wake as a change, not a deletion, when the new copy is written at once, and the new file goes on being watched', async (t) => { + const env = new Env(t); + env.writeDisk('gate.json', 'v1'); + // A short debounce, so a pass can run soon after the rename. + const m = await env.open({ watch: 'always', watchDebounceMs: 50 }); + const mount = (m as any).mounts.get('work'); + const blob = () => env.store.treeGet(TREE, 'gate.json')?.blobHash; + await until(() => !!mount.watcherReadyAt && blob() !== undefined, 'the watcher is ready and the catch-up scan took gate.json'); + const v1 = blob(); + env.events.length = 0; + + // vim's save with a backup: the old file moves aside, the new one is + // written in its place, and the backup is removed. + renameSync(env.disk('gate.json'), env.disk('gate.json~')); + env.writeDisk('gate.json', 'v2'); + unlinkSync(env.disk('gate.json~')); + const named = (type: string, path: string) => env.events.some((e: any) => e.type === type && e.paths?.includes(path)); + await until(() => blob() !== v1 && named('workspace:modified', 'work/gate.json'), 'the save is taken in, as a change'); + // Time for anything that shouldn't come to arrive. + await new Promise((r) => setTimeout(r, 400)); + assert.ok(!named('workspace:deleted', 'work/gate.json'), 'no wake sees a deletion'); + assert.ok(!env.events.some((e: any) => e.paths?.includes('work/gate.json~')), 'nor the backup'); + assert.equal(env.store.treeGet(TREE, 'gate.json~'), null); + + const v2 = blob(); + env.writeDisk('gate.json', 'v3'); + await until(() => blob() !== v2, 'a later edit of the new file is taken in'); + }); +}); + describe('disk counterparts that are not stored', () => { test('a binary counterpart keeps the draft and a disk reference that reports a later change', async (t) => { const env = new Env(t); @@ -1064,6 +1716,33 @@ describe('restarts and faults', () => { assert.equal(await stateOf(m, 'a.txt'), 'synced'); }); + test('an edit of a path the ignore list covers sees the disk version its settled adoption takes in too, since it names the path', async (t) => { + const env = new Env(t); + let m = await pendingAdoption(env); + m = await env.restart(m, { ignore: ['a.txt'] }); + assert.equal((m as any).agreement.get('work', 'a.txt').kind, 'pending'); + env.writeDisk('a.txt', 'C'); + const error = await refused(m, 'edit', { path: 'work/a.txt', oldString: 'A', newString: 'B' }); + assert.match(error, /String not found/, 'the entry is disk\'s C by then, so an edit of A has nothing to replace'); + assert.equal(await contentOf(m, 'a.txt'), 'C'); + }); + + test('a delete of a path the ignore list covers settles its pending adoption first too, since it names the path, and deletes what the workspace then holds', async (t) => { + // Since #456 an edit makes a pass of its own; write, delete and + // writeBinary settle through settleBeforeMutation's. + const env = new Env(t); + let m = await pendingAdoption(env); + m = await env.restart(m, { ignore: ['a.txt'] }); + env.writeDisk('a.txt', 'C'); + await call(m, 'delete', { path: 'work/a.txt' }); + // The deletion is of disk's C, which the settled adoption took in, so a + // materialize of the path that applies deletions carries it to disk. + // Settled at the commit instead, from the adoption's B, it would leave C + // as a copy disk changed since. + await call(m, 'materialize', { path: 'work/a.txt', applyDeletions: true }); + assert.equal(existsSync(env.disk('a.txt')), false, "disk's C, the copy the deletion took in, is unlinked"); + }); + test('a delete settles the pending adoption first, and deletes what the workspace then holds', async (t) => { const env = new Env(t); const m = await pendingAdoption(env); @@ -2849,7 +3528,7 @@ describe('a mount larger than its walk cap (agent-framework #272)', () => { test('a model of the walk: on random trees that change between passes, each pass takes the cyclic window after its place', async (t) => { const env = new Env(t); const rootReal = realpathSync(env.dir); - const view = { root: env.dir, followSymlinks: false, maxFileSize: 1 << 20, ignore: [] } as unknown as MountView; + const view = { root: env.dir, followSymlinks: false, maxFileSize: 1 << 20, ignore: [], others: [] } as unknown as MountView; // A seeded generator, so a failure replays. let seed = 272; const rand = (n: number): number => { @@ -2981,7 +3660,7 @@ describe('the mount boundary', () => { mkdirSync(env.disk('sub'), { recursive: true }); env.writeDisk('sub/a.txt', 'A'); const st = statSync(env.dir, { bigint: true }); - const view = { root: env.dir, rootIdentity: { dev: String(st.dev), ino: String(st.ino) }, followSymlinks: false, maxFileSize: 1e6, ignore: [] } as unknown as MountView; + const view = { root: env.dir, rootIdentity: { dev: String(st.dev), ino: String(st.ino) }, followSymlinks: false, maxFileSize: 1e6, ignore: [], others: [] } as unknown as MountView; const away = `${env.dir}-away`; // The code checks the root by its resolved path (on macOS the temp // directory is under /var, a symlink to /private/var). diff --git a/test/workspace-watcher.test.ts b/test/workspace-watcher.test.ts index 45a913598..66cedb521 100644 --- a/test/workspace-watcher.test.ts +++ b/test/workspace-watcher.test.ts @@ -44,11 +44,12 @@ function makeConfig(): MountConfig { function collect( lifecycle?: WatcherLifecycle, config: MountConfig = makeConfig(), + others: readonly string[] = [], ): { watcher: MountWatcher; batches: FsChange[][] } { const batches: FsChange[][] = []; const watcher = new MountWatcher(config, (changes) => { batches.push(changes); - }, lifecycle); + }, lifecycle, others); watcher.start(); return { watcher, batches }; } @@ -86,8 +87,9 @@ function collectWithPolling(lifecycle?: WatcherLifecycle): { function collectWithNative( lifecycle?: WatcherLifecycle, config?: MountConfig, + others?: readonly string[], ): { watcher: MountWatcher; batches: FsChange[][] } { - return withNativeBackend(() => collect(lifecycle, config)); + return withNativeBackend(() => collect(lifecycle, config, others)); } async function wait(ms: number) { @@ -411,3 +413,55 @@ describe('MountWatcher lifecycle', () => { } }); }); + +describe('MountWatcher ignores what a walk passes over by design (connectome-host #278)', () => { + // The mount sits under a directory with a dot in its name, as one under + // ~/.local does: chokidar's own filter for editors' files tested the whole + // path, so there it hid every `*.swp` beneath the root. + async function reported( + collectOn: (lifecycle?: WatcherLifecycle, config?: MountConfig, others?: readonly string[]) => { watcher: MountWatcher; batches: FsChange[][] }, + ignore: string[], + files: string[], + expected: string[], + others: readonly string[] = [], + ): Promise { + const root = join(tmp, '.local', 'work'); + mkdirSync(join(root, 'cache'), { recursive: true }); + mkdirSync(join(root, 'drafts~'), { recursive: true }); + for (const other of others) mkdirSync(join(root, other), { recursive: true }); + let ready = false; + const { watcher, batches } = collectOn({ onReady: () => { ready = true; } }, { ...makeConfig(), path: root, ignore }, others); + try { + await waitFor(() => ready, 'expected the watcher to be ready', 10_000); + for (const f of files) writeFileSync(join(root, f), 'x'); + // What must be reported, then time for anything that mustn't be. + const seen = (p: string) => batches.flat().some((c) => c.path === p); + await waitFor(() => expected.every(seen), `expected ${expected.join(', ')} to be reported`, 10_000); + await wait(SETTLE_MS); + } finally { + await watcher.stop(); + } + return [...new Set(batches.flat().map((c) => c.path))].sort(); + } + + const EDITORS = ['.gate.json.swp', '.gate.json.swx', 'gate.json~', '.x.subl3f9.tmp', 'drafts~/x.txt']; + const PLAIN = ['notes.swp', 'report.v2.swp', 'notes.sublime.tmpl', 'backup.txt.bak', 'kept.txt']; + + for (const [backend, collectOn] of [ + ['native', collectWithNative], + ['polling', (l?: WatcherLifecycle, c?: MountConfig, o?: readonly string[]) => withPollingBackend(() => collect(l, c, o))], + ] as const) { + it(`reports a plain name and passes over editors' files, by their own name (${backend})`, async () => { + assert.deepEqual(await reported(collectOn, [], [...EDITORS, ...PLAIN], PLAIN), [...PLAIN].sort()); + }); + + it(`passes over another mount's directory nested in this one, and reports what lies beside it (${backend})`, async () => { + const files = ['inner/mount/x.txt', 'inner/kept.txt', 'kept.txt']; + assert.deepEqual(await reported(collectOn, [], files, ['inner/kept.txt', 'kept.txt'], ['inner/mount']), ['inner/kept.txt', 'kept.txt']); + }); + } + + it("passes over what the mount's ignore list covers, by the path relative to the mount", async () => { + assert.deepEqual(await reported(collectWithNative, ['*.log', 'cache/**'], ['new.log', 'cache/blob.txt', 'kept.txt'], ['kept.txt']), ['kept.txt']); + }); +});