From 4928a63dcbd7ad8a0315c71eaa4a41b01c32a4da Mon Sep 17 00:00:00 2001 From: Beth-2709 Date: Mon, 19 Oct 2026 01:51:45 -0400 Subject: [PATCH 01/13] feat(workspace): a read-only direct mount, its tools reading disk as it is, with none of its files in the store A mount whose config sets `backend: 'direct'` is read straight from disk: `read`, `read_image`, `ls`, `glob` and `grep` give its files as they are when they run, no scan, watcher or pass runs on it, and none of its files go into the store. It's read-only. A mount that names no kind is the store-backed kind (`'chronicle'`) and works as before. Each read tool keeps one path for its path rules, limits, refusals and rendering, and asks where the mount keeps what it reads: - read: bytesOf, the store's copy as before, or bytesOnDisk, disk's bytes through openContainedFile, at most maxFileSize, refusing a binary file and one over the limit as a store-backed read refuses what it never stored; - read_image: the store's copy then disk's, as before, or disk's alone, with one renderer (imageResult); - ls and glob: listingOf, the store-backed pass (listingFromStore, moved from the handlers) or listingFromDisk, diskWalk's walk with each entry observed (observePath), 16 at a time: a file's size, or a note for what the mount doesn't take; - grep: textsFromStore (moved unchanged) or textsFromDisk, each file read with readBytes (observe.ts: openContained, at most the limit, no hash), 16 at a time in path order (inOrder), skipped with why where it's binary, over the limit or not read; results say version 'disk'. A direct mount's walks are named, as a path sync's are, so a path a tool names is walked whatever the ignore list covers there, and walkStart stops only before another mount's directory. Its ignore list defaults to node_modules, dist, .git, build, coverage, target, __pycache__ and .venv. Its view has no root identity, and readRoot checks none, so it reads beneath whatever directory is at its root, checking no evidence. The constructor refuses a backend other than the two, and a direct mount that isn't read-only or sets watch other than 'never', watchDebounceMs, watchRootPollMs, a wakeOnChange that would wake, or autoMaterialize (directRefusals). initStore registers no states for a direct mount; where a store-backed run left them, it reads what the store holds there that disk lacks (heldChanges, which status now counts with too), and a mount holding any runs store-backed that session, unwatched, which ensureRunning says as an ops alert (workspace-mount-not-direct) and on stderr, naming the paths and the two ways to start it direct. ls with no path and status give a direct mount's backend and ignore list; sync lists it under skipped; readBinary reads its disk (bytesFromDisk, factored out of readBinaryFromDisk unchanged); getTools adds a sentence on the direct kind to ls, grep and sync only where a mount runs direct. Designed by Gladys-2592, on Ada-1017's calls; tests in test/workspace-direct.test.ts. Co-Authored-By: Claude Opus 5.5 --- changelog.d/workspace-direct-mount.added.md | 7 + src/modules/workspace/index.ts | 899 ++++++++++++++------ src/modules/workspace/observe.ts | 47 + src/modules/workspace/types.ts | 42 + test/workspace-direct.test.ts | 690 +++++++++++++++ 5 files changed, 1434 insertions(+), 251 deletions(-) create mode 100644 changelog.d/workspace-direct-mount.added.md create mode 100644 test/workspace-direct.test.ts diff --git a/changelog.d/workspace-direct-mount.added.md b/changelog.d/workspace-direct-mount.added.md new file mode 100644 index 000000000..71e4a4387 --- /dev/null +++ b/changelog.d/workspace-direct-mount.added.md @@ -0,0 +1,7 @@ +- **A direct workspace mount, for a checkout an agent reads:** a mount whose config sets `backend: 'direct'` (the default is `'chronicle'`, the store-backed kind every mount has been) is read straight from disk. `read`, `read_image`, `ls`, `glob` and `grep` give its files as they are when they run, with no scan, watcher or pass, so no call waits behind one, and none of its files go into the store. It's read-only: the workspace refuses a direct mount with any `mode` but `'read-only'`, and one that sets `watch` other than `'never'`, `watchDebounceMs`, `watchRootPollMs`, a `wakeOnChange` that would wake, or `autoMaterialize: true`, since none of them would ever act there. Its paths, links, other mounts' directories, size limit and walk cap hold as on a store-backed mount. Where it differs: + - **What its listings say:** each file with its `size` and no state, or, for a link the mount doesn't follow or another entry it doesn't take, with why as `note`; `grep`'s results say `version: 'disk'`, and it lists under `skipped` a binary file, one over the size limit and one the mount doesn't take, each with why. `read` refuses a binary file and one over the size limit, as a store-backed mount, which never stores them, answers too. + - **Its ignore list:** with none in its config, it takes `node_modules`, `dist`, `.git`, `build`, `coverage`, `target`, `__pycache__` and `.venv`. Walks of the whole mount pass over what the list covers, and a path a listing, glob or grep names, or its pattern's directory, is walked all the same, as a sync naming a path walks it on a store-backed mount. `ls` without a path, and `status`, give a direct mount's `backend` and the list it goes by. + - **Its root:** it reads beneath whatever directory is at its path now, with no check against the one a store-backed run of it last agreed with. + - **`sync`:** passes over it, listing it under `skipped`. `materialize` refuses it, as it refuses any read-only mount. + - **A mount the store holds changes on that disk lacks** (a draft, a workspace deletion, a conflict, on the branch selected at start) runs store-backed that session, so they stay in view, and the workspace says so with an ops alert (`workspace-mount-not-direct`) naming them and the ways to start it direct: finish them, or give them up with a sync of each path. A mount that ran direct and runs store-backed again takes in what disk changed meanwhile. + - **Tool descriptions:** `ls`, `grep` and `sync` gain a sentence about the direct kind only on a host with a mount that runs direct; elsewhere they're unchanged. diff --git a/src/modules/workspace/index.ts b/src/modules/workspace/index.ts index 825d67f59..74f831d21 100644 --- a/src/modules/workspace/index.ts +++ b/src/modules/workspace/index.ts @@ -40,7 +40,7 @@ import { MountWatcher, type FsChange } from './watcher.js'; import { hashContent, DEFAULT_MAX_FILE_SIZE } from './sync.js'; import { DiskAgreement, sameRootIdentity } from './disk-agreement.js'; import { BranchIntents } from './branch-intent.js'; -import { IGNORED, NONBLOCK, OTHER_MOUNT, ROOT_REPLACED, ROOT_UNAVAILABLE, checkDir, checkRoot, diskSpelling, filesystemTrustsCtime, inOtherMount, leadsInto, linkLeads, linkOnTheWay, linkRefusal, linkWords, locateRoot, nestedAtReal, nestedHolder, passedOverAt, readPrefix, type CheckedDir, type LinkFound, type MountView, type NestedMount } from './observe.js'; +import { IGNORED, NONBLOCK, OTHER_MOUNT, ROOT_REPLACED, ROOT_UNAVAILABLE, checkDir, checkRoot, diskSpelling, filesystemTrustsCtime, inOtherMount, leadsInto, linkLeads, linkOnTheWay, linkRefusal, linkWords, locateRoot, looksBinary, nestedAtReal, nestedHolder, observePath, passedOverAt, readBytes, readPrefix, sniffImageMime, walkScope, type CheckedDir, type LinkFound, type MountView, type NestedMount } from './observe.js'; import { TOOL_BATCH_HOOK_TIMEOUT_MS, ensureStateRegistered } from '../../module-registry.js'; import { type MountRuntime, @@ -73,6 +73,18 @@ const CATCH_UP_BOUND_MS = 30_000; const NOT_RUNNING = 'The workspace has stopped: it takes no more work until it starts again'; /** The workspace's own deadline must end before the framework's hook bound. */ const AGENT_ACTION_SCAN_DEADLINE_MAX_MS = TOOL_BATCH_HOOK_TIMEOUT_MS - 1_000; +/** A direct mount's ignore list when its config names none (MountConfig.ignore). */ +const DIRECT_MOUNT_IGNORE: readonly string[] = ['node_modules', 'dist', '.git', 'build', 'coverage', 'target', '__pycache__', '.venv']; +/** + * How many files a direct mount's grep reads, and its listings look at, at + * once. Over a layout of seven checkouts, 5,929 files, a direct grep reading + * 16 at a time on Node's default threadpool had a median time below the + * store-backed grep's over the same files at every load tried, and one + * reading them in turn took 2 to 2.4 times as long. + */ +const DIRECT_READS_AT_ONCE = 16; +/** Why sync passes over a direct mount, as its result's `skipped` says. */ +const DIRECT_SYNC_SKIPPED = "a direct mount: its tools read disk as it is, so there's nothing to take in"; export type { WorkspaceConfig, @@ -835,17 +847,29 @@ export class WorkspaceModule implements Module { * mounts, so restoration must be second-callback-safe like watcher * startup already is (issue #72). Cleared after application. */ private savedState: WorkspaceModuleState | null = null; + /** + * The mounts configured direct that run as 'chronicle' this session, with + * the changes the store held there that disk lacked (initStore), for + * ensureRunning to say once both halves of the lifecycle have run. + */ + private notDirect = new Map>(); constructor(config: WorkspaceConfig) { for (const mount of config.mounts) { const cap: unknown = mount.walkCap; if (cap !== undefined && !(Number.isSafeInteger(cap) && (cap as number) > 0)) { - // Numbers as JavaScript writes them (JSON would say null for NaN), anything else as JSON. - const shown = typeof cap === 'number' ? String(cap) : typeof cap === 'bigint' ? `${cap}n` : JSON.stringify(cap) ?? String(cap); throw new Error( - `[workspace] Mount "${mount.name}": walkCap must be a positive integer no larger than ${Number.MAX_SAFE_INTEGER}, not ${shown}`, + `[workspace] Mount "${mount.name}": walkCap must be a positive integer no larger than ${Number.MAX_SAFE_INTEGER}, not ${shownValue(cap)}`, ); } + const backend: unknown = mount.backend; + if (backend !== undefined && backend !== 'chronicle' && backend !== 'direct') { + throw new Error(`[workspace] Mount "${mount.name}": backend must be 'chronicle' or 'direct', not ${shownValue(backend)}`); + } + const refused = backend === 'direct' ? directRefusals(mount) : []; + if (refused.length > 0) { + throw new Error(`[workspace] Mount "${mount.name}" is direct, which reads disk as it is with no store, watcher or push: ${refused.join('; ')}`); + } } // Detect overlapping mount paths: a mount nested inside another is that // mount's directory, which the outer mount passes over in every pass, @@ -888,23 +912,31 @@ export class WorkspaceModule implements Module { // before the call, knowing no mount, and a later initStore starts clean. const mounts = new Map(); const intents = new Map(); + // A direct mount's files never go into the store, so its states aren't + // made. One that ran as 'chronicle' before has them, and they're read + // below for what the store holds there that disk lacks. + const made = this.config.mounts.some((m) => m.backend === 'direct') ? new Set(store.listStates().map((s) => s.id)) : null; // Register tree states for each mount: the workspace tree, and the // branch-local intent that qualifies it. for (const mount of this.config.mounts) { const treeStateId = `workspace/${mount.name}/tree`; const intentTreeStateId = `workspace/${mount.name}/intent`; - for (const id of [treeStateId, intentTreeStateId]) { - ensureStateRegistered(store, { - id, - strategy: 'tree', - deltaSnapshotEvery: this.config.deltaSnapshotEvery ?? 50, - fullSnapshotEvery: this.config.fullSnapshotEvery ?? 10, - }); + const direct = mount.backend === 'direct'; + if (!direct) { + for (const id of [treeStateId, intentTreeStateId]) { + ensureStateRegistered(store, { + id, + strategy: 'tree', + deltaSnapshotEvery: this.config.deltaSnapshotEvery ?? 50, + fullSnapshotEvery: this.config.fullSnapshotEvery ?? 10, + }); + } } mounts.set(mount.name, { config: mount, + direct, treeStateId, intentTreeStateId, lastMaterializedSeq: 0, @@ -914,7 +946,9 @@ export class WorkspaceModule implements Module { watcherReadyAt: null, watcherError: null, }); - intents.set(mount.name, new BranchIntents(store, intentTreeStateId)); + if (!direct || (made!.has(treeStateId) && made!.has(intentTreeStateId))) { + intents.set(mount.name, new BranchIntents(store, intentTreeStateId)); + } } this.store = store; @@ -926,6 +960,24 @@ export class WorkspaceModule implements Module { this.agreement = new DiskAgreement(store, { modesObserved: () => this.modesObserved }); this.agreement.open(this.config.mounts.map((m) => ({ name: m.name, root: resolve(m.path) }))); + // A mount configured direct runs as 'chronicle' this session while the + // store holds changes there that disk lacks, so they stay in view + // (MountConfig.backend); ensureRunning says so. A path held more than + // one way is named for the last way set here. + this.notDirect = new Map(); + for (const [name, state] of mounts) { + if (!state.direct || !this.intents.has(name)) continue; + const held = this.heldChanges(state); + const what = new Map(); + for (const path of held.owed) what.set(path, 'changed in the workspace'); + for (const path of held.unreadable) what.set(path, "its workspace intent can't be read"); + for (const path of held.deletions) what.set(path, 'deleted in the workspace'); + for (const path of held.conflicts) what.set(path, 'in conflict'); + if (what.size === 0) continue; + state.direct = false; + this.notDirect.set(name, [...what].sort(([a], [b]) => a.localeCompare(b)).map(([path, label]) => ({ path, what: label }))); + } + this.applySavedState(); this.ensureRunning(); } @@ -958,7 +1010,7 @@ export class WorkspaceModule implements Module { * A mount as observation reads it, with the root identity the workspace * recorded for it, if any: what a pass reads beneath (runtime), with * rootIdentity's, and what a disk read checks its root against (readRoot), - * with rootIdentityForRead's. + * with rootIdentityForRead's. A direct mount's has none (diskView). */ private viewOf(mount: MountState, rootIdentity: MountView['rootIdentity']): MountView { return { @@ -967,12 +1019,28 @@ export class WorkspaceModule implements Module { rootIdentity, followSymlinks: mount.config.followSymlinks === true, maxFileSize: mount.config.maxFileSize ?? DEFAULT_MAX_FILE_SIZE, - ignore: mount.config.ignore ?? [], + ignore: this.ignoreOf(mount), others: this.othersOf(mount.config.name), nested: this.nestedOf(mount.config.name), }; } + /** A mount's ignore list as it runs: its config's, or, for a direct mount that names none, the direct kind's default (MountConfig.ignore). */ + private ignoreOf(mount: MountState): string[] { + return mount.config.ignore ?? (mount.direct ? [...DIRECT_MOUNT_IGNORE] : []); + } + + /** + * A direct mount as its tools read it: its view, with no root identity, + * since it promises what's at its root now and keeps no evidence to check + * that against (MountConfig.backend), and its root as realpath resolves it, + * or as given where that fails, as a pass's runtime takes it. + */ + private async diskView(mount: MountState): Promise<{ view: MountView; rootReal: string }> { + const root = resolve(mount.config.path); + return { view: this.viewOf(mount, undefined), rootReal: await realpath(root).catch(() => root) }; + } + /** The view reconciliation needs of a mount, refreshed per pass. */ private async runtime(mount: MountState): Promise { const root = resolve(mount.config.path); @@ -992,7 +1060,7 @@ export class WorkspaceModule implements Module { intents: this.intents.get(mount.config.name)!, readOnly: mount.config.mode === 'read-only', walkCap: mount.config.walkCap ?? WALK_CAP, - catchUpScan: (mount.config.watch ?? 'always') === 'always', + catchUpScan: watchOf(mount.config) === 'always', modesObserved: this.modesObserved, }; } @@ -1006,15 +1074,18 @@ export class WorkspaceModule implements Module { * spelling beside their own. So the start stops before a segment spelled * any other way, not there, or not a directory, and before a link, which * a walk of the whole mount doesn't go through either. It lists only - * directories it has checked lie inside the mount, and none the mount - * passes over by design, which the walk then answers for. + * directories it has checked lie inside the mount, and, on a 'chronicle' + * mount, none it passes over by design, which the walk then answers for. + * A direct mount walks a path its tools name whatever its ignore list or + * an editor's file name covers there (MountConfig.ignore), so there it + * lists such directories too, and only another mount's it doesn't. */ private async walkStart(mount: MountState, base: string, beneath: string): Promise { if (beneath === '') return base; - const { view, rootReal } = await this.runtime(mount); + const { view, rootReal } = mount.direct ? await this.diskView(mount) : await this.runtime(mount); let start = base; for (const segment of beneath.split('/')) { - if (start !== '' && passedOverAt(start, view.ignore, view.others) !== undefined) break; + if (start !== '' && (mount.direct ? inOtherMount(start, view.others) : passedOverAt(start, view.ignore, view.others) !== undefined)) break; const here = await checkDir(view, rootReal, start); if (here.kind !== 'checked') break; const entry = (await readdir(here.dir.real, { withFileTypes: true }).catch(() => [])).find((e) => e.name === segment); @@ -1205,12 +1276,12 @@ export class WorkspaceModule implements Module { */ private ensureRunning(): void { if (!this.ctx || this.mounts.size === 0) return; + for (const [name, held] of this.notDirect) this.sayNotDirect(name, held); for (const [name, mount] of this.mounts) { if (this.watchers.has(name)) continue; - const watchMode = mount.config.watch ?? 'always'; - if (watchMode === 'always') { + if (watchOf(mount.config) === 'always') { // Chokidar is started with ignoreInitial:true, so changes made while // the session was down would be invisible. A catch-up pass takes them // in under the three-way rule: only real differences produce tree @@ -1294,6 +1365,24 @@ export class WorkspaceModule implements Module { } } + /** + * Say, for whoever runs the host, that a mount configured direct runs as + * 'chronicle' this session (initStore), which changes the store holds there + * that disk lacks are why, and the two ways to start it direct: as an ops + * alert (the framework's failures log, its `ops:alert` trace and, where + * one is set, its webhook), and on stderr. + */ + private sayNotDirect(name: string, held: Array<{ path: string; what: string }>): void { + const shown = held.slice(0, 10).map((h) => `"${name}/${h.path}" (${h.what})`).join(', '); + const more = held.length > 10 ? `, and ${held.length - 10} more` : ''; + const message = `Mount "${name}" runs as 'chronicle' this session, though its config makes it direct: ` + + `the workspace holds changes there that disk lacks, which a direct mount wouldn't show: ${shown}${more}. ` + + 'To start it direct, finish them (with the mount configured \'chronicle\' and read-write, materialize them) ' + + 'or give them up (sync each path, which takes disk\'s copy); the first start that finds none starts it direct.'; + console.warn(`[workspace] ${message}`); + this.ctx?.notifyOps?.('workspace-mount-not-direct', 'framework', message, { mount: name, held: held.length }); + } + private async initialScan(mountName: string): Promise { const store = this.store; const mount = this.mounts.get(mountName); @@ -1375,6 +1464,18 @@ export class WorkspaceModule implements Module { // ========================================================================== getTools(): ToolDefinition[] { + // Where a mount runs direct, three descriptions say what that changes. + // A host with none keeps them as they were, and its residents' caches. + const direct = [...this.mounts.values()].some((m) => m.direct); + const lsDirect = direct + ? ' A direct mount (ls without a path says which, with its ignore list) lists disk\'s files as they are now: each with ' + + 'its size and no state, or a note where the mount doesn\'t take it. A path its ignore list covers is listed when you name it.' + : ''; + const grepDirect = direct + ? ' On a direct mount, grep searches disk\'s files as they are now (version disk), and lists as skipped, with why, ' + + 'those it doesn\'t: binary, over the size limit, or not read, such as a link the mount doesn\'t follow.' + : ''; + const syncDirect = direct ? ' A direct mount is listed under `skipped`: its tools read disk as it is, so there\'s nothing to take in.' : ''; return [ { name: 'read', @@ -1455,7 +1556,7 @@ export class WorkspaceModule implements Module { + 'interrupted) — sync the path to take disk, or materialize that path with force to take the workspace, ' + 'with applyDeletions too where the workspace deleted it; ' + 'disk-missing-provenance-unknown; unverified (disk could not be checked). `incomplete` names regions ' - + 'the listing could not see.', + + 'the listing could not see.' + lsDirect, inputSchema: { type: 'object' as const, properties: { @@ -1482,7 +1583,7 @@ export class WorkspaceModule implements Module { + 'which version matched: the workspace\'s (with its state); the disk version kept with a conflict ' + '(conflicting-disk); or, once disk has changed since the conflict was recorded, that recorded version ' + '(recorded-disk). Disk versions it could not search are listed as skipped: disk-only files (binary or ' - + 'over the size limit), and a conflict\'s disk version that is binary, oversize, or newer than the one recorded.', + + 'over the size limit), and a conflict\'s disk version that is binary, oversize, or newer than the one recorded.' + grepDirect, inputSchema: { type: 'object' as const, properties: { @@ -1548,7 +1649,7 @@ export class WorkspaceModule implements Module { + 'in another mount\'s directory isn\'t synced even when you name it: use the mount that holds it. Naming ' + 'an ignored directory sets aside the patterns that cover it or a directory above it; one that ' + 'covers only something inside it still applies, and what it covers is passed over, as are editors\' ' - + 'files inside it.', + + 'files inside it.' + syncDirect, inputSchema: { type: 'object' as const, properties: { @@ -1855,8 +1956,10 @@ export class WorkspaceModule implements Module { /** * Read a file's raw bytes from a mount (through the Chronicle tree, synced - * first like the `read` tool). Public API for the framework's synthesized - * `read_image` tool and other peer callers that need binary content. + * first like the `read` tool; on a direct mount, from disk as + * readBinaryFromDisk reads it, at most the mount's `maxFileSize`). Public + * API for the framework's synthesized `read_image` tool and other peer + * callers that need binary content. */ async readBinary( mountPrefixedPath: string, @@ -1875,6 +1978,8 @@ export class WorkspaceModule implements Module { } catch (error) { return { error: error instanceof Error ? error.message : String(error) }; } + // A direct mount's bytes are disk's, as read takes them, at most its size limit. + if (mount.direct) return this.bytesFromDisk(mount, relativePath, mountPrefixedPath, mount.config.maxFileSize ?? DEFAULT_MAX_FILE_SIZE); const store = this.getStore(); return this.withMount(mount, async () => { await this.ensureSynced(mount, relativePath); @@ -1920,8 +2025,17 @@ export class WorkspaceModule implements Module { } catch (error) { return { error: error instanceof Error ? error.message : String(error) }; } - const absolutePath = resolve(mount.config.path, relativePath); - const maxBytes = opts.maxBytes; + const read = await this.bytesFromDisk(mount, relativePath, mountPrefixedPath, opts.maxBytes); + return 'error' in read ? read : { data: read.data, absolutePath: resolve(mount.config.path, relativePath) }; + } + + /** A file's bytes as readBinaryFromDisk reads them, once its path is parsed. */ + private async bytesFromDisk( + mount: MountState, + relativePath: string, + mountPrefixedPath: string, + maxBytes: number | undefined, + ): Promise<{ data: Buffer } | { error: string }> { let tooLarge: string | undefined; try { const { handle, fileStat } = await this.openContainedFile(mount, relativePath, mountPrefixedPath, (fileStat) => { @@ -1931,14 +2045,14 @@ export class WorkspaceModule implements Module { } }); try { - if (maxBytes === undefined) return { data: await handle.readFile(), absolutePath }; + if (maxBytes === undefined) return { data: await handle.readFile() }; // No more than maxBytes is loaded: a file grown past it since the // fstat above is refused here, as one already over it was there. Its // size is an fstat's after the read, and at least one more than the // bytes kept, so the refusal's size is always past maxBytes. const { bytes, more } = await readPrefix(handle, maxBytes, fileStat.size); if (more) return { error: `File too large: ${Math.max((await handle.stat()).size, bytes.length + 1)} > ${maxBytes} bytes` }; - return { data: bytes, absolutePath }; + return { data: bytes }; } finally { await handle.close(); } @@ -2232,6 +2346,8 @@ export class WorkspaceModule implements Module { * which answers while the evidence awaits reopening; where even it can't * be read, as when the evidence was never read back whole, the read is * refused too, with an errno only where no directory is at the root's path. + * A direct mount's root is checked against no identity, and its read reads + * no evidence: it promises what's at its root now (diskView). */ private async readRoot(mount: MountState): Promise<{ real: string; dir: CheckedDir }> { const name = mount.config.name; @@ -2243,13 +2359,15 @@ export class WorkspaceModule implements Module { throw new WorkspaceReadError('mount_unavailable', `Mount unavailable: ${name}`, 'realpath_root', errnoOf(err)); } let rootIdentity: MountView['rootIdentity']; - try { - rootIdentity = this.agreementOrThrow().rootIdentityForRead(name); - } catch (err) { - // The evidence can't be read back, so neither can the check be made. - // Where no directory is at the root's path, its errno still says so. - throw new WorkspaceReadError('mount_unavailable', - `Mount unavailable: ${name}: ${err instanceof Error ? err.message : String(err)}`, 'root', await rootAbsence(root)); + if (!mount.direct) { + try { + rootIdentity = this.agreementOrThrow().rootIdentityForRead(name); + } catch (err) { + // The evidence can't be read back, so neither can the check be made. + // Where no directory is at the root's path, its errno still says so. + throw new WorkspaceReadError('mount_unavailable', + `Mount unavailable: ${name}: ${err instanceof Error ? err.message : String(err)}`, 'root', await rootAbsence(root)); + } } const checked = await checkRoot(this.viewOf(mount, rootIdentity), real); if ('dir' in checked) return { real, dir: checked.dir }; @@ -2492,7 +2610,9 @@ export class WorkspaceModule implements Module { * - Once the workspace has recorded the mount root's identity, with the * first evidence it gathers there, a root that is another directory is * an unavailable mount: a replaced directory, a link swapped in for it, - * or an unmounted drive's mountpoint. In any mount, recorded or not, a + * or an unmounted drive's mountpoint. A direct mount's reads check no + * recorded identity: they read beneath whatever directory is at its + * root (MountConfig.backend). In any mount, recorded or not, a * root that stops being the directory the read checked until the file * has been opened and checked is unavailable too: with ENOENT or * ENOTDIR where no directory is there, as before the read, and with no @@ -2810,6 +2930,60 @@ export class WorkspaceModule implements Module { } } + /** + * A file's bytes for read, from where the mount keeps what it reads: the + * workspace's copy, taken in from disk first where it holds none + * (ensureSynced); or, on a direct mount, disk's file as it is now + * (bytesOnDisk). An error says why there are none to read. + */ + private async bytesOf(mount: MountState, relativePath: string, path: string): Promise<{ bytes: Buffer } | { error: string }> { + if (mount.direct) return this.bytesOnDisk(mount, relativePath, path); + const store = this.getStore(); + return this.withMount(mount, async () => { + const report = await this.ensureSynced(mount, relativePath); + const entry = store.treeGet(mount.treeStateId, relativePath); + if (!entry) return { error: this.notInWorkspace(path, report) }; + const blob = store.getBlob(entry.blobHash); + if (!blob) return { error: `Blob not found for: ${path}` }; + return { bytes: blob }; + }); + } + + /** + * A direct mount's file for read: its bytes on disk as they are now, + * through the mount's boundary as readFileFromDisk reaches them + * (openContainedFile), whose refusals are the answer. What a 'chronicle' + * mount never stores, it doesn't show either: a file over the mount's + * size limit, and a binary one, with a NUL in its first 8 KB, which + * read_image reads where it's an image. + */ + private async bytesOnDisk(mount: MountState, relativePath: string, path: string): Promise<{ bytes: Buffer } | { error: string }> { + const maxSize = mount.config.maxFileSize ?? DEFAULT_MAX_FILE_SIZE; + const over = `${path} is over the size limit (${maxSize} bytes), so read doesn't show it.`; + let opened: Awaited>; + try { + opened = await this.openContainedFile(mount, relativePath, path, (fileStat) => { + if (fileStat.size > maxSize) throw new Error(over); + }); + } catch (err) { + return { error: err instanceof Error ? err.message : String(err) }; + } + try { + // No more than the limit is loaded, as for a file over it at its fstat. + const { bytes, more } = await readPrefix(opened.handle, maxSize, opened.fileStat.size); + if (more) return { error: over }; + if (looksBinary(bytes)) { + return { error: `${path} is binary, so read doesn't show it as text${sniffImageMime(bytes) ? '; read it with read_image.' : '.'}` }; + } + return { bytes }; + } catch (err) { + const errno = errnoOf(err); + return { error: `Unable to read file: ${path}${errno ? ` (${errno})` : ''}` }; + } finally { + await opened.handle.close(); + } + } + // ========================================================================== // Tool Handlers // ========================================================================== @@ -2827,17 +3001,9 @@ export class WorkspaceModule implements Module { if (content === undefined) { const { mount, relativePath, path } = await this.parseFilePath(input.path); input = { ...input, path }; - const store = this.getStore(); - const found = await this.withMount(mount, async (): Promise<{ text: string } | { error: string }> => { - const report = await this.ensureSynced(mount, relativePath); - const entry = store.treeGet(mount.treeStateId, relativePath); - if (!entry) return { error: this.notInWorkspace(input.path, report) }; - const blob = store.getBlob(entry.blobHash); - if (!blob) return { error: `Blob not found for: ${input.path}` }; - return { text: blob.toString('utf-8') }; - }); + const found = await this.bytesOf(mount, relativePath, path); if ('error' in found) return { success: false, error: found.error, isError: true }; - content = found.text; + content = found.bytes.toString('utf-8'); } if (characterPaging) { const start = Math.min(offsetChars, content.length); @@ -2931,48 +3097,23 @@ export class WorkspaceModule implements Module { const maxSize = mount.config.maxFileSize ?? DEFAULT_MAX_FILE_SIZE; let treeError: WorkspaceImageReadError | null = null; - try { - const image = this.tryReadImageFromTree(mount, relativePath, input.path, maxSize); - if (image) { - return { - success: true, - data: [ - { - type: 'text', - text: `Path: ${input.path}\nMIME: ${image.mimeType}\nBytes: ${image.bytes.byteLength}`, - }, - { - type: 'image', - data: image.bytes.toString('base64'), - mimeType: image.mimeType, - }, - ], - }; - } - } catch (err) { - if (err instanceof WorkspaceImageReadError) { - treeError = err; - } else { - throw err; + // The workspace's copy first, where it holds one, then disk's. A direct + // mount holds none, so its image is disk's. + if (!mount.direct) { + try { + const image = this.tryReadImageFromTree(mount, relativePath, input.path, maxSize); + if (image) return imageResult(input.path, image); + } catch (err) { + if (err instanceof WorkspaceImageReadError) { + treeError = err; + } else { + throw err; + } } } try { - const image = await this.readImageFromFilesystem(mount, relativePath, input.path, maxSize); - return { - success: true, - data: [ - { - type: 'text', - text: `Path: ${input.path}\nMIME: ${image.mimeType}\nBytes: ${image.bytes.byteLength}`, - }, - { - type: 'image', - data: image.bytes.toString('base64'), - mimeType: image.mimeType, - }, - ], - }; + return imageResult(input.path, await this.readImageFromFilesystem(mount, relativePath, input.path, maxSize)); } catch (err) { if (err instanceof WorkspaceImageReadError) { if (err.code === 'not_found' && treeError) { @@ -3215,14 +3356,14 @@ export class WorkspaceModule implements Module { } private async handleLs(input: LsInput): Promise { - const store = this.getStore(); - if (!input.path) { - // List all mounts + // List all mounts, with a direct mount's kind and the ignore list it + // walks by, whose default no config shows. const mounts = [...this.mounts.entries()].map(([name, m]) => ({ name, path: m.config.path, mode: m.config.mode, + ...(m.direct ? { backend: 'direct', ignore: this.ignoreOf(m) } : {}), })); return { success: true, data: { mounts } }; } @@ -3232,46 +3373,120 @@ export class WorkspaceModule implements Module { const recursive = input.recursive === true; // Every listing checks the paths it shows against disk first, under the - // three-way rule, whatever the mount's watch mode. - return this.withMount(mount, async (): Promise => { - const pass = await this.passUnlocked(mount, { kind: 'dir', dir: relativePath, recursive }); - const reports = [...pass.reports.values()].sort((a, b) => a.path.localeCompare(b.path)); - const incomplete = pass.incomplete.length > 0 ? { incomplete: pass.incomplete } : {}; - - if (recursive) { - return { - success: true, - data: { - path: input.path, - entries: reports.map((r) => ({ path: r.path, ...stateView(r) })), - count: reports.length, - ...incomplete, - }, - }; - } - - // Non-recursive: the scope's files, and the directories beneath it that - // disk or the workspace has. - const prefix = relativePath ? relativePath + '/' : ''; - const dirNames = new Set(pass.dirs.map((d) => d.slice(prefix.length))); - for (const entry of store.treeList(mount.treeStateId, prefix || undefined)) { - const slash = entry.path.indexOf('/', prefix.length); - if (slash >= 0) dirNames.add(entry.path.slice(prefix.length, slash)); - } - const children: Array> = [ - ...[...dirNames].sort().map((name) => ({ name, type: 'directory' })), - ...reports.map((r) => ({ name: r.path.slice(prefix.length), type: 'file', ...stateView(r) })), - ]; + // three-way rule, whatever the mount's watch mode; a direct mount's + // shows disk's. + const listing = await this.listingOf(mount, relativePath, recursive); + const incomplete = listing.incomplete.length > 0 ? { incomplete: listing.incomplete } : {}; + if (recursive) { return { success: true, data: { path: input.path, - entries: children, - count: children.length, + entries: listing.files, + count: listing.files.length, ...incomplete, }, }; + } + + // Non-recursive: the scope's files, and the directories beneath it. + const prefix = relativePath ? relativePath + '/' : ''; + const children: Array> = [ + ...listing.dirs.map((d) => d.slice(prefix.length)).sort().map((name) => ({ name, type: 'directory' })), + ...listing.files.map(({ path: file, ...view }) => ({ name: file.slice(prefix.length), type: 'file', ...view })), + ]; + return { + success: true, + data: { + path: input.path, + entries: children, + count: children.length, + ...incomplete, + }, + }; + } + + /** + * What a listing shows of a directory scope, from where the mount keeps + * what it reads: the store-backed kind's pass (listingFromStore), or, on a + * direct mount, disk (listingFromDisk). `keep` chooses the files shown. + */ + private listingOf(mount: MountState, dir: string, recursive: boolean, keep?: (path: string) => boolean): Promise { + return mount.direct ? this.listingFromDisk(mount, dir, recursive, keep) : this.listingFromStore(mount, dir, recursive, keep); + } + + /** + * A 'chronicle' mount's listing: a pass over the scope under the three-way + * rule, each path it decided with its state, and the directories beneath + * the scope that disk or the workspace has. + */ + private listingFromStore(mount: MountState, dir: string, recursive: boolean, keep?: (path: string) => boolean): Promise { + const store = this.getStore(); + return this.withMount(mount, async (): Promise => { + const pass = await this.passUnlocked(mount, { kind: 'dir', dir, recursive }); + const files = [...pass.reports.values()] + .sort((a, b) => a.path.localeCompare(b.path)) + .filter((report) => keep === undefined || keep(report.path)) + .map((report) => ({ path: report.path, ...stateView(report) })); + if (recursive) return { files, dirs: [], incomplete: pass.incomplete }; + // Read with the pass's reports, on the branch it decided on, before + // anything else takes the mount's turn. + const dirs = new Set(pass.dirs); + const prefix = dir ? dir + '/' : ''; + for (const entry of store.treeList(mount.treeStateId, prefix || undefined)) { + const slash = entry.path.indexOf('/', prefix.length); + if (slash >= 0) dirs.add(entry.path.slice(0, slash)); + } + return { files, dirs: [...dirs], incomplete: pass.incomplete }; + }); + } + + /** + * A direct mount's listing: its walk of the scope (diskWalk), with each + * entry it took as disk has it now, observed as a pass observes it: a + * file's size, or why the mount doesn't take it (a link it doesn't follow, + * one that leaves the mount, what isn't a regular file). A file gone since + * the walk listed it is no longer there to show. + */ + private async listingFromDisk(mount: MountState, dir: string, recursive: boolean, keep?: (path: string) => boolean): Promise { + const { view, rootReal, taken, dirs, incomplete } = await this.diskWalk(mount, dir, recursive); + const files: Listing['files'] = []; + await inOrder(taken.filter((path) => keep === undefined || keep(path)), DIRECT_READS_AT_ONCE, (path) => observePath(view, rootReal, path), (seen, path) => { + if (seen.kind === 'file') files.push({ path, size: seen.size }); + else if (seen.kind === 'unobserved') files.push({ path, note: seen.reason }); }); + return { files, dirs, incomplete }; + } + + /** + * A direct mount's walk of a scope, from its view (diskView): the entries + * it took that aren't directories (files, links and the like), in path + * order; the directories directly beneath a scope it doesn't descend; and + * the regions it couldn't see that a pass reports beside no tracked file: + * what it couldn't list, what the cap left, and a scope in another mount's + * directory, said as a listing says it. A path the walk names is walked + * whatever the mount's ignore list or an editor's file name covers there + * (MountConfig.ignore), as a path sync's walk is. + */ + private async diskWalk(mount: MountState, dir: string, recursive: boolean): Promise<{ + view: MountView; rootReal: string; taken: string[]; dirs: string[]; incomplete: Array<{ path: string; reason: string }>; + }> { + const { view, rootReal } = await this.diskView(mount); + const walk = await walkScope(view, rootReal, dir, recursive, mount.config.walkCap ?? WALK_CAP, { named: true }); + const incomplete = walk.incomplete + .filter((region) => region.kind === 'error' || region.kind === 'cap' || (region.kind === 'ignored' && region.path === dir && dir !== '')) + .map((region) => ({ + path: region.path, + reason: region.reason === OTHER_MOUNT ? this.otherMountNote(mount.config.name, region.path, 'list') : region.reason, + })); + return { + view, + rootReal, + taken: [...walk.files, ...walk.othersTaken].sort((a, b) => a.localeCompare(b)), + // A walk that doesn't descend sees the scope's own subdirectories alone. + dirs: recursive ? [] : [...walk.dirs], + incomplete, + }; } private async handleGlob(input: GlobInput): Promise { @@ -3290,15 +3505,10 @@ export class WorkspaceModule implements Module { const patternDir = literalDirOf(input.pattern); for (const { mount, relativePath } of mountsToSearch) { const start = await this.walkStart(mount, relativePath, patternDir); - const pass = await this.withMount(mount, () => - this.passUnlocked(mount, { kind: 'dir', dir: start, recursive: true })); - for (const report of [...pass.reports.values()].sort((a, b) => a.path.localeCompare(b.path))) { - const testPath = relativePath ? report.path.slice(relativePath.length + 1) : report.path; - if (regex.test(testPath)) { - matches.push({ path: `${mount.config.name}/${report.path}`, ...stateView(report) }); - } - } - for (const region of pass.incomplete) { + const matched = (path: string) => regex.test(relativePath ? path.slice(relativePath.length + 1) : path); + const listing = await this.listingOf(mount, start, true, matched); + for (const { path, ...view } of listing.files) matches.push({ path: `${mount.config.name}/${path}`, ...view }); + for (const region of listing.incomplete) { incomplete.push({ path: `${mount.config.name}/${region.path}`, reason: region.reason }); } } @@ -3315,7 +3525,6 @@ export class WorkspaceModule implements Module { } private async handleGrep(input: GrepInput): Promise { - const store = this.getStore(); let regex: RegExp; try { regex = new RegExp(input.pattern); @@ -3337,8 +3546,8 @@ export class WorkspaceModule implements Module { : [...this.mounts.values()].map(m => ({ mount: m, relativePath: '' })); type GrepMatch = { line: number; text: string; context?: string[] }; - const results: Array<{ file: string; state: string; version: 'workspace' | 'conflicting-disk' | 'recorded-disk'; matches: GrepMatch[] }> = []; - const skipped: Array<{ file: string; reason: string; size?: number; mimeType?: string }> = []; + const results: Array<{ file: string; state?: string; version: GrepText['version']; matches: GrepMatch[] }> = []; + const skipped: GrepSkip[] = []; const incomplete: Array<{ path: string; reason: string }> = []; const search = (content: string): GrepMatch[] => { @@ -3371,110 +3580,16 @@ export class WorkspaceModule implements Module { const deeper = deeperOf(relativePath, filterDir); if (deeper === null) continue; const start = await this.walkStart(mount, relativePath, deeper === relativePath ? '' : deeper.slice(relativePath ? relativePath.length + 1 : 0)); - await this.withMount(mount, async () => { - // A `path` naming a single file greps just that file; otherwise it is a - // directory prefix. (A file path once produced the prefix "notes.md/" - // and silently matched nothing.) - const fileScope = relativePath !== '' && ( - store.treeGet(mount.treeStateId, relativePath) !== null || - await lstat(join(mount.config.path, relativePath)).then((s) => !s.isDirectory(), () => false) - ); - // What follows reads the tree and branch intent to go with the pass's - // reports, synchronously. They must be the branch the pass decided - // on: one selected in between would label its content with the other - // branch's states. So a pass whose branch is no longer selected runs - // again, and a branch that keeps changing leaves the scope unsearched. - // - // A copy held at a path that now leads into a nested mount's directory - // through a link is that mount's file too: read refuses it - // (parseFilePath), so grep searches none of it and says why. The pass - // can't check such a path's disk, so only an unverified one can be - // one. Asking awaits disk, so it's done before the branch is - // compared, and a branch selected meanwhile runs the pass again. - let pass: PassResult | null = null; - let refused = new Map(); - for (let attempt = 0; attempt < 3 && pass === null; attempt++) { - const next = await this.passUnlocked(mount, fileScope - ? { kind: 'paths', paths: [relativePath] } - : { kind: 'dir', dir: start, recursive: true }); - const asked = new Map(); - for (const report of next.reports.values()) { - if (report.state !== 'unverified') continue; - const refusal = await this.leadsIntoNested(mount, report.path, `${mount.config.name}/${report.path}`); - if (refusal) asked.set(report.path, refusal.message); - } - if (next.branchId === undefined || next.branchId === store.currentBranch().id) { - pass = next; - refused = asked; - } - } - if (pass === null) { - incomplete.push({ - path: `${mount.config.name}/${relativePath}`, - reason: 'the selected branch kept changing while grep ran; nothing here was searched — grep again', - }); - return; - } - for (const region of pass.incomplete) { - incomplete.push({ path: `${mount.config.name}/${region.path}`, reason: region.reason }); - } - - const intents = this.intents.get(mount.config.name)!; - for (const report of [...pass.reports.values()].sort((a, b) => a.path.localeCompare(b.path))) { - if (fileGlob && !fileGlob.test(report.path)) continue; - const file = `${mount.config.name}/${report.path}`; - if (report.state === 'disk-only') { - skipped.push({ - file, - reason: 'disk-only: binary or over the size limit, never stored', - ...(report.size !== undefined ? { size: report.size } : {}), - ...(report.mimeType ? { mimeType: report.mimeType } : {}), - }); - continue; - } - // A path in another mount's directory is that mount's: grep gives - // none of this mount's copy of it, as read refuses it. Beneath a - // scope it's passed over by design, like the directory. - if (inOtherMount(report.path, this.othersOf(mount.config.name))) continue; - const refusal = refused.get(report.path); - if (refusal !== undefined) { - skipped.push({ file, reason: refusal }); - continue; - } - // The workspace's version, labelled with the path's state. - const entry = store.treeGet(mount.treeStateId, report.path); - const blob = entry ? store.getBlob(entry.blobHash) : null; - if (blob) { - const found = search(blob.toString('utf-8')); - if (found.length > 0) results.push({ file, state: report.state, version: 'workspace', matches: found }); - } else if (entry) { - skipped.push({ file, reason: 'the workspace copy is missing from the store, so it was not searched' }); - } - if (report.state !== 'conflict') continue; - // A conflict's disk side. The version recorded with the conflict is - // disk's only while disk still holds it; once disk has changed - // since, it is labelled as recorded, and a file disk holds now - // (never stored) is listed as not searched. A binary or oversize - // disk version was never stored either. - const disk = intents.get(report.path)?.conflict?.disk; - const changed = report.conflict?.diskChangedSinceRecorded === true; - const diskBlob = disk?.stored ? store.getBlob(disk.stored) : null; - if (diskBlob) { - const found = search(diskBlob.toString('utf-8')); - if (found.length > 0) results.push({ file, state: report.state, version: changed ? 'recorded-disk' : 'conflicting-disk', matches: found }); - } - if (disk?.stored && !diskBlob) { - skipped.push({ file, reason: 'conflict: the disk version recorded with the conflict is missing from the store, so it was not searched' }); - } - if (changed) { - if (report.diskNow === 'file') { - skipped.push({ file, reason: 'conflict: disk changed since the conflict was recorded; grep searches stored text, so the file disk holds now was not searched' }); - } - } else if (disk && !disk.stored) { - skipped.push({ file, reason: 'conflict: the disk version is binary or over the size limit, never stored', size: disk.size }); - } - } - }); + const searched = (text: GrepText): void => { + const matches = search(text.text); + if (matches.length > 0) results.push({ file: text.file, ...(text.state !== undefined ? { state: text.state } : {}), version: text.version, matches }); + }; + const kept = (path: string): boolean => fileGlob === null || fileGlob.test(path); + const found = mount.direct + ? await this.textsFromDisk(mount, relativePath, start, kept, searched) + : await this.textsFromStore(mount, relativePath, start, kept, searched); + skipped.push(...found.skipped); + incomplete.push(...found.incomplete); } return { @@ -3489,6 +3604,168 @@ export class WorkspaceModule implements Module { }; } + /** + * The texts a 'chronicle' mount's grep searches beneath `start`, or in the + * single file `relativePath` names, those `keep` lets through, each handed + * to `each` in path order: a pass over the scope, then each path's + * workspace copy, labelled with its state, and a conflict's disk side. + * What it doesn't search is in `skipped`, and what it couldn't see in + * `incomplete`, each mount-prefixed. + */ + private async textsFromStore( + mount: MountState, + relativePath: string, + start: string, + keep: (path: string) => boolean, + each: (text: GrepText) => void, + ): Promise<{ skipped: GrepSkip[]; incomplete: Array<{ path: string; reason: string }> }> { + const store = this.getStore(); + const skipped: GrepSkip[] = []; + const incomplete: Array<{ path: string; reason: string }> = []; + await this.withMount(mount, async () => { + // A `path` naming a single file greps just that file; otherwise it is a + // directory prefix. (A file path once produced the prefix "notes.md/" + // and silently matched nothing.) + const fileScope = relativePath !== '' && ( + store.treeGet(mount.treeStateId, relativePath) !== null || + await lstat(join(mount.config.path, relativePath)).then((s) => !s.isDirectory(), () => false) + ); + // What follows reads the tree and branch intent to go with the pass's + // reports, synchronously. They must be the branch the pass decided + // on: one selected in between would label its content with the other + // branch's states. So a pass whose branch is no longer selected runs + // again, and a branch that keeps changing leaves the scope unsearched. + // + // A copy held at a path that now leads into a nested mount's directory + // through a link is that mount's file too: read refuses it + // (parseFilePath), so grep searches none of it and says why. The pass + // can't check such a path's disk, so only an unverified one can be + // one. Asking awaits disk, so it's done before the branch is + // compared, and a branch selected meanwhile runs the pass again. + let pass: PassResult | null = null; + let refused = new Map(); + for (let attempt = 0; attempt < 3 && pass === null; attempt++) { + const next = await this.passUnlocked(mount, fileScope + ? { kind: 'paths', paths: [relativePath] } + : { kind: 'dir', dir: start, recursive: true }); + const asked = new Map(); + for (const report of next.reports.values()) { + if (report.state !== 'unverified') continue; + const refusal = await this.leadsIntoNested(mount, report.path, `${mount.config.name}/${report.path}`); + if (refusal) asked.set(report.path, refusal.message); + } + if (next.branchId === undefined || next.branchId === store.currentBranch().id) { + pass = next; + refused = asked; + } + } + if (pass === null) { + incomplete.push({ + path: `${mount.config.name}/${relativePath}`, + reason: 'the selected branch kept changing while grep ran; nothing here was searched — grep again', + }); + return; + } + for (const region of pass.incomplete) { + incomplete.push({ path: `${mount.config.name}/${region.path}`, reason: region.reason }); + } + + const intents = this.intents.get(mount.config.name)!; + for (const report of [...pass.reports.values()].sort((a, b) => a.path.localeCompare(b.path))) { + if (!keep(report.path)) continue; + const file = `${mount.config.name}/${report.path}`; + if (report.state === 'disk-only') { + skipped.push({ + file, + reason: 'disk-only: binary or over the size limit, never stored', + ...(report.size !== undefined ? { size: report.size } : {}), + ...(report.mimeType ? { mimeType: report.mimeType } : {}), + }); + continue; + } + // A path in another mount's directory is that mount's: grep gives + // none of this mount's copy of it, as read refuses it. Beneath a + // scope it's passed over by design, like the directory. + if (inOtherMount(report.path, this.othersOf(mount.config.name))) continue; + const refusal = refused.get(report.path); + if (refusal !== undefined) { + skipped.push({ file, reason: refusal }); + continue; + } + // The workspace's version, labelled with the path's state. + const entry = store.treeGet(mount.treeStateId, report.path); + const blob = entry ? store.getBlob(entry.blobHash) : null; + if (blob) { + each({ file, state: report.state, version: 'workspace', text: blob.toString('utf-8') }); + } else if (entry) { + skipped.push({ file, reason: 'the workspace copy is missing from the store, so it was not searched' }); + } + if (report.state !== 'conflict') continue; + // A conflict's disk side. The version recorded with the conflict is + // disk's only while disk still holds it; once disk has changed + // since, it is labelled as recorded, and a file disk holds now + // (never stored) is listed as not searched. A binary or oversize + // disk version was never stored either. + const disk = intents.get(report.path)?.conflict?.disk; + const changed = report.conflict?.diskChangedSinceRecorded === true; + const diskBlob = disk?.stored ? store.getBlob(disk.stored) : null; + if (diskBlob) { + each({ file, state: report.state, version: changed ? 'recorded-disk' : 'conflicting-disk', text: diskBlob.toString('utf-8') }); + } + if (disk?.stored && !diskBlob) { + skipped.push({ file, reason: 'conflict: the disk version recorded with the conflict is missing from the store, so it was not searched' }); + } + if (changed) { + if (report.diskNow === 'file') { + skipped.push({ file, reason: 'conflict: disk changed since the conflict was recorded; grep searches stored text, so the file disk holds now was not searched' }); + } + } else if (disk && !disk.stored) { + skipped.push({ file, reason: 'conflict: the disk version is binary or over the size limit, never stored', size: disk.size }); + } + } + }); + return { skipped, incomplete }; + } + + /** + * The texts a direct mount's grep searches beneath `start`, or in the + * single file `relativePath` names, those `keep` lets through, each handed + * to `each` in path order: each file its walk took (diskWalk), read from + * disk as it is now, DIRECT_READS_AT_ONCE at a time. A file over the size + * limit, a binary one and one the mount doesn't take are in `skipped`, + * each with why, as a 'chronicle' mount's grep lists what it never stored; + * one gone since the walk listed it isn't there to search. + */ + private async textsFromDisk( + mount: MountState, + relativePath: string, + start: string, + keep: (path: string) => boolean, + each: (text: GrepText) => void, + ): Promise<{ skipped: GrepSkip[]; incomplete: Array<{ path: string; reason: string }> }> { + const name = mount.config.name; + const fileScope = relativePath !== '' && await lstat(join(mount.config.path, relativePath)).then((s) => !s.isDirectory(), () => false); + const { view, rootReal, taken, incomplete } = fileScope + ? { ...(await this.diskView(mount)), taken: [relativePath], incomplete: [] } + : await this.diskWalk(mount, start, true); + const maxSize = mount.config.maxFileSize ?? DEFAULT_MAX_FILE_SIZE; + const skipped: GrepSkip[] = []; + await inOrder(taken.filter(keep), DIRECT_READS_AT_ONCE, (path) => readBytes(view, rootReal, path, maxSize), (read, path) => { + const file = `${name}/${path}`; + if (read.kind === 'absent') return; + if (read.kind === 'unobserved') { + skipped.push({ file, reason: read.reason }); + return; + } + const mimeType = sniffImageMime(read.head); + const what = { size: read.size, ...(mimeType ? { mimeType } : {}) }; + if (read.bytes === null) skipped.push({ file, reason: `over the size limit (${maxSize} bytes)`, ...what }); + else if (looksBinary(read.head)) skipped.push({ file, reason: 'binary: a NUL in its first 8 KB', ...what }); + else each({ file, version: 'disk', text: read.bytes.toString('utf-8') }); + }); + return { skipped, incomplete: incomplete.map((region) => ({ path: `${name}/${region.path}`, reason: region.reason })) }; + } + private async handleStatus(input: StatusInput): Promise { const store = this.getStore(); const status: Record = {}; @@ -3498,25 +3775,22 @@ export class WorkspaceModule implements Module { for (const [name, mount] of this.mounts) { if (input.mount !== undefined && name !== input.mount) continue; + if (mount.direct) { + // A direct mount keeps nothing to count: its kind, and the ignore + // list its walks go by, whose default no config shows. + status[name] = { path: mount.config.path, mode: mount.config.mode, backend: 'direct', ignore: this.ignoreOf(mount) }; + continue; + } const entries = store.treeList(mount.treeStateId); const currentSeq = store.currentSequence(); - const own = this.ownPaths(mount); - const intents = this.intents.get(name)!.list().filter(([p]) => own(p)); - const conflictPaths = intents.filter(([, bi]) => bi.conflict).map(([p]) => p); - // A workspace deletion is pending until a push confirms it on disk: - // left there without applyDeletions, or unlinked but unconfirmed. - const deletionPaths = intents.filter(([, bi]) => bi.tombstone).map(([p]) => p); - // Pending is what disk owes by the evidence: every entry it doesn't - // show on disk (a refused or failed push stays owed), conflicts and - // deletions. An entry from before the evidence isn't counted until a - // listing or a materialize checks it: unchecked isn't unpushed. - const pending = new Set([...conflictPaths, ...deletionPaths, ...this.entriesByEvidence(mount).owed]); + const { conflicts: conflictPaths, deletions: deletionPaths, owed } = this.heldChanges(mount); + const pending = new Set([...conflictPaths, ...deletionPaths, ...owed]); const currentBranch = store.currentBranch(); status[name] = { path: mount.config.path, mode: mount.config.mode, - watch: mount.config.watch ?? 'always', + watch: watchOf(mount.config), fileCount: entries.length, lastMaterializedSeq: mount.lastMaterializedSeq, currentSeq, @@ -3534,6 +3808,29 @@ export class WorkspaceModule implements Module { return { success: true, data: status }; } + /** + * What the store holds of a mount that disk lacks, on the selected branch, + * among the mount's own paths. Status counts the first three as pending: + * recorded conflicts; workspace deletions, pending until a push confirms + * them on disk (left there without applyDeletions, or unlinked but + * unconfirmed); and what disk owes by the evidence (entriesByEvidence's + * `owed`: every entry it doesn't show on disk, a refused or failed push's + * included). An entry from before the evidence isn't among them until a + * listing or a materialize checks it: unchecked isn't unpushed. The + * fourth, intents that can't be read, may hold a deletion or a conflict, + * so a mount configured direct counts them too (initStore). + */ + private heldChanges(mount: MountState): { conflicts: string[]; deletions: string[]; owed: string[]; unreadable: string[] } { + const own = this.ownPaths(mount); + const intents = this.intents.get(mount.config.name)!.list().filter(([p]) => own(p)); + return { + conflicts: intents.filter(([, bi]) => bi.conflict).map(([p]) => p), + deletions: intents.filter(([, bi]) => bi.tombstone).map(([p]) => p), + owed: this.entriesByEvidence(mount).owed, + unreadable: intents.filter(([, bi]) => bi.unreadable !== undefined).map(([p]) => p), + }; + } + /** * What a materialize of `mount` should consider: a named file, or every * tracked path beneath a named directory; otherwise what the evidence says @@ -3819,6 +4116,12 @@ export class WorkspaceModule implements Module { } for (const { name, mount, relativePath } of targets) { + // A direct mount's tools read disk as it is, so a sync has nothing to + // take in there, and says it didn't look rather than that nothing changed. + if (mount.direct) { + allSkipped.push({ mount: name, path: relativePath, reason: DIRECT_SYNC_SKIPPED }); + continue; + } const pass = await this.withMount(mount, async () => { if (input.acceptRoot && await this.acceptRootUnlocked(mount)) rootsAccepted.push(name); if (!relativePath) { @@ -3891,7 +4194,7 @@ export class WorkspaceModule implements Module { // without a path passes over it, so nothing else keeps it current // (agent-framework #276). A path in another mount's directory isn't // synced at all, and the note says so. - const passedOver = passedOverAt(relativePath, mount.config.ignore ?? [], this.othersOf(name)); + const passedOver = passedOverAt(relativePath, this.ignoreOf(mount), this.othersOf(name)); if (passedOver !== undefined) { allIgnored.push({ mount: name, @@ -3980,7 +4283,7 @@ export class WorkspaceModule implements Module { async onToolBatchComplete(_agentName?: string): Promise { // Every agent's workspace is the store's current branch, so the agent // that ran the batch doesn't choose what is scanned. - const mounts = [...this.mounts.values()].filter((m) => m.config.watch === 'on-agent-action'); + const mounts = [...this.mounts.values()].filter((m) => watchOf(m.config) === 'on-agent-action'); if (mounts.length === 0 || !this.store || !this.running()) return; const deadlineMs = this.agentActionScanDeadlineMs(); await Promise.all(mounts.map(async (mount) => { @@ -4129,6 +4432,68 @@ export class WorkspaceModule implements Module { } +/** + * A text grep searches (textsFromStore, textsFromDisk): its mount-prefixed + * file, which copy it is, and on a 'chronicle' mount, the path's state. + */ +interface GrepText { + file: string; + state?: string; + version: 'workspace' | 'conflicting-disk' | 'recorded-disk' | 'disk'; + text: string; +} + +/** A file, or a copy of one, grep doesn't search, with why. */ +type GrepSkip = { file: string; reason: string; size?: number; mimeType?: string }; + +/** What a listing shows of a directory scope (listingOf). */ +interface Listing { + /** Each file it shows, by mount-relative path, in order, with what a listing says of it beyond its name. */ + files: Array<{ path: string } & Record>; + /** For a listing that doesn't descend, the directories directly beneath the scope, mount-relative; none for one that does. */ + dirs: string[]; + /** The regions it couldn't see, mount-relative, each with why. */ + incomplete: Array<{ path: string; reason: string }>; +} + +/** + * Run `work` on each item, at most `width` at once, and hand each result to + * `each` in the items' order, as soon as it and those before it are done. + * A `work` that rejects rejects this when its turn comes; any started after + * it finish unheard. Exported for its tests. + */ +export async function inOrder(items: readonly T[], width: number, work: (item: T) => Promise, each: (result: R, item: T) => void): Promise { + const started: Array> = []; + for (let i = 0; i < items.length; i++) { + while (started.length < items.length && started.length < i + width) { + const running = work(items[started.length]!); + // Handled from the start, so a rejection that comes before its turn + // isn't an unhandled one; awaiting it below still throws it. + running.catch(() => {}); + started.push(running); + } + each(await started[i]!, items[i]!); + } +} + +/** read_image's answer for an image, wherever it was read from. */ +function imageResult(path: string, image: { bytes: Buffer; mimeType: SupportedImageMimeType }): ToolResult { + return { + success: true, + data: [ + { + type: 'text', + text: `Path: ${path}\nMIME: ${image.mimeType}\nBytes: ${image.bytes.byteLength}`, + }, + { + type: 'image', + data: image.bytes.toString('base64'), + mimeType: image.mimeType, + }, + ], + }; +} + /** What a listing shows of a path beyond its name. */ function stateView(report: PathReport): Record { return { @@ -4147,6 +4512,38 @@ function stateView(report: PathReport): Record { /** A lone UTF-16 surrogate, which UTF-8 can't store: in `u` mode, a pair is one code point. */ const LONE_SURROGATE = /\p{Cs}/u; +/** + * A mount's watch mode as it runs. A mount configured direct watches + * nothing, whichever kind it runs as this session: its config takes no + * other mode (directRefusals), and leaving it out means 'never' there. + */ +function watchOf(config: MountConfig): 'always' | 'on-agent-action' | 'never' { + return config.watch ?? (config.backend === 'direct' ? 'never' : 'always'); +} + +/** A config value for a refusal: a number as JavaScript writes it (JSON would say null for NaN), anything else as JSON. */ +function shownValue(value: unknown): string { + return typeof value === 'number' ? String(value) : typeof value === 'bigint' ? `${value}n` : JSON.stringify(value) ?? String(value); +} + +/** + * What a direct mount's config sets that would never act there, each with + * the way out (MountConfig.backend): a direct mount is read-only, and has no + * watcher to tune, no change event to wake on and no push to make. + */ +function directRefusals(mount: MountConfig): string[] { + const refused: string[] = []; + if (mount.mode !== 'read-only') refused.push(`mode must be 'read-only', not ${shownValue(mount.mode)}`); + if (mount.watch !== undefined && mount.watch !== 'never') refused.push(`watch ${shownValue(mount.watch)} would watch nothing (leave it out, or set 'never')`); + for (const key of ['watchDebounceMs', 'watchRootPollMs'] as const) { + if (mount[key] !== undefined) refused.push(`${key} would tune no watcher (leave it out)`); + } + const wake = mount.wakeOnChange; + if (wake === true || (Array.isArray(wake) && wake.length > 0)) refused.push('wakeOnChange would never wake (leave it out)'); + if (mount.autoMaterialize === true) refused.push('autoMaterialize would never push (leave it out)'); + return refused; +} + /** * The answer to a write, edit or delete that the workspace stored but * couldn't finish: its push to disk didn't complete, or what follows the tree diff --git a/src/modules/workspace/observe.ts b/src/modules/workspace/observe.ts index 6986d4cf1..1c0209c0b 100644 --- a/src/modules/workspace/observe.ts +++ b/src/modules/workspace/observe.ts @@ -1035,6 +1035,53 @@ export async function readPath( } } +/** A file's bytes as readBytes read them, none hashed. */ +export interface BytesResult { + kind: 'bytes'; + /** + * Its size in bytes: as read, for a file within the limit; as its fstat + * says, for one over it, and at least one past the limit where the reads + * found it grown past it. + */ + size: number; + /** The whole file, when it is at most the limit both at its fstat and as read; null otherwise. */ + bytes: Buffer | null; + /** Its first bytes (at most HEAD_BYTES), for binary detection and MIME sniffing. */ + head: Buffer; +} + +/** + * A mount-relative file's bytes, through the same located descriptor as + * readPath, for a direct mount's tools, which keep nothing: at most `limit` + * of them, and none hashed. A file over `limit` at its fstat has only its + * head read, and one that grew past it since is read no further than it. + */ +export async function readBytes( + view: MountView, + rootReal: string, + relativePath: string, + limit: number, +): Promise> { + const opened = await openContained(view, rootReal, relativePath); + if (!('handle' in opened)) return opened; + const { handle, info } = opened; + try { + const size = Number(info.size); + if (size > limit) { + const head = Buffer.alloc(Math.min(HEAD_BYTES, size)); + const { bytesRead } = await handle.read(head, 0, head.length, 0); + return { kind: 'bytes', size, bytes: null, head: head.subarray(0, bytesRead) }; + } + const { bytes, more } = await readPrefix(handle, limit, size); + if (more) return { kind: 'bytes', size: Math.max(Number((await handle.stat({ bigint: true })).size), bytes.length + 1), bytes: null, head: bytes.subarray(0, HEAD_BYTES) }; + return { kind: 'bytes', size: bytes.byteLength, bytes, head: bytes.subarray(0, HEAD_BYTES) }; + } catch (err) { + return { kind: 'unobserved', reason: `cannot read (${errno(err) ?? String(err)})` }; + } finally { + await handle.close(); + } +} + /** * The first bytes of a mount-relative file and its size, through the same * located descriptor as readPath — for listing a file nothing tracks, which diff --git a/src/modules/workspace/types.ts b/src/modules/workspace/types.ts index 36c49832a..340cfad42 100644 --- a/src/modules/workspace/types.ts +++ b/src/modules/workspace/types.ts @@ -16,6 +16,33 @@ export interface MountConfig { path: string; /** Access mode */ mode: 'read-write' | 'read-only'; + /** + * Where the mount's tools get what they read (default: 'chronicle'). + * - 'chronicle': the workspace keeps the mount's files in its store, where + * writes and edits land first, and takes in disk's changes under the + * three-way rule as its listings, reads and syncs reach them. + * - 'direct': for a checkout an agent reads, not one it changes. Every + * read, listing and search is disk's own copy as it is when it runs, + * and none of its files go into the store. No scan or watcher + * runs, so no call waits behind one, and `sync`, `materialize` and the + * watch settings have nothing to act on. Only with `mode: 'read-only'`, + * and not with `watch` other than 'never', `watchDebounceMs`, + * `watchRootPollMs`, `wakeOnChange` or `autoMaterialize`: each of those + * would never act, and the workspace refuses the mount. Its `ignore` + * has a default (see there). Paths, links, other mounts' directories, + * size limits and the walk cap hold as on a 'chronicle' mount. Its + * tools read beneath whatever directory is at its root, with no check + * against the one a 'chronicle' run of it last agreed with. + * + * A mount configured 'direct' that the store holds changes on that disk + * lacks, on the branch selected when the store is given (a draft, a + * workspace deletion, a conflict, as `status` counts them pending), runs + * as 'chronicle' that session, so they stay in view; the workspace says + * so through the framework's ops alerts, naming them and the ways to + * start it direct. A mount that ran direct and runs as 'chronicle' again + * takes in what disk changed meanwhile, as after any time away. + */ + backend?: 'chronicle' | 'direct'; /** * Watch mode for filesystem changes: * - 'always': chokidar watches continuously; each event re-observes its paths @@ -43,6 +70,15 @@ export interface MountConfig { * says, a directory with what lies beneath it: `.name.swp` and `.name.swx` * (the whole name), and names ending in `~` or in `.subl…tmp`, matched by * `^\..+\.sw[px]$`, `~$` and `\.subl.*\.tmp$`. + * + * A direct mount (`backend`) whose config names no list takes + * `node_modules`, `dist`, `.git`, `build`, `coverage`, `target`, + * `__pycache__` and `.venv`, each matching that name at any depth. Its + * walks of the whole mount, and of any path no pattern covers, pass over + * what the list covers. A listing, glob or grep whose path, or whose + * pattern's directory, the list covers is walked all the same, the + * patterns covering it set aside and the rest applying beneath it, as a + * sync that names a path walks it on a 'chronicle' mount. */ ignore?: string[]; /** @@ -148,6 +184,12 @@ export interface WorkspaceConfig { export interface MountState { /** The mount config */ config: MountConfig; + /** + * Whether the mount runs direct this session (`MountConfig.backend`), as + * initStore decided: a mount configured direct runs as 'chronicle' while + * the store holds changes there that disk lacks. + */ + direct: boolean; /** Tree state ID in Chronicle */ treeStateId: string; /** Branch-local intent (tombstones, store origin, conflicts) for this mount's paths */ diff --git a/test/workspace-direct.test.ts b/test/workspace-direct.test.ts new file mode 100644 index 000000000..9e725b84a --- /dev/null +++ b/test/workspace-direct.test.ts @@ -0,0 +1,690 @@ +/** + * A read-only direct mount (MountConfig.backend 'direct'): its tools read + * disk as it is when they run, record nothing in the store, and give the + * answers a 'chronicle' mount's tools give where the two should agree. And + * the switch between the kinds, both ways. + */ + +import test, { type TestContext } from 'node:test'; +import assert from 'node:assert/strict'; +import { appendFileSync, chmodSync, mkdirSync, mkdtempSync, renameSync, rmSync, symlinkSync, unlinkSync, writeFileSync } from 'node:fs'; +import { createRequire, syncBuiltinESMExports } from 'node:module'; +import { tmpdir } from 'node:os'; +import { dirname, join } from 'node:path'; +import { JsStore } from '@animalabs/chronicle'; +// A namespace, so this file loads where the module lacks inOrder, and only +// the tests that call it fail there. +import * as workspaceModule from '../src/modules/workspace/index.js'; +import { WorkspaceModule } from '../src/modules/workspace/index.js'; +import type { MountConfig } from '../src/modules/workspace/types.js'; +import type { ModuleContext } from '../src/types/module.js'; +import type { ProcessEvent, ToolResult } from '../src/types/events.js'; +import { changedAfterFstat, countingLoads, withFileHandle } from './helpers/file-handle.js'; + +const IS_ROOT = typeof process.getuid === 'function' && process.getuid() === 0; + +const PNG = Buffer.from( + 'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR4nGP4////fwAJ+wP9KobjigAAAABJRU5ErkJggg==', + 'base64', +); +const GIF = Buffer.from('R0lGODdhAQABAIEAAP///wAAAAAAAAAAACwAAAAAAQABAAAIBAABBAQAOw==', 'base64'); + +type Alert = { kind: string; agent: string; message: string; data?: Record }; + +class Env { + readonly root = mkdtempSync(join(tmpdir(), 'af-ws-direct-')); + readonly ref = join(this.root, 'ref'); + readonly storePath = join(this.root, 'ws.chronicle'); + store: JsStore; + events: ProcessEvent[] = []; + alerts: Alert[] = []; + private live: WorkspaceModule[] = []; + + constructor(t: TestContext) { + mkdirSync(this.ref, { recursive: true }); + this.store = JsStore.openOrCreate({ path: this.storePath }); + t.after(async () => { + for (const m of this.live) await m.stop().catch(() => {}); + if (!this.store.isClosed()) this.store.close(); + rmSync(this.root, { recursive: true, force: true }); + }); + } + + file(rel: string, content: string | Buffer): void { + mkdirSync(dirname(join(this.ref, rel)), { recursive: true }); + writeFileSync(join(this.ref, rel), content); + } + + /** A module whose `ref` mount is direct and read-only unless `mount` says otherwise, with `more` mounts beside it. */ + async open(mount: Partial = {}, more: MountConfig[] = []): Promise { + const m = new WorkspaceModule({ + mounts: [{ name: 'ref', path: this.ref, mode: 'read-only', backend: 'direct', ...mount }, ...more], + }); + m.initStore(this.store); + await m.start({ + isRestart: false, + getState: () => null, + setState: () => {}, + pushEvent: (e: ProcessEvent) => this.events.push(e), + notifyOps: (kind: string, agent: string, message: string, data?: Record) => this.alerts.push({ kind, agent, message, data }), + } as unknown as ModuleContext); + this.live.push(m); + return m; + } + + /** The same mount as a 'chronicle' one, read-write and unwatched. */ + openChronicle(): Promise { + return this.open({ backend: 'chronicle', mode: 'read-write', watch: 'never' }); + } + + async close(m: WorkspaceModule): Promise { + await m.stop(); + this.live = this.live.filter((x) => x !== m); + } + + stateIds(): string[] { + return this.store.listStates().map((s) => s.id).sort(); + } +} + +/** The mounts a module watches. */ +function watched(m: WorkspaceModule): string[] { + return [...(m as unknown as { watchers: Map }).watchers.keys()]; +} + +/** Run `fn` with `node:fs/promises`'s `lstat`, as the module's imports see it, replaced by what `make` makes of it. */ +async function withLstat(make: (original: (...args: any[]) => Promise) => (...args: any[]) => Promise, fn: () => Promise): Promise { + const fsp = createRequire(import.meta.url)('node:fs/promises') as Record Promise>; + const original = fsp.lstat!; + fsp.lstat = make(original); + syncBuiltinESMExports(); + try { + return await fn(); + } finally { + fsp.lstat = original; + syncBuiltinESMExports(); + } +} + +function call(m: WorkspaceModule, name: string, input: Record = {}): Promise { + return m.handleToolCall({ id: name, name, input }); +} + +async function data>(m: WorkspaceModule, name: string, input: Record = {}): Promise { + const result = await call(m, name, input); + assert.equal(result.success, true, `${name} ${JSON.stringify(input)}: ${result.error}`); + return result.data as T; +} + +async function failure(m: WorkspaceModule, name: string, input: Record): Promise { + const result = await call(m, name, input); + assert.equal(result.success, false, `${name} ${JSON.stringify(input)} succeeded: ${JSON.stringify(result.data)}`); + return result.error!; +} + +test('a direct mount reads disk as it is now, and its tools record nothing in the store', async (t) => { + const env = new Env(t); + env.file('src/a.ts', 'one\n'); + env.file('img/dot.png', PNG); + const m = await env.open(); + const states = env.stateIds(); + const sequence = env.store.currentSequence(); + + assert.equal((await data<{ content: string }>(m, 'read', { path: 'ref/src/a.ts' })).content, ' 1\tone'); + appendFileSync(join(env.ref, 'src/a.ts'), 'two\n'); + assert.equal((await data<{ content: string }>(m, 'read', { path: 'ref/src/a.ts' })).content, ' 1\tone\n 2\ttwo'); + await data(m, 'ls', { path: 'ref', recursive: true }); + await data(m, 'ls', { path: 'ref/src' }); + await data(m, 'glob', { pattern: '**/*.ts' }); + await data(m, 'grep', { pattern: 'two' }); + await data(m, 'read_image', { path: 'ref/img/dot.png' }); + await data(m, 'status'); + await data(m, 'sync'); + assert.deepEqual('data' in (await m.readBinary('ref/src/a.ts')), true); + + assert.deepEqual(env.stateIds(), states, 'no state was made'); + assert.ok(!states.some((id) => id.startsWith('workspace/ref/')), `the mount's states: ${states}`); + assert.equal(env.store.currentSequence(), sequence, 'nothing was recorded'); + assert.deepEqual(env.events.filter((e) => e.type !== 'workspace:mounted').map((e) => e.type), [], 'no change was evented'); + assert.deepEqual(watched(m), [], 'no watcher runs, though watch is left out'); +}); + +test("read on a direct mount refuses what a 'chronicle' mount never stores: a binary file, and one over the size limit", async (t) => { + const env = new Env(t); + env.file('bin.dat', Buffer.from([0x61, 0x00, 0x62])); + env.file('dot.png', PNG); + env.file('big.txt', 'x'.repeat(9)); + env.file('fits.txt', 'x'.repeat(8)); + const m = await env.open({ maxFileSize: 8 }); + assert.equal(await failure(m, 'read', { path: 'ref/bin.dat' }), "ref/bin.dat is binary, so read doesn't show it as text."); + assert.equal(await failure(m, 'read', { path: 'ref/dot.png' }), "ref/dot.png is over the size limit (8 bytes), so read doesn't show it."); + assert.equal(await failure(m, 'read', { path: 'ref/big.txt' }), "ref/big.txt is over the size limit (8 bytes), so read doesn't show it."); + assert.equal((await data<{ content: string }>(m, 'read', { path: 'ref/fits.txt' })).content, ' 1\txxxxxxxx'); + + const roomy = await env.open({ maxFileSize: 1024 }); + assert.equal(await failure(roomy, 'read', { path: 'ref/dot.png' }), "ref/dot.png is binary, so read doesn't show it as text; read it with read_image."); +}); + +test("read on a direct mount answers with the disk reader's refusals", async (t) => { + const env = new Env(t); + env.file('src/a.ts', 'a\n'); + symlinkSync(join(env.ref, 'src/a.ts'), join(env.ref, 'link.ts')); + symlinkSync(join(env.ref, 'src'), join(env.ref, 'srclink')); + const m = await env.open(); + assert.equal(await failure(m, 'read', { path: 'ref/missing.ts' }), 'File not found: ref/missing.ts'); + assert.equal(await failure(m, 'read', { path: 'ref/src' }), 'Path is a directory: ref/src'); + assert.equal(await failure(m, 'read', { path: 'ref/link.ts' }), 'Symlinks are not allowed: ref/link.ts'); + assert.equal( + await failure(m, 'read', { path: 'ref/srclink/a.ts' }), + 'ref/srclink/a.ts: "ref/srclink" is a symlink to "ref/src", which this mount does not follow; the mount\'s host can set followSymlinks: true', + ); + assert.match(await failure(m, 'read', { path: 'ref/../escape.txt' }), /^Path traversal detected/); +}); + +test("read_image on a direct mount never shows the copy a 'chronicle' run stored", async (t) => { + const env = new Env(t); + const chronicle = await env.open({ backend: 'chronicle', mode: 'read-write', watch: 'never', autoMaterialize: true }); + assert.equal((await chronicle.writeBinary('ref/dot.img', PNG, 'image/png')).success, true); + await env.close(chronicle); + env.file('dot.img', GIF); + const direct = await env.open(); + assert.deepEqual(env.alerts, [], 'it starts direct'); + const image = await data>(direct, 'read_image', { path: 'ref/dot.img' }); + assert.deepEqual([image[1]!.mimeType, image[1]!.data], ['image/gif', GIF.toString('base64')]); +}); + +test("read_image on a direct mount reads disk's image as it is now", async (t) => { + const env = new Env(t); + env.file('dot.img', PNG); + const m = await env.open(); + const first = await data>(m, 'read_image', { path: 'ref/dot.img' }); + assert.equal(first[0]!.text, `Path: ref/dot.img\nMIME: image/png\nBytes: ${PNG.byteLength}`); + assert.equal(first[1]!.data, PNG.toString('base64')); + env.file('dot.img', GIF); + const second = await data>(m, 'read_image', { path: 'ref/dot.img' }); + assert.equal(second[1]!.mimeType, 'image/gif'); + assert.equal(second[1]!.data, GIF.toString('base64')); +}); + +test('ls on a direct mount lists disk\'s entries with their sizes and no state, and a link with why it isn\'t taken', async (t) => { + const env = new Env(t); + env.file('a.txt', 'abc'); + env.file('src/b.ts', 'bb'); + symlinkSync(join(env.ref, 'a.txt'), join(env.ref, 'link.txt')); + const m = await env.open(); + assert.deepEqual((await data(m, 'ls', { path: 'ref' })).entries, [ + { name: 'src', type: 'directory' }, + { name: 'a.txt', type: 'file', size: 3 }, + { name: 'link.txt', type: 'file', note: 'a symlink, which this mount does not follow' }, + ]); + assert.deepEqual(await data(m, 'ls', { path: 'ref', recursive: true }), { + path: 'ref', + entries: [ + { path: 'a.txt', size: 3 }, + { path: 'link.txt', note: 'a symlink, which this mount does not follow' }, + { path: 'src/b.ts', size: 2 }, + ], + count: 3, + }); + + const following = await env.open({ followSymlinks: true }); + assert.deepEqual((await data(following, 'ls', { path: 'ref' })).entries, [ + { name: 'src', type: 'directory' }, + { name: 'a.txt', type: 'file', size: 3 }, + { name: 'link.txt', type: 'file', size: 3 }, + ]); +}); + +test('the default ignore list keeps whole walks out of what it covers, and a path a tool names is walked', async (t) => { + const env = new Env(t); + env.file('src/a.js', 'needle\n'); + env.file('node_modules/pkg/index.js', 'needle\n'); + env.file('node_modules/pkg/dist/built.js', 'needle\n'); + env.file('dist/out.js', 'needle\n'); + const m = await env.open(); + assert.deepEqual((await data(m, 'ls', { path: 'ref' })).entries, [{ name: 'src', type: 'directory' }]); + assert.deepEqual((await data<{ matches: unknown[] }>(m, 'glob', { pattern: '**/*.js' })).matches, [{ path: 'ref/src/a.js', size: 7 }]); + assert.deepEqual( + (await data<{ results: Array<{ file: string }> }>(m, 'grep', { pattern: 'needle' })).results.map((r) => r.file), + ['ref/src/a.js'], + ); + // Named, by a path or a pattern's directory: walked, the rest of the list + // applying beneath it. + assert.deepEqual((await data(m, 'ls', { path: 'ref/node_modules/pkg' })).entries, [{ name: 'index.js', type: 'file', size: 7 }]); + assert.deepEqual((await data<{ matches: unknown[] }>(m, 'glob', { pattern: 'node_modules/pkg/*.js' })).matches, [{ path: 'ref/node_modules/pkg/index.js', size: 7 }]); + assert.deepEqual( + (await data<{ results: Array<{ file: string }> }>(m, 'grep', { pattern: 'needle', path: 'ref/node_modules' })).results.map((r) => r.file), + ['ref/node_modules/pkg/index.js'], + ); + assert.deepEqual( + (await data<{ results: Array<{ file: string }> }>(m, 'grep', { pattern: 'needle', glob: 'node_modules/**' })).results.map((r) => r.file), + ['ref/node_modules/pkg/index.js'], + ); + // A list the config names replaces it. + const own = await env.open({ ignore: ['dist'] }); + assert.deepEqual( + (await data<{ matches: Array<{ path: string }> }>(own, 'glob', { pattern: '**/*.js' })).matches.map((x) => x.path), + ['ref/node_modules/pkg/index.js', 'ref/src/a.js'], + ); +}); + +test('glob and grep on a direct mount find a file the moment disk has it, as disk has it', async (t) => { + const env = new Env(t); + env.file('src/a.ts', 'alpha\n'); + env.file('src/notes.md', 'notes\n'); + const m = await env.open(); + assert.equal((await data<{ count: number }>(m, 'glob', { pattern: 'src/*.ts' })).count, 1); + env.file('src/b.ts', 'beta\nalpha again\n'); + assert.deepEqual((await data<{ matches: unknown[] }>(m, 'glob', { pattern: 'src/*.ts' })).matches, [ + { path: 'ref/src/a.ts', size: 6 }, + { path: 'ref/src/b.ts', size: 17 }, + ]); + assert.deepEqual(await data(m, 'grep', { pattern: 'alpha', path: 'ref/src', contextAfter: 1 }), { + pattern: 'alpha', + results: [ + { file: 'ref/src/a.ts', version: 'disk', matches: [{ line: 1, text: 'alpha', context: ['alpha'] }] }, + { file: 'ref/src/b.ts', version: 'disk', matches: [{ line: 2, text: 'alpha again', context: ['alpha again'] }] }, + ], + totalMatches: 2, + }); + assert.deepEqual((await data<{ results: unknown[] }>(m, 'grep', { pattern: 'beta', path: 'ref/src/b.ts' })).results, [ + { file: 'ref/src/b.ts', version: 'disk', matches: [{ line: 1, text: 'beta' }] }, + ]); + assert.deepEqual( + (await data<{ results: Array<{ file: string }> }>(m, 'grep', { pattern: 'alpha', glob: 'src/b.*' })).results.map((r) => r.file), + ['ref/src/b.ts'], + ); +}); + +test("a direct mount's walks keep the cap, start where their pattern names, and say what they couldn't reach", async (t) => { + const env = new Env(t); + for (const name of ['1', '2', '3']) env.file(`node_modules/aaa/${name}.js`, 'x\n'); + env.file('node_modules/pkg/index.js', 'x\n'); + const m = await env.open({ walkCap: 2 }); + // The start descends into the directory the pattern names, past what the + // cap would leave of a walk from above it. + assert.deepEqual((await data<{ matches: unknown[] }>(m, 'glob', { pattern: 'node_modules/pkg/*.js' })).matches, [{ path: 'ref/node_modules/pkg/index.js', size: 2 }]); + const capped = [ + { path: 'node_modules/aaa', reason: 'the walk cap (2) was reached before everything here was taken' }, + { path: 'node_modules/pkg', reason: 'not visited: the walk cap (2) was reached' }, + ]; + assert.deepEqual(await data(m, 'ls', { path: 'ref/node_modules', recursive: true }), { + path: 'ref/node_modules', + entries: [{ path: 'node_modules/aaa/1.js', size: 2 }, { path: 'node_modules/aaa/2.js', size: 2 }], + count: 2, + incomplete: capped, + }); + assert.deepEqual( + (await data<{ incomplete: unknown[] }>(m, 'grep', { pattern: 'x', path: 'ref/node_modules' })).incomplete, + capped.map((region) => ({ ...region, path: `ref/${region.path}` })), + ); +}); + +test("a direct listing names a directory it couldn't list", { skip: IS_ROOT ? 'root lists every directory' : false }, async (t) => { + const env = new Env(t); + env.file('open/a.ts', 'a\n'); + env.file('shut/b.ts', 'b\n'); + const m = await env.open(); + chmodSync(join(env.ref, 'shut'), 0o000); + try { + assert.deepEqual(await data(m, 'ls', { path: 'ref', recursive: true }), { + path: 'ref', + entries: [{ path: 'open/a.ts', size: 2 }], + count: 1, + incomplete: [{ path: 'shut', reason: 'cannot list (EACCES)' }], + }); + } finally { + chmodSync(join(env.ref, 'shut'), 0o755); + } +}); + +test("a direct mount's read and grep load no more than the size limit, and refuse a file that grows past it as it's read", async (t) => { + const env = new Env(t); + env.file('grows.txt', 'x'.repeat(8)); + env.file('huge.txt', 'y'.repeat(100_000)); + const m = await env.open({ maxFileSize: 10 }); + const over = (path: string) => `${path} is over the size limit (10 bytes), so read doesn't show it.`; + // Over the limit at its open: read loads none of it, and grep its head. + const read = await countingLoads(() => call(m, 'read', { path: 'ref/huge.txt' })); + assert.deepEqual([read.result.error, read.loaded], [over('ref/huge.txt'), 0]); + const grep = await countingLoads(() => call(m, 'grep', { pattern: 'y', path: 'ref/huge.txt' })); + assert.deepEqual((grep.result.data as { skipped: unknown[] }).skipped, [{ file: 'ref/huge.txt', reason: 'over the size limit (10 bytes)', size: 100_000 }]); + assert.equal(grep.loaded, 8192); + // Grown past it after its open's fstat: no more than the limit and a byte. + const grows = join(env.ref, 'grows.txt'); + const grown = await changedAfterFstat(grows, () => appendFileSync(grows, 'x'.repeat(1000)), + () => countingLoads(() => call(m, 'read', { path: 'ref/grows.txt' }))); + assert.equal(grown.result.error, over('ref/grows.txt')); + assert.ok(grown.loaded <= 11, `loaded ${grown.loaded}`); + writeFileSync(grows, 'x'.repeat(8)); + const grownGrep = await changedAfterFstat(grows, () => appendFileSync(grows, 'x'.repeat(1000)), + () => countingLoads(() => call(m, 'grep', { pattern: 'x', path: 'ref/grows.txt' }))); + assert.deepEqual((grownGrep.result.data as { skipped: unknown[] }).skipped, [{ file: 'ref/grows.txt', reason: 'over the size limit (10 bytes)', size: 1008 }]); + assert.ok(grownGrep.loaded <= 11, `loaded ${grownGrep.loaded}`); +}); + +test("a direct mount's read and grep say so where reading a file fails", async (t) => { + const env = new Env(t); + env.file('a.txt', 'needle\n'); + const m = await env.open(); + const failing = { read: () => async function () { throw Object.assign(new Error('EIO: i/o error, read'), { code: 'EIO' }); } }; + assert.equal((await withFileHandle(failing, () => call(m, 'read', { path: 'ref/a.txt' }))).error, 'Unable to read file: ref/a.txt (EIO)'); + assert.deepEqual((await withFileHandle(failing, () => call(m, 'grep', { pattern: 'needle' }))).data, { + pattern: 'needle', + results: [], + totalMatches: 0, + skipped: [{ file: 'ref/a.txt', reason: 'cannot read (EIO)' }], + }); +}); + +test('a file gone between the walk that listed it and its read is neither listed nor searched', async (t) => { + const env = new Env(t); + const names = Array.from({ length: 40 }, (_, i) => `f${String(i).padStart(2, '0')}.txt`); + for (const name of names) env.file(name, 'needle\n'); + const m = await env.open(); + const gone = join(env.ref, 'f30.txt'); + const left = names.filter((name) => name !== 'f30.txt'); + // grep reads 16 at a time, in order: the 31st file is opened only once the + // 15th is done, so after the 2nd file's open, when it's removed. + const grep = await changedAfterFstat(join(env.ref, 'f01.txt'), () => unlinkSync(gone), + () => data<{ results: Array<{ file: string }>; skipped?: unknown }>(m, 'grep', { pattern: 'needle' })); + assert.deepEqual(grep.results.map((r) => r.file), left.map((name) => `ref/${name}`)); + assert.equal(grep.skipped, undefined); + // A listing looks at each file by its path, where this one is removed first. + env.file('f30.txt', 'needle\n'); + const listed = await withLstat((lstat) => async (...args: any[]) => { + if (args[0] === gone) rmSync(gone, { force: true }); + return lstat(...args); + }, () => data<{ entries: Array<{ path: string }> }>(m, 'ls', { path: 'ref', recursive: true })); + assert.deepEqual(listed.entries.map((e) => e.path), left); +}); + +test('grep on a direct mount lists what it doesn\'t search, with why', async (t) => { + const env = new Env(t); + env.file('text.txt', 'needle\n'); + env.file('bin.dat', Buffer.concat([Buffer.from('needle'), Buffer.from([0])])); + env.file('dot.png', PNG); + env.file('big.txt', 'needle '.repeat(40)); + symlinkSync(join(env.ref, 'text.txt'), join(env.ref, 'link.txt')); + const m = await env.open({ maxFileSize: 100 }); + assert.deepEqual(await data(m, 'grep', { pattern: 'needle' }), { + pattern: 'needle', + results: [{ file: 'ref/text.txt', version: 'disk', matches: [{ line: 1, text: 'needle' }] }], + totalMatches: 1, + skipped: [ + { file: 'ref/big.txt', reason: 'over the size limit (100 bytes)', size: 280 }, + { file: 'ref/bin.dat', reason: 'binary: a NUL in its first 8 KB', size: 7 }, + { file: 'ref/dot.png', reason: 'binary: a NUL in its first 8 KB', size: PNG.byteLength, mimeType: 'image/png' }, + { file: 'ref/link.txt', reason: 'a symlink, which this mount does not follow' }, + ], + }); +}); + +test("grep's results come in path order, however its reads finish", async (t) => { + const env = new Env(t); + const names = Array.from({ length: 40 }, (_, i) => `f${String(i).padStart(2, '0')}.txt`); + for (const name of [...names].reverse()) env.file(`many/${name}`, `${name} needle\n`.repeat(1 + (names.indexOf(name) % 7) * 500)); + const m = await env.open(); + const files = (await data<{ results: Array<{ file: string }> }>(m, 'grep', { pattern: 'needle' })).results.map((r) => r.file); + assert.deepEqual(files, names.map((name) => `ref/many/${name}`)); +}); + +test("ls without a path and status give a direct mount's kind and ignore list, and a 'chronicle' mount's as before", async (t) => { + const env = new Env(t); + const work = join(env.root, 'work'); + mkdirSync(work); + const m = await env.open({}, [{ name: 'work', path: work, mode: 'read-write', watch: 'never' }]); + const ignore = ['node_modules', 'dist', '.git', 'build', 'coverage', 'target', '__pycache__', '.venv']; + assert.deepEqual((await data(m, 'ls')).mounts, [ + { name: 'ref', path: env.ref, mode: 'read-only', backend: 'direct', ignore }, + { name: 'work', path: work, mode: 'read-write' }, + ]); + const status = await data>>(m, 'status'); + assert.deepEqual(status.ref, { path: env.ref, mode: 'read-only', backend: 'direct', ignore }); + assert.deepEqual(Object.keys(status.work!).sort(), [ + 'canMaterialize', 'conflicts', 'currentBranch', 'currentSeq', 'fileCount', 'initialSyncDone', 'lastMaterializedBranch', + 'lastMaterializedSeq', 'mode', 'path', 'pendingChanges', 'pendingWorkspaceDeletions', 'watch', + ]); + const named = await env.open({ ignore: ['vendor'] }); + assert.deepEqual((await data>>(named, 'status')).ref!.ignore, ['vendor']); +}); + +test('sync passes over a direct mount, and says so rather than that nothing changed', async (t) => { + const env = new Env(t); + env.file('src/a.ts', 'a\n'); + const m = await env.open(); + const reason = "a direct mount: its tools read disk as it is, so there's nothing to take in"; + assert.deepEqual(await data(m, 'sync'), { results: [], totalSynced: 0, totalConflicts: 0, skipped: [{ mount: 'ref', path: '', reason }] }); + assert.deepEqual((await data(m, 'sync', { mount: 'ref', acceptRoot: true })).skipped, [{ mount: 'ref', path: '', reason }]); + assert.deepEqual((await data(m, 'sync', { path: 'ref/src' })).skipped, [{ mount: 'ref', path: 'src', reason }]); + assert.deepEqual((await data(m, 'materialize', { mount: 'ref' })).skipped, [{ mount: 'ref', reason: 'a read-only mount: materialize writes nothing to it' }]); +}); + +test("readBinary reads a direct mount's disk, a binary file included, up to the mount's size limit", async (t) => { + const env = new Env(t); + env.file('bin.dat', Buffer.from([1, 0, 2])); + env.file('big.dat', Buffer.alloc(9)); + const m = await env.open({ maxFileSize: 8 }); + assert.deepEqual(await m.readBinary('ref/bin.dat'), { data: Buffer.from([1, 0, 2]) }); + assert.deepEqual(await m.readBinary('ref/big.dat'), { error: 'File too large: 9 > 8 bytes' }); + assert.deepEqual(await m.readBinary('ref/none.dat'), { error: 'File not found on disk: ref/none.dat' }); +}); + +test('a direct mount reads beneath whatever directory is at its root now', async (t) => { + const env = new Env(t); + env.file('a.txt', 'old\n'); + // A 'chronicle' run records the root it agreed with. + const chronicle = await env.openChronicle(); + assert.equal((await data<{ content: string }>(chronicle, 'read', { path: 'ref/a.txt' })).content, ' 1\told'); + await env.close(chronicle); + // The checkout is cloned again: another directory at the same path. + renameSync(env.ref, join(env.root, 'ref-before')); + env.file('a.txt', 'new\n'); + env.file('sub/x.txt', 'x\n'); + // A cap of one entry, so a search's start that the old root's identity + // stopped short would leave its directory unvisited. + const direct = await env.open({ walkCap: 1 }); + assert.equal((await data<{ content: string }>(direct, 'read', { path: 'ref/a.txt' })).content, ' 1\tnew'); + assert.deepEqual((await data<{ matches: unknown[] }>(direct, 'glob', { pattern: '*.txt' })).matches, [{ path: 'ref/a.txt', size: 4 }]); + assert.deepEqual((await data<{ matches: unknown[] }>(direct, 'glob', { pattern: 'sub/*.txt' })).matches, [{ path: 'ref/sub/x.txt', size: 2 }]); + await env.close(direct); + // A 'chronicle' run checks the root against the one it agreed with. + const again = await env.openChronicle(); + assert.match(JSON.stringify((await data(again, 'ls', { path: 'ref' })).incomplete), /not the directory disk last agreed with/); +}); + +test('a direct mount whose config sets what would never act there is refused, naming the mount and each setting', () => { + const base: MountConfig = { name: 'ref', path: '/tmp/af-ws-direct-none', mode: 'read-only', backend: 'direct' }; + const refused = (mount: Partial): string => { + try { + new WorkspaceModule({ mounts: [{ ...base, ...mount }] }); + } catch (err) { + return (err as Error).message; + } + assert.fail(`accepted ${JSON.stringify(mount)}`); + }; + const lead = '[workspace] Mount "ref" is direct, which reads disk as it is with no store, watcher or push: '; + assert.equal(refused({ mode: 'read-write' }), `${lead}mode must be 'read-only', not "read-write"`); + assert.equal(refused({ watch: 'always' }), `${lead}watch "always" would watch nothing (leave it out, or set 'never')`); + assert.equal(refused({ watch: 'on-agent-action' }), `${lead}watch "on-agent-action" would watch nothing (leave it out, or set 'never')`); + assert.equal(refused({ watchDebounceMs: 100 }), `${lead}watchDebounceMs would tune no watcher (leave it out)`); + assert.equal(refused({ watchRootPollMs: 100 }), `${lead}watchRootPollMs would tune no watcher (leave it out)`); + assert.equal(refused({ wakeOnChange: true }), `${lead}wakeOnChange would never wake (leave it out)`); + assert.equal(refused({ wakeOnChange: ['created'] }), `${lead}wakeOnChange would never wake (leave it out)`); + assert.equal(refused({ autoMaterialize: true }), `${lead}autoMaterialize would never push (leave it out)`); + assert.equal( + refused({ mode: 'read-write', autoMaterialize: true }), + `${lead}mode must be 'read-only', not "read-write"; autoMaterialize would never push (leave it out)`, + ); + for (const accepted of [{}, { watch: 'never' as const }, { wakeOnChange: false }, { wakeOnChange: [] }, { autoMaterialize: false }]) { + new WorkspaceModule({ mounts: [{ ...base, ...accepted }] }); + } + assert.throws( + () => new WorkspaceModule({ mounts: [{ ...base, backend: 'disk' as MountConfig['backend'] }] }), + { message: `[workspace] Mount "ref": backend must be 'chronicle' or 'direct', not "disk"` }, + ); +}); + +test('tool descriptions mention the direct kind only on a host with a direct mount', async (t) => { + const env = new Env(t); + const descriptions = (m: WorkspaceModule) => Object.fromEntries(m.getTools().map((tool) => [tool.name, tool.description])); + const chronicle = descriptions(await env.openChronicle()); + for (const [name, description] of Object.entries(chronicle)) assert.doesNotMatch(description, /\bdirect\b/, name); + const direct = descriptions(await env.open()); + for (const name of Object.keys(chronicle)) { + if (name === 'ls' || name === 'grep' || name === 'sync') { + assert.ok(direct[name]!.startsWith(chronicle[name]!) && direct[name]!.length > chronicle[name]!.length, name); + assert.match(direct[name]!, /direct mount/, name); + } else { + assert.equal(direct[name], chronicle[name], name); + } + } +}); + +test("a mount the store holds changes on that disk lacks runs as 'chronicle', says so, and starts direct once they're given up", async (t) => { + const env = new Env(t); + env.file('drafted.ts', 'disk\n'); + env.file('deleted.ts', 'kept on disk\n'); + env.file('both.ts', 'v1\n'); + const chronicle = await env.openChronicle(); + await data(chronicle, 'ls', { path: 'ref' }); + await data(chronicle, 'write', { path: 'ref/drafted.ts', content: 'draft\n' }); + await data(chronicle, 'delete', { path: 'ref/deleted.ts' }); + await data(chronicle, 'write', { path: 'ref/both.ts', content: 'v2\n' }); + env.file('both.ts', 'v3\n'); + assert.equal((await data<{ entries: Array<{ name: string; state: string }> }>(chronicle, 'ls', { path: 'ref' })).entries.find((e) => e.name === 'both.ts')!.state, 'conflict'); + await env.close(chronicle); + + const held = await env.open({ walkCap: 1 }); + assert.deepEqual(env.alerts, [{ + kind: 'workspace-mount-not-direct', + agent: 'framework', + message: 'Mount "ref" runs as \'chronicle\' this session, though its config makes it direct: the workspace holds changes ' + + 'there that disk lacks, which a direct mount wouldn\'t show: "ref/both.ts" (in conflict), "ref/deleted.ts" (deleted in ' + + 'the workspace), "ref/drafted.ts" (changed in the workspace). To start it direct, finish them (with the mount configured ' + + '\'chronicle\' and read-write, materialize them) or give them up (sync each path, which takes disk\'s copy); the first ' + + 'start that finds none starts it direct.', + data: { mount: 'ref', held: 3 }, + }]); + // As a 'chronicle' mount, unwatched, with no catch-up scan to take turns + // with its syncs, its changes in view. + const status = await data>>(held, 'status'); + assert.equal(status.ref!.backend, undefined); + assert.equal(status.ref!.watch, 'never'); + assert.deepEqual(watched(held), []); + assert.equal(status.ref!.pendingChanges, 3); + const capped = (await data<{ incomplete: Array<{ path: string; reason: string }> }>(held, 'sync')).incomplete.map((region) => region.reason); + assert.ok(capped.some((reason) => /^the walk cap \(1\) was reached: the next sync of the mount resumes after ref\//.test(reason)), JSON.stringify(capped)); + assert.equal((await data<{ content: string }>(held, 'read', { path: 'ref/drafted.ts' })).content, ' 1\tdraft'); + assert.ok(held.getTools().every((tool) => !/\bdirect\b/.test(tool.description))); + + for (const path of ['ref/drafted.ts', 'ref/deleted.ts', 'ref/both.ts']) await data(held, 'sync', { path }); + await env.close(held); + env.alerts = []; + const direct = await env.open(); + assert.deepEqual(env.alerts, []); + assert.equal((await data>>(direct, 'status')).ref!.backend, 'direct'); + assert.equal((await data<{ content: string }>(direct, 'read', { path: 'ref/drafted.ts' })).content, ' 1\tdisk'); +}); + +test("the changes that keep a mount from starting direct are named ten at a time, an intent that can't be read among them", async (t) => { + const env = new Env(t); + const chronicle = await env.openChronicle(); + for (let i = 0; i < 11; i++) await data(chronicle, 'write', { path: `ref/d${String(i).padStart(2, '0')}.ts`, content: `${i}\n` }); + await env.close(chronicle); + // A recorded intent whose blob the store doesn't hold. + env.store.treeSet('workspace/ref/intent', 'aa.ts', { blobHash: 'f'.repeat(64), size: 2, mode: 0 }); + + await env.open(); + assert.equal(env.alerts.length, 1); + const shown = Array.from({ length: 9 }, (_, i) => `"ref/d0${i}.ts" (changed in the workspace)`).join(', '); + assert.match(env.alerts[0]!.message, new RegExp(`disk lacks, which a direct mount wouldn't show: "ref/aa\\.ts" \\(its workspace intent can't be read\\), ${shown.replace(/[.()]/g, '\\$&')}, and 2 more\\. To start it direct`)); + assert.deepEqual(env.alerts[0]!.data, { mount: 'ref', held: 12 }); +}); + +test("a mount that ran direct takes in what disk changed meanwhile when it runs as 'chronicle' again", async (t) => { + const env = new Env(t); + env.file('changed.ts', 'before\n'); + env.file('removed.ts', 'gone soon\n'); + const first = await env.openChronicle(); + assert.equal((await data<{ count: number }>(first, 'ls', { path: 'ref', recursive: true })).count, 2); + await env.close(first); + + const direct = await env.open(); + env.file('changed.ts', 'after\n'); + unlinkSync(join(env.ref, 'removed.ts')); + env.file('added.ts', 'new\n'); + assert.equal((await data<{ content: string }>(direct, 'read', { path: 'ref/changed.ts' })).content, ' 1\tafter'); + await env.close(direct); + + const again = await env.openChronicle(); + assert.deepEqual((await data(again, 'ls', { path: 'ref', recursive: true })).entries, [ + { path: 'added.ts', state: 'synced', size: 4 }, + { path: 'changed.ts', state: 'synced', size: 6 }, + ]); + assert.equal((await data<{ content: string }>(again, 'read', { path: 'ref/changed.ts' })).content, ' 1\tafter'); + const status = (await data>>(again, 'status')).ref!; + assert.deepEqual([status.pendingChanges, status.conflicts], [0, 0]); +}); + +test("a direct mount's listings pass over another mount's directory, naming the mount to use", async (t) => { + const env = new Env(t); + env.file('inner/x.ts', 'x\n'); + env.file('top.ts', 't\n'); + const m = await env.open({}, [{ name: 'inner', path: join(env.ref, 'inner'), mode: 'read-only', watch: 'never' }]); + assert.deepEqual((await data(m, 'ls', { path: 'ref' })).entries, [{ name: 'top.ts', type: 'file', size: 2 }]); + assert.deepEqual(await data(m, 'ls', { path: 'ref/inner' }), { + path: 'ref/inner', + entries: [], + count: 0, + incomplete: [{ path: 'inner', reason: 'it lies in mount "inner", nested in this one: use "inner"' }], + }); + assert.match(await failure(m, 'read', { path: 'ref/inner/x.ts' }), /lies in mount "inner", nested in mount "ref"/); +}); + +test('inOrder hands each result on in order, at most `width` at once', async () => { + const running = { now: 0, most: 0 }; + const order: number[] = []; + const delays = [30, 5, 20, 1, 10, 2, 25, 3]; + await workspaceModule.inOrder(delays.map((_, i) => i), 3, async (i) => { + running.now++; + running.most = Math.max(running.most, running.now); + await new Promise((resolve) => setTimeout(resolve, delays[i])); + running.now--; + return i * 10; + }, (result, i) => { + assert.equal(result, i * 10); + order.push(i); + }); + assert.deepEqual(order, [0, 1, 2, 3, 4, 5, 6, 7]); + assert.equal(running.most, 3); +}); + +test('inOrder rejects in turn for a work that rejects before its turn, leaving no rejection unhandled', async () => { + const unhandled: unknown[] = []; + const record = (reason: unknown) => { unhandled.push(reason); }; + process.on('unhandledRejection', record); + try { + let second!: () => void; + const secondRejected = new Promise((resolve) => { second = resolve; }); + const seen: number[] = []; + await assert.rejects(workspaceModule.inOrder([0, 1, 2], 3, async (i) => { + if (i === 1) { + queueMicrotask(second); + throw new Error('the second'); + } + // The first finishes only after the second has rejected and a turn of + // the event loop has passed, when Node reports a rejection unhandled. + if (i === 0) { + await secondRejected; + await new Promise((resolve) => setImmediate(resolve)); + } + return i; + }, (_, i) => { seen.push(i); }), { message: 'the second' }); + await new Promise((resolve) => setImmediate(resolve)); + assert.deepEqual(seen, [0]); + assert.deepEqual(unhandled, []); + } finally { + process.off('unhandledRejection', record); + } +}); From 42e0533a33babea428b07eb9c6eefae5b4f380f0 Mon Sep 17 00:00:00 2001 From: Beth-2709 Date: Mon, 19 Oct 2026 03:21:15 -0400 Subject: [PATCH 02/13] fix(workspace): the direct mount's first review round, Esther-2584's read, and one spelling per path on every mount Answers an internal review of 6aadccb6b, and Esther-2584's read of it. The switch between kinds: - chooseKinds decides whether a mount configured direct runs direct when initStore gives the store, and again in ensureRunning at each start, which knows how the mounts nest, so a later start of the same module that finds no changes held starts it direct. - One ops alert per start names every such mount, with what's happening and the ways out (sync a path to take disk's copy, or materialize from a 'chronicle' read-write mount as materialize's answer directs) before the list of changes, which the framework's webhook may cut. - A mount configured direct walks by the direct default ignore list where its config names none, whichever kind it runs as (ignoreOf by the configured kind, as watchOf goes; found by Esther-2584). - DiskAgreement.open takes only mounts whose states exist, so a direct mount that never ran store-backed begins no evidence in the store. The direct readers: - inOrder lets each result go once handed on, so a grep holds at most its width of file contents. - A direct read of a file it can't open says why with its errno (Esther-2584). - readBytes names a link to a directory as a listing does (Esther-2584). - A root replaced while a direct read runs is refused in words of its own: reading again reads the directory there now. Paths, on every mount (Ada-1017's call; taken out again two commits on): parsePathAsGiven drops '.' and empty segments and a trailing slash, and reads '..' lexically, as its traversal check already did, still refusing a path that leaves the mount; parsePath answers with the spelling so made. Fragment workspace-path-one-spelling.changed.md. Text: glob gains a direct sentence where a mount runs direct; an omitted mode on a direct mount is refused as left out, read-write; types.ts's backend doc names a 'chronicle' read's behaviour and an unreadable intent among what's held; the changelog fragment drops 'never stores' and names no MIME type on listings. Tests: inOrder letting results go (gc), a same-module restart, every held kind alone, two mounts in one alert, a branch not selected, the fallback's stderr line and every tool there, 16 reads at a time at grep and listing, every handle closed, a skipped file's size when it changes as read, a root replaced mid-read, paths with dot and empty segments on both kinds, a mixed host's descriptions, the omitted mode, no incomplete for ignored children, and grep's order with its first file held back. Co-Authored-By: Claude Opus 5.5 --- changelog.d/workspace-direct-mount.added.md | 6 +- .../workspace-path-one-spelling.changed.md | 1 + src/modules/workspace/index.ts | 150 ++++--- src/modules/workspace/observe.ts | 13 +- src/modules/workspace/types.ts | 36 +- test/workspace-direct.test.ts | 408 +++++++++++++++--- test/workspace-parent-link.test.ts | 4 +- test/workspace-three-way.test.ts | 5 +- 8 files changed, 502 insertions(+), 121 deletions(-) create mode 100644 changelog.d/workspace-path-one-spelling.changed.md diff --git a/changelog.d/workspace-direct-mount.added.md b/changelog.d/workspace-direct-mount.added.md index 71e4a4387..61fb6e45c 100644 --- a/changelog.d/workspace-direct-mount.added.md +++ b/changelog.d/workspace-direct-mount.added.md @@ -1,7 +1,7 @@ - **A direct workspace mount, for a checkout an agent reads:** a mount whose config sets `backend: 'direct'` (the default is `'chronicle'`, the store-backed kind every mount has been) is read straight from disk. `read`, `read_image`, `ls`, `glob` and `grep` give its files as they are when they run, with no scan, watcher or pass, so no call waits behind one, and none of its files go into the store. It's read-only: the workspace refuses a direct mount with any `mode` but `'read-only'`, and one that sets `watch` other than `'never'`, `watchDebounceMs`, `watchRootPollMs`, a `wakeOnChange` that would wake, or `autoMaterialize: true`, since none of them would ever act there. Its paths, links, other mounts' directories, size limit and walk cap hold as on a store-backed mount. Where it differs: - - **What its listings say:** each file with its `size` and no state, or, for a link the mount doesn't follow or another entry it doesn't take, with why as `note`; `grep`'s results say `version: 'disk'`, and it lists under `skipped` a binary file, one over the size limit and one the mount doesn't take, each with why. `read` refuses a binary file and one over the size limit, as a store-backed mount, which never stores them, answers too. + - **What its listings say:** each file with its `size`, and no state or MIME type, or, for a link the mount doesn't follow or another entry it doesn't take, with why as `note`. A listing looks at each entry with `lstat`, so it lists a file it can't open with its size, and reading that file says why. `grep`'s results say `version: 'disk'`, and it lists under `skipped` a binary file, one over the size limit and one it doesn't read, each with why. `read` refuses a binary file and one over the size limit, as a store-backed mount's `read` refuses one it never took in from disk. - **Its ignore list:** with none in its config, it takes `node_modules`, `dist`, `.git`, `build`, `coverage`, `target`, `__pycache__` and `.venv`. Walks of the whole mount pass over what the list covers, and a path a listing, glob or grep names, or its pattern's directory, is walked all the same, as a sync naming a path walks it on a store-backed mount. `ls` without a path, and `status`, give a direct mount's `backend` and the list it goes by. - **Its root:** it reads beneath whatever directory is at its path now, with no check against the one a store-backed run of it last agreed with. - **`sync`:** passes over it, listing it under `skipped`. `materialize` refuses it, as it refuses any read-only mount. - - **A mount the store holds changes on that disk lacks** (a draft, a workspace deletion, a conflict, on the branch selected at start) runs store-backed that session, so they stay in view, and the workspace says so with an ops alert (`workspace-mount-not-direct`) naming them and the ways to start it direct: finish them, or give them up with a sync of each path. A mount that ran direct and runs store-backed again takes in what disk changed meanwhile. - - **Tool descriptions:** `ls`, `grep` and `sync` gain a sentence about the direct kind only on a host with a mount that runs direct; elsewhere they're unchanged. + - **A mount the store holds changes on that disk lacks** (a draft, a workspace deletion, a conflict, or an intent it can't read, on the branch selected at a start) runs store-backed from that start, so they stay in view. It runs unwatched, and with the direct default ignore list where its config names none. The workspace says so on stderr and in one ops alert (`workspace-mount-not-direct`) for every such mount, naming the changes and the two ways to start it direct: give each change up with a sync of its path, or materialize it with the mount configured `'chronicle'` and read-write, as materialize's answer directs. The first start that finds none starts it direct. A mount that ran direct and runs store-backed again takes in what disk changed meanwhile. + - **Tool descriptions:** `ls`, `glob`, `grep` and `sync` gain a sentence about the direct kind only on a host with a mount that runs direct; elsewhere they're unchanged. diff --git a/changelog.d/workspace-path-one-spelling.changed.md b/changelog.d/workspace-path-one-spelling.changed.md new file mode 100644 index 000000000..8a3a78053 --- /dev/null +++ b/changelog.d/workspace-path-one-spelling.changed.md @@ -0,0 +1 @@ +- **One spelling per workspace path, on every mount:** a path with `.` or empty segments, a trailing slash, or a `..` that stays inside the mount now reaches the file it names, and answers spell it without them. Before, a store-backed mount refused a `.` or `..` segment with the store's "Invalid tree path", refused an empty segment right after the mount's name as traversal, answered a file's path with a trailing slash with "File not found", and took any other empty segment as given, so a search whose path ended in a slash answered with `//` in each file's path. A `..` is read lexically, as a shell's `cd` reads `link/..`, and a path that leaves the mount is still refused. diff --git a/src/modules/workspace/index.ts b/src/modules/workspace/index.ts index d03110ec7..dd3bba00a 100644 --- a/src/modules/workspace/index.ts +++ b/src/modules/workspace/index.ts @@ -851,7 +851,7 @@ export class WorkspaceModule implements Module { private savedState: WorkspaceModuleState | null = null; /** * The mounts configured direct that run as 'chronicle' this session, with - * the changes the store held there that disk lacked (initStore), for + * the changes the store held there that disk lacked (chooseKinds), for * ensureRunning to say once both halves of the lifecycle have run. */ private notDirect = new Map>(); @@ -958,17 +958,33 @@ export class WorkspaceModule implements Module { for (const [name, intent] of intents) this.intents.set(name, intent); // What disk last agreed with, per mount and path: global records, so a - // branch switch never rewinds it. + // branch switch never rewinds it. A mount configured direct has its + // evidence read only where a 'chronicle' run left its states, for what + // the store holds there that disk lacks (chooseKinds); one that never ran + // so has none to read, and none is begun for it. this.agreement = new DiskAgreement(store, { modesObserved: () => this.modesObserved }); - this.agreement.open(this.config.mounts.map((m) => ({ name: m.name, root: resolve(m.path) }))); + this.agreement.open(this.config.mounts.filter((m) => intents.has(m.name)).map((m) => ({ name: m.name, root: resolve(m.path) }))); - // A mount configured direct runs as 'chronicle' this session while the - // store holds changes there that disk lacks, so they stay in view - // (MountConfig.backend); ensureRunning says so. A path held more than - // one way is named for the last way set here. + this.chooseKinds(); + this.applySavedState(); + this.ensureRunning(); + } + + /** + * Whether each mount configured direct runs direct: not while the store + * holds changes there that disk lacks, on the branch selected then, so + * they stay in view (MountConfig.backend); it runs as 'chronicle' instead, + * and ensureRunning says so. Decided when initStore gives the store, and + * again at each start, which knows how the mounts nest, so a start that + * finds none held starts the mount direct. A path held more than one way + * is named for the last way set here. + */ + private chooseKinds(): void { this.notDirect = new Map(); - for (const [name, state] of mounts) { - if (!state.direct || !this.intents.has(name)) continue; + for (const [name, state] of this.mounts) { + if (state.config.backend !== 'direct') continue; + state.direct = true; + if (!this.intents.has(name)) continue; const held = this.heldChanges(state); const what = new Map(); for (const path of held.owed) what.set(path, 'changed in the workspace'); @@ -979,9 +995,6 @@ export class WorkspaceModule implements Module { state.direct = false; this.notDirect.set(name, [...what].sort(([a], [b]) => a.localeCompare(b)).map(([path, label]) => ({ path, what: label }))); } - - this.applySavedState(); - this.ensureRunning(); } // ========================================================================== @@ -1027,9 +1040,14 @@ export class WorkspaceModule implements Module { }; } - /** A mount's ignore list as it runs: its config's, or, for a direct mount that names none, the direct kind's default (MountConfig.ignore). */ + /** + * A mount's ignore list: its config's, or, for a mount configured direct + * that names none, the direct kind's default (MountConfig.ignore), whichever + * kind it runs as this session, as its watch mode goes by its config + * (watchOf). + */ private ignoreOf(mount: MountState): string[] { - return mount.config.ignore ?? (mount.direct ? [...DIRECT_MOUNT_IGNORE] : []); + return mount.config.ignore ?? (mount.config.backend === 'direct' ? [...DIRECT_MOUNT_IGNORE] : []); } /** @@ -1278,11 +1296,14 @@ export class WorkspaceModule implements Module { /** * Start chokidar watchers and emit workspace:mounted events. * Idempotent: safe to call from both start() and initStore(), fires once - * per mount regardless of which runs second. + * per mount regardless of which runs second. Once both have run, it + * decides again which mounts configured direct run direct, now that the + * mounts' nesting is known, and says which don't, once for this start. */ private ensureRunning(): void { if (!this.ctx || this.mounts.size === 0) return; - for (const [name, held] of this.notDirect) this.sayNotDirect(name, held); + this.chooseKinds(); + if (this.notDirect.size > 0) this.sayNotDirect(); for (const [name, mount] of this.mounts) { if (this.watchers.has(name)) continue; @@ -1372,21 +1393,30 @@ export class WorkspaceModule implements Module { } /** - * Say, for whoever runs the host, that a mount configured direct runs as - * 'chronicle' this session (initStore), which changes the store holds there - * that disk lacks are why, and the two ways to start it direct: as an ops - * alert (the framework's failures log, its `ops:alert` trace and, where - * one is set, its webhook), and on stderr. + * Say, for whoever runs the host, which mounts configured direct run as + * 'chronicle' this session (chooseKinds), which changes the store holds + * there that disk lacks are why, and the two ways to start one direct: on + * stderr, and as one ops alert for them all (the framework's failures log, + * its `ops:alert` trace and, where one is set, its webhook, which posts + * one alert of a kind per cooldown and only the start and end of a long + * one). So the ways out come before the list of what's held. */ - private sayNotDirect(name: string, held: Array<{ path: string; what: string }>): void { - const shown = held.slice(0, 10).map((h) => `"${name}/${h.path}" (${h.what})`).join(', '); - const more = held.length > 10 ? `, and ${held.length - 10} more` : ''; - const message = `Mount "${name}" runs as 'chronicle' this session, though its config makes it direct: ` + - `the workspace holds changes there that disk lacks, which a direct mount wouldn't show: ${shown}${more}. ` + - 'To start it direct, finish them (with the mount configured \'chronicle\' and read-write, materialize them) ' + - 'or give them up (sync each path, which takes disk\'s copy); the first start that finds none starts it direct.'; + private sayNotDirect(): void { + const mounts = [...this.notDirect]; + const names = mounts.map(([name]) => `"${name}"`); + const named = names.length === 1 ? names[0] : `${names.slice(0, -1).join(', ')} and ${names[names.length - 1]}`; + const changes = (n: number) => `${n} change${n === 1 ? '' : 's'}`; + const why = mounts.length === 1 + ? `Mount ${named} runs as 'chronicle' this session, not direct: the workspace holds ${changes(mounts[0]![1].length)} there that disk lacks.` + : `Mounts ${named} run as 'chronicle' this session, not direct: the workspace holds changes there that disk lacks (` + + `${mounts.map(([name, held]) => `${held.length} in "${name}"`).join(', ')}).`; + const held = mounts.flatMap(([name, paths]) => paths.map((h) => `"${name}/${h.path}" (${h.what})`)); + const message = `${why} To start ${mounts.length === 1 ? 'it' : 'one'} direct, settle each: sync its path to take disk's copy, ` + + "or materialize it with the mount configured 'chronicle' and read-write, as materialize's answer directs. " + + `The first start that finds none starts it direct. Held: ${held.slice(0, 10).join(', ')}` + + `${held.length > 10 ? `, and ${held.length - 10} more` : ''}.`; console.warn(`[workspace] ${message}`); - this.ctx?.notifyOps?.('workspace-mount-not-direct', 'framework', message, { mount: name, held: held.length }); + this.ctx?.notifyOps?.('workspace-mount-not-direct', 'framework', message, { held: Object.fromEntries(mounts.map(([name, paths]) => [name, paths.length])) }); } private async initialScan(mountName: string): Promise { @@ -1470,7 +1500,7 @@ export class WorkspaceModule implements Module { // ========================================================================== getTools(): ToolDefinition[] { - // Where a mount runs direct, three descriptions say what that changes. + // Where a mount runs direct, four descriptions say what that changes. // A host with none keeps them as they were, and its residents' caches. const direct = [...this.mounts.values()].some((m) => m.direct); const lsDirect = direct @@ -1481,6 +1511,7 @@ export class WorkspaceModule implements Module { ? ' On a direct mount, grep searches disk\'s files as they are now (version disk), and lists as skipped, with why, ' + 'those it doesn\'t: binary, over the size limit, or not read, such as a link the mount doesn\'t follow.' : ''; + const globDirect = direct ? ' On a direct mount, a match carries its size, or a note where the mount doesn\'t take it, and no state.' : ''; const syncDirect = direct ? ' A direct mount is listed under `skipped`: its tools read disk as it is, so there\'s nothing to take in.' : ''; return [ { @@ -1573,7 +1604,7 @@ export class WorkspaceModule implements Module { }, { name: 'glob', - description: 'Find files matching a glob pattern, checked against disk first; each match carries its state, as ls describes.', + description: 'Find files matching a glob pattern, checked against disk first; each match carries its state, as ls describes.' + globDirect, inputSchema: { type: 'object' as const, properties: { @@ -2262,7 +2293,7 @@ export class WorkspaceModule implements Module { return { mount: given.mount, relativePath, - path: relativePath === given.relativePath ? path : `${given.mount.config.name}/${relativePath}`, + path: relativePath === given.relativePath ? given.path : `${given.mount.config.name}/${relativePath}`, }; } @@ -2290,9 +2321,13 @@ export class WorkspaceModule implements Module { } /** - * Parse a mount-prefixed path into (mountName, relativePath), as spelled. + * Parse a mount-prefixed path into its mount and mount-relative path, with + * its `.` and empty segments and a trailing slash dropped and each `..` + * taken lexically, and `path` the mount-prefixed path so spelled: the path + * as given where nothing was dropped, so each file has one spelling. A + * path that leaves the mount is refused. */ - private parsePathAsGiven(path: string): { mount: MountState; relativePath: string } { + private parsePathAsGiven(path: string): { mount: MountState; relativePath: string; path: string } { // A file's name on disk is UTF-8, which can't hold a lone surrogate: the // store would keep one name and disk another. if (LONE_SURROGATE.test(path)) { @@ -2300,7 +2335,7 @@ export class WorkspaceModule implements Module { } const slashIdx = path.indexOf('/'); const mountName = slashIdx >= 0 ? path.slice(0, slashIdx) : path; - const relativePath = slashIdx >= 0 ? path.slice(slashIdx + 1) : ''; + const asGiven = slashIdx >= 0 ? path.slice(slashIdx + 1) : ''; const mount = this.mounts.get(mountName); if (!mount) { @@ -2313,13 +2348,18 @@ export class WorkspaceModule implements Module { // prefix-matches there and every in-mount path was rejected as traversal. // Against the resolved root, so a root configured with a trailing // separator, or relative, contains its own paths, the root included. + // The path the tools take is the one this resolves to, so each file has + // one spelling: `.` and empty segments are dropped, a trailing slash + // with them, and a `..` takes back the segment before it, lexically, as + // a shell's `cd` reads `link/..`. const root = resolve(mount.config.path); - const resolved = resolve(root, relativePath); + const resolved = resolve(root, asGiven.split('/').filter((segment) => segment !== '' && segment !== '.').join('/')); if (!isContainedPath(root, resolved)) { throw new Error(`Path traversal detected: "${path}" resolves outside mount "${mountName}"`); } + const relativePath = resolved === root ? '' : relative(root, resolved).split(sep).join('/'); - return { mount, relativePath }; + return { mount, relativePath, path: relativePath === asGiven ? path : relativePath ? `${mountName}/${relativePath}` : mountName }; } /** @@ -2424,11 +2464,13 @@ export class WorkspaceModule implements Module { * with the code and errno readRoot gives what is at its path now (its stage * and words are this check's): gone, or not a directory, with ENOENT or * ENOTDIR, so a caller reads a root gone mid-read as one gone before it; or - * another directory, with none. + * another directory, with none. A direct mount reads whatever directory is + * at its root (MountConfig.backend), so for it, another directory there is + * one the next read reads. */ private async rootMoved(mount: MountState): Promise { const errno = await rootAbsence(resolve(mount.config.path)); - const reason = errno === undefined ? ROOT_REPLACED : ROOT_UNAVAILABLE; + const reason = errno !== undefined ? ROOT_UNAVAILABLE : mount.direct ? DIRECT_ROOT_REPLACED : ROOT_REPLACED; return new WorkspaceReadError('mount_unavailable', `Mount unavailable: ${mount.config.name}: ${reason}`, 'root', errno); } @@ -3007,7 +3049,10 @@ export class WorkspaceModule implements Module { if (fileStat.size > maxSize) throw new Error(over); }); } catch (err) { - return { error: err instanceof Error ? err.message : String(err) }; + // A refusal that names only the path says why with its errno, as a + // failed read below does. + const bare = err instanceof WorkspaceReadError && err.errno !== undefined && err.message === `Unable to read file: ${path}`; + return { error: bare ? `${err.message} (${err.errno})` : err instanceof Error ? err.message : String(err) }; } try { // No more than the limit is loaded, as for a file over it at its fstat. @@ -4500,20 +4545,24 @@ interface Listing { /** * Run `work` on each item, at most `width` at once, and hand each result to * `each` in the items' order, as soon as it and those before it are done. - * A `work` that rejects rejects this when its turn comes; any started after - * it finish unheard. Exported for its tests. + * A result handed on is let go, so what's held at once is at most `width` + * results, as a direct grep's file contents are. A `work` that rejects + * rejects this when its turn comes; any started after it finish unheard. + * Exported for its tests. */ export async function inOrder(items: readonly T[], width: number, work: (item: T) => Promise, each: (result: R, item: T) => void): Promise { - const started: Array> = []; + const waiting: Array> = []; + let started = 0; for (let i = 0; i < items.length; i++) { - while (started.length < items.length && started.length < i + width) { - const running = work(items[started.length]!); + while (started < items.length && started < i + width) { + const running = work(items[started]!); // Handled from the start, so a rejection that comes before its turn // isn't an unhandled one; awaiting it below still throws it. running.catch(() => {}); - started.push(running); + waiting.push(running); + started++; } - each(await started[i]!, items[i]!); + each(await waiting.shift()!, items[i]!); } } @@ -4553,6 +4602,9 @@ function stateView(report: PathReport): Record { /** A lone UTF-16 surrogate, which UTF-8 can't store: in `u` mode, a pair is one code point. */ const LONE_SURROGATE = /\p{Cs}/u; +/** Why a direct mount's read stopped when another directory was put at its root as it read. */ +const DIRECT_ROOT_REPLACED = 'the mount root was replaced while the file was read: reading it again reads the directory there now'; + /** * A mount's watch mode as it runs. A mount configured direct watches * nothing, whichever kind it runs as this session: its config takes no @@ -4574,7 +4626,9 @@ function shownValue(value: unknown): string { */ function directRefusals(mount: MountConfig): string[] { const refused: string[] = []; - if (mount.mode !== 'read-only') refused.push(`mode must be 'read-only', not ${shownValue(mount.mode)}`); + if (mount.mode !== 'read-only') { + refused.push(mount.mode === undefined ? "mode must be 'read-only' (left out, it's 'read-write')" : `mode must be 'read-only', not ${shownValue(mount.mode)}`); + } if (mount.watch !== undefined && mount.watch !== 'never') refused.push(`watch ${shownValue(mount.watch)} would watch nothing (leave it out, or set 'never')`); for (const key of ['watchDebounceMs', 'watchRootPollMs'] as const) { if (mount[key] !== undefined) refused.push(`${key} would tune no watcher (leave it out)`); diff --git a/src/modules/workspace/observe.ts b/src/modules/workspace/observe.ts index 5147d08cc..eb68e1bd1 100644 --- a/src/modules/workspace/observe.ts +++ b/src/modules/workspace/observe.ts @@ -1264,7 +1264,8 @@ export interface BytesResult { * A mount-relative file's bytes, through the same located descriptor as * readPath, for a direct mount's tools, which keep nothing: at most `limit` * of them, and none hashed. A file over `limit` at its fstat has only its - * head read, and one that grew past it since is read no further than it. + * head read, and one that grew past it since is read no further than it. A + * link to a directory gets the reason a listing gives it. */ export async function readBytes( view: MountView, @@ -1273,7 +1274,15 @@ export async function readBytes( limit: number, ): Promise> { const opened = await openContained(view, rootReal, relativePath); - if (!('handle' in opened)) return opened; + if (!('handle' in opened)) { + // A walk takes a link to a directory as an entry, where the mount follows + // links: it's named as a listing names it (observePath). + if (opened.kind === 'unobserved' && opened.reason === 'a directory') { + const seen = await observePath(view, rootReal, relativePath); + return seen.kind === 'file' ? opened : seen; + } + return opened; + } const { handle, info } = opened; try { const size = Number(info.size); diff --git a/src/modules/workspace/types.ts b/src/modules/workspace/types.ts index 340cfad42..3a4cdc0e1 100644 --- a/src/modules/workspace/types.ts +++ b/src/modules/workspace/types.ts @@ -20,7 +20,9 @@ export interface MountConfig { * Where the mount's tools get what they read (default: 'chronicle'). * - 'chronicle': the workspace keeps the mount's files in its store, where * writes and edits land first, and takes in disk's changes under the - * three-way rule as its listings, reads and syncs reach them. + * three-way rule as its listings, searches, syncs and scans reach them. + * A read of a path the store doesn't hold takes in a new file there; + * otherwise it reads the store's copy. * - 'direct': for a checkout an agent reads, not one it changes. Every * read, listing and search is disk's own copy as it is when it runs, * and none of its files go into the store. No scan or watcher @@ -34,13 +36,16 @@ export interface MountConfig { * tools read beneath whatever directory is at its root, with no check * against the one a 'chronicle' run of it last agreed with. * - * A mount configured 'direct' that the store holds changes on that disk - * lacks, on the branch selected when the store is given (a draft, a - * workspace deletion, a conflict, as `status` counts them pending), runs - * as 'chronicle' that session, so they stay in view; the workspace says - * so through the framework's ops alerts, naming them and the ways to - * start it direct. A mount that ran direct and runs as 'chronicle' again - * takes in what disk changed meanwhile, as after any time away. + * A mount configured 'direct' runs as 'chronicle' from a start that finds + * the store holding changes there that disk lacks, on the branch selected + * then: a draft, a workspace deletion or a conflict, as `status` counts + * them pending, or an intent the workspace can't read, which may hold one + * of those. They stay in view, and the workspace says so on stderr and + * through the framework's ops alerts, naming them and the ways to start it + * direct. It runs unwatched, and walks by the direct kind's default + * ignore list where its config names none. A mount that ran direct and + * runs as 'chronicle' again takes in what disk changed meanwhile, as + * after any time away. */ backend?: 'chronicle' | 'direct'; /** @@ -71,14 +76,15 @@ export interface MountConfig { * (the whole name), and names ending in `~` or in `.subl…tmp`, matched by * `^\..+\.sw[px]$`, `~$` and `\.subl.*\.tmp$`. * - * A direct mount (`backend`) whose config names no list takes + * A mount configured direct (`backend`) whose config names no list takes * `node_modules`, `dist`, `.git`, `build`, `coverage`, `target`, - * `__pycache__` and `.venv`, each matching that name at any depth. Its - * walks of the whole mount, and of any path no pattern covers, pass over - * what the list covers. A listing, glob or grep whose path, or whose - * pattern's directory, the list covers is walked all the same, the - * patterns covering it set aside and the rest applying beneath it, as a - * sync that names a path walks it on a 'chronicle' mount. + * `__pycache__` and `.venv`, each matching that name at any depth, + * whichever kind it runs as. A direct mount's walks of the whole mount, + * and of any path no pattern covers, pass over what the list covers. A + * listing, glob or grep whose path, or whose pattern's directory, the + * list covers is walked all the same, the patterns covering it set aside + * and the rest applying beneath it, as a sync that names a path walks it + * on a 'chronicle' mount. */ ignore?: string[]; /** diff --git a/test/workspace-direct.test.ts b/test/workspace-direct.test.ts index 9e725b84a..3929ff7e2 100644 --- a/test/workspace-direct.test.ts +++ b/test/workspace-direct.test.ts @@ -7,15 +7,18 @@ import test, { type TestContext } from 'node:test'; import assert from 'node:assert/strict'; -import { appendFileSync, chmodSync, mkdirSync, mkdtempSync, renameSync, rmSync, symlinkSync, unlinkSync, writeFileSync } from 'node:fs'; +import { appendFileSync, chmodSync, mkdirSync, mkdtempSync, renameSync, rmSync, statSync, symlinkSync, truncateSync, unlinkSync, writeFileSync } from 'node:fs'; import { createRequire, syncBuiltinESMExports } from 'node:module'; import { tmpdir } from 'node:os'; import { dirname, join } from 'node:path'; +import { setFlagsFromString } from 'node:v8'; +import { runInNewContext } from 'node:vm'; import { JsStore } from '@animalabs/chronicle'; // A namespace, so this file loads where the module lacks inOrder, and only // the tests that call it fail there. import * as workspaceModule from '../src/modules/workspace/index.js'; import { WorkspaceModule } from '../src/modules/workspace/index.js'; +import { BranchIntents } from '../src/modules/workspace/branch-intent.js'; import type { MountConfig } from '../src/modules/workspace/types.js'; import type { ModuleContext } from '../src/types/module.js'; import type { ProcessEvent, ToolResult } from '../src/types/events.js'; @@ -61,15 +64,26 @@ class Env { mounts: [{ name: 'ref', path: this.ref, mode: 'read-only', backend: 'direct', ...mount }, ...more], }); m.initStore(this.store); - await m.start({ + await m.start(this.context()); + this.live.push(m); + return m; + } + + /** A module context as a host gives one, recording events and ops alerts. */ + context(): ModuleContext { + return { isRestart: false, getState: () => null, setState: () => {}, pushEvent: (e: ProcessEvent) => this.events.push(e), notifyOps: (kind: string, agent: string, message: string, data?: Record) => this.alerts.push({ kind, agent, message, data }), - } as unknown as ModuleContext); - this.live.push(m); - return m; + } as unknown as ModuleContext; + } + + /** Stop the module and start it again, with the store it has. */ + async restart(m: WorkspaceModule): Promise { + await m.stop(); + await m.start(this.context()); } /** The same mount as a 'chronicle' one, read-write and unwatched. */ @@ -92,20 +106,37 @@ function watched(m: WorkspaceModule): string[] { return [...(m as unknown as { watchers: Map }).watchers.keys()]; } -/** Run `fn` with `node:fs/promises`'s `lstat`, as the module's imports see it, replaced by what `make` makes of it. */ -async function withLstat(make: (original: (...args: any[]) => Promise) => (...args: any[]) => Promise, fn: () => Promise): Promise { +/** Run `fn` with `node:fs/promises`'s `name`, as the module's imports see it, replaced by what `make` makes of it. */ +async function withFsPromise(name: 'lstat' | 'open', make: (original: (...args: any[]) => Promise) => (...args: any[]) => Promise, fn: () => Promise): Promise { const fsp = createRequire(import.meta.url)('node:fs/promises') as Record Promise>; - const original = fsp.lstat!; - fsp.lstat = make(original); + const original = fsp[name]!; + fsp[name] = make(original); syncBuiltinESMExports(); try { return await fn(); } finally { - fsp.lstat = original; + fsp[name] = original; syncBuiltinESMExports(); } } +/** The most calls of `node:fs/promises`'s `name` under way at once while `fn` runs, each held a turn so the others can start. */ +async function mostAtOnce(name: 'lstat' | 'open', fn: () => Promise): Promise { + let now = 0; + let most = 0; + await withFsPromise(name, (original) => async (...args: any[]) => { + now++; + most = Math.max(most, now); + try { + await new Promise((resolve) => setImmediate(resolve)); + return await original(...args); + } finally { + now--; + } + }, fn); + return most; +} + function call(m: WorkspaceModule, name: string, input: Record = {}): Promise { return m.handleToolCall({ id: name, name, input }); } @@ -122,13 +153,13 @@ async function failure(m: WorkspaceModule, name: string, input: Record { +test('a direct mount reads disk as it is now, and records nothing in the store from its start on', async (t) => { const env = new Env(t); env.file('src/a.ts', 'one\n'); env.file('img/dot.png', PNG); - const m = await env.open(); const states = env.stateIds(); const sequence = env.store.currentSequence(); + const m = await env.open(); assert.equal((await data<{ content: string }>(m, 'read', { path: 'ref/src/a.ts' })).content, ' 1\tone'); appendFileSync(join(env.ref, 'src/a.ts'), 'two\n'); @@ -141,10 +172,10 @@ test('a direct mount reads disk as it is now, and its tools record nothing in th await data(m, 'status'); await data(m, 'sync'); assert.deepEqual('data' in (await m.readBinary('ref/src/a.ts')), true); + await m.onToolBatchComplete(); assert.deepEqual(env.stateIds(), states, 'no state was made'); - assert.ok(!states.some((id) => id.startsWith('workspace/ref/')), `the mount's states: ${states}`); - assert.equal(env.store.currentSequence(), sequence, 'nothing was recorded'); + assert.equal(env.store.currentSequence(), sequence, 'nothing was recorded, its disk evidence included'); assert.deepEqual(env.events.filter((e) => e.type !== 'workspace:mounted').map((e) => e.type), [], 'no change was evented'); assert.deepEqual(watched(m), [], 'no watcher runs, though watch is left out'); }); @@ -242,12 +273,13 @@ test('the default ignore list keeps whole walks out of what it covers, and a pat env.file('node_modules/pkg/dist/built.js', 'needle\n'); env.file('dist/out.js', 'needle\n'); const m = await env.open(); - assert.deepEqual((await data(m, 'ls', { path: 'ref' })).entries, [{ name: 'src', type: 'directory' }]); - assert.deepEqual((await data<{ matches: unknown[] }>(m, 'glob', { pattern: '**/*.js' })).matches, [{ path: 'ref/src/a.js', size: 7 }]); - assert.deepEqual( - (await data<{ results: Array<{ file: string }> }>(m, 'grep', { pattern: 'needle' })).results.map((r) => r.file), - ['ref/src/a.js'], - ); + // A whole walk passes over what the list covers, and doesn't count it as + // a region it couldn't reach. + assert.deepEqual(await data(m, 'ls', { path: 'ref' }), { path: 'ref', entries: [{ name: 'src', type: 'directory' }], count: 1 }); + assert.deepEqual(await data(m, 'glob', { pattern: '**/*.js' }), { pattern: '**/*.js', matches: [{ path: 'ref/src/a.js', size: 7 }], count: 1 }); + const grep = await data<{ results: Array<{ file: string }>; incomplete?: unknown }>(m, 'grep', { pattern: 'needle' }); + assert.deepEqual(grep.results.map((r) => r.file), ['ref/src/a.js']); + assert.equal(grep.incomplete, undefined); // Named, by a path or a pattern's directory: walked, the rest of the list // applying beneath it. assert.deepEqual((await data(m, 'ls', { path: 'ref/node_modules/pkg' })).entries, [{ name: 'index.js', type: 'file', size: 7 }]); @@ -338,7 +370,7 @@ test("a direct listing names a directory it couldn't list", { skip: IS_ROOT ? 'r } }); -test("a direct mount's read and grep load no more than the size limit, and refuse a file that grows past it as it's read", async (t) => { +test("a direct mount's read loads none of a file over the size limit and grep only its head, and both refuse one that grows past it as it's read", async (t) => { const env = new Env(t); env.file('grows.txt', 'x'.repeat(8)); env.file('huge.txt', 'y'.repeat(100_000)); @@ -377,6 +409,22 @@ test("a direct mount's read and grep say so where reading a file fails", async ( }); }); +test("read on a direct mount says why it couldn't open a file", { skip: IS_ROOT ? 'root opens every file' : false }, async (t) => { + const env = new Env(t); + env.file('noperm.txt', 'secret\n'); + env.file('shut/b.ts', 'b\n'); + const m = await env.open(); + chmodSync(join(env.ref, 'noperm.txt'), 0o000); + chmodSync(join(env.ref, 'shut'), 0o000); + try { + assert.equal(await failure(m, 'read', { path: 'ref/noperm.txt' }), 'Unable to read file: ref/noperm.txt (EACCES)'); + assert.equal(await failure(m, 'read', { path: 'ref/shut/b.ts' }), 'Unable to read file: ref/shut/b.ts (EACCES)'); + } finally { + chmodSync(join(env.ref, 'noperm.txt'), 0o644); + chmodSync(join(env.ref, 'shut'), 0o755); + } +}); + test('a file gone between the walk that listed it and its read is neither listed nor searched', async (t) => { const env = new Env(t); const names = Array.from({ length: 40 }, (_, i) => `f${String(i).padStart(2, '0')}.txt`); @@ -392,7 +440,7 @@ test('a file gone between the walk that listed it and its read is neither listed assert.equal(grep.skipped, undefined); // A listing looks at each file by its path, where this one is removed first. env.file('f30.txt', 'needle\n'); - const listed = await withLstat((lstat) => async (...args: any[]) => { + const listed = await withFsPromise('lstat', (lstat) => async (...args: any[]) => { if (args[0] === gone) rmSync(gone, { force: true }); return lstat(...args); }, () => data<{ entries: Array<{ path: string }> }>(m, 'ls', { path: 'ref', recursive: true })); @@ -420,13 +468,129 @@ test('grep on a direct mount lists what it doesn\'t search, with why', async (t) }); }); +test('grep on a direct mount that follows links names a link to a directory as its listing does', async (t) => { + const env = new Env(t); + env.file('dir/x.ts', 'needle\n'); + symlinkSync(join(env.ref, 'dir'), join(env.ref, 'linkdir')); + const m = await env.open({ followSymlinks: true }); + const listed = await data<{ entries: Array<{ path: string }> }>(m, 'ls', { path: 'ref', recursive: true }); + assert.deepEqual(listed.entries.find((e) => e.path === 'linkdir'), { path: 'linkdir', note: 'a symlink to a directory, not walked' }); + assert.deepEqual((await data<{ skipped?: unknown[] }>(m, 'grep', { pattern: 'needle' })).skipped, [ + { file: 'ref/linkdir', reason: 'a symlink to a directory, not walked' }, + ]); +}); + test("grep's results come in path order, however its reads finish", async (t) => { const env = new Env(t); const names = Array.from({ length: 40 }, (_, i) => `f${String(i).padStart(2, '0')}.txt`); for (const name of [...names].reverse()) env.file(`many/${name}`, `${name} needle\n`.repeat(1 + (names.indexOf(name) % 7) * 500)); const m = await env.open(); - const files = (await data<{ results: Array<{ file: string }> }>(m, 'grep', { pattern: 'needle' })).results.map((r) => r.file); - assert.deepEqual(files, names.map((name) => `ref/many/${name}`)); + // The first file's reads are held back, so it finishes after the rest. + const first = String(statSync(join(env.ref, 'many', names[0]!), { bigint: true }).ino); + const slow = new WeakSet(); + const result = await withFileHandle({ + stat: (stat) => async function (this: object, ...args: unknown[]) { + const seen = await stat.apply(this, args); + if (String(seen.ino) === first) slow.add(this); + return seen; + }, + read: (read) => async function (this: object, ...args: unknown[]) { + if (slow.has(this)) await new Promise((resolve) => setTimeout(resolve, 50)); + return read.apply(this, args); + }, + }, () => data<{ results: Array<{ file: string }> }>(m, 'grep', { pattern: 'needle' })); + assert.deepEqual(result.results.map((r) => r.file), names.map((name) => `ref/many/${name}`)); +}); + +test('a direct grep and listing read 16 files at a time', async (t) => { + const env = new Env(t); + for (let i = 0; i < 40; i++) env.file(`f${String(i).padStart(2, '0')}.txt`, 'needle\n'); + const m = await env.open(); + assert.equal(await mostAtOnce('open', () => data(m, 'grep', { pattern: 'needle' })), 16); + assert.equal(await mostAtOnce('lstat', () => data(m, 'ls', { path: 'ref', recursive: true })), 16); +}); + +test("a direct mount's reads close every file they open, whatever they answer", async (t) => { + const env = new Env(t); + env.file('a.txt', 'needle\n'); + env.file('bin.dat', Buffer.from([0x61, 0x00, 0x62])); + env.file('big.txt', 'x'.repeat(100)); + env.file('dot.png', PNG); + const m = await env.open({ maxFileSize: 90 }); + // Each handle a read opens is fstat'd; a closed one's fd reads -1. + const opened = new Set<{ fd: number }>(); + await withFileHandle({ + stat: (stat) => async function (this: { fd: number }, ...args: unknown[]) { + opened.add(this); + return stat.apply(this, args); + }, + }, async () => { + for (const path of ['ref/a.txt', 'ref/bin.dat', 'ref/big.txt']) await call(m, 'read', { path }); + await call(m, 'grep', { pattern: 'needle' }); + await call(m, 'read_image', { path: 'ref/dot.png' }); + await m.readBinary('ref/bin.dat'); + }); + assert.ok(opened.size >= 9, `opened ${opened.size}`); + assert.deepEqual([...opened].map((handle) => handle.fd).filter((fd) => fd !== -1), []); +}); + +test("grep gives a skipped file's size as its reads found it, when the file changes as it's read", async (t) => { + const env = new Env(t); + env.file('grows.txt', 'x'.repeat(8)); + env.file('bin.dat', Buffer.from('a\0bcdefg')); + const m = await env.open({ maxFileSize: 10 }); + // Grown past the limit after its open's fstat, then cut short before its + // size is read again: at least a byte past the limit, as its reads found. + const grows = join(env.ref, 'grows.txt'); + let stats = 0; + const grown = await withFileHandle({ + stat: (stat) => async function (this: object, ...args: unknown[]) { + stats++; + if (stats === 2) truncateSync(grows, 5); + const seen = await stat.apply(this, args); + if (stats === 1) appendFileSync(grows, 'x'.repeat(1000)); + return seen; + }, + }, () => data<{ skipped: unknown[] }>(m, 'grep', { pattern: 'x', path: 'ref/grows.txt' })); + assert.deepEqual(grown.skipped, [{ file: 'ref/grows.txt', reason: 'over the size limit (10 bytes)', size: 11 }]); + // Cut short after its fstat, within the limit: the bytes read. + const bin = join(env.ref, 'bin.dat'); + const shrunk = await changedAfterFstat(bin, () => truncateSync(bin, 4), () => data<{ skipped: unknown[] }>(m, 'grep', { pattern: 'b', path: 'ref/bin.dat' })); + assert.deepEqual(shrunk.skipped, [{ file: 'ref/bin.dat', reason: 'binary: a NUL in its first 8 KB', size: 4 }]); +}); + +test('a direct read whose root is replaced as it reads says to read again', async (t) => { + const env = new Env(t); + env.file('a.txt', 'a\n'); + const m = await env.open(); + const error = await changedAfterFstat(join(env.ref, 'a.txt'), () => { + renameSync(env.ref, `${env.ref}-before`); + mkdirSync(env.ref); + }, () => failure(m, 'read', { path: 'ref/a.txt' })); + assert.equal(error, 'Mount unavailable: ref: the mount root was replaced while the file was read: reading it again reads the directory there now'); + assert.equal(await failure(m, 'read', { path: 'ref/a.txt' }), 'File not found: ref/a.txt'); +}); + +test("every mount takes a path's '.' and empty segments, trailing slash and '..' as the file they name, and answers with one spelling", async (t) => { + const env = new Env(t); + env.file('a.ts', 'alpha\n'); + env.file('dir/x.ts', 'alpha x\n'); + symlinkSync(join(env.root, 'elsewhere'), join(env.ref, 'link')); + for (const m of [await env.open(), await env.openChronicle()]) { + const kind = m.getTools().some((tool) => /\bdirect\b/.test(tool.description)) ? 'direct' : 'chronicle'; + for (const path of ['ref/./a.ts', 'ref/dir/../a.ts', 'ref/a.ts/', 'ref//a.ts', 'ref/link/../a.ts', 'ref/../ref/a.ts']) { + assert.deepEqual(await data(m, 'read', { path }), { path: 'ref/a.ts', totalLines: 1, fromLine: 1, toLine: 1, content: ' 1\talpha' }, `${kind} ${path}`); + } + assert.equal((await data<{ path: string }>(m, 'ls', { path: 'ref/dir/' })).path, 'ref/dir', kind); + assert.equal((await data<{ path: string }>(m, 'ls', { path: 'ref/dir/..' })).path, 'ref', kind); + for (const path of ['ref/dir/', 'ref/./dir', 'ref//dir']) { + assert.deepEqual((await data<{ results: Array<{ file: string }> }>(m, 'grep', { pattern: 'alpha', path })).results.map((r) => r.file), ['ref/dir/x.ts'], `${kind} ${path}`); + assert.deepEqual((await data<{ matches: Array<{ path: string }> }>(m, 'glob', { pattern: '*.ts', path })).matches.map((x) => x.path), ['ref/dir/x.ts'], `${kind} ${path}`); + } + for (const path of ['ref/../outside.txt', 'ref/dir/../../outside.txt']) { + assert.match(await failure(m, 'read', { path }), /^Path traversal detected/, `${kind} ${path}`); + } + } }); test("ls without a path and status give a direct mount's kind and ignore list, and a 'chronicle' mount's as before", async (t) => { @@ -505,6 +669,7 @@ test('a direct mount whose config sets what would never act there is refused, na }; const lead = '[workspace] Mount "ref" is direct, which reads disk as it is with no store, watcher or push: '; assert.equal(refused({ mode: 'read-write' }), `${lead}mode must be 'read-only', not "read-write"`); + assert.equal(refused({ mode: undefined }), `${lead}mode must be 'read-only' (left out, it's 'read-write')`); assert.equal(refused({ watch: 'always' }), `${lead}watch "always" would watch nothing (leave it out, or set 'never')`); assert.equal(refused({ watch: 'on-agent-action' }), `${lead}watch "on-agent-action" would watch nothing (leave it out, or set 'never')`); assert.equal(refused({ watchDebounceMs: 100 }), `${lead}watchDebounceMs would tune no watcher (leave it out)`); @@ -530,13 +695,18 @@ test('tool descriptions mention the direct kind only on a host with a direct mou const descriptions = (m: WorkspaceModule) => Object.fromEntries(m.getTools().map((tool) => [tool.name, tool.description])); const chronicle = descriptions(await env.openChronicle()); for (const [name, description] of Object.entries(chronicle)) assert.doesNotMatch(description, /\bdirect\b/, name); - const direct = descriptions(await env.open()); - for (const name of Object.keys(chronicle)) { - if (name === 'ls' || name === 'grep' || name === 'sync') { - assert.ok(direct[name]!.startsWith(chronicle[name]!) && direct[name]!.length > chronicle[name]!.length, name); - assert.match(direct[name]!, /direct mount/, name); - } else { - assert.equal(direct[name], chronicle[name], name); + // A host with a writable 'chronicle' mount beside a direct one, as one + // coding in a checkout with its neighbours mounted to read. + const work = join(env.root, 'work'); + mkdirSync(work); + for (const direct of [descriptions(await env.open()), descriptions(await env.open({}, [{ name: 'work', path: work, mode: 'read-write', watch: 'never' }]))]) { + for (const name of Object.keys(chronicle)) { + if (name === 'ls' || name === 'glob' || name === 'grep' || name === 'sync') { + assert.ok(direct[name]!.startsWith(chronicle[name]!) && direct[name]!.length > chronicle[name]!.length, name); + assert.match(direct[name]!, /direct mount/, name); + } else { + assert.equal(direct[name], chronicle[name], name); + } } } }); @@ -546,40 +716,54 @@ test("a mount the store holds changes on that disk lacks runs as 'chronicle', sa env.file('drafted.ts', 'disk\n'); env.file('deleted.ts', 'kept on disk\n'); env.file('both.ts', 'v1\n'); + env.file('dot.img', GIF); const chronicle = await env.openChronicle(); await data(chronicle, 'ls', { path: 'ref' }); await data(chronicle, 'write', { path: 'ref/drafted.ts', content: 'draft\n' }); await data(chronicle, 'delete', { path: 'ref/deleted.ts' }); await data(chronicle, 'write', { path: 'ref/both.ts', content: 'v2\n' }); + assert.equal((await chronicle.writeBinary('ref/dot.img', PNG, 'image/png')).success, true); env.file('both.ts', 'v3\n'); assert.equal((await data<{ entries: Array<{ name: string; state: string }> }>(chronicle, 'ls', { path: 'ref' })).entries.find((e) => e.name === 'both.ts')!.state, 'conflict'); await env.close(chronicle); + const warned = t.mock.method(console, 'warn', () => {}); const held = await env.open({ walkCap: 1 }); - assert.deepEqual(env.alerts, [{ - kind: 'workspace-mount-not-direct', - agent: 'framework', - message: 'Mount "ref" runs as \'chronicle\' this session, though its config makes it direct: the workspace holds changes ' - + 'there that disk lacks, which a direct mount wouldn\'t show: "ref/both.ts" (in conflict), "ref/deleted.ts" (deleted in ' - + 'the workspace), "ref/drafted.ts" (changed in the workspace). To start it direct, finish them (with the mount configured ' - + '\'chronicle\' and read-write, materialize them) or give them up (sync each path, which takes disk\'s copy); the first ' - + 'start that finds none starts it direct.', - data: { mount: 'ref', held: 3 }, - }]); + warned.mock.restore(); + // The ways out come before the list of what's held, so a webhook that + // keeps only a long alert's start and end still gives them. + const message = 'Mount "ref" runs as \'chronicle\' this session, not direct: the workspace holds 4 changes there that disk lacks. ' + + 'To start it direct, settle each: sync its path to take disk\'s copy, or materialize it with the mount configured ' + + '\'chronicle\' and read-write, as materialize\'s answer directs. The first start that finds none starts it direct. ' + + 'Held: "ref/both.ts" (in conflict), "ref/deleted.ts" (deleted in the workspace), "ref/dot.img" (in conflict), ' + + '"ref/drafted.ts" (changed in the workspace).'; + assert.deepEqual(env.alerts, [{ kind: 'workspace-mount-not-direct', agent: 'framework', message, data: { held: { ref: 4 } } }]); + assert.deepEqual(warned.mock.calls.map((c) => c.arguments), [[`[workspace] ${message}`]]); // As a 'chronicle' mount, unwatched, with no catch-up scan to take turns // with its syncs, its changes in view. const status = await data>>(held, 'status'); assert.equal(status.ref!.backend, undefined); assert.equal(status.ref!.watch, 'never'); assert.deepEqual(watched(held), []); - assert.equal(status.ref!.pendingChanges, 3); + assert.equal(status.ref!.pendingChanges, 4); const capped = (await data<{ incomplete: Array<{ path: string; reason: string }> }>(held, 'sync')).incomplete.map((region) => region.reason); assert.ok(capped.some((reason) => /^the walk cap \(1\) was reached: the next sync of the mount resumes after ref\//.test(reason)), JSON.stringify(capped)); - assert.equal((await data<{ content: string }>(held, 'read', { path: 'ref/drafted.ts' })).content, ' 1\tdraft'); assert.ok(held.getTools().every((tool) => !/\bdirect\b/.test(tool.description))); - - for (const path of ['ref/drafted.ts', 'ref/deleted.ts', 'ref/both.ts']) await data(held, 'sync', { path }); await env.close(held); + + // Every tool reads the workspace's copy there, the draft and the image. + env.alerts = []; + const again = await env.open(); + assert.equal(env.alerts.length, 1, 'each start that finds them says so'); + assert.equal((await data<{ content: string }>(again, 'read', { path: 'ref/drafted.ts' })).content, ' 1\tdraft'); + assert.equal((await data<{ entries: Array<{ name: string; state?: string }> }>(again, 'ls', { path: 'ref' })).entries.find((e) => e.name === 'drafted.ts')?.state, 'workspace-draft'); + assert.deepEqual((await data<{ matches: Array<{ path: string; state?: string }> }>(again, 'glob', { pattern: 'drafted.ts' })).matches.map((x) => [x.path, x.state]), [['ref/drafted.ts', 'workspace-draft']]); + assert.deepEqual((await data<{ results: Array<{ file: string; version: string }> }>(again, 'grep', { pattern: 'draft' })).results.map((r) => [r.file, r.version]), [['ref/drafted.ts', 'workspace']]); + assert.equal((await data>(again, 'read_image', { path: 'ref/dot.img' }))[1]!.mimeType, 'image/png'); + assert.deepEqual(await again.readBinary('ref/dot.img'), { data: PNG }); + + for (const path of ['ref/drafted.ts', 'ref/deleted.ts', 'ref/both.ts', 'ref/dot.img']) await data(again, 'sync', { path }); + await env.close(again); env.alerts = []; const direct = await env.open(); assert.deepEqual(env.alerts, []); @@ -589,17 +773,121 @@ test("a mount the store holds changes on that disk lacks runs as 'chronicle', sa test("the changes that keep a mount from starting direct are named ten at a time, an intent that can't be read among them", async (t) => { const env = new Env(t); + env.file('d00.ts', 'disk\n'); const chronicle = await env.openChronicle(); + await data(chronicle, 'ls', { path: 'ref' }); for (let i = 0; i < 11; i++) await data(chronicle, 'write', { path: `ref/d${String(i).padStart(2, '0')}.ts`, content: `${i}\n` }); await env.close(chronicle); - // A recorded intent whose blob the store doesn't hold. - env.store.treeSet('workspace/ref/intent', 'aa.ts', { blobHash: 'f'.repeat(64), size: 2, mode: 0 }); + // Recorded intents whose blob the store doesn't hold: one alone, and one at + // a changed file, which is named for the intent. + for (const path of ['aa.ts', 'd00.ts']) env.store.treeSet('workspace/ref/intent', path, { blobHash: 'f'.repeat(64), size: 2, mode: 0 }); await env.open(); assert.equal(env.alerts.length, 1); - const shown = Array.from({ length: 9 }, (_, i) => `"ref/d0${i}.ts" (changed in the workspace)`).join(', '); - assert.match(env.alerts[0]!.message, new RegExp(`disk lacks, which a direct mount wouldn't show: "ref/aa\\.ts" \\(its workspace intent can't be read\\), ${shown.replace(/[.()]/g, '\\$&')}, and 2 more\\. To start it direct`)); - assert.deepEqual(env.alerts[0]!.data, { mount: 'ref', held: 12 }); + const unreadable = (path: string) => `"ref/${path}" (its workspace intent can't be read)`; + const shown = [unreadable('aa.ts'), unreadable('d00.ts'), ...Array.from({ length: 8 }, (_, i) => `"ref/d0${i + 1}.ts" (changed in the workspace)`)]; + assert.ok(env.alerts[0]!.message.endsWith(`Held: ${shown.join(', ')}, and 2 more.`), env.alerts[0]!.message); + assert.deepEqual(env.alerts[0]!.data, { held: { ref: 12 } }); +}); + +test('each kind of change the store holds keeps a mount configured direct from starting direct on its own', async (t) => { + const cases: Array<[string, (env: Env) => Promise, string]> = [ + ['a workspace deletion', async (env) => { + env.file('gone.ts', 'kept on disk\n'); + const chronicle = await env.openChronicle(); + await data(chronicle, 'ls', { path: 'ref' }); + await data(chronicle, 'delete', { path: 'ref/gone.ts' }); + await env.close(chronicle); + }, '"ref/gone.ts" (deleted in the workspace)'], + ['a conflict', async (env) => { + await env.close(await env.openChronicle()); + new BranchIntents(env.store, 'workspace/ref/intent').put('c.ts', { conflict: { kind: 'both-changed', at: 0, disk: null } }); + }, '"ref/c.ts" (in conflict)'], + ["an intent that can't be read", async (env) => { + await env.close(await env.openChronicle()); + env.store.treeSet('workspace/ref/intent', 'u.ts', { blobHash: 'f'.repeat(64), size: 2, mode: 0 }); + }, `"ref/u.ts" (its workspace intent can't be read)`], + ]; + for (const [kind, hold, named] of cases) { + const env = new Env(t); + await hold(env); + const m = await env.open(); + assert.equal(env.alerts.length, 1, kind); + assert.ok(env.alerts[0]!.message.endsWith(`Held: ${named}.`), env.alerts[0]!.message); + assert.equal((await data>>(m, 'status')).ref!.backend, undefined, kind); + } +}); + +test('mounts configured direct that the store holds changes on are named in one alert', async (t) => { + const env = new Env(t); + const other = join(env.root, 'other'); + mkdirSync(other); + const chronicle = await env.open({ backend: 'chronicle', mode: 'read-write', watch: 'never' }, [{ name: 'other', path: other, mode: 'read-write', watch: 'never' }]); + await data(chronicle, 'write', { path: 'ref/a.ts', content: 'a\n' }); + await data(chronicle, 'write', { path: 'other/b.ts', content: 'b\n' }); + await data(chronicle, 'write', { path: 'other/c.ts', content: 'c\n' }); + await env.close(chronicle); + + await env.open({}, [{ name: 'other', path: other, mode: 'read-only', backend: 'direct' }]); + assert.equal(env.alerts.length, 1); + assert.ok(env.alerts[0]!.message.startsWith('Mounts "ref" and "other" run as \'chronicle\' this session, not direct: the workspace holds ' + + 'changes there that disk lacks (1 in "ref", 2 in "other"). To start one direct, settle each:'), env.alerts[0]!.message); + assert.ok(env.alerts[0]!.message.endsWith('Held: "ref/a.ts" (changed in the workspace), "other/b.ts" (changed in the workspace), ' + + '"other/c.ts" (changed in the workspace).'), env.alerts[0]!.message); + assert.deepEqual(env.alerts[0]!.data, { held: { ref: 1, other: 2 } }); +}); + +test('a start of the same module decides again whether a mount configured direct runs direct', async (t) => { + const env = new Env(t); + env.file('drafted.ts', 'disk\n'); + const chronicle = await env.openChronicle(); + await data(chronicle, 'write', { path: 'ref/drafted.ts', content: 'draft\n' }); + await env.close(chronicle); + + const m = await env.open(); + assert.equal(env.alerts.length, 1); + await env.restart(m); + assert.equal(env.alerts.length, 2, 'a start that finds the change still held says so again'); + await data(m, 'sync', { path: 'ref/drafted.ts' }); + await env.restart(m); + assert.equal(env.alerts.length, 2, 'the start that finds none says nothing'); + assert.equal((await data>>(m, 'status')).ref!.backend, 'direct'); + assert.equal((await data<{ content: string }>(m, 'read', { path: 'ref/drafted.ts' })).content, ' 1\tdisk'); +}); + +test('changes held on a branch not selected at the start keep no mount from starting direct', async (t) => { + const env = new Env(t); + env.file('drafted.ts', 'disk\n'); + const main = env.store.currentBranch().name; + env.store.createBranch('side', main); + env.store.switchBranch('side'); + const chronicle = await env.openChronicle(); + await data(chronicle, 'write', { path: 'ref/drafted.ts', content: 'draft\n' }); + await env.close(chronicle); + + env.store.switchBranch(main); + const direct = await env.open(); + assert.deepEqual(env.alerts, [], 'the branch selected holds none'); + await env.close(direct); + env.store.switchBranch('side'); + await env.open(); + assert.equal(env.alerts.length, 1, 'the branch that holds it keeps the mount store-backed'); +}); + +test("a mount configured direct with no ignore list walks by the direct kind's default while it runs as 'chronicle'", async (t) => { + const env = new Env(t); + env.file('src/a.ts', 'a\n'); + env.file('node_modules/p/i.js', 'i\n'); + env.file('.git/HEAD', 'ref: refs/heads/main\n'); + const chronicle = await env.openChronicle(); + await data(chronicle, 'write', { path: 'ref/drafted.ts', content: 'draft\n' }); + await env.close(chronicle); + + const held = await env.open(); + assert.equal(env.alerts.length, 1, "it runs as 'chronicle'"); + const listed = (await data<{ entries: Array<{ path: string }> }>(held, 'ls', { path: 'ref', recursive: true })).entries.map((e) => e.path); + assert.deepEqual(listed.filter((path) => !['drafted.ts', 'src/a.ts'].includes(path)), []); + assert.deepEqual(env.store.treeList('workspace/ref/tree').map((e) => e.path).sort(), ['drafted.ts', 'src/a.ts']); }); test("a mount that ran direct takes in what disk changed meanwhile when it runs as 'chronicle' again", async (t) => { @@ -660,6 +948,26 @@ test('inOrder hands each result on in order, at most `width` at once', async () assert.equal(running.most, 3); }); +test('inOrder lets each result go once it has handed it on', async () => { + setFlagsFromString('--expose_gc'); + const gc = runInNewContext('gc') as () => void; + const handed: Array> = []; + let firstKept: boolean | undefined; + // Each work ends in a later task than the one before it, so a WeakRef made + // in one is no longer held for its task when gc runs in another. + await workspaceModule.inOrder(Array.from({ length: 10 }, (_, i) => i), 2, async (i) => { + await new Promise((resolve) => setImmediate(resolve)); + return { i, bytes: Buffer.alloc(64 * 1024) }; + }, (result, i) => { + handed.push(new WeakRef(result)); + if (i === 9) { + gc(); + firstKept = handed[0]!.deref() !== undefined; + } + }); + assert.equal(firstKept, false); +}); + test('inOrder rejects in turn for a work that rejects before its turn, leaving no rejection unhandled', async () => { const unhandled: unknown[] = []; const record = (reason: unknown) => { unhandled.push(reason); }; diff --git a/test/workspace-parent-link.test.ts b/test/workspace-parent-link.test.ts index b71b8f6fd..dbc30f218 100644 --- a/test/workspace-parent-link.test.ts +++ b/test/workspace-parent-link.test.ts @@ -291,7 +291,9 @@ describe('a symlinked parent on a mount that does not follow symlinks', () => { env.writeDisk('l/x.txt', 'v1\n'); const m = await env.open(t); env.swapLForLink(target); - const expected = `work/nope/../l/x.txt: ${linkedReason(target)}`; + // The path is read as `work/l/x.txt`, its `..` taken lexically, as + // every workspace path is, and named so. + const expected = `work/l/x.txt: ${linkedReason(target)}`; await assert.rejects(m.readFileFromDisk('work/nope/../l/x.txt'), (err: unknown) => { assert.ok(err instanceof WorkspaceReadError, String(err)); assert.equal(err.code, linkedCode(target)); diff --git a/test/workspace-three-way.test.ts b/test/workspace-three-way.test.ts index 557f1ad8e..f42446af7 100644 --- a/test/workspace-three-way.test.ts +++ b/test/workspace-three-way.test.ts @@ -1097,10 +1097,11 @@ describe("another mount's directory, to the outer mount's file tools and pushes" assert.deepEqual(whole.results, []); assert.deepEqual(regions(whole), []); // Named, it's the answer to what was asked: its region names the holder. - for (const [path, use] of [['work/a/b', 'deep'], ['work/a/b/inner.txt', 'deep/inner.txt'], ['work/a//b/inner.txt', 'deep/inner.txt']]) { + // A path with an empty segment is named as it reads, without it. + for (const [path, named, use] of [['work/a/b', 'work/a/b', 'deep'], ['work/a/b/inner.txt', 'work/a/b/inner.txt', 'deep/inner.txt'], ['work/a//b/inner.txt', 'work/a/b/inner.txt', 'deep/inner.txt']]) { const found = await call(m, 'grep', { path, pattern: 'stale' }); assert.deepEqual(found.results, [], path); - assert.deepEqual(regions(found), [[path, `it lies in mount "deep", nested in this one: use "${use}"`]], path); + assert.deepEqual(regions(found), [[named, `it lies in mount "deep", nested in this one: use "${use}"`]], path); } // A file there the outer store doesn't hold is named too. env.writeDisk('a/b/other.txt', 'stale words'); From da45fa745a55dac13a514c37f10a270fabdb12ba Mon Sep 17 00:00:00 2001 From: Beth-2709 Date: Mon, 19 Oct 2026 03:46:03 -0400 Subject: [PATCH 03/13] fix(workspace): every mount's path parse stays as it was; the direct kind refuses '.', '..' and a file's trailing '/' Takes out the path normalization 42e0533a3 gave every mount: #536 already normalizes every mount's paths in parsePathAsGiven, with rules of its own, so that rule is #536's to make (Ada-1017's call). parsePathAsGiven and parsePath are as 6aadccb6b has them, the two existing tests' spelling edits are undone, and changelog.d/workspace-path-one-spelling.changed.md goes. In its place, the direct kind refuses these spellings itself: parsePath refuses a direct mount's path with a '.' or '..' segment, which a store-backed mount's store refuses as an invalid tree path, and parseFilePath, which the file operations parse with, a direct mount's path ending in '/', where a store-backed mount's read finds no file. Both read what parsePathAsGiven gives, so once #536's parse lands they see no such path. A directory's path may still end in '/'. From Esther-2584's read of 6aadccb6b..42e0533a3: - The alert that a mount configured direct runs as 'chronicle' counts an intent the workspace can't read as one, not as a change, for one mount and for several. - ls without a path and status give a mount configured direct its backend as it runs this session ('chronicle' for the fallback) and the ignore list it walks by, which no config shows. - The fragment says one ops alert names every such mount. Tests: the normalization test is replaced by one of the direct kind's refusals; the fallback test checks its ls and status; the held-kinds, ten- at-a-time and two-mount tests check the alert's counts. Co-Authored-By: Claude Opus 5.5 --- changelog.d/workspace-direct-mount.added.md | 5 +- .../workspace-path-one-spelling.changed.md | 1 - src/modules/workspace/index.ts | 68 +++++++++++------ src/modules/workspace/types.ts | 16 ++-- test/workspace-direct.test.ts | 73 +++++++++++-------- test/workspace-parent-link.test.ts | 4 +- test/workspace-three-way.test.ts | 5 +- 7 files changed, 105 insertions(+), 67 deletions(-) delete mode 100644 changelog.d/workspace-path-one-spelling.changed.md diff --git a/changelog.d/workspace-direct-mount.added.md b/changelog.d/workspace-direct-mount.added.md index 61fb6e45c..3bee83120 100644 --- a/changelog.d/workspace-direct-mount.added.md +++ b/changelog.d/workspace-direct-mount.added.md @@ -1,7 +1,8 @@ - **A direct workspace mount, for a checkout an agent reads:** a mount whose config sets `backend: 'direct'` (the default is `'chronicle'`, the store-backed kind every mount has been) is read straight from disk. `read`, `read_image`, `ls`, `glob` and `grep` give its files as they are when they run, with no scan, watcher or pass, so no call waits behind one, and none of its files go into the store. It's read-only: the workspace refuses a direct mount with any `mode` but `'read-only'`, and one that sets `watch` other than `'never'`, `watchDebounceMs`, `watchRootPollMs`, a `wakeOnChange` that would wake, or `autoMaterialize: true`, since none of them would ever act there. Its paths, links, other mounts' directories, size limit and walk cap hold as on a store-backed mount. Where it differs: + - **Spellings it refuses:** a path with a `.` or `..` segment, which a store-backed mount's store refuses as an invalid tree path, and, for `read`, `read_image` and the host's binary and disk reads, a path ending in `/`, after which a store-backed mount's `read` finds no file. It says why in words of its own. A directory's path may end in `/`. - **What its listings say:** each file with its `size`, and no state or MIME type, or, for a link the mount doesn't follow or another entry it doesn't take, with why as `note`. A listing looks at each entry with `lstat`, so it lists a file it can't open with its size, and reading that file says why. `grep`'s results say `version: 'disk'`, and it lists under `skipped` a binary file, one over the size limit and one it doesn't read, each with why. `read` refuses a binary file and one over the size limit, as a store-backed mount's `read` refuses one it never took in from disk. - - **Its ignore list:** with none in its config, it takes `node_modules`, `dist`, `.git`, `build`, `coverage`, `target`, `__pycache__` and `.venv`. Walks of the whole mount pass over what the list covers, and a path a listing, glob or grep names, or its pattern's directory, is walked all the same, as a sync naming a path walks it on a store-backed mount. `ls` without a path, and `status`, give a direct mount's `backend` and the list it goes by. + - **Its ignore list:** with none in its config, it takes `node_modules`, `dist`, `.git`, `build`, `coverage`, `target`, `__pycache__` and `.venv`. Walks of the whole mount pass over what the list covers, and a path a listing, glob or grep names, or its pattern's directory, is walked all the same, as a sync naming a path walks it on a store-backed mount. `ls` without a path, and `status`, give a mount configured direct its `backend`, as it runs this session, and the list it goes by. - **Its root:** it reads beneath whatever directory is at its path now, with no check against the one a store-backed run of it last agreed with. - **`sync`:** passes over it, listing it under `skipped`. `materialize` refuses it, as it refuses any read-only mount. - - **A mount the store holds changes on that disk lacks** (a draft, a workspace deletion, a conflict, or an intent it can't read, on the branch selected at a start) runs store-backed from that start, so they stay in view. It runs unwatched, and with the direct default ignore list where its config names none. The workspace says so on stderr and in one ops alert (`workspace-mount-not-direct`) for every such mount, naming the changes and the two ways to start it direct: give each change up with a sync of its path, or materialize it with the mount configured `'chronicle'` and read-write, as materialize's answer directs. The first start that finds none starts it direct. A mount that ran direct and runs store-backed again takes in what disk changed meanwhile. + - **A mount the store holds changes on that disk lacks** (a draft, a workspace deletion, a conflict, or an intent it can't read, on the branch selected at a start) runs store-backed from that start, so they stay in view. It runs unwatched, and with the direct default ignore list where its config names none. The workspace says so on stderr and in one ops alert (`workspace-mount-not-direct`) naming every such mount, what's held there, an intent it can't read counted as one, and the two ways to start one direct: give each change up with a sync of its path, or materialize it with the mount configured `'chronicle'` and read-write, as materialize's answer directs. The first start that finds none starts it direct. A mount that ran direct and runs store-backed again takes in what disk changed meanwhile. - **Tool descriptions:** `ls`, `glob`, `grep` and `sync` gain a sentence about the direct kind only on a host with a mount that runs direct; elsewhere they're unchanged. diff --git a/changelog.d/workspace-path-one-spelling.changed.md b/changelog.d/workspace-path-one-spelling.changed.md deleted file mode 100644 index 8a3a78053..000000000 --- a/changelog.d/workspace-path-one-spelling.changed.md +++ /dev/null @@ -1 +0,0 @@ -- **One spelling per workspace path, on every mount:** a path with `.` or empty segments, a trailing slash, or a `..` that stays inside the mount now reaches the file it names, and answers spell it without them. Before, a store-backed mount refused a `.` or `..` segment with the store's "Invalid tree path", refused an empty segment right after the mount's name as traversal, answered a file's path with a trailing slash with "File not found", and took any other empty segment as given, so a search whose path ended in a slash answered with `//` in each file's path. A `..` is read lexically, as a shell's `cd` reads `link/..`, and a path that leaves the mount is still refused. diff --git a/src/modules/workspace/index.ts b/src/modules/workspace/index.ts index dd3bba00a..28adeee45 100644 --- a/src/modules/workspace/index.ts +++ b/src/modules/workspace/index.ts @@ -988,7 +988,7 @@ export class WorkspaceModule implements Module { const held = this.heldChanges(state); const what = new Map(); for (const path of held.owed) what.set(path, 'changed in the workspace'); - for (const path of held.unreadable) what.set(path, "its workspace intent can't be read"); + for (const path of held.unreadable) what.set(path, UNREADABLE_INTENT); for (const path of held.deletions) what.set(path, 'deleted in the workspace'); for (const path of held.conflicts) what.set(path, 'in conflict'); if (what.size === 0) continue; @@ -1405,10 +1405,21 @@ export class WorkspaceModule implements Module { const mounts = [...this.notDirect]; const names = mounts.map(([name]) => `"${name}"`); const named = names.length === 1 ? names[0] : `${names.slice(0, -1).join(', ')} and ${names[names.length - 1]}`; - const changes = (n: number) => `${n} change${n === 1 ? '' : 's'}`; + // An intent the workspace can't read may hold a change or may not, so + // it's counted as what it is. + const unreadable = (paths: Array<{ what: string }>) => paths.filter((h) => h.what === UNREADABLE_INTENT).length; + const counted = (paths: Array<{ what: string }>) => { + const u = unreadable(paths); + const c = paths.length - u; + return [ + c > 0 ? `${c} change${c === 1 ? '' : 's'} that disk lacks` : '', + u > 0 ? `${u} intent${u === 1 ? '' : 's'} it can't read` : '', + ].filter((part) => part !== '').join(' and '); + }; const why = mounts.length === 1 - ? `Mount ${named} runs as 'chronicle' this session, not direct: the workspace holds ${changes(mounts[0]![1].length)} there that disk lacks.` - : `Mounts ${named} run as 'chronicle' this session, not direct: the workspace holds changes there that disk lacks (` + + ? `Mount ${named} runs as 'chronicle' this session, not direct: there the workspace holds ${counted(mounts[0]![1])}.` + : `Mounts ${named} run as 'chronicle' this session, not direct: there the workspace holds changes that disk lacks` + + `${mounts.some(([, paths]) => unreadable(paths) > 0) ? " or intents it can't read" : ''} (` + `${mounts.map(([name, held]) => `${held.length} in "${name}"`).join(', ')}).`; const held = mounts.flatMap(([name, paths]) => paths.map((h) => `"${name}/${h.path}" (${h.what})`)); const message = `${why} To start ${mounts.length === 1 ? 'it' : 'one'} direct, settle each: sync its path to take disk's copy, ` + @@ -2232,7 +2243,13 @@ export class WorkspaceModule implements Module { * it, naming the innermost mount that holds it and the path to use there. */ private async parseFilePath(path: string): Promise<{ mount: MountState; relativePath: string; path: string }> { - const parsed = this.refuseNested(await this.parsePath(path)); + const spelled = await this.parsePath(path); + // After a trailing slash a store-backed mount's read finds no file. A + // direct mount's would read the file and answer with the slash. + if (spelled.mount.direct && spelled.relativePath.endsWith('/')) { + throw new Error(`Path not taken: ${JSON.stringify(path)} ends with "/", which a direct mount's file operations don't read: name the file without it`); + } + const parsed = this.refuseNested(spelled); // A path can also reach a nested mount's directory through a symlink on // its way. No push of this mount writes it there and no read reads it, // so it's refused here, before anything is stored, and the checks where @@ -2289,11 +2306,17 @@ export class WorkspaceModule implements Module { */ private async parsePath(path: string): Promise<{ mount: MountState; relativePath: string; path: string }> { const given = this.parsePathAsGiven(path); + // A store-backed mount's store refuses a `.` or `..` segment as an + // invalid tree path. A direct mount reads disk, which would take it and + // answer with the path so spelled, giving one file several names. + if (given.mount.direct && given.relativePath.split('/').some((segment) => segment === '.' || segment === '..')) { + throw new Error(`Path not taken: ${JSON.stringify(path)} has a "." or ".." segment, which a direct mount doesn't read: name the file from the mount's root without it`); + } const relativePath = await diskSpelling(resolve(given.mount.config.path), given.relativePath, given.mount.direct ? undefined : this.entriesOf(given.mount)); return { mount: given.mount, relativePath, - path: relativePath === given.relativePath ? given.path : `${given.mount.config.name}/${relativePath}`, + path: relativePath === given.relativePath ? path : `${given.mount.config.name}/${relativePath}`, }; } @@ -2321,13 +2344,9 @@ export class WorkspaceModule implements Module { } /** - * Parse a mount-prefixed path into its mount and mount-relative path, with - * its `.` and empty segments and a trailing slash dropped and each `..` - * taken lexically, and `path` the mount-prefixed path so spelled: the path - * as given where nothing was dropped, so each file has one spelling. A - * path that leaves the mount is refused. + * Parse a mount-prefixed path into (mountName, relativePath), as spelled. */ - private parsePathAsGiven(path: string): { mount: MountState; relativePath: string; path: string } { + private parsePathAsGiven(path: string): { mount: MountState; relativePath: string } { // A file's name on disk is UTF-8, which can't hold a lone surrogate: the // store would keep one name and disk another. if (LONE_SURROGATE.test(path)) { @@ -2335,7 +2354,7 @@ export class WorkspaceModule implements Module { } const slashIdx = path.indexOf('/'); const mountName = slashIdx >= 0 ? path.slice(0, slashIdx) : path; - const asGiven = slashIdx >= 0 ? path.slice(slashIdx + 1) : ''; + const relativePath = slashIdx >= 0 ? path.slice(slashIdx + 1) : ''; const mount = this.mounts.get(mountName); if (!mount) { @@ -2348,18 +2367,13 @@ export class WorkspaceModule implements Module { // prefix-matches there and every in-mount path was rejected as traversal. // Against the resolved root, so a root configured with a trailing // separator, or relative, contains its own paths, the root included. - // The path the tools take is the one this resolves to, so each file has - // one spelling: `.` and empty segments are dropped, a trailing slash - // with them, and a `..` takes back the segment before it, lexically, as - // a shell's `cd` reads `link/..`. const root = resolve(mount.config.path); - const resolved = resolve(root, asGiven.split('/').filter((segment) => segment !== '' && segment !== '.').join('/')); + const resolved = resolve(root, relativePath); if (!isContainedPath(root, resolved)) { throw new Error(`Path traversal detected: "${path}" resolves outside mount "${mountName}"`); } - const relativePath = resolved === root ? '' : relative(root, resolved).split(sep).join('/'); - return { mount, relativePath, path: relativePath === asGiven ? path : relativePath ? `${mountName}/${relativePath}` : mountName }; + return { mount, relativePath }; } /** @@ -3443,13 +3457,14 @@ export class WorkspaceModule implements Module { private async handleLs(input: LsInput): Promise { if (!input.path) { - // List all mounts, with a direct mount's kind and the ignore list it - // walks by, whose default no config shows. + // List all mounts, with the kind a mount configured direct runs as + // this session and the ignore list it walks by, whose default no + // config shows. const mounts = [...this.mounts.entries()].map(([name, m]) => ({ name, path: m.config.path, mode: m.config.mode, - ...(m.direct ? { backend: 'direct', ignore: this.ignoreOf(m) } : {}), + ...(m.config.backend === 'direct' ? { backend: m.direct ? 'direct' : 'chronicle', ignore: this.ignoreOf(m) } : {}), })); return { success: true, data: { mounts } }; } @@ -3876,6 +3891,10 @@ export class WorkspaceModule implements Module { status[name] = { path: mount.config.path, mode: mount.config.mode, + // A mount configured direct runs as 'chronicle' this session + // (chooseKinds), by the direct kind's ignore list where its config + // names none. + ...(mount.config.backend === 'direct' ? { backend: 'chronicle', ignore: this.ignoreOf(mount) } : {}), watch: watchOf(mount.config), fileCount: entries.length, lastMaterializedSeq: mount.lastMaterializedSeq, @@ -4605,6 +4624,9 @@ const LONE_SURROGATE = /\p{Cs}/u; /** Why a direct mount's read stopped when another directory was put at its root as it read. */ const DIRECT_ROOT_REPLACED = 'the mount root was replaced while the file was read: reading it again reads the directory there now'; +/** How the alert that a mount configured direct runs as 'chronicle' names an intent the workspace can't read. */ +const UNREADABLE_INTENT = "its workspace intent can't be read"; + /** * A mount's watch mode as it runs. A mount configured direct watches * nothing, whichever kind it runs as this session: its config takes no diff --git a/src/modules/workspace/types.ts b/src/modules/workspace/types.ts index 3a4cdc0e1..36df2297d 100644 --- a/src/modules/workspace/types.ts +++ b/src/modules/workspace/types.ts @@ -32,9 +32,12 @@ export interface MountConfig { * `watchRootPollMs`, `wakeOnChange` or `autoMaterialize`: each of those * would never act, and the workspace refuses the mount. Its `ignore` * has a default (see there). Paths, links, other mounts' directories, - * size limits and the walk cap hold as on a 'chronicle' mount. Its - * tools read beneath whatever directory is at its root, with no check - * against the one a 'chronicle' run of it last agreed with. + * size limits and the walk cap hold as on a 'chronicle' mount. It + * refuses a path with a `.` or `..` segment, which a 'chronicle' + * mount's store refuses, and a file's path ending in `/`, where a + * 'chronicle' mount's read finds no file. Its tools read beneath + * whatever directory is at its root, with no check against the one a + * 'chronicle' run of it last agreed with. * * A mount configured 'direct' runs as 'chronicle' from a start that finds * the store holding changes there that disk lacks, on the branch selected @@ -43,9 +46,10 @@ export interface MountConfig { * of those. They stay in view, and the workspace says so on stderr and * through the framework's ops alerts, naming them and the ways to start it * direct. It runs unwatched, and walks by the direct kind's default - * ignore list where its config names none. A mount that ran direct and - * runs as 'chronicle' again takes in what disk changed meanwhile, as - * after any time away. + * ignore list where its config names none; `ls` without a path, and + * `status`, give its `backend` as 'chronicle' and that list. A mount that + * ran direct and runs as 'chronicle' again takes in what disk changed + * meanwhile, as after any time away. */ backend?: 'chronicle' | 'direct'; /** diff --git a/test/workspace-direct.test.ts b/test/workspace-direct.test.ts index 3929ff7e2..c23274688 100644 --- a/test/workspace-direct.test.ts +++ b/test/workspace-direct.test.ts @@ -571,26 +571,26 @@ test('a direct read whose root is replaced as it reads says to read again', asyn assert.equal(await failure(m, 'read', { path: 'ref/a.txt' }), 'File not found: ref/a.txt'); }); -test("every mount takes a path's '.' and empty segments, trailing slash and '..' as the file they name, and answers with one spelling", async (t) => { +test("a direct mount refuses a '.' or '..' segment, and a file operation's trailing slash, in words of its own", async (t) => { const env = new Env(t); env.file('a.ts', 'alpha\n'); env.file('dir/x.ts', 'alpha x\n'); - symlinkSync(join(env.root, 'elsewhere'), join(env.ref, 'link')); - for (const m of [await env.open(), await env.openChronicle()]) { - const kind = m.getTools().some((tool) => /\bdirect\b/.test(tool.description)) ? 'direct' : 'chronicle'; - for (const path of ['ref/./a.ts', 'ref/dir/../a.ts', 'ref/a.ts/', 'ref//a.ts', 'ref/link/../a.ts', 'ref/../ref/a.ts']) { - assert.deepEqual(await data(m, 'read', { path }), { path: 'ref/a.ts', totalLines: 1, fromLine: 1, toLine: 1, content: ' 1\talpha' }, `${kind} ${path}`); - } - assert.equal((await data<{ path: string }>(m, 'ls', { path: 'ref/dir/' })).path, 'ref/dir', kind); - assert.equal((await data<{ path: string }>(m, 'ls', { path: 'ref/dir/..' })).path, 'ref', kind); - for (const path of ['ref/dir/', 'ref/./dir', 'ref//dir']) { - assert.deepEqual((await data<{ results: Array<{ file: string }> }>(m, 'grep', { pattern: 'alpha', path })).results.map((r) => r.file), ['ref/dir/x.ts'], `${kind} ${path}`); - assert.deepEqual((await data<{ matches: Array<{ path: string }> }>(m, 'glob', { pattern: '*.ts', path })).matches.map((x) => x.path), ['ref/dir/x.ts'], `${kind} ${path}`); - } - for (const path of ['ref/../outside.txt', 'ref/dir/../../outside.txt']) { - assert.match(await failure(m, 'read', { path }), /^Path traversal detected/, `${kind} ${path}`); - } - } + env.file('dot.png', PNG); + const m = await env.open(); + const segment = (path: string) => `Path not taken: ${JSON.stringify(path)} has a "." or ".." segment, which a direct mount doesn't read: name the file from the mount's root without it`; + for (const path of ['ref/./a.ts', 'ref/dir/../a.ts', 'ref/dir/..']) assert.equal(await failure(m, 'read', { path }), segment(path), path); + assert.equal(await failure(m, 'ls', { path: 'ref/./dir' }), segment('ref/./dir')); + assert.equal(await failure(m, 'glob', { pattern: '*.ts', path: 'ref/dir/..' }), segment('ref/dir/..')); + assert.equal(await failure(m, 'grep', { pattern: 'alpha', path: 'ref/./dir' }), segment('ref/./dir')); + // A path that leaves the mount is refused as traversal, as before. + assert.match(await failure(m, 'read', { path: 'ref/../outside.txt' }), /^Path traversal detected/); + + const slash = (path: string) => `Path not taken: ${JSON.stringify(path)} ends with "/", which a direct mount's file operations don't read: name the file without it`; + assert.equal(await failure(m, 'read', { path: 'ref/a.ts/' }), slash('ref/a.ts/')); + assert.equal(await failure(m, 'read_image', { path: 'ref/dot.png/' }), slash('ref/dot.png/')); + assert.deepEqual(await m.readBinary('ref/a.ts/'), { error: slash('ref/a.ts/') }); + // A directory's path may end with one. + assert.deepEqual((await data(m, 'ls', { path: 'ref/dir/' })).entries, [{ name: 'x.ts', type: 'file', size: 8 }]); }); test("ls without a path and status give a direct mount's kind and ignore list, and a 'chronicle' mount's as before", async (t) => { @@ -732,7 +732,7 @@ test("a mount the store holds changes on that disk lacks runs as 'chronicle', sa warned.mock.restore(); // The ways out come before the list of what's held, so a webhook that // keeps only a long alert's start and end still gives them. - const message = 'Mount "ref" runs as \'chronicle\' this session, not direct: the workspace holds 4 changes there that disk lacks. ' + const message = 'Mount "ref" runs as \'chronicle\' this session, not direct: there the workspace holds 4 changes that disk lacks. ' + 'To start it direct, settle each: sync its path to take disk\'s copy, or materialize it with the mount configured ' + '\'chronicle\' and read-write, as materialize\'s answer directs. The first start that finds none starts it direct. ' + 'Held: "ref/both.ts" (in conflict), "ref/deleted.ts" (deleted in the workspace), "ref/dot.img" (in conflict), ' @@ -740,9 +740,13 @@ test("a mount the store holds changes on that disk lacks runs as 'chronicle', sa assert.deepEqual(env.alerts, [{ kind: 'workspace-mount-not-direct', agent: 'framework', message, data: { held: { ref: 4 } } }]); assert.deepEqual(warned.mock.calls.map((c) => c.arguments), [[`[workspace] ${message}`]]); // As a 'chronicle' mount, unwatched, with no catch-up scan to take turns - // with its syncs, its changes in view. + // with its syncs, its changes in view, and saying what it runs as and the + // ignore list it walks by. + const ignore = ['node_modules', 'dist', '.git', 'build', 'coverage', 'target', '__pycache__', '.venv']; + assert.deepEqual((await data(held, 'ls')).mounts, [{ name: 'ref', path: env.ref, mode: 'read-only', backend: 'chronicle', ignore }]); const status = await data>>(held, 'status'); - assert.equal(status.ref!.backend, undefined); + assert.equal(status.ref!.backend, 'chronicle'); + assert.deepEqual(status.ref!.ignore, ignore); assert.equal(status.ref!.watch, 'never'); assert.deepEqual(watched(held), []); assert.equal(status.ref!.pendingChanges, 4); @@ -786,35 +790,38 @@ test("the changes that keep a mount from starting direct are named ten at a time assert.equal(env.alerts.length, 1); const unreadable = (path: string) => `"ref/${path}" (its workspace intent can't be read)`; const shown = [unreadable('aa.ts'), unreadable('d00.ts'), ...Array.from({ length: 8 }, (_, i) => `"ref/d0${i + 1}.ts" (changed in the workspace)`)]; + // An intent it can't read is counted as one, not as a change. + assert.ok(env.alerts[0]!.message.startsWith("Mount \"ref\" runs as 'chronicle' this session, not direct: there the workspace holds 10 changes that disk lacks and 2 intents it can't read. "), env.alerts[0]!.message); assert.ok(env.alerts[0]!.message.endsWith(`Held: ${shown.join(', ')}, and 2 more.`), env.alerts[0]!.message); assert.deepEqual(env.alerts[0]!.data, { held: { ref: 12 } }); }); test('each kind of change the store holds keeps a mount configured direct from starting direct on its own', async (t) => { - const cases: Array<[string, (env: Env) => Promise, string]> = [ + const cases: Array<[string, (env: Env) => Promise, string, string]> = [ ['a workspace deletion', async (env) => { env.file('gone.ts', 'kept on disk\n'); const chronicle = await env.openChronicle(); await data(chronicle, 'ls', { path: 'ref' }); await data(chronicle, 'delete', { path: 'ref/gone.ts' }); await env.close(chronicle); - }, '"ref/gone.ts" (deleted in the workspace)'], + }, '1 change that disk lacks', '"ref/gone.ts" (deleted in the workspace)'], ['a conflict', async (env) => { await env.close(await env.openChronicle()); new BranchIntents(env.store, 'workspace/ref/intent').put('c.ts', { conflict: { kind: 'both-changed', at: 0, disk: null } }); - }, '"ref/c.ts" (in conflict)'], + }, '1 change that disk lacks', '"ref/c.ts" (in conflict)'], ["an intent that can't be read", async (env) => { await env.close(await env.openChronicle()); env.store.treeSet('workspace/ref/intent', 'u.ts', { blobHash: 'f'.repeat(64), size: 2, mode: 0 }); - }, `"ref/u.ts" (its workspace intent can't be read)`], + }, "1 intent it can't read", `"ref/u.ts" (its workspace intent can't be read)`], ]; - for (const [kind, hold, named] of cases) { + for (const [kind, hold, counted, named] of cases) { const env = new Env(t); await hold(env); const m = await env.open(); assert.equal(env.alerts.length, 1, kind); + assert.ok(env.alerts[0]!.message.startsWith(`Mount "ref" runs as 'chronicle' this session, not direct: there the workspace holds ${counted}. `), env.alerts[0]!.message); assert.ok(env.alerts[0]!.message.endsWith(`Held: ${named}.`), env.alerts[0]!.message); - assert.equal((await data>>(m, 'status')).ref!.backend, undefined, kind); + assert.equal((await data>>(m, 'status')).ref!.backend, 'chronicle', kind); } }); @@ -828,13 +835,21 @@ test('mounts configured direct that the store holds changes on are named in one await data(chronicle, 'write', { path: 'other/c.ts', content: 'c\n' }); await env.close(chronicle); - await env.open({}, [{ name: 'other', path: other, mode: 'read-only', backend: 'direct' }]); + const both = await env.open({}, [{ name: 'other', path: other, mode: 'read-only', backend: 'direct' }]); assert.equal(env.alerts.length, 1); - assert.ok(env.alerts[0]!.message.startsWith('Mounts "ref" and "other" run as \'chronicle\' this session, not direct: the workspace holds ' - + 'changes there that disk lacks (1 in "ref", 2 in "other"). To start one direct, settle each:'), env.alerts[0]!.message); + assert.ok(env.alerts[0]!.message.startsWith('Mounts "ref" and "other" run as \'chronicle\' this session, not direct: there the workspace holds ' + + 'changes that disk lacks (1 in "ref", 2 in "other"). To start one direct, settle each:'), env.alerts[0]!.message); assert.ok(env.alerts[0]!.message.endsWith('Held: "ref/a.ts" (changed in the workspace), "other/b.ts" (changed in the workspace), ' + '"other/c.ts" (changed in the workspace).'), env.alerts[0]!.message); assert.deepEqual(env.alerts[0]!.data, { held: { ref: 1, other: 2 } }); + await env.close(both); + + // An intent it can't read, in either mount, is named as one. + env.store.treeSet('workspace/other/intent', 'u.ts', { blobHash: 'f'.repeat(64), size: 2, mode: 0 }); + env.alerts = []; + await env.open({}, [{ name: 'other', path: other, mode: 'read-only', backend: 'direct' }]); + assert.ok(env.alerts[0]!.message.startsWith('Mounts "ref" and "other" run as \'chronicle\' this session, not direct: there the workspace holds ' + + "changes that disk lacks or intents it can't read (1 in \"ref\", 3 in \"other\"). "), env.alerts[0]!.message); }); test('a start of the same module decides again whether a mount configured direct runs direct', async (t) => { diff --git a/test/workspace-parent-link.test.ts b/test/workspace-parent-link.test.ts index dbc30f218..b71b8f6fd 100644 --- a/test/workspace-parent-link.test.ts +++ b/test/workspace-parent-link.test.ts @@ -291,9 +291,7 @@ describe('a symlinked parent on a mount that does not follow symlinks', () => { env.writeDisk('l/x.txt', 'v1\n'); const m = await env.open(t); env.swapLForLink(target); - // The path is read as `work/l/x.txt`, its `..` taken lexically, as - // every workspace path is, and named so. - const expected = `work/l/x.txt: ${linkedReason(target)}`; + const expected = `work/nope/../l/x.txt: ${linkedReason(target)}`; await assert.rejects(m.readFileFromDisk('work/nope/../l/x.txt'), (err: unknown) => { assert.ok(err instanceof WorkspaceReadError, String(err)); assert.equal(err.code, linkedCode(target)); diff --git a/test/workspace-three-way.test.ts b/test/workspace-three-way.test.ts index f42446af7..557f1ad8e 100644 --- a/test/workspace-three-way.test.ts +++ b/test/workspace-three-way.test.ts @@ -1097,11 +1097,10 @@ describe("another mount's directory, to the outer mount's file tools and pushes" assert.deepEqual(whole.results, []); assert.deepEqual(regions(whole), []); // Named, it's the answer to what was asked: its region names the holder. - // A path with an empty segment is named as it reads, without it. - for (const [path, named, use] of [['work/a/b', 'work/a/b', 'deep'], ['work/a/b/inner.txt', 'work/a/b/inner.txt', 'deep/inner.txt'], ['work/a//b/inner.txt', 'work/a/b/inner.txt', 'deep/inner.txt']]) { + for (const [path, use] of [['work/a/b', 'deep'], ['work/a/b/inner.txt', 'deep/inner.txt'], ['work/a//b/inner.txt', 'deep/inner.txt']]) { const found = await call(m, 'grep', { path, pattern: 'stale' }); assert.deepEqual(found.results, [], path); - assert.deepEqual(regions(found), [[named, `it lies in mount "deep", nested in this one: use "${use}"`]], path); + assert.deepEqual(regions(found), [[path, `it lies in mount "deep", nested in this one: use "${use}"`]], path); } // A file there the outer store doesn't hold is named too. env.writeDisk('a/b/other.txt', 'stale words'); From a25a4259465d028bbb0e259c9d9a9ba7b59f6956 Mon Sep 17 00:00:00 2001 From: Beth-2709 Date: Mon, 19 Oct 2026 04:11:45 -0400 Subject: [PATCH 04/13] fix(workspace): the direct mount's second review round Answers an internal review of e566cc69. The fallback alert: - Its words of fixed length come first: what's happening and the two ways to start a mount direct, then the mounts with what each holds, then the held list, so the webhook's kept head (about 346 of 500 characters) holds the ways out however many mounts it names and however long their names. - The materialize route reads as steps: configure the mount 'chronicle' and read-write, materialize it, and follow materialize's answer. - With several mounts it says to settle each change held at a mount to start that mount direct. - An intent the workspace can't read is named with the way that settles it, a sync of the path, since materialize's selection leaves such an intent with no workspace entry unselected. The decision: - chooseKinds decides every mount before it sets any, so a read of the store that throws leaves the earlier decisions and fails the start. - A mount configured direct that ran as 'chronicle' starts store-backed in initStore, and runs direct only once a decision finds nothing held. - Tests in the host's order (start, then initStore) and the store-first order, and of a decision that can't read the store. readBytes: every refusal from the open asks observePath, so a link to a directory it can't open gets the listing's reason, and a regular file it can't open keeps the open's. Tests: the grep order test holds its first file's reads until fifteen others are read, and asserts it; the inOrder test settles its works in reverse by hand; each held kind is settled by a sync of its path and the next start is direct. Text: a mount that ran direct and runs as 'chronicle' again takes in disk's changes as its listings, searches and syncs reach them; a direct read that finds its root replaced as it opens the file is refused; watch 'never''s doc says a read observes only a path the store doesn't hold; MountState.direct's doc names both decisions. Co-Authored-By: Claude Opus 5.5 --- changelog.d/workspace-direct-mount.added.md | 2 +- src/modules/workspace/index.ts | 77 ++++--- src/modules/workspace/observe.ts | 12 +- src/modules/workspace/types.ts | 14 +- test/workspace-direct.test.ts | 227 ++++++++++++++++---- 5 files changed, 255 insertions(+), 77 deletions(-) diff --git a/changelog.d/workspace-direct-mount.added.md b/changelog.d/workspace-direct-mount.added.md index 3bee83120..47931174b 100644 --- a/changelog.d/workspace-direct-mount.added.md +++ b/changelog.d/workspace-direct-mount.added.md @@ -4,5 +4,5 @@ - **Its ignore list:** with none in its config, it takes `node_modules`, `dist`, `.git`, `build`, `coverage`, `target`, `__pycache__` and `.venv`. Walks of the whole mount pass over what the list covers, and a path a listing, glob or grep names, or its pattern's directory, is walked all the same, as a sync naming a path walks it on a store-backed mount. `ls` without a path, and `status`, give a mount configured direct its `backend`, as it runs this session, and the list it goes by. - **Its root:** it reads beneath whatever directory is at its path now, with no check against the one a store-backed run of it last agreed with. - **`sync`:** passes over it, listing it under `skipped`. `materialize` refuses it, as it refuses any read-only mount. - - **A mount the store holds changes on that disk lacks** (a draft, a workspace deletion, a conflict, or an intent it can't read, on the branch selected at a start) runs store-backed from that start, so they stay in view. It runs unwatched, and with the direct default ignore list where its config names none. The workspace says so on stderr and in one ops alert (`workspace-mount-not-direct`) naming every such mount, what's held there, an intent it can't read counted as one, and the two ways to start one direct: give each change up with a sync of its path, or materialize it with the mount configured `'chronicle'` and read-write, as materialize's answer directs. The first start that finds none starts it direct. A mount that ran direct and runs store-backed again takes in what disk changed meanwhile. + - **A mount the store holds changes on that disk lacks** (a draft, a workspace deletion, a conflict, or an intent it can't read, on the branch selected at a start) runs store-backed from that start, so they stay in view. It runs unwatched, and with the direct default ignore list where its config names none. The workspace says so on stderr and in one ops alert (`workspace-mount-not-direct`). The alert first gives the two ways to start such a mount direct: give each change up with a sync of its path, or configure the mount `'chronicle'` and read-write, materialize it, and follow materialize's answer; its next start then runs it direct. Then it names every such mount and what's held there, counting an intent it can't read as one. A mount that ran direct and runs store-backed again takes in what disk changed meanwhile as its listings, searches and syncs reach it, as an unwatched store-backed mount does. - **Tool descriptions:** `ls`, `glob`, `grep` and `sync` gain a sentence about the direct kind only on a host with a mount that runs direct; elsewhere they're unchanged. diff --git a/src/modules/workspace/index.ts b/src/modules/workspace/index.ts index 28adeee45..338321efa 100644 --- a/src/modules/workspace/index.ts +++ b/src/modules/workspace/index.ts @@ -925,6 +925,9 @@ export class WorkspaceModule implements Module { const treeStateId = `workspace/${mount.name}/tree`; const intentTreeStateId = `workspace/${mount.name}/intent`; const direct = mount.backend === 'direct'; + // A mount configured direct that ran as 'chronicle' runs direct only + // once chooseKinds finds nothing held there. + const ranStoreBacked = direct && made!.has(treeStateId) && made!.has(intentTreeStateId); if (!direct) { for (const id of [treeStateId, intentTreeStateId]) { ensureStateRegistered(store, { @@ -938,7 +941,7 @@ export class WorkspaceModule implements Module { mounts.set(mount.name, { config: mount, - direct, + direct: direct && !ranStoreBacked, treeStateId, intentTreeStateId, lastMaterializedSeq: 0, @@ -948,7 +951,7 @@ export class WorkspaceModule implements Module { watcherReadyAt: null, watcherError: null, }); - if (!direct || (made!.has(treeStateId) && made!.has(intentTreeStateId))) { + if (!direct || ranStoreBacked) { intents.set(mount.name, new BranchIntents(store, intentTreeStateId)); } } @@ -980,20 +983,28 @@ export class WorkspaceModule implements Module { * is named for the last way set here. */ private chooseKinds(): void { - this.notDirect = new Map(); + // Every mount is decided before any is set, so a read of the store that + // throws leaves the decisions as they were. + const decided = new Map>(); for (const [name, state] of this.mounts) { if (state.config.backend !== 'direct') continue; - state.direct = true; - if (!this.intents.has(name)) continue; + // One with no states from a 'chronicle' run holds nothing. + if (!this.intents.has(name)) { + decided.set(name, []); + continue; + } const held = this.heldChanges(state); const what = new Map(); for (const path of held.owed) what.set(path, 'changed in the workspace'); for (const path of held.unreadable) what.set(path, UNREADABLE_INTENT); for (const path of held.deletions) what.set(path, 'deleted in the workspace'); for (const path of held.conflicts) what.set(path, 'in conflict'); - if (what.size === 0) continue; - state.direct = false; - this.notDirect.set(name, [...what].sort(([a], [b]) => a.localeCompare(b)).map(([path, label]) => ({ path, what: label }))); + decided.set(name, [...what].sort(([a], [b]) => a.localeCompare(b)).map(([path, label]) => ({ path, what: label }))); + } + this.notDirect = new Map(); + for (const [name, held] of decided) { + this.mounts.get(name)!.direct = held.length === 0; + if (held.length > 0) this.notDirect.set(name, held); } } @@ -1394,37 +1405,39 @@ export class WorkspaceModule implements Module { /** * Say, for whoever runs the host, which mounts configured direct run as - * 'chronicle' this session (chooseKinds), which changes the store holds - * there that disk lacks are why, and the two ways to start one direct: on - * stderr, and as one ops alert for them all (the framework's failures log, - * its `ops:alert` trace and, where one is set, its webhook, which posts - * one alert of a kind per cooldown and only the start and end of a long - * one). So the ways out come before the list of what's held. + * 'chronicle' this session (chooseKinds), what the workspace holds there + * that keeps them so, and the two ways to start one direct: on stderr, + * and as one ops alert for them all (the framework's failures log, its + * `ops:alert` trace and, where one is set, its webhook, which posts one + * alert of a kind per cooldown and keeps only the start and end of a long + * one). So the words of fixed length, what's happening and the ways out, + * come first, and the mounts and what they hold after them. */ private sayNotDirect(): void { const mounts = [...this.notDirect]; - const names = mounts.map(([name]) => `"${name}"`); - const named = names.length === 1 ? names[0] : `${names.slice(0, -1).join(', ')} and ${names[names.length - 1]}`; + const one = mounts.length === 1; // An intent the workspace can't read may hold a change or may not, so // it's counted as what it is. - const unreadable = (paths: Array<{ what: string }>) => paths.filter((h) => h.what === UNREADABLE_INTENT).length; const counted = (paths: Array<{ what: string }>) => { - const u = unreadable(paths); + const u = paths.filter((h) => h.what === UNREADABLE_INTENT).length; const c = paths.length - u; return [ c > 0 ? `${c} change${c === 1 ? '' : 's'} that disk lacks` : '', u > 0 ? `${u} intent${u === 1 ? '' : 's'} it can't read` : '', ].filter((part) => part !== '').join(' and '); }; - const why = mounts.length === 1 - ? `Mount ${named} runs as 'chronicle' this session, not direct: there the workspace holds ${counted(mounts[0]![1])}.` - : `Mounts ${named} run as 'chronicle' this session, not direct: there the workspace holds changes that disk lacks` + - `${mounts.some(([, paths]) => unreadable(paths) > 0) ? " or intents it can't read" : ''} (` + - `${mounts.map(([name, held]) => `${held.length} in "${name}"`).join(', ')}).`; + const each = mounts.map(([name, paths]) => `"${name}" (${counted(paths)})`); + const which = one + ? `Mount "${mounts[0]![0]}" holds ${counted(mounts[0]![1])}` + : `Mounts ${each.slice(0, -1).join(', ')} and ${each[each.length - 1]} hold`; const held = mounts.flatMap(([name, paths]) => paths.map((h) => `"${name}/${h.path}" (${h.what})`)); - const message = `${why} To start ${mounts.length === 1 ? 'it' : 'one'} direct, settle each: sync its path to take disk's copy, ` + - "or materialize it with the mount configured 'chronicle' and read-write, as materialize's answer directs. " + - `The first start that finds none starts it direct. Held: ${held.slice(0, 10).join(', ')}` + + const message = (one + ? "A mount configured direct runs as 'chronicle' this session, keeping what the workspace holds there in view. " + + 'To start it direct, settle each held change: ' + : "Mounts configured direct run as 'chronicle' this session, keeping what the workspace holds there in view. " + + 'To start a mount direct, settle each change held at it: ') + + "sync its path to take disk's copy, or configure the mount 'chronicle' and read-write, materialize it, " + + `and follow materialize's answer. Its next start then runs it direct. ${which}: ${held.slice(0, 10).join(', ')}` + `${held.length > 10 ? `, and ${held.length - 10} more` : ''}.`; console.warn(`[workspace] ${message}`); this.ctx?.notifyOps?.('workspace-mount-not-direct', 'framework', message, { held: Object.fromEntries(mounts.map(([name, paths]) => [name, paths.length])) }); @@ -4621,11 +4634,15 @@ function stateView(report: PathReport): Record { /** A lone UTF-16 surrogate, which UTF-8 can't store: in `u` mode, a pair is one code point. */ const LONE_SURROGATE = /\p{Cs}/u; -/** Why a direct mount's read stopped when another directory was put at its root as it read. */ -const DIRECT_ROOT_REPLACED = 'the mount root was replaced while the file was read: reading it again reads the directory there now'; +/** Why a direct mount's read stopped when it found another directory put at its root as it opened the file. */ +const DIRECT_ROOT_REPLACED = 'the mount root was replaced as the file was opened: reading it again reads the directory there now'; -/** How the alert that a mount configured direct runs as 'chronicle' names an intent the workspace can't read. */ -const UNREADABLE_INTENT = "its workspace intent can't be read"; +/** + * How the alert that a mount configured direct runs as 'chronicle' names an + * intent the workspace can't read: materialize doesn't select one with no + * workspace entry, and a sync of its path replaces it. + */ +const UNREADABLE_INTENT = "its workspace intent can't be read; a sync of the path settles it"; /** * A mount's watch mode as it runs. A mount configured direct watches diff --git a/src/modules/workspace/observe.ts b/src/modules/workspace/observe.ts index eb68e1bd1..79d23417a 100644 --- a/src/modules/workspace/observe.ts +++ b/src/modules/workspace/observe.ts @@ -1264,8 +1264,10 @@ export interface BytesResult { * A mount-relative file's bytes, through the same located descriptor as * readPath, for a direct mount's tools, which keep nothing: at most `limit` * of them, and none hashed. A file over `limit` at its fstat has only its - * head read, and one that grew past it since is read no further than it. A - * link to a directory gets the reason a listing gives it. + * head read, and one that grew past it since is read no further than it. + * What the open refuses gets the reason a listing gives it, such as a link + * to a directory, except a regular file it couldn't open, which keeps the + * open's. */ export async function readBytes( view: MountView, @@ -1276,8 +1278,10 @@ export async function readBytes( const opened = await openContained(view, rootReal, relativePath); if (!('handle' in opened)) { // A walk takes a link to a directory as an entry, where the mount follows - // links: it's named as a listing names it (observePath). - if (opened.kind === 'unobserved' && opened.reason === 'a directory') { + // links, and the open may refuse it as a directory or, where it can't + // be opened, by its errno: either way it's named as a listing names it + // (observePath). + if (opened.kind === 'unobserved') { const seen = await observePath(view, rootReal, relativePath); return seen.kind === 'file' ? opened : seen; } diff --git a/src/modules/workspace/types.ts b/src/modules/workspace/types.ts index 36df2297d..8cfa7cd69 100644 --- a/src/modules/workspace/types.ts +++ b/src/modules/workspace/types.ts @@ -49,7 +49,8 @@ export interface MountConfig { * ignore list where its config names none; `ls` without a path, and * `status`, give its `backend` as 'chronicle' and that list. A mount that * ran direct and runs as 'chronicle' again takes in what disk changed - * meanwhile, as after any time away. + * meanwhile as its listings, searches and syncs reach it, as an unwatched + * 'chronicle' mount does after any time away. */ backend?: 'chronicle' | 'direct'; /** @@ -61,8 +62,8 @@ export interface MountConfig { * the round goes on. Each scan walks from the mount's start, so on a * mount over its `walkCap` it covers the first `walkCap` entries every * time, and files past them come in as `walkCap` describes - * - 'never': no background reads; listings and lazy reads still observe - * the paths they show + * - 'never': no background reads; listings still observe the paths they + * show, and a read observes a path the store doesn't hold */ watch?: 'always' | 'on-agent-action' | 'never'; /** Debounce window in ms for watch: 'always' mode (default: 300) */ @@ -195,9 +196,10 @@ export interface MountState { /** The mount config */ config: MountConfig; /** - * Whether the mount runs direct this session (`MountConfig.backend`), as - * initStore decided: a mount configured direct runs as 'chronicle' while - * the store holds changes there that disk lacks. + * Whether the mount runs direct (`MountConfig.backend`), as the workspace + * decides when initStore gives the store and again at each start: a mount + * configured direct runs as 'chronicle' while the store holds changes + * there that disk lacks. */ direct: boolean; /** Tree state ID in Chronicle */ diff --git a/test/workspace-direct.test.ts b/test/workspace-direct.test.ts index c23274688..39f24362a 100644 --- a/test/workspace-direct.test.ts +++ b/test/workspace-direct.test.ts @@ -480,25 +480,64 @@ test('grep on a direct mount that follows links names a link to a directory as i ]); }); +test("grep on a direct mount names a link to a directory it can't open as its listing does, and a file it can't open by the open's reason", { skip: IS_ROOT ? 'root opens everything' : false }, async (t) => { + const env = new Env(t); + env.file('shut/x.ts', 'needle\n'); + env.file('noperm.txt', 'needle\n'); + symlinkSync(join(env.ref, 'shut'), join(env.ref, 'linkshut')); + const m = await env.open({ followSymlinks: true, ignore: ['shut'] }); + chmodSync(join(env.ref, 'shut'), 0o311); + chmodSync(join(env.ref, 'noperm.txt'), 0o000); + try { + const listed = await data<{ entries: Array<{ path: string }> }>(m, 'ls', { path: 'ref', recursive: true }); + assert.deepEqual(listed.entries.find((e) => e.path === 'linkshut'), { path: 'linkshut', note: 'a symlink to a directory, not walked' }); + assert.deepEqual((await data<{ skipped?: unknown[] }>(m, 'grep', { pattern: 'needle' })).skipped, [ + { file: 'ref/linkshut', reason: 'a symlink to a directory, not walked' }, + { file: 'ref/noperm.txt', reason: 'cannot open (EACCES)' }, + ]); + } finally { + chmodSync(join(env.ref, 'shut'), 0o755); + chmodSync(join(env.ref, 'noperm.txt'), 0o644); + } +}); + test("grep's results come in path order, however its reads finish", async (t) => { const env = new Env(t); const names = Array.from({ length: 40 }, (_, i) => `f${String(i).padStart(2, '0')}.txt`); for (const name of [...names].reverse()) env.file(`many/${name}`, `${name} needle\n`.repeat(1 + (names.indexOf(name) % 7) * 500)); const m = await env.open(); - // The first file's reads are held back, so it finishes after the rest. + // The first file's reads wait until the other fifteen files read beside + // it have been read, so it finishes after them. const first = String(statSync(join(env.ref, 'many', names[0]!), { bigint: true }).ino); const slow = new WeakSet(); + const read = new WeakSet(); + let othersRead = 0; + let fifteenRead!: () => void; + const fifteen = new Promise((resolve) => { fifteenRead = resolve; }); + let readBeforeFirst = -1; const result = await withFileHandle({ stat: (stat) => async function (this: object, ...args: unknown[]) { const seen = await stat.apply(this, args); if (String(seen.ino) === first) slow.add(this); return seen; }, - read: (read) => async function (this: object, ...args: unknown[]) { - if (slow.has(this)) await new Promise((resolve) => setTimeout(resolve, 50)); - return read.apply(this, args); + read: (original) => async function (this: object, ...args: unknown[]) { + if (slow.has(this)) { + let timer: ReturnType | undefined; + await Promise.race([fifteen, new Promise((resolve) => { timer = setTimeout(resolve, 5000); })]); + clearTimeout(timer); + readBeforeFirst = othersRead; + return original.apply(this, args); + } + const answer = await original.apply(this, args); + if (!read.has(this)) { + read.add(this); + if (++othersRead === 15) fifteenRead(); + } + return answer; }, }, () => data<{ results: Array<{ file: string }> }>(m, 'grep', { pattern: 'needle' })); + assert.ok(readBeforeFirst >= 15, `${readBeforeFirst} other files were read before the first`); assert.deepEqual(result.results.map((r) => r.file), names.map((name) => `ref/many/${name}`)); }); @@ -559,7 +598,7 @@ test("grep gives a skipped file's size as its reads found it, when the file chan assert.deepEqual(shrunk.skipped, [{ file: 'ref/bin.dat', reason: 'binary: a NUL in its first 8 KB', size: 4 }]); }); -test('a direct read whose root is replaced as it reads says to read again', async (t) => { +test('a direct read that finds its root replaced as it opens the file says to read again', async (t) => { const env = new Env(t); env.file('a.txt', 'a\n'); const m = await env.open(); @@ -567,7 +606,7 @@ test('a direct read whose root is replaced as it reads says to read again', asyn renameSync(env.ref, `${env.ref}-before`); mkdirSync(env.ref); }, () => failure(m, 'read', { path: 'ref/a.txt' })); - assert.equal(error, 'Mount unavailable: ref: the mount root was replaced while the file was read: reading it again reads the directory there now'); + assert.equal(error, 'Mount unavailable: ref: the mount root was replaced as the file was opened: reading it again reads the directory there now'); assert.equal(await failure(m, 'read', { path: 'ref/a.txt' }), 'File not found: ref/a.txt'); }); @@ -730,13 +769,11 @@ test("a mount the store holds changes on that disk lacks runs as 'chronicle', sa const warned = t.mock.method(console, 'warn', () => {}); const held = await env.open({ walkCap: 1 }); warned.mock.restore(); - // The ways out come before the list of what's held, so a webhook that - // keeps only a long alert's start and end still gives them. - const message = 'Mount "ref" runs as \'chronicle\' this session, not direct: there the workspace holds 4 changes that disk lacks. ' - + 'To start it direct, settle each: sync its path to take disk\'s copy, or materialize it with the mount configured ' - + '\'chronicle\' and read-write, as materialize\'s answer directs. The first start that finds none starts it direct. ' - + 'Held: "ref/both.ts" (in conflict), "ref/deleted.ts" (deleted in the workspace), "ref/dot.img" (in conflict), ' - + '"ref/drafted.ts" (changed in the workspace).'; + const message = "A mount configured direct runs as 'chronicle' this session, keeping what the workspace holds there in view. " + + "To start it direct, settle each held change: sync its path to take disk's copy, or configure the mount 'chronicle' " + + "and read-write, materialize it, and follow materialize's answer. Its next start then runs it direct. " + + 'Mount "ref" holds 4 changes that disk lacks: "ref/both.ts" (in conflict), "ref/deleted.ts" (deleted in the workspace), ' + + '"ref/dot.img" (in conflict), "ref/drafted.ts" (changed in the workspace).'; assert.deepEqual(env.alerts, [{ kind: 'workspace-mount-not-direct', agent: 'framework', message, data: { held: { ref: 4 } } }]); assert.deepEqual(warned.mock.calls.map((c) => c.arguments), [[`[workspace] ${message}`]]); // As a 'chronicle' mount, unwatched, with no catch-up scan to take turns @@ -788,15 +825,14 @@ test("the changes that keep a mount from starting direct are named ten at a time await env.open(); assert.equal(env.alerts.length, 1); - const unreadable = (path: string) => `"ref/${path}" (its workspace intent can't be read)`; + const unreadable = (path: string) => `"ref/${path}" (its workspace intent can't be read; a sync of the path settles it)`; const shown = [unreadable('aa.ts'), unreadable('d00.ts'), ...Array.from({ length: 8 }, (_, i) => `"ref/d0${i + 1}.ts" (changed in the workspace)`)]; // An intent it can't read is counted as one, not as a change. - assert.ok(env.alerts[0]!.message.startsWith("Mount \"ref\" runs as 'chronicle' this session, not direct: there the workspace holds 10 changes that disk lacks and 2 intents it can't read. "), env.alerts[0]!.message); - assert.ok(env.alerts[0]!.message.endsWith(`Held: ${shown.join(', ')}, and 2 more.`), env.alerts[0]!.message); + assert.ok(env.alerts[0]!.message.endsWith(`Mount "ref" holds 10 changes that disk lacks and 2 intents it can't read: ${shown.join(', ')}, and 2 more.`), env.alerts[0]!.message); assert.deepEqual(env.alerts[0]!.data, { held: { ref: 12 } }); }); -test('each kind of change the store holds keeps a mount configured direct from starting direct on its own', async (t) => { +test('each kind of change the store holds keeps a mount configured direct from starting direct on its own, until a sync of its path settles it', async (t) => { const cases: Array<[string, (env: Env) => Promise, string, string]> = [ ['a workspace deletion', async (env) => { env.file('gone.ts', 'kept on disk\n'); @@ -812,16 +848,23 @@ test('each kind of change the store holds keeps a mount configured direct from s ["an intent that can't be read", async (env) => { await env.close(await env.openChronicle()); env.store.treeSet('workspace/ref/intent', 'u.ts', { blobHash: 'f'.repeat(64), size: 2, mode: 0 }); - }, "1 intent it can't read", `"ref/u.ts" (its workspace intent can't be read)`], + }, "1 intent it can't read", `"ref/u.ts" (its workspace intent can't be read; a sync of the path settles it)`], ]; for (const [kind, hold, counted, named] of cases) { const env = new Env(t); await hold(env); const m = await env.open(); assert.equal(env.alerts.length, 1, kind); - assert.ok(env.alerts[0]!.message.startsWith(`Mount "ref" runs as 'chronicle' this session, not direct: there the workspace holds ${counted}. `), env.alerts[0]!.message); - assert.ok(env.alerts[0]!.message.endsWith(`Held: ${named}.`), env.alerts[0]!.message); + assert.ok(env.alerts[0]!.message.endsWith(`Mount "ref" holds ${counted}: ${named}.`), env.alerts[0]!.message); assert.equal((await data>>(m, 'status')).ref!.backend, 'chronicle', kind); + + // The alert's way out: a sync of the path, then the next start is direct. + await data(m, 'sync', { path: named.slice(1, named.indexOf('"', 1)) }); + await env.close(m); + env.alerts = []; + const again = await env.open(); + assert.deepEqual(env.alerts, [], kind); + assert.equal((await data>>(again, 'status')).ref!.backend, 'direct', kind); } }); @@ -837,19 +880,40 @@ test('mounts configured direct that the store holds changes on are named in one const both = await env.open({}, [{ name: 'other', path: other, mode: 'read-only', backend: 'direct' }]); assert.equal(env.alerts.length, 1); - assert.ok(env.alerts[0]!.message.startsWith('Mounts "ref" and "other" run as \'chronicle\' this session, not direct: there the workspace holds ' - + 'changes that disk lacks (1 in "ref", 2 in "other"). To start one direct, settle each:'), env.alerts[0]!.message); - assert.ok(env.alerts[0]!.message.endsWith('Held: "ref/a.ts" (changed in the workspace), "other/b.ts" (changed in the workspace), ' - + '"other/c.ts" (changed in the workspace).'), env.alerts[0]!.message); + assert.equal(env.alerts[0]!.message, "Mounts configured direct run as 'chronicle' this session, keeping what the workspace holds there in view. " + + "To start a mount direct, settle each change held at it: sync its path to take disk's copy, or configure the mount 'chronicle' " + + "and read-write, materialize it, and follow materialize's answer. Its next start then runs it direct. " + + 'Mounts "ref" (1 change that disk lacks) and "other" (2 changes that disk lacks) hold: "ref/a.ts" (changed in the workspace), ' + + '"other/b.ts" (changed in the workspace), "other/c.ts" (changed in the workspace).'); assert.deepEqual(env.alerts[0]!.data, { held: { ref: 1, other: 2 } }); await env.close(both); - // An intent it can't read, in either mount, is named as one. + // An intent it can't read, in either mount, is counted as one. env.store.treeSet('workspace/other/intent', 'u.ts', { blobHash: 'f'.repeat(64), size: 2, mode: 0 }); env.alerts = []; await env.open({}, [{ name: 'other', path: other, mode: 'read-only', backend: 'direct' }]); - assert.ok(env.alerts[0]!.message.startsWith('Mounts "ref" and "other" run as \'chronicle\' this session, not direct: there the workspace holds ' - + "changes that disk lacks or intents it can't read (1 in \"ref\", 3 in \"other\"). "), env.alerts[0]!.message); + assert.ok(env.alerts[0]!.message.includes("Mounts \"ref\" (1 change that disk lacks) and \"other\" (2 changes that disk lacks and 1 intent it can't read) hold: "), env.alerts[0]!.message); +}); + +test("the alert's ways out come within the start a webhook keeps of it, however many mounts it names and however long their names", async (t) => { + const env = new Env(t); + const extra = ['another-checkout-whose-name-is-rather-long', 'a-third-checkout-with-a-long-name-too'].map((name) => { + mkdirSync(join(env.root, name)); + return { name, path: join(env.root, name) }; + }); + const chronicle = await env.open({ backend: 'chronicle', mode: 'read-write', watch: 'never' }, extra.map((m) => ({ ...m, mode: 'read-write' as const, watch: 'never' as const }))); + for (const name of ['ref', ...extra.map((m) => m.name)]) await data(chronicle, 'write', { path: `${name}/draft.ts`, content: 'draft\n' }); + await env.close(chronicle); + + await env.open({}, extra.map((m) => ({ ...m, mode: 'read-only' as const, backend: 'direct' as const }))); + const message = env.alerts[0]!.message; + // The framework's webhook keeps the first 346 characters of an alert over + // 500, with its end (framework.ts, OPS_WEBHOOK_MESSAGE_MAX_CHARS and + // boundFailureText). + assert.ok(message.length > 500, `${message.length}`); + const last = 'Its next start then runs it direct.'; + const at = message.indexOf(last); + assert.ok(at >= 0 && at + last.length <= 340, `${at}`); }); test('a start of the same module decides again whether a mount configured direct runs direct', async (t) => { @@ -870,6 +934,79 @@ test('a start of the same module decides again whether a mount configured direct assert.equal((await data<{ content: string }>(m, 'read', { path: 'ref/drafted.ts' })).content, ' 1\tdisk'); }); +test('whichever of start and initStore comes first, a mount configured direct is decided once the store is given, and said once', async (t) => { + const env = new Env(t); + env.file('drafted.ts', 'disk\n'); + const chronicle = await env.openChronicle(); + await data(chronicle, 'write', { path: 'ref/drafted.ts', content: 'draft\n' }); + await env.close(chronicle); + const config = { mounts: [{ name: 'ref', path: env.ref, mode: 'read-only' as const, backend: 'direct' as const }] }; + const status = async (m: WorkspaceModule) => (await data>>(m, 'status')).ref!.backend; + + // The host's order: start, then initStore. + const host = new WorkspaceModule(config); + try { + await host.start(env.context()); + assert.deepEqual(env.alerts, []); + host.initStore(env.store); + assert.equal(env.alerts.length, 1); + assert.equal(await status(host), 'chronicle'); + assert.equal((await data<{ content: string }>(host, 'read', { path: 'ref/drafted.ts' })).content, ' 1\tdraft'); + } finally { + await host.stop(); + } + + // The store first: decided before start, so a read between the two reads + // the change, and start says so once. + env.alerts = []; + const early = new WorkspaceModule(config); + try { + early.initStore(env.store); + assert.equal(await status(early), 'chronicle'); + assert.equal((await data<{ content: string }>(early, 'read', { path: 'ref/drafted.ts' })).content, ' 1\tdraft'); + await early.start(env.context()); + assert.equal(env.alerts.length, 1); + await data(early, 'sync', { path: 'ref/drafted.ts' }); + } finally { + await early.stop(); + } + + // With nothing held, a mount that ran as 'chronicle' is direct from + // initStore on. + const settled = new WorkspaceModule(config); + try { + settled.initStore(env.store); + assert.equal(await status(settled), 'direct'); + } finally { + await settled.stop(); + } +}); + +test("a decision that can't read the store leaves a mount that ran as 'chronicle' as it was, never direct", async (t) => { + const env = new Env(t); + env.file('drafted.ts', 'disk\n'); + const chronicle = await env.openChronicle(); + await data(chronicle, 'write', { path: 'ref/drafted.ts', content: 'draft\n' }); + await env.close(chronicle); + type Deciding = { heldChanges: () => never }; + const unreadable = () => { throw new Error('the evidence could not be read'); }; + + // At a later start, the earlier decision stands, and the start fails. + const m = await env.open(); + const atStart = t.mock.method(m as unknown as Deciding, 'heldChanges', unreadable); + await assert.rejects(env.restart(m), /the evidence could not be read/); + atStart.mock.restore(); + assert.equal((await data>>(m, 'status')).ref!.backend, 'chronicle'); + assert.equal((await data<{ content: string }>(m, 'read', { path: 'ref/drafted.ts' })).content, ' 1\tdraft'); + + // At initStore, before any decision has been made, it isn't direct. + const fresh = new WorkspaceModule({ mounts: [{ name: 'ref', path: env.ref, mode: 'read-only', backend: 'direct' }] }); + const atInit = t.mock.method(fresh as unknown as Deciding, 'heldChanges', unreadable); + assert.throws(() => fresh.initStore(env.store), /the evidence could not be read/); + atInit.mock.restore(); + assert.equal((await data<{ content: string }>(fresh, 'read', { path: 'ref/drafted.ts' })).content, ' 1\tdraft'); +}); + test('changes held on a branch not selected at the start keep no mount from starting direct', async (t) => { const env = new Env(t); env.file('drafted.ts', 'disk\n'); @@ -946,21 +1083,39 @@ test("a direct mount's listings pass over another mount's directory, naming the }); test('inOrder hands each result on in order, at most `width` at once', async () => { - const running = { now: 0, most: 0 }; + const settle: Array<() => void> = []; + const settled: number[] = []; const order: number[] = []; - const delays = [30, 5, 20, 1, 10, 2, 25, 3]; - await workspaceModule.inOrder(delays.map((_, i) => i), 3, async (i) => { - running.now++; - running.most = Math.max(running.most, running.now); - await new Promise((resolve) => setTimeout(resolve, delays[i])); - running.now--; - return i * 10; + let running = 0; + let most = 0; + const done = workspaceModule.inOrder([0, 1, 2, 3, 4, 5, 6, 7], 3, (i) => { + running++; + most = Math.max(most, running); + return new Promise((resolve) => { + settle[i] = () => { + running--; + settled.push(i); + resolve(i * 10); + }; + }); }, (result, i) => { assert.equal(result, i * 10); order.push(i); }); + // Each window's works settle last to first, so each settles before the + // ones ahead of it are handed on. + const turn = () => new Promise((resolve) => setImmediate(resolve)); + for (const window of [[2, 1, 0], [5, 4, 3], [7, 6]]) { + await turn(); + for (const i of window) { + settle[i]!(); + await turn(); + } + } + await done; + assert.deepEqual(settled, [2, 1, 0, 5, 4, 3, 7, 6]); assert.deepEqual(order, [0, 1, 2, 3, 4, 5, 6, 7]); - assert.equal(running.most, 3); + assert.equal(most, 3); }); test('inOrder lets each result go once it has handed it on', async () => { From dac5bf4cb61b4eddec8ac237b9efe17b58df2080 Mon Sep 17 00:00:00 2001 From: Beth-2709 Date: Mon, 19 Oct 2026 04:45:04 -0400 Subject: [PATCH 05/13] fix(workspace): the direct mount's third review round Answers an internal review of a25a42594, and Esther-2584's read of it. The fallback alert: - The materialize route ends by restoring the mount's direct config, after "then start it again"; it left the mount configured 'chronicle' and read-write before, so its next start ran as configured (found by Esther-2584). - Where an intent the workspace can't read is held, the ways out say only a sync settles one, past the ten paths named too, and its label says only a sync of the path settles it: materialize doesn't, and a sync does where it can read disk's copy. - The lead no longer says why it keeps the mount store-backed, which the mounts and their counts say after the ways out, so with several mounts the fixed head runs 332 characters, inside the webhook's kept start. The direct kind's refusals say "doesn't take" and "give the path ... without it", since ls, glob and grep name directories and write, edit and delete don't read; parsePath's comment gives their reason, the store-backed kind's own refusals, which an empty segment isn't among. Text: types.ts says the refusals are the tools' and the host's reads', and that a generated file is answered, and resolveAbsolutePath maps a path, as on any mount. Tests: a decision whose read of a second mount fails leaves the first as it was; the same module started again on a branch that holds a draft runs as 'chronicle'; the intent sentence past the ten paths named, and within the webhook's kept start beside long mount names. Co-Authored-By: Claude Opus 5.5 --- changelog.d/workspace-direct-mount.added.md | 2 +- src/modules/workspace/index.ts | 28 ++++--- src/modules/workspace/types.ts | 14 ++-- test/workspace-direct.test.ts | 84 +++++++++++++++++---- 4 files changed, 95 insertions(+), 33 deletions(-) diff --git a/changelog.d/workspace-direct-mount.added.md b/changelog.d/workspace-direct-mount.added.md index 47931174b..fabf1112d 100644 --- a/changelog.d/workspace-direct-mount.added.md +++ b/changelog.d/workspace-direct-mount.added.md @@ -4,5 +4,5 @@ - **Its ignore list:** with none in its config, it takes `node_modules`, `dist`, `.git`, `build`, `coverage`, `target`, `__pycache__` and `.venv`. Walks of the whole mount pass over what the list covers, and a path a listing, glob or grep names, or its pattern's directory, is walked all the same, as a sync naming a path walks it on a store-backed mount. `ls` without a path, and `status`, give a mount configured direct its `backend`, as it runs this session, and the list it goes by. - **Its root:** it reads beneath whatever directory is at its path now, with no check against the one a store-backed run of it last agreed with. - **`sync`:** passes over it, listing it under `skipped`. `materialize` refuses it, as it refuses any read-only mount. - - **A mount the store holds changes on that disk lacks** (a draft, a workspace deletion, a conflict, or an intent it can't read, on the branch selected at a start) runs store-backed from that start, so they stay in view. It runs unwatched, and with the direct default ignore list where its config names none. The workspace says so on stderr and in one ops alert (`workspace-mount-not-direct`). The alert first gives the two ways to start such a mount direct: give each change up with a sync of its path, or configure the mount `'chronicle'` and read-write, materialize it, and follow materialize's answer; its next start then runs it direct. Then it names every such mount and what's held there, counting an intent it can't read as one. A mount that ran direct and runs store-backed again takes in what disk changed meanwhile as its listings, searches and syncs reach it, as an unwatched store-backed mount does. + - **A mount the store holds changes on that disk lacks** (a draft, a workspace deletion, a conflict, or an intent it can't read, on the branch selected at a start) runs store-backed from that start, so they stay in view. It runs unwatched, and with the direct default ignore list where its config names none. The workspace says so on stderr and in one ops alert (`workspace-mount-not-direct`). The alert first gives the two ways to start such a mount direct: give each change up with a sync of its path, or configure the mount `'chronicle'` and read-write, materialize it, follow materialize's answer, and restore its direct config; its next start then runs it direct. Where an intent it can't read is held, it says only a sync settles one. Then it names every such mount and what's held there, counting an intent it can't read as one. A mount that ran direct and runs store-backed again takes in what disk changed meanwhile as its listings, searches and syncs reach it, as an unwatched store-backed mount does. - **Tool descriptions:** `ls`, `glob`, `grep` and `sync` gain a sentence about the direct kind only on a host with a mount that runs direct; elsewhere they're unchanged. diff --git a/src/modules/workspace/index.ts b/src/modules/workspace/index.ts index 338321efa..dceda7ea4 100644 --- a/src/modules/workspace/index.ts +++ b/src/modules/workspace/index.ts @@ -1431,13 +1431,16 @@ export class WorkspaceModule implements Module { ? `Mount "${mounts[0]![0]}" holds ${counted(mounts[0]![1])}` : `Mounts ${each.slice(0, -1).join(', ')} and ${each[each.length - 1]} hold`; const held = mounts.flatMap(([name, paths]) => paths.map((h) => `"${name}/${h.path}" (${h.what})`)); + // Materialize doesn't settle an intent the workspace can't read, so the + // ways out say so wherever one is held, past the ten paths named too. + const intents = mounts.some(([, paths]) => paths.some((h) => h.what === UNREADABLE_INTENT)) + ? "Only a sync settles an intent it can't read. " + : ''; const message = (one - ? "A mount configured direct runs as 'chronicle' this session, keeping what the workspace holds there in view. " + - 'To start it direct, settle each held change: ' - : "Mounts configured direct run as 'chronicle' this session, keeping what the workspace holds there in view. " + - 'To start a mount direct, settle each change held at it: ') + + ? "A mount configured direct runs as 'chronicle' this session. To start it direct, settle each held change, then start it again: " + : "Mounts configured direct run as 'chronicle' this session. To start a mount direct, settle each change held at it, then start it again: ") + "sync its path to take disk's copy, or configure the mount 'chronicle' and read-write, materialize it, " + - `and follow materialize's answer. Its next start then runs it direct. ${which}: ${held.slice(0, 10).join(', ')}` + + `follow the answer, and restore its direct config. ${intents}${which}: ${held.slice(0, 10).join(', ')}` + `${held.length > 10 ? `, and ${held.length - 10} more` : ''}.`; console.warn(`[workspace] ${message}`); this.ctx?.notifyOps?.('workspace-mount-not-direct', 'framework', message, { held: Object.fromEntries(mounts.map(([name, paths]) => [name, paths.length])) }); @@ -2260,7 +2263,7 @@ export class WorkspaceModule implements Module { // After a trailing slash a store-backed mount's read finds no file. A // direct mount's would read the file and answer with the slash. if (spelled.mount.direct && spelled.relativePath.endsWith('/')) { - throw new Error(`Path not taken: ${JSON.stringify(path)} ends with "/", which a direct mount's file operations don't read: name the file without it`); + throw new Error(`Path not taken: ${JSON.stringify(path)} ends with "/", which a direct mount's file operations don't take: give the file's path without it`); } const parsed = this.refuseNested(spelled); // A path can also reach a nested mount's directory through a symlink on @@ -2319,11 +2322,11 @@ export class WorkspaceModule implements Module { */ private async parsePath(path: string): Promise<{ mount: MountState; relativePath: string; path: string }> { const given = this.parsePathAsGiven(path); - // A store-backed mount's store refuses a `.` or `..` segment as an - // invalid tree path. A direct mount reads disk, which would take it and - // answer with the path so spelled, giving one file several names. + // A direct mount refuses what a store-backed mount's store refuses as an + // invalid tree path, a `.` or `..` segment, which disk would otherwise + // take, so neither kind takes a path the other refuses. if (given.mount.direct && given.relativePath.split('/').some((segment) => segment === '.' || segment === '..')) { - throw new Error(`Path not taken: ${JSON.stringify(path)} has a "." or ".." segment, which a direct mount doesn't read: name the file from the mount's root without it`); + throw new Error(`Path not taken: ${JSON.stringify(path)} has a "." or ".." segment, which a direct mount doesn't take: give the path from the mount's root without it`); } const relativePath = await diskSpelling(resolve(given.mount.config.path), given.relativePath, given.mount.direct ? undefined : this.entriesOf(given.mount)); return { @@ -4640,9 +4643,10 @@ const DIRECT_ROOT_REPLACED = 'the mount root was replaced as the file was opened /** * How the alert that a mount configured direct runs as 'chronicle' names an * intent the workspace can't read: materialize doesn't select one with no - * workspace entry, and a sync of its path replaces it. + * workspace entry, or settle one with an entry, and a sync of its path + * replaces it where it can read disk's copy. */ -const UNREADABLE_INTENT = "its workspace intent can't be read; a sync of the path settles it"; +const UNREADABLE_INTENT = "its workspace intent can't be read, and only a sync of the path settles it"; /** * A mount's watch mode as it runs. A mount configured direct watches diff --git a/src/modules/workspace/types.ts b/src/modules/workspace/types.ts index 8cfa7cd69..4114a823a 100644 --- a/src/modules/workspace/types.ts +++ b/src/modules/workspace/types.ts @@ -32,12 +32,14 @@ export interface MountConfig { * `watchRootPollMs`, `wakeOnChange` or `autoMaterialize`: each of those * would never act, and the workspace refuses the mount. Its `ignore` * has a default (see there). Paths, links, other mounts' directories, - * size limits and the walk cap hold as on a 'chronicle' mount. It - * refuses a path with a `.` or `..` segment, which a 'chronicle' - * mount's store refuses, and a file's path ending in `/`, where a - * 'chronicle' mount's read finds no file. Its tools read beneath - * whatever directory is at its root, with no check against the one a - * 'chronicle' run of it last agreed with. + * size limits and the walk cap hold as on a 'chronicle' mount. Its + * tools, and the host's reads of its files, refuse a path with a `.` or + * `..` segment, which a 'chronicle' mount's store refuses, and a file's + * path ending in `/`, where a 'chronicle' mount's read finds no file; + * a generated file is answered, and `resolveAbsolutePath` maps a path, + * as on any mount. Its tools read beneath whatever directory is at its + * root, with no check against the one a 'chronicle' run of it last + * agreed with. * * A mount configured 'direct' runs as 'chronicle' from a start that finds * the store holding changes there that disk lacks, on the branch selected diff --git a/test/workspace-direct.test.ts b/test/workspace-direct.test.ts index 39f24362a..c8104e7ec 100644 --- a/test/workspace-direct.test.ts +++ b/test/workspace-direct.test.ts @@ -616,7 +616,7 @@ test("a direct mount refuses a '.' or '..' segment, and a file operation's trail env.file('dir/x.ts', 'alpha x\n'); env.file('dot.png', PNG); const m = await env.open(); - const segment = (path: string) => `Path not taken: ${JSON.stringify(path)} has a "." or ".." segment, which a direct mount doesn't read: name the file from the mount's root without it`; + const segment = (path: string) => `Path not taken: ${JSON.stringify(path)} has a "." or ".." segment, which a direct mount doesn't take: give the path from the mount's root without it`; for (const path of ['ref/./a.ts', 'ref/dir/../a.ts', 'ref/dir/..']) assert.equal(await failure(m, 'read', { path }), segment(path), path); assert.equal(await failure(m, 'ls', { path: 'ref/./dir' }), segment('ref/./dir')); assert.equal(await failure(m, 'glob', { pattern: '*.ts', path: 'ref/dir/..' }), segment('ref/dir/..')); @@ -624,7 +624,7 @@ test("a direct mount refuses a '.' or '..' segment, and a file operation's trail // A path that leaves the mount is refused as traversal, as before. assert.match(await failure(m, 'read', { path: 'ref/../outside.txt' }), /^Path traversal detected/); - const slash = (path: string) => `Path not taken: ${JSON.stringify(path)} ends with "/", which a direct mount's file operations don't read: name the file without it`; + const slash = (path: string) => `Path not taken: ${JSON.stringify(path)} ends with "/", which a direct mount's file operations don't take: give the file's path without it`; assert.equal(await failure(m, 'read', { path: 'ref/a.ts/' }), slash('ref/a.ts/')); assert.equal(await failure(m, 'read_image', { path: 'ref/dot.png/' }), slash('ref/dot.png/')); assert.deepEqual(await m.readBinary('ref/a.ts/'), { error: slash('ref/a.ts/') }); @@ -769,9 +769,9 @@ test("a mount the store holds changes on that disk lacks runs as 'chronicle', sa const warned = t.mock.method(console, 'warn', () => {}); const held = await env.open({ walkCap: 1 }); warned.mock.restore(); - const message = "A mount configured direct runs as 'chronicle' this session, keeping what the workspace holds there in view. " - + "To start it direct, settle each held change: sync its path to take disk's copy, or configure the mount 'chronicle' " - + "and read-write, materialize it, and follow materialize's answer. Its next start then runs it direct. " + const message = "A mount configured direct runs as 'chronicle' this session. To start it direct, settle each held change, then " + + "start it again: sync its path to take disk's copy, or configure the mount 'chronicle' and read-write, materialize it, " + + "follow the answer, and restore its direct config. " + 'Mount "ref" holds 4 changes that disk lacks: "ref/both.ts" (in conflict), "ref/deleted.ts" (deleted in the workspace), ' + '"ref/dot.img" (in conflict), "ref/drafted.ts" (changed in the workspace).'; assert.deepEqual(env.alerts, [{ kind: 'workspace-mount-not-direct', agent: 'framework', message, data: { held: { ref: 4 } } }]); @@ -825,13 +825,30 @@ test("the changes that keep a mount from starting direct are named ten at a time await env.open(); assert.equal(env.alerts.length, 1); - const unreadable = (path: string) => `"ref/${path}" (its workspace intent can't be read; a sync of the path settles it)`; + const unreadable = (path: string) => `"ref/${path}" (its workspace intent can't be read, and only a sync of the path settles it)`; const shown = [unreadable('aa.ts'), unreadable('d00.ts'), ...Array.from({ length: 8 }, (_, i) => `"ref/d0${i + 1}.ts" (changed in the workspace)`)]; // An intent it can't read is counted as one, not as a change. assert.ok(env.alerts[0]!.message.endsWith(`Mount "ref" holds 10 changes that disk lacks and 2 intents it can't read: ${shown.join(', ')}, and 2 more.`), env.alerts[0]!.message); assert.deepEqual(env.alerts[0]!.data, { held: { ref: 12 } }); }); +test("the ways out say only a sync settles an intent it can't read, where one is held past the ten paths named too", async (t) => { + const env = new Env(t); + const chronicle = await env.openChronicle(); + for (let i = 0; i < 11; i++) await data(chronicle, 'write', { path: `ref/d${String(i).padStart(2, '0')}.ts`, content: `${i}\n` }); + await env.close(chronicle); + const sentence = "and restore its direct config. Only a sync settles an intent it can't read. Mount \"ref\" holds"; + + await env.close(await env.open()); + assert.ok(!env.alerts[0]!.message.includes('Only a sync'), env.alerts[0]!.message); + // One that sorts after every change, so it isn't among the ten named. + env.store.treeSet('workspace/ref/intent', 'zz.ts', { blobHash: 'f'.repeat(64), size: 2, mode: 0 }); + env.alerts = []; + await env.open(); + assert.ok(!env.alerts[0]!.message.includes('"ref/zz.ts"'), env.alerts[0]!.message); + assert.ok(env.alerts[0]!.message.includes(sentence), env.alerts[0]!.message); +}); + test('each kind of change the store holds keeps a mount configured direct from starting direct on its own, until a sync of its path settles it', async (t) => { const cases: Array<[string, (env: Env) => Promise, string, string]> = [ ['a workspace deletion', async (env) => { @@ -848,7 +865,7 @@ test('each kind of change the store holds keeps a mount configured direct from s ["an intent that can't be read", async (env) => { await env.close(await env.openChronicle()); env.store.treeSet('workspace/ref/intent', 'u.ts', { blobHash: 'f'.repeat(64), size: 2, mode: 0 }); - }, "1 intent it can't read", `"ref/u.ts" (its workspace intent can't be read; a sync of the path settles it)`], + }, "1 intent it can't read", `"ref/u.ts" (its workspace intent can't be read, and only a sync of the path settles it)`], ]; for (const [kind, hold, counted, named] of cases) { const env = new Env(t); @@ -880,9 +897,9 @@ test('mounts configured direct that the store holds changes on are named in one const both = await env.open({}, [{ name: 'other', path: other, mode: 'read-only', backend: 'direct' }]); assert.equal(env.alerts.length, 1); - assert.equal(env.alerts[0]!.message, "Mounts configured direct run as 'chronicle' this session, keeping what the workspace holds there in view. " - + "To start a mount direct, settle each change held at it: sync its path to take disk's copy, or configure the mount 'chronicle' " - + "and read-write, materialize it, and follow materialize's answer. Its next start then runs it direct. " + assert.equal(env.alerts[0]!.message, "Mounts configured direct run as 'chronicle' this session. To start a mount direct, settle each change " + + "held at it, then start it again: sync its path to take disk's copy, or configure the mount 'chronicle' and read-write, " + + "materialize it, follow the answer, and restore its direct config. " + 'Mounts "ref" (1 change that disk lacks) and "other" (2 changes that disk lacks) hold: "ref/a.ts" (changed in the workspace), ' + '"other/b.ts" (changed in the workspace), "other/c.ts" (changed in the workspace).'); assert.deepEqual(env.alerts[0]!.data, { held: { ref: 1, other: 2 } }); @@ -904,6 +921,8 @@ test("the alert's ways out come within the start a webhook keeps of it, however const chronicle = await env.open({ backend: 'chronicle', mode: 'read-write', watch: 'never' }, extra.map((m) => ({ ...m, mode: 'read-write' as const, watch: 'never' as const }))); for (const name of ['ref', ...extra.map((m) => m.name)]) await data(chronicle, 'write', { path: `${name}/draft.ts`, content: 'draft\n' }); await env.close(chronicle); + // An intent it can't read too, whose sentence is the last of the ways out. + env.store.treeSet('workspace/ref/intent', 'u.ts', { blobHash: 'f'.repeat(64), size: 2, mode: 0 }); await env.open({}, extra.map((m) => ({ ...m, mode: 'read-only' as const, backend: 'direct' as const }))); const message = env.alerts[0]!.message; @@ -911,7 +930,7 @@ test("the alert's ways out come within the start a webhook keeps of it, however // 500, with its end (framework.ts, OPS_WEBHOOK_MESSAGE_MAX_CHARS and // boundFailureText). assert.ok(message.length > 500, `${message.length}`); - const last = 'Its next start then runs it direct.'; + const last = "Only a sync settles an intent it can't read."; const at = message.indexOf(last); assert.ok(at >= 0 && at + last.length <= 340, `${at}`); }); @@ -1007,6 +1026,38 @@ test("a decision that can't read the store leaves a mount that ran as 'chronicle assert.equal((await data<{ content: string }>(fresh, 'read', { path: 'ref/drafted.ts' })).content, ' 1\tdraft'); }); +test('a decision that fails at a later mount sets none of the mounts it decided before it', async (t) => { + const env = new Env(t); + const other = join(env.root, 'other'); + mkdirSync(other); + env.file('drafted.ts', 'disk\n'); + const chronicle = await env.open({ backend: 'chronicle', mode: 'read-write', watch: 'never' }, [{ name: 'other', path: other, mode: 'read-write', watch: 'never' }]); + await data(chronicle, 'write', { path: 'ref/drafted.ts', content: 'draft\n' }); + await data(chronicle, 'ls', { path: 'other' }); + await env.close(chronicle); + + const m = await env.open({}, [{ name: 'other', path: other, mode: 'read-only', backend: 'direct' }]); + const status = async () => { + const all = await data>>(m, 'status'); + return [all.ref!.backend, all.other!.backend]; + }; + assert.deepEqual(await status(), ['chronicle', 'direct']); + // "ref" is settled, so a decision would start it direct, but reading + // "other", decided after it, fails. + await data(m, 'sync', { path: 'ref/drafted.ts' }); + type Deciding = { heldChanges: (state: { config: { name: string } }) => unknown }; + const original = (m as unknown as Deciding).heldChanges; + const failing = t.mock.method(m as unknown as Deciding, 'heldChanges', function (this: unknown, state: { config: { name: string } }) { + if (state.config.name === 'other') throw new Error('the evidence could not be read'); + return original.call(this, state); + }); + await assert.rejects(env.restart(m), /the evidence could not be read/); + failing.mock.restore(); + assert.deepEqual(await status(), ['chronicle', 'direct']); + await env.restart(m); + assert.deepEqual(await status(), ['direct', 'direct']); +}); + test('changes held on a branch not selected at the start keep no mount from starting direct', async (t) => { const env = new Env(t); env.file('drafted.ts', 'disk\n'); @@ -1018,12 +1069,17 @@ test('changes held on a branch not selected at the start keep no mount from star await env.close(chronicle); env.store.switchBranch(main); - const direct = await env.open(); + const m = await env.open(); assert.deepEqual(env.alerts, [], 'the branch selected holds none'); - await env.close(direct); + assert.equal((await data>>(m, 'status')).ref!.backend, 'direct'); + // The same module, started again on the branch that holds it, runs + // store-backed and says so. + await m.stop(); env.store.switchBranch('side'); - await env.open(); + await m.start(env.context()); assert.equal(env.alerts.length, 1, 'the branch that holds it keeps the mount store-backed'); + assert.equal((await data>>(m, 'status')).ref!.backend, 'chronicle'); + assert.equal((await data<{ content: string }>(m, 'read', { path: 'ref/drafted.ts' })).content, ' 1\tdraft'); }); test("a mount configured direct with no ignore list walks by the direct kind's default while it runs as 'chronicle'", async (t) => { From 6c6cef46f0522404676a4b3f7668198c27fced3e Mon Sep 17 00:00:00 2001 From: Beth-2709 Date: Mon, 19 Oct 2026 05:28:10 -0400 Subject: [PATCH 06/13] test(workspace): the 16-at-a-time test holds the files' calls until sixteen are under way The test counted the most opens (for grep) and lstats (for the listing) under way at once, holding each for a turn. Before its file's lstat, each of the listing's observations resolves the file's parent directory, so the count depended on how those steps interleaved: CI's macOS Node 20 leg counted 15 for the listing, and holding every sixteenth realpath for 200 ms gives 15 on Linux too. inOrder starts all sixteen observations before it awaits the first. The helper now counts only the files' own calls, and holds them until sixteen are under way, and 50 ms longer, in which a reader wider than sixteen would start more. So the count shows the reader's width however the steps before each call interleave: with that same hold, it counts 16. A narrower reader is let go after 5 seconds, so it fails rather than hangs. Co-Authored-By: Claude Opus 5.5 --- test/workspace-direct.test.ts | 48 +++++++++++++++++++++++++---------- 1 file changed, 34 insertions(+), 14 deletions(-) diff --git a/test/workspace-direct.test.ts b/test/workspace-direct.test.ts index c8104e7ec..e23b4172a 100644 --- a/test/workspace-direct.test.ts +++ b/test/workspace-direct.test.ts @@ -120,20 +120,38 @@ async function withFsPromise(name: 'lstat' | 'open', make: (original: (...arg } } -/** The most calls of `node:fs/promises`'s `name` under way at once while `fn` runs, each held a turn so the others can start. */ -async function mostAtOnce(name: 'lstat' | 'open', fn: () => Promise): Promise { +/** + * The most calls of `node:fs/promises`'s `name` on the files `counted` names + * under way at once while `fn` runs. The first are held until `width` of + * them are under way, and a moment longer, in which a reader wider than + * `width` would start more, so a reader `width` wide shows as `width` however + * fast the file system answers each step before the call. A narrower one is + * let go after 5 seconds, so it fails rather than hangs. + */ +async function mostAtOnce(name: 'lstat' | 'open', width: number, counted: RegExp, fn: () => Promise): Promise { let now = 0; let most = 0; - await withFsPromise(name, (original) => async (...args: any[]) => { - now++; - most = Math.max(most, now); - try { - await new Promise((resolve) => setImmediate(resolve)); - return await original(...args); - } finally { - now--; - } - }, fn); + let release!: () => void; + const released = new Promise((resolve) => { release = resolve; }); + const fallback = setTimeout(release, 5000); + let settle: NodeJS.Timeout | undefined; + try { + await withFsPromise(name, (original) => async (...args: any[]) => { + if (!counted.test(String(args[0]))) return original(...args); + now++; + most = Math.max(most, now); + if (now === width && settle === undefined) settle = setTimeout(release, 50); + try { + await released; + return await original(...args); + } finally { + now--; + } + }, fn); + } finally { + clearTimeout(fallback); + clearTimeout(settle); + } return most; } @@ -545,8 +563,10 @@ test('a direct grep and listing read 16 files at a time', async (t) => { const env = new Env(t); for (let i = 0; i < 40; i++) env.file(`f${String(i).padStart(2, '0')}.txt`, 'needle\n'); const m = await env.open(); - assert.equal(await mostAtOnce('open', () => data(m, 'grep', { pattern: 'needle' })), 16); - assert.equal(await mostAtOnce('lstat', () => data(m, 'ls', { path: 'ref', recursive: true })), 16); + // The files' own calls: a walk's are on directories. + const files = /[\\/]f\d\d\.txt$/; + assert.equal(await mostAtOnce('open', 16, files, () => data(m, 'grep', { pattern: 'needle' })), 16); + assert.equal(await mostAtOnce('lstat', 16, files, () => data(m, 'ls', { path: 'ref', recursive: true })), 16); }); test("a direct mount's reads close every file they open, whatever they answer", async (t) => { From 6c52d76bd561d0de9ebf40340e3fe3c6934736fc Mon Sep 17 00:00:00 2001 From: Beth-2709 Date: Mon, 19 Oct 2026 09:48:33 -0400 Subject: [PATCH 07/13] fix(workspace): a direct mount whose root has moved nests what lies beneath it now A direct mount reads beneath whatever directory is at its root at each call, with no check against an earlier root, but which mounts are nested in it was placed at start. So a root repointed through a link read another mount's directory as its own. Now, where a check finds a direct mount's root elsewhere than the last start placed it, it nests in it what a start would nest there now (movedNesting): a mount configured beneath its path, at its configured directory and at where that directory resolves beneath the root, and any other mount whose root, as the last start placed it, lies beneath where its root lies now, each directory also as disk spells it. Each check judges a path by the root it resolved. A file tool's parse resolves the root once, for its lexical check and for where the path leads beneath that root. A read, a walk and grep's check of a path it's given each resolve it for themselves, so a root repointed during a call can only have a later check refuse what an earlier one let through. A root that's gone is no move, and its read says it's unavailable. Worked out from the configured paths and the start's placement, it resolves nothing but directories beneath the moved root and changes nothing shared: no other mount's placement, watcher or status moves, and a share that doesn't answer holds only its own mount's calls, unless a configured mount's directory beneath the moved root is a link into that share, where that mount's files lie. What waits for the next start is a mount configured elsewhere whose own root has moved too, its spellings through other links, and the other mounts' own placements. So a direct root moved into another mount's directory is that mount's too, as a nested mount's root moved while running stays placed. resolveAbsolutePath, being synchronous, keeps the start's placement, and its doc says so. Where the alert names an intent the workspace can't read, it now says to sync that path, which settles it, in place of saying only a sync does: a forced materialize settles one where the workspace holds a copy there, but where it holds none, materialize neither selects nor names it. The intent's label says a sync of the path settles it or says why not, since where disk's file can't be read the sync skips it and says so. The 16-at-a-time test holds the sixteenth call half a second, so a wider reader's next call can arrive under load; the write tools' test checks materialize too; types.ts says which wakeOnChange and autoMaterialize a direct mount refuses, and the changelog that materialize passes over a direct mount, as any read-only one. Co-Authored-By: Claude Opus 5.5 --- changelog.d/workspace-direct-mount.added.md | 6 +- src/modules/workspace/index.ts | 156 +++++-- src/modules/workspace/types.ts | 14 +- test/workspace-direct.test.ts | 424 +++++++++++++++++++- 4 files changed, 552 insertions(+), 48 deletions(-) diff --git a/changelog.d/workspace-direct-mount.added.md b/changelog.d/workspace-direct-mount.added.md index fabf1112d..869b38375 100644 --- a/changelog.d/workspace-direct-mount.added.md +++ b/changelog.d/workspace-direct-mount.added.md @@ -2,7 +2,7 @@ - **Spellings it refuses:** a path with a `.` or `..` segment, which a store-backed mount's store refuses as an invalid tree path, and, for `read`, `read_image` and the host's binary and disk reads, a path ending in `/`, after which a store-backed mount's `read` finds no file. It says why in words of its own. A directory's path may end in `/`. - **What its listings say:** each file with its `size`, and no state or MIME type, or, for a link the mount doesn't follow or another entry it doesn't take, with why as `note`. A listing looks at each entry with `lstat`, so it lists a file it can't open with its size, and reading that file says why. `grep`'s results say `version: 'disk'`, and it lists under `skipped` a binary file, one over the size limit and one it doesn't read, each with why. `read` refuses a binary file and one over the size limit, as a store-backed mount's `read` refuses one it never took in from disk. - **Its ignore list:** with none in its config, it takes `node_modules`, `dist`, `.git`, `build`, `coverage`, `target`, `__pycache__` and `.venv`. Walks of the whole mount pass over what the list covers, and a path a listing, glob or grep names, or its pattern's directory, is walked all the same, as a sync naming a path walks it on a store-backed mount. `ls` without a path, and `status`, give a mount configured direct its `backend`, as it runs this session, and the list it goes by. - - **Its root:** it reads beneath whatever directory is at its path now, with no check against the one a store-backed run of it last agreed with. - - **`sync`:** passes over it, listing it under `skipped`. `materialize` refuses it, as it refuses any read-only mount. - - **A mount the store holds changes on that disk lacks** (a draft, a workspace deletion, a conflict, or an intent it can't read, on the branch selected at a start) runs store-backed from that start, so they stay in view. It runs unwatched, and with the direct default ignore list where its config names none. The workspace says so on stderr and in one ops alert (`workspace-mount-not-direct`). The alert first gives the two ways to start such a mount direct: give each change up with a sync of its path, or configure the mount `'chronicle'` and read-write, materialize it, follow materialize's answer, and restore its direct config; its next start then runs it direct. Where an intent it can't read is held, it says only a sync settles one. Then it names every such mount and what's held there, counting an intent it can't read as one. A mount that ran direct and runs store-backed again takes in what disk changed meanwhile as its listings, searches and syncs reach it, as an unwatched store-backed mount does. + - **Its root:** it reads beneath whatever directory is at its path now, with no check against the one a store-backed run of it last agreed with. Where its root now lies elsewhere than the last start placed it, as when a link it's configured through is repointed, it nests what a start would nest there now: a mount configured beneath its path, where that path leads, and a mount whose root lies beneath where its root lies now. Their files are those mounts'. A mount configured elsewhere whose own root has moved since, and the other mounts' own placements, wait for the next start, so a direct root moved into another mount's directory is still that mount's directory too. + - **`sync`:** passes over it, listing it under `skipped`. `materialize` passes over it too, naming it under `skipped`, as it does any read-only mount. + - **A mount the store holds changes on that disk lacks** (a draft, a workspace deletion, a conflict, or an intent it can't read, on the branch selected at a start) runs store-backed from that start, so they stay in view. It runs unwatched, and with the direct default ignore list where its config names none. The workspace says so on stderr and in one ops alert (`workspace-mount-not-direct`). The alert first gives the two ways to start such a mount direct: give each change up with a sync of its path, or configure the mount `'chronicle'` and read-write, materialize it, follow materialize's answer, and restore its direct config; its next start then runs it direct. Where an intent it can't read is held, it says to sync that path. Then it names every such mount and what's held there, counting an intent it can't read as one. A mount that ran direct and runs store-backed again takes in what disk changed meanwhile as its listings, searches and syncs reach it, as an unwatched store-backed mount does. - **Tool descriptions:** `ls`, `glob`, `grep` and `sync` gain a sentence about the direct kind only on a host with a mount that runs direct; elsewhere they're unchanged. diff --git a/src/modules/workspace/index.ts b/src/modules/workspace/index.ts index 46a95a580..1874e0d5e 100644 --- a/src/modules/workspace/index.ts +++ b/src/modules/workspace/index.ts @@ -842,6 +842,8 @@ export class WorkspaceModule implements Module { private otherMounts = new Map(); /** Whether each mount's root folds case (foldsCase), where disk has it, as the last start read it. */ private rootFoldsCase = new Map(); + /** Where the last start placed each mount's root (readNesting's `reals`). */ + private placedRoots = new Map(); /** * Whether a file's mode is observed, taken in and applied (MountRuntime's * `modesObserved`): not on Windows, which reports no POSIX modes. A test @@ -1071,9 +1073,12 @@ export class WorkspaceModule implements Module { * that against (MountConfig.backend), and its root as realpath resolves it, * or as given where that fails, as a pass's runtime takes it. */ - private async diskView(mount: MountState): Promise<{ view: MountView; rootReal: string }> { + private async diskView(mount: MountState): Promise<{ view: MountView; rootReal: string; moved: NestedMount[] | null }> { const root = resolve(mount.config.path); - return { view: this.viewOf(mount, undefined), rootReal: await realpath(root).catch(() => root) }; + const rootReal = await realpath(root).catch(() => root); + const view = this.viewOf(mount, undefined); + const moved = await this.movedNesting(mount, rootReal); + return { view: moved === null ? view : { ...view, nested: moved, others: moved.map((n) => n.dir) }, rootReal, moved }; } /** The view reconciliation needs of a mount, refreshed per pass. */ @@ -1172,6 +1177,7 @@ export class WorkspaceModule implements Module { if (this.stopsBegun !== stops) return; this.otherMounts = nesting; this.rootFoldsCase = rootFoldsCase; + this.placedRoots = reals; this.ctx = ctx; this.stopped = false; @@ -1435,10 +1441,12 @@ export class WorkspaceModule implements Module { ? `Mount "${mounts[0]![0]}" holds ${counted(mounts[0]![1])}` : `Mounts ${each.slice(0, -1).join(', ')} and ${each[each.length - 1]} hold`; const held = mounts.flatMap(([name, paths]) => paths.map((h) => `"${name}/${h.path}" (${h.what})`)); - // Materialize doesn't settle an intent the workspace can't read, so the - // ways out say so wherever one is held, past the ten paths named too. + // Materialize settles an intent the workspace can't read only where the + // workspace holds a copy there, and says nothing of one it doesn't, so + // the ways out say what does wherever one is held, past the ten paths + // named too. const intents = mounts.some(([, paths]) => paths.some((h) => h.what === UNREADABLE_INTENT)) - ? "Only a sync settles an intent it can't read. " + ? "Sync the path of an intent it can't read. " : ''; const message = (one ? "A mount configured direct runs as 'chronicle' this session. To start it direct, settle each held change, then start it again: " @@ -1961,6 +1969,10 @@ export class WorkspaceModule implements Module { * path for resolution-only purposes (write targets, nonexistent paths, * display); for reading file content, use {@link readFileFromDisk}, which * enforces the mount's `followSymlinks` policy and canonical containment. + * It judges other mounts' directories by where the last start placed them, + * so for a direct mount whose root has moved since, it can differ from + * `readFileFromDisk`, which takes the nesting where the root lies now + * (movedNesting), about which paths lie in another mount's directory. * * @deprecated for direct filesystem reads — call `readFileFromDisk()`. */ @@ -2277,6 +2289,72 @@ export class WorkspaceModule implements Module { return (this.otherMounts.get(mountName) ?? []).flatMap((o) => o.spellings.map((s) => ({ dir: s.dir, name: o.name, real: o.real }))); } + /** + * The mounts nested in `mount` as a call of it that resolved its root to + * `rootReal` sees them: movedNesting where that differs from where the + * last start placed it, the start's placement (nestedOf) otherwise. + */ + private async nestedFor(mount: MountState, rootReal: string): Promise { + return (await this.movedNesting(mount, rootReal)) ?? this.nestedOf(mount.config.name); + } + + /** + * The mounts nested in a direct mount whose root a check resolved to + * `rootReal`, where that's elsewhere than the last start placed it, as when + * a link it's configured through is repointed; null where the start's + * placement holds, or where nothing is at `rootReal`, whose read says so. A + * direct mount reads beneath whatever directory is at its root, so each + * check judges a path by the root it resolved: a file tool's parse resolves + * it once for both its checks, and a read, a walk and grep's check of a + * path it's given each resolve it for themselves, so a root repointed + * during a call can only have a later check refuse what an earlier one let + * through. A mount configured beneath its configured path lies where that + * path now leads, as a start would place it: at its configured directory, + * and where that directory resolves beneath `rootReal` (realBeneath), its + * real root, each also as disk spells it (diskSpelling). Any other mount + * lies where the last start placed it, and is nested where that lies + * beneath `rootReal`. It resolves nothing but directories beneath this + * mount's own root, so a share elsewhere that doesn't answer holds only its + * own calls, unless such a directory is a link into it, where that mount's + * files lie. What it leaves to the next start: a mount configured elsewhere + * whose own root has moved since, as by a second link repointed, that + * mount's spellings through other links, and the other mounts' own + * placements, so a direct root moved into another mount's directory is + * that mount's too, as a nested mount's root moved while running stays + * placed. + */ + private async movedNesting(mount: MountState, rootReal: string): Promise { + const placed = this.placedRoots.get(mount.config.name); + if (!mount.direct || placed === undefined || placed === rootReal) return null; + if (!(await stat(rootReal).then(() => true, () => false))) return null; + const outerPath = resolve(mount.config.path); + const nested: NestedMount[] = []; + // A real root that holds this mount's own would claim its every path, so + // such a mount is nested by its directories alone, as a start nests it. + const at = (dir: string, name: string, real: string): NestedMount => (isContainedPath(real, rootReal) ? { dir, name } : { dir, name, real }); + for (const inner of this.config.mounts) { + if (inner.name === mount.config.name) continue; + const configured = nestedDir(outerPath, resolve(inner.path)); + if (configured !== undefined) { + const real = await realBeneath(join(rootReal, ...configured.split('/'))); + nested.push(at(configured, inner.name, real)); + const there = nestedDir(rootReal, real); + if (there !== undefined && there !== configured) nested.push(at(there, inner.name, real)); + continue; + } + const real = this.placedRoots.get(inner.name); + const fromStart = real === undefined ? undefined : nestedDir(rootReal, real); + if (fromStart !== undefined) nested.push(at(fromStart, inner.name, real!)); + } + // Each directory as disk spells it too, as a start adds it, so a path + // parsePath has respelled is judged on a disk that folds names. + for (const n of [...nested]) { + const dir = await diskSpelling(rootReal, n.dir); + if (!nested.some((m) => m.name === n.name && m.dir === dir)) nested.push({ ...n, dir }); + } + return nested; + } + /** * The innermost mount nested in `mountName` whose directory holds a path * of that mount, judged lexically where the path lands (`.`, `..` and @@ -2284,12 +2362,17 @@ export class WorkspaceModule implements Module { * outer mount isn't followed), with the path to use there; undefined when * no nested mount's directory holds it. */ - private holderOf(mountName: string, relativePath: string): { name: string; use: string } | undefined { + private holderOf(mountName: string, relativePath: string, moved: NestedMount[] | null = null): { name: string; use: string } | undefined { const mount = this.mounts.get(mountName); if (!mount) return undefined; const resolved = resolve(mount.config.path, relativePath); - const holder = (this.otherMounts.get(mountName) ?? []) - .flatMap((o) => o.spellings.filter((s) => isContainedPath(s.root, resolved)).map((s) => ({ name: o.name, root: s.root }))) + // A direct mount whose root a call found moved is judged by that call's + // nesting (movedNesting), each directory beneath its configured path. + const spelled = moved !== null + ? moved.map((n) => ({ name: n.name, root: join(resolve(mount.config.path), ...n.dir.split('/')) })) + : (this.otherMounts.get(mountName) ?? []).flatMap((o) => o.spellings.map((s) => ({ name: o.name, root: s.root }))); + const holder = spelled + .filter((s) => isContainedPath(s.root, resolved)) .sort((a, b) => b.root.length - a.root.length)[0]; if (!holder) return undefined; const rest = relative(holder.root, resolved).split(sep).join('/'); @@ -2301,8 +2384,8 @@ export class WorkspaceModule implements Module { * mount and the path to use: for a sync or a push, which of them leaves * it; for a listing, whose the path is. */ - private otherMountNote(mountName: string, relativePath: string, by: 'sync' | 'push' | 'list'): string { - const holder = this.holderOf(mountName, relativePath); + private otherMountNote(mountName: string, relativePath: string, by: 'sync' | 'push' | 'list', moved: NestedMount[] | null = null): string { + const holder = this.holderOf(mountName, relativePath, moved); const what = by === 'sync' ? ', which no sync of this mount takes in' : by === 'push' ? ', which no push of this mount writes' : ''; return holder ? `it lies in mount "${holder.name}", nested in this one${what}: use "${holder.use}"` @@ -2323,12 +2406,18 @@ export class WorkspaceModule implements Module { if (spelled.mount.direct && spelled.relativePath.endsWith('/')) { throw new Error(`Path not taken: ${JSON.stringify(path)} ends with "/", which a direct mount's file operations don't take: give the file's path without it`); } - const parsed = this.refuseNested(spelled); + // A direct mount's root is resolved once here, so both checks below + // judge the path by one root and its nesting (movedNesting). + const mount = spelled.mount; + const root = resolve(mount.config.path); + const rootReal = mount.direct ? await realpath(root).catch(() => root) : root; + const moved = mount.direct ? await this.movedNesting(mount, rootReal) : null; + const parsed = this.refuseNested(spelled, moved); // A path can also reach a nested mount's directory through a symlink on // its way. No push of this mount writes it there and no read reads it, // so it's refused here, before anything is stored, and the checks where // disk is acted on refuse it again. - const refusal = await this.leadsIntoNested(parsed.mount, parsed.relativePath, parsed.path); + const refusal = await this.leadsIntoNested(mount, parsed.relativePath, parsed.path, moved ?? this.nestedOf(mount.config.name), rootReal); if (refusal) throw refusal; return parsed; } @@ -2340,10 +2429,9 @@ export class WorkspaceModule implements Module { * (linkWords), as the readers and pushes give it. A mount with nothing * nested has nothing to find, so it pays nothing. */ - private async leadsIntoNested(mount: MountState, relativePath: string, path: string): Promise { - const nested = this.nestedOf(mount.config.name); + private async leadsIntoNested(mount: MountState, relativePath: string, path: string, nested = this.nestedOf(mount.config.name), root = resolve(mount.config.path)): Promise { if (nested.length === 0) return null; - const holder = nestedAtReal(await realBeneath(resolve(mount.config.path, relativePath)), nested); + const holder = nestedAtReal(await realBeneath(resolve(root, relativePath)), nested); if (!holder) return null; if (mount.config.followSymlinks !== true) { const found = await linkOnTheWay({ root: mount.config.path, followSymlinks: false }, relativePath); @@ -2352,9 +2440,9 @@ export class WorkspaceModule implements Module { return new NestedMountError(path, mount.config.name, holder.name, holder.use, 'leads into'); } - /** A parsed path, refused if it lies in a nested mount's directory (parseFilePath). */ - private refuseNested

(parsed: P): P { - const holder = this.holderOf(parsed.mount.config.name, parsed.relativePath); + /** A parsed path, refused if it lies in a nested mount's directory (parseFilePath), by a call's moved nesting where it has one. */ + private refuseNested

(parsed: P, moved: NestedMount[] | null = null): P { + const holder = this.holderOf(parsed.mount.config.name, parsed.relativePath, moved); if (holder) throw new NestedMountError(parsed.path, parsed.mount.config.name, holder.name, holder.use); return parsed; } @@ -2634,8 +2722,10 @@ export class WorkspaceModule implements Module { // directories on the way, wherever it leads. Asked again of the path's // resolution after the open, below. const linkView = { root: mount.config.path, followSymlinks: follow }; + // Every check below judges by the root this read opens under. + const nested = await this.nestedFor(mount, realMountRoot); const onTheWay = await linkOnTheWay(linkView, relativePath, 'parents'); - if (onTheWay !== null) throw await linkReadError(mount, onTheWay, mountPrefixedPath, this.nestedOf(mount.config.name)); + if (onTheWay !== null) throw await linkReadError(mount, onTheWay, mountPrefixedPath, nested); let fileInfo: Awaited>; try { @@ -2724,12 +2814,12 @@ export class WorkspaceModule implements Module { // A directory swapped for a link since the walk above is caught here, by // what the path resolved to after the open. const resolvedThrough = await linkRefusal(linkView, realMountRoot, relativePath, realFilePath); - if (resolvedThrough !== null) throw await linkReadError(mount, resolvedThrough, mountPrefixedPath, this.nestedOf(mount.config.name)); + if (resolvedThrough !== null) throw await linkReadError(mount, resolvedThrough, mountPrefixedPath, nested); if (!isContainedPath(realMountRoot, realFilePath)) { throw new WorkspaceReadError('escape', `Symlink escape detected: ${mountPrefixedPath}`, 'realpath'); } - const holder = nestedAtReal(realFilePath, this.nestedOf(mount.config.name)); + const holder = nestedAtReal(realFilePath, nested); if (holder) throw new WorkspaceReadError('other_mount', leadsInto(mount.config.name, relativePath, holder), 'realpath'); let pathStat: Stats; @@ -3997,13 +4087,13 @@ export class WorkspaceModule implements Module { private async diskWalk(mount: MountState, dir: string, recursive: boolean): Promise<{ view: MountView; rootReal: string; taken: string[]; dirs: string[]; incomplete: Array<{ path: string; reason: string }>; }> { - const { view, rootReal } = await this.diskView(mount); + const { view, rootReal, moved } = await this.diskView(mount); const walk = await walkScope(view, rootReal, dir, recursive, mount.config.walkCap ?? WALK_CAP, { named: true }); const incomplete = walk.incomplete .filter((region) => region.kind === 'error' || region.kind === 'cap' || (region.kind === 'ignored' && region.path === dir && dir !== '')) .map((region) => ({ path: region.path, - reason: region.reason === OTHER_MOUNT ? this.otherMountNote(mount.config.name, region.path, 'list') : region.reason, + reason: region.reason === OTHER_MOUNT ? this.otherMountNote(mount.config.name, region.path, 'list', moved) : region.reason, })); return { view, @@ -4096,9 +4186,12 @@ export class WorkspaceModule implements Module { for (const { mount, relativePath } of mountsToSearch) { // A path named in another mount's directory, as holderOf resolves it, // is that mount's: grep searches nothing of this mount's there, and - // says whose it is. - if (this.holderOf(mount.config.name, relativePath)) { - incomplete.push({ path: `${mount.config.name}/${relativePath}`, reason: this.otherMountNote(mount.config.name, relativePath, 'list') }); + // says whose it is. A direct mount's is judged where its root lies now + // (movedNesting). + const root = resolve(mount.config.path); + const moved = mount.direct && relativePath !== '' ? await this.movedNesting(mount, await realpath(root).catch(() => root)) : null; + if (this.holderOf(mount.config.name, relativePath, moved)) { + incomplete.push({ path: `${mount.config.name}/${relativePath}`, reason: this.otherMountNote(mount.config.name, relativePath, 'list', moved) }); continue; } // A `path` and a filter whose directories don't overlap leave no file @@ -5051,11 +5144,14 @@ const DIRECT_ROOT_REPLACED = 'the mount root was replaced as the file was opened /** * How the alert that a mount configured direct runs as 'chronicle' names an - * intent the workspace can't read: materialize doesn't select one with no - * workspace entry, or settle one with an entry, and a sync of its path - * replaces it where it can read disk's copy. + * intent the workspace can't read. A sync of its path settles it, with + * disk's copy, or with none where disk has none, and where disk's file + * can't be read it skips the path and says why. Materialize settles one + * only where the workspace holds a copy there: a plain one skips it and says + * to sync, a forced one writes the copy, and where the workspace holds none + * it neither selects nor names it. */ -const UNREADABLE_INTENT = "its workspace intent can't be read, and only a sync of the path settles it"; +const UNREADABLE_INTENT = "its workspace intent can't be read; a sync of the path settles it or says why not"; /** * A mount's watch mode as it runs. A mount configured direct watches diff --git a/src/modules/workspace/types.ts b/src/modules/workspace/types.ts index 689ae2cfd..14bb9a46c 100644 --- a/src/modules/workspace/types.ts +++ b/src/modules/workspace/types.ts @@ -29,8 +29,9 @@ export interface MountConfig { * runs, so no call waits behind one, and `sync`, `materialize` and the * watch settings have nothing to act on. Only with `mode: 'read-only'`, * and not with `watch` other than 'never', `watchDebounceMs`, - * `watchRootPollMs`, `wakeOnChange` or `autoMaterialize`: each of those - * would never act, and the workspace refuses the mount. Its `ignore` + * `watchRootPollMs`, a `wakeOnChange` that would wake (true, or a + * non-empty list) or `autoMaterialize: true`: each of those would never + * act, and the workspace refuses the mount. Its `ignore` * has a default (see there). Paths, links, other mounts' directories, * size limits and the walk cap hold as on a 'chronicle' mount. Its * tools, and the host's reads of its files, refuse a path with a `.` or @@ -39,7 +40,14 @@ export interface MountConfig { * a generated file is answered, and `resolveAbsolutePath` maps a path, * as on any mount. Its tools read beneath whatever directory is at its * root, with no check against the one a 'chronicle' run of it last - * agreed with. + * agreed with. Where that root now lies elsewhere than the last start + * placed it, each call nests in it, as a start would now, every mount + * configured beneath its path, where that path leads beneath its root, + * and every other mount whose root, as that start placed it, lies + * beneath where its root lies now. A mount configured elsewhere whose + * own root has moved since, and the other mounts' own placements, wait + * for the next start, so a direct root moved into another mount's + * directory is still that mount's too. * * A mount configured 'direct' runs as 'chronicle' from a start that finds * the store holding changes there that disk lacks, on the branch selected diff --git a/test/workspace-direct.test.ts b/test/workspace-direct.test.ts index b35d145c5..9774fa992 100644 --- a/test/workspace-direct.test.ts +++ b/test/workspace-direct.test.ts @@ -7,7 +7,7 @@ import test, { type TestContext } from 'node:test'; import assert from 'node:assert/strict'; -import { appendFileSync, chmodSync, mkdirSync, mkdtempSync, renameSync, rmSync, statSync, symlinkSync, truncateSync, unlinkSync, writeFileSync } from 'node:fs'; +import { appendFileSync, chmodSync, existsSync, mkdirSync, mkdtempSync, renameSync, rmSync, statSync, symlinkSync, truncateSync, unlinkSync, writeFileSync } from 'node:fs'; import { createRequire, syncBuiltinESMExports } from 'node:module'; import { tmpdir } from 'node:os'; import { dirname, join } from 'node:path'; @@ -17,7 +17,7 @@ import { JsStore } from '@animalabs/chronicle'; // A namespace, so this file loads where the module lacks inOrder, and only // the tests that call it fail there. import * as workspaceModule from '../src/modules/workspace/index.js'; -import { WorkspaceModule } from '../src/modules/workspace/index.js'; +import { WorkspaceModule, WorkspaceReadError } from '../src/modules/workspace/index.js'; import { BranchIntents } from '../src/modules/workspace/branch-intent.js'; import type { MountConfig } from '../src/modules/workspace/types.js'; import type { ModuleContext } from '../src/types/module.js'; @@ -107,7 +107,7 @@ function watched(m: WorkspaceModule): string[] { } /** Run `fn` with `node:fs/promises`'s `name`, as the module's imports see it, replaced by what `make` makes of it. */ -async function withFsPromise(name: 'lstat' | 'open', make: (original: (...args: any[]) => Promise) => (...args: any[]) => Promise, fn: () => Promise): Promise { +async function withFsPromise(name: 'lstat' | 'open' | 'realpath', make: (original: (...args: any[]) => Promise) => (...args: any[]) => Promise, fn: () => Promise): Promise { const fsp = createRequire(import.meta.url)('node:fs/promises') as Record Promise>; const original = fsp[name]!; fsp[name] = make(original); @@ -123,7 +123,7 @@ async function withFsPromise(name: 'lstat' | 'open', make: (original: (...arg /** * The most calls of `node:fs/promises`'s `name` on the files `counted` names * under way at once while `fn` runs. The first are held until `width` of - * them are under way, and a moment longer, in which a reader wider than + * them are under way, and half a second longer, in which a reader wider than * `width` would start more, so a reader `width` wide shows as `width` however * fast the file system answers each step before the call. A narrower one is * let go after 5 seconds, so it fails rather than hangs. @@ -140,7 +140,7 @@ async function mostAtOnce(name: 'lstat' | 'open', width: number, counted: RegExp if (!counted.test(String(args[0]))) return original(...args); now++; most = Math.max(most, now); - if (now === width && settle === undefined) settle = setTimeout(release, 50); + if (now === width && settle === undefined) settle = setTimeout(release, 500); try { await released; return await original(...args); @@ -652,6 +652,379 @@ test("a direct mount refuses a '.' or '..' segment, and a file operation's trail assert.deepEqual((await data(m, 'ls', { path: 'ref/dir/' })).entries, [{ name: 'x.ts', type: 'file', size: 8 }]); }); +test("a direct mount whose root is repointed through a link reads the mounts where they lie now", async (t) => { + const env = new Env(t); + // `ref` reaches `a` through a link, and `work` lies at `b/work` with a + // draft disk lacks. + const a = join(env.root, 'a'); + const b = join(env.root, 'b'); + mkdirSync(join(a, 'work'), { recursive: true }); + mkdirSync(join(b, 'work', 'sub'), { recursive: true }); + writeFileSync(join(a, 'work', 'file.txt'), 'a\n'); + writeFileSync(join(b, 'work', 'file.txt'), 'disk\n'); + const link = join(env.root, 'link'); + symlinkSync(a, link); + const m = await env.open({ path: link }, [{ name: 'work', path: join(b, 'work'), mode: 'read-write', watch: 'never' }]); + await data(m, 'write', { path: 'work/file.txt', content: 'draft\n' }); + assert.equal((await data<{ content: string }>(m, 'read', { path: 'ref/work/file.txt' })).content, ' 1\ta'); + + // Repointed to `b`, `ref/work` is `work`'s directory, for every read. + rmSync(link); + symlinkSync(b, link); + const all = await data<{ results: Array<{ file: string }> }>(m, 'grep', { pattern: 'disk' }); + assert.deepEqual(all.results.map((r) => r.file).filter((file) => file.startsWith('ref/')), []); + assert.match(await failure(m, 'read', { path: 'ref/work/file.txt' }), /lies in mount "work", nested in mount "ref"/); + await assert.rejects(m.readFileFromDisk('ref/work/file.txt'), (err: unknown) => err instanceof WorkspaceReadError && err.code === 'other_mount'); + assert.deepEqual((await data(m, 'ls', { path: 'ref', recursive: true })).entries, []); + assert.deepEqual((await data<{ results: unknown[] }>(m, 'grep', { pattern: 'disk', path: 'ref' })).results, []); + const named = await data<{ results: unknown[]; incomplete: Array<{ path: string; reason: string }> }>(m, 'grep', { pattern: 'disk', path: 'ref/work' }); + assert.deepEqual([named.results, named.incomplete], [[], [{ path: 'ref/work', reason: 'it lies in mount "work", nested in this one: use "work"' }]]); + assert.deepEqual((await data(m, 'ls', { path: 'ref/work' })).incomplete, [{ path: 'work', reason: 'it lies in mount "work", nested in this one: use "work"' }]); + assert.equal((await data<{ content: string }>(m, 'read', { path: 'work/file.txt' })).content, ' 1\tdraft'); + + // Pointed back at `a`, `ref/work` is `ref`'s own again. + rmSync(link); + symlinkSync(a, link); + assert.equal((await data<{ content: string }>(m, 'read', { path: 'ref/work/file.txt' })).content, ' 1\ta'); + + // Repointed into `work`'s directory, `ref` reads what's there, and `work` + // keeps the start's placement until the next start: the directory is + // still its own too. + rmSync(link); + symlinkSync(join(b, 'work', 'sub'), link); + writeFileSync(join(b, 'work', 'sub', 'inside.txt'), 'i\n'); + assert.equal((await data<{ content: string }>(m, 'read', { path: 'ref/inside.txt' })).content, ' 1\ti'); + const listed = await data<{ entries: Array<{ path: string }> }>(m, 'ls', { path: 'work', recursive: true }); + assert.ok(listed.entries.some((e) => e.path === 'sub/inside.txt'), JSON.stringify(listed.entries)); +}); + +test("a direct mount moved away from a mount nested in it reads what's there as its own, in read, the host's read, ls and grep", async (t) => { + const env = new Env(t); + // `ref` starts at `b` through a link, where `work` lies nested at `b/work`. + const a = join(env.root, 'a'); + const b = join(env.root, 'b'); + mkdirSync(join(a, 'work'), { recursive: true }); + mkdirSync(join(b, 'work'), { recursive: true }); + writeFileSync(join(a, 'work', 'file.txt'), 'mine\n'); + writeFileSync(join(b, 'work', 'file.txt'), 'theirs\n'); + const link = join(env.root, 'link'); + symlinkSync(b, link); + const m = await env.open({ path: link }, [{ name: 'work', path: join(b, 'work'), mode: 'read-write', watch: 'never' }]); + assert.match(await failure(m, 'read', { path: 'ref/work/file.txt' }), /lies in mount "work", nested in mount "ref"/); + + // Repointed to `a`, nothing lies nested in `ref`, so `a/work` is its own. + rmSync(link); + symlinkSync(a, link); + assert.equal((await data<{ content: string }>(m, 'read', { path: 'ref/work/file.txt' })).content, ' 1\tmine'); + assert.equal((await m.readFileFromDisk('ref/work/file.txt')).bytes.toString(), 'mine\n'); + assert.deepEqual((await data<{ entries: Array<{ path: string }> }>(m, 'ls', { path: 'ref', recursive: true })).entries.map((e) => e.path), ['work/file.txt']); + assert.deepEqual((await data<{ results: Array<{ file: string }> }>(m, 'grep', { pattern: 'mine', path: 'ref/work' })).results.map((r) => r.file), ['ref/work/file.txt']); +}); + +test("a mount configured beneath a direct mount's path is nested where its moved root now puts it, as a start there nests it", async (t) => { + const env = new Env(t); + // `ref` is configured through `link`, and `work` beneath it at + // `link/work`, as a deploy's `current` link and a mount inside it are. In + // `b`, a link to `work` and one to its file. + const a = join(env.root, 'a'); + const b = join(env.root, 'b'); + mkdirSync(join(a, 'work'), { recursive: true }); + mkdirSync(join(b, 'work'), { recursive: true }); + writeFileSync(join(a, 'work', 'file.txt'), 'old\n'); + writeFileSync(join(b, 'work', 'file.txt'), 'new\n'); + writeFileSync(join(b, 'top.txt'), 'top\n'); + symlinkSync('work', join(b, 'alias')); + symlinkSync(join('work', 'file.txt'), join(b, 'alias.txt')); + const link = join(env.root, 'link'); + symlinkSync(a, link); + const mounts: [Partial, MountConfig[]] = [{ path: link, followSymlinks: true }, [{ name: 'work', path: join(link, 'work'), mode: 'read-write', watch: 'never' }]]; + const moved = await env.open(...mounts); + + // Repointed to `b`, `work`'s path leads to `b/work`, which is still + // `work`'s: by its directory, and by where a link into it lands. + rmSync(link); + symlinkSync(b, link); + const started = await new Env(t).open(...mounts); + const answers = async (m: WorkspaceModule) => ({ + read: await failure(m, 'read', { path: 'ref/work/file.txt' }), + throughLink: await failure(m, 'read', { path: 'ref/alias/file.txt' }), + fileLink: await call(m, 'read', { path: 'ref/alias.txt' }), + disk: await m.readFileFromDisk('ref/work/file.txt').then(() => 'read', (err: unknown) => (err instanceof WorkspaceReadError ? err.code : String(err))), + listed: (await data<{ entries: unknown[] }>(m, 'ls', { path: 'ref', recursive: true })).entries, + found: await data(m, 'grep', { pattern: 'new', path: 'ref' }), + }); + const fromStart = await answers(started); + assert.match(fromStart.read, /lies in mount "work", nested in mount "ref"/); + assert.match(fromStart.throughLink, /"ref\/alias\/file\.txt" leads into mount "work", nested in mount "ref"/); + assert.equal(fromStart.disk, 'other_mount'); + assert.deepEqual((fromStart.found as { results: unknown[] }).results, []); + assert.deepEqual(await answers(moved), fromStart); +}); + +test("a mount configured beneath a moved direct root, at a directory that's a link there, is nested as a start there nests it", async (t) => { + const env = new Env(t); + // `work` is configured at `link/work`, and in `b` that's a link to + // `real-work`, a directory beside it. + const a = join(env.root, 'a'); + const b = join(env.root, 'b'); + mkdirSync(join(a, 'work'), { recursive: true }); + mkdirSync(join(b, 'real-work'), { recursive: true }); + symlinkSync('real-work', join(b, 'work')); + writeFileSync(join(b, 'real-work', 'file.txt'), 'b work\n'); + writeFileSync(join(b, 'own.txt'), 'own\n'); + const link = join(env.root, 'link'); + symlinkSync(a, link); + const mounts: [Partial, MountConfig[]] = [{ path: link }, [{ name: 'work', path: join(link, 'work'), mode: 'read-write', watch: 'never' }]]; + const moved = await env.open(...mounts); + rmSync(link); + symlinkSync(b, link); + const started = await new Env(t).open(...mounts); + const answers = async (m: WorkspaceModule) => ({ + read: await failure(m, 'read', { path: 'ref/real-work/file.txt' }), + disk: await m.readFileFromDisk('ref/real-work/file.txt').then(() => 'read', (err: unknown) => (err instanceof WorkspaceReadError ? err.code : String(err))), + own: (await data<{ content: string }>(m, 'read', { path: 'ref/own.txt' })).content, + listed: (await data<{ entries: Array<{ path: string }> }>(m, 'ls', { path: 'ref', recursive: true })).entries.map((e) => e.path), + found: (await data<{ results: Array<{ file: string }> }>(m, 'grep', { pattern: 'b work', path: 'ref' })).results.map((r) => r.file), + }); + const fromStart = await answers(started); + assert.match(fromStart.read, /lies in mount "work", nested in mount "ref"/); + assert.deepEqual([fromStart.disk, fromStart.found], ['other_mount', []]); + assert.deepEqual(await answers(moved), fromStart); +}); + +test("a mount configured beneath a moved direct root, at a link that leads above it, is nested by its directory alone, as a start there nests it", async (t) => { + const env = new Env(t); + // In `b`, `work` leads to `b`'s parent, which holds `b`: through it, + // `work` would claim every path of `ref`'s. + const a = join(env.root, 'a'); + const b = join(env.root, 'b'); + mkdirSync(join(a, 'work'), { recursive: true }); + mkdirSync(b, { recursive: true }); + symlinkSync('..', join(b, 'work')); + writeFileSync(join(b, 'own.txt'), 'own\n'); + const link = join(env.root, 'link'); + symlinkSync(a, link); + const mounts: [Partial, MountConfig[]] = [{ path: link }, [{ name: 'work', path: join(link, 'work'), mode: 'read-only', backend: 'direct' }]]; + const moved = await env.open(...mounts); + rmSync(link); + symlinkSync(b, link); + const started = await new Env(t).open(...mounts); + const answers = async (m: WorkspaceModule) => ({ + own: (await data<{ content: string }>(m, 'read', { path: 'ref/own.txt' })).content, + listed: (await data<{ entries: Array<{ path: string }> }>(m, 'ls', { path: 'ref', recursive: true })).entries.map((e) => e.path), + }); + const fromStart = await answers(started); + assert.equal(fromStart.own, ' 1\town'); + assert.deepEqual(await answers(moved), fromStart); +}); + +test("a file tool's parse judges where a link on the way leads beneath the root it resolved, though the root moves after", async (t) => { + const env = new Env(t); + // At `b`, `alias` is a link into `work`'s directory; at `a`, a directory of + // `ref`'s own. + const a = join(env.root, 'a'); + const b = join(env.root, 'b'); + mkdirSync(join(a, 'alias'), { recursive: true }); + writeFileSync(join(a, 'alias', 'file.txt'), 'mine\n'); + mkdirSync(join(b, 'work'), { recursive: true }); + writeFileSync(join(b, 'work', 'file.txt'), 'theirs\n'); + symlinkSync('work', join(b, 'alias')); + const link = join(env.root, 'link'); + symlinkSync(b, link); + const m = await env.open({ path: link }, [{ name: 'work', path: join(b, 'work'), mode: 'read-write', watch: 'never' }]); + // The parse resolves the root at `b`, and the link is repointed to `a` as + // that answers: the parse's check of where the path leads is made beneath + // `b`, as its nesting is. + let repointed = false; + const err = await withFsPromise('realpath', (original) => async (...args: any[]) => { + const real = await original(...args); + if (!repointed && args[0] === link) { + repointed = true; + rmSync(link); + symlinkSync(a, link); + } + return real; + }, () => m.readFileFromDisk('ref/alias/file.txt').then(() => null, (e: unknown) => e)); + assert.ok(repointed); + assert.ok(err instanceof WorkspaceReadError, String(err)); + assert.deepEqual([err.code, err.stage], ['other_mount', 'parse']); +}); + +test("a direct mount whose root moves between a read's parse and its open names the mount a link on the way leads into, by the root it opens under", async (t) => { + const env = new Env(t); + const a = join(env.root, 'a'); + const b = join(env.root, 'b'); + mkdirSync(a, { recursive: true }); + mkdirSync(join(b, 'work'), { recursive: true }); + writeFileSync(join(b, 'work', 'file.txt'), 'disk\n'); + symlinkSync('work', join(b, 'alias')); + const link = join(env.root, 'link'); + symlinkSync(a, link); + const m = await env.open({ path: link }, [{ name: 'work', path: join(b, 'work'), mode: 'read-write', watch: 'never' }]); + // The parse resolves the root at `a`, which has no `alias`; the open finds + // `b`, where `alias` is a link into `work`'s directory. + let repointed = false; + const err = await withFsPromise('realpath', (original) => async (...args: any[]) => { + const real = await original(...args); + if (!repointed && args[0] === link) { + repointed = true; + rmSync(link); + symlinkSync(b, link); + } + return real; + }, () => m.readFileFromDisk('ref/alias/file.txt').then(() => null, (e: unknown) => e)); + assert.ok(repointed); + assert.ok(err instanceof WorkspaceReadError, String(err)); + assert.match(err.message, /lies in mount "work", nested in mount "ref"/); +}); + +test("a direct mount whose root has moved resolves no other mount's root but those configured beneath its path", async (t) => { + const env = new Env(t); + // `ref` reaches `a` through `link`, and `x` is configured through a link + // of its own, elsewhere. + const a = join(env.root, 'a'); + const b = join(env.root, 'b'); + const c = join(env.root, 'c'); + mkdirSync(a, { recursive: true }); + mkdirSync(b, { recursive: true }); + mkdirSync(join(c, 'x'), { recursive: true }); + writeFileSync(join(b, 'own.txt'), 'own\n'); + const link = join(env.root, 'link'); + const xlink = join(env.root, 'xlink'); + symlinkSync(a, link); + symlinkSync(c, xlink); + const m = await env.open({ path: link }, [{ name: 'x', path: join(xlink, 'x'), mode: 'read-only', backend: 'direct' }]); + rmSync(link); + symlinkSync(b, link); + const resolved: string[] = []; + await withFsPromise('realpath', (original) => async (...args: any[]) => { + resolved.push(String(args[0])); + return original(...args); + }, async () => { + assert.equal((await data<{ content: string }>(m, 'read', { path: 'ref/own.txt' })).content, ' 1\town'); + await data(m, 'ls', { path: 'ref', recursive: true }); + await data(m, 'grep', { pattern: 'own', path: 'ref' }); + await m.readFileFromDisk('ref/own.txt'); + }); + assert.ok(resolved.some((path) => path === link), JSON.stringify(resolved)); + assert.deepEqual(resolved.filter((path) => path.startsWith(xlink) || path.startsWith(c)), []); +}); + +test("a direct mount whose root hasn't moved keeps the start's placement, of a mount configured beneath it whose directory has moved since too", async (t) => { + const env = new Env(t); + // `work` is configured at `ref/work`, a link to `x1` at the start. + env.file('x1/f.txt', 'x1\n'); + env.file('x2/f.txt', 'x2\n'); + symlinkSync('x1', join(env.ref, 'work')); + const m = await env.open({}, [{ name: 'work', path: join(env.ref, 'work'), mode: 'read-only', backend: 'direct' }]); + assert.match(await failure(m, 'read', { path: 'ref/x1/f.txt' }), /lies in mount "work", nested in mount "ref"/); + // Repointed to `x2`, `work`'s files are there now, and `ref`'s root is + // where the start placed it, so `x1` stays `work`'s until the next start. + rmSync(join(env.ref, 'work')); + symlinkSync('x2', join(env.ref, 'work')); + assert.match(await failure(m, 'read', { path: 'ref/x1/f.txt' }), /lies in mount "work", nested in mount "ref"/); + assert.equal((await data<{ content: string }>(m, 'read', { path: 'ref/x2/f.txt' })).content, ' 1\tx2'); +}); + +/** Whether the temp directory folds case, as macOS's default disk does. */ +const FOLDS_CASE = (() => { + const dir = mkdtempSync(join(tmpdir(), 'workspace-direct-folds-')); + try { + writeFileSync(join(dir, 'Fold'), ''); + return existsSync(join(dir, 'FOLD')); + } finally { + rmSync(dir, { recursive: true, force: true }); + } +})(); + +test("on a disk that folds case, a mount configured beneath a moved direct root, spelled otherwise than disk spells it, is nested as a start there nests it", { + skip: FOLDS_CASE ? false : "the temp directory tells case apart, so a path spelled in another case reaches nothing: this runs on a disk that folds case, such as macOS's default", +}, async (t) => { + const env = new Env(t); + const a = join(env.root, 'a'); + const b = join(env.root, 'b'); + mkdirSync(join(a, 'work'), { recursive: true }); + mkdirSync(join(b, 'work'), { recursive: true }); + writeFileSync(join(b, 'work', 'file.txt'), 'new\n'); + writeFileSync(join(b, 'top.txt'), 'top\n'); + const link = join(env.root, 'link'); + symlinkSync(a, link); + // Configured as `Work`, which disk spells `work`. + const mounts: [Partial, MountConfig[]] = [{ path: link }, [{ name: 'work', path: join(link, 'Work'), mode: 'read-only', backend: 'direct' }]]; + const moved = await env.open(...mounts); + rmSync(link); + symlinkSync(b, link); + const started = await new Env(t).open(...mounts); + const answers = async (m: WorkspaceModule) => ({ + asDisk: await call(m, 'read', { path: 'ref/work/file.txt' }), + otherwise: await call(m, 'read', { path: 'ref/WORK/file.txt' }), + listed: (await data<{ entries: unknown[] }>(m, 'ls', { path: 'ref', recursive: true })).entries, + found: await data(m, 'grep', { pattern: 'new', path: 'ref' }), + }); + const fromStart = await answers(started); + assert.equal(fromStart.asDisk.success, false); + assert.deepEqual(await answers(moved), fromStart); +}); + +test("a direct mount whose root is gone, beneath a link, keeps the start's placement, as one beneath no link does", async (t) => { + const env = new Env(t); + // `ref` is configured beneath `link`, so its root as the start placed it + // is `a/ref`, spelled otherwise than its configured path. + const a = join(env.root, 'a'); + mkdirSync(join(a, 'ref', 'work'), { recursive: true }); + const link = join(env.root, 'link'); + symlinkSync(a, link); + const m = await env.open({ path: join(link, 'ref') }, [{ name: 'work', path: join(a, 'ref', 'work'), mode: 'read-only', backend: 'direct' }]); + assert.match(await failure(m, 'read', { path: 'ref/work/file.txt' }), /lies in mount "work", nested in mount "ref"/); + rmSync(join(a, 'ref'), { recursive: true, force: true }); + assert.match(await failure(m, 'read', { path: 'ref/work/file.txt' }), /lies in mount "work", nested in mount "ref"/); +}); + +test("a direct mount whose root moves between a read's parse and its open judges the file by the root it opens under", async (t) => { + const env = new Env(t); + const a = join(env.root, 'a'); + const b = join(env.root, 'b'); + mkdirSync(join(a, 'work'), { recursive: true }); + mkdirSync(join(b, 'work'), { recursive: true }); + writeFileSync(join(a, 'work', 'file.txt'), 'a\n'); + writeFileSync(join(b, 'work', 'file.txt'), 'disk\n'); + const link = join(env.root, 'link'); + symlinkSync(a, link); + const m = await env.open({ path: link }, [{ name: 'work', path: join(b, 'work'), mode: 'read-write', watch: 'never' }]); + // The parse resolves the root at `a`, where nothing is nested, and the + // link is repointed to `b` as that answers, so the open finds `b`, where + // `work` lies. + let repointed = false; + const err = await withFsPromise('realpath', (original) => async (...args: any[]) => { + const real = await original(...args); + if (!repointed && args[0] === link) { + repointed = true; + rmSync(link); + symlinkSync(b, link); + } + return real; + }, () => m.readFileFromDisk('ref/work/file.txt').then(() => null, (e: unknown) => e)); + assert.ok(repointed); + assert.ok(err instanceof WorkspaceReadError, String(err)); + assert.deepEqual([err.code, err.stage], ['other_mount', 'realpath']); +}); + +test("a link inside a moved direct root that leads into a mount nested there is refused as that mount's, at a path only its draft has", async (t) => { + const env = new Env(t); + const a = join(env.root, 'a'); + const b = join(env.root, 'b'); + mkdirSync(a, { recursive: true }); + mkdirSync(join(b, 'work'), { recursive: true }); + symlinkSync('work', join(b, 'alias')); + const link = join(env.root, 'link'); + symlinkSync(a, link); + const m = await env.open({ path: link, followSymlinks: true }, [{ name: 'work', path: join(b, 'work'), mode: 'read-write', watch: 'never' }]); + await data(m, 'write', { path: 'work/new.txt', content: 'draft\n' }); + rmSync(link); + symlinkSync(b, link); + assert.match(await failure(m, 'read', { path: 'ref/alias/new.txt' }), /"ref\/alias\/new\.txt" leads into mount "work", nested in mount "ref"/); +}); + test("ls without a path and status give a direct mount's kind and ignore list, and a 'chronicle' mount's as before", async (t) => { const env = new Env(t); const work = join(env.root, 'work'); @@ -672,7 +1045,7 @@ test("ls without a path and status give a direct mount's kind and ignore list, a assert.deepEqual((await data>>(named, 'status')).ref!.ignore, ['vendor']); }); -test('a direct mount refuses every tool that writes, edit_many included, as read-only', async (t) => { +test('a direct mount refuses every tool that writes, edit_many and materialize included, as read-only', async (t) => { const env = new Env(t); env.file('a.ts', 'alpha\n'); const m = await env.open(); @@ -684,6 +1057,9 @@ test('a direct mount refuses every tool that writes, edit_many included, as read ['delete', { path: 'ref/a.ts' }], ]; for (const [name, input] of calls) assert.match(await failure(m, name, input), /Mount "ref" is read-only/, name); + // materialize names a read-only mount among what it skipped, as on any. + const materialized = await data<{ count: number; skipped: unknown[] }>(m, 'materialize', { mount: 'ref' }); + assert.deepEqual([materialized.count, materialized.skipped], [0, [{ mount: 'ref', reason: 'a read-only mount: materialize writes nothing to it' }]]); assert.equal((await data<{ content: string }>(m, 'read', { path: 'ref/a.ts' })).content, ' 1\talpha'); assert.deepEqual(env.stateIds(), states, 'no state was made'); }); @@ -861,22 +1237,22 @@ test("the changes that keep a mount from starting direct are named ten at a time await env.open(); assert.equal(env.alerts.length, 1); - const unreadable = (path: string) => `"ref/${path}" (its workspace intent can't be read, and only a sync of the path settles it)`; + const unreadable = (path: string) => `"ref/${path}" (its workspace intent can't be read; a sync of the path settles it or says why not)`; const shown = [unreadable('aa.ts'), unreadable('d00.ts'), ...Array.from({ length: 8 }, (_, i) => `"ref/d0${i + 1}.ts" (changed in the workspace)`)]; // An intent it can't read is counted as one, not as a change. assert.ok(env.alerts[0]!.message.endsWith(`Mount "ref" holds 10 changes that disk lacks and 2 intents it can't read: ${shown.join(', ')}, and 2 more.`), env.alerts[0]!.message); assert.deepEqual(env.alerts[0]!.data, { held: { ref: 12 } }); }); -test("the ways out say only a sync settles an intent it can't read, where one is held past the ten paths named too", async (t) => { +test("the ways out say to sync the path of an intent it can't read, where one is held past the ten paths named too", async (t) => { const env = new Env(t); const chronicle = await env.openChronicle(); for (let i = 0; i < 11; i++) await data(chronicle, 'write', { path: `ref/d${String(i).padStart(2, '0')}.ts`, content: `${i}\n` }); await env.close(chronicle); - const sentence = "and restore its direct config. Only a sync settles an intent it can't read. Mount \"ref\" holds"; + const sentence = "and restore its direct config. Sync the path of an intent it can't read. Mount \"ref\" holds"; await env.close(await env.open()); - assert.ok(!env.alerts[0]!.message.includes('Only a sync'), env.alerts[0]!.message); + assert.ok(!env.alerts[0]!.message.includes('Sync the path'), env.alerts[0]!.message); // One that sorts after every change, so it isn't among the ten named. env.store.treeSet('workspace/ref/intent', 'zz.ts', { blobHash: 'f'.repeat(64), size: 2, mode: 0 }); env.alerts = []; @@ -885,6 +1261,30 @@ test("the ways out say only a sync settles an intent it can't read, where one is assert.ok(env.alerts[0]!.message.includes(sentence), env.alerts[0]!.message); }); +test("an intent it can't read where the workspace holds no copy is left by materialize and settled by a sync of its path, as the alert says", async (t) => { + const env = new Env(t); + env.file('u.ts', 'disk\n'); + const chronicle = await env.openChronicle(); + env.store.treeSet('workspace/ref/intent', 'u.ts', { blobHash: 'f'.repeat(64), size: 2, mode: 0 }); + await env.close(chronicle); + await env.close(await env.open()); + assert.match(env.alerts[0]!.message, /Sync the path of an intent it can't read\./); + + // The alert's materialize route, forced too, neither settles it nor names it. + const again = await env.openChronicle(); + for (const input of [{ mount: 'ref' }, { mount: 'ref', force: true }]) { + assert.deepEqual(await data(again, 'materialize', input), { materialized: [], count: 0 }); + } + assert.notEqual(env.store.treeGet('workspace/ref/intent', 'u.ts'), null); + // A sync of its path settles it, and the next start runs direct. + assert.deepEqual((await data<{ totalSynced: number }>(again, 'sync', { path: 'ref/u.ts' })).totalSynced, 1); + assert.equal(env.store.treeGet('workspace/ref/intent', 'u.ts'), null); + await env.close(again); + env.alerts = []; + const direct = await env.open(); + assert.deepEqual([env.alerts.length, (await data>>(direct, 'status')).ref!.backend], [0, 'direct']); +}); + test('each kind of change the store holds keeps a mount configured direct from starting direct on its own, until a sync of its path settles it', async (t) => { const cases: Array<[string, (env: Env) => Promise, string, string]> = [ ['a workspace deletion', async (env) => { @@ -901,7 +1301,7 @@ test('each kind of change the store holds keeps a mount configured direct from s ["an intent that can't be read", async (env) => { await env.close(await env.openChronicle()); env.store.treeSet('workspace/ref/intent', 'u.ts', { blobHash: 'f'.repeat(64), size: 2, mode: 0 }); - }, "1 intent it can't read", `"ref/u.ts" (its workspace intent can't be read, and only a sync of the path settles it)`], + }, "1 intent it can't read", `"ref/u.ts" (its workspace intent can't be read; a sync of the path settles it or says why not)`], ]; for (const [kind, hold, counted, named] of cases) { const env = new Env(t); @@ -966,7 +1366,7 @@ test("the alert's ways out come within the start a webhook keeps of it, however // 500, with its end (framework.ts, OPS_WEBHOOK_MESSAGE_MAX_CHARS and // boundFailureText). assert.ok(message.length > 500, `${message.length}`); - const last = "Only a sync settles an intent it can't read."; + const last = "Sync the path of an intent it can't read."; const at = message.indexOf(last); assert.ok(at >= 0 && at + last.length <= 340, `${at}`); }); From 50e8fb7504fe625aafebfff4aa7e275ac675dd1f Mon Sep 17 00:00:00 2001 From: Beth-2709 Date: Mon, 19 Oct 2026 17:20:56 -0400 Subject: [PATCH 08/13] fix(workspace): a direct mount's checks read disk beneath the root they resolved, and a moved root nests what a start nests there A direct mount's parse resolves its root once, and its checks of a path took the nesting there, but some of their reads of disk still went through the configured path. In a call whose root was repointed between the parse and one of them, two roots met in one check. landingRefusal then read a link on the path's way at the new root and named its target by the old root's nesting, so a refusal could name a file another mount's include leaves out. Now each of the parse's checks reads disk beneath the root the parse resolved: landingRefusal's link lookups, namesFile's and refusesAsOutside's directory reads, and linkReadError, which takes the root the check that found the link resolved. A read's own link checks read beneath the root it opens under, as its other checks judge. A direct mount whose root has moved nested a mount configured elsewhere only where that mount's real root lies beneath the moved root. A start also nests a mount where its configured path lies beneath a directory that is the outer mount's root on disk (readNesting's throughRoot). With a mount configured at b/work, a link to b itself, a root moved to b didn't nest work there, though a start there nests work at work, so its answers for ref/work weren't a start's. movedNesting now nests a mount by its configured path too, beneath each directory the last start found where the moved root lies, from that start's placement (placedAbove), resolving nothing new. On a disk that folds names, a read still judges the include by the path as its parse spelled it, so a root repointed between the two can have it serve a file the new root spells otherwise, as a rename during a call can on any mount. movedNesting's doc and the description say so. Texts: the include's doc says a direct mount has no history, so disk alone decides there; the description and changelog say a direct mount's status gives only its path, mode, backend and ignore list; and generatedFileKey's comment names the traversal guard it matches. Tests: a root moved after the parse is judged against the include beneath the parse's root, and its refusal names nothing another mount's include leaves out; a mount configured through a link to the moved root's new directory is nested as a start there nests it; the moved-away test gives sync and materialize too; and an image the include leaves out is refused by read_image in the include's words. The parse-root race test now compares the raced read with one made at the parse's root, which it now matches: the link on the way is read beneath that root, so the refusal is the link's. Co-Authored-By: Claude Opus 5.5 --- changelog.d/workspace-direct-mount.added.md | 4 +- src/modules/workspace/index.ts | 163 ++++++++++++-------- src/modules/workspace/types.ts | 12 +- test/workspace-direct.test.ts | 121 ++++++++++++++- 4 files changed, 222 insertions(+), 78 deletions(-) diff --git a/changelog.d/workspace-direct-mount.added.md b/changelog.d/workspace-direct-mount.added.md index 62cad91ae..887daaedf 100644 --- a/changelog.d/workspace-direct-mount.added.md +++ b/changelog.d/workspace-direct-mount.added.md @@ -1,7 +1,7 @@ - **A direct workspace mount, for a checkout an agent reads:** a mount whose config sets `backend: 'direct'` (the default is `'chronicle'`, the store-backed kind every mount has been) is read straight from disk. `read`, `read_image`, `ls`, `glob` and `grep` give its files as they are when they run, with no scan, watcher or pass, so no call waits behind one, and none of its files go into the store. It's read-only: the workspace refuses a direct mount with any `mode` but `'read-only'`, and one that sets `watch` other than `'never'`, `watchDebounceMs`, `watchRootPollMs`, a `wakeOnChange` that would wake, or `autoMaterialize: true`, since none of them would ever act there. Its paths, links, `include`, other mounts' directories, size limit and walk cap hold as on a store-backed mount. Where it differs: - **What its listings say:** each file with its `size`, and no state or MIME type, or, for a link the mount doesn't follow or another entry it doesn't take, with why as `note`. A listing looks at each entry with `lstat`, so it lists a file it can't open with its size, and reading that file says why. `grep`'s results say `version: 'disk'`, and it lists under `skipped` a binary file, one over the size limit and one it doesn't read, each with why. `read` refuses a binary file and one over the size limit, as a store-backed mount's `read` refuses one it never took in from disk. - - **Its ignore list:** with none in its config, it takes `node_modules`, `dist`, `.git`, `build`, `coverage`, `target`, `__pycache__` and `.venv`. Walks of the whole mount pass over what the list covers, and a path a listing, glob or grep names, or its pattern's directory, is walked all the same, as a sync naming a path walks it on a store-backed mount. `ls` without a path, and `status`, give a mount configured direct its `backend`, as it runs this session, and the list it goes by. - - **Its root:** it reads beneath whatever directory is at its path now, with no check against the one a store-backed run of it last agreed with. Where its root now lies elsewhere than the last start placed it, as when a link it's configured through is repointed, it nests what a start would nest there now: a mount configured beneath its path, where that path leads, and a mount whose root lies beneath where its root lies now. Their files are those mounts'. A mount configured elsewhere whose own root has moved since, and the other mounts' own placements, wait for the next start, so a direct root moved into another mount's directory is still that mount's directory too. + - **Its ignore list:** with none in its config, it takes `node_modules`, `dist`, `.git`, `build`, `coverage`, `target`, `__pycache__` and `.venv`. Walks of the whole mount pass over what the list covers, and a path a listing, glob or grep names, or its pattern's directory, is walked all the same, as a sync naming a path walks it on a store-backed mount. `ls` without a path, and `status`, give a mount configured direct its `backend`, as it runs this session, and the list it goes by. A direct mount's `status` gives only its `path`, `mode`, `backend` and `ignore`, since it keeps nothing to count. + - **Its root:** it reads beneath whatever directory is at its path now, with no check against the one a store-backed run of it last agreed with. Where its root now lies elsewhere than the last start placed it, as when a link it's configured through is repointed, it nests what a start would nest there now: a mount configured beneath its path, where that path leads, and a mount whose root the last start placed beneath where its root lies now, or whose configured path lies beneath a directory that start found there. Their files are those mounts'. A mount configured elsewhere whose own root has moved since, and the other mounts' own placements, wait for the next start, so a direct root moved into another mount's directory is still that mount's directory too. - **`sync`:** passes over it, listing it under `skipped`. `materialize` passes over it too, naming it under `skipped`, as it does any read-only mount. - **A mount the store holds changes on that disk lacks** (a draft, a workspace deletion, a conflict, or an intent it can't read, on the branch selected at a start, where its `include` covers it) runs store-backed from that start, so they stay in view. It runs unwatched, and with the direct default ignore list where its config names none. The workspace says so on stderr and in one ops alert (`workspace-mount-not-direct`). The alert first gives the two ways to start such a mount direct: give each change up with a sync of its path, or configure the mount `'chronicle'` and read-write, materialize it, follow materialize's answer, and restore its direct config; its next start then runs it direct. Where an intent it can't read is held, it says to sync that path. Then it names every such mount and what's held there, counting an intent it can't read as one. A mount that ran direct and runs store-backed again takes in what disk changed meanwhile as its listings, searches and syncs reach it, as an unwatched store-backed mount does. - **Tool descriptions:** `ls`, `glob`, `grep` and `sync` gain a sentence about the direct kind only on a host with a mount that runs direct; elsewhere they're unchanged. diff --git a/src/modules/workspace/index.ts b/src/modules/workspace/index.ts index 1a097bbf1..f6fcf5d2a 100644 --- a/src/modules/workspace/index.ts +++ b/src/modules/workspace/index.ts @@ -334,7 +334,10 @@ function matchesPartialPrefix(buffer: Buffer, prefix: Buffer): boolean { * among them, the code of a link the mount doesn't follow where it stands: * `symlink` at the path itself, as any other link there gets, and * `parent_symlink` among its directories, judged on the parsed path - * (`relativePath`), whatever spelling the caller gave. + * (`relativePath`), whatever spelling the caller gave. Where the link leads is + * read beneath `rootReal`, the root the check that found it resolved, so the + * link and its target are judged beneath one root, by that root's nesting + * (`nested`); left out, it's the mount's configured root, resolved now. */ async function linkReadError( mount: { config: { name: string; path: string; include?: readonly string[] } }, @@ -342,12 +345,13 @@ async function linkReadError( mountPrefixedPath: string, relativePath: string, nested: readonly NestedMount[], + rootReal?: string, ): Promise { if (!('link' in found)) { return new WorkspaceReadError('not_found', `Unable to read file: ${mountPrefixedPath}: ${linkWords(mount.config.name, found, null)}`, 'readlink', found.code); } - const rootReal = await realpath(mount.config.path).catch(() => resolve(mount.config.path)); - const leads = await shownLeads(mount.config.path, rootReal, found, mount.config.include); + const real = rootReal ?? await realpath(mount.config.path).catch(() => resolve(mount.config.path)); + const leads = await shownLeads(rootReal ?? mount.config.path, real, found, mount.config.include); const reason = `${mountPrefixedPath}: ${linkWords(mount.config.name, found, leads, nested, mount.config.include)}`; const holder = leads?.kind === 'inside' ? nestedHolder(leads.target, nested) : undefined; const code = leads?.kind === 'outside' ? 'escape' @@ -908,6 +912,8 @@ export class WorkspaceModule implements Module { private rootFoldsCase = new Map(); /** Where the last start placed each mount's root (readNesting's `reals`). */ private placedRoots = new Map(); + /** Each directory above each mount's configured path, with where the last start found it on disk (readNesting's `above`). */ + private placedAbove = new Map>(); /** * Whether a file's mode is observed, taken in and applied (MountRuntime's * `modesObserved`): not on Windows, which reports no POSIX modes. A test @@ -1269,13 +1275,14 @@ export class WorkspaceModule implements Module { // before they're known. A stop begun meanwhile stands, and this start then // sets nothing. const stops = this.stopsBegun; - const { nesting, reals } = await this.readNesting(); + const { nesting, reals, above } = await this.readNesting(); const rootFoldsCase = new Map(); for (const mount of this.config.mounts) rootFoldsCase.set(mount.name, await foldsCase(reals.get(mount.name)!)); if (this.stopsBegun !== stops) return; this.otherMounts = nesting; this.rootFoldsCase = rootFoldsCase; this.placedRoots = reals; + this.placedAbove = above; this.ctx = ctx; this.stopped = false; @@ -1907,7 +1914,7 @@ export class WorkspaceModule implements Module { const mountName = slash < 0 ? path : path.slice(0, slash); const mount = this.config.mounts.find(m => m.name === mountName); if (!mount) throw new Error(`Unknown mount: "${mountName}"`); - // Against the resolved root, as parsePath judges it. + // Against the resolved root, as parsePathAsGiven's traversal guard judges it. const root = resolve(mount.path); const resolved = resolve(root, slash < 0 ? '' : path.slice(slash + 1)); if (!isContainedPath(root, resolved)) throw new Error(`Path traversal detected: "${path}"`); @@ -2344,9 +2351,11 @@ export class WorkspaceModule implements Module { * as configured still names it as configured (resolveAbsolutePath). A * directory not on disk keeps the spellings the roots give alone. Each * mount's real root comes back beside the nesting (`reals`), for what - * start reads there. + * start reads there, and each directory above each configured path with + * where it lies on disk (`above`), for a direct mount whose root moves + * (movedNesting). */ - private async readNesting(): Promise<{ nesting: Map; reals: Map }> { + private async readNesting(): Promise<{ nesting: Map; reals: Map; above: Map> }> { const reals = new Map(); // Each directory above a mount's configured path, with where it lies on // disk, so a path that reaches another mount's root through a link, as @@ -2396,7 +2405,7 @@ export class WorkspaceModule implements Module { } } } - return { nesting, reals }; + return { nesting, reals, above }; } /** @@ -2424,28 +2433,32 @@ export class WorkspaceModule implements Module { * placement holds, or where nothing is at `rootReal`, whose read says so. A * direct mount reads beneath whatever directory is at its root, so a check * judges which mount a path is by the root it resolved. A tool's parse - * resolves it once: its respelling reads disk beneath it, its checks of the - * path it's given (parseFilePath's, refusesAsOutside, grep's) take the - * nesting there, and parseFilePath's resolves where the path lands beneath - * it; their other reads of disk go through the configured path. A read and - * a walk each resolve it for themselves, and judge the include and the - * nesting again beneath it. So a root repointed during a call can change - * which answer or refusal a path gets, but nothing a read or a walk serves - * is a file the include leaves out, or in another mount's directory as that - * nesting places it. A mount configured beneath its configured path lies - * where that path now leads, as a start would place it: at its configured - * directory, and where that directory resolves beneath `rootReal` - * (realBeneath), its real root, each also as disk spells it (diskSpelling). - * Any other mount lies where the last start placed it, and is nested where - * that lies beneath `rootReal`. Each carries its own include, as nestedOf - * gives it. It resolves nothing but directories beneath this mount's own - * root, so a share elsewhere that doesn't answer holds only its own calls, - * unless such a directory is a link into it, where that mount's files lie. - * What it leaves to the next start: a mount configured elsewhere whose own - * root has moved since, as by a second link repointed, that mount's - * spellings through other links, and the other mounts' own placements, so a - * direct root moved into another mount's directory is that mount's too, as - * a nested mount's root moved while running stays placed. + * resolves it once, and its respelling and its checks of the path it's + * given (parseFilePath's, refusesAsOutside, grep's) read disk beneath that + * root and judge by the nesting there. A read and a walk each resolve it + * for themselves, and judge the include and the nesting again beneath it. + * So a root repointed during a call can change which answer or refusal a + * path gets, but nothing a read or a walk serves is a file the include + * leaves out, or in another mount's directory as its own nesting places it, + * except that on a disk that folds names a read judges the include by the + * path as its parse spelled it, so a root repointed between the two can + * have it serve a file the new root spells otherwise, as a rename during a + * call can on any mount (sameLevels). A mount configured beneath its + * configured path lies where that path now leads, as a start would place + * it: at its configured directory, and where that directory resolves + * beneath `rootReal` (realBeneath), its real root, each also as disk spells + * it (diskSpelling). Any other mount lies where the last start placed it: + * it's nested where its real root lies beneath `rootReal`, and where its + * configured path lies beneath a directory that start found at `rootReal`. + * Each carries its own include, as nestedOf gives it. It resolves nothing + * but directories beneath this mount's own root, so a share elsewhere that + * doesn't answer holds only its own calls, unless such a directory is a + * link into it, where that mount's files lie. What it leaves to the next + * start: a mount configured elsewhere whose own root has moved since, as by + * a second link repointed, that mount's spellings through other links, and + * the other mounts' own placements, so a direct root moved into another + * mount's directory is that mount's too, as a nested mount's root moved + * while running stays placed. */ private async movedNesting(mount: MountState, rootReal: string): Promise { const placed = this.placedRoots.get(mount.config.name); @@ -2468,9 +2481,20 @@ export class WorkspaceModule implements Module { if (there !== undefined && there !== configured) nested.push(at(there, inner.name, real)); continue; } + // As a start nests it: where its real root, as the last start placed + // it, lies beneath this root, and where its configured path lies + // beneath a directory that start found to be where this root now lies + // (readNesting's throughRoot). const real = this.placedRoots.get(inner.name); - const fromStart = real === undefined ? undefined : nestedDir(rootReal, real); - if (fromStart !== undefined) nested.push(at(fromStart, inner.name, real!)); + if (real === undefined) continue; + const innerPath = resolve(inner.path); + const dirs = [ + nestedDir(rootReal, real), + ...(this.placedAbove.get(inner.name) ?? []).filter((a) => a.real === rootReal).map((a) => nestedDir(a.dir, innerPath)), + ]; + for (const dir of dirs) { + if (dir !== undefined && !nested.some((n) => n.name === inner.name && n.dir === dir)) nested.push(at(dir, inner.name, real)); + } } // Each directory as disk spells it too, as a start adds it, so a path // parsePath has respelled is judged on a disk that folds names. @@ -2541,8 +2565,8 @@ export class WorkspaceModule implements Module { */ private async parseFilePath(path: string): Promise<{ mount: MountState; relativePath: string; path: string }> { const spelled = await this.parsePath(path); - // Both checks take the nesting the parse found, and the second resolves - // where the path lands beneath the root the parse resolved. + // Both checks judge by the nesting the parse found, and the second reads + // disk beneath the root the parse resolved. const { mount, root, moved } = spelled; const parsed = this.refuseNotHeld(spelled, path, false, moved); // A path can also reach a nested mount's directory, or a file this @@ -2568,8 +2592,8 @@ export class WorkspaceModule implements Module { * on a file this mount's include leaves out. A mount with nothing nested, * and without both an include and links followed, has nothing to find, so * it pays nothing. A direct mount's file operation passes the nesting and - * the root its parse found (`nested`, `root`), so where the path lands is - * judged beneath the root that parse resolved. + * the root its parse found (`nested`, `root`), so where the path lands, and + * any link on its way, is judged beneath the root that parse resolved. */ private async landingRefusal(mount: MountState, relativePath: string, path: string, nested = this.nestedOf(mount.config.name), root = resolve(mount.config.path)): Promise { const judged = mount.config.followSymlinks === true && mount.config.include !== undefined; @@ -2590,8 +2614,9 @@ export class WorkspaceModule implements Module { return await coversLanding({ include: mount.config.include }, rootReal, real, relativePath) ? null : new WorkspaceReadError('escape', `Symlink escape detected: ${path}`, 'realpath'); } if (mount.config.followSymlinks !== true) { - const found = await linkOnTheWay({ root: mount.config.path, followSymlinks: false }, relativePath); - if (found !== null) return await linkReadError(mount, found, path, relativePath, nested); + const rootReal = await realBeneath(root); + const found = await linkOnTheWay({ root: rootReal, followSymlinks: false }, relativePath); + if (found !== null) return await linkReadError(mount, found, path, relativePath, nested, rootReal); } // On a mount with an include, the path to use there names each directory // on the place's way as that mount's, so a place beneath anything else @@ -2631,12 +2656,13 @@ export class WorkspaceModule implements Module { * cover names no file of the mount's, whatever disk or history holds * there, so one refusesAsOutside lets through is a directory. A direct * mount has no history: what a run of it as 'chronicle' left in the store - * isn't its own, so disk alone decides. + * isn't its own, so disk alone decides, beneath the root its parse resolved + * (`root`). */ - private async namesFile(mount: MountState, relativePath: string): Promise { + private async namesFile(mount: MountState, relativePath: string, root = resolve(mount.config.path)): Promise { return relativePath !== '' && insideMount(relativePath, mount.config.include) && ( (!mount.direct && this.getStore().treeGet(mount.treeStateId, relativePath) !== null) || - await lstat(join(mount.config.path, relativePath)).then((s) => !s.isDirectory(), () => false) + await lstat(join(root, relativePath)).then((s) => !s.isDirectory(), () => false) ); } @@ -2658,11 +2684,11 @@ export class WorkspaceModule implements Module { * mount. A * path in another mount's directory nested in this one is that mount's, * whatever this mount's include says, so it isn't refused here: each tool - * names the mount that holds it, as the file operations do, a direct - * mount's judged by the nesting its parse found (`moved`, movedNesting). A - * direct mount has no history, as namesFile says. + * names the mount that holds it, as the file operations do. A direct mount + * has no history, as namesFile says, and is judged beneath the root its + * parse resolved (`root`), by the nesting there (`moved`, movedNesting). */ - private async refusesAsOutside(mount: MountState, relativePath: string, moved: NestedMount[] | null): Promise { + private async refusesAsOutside(mount: MountState, relativePath: string, moved: NestedMount[] | null, root: string): Promise { if (this.holderOf(mount.config.name, relativePath, false, moved)) return false; if (insideMount(relativePath, mount.config.include)) return false; const own = { include: mount.config.include, nested: this.nestedOf(mount.config.name) }; @@ -2672,7 +2698,7 @@ export class WorkspaceModule implements Module { // walk reaches it: through no link, since the walk descends none. const segments = relativePath.split('/'); for (let i = 1; i <= segments.length; i++) { - const entry = await lstat(join(mount.config.path, ...segments.slice(0, i))).catch(() => null); + const entry = await lstat(join(root, ...segments.slice(0, i))).catch(() => null); if (!entry?.isDirectory()) return true; } return false; @@ -2710,8 +2736,8 @@ export class WorkspaceModule implements Module { private async parsePath(path: string): Promise { const given = this.parsePathAsGiven(path); // A direct mount's root is resolved once here, so the respelling and the - // checks of the path that follow take one nesting (movedNesting), and a - // file operation's landing is resolved beneath this root. + // checks of the path that follow read disk beneath one root and judge by + // the nesting there (movedNesting). const configured = resolve(given.mount.config.path); const root = given.mount.direct ? await realpath(configured).catch(() => configured) : configured; const moved = given.mount.direct ? await this.movedNesting(given.mount, root) : null; @@ -3022,11 +3048,12 @@ export class WorkspaceModule implements Module { // mount that doesn't follow symlinks goes through none among the // directories on the way, wherever it leads. Asked again of the path's // resolution after the open, below. - const linkView = { root: mount.config.path, followSymlinks: follow }; - // Every check below judges by the root this read opens under. + // Every check below judges by the root this read opens under, and reads + // the links on the path's way beneath it. + const linkView = { root: realMountRoot, followSymlinks: follow }; const nested = await this.nestedFor(mount, realMountRoot); const onTheWay = await linkOnTheWay(linkView, relativePath, 'parents'); - if (onTheWay !== null) throw await linkReadError(mount, onTheWay, mountPrefixedPath, relativePath, nested); + if (onTheWay !== null) throw await linkReadError(mount, onTheWay, mountPrefixedPath, relativePath, nested, realMountRoot); let fileInfo: Awaited>; try { @@ -3118,7 +3145,7 @@ export class WorkspaceModule implements Module { // leading to the opened file itself, passes both: a race this doesn't // close. const resolvedThrough = await linkRefusal(linkView, realMountRoot, relativePath, realFilePath); - if (resolvedThrough !== null) throw await linkReadError(mount, resolvedThrough, mountPrefixedPath, relativePath, nested); + if (resolvedThrough !== null) throw await linkReadError(mount, resolvedThrough, mountPrefixedPath, relativePath, nested, realMountRoot); const inside = isContainedPath(realMountRoot, realFilePath); const holder = inside ? nestedAtReal(realFilePath, nested) : undefined; @@ -4351,10 +4378,10 @@ export class WorkspaceModule implements Module { return { success: true, data: { mounts } }; } - const { mount, relativePath, path, moved } = await this.parsePath(input.path); + const { mount, relativePath, path, moved, root } = await this.parsePath(input.path); // A path outside the mount's include that's no scope (refusesAsOutside) // is refused, as glob refuses it, and shown as given. - if (await this.refusesAsOutside(mount, relativePath, moved)) { + if (await this.refusesAsOutside(mount, relativePath, moved, root)) { return { success: false, error: new OutsideIncludeError(input.path, mount.config).message, isError: true }; } input = { ...input, path }; @@ -4490,13 +4517,13 @@ export class WorkspaceModule implements Module { // Search across mounts const mountsToSearch = input.path ? [await this.parsePath(input.path)] - : [...this.mounts.values()].map(m => ({ mount: m, relativePath: '', moved: null })); + : [...this.mounts.values()].map(m => ({ mount: m, relativePath: '', moved: null, root: resolve(m.config.path) })); // The path is a directory to search in. One outside the mount's include // that's no scope (refusesAsOutside) is refused, as grep refuses it, // rather than searched and reported as matching nothing. if (input.path) { - const { mount, relativePath, moved } = mountsToSearch[0]!; - if (await this.refusesAsOutside(mount, relativePath, moved)) { + const { mount, relativePath, moved, root } = mountsToSearch[0]!; + if (await this.refusesAsOutside(mount, relativePath, moved, root)) { return { success: false, error: new OutsideIncludeError(input.path, mount.config).message, isError: true }; } } @@ -4545,12 +4572,12 @@ export class WorkspaceModule implements Module { const mountsToSearch = input.path ? [await this.parsePath(input.path)] - : [...this.mounts.values()].map(m => ({ mount: m, relativePath: '', moved: null })); + : [...this.mounts.values()].map(m => ({ mount: m, relativePath: '', moved: null, root: resolve(m.config.path) })); // A path outside the mount's include that's no scope is refused, as read // refuses it, rather than searched and reported as holding no match. if (input.path) { - const { mount, relativePath, moved } = mountsToSearch[0]!; - if (await this.refusesAsOutside(mount, relativePath, moved)) { + const { mount, relativePath, moved, root } = mountsToSearch[0]!; + if (await this.refusesAsOutside(mount, relativePath, moved, root)) { return { success: false, error: new OutsideIncludeError(input.path, mount.config).message, isError: true }; } } @@ -4577,7 +4604,7 @@ export class WorkspaceModule implements Module { return found; }; - for (const { mount, relativePath, moved } of mountsToSearch) { + for (const { mount, relativePath, moved, root } of mountsToSearch) { // A path named in another mount's directory, as holderOf resolves it, // is that mount's: grep searches nothing of this mount's there, and // says whose it is. A direct mount's is judged by the nesting its parse @@ -4597,7 +4624,7 @@ export class WorkspaceModule implements Module { }; const kept = (path: string): boolean => fileGlob === null || fileGlob.test(path); const found = mount.direct - ? await this.textsFromDisk(mount, relativePath, start, kept, searched) + ? await this.textsFromDisk(mount, relativePath, root, start, kept, searched) : await this.textsFromStore(mount, relativePath, start, kept, searched); skipped.push(...found.skipped); incomplete.push(...found.incomplete); @@ -4739,7 +4766,8 @@ export class WorkspaceModule implements Module { /** * The texts a direct mount's grep searches beneath `start`, or in the - * single file `relativePath` names, those `keep` lets through, each handed + * single file `relativePath` names, as namesFile judges it beneath `root`, + * the root grep's parse resolved, those `keep` lets through, each handed * to `each` in path order: each file its walk took (diskWalk), read from * disk as it is now, DIRECT_READS_AT_ONCE at a time. A file over the size * limit, a binary one and one the mount doesn't take are in `skipped`, @@ -4749,12 +4777,13 @@ export class WorkspaceModule implements Module { private async textsFromDisk( mount: MountState, relativePath: string, + root: string, start: string, keep: (path: string) => boolean, each: (text: GrepText) => void, ): Promise<{ skipped: GrepSkip[]; incomplete: Array<{ path: string; reason: string }> }> { const name = mount.config.name; - const fileScope = await this.namesFile(mount, relativePath); + const fileScope = await this.namesFile(mount, relativePath, root); const { view, rootReal, taken, incomplete } = fileScope ? { ...(await this.diskView(mount)), taken: [relativePath], incomplete: [] } : await this.diskWalk(mount, start, true); @@ -4960,7 +4989,7 @@ export class WorkspaceModule implements Module { // A path outside the mount's include that's no scope (refusesAsOutside) // is refused; beneath a directory, the push leaves out what the include // doesn't cover. - if (await this.refusesAsOutside(parsed.mount, parsed.relativePath, parsed.moved)) { + if (await this.refusesAsOutside(parsed.mount, parsed.relativePath, parsed.moved, parsed.root)) { return { success: false, error: new OutsideIncludeError(input.path, explicit.mount.config).message, isError: true }; } } @@ -5157,11 +5186,11 @@ export class WorkspaceModule implements Module { // A path names its own mount; otherwise the given mount, or every mount. let targets: Array<{ name: string; mount: MountState; relativePath: string }>; if (input.path) { - const { mount, relativePath, moved } = await this.parsePath(input.path); + const { mount, relativePath, moved, root } = await this.parsePath(input.path); // A path outside the mount's include that's no scope (refusesAsOutside) // is refused; beneath a directory, the walk passes over what the include // doesn't cover. - if (await this.refusesAsOutside(mount, relativePath, moved)) { + if (await this.refusesAsOutside(mount, relativePath, moved, root)) { return { success: false, error: new OutsideIncludeError(input.path, mount.config).message, isError: true }; } targets = [{ name: mount.config.name, mount, relativePath }]; diff --git a/src/modules/workspace/types.ts b/src/modules/workspace/types.ts index c6e5f03b1..5e97cbf5d 100644 --- a/src/modules/workspace/types.ts +++ b/src/modules/workspace/types.ts @@ -40,10 +40,11 @@ export interface MountConfig { * placed it, each call nests in it, as a start would now, every mount * configured beneath its path, where that path leads beneath its root, * and every other mount whose root, as that start placed it, lies - * beneath where its root lies now. A mount configured elsewhere whose - * own root has moved since, and the other mounts' own placements, wait - * for the next start, so a direct root moved into another mount's - * directory is still that mount's too. + * beneath where its root lies now, or whose configured path lies + * beneath a directory that start found there. A mount configured + * elsewhere whose own root has moved since, and the other mounts' own + * placements, wait for the next start, so a direct root moved into + * another mount's directory is still that mount's too. * * A mount configured 'direct' runs as 'chronicle' from a start that finds * the store holding changes there that disk lacks, on the branch selected @@ -151,7 +152,8 @@ export interface MountConfig { * include doesn't cover, and a listing that isn't recursive names each * directory directly beneath its path that disk has, and each that a * covered entry in history implies, with only the files the include - * covers. `ignore` still applies to what `include` covers. + * covers. A direct mount has no history (`backend`), so there disk alone + * decides each of these. `ignore` still applies to what `include` covers. * * On a disk that folds names, the parse reads a path in disk's spelling, * as the walk names what it lists, so a covered name whose file disk diff --git a/test/workspace-direct.test.ts b/test/workspace-direct.test.ts index 66eb2cf34..29b064ad8 100644 --- a/test/workspace-direct.test.ts +++ b/test/workspace-direct.test.ts @@ -692,6 +692,8 @@ test("a direct mount's include holds as a 'chronicle' mount's does", async (t) = for (const path of ['gate.json', 'gate.json.bak', 'notes.md', 'other.txt', 'sub/gate.json', 'sub/x.md', 'sub/y.txt', 'docs/a/b.txt', 'docsx/c.txt']) { env.file(path, `${path} needle\n`); } + env.file('pic.png', PNG); + env.file('docs/pic.png', PNG); const include = ['gate.json', '*.md', 'docs/**']; const calls: Array<[string, Record]> = [ ['ls', { path: 'ref', recursive: true }], @@ -709,16 +711,22 @@ test("a direct mount's include holds as a 'chronicle' mount's does", async (t) = ['read', { path: 'ref/sub/y.txt' }], ['read', { path: 'ref/docsx' }], ['read_image', { path: 'ref/other.txt' }], + ['read_image', { path: 'ref/pic.png' }], + ['read_image', { path: 'ref/docs/pic.png' }], ]; - const hostPaths = ['ref/notes.md', 'ref/other.txt', 'ref/docsx/c.txt']; + const hostPaths = ['ref/notes.md', 'ref/other.txt', 'ref/docsx/c.txt', 'ref/pic.png']; const direct = await env.open({ include }); const fromDisk = await answersOf(direct, calls, hostPaths); // The include's refusal is the shared parse's, in the same words, and `ls` // without a path lists it beside the kind. const said = /^"ref\/other\.txt" is outside the mount's include: mount "ref" holds only gate\.json, \*\.md, docs\/\*\*\./; assert.match(await failure(direct, 'read', { path: 'ref/other.txt' }), said); + // An image the include leaves out is refused in the include's words too, + // and one it covers is read. + assert.match(await failure(direct, 'read_image', { path: 'ref/pic.png' }), /^"ref\/pic\.png" is outside the mount's include: mount "ref" holds only /); + assert.ok((await call(direct, 'read_image', { path: 'ref/docs/pic.png' })).success); assert.deepEqual((await data<{ mounts: Array> }>(direct, 'ls')).mounts[0]!.include, include); - assert.deepEqual((fromDisk[0] as [string, unknown, { entries: string[] }])[2].entries, ['docs/a/b.txt', 'gate.json', 'notes.md', 'sub/gate.json', 'sub/x.md']); + assert.deepEqual((fromDisk[0] as [string, unknown, { entries: string[] }])[2].entries, ['docs/a/b.txt', 'docs/pic.png', 'gate.json', 'notes.md', 'sub/gate.json', 'sub/x.md']); await env.close(direct); assert.deepEqual(fromDisk, await answersOf(await env.open({ backend: 'chronicle', mode: 'read-write', watch: 'never', include }), calls, hostPaths)); }); @@ -980,9 +988,12 @@ test("a file tool's parse judges where a link on the way leads beneath the root const link = join(env.root, 'link'); symlinkSync(b, link); const m = await env.open({ path: link }, [{ name: 'work', path: join(b, 'work'), mode: 'read-write', watch: 'never' }]); + const atB = await m.readFileFromDisk('ref/alias/file.txt').then(() => null, (e: unknown) => e); + assert.ok(atB instanceof WorkspaceReadError, String(atB)); // The parse resolves the root at `b`, and the link is repointed to `a` as // that answers: the parse's check of where the path leads is made beneath - // `b`, as its nesting is. + // `b`, as its nesting is, the link on the way included, so the read + // answers as one made at `b` does. let repointed = false; const err = await withFsPromise('realpath', (original) => async (...args: any[]) => { const real = await original(...args); @@ -995,7 +1006,107 @@ test("a file tool's parse judges where a link on the way leads beneath the root }, () => m.readFileFromDisk('ref/alias/file.txt').then(() => null, (e: unknown) => e)); assert.ok(repointed); assert.ok(err instanceof WorkspaceReadError, String(err)); - assert.deepEqual([err.code, err.stage], ['other_mount', 'parse']); + assert.deepEqual([err.code, err.stage, err.message], [atB.code, atB.stage, atB.message]); + assert.deepEqual([err.code, err.stage], ['other_mount', 'readlink']); +}); + +test("a file tool's parse judges where a followed link lands against the include beneath the root it resolved, though the root moves after", async (t) => { + const env = new Env(t); + // At `b`, `link.md` leads to a file the include leaves out; at `a`, it's a + // covered file of its own. + const a = join(env.root, 'a'); + const b = join(env.root, 'b'); + mkdirSync(a, { recursive: true }); + mkdirSync(b, { recursive: true }); + writeFileSync(join(a, 'link.md'), 'a link.md\n'); + writeFileSync(join(b, 'other.txt'), 'b other\n'); + symlinkSync('other.txt', join(b, 'link.md')); + const link = join(env.root, 'link'); + symlinkSync(b, link); + const m = await env.open({ path: link, followSymlinks: true, include: ['*.md'] }); + // The parse resolves the root at `b`, and the link is repointed to `a` as + // that answers: the parse's check of where the path lands is made beneath + // `b`, where it lands on a file the include leaves out. + let repointed = false; + const err = await withFsPromise('realpath', (original) => async (...args: any[]) => { + const real = await original(...args); + if (!repointed && args[0] === link) { + repointed = true; + rmSync(link); + symlinkSync(a, link); + } + return real; + }, () => m.readFileFromDisk('ref/link.md').then(() => null, (e: unknown) => e)); + assert.ok(repointed); + assert.ok(err instanceof WorkspaceReadError, String(err)); + assert.deepEqual([err.code, err.message], ['escape', 'Symlink escape detected: ref/link.md']); +}); + +test("a file tool's parse reads a link on the way beneath the root it resolved, so a root moved after names nothing another mount's include leaves out", async (t) => { + const env = new Env(t); + // At `a`, `alias` leads into `old`, nested there; at `b`, an `alias` leads + // to a file `private`'s include leaves out. + const a = join(env.root, 'a'); + const b = join(env.root, 'b'); + mkdirSync(join(a, 'old'), { recursive: true }); + writeFileSync(join(a, 'old', 'public.json'), '{}\n'); + symlinkSync(join('old', 'public.json'), join(a, 'alias')); + mkdirSync(join(b, 'private'), { recursive: true }); + writeFileSync(join(b, 'private', 'secret.env'), 'S=1\n'); + symlinkSync(join('private', 'secret.env'), join(b, 'alias')); + const link = join(env.root, 'link'); + symlinkSync(a, link); + const m = await env.open({ path: link }, [ + { name: 'old', path: join(link, 'old'), mode: 'read-only', backend: 'direct' }, + { name: 'private', path: join(b, 'private'), mode: 'read-only', backend: 'direct', include: ['*.json'] }, + ]); + // The parse resolves the root at `a`, and the link is repointed to `b` as + // that answers: the refusal reads `alias` beneath `a`, where it leads into + // `old`. + let repointed = false; + const err = await withFsPromise('realpath', (original) => async (...args: any[]) => { + const real = await original(...args); + if (!repointed && args[0] === link) { + repointed = true; + rmSync(link); + symlinkSync(b, link); + } + return real; + }, () => m.readFileFromDisk('ref/alias').then(() => null, (e: unknown) => e)); + assert.ok(repointed); + assert.ok(err instanceof WorkspaceReadError, String(err)); + assert.equal(err.code, 'other_mount'); + assert.match(err.message, /old/); + assert.doesNotMatch(err.message, /private|secret/); +}); + +test("a mount configured in a moved direct root's new directory, through a link to that directory, is nested as a start there nests it", async (t) => { + const env = new Env(t); + // `work` is configured at `b/work`, a link to `b` itself, so a start with + // `ref` at `b` nests `work` at `work`, by its configured path alone. + const a = join(env.root, 'a'); + const b = join(env.root, 'b'); + mkdirSync(a, { recursive: true }); + mkdirSync(b, { recursive: true }); + writeFileSync(join(b, 'file.txt'), 'b file\n'); + symlinkSync('.', join(b, 'work')); + const link = join(env.root, 'link'); + symlinkSync(a, link); + const mounts: [Partial, MountConfig[]] = [{ path: link, followSymlinks: true }, [{ name: 'work', path: join(b, 'work'), mode: 'read-only', backend: 'direct' }]]; + const moved = await env.open(...mounts); + rmSync(link); + symlinkSync(b, link); + const started = await new Env(t).open(...mounts); + const answers = async (m: WorkspaceModule) => ({ + read: await call(m, 'read', { path: 'ref/work/file.txt' }), + disk: await m.readFileFromDisk('ref/work/file.txt').then(() => 'read', (err: unknown) => (err instanceof WorkspaceReadError ? err.code : String(err))), + own: (await data<{ content: string }>(m, 'read', { path: 'ref/file.txt' })).content, + found: (await data<{ results: Array<{ file: string }> }>(m, 'grep', { pattern: 'b file', path: 'ref' })).results.map((r) => r.file), + }); + const fromStart = await answers(started); + assert.match(fromStart.read.error ?? '', /lies in mount "work", nested in mount "ref"/); + assert.equal(fromStart.disk, 'other_mount'); + assert.deepEqual(await answers(moved), fromStart); }); test("a direct mount whose root moves between a read's parse and its open names the mount a link on the way leads into, by the root it opens under", async (t) => { @@ -1259,6 +1370,8 @@ test("a direct mount moved away from a mount nested in it judges a path its incl await call(m, 'glob', { pattern: '*', path: 'ref/work' }), await call(m, 'grep', { pattern: 'needle', path: 'ref/work' }), await call(m, 'read', { path: 'ref/work' }), + await call(m, 'sync', { path: 'ref/work' }), + await call(m, 'materialize', { path: 'ref/work' }), ]; const fromStart = await answers(started); // Nothing is nested in `ref` at `a`, so `work` is a file of `ref`'s that From 09ccb5e2deafdaef77fe9d2ed7ce2f82635260a9 Mon Sep 17 00:00:00 2001 From: Beth-2709 Date: Mon, 19 Oct 2026 17:23:54 -0400 Subject: [PATCH 09/13] test(workspace): a moved direct root nests a mount its real root alone places there movedNesting places another mount two ways, as a start's readNesting does: by its real root as the last start placed it, where that lies beneath the moved root, and by its configured path beneath a directory that start found where the moved root lies (throughRoot). Since the second was added, every moved-root test's mount was placed both ways, so dropping the first changed no answer. This test configures work at w, a link to b/deep/work, so no directory above w is b, and only its real root nests it in ref once ref's link is repointed to b. The moved module's answers to a read, a host read and a grep are compared with a module started fresh with ref at b, which nests work at deep/work. Co-Authored-By: Claude Opus 5.5 --- test/workspace-direct.test.ts | 31 +++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+) diff --git a/test/workspace-direct.test.ts b/test/workspace-direct.test.ts index 29b064ad8..c39c0ca44 100644 --- a/test/workspace-direct.test.ts +++ b/test/workspace-direct.test.ts @@ -1109,6 +1109,37 @@ test("a mount configured in a moved direct root's new directory, through a link assert.deepEqual(await answers(moved), fromStart); }); +test("a mount configured at a link into a moved direct root's new directory is nested where the link leads, as a start there nests it", async (t) => { + const env = new Env(t); + // `work` is configured at `w`, a link to `b/deep/work`, so a start with + // `ref` at `b` nests it by its real root alone: no directory above `w` + // is `b`. + const a = join(env.root, 'a'); + const b = join(env.root, 'b'); + mkdirSync(a, { recursive: true }); + mkdirSync(join(b, 'deep', 'work'), { recursive: true }); + writeFileSync(join(b, 'deep', 'work', 'file.txt'), 'work file\n'); + writeFileSync(join(b, 'own.txt'), 'b own file\n'); + symlinkSync(join(b, 'deep', 'work'), join(env.root, 'w')); + const link = join(env.root, 'link'); + symlinkSync(a, link); + const mounts: [Partial, MountConfig[]] = [{ path: link }, [{ name: 'work', path: join(env.root, 'w'), mode: 'read-only', backend: 'direct' }]]; + const moved = await env.open(...mounts); + rmSync(link); + symlinkSync(b, link); + const started = await new Env(t).open(...mounts); + const answers = async (m: WorkspaceModule) => ({ + read: await call(m, 'read', { path: 'ref/deep/work/file.txt' }), + disk: await m.readFileFromDisk('ref/deep/work/file.txt').then(() => 'read', (err: unknown) => (err instanceof WorkspaceReadError ? err.code : String(err))), + found: (await data<{ results: Array<{ file: string }> }>(m, 'grep', { pattern: 'file', path: 'ref' })).results.map((r) => r.file), + }); + const fromStart = await answers(started); + assert.match(fromStart.read.error ?? '', /lies in mount "work", nested in mount "ref"/); + assert.equal(fromStart.disk, 'other_mount'); + assert.deepEqual(fromStart.found, ['ref/own.txt']); + assert.deepEqual(await answers(moved), fromStart); +}); + test("a direct mount whose root moves between a read's parse and its open names the mount a link on the way leads into, by the root it opens under", async (t) => { const env = new Env(t); const a = join(env.root, 'a'); From e07eb6b8a4b6b3fa7301976dd7bc1d51124dca0a Mon Sep 17 00:00:00 2001 From: Beth-2709 Date: Mon, 19 Oct 2026 18:17:27 -0400 Subject: [PATCH 10/13] fix(workspace): a moved direct root nests a configured mount beneath each way to it, and a direct view reads beneath the root it resolved A direct mount whose root has moved nested a mount configured beneath its path at its configured directory and where that directory resolves, but not beneath a directory on the way that is the root itself, as a start's readNesting nests it (throughRoot). With work configured at link/sub/work, where sub and work lead to . at the new root, a start there nests work at sub/work and at work, and a moved call nested it at sub/work alone, so its answers for ref/work/file.txt weren't a start's. movedNesting now resolves each directory on the way beneath the moved root, and nests the mount beneath each that resolves to the root, and beneath a directory above the configured path that the last start found where the root now lies. A mount configured at the moved mount's own path lies where its root does, so a start nests it nowhere in it. movedNesting took it for a mount configured elsewhere and nested it where the last start had placed its root, which could lie beneath the new root. Now it's nested nowhere. movedNesting left the start's placement in place for any failure to look at the moved root, so an error other than absence had a call judge a moved root by the nesting of the root it left. Now only absence (ENOENT, ENOTDIR) leaves it, as the doc says: the parse's checks then keep the start's placement, and a read says the root is gone. It also takes one start's placements before its first await, so a start in the meantime can't give one call a nesting from two starts. A direct mount's walks and grep's reads opened their files through the configured path, while judging containment and nesting beneath the root their view resolved. A root repointed between the two had a read open a file its checks never judged, and by our reading that could serve a file a nested mount's include leaves out in that mount's directory at the new root. Their view now reads beneath the root it resolved, so opening and judging use one root. linkReadError's, namesFile's and landingRefusal's root, and landingRefusal's nesting, are now required, so no caller reads the configured path where it meant the root a parse resolved. Store-backed callers give the configured path and the start's nesting. Texts: openBeneathRoot's comment says its link checks read beneath the root step 0 checked while its lstat and open go through the configured path, and step 5 refuses a read whose root moved; the include's doc says a mount running direct has no history; and movedNesting's comments say what it takes from the last start and what waits for the next one. Tests: a mount configured beneath a moved root through a directory that's the root itself; one beneath a root moved to a directory above its configured path; two mounts at one path; a moved root that stat can't look at for a reason other than absence; a grep of a named file whose root is repointed before the open; grep's and a listing's judgements of a path made beneath the parse's root when the root moves after; and a directory a nesting from an unfiltered placement would claim, in the test of a mount configured through a link to the new root. Co-Authored-By: Claude Opus 5.5 --- src/modules/workspace/index.ts | 127 +++++++++++++-------- src/modules/workspace/types.ts | 5 +- test/workspace-direct.test.ts | 203 ++++++++++++++++++++++++++++++++- 3 files changed, 279 insertions(+), 56 deletions(-) diff --git a/src/modules/workspace/index.ts b/src/modules/workspace/index.ts index f6fcf5d2a..b7d30108a 100644 --- a/src/modules/workspace/index.ts +++ b/src/modules/workspace/index.ts @@ -337,7 +337,7 @@ function matchesPartialPrefix(buffer: Buffer, prefix: Buffer): boolean { * (`relativePath`), whatever spelling the caller gave. Where the link leads is * read beneath `rootReal`, the root the check that found it resolved, so the * link and its target are judged beneath one root, by that root's nesting - * (`nested`); left out, it's the mount's configured root, resolved now. + * (`nested`). */ async function linkReadError( mount: { config: { name: string; path: string; include?: readonly string[] } }, @@ -345,13 +345,12 @@ async function linkReadError( mountPrefixedPath: string, relativePath: string, nested: readonly NestedMount[], - rootReal?: string, + rootReal: string, ): Promise { if (!('link' in found)) { return new WorkspaceReadError('not_found', `Unable to read file: ${mountPrefixedPath}: ${linkWords(mount.config.name, found, null)}`, 'readlink', found.code); } - const real = rootReal ?? await realpath(mount.config.path).catch(() => resolve(mount.config.path)); - const leads = await shownLeads(rootReal ?? mount.config.path, real, found, mount.config.include); + const leads = await shownLeads(rootReal, rootReal, found, mount.config.include); const reason = `${mountPrefixedPath}: ${linkWords(mount.config.name, found, leads, nested, mount.config.include)}`; const holder = leads?.kind === 'inside' ? nestedHolder(leads.target, nested) : undefined; const code = leads?.kind === 'outside' ? 'escape' @@ -1162,12 +1161,14 @@ export class WorkspaceModule implements Module { * A direct mount as its tools read it: its view, with no root identity, * since it promises what's at its root now and keeps no evidence to check * that against (MountConfig.backend), and its root as realpath resolves it, - * or as given where that fails, as a pass's runtime takes it. + * or as given where that fails, as a pass's runtime takes it. The view + * reads beneath that root, so a walk or a read judges and opens its files + * beneath one root, however its configured path is repointed meanwhile. */ private async diskView(mount: MountState): Promise<{ view: MountView; rootReal: string; moved: NestedMount[] | null }> { const root = resolve(mount.config.path); const rootReal = await realpath(root).catch(() => root); - const view = this.viewOf(mount, undefined); + const view = { ...this.viewOf(mount, undefined), root: rootReal }; const moved = await this.movedNesting(mount, rootReal); return { view: moved === null ? view : { ...view, nested: moved, others: moved.map((n) => n.dir) }, rootReal, moved }; } @@ -2430,7 +2431,8 @@ export class WorkspaceModule implements Module { * The mounts nested in a direct mount whose root a check resolved to * `rootReal`, where that's elsewhere than the last start placed it, as when * a link it's configured through is repointed; null where the start's - * placement holds, or where nothing is at `rootReal`, whose read says so. A + * placement holds, or where nothing is at `rootReal`: there the parse's + * checks keep the start's placement, and a read says the root is gone. A * direct mount reads beneath whatever directory is at its root, so a check * judges which mount a path is by the root it resolved. A tool's parse * resolves it once, and its respelling and its checks of the path it's @@ -2445,25 +2447,34 @@ export class WorkspaceModule implements Module { * have it serve a file the new root spells otherwise, as a rename during a * call can on any mount (sameLevels). A mount configured beneath its * configured path lies where that path now leads, as a start would place - * it: at its configured directory, and where that directory resolves - * beneath `rootReal` (realBeneath), its real root, each also as disk spells - * it (diskSpelling). Any other mount lies where the last start placed it: - * it's nested where its real root lies beneath `rootReal`, and where its + * it: at its configured directory, beneath each directory on the way there + * that resolves to `rootReal` (realBeneath), and where its directory + * resolves beneath `rootReal`, its real root, each also as disk spells it + * (diskSpelling). Any other mount lies where the last start placed it: it's + * nested where its real root lies beneath `rootReal`, and where its * configured path lies beneath a directory that start found at `rootReal`. - * Each carries its own include, as nestedOf gives it. It resolves nothing - * but directories beneath this mount's own root, so a share elsewhere that - * doesn't answer holds only its own calls, unless such a directory is a - * link into it, where that mount's files lie. What it leaves to the next - * start: a mount configured elsewhere whose own root has moved since, as by - * a second link repointed, that mount's spellings through other links, and - * the other mounts' own placements, so a direct root moved into another - * mount's directory is that mount's too, as a nested mount's root moved - * while running stays placed. + * A mount configured at this mount's own path lies where its root does, so + * it's nested nowhere in it. Each carries its own include, as nestedOf + * gives it. It resolves nothing but directories beneath this mount's own + * root, so a share elsewhere that doesn't answer holds only its own calls, + * unless such a directory is a link into it, where that mount's files lie. + * What it leaves to the next start: a mount configured elsewhere whose own + * root has moved since, as when a link on its configured path is repointed, + * this mount's own reached through another link among them, that mount's + * spellings through other links, and the other mounts' own placements, so a + * direct root moved into another mount's directory is that mount's too, as + * a nested mount's root moved while running stays placed. */ private async movedNesting(mount: MountState, rootReal: string): Promise { - const placed = this.placedRoots.get(mount.config.name); + // One start's placements, whatever a start in the meantime sets. + const { placedRoots, placedAbove } = this; + const placed = placedRoots.get(mount.config.name); if (!mount.direct || placed === undefined || placed === rootReal) return null; - if (!(await stat(rootReal).then(() => true, () => false))) return null; + const there = await stat(rootReal).then(() => true, (err: unknown) => { + const code = errnoOf(err); + return code !== 'ENOENT' && code !== 'ENOTDIR'; + }); + if (!there) return null; const outerPath = resolve(mount.config.path); const nested: NestedMount[] = []; // A real root that holds this mount's own would claim its every path, so @@ -2471,30 +2482,41 @@ export class WorkspaceModule implements Module { const at = (dir: string, name: string, real: string): NestedMount => ({ dir, name, ...(isContainedPath(real, rootReal) ? {} : { real }), include: this.mounts.get(name)?.config.include, }); + const add = (dirs: Array, name: string, real: string) => { + for (const dir of dirs) { + if (dir !== undefined && !nested.some((n) => n.name === name && n.dir === dir)) nested.push(at(dir, name, real)); + } + }; for (const inner of this.config.mounts) { if (inner.name === mount.config.name) continue; - const configured = nestedDir(outerPath, resolve(inner.path)); + const innerPath = resolve(inner.path); + // A mount configured at this mount's own path lies where this root + // does, so a start nests it nowhere in it. + if (innerPath === outerPath) continue; + // Each directory above its configured path that is this root on disk, + // as a start finds it (readNesting's throughRoot): one beneath this + // mount's configured path where it resolves beneath this root now, and + // one above it where the last start found it. + const above = (placedAbove.get(inner.name) ?? []).filter((a) => nestedDir(outerPath, a.dir) === undefined && a.real === rootReal).map((a) => nestedDir(a.dir, innerPath)); + const configured = nestedDir(outerPath, innerPath); if (configured !== undefined) { const real = await realBeneath(join(rootReal, ...configured.split('/'))); - nested.push(at(configured, inner.name, real)); - const there = nestedDir(rootReal, real); - if (there !== undefined && there !== configured) nested.push(at(there, inner.name, real)); + const segments = configured.split('/'); + const between: string[] = []; + for (let i = 1; i < segments.length; i++) { + if (await realBeneath(join(rootReal, ...segments.slice(0, i))) === rootReal) between.push(segments.slice(i).join('/')); + } + add([configured, ...between, ...above, nestedDir(rootReal, real)], inner.name, real); continue; } - // As a start nests it: where its real root, as the last start placed - // it, lies beneath this root, and where its configured path lies - // beneath a directory that start found to be where this root now lies - // (readNesting's throughRoot). - const real = this.placedRoots.get(inner.name); + // Any other mount, as the last start placed it: where its real root + // lies beneath this root, and where its configured path lies beneath a + // directory that start found to be where this root now lies. One that + // leads here only since, as through this mount's own link, waits for + // the next start. + const real = placedRoots.get(inner.name); if (real === undefined) continue; - const innerPath = resolve(inner.path); - const dirs = [ - nestedDir(rootReal, real), - ...(this.placedAbove.get(inner.name) ?? []).filter((a) => a.real === rootReal).map((a) => nestedDir(a.dir, innerPath)), - ]; - for (const dir of dirs) { - if (dir !== undefined && !nested.some((n) => n.name === inner.name && n.dir === dir)) nested.push(at(dir, inner.name, real)); - } + add([nestedDir(rootReal, real), ...above], inner.name, real); } // Each directory as disk spells it too, as a start adds it, so a path // parsePath has respelled is judged on a disk that folds names. @@ -2591,11 +2613,13 @@ export class WorkspaceModule implements Module { * leaves this one, in the disk readers' words, as it does where it lands * on a file this mount's include leaves out. A mount with nothing nested, * and without both an include and links followed, has nothing to find, so - * it pays nothing. A direct mount's file operation passes the nesting and - * the root its parse found (`nested`, `root`), so where the path lands, and - * any link on its way, is judged beneath the root that parse resolved. + * it pays nothing. The caller gives the nesting and the root to judge by + * (`nested`, `root`): a direct mount's file operation those its parse + * found, so where the path lands, and any link on its way, is judged + * beneath the root that parse resolved; otherwise the start's nesting and + * the configured path. */ - private async landingRefusal(mount: MountState, relativePath: string, path: string, nested = this.nestedOf(mount.config.name), root = resolve(mount.config.path)): Promise { + private async landingRefusal(mount: MountState, relativePath: string, path: string, nested: readonly NestedMount[], root: string): Promise { const judged = mount.config.followSymlinks === true && mount.config.include !== undefined; if (nested.length === 0 && !judged) return null; const unresolved: { code?: string } = {}; @@ -2656,10 +2680,11 @@ export class WorkspaceModule implements Module { * cover names no file of the mount's, whatever disk or history holds * there, so one refusesAsOutside lets through is a directory. A direct * mount has no history: what a run of it as 'chronicle' left in the store - * isn't its own, so disk alone decides, beneath the root its parse resolved - * (`root`). + * isn't its own, so disk alone decides. Disk is read beneath `root`: a + * direct mount's is the root its parse resolved, a store-backed mount's + * its configured path. */ - private async namesFile(mount: MountState, relativePath: string, root = resolve(mount.config.path)): Promise { + private async namesFile(mount: MountState, relativePath: string, root: string): Promise { return relativePath !== '' && insideMount(relativePath, mount.config.include) && ( (!mount.direct && this.getStore().treeGet(mount.treeStateId, relativePath) !== null) || await lstat(join(root, relativePath)).then((s) => !s.isDirectory(), () => false) @@ -3048,8 +3073,10 @@ export class WorkspaceModule implements Module { // mount that doesn't follow symlinks goes through none among the // directories on the way, wherever it leads. Asked again of the path's // resolution after the open, below. - // Every check below judges by the root this read opens under, and reads - // the links on the path's way beneath it. + // The checks below judge by the root step 0 checked, and read the links + // on the path's way beneath it. The lstat and the open go through the + // configured path, and step 5 refuses a read whose root is no longer the + // directory step 0 checked, so what's served is judged beneath one root. const linkView = { root: realMountRoot, followSymlinks: follow }; const nested = await this.nestedFor(mount, realMountRoot); const onTheWay = await linkOnTheWay(linkView, relativePath, 'parents'); @@ -4664,7 +4691,7 @@ export class WorkspaceModule implements Module { // A `path` naming a single file greps just that file; otherwise it is a // directory prefix. (A file path once produced the prefix "notes.md/" // and silently matched nothing.) - const fileScope = await this.namesFile(mount, relativePath); + const fileScope = await this.namesFile(mount, relativePath, resolve(mount.config.path)); // What follows reads the tree and branch intent to go with the pass's // reports, synchronously. They must be the branch the pass decided // on: one selected in between would label its content with the other @@ -4688,7 +4715,7 @@ export class WorkspaceModule implements Module { const asked = new Map(); for (const report of next.reports.values()) { if (report.state !== 'unverified') continue; - const refusal = await this.landingRefusal(mount, report.path, `${mount.config.name}/${report.path}`); + const refusal = await this.landingRefusal(mount, report.path, `${mount.config.name}/${report.path}`, this.nestedOf(mount.config.name), resolve(mount.config.path)); if (refusal) asked.set(report.path, refusal.message); } if (next.branchId === undefined || next.branchId === store.currentBranch().id) { diff --git a/src/modules/workspace/types.ts b/src/modules/workspace/types.ts index 5e97cbf5d..d7e296231 100644 --- a/src/modules/workspace/types.ts +++ b/src/modules/workspace/types.ts @@ -152,8 +152,9 @@ export interface MountConfig { * include doesn't cover, and a listing that isn't recursive names each * directory directly beneath its path that disk has, and each that a * covered entry in history implies, with only the files the include - * covers. A direct mount has no history (`backend`), so there disk alone - * decides each of these. `ignore` still applies to what `include` covers. + * covers. A mount running direct has no history (`backend`), so there + * disk alone decides each of these. `ignore` still applies to what + * `include` covers. * * On a disk that folds names, the parse reads a path in disk's spelling, * as the walk names what it lists, so a covered name whose file disk diff --git a/test/workspace-direct.test.ts b/test/workspace-direct.test.ts index c39c0ca44..fcbcdd4d6 100644 --- a/test/workspace-direct.test.ts +++ b/test/workspace-direct.test.ts @@ -7,7 +7,7 @@ import test, { type TestContext } from 'node:test'; import assert from 'node:assert/strict'; -import { appendFileSync, chmodSync, existsSync, mkdirSync, mkdtempSync, renameSync, rmSync, statSync, symlinkSync, truncateSync, unlinkSync, writeFileSync } from 'node:fs'; +import { appendFileSync, chmodSync, existsSync, mkdirSync, mkdtempSync, realpathSync, renameSync, rmSync, statSync, symlinkSync, truncateSync, unlinkSync, writeFileSync } from 'node:fs'; import { createRequire, syncBuiltinESMExports } from 'node:module'; import { tmpdir } from 'node:os'; import { dirname, join } from 'node:path'; @@ -107,7 +107,7 @@ function watched(m: WorkspaceModule): string[] { } /** Run `fn` with `node:fs/promises`'s `name`, as the module's imports see it, replaced by what `make` makes of it. */ -async function withFsPromise(name: 'lstat' | 'open' | 'realpath', make: (original: (...args: any[]) => Promise) => (...args: any[]) => Promise, fn: () => Promise): Promise { +async function withFsPromise(name: 'lstat' | 'open' | 'realpath' | 'stat', make: (original: (...args: any[]) => Promise) => (...args: any[]) => Promise, fn: () => Promise): Promise { const fsp = createRequire(import.meta.url)('node:fs/promises') as Record Promise>; const original = fsp[name]!; fsp[name] = make(original); @@ -1083,12 +1083,14 @@ test("a file tool's parse reads a link on the way beneath the root it resolved, test("a mount configured in a moved direct root's new directory, through a link to that directory, is nested as a start there nests it", async (t) => { const env = new Env(t); // `work` is configured at `b/work`, a link to `b` itself, so a start with - // `ref` at `b` nests `work` at `work`, by its configured path alone. + // `ref` at `b` nests `work` at `work`, by its configured path alone, and + // not at `b/work`, which `b` holds too, as no directory above `b` is `b`. const a = join(env.root, 'a'); const b = join(env.root, 'b'); mkdirSync(a, { recursive: true }); - mkdirSync(b, { recursive: true }); + mkdirSync(join(b, 'b', 'work'), { recursive: true }); writeFileSync(join(b, 'file.txt'), 'b file\n'); + writeFileSync(join(b, 'b', 'work', 'own.txt'), 'b own\n'); symlinkSync('.', join(b, 'work')); const link = join(env.root, 'link'); symlinkSync(a, link); @@ -1101,6 +1103,7 @@ test("a mount configured in a moved direct root's new directory, through a link read: await call(m, 'read', { path: 'ref/work/file.txt' }), disk: await m.readFileFromDisk('ref/work/file.txt').then(() => 'read', (err: unknown) => (err instanceof WorkspaceReadError ? err.code : String(err))), own: (await data<{ content: string }>(m, 'read', { path: 'ref/file.txt' })).content, + alike: (await data<{ content: string }>(m, 'read', { path: 'ref/b/work/own.txt' })).content, found: (await data<{ results: Array<{ file: string }> }>(m, 'grep', { pattern: 'b file', path: 'ref' })).results.map((r) => r.file), }); const fromStart = await answers(started); @@ -1140,6 +1143,198 @@ test("a mount configured at a link into a moved direct root's new directory is n assert.deepEqual(await answers(moved), fromStart); }); +test("a mount configured beneath a moved direct root, through a directory there that's the root itself, is nested beneath that directory too, as a start there nests it", async (t) => { + const env = new Env(t); + // `work` is configured at `link/sub/work`. At `b`, `sub` and `work` are + // links to `.`, so a start with `ref` at `b` nests `work` at `sub/work` + // and, beneath `sub`, which is `b` itself, at `work`. + const a = join(env.root, 'a'); + const b = join(env.root, 'b'); + mkdirSync(join(a, 'sub', 'work'), { recursive: true }); + mkdirSync(b, { recursive: true }); + writeFileSync(join(b, 'file.txt'), 'b file\n'); + symlinkSync('.', join(b, 'sub')); + symlinkSync('.', join(b, 'work')); + const link = join(env.root, 'link'); + symlinkSync(a, link); + const mounts: [Partial, MountConfig[]] = [{ path: link, followSymlinks: true }, [{ name: 'work', path: join(link, 'sub', 'work'), mode: 'read-only', backend: 'direct' }]]; + const moved = await env.open(...mounts); + rmSync(link); + symlinkSync(b, link); + const started = await new Env(t).open(...mounts); + const answers = async (m: WorkspaceModule) => ({ + read: await call(m, 'read', { path: 'ref/work/file.txt' }), + disk: await m.readFileFromDisk('ref/work/file.txt').then(() => 'read', (err: unknown) => (err instanceof WorkspaceReadError ? err.code : String(err))), + ls: await call(m, 'ls', { path: 'ref/work' }), + }); + const fromStart = await answers(started); + assert.match(fromStart.read.error ?? '', /lies in mount "work", nested in mount "ref"/); + assert.equal(fromStart.disk, 'other_mount'); + assert.deepEqual(await answers(moved), fromStart); +}); + +test("a mount configured beneath a direct root moved to a directory above its configured path is nested beneath that directory too, as a start there nests it", async (t) => { + const env = new Env(t); + // `ref` is configured at `b/link`, which leads first to `a` and then to + // `.`, `b` itself; `work` is configured at `b/link/work`. A start with + // `ref` at `b` nests `work` at `work` and, beneath `b`, at `link/work`. + const a = join(env.root, 'a'); + const b = join(env.root, 'b'); + mkdirSync(join(a, 'work'), { recursive: true }); + mkdirSync(join(b, 'work'), { recursive: true }); + writeFileSync(join(b, 'work', 'f.txt'), 'work f\n'); + const link = join(b, 'link'); + symlinkSync(a, link); + const mounts: [Partial, MountConfig[]] = [{ path: link, followSymlinks: true }, [{ name: 'work', path: join(link, 'work'), mode: 'read-only', backend: 'direct' }]]; + const moved = await env.open(...mounts); + rmSync(link); + symlinkSync('.', link); + const started = await new Env(t).open(...mounts); + const answers = async (m: WorkspaceModule) => ({ + read: await call(m, 'read', { path: 'ref/link/work/f.txt' }), + disk: await m.readFileFromDisk('ref/link/work/f.txt').then(() => 'read', (err: unknown) => (err instanceof WorkspaceReadError ? err.code : String(err))), + }); + const fromStart = await answers(started); + assert.match(fromStart.read.error ?? '', /lies in mount "work", nested in mount "ref"/); + assert.equal(fromStart.disk, 'other_mount'); + assert.deepEqual(await answers(moved), fromStart); +}); + +test("a mount configured at a moved direct root's own path is nested nowhere in it, as a start there nests it", async (t) => { + const env = new Env(t); + // `ref` and `twin` are both configured at `link`. It leads first to `a`, + // which lies in `b` as `sub`, and then to `b`, where a start nests + // neither mount in the other. + const b = join(env.root, 'b'); + const a = join(b, 'sub'); + mkdirSync(a, { recursive: true }); + writeFileSync(join(a, 'f.txt'), 'sub f\n'); + const link = join(env.root, 'link'); + symlinkSync(a, link); + const mounts: [Partial, MountConfig[]] = [{ path: link }, [{ name: 'twin', path: link, mode: 'read-only', backend: 'direct' }]]; + const moved = await env.open(...mounts); + rmSync(link); + symlinkSync(b, link); + const started = await new Env(t).open(...mounts); + const answers = async (m: WorkspaceModule) => ({ + read: await call(m, 'read', { path: 'ref/sub/f.txt' }), + disk: await m.readFileFromDisk('ref/sub/f.txt').then(() => 'read', (err: unknown) => (err instanceof WorkspaceReadError ? err.code : String(err))), + }); + const fromStart = await answers(started); + assert.equal((fromStart.read.data as { content: string }).content, ' 1\tsub f'); + assert.equal(fromStart.disk, 'read'); + assert.deepEqual(await answers(moved), fromStart); +}); + +test("a direct mount whose moved root can't be looked at for a reason other than absence nests what a start there would, not what the last start placed", async (t) => { + const env = new Env(t); + // `work` lies at `b/work`, outside `a`, so the start with `ref` at `a` + // nests nothing; at `b`, it's nested at `work`. + const a = join(env.root, 'a'); + const b = join(env.root, 'b'); + mkdirSync(a, { recursive: true }); + mkdirSync(join(b, 'work'), { recursive: true }); + writeFileSync(join(b, 'work', 'file.txt'), 'work file\n'); + const link = join(env.root, 'link'); + symlinkSync(a, link); + const m = await env.open({ path: link }, [{ name: 'work', path: join(b, 'work'), mode: 'read-only', backend: 'direct' }]); + rmSync(link); + symlinkSync(b, link); + // A look at the moved root fails, as on an I/O error, though everything + // else reads it: the nesting is still judged beneath it. + const realB = realpathSync(b); + const answer = await withFsPromise('stat', (original) => async (...args: any[]) => { + if (args[0] === realB) throw Object.assign(new Error(`EIO: i/o error, stat '${realB}'`), { code: 'EIO' }); + return original(...args); + }, () => call(m, 'read', { path: 'ref/work/file.txt' })); + assert.match(answer.error ?? '', /lies in mount "work", nested in mount "ref"/); +}); + +test("a direct grep of a named file reads it beneath the root its read resolved, though the root is repointed before the open", async (t) => { + const env = new Env(t); + // `b` lies in `a` as `sub`, and at `b`, `file.txt` is a link into `work`'s + // directory there, at a file `work`'s include leaves out. + const a = join(env.root, 'a'); + const b = join(a, 'sub'); + mkdirSync(join(a, 'work'), { recursive: true }); + mkdirSync(join(b, 'work'), { recursive: true }); + writeFileSync(join(a, 'file.txt'), 'a needle\n'); + writeFileSync(join(b, 'work', 'secret.env'), 'S=needle\n'); + symlinkSync(join('work', 'secret.env'), join(b, 'file.txt')); + const link = join(env.root, 'link'); + symlinkSync(a, link); + const m = await env.open({ path: link, followSymlinks: true }, [{ name: 'work', path: join(link, 'work'), mode: 'read-only', backend: 'direct', include: ['*.json'] }]); + // The parse resolves the root at `a`, and the read resolves it again; the + // link is repointed to `b` as that answers, before the file is opened. + let resolved = 0; + const found = await withFsPromise('realpath', (original) => async (...args: any[]) => { + const real = await original(...args); + if (args[0] === link && ++resolved === 2) { + rmSync(link); + symlinkSync(b, link); + } + return real; + }, () => data<{ results: Array<{ file: string; matches: Array<{ text: string }> }> }>(m, 'grep', { pattern: 'needle', path: 'ref/file.txt' })); + assert.equal(resolved, 2); + assert.deepEqual(found.results.map((r) => [r.file, r.matches.map((x) => x.text)]), [['ref/file.txt', ['a needle']]]); +}); + +test("a direct grep judges whether a path it's given names a file beneath the root its parse resolved, though the root moves after", async (t) => { + const env = new Env(t); + // At `a`, `notes` is a file; at `b`, a directory. + const a = join(env.root, 'a'); + const b = join(env.root, 'b'); + mkdirSync(a, { recursive: true }); + mkdirSync(join(b, 'notes'), { recursive: true }); + writeFileSync(join(a, 'notes'), 'a needle\n'); + writeFileSync(join(b, 'notes', 'inner.md'), 'b needle\n'); + const link = join(env.root, 'link'); + symlinkSync(a, link); + const m = await env.open({ path: link }); + // The parse resolves the root at `a`, and the link is repointed to `b` as + // that answers: the path is judged a file beneath `a`, and the read, + // which resolves the root again, finds a directory there. + let repointed = false; + const found = await withFsPromise('realpath', (original) => async (...args: any[]) => { + const real = await original(...args); + if (!repointed && args[0] === link) { + repointed = true; + rmSync(link); + symlinkSync(b, link); + } + return real; + }, () => data<{ results: unknown[]; skipped: Array<{ file: string; reason: string }> }>(m, 'grep', { pattern: 'needle', path: 'ref/notes' })); + assert.ok(repointed); + assert.deepEqual([found.results, found.skipped], [[], [{ file: 'ref/notes', reason: 'a directory' }]]); +}); + +test("a direct listing judges whether a path its include leaves out is a directory beneath the root its parse resolved, though the root moves after", async (t) => { + const env = new Env(t); + // The include covers `*.md` alone. At `a`, `dir` is a directory, which a + // listing may name; at `b`, a file it leaves out. + const a = join(env.root, 'a'); + const b = join(env.root, 'b'); + mkdirSync(join(a, 'dir'), { recursive: true }); + mkdirSync(b, { recursive: true }); + writeFileSync(join(b, 'dir'), 'b dir\n'); + const link = join(env.root, 'link'); + symlinkSync(a, link); + const m = await env.open({ path: link, include: ['*.md'] }); + assert.match(await failure(await new Env(t).open({ path: b, include: ['*.md'] }), 'ls', { path: 'ref/dir' }), /is outside the mount's include/); + let repointed = false; + const listed = await withFsPromise('realpath', (original) => async (...args: any[]) => { + const real = await original(...args); + if (!repointed && args[0] === link) { + repointed = true; + rmSync(link); + symlinkSync(b, link); + } + return real; + }, () => call(m, 'ls', { path: 'ref/dir' })); + assert.ok(repointed); + assert.doesNotMatch(listed.error ?? '', /is outside the mount's include/); +}); + test("a direct mount whose root moves between a read's parse and its open names the mount a link on the way leads into, by the root it opens under", async (t) => { const env = new Env(t); const a = join(env.root, 'a'); From c5eb97eb8465012fae0aa98de5a56058a5b570e4 Mon Sep 17 00:00:00 2001 From: Beth-2709 Date: Mon, 19 Oct 2026 19:16:02 -0400 Subject: [PATCH 11/13] refactor(workspace): a start and a moved direct root place a mount with one function A direct mount whose root has moved nests the other mounts as a start there would. movedNesting wrote that rule out a second time, beside readNesting's, and the two could part on a link layout: a directory above the moved mount's own path that leads to the root since the last start, or a second mount configured at the same path whose parent is the root, gave a moved call answers a start there wouldn't, and by our reading so could the order of two mounts' spellings on a disk that folds case. placementOf now holds the rule once: a mount is nested at its directory as configured beneath the outer mount's path, beneath each directory above its configured path that is the outer root on disk, and where its real root lies beneath the outer root, each also as disk spells it, with its real root claimed where it doesn't hold the outer root. readNesting places every pair with it from what the start resolves. movedNesting places each mount with it from where the mount's real root and the directories above its configured path lie on disk: resolved now for a mount configured at the moved mount's path or beneath it, beneath the moved root where they lie beneath that path and as resolving the path finds them above it, and as the last start found them for any other mount. So a moved root nests as a start would by construction, wherever it knows what a start would know, and the gap for a mount configured elsewhere whose own root has moved since stays as the doc names it. The skip of a mount at the moved mount's own path, and the filter on the last start's directories beneath it, go: the shared function places such mounts as a start does. The doc of what a read or a walk serves after a root repointed mid-call now says what holds: no file the include leaves out, nothing a walk serves in another mount's directory as its own nesting places it, and nothing a read serves in the directory of a mount its own nesting claims by real root. It names the two races left to whoever writes the disk, as a rename during a call is on any mount: on a disk that folds names, a read judging the include by the path as its parse spelled it, and a mount nested by its directories alone, judged by the parse's nesting where the root moves between a read's parse and its open. Texts: MountView.root's comment says it's the directory its readers read beneath, a direct mount's resolved root among them; diskView's doc says which reads open beneath its root and that a read of one file goes through openContainedFile; and landingRefusal's doc says what each caller gives it. Tests: four that compare a moved root's answers with a fresh start's, for a mount at the moved mount's own path, nested beneath a directory above that path that's the root now; a mount beneath a root whose path's own directory above leads elsewhere now; one beneath a directory on the way that's the root though it's not the first, and beneath none that isn't; and a root moved to a directory beneath its own configured path. A fifth checks that a root which now leads through a file keeps the start's placement, as a gone one does. Co-Authored-By: Claude Opus 5.5 --- src/modules/workspace/index.ts | 182 +++++++++++++++++-------------- src/modules/workspace/observe.ts | 6 +- test/workspace-direct.test.ts | 127 +++++++++++++++++++++ 3 files changed, 233 insertions(+), 82 deletions(-) diff --git a/src/modules/workspace/index.ts b/src/modules/workspace/index.ts index b7d30108a..3357f74a8 100644 --- a/src/modules/workspace/index.ts +++ b/src/modules/workspace/index.ts @@ -372,6 +372,48 @@ function nestedDir(from: string, to: string): string | undefined { return rel.split(sep).join('/'); } +/** Each directory above `path`, innermost first, to the filesystem's root. */ +function ancestorsOf(path: string): string[] { + const dirs: string[] = []; + for (let dir = dirname(path); ; dir = dirname(dir)) { + dirs.push(dir); + if (dirname(dir) === dir) return dirs; + } +} + +/** + * Where a mount is nested in an outer mount, as a start places it: at its + * directory as configured beneath the outer mount's configured path + * (`outerPath`), beneath each directory above its configured path that is + * the outer root on disk (`above`, each with where it lies on disk), and + * where its real root (`real`) lies beneath the outer root (`outerReal`), + * each also as disk spells it beneath `spellAt`. Its real root is claimed + * where it doesn't hold the outer root, which it would otherwise claim + * whole. Undefined where it's nested nowhere. + */ +async function placementOf( + outerPath: string, + outerReal: string, + innerPath: string, + real: string, + above: ReadonlyArray<{ dir: string; real: string }>, + spellAt: string, +): Promise<{ configured: string | undefined; fromRoots: Array<{ dir: string; root: string }>; spellings: Array<{ dir: string; root: string }>; claims: boolean } | undefined> { + const configured = nestedDir(outerPath, innerPath); + const throughRoot = above.filter((a) => a.real === outerReal).map((a) => nestedDir(a.dir, innerPath)); + const fromRoots: Array<{ dir: string; root: string }> = []; + for (const dir of [configured, ...throughRoot, nestedDir(outerReal, real)]) { + if (dir !== undefined && !fromRoots.some((s) => s.dir === dir)) fromRoots.push({ dir, root: join(outerPath, ...dir.split('/')) }); + } + if (fromRoots.length === 0) return undefined; + const spellings = [...fromRoots]; + for (const found of fromRoots) { + const dir = await diskSpelling(spellAt, found.dir); + if (!spellings.some((s) => s.dir === dir)) spellings.push({ dir, root: join(outerPath, ...dir.split('/')) }); + } + return { configured, fromRoots, spellings, claims: !isContainedPath(real, outerReal) }; +} + /** A mount nested in another, as the outer mount's otherMounts holds it. */ interface NestedMountEntry { name: string; @@ -1162,8 +1204,11 @@ export class WorkspaceModule implements Module { * since it promises what's at its root now and keeps no evidence to check * that against (MountConfig.backend), and its root as realpath resolves it, * or as given where that fails, as a pass's runtime takes it. The view - * reads beneath that root, so a walk or a read judges and opens its files - * beneath one root, however its configured path is repointed meanwhile. + * reads beneath that root, so a walk, a listing and grep's reads judge + * and open their files beneath one root, however its configured path is + * repointed meanwhile. A read of one file goes through openContainedFile + * instead, which refuses it once its root is no longer the directory it + * checked. */ private async diskView(mount: MountState): Promise<{ view: MountView; rootReal: string; moved: NestedMount[] | null }> { const root = resolve(mount.config.path); @@ -2366,10 +2411,7 @@ export class WorkspaceModule implements Module { const given = resolve(mount.path); reals.set(mount.name, await realBeneath(given)); const dirs: Array<{ dir: string; real: string }> = []; - for (let dir = dirname(given); ; dir = dirname(dir)) { - dirs.push({ dir, real: await realBeneath(dir) }); - if (dirname(dir) === dir) break; - } + for (const dir of ancestorsOf(given)) dirs.push({ dir, real: await realBeneath(dir) }); above.set(mount.name, dirs); } const nesting = new Map(); @@ -2379,23 +2421,9 @@ export class WorkspaceModule implements Module { for (const inner of this.config.mounts) { if (outer === inner) continue; const real = reals.get(inner.name)!; - const innerPath = resolve(inner.path); - const configured = nestedDir(outerPath, innerPath); - const fromRoots: Array<{ dir: string; root: string }> = []; - // Its directory as configured, read against this mount's root as - // configured and from each directory above it that is this mount's - // root on disk, and its real root against this mount's. - const throughRoot = above.get(inner.name)!.filter((a) => a.real === outerReal).map((a) => nestedDir(a.dir, innerPath)); - for (const dir of [configured, ...throughRoot, nestedDir(outerReal, real)]) { - if (dir !== undefined && !fromRoots.some((s) => s.dir === dir)) fromRoots.push({ dir, root: join(outerPath, ...dir.split('/')) }); - } - if (fromRoots.length === 0) continue; - const spellings = [...fromRoots]; - for (const found of fromRoots) { - const dir = await diskSpelling(outerPath, found.dir); - if (!spellings.some((s) => s.dir === dir)) spellings.push({ dir, root: join(outerPath, ...dir.split('/')) }); - } - const claims = !isContainedPath(real, outerReal); + const at = await placementOf(outerPath, outerReal, resolve(inner.path), real, above.get(inner.name)!, outerPath); + if (at === undefined) continue; + const { configured, fromRoots, spellings, claims } = at; nesting.set(outer.name, [...(nesting.get(outer.name) ?? []), { name: inner.name, ...(claims ? { real } : {}), fromRoots, spellings }]); const placed = fromRoots.filter((s) => s.dir !== configured); if (placed.length > 0) { @@ -2441,21 +2469,24 @@ export class WorkspaceModule implements Module { * for themselves, and judge the include and the nesting again beneath it. * So a root repointed during a call can change which answer or refusal a * path gets, but nothing a read or a walk serves is a file the include - * leaves out, or in another mount's directory as its own nesting places it, - * except that on a disk that folds names a read judges the include by the + * leaves out, nothing a walk serves lies in another mount's directory as + * its own nesting places it, and nothing a read serves lies in the + * directory of a mount its own nesting claims by real root. Whoever writes + * the disk can race a read in two ways, as a rename during a call can on + * any mount. On a disk that folds names, a read judges the include by the * path as its parse spelled it, so a root repointed between the two can - * have it serve a file the new root spells otherwise, as a rename during a - * call can on any mount (sameLevels). A mount configured beneath its - * configured path lies where that path now leads, as a start would place - * it: at its configured directory, beneath each directory on the way there - * that resolves to `rootReal` (realBeneath), and where its directory - * resolves beneath `rootReal`, its real root, each also as disk spells it - * (diskSpelling). Any other mount lies where the last start placed it: it's - * nested where its real root lies beneath `rootReal`, and where its - * configured path lies beneath a directory that start found at `rootReal`. - * A mount configured at this mount's own path lies where its root does, so - * it's nested nowhere in it. Each carries its own include, as nestedOf - * gives it. It resolves nothing but directories beneath this mount's own + * have it serve a file the new root spells otherwise (sameLevels). And a + * mount nested by its directories alone, since its real root holds this + * one, is judged by the parse's nesting, so a root repointed between a + * read's parse and its open can have it serve a path in that mount's + * directory. Each mount is placed as a start places it (placementOf), from + * where its real root and the directories above its configured path lie on + * disk. For a mount configured at this mount's own path or beneath it, + * that's where they lie now: beneath `rootReal` where they lie beneath this + * mount's path, and the directories above this mount's path as resolving + * that path finds them. For any other mount, it's where the last start + * found them. Each carries its own include, as nestedOf gives it. It + * resolves nothing but this mount's own path and directories beneath its * root, so a share elsewhere that doesn't answer holds only its own calls, * unless such a directory is a link into it, where that mount's files lie. * What it leaves to the next start: a mount configured elsewhere whose own @@ -2477,52 +2508,41 @@ export class WorkspaceModule implements Module { if (!there) return null; const outerPath = resolve(mount.config.path); const nested: NestedMount[] = []; - // A real root that holds this mount's own would claim its every path, so - // such a mount is nested by its directories alone, as a start nests it. - const at = (dir: string, name: string, real: string): NestedMount => ({ - dir, name, ...(isContainedPath(real, rootReal) ? {} : { real }), include: this.mounts.get(name)?.config.include, - }); - const add = (dirs: Array, name: string, real: string) => { - for (const dir of dirs) { - if (dir !== undefined && !nested.some((n) => n.name === name && n.dir === dir)) nested.push(at(dir, name, real)); - } - }; + // The directories above this mount's own path, as they lie on disk now, + // resolved as resolving the path itself does. + let ownAbove: Array<{ dir: string; real: string }> | undefined; + const aboveOwn = async () => ownAbove ??= await Promise.all(ancestorsOf(outerPath).map(async (dir) => ({ dir, real: await realBeneath(dir) }))); for (const inner of this.config.mounts) { if (inner.name === mount.config.name) continue; const innerPath = resolve(inner.path); - // A mount configured at this mount's own path lies where this root - // does, so a start nests it nowhere in it. - if (innerPath === outerPath) continue; - // Each directory above its configured path that is this root on disk, - // as a start finds it (readNesting's throughRoot): one beneath this - // mount's configured path where it resolves beneath this root now, and - // one above it where the last start found it. - const above = (placedAbove.get(inner.name) ?? []).filter((a) => nestedDir(outerPath, a.dir) === undefined && a.real === rootReal).map((a) => nestedDir(a.dir, innerPath)); const configured = nestedDir(outerPath, innerPath); - if (configured !== undefined) { - const real = await realBeneath(join(rootReal, ...configured.split('/'))); - const segments = configured.split('/'); - const between: string[] = []; - for (let i = 1; i < segments.length; i++) { - if (await realBeneath(join(rootReal, ...segments.slice(0, i))) === rootReal) between.push(segments.slice(i).join('/')); + let real: string | undefined; + let above: ReadonlyArray<{ dir: string; real: string }>; + if (configured !== undefined || innerPath === outerPath) { + // A mount configured on this mount's own path, or beneath it, lies + // where that path now leads: its real root, and each directory above + // it, resolved now, beneath this root where they lie beneath it. + const segments = configured?.split('/') ?? []; + real = segments.length === 0 ? rootReal : await realBeneath(join(rootReal, ...segments)); + const beneath: Array<{ dir: string; real: string }> = []; + for (let i = segments.length - 1; i >= 1; i--) { + beneath.push({ dir: join(outerPath, ...segments.slice(0, i)), real: await realBeneath(join(rootReal, ...segments.slice(0, i))) }); } - add([configured, ...between, ...above, nestedDir(rootReal, real)], inner.name, real); - continue; + // This mount's own path, which is the root, would add only the + // configured directory, which placementOf adds anyway. + above = [...beneath, ...await aboveOwn()]; + } else { + // Any other mount lies where the last start found it and the + // directories above its path. One leading here only since, as + // through this mount's own link, waits for the next start. + real = placedRoots.get(inner.name); + if (real === undefined) continue; + above = placedAbove.get(inner.name) ?? []; } - // Any other mount, as the last start placed it: where its real root - // lies beneath this root, and where its configured path lies beneath a - // directory that start found to be where this root now lies. One that - // leads here only since, as through this mount's own link, waits for - // the next start. - const real = placedRoots.get(inner.name); - if (real === undefined) continue; - add([nestedDir(rootReal, real), ...above], inner.name, real); - } - // Each directory as disk spells it too, as a start adds it, so a path - // parsePath has respelled is judged on a disk that folds names. - for (const n of [...nested]) { - const dir = await diskSpelling(rootReal, n.dir); - if (!nested.some((m) => m.name === n.name && m.dir === dir)) nested.push({ ...n, dir }); + const at = await placementOf(outerPath, rootReal, innerPath, real, above, rootReal); + if (at === undefined) continue; + const include = this.mounts.get(inner.name)?.config.include; + for (const s of at.spellings) nested.push({ dir: s.dir, name: inner.name, ...(at.claims ? { real } : {}), include }); } return nested; } @@ -2614,10 +2634,10 @@ export class WorkspaceModule implements Module { * on a file this mount's include leaves out. A mount with nothing nested, * and without both an include and links followed, has nothing to find, so * it pays nothing. The caller gives the nesting and the root to judge by - * (`nested`, `root`): a direct mount's file operation those its parse - * found, so where the path lands, and any link on its way, is judged - * beneath the root that parse resolved; otherwise the start's nesting and - * the configured path. + * (`nested`, `root`). A direct mount's file operation gives those its + * parse found, so where the path lands, and any link on its way, is judged + * beneath the root that parse resolved; any other caller gives the start's + * nesting and the configured path. */ private async landingRefusal(mount: MountState, relativePath: string, path: string, nested: readonly NestedMount[], root: string): Promise { const judged = mount.config.followSymlinks === true && mount.config.include !== undefined; diff --git a/src/modules/workspace/observe.ts b/src/modules/workspace/observe.ts index 911d386c3..a19794623 100644 --- a/src/modules/workspace/observe.ts +++ b/src/modules/workspace/observe.ts @@ -49,7 +49,11 @@ import { basename, dirname, isAbsolute, join, posix, relative, resolve, sep } fr import { sameRootIdentity, type Fingerprint, type RootIdentity } from './disk-agreement.js'; export interface MountView { - /** Mount root as configured. */ + /** + * The directory its readers read beneath: the mount's root as configured, + * or, for a direct mount, the root its view resolved (diskView), so a + * walk opens what it judges beneath one root. + */ root: string; /** The mount's name, which a tool puts before a path in the mount. */ name: string; diff --git a/test/workspace-direct.test.ts b/test/workspace-direct.test.ts index fcbcdd4d6..50b4fa95d 100644 --- a/test/workspace-direct.test.ts +++ b/test/workspace-direct.test.ts @@ -1173,6 +1173,64 @@ test("a mount configured beneath a moved direct root, through a directory there assert.deepEqual(await answers(moved), fromStart); }); +test("a mount configured beneath a moved direct root is nested beneath each directory on the way that's the root, the second among them, and no other", async (t) => { + const env = new Env(t); + // `work` is configured at `link/x/y/work`. At `b`, `x` is a directory and + // `x/y` leads to `..`, `b` itself, and `work` to `.`: a start with `ref` at + // `b` nests `work` at `x/y/work` and, beneath `x/y`, at `work`, and not + // beneath `x`, which isn't `b`, so `y/work` is `ref`'s own. + const a = join(env.root, 'a'); + const b = join(env.root, 'b'); + mkdirSync(join(a, 'x', 'y', 'work'), { recursive: true }); + mkdirSync(join(b, 'x'), { recursive: true }); + mkdirSync(join(b, 'y', 'work'), { recursive: true }); + writeFileSync(join(b, 'file.txt'), 'b file\n'); + writeFileSync(join(b, 'y', 'work', 'own.txt'), 'b own\n'); + symlinkSync('..', join(b, 'x', 'y')); + symlinkSync('.', join(b, 'work')); + const link = join(env.root, 'link'); + symlinkSync(a, link); + const mounts: [Partial, MountConfig[]] = [{ path: link, followSymlinks: true }, [{ name: 'work', path: join(link, 'x', 'y', 'work'), mode: 'read-only', backend: 'direct' }]]; + const moved = await env.open(...mounts); + rmSync(link); + symlinkSync(b, link); + const started = await new Env(t).open(...mounts); + const answers = async (m: WorkspaceModule) => ({ + read: await call(m, 'read', { path: 'ref/work/file.txt' }), + own: await call(m, 'read', { path: 'ref/y/work/own.txt' }), + }); + const fromStart = await answers(started); + assert.match(fromStart.read.error ?? '', /lies in mount "work", nested in mount "ref"/); + assert.equal((fromStart.own.data as { content: string }).content, ' 1\tb own'); + assert.deepEqual(await answers(moved), fromStart); +}); + +test("a direct root moved to a directory beneath its own configured path nests no mount by where the last start found a directory beneath that path", async (t) => { + const env = new Env(t); + // `work` is configured at `link/sub/work`. The start, with `link` at `a`, + // finds `link/sub` at `a/sub`; then `link` leads to `a/sub`, where + // `sub/work` is `work`'s directory and `work` is `ref`'s own. + const a = join(env.root, 'a'); + mkdirSync(join(a, 'sub', 'sub', 'work'), { recursive: true }); + mkdirSync(join(a, 'sub', 'work'), { recursive: true }); + writeFileSync(join(a, 'sub', 'work', 'own.txt'), 'sub own\n'); + const link = join(env.root, 'link'); + symlinkSync(a, link); + const mounts: [Partial, MountConfig[]] = [{ path: link }, [{ name: 'work', path: join(link, 'sub', 'work'), mode: 'read-only', backend: 'direct' }]]; + const moved = await env.open(...mounts); + rmSync(link); + symlinkSync(join(a, 'sub'), link); + const started = await new Env(t).open(...mounts); + const answers = async (m: WorkspaceModule) => ({ + own: await call(m, 'read', { path: 'ref/work/own.txt' }), + nested: await call(m, 'read', { path: 'ref/sub/work/x.txt' }), + }); + const fromStart = await answers(started); + assert.equal((fromStart.own.data as { content: string }).content, ' 1\tsub own'); + assert.match(fromStart.nested.error ?? '', /lies in mount "work", nested in mount "ref"/); + assert.deepEqual(await answers(moved), fromStart); +}); + test("a mount configured beneath a direct root moved to a directory above its configured path is nested beneath that directory too, as a start there nests it", async (t) => { const env = new Env(t); // `ref` is configured at `b/link`, which leads first to `a` and then to @@ -1226,6 +1284,59 @@ test("a mount configured at a moved direct root's own path is nested nowhere in assert.deepEqual(await answers(moved), fromStart); }); +test("a mount configured at a moved direct root's own path is nested beneath a directory above that path that's the root now, as a start there nests it", async (t) => { + const env = new Env(t); + // `ref` and `twin` are both configured at `b/link`, which leads first to + // `b/sub` and then to `.`, `b` itself: a start there nests `twin` in `ref` + // at `link`, beneath `b`, the directory above `b/link`. + const b = join(env.root, 'b'); + mkdirSync(join(b, 'sub'), { recursive: true }); + writeFileSync(join(b, 'f.txt'), 'b f\n'); + const link = join(b, 'link'); + symlinkSync(join(b, 'sub'), link); + const mounts: [Partial, MountConfig[]] = [{ path: link, followSymlinks: true }, [{ name: 'twin', path: link, mode: 'read-only', backend: 'direct' }]]; + const moved = await env.open(...mounts); + rmSync(link); + symlinkSync('.', link); + const started = await new Env(t).open(...mounts); + const answers = async (m: WorkspaceModule) => ({ + read: await call(m, 'read', { path: 'ref/link/f.txt' }), + own: (await data<{ content: string }>(m, 'read', { path: 'ref/f.txt' })).content, + }); + const fromStart = await answers(started); + assert.match(fromStart.read.error ?? '', /lies in mount "twin", nested in mount "ref"/); + assert.deepEqual(await answers(moved), fromStart); +}); + +test("a mount configured beneath a direct root whose path's own directory above leads elsewhere now is nested as a start there nests it", async (t) => { + const env = new Env(t); + // `ref` is configured at `current/ref` and `work` at `current/ref/work`. + // `current` leads first to `a` and then to `b`, where `ref` and `work` lead + // to `.`: a start there finds `current` at `b`, the root, and nests `work` + // at `work` and, beneath `current`, at `ref/work`. + const a = join(env.root, 'a'); + const b = join(env.root, 'b'); + mkdirSync(join(a, 'ref', 'work'), { recursive: true }); + mkdirSync(b, { recursive: true }); + writeFileSync(join(b, 'file.txt'), 'b file\n'); + symlinkSync('.', join(b, 'ref')); + symlinkSync('.', join(b, 'work')); + const current = join(env.root, 'current'); + symlinkSync(a, current); + const mounts: [Partial, MountConfig[]] = [{ path: join(current, 'ref'), followSymlinks: true }, [{ name: 'work', path: join(current, 'ref', 'work'), mode: 'read-only', backend: 'direct' }]]; + const moved = await env.open(...mounts); + rmSync(current); + symlinkSync(b, current); + const started = await new Env(t).open(...mounts); + const answers = async (m: WorkspaceModule) => ({ + through: await call(m, 'read', { path: 'ref/ref/work/file.txt' }), + read: await call(m, 'read', { path: 'ref/work/file.txt' }), + }); + const fromStart = await answers(started); + assert.match(fromStart.through.error ?? '', /lies in mount "work", nested in mount "ref"/); + assert.deepEqual(await answers(moved), fromStart); +}); + test("a direct mount whose moved root can't be looked at for a reason other than absence nests what a start there would, not what the last start placed", async (t) => { const env = new Env(t); // `work` lies at `b/work`, outside `a`, so the start with `ref` at `a` @@ -1488,6 +1599,22 @@ test("a direct mount whose root is gone, beneath a link, keeps the start's place assert.match(await failure(m, 'read', { path: 'ref/work/file.txt' }), /lies in mount "work", nested in mount "ref"/); }); +test("a direct mount whose root now leads through a file keeps the start's placement, as one that's gone does", async (t) => { + const env = new Env(t); + // The start, with `link` at `a`, nests `work` at `work`; then `link` leads + // beneath a file, where nothing can be. + const a = join(env.root, 'a'); + mkdirSync(join(a, 'work'), { recursive: true }); + writeFileSync(join(env.root, 'plain'), 'a file\n'); + const link = join(env.root, 'link'); + symlinkSync(a, link); + const m = await env.open({ path: link }, [{ name: 'work', path: join(a, 'work'), mode: 'read-only', backend: 'direct' }]); + assert.match(await failure(m, 'read', { path: 'ref/work/file.txt' }), /lies in mount "work", nested in mount "ref"/); + rmSync(link); + symlinkSync(join(env.root, 'plain', 'sub'), link); + assert.match(await failure(m, 'read', { path: 'ref/work/file.txt' }), /lies in mount "work", nested in mount "ref"/); +}); + test("a direct mount whose root moves between a read's parse and its open judges the file by the root it opens under", async (t) => { const env = new Env(t); const a = join(env.root, 'a'); From 4d182ebaf3723026ee26c6ec3040bbce6995c90f Mon Sep 17 00:00:00 2001 From: Beth-2709 Date: Mon, 19 Oct 2026 19:26:43 -0400 Subject: [PATCH 12/13] docs(workspace): say what a moved direct root resolves, and the race a read leaves, where callers read them - MountConfig.include names, beside the four cases any nested mount has, the race a direct mount leaves to whoever writes the disk: a root repointed between a read's parse and its open can have the read serve a path in the directory of a mount nested in it by its directories alone. - movedNesting's doc says it resolves the directories on the mount's own configured path, and those beneath its root, not only the path. - A mount at the moved mount's own path takes its real root from the same resolution as one beneath that path, since resolving the moved root gives the moved root. - A test's title says what it pins: a directory on the way that's the root though it's not the first. Co-Authored-By: Claude Opus 5.5 --- src/modules/workspace/index.ts | 21 +++++++++++---------- src/modules/workspace/types.ts | 9 +++++++-- test/workspace-direct.test.ts | 2 +- 3 files changed, 19 insertions(+), 13 deletions(-) diff --git a/src/modules/workspace/index.ts b/src/modules/workspace/index.ts index 3357f74a8..623e2acb3 100644 --- a/src/modules/workspace/index.ts +++ b/src/modules/workspace/index.ts @@ -2486,15 +2486,16 @@ export class WorkspaceModule implements Module { * mount's path, and the directories above this mount's path as resolving * that path finds them. For any other mount, it's where the last start * found them. Each carries its own include, as nestedOf gives it. It - * resolves nothing but this mount's own path and directories beneath its - * root, so a share elsewhere that doesn't answer holds only its own calls, - * unless such a directory is a link into it, where that mount's files lie. - * What it leaves to the next start: a mount configured elsewhere whose own - * root has moved since, as when a link on its configured path is repointed, - * this mount's own reached through another link among them, that mount's - * spellings through other links, and the other mounts' own placements, so a - * direct root moved into another mount's directory is that mount's too, as - * a nested mount's root moved while running stays placed. + * resolves nothing but the directories on this mount's own configured path + * and those beneath its root, so a share elsewhere that doesn't answer + * holds only its own calls, unless such a directory is a link into it, + * where that mount's files lie. What it leaves to the next start: a mount + * configured elsewhere whose own root has moved since, as when a link on + * its configured path is repointed, this mount's own reached through + * another link among them, that mount's spellings through other links, and + * the other mounts' own placements, so a direct root moved into another + * mount's directory is that mount's too, as a nested mount's root moved + * while running stays placed. */ private async movedNesting(mount: MountState, rootReal: string): Promise { // One start's placements, whatever a start in the meantime sets. @@ -2523,7 +2524,7 @@ export class WorkspaceModule implements Module { // where that path now leads: its real root, and each directory above // it, resolved now, beneath this root where they lie beneath it. const segments = configured?.split('/') ?? []; - real = segments.length === 0 ? rootReal : await realBeneath(join(rootReal, ...segments)); + real = await realBeneath(join(rootReal, ...segments)); const beneath: Array<{ dir: string; real: string }> = []; for (let i = segments.length - 1; i >= 1; i--) { beneath.push({ dir: join(outerPath, ...segments.slice(0, i)), real: await realBeneath(join(rootReal, ...segments.slice(0, i))) }); diff --git a/src/modules/workspace/types.ts b/src/modules/workspace/types.ts index d7e296231..82a28c27b 100644 --- a/src/modules/workspace/types.ts +++ b/src/modules/workspace/types.ts @@ -182,7 +182,8 @@ export interface MountConfig { * link on a path's way is refused, as on any such mount (followSymlinks). * A nested mount's root is placed where disk has it at each start, and its * directory is matched by spelling, so four cases fall outside this, as - * for any nested mount: on a disk that folds names, a mount configured + * for any nested mount, and a fifth on a direct one: on a disk that folds + * names, a mount configured * through a link that spells a directory otherwise than disk does isn't * found nested, and this mount holds its files as its own; on such a disk, * a path through a link that spells a nested mount's directory otherwise @@ -195,7 +196,11 @@ export interface MountConfig { * spelling the link's target gives, so a refusal can name a file disk * spells outside that include, or leave a covered one unnamed; and a * nested mount whose root moves while running, as `acceptRoot` takes, - * stays placed where it was until the next start. + * stays placed where it was until the next start; and where a direct + * mount's root is repointed between a read's parse and its open, a race + * left to whoever writes the disk, a mount nested in it by its + * directories alone, since its real root holds that root, can have the + * read serve a path in its directory. * * Text the host registers for a path (`registerGeneratedTextFile`) is * served there whatever the include says, since it isn't one of the diff --git a/test/workspace-direct.test.ts b/test/workspace-direct.test.ts index 50b4fa95d..e249bd326 100644 --- a/test/workspace-direct.test.ts +++ b/test/workspace-direct.test.ts @@ -1173,7 +1173,7 @@ test("a mount configured beneath a moved direct root, through a directory there assert.deepEqual(await answers(moved), fromStart); }); -test("a mount configured beneath a moved direct root is nested beneath each directory on the way that's the root, the second among them, and no other", async (t) => { +test("a mount configured beneath a moved direct root is nested beneath a directory on the way that's the root though it's not the first, and beneath none that isn't", async (t) => { const env = new Env(t); // `work` is configured at `link/x/y/work`. At `b`, `x` is a directory and // `x/y` leads to `..`, `b` itself, and `work` to `.`: a start with `ref` at From 10089c1982d101a3a7fa7a86d033f95681fe1c31 Mon Sep 17 00:00:00 2001 From: Beth-2709 Date: Mon, 19 Oct 2026 20:26:45 -0400 Subject: [PATCH 13/13] test(workspace): the gone-file test finds the listing's look at its file by name At 4d182ebaf3, the direct mount's test that a file gone between the walk that listed it and its read is neither listed nor searched failed on CI's macOS legs, Node 20 and 24: the listing named f30.txt. The test's listing half removes the file when the listing looks at it, and it found that look by comparing the path looked at with one spelled from the test's temp directory. Since e07eb6b8a4, a direct mount's listing reads beneath the root it resolved, and on macOS the temp directory lies under /var, a link to /private/var. So the two spellings differed, the comparison never matched, and the file stayed. The test now finds the look by the file's name, as its grep half already finds its file by inode. With the temp directory reached through a link, on Linux, the direct mount's test file fails this one test at 4d182ebaf3, and none at 50e8fb7504 or with this change, where the two removals this test catches, a listing that shows a gone file and a grep that skips one, are still caught. Co-Authored-By: Claude Opus 5.5 --- test/workspace-direct.test.ts | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/test/workspace-direct.test.ts b/test/workspace-direct.test.ts index e249bd326..9c5e25f5a 100644 --- a/test/workspace-direct.test.ts +++ b/test/workspace-direct.test.ts @@ -10,7 +10,7 @@ import assert from 'node:assert/strict'; import { appendFileSync, chmodSync, existsSync, mkdirSync, mkdtempSync, realpathSync, renameSync, rmSync, statSync, symlinkSync, truncateSync, unlinkSync, writeFileSync } from 'node:fs'; import { createRequire, syncBuiltinESMExports } from 'node:module'; import { tmpdir } from 'node:os'; -import { dirname, join } from 'node:path'; +import { basename, dirname, join } from 'node:path'; import { setFlagsFromString } from 'node:v8'; import { runInNewContext } from 'node:vm'; import { JsStore } from '@animalabs/chronicle'; @@ -456,10 +456,13 @@ test('a file gone between the walk that listed it and its read is neither listed () => data<{ results: Array<{ file: string }>; skipped?: unknown }>(m, 'grep', { pattern: 'needle' })); assert.deepEqual(grep.results.map((r) => r.file), left.map((name) => `ref/${name}`)); assert.equal(grep.skipped, undefined); - // A listing looks at each file by its path, where this one is removed first. + // A listing looks at each file by its path, where this one is removed + // first. The look is known by the file's name, since the listing reads + // beneath the root it resolved, which on macOS isn't spelled as `env.ref` + // is (/var there is a link to /private/var). env.file('f30.txt', 'needle\n'); const listed = await withFsPromise('lstat', (lstat) => async (...args: any[]) => { - if (args[0] === gone) rmSync(gone, { force: true }); + if (basename(String(args[0])) === 'f30.txt') rmSync(gone, { force: true }); return lstat(...args); }, () => data<{ entries: Array<{ path: string }> }>(m, 'ls', { path: 'ref', recursive: true })); assert.deepEqual(listed.entries.map((e) => e.path), left);