From 7ff3cce450004b8d25d735af31f80da0224664f9 Mon Sep 17 00:00:00 2001 From: Abigail-2676 Date: Sat, 17 Oct 2026 14:24:44 -0400 Subject: [PATCH 01/51] feat(workspace): a read says what the text is, cuts a long line, and stops a page before it would spill What the text is, and a bound on one page, built on resultMaxChars from the commit before. What the text is. read decodes through fileText, new in text.ts: the lines read shows, without a BOM and with a CR before a line feed dropped, where each starts in the stored text as character paging reads it, the CRLF and LF counts, and whether the bytes are valid UTF-8. So a CRLF file's lines come without the CR, a BOM isn't shown, and the page's note says when the file has either, with the counts where CRLF and LF ends are mixed. Bytes that aren't UTF-8 show as U+FFFD, as before, and the note now says so, and that edit refuses the file and write replaces it whole (handleEdit's refusal on this branch). A character page still reads the stored text as it is. A long line. A line over 2,000 characters is cut, without splitting a surrogate pair, with how many characters follow and the offsetChars that reads them, counted in the stored text as character paging reads it, BOM and CRs included. A bound on one page. The framework stores a success's data as JSON and writes what passes the call's resultMaxChars to a file, cut mid-JSON. So a line page stops at the last line whose page fits, and a character page at the last whole character that fits (never inside a surrogate pair), each with a note saying where to continue. Each is costed as JSON writes it (jsonLength) beside the rest of the page, allowing for the page's largest numbers; a line page then adds lines while the exact page still fits. Where even the first line would pass, it is cut shorter, at the most that fits, and a character page always moves on by one character, so a limit too small for a page's own fields still gets the reader somewhere. A call without resultMaxChars, one whose caller takes the result in process, uses the framework's default, 24,000. The page logic is src/modules/workspace/read-page.ts (linePage, characterPage); handleRead now only finds the text. read's description says all three. Tests, in test/workspace-text.test.ts: fileText's lines, starts and counts, for CRLF, LF, mixed ends, a lone CR, a BOM, an empty text and invalid UTF-8. In test/workspace-read-text.test.ts: - A CRLF file reads as its lines without the CR, with its note; mixed ends are counted, and a CR that doesn't end a line stays the line's own. - A BOM is noted and not shown, a character page still shows it, a BOM alone is an empty file that has one, and invalid UTF-8 is noted on a line page and a character page. - A long line is cut at 2,000 characters before an emoji's surrogate pair, and the offsetChars it names, past a BOM and a CRLF, reads exactly the rest. - A line page stops before its result passes 24,000, 5,000 and 1,000 characters; one more line would have passed, and the note's offset reads on. - A first line that alone would pass 1,000 characters is cut where it fits. - A character page of escape-heavy text stops within 24,000 and 1,000, its pages put back the whole text, and a stop never splits a surrogate pair, at four neighbouring limits. - A limit of 50, too small for a page's own fields, still moves a character page on by one character, and shows a first line as its marker alone. - The 2000-line default holds for a call whose limit lets it, a script's. Co-Authored-By: Claude Opus 5.5 --- changelog.d/feat-workspace-read.changed.md | 3 + src/modules/workspace/index.ts | 113 +++++---------- src/modules/workspace/read-page.ts | 157 +++++++++++++++++++++ src/modules/workspace/text.ts | 50 ++++++- test/workspace-read-text.test.ts | 140 +++++++++++++++++- test/workspace-text.test.ts | 39 ++++- 6 files changed, 411 insertions(+), 91 deletions(-) create mode 100644 changelog.d/feat-workspace-read.changed.md create mode 100644 src/modules/workspace/read-page.ts diff --git a/changelog.d/feat-workspace-read.changed.md b/changelog.d/feat-workspace-read.changed.md new file mode 100644 index 000000000..865655764 --- /dev/null +++ b/changelog.d/feat-workspace-read.changed.md @@ -0,0 +1,3 @@ +- The workspace's `read` shows a file's text as it is, and says so. A CRLF file's lines come without the CR, and a BOM isn't shown. The page's note says when the file has either, counting a mix of CRLF and LF ends. It also says when the file isn't valid UTF-8: each invalid byte sequence shows as U+FFFD, `edit` refuses the file, and `write` replaces it whole. Before, each line of a CRLF file ended in a CR, a BOM came first in the first line as U+FEFF, and invalid bytes showed as U+FFFD with no word. A character page still reads the stored text as it is, BOM and CRs included. +- A line over 2,000 characters is cut, with how many characters follow and the `offsetChars` that reads them. Before, a line page showed every line whole, however long. +- A `read` page stops before its result would pass `resultMaxChars`, the most of it the framework stores whole. For a model's call that's its agent's inline cap, 24,000 characters by default. Its note says where to continue: the `offset` after its last line, or for a character page the `offsetChars` it also gives as `nextOffsetChars`. Where even a page's first line would pass the limit, that line is cut shorter. Before, a page of up to 2,000 lines, or of `limitChars` characters, went out whole, and the framework wrote what passed the limit to a file, cut mid-JSON. A caller that pages 2,000 lines at a time and starts the next page at line 2,001 now starts where the note says. A running `code_execution` script's call has a limit of 5,000,000 characters, so its pages still stop at 2,000 lines or `limitChars`, unless a page would pass that. diff --git a/src/modules/workspace/index.ts b/src/modules/workspace/index.ts index bcd81c954..e00f096f4 100644 --- a/src/modules/workspace/index.ts +++ b/src/modules/workspace/index.ts @@ -45,6 +45,8 @@ import { TOOL_BATCH_HOOK_TIMEOUT_MS, ensureStateRegistered } from '../../module- import { DEFAULT_TOOL_RESULT_INLINE_MAX_CHARS } from '../../tool-result-history.js'; import { type EditSpec, type FilePlan, decodeForEdit, hintBudget, planFile } from './edit-plan.js'; import { type EditProblem, type EditTool, cutLine, editList, editReceipt, fitted, refusalText } from './edit-text.js'; +import { type FileText, fileText } from './text.js'; +import { DEFAULT_READ_LINE_LIMIT, characterPage, linePage } from './read-page.js'; import { type MountRuntime, type PassOptions, @@ -1290,9 +1292,13 @@ export class WorkspaceModule implements Module { { name: 'read', description: 'Read a file from the workspace. Returns content with line numbers. ' - + 'Reads at most 2000 lines per call by default — use offset/limit to page through larger files. ' + + 'Reads at most 2000 lines per call by default — use offset/limit to page through larger files — ' + + 'and stops sooner where a page would pass the result limit; the note says where to continue. ' + + 'A line over 2000 characters is cut, with the offsetChars that reads the rest. ' + + 'CRLF line ends are shown without the CR, and a BOM isn\'t shown; the note says when the file has either, ' + + 'or isn\'t valid UTF-8. ' + 'For long lines or spill files, use offsetChars/limitChars instead: returns raw text without ' - + 'line numbers, with nextOffsetChars (null at EOF). Keep limitChars when continuing. ' + + 'line numbers, with nextOffsetChars (null at EOF), stopped at the result limit too. Keep limitChars when continuing. ' + 'Character offsets count UTF-16 code units; do not mix character and line parameters.', inputSchema: { type: 'object' as const, @@ -1592,16 +1598,19 @@ export class WorkspaceModule implements Module { return { success: false, isError: true, error: `The ${call.name} call did nothing: ${problem}.` }; } const input = call.input as Record; + // The most of this call's result the framework stores before it spills + // the rest into a file; a call it didn't dispatch gets the default. + const resultMaxChars = call.resultMaxChars ?? DEFAULT_TOOL_RESULT_INLINE_MAX_CHARS; const generated = typeof input?.path === 'string' ? this.findGeneratedTextFile(input.path) : undefined; if (generated) { if (call.name === 'read' && call.callerAgentName === generated.agentName) { - return await this.handleRead(input as unknown as ReadInput, generated.read()); + return await this.handleRead(input as unknown as ReadInput, resultMaxChars, generated.read()); } return { success: false, isError: true, error: this.generatedFileRefusal(generated, call.callerAgentName) }; } switch (call.name) { - case 'read': return await this.handleRead(input as unknown as ReadInput); + case 'read': return await this.handleRead(input as unknown as ReadInput, resultMaxChars); case 'read_image': return await this.handleReadImage(input as unknown as ReadImageInput); case 'write': return await this.handleWrite(input as unknown as WriteInput); case 'edit': return await this.handleEdit(input as unknown as EditInput, call.resultMaxChars); @@ -2651,100 +2660,42 @@ export class WorkspaceModule implements Module { // Tool Handlers // ========================================================================== - private async handleRead(input: ReadInput, generatedContent?: string): Promise { + /** + * A read's page, held to `maxChars`, the most the framework stores of this + * call's result before it spills the rest into a file. + */ + private async handleRead(input: ReadInput, maxChars: number, generatedContent?: string): Promise { const characterPaging = input.offsetChars !== undefined || input.limitChars !== undefined; - const offsetChars = input.offsetChars ?? 0; - const limitChars = input.limitChars ?? 2000; // Each number is a whole number of at least its schema's minimum by now // (inputProblem). if (characterPaging && (input.offset !== undefined || input.limit !== undefined)) { return { success: false, isError: true, error: 'Use either offset/limit (lines) or offsetChars/limitChars (characters), not both.' }; } - let content = generatedContent; - if (content === undefined) { + let file: FileText; + if (generatedContent === undefined) { const { mount, relativePath, path } = await this.parseFilePath(input.path); input = { ...input, path }; const store = this.getStore(); - const found = await this.withMount(mount, async (): Promise<{ text: string } | { error: string }> => { + const found = await this.withMount(mount, async (): Promise<{ bytes: Buffer } | { error: string }> => { const report = await this.ensureSynced(mount, relativePath); const entry = store.treeGet(mount.treeStateId, relativePath); if (!entry) return { error: this.notInWorkspace(input.path, report) }; const blob = store.getBlob(entry.blobHash); if (!blob) return { error: `Blob not found for: ${input.path}` }; - return { text: blob.toString('utf-8') }; + return { bytes: blob }; }); if ('error' in found) return { success: false, error: found.error, isError: true }; - content = found.text; - } - if (characterPaging) { - const start = Math.min(offsetChars, content.length); - const splitsPair = (at: number): boolean => - at > 0 && at < content.length - && content.charCodeAt(at - 1) >= 0xd800 && content.charCodeAt(at - 1) <= 0xdbff - && content.charCodeAt(at) >= 0xdc00 && content.charCodeAt(at) <= 0xdfff; - if (splitsPair(start)) { - return { success: false, isError: true, error: 'offsetChars splits a surrogate pair; use nextOffsetChars from the previous page.' }; - } - let end = start + Math.min(limitChars, content.length - start); - if (splitsPair(end)) end++; - return { - success: true, - data: { - path: input.path, - totalChars: content.length, - offsetChars: start, - nextOffsetChars: end < content.length ? end : null, - content: content.slice(start, end), - }, - }; - } - const lines = linesOf(content); - - // Apply offset/limit. An unlimited read of a large file would inject the - // whole thing into the turn (a 2MB file ≈ 600k tokens → the request blows - // past the model context and 400s), so an omitted limit falls back to a - // default cap; the result reports total/from/to so the agent can page. - const DEFAULT_READ_LINE_LIMIT = 2000; - const startLine = (input.offset ?? 1) - 1; // Convert to 0-indexed - if (startLine >= lines.length) { - // Nothing to show: an empty file, or an offset past the last line. No - // range is claimed, and the line count says where the file ends. - return { - success: true, - data: { - path: input.path, - totalLines: lines.length, - // Reading an empty file from its start is reading it whole. - ...(startLine > 0 - ? { note: `Line ${startLine + 1} is past the end: the file has ${lines.length} line${lines.length === 1 ? '' : 's'}.` } - : {}), - content: '', - }, - }; + file = fileText(found.bytes); + } else { + file = fileText(Buffer.from(generatedContent, 'utf-8')); } - const effectiveLimit = input.limit ?? DEFAULT_READ_LINE_LIMIT; - const endLine = startLine + effectiveLimit; - const slice = lines.slice(startLine, endLine); - const truncated = endLine < lines.length; - - // Format with line numbers (cat -n style) - const formatted = slice - .map((line, i) => `${String(startLine + i + 1).padStart(6)}\t${line}`) - .join('\n'); - - return { - success: true, - data: { - path: input.path, - totalLines: lines.length, - fromLine: startLine + 1, - toLine: Math.min(endLine, lines.length), - ...(truncated - ? { note: `Truncated at ${effectiveLimit} lines (file has ${lines.length}). Use offset/limit to read more.` } - : {}), - content: formatted, - }, - }; + // An unlimited read of a large file would inject the whole thing into the + // turn (a 2MB file ≈ 600k tokens → the request blows past the model + // context and 400s), so an omitted limit falls back to a default, and + // every page stops at the result limit. + return characterPaging + ? characterPage(input.path, file, input.offsetChars ?? 0, input.limitChars ?? 2000, maxChars) + : linePage(input.path, file, input.offset ?? 1, input.limit ?? DEFAULT_READ_LINE_LIMIT, maxChars); } private async handleReadImage(input: ReadImageInput): Promise { diff --git a/src/modules/workspace/read-page.ts b/src/modules/workspace/read-page.ts new file mode 100644 index 000000000..6fc5039ff --- /dev/null +++ b/src/modules/workspace/read-page.ts @@ -0,0 +1,157 @@ +import type { ToolResult } from '../../types/events.js'; +import { type FileText, MAX_LINE_CHARS, jsonCharLength, jsonLength, numberedLine } from './text.js'; + +/** + * One page of a read, held to the result limit: the framework stores a + * success's data as JSON, and spills what passes the caller's limit into a + * file, cut mid-JSON. So a page stops before it would pass the limit, and its + * note says where to continue. + */ + +/** How many lines a read shows when it isn't given a limit. */ +export const DEFAULT_READ_LINE_LIMIT = 2000; + +/** What a read says about the text it shows, besides the lines themselves. */ +export function textNotes(file: FileText): string[] { + const notes: string[] = []; + if (file.crlfEnds > 0) { + notes.push(file.lfEnds > 0 + ? `Line ends are mixed, ${file.crlfEnds} CRLF and ${file.lfEnds} LF, each line shown without a CR. An edit matches text as read shows it, and keeps each line's end.` + : 'Line ends are CRLF, shown without the CR. An edit matches text as read shows it, and keeps the CRLF ends.'); + } + if (file.bom) notes.push('The file starts with a UTF-8 BOM, which isn\'t shown. An edit keeps it.'); + if (!file.utf8) notes.push(NOT_UTF8); + return notes; +} + +const NOT_UTF8 = 'The file isn\'t valid UTF-8: each invalid byte sequence shows as U+FFFD. Edit refuses this file, and write replaces it whole.'; + +const joined = (notes: Array): string | undefined => { + const kept = notes.filter((n): n is string => n !== null); + return kept.length > 0 ? kept.join(' ') : undefined; +}; + +/** + * Lines from `offset` (1-based), at most `limit` of them, each numbered and a + * long one cut with the offsetChars that reads the rest. The page stops where + * one more line would take its data past `maxChars` as JSON; if even the + * first line doesn't fit, that line is cut shorter. + */ +export function linePage(path: string, file: FileText, offset: number, limit: number, maxChars: number, extraNotes: string[] = []): ToolResult { + const { lines, starts } = file; + const total = lines.length; + const first = offset - 1; + const notes = [...textNotes(file), ...extraNotes]; + if (first >= total) { + // Nothing to show: an empty file, or an offset past the last line. No + // range is claimed, and the line count says where the file ends. + const note = joined([ + // Reading an empty file from its start is reading it whole. + first > 0 ? `Line ${first + 1} is past the end: the file has ${total} line${total === 1 ? '' : 's'}.` : null, + ...notes, + ]); + return { success: true, data: { path, totalLines: total, ...(note !== undefined ? { note } : {}), content: '' } }; + } + const last = Math.min(first + limit, total); + const shown = (i: number, maxLineChars = MAX_LINE_CHARS) => + numberedLine(lines[i], i + 1, maxLineChars, (cut) => `read them with offsetChars=${starts[i] + cut}`); + const limitNote = `Truncated at ${limit} lines (file has ${total}). Use offset/limit to read more.`; + const budgetNote = (to: number) => + `Stopped at line ${to} to stay within the ${maxChars}-character result limit (file has ${total}). Continue with offset=${to + 1}.`; + const page = (to: number, content: string, note: string | undefined) => ({ + path, totalLines: total, fromLine: first + 1, toLine: to, ...(note !== undefined ? { note } : {}), content, + }); + // The most anything but the lines can take: the longer note, with the + // largest numbers this page could show. + const overhead = Math.max( + JSON.stringify(page(last, '', joined([limitNote, ...notes]))).length, + JSON.stringify(page(last, '', joined([budgetNote(last), ...notes]))).length, + ); + const shownLines: string[] = []; + let used = overhead; + for (let i = first; i < last; i++) { + const line = shown(i); + // Each line after the first comes after a line feed, which JSON writes as two characters. + const cost = jsonLength(line) + (i > first ? 2 : 0); + if (used + cost > maxChars) break; + shownLines.push(line); + used += cost; + } + const final = () => { + const to = first + shownLines.length; + return page(to, shownLines.join('\n'), joined([to === last ? (to < total ? limitNote : null) : budgetNote(to), ...notes])); + }; + if (shownLines.length === 0) { + // Not even the first line fits: cut it where it does, at the most it can + // show. A longer cut never shows less, so the first that doesn't fit + // bounds the search. + let fits = 0; + let over = Math.min(lines[first].length, MAX_LINE_CHARS) + 1; + while (over - fits > 1) { + const mid = Math.floor((fits + over) / 2); + if (overhead + jsonLength(shown(first, mid)) <= maxChars) fits = mid; else over = mid; + } + shownLines.push(shown(first, fits)); + return { success: true, data: final() }; + } + // The overhead allowed for the largest numbers; a line more may fit beside + // the real ones. + while (first + shownLines.length < last) { + shownLines.push(shown(first + shownLines.length)); + if (JSON.stringify(final()).length > maxChars) { + shownLines.pop(); + break; + } + } + return { success: true, data: final() }; +} + +const splitsPair = (text: string, at: number): boolean => + at > 0 && at < text.length + && text.charCodeAt(at - 1) >= 0xd800 && text.charCodeAt(at - 1) <= 0xdbff + && text.charCodeAt(at) >= 0xdc00 && text.charCodeAt(at) <= 0xdfff; + +/** + * Raw text from `offsetChars`, at most `limitChars` code units (one more to + * keep a surrogate pair whole), stopped sooner where its data would pass + * `maxChars` as JSON. + */ +export function characterPage(path: string, file: FileText, offsetChars: number, limitChars: number, maxChars: number): ToolResult { + const text = file.raw; + const start = Math.min(offsetChars, text.length); + if (splitsPair(text, start)) { + return { success: false, isError: true, error: 'offsetChars splits a surrogate pair; use nextOffsetChars from the previous page.' }; + } + let end = start + Math.min(limitChars, text.length - start); + if (splitsPair(text, end)) end++; + const notes = file.utf8 ? [] : [NOT_UTF8]; + const stoppedNote = (to: number) => + `Stopped at character ${to} to stay within the ${maxChars}-character result limit; continue with offsetChars=${to}.`; + const page = (to: number, note: string | undefined) => ({ + path, + totalChars: text.length, + offsetChars: start, + nextOffsetChars: to < text.length ? to : null, + ...(note !== undefined ? { note } : {}), + content: text.slice(start, to), + }); + const whole = page(end, joined(notes)); + if (JSON.stringify(whole).length <= maxChars) return { success: true, data: whole }; + // As many characters as fit beside the rest of the page, which takes the + // most with the largest offset it could stop at. + const overhead = JSON.stringify({ ...page(end, joined([stoppedNote(end), ...notes])), nextOffsetChars: end, content: '' }).length; + // A page moves by whole characters, a surrogate pair at once, so it never + // stops inside one. + const step = (at: number) => (splitsPair(text, at + 1) ? 2 : 1); + let to = start; + for (let used = overhead; to < end;) { + const cost = jsonCharLength(text, to) + (step(to) === 2 ? jsonCharLength(text, to + 1) : 0); + if (used + cost > maxChars) break; + used += cost; + to += step(to); + } + // A page always moves on, by one character at least, though a limit too + // small for the page's own fields then passes it. + if (to === start) to = Math.min(end, start + step(start)); + return { success: true, data: page(to, joined([stoppedNote(to), ...notes])) }; +} diff --git a/src/modules/workspace/text.ts b/src/modules/workspace/text.ts index cd7448ece..4fb9e17fc 100644 --- a/src/modules/workspace/text.ts +++ b/src/modules/workspace/text.ts @@ -1,6 +1,9 @@ +import { isUtf8 } from 'node:buffer'; + /** - * Lines numbered the way `cat -n` numbers them, with a long one cut, and a - * string's length as JSON, which a budget for a stored result counts. edit's + * A stored file's text as read shows it and grep searches it, lines numbered + * the way `cat -n` numbers them, with a long one cut, and a string's length + * as JSON, which a budget for a stored result counts. Read, grep and edit's * answers use them. */ @@ -9,6 +12,49 @@ export function hasBom(bytes: Uint8Array): boolean { return bytes.length >= 3 && bytes[0] === 0xef && bytes[1] === 0xbb && bytes[2] === 0xbf; } +export interface FileText { + /** The bytes decoded as UTF-8, as character paging reads them: a BOM and every CR kept. */ + raw: string; + /** + * Its lines, as read shows them and grep searches them: without the BOM, a + * CR before a line feed dropped, and a final line end ending the last line + * rather than starting an empty one. So "a\r\nb\r\n" is two lines, and an + * empty text has none. + */ + lines: string[]; + /** Where each line starts in `raw`. */ + starts: number[]; + bom: boolean; + /** How many line ends were CRLF, and how many a bare LF. */ + crlfEnds: number; + lfEnds: number; + /** Whether the bytes are valid UTF-8; where they aren't, `raw` holds U+FFFD. */ + utf8: boolean; +} + +export function fileText(bytes: Buffer): FileText { + const raw = bytes.toString('utf-8'); + const bom = hasBom(bytes); + const lines: string[] = []; + const starts: number[] = []; + let crlfEnds = 0; + let lfEnds = 0; + // A BOM decodes to the single code unit U+FEFF. + for (let at = bom ? 1 : 0; at < raw.length;) { + const lf = raw.indexOf('\n', at); + starts.push(at); + if (lf === -1) { + lines.push(raw.slice(at)); + break; + } + const crlf = lf > at && raw.charCodeAt(lf - 1) === 13; + if (crlf) crlfEnds++; else lfEnds++; + lines.push(raw.slice(at, crlf ? lf - 1 : lf)); + at = lf + 1; + } + return { raw, lines, starts, bom, crlfEnds, lfEnds, utf8: isUtf8(bytes) }; +} + /** How long a line read shows before it cuts the line. */ export const MAX_LINE_CHARS = 2000; diff --git a/test/workspace-read-text.test.ts b/test/workspace-read-text.test.ts index c3869be5b..f0ec23fb8 100644 --- a/test/workspace-read-text.test.ts +++ b/test/workspace-read-text.test.ts @@ -6,7 +6,7 @@ import { join } from 'node:path'; import { JsStore } from '@animalabs/chronicle'; import { WorkspaceModule } from '../src/modules/workspace/index.js'; -async function setup(t: TestContext, content: string) { +async function setup(t: TestContext, content: string | Buffer) { const root = mkdtempSync(join(tmpdir(), 'workspace-read-text-')); const store = JsStore.openOrCreate({ path: join(root, 'store') }); const workspace = new WorkspaceModule({ @@ -15,12 +15,18 @@ async function setup(t: TestContext, content: string) { workspace.initStore(store); t.after(() => { store.close(); rmSync(root, { recursive: true, force: true }); }); assert.equal((await workspace.writeBinary('work/text.txt', Buffer.from(content), 'text/plain')).success, true); - const read = (input: Record = {}) => workspace.handleToolCall({ - id: 'read', name: 'read', input: { path: 'work/text.txt', ...input }, + // A call the framework dispatched carries the most of its result it stores + // (resultMaxChars); one without it gets the framework's default, 24,000. + const read = (input: Record = {}, resultMaxChars?: number) => workspace.handleToolCall({ + id: 'read', name: 'read', input: { path: 'work/text.txt', ...input }, ...(resultMaxChars !== undefined ? { resultMaxChars } : {}), }); return { workspace, read }; } +type LinePage = { totalLines: number; fromLine?: number; toLine?: number; note?: string; content: string }; +type CharPage = { totalChars: number; offsetChars: number; nextOffsetChars: number | null; note?: string; content: string }; +const stored = (result: { data?: unknown }): number => JSON.stringify(result.data).length; + test('line reads retain numbering, ranges, and the 2000-line default', async t => { const { read } = await setup(t, Array.from({ length: 2005 }, (_, i) => `line ${i + 1}`).join('\n')); const result = await read({ offset: 2, limit: 2 }); @@ -29,11 +35,137 @@ test('line reads retain numbering, ranges, and the 2000-line default', async t = note: 'Truncated at 2 lines (file has 2005). Use offset/limit to read more.', content: ' 2\tline 2\n 3\tline 3', }); - const defaults = (await read()).data as { toLine: number; content: string }; + // A script's call can take far more than a turn's, so its default page is the 2000 lines. + const defaults = (await read({}, 5_000_000)).data as LinePage; assert.equal(defaults.toLine, 2000); assert.equal(defaults.content.split('\n').length, 2000); }); +test('a CRLF file reads as its lines without the CR, and says so', async t => { + const { read } = await setup(t, 'one\r\ntwo\r\n'); + assert.deepEqual((await read()).data, { + path: 'work/text.txt', totalLines: 2, fromLine: 1, toLine: 2, + note: 'Line ends are CRLF, shown without the CR. An edit matches text as read shows it, and keeps the CRLF ends.', + content: ' 1\tone\n 2\ttwo', + }); + const { read: readMixed } = await setup(t, 'a\r\nb\nc\r\nd'); + const page = (await readMixed()).data as LinePage; + assert.equal(page.content, ' 1\ta\n 2\tb\n 3\tc\n 4\td'); + assert.equal(page.note, 'Line ends are mixed, 2 CRLF and 1 LF, each line shown without a CR. An edit matches text as read shows it, and keeps each line\'s end.'); + // A CR that doesn't end a line is the line's own. + const { read: readCr } = await setup(t, 'a\rb\n'); + assert.deepEqual((await readCr()).data, { path: 'work/text.txt', totalLines: 1, fromLine: 1, toLine: 1, content: ' 1\ta\rb' }); +}); + +test('a BOM is noted and not shown, and bytes that aren\'t UTF-8 are noted', async t => { + const { read } = await setup(t, Buffer.concat([Buffer.from([0xef, 0xbb, 0xbf]), Buffer.from('first\nsecond\n')])); + assert.deepEqual((await read()).data, { + path: 'work/text.txt', totalLines: 2, fromLine: 1, toLine: 2, + note: 'The file starts with a UTF-8 BOM, which isn\'t shown. An edit keeps it.', + content: ' 1\tfirst\n 2\tsecond', + }); + // Character paging reads the stored text as it is, the BOM included. + assert.equal(((await read({ offsetChars: 0, limitChars: 3 })).data as CharPage).content, '\ufefffi'); + // A BOM alone is an empty file that has one. + const { read: readBomOnly } = await setup(t, Buffer.from([0xef, 0xbb, 0xbf])); + assert.deepEqual((await readBomOnly()).data, { + path: 'work/text.txt', totalLines: 0, note: 'The file starts with a UTF-8 BOM, which isn\'t shown. An edit keeps it.', content: '', + }); + const notUtf8 = 'The file isn\'t valid UTF-8: each invalid byte sequence shows as U+FFFD. Edit refuses this file, and write replaces it whole.'; + const { read: readLatin1 } = await setup(t, Buffer.from('caf\xe9\n', 'latin1')); + assert.deepEqual((await readLatin1()).data, { + path: 'work/text.txt', totalLines: 1, fromLine: 1, toLine: 1, note: notUtf8, content: ' 1\tcaf\ufffd', + }); + assert.equal(((await readLatin1({ offsetChars: 0 })).data as CharPage).note, notUtf8); +}); + +test('a long line is cut at 2000 characters, with the offsetChars that reads the rest', async t => { + // A BOM and a CRLF before the long line, so its offset counts both. + const long = 'x'.repeat(1999) + '😀' + 'y'.repeat(1000); + const { read } = await setup(t, Buffer.concat([Buffer.from([0xef, 0xbb, 0xbf]), Buffer.from(`short\r\n${long}\r\nlast\r\n`)])); + const page = (await read()).data as LinePage; + const lines = page.content.split('\n'); + assert.equal(lines.length, 3); + // The cut doesn't split the emoji's surrogate pair: it falls before it. + const rest = long.length - 1999; + const at = 1 + 'short\r\n'.length + 1999; + assert.equal(lines[1], ` 2\t${'x'.repeat(1999)} [… ${rest} more chars; read them with offsetChars=${at}]`); + const more = (await read({ offsetChars: at, limitChars: rest })).data as CharPage; + assert.equal(more.content, '😀' + 'y'.repeat(1000)); + assert.equal(lines[2], ' 3\tlast'); +}); + +test('a line page stops before its result would pass the limit, and says where to continue', async t => { + const text = Array.from({ length: 3000 }, (_, i) => `line ${i + 1}: ${'"'.repeat(i % 7)}`).join('\n'); + const { read } = await setup(t, text); + for (const limit of [24_000, 5_000, 1_000]) { + const result = await read({}, limit); + assert.ok(stored(result) <= limit, `${stored(result)} > ${limit}`); + const page = result.data as LinePage; + assert.ok(page.toLine! < 2000); + assert.equal(page.note, `Stopped at line ${page.toLine} to stay within the ${limit}-character result limit (file has 3000). Continue with offset=${page.toLine! + 1}.`); + // One more line would have passed the limit. + const more = await read({ limit: page.toLine! + 1 }, 5_000_000); + const withNext = { ...(result.data as object), toLine: page.toLine! + 1, content: (more.data as LinePage).content }; + assert.ok(JSON.stringify(withNext).length > limit); + // Continuing reads on from the next line. + const next = (await read({ offset: page.toLine! + 1 }, limit)).data as LinePage; + assert.equal(next.fromLine, page.toLine! + 1); + assert.ok(next.content.startsWith(`${String(page.toLine! + 1).padStart(6)}\tline ${page.toLine! + 1}:`)); + } +}); + +test('a first line that alone would pass the limit is cut where it fits', async t => { + // Each quote takes two characters as JSON. + const { read } = await setup(t, '"'.repeat(1500) + '\nnext\n'); + const result = await read({}, 1_000); + assert.ok(stored(result) <= 1_000, `${stored(result)}`); + const page = result.data as LinePage; + assert.equal(page.toLine, 1); + const [, shown, rest, at] = /^ {5}1\t("*) \[… (\d+) more chars; read them with offsetChars=(\d+)\]$/.exec(page.content)!; + assert.ok(shown.length > 0); + assert.equal(Number(at), shown.length); + assert.equal(shown.length + Number(rest), 1500); + assert.match(page.note!, /^Stopped at line 1 to stay within the 1000-character result limit \(file has 2\)\. Continue with offset=2\.$/); + // A limit with no room beside the page's own fields still shows where the line's text is. + const tiny = (await read({}, 50)).data as LinePage; + assert.equal(tiny.toLine, 1); + assert.equal(tiny.content, ' 1\t [… 1500 more chars; read them with offsetChars=0]'); +}); + +test('a character page stops at the result limit too, with nextOffsetChars', async t => { + // Control characters take six characters each as JSON. + const text = 'abc\u0001'.repeat(20_000); + const { read } = await setup(t, text); + for (const limit of [24_000, 1_000]) { + let offsetChars = 0; + let recovered = ''; + for (let pages = 0; ; pages++) { + assert.ok(pages < 1_000, 'every page moves on'); + const result = await read({ offsetChars, limitChars: 100_000 }, limit); + assert.ok(stored(result) <= limit, `${stored(result)} > ${limit}`); + const page = result.data as CharPage; + recovered += page.content; + if (page.nextOffsetChars === null) break; + assert.equal(page.note, `Stopped at character ${page.nextOffsetChars} to stay within the ${limit}-character result limit; continue with offsetChars=${page.nextOffsetChars}.`); + offsetChars = page.nextOffsetChars; + } + assert.equal(recovered, text); + } + // A limit with no room beside the page's own fields still moves the page on. + const tiny = (await read({ offsetChars: 5, limitChars: 100 }, 50)).data as CharPage; + assert.equal(tiny.offsetChars, 5); + assert.equal(tiny.nextOffsetChars, 6); + assert.equal(tiny.content, text[5]); + // A surrogate pair is never split at the stop, whatever the limit leaves room for. + const { read: readEmoji } = await setup(t, '😀'.repeat(20_000)); + for (let limit = 1_000; limit < 1_004; limit++) { + const page = (await readEmoji({ offsetChars: 0, limitChars: 100_000 }, limit)).data as CharPage; + assert.equal(page.nextOffsetChars! % 2, 0, `at ${limit}`); + assert.equal(page.content.length % 2, 0, `at ${limit}`); + } +}); + test('character mode can be selected by either parameter and reports EOF', async t => { const { read } = await setup(t, 'a'.repeat(3000)); assert.deepEqual((await read({ offsetChars: 0 })).data, { diff --git a/test/workspace-text.test.ts b/test/workspace-text.test.ts index 09d596f8b..c6937b95d 100644 --- a/test/workspace-text.test.ts +++ b/test/workspace-text.test.ts @@ -1,9 +1,40 @@ -// The text helpers the workspace's read, grep and edit share: numbered lines -// with a long one cut, a BOM's check, and a string's length as JSON, which a -// page's budget is counted in. +// The text helpers the workspace's read, grep and edit share: a file's lines as +// read shows them, numbered lines with a long one cut, a BOM's check, and a +// string's length as JSON, which a page's budget is counted in. import test from 'node:test'; import assert from 'node:assert/strict'; -import { hasBom, jsonLength, numberedLine, safeCut } from '../src/modules/workspace/text.js'; +import { fileText, hasBom, jsonLength, numberedLine, safeCut } from '../src/modules/workspace/text.js'; + +test('fileText gives the lines read shows, where each starts, and what the text is', () => { + const lines = (text: string | Buffer) => fileText(Buffer.from(text)).lines; + assert.deepEqual(lines(''), []); + assert.deepEqual(lines('a'), ['a']); + assert.deepEqual(lines('a\nb\n'), ['a', 'b']); + assert.deepEqual(lines('a\n\n'), ['a', '']); + assert.deepEqual(lines('\n'), ['']); + assert.deepEqual(lines('a\r\nb\r\n'), ['a', 'b']); + assert.deepEqual(lines('a\rb\r\n\r\n'), ['a\rb', '']); + assert.deepEqual(lines('a\r'), ['a\r'], 'a CR that ends the text ends no line'); + + const crlf = fileText(Buffer.from('one\r\ntwo\nthree')); + assert.deepEqual(crlf.starts, [0, 5, 9]); + assert.equal(crlf.crlfEnds, 1); + assert.equal(crlf.lfEnds, 1); + assert.equal(crlf.bom, false); + assert.equal(crlf.utf8, true); + assert.equal(crlf.raw, 'one\r\ntwo\nthree'); + + const bom = fileText(Buffer.from([0xef, 0xbb, 0xbf, 0x61, 0x0a, 0x62])); + assert.equal(bom.bom, true); + assert.deepEqual(bom.lines, ['a', 'b']); + assert.deepEqual(bom.starts, [1, 3], 'offsets count the BOM, as character paging does'); + assert.equal(bom.raw, '\ufeffa\nb'); + assert.deepEqual(fileText(Buffer.from([0xef, 0xbb, 0xbf])).lines, []); + + const latin1 = fileText(Buffer.from('caf\xe9', 'latin1')); + assert.equal(latin1.utf8, false); + assert.deepEqual(latin1.lines, ['caf\ufffd']); +}); test('numberedLine numbers a line as cat -n does, and cuts a long one without splitting a surrogate pair', () => { assert.equal(numberedLine('text', 7), ' 7\ttext'); From 57aff8691695d28723d23170ac2ce7598d841e3d Mon Sep 17 00:00:00 2001 From: Abigail-2676 Date: Sat, 17 Oct 2026 14:24:57 -0400 Subject: [PATCH 02/51] feat(workspace): grep matches and shows a CRLF file's lines as read shows them grep drops the CR before each line feed with read, for matching as well as display, so foo$ finds a CRLF file's line, as it does under ripgrep's --crlf. grep shared linesOf with read, and with hands' planner coming to edit, a CRLF line copied from grep ("foo\r") would miss: the planner makes every CRLF in an edit's old text LF but keeps a lone CR, as hands' match.ts does. grep now searches fileText's lines: without a CR before a line feed and without a BOM, so `foo$` finds a CRLF file's line and `^foo` a BOM file's first. Each file's result says `lineEndings: 'CRLF'` or `'mixed'` when the file has CRLF ends, and says nothing for LF alone. The disk version kept with a conflict is searched the same way. linesOf, which only grep still used, goes. grep's description says so. The cost, which the changelog names: a pattern written to find a CR at a line's end no longer matches. Test, in test/workspace-file-tools.test.ts: `^foo$` with context finds a CRLF file's line with its context lines all without the CR and lineEndings 'CRLF'; a mixed file says 'mixed', an LF file nothing, a BOM file's first line matches; and `\r` finds nothing in a CRLF file. And a test of the copy into an edit: two lines copied from a read of a CRLF file with a BOM, and then a line copied from a grep match, each match in an edit through the planner this branch is stacked on, and the file keeps its CRLF ends and its BOM. Co-Authored-By: Claude Opus 5.5 --- changelog.d/feat-workspace-read.changed.md | 1 + src/modules/workspace/index.ts | 35 ++++++++---------- test/workspace-file-tools.test.ts | 42 ++++++++++++++++++++++ 3 files changed, 58 insertions(+), 20 deletions(-) diff --git a/changelog.d/feat-workspace-read.changed.md b/changelog.d/feat-workspace-read.changed.md index 865655764..fb6a28952 100644 --- a/changelog.d/feat-workspace-read.changed.md +++ b/changelog.d/feat-workspace-read.changed.md @@ -1,3 +1,4 @@ - The workspace's `read` shows a file's text as it is, and says so. A CRLF file's lines come without the CR, and a BOM isn't shown. The page's note says when the file has either, counting a mix of CRLF and LF ends. It also says when the file isn't valid UTF-8: each invalid byte sequence shows as U+FFFD, `edit` refuses the file, and `write` replaces it whole. Before, each line of a CRLF file ended in a CR, a BOM came first in the first line as U+FEFF, and invalid bytes showed as U+FFFD with no word. A character page still reads the stored text as it is, BOM and CRs included. - A line over 2,000 characters is cut, with how many characters follow and the `offsetChars` that reads them. Before, a line page showed every line whole, however long. - A `read` page stops before its result would pass `resultMaxChars`, the most of it the framework stores whole. For a model's call that's its agent's inline cap, 24,000 characters by default. Its note says where to continue: the `offset` after its last line, or for a character page the `offsetChars` it also gives as `nextOffsetChars`. Where even a page's first line would pass the limit, that line is cut shorter. Before, a page of up to 2,000 lines, or of `limitChars` characters, went out whole, and the framework wrote what passed the limit to a file, cut mid-JSON. A caller that pages 2,000 lines at a time and starts the next page at line 2,001 now starts where the note says. A running `code_execution` script's call has a limit of 5,000,000 characters, so its pages still stop at 2,000 lines or `limitChars`, unless a page would pass that. +- The workspace's `grep` leaves out a CR before a line feed, for matching as well as display, as ripgrep's `--crlf` mode does, so `foo$` finds a CRLF file's line too. A match's text and context are the lines as `read` shows them, a BOM isn't part of the first line, and a file with CRLF ends says so in its result's `lineEndings` (`CRLF`, or `mixed`). Before, a CRLF file's lines ended in a CR, so `foo$` missed them and a match's text copied into an edit carried the CR. A pattern written to find a CR at a line's end no longer matches. diff --git a/src/modules/workspace/index.ts b/src/modules/workspace/index.ts index e00f096f4..15732869c 100644 --- a/src/modules/workspace/index.ts +++ b/src/modules/workspace/index.ts @@ -1443,7 +1443,9 @@ export class WorkspaceModule implements Module { + 'which version matched: the workspace\'s (with its state); the disk version kept with a conflict ' + '(conflicting-disk); or, once disk has changed since the conflict was recorded, that recorded version ' + '(recorded-disk). Disk versions it could not search are listed as skipped: disk-only files (binary or ' - + 'over the size limit), and a conflict\'s disk version that is binary, oversize, or newer than the one recorded.', + + 'over the size limit), and a conflict\'s disk version that is binary, oversize, or newer than the one recorded. ' + + 'In a file with CRLF line ends, each line is matched and shown without its CR, as read shows it, ' + + 'and the file\'s result says lineEndings.', inputSchema: { type: 'object' as const, properties: { @@ -3388,12 +3390,16 @@ export class WorkspaceModule implements Module { : [...this.mounts.values()].map(m => ({ mount: m, relativePath: '' })); type GrepMatch = { line: number; text: string; context?: string[] }; - const results: Array<{ file: string; state: string; version: 'workspace' | 'conflicting-disk' | 'recorded-disk'; matches: GrepMatch[] }> = []; + type Searched = { lineEndings?: 'CRLF' | 'mixed'; matches: GrepMatch[] }; + const results: Array<{ file: string; state: string; version: 'workspace' | 'conflicting-disk' | 'recorded-disk' } & Searched> = []; const skipped: Array<{ file: string; reason: string; size?: number; mimeType?: string }> = []; const incomplete: Array<{ path: string; reason: string }> = []; - const search = (content: string): GrepMatch[] => { - const lines = linesOf(content); + // Lines as read shows them: a CR before a line feed is left out, for + // matching as well as display, as ripgrep's --crlf does, so `foo$` finds + // a CRLF file's line too, and the line copies into an edit as read's do. + const search = (bytes: Buffer): Searched => { + const { lines, crlfEnds, lfEnds } = fileText(bytes); const found: GrepMatch[] = []; for (let i = 0; i < lines.length; i++) { if (regex.test(lines[i])) { @@ -3406,7 +3412,7 @@ export class WorkspaceModule implements Module { found.push(match); } } - return found; + return { ...(crlfEnds > 0 ? { lineEndings: lfEnds > 0 ? 'mixed' : 'CRLF' } : {}), matches: found }; }; for (const { mount, relativePath } of mountsToSearch) { @@ -3474,8 +3480,8 @@ export class WorkspaceModule implements Module { const entry = store.treeGet(mount.treeStateId, report.path); const blob = entry ? store.getBlob(entry.blobHash) : null; if (blob) { - const found = search(blob.toString('utf-8')); - if (found.length > 0) results.push({ file, state: report.state, version: 'workspace', matches: found }); + const found = search(blob); + if (found.matches.length > 0) results.push({ file, state: report.state, version: 'workspace', ...found }); } else if (entry) { skipped.push({ file, reason: 'the workspace copy is missing from the store, so it was not searched' }); } @@ -3489,8 +3495,8 @@ export class WorkspaceModule implements Module { const changed = report.conflict?.diskChangedSinceRecorded === true; const diskBlob = disk?.stored ? store.getBlob(disk.stored) : null; if (diskBlob) { - const found = search(diskBlob.toString('utf-8')); - if (found.length > 0) results.push({ file, state: report.state, version: changed ? 'recorded-disk' : 'conflicting-disk', matches: found }); + const found = search(diskBlob); + if (found.matches.length > 0) results.push({ file, state: report.state, version: changed ? 'recorded-disk' : 'conflicting-disk', ...found }); } if (disk?.stored && !diskBlob) { skipped.push({ file, reason: 'conflict: the disk version recorded with the conflict is missing from the store, so it was not searched' }); @@ -4280,17 +4286,6 @@ function storedButUnfinished( return { success: false, isError: true, error: stored + rest + (note !== null ? ` ${note}` : '') }; } -/** - * A text's lines, as read and grep number them: a final line end ends the - * last line rather than starting an empty one, so "a\nb\n" is two lines, - * and an empty text has none. - */ -function linesOf(content: string): string[] { - const lines = content.split('\n'); - if (lines.at(-1) === '') lines.pop(); - return lines; -} - /** * The least each tool's whole-number fields may be, beside their schemas, * whose type (membrane's ToolParameter) has no `minimum`. diff --git a/test/workspace-file-tools.test.ts b/test/workspace-file-tools.test.ts index d53d525e1..3251125e8 100644 --- a/test/workspace-file-tools.test.ts +++ b/test/workspace-file-tools.test.ts @@ -393,6 +393,48 @@ test('grep numbers lines as read does: no match on an empty line after the last' assert.deepEqual((result.data as { results: unknown[] }).results, []); }); +test('grep matches and shows a CRLF file\'s lines without the CR, as read shows them', async (t) => { + const { dir, call } = setup(t); + writeFileSync(join(dir, 'crlf.txt'), 'alpha\r\nfoo\r\nomega\r\n'); + writeFileSync(join(dir, 'mixed.txt'), 'foo\r\nbar\n'); + writeFileSync(join(dir, 'lf.txt'), 'foo\n'); + writeFileSync(join(dir, 'bom.txt'), Buffer.concat([Buffer.from([0xef, 0xbb, 0xbf]), Buffer.from('foo\n')])); + const result = await call('grep', { pattern: '^foo$', contextBefore: 1, contextAfter: 1 }); + assert.equal(result.success, true, result.error); + type Result = { file: string; lineEndings?: string; matches: unknown[] }; + const byFile = Object.fromEntries((result.data as { results: Result[] }).results.map((r) => [r.file, r])); + assert.deepEqual(byFile['work/crlf.txt'], { + file: 'work/crlf.txt', state: 'synced', version: 'workspace', lineEndings: 'CRLF', + matches: [{ line: 2, text: 'foo', context: ['alpha', 'foo', 'omega'] }], + }); + assert.equal(byFile['work/mixed.txt'].lineEndings, 'mixed'); + assert.equal('lineEndings' in byFile['work/lf.txt'], false); + // A BOM is no part of the first line, as read doesn't show it. + assert.deepEqual(byFile['work/bom.txt'].matches, [{ line: 1, text: 'foo', context: ['foo'] }]); + // The cost: a pattern written to find the CR at a line's end no longer does. + const cr = await call('grep', { pattern: '\\r', path: 'work/crlf.txt' }); + assert.deepEqual((cr.data as { results: unknown[] }).results, []); +}); + +test('text copied from a read or a grep of a CRLF file matches in an edit, and the file keeps its line ends and BOM', async (t) => { + const { dir, call, disk } = setup(t, { autoMaterialize: true }); + const bom = Buffer.from([0xef, 0xbb, 0xbf]); + writeFileSync(join(dir, 'w.txt'), Buffer.concat([bom, Buffer.from('one\r\ntwo\r\nthree\r\n')])); + // Two lines as the read shows them, each without its number and tab. + const page = (await call('read', { path: 'work/w.txt' })).data as { content: string }; + const copied = page.content.split('\n').slice(0, 2).map((line) => line.slice(line.indexOf('\t') + 1)).join('\n'); + assert.equal(copied, 'one\ntwo'); + const edit = await call('edit', { path: 'work/w.txt', oldString: copied, newString: 'ONE\nTWO' }); + assert.equal(edit.success, true, edit.error); + assert.deepEqual(disk('w.txt'), Buffer.concat([bom, Buffer.from('ONE\r\nTWO\r\nthree\r\n')])); + // A grep match's text copies the same way. + const found = await call('grep', { pattern: '^three$', path: 'work/w.txt' }); + const text = (found.data as { results: Array<{ matches: Array<{ text: string }> }> }).results[0].matches[0].text; + const second = await call('edit', { path: 'work/w.txt', oldString: text, newString: 'THREE' }); + assert.equal(second.success, true, second.error); + assert.deepEqual(disk('w.txt'), Buffer.concat([bom, Buffer.from('ONE\r\nTWO\r\nTHREE\r\n')])); +}); + test('an offset or limit that is not a whole number of at least 1 is refused', async (t) => { const { dir, call } = setup(t); writeFileSync(join(dir, 'nl.txt'), 'a\nb\n'); From d62d07a3a034fe57eb676ad65d440f6c31e6b005 Mon Sep 17 00:00:00 2001 From: Abigail-2676 Date: Sat, 17 Oct 2026 14:30:06 -0400 Subject: [PATCH 03/51] feat(workspace): a read settles its path against disk first, as a listing and an edit do The read half of the disk check, last here since it changes most what a read does with disk. A read of a path the workspace held didn't look at disk (ensureSynced returned early), so it could show a copy another writer had changed since. Now handleRead passes over its path first, under the three-way rule a listing uses, as handleEdit does on this branch: - disk changed and the store didn't: the change is taken in, and the note says this read shows that text; - disk's copy couldn't be checked: the note says this read shows the workspace's copy; - disk has no copy and nothing records whether it was deleted there (disk-missing-provenance-unknown): the note says so, and that this read shows the workspace's copy, as handleEdit now says for an edit; - disk deleted the file: "File not found", saying disk deleted it and that was taken in (a copy disk made binary or too large keeps the disk-only answer); - the stored copy is a draft and disk changed too: the read shows the draft, and the note names the conflict and both ways to settle it, as the edit's answer does (a refusal would hide the text the agent needs to settle it). A file the workspace didn't hold is taken in with no word, as before. The note rides on a line page and a character page alike (characterPage takes extra notes now). readBinary, the framework's read_image route, settles the same way, so its doc comment's "as the read tool" stays true. A deletion still observes only a path the workspace doesn't hold (ensureSynced, its doc now says why): taking disk's change in first would let its push remove that change unseen, as for a write. Three tests in test/workspace-three-way.test.ts read the store through read, to see what a pass had or hadn't taken in: a sync asked on one branch takes nothing on another, a restored timestamp hides no change, and a tracked link the walk cap passed over isn't read through. A read now settles its own path, so they read the store directly (storedOf, beside contentOf, whose doc now says a read settles first). Before, "the read serves the store" was the first assertion of the second; that is the behaviour this commit changes. Tests, in test/workspace-file-tools.test.ts (6): disk's newer text is shown with the note, the next read has none, and a character page says it too; a file disk deleted; a draft whose disk copy changed shows the draft with the conflict note and leaves disk alone; a directory in the file's place reads the workspace's copy with the unverified note; an entry disk lacks with no record reads with the missing-copy note; readBinary returns disk's new bytes. Red-first: with this commit's source changes undone, all five fail and the other 29 in the file pass. Co-Authored-By: Claude Opus 5.5 --- changelog.d/feat-workspace-read.changed.md | 1 + src/modules/workspace/index.ts | 75 +++++++++++++++----- src/modules/workspace/read-page.ts | 4 +- test/workspace-file-tools.test.ts | 79 ++++++++++++++++++++++ test/workspace-three-way.test.ts | 26 ++++--- 5 files changed, 159 insertions(+), 26 deletions(-) diff --git a/changelog.d/feat-workspace-read.changed.md b/changelog.d/feat-workspace-read.changed.md index fb6a28952..2c1d02921 100644 --- a/changelog.d/feat-workspace-read.changed.md +++ b/changelog.d/feat-workspace-read.changed.md @@ -2,3 +2,4 @@ - A line over 2,000 characters is cut, with how many characters follow and the `offsetChars` that reads them. Before, a line page showed every line whole, however long. - A `read` page stops before its result would pass `resultMaxChars`, the most of it the framework stores whole. For a model's call that's its agent's inline cap, 24,000 characters by default. Its note says where to continue: the `offset` after its last line, or for a character page the `offsetChars` it also gives as `nextOffsetChars`. Where even a page's first line would pass the limit, that line is cut shorter. Before, a page of up to 2,000 lines, or of `limitChars` characters, went out whole, and the framework wrote what passed the limit to a file, cut mid-JSON. A caller that pages 2,000 lines at a time and starts the next page at line 2,001 now starts where the note says. A running `code_execution` script's call has a limit of 5,000,000 characters, so its pages still stop at 2,000 lines or `limitChars`, unless a page would pass that. - The workspace's `grep` leaves out a CR before a line feed, for matching as well as display, as ripgrep's `--crlf` mode does, so `foo$` finds a CRLF file's line too. A match's text and context are the lines as `read` shows them, a BOM isn't part of the first line, and a file with CRLF ends says so in its result's `lineEndings` (`CRLF`, or `mixed`). Before, a CRLF file's lines ended in a CR, so `foo$` missed them and a match's text copied into an edit carried the CR. A pattern written to find a CR at a line's end no longer matches. +- The workspace's `read` first takes in a change on disk to a file the workspace holds, as a listing and an `edit` do, and shows that text. Its note says so when disk's copy had changed since the workspace last agreed with it. It also says when disk's copy couldn't be checked, or disk has none and nothing records whether it was deleted there, so it shows the workspace's copy. A file disk deleted answers that disk deleted it and the deletion was taken in. Where the path's stored copy is a draft and disk changed too, the read shows the draft, and its note names the conflict and both ways to settle it. Before, a read of a path the workspace held didn't look at disk, so it could show a copy another writer had since changed. `readBinary`, which the framework's `read_image` reads through, settles its path the same way. diff --git a/src/modules/workspace/index.ts b/src/modules/workspace/index.ts index 15732869c..f39026e34 100644 --- a/src/modules/workspace/index.ts +++ b/src/modules/workspace/index.ts @@ -1830,9 +1830,10 @@ export class WorkspaceModule implements Module { } /** - * Read a file's raw bytes from a mount (through the Chronicle tree, synced - * first like the `read` tool). Public API for the framework's synthesized - * `read_image` tool and other peer callers that need binary content. + * Read a file's raw bytes from a mount (through the Chronicle tree, its path + * settled against disk first, as the `read` tool settles it). Public API for + * the framework's synthesized `read_image` tool and other peer callers that + * need binary content. */ async readBinary( mountPrefixedPath: string, @@ -1853,7 +1854,7 @@ export class WorkspaceModule implements Module { } const store = this.getStore(); return this.withMount(mount, async () => { - await this.ensureSynced(mount, relativePath); + await this.passUnlocked(mount, { kind: 'paths', paths: [relativePath] }); const entry = store.treeGet(mount.treeStateId, relativePath); if (!entry) return { error: `File not found: ${mountPrefixedPath}` }; const blob = store.getBlob(entry.blobHash); @@ -2623,13 +2624,15 @@ export class WorkspaceModule implements Module { } /** - * Before a read: a path the workspace doesn't hold is observed under the + * Before a deletion: a path the workspace doesn't hold is observed under the * three-way rule, which ingests a new disk file but never brings back a - * workspace deletion or a file absent from this branch. The pass names the - * path, as a file tool names the one it acts on: a path the mount passes - * over by design is observed, as a sync naming it would observe it (#276), - * while a path in another mount's directory is refused before this, and - * passed over by any pass. The caller holds the mount's turn. + * workspace deletion or a file absent from this branch. A path it holds is + * left alone, as a write's is: taking disk's change in first would let the + * deletion's push remove that change unseen. The pass names the path, as a + * file tool names the one it acts on: a path the mount passes over by + * design is observed, as a sync naming it would observe it (#276), while a + * path in another mount's directory is refused before this, and passed + * over by any pass. The caller holds the mount's turn. */ private async ensureSynced(mount: MountState, relativePath: string): Promise { const store = this.getStore(); @@ -2674,20 +2677,36 @@ export class WorkspaceModule implements Module { return { success: false, isError: true, error: 'Use either offset/limit (lines) or offsetChars/limitChars (characters), not both.' }; } let file: FileText; + let notes: string[] = []; if (generatedContent === undefined) { const { mount, relativePath, path } = await this.parseFilePath(input.path); input = { ...input, path }; const store = this.getStore(); - const found = await this.withMount(mount, async (): Promise<{ bytes: Buffer } | { error: string }> => { - const report = await this.ensureSynced(mount, relativePath); + const found = await this.withMount(mount, async (): Promise<{ bytes: Buffer; notes: string[] } | { error: string }> => { + // A read shows the path as disk holds it now: a change on disk to a + // file the workspace holds is taken in first, by the rule a listing + // uses, as an edit's is. Where the stored copy is a draft and disk + // changed too, the read shows the draft and names the conflict. + const pass = await this.passUnlocked(mount, { kind: 'paths', paths: [relativePath] }); + const report = pass.reports.get(relativePath); + const took = (op: TreeOp) => pass.ops.some((o) => o.path === relativePath && o.op === op); const entry = store.treeGet(mount.treeStateId, relativePath); - if (!entry) return { error: this.notInWorkspace(input.path, report) }; + if (!entry) { + // A pass also drops a copy disk made binary or too large, which + // the disk-only answer describes; any other drop is disk's deletion. + return { + error: took('deleted') && report?.state !== 'disk-only' + ? `File not found: ${input.path}. Disk's copy was deleted since the workspace last agreed with it, and that was taken in first.` + : this.notInWorkspace(input.path, report), + }; + } const blob = store.getBlob(entry.blobHash); if (!blob) return { error: `Blob not found for: ${input.path}` }; - return { bytes: blob }; + return { bytes: blob, notes: whichCopyRead(report, took('modified')) }; }); if ('error' in found) return { success: false, error: found.error, isError: true }; file = fileText(found.bytes); + notes = found.notes; } else { file = fileText(Buffer.from(generatedContent, 'utf-8')); } @@ -2696,8 +2715,8 @@ export class WorkspaceModule implements Module { // context and 400s), so an omitted limit falls back to a default, and // every page stops at the result limit. return characterPaging - ? characterPage(input.path, file, input.offsetChars ?? 0, input.limitChars ?? 2000, maxChars) - : linePage(input.path, file, input.offset ?? 1, input.limit ?? DEFAULT_READ_LINE_LIMIT, maxChars); + ? characterPage(input.path, file, input.offsetChars ?? 0, input.limitChars ?? 2000, maxChars, notes) + : linePage(input.path, file, input.offset ?? 1, input.limit ?? DEFAULT_READ_LINE_LIMIT, maxChars, notes); } private async handleReadImage(input: ReadImageInput): Promise { @@ -4286,6 +4305,30 @@ function storedButUnfinished( return { success: false, isError: true, error: stored + rest + (note !== null ? ` ${note}` : '') }; } +/** + * What a read says about the copy it shows, when that isn't simply the one the + * workspace last agreed on with disk: disk's newer text, taken in first; the + * workspace's copy, because disk's couldn't be checked or disk has none that + * anything accounts for; or a draft in conflict with disk's change. A file + * the workspace didn't hold has no older copy, so taking it in needs no word. + */ +function whichCopyRead(report: PathReport | undefined, tookChange: boolean): string[] { + if (tookChange) { + return ['Disk\'s copy had changed since the workspace last agreed with it, so it was taken in first, and this read shows that text.']; + } + if (report?.state === 'unverified') { + return [`The path couldn't be checked against disk (${report.note ?? 'not observed'}), so this read shows the workspace's copy.`]; + } + if (report?.state === 'disk-missing-provenance-unknown') { + return ['Disk has no copy of this file, and nothing records whether it was deleted there, so this read shows the workspace\'s copy.']; + } + if (report?.state === 'conflict') { + return [`The path is in conflict with disk (${report.conflict?.kind ?? 'both-changed'}): this read shows the workspace's copy. ` + + 'Sync the path to take disk\'s copy instead, or materialize it with force to write the workspace\'s over disk\'s.']; + } + return []; +} + /** * The least each tool's whole-number fields may be, beside their schemas, * whose type (membrane's ToolParameter) has no `minimum`. diff --git a/src/modules/workspace/read-page.ts b/src/modules/workspace/read-page.ts index 6fc5039ff..5f025315f 100644 --- a/src/modules/workspace/read-page.ts +++ b/src/modules/workspace/read-page.ts @@ -116,7 +116,7 @@ const splitsPair = (text: string, at: number): boolean => * keep a surrogate pair whole), stopped sooner where its data would pass * `maxChars` as JSON. */ -export function characterPage(path: string, file: FileText, offsetChars: number, limitChars: number, maxChars: number): ToolResult { +export function characterPage(path: string, file: FileText, offsetChars: number, limitChars: number, maxChars: number, extraNotes: string[] = []): ToolResult { const text = file.raw; const start = Math.min(offsetChars, text.length); if (splitsPair(text, start)) { @@ -124,7 +124,7 @@ export function characterPage(path: string, file: FileText, offsetChars: number, } let end = start + Math.min(limitChars, text.length - start); if (splitsPair(text, end)) end++; - const notes = file.utf8 ? [] : [NOT_UTF8]; + const notes = [...(file.utf8 ? [] : [NOT_UTF8]), ...extraNotes]; const stoppedNote = (to: number) => `Stopped at character ${to} to stay within the ${maxChars}-character result limit; continue with offsetChars=${to}.`; const page = (to: number, note: string | undefined) => ({ diff --git a/test/workspace-file-tools.test.ts b/test/workspace-file-tools.test.ts index 3251125e8..ee9c2f6ae 100644 --- a/test/workspace-file-tools.test.ts +++ b/test/workspace-file-tools.test.ts @@ -520,6 +520,85 @@ test("an edit refused for a workspace copy that isn't valid UTF-8 names the path /^The edit did nothing: work\/c\.txt isn't valid UTF-8, .*\n\nThe path is in conflict with disk \(unknown-provenance\): the edit looked in the workspace's copy\. Sync the path to keep disk's copy and give up the workspace's, or materialize it with force to write the workspace's over disk's\.$/); }); +test("a read shows disk's text when disk changed and the store didn't, and says so", async (t) => { + const { dir, call } = setup(t, { autoMaterialize: true }); + writeFileSync(join(dir, 'r.txt'), 'v1\n'); + assert.equal((await call('read', { path: 'work/r.txt' })).success, true); + writeFileSync(join(dir, 'r.txt'), 'v2 from another writer\n'); + const taken = "Disk's copy had changed since the workspace last agreed with it, so it was taken in first, and this read shows that text."; + assert.deepEqual((await call('read', { path: 'work/r.txt' })).data, { + path: 'work/r.txt', totalLines: 1, fromLine: 1, toLine: 1, note: taken, content: ' 1\tv2 from another writer', + }); + // That's the workspace's copy now, so the next read has nothing to say. + assert.deepEqual((await call('read', { path: 'work/r.txt' })).data, { + path: 'work/r.txt', totalLines: 1, fromLine: 1, toLine: 1, content: ' 1\tv2 from another writer', + }); + // A character page settles and says so too. + writeFileSync(join(dir, 'r.txt'), 'v3\n'); + assert.deepEqual((await call('read', { path: 'work/r.txt', offsetChars: 0 })).data, { + path: 'work/r.txt', totalChars: 3, offsetChars: 0, nextOffsetChars: null, note: taken, content: 'v3\n', + }); +}); + +test('a read of a file disk deleted since the workspace read it says so', async (t) => { + const { dir, call } = setup(t, { autoMaterialize: true }); + writeFileSync(join(dir, 'g.txt'), 'here\n'); + assert.equal((await call('read', { path: 'work/g.txt' })).success, true); + rmSync(join(dir, 'g.txt')); + assert.equal(errorOf(await call('read', { path: 'work/g.txt' })), + "File not found: work/g.txt. Disk's copy was deleted since the workspace last agreed with it, and that was taken in first."); +}); + +test('a read of a draft whose disk copy also changed shows the draft and names the conflict', async (t) => { + const { dir, call, disk } = setup(t); + writeFileSync(join(dir, 'k.txt'), 'base\n'); + assert.equal((await call('read', { path: 'work/k.txt' })).success, true); + assert.equal((await call('write', { path: 'work/k.txt', content: 'agent draft\n' })).success, true); + writeFileSync(join(dir, 'k.txt'), 'another writer\n'); + assert.deepEqual((await call('read', { path: 'work/k.txt' })).data, { + path: 'work/k.txt', totalLines: 1, fromLine: 1, toLine: 1, + note: "The path is in conflict with disk (both-changed): this read shows the workspace's copy. " + + "Sync the path to take disk's copy instead, or materialize it with force to write the workspace's over disk's.", + content: ' 1\tagent draft', + }); + assert.equal(disk('k.txt')!.toString(), 'another writer\n', 'the read left disk as it was'); +}); + +test("a read whose disk copy can't be checked says it shows the workspace's copy", async (t) => { + const { dir, call } = setup(t); + writeFileSync(join(dir, 'u.txt'), 'held\n'); + assert.equal((await call('read', { path: 'work/u.txt' })).success, true); + rmSync(join(dir, 'u.txt')); + mkdirSync(join(dir, 'u.txt')); + assert.deepEqual((await call('read', { path: 'work/u.txt' })).data, { + path: 'work/u.txt', totalLines: 1, fromLine: 1, toLine: 1, + note: "The path couldn't be checked against disk (a directory), so this read shows the workspace's copy.", + content: ' 1\theld', + }); +}); + +test("a read of a file disk lacks, with nothing to say whether disk deleted it, says it shows the workspace's copy", async (t) => { + const { store, call } = setup(t); + // An entry from before the evidence (as three-way's legacyEntry makes it). + const bytes = Buffer.from('legacy line\n'); + store.treeSet('workspace/work/tree', 'l.txt', { blobHash: store.storeBlob(bytes, 'text/plain'), size: bytes.byteLength, mode: 0o644 }); + assert.deepEqual((await call('read', { path: 'work/l.txt' })).data, { + path: 'work/l.txt', totalLines: 1, fromLine: 1, toLine: 1, + note: "Disk has no copy of this file, and nothing records whether it was deleted there, so this read shows the workspace's copy.", + content: ' 1\tlegacy line', + }); +}); + +test("readBinary takes in disk's change to a file the workspace holds, as read does", async (t) => { + const { workspace, dir, call } = setup(t, { autoMaterialize: true }); + writeFileSync(join(dir, 'b.txt'), 'old\n'); + assert.equal((await call('read', { path: 'work/b.txt' })).success, true); + writeFileSync(join(dir, 'b.txt'), 'new from another writer\n'); + const read = await workspace.readBinary('work/b.txt'); + assert.ok('data' in read, JSON.stringify(read)); + assert.equal(read.data.toString(), 'new from another writer\n'); +}); + // --------------------------------------------------------------------------- // Part 5: smaller texts // --------------------------------------------------------------------------- diff --git a/test/workspace-three-way.test.ts b/test/workspace-three-way.test.ts index 3b96e58fe..67335c2bf 100644 --- a/test/workspace-three-way.test.ts +++ b/test/workspace-three-way.test.ts @@ -175,12 +175,19 @@ async function stateOf(m: WorkspaceModule, path: string): Promise { return (await entryOf(m, path))?.state ?? 'not listed'; } -/** The workspace's text for a path, unformatted. */ +/** The workspace's text for a path, unformatted, as a read shows it: after its path is settled against disk. */ async function contentOf(m: WorkspaceModule, path: string): Promise { const data = await call(m, 'read', { path: `work/${path}`, offsetChars: 0, limitChars: 1_000_000 }); return data.content as string; } +/** The text the store holds for a path on the selected branch, read without a pass. */ +function storedOf(env: Env, path: string): string | null { + const entry = env.store.treeGet(TREE, path); + if (!entry) return null; + return env.store.getBlob(entry.blobHash)!.toString('utf-8'); +} + /** Written through the workspace and materialized: S = D = P. */ async function seedSynced(env: Env, m: WorkspaceModule, path: string, content: string): Promise { await call(m, 'write', { path: `work/${path}`, content }); @@ -564,9 +571,10 @@ describe('a path sync', () => { const data = await syncing; assert.match(JSON.stringify(data.incomplete), /the selected branch changed before this sync ran/); assert.deepEqual(data.results, []); - assert.equal(await contentOf(m, 'a.txt'), "other's draft", "other's draft is untouched"); + // Read from the store: a read would settle the path against disk itself. + assert.equal(storedOf(env, 'a.txt'), "other's draft", "other's draft is untouched"); env.store.switchBranch(main); - assert.equal(await contentOf(m, 'a.txt'), 'v1', 'and main took nothing either'); + assert.equal(storedOf(env, 'a.txt'), 'v1', 'and main took nothing either'); }); test("lists a deletion it gives up for a disk copy the workspace can't hold, though no workspace entry changes", async (t) => { @@ -4574,9 +4582,10 @@ describe('fingerprints', () => { env.writeDisk('f.txt', 'bbbb'); // same size utimesSync(env.disk('f.txt'), before.atime, before.mtime); - assert.equal(await contentOf(m, 'f.txt'), 'aaaa', 'the read serves the store'); + // Read from the store, so only the listing's pass looks at disk. + assert.equal(storedOf(env, 'f.txt'), 'aaaa', 'nothing has looked at disk yet'); assert.equal(await stateOf(m, 'f.txt'), 'synced'); - assert.equal(await contentOf(m, 'f.txt'), 'bbbb', 'utimes restoring mtime: rehashed and adopted'); + assert.equal(storedOf(env, 'f.txt'), 'bbbb', 'utimes restoring mtime: rehashed and adopted'); await sleep(2100); await call(m, 'ls', { path: 'work' }); @@ -4585,11 +4594,11 @@ describe('fingerprints', () => { const cp = spawnSync('cp', ['-p', join(env.root, 'src.txt'), env.disk('f.txt')]); assert.equal(cp.status, 0, `cp -p: ${cp.stderr}`); assert.equal(await stateOf(m, 'f.txt'), 'synced'); - assert.equal(await contentOf(m, 'f.txt'), 'cccc', 'cp -p: rehashed and adopted'); + assert.equal(storedOf(env, 'f.txt'), 'cccc', 'cp -p: rehashed and adopted'); env.writeDisk('f.txt', 'dddd'); // just after the last hash assert.equal(await stateOf(m, 'f.txt'), 'synced'); - assert.equal(await contentOf(m, 'f.txt'), 'dddd'); + assert.equal(storedOf(env, 'f.txt'), 'dddd'); }); }); @@ -5474,7 +5483,8 @@ describe('a mount larger than its walk cap (agent-framework #272)', () => { env.writeDisk('zz/t3.txt', 'changed behind l3'); await call(m, 'sync', {}); assert.deepEqual(syncedNames(env), ['a.txt', 'l1', 'l2', 'l3', 'zz/t1.txt', 'zz/t2.txt'], 'the third pass took zz/t1.txt and zz/t2.txt'); - assert.equal(await contentOf(m, 'l3'), 'target zz/t3.txt'); + // Read from the store: a read of l3 would pass over l3 itself. + assert.equal(storedOf(env, 'l3'), 'target zz/t3.txt'); }); test('a directory proven missing on the after stretch that is an in-mount link on the wrap proves nothing absent', async (t) => { From b33517e9a44b756ba6b3abf00d6a982aac14cb5e Mon Sep 17 00:00:00 2001 From: Abigail-2676 Date: Sat, 17 Oct 2026 17:28:46 -0400 Subject: [PATCH 04/51] fix(workspace): a read's cut last line is named in its note, readBinary keeps the workspace's copy, and a deletion conflict says what a sync takes What review found in the read half, and the fixes: - A page whose last line is cut, at 2,000 characters or as a first line cut to fit, named the next line as where to continue. Reading on in order starts with that line's rest, which only the line's own marker named. The note now names the offsetChars of the rest before the next line's offset, on a limit's note and a budget's alike. A page that ends the file has no next line to name, so its cut line's marker stands alone, as before. - The first line's cut is sized on the page itself, now that the note names the cut. The line as a page shows it is tried first: shown whole, it carries no marker and its note no cut, so its page can fit where a cut one character shorter doesn't (a one-line file of 900 characters at 1,000: 984 characters whole, 1,032 cut at 899). Below that, a longer cut never makes the page shorter, and a binary search finds the most that fits. The estimate it replaces reserved the longest note, so on a page that ends the file, which carries none, the cut stopped short by that note's length. - readBinary goes back to observing only a path the workspace doesn't hold (ensureSynced), as before this branch. The pass takes no binary disk copy into the tree, so settling dropped a held binary file that disk had changed, and readBinary, a reader of binary content, answered File not found for a file disk still had. read_image doesn't reach readBinary on this module either: the framework calls the module's own read_image handler where the module serves it, and readBinary only where it doesn't. readBinary's doc and ensureSynced's now say why a held path is left alone. - A conflict's note told the reader that a sync takes disk's copy. Where disk has none, a sync takes the deletion and removes the draft, and the note now says so wherever disk has no copy now (the report's diskNow), not by the conflict's kind: a recorded conflict keeps its kind, so one recorded as both-changed before disk deleted the file is still both-changed (which the test below confirms). Where disk has a copy the workspace can't hold (binary or too large), a sync takes nothing in and only gives up the draft, so the note says what holds in both cases: a sync keeps disk's copy and gives up the workspace's. - The mixed-line-ends note said each line is shown without a CR. Only a CR before a line feed is dropped, and the note says that. - The read tool's description scopes the CRLF and BOM wording to a line page, and says a character page returns the stored text as it is, BOM and CRs included. - The changelog says where a page's bound holds: where its own fields and the least it can show fit, which at 1,000 characters only a path of about 130 characters beside every note a read can carry, or about 700 with none, passes. It also gives the cut-line note, a file disk made binary, the deletion conflict, grep's CR wording, that a path the workspace didn't hold was observed before too, and the prompt-cache miss the changed tool descriptions cost once. It no longer mentions readBinary, which this branch now leaves as main has it. Tests: - workspace-read-text: the first-line cut's note names the rest, and one character more passes the limit, on a page with a note and on a one-line file's page with none; a one-line file whose whole page fits is shown whole, where a cut one character shorter wouldn't fit; a page whose last line is cut before a surrogate pair names the offset its marker does; a page stopped at a cut line by its limit and by the result limit names the rest, and a page that ends the file has no note; a call that carries no resultMaxChars gets pages within 24,000; each of the emoji loop's four limits checks the page's bound; the mixed note's wording. - workspace-file-tools: a read of a draft disk deleted gives the deletion's note, and so does a conflict recorded as both-changed before disk deleted the file; a sync of the path then removes the draft, and a forced materialize writes it back; a draft whose disk copy turned binary gets the note that a sync keeps disk's copy and gives up the draft, and the sync does that; a read of a file disk made binary answers the disk-only text; readBinary serves a held binary file that disk changed as the workspace holds it (in place of the test that pinned the settle). Co-Authored-By: Claude Opus 5.5 --- changelog.d/feat-workspace-read.changed.md | 9 +-- src/modules/workspace/index.ts | 53 +++++++++------ src/modules/workspace/read-page.ts | 66 +++++++++++++------ test/workspace-file-tools.test.ts | 76 ++++++++++++++++++++-- test/workspace-read-text.test.ts | 65 +++++++++++++++++- 5 files changed, 217 insertions(+), 52 deletions(-) diff --git a/changelog.d/feat-workspace-read.changed.md b/changelog.d/feat-workspace-read.changed.md index 2c1d02921..2614eef15 100644 --- a/changelog.d/feat-workspace-read.changed.md +++ b/changelog.d/feat-workspace-read.changed.md @@ -1,5 +1,6 @@ -- The workspace's `read` shows a file's text as it is, and says so. A CRLF file's lines come without the CR, and a BOM isn't shown. The page's note says when the file has either, counting a mix of CRLF and LF ends. It also says when the file isn't valid UTF-8: each invalid byte sequence shows as U+FFFD, `edit` refuses the file, and `write` replaces it whole. Before, each line of a CRLF file ended in a CR, a BOM came first in the first line as U+FEFF, and invalid bytes showed as U+FFFD with no word. A character page still reads the stored text as it is, BOM and CRs included. +- The workspace's `read` shows a file's text as it is, and says so. A line page shows a CRLF file's lines without the CR before each line feed, and leaves out a BOM. Its note says when the file has either, counting a mix of CRLF and LF ends. It also says when the file isn't valid UTF-8: each invalid byte sequence shows as U+FFFD, `edit` refuses the file, and `write` replaces it whole. Before, each line of a CRLF file ended in a CR, a BOM came first in the first line as U+FEFF, and invalid bytes showed as U+FFFD with no word. A character page still reads the stored text as it is, BOM and CRs included, and the tool's description now says so. - A line over 2,000 characters is cut, with how many characters follow and the `offsetChars` that reads them. Before, a line page showed every line whole, however long. -- A `read` page stops before its result would pass `resultMaxChars`, the most of it the framework stores whole. For a model's call that's its agent's inline cap, 24,000 characters by default. Its note says where to continue: the `offset` after its last line, or for a character page the `offsetChars` it also gives as `nextOffsetChars`. Where even a page's first line would pass the limit, that line is cut shorter. Before, a page of up to 2,000 lines, or of `limitChars` characters, went out whole, and the framework wrote what passed the limit to a file, cut mid-JSON. A caller that pages 2,000 lines at a time and starts the next page at line 2,001 now starts where the note says. A running `code_execution` script's call has a limit of 5,000,000 characters, so its pages still stop at 2,000 lines or `limitChars`, unless a page would pass that. -- The workspace's `grep` leaves out a CR before a line feed, for matching as well as display, as ripgrep's `--crlf` mode does, so `foo$` finds a CRLF file's line too. A match's text and context are the lines as `read` shows them, a BOM isn't part of the first line, and a file with CRLF ends says so in its result's `lineEndings` (`CRLF`, or `mixed`). Before, a CRLF file's lines ended in a CR, so `foo$` missed them and a match's text copied into an edit carried the CR. A pattern written to find a CR at a line's end no longer matches. -- The workspace's `read` first takes in a change on disk to a file the workspace holds, as a listing and an `edit` do, and shows that text. Its note says so when disk's copy had changed since the workspace last agreed with it. It also says when disk's copy couldn't be checked, or disk has none and nothing records whether it was deleted there, so it shows the workspace's copy. A file disk deleted answers that disk deleted it and the deletion was taken in. Where the path's stored copy is a draft and disk changed too, the read shows the draft, and its note names the conflict and both ways to settle it. Before, a read of a path the workspace held didn't look at disk, so it could show a copy another writer had since changed. `readBinary`, which the framework's `read_image` reads through, settles its path the same way. +- A `read` page stops before its result would pass `resultMaxChars`, the most of it the framework stores whole. For a model's call that's its agent's inline cap, 24,000 characters by default, and a call that carries none gets 24,000 too. Its note says where to continue: the `offset` after its last line, or for a character page the `offsetChars` it also gives as `nextOffsetChars`. Where even a page's first line would pass the limit, that line is cut shorter, at the most the page has room for. Where a page's last line is cut, its note names the `offsetChars` where that line's rest starts before the next line's `offset`. A page always moves on, so where its own fields (the path and its notes) and the least it can show pass a small limit, it passes the limit: at 1,000 characters, only a path of about 130 characters or more beside every note a read can carry does that, or one of about 700 with none. Before, a page of up to 2,000 lines, or of `limitChars` characters, went out whole, and the framework wrote what passed the limit to a file, cut mid-JSON. A caller that pages 2,000 lines at a time and starts the next page at line 2,001 now starts where the note says. A running `code_execution` script's call has a limit of 5,000,000 characters, so its pages still stop at 2,000 lines or `limitChars`, unless a page would pass that. +- The workspace's `grep` leaves out a CR before a line feed, for matching as well as display, as ripgrep's `--crlf` mode does, so `foo$` finds a CRLF file's line too. A match's text and context are the lines as `read` shows them, a BOM isn't part of the first line, and a file with CRLF ends says so in its result's `lineEndings` (`CRLF`, or `mixed`). Before, a CRLF file's lines ended in a CR, so `foo$` missed them and a match's text copied into an edit carried the CR. A pattern written to find the CR of a CRLF line end no longer matches; a CR that doesn't come before a line feed is still part of its line. +- The workspace's `read` first takes in a change on disk to a file the workspace holds, as a listing and an `edit` do, and shows that text. Its note says so when disk's copy had changed since the workspace last agreed with it. It also says when disk's copy couldn't be checked, or disk has none and nothing records whether it was deleted there, so it shows the workspace's copy. A file disk deleted answers that disk deleted it and the deletion was taken in, and a file disk made binary or too large answers that it's on disk but not stored. Where the path's stored copy is a draft and disk changed or deleted it too, the read shows the draft, and its note names the conflict and both ways to settle it: for a deletion, a sync of the path takes it and removes the workspace's copy. Before, a read of a path the workspace held didn't look at disk, so it could show a copy another writer had since changed; a path it didn't hold was observed, as now. +- The `read` and `grep` tool descriptions changed, so the first request after the upgrade that carries the workspace's tools misses the prompt cache from its tool block on. diff --git a/src/modules/workspace/index.ts b/src/modules/workspace/index.ts index f39026e34..055877105 100644 --- a/src/modules/workspace/index.ts +++ b/src/modules/workspace/index.ts @@ -1295,10 +1295,11 @@ export class WorkspaceModule implements Module { + 'Reads at most 2000 lines per call by default — use offset/limit to page through larger files — ' + 'and stops sooner where a page would pass the result limit; the note says where to continue. ' + 'A line over 2000 characters is cut, with the offsetChars that reads the rest. ' - + 'CRLF line ends are shown without the CR, and a BOM isn\'t shown; the note says when the file has either, ' + + 'A line page shows CRLF line ends without the CR, and no BOM; its note says when the file has either, ' + 'or isn\'t valid UTF-8. ' - + 'For long lines or spill files, use offsetChars/limitChars instead: returns raw text without ' - + 'line numbers, with nextOffsetChars (null at EOF), stopped at the result limit too. Keep limitChars when continuing. ' + + 'For long lines or spill files, use offsetChars/limitChars instead: returns the stored text as it is, ' + + 'BOM and CRs included, without line numbers, with nextOffsetChars (null at EOF), stopped at the result limit too. ' + + 'Keep limitChars when continuing. ' + 'Character offsets count UTF-16 code units; do not mix character and line parameters.', inputSchema: { type: 'object' as const, @@ -1830,10 +1831,14 @@ export class WorkspaceModule implements Module { } /** - * Read a file's raw bytes from a mount (through the Chronicle tree, its path - * settled against disk first, as the `read` tool settles it). Public API for - * the framework's synthesized `read_image` tool and other peer callers that - * need binary content. + * Read a file's raw bytes from a mount, through the Chronicle tree, a path + * the workspace doesn't hold observed first (ensureSynced). A path it holds + * is read as the workspace holds it, not settled as the `read` tool settles + * it: the tree takes in no binary disk copy, so a pass over a held binary + * file that disk changed would drop the workspace's copy. Public API for + * the framework's synthesized `read_image` tool, where the workspace module + * doesn't serve read_image itself, and other peer callers that need binary + * content. */ async readBinary( mountPrefixedPath: string, @@ -1854,7 +1859,7 @@ export class WorkspaceModule implements Module { } const store = this.getStore(); return this.withMount(mount, async () => { - await this.passUnlocked(mount, { kind: 'paths', paths: [relativePath] }); + await this.ensureSynced(mount, relativePath); const entry = store.treeGet(mount.treeStateId, relativePath); if (!entry) return { error: `File not found: ${mountPrefixedPath}` }; const blob = store.getBlob(entry.blobHash); @@ -2624,15 +2629,17 @@ export class WorkspaceModule implements Module { } /** - * Before a deletion: a path the workspace doesn't hold is observed under the - * three-way rule, which ingests a new disk file but never brings back a - * workspace deletion or a file absent from this branch. A path it holds is - * left alone, as a write's is: taking disk's change in first would let the - * deletion's push remove that change unseen. The pass names the path, as a - * file tool names the one it acts on: a path the mount passes over by - * design is observed, as a sync naming it would observe it (#276), while a - * path in another mount's directory is refused before this, and passed - * over by any pass. The caller holds the mount's turn. + * Before a deletion, and for readBinary: a path the workspace doesn't hold + * is observed under the three-way rule, which ingests a new disk file but + * never brings back a workspace deletion or a file absent from this branch. + * A path it holds is left alone. For a deletion, as for a write, taking + * disk's change in first would let the push remove that change unseen; for + * readBinary, a pass would drop a held binary file that disk changed, since + * the tree takes in no binary disk copy. The pass names the path, as a file + * tool names the one it acts on: a path the mount passes over by design is + * observed, as a sync naming it would observe it (#276), while a path in + * another mount's directory is refused before this, and passed over by any + * pass. The caller holds the mount's turn. */ private async ensureSynced(mount: MountState, relativePath: string): Promise { const store = this.getStore(); @@ -4323,8 +4330,16 @@ function whichCopyRead(report: PathReport | undefined, tookChange: boolean): str return ['Disk has no copy of this file, and nothing records whether it was deleted there, so this read shows the workspace\'s copy.']; } if (report?.state === 'conflict') { - return [`The path is in conflict with disk (${report.conflict?.kind ?? 'both-changed'}): this read shows the workspace's copy. ` - + 'Sync the path to take disk\'s copy instead, or materialize it with force to write the workspace\'s over disk\'s.']; + const kind = report.conflict?.kind ?? 'both-changed'; + // A sync takes disk's state as it is now, which a recorded conflict's kind + // doesn't follow: where disk has no copy, the sync takes its deletion, and + // the workspace's copy goes with it. Where disk has one, the sync keeps it + // and gives up the workspace's, taking disk's in only where the workspace + // can hold it (not binary or too large). + return [`The path is in conflict with disk (${kind}): this read shows the workspace's copy. ` + + (report.diskNow === 'absent' + ? 'Sync the path to take disk\'s deletion, which removes the workspace\'s copy, or materialize it with force to write the workspace\'s copy back to disk.' + : 'Sync the path to keep disk\'s copy and give up the workspace\'s, or materialize it with force to write the workspace\'s over disk\'s.')]; } return []; } diff --git a/src/modules/workspace/read-page.ts b/src/modules/workspace/read-page.ts index 5f025315f..2d10d5898 100644 --- a/src/modules/workspace/read-page.ts +++ b/src/modules/workspace/read-page.ts @@ -1,5 +1,5 @@ import type { ToolResult } from '../../types/events.js'; -import { type FileText, MAX_LINE_CHARS, jsonCharLength, jsonLength, numberedLine } from './text.js'; +import { type FileText, MAX_LINE_CHARS, jsonCharLength, jsonLength, numberedLine, safeCut } from './text.js'; /** * One page of a read, held to the result limit: the framework stores a @@ -16,7 +16,7 @@ export function textNotes(file: FileText): string[] { const notes: string[] = []; if (file.crlfEnds > 0) { notes.push(file.lfEnds > 0 - ? `Line ends are mixed, ${file.crlfEnds} CRLF and ${file.lfEnds} LF, each line shown without a CR. An edit matches text as read shows it, and keeps each line's end.` + ? `Line ends are mixed, ${file.crlfEnds} CRLF and ${file.lfEnds} LF, and a CRLF end is shown without its CR. An edit matches text as read shows it, and keeps each line's end.` : 'Line ends are CRLF, shown without the CR. An edit matches text as read shows it, and keeps the CRLF ends.'); } if (file.bom) notes.push('The file starts with a UTF-8 BOM, which isn\'t shown. An edit keeps it.'); @@ -35,7 +35,11 @@ const joined = (notes: Array): string | undefined => { * Lines from `offset` (1-based), at most `limit` of them, each numbered and a * long one cut with the offsetChars that reads the rest. The page stops where * one more line would take its data past `maxChars` as JSON; if even the - * first line doesn't fit, that line is cut shorter. + * first line doesn't fit, that line is cut shorter, at the most the page has + * room for. Where the page's last line is cut, its note names where that + * line's rest starts before the next line, since reading on in order starts + * there. A page whose own fields and least content pass `maxChars` still + * moves on, and passes it. */ export function linePage(path: string, file: FileText, offset: number, limit: number, maxChars: number, extraNotes: string[] = []): ToolResult { const { lines, starts } = file; @@ -55,17 +59,24 @@ export function linePage(path: string, file: FileText, offset: number, limit: nu const last = Math.min(first + limit, total); const shown = (i: number, maxLineChars = MAX_LINE_CHARS) => numberedLine(lines[i], i + 1, maxLineChars, (cut) => `read them with offsetChars=${starts[i] + cut}`); - const limitNote = `Truncated at ${limit} lines (file has ${total}). Use offset/limit to read more.`; - const budgetNote = (to: number) => - `Stopped at line ${to} to stay within the ${maxChars}-character result limit (file has ${total}). Continue with offset=${to + 1}.`; + // Where the rest of line i starts in the stored text, shown at most + // maxLineChars long, as its marker says; null for a line shown whole. + const restOf = (i: number, maxLineChars: number): number | null => + lines[i].length > maxLineChars ? starts[i] + safeCut(lines[i], maxLineChars) : null; + const cutLine = (to: number, rest: number) => `Line ${to} is cut: read its rest with offsetChars=${rest}`; + const limitNote = (to: number, rest: number | null) => + `Truncated at ${limit} lines (file has ${total}).${rest === null ? '' : ` ${cutLine(to, rest)}.`} Use offset/limit to read more.`; + const budgetNote = (to: number, rest: number | null) => + `Stopped at line ${to} to stay within the ${maxChars}-character result limit (file has ${total}). ` + + (rest === null ? `Continue with offset=${to + 1}.` : `${cutLine(to, rest)}, then continue with offset=${to + 1}.`); const page = (to: number, content: string, note: string | undefined) => ({ path, totalLines: total, fromLine: first + 1, toLine: to, ...(note !== undefined ? { note } : {}), content, }); // The most anything but the lines can take: the longer note, with the // largest numbers this page could show. const overhead = Math.max( - JSON.stringify(page(last, '', joined([limitNote, ...notes]))).length, - JSON.stringify(page(last, '', joined([budgetNote(last), ...notes]))).length, + JSON.stringify(page(last, '', joined([limitNote(last, file.raw.length), ...notes]))).length, + JSON.stringify(page(last, '', joined([budgetNote(last, file.raw.length), ...notes]))).length, ); const shownLines: string[] = []; let used = overhead; @@ -77,22 +88,39 @@ export function linePage(path: string, file: FileText, offset: number, limit: nu shownLines.push(line); used += cost; } - const final = () => { + // The page with the lines shown so far, its last one shown at most + // lastMax long. + const final = (lastMax = MAX_LINE_CHARS) => { const to = first + shownLines.length; - return page(to, shownLines.join('\n'), joined([to === last ? (to < total ? limitNote : null) : budgetNote(to), ...notes])); + const rest = restOf(to - 1, lastMax); + return page(to, shownLines.join('\n'), joined([to === last ? (to < total ? limitNote(to, rest) : null) : budgetNote(to, rest), ...notes])); }; if (shownLines.length === 0) { - // Not even the first line fits: cut it where it does, at the most it can - // show. A longer cut never shows less, so the first that doesn't fit + // The estimate above found no room for the first line as a page shows it. + // Measured on the page itself, since its note names a cut: the line as a + // page shows it first, then, where that doesn't fit, the most a cut can + // show. Shown whole, a line carries no marker and its note no cut, so its + // page can fit where a cut one character shorter doesn't. Below that a + // longer cut never makes the page shorter, so the first that doesn't fit // bounds the search. - let fits = 0; - let over = Math.min(lines[first].length, MAX_LINE_CHARS) + 1; - while (over - fits > 1) { - const mid = Math.floor((fits + over) / 2); - if (overhead + jsonLength(shown(first, mid)) <= maxChars) fits = mid; else over = mid; + const whole = Math.min(lines[first].length, MAX_LINE_CHARS); + const fitsAt = (chars: number) => { + shownLines[0] = shown(first, chars); + return JSON.stringify(final(chars)).length <= maxChars; + }; + let fits = whole; + if (!fitsAt(whole)) { + fits = 0; + let over = whole; + while (over - fits > 1) { + const mid = Math.floor((fits + over) / 2); + if (fitsAt(mid)) fits = mid; else over = mid; + } } - shownLines.push(shown(first, fits)); - return { success: true, data: final() }; + shownLines[0] = shown(first, fits); + // Cut shorter than a page shows it, the line ends the page: its rest + // comes next. Shown as a page shows it, more lines may fit beside it. + if (fits < whole) return { success: true, data: final(fits) }; } // The overhead allowed for the largest numbers; a line more may fit beside // the real ones. diff --git a/test/workspace-file-tools.test.ts b/test/workspace-file-tools.test.ts index ee9c2f6ae..cce69174e 100644 --- a/test/workspace-file-tools.test.ts +++ b/test/workspace-file-tools.test.ts @@ -558,12 +558,63 @@ test('a read of a draft whose disk copy also changed shows the draft and names t assert.deepEqual((await call('read', { path: 'work/k.txt' })).data, { path: 'work/k.txt', totalLines: 1, fromLine: 1, toLine: 1, note: "The path is in conflict with disk (both-changed): this read shows the workspace's copy. " - + "Sync the path to take disk's copy instead, or materialize it with force to write the workspace's over disk's.", + + "Sync the path to keep disk's copy and give up the workspace's, or materialize it with force to write the workspace's over disk's.", content: ' 1\tagent draft', }); assert.equal(disk('k.txt')!.toString(), 'another writer\n', 'the read left disk as it was'); }); +test("a read of a draft whose disk copy turned binary says a sync keeps disk's copy and gives up the draft, which it does", async (t) => { + const { dir, call } = setup(t); + writeFileSync(join(dir, 'm.txt'), 'base\n'); + assert.equal((await call('read', { path: 'work/m.txt' })).success, true); + assert.equal((await call('write', { path: 'work/m.txt', content: 'agent draft\n' })).success, true); + writeFileSync(join(dir, 'm.txt'), Buffer.from([0, 1, 2, 0, 255, 0])); + assert.equal(((await call('read', { path: 'work/m.txt' })).data as { note: string }).note, + "The path is in conflict with disk (both-changed): this read shows the workspace's copy. " + + "Sync the path to keep disk's copy and give up the workspace's, or materialize it with force to write the workspace's over disk's."); + // The workspace can't hold disk's copy, so the sync takes nothing in: the + // draft goes, and disk keeps its bytes. + assert.equal((await call('sync', { path: 'work/m.txt' })).success, true); + assert.equal(errorOf(await call('read', { path: 'work/m.txt' })), + 'work/m.txt is on disk but not stored in the workspace (binary or over the size limit).'); + assert.deepEqual([...readFileSync(join(dir, 'm.txt'))], [0, 1, 2, 0, 255, 0]); +}); + +test('a read of a draft whose disk copy was deleted says a sync takes the deletion, and each way out does what it says', async (t) => { + const { dir, call, disk } = setup(t); + for (const name of ['d.txt', 'e.txt']) { + writeFileSync(join(dir, name), 'base\n'); + assert.equal((await call('read', { path: `work/${name}` })).success, true); + assert.equal((await call('write', { path: `work/${name}`, content: 'agent draft\n' })).success, true); + rmSync(join(dir, name)); + assert.deepEqual((await call('read', { path: `work/${name}` })).data, { + path: `work/${name}`, totalLines: 1, fromLine: 1, toLine: 1, + note: "The path is in conflict with disk (deleted-on-disk): this read shows the workspace's copy. " + + "Sync the path to take disk's deletion, which removes the workspace's copy, or materialize it with force to write the workspace's copy back to disk.", + content: ' 1\tagent draft', + }); + } + // A conflict recorded before disk deleted the file keeps its kind, and the + // note follows what disk holds now. + writeFileSync(join(dir, 'f.txt'), 'base\n'); + assert.equal((await call('read', { path: 'work/f.txt' })).success, true); + assert.equal((await call('write', { path: 'work/f.txt', content: 'agent draft\n' })).success, true); + writeFileSync(join(dir, 'f.txt'), 'another writer\n'); + const noteOf = async (path: string) => ((await call('read', { path })).data as { note: string }).note; + assert.match(await noteOf('work/f.txt'), /^The path is in conflict with disk \(both-changed\): .*Sync the path to keep disk's copy and give up the workspace's/); + rmSync(join(dir, 'f.txt')); + assert.equal(await noteOf('work/f.txt'), + "The path is in conflict with disk (both-changed): this read shows the workspace's copy. " + + "Sync the path to take disk's deletion, which removes the workspace's copy, or materialize it with force to write the workspace's copy back to disk."); + // A sync of the path takes the deletion: the draft goes. + assert.equal((await call('sync', { path: 'work/d.txt' })).success, true); + assert.equal(errorOf(await call('read', { path: 'work/d.txt' })), 'File not found: work/d.txt'); + // A forced materialize of the path writes the draft back. + assert.equal((await call('materialize', { path: 'work/e.txt', force: true })).success, true); + assert.equal(disk('e.txt')!.toString(), 'agent draft\n'); +}); + test("a read whose disk copy can't be checked says it shows the workspace's copy", async (t) => { const { dir, call } = setup(t); writeFileSync(join(dir, 'u.txt'), 'held\n'); @@ -589,14 +640,25 @@ test("a read of a file disk lacks, with nothing to say whether disk deleted it, }); }); -test("readBinary takes in disk's change to a file the workspace holds, as read does", async (t) => { - const { workspace, dir, call } = setup(t, { autoMaterialize: true }); - writeFileSync(join(dir, 'b.txt'), 'old\n'); +test("a read of a file disk made binary answers the disk-only text, not a deletion", async (t) => { + const { dir, call } = setup(t, { autoMaterialize: true }); + writeFileSync(join(dir, 'b.txt'), 'text\n'); assert.equal((await call('read', { path: 'work/b.txt' })).success, true); - writeFileSync(join(dir, 'b.txt'), 'new from another writer\n'); - const read = await workspace.readBinary('work/b.txt'); + writeFileSync(join(dir, 'b.txt'), Buffer.from([0, 1, 2, 0, 255, 0])); + assert.equal(errorOf(await call('read', { path: 'work/b.txt' })), + 'work/b.txt is on disk but not stored in the workspace (binary or over the size limit).'); +}); + +test('readBinary serves a binary file the workspace holds as it holds it, though disk changed it', async (t) => { + // A settle would drop the held copy, since the tree takes in no binary disk + // copy, and readBinary, a reader of binary content, would find nothing. + const { workspace, dir } = setup(t, { autoMaterialize: true }); + const held = Buffer.from([0x89, 0x50, 0x4e, 0x47, 0, 1, 2, 3]); + assert.equal((await workspace.writeBinary('work/i.png', held, 'image/png')).success, true); + writeFileSync(join(dir, 'i.png'), Buffer.from([0x89, 0x50, 0x4e, 0x47, 9, 9, 9, 9])); + const read = await workspace.readBinary('work/i.png'); assert.ok('data' in read, JSON.stringify(read)); - assert.equal(read.data.toString(), 'new from another writer\n'); + assert.deepEqual([...read.data], [...held]); }); // --------------------------------------------------------------------------- diff --git a/test/workspace-read-text.test.ts b/test/workspace-read-text.test.ts index f0ec23fb8..16314232e 100644 --- a/test/workspace-read-text.test.ts +++ b/test/workspace-read-text.test.ts @@ -51,7 +51,7 @@ test('a CRLF file reads as its lines without the CR, and says so', async t => { const { read: readMixed } = await setup(t, 'a\r\nb\nc\r\nd'); const page = (await readMixed()).data as LinePage; assert.equal(page.content, ' 1\ta\n 2\tb\n 3\tc\n 4\td'); - assert.equal(page.note, 'Line ends are mixed, 2 CRLF and 1 LF, each line shown without a CR. An edit matches text as read shows it, and keeps each line\'s end.'); + assert.equal(page.note, 'Line ends are mixed, 2 CRLF and 1 LF, and a CRLF end is shown without its CR. An edit matches text as read shows it, and keeps each line\'s end.'); // A CR that doesn't end a line is the line's own. const { read: readCr } = await setup(t, 'a\rb\n'); assert.deepEqual((await readCr()).data, { path: 'work/text.txt', totalLines: 1, fromLine: 1, toLine: 1, content: ' 1\ta\rb' }); @@ -93,6 +93,9 @@ test('a long line is cut at 2000 characters, with the offsetChars that reads the const more = (await read({ offsetChars: at, limitChars: rest })).data as CharPage; assert.equal(more.content, '😀' + 'y'.repeat(1000)); assert.equal(lines[2], ' 3\tlast'); + // Where the cut line ends the page, its note names the offset its marker does. + const two = (await read({ limit: 2 })).data as LinePage; + assert.ok(two.note!.startsWith(`Truncated at 2 lines (file has 3). Line 2 is cut: read its rest with offsetChars=${at}. Use offset/limit to read more.`), two.note); }); test('a line page stops before its result would pass the limit, and says where to continue', async t => { @@ -126,11 +129,65 @@ test('a first line that alone would pass the limit is cut where it fits', async assert.ok(shown.length > 0); assert.equal(Number(at), shown.length); assert.equal(shown.length + Number(rest), 1500); - assert.match(page.note!, /^Stopped at line 1 to stay within the 1000-character result limit \(file has 2\)\. Continue with offset=2\.$/); + // Reading on in order means the rest of line 1 first, so the note names it. + assert.equal(page.note, `Stopped at line 1 to stay within the 1000-character result limit (file has 2). Line 1 is cut: read its rest with offsetChars=${at}, then continue with offset=2.`); + // The cut is the most the page has room for: one character more passes the + // limit, on a page with a note and on one that ends the file with none. + const oneMore = (cutPage: LinePage, chars: number) => { + const more = chars + 1; + return JSON.stringify({ + ...cutPage, + ...(cutPage.note !== undefined ? { note: cutPage.note.replace(/offsetChars=\d+/, `offsetChars=${more}`) } : {}), + content: ` 1\t${'"'.repeat(more)} [… ${1500 - more} more chars; read them with offsetChars=${more}]`, + }).length; + }; + assert.ok(oneMore(page, shown.length) > 1_000, `${oneMore(page, shown.length)}`); + const { read: readOneLine } = await setup(t, '"'.repeat(1500)); + const last = await readOneLine({}, 1_000); + assert.ok(stored(last) <= 1_000, `${stored(last)}`); + const lastPage = last.data as LinePage; + assert.equal(lastPage.note, undefined); + const lastShown = /^ {5}1\t("*) \[/.exec(lastPage.content)![1].length; + assert.ok(oneMore(lastPage, lastShown) > 1_000, `${oneMore(lastPage, lastShown)}`); + // Shown whole, a line carries no marker and its note no cut, so its page can + // fit where a cut one character shorter doesn't: it is shown whole. + const { read: readFits } = await setup(t, 'x'.repeat(900)); + const fitting = await readFits({}, 1_000); + assert.ok(stored(fitting) <= 1_000, `${stored(fitting)}`); + assert.deepEqual(fitting.data, { path: 'work/text.txt', totalLines: 1, fromLine: 1, toLine: 1, content: ` 1\t${'x'.repeat(900)}` }); // A limit with no room beside the page's own fields still shows where the line's text is. const tiny = (await read({}, 50)).data as LinePage; assert.equal(tiny.toLine, 1); assert.equal(tiny.content, ' 1\t [… 1500 more chars; read them with offsetChars=0]'); + assert.match(tiny.note!, /Line 1 is cut: read its rest with offsetChars=0, then continue with offset=2\.$/); +}); + +test('a page whose last line is cut names where its rest starts before the next line', async t => { + const long = 'z'.repeat(3000); + const { read } = await setup(t, `short\n${long}\n${long}\n${long}\n`); + const at = (line: number) => 'short\n'.length + (line - 2) * (long.length + 1) + 2000; + // The limit stops the page at the cut line. + const byLimit = (await read({ limit: 2 })).data as LinePage; + assert.equal(byLimit.toLine, 2); + assert.equal(byLimit.note, `Truncated at 2 lines (file has 4). Line 2 is cut: read its rest with offsetChars=${at(2)}. Use offset/limit to read more.`); + // The result limit stops it there: line 4 would pass 5,000 characters. + const byBudget = (await read({}, 5_000)).data as LinePage; + assert.equal(byBudget.toLine, 3); + assert.equal(byBudget.note, `Stopped at line 3 to stay within the 5000-character result limit (file has 4). Line 3 is cut: read its rest with offsetChars=${at(3)}, then continue with offset=4.`); + // A cut line inside a page has its rest named by its own marker alone, and + // a page that ends the file has no next line to name. + const whole = (await read({}, 24_000)).data as LinePage; + assert.equal(whole.toLine, 4); + assert.equal(whole.note, undefined); +}); + +test('a call that carries no resultMaxChars gets pages held to the framework\'s default, 24,000', async t => { + const { read } = await setup(t, Array.from({ length: 3000 }, (_, i) => `line ${i + 1} of a file long enough to pass the default`).join('\n')); + const result = await read(); + assert.ok(stored(result) <= 24_000, `${stored(result)}`); + const page = result.data as LinePage; + assert.ok(page.toLine! < 2000); + assert.match(page.note!, /^Stopped at line \d+ to stay within the 24000-character result limit/); }); test('a character page stops at the result limit too, with nextOffsetChars', async t => { @@ -160,7 +217,9 @@ test('a character page stops at the result limit too, with nextOffsetChars', asy // A surrogate pair is never split at the stop, whatever the limit leaves room for. const { read: readEmoji } = await setup(t, '😀'.repeat(20_000)); for (let limit = 1_000; limit < 1_004; limit++) { - const page = (await readEmoji({ offsetChars: 0, limitChars: 100_000 }, limit)).data as CharPage; + const result = await readEmoji({ offsetChars: 0, limitChars: 100_000 }, limit); + assert.ok(stored(result) <= limit, `${stored(result)} > ${limit}`); + const page = result.data as CharPage; assert.equal(page.nextOffsetChars! % 2, 0, `at ${limit}`); assert.equal(page.content.length % 2, 0, `at ${limit}`); } From c00b269ed8a1ec4e48b9959bc787ed17a0983f4a Mon Sep 17 00:00:00 2001 From: Abigail-2676 Date: Sat, 17 Oct 2026 19:49:24 -0400 Subject: [PATCH 05/51] docs(workspace): grep follows ripgrep's --crlf in matching a CRLF line, not in showing it The comment above grep's search and the changelog said grep leaves out the CR before a line feed for matching as well as display "as ripgrep's --crlf mode does". ripgrep 14.1.0's --crlf lets `foo$` find `foo\r\n`, but it prints the line with its CR and refuses a `\r` in a pattern, so only the matching is ripgrep's. Both now say so. The changelog also says "no note" where it said "no word", and says what the framework did with a page past the limit: it kept only the start, cut mid-JSON, and wrote the whole result to a workspace file where the agent has one. Co-Authored-By: Claude Opus 5.5 --- changelog.d/feat-workspace-read.changed.md | 6 +++--- src/modules/workspace/index.ts | 5 +++-- 2 files changed, 6 insertions(+), 5 deletions(-) diff --git a/changelog.d/feat-workspace-read.changed.md b/changelog.d/feat-workspace-read.changed.md index 2614eef15..5f2265af6 100644 --- a/changelog.d/feat-workspace-read.changed.md +++ b/changelog.d/feat-workspace-read.changed.md @@ -1,6 +1,6 @@ -- The workspace's `read` shows a file's text as it is, and says so. A line page shows a CRLF file's lines without the CR before each line feed, and leaves out a BOM. Its note says when the file has either, counting a mix of CRLF and LF ends. It also says when the file isn't valid UTF-8: each invalid byte sequence shows as U+FFFD, `edit` refuses the file, and `write` replaces it whole. Before, each line of a CRLF file ended in a CR, a BOM came first in the first line as U+FEFF, and invalid bytes showed as U+FFFD with no word. A character page still reads the stored text as it is, BOM and CRs included, and the tool's description now says so. +- The workspace's `read` shows a file's text as it is, and says so. A line page shows a CRLF file's lines without the CR before each line feed, and leaves out a BOM. Its note says when the file has either, counting a mix of CRLF and LF ends. It also says when the file isn't valid UTF-8: each invalid byte sequence shows as U+FFFD, `edit` refuses the file, and `write` replaces it whole. Before, each line of a CRLF file ended in a CR, a BOM came first in the first line as U+FEFF, and invalid bytes showed as U+FFFD with no note. A character page still reads the stored text as it is, BOM and CRs included, and the tool's description now says so. - A line over 2,000 characters is cut, with how many characters follow and the `offsetChars` that reads them. Before, a line page showed every line whole, however long. -- A `read` page stops before its result would pass `resultMaxChars`, the most of it the framework stores whole. For a model's call that's its agent's inline cap, 24,000 characters by default, and a call that carries none gets 24,000 too. Its note says where to continue: the `offset` after its last line, or for a character page the `offsetChars` it also gives as `nextOffsetChars`. Where even a page's first line would pass the limit, that line is cut shorter, at the most the page has room for. Where a page's last line is cut, its note names the `offsetChars` where that line's rest starts before the next line's `offset`. A page always moves on, so where its own fields (the path and its notes) and the least it can show pass a small limit, it passes the limit: at 1,000 characters, only a path of about 130 characters or more beside every note a read can carry does that, or one of about 700 with none. Before, a page of up to 2,000 lines, or of `limitChars` characters, went out whole, and the framework wrote what passed the limit to a file, cut mid-JSON. A caller that pages 2,000 lines at a time and starts the next page at line 2,001 now starts where the note says. A running `code_execution` script's call has a limit of 5,000,000 characters, so its pages still stop at 2,000 lines or `limitChars`, unless a page would pass that. -- The workspace's `grep` leaves out a CR before a line feed, for matching as well as display, as ripgrep's `--crlf` mode does, so `foo$` finds a CRLF file's line too. A match's text and context are the lines as `read` shows them, a BOM isn't part of the first line, and a file with CRLF ends says so in its result's `lineEndings` (`CRLF`, or `mixed`). Before, a CRLF file's lines ended in a CR, so `foo$` missed them and a match's text copied into an edit carried the CR. A pattern written to find the CR of a CRLF line end no longer matches; a CR that doesn't come before a line feed is still part of its line. +- A `read` page stops before its result would pass `resultMaxChars`, the most of it the framework stores whole. For a model's call that's its agent's inline cap, 24,000 characters by default, and a call that carries none gets 24,000 too. Its note says where to continue: the `offset` after its last line, or for a character page the `offsetChars` it also gives as `nextOffsetChars`. Where even a page's first line would pass the limit, that line is cut shorter, at the most the page has room for. Where a page's last line is cut, its note names the `offsetChars` where that line's rest starts before the next line's `offset`. A page always moves on, so where its own fields (the path and its notes) and the least it can show pass a small limit, it passes the limit: at 1,000 characters, only a path of about 130 characters or more beside every note a read can carry does that, or one of about 700 with none. Before, a page of up to 2,000 lines, or of `limitChars` characters, went out whole, and for a result that passed the limit the framework kept only its start, cut mid-JSON, writing the whole of it to a workspace file where the agent has one. A caller that pages 2,000 lines at a time and starts the next page at line 2,001 now starts where the note says. A running `code_execution` script's call has a limit of 5,000,000 characters, so its pages still stop at 2,000 lines or `limitChars`, unless a page would pass that. +- The workspace's `grep` leaves out a CR before a line feed, for matching as well as display, so `foo$` finds a CRLF file's line too, as it does under ripgrep's `--crlf`. A match's text and context are the lines as `read` shows them, a BOM isn't part of the first line, and a file with CRLF ends says so in its result's `lineEndings` (`CRLF`, or `mixed`). Before, a CRLF file's lines ended in a CR, so `foo$` missed them and a match's text copied into an edit carried the CR. A pattern written to find the CR of a CRLF line end no longer matches; a CR that doesn't come before a line feed is still part of its line. - The workspace's `read` first takes in a change on disk to a file the workspace holds, as a listing and an `edit` do, and shows that text. Its note says so when disk's copy had changed since the workspace last agreed with it. It also says when disk's copy couldn't be checked, or disk has none and nothing records whether it was deleted there, so it shows the workspace's copy. A file disk deleted answers that disk deleted it and the deletion was taken in, and a file disk made binary or too large answers that it's on disk but not stored. Where the path's stored copy is a draft and disk changed or deleted it too, the read shows the draft, and its note names the conflict and both ways to settle it: for a deletion, a sync of the path takes it and removes the workspace's copy. Before, a read of a path the workspace held didn't look at disk, so it could show a copy another writer had since changed; a path it didn't hold was observed, as now. - The `read` and `grep` tool descriptions changed, so the first request after the upgrade that carries the workspace's tools misses the prompt cache from its tool block on. diff --git a/src/modules/workspace/index.ts b/src/modules/workspace/index.ts index 055877105..80c3d51fa 100644 --- a/src/modules/workspace/index.ts +++ b/src/modules/workspace/index.ts @@ -3422,8 +3422,9 @@ export class WorkspaceModule implements Module { const incomplete: Array<{ path: string; reason: string }> = []; // Lines as read shows them: a CR before a line feed is left out, for - // matching as well as display, as ripgrep's --crlf does, so `foo$` finds - // a CRLF file's line too, and the line copies into an edit as read's do. + // matching as well as display, so `foo$` finds a CRLF file's line, as it + // does under ripgrep's --crlf, and the line copies into an edit as read's + // do. ripgrep's --crlf still prints the line with its CR. const search = (bytes: Buffer): Searched => { const { lines, crlfEnds, lfEnds } = fileText(bytes); const found: GrepMatch[] = []; From 0a4303ac2303a2610aef3341a07a82088073210a Mon Sep 17 00:00:00 2001 From: Abigail-2676 Date: Sat, 17 Oct 2026 21:14:54 -0400 Subject: [PATCH 06/51] fix(workspace): a read whose disk check gives up says the check didn't finish When the selected branch keeps changing while a read's pass gathers, the pass decides nothing: it leaves no report for the path, only an incomplete reason, "the selected branch kept changing while the scan ran". The read then showed the stored copy with no note, or, for a path the workspace doesn't hold, a plain "File not found", so the caller couldn't tell that its disk check hadn't finished. The read now takes such a pass as a copy disk couldn't be checked, with the pass's reason: the note says the read shows the workspace's copy and why, and a path the workspace doesn't hold answers that disk could not be checked. Tests, in test/workspace-three-way.test.ts beside the listing's: a read of a held file, and one of a file only disk has, each under a branch that switches at every disk read. Co-Authored-By: Claude Opus 5.5 --- src/modules/workspace/index.ts | 6 +++++- test/workspace-three-way.test.ts | 24 ++++++++++++++++++++++++ 2 files changed, 29 insertions(+), 1 deletion(-) diff --git a/src/modules/workspace/index.ts b/src/modules/workspace/index.ts index 80c3d51fa..d6aa9b95f 100644 --- a/src/modules/workspace/index.ts +++ b/src/modules/workspace/index.ts @@ -2695,7 +2695,11 @@ export class WorkspaceModule implements Module { // uses, as an edit's is. Where the stored copy is a draft and disk // changed too, the read shows the draft and names the conflict. const pass = await this.passUnlocked(mount, { kind: 'paths', paths: [relativePath] }); - const report = pass.reports.get(relativePath); + // A pass that gave up (the selected branch kept changing) decided + // nothing and leaves no report. It's read as a copy disk couldn't be + // checked, with its reason, so the answer says the check didn't finish. + const report = pass.reports.get(relativePath) + ?? (pass.incomplete.length > 0 ? { path: relativePath, state: 'unverified' as const, note: pass.incomplete[0]!.reason } : undefined); const took = (op: TreeOp) => pass.ops.some((o) => o.path === relativePath && o.op === op); const entry = store.treeGet(mount.treeStateId, relativePath); if (!entry) { diff --git a/test/workspace-three-way.test.ts b/test/workspace-three-way.test.ts index 67335c2bf..919d3233a 100644 --- a/test/workspace-three-way.test.ts +++ b/test/workspace-three-way.test.ts @@ -6922,6 +6922,30 @@ describe('on-agent-action', () => { assert.deepEqual(data.incomplete, [{ path: '', reason: 'the selected branch kept changing while the scan ran' }]); }); + test("a read whose disk check gives up on a branch that keeps changing says it shows the workspace's copy, and why", async (t) => { + const env = new Env(t); + const m = await env.open(); + await seedSynced(env, m, 'a.txt', 'A'); + const main = env.store.currentBranch().name; + env.store.createBranch('other', main); + const data = await withReadHook((n) => env.store.switchBranch(n % 2 === 1 ? 'other' : main), + () => call(m, 'read', { path: 'work/a.txt' })); + assert.equal(data.content, ' 1\tA'); + assert.equal(data.note, "The path couldn't be checked against disk (the selected branch kept changing while the scan ran), so this read shows the workspace's copy."); + }); + + test("a read of a file the workspace doesn't hold, whose disk check gives up, says disk couldn't be checked", async (t) => { + const env = new Env(t); + const m = await env.open(); + await seedSynced(env, m, 'a.txt', 'A'); + env.writeDisk('b.txt', 'B'); + const main = env.store.currentBranch().name; + env.store.createBranch('other', main); + const error = await withReadHook((n) => env.store.switchBranch(n % 2 === 1 ? 'other' : main), + () => refused(m, 'read', { path: 'work/b.txt' })); + assert.equal(error, 'work/b.txt is not in the workspace, and disk could not be checked (the selected branch kept changing while the scan ran).'); + }); + test('a branch switched away during the root check and back during a read is decided on the branch enumerated', async (t) => { const env = new Env(t); const m = await env.open(); From 83aae288126d86913fac71dc345b658ff0e9e4c4 Mon Sep 17 00:00:00 2001 From: Abigail-2676 Date: Sat, 17 Oct 2026 23:27:56 -0400 Subject: [PATCH 07/51] fix(workspace): the page that ends a file names its cut last line's rest Review finding R1-3 on #465: on the page that ends the file, the note was null even when the page's last line was cut, so a page whose toLine equals totalLines read as complete while its last line held a cut marker. A program that checks toLine < totalLines took the file as read whole. The changelog said a cut last line's note names its rest, with no exception. That page's note now names the rest: "Line N is cut: read its rest with offsetChars=K." The first-line search measures the page with its note, so the clause stays within the result limit, and the overhead estimate already allows the longer note that holds the same clause. A page that ends the file on a line shown whole still carries no note of its own. Tests, in test/workspace-read-text.test.ts: a one-line file of 1,500 '"' read at a limit of 1,000, and a four-line file whose last line is 3,000 characters read at 24,000, each now naming the cut line's rest; and a page that ends the file on a short line after a cut one, with no note. Co-Authored-By: Claude Opus 5.5 --- changelog.d/feat-workspace-read.changed.md | 2 +- src/modules/workspace/read-page.ts | 10 ++++++---- test/workspace-read-text.test.ts | 16 +++++++++++----- 3 files changed, 18 insertions(+), 10 deletions(-) diff --git a/changelog.d/feat-workspace-read.changed.md b/changelog.d/feat-workspace-read.changed.md index 5f2265af6..0acc0be24 100644 --- a/changelog.d/feat-workspace-read.changed.md +++ b/changelog.d/feat-workspace-read.changed.md @@ -1,6 +1,6 @@ - The workspace's `read` shows a file's text as it is, and says so. A line page shows a CRLF file's lines without the CR before each line feed, and leaves out a BOM. Its note says when the file has either, counting a mix of CRLF and LF ends. It also says when the file isn't valid UTF-8: each invalid byte sequence shows as U+FFFD, `edit` refuses the file, and `write` replaces it whole. Before, each line of a CRLF file ended in a CR, a BOM came first in the first line as U+FEFF, and invalid bytes showed as U+FFFD with no note. A character page still reads the stored text as it is, BOM and CRs included, and the tool's description now says so. - A line over 2,000 characters is cut, with how many characters follow and the `offsetChars` that reads them. Before, a line page showed every line whole, however long. -- A `read` page stops before its result would pass `resultMaxChars`, the most of it the framework stores whole. For a model's call that's its agent's inline cap, 24,000 characters by default, and a call that carries none gets 24,000 too. Its note says where to continue: the `offset` after its last line, or for a character page the `offsetChars` it also gives as `nextOffsetChars`. Where even a page's first line would pass the limit, that line is cut shorter, at the most the page has room for. Where a page's last line is cut, its note names the `offsetChars` where that line's rest starts before the next line's `offset`. A page always moves on, so where its own fields (the path and its notes) and the least it can show pass a small limit, it passes the limit: at 1,000 characters, only a path of about 130 characters or more beside every note a read can carry does that, or one of about 700 with none. Before, a page of up to 2,000 lines, or of `limitChars` characters, went out whole, and for a result that passed the limit the framework kept only its start, cut mid-JSON, writing the whole of it to a workspace file where the agent has one. A caller that pages 2,000 lines at a time and starts the next page at line 2,001 now starts where the note says. A running `code_execution` script's call has a limit of 5,000,000 characters, so its pages still stop at 2,000 lines or `limitChars`, unless a page would pass that. +- A `read` page stops before its result would pass `resultMaxChars`, the most of it the framework stores whole. For a model's call that's its agent's inline cap, 24,000 characters by default, and a call that carries none gets 24,000 too. Its note says where to continue: the `offset` after its last line, or for a character page the `offsetChars` it also gives as `nextOffsetChars`. Where even a page's first line would pass the limit, that line is cut shorter, at the most the page has room for. Where a page's last line is cut, its note names the `offsetChars` where that line's rest starts, before the next line's `offset` if the file goes on. A page always moves on, so where its own fields (the path and its notes) and the least it can show pass a small limit, it passes the limit: at 1,000 characters, only a path of about 130 characters or more beside every note a read can carry does that, or one of about 700 with none. Before, a page of up to 2,000 lines, or of `limitChars` characters, went out whole, and for a result that passed the limit the framework kept only its start, cut mid-JSON, writing the whole of it to a workspace file where the agent has one. A caller that pages 2,000 lines at a time and starts the next page at line 2,001 now starts where the note says. A running `code_execution` script's call has a limit of 5,000,000 characters, so its pages still stop at 2,000 lines or `limitChars`, unless a page would pass that. - The workspace's `grep` leaves out a CR before a line feed, for matching as well as display, so `foo$` finds a CRLF file's line too, as it does under ripgrep's `--crlf`. A match's text and context are the lines as `read` shows them, a BOM isn't part of the first line, and a file with CRLF ends says so in its result's `lineEndings` (`CRLF`, or `mixed`). Before, a CRLF file's lines ended in a CR, so `foo$` missed them and a match's text copied into an edit carried the CR. A pattern written to find the CR of a CRLF line end no longer matches; a CR that doesn't come before a line feed is still part of its line. - The workspace's `read` first takes in a change on disk to a file the workspace holds, as a listing and an `edit` do, and shows that text. Its note says so when disk's copy had changed since the workspace last agreed with it. It also says when disk's copy couldn't be checked, or disk has none and nothing records whether it was deleted there, so it shows the workspace's copy. A file disk deleted answers that disk deleted it and the deletion was taken in, and a file disk made binary or too large answers that it's on disk but not stored. Where the path's stored copy is a draft and disk changed or deleted it too, the read shows the draft, and its note names the conflict and both ways to settle it: for a deletion, a sync of the path takes it and removes the workspace's copy. Before, a read of a path the workspace held didn't look at disk, so it could show a copy another writer had since changed; a path it didn't hold was observed, as now. - The `read` and `grep` tool descriptions changed, so the first request after the upgrade that carries the workspace's tools misses the prompt cache from its tool block on. diff --git a/src/modules/workspace/read-page.ts b/src/modules/workspace/read-page.ts index 2d10d5898..73358ae07 100644 --- a/src/modules/workspace/read-page.ts +++ b/src/modules/workspace/read-page.ts @@ -37,9 +37,10 @@ const joined = (notes: Array): string | undefined => { * one more line would take its data past `maxChars` as JSON; if even the * first line doesn't fit, that line is cut shorter, at the most the page has * room for. Where the page's last line is cut, its note names where that - * line's rest starts before the next line, since reading on in order starts - * there. A page whose own fields and least content pass `maxChars` still - * moves on, and passes it. + * line's rest starts, since reading on in order starts there: before the next + * line, or, on the page that ends the file, before the file is read whole. A + * page whose own fields and least content pass `maxChars` still moves on, and + * passes it. */ export function linePage(path: string, file: FileText, offset: number, limit: number, maxChars: number, extraNotes: string[] = []): ToolResult { const { lines, starts } = file; @@ -93,7 +94,8 @@ export function linePage(path: string, file: FileText, offset: number, limit: nu const final = (lastMax = MAX_LINE_CHARS) => { const to = first + shownLines.length; const rest = restOf(to - 1, lastMax); - return page(to, shownLines.join('\n'), joined([to === last ? (to < total ? limitNote(to, rest) : null) : budgetNote(to, rest), ...notes])); + const endNote = to < total ? limitNote(to, rest) : rest === null ? null : `${cutLine(to, rest)}.`; + return page(to, shownLines.join('\n'), joined([to === last ? endNote : budgetNote(to, rest), ...notes])); }; if (shownLines.length === 0) { // The estimate above found no room for the first line as a page shows it. diff --git a/test/workspace-read-text.test.ts b/test/workspace-read-text.test.ts index 16314232e..494c74f20 100644 --- a/test/workspace-read-text.test.ts +++ b/test/workspace-read-text.test.ts @@ -132,7 +132,7 @@ test('a first line that alone would pass the limit is cut where it fits', async // Reading on in order means the rest of line 1 first, so the note names it. assert.equal(page.note, `Stopped at line 1 to stay within the 1000-character result limit (file has 2). Line 1 is cut: read its rest with offsetChars=${at}, then continue with offset=2.`); // The cut is the most the page has room for: one character more passes the - // limit, on a page with a note and on one that ends the file with none. + // limit, on a page that goes on and on one that ends the file. const oneMore = (cutPage: LinePage, chars: number) => { const more = chars + 1; return JSON.stringify({ @@ -146,8 +146,10 @@ test('a first line that alone would pass the limit is cut where it fits', async const last = await readOneLine({}, 1_000); assert.ok(stored(last) <= 1_000, `${stored(last)}`); const lastPage = last.data as LinePage; - assert.equal(lastPage.note, undefined); const lastShown = /^ {5}1\t("*) \[/.exec(lastPage.content)![1].length; + // The page ends the file, and its last line is cut, so its note names the rest. + assert.equal(lastPage.toLine, lastPage.totalLines); + assert.equal(lastPage.note, `Line 1 is cut: read its rest with offsetChars=${lastShown}.`); assert.ok(oneMore(lastPage, lastShown) > 1_000, `${oneMore(lastPage, lastShown)}`); // Shown whole, a line carries no marker and its note no cut, so its page can // fit where a cut one character shorter doesn't: it is shown whole. @@ -174,11 +176,15 @@ test('a page whose last line is cut names where its rest starts before the next const byBudget = (await read({}, 5_000)).data as LinePage; assert.equal(byBudget.toLine, 3); assert.equal(byBudget.note, `Stopped at line 3 to stay within the 5000-character result limit (file has 4). Line 3 is cut: read its rest with offsetChars=${at(3)}, then continue with offset=4.`); - // A cut line inside a page has its rest named by its own marker alone, and - // a page that ends the file has no next line to name. + // A cut line inside a page has its rest named by its own marker alone. The + // page that ends the file names its cut last line too: toLine is the last + // line, but the file isn't read whole until that line's rest is. const whole = (await read({}, 24_000)).data as LinePage; assert.equal(whole.toLine, 4); - assert.equal(whole.note, undefined); + assert.equal(whole.note, `Line 4 is cut: read its rest with offsetChars=${at(4)}.`); + // A page that ends the file on a line shown whole has nothing to say. + const { read: readShort } = await setup(t, `${long}\nshort\n`); + assert.equal(((await readShort({}, 24_000)).data as LinePage).note, undefined); }); test('a call that carries no resultMaxChars gets pages held to the framework\'s default, 24,000', async t => { From cd7f83b838b12008ca2ddaaf5946466004f035b5 Mon Sep 17 00:00:00 2001 From: Abigail-2676 Date: Sat, 17 Oct 2026 23:28:09 -0400 Subject: [PATCH 08/51] fix(workspace): a read made in process pages by its line limit alone, and the changelog says how a script's line pages read Review finding R1-2 on #465: a read whose call carries no resultMaxChars was held to 24,000 characters. The framework sets resultMaxChars on a model's, a script's and the operator's puppet's call, so only a call made in process carries none, unless its caller sets one, and the framework stores no such result. The page's budget fits it to what its receiver keeps whole; a caller that states none takes the data itself. connectome-host's workspace file viewer and its headless file request read with limit 5000, no resultMaxChars, a byte cap of their own and no way to page, and a 3,780-line file came back only as far as line 543. Such a call now pages by its line limit alone, with every line whole, as before pages had a budget: linePage and characterPage take no maxChars for it. CRs and a BOM are left out of its line pages as of any other. ToolCall.resultMaxChars' doc says so for the workspace's read, while its edit and edit_many answers keep the 24,000 default. The changelog names in-process callers. Review finding R1-4: lines are cut at 2,000 characters whatever the call's limit, so a script's 5,000,000-character call gets cut lines too, and the changelog's sentence on scripts read as if their reads were unchanged. Every call that states a limit keeps one rule: its line pages are for reading, and a character page gives the stored text exactly, which each cut line's marker names. The changelog now says so for scripts. Tests, in test/workspace-read-text.test.ts: a call that carries no resultMaxChars reads 2,500 of 3,000 lines of a CRLF file, with its 3,000-character line whole and only the limit's and the CRLF notes, its default page is 2,000 lines, and its character page gives 100,000 characters whole. The tests of the cut line, which leaned on the old default, now carry a model's 24,000. Co-Authored-By: Claude Opus 5.5 --- changelog.d/feat-workspace-read.changed.md | 4 +-- src/modules/workspace/index.ts | 15 +++++---- src/modules/workspace/read-page.ts | 15 ++++++--- src/types/events.ts | 6 ++-- test/workspace-read-text.test.ts | 38 +++++++++++++++------- 5 files changed, 52 insertions(+), 26 deletions(-) diff --git a/changelog.d/feat-workspace-read.changed.md b/changelog.d/feat-workspace-read.changed.md index 0acc0be24..1961babd1 100644 --- a/changelog.d/feat-workspace-read.changed.md +++ b/changelog.d/feat-workspace-read.changed.md @@ -1,6 +1,6 @@ - The workspace's `read` shows a file's text as it is, and says so. A line page shows a CRLF file's lines without the CR before each line feed, and leaves out a BOM. Its note says when the file has either, counting a mix of CRLF and LF ends. It also says when the file isn't valid UTF-8: each invalid byte sequence shows as U+FFFD, `edit` refuses the file, and `write` replaces it whole. Before, each line of a CRLF file ended in a CR, a BOM came first in the first line as U+FEFF, and invalid bytes showed as U+FFFD with no note. A character page still reads the stored text as it is, BOM and CRs included, and the tool's description now says so. -- A line over 2,000 characters is cut, with how many characters follow and the `offsetChars` that reads them. Before, a line page showed every line whole, however long. -- A `read` page stops before its result would pass `resultMaxChars`, the most of it the framework stores whole. For a model's call that's its agent's inline cap, 24,000 characters by default, and a call that carries none gets 24,000 too. Its note says where to continue: the `offset` after its last line, or for a character page the `offsetChars` it also gives as `nextOffsetChars`. Where even a page's first line would pass the limit, that line is cut shorter, at the most the page has room for. Where a page's last line is cut, its note names the `offsetChars` where that line's rest starts, before the next line's `offset` if the file goes on. A page always moves on, so where its own fields (the path and its notes) and the least it can show pass a small limit, it passes the limit: at 1,000 characters, only a path of about 130 characters or more beside every note a read can carry does that, or one of about 700 with none. Before, a page of up to 2,000 lines, or of `limitChars` characters, went out whole, and for a result that passed the limit the framework kept only its start, cut mid-JSON, writing the whole of it to a workspace file where the agent has one. A caller that pages 2,000 lines at a time and starts the next page at line 2,001 now starts where the note says. A running `code_execution` script's call has a limit of 5,000,000 characters, so its pages still stop at 2,000 lines or `limitChars`, unless a page would pass that. +- On a page for a call that carries a `resultMaxChars`, such as a model's or a running script's, a line over 2,000 characters is cut, with how many characters follow and the `offsetChars` that reads them. Before, a line page showed every line whole, however long. +- A `read` page stops before its result would pass `resultMaxChars`, the most of it the framework stores whole. For a model's call that's its agent's inline cap, 24,000 characters by default. Its note says where to continue: the `offset` after its last line, or for a character page the `offsetChars` it also gives as `nextOffsetChars`. Where even a page's first line would pass the limit, that line is cut shorter, at the most the page has room for. Where a page's last line is cut, its note names the `offsetChars` where that line's rest starts, before the next line's `offset` if the file goes on. A page always moves on, so where its own fields (the path and its notes) and the least it can show pass a small limit, it passes the limit: at 1,000 characters, only a path of about 130 characters or more beside every note a read can carry does that, or one of about 700 with none. Before, a page of up to 2,000 lines, or of `limitChars` characters, went out whole, and for a result that passed the limit the framework kept only its start, cut mid-JSON, writing the whole of it to a workspace file where the agent has one. A caller that pages 2,000 lines at a time and starts the next page at line 2,001 now starts where the note says. A running `code_execution` script's call has a limit of 5,000,000 characters, so its pages still stop at 2,000 lines or `limitChars`, unless a page would pass that. Its line pages cut a line over 2,000 characters and leave out CRs and a BOM, as a model's do, and a character page gives it the stored text exactly. A call made in process, through `handleToolCall`, `executeToolCall` or `ModuleContext.callTool`, carries no `resultMaxChars` unless its caller sets one. Its line pages stop at the line limit alone and show every line whole, as they did before, though now without the CRs and the BOM. - The workspace's `grep` leaves out a CR before a line feed, for matching as well as display, so `foo$` finds a CRLF file's line too, as it does under ripgrep's `--crlf`. A match's text and context are the lines as `read` shows them, a BOM isn't part of the first line, and a file with CRLF ends says so in its result's `lineEndings` (`CRLF`, or `mixed`). Before, a CRLF file's lines ended in a CR, so `foo$` missed them and a match's text copied into an edit carried the CR. A pattern written to find the CR of a CRLF line end no longer matches; a CR that doesn't come before a line feed is still part of its line. - The workspace's `read` first takes in a change on disk to a file the workspace holds, as a listing and an `edit` do, and shows that text. Its note says so when disk's copy had changed since the workspace last agreed with it. It also says when disk's copy couldn't be checked, or disk has none and nothing records whether it was deleted there, so it shows the workspace's copy. A file disk deleted answers that disk deleted it and the deletion was taken in, and a file disk made binary or too large answers that it's on disk but not stored. Where the path's stored copy is a draft and disk changed or deleted it too, the read shows the draft, and its note names the conflict and both ways to settle it: for a deletion, a sync of the path takes it and removes the workspace's copy. Before, a read of a path the workspace held didn't look at disk, so it could show a copy another writer had since changed; a path it didn't hold was observed, as now. - The `read` and `grep` tool descriptions changed, so the first request after the upgrade that carries the workspace's tools misses the prompt cache from its tool block on. diff --git a/src/modules/workspace/index.ts b/src/modules/workspace/index.ts index d6aa9b95f..2a717011c 100644 --- a/src/modules/workspace/index.ts +++ b/src/modules/workspace/index.ts @@ -1602,8 +1602,9 @@ export class WorkspaceModule implements Module { } const input = call.input as Record; // The most of this call's result the framework stores before it spills - // the rest into a file; a call it didn't dispatch gets the default. - const resultMaxChars = call.resultMaxChars ?? DEFAULT_TOOL_RESULT_INLINE_MAX_CHARS; + // the rest into a file. A call made in process carries none and takes + // the data itself, so its read pages by its line limit alone. + const resultMaxChars = call.resultMaxChars; const generated = typeof input?.path === 'string' ? this.findGeneratedTextFile(input.path) : undefined; if (generated) { if (call.name === 'read' && call.callerAgentName === generated.agentName) { @@ -2674,9 +2675,11 @@ export class WorkspaceModule implements Module { /** * A read's page, held to `maxChars`, the most the framework stores of this - * call's result before it spills the rest into a file. + * call's result before it spills the rest into a file. A call that carries + * none, made in process, takes the data itself: its page holds to its line + * limit alone, with every line whole, as it did before pages had a budget. */ - private async handleRead(input: ReadInput, maxChars: number, generatedContent?: string): Promise { + private async handleRead(input: ReadInput, maxChars: number | undefined, generatedContent?: string): Promise { const characterPaging = input.offsetChars !== undefined || input.limitChars !== undefined; // Each number is a whole number of at least its schema's minimum by now // (inputProblem). @@ -2723,8 +2726,8 @@ export class WorkspaceModule implements Module { } // An unlimited read of a large file would inject the whole thing into the // turn (a 2MB file ≈ 600k tokens → the request blows past the model - // context and 400s), so an omitted limit falls back to a default, and - // every page stops at the result limit. + // context and 400s), so an omitted limit falls back to a default, and a + // page stops at the call's result limit where it has one. return characterPaging ? characterPage(input.path, file, input.offsetChars ?? 0, input.limitChars ?? 2000, maxChars, notes) : linePage(input.path, file, input.offset ?? 1, input.limit ?? DEFAULT_READ_LINE_LIMIT, maxChars, notes); diff --git a/src/modules/workspace/read-page.ts b/src/modules/workspace/read-page.ts index 73358ae07..35b8fe3c1 100644 --- a/src/modules/workspace/read-page.ts +++ b/src/modules/workspace/read-page.ts @@ -40,9 +40,10 @@ const joined = (notes: Array): string | undefined => { * line's rest starts, since reading on in order starts there: before the next * line, or, on the page that ends the file, before the file is read whole. A * page whose own fields and least content pass `maxChars` still moves on, and - * passes it. + * passes it. With no `maxChars`, for a call made in process, which takes the + * data itself, the page holds to `limit` alone and shows every line whole. */ -export function linePage(path: string, file: FileText, offset: number, limit: number, maxChars: number, extraNotes: string[] = []): ToolResult { +export function linePage(path: string, file: FileText, offset: number, limit: number, maxChars: number | undefined, extraNotes: string[] = []): ToolResult { const { lines, starts } = file; const total = lines.length; const first = offset - 1; @@ -73,6 +74,10 @@ export function linePage(path: string, file: FileText, offset: number, limit: nu const page = (to: number, content: string, note: string | undefined) => ({ path, totalLines: total, fromLine: first + 1, toLine: to, ...(note !== undefined ? { note } : {}), content, }); + if (maxChars === undefined) { + const whole = lines.slice(first, last).map((line, k) => numberedLine(line, first + k + 1, Infinity)).join('\n'); + return { success: true, data: page(last, whole, joined([last < total ? limitNote(last, null) : null, ...notes])) }; + } // The most anything but the lines can take: the longer note, with the // largest numbers this page could show. const overhead = Math.max( @@ -144,9 +149,9 @@ const splitsPair = (text: string, at: number): boolean => /** * Raw text from `offsetChars`, at most `limitChars` code units (one more to * keep a surrogate pair whole), stopped sooner where its data would pass - * `maxChars` as JSON. + * `maxChars` as JSON, if there is one. */ -export function characterPage(path: string, file: FileText, offsetChars: number, limitChars: number, maxChars: number, extraNotes: string[] = []): ToolResult { +export function characterPage(path: string, file: FileText, offsetChars: number, limitChars: number, maxChars: number | undefined, extraNotes: string[] = []): ToolResult { const text = file.raw; const start = Math.min(offsetChars, text.length); if (splitsPair(text, start)) { @@ -166,7 +171,7 @@ export function characterPage(path: string, file: FileText, offsetChars: number, content: text.slice(start, to), }); const whole = page(end, joined(notes)); - if (JSON.stringify(whole).length <= maxChars) return { success: true, data: whole }; + if (maxChars === undefined || JSON.stringify(whole).length <= maxChars) return { success: true, data: whole }; // As many characters as fit beside the rest of the page, which takes the // most with the largest offset it could stop at. const overhead = JSON.stringify({ ...page(end, joined([stoppedNote(end), ...notes])), nextOffsetChars: end, content: '' }).length; diff --git a/src/types/events.ts b/src/types/events.ts index 322f0e2c5..069dbec43 100644 --- a/src/types/events.ts +++ b/src/types/events.ts @@ -253,8 +253,10 @@ export interface ToolCall { * it, as the puppet's does; the framework's own read_image tool passes a * model's call to the workspace without it; and a utility run through * `utils` carries what its `utils` call carried. Without it, a module holds - * its answer to a bound of its own, if any: the workspace's is the house - * default, 24,000. + * its answer to a bound of its own, if any: the workspace's `edit` and + * `edit_many` answers hold to the house default, 24,000, and a `read` holds + * its page to its line limit alone, with every line whole, since such a + * caller takes the data itself. */ resultMaxChars?: number; } diff --git a/test/workspace-read-text.test.ts b/test/workspace-read-text.test.ts index 494c74f20..abcb940ce 100644 --- a/test/workspace-read-text.test.ts +++ b/test/workspace-read-text.test.ts @@ -16,7 +16,7 @@ async function setup(t: TestContext, content: string | Buffer) { t.after(() => { store.close(); rmSync(root, { recursive: true, force: true }); }); assert.equal((await workspace.writeBinary('work/text.txt', Buffer.from(content), 'text/plain')).success, true); // A call the framework dispatched carries the most of its result it stores - // (resultMaxChars); one without it gets the framework's default, 24,000. + // (resultMaxChars); one without it is made in process. const read = (input: Record = {}, resultMaxChars?: number) => workspace.handleToolCall({ id: 'read', name: 'read', input: { path: 'work/text.txt', ...input }, ...(resultMaxChars !== undefined ? { resultMaxChars } : {}), }); @@ -83,7 +83,7 @@ test('a long line is cut at 2000 characters, with the offsetChars that reads the // A BOM and a CRLF before the long line, so its offset counts both. const long = 'x'.repeat(1999) + '😀' + 'y'.repeat(1000); const { read } = await setup(t, Buffer.concat([Buffer.from([0xef, 0xbb, 0xbf]), Buffer.from(`short\r\n${long}\r\nlast\r\n`)])); - const page = (await read()).data as LinePage; + const page = (await read({}, 24_000)).data as LinePage; const lines = page.content.split('\n'); assert.equal(lines.length, 3); // The cut doesn't split the emoji's surrogate pair: it falls before it. @@ -94,7 +94,7 @@ test('a long line is cut at 2000 characters, with the offsetChars that reads the assert.equal(more.content, '😀' + 'y'.repeat(1000)); assert.equal(lines[2], ' 3\tlast'); // Where the cut line ends the page, its note names the offset its marker does. - const two = (await read({ limit: 2 })).data as LinePage; + const two = (await read({ limit: 2 }, 24_000)).data as LinePage; assert.ok(two.note!.startsWith(`Truncated at 2 lines (file has 3). Line 2 is cut: read its rest with offsetChars=${at}. Use offset/limit to read more.`), two.note); }); @@ -169,7 +169,7 @@ test('a page whose last line is cut names where its rest starts before the next const { read } = await setup(t, `short\n${long}\n${long}\n${long}\n`); const at = (line: number) => 'short\n'.length + (line - 2) * (long.length + 1) + 2000; // The limit stops the page at the cut line. - const byLimit = (await read({ limit: 2 })).data as LinePage; + const byLimit = (await read({ limit: 2 }, 24_000)).data as LinePage; assert.equal(byLimit.toLine, 2); assert.equal(byLimit.note, `Truncated at 2 lines (file has 4). Line 2 is cut: read its rest with offsetChars=${at(2)}. Use offset/limit to read more.`); // The result limit stops it there: line 4 would pass 5,000 characters. @@ -187,13 +187,29 @@ test('a page whose last line is cut names where its rest starts before the next assert.equal(((await readShort({}, 24_000)).data as LinePage).note, undefined); }); -test('a call that carries no resultMaxChars gets pages held to the framework\'s default, 24,000', async t => { - const { read } = await setup(t, Array.from({ length: 3000 }, (_, i) => `line ${i + 1} of a file long enough to pass the default`).join('\n')); - const result = await read(); - assert.ok(stored(result) <= 24_000, `${stored(result)}`); - const page = result.data as LinePage; - assert.ok(page.toLine! < 2000); - assert.match(page.note!, /^Stopped at line \d+ to stay within the 24000-character result limit/); +test('a call that carries no resultMaxChars pages by its line limit alone, with every line whole', async t => { + // Such a call is made in process, and its caller takes the data itself, as + // connectome-host's file viewer does with limit 5000 and a cap of its own. + const long = 'q'.repeat(3000); + const lines = Array.from({ length: 3000 }, (_, i) => (i === 10 ? long : `line ${i + 1} of a file long enough to pass 24,000 characters`)); + const { read } = await setup(t, lines.join('\r\n')); + const page = (await read({ limit: 2500 })).data as LinePage; + assert.equal(page.toLine, 2500); + assert.equal(page.note, 'Truncated at 2500 lines (file has 3000). Use offset/limit to read more. Line ends are CRLF, shown without the CR. An edit matches text as read shows it, and keeps the CRLF ends.'); + const shown = page.content.split('\n'); + assert.equal(shown.length, 2500); + assert.equal(shown[10], ` 11\t${long}`); + assert.ok(JSON.stringify(page).length > 100_000, `${JSON.stringify(page).length}`); + // The default is 2000 lines, and the page that ends the file has no note but the text's. + assert.equal(((await read()).data as LinePage).toLine, 2000); + const end = (await read({ offset: 2001 })).data as LinePage; + assert.equal(end.toLine, 3000); + assert.equal(end.note, 'Line ends are CRLF, shown without the CR. An edit matches text as read shows it, and keeps the CRLF ends.'); + // A character page gives its limitChars whole too. + const chars = (await read({ offsetChars: 0, limitChars: 100_000 })).data as CharPage; + assert.equal(chars.content.length, 100_000); + assert.equal(chars.nextOffsetChars, 100_000); + assert.equal(chars.note, undefined); }); test('a character page stops at the result limit too, with nextOffsetChars', async t => { From eec578167685b2dfb616e4b7e6fddb388d7e1ef3 Mon Sep 17 00:00:00 2001 From: Abigail-2676 Date: Sun, 18 Oct 2026 16:11:01 -0400 Subject: [PATCH 09/51] fix(workspace): a read's own disk check names its path, as a file tool's does Review finding R1-1 on #465: merged with main, read's own pass over its path lacked the { named: true } that #460 gave ensureSynced. So a read of a file the mount passes over by design (beneath an ignored directory, ignored by its own name, or an editor's backup) that the store doesn't hold answered that disk could not be checked, "ignored by the mount" or "an editor's swap or backup file", where main's read observes it. The three #276 tests in test/workspace-three-way.test.ts that read such a file failed. The pass now names the path, as a file tool names the one it acts on, so a read observes it as a sync naming it would, and those tests pass. Tests, in test/workspace-three-way.test.ts: a read of a held file beneath an ignored directory, and of one ignored by its own name, takes disk's change in first, says so and stores it, and then takes disk's deletion in first, as a read of any held file does. The #276 tests cover only files the store doesn't hold. Co-Authored-By: Claude Opus 5.5 --- src/modules/workspace/index.ts | 7 +++++-- test/workspace-three-way.test.ts | 29 +++++++++++++++++++++++++++++ 2 files changed, 34 insertions(+), 2 deletions(-) diff --git a/src/modules/workspace/index.ts b/src/modules/workspace/index.ts index 2a717011c..a19a33e5a 100644 --- a/src/modules/workspace/index.ts +++ b/src/modules/workspace/index.ts @@ -2696,8 +2696,11 @@ export class WorkspaceModule implements Module { // A read shows the path as disk holds it now: a change on disk to a // file the workspace holds is taken in first, by the rule a listing // uses, as an edit's is. Where the stored copy is a draft and disk - // changed too, the read shows the draft and names the conflict. - const pass = await this.passUnlocked(mount, { kind: 'paths', paths: [relativePath] }); + // changed too, the read shows the draft and names the conflict. The + // pass names the path, as a file tool names the one it acts on, so a + // path the mount passes over by design is observed, as a sync naming + // it would observe it (#276). + const pass = await this.passUnlocked(mount, { kind: 'paths', paths: [relativePath] }, { named: true }); // A pass that gave up (the selected branch kept changing) decided // nothing and leaves no report. It's read as a copy disk couldn't be // checked, with its reason, so the answer says the check didn't finish. diff --git a/test/workspace-three-way.test.ts b/test/workspace-three-way.test.ts index 919d3233a..820fce561 100644 --- a/test/workspace-three-way.test.ts +++ b/test/workspace-three-way.test.ts @@ -924,6 +924,35 @@ describe('a file tool naming a path passed over by design', () => { }); } + // A read names its path too, so a held file passed over by design is read + // as any held file is: disk's change, and disk's deletion, are taken in + // first. + for (const [kind, ignore, path] of [ + ['beneath a directory an ignore entry covers', ['node_modules'], 'node_modules/pkg/a.txt'], + ['that an ignore entry covers by its own name', ['*.log'], 'build.log'], + ] as const) { + test(`a read of a held file ${kind} takes disk's change and disk's deletion in first, as a read of any held file does`, async (t) => { + const env = new Env(t); + env.writeDisk(path, 'v1\n'); + const m = await env.open({ ignore: [...ignore] }); + await call(m, 'sync', { path: `work/${path}` }); + const stored = () => { + const entry = env.store.treeGet(TREE, path); + return entry ? env.store.getBlob(entry.blobHash)!.toString('utf-8') : null; + }; + assert.equal(stored(), 'v1\n', 'the named sync stored it'); + env.writeDisk(path, 'v2\n'); + const changed = await call(m, 'read', { path: `work/${path}` }); + assert.equal(changed.content, ' 1\tv2', "the read shows disk's text"); + assert.match(String(changed.note), /Disk's copy had changed since the workspace last agreed with it, so it was taken in first/); + assert.equal(stored(), 'v2\n', "and stores disk's change"); + env.rmDisk(path); + const gone = await refused(m, 'read', { path: `work/${path}` }); + assert.equal(gone, `File not found: work/${path}. Disk's copy was deleted since the workspace last agreed with it, and that was taken in first.`); + assert.equal(stored(), null, "and disk's deletion is stored"); + }); + } + test('edit_many edits files passed over by design, each by its path, as a sync naming them takes them in (#276)', async (t) => { const env = new Env(t); env.writeDisk('node_modules/pkg/a.d.ts', 'a v1\n'); From fd7a44b49befb07e093f87deb919dec917ec81b0 Mon Sep 17 00:00:00 2001 From: Abigail-2676 Date: Sun, 18 Oct 2026 15:24:09 -0400 Subject: [PATCH 10/51] docs(framework): executeToolCall's doc names the caller it has The doc said SubagentModule dispatches ephemeral agents' tool calls through executeToolCall. Nothing in src/ calls it but the operator's puppet (puppetToolCall), which puts the agent's inline cap on the call as resultMaxChars, and connectome-host's main doesn't call it. The doc now names the puppet, and says a host may call it as well, as ToolCall.resultMaxChars' doc already describes such a call. Two older lines spoke of callers it doesn't have, the comment beside ModuleContext.callTool's closure and the doc's own reason for agent origin. They now say its callers act for an agent: the puppet as the agent, and a host for the model it runs. Co-Authored-By: Claude Opus 5.5 --- src/framework.ts | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) diff --git a/src/framework.ts b/src/framework.ts index 009a30b0d..a16597525 100644 --- a/src/framework.ts +++ b/src/framework.ts @@ -2531,8 +2531,8 @@ export class AgentFramework { onTrace: (listener) => this.onTrace(listener), // The ONLY entry that confers module origin — a private closure the // framework hands to its own module registry. The public - // executeToolCall() is shared with model/ephemeral callers and always - // stamps agent origin (see there). + // executeToolCall(), whose callers act for an agent (the puppet, a + // host), always stamps agent origin (see there). callTool: (call) => this.executeToolCallFrom(call, { kind: 'module' }), notifyOps: (kind, agentName, message, data) => this.notifyOps(kind, agentName, message, data), }); @@ -17567,12 +17567,15 @@ export class AgentFramework { /** * Execute a tool call and return the result. * Routes to the appropriate handler (module registry or MCPL). - * Used by SubagentModule to dispatch tool calls for ephemeral agents. + * Its one caller in the framework is the operator's puppet + * (puppetToolCall), which puts the agent's inline cap on the call as + * resultMaxChars; a host may call it as well. * - * This public entry always carries AGENT origin. Ephemeral/promise-based - * callers are models, and provenance must identify which internal path a - * call entered through — a public method shared with model callers can - * never confer module trust, whatever the input claims. Module origin + * This public entry always carries AGENT origin. Its callers act for an + * agent, the puppet as the agent and a host for the model it runs, and + * provenance must identify which internal path a call entered through — a + * public method shared with model callers can never confer module trust, + * whatever the input claims. Module origin * exists only through the private closure handed to ModuleRegistry * (ctx.callTool → executeToolCallFrom with {kind:'module'}), which is exempt * from agent allowlists. Registered agents' allowedTools is enforced at From 22becb53d5744505257ac9cdfe32bfbd51cb30b0 Mon Sep 17 00:00:00 2001 From: Abigail-2676 Date: Sun, 18 Oct 2026 17:55:15 -0400 Subject: [PATCH 11/51] fix(workspace): a line page is the longest page that fits its result limit An internal review found that a line page stopped at the first line whose page passed the limit, though a longer page could still fit. A page's length isn't monotone in its lines: the page that ends the file carries no stop note, the page at the line limit a shorter note than a stop, and a page whose last line is cut a longer one. So a file of 900 x's and "a", read at a limit of 1,000, came back as line 1 cut short, though both lines with no note come to about 990 characters, and a file of 300 a's, 550 b's and "c" came back as line 1 alone, 496 characters, though all three fit. linePage now measures each candidate page exactly: its fields and note, measured with no content, beside its lines' length as JSON. It keeps the longest page that fits, and stops looking once a page passes the limit with only the notes every page carries, since a longer page shows more and carries at least those. The overhead estimate it replaces is gone. Where no page fits with its first line as a page shows it, that line is cut shorter, at the most the page has room for, as before. Tests, in test/workspace-read-text.test.ts: the two files above; a page at the line limit that fits at its own length, where the page a line shorter with a stop note doesn't; a page a line past a cut line, which fits where the page ending at the cut line doesn't; and a CRLF file's line pages and an invalid-UTF-8 file's character pages held to the limit with their notes. Co-Authored-By: Claude Opus 5.5 --- src/modules/workspace/read-page.ts | 100 +++++++++++++---------------- test/workspace-read-text.test.ts | 53 +++++++++++++++ 2 files changed, 96 insertions(+), 57 deletions(-) diff --git a/src/modules/workspace/read-page.ts b/src/modules/workspace/read-page.ts index 35b8fe3c1..57fa845bf 100644 --- a/src/modules/workspace/read-page.ts +++ b/src/modules/workspace/read-page.ts @@ -33,9 +33,9 @@ const joined = (notes: Array): string | undefined => { /** * Lines from `offset` (1-based), at most `limit` of them, each numbered and a - * long one cut with the offsetChars that reads the rest. The page stops where - * one more line would take its data past `maxChars` as JSON; if even the - * first line doesn't fit, that line is cut shorter, at the most the page has + * long one cut with the offsetChars that reads the rest. The page is the + * longest that fits within `maxChars` as JSON; where none fits with its first + * line as a page shows it, that line is cut shorter, at the most the page has * room for. Where the page's last line is cut, its note names where that * line's rest starts, since reading on in order starts there: before the next * line, or, on the page that ends the file, before the file is read whole. A @@ -78,67 +78,53 @@ export function linePage(path: string, file: FileText, offset: number, limit: nu const whole = lines.slice(first, last).map((line, k) => numberedLine(line, first + k + 1, Infinity)).join('\n'); return { success: true, data: page(last, whole, joined([last < total ? limitNote(last, null) : null, ...notes])) }; } - // The most anything but the lines can take: the longer note, with the - // largest numbers this page could show. - const overhead = Math.max( - JSON.stringify(page(last, '', joined([limitNote(last, file.raw.length), ...notes]))).length, - JSON.stringify(page(last, '', joined([budgetNote(last, file.raw.length), ...notes]))).length, - ); + // The note of the page that ends at line `to`, its last line shown at most + // lastMax long: the page that ends where it was asked to, at the file's end + // or the line limit, says so, and any other says it stopped for the limit. + const noteAt = (to: number, lastMax = MAX_LINE_CHARS) => { + const rest = restOf(to - 1, lastMax); + const endNote = to < total ? limitNote(to, rest) : rest === null ? null : `${cutLine(to, rest)}.`; + return joined([to === last ? endNote : budgetNote(to, rest), ...notes]); + }; + // A page's length as JSON: its fields' and note's, measured with no content, + // and its lines', which JSON writes inside the content's quotes. + const lengthAt = (to: number, contentChars: number, note: string | undefined) => + JSON.stringify(page(to, '', note)).length + contentChars; + // The page is the longest that fits. Its length isn't monotone in its + // lines: the page that ends the file carries no stop note, the one at the + // line limit a shorter one than a stop, and one whose last line is cut a + // longer one. But every page carries `notes` and a longer page shows more, + // so once a page passes the limit with those alone, no longer one fits. const shownLines: string[] = []; - let used = overhead; + let contentChars = 0; + let best = 0; for (let i = first; i < last; i++) { const line = shown(i); - // Each line after the first comes after a line feed, which JSON writes as two characters. - const cost = jsonLength(line) + (i > first ? 2 : 0); - if (used + cost > maxChars) break; shownLines.push(line); - used += cost; + // Each line after the first comes after a line feed, which JSON writes as two characters. + contentChars += jsonLength(line) + (i > first ? 2 : 0); + if (lengthAt(i + 1, contentChars, noteAt(i + 1)) <= maxChars) best = shownLines.length; + else if (lengthAt(i + 1, contentChars, joined(notes)) > maxChars) break; } - // The page with the lines shown so far, its last one shown at most - // lastMax long. - const final = (lastMax = MAX_LINE_CHARS) => { - const to = first + shownLines.length; - const rest = restOf(to - 1, lastMax); - const endNote = to < total ? limitNote(to, rest) : rest === null ? null : `${cutLine(to, rest)}.`; - return page(to, shownLines.join('\n'), joined([to === last ? endNote : budgetNote(to, rest), ...notes])); - }; - if (shownLines.length === 0) { - // The estimate above found no room for the first line as a page shows it. - // Measured on the page itself, since its note names a cut: the line as a - // page shows it first, then, where that doesn't fit, the most a cut can - // show. Shown whole, a line carries no marker and its note no cut, so its - // page can fit where a cut one character shorter doesn't. Below that a - // longer cut never makes the page shorter, so the first that doesn't fit - // bounds the search. - const whole = Math.min(lines[first].length, MAX_LINE_CHARS); - const fitsAt = (chars: number) => { - shownLines[0] = shown(first, chars); - return JSON.stringify(final(chars)).length <= maxChars; - }; - let fits = whole; - if (!fitsAt(whole)) { - fits = 0; - let over = whole; - while (over - fits > 1) { - const mid = Math.floor((fits + over) / 2); - if (fitsAt(mid)) fits = mid; else over = mid; - } - } - shownLines[0] = shown(first, fits); - // Cut shorter than a page shows it, the line ends the page: its rest - // comes next. Shown as a page shows it, more lines may fit beside it. - if (fits < whole) return { success: true, data: final(fits) }; + if (best > 0) { + shownLines.length = best; + return { success: true, data: page(first + best, shownLines.join('\n'), noteAt(first + best)) }; } - // The overhead allowed for the largest numbers; a line more may fit beside - // the real ones. - while (first + shownLines.length < last) { - shownLines.push(shown(first + shownLines.length)); - if (JSON.stringify(final()).length > maxChars) { - shownLines.pop(); - break; - } + // No page fits with its first line as a page shows it, so that line is cut + // shorter, at the most the page has room for, and ends the page: its note + // names where the line's rest starts, since that comes next. Measured on + // the page itself, since its note names the cut. Shown whole, a line carries + // no marker and its note no cut, so its page can fit where a cut one + // character shorter doesn't. Below that a longer cut never makes the page + // shorter, so the first that doesn't fit bounds the search. + const pageWith = (chars: number) => page(first + 1, shown(first, chars), noteAt(first + 1, chars)); + let fits = 0; + let over = Math.min(lines[first].length, MAX_LINE_CHARS); + while (over - fits > 1) { + const mid = Math.floor((fits + over) / 2); + if (JSON.stringify(pageWith(mid)).length <= maxChars) fits = mid; else over = mid; } - return { success: true, data: final() }; + return { success: true, data: pageWith(fits) }; } const splitsPair = (text: string, at: number): boolean => diff --git a/test/workspace-read-text.test.ts b/test/workspace-read-text.test.ts index abcb940ce..4ca4ca663 100644 --- a/test/workspace-read-text.test.ts +++ b/test/workspace-read-text.test.ts @@ -118,6 +118,59 @@ test('a line page stops before its result would pass the limit, and says where t } }); +test('a line page is the longest that fits, though the page that ends where it was asked to carries a shorter note', async t => { + // The page that ends the file has no stop note, so it can fit where a + // shorter page with one doesn't: both lines, rather than the first cut short. + const { read } = await setup(t, `${'x'.repeat(900)}\na\n`); + const both = await read({}, 1_000); + assert.ok(stored(both) <= 1_000, `${stored(both)}`); + assert.deepEqual(both.data, { path: 'work/text.txt', totalLines: 2, fromLine: 1, toLine: 2, content: ` 1\t${'x'.repeat(900)}\n 2\ta` }); + const { read: readThree } = await setup(t, `${'a'.repeat(300)}\n${'b'.repeat(550)}\nc\n`); + const three = await readThree({}, 1_000); + assert.ok(stored(three) <= 1_000, `${stored(three)}`); + assert.equal((three.data as LinePage).toLine, 3); + assert.equal((three.data as LinePage).note, undefined); + // The page at the line limit says so in fewer words than a stop, so it fits + // at its own length, where the page a line shorter, with a stop note, doesn't. + const { read: readLimit } = await setup(t, `${'a'.repeat(800)}\nb\nc\n`); + const atLimit = await readLimit({ limit: 2 }, 5_000_000); + assert.equal((atLimit.data as LinePage).note, 'Truncated at 2 lines (file has 3). Use offset/limit to read more.'); + assert.deepEqual((await readLimit({ limit: 2 }, stored(atLimit))).data, atLimit.data); + // A page whose last line is cut names that line's rest, in more words than + // a page whose last line is whole, so the page a line longer can fit where it doesn't. + const { read: readCut } = await setup(t, `${'q'.repeat(2500)}\nz\n${'w'.repeat(300)}\n`); + const two = (limit: number) => ({ + path: 'work/text.txt', totalLines: 3, fromLine: 1, toLine: 2, + note: `Stopped at line 2 to stay within the ${limit}-character result limit (file has 3). Continue with offset=3.`, + content: ` 1\t${'q'.repeat(2000)} [… 500 more chars; read them with offsetChars=2000]\n 2\tz`, + }); + const limit = JSON.stringify(two(9_999)).length; + assert.equal(String(limit).length, 4, 'the limit has as many digits as the 9,999 it was measured with'); + assert.deepEqual((await readCut({}, limit)).data, two(limit)); +}); + +test('a page with notes holds to the result limit with its notes', async t => { + // A CRLF file's note shares a line page's room with its lines and its stop note. + const { read } = await setup(t, Array.from({ length: 300 }, (_, i) => `l${i}`).join('\r\n') + '\r\n'); + for (const limit of [1_000, 1_500]) { + const result = await read({}, limit); + assert.ok(stored(result) <= limit, `${stored(result)} > ${limit}`); + const page = result.data as LinePage; + assert.equal(page.note, `Stopped at line ${page.toLine} to stay within the ${limit}-character result limit (file has 300). Continue with offset=${page.toLine! + 1}. Line ends are CRLF, shown without the CR. An edit matches text as read shows it, and keeps the CRLF ends.`); + // One more line would have passed the limit. + const more = await read({ limit: page.toLine! + 1 }, 5_000_000); + assert.ok(JSON.stringify({ ...page, toLine: page.toLine! + 1, content: (more.data as LinePage).content }).length > limit); + } + // An invalid-UTF-8 file's note shares a character page's room. + const { read: readBad } = await setup(t, Buffer.concat([Buffer.from('a'.repeat(5000)), Buffer.from([0xff]), Buffer.from('b'.repeat(5000))])); + for (const limit of [1_000, 1_500]) { + const result = await readBad({ offsetChars: 0, limitChars: 100_000 }, limit); + assert.ok(stored(result) <= limit, `${stored(result)} > ${limit}`); + const page = result.data as CharPage; + assert.equal(page.note, `Stopped at character ${page.nextOffsetChars} to stay within the ${limit}-character result limit; continue with offsetChars=${page.nextOffsetChars}. The file isn't valid UTF-8: each invalid byte sequence shows as U+FFFD. Edit refuses this file, and write replaces it whole.`); + } +}); + test('a first line that alone would pass the limit is cut where it fits', async t => { // Each quote takes two characters as JSON. const { read } = await setup(t, '"'.repeat(1500) + '\nnext\n'); From 4bb31cf7c28f78ae1a73cf410af4bcec9393df1f Mon Sep 17 00:00:00 2001 From: Abigail-2676 Date: Sun, 18 Oct 2026 17:55:16 -0400 Subject: [PATCH 12/51] fix(workspace): a character page that holds all it was asked for names no stop An internal review found that a character page whose limit left no room beside its own fields always said it had stopped for the limit. The page moves on by one character at least, and where that reaches what it was asked for, as with limitChars 1 or the file's last character, it holds all of it: a one-character file read with limitChars 1 at a limit of 50 said "Stopped at character 1 ...; continue with offsetChars=1" with nextOffsetChars null. Such a page now names no stop. Tests, in test/workspace-read-text.test.ts: that page, one that holds its limitChars of a longer file, and the forced step over a surrogate pair, which moves the page on by both its code units. Co-Authored-By: Claude Opus 5.5 --- src/modules/workspace/read-page.ts | 5 +++-- test/workspace-read-text.test.ts | 20 ++++++++++++++++++++ 2 files changed, 23 insertions(+), 2 deletions(-) diff --git a/src/modules/workspace/read-page.ts b/src/modules/workspace/read-page.ts index 57fa845bf..8b97ecaff 100644 --- a/src/modules/workspace/read-page.ts +++ b/src/modules/workspace/read-page.ts @@ -172,7 +172,8 @@ export function characterPage(path: string, file: FileText, offsetChars: number, to += step(to); } // A page always moves on, by one character at least, though a limit too - // small for the page's own fields then passes it. + // small for the page's own fields then passes it. Where that reaches the + // page's own end, it holds all it was asked for, so it names no stop. if (to === start) to = Math.min(end, start + step(start)); - return { success: true, data: page(to, joined([stoppedNote(to), ...notes])) }; + return { success: true, data: page(to, joined([to < end ? stoppedNote(to) : null, ...notes])) }; } diff --git a/test/workspace-read-text.test.ts b/test/workspace-read-text.test.ts index 4ca4ca663..13fc035a4 100644 --- a/test/workspace-read-text.test.ts +++ b/test/workspace-read-text.test.ts @@ -300,6 +300,26 @@ test('a character page stops at the result limit too, with nextOffsetChars', asy } }); +test('a character page moved on by a whole character keeps a surrogate pair whole, and one holding all it was asked for names no stop', async t => { + // A limit too small for the page's own fields still moves the page on, by a + // whole character: an emoji's two code units. + const { read } = await setup(t, '😀abc'); + assert.deepEqual((await read({ offsetChars: 0, limitChars: 100 }, 50)).data, { + path: 'work/text.txt', totalChars: 5, offsetChars: 0, nextOffsetChars: 2, + note: 'Stopped at character 2 to stay within the 50-character result limit; continue with offsetChars=2.', + content: '😀', + }); + // Where that step reaches what the page was asked for, the page holds all of it. + const { read: readOne } = await setup(t, 'a'); + assert.deepEqual((await readOne({ offsetChars: 0, limitChars: 1 }, 50)).data, { + path: 'work/text.txt', totalChars: 1, offsetChars: 0, nextOffsetChars: null, content: 'a', + }); + const { read: readSix } = await setup(t, 'abcdef'); + assert.deepEqual((await readSix({ offsetChars: 0, limitChars: 1 }, 50)).data, { + path: 'work/text.txt', totalChars: 6, offsetChars: 0, nextOffsetChars: 1, content: 'a', + }); +}); + test('character mode can be selected by either parameter and reports EOF', async t => { const { read } = await setup(t, 'a'.repeat(3000)); assert.deepEqual((await read({ offsetChars: 0 })).data, { From fb0f7647b75703ba63382ff890ff4c95c947097f Mon Sep 17 00:00:00 2001 From: Abigail-2676 Date: Sun, 18 Oct 2026 17:55:16 -0400 Subject: [PATCH 13/51] fix(workspace): a read says disk's copy was taken in only where its text changed An internal review found that a read's note counted a mode taken in alone, such as a chmod on disk, as disk's copy having changed, and said this read shows that text. The pass marks such an op modeOnly, and edit already leaves it out of its notes. The read now leaves it out too. Test, in test/workspace-three-way.test.ts: a read of a held file whose mode alone changed on disk takes the mode in and carries no note. Co-Authored-By: Claude Opus 5.5 --- src/modules/workspace/index.ts | 4 +++- test/workspace-three-way.test.ts | 13 +++++++++++++ 2 files changed, 16 insertions(+), 1 deletion(-) diff --git a/src/modules/workspace/index.ts b/src/modules/workspace/index.ts index 4177dbfc5..dd5d41872 100644 --- a/src/modules/workspace/index.ts +++ b/src/modules/workspace/index.ts @@ -2989,7 +2989,9 @@ export class WorkspaceModule implements Module { // checked, with its reason, so the answer says the check didn't finish. const report = pass.reports.get(relativePath) ?? (pass.incomplete.length > 0 ? { path: relativePath, state: 'unverified' as const, note: pass.incomplete[0]!.reason } : undefined); - const took = (op: TreeOp) => pass.ops.some((o) => o.path === relativePath && o.op === op); + // A mode taken in alone isn't a change to the text, so it leaves the + // read nothing to say, as it leaves an edit nothing. + const took = (op: TreeOp) => pass.ops.some((o) => o.path === relativePath && o.op === op && !o.modeOnly); const entry = store.treeGet(mount.treeStateId, relativePath); if (!entry) { // A pass also drops a copy disk made binary or too large, which diff --git a/test/workspace-three-way.test.ts b/test/workspace-three-way.test.ts index 8af757e71..1fca1ef9e 100644 --- a/test/workspace-three-way.test.ts +++ b/test/workspace-three-way.test.ts @@ -3278,6 +3278,19 @@ describe("a file's mode is kept with its bytes", () => { assert.match(keptReasons(pushed), /blob\.bin: written, keeping disk's mode \(0600\)/); }); + test('a read of a held file whose mode alone changed on disk takes the mode in, and says nothing of its text', async (t) => { + const env = new Env(t); + const m = await env.open(); + env.writeDisk('tool.sh', 'echo tool\n'); + chmodSync(env.disk('tool.sh'), 0o644); + await call(m, 'sync', {}); + chmodSync(env.disk('tool.sh'), 0o755); + const data = await call(m, 'read', { path: 'work/tool.sh' }); + assert.equal(data.content, ' 1\techo tool'); + assert.equal(data.note, undefined, "a mode taken in alone isn't a change to the text"); + assert.equal(storedMode(env, 'tool.sh'), 0o100755, 'the read took the mode in'); + }); + test('a chmod on disk is taken in as a change on the same bytes, and makes no event', async (t) => { const env = new Env(t); const m = await env.open({ wakeOnChange: true }); From 62d223d090ec6c9bd662535f4895bb1492c5f165 Mon Sep 17 00:00:00 2001 From: Abigail-2676 Date: Sun, 18 Oct 2026 17:55:16 -0400 Subject: [PATCH 14/51] test(workspace): the read tests an internal review found missing - A generated file read under a result limit stops within it and says where to continue, and read in process comes whole (test/workspace-read-text.test.ts). - A conflict's disk version with CRLF ends is searched and shown without the CR, and grep's result names its lineEndings. - A read whose disk check gives up says so on a character page and on a line page under a result limit, as on a plain line page. - Five scan and sync tests checked what disk's edit left through a read, which now settles its path and so takes the edit in itself. They read the store instead, as the three this branch moved earlier do. Co-Authored-By: Claude Opus 5.5 --- test/workspace-read-text.test.ts | 15 +++++++++++++++ test/workspace-three-way.test.ts | 33 +++++++++++++++++++++++++++----- 2 files changed, 43 insertions(+), 5 deletions(-) diff --git a/test/workspace-read-text.test.ts b/test/workspace-read-text.test.ts index 13fc035a4..f61ebacce 100644 --- a/test/workspace-read-text.test.ts +++ b/test/workspace-read-text.test.ts @@ -171,6 +171,21 @@ test('a page with notes holds to the result limit with its notes', async t => { } }); +test('a generated file reads under the call\'s result limit too', async t => { + const { workspace } = await setup(t, 'unused'); + const catalogue = Array.from({ length: 500 }, (_, i) => `tool ${i + 1}: a description long enough to fill a page`).join('\n'); + workspace.registerGeneratedTextFile('work/tools.md', () => catalogue, 'ada'); + const read = (resultMaxChars?: number) => workspace.handleToolCall({ + id: 'generated', name: 'read', input: { path: 'work/tools.md' }, callerAgentName: 'ada', ...(resultMaxChars !== undefined ? { resultMaxChars } : {}), + }); + const limited = await read(1_000); + assert.ok(stored(limited) <= 1_000, `${stored(limited)}`); + const page = limited.data as LinePage; + assert.equal(page.note, `Stopped at line ${page.toLine} to stay within the 1000-character result limit (file has 500). Continue with offset=${page.toLine! + 1}.`); + // A call made in process takes it whole. + assert.equal(((await read()).data as LinePage).toLine, 500); +}); + test('a first line that alone would pass the limit is cut where it fits', async t => { // Each quote takes two characters as JSON. const { read } = await setup(t, '"'.repeat(1500) + '\nnext\n'); diff --git a/test/workspace-three-way.test.ts b/test/workspace-three-way.test.ts index 1fca1ef9e..00aa6e27c 100644 --- a/test/workspace-three-way.test.ts +++ b/test/workspace-three-way.test.ts @@ -5090,7 +5090,7 @@ describe('a mount larger than its walk cap (agent-framework #272)', () => { env.writeDisk('a.txt', 'edited before the wrap'); await call(m, 'sync', {}); assert.deepEqual(syncedNames(env), ['a.txt', 'b/x.txt', 'b/y.txt', 'm.txt', 'z.txt']); - assert.equal(await contentOf(m, 'a.txt'), 'edited before the wrap', 'the third sync wrapped to a.txt'); + assert.equal(storedOf(env, 'a.txt'), 'edited before the wrap', 'the third sync wrapped to a.txt'); }); test('the walk takes files by name, whatever order they were written in', async (t) => { @@ -5129,7 +5129,7 @@ describe('a mount larger than its walk cap (agent-framework #272)', () => { env.writeDisk('c.txt', 'edited past the cap'); await call(m, 'sync', {}); await call(m, 'sync', {}); - assert.equal(await contentOf(m, 'c.txt'), 'edited past the cap'); + assert.equal(storedOf(env, 'c.txt'), 'edited past the cap'); }); test('a deletion in a directory larger than the cap is proven by a listing read whole', async (t) => { @@ -5244,7 +5244,7 @@ describe('a mount larger than its walk cap (agent-framework #272)', () => { env.writeDisk('a/1.txt', 'edited before the wrap'); await call(m, 'sync', {}); assert.deepEqual(syncedNames(env), ['a/1.txt', 'a/2.txt', 'a/3.txt', 'm/1.txt', 'm/2.txt']); - assert.equal(await contentOf(m, 'a/1.txt'), 'edited before the wrap'); + assert.equal(storedOf(env, 'a/1.txt'), 'edited before the wrap'); }); test('once the cap is reached no further directory is listed, so a deletion there waits for a pass that lists it', async (t) => { @@ -5710,7 +5710,7 @@ describe('a mount larger than its walk cap (agent-framework #272)', () => { return readdir(path, ...rest); }, () => call(m, 'sync', {})); assert.ok(removed && restored); - assert.equal(await contentOf(m, 'm/1.txt'), 'm/1.txt edited while it was gone'); + assert.equal(storedOf(env, 'm/1.txt'), 'm/1.txt edited while it was gone'); }); test('a directory sync whose own directory is gone when the after stretch lists it, and another by the wrap, proves nothing absent', async (t) => { @@ -6705,6 +6705,18 @@ describe('grep', () => { assert.deepEqual((one.results as Array<{ file: string }>).map((r) => r.file), ['work/draft.txt']); }); + test("a conflict's disk version with CRLF ends is searched and shown as read shows its lines, and says so", async (t) => { + const env = new Env(t); + const m = await env.open(); + await seedSynced(env, m, 'a.txt', 'v1'); + env.writeDisk('a.txt', 'disk says ALPHA\r\nsecond\r\n'); + await call(m, 'write', { path: 'work/a.txt', content: 'workspace says GAMMA\n' }); + assert.equal((await entryOf(m, 'a.txt'))?.conflict?.diskCopy, 'stored'); + const data = await call(m, 'grep', { pattern: 'ALPHA$' }); + assert.deepEqual(data.results.map((r: { version: string; lineEndings?: string; matches: Array<{ text: string }> }) => [r.version, r.lineEndings, r.matches[0]!.text]), + [['conflicting-disk', 'CRLF', 'disk says ALPHA']]); + }); + test("once disk changes again, a conflict's stored disk version is labelled as recorded, and what disk holds now as not searched", async (t) => { const env = new Env(t); const m = await env.open(); @@ -7034,6 +7046,17 @@ describe('on-agent-action', () => { () => call(m, 'read', { path: 'work/a.txt' })); assert.equal(data.content, ' 1\tA'); assert.equal(data.note, "The path couldn't be checked against disk (the selected branch kept changing while the scan ran), so this read shows the workspace's copy."); + // A character page and a line page under a result limit say so too, + // each read starting on main, as the first did. + const switching = (n: number) => env.store.switchBranch(n % 2 === 1 ? 'other' : main); + env.store.switchBranch(main); + const chars = await withReadHook(switching, () => call(m, 'read', { path: 'work/a.txt', offsetChars: 0, limitChars: 10 })); + assert.equal(chars.content, 'A'); + assert.equal(chars.note, data.note); + env.store.switchBranch(main); + const limited = await withReadHook(switching, () => m.handleToolCall({ id: 't', name: 'read', input: { path: 'work/a.txt' }, resultMaxChars: 1_000 })); + assert.equal(limited.success, true, String(limited.error)); + assert.equal((limited.data as { note?: string }).note, data.note); }); test("a read of a file the workspace doesn't hold, whose disk check gives up, says disk couldn't be checked", async (t) => { @@ -7468,7 +7491,7 @@ describe('a bare materialize after a restart', () => { await call(m, 'materialize', {}); env.writeDisk('a.txt', 'v2 from a shell'); await call(m, 'sync', {}); - assert.equal(await contentOf(m, 'a.txt'), 'v2 from a shell', 'disk adopted: a tree change disk already holds'); + assert.equal(storedOf(env, 'a.txt'), 'v2 from a shell', 'disk adopted: a tree change disk already holds'); // An entry from before the evidence is unchecked, not unpushed. env.legacyEntry('legacy.txt', 'never checked'); assert.equal((await call(m, 'status', {})).work.pendingChanges, 0); From 0f02d3e2b11bb3e0ab4b235f5e91d5b92ba1a176 Mon Sep 17 00:00:00 2001 From: Abigail-2676 Date: Sun, 18 Oct 2026 17:55:16 -0400 Subject: [PATCH 15/51] docs(changelog): grep shows each matched line whole The grep bullet said a match's text and context are the lines as read shows them. They are without the CR, as read shows them, but whole: grep doesn't cut a line over 2,000 characters as a read page for a call with a result limit does. Co-Authored-By: Claude Opus 5.5 --- changelog.d/feat-workspace-read.changed.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/changelog.d/feat-workspace-read.changed.md b/changelog.d/feat-workspace-read.changed.md index 1961babd1..6bb80c108 100644 --- a/changelog.d/feat-workspace-read.changed.md +++ b/changelog.d/feat-workspace-read.changed.md @@ -1,6 +1,6 @@ - The workspace's `read` shows a file's text as it is, and says so. A line page shows a CRLF file's lines without the CR before each line feed, and leaves out a BOM. Its note says when the file has either, counting a mix of CRLF and LF ends. It also says when the file isn't valid UTF-8: each invalid byte sequence shows as U+FFFD, `edit` refuses the file, and `write` replaces it whole. Before, each line of a CRLF file ended in a CR, a BOM came first in the first line as U+FEFF, and invalid bytes showed as U+FFFD with no note. A character page still reads the stored text as it is, BOM and CRs included, and the tool's description now says so. - On a page for a call that carries a `resultMaxChars`, such as a model's or a running script's, a line over 2,000 characters is cut, with how many characters follow and the `offsetChars` that reads them. Before, a line page showed every line whole, however long. - A `read` page stops before its result would pass `resultMaxChars`, the most of it the framework stores whole. For a model's call that's its agent's inline cap, 24,000 characters by default. Its note says where to continue: the `offset` after its last line, or for a character page the `offsetChars` it also gives as `nextOffsetChars`. Where even a page's first line would pass the limit, that line is cut shorter, at the most the page has room for. Where a page's last line is cut, its note names the `offsetChars` where that line's rest starts, before the next line's `offset` if the file goes on. A page always moves on, so where its own fields (the path and its notes) and the least it can show pass a small limit, it passes the limit: at 1,000 characters, only a path of about 130 characters or more beside every note a read can carry does that, or one of about 700 with none. Before, a page of up to 2,000 lines, or of `limitChars` characters, went out whole, and for a result that passed the limit the framework kept only its start, cut mid-JSON, writing the whole of it to a workspace file where the agent has one. A caller that pages 2,000 lines at a time and starts the next page at line 2,001 now starts where the note says. A running `code_execution` script's call has a limit of 5,000,000 characters, so its pages still stop at 2,000 lines or `limitChars`, unless a page would pass that. Its line pages cut a line over 2,000 characters and leave out CRs and a BOM, as a model's do, and a character page gives it the stored text exactly. A call made in process, through `handleToolCall`, `executeToolCall` or `ModuleContext.callTool`, carries no `resultMaxChars` unless its caller sets one. Its line pages stop at the line limit alone and show every line whole, as they did before, though now without the CRs and the BOM. -- The workspace's `grep` leaves out a CR before a line feed, for matching as well as display, so `foo$` finds a CRLF file's line too, as it does under ripgrep's `--crlf`. A match's text and context are the lines as `read` shows them, a BOM isn't part of the first line, and a file with CRLF ends says so in its result's `lineEndings` (`CRLF`, or `mixed`). Before, a CRLF file's lines ended in a CR, so `foo$` missed them and a match's text copied into an edit carried the CR. A pattern written to find the CR of a CRLF line end no longer matches; a CR that doesn't come before a line feed is still part of its line. +- The workspace's `grep` leaves out a CR before a line feed, for matching as well as display, so `foo$` finds a CRLF file's line too, as it does under ripgrep's `--crlf`. A match's text and context are its lines without the CR, as `read` shows them, but each line whole, where a `read` page for a call with a result limit cuts a line over 2,000 characters. A BOM isn't part of the first line, and a file with CRLF ends says so in its result's `lineEndings` (`CRLF`, or `mixed`). Before, a CRLF file's lines ended in a CR, so `foo$` missed them and a match's text copied into an edit carried the CR. A pattern written to find the CR of a CRLF line end no longer matches; a CR that doesn't come before a line feed is still part of its line. - The workspace's `read` first takes in a change on disk to a file the workspace holds, as a listing and an `edit` do, and shows that text. Its note says so when disk's copy had changed since the workspace last agreed with it. It also says when disk's copy couldn't be checked, or disk has none and nothing records whether it was deleted there, so it shows the workspace's copy. A file disk deleted answers that disk deleted it and the deletion was taken in, and a file disk made binary or too large answers that it's on disk but not stored. Where the path's stored copy is a draft and disk changed or deleted it too, the read shows the draft, and its note names the conflict and both ways to settle it: for a deletion, a sync of the path takes it and removes the workspace's copy. Before, a read of a path the workspace held didn't look at disk, so it could show a copy another writer had since changed; a path it didn't hold was observed, as now. - The `read` and `grep` tool descriptions changed, so the first request after the upgrade that carries the workspace's tools misses the prompt cache from its tool block on. From db840ca20b17153a0c3194ad215a7edf899cbdaf Mon Sep 17 00:00:00 2001 From: Abigail-2676 Date: Sun, 18 Oct 2026 18:36:42 -0400 Subject: [PATCH 16/51] fix(workspace): a read's conflict note on a read-only mount names only the sync An internal review found that a read of a path in conflict with disk on a read-only mount offered a forced materialize, which writes nothing to a read-only mount. A read-only mount can hold a copy disk doesn't: a branch switch can select one, and when disk then changes, the path reads as in conflict. The note now names only the sync there, and says the mount is read-only, so nothing writes the workspace's copy to disk. The advice is edit's wording, which never reaches a read-only mount, since edit refuses there first. Test, in test/workspace-three-way.test.ts: a read-only mount switched to a branch holding the copy disk had before, whose disk copy then changes. The changelog says a forced materialize settles a conflict only on a mount that isn't read-only. Co-Authored-By: Claude Opus 5.5 --- changelog.d/feat-workspace-read.changed.md | 2 +- src/modules/workspace/index.ts | 22 +++++++++++++++------- test/workspace-three-way.test.ts | 18 ++++++++++++++++++ 3 files changed, 34 insertions(+), 8 deletions(-) diff --git a/changelog.d/feat-workspace-read.changed.md b/changelog.d/feat-workspace-read.changed.md index 6bb80c108..9e8aac777 100644 --- a/changelog.d/feat-workspace-read.changed.md +++ b/changelog.d/feat-workspace-read.changed.md @@ -2,5 +2,5 @@ - On a page for a call that carries a `resultMaxChars`, such as a model's or a running script's, a line over 2,000 characters is cut, with how many characters follow and the `offsetChars` that reads them. Before, a line page showed every line whole, however long. - A `read` page stops before its result would pass `resultMaxChars`, the most of it the framework stores whole. For a model's call that's its agent's inline cap, 24,000 characters by default. Its note says where to continue: the `offset` after its last line, or for a character page the `offsetChars` it also gives as `nextOffsetChars`. Where even a page's first line would pass the limit, that line is cut shorter, at the most the page has room for. Where a page's last line is cut, its note names the `offsetChars` where that line's rest starts, before the next line's `offset` if the file goes on. A page always moves on, so where its own fields (the path and its notes) and the least it can show pass a small limit, it passes the limit: at 1,000 characters, only a path of about 130 characters or more beside every note a read can carry does that, or one of about 700 with none. Before, a page of up to 2,000 lines, or of `limitChars` characters, went out whole, and for a result that passed the limit the framework kept only its start, cut mid-JSON, writing the whole of it to a workspace file where the agent has one. A caller that pages 2,000 lines at a time and starts the next page at line 2,001 now starts where the note says. A running `code_execution` script's call has a limit of 5,000,000 characters, so its pages still stop at 2,000 lines or `limitChars`, unless a page would pass that. Its line pages cut a line over 2,000 characters and leave out CRs and a BOM, as a model's do, and a character page gives it the stored text exactly. A call made in process, through `handleToolCall`, `executeToolCall` or `ModuleContext.callTool`, carries no `resultMaxChars` unless its caller sets one. Its line pages stop at the line limit alone and show every line whole, as they did before, though now without the CRs and the BOM. - The workspace's `grep` leaves out a CR before a line feed, for matching as well as display, so `foo$` finds a CRLF file's line too, as it does under ripgrep's `--crlf`. A match's text and context are its lines without the CR, as `read` shows them, but each line whole, where a `read` page for a call with a result limit cuts a line over 2,000 characters. A BOM isn't part of the first line, and a file with CRLF ends says so in its result's `lineEndings` (`CRLF`, or `mixed`). Before, a CRLF file's lines ended in a CR, so `foo$` missed them and a match's text copied into an edit carried the CR. A pattern written to find the CR of a CRLF line end no longer matches; a CR that doesn't come before a line feed is still part of its line. -- The workspace's `read` first takes in a change on disk to a file the workspace holds, as a listing and an `edit` do, and shows that text. Its note says so when disk's copy had changed since the workspace last agreed with it. It also says when disk's copy couldn't be checked, or disk has none and nothing records whether it was deleted there, so it shows the workspace's copy. A file disk deleted answers that disk deleted it and the deletion was taken in, and a file disk made binary or too large answers that it's on disk but not stored. Where the path's stored copy is a draft and disk changed or deleted it too, the read shows the draft, and its note names the conflict and both ways to settle it: for a deletion, a sync of the path takes it and removes the workspace's copy. Before, a read of a path the workspace held didn't look at disk, so it could show a copy another writer had since changed; a path it didn't hold was observed, as now. +- The workspace's `read` first takes in a change on disk to a file the workspace holds, as a listing and an `edit` do, and shows that text. Its note says so when disk's copy had changed since the workspace last agreed with it. It also says when disk's copy couldn't be checked, or disk has none and nothing records whether it was deleted there, so it shows the workspace's copy. A file disk deleted answers that disk deleted it and the deletion was taken in, and a file disk made binary or too large answers that it's on disk but not stored. Where the path's stored copy is a draft and disk changed or deleted it too, the read shows the draft, and its note names the conflict and the ways to settle it, a sync, or a forced materialize on a mount that isn't read-only: for a deletion, a sync of the path takes it and removes the workspace's copy. Before, a read of a path the workspace held didn't look at disk, so it could show a copy another writer had since changed; a path it didn't hold was observed, as now. - The `read` and `grep` tool descriptions changed, so the first request after the upgrade that carries the workspace's tools misses the prompt cache from its tool block on. diff --git a/src/modules/workspace/index.ts b/src/modules/workspace/index.ts index dd5d41872..4cdca944d 100644 --- a/src/modules/workspace/index.ts +++ b/src/modules/workspace/index.ts @@ -3004,7 +3004,7 @@ export class WorkspaceModule implements Module { } const blob = store.getBlob(entry.blobHash); if (!blob) return { error: `Blob not found for: ${input.path}` }; - return { bytes: blob, notes: whichCopyRead(report, took('modified')) }; + return { bytes: blob, notes: whichCopyRead(report, took('modified'), mount.config.mode === 'read-only') }; }); if ('error' in found) return { success: false, error: found.error, isError: true }; file = fileText(found.bytes); @@ -4636,8 +4636,9 @@ function storedButUnfinished( * workspace's copy, because disk's couldn't be checked or disk has none that * anything accounts for; or a draft in conflict with disk's change. A file * the workspace didn't hold has no older copy, so taking it in needs no word. + * A conflict's note names the ways to settle it that work on the path's mount. */ -function whichCopyRead(report: PathReport | undefined, tookChange: boolean): string[] { +function whichCopyRead(report: PathReport | undefined, tookChange: boolean, readOnly: boolean): string[] { if (tookChange) { return ['Disk\'s copy had changed since the workspace last agreed with it, so it was taken in first, and this read shows that text.']; } @@ -4653,11 +4654,18 @@ function whichCopyRead(report: PathReport | undefined, tookChange: boolean): str // doesn't follow: where disk has no copy, the sync takes its deletion, and // the workspace's copy goes with it. Where disk has one, the sync keeps it // and gives up the workspace's, taking disk's in only where the workspace - // can hold it (not binary or too large). - return [`The path is in conflict with disk (${kind}): this read shows the workspace's copy. ` - + (report.diskNow === 'absent' - ? 'Sync the path to take disk\'s deletion, which removes the workspace\'s copy, or materialize it with force to write the workspace\'s copy back to disk.' - : 'Sync the path to keep disk\'s copy and give up the workspace\'s, or materialize it with force to write the workspace\'s over disk\'s.')]; + // can hold it (not binary or too large). A read-only mount can hold a + // copy disk doesn't, such as one a branch switch selected, but a + // materialize, forced or not, writes nothing to it. + const sync = report.diskNow === 'absent' + ? 'Sync the path to take disk\'s deletion, which removes the workspace\'s copy' + : 'Sync the path to keep disk\'s copy and give up the workspace\'s'; + const force = readOnly + ? '. The mount is read-only, so nothing writes the workspace\'s copy to disk.' + : report.diskNow === 'absent' + ? ', or materialize it with force to write the workspace\'s copy back to disk.' + : ', or materialize it with force to write the workspace\'s over disk\'s.'; + return [`The path is in conflict with disk (${kind}): this read shows the workspace's copy. ${sync}${force}`]; } return []; } diff --git a/test/workspace-three-way.test.ts b/test/workspace-three-way.test.ts index 00aa6e27c..c75a9900d 100644 --- a/test/workspace-three-way.test.ts +++ b/test/workspace-three-way.test.ts @@ -7059,6 +7059,24 @@ describe('on-agent-action', () => { assert.equal((limited.data as { note?: string }).note, data.note); }); + test("a read of a conflict on a read-only mount names only the sync, since nothing writes the workspace's copy there", async (t) => { + const env = new Env(t); + const m = await env.open({ mode: 'read-only' }); + env.writeDisk('a.txt', 'v1\n'); + await call(m, 'sync', {}); + const main = env.store.currentBranch().name; + env.store.createBranch('earlier', main); + env.writeDisk('a.txt', 'v2\n'); + await call(m, 'sync', {}); + // The earlier branch holds the copy disk had before, and disk then changes again. + env.store.switchBranch('earlier'); + env.writeDisk('a.txt', 'v3 from a shell\n'); + const data = await call(m, 'read', { path: 'work/a.txt' }); + assert.equal(data.content, ' 1\tv1'); + assert.equal(data.note, "The path is in conflict with disk (both-changed): this read shows the workspace's copy. " + + "Sync the path to keep disk's copy and give up the workspace's. The mount is read-only, so nothing writes the workspace's copy to disk."); + }); + test("a read of a file the workspace doesn't hold, whose disk check gives up, says disk couldn't be checked", async (t) => { const env = new Env(t); const m = await env.open(); From 966ae54283f722e69a8881e0753499bc767d7265 Mon Sep 17 00:00:00 2001 From: Abigail-2676 Date: Sun, 18 Oct 2026 18:36:42 -0400 Subject: [PATCH 17/51] test(workspace): a continued line page ends where its note says An internal review found that no test checked a line page read from an offset under a result limit for its toLine or its note: a page that counted its lines from the file's start, giving as toLine the number of lines it shows and sending a reader back to an earlier offset, passed every workspace test. The bounded line page test now checks, for the page that continues from its note's offset, that its toLine is its last shown line's number and that its note says to continue after it. Co-Authored-By: Claude Opus 5.5 --- test/workspace-read-text.test.ts | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/test/workspace-read-text.test.ts b/test/workspace-read-text.test.ts index f61ebacce..cd809113a 100644 --- a/test/workspace-read-text.test.ts +++ b/test/workspace-read-text.test.ts @@ -111,10 +111,14 @@ test('a line page stops before its result would pass the limit, and says where t const more = await read({ limit: page.toLine! + 1 }, 5_000_000); const withNext = { ...(result.data as object), toLine: page.toLine! + 1, content: (more.data as LinePage).content }; assert.ok(JSON.stringify(withNext).length > limit); - // Continuing reads on from the next line. + // Continuing reads on from the next line, ends at the line its last shown + // line is, and says to go on after it. const next = (await read({ offset: page.toLine! + 1 }, limit)).data as LinePage; assert.equal(next.fromLine, page.toLine! + 1); assert.ok(next.content.startsWith(`${String(page.toLine! + 1).padStart(6)}\tline ${page.toLine! + 1}:`)); + const lastShown = next.content.slice(next.content.lastIndexOf('\n') + 1); + assert.ok(lastShown.startsWith(`${String(next.toLine).padStart(6)}\tline ${next.toLine}:`), lastShown); + assert.equal(next.note, `Stopped at line ${next.toLine} to stay within the ${limit}-character result limit (file has 3000). Continue with offset=${next.toLine! + 1}.`); } }); From 348108655580c8de5d0834ee14fd9ca1858bf3be Mon Sep 17 00:00:00 2001 From: Abigail-2676 Date: Sun, 18 Oct 2026 18:36:43 -0400 Subject: [PATCH 18/51] docs(workspace): the page bound's figures, and comments a review misread An internal review read the changelog's figure for when a page passes a small limit as a bound, but a note can name a symlink's target, of any length. The changelog now gives the figures as measured at a limit of 1,000 with a first line cut to its least: a path of about 700 characters with no notes, and of 120 to 160 with all three notes on a file's text and a conflict's. It says a note naming a symlink's target takes as much room as the target is long. It also says a script's line pages stop at the line limit and its character pages at limitChars, where it said its pages "still stop at 2,000 lines or limitChars". Comments: linePage's fallback says the loop has already tried the first line as a page shows it, which is why that page bounds the search; a test comment names the two pages it compares; another says only the line page is read under a result limit. Co-Authored-By: Claude Opus 5.5 --- changelog.d/feat-workspace-read.changed.md | 2 +- src/modules/workspace/read-page.ts | 16 +++++++++------- test/workspace-read-text.test.ts | 5 +++-- test/workspace-three-way.test.ts | 4 ++-- 4 files changed, 15 insertions(+), 12 deletions(-) diff --git a/changelog.d/feat-workspace-read.changed.md b/changelog.d/feat-workspace-read.changed.md index 9e8aac777..45aa75404 100644 --- a/changelog.d/feat-workspace-read.changed.md +++ b/changelog.d/feat-workspace-read.changed.md @@ -1,6 +1,6 @@ - The workspace's `read` shows a file's text as it is, and says so. A line page shows a CRLF file's lines without the CR before each line feed, and leaves out a BOM. Its note says when the file has either, counting a mix of CRLF and LF ends. It also says when the file isn't valid UTF-8: each invalid byte sequence shows as U+FFFD, `edit` refuses the file, and `write` replaces it whole. Before, each line of a CRLF file ended in a CR, a BOM came first in the first line as U+FEFF, and invalid bytes showed as U+FFFD with no note. A character page still reads the stored text as it is, BOM and CRs included, and the tool's description now says so. - On a page for a call that carries a `resultMaxChars`, such as a model's or a running script's, a line over 2,000 characters is cut, with how many characters follow and the `offsetChars` that reads them. Before, a line page showed every line whole, however long. -- A `read` page stops before its result would pass `resultMaxChars`, the most of it the framework stores whole. For a model's call that's its agent's inline cap, 24,000 characters by default. Its note says where to continue: the `offset` after its last line, or for a character page the `offsetChars` it also gives as `nextOffsetChars`. Where even a page's first line would pass the limit, that line is cut shorter, at the most the page has room for. Where a page's last line is cut, its note names the `offsetChars` where that line's rest starts, before the next line's `offset` if the file goes on. A page always moves on, so where its own fields (the path and its notes) and the least it can show pass a small limit, it passes the limit: at 1,000 characters, only a path of about 130 characters or more beside every note a read can carry does that, or one of about 700 with none. Before, a page of up to 2,000 lines, or of `limitChars` characters, went out whole, and for a result that passed the limit the framework kept only its start, cut mid-JSON, writing the whole of it to a workspace file where the agent has one. A caller that pages 2,000 lines at a time and starts the next page at line 2,001 now starts where the note says. A running `code_execution` script's call has a limit of 5,000,000 characters, so its pages still stop at 2,000 lines or `limitChars`, unless a page would pass that. Its line pages cut a line over 2,000 characters and leave out CRs and a BOM, as a model's do, and a character page gives it the stored text exactly. A call made in process, through `handleToolCall`, `executeToolCall` or `ModuleContext.callTool`, carries no `resultMaxChars` unless its caller sets one. Its line pages stop at the line limit alone and show every line whole, as they did before, though now without the CRs and the BOM. +- A `read` page stops before its result would pass `resultMaxChars`, the most of it the framework stores whole. For a model's call that's its agent's inline cap, 24,000 characters by default. Its note says where to continue: the `offset` after its last line, or for a character page the `offsetChars` it also gives as `nextOffsetChars`. Where even a page's first line would pass the limit, that line is cut shorter, at the most the page has room for. Where a page's last line is cut, its note names the `offsetChars` where that line's rest starts, before the next line's `offset` if the file goes on. A page always moves on, so where its own fields (the path and its notes) and the least it can show pass a small limit, it passes the limit. At 1,000 characters, a page with no notes does that only for a path of about 700 characters or more. Its notes take room from that: with all three notes on a file's text and a conflict's, a path of 120 to 160 characters does it, and a note that names a symlink's target, as one saying disk couldn't be checked can, takes as much room as the target is long. Before, a page of up to 2,000 lines, or of `limitChars` characters, went out whole, and for a result that passed the limit the framework kept only its start, cut mid-JSON, writing the whole of it to a workspace file where the agent has one. A caller that pages 2,000 lines at a time and starts the next page at line 2,001 now starts where the note says. A running `code_execution` script's call has a limit of 5,000,000 characters, so its line pages still stop at the line limit, 2,000 by default, and its character pages at `limitChars`, unless a page would pass that. Its line pages cut a line over 2,000 characters and leave out CRs and a BOM, as a model's do, and a character page gives it the stored text exactly. A call made in process, through `handleToolCall`, `executeToolCall` or `ModuleContext.callTool`, carries no `resultMaxChars` unless its caller sets one. Its line pages stop at the line limit alone and show every line whole, as they did before, though now without the CRs and the BOM. - The workspace's `grep` leaves out a CR before a line feed, for matching as well as display, so `foo$` finds a CRLF file's line too, as it does under ripgrep's `--crlf`. A match's text and context are its lines without the CR, as `read` shows them, but each line whole, where a `read` page for a call with a result limit cuts a line over 2,000 characters. A BOM isn't part of the first line, and a file with CRLF ends says so in its result's `lineEndings` (`CRLF`, or `mixed`). Before, a CRLF file's lines ended in a CR, so `foo$` missed them and a match's text copied into an edit carried the CR. A pattern written to find the CR of a CRLF line end no longer matches; a CR that doesn't come before a line feed is still part of its line. - The workspace's `read` first takes in a change on disk to a file the workspace holds, as a listing and an `edit` do, and shows that text. Its note says so when disk's copy had changed since the workspace last agreed with it. It also says when disk's copy couldn't be checked, or disk has none and nothing records whether it was deleted there, so it shows the workspace's copy. A file disk deleted answers that disk deleted it and the deletion was taken in, and a file disk made binary or too large answers that it's on disk but not stored. Where the path's stored copy is a draft and disk changed or deleted it too, the read shows the draft, and its note names the conflict and the ways to settle it, a sync, or a forced materialize on a mount that isn't read-only: for a deletion, a sync of the path takes it and removes the workspace's copy. Before, a read of a path the workspace held didn't look at disk, so it could show a copy another writer had since changed; a path it didn't hold was observed, as now. - The `read` and `grep` tool descriptions changed, so the first request after the upgrade that carries the workspace's tools misses the prompt cache from its tool block on. diff --git a/src/modules/workspace/read-page.ts b/src/modules/workspace/read-page.ts index 8b97ecaff..2ebab6b5e 100644 --- a/src/modules/workspace/read-page.ts +++ b/src/modules/workspace/read-page.ts @@ -110,13 +110,15 @@ export function linePage(path: string, file: FileText, offset: number, limit: nu shownLines.length = best; return { success: true, data: page(first + best, shownLines.join('\n'), noteAt(first + best)) }; } - // No page fits with its first line as a page shows it, so that line is cut - // shorter, at the most the page has room for, and ends the page: its note - // names where the line's rest starts, since that comes next. Measured on - // the page itself, since its note names the cut. Shown whole, a line carries - // no marker and its note no cut, so its page can fit where a cut one - // character shorter doesn't. Below that a longer cut never makes the page - // shorter, so the first that doesn't fit bounds the search. + // No page fits with its first line as a page shows it: the loop tried that + // page first, since a read's limit is at least 1, and it is pageWith(over). + // So the line is cut shorter, at the most the page has room for, and ends + // the page: its note names where the line's rest starts, since that comes + // next. Each cut is measured on the page itself, since its note names the + // cut. Every cut below `over` shows a marker, and a longer one never makes + // the page shorter: the text gains at least a character where the marker's + // count loses at most a digit, and the offsets only grow. So the search + // finds the longest cut that fits. const pageWith = (chars: number) => page(first + 1, shown(first, chars), noteAt(first + 1, chars)); let fits = 0; let over = Math.min(lines[first].length, MAX_LINE_CHARS); diff --git a/test/workspace-read-text.test.ts b/test/workspace-read-text.test.ts index cd809113a..e61367799 100644 --- a/test/workspace-read-text.test.ts +++ b/test/workspace-read-text.test.ts @@ -140,8 +140,9 @@ test('a line page is the longest that fits, though the page that ends where it w const atLimit = await readLimit({ limit: 2 }, 5_000_000); assert.equal((atLimit.data as LinePage).note, 'Truncated at 2 lines (file has 3). Use offset/limit to read more.'); assert.deepEqual((await readLimit({ limit: 2 }, stored(atLimit))).data, atLimit.data); - // A page whose last line is cut names that line's rest, in more words than - // a page whose last line is whole, so the page a line longer can fit where it doesn't. + // The page that ends at a cut line names that line's rest, in more words + // than a page that ends at a whole line: so the page of lines 1 and 2 fits + // where the page of line 1 alone, cut at 2,000 characters, doesn't. const { read: readCut } = await setup(t, `${'q'.repeat(2500)}\nz\n${'w'.repeat(300)}\n`); const two = (limit: number) => ({ path: 'work/text.txt', totalLines: 3, fromLine: 1, toLine: 2, diff --git a/test/workspace-three-way.test.ts b/test/workspace-three-way.test.ts index c75a9900d..e35de07a8 100644 --- a/test/workspace-three-way.test.ts +++ b/test/workspace-three-way.test.ts @@ -7046,8 +7046,8 @@ describe('on-agent-action', () => { () => call(m, 'read', { path: 'work/a.txt' })); assert.equal(data.content, ' 1\tA'); assert.equal(data.note, "The path couldn't be checked against disk (the selected branch kept changing while the scan ran), so this read shows the workspace's copy."); - // A character page and a line page under a result limit say so too, - // each read starting on main, as the first did. + // A character page says so too, and so does a line page under a result + // limit, each read starting on main, as the first did. const switching = (n: number) => env.store.switchBranch(n % 2 === 1 ? 'other' : main); env.store.switchBranch(main); const chars = await withReadHook(switching, () => call(m, 'read', { path: 'work/a.txt', offsetChars: 0, limitChars: 10 })); From c7cdbbd3f0524168c0708704c2623048063f1ac0 Mon Sep 17 00:00:00 2001 From: Abigail-2676 Date: Sun, 18 Oct 2026 21:05:07 -0400 Subject: [PATCH 19/51] fix(workspace): a line whose own text ends in a CR shows its line end's CR too An internal review found that a read of a file whose lines end in CR CR LF showed each such line with its own CR alone, as "x\r", and that text copied from it no longer matched where it came from: edit reads a CR before a line feed in its old text as part of the line end, so "x\r\ny\r", copied from lines 1 and 2 of "x\r\r\ny\r\r\nx\r\ny\r\n", edited lines 3 and 4, and with no such lines later the edit was refused as missing. Before this change, when read showed every line as stored, the same copy edited lines 1 and 2. fileText now keeps the line end's CR on a line whose own text ends in a CR, so the line shows as stored, "x\r\r", and counts such lines. The read's note on CRLF ends says how many lines show both CRs. grep, which searches the same lines, shows them the same way. Tests: fileText's lines and counts for such a file (test/workspace-text.test.ts); and, in test/workspace-file-tools.test.ts, two lines copied from a read of that file editing lines 1 and 2, and a grep match's text editing its own line. Co-Authored-By: Claude Opus 5.5 --- changelog.d/feat-workspace-read.changed.md | 2 +- src/modules/workspace/read-page.ts | 8 ++++++-- src/modules/workspace/text.ts | 14 ++++++++++--- test/workspace-file-tools.test.ts | 23 ++++++++++++++++++++++ test/workspace-text.test.ts | 7 +++++++ 5 files changed, 48 insertions(+), 6 deletions(-) diff --git a/changelog.d/feat-workspace-read.changed.md b/changelog.d/feat-workspace-read.changed.md index 45aa75404..293690ddd 100644 --- a/changelog.d/feat-workspace-read.changed.md +++ b/changelog.d/feat-workspace-read.changed.md @@ -1,4 +1,4 @@ -- The workspace's `read` shows a file's text as it is, and says so. A line page shows a CRLF file's lines without the CR before each line feed, and leaves out a BOM. Its note says when the file has either, counting a mix of CRLF and LF ends. It also says when the file isn't valid UTF-8: each invalid byte sequence shows as U+FFFD, `edit` refuses the file, and `write` replaces it whole. Before, each line of a CRLF file ended in a CR, a BOM came first in the first line as U+FEFF, and invalid bytes showed as U+FFFD with no note. A character page still reads the stored text as it is, BOM and CRs included, and the tool's description now says so. +- The workspace's `read` shows a file's text as it is, and says so. A line page shows a CRLF file's lines without the CR before each line feed, and leaves out a BOM. A line whose own text ends in a CR shows its line end's CR too, since `edit` reads a CR before a line feed as part of the line end, so text copied from the line still matches it. Its note says when the file has CRLF ends or a BOM, counting a mix of CRLF and LF ends and the lines that show both CRs. It also says when the file isn't valid UTF-8: each invalid byte sequence shows as U+FFFD, `edit` refuses the file, and `write` replaces it whole. Before, each line of a CRLF file ended in a CR, a BOM came first in the first line as U+FEFF, and invalid bytes showed as U+FFFD with no note. A character page still reads the stored text as it is, BOM and CRs included, and the tool's description now says so. - On a page for a call that carries a `resultMaxChars`, such as a model's or a running script's, a line over 2,000 characters is cut, with how many characters follow and the `offsetChars` that reads them. Before, a line page showed every line whole, however long. - A `read` page stops before its result would pass `resultMaxChars`, the most of it the framework stores whole. For a model's call that's its agent's inline cap, 24,000 characters by default. Its note says where to continue: the `offset` after its last line, or for a character page the `offsetChars` it also gives as `nextOffsetChars`. Where even a page's first line would pass the limit, that line is cut shorter, at the most the page has room for. Where a page's last line is cut, its note names the `offsetChars` where that line's rest starts, before the next line's `offset` if the file goes on. A page always moves on, so where its own fields (the path and its notes) and the least it can show pass a small limit, it passes the limit. At 1,000 characters, a page with no notes does that only for a path of about 700 characters or more. Its notes take room from that: with all three notes on a file's text and a conflict's, a path of 120 to 160 characters does it, and a note that names a symlink's target, as one saying disk couldn't be checked can, takes as much room as the target is long. Before, a page of up to 2,000 lines, or of `limitChars` characters, went out whole, and for a result that passed the limit the framework kept only its start, cut mid-JSON, writing the whole of it to a workspace file where the agent has one. A caller that pages 2,000 lines at a time and starts the next page at line 2,001 now starts where the note says. A running `code_execution` script's call has a limit of 5,000,000 characters, so its line pages still stop at the line limit, 2,000 by default, and its character pages at `limitChars`, unless a page would pass that. Its line pages cut a line over 2,000 characters and leave out CRs and a BOM, as a model's do, and a character page gives it the stored text exactly. A call made in process, through `handleToolCall`, `executeToolCall` or `ModuleContext.callTool`, carries no `resultMaxChars` unless its caller sets one. Its line pages stop at the line limit alone and show every line whole, as they did before, though now without the CRs and the BOM. - The workspace's `grep` leaves out a CR before a line feed, for matching as well as display, so `foo$` finds a CRLF file's line too, as it does under ripgrep's `--crlf`. A match's text and context are its lines without the CR, as `read` shows them, but each line whole, where a `read` page for a call with a result limit cuts a line over 2,000 characters. A BOM isn't part of the first line, and a file with CRLF ends says so in its result's `lineEndings` (`CRLF`, or `mixed`). Before, a CRLF file's lines ended in a CR, so `foo$` missed them and a match's text copied into an edit carried the CR. A pattern written to find the CR of a CRLF line end no longer matches; a CR that doesn't come before a line feed is still part of its line. diff --git a/src/modules/workspace/read-page.ts b/src/modules/workspace/read-page.ts index 2ebab6b5e..383c84820 100644 --- a/src/modules/workspace/read-page.ts +++ b/src/modules/workspace/read-page.ts @@ -15,9 +15,13 @@ export const DEFAULT_READ_LINE_LIMIT = 2000; export function textNotes(file: FileText): string[] { const notes: string[] = []; if (file.crlfEnds > 0) { + // A line whose own text ends in a CR shows its line end's CR too (fileText). + const kept = file.crKept > 0 + ? `, except on ${file.crKept} ${file.crKept === 1 ? 'line whose own text ends in a CR, which shows' : 'lines whose own text ends in a CR, which show'} both CRs` + : ''; notes.push(file.lfEnds > 0 - ? `Line ends are mixed, ${file.crlfEnds} CRLF and ${file.lfEnds} LF, and a CRLF end is shown without its CR. An edit matches text as read shows it, and keeps each line's end.` - : 'Line ends are CRLF, shown without the CR. An edit matches text as read shows it, and keeps the CRLF ends.'); + ? `Line ends are mixed, ${file.crlfEnds} CRLF and ${file.lfEnds} LF, and a CRLF end is shown without its CR${kept}. An edit matches text as read shows it, and keeps each line's end.` + : `Line ends are CRLF, shown without the CR${kept}. An edit matches text as read shows it, and keeps the CRLF ends.`); } if (file.bom) notes.push('The file starts with a UTF-8 BOM, which isn\'t shown. An edit keeps it.'); if (!file.utf8) notes.push(NOT_UTF8); diff --git a/src/modules/workspace/text.ts b/src/modules/workspace/text.ts index 4fb9e17fc..dc9298e21 100644 --- a/src/modules/workspace/text.ts +++ b/src/modules/workspace/text.ts @@ -19,7 +19,10 @@ export interface FileText { * Its lines, as read shows them and grep searches them: without the BOM, a * CR before a line feed dropped, and a final line end ending the last line * rather than starting an empty one. So "a\r\nb\r\n" is two lines, and an - * empty text has none. + * empty text has none. A line whose own text ends in a CR keeps its line + * end's CR too, as stored: an edit reads a CR before a line feed in its old + * text as part of the line end, so text copied from "a\r" and the next line + * would lose the CR that is the line's own. */ lines: string[]; /** Where each line starts in `raw`. */ @@ -28,6 +31,8 @@ export interface FileText { /** How many line ends were CRLF, and how many a bare LF. */ crlfEnds: number; lfEnds: number; + /** How many lines with a CRLF end keep its CR, since their own text ends in a CR. */ + crKept: number; /** Whether the bytes are valid UTF-8; where they aren't, `raw` holds U+FFFD. */ utf8: boolean; } @@ -39,6 +44,7 @@ export function fileText(bytes: Buffer): FileText { const starts: number[] = []; let crlfEnds = 0; let lfEnds = 0; + let crKept = 0; // A BOM decodes to the single code unit U+FEFF. for (let at = bom ? 1 : 0; at < raw.length;) { const lf = raw.indexOf('\n', at); @@ -49,10 +55,12 @@ export function fileText(bytes: Buffer): FileText { } const crlf = lf > at && raw.charCodeAt(lf - 1) === 13; if (crlf) crlfEnds++; else lfEnds++; - lines.push(raw.slice(at, crlf ? lf - 1 : lf)); + const keepsCr = crlf && lf - 1 > at && raw.charCodeAt(lf - 2) === 13; + if (keepsCr) crKept++; + lines.push(raw.slice(at, crlf && !keepsCr ? lf - 1 : lf)); at = lf + 1; } - return { raw, lines, starts, bom, crlfEnds, lfEnds, utf8: isUtf8(bytes) }; + return { raw, lines, starts, bom, crlfEnds, lfEnds, crKept, utf8: isUtf8(bytes) }; } /** How long a line read shows before it cuts the line. */ diff --git a/test/workspace-file-tools.test.ts b/test/workspace-file-tools.test.ts index cce69174e..3a3558968 100644 --- a/test/workspace-file-tools.test.ts +++ b/test/workspace-file-tools.test.ts @@ -435,6 +435,29 @@ test('text copied from a read or a grep of a CRLF file matches in an edit, and t assert.deepEqual(disk('w.txt'), Buffer.concat([bom, Buffer.from('ONE\r\nTWO\r\nTHREE\r\n')])); }); +test('lines whose own text ends in a CR, copied from a read, edit the lines they came from', async (t) => { + const { dir, call, disk } = setup(t, { autoMaterialize: true }); + // Lines 1 and 2 end in a CR before their CRLF end; lines 3 and 4 hold the + // same text with a CRLF end alone. + writeFileSync(join(dir, 'cr.txt'), 'x\r\r\ny\r\r\nx\r\ny\r\n'); + const page = (await call('read', { path: 'work/cr.txt' })).data as { content: string; note: string }; + assert.equal(page.note, 'Line ends are CRLF, shown without the CR, except on 2 lines whose own text ends in a CR, which show both CRs. ' + + 'An edit matches text as read shows it, and keeps the CRLF ends.'); + const copied = page.content.split('\n').slice(0, 2).map((line) => line.slice(line.indexOf('\t') + 1)).join('\n'); + assert.equal(copied, 'x\r\r\ny\r\r'); + const edit = await call('edit', { path: 'work/cr.txt', oldString: copied, newString: 'Q\nR' }); + assert.equal(edit.success, true, edit.error); + assert.equal(disk('cr.txt')!.toString(), 'Q\r\nR\r\nx\r\ny\r\n'); + // A grep match's text copies the same way. + writeFileSync(join(dir, 'one.txt'), 'w\r\r\nw\r\n'); + const found = await call('grep', { pattern: '^w', path: 'work/one.txt' }); + const texts = (found.data as { results: Array<{ matches: Array<{ text: string }> }> }).results[0].matches.map((m) => m.text); + assert.deepEqual(texts, ['w\r\r', 'w']); + const second = await call('edit', { path: 'work/one.txt', oldString: texts[0], newString: 'V' }); + assert.equal(second.success, true, second.error); + assert.equal(disk('one.txt')!.toString(), 'V\r\nw\r\n'); +}); + test('an offset or limit that is not a whole number of at least 1 is refused', async (t) => { const { dir, call } = setup(t); writeFileSync(join(dir, 'nl.txt'), 'a\nb\n'); diff --git a/test/workspace-text.test.ts b/test/workspace-text.test.ts index eb477beb6..84421b1f4 100644 --- a/test/workspace-text.test.ts +++ b/test/workspace-text.test.ts @@ -15,6 +15,13 @@ test('fileText gives the lines read shows, where each starts, and what the text assert.deepEqual(lines('a\r\nb\r\n'), ['a', 'b']); assert.deepEqual(lines('a\rb\r\n\r\n'), ['a\rb', '']); assert.deepEqual(lines('a\r'), ['a\r'], 'a CR that ends the text ends no line'); + // A line whose own text ends in a CR keeps its line end's CR too, so text + // copied from it keeps the CR an edit would read as part of the line end. + const crKept = fileText(Buffer.from('x\r\r\ny\r\nz\r\r\n')); + assert.deepEqual(crKept.lines, ['x\r\r', 'y', 'z\r\r']); + assert.deepEqual([crKept.crlfEnds, crKept.lfEnds, crKept.crKept], [3, 0, 2]); + assert.equal(fileText(Buffer.from('\r\r\n')).crKept, 1); + assert.equal(fileText(Buffer.from('a\r\n\r\n')).crKept, 0, 'an empty line after a CRLF end is no CR of its own'); const crlf = fileText(Buffer.from('one\r\ntwo\nthree')); assert.deepEqual(crlf.starts, [0, 5, 9]); From 8b2865218245e161ecce6a41c38707ba0a1769f2 Mon Sep 17 00:00:00 2001 From: Abigail-2676 Date: Sun, 18 Oct 2026 21:05:08 -0400 Subject: [PATCH 20/51] fix(workspace): a cut line's note gives the limitChars that reads its rest An internal review found that a page whose last line is cut told the reader to read its rest with offsetChars=R, then continue with offset=X+1. A character read given offsetChars alone takes 2,000 code units, so it ran past the cut line into the lines after it, which the next page then showed again: on a file of long lines, up to a page of text twice for every page. The note now gives limitChars as the length of the line's rest as a page shows it, the count its marker gives, so the rest-read ends with the line. The note is longer by that clause, so the figures for a small result limit in the changelog are measured again: at 1,000 characters, a page with no notes passes the limit for a path of about 690 characters, and with every note a file's text and a conflict can carry, of 100 to 140. Tests in test/workspace-read-text.test.ts pin the new notes. Co-Authored-By: Claude Opus 5.5 --- changelog.d/feat-workspace-read.changed.md | 2 +- src/modules/workspace/read-page.ts | 16 ++++++++++------ test/workspace-read-text.test.ts | 16 ++++++++-------- 3 files changed, 19 insertions(+), 15 deletions(-) diff --git a/changelog.d/feat-workspace-read.changed.md b/changelog.d/feat-workspace-read.changed.md index 293690ddd..1e7f4fffc 100644 --- a/changelog.d/feat-workspace-read.changed.md +++ b/changelog.d/feat-workspace-read.changed.md @@ -1,6 +1,6 @@ - The workspace's `read` shows a file's text as it is, and says so. A line page shows a CRLF file's lines without the CR before each line feed, and leaves out a BOM. A line whose own text ends in a CR shows its line end's CR too, since `edit` reads a CR before a line feed as part of the line end, so text copied from the line still matches it. Its note says when the file has CRLF ends or a BOM, counting a mix of CRLF and LF ends and the lines that show both CRs. It also says when the file isn't valid UTF-8: each invalid byte sequence shows as U+FFFD, `edit` refuses the file, and `write` replaces it whole. Before, each line of a CRLF file ended in a CR, a BOM came first in the first line as U+FEFF, and invalid bytes showed as U+FFFD with no note. A character page still reads the stored text as it is, BOM and CRs included, and the tool's description now says so. - On a page for a call that carries a `resultMaxChars`, such as a model's or a running script's, a line over 2,000 characters is cut, with how many characters follow and the `offsetChars` that reads them. Before, a line page showed every line whole, however long. -- A `read` page stops before its result would pass `resultMaxChars`, the most of it the framework stores whole. For a model's call that's its agent's inline cap, 24,000 characters by default. Its note says where to continue: the `offset` after its last line, or for a character page the `offsetChars` it also gives as `nextOffsetChars`. Where even a page's first line would pass the limit, that line is cut shorter, at the most the page has room for. Where a page's last line is cut, its note names the `offsetChars` where that line's rest starts, before the next line's `offset` if the file goes on. A page always moves on, so where its own fields (the path and its notes) and the least it can show pass a small limit, it passes the limit. At 1,000 characters, a page with no notes does that only for a path of about 700 characters or more. Its notes take room from that: with all three notes on a file's text and a conflict's, a path of 120 to 160 characters does it, and a note that names a symlink's target, as one saying disk couldn't be checked can, takes as much room as the target is long. Before, a page of up to 2,000 lines, or of `limitChars` characters, went out whole, and for a result that passed the limit the framework kept only its start, cut mid-JSON, writing the whole of it to a workspace file where the agent has one. A caller that pages 2,000 lines at a time and starts the next page at line 2,001 now starts where the note says. A running `code_execution` script's call has a limit of 5,000,000 characters, so its line pages still stop at the line limit, 2,000 by default, and its character pages at `limitChars`, unless a page would pass that. Its line pages cut a line over 2,000 characters and leave out CRs and a BOM, as a model's do, and a character page gives it the stored text exactly. A call made in process, through `handleToolCall`, `executeToolCall` or `ModuleContext.callTool`, carries no `resultMaxChars` unless its caller sets one. Its line pages stop at the line limit alone and show every line whole, as they did before, though now without the CRs and the BOM. +- A `read` page stops before its result would pass `resultMaxChars`, the most of it the framework stores whole. For a model's call that's its agent's inline cap, 24,000 characters by default. Its note says where to continue: the `offset` after its last line, or for a character page the `offsetChars` it also gives as `nextOffsetChars`. Where even a page's first line would pass the limit, that line is cut shorter, at the most the page has room for. Where a page's last line is cut, its note names the `offsetChars` where that line's rest starts and the `limitChars` that reads it to the line's end, before the next line's `offset` if the file goes on. A page always moves on, so where its own fields (the path and its notes) and the least it can show pass a small limit, it passes the limit. At 1,000 characters, a page with no notes does that only for a path of about 690 characters or more. Its notes take room from that: with all three notes on a file's text and a conflict's, a path of 100 to 140 characters does it, and a note that names a symlink's target, as one saying disk couldn't be checked can, takes as much room as the target is long. Before, a page of up to 2,000 lines, or of `limitChars` characters, went out whole, and for a result that passed the limit the framework kept only its start, cut mid-JSON, writing the whole of it to a workspace file where the agent has one. A caller that pages 2,000 lines at a time and starts the next page at line 2,001 now starts where the note says. A running `code_execution` script's call has a limit of 5,000,000 characters, so its line pages still stop at the line limit, 2,000 by default, and its character pages at `limitChars`, unless a page would pass that. Its line pages cut a line over 2,000 characters and leave out CRs and a BOM, as a model's do, and a character page gives it the stored text exactly. A call made in process, through `handleToolCall`, `executeToolCall` or `ModuleContext.callTool`, carries no `resultMaxChars` unless its caller sets one. Its line pages stop at the line limit alone and show every line whole, as they did before, though now without the CRs and the BOM. - The workspace's `grep` leaves out a CR before a line feed, for matching as well as display, so `foo$` finds a CRLF file's line too, as it does under ripgrep's `--crlf`. A match's text and context are its lines without the CR, as `read` shows them, but each line whole, where a `read` page for a call with a result limit cuts a line over 2,000 characters. A BOM isn't part of the first line, and a file with CRLF ends says so in its result's `lineEndings` (`CRLF`, or `mixed`). Before, a CRLF file's lines ended in a CR, so `foo$` missed them and a match's text copied into an edit carried the CR. A pattern written to find the CR of a CRLF line end no longer matches; a CR that doesn't come before a line feed is still part of its line. - The workspace's `read` first takes in a change on disk to a file the workspace holds, as a listing and an `edit` do, and shows that text. Its note says so when disk's copy had changed since the workspace last agreed with it. It also says when disk's copy couldn't be checked, or disk has none and nothing records whether it was deleted there, so it shows the workspace's copy. A file disk deleted answers that disk deleted it and the deletion was taken in, and a file disk made binary or too large answers that it's on disk but not stored. Where the path's stored copy is a draft and disk changed or deleted it too, the read shows the draft, and its note names the conflict and the ways to settle it, a sync, or a forced materialize on a mount that isn't read-only: for a deletion, a sync of the path takes it and removes the workspace's copy. Before, a read of a path the workspace held didn't look at disk, so it could show a copy another writer had since changed; a path it didn't hold was observed, as now. - The `read` and `grep` tool descriptions changed, so the first request after the upgrade that carries the workspace's tools misses the prompt cache from its tool block on. diff --git a/src/modules/workspace/read-page.ts b/src/modules/workspace/read-page.ts index 383c84820..4b6ed1499 100644 --- a/src/modules/workspace/read-page.ts +++ b/src/modules/workspace/read-page.ts @@ -41,11 +41,12 @@ const joined = (notes: Array): string | undefined => { * longest that fits within `maxChars` as JSON; where none fits with its first * line as a page shows it, that line is cut shorter, at the most the page has * room for. Where the page's last line is cut, its note names where that - * line's rest starts, since reading on in order starts there: before the next - * line, or, on the page that ends the file, before the file is read whole. A - * page whose own fields and least content pass `maxChars` still moves on, and - * passes it. With no `maxChars`, for a call made in process, which takes the - * data itself, the page holds to `limit` alone and shows every line whole. + * line's rest starts and how long it is, since reading on in order starts + * there: before the next line, or, on the page that ends the file, before the + * file is read whole. A page whose own fields and least content pass + * `maxChars` still moves on, and passes it. With no `maxChars`, for a call + * made in process, which takes the data itself, the page holds to `limit` + * alone and shows every line whole. */ export function linePage(path: string, file: FileText, offset: number, limit: number, maxChars: number | undefined, extraNotes: string[] = []): ToolResult { const { lines, starts } = file; @@ -69,7 +70,10 @@ export function linePage(path: string, file: FileText, offset: number, limit: nu // maxLineChars long, as its marker says; null for a line shown whole. const restOf = (i: number, maxLineChars: number): number | null => lines[i].length > maxLineChars ? starts[i] + safeCut(lines[i], maxLineChars) : null; - const cutLine = (to: number, rest: number) => `Line ${to} is cut: read its rest with offsetChars=${rest}`; + // The rest's length is what's left of the line as a page shows it, so a + // character read of it ends where the line does. + const cutLine = (to: number, rest: number) => + `Line ${to} is cut: read its rest with offsetChars=${rest} and limitChars=${starts[to - 1]! + lines[to - 1]!.length - rest}`; const limitNote = (to: number, rest: number | null) => `Truncated at ${limit} lines (file has ${total}).${rest === null ? '' : ` ${cutLine(to, rest)}.`} Use offset/limit to read more.`; const budgetNote = (to: number, rest: number | null) => diff --git a/test/workspace-read-text.test.ts b/test/workspace-read-text.test.ts index e61367799..b26dc4600 100644 --- a/test/workspace-read-text.test.ts +++ b/test/workspace-read-text.test.ts @@ -95,7 +95,7 @@ test('a long line is cut at 2000 characters, with the offsetChars that reads the assert.equal(lines[2], ' 3\tlast'); // Where the cut line ends the page, its note names the offset its marker does. const two = (await read({ limit: 2 }, 24_000)).data as LinePage; - assert.ok(two.note!.startsWith(`Truncated at 2 lines (file has 3). Line 2 is cut: read its rest with offsetChars=${at}. Use offset/limit to read more.`), two.note); + assert.ok(two.note!.startsWith(`Truncated at 2 lines (file has 3). Line 2 is cut: read its rest with offsetChars=${at} and limitChars=${rest}. Use offset/limit to read more.`), two.note); }); test('a line page stops before its result would pass the limit, and says where to continue', async t => { @@ -203,14 +203,14 @@ test('a first line that alone would pass the limit is cut where it fits', async assert.equal(Number(at), shown.length); assert.equal(shown.length + Number(rest), 1500); // Reading on in order means the rest of line 1 first, so the note names it. - assert.equal(page.note, `Stopped at line 1 to stay within the 1000-character result limit (file has 2). Line 1 is cut: read its rest with offsetChars=${at}, then continue with offset=2.`); + assert.equal(page.note, `Stopped at line 1 to stay within the 1000-character result limit (file has 2). Line 1 is cut: read its rest with offsetChars=${at} and limitChars=${rest}, then continue with offset=2.`); // The cut is the most the page has room for: one character more passes the // limit, on a page that goes on and on one that ends the file. const oneMore = (cutPage: LinePage, chars: number) => { const more = chars + 1; return JSON.stringify({ ...cutPage, - ...(cutPage.note !== undefined ? { note: cutPage.note.replace(/offsetChars=\d+/, `offsetChars=${more}`) } : {}), + ...(cutPage.note !== undefined ? { note: cutPage.note.replace(/offsetChars=\d+ and limitChars=\d+/, `offsetChars=${more} and limitChars=${1500 - more}`) } : {}), content: ` 1\t${'"'.repeat(more)} [… ${1500 - more} more chars; read them with offsetChars=${more}]`, }).length; }; @@ -222,7 +222,7 @@ test('a first line that alone would pass the limit is cut where it fits', async const lastShown = /^ {5}1\t("*) \[/.exec(lastPage.content)![1].length; // The page ends the file, and its last line is cut, so its note names the rest. assert.equal(lastPage.toLine, lastPage.totalLines); - assert.equal(lastPage.note, `Line 1 is cut: read its rest with offsetChars=${lastShown}.`); + assert.equal(lastPage.note, `Line 1 is cut: read its rest with offsetChars=${lastShown} and limitChars=${1500 - lastShown}.`); assert.ok(oneMore(lastPage, lastShown) > 1_000, `${oneMore(lastPage, lastShown)}`); // Shown whole, a line carries no marker and its note no cut, so its page can // fit where a cut one character shorter doesn't: it is shown whole. @@ -234,7 +234,7 @@ test('a first line that alone would pass the limit is cut where it fits', async const tiny = (await read({}, 50)).data as LinePage; assert.equal(tiny.toLine, 1); assert.equal(tiny.content, ' 1\t [… 1500 more chars; read them with offsetChars=0]'); - assert.match(tiny.note!, /Line 1 is cut: read its rest with offsetChars=0, then continue with offset=2\.$/); + assert.match(tiny.note!, /Line 1 is cut: read its rest with offsetChars=0 and limitChars=1500, then continue with offset=2\.$/); }); test('a page whose last line is cut names where its rest starts before the next line', async t => { @@ -244,17 +244,17 @@ test('a page whose last line is cut names where its rest starts before the next // The limit stops the page at the cut line. const byLimit = (await read({ limit: 2 }, 24_000)).data as LinePage; assert.equal(byLimit.toLine, 2); - assert.equal(byLimit.note, `Truncated at 2 lines (file has 4). Line 2 is cut: read its rest with offsetChars=${at(2)}. Use offset/limit to read more.`); + assert.equal(byLimit.note, `Truncated at 2 lines (file has 4). Line 2 is cut: read its rest with offsetChars=${at(2)} and limitChars=1000. Use offset/limit to read more.`); // The result limit stops it there: line 4 would pass 5,000 characters. const byBudget = (await read({}, 5_000)).data as LinePage; assert.equal(byBudget.toLine, 3); - assert.equal(byBudget.note, `Stopped at line 3 to stay within the 5000-character result limit (file has 4). Line 3 is cut: read its rest with offsetChars=${at(3)}, then continue with offset=4.`); + assert.equal(byBudget.note, `Stopped at line 3 to stay within the 5000-character result limit (file has 4). Line 3 is cut: read its rest with offsetChars=${at(3)} and limitChars=1000, then continue with offset=4.`); // A cut line inside a page has its rest named by its own marker alone. The // page that ends the file names its cut last line too: toLine is the last // line, but the file isn't read whole until that line's rest is. const whole = (await read({}, 24_000)).data as LinePage; assert.equal(whole.toLine, 4); - assert.equal(whole.note, `Line 4 is cut: read its rest with offsetChars=${at(4)}.`); + assert.equal(whole.note, `Line 4 is cut: read its rest with offsetChars=${at(4)} and limitChars=1000.`); // A page that ends the file on a line shown whole has nothing to say. const { read: readShort } = await setup(t, `${long}\nshort\n`); assert.equal(((await readShort({}, 24_000)).data as LinePage).note, undefined); From 93216a7e6876ea9ce47ff8e60ac0d42fae6ce6d6 Mon Sep 17 00:00:00 2001 From: Abigail-2676 Date: Sun, 18 Oct 2026 21:05:09 -0400 Subject: [PATCH 21/51] fix(workspace): a read whose disk check gave up says to read again An internal review found that a read whose disk check gave up, because the selected branch kept changing while the scan ran, gave the scan's reason and nothing to do, where grep's answer to the same says to grep again. The read's answer now adds "Read again to check it.", for a held file and for one the workspace doesn't hold. Tests in test/workspace-three-way.test.ts pin both answers. Co-Authored-By: Claude Opus 5.5 --- changelog.d/feat-workspace-read.changed.md | 2 +- src/modules/workspace/index.ts | 10 +++++++--- test/workspace-three-way.test.ts | 4 ++-- 3 files changed, 10 insertions(+), 6 deletions(-) diff --git a/changelog.d/feat-workspace-read.changed.md b/changelog.d/feat-workspace-read.changed.md index 1e7f4fffc..9309e1787 100644 --- a/changelog.d/feat-workspace-read.changed.md +++ b/changelog.d/feat-workspace-read.changed.md @@ -2,5 +2,5 @@ - On a page for a call that carries a `resultMaxChars`, such as a model's or a running script's, a line over 2,000 characters is cut, with how many characters follow and the `offsetChars` that reads them. Before, a line page showed every line whole, however long. - A `read` page stops before its result would pass `resultMaxChars`, the most of it the framework stores whole. For a model's call that's its agent's inline cap, 24,000 characters by default. Its note says where to continue: the `offset` after its last line, or for a character page the `offsetChars` it also gives as `nextOffsetChars`. Where even a page's first line would pass the limit, that line is cut shorter, at the most the page has room for. Where a page's last line is cut, its note names the `offsetChars` where that line's rest starts and the `limitChars` that reads it to the line's end, before the next line's `offset` if the file goes on. A page always moves on, so where its own fields (the path and its notes) and the least it can show pass a small limit, it passes the limit. At 1,000 characters, a page with no notes does that only for a path of about 690 characters or more. Its notes take room from that: with all three notes on a file's text and a conflict's, a path of 100 to 140 characters does it, and a note that names a symlink's target, as one saying disk couldn't be checked can, takes as much room as the target is long. Before, a page of up to 2,000 lines, or of `limitChars` characters, went out whole, and for a result that passed the limit the framework kept only its start, cut mid-JSON, writing the whole of it to a workspace file where the agent has one. A caller that pages 2,000 lines at a time and starts the next page at line 2,001 now starts where the note says. A running `code_execution` script's call has a limit of 5,000,000 characters, so its line pages still stop at the line limit, 2,000 by default, and its character pages at `limitChars`, unless a page would pass that. Its line pages cut a line over 2,000 characters and leave out CRs and a BOM, as a model's do, and a character page gives it the stored text exactly. A call made in process, through `handleToolCall`, `executeToolCall` or `ModuleContext.callTool`, carries no `resultMaxChars` unless its caller sets one. Its line pages stop at the line limit alone and show every line whole, as they did before, though now without the CRs and the BOM. - The workspace's `grep` leaves out a CR before a line feed, for matching as well as display, so `foo$` finds a CRLF file's line too, as it does under ripgrep's `--crlf`. A match's text and context are its lines without the CR, as `read` shows them, but each line whole, where a `read` page for a call with a result limit cuts a line over 2,000 characters. A BOM isn't part of the first line, and a file with CRLF ends says so in its result's `lineEndings` (`CRLF`, or `mixed`). Before, a CRLF file's lines ended in a CR, so `foo$` missed them and a match's text copied into an edit carried the CR. A pattern written to find the CR of a CRLF line end no longer matches; a CR that doesn't come before a line feed is still part of its line. -- The workspace's `read` first takes in a change on disk to a file the workspace holds, as a listing and an `edit` do, and shows that text. Its note says so when disk's copy had changed since the workspace last agreed with it. It also says when disk's copy couldn't be checked, or disk has none and nothing records whether it was deleted there, so it shows the workspace's copy. A file disk deleted answers that disk deleted it and the deletion was taken in, and a file disk made binary or too large answers that it's on disk but not stored. Where the path's stored copy is a draft and disk changed or deleted it too, the read shows the draft, and its note names the conflict and the ways to settle it, a sync, or a forced materialize on a mount that isn't read-only: for a deletion, a sync of the path takes it and removes the workspace's copy. Before, a read of a path the workspace held didn't look at disk, so it could show a copy another writer had since changed; a path it didn't hold was observed, as now. +- The workspace's `read` first takes in a change on disk to a file the workspace holds, as a listing and an `edit` do, and shows that text. Its note says so when disk's copy had changed since the workspace last agreed with it. It also says when disk's copy couldn't be checked, or disk has none and nothing records whether it was deleted there, so it shows the workspace's copy; where the check gave up because the selected branch kept changing, it says to read again. A file disk deleted answers that disk deleted it and the deletion was taken in, and a file disk made binary or too large answers that it's on disk but not stored. Where the path's stored copy is a draft and disk changed or deleted it too, the read shows the draft, and its note names the conflict and the ways to settle it, a sync, or a forced materialize on a mount that isn't read-only: for a deletion, a sync of the path takes it and removes the workspace's copy. Before, a read of a path the workspace held didn't look at disk, so it could show a copy another writer had since changed; a path it didn't hold was observed, as now. - The `read` and `grep` tool descriptions changed, so the first request after the upgrade that carries the workspace's tools misses the prompt cache from its tool block on. diff --git a/src/modules/workspace/index.ts b/src/modules/workspace/index.ts index 4db987216..57f04300e 100644 --- a/src/modules/workspace/index.ts +++ b/src/modules/workspace/index.ts @@ -2987,8 +2987,12 @@ export class WorkspaceModule implements Module { // A pass that gave up (the selected branch kept changing) decided // nothing and leaves no report. It's read as a copy disk couldn't be // checked, with its reason, so the answer says the check didn't finish. + const gaveUp = !pass.reports.has(relativePath) && pass.incomplete.length > 0; const report = pass.reports.get(relativePath) - ?? (pass.incomplete.length > 0 ? { path: relativePath, state: 'unverified' as const, note: pass.incomplete[0]!.reason } : undefined); + ?? (gaveUp ? { path: relativePath, state: 'unverified' as const, note: pass.incomplete[0]!.reason } : undefined); + // A check that gave up may well finish on the next read, as grep's + // answer to the same says of grep. + const again = gaveUp ? ['Read again to check it.'] : []; // A mode taken in alone isn't a change to the text, so it leaves the // read nothing to say, as it leaves an edit nothing. const took = (op: TreeOp) => pass.ops.some((o) => o.path === relativePath && o.op === op && !o.modeOnly); @@ -2999,12 +3003,12 @@ export class WorkspaceModule implements Module { return { error: took('deleted') && report?.state !== 'disk-only' ? `File not found: ${input.path}. Disk's copy was deleted since the workspace last agreed with it, and that was taken in first.` - : this.notInWorkspace(input.path, report), + : [this.notInWorkspace(input.path, report), ...again].join(' '), }; } const blob = store.getBlob(entry.blobHash); if (!blob) return { error: `Blob not found for: ${input.path}` }; - return { bytes: blob, notes: whichCopyRead(report, took('modified'), mount.config.mode === 'read-only') }; + return { bytes: blob, notes: [...whichCopyRead(report, took('modified'), mount.config.mode === 'read-only'), ...again] }; }); if ('error' in found) return { success: false, error: found.error, isError: true }; file = fileText(found.bytes); diff --git a/test/workspace-three-way.test.ts b/test/workspace-three-way.test.ts index f83d71a65..3e74aee2f 100644 --- a/test/workspace-three-way.test.ts +++ b/test/workspace-three-way.test.ts @@ -7053,7 +7053,7 @@ describe('on-agent-action', () => { const data = await withReadHook((n) => env.store.switchBranch(n % 2 === 1 ? 'other' : main), () => call(m, 'read', { path: 'work/a.txt' })); assert.equal(data.content, ' 1\tA'); - assert.equal(data.note, "The path couldn't be checked against disk (the selected branch kept changing while the scan ran), so this read shows the workspace's copy."); + assert.equal(data.note, "The path couldn't be checked against disk (the selected branch kept changing while the scan ran), so this read shows the workspace's copy. Read again to check it."); // A character page says so too, and so does a line page under a result // limit, each read starting on main, as the first did. const switching = (n: number) => env.store.switchBranch(n % 2 === 1 ? 'other' : main); @@ -7094,7 +7094,7 @@ describe('on-agent-action', () => { env.store.createBranch('other', main); const error = await withReadHook((n) => env.store.switchBranch(n % 2 === 1 ? 'other' : main), () => refused(m, 'read', { path: 'work/b.txt' })); - assert.equal(error, 'work/b.txt is not in the workspace, and disk could not be checked (the selected branch kept changing while the scan ran).'); + assert.equal(error, 'work/b.txt is not in the workspace, and disk could not be checked (the selected branch kept changing while the scan ran). Read again to check it.'); }); test('a branch switched away during the root check and back during a read is decided on the branch enumerated', async (t) => { From 94138a9610d89b923cfc6cab6f94849ab36c9d90 Mon Sep 17 00:00:00 2001 From: Abigail-2676 Date: Sun, 18 Oct 2026 21:05:09 -0400 Subject: [PATCH 22/51] test(workspace): a first line cut to fit, read from a later offset An internal review found that the first-line cut was tested only from offset 1, where a page's lines counted from the file and from the page are the same. A test now reads a long line at offset 3 under a small result limit, and checks the page's lines, the rest's offset in the stored text, the note, and that the rest-read the note gives ends with the line. Co-Authored-By: Claude Opus 5.5 --- test/workspace-read-text.test.ts | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/test/workspace-read-text.test.ts b/test/workspace-read-text.test.ts index b26dc4600..12655290e 100644 --- a/test/workspace-read-text.test.ts +++ b/test/workspace-read-text.test.ts @@ -237,6 +237,21 @@ test('a first line that alone would pass the limit is cut where it fits', async assert.match(tiny.note!, /Line 1 is cut: read its rest with offsetChars=0 and limitChars=1500, then continue with offset=2\.$/); }); +test('a first line cut to fit, read from a later offset, is numbered and placed in the file', async t => { + const { read } = await setup(t, `a\nb\n${'"'.repeat(1500)}\nnext\n`); + const page = (await read({ offset: 3 }, 1_000)).data as LinePage; + assert.deepEqual([page.fromLine, page.toLine], [3, 3]); + const [, shown, rest, at] = /^ {5}3\t("*) \[… (\d+) more chars; read them with offsetChars=(\d+)\]$/.exec(page.content)!; + assert.ok(shown.length > 0 && shown.length < 1500, `${shown.length}`); + // The rest starts past lines 1 and 2, "a\n" and "b\n", in the stored text. + assert.equal(Number(at), 4 + shown.length); + assert.equal(page.note, `Stopped at line 3 to stay within the 1000-character result limit (file has 4). Line 3 is cut: read its rest with offsetChars=${at} and limitChars=${rest}, then continue with offset=4.`); + // The rest read with both reads to the line's end and no further. + const more = (await read({ offsetChars: Number(at), limitChars: Number(rest) }, 24_000)).data as CharPage; + assert.equal(more.content, '"'.repeat(Number(rest))); + assert.equal(more.nextOffsetChars, 4 + 1500); +}); + test('a page whose last line is cut names where its rest starts before the next line', async t => { const long = 'z'.repeat(3000); const { read } = await setup(t, `short\n${long}\n${long}\n${long}\n`); From 723580051d839efc9db2317273645d60bbd7207a Mon Sep 17 00:00:00 2001 From: Abigail-2676 Date: Sun, 18 Oct 2026 21:05:09 -0400 Subject: [PATCH 23/51] docs: comments and a tool description an internal review found out of date - src/framework.ts: executeToolCallFrom's code_execution comment named "promise-based callers (SubagentModule ephemerals)", the caller executeToolCall's doc no longer names, since SubagentModule doesn't call it. It names the callers that reach that branch: the puppet and a host through executeToolCall, and a module through callTool. - The workspace's onToolBatchComplete doc said a read takes in only a file the workspace doesn't hold yet; a read takes its path in, held or not. - grep's description says a BOM isn't part of the first line, as the changelog does, since a ^-anchored pattern now matches a BOM file's first line. Co-Authored-By: Claude Opus 5.5 --- src/framework.ts | 7 ++++--- src/modules/workspace/index.ts | 8 ++++---- 2 files changed, 8 insertions(+), 7 deletions(-) diff --git a/src/framework.ts b/src/framework.ts index e8a94667e..2799d9d84 100644 --- a/src/framework.ts +++ b/src/framework.ts @@ -18256,9 +18256,10 @@ export class AgentFramework { } if (isPresentationTool(call.name)) return this.editToolPresentation(call.callerAgentName ?? '__ephemeral__', call); - // Client-side programmatic tool calling for promise-based callers - // (SubagentModule ephemerals). Keyed by callerAgentName so each ephemeral - // gets its own interpreter state. + // Client-side programmatic tool calling for callers that await a tool's + // result: the puppet and a host through executeToolCall, and a module + // through callTool. Keyed by callerAgentName so each agent gets its own + // interpreter state. if (call.name === CODE_EXECUTION_TOOL_NAME && this.codeExecutionConfig) { return this.runCodeExecution(call.callerAgentName ?? '__ephemeral__', call); } diff --git a/src/modules/workspace/index.ts b/src/modules/workspace/index.ts index 57f04300e..ea3626a2e 100644 --- a/src/modules/workspace/index.ts +++ b/src/modules/workspace/index.ts @@ -1539,7 +1539,7 @@ export class WorkspaceModule implements Module { + '(recorded-disk). Disk versions it could not search are listed as skipped: disk-only files (binary or ' + 'over the size limit), and a conflict\'s disk version that is binary, oversize, or newer than the one recorded. ' + 'In a file with CRLF line ends, each line is matched and shown without its CR, as read shows it, ' - + 'and the file\'s result says lineEndings.', + + 'and the file\'s result says lineEndings. A BOM isn\'t part of the first line.', inputSchema: { type: 'object' as const, properties: { @@ -4374,9 +4374,9 @@ export class WorkspaceModule implements Module { * on disk (a shell command's files, a deletion) is in the workspace when the * agent next looks. On a mount over its walkCap, that holds for the first * walkCap entries in the walk's order, which every batch's scan takes; files - * past them come in through syncs and the listings that reach them, while a - * read takes in only a file the workspace doesn't hold yet. A scan that - * outlasts the deadline stops holding the round: the miss is recorded on + * past them come in through syncs, the listings that reach them, and a read + * of the path, which takes disk's change in first as a listing does. A scan + * that outlasts the deadline stops holding the round: the miss is recorded on * the mount's status and pushed as a `workspace:agent-action-scan-incomplete` * event, and the scan continues, deciding on the branch selected when it * applies. From b5e97c5a437f2ff39aaf7a3207a4feb8342bd157 Mon Sep 17 00:00:00 2001 From: Abigail-2676 Date: Sun, 18 Oct 2026 22:25:01 -0400 Subject: [PATCH 24/51] fix(workspace): a stopped character page gives the limitChars left of what it was asked for Anarchid's nightly review R2-1 (#465, at 723580051): since 8b2865218 the note of a page whose last line is cut gives that line's rest as offsetChars and limitChars. A character page that then stopped for the result limit said only "continue with offsetChars=N", and the read tool's description said "Keep limitChars when continuing", so once the rest ran past one page, following both read on past the line's end, to the end of the file. After a line of 60,000 characters and 3,000 short lines, read at 24,000, the rest's pages ended at 25,791, 49,576, 71,294 and then at the file's end, 76,896, where the line ended at 60,005. A character page that stops short of what it was asked for now says "continue with offsetChars=N and limitChars=M", M being what's left of that. Its room for the note is measured with the note at its widest. The tool description now says: when a page stops at the result limit, its note gives the offsetChars and limitChars that read the rest of what you asked for; otherwise, continue from nextOffsetChars with the same limitChars. A spill reader loses nothing: once the range is read, it goes on with its own limitChars. Tests in test/workspace-read-text.test.ts: a cut line's rest of 5,000 characters read at a limit of 1,000 through several character pages, following only the notes, ends exactly where the line does; and the three tests that pin a stopped character page's note pin the new one, the walk through escape-heavy text now following the notes. Co-Authored-By: Claude Opus 5.5 --- changelog.d/feat-workspace-read.changed.md | 2 +- src/modules/workspace/index.ts | 3 +- src/modules/workspace/read-page.ts | 13 +++--- test/workspace-read-text.test.ts | 48 ++++++++++++++++++++-- 4 files changed, 55 insertions(+), 11 deletions(-) diff --git a/changelog.d/feat-workspace-read.changed.md b/changelog.d/feat-workspace-read.changed.md index 9309e1787..45b761893 100644 --- a/changelog.d/feat-workspace-read.changed.md +++ b/changelog.d/feat-workspace-read.changed.md @@ -1,6 +1,6 @@ - The workspace's `read` shows a file's text as it is, and says so. A line page shows a CRLF file's lines without the CR before each line feed, and leaves out a BOM. A line whose own text ends in a CR shows its line end's CR too, since `edit` reads a CR before a line feed as part of the line end, so text copied from the line still matches it. Its note says when the file has CRLF ends or a BOM, counting a mix of CRLF and LF ends and the lines that show both CRs. It also says when the file isn't valid UTF-8: each invalid byte sequence shows as U+FFFD, `edit` refuses the file, and `write` replaces it whole. Before, each line of a CRLF file ended in a CR, a BOM came first in the first line as U+FEFF, and invalid bytes showed as U+FFFD with no note. A character page still reads the stored text as it is, BOM and CRs included, and the tool's description now says so. - On a page for a call that carries a `resultMaxChars`, such as a model's or a running script's, a line over 2,000 characters is cut, with how many characters follow and the `offsetChars` that reads them. Before, a line page showed every line whole, however long. -- A `read` page stops before its result would pass `resultMaxChars`, the most of it the framework stores whole. For a model's call that's its agent's inline cap, 24,000 characters by default. Its note says where to continue: the `offset` after its last line, or for a character page the `offsetChars` it also gives as `nextOffsetChars`. Where even a page's first line would pass the limit, that line is cut shorter, at the most the page has room for. Where a page's last line is cut, its note names the `offsetChars` where that line's rest starts and the `limitChars` that reads it to the line's end, before the next line's `offset` if the file goes on. A page always moves on, so where its own fields (the path and its notes) and the least it can show pass a small limit, it passes the limit. At 1,000 characters, a page with no notes does that only for a path of about 690 characters or more. Its notes take room from that: with all three notes on a file's text and a conflict's, a path of 100 to 140 characters does it, and a note that names a symlink's target, as one saying disk couldn't be checked can, takes as much room as the target is long. Before, a page of up to 2,000 lines, or of `limitChars` characters, went out whole, and for a result that passed the limit the framework kept only its start, cut mid-JSON, writing the whole of it to a workspace file where the agent has one. A caller that pages 2,000 lines at a time and starts the next page at line 2,001 now starts where the note says. A running `code_execution` script's call has a limit of 5,000,000 characters, so its line pages still stop at the line limit, 2,000 by default, and its character pages at `limitChars`, unless a page would pass that. Its line pages cut a line over 2,000 characters and leave out CRs and a BOM, as a model's do, and a character page gives it the stored text exactly. A call made in process, through `handleToolCall`, `executeToolCall` or `ModuleContext.callTool`, carries no `resultMaxChars` unless its caller sets one. Its line pages stop at the line limit alone and show every line whole, as they did before, though now without the CRs and the BOM. +- A `read` page stops before its result would pass `resultMaxChars`, the most of it the framework stores whole. For a model's call that's its agent's inline cap, 24,000 characters by default. Its note says where to continue: the `offset` after its last line, or for a character page the `offsetChars` it also gives as `nextOffsetChars`, with the `limitChars` left of what it was asked for, so a reader that follows the notes ends where it asked to. The read tool's description now says so, where it said to keep `limitChars` when continuing. Where even a page's first line would pass the limit, that line is cut shorter, at the most the page has room for. Where a page's last line is cut, its note names the `offsetChars` where that line's rest starts and the `limitChars` that reads it to the line's end, before the next line's `offset` if the file goes on. A page always moves on, so where its own fields (the path and its notes) and the least it can show pass a small limit, it passes the limit. At 1,000 characters, a page with no notes does that only for a path of about 690 characters or more. Its notes take room from that: with all three notes on a file's text and a conflict's, a path of 100 to 140 characters does it, and a note that names a symlink's target, as one saying disk couldn't be checked can, takes as much room as the target is long. Before, a page of up to 2,000 lines, or of `limitChars` characters, went out whole, and for a result that passed the limit the framework kept only its start, cut mid-JSON, writing the whole of it to a workspace file where the agent has one. A caller that pages 2,000 lines at a time and starts the next page at line 2,001 now starts where the note says. A running `code_execution` script's call has a limit of 5,000,000 characters, so its line pages still stop at the line limit, 2,000 by default, and its character pages at `limitChars`, unless a page would pass that. Its line pages cut a line over 2,000 characters and leave out CRs and a BOM, as a model's do, and a character page gives it the stored text exactly. A call made in process, through `handleToolCall`, `executeToolCall` or `ModuleContext.callTool`, carries no `resultMaxChars` unless its caller sets one. Its line pages stop at the line limit alone and show every line whole, as they did before, though now without the CRs and the BOM. - The workspace's `grep` leaves out a CR before a line feed, for matching as well as display, so `foo$` finds a CRLF file's line too, as it does under ripgrep's `--crlf`. A match's text and context are its lines without the CR, as `read` shows them, but each line whole, where a `read` page for a call with a result limit cuts a line over 2,000 characters. A BOM isn't part of the first line, and a file with CRLF ends says so in its result's `lineEndings` (`CRLF`, or `mixed`). Before, a CRLF file's lines ended in a CR, so `foo$` missed them and a match's text copied into an edit carried the CR. A pattern written to find the CR of a CRLF line end no longer matches; a CR that doesn't come before a line feed is still part of its line. - The workspace's `read` first takes in a change on disk to a file the workspace holds, as a listing and an `edit` do, and shows that text. Its note says so when disk's copy had changed since the workspace last agreed with it. It also says when disk's copy couldn't be checked, or disk has none and nothing records whether it was deleted there, so it shows the workspace's copy; where the check gave up because the selected branch kept changing, it says to read again. A file disk deleted answers that disk deleted it and the deletion was taken in, and a file disk made binary or too large answers that it's on disk but not stored. Where the path's stored copy is a draft and disk changed or deleted it too, the read shows the draft, and its note names the conflict and the ways to settle it, a sync, or a forced materialize on a mount that isn't read-only: for a deletion, a sync of the path takes it and removes the workspace's copy. Before, a read of a path the workspace held didn't look at disk, so it could show a copy another writer had since changed; a path it didn't hold was observed, as now. - The `read` and `grep` tool descriptions changed, so the first request after the upgrade that carries the workspace's tools misses the prompt cache from its tool block on. diff --git a/src/modules/workspace/index.ts b/src/modules/workspace/index.ts index ea3626a2e..b5982b4b4 100644 --- a/src/modules/workspace/index.ts +++ b/src/modules/workspace/index.ts @@ -1392,7 +1392,8 @@ export class WorkspaceModule implements Module { + 'or isn\'t valid UTF-8. ' + 'For long lines or spill files, use offsetChars/limitChars instead: returns the stored text as it is, ' + 'BOM and CRs included, without line numbers, with nextOffsetChars (null at EOF), stopped at the result limit too. ' - + 'Keep limitChars when continuing. ' + + 'When a page stops at the result limit, its note gives the offsetChars and limitChars that read the rest of what you asked for. ' + + 'Otherwise, continue from nextOffsetChars with the same limitChars. ' + 'Character offsets count UTF-16 code units; do not mix character and line parameters.', inputSchema: { type: 'object' as const, diff --git a/src/modules/workspace/read-page.ts b/src/modules/workspace/read-page.ts index 4b6ed1499..e9e65f663 100644 --- a/src/modules/workspace/read-page.ts +++ b/src/modules/workspace/read-page.ts @@ -156,8 +156,11 @@ export function characterPage(path: string, file: FileText, offsetChars: number, let end = start + Math.min(limitChars, text.length - start); if (splitsPair(text, end)) end++; const notes = [...(file.utf8 ? [] : [NOT_UTF8]), ...extraNotes]; - const stoppedNote = (to: number) => - `Stopped at character ${to} to stay within the ${maxChars}-character result limit; continue with offsetChars=${to}.`; + // A page that stops short of what it was asked for gives what's left of + // that as limitChars, so a reader that follows its notes ends where it + // asked to: at a cut line's end, for the rest a line page's note names. + const stoppedNote = (to: number, left: number) => + `Stopped at character ${to} to stay within the ${maxChars}-character result limit; continue with offsetChars=${to} and limitChars=${left}.`; const page = (to: number, note: string | undefined) => ({ path, totalChars: text.length, @@ -169,8 +172,8 @@ export function characterPage(path: string, file: FileText, offsetChars: number, const whole = page(end, joined(notes)); if (maxChars === undefined || JSON.stringify(whole).length <= maxChars) return { success: true, data: whole }; // As many characters as fit beside the rest of the page, which takes the - // most with the largest offset it could stop at. - const overhead = JSON.stringify({ ...page(end, joined([stoppedNote(end), ...notes])), nextOffsetChars: end, content: '' }).length; + // most with the largest offset it could stop at and the most left. + const overhead = JSON.stringify({ ...page(end, joined([stoppedNote(end, end - start), ...notes])), nextOffsetChars: end, content: '' }).length; // A page moves by whole characters, a surrogate pair at once, so it never // stops inside one. const step = (at: number) => (splitsPair(text, at + 1) ? 2 : 1); @@ -185,5 +188,5 @@ export function characterPage(path: string, file: FileText, offsetChars: number, // small for the page's own fields then passes it. Where that reaches the // page's own end, it holds all it was asked for, so it names no stop. if (to === start) to = Math.min(end, start + step(start)); - return { success: true, data: page(to, joined([to < end ? stoppedNote(to) : null, ...notes])) }; + return { success: true, data: page(to, joined([to < end ? stoppedNote(to, end - to) : null, ...notes])) }; } diff --git a/test/workspace-read-text.test.ts b/test/workspace-read-text.test.ts index 12655290e..6a0842eff 100644 --- a/test/workspace-read-text.test.ts +++ b/test/workspace-read-text.test.ts @@ -172,7 +172,7 @@ test('a page with notes holds to the result limit with its notes', async t => { const result = await readBad({ offsetChars: 0, limitChars: 100_000 }, limit); assert.ok(stored(result) <= limit, `${stored(result)} > ${limit}`); const page = result.data as CharPage; - assert.equal(page.note, `Stopped at character ${page.nextOffsetChars} to stay within the ${limit}-character result limit; continue with offsetChars=${page.nextOffsetChars}. The file isn't valid UTF-8: each invalid byte sequence shows as U+FFFD. Edit refuses this file, and write replaces it whole.`); + assert.equal(page.note, `Stopped at character ${page.nextOffsetChars} to stay within the ${limit}-character result limit; continue with offsetChars=${page.nextOffsetChars} and limitChars=${page.totalChars - page.nextOffsetChars!}. The file isn't valid UTF-8: each invalid byte sequence shows as U+FFFD. Edit refuses this file, and write replaces it whole.`); } }); @@ -252,6 +252,42 @@ test('a first line cut to fit, read from a later offset, is numbered and placed assert.equal(more.nextOffsetChars, 4 + 1500); }); +test("a cut line's rest read through several character pages, following only the notes, ends where the line does", async t => { + // A rest longer than a page at a limit of 1,000, with more lines after it: + // each character page that stops gives what's left of the rest, so reading + // on as the notes say reaches the line's end and no further. + const long = 'z'.repeat(5_000); + const { read } = await setup(t, `head\n${long}\n${Array.from({ length: 50 }, (_, i) => `after ${i}`).join('\n')}\n`); + // From line 2, the line is cut to fit the page. + const linePage = (await read({ offset: 2 }, 1_000)).data as LinePage; + const [, at, left] = /Line 2 is cut: read its rest with offsetChars=(\d+) and limitChars=(\d+), then continue with offset=3\.$/.exec(linePage.note!)!; + const lineEnd = 'head\n'.length + long.length; + assert.equal(Number(at) + Number(left), lineEnd); + let offsetChars = Number(at); + let limitChars = Number(left); + let rest = ''; + let pages = 0; + for (;;) { + assert.ok(++pages < 100, 'every page moves on'); + const result = await read({ offsetChars, limitChars }, 1_000); + assert.ok(stored(result) <= 1_000, `${stored(result)}`); + const page = result.data as CharPage; + rest += page.content; + const next = page.note === undefined ? null : /continue with offsetChars=(\d+) and limitChars=(\d+)/.exec(page.note); + if (next === null) { + // The last page holds all it was asked for, and ends at the line's end. + assert.equal(page.note, undefined); + assert.equal(page.nextOffsetChars, lineEnd); + break; + } + offsetChars = Number(next[1]); + limitChars = Number(next[2]); + assert.equal(offsetChars + limitChars, lineEnd); + } + assert.ok(pages > 2, `${pages} pages`); + assert.equal(rest, long.slice(Number(at) - 'head\n'.length)); +}); + test('a page whose last line is cut names where its rest starts before the next line', async t => { const long = 'z'.repeat(3000); const { read } = await setup(t, `short\n${long}\n${long}\n${long}\n`); @@ -305,17 +341,21 @@ test('a character page stops at the result limit too, with nextOffsetChars', asy const text = 'abc\u0001'.repeat(20_000); const { read } = await setup(t, text); for (const limit of [24_000, 1_000]) { + // Following each page's note, from a first page asked for more than the file holds. let offsetChars = 0; + let limitChars = 100_000; let recovered = ''; for (let pages = 0; ; pages++) { assert.ok(pages < 1_000, 'every page moves on'); - const result = await read({ offsetChars, limitChars: 100_000 }, limit); + const result = await read({ offsetChars, limitChars }, limit); assert.ok(stored(result) <= limit, `${stored(result)} > ${limit}`); const page = result.data as CharPage; recovered += page.content; if (page.nextOffsetChars === null) break; - assert.equal(page.note, `Stopped at character ${page.nextOffsetChars} to stay within the ${limit}-character result limit; continue with offsetChars=${page.nextOffsetChars}.`); + // What's left of what was asked for is the rest of the text. + assert.equal(page.note, `Stopped at character ${page.nextOffsetChars} to stay within the ${limit}-character result limit; continue with offsetChars=${page.nextOffsetChars} and limitChars=${text.length - page.nextOffsetChars}.`); offsetChars = page.nextOffsetChars; + limitChars = text.length - page.nextOffsetChars; } assert.equal(recovered, text); } @@ -341,7 +381,7 @@ test('a character page moved on by a whole character keeps a surrogate pair whol const { read } = await setup(t, '😀abc'); assert.deepEqual((await read({ offsetChars: 0, limitChars: 100 }, 50)).data, { path: 'work/text.txt', totalChars: 5, offsetChars: 0, nextOffsetChars: 2, - note: 'Stopped at character 2 to stay within the 50-character result limit; continue with offsetChars=2.', + note: 'Stopped at character 2 to stay within the 50-character result limit; continue with offsetChars=2 and limitChars=3.', content: '😀', }); // Where that step reaches what the page was asked for, the page holds all of it. From eea460bacf0503ede1d4f82fac17775c02531024 Mon Sep 17 00:00:00 2001 From: Abigail-2676 Date: Sun, 18 Oct 2026 22:25:02 -0400 Subject: [PATCH 25/51] fix(workspace): a first line cut to fit is the longest cut that fits, past the dip a digit makes Anarchid's nightly review R2-2 (#465, at 723580051): the rest's length is in the cut line's marker and, since 8b2865218, in the note's limitChars, so at the cut where it loses a digit the page loses two characters for the one it gains, and is a character shorter than at the cut before. The binary search took the page as growing with the cut, and could stop two characters short: one line of 1,500 "a" at a limit of 712 got a cut of 499 where 501 fits. Such a dip is one cut, and the next is many cuts away, so after the search the next cuts, up to four code units on, are measured too, and the longest that fits is kept. The comment gives the corrected reasoning. Test in test/workspace-read-text.test.ts: a line of 1,500 "a" read at each limit from 700 to 760, where no cut up to four longer than the one shown fits, and the range covers the dip: cuts of 499 and 500 are never shown, since each page is as long as the one two cuts on. Co-Authored-By: Claude Opus 5.5 --- src/modules/workspace/read-page.ts | 27 ++++++++++++++++++--------- test/workspace-read-text.test.ts | 26 ++++++++++++++++++++++++++ 2 files changed, 44 insertions(+), 9 deletions(-) diff --git a/src/modules/workspace/read-page.ts b/src/modules/workspace/read-page.ts index e9e65f663..9ef60213c 100644 --- a/src/modules/workspace/read-page.ts +++ b/src/modules/workspace/read-page.ts @@ -119,21 +119,30 @@ export function linePage(path: string, file: FileText, offset: number, limit: nu return { success: true, data: page(first + best, shownLines.join('\n'), noteAt(first + best)) }; } // No page fits with its first line as a page shows it: the loop tried that - // page first, since a read's limit is at least 1, and it is pageWith(over). + // page first, since a read's limit is at least 1, and it is pageWith(top). // So the line is cut shorter, at the most the page has room for, and ends - // the page: its note names where the line's rest starts, since that comes - // next. Each cut is measured on the page itself, since its note names the - // cut. Every cut below `over` shows a marker, and a longer one never makes - // the page shorter: the text gains at least a character where the marker's - // count loses at most a digit, and the offsets only grow. So the search - // finds the longest cut that fits. + // the page: its note names where the line's rest starts and how long it is, + // since that comes next. Each cut is measured on the page itself, since its + // note names the cut. Every cut below `top` shows a marker. A longer cut + // adds a character of text at least, and the offsets only grow, but the + // rest's length is in the marker and in the note, so where it loses a digit + // the page loses two characters: it can be one character shorter than the + // page a cut before. The search takes the page as growing with the cut. + // Such a dip is one cut, and the next is many cuts away, so the longest cut + // that fits is the one the search finds or, past a dip, at most two cuts + // further, four code units at most, and those are tried too. const pageWith = (chars: number) => page(first + 1, shown(first, chars), noteAt(first + 1, chars)); + const fitsAt = (chars: number) => JSON.stringify(pageWith(chars)).length <= maxChars; + const top = Math.min(lines[first].length, MAX_LINE_CHARS); let fits = 0; - let over = Math.min(lines[first].length, MAX_LINE_CHARS); + let over = top; while (over - fits > 1) { const mid = Math.floor((fits + over) / 2); - if (JSON.stringify(pageWith(mid)).length <= maxChars) fits = mid; else over = mid; + if (fitsAt(mid)) fits = mid; else over = mid; } + // The search ends with `fits + 1` measured as too long, or as `top`. + const found = fits; + for (let chars = found + 2; chars <= found + 4 && chars < top; chars++) if (fitsAt(chars)) fits = chars; return { success: true, data: pageWith(fits) }; } diff --git a/test/workspace-read-text.test.ts b/test/workspace-read-text.test.ts index 6a0842eff..83b2a090d 100644 --- a/test/workspace-read-text.test.ts +++ b/test/workspace-read-text.test.ts @@ -237,6 +237,32 @@ test('a first line that alone would pass the limit is cut where it fits', async assert.match(tiny.note!, /Line 1 is cut: read its rest with offsetChars=0 and limitChars=1500, then continue with offset=2\.$/); }); +test("a first line cut to fit is the longest cut that fits, where the rest's length loses a digit too", async t => { + // The rest's length is in the marker and in the note, so at the cut where it + // goes from 1,000 to 999 the page loses two characters for the one it gains, + // and is a character shorter than at the cut before. Plain letters take one + // character each as JSON, so some limit in this range lands on that dip. + const { read } = await setup(t, 'a'.repeat(1_500)); + const withCut = (page: LinePage, cut: number) => JSON.stringify({ + ...page, + note: `Line 1 is cut: read its rest with offsetChars=${cut} and limitChars=${1_500 - cut}.`, + content: ` 1\t${'a'.repeat(cut)} [… ${1_500 - cut} more chars; read them with offsetChars=${cut}]`, + }).length; + const cuts = new Set(); + for (let limit = 700; limit <= 760; limit++) { + const result = await read({}, limit); + assert.ok(stored(result) <= limit, `${stored(result)} > ${limit}`); + const page = result.data as LinePage; + const cut = /^ {5}1\t(a*) \[/.exec(page.content)![1].length; + cuts.add(cut); + assert.equal(withCut(page, cut), stored(result)); + for (let more = 1; more <= 4; more++) assert.ok(withCut(page, cut + more) > limit, `at ${limit}, a cut of ${cut + more} fits where ${cut} was shown`); + } + // The range covers the dip. Cuts of 499 and 500 are never the longest that + // fits, since the page of each is as long as the page two cuts on. + assert.ok(cuts.has(498) && cuts.has(501) && !cuts.has(499) && !cuts.has(500), [...cuts].join(' ')); +}); + test('a first line cut to fit, read from a later offset, is numbered and placed in the file', async t => { const { read } = await setup(t, `a\nb\n${'"'.repeat(1500)}\nnext\n`); const page = (await read({ offset: 3 }, 1_000)).data as LinePage; From bf379817f2d7d608a23ec1b57514116beb5a5ed8 Mon Sep 17 00:00:00 2001 From: Abigail-2676 Date: Sun, 18 Oct 2026 22:25:02 -0400 Subject: [PATCH 26/51] test(workspace): a script's line page cuts a long line too Anarchid's nightly review R2-3 (#465, at 723580051): R1-4 kept the 2,000-character cut for a running script's line pages, as the changelog says, but no test read a long line at a script's limit of 5,000,000. The long-line test now reads at that limit too, and the line shows the same marker. Co-Authored-By: Claude Opus 5.5 --- test/workspace-read-text.test.ts | 2 ++ 1 file changed, 2 insertions(+) diff --git a/test/workspace-read-text.test.ts b/test/workspace-read-text.test.ts index 83b2a090d..a74740e9d 100644 --- a/test/workspace-read-text.test.ts +++ b/test/workspace-read-text.test.ts @@ -90,6 +90,8 @@ test('a long line is cut at 2000 characters, with the offsetChars that reads the const rest = long.length - 1999; const at = 1 + 'short\r\n'.length + 1999; assert.equal(lines[1], ` 2\t${'x'.repeat(1999)} [… ${rest} more chars; read them with offsetChars=${at}]`); + // A script's call, at 5,000,000 characters, cuts the line the same way. + assert.equal(((await read({}, 5_000_000)).data as LinePage).content.split('\n')[1], lines[1]); const more = (await read({ offsetChars: at, limitChars: rest })).data as CharPage; assert.equal(more.content, '😀' + 'y'.repeat(1000)); assert.equal(lines[2], ' 3\tlast'); From ac741bf916bb371b42317a65e608c2f7d908491f Mon Sep 17 00:00:00 2001 From: Abigail-2676 Date: Mon, 19 Oct 2026 02:29:15 -0400 Subject: [PATCH 27/51] docs(workspace): read and grep say a CR CR LF line keeps its end's CR, and how a cut line's rest is read af-ws-read-whole-r1 (an internal review of 60e17484a..bf379817f) found the text behind the code in three places. - I2: the read and grep tool descriptions, grep's comment and the grep changelog bullet said a CRLF file's lines are shown and matched without the CR, with no word of a line whose own text ends in a CR, which since c7cdbbd3f keeps its line end's CR too. Each now says so: the read description in a sentence of its own, grep's as "without its line end's CR unless its own text ends in a CR", and the changelog's last grep sentence names the exception to "no longer matches". - I12: the read description said a cut line comes "with the offsetChars that reads the rest", but offsetChars alone reads up to 2,000 code units and on into the next lines. It now says the marker gives where the rest starts and how many characters it has, and to read the rest with that count as limitChars. The changelog's cut-line sentence says the same. - I8: "When a page stops at the result limit, its note gives the offsetChars and limitChars ..." read as covering line pages, whose stop note gives offset=. It now says "a character page", the case it was written for. Co-Authored-By: Claude Opus 5.5 --- changelog.d/feat-workspace-read.changed.md | 4 ++-- src/modules/workspace/index.ts | 20 +++++++++++--------- 2 files changed, 13 insertions(+), 11 deletions(-) diff --git a/changelog.d/feat-workspace-read.changed.md b/changelog.d/feat-workspace-read.changed.md index 45b761893..0d478f9b9 100644 --- a/changelog.d/feat-workspace-read.changed.md +++ b/changelog.d/feat-workspace-read.changed.md @@ -1,6 +1,6 @@ - The workspace's `read` shows a file's text as it is, and says so. A line page shows a CRLF file's lines without the CR before each line feed, and leaves out a BOM. A line whose own text ends in a CR shows its line end's CR too, since `edit` reads a CR before a line feed as part of the line end, so text copied from the line still matches it. Its note says when the file has CRLF ends or a BOM, counting a mix of CRLF and LF ends and the lines that show both CRs. It also says when the file isn't valid UTF-8: each invalid byte sequence shows as U+FFFD, `edit` refuses the file, and `write` replaces it whole. Before, each line of a CRLF file ended in a CR, a BOM came first in the first line as U+FEFF, and invalid bytes showed as U+FFFD with no note. A character page still reads the stored text as it is, BOM and CRs included, and the tool's description now says so. -- On a page for a call that carries a `resultMaxChars`, such as a model's or a running script's, a line over 2,000 characters is cut, with how many characters follow and the `offsetChars` that reads them. Before, a line page showed every line whole, however long. +- On a page for a call that carries a `resultMaxChars`, such as a model's or a running script's, a line over 2,000 characters is cut, with the `offsetChars` where its rest starts and how many characters the rest has, and the tool's description says to read the rest with that count as `limitChars`. Before, a line page showed every line whole, however long. - A `read` page stops before its result would pass `resultMaxChars`, the most of it the framework stores whole. For a model's call that's its agent's inline cap, 24,000 characters by default. Its note says where to continue: the `offset` after its last line, or for a character page the `offsetChars` it also gives as `nextOffsetChars`, with the `limitChars` left of what it was asked for, so a reader that follows the notes ends where it asked to. The read tool's description now says so, where it said to keep `limitChars` when continuing. Where even a page's first line would pass the limit, that line is cut shorter, at the most the page has room for. Where a page's last line is cut, its note names the `offsetChars` where that line's rest starts and the `limitChars` that reads it to the line's end, before the next line's `offset` if the file goes on. A page always moves on, so where its own fields (the path and its notes) and the least it can show pass a small limit, it passes the limit. At 1,000 characters, a page with no notes does that only for a path of about 690 characters or more. Its notes take room from that: with all three notes on a file's text and a conflict's, a path of 100 to 140 characters does it, and a note that names a symlink's target, as one saying disk couldn't be checked can, takes as much room as the target is long. Before, a page of up to 2,000 lines, or of `limitChars` characters, went out whole, and for a result that passed the limit the framework kept only its start, cut mid-JSON, writing the whole of it to a workspace file where the agent has one. A caller that pages 2,000 lines at a time and starts the next page at line 2,001 now starts where the note says. A running `code_execution` script's call has a limit of 5,000,000 characters, so its line pages still stop at the line limit, 2,000 by default, and its character pages at `limitChars`, unless a page would pass that. Its line pages cut a line over 2,000 characters and leave out CRs and a BOM, as a model's do, and a character page gives it the stored text exactly. A call made in process, through `handleToolCall`, `executeToolCall` or `ModuleContext.callTool`, carries no `resultMaxChars` unless its caller sets one. Its line pages stop at the line limit alone and show every line whole, as they did before, though now without the CRs and the BOM. -- The workspace's `grep` leaves out a CR before a line feed, for matching as well as display, so `foo$` finds a CRLF file's line too, as it does under ripgrep's `--crlf`. A match's text and context are its lines without the CR, as `read` shows them, but each line whole, where a `read` page for a call with a result limit cuts a line over 2,000 characters. A BOM isn't part of the first line, and a file with CRLF ends says so in its result's `lineEndings` (`CRLF`, or `mixed`). Before, a CRLF file's lines ended in a CR, so `foo$` missed them and a match's text copied into an edit carried the CR. A pattern written to find the CR of a CRLF line end no longer matches; a CR that doesn't come before a line feed is still part of its line. +- The workspace's `grep` leaves out a CRLF end's CR, for matching as well as display, as `read` does, so `foo$` finds a CRLF file's line too, as it does under ripgrep's `--crlf`. A match's text and context are its lines as `read` shows them, but each line whole, where a `read` page for a call with a result limit cuts a line over 2,000 characters. A BOM isn't part of the first line, and a file with CRLF ends says so in its result's `lineEndings` (`CRLF`, or `mixed`). Before, a CRLF file's lines ended in a CR, so `foo$` missed them and a match's text copied into an edit carried the CR. A pattern written to find the CR of a CRLF line end no longer matches, except on a line whose own text ends in a CR, which keeps its line end's CR as `read` shows it; a CR that doesn't come before a line feed is still part of its line. - The workspace's `read` first takes in a change on disk to a file the workspace holds, as a listing and an `edit` do, and shows that text. Its note says so when disk's copy had changed since the workspace last agreed with it. It also says when disk's copy couldn't be checked, or disk has none and nothing records whether it was deleted there, so it shows the workspace's copy; where the check gave up because the selected branch kept changing, it says to read again. A file disk deleted answers that disk deleted it and the deletion was taken in, and a file disk made binary or too large answers that it's on disk but not stored. Where the path's stored copy is a draft and disk changed or deleted it too, the read shows the draft, and its note names the conflict and the ways to settle it, a sync, or a forced materialize on a mount that isn't read-only: for a deletion, a sync of the path takes it and removes the workspace's copy. Before, a read of a path the workspace held didn't look at disk, so it could show a copy another writer had since changed; a path it didn't hold was observed, as now. - The `read` and `grep` tool descriptions changed, so the first request after the upgrade that carries the workspace's tools misses the prompt cache from its tool block on. diff --git a/src/modules/workspace/index.ts b/src/modules/workspace/index.ts index 8cab9a32d..cb0868a99 100644 --- a/src/modules/workspace/index.ts +++ b/src/modules/workspace/index.ts @@ -1387,12 +1387,13 @@ export class WorkspaceModule implements Module { description: 'Read a file from the workspace. Returns content with line numbers. ' + 'Reads at most 2000 lines per call by default — use offset/limit to page through larger files — ' + 'and stops sooner where a page would pass the result limit; the note says where to continue. ' - + 'A line over 2000 characters is cut, with the offsetChars that reads the rest. ' + + 'A line over 2000 characters is cut, with the offsetChars where its rest starts and how many characters the rest has; ' + + 'read the rest with that count as limitChars. ' + 'A line page shows CRLF line ends without the CR, and no BOM; its note says when the file has either, ' - + 'or isn\'t valid UTF-8. ' + + 'or isn\'t valid UTF-8. A line whose own text ends in a CR shows its line end\'s CR too. ' + 'For long lines or spill files, use offsetChars/limitChars instead: returns the stored text as it is, ' + 'BOM and CRs included, without line numbers, with nextOffsetChars (null at EOF), stopped at the result limit too. ' - + 'When a page stops at the result limit, its note gives the offsetChars and limitChars that read the rest of what you asked for. ' + + 'When a character page stops at the result limit, its note gives the offsetChars and limitChars that read the rest of what you asked for. ' + 'Otherwise, continue from nextOffsetChars with the same limitChars. ' + 'Character offsets count UTF-16 code units; do not mix character and line parameters.', inputSchema: { @@ -1539,8 +1540,8 @@ export class WorkspaceModule implements Module { + '(conflicting-disk); or, once disk has changed since the conflict was recorded, that recorded version ' + '(recorded-disk). Disk versions it could not search are listed as skipped: disk-only files (binary or ' + 'over the size limit), and a conflict\'s disk version that is binary, oversize, or newer than the one recorded. ' - + 'In a file with CRLF line ends, each line is matched and shown without its CR, as read shows it, ' - + 'and the file\'s result says lineEndings. A BOM isn\'t part of the first line.', + + 'In a file with CRLF line ends, each line is matched and shown as read shows it, without its line end\'s CR ' + + 'unless its own text ends in a CR, and the file\'s result says lineEndings. A BOM isn\'t part of the first line.', inputSchema: { type: 'object' as const, properties: { @@ -3747,10 +3748,11 @@ export class WorkspaceModule implements Module { const skipped: Array<{ file: string; reason: string; size?: number; mimeType?: string }> = []; const incomplete: Array<{ path: string; reason: string }> = []; - // Lines as read shows them: a CR before a line feed is left out, for - // matching as well as display, so `foo$` finds a CRLF file's line, as it - // does under ripgrep's --crlf, and the line copies into an edit as read's - // do. ripgrep's --crlf still prints the line with its CR. + // Lines as read shows them (fileText): a CRLF end's CR is left out, for + // matching as well as display, unless the line's own text ends in a CR, so + // `foo$` finds a CRLF file's line, as it does under ripgrep's --crlf, and + // the line copies into an edit as read's do. ripgrep's --crlf still prints + // the line with its CR. const search = (bytes: Buffer): Searched => { const { lines, crlfEnds, lfEnds } = fileText(bytes); const found: GrepMatch[] = []; From c9c39033c6e8177769076eb5c4ed73e2b98db8b8 Mon Sep 17 00:00:00 2001 From: Abigail-2676 Date: Mon, 19 Oct 2026 02:29:15 -0400 Subject: [PATCH 28/51] docs(framework): the utils comments name the callers that reach executeToolCallFrom af-ws-read-whole-r1's I9: the comment on the utils branch in executeToolCallFrom named "code_execution scripts, ephemerals, ModuleContext.callTool" as its callers, and handleUtilsToolCall's doc said the "programmatic path (executeToolCall)" lets code_execution scripts and ephemerals reach utilities. A running script's call goes through dispatchScriptToolCall to the model-facing dispatcher (dispatchToolCall, then dispatchUtilsToolCall). So does an ephemeral agent's, such as SubagentModule's: runEphemeralToCompletion registers it among the framework's agents "so the event loop can drive it". The comment by the code_execution branch just above, which 723580051 rewrote, already names the puppet and a host through executeToolCall and a module through callTool. These two now name the same callers, and the doc says a script's calls take the model-facing dispatcher. Co-Authored-By: Claude Opus 5.5 --- src/framework.ts | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/src/framework.ts b/src/framework.ts index 2799d9d84..ded5f8977 100644 --- a/src/framework.ts +++ b/src/framework.ts @@ -18293,9 +18293,9 @@ export class AgentFramework { return this.draftsToolResult(origin.agentName, call.input); } - // The utils meta-tool — programmatic callers (code_execution scripts, - // ephemerals, ModuleContext.callTool) reach utilities the same way the - // model does. + // The utils meta-tool: callers that await a tool's result (the puppet and + // a host through executeToolCall, and a module through callTool) reach + // utilities the same way the model does. if (call.name === 'utils') { return this.handleUtilsToolCall(call.callerAgentName ?? '__ephemeral__', call, { inProcess: true, trustedModule: origin.kind === 'module' }); } @@ -25751,9 +25751,10 @@ export class AgentFramework { } /** The `utils` meta-tool: list/describe/run over module utilities - * (Module.getUtilities). Shared by the model-facing dispatcher and the - * programmatic path (executeToolCall), so code_execution scripts and - * ephemerals reach utilities too. */ + * (Module.getUtilities). Shared by the model-facing dispatcher, which a + * running code_execution script's calls take too, and executeToolCallFrom, + * which the puppet and a host reach through executeToolCall and a module + * through callTool. */ private async handleUtilsToolCall(agentName: string, call: ToolCall, options: { inProcess?: boolean; trustedModule?: boolean } = {}): Promise { const input = (call.input ?? {}) as { action?: unknown; name?: unknown; args?: unknown }; const allUtilities = this.moduleRegistry.getAllUtilities(); From dcb7968c13e8012093ddaaaaebcfe0b5244038c7 Mon Sep 17 00:00:00 2001 From: Abigail-2676 Date: Mon, 19 Oct 2026 02:29:15 -0400 Subject: [PATCH 29/51] test(workspace): pin a mixed file's kept-CR note, a bare LF line's count, the cut's 2,000 bound and a gone root's note af-ws-read-whole-r1's I3, I5, I6 and I7 each found a piece of the change that a removal would leave every test passing. - I3: the note's kept-CR clause was asserted only for a CRLF-only file with two such lines. A file of mixed ends with one now pins the mixed template's clause and its singular wording. - I6: fileText counts a kept CR only on a CRLF line. "a\rb\nc\r\n", whose LF-ended first line has a CR before its last character, keeps none. - I5: the first-line cut's measuring after the search stops at 2,000. At the limit where a 3,000-character line's cut at 2,000 is one character too long, the cut at 2,001 would fit, since its rest's length, 999, has a digit fewer, and the page shows 1,999. - I7: "Read again to check it." goes only with a pass that gave up on its path. A read after the mount's root is renamed away gets an unverified report with an incomplete region, and its note says disk couldn't be checked without saying to read again. Removals R65 to R69 each fail one of these. Co-Authored-By: Claude Opus 5.5 --- test/workspace-file-tools.test.ts | 17 ++++++++++++++++- test/workspace-read-text.test.ts | 25 +++++++++++++++++++++++++ test/workspace-text.test.ts | 4 ++++ 3 files changed, 45 insertions(+), 1 deletion(-) diff --git a/test/workspace-file-tools.test.ts b/test/workspace-file-tools.test.ts index 3a3558968..c68be372d 100644 --- a/test/workspace-file-tools.test.ts +++ b/test/workspace-file-tools.test.ts @@ -4,7 +4,7 @@ import { spawnSync } from 'node:child_process'; import test, { type TestContext } from 'node:test'; import assert from 'node:assert/strict'; -import { chmodSync, existsSync, linkSync, lstatSync, mkdirSync, mkdtempSync, readFileSync, rmSync, statSync, symlinkSync, writeFileSync } from 'node:fs'; +import { chmodSync, existsSync, linkSync, lstatSync, mkdirSync, mkdtempSync, readFileSync, renameSync, rmSync, statSync, symlinkSync, writeFileSync } from 'node:fs'; import { tmpdir } from 'node:os'; import { join } from 'node:path'; import { JsStore } from '@animalabs/chronicle'; @@ -651,6 +651,21 @@ test("a read whose disk copy can't be checked says it shows the workspace's copy }); }); +test("a read of a held file whose mount's root is gone says disk couldn't be checked, and not to read again", async (t) => { + // The pass reports the path as unverified and leaves the root's region + // incomplete. It didn't give up on a branch that kept changing, so reading + // again won't check it. + const { dir, call } = setup(t); + writeFileSync(join(dir, 'u.txt'), 'held\n'); + assert.equal((await call('read', { path: 'work/u.txt' })).success, true); + renameSync(dir, join(dir, '..', 'gone')); + assert.deepEqual((await call('read', { path: 'work/u.txt' })).data, { + path: 'work/u.txt', totalLines: 1, fromLine: 1, toLine: 1, + note: "The path couldn't be checked against disk (the mount root is unavailable), so this read shows the workspace's copy.", + content: ' 1\theld', + }); +}); + test("a read of a file disk lacks, with nothing to say whether disk deleted it, says it shows the workspace's copy", async (t) => { const { store, call } = setup(t); // An entry from before the evidence (as three-way's legacyEntry makes it). diff --git a/test/workspace-read-text.test.ts b/test/workspace-read-text.test.ts index a74740e9d..b8d8e744a 100644 --- a/test/workspace-read-text.test.ts +++ b/test/workspace-read-text.test.ts @@ -52,6 +52,11 @@ test('a CRLF file reads as its lines without the CR, and says so', async t => { const page = (await readMixed()).data as LinePage; assert.equal(page.content, ' 1\ta\n 2\tb\n 3\tc\n 4\td'); assert.equal(page.note, 'Line ends are mixed, 2 CRLF and 1 LF, and a CRLF end is shown without its CR. An edit matches text as read shows it, and keeps each line\'s end.'); + // One line whose own text ends in a CR, in a file of mixed ends: the note counts it, in the singular. + const { read: readKept } = await setup(t, 'x\r\r\ny\nz\r\n'); + const kept = (await readKept()).data as LinePage; + assert.equal(kept.content, ' 1\tx\r\r\n 2\ty\n 3\tz'); + assert.equal(kept.note, 'Line ends are mixed, 2 CRLF and 1 LF, and a CRLF end is shown without its CR, except on 1 line whose own text ends in a CR, which shows both CRs. An edit matches text as read shows it, and keeps each line\'s end.'); // A CR that doesn't end a line is the line's own. const { read: readCr } = await setup(t, 'a\rb\n'); assert.deepEqual((await readCr()).data, { path: 'work/text.txt', totalLines: 1, fromLine: 1, toLine: 1, content: ' 1\ta\rb' }); @@ -239,6 +244,26 @@ test('a first line that alone would pass the limit is cut where it fits', async assert.match(tiny.note!, /Line 1 is cut: read its rest with offsetChars=0 and limitChars=1500, then continue with offset=2\.$/); }); +test("a first line cut to fit stays within 2,000 characters, though a longer cut would fit once its rest loses a digit", async t => { + const { read } = await setup(t, 'a'.repeat(3_000)); + const withCut = (page: LinePage, cut: number) => JSON.stringify({ + ...page, + note: `Line 1 is cut: read its rest with offsetChars=${cut} and limitChars=${3_000 - cut}.`, + content: ` 1\t${'a'.repeat(cut)} [… ${3_000 - cut} more chars; read them with offsetChars=${cut}]`, + }).length; + const shape = (await read({}, 24_000)).data as LinePage; + // At this limit the cut at 2,000 is a character too long, and the cut at + // 2,001 would fit, since its rest's length, 999, has a digit fewer than + // 1,000, in the marker and in the note. + const limit = withCut(shape, 2_000) - 1; + assert.equal(withCut(shape, 2_001), limit); + const result = await read({}, limit); + assert.ok(stored(result) <= limit, `${stored(result)} > ${limit}`); + const page = result.data as LinePage; + assert.equal(/^ {5}1\t(a*) \[/.exec(page.content)![1].length, 1_999); + assert.equal(stored(result), withCut(page, 1_999)); +}); + test("a first line cut to fit is the longest cut that fits, where the rest's length loses a digit too", async t => { // The rest's length is in the marker and in the note, so at the cut where it // goes from 1,000 to 999 the page loses two characters for the one it gains, diff --git a/test/workspace-text.test.ts b/test/workspace-text.test.ts index 84421b1f4..6830e65b4 100644 --- a/test/workspace-text.test.ts +++ b/test/workspace-text.test.ts @@ -22,6 +22,10 @@ test('fileText gives the lines read shows, where each starts, and what the text assert.deepEqual([crKept.crlfEnds, crKept.lfEnds, crKept.crKept], [3, 0, 2]); assert.equal(fileText(Buffer.from('\r\r\n')).crKept, 1); assert.equal(fileText(Buffer.from('a\r\n\r\n')).crKept, 0, 'an empty line after a CRLF end is no CR of its own'); + // A line ended by a bare LF keeps nothing, though a CR sits before its last character. + const lfLine = fileText(Buffer.from('a\rb\nc\r\n')); + assert.deepEqual(lfLine.lines, ['a\rb', 'c']); + assert.deepEqual([lfLine.crlfEnds, lfLine.lfEnds, lfLine.crKept], [1, 1, 0]); const crlf = fileText(Buffer.from('one\r\ntwo\nthree')); assert.deepEqual(crlf.starts, [0, 5, 9]); From 196141297fc0637c02860d71b6c404517218be46 Mon Sep 17 00:00:00 2001 From: Abigail-2676 Date: Mon, 19 Oct 2026 09:46:58 -0400 Subject: [PATCH 30/51] fix(workspace): read and grep keep main's tool descriptions, and the results say the rest MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit An agent's request starts with its tool list, so a changed tool description makes every agent with the workspace's tools rewrite its whole cached context once, at its next request: a request measured after such a change read nothing of its prefix from the cache and wrote it all again. A tool's text is worth changing only for what a caller needs before calling, and everything the read and grep descriptions had gained here is learned by calling, so both go back to main's exact text. Each piece is said in the result instead: - a page that stops at the result limit, CRLF ends, a BOM, invalid UTF-8 and lines that keep a CR: the page's notes already said each; - "the stored text as it is, BOM and CRs included": main's "returns raw text" says it; - reading a cut line's rest: the marker now names both numbers, "[… read the rest with offsetChars=X and limitChars=N]" in place of "[… N more chars; read them with offsetChars=X]". numberedLine prints the read-on clause in place of the count when it's given, so edit's excerpts keep "[… N more chars]". The rest's length is still in the marker once, so the first-line cut's search and its four-code-unit bound stand; - a stopped character page: its note says "read the rest of what you asked for with offsetChars=X and limitChars=M", so it reads as the exception to main's "Keep limitChars when continuing". The changelog drops its cache line and says the marker gives both numbers. A page's least content is a cut first line, so the page-size figures there are measured again: 687 and 677 characters of path for a page with no notes, 347 and 329 with the text notes, 111 to 129 and 93 to 111 with a conflict's too. Co-Authored-By: Claude Opus 5.5 --- changelog.d/feat-workspace-read.changed.md | 7 +++---- src/modules/workspace/index.ts | 17 ++++----------- src/modules/workspace/read-page.ts | 5 +++-- src/modules/workspace/text.ts | 4 ++-- test/workspace-read-text.test.ts | 24 +++++++++++----------- test/workspace-text.test.ts | 2 +- 6 files changed, 25 insertions(+), 34 deletions(-) diff --git a/changelog.d/feat-workspace-read.changed.md b/changelog.d/feat-workspace-read.changed.md index 0d478f9b9..cd129b62d 100644 --- a/changelog.d/feat-workspace-read.changed.md +++ b/changelog.d/feat-workspace-read.changed.md @@ -1,6 +1,5 @@ -- The workspace's `read` shows a file's text as it is, and says so. A line page shows a CRLF file's lines without the CR before each line feed, and leaves out a BOM. A line whose own text ends in a CR shows its line end's CR too, since `edit` reads a CR before a line feed as part of the line end, so text copied from the line still matches it. Its note says when the file has CRLF ends or a BOM, counting a mix of CRLF and LF ends and the lines that show both CRs. It also says when the file isn't valid UTF-8: each invalid byte sequence shows as U+FFFD, `edit` refuses the file, and `write` replaces it whole. Before, each line of a CRLF file ended in a CR, a BOM came first in the first line as U+FEFF, and invalid bytes showed as U+FFFD with no note. A character page still reads the stored text as it is, BOM and CRs included, and the tool's description now says so. -- On a page for a call that carries a `resultMaxChars`, such as a model's or a running script's, a line over 2,000 characters is cut, with the `offsetChars` where its rest starts and how many characters the rest has, and the tool's description says to read the rest with that count as `limitChars`. Before, a line page showed every line whole, however long. -- A `read` page stops before its result would pass `resultMaxChars`, the most of it the framework stores whole. For a model's call that's its agent's inline cap, 24,000 characters by default. Its note says where to continue: the `offset` after its last line, or for a character page the `offsetChars` it also gives as `nextOffsetChars`, with the `limitChars` left of what it was asked for, so a reader that follows the notes ends where it asked to. The read tool's description now says so, where it said to keep `limitChars` when continuing. Where even a page's first line would pass the limit, that line is cut shorter, at the most the page has room for. Where a page's last line is cut, its note names the `offsetChars` where that line's rest starts and the `limitChars` that reads it to the line's end, before the next line's `offset` if the file goes on. A page always moves on, so where its own fields (the path and its notes) and the least it can show pass a small limit, it passes the limit. At 1,000 characters, a page with no notes does that only for a path of about 690 characters or more. Its notes take room from that: with all three notes on a file's text and a conflict's, a path of 100 to 140 characters does it, and a note that names a symlink's target, as one saying disk couldn't be checked can, takes as much room as the target is long. Before, a page of up to 2,000 lines, or of `limitChars` characters, went out whole, and for a result that passed the limit the framework kept only its start, cut mid-JSON, writing the whole of it to a workspace file where the agent has one. A caller that pages 2,000 lines at a time and starts the next page at line 2,001 now starts where the note says. A running `code_execution` script's call has a limit of 5,000,000 characters, so its line pages still stop at the line limit, 2,000 by default, and its character pages at `limitChars`, unless a page would pass that. Its line pages cut a line over 2,000 characters and leave out CRs and a BOM, as a model's do, and a character page gives it the stored text exactly. A call made in process, through `handleToolCall`, `executeToolCall` or `ModuleContext.callTool`, carries no `resultMaxChars` unless its caller sets one. Its line pages stop at the line limit alone and show every line whole, as they did before, though now without the CRs and the BOM. +- The workspace's `read` shows a file's text as it is, and says so. A line page shows a CRLF file's lines without the CR before each line feed, and leaves out a BOM. A line whose own text ends in a CR shows its line end's CR too, since `edit` reads a CR before a line feed as part of the line end, so text copied from the line still matches it. Its note says when the file has CRLF ends or a BOM, counting a mix of CRLF and LF ends and the lines that show both CRs. It also says when the file isn't valid UTF-8: each invalid byte sequence shows as U+FFFD, `edit` refuses the file, and `write` replaces it whole. Before, each line of a CRLF file ended in a CR, a BOM came first in the first line as U+FEFF, and invalid bytes showed as U+FFFD with no note. A character page still reads the stored text as it is, BOM and CRs included. +- On a page for a call that carries a `resultMaxChars`, such as a model's or a running script's, a line over 2,000 characters is cut, with a marker that gives the `offsetChars` and `limitChars` that read its rest. Before, a line page showed every line whole, however long. +- A `read` page stops before its result would pass `resultMaxChars`, the most of it the framework stores whole. For a model's call that's its agent's inline cap, 24,000 characters by default. Its note says where to continue: the `offset` after its last line, or for a character page the `offsetChars` it also gives as `nextOffsetChars`, with the `limitChars` left of what it was asked for, so a reader that follows the notes ends where it asked to. Where even a page's first line would pass the limit, that line is cut shorter, at the most the page has room for. Where a page's last line is cut, its note names the `offsetChars` where that line's rest starts and the `limitChars` that reads it to the line's end, before the next line's `offset` if the file goes on. A page always moves on, so where its own fields (the path and its notes) and the least it can show pass a small limit, it passes the limit. At 1,000 characters, a page with no notes does that only for a path of about 680 characters or more. Its notes take room from that: with all three notes on a file's text and a conflict's, a path of 90 to 130 characters does it, and a note that names a symlink's target, as one saying disk couldn't be checked can, takes as much room as the target is long. Before, a page of up to 2,000 lines, or of `limitChars` characters, went out whole, and for a result that passed the limit the framework kept only its start, cut mid-JSON, writing the whole of it to a workspace file where the agent has one. A caller that pages 2,000 lines at a time and starts the next page at line 2,001 now starts where the note says. A running `code_execution` script's call has a limit of 5,000,000 characters, so its line pages still stop at the line limit, 2,000 by default, and its character pages at `limitChars`, unless a page would pass that. Its line pages cut a line over 2,000 characters and leave out CRs and a BOM, as a model's do, and a character page gives it the stored text exactly. A call made in process, through `handleToolCall`, `executeToolCall` or `ModuleContext.callTool`, carries no `resultMaxChars` unless its caller sets one. Its line pages stop at the line limit alone and show every line whole, as they did before, though now without the CRs and the BOM. - The workspace's `grep` leaves out a CRLF end's CR, for matching as well as display, as `read` does, so `foo$` finds a CRLF file's line too, as it does under ripgrep's `--crlf`. A match's text and context are its lines as `read` shows them, but each line whole, where a `read` page for a call with a result limit cuts a line over 2,000 characters. A BOM isn't part of the first line, and a file with CRLF ends says so in its result's `lineEndings` (`CRLF`, or `mixed`). Before, a CRLF file's lines ended in a CR, so `foo$` missed them and a match's text copied into an edit carried the CR. A pattern written to find the CR of a CRLF line end no longer matches, except on a line whose own text ends in a CR, which keeps its line end's CR as `read` shows it; a CR that doesn't come before a line feed is still part of its line. - The workspace's `read` first takes in a change on disk to a file the workspace holds, as a listing and an `edit` do, and shows that text. Its note says so when disk's copy had changed since the workspace last agreed with it. It also says when disk's copy couldn't be checked, or disk has none and nothing records whether it was deleted there, so it shows the workspace's copy; where the check gave up because the selected branch kept changing, it says to read again. A file disk deleted answers that disk deleted it and the deletion was taken in, and a file disk made binary or too large answers that it's on disk but not stored. Where the path's stored copy is a draft and disk changed or deleted it too, the read shows the draft, and its note names the conflict and the ways to settle it, a sync, or a forced materialize on a mount that isn't read-only: for a deletion, a sync of the path takes it and removes the workspace's copy. Before, a read of a path the workspace held didn't look at disk, so it could show a copy another writer had since changed; a path it didn't hold was observed, as now. -- The `read` and `grep` tool descriptions changed, so the first request after the upgrade that carries the workspace's tools misses the prompt cache from its tool block on. diff --git a/src/modules/workspace/index.ts b/src/modules/workspace/index.ts index cb0868a99..ce0a74ecc 100644 --- a/src/modules/workspace/index.ts +++ b/src/modules/workspace/index.ts @@ -1385,16 +1385,9 @@ export class WorkspaceModule implements Module { { name: 'read', description: 'Read a file from the workspace. Returns content with line numbers. ' - + 'Reads at most 2000 lines per call by default — use offset/limit to page through larger files — ' - + 'and stops sooner where a page would pass the result limit; the note says where to continue. ' - + 'A line over 2000 characters is cut, with the offsetChars where its rest starts and how many characters the rest has; ' - + 'read the rest with that count as limitChars. ' - + 'A line page shows CRLF line ends without the CR, and no BOM; its note says when the file has either, ' - + 'or isn\'t valid UTF-8. A line whose own text ends in a CR shows its line end\'s CR too. ' - + 'For long lines or spill files, use offsetChars/limitChars instead: returns the stored text as it is, ' - + 'BOM and CRs included, without line numbers, with nextOffsetChars (null at EOF), stopped at the result limit too. ' - + 'When a character page stops at the result limit, its note gives the offsetChars and limitChars that read the rest of what you asked for. ' - + 'Otherwise, continue from nextOffsetChars with the same limitChars. ' + + 'Reads at most 2000 lines per call by default — use offset/limit to page through larger files. ' + + 'For long lines or spill files, use offsetChars/limitChars instead: returns raw text without ' + + 'line numbers, with nextOffsetChars (null at EOF). Keep limitChars when continuing. ' + 'Character offsets count UTF-16 code units; do not mix character and line parameters.', inputSchema: { type: 'object' as const, @@ -1539,9 +1532,7 @@ export class WorkspaceModule implements Module { + 'which version matched: the workspace\'s (with its state); the disk version kept with a conflict ' + '(conflicting-disk); or, once disk has changed since the conflict was recorded, that recorded version ' + '(recorded-disk). Disk versions it could not search are listed as skipped: disk-only files (binary or ' - + 'over the size limit), and a conflict\'s disk version that is binary, oversize, or newer than the one recorded. ' - + 'In a file with CRLF line ends, each line is matched and shown as read shows it, without its line end\'s CR ' - + 'unless its own text ends in a CR, and the file\'s result says lineEndings. A BOM isn\'t part of the first line.', + + 'over the size limit), and a conflict\'s disk version that is binary, oversize, or newer than the one recorded.', inputSchema: { type: 'object' as const, properties: { diff --git a/src/modules/workspace/read-page.ts b/src/modules/workspace/read-page.ts index 9ef60213c..cabd29ffe 100644 --- a/src/modules/workspace/read-page.ts +++ b/src/modules/workspace/read-page.ts @@ -65,7 +65,8 @@ export function linePage(path: string, file: FileText, offset: number, limit: nu } const last = Math.min(first + limit, total); const shown = (i: number, maxLineChars = MAX_LINE_CHARS) => - numberedLine(lines[i], i + 1, maxLineChars, (cut) => `read them with offsetChars=${starts[i] + cut}`); + numberedLine(lines[i], i + 1, maxLineChars, + (cut) => `read the rest with offsetChars=${starts[i] + cut} and limitChars=${lines[i].length - cut}`); // Where the rest of line i starts in the stored text, shown at most // maxLineChars long, as its marker says; null for a line shown whole. const restOf = (i: number, maxLineChars: number): number | null => @@ -169,7 +170,7 @@ export function characterPage(path: string, file: FileText, offsetChars: number, // that as limitChars, so a reader that follows its notes ends where it // asked to: at a cut line's end, for the rest a line page's note names. const stoppedNote = (to: number, left: number) => - `Stopped at character ${to} to stay within the ${maxChars}-character result limit; continue with offsetChars=${to} and limitChars=${left}.`; + `Stopped at character ${to} to stay within the ${maxChars}-character result limit; read the rest of what you asked for with offsetChars=${to} and limitChars=${left}.`; const page = (to: number, note: string | undefined) => ({ path, totalChars: text.length, diff --git a/src/modules/workspace/text.ts b/src/modules/workspace/text.ts index dc9298e21..0a00189d9 100644 --- a/src/modules/workspace/text.ts +++ b/src/modules/workspace/text.ts @@ -80,13 +80,13 @@ export function safeCut(value: string, at: number): number { /** * One line numbered the way `cat -n` numbers it: a six-wide number, a tab and * the line. A line longer than `maxChars` is cut, saying how many characters - * it holds past the cut, and `readOn` adds how to read them. + * it holds past the cut, or, where `readOn` is given, how to read them. */ export function numberedLine(line: string, n: number, maxChars = MAX_LINE_CHARS, readOn?: (cut: number) => string): string { const number = String(n).padStart(6); if (line.length <= maxChars) return `${number}\t${line}`; const cut = safeCut(line, maxChars); - return `${number}\t${line.slice(0, cut)} [… ${line.length - cut} more chars${readOn ? `; ${readOn(cut)}` : ''}]`; + return `${number}\t${line.slice(0, cut)} [… ${readOn ? readOn(cut) : `${line.length - cut} more chars`}]`; } /** Lines numbered from `start`, each as numberedLine shows it. */ diff --git a/test/workspace-read-text.test.ts b/test/workspace-read-text.test.ts index b8d8e744a..1c3b4a1bd 100644 --- a/test/workspace-read-text.test.ts +++ b/test/workspace-read-text.test.ts @@ -94,7 +94,7 @@ test('a long line is cut at 2000 characters, with the offsetChars that reads the // The cut doesn't split the emoji's surrogate pair: it falls before it. const rest = long.length - 1999; const at = 1 + 'short\r\n'.length + 1999; - assert.equal(lines[1], ` 2\t${'x'.repeat(1999)} [… ${rest} more chars; read them with offsetChars=${at}]`); + assert.equal(lines[1], ` 2\t${'x'.repeat(1999)} [… read the rest with offsetChars=${at} and limitChars=${rest}]`); // A script's call, at 5,000,000 characters, cuts the line the same way. assert.equal(((await read({}, 5_000_000)).data as LinePage).content.split('\n')[1], lines[1]); const more = (await read({ offsetChars: at, limitChars: rest })).data as CharPage; @@ -154,7 +154,7 @@ test('a line page is the longest that fits, though the page that ends where it w const two = (limit: number) => ({ path: 'work/text.txt', totalLines: 3, fromLine: 1, toLine: 2, note: `Stopped at line 2 to stay within the ${limit}-character result limit (file has 3). Continue with offset=3.`, - content: ` 1\t${'q'.repeat(2000)} [… 500 more chars; read them with offsetChars=2000]\n 2\tz`, + content: ` 1\t${'q'.repeat(2000)} [… read the rest with offsetChars=2000 and limitChars=500]\n 2\tz`, }); const limit = JSON.stringify(two(9_999)).length; assert.equal(String(limit).length, 4, 'the limit has as many digits as the 9,999 it was measured with'); @@ -179,7 +179,7 @@ test('a page with notes holds to the result limit with its notes', async t => { const result = await readBad({ offsetChars: 0, limitChars: 100_000 }, limit); assert.ok(stored(result) <= limit, `${stored(result)} > ${limit}`); const page = result.data as CharPage; - assert.equal(page.note, `Stopped at character ${page.nextOffsetChars} to stay within the ${limit}-character result limit; continue with offsetChars=${page.nextOffsetChars} and limitChars=${page.totalChars - page.nextOffsetChars!}. The file isn't valid UTF-8: each invalid byte sequence shows as U+FFFD. Edit refuses this file, and write replaces it whole.`); + assert.equal(page.note, `Stopped at character ${page.nextOffsetChars} to stay within the ${limit}-character result limit; read the rest of what you asked for with offsetChars=${page.nextOffsetChars} and limitChars=${page.totalChars - page.nextOffsetChars!}. The file isn't valid UTF-8: each invalid byte sequence shows as U+FFFD. Edit refuses this file, and write replaces it whole.`); } }); @@ -205,7 +205,7 @@ test('a first line that alone would pass the limit is cut where it fits', async assert.ok(stored(result) <= 1_000, `${stored(result)}`); const page = result.data as LinePage; assert.equal(page.toLine, 1); - const [, shown, rest, at] = /^ {5}1\t("*) \[… (\d+) more chars; read them with offsetChars=(\d+)\]$/.exec(page.content)!; + const [, shown, at, rest] = /^ {5}1\t("*) \[… read the rest with offsetChars=(\d+) and limitChars=(\d+)\]$/.exec(page.content)!; assert.ok(shown.length > 0); assert.equal(Number(at), shown.length); assert.equal(shown.length + Number(rest), 1500); @@ -218,7 +218,7 @@ test('a first line that alone would pass the limit is cut where it fits', async return JSON.stringify({ ...cutPage, ...(cutPage.note !== undefined ? { note: cutPage.note.replace(/offsetChars=\d+ and limitChars=\d+/, `offsetChars=${more} and limitChars=${1500 - more}`) } : {}), - content: ` 1\t${'"'.repeat(more)} [… ${1500 - more} more chars; read them with offsetChars=${more}]`, + content: ` 1\t${'"'.repeat(more)} [… read the rest with offsetChars=${more} and limitChars=${1500 - more}]`, }).length; }; assert.ok(oneMore(page, shown.length) > 1_000, `${oneMore(page, shown.length)}`); @@ -240,7 +240,7 @@ test('a first line that alone would pass the limit is cut where it fits', async // A limit with no room beside the page's own fields still shows where the line's text is. const tiny = (await read({}, 50)).data as LinePage; assert.equal(tiny.toLine, 1); - assert.equal(tiny.content, ' 1\t [… 1500 more chars; read them with offsetChars=0]'); + assert.equal(tiny.content, ' 1\t [… read the rest with offsetChars=0 and limitChars=1500]'); assert.match(tiny.note!, /Line 1 is cut: read its rest with offsetChars=0 and limitChars=1500, then continue with offset=2\.$/); }); @@ -249,7 +249,7 @@ test("a first line cut to fit stays within 2,000 characters, though a longer cut const withCut = (page: LinePage, cut: number) => JSON.stringify({ ...page, note: `Line 1 is cut: read its rest with offsetChars=${cut} and limitChars=${3_000 - cut}.`, - content: ` 1\t${'a'.repeat(cut)} [… ${3_000 - cut} more chars; read them with offsetChars=${cut}]`, + content: ` 1\t${'a'.repeat(cut)} [… read the rest with offsetChars=${cut} and limitChars=${3_000 - cut}]`, }).length; const shape = (await read({}, 24_000)).data as LinePage; // At this limit the cut at 2,000 is a character too long, and the cut at @@ -273,7 +273,7 @@ test("a first line cut to fit is the longest cut that fits, where the rest's len const withCut = (page: LinePage, cut: number) => JSON.stringify({ ...page, note: `Line 1 is cut: read its rest with offsetChars=${cut} and limitChars=${1_500 - cut}.`, - content: ` 1\t${'a'.repeat(cut)} [… ${1_500 - cut} more chars; read them with offsetChars=${cut}]`, + content: ` 1\t${'a'.repeat(cut)} [… read the rest with offsetChars=${cut} and limitChars=${1_500 - cut}]`, }).length; const cuts = new Set(); for (let limit = 700; limit <= 760; limit++) { @@ -294,7 +294,7 @@ test('a first line cut to fit, read from a later offset, is numbered and placed const { read } = await setup(t, `a\nb\n${'"'.repeat(1500)}\nnext\n`); const page = (await read({ offset: 3 }, 1_000)).data as LinePage; assert.deepEqual([page.fromLine, page.toLine], [3, 3]); - const [, shown, rest, at] = /^ {5}3\t("*) \[… (\d+) more chars; read them with offsetChars=(\d+)\]$/.exec(page.content)!; + const [, shown, at, rest] = /^ {5}3\t("*) \[… read the rest with offsetChars=(\d+) and limitChars=(\d+)\]$/.exec(page.content)!; assert.ok(shown.length > 0 && shown.length < 1500, `${shown.length}`); // The rest starts past lines 1 and 2, "a\n" and "b\n", in the stored text. assert.equal(Number(at), 4 + shown.length); @@ -326,7 +326,7 @@ test("a cut line's rest read through several character pages, following only the assert.ok(stored(result) <= 1_000, `${stored(result)}`); const page = result.data as CharPage; rest += page.content; - const next = page.note === undefined ? null : /continue with offsetChars=(\d+) and limitChars=(\d+)/.exec(page.note); + const next = page.note === undefined ? null : /read the rest of what you asked for with offsetChars=(\d+) and limitChars=(\d+)/.exec(page.note); if (next === null) { // The last page holds all it was asked for, and ends at the line's end. assert.equal(page.note, undefined); @@ -406,7 +406,7 @@ test('a character page stops at the result limit too, with nextOffsetChars', asy recovered += page.content; if (page.nextOffsetChars === null) break; // What's left of what was asked for is the rest of the text. - assert.equal(page.note, `Stopped at character ${page.nextOffsetChars} to stay within the ${limit}-character result limit; continue with offsetChars=${page.nextOffsetChars} and limitChars=${text.length - page.nextOffsetChars}.`); + assert.equal(page.note, `Stopped at character ${page.nextOffsetChars} to stay within the ${limit}-character result limit; read the rest of what you asked for with offsetChars=${page.nextOffsetChars} and limitChars=${text.length - page.nextOffsetChars}.`); offsetChars = page.nextOffsetChars; limitChars = text.length - page.nextOffsetChars; } @@ -434,7 +434,7 @@ test('a character page moved on by a whole character keeps a surrogate pair whol const { read } = await setup(t, '😀abc'); assert.deepEqual((await read({ offsetChars: 0, limitChars: 100 }, 50)).data, { path: 'work/text.txt', totalChars: 5, offsetChars: 0, nextOffsetChars: 2, - note: 'Stopped at character 2 to stay within the 50-character result limit; continue with offsetChars=2 and limitChars=3.', + note: 'Stopped at character 2 to stay within the 50-character result limit; read the rest of what you asked for with offsetChars=2 and limitChars=3.', content: '😀', }); // Where that step reaches what the page was asked for, the page holds all of it. diff --git a/test/workspace-text.test.ts b/test/workspace-text.test.ts index 6830e65b4..f1cb05dfa 100644 --- a/test/workspace-text.test.ts +++ b/test/workspace-text.test.ts @@ -53,7 +53,7 @@ test('numberedLine numbers a line as cat -n does, and cuts a long one without sp assert.equal(numberedLine('x'.repeat(2001), 1), ` 1\t${'x'.repeat(2000)} [… 1 more chars]`); const emoji = 'x'.repeat(4) + '😀' + 'y'; assert.equal(safeCut(emoji, 5), 4); - assert.equal(numberedLine(emoji, 2, 5, (cut) => `from ${cut}`), ` 2\txxxx [… 3 more chars; from 4]`); + assert.equal(numberedLine(emoji, 2, 5, (cut) => `from ${cut}`), ` 2\txxxx [… from 4]`); assert.equal(numberedLine('abc', 3, 0), ' 3\t [… 3 more chars]'); }); From b7a5a1dfcadc5800feba60504939066e7cb730b8 Mon Sep 17 00:00:00 2001 From: Abigail-2676 Date: Mon, 19 Oct 2026 09:46:59 -0400 Subject: [PATCH 31/51] feat(workspace): grep says why a pattern for a CR can't find a CRLF end's With the descriptions back to main's, nothing told an agent that grep matches lines as read shows them, without a CRLF end's CR, before it drew the wrong conclusion from a CR search that found nothing. Now, when the pattern names a CR (a literal one, or `\r` whose backslash isn't itself escaped) and a file it searched has CRLF ends, the answer carries a note: "The pattern names a CR, and a file searched has CRLF line ends. Lines are matched as read shows them, which leaves out a CRLF end's CR unless the line's own text ends in a CR. read with offsetChars shows every CR." A CR pattern over files with no CRLF end finds every CR, as on main, and gets no note. Its test greps a CRLF file with `\r`, a literal CR, `o[\r]?$` and `\\\r`, each with the note, and `foo`, `\\r` and an LF file with none, and finds a CR inside a line of a file without CRLF ends. Co-Authored-By: Claude Opus 5.5 --- changelog.d/feat-workspace-read.changed.md | 2 +- src/modules/workspace/index.ts | 20 +++++++++++++++++++ test/workspace-file-tools.test.ts | 23 +++++++++++++++++++--- 3 files changed, 41 insertions(+), 4 deletions(-) diff --git a/changelog.d/feat-workspace-read.changed.md b/changelog.d/feat-workspace-read.changed.md index cd129b62d..69523c593 100644 --- a/changelog.d/feat-workspace-read.changed.md +++ b/changelog.d/feat-workspace-read.changed.md @@ -1,5 +1,5 @@ - The workspace's `read` shows a file's text as it is, and says so. A line page shows a CRLF file's lines without the CR before each line feed, and leaves out a BOM. A line whose own text ends in a CR shows its line end's CR too, since `edit` reads a CR before a line feed as part of the line end, so text copied from the line still matches it. Its note says when the file has CRLF ends or a BOM, counting a mix of CRLF and LF ends and the lines that show both CRs. It also says when the file isn't valid UTF-8: each invalid byte sequence shows as U+FFFD, `edit` refuses the file, and `write` replaces it whole. Before, each line of a CRLF file ended in a CR, a BOM came first in the first line as U+FEFF, and invalid bytes showed as U+FFFD with no note. A character page still reads the stored text as it is, BOM and CRs included. - On a page for a call that carries a `resultMaxChars`, such as a model's or a running script's, a line over 2,000 characters is cut, with a marker that gives the `offsetChars` and `limitChars` that read its rest. Before, a line page showed every line whole, however long. - A `read` page stops before its result would pass `resultMaxChars`, the most of it the framework stores whole. For a model's call that's its agent's inline cap, 24,000 characters by default. Its note says where to continue: the `offset` after its last line, or for a character page the `offsetChars` it also gives as `nextOffsetChars`, with the `limitChars` left of what it was asked for, so a reader that follows the notes ends where it asked to. Where even a page's first line would pass the limit, that line is cut shorter, at the most the page has room for. Where a page's last line is cut, its note names the `offsetChars` where that line's rest starts and the `limitChars` that reads it to the line's end, before the next line's `offset` if the file goes on. A page always moves on, so where its own fields (the path and its notes) and the least it can show pass a small limit, it passes the limit. At 1,000 characters, a page with no notes does that only for a path of about 680 characters or more. Its notes take room from that: with all three notes on a file's text and a conflict's, a path of 90 to 130 characters does it, and a note that names a symlink's target, as one saying disk couldn't be checked can, takes as much room as the target is long. Before, a page of up to 2,000 lines, or of `limitChars` characters, went out whole, and for a result that passed the limit the framework kept only its start, cut mid-JSON, writing the whole of it to a workspace file where the agent has one. A caller that pages 2,000 lines at a time and starts the next page at line 2,001 now starts where the note says. A running `code_execution` script's call has a limit of 5,000,000 characters, so its line pages still stop at the line limit, 2,000 by default, and its character pages at `limitChars`, unless a page would pass that. Its line pages cut a line over 2,000 characters and leave out CRs and a BOM, as a model's do, and a character page gives it the stored text exactly. A call made in process, through `handleToolCall`, `executeToolCall` or `ModuleContext.callTool`, carries no `resultMaxChars` unless its caller sets one. Its line pages stop at the line limit alone and show every line whole, as they did before, though now without the CRs and the BOM. -- The workspace's `grep` leaves out a CRLF end's CR, for matching as well as display, as `read` does, so `foo$` finds a CRLF file's line too, as it does under ripgrep's `--crlf`. A match's text and context are its lines as `read` shows them, but each line whole, where a `read` page for a call with a result limit cuts a line over 2,000 characters. A BOM isn't part of the first line, and a file with CRLF ends says so in its result's `lineEndings` (`CRLF`, or `mixed`). Before, a CRLF file's lines ended in a CR, so `foo$` missed them and a match's text copied into an edit carried the CR. A pattern written to find the CR of a CRLF line end no longer matches, except on a line whose own text ends in a CR, which keeps its line end's CR as `read` shows it; a CR that doesn't come before a line feed is still part of its line. +- The workspace's `grep` leaves out a CRLF end's CR, for matching as well as display, as `read` does, so `foo$` finds a CRLF file's line too, as it does under ripgrep's `--crlf`. A match's text and context are its lines as `read` shows them, but each line whole, where a `read` page for a call with a result limit cuts a line over 2,000 characters. A BOM isn't part of the first line, and a file with CRLF ends says so in its result's `lineEndings` (`CRLF`, or `mixed`). Before, a CRLF file's lines ended in a CR, so `foo$` missed them and a match's text copied into an edit carried the CR. A pattern written to find the CR of a CRLF line end no longer matches, except on a line whose own text ends in a CR, which keeps its line end's CR as `read` shows it. When a pattern names a CR and a file it searched has CRLF ends, the result's `note` says so, and that a character `read` shows every CR. A CR that doesn't come before a line feed is still part of its line. - The workspace's `read` first takes in a change on disk to a file the workspace holds, as a listing and an `edit` do, and shows that text. Its note says so when disk's copy had changed since the workspace last agreed with it. It also says when disk's copy couldn't be checked, or disk has none and nothing records whether it was deleted there, so it shows the workspace's copy; where the check gave up because the selected branch kept changing, it says to read again. A file disk deleted answers that disk deleted it and the deletion was taken in, and a file disk made binary or too large answers that it's on disk but not stored. Where the path's stored copy is a draft and disk changed or deleted it too, the read shows the draft, and its note names the conflict and the ways to settle it, a sync, or a forced materialize on a mount that isn't read-only: for a deletion, a sync of the path takes it and removes the workspace's copy. Before, a read of a path the workspace held didn't look at disk, so it could show a copy another writer had since changed; a path it didn't hold was observed, as now. diff --git a/src/modules/workspace/index.ts b/src/modules/workspace/index.ts index ce0a74ecc..9a32822fa 100644 --- a/src/modules/workspace/index.ts +++ b/src/modules/workspace/index.ts @@ -3744,8 +3744,10 @@ export class WorkspaceModule implements Module { // `foo$` finds a CRLF file's line, as it does under ripgrep's --crlf, and // the line copies into an edit as read's do. ripgrep's --crlf still prints // the line with its CR. + let searchedCrlf = false; const search = (bytes: Buffer): Searched => { const { lines, crlfEnds, lfEnds } = fileText(bytes); + if (crlfEnds > 0) searchedCrlf = true; const found: GrepMatch[] = []; for (let i = 0; i < lines.length; i++) { if (regex.test(lines[i])) { @@ -3886,6 +3888,9 @@ export class WorkspaceModule implements Module { pattern: input.pattern, results, totalMatches: results.reduce((sum, r) => sum + r.matches.length, 0), + // A pattern written for a CR can't find a CRLF end's, which lines as + // read shows them leave out, so the answer says why it didn't. + ...(searchedCrlf && namesCr(input.pattern) ? { note: CRLF_CR_NOTE } : {}), ...(skipped.length > 0 ? { skipped } : {}), ...(incomplete.length > 0 ? { incomplete } : {}), }, @@ -4762,6 +4767,21 @@ function inputProblem( return null; } +const CRLF_CR_NOTE = 'The pattern names a CR, and a file searched has CRLF line ends. Lines are matched as read shows them, ' + + 'which leaves out a CRLF end\'s CR unless the line\'s own text ends in a CR. read with offsetChars shows every CR.'; + +/** Whether a grep pattern names a CR: a literal one, or `\r` whose backslash isn't itself escaped. */ +function namesCr(pattern: string): boolean { + for (let i = 0; i < pattern.length; i++) { + if (pattern[i] === '\r') return true; + if (pattern[i] === '\\') { + if (pattern[i + 1] === 'r') return true; + i++; + } + } + return false; +} + /** * Convert a glob pattern to a RegExp. Throws, naming the pattern, for a brace * group it can't read: a "{" that never closes, a group inside another, or a diff --git a/test/workspace-file-tools.test.ts b/test/workspace-file-tools.test.ts index c68be372d..a5f5de60b 100644 --- a/test/workspace-file-tools.test.ts +++ b/test/workspace-file-tools.test.ts @@ -411,9 +411,26 @@ test('grep matches and shows a CRLF file\'s lines without the CR, as read shows assert.equal('lineEndings' in byFile['work/lf.txt'], false); // A BOM is no part of the first line, as read doesn't show it. assert.deepEqual(byFile['work/bom.txt'].matches, [{ line: 1, text: 'foo', context: ['foo'] }]); - // The cost: a pattern written to find the CR at a line's end no longer does. - const cr = await call('grep', { pattern: '\\r', path: 'work/crlf.txt' }); - assert.deepEqual((cr.data as { results: unknown[] }).results, []); + // The cost: a pattern written to find the CR at a line's end no longer + // does, and the answer says why. + const note = "The pattern names a CR, and a file searched has CRLF line ends. Lines are matched as read shows them, " + + "which leaves out a CRLF end's CR unless the line's own text ends in a CR. read with offsetChars shows every CR."; + for (const pattern of ['\\r', '\r', 'o[\\r]?$', '\\\\\\r']) { + const cr = await call('grep', { pattern, path: 'work/crlf.txt' }); + const data = cr.data as { results: Array<{ matches: unknown[] }>; note?: string }; + assert.equal(data.note, note, JSON.stringify(pattern)); + if (pattern !== 'o[\\r]?$') assert.deepEqual(data.results, [], JSON.stringify(pattern)); + } + // A pattern that names no CR, and a CR pattern over files without a CRLF + // end, get no note. + for (const [pattern, path] of [['foo', 'work/crlf.txt'], ['\\\\r', 'work/crlf.txt'], ['\\r', 'work/lf.txt']]) { + const data = (await call('grep', { pattern, path })).data as object; + assert.equal('note' in data, false, JSON.stringify([pattern, path])); + } + writeFileSync(join(dir, 'lone.txt'), 'a\rb\n'); + const lone = (await call('grep', { pattern: '\\r', path: 'work/lone.txt' })).data as { results: Array<{ matches: unknown[] }> }; + assert.equal('note' in lone, false); + assert.deepEqual(lone.results[0].matches, [{ line: 1, text: 'a\rb' }]); }); test('text copied from a read or a grep of a CRLF file matches in an edit, and the file keeps its line ends and BOM', async (t) => { From 813c60dd11d040954f067d24fd9408c84513a24a Mon Sep 17 00:00:00 2001 From: Abigail-2676 Date: Mon, 19 Oct 2026 10:09:07 -0400 Subject: [PATCH 32/51] fix(workspace): grep's CR note takes the usual escapes for a CR, not \r alone grep compiles new RegExp(pattern) with no flags, which reads \x0d, \x0D, \u000d, \cM, \cm and \015 (and so [\x0d]) as a CR, and namesCr took only a literal CR and \r, so a CR search written the second way got no note. namesCr now takes any of r, x0[dD], u000[dD], c[mM] and 015 after a backslash that isn't itself escaped. \u{d} needs the u flag, which grep doesn't set, and \R or \C are identity escapes there, so neither counts. A pattern can still match a CR another way, such as \15 in a pattern of fewer than 15 groups or a class range that spans U+000D, and gets no note then. The grep test adds each of those spellings to the patterns that get the note, and \\x0d (an escaped backslash) and \R to those that don't. Co-Authored-By: Claude Opus 5.5 --- src/modules/workspace/index.ts | 8 ++++++-- test/workspace-file-tools.test.ts | 4 ++-- 2 files changed, 8 insertions(+), 4 deletions(-) diff --git a/src/modules/workspace/index.ts b/src/modules/workspace/index.ts index 9a32822fa..efb1ba04b 100644 --- a/src/modules/workspace/index.ts +++ b/src/modules/workspace/index.ts @@ -4770,12 +4770,16 @@ function inputProblem( const CRLF_CR_NOTE = 'The pattern names a CR, and a file searched has CRLF line ends. Lines are matched as read shows them, ' + 'which leaves out a CRLF end\'s CR unless the line\'s own text ends in a CR. read with offsetChars shows every CR.'; -/** Whether a grep pattern names a CR: a literal one, or `\r` whose backslash isn't itself escaped. */ +/** + * Whether a grep pattern names a CR: a literal one, or an escape that a + * pattern compiled without flags reads as one (`\r`, `\x0d`, `\u000d`, `\cM`, + * `\015`) whose backslash isn't itself escaped. + */ function namesCr(pattern: string): boolean { for (let i = 0; i < pattern.length; i++) { if (pattern[i] === '\r') return true; if (pattern[i] === '\\') { - if (pattern[i + 1] === 'r') return true; + if (/^(?:r|x0[dD]|u000[dD]|c[mM]|015)/.test(pattern.slice(i + 1, i + 6))) return true; i++; } } diff --git a/test/workspace-file-tools.test.ts b/test/workspace-file-tools.test.ts index a5f5de60b..f97133880 100644 --- a/test/workspace-file-tools.test.ts +++ b/test/workspace-file-tools.test.ts @@ -415,7 +415,7 @@ test('grep matches and shows a CRLF file\'s lines without the CR, as read shows // does, and the answer says why. const note = "The pattern names a CR, and a file searched has CRLF line ends. Lines are matched as read shows them, " + "which leaves out a CRLF end's CR unless the line's own text ends in a CR. read with offsetChars shows every CR."; - for (const pattern of ['\\r', '\r', 'o[\\r]?$', '\\\\\\r']) { + for (const pattern of ['\\r', '\r', 'o[\\r]?$', '\\\\\\r', '\\x0d', '\\x0D', '\\u000d', '\\cM', '\\cm', '\\015', '[\\x0d]']) { const cr = await call('grep', { pattern, path: 'work/crlf.txt' }); const data = cr.data as { results: Array<{ matches: unknown[] }>; note?: string }; assert.equal(data.note, note, JSON.stringify(pattern)); @@ -423,7 +423,7 @@ test('grep matches and shows a CRLF file\'s lines without the CR, as read shows } // A pattern that names no CR, and a CR pattern over files without a CRLF // end, get no note. - for (const [pattern, path] of [['foo', 'work/crlf.txt'], ['\\\\r', 'work/crlf.txt'], ['\\r', 'work/lf.txt']]) { + for (const [pattern, path] of [['foo', 'work/crlf.txt'], ['\\\\r', 'work/crlf.txt'], ['\\\\x0d', 'work/crlf.txt'], ['\\R', 'work/crlf.txt'], ['\\r', 'work/lf.txt']]) { const data = (await call('grep', { pattern, path })).data as object; assert.equal('note' in data, false, JSON.stringify([pattern, path])); } From 456e5db1b42ecf8130ae1024eacf974ca4bebe64 Mon Sep 17 00:00:00 2001 From: Abigail-2676 Date: Mon, 19 Oct 2026 11:34:42 -0400 Subject: [PATCH 33/51] fix(workspace): grep's note names the workspace's copy, takes a backslash before a literal CR, and has a U+FEFF twin - The CR note said "read with offsetChars shows every CR", but grep also searches a conflict's stored disk version, which read never shows. The note now says read shows every CR in the workspace's copy. - A backslash before a literal CR is an identity escape that a pattern compiled without flags reads as a CR, and namesCr skipped the character after every backslash. It now takes one there too. - The grep description this change had written said a BOM isn't part of the first line, and with main's description back nothing said it, where on main a BOM was part of the first line. A pattern that names U+FEFF (a literal one, \uFEFF in either case, or a backslash before a literal one) over a file that starts with a BOM now gets "The pattern names U+FEFF, and a file searched starts with a BOM. Lines are matched as read shows them, and a BOM isn't part of the first line. read with offsetChars shows it in the workspace's copy." Where both apply, the two notes are joined. namesCr becomes namedLeftOut, which reports both, and leftOutNote builds the note. - namedLeftOut's doc names the spellings it takes, and the ones it doesn't (\15 in a pattern of fewer than 15 groups, a class range that spans U+000D), where "names a CR" read as complete. The changelog lists the spellings too. Tests: \u000D and a backslash before a literal CR join the patterns that get the note; three files in one directory, an LF file on each side of a CRLF one, give the note whichever order the walk takes; a glob that passes over the CRLF file gives none; a conflict whose disk version alone has CRLF ends gives the note for \r$; and a new test pins the U+FEFF note, its spellings, an escaped backslash, a U+FEFF inside a line with no note, and neither note crossing to the other's file. Co-Authored-By: Claude Opus 5.5 --- changelog.d/feat-workspace-read.changed.md | 2 +- src/modules/workspace/index.ts | 48 +++++++++++++++------- test/workspace-file-tools.test.ts | 37 ++++++++++++++++- test/workspace-three-way.test.ts | 5 +++ 4 files changed, 75 insertions(+), 17 deletions(-) diff --git a/changelog.d/feat-workspace-read.changed.md b/changelog.d/feat-workspace-read.changed.md index 69523c593..635bb844d 100644 --- a/changelog.d/feat-workspace-read.changed.md +++ b/changelog.d/feat-workspace-read.changed.md @@ -1,5 +1,5 @@ - The workspace's `read` shows a file's text as it is, and says so. A line page shows a CRLF file's lines without the CR before each line feed, and leaves out a BOM. A line whose own text ends in a CR shows its line end's CR too, since `edit` reads a CR before a line feed as part of the line end, so text copied from the line still matches it. Its note says when the file has CRLF ends or a BOM, counting a mix of CRLF and LF ends and the lines that show both CRs. It also says when the file isn't valid UTF-8: each invalid byte sequence shows as U+FFFD, `edit` refuses the file, and `write` replaces it whole. Before, each line of a CRLF file ended in a CR, a BOM came first in the first line as U+FEFF, and invalid bytes showed as U+FFFD with no note. A character page still reads the stored text as it is, BOM and CRs included. - On a page for a call that carries a `resultMaxChars`, such as a model's or a running script's, a line over 2,000 characters is cut, with a marker that gives the `offsetChars` and `limitChars` that read its rest. Before, a line page showed every line whole, however long. - A `read` page stops before its result would pass `resultMaxChars`, the most of it the framework stores whole. For a model's call that's its agent's inline cap, 24,000 characters by default. Its note says where to continue: the `offset` after its last line, or for a character page the `offsetChars` it also gives as `nextOffsetChars`, with the `limitChars` left of what it was asked for, so a reader that follows the notes ends where it asked to. Where even a page's first line would pass the limit, that line is cut shorter, at the most the page has room for. Where a page's last line is cut, its note names the `offsetChars` where that line's rest starts and the `limitChars` that reads it to the line's end, before the next line's `offset` if the file goes on. A page always moves on, so where its own fields (the path and its notes) and the least it can show pass a small limit, it passes the limit. At 1,000 characters, a page with no notes does that only for a path of about 680 characters or more. Its notes take room from that: with all three notes on a file's text and a conflict's, a path of 90 to 130 characters does it, and a note that names a symlink's target, as one saying disk couldn't be checked can, takes as much room as the target is long. Before, a page of up to 2,000 lines, or of `limitChars` characters, went out whole, and for a result that passed the limit the framework kept only its start, cut mid-JSON, writing the whole of it to a workspace file where the agent has one. A caller that pages 2,000 lines at a time and starts the next page at line 2,001 now starts where the note says. A running `code_execution` script's call has a limit of 5,000,000 characters, so its line pages still stop at the line limit, 2,000 by default, and its character pages at `limitChars`, unless a page would pass that. Its line pages cut a line over 2,000 characters and leave out CRs and a BOM, as a model's do, and a character page gives it the stored text exactly. A call made in process, through `handleToolCall`, `executeToolCall` or `ModuleContext.callTool`, carries no `resultMaxChars` unless its caller sets one. Its line pages stop at the line limit alone and show every line whole, as they did before, though now without the CRs and the BOM. -- The workspace's `grep` leaves out a CRLF end's CR, for matching as well as display, as `read` does, so `foo$` finds a CRLF file's line too, as it does under ripgrep's `--crlf`. A match's text and context are its lines as `read` shows them, but each line whole, where a `read` page for a call with a result limit cuts a line over 2,000 characters. A BOM isn't part of the first line, and a file with CRLF ends says so in its result's `lineEndings` (`CRLF`, or `mixed`). Before, a CRLF file's lines ended in a CR, so `foo$` missed them and a match's text copied into an edit carried the CR. A pattern written to find the CR of a CRLF line end no longer matches, except on a line whose own text ends in a CR, which keeps its line end's CR as `read` shows it. When a pattern names a CR and a file it searched has CRLF ends, the result's `note` says so, and that a character `read` shows every CR. A CR that doesn't come before a line feed is still part of its line. +- The workspace's `grep` leaves out a CRLF end's CR, for matching as well as display, as `read` does, so `foo$` finds a CRLF file's line too, as it does under ripgrep's `--crlf`. A match's text and context are its lines as `read` shows them, but each line whole, where a `read` page for a call with a result limit cuts a line over 2,000 characters. A BOM isn't part of the first line, and a file with CRLF ends says so in its result's `lineEndings` (`CRLF`, or `mixed`). Before, a CRLF file's lines ended in a CR, so `foo$` missed them and a match's text copied into an edit carried the CR. A pattern written to find the CR of a CRLF line end no longer matches, except on a line whose own text ends in a CR, which keeps its line end's CR as `read` shows it. When a pattern names a CR, as a literal one or as `\r`, `\x0d`, `\u000d`, `\cM` or `\015`, and a file it searched has CRLF ends, the result's `note` says so, and that a character `read` shows every CR in the workspace's copy. A pattern that names U+FEFF, over a file that starts with a BOM, gets a note that a BOM isn't part of the first line. A CR that doesn't come before a line feed is still part of its line. - The workspace's `read` first takes in a change on disk to a file the workspace holds, as a listing and an `edit` do, and shows that text. Its note says so when disk's copy had changed since the workspace last agreed with it. It also says when disk's copy couldn't be checked, or disk has none and nothing records whether it was deleted there, so it shows the workspace's copy; where the check gave up because the selected branch kept changing, it says to read again. A file disk deleted answers that disk deleted it and the deletion was taken in, and a file disk made binary or too large answers that it's on disk but not stored. Where the path's stored copy is a draft and disk changed or deleted it too, the read shows the draft, and its note names the conflict and the ways to settle it, a sync, or a forced materialize on a mount that isn't read-only: for a deletion, a sync of the path takes it and removes the workspace's copy. Before, a read of a path the workspace held didn't look at disk, so it could show a copy another writer had since changed; a path it didn't hold was observed, as now. diff --git a/src/modules/workspace/index.ts b/src/modules/workspace/index.ts index efb1ba04b..1d2e2552a 100644 --- a/src/modules/workspace/index.ts +++ b/src/modules/workspace/index.ts @@ -3744,10 +3744,11 @@ export class WorkspaceModule implements Module { // `foo$` finds a CRLF file's line, as it does under ripgrep's --crlf, and // the line copies into an edit as read's do. ripgrep's --crlf still prints // the line with its CR. - let searchedCrlf = false; + let searchedCrlf = false, searchedBom = false; const search = (bytes: Buffer): Searched => { - const { lines, crlfEnds, lfEnds } = fileText(bytes); + const { lines, crlfEnds, lfEnds, bom } = fileText(bytes); if (crlfEnds > 0) searchedCrlf = true; + if (bom) searchedBom = true; const found: GrepMatch[] = []; for (let i = 0; i < lines.length; i++) { if (regex.test(lines[i])) { @@ -3888,9 +3889,10 @@ export class WorkspaceModule implements Module { pattern: input.pattern, results, totalMatches: results.reduce((sum, r) => sum + r.matches.length, 0), - // A pattern written for a CR can't find a CRLF end's, which lines as - // read shows them leave out, so the answer says why it didn't. - ...(searchedCrlf && namesCr(input.pattern) ? { note: CRLF_CR_NOTE } : {}), + // A pattern written for a CR can't find a CRLF end's, nor one written + // for U+FEFF a BOM, which lines as read shows them leave out, so the + // answer says why it didn't. + ...leftOutNote(input.pattern, searchedCrlf, searchedBom), ...(skipped.length > 0 ? { skipped } : {}), ...(incomplete.length > 0 ? { incomplete } : {}), }, @@ -4768,22 +4770,40 @@ function inputProblem( } const CRLF_CR_NOTE = 'The pattern names a CR, and a file searched has CRLF line ends. Lines are matched as read shows them, ' - + 'which leaves out a CRLF end\'s CR unless the line\'s own text ends in a CR. read with offsetChars shows every CR.'; + + 'which leaves out a CRLF end\'s CR unless the line\'s own text ends in a CR. read with offsetChars shows every CR in the workspace\'s copy.'; +const BOM_NOTE = 'The pattern names U+FEFF, and a file searched starts with a BOM. Lines are matched as read shows them, ' + + 'and a BOM isn\'t part of the first line. read with offsetChars shows it in the workspace\'s copy.'; + +/** grep's note for a pattern that names a character its lines leave out, where a file it searched has one. */ +function leftOutNote(pattern: string, searchedCrlf: boolean, searchedBom: boolean): { note?: string } { + if (!searchedCrlf && !searchedBom) return {}; + const named = namedLeftOut(pattern); + const notes = [named.cr && searchedCrlf ? CRLF_CR_NOTE : null, named.bom && searchedBom ? BOM_NOTE : null].filter((n) => n !== null); + return notes.length ? { note: notes.join(' ') } : {}; +} /** - * Whether a grep pattern names a CR: a literal one, or an escape that a - * pattern compiled without flags reads as one (`\r`, `\x0d`, `\u000d`, `\cM`, - * `\015`) whose backslash isn't itself escaped. + * Which of the characters grep's lines leave out a pattern names: a CR, as a + * literal one or as `\r`, `\x0d`, `\u000d`, `\cM`, `\015` or a backslash + * before a literal CR; and U+FEFF, as a literal one, `\uFEFF` or a backslash + * before a literal one. An escape counts only where its backslash isn't itself + * escaped. A pattern compiled without flags can match a CR other ways too, + * such as `\15` in a pattern of fewer than 15 groups or a class range that + * spans U+000D, and those go unnoticed. */ -function namesCr(pattern: string): boolean { +function namedLeftOut(pattern: string): { cr: boolean; bom: boolean } { + const named = { cr: false, bom: false }; for (let i = 0; i < pattern.length; i++) { - if (pattern[i] === '\r') return true; - if (pattern[i] === '\\') { - if (/^(?:r|x0[dD]|u000[dD]|c[mM]|015)/.test(pattern.slice(i + 1, i + 6))) return true; + if (pattern[i] === '\r') named.cr = true; + else if (pattern[i] === '\uFEFF') named.bom = true; + else if (pattern[i] === '\\') { + const next = pattern.slice(i + 1, i + 6); + if (/^(?:r|\r|x0[dD]|u000[dD]|c[mM]|015)/.test(next)) named.cr = true; + else if (/^(?:u[fF][eE][fF][fF]|\uFEFF)/.test(next)) named.bom = true; i++; } } - return false; + return named; } /** diff --git a/test/workspace-file-tools.test.ts b/test/workspace-file-tools.test.ts index f97133880..d1cb6bea4 100644 --- a/test/workspace-file-tools.test.ts +++ b/test/workspace-file-tools.test.ts @@ -414,8 +414,8 @@ test('grep matches and shows a CRLF file\'s lines without the CR, as read shows // The cost: a pattern written to find the CR at a line's end no longer // does, and the answer says why. const note = "The pattern names a CR, and a file searched has CRLF line ends. Lines are matched as read shows them, " - + "which leaves out a CRLF end's CR unless the line's own text ends in a CR. read with offsetChars shows every CR."; - for (const pattern of ['\\r', '\r', 'o[\\r]?$', '\\\\\\r', '\\x0d', '\\x0D', '\\u000d', '\\cM', '\\cm', '\\015', '[\\x0d]']) { + + "which leaves out a CRLF end's CR unless the line's own text ends in a CR. read with offsetChars shows every CR in the workspace's copy."; + for (const pattern of ['\\r', '\r', 'o[\\r]?$', '\\\\\\r', '\\x0d', '\\x0D', '\\u000d', '\\u000D', '\\cM', '\\cm', '\\015', '[\\x0d]', '\\\r']) { const cr = await call('grep', { pattern, path: 'work/crlf.txt' }); const data = cr.data as { results: Array<{ matches: unknown[] }>; note?: string }; assert.equal(data.note, note, JSON.stringify(pattern)); @@ -431,6 +431,39 @@ test('grep matches and shows a CRLF file\'s lines without the CR, as read shows const lone = (await call('grep', { pattern: '\\r', path: 'work/lone.txt' })).data as { results: Array<{ matches: unknown[] }> }; assert.equal('note' in lone, false); assert.deepEqual(lone.results[0].matches, [{ line: 1, text: 'a\rb' }]); + // Any CRLF file searched gives the note, whatever comes after it: here an LF + // file is searched on either side of the CRLF one, whichever order the walk + // takes. A file the glob passes over isn't searched, so it gives none. + mkdirSync(join(dir, 'many')); + writeFileSync(join(dir, 'many', 'a.txt'), 'x\n'); + writeFileSync(join(dir, 'many', 'b.txt'), 'x\r\n'); + writeFileSync(join(dir, 'many', 'c.txt'), 'x\n'); + assert.equal(((await call('grep', { pattern: '\\r', path: 'work/many' })).data as { note?: string }).note, note); + assert.equal('note' in ((await call('grep', { pattern: '\\r', path: 'work/many', glob: 'many/a.txt' })).data as object), false); +}); + +test("grep says a BOM isn't part of the first line when a pattern for U+FEFF searched a file that starts with one", async (t) => { + const { dir, call } = setup(t); + writeFileSync(join(dir, 'bom.txt'), Buffer.concat([Buffer.from([0xef, 0xbb, 0xbf]), Buffer.from('foo\n')])); + writeFileSync(join(dir, 'inner.txt'), 'a\uFEFFb\n'); + const note = "The pattern names U+FEFF, and a file searched starts with a BOM. Lines are matched as read shows them, " + + "and a BOM isn't part of the first line. read with offsetChars shows it in the workspace's copy."; + for (const pattern of ['\\uFEFF', '^\\ufeff', '\uFEFF', '\\\uFEFF']) { + const data = (await call('grep', { pattern, path: 'work/bom.txt' })).data as { results: unknown[]; note?: string }; + assert.equal(data.note, note, JSON.stringify(pattern)); + assert.deepEqual(data.results, [], JSON.stringify(pattern)); + } + // An escaped backslash names no U+FEFF, and a U+FEFF inside a line, in a + // file with no BOM, is found with no note. + assert.equal('note' in ((await call('grep', { pattern: '\\\\uFEFF', path: 'work/bom.txt' })).data as object), false); + const inner = (await call('grep', { pattern: '\\uFEFF', path: 'work/inner.txt' })).data as { results: Array<{ matches: unknown[] }> }; + assert.equal('note' in inner, false); + assert.deepEqual(inner.results[0].matches, [{ line: 1, text: 'a\uFEFFb' }]); + // A CR pattern over the BOM file, which has no CRLF end, and a U+FEFF + // pattern over a CRLF file with no BOM, get no note. + writeFileSync(join(dir, 'crlf.txt'), 'x\r\n'); + assert.equal('note' in ((await call('grep', { pattern: '\\r', path: 'work/bom.txt' })).data as object), false); + assert.equal('note' in ((await call('grep', { pattern: '\\uFEFF', path: 'work/crlf.txt' })).data as object), false); }); test('text copied from a read or a grep of a CRLF file matches in an edit, and the file keeps its line ends and BOM', async (t) => { diff --git a/test/workspace-three-way.test.ts b/test/workspace-three-way.test.ts index 3e74aee2f..66fb08388 100644 --- a/test/workspace-three-way.test.ts +++ b/test/workspace-three-way.test.ts @@ -6723,6 +6723,11 @@ describe('grep', () => { const data = await call(m, 'grep', { pattern: 'ALPHA$' }); assert.deepEqual(data.results.map((r: { version: string; lineEndings?: string; matches: Array<{ text: string }> }) => [r.version, r.lineEndings, r.matches[0]!.text]), [['conflicting-disk', 'CRLF', 'disk says ALPHA']]); + // A CR pattern searched only the disk version's CRLF ends, and says why it + // finds no CR there. + const cr = await call(m, 'grep', { pattern: '\\r$' }); + assert.deepEqual(cr.results, []); + assert.match(cr.note, /^The pattern names a CR, and a file searched has CRLF line ends\./); }); test("once disk changes again, a conflict's stored disk version is labelled as recorded, and what disk holds now as not searched", async (t) => { From 8db582022899a8f5c19fe61c4553598f41529567 Mon Sep 17 00:00:00 2001 From: Abigail-2676 Date: Mon, 19 Oct 2026 11:35:18 -0400 Subject: [PATCH 34/51] docs(workspace): the utils comment says a module is trusted, and four texts say what they pin - The comment on executeToolCallFrom's utils branch said its callers "reach utilities the same way the model does", but a module's call carries trustedModule, which skips the agent's allowedTools for list, describe and run. It now says the puppet and a host reach what the agent's allowedTools permits, as the model does, and a module is trusted and reaches them all. - linePage's doc and the long-line test's title said a cut line comes with "the offsetChars that reads the rest"; both now name the limitChars the marker gives too. - The gone-root test's title said the read says "not to read again", as if its note said so; it says the note doesn't say to read again. - The changelog gave the conflict-note page figure as 90 to 130 characters of path where the measurement was 93 to 129, and now says 93 to 129. Co-Authored-By: Claude Opus 5.5 --- changelog.d/feat-workspace-read.changed.md | 2 +- src/framework.ts | 7 ++++--- src/modules/workspace/read-page.ts | 20 ++++++++++---------- test/workspace-file-tools.test.ts | 2 +- test/workspace-read-text.test.ts | 2 +- 5 files changed, 17 insertions(+), 16 deletions(-) diff --git a/changelog.d/feat-workspace-read.changed.md b/changelog.d/feat-workspace-read.changed.md index 635bb844d..990e4f533 100644 --- a/changelog.d/feat-workspace-read.changed.md +++ b/changelog.d/feat-workspace-read.changed.md @@ -1,5 +1,5 @@ - The workspace's `read` shows a file's text as it is, and says so. A line page shows a CRLF file's lines without the CR before each line feed, and leaves out a BOM. A line whose own text ends in a CR shows its line end's CR too, since `edit` reads a CR before a line feed as part of the line end, so text copied from the line still matches it. Its note says when the file has CRLF ends or a BOM, counting a mix of CRLF and LF ends and the lines that show both CRs. It also says when the file isn't valid UTF-8: each invalid byte sequence shows as U+FFFD, `edit` refuses the file, and `write` replaces it whole. Before, each line of a CRLF file ended in a CR, a BOM came first in the first line as U+FEFF, and invalid bytes showed as U+FFFD with no note. A character page still reads the stored text as it is, BOM and CRs included. - On a page for a call that carries a `resultMaxChars`, such as a model's or a running script's, a line over 2,000 characters is cut, with a marker that gives the `offsetChars` and `limitChars` that read its rest. Before, a line page showed every line whole, however long. -- A `read` page stops before its result would pass `resultMaxChars`, the most of it the framework stores whole. For a model's call that's its agent's inline cap, 24,000 characters by default. Its note says where to continue: the `offset` after its last line, or for a character page the `offsetChars` it also gives as `nextOffsetChars`, with the `limitChars` left of what it was asked for, so a reader that follows the notes ends where it asked to. Where even a page's first line would pass the limit, that line is cut shorter, at the most the page has room for. Where a page's last line is cut, its note names the `offsetChars` where that line's rest starts and the `limitChars` that reads it to the line's end, before the next line's `offset` if the file goes on. A page always moves on, so where its own fields (the path and its notes) and the least it can show pass a small limit, it passes the limit. At 1,000 characters, a page with no notes does that only for a path of about 680 characters or more. Its notes take room from that: with all three notes on a file's text and a conflict's, a path of 90 to 130 characters does it, and a note that names a symlink's target, as one saying disk couldn't be checked can, takes as much room as the target is long. Before, a page of up to 2,000 lines, or of `limitChars` characters, went out whole, and for a result that passed the limit the framework kept only its start, cut mid-JSON, writing the whole of it to a workspace file where the agent has one. A caller that pages 2,000 lines at a time and starts the next page at line 2,001 now starts where the note says. A running `code_execution` script's call has a limit of 5,000,000 characters, so its line pages still stop at the line limit, 2,000 by default, and its character pages at `limitChars`, unless a page would pass that. Its line pages cut a line over 2,000 characters and leave out CRs and a BOM, as a model's do, and a character page gives it the stored text exactly. A call made in process, through `handleToolCall`, `executeToolCall` or `ModuleContext.callTool`, carries no `resultMaxChars` unless its caller sets one. Its line pages stop at the line limit alone and show every line whole, as they did before, though now without the CRs and the BOM. +- A `read` page stops before its result would pass `resultMaxChars`, the most of it the framework stores whole. For a model's call that's its agent's inline cap, 24,000 characters by default. Its note says where to continue: the `offset` after its last line, or for a character page the `offsetChars` it also gives as `nextOffsetChars`, with the `limitChars` left of what it was asked for, so a reader that follows the notes ends where it asked to. Where even a page's first line would pass the limit, that line is cut shorter, at the most the page has room for. Where a page's last line is cut, its note names the `offsetChars` where that line's rest starts and the `limitChars` that reads it to the line's end, before the next line's `offset` if the file goes on. A page always moves on, so where its own fields (the path and its notes) and the least it can show pass a small limit, it passes the limit. At 1,000 characters, a page with no notes does that only for a path of about 680 characters or more. Its notes take room from that: with all three notes on a file's text and a conflict's, a path of 93 to 129 characters does it, and a note that names a symlink's target, as one saying disk couldn't be checked can, takes as much room as the target is long. Before, a page of up to 2,000 lines, or of `limitChars` characters, went out whole, and for a result that passed the limit the framework kept only its start, cut mid-JSON, writing the whole of it to a workspace file where the agent has one. A caller that pages 2,000 lines at a time and starts the next page at line 2,001 now starts where the note says. A running `code_execution` script's call has a limit of 5,000,000 characters, so its line pages still stop at the line limit, 2,000 by default, and its character pages at `limitChars`, unless a page would pass that. Its line pages cut a line over 2,000 characters and leave out CRs and a BOM, as a model's do, and a character page gives it the stored text exactly. A call made in process, through `handleToolCall`, `executeToolCall` or `ModuleContext.callTool`, carries no `resultMaxChars` unless its caller sets one. Its line pages stop at the line limit alone and show every line whole, as they did before, though now without the CRs and the BOM. - The workspace's `grep` leaves out a CRLF end's CR, for matching as well as display, as `read` does, so `foo$` finds a CRLF file's line too, as it does under ripgrep's `--crlf`. A match's text and context are its lines as `read` shows them, but each line whole, where a `read` page for a call with a result limit cuts a line over 2,000 characters. A BOM isn't part of the first line, and a file with CRLF ends says so in its result's `lineEndings` (`CRLF`, or `mixed`). Before, a CRLF file's lines ended in a CR, so `foo$` missed them and a match's text copied into an edit carried the CR. A pattern written to find the CR of a CRLF line end no longer matches, except on a line whose own text ends in a CR, which keeps its line end's CR as `read` shows it. When a pattern names a CR, as a literal one or as `\r`, `\x0d`, `\u000d`, `\cM` or `\015`, and a file it searched has CRLF ends, the result's `note` says so, and that a character `read` shows every CR in the workspace's copy. A pattern that names U+FEFF, over a file that starts with a BOM, gets a note that a BOM isn't part of the first line. A CR that doesn't come before a line feed is still part of its line. - The workspace's `read` first takes in a change on disk to a file the workspace holds, as a listing and an `edit` do, and shows that text. Its note says so when disk's copy had changed since the workspace last agreed with it. It also says when disk's copy couldn't be checked, or disk has none and nothing records whether it was deleted there, so it shows the workspace's copy; where the check gave up because the selected branch kept changing, it says to read again. A file disk deleted answers that disk deleted it and the deletion was taken in, and a file disk made binary or too large answers that it's on disk but not stored. Where the path's stored copy is a draft and disk changed or deleted it too, the read shows the draft, and its note names the conflict and the ways to settle it, a sync, or a forced materialize on a mount that isn't read-only: for a deletion, a sync of the path takes it and removes the workspace's copy. Before, a read of a path the workspace held didn't look at disk, so it could show a copy another writer had since changed; a path it didn't hold was observed, as now. diff --git a/src/framework.ts b/src/framework.ts index ded5f8977..505b7e1ea 100644 --- a/src/framework.ts +++ b/src/framework.ts @@ -18293,9 +18293,10 @@ export class AgentFramework { return this.draftsToolResult(origin.agentName, call.input); } - // The utils meta-tool: callers that await a tool's result (the puppet and - // a host through executeToolCall, and a module through callTool) reach - // utilities the same way the model does. + // The utils meta-tool, for callers that await a tool's result. The puppet + // and a host, through executeToolCall, reach the utilities the agent's + // allowedTools permits, as the model does; a module, through callTool, is + // trusted and reaches them all. if (call.name === 'utils') { return this.handleUtilsToolCall(call.callerAgentName ?? '__ephemeral__', call, { inProcess: true, trustedModule: origin.kind === 'module' }); } diff --git a/src/modules/workspace/read-page.ts b/src/modules/workspace/read-page.ts index cabd29ffe..9a3373bd0 100644 --- a/src/modules/workspace/read-page.ts +++ b/src/modules/workspace/read-page.ts @@ -37,16 +37,16 @@ const joined = (notes: Array): string | undefined => { /** * Lines from `offset` (1-based), at most `limit` of them, each numbered and a - * long one cut with the offsetChars that reads the rest. The page is the - * longest that fits within `maxChars` as JSON; where none fits with its first - * line as a page shows it, that line is cut shorter, at the most the page has - * room for. Where the page's last line is cut, its note names where that - * line's rest starts and how long it is, since reading on in order starts - * there: before the next line, or, on the page that ends the file, before the - * file is read whole. A page whose own fields and least content pass - * `maxChars` still moves on, and passes it. With no `maxChars`, for a call - * made in process, which takes the data itself, the page holds to `limit` - * alone and shows every line whole. + * long one cut with the offsetChars and limitChars that read the rest. The + * page is the longest that fits within `maxChars` as JSON; where none fits + * with its first line as a page shows it, that line is cut shorter, at the + * most the page has room for. Where the page's last line is cut, its note + * names where that line's rest starts and how long it is, since reading on in + * order starts there: before the next line, or, on the page that ends the + * file, before the file is read whole. A page whose own fields and least + * content pass `maxChars` still moves on, and passes it. With no `maxChars`, + * for a call made in process, which takes the data itself, the page holds to + * `limit` alone and shows every line whole. */ export function linePage(path: string, file: FileText, offset: number, limit: number, maxChars: number | undefined, extraNotes: string[] = []): ToolResult { const { lines, starts } = file; diff --git a/test/workspace-file-tools.test.ts b/test/workspace-file-tools.test.ts index d1cb6bea4..aef86e31a 100644 --- a/test/workspace-file-tools.test.ts +++ b/test/workspace-file-tools.test.ts @@ -701,7 +701,7 @@ test("a read whose disk copy can't be checked says it shows the workspace's copy }); }); -test("a read of a held file whose mount's root is gone says disk couldn't be checked, and not to read again", async (t) => { +test("a read of a held file whose mount's root is gone says disk couldn't be checked, and doesn't say to read again", async (t) => { // The pass reports the path as unverified and leaves the root's region // incomplete. It didn't give up on a branch that kept changing, so reading // again won't check it. diff --git a/test/workspace-read-text.test.ts b/test/workspace-read-text.test.ts index 1c3b4a1bd..38b033c2f 100644 --- a/test/workspace-read-text.test.ts +++ b/test/workspace-read-text.test.ts @@ -84,7 +84,7 @@ test('a BOM is noted and not shown, and bytes that aren\'t UTF-8 are noted', asy assert.equal(((await readLatin1({ offsetChars: 0 })).data as CharPage).note, notUtf8); }); -test('a long line is cut at 2000 characters, with the offsetChars that reads the rest', async t => { +test('a long line is cut at 2000 characters, with the offsetChars and limitChars that read the rest', async t => { // A BOM and a CRLF before the long line, so its offset counts both. const long = 'x'.repeat(1999) + '😀' + 'y'.repeat(1000); const { read } = await setup(t, Buffer.concat([Buffer.from([0xef, 0xbb, 0xbf]), Buffer.from(`short\r\n${long}\r\nlast\r\n`)])); From 7d02af8d794ffceccae2244a0b4a281bb8100f9f Mon Sep 17 00:00:00 2001 From: Abigail-2676 Date: Mon, 19 Oct 2026 12:48:50 -0400 Subject: [PATCH 35/51] fix(workspace): grep's U+FEFF note names only a BOM the workspace's copy has, and grep's notes are pinned where they weren't grep searches a conflict's stored disk version too, and read never shows it. Where only that version starts with a BOM, the U+FEFF note's "read with offsetChars shows it in the workspace's copy" named a BOM the workspace's copy doesn't have. It now says "read with offsetChars shows any BOM the workspace's copy has", as the CR note says every CR in the workspace's copy. A three-way test greps such a conflict for ^\uFEFF and gets the note, and reads the workspace's copy without one. The other tests pin what changing the code could break unseen: - a pattern that names both a CR and U+FEFF, over a file with a BOM and CRLF ends, gets both notes, joined by a space with the CR's first; - the BOM note comes from a BOM file searched with a file without one on each side of it, as the CR note's latch is tested; - the glob case finds the file its glob takes, with no note, where it showed only that no note came; - \X0D, \U000D, \CM and \UFEFF, identity escapes in a pattern compiled without flags, and an escaped backslash before cM or u000d, name neither character. Co-Authored-By: Claude Opus 5.5 --- src/modules/workspace/index.ts | 2 +- test/workspace-file-tools.test.ts | 35 ++++++++++++++++++++++++------- test/workspace-three-way.test.ts | 15 +++++++++++++ 3 files changed, 44 insertions(+), 8 deletions(-) diff --git a/src/modules/workspace/index.ts b/src/modules/workspace/index.ts index 1d2e2552a..7ffe75ea7 100644 --- a/src/modules/workspace/index.ts +++ b/src/modules/workspace/index.ts @@ -4772,7 +4772,7 @@ function inputProblem( const CRLF_CR_NOTE = 'The pattern names a CR, and a file searched has CRLF line ends. Lines are matched as read shows them, ' + 'which leaves out a CRLF end\'s CR unless the line\'s own text ends in a CR. read with offsetChars shows every CR in the workspace\'s copy.'; const BOM_NOTE = 'The pattern names U+FEFF, and a file searched starts with a BOM. Lines are matched as read shows them, ' - + 'and a BOM isn\'t part of the first line. read with offsetChars shows it in the workspace\'s copy.'; + + 'and a BOM isn\'t part of the first line. read with offsetChars shows any BOM the workspace\'s copy has.'; /** grep's note for a pattern that names a character its lines leave out, where a file it searched has one. */ function leftOutNote(pattern: string, searchedCrlf: boolean, searchedBom: boolean): { note?: string } { diff --git a/test/workspace-file-tools.test.ts b/test/workspace-file-tools.test.ts index aef86e31a..b4c0725e1 100644 --- a/test/workspace-file-tools.test.ts +++ b/test/workspace-file-tools.test.ts @@ -423,7 +423,8 @@ test('grep matches and shows a CRLF file\'s lines without the CR, as read shows } // A pattern that names no CR, and a CR pattern over files without a CRLF // end, get no note. - for (const [pattern, path] of [['foo', 'work/crlf.txt'], ['\\\\r', 'work/crlf.txt'], ['\\\\x0d', 'work/crlf.txt'], ['\\R', 'work/crlf.txt'], ['\\r', 'work/lf.txt']]) { + for (const [pattern, path] of [['foo', 'work/crlf.txt'], ['\\\\r', 'work/crlf.txt'], ['\\\\x0d', 'work/crlf.txt'], ['\\\\cM', 'work/crlf.txt'], + ['\\\\u000d', 'work/crlf.txt'], ['\\R', 'work/crlf.txt'], ['\\X0D', 'work/crlf.txt'], ['\\U000D', 'work/crlf.txt'], ['\\CM', 'work/crlf.txt'], ['\\r', 'work/lf.txt']]) { const data = (await call('grep', { pattern, path })).data as object; assert.equal('note' in data, false, JSON.stringify([pattern, path])); } @@ -433,13 +434,16 @@ test('grep matches and shows a CRLF file\'s lines without the CR, as read shows assert.deepEqual(lone.results[0].matches, [{ line: 1, text: 'a\rb' }]); // Any CRLF file searched gives the note, whatever comes after it: here an LF // file is searched on either side of the CRLF one, whichever order the walk - // takes. A file the glob passes over isn't searched, so it gives none. + // takes. A file the glob passes over isn't searched, so it gives none: here + // the glob takes a.txt alone, which the search finds. mkdirSync(join(dir, 'many')); writeFileSync(join(dir, 'many', 'a.txt'), 'x\n'); writeFileSync(join(dir, 'many', 'b.txt'), 'x\r\n'); writeFileSync(join(dir, 'many', 'c.txt'), 'x\n'); assert.equal(((await call('grep', { pattern: '\\r', path: 'work/many' })).data as { note?: string }).note, note); - assert.equal('note' in ((await call('grep', { pattern: '\\r', path: 'work/many', glob: 'many/a.txt' })).data as object), false); + const globbed = (await call('grep', { pattern: 'x|\\r', path: 'work/many', glob: 'many/a.txt' })).data as { results: Array<{ file: string }> }; + assert.deepEqual(globbed.results.map((r) => r.file), ['work/many/a.txt']); + assert.equal('note' in globbed, false); }); test("grep says a BOM isn't part of the first line when a pattern for U+FEFF searched a file that starts with one", async (t) => { @@ -447,15 +451,17 @@ test("grep says a BOM isn't part of the first line when a pattern for U+FEFF sea writeFileSync(join(dir, 'bom.txt'), Buffer.concat([Buffer.from([0xef, 0xbb, 0xbf]), Buffer.from('foo\n')])); writeFileSync(join(dir, 'inner.txt'), 'a\uFEFFb\n'); const note = "The pattern names U+FEFF, and a file searched starts with a BOM. Lines are matched as read shows them, " - + "and a BOM isn't part of the first line. read with offsetChars shows it in the workspace's copy."; + + "and a BOM isn't part of the first line. read with offsetChars shows any BOM the workspace's copy has."; for (const pattern of ['\\uFEFF', '^\\ufeff', '\uFEFF', '\\\uFEFF']) { const data = (await call('grep', { pattern, path: 'work/bom.txt' })).data as { results: unknown[]; note?: string }; assert.equal(data.note, note, JSON.stringify(pattern)); assert.deepEqual(data.results, [], JSON.stringify(pattern)); } - // An escaped backslash names no U+FEFF, and a U+FEFF inside a line, in a - // file with no BOM, is found with no note. - assert.equal('note' in ((await call('grep', { pattern: '\\\\uFEFF', path: 'work/bom.txt' })).data as object), false); + // An escaped backslash names no U+FEFF, nor does \U, an identity escape, + // and a U+FEFF inside a line, in a file with no BOM, is found with no note. + for (const pattern of ['\\\\uFEFF', '\\UFEFF']) { + assert.equal('note' in ((await call('grep', { pattern, path: 'work/bom.txt' })).data as object), false, JSON.stringify(pattern)); + } const inner = (await call('grep', { pattern: '\\uFEFF', path: 'work/inner.txt' })).data as { results: Array<{ matches: unknown[] }> }; assert.equal('note' in inner, false); assert.deepEqual(inner.results[0].matches, [{ line: 1, text: 'a\uFEFFb' }]); @@ -464,6 +470,21 @@ test("grep says a BOM isn't part of the first line when a pattern for U+FEFF sea writeFileSync(join(dir, 'crlf.txt'), 'x\r\n'); assert.equal('note' in ((await call('grep', { pattern: '\\r', path: 'work/bom.txt' })).data as object), false); assert.equal('note' in ((await call('grep', { pattern: '\\uFEFF', path: 'work/crlf.txt' })).data as object), false); + // Any BOM file searched gives the note, whatever comes after it: a file + // without one is searched on either side of the BOM file. + mkdirSync(join(dir, 'boms')); + writeFileSync(join(dir, 'boms', 'a.txt'), 'x\n'); + writeFileSync(join(dir, 'boms', 'b.txt'), Buffer.concat([Buffer.from([0xef, 0xbb, 0xbf]), Buffer.from('x\n')])); + writeFileSync(join(dir, 'boms', 'c.txt'), 'x\n'); + assert.equal(((await call('grep', { pattern: '\\uFEFF', path: 'work/boms' })).data as { note?: string }).note, note); + // A pattern that names both, over a file with a BOM and CRLF ends, gets both + // notes, the CR's first. + writeFileSync(join(dir, 'both.txt'), Buffer.concat([Buffer.from([0xef, 0xbb, 0xbf]), Buffer.from('x\r\n')])); + const crNote = "The pattern names a CR, and a file searched has CRLF line ends. Lines are matched as read shows them, " + + "which leaves out a CRLF end's CR unless the line's own text ends in a CR. read with offsetChars shows every CR in the workspace's copy."; + const both = (await call('grep', { pattern: '^\\uFEFF|\\r$', path: 'work/both.txt' })).data as { results: unknown[]; note?: string }; + assert.equal(both.note, `${crNote} ${note}`); + assert.deepEqual(both.results, []); }); test('text copied from a read or a grep of a CRLF file matches in an edit, and the file keeps its line ends and BOM', async (t) => { diff --git a/test/workspace-three-way.test.ts b/test/workspace-three-way.test.ts index 66fb08388..98356a022 100644 --- a/test/workspace-three-way.test.ts +++ b/test/workspace-three-way.test.ts @@ -6730,6 +6730,21 @@ describe('grep', () => { assert.match(cr.note, /^The pattern names a CR, and a file searched has CRLF line ends\./); }); + test("a U+FEFF pattern over a conflict whose disk version alone starts with a BOM gets the note, which names only the workspace's copy as what read shows", async (t) => { + const env = new Env(t); + const m = await env.open(); + await seedSynced(env, m, 'a.txt', 'v1'); + env.writeDisk('a.txt', '\uFEFFdisk says ALPHA\n'); + await call(m, 'write', { path: 'work/a.txt', content: 'workspace says GAMMA\n' }); + assert.equal((await entryOf(m, 'a.txt'))?.conflict?.diskCopy, 'stored'); + const data = await call(m, 'grep', { pattern: '^\\uFEFF' }); + assert.deepEqual(data.results, []); + assert.equal(data.note, "The pattern names U+FEFF, and a file searched starts with a BOM. Lines are matched as read shows them, " + + "and a BOM isn't part of the first line. read with offsetChars shows any BOM the workspace's copy has."); + // The workspace's copy, which read shows, has none. + assert.equal((await call(m, 'read', { path: 'work/a.txt', offsetChars: 0 })).content, 'workspace says GAMMA\n'); + }); + test("once disk changes again, a conflict's stored disk version is labelled as recorded, and what disk holds now as not searched", async (t) => { const env = new Env(t); const m = await env.open(); From ac6f05d8f5c23486d09d068c94363035adf45ef3 Mon Sep 17 00:00:00 2001 From: Abigail-2676 Date: Mon, 19 Oct 2026 12:48:50 -0400 Subject: [PATCH 36/51] docs(framework): the utils comment says an unidentified caller is unrestricted The comment said the puppet and a host, through executeToolCall, reach the utilities the agent's allowedTools permits. A call naming no agent runs as "__ephemeral__", which canAgentUseTool passes for every utility unless that name has been an ephemeral's, so such a caller reaches them all, as executeToolCall's doc says of unidentified callers. The comment now says a call through executeToolCall reaches what its agent's allowedTools permits, as the model's does, and an unidentified caller is unrestricted. Co-Authored-By: Claude Opus 5.5 --- src/framework.ts | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/src/framework.ts b/src/framework.ts index 505b7e1ea..49a36fb3f 100644 --- a/src/framework.ts +++ b/src/framework.ts @@ -18293,10 +18293,11 @@ export class AgentFramework { return this.draftsToolResult(origin.agentName, call.input); } - // The utils meta-tool, for callers that await a tool's result. The puppet - // and a host, through executeToolCall, reach the utilities the agent's - // allowedTools permits, as the model does; a module, through callTool, is - // trusted and reaches them all. + // The utils meta-tool, for callers that await a tool's result. A call + // through executeToolCall (the puppet, a host) reaches what its agent's + // allowedTools permits, as the model's does, and an unidentified caller + // is unrestricted, as executeToolCall's doc says; a module, through + // callTool, is trusted and reaches every utility. if (call.name === 'utils') { return this.handleUtilsToolCall(call.callerAgentName ?? '__ephemeral__', call, { inProcess: true, trustedModule: origin.kind === 'module' }); } From e62ea559633eca644f9082f793975a6797bc5045 Mon Sep 17 00:00:00 2001 From: Abigail-2676 Date: Mon, 19 Oct 2026 22:06:21 -0400 Subject: [PATCH 37/51] test(workspace): a direct mount's read and grep give a 'chronicle' mount's answers Anarchid's nightly review R3-1 (#465, at ac6f05d8f) asked for the CRLF, BOM, budget, long-line and grep tests on a direct mount too. Two tests on a direct mount, each of which compares its answers with a 'chronicle' mount's over the same files: - read: a CRLF file, one that starts with a BOM, one that isn't valid UTF-8, one with mixed ends and a line whose own text ends in a CR, a 60,000-character line and a 3,000-line file, each read as a page of lines and of characters at a result limit of 24,000, of 1,000, and with none. Beside the comparison, it checks the CRLF and BOM pages and their notes, the cut line's marker, and that the long line's page at 24,000 and the 3,000-line file's pages at both limits stay within them. - grep: ^foo$ over CRLF, mixed, LF and BOM files, with each file's lineEndings, and the notes on patterns that name a CR, U+FEFF or both. With direct mounts left on main's reading and every other line of the merge as it is, these two fail, and every other test in the five workspace test files this change touches passes. Co-Authored-By: Claude Opus 5.5 --- test/workspace-direct.test.ts | 88 +++++++++++++++++++++++++++++++++++ 1 file changed, 88 insertions(+) diff --git a/test/workspace-direct.test.ts b/test/workspace-direct.test.ts index 9c5e25f5a..4b91a28fd 100644 --- a/test/workspace-direct.test.ts +++ b/test/workspace-direct.test.ts @@ -2315,6 +2315,94 @@ test("a direct mount's listings pass over another mount's directory, naming the assert.match(await failure(m, 'read', { path: 'ref/inner/x.ts' }), /lies in mount "inner", nested in mount "ref"/); }); +test("a direct mount's read shows a file's text as a 'chronicle' mount's does: lines without a CRLF end's CR, no BOM, a long line cut, and each page held to the call's result limit, with the notes that say so", async (t) => { + const env = new Env(t); + env.file('crlf.txt', 'one\r\ntwo\r\n'); + env.file('bom.txt', Buffer.concat([Buffer.from([0xef, 0xbb, 0xbf]), Buffer.from('first\n')])); + env.file('bad.txt', Buffer.from([0x61, 0xff, 0x62, 0x0a])); + env.file('mixed.txt', 'a\r\nb\nc\r\r\nd\n'); + env.file('long.txt', `${'x'.repeat(60000)}\ny\n`); + env.file('big.txt', Array.from({ length: 3000 }, (_, i) => `line ${i + 1} of a file with fairly ordinary lines`).join('\n')); + // Each file read as a page of lines and as a page of characters, at a + // model's limit, at a small one, and with none, as an in-process call makes it. + const reads: Array<[Record, number | undefined]> = []; + for (const path of ['crlf.txt', 'bom.txt', 'bad.txt', 'mixed.txt', 'long.txt', 'big.txt']) { + for (const cap of [24000, 1000, undefined]) { + reads.push([{ path: `ref/${path}` }, cap], [{ path: `ref/${path}`, offsetChars: 0, limitChars: 5000 }, cap]); + } + } + const answers = async (m: WorkspaceModule): Promise, number | undefined, Record]>> => { + const out: Array<[Record, number | undefined, Record]> = []; + for (const [input, cap] of reads) { + const r = await m.handleToolCall({ id: 'read', name: 'read', input, ...(cap !== undefined ? { resultMaxChars: cap } : {}) }); + out.push([input, cap, r.success ? r.data as Record : { error: r.error }]); + } + return out; + }; + const direct = await env.open(); + const fromDisk = await answers(direct); + const page = (path: string, cap: number | undefined, chars = false) => + fromDisk.find(([input, c]) => input.path === `ref/${path}` && c === cap && ('offsetChars' in input) === chars)![2]; + assert.equal(page('crlf.txt', 24000).content, ' 1\tone\n 2\ttwo'); + assert.match(page('crlf.txt', 24000).note, /^Line ends are CRLF, shown without the CR\./); + assert.equal(page('bom.txt', 24000).content, ' 1\tfirst'); + assert.match(page('bom.txt', 24000).note, /BOM/); + assert.match(page('bad.txt', 24000).note, /isn't valid UTF-8/); + assert.equal(page('mixed.txt', 24000).content, ' 1\ta\n 2\tb\n 3\tc\r\r\n 4\td'); + const long = page('long.txt', 24000); + assert.ok(JSON.stringify(long).length <= 24000, `${JSON.stringify(long).length}`); + assert.equal(long.content, ` 1\t${'x'.repeat(2000)} [… read the rest with offsetChars=2000 and limitChars=58000]\n 2\ty`); + for (const cap of [24000, 1000]) { + for (const chars of [false, true]) assert.ok(JSON.stringify(page('big.txt', cap, chars)).length <= cap, `${cap} ${chars}`); + } + assert.ok(page('big.txt', 24000).toLine < 3000); + assert.equal(page('long.txt', undefined).content.length, ' 1\t'.length + 60000 + '\n 2\ty'.length); + await env.close(direct); + // The same answers as a 'chronicle' mount's, whose first read of a file + // takes it in with nothing more to say of it. + assert.deepEqual(fromDisk, await answers(await env.openChronicle())); +}); + +test("a direct mount's grep matches lines as read shows them, as a 'chronicle' mount's grep does, with each file's lineEndings and the notes on a pattern for a CR or U+FEFF", async (t) => { + const env = new Env(t); + env.file('crlf.txt', 'alpha\r\nfoo\r\nomega\r\n'); + env.file('mixed.txt', 'foo\r\nbar\n'); + env.file('lf.txt', 'foo\n'); + env.file('bom.txt', Buffer.concat([Buffer.from([0xef, 0xbb, 0xbf]), Buffer.from('foo\n')])); + env.file('both.txt', Buffer.concat([Buffer.from([0xef, 0xbb, 0xbf]), Buffer.from('x\r\n')])); + const calls: Array> = [ + { pattern: '^foo$' }, + { pattern: '\\r$', path: 'ref/crlf.txt' }, + { pattern: '^\\uFEFF', path: 'ref/bom.txt' }, + { pattern: '^\\uFEFF|\\r$', path: 'ref/both.txt' }, + { pattern: '\\r', path: 'ref/lf.txt' }, + ]; + // A 'chronicle' mount labels each result with its path's state and the + // workspace's copy, a direct mount's with disk's file: the rest is the same. + const answers = async (m: WorkspaceModule): Promise, Array>, string | null]>> => { + const out: Array<[Record, Array>, string | null]> = []; + for (const input of calls) { + const d = await data<{ results: Array>; note?: string }>(m, 'grep', input); + out.push([input, d.results.map(({ state: _state, version: _version, ...rest }) => rest), d.note ?? null]); + } + return out; + }; + const direct = await env.open(); + const fromDisk = await answers(direct); + const byFile = Object.fromEntries(fromDisk[0]![1].map((r) => [r.file, r])); + assert.deepEqual(byFile['ref/crlf.txt'], { file: 'ref/crlf.txt', lineEndings: 'CRLF', matches: [{ line: 2, text: 'foo' }] }); + assert.equal(byFile['ref/mixed.txt']!.lineEndings, 'mixed'); + assert.deepEqual(byFile['ref/bom.txt'], { file: 'ref/bom.txt', matches: [{ line: 1, text: 'foo' }] }); + assert.deepEqual(byFile['ref/lf.txt'], { file: 'ref/lf.txt', matches: [{ line: 1, text: 'foo' }] }); + const cr = "The pattern names a CR, and a file searched has CRLF line ends. Lines are matched as read shows them, " + + "which leaves out a CRLF end's CR unless the line's own text ends in a CR. read with offsetChars shows every CR in the workspace's copy."; + const bom = "The pattern names U+FEFF, and a file searched starts with a BOM. Lines are matched as read shows them, " + + "and a BOM isn't part of the first line. read with offsetChars shows any BOM the workspace's copy has."; + assert.deepEqual(fromDisk.slice(1).map(([, results, note]) => [results.length, note]), [[0, cr], [0, bom], [0, `${cr} ${bom}`], [0, null]]); + await env.close(direct); + assert.deepEqual(fromDisk, await answers(await env.openChronicle())); +}); + test('inOrder hands each result on in order, at most `width` at once', async () => { const settle: Array<() => void> = []; const settled: number[] = []; From ac535df13e218511e8b04db7bcbfdbedee942d5d Mon Sep 17 00:00:00 2001 From: Abigail-2676 Date: Mon, 19 Oct 2026 22:07:21 -0400 Subject: [PATCH 38/51] docs(changelog): a direct mount's read and grep show text as a store-backed mount's do Anarchid's nightly review R3-1 (#465, at ac6f05d8f): with direct mounts left on main's reading, the changelog's claims on read's pages and grep's lines would be false for them. The changelog's sentences on read's pages and grep's lines now hold for direct mounts too, and one line says so: the same pages, cut lines, notes and lineEndings, with the disk check a store-backed mount's alone, since a direct mount reads disk as it is and keeps no copy. Direct mounts are new in the same release, so there is no earlier behaviour of theirs to name. Co-Authored-By: Claude Opus 5.5 --- changelog.d/feat-workspace-read.changed.md | 1 + 1 file changed, 1 insertion(+) diff --git a/changelog.d/feat-workspace-read.changed.md b/changelog.d/feat-workspace-read.changed.md index 990e4f533..567ca4364 100644 --- a/changelog.d/feat-workspace-read.changed.md +++ b/changelog.d/feat-workspace-read.changed.md @@ -3,3 +3,4 @@ - A `read` page stops before its result would pass `resultMaxChars`, the most of it the framework stores whole. For a model's call that's its agent's inline cap, 24,000 characters by default. Its note says where to continue: the `offset` after its last line, or for a character page the `offsetChars` it also gives as `nextOffsetChars`, with the `limitChars` left of what it was asked for, so a reader that follows the notes ends where it asked to. Where even a page's first line would pass the limit, that line is cut shorter, at the most the page has room for. Where a page's last line is cut, its note names the `offsetChars` where that line's rest starts and the `limitChars` that reads it to the line's end, before the next line's `offset` if the file goes on. A page always moves on, so where its own fields (the path and its notes) and the least it can show pass a small limit, it passes the limit. At 1,000 characters, a page with no notes does that only for a path of about 680 characters or more. Its notes take room from that: with all three notes on a file's text and a conflict's, a path of 93 to 129 characters does it, and a note that names a symlink's target, as one saying disk couldn't be checked can, takes as much room as the target is long. Before, a page of up to 2,000 lines, or of `limitChars` characters, went out whole, and for a result that passed the limit the framework kept only its start, cut mid-JSON, writing the whole of it to a workspace file where the agent has one. A caller that pages 2,000 lines at a time and starts the next page at line 2,001 now starts where the note says. A running `code_execution` script's call has a limit of 5,000,000 characters, so its line pages still stop at the line limit, 2,000 by default, and its character pages at `limitChars`, unless a page would pass that. Its line pages cut a line over 2,000 characters and leave out CRs and a BOM, as a model's do, and a character page gives it the stored text exactly. A call made in process, through `handleToolCall`, `executeToolCall` or `ModuleContext.callTool`, carries no `resultMaxChars` unless its caller sets one. Its line pages stop at the line limit alone and show every line whole, as they did before, though now without the CRs and the BOM. - The workspace's `grep` leaves out a CRLF end's CR, for matching as well as display, as `read` does, so `foo$` finds a CRLF file's line too, as it does under ripgrep's `--crlf`. A match's text and context are its lines as `read` shows them, but each line whole, where a `read` page for a call with a result limit cuts a line over 2,000 characters. A BOM isn't part of the first line, and a file with CRLF ends says so in its result's `lineEndings` (`CRLF`, or `mixed`). Before, a CRLF file's lines ended in a CR, so `foo$` missed them and a match's text copied into an edit carried the CR. A pattern written to find the CR of a CRLF line end no longer matches, except on a line whose own text ends in a CR, which keeps its line end's CR as `read` shows it. When a pattern names a CR, as a literal one or as `\r`, `\x0d`, `\u000d`, `\cM` or `\015`, and a file it searched has CRLF ends, the result's `note` says so, and that a character `read` shows every CR in the workspace's copy. A pattern that names U+FEFF, over a file that starts with a BOM, gets a note that a BOM isn't part of the first line. A CR that doesn't come before a line feed is still part of its line. - The workspace's `read` first takes in a change on disk to a file the workspace holds, as a listing and an `edit` do, and shows that text. Its note says so when disk's copy had changed since the workspace last agreed with it. It also says when disk's copy couldn't be checked, or disk has none and nothing records whether it was deleted there, so it shows the workspace's copy; where the check gave up because the selected branch kept changing, it says to read again. A file disk deleted answers that disk deleted it and the deletion was taken in, and a file disk made binary or too large answers that it's on disk but not stored. Where the path's stored copy is a draft and disk changed or deleted it too, the read shows the draft, and its note names the conflict and the ways to settle it, a sync, or a forced materialize on a mount that isn't read-only: for a deletion, a sync of the path takes it and removes the workspace's copy. Before, a read of a path the workspace held didn't look at disk, so it could show a copy another writer had since changed; a path it didn't hold was observed, as now. +- A direct mount's `read` and `grep` show a file's text as a store-backed mount's do, with the same pages, cut lines, notes and `lineEndings`. Only the disk check is a store-backed mount's alone, since a direct mount reads disk as it is and keeps no copy of its own. From 2a2b02eadee197627dcdc55713226178a7adf5ca Mon Sep 17 00:00:00 2001 From: Abigail-2676 Date: Mon, 19 Oct 2026 22:51:22 -0400 Subject: [PATCH 39/51] fix(workspace): grep's CR and BOM notes name disk's file on a host with a direct mount grep's note on a pattern that names a CR or U+FEFF ends by saying where a character read shows what lines leave out: "every CR in the workspace's copy", "any BOM the workspace's copy has". The note names the workspace's copy because grep also searches a conflict's disk version, which read doesn't show. Since the merge with main, a direct mount's grep gives the same notes, and a direct mount keeps no workspace copy: its read shows disk's file. On a host with a mount that runs direct, the notes now end "every CR in the workspace's copy, or in disk's file on a direct mount" and "any BOM in the workspace's copy, or in disk's file on a direct mount". A host with none keeps the notes as they were, as the tools' descriptions mention direct mounts only on a host that has one. The direct grep test checks each host's notes, and that the rest of the two hosts' answers is the same. Co-Authored-By: Claude Opus 5.5 --- src/modules/workspace/index.ts | 29 +++++++++++++++++++++-------- test/workspace-direct.test.ts | 18 +++++++++++++----- 2 files changed, 34 insertions(+), 13 deletions(-) diff --git a/src/modules/workspace/index.ts b/src/modules/workspace/index.ts index 1c1a19aad..2ad9dfded 100644 --- a/src/modules/workspace/index.ts +++ b/src/modules/workspace/index.ts @@ -4687,7 +4687,7 @@ export class WorkspaceModule implements Module { // A pattern written for a CR can't find a CRLF end's, nor one written // for U+FEFF a BOM, which lines as read shows them leave out, so the // answer says why it didn't. - ...leftOutNote(input.pattern, searchedCrlf, searchedBom), + ...leftOutNote(input.pattern, searchedCrlf, searchedBom, [...this.mounts.values()].some((m) => m.direct)), ...(skipped.length > 0 ? { skipped } : {}), ...(incomplete.length > 0 ? { incomplete } : {}), }, @@ -5916,16 +5916,29 @@ function inputProblem( return null; } -const CRLF_CR_NOTE = 'The pattern names a CR, and a file searched has CRLF line ends. Lines are matched as read shows them, ' - + 'which leaves out a CRLF end\'s CR unless the line\'s own text ends in a CR. read with offsetChars shows every CR in the workspace\'s copy.'; -const BOM_NOTE = 'The pattern names U+FEFF, and a file searched starts with a BOM. Lines are matched as read shows them, ' - + 'and a BOM isn\'t part of the first line. read with offsetChars shows any BOM the workspace\'s copy has.'; +const CRLF_CR_MATCHED = 'The pattern names a CR, and a file searched has CRLF line ends. Lines are matched as read shows them, ' + + 'which leaves out a CRLF end\'s CR unless the line\'s own text ends in a CR.'; +const BOM_MATCHED = 'The pattern names U+FEFF, and a file searched starts with a BOM. Lines are matched as read shows them, ' + + 'and a BOM isn\'t part of the first line.'; -/** grep's note for a pattern that names a character its lines leave out, where a file it searched has one. */ -function leftOutNote(pattern: string, searchedCrlf: boolean, searchedBom: boolean): { note?: string } { +/** + * grep's note for a pattern that names a character its lines leave out, where + * a file it searched has one. The note says a character read shows it in the + * workspace's copy, since grep also searches a conflict's disk version, which + * read doesn't show. A direct mount keeps no workspace copy, and its read + * shows disk's file, so on a host with one (`direct`) the note names that + * too, as the tools' descriptions mention direct mounts only on such a host. + */ +function leftOutNote(pattern: string, searchedCrlf: boolean, searchedBom: boolean, direct: boolean): { note?: string } { if (!searchedCrlf && !searchedBom) return {}; const named = namedLeftOut(pattern); - const notes = [named.cr && searchedCrlf ? CRLF_CR_NOTE : null, named.bom && searchedBom ? BOM_NOTE : null].filter((n) => n !== null); + const onDisk = direct ? ', or in disk\'s file on a direct mount' : ''; + const notes = [ + named.cr && searchedCrlf ? `${CRLF_CR_MATCHED} read with offsetChars shows every CR in the workspace's copy${onDisk}.` : null, + named.bom && searchedBom + ? `${BOM_MATCHED} read with offsetChars shows any BOM ${direct ? `in the workspace's copy${onDisk}` : 'the workspace\'s copy has'}.` + : null, + ].filter((n) => n !== null); return notes.length ? { note: notes.join(' ') } : {}; } diff --git a/test/workspace-direct.test.ts b/test/workspace-direct.test.ts index 4b91a28fd..af0098955 100644 --- a/test/workspace-direct.test.ts +++ b/test/workspace-direct.test.ts @@ -2378,7 +2378,8 @@ test("a direct mount's grep matches lines as read shows them, as a 'chronicle' m { pattern: '\\r', path: 'ref/lf.txt' }, ]; // A 'chronicle' mount labels each result with its path's state and the - // workspace's copy, a direct mount's with disk's file: the rest is the same. + // workspace's copy, a direct mount's with disk's file: the rest is the same, + // but for where the notes say read shows what lines leave out. const answers = async (m: WorkspaceModule): Promise, Array>, string | null]>> => { const out: Array<[Record, Array>, string | null]> = []; for (const input of calls) { @@ -2395,12 +2396,19 @@ test("a direct mount's grep matches lines as read shows them, as a 'chronicle' m assert.deepEqual(byFile['ref/bom.txt'], { file: 'ref/bom.txt', matches: [{ line: 1, text: 'foo' }] }); assert.deepEqual(byFile['ref/lf.txt'], { file: 'ref/lf.txt', matches: [{ line: 1, text: 'foo' }] }); const cr = "The pattern names a CR, and a file searched has CRLF line ends. Lines are matched as read shows them, " - + "which leaves out a CRLF end's CR unless the line's own text ends in a CR. read with offsetChars shows every CR in the workspace's copy."; + + "which leaves out a CRLF end's CR unless the line's own text ends in a CR. read with offsetChars shows every CR in the workspace's copy"; const bom = "The pattern names U+FEFF, and a file searched starts with a BOM. Lines are matched as read shows them, " - + "and a BOM isn't part of the first line. read with offsetChars shows any BOM the workspace's copy has."; - assert.deepEqual(fromDisk.slice(1).map(([, results, note]) => [results.length, note]), [[0, cr], [0, bom], [0, `${cr} ${bom}`], [0, null]]); + + "and a BOM isn't part of the first line. read with offsetChars shows any BOM"; + // On a host with a direct mount, which keeps no workspace copy, the notes + // name disk's file too; on a host with none, the workspace's copy alone. + const crDirect = `${cr}, or in disk's file on a direct mount.`; + const bomDirect = `${bom} in the workspace's copy, or in disk's file on a direct mount.`; + const notes = (crNote: string, bomNote: string) => [crNote, bomNote, `${crNote} ${bomNote}`, null]; + assert.deepEqual(fromDisk.slice(1).map(([, results, note]) => [results.length, note]), notes(crDirect, bomDirect).map((n) => [0, n])); await env.close(direct); - assert.deepEqual(fromDisk, await answers(await env.openChronicle())); + const fromStore = await answers(await env.openChronicle()); + assert.deepEqual(fromStore.map(([input, results]) => [input, results]), fromDisk.map(([input, results]) => [input, results])); + assert.deepEqual(fromStore.slice(1).map(([, , note]) => note), notes(`${cr}.`, `${bom} the workspace's copy has.`)); }); test('inOrder hands each result on in order, at most `width` at once', async () => { From 2226bf1e595974336513ed1c798d885a8259d8ca Mon Sep 17 00:00:00 2001 From: Abigail-2676 Date: Mon, 19 Oct 2026 22:51:30 -0400 Subject: [PATCH 40/51] test(workspace): the direct read test holds every page with a result limit to it The test's title says each page is held to the call's result limit, and it checked that only for the 60,000-character line's page at 24,000 and the 3,000-line file's pages at both limits. It now checks every page it asks for with a limit: each file's line page and character page, at 24,000 and at 1,000. The comparison that follows holds a 'chronicle' mount's pages to the same answers. Co-Authored-By: Claude Opus 5.5 --- test/workspace-direct.test.ts | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/test/workspace-direct.test.ts b/test/workspace-direct.test.ts index af0098955..0c9565c54 100644 --- a/test/workspace-direct.test.ts +++ b/test/workspace-direct.test.ts @@ -2349,11 +2349,11 @@ test("a direct mount's read shows a file's text as a 'chronicle' mount's does: l assert.match(page('bom.txt', 24000).note, /BOM/); assert.match(page('bad.txt', 24000).note, /isn't valid UTF-8/); assert.equal(page('mixed.txt', 24000).content, ' 1\ta\n 2\tb\n 3\tc\r\r\n 4\td'); - const long = page('long.txt', 24000); - assert.ok(JSON.stringify(long).length <= 24000, `${JSON.stringify(long).length}`); - assert.equal(long.content, ` 1\t${'x'.repeat(2000)} [… read the rest with offsetChars=2000 and limitChars=58000]\n 2\ty`); - for (const cap of [24000, 1000]) { - for (const chars of [false, true]) assert.ok(JSON.stringify(page('big.txt', cap, chars)).length <= cap, `${cap} ${chars}`); + assert.equal(page('long.txt', 24000).content, ` 1\t${'x'.repeat(2000)} [… read the rest with offsetChars=2000 and limitChars=58000]\n 2\ty`); + // Every page a call with a result limit asked for fits it. + for (const [input, cap, d] of fromDisk) { + const length = JSON.stringify(d).length; + if (cap !== undefined) assert.ok(length <= cap, `${String(input.path)}${'offsetChars' in input ? ' in characters' : ''} at ${cap}: ${length}`); } assert.ok(page('big.txt', 24000).toLine < 3000); assert.equal(page('long.txt', undefined).content.length, ' 1\t'.length + 60000 + '\n 2\ty'.length); From 677715d0997fc82d38dc2a24faabe1f13f8cec2f Mon Sep 17 00:00:00 2001 From: Abigail-2676 Date: Mon, 19 Oct 2026 22:51:31 -0400 Subject: [PATCH 41/51] docs(workspace): MountConfig.backend says a read takes disk's change in too The 'chronicle' backend's doc listed listings, searches, syncs and scans as what takes disk's changes in, and said a read of a path the store holds reads the store's copy. Since read settles its path against disk first, under the same three-way rule, the doc names reads among them, and says a read then shows the store's copy. Co-Authored-By: Claude Opus 5.5 --- src/modules/workspace/types.ts | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/src/modules/workspace/types.ts b/src/modules/workspace/types.ts index 82a28c27b..d2595ea03 100644 --- a/src/modules/workspace/types.ts +++ b/src/modules/workspace/types.ts @@ -20,9 +20,8 @@ export interface MountConfig { * Where the mount's tools get what they read (default: 'chronicle'). * - 'chronicle': the workspace keeps the mount's files in its store, where * writes and edits land first, and takes in disk's changes under the - * three-way rule as its listings, searches, syncs and scans reach them. - * A read of a path the store doesn't hold takes in a new file there; - * otherwise it reads the store's copy. + * three-way rule as its reads, listings, searches, syncs and scans reach + * them. A read then shows the store's copy. * - 'direct': for a checkout an agent reads, not one it changes. Every * read, listing and search is disk's own copy as it is when it runs, * and none of its files go into the store. No scan or watcher From 24b507d6b7b368878dd1b22f80e32625bda6d1c4 Mon Sep 17 00:00:00 2001 From: Abigail-2676 Date: Mon, 19 Oct 2026 23:22:27 -0400 Subject: [PATCH 42/51] refactor(workspace): the tools' descriptions and grep's notes ask one question of whether a mount runs direct getTools mentions direct mounts in four descriptions only where one of the host's mounts runs direct, and grep's notes name disk's file on the same hosts. The two had each written the test inline. anyRunsDirect now asks it for both, so a later change to what counts can't leave the descriptions and the notes disagreeing about the host. Co-Authored-By: Claude Opus 5.5 --- src/modules/workspace/index.ts | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/src/modules/workspace/index.ts b/src/modules/workspace/index.ts index 2ad9dfded..4b0f6d811 100644 --- a/src/modules/workspace/index.ts +++ b/src/modules/workspace/index.ts @@ -1694,10 +1694,19 @@ export class WorkspaceModule implements Module { // Tool Definitions // ========================================================================== + /** + * Whether any of this host's mounts runs direct. The tools' descriptions + * and grep's notes mention direct mounts only on such a host, so both ask + * this one question. + */ + private anyRunsDirect(): boolean { + return [...this.mounts.values()].some((m) => m.direct); + } + getTools(): ToolDefinition[] { // Where a mount runs direct, four descriptions say what that changes. // A host with none keeps them as they were, and its residents' caches. - const direct = [...this.mounts.values()].some((m) => m.direct); + const direct = this.anyRunsDirect(); const lsDirect = direct ? ' A direct mount (ls without a path says which, with its ignore list) lists disk\'s files as they are now: each with ' + 'its size and no state, or a note where the mount doesn\'t take it. A path its ignore list covers is listed when you name it.' @@ -4687,7 +4696,7 @@ export class WorkspaceModule implements Module { // A pattern written for a CR can't find a CRLF end's, nor one written // for U+FEFF a BOM, which lines as read shows them leave out, so the // answer says why it didn't. - ...leftOutNote(input.pattern, searchedCrlf, searchedBom, [...this.mounts.values()].some((m) => m.direct)), + ...leftOutNote(input.pattern, searchedCrlf, searchedBom, this.anyRunsDirect()), ...(skipped.length > 0 ? { skipped } : {}), ...(incomplete.length > 0 ? { incomplete } : {}), }, From 766f9fa75d87a4dd96b86590ab7a0cf3b7367ebf Mon Sep 17 00:00:00 2001 From: Abigail-2676 Date: Mon, 19 Oct 2026 23:22:27 -0400 Subject: [PATCH 43/51] docs(workspace): the 'chronicle' backend's doc and the changelog say which reads take disk's change in MountConfig.backend's doc said a 'chronicle' mount takes in disk's changes as its reads reach them, and that a read then shows the store's copy. Of the reads, only the read tool settles its path first. read_image and readBinary still read a held path as the workspace holds it, and a read that takes in disk's deletion shows no copy. The doc now names `read` among what takes disk's changes in, and says the mount's tools read the store's copy. The changelog's sentence on grep's CR note now says the note names disk's file on a direct mount too, where the host has one. Co-Authored-By: Claude Opus 5.5 --- changelog.d/feat-workspace-read.changed.md | 2 +- src/modules/workspace/types.ts | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/changelog.d/feat-workspace-read.changed.md b/changelog.d/feat-workspace-read.changed.md index 567ca4364..71fe0c187 100644 --- a/changelog.d/feat-workspace-read.changed.md +++ b/changelog.d/feat-workspace-read.changed.md @@ -1,6 +1,6 @@ - The workspace's `read` shows a file's text as it is, and says so. A line page shows a CRLF file's lines without the CR before each line feed, and leaves out a BOM. A line whose own text ends in a CR shows its line end's CR too, since `edit` reads a CR before a line feed as part of the line end, so text copied from the line still matches it. Its note says when the file has CRLF ends or a BOM, counting a mix of CRLF and LF ends and the lines that show both CRs. It also says when the file isn't valid UTF-8: each invalid byte sequence shows as U+FFFD, `edit` refuses the file, and `write` replaces it whole. Before, each line of a CRLF file ended in a CR, a BOM came first in the first line as U+FEFF, and invalid bytes showed as U+FFFD with no note. A character page still reads the stored text as it is, BOM and CRs included. - On a page for a call that carries a `resultMaxChars`, such as a model's or a running script's, a line over 2,000 characters is cut, with a marker that gives the `offsetChars` and `limitChars` that read its rest. Before, a line page showed every line whole, however long. - A `read` page stops before its result would pass `resultMaxChars`, the most of it the framework stores whole. For a model's call that's its agent's inline cap, 24,000 characters by default. Its note says where to continue: the `offset` after its last line, or for a character page the `offsetChars` it also gives as `nextOffsetChars`, with the `limitChars` left of what it was asked for, so a reader that follows the notes ends where it asked to. Where even a page's first line would pass the limit, that line is cut shorter, at the most the page has room for. Where a page's last line is cut, its note names the `offsetChars` where that line's rest starts and the `limitChars` that reads it to the line's end, before the next line's `offset` if the file goes on. A page always moves on, so where its own fields (the path and its notes) and the least it can show pass a small limit, it passes the limit. At 1,000 characters, a page with no notes does that only for a path of about 680 characters or more. Its notes take room from that: with all three notes on a file's text and a conflict's, a path of 93 to 129 characters does it, and a note that names a symlink's target, as one saying disk couldn't be checked can, takes as much room as the target is long. Before, a page of up to 2,000 lines, or of `limitChars` characters, went out whole, and for a result that passed the limit the framework kept only its start, cut mid-JSON, writing the whole of it to a workspace file where the agent has one. A caller that pages 2,000 lines at a time and starts the next page at line 2,001 now starts where the note says. A running `code_execution` script's call has a limit of 5,000,000 characters, so its line pages still stop at the line limit, 2,000 by default, and its character pages at `limitChars`, unless a page would pass that. Its line pages cut a line over 2,000 characters and leave out CRs and a BOM, as a model's do, and a character page gives it the stored text exactly. A call made in process, through `handleToolCall`, `executeToolCall` or `ModuleContext.callTool`, carries no `resultMaxChars` unless its caller sets one. Its line pages stop at the line limit alone and show every line whole, as they did before, though now without the CRs and the BOM. -- The workspace's `grep` leaves out a CRLF end's CR, for matching as well as display, as `read` does, so `foo$` finds a CRLF file's line too, as it does under ripgrep's `--crlf`. A match's text and context are its lines as `read` shows them, but each line whole, where a `read` page for a call with a result limit cuts a line over 2,000 characters. A BOM isn't part of the first line, and a file with CRLF ends says so in its result's `lineEndings` (`CRLF`, or `mixed`). Before, a CRLF file's lines ended in a CR, so `foo$` missed them and a match's text copied into an edit carried the CR. A pattern written to find the CR of a CRLF line end no longer matches, except on a line whose own text ends in a CR, which keeps its line end's CR as `read` shows it. When a pattern names a CR, as a literal one or as `\r`, `\x0d`, `\u000d`, `\cM` or `\015`, and a file it searched has CRLF ends, the result's `note` says so, and that a character `read` shows every CR in the workspace's copy. A pattern that names U+FEFF, over a file that starts with a BOM, gets a note that a BOM isn't part of the first line. A CR that doesn't come before a line feed is still part of its line. +- The workspace's `grep` leaves out a CRLF end's CR, for matching as well as display, as `read` does, so `foo$` finds a CRLF file's line too, as it does under ripgrep's `--crlf`. A match's text and context are its lines as `read` shows them, but each line whole, where a `read` page for a call with a result limit cuts a line over 2,000 characters. A BOM isn't part of the first line, and a file with CRLF ends says so in its result's `lineEndings` (`CRLF`, or `mixed`). Before, a CRLF file's lines ended in a CR, so `foo$` missed them and a match's text copied into an edit carried the CR. A pattern written to find the CR of a CRLF line end no longer matches, except on a line whose own text ends in a CR, which keeps its line end's CR as `read` shows it. When a pattern names a CR, as a literal one or as `\r`, `\x0d`, `\u000d`, `\cM` or `\015`, and a file it searched has CRLF ends, the result's `note` says so, and that a character `read` shows every CR in the workspace's copy, or in disk's file on a direct mount where the host has one. A pattern that names U+FEFF, over a file that starts with a BOM, gets a note that a BOM isn't part of the first line. A CR that doesn't come before a line feed is still part of its line. - The workspace's `read` first takes in a change on disk to a file the workspace holds, as a listing and an `edit` do, and shows that text. Its note says so when disk's copy had changed since the workspace last agreed with it. It also says when disk's copy couldn't be checked, or disk has none and nothing records whether it was deleted there, so it shows the workspace's copy; where the check gave up because the selected branch kept changing, it says to read again. A file disk deleted answers that disk deleted it and the deletion was taken in, and a file disk made binary or too large answers that it's on disk but not stored. Where the path's stored copy is a draft and disk changed or deleted it too, the read shows the draft, and its note names the conflict and the ways to settle it, a sync, or a forced materialize on a mount that isn't read-only: for a deletion, a sync of the path takes it and removes the workspace's copy. Before, a read of a path the workspace held didn't look at disk, so it could show a copy another writer had since changed; a path it didn't hold was observed, as now. - A direct mount's `read` and `grep` show a file's text as a store-backed mount's do, with the same pages, cut lines, notes and `lineEndings`. Only the disk check is a store-backed mount's alone, since a direct mount reads disk as it is and keeps no copy of its own. diff --git a/src/modules/workspace/types.ts b/src/modules/workspace/types.ts index d2595ea03..503755fc0 100644 --- a/src/modules/workspace/types.ts +++ b/src/modules/workspace/types.ts @@ -20,8 +20,8 @@ export interface MountConfig { * Where the mount's tools get what they read (default: 'chronicle'). * - 'chronicle': the workspace keeps the mount's files in its store, where * writes and edits land first, and takes in disk's changes under the - * three-way rule as its reads, listings, searches, syncs and scans reach - * them. A read then shows the store's copy. + * three-way rule as `read`, listings, searches, syncs and scans reach + * them. Its tools read the store's copy. * - 'direct': for a checkout an agent reads, not one it changes. Every * read, listing and search is disk's own copy as it is when it runs, * and none of its files go into the store. No scan or watcher From a40712ef39704c284dd28862b02ff954ed591372 Mon Sep 17 00:00:00 2001 From: Abigail-2676 Date: Mon, 19 Oct 2026 23:22:27 -0400 Subject: [PATCH 44/51] test(workspace): grep's notes on a host with both kinds of mount, and each capped direct read is a page The notes name disk's file wherever any of the host's mounts runs direct, and the only grep test of them used a host with a direct mount alone, where a choice by the mounts a call searched would give the same notes. A new test greps a 'chronicle' mount beside a direct one, for a CR and for U+FEFF, and expects the notes to name disk's file too. The direct read test's check that each page with a result limit fits it also checks that each is a page, since an error's text would fit any limit. Co-Authored-By: Claude Opus 5.5 --- test/workspace-direct.test.ts | 25 ++++++++++++++++++++++--- 1 file changed, 22 insertions(+), 3 deletions(-) diff --git a/test/workspace-direct.test.ts b/test/workspace-direct.test.ts index 0c9565c54..ea4268100 100644 --- a/test/workspace-direct.test.ts +++ b/test/workspace-direct.test.ts @@ -2350,10 +2350,11 @@ test("a direct mount's read shows a file's text as a 'chronicle' mount's does: l assert.match(page('bad.txt', 24000).note, /isn't valid UTF-8/); assert.equal(page('mixed.txt', 24000).content, ' 1\ta\n 2\tb\n 3\tc\r\r\n 4\td'); assert.equal(page('long.txt', 24000).content, ` 1\t${'x'.repeat(2000)} [… read the rest with offsetChars=2000 and limitChars=58000]\n 2\ty`); - // Every page a call with a result limit asked for fits it. + // Every page a call with a result limit asked for is a page, and fits it. for (const [input, cap, d] of fromDisk) { - const length = JSON.stringify(d).length; - if (cap !== undefined) assert.ok(length <= cap, `${String(input.path)}${'offsetChars' in input ? ' in characters' : ''} at ${cap}: ${length}`); + const label = `${String(input.path)}${'offsetChars' in input ? ' in characters' : ''} at ${cap}`; + assert.equal(typeof d.content, 'string', `${label}: ${JSON.stringify(d).slice(0, 200)}`); + if (cap !== undefined) assert.ok(JSON.stringify(d).length <= cap, `${label}: ${JSON.stringify(d).length}`); } assert.ok(page('big.txt', 24000).toLine < 3000); assert.equal(page('long.txt', undefined).content.length, ' 1\t'.length + 60000 + '\n 2\ty'.length); @@ -2411,6 +2412,24 @@ test("a direct mount's grep matches lines as read shows them, as a 'chronicle' m assert.deepEqual(fromStore.slice(1).map(([, , note]) => note), notes(`${cr}.`, `${bom} the workspace's copy has.`)); }); +test("on a host with a direct mount, grep's notes name disk's file too where the search was of a 'chronicle' mount beside it", async (t) => { + const env = new Env(t); + const work = join(env.root, 'work'); + mkdirSync(work); + writeFileSync(join(work, 'crlf.txt'), 'alpha\r\nomega\r\n'); + writeFileSync(join(work, 'bom.txt'), Buffer.concat([Buffer.from([0xef, 0xbb, 0xbf]), Buffer.from('foo\n')])); + // The tools' descriptions mention direct mounts on this host, and so do + // the notes, whichever of its mounts a grep searched. + const m = await env.open({}, [{ name: 'work', path: work, mode: 'read-write', watch: 'never' }]); + const onDisk = ", or in disk's file on a direct mount."; + const cr = await data<{ results: unknown[]; note?: string }>(m, 'grep', { pattern: '\\r$', path: 'work/crlf.txt' }); + assert.deepEqual(cr.results, []); + assert.ok(cr.note?.endsWith(`read with offsetChars shows every CR in the workspace's copy${onDisk}`), cr.note); + const bom = await data<{ results: unknown[]; note?: string }>(m, 'grep', { pattern: '^\\uFEFF', path: 'work/bom.txt' }); + assert.deepEqual(bom.results, []); + assert.ok(bom.note?.endsWith(`read with offsetChars shows any BOM in the workspace's copy${onDisk}`), bom.note); +}); + test('inOrder hands each result on in order, at most `width` at once', async () => { const settle: Array<() => void> = []; const settled: number[] = []; From 5f89cc82bb06fffc14715dca980c1fa4a5281299 Mon Sep 17 00:00:00 2001 From: Abigail-2676 Date: Wed, 21 Oct 2026 11:12:11 -0400 Subject: [PATCH 45/51] fix(workspace): grep's note on a pattern for a CR or U+FEFF names the files that have CRLF ends or a BOM Anarchid's nightly review R4-1 (#465, at a40712ef3): grep matches lines as read shows them, without a CRLF end's CR or a BOM, so a grep for \r$ or ^\uFEFF, main's way of finding which files have CRLF ends or a BOM, found nothing. Its note said that a file searched had them and named none, and lineEndings comes only with a match, so no workspace tool answered it short of reading each file. Whether a file has CRLF ends or a BOM is a fact about the file, so grep now keeps each searched file that has either by its path, and the note on a pattern that names a CR or U+FEFF says how many files searched have it and names them: "3 files searched have CRLF line ends: work/both.json (CRLF), work/mixed.cfg (mixed), work/win.bat (CRLF)". Past 20 files it names the first 20, in the order grep searched them, then "and N more; narrow with path or glob", so the count stays exact. A conflict's disk version is named as that version, since read doesn't show it, and in each note only where the workspace's copy of its path doesn't have that note's character. The results, the tools' descriptions and a search that names no such character are as they were. Co-Authored-By: Claude Opus 5.5 --- src/modules/workspace/index.ts | 85 ++++++++++++++++++++++--------- test/workspace-direct.test.ts | 16 +++--- test/workspace-file-tools.test.ts | 47 ++++++++++++++--- test/workspace-three-way.test.ts | 18 +++++-- 4 files changed, 127 insertions(+), 39 deletions(-) diff --git a/src/modules/workspace/index.ts b/src/modules/workspace/index.ts index 4b0f6d811..ef087b0cd 100644 --- a/src/modules/workspace/index.ts +++ b/src/modules/workspace/index.ts @@ -4641,11 +4641,20 @@ export class WorkspaceModule implements Module { // `foo$` finds a CRLF file's line, as it does under ripgrep's --crlf, and // the line copies into an edit as read's do. ripgrep's --crlf still prints // the line with its CR. - let searchedCrlf = false, searchedBom = false; - const search = (bytes: Buffer): Searched => { - const { lines, crlfEnds, lfEnds, bom } = fileText(bytes); - if (crlfEnds > 0) searchedCrlf = true; - if (bom) searchedBom = true; + // + // Whether a file has CRLF ends or a BOM is a fact about the file, not a + // line, so each searched file that has either is kept here by its path, + // for the note on a pattern that names a CR or U+FEFF to name it. A + // conflict's disk version is kept only where the workspace's copy of its + // path has neither, and is named as that version, since read doesn't show + // it. + const crlfFiles = new Map(); + const bomFiles = new Map(); + const search = (text: GrepText): Searched => { + const { lines, crlfEnds, lfEnds, bom } = fileText(text.bytes); + const lineEndings = crlfEnds > 0 ? (lfEnds > 0 ? 'mixed' : 'CRLF') : undefined; + if (lineEndings !== undefined && !crlfFiles.has(text.file)) crlfFiles.set(text.file, { ends: lineEndings, version: text.version }); + if (bom && !bomFiles.has(text.file)) bomFiles.set(text.file, text.version); const found: GrepMatch[] = []; for (let i = 0; i < lines.length; i++) { if (regex.test(lines[i])) { @@ -4658,7 +4667,7 @@ export class WorkspaceModule implements Module { found.push(match); } } - return { ...(crlfEnds > 0 ? { lineEndings: lfEnds > 0 ? 'mixed' : 'CRLF' } : {}), matches: found }; + return { ...(lineEndings !== undefined ? { lineEndings } : {}), matches: found }; }; for (const { mount, relativePath, moved, root } of mountsToSearch) { @@ -4676,7 +4685,7 @@ export class WorkspaceModule implements Module { if (deeper === null) continue; const start = await this.walkStart(mount, relativePath, deeper === relativePath ? '' : deeper.slice(relativePath ? relativePath.length + 1 : 0)); const searched = (text: GrepText): void => { - const hits = search(text.bytes); + const hits = search(text); if (hits.matches.length > 0) results.push({ file: text.file, ...(text.state !== undefined ? { state: text.state } : {}), version: text.version, ...hits }); }; const kept = (path: string): boolean => fileGlob === null || fileGlob.test(path); @@ -4695,8 +4704,8 @@ export class WorkspaceModule implements Module { totalMatches: results.reduce((sum, r) => sum + r.matches.length, 0), // A pattern written for a CR can't find a CRLF end's, nor one written // for U+FEFF a BOM, which lines as read shows them leave out, so the - // answer says why it didn't. - ...leftOutNote(input.pattern, searchedCrlf, searchedBom, this.anyRunsDirect()), + // answer says why it didn't, and names the files that have them. + ...leftOutNote(input.pattern, crlfFiles, bomFiles, this.anyRunsDirect()), ...(skipped.length > 0 ? { skipped } : {}), ...(incomplete.length > 0 ? { incomplete } : {}), }, @@ -5925,27 +5934,57 @@ function inputProblem( return null; } -const CRLF_CR_MATCHED = 'The pattern names a CR, and a file searched has CRLF line ends. Lines are matched as read shows them, ' - + 'which leaves out a CRLF end\'s CR unless the line\'s own text ends in a CR.'; -const BOM_MATCHED = 'The pattern names U+FEFF, and a file searched starts with a BOM. Lines are matched as read shows them, ' - + 'and a BOM isn\'t part of the first line.'; +/** How many files a grep note names before it gives the count of the rest. */ +const NOTE_FILES_MAX = 20; + +/** + * The files a grep note names, in the order grep searched them, as its + * results are: at most NOTE_FILES_MAX of them, then how many more and how to + * see them. Each is named by its mount-prefixed path, with what `detail` says + * of it. + */ +function namedFiles(files: Map, detail: (value: T) => string): string { + const names = [...files.keys()]; + const shown = names.slice(0, NOTE_FILES_MAX).map((name) => `${name}${detail(files.get(name)!)}`); + const rest = names.length - shown.length; + return `${shown.join(', ')}${rest > 0 ? `, and ${rest} more; narrow with path or glob` : ''}`; +} + +/** What a note says of a conflict's disk version, which read doesn't show. */ +const ofVersion = (version: GrepText['version']): string => + version === 'conflicting-disk' || version === 'recorded-disk' ? ` in its ${version} version` : ''; /** * grep's note for a pattern that names a character its lines leave out, where - * a file it searched has one. The note says a character read shows it in the - * workspace's copy, since grep also searches a conflict's disk version, which - * read doesn't show. A direct mount keeps no workspace copy, and its read - * shows disk's file, so on a host with one (`direct`) the note names that - * too, as the tools' descriptions mention direct mounts only on such a host. + * a file it searched has one: it says how many files searched have it and + * names them, each with whether its ends are CRLF or mixed, so a search for a + * CR or a BOM still finds the files that have them. The note says a + * character read shows it in the workspace's copy, since grep also searches a + * conflict's disk version, which read doesn't show. A direct mount keeps no + * workspace copy, and its read shows disk's file, so on a host with one + * (`direct`) the note names that too, as the tools' descriptions mention + * direct mounts only on such a host. */ -function leftOutNote(pattern: string, searchedCrlf: boolean, searchedBom: boolean, direct: boolean): { note?: string } { - if (!searchedCrlf && !searchedBom) return {}; +function leftOutNote( + pattern: string, + crlfFiles: Map, + bomFiles: Map, + direct: boolean, +): { note?: string } { + if (crlfFiles.size === 0 && bomFiles.size === 0) return {}; const named = namedLeftOut(pattern); const onDisk = direct ? ', or in disk\'s file on a direct mount' : ''; + const count = (n: number, one: string, many: string) => `${n} ${n === 1 ? `file searched ${one}` : `files searched ${many}`}`; const notes = [ - named.cr && searchedCrlf ? `${CRLF_CR_MATCHED} read with offsetChars shows every CR in the workspace's copy${onDisk}.` : null, - named.bom && searchedBom - ? `${BOM_MATCHED} read with offsetChars shows any BOM ${direct ? `in the workspace's copy${onDisk}` : 'the workspace\'s copy has'}.` + named.cr && crlfFiles.size > 0 + ? `The pattern names a CR, and ${count(crlfFiles.size, 'has', 'have')} CRLF line ends: ` + + `${namedFiles(crlfFiles, ({ ends, version }) => ` (${ends}${ofVersion(version)})`)}. Lines are matched as read shows them, ` + + `which leaves out a CRLF end's CR unless the line's own text ends in a CR. read with offsetChars shows every CR in the workspace's copy${onDisk}.` + : null, + named.bom && bomFiles.size > 0 + ? `The pattern names U+FEFF, and ${count(bomFiles.size, 'starts', 'start')} with a BOM: ` + + `${namedFiles(bomFiles, (version) => (ofVersion(version) ? ` (${ofVersion(version).slice(1)})` : ''))}. Lines are matched as read shows them, ` + + `and a BOM isn't part of the first line. read with offsetChars shows any BOM ${direct ? `in the workspace's copy${onDisk}` : 'the workspace\'s copy has'}.` : null, ].filter((n) => n !== null); return notes.length ? { note: notes.join(' ') } : {}; diff --git a/test/workspace-direct.test.ts b/test/workspace-direct.test.ts index ea4268100..80925d1bf 100644 --- a/test/workspace-direct.test.ts +++ b/test/workspace-direct.test.ts @@ -2396,20 +2396,22 @@ test("a direct mount's grep matches lines as read shows them, as a 'chronicle' m assert.equal(byFile['ref/mixed.txt']!.lineEndings, 'mixed'); assert.deepEqual(byFile['ref/bom.txt'], { file: 'ref/bom.txt', matches: [{ line: 1, text: 'foo' }] }); assert.deepEqual(byFile['ref/lf.txt'], { file: 'ref/lf.txt', matches: [{ line: 1, text: 'foo' }] }); - const cr = "The pattern names a CR, and a file searched has CRLF line ends. Lines are matched as read shows them, " + // Each note names the files searched that have what it's about. + const cr = (file: string) => `The pattern names a CR, and 1 file searched has CRLF line ends: ${file} (CRLF). Lines are matched as read shows them, ` + "which leaves out a CRLF end's CR unless the line's own text ends in a CR. read with offsetChars shows every CR in the workspace's copy"; - const bom = "The pattern names U+FEFF, and a file searched starts with a BOM. Lines are matched as read shows them, " + const bom = (file: string) => `The pattern names U+FEFF, and 1 file searched starts with a BOM: ${file}. Lines are matched as read shows them, ` + "and a BOM isn't part of the first line. read with offsetChars shows any BOM"; // On a host with a direct mount, which keeps no workspace copy, the notes // name disk's file too; on a host with none, the workspace's copy alone. - const crDirect = `${cr}, or in disk's file on a direct mount.`; - const bomDirect = `${bom} in the workspace's copy, or in disk's file on a direct mount.`; - const notes = (crNote: string, bomNote: string) => [crNote, bomNote, `${crNote} ${bomNote}`, null]; - assert.deepEqual(fromDisk.slice(1).map(([, results, note]) => [results.length, note]), notes(crDirect, bomDirect).map((n) => [0, n])); + const onDisk = ", or in disk's file on a direct mount."; + const notes = (crEnd: string, bomEnd: string) => [ + `${cr('ref/crlf.txt')}${crEnd}`, `${bom('ref/bom.txt')}${bomEnd}`, `${cr('ref/both.txt')}${crEnd} ${bom('ref/both.txt')}${bomEnd}`, null]; + assert.deepEqual(fromDisk.slice(1).map(([, results, note]) => [results.length, note]), + notes(onDisk, ` in the workspace's copy${onDisk}`).map((n) => [0, n])); await env.close(direct); const fromStore = await answers(await env.openChronicle()); assert.deepEqual(fromStore.map(([input, results]) => [input, results]), fromDisk.map(([input, results]) => [input, results])); - assert.deepEqual(fromStore.slice(1).map(([, , note]) => note), notes(`${cr}.`, `${bom} the workspace's copy has.`)); + assert.deepEqual(fromStore.slice(1).map(([, , note]) => note), notes('.', " the workspace's copy has.")); }); test("on a host with a direct mount, grep's notes name disk's file too where the search was of a 'chronicle' mount beside it", async (t) => { diff --git a/test/workspace-file-tools.test.ts b/test/workspace-file-tools.test.ts index d6940b13f..670d3c133 100644 --- a/test/workspace-file-tools.test.ts +++ b/test/workspace-file-tools.test.ts @@ -631,8 +631,9 @@ test('grep matches and shows a CRLF file\'s lines without the CR, as read shows assert.deepEqual(byFile['work/bom.txt'].matches, [{ line: 1, text: 'foo', context: ['foo'] }]); // The cost: a pattern written to find the CR at a line's end no longer // does, and the answer says why. - const note = "The pattern names a CR, and a file searched has CRLF line ends. Lines are matched as read shows them, " + const noteFor = (files: string) => `The pattern names a CR, and ${files}. Lines are matched as read shows them, ` + "which leaves out a CRLF end's CR unless the line's own text ends in a CR. read with offsetChars shows every CR in the workspace's copy."; + const note = noteFor('1 file searched has CRLF line ends: work/crlf.txt (CRLF)'); for (const pattern of ['\\r', '\r', 'o[\\r]?$', '\\\\\\r', '\\x0d', '\\x0D', '\\u000d', '\\u000D', '\\cM', '\\cm', '\\015', '[\\x0d]', '\\\r']) { const cr = await call('grep', { pattern, path: 'work/crlf.txt' }); const data = cr.data as { results: Array<{ matches: unknown[] }>; note?: string }; @@ -658,7 +659,8 @@ test('grep matches and shows a CRLF file\'s lines without the CR, as read shows writeFileSync(join(dir, 'many', 'a.txt'), 'x\n'); writeFileSync(join(dir, 'many', 'b.txt'), 'x\r\n'); writeFileSync(join(dir, 'many', 'c.txt'), 'x\n'); - assert.equal(((await call('grep', { pattern: '\\r', path: 'work/many' })).data as { note?: string }).note, note); + assert.equal(((await call('grep', { pattern: '\\r', path: 'work/many' })).data as { note?: string }).note, + noteFor('1 file searched has CRLF line ends: work/many/b.txt (CRLF)')); const globbed = (await call('grep', { pattern: 'x|\\r', path: 'work/many', glob: 'many/a.txt' })).data as { results: Array<{ file: string }> }; assert.deepEqual(globbed.results.map((r) => r.file), ['work/many/a.txt']); assert.equal('note' in globbed, false); @@ -668,8 +670,9 @@ test("grep says a BOM isn't part of the first line when a pattern for U+FEFF sea const { dir, call } = setup(t); writeFileSync(join(dir, 'bom.txt'), Buffer.concat([Buffer.from([0xef, 0xbb, 0xbf]), Buffer.from('foo\n')])); writeFileSync(join(dir, 'inner.txt'), 'a\uFEFFb\n'); - const note = "The pattern names U+FEFF, and a file searched starts with a BOM. Lines are matched as read shows them, " + const noteFor = (file: string) => `The pattern names U+FEFF, and 1 file searched starts with a BOM: ${file}. Lines are matched as read shows them, ` + "and a BOM isn't part of the first line. read with offsetChars shows any BOM the workspace's copy has."; + const note = noteFor('work/bom.txt'); for (const pattern of ['\\uFEFF', '^\\ufeff', '\uFEFF', '\\\uFEFF']) { const data = (await call('grep', { pattern, path: 'work/bom.txt' })).data as { results: unknown[]; note?: string }; assert.equal(data.note, note, JSON.stringify(pattern)); @@ -694,17 +697,49 @@ test("grep says a BOM isn't part of the first line when a pattern for U+FEFF sea writeFileSync(join(dir, 'boms', 'a.txt'), 'x\n'); writeFileSync(join(dir, 'boms', 'b.txt'), Buffer.concat([Buffer.from([0xef, 0xbb, 0xbf]), Buffer.from('x\n')])); writeFileSync(join(dir, 'boms', 'c.txt'), 'x\n'); - assert.equal(((await call('grep', { pattern: '\\uFEFF', path: 'work/boms' })).data as { note?: string }).note, note); + assert.equal(((await call('grep', { pattern: '\\uFEFF', path: 'work/boms' })).data as { note?: string }).note, noteFor('work/boms/b.txt')); // A pattern that names both, over a file with a BOM and CRLF ends, gets both // notes, the CR's first. writeFileSync(join(dir, 'both.txt'), Buffer.concat([Buffer.from([0xef, 0xbb, 0xbf]), Buffer.from('x\r\n')])); - const crNote = "The pattern names a CR, and a file searched has CRLF line ends. Lines are matched as read shows them, " + const crNote = "The pattern names a CR, and 1 file searched has CRLF line ends: work/both.txt (CRLF). Lines are matched as read shows them, " + "which leaves out a CRLF end's CR unless the line's own text ends in a CR. read with offsetChars shows every CR in the workspace's copy."; const both = (await call('grep', { pattern: '^\\uFEFF|\\r$', path: 'work/both.txt' })).data as { results: unknown[]; note?: string }; - assert.equal(both.note, `${crNote} ${note}`); + assert.equal(both.note, `${crNote} ${noteFor('work/both.txt')}`); assert.deepEqual(both.results, []); }); +test("grep's note on a pattern for a CR or U+FEFF names the files searched that have CRLF ends or a BOM, counting every one and naming at most 20", async (t) => { + const { dir, call } = setup(t); + const bom = Buffer.from([0xef, 0xbb, 0xbf]); + writeFileSync(join(dir, 'lf.txt'), 'a\nb\n'); + writeFileSync(join(dir, 'win.bat'), 'echo one\r\necho two\r\n'); + writeFileSync(join(dir, 'mixed.cfg'), 'x=1\r\ny=2\nz=3\r\n'); + writeFileSync(join(dir, 'bom.txt'), Buffer.concat([bom, Buffer.from('first\n')])); + writeFileSync(join(dir, 'both.json'), Buffer.concat([bom, Buffer.from('{}\r\n')])); + for (let i = 0; i < 3; i++) writeFileSync(join(dir, `src${i}.ts`), `export const v${i} = ${i};\n`); + const tail = "Lines are matched as read shows them, which leaves out a CRLF end's CR unless the line's own text ends in a CR. " + + "read with offsetChars shows every CR in the workspace's copy."; + // On main, \r$ found these files' lines; here no line keeps a CRLF end's + // CR, so the note names the files, mixed ends as such, sorted as results are. + const cr = (await call('grep', { pattern: '\\r$' })).data as { results: unknown[]; totalMatches: number; note?: string }; + assert.deepEqual([cr.results, cr.totalMatches], [[], 0]); + assert.equal(cr.note, 'The pattern names a CR, and 3 files searched have CRLF line ends: ' + + `work/both.json (CRLF), work/mixed.cfg (mixed), work/win.bat (CRLF). ${tail}`); + const boms = (await call('grep', { pattern: '^\\uFEFF' })).data as { results: unknown[]; note?: string }; + assert.deepEqual(boms.results, []); + assert.equal(boms.note, 'The pattern names U+FEFF, and 2 files searched start with a BOM: work/bom.txt, work/both.json. ' + + "Lines are matched as read shows them, and a BOM isn't part of the first line. read with offsetChars shows any BOM the workspace's copy has."); + // Past 20 files the count stays exact, and the rest are found by narrowing. + mkdirSync(join(dir, 'many')); + for (let i = 0; i < 23; i++) writeFileSync(join(dir, 'many', `f${String(i).padStart(2, '0')}.txt`), 'x\r\n'); + const many = (await call('grep', { pattern: '\\r', path: 'work/many' })).data as { note?: string }; + const first20 = Array.from({ length: 20 }, (_, i) => `work/many/f${String(i).padStart(2, '0')}.txt (CRLF)`).join(', '); + assert.equal(many.note, `The pattern names a CR, and 23 files searched have CRLF line ends: ${first20}, and 3 more; narrow with path or glob. ${tail}`); + const narrowed = (await call('grep', { pattern: '\\r', path: 'work/many', glob: 'many/f2*.txt' })).data as { note?: string }; + assert.equal(narrowed.note, 'The pattern names a CR, and 3 files searched have CRLF line ends: ' + + `work/many/f20.txt (CRLF), work/many/f21.txt (CRLF), work/many/f22.txt (CRLF). ${tail}`); +}); + test('text copied from a read or a grep of a CRLF file matches in an edit, and the file keeps its line ends and BOM', async (t) => { const { dir, call, disk } = setup(t, { autoMaterialize: true }); const bom = Buffer.from([0xef, 0xbb, 0xbf]); diff --git a/test/workspace-three-way.test.ts b/test/workspace-three-way.test.ts index fce6eccea..19aabd48b 100644 --- a/test/workspace-three-way.test.ts +++ b/test/workspace-three-way.test.ts @@ -6775,7 +6775,19 @@ describe('grep', () => { // finds no CR there. const cr = await call(m, 'grep', { pattern: '\\r$' }); assert.deepEqual(cr.results, []); - assert.match(cr.note, /^The pattern names a CR, and a file searched has CRLF line ends\./); + assert.match(cr.note, /^The pattern names a CR, and 1 file searched has CRLF line ends: work\/a\.txt \(CRLF in its conflicting-disk version\)\. /); + }); + + test("a conflict whose workspace copy and disk version both have CRLF ends is named once in a CR pattern's note, as the workspace's copy", async (t) => { + const env = new Env(t); + const m = await env.open(); + await seedSynced(env, m, 'a.txt', 'v1'); + env.writeDisk('a.txt', 'disk says ALPHA\r\n'); + await call(m, 'write', { path: 'work/a.txt', content: 'workspace says GAMMA\r\n' }); + assert.equal((await entryOf(m, 'a.txt'))?.conflict?.diskCopy, 'stored'); + const cr = await call(m, 'grep', { pattern: '\\r$' }); + assert.deepEqual(cr.results, []); + assert.match(cr.note, /^The pattern names a CR, and 1 file searched has CRLF line ends: work\/a\.txt \(CRLF\)\. /); }); test("a U+FEFF pattern over a conflict whose disk version alone starts with a BOM gets the note, which names only the workspace's copy as what read shows", async (t) => { @@ -6787,8 +6799,8 @@ describe('grep', () => { assert.equal((await entryOf(m, 'a.txt'))?.conflict?.diskCopy, 'stored'); const data = await call(m, 'grep', { pattern: '^\\uFEFF' }); assert.deepEqual(data.results, []); - assert.equal(data.note, "The pattern names U+FEFF, and a file searched starts with a BOM. Lines are matched as read shows them, " - + "and a BOM isn't part of the first line. read with offsetChars shows any BOM the workspace's copy has."); + assert.equal(data.note, "The pattern names U+FEFF, and 1 file searched starts with a BOM: work/a.txt (in its conflicting-disk version). " + + "Lines are matched as read shows them, and a BOM isn't part of the first line. read with offsetChars shows any BOM the workspace's copy has."); // The workspace's copy, which read shows, has none. assert.equal((await call(m, 'read', { path: 'work/a.txt', offsetChars: 0 })).content, 'workspace says GAMMA\n'); }); From 1142d08130c2e549fc4341930f23ed59588d4997 Mon Sep 17 00:00:00 2001 From: Abigail-2676 Date: Wed, 21 Oct 2026 11:12:12 -0400 Subject: [PATCH 46/51] docs(changelog): how a grep for \r or ^\uFEFF finds the files it searches with CRLF ends or a BOM Anarchid's nightly review R4-1 (#465, at a40712ef3) asked for a sentence saying how to find CRLF and BOM files now. The grep sentence says the note names the files, each as CRLF or mixed, up to 20 and then how many more, and that a BOM pattern's note names the files that start with one. Co-Authored-By: Claude Opus 5.5 --- changelog.d/feat-workspace-read.changed.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/changelog.d/feat-workspace-read.changed.md b/changelog.d/feat-workspace-read.changed.md index 71fe0c187..f06a5b3a6 100644 --- a/changelog.d/feat-workspace-read.changed.md +++ b/changelog.d/feat-workspace-read.changed.md @@ -1,6 +1,6 @@ - The workspace's `read` shows a file's text as it is, and says so. A line page shows a CRLF file's lines without the CR before each line feed, and leaves out a BOM. A line whose own text ends in a CR shows its line end's CR too, since `edit` reads a CR before a line feed as part of the line end, so text copied from the line still matches it. Its note says when the file has CRLF ends or a BOM, counting a mix of CRLF and LF ends and the lines that show both CRs. It also says when the file isn't valid UTF-8: each invalid byte sequence shows as U+FFFD, `edit` refuses the file, and `write` replaces it whole. Before, each line of a CRLF file ended in a CR, a BOM came first in the first line as U+FEFF, and invalid bytes showed as U+FFFD with no note. A character page still reads the stored text as it is, BOM and CRs included. - On a page for a call that carries a `resultMaxChars`, such as a model's or a running script's, a line over 2,000 characters is cut, with a marker that gives the `offsetChars` and `limitChars` that read its rest. Before, a line page showed every line whole, however long. - A `read` page stops before its result would pass `resultMaxChars`, the most of it the framework stores whole. For a model's call that's its agent's inline cap, 24,000 characters by default. Its note says where to continue: the `offset` after its last line, or for a character page the `offsetChars` it also gives as `nextOffsetChars`, with the `limitChars` left of what it was asked for, so a reader that follows the notes ends where it asked to. Where even a page's first line would pass the limit, that line is cut shorter, at the most the page has room for. Where a page's last line is cut, its note names the `offsetChars` where that line's rest starts and the `limitChars` that reads it to the line's end, before the next line's `offset` if the file goes on. A page always moves on, so where its own fields (the path and its notes) and the least it can show pass a small limit, it passes the limit. At 1,000 characters, a page with no notes does that only for a path of about 680 characters or more. Its notes take room from that: with all three notes on a file's text and a conflict's, a path of 93 to 129 characters does it, and a note that names a symlink's target, as one saying disk couldn't be checked can, takes as much room as the target is long. Before, a page of up to 2,000 lines, or of `limitChars` characters, went out whole, and for a result that passed the limit the framework kept only its start, cut mid-JSON, writing the whole of it to a workspace file where the agent has one. A caller that pages 2,000 lines at a time and starts the next page at line 2,001 now starts where the note says. A running `code_execution` script's call has a limit of 5,000,000 characters, so its line pages still stop at the line limit, 2,000 by default, and its character pages at `limitChars`, unless a page would pass that. Its line pages cut a line over 2,000 characters and leave out CRs and a BOM, as a model's do, and a character page gives it the stored text exactly. A call made in process, through `handleToolCall`, `executeToolCall` or `ModuleContext.callTool`, carries no `resultMaxChars` unless its caller sets one. Its line pages stop at the line limit alone and show every line whole, as they did before, though now without the CRs and the BOM. -- The workspace's `grep` leaves out a CRLF end's CR, for matching as well as display, as `read` does, so `foo$` finds a CRLF file's line too, as it does under ripgrep's `--crlf`. A match's text and context are its lines as `read` shows them, but each line whole, where a `read` page for a call with a result limit cuts a line over 2,000 characters. A BOM isn't part of the first line, and a file with CRLF ends says so in its result's `lineEndings` (`CRLF`, or `mixed`). Before, a CRLF file's lines ended in a CR, so `foo$` missed them and a match's text copied into an edit carried the CR. A pattern written to find the CR of a CRLF line end no longer matches, except on a line whose own text ends in a CR, which keeps its line end's CR as `read` shows it. When a pattern names a CR, as a literal one or as `\r`, `\x0d`, `\u000d`, `\cM` or `\015`, and a file it searched has CRLF ends, the result's `note` says so, and that a character `read` shows every CR in the workspace's copy, or in disk's file on a direct mount where the host has one. A pattern that names U+FEFF, over a file that starts with a BOM, gets a note that a BOM isn't part of the first line. A CR that doesn't come before a line feed is still part of its line. +- The workspace's `grep` leaves out a CRLF end's CR, for matching as well as display, as `read` does, so `foo$` finds a CRLF file's line too, as it does under ripgrep's `--crlf`. A match's text and context are its lines as `read` shows them, but each line whole, where a `read` page for a call with a result limit cuts a line over 2,000 characters. A BOM isn't part of the first line, and a file with CRLF ends says so in its result's `lineEndings` (`CRLF`, or `mixed`). Before, a CRLF file's lines ended in a CR, so `foo$` missed them and a match's text copied into an edit carried the CR. A pattern written to find the CR of a CRLF line end no longer matches, except on a line whose own text ends in a CR, which keeps its line end's CR as `read` shows it. When a pattern names a CR, as a literal one or as `\r`, `\x0d`, `\u000d`, `\cM` or `\015`, and a file it searched has CRLF ends, the result's `note` says how many files searched have them and names them, each as `CRLF` or `mixed`, up to 20 and then how many more. It also says that a character `read` shows every CR in the workspace's copy, or in disk's file on a direct mount where the host has one. A pattern that names U+FEFF, over a file that starts with a BOM, gets a note that names the files that start with one and says a BOM isn't part of the first line. So `grep` for `\r` or `^\uFEFF` still finds which files have CRLF ends or a BOM. A CR that doesn't come before a line feed is still part of its line. - The workspace's `read` first takes in a change on disk to a file the workspace holds, as a listing and an `edit` do, and shows that text. Its note says so when disk's copy had changed since the workspace last agreed with it. It also says when disk's copy couldn't be checked, or disk has none and nothing records whether it was deleted there, so it shows the workspace's copy; where the check gave up because the selected branch kept changing, it says to read again. A file disk deleted answers that disk deleted it and the deletion was taken in, and a file disk made binary or too large answers that it's on disk but not stored. Where the path's stored copy is a draft and disk changed or deleted it too, the read shows the draft, and its note names the conflict and the ways to settle it, a sync, or a forced materialize on a mount that isn't read-only: for a deletion, a sync of the path takes it and removes the workspace's copy. Before, a read of a path the workspace held didn't look at disk, so it could show a copy another writer had since changed; a path it didn't hold was observed, as now. - A direct mount's `read` and `grep` show a file's text as a store-backed mount's do, with the same pages, cut lines, notes and `lineEndings`. Only the disk check is a store-backed mount's alone, since a direct mount reads disk as it is and keeps no copy of its own. From 786cdc1b4bcdb7039f6bddfa13baa368c02958cd Mon Sep 17 00:00:00 2001 From: Abigail-2676 Date: Wed, 21 Oct 2026 11:51:27 -0400 Subject: [PATCH 47/51] test(workspace): grep's notes on a conflict per note, on a recorded disk version, and with skipped files Anarchid's nightly review R4-1 (#465, at a40712ef3) fix, made more exact: - A conflict whose workspace copy and disk version both have CRLF ends and a BOM is named once in each note, as the workspace's copy. - One whose workspace copy starts with a BOM and whose disk version has CRLF ends is named in each note for the version that has it: the CR note names the conflicting-disk version, the BOM note the workspace's copy. - A conflict's recorded disk version, once disk has changed again, is named in each note as the recorded-disk version. - A binary file and one over the size limit, each with a BOM and CRLF ends, are listed as skipped and named in neither note. Co-Authored-By: Claude Opus 5.5 --- test/workspace-file-tools.test.ts | 9 +++++-- test/workspace-three-way.test.ts | 41 ++++++++++++++++++++++++++++--- 2 files changed, 45 insertions(+), 5 deletions(-) diff --git a/test/workspace-file-tools.test.ts b/test/workspace-file-tools.test.ts index 670d3c133..93590313d 100644 --- a/test/workspace-file-tools.test.ts +++ b/test/workspace-file-tools.test.ts @@ -709,7 +709,7 @@ test("grep says a BOM isn't part of the first line when a pattern for U+FEFF sea }); test("grep's note on a pattern for a CR or U+FEFF names the files searched that have CRLF ends or a BOM, counting every one and naming at most 20", async (t) => { - const { dir, call } = setup(t); + const { dir, call } = setup(t, { maxFileSize: 1000 }); const bom = Buffer.from([0xef, 0xbb, 0xbf]); writeFileSync(join(dir, 'lf.txt'), 'a\nb\n'); writeFileSync(join(dir, 'win.bat'), 'echo one\r\necho two\r\n'); @@ -717,12 +717,17 @@ test("grep's note on a pattern for a CR or U+FEFF names the files searched that writeFileSync(join(dir, 'bom.txt'), Buffer.concat([bom, Buffer.from('first\n')])); writeFileSync(join(dir, 'both.json'), Buffer.concat([bom, Buffer.from('{}\r\n')])); for (let i = 0; i < 3; i++) writeFileSync(join(dir, `src${i}.ts`), `export const v${i} = ${i};\n`); + // Files grep doesn't search, binary or over the size limit, are listed as + // skipped and named in no note. + writeFileSync(join(dir, 'blob.bin'), Buffer.concat([bom, Buffer.from('x\r\n\0y\r\n')])); + writeFileSync(join(dir, 'big.txt'), Buffer.concat([bom, Buffer.from('z\r\n'.repeat(400))])); const tail = "Lines are matched as read shows them, which leaves out a CRLF end's CR unless the line's own text ends in a CR. " + "read with offsetChars shows every CR in the workspace's copy."; // On main, \r$ found these files' lines; here no line keeps a CRLF end's // CR, so the note names the files, mixed ends as such, sorted as results are. - const cr = (await call('grep', { pattern: '\\r$' })).data as { results: unknown[]; totalMatches: number; note?: string }; + const cr = (await call('grep', { pattern: '\\r$' })).data as { results: unknown[]; totalMatches: number; note?: string; skipped?: Array<{ file: string }> }; assert.deepEqual([cr.results, cr.totalMatches], [[], 0]); + assert.deepEqual(cr.skipped?.map((s) => s.file), ['work/big.txt', 'work/blob.bin']); assert.equal(cr.note, 'The pattern names a CR, and 3 files searched have CRLF line ends: ' + `work/both.json (CRLF), work/mixed.cfg (mixed), work/win.bat (CRLF). ${tail}`); const boms = (await call('grep', { pattern: '^\\uFEFF' })).data as { results: unknown[]; note?: string }; diff --git a/test/workspace-three-way.test.ts b/test/workspace-three-way.test.ts index 19aabd48b..706d23f8c 100644 --- a/test/workspace-three-way.test.ts +++ b/test/workspace-three-way.test.ts @@ -6778,16 +6778,34 @@ describe('grep', () => { assert.match(cr.note, /^The pattern names a CR, and 1 file searched has CRLF line ends: work\/a\.txt \(CRLF in its conflicting-disk version\)\. /); }); - test("a conflict whose workspace copy and disk version both have CRLF ends is named once in a CR pattern's note, as the workspace's copy", async (t) => { + test("a conflict whose workspace copy and disk version both have CRLF ends and a BOM is named once in each note, as the workspace's copy", async (t) => { const env = new Env(t); const m = await env.open(); await seedSynced(env, m, 'a.txt', 'v1'); - env.writeDisk('a.txt', 'disk says ALPHA\r\n'); - await call(m, 'write', { path: 'work/a.txt', content: 'workspace says GAMMA\r\n' }); + env.writeDisk('a.txt', '\uFEFFdisk says ALPHA\r\n'); + await call(m, 'write', { path: 'work/a.txt', content: '\uFEFFworkspace says GAMMA\r\n' }); assert.equal((await entryOf(m, 'a.txt'))?.conflict?.diskCopy, 'stored'); const cr = await call(m, 'grep', { pattern: '\\r$' }); assert.deepEqual(cr.results, []); assert.match(cr.note, /^The pattern names a CR, and 1 file searched has CRLF line ends: work\/a\.txt \(CRLF\)\. /); + const bom = await call(m, 'grep', { pattern: '^\\uFEFF' }); + assert.deepEqual(bom.results, []); + assert.match(bom.note, /^The pattern names U\+FEFF, and 1 file searched starts with a BOM: work\/a\.txt\. /); + }); + + test("a conflict whose workspace copy starts with a BOM and whose disk version has CRLF ends is named in each note for the version that has it", async (t) => { + const env = new Env(t); + const m = await env.open(); + await seedSynced(env, m, 'a.txt', 'v1'); + env.writeDisk('a.txt', 'disk says ALPHA\r\n'); + await call(m, 'write', { path: 'work/a.txt', content: '\uFEFFworkspace says GAMMA\n' }); + assert.equal((await entryOf(m, 'a.txt'))?.conflict?.diskCopy, 'stored'); + const cr = await call(m, 'grep', { pattern: '\\r$' }); + assert.deepEqual(cr.results, []); + assert.match(cr.note, /^The pattern names a CR, and 1 file searched has CRLF line ends: work\/a\.txt \(CRLF in its conflicting-disk version\)\. /); + const bom = await call(m, 'grep', { pattern: '^\\uFEFF' }); + assert.deepEqual(bom.results, []); + assert.match(bom.note, /^The pattern names U\+FEFF, and 1 file searched starts with a BOM: work\/a\.txt\. /); }); test("a U+FEFF pattern over a conflict whose disk version alone starts with a BOM gets the note, which names only the workspace's copy as what read shows", async (t) => { @@ -6838,6 +6856,23 @@ describe('grep', () => { assert.equal(gone.skipped, undefined, 'and no disk file went unsearched'); assert.equal(await contentOf(m, 'a.txt'), 'workspace says GAMMA', 'the draft is kept'); }); + + test("a conflict's recorded disk version with CRLF ends and a BOM is named as that version in each note", async (t) => { + const env = new Env(t); + const m = await env.open(); + await seedSynced(env, m, 'a.txt', 'v1'); + env.writeDisk('a.txt', '\uFEFFdisk says ALPHA\r\n'); + await call(m, 'write', { path: 'work/a.txt', content: 'workspace says GAMMA\n' }); + assert.equal((await entryOf(m, 'a.txt'))?.conflict?.diskCopy, 'stored'); + env.writeDisk('a.txt', 'disk now says BETA\n'); + assert.equal((await entryOf(m, 'a.txt'))?.conflict?.diskChangedSinceRecorded, true); + const cr = await call(m, 'grep', { pattern: '\\r$' }); + assert.deepEqual(cr.results, []); + assert.match(cr.note, /^The pattern names a CR, and 1 file searched has CRLF line ends: work\/a\.txt \(CRLF in its recorded-disk version\)\. /); + const bom = await call(m, 'grep', { pattern: '^\\uFEFF' }); + assert.deepEqual(bom.results, []); + assert.match(bom.note, /^The pattern names U\+FEFF, and 1 file searched starts with a BOM: work\/a\.txt \(in its recorded-disk version\)\. /); + }); }); describe('grep, when the branch or the store changes under it', () => { From deac5132ccf5a72d90b204022f8d7e5e33d60ef3 Mon Sep 17 00:00:00 2001 From: Abigail-2676 Date: Wed, 21 Oct 2026 11:51:27 -0400 Subject: [PATCH 48/51] docs(workspace): grep keeps a conflict's disk version per note, and only the CR note gives ends Anarchid's nightly review R4-1 (#465, at a40712ef3) fix, worded as the code does it. The comment on grep's files said a conflict's disk version is kept only where the workspace's copy has neither CRLF ends nor a BOM; each note keeps it where the workspace's copy lacks that note's character. leftOutNote's doc said each named file comes with its ends; only the CR note gives them, and either note names a disk version as that version. The changelog's sentence on finding CRLF and BOM files now says it finds them among the files grep searches. Co-Authored-By: Claude Opus 5.5 --- changelog.d/feat-workspace-read.changed.md | 2 +- src/modules/workspace/index.ts | 11 ++++++----- 2 files changed, 7 insertions(+), 6 deletions(-) diff --git a/changelog.d/feat-workspace-read.changed.md b/changelog.d/feat-workspace-read.changed.md index f06a5b3a6..bb829ae1e 100644 --- a/changelog.d/feat-workspace-read.changed.md +++ b/changelog.d/feat-workspace-read.changed.md @@ -1,6 +1,6 @@ - The workspace's `read` shows a file's text as it is, and says so. A line page shows a CRLF file's lines without the CR before each line feed, and leaves out a BOM. A line whose own text ends in a CR shows its line end's CR too, since `edit` reads a CR before a line feed as part of the line end, so text copied from the line still matches it. Its note says when the file has CRLF ends or a BOM, counting a mix of CRLF and LF ends and the lines that show both CRs. It also says when the file isn't valid UTF-8: each invalid byte sequence shows as U+FFFD, `edit` refuses the file, and `write` replaces it whole. Before, each line of a CRLF file ended in a CR, a BOM came first in the first line as U+FEFF, and invalid bytes showed as U+FFFD with no note. A character page still reads the stored text as it is, BOM and CRs included. - On a page for a call that carries a `resultMaxChars`, such as a model's or a running script's, a line over 2,000 characters is cut, with a marker that gives the `offsetChars` and `limitChars` that read its rest. Before, a line page showed every line whole, however long. - A `read` page stops before its result would pass `resultMaxChars`, the most of it the framework stores whole. For a model's call that's its agent's inline cap, 24,000 characters by default. Its note says where to continue: the `offset` after its last line, or for a character page the `offsetChars` it also gives as `nextOffsetChars`, with the `limitChars` left of what it was asked for, so a reader that follows the notes ends where it asked to. Where even a page's first line would pass the limit, that line is cut shorter, at the most the page has room for. Where a page's last line is cut, its note names the `offsetChars` where that line's rest starts and the `limitChars` that reads it to the line's end, before the next line's `offset` if the file goes on. A page always moves on, so where its own fields (the path and its notes) and the least it can show pass a small limit, it passes the limit. At 1,000 characters, a page with no notes does that only for a path of about 680 characters or more. Its notes take room from that: with all three notes on a file's text and a conflict's, a path of 93 to 129 characters does it, and a note that names a symlink's target, as one saying disk couldn't be checked can, takes as much room as the target is long. Before, a page of up to 2,000 lines, or of `limitChars` characters, went out whole, and for a result that passed the limit the framework kept only its start, cut mid-JSON, writing the whole of it to a workspace file where the agent has one. A caller that pages 2,000 lines at a time and starts the next page at line 2,001 now starts where the note says. A running `code_execution` script's call has a limit of 5,000,000 characters, so its line pages still stop at the line limit, 2,000 by default, and its character pages at `limitChars`, unless a page would pass that. Its line pages cut a line over 2,000 characters and leave out CRs and a BOM, as a model's do, and a character page gives it the stored text exactly. A call made in process, through `handleToolCall`, `executeToolCall` or `ModuleContext.callTool`, carries no `resultMaxChars` unless its caller sets one. Its line pages stop at the line limit alone and show every line whole, as they did before, though now without the CRs and the BOM. -- The workspace's `grep` leaves out a CRLF end's CR, for matching as well as display, as `read` does, so `foo$` finds a CRLF file's line too, as it does under ripgrep's `--crlf`. A match's text and context are its lines as `read` shows them, but each line whole, where a `read` page for a call with a result limit cuts a line over 2,000 characters. A BOM isn't part of the first line, and a file with CRLF ends says so in its result's `lineEndings` (`CRLF`, or `mixed`). Before, a CRLF file's lines ended in a CR, so `foo$` missed them and a match's text copied into an edit carried the CR. A pattern written to find the CR of a CRLF line end no longer matches, except on a line whose own text ends in a CR, which keeps its line end's CR as `read` shows it. When a pattern names a CR, as a literal one or as `\r`, `\x0d`, `\u000d`, `\cM` or `\015`, and a file it searched has CRLF ends, the result's `note` says how many files searched have them and names them, each as `CRLF` or `mixed`, up to 20 and then how many more. It also says that a character `read` shows every CR in the workspace's copy, or in disk's file on a direct mount where the host has one. A pattern that names U+FEFF, over a file that starts with a BOM, gets a note that names the files that start with one and says a BOM isn't part of the first line. So `grep` for `\r` or `^\uFEFF` still finds which files have CRLF ends or a BOM. A CR that doesn't come before a line feed is still part of its line. +- The workspace's `grep` leaves out a CRLF end's CR, for matching as well as display, as `read` does, so `foo$` finds a CRLF file's line too, as it does under ripgrep's `--crlf`. A match's text and context are its lines as `read` shows them, but each line whole, where a `read` page for a call with a result limit cuts a line over 2,000 characters. A BOM isn't part of the first line, and a file with CRLF ends says so in its result's `lineEndings` (`CRLF`, or `mixed`). Before, a CRLF file's lines ended in a CR, so `foo$` missed them and a match's text copied into an edit carried the CR. A pattern written to find the CR of a CRLF line end no longer matches, except on a line whose own text ends in a CR, which keeps its line end's CR as `read` shows it. When a pattern names a CR, as a literal one or as `\r`, `\x0d`, `\u000d`, `\cM` or `\015`, and a file it searched has CRLF ends, the result's `note` says how many files searched have them and names them, each as `CRLF` or `mixed`, up to 20 and then how many more. It also says that a character `read` shows every CR in the workspace's copy, or in disk's file on a direct mount where the host has one. A pattern that names U+FEFF, over a file that starts with a BOM, gets a note that names the files that start with one and says a BOM isn't part of the first line. So a `grep` for `\r` or `^\uFEFF` still finds which of the files it searches have CRLF ends or a BOM. A CR that doesn't come before a line feed is still part of its line. - The workspace's `read` first takes in a change on disk to a file the workspace holds, as a listing and an `edit` do, and shows that text. Its note says so when disk's copy had changed since the workspace last agreed with it. It also says when disk's copy couldn't be checked, or disk has none and nothing records whether it was deleted there, so it shows the workspace's copy; where the check gave up because the selected branch kept changing, it says to read again. A file disk deleted answers that disk deleted it and the deletion was taken in, and a file disk made binary or too large answers that it's on disk but not stored. Where the path's stored copy is a draft and disk changed or deleted it too, the read shows the draft, and its note names the conflict and the ways to settle it, a sync, or a forced materialize on a mount that isn't read-only: for a deletion, a sync of the path takes it and removes the workspace's copy. Before, a read of a path the workspace held didn't look at disk, so it could show a copy another writer had since changed; a path it didn't hold was observed, as now. - A direct mount's `read` and `grep` show a file's text as a store-backed mount's do, with the same pages, cut lines, notes and `lineEndings`. Only the disk check is a store-backed mount's alone, since a direct mount reads disk as it is and keeps no copy of its own. diff --git a/src/modules/workspace/index.ts b/src/modules/workspace/index.ts index ef087b0cd..d6880d27a 100644 --- a/src/modules/workspace/index.ts +++ b/src/modules/workspace/index.ts @@ -4645,9 +4645,9 @@ export class WorkspaceModule implements Module { // Whether a file has CRLF ends or a BOM is a fact about the file, not a // line, so each searched file that has either is kept here by its path, // for the note on a pattern that names a CR or U+FEFF to name it. A - // conflict's disk version is kept only where the workspace's copy of its - // path has neither, and is named as that version, since read doesn't show - // it. + // conflict's disk version is kept, for each, only where the workspace's + // copy of its path doesn't have it, and is named as that version, since + // read doesn't show it. const crlfFiles = new Map(); const bomFiles = new Map(); const search = (text: GrepText): Searched => { @@ -5957,8 +5957,9 @@ const ofVersion = (version: GrepText['version']): string => /** * grep's note for a pattern that names a character its lines leave out, where * a file it searched has one: it says how many files searched have it and - * names them, each with whether its ends are CRLF or mixed, so a search for a - * CR or a BOM still finds the files that have them. The note says a + * names them, so a search for a CR or a BOM still finds the files that have + * them. The CR note gives each file's ends, CRLF or mixed, and either note + * names a conflict's disk version as that version. The note says a * character read shows it in the workspace's copy, since grep also searches a * conflict's disk version, which read doesn't show. A direct mount keeps no * workspace copy, and its read shows disk's file, so on a host with one From 754b17fa0c6a016fab83a5942ccd98bc2cc6dcd1 Mon Sep 17 00:00:00 2001 From: Abigail-2676 Date: Wed, 21 Oct 2026 12:07:55 -0400 Subject: [PATCH 49/51] docs(workspace): a read's limit is what reaches its caller whole, a script's included Main's code_execution now gives a running script's call the longest result that reaches the script whole, 7,456,527 characters at the default 128 MiB of script memory, and answers a longer one with ResultTooLarge, cutting and saving nothing. The workspace's comments said a read's limit was the most the framework stores before it spills the rest into a file, which holds for a model's call alone. They now say it's the most of the call's result that reaches its caller whole, and what becomes of a longer one on each path. The changelog gives a script's figure beside a model's, and the read tests that stand for a script's call use that figure, where they used the 5,000,000 a script's call carried before. Co-Authored-By: Claude Opus 5.5 --- changelog.d/feat-workspace-read.changed.md | 2 +- src/modules/workspace/index.ts | 15 ++++++++------- src/modules/workspace/read-page.ts | 9 +++++---- test/workspace-read-text.test.ts | 8 +++++--- 4 files changed, 19 insertions(+), 15 deletions(-) diff --git a/changelog.d/feat-workspace-read.changed.md b/changelog.d/feat-workspace-read.changed.md index bb829ae1e..cce30f56f 100644 --- a/changelog.d/feat-workspace-read.changed.md +++ b/changelog.d/feat-workspace-read.changed.md @@ -1,6 +1,6 @@ - The workspace's `read` shows a file's text as it is, and says so. A line page shows a CRLF file's lines without the CR before each line feed, and leaves out a BOM. A line whose own text ends in a CR shows its line end's CR too, since `edit` reads a CR before a line feed as part of the line end, so text copied from the line still matches it. Its note says when the file has CRLF ends or a BOM, counting a mix of CRLF and LF ends and the lines that show both CRs. It also says when the file isn't valid UTF-8: each invalid byte sequence shows as U+FFFD, `edit` refuses the file, and `write` replaces it whole. Before, each line of a CRLF file ended in a CR, a BOM came first in the first line as U+FEFF, and invalid bytes showed as U+FFFD with no note. A character page still reads the stored text as it is, BOM and CRs included. - On a page for a call that carries a `resultMaxChars`, such as a model's or a running script's, a line over 2,000 characters is cut, with a marker that gives the `offsetChars` and `limitChars` that read its rest. Before, a line page showed every line whole, however long. -- A `read` page stops before its result would pass `resultMaxChars`, the most of it the framework stores whole. For a model's call that's its agent's inline cap, 24,000 characters by default. Its note says where to continue: the `offset` after its last line, or for a character page the `offsetChars` it also gives as `nextOffsetChars`, with the `limitChars` left of what it was asked for, so a reader that follows the notes ends where it asked to. Where even a page's first line would pass the limit, that line is cut shorter, at the most the page has room for. Where a page's last line is cut, its note names the `offsetChars` where that line's rest starts and the `limitChars` that reads it to the line's end, before the next line's `offset` if the file goes on. A page always moves on, so where its own fields (the path and its notes) and the least it can show pass a small limit, it passes the limit. At 1,000 characters, a page with no notes does that only for a path of about 680 characters or more. Its notes take room from that: with all three notes on a file's text and a conflict's, a path of 93 to 129 characters does it, and a note that names a symlink's target, as one saying disk couldn't be checked can, takes as much room as the target is long. Before, a page of up to 2,000 lines, or of `limitChars` characters, went out whole, and for a result that passed the limit the framework kept only its start, cut mid-JSON, writing the whole of it to a workspace file where the agent has one. A caller that pages 2,000 lines at a time and starts the next page at line 2,001 now starts where the note says. A running `code_execution` script's call has a limit of 5,000,000 characters, so its line pages still stop at the line limit, 2,000 by default, and its character pages at `limitChars`, unless a page would pass that. Its line pages cut a line over 2,000 characters and leave out CRs and a BOM, as a model's do, and a character page gives it the stored text exactly. A call made in process, through `handleToolCall`, `executeToolCall` or `ModuleContext.callTool`, carries no `resultMaxChars` unless its caller sets one. Its line pages stop at the line limit alone and show every line whole, as they did before, though now without the CRs and the BOM. +- A `read` page stops before its result would pass `resultMaxChars`, the most of it that reaches its caller whole. For a model's call that's its agent's inline cap, 24,000 characters by default, and for a running script's call what the script's memory takes, 7,456,527 characters at the default 128 MiB. Its note says where to continue: the `offset` after its last line, or for a character page the `offsetChars` it also gives as `nextOffsetChars`, with the `limitChars` left of what it was asked for, so a reader that follows the notes ends where it asked to. Where even a page's first line would pass the limit, that line is cut shorter, at the most the page has room for. Where a page's last line is cut, its note names the `offsetChars` where that line's rest starts and the `limitChars` that reads it to the line's end, before the next line's `offset` if the file goes on. A page always moves on, so where its own fields (the path and its notes) and the least it can show pass a small limit, it passes the limit. At 1,000 characters, a page with no notes does that only for a path of about 680 characters or more. Its notes take room from that: with all three notes on a file's text and a conflict's, a path of 93 to 129 characters does it, and a note that names a symlink's target, as one saying disk couldn't be checked can, takes as much room as the target is long. Before, a page of up to 2,000 lines, or of `limitChars` characters, went out whole, and for a result that passed the limit the framework kept only its start, cut mid-JSON, writing the whole of it to a workspace file where the agent has one. A caller that pages 2,000 lines at a time and starts the next page at line 2,001 now starts where the note says. A running `code_execution` script's call has that larger limit, so its line pages still stop at the line limit, 2,000 by default, and its character pages at `limitChars`, unless a page would pass that. Its line pages cut a line over 2,000 characters and leave out CRs and a BOM, as a model's do, and a character page gives it the stored text exactly. A call made in process, through `handleToolCall`, `executeToolCall` or `ModuleContext.callTool`, carries no `resultMaxChars` unless its caller sets one. Its line pages stop at the line limit alone and show every line whole, as they did before, though now without the CRs and the BOM. - The workspace's `grep` leaves out a CRLF end's CR, for matching as well as display, as `read` does, so `foo$` finds a CRLF file's line too, as it does under ripgrep's `--crlf`. A match's text and context are its lines as `read` shows them, but each line whole, where a `read` page for a call with a result limit cuts a line over 2,000 characters. A BOM isn't part of the first line, and a file with CRLF ends says so in its result's `lineEndings` (`CRLF`, or `mixed`). Before, a CRLF file's lines ended in a CR, so `foo$` missed them and a match's text copied into an edit carried the CR. A pattern written to find the CR of a CRLF line end no longer matches, except on a line whose own text ends in a CR, which keeps its line end's CR as `read` shows it. When a pattern names a CR, as a literal one or as `\r`, `\x0d`, `\u000d`, `\cM` or `\015`, and a file it searched has CRLF ends, the result's `note` says how many files searched have them and names them, each as `CRLF` or `mixed`, up to 20 and then how many more. It also says that a character `read` shows every CR in the workspace's copy, or in disk's file on a direct mount where the host has one. A pattern that names U+FEFF, over a file that starts with a BOM, gets a note that names the files that start with one and says a BOM isn't part of the first line. So a `grep` for `\r` or `^\uFEFF` still finds which of the files it searches have CRLF ends or a BOM. A CR that doesn't come before a line feed is still part of its line. - The workspace's `read` first takes in a change on disk to a file the workspace holds, as a listing and an `edit` do, and shows that text. Its note says so when disk's copy had changed since the workspace last agreed with it. It also says when disk's copy couldn't be checked, or disk has none and nothing records whether it was deleted there, so it shows the workspace's copy; where the check gave up because the selected branch kept changing, it says to read again. A file disk deleted answers that disk deleted it and the deletion was taken in, and a file disk made binary or too large answers that it's on disk but not stored. Where the path's stored copy is a draft and disk changed or deleted it too, the read shows the draft, and its note names the conflict and the ways to settle it, a sync, or a forced materialize on a mount that isn't read-only: for a deletion, a sync of the path takes it and removes the workspace's copy. Before, a read of a path the workspace held didn't look at disk, so it could show a copy another writer had since changed; a path it didn't hold was observed, as now. - A direct mount's `read` and `grep` show a file's text as a store-backed mount's do, with the same pages, cut lines, notes and `lineEndings`. Only the disk check is a store-backed mount's alone, since a direct mount reads disk as it is and keeps no copy of its own. diff --git a/src/modules/workspace/index.ts b/src/modules/workspace/index.ts index 4132838cf..9c38af80f 100644 --- a/src/modules/workspace/index.ts +++ b/src/modules/workspace/index.ts @@ -2027,9 +2027,10 @@ export class WorkspaceModule implements Module { return { success: false, isError: true, error: `The ${call.name} call did nothing: ${problem}.` }; } const input = call.input as Record; - // The most of this call's result the framework stores before it spills - // the rest into a file. A call made in process carries none and takes - // the data itself, so its read pages by its line limit alone. + // The most of this call's result that reaches its caller whole: past + // it, a model's result spills into a file, and a script gets + // ResultTooLarge in its place. A call made in process carries none and + // takes the data itself, so its read pages by its line limit alone. const resultMaxChars = call.resultMaxChars; const generated = typeof input?.path === 'string' ? this.findGeneratedTextFile(input.path) : undefined; if (generated) { @@ -3821,10 +3822,10 @@ export class WorkspaceModule implements Module { // ========================================================================== /** - * A read's page, held to `maxChars`, the most the framework stores of this - * call's result before it spills the rest into a file. A call that carries - * none, made in process, takes the data itself: its page holds to its line - * limit alone, with every line whole, as it did before pages had a budget. + * A read's page, held to `maxChars`, the most of this call's result that + * reaches its caller whole. A call that carries none, made in process, + * takes the data itself: its page holds to its line limit alone, with every + * line whole, as it did before pages had a budget. */ private async handleRead(input: ReadInput, maxChars: number | undefined, generatedContent?: string): Promise { const characterPaging = input.offsetChars !== undefined || input.limitChars !== undefined; diff --git a/src/modules/workspace/read-page.ts b/src/modules/workspace/read-page.ts index 9a3373bd0..6d880cc9a 100644 --- a/src/modules/workspace/read-page.ts +++ b/src/modules/workspace/read-page.ts @@ -2,10 +2,11 @@ import type { ToolResult } from '../../types/events.js'; import { type FileText, MAX_LINE_CHARS, jsonCharLength, jsonLength, numberedLine, safeCut } from './text.js'; /** - * One page of a read, held to the result limit: the framework stores a - * success's data as JSON, and spills what passes the caller's limit into a - * file, cut mid-JSON. So a page stops before it would pass the limit, and its - * note says where to continue. + * One page of a read, held to the result limit: the framework gives a + * success's data as JSON, and what passes the caller's limit doesn't reach it + * whole. A model's result spills into a file, cut mid-JSON, and a script gets + * ResultTooLarge in its place. So a page stops before it would pass the limit, + * and its note says where to continue. */ /** How many lines a read shows when it isn't given a limit. */ diff --git a/test/workspace-read-text.test.ts b/test/workspace-read-text.test.ts index 38b033c2f..f8b81fb9d 100644 --- a/test/workspace-read-text.test.ts +++ b/test/workspace-read-text.test.ts @@ -26,6 +26,8 @@ async function setup(t: TestContext, content: string | Buffer) { type LinePage = { totalLines: number; fromLine?: number; toLine?: number; note?: string; content: string }; type CharPage = { totalChars: number; offsetChars: number; nextOffsetChars: number | null; note?: string; content: string }; const stored = (result: { data?: unknown }): number => JSON.stringify(result.data).length; +// A running script's call's resultMaxChars at the default 128 MiB of script memory. +const SCRIPT_LIMIT = 7_456_527; test('line reads retain numbering, ranges, and the 2000-line default', async t => { const { read } = await setup(t, Array.from({ length: 2005 }, (_, i) => `line ${i + 1}`).join('\n')); @@ -36,7 +38,7 @@ test('line reads retain numbering, ranges, and the 2000-line default', async t = content: ' 2\tline 2\n 3\tline 3', }); // A script's call can take far more than a turn's, so its default page is the 2000 lines. - const defaults = (await read({}, 5_000_000)).data as LinePage; + const defaults = (await read({}, SCRIPT_LIMIT)).data as LinePage; assert.equal(defaults.toLine, 2000); assert.equal(defaults.content.split('\n').length, 2000); }); @@ -95,8 +97,8 @@ test('a long line is cut at 2000 characters, with the offsetChars and limitChars const rest = long.length - 1999; const at = 1 + 'short\r\n'.length + 1999; assert.equal(lines[1], ` 2\t${'x'.repeat(1999)} [… read the rest with offsetChars=${at} and limitChars=${rest}]`); - // A script's call, at 5,000,000 characters, cuts the line the same way. - assert.equal(((await read({}, 5_000_000)).data as LinePage).content.split('\n')[1], lines[1]); + // A script's call cuts the line the same way. + assert.equal(((await read({}, SCRIPT_LIMIT)).data as LinePage).content.split('\n')[1], lines[1]); const more = (await read({ offsetChars: at, limitChars: rest })).data as CharPage; assert.equal(more.content, '😀' + 'y'.repeat(1000)); assert.equal(lines[2], ' 3\tlast'); From 4060c27bc807016114aec0ce4786e5c400e35d19 Mon Sep 17 00:00:00 2001 From: Abigail-2676 Date: Thu, 22 Oct 2026 10:28:32 -0400 Subject: [PATCH 50/51] docs(workspace): a read's limit is the most that can reach its caller whole Main's ToolCall.resultMaxChars doc now says a model's result within the agent's inline cap is stored whole unless its round's results together pass the readers' per-message bound, when each takes an equal share. So a read page held to the cap may still spill, beside other large results of the same round. The workspace's comments said the limit is the most of the call's result that reaches its caller whole; they now say the most that can reach it whole, and the changelog says when a model's result within the cap is stored whole. Co-Authored-By: Claude Opus 5.5 --- changelog.d/feat-workspace-read.changed.md | 2 +- src/modules/workspace/index.ts | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/changelog.d/feat-workspace-read.changed.md b/changelog.d/feat-workspace-read.changed.md index cce30f56f..c94feec88 100644 --- a/changelog.d/feat-workspace-read.changed.md +++ b/changelog.d/feat-workspace-read.changed.md @@ -1,6 +1,6 @@ - The workspace's `read` shows a file's text as it is, and says so. A line page shows a CRLF file's lines without the CR before each line feed, and leaves out a BOM. A line whose own text ends in a CR shows its line end's CR too, since `edit` reads a CR before a line feed as part of the line end, so text copied from the line still matches it. Its note says when the file has CRLF ends or a BOM, counting a mix of CRLF and LF ends and the lines that show both CRs. It also says when the file isn't valid UTF-8: each invalid byte sequence shows as U+FFFD, `edit` refuses the file, and `write` replaces it whole. Before, each line of a CRLF file ended in a CR, a BOM came first in the first line as U+FEFF, and invalid bytes showed as U+FFFD with no note. A character page still reads the stored text as it is, BOM and CRs included. - On a page for a call that carries a `resultMaxChars`, such as a model's or a running script's, a line over 2,000 characters is cut, with a marker that gives the `offsetChars` and `limitChars` that read its rest. Before, a line page showed every line whole, however long. -- A `read` page stops before its result would pass `resultMaxChars`, the most of it that reaches its caller whole. For a model's call that's its agent's inline cap, 24,000 characters by default, and for a running script's call what the script's memory takes, 7,456,527 characters at the default 128 MiB. Its note says where to continue: the `offset` after its last line, or for a character page the `offsetChars` it also gives as `nextOffsetChars`, with the `limitChars` left of what it was asked for, so a reader that follows the notes ends where it asked to. Where even a page's first line would pass the limit, that line is cut shorter, at the most the page has room for. Where a page's last line is cut, its note names the `offsetChars` where that line's rest starts and the `limitChars` that reads it to the line's end, before the next line's `offset` if the file goes on. A page always moves on, so where its own fields (the path and its notes) and the least it can show pass a small limit, it passes the limit. At 1,000 characters, a page with no notes does that only for a path of about 680 characters or more. Its notes take room from that: with all three notes on a file's text and a conflict's, a path of 93 to 129 characters does it, and a note that names a symlink's target, as one saying disk couldn't be checked can, takes as much room as the target is long. Before, a page of up to 2,000 lines, or of `limitChars` characters, went out whole, and for a result that passed the limit the framework kept only its start, cut mid-JSON, writing the whole of it to a workspace file where the agent has one. A caller that pages 2,000 lines at a time and starts the next page at line 2,001 now starts where the note says. A running `code_execution` script's call has that larger limit, so its line pages still stop at the line limit, 2,000 by default, and its character pages at `limitChars`, unless a page would pass that. Its line pages cut a line over 2,000 characters and leave out CRs and a BOM, as a model's do, and a character page gives it the stored text exactly. A call made in process, through `handleToolCall`, `executeToolCall` or `ModuleContext.callTool`, carries no `resultMaxChars` unless its caller sets one. Its line pages stop at the line limit alone and show every line whole, as they did before, though now without the CRs and the BOM. +- A `read` page stops before its result would pass `resultMaxChars`, the most of it that can reach its caller whole. For a model's call that's its agent's inline cap, 24,000 characters by default, within which a result is stored whole unless its round's results together pass the readers' bound; for a running script's call it's what the script's memory takes, 7,456,527 characters at the default 128 MiB. Its note says where to continue: the `offset` after its last line, or for a character page the `offsetChars` it also gives as `nextOffsetChars`, with the `limitChars` left of what it was asked for, so a reader that follows the notes ends where it asked to. Where even a page's first line would pass the limit, that line is cut shorter, at the most the page has room for. Where a page's last line is cut, its note names the `offsetChars` where that line's rest starts and the `limitChars` that reads it to the line's end, before the next line's `offset` if the file goes on. A page always moves on, so where its own fields (the path and its notes) and the least it can show pass a small limit, it passes the limit. At 1,000 characters, a page with no notes does that only for a path of about 680 characters or more. Its notes take room from that: with all three notes on a file's text and a conflict's, a path of 93 to 129 characters does it, and a note that names a symlink's target, as one saying disk couldn't be checked can, takes as much room as the target is long. Before, a page of up to 2,000 lines, or of `limitChars` characters, went out whole, and for a result that passed the limit the framework kept only its start, cut mid-JSON, writing the whole of it to a workspace file where the agent has one. A caller that pages 2,000 lines at a time and starts the next page at line 2,001 now starts where the note says. A running `code_execution` script's call has that larger limit, so its line pages still stop at the line limit, 2,000 by default, and its character pages at `limitChars`, unless a page would pass that. Its line pages cut a line over 2,000 characters and leave out CRs and a BOM, as a model's do, and a character page gives it the stored text exactly. A call made in process, through `handleToolCall`, `executeToolCall` or `ModuleContext.callTool`, carries no `resultMaxChars` unless its caller sets one. Its line pages stop at the line limit alone and show every line whole, as they did before, though now without the CRs and the BOM. - The workspace's `grep` leaves out a CRLF end's CR, for matching as well as display, as `read` does, so `foo$` finds a CRLF file's line too, as it does under ripgrep's `--crlf`. A match's text and context are its lines as `read` shows them, but each line whole, where a `read` page for a call with a result limit cuts a line over 2,000 characters. A BOM isn't part of the first line, and a file with CRLF ends says so in its result's `lineEndings` (`CRLF`, or `mixed`). Before, a CRLF file's lines ended in a CR, so `foo$` missed them and a match's text copied into an edit carried the CR. A pattern written to find the CR of a CRLF line end no longer matches, except on a line whose own text ends in a CR, which keeps its line end's CR as `read` shows it. When a pattern names a CR, as a literal one or as `\r`, `\x0d`, `\u000d`, `\cM` or `\015`, and a file it searched has CRLF ends, the result's `note` says how many files searched have them and names them, each as `CRLF` or `mixed`, up to 20 and then how many more. It also says that a character `read` shows every CR in the workspace's copy, or in disk's file on a direct mount where the host has one. A pattern that names U+FEFF, over a file that starts with a BOM, gets a note that names the files that start with one and says a BOM isn't part of the first line. So a `grep` for `\r` or `^\uFEFF` still finds which of the files it searches have CRLF ends or a BOM. A CR that doesn't come before a line feed is still part of its line. - The workspace's `read` first takes in a change on disk to a file the workspace holds, as a listing and an `edit` do, and shows that text. Its note says so when disk's copy had changed since the workspace last agreed with it. It also says when disk's copy couldn't be checked, or disk has none and nothing records whether it was deleted there, so it shows the workspace's copy; where the check gave up because the selected branch kept changing, it says to read again. A file disk deleted answers that disk deleted it and the deletion was taken in, and a file disk made binary or too large answers that it's on disk but not stored. Where the path's stored copy is a draft and disk changed or deleted it too, the read shows the draft, and its note names the conflict and the ways to settle it, a sync, or a forced materialize on a mount that isn't read-only: for a deletion, a sync of the path takes it and removes the workspace's copy. Before, a read of a path the workspace held didn't look at disk, so it could show a copy another writer had since changed; a path it didn't hold was observed, as now. - A direct mount's `read` and `grep` show a file's text as a store-backed mount's do, with the same pages, cut lines, notes and `lineEndings`. Only the disk check is a store-backed mount's alone, since a direct mount reads disk as it is and keeps no copy of its own. diff --git a/src/modules/workspace/index.ts b/src/modules/workspace/index.ts index 696337853..33ac47be0 100644 --- a/src/modules/workspace/index.ts +++ b/src/modules/workspace/index.ts @@ -2037,7 +2037,7 @@ export class WorkspaceModule implements Module { return { success: false, isError: true, error: `The ${call.name} call did nothing: ${problem}.` }; } const input = call.input as Record; - // The most of this call's result that reaches its caller whole: past + // The most of this call's result that can reach its caller whole: past // it, a model's result spills into a file, and a script gets // ResultTooLarge in its place. A call made in process carries none and // takes the data itself, so its read pages by its line limit alone. @@ -3844,7 +3844,7 @@ export class WorkspaceModule implements Module { /** * A read's page, held to `maxChars`, the most of this call's result that - * reaches its caller whole. A call that carries none, made in process, + * can reach its caller whole. A call that carries none, made in process, * takes the data itself: its page holds to its line limit alone, with every * line whole, as it did before pages had a budget. */ From 8b46eac2a909accbacdcae4a7ac3b099dd406b04 Mon Sep 17 00:00:00 2001 From: Abigail-2676 Date: Thu, 22 Oct 2026 10:40:15 -0400 Subject: [PATCH 51/51] refactor(workspace): read decides that its pass decided nothing as the other file tools do main's undecidedOf reads a pass that decided nothing by its missing branchId, and delete, readBinary and the edit plan use it. read's disk check spelled the same case as a path with no report beside a region left incomplete. The two agree for every pass a read runs: a pass that decides nothing reports no path and leaves its reason as a region, and one that decides with a region incomplete, as under a moved or replaced mount root, still reports the path as unverified. One predicate keeps them from drifting apart. read's answers are unchanged. Co-Authored-By: Claude Opus 5.5 --- src/modules/workspace/index.ts | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/src/modules/workspace/index.ts b/src/modules/workspace/index.ts index 33ac47be0..1a67fc85f 100644 --- a/src/modules/workspace/index.ts +++ b/src/modules/workspace/index.ts @@ -3772,15 +3772,15 @@ export class WorkspaceModule implements Module { // path the mount passes over by design is observed, as a sync naming // it would observe it (#276). const pass = await this.passUnlocked(mount, { kind: 'paths', paths: [relativePath] }, { named: true }); - // A pass that gave up (the selected branch kept changing) decided - // nothing and leaves no report. It's read as a copy disk couldn't be + // A pass that decided nothing (undecidedOf: the selected branch kept + // changing) leaves no report. It's read as a copy disk couldn't be // checked, with its reason, so the answer says the check didn't finish. - const gaveUp = !pass.reports.has(relativePath) && pass.incomplete.length > 0; + const undecided = undecidedOf(pass); const report = pass.reports.get(relativePath) - ?? (gaveUp ? { path: relativePath, state: 'unverified' as const, note: pass.incomplete[0]!.reason } : undefined); - // A check that gave up may well finish on the next read, as grep's - // answer to the same says of grep. - const again = gaveUp ? ['Read again to check it.'] : []; + ?? (undecided !== null ? { path: relativePath, state: 'unverified' as const, note: undecided } : undefined); + // A check that decided nothing may well finish on the next read, as + // grep's answer to the same says of grep. + const again = undecided !== null ? ['Read again to check it.'] : []; // A mode taken in alone isn't a change to the text, so it leaves the // read nothing to say, as it leaves an edit nothing. const took = (op: TreeOp) => pass.ops.some((o) => o.path === relativePath && o.op === op && !o.modeOnly);