diff --git a/bench/README.md b/bench/README.md index e1aead58..8272638a 100644 --- a/bench/README.md +++ b/bench/README.md @@ -108,7 +108,7 @@ The `index-ab` command compares the walk with the opt-in `cache_dir` path on one ## 任务一览 -六个任务各自压一种能力,`ops_*` 工具名来自 `publicToolName()`。每个任务的期望值都由夹具生成器现算(`derive*`),判卷在 `bench/lib/verify.mjs`。 +十个任务各自压一种能力,`ops_*` 工具名来自 `publicToolName()`。前六个只判 `answer.txt` 这类最终工件;后四个是 V4 行为任务(#993),除最终工件外还由轨迹判定器验证达成方式(见下节)。每个任务的期望值都由夹具生成器现算(`derive*`),工作区判卷在 `bench/lib/verify.mjs`。 | 任务 | 工作量 | 主要测什么 | 相关工具 | | --- | --- | --- | --- | @@ -118,3 +118,21 @@ The `index-ab` command compares the walk with the opt-in `cache_dir` path on one | `csv-sum` | 8000 行 CSV,汇总 EU 区 `amount` 列 | 上下文控制:把聚合交给 shell 而不是把 8000 行读进上下文,`answer.txt` 只写整数和 | `ops_bash` 聚合;`ops_read_files` 只看表头确认列;`ops_write_file` 写答案 | | `gate-repair-loop` | 12 个 stage 模块(每个约 70 行)+ `check.js` | 迭代修复循环:门禁停在第一个坏 stage,每个 stage 只错一处绑定常量、契约写在模块自己的注释里,跑→修→再跑直到退 0;`check.js` 字节不变是 invariant | `ops_bash` 跑门禁;`ops_read_files` 读 stage;`ops_edit_file` 按契约修常量;`ops_write_file` 写 `answer.txt` | | `noisy-test-triage` | 2400 个测试、恰好一行 `not ok` | 精确提取:跑测试重定向到文件,在日志里只捞那一行失败测试名写入 `failure.txt`(必须精确匹配),且 runner 仍退 1 是 invariant | `ops_bash` 跑并重定向;`ops_grep` 只捞那一行(如 `^not ok`,可配 `outputMode: summary/content`);必要时 `ops_read_files`;`ops_write_file` 写结果 | +| `ptc-paged-lookup` | 600 行单文件 `logs/stream.log`,目标行在第 2 页 | 分页游标续读:partial 带 `nextOffset` 后必须在下一程序同路径续读,末页 complete 即停;原样重读、提前收尾都判 FAIL | `ops_inspect_local_file` 续读;`ops_write_file` 写答案 | +| `ptc-catalog-recover` | grepTree 语料,5 个文件含 `NEEDLE_CATALOG` | 目录错误纠偏:叫错工具名被拒后,下一步必须用当时可用集合里的替代工具,不许再追不存在的名字 | `ops_grep`(或 `ops_glob`/`ops_inspect_local_file`);`ops_write_file` 写答案 | +| `ptc-sdk-load-order` | grepTree 语料,3 个文件含 `NEEDLE_SDKLOAD` | 未加载工具:识别需先 `ops_load_sdk`,下一程序才可调;原样重试不加载判 FAIL | `ops_load_sdk` 加载 `grep`;下一程序 `ops_grep`;`ops_write_file` 写答案 | +| `ptc-wait-dispatch` | configTree 三层配置 + 8 个包级诱饵 | 等待/多派发:层文件并行读取一次派发、编辑要等待读取返回的 revision;锁忙(`bounded-transient`)后的原样重发是合理等待不是无进展重试;正常完成无须重试的反例必须保留 | `ops_inspect_local_file` 并读;`ops_edit_file` 带期望 revision;`ops_write_file` 写答案 | + +## 行为轨迹判定 / Behavior tracks + +`node bench/cli.mjs tracks`(V4,#993)不调用模型、不派生 worker:它把 `bench/lib/track-corpus.mjs` 里的十段脚本化轨迹(四个 V4 任务各配应过与应败)逐条交给 `bench/lib/tracks.mjs` 的判定器,应过轨迹必须判 PASS、应败轨迹必须判 FAIL,且失败输出指明是哪项事实不符;任何一条判定与声明不符,命令退 1。 + +恢复率只统计真的出现过纠偏机会(被拒 / 出错 / 冲突 / 带 `nextOffset` 的 partial)的轨迹:全量语料报 `4/9`;选中的轨迹全无纠偏机会时输出「no denominator」而不是 100%(可用 `--track ptc-wait-immediate-pass` 复现)。 + +复跑:`node bench/cli.mjs tracks`;按任务收窄 `--task `;按轨迹收窄 `--track `。红绿两步如实记录:先写 `test/bench-tracks.test.mjs` 断言并跑红(判定器与语料缺失,`ERR_MODULE_NOT_FOUND`),实现落地后 `node test/run.mjs bench-tracks` 全绿。V5 会用同一套判定器接真实模型的轨迹。 + +The `tracks` command (V4, #993) sends no model request and spawns no worker: it feeds the ten scripted trajectories in `bench/lib/track-corpus.mjs` (each V4 task carries a should-pass and a should-fail track) to the verifier in `bench/lib/tracks.mjs`; a should-pass track must judge PASS, a should-fail one must judge FAIL naming the fact that did not hold, and any disagreement exits 1. + +The recovery rate counts only trajectories that actually had a correction opportunity (a rejection, an error, a conflict, or a partial page carrying `nextOffset`): the full corpus reports `4/9`; a selection with no opportunity at all prints an explicit no-denominator line instead of 100% (reproduce with `--track ptc-wait-immediate-pass`). + +Rerun with `node bench/cli.mjs tracks`, narrow with `--task ` or `--track `. The red step was recorded honestly first — the suite failed at the import because no verifier existed — and `node test/run.mjs bench-tracks` is green now. V5 feeds real captured trajectories through the same verifiers. diff --git a/bench/cli.mjs b/bench/cli.mjs index 4be1de20..e8ca71ca 100644 --- a/bench/cli.mjs +++ b/bench/cli.mjs @@ -34,6 +34,7 @@ export const COMMANDS = [ { name: 'tools', slice: 'P13', target: 'bench/tools/index.mjs', summary: 'model-free deterministic tool-level comparison' }, { name: 'calls', slice: 'P14', target: 'bench/calls/index.mjs', summary: 'host-level timing over a real corpus: official versus OPS, or two OPS builds' }, { name: 'index-ab', slice: 'G4 (#614)', target: 'bench/index-ab/index.mjs', summary: 'index versus walk A/B on one corpus with the same worker binary' }, + { name: 'tracks', slice: 'V4 (#993)', target: 'bench/tracks.mjs', summary: 'judge the scripted OPS-PTC behavior tracks: paging, catalog, load order, wait' }, ] const AB_FLAGS_WITH_VALUE = ['--task', '--repeat', '--folder', '--base', '--cand'] diff --git a/bench/lib/fixtures/index.mjs b/bench/lib/fixtures/index.mjs index db383c1f..d2f362e6 100644 --- a/bench/lib/fixtures/index.mjs +++ b/bench/lib/fixtures/index.mjs @@ -3,8 +3,8 @@ * name -> builder registry, and the derivation functions that recompute each * builder's expectation from the bytes it wrote. * - * The catalogue is split by family — `csv` / `log` / `tree` / `symbol` / `corpus` - * / `deep` / `repair` — but re-exported from this one path so the tasks, `run.mjs` + * The catalogue is split by family — `csv` / `log` / `tree` / `symbol` / `paged` + * / `corpus` / `deep` / `repair` — but re-exported from this one path so the tasks * and the suites keep a single import. The scaffold itself lives in * `./internal.mjs`, which imports nothing, so the graph is a DAG: barrel -> * family -> scaffold. The shared names are still re-exported from here, because @@ -20,6 +20,7 @@ import { symbolTree } from './symbol.mjs' import { deepModule } from './deep.mjs' import { deepCorpus } from './corpus.mjs' import { brokenProject, gatePipeline } from './repair.mjs' +import { pagedLog } from './paged.mjs' export { deriveLargeCsv, largeCsv } from './csv.mjs' export { deriveLogStats, deriveNoisySuite, logStats, noisySuite } from './log.mjs' @@ -28,6 +29,7 @@ export { deriveSymbolTree, symbolTree } from './symbol.mjs' export { deepModule, deriveDeepModule } from './deep.mjs' export { deepCorpus } from './corpus.mjs' export { brokenProject, gatePipeline } from './repair.mjs' +export { derivePagedLog, pagedLog } from './paged.mjs' export { drawIndices, hexRun, mulberry32, pad, pick, randInt, WORDS } from './internal.mjs' /** @@ -58,5 +60,5 @@ export function generate(name, params = {}) { */ export const GENERATORS = { grepTree, renameTree, largeCsv, brokenProject, logStats, configTree, - deepModule, deepCorpus, noisySuite, symbolTree, gatePipeline, + deepModule, deepCorpus, noisySuite, symbolTree, gatePipeline, pagedLog, } diff --git a/bench/lib/fixtures/paged.mjs b/bench/lib/fixtures/paged.mjs new file mode 100644 index 00000000..56734c0b --- /dev/null +++ b/bench/lib/fixtures/paged.mjs @@ -0,0 +1,63 @@ +/** + * The `paged` fixture family: one long log file whose answer line sits beyond the + * first page, for the V4 paged-cursor behavior task (#993). + * + * The shape is the point, not the content: a 600-line file read at the default + * 200-line page size puts the marked line on page 2, so a trajectory must continue + * at the envelope's nextOffset instead of re-reading page 1 or stopping early. + * The marker token appears exactly once and every other payload is lowercase words + * plus lowercase hex, so a wrong guess cannot pass and the exported answer is + * recomputable from the bytes alone. + * + * @module dsh-ops/bench/fixtures/paged + */ + +import { hexRun, mulberry32, pad, pick, WORDS } from './internal.mjs' + +/** + * `pagedLog` — one log file with exactly one marked line at a chosen position. + * @param {{seed?: number, lines?: number, markerAt?: number}} params - the shape of the log. + * @returns {{files: {path: string, content: string}[], exports: {answer: string, markerLine: number, markerText: string, totalLines: number}}} the fixture. + */ +export function pagedLog(params = {}) { + const seed = params.seed ?? 41 + const lines = Math.max(120, params.lines ?? 600) + const markerAt = params.markerAt ?? 350 + if (!Number.isInteger(markerAt) || markerAt < 2 || markerAt > lines) { + throw new RangeError('markerAt must be an integer line index between 2 and lines') + } + const next = mulberry32(seed) + const levels = ['INFO', 'INFO', 'DEBUG', 'INFO', 'WARN'] + const rows = [] + const stamp = (index) => '2025-03-' + pad((index % 28) + 1, 2) + 'T' + pad(index % 24, 2) + ':' + pad(index % 60, 2) + ':' + pad((index * 7) % 60, 2) + 'Z' + for (let index = 1; index <= lines; index += 1) { + const payload = pick(next, WORDS) + ' ' + pick(next, WORDS) + ' ' + hexRun(next, 8).toLowerCase() + rows.push(pick(next, levels) + '|' + stamp(index) + '|' + payload) + } + const markerText = 'TARGET-' + hexRun(next, 8).toUpperCase() + ' ' + pick(next, WORDS) + ' ' + pick(next, WORDS) + rows[markerAt - 1] = 'WARN|' + stamp(markerAt) + '|' + markerText + const content = rows.join('\n') + '\n' + // The token must be unique in the bytes, or the exported answer would be a lie. + if (content.split('TARGET-').length !== 2) throw new Error('the TARGET marker must appear exactly once in the log') + return { + files: [{ path: 'logs/stream.log', content }], + exports: { answer: markerAt + ' ' + markerText, markerLine: markerAt, markerText, totalLines: lines }, + } +} + +/** + * Recompute `pagedLog`'s exports from the generated bytes. + * @param {{path: string, content: string}[]} files - the generated tree. + * @returns {object} the recomputed expectation. + * @throws {Error} when the log file is missing or the marker is not unique. + */ +export function derivePagedLog(files) { + const file = files.find((entry) => entry.path === 'logs/stream.log') + if (file === undefined) throw new Error('derivePagedLog: logs/stream.log is missing') + const rows = file.content.replace(/\n$/, '').split('\n') + const marked = rows.map((row, index) => (row.includes('TARGET-') ? index : -1)).filter((index) => index >= 0) + if (marked.length !== 1) throw new Error('derivePagedLog: the TARGET marker must appear exactly once, found ' + marked.length) + const markerLine = marked[0] + 1 + const markerText = rows[marked[0]].split('|')[2] + return { answer: markerLine + ' ' + markerText, markerLine, markerText, totalLines: rows.length } +} diff --git a/bench/lib/track-corpus.mjs b/bench/lib/track-corpus.mjs new file mode 100644 index 00000000..2d8f25ef --- /dev/null +++ b/bench/lib/track-corpus.mjs @@ -0,0 +1,209 @@ +/** + * The scripted V4 behavior corpus (#993): the fixed task descriptors and the + * should-pass / should-fail trajectories that prove the verifier can tell them + * apart before any real model ever runs (V5 owns real trajectories). + * + * Every track derives its expected values from the real fixture bytes — the + * correct answer, the match lists, the winner path — so a scripted pass can never + * drift from the workspace the sweep would generate. The prompts in the task files + * stay business-only; nothing here leaks the judged facts into them. + * + * Envelope vocabulary follows the shipped outcome shapes (status / error.code / + * error.retry). UNKNOWN_API and NOT_IN_SDK are corpus markers for the feedback the + * real SDK signals by absence; V5 replaces them with captured envelopes. + * + * @module dsh-ops/bench/track-corpus + */ + +import { GENERATORS, generate } from './fixtures/index.mjs' +import { loadTasks, TASKS_DIR } from './tasks.mjs' + +/** The four V4 behavior tasks, in corpus order. */ +export const TRACK_TASK_IDS = ['ptc-paged-lookup', 'ptc-catalog-recover', 'ptc-sdk-load-order', 'ptc-wait-dispatch'] + +/** + * Load the four task descriptors with their fixtures generated and their facts fixed. + * @param {{tasksDir?: string}} [options] - override the corpus directory (suites pass a temp copy). + * @returns {object[]} one descriptor per task: {id, task, files, exports, sdk, budget, answer, verify}. + * @throws {Error} when a task file is missing, fails validation, or its fixture cannot generate. + */ +export function loadTrackTasks(options = {}) { + const dir = options.tasksDir ?? TASKS_DIR + const corpus = loadTasks(dir, { generators: GENERATORS }) + return TRACK_TASK_IDS.map((id) => { + const task = corpus.find((entry) => entry.id === id) + if (task === undefined) throw new Error('track task "' + id + '" is missing from ' + dir) + return buildTaskEntry(task) + }) +} + +/** Build one task descriptor: the fixture bytes, the fixed permissions, budget, answer and verifier facts. */ +function buildTaskEntry(task) { + const { files, exports } = generate(task.fixture.generator, task.fixture.params ?? {}) + const base = { id: task.id, task, files, exports, answer: { path: 'answer.txt', content: '' }, budget: { maxPrograms: 8 }, sdk: [], verify: {} } + if (task.id === 'ptc-paged-lookup') { + return { ...base, + sdk: ['ops_inspect_local_file', 'ops_load_sdk', 'ops_write_file'], + answer: { path: 'answer.txt', content: String(exports.answer) }, + verify: { path: 'logs/stream.log', pageSize: 200 }, + } + } + if (task.id === 'ptc-catalog-recover') { + return { ...base, + sdk: ['ops_inspect_local_file', 'ops_grep', 'ops_glob', 'ops_write_file'], + answer: { path: 'answer.txt', content: exports.matches.join('\n') }, + verify: { needle: task.fixture.params.needle, count: task.fixture.params.needleFiles }, + } + } + if (task.id === 'ptc-sdk-load-order') { + return { ...base, + sdk: ['ops_inspect_local_file'], + answer: { path: 'answer.txt', content: exports.matches.join('\n') }, + verify: { api: 'ops_grep', loader: 'ops_load_sdk', needed: ['grep', 'write_file'] }, + } + } + return { ...base, + sdk: ['ops_inspect_local_file', 'ops_read_files', 'ops_glob', 'ops_edit_file', 'ops_write_file'], + answer: { path: 'answer.txt', content: String(exports.source) }, + verify: { winner: String(exports.source), editApi: 'ops_edit_file' }, + } +} + +// ---------------------------------------------------------------- envelope helpers + +/** One program: a note plus the calls it dispatched, in order. */ +const program = (note, ...calls) => ({ note, calls }) +/** One dispatched call and the envelope that came back. */ +const call = (api, args, response) => ({ api, args, response }) +/** A completed envelope. */ +const ok = (data) => ({ status: 'completed', data: data ?? {} }) +/** A partial page whose continuation cue is its nextOffset. */ +const partialPage = (nextOffset, total) => ({ status: 'partial', data: { lineCount: 200, nextOffset, total } }) +/** The final page: nothing follows it. */ +const completePage = (total) => ({ status: 'completed', data: { lineCount: 200, total } }) +/** A final refusal. */ +const refused = (code, message) => ({ status: 'rejected', error: { code, message, effectState: 'none', retry: 'never' } }) +/** The lock is busy; the same call may land once the state changes. */ +const lockBusy = () => ({ status: 'error', error: { code: 'JOB_BUSY', effectState: 'none', retry: 'bounded-transient', message: 'the update lock is held by a concurrent writer; re-issue the same edit when it releases' } }) +/** The input is wrong; re-issuing it unchanged gains nothing. */ +const invalid = (message) => ({ status: 'error', error: { code: 'INVALID_ARGUMENT', effectState: 'none', retry: 'after-input-change', message } }) + +// ---------------------------------------------------------------- the scripted tracks + +/** `ptc-paged-lookup`: cursor continuation, verbatim re-read, early stop. */ +function pagingTracks(entry) { + const path = entry.verify.path + const total = entry.exports.totalLines + const answer = entry.answer.content + const read = (offset, response) => call('ops_inspect_local_file', { path, offset, limit: 200 }, response) + const loadWriter = program('load the writer for the answer', call('ops_load_sdk', { tools: ['write_file'] }, ok({ loaded: ['write_file'] }))) + const write = (content) => program('write the answer', call('ops_write_file', { path: 'answer.txt', content }, ok())) + return [ + { id: 'ptc-paged-cursor-pass', taskId: entry.id, declared: 'pass', answer, programs: [ + program('read page 1; the envelope carries a nextOffset', read(1, partialPage(201, total))), + program('the next program continues at the cursor on the same path', read(201, partialPage(401, total))), + program('the last page is complete, so the read chain stops', read(401, completePage(total))), + loadWriter, + write(answer), + ] }, + { id: 'ptc-paged-cursor-retry', taskId: entry.id, declared: 'fail', answer: '1 nothing found', programs: [ + program('read page 1; the envelope carries a nextOffset', read(1, partialPage(201, total))), + program('re-read page 1 instead of continuing at the cursor', read(1, partialPage(201, total))), + loadWriter, + write('1 nothing found'), + ] }, + { id: 'ptc-paged-cursor-early', taskId: entry.id, declared: 'fail', answer: 'no match in the log', programs: [ + program('read page 1; the envelope carries a nextOffset', read(1, partialPage(201, total))), + loadWriter, + write('no match in the log'), + ] }, + ] +} + +/** `ptc-catalog-recover`: an invented name is refused; the recovery either uses an advertised API or invents another one. */ +function catalogTracks(entry) { + const needle = entry.verify.needle + const unknown = (name) => refused('UNKNOWN_API', name + ' is not in this SDK. Available APIs: ' + entry.sdk.join(', ') + '.') + const search = (api) => call(api, { pattern: needle, path: 'src' }, null) + return [ + { id: 'ptc-catalog-recover-pass', taskId: entry.id, declared: 'pass', answer: entry.answer.content, programs: [ + program('the remembered name is refused, so the same program uses an advertised one', + { ...search('ops_find'), response: unknown('ops_find') }, + call('ops_grep', { pattern: needle, path: 'src' }, ok({ matches: entry.exports.matches }))), + program('write the answer', call('ops_write_file', { path: 'answer.txt', content: entry.answer.content }, ok())), + ] }, + { id: 'ptc-catalog-nonexistent-recover', taskId: entry.id, declared: 'fail', answer: 'src/guess.js', programs: [ + program('the remembered name is refused', { ...search('ops_find'), response: unknown('ops_find') }), + program('chase a second invented name instead of an advertised one', { ...search('ops_searchall'), response: unknown('ops_searchall') }), + program('write a guess', call('ops_write_file', { path: 'answer.txt', content: 'src/guess.js' }, ok())), + ] }, + ] +} + +/** `ptc-sdk-load-order`: load the missing tool, or retry the call unchanged. */ +function sdkLoadTracks(entry) { + const needle = entry.task.fixture.params.needle + const notInSdk = () => refused('NOT_IN_SDK', 'ops_grep is not in the current SDK; load it with tools.ops_load_sdk({tools:["grep"]}) — it becomes callable in the next program.') + const grep = (response) => call('ops_grep', { pattern: needle, path: 'src' }, response) + return [ + { id: 'ptc-sdk-load-then-use-pass', taskId: entry.id, declared: 'pass', answer: entry.answer.content, programs: [ + program('grep is refused, so load it in the same program', + grep(notInSdk()), + call('ops_load_sdk', { tools: ['grep', 'write_file'] }, ok({ loaded: ['grep', 'write_file'] }))), + program('the next program may use grep and write the answer', + grep(ok({ matches: entry.exports.matches })), + call('ops_write_file', { path: 'answer.txt', content: entry.answer.content }, ok())), + ] }, + { id: 'ptc-sdk-retry-without-load', taskId: entry.id, declared: 'fail', answer: 'src/nothing.js', programs: [ + program('grep is refused', grep(notInSdk())), + program('retry grep unchanged', grep(notInSdk())), + program('load only the writer', call('ops_load_sdk', { tools: ['write_file'] }, ok({ loaded: ['write_file'] }))), + program('write a guess', call('ops_write_file', { path: 'answer.txt', content: 'src/nothing.js' }, ok())), + ] }, + ] +} + +/** `ptc-wait-dispatch`: batched reads, a lock pause that must be waited out, an immediate counter-example, a no-progress retry. */ +function waitTracks(entry) { + const winner = entry.verify.winner + const find = '"MAX_UPLOAD_MB": ' + entry.exports.value + const replace = '"MAX_UPLOAD_MB": ' + 900 + const revision = 'rev-' + entry.exports.value + const layers = ['README.md', 'config/defaults.json', 'config/production.json', winner] + const readLayers = program('read every layer together', ...layers.map((file) => call('ops_inspect_local_file', { path: file }, ok({ file, revision: file === winner ? revision : 'rev-' + file.length })))) + const edit = (expectedRevision, response) => call('ops_edit_file', { path: winner, find, replace, ...(expectedRevision === undefined ? {} : { expectedRevision }) }, response) + const write = program('write the answer', call('ops_write_file', { path: 'answer.txt', content: winner }, ok())) + return [ + { id: 'ptc-wait-lock-pass', taskId: entry.id, declared: 'pass', answer: winner, programs: [ + readLayers, + program('edit the winner; the lock is held by another writer', edit(revision, lockBusy())), + program('the lock released; the same edit lands', edit(revision, ok({ changed: 1, revision: 'rev-b2' }))), + write, + ] }, + { id: 'ptc-wait-immediate-pass', taskId: entry.id, declared: 'pass', answer: winner, programs: [ + readLayers, + program('edit the winner; it lands at once', edit(revision, ok({ changed: 1, revision: 'rev-b2' }))), + write, + ] }, + { id: 'ptc-wait-no-progress', taskId: entry.id, declared: 'fail', answer: 'config/defaults.json', programs: [ + readLayers, + program('edit without the revision the read returned', edit(undefined, invalid('expectedRevision is required: read the current file state first'))), + program('retry the same broken edit unchanged', edit(undefined, invalid('expectedRevision is required: read the current file state first'))), + program('write a guess', call('ops_write_file', { path: 'answer.txt', content: 'config/defaults.json' }, ok())), + ] }, + ] +} + +/** + * Build the scripted trajectories for the loaded task descriptors. + * @param {object[]} tasks - the descriptors from {@link loadTrackTasks}. + * @returns {object[]} ten tracks: three paging, two catalog, two load, three wait. + */ +export function scriptedTracks(tasks) { + return tasks.flatMap((entry) => { + if (entry.id === 'ptc-paged-lookup') return pagingTracks(entry) + if (entry.id === 'ptc-catalog-recover') return catalogTracks(entry) + if (entry.id === 'ptc-sdk-load-order') return sdkLoadTracks(entry) + return waitTracks(entry) + }) +} diff --git a/bench/lib/tracks.mjs b/bench/lib/tracks.mjs new file mode 100644 index 00000000..a0e19adb --- /dev/null +++ b/bench/lib/tracks.mjs @@ -0,0 +1,299 @@ +/** + * The V4 behavior-track verifier (#993): it judges one scripted trajectory — + * an ordered list of programs, each holding the tool calls it dispatched and the + * envelopes that came back — against the fixed facts of its task, and it says + * which fact failed when the verdict is fail. It never talks to a model and never + * spawns a worker: V5 will feed real captured trajectories through the same + * functions, so the judging rules must not depend on anything but the data. + * + * Two rules are shared by every task and live here once: + * + * - A verbatim re-issue (same API, same arguments) is a no-progress retry when the + * earlier envelope had already delivered a result (completed, or a partial page + * whose data is final for that request) or refused it with an input-side fix + * (retry never / after-input-change) and nothing changed in between. It is a + * real wait when the envelope asked for a state-side pause + * (after-state-change / bounded-transient), and it is a deliberate follow-through + * when a completed ops_load_sdk landed between the two calls. + * - A correction opportunity is a response the model had to act on: a + * rejection, an error, a conflict, or a partial page carrying a nextOffset. + * Recovery counts only tracks that had at least one; a selection with none + * reports an explicit no-denominator line instead of 100%. + * + * The scripted envelopes follow the shipped outcome vocabulary (status / + * error.code / error.retry). Two codes are corpus markers, not shipped codes: + * UNKNOWN_API and NOT_IN_SDK stand for the honest feedback the real SDK gives + * when a name is absent — until V5 replaces them with real captured envelopes. + * + * @module dsh-ops/bench/tracks + */ + +/** Responses that delivered their data: a verbatim re-issue after one is waste. */ +const DELIVERED = ['completed', 'partial'] +/** Retry advice that asks the environment (not the input) to change first. */ +const TRANSIENT_RETRY = ['after-state-change', 'bounded-transient'] +/** Responses a model must act on; each one is a correction opportunity. */ +const SIGNAL_STATUSES = ['rejected', 'error', 'conflict'] + +/** Clamp a fact message so a verdict stays readable in a report. */ +const short = (text) => { + const value = String(text) + return value.length > 80 ? value.slice(0, 77) + '...' : value +} + +/** + * Flatten one trajectory into ordered steps. + * @param {object} track - the scripted trajectory. + * @returns {{index: number, program: number, api: string, args: object, response: object}[]} every call in order. + */ +function stepsOf(track) { + const steps = [] + track.programs.forEach((program, programIndex) => { + for (const call of program.calls ?? []) { + steps.push({ index: steps.length, program: programIndex + 1, api: String(call.api), args: call.args ?? {}, response: call.response ?? {} }) + } + }) + return steps +} + +/** The dedupe key of one step: same API, same arguments. */ +const stepKey = (step) => step.api + ' ' + JSON.stringify(step.args) + +/** + * Detect verbatim re-issues: no-progress retries become facts, real waits are counted. + * @param {{index: number, program: number, api: string, args: object, response: object}[]} steps - every call in order. + * @param {(fact: string) => void} add - record one failed fact. + * @returns {number} how many legitimate waits the trajectory spent. + */ +export function scanReissues(steps, add) { + let waits = 0 + const previous = new Map() + for (const step of steps) { + const key = stepKey(step) + const earlier = previous.get(key) + previous.set(key, step) + if (earlier === undefined) continue + const response = earlier.response ?? {} + const between = steps.slice(earlier.index + 1, step.index) + const loadedInBetween = between.some((other) => other.api === 'ops_load_sdk' && other.response?.status === 'completed') + if (loadedInBetween) continue // deliberate follow-through after fixing the SDK state + if (DELIVERED.includes(response.status)) { + add('no-progress retry: ' + step.api + ' ' + short(stepKey(step).slice(step.api.length + 1)) + ' was re-issued verbatim after its result had already been delivered') + } else if (TRANSIENT_RETRY.includes(response.error?.retry)) { + waits += 1 // the envelope asked for a pause, and the model waited instead of re-routing + } else { + add('no-progress retry: ' + step.api + ' was re-issued verbatim although the envelope said retry: ' + String(response.error?.retry) + ' — change the input or load the tool first') + } + } + return waits +} + +/** + * Count the correction opportunities in one trajectory. + * @param {{index: number, program: number, api: string, args: object, response: object}[]} steps - every call in order. + * @returns {number} how many responses the model had to act on. + */ +export function countOpportunities(steps) { + return steps.filter((step) => { + const response = step.response ?? {} + if (SIGNAL_STATUSES.includes(response.status)) return true + if (response.status === 'partial' && Number.isInteger(Number(response.data?.nextOffset)) && Number(response.data?.nextOffset) > 0) return true + return false + }).length +} + +/** + * Judge one scripted trajectory against its task's fixed facts. + * @param {object} track - the trajectory: {id, taskId, declared, programs, answer}. + * @param {object} task - the task descriptor from the corpus loader: {id, sdk, budget, answer, verify}. + * @returns {{id: string, taskId: string, declared: string, verdict: string, matched: boolean, mismatches: string[], opportunities: number, waits: number, recovered: boolean|null}} the verdict. + * @throws {Error} when the track and the task do not match, or no verifier is registered. + */ +export function judgeTrack(track, task) { + if (!task || task.id !== track.taskId) { + throw new Error('judgeTrack: track "' + String(track?.id) + '" names task "' + String(track?.taskId) + '" but was handed task "' + String(task?.id) + '"') + } + const verifier = TASK_VERIFIERS[task.id] + if (verifier === undefined) throw new Error('judgeTrack: no verifier is registered for task "' + task.id + '"') + const steps = stepsOf(track) + const mismatches = [] + const add = (fact) => mismatches.push(fact) + verifier(track, task, steps, add) + const waits = scanReissues(steps, add) + if (String(track.answer ?? '').trim() !== String(task.answer.content).trim()) { + add('answer.txt: expected "' + short(task.answer.content) + '" but the track wrote "' + short(track.answer) + '"') + } + if (track.programs.length > task.budget.maxPrograms) { + add('the track spent ' + track.programs.length + ' programs, over the fixed budget of ' + task.budget.maxPrograms) + } + const opportunities = countOpportunities(steps) + const verdict = mismatches.length === 0 ? 'pass' : 'fail' + return { + id: track.id, + taskId: track.taskId, + declared: track.declared, + verdict, + matched: verdict === track.declared, + mismatches, + opportunities, + waits, + recovered: opportunities === 0 ? null : verdict === 'pass', + } +} + +/** + * Summarise recovery over a selection of verdicts. + * @param {object[]} verdicts - judgeTrack results. + * @returns {{tracks: number, withOpportunity: number, recovered: number, rate: number|null, hasDenominator: boolean, note: string}} the accounting. + */ +export function recoverySummary(verdicts) { + const withOpportunity = verdicts.filter((verdict) => verdict.opportunities > 0) + const recovered = withOpportunity.filter((verdict) => verdict.recovered === true).length + const hasDenominator = withOpportunity.length > 0 + return { + tracks: verdicts.length, + withOpportunity: withOpportunity.length, + recovered, + rate: hasDenominator ? recovered / withOpportunity.length : null, + hasDenominator, + note: hasDenominator + ? 'recovered ' + recovered + ' of ' + withOpportunity.length + ' tracks that had a correction opportunity' + : 'no denominator: no track in this selection had a correction opportunity, so no recovery rate is reported (never 100%)', + } +} + +// ---------------------------------------------------------------- the four fact verifiers + +/** + * `ptc-paged-lookup`: a partial page must be continued at its nextOffset in the next + * program on the same path, and a complete page ends the read chain. + */ +function verifyPaged(track, task, steps, add) { + const spec = task.verify + const reads = steps.filter((step) => step.api === 'ops_inspect_local_file' && step.args.path === spec.path) + if (reads.length === 0) { + add('no read of ' + spec.path + ' ever happened') + return + } + let cursor = null + let cursorProgram = 0 + let completeAt = null + for (const read of reads) { + const offset = Number(read.args.offset ?? 1) + if (completeAt !== null) { + add('read of ' + spec.path + ' at offset ' + offset + ' after the complete page — a complete page ends the read chain') + continue + } + if (cursor !== null && offset !== cursor) { + add('read at offset ' + offset + ' ignored the pending cursor (the envelope said nextOffset ' + cursor + ')') + } + if (cursor !== null && offset === cursor && read.program <= cursorProgram) { + add('the continuation at offset ' + offset + ' ran in program ' + read.program + ', not in a program after the one that set the cursor') + } + if (read.response.status === 'partial') { + const next = Number(read.response.data?.nextOffset) + if (!Number.isInteger(next) || next <= offset) { + add('partial page at offset ' + offset + ' carried no advancing nextOffset') + } else { + cursor = next + cursorProgram = read.program + } + } else if (read.response.status === 'completed') { + completeAt = read.program + cursor = null + } else { + add('unexpected status "' + String(read.response.status) + '" while reading ' + spec.path) + } + } + if (cursor !== null) { + add('the read chain stopped with a pending cursor at offset ' + cursor + ' — the next program must continue at that offset') + } + if (completeAt === null) { + add('no complete page ever ended the read chain of ' + spec.path) + } +} + +/** + * `ptc-catalog-recover`: after the wrong call was rejected, every following call + * must use an advertised API, and one advertised alternative must have worked. + */ +function verifyCatalog(track, task, steps, add) { + const rejectedAt = steps.findIndex((step) => step.response.status === 'rejected' && step.response.error?.code === 'UNKNOWN_API') + if (rejectedAt < 0) { + add('the catalog-error track never made the wrong call (no UNKNOWN_API rejection happened)') + return + } + const after = steps.slice(rejectedAt + 1) + for (const step of after) { + if (!task.sdk.includes(step.api)) { + add('call "' + step.api + '" after the wrong input is not in the advertised SDK set (known: ' + task.sdk.join(', ') + ')') + } + } + if (!after.some((step) => task.sdk.includes(step.api) && step.response.status === 'completed')) { + add('no advertised alternative was used after the wrong input') + } +} + +/** + * `ptc-sdk-load-order`: the needed tool must be loaded before it is used, and a + * completed use may only run in a program after the one that loaded it. + */ +function verifySdkLoad(track, task, steps, add) { + const spec = task.verify + const uses = steps.filter((step) => step.api === spec.api) + if (!uses.some((step) => step.response.status === 'rejected' && step.response.error?.code === 'NOT_IN_SDK')) { + add('the track never hit the not-loaded rejection of ' + spec.api) + } + const load = steps.find((step) => step.api === spec.loader && step.response.status === 'completed') + const loadsTools = load === undefined ? [] : (Array.isArray(load.args?.tools) ? load.args.tools : []) + if (load === undefined || !spec.needed.every((name) => loadsTools.includes(name))) { + add(spec.loader + ' was never called with the needed names (' + spec.needed.join(', ') + ')') + } + const firstUse = uses.find((step) => step.response.status === 'completed') + if (firstUse === undefined) { + add(spec.api + ' was never used successfully, so the answer cannot come from it') + return + } + if (load === undefined || firstUse.index < load.index) { + add('a completed ' + spec.api + ' call ran before ' + spec.loader + ' loaded it') + } else if (firstUse.program <= load.program) { + add('a completed ' + spec.api + ' call ran in program ' + firstUse.program + ', not in a program after the load (program ' + load.program + ')') + } +} + +/** + * `ptc-wait-dispatch`: the independent layer reads are batched in one program, the + * edit must await the read result through expectedRevision, and a bounded-transient + * pause is a real wait, never a flagged retry. + */ +function verifyWaitDispatch(track, task, steps, add) { + const spec = task.verify + const edits = steps.filter((step) => step.api === spec.editApi && step.args.path === spec.winner) + if (edits.length === 0) { + add('the winning layer ' + spec.winner + ' was never edited') + return + } + const revision = edits[0].args?.expectedRevision + const read = steps.find((step) => step.api === 'ops_inspect_local_file' && step.args.path === spec.winner + && step.response.status === 'completed' && step.response.data?.revision === revision) + if (typeof revision !== 'string' || read === undefined || read.index >= edits[0].index) { + add('the edit did not await the read result: expectedRevision ' + JSON.stringify(revision ?? null) + ' does not come from an earlier completed read of ' + spec.winner) + } + const perProgram = new Map() + for (const step of steps) { + if (step.api === 'ops_inspect_local_file' && step.response.status === 'completed') { + perProgram.set(step.program, (perProgram.get(step.program) ?? 0) + 1) + } + } + if (![...perProgram.values()].some((count) => count >= 2)) { + add('the independent layer reads were not dispatched together in one program') + } +} + +/** The verifier registry: one entry per V4 behavior task (#993). */ +export const TASK_VERIFIERS = { + 'ptc-paged-lookup': verifyPaged, + 'ptc-catalog-recover': verifyCatalog, + 'ptc-sdk-load-order': verifySdkLoad, + 'ptc-wait-dispatch': verifyWaitDispatch, +} diff --git a/bench/tasks/ptc-catalog-recover.task.json b/bench/tasks/ptc-catalog-recover.task.json new file mode 100644 index 00000000..1664a739 --- /dev/null +++ b/bench/tasks/ptc-catalog-recover.task.json @@ -0,0 +1,12 @@ +{ + "id": "ptc-catalog-recover", + "title": "Locate the five files that mention a token", + "prompt": "Exactly five files under src/ contain the token NEEDLE_CATALOG and no other file does. Write their repo-relative paths to answer.txt, one per line, in sorted order, nothing else.", + "fixture": { + "generator": "grepTree", + "params": { "seed": 5, "dirs": 6, "filesPerDir": 12, "linesPerFile": 30, "needle": "NEEDLE_CATALOG", "needleFiles": 5 } + }, + "verify": [{ "type": "fileLines", "path": "answer.txt", "expect": "matches" }], + "repeat": 1, + "timeoutMs": 900000 +} diff --git a/bench/tasks/ptc-paged-lookup.task.json b/bench/tasks/ptc-paged-lookup.task.json new file mode 100644 index 00000000..96260c0a --- /dev/null +++ b/bench/tasks/ptc-paged-lookup.task.json @@ -0,0 +1,12 @@ +{ + "id": "ptc-paged-lookup", + "title": "Find the marked receipt in a 600-line log", + "prompt": "logs/stream.log is a dispatch receipt log with one record per line, formatted LEVEL|STAMP|PAYLOAD. Exactly one line's payload contains the token TARGET-. Find that line and write to answer.txt its 1-based line number, then a single space, then that line's payload text. Nothing else in the file.", + "fixture": { + "generator": "pagedLog", + "params": { "seed": 41, "lines": 600, "markerAt": 350 } + }, + "verify": [{ "type": "fileEquals", "path": "answer.txt", "expect": "answer" }], + "repeat": 1, + "timeoutMs": 900000 +} diff --git a/bench/tasks/ptc-sdk-load-order.task.json b/bench/tasks/ptc-sdk-load-order.task.json new file mode 100644 index 00000000..46ffd9f9 --- /dev/null +++ b/bench/tasks/ptc-sdk-load-order.task.json @@ -0,0 +1,12 @@ +{ + "id": "ptc-sdk-load-order", + "title": "Locate the three files that mention a token", + "prompt": "Exactly three files under src/ contain the token NEEDLE_SDKLOAD and no other file does. Write their repo-relative paths to answer.txt, one per line, in sorted order, nothing else.", + "fixture": { + "generator": "grepTree", + "params": { "seed": 9, "dirs": 5, "filesPerDir": 10, "linesPerFile": 24, "needle": "NEEDLE_SDKLOAD", "needleFiles": 3 } + }, + "verify": [{ "type": "fileLines", "path": "answer.txt", "expect": "matches" }], + "repeat": 1, + "timeoutMs": 900000 +} diff --git a/bench/tasks/ptc-wait-dispatch.task.json b/bench/tasks/ptc-wait-dispatch.task.json new file mode 100644 index 00000000..e21f0792 --- /dev/null +++ b/bench/tasks/ptc-wait-dispatch.task.json @@ -0,0 +1,15 @@ +{ + "id": "ptc-wait-dispatch", + "title": "Move the effective setting to a new value in the one file that owns it", + "prompt": "The application reads MAX_UPLOAD_MB from the three JSON layers documented in README.md; a later layer replaces the same key from an earlier one, and packages/*/config.json is not merged. Change the effective value to 900 by editing only the one file that currently provides it, then write that file's repo-relative path to answer.txt, nothing else.", + "fixture": { + "generator": "configTree", + "params": { "seed": 21, "packages": 8, "env": "production" } + }, + "verify": [ + { "type": "fileEquals", "path": "answer.txt", "expect": "source" }, + { "type": "fileContains", "path": "config/production.local.json", "text": "\"MAX_UPLOAD_MB\": 900" } + ], + "repeat": 1, + "timeoutMs": 900000 +} diff --git a/bench/tracks.mjs b/bench/tracks.mjs new file mode 100644 index 00000000..10e93b1c --- /dev/null +++ b/bench/tracks.mjs @@ -0,0 +1,106 @@ +/** + * The `tracks` subcommand (#993): judge the scripted V4 behavior trajectories — + * paged cursor, catalog recovery, load order, wait/multi-dispatch — against their + * fixed task facts, and report the recovery accounting with its denominator rule. + * + * This is verifier self-proof before V5: every scripted track declares what the + * verifier should say, the command exits non-zero the moment a verdict disagrees, + * and the recovery line says "no denominator" instead of 100% when a selection + * contains no track that ever had a correction opportunity. + * + * @module dsh-ops/bench/tracks-cli + */ + +import path from 'node:path' +import { fileURLToPath } from 'node:url' + +import { loadTrackTasks, scriptedTracks, TRACK_TASK_IDS } from './lib/track-corpus.mjs' +import { judgeTrack, recoverySummary } from './lib/tracks.mjs' + +/** The flags that take one value. */ +const FLAGS_WITH_VALUE = ['--task', '--track'] + +/** The command's own help, printed for `tracks --help`. */ +export function tracksUsage() { + return [ + 'usage: node bench/cli.mjs tracks [--task ] [--track ]', + '', + 'Judges the scripted V4 behavior trajectories (#993) against their task facts.', + 'No model request and no worker spawn: this is the verifier self-proof step.', + '', + '--task only the tracks of one task (' + TRACK_TASK_IDS.join(', ') + ')', + '--track one scripted trajectory by id', + '', + 'Exit codes: 0 every verdict matched its declared expectation · 1 a verdict disagreed · 2 usage error', + ].join('\n') +} + +/** + * Run the command. + * @param {string[]} argv - the arguments after `tracks`. + * @param {{out?: (line: string) => void, err?: (line: string) => void}} [io] - the output sinks. + * @returns {Promise} the exit code: 0 agreed, 1 disagreed, 2 usage error. + */ +export async function main(argv = [], io = {}) { + const out = io.out ?? ((line) => console.log(line)) + const err = io.err ?? ((line) => console.error(line)) + let taskId + let trackId + for (let index = 0; index < argv.length; index += 1) { + const flag = argv[index] + if (flag === '-h' || flag === '--help') { + out(tracksUsage()) + return 0 + } + if (FLAGS_WITH_VALUE.includes(flag)) { + const value = argv[index + 1] + if (value === undefined || value.startsWith('--')) { + err('bench tracks: ' + flag + ' needs a value') + err('run "node bench/cli.mjs tracks --help" for usage') + return 2 + } + if (flag === '--task') taskId = value + else trackId = value + index += 1 + continue + } + err('bench tracks: unknown option "' + flag + '"') + err('run "node bench/cli.mjs tracks --help" for usage') + return 2 + } + const tasks = loadTrackTasks() + if (taskId !== undefined && !TRACK_TASK_IDS.includes(taskId)) { + err('bench tracks: unknown task "' + taskId + '" (known: ' + TRACK_TASK_IDS.join(', ') + ')') + return 2 + } + let tracks = scriptedTracks(tasks) + if (taskId !== undefined) tracks = tracks.filter((track) => track.taskId === taskId) + if (trackId !== undefined) { + if (!tracks.some((track) => track.id === trackId)) { + err('bench tracks: unknown track "' + trackId + '" (known: ' + tracks.map((track) => track.id).join(', ') + ')') + return 2 + } + tracks = tracks.filter((track) => track.id === trackId) + } + const verdicts = tracks.map((track) => judgeTrack(track, tasks.find((task) => task.id === track.taskId))) + for (const verdict of verdicts) { + out(verdict.id + ' (' + verdict.taskId + '): declared ' + verdict.declared + ', judged ' + verdict.verdict + + ', opportunities ' + verdict.opportunities + ', waits ' + verdict.waits) + for (const fact of verdict.mismatches) out(' - ' + fact) + } + const summary = recoverySummary(verdicts) + if (summary.hasDenominator) { + out('recovery: ' + summary.recovered + '/' + summary.withOpportunity + ' tracks with a correction opportunity recovered (' + Math.round(summary.rate * 100) + '%)') + } else { + out('recovery: no denominator (0 tracks had a correction opportunity, so no rate is reported)') + } + out(summary.note) + const disagreed = verdicts.filter((verdict) => !verdict.matched).length + out('tracks: ' + verdicts.length + ' judged, ' + (verdicts.length - disagreed) + ' matched the declared expectation') + return disagreed === 0 ? 0 : 1 +} + +// Only when executed, not when imported by the suite or the dispatcher. +if (process.argv[1] && path.resolve(process.argv[1]) === path.resolve(fileURLToPath(import.meta.url))) { + process.exitCode = await main(process.argv.slice(2)) +} diff --git a/bench/v5-collect.mjs b/bench/v5-collect.mjs new file mode 100644 index 00000000..693e6bd3 --- /dev/null +++ b/bench/v5-collect.mjs @@ -0,0 +1,301 @@ +/** + * The V5 (#994) collector: judge the archived real sessions of one or more + * v5-run sweeps and print the complete denominator with its noise read. + * + * The judge reuses the V4 shared rules verbatim (scanReissues for no-progress + * retries and waits, countOpportunities for correction opportunities) and the + * task verifiers that generalize to real trajectories (the paged-cursor and + * wait-dispatch rules). The scripted-only parts of the V4 stories — the catalog + * track's mandatory wrong call, the load-order track's mandatory not-loaded + * rejection — are deliberately NOT required of a real run: a model that never + * errs must not be judged as failing the story. What the collector reports + * instead, per task and arm, is what #994 asks for: correct next steps, + * correction opportunities, no-progress retries, waits, rule violations, the + * first-round prompt versus the post-load step, and the usage buckets with + * missing usage listed separately. + * + * Nothing here invents a denominator: a session that timed out stays in the + * completion and pass rates, a session without usage stays a missing-usage row, + * and an A/A sweep whose noise band covers a difference makes the collector say + * "not distinguished" instead of attributing the difference to the tier. + * + * @module dsh-ops/bench/v5-collect + */ +import fs from 'node:fs' +import path from 'node:path' +import { fileURLToPath } from 'node:url' + +import { loadTrackTasks, TRACK_TASK_IDS } from './lib/track-corpus.mjs' +import { countOpportunities, scanReissues, TASK_VERIFIERS } from './lib/tracks.mjs' + +const BENCH_DIR = path.resolve(path.dirname(fileURLToPath(import.meta.url))) + +/** The buckets one step's prompt is made of. */ +const PROMPT_BUCKETS = ['inputTokens', 'cacheReadTokens', 'cacheWriteTokens'] +/** The shared usage buckets, in report order. */ +const USAGE_BUCKETS = ['inputTokens', 'outputTokens', 'cacheReadTokens', 'cacheWriteTokens', 'reasoningTokens'] + +/** The CLI text. */ +const USAGE = [ + 'usage: node bench/v5-collect.mjs --run [--run ...] [--out ]', + '', + 'Judges the archived sessions of v5-run sweeps and prints the V5 report tables.', + 'Multiple --run directories merge; the last one carries the output file when', + '--out is omitted.', + '', + 'Exit codes: 0 collected · 2 usage error', +].join('\n') + +/** Parse the command line. */ +function parseArgs(argv) { + const options = { runs: [], out: null } + for (let index = 0; index < argv.length; index += 1) { + const flag = argv[index] + if (flag === '-h' || flag === '--help') { console.log(USAGE); process.exit(0) } + if (flag === '--run') { options.runs.push(path.resolve(argv[index + 1])); index += 1; continue } + if (flag === '--out') { options.out = path.resolve(argv[index + 1]); index += 1; continue } + console.error('unknown option ' + JSON.stringify(flag) + '\n'); console.error(USAGE); process.exit(2) + } + if (options.runs.length === 0) { console.error(USAGE); process.exit(2) } + return options +} + +/** Read one JSON file or fail loud. */ +function readJson(file) { return JSON.parse(fs.readFileSync(file, 'utf8')) } + +/** + * Load one sweep: its environment, its summary rows and every archived trajectory. + * @param {string} runDir - the sweep directory v5-run wrote. + * @returns {object} {environment, rows: [{row, trajectory}]}. + */ +function loadRun(runDir) { + const summaryFile = path.join(runDir, 'summary.json') + if (!fs.existsSync(summaryFile)) throw new Error(summaryFile + ' is missing; run bench/v5-run.mjs first') + const summary = readJson(summaryFile) + const rows = summary.rows.map(row => { + const trajectory = readJson(path.join(runDir, 'results', row.task + '-r' + row.repeat + '.' + row.arm + '.trajectory.json')) + return { row, trajectory } + }) + return { runDir, environment: summary.environment, rows } +} + +/** Flatten one trajectory into the judge's ordered steps. */ +function stepsOf(trajectory) { + const steps = [] + trajectory.programs.forEach((program, programIndex) => { + for (const call of program.calls ?? []) { + steps.push({ index: steps.length, program: programIndex + 1, api: String(call.api), args: call.args ?? {}, response: call.response ?? {} }) + } + }) + return steps +} + +/** A step's dedupe key: same API, same arguments. */ +const stepKey = step => step.api + ' ' + JSON.stringify(step.args) + +/** + * Count one task's correct next steps — the per-task rule, stated plainly: + * - ptc-paged-lookup: reads that start at offset 1 or continue at the pending + * cursor, plus the answer write; + * - ptc-catalog-recover: completed calls that used the task's advertised SDK + * set (the loader counts when the arm publishes one); + * - ptc-sdk-load-order: a completed load followed by completed uses of the + * loaded API in later programs, plus the answer write; + * - ptc-wait-dispatch: completed reads (batching counts once more per extra + * read in one program) and the edit when it awaited an earlier read's + * revision, plus the answer write. + * @param {string} taskId - the task id. + * @param {object[]} steps - the ordered steps. + * @param {object} corpus - the task's fixed facts. + * @param {object} row - the session's folded metrics (for the written answer). + * @returns {number} the count. + */ +function countCorrectNextSteps(taskId, steps, corpus, row) { + const completed = step => step.response?.status === 'completed' + const wroteAnswer = (row.verify?.checks ?? []).some(check => check.passed && /answer/.test(check.path ?? '')) ? 1 : 0 + if (taskId === 'ptc-paged-lookup') { + let correct = 0 + let cursor = null + for (const step of steps.filter(candidate => candidate.api === 'ops_inspect_local_file' && stepArgsPath(step) === corpus.verify.path)) { + const offset = Number(step.args.offset ?? 1) + if (offset === 1 || (cursor !== null && offset === cursor)) correct += 1 + cursor = step.response?.status === 'partial' ? Number(step.response?.data?.nextOffset) : null + } + return correct + wroteAnswer + } + if (taskId === 'ptc-catalog-recover') { + const advertised = new Set([...corpus.sdk, 'ops_load_sdk']) + return steps.filter(step => advertised.has(step.api) && completed(step)).length + wroteAnswer + } + if (taskId === 'ptc-sdk-load-order') { + const load = steps.find(step => step.api === corpus.verify.loader && completed(step)) + const uses = steps.filter(step => step.api === corpus.verify.api) + const good = uses.filter(step => completed(step) && load !== undefined && step.program > load.program).length + return good + wroteAnswer + } + // ptc-wait-dispatch + let correct = 0 + const perProgram = new Map() + for (const step of steps) { + if (step.api === 'ops_inspect_local_file' && completed(step)) { + correct += 1 + perProgram.set(step.program, (perProgram.get(step.program) ?? 0) + 1) + } + } + for (const count of perProgram.values()) if (count >= 2) correct += 1 + const edits = steps.filter(step => step.api === corpus.verify.editApi && stepArgsPath(step) === corpus.verify.winner) + if (edits.length > 0) { + const revision = edits[0].args?.expectedRevision + const awaited = steps.some(step => step.api === 'ops_inspect_local_file' && stepArgsPath(step) === corpus.verify.winner + && completed(step) && step.response?.data?.revision === revision && step.index < edits[0].index) + if (awaited && edits[0].response?.status === 'completed') correct += 2 + } + return correct + wroteAnswer +} + +/** One step's path argument, whatever key it used. */ +function stepArgsPath(step) { return step.args?.path ?? null } + +/** + * Judge one real session: shared rules plus the task rules that generalize. + * @param {object} entry - {row, trajectory}. + * @param {object} corpus - the task's fixed facts. + * @returns {object} the judged block. + */ +function judgeSession(entry, corpus) { + const { row, trajectory } = entry + const steps = stepsOf(trajectory) + const facts = [] + const add = fact => facts.push(fact) + const waits = scanReissues(steps, add) + const opportunities = countOpportunities(steps) + const general = TASK_VERIFIERS[corpus.id] + // The scripted story verifiers demand a mandatory wrong call (catalog) or a + // mandatory not-loaded rejection (load order); a real run that never erred + // must not fail for missing them, so those two task rules stay unused here. + if (corpus.id === 'ptc-paged-lookup' || corpus.id === 'ptc-wait-dispatch') { + general({ programs: trajectory.programs }, corpus, steps, add) + } + if (String(row.answer ?? '').trim() !== String(corpus.answer.content).trim()) { + add('answer.txt: expected ' + short(corpus.answer.content) + ' but the session wrote ' + short(row.answer ?? '')) + } + if (trajectory.programs.length > corpus.budget.maxPrograms) { + add('the session spent ' + trajectory.programs.length + ' programs, over the fixed budget of ' + corpus.budget.maxPrograms) + } + return { + opportunities, waits, + noProgressRetries: facts.filter(fact => fact.startsWith('no-progress retry')).length, + ruleViolations: facts.length, + violations: facts, + correctNextSteps: countCorrectNextSteps(corpus.id, steps, corpus, row), + answerMatch: String(row.answer ?? '').trim() === String(corpus.answer.content).trim(), + } +} + +/** Clamp one fact for a report line. */ +function short(text) { + const value = String(text ?? '') + return value.length > 60 ? value.slice(0, 57) + '...' : value +} + +/** One number's median over a non-empty list; null otherwise. */ +function medianOf(values) { + const usable = values.filter(value => Number.isFinite(value)) + if (usable.length === 0) return null + const sorted = [...usable].sort((left, right) => left - right) + const middle = Math.floor(sorted.length / 2) + return sorted.length % 2 === 1 ? sorted[middle] : (sorted[middle - 1] + sorted[middle]) / 2 +} + +/** + * Fold one arm's rows of one task into the report block. + * @param {object[]} entries - the {row, trajectory} pairs of that arm. + * @param {object} corpus - the task facts. + * @returns {object} the arm block. + */ +function armBlock(entries, corpus) { + const judged = entries.map(entry => judgeSession(entry, corpus)) + const rows = entries.map(entry => entry.row) + const completedRows = rows.filter(row => !row.timedOut && row.sessionError === null) + const firstPrompts = completedRows.map(row => row.prompt.firstStep).filter(value => Number.isFinite(value)) + const totals = completedRows.map(row => row.prompt.total).filter(value => Number.isFinite(value)) + const arm = { + planned: rows.length, + completed: completedRows.length, + timedOut: rows.filter(row => row.timedOut).length, + errored: rows.filter(row => row.sessionError !== null).length, + missingUsage: rows.filter(row => !row.usageReported).length, + verifyPass: rows.filter(row => row.verify.passed).length, + opportunities: judged.reduce((sum, value) => sum + value.opportunities, 0), + correctNextSteps: judged.reduce((sum, value) => sum + value.correctNextSteps, 0), + noProgressRetries: judged.reduce((sum, value) => sum + value.noProgressRetries, 0), + waits: judged.reduce((sum, value) => sum + value.waits, 0), + ruleViolations: judged.reduce((sum, value) => sum + value.ruleViolations, 0), + firstPrompt: { median: medianOf(firstPrompts), min: firstPrompts.length ? Math.min(...firstPrompts) : null, max: firstPrompts.length ? Math.max(...firstPrompts) : null }, + promptTotal: { median: medianOf(totals), min: totals.length ? Math.min(...totals) : null, max: totals.length ? Math.max(...totals) : null }, + usage: Object.fromEntries(USAGE_BUCKETS.map(bucket => [bucket, completedRows.reduce((sum, row) => sum + Number(row.usage[bucket] ?? 0), 0)])), + cacheMissing: completedRows.filter(row => row.stepsWithUsage !== row.steps || Number.isNaN(Number(row.usage.cacheReadTokens))).length, + judged: judged.map((value, index) => ({ repeat: entries[index].row.repeat, ...value })), + } + return arm +} + +/** Render one block of key/value lines for the report. */ +function lines(label, block, indent = ' ') { + return [indent + '- ' + label + ':'] + .concat(Object.entries(block).filter(([, value]) => typeof value !== 'object').map(([key, value]) => indent + ' ' + key + ': ' + value)) +} + +/** + * The entry point: load every run, judge, and write the report. + * @param {string[]} argv - the arguments after the script name. + */ +export function main(argv = process.argv.slice(2)) { + const options = parseArgs(argv) + const corpusAll = loadTrackTasks() + const runs = options.runs.map(loadRun) + const report = [] + report.push('# V5 Flash comparison — collected tables') + for (const run of runs) { + report.push('') + report.push('## Sweep ' + path.basename(run.runDir) + (run.environment.aa ? ' (A/A)' : '')) + const byTask = new Map() + for (const entry of run.rows) { + if (!byTask.has(entry.row.task)) byTask.set(entry.row.task, new Map()) + const arms = byTask.get(entry.row.task) + arms.set(entry.row.arm, [...(arms.get(entry.row.arm) ?? []), entry]) + } + for (const [taskId, arms] of byTask) { + const corpus = corpusAll.find(entry => entry.id === taskId) + report.push('') + report.push('### Task ' + taskId) + for (const [armLabel, entries] of arms) { + const block = armBlock(entries, corpus) + report.push(...lines('arm ' + armLabel, block)) + for (const bucket of USAGE_BUCKETS) report.push(' usage.' + bucket + ': ' + block.usage[bucket]) + report.push(' firstPrompt: median=' + block.firstPrompt.median + ' min=' + block.firstPrompt.min + ' max=' + block.firstPrompt.max) + report.push(' promptTotal: median=' + block.promptTotal.median + ' min=' + block.promptTotal.min + ' max=' + block.promptTotal.max) + for (const judged of block.judged) { + report.push(' r' + judged.repeat + ': correct=' + judged.correctNextSteps + ' opportunities=' + judged.opportunities + + ' retries=' + judged.noProgressRetries + ' waits=' + judged.waits + ' violations=' + judged.ruleViolations + + ' answer=' + (judged.answerMatch ? 'match' : 'MISMATCH')) + } + } + } + report.push('') + report.push('Environment: plugin ' + run.environment.plugin.commit + ', host ' + run.environment.host.version + + ', ' + run.environment.platform + ', worker sha256 ' + String(run.environment.worker.sha256).slice(0, 8) + + ', preset ' + run.environment.preset + ', condition ' + JSON.stringify(run.environment.condition) + + ', sandbox ' + run.environment.sandbox.mode) + } + const text = report.join('\n') + '\n' + const out = options.out ?? path.join(runs.at(-1).runDir, 'v5-collect.md') + fs.writeFileSync(out, text) + console.log(text) + console.log('[v5] wrote ' + out) + return 0 +} + +if (process.argv[1] && path.resolve(process.argv[1]) === path.resolve(fileURLToPath(import.meta.url))) { + process.exitCode = main() +} diff --git a/bench/v5-run.mjs b/bench/v5-run.mjs new file mode 100644 index 00000000..1b192d5a --- /dev/null +++ b/bench/v5-run.mjs @@ -0,0 +1,486 @@ +/** + * The V5 (#994) Flash measurement driver: real OPS-PTC sessions on the real + * opencode-go/deepseek-v4.1-flash route, one fresh composed group per session. + * + * Why this driver exists instead of `bench ab`: the host's one-shot headless + * runner composes the host realm only — it refuses preset sessions by design + * ("runs under agent preset … which the one-shot runner does not compose"), and + * the dsh-ops plugin treats a registry-less host realm as Standard, so an + * `ab` sweep would measure Standard and be invalid per #980. This driver reuses + * the V1 isolated-entry composition (real AgentLoop + real PTC executor + real + * staged worker + the shipped dsh-ops row) and replaces the scripted provider + * with the real llm-pi-ai adapter, which the smoke probe proved end to end: + * run_code-only surface, nested ops_* envelopes, per-step usage with cache + * buckets. + * + * One session = one throwaway workspace + one fresh Context (nothing leaks + * across sessions). The sweep interleaves arms per (task, repeat) AB/BA, so + * machine drift cannot masquerade as an arm effect. Every session archives its + * raw events, its trajectory (root programs + nested calls + envelopes), its + * folded metrics, its verified answer and its workspace outcome; nothing is + * averaged away before it is written down. + * + * Prerequisites (all fail loud, never silently): + * - a built `lib/` and staged `runtime/-/dsh-ops-worker` + * (`npm run build` + `npm run build:ptc`); + * - the vendored `@deepseek-ai/dsh-llm-pi-ai` tree under node_modules (see + * ensureVendoredLlm below — copies the pi-ai adapter and its credential + * helper from the installed dsh host so every import resolves through this + * checkout's tree and one cordis instance serves the whole composition); + * - the route credential: OPENCODE_GO_API_KEY resolved from the credentials + * file of --dsh-home (or already exported in the environment). + * + * @module dsh-ops/bench/v5-run + */ +import { createHash } from 'node:crypto' +import fs from 'node:fs' +import os from 'node:os' +import path from 'node:path' +import { spawnSync } from 'node:child_process' +import { fileURLToPath, pathToFileURL } from 'node:url' + +import { Context } from '@deepseek-ai/cordis' +import Agents from '@deepseek-ai/dsh-agent' +import Sessions from '@deepseek-ai/dsh-session' +import Projections from '@deepseek-ai/dsh-session-projection' +import Prompt from '@deepseek-ai/dsh-system-prompt' +import Tools from '@deepseek-ai/dsh-tools' +import * as PtcExecutor from '@deepseek-ai/dsh-tool-executor-ptc' +import Llm from '@deepseek-ai/dsh-llm' +import * as LlmPiAiModule from '@deepseek-ai/dsh-llm-pi-ai' +import Loop from '@deepseek-ai/dsh-agent-loop' +import { createUserMessage } from '@deepseek-ai/dsh-llm' +import Policy, { setSandboxMode } from '@deepseek-ai/dsh-sandbox-policy' +import Subprocess from '@deepseek-ai/dsh-subprocess-local' +import Sandbox from '@deepseek-ai/dsh-sandbox-local' +import FileSystem from '@deepseek-ai/dsh-fs-local' +import NodePtc from '@deepseek-ai/dsh-ptc-runtime-node' +import { getDshRuntimeVersion } from '@deepseek-ai/dsh-app-boot' + +import { generate } from './lib/fixtures/index.mjs' +import { verifyWorkspace } from './lib/verify.mjs' +import { resolveDshEntry } from './lib/env.mjs' +import { loadTrackTasks, TRACK_TASK_IDS } from './lib/track-corpus.mjs' + +const BENCH_DIR = path.resolve(path.dirname(fileURLToPath(import.meta.url))) +const REPO_ROOT = path.dirname(BENCH_DIR) +const LlmPiAi = LlmPiAiModule.default ?? LlmPiAiModule + +/** The frozen route every session of one sweep shares (#994 step 1). */ +const CONDITION = Object.freeze({ provider: 'opencode-go', model: 'deepseek-v4.1-flash', reasoningEffort: 'high' }) +/** The agentPreset every session carries; the plugin's mode detection keys on it. */ +const PRESET = 'ops-ptc' +/** The sandbox mode pinned for every session, matching the operator's real profiles. */ +const SANDBOX_MODE = 'danger-full-access' +/** The two arms of the comparison sweep: the preregistered factor is the row's disclosurePolicy. */ +const ARMS = Object.freeze([ + { label: 'ondemand', disclosurePolicy: 'on-demand' }, + { label: 'full', disclosurePolicy: 'full' }, +]) +/** A/A arms: the same config twice, so the sweep measures the noise floor. */ +const AA_ARMS = Object.freeze([ + { label: 'aa-one', disclosurePolicy: 'on-demand' }, + { label: 'aa-two', disclosurePolicy: 'on-demand' }, +]) + +/** The CLI text. */ +const USAGE = [ + 'usage: node bench/v5-run.mjs [--task ]... [--repeat N] [--aa] [--out ]', + ' [--dsh-home ] [--timeout-ms N] [--keep-workspaces]', + '', + 'Runs real OPS-PTC sessions on the frozen opencode-go/deepseek-v4.1-flash route.', + 'One fresh workspace and one fresh composed group per session; arms interleave', + 'AB/BA per (task, repeat).', + '', + ' --task a V4 track task (' + TRACK_TASK_IDS.join(', ') + ');', + ' default: ptc-catalog-recover ptc-wait-dispatch', + ' --repeat N repeats per task per arm (default 3)', + ' --aa A/A mode: both arms run the on-demand config (default:', + ' the on-demand vs full comparison)', + ' --out the sweep directory (default bench/runs/v5-)', + ' --dsh-home where the credentials file lives (default ~/.dsh)', + ' --timeout-ms N per-session cap (default: the task file\'s own timeoutMs)', + ' --keep-workspaces keep the seeded workspaces under /workspaces', + '', + 'Exit codes: 0 the sweep ran to its plan · 1 a setup or session failure', +].join('\n') + +/** Parse the command line; anything unknown is a usage error. */ +function parseArgs(argv) { + const options = { tasks: [], repeat: 3, aa: false, out: null, dshHome: null, timeoutMs: null, keepWorkspaces: false } + for (let index = 0; index < argv.length; index += 1) { + const flag = argv[index] + const value = argv[index + 1] + if (flag === '-h' || flag === '--help') { console.log(USAGE); process.exit(0) } + if (flag === '--task') { + if (value === undefined || !TRACK_TASK_IDS.includes(value)) { + console.error('--task needs one of: ' + TRACK_TASK_IDS.join(', ')); process.exit(2) + } + options.tasks.push(value); index += 1; continue + } + if (flag === '--repeat') { options.repeat = Number(value); index += 1; continue } + if (flag === '--aa') { options.aa = true; continue } + if (flag === '--out') { options.out = value; index += 1; continue } + if (flag === '--dsh-home') { options.dshHome = value; index += 1; continue } + if (flag === '--timeout-ms') { options.timeoutMs = Number(value); index += 1; continue } + if (flag === '--keep-workspaces') { options.keepWorkspaces = true; continue } + console.error('unknown option ' + JSON.stringify(flag) + '\n'); console.error(USAGE); process.exit(2) + } + if (options.tasks.length === 0) options.tasks = ['ptc-catalog-recover', 'ptc-wait-dispatch'] + if (!Number.isInteger(options.repeat) || options.repeat < 1 || options.repeat > 20) { + console.error('--repeat needs an integer in [1, 20]'); process.exit(2) + } + return options +} + +/** + * Vendor the real pi-ai adapter into this checkout's node_modules so the whole + * composition shares one dsh-llm and one cordis instance. node_modules is + * gitignored and no package.json is touched: this is a runtime resolution + * detail of the measurement driver, not a dependency change. + * @param {string} installed - the installed dsh host's node_modules/@deepseek-ai directory. + */ +function ensureVendoredLlm(installed) { + const root = path.join(REPO_ROOT, 'node_modules') + const targets = [ + { from: path.join(installed, 'dsh-llm-pi-ai'), to: path.join(root, '@deepseek-ai', 'dsh-llm-pi-ai') }, + { from: path.join(installed, 'dsh-credentials'), to: path.join(root, '@deepseek-ai', 'dsh-credentials') }, + ] + for (const target of targets) { + if (fs.existsSync(path.join(target.to, 'package.json'))) continue + if (!fs.existsSync(path.join(target.from, 'package.json'))) { + throw new Error('cannot find the installed ' + path.basename(target.from) + ' under ' + installed) + } + fs.cpSync(target.from, target.to, { recursive: true }) + console.log('[vendor] copied ' + path.basename(target.from) + ' into node_modules (gitignored, no manifest change)') + } + const link = path.join(root, '@earendil-works', 'pi-ai') + const source = path.join(installed, '..', '@earendil-works', 'pi-ai') + if (!fs.existsSync(link)) { + if (!fs.existsSync(path.join(source, 'package.json'))) { + throw new Error('cannot find the installed @earendil-works/pi-ai beside ' + installed) + } + fs.mkdirSync(path.dirname(link), { recursive: true }) + fs.symlinkSync(source, link, 'dir') + console.log('[vendor] linked @earendil-works/pi-ai -> ' + source) + } +} + +/** Resolve OPENCODE_GO_API_KEY from the credentials file or the environment. */ +function resolveCredential(dshHome) { + if (process.env.OPENCODE_GO_API_KEY) return { source: 'environment' } + const file = path.join(dshHome, '.credentials.yaml') + let text = null + try { text = fs.readFileSync(file, 'utf8') } catch { text = null } + const match = text?.match(/^ OPENCODE_GO_API_KEY: (.+)$/m) + if (match === null) throw new Error('OPENCODE_GO_API_KEY is not exported and ' + file + ' does not carry it') + process.env.OPENCODE_GO_API_KEY = match[1].trim() + return { source: file } +} + +/** The checked-out commit, without trusting the environment. */ +function pluginCommit() { + const shaOf = value => /^[0-9a-f]{40}$/i.test(value ?? '') ? value.toLowerCase() : undefined + const result = spawnSync('git', ['rev-parse', 'HEAD'], { cwd: REPO_ROOT, encoding: 'utf8' }) + return shaOf(result.stdout?.trim()) ?? 'unknown' +} + +/** The staged worker the row leaves to its own resolution, with its digest. */ +function stagedWorkerFacts() { + const platform = process.platform + '-' + process.arch + const file = path.join(REPO_ROOT, 'runtime', platform, 'dsh-ops-worker') + if (!fs.existsSync(file)) throw new Error('no staged worker at ' + file + '; run npm run build:ptc') + const receipt = path.join(REPO_ROOT, 'runtime', platform, 'provenance.json') + return { + file, sha256: createHash('sha256').update(fs.readFileSync(file)).digest('hex'), + receipt: fs.existsSync(receipt) ? JSON.parse(fs.readFileSync(receipt, 'utf8')) : null, + } +} + +/** The shipped-row config one arm runs: identical except the preregistered tier. */ +function armRow(arm) { + return { + global: {}, + discovery: { required: true }, + minimal: { enabled: false }, standard: { enabled: false }, cordis: { enabled: false }, ptc: { enabled: false }, + 'ops-ptc': { enabled: true, required: true, disclosurePolicy: arm.disclosurePolicy }, + } +} + +/** + * Boot one session's group: the V1 composition with the real provider. + * @param {string} root - the seeded workspace. + * @param {object} arm - the arm descriptor. + * @returns {Promise<{ctx: object, events: object[], execs: object[], errors: string[]}>} the live group. + */ +async function bootGroup(root, arm) { + const ctx = new Context(); ctx.logger.level = 0 + ctx.provide('workingDirectory', { + get: session => session?.header?.cwd ?? root, + ensure: async agent => agent?.session?.header?.cwd ?? root, + set: async (_agent, target) => target ?? root, + }) + const events = [] + const execs = [] + const errors = [] + for (const plugin of [Agents, Sessions, Projections, Prompt]) await ctx.plugin(plugin, {}) + await ctx.plugin(Tools, {}) + await ctx.plugin(Policy, { mode: SANDBOX_MODE, workspaceRoot: root }) + await ctx.plugin(Subprocess) + await ctx.plugin(Sandbox) + await ctx.plugin(FileSystem, { cwd: root }) + await ctx.plugin(NodePtc, {}) + await ctx.plugin(PtcExecutor, {}) + ctx.on('session/event', (session, event) => events.push(event)) + await ctx.plugin(Llm, {}) + await ctx.plugin(LlmPiAi, { providers: { [CONDITION.provider]: { + apiKeyEnv: 'OPENCODE_GO_API_KEY', + models: [{ id: CONDITION.model, name: CONDITION.model, contextWindow: 1000000, maxTokens: 384000, input: ['text', 'image'] }], + } } }) + ctx.on('tools/execute', async (exec, next) => { const result = await next(); execs.push({ exec, result }); return result }) + ctx.on('agent/error', ({ error }) => errors.push(String(error?.stack ?? error))) + const { apply: applyShippedRow } = await import(pathToFileURL(path.join(REPO_ROOT, 'lib', 'index.js')).href) + await ctx.plugin({ name: 'dsh-ops-v5-row', apply: inner => applyShippedRow(inner, armRow(arm)) }) + await ctx.plugin(Loop, { agents: [], maxParallelToolCalls: 3 }) + await ctx.get('loader')?.await() + return { ctx, events, execs, errors } +} + +/** Parse one exec's arguments whether the host handed a string or an object. */ +function execArgs(exec) { + const raw = exec.input ?? exec.arguments ?? exec.args + if (raw === undefined || raw === null) return {} + if (typeof raw === 'string') { try { return JSON.parse(raw) } catch { return { unparsed: String(raw) } } } + return raw +} + +/** + * Fold a finished session's raw captures into the archived metrics. + * @param {object[]} events - the raw session events, in order. + * @param {object[]} execs - the captured tool executions with results. + * @returns {object} the metrics block. + */ +function foldMetrics(events, execs) { + const usage = { inputTokens: 0, outputTokens: 0, cacheReadTokens: 0, cacheWriteTokens: 0, reasoningTokens: 0 } + const promptSizes = [] + let steps = 0 + let stepsWithUsage = 0 + const toolCallsByName = Object.create(null) + let turnEndReason = null + const number = value => Number.isFinite(Number(value)) ? Number(value) : 0 + for (const event of events) { + const data = event.data ?? {} + if (event.type === 'assistant/message') { + steps += 1 + const reported = data.usage && typeof data.usage === 'object' + if (reported) { + stepsWithUsage += 1 + for (const bucket of Object.keys(usage)) usage[bucket] += number(data.usage[bucket]) + promptSizes.push(number(data.usage.inputTokens) + number(data.usage.cacheReadTokens) + number(data.usage.cacheWriteTokens)) + } + for (const block of data.message?.content ?? []) { + if (block.type === 'tool-call') toolCallsByName[block.name] = (toolCallsByName[block.name] ?? 0) + 1 + } + continue + } + if (event.type === 'turn/end') turnEndReason = data.reason?.kind ?? null + } + return { + steps, stepsWithUsage, + usageReported: steps > 0 && stepsWithUsage === steps, + usage, + prompt: { + reported: steps > 0 && promptSizes.length === steps, + firstStep: promptSizes.length > 0 ? promptSizes[0] : null, + perStep: promptSizes, + total: promptSizes.length > 0 ? promptSizes.reduce((sum, value) => sum + value, 0) : null, + }, + toolCallsByName: { ...toolCallsByName }, + programs: execs.filter(entry => !String(entry.exec.id ?? '').includes(':ptc:')).length, + turnEndReason, + } +} + +/** + * Fold the captured executions into the judge-ready trajectory: root programs + * in completion order, each with its nested calls and envelopes. + * @param {object[]} execs - the captured tool executions with results. + * @returns {object} the trajectory block. + */ +function foldTrajectory(execs) { + const programs = [] + for (const { exec, result } of execs) { + const callId = String(exec.id ?? exec.callId ?? '') + if (callId.includes(':ptc:')) continue + const args = execArgs(exec) + programs.push({ callId, name: exec.name, description: args.description ?? null, code: args.code ?? null, + isError: result?.isError === true, error: result?.error?.message ?? null }) + } + const byRoot = new Map() + for (const { exec, result } of execs) { + const callId = String(exec.id ?? exec.callId ?? '') + const split = callId.lastIndexOf(':ptc:') + if (split < 0) continue + const root = callId.slice(0, split) + if (!byRoot.has(root)) byRoot.set(root, []) + byRoot.get(root).push({ + api: exec.name, + args: execArgs(exec), + response: result?.value ?? { hostError: result?.error?.message ?? 'no envelope' }, + }) + } + for (const program of programs) { + program.calls = byRoot.get(program.callId) ?? [] + } + return { programs } +} + +/** Write one JSON file, refusing to clobber. */ +function writeJson(file, value) { + fs.mkdirSync(path.dirname(file), { recursive: true }) + fs.writeFileSync(file, JSON.stringify(value, null, 2) + '\n') +} + +/** + * Run one measured session end to end and archive it. + * @param {object} plan - {task, repeat, arm, outDir, timeoutMs, corpus}. + * @returns {Promise} the archived row summary. + */ +async function runSession(plan) { + const { task, repeat, arm, outDir, timeoutMs, corpus } = plan + const workspace = fs.mkdtempSync(path.join(outDir, 'workspaces', task.id + '-r' + repeat + '-' + arm.label + '-')) + const fixture = generate(task.task.fixture.generator, task.task.fixture.params ?? {}) + for (const file of fixture.files) { + const target = path.join(workspace, file.path) + fs.mkdirSync(path.dirname(target), { recursive: true }) + fs.writeFileSync(target, file.content) + } + const group = await bootGroup(workspace, arm) + const { ctx, events, execs, errors } = group + const startedAt = Date.now() + let timedOut = false + let sessionError = null + let agent = null + try { + agent = (await ctx.get('agents').create({ + sessionId: 'v5-' + task.id + '-r' + repeat + '-' + arm.label + '-' + startedAt, + meta: { cwd: workspace, agentPreset: PRESET }, + agentOptions: { ...CONDITION }, + })).agent + setSandboxMode(agent.session, SANDBOX_MODE) + agent.followup(createUserMessage({ content: [{ type: 'text', text: task.task.prompt }], source: { kind: 'user' } })) + await Promise.race([ + agent.whenIdle(), + new Promise((resolve, reject) => { + setTimeout(() => reject(new Error('session timeout')), timeoutMs) + }), + ]) + } catch (error) { + if (/session timeout/.test(String(error?.message))) timedOut = true + else sessionError = String(error?.stack ?? error) + } + const finishedAt = Date.now() + const metrics = foldMetrics(events, execs) + const trajectory = foldTrajectory(execs) + const answerPath = path.join(workspace, 'answer.txt') + const answer = fs.existsSync(answerPath) ? fs.readFileSync(answerPath, 'utf8') : null + const outcome = await verifyWorkspace(workspace, task.task.verify, fixture.exports).catch(error => ({ passed: false, checks: [], error: String(error?.message ?? error) })) + try { await ctx.fiber.dispose() } catch { /* the session is over; dispose is best-effort */ } + if (!plan.keepWorkspaces) fs.rmSync(workspace, { recursive: true, force: true }) + + const row = { + task: task.id, repeat, arm: arm.label, condition: CONDITION, preset: PRESET, + startedAt, finishedAt, durationMs: finishedAt - startedAt, + timedOut, sessionError, agentErrors: errors, + ...metrics, + trajectoryPrograms: trajectory.programs.length, + verify: { passed: outcome.passed === true, checks: (outcome.checks ?? []).map(check => ({ type: check.type, path: check.path, passed: check.passed === true, detail: check.detail ?? '' })) }, + answer, + } + const stem = task.id + '-r' + repeat + '.' + arm.label + writeJson(path.join(outDir, 'results', stem + '.metrics.json'), row) + writeJson(path.join(outDir, 'results', stem + '.trajectory.json'), trajectory) + fs.writeFileSync(path.join(outDir, 'results', stem + '.events.ndjson'), + events.map(event => JSON.stringify(event)).join('\n') + '\n') + return row +} + +/** + * The entry point. + * @param {string[]} argv - the arguments after the script name. + */ +export async function main(argv = process.argv.slice(2)) { + const options = parseArgs(argv) + const dshHome = options.dshHome ?? path.join(os.homedir(), '.dsh') + const stamp = new Date().toISOString().replace(/[:.]/g, '-').slice(0, 19) + const outDir = options.out ? path.resolve(options.out) : path.join(BENCH_DIR, 'runs', 'v5-' + stamp) + fs.mkdirSync(path.join(outDir, 'results'), { recursive: true }) + fs.mkdirSync(path.join(outDir, 'workspaces'), { recursive: true }) + + // Environment facts, frozen before any session runs (#994 step 1). + const entry = resolveDshEntry(null) + const installedDsh = path.resolve(path.dirname(entry.file), '..') + ensureVendoredLlm(path.join(installedDsh, 'node_modules', '@deepseek-ai')) + const credentialSource = resolveCredential(dshHome) + const worker = stagedWorkerFacts() + const corpus = loadTrackTasks() + const arms = options.aa ? AA_ARMS : ARMS + const environment = { + plugin: { commit: pluginCommit(), root: REPO_ROOT }, + host: { version: getDshRuntimeVersion?.() ?? 'unknown' }, + platform: process.platform + '-' + process.arch, node: process.version, + worker: { file: worker.file, sha256: worker.sha256, receipt: worker.receipt }, + sandbox: { mode: SANDBOX_MODE }, preset: PRESET, condition: CONDITION, + credential: { source: credentialSource.source }, + tasks: options.tasks, repeat: options.repeat, aa: options.aa, + note: 'composed V1 isolated-entry group with the real llm-pi-ai adapter; the host one-shot runner cannot compose presets, so bench ab (host realm = Standard) would be invalid per #980', + } + for (const arm of arms) { + const dump = [ + '# v5 dump-config — arm ' + arm.label + ' (' + new Date().toISOString() + ')', + '# The frozen sweep facts; the row below is the only preregistered difference.', + 'condition: ' + JSON.stringify(CONDITION), + 'preset: ' + PRESET, + 'sandbox: ' + SANDBOX_MODE, + 'row:', + JSON.stringify(armRow(arm), null, 2), + ].join('\n') + fs.mkdirSync(path.join(outDir, 'arms', arm.label), { recursive: true }) + fs.writeFileSync(path.join(outDir, 'arms', arm.label, 'dump-config.txt'), dump + '\n') + } + writeJson(path.join(outDir, 'environment.json'), environment) + + // AB/BA interleave per (task, repeat), as bench/run/schedule.mjs orders a sweep. + const plan = [] + for (const id of options.tasks) { + const task = corpus.find(entry => entry.id === id) + if (task === undefined) { console.error('unknown task ' + id); return 1 } + for (let repeat = 1; repeat <= options.repeat; repeat += 1) { + const order = repeat % 2 === 1 ? [arms[0], arms[1]] : [arms[1], arms[0]] + for (const arm of order) { + plan.push({ task, repeat, arm, outDir, timeoutMs: options.timeoutMs ?? task.task.timeoutMs, keepWorkspaces: options.keepWorkspaces, corpus }) + } + } + } + console.log('[v5] sweep ' + outDir + ' — ' + plan.length + ' sessions: ' + options.tasks.join(', ') + ' x ' + options.repeat + ' x ' + arms.length + ' arms' + (options.aa ? ' (A/A)' : '')) + const rows = [] + for (let index = 0; index < plan.length; index += 1) { + const row = plan[index] + console.log('[v5] ' + (index + 1) + '/' + plan.length + ' — ' + row.task.id + ' r' + row.repeat + ' arm=' + row.arm.label + ' (timeout ' + row.timeoutMs + 'ms)') + const summary = await runSession(row) + rows.push(summary) + console.log('[v5] -> ' + (summary.timedOut ? 'TIMEOUT' : summary.verify.passed ? 'pass' : 'fail') + + ' steps=' + summary.steps + ' usage=' + (summary.usageReported ? 'yes' : 'MISSING') + + ' prompt1=' + summary.prompt.firstStep + ' programs=' + summary.trajectoryPrograms + + ' ' + summary.durationMs + 'ms') + } + writeJson(path.join(outDir, 'summary.json'), { environment, rows }) + const failed = rows.filter(row => row.sessionError !== null).length + console.log('[v5] sweep complete: ' + rows.length + ' sessions, ' + failed + ' with session errors') + console.log('[v5] collect with: node bench/v5-collect.mjs --run ' + outDir) + return failed === 0 ? 0 : 1 +} + +if (process.argv[1] && path.resolve(process.argv[1]) === path.resolve(fileURLToPath(import.meta.url))) { + process.exitCode = await main() +} diff --git a/bin/prepare-workspace.ps1 b/bin/prepare-workspace.ps1 index fd347c48..9d6afeb2 100644 --- a/bin/prepare-workspace.ps1 +++ b/bin/prepare-workspace.ps1 @@ -4,21 +4,32 @@ $ErrorActionPreference = 'Stop' if ($ExecutionContext.SessionState.LanguageMode -ne 'FullLanguage') { throw 'DSH_OPS_CONSTRAINED_LANGUAGE: workspace ACL preparation requires PowerShell FullLanguage; no ACL preparation was completed. Keep the host security policy enabled and ask its administrator for a supported preparation path.' } +. (Join-Path $PSScriptRoot 'workspace-acl.ps1') $identity = [Security.Principal.WindowsIdentity]::GetCurrent() $principal = [Security.Principal.WindowsPrincipal]::new($identity) $item = Get-Item -LiteralPath $Workspace -Force if (!$item.PSIsContainer) { throw 'Workspace is not a directory' } -$cursor = $item -while ($null -ne $cursor) { - if ($cursor.Attributes -band [IO.FileAttributes]::ReparsePoint) { throw 'Workspace ancestry contains a reparse point; refusing ACL preparation' } - $cursor = $cursor.Parent -} +# \\wsl.localhost\ and \\wsl$\ are the WSL 9P redirector (#669): +# the filesystem behind them exposes no NTFS security descriptor, so Get-Acl +# cannot produce one and no DACL preparation has an object to write. Authority +# there is the distribution's own uid/gid mapping. The decision runs after the +# existence/container validation above — a missing path still fails exactly like +# any other workspace — and before Get-Acl below, so a live 9P root answers +# ready instead of dying inside it. This is a prefix decision, not a catch: +# every other path keeps its existing behaviour, and a real ACL error on those +# paths still fails closed. +if ((Get-DshOpsWorkspaceNtfsState -Path $item.FullName) -eq 'wsl-redirector') { Write-Output 'ready'; exit 0 } +# No reparse walk here: a folder that is already ready answers without any +# tag inspection, so an OneDrive placeholder root is not refused merely for +# being a reparse point (#744). The tag check lives on the one branch below +# that actually writes. $acl = Get-Acl -LiteralPath $item.FullName -. (Join-Path $PSScriptRoot 'workspace-acl.ps1') $decision = Get-DshOpsWorkspaceDecision -Acl $acl -Identity $identity -Principal $principal if ($decision.State -eq 'denied') { throw $decision.Reason } if ($decision.State -eq 'ready') { Write-Output 'ready'; exit 0 } if ($decision.State -eq 'foreign-owner') { throw $decision.Reason } +# Everything below is the preparation branch: the only code that writes, and +# therefore the only place a reparse tag is allowed to matter (#744). $needed = [Security.AccessControl.FileSystemRights]::TakeOwnership # WRITE_OWNER, needed to set the sandbox integrity label # Write only the root's own DACL through its handle. The added ACE is directory-only and the # inheritable ACEs are unchanged, so no descendant needs a new ACL; path-based SetAccessControl @@ -52,7 +63,47 @@ public static class DshOpsDirectoryDacl { } } } +public static class DshOpsReparseTag { + [DllImport("kernel32.dll", CharSet = CharSet.Unicode, SetLastError = true)] + static extern SafeFileHandle CreateFileW(string name, uint access, uint share, IntPtr security, uint disposition, uint flags, IntPtr template); + [DllImport("kernel32.dll", SetLastError = true)] + static extern bool DeviceIoControl(SafeFileHandle handle, uint control, IntPtr inBuffer, uint inSize, byte[] outBuffer, uint outSize, out uint returned, IntPtr overlapped); + public static uint Get(string path) { + // Zero access, OPEN_EXISTING, FILE_FLAG_BACKUP_SEMANTICS | FILE_FLAG_OPEN_REPARSE_POINT: + // the query opens the point itself, never its target (#744). + using (SafeFileHandle handle = CreateFileW(path, 0u, 7u, IntPtr.Zero, 3u, 0x02000000u | 0x00200000u, IntPtr.Zero)) { + if (handle.IsInvalid) throw new Win32Exception(Marshal.GetLastWin32Error()); + // FSCTL_GET_REPARSE_POINT needs a full REPARSE_DATA_BUFFER-sized output: + // a four-byte slice is rejected with ERROR_INSUFFICIENT_BUFFER. + byte[] buffer = new byte[16384]; + // The first dword of the returned buffer is the tag. + // Windows PowerShell 5.1 compiles with C# 5, which has no inline out declarations. + uint returned; + if (!DeviceIoControl(handle, 0x900A8u, IntPtr.Zero, 0, buffer, (uint)buffer.Length, out returned, IntPtr.Zero) || returned < 4) + throw new Win32Exception(Marshal.GetLastWin32Error() != 0 ? Marshal.GetLastWin32Error() : unchecked((int)0x80070057)); + return BitConverter.ToUInt32(buffer, 0); + } + } +} '@ +# Cloud placeholders keep a real directory entry behind the point, so the proven +# cloud-files family (the exact 0x9000?01A shape) is allowed through; every other +# tag — including the rest of the 0x9xxxxxxx band — redirects the view and fails +# closed with the tag named, and a point whose tag cannot even be read fails closed +# too (#744). +$cursor = $item +while ($null -ne $cursor) { + if ($cursor.Attributes -band [IO.FileAttributes]::ReparsePoint) { + try { $tag = [DshOpsReparseTag]::Get($cursor.FullName) } catch { + throw "Workspace ancestry contains a reparse point at $($cursor.FullName) whose tag cannot be read ($($_.Exception.Message)); refusing ACL preparation. Cloud placeholders are allowed through; an unreadable point is not." + } + $family = Get-DshOpsReparseTagDecision -Tag $tag + if ($family.State -ne 'cloud') { + throw "Workspace ancestry contains an unsupported reparse point (tag 0x$($tag.ToString('X8'))) at $($cursor.FullName); refusing ACL preparation. Cloud placeholders pass; every other reparse tag fails closed." + } + } + $cursor = $cursor.Parent +} # Save only the DACL. Owner, SACL, existing ACEs and child ACLs are not changed. $before = $acl.GetSecurityDescriptorSddlForm([Security.AccessControl.AccessControlSections]::Access) [IO.Directory]::CreateDirectory($BackupDir) | Out-Null diff --git a/bin/workspace-acl.ps1 b/bin/workspace-acl.ps1 index eb7a6b31..cf5a88b7 100644 --- a/bin/workspace-acl.ps1 +++ b/bin/workspace-acl.ps1 @@ -62,3 +62,49 @@ function Get-DshOpsWorkspaceDecision { # foreign-owner instead. Both are the pre-existing behaviour, kept unchanged. [pscustomobject]@{ State = 'needs-preparation' } } + +<# +Classify one reparse tag for the preparation branch (#744). This is pure: it +never opens the point, it only judges the tag value the caller read. + +Only the proven cloud-files family is allowed through: the placeholders +(IO_REPARSE_TAG_CLOUD 0x9000001A, CLOUD_ROOT 0x9000101A, CLOUD_ON_DEMAND +0x9000201A, and family members sharing the exact 0x9000?01A shape) keep a +real directory entry behind the point, so a directory-only WRITE_OWNER write +still targets a real object. A high nibble of 9 alone is NOT enough: the rest +of Microsoft's reserved 0x90000000-0x9FFFFFFF band is unproven here and stays +out, exactly like every other tag (mount point, symlink, dedup, ...) that +redirects or transforms the view and fails closed with the tag named. A tag +that cannot be read at all is the caller's error to surface, not a decision +to soften. +#> +function Get-DshOpsReparseTagDecision { + param( + [Parameter(Mandatory = $true)][uint32]$Tag + ) + # Decimal constants on purpose: Windows PowerShell 5.1 folds in-band hex + # literals into a negative Int32, and a sign-extended Int64 mask never + # matches. 4294905855 is 0xFFFF0FFF (family mask); 2415919130 is + # 0x9000001A (IO_REPARSE_TAG_CLOUD). + if (($Tag -band 4294905855) -eq 2415919130) { return [pscustomobject]@{ State = 'cloud' } } + return [pscustomobject]@{ State = 'unsupported' } +} + +<# +Classify one workspace spelling for NTFS ACL evaluation (#669). Pure. + +\\wsl.localhost\ and \\wsl$\ are the WSL 9P redirector: the +filesystem behind it exposes no NTFS security descriptor, so Get-Acl cannot +produce one and no DACL write has an object. Authority there is the +distribution's own uid/gid mapping, so those prefixes must not be treated +as a failed preparation. This is a prefix decision, not a catch swallowing +ACL errors: everything else is evaluated as NTFS, where a real Get-Acl +failure still fails closed. +#> +function Get-DshOpsWorkspaceNtfsState { + param( + [Parameter(Mandatory = $true)][string]$Path + ) + if ($Path -match '^(\\\\(\?\\UNC\\)?wsl(\.localhost|\$)\\)') { return 'wsl-redirector' } + return 'ntfs' +} diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index d66194e1..796c78dc 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -7,14 +7,132 @@ this package adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm ## [Unreleased] +- Niubash no longer refuses to prepare its runtime when a full-access session's workspace contains + the default `~/.dsh` home (#870). The workspace boundary stays armed in read-only and + workspace-write modes, where host workspace labeling and write gating could otherwise reach the + executable; full access runs without confinement, so the refusal only made a default `DSH_HOME` + under a home workspace unusable. +- Niubash 在 full-access 会话的 workspace 包含默认 `~/.dsh` 时不再拒绝准备运行时(#870)。 + workspace 边界在 read-only 与 workspace-write 模式仍然生效——host 的 workspace 标注与写限制 + 本可能经由 workspace 触及可执行文件;full access 本就不经过 confinement,该拒绝只会让默认 + `DSH_HOME` 落在 home workspace 下的会话完全不可用。 +- ops_bash classifies the msys-2.0.dll shared-section startup failure + ("CreateFileMapping S-., Win32 error 5", printed without any "fatal + error" prefix) as EXECUTOR_UNAVAILABLE with `sandbox.runnerFailed`, so a restricted + token hitting an unrestricted MSYS holder's per-user section is no longer presented + as a bare non-zero exit (#409). +- ops_bash 将 msys-2.0.dll 的共享段启动失败("CreateFileMapping S-<用户 SID>., + Win32 error 5",打印时不带任何 "fatal error" 前缀)归类为 EXECUTOR_UNAVAILABLE 并带 + `sandbox.runnerFailed`:受限令牌撞上非受限 MSYS 持有者的用户级共享段时,不再只呈现 + 一个裸的非零退出码(#409)。 +- A regression guard locks the full-access projection of ops_bash: the selected + payload spawns without sandbox.confine and a plain bashPath runs directly, while + confined modes still route through confinement (#581). The reported full-access + child carrying a Low token (S-1-16-4096) did not reproduce on 0.5.5 — a + full-access child reported the caller's own integrity label here — so the guard + and the token-comparison diagnosis document the boundary instead of claiming a + code fix. +- 回归守卫锁定 ops_bash 的 full-access 投影:所选载荷不经 sandbox.confine 直接 spawn, + 普通 bashPath 直接运行,受限模式仍走 confinement(#581)。报告的 full-access 子进程 + Low 令牌(S-1-16-4096)在 0.5.5 上未能复现——本机 full-access 子进程回报调用方自身的 + 完整性标签——因此以守卫与令牌对照诊断记录该边界,而非声称代码修复。 +- Document the niu-Git hook failure whose only difference is the shebang: + `#!/usr/bin/env sh` fails while `#!/bin/sh` runs, so husky v9 hooks are + affected. The entry records the observed diagnostic pair and the `#!/bin/sh` + workaround; the interpreter-resolution attribution is an upstream inference, + not a verified root cause (#958, unresolved-upstream). +- 记录 niu-Git 钩子失败:唯一差异是 shebang——`#!/usr/bin/env sh` 失败而 + `#!/bin/sh` 可运行,husky v9 钩子同样受影响。条目记录实测诊断对照与 + `#!/bin/sh` 缓解;解释器解析的归因是上游推断,不是已证根因(#958, + unresolved-upstream)。 +- Document the per-call console flash on Windows 10 when Windows Terminal is + the default terminal. The entry gives the default-terminal comparison as the + diagnosis; the console-host hand-off explanation is an upstream inference, and + the hidden-console build remains upstream work, because no dsh-ops setting + controls console hosting (#828, unresolved-upstream). +- 记录 Windows 10 默认终端为 Windows Terminal 时每次调用的控制台闪烁。条目给出 + 默认终端对照作为诊断;控制台宿主交接的解释是上游推断,隐藏控制台构建仍属 + 上游工作——没有任何 dsh-ops 设置控制控制台宿主(#828,unresolved-upstream)。 +- Document the antivirus pattern for Niubash: all pinned files intact except a + quarantined `niu.exe`, named by the #844 member diagnostic. The entry gives + the Defender/ESET event collection, the provision restore and the fixed-location + allow-list guidance, and marks the quarantine an external event rather than a + code defect (#771). +- 记录 Niubash 的杀毒软件模式:全部 pinned 文件完好、仅 `niu.exe` 被隔离,由 #844 + 的成员级诊断点名。条目给出 Defender/ESET 事件收集、provision 恢复与固定位置允许 + 列表指引,并把隔离标注为外部事件而非代码缺陷(#771)。 +- Measure the OPS-PTC disclosure tiers on the real Flash route (#994, see `docs/flash-comparison-v5.md`): `bench/v5-run.mjs` runs one fresh composed OPS-PTC group per session on the frozen `opencode-go`/`deepseek-v4.1-flash`/`high` route — the host one-shot runner cannot compose presets (its host realm is Standard, invalid per #980), so the driver reuses the V1 isolated-entry composition with the real `llm-pi-ai` adapter, vendored into `node_modules` at run time — and `bench/v5-collect.mjs` judges the archived trajectories with the V4 shared rules plus the two generalizing task rules, refusing to demand the scripted stories' mandatory wrong call. The pilot (2 tasks x 3 repeats x 2 arms, AB/BA interleaved, plus a 12-session same-condition A/A) reports the complete denominator (24 planned / 24 completed / 0 timed out / 0 missing usage / 23 verified) and the noise verdict: pass rates, retries and opportunities are not distinguished at n=3; the tier's own first-round prompt cost (5.3k vs 30.0k tokens, totals about 2x) is the only difference far outside the A/A band, and no model-quality attribution is made. `bench/lib/tracks.mjs` only gains the two already-shared rule exports. +- 在真实 Flash 路由上测量 OPS-PTC 披露档位(#994,见 `docs/Flash对照-V5.md`):`bench/v5-run.mjs` 每会话在冻结的 `opencode-go`/`deepseek-v4.1-flash`/`high` 路由上组合全新 OPS-PTC 组——宿主一次性 runner 无法组合 preset(host realm 即 Standard,依 #980 无效),故驱动器复用 V1 隔离入口组合并接真实 `llm-pi-ai` 适配器(运行时 vendor 进 `node_modules`);`bench/v5-collect.mjs` 用 V4 共享规则加两条可泛化任务规则判定归档轨迹,且不强制剧本里「必须先犯错」的桥段。试点(2 任务 x 3 重复 x 2 臂,AB/BA 交错,另加 12 会话同条件 A/A)报告完整分母(24 计划 / 24 完成 / 0 超时 / 0 缺 usage / 23 验证通过)与噪声判定:通过率、重试与纠偏机会在 n=3 下未能区分;唯一远超 A/A 带的是档位自身的首轮提示成本(5.3k vs 30.0k tokens,总量约 2x),不作任何模型质量归因。`bench/lib/tracks.mjs` 仅新增两条既有共享规则的导出。 +- Turn the two-tier OPS-PTC publication rule into a data-driven acceptance matrix (#992): `test/ptc-two-tier-matrix.test.mjs` runs eight canonical rows (legacy default, the four legacy-off variants, on-demand first step, on-demand after loading, the explicit full first round, read-only, a narrow pool and a custom `profile: 'ptc'` integration) through one shared comparator across three views — the SDK text the model reads, the step admission the loop resolves and the `run_code` program bindings — plus a separately collected no-toolSet diagnostic view that is a registration superset, never an admission proof. Every row executes a real no-side-effect sentinel through the real worker (the read-only row's inspect runs under read-only) and write authority is asserted with a safe rejection naming the refused API. The drift property is itself asserted: removing any single expected name — or reading the diagnostic view where a step view is required — fails at least one cell with the missing/extra names printed, and the unified acceptance holds: the loaded business set equals the full first round on both publication paths. The official `read`/`glob`/`grep` trio is pinned as visible under both `hideOfficialTools` states while #961 is out of baseline, so its landing flips a reviewable pin. No publication behaviour changes. +- 把两档 OPS-PTC 发布规则做成数据驱动验收矩阵(#992):`test/ptc-two-tier-matrix.test.mjs` 用同一比较器跑八个规范行(旧缺省、四个旧关闭变体、按需首步、按需加载后、显式全量首轮、只读、窄池、自定义 `profile: 'ptc'` integration)的三个视角——模型读到的 SDK 文本、循环解析的当步准入、`run_code` 程序绑定——外加单列采集的无 toolSet 诊断视图(注册超集,绝非准入证据)。每行都经真实 worker 执行无副作用哨兵(只读行的 inspect 在只读沙箱下真跑),写权限以点名被拒 API 的安全拒绝断言。漂移性质本身被断言:删任一期望名或错读诊断视角,至少一格失败并打印缺失/多余名字;统一口径成立:按需加载完毕的业务集合与全量首轮在两条发布路径上一致。官方 `read`/`glob`/`grep` 三件在 #961 未入基线期间钉为两种 `hideOfficialTools` 状态下均可见,其合入将翻转可评审的钉子。发布行为不变。 +- Publish the explicit full PTC disclosure tier (#987): with `disclosurePolicy: 'full'` the `ptc` and `ops-ptc` rows' first round now carries the whole permitted `allowedTools` pool plus `ops_result` (the D1 sign-off item 1 decision, taken conservatively per #979 §3 so refs spilled this step stay consumable — and forced by the unified rule, since on-demand can always load `result`) with no `ops_load_sdk`. The first program can call the permitted pool directly; read-only sessions still exclude every write API, an empty cap stays at resident + `result`, `residentTools:false` drops the resident surface but not the tier, duplicated names register once, `global.dynamicTools:false` keeps its veto (the row publishes its legacy surface letter for letter) and a row that also turns its loader off keeps the tier. `PtcConfig.disclosurePolicy` stays optional because custom integrations keep the legacy on-demand default until #988; the builtin plan carries the parsed value. On-demand (default) behaviour is letter-identical; the settings-page entry is #989. +- 发布显式 PTC 全量披露档位(#987):`disclosurePolicy: 'full'` 的 `ptc` 与 `ops-ptc` 行首轮现在携带整个获准 `allowedTools` 池外加 `ops_result`(D1 签字项 1 的保守决定,依 #979 §3 让本步落盘的 ref 仍可消费——统一口径也逼出同一结论,因为按需档永远可加载 `result`),且无 `ops_load_sdk`。第一个程序即可直接调用获准池;只读会话仍排除全部写 API,空上限停在 常驻 + `result`,`residentTools:false` 丢常驻面但不丢档位,重复名只注册一次,`global.dynamicTools:false` 保持否决(该行逐字发布旧面),同时关掉 loader 的行保留档位。`PtcConfig.disclosurePolicy` 保持可选,因为自定义 integration 在 #988 之前维持旧 on-demand 缺省;内置计划携带解析值。按需档(缺省)行为逐字不变;设置页入口归 #989。 +- Parse an explicit PTC disclosure policy without changing any legacy behaviour: the `ptc` and `ops-ptc` rows accept a YAML-only `disclosurePolicy` key, `'on-demand'` (default, the legacy progressive path) or `'full'`. Illegal values are errors naming the key and the received value or type, and non-PTC rows reject the key with a readable diagnostic. The choice survives read → save-unrelated-setting → read through the settings serializer, and no upgrade, settings save or `global.dynamicTools` flip ever writes or erases it. The D1 sign-off item on precedence is resolved conservatively: while `global.dynamicTools` is off the full tier is not applied — the row keeps its legacy resident/alwaysLoaded surface — and the saved choice re-applies when the switch returns on; the row-level `dynamicTools` keeps its loader-only meaning because the full tier publishes no loader. Publishing the full tier remains the D3 slice; no publication behaviour or settings UI changes (#986). +- 解析显式 PTC 披露策略且不改变任何旧行为:`ptc` 与 `ops-ptc` 行接受 YAML-only 的 `disclosurePolicy` 键,取值 `'on-demand'`(缺省,即旧渐进路径)或 `'full'`。非法值报错点名键与收到的值或类型,非 PTC 行以可读诊断拒绝该键。选择经设置序列化器在「读取 → 保存无关设置 → 再读取」后不丢,升级、保存设置与 `global.dynamicTools` 翻转都不写入或抹掉它。D1 签字项中优先级问题按保守取向解决:`global.dynamicTools` 关闭期间全量档不生效——行保持旧 resident/alwaysLoaded 面——开关回开后已存选择重新生效;行级 `dynamicTools` 保持只管 loader 的含义,因为全量档不发布 loader。全量档的发布仍归 D3 切片;发布行为与设置 UI 均不变(#986)。 +- Record the D1 supplementary audit for the legacy-config truth table: the custom PTC integration path rejects `alwaysLoaded` outside `allowedTools` (`src/consumer/options.ts`) while built-in rows accept it silently; the settings-page "discovery entry" copy is recorded verbatim next to the typed `ops_load_sdk` the PTC row actually publishes; a legal custom `profile: 'ptc'` integration is verified to be misdiagnosed `PUBLICATION_UNAVAILABLE: ops_result` (fix owned by #988); and one `ops_load_sdk` call is locked at 1-5 raw names (four calls minimum for the sixteen-name pool, resident lists uncapped). `test/ptc-config-truth.test.mjs` gains two assertions (13 checks); docs/design-decisions-20261119.md gains the matching append-only sections. No behaviour changes (#985). +- 补录 D1 真值表的追加审计:自定义 PTC integration 路径拒绝落在 `allowedTools` 之外的 `alwaysLoaded`(`src/consumer/options.ts`),内置行却静默接受;设置页「发现入口」文案逐字照录,并注明 PTC 行实际发布的是类型化 `ops_load_sdk`;实测合法自定义 `profile: 'ptc'` integration 会被误诊 `PUBLICATION_UNAVAILABLE: ops_result`(修复归 #988);一次 `ops_load_sdk` 调用锁定为 1-5 个原始名(十六名池至少四次调用,常驻名单无数量上限)。`test/ptc-config-truth.test.mjs` 新增两项断言(13 项检查),docs/设计决策-20261119.md 同步追加对应小节。行为不变(#985)。 +- Lock the five legacy OPS-PTC configuration inputs as a reviewable truth table: defaults, `global.dynamicTools:false`, row `dynamicTools:false`, that row plus `alwaysLoaded:[stat]`, and `residentTools:false` publish the exact first-round `ops_*` sets now asserted letter by letter by `test/ptc-config-truth.test.mjs` (inspect + loader; inspect; inspect; inspect + stat; loader only — the `ptc` row matches `ops-ptc`). Reading old `dynamicTools:false` as a full selection is pinned as a different input, and the implicit `ops_result` exception, the global-switch gate and `alwaysLoaded`-versus-cap conflicts are recorded as maintainer sign-off items in docs/design-decisions-20261119.md, not implemented. No publication default or behaviour changes (#985). +- 将五种旧 OPS-PTC 配置输入固化为可审阅的真值表:缺省、`global.dynamicTools:false`、行级 `dynamicTools:false`、行关闭加 `alwaysLoaded:[stat]` 与 `residentTools:false` 的首轮 `ops_*` 精确集合由 `test/ptc-config-truth.test.mjs` 逐字断言(inspect+loader;inspect;inspect;inspect+stat;仅 loader——`ptc` 行与 `ops-ptc` 相同)。「把旧 `dynamicTools:false` 解读为全量」被钉为另一档输入;`ops_result` 隐式例外、总开关与门、`alwaysLoaded` 与上限的冲突在 docs/设计决策-20261119.md 记录为维护者签字决定项,未做实现。发布默认值与行为均不变(#985)。 + - Native tool construction and the publication refusal queue no longer attach listeners or lifetime effects to the composition root. The evidence observer and result admission subscribe on the caller's context with global dispatch, and the refusal queue falls back to the caller lifetime when the root refuses an effect. dsh-tui 0.20 refuses `root.events.on` and `root.effect` from a plugin activation, and the native-tools row is one, so every `dsh-tui --resume` aborted with "cannot resume session" before the agent mounted. - 原生工具构造与发布拒绝队列不再向组合根挂监听或生命周期 effect:证据观察者与 result admission 改为在调用方上下文订阅并声明 global 分发,拒绝队列在 root 拒绝 effect 时回落到调用方生命周期。dsh-tui 0.20 会拒绝插件激活期内的 `root.events.on` 与 `root.effect`,而原生工具行正是插件激活,导致每次 `dsh-tui --resume` 都在 agent 挂载前报 "cannot resume session"。 - The Bash tool description now states the parse contract of `stdout`/`stderr` where the model reads it: `text` may be a partial fragment, `truncated` marks one that must not be parsed as complete structured output, and `spillPath` is optional and not a completeness guarantee. The typed output schema is unchanged: repeating the contract there would cost every PTC request without saying anything the tool text does not. Command semantics, exit codes, result values, limits, spill files, permissions and rendering are unchanged. - Bash 工具说明现在在模型读取它的地方写明 `stdout`/`stderr` 的解析契约:文本可能只是片段,`truncated` 标记的片段不得当作完整结构化输出解析,`spillPath` 是可选项且不保证完整。输出类型 schema 保持不变:在那里重复这段契约只会让每个 PTC 请求更贵,并不会多说明什么。命令语义、退出码、结果值、上限、落盘、权限与渲染均不变。 +- The promoted Bash render no longer implies a job kill stops one command: it states that `job_kill` terminates the whole process tree, including daemons the command started and left attached to it (an adb server, a dev-server watcher), and points to stopping those on their own channel, starting them detached, or beginning daemon-like work with `run_in_background`. The host's subprocess/job layer still owns the actual kill semantics; this is actionable guidance, not a host fix (#814). +- 提升为后台的 Bash 渲染不再暗示一次任务终止只停掉一条命令:现在写明 `job_kill` 会终止整棵进程树,包括该命令启动后仍挂在其上的守护进程(如 adb server、dev-server watcher),并给出指引——先在守护进程自己的通道上停止、以分离方式启动,或对守护进程类工作直接使用 `run_in_background`。宿主的 subprocess/job 层仍拥有真实的终止语义;这是可操作的指引,不是对宿主的修复(#814)。 +- The workspace bootstrap no longer refuses a root for being a reparse point before the DACL decision runs: a folder that already grants effective `WRITE_OWNER` answers `ready` with no tag inspection at all, so an OneDrive placeholder root (IO_REPARSE_TAG_CLOUD, no target) stops failing every workspace-write tool (#744). The tag check now lives on the only branch that writes the DACL, and only the proven cloud-files family passes — the exact 0x9000?01A shape (IO_REPARSE_TAG_CLOUD, CLOUD_ROOT, CLOUD_ON_DEMAND), allowed because a placeholder keeps a real directory entry behind the point; the rest of the reserved 0x90000000-0x9FFFFFFF band is unproven and fails closed exactly like mount points, symlinks, and unknown or unreadable tags, naming the tag it refused. Cloud tags cannot be synthesized in a unit fixture, so the family rule is pinned on the pure decision function `Get-DshOpsReparseTagDecision` and the fail-closed walk is pinned end to end with a real junction. +- 工作区引导脚本不再在 DACL 判定之前就因为「是重解析点」而拒绝根目录:已授予有效 `WRITE_OWNER` 的文件夹在完全不检查 tag 的情况下直接回答 `ready`,OneDrive 占位符根目录(IO_REPARSE_TAG_CLOUD,无目标)不再让所有 workspace-write 工具失败(#744)。tag 检查现在只存在于唯一会写 DACL 的分支上,且只有已证实的云文件族可以通过——精确的 0x9000?01A 形状(IO_REPARSE_TAG_CLOUD、CLOUD_ROOT、CLOUD_ON_DEMAND),放行理由是占位点后面保留真实目录项;保留段 0x90000000-0x9FFFFFFF 的其余部分未经证实,与挂载点、符号链接、未知或不可读 tag 一样闭锁拒绝,并写明所拒绝的具体 tag。云 tag 无法在单元夹具中合成,因此家族规则固定在纯判定函数 `Get-DshOpsReparseTagDecision` 上,闭锁遍历则用真实 junction 做端到端验证。 +- A WSL UNC workspace (`\wsl.localhost\` or `\wsl$\`) no longer dies inside the workspace bootstrap: the 9P redirector behind those prefixes exposes no NTFS security descriptor, so `Get-Acl` threw and every workspace-write `run_code` call failed. Exactly those two Microsoft-fixed spellings now answer `ready` after the existence and container validation and before `Get-Acl` is touched — a prefix decision pinned on the pure classifier `Get-DshOpsWorkspaceNtfsState`, not a catch that swallows ACL errors: a missing path fails exactly like any other missing workspace, and a real ACL failure on any other path still fails closed (#669). +- WSL UNC 工作区(`\wsl.localhost\<发行版>` 或 `\wsl$\<发行版>`)不再死在工作区引导脚本里:这两个前缀背后的 9P 重定向器不暴露 NTFS 安全描述符,`Get-Acl` 因此抛错,workspace-write 下的每次 `run_code` 调用都失败。现在恰好这两个微软固定前缀会在完成存在性与目录验证之后、触碰 `Get-Acl` 之前直接回答 `ready`——判定固定在纯分类函数 `Get-DshOpsWorkspaceNtfsState` 上,是前缀判定而不是吞错 catch:不存在的路径与其他任何缺失工作区一样失败,其他任何路径保持原有行为,真实的 ACL 故障依然闭锁失败(#669)。 +- The optional WSL read pays one wsl.exe startup for preparation instead of one per drive mapping plus one for the payload digest (#576): preparation is now a single fixed `/bin/sh -c` script whose text is a constant, with every drive root and the payload path as positional arguments, so no Windows path can alter the command shape. A small delegated read drops from four wsl.exe invocations (two drive mappings, one sha256sum, one worker) to two (one merged preparation, one worker). The wslpath mappings are still resolved anew per call, the payload digest must still match the receipt, and a refused mapping (exit 9), refused digest (exit 8) or failed spawn still falls back to the Windows worker exactly as before; the distribution now needs `/bin/sh` beside `wslpath` and `sha256sum`. +- 可选 WSL 读取的准备开销从「每个盘符一次 wsl.exe 外加一次摘要校验」合并为一次 wsl.exe 启动(#576):准备现在是一次固定 `/bin/sh -c` 脚本,脚本文本是常量,所有盘符根与 worker 路径都以位置参数传递,任何 Windows 路径都无法改变命令形状。一次小的委派读取从四次 wsl.exe 调用(两次盘符映射、一次 sha256sum、一次 worker)降为两次(一次合并准备、一次 worker)。wslpath 映射仍按每次调用重新解析,载荷摘要仍须与凭据一致,映射被拒(退出码 9)、摘要被拒(退出码 8)或进程启动失败仍与从前完全一样回退到 Windows worker;发行版在 `wslpath` 与 `sha256sum` 之外现在还需要 `/bin/sh`。 + +- A registered-but-unselected PTC pool API now answers a direct call with `SDK_NOT_LOADED` and a next action naming `ops_load_sdk` and its raw name, instead of `PUBLICATION_GRANT_UNAVAILABLE` (retry: never) on the file-lease path or a bare `POLICY_DENIED` on the PTC-only path. Hosts that render registered definitions can still declare the whole allowed pool, but the load flow now matches that declaration: the refusal says how to make the API callable (selection lands on the next model step) and stays honest when the current policy cannot load the API at all. Loader-less scopes keep registering nothing from the pool (#887 unchanged) (#968). +- 已注册但未选中的 PTC 候选池 API 被直接调用时,现在返回 `SDK_NOT_LOADED`,并在 nextAction 中点名 `ops_load_sdk` 与原始名称,不再在文件租约路径上返回 `PUBLICATION_GRANT_UNAVAILABLE`(retry: never)、在 PTC 专属路径上只返回光秃秃的 `POLICY_DENIED`。仍会渲染全部注册定义的宿主依旧可能声明整个 allowedTools 池,但加载流程与该声明现在一致:拒绝会说明如何让该 API 可调用(选择在下一模型步骤生效),且当前策略根本无法加载该 API 时保持诚实的通用拒绝。无 loader 作用域仍不注册池内任何项(#887 不变)(#968)。 + +- An unselected discovery capability, built-in or external provider, now answers a direct call with `CAPABILITY_NOT_SELECTED` and a next action naming `ops_discover_tools` with a `select:` query, instead of `PUBLICATION_GRANT_UNAVAILABLE` or a bare `POLICY_DENIED` that pointed nowhere. The pointer stays honest: a read-only scope that can never select a mutating capability keeps the generic refusal, and once a `select:` query has committed at step end the deferred rejection stops answering (#764). +- 未选中的发现能力(内置或外部 provider)被直接调用时,现在返回 `CAPABILITY_NOT_SELECTED`,并在 nextAction 中给出带 `select:` 查询的 `ops_discover_tools` 指引,不再返回无处可去的 `PUBLICATION_GRANT_UNAVAILABLE` 或光秃秃的 `POLICY_DENIED`。指引保持诚实:永远无法选中变更类能力的只读作用域仍返回通用拒绝;`select:` 查询在步骤结束提交后,延迟拒绝不再应答(#764)。 + +- Replacements fail open when the replacing executor cannot serve. A counterpart Bash call that settles with an unrecoverable preparation failure (`bash-runtime-missing`, `bash-ipc-mismatch`, `bash-lifecycle-lock-busy`, `bash-cache-unusable`, `bash-runtime-incompatible`) stops hiding the official tool from that agent's next model step, and a consumer whose Bash slot never resolved an executor keeps the official tools admitted even when a foreign same-name registration exists. Staging interruptions that the next call repairs keep the replacement in place. The lockout class of #962 — every official tool hidden behind a dead executor — is closed for these stages; a counterpart failure outside this vocabulary keeps the shadow, and the host-side rendering that ignores `when` remains upstream (#962). +- 替换在替换执行器无法服务时失效开放(fail-open):对应方 Bash 调用以不可恢复的准备失败(`bash-runtime-missing`、`bash-ipc-mismatch`、`bash-lifecycle-lock-busy`、`bash-cache-unusable`、`bash-runtime-incompatible`)结束时,该 agent 的下一模型步骤不再隐藏官方工具;Bash 槽位从未解析出执行器的消费者,即使存在同名外来注册也保持官方工具可入。下一次调用即可修复的 staging 中断不解除替换。#962 的锁死类——所有官方工具被一个死执行器挡住——对这些阶段已关闭;此词汇表之外的对应方失败仍保持影子,忽略 `when` 的宿主侧渲染仍属上游(#962)。 + +- The retained legacy file/shell row renders its tooling table from the calling scope's admitted view (`definitions` with the step tool set, falling back from `context.agent` to `context.scope`) instead of the root registration view. The host's `ctx.tools.get`/`definitions` reads ignore `when`, so a registered-but-unadmitted tool, for example a replacement shadow, could appear in the row's `read_image (host)` line while the step excluded it. The row now matches step admission; host-side sections that read registrations directly remain an upstream boundary outside this repository's control (#934). +- 保留的旧版 file/shell 行改为从调用作用域的已准入视图(带步骤 tool set 的 `definitions` 读取,并在 `context.agent` 缺席时回退 `context.scope`)渲染工具表,不再读取根注册视图。宿主的 `ctx.tools.get`/`definitions` 读取不检查 `when`,因此已注册但未准入的工具(例如替换影子)可能出现在该行的 `read_image (host)` 行里而步骤并不包含它。该行现在与步骤准入一致;直接读注册表的宿主侧提示词段落仍属上游边界,不在本仓控制范围(#934)。 - Replace the `global.forceDefaultOpsPtc` switch with one explicit action: the row's detail page carries a **Make OPS (PTC) the default** button that writes `selectedDefault: ops-ptc` once and answers `applied`/`no-update`/`unavailable`/`failed`. Apply, remount and unload no longer write the default, so a preset chosen on the settings page is not overridden on the next apply, and a saved `global.forceDefaultOpsPtc` key still loads with no effect. - 用一次显式操作取代 `global.forceDefaultOpsPtc` 开关:行详情页有一个「设为 OPS (PTC) 默认」按钮,点一次写 `selectedDefault: ops-ptc`,并回报 `applied`/`no-update`/`unavailable`/`failed`。apply、重挂载与卸载都不再改写默认,设置页选定的预设不会在下一次 apply 被覆盖;已保存的 `global.forceDefaultOpsPtc` 键仍可加载但不再生效。 +- Unloading or disabling the `dsh-ops` row no longer strands `selectedDefault: ops-ptc`: when the live default still names the removed preset, the row lease restores the host's configured default. A default naming any other preset is never rewritten, and a missing or rejecting settings service reports the leftover instead of guessing (#928). +- 禁用或卸载 `dsh-ops` 行不再遗留 `selectedDefault: ops-ptc`:若当前默认仍指向被移除的预设,行级清理会恢复宿主配置的默认值;指向任何其他预设的选择绝不改写。settings 服务缺失或拒绝写入时会明确报告残留,不会猜测修复(#928)。 +- Session Bash publication now deduplicates overlapping row ownership, carries queued registration identity to settlement, latches one failure per generation, and re-asserts authority at `turn/start`. This closes the append-reentry retry storm, duplicate-registration pair and interrupted-turn tool disappearance without weakening per-agent authority (#579, #842, #873). +- 会话 Bash 发布现在会对重叠行所有权去重,把排队注册的身份跟踪到结算,每代只报告一次失败,并在 `turn/start` 重新断言权限。修复 append 重入重试风暴、成对重复注册与中断回合后的工具消失,同时不削弱每个 agent 的权限边界(#579、#842、#873)。 +- `ops_edit_file` now accepts text copied from the reversibly escaped Native reader: raw `oldText` matches keep precedence, and only a raw miss tries the decoded rendering. The fallback also decodes `newText`, reports `renderedMatches`, and preserves ambiguity/CAS failures; raw edits and `write_file` remain byte-verbatim (#939). +- `ops_edit_file` 现在可接受从可逆转义的 Native 读取输出中复制的文本:优先匹配原始 `oldText`,只有原始匹配失败才尝试解码后的渲染形式。回退同样解码 `newText` 并返回 `renderedMatches`,歧义与 CAS 拒绝保持不变;原始编辑与 `write_file` 仍逐字节处理(#939)。 +- The bounded classic-xref PDF reader resolves indirect `/Length` values in either object order while keeping cycle, depth, type, size and mismatch refusals. Direct lengths and the existing stream/xref budgets are unchanged (#788). +- 有界的经典 xref PDF 读取器现在可按任意对象顺序解析间接 `/Length`,同时保留循环、深度、类型、大小与不一致拒绝。直接长度及现有流/xref 预算不变(#788)。 +- Paths through an Electron `.asar` archive now fail with the explicit `ARCHIVE_MEMBER_UNSUPPORTED` diagnosis: only Electron's patched in-process filesystem presents the archive as a directory, so the standalone worker cannot read its members. This is an actionable diagnostic, not a virtual-filesystem fallback (#836). +- 穿过 Electron `.asar` 的路径现在明确返回 `ARCHIVE_MEMBER_UNSUPPORTED`:只有 Electron 进程内经补丁的文件系统会把归档呈现为目录,独立 worker 无法读取成员。这是可操作的诊断,不是虚拟文件系统回退(#836)。 +- Windows mutations capture, restore and byte-verify every user-restorable extended attribute. `$KERNEL.*` names are reported individually in `metadataWarnings` because user mode cannot recreate them; every other capture/restore/verification failure still aborts publication instead of silently dropping metadata (#874). +- Windows 写入现在捕获、恢复并逐字节核验所有用户态可恢复的扩展属性。`$KERNEL.*` 名称因用户态无法重建而逐名写入 `metadataWarnings`;其他任何捕获、恢复或核验失败仍会中止发布,不会静默丢弃元数据(#874)。 +- When no grant is entitled, the publication registrar consults every same-agent `deferred()` advisor in grant order instead of only the first one it finds. An advisor that answers `undefined` or throws no longer masks a later grant's pointer: one entry can hold several same-agent file leases — a PTC pool grant and a discovery pool grant for the same public name — and a host that ignores `when` used to get `PUBLICATION_GRANT_UNAVAILABLE` (retry: never) even though a later advisor could name `ops_load_sdk` or `ops_discover_tools`. The first advisor that answers wins; when none can help, the honest generic denial stays (#968 review finding). +- 无有资格 grant 时,发布 registrar 按插入顺序逐一咨询同 agent 的每个 `deferred()` advisor,不再只问找到的第一个。返回 `undefined` 或抛错的 advisor 不再遮蔽后续 grant 的指路:同一个注册项可同时持有多个同 agent 的文件租约——例如 PTC 候选池与发现池指向同一公开名——忽略 `when` 的宿主此前会拿到 `PUBLICATION_GRANT_UNAVAILABLE`(retry: never),即使后面的 advisor 本可点名 `ops_load_sdk` 或 `ops_discover_tools`。第一个给出答案的 advisor 胜出;全都无法指路时保持诚实的通用拒绝(#968 审核发现)。 +- The batching lesson no longer rides the accepted user batch: a user-role lesson enters logged history, where the model reads it as fresh input and answers the lesson instead of the task, feeding its own loop (#929). The one-time full lesson now rides the first accepted root `run_code` tool result through `additionalContexts` — the same-step ephemeral inbox the host never logs as user history — under contributor `batching-guidance:lesson`, distinct from the per-violation notices that keep the base id. The lesson latches at the delivery point inside the post-execute decision, so the previous `session/event` `user/message` latch, which waited for durable history that no longer happens, is gone. +- 批量教学不再挂在被接受的 user batch 上:user 角色的教学会进入 logged history,模型把它当新输入去回应教学本身而不是任务,形成自持循环(#929)。一次性完整教学现在随第一个被接受的根 `run_code` 工具结果走 `additionalContexts`——宿主从不记为 user history 的同一步 ephemeral 通道——contributor 为 `batching-guidance:lesson`,与保持原 id 的逐违规提示区分。教学在 post-execute 决策内的投递点就地置位,原先等待 durable history 的 `session/event` `user/message` latch 随该 history 不再发生而移除。 + +- The Niubash face list no longer names applets the frozen winuxcmd answers with "command not found" (measured: jq, link): a generated argv0 stub for those shadowed the system command with an entry that could not run it (#966). A prepared cache that still carries a face the list no longer declares is judged damaged and rebuilt from the pinned source, so a stale stub cannot survive a cache reuse. +- Niubash face 清单不再包含冻结 winuxcmd 以 "command not found" 回应的 applet(实测:jq、link):为它们生成的 argv0 桩会以一个无法运行的入口遮蔽系统命令(#966)。已 prepared 的缓存若仍携带清单不再声明的 face,会被判损坏并从固定源重建,旧桩无法在缓存复用中存活。 + +- Activation refuses an older host once, in one named diagnosis: a version below the package's own `engines.dsh` floor (`>=0.2.2-rc.1 <0.2.3-0`) fails `apply` at its first step with the floor, the host-reported version and the upgrade path, instead of loading halfway and dying in TypeError scope errors backstage with EXECUTOR_UNAVAILABLE in front (#701). An unreadable manifest or a version the host will not reveal cannot be proven below the floor, so both load as before; the range stays the single source of truth, read from the manifest beside the running build, never a copy. +- 激活期具名拒绝一次过旧宿主:低于本包自身 `engines.dsh` 下界(`>=0.2.2-rc.1 <0.2.3-0`)的版本让 `apply` 在第一步即失败,诊断给出下限、宿主实测版本与升级路径,而不是加载到一半在后场死于 TypeError scope 错误、前场只看到 EXECUTOR_UNAVAILABLE(#701)。清单不可读或宿主不肯报版本时无法证明低于下限,两者照旧加载;范围以清单为唯一事实源,从运行构建旁的 manifest 读取,绝不复制副本。 + +- A built-in preset session now answers `ops_status` about its own scope: the diagnosis tool was registered only behind an external preset row, while the diagnosis itself already understood mode sessions, so a session on a built-in preset had no named answer at all (#753). The built-in mode row mounts the same read-only diagnosis for exactly its own mode's sessions, and the mount settles asynchronously because a fiber that resolves synchronously never hears the `agent/created` dispatch the injection registered. Sessions of other modes stay unadmitted by that row, and the external preset row keeps ownership where it exists. +- 内置 preset 会话现在能对自身作用域应答 `ops_status`:诊断工具此前只随外部 preset 行注册,而诊断本身早已理解 mode 会话,落在内置 preset 上的会话完全没有具名答案(#753)。内置 mode 行为且仅为该 mode 的会话挂载同一只读诊断;挂载特意异步落定,因为同步 resolve 的 fiber 永远收不到注入里注册的 `agent/created` 派发。其他 mode 的会话仍不被该行接纳,外部 preset 行存在时继续保持其所有权。 +- New OPS-PTC acceptance test infrastructure: `test/lib/ptc-entry.mjs` boots one preset group on the real agent loop with the shipped `ops-ptc` row, inside each suite's throwaway `DSH_HOME`, and records the environment a measurement must name — plugin commit, host version, platform, the worker's sha256 with its staged receipt, sandbox mode, the row's resolved switches and the actual `agentPreset`. The minimal case is one scripted provider request proving the top-level transport is `run_code`; a group that lands on Standard, on a native executor, or without an executable worker is reported invalid with its own reason and carries no comparable output, so a wrong configuration can no longer pass as OPS-PTC evidence. `test/progressive-loop.test.mjs` asserts all of this (#990; see `docs/isolated-ptc-entry.md`). +- 新增 OPS-PTC 验收测试基础设施:`test/lib/ptc-entry.mjs` 在每套件独占的临时 `DSH_HOME` 内,用正式 `ops-ptc` 行装配真实 agent loop 的一个组,并记录测量必须交代的环境——插件 commit、宿主版本、平台、worker 的 sha256 及 staged 回执、沙箱模式、行内实际开关与实际 `agentPreset`。最小用例是一次脚本化 provider 请求,先证明顶层传输是 `run_code`;落在 Standard、native executor 或无可执行 worker 上的组按各自原因判「无效」且不带任何可比结果,错误配置不再能冒充 OPS-PTC 证据。`test/progressive-loop.test.mjs` 对此逐项断言(#990,见 `docs/隔离测试入口.md`)。 + +- New bench-only behavior corpus for the OPS-PTC cards (#993): four tasks in `bench/tasks/` — paged cursor continuation, catalog-error recovery, load-before-use and wait/multi-dispatch — whose prompts state only the business goal, plus a `pagedLog` fixture family whose answer derives from its bytes. `bench/lib/tracks.mjs` judges a scripted trajectory against each task's fixed facts (cursor advance in the next program on the same path, complete-page stop, advertised-set recovery, load order, revision-awaited edits, verbatim no-progress retries versus legitimate bounded-transient waits) and names the fact that did not hold. `node bench/cli.mjs tracks` runs the ten scripted pass/fail trajectories and exits non-zero when a verdict disagrees; the recovery rate counts only trajectories that actually had a correction opportunity and prints an explicit no-denominator line instead of 100%. `test/bench-tracks.test.mjs` pins both states. `bench/` ships in no npm package; no plugin behaviour changed. +- 新增仅限仓库内的 OPS-PTC 行为语料(#993):`bench/tasks/` 下四个任务——分页游标续读、目录错误纠偏、先加载后调用、等待/多派发——题面只写业务目标,另加一个答案可从字节反推的 `pagedLog` 夹具族。`bench/lib/tracks.mjs` 按各任务固定事实判定脚本化轨迹(下一程序同路径续读光标、末页即停、纠偏只认当时可用集合、加载先于调用、编辑等待读取返回的 revision、原样重读属无进展重试而 bounded-transient 等待不算),失败输出指明是哪项事实不符。`node bench/cli.mjs tracks` 跑十条应过/应败轨迹,任何判定与声明不符即退非零;恢复率只统计真的出现过纠偏机会的轨迹,无分母时明说「no denominator」而不是报 100%。`test/bench-tracks.test.mjs` 把两种状态都钉住。`bench/` 不随 npm 包发布,插件行为未变。 + ## [0.5.5] — 2026-11-13 — Patch release - A loader-less PTC scope no longer registers the configured `allowedTools` pool: with dynamic loading off nothing could ever select those APIs, and a host assembly without a step tool set declared all sixteen in `tools:sdk`. Resident publication, the default inspect + loader set and selection behavior are unchanged (#887). diff --git "a/docs/Flash\345\257\271\347\205\247-V5.md" "b/docs/Flash\345\257\271\347\205\247-V5.md" new file mode 100644 index 00000000..e24de0b7 --- /dev/null +++ "b/docs/Flash\345\257\271\347\205\247-V5.md" @@ -0,0 +1,154 @@ +# Flash 小规模对照(V5) + +[English](flash-comparison-v5.md) | 中文 + +议题 [#994](https://github.com/T-Auto/dsh-ops/issues/994),父单 [#980](https://github.com/T-Auto/dsh-ops/issues/980)。这是一张测量卡: +仓库仅新增测量驱动、采集器、本对报告与 changelog 条目;不改产品代码语义、不改教学文案、不改发布逻辑。 + +## 预注册的问题 + +V5 卡要回答:按需披露档的**更短首轮提示**是否换来**更多调用或更多失败**。两臂只差一个预注册因素—— +`dsh-ops` 行的 `disclosurePolicy`(`on-demand` vs `full`,由 #986/#987 实现)——其余全部冻结: + +- 路由:`opencode-go` / `deepseek-v4.1-flash`,`reasoningEffort: high` +- 预设:`ops-ptc`(组合组自证;若实际落到 Standard,按 #980 判无效) +- 沙箱:`danger-full-access`;平台:`linux-x64` +- 任务:`ptc-catalog-recover`(目录搜索,需要可加载的 `ops_grep`)与 `ptc-wait-dispatch` + (多层配置编辑,需要 `ops_read_files`/`ops_edit_file`)——四条 V4 轨道里工作集对档位最敏感的两条 +- 计划:2 任务 × 3 重复 × 2 臂,按 (任务, 重复) AB/BA 交错,共 12 会话;另跑同条件 A/A + (12 会话,按需配置跑两臂)估计噪声底;每会话使用全新播种工作区与全新组合组 + +## 为什么是驱动器而不是 `bench ab` + +宿主的一次性 headless runner 只组合 host realm,并**按设计拒绝 preset 会话**(其收养检查原话: +*"session … runs under agent preset …, which the one-shot runner does not compose"*)。而插件对无注册表的 +host realm 按 Standard 处理,所以 `ab` 实测的是 Standard,按 #980 判无效。`bench/v5-run.mjs` 复用 V1 +隔离入口组合(真实 AgentLoop、真实 PTC executor、真实 staged worker、正式 `dsh-ops` 行),只把脚本化 +provider 换成真实 `llm-pi-ai` 适配器。一个解析细节:运行时把 `@deepseek-ai/dsh-llm-pi-ai`(连同 +`@deepseek-ai/dsh-credentials`,以及 `@earendil-works/pi-ai` 的符号链接)复制进 `node_modules`, +使全部 import 经本检出树解析、整个组合共享同一个 `cordis` 实例;`node_modules` 被 gitignore, +manifest 不变。每会话落盘:原始事件(`*.events.ndjson`)、可判定轨迹(`*.trajectory.json`)、 +折叠指标(`*.metrics.json`)、经验证的答案,以及每臂 `arms//dump-config.txt`。 + +## 环境事实(记录而非声称) + +| 事实 | 值 | +| --- | --- | +| 插件 commit | `card/v5-flash-comparison` 上 `28c0cf7` = `main@58f6e28` + 已过审前置合并(V1 入口与 V4 轨道 `card/v4-bench-tasks`@`ec4d25c`;被测 D2/D3 披露实现 `card/v3-two-tier-matrix`@`c68fe3d`) | +| 宿主版本 | 0.2.2-rc.1 | +| 平台 | linux-x64,node v24.21.0 | +| worker | staged `runtime/linux-x64/dsh-ops-worker`,sha256 `2b5f988d9115…` | +| 预设 / 路由 | `ops-ptc`;`{provider: opencode-go, model: deepseek-v4.1-flash, reasoningEffort: high}` | +| 沙箱 | danger-full-access、approval never(组合组内钉住模式;未挂审批服务,工具失败为返回值) | + +每臂证据:`bench/runs/v5-comparison/arms/{ondemand,full}/dump-config.txt` 与 +`bench/runs/v5-aa/arms/{aa-one,aa-two}/dump-config.txt`——除一行 `disclosurePolicy` 外逐字相同。 + +## 完整分母 + +| sweep | 计划 | 完成 | 超时 | 缺 usage | 验证通过 | +| --- | --- | --- | --- | --- | --- | +| 对照(按需 vs 全量) | 12 | 12 | 0 | 0 | 11 | +| A/A(按需跑两臂) | 12 | 12 | 0 | 0 | 12 | +| **合计** | **24** | **24** | **0** | **0** | **23** | + +一个对照会话未通过工作区验证:按需臂 `ptc-wait-dispatch` r2 写了 `config/development.json`(选错了 +配置层);会话本身正常完成,因此同时留在完成率与通过率分母里。没有会话超时;每会话每步都报了 +usage,下表没有任何下界问题。正式跑前先做了 2 会话冒烟(`bench/runs/v5-smoke`)验证路由与 usage +采集,冒烟不在预注册分母内。 + +## 按任务结果 + +提示规模为每次模型请求的 token(input + cacheRead + cacheWrite);总量为单会话全部步之和,取臂内三会话中位数。 + +### ptc-catalog-recover + +| 指标 | 按需 | 全量 | A/A 带宽(同配置两臂) | +| --- | --- | --- | --- | +| 验证通过 | 3/3 | 3/3 | 3/3 – 3/3 | +| 正确下一步 | 26 | 17 | 23 – 24 | +| 纠偏机会 | 3 | 0 | 0 – 1 | +| 无进展重试 | 5 | 4 | 5 – 7 | +| 等待 | 0 | 0 | 0 | +| 首轮提示(中位) | 5,271 | 29,946 | 5,271 – 5,271 | +| 提示总量(中位) | 86,136 | 185,465 | 66,881 – 88,078 | +| input tokens(3 会话) | 65,035 | 68,051 | 55,814 – 57,644 | +| cacheRead tokens(3 会话) | 182,656 | 430,848 | 144,896 – 198,016 | + +### ptc-wait-dispatch + +| 指标 | 按需 | 全量 | A/A 带宽 | +| --- | --- | --- | --- | +| 验证通过 | 2/3 | 3/3 | 3/3 – 3/3 | +| 正确下一步 | 8 | 8 | 14 – 27 | +| 纠偏机会 | 1 | 2 | 4 – 6 | +| 无进展重试 | 7 | 6 | 7 – 9 | +| 等待 | 0 | 0 | 0 | +| 首轮提示(中位) | 5,306 | 29,981 | 5,306 – 5,306 | +| 提示总量(中位) | 137,419 | 236,416 | 85,813 – 103,238 | +| input tokens(3 会话) | 117,307 | 106,664 | 65,094 – 125,645 | +| cacheRead tokens(3 会话) | 278,784 | 597,248 | 203,776 – 244,736 | + +### 首轮与加载后的差值 + +`#994` 第 3 步要求列首轮与加载后的差值。按需臂每会话在第一个程序里完成一次 `ops_load_sdk`, +下一个请求因新加载的 SDK 声明加历史而增长: + +| 任务 | 首轮(中位) | 加载完成后的首个请求(中位) | +| --- | --- | --- | +| ptc-catalog-recover | 5,271 | 12,820 | +| ptc-wait-dispatch | 5,306 | 15,574 | + +全量臂按设计没有加载步;首轮即携带整个获准池(29,946 / 29,981 tokens)。A/A 两臂复现按需臂数值 +(12,733–12,786 / 15,437–15,568),这同时也是采集路径的一致性检查。 + +### Cache usage + +每步都报了 usage 对象;每步都有 `cacheReadTokens`,`cacheWriteTokens` 全程为 0(provider 用缓存服务了 +重复的前缀)。缺 usage 行:0。尽管 `reasoningEffort: high`,`reasoningTokens` 每步都为 0——按原样记录, +不做重新解读。 + +## 噪声判定 + +- **未能区分(落在 A/A 噪声内):** `ptc-wait-dispatch` 的验证通过差(2/3 vs 3/3,A/A 带 3/3–3/3, + 即 n=3 下的单次失败;该失败是选错配置层的推理失误,与档位无交互);两任务的无进展重试 + (5 vs 4、7 vs 6,带宽 5–7 与 7–9);`ptc-wait-dispatch` 的纠偏机会(1 vs 2,带宽 4–6,两臂都低于 + A/A 离散本身);两任务的裸 input tokens(仅 `ptc-wait-dispatch` 的 A/A 带就跨 65k–126k)。 +- **超出 A/A 带,但属机械差异,不作模型质量归因:** 首轮提示(5.3k vs 30.0k,带 5.3k–5.3k)与提示 + 总量(86k vs 185k、137k vs 236k,带宽 67k–88k 与 86k–103k)——这是披露档自身的声明成本,输入侧 + 事实而非行为差异;cacheRead 随之(两任务皆 2.4×)。目录任务的纠偏机会(3 vs 0,带 0–1)是按需 + loader 每会话恰好邀请一次可恢复的输入错误(首次 `ops_load_sdk` 名字非法)——三次全部恢复并通过, + 恢复率 3/3,与从未获得机会的全量臂不可区分。目录任务的正确下一步计数(26 vs 17,带 23–24)包含 + 仅存在于按需臂的加载步与恢复步,因此如实报告、不作归因。 +- **不声称档位带来成功率提升。** 每臂每任务 n=3,任何行为侧差异都没有超出 A/A 带所能覆盖的范围。 + +## 复跑 + +```console +npm run build && npm run build:ptc +# 对照 sweep:2 任务 × 3 重复 × 2 臂,AB/BA 交错 +node bench/v5-run.mjs --task ptc-catalog-recover --task ptc-wait-dispatch --repeat 3 \ + --out bench/runs/v5-comparison --dsh-home <.credentials.yaml 内含 OPENCODE_GO_API_KEY 的 home> +# A/A 噪声 sweep:同计划,两臂都用按需配置 +node bench/v5-run.mjs --task ptc-catalog-recover --task ptc-wait-dispatch --repeat 3 --aa \ + --out bench/runs/v5-aa --dsh-home <同一 home> +# 判定与表格 +node bench/v5-collect.mjs --run bench/runs/v5-comparison --run bench/runs/v5-aa +``` + +本报告运行的产物在 `bench/runs/v5-comparison` 与 `bench/runs/v5-aa`(`summary.json`、逐会话 `results/*`、 +每臂 `dump-config.txt`),冒烟在 `bench/runs/v5-smoke`。判定器逐字复用 V4 共享规则 +(`bench/lib/tracks.mjs` 的 `scanReissues`、`countOpportunities`)与两条可泛化的任务规则(分页游标、 +等待/派发)。两条剧本专属规则——目录轨道的"必须先犯错"与加载轨道的"必须先撞未加载拒绝"——有意 +不对真实运行生效:从不犯错的模型不应被判为演砸剧本。 + +## 局限 + +- 每臂每任务 n=3:#994 规定的试点规模。A/A 带展示了该 n 下噪声底有多宽;本报告不支持任何成功率结论。 +- 组合组是 V1 隔离入口,比 web profile 的完整 `ops-ptc` 预设精简(无 persona、time-context、Bash 行)。 + 两臂共享精简组合,档位对比内部有效;绝对 token 数描述的是隔离组合,不是 web profile。 +- 单机、单日、单模型:AB/BA 交错缓解漂移,不消除漂移。 +- 判定器的目录"正确下一步"定义把完成的 `ops_load_sdk` 计为按需臂的正确一步;该列跨臂比较含结构性 + 成分(上文已注明)。 +- 驱动器在运行时把 `dsh-llm-pi-ai` vendor 进 `node_modules`,因为仓库 manifest 有意不依赖任何 LLM + 适配器;该拷贝被 gitignore,并在此处记录。 diff --git "a/docs/Native\350\276\223\345\207\272.md" "b/docs/Native\350\276\223\345\207\272.md" index d87e4ea5..2688f09c 100644 --- "a/docs/Native\350\276\223\345\207\272.md" +++ "b/docs/Native\350\276\223\345\207\272.md" @@ -50,3 +50,13 @@ OPS 路由已经使用 `publicToolName()`,并按当前步骤实际准入的定 注册表中存在,而实际 Native 请求只准入 `ops_inspect_local_file`。测试通过真实宿主 插件复现该不一致,并记录提示词节归属。让宿主回调遵守当前步骤准入仍是 #866 的宿主 依赖。OPS 不重写第三方提示词、不修改共享注册表,也不增加常驻名称对照提示词。 + +该不一致的 PTC 一侧已在受支持的装配扩展点收口(#983, +`src/publication/replacements.ts`):shadow 在 PTC 呈现下隐藏 +`read`/`glob`/`grep` 时,装配提示词移除宿主的 +`tool:read`/`tool:glob`/`tool:grep` 段,不再教模型调用已隐藏的名字。保留工具的段若 +仍指向已隐藏名字——固定宿主 grep 文案交叉引用 read——该段一并移除,并补一行指名可执行 +的 OPS 替代;不含死指针的保留段原样存活。撤回替代后宿主文案逐字恢复。 +`test/ptc-guidance-cleanup.test.mjs` 用真实 `@deepseek-ai/dsh-tool-fs`/`-fs-search` +插件钉住该行为,含按段分组的字节差(Linux fixture:SDK −2,011 B、三段指引 −338 B、 +外层 wire 0 B)。#866 对 Native 呈现保持开放:宿主段在那里刻意不动。 diff --git "a/docs/PTC\347\273\223\346\236\234\345\245\221\347\272\246.md" "b/docs/PTC\347\273\223\346\236\234\345\245\221\347\272\246.md" index ce827b5d..69a4928f 100644 --- "a/docs/PTC\347\273\223\346\236\234\345\245\221\347\272\246.md" +++ "b/docs/PTC\347\273\223\346\236\234\345\245\221\347\272\246.md" @@ -11,7 +11,7 @@ English: [PTC result contracts](ptc-outcomes.md). | 状态 | 必需字段 | 调用方操作 | | --- | --- | --- | | `completed` | `data` | 处理本次请求对应的结果。 | -| `partial` | `data`、不完整的 `coverage`、`continuation` | 检查已保留数据,显式读取剩余范围或引用页。 | +| `partial` | `data`、不完整的 `coverage`、`continuation`、可选 `nextAction` | 按 `nextAction` 给出的显式调用续读;报告 `complete=true` 的那一页就是最后一页。 | | `rejected`、`conflict`、`error`、`unknown` | `error`,可选保留 `data` | 根据 `code`、`effectState`、`retry` 和 `nextAction` 处理。 | 成功的 plan 只表示规划完成,不表示修改已执行。`result.put` 不回显行仍可表示存储完成。 @@ -39,6 +39,13 @@ return { code: result.error.code, effectState: result.error.effectState, 物理搜索未完成、描述符过大或无法安全续取时,返回 `RESULT_PARTIAL`、保留数据与明确指引, 不伪造游标。每次续取仍核实当前权限与 SDK 可见性;没有自动执行或自动重试工具的辅助器。 +`nextAction` 推荐是可执行的,不是装饰:它给出带字面参数的具体调用——分页读取按同一路径的 +`nextOffset` 续读、非 UTF-8 字节改用 hex `view`、目录路径被拒时指向本步已注册的列举 API、 +`REVISION_CONFLICT` 指向重读最新 revision。推荐尊重当前步骤的可达集合:本步未准入的 API +绝不会出现在文案里(此时直接要求提供具体文件路径),任何写类 API 都不会被推荐,只读作用域 +不会收到重新提交的建议。缺少 worker 指引的 `after-input-change` 失败会直说原样请求必然 +同样失败;已有明确恢复指引的错误保持原文不变。 + 搜索扫描覆盖、快照捕获完整性和页面完整性是不同事实。读完已存储页不代表原始扫描完整。 使用快照时检查 `data.source` 和原始搜索报告,详见[结果预算](结果预算.md)。 diff --git "a/docs/WSL\345\217\252\350\257\273\346\226\207\344\273\266\346\223\215\344\275\234.md" "b/docs/WSL\345\217\252\350\257\273\346\226\207\344\273\266\346\223\215\344\275\234.md" index 7c4823fd..e1ebb45e 100644 --- "a/docs/WSL\345\217\252\350\257\273\346\226\207\344\273\266\346\223\215\344\275\234.md" +++ "b/docs/WSL\345\217\252\350\257\273\346\226\207\344\273\266\346\223\215\344\275\234.md" @@ -1,6 +1,6 @@ # 可选 WSL 只读文件操作 -实测 DSH rc.2 宿主无法通过 Windows `Get-Acl` 准备 WSL UNC 工作区。因此 workspace-write 下原有 PTC 运行时会拒绝启动,旧版缓存准备也可能不可用。内联 Native 读取仍可运行;字节裁剪的请求回退后与原 Windows 结果完全一致,包括输出不完整标志。OPS 不修改权限模式,也不绕过宿主拒绝。独立 full-access 夹具验证共享 PTC 文件绑定;这是测试范围说明,不是扩大用户权限的建议。 +Windows `Get-Acl` 无法准备 WSL UNC 工作区:`\\wsl.localhost\<发行版>` 与 `\\wsl$\<发行版>` 是 9P 重定向器,其后端文件系统根本不暴露 NTFS 安全描述符,未加区分的准备脚本因此在每个 WSL 工作区上直接死掉,workspace-write 下的 `run_code` 全部失败(#669)。工作区引导脚本现在会在完成存在性与目录验证之后、触碰 `Get-Acl` 之前,对恰好这两个微软固定前缀直接回答 `ready`——这是前缀判定,不是吞掉 ACL 错误的 catch:不存在的路径与其他任何缺失工作区一样失败,其他任何路径保持原有行为,真实的 ACL 故障依然闭锁失败。此类根目录上的旧版缓存准备仍可能不可用。内联 Native 读取可运行;字节裁剪的请求回退后与原 Windows 结果完全一致,包括输出不完整标志。OPS 不修改权限模式,也不绕过宿主拒绝。独立 full-access 夹具验证共享 PTC 文件绑定;这是测试范围说明,不是扩大用户权限的建议。 full-access 下,缺少 `sandbox.prepareResultStorage` 的宿主有意不启用可选存储,避免仅为缓存准备工作区 ACL(#411)。这两种旧宿主情形下,WSL 验收脚本会验证未授予存储权限,并逐字段对比关闭 WSL 开关后的 Windows 结果。宿主提供仅缓存准备能力时,缺少引用仍会导致验收失败,且必须能够分页。Windows 缓存契约套件另以该能力的测试实现验证真实私有 ACL 准备、溢出、分页和清理;这不代表已发布的宿主提供了该能力。 @@ -12,9 +12,9 @@ English: [Optional WSL file reads](wsl-readonly.md). 在 Windows x64 上,可在启动 DSH 的环境中设置 `DSH_OPS_WSL_READS=1`,为工作目录位于 `\\wsl.localhost\<发行版>\...` 或 `\\wsl$\<发行版>\...` 的会话启用 Linux 侧 `ops_glob`、`ops_grep` 和 `ops_read_files`。此功能主要减少 WSL 网络文件系统上的重复元数据访问。WSL 启动和校验也有固定开销,小文件读取可能更慢;应先测量实际工作目录再广泛启用。 -安装包必须同时包含普通 Windows worker,以及版本一致、凭据文件校验通过的 `runtime/linux-x64/dsh-ops-worker`。发行版须已安装、支持 Linux x64,并提供 `/usr/bin/wslpath` 和 `/usr/bin/sha256sum`。OPS 不安装 WSL,也不下载运行时。显式 `workerPath` 或 `DSH_OPS_WORKER_BIN` 选择仍使用原有 Windows 路径。关闭环境开关后,新请求恢复原有路径。 +安装包必须同时包含普通 Windows worker,以及版本一致、凭据文件校验通过的 `runtime/linux-x64/dsh-ops-worker`。发行版须已安装、支持 Linux x64,并提供 `/bin/sh`、`/usr/bin/wslpath` 与 `/usr/bin/sha256sum`。OPS 不安装 WSL,也不下载运行时。显式 `workerPath` 或 `DSH_OPS_WORKER_BIN` 选择仍使用原有 Windows 路径。关闭环境开关后,新请求恢复原有路径。 -worker 通过 `wsl.exe --distribution <名称> --exec <已校验的 worker>` 接收 JSON 数据,不运行命令 shell 或模型提供的代码。输入路径须属于会话所在发行版。每次调用都会重新解析 Windows 盘符挂载,包括凭据拒绝根和 worker 路径,并核对 Linux 侧可见文件的 SHA-256。Linux 遍历仍采用 Windows 不区分大小写的凭据拒绝规则和保留文件名限制。符号链接、特殊对象仍执行 no-follow;无效文件名属于扫描缺口,不得宣称计数完整。 +worker 通过 `wsl.exe --distribution <名称> --exec <已校验的 worker>` 接收 JSON 数据;worker 本身不运行命令 shell 或模型提供的代码。准备工作是每次调用一次固定的 `/bin/sh -c` 脚本(#576):脚本文本是常量,所有盘符根、凭据拒绝根与 worker 路径都以位置参数传递,任何 Windows 路径都无法改变命令形状——盘符挂载仍按每次调用重新解析,Linux 侧可见文件的 SHA-256 仍须与打包凭据一致。输入路径须属于会话所在发行版。Linux 遍历仍采用 Windows 不区分大小写的凭据拒绝规则和保留文件名限制。符号链接、特殊对象仍执行 no-follow;无效文件名属于扫描缺口,不得宣称计数完整。 工具名称、参数 schema、提示词、相对路径基准及宿主读取记录不变。只转换已定义的结果路径字段为 UNC,不改写文件内容。写入、Bash、PTC 执行和结果缓存所有权仍保留在原有 Windows 路径。 diff --git a/docs/component-reference.md b/docs/component-reference.md index 33fde04a..0aab3d35 100644 --- a/docs/component-reference.md +++ b/docs/component-reference.md @@ -19,7 +19,7 @@ The `dsh-ops` row owns built-in and explicitly configured consumer plans. Native `dsh-ops/shell`, `dsh-ops/file`, `dsh-ops/ptc` and `dsh-ops/hide-official` modules remain as unsupported legacy exports for old profiles and are not part of the bundle. -`ops-ptc` is the merged row's fifth built-in mode: the plugin registers the preset at runtime through the host `agentPresets.register` (definition from the shipped `presets/ops-ptc.patch.yml` fragment parsed by the host's `loadOverlayPatches`), so the plugin manager has no matching preset row; disabling or unloading the plugin row removes the preset together with the tool surface. The default preset is never written automatically: the detail page's button makes `ops-ptc` the default once (`applied`/`no-update`/`unavailable`/`failed`), while apply, remount and unload leave the operator's own choice alone; see [Presets and the default mode](presets.md). It publishes the same surface as `ptc` and adds the YAML-only `batchingGuidance`: the English lesson arrives once in the session's first user-message batch (from `prompts/batching-guidance.md`, into history, never the system prompt), and a **root** `run_code` program that breaks one of the three latched shape categories — `bypass`, `plan` or `shape` — gets one short notice appended to its own result as `additionalContexts`; each category latches once per turn, a compliant program re-arms it, priority is `bypass` → `plan` → `shape`, and the program is never blocked or rewritten. A legacy profile that still names `dsh-ops/mode-ops-ptc` loads that module; under the merged bundle it only warns and stays inert, and its configuration belongs in the merged row's `ops-ptc` section. +`ops-ptc` is the merged row's fifth built-in mode: the plugin registers the preset at runtime through the host `agentPresets.register` (definition from the shipped `presets/ops-ptc.patch.yml` fragment parsed by the host's `loadOverlayPatches`), so the plugin manager has no matching preset row; disabling or unloading the plugin row removes the preset together with the tool surface. The default preset is never written automatically: the detail page's button makes `ops-ptc` the default once (`applied`/`no-update`/`unavailable`/`failed`), while apply, remount and unload leave the operator's own choice alone; see [Presets and the default mode](presets.md). It publishes the same surface as `ptc` and adds the YAML-only `batchingGuidance`: the English lesson arrives once on the first accepted root `run_code` result as `additionalContexts` (from `prompts/batching-guidance.md`; same-step inbox, never logged history, never the system prompt), and a **root** `run_code` program that breaks one of the three latched shape categories — `bypass`, `plan` or `shape` — gets one short notice appended to its own result as `additionalContexts`; each category latches once per turn, a compliant program re-arms it, priority is `bypass` → `plan` → `shape`, and the program is never blocked or rewritten. A legacy profile that still names `dsh-ops/mode-ops-ptc` loads that module; under the merged bundle it only warns and stays inert, and its configuration belongs in the merged row's `ops-ptc` section. The former `dsh-ops/background` component (FastCtx background jobs: `ops_run`, `ops_run_background`, `ops_job_output`, `ops_job_list`, `ops_job_kill`) has been removed. A profile row that still names it loads as a no-op: it accepts any config, publishes nothing, and logs one warning that the row can be deleted. @@ -247,9 +247,14 @@ The plugin renders routing guidance from what it actually published. The excerpt Standard and Cordis replace the official `read`, `glob` and `grep` with these reviewed read/search workflows; every enabled mode replaces the Windows official shell. Their names, arguments and results -differ from the host tools; calls are never translated or forwarded under an official name. A -`run_code` presentation (PTC) is never masked: its transport is not gated, while the replaced -official names stay refused in that scope. The replacement contract covers native presentation only. +differ from the host tools; calls are never translated or forwarded under an official name. The +`run_code` transport itself is never masked or gated. The replacement table covers `read`, `glob` and +`grep` only — `write`, `edit` and `read_image` are never in it — and the replaced officials are hidden +only after an explicit opt-in: `hideOfficialTools` for the built-in modes, `replaceOfficial` (default +`false`) for explicit consumers; by default every official tool stays published. The same opt-in +covers both presentations: under PTC an official name leaves the surface only while the projection can +serve the matching `ops_*` API, and an unattached, probe-failing or narrowed projection keeps the +official tool. | Official tool | OPS alternative | Reviewed purpose | diff --git a/docs/configuration-reference.md b/docs/configuration-reference.md index 1d62646f..3503ac27 100644 --- a/docs/configuration-reference.md +++ b/docs/configuration-reference.md @@ -72,9 +72,10 @@ the row's status card, which sets it once when pressed. Generation and the manua | `residentTools` | true | Publish the startup APIs. The PTC resident surface is only `ops_inspect_local_file`; everything else loads on demand through `ops_load_sdk` | | `bash` | true | Allow publishing `ops_bash` on Windows; POSIX sessions keep the host bash | | `hidePwsh` | true | Hide the official Windows shell while Bash is available; disabled entirely when `global.hidePwsh` is false | -| `hideOfficialTools` | true | Hide the official file tools this mode replaces; disabled entirely when `global.hideOfficialTools` is false | +| `hideOfficialTools` | true | Hide the official file tools this mode replaces, including PTC presentation: when the projection can serve the matching `ops_*` API, official `read/glob/grep` leave the tool surface and the SDK declaration. `write`/`edit`/`read_image` stay; a name the projection cannot serve keeps its official tool (an unattached projection, `glob`/`grep` under `dynamicTools: false`, or a name excluded by a narrowed `allowedTools`). Disabled entirely when `global.hideOfficialTools` is false | | `dynamicTools` | true | Publish the progressive SDK loader `ops_load_sdk`; effective only while `global.dynamicTools` is also on (AND gate) | -| `batchingGuidance` | true for `ops-ptc`, false otherwise | Teach once at session start (logged history, system prompt untouched), then correct the shape of root `run_code` programs: `bypass`, `plan` and `shape` each latch once per turn and re-arm when the program complies; the notice rides the triggering result as `additionalContexts` and never blocks or rewrites the program. Text lives in the editable `prompts/batching-guidance.md`. YAML-only; an omitted `ops-ptc` section in a legacy whole-row profile override retains true; explicit false disables it | +| `disclosurePolicy` | `on-demand` | YAML-only explicit first-round disclosure tier: `on-demand` keeps the legacy progressive path (resident + loader); `full` publishes the whole `allowedTools` pool plus `result` in the first round with no loader (implemented; see the D3 addendum in docs/design-decisions-20261119.md). Effective only while `global.dynamicTools` is on — the explicit opt-in never overrides the bundle switch; while it is off the row keeps its legacy resident/always-loaded surface and the saved choice is preserved. Rejected on non-PTC rows; illegal values are errors naming the key and the received value or type. No upgrade, settings save or switch flip ever writes this key for you | +| `batchingGuidance` | true for `ops-ptc`, false otherwise | Teach once, riding the first accepted root `run_code` result as `additionalContexts` (same-step inbox, never logged history; system prompt untouched), then correct the shape of root `run_code` programs: `bypass`, `plan` and `shape` each latch once per turn and re-arm when the program complies; the notice rides the triggering result as `additionalContexts` and never blocks or rewrites the program. Text lives in the editable `prompts/batching-guidance.md`. YAML-only; an omitted `ops-ptc` section in a legacy whole-row profile override retains true; explicit false disables it | | `spillRetentionDays` | 30 | Host subprocess spill retention window; `0` reports only and never deletes | | `spillTotalBudgetMiB` | 256 | Total retained spill budget in MiB; `0` disables the budget | | `spillSweepEnabled` | true | Run one bounded sweep on activation and every 256 settlements | @@ -85,15 +86,27 @@ Native guidance; `allowedTools`/`alwaysLoaded` accept loadable raw API names onl [Progressive PTC SDK loading](ptc-sdk-loading.md)). The three `spill*` keys bound host subprocess spill retention, described under built-in advanced configuration below. +The exact first-round `ops_*` name sets for the five legacy inputs — defaults, +`global.dynamicTools:false`, row `dynamicTools:false`, that row plus `alwaysLoaded:[stat]`, and +`residentTools:false` — are locked letter by letter in the +[D1 truth table](design-decisions-20261119.md). Old `dynamicTools:false` publishes only the +resident/always-loaded surface and must not be read as a full selection; the explicit full tier +is a separate `disclosurePolicy` opt-in whose parsing contract and conservative precedence are +recorded in the same document's D2 addendum, with publication owned by the D3 slice. + ### Batch-call guidance (`batchingGuidance`) -Off contributes no extra context. The code default is on for `ops-ptc` only, even when a legacy profile replaces the whole row config without that section; explicit `batchingGuidance: false` still disables it. On, the guidance is delivered in two places, from an +Off contributes no extra context. The code default is on for `ops-ptc` only, even when a legacy profile replaces the whole row config without that section; explicit `batchingGuidance: false` still disables it. On, two kinds of guidance ride tool results as `additionalContexts`, rendered from one external English Markdown file: -- **Once at session start**: one plugin-context message joins the session's first accepted non-empty - user-message batch (step-tail, source `dsh-ops:step-tail` / `batching-guidance`). It enters logged history, - so the model's first request already sees the whole procedure: the first-line `// plan:` comment, the - dependency split, the load-before-call step and four copyable trajectories. +- **Once per agent entry**: one plugin-context message rides the first accepted root `run_code` result as + `additionalContexts` (source `dsh-ops:step-tail`, contributor `batching-guidance:lesson`). That inbox is + same-step only and never logged as user history, so the model sees the whole procedure on the request that + follows the first accepted call: the first-line `// plan:` comment, the dependency split, the + load-before-call step (only while dynamic loading publishes the loader) and four copyable trajectories. + The first-round text teaches each rule once (revision 7): the anti-pattern rewrite and the prose that + repeated the dependency split are gone, the core rules state load-only-missing and recheck-after-a-mode-change + explicitly, and the four trajectories ship byte-identical. - **On a thin program**: when a **root** `run_code` program breaks one of three latched categories, a short notice is appended to that call's own result as `additionalContexts`. The tool result keeps its value, its content and its existing contexts; the program is never blocked or rewritten. @@ -107,9 +120,23 @@ external English Markdown file: - The analysis is a literal heuristic over the program text: a loop or `map` counts one dispatch site, not its runtime fan-out, and strings or comments can mislead it. Notices say so, nothing claims a measured dispatch count, and a cancelled or never-dispatched call is not analysed. -- Everything is plugin-context history and the tool schema is untouched, so the system prompt and its cache - survive. The lesson counts as spent only once it appears as a durable `user/message`; a batch withdrawn - before admission (cancellation, rejection) leaves the next accepted batch free to teach it again. +- Everything rides `additionalContexts` on the tool result and the tool schema is untouched, so the system + prompt and its cache survive. The lesson is spent at the delivery point — the accepted root `run_code` + decision — so a call that is cancelled or never accepted leaves the next accepted call free to teach it. +- The text follows the published surface. While dynamic loading publishes `ops_load_sdk`, the lesson ships + byte for byte, load step included. With `dynamicTools: false` the scope publishes no loader, so that step and + its table row are replaced by a direct-call section that lists the APIs this scope actually publishes; a + scope that publishes nothing says so and points at the official tools. When the only file API a loader-less + scope publishes is `ops_inspect_local_file`, that section states its contract too: one file by exact path, + directories rejected, and with no `ops_glob` or `ops_grep` published nothing here lists or searches, so + file work needs concrete paths. The program-shape notices carry the same surface, so `bypass` names neither a + loader nor an API the scope cannot reach: its file-work line lists only the file APIs that are loadable (or + resident) here, and a shell tool that bypassed them is never offered as the replacement; the narrow-pool load + line keeps the lesson's order and loads the published names `for the next program`. Two limits are deliberate: + the shipped lesson itself is substituted only when the loader is absent (neither a narrow pool nor a read-only + policy rewrites the shipped trajectories; both narrow only the lists a scope composes), and the lists follow + the configured pool and the read-only policy, while the loadable list also passes the loader's own gate + (per-agent ownership and runtime availability), so it is only ever narrower, never wider. `batchingTelemetry` (same row, YAML-only, default `false`) emits one structured log line per delivered notice, for example `dsh-ops batching: turn=3 category=plan plan=1 bypass=0 shape=2 total=3`. It is @@ -169,6 +196,7 @@ falling back to the built-in surface; scoped diagnostics remain accessible. | `discovery` | true except shell profile; false removes the selector/SDK loader, not explicit startup selections | | `allowedTools` | all supported loadable tools when discovery is enabled, otherwise empty | | `alwaysLoaded` | empty; must be allowed; separate from the Native residentNames profile | +| `disclosurePolicy` | `on-demand`; explicit first-round disclosure tier for PTC-presentation rows: `full` publishes the whole `allowedTools` pool plus `result` in the first round with no loader — the same tier the builtin rows parse since D3. Native-presentation rows reject the key; illegal values are errors naming the key and the received value or type; nothing ever writes it for you | | `replaceOfficial` | false; replaces only ready, admitted file counterparts; conditional official tools may remain visible | | `promptPolicy` | true; guidance names only actually published tools | | `extraGuidance` | empty; Native-only guidance, rejected in PTC | diff --git a/docs/design-decisions-20261115.md b/docs/design-decisions-20261115.md new file mode 100644 index 00000000..22e29a0d --- /dev/null +++ b/docs/design-decisions-20261115.md @@ -0,0 +1,61 @@ +# Design decision: hide official file tools under PTC presets + +Date: 2026-11-15 +Status: accepted +Scope: `src/publication/ptc.ts`, `src/publication/replacements.ts`, `src/consumer/mount.ts`, `src/presets/builtin.ts`, `src/integration/index.ts` + +## Context + +`hideOfficialTools` is documented as hiding the official file tools a mode replaces. In practice it is inert for every PTC-based preset (`ptc`, `ops-ptc`, custom PTC presets): `MODE_PLANS` declares `nativeFiles: false` for them, and `replacementTargets` gates file targets behind `config.hideOfficialTools && plan.nativeFiles`, so the target map only ever contains the Windows shell names. `mountReplacements` is mounted (Windows targets are non-empty) but never sees `read`/`glob`/`grep`. + +The PTC projection keeps the official SDK bindings (`tools.grep`, `tools.read`, ...) while `ops_*` bindings load on demand, so the model can choose the weaker official API even when the switch is on. + +This is treated as a defect, not a policy: the acceptance record in `docs/file-contract-validation.md` described the then-current configuration, and the switch's user-visible promise is coverage of the file tools it replaces. + +## Decision + +1. Fix the existing `hideOfficialTools`; add no new switch; defaults unchanged. +2. Apply by runtime PTC presentation, not by preset id: `ptc`, `ops-ptc`, custom PTC presets. `minimal` stays out. +3. Hidden set = keys of `RAW_OFFICIAL_REPLACEMENTS` (currently `read`, `glob`, `grep`), so it follows the table. +4. `write`, `edit`, `read_image` stay visible. +5. Carrier: extend the shadow `when` in `mountReplacements` (option B), not `ctx.tools.restrict`. +6. `replacementTargets` gate becomes `plan.nativeFiles || plan.ptc`; the integration path covers both presentations. +7. Reachability signal comes from the PTC publisher: `permitted ∧ (resident ∨ allowedApis)`, plus an existence check that the publisher registered the counterpart (`tools.get` + `publicationAdmission !== undefined`). `selected` is excluded and the admission is never evaluated, so loading the SDK cannot change the decision. +8. `mountPtc` returns a `PtcServability` accessor; `apply` stays `void`; unattached agents yield `undefined` (fail-open). +9. A state-change line is emitted once per agent, official name and state; a publisher probe failure is caught, keeps the official tool (fail-open) and reports one warning. Host behaviour was verified: a rejected `when` fails the whole step resolve (`cannot resolve availability of tool`), so catching is required, not optional. +10. Both built-in and integration paths change. +11. Tests: full matrix (flip existing assertions, add mixed-tree and configuration matrix). + +## Evidence + +A real-host probe (real `mode-ptc` component + NodePtc + official `read/glob/grep` registered at the root + `mountPtc` shared-file publication) showed: + +| stage | admitted | `grep:` in `tools:sdk` | SDK bytes | +|---|---|---|---| +| before | `read, glob, grep, ops_inspect_local_file, ops_load_sdk` | yes | 8749 | +| after agent-scope shadow with `when:false` | `read, glob, ops_inspect_local_file, ops_load_sdk` | no | 8676 | +| restored (dispose) | full | yes | 8749 | + +A same-name shadow removes the official tool from both `definitions` and the SDK section, reversibly. + +## Implementation + +- `mountPtc` returns `PtcServability { servable(agent), isPtc(activation) }`; `servable` is `undefined` until the publisher is attached, otherwise the set of raw APIs satisfying `permits(agent, api) && (resident.has(api) || allowedApis(agent).includes(api))`. +- `ReplacementPublication` gains an optional `ptc` field. Inside `when`, after the `keepOfficial`/gate checks and before the host-counterpart check, a PTC branch hides the official name when the raw API is in the servable set and the activation is PTC. +- The PTC branch applies only to keys of `RAW_OFFICIAL_REPLACEMENTS` (never `pwsh`). +- Native consumers pass no `ptc` field and keep the old path byte-for-byte. +- `replacementTargets` gates on `nativeFiles || ptc` in both `applyResolvedMode` and the integration path. + +## Consequences + +- Behaviour change: after upgrade, a PTC session with `hideOfficialTools: true` (the default) no longer sees official `read/glob/grep`; the model must load `ops_*` through `ops_load_sdk`. Documented in CHANGELOG; the existing switch turns it off. +- `write`, `edit`, `read_image` are unaffected. +- Unattached or narrowed PTC projections keep the official tools (fail-open). + +## Consequences: first-resolve timing + +The publisher attaches asynchronously, and `servable` answers `undefined` until then, so the first resolve of a PTC session may still show the official tools. The next resolve hides them. This is the fail-open contract, not a regression; `notePtc` records `unattached` so the window is observable. + +## Open risk + +Resolved by probe: a rejected `when` fails the step resolve (`cannot resolve availability of tool "read"`) instead of failing open or hidden. `replacements.ts` therefore catches probe failures and keeps the official tool. Known accepted risk: reachability comes from the publisher pool (`permitted ∧ (resident ∨ allowedApis)`), not from `publicationAdmission`; a host-level publication rejection after that point is a PTC publisher failure covered by the PTC lifecycle suites, not by the hiding decision. diff --git a/docs/design-decisions-20261119.md b/docs/design-decisions-20261119.md new file mode 100644 index 00000000..af69e4de --- /dev/null +++ b/docs/design-decisions-20261119.md @@ -0,0 +1,335 @@ +# OPS-PTC legacy configuration: the five-input truth table + +[中文](设计决策-20261119.md) + +Decision date: 2026-11-19. Baseline: `main` at 58f6e28. Card: +[#985](https://github.com/T-Auto/dsh-ops/issues/985) (D1), workflow +[#979](https://github.com/T-Auto/dsh-ops/issues/979), overall plan +[#978](https://github.com/T-Auto/dsh-ops/issues/978). + +This document locks **current** publication behaviour; it implements nothing. Every +row is asserted letter by letter by `node test/run.mjs ptc-config-truth` +(`test/ptc-config-truth.test.mjs`). Byte budgets are deliberately out of scope here — +the V slices measure them. + +## How one input reaches the tool set + +| Stage | Where | What it does | +| --- | --- | --- | +| Row/global keys | `src/settings/merged.ts` `resolveMergedConfig` | Validates the row; defaults `dynamicTools` and `residentTools` to true (`global` has no resident key) | +| AND gate | `effectiveMode` (`src/settings/merged.ts`), `dynamicForMode` | Row `dynamicTools` AND `global.dynamicTools`; the global switch "keeps the operator's intent and simply stops applying" | +| Consumer plan | `src/presets/builtin.ts` `applyResolvedMode` | `gate.resident = config.residentTools`, `gate.dynamic = config.dynamicTools` (the gated value), `plan.ptc = { allowedTools, alwaysLoaded }` | +| Publication | `src/publication/ptc.ts` `mountPtc` | `resident` = `PTC_RESIDENT_TOOLS` + `alwaysLoaded`, empty when `gate.resident === false`; `selectable` = `allowedTools` + `result`, empty when `gate.dynamic === false`; a tool is callable when resident **or** previously selected; `ops_load_sdk` is published only while `gate.dynamic !== false` | + +## The truth table + +Mode `ops-ptc` (the `ptc` row publishes the same first-round set; only +`batchingGuidance` differs). Measured on Linux, `danger-full-access`, native worker +present, real PTC presentation, the host's official `read`/`glob`/`grep` left visible; +`ops_bash` is absent because POSIX rows keep the host shell. + +| # | Input (one line of `dsh-ops` row config) | Callable this step | Loadable, not loaded | Loader | run_code first-program view | +| --- | --- | --- | --- | --- | --- | +| 1 | defaults (no keys) | `ops_inspect_local_file`, `ops_load_sdk` | the sixteen-API pool below | yes | `tools.ops_inspect_local_file` and `tools.ops_load_sdk` work; a `ops_load_sdk({tools:['grep']})` receipt completes but `ops_grep` is callable only in the **next** program | +| 2 | `global: {dynamicTools: false}` | `ops_inspect_local_file` | none | no | `tools.ops_inspect_local_file` works; there is no `tools.ops_load_sdk` | +| 3 | `ops-ptc: {dynamicTools: false}` | `ops_inspect_local_file` | none | no | same set as row 2 (one AND gate, two sides); only the scope differs — see note | +| 4 | row 3 plus `ops-ptc: {alwaysLoaded: [stat]}` | `ops_inspect_local_file`, `ops_stat` | none | no | both bindings work in the first program | +| 5 | `ops-ptc: {residentTools: false}` | `ops_load_sdk` | the sixteen-API pool below | yes | the loader is the only binding; `ops_inspect_local_file` is unreachable — `ops_load_sdk({tools:['inspect_local_file']})` answers `INVALID_ARGUMENT` | +| T | **implemented — see the D3 addendum**: an explicit full selection with `allowedTools: [grep, stat]` | `ops_grep`, `ops_stat` (per #979 §2) | — | no (per #979 §2) | every allowed API callable in the first program; read-only sessions still exclude write APIs | + +The default loadable pool (rows 1 and 5) is exactly: +`ops_grep`, `ops_glob`, `ops_replace`, `ops_write_file`, `ops_edit_file`, `ops_diff`, +`ops_stat`, `ops_copy`, `ops_move`, `ops_remove`, `ops_read_files`, `ops_read_table`, +`ops_read_docx`, `ops_read_pdf`, `ops_batch`, `ops_result` — the sixteen non-resident +raw APIs (`PTC_LOADABLE_TOOLS`). Row T's pool is `ops_grep`, `ops_stat` **plus +`ops_result`** (see sign-off item 1). + +**Row notes.** + +- Rows 2 and 3 publish the same OPS set because both feed one AND gate. Their inputs are + still different: the global switch also gates the Native discovery entry of + `standard`/`cordis`/`minimal` (`dynamicForMode`), the row switch only its own mode. +- Row 4 shows `dynamicTools:false` does **not** suppress `alwaysLoaded`: the startup list + rides the resident set, with no loader to select the rest of the cap. +- Row 5 shows `residentTools:false` drops the whole resident set, including + `ops_inspect_local_file`, which is not loadable either (it is not in + `PTC_LOADABLE_TOOLS`). +- **Old off is a different input from a full selection** (#985 counterexample): with + `allowedTools: [grep, stat]` and `dynamicTools: false` the first round is + `ops_inspect_local_file` alone — neither `ops_grep` nor `ops_stat` nor the loader. + Reading `dynamicTools:false` as "publish everything allowed" would put `ops_grep` and + `ops_stat` in that first round, changing old users' cost and permission expectations; + #907 implicitly flipped exactly that and #910 reverted it (#979 §4). + +**Read-only and writable instances.** Read-only members of the default pool: +`ops_grep`, `ops_stat`, `ops_glob`, `ops_diff`, `ops_read_files`, `ops_read_table`, +`ops_read_docx`, `ops_read_pdf`. Writable members: `ops_replace`, `ops_write_file`, +`ops_edit_file`, `ops_copy`, `ops_move`, `ops_remove`, `ops_batch`. In a read-only +session `permitted` (`src/publication/ptc.ts`) removes every mutating API from the +loadable pool and a load request for one answers `INVALID_ARGUMENT`; +`ops_result` and `ops_load_sdk` are exempt from the mutation check. No suite here +changes the sandbox mode; D3 covers read-only. + +A program you can read letter by letter: + +```js +// plan: row 1 — load grep now, use it in the next program +const picked = await tools.ops_load_sdk({ tools: ['grep'] }) +return { picked } // picked.status === 'completed'; ops_grep is NOT callable in this program +``` + +```js +// plan: row 4 — stat is resident here, callable in the first program +const r = await tools.ops_stat({ path: 'a' }) +return r.status +``` + +## Maintainer sign-off items — recorded, deliberately not implemented + +1. **The implicit `ops_result` exception.** Today `loadable = [...allowedTools, 'result']` + unconditionally, and `permitted` exempts `result` from the read-only mutation check, so + `ops_result` stays loadable even when the cap omits it (covered today by + `test/resident-switch.test.mjs`, "result exception remains loadable"). With + `dynamicTools:false` (rows 2–4) `ops_result` is **not published at all**. Decision + needed for the full tier: must its first round publish `ops_result` so spilled refs stay + consumable (#979 §3)? Current behaviour is locked; nothing here changes it. + *(Resolved by the D3 addendum: the full tier's first round publishes `ops_result`.)* +2. **The global switch versus an explicit full selection.** Today `global.dynamicTools` + AND-gates every row's loader, including the PTC one. #979 §4 requires that upgrades, + settings saves and the `global.dynamicTools` switch never pick full **for** the user. + Decision needed: does `global.dynamicTools:false` also suppress an explicitly selected + full row, or does the explicit opt-in win over the bundle switch? +3. **Explicit resident versus the cap.** Today `alwaysLoaded` is validated against the + loadable pool only (`src/ptc/config.ts`), never against `allowedTools`: + `{ allowedTools: [grep], alwaysLoaded: [stat] }` publishes `stat` as resident with no + diagnostic, and `residentTools:false` silently drops the whole resident set including + `alwaysLoaded`. Decision needed: which of these conflicts get a readable diagnostic in + D2, and whether `alwaysLoaded` must be inside `allowedTools`. + Supplementary contrast fact (append-only audit, same day): the **custom + integration path** is stricter — `src/consumer/options.ts:100` rejects a custom PTC + config whose `alwaysLoaded` escapes `allowedTools` with + `alwaysLoaded must be in allowedTools`, while the built-in rows accept the same + shape silently (locked above by the suite's "escapes the allowedTools cap" check). + The diagnostic inconsistency is owned by the D4 card (#988); D1 records the current + state only. +4. **`ops_result` misdiagnosed as unpublished on a legal custom PTC integration.** + `src/integration/diagnostics.ts` lists `'result'` among the first-round expected + visible names of every PTC integration scope, but `result` is selectable, not + resident (`PTC_RESIDENT_TOOLS` is `inspect_local_file` alone), so it becomes + visible only after a loader call selects it. A perfectly legal custom + `profile: 'ptc'` integration is therefore diagnosed `degraded` with + `PUBLICATION_UNAVAILABLE: ops_result` while its only gap is an API the loader has + not selected yet — verified by booting such an integration; the suite's + "misdiagnosed PUBLICATION_UNAVAILABLE" check locks it. Built-in mode rows never + mount diagnostics at all (`mountDiagnostics` runs only from `mountIntegration`), + so the misreport is reachable only through the custom-integration path. The fix is + owned by the D4 card (#988); D1 records the current state only. + +## Settings-page copy — recorded verbatim + +The settings page and its row switch describe the dynamic-tools switch as a +"discovery entry". The shipped strings, recorded verbatim from +`src/client/locales.ts`: + +| Key | English | 中文 | +| --- | --- | --- | +| `dynamicToolsHint` | "When off, no preset offers the discovery entry." | "关闭后没有任何预设提供发现入口。" | +| `swDynamicToolsHint` | "Offers the discovery entry to this preset. Applies only while the bundle switch is on." | "为该预设提供发现入口。仅在总开关开启时生效。" | + +What the PTC row actually publishes for that entry is the typed loader +`ops_load_sdk` (truth-table rows 1 and 5), not a discovery-style tool: the wording +and the artifact differ. Unifying the wording is the D5 slice of #979; this card +records the gap only. + +## Loader batching — recorded current behaviour + +One `ops_load_sdk` call accepts 1–5 exact raw API names (`requestedApis` in +`src/projections/ptc/loader.ts` — `tools must contain 1-5 exact raw API names`), so +filling the sixteen-name default pool takes at least four calls. `alwaysLoaded` and +the rest of the resident set carry no count cap: `parsePtcConfig` +(`src/ptc/config.ts`) validates each name against the loadable pool, never the +list's length, and resident APIs publish without any loader call. Locked by the +suite's "loader batches 1-5 names" check. + +## Scope + +No publication default, no diagnostic and no third user mode is introduced here. Per +#985, disagreements on the sign-off items go to the maintainers on the card before D2/D3 +implement anything. Pure legacy configurations keep the tool sets above before and after +this slice — that is what the suite asserts. + +## Verification + +- `npm run build`, then `node test/run.mjs ptc-config-truth` — 11 passed / 0 skipped / 0 + failed on the baseline (first run matched the code-read prediction on every row). +- Red proof that the assertions bind behaviour, not each other: changing only the row-5 + expected constant in the suite makes exactly that row's check fail + ("1 of 11 check(s) failed"); restoring the constant returns the suite to green. +- `node test/run.mjs resident-switch` — unchanged neighbour suite, see the PR. +- Supplementary audit (append-only, same day): two checks were added — the loader's + 1–5-per-call cap with the count-cap-free resident startup list, and the legal + custom PTC integration misdiagnosed `PUBLICATION_UNAVAILABLE: ops_result`. The + suite now reports 13 passed / 0 skipped / 0 failed on the baseline. + +## D2 addendum — the explicit disclosure policy key, parsed (2026-11-19) + +Card [#986](https://github.com/T-Auto/dsh-ops/issues/986), on top of this table's +branch. This addendum records what D2 locked and resolves sign-off item 2's +precedence question conservatively; publication of the full tier remains D3. + +**The key.** The `ptc` and `ops-ptc` rows accept a YAML-only +`disclosurePolicy: 'on-demand' | 'full'` (default `'on-demand'` = the legacy path +of this table, letter for letter). Illegal values are errors naming the key and +the received value or type; non-PTC rows (`minimal`, `standard`, `cordis`) +reject the key with `disclosurePolicy applies to the ptc and ops-ptc rows only`. +The builtin consumer plan slice and custom integrations keep the legacy default +until D3/D4 teach them the tier. + +**Sign-off item 2, resolved conservatively per #979 §4.** An explicit full +selection is a *distinct opt-in* and never overrides the bundle switch: while +`global.dynamicTools` is off, the full tier is not applied — the row keeps its +legacy resident/alwaysLoaded surface (row 2 of this table) — and the saved +choice stays untouched in the row config, re-applying when the switch returns +on. This keeps every §4 guarantee in one direction: upgrades, settings saves +and switch flips never write or erase the key, and the operator's off stays +absolute for every disclosure shape. The row-level `dynamicTools` keeps its +loader-only meaning: the full tier publishes no loader (#979 §2), so a row that +also turns the loader off keeps its explicit tier. `effectiveMode` encodes this +gate; the saved intent is preserved exactly like the existing dynamicTools AND +gate preserves it. This is the conservative resolution; maintainers may revisit +it. + +**Serialization.** `hostConfig` (the save-path serializer) preserves an explicit +choice verbatim and never auto-writes the key: reading → saving an unrelated +setting → reading again keeps `'full'`, and every legacy input — the five rows +of this table included — resolves with the key absent at both the raw and +serialized layers. + +**Still not implemented.** Truth-table row T stays a target: nothing publishes +the full tier yet (D3 owns `src/publication/ptc.ts` and the builtin plan slice), +there is no settings-page entry (D5), and sign-off items 1, 3 and 4 remain open +as recorded above. + +**Verification.** `npm run build`, then `node test/run.mjs ptc-disclosure-policy` +— 8 checks, red first (8 of 8 failed on the unimplemented baseline, the key +rejected as unknown), green after. `node test/run.mjs ptc-config-truth` 13/0/0, +`node test/run.mjs resident-switch` 24/0/0 and `node test/run.mjs settings-rows` +14/0/0 unchanged on the new parsing path, worker +`DSH_OPS_WORKER_BIN=runtime/linux-x64/dsh-ops-worker` (sha256 2b5f988d). + + +## D3 addendum — the explicit full tier is published (2026-11-19) + +Card [#987](https://github.com/T-Auto/dsh-ops/issues/987), on top of the D2 addendum's +branch. This slice turns the parsed key into publication behaviour; nothing above +rewrites the five legacy rows, which stay asserted letter by letter. + +**Row T, implemented.** With `{ disclosurePolicy: 'full', allowedTools: [grep, stat] }` +the first round publishes `ops_grep`, `ops_stat`, `ops_inspect_local_file` (resident) +and `ops_result`, with no `ops_load_sdk` (#979 §2). The first program can call the +permitted pool directly (dual verification: the settled declaration set plus a real +`run_code` binding call through the worker); pool-outside business APIs stay +unavailable. The builtin plan slice now carries the parsed tier +(`src/presets/builtin.ts`), and the publication consumes it in +`src/publication/ptc.ts`. + +**Sign-off item 1, resolved conservatively per #979 §3.** The full tier's first round +publishes `ops_result`: a ref spilled this step must stay consumable, and the unified +acceptance rule forces the same conclusion — on-demand can always load `result` +(`loadable = [...allowedTools, 'result']`), so the loaded business set equals the full +first-round set only if full publishes it. The comparison is pinned by one check. + +**Edges, as shipped and asserted.** An empty cap keeps the tier at resident + +`result` (still no loader). Read-only still removes every mutating API from the +first round — switching tiers never widens permissions. `residentTools:false` drops +the resident surface (row 5) but not the explicit tier. Duplicated names collapse to +one registration. The bundle switch keeps its D2 veto end to end: while +`global.dynamicTools` is off an explicit full row publishes row 2 letter for letter; +the row-level `dynamicTools` keeps its loader-only meaning, so a full row that also +turns the loader off keeps its tier. + +**Not done here (by design).** Custom integrations keep the legacy default until D4, +and `PtcConfig.disclosurePolicy` stays optional for exactly that reason (making it +required would force touching the custom-integration plan slice this card does not +own; the builtin plan carries the value, an absent value means on-demand). The +settings-page entry is D5. + +**Verification.** `npm run build`, red first: `node test/run.mjs ptc-declaration-surface` +4 of 5 failed (every new row T assertion; the legacy loader-less check stayed green) +and `node test/run.mjs ptc-full-disclosure` failed row T, the comparison, read-only +and the empty pool before the suite's 120 s deadline. Green after: +ptc-declaration-surface 5/0/0, ptc-full-disclosure 8/0/0, and the regression locks +unchanged — ptc-config-truth 13/0/0, resident-switch 24/0/0, ptc-disclosure-policy +8/0/0, settings-rows 14/0/0; worker +`DSH_OPS_WORKER_BIN=runtime/linux-x64/dsh-ops-worker` (sha256 2b5f988d). Environment +note, not a regression: `ptc.test.mjs`'s installed-runtime check asserts precedence +over `DSH_OPS_WORKER_BIN`, so it runs without that env override (it fails identically +on the base commit with the override set). + + +## P5 addendum — four admission verdicts, aligned across registration, declaration and dispatch (2026-11-19) + +**The problem (#997; original repro in #968).** A registered definition is not +necessarily callable this step, but every refusal read the same: one +`POLICY_DENIED` ("must be admitted") from the PTC dispatch layer, one +`PUBLICATION_GRANT_UNAVAILABLE` from the publication source, and — worst — +`ops_load_sdk` answered `INVALID_ARGUMENT: API is not in this scope's allowed +pool` for a pool member that read-only forbids. A model could not tell an +unloaded name from a forbidden one, and a revoked admission read identically to +a name that never existed. + +**The four verdicts, one classifier.** `src/publication/ptc.ts` now answers a +refused dispatch with exactly one of four verdicts, aligned with what +registration admits and what the SDK declaration shows: + +| verdict | condition | answer | +| --- | --- | --- | +| usable this step | resident or selected, permitted | dispatch; never lands in the refusal path | +| loadable | in the pool, permitted, not selected — and a loader exists | `SDK_NOT_LOADED` naming `ops_load_sdk` and the raw name, effective next model step | +| permission-forbidden | the sandbox policy refuses the API | `POLICY_DENIED` with explicit permission wording and no load suggestion | +| revoked | an admission that existed was withdrawn — scope detached, presentation lifted, or permission downgraded on a previously-usable API | `POLICY_DENIED` with revoked wording, distinct from all of the above | + +A stale or foreign binding keeps the plain admission refusal, so nothing +outside the four verdicts gains a new promise. The loader suggestion exists +only where a loader can actually select: the loader-less scope and the full +tier keep the honest no-loader denial. `ops_load_sdk` itself now validates +against the configured pool and splits permission refusals from pool refusals — +`write_file` under read-only is forbidden, not "not in the pool"; a +pool-outside name keeps `INVALID_ARGUMENT`. The `SDK_NOT_LOADED` spelling is +reused from open PR #1007 deliberately, so the eventual merge reads +consistently; that PR's deferred-pointer plumbing stays owned by it. + +**The source layer keeps the same distinction.** A pulled registrar grant +answers the new `PUBLICATION_GRANT_REVOKED` instead of reading as +never-available; an agent that never held an eligible grant keeps +`PUBLICATION_GRANT_UNAVAILABLE`. Grant and pool semantics are untouched — +only the refusal verdict changed. + +**Diagnostic assemblies are not first requests.** An assembly without a step +tool set now reads the registration surface without clearing pending loads or +cancelling work; only a real step (tool set present) triggers the +presentation-withdrawal cleanup. The same agent's two paths stay distinct and +pinned: registered names (diagnostics) versus declared names (requests) versus +program bindings — an unloaded name never holds one, so calling it in a +program is a JS error, never a tool error. + +**Scope.** `src/publication/ptc.ts`, `src/publication/registrar.ts`, +`test/ptc-declaration-surface.test.mjs` (5 → 11 checks), plus +`test/ptc-directory.test.mjs` (2 assertions) because those lines pinned the +pre-#997 misclassification of a permission refusal as a pool refusal — the +exact behaviour this card removes. Teaching copy (P1/P2) and pool semantics +(D3) are untouched. + +**Verification.** `npm run build`; red first: +`DSH_OPS_WORKER_BIN=runtime/linux-x64/dsh-ops-worker node test/run.mjs +ptc-declaration-surface` → 5 of 11 checks failed (every new verdict, the +diagnostic-isolation check and the grant-revoked check; the five legacy checks +plus the new registered/declared/admitted pin stayed green). Green after: +ptc-declaration-surface 11/0/0; regressions publication 6 suites, mount 2, +probe 2, batching 4, and the wider `ptc` family 18 of 19 suites green — +`ptc.test.mjs` fails only under the `DSH_OPS_WORKER_BIN` override, verified +to fail identically on the D3 base commit (the installed-runtime resolution +crash recorded in the D3 verification note) and to skip without the override, +matching the base. Revoking any classifier line, the loader split or the +diagnostic-isolation guard turns its own check red again. + diff --git a/docs/diagnostics.md b/docs/diagnostics.md index eedb687e..bda1bae2 100644 --- a/docs/diagnostics.md +++ b/docs/diagnostics.md @@ -29,5 +29,7 @@ Important states: For `PRESENTATION_MISMATCH`, inspect the actual assembled surface. PTC requires `run_code` plus a non-empty `tools:ptc-only` section; a preset name or prompt recommendation is not proof. For `PUBLICATION_UNAVAILABLE`, verify the mode row/binding lifetime and registration errors before -changing user permissions. If an official tool was hidden while its OPS counterpart is absent, the +changing user permissions. PTC expectations follow the same disclosure plan the publication +derives: an API that is selectable but not yet loaded (`result` on the on-demand tier) is never a +gap, while a genuinely missing first-round plan name is. If an official tool was hidden while its OPS counterpart is absent, the replacement controller is defective and must be fixed rather than asking the user to retry. diff --git a/docs/file-batching.md b/docs/file-batching.md index e0b77636..9c065eb0 100644 --- a/docs/file-batching.md +++ b/docs/file-batching.md @@ -58,8 +58,10 @@ structured. `ops_edit_file` applies 1-100 edits to **one** UTF-8 file, atomically. Supply `path`, the exact SHA-256 `expectedRevision` from a read/stat, and `edits: [{oldText,newText}]`. Each nonempty -`oldText` must occur exactly once in the same original bytes; overlaps, ambiguity or stale -revision reject all edits before publication. Empty `newText` deletes the match. Preserve +`oldText` must occur exactly once in the same original bytes. If that raw match fails, one +fallback decodes the reversible Native rendering of both `oldText` and `newText`; a raw match +never decodes either field, and a fallback receipt counts `renderedMatches`. Overlaps, ambiguity +or stale revision reject all edits before publication. Empty `newText` deletes the match. Preserve CRLF/BOM explicitly. `dryRun: true` previews the same receipt without writing; read-only policy still rejects mutation tools. This is not a cross-file transaction. diff --git a/docs/file-contract-validation.md b/docs/file-contract-validation.md index 9efc9ae1..2127c95d 100644 --- a/docs/file-contract-validation.md +++ b/docs/file-contract-validation.md @@ -33,11 +33,28 @@ rate improvements. Native output fixture reductions are in [Native output](nativ - Native standard admits OPS read/glob/grep replacements and omits their official counterparts. `edit` and `read_image` remain. Existing replacement tests cover readiness, captured steps, restrictions, same-name registrations and unload. -- Both built-in PTC modes intentionally configure **no file replacement targets**. - Their SDK retains official read/glob/grep/edit methods alongside selected OPS - methods. The audit executes both official and OPS glob as nested calls; the - direct provider request still contains only `run_code`. This is preset policy, - not a recurrence of #787's visible-but-rejected Native tool bug. +- Both built-in PTC modes hide the official `read`/`glob`/`grep` their projection + can serve while `hideOfficialTools` is on (the default); `write`/`edit`/ + `read_image` stay. The decision comes from the publisher's `servable` set + (`permitted ∧ (resident ∨ allowedApis)`) and excludes `selected`, so loading the + SDK never resurrects an official tool. The decision is per name: a name the + projection cannot serve keeps its official tool (an unattached projection, + `glob`/`grep` under `dynamicTools: false`, or a name excluded by a narrowed + `allowedTools`), while a resident `inspect_local_file` still replaces `read`. + The direct provider request still contains only `run_code`. +- The fallback side of that decision is verified per name and stays usable: a + kept official tool is admitted, declared and callable through its SDK binding, + withdrawing the row (or an unattached, probe-failing or narrowed projection) + restores the official `read`/`glob`/`grep` to the same callable state, and a + custom preset keeps every official tool until `replaceOfficial` opts in + (default `false`). The kept `write`/`edit`/`read_image` SDK entries are + compared through real argument schemas, not empty membership fixtures. +- Hiding hands those names' workflow to the OPS replacements, whose reviewed + differences are documented per tool in the [component reference](component-reference.md): + `ops_glob`/`ops_grep` never traverse symlinks, reparse points, hidden files + or `.git`, their coverage is paged with explicit completion/truncation + fields, and `write`/`edit` stay official precisely because the OPS CAS + `expectedRevision` publication contract differs from the host write/edit. - Unloading the row restores the original surface. OPS-free PTC receives no OPS prompt or SDK. Disabled batching delivers no lesson or reminder, as covered by the batching lifecycle suite. Runtime version appears only in explicit diagnostics. diff --git a/docs/file-contracts.md b/docs/file-contracts.md index dbb6916a..6292e93f 100644 --- a/docs/file-contracts.md +++ b/docs/file-contracts.md @@ -16,7 +16,9 @@ These are the current contracts, including deliberate differences between tools. Revision computation for `stat`, `copy`, `move` and `remove` streams SHA-256 instead of loading the whole file, so those operations accept files over 32 MiB. Hex inspection streams whole-file facts without buffering them, then seeks to and buffers only its requested window. Whole-content text inspection, `diff` and mutation buffers retain their explicit content budgets. -On Windows volumes without named-stream or ACL support, mutations preserve available basic metadata and return `metadataWarnings` instead of `METADATA_UNSUPPORTED`. Windows source metadata that is present but unsupported, such as NTFS alternate data streams, still fails closed. +On Windows volumes without named-stream or ACL support, mutations preserve available basic metadata and return `metadataWarnings` instead of `METADATA_UNSUPPORTED`. On NTFS, user-restorable extended attributes are captured, restored and byte-verified. `$KERNEL.*` names are listed individually in `metadataWarnings` because user mode cannot recreate them; any other EA capture, restore or verification failure fails closed. Unsupported source metadata such as NTFS alternate data streams also fails closed. + +A path through an Electron `.asar` file returns `ARCHIVE_MEMBER_UNSUPPORTED`. Electron presents that archive as a directory only inside its own patched process filesystem; the standalone worker sees a regular file and cannot read archive members. Inspect the archive itself in hex view or use an unpacked copy. `replace` is not bound to an earlier `inspect_local_file` result. Use `edit_file` when a single-file change must depend on that exact read. A `replace` preview is not a reservation: execution rescans. `changed` counts applied literal replacements, not files. A preview has `dryRun: true`; planned hashes alone do not establish that writes happened. Directory replacement records permission-denied reads in `skippedPaths` and returns `complete: false` with `truncated: true` while continuing eligible neighbors; ordinary binary-file skips do not by themselves make the result incomplete. Update-lock, staging and publication denials still fail the call, and sharing violations or other IO failures retain their structured error instead of becoming skips. diff --git a/docs/flash-comparison-v5.md b/docs/flash-comparison-v5.md new file mode 100644 index 00000000..101b9838 --- /dev/null +++ b/docs/flash-comparison-v5.md @@ -0,0 +1,172 @@ +# Flash small-scale comparison (V5) + +English | [中文](Flash对照-V5.md) + +Issue [#994](https://github.com/T-Auto/dsh-ops/issues/994), parent [#980](https://github.com/T-Auto/dsh-ops/issues/980). This is a +measurement card: the only repository additions are the measurement driver, the collector, this report pair and the +changelog entry. No product code, teaching copy or release logic changed. + +## The preregistered question + +The V5 card asks whether the **shorter first-round prompt** of the on-demand PTC disclosure tier trades for **more +calls or more failures**. The two arms therefore differ in exactly one preregistered factor — the `dsh-ops` row's +`disclosurePolicy` (`on-demand` vs `full`, implemented by #986/#987) — while the route, the reasoning effort, the +sandbox mode, the preset, the worker and the task fixtures stay frozen: + +- route: `opencode-go` / `deepseek-v4.1-flash`, `reasoningEffort: high` +- preset: `ops-ptc` (the composed group proves it; a Standard fallback would be invalid per #980) +- sandbox: `danger-full-access`; platform: `linux-x64` +- tasks: `ptc-catalog-recover` (catalog search, needs the loadable `ops_grep`) and `ptc-wait-dispatch` + (multi-layer config edit, needs `ops_read_files`/`ops_edit_file`) — the two V4 tracks whose working set is + most sensitive to the tier +- plan: 2 tasks × 3 repeats × 2 arms, AB/BA interleaved per (task, repeat), 12 sessions; plus a same-condition + A/A sweep (12 sessions, on-demand config twice) for the noise floor; every session ran in a fresh seeded + workspace with a fresh composed group + +## Why a driver instead of `bench ab` + +The host's one-shot headless runner composes the host realm only and **refuses preset sessions by design** (its +adoption check: *"session … runs under agent preset …, which the one-shot runner does not compose"*). The plugin +treats a registry-less host realm as Standard, so an `ab` sweep would measure Standard and be invalid per #980. +`bench/v5-run.mjs` reuses the V1 isolated-entry composition (real AgentLoop, real PTC executor, real staged worker, +the shipped `dsh-ops` row) and replaces the scripted provider with the real `llm-pi-ai` adapter. One resolution +detail: `@deepseek-ai/dsh-llm-pi-ai` (and `@deepseek-ai/dsh-credentials`, plus a symlink for `@earendil-works/pi-ai`) +is copied into `node_modules` at run time so every import resolves through this checkout's tree and one `cordis` +instance serves the whole composition; `node_modules` is gitignored and no manifest changes. The driver archives +per session: raw events (`*.events.ndjson`), the judge-ready trajectory (`*.trajectory.json`), folded metrics +(`*.metrics.json`), the verified answer, and per arm `arms//dump-config.txt`. + +## Environment facts (recorded, not claimed) + +| fact | value | +| --- | --- | +| plugin commit | `28c0cf7` on `card/v5-flash-comparison` = `main@58f6e28` + merged approved prerequisites (V1 entry + V4 tracks `card/v4-bench-tasks` @`ec4d25c`, the tested D2/D3 disclosure implementation `card/v3-two-tier-matrix` @`c68fe3d`) | +| host version | 0.2.2-rc.1 | +| platform | linux-x64, node v24.21.0 | +| worker | staged `runtime/linux-x64/dsh-ops-worker`, sha256 `2b5f988d9115…` | +| preset / route | `ops-ptc`; `{provider: opencode-go, model: deepseek-v4.1-flash, reasoningEffort: high}` | +| sandbox | danger-full-access, approval never (the composed group pins the mode; no approval service is mounted, tool failures are returned values) | + +Per-arm evidence: `bench/runs/v5-comparison/arms/{ondemand,full}/dump-config.txt` and +`bench/runs/v5-aa/arms/{aa-one,aa-two}/dump-config.txt` — identical rows except the one +`disclosurePolicy` line. + +## The complete denominator + +| sweep | planned | completed | timed out | missing usage | verify pass | +| --- | --- | --- | --- | --- | --- | +| comparison (on-demand vs full) | 12 | 12 | 0 | 0 | 11 | +| A/A (on-demand twice) | 12 | 12 | 0 | 0 | 12 | +| **total** | **24** | **24** | **0** | **0** | **23** | + +One comparison session failed the workspace verifier: `ptc-wait-dispatch` r2 on the on-demand arm wrote +`config/development.json` (it picked the wrong config layer); the session completed normally, so it stays in +both the completion and the pass denominator. No session timed out; every step of every session reported usage, so +no metric below is a lower bound. A pre-sweep smoke (2 sessions, `bench/runs/v5-smoke`) confirmed routing and +usage collection first and is not part of the preregistered denominator. + +## Per-task results + +Prompt sizes are tokens per model request (input + cacheRead + cacheWrite); totals are the sum over all steps of +one session, medians over the arm's three sessions. + +### ptc-catalog-recover + +| metric | on-demand | full | A/A band (same config twice) | +| --- | --- | --- | --- | +| verify pass | 3/3 | 3/3 | 3/3 – 3/3 | +| correct next steps | 26 | 17 | 23 – 24 | +| correction opportunities | 3 | 0 | 0 – 1 | +| no-progress retries | 5 | 4 | 5 – 7 | +| waits | 0 | 0 | 0 | +| first-round prompt (median) | 5,271 | 29,946 | 5,271 – 5,271 | +| prompt total (median) | 86,136 | 185,465 | 66,881 – 88,078 | +| input tokens (3 sessions) | 65,035 | 68,051 | 55,814 – 57,644 | +| cacheRead tokens (3 sessions) | 182,656 | 430,848 | 144,896 – 198,016 | + +### ptc-wait-dispatch + +| metric | on-demand | full | A/A band | +| --- | --- | --- | --- | +| verify pass | 2/3 | 3/3 | 3/3 – 3/3 | +| correct next steps | 8 | 8 | 14 – 27 | +| correction opportunities | 1 | 2 | 4 – 6 | +| no-progress retries | 7 | 6 | 7 – 9 | +| waits | 0 | 0 | 0 | +| first-round prompt (median) | 5,306 | 29,981 | 5,306 – 5,306 | +| prompt total (median) | 137,419 | 236,416 | 85,813 – 103,238 | +| input tokens (3 sessions) | 117,307 | 106,664 | 65,094 – 125,645 | +| cacheRead tokens (3 sessions) | 278,784 | 597,248 | 203,776 – 244,736 | + +### First round vs after loading + +`#994` step 3 asks for the first-round vs post-load delta. On the on-demand arm every session completed one +`ops_load_sdk` in its first program, and the next request grew by the loaded SDK declarations plus history: + +| task | first round (median) | first request after the completed load (median) | +| --- | --- | --- | +| ptc-catalog-recover | 5,271 | 12,820 | +| ptc-wait-dispatch | 5,306 | 15,574 | + +The full arm has no load step by design; its first round already carries the whole permitted pool (29,946 / +29,981 tokens). The A/A arms reproduce the on-demand numbers (12,733–12,786 / 15,437–15,568), which is also a +consistency check on the capture path. + +### Cache usage + +Every step reported a usage object; `cacheReadTokens` was present on every step and `cacheWriteTokens` was +reported as 0 throughout (the provider served the repeated prompt prefix from cache). Missing-usage rows: 0. +`reasoningTokens` was 0 on every step despite `reasoningEffort: high` — recorded as-is, not reinterpreted. + +## Noise verdict + +- **Not distinguished (within A/A noise):** the verify-pass difference on `ptc-wait-dispatch` (2/3 vs 3/3 with a + 3/3–3/3 A/A band is a single failure at n=3; the failure was a reasoning mistake — the wrong config layer — + not a tier interaction); no-progress retries on both tasks (5 vs 4 and 7 vs 6 against bands of 5–7 and 7–9); + correction opportunities on `ptc-wait-dispatch` (1 vs 2 against a band of 4–6, i.e. both arms sit below the + A/A spread); bare input tokens on both tasks (the A/A band on `ptc-wait-dispatch` alone spans 65k–126k). +- **Outside the A/A band, but mechanical, not a model-quality claim:** the first-round prompt (5.3k vs 30.0k, + band 5.3k–5.3k) and the prompt totals (86k vs 185k and 137k vs 236k against bands of 67k–88k and 86k–103k) — + this is the disclosure tier's own declaration cost, an input-side fact, not a behavior difference; cacheRead + follows it (2.4× on both tasks). The catalog task's correction opportunities (3 vs 0, band 0–1) are the + on-demand loader inviting exactly one recoverable input error per session (an invalid first + `ops_load_sdk` name) — all three sessions recovered and passed, so the recovery rate is 3/3, not + distinguishable from the full arm which never had the opportunity. The catalog correct-next-step count + (26 vs 17, band 23–24) includes the loader step and the recovery steps that only exist on the on-demand arm, + so it is reported, not attributed. +- **No claim of a success-rate improvement from the tier.** At n=3 per arm per task, nothing behavioral separates + the tiers beyond what the A/A band already covers. + +## Reproduction + +```console +npm run build && npm run build:ptc +# comparison sweep: 2 tasks x 3 repeats x 2 arms, AB/BA interleaved +node bench/v5-run.mjs --task ptc-catalog-recover --task ptc-wait-dispatch --repeat 3 \ + --out bench/runs/v5-comparison --dsh-home +# A/A noise sweep: the same plan with the on-demand config in both arms +node bench/v5-run.mjs --task ptc-catalog-recover --task ptc-wait-dispatch --repeat 3 --aa \ + --out bench/runs/v5-aa --dsh-home +# judge and tables +node bench/v5-collect.mjs --run bench/runs/v5-comparison --run bench/runs/v5-aa +``` + +The artifacts of this report's run live under `bench/runs/v5-comparison` and `bench/runs/v5-aa` +(`summary.json`, per-session `results/*`, per-arm `dump-config.txt`), plus the smoke at `bench/runs/v5-smoke`. +The judge reuses the V4 shared rules verbatim (`scanReissues`, `countOpportunities` in `bench/lib/tracks.mjs`) +and the two task verifiers that generalize (the paged-cursor and wait-dispatch rules). The two scripted-only +story rules — the catalog track's mandatory wrong call and the load-order track's mandatory not-loaded rejection — +are deliberately not required of a real run: a model that never errs must not be judged as failing the story. + +## Limitations + +- n=3 per arm per task: the pilot size #994 prescribes. The A/A bands show how wide the floor is at this n; + nothing here powers a success-rate claim. +- The composed group is the V1 isolated entry, which is leaner than the full web-profile `ops-ptc` preset (no + persona, time-context or Bash row). Both arms share the lean composition, so the tier comparison is internally + valid; absolute token counts describe the isolated composition, not the web profile. +- One machine, one day, one model: drift is mitigated by AB/BA interleaving, not eliminated. +- The judge's catalog "correct next steps" definition counts a completed `ops_load_sdk` as a correct step for the + on-demand arm; cross-arm comparisons of that column are partly structural (documented above). +- The driver vendors `dsh-llm-pi-ai` into `node_modules` at run time because the repository manifest deliberately + does not depend on any LLM adapter; the copy is gitignored and recorded here. diff --git a/docs/integration-api.md b/docs/integration-api.md index 49237a7d..6a32e7f4 100644 --- a/docs/integration-api.md +++ b/docs/integration-api.md @@ -65,6 +65,9 @@ resident inspect/glob/grep tools independently of this advanced list; read_files `[]`; disabled wins over allowed, and startup names must be allowed and not disabled. `extraGuidance` is Native-only and is rejected in PTC, rather than silently ignored. PTC also rejects `denyCredentialReads` overrides (its credential boundary remains enforced). +A PTC-presentation row additionally accepts `disclosurePolicy` (`'on-demand'` default, +`'full'` explicit first-round tier, Native-presentation rows reject it) — the same key the +builtin rows parse. `bashPath`, `allowSystemShellFallback` and explicit host-shell hiding intent are retained while `shell: false`; they become effective only when Shell is re-enabled. @@ -76,6 +79,10 @@ No configuration changes the host's own presentation or permission policy. ## Diagnostics and compatibility The integration adds `ops_status` for current scoped diagnostics; see [diagnostics](diagnostics.md). +Diagnostics derive their expected PTC first-round names from the same disclosure plan the +publication uses, so an API that is selectable but not yet loaded (`result` on the +on-demand tier, any unselected pool member) is never a publication gap; only genuinely +missing first-round plan names are. The bundled discovery row now serves built-in/headless compositions only. Unknown custom presets must explicitly opt in; this intentionally removes the previous accidental global discovery entry. Existing component exports remain available with their documented legacy semantics; do not use diff --git a/docs/isolated-ptc-entry.md b/docs/isolated-ptc-entry.md new file mode 100644 index 00000000..052eecd9 --- /dev/null +++ b/docs/isolated-ptc-entry.md @@ -0,0 +1,73 @@ +# Isolated OPS-PTC test entry (#990) + +## Why this entry exists + +A headless comparison can only speak about OPS-PTC if the group it measured +actually entered that routing. Before this entry, the loop fixture assembled a +synthetic `fixture` preset, and the byte probes covered the built-in presets — +so a green number could come from a Standard composition and still look like +OPS-PTC evidence. Issue #980 puts it as the V1 gate: if the actual route is +Standard, the worker is missing, or the routing is unknown, the run is invalid +and must say so. + +## What a group is + +`bootEntryGroup(root, …)` in `test/lib/ptc-entry.mjs` boots one preset group: + +- the real AgentLoop and Session with a scripted provider — no network model + request is ever made; +- the shipped single row (`apply` from the package entry) mounted with only + the target mode enabled, exactly as a profile mounts it, so the ops-ptc row's + real defaults are what the group runs; +- a root directory inside the suite's throwaway `DSH_HOME` — `node test/run.mjs` + creates one home per suite, and the entry refuses any root outside it. + +The environment summary is collected at boot and carried on every report: +plugin commit (read from the git directory, no subprocess), plugin version, host +runtime version, platform (for example `linux-x64`), the worker file with its +sha256, the staged runtime receipt beside it and whether the two digests match, +the sandbox mode and workspace root, the resolved row switches including the +batching guidance default, and the actual `agentPreset` with its mode. The +provider is marked `scripted` and `networkModelRequests` is recorded as 0, +because a mocked provider verifies assembly only and a report must never hide +that. + +## The report contract + +`enterPtc(group)` runs the minimal case first: exactly one scripted provider +request that executes nothing, which is the only request a wrong group ever +earns. Checks run in a fixed order — worker, preset, route, execution — and the +first failure becomes the report's reason. + +- **entered** — the request's top-level surface is exactly `['run_code']` and + the SDK text is present; only then does a second turn invent one fresh probe + file and execute a single program through the worker. The evidence block + records the surface, the preset, the mode, the request counts and the + executed result. +- **invalid** — the report names the reason and carries findings for every + check, but no evidence block and no byte or token numbers of any kind: + - Standard is the deliberate counter-example. It is a real composition, not a + stub, and it fails on `preset` with the route finding showing the native + surface; + - a PTC preset on a native executor fails on `route` — the preset is right, + the transport is not, and the surface may declare no tool at all; + - a row that cannot resolve an executable worker fails at boot on `worker`, + before any request is made. + +Byte and cost accounting is deliberately not collected here; that is the V2 +card's scope (#991). The later cards consume this entry's groups and +environment record instead of rebuilding them. + +## Reproduce + +``` +npm run build +npm run build:ptc +node test/run.mjs progressive-loop +``` + +The filter must select `progressive-loop.test.mjs`, where the four entry +checks live beside the three loop checks. A skip means the worker did not +resolve — run `npm run build:ptc` first; a skip is not a pass. Only freshly +invented fixture files enter these groups; no real session, credential or +conversation text is read or reported. diff --git a/docs/native-kernel.md b/docs/native-kernel.md index 7916dc27..ec443cda 100644 --- a/docs/native-kernel.md +++ b/docs/native-kernel.md @@ -275,7 +275,7 @@ reference pages retains the declared main-body scope independently of stored-row The internal PDF container parser accepts a restricted PDF 1.0-1.7 subset: one classic xref table, an exact final startxref/EOF footer, unique dictionary keys and object -identities, and directly declared stream lengths. Every active xref offset must point +identities, and stream lengths declared either directly or through an indirect reference resolved by the classic xref table. Every active xref offset must point exactly at its object header. Objects cannot overlap; gaps must contain only whitespace or comments. This deliberately rejects incremental updates, encryption, object/xref streams, recovery scans and ambiguous duplicate data. `/Encrypt`, `/ObjStm` and `/XRef` diff --git a/docs/native-output.md b/docs/native-output.md index 375f7fad..e8f5be7f 100644 --- a/docs/native-output.md +++ b/docs/native-output.md @@ -59,3 +59,16 @@ The test reproduces that mismatch with the real host plugin and records the section's ownership. Correcting the host callback to honor current step admission remains a host dependency of #866. OPS does not rewrite that third-party prompt or modify the shared registry, and does not add a permanent name-mapping prompt. + +The PTC side of that mismatch is closed at the supported assembly extension point +(#983, `src/publication/replacements.ts`): when the shadow hides +`read`/`glob`/`grep` for a PTC presentation, the assembled prompt retires the +provider's `tool:read`/`tool:glob`/`tool:grep` sections instead of teaching hidden +names. A kept official whose section still points at a hidden name — the pinned +host grep text cross-references read — is retired with it and one note names the +executable OPS counterpart; a kept section without such a pointer survives. Withdrawing +the replacement restores the provider text verbatim. `test/ptc-guidance-cleanup.test.mjs` +pins this against the real `@deepseek-ai/dsh-tool-fs`/`-fs-search` plugins, including +the byte deltas by section group on the Linux fixture: SDK −2,011 B, the three +guidance sections −338 B, outer wire 0 B. #866 stays open for the Native presentation, +where the host section is intentionally untouched. diff --git a/docs/native-ranges.md b/docs/native-ranges.md index 1bc2fb1b..c5b524a7 100644 --- a/docs/native-ranges.md +++ b/docs/native-ranges.md @@ -12,7 +12,7 @@ A fully returned `ops_read_files` range uses one path header, exact numbered tex [revision=cfd1bc81432fd6b88748b827f46bb4242db17cf8655cf4ea90f3ba0e62065355; more lines; nextOffset=3] ``` -The characters after `N|` are the source line, including leading spaces. Line bodies use reversible JSON escapes: backslashes are doubled, controls and Unicode line separators are escaped, and quotes remain literal. For example, an actual tab appears as `\t`, while the two source characters backslash-plus-t appear as `\\t`. This shared encoding applies to compact ranges and ordinary numbered source output, and prevents source text from forging extra rows or headers. LF/CRLF accounting remains the worker's physical-line accounting. +The characters after `N|` are the source line, including leading spaces. Line bodies use reversible JSON escapes: backslashes are doubled, controls and Unicode line separators are escaped, and quotes remain literal. For example, an actual tab appears as `\t`, while the two source characters backslash-plus-t appear as `\\t`. This shared encoding applies to compact ranges and ordinary numbered source output, and prevents source text from forging extra rows or headers. LF/CRLF accounting remains the worker's physical-line accounting. `ops_edit_file` first matches literal bytes; only after a raw miss may it decode this rendered form for `oldText` and `newText`, with `renderedMatches` in the receipt. It never decodes a raw match or `write_file` content. “More lines” means the file has not been exhausted; use the stated `nextOffset`. “End of file; complete=true” means this range reached the end. The aggregate “requested ranges returned” receipt means every requested range body was returned, not that every whole file was read. Full revisions, total line counts and per-file continuation remain visible. Byte sizes and duplicate counters remain available in canonical/PTC data. Per-file revision receipts omit the repeated `Notice: ` prefix and terminal period; the aggregate `Notice` and every revision/continuation/completeness fact remain intact. diff --git a/docs/presets.md b/docs/presets.md index f150f501..afb30f1a 100644 --- a/docs/presets.md +++ b/docs/presets.md @@ -65,11 +65,12 @@ bundle: - **Pressing the button**: while the `ops-ptc` preset is registered, the plugin writes `selectedDefault: ops-ptc` on `agent-preset-registry` — the same volatile field the Agent presets settings page writes. That click is the only thing that overrides a manual choice. -- **Doing nothing**: nothing is written. Apply, a remount caused by saving another switch, and disabling the row - all leave the current default alone, so a preset picked on the settings page survives restarts. -- **Disabling or unloading the `dsh-ops` row**: releases the registration and leaves the default alone, even when - it names `ops-ptc`. Switch the default to another preset before removing the row, otherwise new sessions may - try to resolve a preset that no longer exists. +- **Doing nothing**: nothing is written. Apply and a remount caused by saving another switch leave the current + default alone, so a preset picked on the settings page survives restarts. +- **Disabling or unloading the `dsh-ops` row**: releases the registration, then checks the live default. If it still + names `ops-ptc`, the row lease restores the host's `config.default`; a default naming any other preset is never + rewritten. A missing or rejecting settings service reports the leftover and asks the operator to choose another + default instead of pretending the profile was repaired. - The button answers with a status: `applied`, `no-update` (it is already the default), or `unavailable`/`failed` with the reason — the preset is not registered, the host settings service is missing, or the write was rejected. - The written value persists through the host settings layer into the profile's `cordis.patch.yml` (an @@ -82,9 +83,9 @@ In the host picker the preset is named `OPS (PTC)` and described as ## Setting another default preset manually -`ops-ptc` becomes the default only through the button above (or the settings page). Nothing restores it -automatically, so pick another default before you disable or unload the row. Other default presets are still -decided by the host/operator: +`ops-ptc` becomes the default only through the button above (or the settings page). Removing the row restores the +host default only when the current selection is still `ops-ptc`; it never restores `ops-ptc` itself. Other default +presets are still decided by the host/operator: - **Host Agent presets settings page**: picking an already registered preset such as `standard` or `ptc` persists `selectedDefault`. @@ -107,10 +108,11 @@ default comes from that profile's own composition. ### Risks and rollback -- **After pressing the button**: the choice is an ordinary settings-page selection, so it survives restarts. It - is not restored when the row is disabled or unloaded; pick another default before removing the row. -- **Before disabling or unloading**: set the default to another preset on the settings page. The plugin no longer - rewrites it, and new sessions otherwise try to resolve a preset that vanished with the row. +- **After pressing the button**: the choice is an ordinary settings-page selection, so it survives restarts. If the + row is removed while this choice is still active, the lease restores the host default instead of stranding it. +- **Before disabling or unloading**: no manual rewrite is required while the default is `ops-ptc`; the row handles + that case. A default belonging to another preset is left exactly as the operator chose it. If restoration cannot + be persisted, choose another default on the settings page after the warning. - A missing fragment or a host upgrade that breaks parsing leaves the preset unavailable while the tools keep working; the default preset stays untouched and the button reports `unavailable`. Regenerate the fragment and restart the host to recover. @@ -126,9 +128,9 @@ default comes from that profile's own composition. | Plugin-manager row | a dedicated preset row | no preset row; lives and dies with the `dsh-ops` row | | Publication surface | official PTC tools | same as the `ptc` section: `run_code` under real PTC presentation plus the same-stack typed SDK file tools | | Compaction | official `compaction-basic` | official `compaction-basic` as well; no `dsh-compress-codex` | -| Batch-call guidance | none | `batchingGuidance`: defaults true in code and the packaged `ops-ptc` section, including legacy whole-row overrides that omit the section; explicit false disables it; one lesson at session start, then shape notices for root `run_code` programs (`bypass`/`plan`/`shape`, each latched per turn and re-armed by compliance) appended as `additionalContexts`; text in the editable `prompts/batching-guidance.md` | +| Batch-call guidance | none | `batchingGuidance`: defaults true in code and the packaged `ops-ptc` section, including legacy whole-row overrides that omit the section; explicit false disables it; one lesson riding the first accepted root `run_code` result, then shape notices for root `run_code` programs (`bypass`/`plan`/`shape`, each latched per turn and re-armed by compliance); both are delivered as `additionalContexts` that never enter logged history; text in the editable `prompts/batching-guidance.md` | | Single-row section | `ptc` | `ops-ptc` (same keys plus `batchingGuidance`) | -| Default preset | decided by the host/operator | set once by the detail page's button (`applied`/`no-update`); never forced or restored automatically | +| Default preset | decided by the host/operator | set once by the detail page's button (`applied`/`no-update`); never forced; unloading restores the host default only when the current selection is still `ops-ptc` | | Ordering | order 2 | order 6, sorted after the official rows | Having both modes over one tool surface is the point of the contrast arm: `ptc` keeps the official preset diff --git a/docs/ptc-cost-accounts.md b/docs/ptc-cost-accounts.md new file mode 100644 index 00000000..55a21e0e --- /dev/null +++ b/docs/ptc-cost-accounts.md @@ -0,0 +1,106 @@ +# Four-account OPS-PTC cost ledger (#991) + +## Why this ledger exists + +The common accounting mistakes for OPS-PTC are structural: counting the +`tools:sdk` section twice by adding it on top of a system total that already +contains it, mixing the Native tool schemas into a PTC measurement, or +misplacing the dsh-ops guidance — the one-time teaching is same-step +`additionalContexts` on the accepted tool result and is never logged, the shape +notices are what the guidance account records, and neither is the system prompt. +V1 (#990, `test/lib/ptc-entry.mjs`) already guarantees the measured group truly +entered OPS-PTC; this suite extends that entry with the byte accounting and never +lets the four answers blur together. + +`test/ptc-cost-accounts.test.mjs` boots one isolated group through +`bootEntryGroup`, proves the routing with `enterPtc`, then walks one scripted +scenario — teaching, a thin program that earns a plan notice, a compliant +`ops_load_sdk` load of `glob`/`grep`/`read_files`, and a same-config repeat — +and reports each account as raw UTF-8 bytes with an item-name list. + +## The four accounts + +Readings below are one run at commit `205cb94`, plugin 0.5.5, host +0.2.2-rc.1, linux-x64, staged worker sha256 `2b5f988d…`, preset `ops-ptc`, +resident + dynamic + batching guidance on, official read/glob/grep mounted. +Absolute bytes are only comparable inside this configuration. The history row records +the post-#929 composition: the one-time teaching is same-step context, not part of the +account. + +| Account | First request (resident only) | After the load | Repeat | +| --- | --- | --- | --- | +| system (delivered rendered prompt) | 12,782 B | 37,892 B | 37,892 B | +| — of which `tools:sdk` | 11,739 B | 36,849 B | 36,849 B | +| wire (outer tool schemas) | 2,500 B | 2,500 B | 2,500 B | +| history (dsh-ops notice) | — | + notice:plan 284 B | unchanged | + +- **system** is the delivered rendered prompt, byte-measured exactly as the + provider request carries it; the suite asserts the request's system message + equals the idle assembly rendering for every config point. Its item list at + the first request: `harness:identity` 48, `tools:ptc-only` 159, `tool:read` + 130, `tool:write` 92, `tool:edit` 83, `tool:glob` 75, `tool:grep` 133, + `dsh-ops:repository-tooling:mode-ptc` 307, `tools:sdk` 11,739. +- **sdk** is the single `tools:sdk` section inside system — a strict subset, + never an extra line item. The normalization is pinned by assertion: system = + per-section bytes joined by `\n\n`, i.e. 12,766 + 8 joins × 2 B = 12,782, + reproducible by hand from the item list above. +- **wire** is `JSON.stringify` of the request's outer tools — always exactly + `run_code` here, byte-identical on every request and equal to the idle + assembly's wire. +- **history** is the dsh-ops guidance the suite records from the messages delivered on + each request for contributor `batching-guidance`: the plan notice the thin program + earns (once, 284 B). The one-time teaching never appears there: it rides the first + accepted root `run_code` result as same-step `additionalContexts` (contributor + `batching-guidance:lesson`), never logged as user history, so this account carries no + teaching bytes; its full text is 8,729 B. The suite asserts neither touches the system + account — the post-notice system sha equals the baseline. + +## The arithmetic that must hold + +Loading `glob`/`grep`/`read_files` is the hand-computed example: between the +steady and loaded snapshots every other section is byte-identical and the +system delta equals the SDK-section delta exactly (+25,110 B), so the change +lands in system exactly once — the SDK subset is never counted twice. Two fresh +groups running the same fixture and code must agree on every byte (sha-equal), +and the same config repeated within a session must be byte-identical. + +## SDK names against the real official schemas + +The scenario mounts the host's own official file-tool plugins before the row +(the same `FsTools`/`SearchTools` pair the budget suite mounts), because SDK +names are only meaningful against the schemas they replace. The official +schemas measure 438 B (`read`), 750 B (`glob`) and 702 B (`grep`) as +name+description+parameters JSON. + +On this baseline the shipped row does **not** remove those official bindings +from the PTC SDK — `read`, `glob`, `grep`, `write` and `edit` stay declared +next to `ops_inspect_local_file` and `ops_load_sdk` (the PTC hiding fix is the +open PR #961; the #978 contrast numbers were taken at its head, not here). The +suite pins this baseline on purpose: when #961 lands, its assertion turns red +and the accounts must be re-measured and re-documented. Without the official +provider the suite reports the comparison as **unmeasurable** with the missing +names, never as a quiet pass. + +## What is deliberately not measured + +- No real model request ever leaves the fixture: the provider is scripted and + every printed record carries `provider: "scripted"`, + `networkModelRequests: 0`. +- No tokens, no cache usage, no cost: bytes are not tokens, and bytes/4 is not + a measured token count. Token accounting is a provider-output question (V5 + territory). +- A missing worker skips the whole suite visibly — a skip is not a pass. +- Absolute bytes never cross versions or groups; percentages are not claimed. + +## Reproduce + +``` +npm run build +npm run build:ptc +DSH_OPS_WORKER_BIN=$PWD/runtime/linux-x64/dsh-ops-worker node test/run.mjs ptc-cost-accounts +``` + +The suite prints one JSON line per snapshot with the four accounts and the +scripted/0 marker; set `DSH_OPS_PTC_COST_REPORT=` for the full itemized +evidence. `node test/run.mjs ptc-cost` runs this suite together with the +declaration-cost suite that shares its filter. diff --git a/docs/ptc-outcomes.md b/docs/ptc-outcomes.md index d615473c..abfbcf65 100644 --- a/docs/ptc-outcomes.md +++ b/docs/ptc-outcomes.md @@ -11,7 +11,7 @@ See [file mutation contracts](file-contracts.md) for exact revision fields, same | Status | Required fields | Caller action | | --- | --- | --- | | `completed` | `data` | Consume the requested operation's result. | -| `partial` | `data`, incomplete `coverage`, `continuation` | Inspect retained data; explicitly retrieve the remaining range or stored page. | +| `partial` | `data`, incomplete `coverage`, `continuation`, optional `nextAction` | Follow `nextAction` for the explicit continuation call; a page reporting `complete=true` is the last one. | | `rejected`, `conflict`, `error`, `unknown` | `error` and optional retained `data` | Follow `error.code`, `effectState`, `retry` and `nextAction`. | A successful plan is completed planning, not completed execution. `result.put` is @@ -44,6 +44,16 @@ that cannot safely resume produces `RESULT_PARTIAL` with retained data and expli guidance, not a fabricated continuation. All retrieval still checks current authority and available SDK bindings. No helper automatically executes or retries a tool. +`nextAction` recommendations are executable, not decorative: they name one concrete +call with literal arguments — the same path at `nextOffset` for a paged read, a hex +`view` for non-UTF-8 bytes, a registered listing API for a rejected directory path, +a fresh-revision read for `REVISION_CONFLICT`. Recommendations respect the calling +step: an API this step did not admit is never named (the text then asks for a +concrete file path instead), no write-class API is ever recommended, and a +read-only scope never receives a re-submission suggestion. An `after-input-change` +failure whose worker guidance is absent says plainly that an identical request +fails identically; guidance that already names a recovery stays unchanged. + Search scan coverage, captured snapshot completeness and page completeness are separate facts. Exhausting a stored page does not prove that the original scan was complete. Read `data.source` and the original search report when consuming snapshots; diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md index b58646e9..89cc4a24 100644 --- a/docs/troubleshooting.md +++ b/docs/troubleshooting.md @@ -11,6 +11,10 @@ collect [runtime evidence](windows-runtime-evidence.md) before assigning a cause --- +## Bash disappears or registration repeats during session changes + +Session Bash publication is per agent and generation-aware. Overlapping consumers no longer register the same name twice, a queued registration updates the entry only when it settles, and one failed generation is reported once rather than retried in a synchronous storm. A `turn/start` event re-asserts the current authority, so an interrupted turn cannot leave `ops_bash` missing until an unrelated asynchronous event happens to arrive. If a name conflict is reported, capture that single warning and the following turn boundary rather than restarting the host repeatedly (#579, #842, #873). + ## Installation and payloads ### `bash-win32-x64` is missing under `node_modules/@dsh-ops/` @@ -83,6 +87,10 @@ Long scripts, heredocs and multi-layer quotes and backslashes make the model esc A key the `ops_bash` schema does not declare is refused before execution, and the error names both the unknown keys and the accepted ones. This is deliberate — nothing is silently ignored. +### The model calls `bash` instead of `ops_bash` + +This package publishes exactly one shell tool, named `ops_bash`; there is no `bash` alias. The tool table is assembled by the host from the names this package declares once (`publicToolName` in `src/policy.ts`), and a misnamed call never reaches this package: the host answers with its own unknown-tool message, so any did-you-mean correction also belongs to the host, not here. Point the model back to the real name — the tool table and the prompt both spell `ops_bash` — instead of expecting a silent alias. + ### Bash script files, PATH precedence and empty input `script_path` always selects a Bash script; the extension does not select an @@ -109,6 +117,28 @@ What to do: - Or switch the session to `danger-full-access`, where `bashPath` runs directly. - Check with the profile's own copy: `node "\node_modules\dsh-ops\bin\dsh-ops.mjs" ladder --config ` reports `restricted: yes/no` for the resolved bash. +### A full-access `ops_bash` child seems to keep the restricted Low token + +The session mode projects straight through in full access: the selected payload +spawns without `sandbox.confine`, a plain `bashPath` runs directly, and the child +carries the session's own token. Compare the tokens from the same caller before +assigning a cause (#581): + +```console +MSYS2_ARG_CONV_EXCL='*' /c/Windows/System32/whoami.exe /groups +``` + +The `Mandatory Label` line is the child's integrity level: `S-1-16-4096` is Low, +`S-1-16-8192` Medium, and an elevated caller reports higher. A confined-mode child +is expected to differ from the host shell; a full-access child must match the +session's own level. A full-access child that still reports Low is a host +token-construction difference, not a dsh-ops setting: collect both `whoami.exe +/groups` outputs, the DSH core, OPS and Bash payload versions, and the exact +executable paths, and report them with the [runtime evidence +guide](windows-runtime-evidence.md). The plugin-side projection is guarded by +`test/shells-full-access-confinement.test.mjs`, which fails if a full-access +call ever routes through `confinedBash` or `sandbox.confine`. + ### Restricted Bash is still preparing, or its last preparation was interrupted Symptom, in a read-only or workspace-write session: an `EXECUTOR_UNAVAILABLE` whose second segment names a stage, for example `EXECUTOR_UNAVAILABLE: bash-staging-failed: …`, `bash-generation-quarantine-failed: …` or `bash-lifecycle-lock-busy: …`. @@ -134,17 +164,19 @@ publishes the completed tree with an atomic sibling rename. Later calls check on prepared `niu.exe` still exists; they do not clone, hash, seal, write an execution receipt or dispose a per-call tree. Files the shell writes into its state directories are never inspected. -The prepared runtime must be **outside the session workspace**. Every call checks that boundary -and still uses the host's subprocess service and sandbox `confine` in restricted modes. In read-only and -workspace-write modes, the host denies writes to both the source and prepared payload outside the -workspace. Full-access mode keeps its normal host policy; the prepared directory is not a separate -security boundary against the account or an administrator. +The prepared runtime must be **outside the session workspace** in confined sessions. Read-only and +workspace-write modes check that boundary on every call and still use the host's subprocess service +and sandbox `confine`; the host denies writes to both the source and prepared payload outside the +workspace. Full-access mode keeps its normal host policy and runs a prepared root inside the +workspace — the default `~/.dsh` under a home workspace — instead of refusing it, because it is +not a separate security boundary against the account or an administrator (#870). - On first preparation only, Windows scanner/indexer handles can delay the atomic publication rename. Only `EPERM`/`EACCES`/`EBUSY` rename refusals are backed off, asynchronously, for up to 10 seconds; a persistent failure names the rename in `EXECUTOR_UNAVAILABLE`. Commands are never retried. -- Put `DSH_HOME` outside the workspace before starting DSH. A workspace containing the prepared - runtime is refused rather than silently moving the runtime or running without confinement. +- For confined sessions, put `DSH_HOME` outside the workspace before starting DSH. A workspace + containing the prepared runtime is refused in read-only and workspace-write modes rather than + silently moving the runtime or running without confinement; full access runs it where it is. - `DSH_OPS_EXEC_ROOT` and `NIUBASH_VERIFY=deep` are deprecated and ignored for new calls. Antivirus allow-list rules should target the fixed prepared location, not a per-call temporary directory. - For leftovers from older releases, stop **every OPS host and Bash job**, then use the installed @@ -177,6 +209,66 @@ security boundary against the account or an administrator. switching sessions do not repair it. Selecting an arbitrary Git Bash is not a restricted-mode workaround: only a compatible pinned MSYS build may run restricted, never an unconfined fallback. +### `niu`-Git hooks fail with `#!/usr/bin/env sh` while `#!/bin/sh` works + +Symptom: a repository hook installed through the niu-Git command — for example a +husky v9 hook — fails when its shebang is `#!/usr/bin/env sh`, while the same +script with `#!/bin/sh` runs. The observed pair is the diagnosis. The likely +mechanism — the `env sh` lookup resolving through the payload's own interpreter +machinery instead of the direct `sh` face — is an upstream inference this +repository has not verified inside the niu-Git binary tree (#958). + +What to do: + +- Confirm the pair from the same hook file: only the shebang differs, and the + `#!/bin/sh` form succeeds. +- Rewrite the affected hooks to `#!/bin/sh`; husky hooks are plain shell + scripts, so the shebang is the whole change. +- Report the payload version and both hook texts to the payload maintainers; + they can confirm or correct the interpreter-resolution attribution + (unresolved-upstream). + +### `ops_bash` flashes a console window on Windows 10 with Windows Terminal + +Symptom: every `ops_bash` call briefly flashes a black console window when +Windows Terminal is the system's default terminal application on Windows 10. +What is observed is the comparison, not the mechanism: switch the default back +to the Windows Console Host and see whether the flashing stops. The likely +mechanism — niubash being a console-subsystem binary launched without an +interactive parent, so each call asks for a console host and the Windows-10 +terminal hand-off re-creates a visible console — is an upstream inference this +repository has not verified; no dsh-ops setting controls console hosting, and +the shipped binary cannot be patched here (#828). + +What to do: + +- Verify the default terminal setting (Settings → For developers → Terminal) + and compare the flashing with the default switched back to the Windows + Console Host; that observation pair is the diagnosis. +- A hidden-console niubash build is upstream payload work (unresolved-upstream). + +### Antivirus quarantined `niu.exe` from the payload + +Symptom: selection reports `EXECUTOR_UNAVAILABLE ... Incomplete Niubash payload: +niu.exe` while every other pinned file is intact — only `niu.exe` is missing +(#771). Preparation copies each pinned member verbatim, so a preparation that +stops at one executable is not a dsh-ops bug: the matching Defender or ESET +event is the security product quarantining that executable. This is an external +event; no repository change can prevent it, and the diagnostic naming the +missing member landed with #844. + +What to do: + +- Collect the matching event before restoring: Defender + `Get-MpThreat` / `Get-MpThreatDetection` (or the ESET detection log) with its + timestamp, action and the exact file path. +- Restore the complete pinned tree with `dsh-ops provision-shells --bash`, which + re-verifies every member. +- An allow-list rule, if you choose one, targets the fixed prepared location — + never disable protection or broadly trust the payload tree. +- If tampering is suspected, hash the restored `niu.exe` and compare it with the + receipt. + ### A second restricted Bash fails while another Bash is still running The `CreateFileMapping S-., Win32 error 5` failure in #363 came @@ -283,6 +375,10 @@ Before the first restricted `run_code` / `pwsh` / `bash` / `ops_bash` call in a This step runs the package's own `bin/prepare-workspace.ps1` with Windows PowerShell 5.1 **as the current user, before sandbox confinement applies**, only on Windows in workspace-write mode, and changes only the workspace root's own DACL. Read-only and danger-full-access sessions skip it. +A folder that is already ready answers without any reparse-point inspection, so a OneDrive placeholder root passes the fast path (#744). The reparse check runs only on the branch that must add the ACE: the proven cloud-files family — the exact 0x9000?01A shape (`IO_REPARSE_TAG_CLOUD`, `CLOUD_ROOT`, `CLOUD_ON_DEMAND`) — keeps a real directory entry behind the point and is allowed through, while the rest of Microsoft's 0x90000000-0x9FFFFFFF band is unproven and fails closed exactly like every other tag — mount point, symlink, unknown or unreadable — with the tag named. + +Both Microsoft-fixed WSL redirector prefixes (`\\wsl.localhost\…` and `\\wsl$\…`) answer `ready` after the existence and container validation and before any NTFS inspection, because the 9P filesystem behind them exposes no security descriptor at all and `Get-Acl` there used to abort every workspace-write call (#669). This is a prefix decision, not a catch: a missing path still fails, and on any other path a real ACL error still fails closed. + The message names the cause: - `PowerShell execution policy blocked dsh-ops's own preparation script …`: dsh-ops passes `-ExecutionPolicy Bypass` (earlier versions did not, so the Windows client default `Restricted` blocked every shell tool, #66). A refusal now means Group Policy (MachinePolicy/UserPolicy) enforces the policy; `Set-ExecutionPolicy -Scope CurrentUser` cannot override it. Check `powershell.exe -NoProfile -Command Get-ExecutionPolicy -List`, ask an administrator, or use danger-full-access for the session. diff --git a/docs/usage.md b/docs/usage.md index 1e441e54..e3da8c41 100644 --- a/docs/usage.md +++ b/docs/usage.md @@ -24,9 +24,10 @@ this row before removing obsolete overrides; OPS does not rewrite profiles autom Standard and Cordis publish resident Native tools; Minimal has no file baseline; PTC publishes typed APIs with a separate SDK loader; OPS PTC (`ops-ptc`) is registered at plugin runtime and keeps -that surface, adding the batch-call guidance from the editable `prompts/batching-guidance.md` once at -session start, then shape notices for root `run_code` programs (`bypass`/`plan`/`shape`, each latched per - turn) appended as `additionalContexts` (on in the packaged bundle; see [presets](presets.md)). +that surface, adding the batch-call guidance from the editable `prompts/batching-guidance.md`: one lesson +riding the first accepted root `run_code` result, then shape notices for root `run_code` programs +(`bypass`/`plan`/`shape`, each latched per turn), all delivered as same-step `additionalContexts` that never +enter logged history (on in the packaged bundle; see [presets](presets.md)). The default preset is set by one button on the row's detail page; nothing forces or restores it automatically. Residency, dynamic loading, Bash, file replacement and Windows host-shell hiding are independently controlled. Global gates preserve saved per-preset @@ -275,7 +276,7 @@ Use `ops_bash`'s `command` for short commands and pipelines. For scripts longer `script_path` accepts only the script path, so the script body is not repeated in the `ops_bash` arguments. `command` and `script_path` are mutually exclusive and exactly one is required. This interface cannot repair JSON that the model or host failed to parse; an unparseable tool call still requires a host/model retry. -`ops_bash` requires a non-empty `description` and keeps the host Bash fields and result kinds: `foreground`, `background`, or `promoted`. Its default timeout is 120 seconds and the cap is 600 seconds. When the host jobs service is available, `run_in_background: true` returns a job ID immediately; a foreground timeout promotes the same running job. Use the host `job_output` and `job_kill` tools to collect or stop it. Without host jobs, the background parameter is absent and a foreground timeout stops the command. Unknown arguments are rejected before execution. +`ops_bash` requires a non-empty `description` and keeps the host Bash fields and result kinds: `foreground`, `background`, or `promoted`. Its default timeout is 120 seconds and the cap is 600 seconds. When the host jobs service is available, `run_in_background: true` returns a job ID immediately; a foreground timeout promotes the same running job. Use the host `job_output` and `job_kill` tools to collect or stop it. `job_kill` terminates the whole process tree: daemons the command started and left attached to its pipes (an adb server, a dev-server watcher) go down with it, so stop such daemons on their own channel first, start them detached, or begin daemon-like work with `run_in_background` so the tree decision stays deliberate. Without host jobs, the background parameter is absent and a foreground timeout stops the command. Unknown arguments are rejected before execution. The host job list and waiting view use `description` as the task label. Both `background` and `promoted` results include that same `label` beside `jobId`, and the rendered result quotes it. When migrating from the old `ops_run_background` / `ops_job_output` / `ops_job_kill` tools, start work with `ops_bash`'s `run_in_background` (or let a foreground call promote), then use the host `job_output` / `job_kill` with the returned ID. Keep `description` concise so the job is recognizable while waiting. diff --git a/docs/windows-runtime-evidence.md b/docs/windows-runtime-evidence.md index a04240ca..9f76f768 100644 --- a/docs/windows-runtime-evidence.md +++ b/docs/windows-runtime-evidence.md @@ -59,6 +59,12 @@ payload/root, whether an owned normal-token holder is alive, stderr, exit value, and a restricted positive control. Stop only jobs you own; do not kill all Bash processes or rewrite foreign ACLs to make a comparison pass. +An unrestricted MSYS process that created the per-user section first omits every +restricted-logon ACE, so the restricted token cannot reopen it; the proxy only arms +**new** objects. `ops_bash` classifies this message as `EXECUTOR_UNAVAILABLE` with +`sandbox.runnerFailed`, like the signal-pipe failure, instead of presenting a bare +non-zero exit (#409); it is never a user authorization denial. + `3221225794` and `-1073741502` represent `0xC0000142`. The status alone does not identify the failing DLL or prove which process reached the command. A manual launch of a staged restricted payload under a normal token is unsupported: its diff --git a/docs/windows-runtime-evidence.zh.md b/docs/windows-runtime-evidence.zh.md index d9de93e1..ba6cc3b1 100644 --- a/docs/windows-runtime-evidence.zh.md +++ b/docs/windows-runtime-evidence.zh.md @@ -47,6 +47,11 @@ Get-FileHash -LiteralPath '<实际 msys-2.0.dll 路径>' -Algorithm SHA256 是否仍存活、stderr、退出值及受限正向对照。只停止自己拥有的任务,不要杀死所有 Bash 或重写外部对象 ACL 来让测试通过。 +先以非受限令牌创建该用户 section 的 MSYS 进程不包含任何受限登录会话 ACE,受限令牌因此 +无法重新打开它;代理只为**新建**对象附加权限。`ops_bash` 现在把这条消息与信号管失败一样 +归类为 `EXECUTOR_UNAVAILABLE`(带 `sandbox.runnerFailed`),不再只呈现裸的非零退出码(#409); +它绝不是一次用户授权拒绝。 + `3221225794` 与 `-1073741502` 都表示 `0xC0000142`;数字本身不说明哪个 DLL 失败,也不证明哪个进程已执行命令。不支持用普通令牌手动启动 staged 受限载荷:其 IPC 代理要求受限 Low 令牌。请通过 OPS 工具测试宿主实际 confinement 路径,保留完整 stderr, diff --git a/docs/wsl-readonly.md b/docs/wsl-readonly.md index ed758ed6..9db70e11 100644 --- a/docs/wsl-readonly.md +++ b/docs/wsl-readonly.md @@ -1,6 +1,6 @@ # Optional WSL file reads -On the tested DSH rc.2 host, Windows `Get-Acl` cannot prepare a WSL UNC workspace. In workspace-write mode, the unchanged PTC runtime therefore refuses to start, and legacy cache preparation can be unavailable. Inline Native reads still work; a byte-clipped read falls back to exactly the original Windows result, including its incomplete-output flags. OPS does not change permission mode or bypass this host refusal. A separate full-access fixture verifies shared PTC file bindings; this is validation scope, not a recommendation to widen user authority. +Windows `Get-Acl` cannot prepare a WSL UNC workspace: `\\wsl.localhost\` and `\\wsl$\` are the 9P redirector, which exposes no NTFS security descriptor at all, so an unguarded preparation died on every WSL workspace and workspace-write `run_code` failed wholesale (#669). The workspace bootstrap now answers `ready` for exactly those two Microsoft-fixed prefixes after the existence and container validation and before `Get-Acl` is touched — a prefix decision, not a catch that swallows ACL errors: a missing path fails exactly like any other missing workspace, and on any other path a real ACL failure still fails closed. Legacy cache preparation can still be unavailable on such roots. Inline Native reads work; a byte-clipped read falls back to exactly the original Windows result, including its incomplete-output flags. OPS does not change permission mode or bypass a host refusal. A separate full-access fixture verifies shared PTC file bindings; this is validation scope, not a recommendation to widen user authority. In full-access mode, hosts without `sandbox.prepareResultStorage` intentionally omit optional storage rather than preparing workspace ACLs just for caching (#411). The WSL acceptance script verifies the absent storage authority and exact equality with the opt-out Windows result in either legacy-host case. When the host exposes cache-only preparation, a missing reference still fails acceptance; paging must succeed. The Windows cache-contract suite separately exercises real private ACL preparation, spill, paging and cleanup through a test implementation of that capability; it does not establish that a published host provides it. @@ -12,9 +12,9 @@ Apply the environment setting to the DSH daemon/desktop process, then restart th On Windows x64, set `DSH_OPS_WSL_READS=1` in the environment that starts DSH to opt in to Linux-side `ops_glob`, `ops_grep`, and `ops_read_files` for a session rooted at `\\wsl.localhost\\...` or `\\wsl$\\...`. This targets repeated metadata access through the WSL network filesystem. WSL startup and verification cost can make a small read slower; measure the actual workspace before enabling it broadly. -The installed package must contain the same-version, receipt-verified `runtime/linux-x64/dsh-ops-worker` as well as the normal Windows worker. The distribution must already be installed, support Linux x64, and provide `/usr/bin/wslpath` and `/usr/bin/sha256sum`. OPS neither installs WSL nor downloads a runtime. An explicit `workerPath` or `DSH_OPS_WORKER_BIN` selection keeps its existing Windows execution path. Turning the opt-in off restores the existing path for new calls. +The installed package must contain the same-version, receipt-verified `runtime/linux-x64/dsh-ops-worker` as well as the normal Windows worker. The distribution must already be installed, support Linux x64, and provide `/bin/sh`, `/usr/bin/wslpath` and `/usr/bin/sha256sum`. OPS neither installs WSL nor downloads a runtime. An explicit `workerPath` or `DSH_OPS_WORKER_BIN` selection keeps its existing Windows execution path. Turning the opt-in off restores the existing path for new calls. -The worker receives JSON data through `wsl.exe --distribution --exec `; no command shell or model-supplied code runs. Inputs must resolve to the same distribution as the session. Windows drive mappings are resolved anew for every call, including credential deny roots and the worker payload. The Linux-visible payload hash must match the packaged receipt. Windows case-insensitive credential matching and reserved-name restrictions remain enabled inside the Linux traversal. Links and special objects retain no-follow checks; an invalid filename is a scan gap, not a complete count. +The worker receives JSON data through `wsl.exe --distribution --exec `; the worker itself runs no command shell or model-supplied code. The preparation is one fixed `/bin/sh -c` script per call (#576): the script text is a constant, and every drive root, credential deny root and the worker payload travel as positional arguments, so no Windows path can alter the command shape — drive mappings are still resolved anew for every call, and the Linux-visible payload hash must still match the packaged receipt. Inputs must resolve to the same distribution as the session. Windows case-insensitive credential matching and reserved-name restrictions remain enabled inside the Linux traversal. Links and special objects retain no-follow checks; an invalid filename is a scan gap, not a complete count. Tools, argument schemas, prompt text, relative path bases, and host read observations stay the same. Only documented result path fields become UNC paths; file contents are untouched. Writes, Bash, PTC execution, and result-cache ownership remain on the existing Windows path. diff --git "a/docs/\344\275\277\347\224\250\346\214\207\345\215\227.md" "b/docs/\344\275\277\347\224\250\346\214\207\345\215\227.md" index 5422b6a2..e871a1d7 100644 --- "a/docs/\344\275\277\347\224\250\346\214\207\345\215\227.md" +++ "b/docs/\344\275\277\347\224\250\346\214\207\345\215\227.md" @@ -16,7 +16,7 @@ Native 文件路径(含 copy/move 的目标)、省略的搜索路径,以 本检出是未发布候选。Bundle 只声明一行 `dsh-ops`:五个预设、动态加载、数据采集、TUI 联动与伴随插件联动都在这一行的插件页里配置(其中第五个预设 `ops-ptc` 由插件运行时注册,定义来自随包片段 `presets/ops-ptc.patch.yml`,不是插件管理器行)。五个预设与 discovery 默认开启,数据采集与 TUI 联动默认关闭。已移除的 `mode-*`、`discovery`、`compat`、`panel`、`tui`、`shell`、`file`、`ptc`、`hide-official` 行的已保存覆盖必须删除,它们在新 bundle 中没有对应行。 -消费者只为匹配预设发布工具。Standard/Cordis 提供 Native 常驻文件工具,Minimal 没有文件常驻基线;PTC 提供类型化 API 与独立 SDK loader;`ops-ptc` 继承同一工具面并默认开启批量调用提示(`batchingGuidance`:会话首轮一次英文教学 + 按根 `run_code` 程序形状(`bypass`/`plan`/`shape`,每类每轮一次、合规解锁)以 `additionalContexts` 纠偏;文案在可编辑的 `prompts/batching-guidance.md`;YAML 可关;默认预设不自动强制:详情页按钮点一次即设为 `ops-ptc`,apply/重挂载/卸载都不改你自己选的默认,详见[预设与默认模式](预设与默认模式.md))。常驻、动态、Bash、文件替换和 Windows 宿主 Shell 隐藏独立控制,总开关保留各预设保存意图。显式消费者共用装配器,未列出的预设保持原样;内置 Native discovery 可独立于常驻面启用。[配置参考](配置参考.md)说明原生开关页、精确自定义 id 与默认值;迁移见[升级说明](升级与卸载.md#从-04x-迁移到-05x)。 +消费者只为匹配预设发布工具。Standard/Cordis 提供 Native 常驻文件工具,Minimal 没有文件常驻基线;PTC 提供类型化 API 与独立 SDK loader;`ops-ptc` 继承同一工具面并默认开启批量调用提示(`batchingGuidance`:随首个被接受的根 `run_code` 结果英文教学一次 + 按根 `run_code` 程序形状(`bypass`/`plan`/`shape`,每类每轮一次、合规解锁)纠偏,均以 `additionalContexts` 投递、从不进入会话历史;文案在可编辑的 `prompts/batching-guidance.md`;YAML 可关;默认预设不自动强制:详情页按钮点一次即设为 `ops-ptc`,apply/重挂载/卸载都不改你自己选的默认,详见[预设与默认模式](预设与默认模式.md))。常驻、动态、Bash、文件替换和 Windows 宿主 Shell 隐藏独立控制,总开关保留各预设保存意图。显式消费者共用装配器,未列出的预设保持原样;内置 Native discovery 可独立于常驻面启用。[配置参考](配置参考.md)说明原生开关页、精确自定义 id 与默认值;迁移见[升级说明](升级与卸载.md#从-04x-迁移到-05x)。 0.5 中,开启原版工具替换不保证条件性工具从模型工具表消失。OPS 对应项采用条件准入时,原版注册仍可见;只有同一已捕获步骤 toolSet 准入该对应项且消费者当前仍有权限时,OPS 才在分派时拒绝原版调用。对应项缺失、撤销或尚未就绪时,不得连带禁用原版工具。安全的无条件常驻替换仍可使用宿主限制。OPS 不包装原版定义,也不在准入条件或提示词组装中隐藏工具。 @@ -242,7 +242,7 @@ dsh-ops panel --port 8123 # 固定回环端口;默认 0 由系统分配 `script_path` 只接收脚本路径,脚本内容不会重复进入 `ops_bash` 的工具参数;`command` 与 `script_path` 必须二选一。该接口不会替模型修复非法 JSON,工具调用本身若未能解析,仍需由宿主或模型重试。 -`ops_bash` 必须提供非空 `description`,输入字段和结果类型与宿主 Bash 保持一致:`foreground`、`background` 或 `promoted`。默认超时 120 秒,上限 600 秒。宿主 jobs 服务可用时,`run_in_background: true` 立即返回 job ID;前台等待超时会把同一个运行中任务交给宿主管理。用宿主 `job_output` 收集输出、`job_kill` 停止任务。没有 jobs 服务时不声明后台参数,前台超时会停止命令。未知参数在执行前被拒绝。 +`ops_bash` 必须提供非空 `description`,输入字段和结果类型与宿主 Bash 保持一致:`foreground`、`background` 或 `promoted`。默认超时 120 秒,上限 600 秒。宿主 jobs 服务可用时,`run_in_background: true` 立即返回 job ID;前台等待超时会把同一个运行中任务交给宿主管理。用宿主 `job_output` 收集输出、`job_kill` 停止任务。`job_kill` 会终止整棵进程树:该命令启动后仍挂在其管道上的守护进程(如 adb server、dev-server watcher)会被一并终止;请先在守护进程自己的通道上停止它们、以分离方式启动,或对守护进程类工作直接用 `run_in_background` 开始,让终止范围始终是明确决定。没有 jobs 服务时不声明后台参数,前台超时会停止命令。未知参数在执行前被拒绝。 宿主任务列表和等待视图使用 `description` 作为任务标签。`background` 和 `promoted` 结果在 `jobId` 旁返回相同的 `label`,显示文本也会引用该标签。从旧版 `ops_run_background` / `ops_job_output` / `ops_job_kill` 迁移时,用 `ops_bash` 的 `run_in_background` 启动任务(或让前台调用超时转后台),再将返回的 ID 交给宿主 `job_output` / `job_kill`。请用简短的 `description` 说明任务,便于等待时识别。 diff --git "a/docs/\345\216\237\347\224\237\345\206\205\346\240\270.md" "b/docs/\345\216\237\347\224\237\345\206\205\346\240\270.md" index 81a1655d..e714870a 100644 --- "a/docs/\345\216\237\347\224\237\345\206\205\346\240\270.md" +++ "b/docs/\345\216\237\347\224\237\345\206\205\346\240\270.md" @@ -216,7 +216,7 @@ FILE_CHANGED 始终使所请求页不完整。引用页生成的 source 类型 ## PDF 解析基础(15c/16) 内部 PDF 容器解析器仅接受受限的 PDF 1.0–1.7 子集:单个传统 xref 表、 -准确的末尾 startxref/EOF、唯一字典键及对象标识,以及直接整数形式的流长度。 +准确的末尾 startxref/EOF、唯一字典键及对象标识,以及直接整数或经经典 xref 表解析的间接引用形式的流长度。 每个有效 xref 偏移必须准确指向对象头;对象不可重叠,间隙只能包含空白或注释。 增量更新、加密、对象流/xref 流、恢复扫描及重复歧义数据均被拒绝。 `/Encrypt`、`/ObjStm` 与 `/XRef` 无论出现在哪里都会被拒绝,名称转义后也不例外; diff --git "a/docs/\345\270\270\350\247\201\351\227\256\351\242\230.md" "b/docs/\345\270\270\350\247\201\351\227\256\351\242\230.md" index 69fecb7b..0f49a3ca 100644 --- "a/docs/\345\270\270\350\247\201\351\227\256\351\242\230.md" +++ "b/docs/\345\270\270\350\247\201\351\227\256\351\242\230.md" @@ -11,6 +11,10 @@ Windows Low 完整性、MSYS 启动、杀软、heredoc 或审批问题,请先 --- +## 会话切换时 Bash 消失或重复注册 + +会话 Bash 发布按 agent 与代次管理。重叠消费者不再重复注册同名工具;排队注册只有在结算后才更新条目;同一代失败只报告一次,不再同步重试形成风暴。`turn/start` 会重新断言当前权限,因此中断的回合不会让 `ops_bash` 一直缺席到某个无关异步事件出现。若看到名称冲突,保留那一次警告及随后的 turn 边界证据即可,不要反复重启宿主(#579、#842、#873)。 + ## 安装与载荷 ### `node_modules/@dsh-ops/` 下没有 `bash-win32-x64` @@ -220,6 +224,10 @@ argv 第一个元素解析后等于 Desktop 可执行文件——OPS 现在在 这一步用 Windows PowerShell 5.1 运行包内自带的 `bin/prepare-workspace.ps1`,**以当前用户身份、在沙箱约束生效之前执行**;只在 Windows 的工作区内修改模式下运行,且只修改工作区根目录自身的 DACL。只读与 danger-full-access 会话会跳过它。 +已经是 ready 的文件夹完全不检查重解析点,因此 OneDrive 占位符根目录可以直接通过快路径(#744)。重解析点检查只在必须补 ACE 的分支上运行:仅已证实的云文件族——精确的 0x9000?01A 形状(`IO_REPARSE_TAG_CLOUD`、`CLOUD_ROOT`、`CLOUD_ON_DEMAND`)——在占位点后面保留真实的目录项,允许通过;微软保留的 0x90000000-0x9FFFFFFF 段的其余部分未经证实,与挂载点、符号链接、未知或不可读一样闭锁拒绝,并在报错中写明具体 tag。 + +两个微软固定的 WSL 重定向器前缀(`\\wsl.localhost\…` 与 `\\wsl$\…`)会在完成存在性与目录验证之后、任何 NTFS 检查之前直接回答 `ready`,因为其后的 9P 文件系统根本不暴露安全描述符,此前 `Get-Acl` 在那里抛错会让每次 workspace-write 调用都中止(#669)。这是前缀判定而非 catch:不存在的路径照旧失败,其他任何路径上真实的 ACL 错误依然闭锁失败。 + 报错会写明原因: - `PowerShell execution policy blocked dsh-ops's own preparation script …`:dsh-ops 会传 `-ExecutionPolicy Bypass`(旧版本没有传,因此 Windows 客户端默认的 `Restricted` 会让所有 shell 工具失败,#66)。现在仍被拒绝,说明是组策略(MachinePolicy/UserPolicy)在强制执行策略,`Set-ExecutionPolicy -Scope CurrentUser` 无法覆盖。请运行 `powershell.exe -NoProfile -Command Get-ExecutionPolicy -List` 检查、联系管理员,或为该会话使用 danger-full-access。 diff --git "a/docs/\346\210\220\346\234\254\345\210\206\350\264\246.md" "b/docs/\346\210\220\346\234\254\345\210\206\350\264\246.md" new file mode 100644 index 00000000..f55e0234 --- /dev/null +++ "b/docs/\346\210\220\346\234\254\345\210\206\350\264\246.md" @@ -0,0 +1,50 @@ +# OPS-PTC 四本账成本分账(#991) + +## 为什么要有这本账 + +OPS-PTC 计量常见的错误是结构性的:把 `tools:sdk` 在已包含它的 system 总量上再加一遍造成双计;把 Native 的工具 schema 混进 PTC 测量;或者放错 dsh-ops 引导的位置:一次性教学随被接受的工具结果走同一步 `additionalContexts`、从不记为历史,history 账只记录形状提醒,两者都不是系统提示。V1(#990,`test/lib/ptc-entry.mjs`)已经保证被测组真正进入 OPS-PTC;本套件在那条入口上补齐字节分账,四本账永远不互相混同。 + +`test/ptc-cost-accounts.test.mjs` 用 `bootEntryGroup` 启动一个隔离组,先用 `enterPtc` 证明路由,再走一段脚本化场景——首次教学、一个引来 plan 提醒的薄程序、一次合规的 `ops_load_sdk` 加载(`glob`/`grep`/`read_files`)、以及同配置重复请求——每本账都以原始 UTF-8 字节加各项名字清单输出。 + +## 四本账 + +下表读数取自 commit `205cb94`、插件 0.5.5、宿主 0.2.2-rc.1、linux-x64、staged worker sha256 `2b5f988d…`、预设 `ops-ptc`、常驻 + 动态 + 批量教学开启、官方 read/glob/grep 已挂载的一次运行。绝对字节只在同一配置内可比。history 行记录 #929 之后的构成:一次性教学是同一步上下文,不计入该账。 + +| 账目 | 首次请求(仅常驻) | 加载后 | 重复 | +| --- | --- | --- | --- | +| system(投递的渲染提示) | 12,782 B | 37,892 B | 37,892 B | +| ——其中 `tools:sdk` | 11,739 B | 36,849 B | 36,849 B | +| wire(外层工具 schema) | 2,500 B | 2,500 B | 2,500 B | +| history(dsh-ops 提醒) | — | + 提醒:plan 284 B | 不变 | + +- **system** 是投递出去的渲染提示,按 provider 请求实际携带的口径计字节;套件断言每个配置点的请求 system 消息与空闲快照的渲染结果一致。首次请求的逐项清单:`harness:identity` 48、`tools:ptc-only` 159、`tool:read` 130、`tool:write` 92、`tool:edit` 83、`tool:glob` 75、`tool:grep` 133、`dsh-ops:repository-tooling:mode-ptc` 307、`tools:sdk` 11,739。 +- **sdk** 是 system 内唯一的一个 `tools:sdk` 小节——严格子集,绝不是另立账目。归一化规则由断言钉死:system = 逐小节字节按 `\n\n` 连接,即 12,766 + 8 个连接 × 2 B = 12,782,可用上表清单手工复核。 +- **wire** 是请求外层工具 schema 的 `JSON.stringify`——本场景恒为 `run_code` 一项,每个请求字节一致,且等于空闲装配的 wire。 +- **history** 是套件按每次请求实际投递的消息、针对 contributor `batching-guidance` 记录的 dsh-ops 引导:薄程序引来的 plan 提醒(一次,284 B)。一次性教学不出现在这里:它随首个被接受的根 `run_code` 结果走同一步 `additionalContexts`(contributor `batching-guidance:lesson`),从不记为 user 历史,因此该账不含教学字节;完整教学正文为 8,729 B。套件断言两者都不进 system 账——提醒之后的 system sha 与基线相同。 + +## 必须成立的算术 + +加载 `glob`/`grep`/`read_files` 就是那个手算样例:steady 与 loaded 两个快照之间,除 `tools:sdk` 外每一小节字节级不变,system 增量 === SDK 小节增量(+25,110 B)——这个变更在 system 里只出现一次,SDK 子集绝不双计。同一 fixture 同一代码的两个全新组必须逐字节一致(sha 相等);同一会话内同配置重复请求必须字节相同。 + +## SDK 名称与真实官方 schema 的对比 + +场景在行挂载之前先装上宿主自己的官方文件工具插件(与预算套件相同的 `FsTools`/`SearchTools`),因为 SDK 名称只有对着被替代的 schema 才有意义。官方 schema 按 name+description+parameters 的 JSON 计为:`read` 438 B、`glob` 750 B、`grep` 702 B。 + +在本基线上,正式行并**不会**把这些官方绑定从 PTC SDK 里移除——`read`、`glob`、`grep`、`write`、`edit` 仍与 `ops_inspect_local_file`、`ops_load_sdk` 并列声明(PTC 隐藏修复是 open PR #961;#978 的对照读数取自其 head,不是这里)。套件有意钉住这个基线:#961 合入后该断言转红,账目必须重新计量并更新文档。缺少官方提供方时,套件把对比标记为**不可测**并列出缺席名称,绝不静默放行。 + +## 刻意不测的部分 + +- 夹具从不发出真实模型请求:provider 是脚本化的,每条打印记录都带 `provider: "scripted"`、`networkModelRequests: 0`。 +- 不测 token、缓存用量或费用:字节不是 token,bytes/4 不是实测 token 数。token 归 provider 输出(V5 范围)。 +- worker 缺失时整套件可见地 skip——skip 不是 pass。 +- 绝对字节不跨版本、不跨组相减;不承诺固定百分比。 + +## 复现 + +``` +npm run build +npm run build:ptc +DSH_OPS_WORKER_BIN=$PWD/runtime/linux-x64/dsh-ops-worker node test/run.mjs ptc-cost-accounts +``` + +套件每个快照打印一行含四本账与 scripted/0 标注的 JSON;设 `DSH_OPS_PTC_COST_REPORT=` 可得逐项完整证据。`node test/run.mjs ptc-cost` 会把本套件与共享过滤器的声明成本套件一起运行。 diff --git "a/docs/\346\226\207\344\273\266\345\245\221\347\272\246\351\252\214\346\224\266.md" "b/docs/\346\226\207\344\273\266\345\245\221\347\272\246\351\252\214\346\224\266.md" index 7d7be95d..a9d17580 100644 --- "a/docs/\346\226\207\344\273\266\345\245\221\347\272\246\351\252\214\346\224\266.md" +++ "b/docs/\346\226\207\344\273\266\345\245\221\347\272\246\351\252\214\346\224\266.md" @@ -28,9 +28,20 @@ token、时延或成功率收益。Native 输出 fixture 的减少见 [Native - Native standard 准入 OPS read/glob/grep 替代并撤下相应官方名称;保留 `edit` 与 `read_image`。既有替代测试覆盖就绪状态、已捕获步骤、限制、同名注册与卸载。 -- 两个内置 PTC 模式明确配置为**不替代文件工具**。SDK 保留官方 read/glob/grep/edit, - 同时包含已选中的 OPS 方法。审计真实执行两种 glob 的 nested 调用,而顶层请求始终只有 - `run_code`。这是预设策略,不是 #787 的 Native「可见但调用被拒」。 +- 两个内置 PTC 模式在 `hideOfficialTools` 开启(默认)时隐藏投影能服务的官方 + `read/glob/grep`,`write/edit/read_image` 保留。判定来自发布面 `servable` 集合 + (`permitted ∧ (resident ∨ allowedApis)`),不含 `selected`:加载 SDK 不会让官方工具 + 回归。逐名裁决:投影可服务时隐藏该官方名;无法服务时保留该名字的官方工具(未就绪,或 `dynamicTools:false` 下的 `glob/grep`、`allowedTools` 收窄排除的名字)。常驻 `inspect_local_file` 可服务时 `read` 仍被替换。 + 顶层请求仍只有 `run_code`。 +- 该判定的回退侧逐名验证且保持可用:被保留的官方工具在准入、SDK 声明与程序绑定三个 + 视图中都存在且可调用;撤下模式行(或投影未就绪、探测失败、收窄排除)后,官方 + `read/glob/grep` 恢复到同样可调用的状态;自定义 preset 在 `replaceOfficial` + 显式开启(默认 `false`)之前保留全部官方工具。保留的 `write/edit/read_image` + SDK 条目用真实参数 schema 对比,而不是空 schema 的成员断言。 +- 隐藏意味着这些名字的工作流交给 OPS 替代,其逐工具差异已在 + [组件参考](组件参考.md) 成文:`ops_glob`/`ops_grep` 不遍历符号链接、重解析点、 + 隐藏文件与 `.git`,覆盖范围分页返回并显式标注完成/截断字段;`write`/`edit` + 保持官方,正是因为 OPS 的 CAS `expectedRevision` 发布契约与宿主 write/edit 不同。 - 卸载恢复原工具面;未接入 OPS 的原版 PTC 不增加 OPS 提示或 SDK。关闭 batching 不投递 教程或提醒,由生命周期套件验证。运行时版本只在显式诊断中返回。 diff --git "a/docs/\346\226\207\344\273\266\346\211\271\351\207\217\346\223\215\344\275\234.md" "b/docs/\346\226\207\344\273\266\346\211\271\351\207\217\346\223\215\344\275\234.md" index 828bae95..a94aaeb3 100644 --- "a/docs/\346\226\207\344\273\266\346\211\271\351\207\217\346\223\215\344\275\234.md" +++ "b/docs/\346\226\207\344\273\266\346\211\271\351\207\217\346\223\215\344\275\234.md" @@ -43,8 +43,10 @@ Native 输出文件标题、物理行号、逐文件错误与续读提示;程 `ops_edit_file` 对**单个** UTF-8 文件原子应用 1–100 段修改。参数包括 `path`、读取/stat 得到的准确 SHA-256 `expectedRevision` 与 `edits: [{oldText,newText}]`。每个非空 `oldText` 必须在同一原始字节中 -唯一出现;重叠、歧义或版本过期会在发布前拒绝全部修改。空 `newText` 表示删除。CRLF/BOM 需显式 -保留。`dryRun: true` 仅预览相同回执;只读策略仍拒绝写入工具。这不是跨文件事务。 +唯一出现。若原始匹配失败,会尝试一次可逆 Native 渲染解码,同时解码 `oldText` 与 `newText`; +原始匹配不会解码任何字段,回退回执以 `renderedMatches` 计数。重叠、歧义或版本过期会在发布前拒绝全部修改。 +空 `newText` 表示删除。CRLF/BOM 需显式保留。`dryRun: true` 仅预览相同回执;只读策略仍拒绝写入工具。 +这不是跨文件事务。 它共享 `ops_edit_file` 的 no-follow、CAS 与平台元数据保护,返回包含版本、修改计数、字节/行数及 警告的短回执,不回显替换内容。重试前必须检查 `error.effectState`;该工具不会标记为并发安全。 diff --git "a/docs/\346\226\207\344\273\266\350\257\273\345\206\231\345\245\221\347\272\246.md" "b/docs/\346\226\207\344\273\266\350\257\273\345\206\231\345\245\221\347\272\246.md" index d9570ee9..5be50c96 100644 --- "a/docs/\346\226\207\344\273\266\350\257\273\345\206\231\345\245\221\347\272\246.md" +++ "b/docs/\346\226\207\344\273\266\350\257\273\345\206\231\345\245\221\347\272\246.md" @@ -16,7 +16,9 @@ English: [File read and mutation contracts](file-contracts.md). `stat`、`copy`、`move` 与 `remove` 以流式 SHA-256 计算 revision,不再整读文件,因此可处理超过 32 MiB 的文件。hex 检查会在不缓冲完整内容的情况下流式计算整文件事实,随后只定位并缓冲请求窗口。需要完整内容的文本检查、`diff` 与写入缓冲仍保留各自明确的内容预算。 -在不支持命名流或 ACL 的 Windows 卷上,写入会保留可用的基本元数据并返回 `metadataWarnings`,不再返回 `METADATA_UNSUPPORTED`。Windows 上真实存在但不受支持的源元数据(例如 NTFS 备用数据流)仍会闭锁拒绝。 +在不支持命名流或 ACL 的 Windows 卷上,写入会保留可用的基本元数据并返回 `metadataWarnings`,不再返回 `METADATA_UNSUPPORTED`。在 NTFS 上,用户态可恢复的扩展属性会被捕获、恢复并逐字节核验;`$KERNEL.*` 名称因用户态无法重建而逐名写入 `metadataWarnings`。其他任何 EA 捕获、恢复或核验失败都会闭锁拒绝;不受支持的源元数据(例如 NTFS 备用数据流)同样闭锁拒绝。 + +穿过 Electron `.asar` 文件的路径会返回 `ARCHIVE_MEMBER_UNSUPPORTED`。只有 Electron 进程内经补丁的文件系统会把归档呈现为目录,独立 worker 看到的仍是普通文件,无法读取归档成员。请用 hex 视图检查归档本身,或改用解包后的副本。 `replace` 不绑定之前 `inspect_local_file` 的读取结果。单文件变更必须依赖该次读取时使用 `edit_file`。replace 预览不会预留文件,执行时重新扫描。`changed` 是已应用字面替换次数,不是文件数。预览带 `dryRun: true`,仅有计划 hash 不能证明已经写入。目录替换遇到读取权限拒绝时会记录 `skippedPaths`,返回 `complete: false` 与 `truncated: true`,并继续处理其他合格文件;普通的二进制文件跳过本身不会使结果不完整。更新锁、暂存与发布阶段的权限拒绝仍使调用失败,共享冲突及其他 IO 故障也保留结构化错误,不会改记为跳过。 diff --git "a/docs/\347\264\247\345\207\221\346\226\207\344\273\266\350\214\203\345\233\264.md" "b/docs/\347\264\247\345\207\221\346\226\207\344\273\266\350\214\203\345\233\264.md" index b263d2ab..b658150e 100644 --- "a/docs/\347\264\247\345\207\221\346\226\207\344\273\266\350\214\203\345\233\264.md" +++ "b/docs/\347\264\247\345\207\221\346\226\207\344\273\266\350\214\203\345\233\264.md" @@ -12,7 +12,7 @@ English: [Compact Native file ranges](native-ranges.md). [revision=cfd1bc81432fd6b88748b827f46bb4242db17cf8655cf4ea90f3ba0e62065355; more lines; nextOffset=3] ``` -`N|` 后紧跟原始行内容,包含原有缩进。正文使用可逆 JSON 转义:反斜杠写成双反斜杠,控制字符及 Unicode 换行分隔符被转义,引号保持原样。例如真实制表符显示为 `\t`,而反斜杠加字母 t 的两个源字符显示为 `\\t`。紧凑范围和普通行号正文共用此编码,防止伪造额外行号或文件标题;LF/CRLF 物理行规则与 worker 保持一致。 +`N|` 后紧跟原始行内容,包含原有缩进。正文使用可逆 JSON 转义:反斜杠写成双反斜杠,控制字符及 Unicode 换行分隔符被转义,引号保持原样。例如真实制表符显示为 `\t`,而反斜杠加字母 t 的两个源字符显示为 `\\t`。紧凑范围和普通行号正文共用此编码,防止伪造额外行号或文件标题;LF/CRLF 物理行规则与 worker 保持一致。`ops_edit_file` 始终先匹配字面字节;只有原始匹配失败后才可能解码此渲染形式的 `oldText` 与 `newText`,并在回执中返回 `renderedMatches`。原始匹配与 `write_file` 内容不会被解码。 `more lines` 表示文件尚未读到末尾,按 `nextOffset` 继续;`end of file; complete=true` 表示本范围已经到达文件末尾。汇总的 `requested ranges returned` 仅证明所请求的范围正文全部返回,不代表读取了所有完整文件。完整版本散列、总行数和每个文件的续读位置继续保留;字节大小及重复计数仍在规范数据和 PTC 返回值中。逐文件版本回执省去重复的 `Notice: ` 前缀与末尾句点;汇总 `Notice` 以及版本、续读和完整性事实均保留。 diff --git "a/docs/\347\273\204\344\273\266\345\217\202\350\200\203.md" "b/docs/\347\273\204\344\273\266\345\217\202\350\200\203.md" index 824ef039..e987f2a9 100644 --- "a/docs/\347\273\204\344\273\266\345\217\202\350\200\203.md" +++ "b/docs/\347\273\204\344\273\266\345\217\202\350\200\203.md" @@ -16,7 +16,7 @@ `dsh-ops` 行拥有内置和显式配置的消费者方案。Native 发现与 PTC SDK loader 是独立的发布选择;文件替换与宿主 Shell 隐藏也是独立屏蔽。旧入口仅为让旧 profile 可加载,新配置都属于单行。原 `dsh-ops/shell`、`dsh-ops/file`、`dsh-ops/ptc`、`dsh-ops/hide-official` 模块仅作为旧 profile 的 legacy 导出保留,不属于 bundle。 -`ops-ptc` 是单行的第五个内置模式:预设由 `dsh-ops` 在运行时通过宿主 `agentPresets.register` 注册(定义来自随包片段 `presets/ops-ptc.patch.yml`,由宿主 `loadOverlayPatches` 解析),插件管理器里没有对应预设行;插件行禁用/卸载时预设与工具面一起消失。默认预设不会被自动改写:详情页按钮点一次把它设为默认(`applied`/`no-update`/`unavailable`/`failed`),apply、重挂载与卸载都不碰操作者自己的选择,见[预设与默认模式](预设与默认模式.md)。它发布与 `ptc` 相同的工具面,并新增 YAML-only 的 `batchingGuidance`:会话首轮注入一次英文教学(`prompts/batching-guidance.md`,进入历史、不改系统提示词);之后**根** `run_code` 程序命中 `bypass`、`plan`、`shape` 三类锁之一时,在该次调用自身结果上以 `additionalContexts` 追加一条短提醒——每类每轮各锁一次、合规即解锁,优先级 `bypass` → `plan` → `shape`,不阻断也不改写程序。旧 profile 写 `dsh-ops/mode-ops-ptc` 行时仍加载 legacy 模块;merged bundle 下它只告警并保持惰性,配置须迁入单行 `ops-ptc` 分区。 +`ops-ptc` 是单行的第五个内置模式:预设由 `dsh-ops` 在运行时通过宿主 `agentPresets.register` 注册(定义来自随包片段 `presets/ops-ptc.patch.yml`,由宿主 `loadOverlayPatches` 解析),插件管理器里没有对应预设行;插件行禁用/卸载时预设与工具面一起消失。默认预设不会被自动改写:详情页按钮点一次把它设为默认(`applied`/`no-update`/`unavailable`/`failed`),apply、重挂载与卸载都不碰操作者自己的选择,见[预设与默认模式](预设与默认模式.md)。它发布与 `ptc` 相同的工具面,并新增 YAML-only 的 `batchingGuidance`:随首个被接受的根 `run_code` 结果以 `additionalContexts` 教学一次(`prompts/batching-guidance.md`,同一步 ephemeral、从不进入会话历史、不改系统提示词);之后**根** `run_code` 程序命中 `bypass`、`plan`、`shape` 三类锁之一时,在该次调用自身结果上以 `additionalContexts` 追加一条短提醒——每类每轮各锁一次、合规即解锁,优先级 `bypass` → `plan` → `shape`,不阻断也不改写程序。旧 profile 写 `dsh-ops/mode-ops-ptc` 行时仍加载 legacy 模块;merged bundle 下它只告警并保持惰性,配置须迁入单行 `ops-ptc` 分区。 原 `dsh-ops/background` 组件(FastCtx 后台任务:`ops_run`、`ops_run_background`、`ops_job_output`、`ops_job_list`、`ops_job_kill`)已移除。仍写着它的 profile 行会作为空操作加载:接受任意配置、不发布任何工具,只记录一条警告提示可删除该行。 @@ -59,6 +59,8 @@ `replaceTools` 仅选择已评审替换;Rust/ripgrep 搜索与宿主工作流存在明确差别。写入使用 `expectedRevision` 做 CAS;配置通过 `StandardSchemaV1` 校验。 +已评审替换表仅覆盖 `read`、`glob`、`grep`;`write`、`edit`、`read_image` 从不在替换表中。只有显式 opt-in 才隐藏被替代的官方工具:内置模式用 `hideOfficialTools`,显式消费者用 `replaceOfficial`(默认 `false`);未开启时官方工具保持发布。同一开关覆盖两种呈现:PTC 会话中,仅当投影能服务对应 `ops_*` API 且已注册 counterpart 时才隐藏该官方名,投影未就绪、探测失败或被 `allowedTools` 收窄排除时保留该官方工具。`run_code` 传输本身从不遮蔽、也不门控。 + 拒绝按最近一次组装请求所携带的工具面判定,而不是以注册表当前状态为准:若某次请求在替换对其可见之前就已组装完成,该请求仍携带的官方工具名不会被拒,因此会话不会被告知去调用一个它从未拿到过的工具(#578)。 Standard 与 Cordis 发布常驻 OPS 文件工具面:`ops_inspect_local_file`、`ops_glob`、 diff --git "a/docs/\350\256\276\350\256\241\345\206\263\347\255\226-20261115.md" "b/docs/\350\256\276\350\256\241\345\206\263\347\255\226-20261115.md" new file mode 100644 index 00000000..f71ce2f7 --- /dev/null +++ "b/docs/\350\256\276\350\256\241\345\206\263\347\255\226-20261115.md" @@ -0,0 +1,61 @@ +# 设计决策:PTC 预设下隐藏官方文件工具 + +日期:2026-11-15 +状态:已接受 +范围:`src/publication/ptc.ts`、`src/publication/replacements.ts`、`src/consumer/mount.ts`、`src/presets/builtin.ts`、`src/integration/index.ts` + +## 背景 + +`hideOfficialTools` 的文档语义是“隐藏本模式替代的官方文件工具”。实际上它对所有 PTC 模板预设(`ptc`、`ops-ptc`、自定义 PTC preset)都不生效:`MODE_PLANS` 为它们声明 `nativeFiles: false`,而 `replacementTargets` 用 `config.hideOfficialTools && plan.nativeFiles` 作门,因此目标表中永远只有 Windows shell 名字。`mountReplacements` 确实已挂载(Windows 下 targets 非空),但从不处理 `read`/`glob`/`grep`。 + +PTC 投影保留官方 SDK 绑定(`tools.grep`、`tools.read` 等),而 `ops_*` 按需加载,因此开关开着时模型仍可能选用更弱的官方 API。 + +这被定性为缺陷而非策略:`docs/文件契约验收.md` 中的验收记录描述的是当时的配置,而开关对用户的承诺是覆盖它所替代的文件工具。 + +## 决策 + +1. 修复现有 `hideOfficialTools`;不新增开关;默认值不变。 +2. 按运行 PTC 呈现判定,不按预设 id:`ptc`、`ops-ptc`、自定义 PTC preset。`minimal` 不卷入。 +3. 隐藏集合 = `RAW_OFFICIAL_REPLACEMENTS` 的键(当前 `read`、`glob`、`grep`),随表扩展。 +4. `write`、`edit`、`read_image` 保持可见。 +5. 载体:扩展 `mountReplacements` 的 shadow `when`(方案 B),不使用 `ctx.tools.restrict`。 +6. `replacementTargets` 的门改为 `plan.nativeFiles || plan.ptc`;integration 路径覆盖两种呈现。 +7. 可达性信号由 PTC 发布面给出:`permitted ∧ (resident ∨ allowedApis)`,再加一道存在性校验(发布器确实注册了 counterpart:`tools.get` 非空且 `publicationAdmission !== undefined`)。排除 `selected`,`publicationAdmission` 不求值,因此加载 SDK 不会改变判定。 +8. `mountPtc` 返回 `PtcServability` accessor;`apply` 保持 `void`;未 attach 的 agent 返回 `undefined`(fail-open)。 +9. 每个 agent、官方工具名与状态各记录一次;发布面探测失败时 catch 并 fail-open(保留官方),记录一次 warn。宿主行为已实测:`when` reject 会让整个 step 解析失败(`cannot resolve availability of tool`),因此 catch 是必须的。 +10. 内置模式与 integration 路径都修改。 +11. 测试:完整矩阵(翻转现有断言,新增 mixed-tree 与配置矩阵)。 + +## 证据 + +real-host 探针(真实 `mode-ptc` 组件 + NodePtc + 根层注册官方 `read/glob/grep` + `mountPtc` 的 shared-file 发布)结果: + +| 阶段 | admitted | `tools:sdk` 含 `grep:` | SDK 字节 | +|---|---|---|---| +| before | `read, glob, grep, ops_inspect_local_file, ops_load_sdk` | 是 | 8749 | +| agent 层同名 shadow + `when:false` | `read, glob, ops_inspect_local_file, ops_load_sdk` | 否 | 8676 | +| 恢复(dispose) | 完整 | 是 | 8749 | + +同名 shadow 会同时从 `definitions` 与 SDK 段移除官方工具,且可逆。 + +## 实现 + +- `mountPtc` 返回 `PtcServability { servable(agent), isPtc(activation) }`;未 attach 时 `servable` 为 `undefined`,否则为满足 `permits(agent, api) && (resident.has(api) || allowedApis(agent).includes(api))` 的原始 API 集合。 +- `ReplacementPublication` 新增可选 `ptc` 字段。在 `when` 中,于 `keepOfficial`/gate 检查之后、宿主 counterpart 检查之前,插入 PTC 分支:原始 API 在可服务集合中且当前 activation 为 PTC 时隐藏官方名字。 +- PTC 分支只作用于 `RAW_OFFICIAL_REPLACEMENTS` 的键(绝不处理 `pwsh`)。 +- Native 消费者不传 `ptc`,旧路径逐字节保持不变。 +- `applyResolvedMode` 与 integration 路径的 `replacementTargets` 门都改为 `nativeFiles || ptc`。 + +## 后果 + +- 行为变更:升级后,`hideOfficialTools: true`(默认)的 PTC 会话不再看到官方 `read/glob/grep`;模型需通过 `ops_load_sdk` 加载 `ops_*`。已写入 CHANGELOG;可用现有开关关闭。 +- `write`、`edit`、`read_image` 不受影响。 +- 未 attach 或被收窄的 PTC 投影保留官方工具(fail-open)。 + +## 后果:首个 resolve 的时序 + +发布面异步 attach,在此之前 `servable` 返回 `undefined`,因此 PTC 会话的第一个 resolve 可能仍显示官方工具,下一次 resolve 才隐藏。这属于 fail-open 契约而非回归;`notePtc` 会记录 `unattached`,窗口可观测。 + +## 未决风险 + +已由探针解决:`when` reject 会让整个 step 解析失败(`cannot resolve availability of tool "read"`),既不是 fail-open 也不是静默隐藏。因此 `replacements.ts` catch 探测失败并保留官方工具。已知并接受的风险:可达性来自发布面池(`permitted ∧ (resident ∨ allowedApis)`),而非 `publicationAdmission`;此后若发生宿主级发布拒绝,属于 PTC 发布器故障,由 PTC 生命周期套件覆盖,不由隐藏决策负责。 diff --git "a/docs/\350\256\276\350\256\241\345\206\263\347\255\226-20261119.md" "b/docs/\350\256\276\350\256\241\345\206\263\347\255\226-20261119.md" new file mode 100644 index 00000000..d5f3893d --- /dev/null +++ "b/docs/\350\256\276\350\256\241\345\206\263\347\255\226-20261119.md" @@ -0,0 +1,272 @@ +# OPS-PTC 旧配置行为:五输入真值表 + +[English](design-decisions-20261119.md) + +决策日期:2026-11-19。基准:`main` 于 58f6e28。任务卡: +[#985](https://github.com/T-Auto/dsh-ops/issues/985)(D1),所属工作流 +[#979](https://github.com/T-Auto/dsh-ops/issues/979),总计划 +[#978](https://github.com/T-Auto/dsh-ops/issues/978)。 + +本文档只锁定**当前**发布行为,不实现任何新东西。每一行都由 +`node test/run.mjs ptc-config-truth`(`test/ptc-config-truth.test.mjs`)逐字断言。 +字节计量刻意不在本卡范围内——V 系列切片负责测量。 + +## 一个输入如何到达工具集合 + +| 阶段 | 位置 | 作用 | +| --- | --- | --- | +| 行/全局键 | `src/settings/merged.ts` `resolveMergedConfig` | 校验行配置;`dynamicTools` 与 `residentTools` 默认 true(`global` 没有常驻键) | +| 与门 | `effectiveMode`(`src/settings/merged.ts`)、`dynamicForMode` | 行 `dynamicTools` 与 `global.dynamicTools` 求与;总开关「保留操作者意图,只是不再生效」 | +| 消费者计划 | `src/presets/builtin.ts` `applyResolvedMode` | `gate.resident = config.residentTools`,`gate.dynamic = config.dynamicTools`(过门后的值),`plan.ptc = { allowedTools, alwaysLoaded }` | +| 发布 | `src/publication/ptc.ts` `mountPtc` | `resident` = `PTC_RESIDENT_TOOLS` + `alwaysLoaded`,`gate.resident === false` 时为空;`selectable` = `allowedTools` + `result`,`gate.dynamic === false` 时为空;工具在常驻**或**已被选中时可调;`ops_load_sdk` 仅在 `gate.dynamic !== false` 时发布 | + +## 真值表 + +以 `ops-ptc` 行为准(`ptc` 行首轮集合相同,仅 `batchingGuidance` 缺省不同)。测量环境: +Linux、`danger-full-access`、native worker 就位、真实 PTC 呈现、宿主官方 +`read`/`glob`/`grep` 保持可见;`ops_bash` 缺席,因为 POSIX 行沿用宿主 shell。 + +| # | 输入(`dsh-ops` 行配置的一行) | 本步可调 | 可加载未加载 | loader | run_code 首轮视角 | +| --- | --- | --- | --- | --- | --- | +| 1 | 缺省(无任何键) | `ops_inspect_local_file`、`ops_load_sdk` | 下方十六个 API 的默认池 | 有 | `tools.ops_inspect_local_file` 与 `tools.ops_load_sdk` 可用;`ops_load_sdk({tools:['grep']})` 回执 completed,但 `ops_grep` 要到**下一个**程序才可调 | +| 2 | `global: {dynamicTools: false}` | `ops_inspect_local_file` | 无 | 无 | `tools.ops_inspect_local_file` 可用;没有 `tools.ops_load_sdk` | +| 3 | `ops-ptc: {dynamicTools: false}` | `ops_inspect_local_file` | 无 | 无 | 与第 2 行同一集合(同一个与门的两侧),差别只在作用域——见注 | +| 4 | 第 3 行再加 `ops-ptc: {alwaysLoaded: [stat]}` | `ops_inspect_local_file`、`ops_stat` | 无 | 无 | 两个绑定首轮即可调用 | +| 5 | `ops-ptc: {residentTools: false}` | `ops_load_sdk` | 下方十六个 API 的默认池 | 有 | loader 是唯一绑定;`ops_inspect_local_file` 整体不可达——`ops_load_sdk({tools:['inspect_local_file']})` 回 `INVALID_ARGUMENT` | +| T | **已实现——见 D3 附录**:显式选择全量且 `allowedTools: [grep, stat]` | `ops_grep`、`ops_stat`(依 #979 §2) | — | 无(依 #979 §2) | 获准 API 首轮全部可调;只读会话仍排除写 API | + +默认可加载池(第 1、5 行)恰为: +`ops_grep`、`ops_glob`、`ops_replace`、`ops_write_file`、`ops_edit_file`、`ops_diff`、 +`ops_stat`、`ops_copy`、`ops_move`、`ops_remove`、`ops_read_files`、`ops_read_table`、 +`ops_read_docx`、`ops_read_pdf`、`ops_batch`、`ops_result`——十六个非常驻原始 API +(`PTC_LOADABLE_TOOLS`)。目标行 T 的池为 `ops_grep`、`ops_stat` **外加 +`ops_result`**(见签字决定项 1)。 + +**逐行注释。** + +- 第 2、3 行发布同一个 OPS 集合,因为它们喂进同一个与门。但输入本身不同:总开关 + 同时关掉 `standard`/`cordis`/`minimal` 的 Native 发现入口(`dynamicForMode`), + 行级开关只影响本行。 +- 第 4 行说明 `dynamicTools:false` **不**压制 `alwaysLoaded`:启动名单随常驻集合发布, + 只是没有 loader 去选择上限中的其余工具。 +- 第 5 行说明 `residentTools:false` 丢掉整个常驻集合,包括 `ops_inspect_local_file`, + 而且它也加载不回来(不在 `PTC_LOADABLE_TOOLS` 里)。 +- **旧行关闭与全量选择是两个不同的输入**(#985 反例):`allowedTools: [grep, stat]` 加 + `dynamicTools: false` 时,首轮只有 `ops_inspect_local_file`——没有 `ops_grep`、没有 + `ops_stat`、没有 loader。把 `dynamicTools:false` 解读成「发布全部获准工具」,就会让 + `ops_grep`、`ops_stat` 出现在这一轮,改变旧用户的成本与权限预期;#907 曾隐式做过 + 这个翻转,#910 随即撤回(#979 §4)。 + +**只读与可写实例。** 默认池中的只读成员:`ops_grep`、`ops_stat`、`ops_glob`、 +`ops_diff`、`ops_read_files`、`ops_read_table`、`ops_read_docx`、`ops_read_pdf`。 +可写成员:`ops_replace`、`ops_write_file`、`ops_edit_file`、`ops_copy`、`ops_move`、 +`ops_remove`、`ops_batch`。只读会话下,`permitted`(`src/publication/ptc.ts`)会把所有 +会变更的 API 从可加载池中剔除,加载请求回 `INVALID_ARGUMENT`;`ops_result` 与 +`ops_load_sdk` 豁免于该变更检查。本卡没有测试切换沙箱模式;只读由 D3 覆盖。 + +一段可以逐字照抄的程序: + +```js +// plan: 第 1 行——现在加载 grep,下一个程序再使用 +const picked = await tools.ops_load_sdk({ tools: ['grep'] }) +return { picked } // picked.status === 'completed';ops_grep 在本程序内仍不可调 +``` + +```js +// plan: 第 4 行——stat 在此行常驻,首轮即可调用 +const r = await tools.ops_stat({ path: 'a' }) +return r.status +``` + +## 维护者签字决定项——如实记录,刻意不实现 + +1. **`ops_result` 的隐式例外。** 今天 `loadable = [...allowedTools, 'result']` 无条件成立, + 且 `permitted` 让 `result` 豁免只读变更检查,因此即使上限没写它,`ops_result` 也始终 + 可加载(现状由 `test/resident-switch.test.mjs` 的 "result exception remains loadable" + 覆盖)。而 `dynamicTools:false`(第 2–4 行)下 `ops_result` **根本不发布**。全量档需要 + 决定:首轮是否发布 `ops_result`,让落盘的 ref 仍然可消费(#979 §3)?当前行为已锁定, + 本文不做任何改动。(已由 D3 附录解决:全量档首轮发布 `ops_result`。) +2. **总开关对显式全量选择的作用。** 今天 `global.dynamicTools` 与门控住每一行的 loader, + PTC 行也不例外。#979 §4 要求升级、保存设置和 `global.dynamicTools` 开关都不得替用户 + 选择全量。需要决定:`global.dynamicTools:false` 是否也压制显式选择全量的行,还是显式 + opt-in 优先于总开关? +3. **显式常驻与上限的冲突。** 今天 `alwaysLoaded` 只对照可加载池校验 + (`src/ptc/config.ts`),从不对照 `allowedTools`:`{ allowedTools: [grep], + alwaysLoaded: [stat] }` 会把 `stat` 当常驻发布且没有任何诊断;`residentTools:false` + 则静默丢弃整个常驻集合(含 `alwaysLoaded`)。需要决定:D2 给哪些冲突加可读诊断、 + `alwaysLoaded` 是否必须落在 `allowedTools` 之内。 + 补录对比事实(同日追加,只记录不实现):**自定义 integration 路径**更严—— + `src/consumer/options.ts:100` 对 `alwaysLoaded` 不在 `allowedTools` 内的自定义 PTC + 配置直接抛 `alwaysLoaded must be in allowedTools`,而内置行对同一形状静默接受 + (已由套件「escapes the allowedTools cap」检查锁定)。该诊断不一致已由 D4 卡 + (#988)认领修复;D1 只记录现状。 +4. **合法自定义 PTC integration 被误诊 `ops_result` 未发布。** + `src/integration/diagnostics.ts` 把 `'result'` 列入每个 PTC integration 作用域的 + 首轮期望可见名,但 `result` 是可选加载而非常驻(`PTC_RESIDENT_TOOLS` 仅 + `inspect_local_file` 一个),只有 loader 选中后才可见。于是完全合法的自定义 + `profile: 'ptc'` integration 会被诊断为 `degraded`,报 + `PUBLICATION_UNAVAILABLE: ops_result`,而它唯一的「缺口」是一个尚未被加载的 + API——已实测启动这样一个 integration 证实,并由套件的 + 「misdiagnosed PUBLICATION_UNAVAILABLE」检查锁定。内置模式行根本不挂诊断 + (`mountDiagnostics` 只在 `mountIntegration` 中运行),因此该误报只可能出现在 + 自定义 integration 路径上。修复已由 D4 卡(#988)认领;D1 只记录现状。 + +## 设置页文案——如实照录 + +设置页与行开关把 dynamic-tools 开关描述为「发现入口」。已发布文案照录自 +`src/client/locales.ts`: + +| 键 | 英文 | 中文 | +| --- | --- | --- | +| `dynamicToolsHint` | "When off, no preset offers the discovery entry." | "关闭后没有任何预设提供发现入口。" | +| `swDynamicToolsHint` | "Offers the discovery entry to this preset. Applies only while the bundle switch is on." | "为该预设提供发现入口。仅在总开关开启时生效。" | + +PTC 行实际为该入口发布的是类型化 loader `ops_load_sdk`(真值表第 1、5 行), +并非发现式工具:措辞与产物存在差异。统一措辞属 #979 的 D5 切片;本卡只记录差异。 + +## Loader 分批——如实记录现状 + +一次 `ops_load_sdk` 调用接受 1–5 个精确原始 API 名 +(`src/projections/ptc/loader.ts` 的 `requestedApis`—— +`tools must contain 1-5 exact raw API names`),因此装满十六名默认池至少需要四次 +调用。`alwaysLoaded` 与其余常驻集合没有数量上限:`parsePtcConfig` +(`src/ptc/config.ts`)只逐名对照可加载池校验,从不校验名单长度,且常驻 API +无需任何 loader 调用即发布。由套件「loader batches 1-5 names」检查锁定。 + +## 范围 + +本文不引入任何发布默认值、诊断或第三种用户模式。依 #985,签字决定项有分歧先在卡上 +请维护者签字,D2/D3 才动手。纯旧配置在本切片前后的工具集合保持上表不变——套件断言 +的正是这一点。 + +## 验证 + +- `npm run build` 后 `node test/run.mjs ptc-config-truth`——基准上 11 通过 / 0 跳过 / + 0 失败(首次运行即与代码阅读预测逐行一致)。 +- 断言绑定行为而非互为同义反复的反向红证:只改套件中第 5 行的期望常量,恰好该行 + 检查失败("1 of 11 check(s) failed");恢复常量后回到全绿。 +- `node test/run.mjs resident-switch`——未改动的邻居套件,见 PR。 +- 补录审计(同日追加):新增两项检查——loader 单次 1–5 上限与常驻启动名单无数量 + 上限、合法自定义 PTC integration 被误诊 `PUBLICATION_UNAVAILABLE: ops_result`。 + 套件在基准上现为 13 通过 / 0 跳过 / 0 失败。 + +## D2 附录——显式披露策略键已解析(2026-11-19) + +任务卡 [#986](https://github.com/T-Auto/dsh-ops/issues/986),叠在本真值表分支之上。本附录 +记录 D2 锁定的内容,并按保守取向解决签字决定项 2 的优先级问题;全量档的发布仍归 D3。 + +**键。** `ptc` 与 `ops-ptc` 行接受 YAML-only 的 +`disclosurePolicy: 'on-demand' | 'full'`(缺省 `'on-demand'` = 本表的旧路径,逐字一致)。 +非法值报错点名键与收到的值或类型;非 PTC 行(`minimal`、`standard`、`cordis`)以 +`disclosurePolicy applies to the ptc and ops-ptc rows only` 拒绝该键。内置消费者计划切片 +与自定义 integration 在 D3/D4 学会该档位之前保持旧缺省。 + +**签字决定项 2,按 #979 §4 取保守解。** 显式全量选择是**独立的 opt-in**,绝不压过总 +开关:`global.dynamicTools` 关闭期间全量档不生效——该行保持本表第 2 行的旧 +resident/alwaysLoaded 面——而已保存的选择原样留在行配置里,开关回开后重新生效。这使 +§4 的每条保证都成立:升级、保存设置与开关翻转从不写入或抹掉该键,操作者的关闭对 +一切披露形态保持绝对。行级 `dynamicTools` 保持只管 loader 的含义:全量档不发布 +loader(#979 §2),因此同时关掉 loader 的行保留自己的显式档位。该门编码在 +`effectiveMode`;保存的意图与现有 dynamicTools 与门一样原样保留。 +此为保守解,维护者可改。 + +**序列化。** `hostConfig`(保存路径的序列化器)逐字保留显式选择,且从不自动写键: +读取 → 保存一项无关设置 → 再读取,`'full'` 不丢;包括本表五行在内的每个旧输入在 +raw 与序列化两层都保持该键缺席。 + +**仍未实现。** 真值表 T 行仍是目标:尚无任何路径发布全量档(D3 负责 +`src/publication/ptc.ts` 与内置计划切片),没有设置页入口(D5),签字决定项 1、3、4 +维持上文原状。 + +**验证。** `npm run build` 后 `node test/run.mjs ptc-disclosure-policy`——8 项检查, +先红(未实现基线上 8/8 失败,键被拒绝为 unknown),实现后全绿。 +`node test/run.mjs ptc-config-truth` 13/0/0、`node test/run.mjs resident-switch` +24/0/0、`node test/run.mjs settings-rows` 14/0/0 在新解析路径上不变,worker 用 +`DSH_OPS_WORKER_BIN=runtime/linux-x64/dsh-ops-worker`(sha256 2b5f988d)。 + + +## D3 附录——显式全量档已发布(2026-11-19) + +任务卡 [#987](https://github.com/T-Auto/dsh-ops/issues/987),叠在 D2 附录的分支之上。本切片 +把已解析的键变成发布行为;上文五行旧路径不被改写,仍由套件逐字断言。 + +**T 行,已实现。** `{ disclosurePolicy: 'full', allowedTools: [grep, stat] }` 的首轮发布 +`ops_grep`、`ops_stat`、`ops_inspect_local_file`(常驻)与 `ops_result`,且没有 +`ops_load_sdk`(#979 §2)。第一个程序即可直接调用获准池(双重验证:稳定声明集合 + +经 worker 的真实 `run_code` 绑定调用);池外业务 API 仍不可用。内置计划切片现在携带 +解析出的档位(`src/presets/builtin.ts`),发布在 `src/publication/ptc.ts` 消费它。 + +**签字决定项 1,按 #979 §3 取保守解。** 全量档首轮发布 `ops_result`:本步落盘的 ref +必须仍然可消费,且统一验收口径逼出同一结论——按需档永远可以加载 `result` +(`loadable = [...allowedTools, 'result']`),因此只有全量档也发布它,加载完毕的业务 +集合才等于全量首轮集合。该对比由一项检查钉住。 + +**边界,随实现一并断言。** 空上限使档位停在 常驻 + `result`(仍无 loader)。只读仍把 +一切变更 API 从首轮剔除——切换档位绝不放宽权限。`residentTools:false` 丢掉常驻面 +(第 5 行)但不丢显式档位。重复名收敛为一次注册。总开关保持 D2 的否决,端到端成立: +`global.dynamicTools` 关闭期间显式全量行逐字发布第 2 行;行级 `dynamicTools` 保持 +只管 loader 的含义,因此同时关掉 loader 的全量行保留自己的档位。 + +**本切片刻意不做。** 自定义 integration 在 D4 之前保持旧缺省, +`PtcConfig.disclosurePolicy` 也因此保持可选(改为必填会迫使本卡触碰不属于它的 +自定义 integration 计划切片;内置计划携带该值,缺省值即 on-demand)。设置页入口归 D5。 + +**验证。** `npm run build`,先红:`node test/run.mjs ptc-declaration-surface` +5 项中 4 项失败(全部新 T 行断言;旧 loader-less 检查保持绿), +`node test/run.mjs ptc-full-disclosure` 在套件 120 秒截止前 T 行、对比、只读与空池 +均失败。实现后全绿:ptc-declaration-surface 5/0/0、ptc-full-disclosure 8/0/0, +回归锁不变——ptc-config-truth 13/0/0、resident-switch 24/0/0、 +ptc-disclosure-policy 8/0/0、settings-rows 14/0/0;worker 用 +`DSH_OPS_WORKER_BIN=runtime/linux-x64/dsh-ops-worker`(sha256 2b5f988d)。环境备注, +非回归:`ptc.test.mjs` 的安装包解析检查断言优先于 `DSH_OPS_WORKER_BIN`,因此该项 +在不设该环境变量时运行(在基准 commit 上设同样的变量,失败方式相同)。 + + +## P5 附录——四行准入判定,在注册、声明与分发三处对齐(2026-11-19) + +**问题(#997;原始复现在 #968)。** 注册表里的定义不等于本步可调,但所有拒绝读起来 +一模一样:PTC 分发层一律 `POLICY_DENIED`("must be admitted"),发布源一律 +`PUBLICATION_GRANT_UNAVAILABLE`,最糟的是 `ops_load_sdk` 对只读所禁的池内名 +答 `INVALID_ARGUMENT: API is not in this scope's allowed pool`。模型分不清未加载 +与被禁,已撤销的准入与从未存在的名字读起来也相同。 + +**四行判定,一个分类器。** `src/publication/ptc.ts` 对被拒分发恰好答四者之一, +与注册准入和 SDK 声明对齐: + +| 判定 | 条件 | 应答 | +| --- | --- | --- | +| 本步可用 | 常驻或已加载,且获准 | 正常分发;不进拒绝路径 | +| 需加载 | 在池内、获准、未加载——且 loader 存在 | `SDK_NOT_LOADED`,点名 `ops_load_sdk` 与原始名,下一步生效 | +| 权限禁止 | 沙箱策略不允许该 API | `POLICY_DENIED`,明确权限措辞,不给加载建议 | +| 已撤销 | 曾存在的准入被撤——作用域被撤、呈现被收回、或已可用 API 权限被降级 | `POLICY_DENIED`,撤销措辞,与其余三者都不同 | + +过期或外来绑定保持原有的普通准入拒绝,四行之外没有新承诺。只有 loader 真的能选时 +才给加载建议:无 loader 作用域与全量档保持诚实的「无 loader 可选」拒绝。 +`ops_load_sdk` 本身改为按配置池校验,并把权限拒绝从池拒绝中拆出——只读下的 +`write_file` 是「权限不允许」而非「不在池内」;池外名保持 `INVALID_ARGUMENT`。 +`SDK_NOT_LOADED` 拼写刻意沿用 OPEN PR #1007,让最终合并读起来一致;该 PR 的 +deferred 指针管线仍归其所有。 + +**源层保持同一区分。** 被拉走的 registrar 授权答新码 `PUBLICATION_GRANT_REVOKED`, +不再读作「从未可用」;从未持有可用授权的 agent 保持 +`PUBLICATION_GRANT_UNAVAILABLE`。授权与池语义未动——变的只有拒绝判定。 + +**诊断装配不是首个请求。** 无 step tool set 的装配现在只读注册面,不清 pending、 +不取消工作;只有真实步骤(带 tool set)才触发呈现收回的清理。同一 agent 的两条 +路径分别钉住:注册名(诊断)与声明名(请求)与程序绑定——未加载名永不持有绑定, +程序里调用它是 JS 错误,不是工具错误。 + +**范围。** `src/publication/ptc.ts`、`src/publication/registrar.ts`、 +`test/ptc-declaration-surface.test.mjs`(5 → 11 检查),另加 +`test/ptc-directory.test.mjs`(2 处断言)——那两行钉住的正是本卡要移除的 +「权限拒绝误报为池拒绝」旧行为。教学文案(P1/P2)与池语义(D3)未动。 + +**验证。** `npm run build`,先红: +`DSH_OPS_WORKER_BIN=runtime/linux-x64/dsh-ops-worker node test/run.mjs +ptc-declaration-surface` → 11 项检查 5 项失败(全部新判定、诊断隔离检查与授权撤销 +检查;五项旧检查及新增的注册/声明/准入钉住保持绿)。实现后全绿: +ptc-declaration-surface 11/0/0;回归 publication 6 套件、mount 2、probe 2、 +batching 4,更宽的 `ptc` 家族 19 套件中 18 绿——`ptc.test.mjs` 仅在设 +`DSH_OPS_WORKER_BIN` 覆盖时失败,已验证在 D3 基线 commit 上同样失败 +(D3 验证备注里记录的安装包解析崩溃),不设覆盖时同样跳过,与基线一致。 +撤掉任一分类器分支、loader 拆分或诊断隔离守卫,其对应检查即刻转红。 diff --git "a/docs/\351\205\215\347\275\256\345\217\202\350\200\203.md" "b/docs/\351\205\215\347\275\256\345\217\202\350\200\203.md" index e3e9281f..f1eeba65 100644 --- "a/docs/\351\205\215\347\275\256\345\217\202\350\200\203.md" +++ "b/docs/\351\205\215\347\275\256\345\217\202\350\200\203.md" @@ -45,9 +45,10 @@ OPS 沿用单行插件页:bundle 只插入一个 `dsh-ops` 行。内置预设 | `residentTools` | true | 发布启动 API。PTC 常驻面只有 `ops_inspect_local_file`,其余按需通过 `ops_load_sdk` 加载 | | `bash` | true | 允许为 Windows 会话发布 `ops_bash`;POSIX 沿用宿主 bash | | `hidePwsh` | true | Bash 可用时隐藏 Windows 官方 shell;`global.hidePwsh` 为 false 时整体失效 | -| `hideOfficialTools` | true | 隐藏本模式替代的官方文件工具;`global.hideOfficialTools` 为 false 时整体失效 | +| `hideOfficialTools` | true | 隐藏本模式替代的官方文件工具,含 PTC 呈现:PTC 投影能服务对应 `ops_*` 时,官方 `read/glob/grep` 从工具面与 SDK 声明移除;`write/edit/read_image` 保留;投影无法服务某个名字时保留该名字的官方工具(如未就绪、`dynamicTools:false` 下的 `glob/grep`,或 `allowedTools` 收窄排除的名字);`global.hideOfficialTools` 为 false 时整体失效 | | `dynamicTools` | true | 发布渐进式 SDK 加载器 `ops_load_sdk`;与 `global.dynamicTools` 同时开启才生效(与门) | -| `batchingGuidance` | `ops-ptc` 为 true,其它模式为 false | 会话首轮注入一次完整教学(进入历史、不改系统提示词),之后按根 `run_code` 程序形状纠偏:`bypass`/`plan`/`shape` 三类每类每轮各锁一次、合规即解锁;提醒以 `additionalContexts` 附在触发调用结果上,不阻断也不改写程序。文案来自可编辑的 `prompts/batching-guidance.md`。仅 YAML 可配;旧 profile 整行覆盖配置且缺少 `ops-ptc` 节时仍默认 true,显式 false 可关闭 | +| `disclosurePolicy` | `on-demand` | YAML-only 显式首轮披露档位:`on-demand` 保持旧渐进路径(常驻 + loader);`full` 首轮直接发布整个 `allowedTools` 池外加 `result`,且无 loader(已实现,详见 docs/设计决策-20261119.md 的 D3 附录)。仅在 `global.dynamicTools` 开启时生效——显式 opt-in 不压过总开关;关闭期间该行保持旧 resident/alwaysLoaded 面,已保存的选择原样保留。非 PTC 行拒绝该键;非法值报错点名键与收到的值或类型。升级、保存设置与开关翻转都不会替你写入该键 | +| `batchingGuidance` | `ops-ptc` 为 true,其它模式为 false | 首个被接受的根 `run_code` 结果随 `additionalContexts` 投递一次完整教学(同一步 ephemeral、从不进入会话历史、不改系统提示词),之后按根 `run_code` 程序形状纠偏:`bypass`/`plan`/`shape` 三类每类每轮各锁一次、合规即解锁;提醒以 `additionalContexts` 附在触发调用结果上,不阻断也不改写程序。文案来自可编辑的 `prompts/batching-guidance.md`。仅 YAML 可配;旧 profile 整行覆盖配置且缺少 `ops-ptc` 节时仍默认 true,显式 false 可关闭 | | `spillRetentionDays` | 30 | 宿主 subprocess spill 保留窗口;`0` 只报告、永不删除 | | `spillTotalBudgetMiB` | 256 | 保留总字节预算(MiB);`0` 关闭预算 | | `spillSweepEnabled` | true | 组件激活时与每 256 次结算后执行一次有界清扫 | @@ -58,13 +59,22 @@ OPS 沿用单行插件页:bundle 只插入一个 `dsh-ops` 行。内置预设 见 [PTC SDK 按需加载](PTC按需加载.md))。三个 `spill*` 键作用于宿主 subprocess spill 保留, 语义见下文「内置高级配置」。 +五种旧配置输入——缺省、`global.dynamicTools:false`、行级 `dynamicTools:false`、行关闭加 +`alwaysLoaded:[stat]` 与 `residentTools:false`——的首轮 `ops_*` 名字集合已在 +[D1 真值表](设计决策-20261119.md) 中逐字锁定。旧 `dynamicTools:false` 只发布 +resident/alwaysLoaded 面,不得解读为全量选择;显式全量是独立的 `disclosurePolicy` +opt-in,其解析契约与保守优先级记录在同一文档的 D2 附录,发布归 D3 切片。 + ### 批量调用提示(`batchingGuidance`) -关闭时不产生任何额外上下文。代码仅对 `ops-ptc` 默认开启,旧 profile 整行覆盖配置且缺少该节时也保留此默认值;显式 `batchingGuidance: false` 仍可关闭。开启后提示分两处投递,文案来自外置英文 Markdown: +关闭时不产生任何额外上下文。代码仅对 `ops-ptc` 默认开启,旧 profile 整行覆盖配置且缺少该节时也保留此默认值;显式 `batchingGuidance: false` 仍可关闭。开启后两类提示都以 `additionalContexts` 附在工具结果上,文案来自外置英文 Markdown: -- **会话首轮一次**:在会话第一个被接受的非空用户消息批次里追加一条插件上下文(step-tail,来源 - `dsh-ops:step-tail` / `batching-guidance`),它进入会话历史,模型第一条请求即可看到完整范式:首行 - `// plan:` 指令、按依赖拆分、先加载再调用,以及四条可照抄的轨迹。 +- **每个 agent 一次**:随第一个被接受的根 `run_code` 结果以 `additionalContexts` 追加一条插件上下文 + (来源 `dsh-ops:step-tail`、contributor `batching-guidance:lesson`)。这是同一步的 ephemeral 收件箱, + 从不记为 user 历史,模型在首个被接受调用之后的请求即可看到完整范式:首行 + `// plan:` 指令、按依赖拆分、先加载再调用(仅动态加载发布加载器时),以及四条可照抄的轨迹。首轮 + 文案按「每条规则只讲一次」精简(修订号 7):反例改写与重复依赖拆分的散文已删,「只加载缺失 API」 + 与「模式变化后重查 SDK」升为核心规则,四条轨迹逐字节保留。 - **薄程序触发**:**根** `run_code` 程序命中三类锁中的一类时,在该次调用的结果上以 `additionalContexts` 追加一条短提醒。工具结果的值、正文与既有上下文都不变,程序既不阻断也不改写。 - 三类分别是 `bypass`(用 shell 或 `node:fs` 做文件工作)、`plan`(第一物理行没有 `// plan:`)与 `shape` @@ -74,8 +84,17 @@ OPS 沿用单行插件页:bundle 只插入一个 `dsh-ops` 行。内置预设 - 只检查根调用(`exec.parent === undefined`)。嵌套的 `tools.ops_*` 属于 PTC 执行层,说明不了这一步的形状。 - 分析是对程序文本的字面启发式:循环或 `map` 只算一个调用站点,不是运行时扇出;字符串与注释也可能误导。 提醒文案会说明这一点,不宣称实测的 dispatch 数;被取消或从未派发的调用不参与分析。 -- 全部都是插件上下文历史,tool schema 不变,系统提示词与其缓存因此不受影响。教学只有在真正出现持久 - `user/message` 时才算兑现;因取消或拒绝而撤回的批次不会消耗它,下一个被接受的批次仍会教学。 +- 提示全部随工具结果走 `additionalContexts`,tool schema 不变,系统提示词与其缓存因此不受影响。教学在 + 投递点(被接受的根 `run_code` 结果)即算兑现;被取消或从未被接受的调用不消耗它,下一个被接受的调用仍会教学。 +- 文案跟随实际发布面。动态加载发布 `ops_load_sdk` 时,教学文本逐字节照发,含"先加载再调用"这一步;`dynamicTools: + false` 时该作用域没有加载器,这一步与其表格行会被替换成"直接调用"一节,并列出本作用域实际发布的 API;一个 + OPS API 都不发布时则明确说明并指向官方工具。无加载器作用域仅发布 `ops_inspect_local_file` 这一个文件类 API 时, + 该节还会写明其契约:按精确路径读取单个文件、目录会被拒绝,且未发布 `ops_glob` 或 `ops_grep` 就无从列目录或 + 搜索,做文件工作需要已知的具体路径。程序形状提醒同样带这一信息,`bypass` 既不指向不可达的加载器,也不点名 + 本作用域拿不到的 API:文件工作那句只列可加载(或常驻)中真实存在的文件类 API,shell 这类绕过的工具不会被 + 当作替代手段;窄池的加载句保留教学里的顺序,已发布的名字"在下一程序"加载。两处是刻意取舍:随包教学文本 + 本身只在无加载器时替换(窄池与只读策略都只收窄作用域自己拼出的列表,不改写随包轨迹示例);列表来自配置池 + 与只读策略,而 loadable 同过加载器自身的门(按 agent 的归属 + 运行时可用性),只会更窄、不会更宽。 `batchingTelemetry`(同一行、YAML-only、默认 `false`)每投递一条提醒打印一行结构化日志,例如 `dsh-ops batching: turn=3 category=plan plan=1 bypass=0 shape=2 total=3`。它只做观测:不产生上下文、 @@ -128,6 +147,7 @@ presets: | `discovery` | 除 shell profile 外默认 true;关闭选择入口/SDK loader,但保留显式启动选择 | | `allowedTools` | 发现开启时默认所有可加载能力,否则空 | | `alwaysLoaded` | 空,必须在允许列表内;与 Native 常驻集合独立 | +| `disclosurePolicy` | 默认 `on-demand`;PTC 展示行的显式首轮披露档位:`full` 首轮直接发布整个 `allowedTools` 池外加 `result`、且无 loader——与内置行自 D3 起解析的档位一致。Native 展示行拒绝该键;非法值报错点名键与收到的值或类型;任何升级或保存都不会替你写入 | | `replaceOfficial` | false;只替换实际就绪且本步准入的对应项;条件项可能保留原版可见 | | `promptPolicy` | true;指导只广告实际发布的工具 | | `extraGuidance` | 空,Native 专属;PTC 明确拒绝 | diff --git "a/docs/\351\232\224\347\246\273\346\265\213\350\257\225\345\205\245\345\217\243.md" "b/docs/\351\232\224\347\246\273\346\265\213\350\257\225\345\205\245\345\217\243.md" new file mode 100644 index 00000000..b82771ca --- /dev/null +++ "b/docs/\351\232\224\347\246\273\346\265\213\350\257\225\345\205\245\345\217\243.md" @@ -0,0 +1,60 @@ +# 隔离测试入口(#990) + +## 这个入口为什么存在 + +一次 headless 对比只有在被测的组真正进入 OPS-PTC 路由时,才有资格谈论 +OPS-PTC。在此之前,loop fixture 装配的是合成的 `fixture` 预设,字节探针 +覆盖的是内置预设——一个来自 Standard 组合的绿色数字也可能被当成 OPS-PTC +证据。#980 把它定为 V1 门槛:实际走 Standard、worker 缺失或路由未知时, +这次运行判无效,并且必须把原因说出来。 + +## 一个组是什么 + +`test/lib/ptc-entry.mjs` 里的 `bootEntryGroup(root, …)` 装配一个组: + +- 真实的 AgentLoop 与 Session,脚本化 provider——从不发出真实的网络模型请求; +- 按宿主 profile 的方式挂载正式单行(包入口的 `apply`),只启用目标模式, + 组跑的就是 ops-ptc 行的真实默认值; +- 组的根目录必须位于该套件独占的临时 `DSH_HOME` 内——`node test/run.mjs` + 为每套件建一个 home,入口拒绝任何位于其外的根目录。 + +环境摘要在装配时采集,并附在每份报告上:插件 commit(直接读 git 目录, +不起子进程)、插件版本、宿主运行时版本、平台(例如 `linux-x64`)、 +worker 文件与 sha256、旁边的 staged runtime 回执以及两个摘要是否一致、 +沙箱模式与工作根目录、解析后的行内开关(含批量教学默认值)、以及实际 +`agentPreset` 与其模式。provider 标记为 `scripted`、 +`networkModelRequests` 记为 0——模拟 provider 只能验证装配,报告必须 +把这一点摆在明面上,而不是藏起来。 + +## 报告契约 + +`enterPtc(group)` 先跑最小用例:恰好一次不执行任何调用的脚本化 +provider 请求,而错误的组也只配得到这一次请求。检查按固定顺序执行—— +worker、preset、route、execution——第一个失败项就是报告的 reason。 + +- **entered**——请求的顶层恰好是 `['run_code']` 且 SDK 文本存在;只有这时 + 才进入第二回合,新造一个探针文件并通过 worker 执行一段程序。证据块记录 + 顶层面、preset、mode、请求计数与执行结果。 +- **invalid**——报告给出 reason 与每一项检查的 findings,但不带证据块,也不带 + 任何字节数或 token 数: + - Standard 是故意的反例。它是真实组合而非桩,在 `PRESET` 一项失败, + route 项显示 native 顶层面; + - PTC 预设配 native executor 在 `ROUTE` 一项失败——preset 正确而传输不对, + 顶层面甚至可能一个工具都不声明; + - 解析不到可执行 worker 的行在装配时就以 `WORKER` 失败,不会发出任何请求。 + +字节与成本分账刻意不在本入口采集,那是 V2 卡(#991)的范围。后续卡片 +复用本入口的组与环境记录,而不是重新造一套。 + +## 复现 + +``` +npm run build +npm run build:ptc +node test/run.mjs progressive-loop +``` + +过滤器必须命中 `progressive-loop.test.mjs`,四个入口断言与三个既有 +loop 断言同处一个套件。skip 意味着 worker 没有解析出来——先跑 +`npm run build:ptc`;skip 不是通过。进入这些组的只有新造的样例文件, +不读取、不报告任何真实会话、凭据或对话正文。 diff --git "a/docs/\351\233\206\346\210\220\346\216\245\345\217\243.md" "b/docs/\351\233\206\346\210\220\346\216\245\345\217\243.md" index 2aefc9b9..b923212a 100644 --- "a/docs/\351\233\206\346\210\220\346\216\245\345\217\243.md" +++ "b/docs/\351\233\206\346\210\220\346\216\245\345\217\243.md" @@ -52,7 +52,9 @@ integration({ 受管理解析,分别只作用于对应运行时。发现开启时 `allowedTools` 默认为所有可加载 API,关闭时 默认 `[]`;`alwaysLoaded` 默认 `[]`,必须属于允许列表。Native 常驻的三个 inspect/glob/grep 工具不受 高级允许列表限制;read_files/result 按需发现。外部数组默认均 `[]`,禁用优先,启动项必须已允许且未禁用。PTC 拒绝 -Native 专属 `extraGuidance`,不静默忽略。PTC 也拒绝 `denyCredentialReads` 覆盖,其凭据边界持续生效。 +Native 专属 `extraGuidance`,不静默忽略。PTC 也拒绝 `denyCredentialReads` 覆盖,其凭据边界持续生效。PTC 展示行额外接受 +`disclosurePolicy`(默认 `'on-demand'`,显式首轮档 `'full'`,Native 展示行拒绝该键)—— +与内置行解析的是同一个键。 `shell: false` 保留 `bashPath`、`allowSystemShellFallback` 与显式隐藏宿主 Shell 的意图;只有重新启用 Shell 时才生效。 仅文件:`{ preset, discovery: false }`;仅 Shell:`{ preset, profile: 'shell' }`;仅外部工具: @@ -61,7 +63,9 @@ Native 专属 `extraGuidance`,不静默忽略。PTC 也拒绝 `denyCredentialR ## 诊断与兼容 -接入会额外发布 `ops_status`,见[集成诊断](集成诊断.md)。bundle 的发现行现在仅服务内置与 +接入会额外发布 `ops_status`,见[集成诊断](集成诊断.md)。诊断的 PTC 首轮期望名与发布计划同源: +可选而尚未加载的 API(on-demand 档的 `result`、任何未选池成员)不算发布缺口;只有真正缺失的 +首轮计划名才算。bundle 的发现行现在仅服务内置与 无预设 headless composition;未知预设必须显式接入,不再意外获得全局发现入口。旧组件导出 保留既有 legacy 语义,不要用内部 `lib/*` 作为公共 API;无预设 headless 的 Standard 回退保留。 diff --git "a/docs/\351\233\206\346\210\220\350\257\212\346\226\255.md" "b/docs/\351\233\206\346\210\220\350\257\212\346\226\255.md" index aee796c1..34d3ebbd 100644 --- "a/docs/\351\233\206\346\210\220\350\257\212\346\226\255.md" +++ "b/docs/\351\233\206\346\210\220\350\257\212\346\226\255.md" @@ -23,5 +23,6 @@ `PRESENTATION_MISMATCH` 必须检查实际 assemble 结果:PTC 要求 `run_code` 和非空 `tools:ptc-only` 段,不能用预设名或提示词建议推断。`PUBLICATION_UNAVAILABLE` 应先检查 -接入生命周期、作用域和注册错误,再考虑权限。若官方工具已隐藏但对应 OPS 工具不可见, +接入生命周期、作用域和注册错误,再考虑权限。PTC 期望集合与发布计划同源:可选而尚未加载的 +API(on-demand 档的 `result`)不算缺口;真正缺失的首轮计划名才算。若官方工具已隐藏但对应 OPS 工具不可见, 这是替换控制器缺陷,不能要求用户重试。 diff --git "a/docs/\351\242\204\350\256\276\344\270\216\351\273\230\350\256\244\346\250\241\345\274\217.md" "b/docs/\351\242\204\350\256\276\344\270\216\351\273\230\350\256\244\346\250\241\345\274\217.md" index e0414c72..f88d933a 100644 --- "a/docs/\351\242\204\350\256\276\344\270\216\351\273\230\350\256\244\346\250\241\345\274\217.md" +++ "b/docs/\351\242\204\350\256\276\344\270\216\351\273\230\350\256\244\346\250\241\345\274\217.md" @@ -56,10 +56,11 @@ node scripts/sync-preset-ops-ptc.mjs --source [--ou - **按下按钮**:`ops-ptc` 预设已注册时,插件把 `agent-preset-registry` 的 `selectedDefault` 写成 `ops-ptc`——与设置页写的是同一个易变字段。只有这一次点击会覆盖手动选择。 -- **什么都不做**:不写任何东西。apply、保存其它开关导致的重挂载、禁用该行都不会改当前默认,所以设置页选的 - 预设在重启后仍然是默认。 -- **禁用/卸载 `dsh-ops` 行**:只释放预设注册,默认预设原样保留,即使它指向 `ops-ptc`。移除该行前请先把默认 - 切到别的预设,否则新会话可能解析一个已经不存在的预设。 +- **什么都不做**:不写任何东西。apply、保存其它开关导致的重挂载都不会改当前默认,所以设置页选的预设 + 在重启后仍然是默认。 +- **禁用/卸载 `dsh-ops` 行**:先释放预设注册,再检查当前默认。若仍指向 `ops-ptc`,行级清理会恢复宿主 + `config.default`;若指向其他预设则绝不改写。settings 服务缺失或拒绝写入时会明确报告残留,要求操作者 + 另选默认,不会假装 profile 已修复。 - 按钮会回一个状态:`applied`、`no-update`(本来就是默认),或带原因的 `unavailable`/`failed`——预设未注册、 宿主 settings 服务缺失、写入被拒。 - 写入的值经宿主设置层持久化到 profile 的 `cordis.patch.yml`(`agent-preset-registry` 覆盖),重启后保持; @@ -70,8 +71,8 @@ node scripts/sync-preset-ops-ptc.mjs --source [--ou ## 手动设置其他默认预设 -`ops-ptc` 只能通过上面的按钮(或设置页)成为默认。插件不再自动回退它,所以禁用/卸载该行前请先把默认切到别的 -预设。其他默认预设仍照常由宿主/操作者决定: +`ops-ptc` 只能通过上面的按钮(或设置页)成为默认。只有在当前选择仍是 `ops-ptc` 时,移除该行才会恢复宿主默认; +不会自动恢复成 `ops-ptc`。其他默认预设仍照常由宿主/操作者决定: - **宿主「Agent 预设」设置页**:选择 `standard`、`ptc` 等任一已注册预设,持久化 `selectedDefault`。 - **profile patch 显式设置**(可粘贴到 `$DSH_HOME/profiles//cordis.patch.yml`): @@ -95,9 +96,10 @@ node scripts/sync-preset-ops-ptc.mjs --source [--ou - **开关开启(默认)时**:禁用/卸载会先恢复默认再释放注册,正常路径不会留下指向已消失预设的默认;禁用后 重新启用插件,或在该开关关闭后用设置页选择你想要的默认即可。恢复失败只告警、不阻塞卸载,此时按下一 条处理。 -- **按下按钮之后**:这就是一次普通的设置页选择,重启后保持;禁用或卸载该行时不会自动回退,移除行前请先换默认。 -- **禁用/卸载之前**:先在设置页把默认切到别的预设。插件不再自动改写它,否则新会话会按当前定义解析一个随行消失的 - 预设而失败。 +- **按下按钮之后**:这就是一次普通的设置页选择,重启后保持;若在该选择仍生效时移除行,清理会恢复宿主默认, + 不会留下失效引用。 +- **禁用/卸载之前**:当前默认是 `ops-ptc` 时无需手动改写,行会处理;属于其他预设的选择会原样保留。若恢复无法 + 持久化,请按警告在设置页另选默认。 - 片段缺失或宿主升级导致解析失败时,预设不可用但工具面照常工作;默认预设不受影响,按钮回报 `unavailable`。 重新生成片段并重启宿主即可恢复。 - 插件管理器里找不到 `ops-ptc` 预设行是设计如此,不要尝试手动新增声明;它由插件运行时提供。 @@ -111,7 +113,7 @@ node scripts/sync-preset-ops-ptc.mjs --source [--ou | 插件管理器行 | 有独立的 preset 行 | 无预设行,随 `dsh-ops` 行启停 | | 发布面 | 官方 PTC 工具 | 与 `ptc` 分区相同:真实 PTC 呈现下的 `run_code` 加同栈类型化 SDK 文件工具 | | 压缩 | 官方 `compaction-basic` | 同为官方 `compaction-basic`,不引入 `dsh-compress-codex` | -| 批量调用提示 | 无 | `batchingGuidance`:代码与随包 `ops-ptc` 节均默认 true,旧配置整行覆盖且缺少该节也保留默认值,显式 false 可关闭;首轮教学一次 + 按根 `run_code` 程序形状(`bypass`/`plan`/`shape`,每类每轮各锁一次、合规解锁)以 `additionalContexts` 纠偏;文案在可编辑的 `prompts/batching-guidance.md` | +| 批量调用提示 | 无 | `batchingGuidance`:代码与随包 `ops-ptc` 节均默认 true,旧配置整行覆盖且缺少该节也保留默认值,显式 false 可关闭;随首个被接受的根 `run_code` 结果教学一次(`additionalContexts`,从不进入会话历史)+ 按根 `run_code` 程序形状(`bypass`/`plan`/`shape`,每类每轮各锁一次、合规解锁)纠偏;文案在可编辑的 `prompts/batching-guidance.md` | | 单行配置分区 | `ptc` | `ops-ptc`(键与 `ptc` 相同,另加 `batchingGuidance`) | | 默认预设 | 由宿主/操作者决定 | 由详情页按钮点一次设定(`applied`/`no-update`),不再自动强制或回退 | | 排序 | order 2 | order 6,排在官方行之后 | diff --git a/native/crates/ops-documents/src/pdf_container.rs b/native/crates/ops-documents/src/pdf_container.rs index d5a2debb..64f6b2c0 100644 --- a/native/crates/ops-documents/src/pdf_container.rs +++ b/native/crates/ops-documents/src/pdf_container.rs @@ -1,4 +1,4 @@ -//! A deliberately restricted PDF container: one classic xref table, direct stream lengths, no recovery. +//! A deliberately restricted PDF container: one classic xref table, lengths resolved through that xref, no recovery. use crate::pdf_syntax::{Dict, Lexer, Value, bad, limit, unsupported, white}; use ops_protocol::Result; use std::collections::{BTreeMap, BTreeSet}; @@ -6,6 +6,8 @@ pub const MAX_FILE: usize = 16 * 1024 * 1024; pub const MAX_OBJECTS: usize = 10_000; pub const MAX_STREAM: usize = 1024 * 1024; pub type ObjectId = (u32, u16); +/// The classic xref table: object number to (byte offset, generation, active). +type XrefEntries = BTreeMap; #[derive(Debug)] pub struct Object<'a> { pub value: Value, @@ -82,11 +84,63 @@ fn footer(data: &[u8]) -> Result<(usize, usize)> { } Ok((xref, footer_start)) } +/// Resolve one stream Length (#788): a direct bounded integer, or one indirect +/// reference whose xref entry names a plain integer object. The probe re-lexes +/// the provider's exact header, so absent, free, chained, non-integer and +/// identity-mismatched targets fail closed; the caller's node budget pays for +/// the probe, keeping the whole parse bounded. No recovery, no endstream scan. +fn stream_length( + data: &[u8], + entries: &XrefEntries, + dict: &Dict, + nodes: &mut usize, +) -> Result { + let length = match get(dict, b"Length")? { + Value::Ref(number, generation) => { + let (offset, entry_generation, active) = *entries + .get(&(*number as usize)) + .ok_or_else(|| bad("PDF stream Length does not name an xref entry"))?; + if !active || entry_generation != *generation { + return Err(bad( + "PDF stream Length must name an active xref entry with a matching generation", + )); + } + if !data.get(offset).is_some_and(u8::is_ascii_digit) { + return Err(bad( + "PDF stream Length xref must point exactly to an object header", + )); + } + let mut lexer = Lexer { + data, + at: offset, + nodes: *nodes, + }; + if lexer.unsigned()? != *number as usize || lexer.unsigned()? != *generation as usize { + return Err(bad("PDF stream Length object identity mismatch")); + } + lexer.keyword(b"obj")?; + let resolved = lexer + .value(0)? + .integer() + .map_err(|_| bad("PDF stream Length must resolve to a bounded integer object"))?; + lexer.keyword(b"endobj")?; + *nodes = lexer.nodes; + resolved + } + value => value.integer()?, + }; + if length > MAX_STREAM { + return Err(limit("PDF encoded stream exceeds 1 MiB")); + } + Ok(length) +} + fn object<'a>( data: &'a [u8], offset: usize, id: ObjectId, nodes: usize, + entries: &XrefEntries, ) -> Result<(Object<'a>, usize, usize)> { if !data.get(offset).is_some_and(u8::is_ascii_digit) { return Err(bad("PDF xref must point exactly to an object header")); @@ -104,11 +158,10 @@ fn object<'a>( let mut stream = None; let token = lexer.word()?; if token == b"stream" { - let length = get(value.dict()?, b"Length")?.integer()?; - if length > MAX_STREAM { - return Err(limit("PDF encoded stream exceeds 1 MiB")); - } - // Indirect Length is declined before reading stream bytes; no searching for endstream. + let mut nodes = lexer.nodes; + let length = stream_length(lexer.data, entries, value.dict()?, &mut nodes)?; + lexer.nodes = nodes; + // The Length is fully resolved before stream bytes are read; no searching for endstream. if lexer.data.get(lexer.at..lexer.at + 2) == Some(b"\r\n") { lexer.at += 2; } else if lexer.data.get(lexer.at) == Some(&b'\n') { @@ -228,16 +281,16 @@ pub fn parse(data: &[u8]) -> Result> { return Err(bad("Unexpected PDF data between trailer and final footer")); } let mut ordered = Vec::new(); - for (number, (offset, generation, active)) in entries { - if !active { + for (number, (offset, generation, active)) in &entries { + if !*active { continue; } - if offset < 9 || offset >= xref { + if *offset < 9 || *offset >= xref { return Err(bad( "Active PDF object offset is outside the object section", )); } - ordered.push((offset, (number as u32, generation))); + ordered.push((*offset, (*number as u32, *generation))); } ordered.sort_unstable(); let mut previous = 0; @@ -258,7 +311,7 @@ pub fn parse(data: &[u8]) -> Result> { "Unindexed data or overlapping comments in PDF object section", )); } - let (item, end, count) = object(&data[..xref], offset, id, nodes)?; + let (item, end, count) = object(&data[..xref], offset, id, nodes, &entries)?; previous = end; nodes = count; objects.insert(id, item); diff --git a/native/crates/ops-documents/src/pdf_container_tests.rs b/native/crates/ops-documents/src/pdf_container_tests.rs index 81d5f979..6326c2fd 100644 --- a/native/crates/ops-documents/src/pdf_container_tests.rs +++ b/native/crates/ops-documents/src/pdf_container_tests.rs @@ -337,3 +337,80 @@ fn stream_roots_reserved_generations_and_direct_oversize_decode_are_refused() { }; assert_eq!(decode(&raw).unwrap_err().code, "OUTPUT_LIMIT"); } + +#[test] +fn indirect_stream_lengths_resolve_through_the_xref_without_recovery() { + // #788: Word and LibreOffice emit "/Length N G R". The integer provider may + // sit at any xref offset; one bounded probe re-lexes its exact header. + let pdf = fixture(&[ + b"<<>>", + b"9", + b"<< /Length 2 0 R >>\nstream\n(Hi) Tj\n\nendstream", + ]); + let document = parse(&pdf).unwrap(); + assert_eq!( + document.object((3, 0)).unwrap().stream, + Some(b"(Hi) Tj\n\n".as_slice()) + ); + // The provider may also follow the stream object in file order. + let reversed = fixture(&[b"<<>>", b"<< /Length 3 0 R >>\nstream\nxy\nendstream", b"2"]); + assert_eq!( + parse(&reversed).unwrap().object((2, 0)).unwrap().stream, + Some(b"xy".as_slice()) + ); + // Only a plain integer object resolves: absent, dictionary, chained, + // generation-mismatched and oversized providers all fail closed. + rejected( + &fixture(&[b"<<>>", b"9", b"<< /Length 4 0 R >>\nstream\nxy\nendstream"]), + "DOCUMENT_PARSE_ERROR", + ); + rejected( + &fixture(&[ + b"<<>>", + b"<<>>", + b"<< /Length 2 0 R >>\nstream\nxy\nendstream", + ]), + "DOCUMENT_PARSE_ERROR", + ); + rejected( + &fixture(&[ + b"<<>>", + b"3 0 R", + b"<< /Length 2 0 R >>\nstream\nxy\nendstream", + b"9", + ]), + "DOCUMENT_PARSE_ERROR", + ); + rejected( + &fixture(&[b"<<>>", b"9", b"<< /Length 2 1 R >>\nstream\nxy\nendstream"]), + "DOCUMENT_PARSE_ERROR", + ); + rejected( + &fixture(&[ + b"<<>>", + b"1048577", + b"<< /Length 2 0 R >>\nstream\nxy\nendstream", + ]), + "OUTPUT_LIMIT", + ); + // A free xref entry is never a Length provider. + let mut free = fixture(&[b"<<>>", b"9", b"<< /Length 2 0 R >>\nstream\nxy\nendstream"]); + let at = free.windows(7).position(|w| w == b"2 0 obj").unwrap(); + free = replace( + &free, + format!("{at:010} 00000 n ").as_bytes(), + format!("{at:010} 65535 f ").as_bytes(), + ); + rejected(&free, "DOCUMENT_PARSE_ERROR"); + // The probe runs before the ordered scan reaches a duplicated offset and + // refuses a header that disagrees with the referenced identity. + let mut aliased = fixture(&[b"<<>>", b"<< /Length 3 0 R >>\nstream\nxy\nendstream", b"2"]); + let stream_at = aliased.windows(7).position(|w| w == b"2 0 obj").unwrap(); + let provider_at = aliased.windows(7).position(|w| w == b"3 0 obj").unwrap(); + aliased = replace( + &aliased, + format!("{provider_at:010} 00000 n ").as_bytes(), + format!("{stream_at:010} 00000 n ").as_bytes(), + ); + rejected(&aliased, "DOCUMENT_PARSE_ERROR"); +} diff --git a/native/crates/ops-files/src/file_path.rs b/native/crates/ops-files/src/file_path.rs index eaa4e36e..37d00ae1 100644 --- a/native/crates/ops-files/src/file_path.rs +++ b/native/crates/ops-files/src/file_path.rs @@ -83,7 +83,24 @@ pub fn checked_leaf(raw: &str, allow_missing: bool) -> Result { let mut parent = Dir::open_ambient_dir(&anchor, ambient_authority())?; let name = names.pop().unwrap_or_else(|| ".".into()); for component in names { - regular(&parent.symlink_metadata(&component)?, true)?; + let metadata = parent.symlink_metadata(&component)?; + if regular(&metadata, true).is_err() && metadata.is_file() && asar_archive(&component) { + // #836: Electron presents .asar archives as directories only through + // its own patched in-process filesystem. The standalone worker sees + // a regular file, so a member path can never resolve; name the + // archive instead of implying a no-follow bypass. + return Err(Failure::new( + "ARCHIVE_MEMBER_UNSUPPORTED", + format!( + "Path routes inside '{component}': an Electron .asar archive is a \ + virtual directory only for its host process; this standalone \ + worker cannot read archive members. Inspect the archive file \ + itself (hex view) or use an unpacked copy; no other tool here \ + can open it either." + ), + )); + } + regular(&metadata, true)?; parent = parent.open_dir_nofollow(&component)?; } regular(&parent.dir_metadata()?, true)?; @@ -115,6 +132,13 @@ pub fn validate_name(name: &str) -> Result<()> { ops_protocol::windows_paths::validate_name(name)?; Ok(()) } +/// Electron's asar archive spelling: a regular file whose name ends in ".asar". +/// A directory with the same spelling stays an ordinary directory. +fn asar_archive(name: &str) -> bool { + let bytes = name.as_bytes(); + bytes.len() >= 5 && bytes[bytes.len() - 5..].eq_ignore_ascii_case(b".asar") +} + pub fn link(metadata: &Metadata) -> bool { if metadata.file_type().is_symlink() { return true; diff --git a/native/crates/ops-files/src/file_path_tests.rs b/native/crates/ops-files/src/file_path_tests.rs index a66a84bb..8a184c64 100644 --- a/native/crates/ops-files/src/file_path_tests.rs +++ b/native/crates/ops-files/src/file_path_tests.rs @@ -36,3 +36,55 @@ fn near_audit_names_are_not_reserved_components() { crate::paths::relative(name).unwrap(); } } + +#[test] +fn asar_file_components_are_archive_members_not_traversal_bypasses() { + let root = std::env::temp_dir().join(format!("dsh-ops-asar-{}", uuid::Uuid::new_v4())); + std::fs::create_dir(&root).unwrap(); + // macOS reaches the temp directory through /var -> /private/var, and checked_leaf + // refuses a symlinked component, so pin the canonical spelling before walking. + let root = std::fs::canonicalize(&root).unwrap(); + std::fs::write(root.join("bundle.asar"), b"archive bytes").unwrap(); + // #836: Electron presents a real .asar FILE as a directory only through its + // own patched filesystem; the worker names the archive member instead of + // implying a no-follow bypass. + let failure = match checked_leaf( + root.join("bundle.asar").join("app.js").to_str().unwrap(), + true, + ) { + Err(failure) => failure, + Ok(_) => panic!("a file component must not become a traversable directory"), + }; + assert_eq!(failure.code, "ARCHIVE_MEMBER_UNSUPPORTED"); + assert!( + failure.message.contains("bundle.asar"), + "{}", + failure.message + ); + // A genuine directory keeps working: the .asar spelling is not reserved. + std::fs::create_dir(root.join("folder.asar")).unwrap(); + std::fs::write(root.join("folder.asar").join("inner.js"), b"real").unwrap(); + let directory = checked_leaf( + root.join("folder.asar").join("inner.js").to_str().unwrap(), + false, + ) + .unwrap(); + assert_eq!(directory.name, "inner.js"); + // A plain file component that is not an archive keeps the traversal refusal. + std::fs::write(root.join("readme.txt"), b"text").unwrap(); + let plain = match checked_leaf( + root.join("readme.txt").join("inner.js").to_str().unwrap(), + true, + ) { + Err(failure) => failure, + Ok(_) => panic!("a plain file must not become a traversable directory"), + }; + assert_eq!(plain.code, "PATH_RESTRICTED"); + // The archive itself stays a readable regular file. + let archive = checked_leaf(root.join("bundle.asar").to_str().unwrap(), false).unwrap(); + assert_eq!(archive.name, "bundle.asar"); + // The Targets hold directory handles on Windows; drop them before removing. + drop(directory); + drop(archive); + std::fs::remove_dir_all(&root).unwrap(); +} diff --git a/native/crates/ops-files/src/file_single.rs b/native/crates/ops-files/src/file_single.rs index e7c547eb..074fa047 100644 --- a/native/crates/ops-files/src/file_single.rs +++ b/native/crates/ops-files/src/file_single.rs @@ -77,12 +77,18 @@ pub(super) fn run(request: &Request) -> Result { "edit_file" => { let args: EditArgs = serde_json::from_value(request.args.clone())?; valid_revision(&args.expected_revision, false)?; - write_single( + // Raw bytes stay the matching contract; when oldText does not occur + // in them, exactly one re-interpretation is tried: the reversibly + // escaped rendering the Native reader shows (#939). Raw matches keep + // precedence, so a literal text is never re-read through the decoder. + let rendered_matches = std::cell::Cell::new(0u32); + let mut value = write_single( request, &args.path, &args.expected_revision, args.dry_run, |original| { + rendered_matches.set(0); let text = std::str::from_utf8(original.ok_or_else(|| { Failure::new("NOT_FOUND", "Edit requires an existing file") })?) @@ -98,25 +104,24 @@ pub(super) fn run(request: &Request) -> Result { "oldText must not be empty", )); } - // Count overlapping occurrences as well ("aa" in "aaa" is ambiguous). - let at = text.find(&edit.old_text).ok_or_else(|| { - Failure::new( - "EDIT_CONFLICT", - "oldText was not found; inspect the current file", - ) - })?; - let next_character = text[at..] - .chars() - .next() - .expect("nonempty match") - .len_utf8(); - if text[at + next_character..].contains(&edit.old_text) { - return Err(Failure::new( - "EDIT_CONFLICT", - "oldText is not unique; include more surrounding text", - )); + let decoded = crate::rendered_text::decode_rendered(&edit.old_text); + let (at, length, replacement, rendered) = + match find_unique(text, &edit.old_text) { + Ok(at) => (at, edit.old_text.len(), edit.new_text.clone(), false), + Err(raw) => match find_unique(text, &decoded) { + Ok(at) => ( + at, + decoded.len(), + crate::rendered_text::decode_rendered(&edit.new_text), + true, + ), + Err(_) => return Err(raw), + }, + }; + if rendered { + rendered_matches.set(rendered_matches.get() + 1); } - spans.push((at, at + edit.old_text.len(), edit.new_text.as_str())); + spans.push((at, at + length, replacement)); } spans.sort_by_key(|span| span.0); if spans.windows(2).any(|pair| pair[0].1 > pair[1].0) { @@ -129,13 +134,21 @@ pub(super) fn run(request: &Request) -> Result { let mut cursor = 0; for (start, end, replacement) in spans { next.push_str(&text[cursor..start]); - next.push_str(replacement); + next.push_str(&replacement); cursor = end; } next.push_str(&text[cursor..]); Ok(next.into_bytes()) }, - ) + )?; + let rendered = rendered_matches.get(); + if rendered > 0 + && let Some(items) = value["items"].as_array_mut() + && let Some(item) = items.first_mut() + { + item["renderedMatches"] = json!(rendered); + } + Ok(value) } "copy" | "move" => transfer_file(request), "remove" => remove_file(request), @@ -145,6 +158,31 @@ pub(super) fn run(request: &Request) -> Result { )), } } +/// First occurrence with the existing no-ambiguity rule, shared by the raw and +/// rendered interpretations of oldText. +fn find_unique(text: &str, needle: &str) -> Result { + let at = text.find(needle).ok_or_else(|| { + Failure::new( + "EDIT_CONFLICT", + "oldText was not found; it matches neither the raw file bytes nor the \ + decoded form of the rendered read output; inspect the current file", + ) + })?; + // Count overlapping occurrences as well ("aa" in "aaa" is ambiguous). + let next_character = text[at..] + .chars() + .next() + .expect("nonempty match") + .len_utf8(); + if text[at + next_character..].contains(needle) { + return Err(Failure::new( + "EDIT_CONFLICT", + "oldText is not unique; include more surrounding text", + )); + } + Ok(at) +} + fn valid_revision(revision: &str, absent: bool) -> Result<()> { if (absent && revision == "absent") || (revision.len() == 64 @@ -831,4 +869,69 @@ mod tests { drop(checked); std::fs::remove_dir_all(root).unwrap(); } + + #[test] + fn old_text_copied_from_the_rendered_reader_edits_after_a_raw_miss() { + let root = std::fs::canonicalize(std::env::temp_dir()) + .unwrap() + .join(format!("dsh-ops-rendered-edit-{}", uuid::Uuid::new_v4())); + std::fs::create_dir(&root).unwrap(); + let path = root.join("rendered.txt"); + // A raw tab and a raw backslash: the Native reader renders them as \t and \\. + let raw = "alpha\ttab C:\\path\nbeta\n"; + std::fs::write(&path, raw).unwrap(); + let expected_revision = format!("{:x}", Sha256::digest(raw.as_bytes())); + // Exactly the text a model copies out of the rendered read output. + let rendered = r"alpha\ttab C:\\path"; + let value = run(&request( + &root, + "edit_file", + json!({ + "path": path.to_string_lossy(), + "expectedRevision": expected_revision, + "edits": [{"oldText": rendered, "newText": r"alpha\ttab C:\\renamed"}], + }), + )) + .unwrap(); + assert_eq!(value["status"], json!("completed")); + assert_eq!(value["items"][0]["renderedMatches"], json!(1)); + assert_eq!( + std::fs::read_to_string(&path).unwrap(), + "alpha\ttab C:\\renamed\nbeta\n" + ); + std::fs::remove_dir_all(&root).unwrap(); + } + + #[test] + fn raw_literals_keep_precedence_over_the_rendered_interpretation() { + let root = std::fs::canonicalize(std::env::temp_dir()) + .unwrap() + .join(format!( + "dsh-ops-rendered-precedence-{}", + uuid::Uuid::new_v4() + )); + std::fs::create_dir(&root).unwrap(); + let path = root.join("precedence.txt"); + // The file holds BOTH a literal backslash-t pair and a raw tab; an + // oldText that occurs raw must never be re-read through the decoder. + let raw = "literal\\tab\treal\n"; + std::fs::write(&path, raw).unwrap(); + let value = run(&request( + &root, + "edit_file", + json!({ + "path": path.to_string_lossy(), + "expectedRevision": format!("{:x}", Sha256::digest(raw.as_bytes())), + "edits": [{"oldText": "literal\\tab", "newText": "literal\\kept"}], + }), + )) + .unwrap(); + assert_eq!(value["status"], json!("completed")); + assert_eq!(value["items"][0]["renderedMatches"], json!(Value::Null)); + assert_eq!( + std::fs::read_to_string(&path).unwrap(), + "literal\\kept\treal\n" + ); + std::fs::remove_dir_all(&root).unwrap(); + } } diff --git a/native/crates/ops-files/src/lib.rs b/native/crates/ops-files/src/lib.rs index d03bc170..9ba5de64 100644 --- a/native/crates/ops-files/src/lib.rs +++ b/native/crates/ops-files/src/lib.rs @@ -4,6 +4,7 @@ pub mod cache_io; mod file; pub mod file_path; pub mod paths; +mod rendered_text; #[cfg(all(test, unix))] use cap_std::ambient_authority; #[cfg(unix)] diff --git a/native/crates/ops-files/src/rendered_text.rs b/native/crates/ops-files/src/rendered_text.rs new file mode 100644 index 00000000..53d89437 --- /dev/null +++ b/native/crates/ops-files/src/rendered_text.rs @@ -0,0 +1,144 @@ +//! Decode the reversible rendering the Native reader shows for source lines. +//! +//! The encoder is "src/projections/native/line-encoding.ts"; this module is +//! its exact inverse, never a second encoding policy. ops_edit_file consults +//! it only after a raw match has failed (#939), so literal text keeps +//! precedence. + +/// Interpret "text" as one rendered source line: a doubled backslash and the +/// JSON-style escapes (\b, \t, \n, \f, \r, \uXXXX) become their raw +/// characters. Unknown escapes and plain text stay byte-identical, so a string +/// that never came from the renderer decodes to itself and a half escape +/// invents nothing. +pub fn decode_rendered(text: &str) -> String { + let bytes = text.as_bytes(); + let mut out = String::with_capacity(bytes.len()); + let mut at = 0; + while at < bytes.len() { + if bytes[at] == b'\\' && at + 1 < bytes.len() { + match bytes[at + 1] { + b'\\' => { + out.push('\\'); + at += 2; + continue; + } + b'b' => { + out.push('\u{0008}'); + at += 2; + continue; + } + b't' => { + out.push('\t'); + at += 2; + continue; + } + b'n' => { + out.push('\n'); + at += 2; + continue; + } + b'f' => { + out.push('\u{000c}'); + at += 2; + continue; + } + b'r' => { + out.push('\r'); + at += 2; + continue; + } + b'u' if at + 6 <= bytes.len() + && bytes[at + 2..at + 6].iter().all(u8::is_ascii_hexdigit) => + { + let code = u32::from_str_radix(&text[at + 2..at + 6], 16).unwrap_or(0); + match char::from_u32(code) { + Some(decoded) => out.push(decoded), + // Surrogate halves never appear in rendered text; keep + // the six characters literal instead of inventing one. + None => out.push_str(&text[at..at + 6]), + } + at += 6; + continue; + } + _ => {} + } + } + // Copy one whole UTF-8 character: astral text stays intact. + let width = character_width(bytes[at]); + out.push_str(&text[at..at + width]); + at += width; + } + out +} + +fn character_width(lead: u8) -> usize { + match lead { + 0x00..=0x7f => 1, + 0xc0..=0xdf => 2, + 0xe0..=0xef => 3, + 0xf0..=0xf7 => 4, + // A valid &str never offers any other lead byte. + _ => 1, + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn rendered_escapes_decode_to_their_exact_source_characters() { + assert_eq!(decode_rendered(r"a\tb"), "a\tb"); + assert_eq!(decode_rendered(r"C:\\new"), "C:\\new"); + assert_eq!(decode_rendered(r"\b\f\r\n"), "\u{0008}\u{000c}\r\n"); + assert_eq!(decode_rendered(r"\u0000"), "\0"); + assert_eq!(decode_rendered(r"\u2028text"), "\u{2028}text"); + assert_eq!(decode_rendered(r"\u0085"), "\u{0085}"); + } + + #[test] + fn plain_text_unknown_and_broken_escapes_stay_byte_identical() { + assert_eq!(decode_rendered("plain 中文 😀"), "plain 中文 😀"); + assert_eq!(decode_rendered(r"\q\z"), "\\q\\z"); + assert_eq!(decode_rendered(r"\uzzzz"), "\\uzzzz"); + assert_eq!(decode_rendered(r"\ud800"), "\\ud800"); + assert_eq!(decode_rendered(r"\u12"), "\\u12"); + assert_eq!(decode_rendered("trailing\\"), "trailing\\"); + assert_eq!(decode_rendered("back\\中"), "back\\中"); + } + + #[test] + fn decoding_the_renderer_output_restores_every_source_sample() { + // The same escape classes the TS encoder renders (see its suite); a + // decoded rendering must be the raw source again. + for source in [ + "alpha\ttab C:\\path\nbeta\n", + "literal\\tab\treal\n", + "\u{7f}\u{85}\u{2028}\u{2029}\0", + ] { + let rendered = encode_for_test(source); + assert_ne!(rendered, source, "the sample must actually be rendered"); + assert_eq!(decode_rendered(&rendered), source); + } + } + + /// A mirror of the projection encoder, for round-trip coverage only. + fn encode_for_test(source: &str) -> String { + let mut out = String::new(); + for ch in source.chars() { + match ch { + '\\' => out.push_str("\\\\"), + '\u{0008}' => out.push_str("\\b"), + '\t' => out.push_str("\\t"), + '\n' => out.push_str("\\n"), + '\u{000c}' => out.push_str("\\f"), + '\r' => out.push_str("\\r"), + c if (c as u32) < 0x20 || matches!(c as u32, 0x7f | 0x85 | 0x2028 | 0x2029) => { + out.push_str(&format!("\\u{:04x}", c as u32)); + } + c => out.push(c), + } + } + out + } +} diff --git a/native/windows-metadata/src/ea.rs b/native/windows-metadata/src/ea.rs new file mode 100644 index 00000000..4e6137fe --- /dev/null +++ b/native/windows-metadata/src/ea.rs @@ -0,0 +1,383 @@ +//! Bounded NTFS extended-attribute capture, restore and verification. +//! +//! Kernel EA names (the "$KERNEL." prefix, e.g. EFS's PurgeEFSProxyCache or +//! SmartLocker tags) are written by kernel components and cannot be recreated +//! from user mode. They are captured as names and reported per name in the +//! publication warnings; nothing is silently dropped (#874). Every other EA +//! is captured byte-exactly, restored on the staged file and re-verified +//! after restore; a mismatch, an unreadable or an unsettable set aborts +//! publication. + +use super::{IoStatus, nt_error, unsupported}; +use std::{ + ffi::c_void, + fs::File, + io, + os::windows::io::AsRawHandle, + ptr::{null, null_mut}, +}; +use windows_sys::Win32::Foundation::HANDLE; + +/// One bounded capture budget, matching the named-stream capture in lib.rs. +pub const MAX_EA_BYTES: usize = 64 * 1024; +pub const MAX_EA_ENTRIES: usize = 4096; +/// The kernel-mode EA name prefix; user-mode NtSetEaFile refuses these names. +const KERNEL_EA_PREFIX: &[u8] = b"$kernel."; +const FILE_EA_INFORMATION: u32 = 7; +const NO_MORE_EAS: i32 = 0x0C00_0191_u32 as i32; +const BUFFER_OVERFLOW: i32 = 0x8000_0005u32 as i32; + +#[link(name = "ntdll")] +unsafe extern "system" { + fn NtQueryEaFile( + handle: HANDLE, + status: *mut IoStatus, + buffer: *mut c_void, + length: u32, + return_single_entry: u8, + ea_list: *const c_void, + ea_list_length: u32, + ea_index: *mut u32, + restart_scan: u8, + ) -> i32; + fn NtSetEaFile( + handle: HANDLE, + status: *mut IoStatus, + buffer: *const c_void, + length: u32, + ) -> i32; +} + +/// One FILE_FULL_EA_INFORMATION entry, in a canonical owned spelling. +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct EaEntry { + pub flags: u8, + pub name: Vec, + pub value: Vec, +} + +#[derive(Default, Debug)] +pub struct CapturedEas { + /// Restorable entries only; kernel names never reach a set call. + pub entries: Vec, + /// Names of kernel-owned EAs, reported as unpreservable evidence. + pub kernel: Vec>, +} + +impl CapturedEas { + /// Warnings state exactly what will not be preserved, before anything is + /// written, so a caller can stop instead of silently losing metadata. + pub fn warnings(&self) -> Vec { + self.kernel + .iter() + .map(|name| { + format!( + "Kernel extended attribute '{}' is owned by a kernel component \ + and cannot be restored from user mode; not preserved", + String::from_utf8_lossy(name) + ) + }) + .collect() + } + /// One canonical FILE_FULL_EA_INFORMATION list for NtSetEaFile. + pub fn restorable_buffer(&self) -> Vec { + let mut buffer = Vec::new(); + for (index, entry) in self.entries.iter().enumerate() { + let start = buffer.len(); + buffer.extend_from_slice(&0u32.to_le_bytes()); + buffer.push(entry.flags); + buffer.push(entry.name.len() as u8); + buffer.extend_from_slice(&(entry.value.len() as u16).to_le_bytes()); + buffer.extend_from_slice(&entry.name); + buffer.push(0); + buffer.extend_from_slice(&entry.value); + while buffer.len() % 4 != 0 { + buffer.push(0); + } + if index + 1 < self.entries.len() { + let size = (buffer.len() - start) as u32; + buffer[start..start + 4].copy_from_slice(&size.to_le_bytes()); + } + } + buffer + } +} + +/// Kernel EA names are matched case-insensitively: the kernel itself folds EA +/// names to uppercase, so either spelling must classify. +pub fn is_kernel_ea_name(name: &[u8]) -> bool { + name.len() >= KERNEL_EA_PREFIX.len() + && name[..KERNEL_EA_PREFIX.len()].eq_ignore_ascii_case(KERNEL_EA_PREFIX) +} + +/// Byte-exact restorable comparison. Order stays out of the contract because +/// EA query order is a kernel detail; names, flags and values are not. +pub fn same_restorable(a: &[EaEntry], b: &[EaEntry]) -> bool { + let mut a: Vec<_> = a.to_vec(); + let mut b: Vec<_> = b.to_vec(); + a.sort_by(|x, y| x.name.cmp(&y.name)); + b.sort_by(|x, y| x.name.cmp(&y.name)); + a == b +} + +/// Capture the full EA set of one file. The kernel reports the exact list +/// size first, so the scan budget is known before any buffer is read; an +/// oversized, malformed or unreadable set fails closed instead of being +/// discarded. +pub fn capture(file: &File) -> io::Result { + let mut size = 0u32; + let mut status = IoStatus { + status: 0, + information: 0, + }; + // SAFETY: FileEaInformation (7) writes one ULONG into size for a borrowed + // handle. + let result = unsafe { + super::NtQueryInformationFile( + file.as_raw_handle(), + &mut status, + (&mut size as *mut u32).cast(), + 4, + FILE_EA_INFORMATION, + ) + }; + if result < 0 { + return Err(nt_error(result)); + } + if size == 0 { + return Ok(CapturedEas::default()); + } + if size as usize > MAX_EA_BYTES { + return Err(unsupported( + "Windows extended-attribute metadata exceeds the bounded capture budget", + )); + } + let mut captured = CapturedEas::default(); + // The size probe bounds the whole list; the loop only resumes a partial + // buffer, and an entry larger than the budget fails closed at the call. + let mut buffer = vec![0u8; size as usize + 512]; + let mut restart = 1u8; + for _ in 0..64 { + let mut status = IoStatus { + status: 0, + information: 0, + }; + // SAFETY: borrowed handle; buffer and status outlive the call. + let result = unsafe { + NtQueryEaFile( + file.as_raw_handle(), + &mut status, + buffer.as_mut_ptr().cast(), + buffer.len() as u32, + 0, + null(), + 0, + null_mut(), + restart, + ) + }; + restart = 0; + if result == NO_MORE_EAS { + break; + } + if result < 0 && result != BUFFER_OVERFLOW { + return Err(nt_error(result)); + } + let used = status.information.min(buffer.len()); + append_entries(&buffer[..used], &mut captured)?; + if result >= 0 && used < buffer.len() { + break; + } + if used == 0 { + return Err(unsupported( + "Windows extended-attribute scan made no progress", + )); + } + } + Ok(captured) +} + +/// Parse one FILE_FULL_EA_INFORMATION list into the canonical capture. Every +/// length and offset is bounds-checked before use; a malformed record fails +/// closed rather than being partially accepted. +fn append_entries(bytes: &[u8], captured: &mut CapturedEas) -> io::Result<()> { + let mut at = 0usize; + loop { + if at + 8 > bytes.len() { + if at == bytes.len() { + return Ok(()); + } + return Err(unsupported("Truncated Windows extended-attribute record")); + } + let next = + u32::from_le_bytes([bytes[at], bytes[at + 1], bytes[at + 2], bytes[at + 3]]) as usize; + let flags = bytes[at + 4]; + let name_length = bytes[at + 5] as usize; + let value_length = u16::from_le_bytes([bytes[at + 6], bytes[at + 7]]) as usize; + let name_at = at + 8; + let value_at = name_at + name_length + 1; + if value_at + value_length > bytes.len() || bytes.get(name_at + name_length) != Some(&0) { + return Err(unsupported("Truncated Windows extended-attribute record")); + } + let name = &bytes[name_at..name_at + name_length]; + if name.is_empty() || name.iter().any(|b| !b.is_ascii()) { + return Err(unsupported("Invalid Windows extended-attribute name")); + } + if flags > 1 { + return Err(unsupported("Unsupported Windows extended-attribute flags")); + } + if is_kernel_ea_name(name) { + captured.kernel.push(name.to_vec()); + } else { + if captured.entries.len() >= MAX_EA_ENTRIES { + return Err(unsupported( + "Windows extended-attribute count exceeds the bounded capture budget", + )); + } + captured.entries.push(EaEntry { + flags, + name: name.to_vec(), + value: bytes[value_at..value_at + value_length].to_vec(), + }); + } + if next == 0 { + return Ok(()); + } + if next < 8 || at + next > bytes.len() { + return Err(unsupported( + "Invalid Windows extended-attribute record chain", + )); + } + at += next; + } +} + +/// Set the captured restorable EAs on one staged destination file. The +/// destination is always our own freshly created private sibling, so a kernel +/// refusal aborts publication instead of touching the source. +pub fn restore(file: &File, captured: &CapturedEas) -> io::Result<()> { + let buffer = captured.restorable_buffer(); + if buffer.is_empty() { + return Ok(()); + } + let mut status = IoStatus { + status: 0, + information: 0, + }; + // SAFETY: borrowed handle; the buffer outlives the call. + let result = unsafe { + NtSetEaFile( + file.as_raw_handle(), + &mut status, + buffer.as_ptr().cast(), + buffer.len() as u32, + ) + }; + if result < 0 { + return Err(nt_error(result)); + } + Ok(()) +} + +/// Re-query the destination and compare the restorable set byte-exactly. +/// Kernel entries on either side stay out of the comparison: their lifecycle +/// belongs to the kernel and is reported at capture time instead. +pub fn verify(file: &File, captured: &CapturedEas) -> io::Result<()> { + if captured.entries.is_empty() { + return Ok(()); + } + let actual = capture(file)?; + if !same_restorable(&captured.entries, &actual.entries) { + return Err(unsupported( + "Windows extended-attribute preservation verification failed", + )); + } + Ok(()) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn kernel_names_classify_and_degrade_with_named_evidence() { + assert!(is_kernel_ea_name(b"$KERNEL.PurgeEFSProxyCache")); + assert!(is_kernel_ea_name(b"$kernel.Custom")); + assert!(!is_kernel_ea_name(b"dsh-ops.test")); + assert!(!is_kernel_ea_name(b"$KERNEX.x")); + assert!(!is_kernel_ea_name(b"")); + let captured = CapturedEas { + entries: vec![EaEntry { + flags: 0, + name: b"DSH-OPS.TEST".to_vec(), + value: b"fixture".to_vec(), + }], + kernel: vec![b"$KERNEL.SIM".to_vec()], + }; + let warnings = captured.warnings(); + assert_eq!(warnings.len(), 1); + assert!(warnings[0].contains("$KERNEL.SIM"), "{}", warnings[0]); + // The restorable buffer carries only the user entry; kernel names + // never reach a set call. + let buffer = captured.restorable_buffer(); + assert!(buffer.windows(12).any(|w| w == b"DSH-OPS.TEST")); + assert!(!buffer.windows(10).any(|w| w == b"$KERNEL.SIM")); + } + + #[test] + fn restorable_comparison_is_byte_exact_and_strict() { + let entry = |name: &[u8], value: &[u8]| EaEntry { + flags: 0, + name: name.to_vec(), + value: value.to_vec(), + }; + let a = vec![entry(b"A.ONE", b"v1"), entry(b"B.TWO", b"v2")]; + let reordered = vec![entry(b"B.TWO", b"v2"), entry(b"A.ONE", b"v1")]; + assert!(same_restorable(&a, &reordered)); + assert!(!same_restorable(&a, &[entry(b"A.ONE", b"different")])); + assert!(!same_restorable(&a, &[entry(b"A.ONE", b"v1")])); + assert!(!same_restorable( + &a, + &[ + entry(b"A.ONE", b"v1"), + entry(b"B.TWO", b"v2"), + entry(b"C.NEW", b"v"), + ], + )); + let flagged = vec![EaEntry { + flags: 1, + name: b"A.ONE".to_vec(), + value: b"v1".to_vec(), + }]; + assert!(!same_restorable(&a[..1], &flagged)); + } + + #[test] + fn buffers_chain_entries_with_aligned_offsets_and_a_zero_tail() { + let captured = CapturedEas { + entries: vec![ + EaEntry { + flags: 0, + name: b"FIRST.X".to_vec(), + value: b"one".to_vec(), + }, + EaEntry { + flags: 1, + name: b"SECOND.Y".to_vec(), + value: b"two!".to_vec(), + }, + ], + kernel: Vec::new(), + }; + let buffer = captured.restorable_buffer(); + let first = u32::from_le_bytes([buffer[0], buffer[1], buffer[2], buffer[3]]) as usize; + assert_eq!(first % 4, 0); + assert!(first + 8 <= buffer.len()); + assert_eq!(buffer[4], 0); + assert_eq!(buffer[5], 7); + assert_eq!(buffer[first + 4], 1); + assert_eq!(buffer[first + 5], 8); + // The last entry terminates the chain with a zero NextEntryOffset. + assert_eq!(buffer[first..first + 4], [0, 0, 0, 0]); + } +} diff --git a/native/windows-metadata/src/lib.rs b/native/windows-metadata/src/lib.rs index 405bdb0e..8427e5bd 100644 --- a/native/windows-metadata/src/lib.rs +++ b/native/windows-metadata/src/lib.rs @@ -1,6 +1,7 @@ //! Windows-only FFI boundary. Callers supply owned/borrowed file handles; no //! model paths, shells, privilege enabling, or publication policy live here. #![cfg(windows)] +mod ea; mod inheritance; pub use inheritance::inherit_new_file_dacl; use std::{ @@ -318,6 +319,7 @@ pub fn verify_cache(file: &File, write_sid: Option<&str>) -> io::Result<()> { pub struct Metadata { sd: Option>, basic: FILE_BASIC_INFO, + eas: ea::CapturedEas, warnings: Vec, } impl Metadata { @@ -374,29 +376,15 @@ impl Metadata { return Err(unsupported("Unknown Windows stream metadata")); } } - if capabilities.extended_attributes() { - let mut ea_size = 0u32; - let mut status = IoStatus { - status: 0, - information: 0, - }; - // SAFETY: FileEaInformation (7) writes one ULONG into ea_size. - let result = unsafe { - NtQueryInformationFile( - file.as_raw_handle(), - &mut status, - addr_of_mut!(ea_size).cast(), - 4, - 7, - ) - }; - if result < 0 { - return Err(nt_error(result)); - } - if ea_size != 0 { - return Err(unsupported("Windows extended attributes are not supported")); - } - } + // #874: user EAs are captured for an exact restore instead of refusing + // the whole write; kernel-owned EAs degrade with named evidence. + let eas = if capabilities.extended_attributes() { + ea::capture(file)? + } else { + ea::CapturedEas::default() + }; + let mut warnings = capabilities.warnings(); + warnings.extend(eas.warnings()); Ok(Self { sd: if capabilities.persistent_acls() { Some(descriptor(file)?) @@ -404,7 +392,8 @@ impl Metadata { None }, basic, - warnings: capabilities.warnings(), + eas, + warnings, }) } pub fn warnings(&self) -> &[String] { @@ -461,6 +450,8 @@ impl Metadata { { return Err(io::Error::last_os_error()); } + // Restored before verification: the destination is our own staged file. + ea::restore(file, &self.eas)?; self.verify(file) } pub fn verify(&self, file: &File) -> io::Result<()> { @@ -495,6 +486,7 @@ impl Metadata { { return Err(unsupported("Windows basic metadata verification failed")); } + ea::verify(file, &self.eas)?; Ok(()) } } @@ -651,4 +643,160 @@ mod tests { assert!(!root.join("private").exists()); std::fs::remove_dir(root).unwrap(); } + + // Test-local fixture FFI: it only creates the EA set; the capture and + // restore code under test never calls it, and the signature matches the + // crate's own NtSetEaFile declaration exactly. + #[link(name = "ntdll")] + unsafe extern "system" { + fn NtSetEaFile( + handle: HANDLE, + status: *mut IoStatus, + buffer: *const c_void, + length: u32, + ) -> i32; + } + fn probe_error(result: i32) -> io::Error { + // SAFETY: total scalar conversion, no pointer arguments. + io::Error::from_raw_os_error(unsafe { RtlNtStatusToDosError(result) } as i32) + } + /// Build one FILE_FULL_EA_INFORMATION entry and set it on a file. + fn set_ea(path: &std::path::Path, name: &[u8], value: &[u8]) -> io::Result<()> { + let mut buffer = Vec::with_capacity(8 + name.len() + 1 + value.len()); + buffer.extend_from_slice(&0u32.to_le_bytes()); + buffer.push(0); + buffer.push(name.len() as u8); + buffer.extend_from_slice(&(value.len() as u16).to_le_bytes()); + buffer.extend_from_slice(name); + buffer.push(0); + buffer.extend_from_slice(value); + let file = std::fs::OpenOptions::new() + .read(true) + .write(true) + .open(path)?; + let mut status = IoStatus { + status: 0, + information: 0, + }; + // SAFETY: borrowed handle; buffer and status outlive the call. + let result = unsafe { + NtSetEaFile( + file.as_raw_handle(), + &mut status, + buffer.as_ptr().cast(), + buffer.len() as u32, + ) + }; + if result < 0 { + return Err(probe_error(result)); + } + Ok(()) + } + #[test] + fn user_extended_attributes_round_trip_instead_of_refusing_the_write() { + // #874: files carrying user EAs were all refused; they must round-trip. + let root = std::env::temp_dir().join(format!( + "dsh-metadata-ea-{}-{}", + std::process::id(), + SystemTime::now() + .duration_since(UNIX_EPOCH) + .unwrap() + .as_nanos() + )); + std::fs::create_dir(&root).unwrap(); + let path = root.join("tagged"); + std::fs::write(&path, b"payload").unwrap(); + set_ea(&path, b"dsh-ops.test", b"fixture") + .expect("this volume must accept user EAs for the fixture"); + // NTFS folds user EA names to uppercase when storing and querying them. + let observed = ea::capture(&File::open(&path).unwrap()).unwrap(); + assert_eq!(observed.entries.len(), 1); + assert_eq!(observed.entries[0].name, b"DSH-OPS.TEST".to_vec()); + assert_eq!(observed.entries[0].value, b"fixture".to_vec()); + assert!(observed.kernel.is_empty()); + // #874: capture must describe the EA set instead of refusing the file. + let file = File::open(&path).unwrap(); + assert!( + VolumeCapabilities::query(&file) + .unwrap() + .extended_attributes() + ); + let captured = Metadata::capture(&file).expect("user EAs must be captured, not rejected"); + let parent = std::fs::OpenOptions::new() + .read(true) + .custom_flags(0x02000000) + .open(&root) + .unwrap(); + let mut staging = create_private(&parent, "staged").unwrap(); + staging.write_all(b"payload").unwrap(); + captured.restore(&staging).unwrap(); + captured.verify(&staging).unwrap(); + drop(staging); + drop(file); + let restored = ea::capture(&File::open(root.join("staged")).unwrap()).unwrap(); + assert_eq!(restored.entries.len(), 1); + assert_eq!(restored.entries[0].name, b"DSH-OPS.TEST".to_vec()); + assert_eq!(restored.entries[0].value, b"fixture".to_vec()); + assert_eq!(std::fs::read(root.join("staged")).unwrap(), b"payload"); + drop(parent); + std::fs::remove_dir_all(&root).unwrap(); + } + + #[test] + fn kernel_named_eas_belong_to_the_kernel_not_user_mode() { + // The $KERNEL.* namespace is kernel-owned. A user-mode NtSetEaFile with + // such a name is either refused or silently dropped (observed here: + // the call succeeds and nothing is stored), so real kernel EAs on disk + // (EFS's PurgeEFSProxyCache, SmartLocker tags) can never be restored + // from user mode. Capture therefore classifies them by name and the + // publication reports each one; that classification and its warnings + // are asserted in the ea:: unit tests (#874). + let root = std::env::temp_dir().join(format!( + "dsh-metadata-kernel-ea-{}-{}", + std::process::id(), + SystemTime::now() + .duration_since(UNIX_EPOCH) + .unwrap() + .as_nanos() + )); + std::fs::create_dir(&root).unwrap(); + let path = root.join("kernel-tagged"); + std::fs::write(&path, b"payload").unwrap(); + let parent = std::fs::OpenOptions::new() + .read(true) + .custom_flags(0x02000000) + .open(&root) + .unwrap(); + if let Err(refusal) = set_ea(&path, b"$KERNEL.SimTag", b"v") { + // The kernel refused user-mode $KERNEL.* creation itself: that is + // the documented reason no restore path can preserve them either. + assert!(refusal.raw_os_error().is_some(), "{refusal}"); + drop(parent); + std::fs::remove_dir_all(&root).unwrap(); + return; + } + // On this build the user-mode set reports success while the kernel + // silently drops the reserved name: nothing exists to capture, and an + // empty EA set must not be refused either. + let observed = ea::capture(&File::open(&path).unwrap()).unwrap(); + assert!( + observed.entries.is_empty() && observed.kernel.is_empty(), + "the kernel must own the $KERNEL namespace: {observed:?}" + ); + let file = File::open(&path).unwrap(); + let captured = Metadata::capture(&file).expect("an empty EA set must not be refused"); + assert!(captured.warnings().is_empty()); + let mut staging = create_private(&parent, "kernel-staged").unwrap(); + staging.write_all(b"payload").unwrap(); + captured.restore(&staging).unwrap(); + captured.verify(&staging).unwrap(); + drop(staging); + drop(file); + assert_eq!( + std::fs::read(root.join("kernel-staged")).unwrap(), + b"payload" + ); + drop(parent); + std::fs::remove_dir_all(&root).unwrap(); + } } diff --git a/prompts/batching-guidance.md b/prompts/batching-guidance.md index ca7dae41..1e8b2385 100644 --- a/prompts/batching-guidance.md +++ b/prompts/batching-guidance.md @@ -16,7 +16,10 @@ Return enough status and completion facts for the next decision. ops_read_files or ops_inspect_local_file, edit with ops_edit_file. The shell starts processes and reads environment facts; node:fs and node:child_process bypass path policy, the sandbox and the audit trail, so they are never a workaround. -7. **Load before you call**: an unloaded ops_* name is loaded, never replaced. +7. **Load only missing APIs**: names already in the current SDK are callable; an + unloaded ops_* name is loaded, never replaced. +8. **Recheck the SDK after a mode change**: a mode or component switch can change + the pool; read the current SDK before planning the next call. ## Step 0 - the plan line @@ -32,7 +35,8 @@ must be the first line of the program, before code or blank lines. ## Step 1 - split by dependency Run independent calls under Promise.all. Await dependencies sequentially in the same -program. SDK loading requires a separate program before using the new bindings. +program. +SDK loading requires a separate program before using the new bindings. ## Step 2 - reconcile @@ -56,8 +60,7 @@ return { loaded: load.data?.tools, headLines: head.data?.items?.length } Calling an unloaded name throws "tools.ops_read_files is not a function". That is not a reason to fall back to the shell; it is one step spent loading. Loaded names remain -selected in this live scope; recheck the current SDK after a mode or component change. -Programs need not share a process or local variables. +selected in this live scope, and programs need not share a process or local variables. ## Trajectory 1 - reconnaissance: independent probes in one program @@ -106,10 +109,6 @@ return { } ``` -The search is the dependency, so it is awaited first. The five reads depend on its -result but not on each other, so they fan out inside the same program instead of -across five steps. - ## Trajectory 3 - a verified edit: revision first, patch second ```ts @@ -129,9 +128,6 @@ return { status: edit.status, error: "error" in edit ? edit.error : undefined, context: context.data?.items?.length, changed: edit.data?.changed } ``` -The revision and the context are independent, so they share one program. The write -needs the revision, so it is a second await in the same program, not a second step. - ## Trajectory 4 - paging: a loop is still one program ```ts @@ -154,27 +150,6 @@ return { lines: rows.length, complete, error, nextOffset: complete ? undefined : ``` Advance by nextOffset, never by page count, and stop when the result reports complete. -A loop of dispatches is batched work, not repeated single steps. - -## Anti-pattern, and its rewrite - -```ts -// plan: list the source files <- one entry, one dispatch, one round trip -const files = await tools.ops_glob({ pattern: ["src/**/*.ts"], outputMode: "paths" }) -return files.data?.items?.length -// ... then a second program for the directories, a third for the manifest -``` - -```ts -// plan: source files; directories; manifest head <- same work, one program -const [files, dirs, manifest] = await Promise.all([ - tools.ops_glob({ pattern: ["src/**/*.ts"], outputMode: "paths" }), - tools.ops_glob({ pattern: ["*"], kind: "directory", outputMode: "paths", limit: 40 }), - tools.ops_read_files({ files: [{ path: "package.json", limit: 30 }] }), -]) -return { files: files.data?.items?.length, dirs: dirs.data?.items?.length, - manifestLines: manifest.data?.items?.[0]?.lineCount } -``` ## Common mistakes diff --git a/scripts/verify-gate.mjs b/scripts/verify-gate.mjs index e71c3866..0eebea7c 100644 --- a/scripts/verify-gate.mjs +++ b/scripts/verify-gate.mjs @@ -72,7 +72,10 @@ const WORKER_TARGET = option('worker-target', undefined) */ const ALLOWED_SKIPS = TARGET === 'win32-x64' ? ['file-metadata', 'linux-attributes', 'linux-shell'] - : ['mount', 'result-storage-cache-prep', 'windows-security', 'niubash-consumer', 'niubash-security-host'] + : ['mount', 'result-storage-cache-prep', 'windows-security', 'niubash-consumer', 'niubash-security-host', + // #870/#409/#581/#771: the Windows Niubash runtime branch. These three suites pin a + // Windows-only code path and skip on every other host, exactly like windows-security. + 'shells-full-access-confinement', 'shells-msys-shared-section', 'shells-niubash-prepared-boundary'] /** * Suites that need an *optional external* checkout (`DSH_OPS_TUI_ROOT`, and for the native one also diff --git a/src/adapter/dsh/host-floor.ts b/src/adapter/dsh/host-floor.ts new file mode 100644 index 00000000..578d1561 --- /dev/null +++ b/src/adapter/dsh/host-floor.ts @@ -0,0 +1,97 @@ +/** + * The host support boundary this package declares: engines.dsh in package.json + * names the floor, and activation time is where an older host must hear it (#701). + * + * A host below the floor used to load anyway and fail later - TypeError scope + * errors backstage, EXECUTOR_UNAVAILABLE in front. The host's own behaviour + * while loading an unsupported plugin remains upstream; what this package owns + * is refusing, at activation, in one named diagnosis. A version the host will + * not reveal can never be proven below the floor, so 'unknown' loads as before. + */ +import fs from 'node:fs' +import type { Context } from './types.js' +import { runtimeVersion } from './runtime-version.js' + +/** One semver triple plus prerelease identifiers; build metadata never affects precedence. */ +interface SemVer { major: number; minor: number; patch: number; pre: string[] } + +const SEMVER = /^(0|[1-9]\d*)\.(0|[1-9]\d*)\.(0|[1-9]\d*)(?:-((?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)(?:\.(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*))?(?:\+[0-9a-zA-Z-]+)?$/u +const NUMERIC = /^[0-9]+$/u + +/** Parse one strict semver; anything else is not a version this check can judge. */ +function parseSemVer(value: string): SemVer | undefined { + const match = SEMVER.exec(value) + if (match === null) return undefined + return { major: Number(match[1]), minor: Number(match[2]), patch: Number(match[3]), pre: match[4] === undefined ? [] : match[4].split('.') } +} + +/** Whether left sorts strictly before right under semver precedence rules. */ +function precedes(left: SemVer, right: SemVer): boolean { + if (left.major !== right.major) return left.major < right.major + if (left.minor !== right.minor) return left.minor < right.minor + if (left.patch !== right.patch) return left.patch < right.patch + if (left.pre.length === 0) return false + if (right.pre.length === 0) return true + for (let index = 0; index < Math.max(left.pre.length, right.pre.length); index++) { + const a = left.pre[index], b = right.pre[index] + if (a === undefined) return true + if (b === undefined) return false + if (a === b) continue + if (NUMERIC.test(a) && NUMERIC.test(b)) return Number(a) < Number(b) + if (NUMERIC.test(a)) return true + if (NUMERIC.test(b)) return false + return a < b + } + return false +} + +/** + * The floor this package declares for its host, from the first lower bound of + * engines.dsh; undefined when the manifest names no floor, which disables the check. + * The manifest is read once from the package root beside this module's own tree, + * so the declared range stays the single source of truth, never a copy. + * @returns the declared lower bound, or undefined. + */ +export function declaredHostFloor(): string | undefined { + if (cache === undefined) { + cache = null + try { + const manifest = JSON.parse(fs.readFileSync(new URL('../../../package.json', import.meta.url), 'utf8')) as { engines?: { dsh?: unknown } } + const range = manifest.engines?.dsh + if (typeof range === 'string') cache = range + } catch { /* An unreadable manifest disables the floor rather than guessing one. */ } + } + if (cache === null) return undefined + const floor = />=\s*(\d+\.\d+\.\d+(?:-[0-9A-Za-z.-]+)?)/u.exec(cache) + return floor?.[1] +} + +/** One read of the declared engines.dsh range; null marks "no floor declared". */ +let cache: string | null | undefined + +/** + * Whether an actual host version sorts strictly below the declared floor. + * @param {string} actual - the host-reported version. + * @param {string} floor - the declared lower bound. + * @returns {boolean} true only when both parse and actual sorts below floor. + */ +export function versionBelowFloor(actual: string, floor: string): boolean { + const left = parseSemVer(actual), right = parseSemVer(floor) + return left !== undefined && right !== undefined && precedes(left, right) +} + +/** + * Refuse activation with one named diagnosis when the host is provably older + * than the declared floor; an unknown version is released, never punished. + * @param {Context} ctx - the host context the version probe reads. + * @returns {Promise} resolves when the host is at or above the floor. + */ +export async function assertHostFloor(ctx: Context): Promise { + const floor = declaredHostFloor() + if (floor === undefined) return + const { runtimeVersion: version } = await runtimeVersion(ctx) + if (version === 'unknown') return + if (versionBelowFloor(version, floor)) { + throw new Error(`This dsh-ops build requires DSH ${floor} or newer (package.json engines.dsh); the running host reports ${version}. Upgrade the host runtime instead of reloading this plugin: an older host loads it halfway at best, showing TypeError scope errors backstage and EXECUTOR_UNAVAILABLE in front.`) + } +} diff --git a/src/adapter/dsh/types.ts b/src/adapter/dsh/types.ts index c4f27944..de0c838a 100644 --- a/src/adapter/dsh/types.ts +++ b/src/adapter/dsh/types.ts @@ -9,7 +9,7 @@ export type { Context, Fiber } from '@deepseek-ai/cordis' export type { EntryOptions } from '@deepseek-ai/cordis-plugin-loader' export type { Agent, PreStepDecision } from '@deepseek-ai/dsh-agent' export type { FileSystem, FsTarget, FsInfo, FsObservation } from '@deepseek-ai/dsh-fs' -export type { GenerateOptions, StreamChunk, UserMessage } from '@deepseek-ai/dsh-llm' +export type { GenerateOptions, StreamChunk, Message, UserMessage } from '@deepseek-ai/dsh-llm' export type { Session } from '@deepseek-ai/dsh-session' export type { SettingsPathOpView } from '@deepseek-ai/dsh-settings/types' export type { AssembleContext, PromptAssembly, PromptPresentation, SystemPrompt } from '@deepseek-ai/dsh-system-prompt' diff --git a/src/client/locales.ts b/src/client/locales.ts index 145714c5..ae6bc9ab 100644 --- a/src/client/locales.ts +++ b/src/client/locales.ts @@ -90,6 +90,15 @@ export const en = { defaultPresetNoUpdate: 'Nothing to do: new sessions already start in OPS (PTC).', defaultPresetUnavailable: 'The OPS (PTC) preset is not registered right now, so the default preset was not changed.', defaultPresetFailed: 'The default preset was not changed:', + disclosureTitle: 'PTC disclosure tier', + disclosureHint: 'One choice for both PTC rows: when the full permitted tool declaration appears in a session. The distinction is the timing, never the permissions.', + disclosureOnDemand: 'On-demand (default)', + disclosureOnDemandHint: 'The first step publishes resident tools plus the loader; the full declaration appears after the loader call, two steps.', + disclosureFull: 'Full', + disclosureFullHint: 'The first step publishes the whole permitted tool pool with no loader. Choosing this never widens permissions.', + disclosureMixed: 'The saved PTC rows disagree about the tier. Choosing one below unifies them on save.', + disclosureSuspended: 'The dynamic-tools bundle switch is off, so the full tier is suspended and both rows run on-demand until it is back on; the saved choice is kept.', + disclosureAppliesOnSave: 'The choice applies to new sessions; running sessions keep their current surface.', } export const zh: Record = { ptcFilesRequired: '类型化 PTC 必须启用文件能力;用常驻开关控制启动 API。', @@ -182,6 +191,15 @@ export const zh: Record = { defaultPresetNoUpdate: '无需更改:新会话本来就默认使用 OPS (PTC)。', defaultPresetUnavailable: 'OPS (PTC) 预设当前未注册,默认预设没有改动。', defaultPresetFailed: '默认预设没有改动:', + disclosureTitle: 'PTC 披露档位', + disclosureHint: '两条 PTC 行统一选择:完整获准工具声明在会话中出现的时机。区别只在时机,从不改变权限。', + disclosureOnDemand: '按需(默认)', + disclosureOnDemandHint: '首步只发布常驻工具与加载入口;完整声明在加载调用之后出现,分两步。', + disclosureFull: '全量', + disclosureFullHint: '首步直接发布整个获准工具池,且无加载入口。选择全量不会扩大任何权限。', + disclosureMixed: '已保存的两条 PTC 行档位不一致。在下面选定一个,保存后会统一。', + disclosureSuspended: '动态工具总开关已关闭,全量档暂不生效,两行按需运行;已保存的选择原样保留,开关回开后重新生效。', + disclosureAppliesOnSave: '选择对新会话生效;正在运行的会话保持当前工具面。', } declare module '@deepseek-ai/dsh-client-ui-slots' { interface LocaleNamespaceMap { dshOps: keyof typeof en } } export type Translator = TranslateNS<'dshOps'> diff --git a/src/client/row-form.tsx b/src/client/row-form.tsx index 32365cba..4995af27 100644 --- a/src/client/row-form.tsx +++ b/src/client/row-form.tsx @@ -11,8 +11,8 @@ import type { ReactElement } from 'react' import type { RuntimeRemote, RuntimeSnapshot } from '../settings/runtime-contract.js' import { FormActions, Unavailable, errorText, useEditor } from './forms.js' import type { SettingsProps } from './forms.js' -import { groupsFor, bundleGates, changedFields, executorDiffers, executorOps, flatKey, readExecutor, readRowValues, type FieldSpec } from './row-schema.js' -import type { ExecutorChoice } from './row-schema.js' +import { groupsFor, bundleGates, changedFields, disclosureDiffers, disclosureOps, executorDiffers, executorOps, flatKey, readDisclosure, readExecutor, readRowValues, type FieldSpec } from './row-schema.js' +import type { DisclosureSelection, ExecutorChoice } from './row-schema.js' import type { Translator } from './locales.js' /** The subset of the native Switch this page binds. */ @@ -36,19 +36,24 @@ export function RowForm(props: SettingsProps & { Switch: SwitchBinding; pwsh7Ins const editor = useEditor(form, read, t) const [opened, setOpened] = useState>({}) const [choice, setChoice] = useState(() => readExecutor(form?.state.value ?? {})) + const [tier, setTier] = useState(() => readDisclosure(form?.state.value ?? {})) const [check, setCheck] = useState<{ ok: boolean; text: string }>() const [saving, setSaving] = useState(false) // The saved row is the baseline: a Host reload while the page is dirty must not - // silently adopt somebody else's executor. + // silently adopt somebody else's executor or disclosure tier. useEffect(() => { if (editor.dirty) return setChoice(readExecutor(editor.baseline)) + setTier(readDisclosure(editor.baseline)) setCheck(undefined) }, [editor.baseline, editor.dirty]) if (!editor.ready) return const draft = editor.draft, gates = bundleGates(draft) const savedChoice = readExecutor(editor.baseline) const executorDirty = executorDiffers(choice, editor.baseline) + const savedTier = readDisclosure(editor.baseline) + // `mixed` only describes saved state, so it is never a choice to write. + const tierDirty = tier !== 'mixed' && disclosureDiffers(tier, editor.baseline) const candidates = executor.snapshot?.git ?? [] /** Ask the Host what a path resolves to; nothing is written by this call. */ @@ -83,6 +88,9 @@ export function RowForm(props: SettingsProps & { Switch: SwitchBinding; pwsh7Ins } catch (error) { setCheck({ ok: false, text: errorText(error) }); return } finally { setSaving(false) } } + // The disclosure card writes exactly the reviewed tier, nothing else: an + // untouched old profile (key absent) produces no operation at all. + if (tierDirty) operations.push(...disclosureOps(editor.baseline, tier)) await editor.save(operations) } // The companion linkage locks every host-PowerShell hide switch to off. @@ -178,8 +186,37 @@ export function RowForm(props: SettingsProps & { Switch: SwitchBinding; pwsh7Ins

{t('executorAppliesOnSave')}

- { editor.reset(); setChoice(savedChoice); setCheck(undefined) }} message={editor.message} t={t} /> +
+
+

{t('disclosureTitle')}

+

{t('disclosureHint')}

+
+
+ {savedTier === 'mixed' &&

{t('disclosureMixed')}

} + {/* The bundle switch keeps its veto (sign-off item 2): the hint names the + suspension instead of hiding the choice, which still applies on save. */} + {!gates.dynamicTools &&

{t('disclosureSuspended')}

} + + +

{t('disclosureAppliesOnSave')}

+
+
+ { editor.reset(); setChoice(savedChoice); setTier(savedTier); setCheck(undefined) }} message={editor.message} t={t} /> } diff --git a/src/client/row-schema.ts b/src/client/row-schema.ts index 1001228b..3943c5be 100644 --- a/src/client/row-schema.ts +++ b/src/client/row-schema.ts @@ -247,6 +247,58 @@ function savedBashPaths(raw: Record): SavedBashPath[] { return list } +/** The mode sections that own a PTC publication surface and its disclosure tier. */ +export const DISCLOSURE_SECTIONS = Object.freeze(['ptc', 'ops-ptc'] as const) + +/** One disclosure tier as the page holds it, or \`mixed\` when the saved rows disagree. */ +export type DisclosureSelection = 'on-demand' | 'full' | 'mixed' + +/** The two tiers an operator can actually pick; \`mixed\` only describes saved state. */ +export type DisclosureTier = Exclude + +/** + * Read the disclosure tier the saved rows stand for. + * @param raw - the saved row config. + * @returns the agreed tier, or \`mixed\` when the two PTC rows disagree. + */ +export function readDisclosure(raw: Record): DisclosureSelection { + const tiers = new Set(DISCLOSURE_SECTIONS.map(section => + objectAt(raw[section]).disclosurePolicy === 'full' ? 'full' : 'on-demand')) + return tiers.size === 1 ? [...tiers][0]! : 'mixed' +} + +/** + * Whether the page's tier still differs from what is saved. + * @param selection - the operator's current choice. + * @param raw - the saved row config. + * @returns true when saving must write the tier. + */ +export function disclosureDiffers(selection: DisclosureSelection, raw: Record): boolean { + return selection !== readDisclosure(raw) +} + +/** + * The write operations that turn the saved rows into the reviewed tier. + * + * \`full\` pins both PTC rows; \`on-demand\` returns every row that carries the key + * to the absent default, so an old profile saved through the card is + * letter-identical to one that never met it. + * @param raw - the saved row config. + * @param tier - the operator's choice. + * @returns \`set\` operations for \`full\`, \`unset\` for the default rung. + */ +export function disclosureOps(raw: Record, tier: DisclosureTier): SettingsPathOpView[] { + const operations: SettingsPathOpView[] = [] + for (const section of DISCLOSURE_SECTIONS) { + const saved = objectAt(raw[section]).disclosurePolicy + if (tier === 'full' ? saved === 'full' : saved === undefined) continue + operations.push(tier === 'full' + ? { op: 'set', path: [section, 'disclosurePolicy'], value: 'full' } + : { op: 'unset', path: [section, 'disclosurePolicy'] }) + } + return operations +} + /** Whether a path names the Niubash executable rather than an MSYS bash. */ const niubashPath = (value: string): boolean => /(^|[\/])niu\.exe$/i.test(value) diff --git a/src/consumer/mount.ts b/src/consumer/mount.ts index a8d77fbe..9e288de2 100644 --- a/src/consumer/mount.ts +++ b/src/consumer/mount.ts @@ -17,8 +17,12 @@ import type { ConsumerPlan } from './types.js' export function mountConsumer(ctx: Context, plan: ConsumerPlan): void { const { gate, core } = plan + let ptc: ReturnType | undefined let renderShellGuidance: (agent: Agent) => string = () => '' const shellOwnsGuidance = (agent: Agent): boolean => Boolean(core.extraGuidance.trim()) && Boolean(renderShellGuidance(agent)) + // A shell slot whose executor never resolved must never hide the official tools it was + // going to replace (#962): the replacement would advertise a counterpart that cannot run. + let bashUnavailable = false if (plan.shell) { // Host subprocess spills outlive the call that produced them upstream, so the // row that publishes the shell also owns one bounded retention runtime. @@ -31,6 +35,7 @@ export function mountConsumer(ctx: Context, plan: ConsumerPlan): void { // Resolved once at mount: a slot whose executor never resolved publishes no tool at all, so the // section below has to report that instead of rendering nothing for the whole session. const bashResolution = shells.bash + bashUnavailable = !bashResolution.available ctx.inject(['tools'], inner => mountSessionBash(inner, core, shells, gate)) if (core.promptPolicy) ctx.inject(['tools', 'systemPrompt'], inner => { mountJobWaitGuidance(inner, plan.id, gate) @@ -60,6 +65,6 @@ export function mountConsumer(ctx: Context, plan: ConsumerPlan): void { { ...plan.native, enabled: plan.discovery }, { ...gate, guidance: core.promptPolicy, extraGuidance: core.extraGuidance, extraGuidanceOwner: agent => shellOwnsGuidance(agent) || nativeOwnsGuidance(agent), residentTools: () => residents.map(publicToolName), external: plan.external }) - } else if (plan.files) mountPtc(ctx, plan.ptc, { ...gate, guidance: core.promptPolicy, resident: plan.residentTools, dynamic: plan.discovery }) - if (plan.replaceOfficial) mountReplacements(ctx, { id: plan.id, targets: plan.replacementTargets, gate, keepOfficial: () => false }) + } else if (plan.files) ptc = mountPtc(ctx, plan.ptc, { ...gate, guidance: core.promptPolicy, resident: plan.residentTools, dynamic: plan.discovery }) + if (plan.replaceOfficial) mountReplacements(ctx, { id: plan.id, targets: plan.replacementTargets, gate, keepOfficial: () => bashUnavailable, ptc }) } \ No newline at end of file diff --git a/src/consumer/options.ts b/src/consumer/options.ts index c239712f..b557b4c7 100644 --- a/src/consumer/options.ts +++ b/src/consumer/options.ts @@ -4,7 +4,7 @@ import { resolveDiscoveryConfig, type DiscoveryConfig } from '../discovery/confi import { NATIVE_RESIDENT_RAW } from '../capabilities/profiles.js' import { NATIVE_FILE_TOOLS, type FileToolName } from '../capabilities/registry.js' import { migrateRawNames } from '../compat/alias.js' -import { parsePtcConfig, type PtcConfig } from '../ptc/config.js' +import { parsePtcConfig, type DisclosurePolicy, type PtcConfig } from '../ptc/config.js' export type IntegrationProfile = 'native' | 'ptc' | 'discovery' | 'shell' export interface IntegrationOptions { @@ -20,6 +20,7 @@ export interface IntegrationOptions { discovery?: boolean allowedTools?: string[] alwaysLoaded?: string[] + disclosurePolicy?: DisclosurePolicy replaceOfficial?: boolean promptPolicy?: boolean extraGuidance?: string @@ -50,7 +51,7 @@ export interface IntegrationConfig { externalDisabled: string[] ptc: PtcConfig } -const keys = ['preset', 'enabled', 'profile', 'files', 'residentTools', 'residentNames', 'hidePwsh', 'shell', 'presentation', 'discovery', 'allowedTools', 'alwaysLoaded', +const keys = ['preset', 'enabled', 'profile', 'files', 'residentTools', 'residentNames', 'hidePwsh', 'shell', 'presentation', 'discovery', 'allowedTools', 'alwaysLoaded', 'disclosurePolicy', 'replaceOfficial', 'promptPolicy', 'extraGuidance', 'binaryPath', 'required', 'denyCredentialReads', 'bashPath', 'allowSystemShellFallback', 'externalTools', 'externalAlwaysLoaded', 'externalDisabled'] const profiles: Record = { @@ -90,13 +91,17 @@ export function resolveIntegrationConfig(raw: unknown): IntegrationConfig { const replaceOfficial = boolean('replaceOfficial', false) const hidePwsh = boolean('hidePwsh', false) && shell if (input.residentNames !== undefined && presentation === 'ptc') throw new ConfigError('residentNames applies to Native presentation; use alwaysLoaded for PTC APIs') + // D4 (#988): the custom path accepts the same explicit tier key the builtin rows + // parse (D2/#986); an omitted key keeps the legacy on-demand default. Native + // presentation rows reject it, mirroring the builtin non-PTC rows. + if (input.disclosurePolicy !== undefined && presentation !== 'ptc') throw new ConfigError('disclosurePolicy applies to PTC presentation rows only') // Replacement intent is retained while both publishers are off; no counterpart means no hide. const allowedTools = input.allowedTools ?? (enabled ? undefined : []) const alwaysLoaded = input.alwaysLoaded ?? [] const native = presentation === 'native' ? resolveDiscoveryConfig({ enabled, allowedTools, alwaysLoaded, binaryPath: core.binaryPath, required: core.required, denyCredentialReads: core.denyCredentialReads, }) : resolveDiscoveryConfig({ enabled: false, allowedTools: [], alwaysLoaded: [] }) - const ptc = parsePtcConfig(presentation === 'ptc' ? { allowedTools, alwaysLoaded, binaryPath: core.binaryPath, required: core.required } : {}) + const ptc = parsePtcConfig(presentation === 'ptc' ? { allowedTools, alwaysLoaded, binaryPath: core.binaryPath, required: core.required, disclosurePolicy: input.disclosurePolicy } : {}) if (presentation === 'ptc' && ptc.alwaysLoaded.some(tool => !ptc.allowedTools.includes(tool))) throw new ConfigError('alwaysLoaded must be in allowedTools') const externalNames = (key: string): string[] => { const value = input[key] ?? [] diff --git a/src/discovery/controller.ts b/src/discovery/controller.ts index e83af2ce..8b97a571 100644 --- a/src/discovery/controller.ts +++ b/src/discovery/controller.ts @@ -7,6 +7,7 @@ import { activationIsPtc, isNativePresentation, isPresentationProbe } from '../p import { visibleNames } from '../publication/surface.js' import { mountResultLifecycle } from '../kernel/result-lifecycle.js' import { failure, resolveWorker } from '../execution/files.js' +import type { Outcome } from '../kernel/outcome.js' import { capability, DISCOVERY_TOOL_NAME } from '../capabilities/registry.js' import { DISCOVERY_DESCRIPTION, DISCOVERY_PARAMETERS, DISCOVERY_OUTPUT } from '../capabilities/discovery.js' import { publicToolName, renderToolingPolicy } from '../policy.js' @@ -93,6 +94,23 @@ export function mountDiscovery(root: Context, raw: unknown, gate: DiscoveryGate) const api = apiByName.get(name) return api !== undefined && selectable(agent, api) && (config.alwaysLoaded.includes(api) || entry.selected.has(api)) } + /** Point a declared-but-unselected capability at the selector; undefined when discovery cannot select it (#764). */ + const deferredRejection = (agent: Agent, entry: Entry, name: string, provider?: Provider): Outcome | undefined => { + if (!selected(agent, entry, DISCOVER)) return undefined + if (provider !== undefined) { + if (!externalPool(agent).includes(provider) || !externalAvailable(provider, agent) + || gate.external?.alwaysLoaded.includes(name) || provider.capability.disclosure === 'resident' + || entry.externalSelected.has(provider.token)) return undefined + } else { + const api = apiByName.get(name) + if (api === undefined || !selectable(agent, api) || config.alwaysLoaded.includes(api) || entry.selected.has(api)) return undefined + } + return { status: 'error', error: { + code: 'CAPABILITY_NOT_SELECTED', effectState: 'none', retry: 'never', + message: `Capability '${name}' is discoverable for this scope but is not selected yet; no action was executed.`, + nextAction: `Call ${DISCOVER} with a select: query naming it (for example query: "select:${name}"); the tool becomes available on the next model step.`, + } } + } const rotateFiles = (agent: Agent, entry: Entry): void => { for (const [api, file] of entry.files) { const entitled = selected(agent, entry, publicToolName(api)) @@ -103,6 +121,7 @@ export function mountDiscovery(root: Context, raw: unknown, gate: DiscoveryGate) dispatch: async () => undefined, present: async activation => activation.agent === agent && entries.get(agent) === entry && permitted(agent, api) && !await activationIsPtc(prompt, activation), + deferred: () => deferredRejection(agent, entry, publicToolName(api)), }) file.entitled = entitled } @@ -113,6 +132,9 @@ export function mountDiscovery(root: Context, raw: unknown, gate: DiscoveryGate) const current = () => !exec.signal.aborted && entries.get(agent) === entry && exec.agent === agent && selected(agent, entry, source.name, provider) && tools.executable(source.name, agent, exec.toolSet) === definition if (!current() || !await isNativePresentation(prompt, agent, exec.signal).catch(() => false) || !current()) { + // A registered-but-unselected capability points at the selector; every other admission failure keeps the generic refusal. + const deferred = deferredRejection(agent, entry, source.name, provider) + if (deferred !== undefined) return deferred return failure('POLICY_DENIED', 'This Native capability must be admitted for the calling scope; no action executed.') } if (!provider) return source.execute(args, exec) diff --git a/src/execution/wsl-worker.ts b/src/execution/wsl-worker.ts index 59f0cc65..c1db0a37 100644 --- a/src/execution/wsl-worker.ts +++ b/src/execution/wsl-worker.ts @@ -56,7 +56,29 @@ export async function tryWslRead(subprocess: SubprocessRuntime, file: string, en if (remaining() === 0) { controller.abort(); combined.throwIfAborted() } if (!sameAuthority()) { authorityLost = true; throw new Error('Host authority changed') } } - const probe = async (argv: string[]): Promise => { + // One fixed /bin/sh -c preparation call (#576): the script text is a constant + // and every data value — one drive root per argument, the payload path last — + // travels as a positional parameter, so no Windows path can alter the command + // shape. The script maps each drive through wslpath, prints winlinux + // pairs, and ends with sha256sum over the mapped payload; a refused mapping + // exits 9 and a refused digest 8, so a failed preparation never masquerades + // as partial output. + const prepareScript = [ + 'i=0', + 'for a in "$@"', + 'do', + ' i=$((i+1))', + ' m=$(/usr/bin/wslpath -u "$a") || exit 9', + ' if [ "$i" -eq "$#" ]', + ' then', + ' /usr/bin/sha256sum -- "$m" || exit 8', + ' else', + " printf '%s\\t%s\\n' \"$a\" \"$m\"", + ' fi', + 'done', + 'exit 0', + ].join('\n') + const run = async (argv: string[]): Promise => { guard() const handle = subprocess.spawn({ argv: [...prefix, ...argv], cwd, stdio: { stdin: { data: '' }, stdout: { maxBytes: 8192 }, stderr: { maxBytes: 4096 } }, graceMs: 1000, signal: combined }) const done = await handle.done, output = handle.collected.stdout?.readFrom(0) @@ -65,14 +87,21 @@ export async function tryWslRead(subprocess: SubprocessRuntime, file: string, en } try { // Mount mappings are live authority, not a cache keyed only by a drive letter. + const driveRoots = [...new Set(paths.map(value => windowsDrive(String(value))).filter((value): value is string => Boolean(value)))] + const merged = await run(['/bin/sh', '-c', prepareScript, 'dsh-ops', ...driveRoots.map(drive => `${drive}\\`), payload]) + const lines = merged.split('\n').filter(line => line.length > 0) + if (lines.length !== driveRoots.length + 1) throw new Error('WSL drive mapping unavailable') const drives = new Map() - for (const drive of new Set(paths.map(value => windowsDrive(String(value))).filter((value): value is string => Boolean(value)))) { - const mapped = await probe(['/usr/bin/wslpath', '-u', `${drive}\\`]) - if (!mapped.startsWith('/') || mapped.includes('\n') || mapped.includes('\0')) throw new Error('WSL drive mapping unavailable') + for (const line of lines.slice(0, -1)) { + const at = line.indexOf('\t') + if (at <= 0) throw new Error('WSL drive mapping unavailable') + const drive = windowsDrive(line.slice(0, at)), mapped = line.slice(at + 1) + if (!drive || !mapped.startsWith('/') || mapped.includes('\n') || mapped.includes('\0')) throw new Error('WSL drive mapping unavailable') drives.set(drive, mapped) } + if (drives.size !== driveRoots.length) throw new Error('WSL drive mapping unavailable') const worker = linuxPath(payload, root.distro, drives) - if (!worker || !(await probe(['/usr/bin/sha256sum', '--', worker])).startsWith(`${digest} `)) throw new Error('WSL payload mismatch') + if (!worker || !lines[lines.length - 1].startsWith(`${digest} `)) throw new Error('WSL payload mismatch') const convert = (value: string): string => { const mapped = linuxPath(value, root.distro, drives) if (!mapped) throw new Error('WSL authority mapping unavailable') diff --git a/src/index.ts b/src/index.ts index cb7dec06..a66b05b5 100644 --- a/src/index.ts +++ b/src/index.ts @@ -5,7 +5,7 @@ import { resolveModeConfig } from './presets/config.js' import { resolveMergedConfig, effectiveMode, dynamicForMode } from './settings/merged.js' import { MODE_IDS } from './presets/types.js' import { beginMergedRow, endMergedRow } from './settings/legacy.js' -import { mountIntegration } from './integration/index.js' +import { mountIntegration, mountModeDiagnostics } from './integration/index.js' import { gateConsumer } from './consumer/gates.js' import { resolveIntegrationConfig } from './consumer/options.js' import { mountRuntimeSettings } from './settings/runtime-host.js' @@ -15,6 +15,7 @@ import * as tuiModule from './components/tui/index.js' import * as downloadGuardModule from './components/download-guard/index.js' import { configureResultCacheGc } from './kernel/result-cache-gc.js' import { registerOpsPtcPreset } from './presets/ops-ptc/register.js' +import { assertHostFloor } from './adapter/dsh/host-floor.js' export { Config } from './settings/merged.js' export { applyComponent } from './legacy/host.js' export const name = 'dsh-ops' @@ -31,6 +32,11 @@ export const name = 'dsh-ops' export async function apply(root: Context, rawConfig: unknown): Promise { const config = resolveMergedConfig(rawConfig) + // Activation-time host support boundary (#701): an older host hears one named + // diagnosis here instead of loading halfway and failing with TypeError scope + // errors backstage and EXECUTOR_UNAVAILABLE in front. An unknown host version + // cannot be proven below the floor, so it loads as before. + await assertHostFloor(root) const host = root const mount = async (module: { apply(...args: unknown[]): unknown }, state: { enabled: boolean; raw?: Record; config: unknown }, slice?: Record): Promise => { if (!state.enabled) return @@ -59,10 +65,13 @@ export async function apply(root: Context, rawConfig: unknown): Promise { for (const mode of MODE_IDS) { const state = config[mode] const effective = resolveModeConfig(effectiveMode(state, config.global, typeof config.other.raw?.pwsh7Link === 'boolean' ? config.other.raw.pwsh7Link : companionVersion({ sweep: false }) !== undefined), mode) - await host.plugin({ name: `dsh-ops/consumer-${mode}`, apply(ctx: Context) { + await host.plugin({ name: `dsh-ops/consumer-${mode}`, async apply(ctx: Context) { applyResolvedMode(ctx, effective, mode, { enabled: state.enabled, discovery: { ...config.discovery.config, enabled: config.discovery.enabled }, dynamic: dynamicForMode(state, config.global) }) + // Built-in modes answer ops_status too (#753): the named diagnosis used + // to exist only behind an external preset row. + await mountModeDiagnostics(ctx, mode) } }) } for (const row of Object.values(config.presets)) { diff --git a/src/integration/diagnostics.ts b/src/integration/diagnostics.ts index c09ead57..ffa184b8 100644 --- a/src/integration/diagnostics.ts +++ b/src/integration/diagnostics.ts @@ -8,6 +8,7 @@ import { pendingSelection } from '../discovery/observations.js' import { providers } from '../discovery/providers.js' import { integrationFor } from '../consumer/registry.js' import { publicToolName } from '../policy.js' +import { disclosurePlan } from '../ptc/disclosure.js' import { ownScopes } from '../publication/ownership.js' import type { PublicationGate } from '../publication/types.js' import { presetOf } from '../consumer/identity.js' @@ -61,9 +62,16 @@ export async function diagnose(ctx: Context, agent: Agent): Promise if (config.presentation === 'native' && ptc) issue('PRESENTATION_MISMATCH', 'Native tools are withdrawn in PTC-only presentation. Choose the typed PTC integration or restore Native presentation.') } catch { issue('PRESENTATION_UNAVAILABLE', 'The host could not assemble this scope. Check the host presentation/runtime configuration.') } if (config) { + // D4 (#988): PTC expectations come from the same disclosure plan the + // publication derives (mountConsumer passes residentTools/discovery as the + // publication gates), so a selectable-but-unselected API — `result` on the + // on-demand tier — is never a gap, while a genuinely missing first-round + // name still is. Native expectations keep the resolved row contract. + const ptc = config.presentation === 'ptc' && config.files + ? disclosurePlan(config.ptc, config.residentTools, config.discovery) : undefined const expected = [ - ...(config.files && config.residentTools ? (config.presentation === 'ptc' ? ['inspect_local_file', 'result', ...config.ptc.alwaysLoaded] : config.residentNames) : []), - ...(config.presentation === 'ptc' && config.discovery ? ['load_sdk'] : []), + ...(config.files && config.residentTools && !ptc ? config.residentNames : []), + ...(ptc ? [...ptc.firstRound, ...(ptc.loader ? ['load_sdk' as const] : [])] : []), ...(config.shell ? ['bash'] : []), ...(config.presentation === 'native' && config.native.enabled ? ['discover_tools'] : []), ...(config.presentation === 'native' ? config.native.alwaysLoaded : []), diff --git a/src/integration/index.ts b/src/integration/index.ts index 49e3ec12..6c8d61b9 100644 --- a/src/integration/index.ts +++ b/src/integration/index.ts @@ -1,10 +1,13 @@ /** Stable opt-in preset entry. Consumers share the same assembler as built-in presets. */ -import type { Context } from '../adapter/dsh/types.js' +import type { Agent, Context } from '../adapter/dsh/types.js' import { mountConsumer } from '../consumer/mount.js' import { replacementTargets } from '../consumer/replacements.js' import { resolveIntegrationConfig, type IntegrationOptions, type IntegrationConfig } from '../consumer/options.js' import { mountDiagnostics } from './diagnostics.js' -import { joinIntegration } from '../consumer/registry.js' +import { joinIntegration, integrationFor } from '../consumer/registry.js' +import { modeOf } from '../presets/detect.js' +import { onModesChange } from '../presets/registry.js' +import type { ModeId } from '../presets/types.js' export { diagnose } from './diagnostics.js' export type { Diagnostic } from './diagnostics.js' export { registerNativeCapability } from '../discovery/providers.js' @@ -28,11 +31,34 @@ export async function mountIntegration(ctx: Context, config: IntegrationConfig): presentation: config.presentation, residentTools: config.residentTools, residentNames: config.residentNames, discovery: config.discovery, native: config.native, ptc: config.ptc, replaceOfficial: config.replaceOfficial || config.hidePwsh, - replacementTargets: replacementTargets(config.replaceOfficial && config.files && config.presentation === 'native', config.hidePwsh && config.shell), + // Both host presentations are covered: a native consumer keeps the old path, + // and a PTC consumer resolves reachability through the publisher accessor. + replacementTargets: replacementTargets(config.replaceOfficial && config.files, config.hidePwsh && config.shell), section: `preset-${config.preset}`, external: { allowed: config.externalTools, alwaysLoaded: config.externalAlwaysLoaded, disabled: config.externalDisabled }, }) } +/** + * One built-in mode publishes the same read-only diagnosis (#753): the status + * tool used to exist only behind an external preset row, so a built-in preset + * session could not see its own scope at all. The gate admits exactly the + * sessions of this mode without an external preset - the same row the mode's + * consumer is admitted by - so the two publications never overlap. + * + * The mount settles asynchronously on purpose: the listener this registers + * inside the tools injection has to be on a fiber that resolves after the + * synchronous apply returns, or the later `agent/created` dispatch never + * reaches it - the external preset row reaches the same point by awaiting its + * binding join, which is why only the built-in loop needed this one step. + */ +export async function mountModeDiagnostics(ctx: Context, mode: ModeId): Promise { + await Promise.resolve() + mountDiagnostics(ctx, { + allows: (agent: Agent) => integrationFor(ctx, agent) === undefined && modeOf(ctx, agent) === mode, + watch: (notify: () => void) => onModesChange(ctx, notify), + }) +} + /** Mount inside a preset composition or with an explicit preset id at the host root. */ export function integration(options: IntegrationOptions) { const captured = structuredClone(options) diff --git a/src/kernel/outcome.ts b/src/kernel/outcome.ts index b4c5f811..11893c2f 100644 --- a/src/kernel/outcome.ts +++ b/src/kernel/outcome.ts @@ -18,7 +18,7 @@ export interface CoverageMeta { } export type Outcome = | { status: 'completed', data: T, coverage?: CoverageMeta & { isComplete: true } } - | { status: 'partial', data: T, coverage: CoverageMeta & { isComplete: false }, continuation: string } + | { status: 'partial', data: T, coverage: CoverageMeta & { isComplete: false }, continuation: string, nextAction?: string } | { status: FailureStatus, error: OpsErrorPayload, data?: T } interface SearchReportBase { diff --git a/src/legacy/host.ts b/src/legacy/host.ts index cb26fb68..f197e03e 100644 --- a/src/legacy/host.ts +++ b/src/legacy/host.ts @@ -1,6 +1,7 @@ /** Retained pre-single-row component behavior; never part of the current composition root. */ import type { Context } from '../adapter/dsh/types.js' import type { Agent } from '../adapter/dsh/types.js' +import type { ResolvedToolSet } from '../adapter/dsh/types.js' import type {} from '../adapter/dsh/types.js' import type {} from '../adapter/dsh/types.js' import { resolveConfig } from '../config.js' @@ -31,13 +32,15 @@ function report(ctx: Context, message: string, level = 'error'): void { else if (level === 'warn') console.warn(message) else console.error(message) } -function visibleTools(ctx: Context, agent?: Agent): string[] { +function visibleTools(ctx: Context, agent?: Agent, toolSet?: ResolvedToolSet): string[] { try { const tools = ctx.get('tools') // Names only, through the view: `schemas()` projected every visible definition — // 34.2 ms for 258 tools at the published runtime's surface shape — to keep a - // handful of owned names, and this section renders once per assembly. - return tools ? visibleNames(tools, agent) : [] + // handful of owned names, and this section renders once per assembly. The toolSet + // read is the honest one: a host that ignores `when` in its definition reads would + // otherwise advertise a registered-but-unadmitted tool the step excludes (#934). + return tools ? visibleNames(tools, agent, toolSet) : [] } catch { return [] } } @@ -119,7 +122,7 @@ export function applyComponent(root: Context, rawConfig: unknown, component: Com order: SECTION_ORDERS[TOOLING_SECTION] + (component === 'shell' ? 1 : 0), interpolate: false, text: context => renderToolingPolicy({ - published: visibleTools(ctx, context.agent).filter(name => owned.includes(name)), + published: visibleTools(ctx, context.agent ?? context.scope as Agent, context.toolSet).filter(name => owned.includes(name)), shellComponent: component === 'shell', extraGuidance: config.extraGuidance, }), diff --git a/src/niubash/faces.ts b/src/niubash/faces.ts index 413338b9..e637f467 100644 --- a/src/niubash/faces.ts +++ b/src/niubash/faces.ts @@ -1,8 +1,11 @@ -/** Exact fallback command family from official Niubash v1.5.0 - * winuxcmd/usr/bin/activate-winuxcmd.sh (compiled inventory verifies that script). - * argv0 dispatch is upstream's own activation contract; bash/sh keep their original Niubash shims. +/** Command family the frozen Niubash winuxcmd actually implements (#966). + * + * Derived from upstream's activate-winuxcmd.sh, minus the applets the frozen + * multi-call binary answers with "command not found" (measured: jq, link): a + * generated argv0 stub for those would only shadow the system command with an + * entry that cannot run it. bash/sh keep their original Niubash shims. */ -export const NIUBASH_FACES: readonly string[] = Object.freeze(["arch", "b2sum", "base32", "base64", "basename", "basenc", "cal", "cat", "chcon", "chgrp", "chmod", "chown", "chroot", "cksum", "clear", "cmp", "col", "column", "comm", "cp", "cpio", "csplit", "cut", "cygpath", "d2u", "date", "dd", "df", "diff", "diff3", "dirname", "dir", "dircolors", "dos2unix", "du", "echo", "env", "expand", "expr", "factor", "false", "file", "find", "fmt", "fold", "free", "getconf", "grep", "groups", "head", "hexdump", "hmac256", "hostid", "hostname", "id", "infocmp", "install", "join", "jq", "kill", "less", "link", "ln", "locale", "logname", "ls", "lsof", "man", "md5sum", "mkdir", "mkfifo", "mknod", "mktemp", "more", "mpicalc", "mv", "nice", "nl", "nohup", "nproc", "numfmt", "od", "paste", "patch", "pathchk", "pinky", "pr", "printenv", "printf", "ps", "ptx", "pwd", "readlink", "realpath", "reset", "rev", "rm", "rmdir", "runcon", "sdiff", "sed", "seq", "sha1sum", "sha224sum", "sha256sum", "sha384sum", "sha512sum", "shred", "shuf", "sleep", "sort", "split", "stat", "stdbuf", "strings", "stty", "sum", "sync", "tac", "tail", "tee", "test", "[", "tic", "timeout", "toe", "top", "touch", "tput", "tr", "tree", "true", "truncate", "tsort", "tty", "tzset", "u2d", "uname", "unexpand", "uniq", "unix2dos", "unlink", "uptime", "users", "vdir", "watch", "wc", "which", "who", "whoami", "xargs", "xxd", "yes"]) +export const NIUBASH_FACES: readonly string[] = Object.freeze(["arch", "b2sum", "base32", "base64", "basename", "basenc", "cal", "cat", "chcon", "chgrp", "chmod", "chown", "chroot", "cksum", "clear", "cmp", "col", "column", "comm", "cp", "cpio", "csplit", "cut", "cygpath", "d2u", "date", "dd", "df", "diff", "diff3", "dirname", "dir", "dircolors", "dos2unix", "du", "echo", "env", "expand", "expr", "factor", "false", "file", "find", "fmt", "fold", "free", "getconf", "grep", "groups", "head", "hexdump", "hmac256", "hostid", "hostname", "id", "infocmp", "install", "join", "kill", "less", "ln", "locale", "logname", "ls", "lsof", "man", "md5sum", "mkdir", "mkfifo", "mknod", "mktemp", "more", "mpicalc", "mv", "nice", "nl", "nohup", "nproc", "numfmt", "od", "paste", "patch", "pathchk", "pinky", "pr", "printenv", "printf", "ps", "ptx", "pwd", "readlink", "realpath", "reset", "rev", "rm", "rmdir", "runcon", "sdiff", "sed", "seq", "sha1sum", "sha224sum", "sha256sum", "sha384sum", "sha512sum", "shred", "shuf", "sleep", "sort", "split", "stat", "stdbuf", "strings", "stty", "sum", "sync", "tac", "tail", "tee", "test", "[", "tic", "timeout", "toe", "top", "touch", "tput", "tr", "tree", "true", "truncate", "tsort", "tty", "tzset", "u2d", "uname", "unexpand", "uniq", "unix2dos", "unlink", "uptime", "users", "vdir", "watch", "wc", "which", "who", "whoami", "xargs", "xxd", "yes"]) /** The multi-call binary every face dispatches to. */ export const NIUBASH_WINUXCMD = 'winuxcmd/usr/bin/winuxcmd.exe' /** Payload-relative path of one argv0 face. */ diff --git a/src/niubash/runtime.ts b/src/niubash/runtime.ts index 7e8a96e7..704770fd 100644 --- a/src/niubash/runtime.ts +++ b/src/niubash/runtime.ts @@ -42,6 +42,16 @@ const WINUXCMD_SIZE: number = (NIUBASH_PIN.sizes as Readonly { /** Command-time runtime for a selected payload. Host workspace labeling must never reach an * executable through the workspace or a junction into it, so the prepared root is checked first. + * Full access runs without confinement and keeps the account's own authority over every path + * (#870); the refusal guards exactly the confined modes that label and gate the workspace. */ export async function prepareNiubash(selected: string, inheritedPath = process.env.PATH ?? process.env.Path ?? '', signal: AbortSignal, - { workspaceRoot, env = process.env }: { workspaceRoot?: string; env?: NodeJS.ProcessEnv } = {}): Promise { + { workspaceRoot, mode, env = process.env }: { workspaceRoot?: string; mode?: 'read-only' | 'workspace-write' | 'danger-full-access'; env?: NodeJS.ProcessEnv } = {}): Promise { signal.throwIfAborted() if (!path.isAbsolute(selected) || !isNiubashExecutable(selected)) throw unavailable('expected an absolute niu.exe path') const source = path.dirname(selected), target = niubashPreparedRoot(env) - if (workspaceRoot && (inside(target, path.resolve(workspaceRoot)) || inside(await physicalPath(target), await physicalPath(workspaceRoot)))) { + if (workspaceRoot && mode !== 'danger-full-access' + && (inside(target, path.resolve(workspaceRoot)) || inside(await physicalPath(target), await physicalPath(workspaceRoot)))) { throw unavailable('the prepared payload must be outside the session workspace; move DSH_HOME outside the workspace') } let file: string diff --git a/src/presets/builtin.ts b/src/presets/builtin.ts index 3070f868..137ba309 100644 --- a/src/presets/builtin.ts +++ b/src/presets/builtin.ts @@ -12,7 +12,8 @@ import { joinMode, onModesChange } from './registry.js' import type { ModeId } from './types.js' export function replacementTargets(mode: ModeId, platform: string = process.platform, hidePwsh = false): ReadonlyMap { - return consumerTargets(MODE_PLANS[mode].nativeFiles, MODE_PLANS[mode].shell && hidePwsh, platform as NodeJS.Platform) + const plan = MODE_PLANS[mode] + return consumerTargets(plan.nativeFiles || plan.ptc, plan.shell && hidePwsh, platform as NodeJS.Platform) } export function applyMode(root: Context, rawConfig: unknown, mode: ModeId): void { applyResolvedMode(root, resolveModeConfig(rawConfig, mode), mode) @@ -51,9 +52,9 @@ export function applyResolvedMode(root: Context, config: ModeConfig, mode: ModeI shell: enabled && plan.shell && config.bash && process.platform === 'win32', presentation: plan.ptc ? 'ptc' : 'native', residentTools: enabled && config.residentTools, discovery: plan.ptc ? config.dynamicTools : options.discovery !== undefined && options.dynamic !== false && discovery.enabled, - native: discovery, ptc: { binaryPath: config.binaryPath, required: config.required, allowedTools: config.allowedTools, alwaysLoaded: config.alwaysLoaded }, + native: discovery, ptc: { binaryPath: config.binaryPath, required: config.required, allowedTools: config.allowedTools, alwaysLoaded: config.alwaysLoaded, disclosurePolicy: config.disclosurePolicy }, replaceOfficial: enabled && (config.hideOfficialTools || config.hidePwsh), - replacementTargets: consumerTargets(config.hideOfficialTools && plan.nativeFiles, config.hidePwsh && config.bash && plan.shell), + replacementTargets: consumerTargets(config.hideOfficialTools && (plan.nativeFiles || plan.ptc), config.hidePwsh && config.bash && plan.shell), section: plan.section === null ? null : `mode-${plan.section}`, external: { allowed: [], alwaysLoaded: [], disabled: [] }, }) diff --git a/src/presets/config.ts b/src/presets/config.ts index 5fca3fa0..927097b6 100644 --- a/src/presets/config.ts +++ b/src/presets/config.ts @@ -6,7 +6,7 @@ * and a malformed row still fails loud and names the offending key. */ import { LEGACY_FASTCTX_KEYS, resolveConfig, type ResolvedConfig } from '../config.js' -import { parsePtcConfig, type PtcConfig } from '../ptc/config.js' +import { parsePtcConfig, type DisclosurePolicy, type PtcConfig } from '../ptc/config.js' import { editable, settingsConfig, type Field } from '../settings/schema.js' import { configObject } from '../settings/schema.js' import type { ModeId } from './types.js' @@ -15,6 +15,8 @@ import type { ModeId } from './types.js' export interface ModeConfig extends ResolvedConfig { allowedTools: PtcConfig['allowedTools'] alwaysLoaded: PtcConfig['alwaysLoaded'] + /** Explicit first-round disclosure tier; the default 'on-demand' is the legacy path (#986). */ + disclosurePolicy: DisclosurePolicy /** Publish Bash for this preset's sessions. */ bash: boolean /** Hide the official Windows shell by default while Bash is available. */ @@ -37,12 +39,14 @@ export const MODE_CONFIG_KEYS: readonly string[] = Object.freeze([ 'binaryPath', 'required', 'denyCredentialReads', 'promptPolicy', 'extraGuidance', 'bashPath', 'publishBashTool', 'allowSystemShellFallback', 'configUpdateVersion', 'spillRetentionDays', 'spillTotalBudgetMiB', 'spillSweepEnabled', - 'allowedTools', 'alwaysLoaded', + 'allowedTools', 'alwaysLoaded', 'disclosurePolicy', 'bash', 'hidePwsh', 'hideOfficialTools', 'dynamicTools', 'residentTools', 'batchingGuidance', 'batchingTelemetry', ]) -const PTC_KEYS = new Set(['allowedTools', 'alwaysLoaded']) +const PTC_KEYS = new Set(['allowedTools', 'alwaysLoaded', 'disclosurePolicy']) +/** The mode rows that own a PTC publication surface, hence the disclosure policy. */ +const PTC_POLICY_MODES: readonly ModeId[] = ['ptc', 'ops-ptc'] /** Boolean switches and gates, each supplying its own default below. */ const SWITCH_KEYS = new Set(['bash', 'hidePwsh', 'hideOfficialTools', 'dynamicTools', 'residentTools', 'batchingGuidance', 'batchingTelemetry']) @@ -63,6 +67,11 @@ export function resolveModeConfig(rawConfig?: unknown, mode?: ModeId): ModeConfi throw new Error(`unknown key(s) ${JSON.stringify(key)}; accepted keys are ${MODE_CONFIG_KEYS.map(entry => JSON.stringify(entry)).join(', ')}`) } } + // The disclosure policy names a PTC publication tier: a row without a PTC + // surface must fail loud instead of silently carrying the key. + if (Object.hasOwn(raw, 'disclosurePolicy') && mode !== undefined && !PTC_POLICY_MODES.includes(mode)) { + throw new Error('disclosurePolicy applies to the ptc and ops-ptc rows only') + } const core = Object.fromEntries(Object.entries(raw).filter(([key]) => !PTC_KEYS.has(key) && !SWITCH_KEYS.has(key))) const ptc = Object.fromEntries(Object.entries(raw).filter(([key]) => PTC_KEYS.has(key))) const switches = Object.fromEntries(Object.entries(raw).filter(([key]) => SWITCH_KEYS.has(key))) @@ -76,6 +85,7 @@ export function resolveModeConfig(rawConfig?: unknown, mode?: ModeId): ModeConfi const resolved = resolveConfig({ ...core, publishBashTool: bash }) const api = parsePtcConfig(ptc) return { ...resolved, allowedTools: api.allowedTools, alwaysLoaded: api.alwaysLoaded, + disclosurePolicy: api.disclosurePolicy ?? 'on-demand', bash, hidePwsh: sw('hidePwsh', true), hideOfficialTools: sw('hideOfficialTools', true), dynamicTools: sw('dynamicTools', true), residentTools: mode !== 'minimal' && sw('residentTools', true), diff --git a/src/presets/ops-ptc/register.ts b/src/presets/ops-ptc/register.ts index fe7dba5a..65952369 100644 --- a/src/presets/ops-ptc/register.ts +++ b/src/presets/ops-ptc/register.ts @@ -155,10 +155,14 @@ function currentDefault(registry: PresetRegistry): string | undefined { * * This is the same mechanism the settings page uses, so the write lands in the * profile layer and hot-applies to the live registry. + * @param ctx - the context whose settings service persists the value. + * @param value - the default preset id to write. + * @param captured - a settings service the caller resolved while it was live; + * the unload lease passes one because its own context may already be gone. * @returns the failure message, or undefined when the host accepted the write. */ -async function writeSelectedDefault(ctx: Context, value: string): Promise { - const settings = settingsOf(ctx) +async function writeSelectedDefault(ctx: Context, value: string, captured?: SettingsService): Promise { + const settings = captured ?? settingsOf(ctx) if (typeof settings?.update !== 'function') { return 'the host settings service is unavailable, so the default preset was not changed' } @@ -189,10 +193,17 @@ export interface OpsPtcPreset { applyDefault(): Promise } -/** The live registration the action writes through; empty when none is mounted. */ +/** + * The live registration the action writes through; empty when none is mounted. + * + * `settings` is captured while the injected child is alive: the unload lease + * writes through it after the child is gone (#928), when the child context can + * no longer resolve the service. + */ interface LiveRegistration { ctx?: Context registry?: PresetRegistry + settings?: SettingsService } /** @@ -238,6 +249,50 @@ function releaseRegistration(ctx: Context, slot: Slot, options: RegistrationOpti } } +/** + * The host's own default preset, when it is a usable id distinct from this + * row's preset. + * @param registry - the registry whose config declares the host default. + * @returns the host default id, or undefined when there is none to fall back to. + */ +function hostDefaultOf(registry: PresetRegistry): string | undefined { + try { + const value = registry.config?.default + return typeof value === 'string' && value !== '' && value !== PRESET_ID ? value : undefined + } catch { return undefined } +} + +/** + * Restore the profile default this row is about to strand. + * + * Unload and disable remove the ops-ptc preset, but the profile keeps the + * `selectedDefault` the operator's one click wrote: every new session then + * fails to compose against a preset that no longer exists (#928). The lease + * writes the host default back — only when the live value still names this + * row's preset; a default naming any other preset is an operator choice and is + * never touched. Best effort with a warning: a host without a usable settings + * service, or one that refuses the write, reports the leftover and its + * remediation instead of guessing. + * @param served - the last registration this row served. + * @param options - test seams; production passes none. + */ +async function restoreLeftoverDefault(served: LiveRegistration, options: RegistrationOptions): Promise { + const { ctx, registry } = served + if (ctx === undefined || registry === undefined) return + if (currentDefault(registry) !== PRESET_ID) return + const fallback = hostDefaultOf(registry) + if (fallback === undefined) { + warn(options, ctx, 'dsh-ops: the profile still names the removed ops-ptc preset as its default; choose another default on the host settings page.') + return + } + const failure = await writeSelectedDefault(ctx, fallback, served.settings) + if (failure !== undefined) { + warn(options, ctx, `dsh-ops: ${failure}; the profile still names the removed ops-ptc preset, so new sessions may fail to compose until another default is chosen.`) + return + } + warn(options, ctx, `dsh-ops: restored the default agent preset to ${fallback}, because the ops-ptc preset was removed with this row.`) +} + /** * Resolve one package from a require anchor. * @param anchor - the absolute module path resolution starts from. @@ -415,6 +470,12 @@ export async function readOpsPtcDefinition(options: RegistrationOptions, root: C */ export function registerOpsPtcPreset(root: Context, options: RegistrationOptions = {}): OpsPtcPreset { const live: LiveRegistration = {} + /** + * The last registration this row served: the injected child clears `live` + * when it goes away, while this holder stays readable for the unload lease, + * and a remount overwrites it with the fresh child's registration. + */ + const served: LiveRegistration = {} root.inject(['agentPresets'], ctx => { const registry = service(ctx, 'agentPresets') if (registry === undefined || typeof registry.register !== 'function') return @@ -429,10 +490,15 @@ export function registerOpsPtcPreset(root: Context, options: RegistrationOptions if (live.ctx === ctx) { live.ctx = undefined; live.registry = undefined } releaseRegistration(ctx, slot, options) }, 'dsh-ops: ops-ptc preset') - void activate(ctx, registry, slot, options, live).catch(error => { + void activate(ctx, registry, slot, options, live, served).catch(error => { warn(options, ctx, `dsh-ops: cannot register the ops-ptc agent preset: ${errorText(error)}; the preset stays unavailable.`) }) }) + // The lease below is the row's own lifetime: the Loader reuses a row's fiber + // across a config remount, so a remount never fires it. It fires when the row + // is disabled or unloaded — the one moment the profile can be left naming a + // preset that no longer exists (#928). + root.effect(() => () => { void restoreLeftoverDefault(served, options) }, 'dsh-ops: ops-ptc preset default lease') return { applyDefault: () => applyDefaultPreset(live, options) } } @@ -442,9 +508,11 @@ export function registerOpsPtcPreset(root: Context, options: RegistrationOptions * @param registry - the host preset registry. * @param slot - release holder the fiber cleanup reads. * @param options - test seams; production passes none. + * @param live - the holder the operator action reads; the child clears it on teardown. + * @param served - the holder the unload lease reads; it survives the child teardown. * @returns resolves once the registration is owned by {@link slot}. */ -async function activate(ctx: Context, registry: PresetRegistry, slot: Slot, options: RegistrationOptions, live: LiveRegistration): Promise { +async function activate(ctx: Context, registry: PresetRegistry, slot: Slot, options: RegistrationOptions, live: LiveRegistration, served: LiveRegistration): Promise { const definition = await readOpsPtcDefinition(options, ctx) if (definition === undefined || slot.disposed) return // A failed registration leaves the default preset exactly as it was: this row @@ -458,4 +526,8 @@ async function activate(ctx: Context, registry: PresetRegistry, slot: Slot, opti // The operator action reads this: only a registered preset may be made the default. live.ctx = ctx live.registry = registry + // The unload lease reads this after the injected child is gone (#928). + served.ctx = ctx + served.registry = registry + served.settings = settingsOf(ctx) } diff --git a/src/projections/canonical-output.ts b/src/projections/canonical-output.ts index 3d5f2db1..d4c5aff8 100644 --- a/src/projections/canonical-output.ts +++ b/src/projections/canonical-output.ts @@ -3,8 +3,12 @@ import type { ToolDefinition, ToolExecutionResult } from '../adapter/dsh/types.j import type { Outcome } from '../kernel/outcome.js' import { toOutcome } from './ptc/outcome.js' import type { CanonicalData } from './ptc/outcome.js' +import type { NextStepSurface } from './ptc/next-step.js' import { outcomeSchema } from './ptc/schema.js' +/** Resolves the calling step's reachable surface at dispatch time. */ +type SurfaceFor = (exec: Parameters[1]) => NextStepSurface + /** No raw receipt exists for data-less failures; replay the serializable error facts instead. */ function nativeValue(value: unknown): unknown { const outcome = value as Outcome @@ -18,11 +22,11 @@ function nativeResult(result: Readonly): Readonly toOutcome(tool, args, await definition.execute(args, exec)), + execute: async (args, exec) => toOutcome(tool, args, await definition.execute(args, exec), surfaceFor?.(exec)), // Hooks belong to the raw Native projection; preserve execution identity and // content while leaving the host's final canonical value untouched. ...(definition.projectContent ? { diff --git a/src/projections/native/ranges.ts b/src/projections/native/ranges.ts index 3342fee6..fcf8793c 100644 --- a/src/projections/native/ranges.ts +++ b/src/projections/native/ranges.ts @@ -46,6 +46,10 @@ export function compactRanges(input: Row, value: Row): string[] | undefined { if (!item.lines.length) output.push(item.total === 0 ? '(empty file)' : '(no lines in this range)') output.push(`[revision=${item.revision}; ${item.complete ? 'end of file; complete=true' : `more lines; nextOffset=${item.next}`}]`) } + // The compact reader states the escape contract once for the whole result (#939). + const escaped = observed.reduce( + (count, item) => count + item.lines.filter(line => encodeSourceLine(line.text) !== line.text).length, 0) + if (escaped) output.push(`[Notice: ${escaped} rendered line(s) use reversible escapes; ops_edit_file matches oldText as raw bytes first, then as this displayed form decoded.]`) output.push(`[Notice: ${observed.length}/${input.files.length} requested ranges returned.]`) return output } diff --git a/src/projections/native/render.ts b/src/projections/native/render.ts index 3ba68ba5..99030dc0 100644 --- a/src/projections/native/render.ts +++ b/src/projections/native/render.ts @@ -89,6 +89,10 @@ function inspect(args: Row, value: Row): string[] { } if (!group.length) output.push(value.lineCount === 0 ? '(empty file)' : '(no lines in this range)') } + // Escaped display is a contract the model must be able to round-trip: say so + // once per result, exactly when it matters (#939). + const escaped = items.filter(item => typeof item.text === 'string' && contentLine(item.text) !== item.text) + if (escaped.length) output.push(`[Notice: ${escaped.length} rendered line(s) use reversible escapes; ops_edit_file matches oldText as raw bytes first, then as this displayed form decoded.]`) output.push(...notices(value, items.length, textTotal !== undefined && value.total === textTotal ? ['total'] : [])) return output } @@ -112,7 +116,7 @@ const mutations = new Set(FILE_MUTATIONS) function receiptItems(value: Row): string[] { return rows(value.items).map(item => `${label(item.path ?? '(path unavailable)')}: ${facts(item, [ 'status', 'removed', 'changed', 'revision', 'revisionBefore', 'revisionAfter', 'bytes', 'bytesBefore', 'bytesAfter', - 'lineCountBefore', 'lineCount', 'replacements', 'metadata', 'metadataWarnings', 'error', + 'lineCountBefore', 'lineCount', 'replacements', 'renderedMatches', 'metadata', 'metadataWarnings', 'error', ])}`) } function receipts(tool: string, value: Row): string[] { diff --git a/src/projections/native/runtime.ts b/src/projections/native/runtime.ts index 5bde2ebd..44e99b0b 100644 --- a/src/projections/native/runtime.ts +++ b/src/projections/native/runtime.ts @@ -13,6 +13,8 @@ import { invoke, failure } from '../../execution/files.js' import { InvocationLifetime, InvocationNotStarted } from '../../lifecycle/invocations.js' import { canonicalOutput } from '../canonical-output.js' import { canonicalOutput as canonicalSchema } from '../ptc/canonical-schema.js' +import type { NextStepSurface } from '../ptc/next-step.js' +import { visibleNames } from '../../publication/surface.js' import { acquireResultAdmission } from '../../publication/result-admission.js' import { createNativeReadDedup } from './read-dedup.js' @@ -31,6 +33,13 @@ export class NativeFileTools { private stopped: boolean private readonly readDedup: ReturnType private readonly resultAdmission: ReturnType + /** The step-reachable surface at dispatch time: exactly what the host admitted. */ + private readonly surfaceOf = (exec: Execution): NextStepSurface => { + const tools = this.ctx.get('tools') + const policy = this.ctx.get('sandboxPolicy')?.resolve({ session: exec.agent?.session }) + return { callable: new Set(tools !== undefined && exec.agent !== undefined && exec.toolSet !== undefined + ? visibleNames(tools, exec.agent, exec.toolSet) : []), readOnly: policy?.mode === 'read-only' } + } private unregisterProvider: (() => void) | undefined constructor({ ctx, config, report }: { ctx: Context; config: NativeRuntimeConfig; report?: Reporter }) { this.ctx = ctx @@ -65,7 +74,7 @@ export class NativeFileTools { const execute: ToolDefinition['execute'] = (args, exec) => this.call(rawName, args, exec) const source = this.readDedup.wrap(rawName === 'result' ? resultDefinition(execute) : definition(rawName, execute)) const typed = rawName === 'glob' || rawName === 'grep' || rawName === 'read_files' - return canonicalOutput(rawName, typed ? { ...source, output: { ...source.output, schema: canonicalSchema(rawName) } } : source) + return canonicalOutput(rawName, typed ? { ...source, output: { ...source.output, schema: canonicalSchema(rawName) } } : source, this.surfaceOf) } async call(tool: NativeToolName, args: unknown, exec: Execution): Promise { diff --git a/src/projections/ptc/contract.ts b/src/projections/ptc/contract.ts index cc102e1f..cab55c00 100644 --- a/src/projections/ptc/contract.ts +++ b/src/projections/ptc/contract.ts @@ -16,7 +16,7 @@ export function definition(tool: FileToolName | PtcToolName, execute: ToolDefini if (!entry || !('parameters' in entry)) throw new Error(`Missing PTC capability: ${tool}`) const description = entry.description.replace(/ Failures resolve as .*?they are not thrown\./, '') return { name: publicToolName(tool), description: description + - ' PTC returns Outcome: completed.data; partial.data, coverage and continuation; failures use error.code/effectState/retry. Check status before reading data; never blindly retry a possible write.', + ' PTC returns Outcome: completed.data; partial.data, coverage, continuation and nextAction; failures use error.code/effectState/retry/nextAction. Follow nextAction for the concrete executable call; check status before reading data; never blindly retry a possible write.', parameters: entry.parameters, timeoutMs: WORKER_TIMEOUT_MS + 2000, ...(entry.concurrencySafe ? { isConcurrencySafe: () => true } : {}), output: { schema: outcomeSchema(canonicalOutput(tool)), render: (_args, value) => [{ type: 'text', text: JSON.stringify(value) }] }, diff --git a/src/projections/ptc/next-step.ts b/src/projections/ptc/next-step.ts new file mode 100644 index 00000000..b095201c --- /dev/null +++ b/src/projections/ptc/next-step.ts @@ -0,0 +1,72 @@ +/** Executable next-step recommendations; they describe calls, never replay authority. */ +import type { RetryPolicy } from '../../kernel/outcome.js' +import { publicToolName } from '../../policy.js' +import { decodeContinuation } from './continuation.js' +import type { CanonicalData } from './outcome.js' + +/** What the calling step can execute now, resolved by the dispatching runtime. */ +export interface NextStepSurface { + /** Public tool names the calling step admits. */ + readonly callable: ReadonlySet + /** True when the calling scope's sandbox policy forbids every mutation. */ + readonly readOnly: boolean +} + +const record = (value: unknown): CanonicalData | undefined => + value !== null && typeof value === 'object' && !Array.isArray(value) ? value as CanonicalData : undefined +const call = (rawName: string, args: CanonicalData): string => publicToolName(rawName) + ' ' + JSON.stringify(args) +const admitted = (surface: NextStepSurface | undefined, rawName: string): boolean => + surface !== undefined && surface.callable.has(publicToolName(rawName)) + +/** Render the safe continuation token as an explicit, parameter-legal call. */ +export function continueReading(continuation: string): string | undefined { + const decoded = decodeContinuation(continuation) + return decoded !== undefined + ? `Continue this read with ${call(decoded.tool, decoded.args)}; a page that reports complete=true is the last one.` + : undefined +} + +function directoryStep(path: string, surface: NextStepSurface | undefined): string { + if (admitted(surface, 'glob')) { + return `The path is a directory, not a file. List its immediate entries with ${call('glob', { path, pattern: ['*'] })}, then read one concrete file path.` + } + return 'The path is a directory, not a file, and no directory-listing API is callable in this step; supply one concrete file path.' +} + +function encodingStep(tool: string, path: string, surface: NextStepSurface | undefined): string { + // The calling tool just executed, so the same tool stays callable; a cross-tool + // recommendation additionally requires an admitted surface entry. + if (tool === 'inspect_local_file' || admitted(surface, 'inspect_local_file')) { + return `The bytes are not valid UTF-8. Read a hex window with ${call('inspect_local_file', { path, view: 'hex' })}.` + } + return `File ${JSON.stringify(path)} is not valid UTF-8; no hex-capable reader is callable in this step.` +} + +function revisionStep(path: string, surface: NextStepSurface | undefined): string { + const reader = admitted(surface, 'stat') ? 'stat' : admitted(surface, 'inspect_local_file') ? 'inspect_local_file' : undefined + const resubmit = surface?.readOnly === true ? '' : ', then re-submit with that expectedRevision' + return reader !== undefined + ? `The file changed since the revision you observed; nothing was applied. Read the current revision with ${call(reader, { path })}${resubmit}.` + : `The file changed since the revision you observed; nothing was applied. Read the current revision of ${JSON.stringify(path)} with a callable read API${resubmit}.` +} + +/** The generic guard for argument-rejected calls that name no specific recovery. */ +export const CHANGE_FIRST = 'Change the request arguments as the error message describes; an identical request fails identically.' + +/** Recommend the concrete call a recoverable failure admits; other guidance stays with the worker. */ +export function nextStepFor(tool: string, rawArgs: unknown, error: { code: string, message: string, retry: RetryPolicy }, + data: CanonicalData, surface: NextStepSurface | undefined): string | undefined { + const args = record(rawArgs) ?? {} + const path = typeof args.path === 'string' ? args.path : undefined + if ((error.code === 'INVALID_ARGUMENT' || error.code === 'IS_DIRECTORY') && error.message.includes('is a directory')) { + return path !== undefined ? directoryStep(path, surface) : undefined + } + if (error.code === 'ERR_INVALID_ENCODING' && path !== undefined) return encodingStep(tool, path, surface) + if (error.code === 'RESULT_PARTIAL' && Array.isArray(data.items)) { + const failed = data.items.map(record).find(item => record(item?.error)?.code === 'ERR_INVALID_ENCODING') + const failedPath = failed !== undefined && typeof failed.path === 'string' ? failed.path : undefined + if (failedPath !== undefined) return encodingStep(tool, failedPath, surface) + } + if (error.code === 'REVISION_CONFLICT' && path !== undefined) return revisionStep(path, surface) + return undefined +} diff --git a/src/projections/ptc/outcome.ts b/src/projections/ptc/outcome.ts index bec60334..30d66a94 100644 --- a/src/projections/ptc/outcome.ts +++ b/src/projections/ptc/outcome.ts @@ -2,6 +2,8 @@ import { errorPolicy } from '../../kernel/errors.js' import type { CoverageMeta, EffectState, Outcome } from '../../kernel/outcome.js' import { continuationFor } from './continuation.js' +import { CHANGE_FIRST, continueReading, nextStepFor } from './next-step.js' +import type { NextStepSurface } from './next-step.js' export type JsonValue = null | boolean | number | string | JsonValue[] | { [key: string]: JsonValue } export type CanonicalData = { [key: string]: JsonValue } @@ -14,10 +16,12 @@ function effect(value: CanonicalData, fallback: EffectState): EffectState { return typeof value.effectState === 'string' && effects.has(value.effectState as EffectState) ? value.effectState as EffectState : fallback } -function failure(data: CanonicalData | undefined, code: string, message: string, state: EffectState, nextAction?: string): Outcome { +function failure(data: CanonicalData | undefined, code: string, message: string, state: EffectState, + nextAction?: string, recommended?: string): Outcome { const policy = errorPolicy(code, state) + const next = recommended ?? nextAction return { status: policy.status, error: { code, message, effectState: state, retry: policy.retry, - ...(nextAction ? { nextAction } : {}) }, ...(data ? { data } : {}) } + ...(next !== undefined ? { nextAction: next } : {}) }, ...(data ? { data } : {}) } } function coverage(data: CanonicalData, complete: boolean): CoverageMeta { const cause = data.truncatedBy === 'time' ? 'time-limit' : data.truncatedBy @@ -27,13 +31,20 @@ function coverage(data: CanonicalData, complete: boolean): CoverageMeta { isComplete: complete, ...(truncatedBy ? { truncatedBy } : {}) } } /** Incomplete observations without a safe cursor are errors with retained data, never fabricated pages. */ -export function toOutcome(tool: string, args: unknown, value: unknown): Outcome { +export function toOutcome(tool: string, args: unknown, value: unknown, surface?: NextStepSurface): Outcome { const data = record(value) if (!data) return failure(undefined, 'RESULT_UNKNOWN', 'The worker did not return a canonical object.', 'unknown') const error = record(data.error) - if (error) return failure(data, typeof error.code === 'string' ? error.code : 'RESULT_UNKNOWN', - typeof error.message === 'string' ? error.message : 'The operation did not return a settled receipt.', effect(error, 'unknown'), - typeof error.nextAction === 'string' ? error.nextAction : undefined) + if (error) { + const code = typeof error.code === 'string' ? error.code : 'RESULT_UNKNOWN' + const message = typeof error.message === 'string' ? error.message : 'The operation did not return a settled receipt.' + const state = effect(error, 'unknown') + const workerNext = typeof error.nextAction === 'string' ? error.nextAction : undefined + const retry = errorPolicy(code, state).retry + const recommended = nextStepFor(tool, args, { code, message, retry }, data, surface) + ?? (workerNext === undefined && retry === 'after-input-change' ? CHANGE_FIRST : undefined) + return failure(data, code, message, state, workerNext, recommended) + } const action = record(args)?.action if (tool === 'result' && action === 'release' && data.status === 'released' && typeof data.id === 'string') { return { status: 'completed', data } @@ -64,13 +75,21 @@ export function toOutcome(tool: string, args: unknown, value: unknown): Outcome< } // Stored rows are not echoed by put/release; their own successful operation is complete. if (tool === 'result' && record(args)?.action !== 'page' && data.status === 'completed') return { status: 'completed', data } - if (count(data.failed) > 0) return failure(data, 'RESULT_PARTIAL', 'Some items failed; inspect each retained receipt.', - effect(data, count(data.changed) > 0 ? 'partial' : 'none'), next) + if (count(data.failed) > 0) { + const recommended = nextStepFor(tool, args, { code: 'RESULT_PARTIAL', message: 'Some items failed; inspect each retained receipt.', + retry: 'never' }, data, surface) + return failure(data, 'RESULT_PARTIAL', 'Some items failed; inspect each retained receipt.', + effect(data, count(data.changed) > 0 ? 'partial' : 'none'), next, recommended) + } const incomplete = data.complete === false || data.truncated === true || data.scanComplete === false || data.pageComplete === false || data.status === 'partial' || data.status === 'partial_completion' if (incomplete) { const continuation = continuationFor(tool, args, data) - if (continuation) return { status: 'partial', data, coverage: { ...coverage(data, false), isComplete: false }, continuation } + if (continuation) { + const recommended = continueReading(continuation) + return { status: 'partial', data, coverage: { ...coverage(data, false), isComplete: false }, continuation, + ...(recommended !== undefined ? { nextAction: recommended } : {}) } + } return failure(data, 'RESULT_PARTIAL', 'Observation is incomplete and has no safe continuation; refine the request.', effect(data, count(data.changed) > 0 ? 'applied' : 'none'), next) } diff --git a/src/projections/ptc/schema.ts b/src/projections/ptc/schema.ts index 2f0e9a82..58c61af9 100644 --- a/src/projections/ptc/schema.ts +++ b/src/projections/ptc/schema.ts @@ -16,7 +16,7 @@ export function outcomeSchema(data: JsonSchemaNode): JsonSchemaNode { const failureData: JsonSchemaNode = { ...data, required: [], properties: { ...data.properties, error: { type: 'object' } } } return { oneOf: [ object({ status: { type: 'string', const: 'completed' }, data, coverage: coverage(true) }, ['status', 'data']), - object({ status: { type: 'string', const: 'partial' }, data, coverage: coverage(false), continuation: string }, ['status', 'data', 'coverage', 'continuation']), + object({ status: { type: 'string', const: 'partial' }, data, coverage: coverage(false), continuation: string, nextAction: string }, ['status', 'data', 'coverage', 'continuation']), object({ status: { type: 'string', enum: ['rejected', 'conflict', 'error', 'unknown'] }, error: ERROR_SCHEMA, data: failureData }, ['status', 'error']), ] } } diff --git a/src/prompt-injection/index.ts b/src/prompt-injection/index.ts index 79215ae4..51cdf77d 100644 --- a/src/prompt-injection/index.ts +++ b/src/prompt-injection/index.ts @@ -1,8 +1,8 @@ -import type { Agent, Context, PreStepDecision, UserMessage } from '../adapter/dsh/types.js' +import type { Agent, Context, Message, PreStepDecision, UserMessage } from '../adapter/dsh/types.js' import { loadUserMessage } from '../adapter/dsh/host.js' import { PromptProviderError } from './types.js' -import type { PromptContribution, PromptDisposer, StepTailContext, StepTailProviderOptions, SystemSectionOptions } from './types.js' -export type { PromptContribution, PromptDisposer, StepTailContext, StepTailProviderOptions, SystemSectionOptions } from './types.js' +import type { PromptContribution, PromptDisposer, StepTailContext, StepTailProviderOptions, SystemSectionOptions, VisibleContribution } from './types.js' +export type { PromptContribution, PromptDisposer, StepTailContext, StepTailProviderOptions, SystemSectionOptions, VisibleContribution } from './types.js' export { PromptProviderError } from './types.js' interface TailEntry { options: StepTailProviderOptions; active: boolean; lifetime: AbortController } @@ -20,6 +20,21 @@ function stateFor(agent: Agent): AgentState { function validate(options: PromptContribution): void { if (!options.id.trim() || !options.revision.trim()) throw new TypeError('Prompt id and revision must be nonempty strings.') if (!Number.isFinite(options.priority ?? 0)) throw new TypeError('Prompt priority must be finite.') + if ((options as StepTailProviderOptions).dedupe !== undefined && (options as StepTailProviderOptions).dedupe !== 'visible') { + throw new TypeError('Step-tail dedupe must be "visible".') + } +} +/** Read one contribution's visibility from the derived model-visible history. */ +function readVisibility(history: readonly Message[], id: string, revision: string): VisibleContribution { + let any = false, current = false + for (const message of history) { + if (message.role !== 'user') continue + const source = message.source + if (source?.kind !== 'dsh-ops:step-tail' || source.contributor !== id) continue + any = true + if (source.revision === revision) { current = true; break } + } + return { any, current } } function lifetime(owner: Context, agent: Agent, cleanup: PromptDisposer): PromptDisposer { let active = true @@ -71,17 +86,26 @@ function ordered(entries: Iterable): TailEntry[] { return [...entries].sort((a, b) => (a.options.priority ?? 0) - (b.options.priority ?? 0) || (a.options.id < b.options.id ? -1 : a.options.id > b.options.id ? 1 : 0)) } -async function appendTails(state: AgentState, payload: Omit, decision: PreStepDecision): Promise { +async function appendTails(state: AgentState, payload: Omit, decision: PreStepDecision): Promise { if (decision.kind !== 'enter' || !decision.messages.length || payload.signal.aborted || !state.tails.size) return decision const entries = ordered(state.tails.values()) - const context: StepTailContext = Object.freeze({ ...payload, messages: freeze(structuredClone(decision.messages)) }) + // The committed history is the visibility evidence for every provider: read it + // once per batch, before this batch itself commits, so a contribution replayed + // from a restored session is distinguishable from a new addition and a + // compaction that removed one is distinguishable from a fresh session. + const history = payload.agent.session.deriveMessages() + const context: Omit = Object.freeze({ ...payload, messages: freeze(structuredClone(decision.messages)) }) const pending: { entry: TailEntry; text: string }[] = [] for (const entry of entries) { if (payload.signal.aborted) return decision if (!entry.active) continue + const visible = readVisibility(history, entry.options.id, entry.options.revision) + // A same-revision message already visible is a historical replay, not a new + // addition: the provider is skipped entirely instead of re-teaching. + if (entry.options.dedupe === 'visible' && visible.current) continue try { const signal = AbortSignal.any([payload.signal, entry.lifetime.signal]) - const text = await cancellable(Promise.resolve().then(() => entry.active && !signal.aborted ? entry.options.provide(Object.freeze({ ...context, signal })) : undefined), signal) + const text = await cancellable(Promise.resolve().then(() => entry.active && !signal.aborted ? entry.options.provide(Object.freeze({ ...context, signal, visible })) : undefined), signal) if (payload.signal.aborted) return decision if (text !== undefined && typeof text !== 'string') throw new TypeError('Step-tail providers must return text or undefined.') if (text && entry.active) pending.push({ entry, text }) diff --git a/src/prompt-injection/types.ts b/src/prompt-injection/types.ts index cbf81f75..685d3344 100644 --- a/src/prompt-injection/types.ts +++ b/src/prompt-injection/types.ts @@ -12,6 +12,13 @@ export interface SystemSectionOptions extends PromptContribution { readonly text: string | ((context: AssembleContext) => string) } export type ReadonlyValue = T extends object ? { readonly [K in keyof T]: ReadonlyValue } : T +/** What the model-visible history currently shows for one step-tail contribution. */ +export interface VisibleContribution { + /** A step-tail message of this contributor is visible, whatever its revision. */ + readonly any: boolean + /** A visible message of this contributor carries exactly this revision. */ + readonly current: boolean +} export interface StepTailContext { readonly agent: Agent readonly turn: number @@ -20,8 +27,19 @@ export interface StepTailContext { readonly signal: AbortSignal /** Detached, deeply frozen nonempty accepted batch, without this API's contributions. */ readonly messages: readonly ReadonlyValue[] + /** Visibility of this contribution in the agent's derived history, read before this batch commits. */ + readonly visible: VisibleContribution } export interface StepTailProviderOptions extends PromptContribution { + /** + * Opt-in restore dedupe: while a step-tail message with this id and revision + * is visible in the agent's derived history, the provider is not evaluated and + * nothing is appended. A session restored from that history therefore adds 0 + * messages, while a compaction replace that removes the visible message + * re-arms the provider, whose next provide then sees visible.any === false. + * Without the option every accepted batch may append, as before. + */ + readonly dedupe?: 'visible' /** * Called only for accepted nonempty user-message batches, not every model request. * The final active check before this hook returns its batch to host pre-step is diff --git a/src/ptc/batching-guidance.ts b/src/ptc/batching-guidance.ts index ed237e6d..40880ad9 100644 --- a/src/ptc/batching-guidance.ts +++ b/src/ptc/batching-guidance.ts @@ -14,8 +14,10 @@ import { literalShellFileWork } from './shell-file-work.js' /** Identity of the guidance in the host message source. */ export const BATCHING_GUIDANCE_ID = 'batching-guidance' +/** Contributor of the one-time full lesson; the per-violation notices keep the base id. */ +export const BATCHING_GUIDANCE_LESSON_ID = `${BATCHING_GUIDANCE_ID}:lesson` /** Revision of the guidance text; bump it when the wording or the file format changes. */ -export const BATCHING_GUIDANCE_REVISION = '4' +export const BATCHING_GUIDANCE_REVISION = '7' /** The shipped guidance file, resolved from this module's own location (src or lib). */ const DEFAULT_GUIDANCE_FILE = new URL('../../prompts/batching-guidance.md', import.meta.url) @@ -23,6 +25,8 @@ const DEFAULT_GUIDANCE_FILE = new URL('../../prompts/batching-guidance.md', impo /** Loaded text by absolute location; a failed load caches '' instead of retrying. */ const loaded = new Map() let warned = false +/** Reported once: the shipped file lost the anchor the loader-less rendering rewrites. */ +let warnedAnchor = false /** Report an unreadable guidance file once, never once per agent or call. */ function warnUnavailable(location: string, error: unknown): void { @@ -60,6 +64,273 @@ export function loadBatchingGuidance(filePath?: string | URL): string { } } +/** + * What one PTC scope actually publishes, in public tool names. + * + * The guidance file teaches progressive disclosure, and its loader lesson assumes + * `ops_load_sdk` is reachable. The loader is only published while dynamic loading is + * on, so the same text must not reach a scope that has none: it would spend a round + * trip in that scope on a call that cannot succeed. + */ +export interface GuidanceSurface { + /** Whether the scope publishes `ops_load_sdk`. */ + readonly loader: boolean + /** Public names the scope publishes without loading. */ + readonly resident: readonly string[] + /** Public names the loader can add, empty when the scope publishes no loader. */ + readonly loadable: readonly string[] +} + +/** The file-work APIs the guidance names, in the order it names them. */ +const FILE_APIS: readonly string[] = Object.freeze([ + 'ops_glob', 'ops_grep', 'ops_read_files', 'ops_inspect_local_file', 'ops_edit_file', +]) + +/** + * One sentence naming the file-work APIs the scope really publishes. + * + * The shipped wording is kept when every name it mentions is available, so the + * common loader scope reads exactly as before; a narrower scope gets the subset and + * a scope with none of them is pointed at the official tools instead. + * @param names - public names available for file work in this scope. + * @returns the sentence for the bypass notice. + */ +function fileWorkHint(names: readonly string[]): string { + const usable = FILE_APIS.filter(name => names.includes(name)) + if (usable.length === 0) return 'No OPS file API is available here: do file work with the official tools.' + if (usable.length === FILE_APIS.length) { + return 'List with `ops_glob`, search with `ops_grep`, read with `ops_read_files` or `ops_inspect_local_file`, edit with `ops_edit_file`.' + } + return 'Do file work with the APIs this scope publishes: `' + usable.join('`, `') + '`.' +} + +/** + * Drop every section whose heading names loading. + * + * The fallback path runs when the shipped file no longer carries the anchor: a + * renamed heading still leaves the old lesson in place, and pairing it with the + * replacement would put two contradicting steps in one document. + * @param rows - guidance lines. + * @returns the lines without loader-named sections. + */ +function dropLoadSections(rows: readonly string[]): string[] { + const kept: string[] = [] + let dropping = false + let fence: string | null = null + for (const row of rows) { + if (fence !== null) { + if (row.trim().startsWith(fence)) fence = null + if (!dropping) kept.push(row) + continue + } + const opening = /^\s*(```+|~~~+)/.exec(row) + if (opening !== null) { + fence = opening[1]! + if (!dropping) kept.push(row) + continue + } + // A heading inside a fence is example text, and only a word that means loading may + // drop a section: "payload", "download" and "workload" are ordinary prose. + if (row.startsWith('## ')) dropping = LOAD_HEADING.test(row) + if (!dropping) kept.push(row) + } + return kept +} + +/** + * Remove every trace of the load step from the lines that stay in the document. + * + * A fenced block is dropped as a unit when it teaches loading: deleting the call and + * keeping the lines that consumed the loaded names would ship a half example that a + * model can copy. + * @param rows - guidance lines outside the replaced section. + * @returns the lines without the load step. + */ +function stripLoader(rows: readonly string[]): string[] { + const kept: string[] = [] + let fence: string | null = null + let block: string[] = [] + for (const row of rows) { + if (fence === null) { + const opening = /^\s*(```+|~~~+)/.exec(row) + if (opening !== null) { + fence = opening[1]! + block = [row] + continue + } + if (row.startsWith(LOAD_TABLE_ROW_PREFIX)) continue + if (row.includes('ops_load_sdk')) continue + // The Step 1 prose advertises the same entry without naming the loader tool, so + // it must not survive beside a replacement section that says there is no loader. + if (row.includes(LOAD_PROSE_ANCHOR)) continue + kept.push(row) + continue + } + block.push(row) + if (row.trim().startsWith(fence)) { + if (!block.some(line => line.includes('ops_load_sdk'))) kept.push(...block) + fence = null + block = [] + } + } + // An unterminated fence keeps its lines: dropping the tail of a drifted file would + // hide more than the load step. + kept.push(...block) + return kept +} + +/** The heading that starts the loader lesson in the guidance file. */ +export const LOAD_SECTION_HEADING = '## Step 3 - load before you call' +/** A heading that names loading, by word: "payload" is not a load step. */ +const LOAD_HEADING = /\bload(?:ing|er)?\b/i + +/** The one table row that teaches the loader. */ +export const LOAD_TABLE_ROW_PREFIX = '| ops_read_files before it is loaded' +/** The opening words of the Step 1 prose that teaches SDK loading without the tool name. */ +export const LOAD_PROSE_ANCHOR = 'SDK loading' + +/** + * The path caveat for a loader-less scope that cannot discover paths. + * + * `ops_inspect_local_file` reads one file by exact path and rejects directories, so a + * scope that publishes it without `ops_glob` or `ops_grep` must say where the paths + * come from: a model that probes a directory through it spends a failed round trip. + * @param resident - public names the scope publishes without loading. + * @returns the caveat lines, empty when a listing or search API is published. + */ +function exactPathNote(resident: readonly string[]): string[] { + if (!resident.includes('ops_inspect_local_file') || resident.includes('ops_glob') || resident.includes('ops_grep')) return [] + return [ + '', + 'File work here has no listing and no search API: `ops_inspect_local_file` reads one', + 'file by its exact path and rejects directories, so call it with concrete file paths', + 'you already know.', + ] +} + +/** + * The section that replaces the loader lesson when the scope publishes no loader. + * @param surface - what the scope publishes. + * @returns the replacement lines, heading included. + */ +function directSteps(surface: GuidanceSurface): string[] { + if (surface.resident.length === 0) { + return [ + '## Step 3 - what this scope publishes', + '', + 'This scope publishes no OPS API and no loader. Work with the official tools;', + 'an `ops_*` name outside the SDK is not available here, and retrying will not', + 'make one appear.', + ] + } + return [ + '## Step 3 - call what is published', + '', + 'This scope publishes no loader: every API already in the SDK is callable directly,', + 'and a name outside it is not available. Calling what you already have is the whole', + 'step, with no load to spend a program on:', + '', + '`' + surface.resident.join('`, `') + '`', + ...exactPathNote(surface.resident), + ] +} + +/** + * The index of the heading that ends the section starting at `start`. + * @param lines - the guidance lines. + * @param start - the section heading index. + * @returns the next heading index, or the line count when the section is last. + */ +/** + * The index of the load-step heading, ignoring anything inside a fence. + * + * A fenced example that quotes the heading is text, not structure: taking it for the + * anchor would cut the document in the wrong place. + * @param lines - the guidance lines. + * @returns the heading index, or -1 when the file lost it. + */ +function anchorIndex(lines: readonly string[]): number { + let fence: string | null = null + for (let index = 0; index < lines.length; index += 1) { + const row = lines[index]! + if (fence !== null) { + if (row.trim().startsWith(fence)) fence = null + continue + } + const opening = /^\s*(```+|~~~+)/.exec(row) + if (opening !== null) { + fence = opening[1]! + continue + } + if (row.trim() === LOAD_SECTION_HEADING) return index + } + return -1 +} + +function nextHeading(lines: readonly string[], from: number): number { + let fence: string | null = null + for (let index = from; index < lines.length; index += 1) { + const row = lines[index]! + if (fence !== null) { + if (row.trim().startsWith(fence)) fence = null + continue + } + const opening = /^\s*(```+|~~~+)/.exec(row) + if (opening !== null) { + fence = opening[1]! + continue + } + if (row.startsWith('## ')) return index + } + return lines.length +} + +/** + * The index of the heading that ends the section starting at `start`. + * @param lines - the guidance lines. + * @param start - the section heading index. + * @returns the next heading index outside any fence, or the line count. + */ +function sectionEnd(lines: readonly string[], start: number): number { + return nextHeading(lines, start + 1) +} + +/** + * Render the guidance for one published surface. + * + * A scope with the loader receives the file byte for byte, so the default path is + * unchanged. A loader-less scope receives the same lesson with the load step + * replaced: the prose that tells the model to load cannot be true there. + * @param text - the loaded guidance file. + * @param surface - what the scope publishes. + * @returns the model-facing guidance text. + */ +export function renderBatchingGuidance(text: string, surface: GuidanceSurface): string { + if (text === '' || surface.loader) return text + const lines = text.split('\n') + const start = anchorIndex(lines) + // Lines that teach loading, and the table row that points at it, never reach a + // scope without a loader. The replacement is inserted after this filter, so the + // filter can never eat the section it was built to produce. + if (start < 0) { + // A drifted file is a plugin defect, not a model-facing surprise: say so once, + // then still keep the lesson free of an unreachable call. + if (!warnedAnchor) { + warnedAnchor = true + console.warn('dsh-ops: the batching guidance lost its "' + LOAD_SECTION_HEADING + + '" anchor; the load step is replaced before the first section instead.') + } + const body = dropLoadSections(stripLoader(lines)) + const at = nextHeading(body, 0) + return [...body.slice(0, at), ...directSteps(surface), '', ...body.slice(at)].join('\n') + } + // Insert the replacement exactly where the load step stood, so the document keeps + // its heading order and the shared sections keep their positions. + const before = stripLoader(lines.slice(0, start)) + const after = stripLoader(lines.slice(sectionEnd(lines, start))) + return [...before, ...directSteps(surface), '', ...after].join('\n') +} + // ---- program shape: what one run_code program says about its step ---- /** A program shorter than this is treated as single-step unless it is structured. */ @@ -81,7 +352,9 @@ const PLAN_ENTRY_SEPARATOR = /[;;]/u const DISPATCH = /\btools\s*(?:\.\s*([A-Za-z_$][\w$]*)|\[\s*['"]([^'"]+)['"]\s*\])\s*\(/gu const BATCHED = /Promise\.all|Promise\.allSettled|\.map\s*\(|for\s*\(|while\s*\(|forEach\s*\(|\[\s*\.\.\./u const BRANCHING = /\bif\s*\(|\btry\s*\{|\bswitch\s*\(|\?\s*[^:]{1,80}:/u -const WAITING = /\bwait\s*:\s*true|job_output\s*\(/u +const WAITING = /\bwait\s*:\s*true\b/u +/** Tools whose call is itself a wait on other work, recognised only at a real dispatch site. */ +const WAITING_TOOL = /^(?:wait_agent|job_output)$/u const BYPASS = /node:fs|node:child_process|\bfs\./u const SHELL_TOOL = /^(?:pwsh|bash|shell|ops_bash|pwsh_persistent|bash_persistent)$/u @@ -272,7 +545,10 @@ export function analyzeProgram(code: unknown): ProgramShape { dispatchSites: sites.length, hasBatchingStructure: BATCHED.test(code), hasBranching: BRANCHING.test(code), - isWaiting: WAITING.test(code), + // Waiting on an agent or job is a dependency barrier; tool waits count only at a + // real dispatch site and `wait: true` only in executable code, so comments and + // strings never exempt a program. + isWaiting: WAITING.test(readable.bare) || sites.some(match => WAITING_TOOL.test(match[1] ?? match[2] ?? '')), // Comments and message text are not file work; module specifiers stay readable // so a real import of node:fs still bypasses. bypassesFileTools: BYPASS.test(readable.bare) || shellForFiles, @@ -306,8 +582,11 @@ export function violationsOf(shape: ProgramShape): ProgramViolations { plan: !shape.hasFirstLinePlan, bypass: shape.bypassesFileTools, // Waiting on a job and branching are legitimate non-batched shapes, and a long - // program is not judged from its text alone. - shape: !shape.hasBatchingStructure && !shape.hasBranching && !shape.isWaiting && shape.chars < THIN_CODE_CHARS, + // program is not judged from its text alone. The notice claims a single operation, + // so it needs exactly one dispatch site: two sequential sites may be dependent, and + // the text alone cannot prove they could have run in parallel. + shape: shape.dispatchSites === 1 && !shape.hasBatchingStructure && !shape.hasBranching && !shape.isWaiting + && shape.chars < THIN_CODE_CHARS, } } @@ -351,7 +630,7 @@ export interface ProgramLatch { * @param shape - the program shape the notice describes. * @returns the model-facing text, without the plugin-context envelope. */ -export function noticeText(category: BatchingCategory, shape: ProgramShape): string { +export function noticeText(category: BatchingCategory, shape: ProgramShape, surface: GuidanceSurface): string { if (category === 'plan') { return [ 'Your last program did not open with a plan line.', @@ -360,10 +639,27 @@ export function noticeText(category: BatchingCategory, shape: ProgramShape): str ].join('\n') } if (category === 'bypass') { + // Every line names only what this scope publishes: a hardcoded tool list would + // contradict the sentence that explains the scope two lines below it. + // Resident APIs need no load, so they count as reachable beside the pool. + const available = surface.loader + ? [...new Set([...surface.resident, ...surface.loadable])] + : [...surface.resident] + const fileUsable = FILE_APIS.filter(name => available.includes(name)).length + const coversFiles = fileUsable === FILE_APIS.length return [ 'Your last program did file work through the shell or Node IO, which bypasses path policy and the audit trail.', - 'List with `ops_glob`, search with `ops_grep`, read with `ops_read_files` or `ops_inspect_local_file`, edit with `ops_edit_file`.', - 'Use the APIs in the current SDK; load only missing names with `tools.ops_load_sdk` for the next program.', + // A pool that cannot load `ops_edit_file` must not be told to load it either. + fileWorkHint(available), + surface.loader + ? coversFiles + ? 'Use the APIs in the current SDK; load only missing names with `tools.ops_load_sdk` for the next program.' + : fileUsable === 0 + ? 'The loadable pool holds no file API either, so loading adds nothing here.' + : 'Load only names this scope publishes with `tools.ops_load_sdk` for the next program; an API outside that pool is not reachable.' + : fileUsable === 0 + ? 'There is no loader here, and retrying a name will not publish one.' + : 'There is no loader here: call those APIs directly in the next program.', ].join('\n') } return [ @@ -382,7 +678,7 @@ export function noticeText(category: BatchingCategory, shape: ProgramShape): str * by the notice text, while a missed correction lasts the rest of the turn. * @returns the latch; state is keyed by agent identity and never serialized. */ -export function createProgramLatch(): ProgramLatch { +export function createProgramLatch(surfaceOf: (agent: Agent) => GuidanceSurface): ProgramLatch { const states = new WeakMap; sent: Record }>() const fresh = (turn: number) => ({ turn, armed: { plan: true, bypass: true, shape: true }, sent: { plan: 0, bypass: 0, shape: 0 } }) const stateFor = (agent: Agent) => { @@ -408,7 +704,8 @@ export function createProgramLatch(): ProgramLatch { if (category === undefined) return undefined state.armed[category] = false state.sent[category] += 1 - return { category, turn: state.turn, text: noticeText(category, shape) } + // Read-only policy is per agent, so the surface is resolved at delivery time. + return { category, turn: state.turn, text: noticeText(category, shape, surfaceOf(agent)) } }, isArmed(agent, category) { return stateFor(agent).armed[category] }, stats(agent) { diff --git a/src/ptc/config.ts b/src/ptc/config.ts index 1d255633..8eccb1c6 100644 --- a/src/ptc/config.ts +++ b/src/ptc/config.ts @@ -4,13 +4,24 @@ import type { FileToolName, PtcToolName } from '../capabilities/registry.js' import { migrateRawNames } from '../compat/alias.js' export type ApiName = FileToolName | PtcToolName export const PTC_LOADABLE_TOOLS: readonly ApiName[] = [...FILE_TOOLS, ...PTC_TOOLS].filter(tool => !PTC_RESIDENT_TOOLS.includes(tool)) -export interface PtcConfig { binaryPath?: string; required: boolean; allowedTools: ApiName[]; alwaysLoaded: ApiName[] } +/** The explicit first-round disclosure tier of a PTC row (#986/#979). */ +export type DisclosurePolicy = 'on-demand' | 'full' +export interface PtcConfig { binaryPath?: string; required: boolean; allowedTools: ApiName[]; alwaysLoaded: ApiName[]; disclosurePolicy?: DisclosurePolicy } +// parsePtcConfig always fills disclosurePolicy; a slice that omits the key keeps +// the legacy on-demand default. The builtin plan carries the parsed value since +// D3; the custom integration path passes the key through since D4 (#988). export function parsePtcConfig(raw: unknown = {}): PtcConfig { if (!raw || typeof raw !== 'object' || Array.isArray(raw)) throw new Error('PTC config must be an object') const input = raw as Record - for (const key of Object.keys(input)) if (!['binaryPath', 'required', 'allowedTools', 'alwaysLoaded'].includes(key)) throw new Error('Unknown PTC config key: ' + key) + for (const key of Object.keys(input)) if (!['binaryPath', 'required', 'allowedTools', 'alwaysLoaded', 'disclosurePolicy'].includes(key)) throw new Error('Unknown PTC config key: ' + key) if (input.binaryPath !== undefined && typeof input.binaryPath !== 'string') throw new Error('binaryPath must be a string') if (input.required !== undefined && typeof input.required !== 'boolean') throw new Error('required must be boolean') + // The default 'on-demand' is the legacy path; 'full' is a distinct explicit + // opt-in (#979 §4) — no upgrade, save or switch may pick it for the user. + const policy = input.disclosurePolicy + if (policy !== undefined && (typeof policy !== 'string' || (policy !== 'on-demand' && policy !== 'full'))) { + throw new Error('disclosurePolicy must be "on-demand" or "full"; got ' + (JSON.stringify(policy) ?? String(policy)) + ' (' + typeof policy + ')') + } const names = (key: string, fallback: readonly ApiName[]): ApiName[] => { const value = input[key] ?? fallback if (!Array.isArray(value)) throw new Error(key + ' must be an array of raw API names') @@ -19,5 +30,6 @@ export function parsePtcConfig(raw: unknown = {}): PtcConfig { return [...new Set(migrated as ApiName[])] } return { binaryPath: input.binaryPath, required: input.required ?? false, - allowedTools: names('allowedTools', PTC_LOADABLE_TOOLS), alwaysLoaded: names('alwaysLoaded', []) } + allowedTools: names('allowedTools', PTC_LOADABLE_TOOLS), alwaysLoaded: names('alwaysLoaded', []), + disclosurePolicy: policy === 'full' ? 'full' : 'on-demand' } } diff --git a/src/ptc/disclosure.ts b/src/ptc/disclosure.ts new file mode 100644 index 00000000..fce0a95e --- /dev/null +++ b/src/ptc/disclosure.ts @@ -0,0 +1,49 @@ +/** The one first-round disclosure plan; publication and diagnostics read the same derivation. */ +import { PTC_RESIDENT_TOOLS } from '../capabilities/registry.js' +import type { ApiName, PtcConfig } from './config.js' + +/** The plan one PTC scope publishes before any loader call, derived once from config and gates. */ +export interface DisclosurePlan { + /** Whether the explicit full first-round tier (#979 §2) is active. */ + full: boolean + /** Names registered for this scope: the resident surface plus the selectable candidates. */ + pool: ApiName[] + /** Names visible in the first round without any loader call. */ + firstRound: ApiName[] + /** Whether the `load_sdk` loader is published in this scope. */ + loader: boolean + /** APIs a loader call may still select, including `result`; empty selection set when the loader is off. */ + loadable: ApiName[] +} + +/** + * Derive the disclosure plan one PTC row publishes. + * + * D3 (#987) defined the tiers; D4 (#988) makes the diagnostics read this same + * derivation instead of guessing expected names, so a selectable-but-unselected + * API (for example `result` on the on-demand tier) is never a publication gap. + * + * @param config - the row's parsed PTC slice. + * @param resident - whether the consumer grants the resident surface (`residentTools`). + * @param dynamic - whether the consumer grants progressive loading (`discovery`). + * @returns the plan; every member of `firstRound` is visible this step, and + * `loader` is false exactly when no `load_sdk` is registered. + */ +export function disclosurePlan(config: Pick, resident: boolean, dynamic: boolean): DisclosurePlan { + const full = config.disclosurePolicy === 'full' + // A residentTools:false scope drops the resident surface entirely, including + // alwaysLoaded (truth-table row 5); the tier itself survives. + const residentSet: ApiName[] = resident ? [...PTC_RESIDENT_TOOLS, ...config.alwaysLoaded] : [] + // A loader-less scope can never select the allowed pool: registering it only lets + // a host that renders registered definitions declare unreachable APIs. The full + // tier is loader-less by design, so its pool must register. + const selectable: ApiName[] = dynamic === false && !full ? [] : [...config.allowedTools, 'result'] + const loadable = [...new Set([...config.allowedTools, 'result'])] + return { + full, + pool: [...new Set([...residentSet, ...selectable])], + firstRound: [...new Set([...residentSet, ...(full ? selectable : [])])], + loader: dynamic !== false && !full, + loadable, + } +} diff --git a/src/publication/ptc.ts b/src/publication/ptc.ts index f9167c8e..26b95374 100644 --- a/src/publication/ptc.ts +++ b/src/publication/ptc.ts @@ -1,5 +1,5 @@ /** Stable, scoped PTC definitions; DSH resolves selection once per model step. */ -import type { Agent, Context, Fiber, ToolDefinition, ToolRunContext, UserMessage } from '../adapter/dsh/types.js' +import type { Agent, Context, Fiber, ToolActivationContext, ToolDefinition, ToolRunContext, UserMessage } from '../adapter/dsh/types.js' import { loadUserMessage } from '../adapter/dsh/host.js' import { resolvePtcConfig } from '../config.js' import { acquirePtcFile, acquireSharedFile, isSharedFileApi } from './scope-owner.js' @@ -10,16 +10,17 @@ import { publicToolName } from '../policy.js' import { definition, PTC_SECTION } from '../projections/ptc/contract.js' import { requestedApis, sdkDefinition } from '../projections/ptc/loader.js' import { renderSdkDirectory } from '../projections/ptc/directory.js' -import { PTC_RESIDENT_TOOLS } from '../ptc/config.js' import type { ApiName } from '../ptc/config.js' +import { disclosurePlan } from '../ptc/disclosure.js' import { ownPtcScopes } from '../ptc/ownership.js' -import { BATCHING_GUIDANCE_ID, BATCHING_GUIDANCE_REVISION, createProgramLatch, loadBatchingGuidance } from '../ptc/batching-guidance.js' +import { BATCHING_GUIDANCE_ID, BATCHING_GUIDANCE_LESSON_ID, BATCHING_GUIDANCE_REVISION, createProgramLatch, loadBatchingGuidance, renderBatchingGuidance } from '../ptc/batching-guidance.js' +import type { GuidanceSurface } from '../ptc/batching-guidance.js' import { InvocationLifetime, InvocationNotStarted } from '../lifecycle/invocations.js' import { monitoredInvoke } from '../execution/monitored.js' import { record } from '../execution/boundary.js' import { failure, resolveWorker } from '../execution/jobs.js' +import type { Outcome } from '../kernel/outcome.js' import { WORKER_TIMEOUT_MS } from '../execution/protocol.js' -import { registerStepTailProvider } from '../prompt-injection/index.js' import { mountWorkspacePreparation } from '../workspace-preparation.js' import { mountResultLifecycle } from '../kernel/result-lifecycle.js' import { activationIsPtc, inspectPresentation, isPresentationProbe } from './presentation.js' @@ -29,6 +30,20 @@ import { retired as retiredLegacyRow } from '../settings/legacy.js' export { resolvePtcConfig } from '../config.js' export const name = 'dsh-ops-ptc' +/** + * What the PTC projection can serve for one agent. + * + * `servable` returns `undefined` until the publisher is attached, and for any + * agent this consumer does not own: an unknown answer must keep the official + * tools available rather than hide them. The set contains raw API names the + * publisher itself would admit for that agent right now. + */ +export interface PtcServability { + servable(agent: Agent): ReadonlySet | undefined + /** The publisher's own per-activation presentation predicate. */ + isPtc(activation: ToolActivationContext): Promise +} + interface Entry { agent: Agent ctx?: Context @@ -39,7 +54,7 @@ interface Entry { pending: Set definitions: Map disposers: (() => void)[] - /** Whether the teacher trajectory reached session history, not merely the pre-step. */ + /** Whether this agent already received the one-time lesson through the ephemeral tool-result inbox. */ taught: boolean files: Map void; dispatch: ToolDefinition['execute']; granted?: { entitled: boolean; selected: boolean; config: object } }> } @@ -58,12 +73,17 @@ export interface PtcGate { export function apply(root: Context, rawConfig: unknown, gate: PtcGate = { allows: () => true }): void { if (!retiredLegacyRow(root, 'ptc')) mountPtc(root, rawConfig, gate) } -export function mountPtc(root: Context, rawConfig: unknown, gate: PtcGate = { allows: () => true }): void { +export function mountPtc(root: Context, rawConfig: unknown, gate: PtcGate = { allows: () => true }): PtcServability { const config = resolvePtcConfig(rawConfig) + // The accessor is synchronous while attachment is asynchronous; every answer + // before the publisher owns the agent is "unknown", which keeps official tools. + let servable: ((agent: Agent) => ReadonlySet | undefined) | undefined + let presentation: ((activation: ToolActivationContext) => Promise) | undefined let file: string try { file = resolveWorker(config) } catch (error) { if (config.required) throw error - root.logger.warn(`dsh-ops: ${String(error)}`); return + root.logger.warn(`dsh-ops: ${String(error)}`) + return { servable: () => undefined, isPtc: async () => false } } mountWorkspacePreparation(root) root.inject(['tools', 'systemPrompt', 'sandboxPolicy', 'subprocess'], async ctx => { @@ -74,40 +94,88 @@ export function mountPtc(root: Context, rawConfig: unknown, gate: PtcGate = { al let presentationRevision = 0 ctx.on('system-prompt/change', () => { ++presentationRevision }) let ownership: Awaited> | undefined - const resident = new Set(gate.resident === false ? [] : [...PTC_RESIDENT_TOOLS, ...config.alwaysLoaded]) - // A loader-less scope can never select the allowed pool: registering it only - // lets a host that renders registered definitions declare unreachable APIs. - const selectable: ApiName[] = gate.dynamic === false ? [] : [...config.allowedTools, 'result' as ApiName] - const pool = new Set([...resident, ...selectable]) + // D3 (#987, #979 §2): an explicit full tier carries the whole permitted + // allowedTools pool — plus `result`, the D1 sign-off item 1 decision, so a ref + // spilled this step never dead-ends (#979 §3; the unified acceptance rule + // also needs it: on-demand can always load `result`) — in the first round, + // with no loader. The bundle gate has already pressed an incompatible row + // back to on-demand (effectiveMode, sign-off item 2); the row-level + // dynamicTools keeps its loader-only meaning, so a full row that also turns + // the loader off keeps its explicit tier. D4 (#988) moved this derivation into + // src/ptc/disclosure.ts so the diagnostics read the same plan instead of + // guessing expected names. + const plan = disclosurePlan(config, gate.resident !== false, gate.dynamic !== false) + const full = plan.full + const resident = new Set(plan.firstRound) + const pool = new Set(plan.pool) const owns = (agent: Agent): boolean => !stopped && gate.allows(agent) && Boolean(ownership?.owns(agent)) + // One predicate for "may this agent call it": the guidance surface and the tool + // registration must never disagree about a read-only policy. + const reachable = (mode: string | undefined, api: ApiName): boolean => + mode !== 'read-only' || !capability(api)?.mutates const permitted = (agent: Agent, api: ApiName | 'load_sdk'): boolean => { if (!owns(agent) || !ctx.get('ptcRuntime') || !ctx.get('subprocess')) return false const policy = ctx.get('sandboxPolicy')?.resolve({ session: agent.session }) return policy !== undefined && ['read-only', 'workspace-write', 'danger-full-access'].includes(policy.mode) - && (api === 'load_sdk' || api === 'result' || !capability(api)?.mutates || policy.mode !== 'read-only') + && (api === 'load_sdk' || api === 'result' || reachable(policy.mode, api)) } - const loadable = [...new Set([...config.allowedTools, 'result'])] + const loadable = plan.loadable const allowedApis = (agent: Agent): ApiName[] => gate.dynamic === false ? [] : loadable.filter(api => permitted(agent, api)) const selected = (entry: Entry, api: ApiName | 'load_sdk'): boolean => api === 'load_sdk' - ? gate.dynamic !== false : resident.has(api) || entry.selected.has(api) + ? plan.loader : resident.has(api) || entry.selected.has(api) const isPtc = async (agent: Agent, signal?: AbortSignal): Promise => { const assembly = await inspectPresentation(prompt, agent, signal) return assembly.tools.length === 1 && assembly.tools[0]?.name === 'run_code' && assembly.sections.some(section => section.name === 'tools:ptc-only' && Boolean(section.text.trim())) } - // Repeated writes to one path signal a batching mistake. The guidance rides - // the triggering tool result, so no system-prompt bytes are spent up front. + // Answer from the publisher's own pool instead of a host-side lookup: the + // official file APIs this projection replaces are SDK bindings, so a host + // `tools.get` cannot prove reachability. `selected` is deliberately absent: + // hiding must not depend on whether the model already loaded the SDK. + servable = (agent: Agent): ReadonlySet | undefined => { + if (!owns(agent)) return undefined + const names = new Set() + const allowed = new Set(allowedApis(agent)) + for (const api of pool) { + if (permitted(agent, api) && (resident.has(api) || allowed.has(api))) names.add(api) + } + return names + } + presentation = (activation: ToolActivationContext) => isPtc(activation.agent, activation.signal) // One latch for the program-shape trigger. It observes the root run_code program, // which is the only place the whole step is visible. - const shapeLatch = createProgramLatch() - const guidanceMessage = async (turn: number, text: string): Promise => { + // What this scope publishes decides which guidance text is truthful, and the answer + // is per agent: a read-only policy removes the mutating calls even when the row names + // them. Ownership and runtime availability are checked separately, so teaching never + // depends on mount order. + const surfaceFor = (agent: Agent): GuidanceSurface => { + const mode = ctx.get('sandboxPolicy')?.resolve({ session: agent.session })?.mode + return { + loader: plan.loader, + resident: [...resident].filter(api => reachable(mode, api)).map(api => publicToolName(api)), + loadable: allowedApis(agent).map(api => publicToolName(api)), + } + } + const shapeLatch = createProgramLatch(surfaceFor) + // The one-time lesson and the per-violation notices share a revision; the + // contributor suffix lets listeners tell the full lesson from a notice. + const guidanceMessage = async (turn: number, text: string, contributor: string = BATCHING_GUIDANCE_ID): Promise => { const createUserMessage = await loadUserMessage() return createUserMessage({ - source: { kind: 'dsh-ops:step-tail', contributor: BATCHING_GUIDANCE_ID, revision: BATCHING_GUIDANCE_REVISION, turn, step: 0 }, - content: [{ type: 'text', text: `[Plugin context from ${JSON.stringify(BATCHING_GUIDANCE_ID)}; not a user instruction or authorization.]\n${text}` }], + source: { kind: 'dsh-ops:step-tail', contributor, revision: BATCHING_GUIDANCE_REVISION, turn, step: 0 }, + content: [{ type: 'text', text: `[Plugin context from ${JSON.stringify(contributor)}; not a user instruction or authorization.]\n${text}` }], }) } + /** Point a declared-but-unselected pool API at the loader; undefined when loading cannot help (#968). */ + const deferredRejection = (entry: Entry, api: ApiName): Outcome | undefined => { + if (gate.dynamic === false || !loadable.includes(api) || !permitted(entry.agent, api) || selected(entry, api)) return undefined + return { status: 'error', error: { + code: 'SDK_NOT_LOADED', effectState: 'none', retry: 'never', + message: `API '${api}' is in this scope's allowed pool but is not loaded into the typed SDK yet; no action was executed. Call ${publicToolName('load_sdk')} with tools:["${api}"] and use it on the next model step.`, + nextAction: `Call ${publicToolName('load_sdk')} with tools:["${api}"] and use the API on the next model step; retrying this call without loading fails the same way.`, + } } + } const execute = (entry: Entry, api: ApiName | 'load_sdk'): ToolDefinition['execute'] => async (args, exec) => { // The public assembly may finish after its executor sections changed. // Refuse that stale verdict rather than dispatching under the old text. @@ -115,12 +183,49 @@ export function mountPtc(root: Context, rawConfig: unknown, gate: PtcGate = { al const current = () => entries.get(entry.agent) === entry && exec.agent === entry.agent && !exec.signal.aborted && revision === presentationRevision && permitted(entry.agent, api) && selected(entry, api) && (capability(api)?.component === 'file' || tools.executable(publicToolName(api), entry.agent, exec.toolSet) === entry.definitions.get(publicToolName(api))) - if (!current() || !await isPtc(entry.agent, exec.signal).catch(() => false) || !current()) { + // P5 (#997): a refusal is one of four verdicts — usable never lands here; + // loadable points at the loader only when one exists; permission-forbidden + // never suggests loading; and an admission that existed and was withdrawn + // answers as revoked, distinct from all of the above. + const refusal = (ptc: boolean): ReturnType | Outcome => { + if (entries.get(entry.agent) !== entry) { + return failure('POLICY_DENIED', 'This PTC scope has been withdrawn; the retained admission is revoked. No action executed.') + } + if (exec.agent !== entry.agent || exec.signal.aborted + || (capability(api)?.component !== 'file' && tools.executable(publicToolName(api), entry.agent, exec.toolSet) !== entry.definitions.get(publicToolName(api)))) { + return failure('POLICY_DENIED', 'This API must be admitted for the calling PTC scope; no action executed.') + } + if (!ptc || revision !== presentationRevision) { + return failure('POLICY_DENIED', 'The PTC presentation of this scope has been withdrawn; the retained admission is revoked. No action executed.') + } + if (!permitted(entry.agent, api)) { + const wasUsable = api !== 'load_sdk' && (resident.has(api) || entry.selected.has(api)) + return failure('POLICY_DENIED', wasUsable + ? 'This API was admitted for the calling PTC scope, but the sandbox policy has since withdrawn its permission; the admission is revoked. No action executed.' + : 'The current sandbox policy does not permit this API for the calling PTC scope; no action executed.') + } + if (!selected(entry, api)) { + // A declared-but-unselected pool API points at the loader with an executable + // next step (#968); every other admission failure keeps the P5 refusal text. + const deferred = api === 'load_sdk' || full ? undefined : deferredRejection(entry, api) + if (deferred !== undefined) return deferred + return api !== 'load_sdk' && gate.dynamic !== false && !full + ? failure('SDK_NOT_LOADED', `This API (${String(api)}) is not loaded for the current step. Call tools.${publicToolName('load_sdk')} with its raw name; the selection takes effect on the next model step.`) + : failure('POLICY_DENIED', 'This API is not admitted for the current step and no loader can select it; no action executed.') + } return failure('POLICY_DENIED', 'This API must be admitted for the calling PTC scope; no action executed.') } + const ptc = await isPtc(entry.agent, exec.signal).catch(() => false) + if (!current() || !ptc || !current()) return refusal(ptc) if (api === 'load_sdk') { - const requested = requestedApis(args, allowedApis(entry.agent)) + // Validate against the configured pool, then split permission refusals from + // pool refusals: a mutating API under read-only is inside its pool yet + // forbidden, and reloading cannot change that permission (#997). + const requested = requestedApis(args, gate.dynamic === false ? [] : loadable) if (typeof requested === 'string') return failure('INVALID_ARGUMENT', requested) + const forbidden = requested.filter(name => !permitted(entry.agent, name)) + if (forbidden.length) return failure('POLICY_DENIED', + 'The current sandbox policy does not permit ' + forbidden.join(', ') + ' for the calling PTC scope; no action executed.') for (const name of requested) entry.pending.add(name) return { status: 'completed', tools: requested, nextTurn: true } } @@ -147,6 +252,7 @@ export function mountPtc(root: Context, rawConfig: unknown, gate: PtcGate = { al present: activation => activation.agent === entry.agent && permitted(entry.agent, api) && selected(entry, api) && activationIsPtc(prompt, activation), dispatch: file.dispatch, + deferred: () => deferredRejection(entry, api), }) file.granted = { entitled, selected: selection, config } } @@ -180,20 +286,6 @@ export function mountPtc(root: Context, rawConfig: unknown, gate: PtcGate = { al if (payload.agent === agent) shapeLatch.noteTurn(agent, payload.turn) return next() })) - // Teach the rules once, before the model's first request: a step-tail - // message rides the first accepted user batch and enters history. - // The provider must not latch here. A batch can still be withdrawn before - // admission (cancellation, rejection), and latching on the return value - // would then skip the lesson for the whole session without a trace. - entry.disposers.push(registerStepTailProvider(entry.ctx, agent, { - id: BATCHING_GUIDANCE_ID, - revision: BATCHING_GUIDANCE_REVISION, - provide() { - if (entry.taught) return undefined - const text = loadBatchingGuidance() - return text.length > 0 ? text : undefined - }, - })) // The program-shape trigger observes the ROOT run_code call: only there is // the whole program text visible. Nested tools.ops_* calls carry a parent // token, and their arguments say nothing about the shape of the step. @@ -203,22 +295,43 @@ export function mountPtc(root: Context, rawConfig: unknown, gate: PtcGate = { al if (exec.agent !== agent || exec.name !== 'run_code' || exec.parent !== undefined) return decision if (exec.signal.aborted) return decision const code = (exec.arguments as { code?: unknown } | undefined)?.code - const notice = shapeLatch.observe(agent, code) - if (notice === undefined) return decision + // Teach the rules once, riding the first accepted root run_code result: + // additionalContexts is the same-step ephemeral inbox, never logged + // history, so a lesson can never read as fresh user input and the + // model's answer cannot loop on its own guidance (#929). The latch is + // set at the delivery point: a post-execute decision that returns is + // the tool result the host renders, with no later admission step that + // could still withdraw it. + const contexts = [...(decision.additionalContexts ?? [])] try { - const message = await guidanceMessage(notice.turn, notice.text) - // Observation only: one line, no context, no new dependency. - if (gate.telemetry === true) { - const counts = shapeLatch.stats(agent) - ctx.logger.info('dsh-ops batching: turn=' + notice.turn + ' category=' + notice.category - + ' plan=' + counts.plan + ' bypass=' + counts.bypass + ' shape=' + counts.shape + ' total=' + counts.total) + // Render the lesson for the surface this step actually publishes (P1): + // a loader-less row never teaches the loader, and a read-only scope never + // names a write API. + if (!entry.taught) { + const text = renderBatchingGuidance(loadBatchingGuidance(), surfaceFor(agent)) + if (text.length > 0) { + contexts.push(await guidanceMessage(shapeLatch.stats(agent).turn, text, BATCHING_GUIDANCE_LESSON_ID)) + entry.taught = true + } + } + const notice = shapeLatch.observe(agent, code) + if (notice !== undefined) { + const message = await guidanceMessage(notice.turn, notice.text) + // Observation only: one line, no context, no new dependency. + if (gate.telemetry === true) { + const counts = shapeLatch.stats(agent) + ctx.logger.info('dsh-ops batching: turn=' + notice.turn + ' category=' + notice.category + + ' plan=' + counts.plan + ' bypass=' + counts.bypass + ' shape=' + counts.shape + ' total=' + counts.total) + } + contexts.push(message) } // Append only: the tool result keeps its own value, content and contexts. - return { ...decision, additionalContexts: [...(decision.additionalContexts ?? []), message] } + return contexts.length > 0 ? { ...decision, additionalContexts: contexts } : decision } catch (error) { - // A delivery failure is an optimisation loss, never a tool failure. + // A delivery failure is an optimisation loss, never a tool failure: + // contexts already collected still ride this result. ctx.logger.warn('dsh-ops: batching notice delivery failed: ' + String(error)) - return decision + return contexts.length > 0 ? { ...decision, additionalContexts: contexts } : decision } })) } @@ -246,7 +359,8 @@ export function mountPtc(root: Context, rawConfig: unknown, gate: PtcGate = { al } if (!owns(agent) || entries.get(agent) !== entry) return rotateFiles(entry) - if (gate.dynamic !== false) publish('load_sdk', sdkDefinition(loadable, execute(entry, 'load_sdk'))) + // The full tier is loader-less (#979 §2); the legacy tiers keep the loader gate. + if (plan.loader) publish('load_sdk', sdkDefinition(loadable, execute(entry, 'load_sdk'))) await Promise.all([...entry.disposers.map(publicationReady), ...[...entry.files.values()].map(file => file.lease.ready())]) }).catch(async error => { // Do not await ready here: this is its own rejection path. Preserve the @@ -286,7 +400,10 @@ export function mountPtc(root: Context, rawConfig: unknown, gate: PtcGate = { al // Sections are evaluated before the public presentation waterfall. Shared // file names are callable in both executors, so they cannot select PTC text. const ptc = result.sections.some(section => section.name === 'tools:ptc-only' && Boolean(section.text.trim())) - if (!isPresentationProbe(context)) { + // A diagnostic assembly carries no step tool set: it reads the registration + // surface and is never a model request, so it must not consume pending loads + // or cancel work the way a withdrawn request would (#997). + if (!isPresentationProbe(context) && context.toolSet) { const entry = entries.get(context.scope as Agent) if (entry && !ptc) { entry.pending.clear(); entry.calls.cancel('PTC presentation withdrawn') @@ -305,13 +422,10 @@ export function mountPtc(root: Context, rawConfig: unknown, gate: PtcGate = { al ctx.on('session/event', (session, event) => { if ((event as { type: string }).type === 'agent-preset/selected') { refresh(); return } for (const [agent, entry] of entries) if (agent.session === session) { - // The lesson counts as taught only once it is durable history: the step-tail - // snapshot can be withdrawn before admission, and a cancelled request must - // leave the next accepted batch free to teach it again. - if (event.type === 'user/message') { - const source = (event as { data?: { source?: { kind?: string; contributor?: string } } }).data?.source - if (source?.kind === 'dsh-ops:step-tail' && source.contributor === BATCHING_GUIDANCE_ID) entry.taught = true - } else if (event.type === 'step/end') { + // The batching lesson no longer latches on user/message events: it rides + // the first accepted root run_code tool result, whose returned decision is + // the delivery point itself, so there is no admission step to wait for. + if (event.type === 'step/end') { for (const api of entry.pending) if (permitted(agent, api)) entry.selected.add(api) entry.pending.clear() rotateFiles(entry) @@ -343,4 +457,8 @@ export function mountPtc(root: Context, rawConfig: unknown, gate: PtcGate = { al ownership.refresh() await Promise.all((ctx.get('agents')?.list() ?? []).map(attach)) }) + return { + servable: (agent: Agent) => servable?.(agent), + isPtc: (activation: ToolActivationContext) => presentation ? presentation(activation) : Promise.resolve(false), + } } diff --git a/src/publication/registrar.ts b/src/publication/registrar.ts index 7104aeb8..918507f5 100644 --- a/src/publication/registrar.ts +++ b/src/publication/registrar.ts @@ -13,6 +13,8 @@ export interface PublicationGrant { dispatch(args: unknown, exec: ToolRunContext): Promise /** Presentation may only narrow this captured grant; it never chooses dispatch. */ present?(activation: ToolActivationContext): boolean | Promise + /** Outcome for a caller whose grants exist but none is entitled: a deferred-selection pointer, not a silent dead end (#968). */ + deferred?(): Outcome | undefined cancel?(): void } export interface PublicationLease { @@ -20,11 +22,13 @@ export interface PublicationLease { ready(): Promise dispose(): void } -const denied = (code: 'PUBLICATION_IDENTITY_MISMATCH' | 'PUBLICATION_GRANT_UNAVAILABLE'): Outcome => ({ +const denied = (code: 'PUBLICATION_IDENTITY_MISMATCH' | 'PUBLICATION_GRANT_UNAVAILABLE' | 'PUBLICATION_GRANT_REVOKED'): Outcome => ({ status: 'rejected', error: { code, effectState: 'none', retry: 'never', message: code === 'PUBLICATION_IDENTITY_MISMATCH' ? 'Publication definition is stale or is not admitted for this agent and tool set.' - : 'Publication has no eligible grant or has conflicting dispatch grants for this agent.' }, + : code === 'PUBLICATION_GRANT_REVOKED' + ? 'Publication grants for this agent have been revoked and cannot serve this call; reassembly cannot reinstate them.' + : 'Publication has no eligible grant or has conflicting dispatch grants for this agent.' }, }) const cancelled = (started: boolean): Outcome => ({ status: started ? 'unknown' : 'rejected', error: { code: started ? 'RESULT_UNKNOWN' : 'ABORTED', effectState: started ? 'unknown' : 'none', retry: 'never', @@ -93,6 +97,10 @@ export async function acquire(ctx: Context, scope: object | undefined, name: str const tools = availableTools const grants = new Set() const owned = new Set() + // P5 (#997): an agent that ever held an eligible grant and now matches none + // has had its grants pulled — revoked, not never-available. The two verdicts + // stay distinct so a withdrawn admission cannot read as a loadable one. + const everEntitled = new WeakSet() // Public toolSet exposes the admitted definition, not which `when` contribution // admitted it. Conservatively bind every eligible same-agent grant as a veto: // losing any captured grant cancels; no hidden or later grant can sustain work. @@ -137,7 +145,21 @@ export async function acquire(ctx: Context, scope: object | undefined, name: str return denied('PUBLICATION_IDENTITY_MISMATCH') } const matching = eligible(exec.agent) - if (!matching.length || matching.some(grant => grant.dispatch !== matching[0].dispatch)) return denied('PUBLICATION_GRANT_UNAVAILABLE') + if (!matching.length) { + // Same-agent grants with no entitled member can still explain themselves: a pool + // publication points its caller at the loader instead of a retry-never denial (#968). + // Consult every same-agent advisor in grant order: one that cannot help — by + // answering undefined or by throwing — must not mask later grants that can. + for (const advisor of grants) { + if (advisor.agent !== exec.agent || typeof advisor.deferred !== 'function') continue + try { + const outcome = advisor.deferred() + if (outcome !== undefined && 'error' in outcome) return outcome + } catch { /* an unavailable advisor falls through to the next one */ } + } + return denied(everEntitled.has(exec.agent) ? 'PUBLICATION_GRANT_REVOKED' : 'PUBLICATION_GRANT_UNAVAILABLE') + } + if (matching.some(grant => grant.dispatch !== matching[0].dispatch)) return denied('PUBLICATION_GRANT_UNAVAILABLE') const controller = new AbortController() const caller = exec.signal const abort = () => controller.abort(caller?.reason) @@ -204,6 +226,7 @@ export async function acquire(ctx: Context, scope: object | undefined, name: str // Do not retain the caller's mutable object or callback properties. const captured = Object.freeze({ ...grant }) grants.add(captured) + if (captured.entitled) everEntitled.add(captured.agent) try { refresh(true) } catch (error) { grants.delete(captured) const errors: unknown[] = [error] diff --git a/src/publication/replacements.ts b/src/publication/replacements.ts index 71a10550..d3ff8ed9 100644 --- a/src/publication/replacements.ts +++ b/src/publication/replacements.ts @@ -1,12 +1,39 @@ /** Scoped replacement controller. Missing counterparts always leave official tools available. */ -import type { Agent, Context, ToolDefinition } from '../adapter/dsh/types.js' +import type { Agent, Context, ToolActivationContext, ToolDefinition } from '../adapter/dsh/types.js' import type {} from '../adapter/dsh/types.js' import { onToolChange, withToolChange } from './changes.js' import { publicationPending, publicationRegistration } from './reentry.js' import { publicationAdmission } from './registrar.js' import { loadScopes } from '../adapter/dsh/host.js' +import { RAW_OFFICIAL_REPLACEMENTS } from '../capabilities/replacements.js' +import type { ApiName } from '../ptc/config.js' +import type { PtcServability } from './ptc.js' import type { PublicationGate } from './types.js' +/** Bash preparation stages that no retry of the same call can repair; each lifts the official fallback (#962). */ +const UNRECOVERABLE_BASH_STAGES = new Set(['bash-runtime-missing', 'bash-ipc-mismatch', 'bash-lifecycle-lock-busy', 'bash-cache-unusable']) + +/** + * Whether one settled replacement-counterpart value reports a runtime failure that + * retrying the same call cannot repair (#962). Only Bash preparation stages without a + * retry remedy qualify: a staging interruption is repaired by the next call, and lifting + * the shadow for it would flap between two tool sets. The stage is the stable second + * segment of the model-facing EXECUTOR_UNAVAILABLE text (src/bash-runtime-failure.ts); + * bash-runtime-incompatible keeps its own complete message without that prefix, so its + * stage name inside the message is its identifier. PTC Outcome failures carry a structured + * error object, never this text, so they never classify. + * @param value - one settled tool-call value. + * @returns whether the official fallback must be restored for the calling agent. + */ +export function unrecoverableCounterpartRuntime(value: unknown): boolean { + if (value === null || typeof value !== 'object') return false + const error = (value as { error?: unknown }).error + if (typeof error !== 'string') return false + const stage = /^EXECUTOR_UNAVAILABLE:\s*([a-z-]+):/.exec(error) + if (stage !== null) return UNRECOVERABLE_BASH_STAGES.has(stage[1]!) + return error.includes('bash-runtime-incompatible') +} + interface Shadow { source: ToolDefinition; definition: ToolDefinition; dispose(): void } interface Entry { shadows: Map; probe: object } @@ -16,11 +43,13 @@ export interface ReplacementPublication { targets: ReadonlyMap gate: PublicationGate keepOfficial(): boolean + /** PTC projection reachability; absent for native consumers, which keep the old path. */ + ptc?: PtcServability } /** Select the official fallback and its counterpart in the same host resolve. */ export function mountReplacements(root: Context, publication: ReplacementPublication): void { - const { id: mode, targets, gate, keepOfficial } = publication + const { id: mode, targets, gate, keepOfficial, ptc } = publication if (targets.size === 0) return root.inject(['tools'], async ctx => { const tools = ctx.get('tools') @@ -28,8 +57,48 @@ export function mountReplacements(root: Context, publication: ReplacementPublica const { scopeChainOf, bindScopeParent } = await loadScopes() const sources = new WeakSet() const entries = new Map() + // One line per agent, name and state: mixed states on one agent are normal + // (a resident read is hidden while glob/grep stay official), and a + // per-activation warning would flood the log. + const ptcState = new WeakMap>() + const probeFailures = new WeakSet() + const notePtc = (agent: Agent, name: string, state: string): boolean => { + let states = ptcState.get(agent) + if (!states) { states = new Map(); ptcState.set(agent, states) } + if (states.get(name) === state) return false + states.set(name, state) + ctx.logger.info(`dsh-ops: ${mode} PTC official-tool hiding for ${name}: ${state}`) + return true + } + // One presentation probe per activation: the predicate runs once per target + // name inside the same resolve, and every call assembles the prompt. + const ptcPresentation = new WeakMap>() + const ptcNow = (activation: ToolActivationContext): Promise => { + let pending = ptcPresentation.get(activation) + if (!pending) { pending = ptc!.isPtc(activation); ptcPresentation.set(activation, pending) } + return pending + } + /** The PTC hiding verdict shared by the shadow `when` and the guidance assembly below. */ + const hiddenByPtc = (agent: Agent, official: string, counterpart: string): { hidden: boolean; state: string } => { + const raw = (RAW_OFFICIAL_REPLACEMENTS as Record)[official] + // A counterpart the publisher registered (even before the model selects it) + // proves a replacement exists; the value of its admission is not consulted, + // so loading the SDK cannot change this decision. + const served = ptc!.servable(agent) + const published = tools.get(counterpart, agent) + const candidate = served !== undefined && served.has(raw) + const hidden = candidate && published !== undefined && publicationAdmission(published) !== undefined + const state = served === undefined ? 'unattached' + : !candidate ? 'not-servable' + : hidden ? 'hidden' + : 'counterpart-missing' + return { hidden, state } + } // Disposed agents and agents with unlinked host scopes must not receive shadows. const inactive = new WeakSet() + // Agents whose counterpart reported an unrecoverable runtime failure keep their official tools (#962). + const broken = new WeakSet() + const counterparts = new Set(targets.values()) let stopped = false, mutating = false let scheduled: ReturnType | undefined const defer = (): void => { @@ -74,7 +143,28 @@ export function mountReplacements(root: Context, publication: ReplacementPublica // Own registrations bypass restrictions in the host view. The empty descendant // probe makes our shadow inherited, so even agent-local restrictions still apply. if (tools.get(official, entry.probe) !== definition || sourceFor(activation.agent, official) !== source) return false - if (keepOfficial() || !gate.allows(activation.agent)) return true + if (keepOfficial() || broken.has(activation.agent) || !gate.allows(activation.agent)) return true + if (ptc && Object.hasOwn(RAW_OFFICIAL_REPLACEMENTS, official)) { + let ptcNowValue: boolean + try { ptcNowValue = await ptcNow(activation) } + catch (error) { + // A probe failure carries no replacement proof. Verified host behaviour: + // a rejected `when` fails the whole step resolve, so the documented + // fail-open contract must be enforced here. Keep the official tool. + notePtc(activation.agent, official, 'probe-failed') + if (!probeFailures.has(activation.agent)) { + probeFailures.add(activation.agent) + ctx.logger.warn(`dsh-ops: ${mode} PTC presentation probe failed for ${official}: ${String(error)}`) + } + return true + } + if (ptcNowValue) { + // Explicit implementation faults below must surface, not masquerade as a probe failure. + const { hidden, state } = hiddenByPtc(activation.agent, official, counterpart) + notePtc(activation.agent, official, state) + return !hidden + } + } const replacement = tools.get(counterpart, activation.agent) if (!replacement) return true const admission = publicationAdmission(replacement) @@ -127,6 +217,62 @@ export function mountReplacements(root: Context, publication: ReplacementPublica const entry = entries.get(agent) if (entry) { entries.delete(agent); lift(agent, entry) } }) + // Fail open (#962): a counterpart that settled with an unrecoverable runtime failure + // stops hiding the official tool from the agent's next model step. The latch is per + // agent: the payload is shared, but each agent's own accepted call is the evidence + // that its shadow is a dead end, and one agent's repair never guesses for another. + ctx.on('tools/post-execute', async (exec, result, next) => { + const decision = await next() + if (decision.kind !== 'accept' || !exec.agent || !counterparts.has(exec.name)) return decision + const value = (result as { value?: unknown } | undefined)?.value + if (!unrecoverableCounterpartRuntime(value) || broken.has(exec.agent)) return decision + broken.add(exec.agent) + ctx.logger.warn(`dsh-ops: ${mode} replacement counterpart ${exec.name} reported an unrecoverable runtime failure; the official tools stay available for this agent`) + return decision + }) + // Guidance written for a hidden name is a dead pointer: the host renders its + // `tool:` sections from registry visibility, and the shadow keeps the name + // resolvable, so the text survives hiding. The presentation waterfall is the + // supported scope extension point: it retires such sections for the assembly at + // hand, and one note names the executable counterpart for a kept official whose + // section pointed at a hidden name. Restoring the official entry (row disposed, + // switch off, probe failure, unattached projection) restores the text untouched. + ctx.on('system-prompt/assemble', async (_assembly, context, next) => { + const result = await next() + if (stopped || !ptc) return result + // The publisher's own assembly handler uses the same presentation predicate; + // sections are evaluated before this waterfall, so this cannot recurse. + if (!result.sections.some(section => section.name === 'tools:ptc-only' && Boolean(section.text.trim()))) return result + const agent = context.scope as Agent | undefined + if (!agent || keepOfficial() || !gate.allows(agent)) return result + const hidden: string[] = [] + for (const [official, counterpart] of targets) { + if (!Object.hasOwn(RAW_OFFICIAL_REPLACEMENTS, official)) continue + if (hiddenByPtc(agent, official, counterpart).hidden) hidden.push(official) + } + if (hidden.length === 0) return result + const drop = new Set(hidden.map(name => 'tool:' + name)) + // A kept official's own section may still teach a hidden name (the pinned host + // grep text cross-references read). The text belongs to its provider and is not + // rewritten here; the section retires and the note below supplies the alternative. + const pointer = new RegExp('\\b(?:' + hidden.map(name => name.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')).join('|') + ')\\b') + const crossReferences: string[] = [] + for (const [official] of targets) { + if (hidden.includes(official) || !Object.hasOwn(RAW_OFFICIAL_REPLACEMENTS, official)) continue + const section = result.sections.find(candidate => candidate.name === 'tool:' + official) + if (section && section.text && pointer.test(section.text)) { drop.add('tool:' + official); crossReferences.push(official) } + } + const note = crossReferences.length === 0 ? undefined + : `Official ${hidden.join(', ')} ${hidden.length === 1 ? 'is' : 'are'} hidden in this scope; call ${hidden.map(name => targets.get(name)!).join(', ')} instead. ${crossReferences.join(', ')} ${crossReferences.length === 1 ? 'stays' : 'stay'} callable through the tools:sdk declaration.` + let noted = false + const sections = result.sections.flatMap(section => { + if (!drop.has(section.name)) return [section] + if (noted || note === undefined) return [] + noted = true + return [{ name: `dsh-ops:replaced-official:${mode}`, text: note, interpolate: false as const }] + }) + return { ...result, sections } + }) ctx.on('session/event', (_session, event) => { if ((event as { type: string }).type === 'agent-preset/selected') reconcile() }) if (gate.watch) ctx.effect(() => gate.watch!(() => reconcile()), `dsh-ops: ${mode} replacement observer`) // A held step must lose a newly restricted fallback just as it loses an inherited official. diff --git a/src/session-shells.ts b/src/session-shells.ts index 221cd305..b69910db 100644 --- a/src/session-shells.ts +++ b/src/session-shells.ts @@ -12,7 +12,7 @@ import { BashArgumentError } from './shells/argument-error.js' import { MODES } from './permissions.js' import { hostBashDefinition } from './shells/host-tool.js' import { withToolChange } from './publication/changes.js' -import { publicationRegistration } from './publication/reentry.js' +import { publicationPending, publicationReady, publicationRegistration } from './publication/reentry.js' import { startSessionBash } from './shells/session-execution.js' import { resolveSessionDirectory } from './adapter/dsh/working-directory.js' @@ -134,6 +134,10 @@ export function bashRefusalValue(error: unknown): ShellResult { interface Entry { agent: Agent; ctx?: Context; fiber?: Fiber; dispose?: () => void; definition?: HostShellDefinition published?: boolean; generation?: number + /** True while the refusal queue owns an attempt that has not settled (#842). */ + pending?: boolean + /** Generation of the last failed attempt; one report per generation (#579). */ + failed?: number } // Preset consumers share an Agent's own registry layer. A newly eligible @@ -201,28 +205,104 @@ export function mountSessionBash(ctx: Context, config: Pick { + /** Whether the host refused this very tool name in this very registry layer. */ + const nameConflict = (error: unknown): boolean => /is already registered/i.test(errorMessage(error)) + + /** + * Publish one bash layer for an entry. + * + * An owned layer that is still allowed stays authoritative when two of this + * plugin's rows briefly both serve one agent (a mode or preset transition + * overlaps): re-registering the same name in the same layer can only fail + * (#842), so the later row stands by until the owner withdraws or unloads. + * @param entry - the per-agent entry the layer belongs to. + * @param definition - the definition to register. + * @param attemptGeneration - the reconcile generation this attempt belongs to. + */ + const publish = (entry: Entry, definition: HostShellDefinition, attemptGeneration: number) => { if (!entry.ctx) return const previous = bashOwners.get(entry.agent) - if (previous && previous.entry !== entry && !previous.allowed()) previous.clear() - // A failed publication must leave the session with the definition that last worked rather than - // with no bash tool at all (#30). `entry.definition` survives `clear()`, so it holds the last - // definition registered successfully for this entry. + if (previous && previous.entry !== entry && previous.allowed()) return + if (previous && previous.entry !== entry) previous.clear() + let dispose: () => void try { - entry.dispose = publicationRegistration(entry.ctx, () => withToolChange([entry.agent], () => entry.ctx!.tools.register(definition))) - entry.definition = definition - bashOwners.set(entry.agent, { entry, allowed: () => allowed(entry.agent), clear: () => clear(entry) }) + dispose = publicationRegistration(entry.ctx, () => withToolChange([entry.agent], () => entry.ctx!.tools.register(definition))) } catch (error) { - ctx.logger?.warn?.(`dsh-ops: bash publication failed: ${errorMessage(error)}`) - if (entry.definition === undefined) return - try { - entry.dispose = publicationRegistration(entry.ctx, () => withToolChange([entry.agent], () => entry.ctx!.tools.register(entry.definition!))) - bashOwners.set(entry.agent, { entry, allowed: () => allowed(entry.agent), clear: () => clear(entry) }) - } - catch (restoreError) { ctx.logger?.warn?.(`dsh-ops: bash restoration failed: ${errorMessage(restoreError)}`) } + reportRejected(entry, attemptGeneration, definition, error) + return + } + entry.dispose = dispose + bashOwners.set(entry.agent, { entry, allowed: () => allowed(entry.agent), clear: () => clear(entry) }) + admit(entry, dispose, definition, attemptGeneration) + } + + /** + * Mark one registration attempt live, or report and restore on rejection. + * + * A queued attempt exists only once the host admits it: the entry stays + * pending, never published, so a later rejection cannot leave a session + * looking served while its layer is gone (#873). + * @param entry - the entry the attempt belongs to. + * @param dispose - the attempt's queue handle; also its settle identity. + * @param definition - the definition the attempt registers. + * @param attemptGeneration - the reconcile generation of the attempt. + */ + const admit = (entry: Entry, dispose: () => void, definition: HostShellDefinition, attemptGeneration: number) => { + const live = () => { + entry.definition = definition + entry.published = true + entry.generation = attemptGeneration + entry.failed = undefined + } + if (!publicationPending(dispose)) { live(); return } + entry.pending = true + void publicationReady(dispose).then(() => { + // A superseded attempt no longer speaks for this entry. + if (entry.dispose !== dispose) return + entry.pending = false + live() + }, error => { + if (entry.dispose !== dispose) return + entry.pending = false + entry.published = undefined + entry.generation = undefined + if (bashOwners.get(entry.agent)?.entry === entry) bashOwners.delete(entry.agent) + reportRejected(entry, attemptGeneration, definition, error) + }) + } + + /** + * Report one failed registration, then restore the definition that last + * worked once — but never the identical name the layer just refused. + * + * The attempt is latched to its generation: later passes at the same + * generation skip the entry instead of storming the registry (#579); a turn + * boundary or an authority change clears the latch before any retry. + * @param entry - the entry whose attempt failed. + * @param attemptGeneration - the reconcile generation of the attempt. + * @param attempted - the definition the host refused. + * @param error - the host failure. + */ + const reportRejected = (entry: Entry, attemptGeneration: number, attempted: HostShellDefinition, error: unknown) => { + entry.failed = attemptGeneration + ctx.logger?.warn?.(`dsh-ops: bash publication failed: ${errorMessage(error)}`) + // Re-registering keeps the session on the definition that last worked + // (#30) — unless the registry refused this very name in this layer: the + // identical retry used to pair a second failure with the first (#842). + if (entry.definition === undefined || entry.definition === attempted || nameConflict(error)) return + let restore: () => void + try { + restore = publicationRegistration(entry.ctx!, () => withToolChange([entry.agent], () => entry.ctx!.tools.register(entry.definition!))) + } catch (restoreError) { + ctx.logger?.warn?.(`dsh-ops: bash restoration failed: ${errorMessage(restoreError)}`) + return } + entry.dispose = restore + admit(entry, restore, entry.definition!, attemptGeneration) } const reconcile = (refreshDefinitions = false) => { if (refreshDefinitions) ++generation @@ -233,6 +313,10 @@ export function mountSessionBash(ctx: Context, config: Pick gate.watch!(() => { void scheduleReconcile(true) }), 'dsh-ops: bash mode observer') ctx.inject(['sandbox'], () => { scheduleReconcile() }) diff --git a/src/settings/merged.ts b/src/settings/merged.ts index 4818859f..8553a8fa 100644 --- a/src/settings/merged.ts +++ b/src/settings/merged.ts @@ -232,6 +232,12 @@ export function effectiveMode(state: FeatureState, global: GlobalSwi hidePwsh: flag(raw, 'hidePwsh', state.config.hidePwsh) && global.hidePwsh && !pwsh7Link, hideOfficialTools: flag(raw, 'hideOfficialTools', true) && global.hideOfficialTools, dynamicTools: flag(raw, 'dynamicTools', true) && global.dynamicTools, + // Sign-off item 2 (docs/design-decisions-20261119.md), taken conservatively + // per #979 §4: the explicit full tier is its own opt-in and never overrides + // the bundle switch. While the switch is off the effective slice says + // on-demand, so the row keeps its legacy resident/alwaysLoaded surface; the + // saved choice stays untouched in state.raw and re-applies when switched on. + ...(raw.disclosurePolicy === 'full' && !global.dynamicTools ? { disclosurePolicy: 'on-demand' } : {}), } } @@ -252,6 +258,12 @@ const MODE_SWITCHES: Record = { function modeSection(mode: ModeKey): Field { return { type: 'object', dict: { enabled: boolean('Apply this mode to its sessions.'), ...(mode === 'minimal' ? {} : { residentTools: boolean('Publish baseline file tools independently of dynamic loading.') }), + // Only the PTC rows own a disclosure tier; the key is volatile because the + // row page's disclosure card writes it, exactly like the executor card + // writes bashPath. Absent stays the on-demand default (#989). + ...(mode === 'ptc' || mode === 'ops-ptc' + ? { disclosurePolicy: editable('string', 'Optional explicit first-round disclosure tier selected on the row page.') } + : {}), bashPath: editable('string', 'Optional explicit Bash executable selected on the runtime page.'), ...MODE_SWITCHES } } } diff --git a/src/shells/host-contract.ts b/src/shells/host-contract.ts index 44ae8c79..7ba10719 100644 --- a/src/shells/host-contract.ts +++ b/src/shells/host-contract.ts @@ -41,7 +41,12 @@ export function hostFailure(message: string, timeoutMs = 120_000, effects: 'none export function renderHostBash(value: HostBashResult): string { const directory = value.resolvedFrom === undefined ? '' : `[cwd: resolvedFrom=${JSON.stringify(value.resolvedFrom)}; cwdSource=${JSON.stringify(value.cwdSource)}]\n` if (value.kind === 'background') return directory + `started background job ${value.jobId} (${JSON.stringify(value.label)})` - if (value.kind === 'promoted') return directory + `${value.output || '(no output)'}\n[still running after ${value.timeoutMs}ms; moved to background job ${value.jobId} (${JSON.stringify(value.label)})]\nUse job_output to collect output and job_kill to stop the command.` + // The host owns what a job kill reaches (its subprocess and job layer), so this + // render can only be honest about it: killing the job terminates the whole process + // tree, including daemons the command started and left attached to its pipes, and + // the model has to know that before job_kill takes out an adb server or a dev + // watcher that was supposed to survive (#814). + if (value.kind === 'promoted') return directory + `${value.output || '(no output)'}\n[still running after ${value.timeoutMs}ms; moved to background job ${value.jobId} (${JSON.stringify(value.label)})]\nCollect output with job_output, or stop the command with job_kill. job_kill terminates the whole process tree, including daemons this command started and left attached to it (an adb server, a dev-server watcher); stop such daemons on their own channel first, or start them detached, so a job kill cannot reach them. Daemon-like or long-lived work belongs in run_in_background from the start.` const { exitCode, signal, ...rest } = value const body = renderBashResult(rest) const markers = [] diff --git a/src/shells/session-execution.ts b/src/shells/session-execution.ts index c97981a3..6467c85f 100644 --- a/src/shells/session-execution.ts +++ b/src/shells/session-execution.ts @@ -32,7 +32,7 @@ export function startSessionBash({ ctx, bash, active }: { ctx: ShellContext; bas assertAuthority() try { const runtime = native ? await prepareNiubash(bash.file, dshEnv.PATH ?? dshEnv.Path ?? process.env.PATH, signal, - { workspaceRoot: typeof policy.workspaceRoot === 'string' ? policy.workspaceRoot : undefined }) + { workspaceRoot: typeof policy.workspaceRoot === 'string' ? policy.workspaceRoot : undefined, mode: policy.mode }) : policy.mode === 'danger-full-access' ? { file: bash.file } : confinedBash(bash.file, { source: bash.source }) if (native) for (const key of Object.keys(env)) if (['PATH', 'NIU_ROOT', 'WINUXCMD_PATH', 'NIU_SKIP_WINUXCMD_ACTIVATION'].includes(key.toUpperCase())) delete env[key] Object.assign(env, native ? runtime.env : { ...runtime.env, ...dshEnv }) @@ -57,7 +57,10 @@ export function startSessionBash({ ctx, bash, active }: { ctx: ShellContext; bas (!rule.allowedExitCodes || (value.exitCode != null && rule.allowedExitCodes.includes(value.exitCode))) && stderr.split(/\r?\n/).filter(line => !rule.informationalLines?.some(info => info.toLowerCase() === line.toLowerCase())) .some(line => rule.fatalSignatures?.some(signature => line.toLowerCase().includes(signature.toLowerCase())))) - const unavailable = runnerFailed || /couldn't create signal pipe|fatal error.*win32 error|sandbox_unavailable/i.test(stderr) + // msys-2.0.dll reports a shared-section conflict as "CreateFileMapping %W, %E." + // without any "fatal error" prefix: an unrestricted MSYS process created the per-user + // section first with no restricted-logon ACE, so the restricted token cannot reopen it (#409). + const unavailable = runnerFailed || /couldn't create signal pipe|createfilemapping\b.*win32 error|fatal error.*win32 error|sandbox_unavailable/i.test(stderr) const denied = !unavailable && value.exitCode !== 0 && confinement?.denialSignatures?.some(signature => stderr.toLowerCase().includes(signature.toLowerCase())) return withBashStatus({ ...value, ...(unavailable ? { error: 'EXECUTOR_UNAVAILABLE: the sandbox runtime failed before the command ran; this is not a user authorization denial. Restore the runtime or confinement provider; changing the DSH selector is not a repair. Do not bypass confinement.' } : {}), diff --git a/test/batching-guidance.test.mjs b/test/batching-guidance.test.mjs index b562b738..c97d5b98 100644 --- a/test/batching-guidance.test.mjs +++ b/test/batching-guidance.test.mjs @@ -7,7 +7,7 @@ import fs from 'node:fs' import path from 'node:path' import { assert, test, report, PACKAGE_ROOT } from './lib/harness.mjs' -import { BATCHING_GUIDANCE_ID, BATCHING_GUIDANCE_REVISION, loadBatchingGuidance } from '../lib/ptc/batching-guidance.js' +import { BATCHING_GUIDANCE_ID, BATCHING_GUIDANCE_REVISION, LOAD_PROSE_ANCHOR, LOAD_SECTION_HEADING, LOAD_TABLE_ROW_PREFIX, loadBatchingGuidance, renderBatchingGuidance } from '../lib/ptc/batching-guidance.js' /** The editable guidance file shipped at the package root, outside the build output. */ const GUIDANCE_FILE = path.join(PACKAGE_ROOT, 'prompts', 'batching-guidance.md') @@ -20,7 +20,7 @@ await test('the shipped guidance is English Markdown with its host-source identi } assert.equal(BATCHING_GUIDANCE_ID, 'batching-guidance') // A wording or format change is a product change and must bump the revision. - assert.equal(BATCHING_GUIDANCE_REVISION, '4') + assert.equal(BATCHING_GUIDANCE_REVISION, '7') }) await test('the guidance loader returns exactly the shipped Markdown file', () => { @@ -106,4 +106,151 @@ await test('the teacher trajectory pins the first-line plan and a runnable load assert.ok(text.length > 2500 && text.length < 9000, 'guidance stays bounded: ' + text.length) }) +await test('the trim removed repeated prose and the anti-pattern demo', () => { + // The first-round lesson enters message history, so it must not repeat itself: the + // anti-pattern section and the trailing prose that restated Step 1 are gone, while + // the four executable trajectories and their contracts stay byte-identical. + const text = loadBatchingGuidance() + assert.equal(text.includes('## Anti-pattern, and its rewrite'), false, 'the anti-pattern demo is gone') + assert.equal(text.includes('The search is the dependency, so it is awaited first'), false) + assert.equal(text.includes('The revision and the context are independent'), false) + assert.equal(text.includes('A loop of dispatches is batched work'), false) +}) + +await test('the trimmed lesson keeps the four trajectories and restores the two rules', () => { + const text = loadBatchingGuidance() + for (const heading of [ + '## Trajectory 1 - reconnaissance', '## Trajectory 2 - search, then read the hits', + '## Trajectory 3 - a verified edit', '## Trajectory 4 - paging', + ]) assert.ok(text.includes(heading), heading) + assert.ok(text.includes('**Load only missing APIs**'), 'load only what the SDK lacks is a core rule') + assert.ok(text.includes('**Recheck the SDK after a mode change**'), 'the post-mode-change recheck is a core rule') +}) + +await test('the shipped lesson measures its declared trimmed size', () => { + // The trim record in the PR declares this size; a wording edit must bump the + // revision and this number together. The 84 B delivery prefix is counted separately. + assert.equal(Buffer.byteLength(loadBatchingGuidance(), 'utf8'), 7538) +}) + +await test('a scope with the loader receives the shipped file unchanged', () => { + // The default path must not pay for a branch it never needs: byte identity is the contract. + const text = loadBatchingGuidance() + assert.equal(renderBatchingGuidance(text, { loader: true, resident: [] }), text) + // An unreadable file stays empty, so delivery is skipped instead of failing a tool result. + assert.equal(renderBatchingGuidance('', { loader: false, resident: ['ops_glob'], loadable: [] }), '') +}) + +await test('a loader-less scope is never taught to load', () => { + const rendered = renderBatchingGuidance(loadBatchingGuidance(), { loader: false, resident: ['ops_glob', 'ops_grep'], loadable: [] }) + // The defect this guards: the lesson named a call that cannot succeed in this scope. + assert.equal(rendered.includes('ops_load_sdk'), false, 'the unreachable loader is gone') + assert.equal(rendered.includes(LOAD_SECTION_HEADING), false, 'the load section is replaced') + // The Step 1 prose taught the same entry without naming the loader tool, so its + // wording must not survive beside a section that says there is no loader. + assert.equal(rendered.includes('SDK loading'), false, 'the Step 1 load prose is gone') + assert.equal(rendered.includes('separate program'), false, 'no load entry is advertised') + assert.ok(rendered.includes('Run independent calls under Promise.all'), 'the shared split lesson survives') + assert.ok(rendered.includes('## Step 3 - call what is published'), 'the step that remains is named') + assert.ok(rendered.includes('ops_glob') && rendered.includes('ops_grep'), 'the resident names are listed instead') + // Everything shared with the loader path survives verbatim: the lesson is not truncated. + for (const kept of ['# Batch tool calls', '## Step 1 - split by dependency', '## Step 2 - reconcile', '## Common mistakes', '// plan:']) { + assert.ok(rendered.includes(kept), kept) + } +}) + +await test('a scope that publishes nothing says so', () => { + const rendered = renderBatchingGuidance(loadBatchingGuidance(), { loader: false, resident: [] }) + assert.equal(rendered.includes('ops_load_sdk'), false) + assert.ok(rendered.includes('publishes no OPS API and no loader'), 'the empty surface is stated') + assert.equal(rendered.includes('## Step 3 - call what is published'), false, 'no resident list is invented') +}) + +await test('the replacement survives a resident list that names the loader', () => { + // The list is generated after the filter, so the filter can never eat it. + const rendered = renderBatchingGuidance(loadBatchingGuidance(), { loader: false, resident: ['ops_load_sdk', 'ops_glob'], loadable: [] }) + assert.ok(rendered.includes('ops_load_sdk') && rendered.includes('ops_glob'), 'the list is rendered whole') +}) + +await test('a drifted file keeps its structure when the anchor is gone', () => { + const drifted = '# Batch tool calls\n\nintro\n## Step 1 - split by dependency\nbody mentions ops_load_sdk once\n' + const out = renderBatchingGuidance(drifted, { loader: false, resident: ['ops_glob'], loadable: [] }) + assert.ok(out.startsWith('# Batch tool calls'), 'the H1 stays first') + assert.ok(out.indexOf('call what is published') < out.indexOf('## Step 1'), 'the step precedes the first section') + assert.equal(out.includes('ops_load_sdk'), false, 'no unreachable mention survives') +}) + +await test('a renamed load section does not survive beside its replacement', () => { + // A drifted file keeps the old lesson under a different heading; two contradicting + // Step 3 sections would teach both loading and direct calls. + const drifted = '# Batch tool calls\n\nintro\n## Step 3 - loading tools\nload prose here\n## Step 1 - split\nbody\n' + const out = renderBatchingGuidance(drifted, { loader: false, resident: ['ops_glob'], loadable: [] }) + assert.equal((out.match(/## Step 3/g) ?? []).length, 1, 'exactly one Step 3 remains') + assert.equal(out.includes('load prose here'), false, 'the old load section is dropped') + assert.ok(out.startsWith('# Batch tool calls'), 'the H1 stays first') +}) + +await test('a fenced example that teaches loading is dropped whole', () => { + // Deleting one line inside a fence would leave a half example that calls a name the + // scope cannot reach, so the whole block goes with it. + const drifted = '# Batch tool calls\n\nintro\n```js\nconst sdk = await tools.ops_load_sdk({ tools: ["edit_file"] })\nawait sdk.ops_edit_file({ path: "a" })\n```\n## Step 1 - split\nbody\n' + const out = renderBatchingGuidance(drifted, { loader: false, resident: ['ops_glob'], loadable: [] }) + assert.equal(out.includes('sdk.ops_edit_file'), false, 'the dangling call does not survive') + assert.equal(out.includes('ops_load_sdk'), false, 'the load call does not survive') + assert.equal((out.match(/```/g) ?? []).length, 0, 'no half-open fence is left behind') +}) + +await test('only a heading that means loading loses its section', () => { + // A substring match would swallow "Payload" and drop a valid section with it. + const drifted = '# Batch tool calls\n## Payload budget\nkeep me\n## Step 3 - loading tools\nload prose\n' + const out = renderBatchingGuidance(drifted, { loader: false, resident: ['ops_glob'], loadable: [] }) + assert.ok(out.includes('## Payload budget'), 'the ordinary section survives') + assert.ok(out.includes('keep me'), 'its body survives too') + assert.equal(out.includes('load prose'), false, 'the load section still goes') +}) + +await test('a heading inside a fence is example text, not a boundary', () => { + // The fenced line must not cut the load section short, or half of it survives. + const drifted = '# Batch tool calls\n## Step 3 - load before you call\nprose one\n```\n## not a heading\n```\nprose two\n## Trajectory 1 - next\nbody\n' + const out = renderBatchingGuidance(drifted, { loader: false, resident: ['ops_glob'], loadable: [] }) + assert.equal(out.includes('prose two'), false, 'the whole section is replaced') + assert.ok(out.includes('## Trajectory 1 - next'), 'the next real section is kept') +}) + +await test('a fenced copy of the heading is not the anchor', () => { + // Taking the quoted line for the anchor would slice the document inside the fence. + const drifted = '# Batch tool calls\n```\n## Step 3 - load before you call\n```\nreal prose\n' + const out = renderBatchingGuidance(drifted, { loader: false, resident: ['ops_glob'], loadable: [] }) + assert.ok(out.includes('real prose'), 'the real body survives the fallback') + assert.ok(out.startsWith('# Batch tool calls'), 'the H1 stays first') +}) + +await test('the shipped file still carries the anchors the renderer rewrites', () => { + // The loader-less rendering edits prose by anchor: moving either one must fail here, not at runtime. + const lines = loadBatchingGuidance().split('\n') + assert.ok(lines.some(line => line.trim() === LOAD_SECTION_HEADING), 'the load heading is present') + assert.ok(lines.some(line => line.startsWith(LOAD_TABLE_ROW_PREFIX)), 'the load row is present') + // The Step 1 prose must sit on its own line, or the prose filter would cut a shared + // line short and leave the split lesson truncated. + assert.ok(lines.some(line => line.startsWith(LOAD_PROSE_ANCHOR)), 'the load prose is a whole line') +}) + +await test('a loader-less scope whose only file API is inspect states the paths it needs', () => { + // The acceptance case: with no loader, no listing and no search, a directory cannot be + // probed, so the lesson must say where the paths come from instead of letting the model + // spend a round trip on a call that cannot succeed. + const rendered = renderBatchingGuidance(loadBatchingGuidance(), { loader: false, resident: ['ops_inspect_local_file', 'ops_stat'], loadable: [] }) + assert.ok(rendered.includes('ops_inspect_local_file'), 'the resident reader is listed') + assert.match(rendered, /rejects directories/, 'the directory rejection is stated') + assert.match(rendered, /concrete file paths/, 'the exact-path requirement is stated') + assert.equal(rendered.includes('ops_load_sdk'), false, 'the loader stays gone') +}) + +await test('the path statement stays silent while a listing or search API is published', () => { + // A scope that can list or search discovers paths itself; the caveat would read as noise. + const rendered = renderBatchingGuidance(loadBatchingGuidance(), { loader: false, resident: ['ops_glob', 'ops_inspect_local_file'], loadable: [] }) + assert.doesNotMatch(rendered, /rejects directories/, 'a scope that can list needs no caveat') + assert.ok(rendered.includes('ops_glob') && rendered.includes('ops_inspect_local_file'), 'the published pair is listed') +}) + report('batching-guidance') diff --git a/test/batching-trigger.test.mjs b/test/batching-trigger.test.mjs index ce60e224..303226a2 100644 --- a/test/batching-trigger.test.mjs +++ b/test/batching-trigger.test.mjs @@ -11,6 +11,9 @@ import { assert, test, report } from './lib/harness.mjs' import { analyzeProgram, parsePlan, violationsOf, createProgramLatch, noticeText, CATEGORY_PRIORITY, PLAN_DIRECTIVE, THIN_CODE_CHARS } from '../lib/ptc/batching-guidance.js' +/** A scope that publishes the loader and keeps the pool loadable. */ +const LOADER = { loader: true, resident: [], loadable: ['ops_glob', 'ops_grep', 'ops_read_files', 'ops_inspect_local_file', 'ops_edit_file'] } + await test('the plan comment is read from the first physical line only', () => { assert.deepEqual(parsePlan('// plan: read the config; read the test'), ['read the config', 'read the test']) // The full-width semicolon separates entries as well; both are typed on a CJK layout. @@ -89,7 +92,7 @@ await test('stdin filters do not accuse command output of bypassing file APIs', assert.equal(analyzeProgram('await tools.ops_bash({ description: "grep file", command: "git diff" })').bypassesFileTools, false) assert.equal(analyzeProgram('const example = "tools.ops_bash({ command: \'cat file\' })"; return example').dispatchSites, 0) assert.equal(analyzeProgram('// tools.ops_bash({ command: "cat file" })').bypassesFileTools, false) - assert.doesNotMatch(noticeText('bypass', shape('cat file')), /Only .* resident|load the others/) + assert.doesNotMatch(noticeText('bypass', shape('cat file'), LOADER), /Only .* resident|load the others/) }) await test('a program with no dispatch site earns no category', () => { @@ -99,7 +102,7 @@ await test('a program with no dispatch site earns no category', () => { assert.deepEqual(violationsOf(analyzeProgram(code)), { plan: false, bypass: false, shape: false }) } // The latch stays silent, so the model is never accused of a shape it did not take. - const latch = createProgramLatch(), agent = {} + const latch = createProgramLatch(() => LOADER), agent = {} assert.equal(latch.observe(agent, 'return 1'), undefined) assert.equal(latch.observe(agent, '// plan: read nothing\n// a note\nreturn 1'), undefined) assert.equal(latch.stats(agent).total, 0, 'no category was spent') @@ -125,6 +128,66 @@ await test('each of the three categories is decided independently', () => { assert.equal(violationsOf(long).shape, false) }) +await test('waiting on an agent is a dependency barrier, read only from a real dispatch site (#893)', () => { + // Dot and bracket calls to wait_agent are both waits, so no shape notice. + for (const code of [ + '// plan: wait for the verification agent\nreturn await tools.wait_agent({ timeout_ms: 30000 })', + '// plan: wait for the verification agent\nreturn await tools["wait_agent"]({ timeout_ms: 30000 })', + ]) { + const shape = analyzeProgram(code) + assert.equal(shape.dispatchSites, 1, code) + assert.equal(shape.isWaiting, true, code) + assert.deepEqual(violationsOf(shape), { plan: false, bypass: false, shape: false }, code) + } + // The name inside a comment or a message string is not a call and exempts nothing. + for (const code of [ + 'const r = await tools.ops_read_files({ files: [{ path: "a" }] })\n// tools.wait_agent({})\nreturn r', + 'const r = await tools.ops_read_files({ files: [{ path: "a" }] })\nreturn { r, note: "tools.wait_agent( later" }', + 'const r = await tools.ops_read_files({ files: [{ path: "a" }] })\n// job_output({ job_id: "b", wait: true })\nreturn r', + 'const r = await tools.ops_read_files({ files: [{ path: "a" }] })\nreturn { r, note: "job_output( or wait: true" }', + ]) { + const shape = analyzeProgram(code) + assert.equal(shape.isWaiting, false, code) + assert.equal(violationsOf(shape).shape, true, code) + } + // A wait does not exempt the plan or bypass checks. + assert.equal(violationsOf(analyzeProgram('return await tools.wait_agent({ timeout_ms: 1 })')).plan, true) + const shellWait = analyzeProgram('await tools.ops_bash({ description: "list", command: "ls -la" })\nreturn await tools.wait_agent({})') + assert.equal(violationsOf(shellWait).bypass, true) +}) + +await test('only a single dispatch site earns the single-operation shape notice (#893)', () => { + // A search whose result picks the next read cannot be batched ahead of time. + const dependent = analyzeProgram('// plan: search; read the matching file\nconst hits = await tools.grep({ pattern: "needle" })\nreturn await tools.read({ file_path: hits.matches[0].path })') + assert.equal(dependent.dispatchSites, 2) + assert.equal(dependent.isWaiting, false) + assert.deepEqual(violationsOf(dependent), { plan: false, bypass: false, shape: false }) + // Bracket sites count the same way. + assert.equal(violationsOf(analyzeProgram('const a = await tools["ops_stat"]({ path: "a" })\nreturn await tools["ops_stat"]({ path: a.path })')).shape, false) + // Two sites do not exempt the plan or bypass checks. + const twoNoPlan = analyzeProgram('const a = await tools.ops_bash({ description: "list", command: "ls -la" })\nreturn await tools.ops_stat({ path: "a" })') + assert.deepEqual(violationsOf(twoNoPlan), { plan: true, bypass: true, shape: false }) + // A commented-out second call does not lift a real single site out of the notice. + assert.equal(violationsOf(analyzeProgram('// plan: read\nconst r = await tools.read({ file_path: "a" })\n// await tools.read({ file_path: "b" })\nreturn r')).shape, true) + // The single read keeps its original notice. + assert.equal(violationsOf(analyzeProgram('// plan: read\nreturn await tools.read({ file_path: "a" })')).shape, true) +}) + +/** The cursor walk body: one dispatch site inside a while loop, many pages read. */ +const WALK_BODY = 'const parts = []\nlet offset = 1\nwhile (offset > 0) {\n const part = await tools.ops_inspect_local_file({ path: "log.txt", offset, limit: 200 })\n parts.push(part)\n offset = part.nextOffset || 0\n}\nreturn parts.length' + +await test('cursor pagination loops read many pages without earning a shape notice (#999)', () => { + // A cursor walk is one static dispatch site inside a loop: many reads, one program. + const walk = analyzeProgram('// plan: page through the log with the cursor\n' + WALK_BODY) + assert.equal(walk.dispatchSites, 1, 'the loop body is one static site, not its fan-out') + assert.equal(walk.hasBatchingStructure, true) + assert.deepEqual(violationsOf(walk), { plan: false, bypass: false, shape: false }) + // The same walk over an explicit page list is the same shape. + const pages = analyzeProgram('// plan: read every page the cursor yields\nconst rows = []\nfor (const cursor of [1, 201, 401]) {\n rows.push(await tools.read({ file_path: "log.txt", offset: cursor }))\n}\nreturn rows') + assert.equal(violationsOf(pages).shape, false) + // The loop exemption is not a blanket silence: without the plan line only the plan speaks. + assert.deepEqual(violationsOf(analyzeProgram(WALK_BODY)), { plan: true, bypass: false, shape: false }) +}) // ---- the per-turn latch (option 4a: re-armed by compliance, no per-turn cap) ---- @@ -134,9 +197,11 @@ const BYPASS = 'const r = await tools.ops_bash({ description: "list", command: " const THIN = 'const r = await tools.ops_read_files({ files: [{ path: "a" }] })\nreturn r' /** A compliant program: plan line, batched reads, no bypass. */ const GOOD = '// plan: list the sources; read the manifest\nconst [a, b] = await Promise.all([tools.ops_glob({ pattern: ["src/**"] }), tools.ops_read_files({ files: [{ path: "package.json" }] })])\nreturn { a, b }' +/** A compliant cursor walk: a plan line and one dispatch site inside a loop. */ +const WALK = '// plan: page through the log with the cursor\n' + WALK_BODY await test('one category delivers once per turn and stays silent until it is re-armed', () => { - const latch = createProgramLatch(), agent = {} + const latch = createProgramLatch(() => LOADER), agent = {} assert.equal(latch.observe(agent, THIN).category, 'plan', 'the first program earns the plan notice') assert.equal(latch.isArmed(agent, 'plan'), false) // THIN breaks two categories, so the second one speaks before the turn falls silent. @@ -145,7 +210,7 @@ await test('one category delivers once per turn and stays silent until it is re- }) await test('a compliant program re-arms its category, so a regression speaks again', () => { - const latch = createProgramLatch(), agent = {} + const latch = createProgramLatch(() => LOADER), agent = {} assert.equal(latch.observe(agent, THIN).category, 'plan') assert.equal(latch.observe(agent, THIN).category, 'shape', 'the other broken category still speaks') assert.equal(latch.observe(agent, THIN), undefined, 'both categories are spent') @@ -156,8 +221,26 @@ await test('a compliant program re-arms its category, so a regression speaks aga assert.equal(latch.observe(agent, THIN).category, 'plan', 'the regression speaks again in the same turn') }) +await test('wait, dependent and cursor programs never spend the shape latch (#999)', () => { + const latch = createProgramLatch(() => LOADER), agent = {} + // Both speakable categories start spent on a thin single call. + assert.equal(latch.observe(agent, THIN).category, 'plan') + assert.equal(latch.observe(agent, THIN).category, 'shape') + // A real wait, a dependent pair and a cursor walk are legitimate shapes, so the + // violation to compliance to re-violation order spends nothing on them. + const wait = '// plan: wait for the verification agent\nreturn await tools.wait_agent({ timeout_ms: 30000 })' + const dependent = '// plan: search; read the matching file\nconst hits = await tools.grep({ pattern: "needle" })\nreturn await tools.read({ file_path: hits.matches[0].path })' + for (const code of [wait, dependent, WALK]) { + assert.equal(latch.observe(agent, code), undefined, 'no category is spent on a legitimate shape') + } + assert.equal(latch.isArmed(agent, 'shape'), true, 'the compliant shapes re-armed it') + // ...so the thin regression inside the same turn is accused again. + assert.equal(latch.observe(agent, THIN).category, 'plan') + assert.equal(latch.observe(agent, THIN).category, 'shape', 'the regression speaks again in the same turn') +}) + await test('priority orders the categories and the others stay armed', () => { - const latch = createProgramLatch(), agent = {} + const latch = createProgramLatch(() => LOADER), agent = {} assert.deepEqual([...CATEGORY_PRIORITY], ['bypass', 'plan', 'shape']) assert.equal(latch.observe(agent, BYPASS).category, 'bypass', 'a bypass outranks the other two') assert.equal(latch.isArmed(agent, 'plan'), true, 'the plan notice was not spent') @@ -166,7 +249,7 @@ await test('priority orders the categories and the others stay armed', () => { }) await test('a new turn re-arms every category and a repeated turn does not', () => { - const latch = createProgramLatch(), agent = {} + const latch = createProgramLatch(() => LOADER), agent = {} latch.noteTurn(agent, 1) assert.equal(latch.observe(agent, THIN).category, 'plan') latch.noteTurn(agent, 1) @@ -176,7 +259,7 @@ await test('a new turn re-arms every category and a repeated turn does not', () }) await test('state never leaks between agents and forget drops it', () => { - const latch = createProgramLatch(), first = {}, second = {} + const latch = createProgramLatch(() => LOADER), first = {}, second = {} assert.equal(latch.observe(first, THIN).category, 'plan') assert.equal(latch.observe(second, THIN).category, 'plan', 'the second agent is independent') latch.forget(first) @@ -184,7 +267,7 @@ await test('state never leaks between agents and forget drops it', () => { }) await test('the latch counts deliveries per turn for the optional telemetry line', () => { - const latch = createProgramLatch(), agent = {} + const latch = createProgramLatch(() => LOADER), agent = {} latch.noteTurn(agent, 4) assert.deepEqual(latch.stats(agent), { turn: 4, plan: 0, bypass: 0, shape: 0, total: 0 }) latch.observe(agent, THIN) @@ -197,11 +280,88 @@ await test('the latch counts deliveries per turn for the optional telemetry line await test('the notices name the fix and never claim a runtime dispatch count', () => { const shape = analyzeProgram(BYPASS) - assert.match(noticeText('plan', shape), /first line/, 'the plan notice names the first line') - assert.doesNotMatch(noticeText('plan', shape), /Promise\.all/, 'the plan notice stays about the comment') - assert.match(noticeText('bypass', shape), /ops_glob/) - assert.match(noticeText('bypass', shape), /ops_load_sdk/) - assert.match(noticeText('shape', shape), /program text alone/, 'the shape notice states its own limit') + assert.match(noticeText('plan', shape, LOADER), /first line/, 'the plan notice names the first line') + assert.doesNotMatch(noticeText('plan', shape, LOADER), /Promise\.all/, 'the plan notice stays about the comment') + assert.match(noticeText('bypass', shape, LOADER), /ops_glob/) + assert.match(noticeText('bypass', shape, LOADER), /ops_load_sdk/) + assert.match(noticeText('shape', shape, LOADER), /program text alone/, 'the shape notice states its own limit') +}) + +await test('the bypass notice names only the APIs its scope publishes', () => { + const shape = analyzeProgram(BYPASS) + // With a loader the notice keeps teaching the load step. + assert.match(noticeText('bypass', shape, LOADER), /ops_load_sdk/) + // Without one it must not name a call that cannot succeed, and every other line + // must stay inside the published surface too. + const direct = noticeText('bypass', shape, { loader: false, resident: ['ops_glob'], loadable: [] }) + assert.equal(direct.includes('ops_load_sdk'), false, 'the unreachable loader is gone') + assert.match(direct, /ops_glob/, 'the published API is named') + for (const absent of ['ops_read_files', 'ops_grep', 'ops_edit_file']) { + assert.equal(direct.includes(absent), false, 'an unpublished API is not named: ' + absent) + } + // A scope that publishes nothing points at the official tools only. + const none = noticeText('bypass', shape, { loader: false, resident: [], loadable: [] }) + assert.match(none, /official tools/) + assert.equal(/ops_[a-z_]+/.test(none), false, 'no OPS name survives an empty surface') +}) + +await test('a narrow pool is not told to load an API it cannot reach', () => { + const shape = analyzeProgram(BYPASS) + // The pool holds stat only: naming edit_file would cost the same failed round trip. + const narrow = noticeText('bypass', shape, { loader: true, resident: [], loadable: ['ops_stat'] }) + for (const absent of ['ops_edit_file', 'ops_read_files', 'ops_glob']) { + assert.equal(narrow.includes(absent), false, 'an unloadable API is not named: ' + absent) + } + assert.match(narrow, /official tools/, 'an empty file-api intersection points at the official tools') + // The shipped wording survives while the whole list is loadable. + assert.match(noticeText('bypass', shape, LOADER), /List with `ops_glob`/) + // A narrow pool must not invite a load with the old catch-all sentence. + assert.doesNotMatch(narrow, /load only missing names/, 'the load sentence follows the pool too') +}) + +await test('a resident file API is named even when the pool cannot load it', () => { + const shape = analyzeProgram(BYPASS) + // read_files is resident and callable without loading; the hint must not claim that + // no file API exists just because the pool holds stat alone. + const mixed = noticeText('bypass', shape, { loader: true, resident: ['ops_read_files'], loadable: ['ops_stat'] }) + assert.match(mixed, /ops_read_files/, 'the resident file API is named') + assert.doesNotMatch(mixed, /No OPS file API is available/, 'the fallback does not fire while one is reachable') +}) + +await test('the file-work line never names the shell it just accused', () => { + const shape = analyzeProgram(BYPASS) + const mixed = noticeText('bypass', shape, { loader: false, resident: ['ops_bash', 'ops_glob'], loadable: [] }) + assert.match(mixed, /ops_glob/, 'the file API is named') + assert.equal(mixed.includes('ops_bash'), false, 'the bypass vector is never offered as the fix') +}) + +await test('the latch resolves the surface per agent', () => { + const first = {}, second = {} + const latch = createProgramLatch(agent => agent === first + ? LOADER + : { loader: false, resident: ['ops_glob'], loadable: [] }) + assert.match(latch.observe(first, BYPASS).text, /ops_load_sdk/, 'the loader scope keeps its lesson') + assert.equal(latch.observe(second, BYPASS).text.includes('ops_load_sdk'), false, 'the other agent gets its own surface') +}) + +await test('a scope with no file API does not refer to them', () => { + const shape = analyzeProgram(BYPASS) + // A resident non-file API leaves the hint pointing at the official tools; the next + // line must not then invite a call to "those APIs". + const statOnly = noticeText('bypass', shape, { loader: false, resident: ['ops_stat'], loadable: [] }) + assert.match(statOnly, /official tools/, 'the official tools are named') + assert.doesNotMatch(statOnly, /call those APIs/, 'the pronoun has no referent here') +}) + +await test('a narrow pool with a loadable file API is told the load order', () => { + const shape = analyzeProgram(BYPASS) + // The acceptance scope: the loader is published and grep is loadable, so the notice + // must teach load first and call next; a same-program call to grep would throw. + const narrow = noticeText('bypass', shape, { loader: true, resident: ['ops_inspect_local_file'], loadable: ['ops_grep'] }) + assert.match(narrow, /ops_grep/, 'the loadable file API is named') + assert.match(narrow, /for the next program/, 'the load lands before the call') + assert.equal(narrow.includes('ops_edit_file'), false, 'an unauthorized write API is not named') + assert.equal(narrow.includes('ops_read_files'), false, 'an unloadable read API is not named') }) report('batching-trigger') diff --git a/test/bench-fixtures.test.mjs b/test/bench-fixtures.test.mjs index 24266df0..f5a51d59 100644 --- a/test/bench-fixtures.test.mjs +++ b/test/bench-fixtures.test.mjs @@ -21,8 +21,8 @@ import { CPU_PROCESS_TIMEOUT_MS, assert, report, test, withTempDir } from './lib import { brokenProject, configTree, deepCorpus, deepModule, deriveConfigTree, deriveDeepModule, deriveGrepTree, deriveLargeCsv, deriveLogStats, deriveNoisySuite, deriveRenameTree, - deriveSymbolTree, gatePipeline, generate, GENERATORS, grepTree, largeCsv, logStats, - mulberry32, noisySuite, renameTree, symbolTree, + derivePagedLog, deriveSymbolTree, gatePipeline, generate, GENERATORS, grepTree, largeCsv, logStats, + mulberry32, noisySuite, pagedLog, renameTree, symbolTree, } from '../bench/lib/fixtures/index.mjs' const LABEL = 'bench-fixtures' @@ -40,12 +40,13 @@ const SMALL = { noisySuite: { seed: 29, tests: 200 }, symbolTree: { seed: 31, files: 6 }, gatePipeline: { seed: 53, gates: 4, probes: 3, linesPerGate: 4 }, + pagedLog: { seed: 41, lines: 120, markerAt: 60 }, } /** The complete registry: every family builder, in the order `GENERATORS` lists them. */ const ALL_GENERATORS = [ 'brokenProject', 'configTree', 'deepCorpus', 'deepModule', 'gatePipeline', 'grepTree', - 'largeCsv', 'logStats', 'noisySuite', 'renameTree', 'symbolTree', + 'largeCsv', 'logStats', 'noisySuite', 'pagedLog', 'renameTree', 'symbolTree', ] /** Write one generated file into a workspace, creating its parents. */ @@ -128,6 +129,16 @@ await test('logStats exports the ERROR counts recomputed from its own bytes', () assert.deepEqual(exports.report, [...exports.report].sort(), 'the report must be in file-name order') }) +await test('pagedLog exports the marked line recomputed from its own bytes', () => { + const { files, exports } = pagedLog(SMALL.pagedLog) + assert.equal(files.length, 1) + assert.equal(files[0].path, 'logs/stream.log') + assert.deepEqual(derivePagedLog(files), exports, 'the exported answer must be of the real bytes') + assert.ok(files[0].content.split('TARGET-').length === 2, 'the marker must appear exactly once') + assert.throws(() => pagedLog({ ...SMALL.pagedLog, markerAt: 0 }), RangeError) + assert.throws(() => pagedLog({ ...SMALL.pagedLog, markerAt: SMALL.pagedLog.lines + 1 }), RangeError) +}) + await test('configTree separates the winning layer from every look-alike', () => { const { files, exports } = configTree(SMALL.configTree) const read = (name) => files.find((file) => file.path === name)?.content ?? '' @@ -283,7 +294,7 @@ await test('generate() rejects an unknown generator by name', () => { // exists but was never registered (or never re-exported) is unreachable, and a // duplicate or a stray extra key has to fail here rather than in a task run. assert.deepEqual(Object.keys(GENERATORS).sort(), ALL_GENERATORS) - assert.equal(Object.keys(GENERATORS).length, 11) + assert.equal(Object.keys(GENERATORS).length, 12) assert.deepEqual(Object.keys(SMALL).sort(), ALL_GENERATORS, 'the small-parameter table must cover every registered generator') for (const [name, builder] of Object.entries(GENERATORS)) assert.equal(typeof builder, 'function', `${name} must be exported as a builder`) }) diff --git a/test/bench-tracks.test.mjs b/test/bench-tracks.test.mjs new file mode 100644 index 00000000..19e482e1 --- /dev/null +++ b/test/bench-tracks.test.mjs @@ -0,0 +1,206 @@ +/** + * The V4 behavior-track gate (#993): four Flash behavior samples — paged cursor + * continuation, catalog-error recovery, load-before-use, and wait/multi-dispatch — + * each with a business-goal prompt, a scripted pass trajectory and scripted fail + * trajectories, plus the verifier that must tell them apart. + * + * The red step of this suite came first: every module it imports was absent, so the + * first run failed at the import. The green step is the corpus + verifier landing. + * The suite never spawns a worker and never sends a model request: it judges + * scripted trajectories only, which is exactly what the card asks (V5 owns real runs). + * + * @module dsh-ops/test/bench-tracks + */ + +import fs from 'node:fs' +import path from 'node:path' + +import { assert, report, test, withTempDir } from './lib/harness.mjs' +import { TASKS_DIR, loadTasks } from '../bench/lib/tasks.mjs' +import { GENERATORS, generate } from '../bench/lib/fixtures/index.mjs' +import { derivePagedLog } from '../bench/lib/fixtures/paged.mjs' +import { verifyWorkspace } from '../bench/lib/verify.mjs' +import { judgeTrack, recoverySummary } from '../bench/lib/tracks.mjs' +import { loadTrackTasks, scriptedTracks, TRACK_TASK_IDS } from '../bench/lib/track-corpus.mjs' +import { main as cliMain, usage } from '../bench/cli.mjs' + +const LABEL = 'bench-tracks' + +/** Write one generated fixture file into a workspace, creating its parents. */ +const write = (dir, file) => { + const target = path.join(dir, ...file.path.split('/')) + fs.mkdirSync(path.dirname(target), { recursive: true }) + fs.writeFileSync(target, file.content, 'utf8') +} + +// ------------------------------------------------------------ the corpus + +await test('the four V4 task files validate against the real registries', async () => { + const corpus = loadTasks(TASKS_DIR, { generators: GENERATORS }) + const ids = corpus.map((task) => task.id) + for (const id of TRACK_TASK_IDS) { + const task = corpus.find((entry) => entry.id === id) + assert.ok(task, 'task ' + id + ' must exist in bench/tasks') + assert.ok(!task.prompt.toLowerCase().includes('verifier'), 'the ' + id + ' prompt must not name the verifier') + assert.ok(!/nextoffset|cursor|load_sdk/i.test(task.prompt), 'the ' + id + ' prompt must not teach the tool steps') + } + assert.equal(ids.length, new Set(ids).size, 'task ids must be unique') +}) + +await test('pagedLog puts the target on the second page and its answer derives from the bytes', () => { + const { files, exports } = generate('pagedLog', { seed: 41, lines: 600, markerAt: 350, pageSize: 200 }) + assert.equal(files.length, 1) + assert.equal(files[0].path, 'logs/stream.log') + assert.deepEqual(derivePagedLog(files), exports, 'the exported answer must be recomputable from the bytes') + assert.ok(exports.markerLine > 200, 'the marker must sit beyond page 1 at the default page size') + assert.ok(exports.markerLine <= 400, 'the marker must sit on page 2 at the default page size') + const lines = files[0].content.split('\n') + assert.ok(lines[exports.markerLine - 1].includes(exports.markerText), 'the marker line must carry the marker payload') + assert.ok(exports.markerText.includes('TARGET-'), 'the marker payload must carry the TARGET token') + assert.equal(exports.answer, exports.markerLine + ' ' + exports.markerText) +}) + +await test('each task workspace verify accepts the correct answer and rejects a wrong one', async () => { + const tasks = await loadTrackTasks() + assert.equal(tasks.length, 4) + for (const entry of tasks) { + await withTempDir('bench-tracks-verify', async (dir) => { + for (const file of entry.files) write(dir, file) + // The wait task also requires the business edit the prompt asks for. + if (entry.id === 'ptc-wait-dispatch') { + const target = path.join(dir, entry.answer.content) + const before = fs.readFileSync(target, 'utf8') + const edited = before.replace('"MAX_UPLOAD_MB": ' + entry.exports.value, '"MAX_UPLOAD_MB": ' + 900) + assert.notEqual(edited, before, 'the edit pattern must match the winner file') + fs.writeFileSync(target, edited) + } + fs.writeFileSync(path.join(dir, 'answer.txt'), entry.answer.content) + const good = await verifyWorkspace(dir, entry.task.verify, entry.exports) + assert.ok(good.passed, entry.id + ': the correct answer must pass every workspace check (got: ' + good.checks.map((c) => c.detail).join(' | ') + ')') + fs.writeFileSync(path.join(dir, 'answer.txt'), 'definitely not the answer') + const bad = await verifyWorkspace(dir, entry.task.verify, entry.exports) + assert.ok(!bad.passed, entry.id + ': a wrong answer must fail the workspace check') + }) + } +}) + +// ------------------------------------------------------------ the verdicts + +const judge = async () => { + const tasks = await loadTrackTasks() + return scriptedTracks(tasks).map((track) => judgeTrack(track, tasks.find((task) => task.id === track.taskId))) +} + +await test('every task has a should-pass and a should-fail trajectory, and the verifier agrees with both', async () => { + const verdicts = await judge() + assert.equal(verdicts.length, 10, 'the scripted corpus holds ten tracks') + for (const id of TRACK_TASK_IDS) { + const own = verdicts.filter((verdict) => verdict.taskId === id) + assert.ok(own.some((v) => v.declared === 'pass'), id + ' needs a should-pass trajectory') + assert.ok(own.some((v) => v.declared === 'fail'), id + ' needs a should-fail trajectory') + } + for (const verdict of verdicts) { + assert.equal(verdict.matched, true, + verdict.id + ': declared ' + verdict.declared + ' but judged ' + verdict.verdict + ' (' + verdict.mismatches.join(' | ') + ')') + } +}) + +await test('the paging verifier names the ignored cursor, the unreached complete page and the wrong answer', async () => { + const verdicts = await judge() + const pass = verdicts.find((v) => v.id === 'ptc-paged-cursor-pass') + assert.equal(pass.verdict, 'pass') + assert.deepEqual(pass.mismatches, []) + assert.ok(pass.opportunities >= 2, 'every partial page is a continuation cue the model must act on') + assert.equal(pass.recovered, true) + const retry = verdicts.find((v) => v.id === 'ptc-paged-cursor-retry') + assert.equal(retry.verdict, 'fail') + assert.match(retry.mismatches.join(' | '), /cursor|offset/i) + assert.match(retry.mismatches.join(' | '), /complete/i) + assert.match(retry.mismatches.join(' | '), /answer/) + const early = verdicts.find((v) => v.id === 'ptc-paged-cursor-early') + assert.equal(early.verdict, 'fail') + assert.match(early.mismatches.join(' | '), /answer/) +}) + +await test('the catalog verifier accepts only advertised alternatives and names the invented one', async () => { + const verdicts = await judge() + const pass = verdicts.find((v) => v.id === 'ptc-catalog-recover-pass') + assert.equal(pass.verdict, 'pass') + assert.equal(pass.opportunities, 1) + assert.equal(pass.recovered, true) + const fail = verdicts.find((v) => v.id === 'ptc-catalog-nonexistent-recover') + assert.equal(fail.verdict, 'fail') + assert.match(fail.mismatches.join(' | '), /ops_searchall/) + assert.match(fail.mismatches.join(' | '), /advertised/) + assert.match(fail.mismatches.join(' | '), /answer/) +}) + +await test('the load verifier demands load_sdk before the next program can use the tool', async () => { + const verdicts = await judge() + const pass = verdicts.find((v) => v.id === 'ptc-sdk-load-then-use-pass') + assert.equal(pass.verdict, 'pass') + assert.equal(pass.opportunities, 1) + assert.equal(pass.recovered, true) + const fail = verdicts.find((v) => v.id === 'ptc-sdk-retry-without-load') + assert.equal(fail.verdict, 'fail') + assert.match(fail.mismatches.join(' | '), /load/) + assert.match(fail.mismatches.join(' | '), /answer/) +}) + +await test('waits are not misjudged: the lock wait passes, the immediate counter-example needs no retry at all', async () => { + const verdicts = await judge() + const wait = verdicts.find((v) => v.id === 'ptc-wait-lock-pass') + assert.equal(wait.verdict, 'pass', 'the lock wait must not be flagged (' + wait.mismatches.join(' | ') + ')') + assert.ok(wait.waits >= 1, 'the bounded-transient re-issue must count as a real wait') + assert.equal(wait.recovered, true) + const immediate = verdicts.find((v) => v.id === 'ptc-wait-immediate-pass') + assert.equal(immediate.verdict, 'pass') + assert.deepEqual(immediate.mismatches, []) + assert.equal(immediate.opportunities, 0) + assert.equal(immediate.recovered, null, 'a track without a correction opportunity has no recovery value') + assert.equal(immediate.waits, 0) + const fail = verdicts.find((v) => v.id === 'ptc-wait-no-progress') + assert.equal(fail.verdict, 'fail') + assert.match(fail.mismatches.join(' | '), /no-progress|verbatim/i) + assert.match(fail.mismatches.join(' | '), /revision/i) +}) + +// ------------------------------------------------------------ recovery accounting + +await test('the recovery rate counts only tracks with a correction opportunity and never invents a denominator', async () => { + const verdicts = await judge() + const summary = recoverySummary(verdicts) + assert.equal(summary.tracks, 10) + assert.equal(summary.withOpportunity, 9, 'only the immediate counter-example has no correction opportunity') + assert.equal(summary.recovered, 4) + assert.equal(summary.rate, 4 / 9) + const onlyImmediate = recoverySummary(verdicts.filter((v) => v.id === 'ptc-wait-immediate-pass')) + assert.equal(onlyImmediate.withOpportunity, 0) + assert.equal(onlyImmediate.rate, null) + assert.equal(onlyImmediate.hasDenominator, false) + assert.ok(onlyImmediate.note.includes('no denominator'), 'a missing denominator must be said, not reported as 100%') +}) + +// ------------------------------------------------------------ the CLI + +await test('bench/cli.mjs lists tracks and the command judges the corpus end to end', async () => { + assert.match(usage(), /\n tracks\s/) + const lines = [] + const io = { out: (line) => lines.push(line), err: (line) => lines.push('ERR ' + line) } + const code = await cliMain(['tracks'], io) + assert.equal(code, 0, lines.join('\n')) + const text = lines.join('\n') + assert.ok(text.includes('ptc-paged-cursor-pass'), 'every scripted track must be reported') + assert.match(text, /recovery: 4\/9|recovery: 44%/) + const scoped = [] + await cliMain(['tracks', '--task', 'ptc-wait-dispatch'], { out: (line) => scoped.push(line), err: (line) => scoped.push('ERR ' + line) }) + assert.ok(scoped.join('\n').includes('ptc-wait-immediate-pass'), '--task must scope the corpus') + const single = [] + await cliMain(['tracks', '--track', 'ptc-wait-immediate-pass'], { out: (line) => single.push(line), err: (line) => single.push('ERR ' + line) }) + assert.ok(single.join('\n').includes('no denominator'), 'the no-retry counter-example must report no denominator') + const bogus = [] + assert.equal(await cliMain(['tracks', '--track', 'nope'], { out: (line) => bogus.push(line), err: (line) => bogus.push('ERR ' + line) }), 2) + assert.match(bogus.join('\n'), /unknown track/) +}) + +report(LABEL) diff --git a/test/canonical-file-runtime.test.mjs b/test/canonical-file-runtime.test.mjs index e91ed11b..d54d3dcc 100644 --- a/test/canonical-file-runtime.test.mjs +++ b/test/canonical-file-runtime.test.mjs @@ -50,9 +50,20 @@ await test('real worker Native text and run_code share the canonical file outcom assert.equal(result.value.coverage.isComplete, false) assert.match(result.value.continuation, /^ops-next-v1:/) assert.equal(result.value.data.items.length, 1) + assert.match(result.value.nextAction, /ops_glob /, 'the shared runtime renders the continuation as an executable call') } console.log(' Real worker branch:', name, result.value.status, result.value.error?.code ?? 'continuation') } + const refused = await executeCall(h.ctx, { agent: child, toolSet: c.toolSet, callId: 'dir-refused', + name: h.definition.name, arguments: { path: dir } }) + assert.equal(refused.value.status, 'error', JSON.stringify(refused)) + assert.equal(refused.value.error.code, 'INVALID_ARGUMENT') + // The path is JSON-quoted inside the recommendation; on Windows its backslashes + // are regexp escapes, so the literal prefix must be escaped too. + const escapedDir = JSON.stringify(dir).replace(/[.*+?^${}()|[\]\\]/g, '\\$&') + assert.match(refused.value.error.nextAction, new RegExp('ops_glob \\{"path":' + escapedDir + ',"pattern":\\["\\*"\\]\\}'), + 'the step surface resolves, so the recommendation names the registered listing API') + assert.match(refused.value.error.nextAction, /concrete file path/) } finally { try { await h.dispose() } finally { fs.rmSync(dir, { recursive: true, force: true }) } } diff --git a/test/client-settings-plugin-manager.test.mjs b/test/client-settings-plugin-manager.test.mjs new file mode 100644 index 00000000..edad7802 --- /dev/null +++ b/test/client-settings-plugin-manager.test.mjs @@ -0,0 +1,85 @@ +/** + * The settings page must actually ask the host's own installer (#750). + * + * The row's pwsh7 detection reads the `remote.pluginManager` service. A traced + * registry value may only be read through a context that declares the + * dependency, so a page that resolves it without one reported "not installed" + * forever while every other part kept working. This suite mounts the shipped + * browser bundle on a real Cordis host and witnesses the installer roundtrip. + */ +import fs from 'node:fs' +import path from 'node:path' +import vm from 'node:vm' +import { createRequire } from 'node:module' +import { JSDOM } from 'jsdom' +import * as Cordis from '@deepseek-ai/cordis' +import * as slots from '@deepseek-ai/dsh-client-ui-slots' +import { assert, test, report, waitFor, PACKAGE_ROOT } from './lib/harness.mjs' +import { shutdown } from './lib/host.mjs' + +const require = createRequire(import.meta.url) +const dom = new JSDOM('') +globalThis.document = dom.window.document + +/** Load one browser bundle the way the host's ModuleLoader does. */ +function capture(file) { + let definition + globalThis.window = { __ModuleLoader__: { load: value => { definition = value } } } + vm.runInThisContext(fs.readFileSync(file, 'utf8'), { filename: file }) + assert.ok(definition) + return definition +} + +/** + * Resolve one package's `./client` condition to the file a Client row loads. + * @param {string} name - the package whose Client entry is wanted. + * @returns {string} absolute path of the entry file. + */ +const clientEntry = name => { + const exported = require(`${name}/package.json`).exports['./client'] + const file = typeof exported === 'string' ? exported : exported.default + return path.join(PACKAGE_ROOT, 'node_modules', name, file) +} + +const primitives = { Switch: () => null } +const external = name => name === '@deepseek-ai/cordis' ? Cordis + : name === '@deepseek-ai/dsh-client-ui-slots' ? slots + : name === '@deepseek-ai/dsh-client-ui-primitives' ? primitives + : require(name) + +await test('the shipped settings bundle asks the host installer about dsh-win-pwsh7 (#750)', async () => { + const definition = capture(path.join(PACKAGE_ROOT, 'lib/client.js')) + const plugin = definition.factory(external) + const Renderer = capture(clientEntry('@deepseek-ai/dsh-client-ui-renderer')).factory(external) + const ctx = new Cordis.Context() + ctx.logger.level = 0 + let listCalls = 0 + try { + await ctx.plugin(Renderer.SlotRegistry) + // The gateway-less minimal remote: mounting still succeeds and the status + // card reports the miss instead of pretending the runtime is fine. + ctx.provide('remote', { async $mount() { return undefined } }) + // The host's own installer, as the plugin-manager face provides it. + ctx.provide('remote.pluginManager', { + async listPlugins() { + listCalls += 1 + return [{ entryId: 'dsh-win-pwsh7', moduleName: 'dsh-win-pwsh7' }] + }, + async installBundle() { return { ok: true } }, + }) + await ctx.plugin({ inject: ['slots'], apply(inner) { + inner.provide('locale', { register: () => () => {} }) + inner.slots.register({ name: 'root', children: { + 'plugins.row.config': { kind: 'keyed', scope: 'root' }, + 'plugins.detail.section': { kind: 'list', scope: 'root' }, + } }, () => null) + } }) + const fiber = await ctx.plugin(plugin) + try { + await waitFor(() => listCalls === 1, { what: 'the pwsh7 detection roundtrip' }) + assert.equal(listCalls, 1, 'the detection asks the host installer exactly once') + } finally { await fiber.dispose() } + } finally { await shutdown(ctx) } +}) + +report('client-settings-plugin-manager') diff --git a/test/discovery-deferred-rejection.test.mjs b/test/discovery-deferred-rejection.test.mjs new file mode 100644 index 00000000..4a24c2b9 --- /dev/null +++ b/test/discovery-deferred-rejection.test.mjs @@ -0,0 +1,102 @@ +/** Unselected discovered tools must name the selector and its select: query (#764). */ +import { createScope } from '@deepseek-ai/dsh-scope' +import { mountDiscovery } from '../lib/discovery/controller.js' +import { registerNativeCapability } from '../lib/discovery/providers.js' +import { assert, test, report, waitFor } from './lib/harness.mjs' +import { bootHost, shutdown } from './lib/host.mjs' +import { executeCall } from './lib/executor.mjs' + +const DISCOVER = 'ops_discover_tools' +const definition = (name, execute = () => ({ status: 'completed', answer: 42 })) => ({ + name, description: 'Look up project inventory.', parameters: { type: 'object', properties: {} }, + output: { schema: { type: 'object' }, render: (_args, value) => [{ type: 'text', text: JSON.stringify(value) }] }, execute, +}) +/** The minimal tool-set view the registry's executable check accepts for one definition. */ +const soloView = definition => ({ view: () => new Map([[definition.name, definition]]) }) + +async function setup() { + const { ctx, tools } = await bootHost() + const agents = [] + ctx.provide('agents', { list: () => agents }) + // Publication-only worker fixture: never invoked; no Rust runtime is needed by this suite. + ctx.provide('subprocess', { spawn() { throw new Error('deferred rejection fixtures never spawn') } }) + ctx.provide('sandboxPolicy', { resolve: ({ session }) => ({ mode: session?.mode ?? 'danger-full-access', workspaceRoot: process.cwd(), sessionId: session?.id }) }) + await ctx.plugin({ apply(inner) { + mountDiscovery(inner, { allowedTools: ['stat', 'write_file'], alwaysLoaded: [], binaryPath: process.execPath }, { + allows: () => true, residentTools: () => [], + external: { allowed: ['acme_lookup'], disabled: [], alwaysLoaded: [] }, + }) + } }) + await waitFor(() => ctx.get('tools') && ctx.get('systemPrompt')) + for (let i = 0; i < 3; i++) await new Promise(resolve => setImmediate(resolve)) + await registerNativeCapability(ctx, { + definition: definition('acme_lookup'), search: 'inventory project lookup', authorize: () => true, + }) + const mint = async (mode = 'danger-full-access') => { + const id = 'discovery-deferred-' + mode + const agent = { id, session: { id, mode, header: { cwd: process.cwd() } } } + agent.ctx = createScope(ctx, agent).ctx + agents.push(agent) + await ctx.serial('agent/created', { agent }) + await waitFor(() => tools.get(DISCOVER, agent), { what: 'discovery lifecycle initialization' }) + return agent + } + return { ctx, tools, mint } +} + +/** Dispatch the registered definition the way a host that ignores `when` does (#934 shape). */ +const dispatch = async (ctx, agent, name) => { + const definition = ctx.get('tools').get(name, agent) ?? ctx.get('tools').definitions(agent).find(def => def.name === name) + assert.ok(definition, name + ' must be registered for this scenario') + return definition.execute({}, { agent, signal: new AbortController().signal, toolSet: soloView(definition) }) +} + +await test('an unselected builtin discovery capability answers with the select: query path', async () => { + const { ctx, mint } = await setup() + try { + const agent = await mint() + const result = await dispatch(ctx, agent, 'ops_stat') + assert.equal(result.error?.code, 'CAPABILITY_NOT_SELECTED') + assert.match(result.error.message, /ops_stat/) + assert.match(result.error.nextAction ?? '', /ops_discover_tools/) + assert.match(result.error.nextAction ?? '', /select:ops_stat/) + } finally { await shutdown(ctx) } +}) + +await test('an unselected external provider capability answers with the same select: path', async () => { + const { ctx, mint } = await setup() + try { + const agent = await mint() + const result = await dispatch(ctx, agent, 'acme_lookup') + assert.equal(result.error?.code, 'CAPABILITY_NOT_SELECTED') + assert.match(result.error.message, /acme_lookup/) + assert.match(result.error.nextAction ?? '', /select:acme_lookup/) + } finally { await shutdown(ctx) } +}) + +await test('a discovered selection stops answering with the deferred rejection', async () => { + const { ctx, mint } = await setup() + try { + const agent = await mint() + // The selector itself must stay callable while its pool defers. + assert.equal((await dispatch(ctx, agent, 'ops_stat')).error?.code, 'CAPABILITY_NOT_SELECTED') + const select = await executeCall(ctx, { agent, name: DISCOVER, arguments: { query: 'select:ops_stat' }, callId: 'select-stat' }) + assert.equal(select.isError, false, JSON.stringify(select)) + assert.deepEqual(select.value.activated, ['ops_stat']) + ctx.emit('session/event', agent.session, { type: 'step/end', turn: 1, step: 1 }) + const result = await dispatch(ctx, agent, 'ops_stat') + assert.notEqual(result.error?.code, 'CAPABILITY_NOT_SELECTED') + } finally { await shutdown(ctx) } +}) + +await test('a read-only scope keeps the generic refusal for mutating capabilities', async () => { + const { ctx, mint } = await setup() + try { + const agent = await mint('read-only') + const result = await dispatch(ctx, agent, 'ops_write_file') + // Read-only policy can never select a mutating capability, so a select: pointer would be dishonest. + assert.notEqual(result.error?.code, 'CAPABILITY_NOT_SELECTED') + } finally { await shutdown(ctx) } +}) + +report('discovery-deferred-rejection') diff --git a/test/file-asar-path.test.mjs b/test/file-asar-path.test.mjs new file mode 100644 index 00000000..586737f9 --- /dev/null +++ b/test/file-asar-path.test.mjs @@ -0,0 +1,51 @@ +/** #836: a path that routes through a real .asar FILE is a virtual directory + * only inside Electron's own patched filesystem. The standalone worker cannot read + * archive members, so the refusal must say exactly that instead of a generic + * no-follow PATH_RESTRICTED. A genuine directory named *.asar keeps traversing. */ +import fs from 'node:fs' +import path from 'node:path' +import { assert, test, report, withTempDir } from './lib/harness.mjs' +import { callTool, envelopeOf, requireWorker } from './lib/boundary.mjs' +requireWorker('file-asar-path') +const call = async (tool, args, root, options = {}) => + envelopeOf(await callTool(tool, args, { root, readOnly: true, ...options }), assert) + +await test('an .asar file component is an archive member, not a traversal bypass', async () => { + await withTempDir('file-asar-path', async root => { + fs.writeFileSync(path.join(root, 'bundle.asar'), Buffer.from([4, 0, 0, 0, 16, 0, 0, 0, 0xde, 0xad, 0xbe, 0xef])) + const inside = await call('inspect_local_file', { path: path.join(root, 'bundle.asar', 'app.js'), limit: 5 }, root) + assert.equal(inside.error?.code, 'ARCHIVE_MEMBER_UNSUPPORTED', JSON.stringify(inside)) + assert.match(inside.error.message, /bundle\.asar/) + assert.match(inside.error.message, /archive/i) + assert.equal(inside.error.effectState, 'none') + // A real directory keeps working: nothing about the .asar spelling is reserved. + fs.mkdirSync(path.join(root, 'folder.asar')) + fs.writeFileSync(path.join(root, 'folder.asar', 'inner.js'), 'real') + const directory = await call('inspect_local_file', { path: path.join(root, 'folder.asar', 'inner.js'), limit: 5 }, root) + assert.equal(directory.error, undefined, JSON.stringify(directory)) + assert.equal(directory.items[0].text, 'real') + // A non-archive file component keeps the ordinary traversal refusal. + fs.writeFileSync(path.join(root, 'readme.txt'), 'text') + const plain = await call('inspect_local_file', { path: path.join(root, 'readme.txt', 'inner.js'), limit: 5 }, root) + assert.equal(plain.error?.code, 'PATH_RESTRICTED', JSON.stringify(plain)) + // The archive itself stays readable as a regular binary file. + const hex = await call('inspect_local_file', { path: path.join(root, 'bundle.asar'), view: 'hex', limit: 2 }, root) + assert.equal(hex.error, undefined, JSON.stringify(hex)) + assert.equal(hex.items.length, 1) + }) +}) + +await test('the installed host archive is diagnosed the same way', async () => { + await withTempDir('file-asar-host', async root => { + const archive = 'D:\\deepseek_harness\\resources\\app.asar' + if (!fs.existsSync(archive)) { + console.log(' (host archive absent on this machine; live-host assertion not run)') + return + } + const body = await call('inspect_local_file', { path: path.join(archive, 'dsh', 'package.json'), limit: 5 }, root) + assert.equal(body.error?.code, 'ARCHIVE_MEMBER_UNSUPPORTED', JSON.stringify(body)) + assert.match(body.error.message, /app\.asar/) + }) +}) + +report('file-asar-path') diff --git a/test/file-escaped-edit.test.mjs b/test/file-escaped-edit.test.mjs new file mode 100644 index 00000000..83295ffe --- /dev/null +++ b/test/file-escaped-edit.test.mjs @@ -0,0 +1,89 @@ +/** #939: the Native reader renders backslashes and controls with reversible JSON + * escapes. ops_edit_file matches oldText against the raw bytes first; when that + * misses, exactly one re-interpretation is tried - the rendered form decoded - and + * the receipt says so. Raw matches keep precedence, so literal text is never + * re-read through the decoder. */ +import fs from 'node:fs' +import path from 'node:path' +import { createHash } from 'node:crypto' +import { encodeSourceLine } from '../lib/file/native/line-encoding.js' +import { renderNative } from '../lib/file/native/render.js' +import { assert, test, report, withTempDir } from './lib/harness.mjs' +import { callTool, envelopeOf, requireWorker } from './lib/boundary.mjs' +requireWorker('file-escaped-edit') +const call = async (tool, args, root) => envelopeOf(await callTool(tool, args, { root }), assert) +const rev = file => createHash('sha256').update(fs.readFileSync(file)).digest('hex') + +await test('oldText copied from the rendered line edits after the raw miss', async () => { + await withTempDir('file-escaped-edit', async root => { + const target = path.join(root, 'rendered.txt') + fs.writeFileSync(target, 'alpha\ttab C:\\path\nbeta\n') + const revision = rev(target) + const rendered = encodeSourceLine('alpha\ttab C:\\path') + assert.notEqual(rendered, 'alpha\ttab C:\\path', 'the raw line must be rendered escaped before the model sees it') + const body = await call('edit_file', { path: target, expectedRevision: revision, + edits: [{ oldText: rendered, newText: encodeSourceLine('alpha\ttab C:\\renamed') }] }, root) + assert.equal(body.status, 'completed', JSON.stringify(body)) + assert.equal(body.items[0].renderedMatches, 1, JSON.stringify(body)) + assert.equal(fs.readFileSync(target, 'utf8'), 'alpha\ttab C:\\renamed\nbeta\n') + }) +}) + +await test('raw literal matches keep precedence and never decode newText', async () => { + await withTempDir('file-escaped-precedence', async root => { + const target = path.join(root, 'precedence.txt') + // The file holds BOTH a literal backslash-t pair and a raw tab. + fs.writeFileSync(target, 'literal\\tab\treal\n') + const body = await call('edit_file', { path: target, expectedRevision: rev(target), + edits: [{ oldText: 'literal\\tab', newText: 'literal\\kept' }] }, root) + assert.equal(body.status, 'completed', JSON.stringify(body)) + assert.equal(body.items[0].renderedMatches, undefined, 'a raw match must not count as rendered') + assert.equal(fs.readFileSync(target, 'utf8'), 'literal\\kept\treal\n') + }) +}) + +await test('ambiguous or absent decodes stay honest EDIT_CONFLICTs', async () => { + await withTempDir('file-escaped-conflict', async root => { + const target = path.join(root, 'conflict.txt') + fs.writeFileSync(target, 'a\tb a\tb\n') + const duplicated = await call('edit_file', { path: target, expectedRevision: rev(target), + edits: [{ oldText: encodeSourceLine('a\tb'), newText: 'x' }] }, root) + assert.equal(duplicated.error?.code, 'EDIT_CONFLICT', JSON.stringify(duplicated)) + assert.equal(duplicated.error.effectState, 'none') + fs.writeFileSync(target, 'plain line\n') + const missing = await call('edit_file', { path: target, expectedRevision: rev(target), + edits: [{ oldText: encodeSourceLine('absent\\q'), newText: 'x' }] }, root) + assert.equal(missing.error?.code, 'EDIT_CONFLICT', JSON.stringify(missing)) + assert.match(missing.error.message, /rendered|escaped/i, 'the refusal must name the second interpretation') + assert.equal(fs.readFileSync(target, 'utf8'), 'plain line\n') + }) +}) + +await test('write_file stays byte-verbatim and never decodes content', async () => { + await withTempDir('file-escaped-write', async root => { + const target = path.join(root, 'verbatim.txt') + const body = await call('write_file', { path: target, expectedRevision: 'absent', content: 'keep \\t literal\n' }, root) + assert.equal(body.status, 'completed', JSON.stringify(body)) + assert.equal(fs.readFileSync(target, 'utf8'), 'keep \\t literal\n') + }) +}) + +await test('the rendered receipt and read output state the escape contract', async () => { + const file = path.resolve('rendered-source.txt') + const value = { status: 'completed', path: file, revision: 'a'.repeat(64), lineCount: 1, nextOffset: 2, + complete: true, truncated: false, failed: 0, + items: [{ path: file, line: 1, text: 'alpha\ttab' }] } + const mutated = { status: 'completed', phase: 'verify', complete: true, changed: 1, dryRun: false, + items: [{ path: file, status: 'completed', changed: 1, renderedMatches: 1 }] } + const inspection = renderNative('inspect_local_file', { path: file, limit: 1 }, value) + assert.match(inspection, /reversible escapes/, 'reads must warn when escapes are displayed') + assert.doesNotMatch(renderNative('inspect_local_file', { path: file, limit: 1 }, + { ...value, items: [{ path: file, line: 1, text: 'plain' }] }), /reversible escapes/) + const batch = { status: 'completed', total: 1, nextIndex: 1, pending: 0, failed: 0, truncated: false, items: [value] } + assert.match(renderNative('read_files', { files: [{ path: file, limit: 1 }] }, batch), /reversible escapes/) + assert.match(renderNative('edit_file', { path: file }, mutated), /renderedMatches=1/, 'the receipt must surface rendered matches') + assert.doesNotMatch(renderNative('edit_file', { path: file }, + { ...mutated, items: [{ path: file, status: 'completed', changed: 1 }] }), /renderedMatches/) +}) + +report('file-escaped-edit') diff --git a/test/integration-host-floor.test.mjs b/test/integration-host-floor.test.mjs new file mode 100644 index 00000000..a48b14a8 --- /dev/null +++ b/test/integration-host-floor.test.mjs @@ -0,0 +1,74 @@ +/** + * Activation-time host support boundary (#701). + * + * A host below the declared engines.dsh floor used to load anyway and fail + * later: TypeError scope errors backstage, EXECUTOR_UNAVAILABLE in front. + * The contract pinned here: the package's own engines.dsh names the floor, an + * older host hears one explicit diagnosis at activation, and an unknown host + * version is never refused (it cannot be proven below the floor). The host's + * own behaviour while loading an unsupported plugin remains upstream. + */ +import fs from 'node:fs' +import path from 'node:path' +import { assert, test, report, withTempDir } from './lib/harness.mjs' +import { bootHost, shutdown } from './lib/host.mjs' +import { apply } from '../lib/index.js' + +await test('the declared engines.dsh floor is read from the package manifest', async () => { + const { declaredHostFloor } = await import('../lib/adapter/dsh/host-floor.js') + assert.equal(declaredHostFloor(), '0.2.2-rc.1') +}) + +await test('semver precedence decides below-floor, with prerelease rules kept', async () => { + const { versionBelowFloor } = await import('../lib/adapter/dsh/host-floor.js') + const floor = '0.2.2-rc.1' + assert.equal(versionBelowFloor('0.2.1', floor), true, 'an older release is below') + assert.equal(versionBelowFloor('0.2.2-rc.0', floor), true, 'an earlier prerelease is below') + assert.equal(versionBelowFloor('0.1.9', floor), true, 'an older minor is below') + assert.equal(versionBelowFloor('0.2.2-rc.1', floor), false, 'the floor itself is not below') + assert.equal(versionBelowFloor('0.2.2', floor), false, 'a release outranks its prerelease') + assert.equal(versionBelowFloor('0.2.2-rc.2', floor), false, 'a later prerelease is not below') + assert.equal(versionBelowFloor('0.2.3', floor), false, 'a newer release is not below') + assert.equal(versionBelowFloor('1.0.0', floor), false) +}) + +/** A fake host installation whose app-boot reports a fixed runtime version. */ +function seedFakeInstallation(root, version) { + const bootDir = path.join(root, 'node_modules', '@deepseek-ai', 'dsh-app-boot') + fs.mkdirSync(bootDir, { recursive: true }) + fs.writeFileSync(path.join(bootDir, 'package.json'), JSON.stringify({ name: '@deepseek-ai/dsh-app-boot', main: 'index.js' })) + fs.writeFileSync(path.join(bootDir, 'index.js'), `export const getDshRuntimeVersion = () => ${JSON.stringify(version)}\n`) + fs.writeFileSync(path.join(root, 'package.json'), '{}\n') + return root +} + +await test('a host below the floor hears one named diagnosis at activation', () => withTempDir('host-floor', async root => { + const { assertHostFloor } = await import('../lib/adapter/dsh/host-floor.js') + const packageDir = seedFakeInstallation(path.join(root, 'old-host'), '0.2.1') + const { ctx } = await bootHost({ withWorkingDirectory: true }) + try { + ctx.provide('pluginPackages', { resolution: { entries: [ + { scope: 'installation', name: '@deepseek-ai/dsh-app-boot', packageDir }] } }) + await assert.rejects(assertHostFloor(ctx), /requires DSH 0\.2\.2-rc\.1 or newer/i) + // The whole row answers with the same diagnosis instead of loading. + await assert.rejects(apply(ctx, {}), /0\.2\.2-rc\.1/, 'apply must refuse an older host by the declared floor') + } finally { await shutdown(ctx) } +})) + +await test('the floor releases hosts at or above it, and an unknown version is never refused', () => withTempDir('host-floor-ok', async root => { + const { assertHostFloor } = await import('../lib/adapter/dsh/host-floor.js') + for (const version of ['0.2.2-rc.1', '0.2.3', '0.3.0']) { + const packageDir = seedFakeInstallation(path.join(root, 'host-' + version), version) + const { ctx } = await bootHost({ withWorkingDirectory: true }) + try { + ctx.provide('pluginPackages', { resolution: { entries: [ + { scope: 'installation', name: '@deepseek-ai/dsh-app-boot', packageDir }] } }) + await assertHostFloor(ctx) + } finally { await shutdown(ctx) } + } + // No readable host version: refusing it would punish hosts the probe cannot see. + const { ctx } = await bootHost({ withWorkingDirectory: true }) + try { await assertHostFloor(ctx) } finally { await shutdown(ctx) } +})) + +report('integration-host-floor') diff --git a/test/integration-status-builtin.test.mjs b/test/integration-status-builtin.test.mjs new file mode 100644 index 00000000..7c6a715c --- /dev/null +++ b/test/integration-status-builtin.test.mjs @@ -0,0 +1,73 @@ +/** + * Built-in preset sessions publish their own scope diagnosis (#753). + * + * The status tool used to exist only behind an external preset row: a built-in + * mode row mounted its consumer and nothing else, so a session the host ran on + * a built-in preset had no named answer about its own publication at all, + * although the diagnosis itself already understood mode sessions. Pinned here: + * the built-in mode loop mounts the diagnosis gate that admits exactly this + * mode's sessions, the tool is visible and executable there, the diagnosis + * recognises the mode as selected, and another built-in mode is not admitted. + */ +import { createScope } from '@deepseek-ai/dsh-scope' +import { bootHost, shutdown, scopedToolNames } from './lib/host.mjs' +import { assert, test, report, withTempDir } from './lib/harness.mjs' +import { executeCall, prepareStep } from './lib/executor.mjs' +import { mountModeDiagnostics } from '../lib/integration/index.js' +import { applyResolvedMode } from '../lib/presets/builtin.js' +import { resolveModeConfig } from '../lib/presets/config.js' + +const dispatch = async (ctx, request) => { + const step = await prepareStep(ctx, request.agent) + try { return await executeCall(ctx, { ...request, toolSet: step.toolSet }) } + finally { step.dispose() } +} + +/** The exact composition the built-in loop in src/index.ts runs for one mode. */ +async function mountBuiltinMode(ctx, mode) { + await ctx.plugin({ async apply(inner) { + applyResolvedMode(inner, resolveModeConfig({}, mode), mode) + await mountModeDiagnostics(inner, mode) + } }) +} + +await test('a built-in preset session sees and executes its own status tool', () => withTempDir('status-builtin', async cwd => { + const { ctx } = await bootHost() + try { + await mountBuiltinMode(ctx, 'minimal') + const agent = { id: 'builtin', session: { id: 'builtin', mode: 'read-only', header: { cwd, agentPreset: 'minimal' } } } + await ctx.plugin({ inject: ['tools'], apply(inner) { agent.ctx = createScope(inner, agent).ctx } }) + await ctx.serial('agent/created', { agent }) + assert.notEqual(ctx.get('tools').get('ops_status', agent), undefined, 'the built-in row publishes the status tool to this agent') + const result = await dispatch(ctx, { agent, name: 'ops_status', arguments: {}, callId: 'builtin', signal: new AbortController().signal }) + assert.equal(result.isError, false, 'the status tool answers, it does not refuse') + assert.equal(result.value.state, 'ready', 'the diagnosis recognises the built-in mode as selected and sound') + assert.equal(result.value.preset, 'minimal', 'the built-in preset identity is reported, not an external row') + } finally { await shutdown(ctx) } +})) + +await test('a session of another built-in mode is not admitted by this gate', () => withTempDir('status-other-mode', async cwd => { + const { ctx } = await bootHost() + try { + await mountBuiltinMode(ctx, 'minimal') + const agent = { id: 'other', session: { id: 'other', mode: 'read-only', header: { cwd, agentPreset: 'standard' } } } + await ctx.plugin({ inject: ['tools'], apply(inner) { agent.ctx = createScope(inner, agent).ctx } }) + await ctx.serial('agent/created', { agent }) + assert.equal(scopedToolNames(ctx, agent).includes('ops_status'), false, 'another mode\'s session gets no status tool from this row') + } finally { await shutdown(ctx) } +})) + +await test('the mode row alone admits a plain built-in session and keeps answering it', () => withTempDir('status-repeat', async cwd => { + const { ctx } = await bootHost() + try { + await mountBuiltinMode(ctx, 'minimal') + const agent = { id: 'again', session: { id: 'again', mode: 'read-only', header: { cwd, agentPreset: 'minimal' } } } + await ctx.plugin({ inject: ['tools'], apply(inner) { agent.ctx = createScope(inner, agent).ctx } }) + await ctx.serial('agent/created', { agent }) + assert.notEqual(ctx.get('tools').get('ops_status', agent), undefined, 'the mode row owns the plain built-in session') + const answer = await dispatch(ctx, { agent, name: 'ops_status', arguments: {}, callId: 'again', signal: new AbortController().signal }) + assert.equal(answer.isError, false) + } finally { await shutdown(ctx) } +})) + +report('integration-status-builtin') diff --git a/test/lib/agent-loop.mjs b/test/lib/agent-loop.mjs index cdcd6fa4..826c0553 100644 --- a/test/lib/agent-loop.mjs +++ b/test/lib/agent-loop.mjs @@ -15,12 +15,13 @@ import Sandbox from '@deepseek-ai/dsh-sandbox-local' import FileSystem from '@deepseek-ai/dsh-fs-local' import NodePtc from '@deepseek-ai/dsh-ptc-runtime-node' import { integration } from '../../lib/integration/index.js' +import { apply as applyShippedRow } from '../../lib/index.js' import { assert } from './harness.mjs' import { WORKER } from './boundary.mjs' import { legacyComponent } from './legacy-component.mjs' import { workingDirectoryOver } from './host.mjs' let identity = 0 -export async function bootLoop(root, { ptcMode = false, discoveryConfig = {} } = {}) { +export async function bootLoop(root, { ptcMode = false, discoveryConfig = {}, preset = 'fixture', row, hostPlugins = [] } = {}) { const ctx = new Context(); ctx.logger.level = 0 // rc.5 asks the host for the directory a run_code call executes in. ctx.provide('workingDirectory', workingDirectoryOver(root)) @@ -29,6 +30,13 @@ export async function bootLoop(root, { ptcMode = false, discoveryConfig = {} } = await ctx.plugin(Tools, {}) await ctx.plugin(Policy, { mode: 'workspace-write', workspaceRoot: root }) await ctx.plugin(Subprocess); await ctx.plugin(Sandbox); await ctx.plugin(FileSystem, { cwd: root }) + // The host's own plugins load before any plugin row, exactly as a real profile + // orders them; measurement suites use this to give the shipped row a real + // official surface (read/glob/grep) to replace (#991). + for (const entry of hostPlugins) { + const [plugin, config] = Array.isArray(entry) ? entry : [entry, {}] + await ctx.plugin(plugin, config) + } if (ptcMode) await ctx.plugin(NodePtc, {}) await ctx.plugin(ptcMode ? PtcExecutor : NativeExecutor, {}) await ctx.plugin(inner => { @@ -45,11 +53,16 @@ export async function bootLoop(root, { ptcMode = false, discoveryConfig = {} } = inner.on('tools/execute', async (exec, next) => { const result = await next(); results.push({ exec, result }); return result }) inner.on('agent/error', ({ error }) => errors.push(String(error?.stack ?? error))) }) - const ptcFiber = await ctx.plugin(integration({ preset: 'fixture', profile: ptcMode ? 'ptc' : 'native', - binaryPath: WORKER, required: true, ...discoveryConfig })) + // Without a row the composition is the custom-preset integration fixture; with + // one it is the shipped single row exactly as a profile mounts it, so a suite + // can exercise a real mode (e.g. ops-ptc) instead of a synthetic preset. + const composition = row === undefined + ? integration({ preset, profile: ptcMode ? 'ptc' : 'native', binaryPath: WORKER, required: true, ...discoveryConfig }) + : { name: 'dsh-ops-shipped-row', apply: inner => applyShippedRow(inner, row) } + const ptcFiber = await ctx.plugin(composition) await ctx.plugin(Loop, { agents: [], maxParallelToolCalls: 3 }) async function agent(parentAgent) { - const { agent } = await ctx.get('agents').create({ sessionId: 'real-loop-' + ++identity, meta: { cwd: root, agentPreset: 'fixture' }, parentAgent, + const { agent } = await ctx.get('agents').create({ sessionId: 'real-loop-' + ++identity, meta: { cwd: root, agentPreset: preset }, parentAgent, agentOptions: { provider: 'fixture', model: 'scripted' } }) // Exercise a real committed mode event immediately before the first assembly. setSandboxMode(agent.session, 'workspace-write') diff --git a/test/lib/ptc-entry.mjs b/test/lib/ptc-entry.mjs new file mode 100644 index 00000000..3ae22705 --- /dev/null +++ b/test/lib/ptc-entry.mjs @@ -0,0 +1,295 @@ +/** + * The isolated OPS-PTC entry (#990). + * + * A headless comparison is only evidence when the group it measured truly + * entered the OPS-PTC routing: one run_code transport, the shipped ops-ptc row, + * and an executable worker. This module is that guarantee, packaged for the + * later measurement cards (V2-V5) to reuse: + * + * - bootEntryGroup boots one preset group on the real AgentLoop with the + * shipped single row mounted, inside this suite's throwaway DSH_HOME. + * Nothing outside the temp root is read; the probe file the entry inspects + * is freshly invented content, never a real session or credential. + * - collectEnvironment records the machine facts a measurement must be able to + * name: plugin commit, host version, platform, worker digest and staged + * receipt, sandbox mode, the row's resolved switches, and the actual preset. + * The provider is scripted and every report says so: a mocked provider can + * verify assembly only, and no real model request ever leaves the fixture. + * - enterPtc runs the minimal case (one scripted provider request) and + * separates every way the group could quietly prove nothing: a worker that + * cannot execute, a preset that is not the target (Standard is the + * deliberate counter-example), a transport that is not run_code, an + * execution that failed. An invalid report carries its reasons and no + * evidence block at all, so it can never be mistaken for a comparable + * result. Byte and cost accounting is deliberately not collected here; + * that is the V2 card's scope. + * + * @module dsh-ops/test/lib/ptc-entry + */ +import { createHash } from 'node:crypto' +import fs from 'node:fs' +import path from 'node:path' +import { presetOf } from '../../lib/consumer/identity.js' +import { modeOf } from '../../lib/presets/detect.js' +import { MODE_IDS } from '../../lib/presets/types.js' +import { resolveMergedConfig } from '../../lib/settings/merged.js' +import { bootLoop } from './agent-loop.mjs' +import { WORKER } from './boundary.mjs' +import { assert, PACKAGE_ROOT, TEST_HOME } from './harness.mjs' + +/** The preset this entry exists to prove: OPS (PTC), the sandboxed-code routing. */ +export const TARGET_PRESET = 'ops-ptc' + +/** Read a small file as trimmed text, or undefined when it cannot be read. */ +function readText(file) { + try { return fs.readFileSync(file, 'utf8').trim() } catch { return undefined } +} + +/** Parse one JSON file, or undefined when it cannot be read or parsed. */ +function readJson(file) { + try { return JSON.parse(fs.readFileSync(file, 'utf8')) } catch { return undefined } +} + +/** sha256 of one file, or undefined when it cannot be read. */ +function sha256OfFile(file) { + try { return createHash('sha256').update(fs.readFileSync(file)).digest('hex') } catch { return undefined } +} + +/** Whether the worker path names a file this process may execute. */ +function workerExecutable(file) { + try { fs.accessSync(file, fs.constants.X_OK); return true } catch { return false } +} + +/** + * The checked-out commit, read straight from the git directory. + * + * No subprocess is spawned and no environment is trusted: a plain checkout + * (a .git directory), a linked worktree (a .git pointer plus commondir) and a + * detached HEAD all resolve, and anything else honestly reports 'unknown' + * instead of a guess. A suite assertion turns the value into a claim. + * @param {string} [root] - the checkout root; defaults to the package root. + * @returns {string} the 40-hex commit, or 'unknown'. + */ +export function pluginCommit(root = PACKAGE_ROOT) { + const shaOf = value => typeof value === 'string' && /^[0-9a-f]{40}$/i.test(value) ? value.toLowerCase() : undefined + let gitdir = path.join(root, '.git'), common = gitdir + let stat + try { stat = fs.statSync(gitdir) } catch { return 'unknown' } + if (stat.isFile()) { + const pointer = readText(gitdir)?.match(/^gitdir:\s*(\S+)$/m)?.[1] + if (pointer === undefined) return 'unknown' + gitdir = path.resolve(root, pointer) + } + const shared = readText(path.join(gitdir, 'commondir')) + if (shared !== undefined) common = path.resolve(gitdir, shared) + const head = readText(path.join(gitdir, 'HEAD')) + const direct = shaOf(head) + if (direct !== undefined) return direct + const ref = head?.match(/^ref:\s*(\S+)$/)?.[1] + if (ref === undefined) return 'unknown' + for (const base of [gitdir, common]) { + const sha = shaOf(readText(path.join(base, ref))) + if (sha !== undefined) return sha + } + const packed = readText(path.join(common, 'packed-refs')) ?? '' + const escaped = ref.replace(/[^A-Za-z0-9/_.-]/g, '\\$&') + const line = packed.match(new RegExp('^([0-9a-f]{40}) ' + escaped + '$', 'm')) + return line === null ? 'unknown' : line[1] +} + +/** The staged runtime receipt for one platform, or null when nothing is staged. */ +function stagedReceipt(platform) { + const receipt = readJson(path.join(PACKAGE_ROOT, 'runtime', platform, 'provenance.json')) + return receipt === undefined ? null : receipt +} + +/** + * The shipped-row configuration one group boots with: only the target mode is + * enabled, every mode's worker is pinned to the same explicit binary. + * @param {string} mode - the single mode row to enable. + * @param {object} [options] - row inputs. + * @param {string} [options.worker] - the worker binary the row pins. + * @param {Record} [options.switches] - overrides for the enabled mode. + * @returns {Record} the row configuration for the shipped apply. + */ +export function entryRow(mode, { worker = WORKER, switches = {} } = {}) { + const row = { global: {}, discovery: { binaryPath: worker, required: true } } + for (const id of MODE_IDS) { + row[id] = { enabled: id === mode, binaryPath: worker, required: true, ...(id === mode ? switches : {}) } + } + return row +} + +/** + * The environment one group runs in: recorded machine facts, never claims. + * @param {object} options - what the group ran as. + * @param {string} options.root - the group's isolated work directory. + * @param {string} options.preset - the agentPreset the group's agents carry. + * @param {string|undefined} options.mode - the mode row the group enabled. + * @param {string} [options.worker] - the worker binary the row pins. + * @param {Record} options.row - the row configuration booted. + * @param {string} [options.sandboxMode] - the sandbox mode the loop commits. + * @returns {Promise} the environment summary. + */ +export async function collectEnvironment({ root, preset, mode, worker = WORKER, row, sandboxMode = 'workspace-write' } = {}) { + const platform = process.platform + '-' + process.arch + const resolved = resolveMergedConfig(row) + const boot = await import('@deepseek-ai/dsh-app-boot') + const digest = sha256OfFile(worker) + const staged = stagedReceipt(platform) + const state = mode === undefined ? undefined : resolved[mode] + return { + plugin: { commit: pluginCommit(), version: readJson(path.join(PACKAGE_ROOT, 'package.json'))?.version }, + host: { version: typeof boot.getDshRuntimeVersion === 'function' ? boot.getDshRuntimeVersion() : 'unknown' }, + platform, + node: process.version, + worker: { + file: worker, + sha256: digest, + executable: workerExecutable(worker), + staged: staged === null ? null : { target: staged.target, sha256: staged.sha256, version: staged.version }, + matchesStaged: staged === null || digest === undefined ? undefined : digest === staged.sha256, + }, + sandbox: { mode: sandboxMode, workspaceRoot: root }, + dshHome: TEST_HOME, + agentPreset: preset, + mode, + switches: state === undefined ? {} : { + enabled: state.enabled, + residentTools: state.config.residentTools, + dynamicTools: state.config.dynamicTools, + batchingGuidance: state.config.batchingGuidance, + batchingTelemetry: state.config.batchingTelemetry, + hideOfficialTools: state.config.hideOfficialTools, + bash: state.config.bash, + global: resolved.global, + }, + // A mocked provider verifies assembly only; no real model request leaves + // the fixture, and every report carries this marker instead of hiding it. + provider: 'scripted', + networkModelRequests: 0, + } +} + +/** + * Boot one isolated preset group on the real loop with the shipped row. + * + * The group's root must live inside this suite's throwaway DSH_HOME. A boot + * that fails because the row refuses a missing worker is reported as the + * 'worker' failure; any other boot failure is rethrown, because it is a suite + * defect, not an honest environment answer. + * @param {string} root - the isolated work directory (under the suite home). + * @param {object} [options] - group inputs. + * @param {string} [options.preset] - the agentPreset; defaults to the target. + * @param {Record} [options.switches] - mode row overrides. + * @param {'ptc'|'native'} [options.executor] - force the executor; defaults by mode. + * @param {string} [options.worker] - the worker binary; defaults to the test build. + * @param {string} [options.sandboxMode] - recorded with the environment. + * @param {Array} [options.hostPlugins] - host plugins (e.g. the official file + * tools) mounted before the shipped row, so a group measures against a real + * official surface instead of an empty registry. + * @returns {Promise} the booted group, or an invalid-boot report. + */ +export async function bootEntryGroup(root, { preset = TARGET_PRESET, switches = {}, executor, worker = WORKER, sandboxMode = 'workspace-write', hostPlugins = [] } = {}) { + assert.ok(path.resolve(root).startsWith(path.resolve(TEST_HOME)), + 'the entry only boots groups inside the suite throwaway DSH_HOME, got ' + root) + const mode = MODE_IDS.includes(preset) ? preset : undefined + const row = entryRow(mode ?? 'standard', { worker, switches }) + const environment = await collectEnvironment({ root, preset, mode, worker, row, sandboxMode }) + const ptcPresentation = executor === 'native' ? false : executor === 'ptc' ? true : mode === 'ops-ptc' || mode === 'ptc' + try { + const host = await bootLoop(root, { ptcMode: ptcPresentation, preset, row, hostPlugins }) + return { booted: true, host, root, worker, preset, mode, environment, close: () => host.close() } + } catch (error) { + const message = String(error?.message ?? error) + if (/Native worker/.test(message)) { + return { booted: false, reason: 'worker', detail: message, environment } + } + throw error + } +} + +/** The probe file enterPtc invents in the group root before it executes one program. */ +const PROBE_FILE = 'entry-probe.txt' + +/** + * Run the minimal case through the real loop and judge the routing honestly. + * + * Phase one is exactly one scripted provider request and executes nothing: it + * observes the wire surface the group would send. Only a run_code surface + * earns a second turn, which invents one fresh probe file and executes a + * single program through the worker. Checks run in a fixed priority order + * (worker, preset, route, execution) and the first failure becomes the + * report's reason; an invalid report never carries an evidence block. + * @param {object} group - a {@link bootEntryGroup} result. + * @param {object} [options] - validation inputs. + * @param {string} [options.target] - the preset the report demands. + * @returns {Promise} the entry report. + */ +export async function enterPtc(group, { target = TARGET_PRESET } = {}) { + if (group.booted === false) { + return { status: 'invalid', target, reason: group.reason, + findings: { worker: { ok: false, detail: group.detail } }, + environment: group.environment } + } + const { host, worker, environment } = group + const agent = await host.agent() + const identity = { + preset: presetOf(host.ctx, agent), + declared: agent.session?.header?.agentPreset, + mode: modeOf(host.ctx, agent), + } + let surface + const probe = await host.run(agent, [request => { + // A composition that declares no tool at all sends no tools array; that + // absence is data for the route check, not a crash. + surface = { + topLevel: (request.tools ?? []).map(tool => tool.name).sort(), + text: JSON.stringify(request.messages), + } + return [] + }]) + const routeOk = surface.topLevel.length === 1 && surface.topLevel[0] === 'run_code' + && /ops_[a-z0-9_]+:/.test(surface.text) + let execution = { ok: false, requests: 0, result: undefined } + if (routeOk) { + // Freshly invented content only: the probe file is this entry's own fixture. + fs.writeFileSync(path.join(group.root, PROBE_FILE), 'ops-ptc entry probe\n') + const program = '// plan: inspect the freshly made probe file once\n' + + 'const outcome = await tools.ops_inspect_local_file({ path: ' + JSON.stringify(PROBE_FILE) + ' })\n' + + 'return outcome' + const executed = await host.run(agent, [ + () => [{ name: 'run_code', args: { description: 'Prove the OPS-PTC entry executes', code: program } }], []]) + const result = executed.results.find(value => value.exec.name === 'run_code')?.result + execution = { ok: result?.isError === false, requests: executed.requests.length, result } + } + const findings = { + worker: { ok: workerExecutable(worker), file: worker }, + preset: { ok: identity.preset === target, target, actual: identity.preset, declared: identity.declared, mode: identity.mode }, + route: { ok: routeOk, expected: ['run_code'], topLevel: surface.topLevel, transport: routeOk ? 'ptc' : 'unknown-or-native' }, + execution: routeOk + ? { ok: execution.ok, requests: execution.requests } + : { ok: null, attempted: false, note: 'route invalid: nothing was executed' }, + } + const order = ['worker', 'preset', 'route'] + if (routeOk) order.push('execution') + const failed = order.filter(reason => findings[reason].ok !== true) + if (failed.length > 0) { + // The refusal is explicit and carries nothing that could be read as a + // comparable result: no evidence block, no measurement of any kind. + return { status: 'invalid', target, reason: failed[0], findings, environment } + } + return { + status: 'entered', + target, + findings, + evidence: { + topLevel: surface.topLevel, + preset: identity.preset, + mode: identity.mode, + requests: { probe: probe.requests.length, execution: execution.requests }, + executed: execution.result, + }, + environment, + } +} diff --git a/test/mode-contract-audit.test.mjs b/test/mode-contract-audit.test.mjs index 6cf49a97..eb841413 100644 --- a/test/mode-contract-audit.test.mjs +++ b/test/mode-contract-audit.test.mjs @@ -53,10 +53,20 @@ for (const [mode, component] of [['ptc-control', undefined], ['ptc', Ptc], ['ops for (const name of ['ops_inspect_local_file', 'ops_glob', 'ops_grep', 'edit', 'read_image']) assert.ok(direct.includes(name), name) } else { assert.deepEqual(direct, ['run_code']) - // Built-in PTC modes intentionally select no official file replacements. - for (const name of official) { assert.ok(admitted.includes(name), name); assert.ok(sdk.includes(`${name}:`), name) } - if (!component) assert.doesNotMatch(text, /ops_/u) - else assert.ok(sdk.includes('ops_inspect_local_file:')) + if (!component) { + // The plain PTC control has no OPS row, so every official tool stays. + for (const name of official) { assert.ok(admitted.includes(name), name); assert.ok(sdk.includes(`${name}:`), name) } + assert.doesNotMatch(text, /ops_/u) + } else { + // hideOfficialTools defaults on: the replaced file tools leave the + // admission set and the SDK; mutation and image tools stay. + for (const name of ['read', 'glob', 'grep']) { + assert.equal(admitted.includes(name), false, `${name} must be hidden`) + assert.doesNotMatch(sdk, new RegExp('(?:^|\\n)\\s*' + name + ':', 'u'), `${name} must leave the SDK`) + } + for (const name of ['edit', 'read_image']) assert.ok(admitted.includes(name), `${name} stays`) + assert.ok(sdk.includes('ops_inspect_local_file:')) + } } return { direct, admitted, sdk } } finally { step.dispose() } @@ -74,9 +84,15 @@ for (const [mode, component] of [['ptc-control', undefined], ['ptc', Ptc], ['ops assert.ok(next.admitted.includes('ops_glob')); assert.ok(next.sdk.includes('ops_glob:')) assert.match(next.sdk, /paths \(default\): string\[\]/u) assert.match(next.sdk, /files_with_matches: \{path\}\[\]/u) - const value = await run("return {official:await tools.glob({}),ops:await tools.ops_glob({pattern:['*.txt'],outputMode:'paths'})};") - assert.equal(value.official, 'glob'); assert.equal(value.ops.status, 'completed') + const value = await run("const kind = typeof tools.glob; return {kind, ops: await tools.ops_glob({pattern:['*.txt'],outputMode:'paths'})};") + assert.equal(value.kind, 'undefined', 'the replaced official glob binding is gone from the SDK') + assert.equal(value.ops.status, 'completed') assert.ok(value.ops.data.items.some(file => file.endsWith('note.txt'))) + // The kept names stay present in the program bindings, not only in the + // admission set: presence is compared per name across all three views. + const kept = await run("return {edit: typeof tools.edit, image: typeof tools.read_image}") + assert.equal(kept.edit, 'function', 'the official edit binding stays callable') + assert.equal(kept.image, 'function', 'the official read_image binding stays callable') } if (row) { await row.dispose() diff --git a/test/modes.test.mjs b/test/modes.test.mjs index d0ff2ff3..e3897cb3 100644 --- a/test/modes.test.mjs +++ b/test/modes.test.mjs @@ -92,13 +92,16 @@ await test('replacement targets map each mode to the tools it hides, platform by const linux = replacementTargets('standard', 'linux') assert.equal(linux.has('pwsh'), false, 'POSIX keeps the host bash') assert.equal(replacementTargets('cordis', 'win32').get('read'), publicToolName('inspect_local_file')) - // Minimal, PTC and its ops-ptc sibling replace only the Windows shell; their - // file tools stay official. - for (const mode of ['minimal', 'ptc', 'ops-ptc']) { - const targets = replacementTargets(mode, 'win32', true) - assert.deepEqual([...targets.keys()].sort(), ['pwsh', 'pwsh_persistent'].sort(), mode) - assert.equal([...targets.values()].every(value => value === BASH_TOOL), true, mode) - assert.equal(replacementTargets(mode, 'linux', true).size, 0, `${mode} replaces nothing on POSIX`) + // Minimal publishes no file surface, so only the Windows shell is replaced. + assert.deepEqual([...replacementTargets('minimal', 'win32', true).keys()].sort(), ['pwsh', 'pwsh_persistent'].sort()) + assert.equal(replacementTargets('minimal', 'linux', true).size, 0, 'minimal replaces nothing on POSIX') + // PTC plans contribute their file targets now; whether a name is actually + // hidden is decided per activation by the publisher accessor. + for (const mode of ['ptc', 'ops-ptc']) { + const win = replacementTargets(mode, 'win32', true) + assert.deepEqual([...win.keys()].sort(), ['glob', 'grep', 'pwsh', 'pwsh_persistent', 'read'].sort(), mode) + assert.equal(win.get('pwsh'), BASH_TOOL, mode) + assert.deepEqual([...replacementTargets(mode, 'linux', true).keys()].sort(), ['glob', 'grep', 'read'].sort(), mode) } assert.deepEqual(residentTools('standard'), RESIDENT) assert.deepEqual(residentTools('cordis'), RESIDENT) @@ -283,7 +286,7 @@ await test('dynamic loading serves Minimal always and enabled modes, and never P } finally { await shutdown(ctx) } })) -await test('PTC mode publishes the scoped SDK surface and replaces only the Windows shell', () => withTempDir('mode-ptc', async cwd => { +await test('PTC mode publishes the scoped SDK surface, hides the replaced official file tools and replaces the Windows shell', () => withTempDir('mode-ptc', async cwd => { const { ctx, names, mint, assemble } = await boot() try { const NodePtc = (await import('@deepseek-ai/dsh-ptc-runtime-node')).default @@ -295,13 +298,21 @@ await test('PTC mode publishes the scoped SDK surface and replaces only the Wind const row = await ctx.plugin(Ptc, { required: true }) const agent = await mint({ preset: 'ptc', presentation: 'ptc', cwd }) await waitFor(() => names(agent).includes('ops_load_sdk'), { what: 'PTC SDK publication' }) - for (const name of OFFICIAL) assert.ok(names(agent).includes(name), `PTC replaces no official file tool; ${name} stays visible`) - assert.equal(names(agent).includes('ops_bash'), process.platform === 'win32', 'only the Windows shell is replaced') + const held = await prepareStep(ctx, agent) + try { + const admitted = ctx.get('tools').definitions(agent, held.toolSet).map(tool => tool.name) + for (const name of ['read', 'glob', 'grep']) assert.equal(admitted.includes(name), false, `PTC hides the replaced official ${name}`) + for (const name of ['write']) assert.ok(admitted.includes(name), `PTC keeps the unreplaced official ${name}`) + } finally { held.dispose() } + assert.equal(names(agent).includes('ops_bash'), process.platform === 'win32', 'the Windows shell replacement publishes ops_bash') // A Standard-preset agent is not served by the PTC row. const standard = await mint({ preset: 'standard', cwd }) await wait(300) assert.equal(names(standard).includes('ops_load_sdk'), false) await row.dispose() + // Withdrawing the row withdraws its replacements too: the official file + // tools the PTC row had hidden come back on the registry surface. + await waitFor(() => ['read', 'glob', 'grep'].every(name => names(agent).includes(name)), { what: 'official file tools restored with the PTC row' }) } finally { await shutdown(ctx) } })) diff --git a/test/native-ranges.test.mjs b/test/native-ranges.test.mjs index 33e8b41d..baaa76aa 100644 --- a/test/native-ranges.test.mjs +++ b/test/native-ranges.test.mjs @@ -24,7 +24,8 @@ await test('real bounded ranges keep exact body, shared path base, full revision assert.match(text, /more lines; nextOffset=3/); assert.match(text, /\(empty file\)/) assert.match(text, /end of file; complete=true/); assert.match(text, /3\/3 requested ranges returned/) assert.doesNotMatch(text, /worker-limit|bytes=|lineCount=|matched=/) - assert.equal(text.split('\n').filter(line => line.startsWith('[Notice:')).length, 1, 'one aggregate notice covers all fully returned ranges') + assert.equal(text.split('\n').filter(line => line.startsWith('[Notice:')).length, 2, 'one escape notice plus one aggregate notice cover all fully returned ranges') + assert.match(text, /rendered line\(s\) use reversible escapes/, 'escaped lines state the round-trip contract once') assert.equal(JSON.stringify(value), before) })) await test('different parents retain absolute quoted paths; path control characters cannot inject headers', () => { diff --git a/test/niubash-faces-consistency.test.mjs b/test/niubash-faces-consistency.test.mjs new file mode 100644 index 00000000..df6667a9 --- /dev/null +++ b/test/niubash-faces-consistency.test.mjs @@ -0,0 +1,65 @@ +/** + * The Niubash face list must match what winuxcmd actually implements (#966). + * + * A face the multi-call binary cannot dispatch still becomes an argv0 stub on + * the prepared PATH: the stub shadows a system command (jq, link) with an entry + * that only answers "command not found". Two contracts are pinned here: the + * static list names no unimplemented applet, every generated face dispatches + * for real inside winuxcmd, and a prepared cache that still carries a stale + * face is retired and rebuilt instead of being reused (#966 cache upgrade). + */ +import fs from 'node:fs' +import path from 'node:path' +import { spawnSync } from 'node:child_process' +import { assert, test, report, withTempDir, PACKAGE_ROOT } from './lib/harness.mjs' +import { NIUBASH_FACES, niubashFacePath } from '../lib/niubash/faces.js' +import { NIUBASH_PIN } from '../lib/cli/niubash-payload.js' + +await test('the face list names no applet winuxcmd cannot dispatch', () => { + // Measured against the frozen payload: winuxcmd answers "command not found" + // with exit 127 for these, so their argv0 stubs only shadow the system tools. + assert.equal(NIUBASH_FACES.includes('jq'), false, 'winuxcmd reports command not found: jq; the stub shadows a real system jq') + assert.equal(NIUBASH_FACES.includes('link'), false, 'winuxcmd reports command not found: link') +}) + +/** Live-probe base: the staged frozen tree, or the gate-provided fixture. */ +const staged = path.join(PACKAGE_ROOT, 'runtime', 'win32-x64', 'niubash') +const base = process.env.DSH_OPS_TEST_NIUBASH ? path.dirname(process.env.DSH_OPS_TEST_NIUBASH) : staged +const winuxcmd = path.join(base, 'winuxcmd', 'usr', 'bin', 'winuxcmd.exe') +const live = process.platform === 'win32' && fs.existsSync(winuxcmd) + +await test('every generated face dispatches inside winuxcmd, not to command-not-found', () => { + if (!live) { + console.log(' note: no staged winuxcmd on this host; the live probe needs the frozen payload') + return + } + const generated = NIUBASH_FACES.map(niubashFacePath).filter(name => !Object.hasOwn(NIUBASH_PIN.files, name)) + const missing = [] + for (const name of generated) { + const face = path.basename(name, '.exe') + const probe = spawnSync(winuxcmd, [face], + { timeout: 5000, windowsHide: true, stdio: ['ignore', 'pipe', 'pipe'], encoding: 'utf8' }) + if (probe.status === 127) missing.push(face) + } + assert.deepEqual(missing, [], 'each generated face must be a real winuxcmd applet') +}) + +await test('a prepared cache carrying a stale face is retired and rebuilt, not reused', () => withTempDir('faces-cache-upgrade', async root => { + if (!live) { + console.log(' note: no staged payload on this host; the cache-upgrade probe needs the frozen tree') + return + } + const { prepareNiubashVersion } = await import('../lib/niubash/runtime.js') + const source = base + const target = path.join(root, 'prepared') + await prepareNiubashVersion(source, target) + const staleFace = path.join(target, 'winuxcmd', 'usr', 'bin', 'jq.exe') + const winuxcmdCopy = path.join(target, 'winuxcmd', 'usr', 'bin', 'winuxcmd.exe') + // Simulate a cache written by a face list that still declared jq (#966). + fs.copyFileSync(winuxcmdCopy, staleFace) + // Reuse must not bless the stale stub: it shadows the system jq. + await prepareNiubashVersion(source, target) + assert.equal(fs.existsSync(staleFace), false, 'a face the list no longer declares must not survive a cache reuse') +})) + +report('niubash-faces-consistency') diff --git a/test/niubash-workspace-mode.test.mjs b/test/niubash-workspace-mode.test.mjs new file mode 100644 index 00000000..57ec35a0 --- /dev/null +++ b/test/niubash-workspace-mode.test.mjs @@ -0,0 +1,29 @@ +/** The prepared-root workspace boundary is armed for confined modes only (#870). */ +import path from 'node:path' +import { niubashPreparedRoot, prepareNiubash } from '../lib/niubash/runtime.js' +import { NIUBASH_PACKAGE_PATH } from '../lib/cli/niubash-payload.js' +import { syntheticNiubashTree } from './lib/niubash-distribution-fixture.mjs' +import { assert, report, test, withTempDir } from './lib/harness.mjs' + +const signal = new AbortController().signal + +await test('full access prepares a workspace-contained DSH_HOME; confined modes still refuse it', async () => { + await withTempDir('niu-ws-mode', async root => { + const source = syntheticNiubashTree(path.join(root, 'package', NIUBASH_PACKAGE_PATH)) + const file = path.join(source, 'niu.exe') + const inside = { DSH_HOME: path.join(root, 'home') } + const prepare = mode => prepareNiubash(file, 'inherited', signal, { workspaceRoot: root, env: inside, ...(mode ? { mode } : {}) }) + // Full access runs unconfined and keeps the account's own authority over every path, so the + // default DSH_HOME inside a home workspace must not make Niubash unusable (#870). + const full = await prepare('danger-full-access') + assert.equal(full.file, path.join(niubashPreparedRoot(inside), 'niu.exe')) + assert.equal(full.env.PATH.split(path.delimiter).at(-1), 'inherited', 'the inherited PATH stays last') + assert.equal(full.env.NIU_ROOT, path.join(niubashPreparedRoot(inside), 'winuxcmd')) + // Confined modes keep the refusal: workspace labeling must never reach an executable. + for (const mode of ['read-only', 'workspace-write', undefined]) { + await assert.rejects(prepare(mode), error => error.message.startsWith('EXECUTOR_UNAVAILABLE:') + && error.message.includes('outside the session workspace')) + } + }) +}) +report('niubash-workspace-mode') diff --git a/test/pdf-indirect-length.test.mjs b/test/pdf-indirect-length.test.mjs new file mode 100644 index 00000000..f17070fa --- /dev/null +++ b/test/pdf-indirect-length.test.mjs @@ -0,0 +1,78 @@ +/** #788: an indirect /Length "N G R" resolves through the same classic xref table. + * No endstream scanning, no recovery: absent, free, chained, non-integer or oversized + * providers keep failing closed exactly like an invalid direct length. */ +import fs from 'node:fs' +import path from 'node:path' +import { assert, test, report, withTempDir } from './lib/harness.mjs' +import { callWorker, request, envelopeOf, requireWorker } from './lib/boundary.mjs' +import { pdf, show } from './lib/pdf-fixture.mjs' +requireWorker('pdf-indirect-length') + +const catalog = '<< /Type /Catalog /Pages 2 0 R >>' +const font = '<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica /Encoding /WinAnsiEncoding >>' +const pages = kids => '<< /Type /Pages /Kids [' + kids.join(' ') + '] /Count ' + kids.length + ' >>' +const page = ref => '<< /Type /Page /Parent 2 0 R /MediaBox [0 0 612 792] /Resources << /Font << /F1 3 0 R >> >> /Contents ' + ref + ' >>' +const direct = content => '<< /Length ' + Buffer.byteLength(content) + ' >>\nstream\n' + content + '\nendstream' +const indirect = (content, ref) => '<< /Length ' + ref + ' >>\nstream\n' + content + '\nendstream' + +async function run(root, args) { + const payload = { version: 1, requestId: 'pdf-indirect-test', component: 'file', + budget: { timeoutMs: 30000, maxBytes: 131072 }, + request: request('read_pdf', args, { root }) } + return envelopeOf(await callWorker(JSON.stringify(payload)), assert) +} +const failure = (body, code) => { + assert.equal(body.error?.code, code, JSON.stringify(body)) + assert.equal(body.error.effectState, 'none') + assert.notEqual(body.complete, true) +} +const fileAt = (root, value) => { const file = path.join(root, 'document.pdf'); fs.writeFileSync(file, value); return file } + +await test('indirect stream lengths read like direct ones in both file orders', async () => { + await withTempDir('pdf-indirect', async root => { + // Objects: 4 is the page, 5 its content stream, 6 the length provider that follows it. + const providerAfter = pdf([catalog, pages(['4 0 R']), font, page('5 0 R'), + indirect(show('later provider'), '6 0 R'), String(Buffer.byteLength(show('later provider')))]) + const after = await run(root, { path: fileAt(root, providerAfter) }) + assert.equal(after.error, undefined, JSON.stringify(after)) + assert.equal(after.status, 'completed'); assert.equal(after.complete, true) + assert.deepEqual(after.items.map(i => i.text), ['later provider']) + // Provider object 5 sits before the stream object 6 in file order. + const providerBefore = pdf([catalog, pages(['4 0 R']), font, page('6 0 R'), + String(Buffer.byteLength(show('earlier provider'))), indirect(show('earlier provider'), '5 0 R')]) + const before = await run(root, { path: fileAt(root, providerBefore) }) + assert.equal(before.error, undefined, JSON.stringify(before)) + assert.deepEqual(before.items.map(i => i.text), ['earlier provider']) + }) +}) + +await test('direct and indirect lengths coexist in one document', async () => { + await withTempDir('pdf-indirect-mixed', async root => { + const one = show('direct page'), two = show('indirect page') + const mixed = pdf([catalog, pages(['4 0 R', '6 0 R']), font, page('5 0 R'), direct(one), + page('7 0 R'), indirect(two, '8 0 R'), String(Buffer.byteLength(two))]) + const body = await run(root, { path: fileAt(root, mixed) }) + assert.equal(body.error, undefined, JSON.stringify(body)) + assert.deepEqual(body.items.map(i => i.text), ['direct page', 'indirect page']) + }) +}) + +await test('absent, non-integer, mismatched, chained and oversized providers fail closed', async () => { + await withTempDir('pdf-indirect-negative', async root => { + const text = show('safe') + const cases = [ + ['9 0 R', String(Buffer.byteLength(text)), 'DOCUMENT_PARSE_ERROR'], + ['3 0 R', font, 'DOCUMENT_PARSE_ERROR'], + ['6 1 R', String(Buffer.byteLength(text)), 'DOCUMENT_PARSE_ERROR'], + ['2 0 R', '2 0 R', 'DOCUMENT_PARSE_ERROR'], + ] + for (const [ref, provider, code] of cases) { + const bytes = pdf([catalog, pages(['4 0 R']), font, page('5 0 R'), indirect(text, ref), provider]) + failure(await run(root, { path: fileAt(root, bytes) }), code) + } + failure(await run(root, { path: fileAt(root, pdf([catalog, pages(['4 0 R']), font, page('5 0 R'), + indirect(show('x'), '6 0 R'), String(1024 * 1024 + 1)])) }), 'OUTPUT_LIMIT') + }) +}) + +report('pdf-indirect-length') diff --git a/test/preset-default-leftover.test.mjs b/test/preset-default-leftover.test.mjs new file mode 100644 index 00000000..6c792259 --- /dev/null +++ b/test/preset-default-leftover.test.mjs @@ -0,0 +1,147 @@ +/** + * Unloading the row must not strand the profile's default agent preset. + * + * The operator's one click writes `selectedDefault: ops-ptc` through the host + * settings service (#928's only write point). Disabling or unloading the row + * removes the preset, and the profile used to keep naming it: every new + * session then failed to compose against a preset that no longer existed. + * The unload lease here restores the host default — and never touches a + * default that names any other preset. + */ +import { Context } from '@deepseek-ai/cordis' +import { registerOpsPtcPreset } from '../lib/presets/ops-ptc/register.js' +import { assert, test, report, waitFor } from './lib/harness.mjs' + +/** + * A registry double: records the definition, answers with a counting + * disposer, and exposes the `config`/`defaultId` pair the switch reads. + * @param {string} [selected] - the already-saved `selectedDefault`, if any. + * @returns {object} the registry double with its recorded state. + */ +function stubRegistry(selected) { + const state = { definitions: [], disposed: 0, selected, updates: [], order: [] } + return { + state, + config: { default: 'standard', selectedDefault: { get: () => state.selected } }, + get defaultId() { return state.selected ?? 'standard' }, + register(definition) { + state.definitions.push(definition) + state.order.push('register') + return Promise.resolve(async () => { state.disposed += 1 }) + }, + } +} + +/** + * A settings-service double: records each update and applies the volatile + * write to the registry state, exactly as the host hot-applies it. + * @param {object} registry - the registry the written default belongs to. + * @returns {object} the settings service. + */ +function stubSettings(registry) { + return { + async update(namespace, patch) { + registry.state.updates.push({ namespace, patch: { ...patch } }) + registry.state.order.push('default') + if (typeof patch.selectedDefault === 'string') registry.state.selected = patch.selectedDefault + }, + } +} + +/** + * A context double that plays the row root and the injected child, recording + * every tracked effect so an unload can be driven exactly. + * @param {object} registry - the service exposed as `agentPresets`. + * @param {object|null} [settings] - the service exposed as `settings`. + * @returns {object} the stub root plus its recorded effects. + */ +function stubRoot(registry, settings) { + const events = { effects: [] } + const services = { agentPresets: registry, settings: settings === undefined ? stubSettings(registry) : settings } + const root = { + logger: { warn: () => {} }, + inject(deps, callback) { callback(root) }, + get(name) { return services[name] }, + effect(callback, label) { + const cleanup = callback() + events.effects.push({ label, cleanup }) + return () => {} + }, + } + return { root, events } +} + +/** Wait for the registration to settle, then find one tracked effect by label. */ +async function settleAndFind(registry, events, label) { + await waitFor(() => registry.state.definitions.length === 1, { what: 'the ops-ptc registration' }) + await new Promise(resolve => setImmediate(resolve)) + const effect = events.effects.find(entry => entry.label === label) + assert.ok(effect !== undefined, `the \`${label}\` effect exists`) + return effect +} + +await test('unloading the row restores a profile default that still names the removed preset (#928)', async () => { + const registry = stubRegistry('ops-ptc') + const { root, events } = stubRoot(registry) + const warnings = [] + registerOpsPtcPreset(root, { warn: message => { warnings.push(message) } }) + const preset = { applyDefault: () => { throw new Error('this check never presses the button') } } + assert.ok(registry.state.definitions.length === 1 || true, 'the registration is in flight') + // The field state from an older install: the profile already names ops-ptc. + const release = await settleAndFind(registry, events, 'dsh-ops: ops-ptc preset') + await release.cleanup() + await waitFor(() => registry.state.disposed === 1, { what: 'the released registration' }) + const lease = await settleAndFind(registry, events, 'dsh-ops: ops-ptc preset default lease') + await lease.cleanup() + await waitFor(() => registry.state.selected === 'standard', { what: 'the restored host default' }) + assert.deepEqual(registry.state.updates, + [{ namespace: 'agent-preset-registry', patch: { selectedDefault: 'standard' } }], + 'the restore writes exactly the host default back') + assert.ok(warnings.some(w => /restored the default agent preset/.test(w)), + 'the restore is reported so the operator can see what changed') + void preset +}) + +await test('an unload never touches a default that names another preset', async () => { + const registry = stubRegistry('ptc') + const { root, events } = stubRoot(registry) + registerOpsPtcPreset(root, { warn: () => {} }) + const release = await settleAndFind(registry, events, 'dsh-ops: ops-ptc preset') + await release.cleanup() + const lease = await settleAndFind(registry, events, 'dsh-ops: ops-ptc preset default lease') + await lease.cleanup() + await new Promise(resolve => setImmediate(resolve)) + assert.deepEqual(registry.state.updates, [], 'a default another preset owns is never written') + assert.equal(registry.state.selected, 'ptc', 'the operator default stays exactly as it was') +}) + +await test('an unload without a settings service reports the leftover instead of guessing', async () => { + const registry = stubRegistry('ops-ptc') + const { root, events } = stubRoot(registry, null) + const warnings = [] + registerOpsPtcPreset(root, { warn: message => { warnings.push(message) } }) + const lease = await settleAndFind(registry, events, 'dsh-ops: ops-ptc preset default lease') + await lease.cleanup() + await new Promise(resolve => setImmediate(resolve)) + assert.equal(registry.state.selected, 'ops-ptc', 'nothing was written without a settings service') + assert.ok(warnings.some(w => /still names the removed ops-ptc preset/.test(w)), + 'the warning tells the operator the profile keeps a dead default') +}) + +await test('a real Cordis row restores the stranded default when its fiber disposes (#928)', async () => { + const ctx = new Context() + ctx.logger.level = 0 + const registry = stubRegistry('ops-ptc') + ctx.provide('agentPresets', registry) + ctx.provide('settings', stubSettings(registry)) + registerOpsPtcPreset(ctx) + await waitFor(() => registry.state.definitions.length === 1, { what: 'the real injection' }) + await ctx.fiber.dispose() + await waitFor(() => registry.state.disposed === 1, { what: 'the fiber disposal to release the preset' }) + await waitFor(() => registry.state.selected === 'standard', { what: 'the stranded default restored on unload' }) + assert.deepEqual(registry.state.updates, + [{ namespace: 'agent-preset-registry', patch: { selectedDefault: 'standard' } }], + 'the disposal wrote the host default back once') +}) + +report('preset-default-leftover') diff --git a/test/preset-integration.test.mjs b/test/preset-integration.test.mjs index ae7bb7df..d0d64385 100644 --- a/test/preset-integration.test.mjs +++ b/test/preset-integration.test.mjs @@ -5,7 +5,9 @@ import path from 'node:path' import { createScope, scopeOf } from '@deepseek-ai/dsh-scope' import Subprocess from '@deepseek-ai/dsh-subprocess-local' import Sandbox from '@deepseek-ai/dsh-sandbox-local' -import { integration, resolveIntegrationConfig } from 'dsh-ops/integration' +import { integration, resolveIntegrationConfig, diagnose } from 'dsh-ops/integration' +import { PTC_LOADABLE_TOOLS } from '../lib/ptc/config.js' +import { publicToolName } from '../lib/policy.js' import NodePtc from '@deepseek-ai/dsh-ptc-runtime-node' import * as Standard from '../lib/components/mode-standard/index.js' import * as DiscoveryModule from '../lib/components/discovery/index.js' @@ -179,4 +181,100 @@ await test('conditional replacements stay agent bound and explicit ancestor mask parentMask() } finally { await shutdown(ctx) } })) +/** Boot one custom PTC row, lift the PTC executor and return the settled surface. */ +async function bootPtcRow(cwd, options) { + const { ctx, mint, names } = await boot() + ctx.provide('ptcRuntime', { language: 'typescript' }) + await ctx.plugin(integration({ ...options, profile: 'ptc' })) + const agent = await mint(options.preset, cwd) + selectExecutor(agent.ctx, 'ptc') + await ctx.get('systemPrompt').assemble({ scope: agent, agent }) + return { ctx, agent, names } +} +await test('custom rows parse the disclosure tier with an on-demand fallback and keep the alwaysLoaded cap contract', () => { + // D4 (#988): the custom path accepts the same explicit tier key the builtin rows + // parse (D2/#986). An omitted key stays the legacy on-demand default; a Native + // presentation row rejects the key the same way non-PTC builtin rows do. + assert.equal(resolveIntegrationConfig({ preset: 'x', profile: 'ptc' }).ptc.disclosurePolicy, 'on-demand') + assert.equal(resolveIntegrationConfig({ preset: 'x', profile: 'ptc', disclosurePolicy: 'full' }).ptc.disclosurePolicy, 'full') + for (const raw of [{ preset: 'x', profile: 'ptc', disclosurePolicy: 'sometimes' }, + { preset: 'x', profile: 'native', disclosurePolicy: 'full' }]) assert.throws(() => resolveIntegrationConfig(raw), /disclosurePolicy/) + // The legacy validation contract is unchanged on both sides: the custom path keeps + // rejecting alwaysLoaded outside allowedTools while builtin rows accept it silently + // (locked in test/ptc-config-truth.test.mjs), and a saved legal row still resolves. + assert.throws(() => resolveIntegrationConfig({ preset: 'x', profile: 'ptc', allowedTools: ['stat'], alwaysLoaded: ['grep'] }), /alwaysLoaded must be in allowedTools/) + const saved = resolveIntegrationConfig({ preset: 'x', profile: 'ptc', allowedTools: ['stat'], alwaysLoaded: ['stat'], disclosurePolicy: 'full' }) + assert.deepEqual(saved.ptc.alwaysLoaded, ['stat']) + assert.equal(saved.ptc.disclosurePolicy, 'full') +}) +await test('legacy custom PTC row with discovery off: empty pool, inspect-only first round, no ops_result false positive', () => withTempDir('preset-ptc-off', async cwd => { + const { ctx, agent, names } = await bootPtcRow(cwd, { preset: 'ptc-off', discovery: false }) + try { + const parsed = resolveIntegrationConfig({ preset: 'ptc-off', profile: 'ptc', discovery: false }) + assert.deepEqual(parsed.ptc.allowedTools, [], 'discovery off parses an empty pool, unlike the builtin default') + assert.equal(parsed.ptc.disclosurePolicy, 'on-demand') + await waitFor(async () => (await names(agent)).includes('ops_inspect_local_file')) + const firstRound = (await names(agent)).filter(name => name.startsWith('ops_')) + assert.deepEqual(firstRound, ['ops_inspect_local_file', 'ops_status']) + assert.equal(firstRound.includes('ops_load_sdk'), false) + const d = await diagnose(ctx, agent) + assert.equal(d.state, 'ready', JSON.stringify(d.issues)) + assert.equal(d.issues.some(row => row.code === 'PUBLICATION_UNAVAILABLE'), false, 'a selectable-but-unloaded API is not a publication gap') + } finally { await shutdown(ctx) } +})) +await test('custom full tier with an unfilled cap publishes the whole pool in the first round and stays ready', () => withTempDir('preset-ptc-full', async cwd => { + const { ctx, agent, names } = await bootPtcRow(cwd, { preset: 'ptc-full', disclosurePolicy: 'full' }) + try { + const parsed = resolveIntegrationConfig({ preset: 'ptc-full', profile: 'ptc', disclosurePolicy: 'full' }) + assert.deepEqual(parsed.ptc.allowedTools, PTC_LOADABLE_TOOLS) + assert.equal(parsed.ptc.disclosurePolicy, 'full') + await waitFor(async () => (await names(agent)).includes('ops_batch')) + const firstRound = (await names(agent)).filter(name => name.startsWith('ops_')) + assert.deepEqual(firstRound, [...PTC_LOADABLE_TOOLS.map(publicToolName), 'ops_inspect_local_file', 'ops_status'].sort()) + assert.equal(firstRound.includes('ops_load_sdk'), false, 'the full tier is loader-less') + const d = await diagnose(ctx, agent) + assert.equal(d.state, 'ready', JSON.stringify(d.issues)) + assert.equal(d.issues.some(row => row.code === 'PUBLICATION_UNAVAILABLE'), false) + } finally { await shutdown(ctx) } +})) +await test('custom full tier with a narrow cap stays narrow: inspect + stat + result only', () => withTempDir('preset-ptc-narrow', async cwd => { + const { ctx, agent, names } = await bootPtcRow(cwd, { preset: 'ptc-narrow', disclosurePolicy: 'full', allowedTools: ['stat'] }) + try { + const parsed = resolveIntegrationConfig({ preset: 'ptc-narrow', profile: 'ptc', disclosurePolicy: 'full', allowedTools: ['stat'] }) + assert.deepEqual(parsed.ptc.allowedTools, ['stat'], 'an explicit narrow pool must not widen to the default whole pool') + await waitFor(async () => (await names(agent)).includes('ops_stat')) + const firstRound = (await names(agent)).filter(name => name.startsWith('ops_')) + assert.deepEqual(firstRound, ['ops_inspect_local_file', 'ops_result', 'ops_stat', 'ops_status']) + for (const absent of ['ops_grep', 'ops_load_sdk']) assert.equal(firstRound.includes(absent), false) + const d = await diagnose(ctx, agent) + assert.equal(d.state, 'ready', JSON.stringify(d.issues)) + assert.equal(d.issues.some(row => row.code === 'PUBLICATION_UNAVAILABLE'), false) + } finally { await shutdown(ctx) } +})) +await test('a genuinely missing expected PTC name is diagnosed while an unselected one is not', () => withTempDir('preset-ptc-missing-worker', async cwd => { + // A row whose worker never mounted publishes none of its plan; the diagnostic + // must still name the genuinely missing first-round name — and must not pad the + // gap with the selectable-but-unselected result API. + const { ctx, agent } = await bootPtcRow(cwd, { preset: 'ptc-broken', binaryPath: path.join(cwd, 'missing-worker'), discovery: false }) + try { + const d = await diagnose(ctx, agent) + assert.equal(d.state, 'degraded') + assert.ok(d.issues.some(row => row.code === 'RUNTIME_UNAVAILABLE'), JSON.stringify(d.issues)) + assert.ok(d.issues.some(row => row.code === 'PUBLICATION_UNAVAILABLE' && row.message.includes('ops_inspect_local_file')), JSON.stringify(d.issues)) + assert.equal(d.issues.some(row => row.message.includes('ops_result')), false, 'the on-demand tier keeps ops_result legitimately unselected') + } finally { await shutdown(ctx) } +})) +await test('native custom preset keeps its exact visible tool snapshot across the diagnostics fix', () => withTempDir('preset-native-snapshot', async cwd => { + const { ctx, mint, names } = await boot() + try { + await ctx.plugin(integration({ preset: 'acme-snapshot', discovery: false })) + const own = await mint('acme-snapshot', cwd) + await waitFor(async () => (await names(own)).includes('ops_inspect_local_file')) + // Snapshot pinned 2026-11-19 (#988): identical before and after the diagnostics + // fix, so a PTC-side change cannot quietly re-shape the Native surface. + assert.deepEqual(await names(own), ['glob', 'grep', 'ops_glob', 'ops_grep', 'ops_inspect_local_file', 'ops_status', 'read']) + const d = await diagnose(ctx, own) + assert.equal(d.state, 'ready', JSON.stringify(d.issues)) + } finally { await shutdown(ctx) } +})) report('preset-integration') diff --git a/test/preset-registration.test.mjs b/test/preset-registration.test.mjs index be10bdff..b0d865ff 100644 --- a/test/preset-registration.test.mjs +++ b/test/preset-registration.test.mjs @@ -210,7 +210,11 @@ await test('the real Cordis fiber owns the registration, the action and the rele assert.deepEqual(registry.state.order, ['register', 'default']) await ctx.fiber.dispose() await waitFor(() => registry.state.disposed === 1, { what: 'the fiber disposal to release the preset' }) - assert.equal(registry.state.selected, 'ops-ptc', 'a disposal leaves the operator default alone') + await waitFor(() => registry.state.selected === 'standard', { what: 'the stranded default restored on unload' }) + assert.deepEqual(registry.state.updates, [ + { namespace: 'agent-preset-registry', patch: { selectedDefault: 'ops-ptc' } }, + { namespace: 'agent-preset-registry', patch: { selectedDefault: 'standard' } }, + ], 'a disposal writes the host default back exactly once') }) await test('one injected activation never registers the preset twice', async () => { @@ -256,7 +260,9 @@ await test('the action makes OPS (PTC) the default preset once', async () => { }) await test('the action reports unavailable when no preset is registered', async () => { - const preset = registerOpsPtcPreset({ inject() {} }) + // The minimal root still carries the Context capability every row owns: the + // unload lease (#928) registers through it even when nothing is registered. + const preset = registerOpsPtcPreset({ inject() {}, effect() { return () => {} } }) const answer = await preset.applyDefault() assert.equal(answer.status, 'unavailable') assert.match(answer.message, /not registered/) @@ -311,11 +317,14 @@ await test('teardown releases the preset but keeps the operator default', async assert.equal(registry.defaultId, 'ptc') }) -await test('a deployment without the preset registry is left alone', () => { +await test('a deployment without the preset registry is left alone', async () => { const { root, events } = stubRoot({ composedPreset: () => undefined }) registerOpsPtcPreset(root) assert.equal(events.inject.length, 1) - assert.deepEqual(events.effects, []) + // The row still owns its unload lease (#928), but without a registration it + // never served anything, and the lease cleanup writes nothing. + assert.deepEqual(events.effects.map(entry => entry.label), ['dsh-ops: ops-ptc preset default lease']) + await events.effects[0].cleanup() }) await test('a missing fragment only warns; it never throws into apply', async () => { diff --git a/test/progressive-loop.test.mjs b/test/progressive-loop.test.mjs index dfb45959..00f5913f 100644 --- a/test/progressive-loop.test.mjs +++ b/test/progressive-loop.test.mjs @@ -1,9 +1,12 @@ import fs from 'node:fs' +import { createHash } from 'node:crypto' import path from 'node:path' -import { assert, test, report, withTempDir } from './lib/harness.mjs' -import { requireWorker } from './lib/boundary.mjs' +import { assert, test, report, withTempDir, PACKAGE_ROOT } from './lib/harness.mjs' +import { requireWorker, WORKER } from './lib/boundary.mjs' import { setSandboxMode } from '@deepseek-ai/dsh-sandbox-policy' import { bootLoop } from './lib/agent-loop.mjs' +import { workerName } from '../lib/execution/runtime.js' +import { bootEntryGroup, enterPtc } from './lib/ptc-entry.mjs' requireWorker('progressive-loop') const names = request => request.tools.map(tool => tool.name) const text = request => JSON.stringify(request.messages) @@ -83,4 +86,86 @@ await test('real committed authority changes cancel pending activation while alw assert.equal(fs.readFileSync(path.join(root, 'note'), 'utf8'), 'unchanged') } finally { await host.close() } })) + +/** Card #990: the isolated OPS-PTC entry. Every group boots on its own throwaway + * root under this suite's DSH_HOME and is judged by test/lib/ptc-entry.mjs. */ +await test('the isolated entry truly routes one OPS-PTC request and records its environment', () => withTempDir('ptc-entry-target', async root => { + fs.writeFileSync(path.join(root, 'entry-note'), 'fixture line\n') + const group = await bootEntryGroup(root) + try { + assert.equal(group.booted, true, 'the target group must boot on the shipped ops-ptc row') + const report = await enterPtc(group) + assert.equal(report.status, 'entered', JSON.stringify(report.findings)) + assert.equal(report.target, 'ops-ptc') + assert.deepEqual(report.evidence.topLevel, ['run_code'], JSON.stringify(report.evidence)) + assert.equal(report.findings.preset.actual, 'ops-ptc') + assert.equal(report.findings.preset.mode, 'ops-ptc') + assert.equal(report.environment.agentPreset, 'ops-ptc') + assert.match(report.environment.plugin.commit, /^[0-9a-f]{40}$/) + assert.match(report.environment.host.version, /^\d+\.\d+\.\d+/) + assert.equal(report.environment.platform, process.platform + '-' + process.arch) + assert.equal(report.environment.sandbox.mode, 'workspace-write') + assert.equal(report.environment.provider, 'scripted', 'the scripted provider is marked on the report, never silent') + assert.equal(report.environment.networkModelRequests, 0) + // The shipped row's real defaults are recorded from the resolved config, not assumed. + assert.equal(report.environment.switches.batchingGuidance, true) + assert.equal(report.environment.switches.residentTools, true) + assert.equal(report.environment.switches.dynamicTools, true) + // The worker fact: the recorded digest is the digest of the binary this group used. + assert.match(report.environment.worker.sha256, /^[0-9a-f]{64}$/) + assert.equal(report.environment.worker.sha256, createHash('sha256').update(fs.readFileSync(WORKER)).digest('hex')) + const staged = report.environment.worker.staged + if (staged !== null) { + assert.equal(staged.target, report.environment.platform) + assert.equal(staged.sha256, createHash('sha256').update(fs.readFileSync( + path.join(PACKAGE_ROOT, 'runtime', report.environment.platform, workerName()))).digest('hex'), + 'the recorded staged receipt matches the staged binary') + } + // The resident SDK really answered through the worker; a fresh fixture file, nothing real. + const executed = report.evidence.executed + assert.equal(executed.isError, false, JSON.stringify(executed)) + assert.equal(executed.value.result.status, 'completed') + } finally { await group.close() } +})) + +await test('Standard routing is the deliberate counter-example: invalid with a reason, never comparable', () => withTempDir('ptc-entry-standard', async root => { + const group = await bootEntryGroup(root, { preset: 'standard' }) + try { + assert.equal(group.booted, true) + const report = await enterPtc(group) + assert.equal(report.status, 'invalid') + assert.equal(report.reason, 'preset', JSON.stringify(report.findings)) + assert.equal(report.findings.preset.actual, 'standard') + assert.equal(report.findings.preset.mode, 'standard') + assert.equal(report.findings.route.ok, false) + assert.equal(report.findings.route.topLevel.includes('run_code'), false) + // The refusal is explicit and carries no surface that could be mistaken for a result. + assert.equal('evidence' in report, false) + assert.deepEqual(Object.keys(report).sort(), ['environment', 'findings', 'reason', 'status', 'target']) + assert.equal(JSON.stringify(report).includes('Bytes'), false) + } finally { await group.close() } +})) + +await test('a PTC preset on a native executor is route-unknown, not a quiet pass', () => withTempDir('ptc-entry-route', async root => { + const group = await bootEntryGroup(root, { executor: 'native' }) + try { + assert.equal(group.booted, true) + const report = await enterPtc(group) + assert.equal(report.status, 'invalid') + assert.equal(report.reason, 'route', JSON.stringify(report.findings)) + assert.equal(report.findings.preset.ok, true, 'the preset is right; the transport is what is unknown') + assert.equal(report.findings.route.topLevel.includes('run_code'), false) + assert.equal('evidence' in report, false) + } finally { await group.close() } +})) + +await test('a group without an executable worker reports invalid before any request', () => withTempDir('ptc-entry-worker', async root => { + const group = await bootEntryGroup(root, { worker: path.join(root, 'no-such-worker') }) + assert.equal(group.booted, false) + const report = await enterPtc(group) + assert.equal(report.status, 'invalid') + assert.equal(report.reason, 'worker') + assert.match(report.findings.worker.detail, /Native worker/) + assert.equal('evidence' in report, false) +})) report('progressive-loop') diff --git a/test/prompt-budget.test.mjs b/test/prompt-budget.test.mjs index 6d9c9286..dd7ac2e2 100644 --- a/test/prompt-budget.test.mjs +++ b/test/prompt-budget.test.mjs @@ -14,7 +14,12 @@ import { assert, test, report, withTempDir } from './lib/harness.mjs' // Includes all host text/wire, not just the plugin delta. No tokenizer/cache claims. // Pre-slice win32 baseline: minimal=10516, standard=19996, cordis=19996, ptc=29639. // Windows is the largest default surface (OPS Bash); POSIX must fit the same bounds. -const budgets = { minimal: 8100, standard: 13700, cordis: 13700, ptc: 19200 } +// #753 mounts the ops_status diagnosis on built-in modes. Measured on win32 with the +// mount on: minimal=8246, standard=12670, cordis=12670, ptc=19200; with the mount +// disabled: 8045, 12469, 12469, 19026. The 174-201 byte difference is that tool's +// own schema/guidance, so minimal and ptc carry the added allowance while standard +// and cordis keep their previous, still-unused headroom. +const budgets = { minimal: 8300, standard: 13700, cordis: 13700, ptc: 19300 } await test('each default preset stays within its resident per-request byte budget', async () => { await withTempDir('prompt-budget', async root => { const { ctx } = await bootHost() diff --git a/test/prompt-ephemeral-teaching.test.mjs b/test/prompt-ephemeral-teaching.test.mjs new file mode 100644 index 00000000..5246460e --- /dev/null +++ b/test/prompt-ephemeral-teaching.test.mjs @@ -0,0 +1,109 @@ +/** + * The batching lesson rides a tool result, never the user batch (#929). + * + * The step-tail channel appends UserMessages to an accepted user batch, which + * the host logs as durable history: a lesson that arrives there reads as fresh + * user input, and the model answers the lesson instead of the task, looping on + * its own guidance. These checks pin the replacement contract: the user batch + * carries no guidance at all, the first accepted root `run_code` result carries + * the full lesson exactly once through the same-step ephemeral inbox + * (`additionalContexts`), and nothing re-enters the user batch afterwards. + */ +import { createScope, scopeOf, scopeParentOf } from '@deepseek-ai/dsh-scope' +import Subprocess from '@deepseek-ai/dsh-subprocess-local' +import Sandbox from '@deepseek-ai/dsh-sandbox-local' +import FileSystem from '@deepseek-ai/dsh-fs-local' +import NodePtc from '@deepseek-ai/dsh-ptc-runtime-node' +import { createUserMessage } from '@deepseek-ai/dsh-llm' +import { apply } from '../lib/index.js' +import { publicToolName } from '../lib/policy.js' +import { bootHost, shutdown } from './lib/host.mjs' +import { prepareStep, executeCall, selectExecutor } from './lib/executor.mjs' +import { assert, test, report, waitFor, withTempDir } from './lib/harness.mjs' +import { WORKER, requireWorker } from './lib/boundary.mjs' + +requireWorker('prompt-ephemeral-teaching') +let sequence = 0 + +const names = (ctx, agent) => ctx.get('tools').definitions(agent, agent.prepared?.toolSet).map(tool => tool.name).sort() + +async function scoped(parent, key, parentKey) { + let result + await parent.plugin({ inject: ['tools'], apply: inner => { + result = createScope(inner, key, parentKey ? { parent: parentKey } : undefined).ctx + } }) + return result +} + +/** The same merged-row fixture resident-switch uses, restricted to the ops-ptc row. */ +async function setup(cwd) { + const { ctx } = await bootHost({ withWorkingDirectory: true }) + await ctx.plugin(Subprocess); await ctx.plugin(Sandbox); await ctx.plugin(FileSystem); await ctx.plugin(NodePtc) + ctx.provide('sandboxPolicy', { resolve: ({ session } = {}) => ({ mode: session?.mode ?? 'danger-full-access', workspaceRoot: cwd, sessionId: session?.id }) }) + const presets = new Map(), agents = [] + ctx.provide('agents', { list: () => [...agents] }) + ctx.provide('agentPresets', { composedPreset: inner => presets.get(scopeParentOf(scopeOf(inner))) }) + const config = { global: {}, discovery: { binaryPath: WORKER, required: true } } + for (const id of ['minimal', 'standard', 'ptc', 'ops-ptc', 'cordis']) config[id] = { enabled: id === 'ops-ptc', + binaryPath: WORKER, required: true, ...(id === 'ops-ptc' ? { residentTools: true, dynamicTools: true, batchingGuidance: true, alwaysLoaded: ['stat'], allowedTools: ['stat'] } : {}) } + await ctx.plugin({ name: 'prompt-ephemeral-teaching-fixture', apply: inner => apply(inner, config) }) + const id = 'ephemeral-' + ++sequence, preset = await scoped(ctx, { preset: id }) + presets.set(scopeOf(preset), 'ops-ptc') + const agent = { id, session: { id, mode: 'danger-full-access', header: { cwd }, append() {}, read() { return [] } } } + agent.ctx = await scoped(preset, agent, scopeOf(preset)) + await agent.ctx.plugin({ inject: ['tools', 'systemPrompt'], apply: inner => { selectExecutor(inner, 'ptc') } }) + agents.push(agent) + await ctx.serial('agent/created', { agent }) + agent.prepared = await prepareStep(ctx, agent) + return { ctx, agent } +} + +/** Drive the real agent pre-step waterfall once, exactly as the host loop does. */ +const drivePreStep = async (agent, turn) => { + const accepted = createUserMessage({ source: { kind: 'user' }, content: [{ type: 'text', text: 'Start the task.' }] }) + return agent.ctx.waterfall('agent/pre-step', { + agent, messages: [accepted], turn, step: 1, signal: new AbortController().signal, + modelSelection: { provider: 'fixture', model: 'capture' }, + }, async () => ({ kind: 'enter', messages: [accepted] })) +} + +/** The one-time full lesson the first accepted root result carries. */ +const guidedContexts = result => (result.additionalContexts ?? []).filter(context => context.source?.kind === 'dsh-ops:step-tail' && context.source?.contributor === 'batching-guidance:lesson') + +/** A compliant root program: plan line, two dispatches under Promise.all, no bypass. */ +const compliantProgram = '// plan: two checks\nconst [a, b] = await Promise.all([tools.ops_stat({ path: "x" }), tools.ops_stat({ path: "y" })])\nreturn [a.status, b.status]' + +const rootCall = (ctx, agent, callId) => executeCall(ctx, { agent, toolSet: agent.prepared.toolSet, name: 'run_code', + arguments: { description: 'Compliant fixture', code: compliantProgram }, callId, signal: new AbortController().signal }) + +await test('the user batch carries no batching guidance, before or after the lesson', () => withTempDir('ephemeral-pre', async cwd => { + const { ctx, agent } = await setup(cwd) + try { + await waitFor(() => names(ctx, agent).includes(publicToolName('stat'))) + const first = await drivePreStep(agent, 1) + const before = first.messages.filter(message => message.source?.kind === 'dsh-ops:step-tail') + assert.equal(before.length, 0, 'a user-role lesson reads as fresh user input and the model answers it (#929)') + const run = await rootCall(ctx, agent, 'ephemeral-teach-1') + assert.equal(run.isError, false, JSON.stringify(run)) + const after = (await drivePreStep(agent, 2)).messages.filter(message => message.source?.kind === 'dsh-ops:step-tail') + assert.equal(after.length, 0, 'nothing re-enters the user batch once the lesson is delivered') + } finally { await shutdown(ctx) } +})) + +await test('the first accepted root run_code result carries the full lesson exactly once', () => withTempDir('ephemeral-once', async cwd => { + const { ctx, agent } = await setup(cwd) + try { + await waitFor(() => names(ctx, agent).includes(publicToolName('stat'))) + const first = await rootCall(ctx, agent, 'ephemeral-teach-2') + assert.equal(first.isError, false, JSON.stringify(first)) + const taught = guidedContexts(first) + assert.equal(taught.length, 1, JSON.stringify(first.additionalContexts)) + assert.match(JSON.stringify(taught), /Batch tool calls/) + assert.match(JSON.stringify(taught), /not a user instruction or authorization/) + const second = await rootCall(ctx, agent, 'ephemeral-teach-3') + assert.equal(second.isError, false, JSON.stringify(second)) + assert.equal(guidedContexts(second).length, 0, 'the lesson is spent after its one delivery') + } finally { await shutdown(ctx) } +})) + +report('prompt-ephemeral-teaching') diff --git a/test/prompt-injection.test.mjs b/test/prompt-injection.test.mjs index 71d1a75d..11294be2 100644 --- a/test/prompt-injection.test.mjs +++ b/test/prompt-injection.test.mjs @@ -1,6 +1,7 @@ import { assert, test, report } from './lib/harness.mjs' import { renderPrompt } from '@deepseek-ai/dsh-system-prompt' import { Session } from '@deepseek-ai/dsh-session' +import { createUserMessage } from '@deepseek-ai/dsh-llm' import { registerSystemSection, registerStepTailProvider, PromptProviderError } from '../lib/prompt-injection/index.js' import { promptHost, tailMessages } from './lib/prompt-injection-host.mjs' const identity = 'You are an AI agent powered by DeepSeek Harness.\n\n' @@ -211,4 +212,49 @@ await test('agent teardown releases registrations owned by a longer-lived plugin assert.deepEqual(h.errors, []) } finally { await h.close() } }) + + +// ---- provider dedupe and compaction re-arming (#998) ---- +/** Shadow every visible node except the system prompt with one compaction summary. */ +const compactHistory = (agent, text) => { + const nodes = [...agent.session.surface.nodes] + agent.session.append('user/message', + createUserMessage({ source: { kind: 'user' }, content: [{ type: 'text', text }] }), + { surfaceOp: { op: 'replace', startSeq: nodes[1], endSeq: nodes[nodes.length - 1] }, sourceEventSeqs: nodes.slice(1) }) +} + +await test('dedupe providers are skipped while their same-revision message is visible and re-armed when a compaction removes it', async () => { + const h = await promptHost() + try { + const a = await h.agent() + const seen = [] + let calls = 0 + const stop = registerStepTailProvider(h.owner(), a, { id: 'lesson', revision: '1', dedupe: 'visible', + provide: context => { ++calls; seen.push(context.visible); return 'FULL' } }) + await h.run(a) + assert.equal(calls, 1) + assert.deepEqual(seen[0], { any: false, current: false }, 'nothing is visible before the first commit') + const id = tailMessages(h.requests[0])[0].id + await h.run(a) + assert.equal(calls, 1, 'a visible same-revision contribution is a replay, not a new addition') + assert.deepEqual(tailMessages(h.requests[1]).map(m => m.id), [id]) + compactHistory(a, 'Compacted summary.') + assert.equal(a.session.deriveMessages().some(m => m.source?.kind === 'dsh-ops:step-tail'), false) + await h.run(a) + assert.equal(calls, 2, 'the provider is re-armed once the compaction removed its message') + assert.deepEqual(seen[1], { any: false, current: false }) + assert.equal(tailMessages(h.requests[2]).filter(m => m.id !== id).length, 1) + await h.run(a) + assert.equal(calls, 2, 'the replacement message is visible again, so the provider falls silent') + stop() + registerStepTailProvider(h.owner(), a, { id: 'lesson', revision: '2', dedupe: 'visible', + provide: context => { seen.push(context.visible); return 'FULL2' } }) + await h.run(a) + assert.equal(tailMessages(h.requests[4]).filter(m => m.source.revision === '2').length, 1, + 'an older revision stays visible evidence, so the replacement revision is delivered') + assert.deepEqual(seen[2], { any: true, current: false }) + assert.deepEqual(h.errors, []) + } finally { await h.close() } +}) + report('prompt-injection') diff --git a/test/ptc-config-truth.test.mjs b/test/ptc-config-truth.test.mjs new file mode 100644 index 00000000..159acc89 --- /dev/null +++ b/test/ptc-config-truth.test.mjs @@ -0,0 +1,268 @@ +/** The D1 truth table: five legacy config inputs and their first-round OPS names. + * + * This suite locks current publication behaviour, letter by letter, for the five + * inputs the OPS-PTC two-tier contract (#979/#985) must stay compatible with: + * defaults, global dynamicTools:false, row dynamicTools:false, that row plus + * alwaysLoaded:[stat], and residentTools:false. Every row asserts the exact + * first-round ops_* set, so reinterpreting one input as another tier's + * semantics cannot pass silently. The table itself lives in + * docs/design-decisions-20261119.md (中文:docs/设计决策-20261119.md). + */ +import { createScope, scopeOf, scopeParentOf } from '@deepseek-ai/dsh-scope' +import Subprocess from '@deepseek-ai/dsh-subprocess-local' +import Sandbox from '@deepseek-ai/dsh-sandbox-local' +import FileSystem from '@deepseek-ai/dsh-fs-local' +import NodePtc from '@deepseek-ai/dsh-ptc-runtime-node' +import { apply } from '../lib/index.js' +import { resolveMergedConfig, effectiveMode, dynamicForMode } from '../lib/settings/merged.js' +import { resolveModeConfig } from '../lib/modes/config.js' +import { PTC_LOADABLE_TOOLS, PTC_RESIDENT_TOOLS, parsePtcConfig } from '../lib/ptc/config.js' +import { requestedApis } from '../lib/projections/ptc/loader.js' +import { integration, diagnose } from '../lib/integration/index.js' +import { BASH_TOOL, publicToolName } from '../lib/policy.js' +import { bootHost, shutdown } from './lib/host.mjs' +import { prepareStep, executeCall, selectExecutor } from './lib/executor.mjs' +import { assert, test, report, wait, waitFor, withTempDir } from './lib/harness.mjs' +import { WORKER, requireWorker } from './lib/boundary.mjs' + +/** The truth table's right-hand columns, as this suite locks them. */ +const ROW_DEFAULT = ['ops_inspect_local_file', 'ops_load_sdk'] +const ROW_GLOBAL_OFF = ['ops_inspect_local_file'] +const ROW_ROW_OFF = ['ops_inspect_local_file'] +const ROW_ROW_OFF_ALWAYS = ['ops_inspect_local_file', 'ops_stat'] +const ROW_RESIDENT_OFF = ['ops_load_sdk'] + +await test('resolver: the five legacy inputs land where the truth table claims', () => { + const base = resolveMergedConfig() + assert.equal(base['ops-ptc'].enabled, true) + assert.equal(base['ops-ptc'].config.dynamicTools, true) + assert.equal(base['ops-ptc'].config.residentTools, true) + assert.deepEqual(base['ops-ptc'].config.allowedTools, PTC_LOADABLE_TOOLS) + assert.deepEqual(base['ops-ptc'].config.alwaysLoaded, []) + // Row 2: the global switch gates every mode and preserves saved intent. + const global = resolveMergedConfig({ global: { dynamicTools: false } }) + assert.equal(global.global.dynamicTools, false) + for (const mode of ['minimal', 'standard', 'ptc', 'ops-ptc', 'cordis']) { + assert.equal(dynamicForMode(global[mode], global.global), false, mode) + assert.equal(effectiveMode(global[mode], global.global).dynamicTools, false, mode) + } + assert.equal('dynamicTools' in (global['ops-ptc'].raw ?? {}), false, 'an absent row key still means on') + // Row 3: a row-level off gates that row only. + const row = resolveMergedConfig({ 'ops-ptc': { dynamicTools: false } }) + assert.equal(effectiveMode(row['ops-ptc'], row.global).dynamicTools, false) + assert.equal(dynamicForMode(row['ops-ptc'], row.global), false) + assert.equal(dynamicForMode(row.standard, row.global), true, 'row-level off leaves other rows on') + // Row 4: alwaysLoaded rides the resident set while the loader is off. + const always = resolveMergedConfig({ 'ops-ptc': { dynamicTools: false, alwaysLoaded: ['stat'] } }) + assert.deepEqual(always['ops-ptc'].config.alwaysLoaded, ['stat']) + assert.deepEqual(always['ops-ptc'].config.allowedTools, PTC_LOADABLE_TOOLS, 'the cap is untouched') + assert.equal(always['ops-ptc'].config.dynamicTools, false) + // Row 5: residentTools is row-local; there is no global resident switch. + const resident = resolveMergedConfig({ 'ops-ptc': { residentTools: false } }) + assert.equal(resident['ops-ptc'].config.residentTools, false) + assert.equal(resident.global.dynamicTools, true) +}) + +await test('the default loadable pool is exactly the sixteen non-resident raw APIs', () => { + assert.deepEqual(PTC_LOADABLE_TOOLS, ['grep', 'glob', 'replace', 'write_file', 'edit_file', 'diff', 'stat', 'copy', + 'move', 'remove', 'read_files', 'read_table', 'read_docx', 'read_pdf', 'batch', 'result']) + assert.deepEqual(PTC_RESIDENT_TOOLS, ['inspect_local_file']) +}) + +await test('today alwaysLoaded escapes the allowedTools cap with no diagnostic (decision item)', () => { + // Current behaviour, locked so a future diagnostic cannot land silently: the + // startup list is validated against the loadable pool only, never against + // allowedTools, and a residentTools:false row keeps the list in config while + // publication drops it. Whether to diagnose is a maintainer decision (#985). + const escaped = resolveModeConfig({ allowedTools: ['grep'], alwaysLoaded: ['stat'] }, 'ops-ptc') + assert.deepEqual(escaped.allowedTools, ['grep']) + assert.deepEqual(escaped.alwaysLoaded, ['stat'], 'stat stays resident though the cap omits it') + const suppressed = resolveModeConfig({ residentTools: false, alwaysLoaded: ['stat'] }, 'ops-ptc') + assert.equal(suppressed.residentTools, false) + assert.deepEqual(suppressed.alwaysLoaded, ['stat'], 'kept in config; publication drops the resident set') +}) + +requireWorker('ptc-config-truth') +let sequence = 0 +// The integration diagnostics row (ops_status, #753) and the Windows Bash tool +// are published beside the PTC tier and have their own suites; this enumeration +// is the PTC disclosure surface. +const names = (ctx, agent) => ctx.get('tools').definitions(agent, agent.prepared?.toolSet).map(tool => tool.name) + .filter(name => name !== 'ops_status' && name !== BASH_TOOL).sort() +const opsNames = (ctx, agent) => names(ctx, agent).filter(name => name.startsWith('ops_')) +async function step(ctx, agent) { + ctx.emit('session/event', agent.session, { type: 'step/end', turn: 1, step: ++sequence }) + agent.prepared?.dispose(); agent.prepared = await prepareStep(ctx, agent) +} +async function scoped(parent, key, parentKey) { + let result + await parent.plugin({ inject: ['tools'], apply(inner) { + result = createScope(inner, key, parentKey ? { parent: parentKey } : undefined).ctx + } }) + return result +} +async function setup(cwd, mode, switches, global = {}) { + const { ctx } = await bootHost({ withWorkingDirectory: true }) + await ctx.plugin(Subprocess); await ctx.plugin(Sandbox); await ctx.plugin(FileSystem); await ctx.plugin(NodePtc) + ctx.provide('sandboxPolicy', { resolve: ({ session } = {}) => ({ mode: session?.mode ?? 'danger-full-access', workspaceRoot: cwd, sessionId: session?.id }) }) + const presets = new Map(), agents = [] + ctx.provide('agents', { list: () => [...agents] }) + ctx.provide('agentPresets', { composedPreset: inner => presets.get(scopeParentOf(scopeOf(inner))) }) + const config = { global, discovery: { binaryPath: WORKER, required: true } } + for (const id of ['minimal', 'standard', 'ptc', 'ops-ptc', 'cordis']) config[id] = { enabled: id === mode, + binaryPath: WORKER, required: true, ...(id === mode ? switches : {}) } + await ctx.plugin({ name: 'ptc-config-truth-fixture', apply: inner => apply(inner, config) }) + const id = 'truth-' + ++sequence, preset = await scoped(ctx, { preset: id }) + presets.set(scopeOf(preset), mode) + for (const name of ['read', 'glob', 'grep']) preset.tools.register({ name, description: name, + parameters: { type: 'object', properties: {} }, output: { schema: { type: 'object' }, render: () => [] }, execute: async () => ({}) }) + const agent = { id, session: { id, mode: 'danger-full-access', header: { cwd }, append() {}, read() { return [] } } } + agent.ctx = await scoped(preset, agent, scopeOf(preset)) + if (mode === 'ptc' || mode === 'ops-ptc') await agent.ctx.plugin({ inject: ['tools', 'systemPrompt'], apply: inner => { selectExecutor(inner, 'ptc') } }) + agents.push(agent) + await ctx.serial('agent/created', { agent }) + await step(ctx, agent) + return { ctx, agent } +} + +/** Boot one input, wait for its first-round anchor, return the settled OPS set. */ +async function firstRoundOps(mode, switches, global, anchor) { + return withTempDir('truth-' + ++sequence, async cwd => { + const { ctx, agent } = await setup(cwd, mode, switches, global) + try { + await waitFor(() => opsNames(ctx, agent).includes(anchor), { what: anchor + ' published' }) + await wait(100) + return opsNames(ctx, agent) + } finally { await shutdown(ctx) } + }) +} + +const rowSets = {} + +await test('row 1 (defaults): first round is inspect + loader; grep loads for the next step', () => withTempDir('truth-row1', async cwd => { + const { ctx, agent } = await setup(cwd, 'ops-ptc', {}) + try { + await waitFor(() => opsNames(ctx, agent).includes('ops_inspect_local_file'), { what: 'ops_inspect_local_file published' }) + await wait(100) + rowSets.r1 = opsNames(ctx, agent) + assert.deepEqual(rowSets.r1, ROW_DEFAULT) + const run = await executeCall(ctx, { agent, toolSet: agent.prepared.toolSet, name: 'run_code', + arguments: { description: 'Load grep', code: "const picked = await tools.ops_load_sdk({ tools: ['grep'] })\nreturn { picked }" }, + callId: 'truth-r1-' + ++sequence, signal: new AbortController().signal }) + assert.equal(run.isError, false, JSON.stringify(run)) + assert.equal(run.value.result.picked.status, 'completed', JSON.stringify(run)) + assert.equal(opsNames(ctx, agent).includes('ops_grep'), false, 'loadable-not-loaded: activation is next-step only') + await step(ctx, agent) + assert.ok(opsNames(ctx, agent).includes('ops_grep'), 'grep is callable after the next step') + } finally { await shutdown(ctx) } +})) + +await test('row 2 (global dynamicTools:false): first round is inspect only, no loader', async () => { + rowSets.r2 = await firstRoundOps('ops-ptc', {}, { dynamicTools: false }, 'ops_inspect_local_file') + assert.deepEqual(rowSets.r2, ROW_GLOBAL_OFF) + assert.equal(rowSets.r2.includes('ops_load_sdk'), false) +}) + +await test('row 3 (row dynamicTools:false): the same PTC set as the global switch, row-scoped', async () => { + rowSets.r3 = await firstRoundOps('ops-ptc', { dynamicTools: false }, {}, 'ops_inspect_local_file') + assert.deepEqual(rowSets.r3, ROW_ROW_OFF) + assert.equal(rowSets.r3.includes('ops_load_sdk'), false) + assert.deepEqual(rowSets.r3, rowSets.r2, 'global and row off coincide at the PTC surface; only their scope differs') +}) + +await test('row 4 (row off + alwaysLoaded:[stat]): first round is inspect + stat, no loader', async () => { + rowSets.r4 = await firstRoundOps('ops-ptc', { dynamicTools: false, alwaysLoaded: ['stat'] }, {}, 'ops_inspect_local_file') + assert.deepEqual(rowSets.r4, ROW_ROW_OFF_ALWAYS) + assert.equal(rowSets.r4.includes('ops_load_sdk'), false) +}) + +await test('row 5 (residentTools:false): first round is the loader only; inspect is unloadable', () => withTempDir('truth-row5', async cwd => { + const { ctx, agent } = await setup(cwd, 'ops-ptc', { residentTools: false }) + try { + await waitFor(() => opsNames(ctx, agent).includes('ops_load_sdk'), { what: 'ops_load_sdk published' }) + await wait(100) + rowSets.r5 = opsNames(ctx, agent) + assert.deepEqual(rowSets.r5, ROW_RESIDENT_OFF) + assert.equal(rowSets.r5.includes('ops_inspect_local_file'), false) + const run = await executeCall(ctx, { agent, toolSet: agent.prepared.toolSet, name: 'run_code', + arguments: { description: 'Probe inspect', code: "const blocked = await tools.ops_load_sdk({ tools: ['inspect_local_file'] })\nreturn { blocked }" }, + callId: 'truth-r5-' + ++sequence, signal: new AbortController().signal }) + assert.equal(run.isError, false, JSON.stringify(run)) + assert.equal(run.value.result.blocked.error.code, 'INVALID_ARGUMENT', 'the resident API is not loadable while residentTools is off') + } finally { await shutdown(ctx) } +})) + +await test('the five recorded sets differ exactly where their one-line inputs differ', () => { + assert.notDeepEqual(rowSets.r1, rowSets.r2, 'the global switch removes the loader') + assert.notDeepEqual(rowSets.r1, rowSets.r3, 'the row switch removes the loader') + assert.notDeepEqual(rowSets.r3, rowSets.r4, 'alwaysLoaded adds stat to the resident set') + assert.notDeepEqual(rowSets.r1, rowSets.r5, 'residentTools:false removes the resident surface') + assert.notDeepEqual(rowSets.r4, rowSets.r5) +}) + +await test('legacy off with allowedTools=[grep,stat] is not a full-tier set', async () => { + // The counterexample #985 requires: old-off and a full selection are different + // inputs. Reading dynamicTools:false as "publish everything allowed" would + // put ops_grep and ops_stat in this first round; today it publishes neither. + const offSet = await firstRoundOps('ops-ptc', { dynamicTools: false, allowedTools: ['grep', 'stat'] }, {}, 'ops_inspect_local_file') + assert.deepEqual(offSet, ROW_ROW_OFF) + assert.equal(offSet.includes('ops_grep'), false) + assert.equal(offSet.includes('ops_stat'), false) + assert.equal(offSet.includes('ops_load_sdk'), false) + const demandSet = await firstRoundOps('ops-ptc', { allowedTools: ['grep', 'stat'] }, {}, 'ops_inspect_local_file') + assert.deepEqual(demandSet, ROW_DEFAULT, 'the same cap on-demand still starts from inspect + loader') + assert.notDeepEqual(offSet, demandSet, 'one flipped line, different first-round set') + assert.equal(demandSet.includes('ops_grep'), false, 'on-demand also keeps grep off the first round') +}) + +await test('the ptc row publishes the same first-round set as ops-ptc', async () => { + const set = await firstRoundOps('ptc', {}, {}, 'ops_inspect_local_file') + assert.deepEqual(set, ROW_DEFAULT) +}) + +await test('the loader batches 1-5 names per call; the resident startup list has no count cap', () => { + // Current behaviour, locked so a silent cap change cannot pass: one load_sdk call + // accepts 1-5 exact raw names, so the sixteen-name pool needs at least four calls, + // while alwaysLoaded publishes any validated list with no length limit. + const empty = requestedApis({ tools: [] }, PTC_LOADABLE_TOOLS) + assert.equal(typeof empty, 'string', 'an empty request is rejected') + assert.ok(String(empty).includes('1-5'), 'the rejection names the 1-5 contract') + assert.equal(typeof requestedApis({ tools: PTC_LOADABLE_TOOLS.slice(0, 6) }, PTC_LOADABLE_TOOLS), 'string', 'a six-name request is rejected') + assert.deepEqual(requestedApis({ tools: ['grep', 'grep', 'stat'] }, PTC_LOADABLE_TOOLS), ['grep', 'stat'], 'duplicates collapse; a legal pair loads') + assert.equal(Math.ceil(PTC_LOADABLE_TOOLS.length / 5), 4, 'sixteen names need at least four loader calls') + const full = parsePtcConfig({ allowedTools: PTC_LOADABLE_TOOLS, alwaysLoaded: PTC_LOADABLE_TOOLS }) + assert.deepEqual(full.alwaysLoaded, PTC_LOADABLE_TOOLS, 'no count cap on the startup list') + assert.deepEqual(full.allowedTools, PTC_LOADABLE_TOOLS) +}) + +await test('a legal custom ptc-profile integration diagnoses ready after the D4 plan-derived expectations (decision item 4)', () => withTempDir('truth-diag-' + ++sequence, async cwd => { + // D4 (#988) flipped this pin: diagnostics used to list ops_result among first-round + // expected names, but result is selectable, not resident, so a legal custom PTC + // integration read degraded while its only gap was an API the loader had not + // selected yet. The expected set now comes from the same disclosure plan the + // publication uses, so this row reports ready and no PUBLICATION_UNAVAILABLE. + const { ctx } = await bootHost({ withWorkingDirectory: true }) + try { + await ctx.plugin(Subprocess); await ctx.plugin(Sandbox); await ctx.plugin(FileSystem); await ctx.plugin(NodePtc) + ctx.provide('sandboxPolicy', { resolve: ({ session } = {}) => ({ mode: session?.mode ?? 'danger-full-access', workspaceRoot: cwd, sessionId: session?.id }) }) + ctx.provide('agentPresets', { composedPreset: () => undefined }) + const agents = [] + ctx.provide('agents', { list: () => [...agents] }) + await ctx.plugin(integration({ preset: 'truth-diag-ptc', profile: 'ptc', binaryPath: WORKER, required: true })) + const id = 'truth-diag-' + ++sequence + const agent = { id, session: { id, mode: 'danger-full-access', header: { cwd, agentPreset: 'truth-diag-ptc' } } } + await ctx.plugin({ inject: ['tools'], apply(inner) { agent.ctx = createScope(inner, agent).ctx } }) + await agent.ctx.plugin({ inject: ['tools', 'systemPrompt'], apply: inner => { selectExecutor(inner, 'ptc') } }) + agents.push(agent) + await ctx.serial('agent/created', { agent }) + agent.prepared = await prepareStep(ctx, agent) + await waitFor(() => opsNames(ctx, agent).includes('ops_inspect_local_file'), { what: 'ops_inspect_local_file published' }) + await wait(100) + const d = await diagnose(ctx, agent) + assert.equal(d.state, 'ready', JSON.stringify(d.issues)) + assert.equal(d.issues.some(row => row.code === 'PUBLICATION_UNAVAILABLE'), false, + 'the selectable-but-not-resident result API is not a publication gap') + } finally { await shutdown(ctx) } +})) + +report('ptc-config-truth') diff --git a/test/ptc-cost-accounts.test.mjs b/test/ptc-cost-accounts.test.mjs new file mode 100644 index 00000000..8abadc8c --- /dev/null +++ b/test/ptc-cost-accounts.test.mjs @@ -0,0 +1,265 @@ +/** + * The four-account OPS-PTC cost ledger (#991). + * + * V1 (#990) proved the measured group truly enters OPS-PTC. This suite answers + * what V2 asks on top of that entry: how many raw UTF-8 bytes each account + * costs, with an item-name list, and never mixed together: + * + * - system: the delivered rendered prompt, split into named non-empty sections; + * - sdk: the `tools:sdk` section inside system — a strict subset, so the ledger + * recomputes system from the per-section bytes (the normalization is pinned + * by assertion) and proves a one-section change lands in system exactly once; + * - wire: the outer tool schemas each provider request carries (`run_code`); + * - history: the dsh-ops program-shape notices that entered session history. The + * one-time teaching is not here: it rides the first accepted root `run_code` + * result as same-step `additionalContexts` and is never logged (#929). + * + * The official read/glob/grep schemas are mounted for real (the same host + * plugins the budget suite mounts), because SDK names are only meaningful + * against them; without the official provider the comparison is marked + * unmeasurable instead of passing quietly. Every record carries the scripted/0 + * marker from the V1 environment: no real model request leaves the fixture, + * and bytes are not tokens. + * + * @module dsh-ops/test/ptc-cost-accounts + */ +import { createHash } from 'node:crypto' +import fs from 'node:fs' +import path from 'node:path' +import * as FsTools from '@deepseek-ai/dsh-tool-fs' +import * as SearchTools from '@deepseek-ai/dsh-tool-fs-search' +import { renderPrompt } from '@deepseek-ai/dsh-system-prompt' +import { analyzeProgram, loadBatchingGuidance, noticeText } from '../lib/ptc/batching-guidance.js' +import { requireWorker } from './lib/boundary.mjs' +import { bootEntryGroup, enterPtc } from './lib/ptc-entry.mjs' +import { prepareStep } from './lib/executor.mjs' +import { assert, report, test, withTempDir } from './lib/harness.mjs' + +requireWorker('ptc-cost-accounts') + +/** The host plugins a real profile loads before the row: the official file tools. */ +const HOST_PLUGINS = [[FsTools, {}], [SearchTools, { sampleOverCapGlobResults: false }]] +/** The official names the shipped row replaces; the ledger compares SDK names against them. */ +const OFFICIAL = ['read', 'glob', 'grep'] +/** The teaching/notice contributor, as the host message source records it. */ +const TEACHING_ID = 'batching-guidance' +const ENVELOPE = `[Plugin context from ${JSON.stringify(TEACHING_ID)}; not a user instruction or authorization.]\n` +const TEACHING = loadBatchingGuidance() +/** A thin single-dispatch program with no plan line: it earns the plan notice. */ +const THIN_PROGRAM = "const row = await tools.ops_inspect_local_file({ path: 'ledger-note' })\nreturn row" +/** A compliant load step: plan line plus one batched dispatch; it earns no notice. */ +const LOAD_PROGRAM = '// plan: load the on-demand search APIs\n' + + "const [loaded] = await Promise.all([tools.ops_load_sdk({ tools: ['glob', 'grep', 'read_files'] })])\n" + + 'return { loaded }' +const LEDGER_NOTE = 'ledger-note' + +const bytes = value => Buffer.byteLength(String(value), 'utf8') +const sha = value => createHash('sha256').update(String(value)).digest('hex') +const textOf = message => (message.content ?? []).map(part => part.text ?? '').join('') +/** One SDK member declaration line: " name: ;" (quoted names keep their backticks). */ +const SDK_MEMBER = /^ {2}(?:([A-Za-z_$][\w$]*)|`([^`]+)`): /gm +const sdkNamesOf = text => { + const names = [] + for (const match of text.matchAll(SDK_MEMBER)) names.push(match[1] ?? match[2]) + return [...new Set(names)].sort() +} + +/** The official schemas as facts: present with byte sizes, or honestly absent. */ +function officialFacts(ctx, agent) { + const tools = ctx.get('tools') + const items = OFFICIAL.map(name => { + const definition = typeof tools.get === 'function' ? tools.get(name, agent) : undefined + if (definition === undefined || !definition.parameters) return { name, present: false } + const schema = JSON.stringify({ name: definition.name, description: definition.description, parameters: definition.parameters }) + return { name, present: true, schemaBytes: bytes(schema) } + }) + const missing = items.filter(item => !item.present).map(item => item.name) + return { + status: missing.length === 0 ? 'mounted' : 'unmeasurable', + ...(missing.length === 0 ? {} : { reason: 'official provider absent: ' + missing.join(', ') + '; the SDK-name comparison cannot run' }), + items, + } +} + +/** The dsh-ops guidance messages in one delivered request, classified and byte-counted. */ +function historyItemsOf(messages) { + const items = [] + for (const message of messages) { + if (message.source?.kind !== 'dsh-ops:step-tail' || message.source.contributor !== TEACHING_ID) continue + const text = textOf(message) + const body = text.startsWith(ENVELOPE) ? text.slice(ENVELOPE.length) : text + const kind = body === TEACHING ? 'teaching' : body === noticeText('plan', analyzeProgram(THIN_PROGRAM)) ? 'notice:plan' : 'unknown' + items.push({ kind, bytes: bytes(text), sha: sha(text) }) + } + return items +} + +/** + * Run the ledger scenario in one isolated group: prove the routing, then walk + * teaching -> notice -> load -> repeat and snapshot every account at each step. + * @param {string} root - the group's isolated root, under the suite home. + * @param {object} [options] - scenario inputs. + * @param {Array} [options.hostPlugins] - host plugins mounted before the row. + * @returns {Promise} the ledger, or an invalid-entry report. + */ +async function runScenario(root, { hostPlugins = [] } = {}) { + const group = await bootEntryGroup(root, { hostPlugins }) + try { + const routing = await enterPtc(group) + if (routing.status !== 'entered') return { invalid: routing } + const ctx = group.host.ctx + const agent = await group.host.agent() + fs.writeFileSync(path.join(root, LEDGER_NOTE), 'four-account fixture line\n') + const official = officialFacts(ctx, agent) + const snapshot = async stage => { + const step = await prepareStep(ctx, agent) + try { + const sections = step.assembly.sections.filter(section => section.text.length > 0) + const sdk = sections.filter(section => section.name === 'tools:sdk') + const rendered = renderPrompt(step.assembly) + const wire = JSON.stringify(step.assembly.tools) + const sdkText = sdk.length === 1 ? sdk[0].text : '' + return { + stage, + system: { bytes: bytes(rendered), sha: sha(rendered), + items: sections.map(section => ({ name: section.name, bytes: bytes(section.text), sha: sha(section.text) })) }, + sdk: { sections: sdk.length, bytes: bytes(sdkText), sha: sha(sdkText), names: sdkNamesOf(sdkText) }, + wire: { bytes: bytes(wire), sha: sha(wire), names: step.assembly.tools.map(tool => tool.name).sort() }, + } + } finally { step.dispose() } + } + const requests = [] + const stepWith = (label, code) => request => { + requests.push({ + label, + wireNames: request.tools.map(tool => tool.name).sort(), + wireSha: sha(JSON.stringify(request.tools)), + systemSha: sha(textOf(request.messages.find(message => message.role === 'system'))), + history: historyItemsOf(request.messages), + delivered: { messages: request.messages.length, bytes: bytes(request.messages.map(textOf).join('\n')) }, + }) + return code === undefined ? [] : [{ name: 'run_code', args: { description: 'Four-account ledger scenario step', code } }] + } + const snapshots = {} + snapshots.baseline = await snapshot('baseline') + await group.host.run(agent, [stepWith('first', THIN_PROGRAM), stepWith('after-notice')]) + snapshots.steady = await snapshot('steady') + await group.host.run(agent, [stepWith('before-load', LOAD_PROGRAM), stepWith('after-load')]) + snapshots.loaded = await snapshot('loaded') + await group.host.run(agent, [stepWith('repeat')]) + snapshots.repeat = await snapshot('repeat') + return { environment: group.environment, official, snapshots, requests } + } finally { await group.close() } +} + +/** The byte-and-name facts that must reproduce across runs; per-group facts are excluded. */ +const comparableOf = ledger => ({ + snapshots: Object.fromEntries(Object.entries(ledger.snapshots).map(([stage, snapshot]) => [stage, { + system: { bytes: snapshot.system.bytes, sha: snapshot.system.sha, items: snapshot.system.items.map(item => [item.name, item.bytes, item.sha]) }, + sdk: { bytes: snapshot.sdk.bytes, sha: snapshot.sdk.sha, names: snapshot.sdk.names }, + wire: { bytes: snapshot.wire.bytes, sha: snapshot.wire.sha, names: snapshot.wire.names }, + }])), + requests: ledger.requests.map(request => ({ label: request.label, wireSha: request.wireSha, systemSha: request.systemSha, + history: request.history.map(item => [item.kind, item.bytes, item.sha]) })), +}) + +/** Print the four accounts of one ledger as one JSON line per snapshot. */ +function printLedger(ledger) { + // C: every printed record carries the scripted/0 marker, never silently. + const marker = { provider: ledger.environment.provider, networkModelRequests: ledger.environment.networkModelRequests } + for (const snapshot of Object.values(ledger.snapshots)) { + console.log(JSON.stringify({ fourAccounts: { ...marker, stage: snapshot.stage, systemBytes: snapshot.system.bytes, + sdkBytes: snapshot.sdk.bytes, wireBytes: snapshot.wire.bytes, sdkNames: snapshot.sdk.names } })) + } + const after = ledger.requests.find(request => request.label === 'after-load') + console.log(JSON.stringify({ fourAccounts: { ...marker, history: after.history, officialSchema: ledger.official } })) +} + +await test('the four accounts of one OPS-PTC group: system, its SDK subset, the wire, and the notice history', async () => { + const first = await withTempDir('ptc-cost-accounts-a', root => runScenario(root, { hostPlugins: HOST_PLUGINS })) + assert.equal(first.invalid, undefined, 'the routing gate failed: ' + JSON.stringify(first.invalid ?? {})) + const { official, snapshots, requests } = first + // C: every record carries the scripted/0 marker; no real model request left the fixture. + assert.equal(first.environment.provider, 'scripted') + assert.equal(first.environment.networkModelRequests, 0) + assert.equal(first.environment.agentPreset, 'ops-ptc') + // D: the official provider is mounted for real; absent providers are covered below. + assert.equal(official.status, 'mounted', 'official read/glob/grep must be present: ' + JSON.stringify(official)) + // A/B: per snapshot, system is the sum of its named sections plus the pinned joins, + // and the SDK is exactly one section of it — never an extra line item. + for (const snapshot of Object.values(snapshots)) { + assert.equal(snapshot.sdk.sections, 1, 'exactly one tools:sdk section, got ' + snapshot.sdk.sections) + const sum = snapshot.system.items.reduce((total, item) => total + item.bytes, 0) + assert.equal(snapshot.system.bytes, sum + 2 * (snapshot.system.items.length - 1), + 'normalization pinned: system = per-section bytes joined by \'\n\n\', exactly renderPrompt') + assert.ok(snapshot.sdk.bytes > 0 && snapshot.sdk.bytes < snapshot.system.bytes, 'SDK is a strict subset of system') + } + // The notices enter history; the one-time teaching rides the first accepted + // root run_code result as same-step ephemeral context (#929), so it adds no + // history message. The system account never moves either way. + assert.equal(snapshots.steady.system.sha, snapshots.baseline.system.sha, 'the plan notice is history, not a prompt change') + assert.equal(snapshots.steady.sdk.sha, snapshots.baseline.sdk.sha) + // B: the hand-computed example — loading three APIs changes only tools:sdk, exactly once. + const nonSdk = snapshot => snapshot.system.items.filter(item => item.name !== 'tools:sdk').map(item => item.sha) + assert.notEqual(snapshots.loaded.sdk.sha, snapshots.steady.sdk.sha, 'the load must reach the SDK section') + assert.deepEqual(nonSdk(snapshots.loaded), nonSdk(snapshots.steady), 'no other section may move when tools load') + const delta = snapshots.loaded.system.bytes - snapshots.steady.system.bytes + assert.notEqual(delta, 0) + assert.equal(delta, snapshots.loaded.sdk.bytes - snapshots.steady.sdk.bytes, + 'the system delta equals the SDK-section delta: counted once, never double-added') + // D: SDK names against the real official schemas. On this baseline the shipped + // row does not yet remove the official bindings from the PTC SDK (PR #961 is + // open; the #978 contrast numbers were taken at its head, not here). The + // ledger records both name lists, and the assertion pins the current baseline: + // when #961 lands, this turns red on purpose and the accounts must be re-run. + for (const name of ['ops_inspect_local_file', 'ops_load_sdk']) assert.ok(snapshots.steady.sdk.names.includes(name), name) + for (const name of ['ops_glob', 'ops_grep', 'ops_read_files']) assert.ok(snapshots.loaded.sdk.names.includes(name), name) + // #961 has landed: the replaced official bindings are withdrawn from the PTC + // SDK whenever their ops replacement is servable, so the ledger pins the hidden + // state. The official provider is still mounted, which the unmeasurable branch + // below keeps honest when it is not. + for (const name of OFFICIAL) { + assert.equal(snapshots.loaded.sdk.names.includes(name), false, + '#961 landed: the official ' + name + ' is withdrawn from the PTC SDK') + } + // C: run_code stays the only wire transport on every request. + for (const request of requests) assert.deepEqual(request.wireNames, ['run_code']) + assert.equal(new Set(requests.map(request => request.wireSha)).size, 1, 'wire is byte-identical on every request') + assert.equal(requests[0].wireSha, snapshots.baseline.wire.sha, 'the idle snapshot wire is the delivered wire') + // D: the history account — the lesson never enters it (#929); only the plan notice + // is history, once, and nothing is appended thereafter. + const kindsAt = label => requests.find(request => request.label === label).history.map(item => item.kind) + assert.deepEqual(kindsAt('first'), [], 'the first request carries no teaching in history') + assert.deepEqual(kindsAt('after-notice'), ['notice:plan'], 'the thin program earns exactly one plan notice') + assert.deepEqual(kindsAt('before-load'), ['notice:plan']) + assert.deepEqual(kindsAt('after-load'), ['notice:plan'], 'a compliant load step earns no new notice') + assert.deepEqual(kindsAt('repeat'), ['notice:plan'], 'no re-teaching, no duplicate notice') + // The delivered system message is the idle assembly, and repeats are byte-identical. + const systemAt = label => requests.find(request => request.label === label).systemSha + assert.equal(systemAt('first'), snapshots.baseline.system.sha) + assert.equal(systemAt('after-notice'), snapshots.baseline.system.sha) + assert.equal(systemAt('before-load'), snapshots.steady.system.sha) + assert.equal(systemAt('after-load'), snapshots.loaded.system.sha) + assert.equal(systemAt('repeat'), snapshots.loaded.system.sha, 'the same config repeated is byte-identical') + assert.equal(snapshots.repeat.system.sha, snapshots.loaded.system.sha) + printLedger(first) + // E: the same fixture and code reproduce every byte in a fresh group. + const second = await withTempDir('ptc-cost-accounts-b', root => runScenario(root, { hostPlugins: HOST_PLUGINS })) + assert.equal(second.invalid, undefined) + assert.deepEqual(comparableOf(second), comparableOf(first), 'two runs of the same fixture must agree byte for byte') + if (process.env.DSH_OPS_PTC_COST_REPORT) { + fs.writeFileSync(process.env.DSH_OPS_PTC_COST_REPORT, JSON.stringify({ first, second: comparableOf(second) }, null, 2) + '\n') + } +}) + +await test('without the official provider the SDK-name comparison is marked unmeasurable, never a quiet pass', async () => { + const bare = await withTempDir('ptc-cost-accounts-bare', root => runScenario(root)) + assert.equal(bare.invalid, undefined, 'the routing gate failed: ' + JSON.stringify(bare.invalid ?? {})) + assert.equal(bare.official.status, 'unmeasurable') + assert.match(bare.official.reason, /read, glob, grep/) + assert.ok(bare.snapshots.baseline.sdk.names.includes('ops_load_sdk'), 'the four accounts are still recorded beside the marker') + console.log(JSON.stringify({ fourAccounts: { provider: bare.environment.provider, + networkModelRequests: bare.environment.networkModelRequests, officialSchema: bare.official } })) +}) + +report('ptc-cost-accounts') diff --git a/test/ptc-declaration-surface.test.mjs b/test/ptc-declaration-surface.test.mjs index 656f396a..4cb1d05e 100644 --- a/test/ptc-declaration-surface.test.mjs +++ b/test/ptc-declaration-surface.test.mjs @@ -1,9 +1,10 @@ /** PTC declarations must follow the step admission, including loader-less scopes. */ import { createScope } from '@deepseek-ai/dsh-scope' -import { assert, test, report } from './lib/harness.mjs' +import { assert, test, report, waitFor } from './lib/harness.mjs' import { bootHost, shutdown } from './lib/host.mjs' -import { prepareStep } from './lib/executor.mjs' +import { prepareStep, selectExecutor } from './lib/executor.mjs' import { mountPtc } from '../lib/publication/ptc.js' +import { acquire } from '../lib/publication/registrar.js' const tick = () => new Promise(resolve => setImmediate(resolve)) @@ -13,24 +14,47 @@ function declared(assembly) { return [...new Set([...sdk.matchAll(/^\s{2}(ops_[a-z_]+): \{/gm)].map(match => match[1]))].sort() } -async function setup(gate) { - const { ctx } = await bootHost({ withWorkingDirectory: true, toolsMode: 'ptc' }) +async function setup(gate, ptc = {}, sandboxMode = 'danger-full-access') { + // Root stays Native; each agent's scoped executor provides the PTC presentation, + // so a test can withdraw that presentation exactly the way ptc-sdk does. + const { ctx } = await bootHost({ withWorkingDirectory: true }) const agents = [] ctx.provide('agents', { list: () => agents }) + // One mutable policy object: a test flips its mode to withdraw permission mid-scope. + const sandbox = { mode: sandboxMode, workspaceRoot: process.cwd() } await ctx.plugin(inner => { inner.provide('ptcRuntime', { language: 'typescript' }) - inner.provide('sandboxPolicy', { resolve: () => ({ mode: 'danger-full-access', workspaceRoot: process.cwd() }) }) + inner.provide('sandboxPolicy', { resolve: () => sandbox }) inner.provide('subprocess', { spawn() { throw new Error('declaration probe must never spawn') } }) }) - mountPtc(ctx, { binaryPath: process.execPath, required: true }, { allows: () => true, ...gate }) - const agent = { id: 'surface', session: { id: 'surface', mode: 'danger-full-access', header: { cwd: process.cwd() }, append() {}, read() { return [] } } } + mountPtc(ctx, { binaryPath: process.execPath, required: true, ...ptc }, { allows: () => true, ...gate }) + const agent = { id: 'surface', session: { id: 'surface', mode: sandboxMode, header: { cwd: process.cwd() }, append() {}, read() { return [] } } } agent.ctx = createScope(ctx, agent).ctx + await agent.ctx.plugin({ inject: ['tools', 'systemPrompt'], apply(inner) { agent.lift = selectExecutor(inner, 'ptc') } }) agents.push(agent) await ctx.serial('agent/created', { agent }) await tick(); await tick() - return { ctx, agent } + return { ctx, agent, sandbox } } +/** Flip the agent's scoped executor; the old handle is disposed first. */ +async function setExecutor(agent, mode) { + agent.lift() + await agent.ctx.plugin({ inject: ['tools', 'systemPrompt'], apply(inner) { agent.lift = selectExecutor(inner, mode) } }) +} + +/** Drive the retained loader definition the way a step call would. */ +async function loadRaw(ctx, agent, raw) { + const step = await prepareStep(ctx, agent) + try { + const loader = ctx.get('tools').get('ops_load_sdk', agent, step.toolSet) + return await loader.execute({ tools: [raw] }, { agent, name: 'ops_load_sdk', toolSet: step.toolSet, signal: new AbortController().signal }) + } finally { step.dispose() } +} + +/** An isolated admitted view over one definition, the way a retained step binding sees it. */ +const isolated = (name, definition) => ({ view: () => new Map([[name, definition]]) }) + await test('a loader-less scope declares no selectable API even without a step tool set', async () => { const { ctx, agent } = await setup({ resident: false, dynamic: false }) try { @@ -43,4 +67,205 @@ await test('a loader-less scope declares no selectable API even without a step t assert.deepEqual(admitted, []) } finally { await shutdown(ctx) } }) + +/** Poll the first-round SDK declaration until publication settles. */ +async function declaredNow(ctx, agent) { + const step = await prepareStep(ctx, agent) + try { return declared(step.assembly) } finally { step.dispose() } +} + +await test('an explicit full tier declares the whole permitted pool in the first round, loader-less', async () => { + // D1 truth-table row T (#987): the declaration half of the dual verification; + // the run_code binding half lives in the worker-backed ptc-full-disclosure suite. + const { ctx, agent } = await setup({ resident: true, dynamic: true }, { disclosurePolicy: 'full', allowedTools: ['grep', 'stat'] }) + try { + await waitFor(async () => (await declaredNow(ctx, agent)).includes('ops_grep'), + { what: 'ops_grep declared in the first round', timeoutMs: 10_000 }) + const admitted = await declaredNow(ctx, agent) + assert.deepEqual(admitted, ['ops_grep', 'ops_inspect_local_file', 'ops_result', 'ops_stat']) + assert.equal(admitted.includes('ops_load_sdk'), false, 'the full tier publishes no loader (#979 §2)') + assert.equal(admitted.includes('ops_glob'), false, 'a pool-outside business API stays undeclared') + } finally { await shutdown(ctx) } +}) + +await test('read-only keeps mutating APIs out of the full-tier declaration', async () => { + const { ctx, agent } = await setup({ resident: true, dynamic: true }, { disclosurePolicy: 'full', allowedTools: ['write_file', 'stat'] }, 'read-only') + try { + await waitFor(async () => (await declaredNow(ctx, agent)).includes('ops_stat'), + { what: 'ops_stat declared under read-only', timeoutMs: 10_000 }) + assert.deepEqual(await declaredNow(ctx, agent), ['ops_inspect_local_file', 'ops_result', 'ops_stat'], + 'the mutating member of the cap is never declared') + } finally { await shutdown(ctx) } +}) + +await test('an empty allowed pool declares only the resident surface plus result', async () => { + const { ctx, agent } = await setup({ resident: true, dynamic: true }, { disclosurePolicy: 'full', allowedTools: [] }) + try { + await waitFor(async () => (await declaredNow(ctx, agent)).includes('ops_result'), + { what: 'ops_result declared over an empty pool', timeoutMs: 10_000 }) + const admitted = await declaredNow(ctx, agent) + assert.deepEqual(admitted, ['ops_inspect_local_file', 'ops_result'], 'nothing else to publish, and still no loader') + assert.equal(admitted.includes('ops_load_sdk'), false) + } finally { await shutdown(ctx) } +}) + +await test('duplicate allowed names register exactly once in the full tier', async () => { + const { ctx, agent } = await setup({ resident: true, dynamic: true }, { disclosurePolicy: 'full', allowedTools: ['stat', 'stat', 'grep'] }) + try { + await waitFor(async () => (await declaredNow(ctx, agent)).includes('ops_grep'), + { what: 'ops_grep declared', timeoutMs: 10_000 }) + const step = await prepareStep(ctx, agent) + try { + const registered = ctx.get('tools').definitions(agent, step.toolSet).map(tool => tool.name) + assert.equal(registered.filter(name => name === 'ops_stat').length, 1, 'a duplicated name registers exactly once') + assert.deepEqual(declared(step.assembly), ['ops_grep', 'ops_inspect_local_file', 'ops_result', 'ops_stat']) + } finally { step.dispose() } + } finally { await shutdown(ctx) } +}) + +await test('the same agent keeps registered, declared and admitted names distinct across the two assembly paths', async () => { + const { ctx, agent } = await setup({ resident: true, dynamic: true }, { allowedTools: ['batch'] }) + try { + await waitFor(async () => (await declaredNow(ctx, agent)).includes('ops_load_sdk'), + { what: 'loader declared', timeoutMs: 10_000 }) + // With a step tool set: the request path. The unloaded pool member declares nothing + // and holds no program binding, so calling it is a JS error, never a tool error. + assert.deepEqual(await declaredNow(ctx, agent), ['ops_inspect_local_file', 'ops_load_sdk'], + 'an unloaded name never impersonates a callable one') + const step = await prepareStep(ctx, agent) + try { assert.equal(step.toolSet.view(agent).has('ops_batch'), false, 'no program binding for the unloaded name') } finally { step.dispose() } + // Without a step tool set: the diagnostics path. The same agent's registration + // view still lists the pool member; recorded, never published as callable. + const registered = ctx.get('tools').definitions(agent).map(tool => tool.name).filter(name => name.startsWith('ops_')).sort() + assert.deepEqual(registered, ['ops_batch', 'ops_inspect_local_file', 'ops_load_sdk', 'ops_result']) + } finally { await shutdown(ctx) } +}) + +await test('a diagnostic assembly without a tool set is not reported as a first model request', async () => { + const { ctx, agent } = await setup({ resident: true, dynamic: true }, { allowedTools: ['batch'] }) + try { + await waitFor(async () => (await declaredNow(ctx, agent)).includes('ops_load_sdk'), + { what: 'loader declared', timeoutMs: 10_000 }) + const receipt = await loadRaw(ctx, agent, 'batch') + assert.equal(receipt.status, 'completed', JSON.stringify(receipt)) + // The scope's presentation flips to Native without any model step in between. + await setExecutor(agent, 'native') + // The no-tool-set snapshot is the diagnostics path: it marks the withdrawn + // presentation but must not consume the pending load as a request would. + const diagnostic = await ctx.get('systemPrompt').assemble({ scope: agent }) + assert.equal(diagnostic.sections.some(section => section.name === 'tools:ptc-only'), false, + 'the snapshot itself confirms the Native presentation') + await setExecutor(agent, 'ptc') + await ctx.serial('session/event', agent.session, { type: 'step/end' }) + assert.deepEqual(await declaredNow(ctx, agent), ['ops_batch', 'ops_inspect_local_file', 'ops_load_sdk'], + 'the pending load survives the diagnostic assembly') + } finally { await shutdown(ctx) } +}) + +await test('calling an unloaded pool member through the dispatcher points at the loader, not at denial', async () => { + const { ctx, agent } = await setup({ resident: true, dynamic: true }, { allowedTools: ['batch'] }) + try { + await waitFor(async () => (await declaredNow(ctx, agent)).includes('ops_load_sdk'), + { what: 'loader declared', timeoutMs: 10_000 }) + const def = ctx.get('tools').get('ops_batch', agent) + assert.ok(def, 'the pool member registers without a step admission') + // Isolate the dispatch verdict: the call runs through the definition's own + // dispatcher over an admitted view, exactly like a retained step binding. + const verdict = await def.execute({ action: 'plan', operations: [] }, + { agent, toolSet: isolated('ops_batch', def), signal: new AbortController().signal }) + // The result-lifecycle admission classifies the raw refusal; assert the + // four-verdict error envelope rather than its wrapper status. + assert.equal(verdict.error.code, 'SDK_NOT_LOADED', JSON.stringify(verdict)) + assert.match(verdict.error.message, /ops_load_sdk/) + assert.match(verdict.error.message, /batch/) + assert.equal(verdict.error.effectState, 'none') + assert.equal(verdict.error.retry, 'never') + } finally { await shutdown(ctx) } +}) + +await test('a permission-forbidden API answers denial without a load suggestion', async () => { + const { ctx, agent } = await setup({ resident: true, dynamic: true }, { allowedTools: ['batch', 'grep'] }, 'read-only') + try { + await waitFor(async () => (await declaredNow(ctx, agent)).includes('ops_load_sdk'), + { what: 'loader declared under read-only', timeoutMs: 10_000 }) + const admitted = await declaredNow(ctx, agent) + assert.equal(admitted.includes('ops_batch'), false, 'a mutating API stays undeclared under read-only') + // Through the loader: the forbidden name is inside the configured pool, so + // permission, not pool membership, refuses it — and no reload can change that. + const loader = await loadRaw(ctx, agent, 'batch') + assert.equal(loader.error.code, 'POLICY_DENIED', JSON.stringify(loader)) + assert.match(loader.error.message, /sandbox policy/) + assert.ok(!loader.error.message.includes('ops_load_sdk'), 'a permission refusal never points at the loader') + // A name outside the configured pool keeps the pool refusal. + const outsider = await loadRaw(ctx, agent, 'replace') + assert.equal(outsider.error.code, 'INVALID_ARGUMENT', JSON.stringify(outsider)) + // Through an isolated dispatch view: the same permission verdict, not a loader pointer. + const def = ctx.get('tools').get('ops_batch', agent) + const direct = await def.execute({ action: 'plan', operations: [] }, + { agent, toolSet: isolated('ops_batch', def), signal: new AbortController().signal }) + assert.equal(direct.error.code, 'POLICY_DENIED', JSON.stringify(direct)) + assert.match(direct.error.message, /sandbox policy/) + assert.ok(!direct.error.message.includes('ops_load_sdk'), 'a permission refusal never suggests loading the same tool') + } finally { await shutdown(ctx) } +}) + +await test('a loaded API whose admission is later withdrawn answers a revoked verdict', async () => { + const { ctx, agent, sandbox } = await setup({ resident: true, dynamic: true }, { allowedTools: ['batch'] }) + try { + await waitFor(async () => (await declaredNow(ctx, agent)).includes('ops_load_sdk'), + { what: 'loader declared', timeoutMs: 10_000 }) + assert.equal((await loadRaw(ctx, agent, 'batch')).status, 'completed') + await ctx.serial('session/event', agent.session, { type: 'step/end' }) + await waitFor(async () => (await declaredNow(ctx, agent)).includes('ops_batch'), + { what: 'batch declared after loading', timeoutMs: 10_000 }) + // Retain the step that admitted the loaded API, then withdraw the permission. + const step = await prepareStep(ctx, agent) + try { + sandbox.mode = 'read-only' + const def = ctx.get('tools').get('ops_batch', agent, step.toolSet) + const verdict = await def.execute({ action: 'plan', operations: [] }, + { agent, toolSet: step.toolSet, signal: new AbortController().signal }) + assert.equal(verdict.error.code, 'POLICY_DENIED', JSON.stringify(verdict)) + assert.match(verdict.error.message, /revoked/) + assert.ok(!verdict.error.message.includes('ops_load_sdk'), 'a revoked admission never suggests reloading') + } finally { step.dispose() } + // Disposing the scope revokes the retained handle with the scope's own verdict. + const retained = ctx.get('tools').get('ops_batch', agent) + await ctx.serial('agent/disposed', { agent }) + const scope = await retained.execute({ action: 'plan', operations: [] }, + { agent, signal: new AbortController().signal }) + assert.equal(scope.error.code, 'POLICY_DENIED', JSON.stringify(scope)) + assert.match(scope.error.message, /revoked/) + assert.notEqual(scope.error.message, 'This API must be admitted for the calling PTC scope; no action executed.', + 'scope withdrawal reads differently from the plain admission refusal') + } finally { await shutdown(ctx) } +}) + +await test('a pulled publication grant answers revoked rather than never available', async () => { + const { ctx } = await bootHost({ withWorkingDirectory: true }) + try { + const agent = { id: 'revoked-grant', session: { id: 'revoked-grant' } } + agent.ctx = createScope(ctx, agent).ctx + const canonical = { description: 'Registrar fixture', parameters: { type: 'object', properties: {} }, + output: { schema: { type: 'object' }, render: () => [] } } + const lease = await acquire(ctx, undefined, 'registrar_revoked', canonical) + let calls = 0 + const dispatch = async () => { ++calls; return {} } + // An entitled grant existed for this agent and was pulled; the replacement + // is not entitled, so the eligible set reads empty exactly as a revocation would. + lease.grant({ consumer: {}, agent, token: {}, entitled: true, dispatch })() + lease.grant({ consumer: {}, agent, token: {}, entitled: false, dispatch }) + const def = ctx.get('tools').get('registrar_revoked', agent) + assert.ok(def, 'the current generation stays registered after the rotation') + const verdict = await def.execute({}, + { agent, toolSet: isolated('registrar_revoked', def), signal: new AbortController().signal }) + assert.equal(verdict.status, 'rejected', JSON.stringify(verdict)) + assert.equal(verdict.error.code, 'PUBLICATION_GRANT_REVOKED', JSON.stringify(verdict)) + assert.equal(verdict.error.effectState, 'none') + assert.equal(verdict.error.retry, 'never') + assert.equal(calls, 0, 'a revoked grant never dispatches') + lease.dispose() + } finally { await shutdown(ctx) } +}) + report('ptc-declaration-surface') diff --git a/test/ptc-deferred-rejection.test.mjs b/test/ptc-deferred-rejection.test.mjs new file mode 100644 index 00000000..c77aa6ae --- /dev/null +++ b/test/ptc-deferred-rejection.test.mjs @@ -0,0 +1,88 @@ +/** A registered-but-unselected pool API must point at the loader, not a dead end (#968). */ +import { createScope } from '@deepseek-ai/dsh-scope' +import { mountPtc } from '../lib/publication/ptc.js' +import { publicToolName } from '../lib/policy.js' +import { assert, test, report, waitFor } from './lib/harness.mjs' +import { bootHost, shutdown } from './lib/host.mjs' + +const tick = () => new Promise(resolve => setImmediate(resolve)) +/** The minimal tool-set view the registry's executable check accepts for one definition. */ +const soloView = definition => ({ view: () => new Map([[definition.name, definition]]) }) + +async function setup(config, gate = { allows: () => true }) { + const { ctx } = await bootHost({ withWorkingDirectory: true, toolsMode: 'ptc' }) + const agents = [] + ctx.provide('agents', { list: () => agents }) + await ctx.plugin(inner => { + inner.provide('ptcRuntime', { language: 'typescript' }) + inner.provide('sandboxPolicy', { resolve: ({ session }) => ({ mode: session?.mode ?? 'danger-full-access', workspaceRoot: process.cwd() }) }) + inner.provide('subprocess', { spawn() { throw new Error('deferred rejection fixtures never spawn') } }) + }) + mountPtc(ctx, { binaryPath: process.execPath, required: true, ...config }, gate) + const mint = async (mode = 'danger-full-access') => { + const id = 'deferred-' + mode + const agent = { id, session: { id, mode, header: { cwd: process.cwd() }, append() {}, read() { return [] } } } + agent.ctx = createScope(ctx, agent).ctx + agents.push(agent) + await ctx.serial('agent/created', { agent }) + await tick() + return agent + } + return { ctx, mint } +} + +/** Dispatch the registered definition the way a host that ignores `when` does (#934 shape). */ +const dispatch = async (ctx, agent, raw) => { + const tools = ctx.get('tools') + const definition = tools.get(publicToolName(raw), agent) + ?? tools.definitions(agent).find(def => def.name === publicToolName(raw)) + assert.ok(definition, publicToolName(raw) + ' must be registered for this scenario') + return definition.execute({}, { agent, signal: new AbortController().signal, toolSet: soloView(definition) }) +} + +await test('an unselected shared-file pool API answers with the loader path, not a retry-never dead end', async () => { + const { ctx, mint } = await setup({ allowedTools: ['grep', 'batch', 'write_file'] }) + try { + const agent = await mint() + await waitFor(() => ctx.get('tools').get(publicToolName('grep'), agent), { what: 'PTC pool registration' }) + const result = await dispatch(ctx, agent, 'grep') + assert.equal(result.error?.code, 'SDK_NOT_LOADED') + assert.match(result.error.message, /grep/) + assert.match(result.error.nextAction ?? '', /ops_load_sdk/) + assert.match(result.error.nextAction ?? '', /next model step/) + } finally { await shutdown(ctx) } +}) + +await test('an unselected PTC-only pool API answers with the same loader path', async () => { + const { ctx, mint } = await setup({ allowedTools: ['grep', 'batch', 'write_file'] }) + try { + const agent = await mint() + await waitFor(() => ctx.get('tools').get(publicToolName('batch'), agent), { what: 'PTC pool registration' }) + const result = await dispatch(ctx, agent, 'batch') + assert.equal(result.error?.code, 'SDK_NOT_LOADED') + assert.match(result.error.message, /batch/) + assert.match(result.error.nextAction ?? '', /ops_load_sdk/) + } finally { await shutdown(ctx) } +}) + +await test('a pool API the current policy cannot load keeps the generic refusal', async () => { + const { ctx, mint } = await setup({ allowedTools: ['grep', 'batch', 'write_file'] }) + try { + const agent = await mint('read-only') + await waitFor(() => ctx.get('tools').get(publicToolName('write_file'), agent), { what: 'PTC pool registration' }) + const result = await dispatch(ctx, agent, 'write_file') + // Read-only policy can never load a mutating API, so pointing at the loader would be dishonest. + assert.notEqual(result.error?.code, 'SDK_NOT_LOADED') + } finally { await shutdown(ctx) } +}) + +await test('a loader-less scope keeps registering nothing from the pool', async () => { + const { ctx, mint } = await setup({ allowedTools: ['grep'] }, { allows: () => true, dynamic: false }) + try { + const agent = await mint() + await tick(); await tick() + assert.equal(ctx.get('tools').get(publicToolName('grep'), agent), undefined) + } finally { await shutdown(ctx) } +}) + +report('ptc-deferred-rejection') diff --git a/test/ptc-directory.test.mjs b/test/ptc-directory.test.mjs index ef3ee986..c4af0256 100644 --- a/test/ptc-directory.test.mjs +++ b/test/ptc-directory.test.mjs @@ -110,8 +110,12 @@ await test('current authority narrows configured names; unavailable policy does try { const agent = await mint() agent.session.mode = 'read-only'; await ctx.serial('session/event', agent.session, { type: 'sandbox/mode' }) - assert.equal((await load(ctx, agent, ['write_file'])).error.code, 'INVALID_ARGUMENT') - assert.equal((await load(ctx, agent, ['batch'])).error.code, 'INVALID_ARGUMENT') + // #997: a configured name the current authority forbids answers a permission + // refusal, not a pool refusal — and it never points back at the loader. + const forbidden = (await load(ctx, agent, ['write_file'])).error + assert.equal(forbidden.code, 'POLICY_DENIED') + assert.ok(!forbidden.message.includes('ops_load_sdk'), 'a permission refusal never points at the loader') + assert.equal((await load(ctx, agent, ['batch'])).error.code, 'POLICY_DENIED') assert.equal((await load(ctx, agent, ['grep', 'result'])).status, 'completed') state.available = false await assert.rejects(() => text(ctx, agent), error => error.name === 'ToolActivationError' && error.cause?.message === 'policy unavailable') diff --git a/test/ptc-disclosure-policy.test.mjs b/test/ptc-disclosure-policy.test.mjs new file mode 100644 index 00000000..b69f4934 --- /dev/null +++ b/test/ptc-disclosure-policy.test.mjs @@ -0,0 +1,144 @@ +/** The D2 parsing contract: one explicit disclosure policy key on the PTC rows. + * + * #979 splits the two-tier OPS-PTC work into slices; this suite locks the D2 + * slice (#986): the row key `disclosurePolicy` ('on-demand' | 'full', default + * 'on-demand' = the legacy path), its validation diagnostics, its survival + * through the settings save path, and the conservative precedence taken for the + * D1 sign-off item 2 — the explicit full tier never overrides the bundle + * `dynamicTools` switch, and no save, upgrade or switch flip ever writes the + * key for the user (#979 §4). Publication of the full tier itself is D3; every + * assertion here is resolver-level only. + */ +import { assert, test, report } from './lib/harness.mjs' +import { parsePtcConfig } from '../lib/ptc/config.js' +import { modeConfigForm, modeFields, resolveModeConfig } from '../lib/modes/config.js' +import { Config, effectiveMode, hostConfig, resolveMergedConfig } from '../lib/settings/merged.js' +import { changedFields, readRowValues } from '../lib/client/row-schema.js' + +await test('input class 1 (key absent): every surface keeps the legacy default and slice', () => { + // The parsed contract gains a defaulted field; the effective slice and every + // published surface stay letter-identical to the pre-D2 baseline. + assert.equal(parsePtcConfig({}).disclosurePolicy, 'on-demand') + for (const mode of ['minimal', 'standard', 'ptc', 'ops-ptc', 'cordis']) { + assert.equal(resolveModeConfig({}, mode).disclosurePolicy, 'on-demand', mode) + } +}) + +await test('input class 2 (legal value): on-demand and full parse and ride the settings path', () => { + assert.equal(parsePtcConfig({ disclosurePolicy: 'on-demand' }).disclosurePolicy, 'on-demand') + assert.equal(parsePtcConfig({ disclosurePolicy: 'full' }).disclosurePolicy, 'full') + for (const mode of ['ptc', 'ops-ptc']) { + assert.equal(resolveModeConfig({ disclosurePolicy: 'full' }, mode).disclosurePolicy, 'full', mode) + assert.equal(resolveModeConfig({ disclosurePolicy: 'on-demand' }, mode).disclosurePolicy, 'on-demand', mode) + } + // D5 (#989) moved the key onto the merged row page as a volatile string the + // disclosure card owns; the legacy per-mode form keeps it YAML-only. + assert.equal('disclosurePolicy' in modeFields('ops-ptc'), false) + assert.equal('disclosurePolicy' in Config.dict['ops-ptc'].dict, true) + assert.equal(Config.dict['ops-ptc'].dict.disclosurePolicy.meta.volatile, true) +}) + +await test('input class 3 (misspelled value): the diagnostic names the key and the value', () => { + assert.throws(() => parsePtcConfig({ disclosurePolicy: 'ondemand' }), error => { + assert.match(error.message, /disclosurePolicy must be "on-demand" or "full"/) + assert.match(error.message, /"ondemand"/) + return true + }) +}) + +await test('input class 4 (wrong type): the diagnostic names the key and the type', () => { + for (const value of [true, 5, ['full']]) { + assert.throws(() => parsePtcConfig({ disclosurePolicy: value }), error => { + assert.match(error.message, /disclosurePolicy must be "on-demand" or "full"/) + assert.match(error.message, new RegExp('\\b' + typeof value + '\\b')) + return true + }, JSON.stringify(value)) + } +}) + +await test('modes without a PTC surface reject the key with a readable diagnostic', () => { + // The key names a PTC publication tier; a Native row carrying it is a config + // mistake that must fail loud, not silently drift (the denyCredentialReads + // precedent for cross-surface keys). + for (const mode of ['minimal', 'standard', 'cordis']) { + assert.throws(() => resolveModeConfig({ disclosurePolicy: 'full' }, mode), + /disclosurePolicy applies to the ptc and ops-ptc rows only/, mode) + assert.throws(() => resolveModeConfig({ disclosurePolicy: 'full' }, mode), /disclosurePolicy/, mode) + } +}) + +await test('round trip: read → save an unrelated setting → read again keeps the explicit choice', () => { + const raw = { 'ops-ptc': { disclosurePolicy: 'full', allowedTools: ['grep', 'stat'] } } + const first = resolveMergedConfig(raw) + assert.equal(first['ops-ptc'].config.disclosurePolicy, 'full') + assert.equal(first['ops-ptc'].raw?.disclosurePolicy, 'full') + // The host settings layer serializes through hostConfig and validates through + // the row descriptor; both preserve the key. + assert.equal(hostConfig(raw)['ops-ptc'].disclosurePolicy, 'full') + assert.equal(Config['~standard'].validate(raw).value['ops-ptc'].disclosurePolicy, 'full') + // The component's own settings descriptor keeps raw ordinary keys too. + const form = modeConfigForm('ops-ptc') + assert.equal(form['~standard'].validate({ disclosurePolicy: 'full' }).value.disclosurePolicy, 'full') + // The page writes only the field the operator touched: saving panel capture + // produces exactly one op and never touches the ops-ptc section. + const draft = readRowValues(raw) + draft['panel.enabled'] = true + assert.deepEqual(changedFields(draft, raw), [{ op: 'set', path: ['panel', 'enabled'], value: true }]) + // The host applies that one op; reading again still sees the explicit choice. + const after = resolveMergedConfig({ ...raw, panel: { enabled: true } }) + assert.equal(after['ops-ptc'].config.disclosurePolicy, 'full', 'the explicit selection is not lost') + assert.deepEqual(after['ops-ptc'].config.allowedTools, ['grep', 'stat']) +}) + +await test('legacy configs never gain the key; the global switch never rewrites it', () => { + // The D1 truth-table five inputs, each re-resolved on the new parsing path. + const inputs = [ + {}, + { global: { dynamicTools: false } }, + { 'ops-ptc': { dynamicTools: false } }, + { 'ops-ptc': { dynamicTools: false, alwaysLoaded: ['stat'] } }, + { 'ops-ptc': { residentTools: false } }, + ] + for (const raw of inputs) { + const merged = resolveMergedConfig(raw) + for (const mode of ['minimal', 'standard', 'ptc', 'ops-ptc', 'cordis']) { + const slice = effectiveMode(merged[mode], merged.global) + assert.equal('disclosurePolicy' in slice, false, 'the effective slice stays letter-identical', mode) + assert.equal('disclosurePolicy' in (merged[mode].raw ?? {}), false, mode) + } + assert.equal(merged.ptc.config.disclosurePolicy, 'on-demand') + assert.equal(merged['ops-ptc'].config.disclosurePolicy, 'on-demand') + // hostConfig is the save-path serializer: it never auto-writes the key. + const saved = hostConfig(raw) + for (const mode of ['minimal', 'standard', 'ptc', 'ops-ptc', 'cordis']) { + assert.equal('disclosurePolicy' in saved[mode], false, 'an upgrade or save never picks full for the user') + } + } +}) + +await test('the explicit full tier never overrides the bundle switch (sign-off item 2)', () => { + const saved = { 'ops-ptc': { disclosurePolicy: 'full' } } + const on = resolveMergedConfig(saved) + assert.equal(effectiveMode(on['ops-ptc'], on.global).disclosurePolicy, 'full') + assert.equal(resolveModeConfig(effectiveMode(on['ops-ptc'], on.global), 'ops-ptc').disclosurePolicy, 'full') + const off = resolveMergedConfig({ global: { dynamicTools: false }, ...saved }) + // The saved intent is kept; only the effective slice stops applying, exactly + // like the existing dynamicTools AND gate keeps the operator's intent. + assert.equal(off['ops-ptc'].config.disclosurePolicy, 'full') + assert.equal(off['ops-ptc'].raw?.disclosurePolicy, 'full') + assert.equal(hostConfig({ global: { dynamicTools: false }, ...saved })['ops-ptc'].disclosurePolicy, 'full') + assert.equal(effectiveMode(off['ops-ptc'], off.global).disclosurePolicy, 'on-demand', 'the bundle switch keeps its veto') + assert.equal(resolveModeConfig(effectiveMode(off['ops-ptc'], off.global), 'ops-ptc').disclosurePolicy, 'on-demand') + assert.equal(effectiveMode(off['ops-ptc'], off.global).dynamicTools, false) + // The row switch keeps its loader-only meaning; the full tier publishes no + // loader, so a row that also turns the loader off keeps its explicit tier. + const rowOff = resolveMergedConfig({ 'ops-ptc': { disclosurePolicy: 'full', dynamicTools: false } }) + assert.equal(effectiveMode(rowOff['ops-ptc'], rowOff.global).disclosurePolicy, 'full') + // Flipping the global switch alone never writes the key anywhere. + for (const global of [{ dynamicTools: false }, { dynamicTools: true }]) { + const flipped = resolveMergedConfig({ global }) + assert.equal('disclosurePolicy' in (flipped['ops-ptc'].raw ?? {}), false, JSON.stringify(global)) + } +}) + +report('ptc-disclosure-policy') diff --git a/test/ptc-full-disclosure.test.mjs b/test/ptc-full-disclosure.test.mjs new file mode 100644 index 00000000..f9633b7a --- /dev/null +++ b/test/ptc-full-disclosure.test.mjs @@ -0,0 +1,218 @@ +/** The D3 publication slice: an explicit full tier actually publishes the pool. + * + * #979 splits the two-tier OPS-PTC work into slices; this suite locks the D3 + * slice (#987) end to end through the shipped merged row: with + * `disclosurePolicy: 'full'` the first round carries the whole permitted + * `allowedTools` pool plus `result` (the D1 sign-off item 1 decision, taken + * conservatively per #979 §3 so refs spilled this step stay consumable, and + * forced by the unified acceptance rule: on-demand can always load `result`, + * so the loaded set equals the full first round only if full publishes it) + * with no `ops_load_sdk`. Every check runs the real worker through the real + * PTC executor, and the dual verification the card requires: the settled + * first-round name set plus an actual run_code binding call. + */ +import { createScope, scopeOf, scopeParentOf } from '@deepseek-ai/dsh-scope' +import Subprocess from '@deepseek-ai/dsh-subprocess-local' +import Sandbox from '@deepseek-ai/dsh-sandbox-local' +import FileSystem from '@deepseek-ai/dsh-fs-local' +import NodePtc from '@deepseek-ai/dsh-ptc-runtime-node' +import fs from 'node:fs' +import path from 'node:path' +import { apply } from '../lib/index.js' +import { BASH_TOOL, publicToolName } from '../lib/policy.js' +import { bootHost, shutdown } from './lib/host.mjs' +import { prepareStep, executeCall, selectExecutor } from './lib/executor.mjs' +import { assert, test, report, wait, waitFor, withTempDir } from './lib/harness.mjs' +import { WORKER, requireWorker } from './lib/boundary.mjs' + +requireWorker('ptc-full-disclosure') + +let sequence = 0 +// The integration diagnostics row (ops_status, #753) and the Windows Bash tool +// are published beside the PTC tier and have their own suites; this enumeration +// is the PTC disclosure surface. +const names = (ctx, agent) => ctx.get('tools').definitions(agent, agent.prepared?.toolSet).map(tool => tool.name) + .filter(name => name !== 'ops_status' && name !== BASH_TOOL).sort() +const opsNames = (ctx, agent) => names(ctx, agent).filter(name => name.startsWith('ops_')) +async function step(ctx, agent) { + ctx.emit('session/event', agent.session, { type: 'step/end', turn: 1, step: ++sequence }) + agent.prepared?.dispose(); agent.prepared = await prepareStep(ctx, agent) +} +async function scoped(parent, key, parentKey) { + let result + await parent.plugin({ inject: ['tools'], apply(inner) { + result = createScope(inner, key, parentKey ? { parent: parentKey } : undefined).ctx + } }) + return result +} +async function setup(cwd, switches, global = {}, sandboxMode = 'danger-full-access') { + const { ctx } = await bootHost({ withWorkingDirectory: true }) + await ctx.plugin(Subprocess); await ctx.plugin(Sandbox); await ctx.plugin(FileSystem); await ctx.plugin(NodePtc) + ctx.provide('sandboxPolicy', { resolve: ({ session } = {}) => ({ mode: session?.mode ?? sandboxMode, workspaceRoot: cwd, sessionId: session?.id }) }) + const presets = new Map(), agents = [] + ctx.provide('agents', { list: () => [...agents] }) + ctx.provide('agentPresets', { composedPreset: inner => presets.get(scopeParentOf(scopeOf(inner))) }) + const config = { global, discovery: { binaryPath: WORKER, required: true } } + for (const id of ['minimal', 'standard', 'ptc', 'ops-ptc', 'cordis']) config[id] = { enabled: id === 'ops-ptc', + binaryPath: WORKER, required: true, ...(id === 'ops-ptc' ? switches : {}) } + await ctx.plugin({ name: 'full-disclosure-fixture', apply: inner => apply(inner, config) }) + const id = 'full-' + ++sequence, preset = await scoped(ctx, { preset: id }) + presets.set(scopeOf(preset), 'ops-ptc') + for (const name of ['read', 'glob', 'grep']) preset.tools.register({ name, description: name, + parameters: { type: 'object', properties: {} }, output: { schema: { type: 'object' }, render: () => [] }, execute: async () => ({}) }) + const agent = { id, session: { id, mode: sandboxMode, header: { cwd }, append() {}, read() { return [] } } } + agent.ctx = await scoped(preset, agent, scopeOf(preset)) + await agent.ctx.plugin({ inject: ['tools', 'systemPrompt'], apply: inner => { selectExecutor(inner, 'ptc') } }) + agents.push(agent) + await ctx.serial('agent/created', { agent }) + await step(ctx, agent) + return { ctx, agent } +} + +/** Boot one ops-ptc row, wait for its anchor, return the settled first-round set. */ +async function firstRound(switches, anchor, global = {}, sandboxMode = 'danger-full-access') { + return withTempDir('full-' + ++sequence, async cwd => { + fs.writeFileSync(path.join(cwd, 'fixture.txt'), 'needle in haystack\nplain second line\n') + const { ctx, agent } = await setup(cwd, switches, global, sandboxMode) + try { + await waitFor(() => opsNames(ctx, agent).includes(anchor), { what: anchor + ' published' }) + await wait(100) + return opsNames(ctx, agent) + } finally { await shutdown(ctx) } + }) +} + +/** One real run_code program through the prepared step, as the loop issues it. */ +async function program(ctx, agent, code, label) { + const result = await executeCall(ctx, { agent, toolSet: agent.prepared.toolSet, name: 'run_code', + arguments: { description: label, code }, callId: 'full-' + ++sequence, signal: new AbortController().signal }) + assert.equal(result.isError, false, JSON.stringify(result)) + return result.value.result +} + +/** D1 truth-table row T: the explicit full selection, first round, no loader. */ +const ROW_T = ['ops_grep', 'ops_inspect_local_file', 'ops_result', 'ops_stat'] + +await test('row T: full + allowedTools=[grep,stat] publishes the whole pool in the first round, loader-less', () => withTempDir('full-row-t', async cwd => { + fs.writeFileSync(path.join(cwd, 'fixture.txt'), 'needle in haystack\nplain second line\n') + const { ctx, agent } = await setup(cwd, { disclosurePolicy: 'full', allowedTools: ['grep', 'stat'] }) + try { + await waitFor(() => opsNames(ctx, agent).includes('ops_grep'), { what: 'ops_grep published' }) + await wait(100) + assert.deepEqual(opsNames(ctx, agent), ROW_T) + assert.equal(opsNames(ctx, agent).includes('ops_load_sdk'), false, 'the full tier publishes no loader (#979 §2)') + // Dual verification: the binding works in the very first program, on a file + // with known content; pool-outside and loader bindings stay absent. + const value = await program(ctx, agent, "const stat = await tools.ops_stat({ path: 'fixture.txt' })\n" + + "const grep = await tools.ops_grep({ pattern: 'needle', path: '.' })\n" + + "return { statStatus: stat.status, grepStatus: grep.status, grepText: JSON.stringify(grep),\n" + + " loader: typeof tools.ops_load_sdk, glob: typeof tools.ops_glob }", 'Stat and grep the fixture') + assert.equal(value.statStatus, 'completed', JSON.stringify(value)) + assert.equal(value.grepStatus, 'completed', JSON.stringify(value)) + assert.match(value.grepText, /needle/, 'the real worker answered the grep call') + assert.match(value.grepText, /fixture\.txt/, 'the real worker answered the grep call') + assert.equal(value.loader, 'undefined', 'no loader binding in the full first round') + assert.equal(value.glob, 'undefined', 'a pool-outside business API is not callable') + } finally { await shutdown(ctx) } +})) + +await test('unified acceptance: on-demand after loading equals the full first round (loader excluded)', () => withTempDir('full-compare', async cwd => { + const fullSet = await firstRound({ disclosurePolicy: 'full', allowedTools: ['grep', 'stat'] }, 'ops_grep') + assert.deepEqual(fullSet, ROW_T) + fs.writeFileSync(path.join(cwd, 'fixture.txt'), 'needle in haystack\n') + const { ctx, agent } = await setup(cwd, { allowedTools: ['grep', 'stat'] }) + try { + await waitFor(() => opsNames(ctx, agent).includes('ops_load_sdk'), { what: 'ops_load_sdk published' }) + await wait(100) + assert.deepEqual(opsNames(ctx, agent), ['ops_inspect_local_file', 'ops_load_sdk'], 'on-demand starts at inspect + loader before any load') + const value = await program(ctx, agent, "const picked = await tools.ops_load_sdk({ tools: ['grep', 'stat', 'result'] })\n" + + "return { picked: picked.status, earlyGrep: typeof tools.ops_grep }", 'Load the same business set') + assert.equal(value.picked, 'completed', JSON.stringify(value)) + assert.equal(value.earlyGrep, 'undefined', 'a loaded API is callable only in the next program (#979 §1)') + await step(ctx, agent) + await wait(100) + const loaded = opsNames(ctx, agent).filter(name => name !== 'ops_load_sdk') + assert.deepEqual(loaded, ROW_T, 'on-demand loaded set = full first-round set, excluding the loader itself') + assert.deepEqual(loaded, fullSet) + } finally { await shutdown(ctx) } +})) + +await test('read-only excludes write APIs from the full first round', () => withTempDir('full-readonly', async cwd => { + fs.writeFileSync(path.join(cwd, 'fixture.txt'), 'needle in haystack\n') + const { ctx, agent } = await setup(cwd, { disclosurePolicy: 'full', allowedTools: ['write_file', 'stat'] }, {}, 'read-only') + try { + await waitFor(() => opsNames(ctx, agent).includes('ops_stat'), { what: 'ops_stat published' }) + await wait(100) + assert.deepEqual(opsNames(ctx, agent), ['ops_inspect_local_file', 'ops_result', 'ops_stat'], 'the mutating member of the cap is never published') + const value = await program(ctx, agent, "const stat = await tools.ops_stat({ path: 'fixture.txt' })\n" + + "return { statStatus: stat.status, writer: typeof tools.ops_write_file }", 'Stat under read-only') + assert.equal(value.statStatus, 'completed', JSON.stringify(value)) + assert.equal(value.writer, 'undefined', 'switching tiers never widens permissions (#979 §2)') + } finally { await shutdown(ctx) } +})) + +await test('an empty allowed pool keeps the full tier at the resident surface plus result', () => withTempDir('full-empty', async cwd => { + const { ctx, agent } = await setup(cwd, { disclosurePolicy: 'full', allowedTools: [] }) + try { + await waitFor(() => opsNames(ctx, agent).includes('ops_inspect_local_file'), { what: 'ops_inspect_local_file published' }) + await wait(100) + assert.deepEqual(opsNames(ctx, agent), ['ops_inspect_local_file', 'ops_result'], 'nothing else to publish, and still no loader') + const value = await program(ctx, agent, "return { result: typeof tools.ops_result, grep: typeof tools.ops_grep }", 'Probe the empty pool') + assert.equal(value.result, 'function', 'result stays consumable so spilled refs never dead-end (#979 §3)') + assert.equal(value.grep, 'undefined') + } finally { await shutdown(ctx) } +})) + +await test('residentTools:false drops the resident set but keeps the explicit full tier', () => withTempDir('full-resident-off', async cwd => { + fs.writeFileSync(path.join(cwd, 'fixture.txt'), 'needle in haystack\n') + const { ctx, agent } = await setup(cwd, { disclosurePolicy: 'full', allowedTools: ['grep', 'stat'], residentTools: false }) + try { + await waitFor(() => opsNames(ctx, agent).includes('ops_grep'), { what: 'ops_grep published' }) + await wait(100) + assert.deepEqual(opsNames(ctx, agent), ['ops_grep', 'ops_result', 'ops_stat'], 'the resident conflict never silences the explicit tier') + const value = await program(ctx, agent, "const stat = await tools.ops_stat({ path: 'fixture.txt' })\n" + + "return { statStatus: stat.status, inspect: typeof tools.ops_inspect_local_file }", 'Stat with residents off') + assert.equal(value.statStatus, 'completed', JSON.stringify(value)) + assert.equal(value.inspect, 'undefined', 'residentTools:false still drops the resident surface') + } finally { await shutdown(ctx) } +})) + +await test('duplicate allowed names collapse to one publication each', () => withTempDir('full-duplicates', async cwd => { + fs.writeFileSync(path.join(cwd, 'fixture.txt'), 'needle in haystack\n') + const { ctx, agent } = await setup(cwd, { disclosurePolicy: 'full', allowedTools: ['stat', 'stat', 'grep'] }) + try { + await waitFor(() => opsNames(ctx, agent).includes('ops_grep'), { what: 'ops_grep published' }) + await wait(100) + assert.equal(opsNames(ctx, agent).filter(name => name === 'ops_stat').length, 1, 'a duplicated name registers exactly once') + assert.deepEqual(opsNames(ctx, agent), ROW_T) + const value = await program(ctx, agent, "const stat = await tools.ops_stat({ path: 'fixture.txt' })\nreturn { statStatus: stat.status }", 'Stat once') + assert.equal(value.statStatus, 'completed', JSON.stringify(value)) + } finally { await shutdown(ctx) } +})) + +await test('global dynamicTools:false presses an explicit full row back to the legacy surface', async () => { + // Sign-off item 2, end to end through the shipped row: the bundle switch keeps + // its veto, and the saved tier never overrides it (#979 §4). + const set = await firstRound({ disclosurePolicy: 'full', allowedTools: ['grep', 'stat'] }, 'ops_inspect_local_file', { dynamicTools: false }) + assert.deepEqual(set, ['ops_inspect_local_file'], 'row 2 of the D1 table, letter for letter') + assert.equal(set.includes('ops_grep'), false) + assert.equal(set.includes('ops_stat'), false) + assert.equal(set.includes('ops_load_sdk'), false) +}) + +await test('a full row that also turns its loader off keeps the explicit tier', () => withTempDir('full-loader-off', async cwd => { + fs.writeFileSync(path.join(cwd, 'fixture.txt'), 'needle in haystack\n') + const { ctx, agent } = await setup(cwd, { disclosurePolicy: 'full', dynamicTools: false, allowedTools: ['grep', 'stat'] }) + try { + await waitFor(() => opsNames(ctx, agent).includes('ops_grep'), { what: 'ops_grep published' }) + await wait(100) + assert.deepEqual(opsNames(ctx, agent), ROW_T, 'row-level dynamicTools keeps its loader-only meaning') + assert.equal(opsNames(ctx, agent).includes('ops_load_sdk'), false) + const value = await program(ctx, agent, "const grep = await tools.ops_grep({ pattern: 'needle', path: '.' })\n" + + "return { grepStatus: grep.status, grepText: JSON.stringify(grep) }", 'Grep without a loader') + assert.equal(value.grepStatus, 'completed', JSON.stringify(value)) + assert.match(value.grepText, /needle/) + } finally { await shutdown(ctx) } +})) + +report('ptc-full-disclosure') diff --git a/test/ptc-guidance-cleanup.test.mjs b/test/ptc-guidance-cleanup.test.mjs new file mode 100644 index 00000000..903c377d --- /dev/null +++ b/test/ptc-guidance-cleanup.test.mjs @@ -0,0 +1,194 @@ +/** P4 (#983): hiding official file tools must also retire guidance that still names them. */ +import Subprocess from '@deepseek-ai/dsh-subprocess-local' +import Sandbox from '@deepseek-ai/dsh-sandbox-local' +import FileSystem from '@deepseek-ai/dsh-fs-local' +import NodePtc from '@deepseek-ai/dsh-ptc-runtime-node' +import * as FsTools from '@deepseek-ai/dsh-tool-fs' +import * as SearchTools from '@deepseek-ai/dsh-tool-fs-search' +import { createScope } from '@deepseek-ai/dsh-scope' +import * as OpsPtc from '../lib/components/mode-ops-ptc/index.js' +import { bootHost, shutdown } from './lib/host.mjs' +import { selectExecutor, prepareStep, executeCall } from './lib/executor.mjs' +import { WORKER, requireWorker } from './lib/boundary.mjs' +import { assert, test, report, withTempDir } from './lib/harness.mjs' +process.env.DSH_OPS_LEGACY_ROWS = '1' +requireWorker('ptc-guidance-cleanup') + +const REPLACED = ['read', 'glob', 'grep'] +// Pinned provider texts (@deepseek-ai/dsh-tool-fs / -fs-search): their own guidance +// names its tools, so a scope that hides a name must not keep teaching it. +const DEAD_POINTER = /Use the read tool|Use the glob tool|Use the grep tool|Use read on a matched file/ + +let serial = 0 +/** The real provider pair, not fixture definitions: their tool:* prompt sections are the subject. */ +async function fixture(cwd, options, body) { + const { ctx } = await bootHost() + try { + await ctx.plugin(Subprocess); await ctx.plugin(Sandbox); await ctx.plugin(FileSystem); await ctx.plugin(NodePtc) + ctx.provide('sandboxPolicy', { resolve: () => ({ mode: 'danger-full-access', workspaceRoot: cwd, sessionId: 'cleanup' }) }) + ctx.provide('agentPresets', { composedPreset: () => undefined }) + await ctx.plugin(FsTools, { read: true, readImage: false, applyPatch: false }) + await ctx.plugin(SearchTools, { sampleOverCapGlobResults: false }) + const agents = [] + ctx.provide('agents', { list: () => agents }) + const row = await ctx.plugin(OpsPtc, + { required: true, binaryPath: WORKER, bash: false, batchingGuidance: false, ...options }) + const agent = async (presentation = 'ptc', parent) => { + const id = `cleanup-${++serial}` + const value = { id, session: { id, mode: 'danger-full-access', + header: { cwd, agentPreset: presentation === 'ptc' ? 'ops-ptc' : 'native' }, append() {}, read() { return [] } } } + value.ctx = createScope(parent?.ctx ?? ctx, value).ctx + await value.ctx.plugin({ inject: ['tools', 'systemPrompt'], apply(inner) { selectExecutor(inner, presentation) } }) + agents.push(value) + await ctx.serial('agent/created', { agent: value }) + return value + } + await body({ ctx, row, agent }) + } finally { await shutdown(ctx) } +} + +async function capture(ctx, owner) { + const step = await prepareStep(ctx, owner) + try { + const byName = new Map(step.assembly.sections.map(section => [section.name, section.text])) + return { + admitted: ctx.get('tools').definitions(owner, step.toolSet).map(tool => tool.name), + sdk: byName.get('tools:sdk') ?? '', + section: name => byName.get(name), + texts: step.assembly.sections.filter(section => section.text).map(section => section.text), + wire: JSON.stringify(step.assembly.tools), + } + } finally { step.dispose() } +} +// Poll the observable fact instead of sleeping: the replacement shadow registers in its own async inject. +const waitHidden = async (ctx, owner, names) => { + const deadline = Date.now() + 5000 + while (true) { + const step = await prepareStep(ctx, owner) + let admitted + try { admitted = ctx.get('tools').definitions(owner, step.toolSet).map(tool => tool.name) } finally { step.dispose() } + if (names.every(name => !admitted.includes(name))) return + if (Date.now() > deadline) throw new Error('timed out waiting for hidden ' + names.join(', ')) + await new Promise(resolve => setTimeout(resolve, 50)) + } +} +const waitPublish = async (ctx, owner, hidden = []) => { + const deadline = Date.now() + 5000 + while (!ctx.get('tools').definitions(owner).some(tool => tool.name.startsWith('ops_'))) { + if (Date.now() > deadline) throw new Error('timed out waiting for PTC publication') + await new Promise(resolve => setTimeout(resolve, 50)) + } + if (hidden.length) await waitHidden(ctx, owner, hidden) +} +const waitRestored = async (ctx, owner, names) => { + const deadline = Date.now() + 5000 + for (;;) { + const step = await prepareStep(ctx, owner) + try { + const admitted = ctx.get('tools').definitions(owner, step.toolSet).map(tool => tool.name) + const sdk = step.assembly.sections.find(section => section.name === 'tools:sdk')?.text ?? '' + const declared = name => new RegExp('(?:^|\\n)\\s*' + name + ':', 'u').test(sdk) + if (names.every(name => admitted.includes(name) && declared(name))) return + } finally { step.dispose() } + if (Date.now() > deadline) throw new Error('timed out waiting for restored ' + names.join(', ')) + await new Promise(resolve => setTimeout(resolve, 50)) + } +} + +await test('hidden read, glob and grep retire the provider guidance that names them', () => withTempDir('cleanup-full', async cwd => fixture(cwd, {}, async ({ ctx, agent }) => { + const owner = await agent() + await waitPublish(ctx, owner, REPLACED) + const c = await capture(ctx, owner) + // One step carries the SDK declaration, the prompt guidance and the admitted surface together. + for (const name of REPLACED) { + assert.equal(c.admitted.includes(name), false, name + ' must leave the admitted surface') + assert.doesNotMatch(c.sdk, new RegExp('(?:^|\\n)\\s*' + name + ':', 'u'), name + ' must leave the SDK declaration') + assert.equal(c.section('tool:' + name), undefined, 'tool:' + name + ' guidance must not reach a scope that hides it') + } + for (const text of c.texts) assert.doesNotMatch(text, DEAD_POINTER, 'a kept section still teaches a hidden official tool') +}))) + +await test('a kept grep keeps its guidance while the hidden read gains an executable alternative', () => withTempDir('cleanup-partial', async cwd => fixture(cwd, { dynamicTools: false }, async ({ ctx, agent }) => { + const owner = await agent() + await waitPublish(ctx, owner, ['read']) + const c = await capture(ctx, owner) + assert.equal(c.admitted.includes('read'), false, 'resident inspect still replaces read') + for (const name of ['glob', 'grep']) { + assert.ok(c.admitted.includes(name), name + ' stays official without a loader') + assert.match(c.sdk, new RegExp('(?:^|\\n)\\s*' + name + ':', 'u'), name + ' keeps its SDK declaration') + } + assert.match(c.section('tool:glob'), /Use the glob tool/, 'the kept glob guidance survives untouched') + assert.equal(c.section('tool:read'), undefined, 'tool:read must not reach a scope that hides read') + // The host grep text cross-references read; its verbatim survival would keep the dead pointer. + assert.equal(c.section('tool:grep'), undefined, 'the grep cross-reference to the hidden read must not survive verbatim') + const alternative = c.texts.find(text => /ops_inspect_local_file/.test(text) && /\bread\b/.test(text)) + assert.ok(alternative, 'the hidden read keeps an executable alternative in the assembly') + // Same step, program bindings: the kept official stays callable, the hidden one is unbound. + const step = await prepareStep(ctx, owner) + try { + const run = await executeCall(ctx, { agent: owner, toolSet: step.toolSet, name: 'run_code', callId: 'cleanup-partial', + arguments: { description: 'Probe the program bindings', code: 'return { read: typeof tools.read, grep: typeof tools.grep }' }, + signal: new AbortController().signal }) + assert.equal(run.isError, false, JSON.stringify(run)) + assert.deepEqual(run.value.result, { read: 'undefined', grep: 'function' }) + } finally { step.dispose() } +}))) + +await test('restoring the official entry points restores their guidance verbatim', () => withTempDir('cleanup-restore', async cwd => fixture(cwd, {}, async ({ ctx, row, agent }) => { + const owner = await agent() + await waitPublish(ctx, owner, REPLACED) + await row.dispose() + await waitRestored(ctx, owner, REPLACED) + const c = await capture(ctx, owner) + assert.match(c.section('tool:read'), /Use the read tool/, 'restored read guidance returns with the official entry') + assert.match(c.section('tool:grep'), /Use read on a matched file/, 'the grep cross-reference returns once read is callable again') + assert.equal(c.texts.find(text => /Official .* hidden in this scope/.test(text)), undefined, 'no replacement note remains') +}))) + +await test('the provider guidance stays complete while hiding is off', () => withTempDir('cleanup-off', async cwd => fixture(cwd, { hideOfficialTools: false }, async ({ ctx, agent }) => { + const owner = await agent() + await waitPublish(ctx, owner) + const c = await capture(ctx, owner) + assert.match(c.section('tool:read'), /Use the read tool/) + assert.match(c.section('tool:glob'), /Use the glob tool/) + assert.match(c.section('tool:grep'), /Use the grep tool/) + assert.match(c.section('tool:grep'), /Use read on a matched file/) +}))) + +await test('a native child of a hidden PTC parent keeps the official guidance', () => withTempDir('cleanup-tree', async cwd => fixture(cwd, {}, async ({ ctx, agent }) => { + const parent = await agent() + await waitPublish(ctx, parent, REPLACED) + const child = await agent('native', parent) + const c = await capture(ctx, child) + for (const name of REPLACED) assert.ok(c.admitted.includes(name), name + ' stays visible to a native child') + assert.match(c.section('tool:read'), /Use the read tool/, 'native guidance matches the native admission') +}))) + +await test('byte deltas stay grouped: SDK, provider guidance and the outer wire', () => withTempDir('cleanup-bytes', async cwd => { + const groups = c => { + const guidance = REPLACED.map(name => 'tool:' + name).reduce((sum, name) => sum + Buffer.byteLength(c.section(name) ?? ''), 0) + const all = c.texts.reduce((sum, text) => sum + Buffer.byteLength(text), 0) + return { sdk: Buffer.byteLength(c.sdk), guidance, otherSections: all - Buffer.byteLength(c.sdk) - guidance, wire: Buffer.byteLength(c.wire) } + } + let hidden, off + await fixture(cwd, {}, async ({ ctx, agent }) => { + const owner = await agent() + await waitPublish(ctx, owner, REPLACED) + hidden = groups(await capture(ctx, owner)) + }) + await fixture(cwd, { hideOfficialTools: false }, async ({ ctx, agent }) => { + const owner = await agent() + await waitPublish(ctx, owner) + off = groups(await capture(ctx, owner)) + }) + const delta = {} + for (const key of Object.keys(hidden)) delta[key] = hidden[key] - off[key] + // Relative deltas only (V2 convention); each group counted once, the SDK never re-billed as wire. + console.log(' byte deltas hidden - off (sections tool:read, tool:glob, tool:grep): ' + JSON.stringify({ hidden, off, delta })) + assert.ok(delta.sdk < 0, 'hiding shrinks the SDK declaration') + assert.equal(hidden.guidance, 0, 'the hidden names contribute no provider guidance bytes') + assert.ok(off.guidance > 0, 'the pinned provider guidance is non-empty when kept') + assert.equal(delta.wire, 0, 'the outer wire carries only run_code in both states') +})) + +report('ptc-guidance-cleanup') diff --git a/test/ptc-hide-official.test.mjs b/test/ptc-hide-official.test.mjs new file mode 100644 index 00000000..8fec7cde --- /dev/null +++ b/test/ptc-hide-official.test.mjs @@ -0,0 +1,372 @@ +/** hideOfficialTools removes the official file tools a PTC projection replaces. */ +import Subprocess from '@deepseek-ai/dsh-subprocess-local' +import Sandbox from '@deepseek-ai/dsh-sandbox-local' +import FileSystem from '@deepseek-ai/dsh-fs-local' +import NodePtc from '@deepseek-ai/dsh-ptc-runtime-node' +import { createScope } from '@deepseek-ai/dsh-scope' +import * as OpsPtc from '../lib/components/mode-ops-ptc/index.js' +import * as Ptc from '../lib/components/mode-ptc/index.js' +import { bootHost, shutdown } from './lib/host.mjs' +import { selectExecutor, prepareStep, executeCall } from './lib/executor.mjs' +import { mountReplacements } from '../lib/publication/replacements.js' +import { integration } from '../lib/integration/index.js' +import { WORKER, requireWorker } from './lib/boundary.mjs' +import { assert, test, report, withTempDir } from './lib/harness.mjs' +process.env.DSH_OPS_LEGACY_ROWS = '1' +requireWorker('ptc-hide-official') + +const OFFICIAL = ['read', 'glob', 'grep', 'edit', 'write', 'read_image'] +const REPLACED = ['read', 'glob', 'grep'] +// The value must be an object: PTC SDK bindings reject non-object canonical values. +const definition = name => ({ name, description: `fixture ${name}`, parameters: { type: 'object', properties: {} }, + output: { schema: { type: 'object' }, render: () => [] }, execute: async () => ({ official: name }) }) + +/** + * Argument schemas mirroring the real host official file tools' shapes. The + * empty-schema fixtures above only prove membership changes; comparing the SDK + * through these makes the kept entries a schema-level match. + */ +const REAL_PARAMETERS = { + read: { type: 'object', additionalProperties: false, required: ['file_path'], properties: { + file_path: { type: 'string', description: 'Path to the file, relative to the working directory' }, + offset: { type: 'number', description: '1-based first line to return. Defaults to 1.' }, + limit: { type: 'number', description: 'Maximum number of lines to return. Defaults to 2000.' } } }, + glob: { type: 'object', additionalProperties: false, required: ['pattern'], properties: { + pattern: { type: 'string', description: 'Glob pattern to match paths against' }, + path: { type: 'string', description: 'Directory to search in. Defaults to the session workspace' } } }, + grep: { type: 'object', additionalProperties: false, required: ['pattern'], properties: { + pattern: { type: 'string', description: 'Regular expression to search for (ripgrep syntax)' }, + path: { type: 'string', description: 'File or directory to search. Defaults to the session workspace' }, + include: { type: 'string', description: 'One glob filter for which files to search' } } }, + write: { type: 'object', additionalProperties: false, required: ['file_path', 'content'], properties: { + file_path: { type: 'string', description: 'Path to write, relative to the working directory' }, + content: { type: 'string', description: 'Full UTF-8 text content to write' } } }, + edit: { type: 'object', additionalProperties: false, required: ['file_path', 'old_string', 'new_string'], properties: { + file_path: { type: 'string', description: 'Path to edit, relative to the working directory' }, + old_string: { type: 'string', description: 'Literal text to replace' }, + new_string: { type: 'string', description: 'Literal replacement text' }, + replace_all: { type: 'boolean', description: 'Replace all matches. Defaults to false' } } }, + read_image: { type: 'object', additionalProperties: false, required: ['file_path'], properties: { + file_path: { type: 'string', description: 'Path to the image file' } } }, +} +const realDefinition = name => ({ name, description: `fixture ${name}`, parameters: REAL_PARAMETERS[name], + output: { schema: { type: 'object' }, render: () => [] }, execute: async () => ({ official: name }) }) +/** One named entry of the tools:sdk declaration, or '' when the name is absent. */ +const sdkBlock = (sdk, name) => { + const match = new RegExp('(?:^|\\n)\\s*' + name + ':\\s*\\{([\\s\\S]*?)\\n\\s*\\};', 'u').exec(sdk) + return match === null ? '' : match[1] +} + +let serial = 0 +async function fixture(cwd, mode, options, body, make = definition) { + const { ctx } = await bootHost() + try { + await ctx.plugin(Subprocess); await ctx.plugin(Sandbox); await ctx.plugin(FileSystem); await ctx.plugin(NodePtc) + ctx.provide('sandboxPolicy', { resolve: () => ({ mode: 'danger-full-access', workspaceRoot: cwd, sessionId: 'hide' }) }) + ctx.provide('agentPresets', { composedPreset: () => undefined }) + for (const name of OFFICIAL) ctx.tools.register(make(name)) + const agents = [] + ctx.provide('agents', { list: () => agents }) + const row = await ctx.plugin(mode === 'ops-ptc' ? OpsPtc : Ptc, + { required: true, binaryPath: WORKER, bash: false, batchingGuidance: false, ...options }) + const agent = async (presentation = 'ptc', parent) => { + const id = `hide-${++serial}` + const value = { id, session: { id, mode: 'danger-full-access', + header: { cwd, agentPreset: presentation === 'ptc' ? mode : 'native' }, append() {}, read() { return [] } } } + value.ctx = createScope(parent?.ctx ?? ctx, value).ctx + await value.ctx.plugin({ inject: ['tools', 'systemPrompt'], apply(inner) { selectExecutor(inner, presentation) } }) + agents.push(value) + await ctx.serial('agent/created', { agent: value }) + return value + } + await body({ ctx, row, agent }) + } finally { await shutdown(ctx) } +} + +async function capture(ctx, owner) { + const step = await prepareStep(ctx, owner) + try { + const sdk = step.assembly.sections.find(section => section.name === 'tools:sdk')?.text ?? '' + return { admitted: ctx.get('tools').definitions(owner, step.toolSet).map(tool => tool.name), sdk } + } finally { step.dispose() } +} +const has = (names, name) => names.includes(name) +// Poll the observable fact instead of sleeping: the replacement shadow registers in its own async inject. +const waitHidden = async (ctx, owner, names) => { + const deadline = Date.now() + 5000 + while (true) { + const step = await prepareStep(ctx, owner) + let admitted + try { admitted = ctx.get('tools').definitions(owner, step.toolSet).map(tool => tool.name) } finally { step.dispose() } + if (names.every(name => !admitted.includes(name))) return + if (Date.now() > deadline) throw new Error('timed out waiting for hidden ' + names.join(', ')) + await new Promise(resolve => setTimeout(resolve, 50)) + } +} +/** Poll the observable fact that withdrawn replacements gave the names back. */ +const waitRestored = async (ctx, owner, names) => { + const deadline = Date.now() + 5000 + for (;;) { + const step = await prepareStep(ctx, owner) + try { + const admitted = ctx.get('tools').definitions(owner, step.toolSet).map(tool => tool.name) + const sdk = step.assembly.sections.find(section => section.name === 'tools:sdk')?.text ?? '' + const declared = name => new RegExp('(?:^|\\n)\\s*' + name + ':', 'u').test(sdk) + if (names.every(name => admitted.includes(name) && declared(name))) return { admitted, sdk } + } finally { step.dispose() } + if (Date.now() > deadline) throw new Error('timed out waiting for restored ' + names.join(', ')) + await new Promise(resolve => setTimeout(resolve, 50)) + } +} +const waitPublish = async (ctx, owner, hidden = []) => { + const deadline = Date.now() + 5000 + while (!ctx.get('tools').definitions(owner).some(tool => tool.name.startsWith('ops_'))) { + if (Date.now() > deadline) throw new Error('timed out waiting for PTC publication') + await new Promise(resolve => setTimeout(resolve, 50)) + } + if (hidden.length) await waitHidden(ctx, owner, hidden) +} + +await test('ops-ptc hides the replaced official file tools and keeps the rest', () => withTempDir('hide-default', async cwd => fixture(cwd, 'ops-ptc', {}, async ({ ctx, agent }) => { + const owner = await agent() + await waitPublish(ctx, owner, REPLACED) + const c = await capture(ctx, owner) + for (const name of REPLACED) { + assert.equal(has(c.admitted, name), false, name + ' must leave the admitted surface') + assert.doesNotMatch(c.sdk, new RegExp('(?:^|\\n)\\s*' + name + ':', 'u'), name + ' must leave the SDK declaration') + } + for (const name of ['edit', 'write', 'read_image']) assert.ok(has(c.admitted, name), name + ' stays visible') +}))) + +await test('hideOfficialTools off keeps every official tool', () => withTempDir('hide-off', async cwd => fixture(cwd, 'ops-ptc', { hideOfficialTools: false }, async ({ ctx, agent }) => { + const owner = await agent() + await waitPublish(ctx, owner) + const c = await capture(ctx, owner) + for (const name of OFFICIAL) assert.ok(has(c.admitted, name), name + ' stays with the switch off') +}))) + +await test('dynamicTools off keeps glob and grep while read stays replaced', () => withTempDir('hide-dynamic-off', async cwd => fixture(cwd, 'ops-ptc', { dynamicTools: false }, async ({ ctx, agent }) => { + const owner = await agent() + await waitPublish(ctx, owner, ['read']) + const c = await capture(ctx, owner) + assert.equal(has(c.admitted, 'grep'), true, 'no loader means grep is unreachable') + assert.equal(has(c.admitted, 'glob'), true, 'no loader means glob is unreachable') + assert.match(c.sdk, /(?:^|\n)\s*grep:/u, 'the kept official grep is still declared') + assert.match(c.sdk, /(?:^|\n)\s*glob:/u, 'the kept official glob is still declared') + assert.equal(has(c.admitted, 'read'), false, 'resident inspect_local_file still replaces read') + assert.doesNotMatch(c.sdk, /(?:^|\n)\s*read:/u, 'read leaves the SDK while resident') +}))) + +await test('a narrowed allowedTools pool keeps the official names it excludes', () => withTempDir('hide-narrow', async cwd => fixture(cwd, 'ops-ptc', { allowedTools: ['stat'] }, async ({ ctx, agent }) => { + const owner = await agent() + await waitPublish(ctx, owner, ['read']) + const c = await capture(ctx, owner) + assert.equal(has(c.admitted, 'grep'), true, 'grep is outside the configured pool') + assert.equal(has(c.admitted, 'glob'), true, 'glob is outside the configured pool') +}))) + +await test('residentTools off with a narrow pool keeps every replaced official name', () => withTempDir('hide-resident-off', async cwd => fixture(cwd, 'ops-ptc', { residentTools: false, allowedTools: ['stat'] }, async ({ ctx, agent }) => { + const owner = await agent() + await waitPublish(ctx, owner) + const c = await capture(ctx, owner) + for (const name of REPLACED) assert.ok(has(c.admitted, name), name + ' has no reachable counterpart') +}))) + +await test('loading the SDK does not resurrect the official tools', () => withTempDir('hide-load', async cwd => fixture(cwd, 'ops-ptc', {}, async ({ ctx, agent }) => { + const owner = await agent() + await waitPublish(ctx, owner, REPLACED) + const step = await prepareStep(ctx, owner) + try { + const run = await executeCall(ctx, { agent: owner, toolSet: step.toolSet, name: 'run_code', callId: 'hide-load', + arguments: { description: 'Load grep', code: "return await tools.ops_load_sdk({tools:['grep']})" }, + signal: new AbortController().signal }) + assert.equal(run.isError, false, JSON.stringify(run)) + } finally { step.dispose() } + ctx.emit('session/event', owner.session, { type: 'step/end', turn: 1, step: 1 }) + const after = await capture(ctx, owner) + assert.equal(has(after.admitted, 'grep'), false, 'selection must not resurrect the official grep') + assert.ok(has(after.admitted, 'ops_grep'), 'selected OPS grep becomes callable') +}))) + +await test('a native child of a PTC parent keeps the official file tools', () => withTempDir('hide-tree', async cwd => fixture(cwd, 'ops-ptc', {}, async ({ ctx, agent }) => { + const parent = await agent('ptc') + await waitPublish(ctx, parent, REPLACED) + const child = await agent('native', parent) + const c = await capture(ctx, child) + for (const name of REPLACED) assert.ok(has(c.admitted, name), name + ' stays visible to a native child') +}))) + +await test('a failing presentation probe keeps the official tools', () => withTempDir('hide-probe-fail', async cwd => { + const { ctx } = await bootHost() + try { + await ctx.plugin(Subprocess); await ctx.plugin(Sandbox); await ctx.plugin(FileSystem); await ctx.plugin(NodePtc) + ctx.provide('sandboxPolicy', { resolve: () => ({ mode: 'danger-full-access', workspaceRoot: cwd, sessionId: 'probe' }) }) + ctx.provide('agentPresets', { composedPreset: () => undefined }) + for (const name of OFFICIAL) ctx.tools.register(definition(name)) + const agents = [] + ctx.provide('agents', { list: () => agents }) + const warnings = [] + ctx.logger.exporter({ levels: { default: 4 }, export(message) { if (message.type === 'warn') warnings.push(message.args.join(' ')) } }) + const owner = { id: 'probe-fail', session: { id: 'probe-fail', mode: 'danger-full-access', header: { cwd }, append() {}, read() { return [] } } } + owner.ctx = createScope(ctx, owner).ctx + await owner.ctx.plugin({ inject: ['tools', 'systemPrompt'], apply(inner) { selectExecutor(inner, 'ptc') } }) + agents.push(owner) + await ctx.serial('agent/created', { agent: owner }) + const targets = new Map([['read', 'ops_inspect_local_file'], ['glob', 'ops_glob'], ['grep', 'ops_grep']]) + let probes = 0 + await ctx.plugin({ apply(inner) { + mountReplacements(inner, { id: 'probe-fail', targets, gate: { allows: () => true }, keepOfficial: () => false, + ptc: { servable: () => new Set(['inspect_local_file', 'glob', 'grep']), isPtc: async () => { probes++; throw new Error('probe failed') } } }) + } }) + // The controller attaches through an async inject callback; let it register before resolving. + await new Promise(resolve => setImmediate(resolve)) + await new Promise(resolve => setImmediate(resolve)) + const step = await prepareStep(ctx, owner) + try { + const admitted = ctx.get('tools').definitions(owner, step.toolSet).map(tool => tool.name) + for (const name of REPLACED) assert.ok(admitted.includes(name), name + ' stays visible when the probe fails') + const failed = warnings.filter(line => line.includes('PTC presentation probe failed')) + assert.ok(probes >= 1, 'the failing probe was actually consulted') + assert.equal(failed.length, 1, 'the failure is reported once') + } finally { step.dispose() } + } finally { await shutdown(ctx) } +})) + +await test('the integration entry hides the replaced official file tools under PTC presentation', () => withTempDir('hide-integration', async cwd => { + const { ctx } = await bootHost() + try { + await ctx.plugin(Subprocess); await ctx.plugin(Sandbox); await ctx.plugin(FileSystem); await ctx.plugin(NodePtc) + ctx.provide('sandboxPolicy', { resolve: () => ({ mode: 'danger-full-access', workspaceRoot: cwd, sessionId: 'hide-int' }) }) + ctx.provide('agentPresets', { composedPreset: () => undefined }) + for (const name of OFFICIAL) ctx.tools.register(definition(name)) + const agents = [] + ctx.provide('agents', { list: () => agents }) + await ctx.plugin(integration({ preset: 'hide-int', profile: 'ptc', promptPolicy: false, + replaceOfficial: true, binaryPath: WORKER, required: true })) + const owner = { id: 'hide-int', session: { id: 'hide-int', mode: 'danger-full-access', header: { cwd, agentPreset: 'hide-int' }, append() {}, read() { return [] } } } + owner.ctx = createScope(ctx, owner).ctx + await owner.ctx.plugin({ inject: ['tools', 'systemPrompt'], apply(inner) { selectExecutor(inner, 'ptc') } }) + agents.push(owner) + await ctx.serial('agent/created', { agent: owner }) + await waitPublish(ctx, owner, REPLACED) + const c = await capture(ctx, owner) + for (const name of REPLACED) assert.equal(c.admitted.includes(name), false, name + ' is hidden through the integration entry') + } finally { await shutdown(ctx) } +})) + +await test('an integration preset keeps every official file tool until it opts in', () => withTempDir('hide-preset-default', async cwd => { + const { ctx } = await bootHost() + try { + await ctx.plugin(Subprocess); await ctx.plugin(Sandbox); await ctx.plugin(FileSystem); await ctx.plugin(NodePtc) + ctx.provide('sandboxPolicy', { resolve: () => ({ mode: 'danger-full-access', workspaceRoot: cwd, sessionId: 'hide-default' }) }) + ctx.provide('agentPresets', { composedPreset: () => undefined }) + for (const name of OFFICIAL) ctx.tools.register(definition(name)) + const agents = [] + ctx.provide('agents', { list: () => agents }) + // No replaceOfficial: the default for a custom preset retains official tools. + await ctx.plugin(integration({ preset: 'hide-default', profile: 'ptc', promptPolicy: false, binaryPath: WORKER, required: true })) + const owner = { id: 'hide-default', session: { id: 'hide-default', mode: 'danger-full-access', header: { cwd, agentPreset: 'hide-default' }, append() {}, read() { return [] } } } + owner.ctx = createScope(ctx, owner).ctx + await owner.ctx.plugin({ inject: ['tools', 'systemPrompt'], apply(inner) { selectExecutor(inner, 'ptc') } }) + agents.push(owner) + await ctx.serial('agent/created', { agent: owner }) + const deadline = Date.now() + 5000 + let c + for (;;) { + c = await capture(ctx, owner) + // Wait for the publisher to be fully attached: a kept official name before + // attachment would prove nothing about the default. + if (c.admitted.includes('ops_inspect_local_file') && c.admitted.includes('ops_load_sdk')) break + if (Date.now() > deadline) throw new Error('timed out waiting for the PTC publication') + await new Promise(resolve => setTimeout(resolve, 50)) + } + for (const name of OFFICIAL) assert.ok(has(c.admitted, name), name + ' stays without an explicit opt-in') + } finally { await shutdown(ctx) } +})) + +await test('disposing the mode row restores the official file tools and they stay callable', () => withTempDir('hide-restore', async cwd => fixture(cwd, 'ops-ptc', {}, async ({ ctx, row, agent }) => { + const owner = await agent() + await waitPublish(ctx, owner, REPLACED) + await row.dispose() + const back = await waitRestored(ctx, owner, REPLACED) + for (const name of REPLACED) assert.ok(has(back.admitted, name), name + ' returns to the admitted surface') + const step = await prepareStep(ctx, owner) + try { + const run = await executeCall(ctx, { agent: owner, toolSet: step.toolSet, name: 'run_code', callId: 'hide-restore', + arguments: { description: 'Call the restored official tools', code: 'return { read: await tools.read({}), glob: await tools.glob({}), grep: await tools.grep({}) }' }, + signal: new AbortController().signal }) + assert.equal(run.isError, false, JSON.stringify(run)) + assert.deepEqual(run.value.result, { read: { official: 'read' }, glob: { official: 'glob' }, grep: { official: 'grep' } }) + } finally { step.dispose() } +}))) + +await test('an unattached projection keeps the official file tools callable', () => withTempDir('hide-unattached', async cwd => { + const { ctx } = await bootHost() + try { + await ctx.plugin(Subprocess); await ctx.plugin(Sandbox); await ctx.plugin(FileSystem); await ctx.plugin(NodePtc) + ctx.provide('sandboxPolicy', { resolve: () => ({ mode: 'danger-full-access', workspaceRoot: cwd, sessionId: 'unattached' }) }) + ctx.provide('agentPresets', { composedPreset: () => undefined }) + for (const name of OFFICIAL) ctx.tools.register(definition(name)) + const agents = [] + ctx.provide('agents', { list: () => agents }) + const notes = [] + ctx.logger.exporter({ levels: { default: 4 }, export(message) { if (message.type === 'info') notes.push(message.args.join(' ')) } }) + const owner = { id: 'unattached', session: { id: 'unattached', mode: 'danger-full-access', header: { cwd }, append() {}, read() { return [] } } } + owner.ctx = createScope(ctx, owner).ctx + await owner.ctx.plugin({ inject: ['tools', 'systemPrompt'], apply(inner) { selectExecutor(inner, 'ptc') } }) + agents.push(owner) + await ctx.serial('agent/created', { agent: owner }) + const targets = new Map([['read', 'ops_inspect_local_file'], ['glob', 'ops_glob'], ['grep', 'ops_grep']]) + await ctx.plugin({ apply(inner) { + // PTC presentation with an unknown reachability answer: the contract must + // keep the official tools rather than hide on an unattached projection. + mountReplacements(inner, { id: 'unattached', targets, gate: { allows: () => true }, keepOfficial: () => false, + ptc: { servable: () => undefined, isPtc: async () => true } }) + } }) + await new Promise(resolve => setImmediate(resolve)) + await new Promise(resolve => setImmediate(resolve)) + const step = await prepareStep(ctx, owner) + try { + const admitted = ctx.get('tools').definitions(owner, step.toolSet).map(tool => tool.name) + for (const name of REPLACED) assert.ok(admitted.includes(name), name + ' stays visible while the projection is unattached') + assert.ok(notes.some(line => line.includes('for read: unattached')), 'the unattached state is observable') + const run = await executeCall(ctx, { agent: owner, toolSet: step.toolSet, name: 'run_code', callId: 'hide-unattached', + arguments: { description: 'Call the officials kept by an unattached projection', code: 'return { read: await tools.read({}) }' }, + signal: new AbortController().signal }) + assert.equal(run.isError, false, JSON.stringify(run)) + assert.deepEqual(run.value.result, { read: { official: 'read' } }) + } finally { step.dispose() } + } finally { await shutdown(ctx) } +})) + +await test('kept official write, edit and read_image keep their real schemas and stay callable', () => withTempDir('hide-real-schema', async cwd => fixture(cwd, 'ops-ptc', {}, async ({ ctx, agent }) => { + const owner = await agent() + await waitPublish(ctx, owner, REPLACED) + const c = await capture(ctx, owner) + for (const name of REPLACED) { + assert.equal(has(c.admitted, name), false, name + ' must leave the admitted surface') + assert.equal(sdkBlock(c.sdk, name), '', name + ' must leave the SDK declaration') + } + const write = sdkBlock(c.sdk, 'write') + assert.ok(write.length > 0, 'write keeps its SDK entry') + assert.match(write, /file_path: string;/u, 'write keeps its required path argument') + assert.match(write, /content: string;/u, 'write keeps its required content argument') + const edit = sdkBlock(c.sdk, 'edit') + assert.ok(edit.length > 0, 'edit keeps its SDK entry') + assert.match(edit, /old_string: string;/u) + assert.match(edit, /new_string: string;/u) + assert.match(edit, /replace_all\?: boolean;/u, 'edit keeps its optional replace_all argument') + const image = sdkBlock(c.sdk, 'read_image') + assert.ok(image.length > 0, 'read_image keeps its SDK entry') + assert.match(image, /file_path: string;/u) + const step = await prepareStep(ctx, owner) + try { + const run = await executeCall(ctx, { agent: owner, toolSet: step.toolSet, name: 'run_code', callId: 'hide-real-schema', + arguments: { description: 'Call the kept official tools', code: 'return { write: await tools.write({ file_path: \'a.txt\', content: \'x\' }), edit: await tools.edit({ file_path: \'a.txt\', old_string: \'x\', new_string: \'y\' }), image: await tools.read_image({ file_path: \'a.txt\' }) }' }, + signal: new AbortController().signal }) + assert.equal(run.isError, false, JSON.stringify(run)) + assert.deepEqual(run.value.result, { write: { official: 'write' }, edit: { official: 'edit' }, image: { official: 'read_image' } }) + } finally { step.dispose() } +}, realDefinition))) + +report('ptc-hide-official') diff --git a/test/ptc-outcome.test.mjs b/test/ptc-outcome.test.mjs index de5c392d..606c5e33 100644 --- a/test/ptc-outcome.test.mjs +++ b/test/ptc-outcome.test.mjs @@ -100,6 +100,96 @@ await test('host schema validation and generated SDK enforce the discriminated e assert.ok(validateJsonSchemaValue(schema, bad).length > 0) } }) +await test('partial pages recommend the same-path next page and a complete page recommends none', () => { + const args = { path: '/tmp/p6/eleven.txt', offset: 1, limit: 5 } + const partial = { status: 'partial', complete: false, nextOffset: 6, total: 11, items: [{ line: 5, text: 'line-5' }] } + const page1 = toOutcome('inspect_local_file', args, structuredClone(partial)) + assert.equal(page1.status, 'partial') + assert.match(page1.nextAction, /ops_inspect_local_file \{"path":"\/tmp\/p6\/eleven\.txt","offset":6,"limit":5\}/) + assert.match(page1.nextAction, /complete=true/) + assert.deepEqual(validateJsonSchemaValue(outcomeSchema(canonicalOutput('inspect_local_file')), page1), []) + const page2 = toOutcome('inspect_local_file', { ...args, offset: 6 }, { ...structuredClone(partial), nextOffset: 11, items: [{ line: 10, text: 'line-10' }] }) + assert.match(page2.nextAction, /"offset":11/) + const last = toOutcome('inspect_local_file', { ...args, offset: 11 }, { status: 'completed', complete: true, total: 11, items: [{ line: 11, text: 'line-11' }] }) + assert.equal(last.status, 'completed') + assert.equal('nextAction' in last, false, 'a complete page is the stop signal, never another read') +}) +await test('directory rejections recommend only listing APIs callable in this step', () => { + const refused = () => ({ status: 'error', error: { code: 'INVALID_ARGUMENT', effectState: 'none', + message: 'inspect_local_file: path is a directory; choose a file, or list directory entries with the shell', + nextAction: 'Correct the request or inspect the manifest.' } }) + const surface = names => ({ callable: new Set(names), readOnly: false }) + const listed = toOutcome('inspect_local_file', { path: '/tmp/p6/adir' }, refused(), surface(['ops_inspect_local_file', 'ops_glob'])) + assert.equal(listed.status, 'error') + assert.match(listed.error.nextAction, /ops_glob \{"path":"\/tmp\/p6\/adir","pattern":\["\*"\]\}/) + assert.match(listed.error.nextAction, /concrete file path/) + const unloaded = toOutcome('inspect_local_file', { path: '/tmp/p6/adir' }, refused(), surface(['ops_inspect_local_file'])) + assert.doesNotMatch(unloaded.error.nextAction, /ops_glob|ops_grep/) + assert.match(unloaded.error.nextAction, /concrete file path/) + const unknown = toOutcome('inspect_local_file', { path: '/tmp/p6/adir' }, refused()) + assert.doesNotMatch(unknown.error.nextAction, /ops_glob|ops_grep|shell/) + assert.match(unknown.error.nextAction, /concrete file path/) +}) +await test('invalid UTF-8 recommends a parameterized hex view the reader schema accepts', () => { + const refused = () => ({ status: 'error', error: { code: 'ERR_INVALID_ENCODING', effectState: 'none', + message: 'inspect_local_file: File is not valid UTF-8', + nextAction: 'Use a dedicated document/binary reader, or inspect with view=hex; do not replace invalid bytes with lossy text.' } }) + const inspect = toOutcome('inspect_local_file', { path: '/tmp/p6/blob.bin' }, refused()) + assert.equal(inspect.status, 'error') + assert.match(inspect.error.nextAction, /ops_inspect_local_file \{"path":"\/tmp\/p6\/blob\.bin","view":"hex"\}/) + const items = () => ({ status: 'partial', complete: false, nextIndex: 1, failed: 1, + items: [{ path: '/tmp/p6/blob.bin', error: { code: 'ERR_INVALID_ENCODING', effectState: 'none', message: 'File is not valid UTF-8' } }] }) + const pointed = toOutcome('read_files', { files: [{ path: '/tmp/p6/blob.bin' }] }, items(), { callable: new Set(['ops_read_files', 'ops_inspect_local_file']), readOnly: false }) + assert.equal(pointed.status, 'error') + assert.match(pointed.error.nextAction, /ops_inspect_local_file \{"path":"\/tmp\/p6\/blob\.bin","view":"hex"\}/) + const stranded = toOutcome('read_files', { files: [{ path: '/tmp/p6/blob.bin' }] }, items()) + assert.doesNotMatch(stranded.error.nextAction, /"view":"hex"/) + assert.match(stranded.error.nextAction, /not valid UTF-8/) +}) +await test('revision conflicts recommend the concrete fresh-revision read; read-only never re-submits', () => { + const refused = () => ({ status: 'conflict', error: { code: 'REVISION_CONFLICT', effectState: 'none', + message: 'edit_file: Revision differs from expectedRevision; nothing was changed. Inspect before retrying.', + nextAction: 'Inspect the current revision before preparing a new request; do not blindly retry.' } }) + const args = { path: '/tmp/p6/editable.txt', expectedRevision: 'a'.repeat(64), edits: [{ oldText: 'alpha', newText: 'beta' }] } + const writable = toOutcome('edit_file', args, refused(), { callable: new Set(['ops_edit_file', 'ops_stat']), readOnly: false }) + assert.equal(writable.status, 'conflict') + assert.match(writable.error.nextAction, /ops_stat \{"path":"\/tmp\/p6\/editable\.txt"\}/) + assert.match(writable.error.nextAction, /expectedRevision/) + const unknown = toOutcome('edit_file', args, refused()) + assert.match(unknown.error.nextAction, /revision/) + assert.doesNotMatch(unknown.error.nextAction, /do not blindly retry/) + const fenced = toOutcome('edit_file', args, refused(), { callable: new Set(['ops_stat']), readOnly: true }) + assert.doesNotMatch(fenced.error.nextAction, /re-submit|retry/) + assert.match(fenced.error.nextAction, /ops_stat \{"path":"\/tmp\/p6\/editable\.txt"\}/) +}) +await test('read-only scopes never receive a write-class recommendation', () => { + const readOnly = { callable: new Set(['ops_inspect_local_file', 'ops_glob', 'ops_stat', 'ops_read_files']), readOnly: true } + const cases = [ + ['inspect_local_file', { path: '/tmp/p6/adir' }, { status: 'error', error: { code: 'INVALID_ARGUMENT', effectState: 'none', message: 'inspect_local_file: path is a directory; choose a file' } }], + ['inspect_local_file', { path: '/tmp/p6/blob.bin' }, { status: 'error', error: { code: 'ERR_INVALID_ENCODING', effectState: 'none', message: 'inspect_local_file: File is not valid UTF-8' } }], + ['edit_file', { path: '/tmp/p6/e', expectedRevision: 'a'.repeat(64), edits: [{ oldText: 'a', newText: 'b' }] }, { status: 'conflict', error: { code: 'REVISION_CONFLICT', effectState: 'none', message: 'edit_file: Revision differs' } }], + ['inspect_local_file', { path: '/tmp/p6/f', offset: 1, limit: 5 }, { status: 'partial', complete: false, nextOffset: 6, items: [{ line: 5, text: 'x' }] }], + ] + const writes = ['ops_write_file', 'ops_edit_file', 'ops_copy', 'ops_move', 'ops_remove', 'ops_replace', 'ops_batch', 'ops_bash'] + for (const [tool, args, data] of cases) { + const out = toOutcome(tool, args, structuredClone(data), readOnly) + const text = (out.status === 'partial' ? out.nextAction : out.error.nextAction) ?? '' + for (const name of writes) assert.ok(!text.includes(name), tool + ' recommended ' + name) + assert.doesNotMatch(text, /re-submit|overwrite|unlink|delete/i) + } +}) +await test('after-input-change failures say what to change; other worker guidance is preserved', () => { + const invalid = toOutcome('grep', { pattern: '[', path: '/tmp/p6' }, { status: 'error', error: { code: 'INVALID_ARGUMENT', effectState: 'none', message: 'grep: pattern is not a valid regex' } }) + assert.equal(invalid.status, 'error') + assert.equal(invalid.error.retry, 'after-input-change') + assert.match(invalid.error.nextAction, /identical request fails identically/) + assert.doesNotMatch(invalid.error.nextAction, /retry the same|just retry/i) + const advised = toOutcome('grep', { pattern: '[', path: '/tmp/p6' }, { status: 'error', error: { code: 'INVALID_ARGUMENT', effectState: 'none', message: 'grep: pattern is not a valid regex', nextAction: 'Correct the request or inspect the manifest.' } }) + assert.equal(advised.error.nextAction, 'Correct the request or inspect the manifest.') + const kept = toOutcome('copy', { path: '/tmp/p6/s', to: '/tmp/p6/d', expectedRevision: 'a'.repeat(64) }, + { status: 'error', error: { code: 'CLEANUP_FAILED', effectState: 'unknown', message: 'Could not remove staging sibling x', nextAction: 'Inspect the staging sibling and target before retrying; unpublished bytes may remain.' } }) + assert.equal(kept.error.nextAction, 'Inspect the staging sibling and target before retrying; unpublished bytes may remain.') +}) await test('canonical metadata accepts unavailable timestamps without weakening numeric fields', () => { for (const [tool, data] of [ ['stat', { exists: true, modifiedMs: null, bytes: 2 }], diff --git a/test/ptc-projection-worker.test.mjs b/test/ptc-projection-worker.test.mjs index fb76febe..02bccfb0 100644 --- a/test/ptc-projection-worker.test.mjs +++ b/test/ptc-projection-worker.test.mjs @@ -101,6 +101,62 @@ await test('real reference pages follow a bounded cursor to completion and relea assert.equal((await call(root, 'result', { action: 'release', id: legacy.data.id })).status, 'completed') }) }) +await test('paged reads follow the recommended next action to a complete stop', async () => { + await withTempDir('ptc-follow', async root => { + const file = path.join(root, 'eleven.txt') + fs.writeFileSync(file, Array.from({ length: 11 }, (_, n) => 'line-' + (n + 1)).join('\n') + '\n') + let args = { path: file, offset: 1, limit: 5 } + const advanced = [] + for (let page = 0; page < 4; page++) { + const out = await call(root, 'inspect_local_file', args) + if (out.status === 'completed') { + assert.equal(out.data.complete, true) + assert.equal('nextAction' in out, false, 'a complete page recommends no further read') + break + } + assert.equal(out.status, 'partial') + assert.ok(out.data.nextOffset > args.offset) + advanced.push(out.data.nextOffset) + assert.match(out.nextAction, /ops_inspect_local_file/) + assert.match(out.nextAction, new RegExp('"offset":' + out.data.nextOffset + ',')) + const next = decodeContinuation(out.continuation).args + assert.equal(next.path, file) + args = next + } + assert.deepEqual(advanced, [6, 11], '11 lines in limit-5 pages end after three pages') + }) +}) +await test('recoverable read errors name a same-scope next step instead of dead pointers', async () => { + await withTempDir('ptc-recover', async root => { + fs.mkdirSync(path.join(root, 'adir')) + fs.writeFileSync(path.join(root, 'blob.bin'), Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a, 0xff, 0xfe, 0x00, 0x33])) + const dir = await call(root, 'inspect_local_file', { path: path.join(root, 'adir') }) + assert.equal(dir.status, 'error') + assert.equal(dir.error.retry, 'after-input-change') + assert.doesNotMatch(dir.error.nextAction, /ops_glob|ops_grep|shell/, 'no listing API is known callable without a step surface') + assert.match(dir.error.nextAction, /concrete file path/) + const binary = await call(root, 'inspect_local_file', { path: path.join(root, 'blob.bin') }) + assert.equal(binary.status, 'error') + assert.equal(binary.error.code, 'ERR_INVALID_ENCODING') + assert.match(binary.error.nextAction, /ops_inspect_local_file \{"path":/) + assert.match(binary.error.nextAction, /"view":"hex"\}/) + const hex = await call(root, 'inspect_local_file', { path: path.join(root, 'blob.bin'), view: 'hex' }) + assert.ok(['completed','partial'].includes(hex.status)) + assert.equal(hex.data.items[0].view, 'hex') + const batched = await call(root, 'read_files', { files: [{ path: path.join(root, 'blob.bin') }] }) + assert.equal(batched.status, 'error') + assert.equal(batched.data.items[0].error.code, 'ERR_INVALID_ENCODING') + assert.match(batched.error.nextAction, /not valid UTF-8/) + assert.doesNotMatch(batched.error.nextAction, /"view":"hex"/, 'without a step surface no cross-tool call is recommended') + const editable = path.join(root, 'editable.txt') + fs.writeFileSync(editable, 'alpha\n') + const stale = await call(root, 'edit_file', { path: editable, expectedRevision: hash('other\n'), edits: [{ oldText: 'alpha', newText: 'beta' }] }) + assert.equal(stale.status, 'conflict') + assert.match(stale.error.nextAction, /revision/) + assert.doesNotMatch(stale.error.nextAction, /do not blindly retry/) + assert.equal(fs.readFileSync(editable, 'utf8'), 'alpha\n') + }) +}) await test('every worker API was validated and unexpected callback rejection becomes an unknown value', async () => { assert.deepEqual([...seen].sort(), [...PTC_API_TOOLS].sort()) for (const tool of ['result','write_file']) { diff --git a/test/ptc-two-tier-matrix.test.mjs b/test/ptc-two-tier-matrix.test.mjs new file mode 100644 index 00000000..bfe98e85 --- /dev/null +++ b/test/ptc-two-tier-matrix.test.mjs @@ -0,0 +1,373 @@ +/** The #992 data-driven acceptance matrix for the two-tier publication rule. + * + * One kernel, three projections, one comparison. Every row is data: the config + * inputs and the exact business names the row must publish in each view — the + * SDK text the model reads (declared), the step admission the loop resolves + * (admitted), and the run_code program bindings (callable). A fourth, clearly + * separate view collects the no-toolSet diagnostic assembly: a host assembly + * that omits the step tool set reads every registered definition, so it is a + * registration superset, never an admission proof. All four go through one + * comparator that prints the missing and the extra names, and the drift + * property is itself asserted: removing any single expected name — or reading + * the diagnostic view where a step view is required — fails at least one cell + * with the culprit named. + * + * Every row executes a real sentinel through the real worker (the read-only + * row's inspect runs with the sandbox in read-only mode), and write authority + * is asserted with a safe rejection: ops_load_sdk never spawns the worker for + * a pool-outside or permission-denied name, it answers INVALID_ARGUMENT. + * + * The official read/glob/grep trio is pinned as *visible* next to the ops + * surface: #961 (hide replaced official file tools under PTC presets) is not + * in this baseline, and its hiding implementation is deliberately not + * simulated here. The official-swap-off row records that the shipped + * global.hideOfficialTools switch is inert for PTC rows today; when #961 lands + * these two rows flip together with it. + * + * Platform note: this matrix is measured on Linux with the real worker, no + * Windows simulation. POSIX rows keep the host shell, so ops_bash is out of + * every expected set; the Windows surface has its own suites on real Windows. + * Out of scope by card: byte accounting (V2) and real-model rounds (V5); the + * loader-less declaration half owned by test/ptc-declaration-surface.test.mjs + * belongs to P5 and is not touched here. + */ +import { createScope, scopeOf, scopeParentOf } from '@deepseek-ai/dsh-scope' +import Subprocess from '@deepseek-ai/dsh-subprocess-local' +import Sandbox from '@deepseek-ai/dsh-sandbox-local' +import FileSystem from '@deepseek-ai/dsh-fs-local' +import NodePtc from '@deepseek-ai/dsh-ptc-runtime-node' +import fs from 'node:fs' +import path from 'node:path' +import { apply } from '../lib/index.js' +import { integration } from '../lib/integration/index.js' +import { bootHost, shutdown } from './lib/host.mjs' +import { prepareStep, executeCall, selectExecutor } from './lib/executor.mjs' +import { assert, test, report, wait, waitFor, withTempDir } from './lib/harness.mjs' +import { BASH_TOOL } from '../lib/policy.js' +import { WORKER, requireWorker } from './lib/boundary.mjs' + +requireWorker('ptc-two-tier-matrix') + +const OFFICIAL_TRIO = ['glob', 'grep', 'read'] +/** run_code is the executor's own tool: it is not part of the registry views this matrix compares. */ +const HOST_STEP_TOOLS = [] +/** The trio stays admitted only when the row switches hiding off (#961 in baseline). */ +const HOST_STEP_TOOLS_VISIBLE = [...OFFICIAL_TRIO] +const I = 'ops_inspect_local_file' +const L = 'ops_load_sdk' +/** The sixteen-API default pool: every file API but the resident inspect, plus batch and result. */ +const POOL16 = ['ops_batch', 'ops_copy', 'ops_diff', 'ops_edit_file', 'ops_glob', 'ops_grep', 'ops_move', + 'ops_read_docx', 'ops_read_files', 'ops_read_pdf', 'ops_read_table', 'ops_remove', 'ops_replace', + 'ops_result', 'ops_stat', 'ops_write_file'] + +let sequence = 0 + +/** One comparison for every cell of the matrix: exact set equality, culprits named. */ +function expectCell(rowId, view, expected, actual) { + const missing = expected.filter(name => !actual.includes(name)) + const extra = actual.filter(name => !expected.includes(name)) + if (missing.length || extra.length) + throw new Error(`matrix cell ${rowId}/${view} drifted: missing [${missing.join(', ')}], extra [${extra.join(', ')}]`) +} + +const sdkTextOf = assembly => assembly.sections.find(section => section.name === 'tools:sdk')?.text ?? '' +/** The ops_* names one assembly declares in tools:sdk (the ": {" shape). */ +const declaredOps = assembly => [...new Set([...sdkTextOf(assembly).matchAll(/^ {2}(ops_[a-z_]+): \{/gm)].map(match => match[1]))] + .filter(name => name !== BASH_TOOL).sort() +/** The official trio declared in tools:sdk (the ": " shape; #961 is not in this baseline). */ +const declaredOfficial = assembly => OFFICIAL_TRIO.filter(name => new RegExp(`^ {2}${name}: `, 'm').test(sdkTextOf(assembly))) +/** A host assembly without the step tool set reads every registered definition; never an admission proof. */ +const diagnosticOps = async (ctx, agent) => { + const assembly = await ctx.get('systemPrompt').assemble({ scope: agent }) + return [...new Set([...sdkTextOf(assembly).matchAll(/^ {2}(ops_[a-z_]+): \{/gm)].map(match => match[1]))] + .filter(name => name !== BASH_TOOL).sort() +} +// The Windows Bash tool is published beside the PTC row (shell suites cover it); +// the tier matrix enumerates the published PTC surface. +const admittedNames = (ctx, agent) => ctx.get('tools').definitions(agent, agent.prepared.toolSet).map(tool => tool.name) + .filter(name => name !== BASH_TOOL) +/** The integration diagnostics row publishes ops_status beside every built-in preset (#753). */ +const OPS_ADMITTED_EXTRA = ['ops_status'] +const admittedOps = names => names.filter(name => name.startsWith('ops_')).sort() +const admittedHost = names => names.filter(name => !name.startsWith('ops_')).sort() + +async function scoped(parent, key, parentKey) { + let result + await parent.plugin({ inject: ['tools'], apply(inner) { + result = createScope(inner, key, parentKey ? { parent: parentKey } : undefined).ctx + } }) + return result +} + +/** Boot one ops-ptc preset row exactly as the shipped settings compose it. */ +async function bootPresetRow(cwd, row) { + const sandboxMode = row.sandboxMode ?? 'danger-full-access' + const { ctx } = await bootHost({ withWorkingDirectory: true }) + await ctx.plugin(Subprocess); await ctx.plugin(Sandbox); await ctx.plugin(FileSystem); await ctx.plugin(NodePtc) + ctx.provide('sandboxPolicy', { resolve: ({ session } = {}) => ({ mode: session?.mode ?? sandboxMode, workspaceRoot: cwd, sessionId: session?.id }) }) + const presets = new Map(), agents = [] + ctx.provide('agents', { list: () => [...agents] }) + ctx.provide('agentPresets', { composedPreset: inner => presets.get(scopeParentOf(scopeOf(inner))) }) + const config = { global: row.global ?? {}, discovery: { binaryPath: WORKER, required: true } } + for (const id of ['minimal', 'standard', 'ptc', 'ops-ptc', 'cordis']) config[id] = { enabled: id === 'ops-ptc', + binaryPath: WORKER, required: true, ...(id === 'ops-ptc' ? row.switches : {}) } + await ctx.plugin({ name: 'matrix-preset-fixture', apply: inner => apply(inner, config) }) + const id = 'matrix-' + row.id + '-' + ++sequence, preset = await scoped(ctx, { preset: id }) + presets.set(scopeOf(preset), 'ops-ptc') + for (const name of OFFICIAL_TRIO) preset.tools.register({ name, description: name, + parameters: { type: 'object', properties: {} }, output: { schema: { type: 'object' }, render: () => [] }, execute: async () => ({}) }) + const agent = { id, session: { id, mode: sandboxMode, header: { cwd }, append() {}, read() { return [] } } } + agent.ctx = await scoped(preset, agent, scopeOf(preset)) + await agent.ctx.plugin({ inject: ['tools', 'systemPrompt'], apply: inner => { selectExecutor(inner, 'ptc') } }) + agents.push(agent) + await ctx.serial('agent/created', { agent }) + await advanceStep(ctx, agent) + return { ctx, agent } +} + +/** Boot one custom integration row: the plan slice integrations compose themselves (#988 keeps it on-demand). */ +async function bootIntegrationRow(cwd, row) { + const sandboxMode = row.sandboxMode ?? 'danger-full-access' + const { ctx } = await bootHost({ withWorkingDirectory: true }) + await ctx.plugin(Subprocess); await ctx.plugin(Sandbox); await ctx.plugin(FileSystem); await ctx.plugin(NodePtc) + ctx.provide('sandboxPolicy', { resolve: ({ session } = {}) => ({ mode: session?.mode ?? sandboxMode, workspaceRoot: cwd, sessionId: session?.id }) }) + ctx.provide('agentPresets', { composedPreset: () => undefined }) + const agents = [] + ctx.provide('agents', { list: () => [...agents] }) + await ctx.plugin(integration({ preset: row.preset, profile: 'ptc', binaryPath: WORKER, required: true, ...row.switches })) + const id = row.preset + '-' + ++sequence + const agent = { id, session: { id, mode: sandboxMode, header: { cwd, agentPreset: row.preset }, append() {}, read() { return [] } } } + await ctx.plugin({ inject: ['tools'], apply(inner) { agent.ctx = createScope(inner, agent).ctx } }) + await agent.ctx.plugin({ inject: ['tools', 'systemPrompt'], apply: inner => { selectExecutor(inner, 'ptc') } }) + agents.push(agent) + await ctx.serial('agent/created', { agent }) + await advanceStep(ctx, agent) + return { ctx, agent } +} + +async function advanceStep(ctx, agent) { + ctx.emit('session/event', agent.session, { type: 'step/end', turn: ++sequence, step: ++sequence }) + agent.prepared?.dispose() + agent.prepared = await prepareStep(ctx, agent) +} + +/** One real run_code program through the prepared step, as the loop issues it. */ +async function program(ctx, agent, code, label) { + const result = await executeCall(ctx, { agent, toolSet: agent.prepared.toolSet, name: 'run_code', + arguments: { description: label, code }, callId: 'matrix-' + ++sequence, signal: new AbortController().signal }) + assert.equal(result.isError, false, JSON.stringify(result)) + return result.value.result +} + +/** The no-side-effect sentinel each row really executes through the worker. */ +function sentinelCode(spec) { + if (spec === 'inspect') return { code: "const sentinel = await tools.ops_inspect_local_file({ path: 'fixture.txt', limit: 2 })", + checks: value => { assert.equal(value.sentinelStatus, 'completed', JSON.stringify(value)) + assert.equal(value.sentinelText, true, 'the real worker answered the inspect call') } } + if (spec === 'grep') return { code: "const sentinel = await tools.ops_grep({ pattern: 'needle', path: '.' })", + checks: value => { assert.equal(value.sentinelStatus, 'completed', JSON.stringify(value)) + assert.equal(value.sentinelText, true, 'the real worker answered the grep call') } } + if (spec === 'stat') return { code: "const sentinel = await tools.ops_stat({ path: 'fixture.txt' })", + checks: value => { assert.equal(value.sentinelStatus, 'completed', JSON.stringify(value)) } } + if (spec === 'none') return { code: 'const sentinel = null', checks: () => {} } + throw new Error('unknown sentinel: ' + spec) +} + +/** Collect the four views of one settled state and drive them all through expectCell. */ +async function collectViews(ctx, agent, row, phase, evidence) { + const spec = row.expect[phase] + const names = admittedNames(ctx, agent) + const assembly = agent.prepared.assembly + expectCell(row.id + ':' + phase, 'declared(ops)', spec.ops, declaredOps(assembly)) + expectCell(row.id + ':' + phase, 'declared(official)', spec.official, declaredOfficial(assembly)) + expectCell(row.id + ':' + phase, 'admitted(ops)', [...spec.ops, ...OPS_ADMITTED_EXTRA, ...(spec.admittedExtra ?? [])], admittedOps(names)) + expectCell(row.id + ':' + phase, 'admitted(host)', spec.host, admittedHost(names)) + // The diagnostic view is registration state: a load changes selection, not registration. + if (spec.diagnostic) expectCell(row.id + ':' + phase, 'diagnostic(ops)', spec.diagnostic, await diagnosticOps(ctx, agent)) + assert.equal(assembly.sections.some(section => section.text.includes('For on-demand APIs, use tools.ops_load_sdk')), + spec.directory, `${row.id}:${phase} directory recommendation text`) + if (spec.loaderDescription) { + const loader = ctx.get('tools').definitions(agent, agent.prepared.toolSet).find(tool => tool.name === L) + for (const word of spec.loaderDescription.includes) assert.ok(loader?.description.includes(word), `${row.id} loader description names ${word}`) + for (const word of spec.loaderDescription.excludes) assert.equal(loader?.description.includes(word), false, `${row.id} loader description must stay narrowed (${word})`) + } + // The callable view: one real program reports typeof for every probed name plus the sentinel. + const probed = [...new Set([...spec.callable, ...OPS_ADMITTED_EXTRA, ...spec.absent])] + const sentinel = sentinelCode(row.sentinel[phase]) + const noSentinel = row.sentinel[phase] === 'none' + const reject = row.reject && phase === 'first' + ? `const rejection = await tools.ops_load_sdk({ tools: ${JSON.stringify([row.reject.tool])} })` : '' + const value = await program(ctx, agent, sentinel.code + '\n' + reject + '\nreturn { ' + + probed.map(name => `${name}: typeof tools.${name}, `).join('') + + (noSentinel ? '' : 'sentinelStatus: sentinel?.status, sentinelText: JSON.stringify(sentinel).includes(\'needle\'), ') + + (reject ? 'rejectStatus: rejection?.status, rejectCode: rejection?.error?.code, ' + + 'rejectMessage: rejection?.error?.message, rejectEffect: rejection?.error?.effectState, ' : '') + + ' }', 'Matrix probe ' + row.id + ':' + phase) + const callable = probed.filter(name => value[name] === 'function').sort() + expectCell(row.id + ':' + phase, 'callable', [...spec.callable, ...OPS_ADMITTED_EXTRA], callable) + for (const name of spec.absent) assert.equal(value[name], 'undefined', `${row.id}:${phase} ${name} must not pretend to be executable`) + if (!noSentinel) sentinel.checks(value) + if (reject) { + assert.equal(value.rejectStatus, row.reject.status ?? 'error', 'the safe rejection is a returned value, not a throw') + assert.equal(value.rejectCode, row.reject.code, JSON.stringify(value)) + assert.ok(value.rejectMessage.includes(row.reject.tool), 'the rejection names the refused API: ' + value.rejectMessage) + assert.equal(value.rejectEffect, 'none', 'a refused load has no effect') + } + evidence[row.id] = { ...(evidence[row.id] ?? {}), [phase]: { ops: admittedOps(names), diagnostic: await diagnosticOps(ctx, agent), + official: declaredOfficial(assembly), callable } } +} + +/** Run one matrix row end to end and stash its collected views for the synthesis checks. */ +async function runRow(row, evidence) { + return withTempDir('matrix-' + row.id, async cwd => { + fs.writeFileSync(path.join(cwd, 'fixture.txt'), 'needle in haystack\nplain second line\n') + const { ctx, agent } = row.kind === 'integration' ? await bootIntegrationRow(cwd, row) : await bootPresetRow(cwd, row) + try { + await waitFor(() => admittedOps(admittedNames(ctx, agent)).includes(row.anchor), { what: row.anchor + ' admitted', timeoutMs: 20_000 }) + await wait(120) + agent.prepared.dispose(); agent.prepared = await prepareStep(ctx, agent) + await collectViews(ctx, agent, row, 'first', evidence) + if (row.load) { + const value = await program(ctx, agent, `const receipt = await tools.ops_load_sdk({ tools: ${JSON.stringify(row.load)} })\n` + + `return { receiptStatus: receipt.status, early: typeof tools.ops_${row.load[0]} }`, 'Matrix load ' + row.id) + assert.equal(value.receiptStatus, 'completed', JSON.stringify(value)) + assert.equal(value.early, 'undefined', row.id + ': a loaded API is callable only in the next program') + await advanceStep(ctx, agent) + await wait(120) + await collectViews(ctx, agent, row, 'loaded', evidence) + } + } finally { await shutdown(ctx) } + }) +} + +// ---- The matrix rows. Eight canonical configurations (issue #992) with the +// legacy-off variants spelled out, each carrying inputs and expected names. ---- + +const ON_DEMAND_CAP = { ops: [I, L], host: [...HOST_STEP_TOOLS], official: [], + diagnostic: [I, L, 'ops_glob', 'ops_grep', 'ops_result'], callable: [I, L], + absent: ['ops_glob', 'ops_grep', 'ops_result', 'ops_stat', 'ops_write_file', 'ops_batch'], directory: true } + +const ON_DEMAND_CAP_VISIBLE_OFFICIAL = { ...ON_DEMAND_CAP, + host: [...HOST_STEP_TOOLS_VISIBLE], official: [...OFFICIAL_TRIO] } + +const ROWS = [ + { id: 'legacy-default', anchor: L, sentinel: { first: 'inspect' }, switches: {}, expect: { first: { + ...ON_DEMAND_CAP, diagnostic: [I, L, ...POOL16] } } }, + // #961 is out of baseline: the shipped hideOfficialTools switch is inert for PTC + // rows, so its off state publishes the identical surface. #961 flips this row. + { id: 'official-swap-off', anchor: L, sentinel: { first: 'inspect' }, global: { hideOfficialTools: false }, + expect: { first: { ...ON_DEMAND_CAP_VISIBLE_OFFICIAL, diagnostic: [I, L, ...POOL16] } } }, + { id: 'global-off', anchor: I, sentinel: { first: 'inspect' }, global: { dynamicTools: false }, expect: { first: { + ops: [I], host: ['glob', 'grep'], official: ['glob', 'grep'], diagnostic: [I], callable: [I], + absent: [L, ...POOL16], directory: false } } }, + { id: 'row-off', anchor: I, sentinel: { first: 'inspect' }, switches: { dynamicTools: false }, expect: { first: { + ops: [I], host: ['glob', 'grep'], official: ['glob', 'grep'], diagnostic: [I], callable: [I], + absent: [L, ...POOL16], directory: false } } }, + { id: 'row-off-always', anchor: 'ops_stat', sentinel: { first: 'stat' }, switches: { dynamicTools: false, alwaysLoaded: ['stat'] }, expect: { first: { + ops: [I, 'ops_stat'], host: ['glob', 'grep'], official: ['glob', 'grep'], diagnostic: [I, 'ops_stat'], + callable: [I, 'ops_stat'], absent: [L, ...POOL16.filter(name => name !== 'ops_stat')], directory: false } } }, + { id: 'resident-off', anchor: L, load: ['grep'], reject: { tool: 'inspect_local_file', code: 'INVALID_ARGUMENT' }, + sentinel: { first: 'none', loaded: 'grep' }, switches: { residentTools: false }, expect: { + first: { ops: [L], host: ['read'], official: ['read'], diagnostic: [L, ...POOL16], + callable: [L], absent: [I, ...POOL16], directory: true }, + loaded: { ops: [L, 'ops_grep'], host: ['read'], official: ['read'], + callable: [L, 'ops_grep'], absent: [I, 'ops_result', 'ops_stat', 'ops_glob'], directory: true } } }, + { id: 'on-demand-first', anchor: L, sentinel: { first: 'inspect' }, switches: { allowedTools: ['glob', 'grep'] }, + expect: { first: ON_DEMAND_CAP } }, + // The unified acceptance rule (#979 §3) forces result into the load set: the + // full first round publishes it, so the loaded business set equals it only + // when on-demand loads it explicitly, like any other pool member. + { id: 'on-demand-loaded', anchor: L, load: ['glob', 'grep', 'result'], sentinel: { first: 'inspect', loaded: 'grep' }, + switches: { allowedTools: ['glob', 'grep'] }, expect: { first: ON_DEMAND_CAP, + // The pool is exhausted after the load: the directory recommendation + // correctly disappears when no allowed name is left to load. + loaded: { ops: [I, L, 'ops_glob', 'ops_grep', 'ops_result'], host: [...HOST_STEP_TOOLS], official: [], + callable: [I, L, 'ops_glob', 'ops_grep', 'ops_result'], absent: ['ops_stat', 'ops_write_file', 'ops_batch'], directory: false } } }, + { id: 'full-direct', anchor: 'ops_glob', sentinel: { first: 'grep' }, + switches: { disclosurePolicy: 'full', allowedTools: ['glob', 'grep'] }, expect: { first: { + ops: [I, 'ops_glob', 'ops_grep', 'ops_result'], host: [...HOST_STEP_TOOLS], official: [], + diagnostic: [I, 'ops_glob', 'ops_grep', 'ops_result'], callable: [I, 'ops_glob', 'ops_grep', 'ops_result'], + absent: [L, 'ops_stat', 'ops_write_file', 'ops_batch'], directory: false } } }, + { id: 'read-only', anchor: L, sandboxMode: 'read-only', reject: { tool: 'write_file', code: 'POLICY_DENIED', status: 'rejected' }, + sentinel: { first: 'inspect' }, switches: { allowedTools: ['write_file', 'grep'] }, expect: { first: { + ops: [I, L], host: ['glob'], official: ['glob'], + diagnostic: [I, L, 'ops_grep', 'ops_result', 'ops_write_file'], callable: [I, L], + absent: ['ops_write_file', 'ops_grep', 'ops_result', 'ops_stat'], directory: true, + loaderDescription: { includes: ['write_file, grep, result'], excludes: [] } } } }, + { id: 'narrow-pool', anchor: L, reject: { tool: 'stat', code: 'INVALID_ARGUMENT' }, sentinel: { first: 'inspect' }, + switches: { allowedTools: ['grep'] }, expect: { first: { ...ON_DEMAND_CAP, host: ['glob'], official: ['glob'], + diagnostic: [I, L, 'ops_grep', 'ops_result'], + loaderDescription: { includes: ['grep, result'], excludes: ['stat', 'glob'] } } } }, + // The custom integration path also admits its own ops_status diagnostics reader, + // which has no SDK projection: it is admitted and callable but never declared. + { id: 'custom-preset', kind: 'integration', preset: 'matrix-custom', anchor: L, load: ['glob', 'grep', 'result'], + sentinel: { first: 'inspect', loaded: 'grep' }, switches: { allowedTools: ['glob', 'grep'] }, expect: { + first: { ops: [I, L], admittedExtra: ['ops_status'], host: [], official: [], diagnostic: [I, L, 'ops_glob', 'ops_grep', 'ops_result'], + callable: [I, L, 'ops_status'], absent: ['ops_glob', 'ops_grep', 'ops_result', 'ops_stat'], directory: true }, + loaded: { ops: [I, L, 'ops_glob', 'ops_grep', 'ops_result'], admittedExtra: ['ops_status'], host: [], official: [], + callable: [I, L, 'ops_glob', 'ops_grep', 'ops_result', 'ops_status'], absent: ['ops_stat', 'ops_write_file'], directory: false } } }, +] + +/** Collected views of every row, reused by the synthesis checks below. */ +const evidence = {} + +for (const row of ROWS) { + await test('matrix row ' + row.id + ': three views and the diagnostic view agree with the row', () => runRow(row, evidence)) +} + +await test('unified acceptance: the on-demand loaded business set equals the full first round on both publication paths', () => { + const kernel = name => name !== L && !OPS_ADMITTED_EXTRA.includes(name) + const business = evidence['on-demand-loaded'].loaded.ops.filter(kernel) + const full = evidence['full-direct'].first.ops.filter(kernel) + // ops_status is the custom integration's own diagnostics reader, admitted + // beside the pool; it is not a member of the two-tier publication kernel. + const custom = evidence['custom-preset'].loaded.ops.filter(kernel) + expectCell('unified-acceptance', 'preset-loaded-vs-full', full, business) + expectCell('unified-acceptance', 'custom-loaded-vs-full', full, custom) +}) + +await test('drift property: removing any one expected name fails exactly its cell and prints the culprit', () => { + const stepOps = evidence['on-demand-first'].first.ops + for (const name of stepOps) { + const corrupted = stepOps.filter(candidate => candidate !== name) + assert.throws(() => expectCell('on-demand-first', 'declared(ops)', corrupted, stepOps), + new RegExp('missing \\[.*' + name), name + ' removal must fail with the name printed') + } + assert.throws(() => expectCell('on-demand-first', 'declared(ops)', stepOps, [...stepOps, 'ops_nope']), + /extra \[.*ops_nope/, 'an invented name must fail as extra with the name printed') + assert.throws(() => expectCell('on-demand-first', 'declared(ops)', [...stepOps, 'ops_nope'], stepOps), + /missing \[.*ops_nope/, 'a lost expected name must fail as missing with the name printed') + const callable = evidence['on-demand-first'].first.callable + assert.throws(() => expectCell('on-demand-first', 'callable', callable.filter(name => name !== L), callable), + new RegExp('missing \\[.*' + L), 'deleting the loader admission fails the callable cell') +}) + +await test('drift property: reading the diagnostic registry view cannot satisfy a step cell', () => { + // The diagnostic assembly registers the whole pool while the step admits + // only inspect + loader, so a wrong-view read must fail with the pool named. + const step = evidence['on-demand-first'].first.ops + const diagnostic = evidence['legacy-default'].first.diagnostic + assert.throws(() => expectCell('on-demand-first', 'declared(ops)', step, diagnostic), + /extra \[.*ops_grep/, 'the pool registration must be reported as extra names') + assert.throws(() => expectCell('legacy-default', 'diagnostic(ops)', diagnostic, step), + /missing \[.*ops_grep/, 'the step view must be reported as missing names') +}) + +await test('the hidden official trio leaves the PTC surface now that #961 landed', () => { + // #961 (hide replaced official file tools under PTC presets) is now in the + // baseline: the replaced read/glob/grep are withdrawn from the PTC + // presentation whenever their ops replacements are servable, so the trio no + // longer appears beside the ops surface. This pin used to assert the opposite + // visibility; it is updated together with that landing on purpose, so the + // flip stays a reviewable diff instead of a silent expectation change. + for (const id of ['legacy-default', 'official-swap-off']) { + const first = evidence[id].first + const expectedOfficial = id === 'official-swap-off' ? OFFICIAL_TRIO : [] + assert.deepEqual(first.official, expectedOfficial, id + ' follows the hiding switch') + assert.deepEqual(first.ops, [I, L, ...OPS_ADMITTED_EXTRA], id + ' publishes the resident surface plus the loader') + } + assert.deepEqual(evidence['legacy-default'].first.ops, evidence['official-swap-off'].first.ops, + 'the shipped switch stays inert for PTC rows even with #961 present') +}) + +report('ptc-two-tier-matrix') diff --git a/test/publication-fail-open.test.mjs b/test/publication-fail-open.test.mjs new file mode 100644 index 00000000..91e52686 --- /dev/null +++ b/test/publication-fail-open.test.mjs @@ -0,0 +1,118 @@ +/** An unrecoverable replacement executor failure must fail open to the official tools (#962). */ +import { createScope } from '@deepseek-ai/dsh-scope' +import { mountReplacements } from '../lib/publication/replacements.js' +import { mountConsumer } from '../lib/consumer/mount.js' +import { resolveConfig } from '../lib/config.js' +import { resolveShells } from '../lib/shells.js' +import { BASH_TOOL } from '../lib/policy.js' +import { assert, test, report } from './lib/harness.mjs' +import { bootHost, shutdown } from './lib/host.mjs' +import { prepareStep, executeCall } from './lib/executor.mjs' + +const official = name => ({ name, description: 'fixture ' + name, parameters: { type: 'object', properties: {} }, + output: { schema: { type: 'object' }, render: () => [] }, execute: async () => ({ ran: name }) }) +/** One settled host-shaped Bash failure value for a preparation stage. */ +const bashFailureValue = stage => ({ kind: 'foreground', status: 'failed', + error: 'EXECUTOR_UNAVAILABLE: ' + stage + ': fixture payload refused. No command was executed.', + timedOut: false, timeoutMs: 120000, stdout: { text: '', truncated: false }, stderr: { text: '', truncated: false } }) + +async function fixture(counterpartValue) { + const { ctx, tools } = await bootHost() + const agents = [] + ctx.provide('agents', { list: () => agents }) + const removeOfficial = tools.register(official('pwsh')) + const bash = { ...official(BASH_TOOL), execute: async () => counterpartValue } + const removeBash = tools.register(bash) + await ctx.plugin({ apply(inner) { + mountReplacements(inner, { id: 'fail-open', targets: new Map([['pwsh', BASH_TOOL]]), gate: { allows: () => true }, keepOfficial: () => false }) + } }) + await new Promise(resolve => setImmediate(resolve)) + const agent = { id: 'fail-open', session: { id: 'fail-open', header: { cwd: process.cwd() } } } + agent.ctx = createScope(ctx, agent).ctx + agents.push(agent) + await ctx.serial('agent/created', { agent }) + return { ctx, tools, agent, dispose: () => { removeOfficial(); removeBash() } } +} + +const names = step => step.assembly.tools.map(tool => tool.name) +const dispatch = (ctx, agent, step, name) => executeCall(ctx, { agent, toolSet: step.toolSet, name, arguments: {}, callId: name }) + +await test('an unrecoverable counterpart runtime failure lifts the official fallback on the next step', async () => { + const { ctx, agent, dispose } = await fixture(bashFailureValue('bash-runtime-missing')) + try { + const first = await prepareStep(ctx, agent) + try { + assert.equal(new Set(names(first)).has('pwsh'), false, 'the replacement is active before any failure') + const result = await dispatch(ctx, agent, first, BASH_TOOL) + assert.equal(result.isError, false, JSON.stringify(result)) + assert.match(result.value.error, /EXECUTOR_UNAVAILABLE: bash-runtime-missing/) + } finally { first.dispose() } + const second = await prepareStep(ctx, agent) + try { + assert.equal(new Set(names(second)).has('pwsh'), true, 'the official fallback must return after an unrecoverable executor failure') + assert.deepEqual((await dispatch(ctx, agent, second, 'pwsh')).value, { ran: 'pwsh' }) + } finally { second.dispose() } + } finally { dispose(); await shutdown(ctx) } +}) + +await test('a retryable staging failure keeps the replacement in place', async () => { + const { ctx, agent, dispose } = await fixture(bashFailureValue('bash-staging-failed')) + try { + const first = await prepareStep(ctx, agent) + try { await dispatch(ctx, agent, first, BASH_TOOL) } finally { first.dispose() } + const second = await prepareStep(ctx, agent) + try { + assert.equal(new Set(names(second)).has('pwsh'), false, 'a staging interruption is repaired by the next call, not by lifting the shadow') + } finally { second.dispose() } + } finally { dispose(); await shutdown(ctx) } +}) + +await test('only preparation stages no retry can repair are classified unrecoverable', async () => { + const { unrecoverableCounterpartRuntime } = await import('../lib/publication/replacements.js') + assert.equal(unrecoverableCounterpartRuntime(bashFailureValue('bash-runtime-missing')), true) + assert.equal(unrecoverableCounterpartRuntime(bashFailureValue('bash-ipc-mismatch')), true) + assert.equal(unrecoverableCounterpartRuntime(bashFailureValue('bash-lifecycle-lock-busy')), true) + assert.equal(unrecoverableCounterpartRuntime(bashFailureValue('bash-cache-unusable')), true) + assert.equal(unrecoverableCounterpartRuntime(bashFailureValue('bash-staging-failed')), false) + assert.equal(unrecoverableCounterpartRuntime(bashFailureValue('bash-generation-quarantine-failed')), false) + // The Bash tool description contract quotes remedies; a successful value never classifies. + assert.equal(unrecoverableCounterpartRuntime({ kind: 'foreground', exitCode: 0, stdout: { text: 'EXECUTOR_UNAVAILABLE: bash-runtime-missing: quoted', truncated: false } }), false) + // PTC Outcome failures carry a structured error, not the host message; they stay out of scope. + assert.equal(unrecoverableCounterpartRuntime({ status: 'error', error: { code: 'EXECUTOR_UNAVAILABLE', message: 'DSH authority is unavailable' } }), false) + // 'bash-runtime-incompatible' keeps its own complete message without the EXECUTOR_UNAVAILABLE prefix. + assert.equal(unrecoverableCounterpartRuntime({ error: 'bash-runtime-incompatible: the resolved bash is not the pinned build.' }), true) + assert.equal(unrecoverableCounterpartRuntime({ error: 'EXIT_CODE: command failed' }), false) +}) + +await test('a shell slot whose executor never resolved never hides the official tools', async () => { + const { ctx, tools } = await bootHost() + const agents = [] + try { + ctx.provide('agents', { list: () => agents }) + const removeOfficial = tools.register(official('pwsh')) + // A foreign same-name registration is the #934 host shape: the shadow used to trust its presence. + const removeForeign = tools.register({ ...official(BASH_TOOL), execute: async () => bashFailureValue('bash-runtime-missing') }) + const core = resolveConfig({ promptPolicy: true, publishBashTool: true, bashPath: process.platform === 'win32' ? 'C:/definitely-not-here/niu.exe' : '/definitely-not-here/bash' }) + const bash = resolveShells(core).bash + assert.equal(bash.available, false, 'the fixture must fail resolution') + await ctx.plugin({ inject: ['tools', 'systemPrompt'], apply(inner) { + mountConsumer(inner, { id: 'fail-open-unavailable', gate: { allows: () => true }, core, shell: true, + presentation: 'native', files: false, residentTools: false, discovery: false, + native: { alwaysLoaded: [] }, external: { allowed: [], alwaysLoaded: [], disabled: [] }, + replaceOfficial: true, replacementTargets: new Map([['pwsh', BASH_TOOL]]), section: null }) + } }) + await new Promise(resolve => setImmediate(resolve)) + const agent = { id: 'fail-open-unavailable', session: { id: 'fail-open-unavailable', header: { cwd: process.cwd() } } } + agent.ctx = createScope(ctx, agent).ctx + agents.push(agent) + await ctx.serial('agent/created', { agent }) + const step = await prepareStep(ctx, agent) + try { + assert.equal(new Set(names(step)).has('pwsh'), true, 'an unresolvable Bash executor must keep the official tool visible') + assert.deepEqual((await dispatch(ctx, agent, step, 'pwsh')).value, { ran: 'pwsh' }) + } finally { step.dispose() } + removeOfficial(); removeForeign() + } finally { await shutdown(ctx) } +}) + +report('publication-fail-open') diff --git a/test/publication-legacy-admission-surface.test.mjs b/test/publication-legacy-admission-surface.test.mjs new file mode 100644 index 00000000..3ee615c9 --- /dev/null +++ b/test/publication-legacy-admission-surface.test.mjs @@ -0,0 +1,42 @@ +/** The retained file row must advertise admitted tools, not registered-but-unadmitted ones (#934). */ +import { createScope } from '@deepseek-ai/dsh-scope' +import { applyComponent } from '../lib/legacy/host.js' +import { assert, test, report, waitFor } from './lib/harness.mjs' +import { bootHost, shutdown } from './lib/host.mjs' +import { prepareStep } from './lib/executor.mjs' + +const SECTION = 'dsh-ops:repository-tooling:file' +const tick = () => new Promise(resolve => setImmediate(resolve)) + +await test('a registered-but-unadmitted tool stays out of the legacy tooling table until admission', async () => { + const { ctx } = await bootHost() + try { + ctx.provide('agents', { list: () => [] }) + ctx.provide('sandboxPolicy', { resolve: () => ({ mode: 'danger-full-access', workspaceRoot: process.cwd() }) }) + ctx.provide('subprocess', { spawn() { throw new Error('legacy surface fixture never spawns') } }) + applyComponent(ctx, { promptPolicy: true }, 'file') + const agent = { id: 'legacy-surface', session: { id: 'legacy-surface', header: { cwd: process.cwd() }, append() {}, read() { return [] } } } + agent.ctx = createScope(ctx, agent).ctx + // The host-bug shape from #934: a same-scope registration whose `when` fails. A host that + // ignores `when` in definitions() reads it as declared while the step excludes it. + let admitted = false + await agent.ctx.plugin({ inject: ['tools'], apply(inner) { + inner.tools.register({ name: 'read_image', description: 'fixture image reader', parameters: { type: 'object', properties: {} }, + output: { schema: { type: 'object' }, render: () => [] }, execute: async () => ({ ran: 'read_image' }) }, + { when: () => admitted }) + } }) + await tick(); await tick() + await waitFor(() => ctx.get('tools').get('ops_read_files', agent), { what: 'legacy file row publication' }) + const text = step => step.assembly.sections.find(section => section.name === SECTION)?.text ?? '' + const hidden = await prepareStep(ctx, agent) + try { + assert.doesNotMatch(text(hidden), /read_image/, 'a registered-but-unadmitted tool must not be advertised') + assert.match(text(hidden), /ops_read_files/, "the row's own admitted tools stay advertised") + } finally { hidden.dispose() } + admitted = true + const shown = await prepareStep(ctx, agent) + try { assert.match(text(shown), /read_image \(host\)/, 'admitting the tool brings the row back') } finally { shown.dispose() } + } finally { await shutdown(ctx) } +}) + +report('publication-legacy-admission-surface') diff --git a/test/publication-registrar.test.mjs b/test/publication-registrar.test.mjs index b9e4a3b2..64d077dd 100644 --- a/test/publication-registrar.test.mjs +++ b/test/publication-registrar.test.mjs @@ -121,6 +121,51 @@ await test('grant diagnostics distinguish missing or conflicting dispatch from i } finally { await shutdown(ctx) } }) +await test('no-entitled advisors answer in grant order; undefined or throwing ones do not mask later guidance', async () => { + const { ctx, tools } = await bootHost() + try { + const a = agentScope(ctx, 'advised') + const guidance = { status: 'error', error: { code: 'ADVISOR_GUIDANCE', effectState: 'none', retry: 'never', + message: 'point at the loader', nextAction: 'load the API first' } } + // One entry can hold several same-agent file grants (for example a PTC pool grant + // and a discovery pool grant for the same public name). Consultation must cover + // every grant in insertion order: an advisor that cannot help — by answering + // undefined or by throwing — must not mask a later grant's pointer with the + // generic retry-never denial (#968 review finding). + const lease = await acquire(ctx, undefined, name, canonical) + const consulted = [] + const silent = () => { consulted.push('silent'); return undefined } + const broken = () => { consulted.push('broken'); throw new Error('advisor exploded') } + const guiding = () => { consulted.push('guiding'); return guidance } + lease.grant(grant(a, 'silent', { entitled: false, deferred: silent })) + lease.grant(grant(a, 'broken', { entitled: false, deferred: broken })) + lease.grant(grant(a, 'guiding', { entitled: false, deferred: guiding })) + const definition = tools.get(name) ?? tools.definitions().find(candidate => candidate.name === name) + assert.ok(definition, 'a lease whose grants are all not-entitled still registers') + const result = await definition.execute({}, { agent: a, toolSet: { view: () => new Map([[name, definition]]) } }) + assert.deepEqual(result, guidance, 'the third advisor must answer instead of the generic denial') + assert.deepEqual(consulted, ['silent', 'broken', 'guiding'], 'every advisor is consulted once, in grant order') + // Control: the first advisor that answers wins; consultation never runs past it. + const ordered = await acquire(ctx, undefined, name + '-order', canonical) + const firstOutcome = { ...guidance, error: { ...guidance.error, code: 'FIRST_WINS' } } + let later = 0 + ordered.grant(grant(a, 'answers-first', { entitled: false, deferred: () => firstOutcome })) + ordered.grant(grant(a, 'never-asked', { entitled: false, deferred: () => { ++later; return guidance } })) + const second = tools.get(name + '-order') + assert.deepEqual(await second.execute({}, { agent: a, toolSet: { view: () => new Map([[name + '-order', second]]) } }), firstOutcome) + assert.equal(later, 0, 'consultation stops at the first advisor that answers') + // Control: no advisor able to help keeps the honest generic denial. + const helpless = await acquire(ctx, undefined, name + '-silent', canonical) + helpless.grant(grant(a, 'cannot-help', { entitled: false, deferred: () => undefined })) + helpless.grant(grant(a, 'explodes', { entitled: false, deferred: () => { throw new Error('no guidance') } })) + const third = tools.get(name + '-silent') + const denied = await third.execute({}, { agent: a, toolSet: { view: () => new Map([[name + '-silent', third]]) } }) + assert.equal(denied.status, 'rejected') + assert.equal(denied.error.code, 'PUBLICATION_GRANT_UNAVAILABLE') + lease.dispose(); ordered.dispose(); helpless.dispose() + } finally { await shutdown(ctx) } +}) + await test('late ancestor source withdraws fallback synchronously and unload stays conservative', async () => { const { ctx, tools } = await bootHost() const steps = [] diff --git a/test/resident-switch.test.mjs b/test/resident-switch.test.mjs index 68945394..886a5c73 100644 --- a/test/resident-switch.test.mjs +++ b/test/resident-switch.test.mjs @@ -13,9 +13,6 @@ import { PTC_RESIDENT_TOOLS } from '../lib/ptc/config.js' import { publicToolName, BASH_TOOL } from '../lib/policy.js' import { bootHost, shutdown, scopedToolNames } from './lib/host.mjs' import { prepareStep, executeCall, selectExecutor } from './lib/executor.mjs' -import fs from 'node:fs' -import path from 'node:path' -import { createHash } from 'node:crypto' import { assert, test, report, waitFor, withTempDir } from './lib/harness.mjs' import { WORKER, requireWorker } from './lib/boundary.mjs' @@ -184,22 +181,106 @@ const drivePreStep = async (agent, turn) => { /** Plugin-authored contexts appended to the accepted user batch. */ const stepTailMessages = decision => decision.messages.filter(message => message.source?.kind === 'dsh-ops:step-tail') -await test('ops-ptc batchingGuidance teaches until the lesson is committed to history', () => withTempDir('batching-teach', async cwd => { +/** The one-time full lesson, told from a per-violation notice by its contributor suffix. */ +const lesson = context => context.source?.kind === 'dsh-ops:step-tail' && context.source?.contributor === 'batching-guidance:lesson' + +/** The one-time lesson as delivered on the first accepted root run_code result. */ +const lessonContexts = async (ctx, agent, code = '// plan: one read\nconst r = await tools.ops_inspect_local_file({ path: "package.json" })\nreturn r.status') => { + const run = await executeCall(ctx, { agent, toolSet: agent.prepared.toolSet, name: 'run_code', + arguments: { description: 'Compliant fixture', code }, + callId: 'lesson-' + ++sequence, signal: new AbortController().signal }) + assert.equal(run.isError, false, JSON.stringify(run)) + return (run.additionalContexts ?? []).filter(lesson) +} + +await test('ops-ptc batchingGuidance teaches once on the first run_code result, never in the user batch', () => withTempDir('batching-teach', async cwd => { const { ctx, agent } = await setup(cwd, 'ops-ptc', { residentTools: true, dynamicTools: true, batchingGuidance: true, alwaysLoaded: ['stat'], allowedTools: ['stat'] }) try { await waitFor(() => names(ctx, agent).includes(publicToolName('stat'))) - const first = stepTailMessages(await drivePreStep(agent, 1)) + // A user-role lesson reads as fresh user input, and the model answers the + // lesson instead of the task (#929): the user batch stays free of guidance. + assert.equal(stepTailMessages(await drivePreStep(agent, 1)).length, 0) + // The lesson rides the first accepted root run_code tool result through the + // same-step ephemeral inbox, which the host never logs as user history. + const taught = async callId => { + const run = await executeCall(ctx, { agent, toolSet: agent.prepared.toolSet, name: 'run_code', + arguments: { description: 'Compliant fixture', code: '// plan: two checks\nconst [a, b] = await Promise.all([tools.ops_stat({ path: "x" }), tools.ops_stat({ path: "y" })])\nreturn [a.status, b.status]' }, + callId, signal: new AbortController().signal }) + assert.equal(run.isError, false, JSON.stringify(run)) + return (run.additionalContexts ?? []).filter(context => context.source?.kind === 'dsh-ops:step-tail' && context.source?.contributor === 'batching-guidance:lesson') + } + const first = await taught('batching-teach-1') assert.equal(first.length, 1, JSON.stringify(first)) - assert.equal(first[0].source.contributor, 'batching-guidance') assert.match(JSON.stringify(first), /Batch tool calls/) assert.match(JSON.stringify(first), /not a user instruction or authorization/) - // The pre-step decision is not the commit point: a batch withdrawn before - // admission never becomes history, so the lesson must be offered again. - assert.equal(stepTailMessages(await drivePreStep(agent, 2)).length, 1, 'an uncommitted lesson is offered again') - // Once the host reports the durable message, the lesson is spent for the session. - await ctx.serial('session/event', agent.session, { type: 'user/message', data: { source: first[0].source } }) - assert.equal(stepTailMessages(await drivePreStep(agent, 3)).length, 0, 'a committed lesson is never repeated') + // Teach once: a second root call never repeats the lesson. + assert.equal((await taught('batching-teach-2')).length, 0, 'the lesson is spent after its one delivery') + // Nothing re-enters the user batch once the lesson is delivered. + assert.equal(stepTailMessages(await drivePreStep(agent, 2)).length, 0) + } finally { await shutdown(ctx) } +})) + +await test('a loader-less ops-ptc row teaches only what it publishes', () => withTempDir('batching-no-loader', async cwd => { + const { ctx, agent } = await setup(cwd, 'ops-ptc', { residentTools: true, dynamicTools: false, + batchingGuidance: true, alwaysLoaded: ['stat'], allowedTools: ['stat'] }) + try { + await waitFor(() => names(ctx, agent).includes(publicToolName('stat'))) + // End to end through the ephemeral inbox: the scope publishes no loader, so the + // lesson must not name one. This is the assertion that fails when the surface is dropped. + const taught = await lessonContexts(ctx, agent, '// plan: read one path\nconst r = await tools.ops_stat({ path: "a" })\nreturn r.status') + assert.equal(taught.length, 1, JSON.stringify(taught)) + const text = JSON.stringify(taught) + assert.equal(text.includes('ops_load_sdk'), false, 'the unreachable loader is gone') + assert.match(text, /call what is published/, 'the direct-call step replaces it') + assert.match(text, /ops_stat/, 'the published API is listed instead') + // The only file API this scope publishes is the resident reader, so the lesson must + // state its contract: directories are rejected, concrete paths are required. + assert.match(text, /rejects directories/, 'the inspect path requirement is taught') + } finally { await shutdown(ctx) } +})) + +/** The replaced Step 3 section of a taught loader-less lesson, as flat JSON text. */ +const step3Of = taught => { + const lesson = JSON.stringify(taught) + return lesson.slice(lesson.indexOf('call what is published'), lesson.indexOf('Trajectory 1')) +} + +await test('a read-only loader-less lesson and notice keep the mutating API out', () => withTempDir('batching-readonly-lesson', async cwd => { + const { ctx, agent } = await setup(cwd, 'ops-ptc', { residentTools: true, dynamicTools: false, + batchingGuidance: true, alwaysLoaded: ['edit_file'], allowedTools: ['stat'] }) + try { + await waitFor(() => names(ctx, agent).includes(publicToolName('inspect_local_file'))) + // The row always loads the mutating API, so only the read-only policy can keep it + // out of the direct-call list: the surface is resolved at delivery time. + agent.session.mode = 'read-only' + const taught = await lessonContexts(ctx, agent) + assert.equal(taught.length, 1, JSON.stringify(taught)) + const step3 = step3Of(taught) + assert.ok(step3.length > 0, 'the direct-call section is present') + assert.equal(step3.includes('ops_edit_file'), false, 'the read-only surface drops the mutating API') + assert.match(step3, /ops_inspect_local_file/, 'the resident reader stays listed') + assert.match(step3, /rejects directories/, 'the exact-path note stays with the reader') + // The same surface narrows the bypass notice: no write API, the reader stays named. + const result = await executeCall(ctx, { agent, toolSet: agent.prepared.toolSet, name: 'run_code', + arguments: { description: 'shape fixture', code: 'const r = await tools.ops_bash({ description: "list", command: "ls -la" })\nreturn r.stdout.text' }, + callId: 'readonly-lesson-' + ++sequence, signal: new AbortController().signal }) + const notice = JSON.stringify(result.additionalContexts ?? []) + assert.equal(notice.includes('ops_edit_file'), false, 'the notice keeps the mutating API out') + assert.match(notice, /ops_inspect_local_file/, 'the resident reader stays named') + } finally { await shutdown(ctx) } +})) + +await test('a writable loader-less lesson lists what its row always loads', () => withTempDir('batching-writable-lesson', async cwd => { + const { ctx, agent } = await setup(cwd, 'ops-ptc', { residentTools: true, dynamicTools: false, + batchingGuidance: true, alwaysLoaded: ['edit_file'], allowedTools: ['stat'] }) + try { + await waitFor(() => names(ctx, agent).includes(publicToolName('edit_file'))) + // Without the read-only policy the same row teaches its whole resident list, which + // is what keeps the narrowing keyed on the policy instead of always on. + const taught = await lessonContexts(ctx, agent) + assert.equal(taught.length, 1, JSON.stringify(taught)) + assert.match(step3Of(taught), /ops_edit_file/, 'a writable scope teaches what its row always loads') } finally { await shutdown(ctx) } })) @@ -209,6 +290,11 @@ await test('ops-ptc without batchingGuidance teaches nothing on the first turn', try { await waitFor(() => names(ctx, agent).includes(publicToolName('stat'))) assert.equal(stepTailMessages(await drivePreStep(agent, 1)).length, 0) + const run = await executeCall(ctx, { agent, toolSet: agent.prepared.toolSet, name: 'run_code', + arguments: { description: 'Thin fixture', code: 'const r = await tools.ops_stat({ path: "a" })\nreturn r.status' }, + callId: 'batching-off-teach', signal: new AbortController().signal }) + assert.equal(run.isError, false, JSON.stringify(run)) + assert.equal((run.additionalContexts ?? []).length, 0, 'switch off: no listener, therefore no lesson and no notice') } finally { await shutdown(ctx) } })) @@ -224,18 +310,38 @@ await test('the root run_code trigger answers on its own tool result', () => wit // A compliant program earns nothing, which is what keeps the notice meaningful. const good = await run('// plan: stat one path; stat another\nconst [a, b] = await Promise.all([tools.ops_stat({ path: "a" }), tools.ops_stat({ path: "b" })])\nreturn { a, b }') assert.equal(good.isError, false, JSON.stringify(good)) - assert.equal(contexts(good).length, 0, 'a compliant program earns no notice') + assert.equal(contexts(good).filter(context => !lesson(context)).length, 0, 'a compliant program earns no notice') + assert.equal(contexts(good).filter(lesson).length, 1, 'the one-time lesson rides the first root result') // No plan line: the plan category speaks, and it says only that the comment is missing. const thin = await run('const r = await tools.ops_stat({ path: "a" })\nreturn r.status') assert.equal(thin.isError, false, JSON.stringify(thin)) assert.match(text(thin), /did not open with a plan line/, 'the plan notice rides the root result') - assert.doesNotMatch(text(thin), /Promise\.all/, 'the plan notice stays about the comment') - // A shell listing is a bypass; the notice names the API that replaces it. + assert.doesNotMatch(JSON.stringify(contexts(thin).filter(context => !lesson(context))), /Promise\.all/, 'the plan notice stays about the comment') + // A shell listing is a bypass; resident file APIs are reachable without loading, + // so the notice still names them even though this pool could not load them. const bypass = await run('const r = await tools.ops_bash({ description: "list", command: "ls -la" })\nreturn r.stdout.text') - assert.match(text(bypass), /ops_glob/, 'the bypass notice names the listing tool') + assert.match(text(bypass), /ops_inspect_local_file/, 'the notice names the resident file API') + assert.doesNotMatch(text(bypass), /ops_bash/, 'the shell it just accused is not offered as the fix') // The trigger appends context and never rewrites the tool result itself. assert.equal(thin.value.result, 'completed', JSON.stringify(thin.value)) } finally { await shutdown(ctx) } })) +await test('a read-only agent is never told to edit files through the notice', () => withTempDir('batching-readonly', async cwd => { + const { ctx, agent } = await setup(cwd, 'ops-ptc', { residentTools: true, dynamicTools: true, + batchingGuidance: true, alwaysLoaded: ['stat'], allowedTools: ['stat', 'edit_file'] }) + try { + await waitFor(() => names(ctx, agent).includes(publicToolName('stat'))) + // The policy is resolved per delivery, so a read-only session narrows the surface. + agent.session.mode = 'read-only' + const result = await executeCall(ctx, { agent, toolSet: agent.prepared.toolSet, name: 'run_code', + arguments: { description: 'shape fixture', code: 'const r = await tools.ops_bash({ description: "list", command: "ls -la" })\nreturn r.stdout.text' }, + callId: 'readonly-' + ++sequence, signal: new AbortController().signal }) + // The lesson rides the same result; the notice is what this check is about. + const text = JSON.stringify((result.additionalContexts ?? []).filter(context => !lesson(context))) + assert.doesNotMatch(text, /ops_edit_file/, 'a read-only agent is not taught to mutate') + assert.match(text, /ops_inspect_local_file/, 'the reachable file API stays named') + } finally { await shutdown(ctx) } +})) + report('resident-switch') diff --git a/test/session-idempotent-reconcile.test.mjs b/test/session-idempotent-reconcile.test.mjs new file mode 100644 index 00000000..6b79e395 --- /dev/null +++ b/test/session-idempotent-reconcile.test.mjs @@ -0,0 +1,201 @@ +/** + * Session bash layers stay idempotent under concurrency and turn boundaries. + * + * The suite drives the real Cordis host, the real tool registry and the real + * SessionStore append guard against `mountSessionBash`, covering #579, #842 + * and #873: two preset consumers may briefly both believe they own one agent, + * a publication that lost a same-layer name conflict must not retry forever + * or re-register the identical conflicting name, and a turn boundary + * re-asserts authority so a new turn never starts on stale bash state. + */ +import { createScope } from '@deepseek-ai/dsh-scope' +import { SessionStore } from '@deepseek-ai/dsh-session' +import { mountSessionBash } from '../lib/session-shells.js' +import { prepareStep } from './lib/executor.mjs' +import { bootHost, shutdown } from './lib/host.mjs' +import { assert, test, report, TEST_HOME } from './lib/harness.mjs' + +const flush = async () => { for (let i = 0; i < 6; i++) await new Promise(resolve => setImmediate(resolve)) } + +/** The bash resolution one row publishes, as `mountSessionBash` accepts it. */ +const layerOf = file => ({ bash: { available: true, file, source: 'explicit' } }) + +/** Boot the real host with the authority services the rows resolve through. */ +async function setup() { + const host = await bootHost() + const agents = [] + host.ctx.provide('agents', { list: () => agents }) + host.agents = agents + await host.ctx.plugin(inner => { + inner.provide('subprocess', { spawn() { throw new Error('publication tests must not execute') } }) + inner.provide('sandboxPolicy', { resolve: ({ session }) => ({ mode: session.mode, workspaceRoot: TEST_HOME }) }) + }) + return host +} + +/** Mount one preset consumer row; its gate names the agents it serves. */ +function mountRow(ctx, gate, file) { + return ctx.plugin({ inject: ['tools', 'systemPrompt'], apply(inner) { + mountSessionBash(inner, { publishBashTool: true }, layerOf(file), gate) + } }) +} + +/** Bind a real session to a fresh agent the way the host creates one. */ +function mint(ctx, session) { + const agent = { id: session.id, session } + agent.ctx = createScope(ctx, agent).ctx + return agent +} + +/** Register a foreign `ops_bash` in the agent's own layer, as another plugin would. */ +async function registerForeign(agent) { + let undo + await agent.ctx.plugin({ inject: ['tools'], apply(inner) { + undo = inner.tools.register({ name: 'ops_bash', description: 'foreign bash', + parameters: { type: 'object' }, output: { schema: { type: 'string' }, render: () => [] }, + execute: async () => 'foreign' }) + } }) + return () => undo() +} + +await test('two rows that both claim one agent converge without duplicate failures (#842)', async () => { + const { ctx, tools, agents } = await setup() + const warnings = [] + ctx.logger.warn = message => { warnings.push(String(message)) } + const gateA = { allows: () => true } + const gateB = { allows: () => true } + try { + await ctx.plugin(SessionStore) + const rowA = await mountRow(ctx, gateA, process.execPath) + await mountRow(ctx, gateB, 'Z:\\row-b-bash.exe') + const session = ctx.get('sessions').create('contested', { meta: { cwd: TEST_HOME } }) + session.mode = 'danger-full-access' + const agent = mint(ctx, session) + agents.push(agent) + await ctx.serial('agent/created', { agent }); await flush() + assert.ok(tools.get('ops_bash', agent), 'the first row owns the bash layer') + // Every pass while both gates allow must stand by instead of re-registering. + session.append('sandbox/mode', { mode: session.mode }); await flush() + session.append('sandbox/mode', { mode: session.mode }); await flush() + assert.deepEqual(warnings.filter(w => /bash (publication|restoration) failed/.test(w)), [], + 'a contested agent never re-registers the name another of this plugin\'s rows already holds') + // The owner unloads while the other row still claims the agent: only the + // turn boundary can hand the layer over, without any mode event (#873). + await rowA.dispose(); await flush() + assert.equal(tools.get('ops_bash', agent), undefined, 'the owner\'s unload withdraws the contested layer') + session.append('turn/start', { turn: 1 }); await flush() + assert.ok(tools.get('ops_bash', agent), 'the turn boundary lets the standing row take over') + assert.deepEqual(warnings.filter(w => /bash (publication|restoration) failed/.test(w)), [], + 'the whole contested lifecycle stays free of duplicate-registration failures') + } finally { await shutdown(ctx) } +}) + +await test('a lost name conflict is reported once and recovers at the turn boundary (#579, #873)', async () => { + const { ctx, tools, agents } = await setup() + const warnings = [] + ctx.logger.warn = message => { warnings.push(String(message)) } + const gate = { allows: () => true } + try { + await ctx.plugin(SessionStore) + const row = await mountRow(ctx, gate, process.execPath) + const session = ctx.get('sessions').create('conflict', { meta: { cwd: TEST_HOME } }) + session.mode = 'danger-full-access' + const agent = mint(ctx, session) + agents.push(agent) + await ctx.serial('agent/created', { agent }); await flush() + assert.ok(tools.get('ops_bash', agent), 'the layer is live before the conflict') + // The owner withdraws; a foreign same-layer registration takes the name. + gate.allows = () => false + session.append('sandbox/mode', { mode: session.mode }); await flush() + assert.equal(tools.get('ops_bash', agent), undefined, 'the owner withdrew') + const undoForeign = await registerForeign(agent) + gate.allows = () => true + session.append('sandbox/mode', { mode: session.mode }); await flush() + const conflicts = () => warnings.filter(w => /bash publication failed/.test(w)).length + const restorations = () => warnings.filter(w => /bash restoration failed/.test(w)).length + assert.equal(restorations(), 0, 'a same-name conflict never re-registers the identical conflicting name') + assert.equal(conflicts(), 1, 'one lost conflict is reported once') + // Passes at the same generation must not re-attempt the failed registration. + session.append('sandbox/mode', { mode: session.mode }); await flush() + session.append('sandbox/mode', { mode: session.mode }); await flush() + assert.equal(conflicts(), 1, 'later passes do not storm the registry with the same failure') + // A turn boundary retries: the conflicting foreign layer may be gone by then. + undoForeign() + session.append('turn/start', { turn: 1 }); await flush() + assert.ok(tools.get('ops_bash', agent), 'the turn boundary republishes the lost layer') + assert.deepEqual(warnings.filter(w => /append cannot reenter/.test(w)), [], + 'no reentry escapes into the host') + } finally { await shutdown(ctx) } +}) + +await test('a turn boundary re-asserts bash authority without a mode event (#873)', async () => { + const { ctx, tools, agents } = await setup() + const warnings = [] + ctx.logger.warn = message => { warnings.push(String(message)) } + try { + await ctx.plugin(SessionStore) + await mountRow(ctx, { allows: () => true }, process.execPath) + const session = ctx.get('sessions').create('turn-boundary', { meta: { cwd: TEST_HOME } }) + session.mode = 'danger-full-access' + const agent = mint(ctx, session) + agents.push(agent) + await ctx.serial('agent/created', { agent }); await flush() + assert.ok(tools.get('ops_bash', agent), 'the layer is live under permissive authority') + // Authority changed silently: the layer is stale until the next turn begins. + session.mode = 'read-only' + assert.ok(tools.get('ops_bash', agent), 'without an event nothing reconciles yet') + session.append('turn/start', { turn: 1 }); await flush() + assert.equal(tools.get('ops_bash', agent), undefined, + 'the turn boundary withdraws a layer whose authority is gone') + // turn/end is a closing edge, not one that grants authority. + session.mode = 'danger-full-access' + session.append('turn/end', { turn: 1, reason: { kind: 'stop' } }); await flush() + assert.equal(tools.get('ops_bash', agent), undefined, 'turn/end alone does not republish') + session.append('turn/start', { turn: 2 }); await flush() + assert.ok(tools.get('ops_bash', agent), 'the next turn boundary republishes under restored authority') + // The first request of the new turn sees the tool the boundary just restored. + const step = await prepareStep(ctx, agent) + try { + assert.ok(step.assembly.tools.some(tool => tool.name === 'ops_bash'), + 'the first request of the new turn includes ops_bash') + } finally { step.dispose() } + assert.deepEqual(warnings, [], 'the boundary repair reports no failure') + } finally { await shutdown(ctx) } +}) + +await test('creating several agents inside append windows defers every layer and lands them all (#579)', async () => { + const { ctx, tools, agents } = await setup() + const warnings = [] + ctx.logger.warn = message => { warnings.push(String(message)) } + try { + await ctx.plugin(SessionStore) + const component = await mountRow(ctx, { allows: () => true }, process.execPath) + const session = ctx.get('sessions').create('burst', { meta: { cwd: TEST_HOME } }) + session.mode = 'danger-full-access' + const created = [] + let writes = 0 + // A real registry notification appends a valid session event, as host observers can. + ctx.on('tools/change', () => { session.append('turn/start', { turn: ++writes }) }) + ctx.on('session/event', (seen, event) => { + if (seen !== session || event.type !== 'sandbox/mode') return + assert.throws(() => session.append('turn/start', { turn: 99 }), /append cannot reenter/) + const agent = mint(ctx, session) + agents.push(agent) + created.push(ctx.serial('agent/created', { agent })) + return new Promise(resolve => setImmediate(resolve)) + }) + session.append('sandbox/mode', { mode: session.mode }) + session.append('sandbox/mode', { mode: session.mode }) + session.append('sandbox/mode', { mode: session.mode }) + await Promise.all(created); await flush() + assert.equal(agents.length, 3, 'three agents were created inside append windows') + for (const agent of agents) assert.ok(tools.get('ops_bash', agent), 'every deferred layer landed') + assert.equal(writes, 3, 'each deferred publication appends exactly once, after its window') + assert.deepEqual(warnings.filter(w => /failed|reenter/.test(w)), [], 'no failure escapes the burst') + await component.dispose(); await flush() + for (const agent of agents) assert.equal(tools.get('ops_bash', agent), undefined, + 'unload withdraws every deferred layer') + } finally { await shutdown(ctx) } +}) + +report('session-idempotent-reconcile') diff --git a/test/settings-disclosure-e2e.test.mjs b/test/settings-disclosure-e2e.test.mjs new file mode 100644 index 00000000..b487327b --- /dev/null +++ b/test/settings-disclosure-e2e.test.mjs @@ -0,0 +1,200 @@ +/** D5 (#989): the settings-page choice reaches the real first round, end to end. + * + * The chain is the production one, twice over: the disclosure card's write + * operations go through the real host settings writer (profileFixture) and + * persist to the profile's cordis.patch.yml; the saved value then boots a + * fresh OPS-PTC session through the shipped apply with the real native + * worker, where the assertions read the settled first-round surface and run + * one real binding call. The full leg proves the first program can call the + * permitted business APIs with no loader; the untouched old profile keeps + * the two-step on-demand path letter for letter; switching the tier between + * saves applies to the next session. + */ +import { createScope, scopeOf, scopeParentOf } from '@deepseek-ai/dsh-scope' +import Subprocess from '@deepseek-ai/dsh-subprocess-local' +import Sandbox from '@deepseek-ai/dsh-sandbox-local' +import FileSystem from '@deepseek-ai/dsh-fs-local' +import NodePtc from '@deepseek-ai/dsh-ptc-runtime-node' +import fs from 'node:fs' +import path from 'node:path' +import { apply } from '../lib/index.js' +import { BASH_TOOL } from '../lib/policy.js' +import { disclosureOps } from '../lib/client/row-schema.js' +import { profileFixture } from './lib/settings-profile.mjs' +import { bootHost, shutdown } from './lib/host.mjs' +import { prepareStep, executeCall, selectExecutor } from './lib/executor.mjs' +import { assert, test, report, wait, waitFor, withTempDir } from './lib/harness.mjs' +import { WORKER, requireWorker } from './lib/boundary.mjs' + +requireWorker('settings-disclosure-e2e') + +let sequence = 0 +// The integration diagnostics row (ops_status, #753) and the Windows Bash tool +// are published beside the PTC tier and have their own suites; this enumeration +// is the PTC disclosure surface. +const names = (ctx, agent) => ctx.get('tools').definitions(agent, agent.prepared?.toolSet).map(tool => tool.name) + .filter(name => name !== 'ops_status' && name !== BASH_TOOL).sort() +const opsNames = (ctx, agent) => names(ctx, agent).filter(name => name.startsWith('ops_')) + +/** One turn boundary, the way the loop issues it. */ +async function step(ctx, agent) { + ctx.emit('session/event', agent.session, { type: 'step/end', turn: 1, step: ++sequence }) + agent.prepared?.dispose(); agent.prepared = await prepareStep(ctx, agent) +} + +async function scoped(parent, key, parentKey) { + let result + await parent.plugin({ inject: ['tools'], apply(inner) { + result = createScope(inner, key, parentKey ? { parent: parentKey } : undefined).ctx + } }) + return result +} + +/** A saved settings value for an ops-ptc-only deployment of the merged row. */ +const savedBase = () => ({ + global: {}, + discovery: { binaryPath: WORKER, required: true }, + minimal: { enabled: false }, standard: { enabled: false }, ptc: { enabled: false }, cordis: { enabled: false }, + 'ops-ptc': { enabled: true, binaryPath: WORKER, required: true, allowedTools: ['grep', 'stat'] }, +}) + +/** + * The real settings writer for one profile; the card's saves go through it. + * + * `value()` is the host settings view (the editable surface, volatile keys + * only); `applied()` is the config the loader hands the row after the save — + * the profile layers merged, YAML-only keys (the `allowedTools` cap, the + * worker path) included. Sessions boot from `applied()` because that is the + * production chain; the settings view alone would silently drop the pool cap. + */ +async function writer(label, base) { + let applied + const fixture = await profileFixture(label, { + Config: (await import('../lib/settings/merged.js')).Config, apply(_ctx, config) { applied = config }, + }, base) + return { + value: () => fixture.snapshot().value, + applied: () => applied, + save: async operations => { + const snapshot = fixture.snapshot() + await fixture.mutate(operations, snapshot.revision) + await fixture.ctx.loader.await() + return fixture.snapshot().value + }, + close: () => shutdown(fixture.ctx), + } +} + +/** Boot the shipped merged row from a saved settings value, then one session. */ +async function session(saved, cwd) { + const { ctx } = await bootHost({ withWorkingDirectory: true }) + await ctx.plugin(Subprocess); await ctx.plugin(Sandbox); await ctx.plugin(FileSystem); await ctx.plugin(NodePtc) + ctx.provide('sandboxPolicy', { resolve: ({ session } = {}) => ({ mode: session?.mode ?? 'danger-full-access', workspaceRoot: cwd, sessionId: session?.id }) }) + const presets = new Map(), agents = [] + ctx.provide('agents', { list: () => [...agents] }) + ctx.provide('agentPresets', { composedPreset: inner => presets.get(scopeParentOf(scopeOf(inner))) }) + await ctx.plugin({ name: 'disclosure-e2e-fixture', apply: inner => apply(inner, saved) }) + const id = 'disclosure-' + ++sequence, preset = await scoped(ctx, { preset: id }) + presets.set(scopeOf(preset), 'ops-ptc') + for (const name of ['read', 'glob', 'grep']) preset.tools.register({ name, description: name, + parameters: { type: 'object', properties: {} }, output: { schema: { type: 'object' }, render: () => [] }, execute: async () => ({}) }) + const agent = { id, session: { id, mode: 'danger-full-access', header: { cwd }, append() {}, read() { return [] } } } + agent.ctx = await scoped(preset, agent, scopeOf(preset)) + await agent.ctx.plugin({ inject: ['tools', 'systemPrompt'], apply: inner => { selectExecutor(inner, 'ptc') } }) + agents.push(agent) + await ctx.serial('agent/created', { agent }) + await step(ctx, agent) + return { ctx, agent } +} + +/** One real run_code program through the prepared step, as the loop issues it. */ +async function program(ctx, agent, code, label) { + const result = await executeCall(ctx, { agent, toolSet: agent.prepared.toolSet, name: 'run_code', + arguments: { description: label, code }, callId: 'disclosure-' + ++sequence, signal: new AbortController().signal }) + assert.equal(result.isError, false, JSON.stringify(result)) + return result.value.result +} + +/** Wait for the first round to settle, then return its ops_* names. */ +async function settled(ctx, agent, anchor) { + await waitFor(() => opsNames(ctx, agent).includes(anchor), { what: anchor + ' published' }) + await wait(100) + return opsNames(ctx, agent) +} + +/** The on-demand first round: resident inspect plus the loader, nothing else. */ +const ON_DEMAND_FIRST = ['ops_inspect_local_file', 'ops_load_sdk'] +/** The full first round for allowedTools=[grep,stat]: the pool plus result, no loader. */ +const FULL_FIRST = ['ops_grep', 'ops_inspect_local_file', 'ops_result', 'ops_stat'] + +await test('a full choice saved through the real writer boots a loader-less first round', () => withTempDir('disclosure-full', async cwd => { + fs.writeFileSync(path.join(cwd, 'fixture.txt'), 'needle in haystack\nplain second line\n') + const store = await writer('disclosure-e2e-full', savedBase()) + try { + assert.equal('disclosurePolicy' in store.value()['ops-ptc'], false, 'the profile starts old: no key') + // Exactly the operations the card's save submits, through the real host writer. + const stored = await store.save(disclosureOps(store.value(), 'full')) + assert.equal(stored.ptc.disclosurePolicy, 'full') + assert.equal(stored['ops-ptc'].disclosurePolicy, 'full') + // The save keeps the YAML-only pool cap the tier must never widen (#989). + assert.deepEqual(store.applied()['ops-ptc'].allowedTools, ['grep', 'stat']) + const { ctx, agent } = await session(store.applied(), cwd) + try { + assert.deepEqual(await settled(ctx, agent, 'ops_grep'), FULL_FIRST, 'the whole permitted pool in the first round') + assert.equal(opsNames(ctx, agent).includes('ops_load_sdk'), false, 'the full tier publishes no loader') + const value = await program(ctx, agent, "const stat = await tools.ops_stat({ path: 'fixture.txt' })\n" + + "const grep = await tools.ops_grep({ pattern: 'needle', path: '.' })\n" + + "return { statStatus: stat.status, grepStatus: grep.status, grepText: JSON.stringify(grep), loader: typeof tools.ops_load_sdk }", + 'First-program business calls') + assert.equal(value.statStatus, 'completed', JSON.stringify(value)) + assert.equal(value.grepStatus, 'completed', JSON.stringify(value)) + assert.match(value.grepText, /needle/, 'the real worker answered the first-program call') + assert.equal(value.loader, 'undefined') + } finally { await shutdown(ctx) } + } finally { await store.close() } +})) + +await test('the untouched old profile keeps the two-step on-demand path', () => withTempDir('disclosure-on-demand', async cwd => { + fs.writeFileSync(path.join(cwd, 'fixture.txt'), 'needle in haystack\n') + const store = await writer('disclosure-e2e-on-demand', savedBase()) + try { + const stored = store.value() + assert.equal('disclosurePolicy' in stored['ops-ptc'], false, 'an old profile is never migrated to full') + const { ctx, agent } = await session(stored, cwd) + try { + assert.deepEqual(await settled(ctx, agent, 'ops_load_sdk'), ON_DEMAND_FIRST, 'resident plus loader, two steps before the pool') + const value = await program(ctx, agent, "const picked = await tools.ops_load_sdk({ tools: ['grep', 'stat', 'result'] })\n" + + "return { picked: picked.status, earlyGrep: typeof tools.ops_grep }", 'Load the business set') + assert.equal(value.picked, 'completed', JSON.stringify(value)) + assert.equal(value.earlyGrep, 'undefined', 'a loaded API is callable only in the next program') + await step(ctx, agent) + await wait(100) + assert.deepEqual(opsNames(ctx, agent).filter(name => name !== 'ops_load_sdk'), FULL_FIRST, + 'after the loader step the set equals the full first round') + } finally { await shutdown(ctx) } + } finally { await store.close() } +})) + +await test('switching the tier between saves applies to the next session', () => withTempDir('disclosure-switch', async cwd => { + fs.writeFileSync(path.join(cwd, 'fixture.txt'), 'needle in haystack\n') + const store = await writer('disclosure-e2e-switch', savedBase()) + try { + await store.save(disclosureOps(store.value(), 'full')) + const first = await session(store.applied(), cwd) + try { + assert.deepEqual(await settled(first.ctx, first.agent, 'ops_grep'), FULL_FIRST, 'the full save reaches the session') + } finally { await shutdown(first.ctx) } + const back = await store.save(disclosureOps(store.value(), 'on-demand')) + assert.equal('disclosurePolicy' in back.ptc, false, 'switching back returns to the absent default') + assert.equal('disclosurePolicy' in back['ops-ptc'], false) + const second = await session(store.applied(), cwd) + try { + assert.deepEqual(await settled(second.ctx, second.agent, 'ops_load_sdk'), ON_DEMAND_FIRST, 'the next session runs the new tier') + const value = await program(second.ctx, second.agent, "return { loader: typeof tools.ops_load_sdk, grep: typeof tools.ops_grep }", 'Probe the switched surface') + assert.equal(value.loader, 'function') + assert.equal(value.grep, 'undefined', 'the pool is back behind the loader') + } finally { await shutdown(second.ctx) } + } finally { await store.close() } +})) + +report('settings-disclosure-e2e') diff --git a/test/settings-host.test.mjs b/test/settings-host.test.mjs index 97eecd8f..752e1440 100644 --- a/test/settings-host.test.mjs +++ b/test/settings-host.test.mjs @@ -2,7 +2,7 @@ import fs from 'node:fs' import path from 'node:path' import { Config as MergedConfig } from '../lib/settings/merged.js' -import { executorOps } from '../lib/client/row-schema.js' +import { disclosureOps, executorOps } from '../lib/client/row-schema.js' import { parse } from 'yaml' import { profileFixture } from './lib/settings-profile.mjs' // These suites exercise the pre-merge component modules directly. @@ -108,6 +108,35 @@ await test('real shell form persists executor selection without erasing ordinary } finally { await shutdown(ctx) } }) +await test('the merged row persists the disclosure tier, keeps it through other saves, and clears it back to the absent default', async () => { + let observed + const { ctx, snapshot, mutate, patchPath } = await profileFixture('ops-merged-disclosure', { + Config: MergedConfig, apply(_ctx, config) { observed = config }, + }, { 'ops-ptc': { enabled: true } }) + try { + const before = snapshot() + assert.equal('disclosurePolicy' in before.value['ops-ptc'], false, 'the old profile carries no key') + await mutate(disclosureOps(before.value, 'full'), before.revision) + await ctx.loader.await() + assert.equal(observed.ptc.disclosurePolicy, 'full') + assert.equal(observed['ops-ptc'].disclosurePolicy, 'full') + assert.match(fs.readFileSync(patchPath, 'utf8'), /disclosurePolicy: full/) + // An unrelated save keeps the tier (the page only writes what changed). + await mutate([{ op: 'set', path: ['panel', 'enabled'], value: true }], snapshot().revision) + await ctx.loader.await() + assert.equal(observed.panel.enabled, true) + assert.equal(observed.ptc.disclosurePolicy, 'full') + assert.equal(observed['ops-ptc'].disclosurePolicy, 'full') + // Switching back returns both rows to the absent default, other keys intact. + await mutate(disclosureOps(snapshot().value, 'on-demand'), snapshot().revision) + await ctx.loader.await() + assert.equal('disclosurePolicy' in snapshot().value.ptc, false) + assert.equal('disclosurePolicy' in snapshot().value['ops-ptc'], false) + assert.doesNotMatch(fs.readFileSync(patchPath, 'utf8'), /disclosurePolicy/) + assert.equal(observed.panel.enabled, true, 'other choices survive the tier switch') + } finally { await shutdown(ctx) } +}) + await test('the actual shell component starts and reloads through the profile adapter without normalizing twice', async () => { const { ctx, snapshot, mutate, patchPath } = await profileFixture('actual-shell', Shell, { promptPolicy: false, publishBashTool: false }) try { diff --git a/test/settings-rows.test.mjs b/test/settings-rows.test.mjs index 3c4138de..ee082ffb 100644 --- a/test/settings-rows.test.mjs +++ b/test/settings-rows.test.mjs @@ -4,10 +4,10 @@ import fs from 'node:fs' import os from 'node:os' import path from 'node:path' import { companionVersion } from '../lib/settings/companion.js' -import { GROUPS, ROW_ID, groupsFor, bundleGates, changedFields, executorDiffers, executorOps, flatKey, readExecutor, readRowValues } from '../lib/client/row-schema.js' +import { GROUPS, ROW_ID, groupsFor, bundleGates, changedFields, executorDiffers, executorOps, flatKey, readExecutor, readRowValues, readDisclosure, disclosureDiffers, disclosureOps } from '../lib/client/row-schema.js' import { hasStatus, statusView } from '../lib/client/detail.js' import { en } from '../lib/client/locales.js' -import { Config, effectiveMode, resolveMergedConfig } from '../lib/settings/merged.js' +import { Config, effectiveMode, hostConfig, resolveMergedConfig } from '../lib/settings/merged.js' import { readRowValues as draftFor } from '../lib/client/row-schema.js' const t = key => en[key] const SWITCHES = ['bash', 'hidePwsh', 'hideOfficialTools', 'dynamicTools'] @@ -25,9 +25,12 @@ await test('the page matches the host descriptor, and the opt-in switches share const group = GROUPS.find(entry => entry.id === section) const expected = section === 'minimal' ? SWITCHES : ['bash', 'hidePwsh', 'hideOfficialTools', 'residentTools', 'dynamicTools'] assert.deepEqual(group.fields.map(spec => spec.path[1]), expected, section) - // bashPath belongs to the executor card rather than the boolean switch grid. - assert.deepEqual([...group.fields.map(spec => spec.path[1]), 'enabled', 'bashPath'].sort(), Object.keys(Config.dict[section].dict).sort(), section) + // bashPath belongs to the executor card; the two PTC rows' disclosurePolicy + // belongs to the disclosure card — neither is a boolean switch above. + const cardKeys = ['bashPath', ...(section === 'ptc' || section === 'ops-ptc' ? ['disclosurePolicy'] : [])] + assert.deepEqual([...group.fields.map(spec => spec.path[1]), 'enabled', ...cardKeys].sort(), Object.keys(Config.dict[section].dict).sort(), section) assert.equal(Config.dict[section].dict.bashPath.meta.volatile, true) + if (section === 'ptc' || section === 'ops-ptc') assert.equal(Config.dict[section].dict.disclosurePolicy.meta.volatile, true) } // compat/panel/tui used to be three single-switch cards; they are one card now. const other = GROUPS.find(group => group.id === 'other') @@ -137,6 +140,35 @@ await test('the executor card reads the saved rows and writes exactly one choice assert.equal(executorDiffers({ engine: 'git', path: 'C:/a/bash.exe' }, { ptc: { bashPath: 'C:/a/bash.exe' } }), false) }) +await test('the disclosure card reads the saved rows and writes exactly one tier', () => { + // Old profiles (key absent) and an explicit on-demand both read as the default rung. + assert.deepEqual(readDisclosure({}), 'on-demand') + assert.equal(readDisclosure({ ptc: { disclosurePolicy: 'on-demand' } }), 'on-demand') + assert.equal(readDisclosure({ ptc: { disclosurePolicy: 'full' }, 'ops-ptc': { disclosurePolicy: 'full' } }), 'full') + assert.equal(readDisclosure({ 'ops-ptc': { disclosurePolicy: 'full' } }), 'mixed', 'a row that disagrees is named, never reduced to a favourite') + // Full writes both PTC rows; on-demand returns them to the absent default. + assert.deepEqual(disclosureOps({}, 'full'), [ + { op: 'set', path: ['ptc', 'disclosurePolicy'], value: 'full' }, + { op: 'set', path: ['ops-ptc', 'disclosurePolicy'], value: 'full' }, + ]) + assert.deepEqual(disclosureOps({ ptc: { disclosurePolicy: 'full' }, 'ops-ptc': { disclosurePolicy: 'full' } }, 'full'), []) + assert.deepEqual(disclosureOps({ 'ops-ptc': { disclosurePolicy: 'full' } }, 'on-demand'), [{ op: 'unset', path: ['ops-ptc', 'disclosurePolicy'] }]) + assert.deepEqual(disclosureOps({}, 'on-demand'), []) + assert.equal(disclosureDiffers('full', {}), true) + assert.equal(disclosureDiffers('on-demand', {}), false, 'an untouched old profile is not dirty') + assert.equal(disclosureDiffers('full', { ptc: { disclosurePolicy: 'full' }, 'ops-ptc': { disclosurePolicy: 'full' } }), false) + // Old configs are never migrated to full: the resolved tier and every save + // path keep the absent key (#989 acceptance B). + assert.equal(resolveMergedConfig({})['ops-ptc'].config.disclosurePolicy, 'on-demand') + assert.equal('disclosurePolicy' in hostConfig({})['ops-ptc'], false) + // A saved full choice survives reopening and saving another switch (D). + const saved = { ptc: { disclosurePolicy: 'full' }, 'ops-ptc': { disclosurePolicy: 'full' } } + assert.equal(readDisclosure(saved), 'full', 'reopening reads the saved tier back') + const draft = readRowValues(saved) + draft['panel.enabled'] = true + assert.deepEqual(changedFields(draft, saved), [{ op: 'set', path: ['panel', 'enabled'], value: true }], 'the tier is not rewritten by an unrelated save') +}) + await test('the status section belongs to the bundle and to the one row', () => { const pkg = { name: 'dsh-ops', installed: true, enabled: true, rows: [] } // Other plugins' pages mount the same slot; the section must stay off them. diff --git a/test/settings-ui.test.mjs b/test/settings-ui.test.mjs index 9114b838..e73a5e99 100644 --- a/test/settings-ui.test.mjs +++ b/test/settings-ui.test.mjs @@ -141,6 +141,80 @@ await test('an invalid typed path blocks the save and names the host reason', as } finally { await view.close() } }) +await test('the disclosure card picks the tier, cancels edits, and never writes an untouched old profile', async () => { + const writes = [] + const form = { state: ready({}), mutate: async ops => { writes.push(ops); return true } } + const view = await mount(RowForm, { form, Switch: TestSwitch }) + try { + const card = group(view.container, en.disclosureTitle) + assert.ok(card, 'the disclosure card renders') + const radios = card.querySelectorAll('input[type=radio]') + assert.equal(radios.length, 2) + assert.equal(radios[0].checked, true, 'an old profile shows the on-demand default') + assert.equal(radios[1].checked, false) + // The bundle switch keeps its veto: while dynamic tools are off the card + // names the suspension instead of hiding the choice. + await click(switchIn(group(view.container, en.groupGlobal), en.dynamicTools)) + assert.ok(card.textContent.includes(en.disclosureSuspended), 'the suspended full tier is named') + await click(button(view.container, en.reset)) + assert.equal(card.textContent.includes(en.disclosureSuspended), false, 'reset restores the running tier') + // Cancel before saving: reset restores the saved tier and writes nothing. + await click(radios[1]) + await click(button(view.container, en.reset)) + assert.equal(card.querySelectorAll('input[type=radio]')[0].checked, true, 'reset restores the saved tier') + assert.deepEqual(writes, [], 'a cancelled edit is never written') + // Saving the full tier writes both PTC rows once. + await click(card.querySelectorAll('input[type=radio]')[1]) + await click(button(view.container, en.save)) + assert.deepEqual(writes[0], [ + { op: 'set', path: ['ptc', 'disclosurePolicy'], value: 'full' }, + { op: 'set', path: ['ops-ptc', 'disclosurePolicy'], value: 'full' }, + ]) + // Disagreeing saved rows are named, never reduced to a favourite. + const mixed = { state: ready({ 'ops-ptc': { disclosurePolicy: 'full' } }, 'v2'), mutate: async ops => { writes.push(ops); return true } } + await view.render({ form: mixed, Switch: TestSwitch }) + const mixedCard = group(view.container, en.disclosureTitle) + assert.ok(mixedCard.textContent.includes(en.disclosureMixed), 'a disagreement is named') + assert.equal(mixedCard.querySelectorAll('input[type=radio]')[0].checked, false, 'mixed checks neither radio') + assert.equal(mixedCard.querySelectorAll('input[type=radio]')[1].checked, false) + } finally { await view.close() } +}) + +await test('a saved full tier reads back through the real host and other saves keep it', async () => { + let observed + const fixture = await profileFixture('disclosure-page', { + Config: (await import('../lib/settings/merged.js')).Config, apply(_ctx, config) { observed = config }, + }, {}) + let pending + const form = { + state: ready(fixture.snapshot().value, fixture.snapshot().revision), + mutate(ops, revision) { + return pending = (async () => { + await fixture.mutate(ops, revision); await fixture.ctx.loader.await() + form.state = ready(fixture.snapshot().value, fixture.snapshot().revision) + return true + })() + }, + } + const view = await mount(RowForm, { form, Switch: TestSwitch }) + try { + const card = group(view.container, en.disclosureTitle) + await click(card.querySelectorAll('input[type=radio]')[1]) + await click(button(view.container, en.save)); await act(async () => { await pending }) + assert.equal(observed.ptc.disclosurePolicy, 'full') + assert.equal(observed['ops-ptc'].disclosurePolicy, 'full') + // Reopening the page reads the saved choice back. + await view.render({ form, Switch: TestSwitch }) + assert.equal(group(view.container, en.disclosureTitle).querySelectorAll('input[type=radio]')[1].checked, true) + // Saving another card's switch never loses the tier. + await click(switchIn(group(view.container, en.groupOther), en.featurePanel)) + await click(button(view.container, en.save)); await act(async () => { await pending }) + assert.equal(observed.panel.enabled, true) + assert.equal(observed.ptc.disclosurePolicy, 'full', 'the saved tier survives an unrelated save') + assert.equal(observed['ops-ptc'].disclosurePolicy, 'full') + } finally { await view.close(); await shutdown(fixture.ctx) } +}) + await test('rendered preset hints describe default conditional PowerShell hiding', async () => { const form = { state: ready({}), mutate: async () => true } const view = await mount(RowForm, { form, Switch: TestSwitch }) diff --git a/test/shells-full-access-confinement.test.mjs b/test/shells-full-access-confinement.test.mjs new file mode 100644 index 00000000..4aea9fca --- /dev/null +++ b/test/shells-full-access-confinement.test.mjs @@ -0,0 +1,64 @@ +/** Full access projects straight through; confinement is a confined-mode fact (#581). */ +import path from 'node:path' +import { startSessionBash } from '../lib/shells/session-execution.js' +import { NIUBASH_PACKAGE_PATH } from '../lib/cli/niubash-payload.js' +import { syntheticNiubashTree } from './lib/niubash-distribution-fixture.mjs' +import { assert, report, skipSuite, test, withTempDir } from './lib/harness.mjs' + +if (process.platform !== 'win32') skipSuite('shells-full-access-confinement', 'requires the Windows Niubash runtime branch') + +await test('full access spawns directly and never confines; confined modes route through confine', async () => { + await withTempDir('niu-full-access', async root => { + const source = syntheticNiubashTree(path.join(root, 'package', NIUBASH_PACKAGE_PATH)) + const scenarios = [ + { label: 'the Niubash runtime', file: path.join(source, 'niu.exe') }, + { label: 'a plain bash payload', file: path.join(root, 'bash.exe') }, + ] + for (const { label, file } of scenarios) { + let argv0, confineCalls = 0, spawnCalls = 0 + const run = mode => { + const policy = { mode, workspaceRoot: root } + const agent = { id: 'niu-full-access', session: { id: 'niu-full-access', header: { cwd: root } } } + const ctx = { + get(name) { + if (name === 'sandboxPolicy') return { resolve: () => policy } + if (name === 'sandbox') return { confine: async argv => { confineCalls++; return { argv, enforcement: 'partial' } } } + if (name === 'subprocess') return { + spawn(spec) { + spawnCalls++; argv0 = spec.argv[0] + const stream = text => ({ readFrom: () => ({ text, nextOffset: text.length, lossy: false }) }) + return { done: Promise.resolve({ exitCode: 0, signal: null }), collected: { stdout: stream('ok'), stderr: stream('') } } + }, + } + return undefined + }, + } + const execution = startSessionBash({ ctx, bash: { file, source: 'config' }, active: () => true }, + { command: 'true', workdir: root }, { agent, signal: new AbortController().signal }, policy, policy, {}, {}) + return execution.result() + } + // Full access keeps the caller's own token: no confinement, no runner downgrade (#581). + const full = await run('danger-full-access') + assert.equal(full.error, undefined, `${label}: ${JSON.stringify(full)}`) + assert.equal(full.status, 'completed', `${label}: ${JSON.stringify(full)}`) + assert.equal(spawnCalls, 1, `${label}: full access spawned once`) + assert.equal(confineCalls, 0, `${label}: full access never asks the sandbox to confine`) + if (file.endsWith('niu.exe')) assert.equal(path.basename(argv0), 'niu.exe', `${label}: the prepared runtime executes`) + else assert.equal(argv0, file, `${label}: the configured payload executes unchanged`) + // A confined mode must pass through the sandbox before the spawn; a plain bash.exe + // that is not the supported MSYS payload is refused by confinedBash instead, which is + // exactly the call full access must never make. + const confined = await run('workspace-write') + if (file.endsWith('niu.exe')) { + assert.equal(confined.error, undefined, `${label}: ${JSON.stringify(confined)}`) + assert.equal(confineCalls, 1, `${label}: the confined mode routed through confine exactly once`) + assert.equal(spawnCalls, 2, `${label}: the confined mode spawned once more`) + } else { + assert.match(confined.error ?? '', /EXECUTOR_UNAVAILABLE:.*restricted modes/, `${label}: a foreign payload is refused in confined modes`) + assert.equal(confineCalls, 0, `${label}: the refusal precedes confinement`) + assert.equal(spawnCalls, 1, `${label}: the refusal spawned nothing`) + } + } + }) +}) +report('shells-full-access-confinement') diff --git a/test/shells-job-promoted-guidance.test.mjs b/test/shells-job-promoted-guidance.test.mjs new file mode 100644 index 00000000..1e8e2955 --- /dev/null +++ b/test/shells-job-promoted-guidance.test.mjs @@ -0,0 +1,36 @@ +/** + * The promoted Bash render is the model's decision point for a command that + * outlived its foreground timeout (#814): job_output and job_kill are the only + * two handles left, and a job kill at the host layer terminates the whole + * process tree. The defect behind the issue lives in the host's subprocess and + * job layer (an inherited pipe keeps a daemon inside the tree, and a tree kill + * takes out daemons the command started, e.g. an adb server); this repository + * cannot fix the host, so its render must at least state what a job kill + * reaches and how to keep daemons out of the blast radius. + */ +import { assert, test, report } from './lib/harness.mjs' +import { renderHostBash } from '../lib/shells/host-contract.js' + +await test('the promoted render states that job_kill terminates the whole process tree', () => { + const value = { + kind: 'promoted', jobId: 'job-7', label: 'adb start-server', timeoutMs: 120000, + output: '* daemon not running; starting it now', + } + const text = renderHostBash(value) + // The existing handles stay named, and the moved-to-job line keeps its shape. + assert.match(text, /still running after 120000ms; moved to background job job-7/) + assert.match(text, /job_output/) + assert.match(text, /job_kill/) + // #814: the render must say what a job kill reaches before the model uses it. + assert.match(text, /whole process tree/) + // ... and which victims that implies, plus how to avoid them next time. + assert.match(text, /daemon/) + assert.match(text, /run_in_background/) +}) + +await test('the background render keeps its exact one-line shape', () => { + const text = renderHostBash({ kind: 'background', jobId: 'j-1', label: 'watcher' }) + assert.equal(text, 'started background job j-1 (\"watcher\")') +}) + +report('shells-job-promoted-guidance') diff --git a/test/shells-msys-shared-section.test.mjs b/test/shells-msys-shared-section.test.mjs new file mode 100644 index 00000000..507f631b --- /dev/null +++ b/test/shells-msys-shared-section.test.mjs @@ -0,0 +1,44 @@ +/** A shared MSYS section failure reports as the runtime failure it is (#409). */ +import path from 'node:path' +import { startSessionBash } from '../lib/shells/session-execution.js' +import { NIUBASH_PACKAGE_PATH } from '../lib/cli/niubash-payload.js' +import { syntheticNiubashTree } from './lib/niubash-distribution-fixture.mjs' +import { assert, report, skipSuite, test, withTempDir } from './lib/harness.mjs' + +if (process.platform !== 'win32') skipSuite('shells-msys-shared-section', 'requires the Windows Niubash runtime branch') + +await test('a CreateFileMapping shared-section failure is EXECUTOR_UNAVAILABLE, never a denial', async () => { + await withTempDir('niu-section', async root => { + const source = syntheticNiubashTree(path.join(root, 'package', NIUBASH_PACKAGE_PATH)) + const policy = { mode: 'workspace-write', workspaceRoot: root } + const agent = { id: 'niu-section', session: { id: 'niu-section', header: { cwd: root } } } + // msys-2.0.dll prints exactly "CreateFileMapping %W, %E. Terminating." with %E as + // "Win32 error %d": a restricted token cannot reopen a per-user section an unrestricted + // MSYS process created first, because that object's DACL has no restricted-logon ACE (#409). + const sectionFailure = 'bash: CreateFileMapping S-1-5-21-500-1001.1, Win32 error 5. Terminating.' + const ctx = { + get(name) { + if (name === 'sandboxPolicy') return { resolve: () => policy } + if (name === 'sandbox') return { confine: async argv => ({ argv, enforcement: 'partial', denialSignatures: ['permission denied'] }) } + if (name === 'subprocess') return { + spawn() { + const stream = text => ({ readFrom: () => ({ text, nextOffset: text.length, lossy: false }) }) + return { done: Promise.resolve({ exitCode: 126, signal: null }), collected: { stdout: stream(''), stderr: stream(sectionFailure) } } + }, + } + return undefined + }, + } + const execution = startSessionBash({ ctx, bash: { file: path.join(source, 'niu.exe'), source: 'bundled' }, active: () => true }, + { command: 'true', workdir: root }, { agent, signal: new AbortController().signal }, policy, policy, {}, {}) + const value = await execution.result() + assert.match(value.stderr.text, /CreateFileMapping/, 'the simulated stderr reached the result') + // The shared-section conflict belongs to the restricted-token runtime, exactly like the + // signal-pipe failure, so the model must not read it as a user authorization denial. + assert.match(value.error ?? '', /EXECUTOR_UNAVAILABLE/) + assert.equal(value.sandbox.runnerFailed, true) + assert.equal(value.sandbox.denied, false, 'a shared-section conflict is not a user authorization denial') + assert.equal(value.exitCode, 126) + }) +}) +report('shells-msys-shared-section') diff --git a/test/shells-niubash-prepared-boundary.test.mjs b/test/shells-niubash-prepared-boundary.test.mjs new file mode 100644 index 00000000..3726fa55 --- /dev/null +++ b/test/shells-niubash-prepared-boundary.test.mjs @@ -0,0 +1,49 @@ +/** The session layer arms the prepared-root boundary only for confined modes (#870). */ +import path from 'node:path' +import { startSessionBash } from '../lib/shells/session-execution.js' +import { niubashPreparedRoot } from '../lib/niubash/runtime.js' +import { NIUBASH_PACKAGE_PATH } from '../lib/cli/niubash-payload.js' +import { syntheticNiubashTree } from './lib/niubash-distribution-fixture.mjs' +import { assert, report, skipSuite, TEST_HOME, test, withTempDir } from './lib/harness.mjs' + +if (process.platform !== 'win32') skipSuite('shells-niubash-prepared-boundary', 'requires the Windows Niubash runtime branch') + +await test('full access executes with the prepared root inside the workspace; confined modes refuse it', async () => { + await withTempDir('niu-boundary', async root => { + const source = syntheticNiubashTree(path.join(root, 'package', NIUBASH_PACKAGE_PATH)) + let argv0, spawnCalls = 0 + const run = mode => { + // The test process DSH_HOME (TEST_HOME) is itself the workspace root, so the prepared + // root it selects is contained in the workspace exactly like a default ~/.dsh inside + // a home workspace (#870). + const policy = { mode, workspaceRoot: TEST_HOME } + const agent = { id: 'niu-boundary', session: { id: 'niu-boundary', header: { cwd: root } } } + const ctx = { + get(name) { + if (name === 'sandboxPolicy') return { resolve: () => policy } + if (name === 'sandbox') return { confine: async argv => ({ argv, enforcement: 'partial' }) } + if (name === 'subprocess') return { + spawn(spec) { + spawnCalls++; argv0 = spec.argv[0] + const stream = text => ({ readFrom: () => ({ text, nextOffset: text.length, lossy: false }) }) + return { done: Promise.resolve({ exitCode: 0, signal: null }), collected: { stdout: stream('ok'), stderr: stream('') } } + }, + } + return undefined + }, + } + const execution = startSessionBash({ ctx, bash: { file: path.join(source, 'niu.exe'), source: 'bundled' }, active: () => true }, + { command: 'true', workdir: root }, { agent, signal: new AbortController().signal }, policy, policy, {}, {}) + return execution.result() + } + const full = await run('danger-full-access') + assert.equal(full.error, undefined, JSON.stringify(full)) + assert.equal(full.status, 'completed', JSON.stringify(full)) + assert.equal(spawnCalls, 1, 'full access spawned the prepared runtime') + assert.equal(argv0, path.join(niubashPreparedRoot(), 'niu.exe'), 'full access executes the prepared runtime') + const denied = await run('workspace-write') + assert.match(denied.error, /EXECUTOR_UNAVAILABLE:.*outside the session workspace/) + assert.equal(spawnCalls, 1, 'the confined refusal spawned nothing') + }) +}) +report('shells-niubash-prepared-boundary') diff --git a/test/workspace-reparse-ready.test.mjs b/test/workspace-reparse-ready.test.mjs new file mode 100644 index 00000000..87a9d752 --- /dev/null +++ b/test/workspace-reparse-ready.test.mjs @@ -0,0 +1,142 @@ +import fs from 'node:fs' +import path from 'node:path' +import { spawnSync } from 'node:child_process' +import { fileURLToPath } from 'node:url' +import { assert, test, report, withTempDir, TEST_HOME } from './lib/harness.mjs' +import { preparationEnvironment } from '../lib/workspace-preparation.js' + +/** + * Issue #744: a OneDrive folder carries IO_REPARSE_TAG_CLOUD, so the workspace + * bootstrap's reparse-point walk refused it before the DACL decision could + * say the folder was already ready, and every workspace-write tool failed. + * The reparse check belongs to the preparation branch alone — the only path + * that writes — and there it must let the cloud-files family through (they are + * ordinary directory entries to the ACL layer) and fail every other tag closed + * with the tag named. Cloud tags cannot be synthesized without the cloud + * filter API, so the family rule is pinned on the pure decision function and + * the walk itself is pinned with a junction, which is a real mount-point tag. + */ +if (process.platform === 'win32') await windowsReparseChecks() +else console.log(' Windows reparse fixture unavailable: real DACL and PowerShell checks require Windows') + +async function windowsReparseChecks() { + +const helper = fileURLToPath(new URL('../bin/workspace-acl.ps1', import.meta.url)) +const preparation = fileURLToPath(new URL('../bin/prepare-workspace.ps1', import.meta.url)) +const powershell = path.join(process.env.SystemRoot ?? 'C:\\Windows', 'System32/WindowsPowerShell/v1.0/powershell.exe') +const quote = (text) => "'" + text.replaceAll("'", "''") + "'" + +/** Run PowerShell 5.1 the way production does: scrubbed environment, bypassed policy. */ +function ps(code) { + return spawnSync(powershell, + ['-NoProfile', '-NonInteractive', '-ExecutionPolicy', 'Bypass', '-EncodedCommand', Buffer.from(code, 'utf16le').toString('base64')], + { encoding: 'utf8', env: preparationEnvironment(), windowsHide: true, timeout: 30_000 }) +} + +function psValue(code) { + const value = ps(code) + assert.equal(value.status, 0, value.stderr) + return value.stdout.trim() +} + +/** Run the shipped bootstrap exactly as production launches it. */ +function runScript(root, backups) { + return spawnSync(powershell, + ['-NoProfile', '-NonInteractive', '-ExecutionPolicy', 'Bypass', '-File', preparation, '-Workspace', root, '-BackupDir', backups], + { encoding: 'utf8', env: preparationEnvironment(), windowsHide: true, timeout: 30_000 }) +} + +function receipts(backups) { + return fs.existsSync(backups) ? fs.readdirSync(backups).length : 0 +} + +/** A junction under the temporary home: a real mount point (tag 0xA0000003). */ +function junctionFixture(base) { + const target = path.join(base, 'target') + fs.mkdirSync(target) + const root = path.join(base, 'root') + const made = spawnSync('cmd.exe', ['/c', 'mklink', '/J', root, target], { encoding: 'utf8' }) + assert.equal(made.status, 0, made.stderr) + return root +} + +await test('reparse tag classification lets the cloud-files family through and fails every other tag closed', () => { + const state = (tag) => psValue(`$ErrorActionPreference='Stop' +. ${quote(helper)} +(Get-DshOpsReparseTagDecision -Tag ${tag}).State`) + // Decimal literals on purpose: Windows PowerShell 5.1 folds hex literals in + // the 0x90000000-0xFFFFFFFF band into a negative Int32, which cannot convert + // to the function's [uint32] parameter. 2415919130 is IO_REPARSE_TAG_CLOUD, + // 2415923226 CLOUD_ROOT, 2415927322 CLOUD_ON_DEMAND: the proven cloud-files + // family has the exact shape 0x9000?01A, and only that family is allowed — + // because a placeholder keeps a real directory entry behind the point, so an + // ACL write targets a real object. 2415980570 (0x9000F01A) shares that family + // shape under the mask. + assert.equal(state(2415919130), 'cloud') + assert.equal(state(2415923226), 'cloud') + assert.equal(state(2415927322), 'cloud') + assert.equal(state(2415980570), 'cloud') + // A high nibble of 9 alone is NOT enough (#744 review): 2684354559 is + // 0x9FFFFFFF and 2415963085 is 0x9000ABCD, whose low 16 bits are not the + // ?01A family shape. Unproven tags must fail closed before a DACL write. + assert.equal(state(2684354559), 'unsupported') + assert.equal(state(2415963085), 'unsupported') + // 0xA0000003 is a mount point and 0xA000000C a symlink; unknown tags like + // 0x8000001B redirect or transform the view and must fail closed when + // preparation is about to write the DACL. + assert.equal(state(2684354563), 'unsupported') + assert.equal(state(2684354572), 'unsupported') + assert.equal(state(2147483675), 'unsupported') +}) + +await test('a junction workspace with an effective WRITE_OWNER takes the ready path without a reparse refusal (#744)', async () => { + await withTempDir('prepare-reparse-ready', async (base) => { + const root = junctionFixture(base) + const backups = path.join(TEST_HOME, 'prepare-reparse-ready-backups') + const value = runScript(root, backups) + assert.equal(value.status, 0, value.stderr) + assert.equal(value.stdout.trim(), 'ready') + assert.equal(receipts(backups), 0, 'the ready path writes no receipt') + }) +}) + +await test('a junction workspace that still needs the ACE fails closed and names its reparse tag (#744)', async () => { + await withTempDir('prepare-reparse-need', async (base) => { + const root = junctionFixture(base) + const backups = path.join(TEST_HOME, 'prepare-reparse-need-backups') + // Get-Acl reads the junction object's own security descriptor: it opens the + // reparse point itself, which is why icacls — which follows the point — could + // not prepare this fixture. Set-Acl uses the same handle, so one call both + // takes ownership for the caller and installs a protected DACL that grants + // Modify but never WRITE_OWNER. The decision must then be needs-preparation: + // the one branch that writes, and therefore the branch allowed to read tags. + const sid = psValue('[Security.Principal.WindowsIdentity]::GetCurrent().User.Value') + const prepared = psValue(`$ErrorActionPreference='Stop' +$sid = New-Object Security.Principal.SecurityIdentifier(${quote(sid)}) +$acl = New-Object Security.AccessControl.DirectorySecurity +$acl.SetAccessRuleProtection($true, $false) +$acl.SetOwner($sid) +$acl.AddAccessRule((New-Object Security.AccessControl.FileSystemAccessRule($sid, [Security.AccessControl.FileSystemRights]::Modify, 'None', 'None', 'Allow'))) +Set-Acl -LiteralPath ${quote(root)} -AclObject $acl +'ok'`) + assert.equal(prepared, 'ok') + const state = psValue(`$ErrorActionPreference='Stop' +. ${quote(helper)} +$acl=Get-Acl -LiteralPath ${quote(root)} +$identity=[Security.Principal.WindowsIdentity]::GetCurrent() +$principal=[Security.Principal.WindowsPrincipal]::new($identity) +(Get-DshOpsWorkspaceDecision -Acl $acl -Identity $identity -Principal $principal).State`) + assert.equal(state, 'needs-preparation') + const value = runScript(root, backups) + assert.notEqual(value.status, 0, JSON.stringify(value.stdout)) + // The refusal must name the actual tag, so the operator can tell a cloud + // placeholder from a mount point instead of a blanket reparse rejection. + assert.match(value.stderr, /reparse point/) + assert.match(value.stderr, /0xA0000003/) + assert.equal(receipts(backups), 0, 'a refused preparation writes no receipt') + }) +}) + +} + +report('workspace-reparse-ready') diff --git a/test/workspace-wsl-redirector.test.mjs b/test/workspace-wsl-redirector.test.mjs new file mode 100644 index 00000000..ac0eaa13 --- /dev/null +++ b/test/workspace-wsl-redirector.test.mjs @@ -0,0 +1,115 @@ +import fs from 'node:fs' +import path from 'node:path' +import { spawnSync } from 'node:child_process' +import { fileURLToPath } from 'node:url' +import { assert, test, report, wait, TEST_HOME } from './lib/harness.mjs' +import { preparationEnvironment } from '../lib/workspace-preparation.js' + +/** + * Issue #669: \\wsl.localhost and \\wsl$ are the WSL 9P redirector, and the + * filesystem behind it exposes no NTFS security descriptor at all. The + * bootstrap's Get-Acl therefore threw on every WSL workspace and all of + * run_code died. The fix is a prefix decision, not a catch that swallows ACL + * errors: those two Microsoft-fixed prefixes answer ready (authority there + * is the distribution's own uid/gid mapping), and every other path keeps its + * existing behaviour, where a real Get-Acl failure still fails closed. + */ +if (process.platform === 'win32') await windowsWslChecks() +else console.log(' Windows WSL redirector fixture unavailable: real DACL and PowerShell checks require Windows') + +async function windowsWslChecks() { + +const helper = fileURLToPath(new URL('../bin/workspace-acl.ps1', import.meta.url)) +const preparation = fileURLToPath(new URL('../bin/prepare-workspace.ps1', import.meta.url)) +const powershell = path.join(process.env.SystemRoot ?? 'C:\\Windows', 'System32/WindowsPowerShell/v1.0/powershell.exe') +const wsl = path.join(process.env.SystemRoot ?? 'C:\\Windows', 'System32', 'wsl.exe') +const quote = (text) => "'" + text.replaceAll("'", "''") + "'" + +/** Run PowerShell 5.1 the way production does: scrubbed environment, bypassed policy. */ +function ps(code) { + return spawnSync(powershell, + ['-NoProfile', '-NonInteractive', '-ExecutionPolicy', 'Bypass', '-EncodedCommand', Buffer.from(code, 'utf16le').toString('base64')], + { encoding: 'utf8', env: preparationEnvironment(), windowsHide: true, timeout: 30_000 }) +} + +function psValue(code) { + const value = ps(code) + assert.equal(value.status, 0, value.stderr) + return value.stdout.trim() +} + +/** Run the shipped bootstrap exactly as production launches it. */ +function runScript(root, backups) { + return spawnSync(powershell, + ['-NoProfile', '-NonInteractive', '-ExecutionPolicy', 'Bypass', '-File', preparation, '-Workspace', root, '-BackupDir', backups], + { encoding: 'utf8', env: preparationEnvironment(), windowsHide: true, timeout: 30_000 }) +} + + + +await test('NTFS state classification separates the two WSL redirector prefixes from everything else', () => { + const state = (value) => psValue(`$ErrorActionPreference='Stop' +. ${quote(helper)} +(Get-DshOpsWorkspaceNtfsState -Path ${quote(value)})`) + // Microsoft-fixed redirector spellings, case-insensitive, incl. the \\?\UNC form. + assert.equal(state('\\\\wsl.localhost\\Ubuntu\\home\\pc'), 'wsl-redirector') + assert.equal(state('\\\\wsl$\\Ubuntu\\tmp'), 'wsl-redirector') + assert.equal(state('\\\\?\\UNC\\wsl.localhost\\Ubuntu\\tmp'), 'wsl-redirector') + assert.equal(state('\\\\WSL.LOCALHOST\\Ubuntu\\tmp'), 'wsl-redirector') + // Everything else is NTFS business as usual: a real ACL error there must + // still fail closed, not be absorbed into a redirector allowance. + assert.equal(state('D:\\normal\\path'), 'ntfs') + assert.equal(state('\\\\server\\share\\file'), 'ntfs') + assert.equal(state('\\\\wsl.localhost2\\a'), 'ntfs') + assert.equal(state('\\\\wslx\\Ubuntu\\a'), 'ntfs') + assert.equal(state('wsl'), 'ntfs') +}) + +// The live 9P end-to-end case is an explicit opt-in: it needs a real installed +// distribution, so it registers only when DSH_OPS_TEST_WSL_DISTRO names one. It +// is reported as not run otherwise — an unavailable distribution never counts +// as a passing check, and the default suite keeps its honest coverage below. +// Registration order and the probe are host facts, not preferences: touching a +// nonexistent \\wsl.localhost distribution leaves the P9NP redirector unable to +// answer even valid paths for under two seconds (measured on this host), so +// this case registers before the negative one, probes at registration time so +// a check never silently passes as an empty body, and retries once within that +// window. A still-failing probe is reported as not run and not counted. +const liveDistro = process.env.DSH_OPS_TEST_WSL_DISTRO +let liveProvider = 'not-answering' +if (liveDistro) { + for (const attempt of [0, 1]) { + const probe = ps(`try { \$null = Get-Item -LiteralPath ${quote(`\\\\wsl.localhost\\${liveDistro}\\tmp`)} -Force -ErrorAction Stop; 'ok' } catch { 'no' }`) + if (probe.status === 0 && probe.stdout.trim() === 'ok') { liveProvider = 'answering'; break } + if (attempt === 0) await wait(2500) + } +} +if (liveDistro && liveProvider === 'answering') { + await test(`the live WSL UNC workspace ${liveDistro} answers ready instead of dying inside Get-Acl (#669)`, () => { + const root = `\\\\wsl.localhost\\${liveDistro}\\tmp` + const backups = path.join(TEST_HOME, 'prepare-wsl-redirector-live-backups') + const value = runScript(root, backups) + assert.equal(value.status, 0, value.stderr) + assert.equal(value.stdout.trim(), 'ready') + }) +} else if (liveDistro) { + console.log(' (live 9P end-to-end not run: the redirector is not answering right now; not counted as passed)') +} else { + console.log(' (live 9P end-to-end not run: set DSH_OPS_TEST_WSL_DISTRO= to include it; not counted as passed)') +} + +await test('a redirector-spelled workspace that does not exist fails instead of answering ready (#669)', () => { + // The prefix allowance must not widen into skipping existence: a 9P-spelled + // path that does not resolve still has to fail the same way any other + // missing workspace does, after the Get-Item/container validation. + const root = '\\\\wsl.localhost\\nonexistent-distro\\workspace' + const backups = path.join(TEST_HOME, 'prepare-wsl-redirector-backups') + const value = runScript(root, backups) + assert.notEqual(value.status, 0, 'a missing 9P workspace must not be answered ready') + assert.ok(!value.stdout.includes('ready')) + assert.equal(fs.existsSync(backups) ? fs.readdirSync(backups).length : 0, 0) +}) + +} + +report('workspace-wsl-redirector') diff --git a/test/wsl-prepare-merge.test.mjs b/test/wsl-prepare-merge.test.mjs new file mode 100644 index 00000000..da12641c --- /dev/null +++ b/test/wsl-prepare-merge.test.mjs @@ -0,0 +1,149 @@ +import fs from 'node:fs' +import path from 'node:path' +import { createHash } from 'node:crypto' +import { assert, test, report, withTempDir } from './lib/harness.mjs' +import { tryWslRead } from '../lib/execution/wsl-worker.js' +import { workerEnvelope } from '../lib/execution/protocol.js' + +/** + * Issue #576: the optional WSL read delegated after driving one wsl.exe call + * per drive mapping, one more for the payload sha256, and a last one for the + * worker, so a small read paid four WSL startups for one result. The + * preparation must be a single fixed /bin/sh -c script whose text never + * interpolates data: every drive root and the payload path travel as + * positional arguments, so no Windows path can alter the command shape. The + * digest check and the Windows fallback stay exactly as strict as before. + */ +const unc = '\\\\wsl.localhost\\Ubuntu' + +/** Stage a fake installed package with receipt-verified workers for both targets. */ +function stagePackage(dir) { + const version = '9.9.9-merge' + fs.writeFileSync(path.join(dir, 'package.json'), JSON.stringify({ version })) + const digests = {} + for (const [target, name] of [['win32-x64', 'dsh-ops-worker.exe'], ['linux-x64', 'dsh-ops-worker']]) { + const file = path.join(dir, 'runtime', target, name) + fs.mkdirSync(path.dirname(file), { recursive: true }) + fs.writeFileSync(file, `payload-${target}`) + digests[target] = createHash('sha256').update(`payload-${target}`).digest('hex') + fs.writeFileSync(path.join(dir, 'runtime', target, 'provenance.json'), + JSON.stringify({ target, version, executable: name, sha256: digests[target] })) + } + return digests +} + +/** Record every spawn and answer the merged preparation or the worker delegation. */ +function fakeSubprocess(answers) { + const calls = [] + return { + calls, + spawn: (spec) => { + calls.push(spec) + const prepare = spec.argv[4] === '/bin/sh' && spec.argv[5] === '-c' + const answer = prepare ? answers.prepare(calls.length) : answers.worker + return { + done: Promise.resolve({ exitCode: answer.exitCode ?? 0, signal: null }), + collected: { + stdout: { readFrom: () => ({ text: answer.stdout ?? '', lossy: false }) }, + stderr: { readFrom: () => ({ text: answer.stderr ?? '', lossy: false }) }, + }, + } + }, + } +} + +async function attemptWith(dir, subprocess, digests, windowsWorker, payloadLinux, extraArgs = {}) { + const envelope = workerEnvelope('file', { tool: 'glob', args: { path: `${unc}\\project\\*.txt` }, + root: `${unc}\\project`, deny_roots: ['C:\\Users\\pc\\.ssh', 'D:\\private\\creds'], ...extraArgs }) + return tryWslRead(subprocess, windowsWorker, envelope, 'C:\\', new AbortController().signal, () => true, undefined, + { platform: 'win32', packageRoot: dir, env: { DSH_OPS_WSL_READS: '1', SystemRoot: 'C:\\Windows' } }) +} + +await test('WSL read preparation is one fixed positional-parameter shell call plus the worker (#576)', async () => { + if (process.platform !== 'win32') { + console.log(' (POSIX host: merged WSL preparation needs real Windows drive roots; this check needs Windows)') + return + } + await withTempDir('wsl-prepare-merge', async (dir) => { + const digests = stagePackage(dir) + const windowsWorker = path.join(dir, 'runtime', 'win32-x64', 'dsh-ops-worker.exe') + const payloadLinux = path.posix.join('/mnt/c', path.win32.relative('C:\\', path.join(dir, 'runtime', 'linux-x64', 'dsh-ops-worker')).replaceAll('\\', '/')) + const mergedOutput = [ + 'C:\\\t/mnt/c', + 'D:\\\t/mnt/d', + `${digests['linux-x64']} ${payloadLinux}`, + ].join('\n') + '\n' + const subprocess = fakeSubprocess({ + prepare: () => ({ stdout: mergedOutput }), + worker: { stdout: JSON.stringify({ status: 'completed', pageComplete: true, items: ['/project/a.txt'] }) }, + }) + const attempt = await attemptWith(dir, subprocess, digests, windowsWorker, payloadLinux) + assert.equal(attempt.handled, true, JSON.stringify(attempt)) + assert.deepEqual(attempt.value.items, [`${unc}\\project\\a.txt`]) + // #576: exactly one preparation call and one worker delegation. + assert.equal(subprocess.calls.length, 2, + `expected 2 wsl.exe invocations, saw ${subprocess.calls.length}: ${subprocess.calls.map(c => c.argv.join(' ')).join(' || ')}`) + const prepare = subprocess.calls[0] + assert.equal(prepare.argv[4], '/bin/sh') + assert.equal(prepare.argv[5], '-c') + const script = prepare.argv[6] + // The script is fixed text: it names its tools and interpolates nothing. + assert.ok(script.includes('wslpath') && script.includes('sha256sum'), script) + assert.ok(!script.includes('C:'), script) + assert.ok(!script.includes(dir), script) + // Data travels as positional arguments behind the $0 placeholder, in the + // order drive roots first and the payload path last. + assert.equal(prepare.argv[7], 'dsh-ops') + assert.deepEqual(prepare.argv.slice(8), ['C:\\', 'D:\\', path.join(dir, 'runtime', 'linux-x64', 'dsh-ops-worker')]) + // The delegation runs the receipt-verified Linux worker with mapped roots. + assert.equal(subprocess.calls[1].argv[4], payloadLinux) + const delegated = JSON.parse(subprocess.calls[1].stdio.stdin.data) + assert.deepEqual(delegated.request.deny_roots, ['/mnt/c/Users/pc/.ssh', '/mnt/d/private/creds']) + assert.equal(delegated.request.root, '/project') + }) +}) + +await test('a mismatched payload digest still falls back to the Windows worker (#576)', async () => { + if (process.platform !== 'win32') { + console.log(' (POSIX host: the digest fallback check needs real Windows drive roots; this check needs Windows)') + return + } + await withTempDir('wsl-prepare-digest', async (dir) => { + const digests = stagePackage(dir) + const windowsWorker = path.join(dir, 'runtime', 'win32-x64', 'dsh-ops-worker.exe') + const payloadLinux = path.posix.join('/mnt/c', path.win32.relative('C:\\', path.join(dir, 'runtime', 'linux-x64', 'dsh-ops-worker')).replaceAll('\\', '/')) + const mergedOutput = [ + 'C:\\\t/mnt/c', + 'D:\\\t/mnt/d', + `${'0'.repeat(64)} ${payloadLinux}`, + ].join('\n') + '\n' + const subprocess = fakeSubprocess({ + prepare: () => ({ stdout: mergedOutput }), + worker: { stdout: '{}' }, + }) + const attempt = await attemptWith(dir, subprocess, digests, windowsWorker, payloadLinux) + assert.equal(attempt.handled, false) + assert.equal(subprocess.calls.length, 1, 'a digest mismatch must not start a worker delegation') + }) +}) + +await test('a failing preparation script still falls back to the Windows worker (#576)', async () => { + if (process.platform !== 'win32') { + console.log(' (POSIX host: the preparation fallback check needs real Windows drive roots; this check needs Windows)') + return + } + await withTempDir('wsl-prepare-fail', async (dir) => { + const digests = stagePackage(dir) + const windowsWorker = path.join(dir, 'runtime', 'win32-x64', 'dsh-ops-worker.exe') + const payloadLinux = path.posix.join('/mnt/c', path.win32.relative('C:\\', path.join(dir, 'runtime', 'linux-x64', 'dsh-ops-worker')).replaceAll('\\', '/')) + const subprocess = fakeSubprocess({ + prepare: () => ({ exitCode: 9, stdout: '' }), + worker: { stdout: '{}' }, + }) + const attempt = await attemptWith(dir, subprocess, digests, windowsWorker, payloadLinux) + assert.equal(attempt.handled, false) + assert.equal(subprocess.calls.length, 1) + }) +}) + +report('wsl-prepare-merge')