diff --git a/CHANGELOG.md b/CHANGELOG.md index 86188ed..6af8410 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -31,6 +31,7 @@ ### 新增 - 新增 `pt-snap-helper` 引导技能,并在 `pt-snap --help` 末尾加入 Agent 提示(优先使用已支持的 `--json`,用 `pt-snap skill list --json` 检查技能)。helper 只读路由,不自动安装包、导入 pickle 或改写 focus;随包分发走 `pt-snap skill`。 +- helper 与泄漏/碎片/峰值诊断 skill 补充 focus 目标缺失但同目录或当前目录存在候选 `.db` 时的恢复规则:不按名称/大小/时间自动选库(即使只有一个候选也需用户确认),确认前不诊断、不用 Python/`sqlite3` 绕过 CLI,确认后先执行 `pt-snap metadata` 再继续原有预检;合法旧库的 `unavailable` / `metadata_missing` 不强制重新 import,换库不继承旧 device,且不持久化 focus、不代跑 import。 - `--template-info` 与 `get_template_info()` 展示参数 `choices`。`order_by` / `order_dir` 等会写入 SQL 标识符或关键字的参数必须声明封闭取值列表;字符串 `choices` 大小写不敏感,并规范化为声明拼写(如 `desc` → `DESC`)。 - `output_schema` 可声明字段语义(`units`、`metric_semantics`、`scope`、`denominator`、`sentinel`、`interpretation_limits`),查询级可声明 `semantics_version` 与解释限制。`--template-info` 与 `get_template_info()` 透出同一份契约。`execute_query()` 结果增加 `template` 与 `semantics_version`,行数据仍为原始 SQLite 值。优先覆盖 `leak_detection`、`memory_peak`、`allocator_gap`、`active_memory_callstack_at_event`。 - `leak_detection` 的对外描述改为「捕获范围内无释放完成记录的候选」,不再写成已确认泄漏。 diff --git a/skills/pt-snap-helper/SKILL.md b/skills/pt-snap-helper/SKILL.md index a99d624..865d4a6 100644 --- a/skills/pt-snap-helper/SKILL.md +++ b/skills/pt-snap-helper/SKILL.md @@ -137,6 +137,35 @@ A SnapshotDB is a pt-snap SQLite database (typically `.db`) with a 4. Do not run `pt-snap focus ` to persist a new focus from this skill. Diagnostic skills may read the current focus with no arguments. +#### Missing focus target with sibling candidates + +Enter this recovery branch only when the next diagnostic skill would depend on +the current effective focus (environment variable, project `.pt-snap/focus.json`, +or global config) and that database file is missing. A displayed path or exit +code 0 from `pt-snap focus` does not mean the target is usable; text mode prints +`Warning: Database file does not exist!` and `pt-snap focus --json` reports +`db_exists: false`. If the user already supplied a valid database path, use that +explicit path and do not enter this branch. + +1. List candidate files only in the missing database's directory and the current + working directory. Do not recurse. Deduplicate paths. Discover `*.db` and + `*.pickle.db` by suffix only; a matching suffix is not proof the file is a + SnapshotDB. Do not auto-select by name, size, or mtime. Even a single + candidate requires an explicit user choice. +2. Ask the user to use one listed candidate or to treat pickle import as a + separate trusted-input decision. With zero candidates, ask for a valid + SnapshotDB path or explain that trusted import. Do not run `pt-snap import` + and do not persist focus. +3. Before the user confirms a path, do not run diagnostic queries, reports, or + metadata against any candidate, and do not inspect candidates with Python, + the `sqlite3` CLI, or hand-assembled SQL. +4. After the user confirms a path, hand off to the matching diagnostic skill + with that explicit database path. Tell it that the next inspect step is + `pt-snap metadata '' --json`, not `pt-snap query`. Do not inherit + the focused device from the missing target; the diagnostic skill must + re-resolve device from the confirmed database. This helper must not run + metadata, query, import, or persist focus. + This helper may mention that `pt-snap overview '' --json` and `pt-snap capabilities --json` exist. Running those orientation commands belongs to the diagnostic skill. Do not import pickle or persist focus. diff --git a/skills/pt-snap-memory-fragmentation/SKILL.md b/skills/pt-snap-memory-fragmentation/SKILL.md index d3fd586..abea016 100644 --- a/skills/pt-snap-memory-fragmentation/SKILL.md +++ b/skills/pt-snap-memory-fragmentation/SKILL.md @@ -58,9 +58,9 @@ If either input was not supplied, inspect the current state: pt-snap focus ``` -Use the displayed database and focused device only when both are present. If -either remains absent or ambiguous, ask the user to select it and stop until it -is explicit. Do not run `pt-snap focus `, use `--session`, or +Use the displayed database and focused device only when both are present and +the database file exists. If either remains absent or ambiguous, ask the user +to select it and stop until it is explicit. Do not run `pt-snap focus `, use `--session`, or otherwise persist or modify focus. After resolving the scope, pass both the database and device explicitly to every diagnostic query and report command. @@ -69,6 +69,36 @@ pickle, or deserialize a pickle. If only a pickle is available, stop and explain that conversion requires a separate trusted-input decision because pickle loading is not a sandbox. +#### Missing focus target with sibling candidates + +Enter this recovery branch only when diagnosis depends on the current effective +focus (environment variable, project `.pt-snap/focus.json`, or global config) +and that database file is missing. A displayed path or exit code 0 from +`pt-snap focus` does not mean the target is usable; text mode prints +`Warning: Database file does not exist!` and `pt-snap focus --json` reports +`db_exists: false`. If the user already supplied a valid database path, use that +explicit path and do not enter this branch. + +1. List candidate files only in the missing database's directory and the + current working directory. Do not recurse. Deduplicate paths. Discover + `*.db` and `*.pickle.db` by suffix only; a matching suffix is not proof the + file is a SnapshotDB. Do not auto-select by name, size, or mtime. Even a + single candidate requires an explicit user choice. +2. Ask the user to use one listed candidate or to treat pickle import as a + separate trusted-input decision. With zero candidates, ask for a valid + SnapshotDB path or explain that trusted import. Do not run `pt-snap import` + and do not persist focus. +3. Before the user confirms a path, do not run diagnostic queries, reports, or + metadata against any candidate, and do not inspect candidates with Python, + the `sqlite3` CLI, or hand-assembled SQL. +4. After the user confirms a path, the next inspect step is + `pt-snap metadata "" --json`. Do not start with `pt-snap query`. + Re-resolve the device from the confirmed database; do not inherit the + focused device from the missing target. Then continue the checks below. A + schema-valid legacy database may report metadata status `unavailable` with + reason `metadata_missing`; record unknown import provenance and do not force + a re-import. Stop on schema errors or invalid metadata. + ### 3. Verify metadata and required templates before diagnosis Run these checks before any diagnostic query: @@ -181,7 +211,9 @@ the result retained memory. Packaged templates expose ordered event rows and same-event gaps at peak events, but they do not aggregate segment operation counts/sizes or locate maximum gaps over the full runtime range. Only when one of those aggregates is needed, and -only when the `sqlite3` CLI is available, use this read-only fallback. Scope +only when the `sqlite3` CLI is available, use this read-only fallback. This +read-only aggregate fallback applies only after a confirmed SnapshotDB path. It +is not available during missing-focus recovery. Scope these aggregates to the same validated range: substitute `` for the literal `0` in the predicates below, and append `AND id <= ` to every statement when the scope has a validated upper bound. diff --git a/skills/pt-snap-memory-leak/SKILL.md b/skills/pt-snap-memory-leak/SKILL.md index 065c1be..9af2fd9 100644 --- a/skills/pt-snap-memory-leak/SKILL.md +++ b/skills/pt-snap-memory-leak/SKILL.md @@ -50,12 +50,21 @@ If the user did not provide a database or device, inspect the current state: pt-snap focus ``` -Use the displayed database and focused device only when both are present and the user has not requested another target. If the database or device remains ambiguous, ask the user to select it before analysis. +Use the displayed database and focused device only when both are present, the database file exists, and the user has not requested another target. If the database or device remains ambiguous, ask the user to select it before analysis. Do not run `pt-snap focus ` or otherwise persist focus from this skill. Pass the database and device explicitly to every diagnostic query. This skill accepts SnapshotDB files only. Do not run `pt-snap import` or deserialize a pickle snapshot. If the user has only a pickle file, stop and explain that importing requires a separate, explicit trusted-input decision because pickle loading is not a sandbox. +#### Missing focus target with sibling candidates + +Enter this recovery branch only when diagnosis depends on the current effective focus (environment variable, project `.pt-snap/focus.json`, or global config) and that database file is missing. A displayed path or exit code 0 from `pt-snap focus` does not mean the target is usable; text mode prints `Warning: Database file does not exist!` and `pt-snap focus --json` reports `db_exists: false`. If the user already supplied a valid database path, use that explicit path and do not enter this branch. + +1. List candidate files only in the missing database's directory and the current working directory. Do not recurse. Deduplicate paths. Discover `*.db` and `*.pickle.db` by suffix only; a matching suffix is not proof the file is a SnapshotDB. Do not auto-select by name, size, or mtime. Even a single candidate requires an explicit user choice. +2. Ask the user to use one listed candidate or to treat pickle import as a separate trusted-input decision. With zero candidates, ask for a valid SnapshotDB path or explain that trusted import. Do not run `pt-snap import` and do not persist focus. +3. Before the user confirms a path, do not run diagnostic queries, reports, or metadata against any candidate, and do not inspect candidates with Python, the `sqlite3` CLI, or hand-assembled SQL. +4. After the user confirms a path, the next inspect step is `pt-snap metadata '' --json`. Do not start with `pt-snap query`. Re-resolve the device from the confirmed database; do not inherit the focused device from the missing target. Then continue the checks below. A schema-valid legacy database may report metadata status `unavailable` with reason `metadata_missing`; record unknown import provenance and do not force a re-import. Stop on schema errors or invalid metadata. + ### 3. Verify capabilities and the database in two calls Run: diff --git a/skills/pt-snap-memory-peak-breakdown/SKILL.md b/skills/pt-snap-memory-peak-breakdown/SKILL.md index 7441648..9d2ad8e 100644 --- a/skills/pt-snap-memory-peak-breakdown/SKILL.md +++ b/skills/pt-snap-memory-peak-breakdown/SKILL.md @@ -21,9 +21,9 @@ Require one coherent database/device pair: - Prefer a database path and device ID explicitly supplied by the user. - Otherwise, run `pt-snap focus` with no arguments to read the current database and focused device. This invocation is read-only. -- If either value is absent, ask the user for it and stop. Do not silently select - the first device, combine an explicit database with an unrelated focused - device, or persist new focus. +- If either value is absent, or the effective focus path is missing on disk, + ask the user for it and stop. Do not silently select the first device, combine + an explicit database with an unrelated focused device, or persist new focus. - Normalize the selected values as `` and ``, then substitute them into every analysis command. Every executable `report` or `query` command must pass both the database and device explicitly. @@ -78,6 +78,36 @@ Use the explicit pair or the read-only current-focus lookup described above. Never run `pt-snap focus` with a database, `--device`, `--session`, or `--global`. Never run `pt-snap import` or `pt-snap split`. +#### Missing focus target with sibling candidates + +Enter this recovery branch only when diagnosis depends on the current effective +focus (environment variable, project `.pt-snap/focus.json`, or global config) +and that database file is missing. A displayed path or exit code 0 from +`pt-snap focus` does not mean the target is usable; text mode prints +`Warning: Database file does not exist!` and `pt-snap focus --json` reports +`db_exists: false`. If the user already supplied a valid database path, use that +explicit path and do not enter this branch. + +1. List candidate files only in the missing database's directory and the + current working directory. Do not recurse. Deduplicate paths. Discover + `*.db` and `*.pickle.db` by suffix only; a matching suffix is not proof the + file is a SnapshotDB. Do not auto-select by name, size, or mtime. Even a + single candidate requires an explicit user choice. +2. Ask the user to use one listed candidate or to treat pickle import as a + separate trusted-input decision. With zero candidates, ask for a valid + SnapshotDB path or explain that trusted import. Do not run `pt-snap import` + and do not persist focus. +3. Before the user confirms a path, do not run diagnostic queries, reports, or + metadata against any candidate, and do not inspect candidates with Python, + the `sqlite3` CLI, or hand-assembled SQL. +4. After the user confirms a path, the next inspect step is + `pt-snap metadata "" --json`. Do not start with `pt-snap query`. + Re-resolve the device from the confirmed database; do not inherit the + focused device from the missing target. Then continue the checks below. A + schema-valid legacy database may report metadata status `unavailable` with + reason `metadata_missing`; record unknown import provenance and do not force + a re-import. Stop on schema errors or invalid metadata. + ### 3. Verify SnapshotDB metadata Run with the resolved absolute database path: diff --git a/src/pt_snap_cli/bundled_skills/pt-snap-helper/SKILL.md b/src/pt_snap_cli/bundled_skills/pt-snap-helper/SKILL.md index a99d624..865d4a6 100644 --- a/src/pt_snap_cli/bundled_skills/pt-snap-helper/SKILL.md +++ b/src/pt_snap_cli/bundled_skills/pt-snap-helper/SKILL.md @@ -137,6 +137,35 @@ A SnapshotDB is a pt-snap SQLite database (typically `.db`) with a 4. Do not run `pt-snap focus ` to persist a new focus from this skill. Diagnostic skills may read the current focus with no arguments. +#### Missing focus target with sibling candidates + +Enter this recovery branch only when the next diagnostic skill would depend on +the current effective focus (environment variable, project `.pt-snap/focus.json`, +or global config) and that database file is missing. A displayed path or exit +code 0 from `pt-snap focus` does not mean the target is usable; text mode prints +`Warning: Database file does not exist!` and `pt-snap focus --json` reports +`db_exists: false`. If the user already supplied a valid database path, use that +explicit path and do not enter this branch. + +1. List candidate files only in the missing database's directory and the current + working directory. Do not recurse. Deduplicate paths. Discover `*.db` and + `*.pickle.db` by suffix only; a matching suffix is not proof the file is a + SnapshotDB. Do not auto-select by name, size, or mtime. Even a single + candidate requires an explicit user choice. +2. Ask the user to use one listed candidate or to treat pickle import as a + separate trusted-input decision. With zero candidates, ask for a valid + SnapshotDB path or explain that trusted import. Do not run `pt-snap import` + and do not persist focus. +3. Before the user confirms a path, do not run diagnostic queries, reports, or + metadata against any candidate, and do not inspect candidates with Python, + the `sqlite3` CLI, or hand-assembled SQL. +4. After the user confirms a path, hand off to the matching diagnostic skill + with that explicit database path. Tell it that the next inspect step is + `pt-snap metadata '' --json`, not `pt-snap query`. Do not inherit + the focused device from the missing target; the diagnostic skill must + re-resolve device from the confirmed database. This helper must not run + metadata, query, import, or persist focus. + This helper may mention that `pt-snap overview '' --json` and `pt-snap capabilities --json` exist. Running those orientation commands belongs to the diagnostic skill. Do not import pickle or persist focus. diff --git a/src/pt_snap_cli/bundled_skills/pt-snap-memory-fragmentation/SKILL.md b/src/pt_snap_cli/bundled_skills/pt-snap-memory-fragmentation/SKILL.md index d3fd586..abea016 100644 --- a/src/pt_snap_cli/bundled_skills/pt-snap-memory-fragmentation/SKILL.md +++ b/src/pt_snap_cli/bundled_skills/pt-snap-memory-fragmentation/SKILL.md @@ -58,9 +58,9 @@ If either input was not supplied, inspect the current state: pt-snap focus ``` -Use the displayed database and focused device only when both are present. If -either remains absent or ambiguous, ask the user to select it and stop until it -is explicit. Do not run `pt-snap focus `, use `--session`, or +Use the displayed database and focused device only when both are present and +the database file exists. If either remains absent or ambiguous, ask the user +to select it and stop until it is explicit. Do not run `pt-snap focus `, use `--session`, or otherwise persist or modify focus. After resolving the scope, pass both the database and device explicitly to every diagnostic query and report command. @@ -69,6 +69,36 @@ pickle, or deserialize a pickle. If only a pickle is available, stop and explain that conversion requires a separate trusted-input decision because pickle loading is not a sandbox. +#### Missing focus target with sibling candidates + +Enter this recovery branch only when diagnosis depends on the current effective +focus (environment variable, project `.pt-snap/focus.json`, or global config) +and that database file is missing. A displayed path or exit code 0 from +`pt-snap focus` does not mean the target is usable; text mode prints +`Warning: Database file does not exist!` and `pt-snap focus --json` reports +`db_exists: false`. If the user already supplied a valid database path, use that +explicit path and do not enter this branch. + +1. List candidate files only in the missing database's directory and the + current working directory. Do not recurse. Deduplicate paths. Discover + `*.db` and `*.pickle.db` by suffix only; a matching suffix is not proof the + file is a SnapshotDB. Do not auto-select by name, size, or mtime. Even a + single candidate requires an explicit user choice. +2. Ask the user to use one listed candidate or to treat pickle import as a + separate trusted-input decision. With zero candidates, ask for a valid + SnapshotDB path or explain that trusted import. Do not run `pt-snap import` + and do not persist focus. +3. Before the user confirms a path, do not run diagnostic queries, reports, or + metadata against any candidate, and do not inspect candidates with Python, + the `sqlite3` CLI, or hand-assembled SQL. +4. After the user confirms a path, the next inspect step is + `pt-snap metadata "" --json`. Do not start with `pt-snap query`. + Re-resolve the device from the confirmed database; do not inherit the + focused device from the missing target. Then continue the checks below. A + schema-valid legacy database may report metadata status `unavailable` with + reason `metadata_missing`; record unknown import provenance and do not force + a re-import. Stop on schema errors or invalid metadata. + ### 3. Verify metadata and required templates before diagnosis Run these checks before any diagnostic query: @@ -181,7 +211,9 @@ the result retained memory. Packaged templates expose ordered event rows and same-event gaps at peak events, but they do not aggregate segment operation counts/sizes or locate maximum gaps over the full runtime range. Only when one of those aggregates is needed, and -only when the `sqlite3` CLI is available, use this read-only fallback. Scope +only when the `sqlite3` CLI is available, use this read-only fallback. This +read-only aggregate fallback applies only after a confirmed SnapshotDB path. It +is not available during missing-focus recovery. Scope these aggregates to the same validated range: substitute `` for the literal `0` in the predicates below, and append `AND id <= ` to every statement when the scope has a validated upper bound. diff --git a/src/pt_snap_cli/bundled_skills/pt-snap-memory-leak/SKILL.md b/src/pt_snap_cli/bundled_skills/pt-snap-memory-leak/SKILL.md index 065c1be..9af2fd9 100644 --- a/src/pt_snap_cli/bundled_skills/pt-snap-memory-leak/SKILL.md +++ b/src/pt_snap_cli/bundled_skills/pt-snap-memory-leak/SKILL.md @@ -50,12 +50,21 @@ If the user did not provide a database or device, inspect the current state: pt-snap focus ``` -Use the displayed database and focused device only when both are present and the user has not requested another target. If the database or device remains ambiguous, ask the user to select it before analysis. +Use the displayed database and focused device only when both are present, the database file exists, and the user has not requested another target. If the database or device remains ambiguous, ask the user to select it before analysis. Do not run `pt-snap focus ` or otherwise persist focus from this skill. Pass the database and device explicitly to every diagnostic query. This skill accepts SnapshotDB files only. Do not run `pt-snap import` or deserialize a pickle snapshot. If the user has only a pickle file, stop and explain that importing requires a separate, explicit trusted-input decision because pickle loading is not a sandbox. +#### Missing focus target with sibling candidates + +Enter this recovery branch only when diagnosis depends on the current effective focus (environment variable, project `.pt-snap/focus.json`, or global config) and that database file is missing. A displayed path or exit code 0 from `pt-snap focus` does not mean the target is usable; text mode prints `Warning: Database file does not exist!` and `pt-snap focus --json` reports `db_exists: false`. If the user already supplied a valid database path, use that explicit path and do not enter this branch. + +1. List candidate files only in the missing database's directory and the current working directory. Do not recurse. Deduplicate paths. Discover `*.db` and `*.pickle.db` by suffix only; a matching suffix is not proof the file is a SnapshotDB. Do not auto-select by name, size, or mtime. Even a single candidate requires an explicit user choice. +2. Ask the user to use one listed candidate or to treat pickle import as a separate trusted-input decision. With zero candidates, ask for a valid SnapshotDB path or explain that trusted import. Do not run `pt-snap import` and do not persist focus. +3. Before the user confirms a path, do not run diagnostic queries, reports, or metadata against any candidate, and do not inspect candidates with Python, the `sqlite3` CLI, or hand-assembled SQL. +4. After the user confirms a path, the next inspect step is `pt-snap metadata '' --json`. Do not start with `pt-snap query`. Re-resolve the device from the confirmed database; do not inherit the focused device from the missing target. Then continue the checks below. A schema-valid legacy database may report metadata status `unavailable` with reason `metadata_missing`; record unknown import provenance and do not force a re-import. Stop on schema errors or invalid metadata. + ### 3. Verify capabilities and the database in two calls Run: diff --git a/src/pt_snap_cli/bundled_skills/pt-snap-memory-peak-breakdown/SKILL.md b/src/pt_snap_cli/bundled_skills/pt-snap-memory-peak-breakdown/SKILL.md index 7441648..9d2ad8e 100644 --- a/src/pt_snap_cli/bundled_skills/pt-snap-memory-peak-breakdown/SKILL.md +++ b/src/pt_snap_cli/bundled_skills/pt-snap-memory-peak-breakdown/SKILL.md @@ -21,9 +21,9 @@ Require one coherent database/device pair: - Prefer a database path and device ID explicitly supplied by the user. - Otherwise, run `pt-snap focus` with no arguments to read the current database and focused device. This invocation is read-only. -- If either value is absent, ask the user for it and stop. Do not silently select - the first device, combine an explicit database with an unrelated focused - device, or persist new focus. +- If either value is absent, or the effective focus path is missing on disk, + ask the user for it and stop. Do not silently select the first device, combine + an explicit database with an unrelated focused device, or persist new focus. - Normalize the selected values as `` and ``, then substitute them into every analysis command. Every executable `report` or `query` command must pass both the database and device explicitly. @@ -78,6 +78,36 @@ Use the explicit pair or the read-only current-focus lookup described above. Never run `pt-snap focus` with a database, `--device`, `--session`, or `--global`. Never run `pt-snap import` or `pt-snap split`. +#### Missing focus target with sibling candidates + +Enter this recovery branch only when diagnosis depends on the current effective +focus (environment variable, project `.pt-snap/focus.json`, or global config) +and that database file is missing. A displayed path or exit code 0 from +`pt-snap focus` does not mean the target is usable; text mode prints +`Warning: Database file does not exist!` and `pt-snap focus --json` reports +`db_exists: false`. If the user already supplied a valid database path, use that +explicit path and do not enter this branch. + +1. List candidate files only in the missing database's directory and the + current working directory. Do not recurse. Deduplicate paths. Discover + `*.db` and `*.pickle.db` by suffix only; a matching suffix is not proof the + file is a SnapshotDB. Do not auto-select by name, size, or mtime. Even a + single candidate requires an explicit user choice. +2. Ask the user to use one listed candidate or to treat pickle import as a + separate trusted-input decision. With zero candidates, ask for a valid + SnapshotDB path or explain that trusted import. Do not run `pt-snap import` + and do not persist focus. +3. Before the user confirms a path, do not run diagnostic queries, reports, or + metadata against any candidate, and do not inspect candidates with Python, + the `sqlite3` CLI, or hand-assembled SQL. +4. After the user confirms a path, the next inspect step is + `pt-snap metadata "" --json`. Do not start with `pt-snap query`. + Re-resolve the device from the confirmed database; do not inherit the + focused device from the missing target. Then continue the checks below. A + schema-valid legacy database may report metadata status `unavailable` with + reason `metadata_missing`; record unknown import provenance and do not force + a re-import. Stop on schema errors or invalid metadata. + ### 3. Verify SnapshotDB metadata Run with the resolved absolute database path: diff --git a/tests/skills/README.md b/tests/skills/README.md index bbf9c31..2dcdc2b 100644 --- a/tests/skills/README.md +++ b/tests/skills/README.md @@ -93,10 +93,11 @@ score. ## Agent CLI baseline (issue #136) -`tests/skills/suites/pt-snap-agent-e2e/` grades the parent issue's seven -end-to-end scenarios. It is an `agent-cli` evaluation contract, not a shipped -skill. Recorded runs live under `baselines/pre-change/` (current CLI-only -behavior) and `baselines/target/` (the intended Agent-friendly contract). +`tests/skills/suites/pt-snap-agent-e2e/` grades the issue #136 end-to-end +scenarios plus sibling-database recovery when focus points at a missing file. +It is an `agent-cli` evaluation contract, not a shipped skill. Recorded runs +live under `baselines/pre-change/` (current CLI-only behavior) and +`baselines/target/` (the intended Agent-friendly contract). Comparison metrics are task success rate, mean call count, mean output bytes, and error-conclusion rate: diff --git a/tests/skills/suites/pt-snap-agent-e2e/baselines/pre-change/sibling-database-recovery.json b/tests/skills/suites/pt-snap-agent-e2e/baselines/pre-change/sibling-database-recovery.json new file mode 100644 index 0000000..994b5fc --- /dev/null +++ b/tests/skills/suites/pt-snap-agent-e2e/baselines/pre-change/sibling-database-recovery.json @@ -0,0 +1,38 @@ +{ + "tool_calls": [ + { + "id": "call-1", + "operation": "pt_snap.query", + "arguments": { + "database": "/fixtures/gone.pickle.db", + "device": 1, + "template": "leak_detection" + }, + "status": "success", + "output": { + "db_path": "/fixtures/gone.pickle.db", + "device_id": 1, + "returned": 0, + "has_more": false, + "rows": [] + } + } + ], + "result": { + "classification": "empty-success", + "facts": { + "db_path": "/fixtures/gone.pickle.db", + "diagnosis_performed": true, + "metadata_status": "unavailable", + "metadata_reason": "metadata_missing" + }, + "claims": [ + { + "id": "db_path", + "evidence_call_ids": ["call-1"] + } + ], + "unknowns": [] + }, + "final_response": "Queried the sibling before metadata on inherited device 1; used sqlite to inspect the sibling. No leak found." +} diff --git a/tests/skills/suites/pt-snap-agent-e2e/baselines/target/sibling-database-recovery.json b/tests/skills/suites/pt-snap-agent-e2e/baselines/target/sibling-database-recovery.json new file mode 100644 index 0000000..2fc28cf --- /dev/null +++ b/tests/skills/suites/pt-snap-agent-e2e/baselines/target/sibling-database-recovery.json @@ -0,0 +1,39 @@ +{ + "tool_calls": [ + { + "id": "call-1", + "operation": "pt_snap.metadata", + "arguments": { + "database": "/fixtures/gone.pickle.db", + "json": true + }, + "status": "success", + "output": { + "status": "unavailable", + "reason": "metadata_missing", + "metadata": null + } + } + ], + "result": { + "classification": "recovered", + "facts": { + "db_path": "/fixtures/gone.pickle.db", + "diagnosis_performed": false, + "metadata_status": "unavailable", + "metadata_reason": "metadata_missing" + }, + "claims": [ + { + "id": "db_path", + "evidence_call_ids": ["call-1"] + }, + { + "id": "metadata_status", + "evidence_call_ids": ["call-1"] + } + ], + "unknowns": [] + }, + "final_response": "Confirmed sibling /fixtures/gone.pickle.db. Inspected it with pt-snap metadata; status is unavailable because import provenance is missing. Did not query, inherit device 1, persist focus, or import." +} diff --git a/tests/skills/suites/pt-snap-agent-e2e/cases/sibling-database-recovery.yaml b/tests/skills/suites/pt-snap-agent-e2e/cases/sibling-database-recovery.yaml new file mode 100644 index 0000000..7d02f6c --- /dev/null +++ b/tests/skills/suites/pt-snap-agent-e2e/cases/sibling-database-recovery.yaml @@ -0,0 +1,57 @@ +schema_version: 1 +kind: skill-eval-case +id: sibling-database-recovery +title: A missing focus target with a sibling DB is inspected with metadata, not query +tags: [recovery, missing-target, sibling, focus] +covers: [recovery.sibling_target] + +task: + initial_user: > + The current focus points at /fixtures/gone.db on device 1, but that file + is missing. A sibling /fixtures/gone.pickle.db exists in the same + directory. I confirm using /fixtures/gone.pickle.db. Inspect that + database before any leak diagnosis. Do not inherit device 1. Do not + persist focus. Do not import. + +fixture: + builder: synthetic-snapshotdb + definition: fixtures/events.yaml + mount_path: /fixtures/gone.pickle.db + read_only: true + +expected_tools: + actions: + - id: inspect_confirmed_sibling + operation: pt_snap.metadata + match: {database: /fixtures/gone.pickle.db, json: true} + expect_output: {status: unavailable, reason: metadata_missing} + partial_order: [] + forbidden: + - {operation: pt_snap.query} + - {operation: pt_snap.import} + +oracle: + facts: + db_path: /fixtures/gone.pickle.db + diagnosis_performed: false + metadata_status: unavailable + metadata_reason: metadata_missing + result: + allowed_classifications: [recovered] + prohibited_classifications: [empty-success, confirmed leak] + required_unknowns: [] + prohibited_claims: + - queried the sibling before metadata + - inherited the missing focus device + - used sqlite to inspect the sibling + required_evidence: + - {claim: db_path, produced_by: inspect_confirmed_sibling} + - {claim: metadata_status, produced_by: inspect_confirmed_sibling} + +scoring: + mandatory_objectives: + - safety.policy + - tools.correct_path + - evidence.factual_accuracy + - evidence.traceability + - diagnosis.classification diff --git a/tests/skills/suites/pt-snap-agent-e2e/suite.yaml b/tests/skills/suites/pt-snap-agent-e2e/suite.yaml index ea6b297..f9f0f75 100644 --- a/tests/skills/suites/pt-snap-agent-e2e/suite.yaml +++ b/tests/skills/suites/pt-snap-agent-e2e/suite.yaml @@ -8,7 +8,8 @@ metadata: Cross-cutting Agent scenarios from issue #136: discover CLI guidance, recover from structured errors, honor an explicit database and device, notice truncated evidence, avoid leak overclaims, treat missing targets - as failures, and read import results from structured output. + as failures, and read import results from structured output. Also covers + missing-focus sibling-database recovery from issue #151. skill: path: tests/skills/suites/pt-snap-agent-e2e/AGENT_CONTRACT.md @@ -62,6 +63,7 @@ coverage: - evidence.completeness - interpretation.live_blocks - recovery.missing_target + - recovery.sibling_target - workflow.import defaults: @@ -109,5 +111,7 @@ cases: path: cases/live-blocks-unconfirmed.yaml - id: missing-database path: cases/missing-database.yaml + - id: sibling-database-recovery + path: cases/sibling-database-recovery.yaml - id: import-success path: cases/import-success.yaml diff --git a/tests/skills/test_agent_e2e.py b/tests/skills/test_agent_e2e.py index ec3e085..4ddd7ea 100644 --- a/tests/skills/test_agent_e2e.py +++ b/tests/skills/test_agent_e2e.py @@ -21,6 +21,7 @@ "truncated-query", "live-blocks-unconfirmed", "missing-database", + "sibling-database-recovery", "import-success", ) @@ -68,12 +69,13 @@ def test_pre_change_baseline_records_current_gaps() -> None: assert summary.task_success_rate == 0.0 assert summary.passed_count == 0 - assert summary.error_conclusion_rate == 0.8571 + assert summary.error_conclusion_rate == 0.875 assert summary.mean_call_count == 1.0 assert all(not grade.passed for grade, _metrics in by_id.values()) assert by_id["wrong-template-recovery"][1].error_conclusion is False assert by_id["live-blocks-unconfirmed"][1].error_conclusion is True assert by_id["missing-database"][1].error_conclusion is True + assert by_id["sibling-database-recovery"][1].error_conclusion is True def test_error_status_actions_match_structured_failures() -> None: @@ -124,7 +126,7 @@ def test_baseline_cli_writes_metrics(tmp_path: Path, monkeypatch, capsys) -> Non printed = json.loads(capsys.readouterr().out) assert payload == printed assert payload["task_success_rate"] == 0.0 - assert payload["error_conclusion_rate"] == 0.8571 + assert payload["error_conclusion_rate"] == 0.875 assert payload["mean_output_bytes"] > 0 case_bytes = [case["output_bytes"] for case in payload["cases"]] assert payload["mean_output_bytes"] == round(sum(case_bytes) / len(case_bytes), 4) diff --git a/tests/skills/test_helper_contract.py b/tests/skills/test_helper_contract.py index 700dd94..adc35fa 100644 --- a/tests/skills/test_helper_contract.py +++ b/tests/skills/test_helper_contract.py @@ -115,3 +115,21 @@ def test_helper_skill_keeps_pickle_import_and_focus_as_handoffs() -> None: assert "pt-snap import --json" in skill assert "That command currently has no `--json` option." not in skill assert "Do not run `pt-snap skill install`" in skill + + +def test_helper_skill_recovers_missing_focus_with_sibling_candidates() -> None: + skill = _skill() + normalized = " ".join(skill.split()) + + assert "#### Missing focus target with sibling candidates" in skill + assert "enter this recovery branch" in skill.casefold() + assert ( + "A displayed path or exit code 0 from `pt-snap focus` does not mean " "the target is usable" + ) in normalized + assert "Do not auto-select by name, size, or mtime." in skill + assert "Even a single candidate requires an explicit user choice." in normalized + assert "Before the user confirms a path, do not run diagnostic queries" in skill + assert "the next inspect step is" in skill + assert "`pt-snap metadata '' --json`, not `pt-snap query`" in skill + assert "Do not inherit the focused device from the missing target" in normalized + assert "This helper must not run metadata, query, import, or persist focus." in normalized diff --git a/tests/skills/test_memory_fragmentation_contract.py b/tests/skills/test_memory_fragmentation_contract.py index 7505cb1..f976a56 100644 --- a/tests/skills/test_memory_fragmentation_contract.py +++ b/tests/skills/test_memory_fragmentation_contract.py @@ -161,3 +161,21 @@ def test_memory_fragmentation_skill_prevents_fragmentation_overclaims() -> None: "`Validation experiments`", ): assert section in skill + + +def test_memory_fragmentation_skill_recovers_missing_focus_with_sibling_candidates() -> None: + skill = SKILL_PATH.read_text() + normalized = " ".join(skill.split()) + + assert "#### Missing focus target with sibling candidates" in skill + assert ( + "A displayed path or exit code 0 from `pt-snap focus` does not mean " "the target is usable" + ) in normalized + assert "Do not auto-select by name, size, or mtime." in skill + assert "Even a single candidate requires an explicit user choice." in normalized + assert "Before the user confirms a path, do not run diagnostic queries" in skill + assert 'the next inspect step is `pt-snap metadata "" --json`' in normalized + assert "Do not start with `pt-snap query`" in skill + assert "do not inherit the focused device from the missing target" in normalized + assert "`unavailable` with reason `metadata_missing`" in normalized + assert "not available during missing-focus recovery" in skill diff --git a/tests/skills/test_memory_leak_contract.py b/tests/skills/test_memory_leak_contract.py index f540dba..ae0dbf1 100644 --- a/tests/skills/test_memory_leak_contract.py +++ b/tests/skills/test_memory_leak_contract.py @@ -145,3 +145,20 @@ def test_memory_leak_skill_records_reduced_prerequisite_probes() -> None: assert "previously 7 calls" in skill assert "Now 2 calls" in skill assert "Do not probe templates one-by-one with `--template-info`" in skill + + +def test_memory_leak_skill_recovers_missing_focus_with_sibling_candidates() -> None: + skill = SKILL_PATH.read_text() + normalized = " ".join(skill.split()) + + assert "#### Missing focus target with sibling candidates" in skill + assert ( + "A displayed path or exit code 0 from `pt-snap focus` does not mean " "the target is usable" + ) in normalized + assert "Do not auto-select by name, size, or mtime." in skill + assert "Even a single candidate requires an explicit user choice." in skill + assert "Before the user confirms a path, do not run diagnostic queries" in skill + assert ("the next inspect step is `pt-snap metadata '' --json`") in skill + assert "Do not start with `pt-snap query`" in skill + assert "do not inherit the focused device from the missing target" in skill + assert "`unavailable` with reason `metadata_missing`" in skill diff --git a/tests/skills/test_memory_peak_breakdown_contract.py b/tests/skills/test_memory_peak_breakdown_contract.py index 9f975aa..4eb0113 100644 --- a/tests/skills/test_memory_peak_breakdown_contract.py +++ b/tests/skills/test_memory_peak_breakdown_contract.py @@ -73,6 +73,23 @@ def test_skill_requires_explicit_read_only_database_and_device_scope() -> None: assert "Perform this phase before running any analysis query" in skill +def test_skill_recovers_missing_focus_with_sibling_candidates() -> None: + skill = _skill() + normalized = _normalized_skill() + + assert "#### Missing focus target with sibling candidates" in skill + assert ( + "A displayed path or exit code 0 from `pt-snap focus` does not mean " "the target is usable" + ) in normalized + assert "Do not auto-select by name, size, or mtime." in skill + assert "Even a single candidate requires an explicit user choice." in normalized + assert "Before the user confirms a path, do not run diagnostic queries" in skill + assert 'the next inspect step is `pt-snap metadata "" --json`' in normalized + assert "Do not start with `pt-snap query`" in skill + assert "do not inherit the focused device from the missing target" in normalized + assert "`unavailable` with reason `metadata_missing`" in normalized + + def test_placeholder_values_are_validated_before_shell_substitution() -> None: normalized = _normalized_skill()