Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,14 @@ templates under category subdirectories are included by
`query/registry.py` recursively loads templates into the singleton registry;
`query/executor.py` renders with Jinja2 `StrictUndefined` and injects device
table names before executing through `Context`.
- Templates that differ by SnapshotDB callstack layout declare
`query_variants.v1` / `query_variants.v2` in the same YAML entry and share
description, parameters, and `output_schema`. Unaffected templates keep a
single `query`. `Context` detects the layout from read-only column/table
checks (with `pt_snap_metadata` as an auxiliary check) and
`QueryExecutor` selects the matching SQL. Conflicting or damaged layouts
leave `callstack_layout` unset and fail only variant templates. Do not
persist layout in `focus.json` or migrate databases on open.
- When template metadata or behavior changes, review the YAML, config, registry,
executor, `core/query_service.py`, CLI/MCP presentation, and focused tests
together.
Expand Down
55 changes: 49 additions & 6 deletions docs/en/database.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,11 +48,17 @@ pt-snap metadata snapshot.pkl.db --json
pt-snap import snapshot.pkl --force
```

Legacy or externally generated compatible databases without this table remain queryable for
templates that do not require the deduplicated callstack schema. Callstack templates require a
database produced by the current importer; otherwise the query reports that the snapshot must be
re-imported. Their metadata status is reported as unavailable, and a later import rebuilds them
once before reuse.
Legacy or externally generated compatible databases without this table remain
Comment thread
codeasier marked this conversation as resolved.
Resolved
queryable for templates that do not require a recognized callstack layout.
`pt-snap` detects the callstack layout from table columns (and treats
`pt_snap_metadata.import_format_version` as an auxiliary check, not the only
source of truth). Callstack templates use the matching SQL automatically.

Opening, focusing, and querying a database never writes the `.db` file and never
migrates it in place. Import cache invalidation still rebuilds a destination
when the current importer's format version does not match; that is separate from
read compatibility. `pt-snap metadata` reports missing metadata as unavailable,
and a later import of the same pickle rebuilds the database once before reuse.

### Query workflow

Expand All @@ -67,7 +73,11 @@ pt-snap query --template-use block --params '{"min_size": 1048576}'
```

Use `pt-snap query --list` and `pt-snap query --template-info <name>` to inspect
the supported query surface. See [Querying](querying.md) for the complete workflow.
the supported query surface. `event`, `callstack_analysis`, and
`active_memory_callstack_at_event` work on both the current deduplicated
callstack schema and older inline-text databases; see
[Callstack layout compatibility](database.md#callstack-layout-compatibility).
See [Querying](querying.md) for the complete workflow.

---

Expand Down Expand Up @@ -311,6 +321,39 @@ CREATE TABLE callstack (
| Scope | Shared by every device in the database; the table has no device suffix |
| Coverage | Every event produced by import resolves to a row; `callstackId` is `NULL` only in externally generated databases |

Current `pt-snap import` writes this v2 layout (`import_format_version = 2`).
Databases produced before callstack deduplication store inline `callstack` TEXT
on each `trace_entry_<device>` table and have no shared `callstack` table
(`import_format_version = 1` when metadata is present).

### Callstack layout compatibility

`pt-snap` reads both layouts without rewriting the file:

| Layout | How to recognize it | Callstack text |
|--------|---------------------|----------------|
| v2 (current import) | `trace_entry_<device>.callstackId` plus shared `callstack` | Join `callstack.id` |
| v1 (legacy) | `trace_entry_<device>.callstack` TEXT, no `callstackId`, no shared table | Read the column directly |

`event`, `callstack_analysis`, and `active_memory_callstack_at_event` keep one
template name, parameter set, and output schema. The query engine selects v1 or
v2 SQL from the detected layout. Other templates do not depend on this split.

Detection is read-only: `focus`, `query`, reports, the Python API, and MCP all
open the database with SQLite `mode=ro`. Layout is not stored in
`.pt-snap/focus.json`. Conflicting columns, mixed devices, a damaged
`callstack` table, or metadata that disagrees with the physical schema leave
the layout unrecognized and fail only callstack templates, instead of guessing
or blocking focus, metadata, and other queries. There is no implicit v1→v2
migration.

```bash
pt-snap focus legacy.db
# Callstack layout: v1 (inline text)
pt-snap query --template-use event
pt-snap query --template-use callstack_analysis
```

### 5. pt_snap_metadata — Import Metadata Table

First-party imports create one metadata row for provenance and cache
Expand Down
2 changes: 1 addition & 1 deletion docs/en/focus-management.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ its device, becomes effective again only after the session override is unset.
pt-snap focus
```

This shows the resolved database path, device ID, and where they came from (project focus, session env, or global config).
This shows the resolved database path, device ID, and where they came from (project focus, session env, or global config). `pt-snap focus`, `pt-snap focus <db>`, `pt-snap focus --global`, and `pt-snap focus --device` also print `Callstack layout: v1 (inline text)` or `v2 (deduplicated)` when the schema is recognized, or a warning when the layout conflicts. `pt-snap focus --session` prints only the `export PT_SNAP_DB_PATH=...` line so it can be evaluated by the shell. The layout is detected read-only from the database; it is not stored in `.pt-snap/focus.json`.

## Override Focus

Expand Down
4 changes: 4 additions & 0 deletions docs/en/querying.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,10 @@ Raw data lookup.
| `event` | Query memory events with flexible field filters |
| `allocation` | Memory allocation timeline (id, allocated, active, reserved) |

`event`, `callstack_analysis`, and `active_memory_callstack_at_event` share one
public contract and select v1 or v2 SQL from the database layout. See
[Callstack layout compatibility](database.md#callstack-layout-compatibility).

### Statistical Queries

Aggregation and analysis.
Expand Down
3 changes: 3 additions & 0 deletions docs/en/snapshot-analyzer-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ state = analyzer.get_focus()
print(state.db_path)
print(state.device_id)
print(state.available_devices)
print(state.callstack_layout)
print(state.source)

state = analyzer.set_focus(
Expand Down Expand Up @@ -61,6 +62,8 @@ explicit `db_path`, a validated device-only analyzer override is reported by
| `device_id` | Analyzer device override, or the device attached to the resolved focus |
| `source` | Resolution source such as `explicit`, `env`, `project`, `global`, or `none` |
| `available_devices` | Device IDs discovered from `trace_entry_<device>` tables |
| `callstack_layout` | `"v1"` for inline callstack text, `"v2"` for deduplicated `callstackId`, or `None` when unrecognized or conflicting |
| `callstack_layout_error` | Conflict reason when the layout cannot be used, otherwise `None` |

See [Focus Management](focus-management.md) for the complete resolution and
persistence model.
Expand Down
46 changes: 42 additions & 4 deletions docs/zh/database.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,9 +44,14 @@ pt-snap metadata snapshot.pkl.db --json
pt-snap import snapshot.pkl --force
```

旧版或外部生成且结构兼容的 DB,仍可用于不依赖去重调用栈结构的查询。调用栈相关模板要求
数据库由当前导入器生成;否则会提示需要重新导入快照。若没有 metadata,查询会返回
unavailable,下一次执行对应 pickle 导入时会重建一次。
旧版或外部生成且结构兼容的 DB 仍可查询不依赖可识别调用栈布局的模板。`pt-snap`
根据表列识别调用栈布局,并把 `pt_snap_metadata.import_format_version` 当作辅助
校验,而不是唯一依据。调用栈相关模板会自动选择匹配的 SQL。

打开、focus 和查询都不会写入 `.db` 文件,也不会原地迁移 schema。导入缓存仍会在
当前导入器格式版本不匹配时重建目标库,这与读取兼容是分开的。没有 metadata 时,
`pt-snap metadata` 会报告 unavailable;之后对同一 pickle 再导入会先重建一次再参与
缓存复用。

### 查询工作流

Expand All @@ -60,7 +65,9 @@ pt-snap query --template-use block --params '{"min_size": 1048576}'
```

使用 `pt-snap query --list` 和 `pt-snap query --template-info <name>` 查看受支持的
查询入口。完整流程见[运行查询](querying.md)。
查询入口。`event`、`callstack_analysis` 和 `active_memory_callstack_at_event`
同时支持当前去重调用栈结构和旧版内联文本库;见
[调用栈布局兼容](database.md#调用栈布局兼容)。完整流程见[运行查询](querying.md)。

---

Expand Down Expand Up @@ -301,6 +308,37 @@ CREATE TABLE callstack (
| 作用域 | 数据库内所有设备共享,表名不带设备后缀 |
| 覆盖度 | 导入产生的每个事件都能解析到一行;只有外部生成的数据库才可能出现 `callstackId` 为 `NULL` |

当前 `pt-snap import` 写入的是这种 v2 布局(`import_format_version = 2`)。
调用栈去重之前生成的数据库在每张 `trace_entry_<device>` 表中内联存储
`callstack` TEXT,没有共享 `callstack` 表(有 metadata 时为
`import_format_version = 1`)。

### 调用栈布局兼容

`pt-snap` 可以读取两种布局,且不会改写数据库文件:

| 布局 | 识别方式 | 调用栈文本 |
|------|----------|------------|
| v2(当前导入) | `trace_entry_<device>.callstackId` 加共享 `callstack` 表 | join `callstack.id` |
| v1(旧版) | `trace_entry_<device>.callstack` TEXT,无 `callstackId`,无共享表 | 直接读该列 |

`event`、`callstack_analysis` 和 `active_memory_callstack_at_event` 保持相同的
模板名、参数和输出契约。查询引擎按识别到的布局选择 v1 或 v2 SQL。其他模板不依赖
这次拆分。

识别过程是只读的:`focus`、`query`、报告、Python API 和 MCP 都以 SQLite
`mode=ro` 打开数据库。布局不会写入 `.pt-snap/focus.json`。列冲突、设备布局不一致、
损坏的 `callstack` 表,或 metadata 与物理结构不一致时,布局保持无法识别,并且
只有调用栈模板会报错;不会猜测布局,也不会阻止 focus、metadata 和其他查询。
默认不会做 v1→v2 迁移。

```bash
pt-snap focus legacy.db
# Callstack layout: v1 (inline text)
pt-snap query --template-use event
pt-snap query --template-use callstack_analysis
```

### 5. pt_snap_metadata — 导入 Metadata 表

第一方导入会创建一条 metadata 记录,用于来源追踪和缓存兼容性检查:
Expand Down
2 changes: 1 addition & 1 deletion docs/zh/focus-management.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ Session focus 只导出 `PT_SNAP_DB_PATH`,因此 `--session` 不能与 `--devi
pt-snap focus
```

显示已解析的数据库路径、设备 ID 及其来源(项目 focus、session 环境变量或全局配置)。
显示已解析的数据库路径、设备 ID 及其来源(项目 focus、session 环境变量或全局配置)。`pt-snap focus`、`pt-snap focus <db>`、`pt-snap focus --global` 和 `pt-snap focus --device` 在识别到调用栈 schema 时还会打印 `Callstack layout: v1 (inline text)` 或 `v2 (deduplicated)`;布局冲突时打印警告。`pt-snap focus --session` 只输出 `export PT_SNAP_DB_PATH=...`,以便被 shell 直接执行。该信息从数据库只读识别,不会写入 `.pt-snap/focus.json`。

## 覆盖焦点

Expand Down
4 changes: 4 additions & 0 deletions docs/zh/querying.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,10 @@ pt-snap query [DB_PATH] [--template-use <template_name>] [--params <json>] \
| `event` | 灵活字段过滤的内存事件查询 |
| `allocation` | 内存分配时间线(id, allocated, active, reserved) |

`event`、`callstack_analysis` 和 `active_memory_callstack_at_event` 对外保持同一契约,
并根据数据库布局选择 v1 或 v2 SQL。见
[调用栈布局兼容](database.md#调用栈布局兼容)。

### Statistical Queries

聚合分析。
Expand Down
3 changes: 3 additions & 0 deletions docs/zh/snapshot-analyzer-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ state = analyzer.get_focus()
print(state.db_path)
print(state.device_id)
print(state.available_devices)
print(state.callstack_layout)
print(state.source)

state = analyzer.set_focus(
Expand Down Expand Up @@ -56,6 +57,8 @@ focus 中的设备。没有显式 `db_path` 时,已验证的 analyzer 设备
| `device_id` | analyzer 设备覆盖;未设置覆盖时为已解析 focus 所附带的设备 |
| `source` | `explicit`、`env`、`project`、`global`、`none` 等解析来源 |
| `available_devices` | 从 `trace_entry_<device>` 表发现的设备 ID |
| `callstack_layout` | 内联调用栈文本为 `"v1"`,去重 `callstackId` 为 `"v2"`,无法识别或冲突时为 `None` |
| `callstack_layout_error` | 布局无法使用时的冲突原因,否则为 `None` |

完整解析和持久化模型见 [Focus 管理](focus-management.md)。

Expand Down
4 changes: 4 additions & 0 deletions src/pt_snap_cli/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,10 @@
- Preserve focus precedence: explicit CLI/API path, `PT_SNAP_DB_PATH`, nearest project `.pt-snap/focus.json`, then legacy global config.
- Keep SQLite access read-only for analysis paths.
- When adding or changing a repository skill under `skills/`, copy the same `SKILL.md` into `bundled_skills/<name>/`.
- `Context.callstack_layout` is `v1`, `v2`, or `None`. Conflicting layouts leave
it `None` and set `callstack_layout_error`; construction still only requires
the `dictionary` table. Query SQL variants are selected by `QueryExecutor`,
not by adapters.

### Testing Requirements
- CLI changes should update/run `tests/test_cli.py` and any affected config/focus tests.
Expand Down
4 changes: 4 additions & 0 deletions src/pt_snap_cli/api.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,8 @@ class FocusState:
device_id: int | None
source: str
available_devices: list[int]
callstack_layout: str | None = None
callstack_layout_error: str | None = None


class SnapshotAnalyzer:
Expand Down Expand Up @@ -71,6 +73,8 @@ def get_focus(self) -> FocusState:
device_id=self._device_id if self._device_id is not None else state.device_id,
source=state.source,
available_devices=state.available_devices,
callstack_layout=state.callstack_layout,
callstack_layout_error=state.callstack_layout_error,
)

def set_focus(self, db_path: str | None = None, device_id: int | None = None) -> FocusState:
Expand Down
12 changes: 12 additions & 0 deletions src/pt_snap_cli/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -151,6 +151,7 @@ def focus_database(
typer.echo(f"Focus file: {state.focus_file}")
if state.device_id is not None:
typer.echo(f"Focused device: {state.device_id}")
_echo_callstack_layout(state.callstack_layout, state.callstack_layout_error)
if not state.db_path.exists():
typer.secho("Warning: Database file does not exist!", fg=typer.colors.YELLOW)
else:
Expand Down Expand Up @@ -179,6 +180,7 @@ def focus_database(
typer.secho(f"Focused device ({state.source}): {device}", fg=typer.colors.GREEN)
if state.source == "project" and state.focus_file:
typer.echo(f"Focus file: {state.focus_file}")
_echo_callstack_layout(state.callstack_layout, state.callstack_layout_error)
raise typer.Exit()

try:
Expand All @@ -203,6 +205,7 @@ def focus_database(
typer.echo(f"Available devices: {', '.join(map(str, state.available_devices))}")
else:
typer.echo("No devices found in database.")
_echo_callstack_layout(state.callstack_layout, state.callstack_layout_error)
except (
DatabaseMissingError,
DatabaseSchemaError,
Expand Down Expand Up @@ -971,6 +974,15 @@ def show_config(
typer.echo(f" {key}: {value}")


def _echo_callstack_layout(layout: str | None, error: str | None = None) -> None:
if layout == "v1":
typer.echo("Callstack layout: v1 (inline text)")
elif layout == "v2":
typer.echo("Callstack layout: v2 (deduplicated)")
elif error:
typer.secho(f"Warning: {error}", fg=typer.colors.YELLOW)


def _error(message: str) -> NoReturn:
typer.secho(f"Error: {message}", fg=typer.colors.RED)
raise typer.Exit(1) from None
Expand Down
Loading
Loading