Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,23 @@ jobs:
- name: basedPyright check
run: python -m basedpyright --pythonpath "$(python -c 'import sys; print(sys.executable)')"

browser:
name: Frame-tree browser regression
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.13"
- name: Install browser test dependencies
run: |
pip install -e ".[dev]"
python -m playwright install --with-deps chromium
- name: Run required Chromium interaction tests
env:
PT_SNAP_REQUIRE_BROWSER: "1"
run: pytest tests/test_fixture_provenance.py tests/test_memory_tree_browser.py

test:
name: Test (Python ${{ matrix.python-version }})
runs-on: ubuntu-latest
Expand Down
11 changes: 11 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,17 @@ python -m basedpyright --pythonpath "$(python -c 'import sys; print(sys.executab
git diff --check
```

HTML interaction regressions run in a separate Chromium CI job. To run locally:

```bash
python -m playwright install chromium
PT_SNAP_REQUIRE_BROWSER=1 pytest tests/test_memory_tree_browser.py
```

Without an installed browser, the ordinary suite skips these browser tests. The
required-browser flag makes missing browser support fail. `PT_SNAP_TEST_CHROMIUM`
may point to an existing Chromium executable instead of downloading one.

- Type checking covers all `src/pt_snap_cli`, including `snapshot/`. Zero errors
is the gate; warnings are informational. Preserve the file-level Any/Unknown
error gates in `api.py`, `core/models.py`, and `snapshot/representation.py`;
Expand Down
9 changes: 9 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,7 @@ accept explicitly selected complete validated datasets, not arbitrary directorie
| `pt-snap overview [database.db]` | Read-only device list, per-device event-id bounds, and import-metadata status |
| `pt-snap query` | Run memory analysis queries |
| `pt-snap report` | Generate higher-level memory analysis reports |
| `pt-snap report memory-tree --event-id <id>` | Frame-level live-memory breakdown; `--format html` emits an interactive flamegraph |
| `pt-snap config` | Manage global configuration |
| `pt-snap skill` | List and install bundled agent skills |

Expand All @@ -109,6 +110,14 @@ Peak reports expose attribution completeness and same-event active-byte coverage
group percentages use included bytes after filtering and ranking. See the
[report guide](docs/en/querying.md#report-command) before interpreting a capped breakdown.

Standalone imports preserve ordered structured callstack frames. Use `event_frames` to
inspect them, or `report memory-tree --event-id 100 --format html > memory-tree.html`
to explore event-time occupancy by caller path. See
[frame memory breakdown](docs/en/querying.md#frame-memory-breakdown); older
text-only databases require re-importing the original snapshot.
`report memory-tree` also accepts `--json`. The HTML view has full-tree search and
keyboard-accessible child lists, so tiny and zero-byte frames remain reachable.

## Agent Skills

For long diagnostic stacks, opt into `stack_bytes` on `event` or
Expand Down
8 changes: 8 additions & 0 deletions README_zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,7 @@ pt-snap split snapshot.pkl --slices 4 --output snapshot-slices
| `pt-snap overview [database.db]` | 只读输出设备列表、各设备 event id 边界和导入 metadata 状态 |
| `pt-snap query` | 运行内存分析查询 |
| `pt-snap report` | 生成高层内存分析报告 |
| `pt-snap report memory-tree --event-id <id>` | 按帧分层拆解存活内存;`--format html` 输出交互式火焰图 |
| `pt-snap config` | 管理全局配置 |
| `pt-snap skill` | 列出并安装随包 agent skill |

Expand All @@ -96,6 +97,13 @@ pt-snap split snapshot.pkl --slices 4 --output snapshot-slices
included bytes 为分母。解读有上限的分组结果前,请参阅
[报告指南](docs/zh/querying.md#report-命令)。

单库导入时保留结构化调用栈帧及原始顺序。用 `event_frames` 查看帧,或执行
`report memory-tree --event-id 100 --format html > memory-tree.html`
按调用路径查看事件时刻的占用。详见[调用栈帧内存拆解](docs/zh/querying.md#调用栈帧内存拆解);
旧文本数据库需要从原始快照重新导入。
`report memory-tree` 同样支持 `--json`。HTML 支持全树搜索和可键盘操作的子节点列表,
极小占用及零字节帧也能进入查看。

## Agent Skills

诊断长调用栈时,可在 `event` 或 `active_memory_callstack_at_event` 上显式设置
Expand Down
57 changes: 54 additions & 3 deletions docs/en/database.md
Original file line number Diff line number Diff line change
Expand Up @@ -331,15 +331,65 @@ CREATE TABLE callstack (

| Property | Guarantee |
|----------|-----------|
| Uniqueness | One row per distinct callstack text, so grouping by `callstackId` equals grouping by text |
| Uniqueness | One row per distinct ordered structured frame array; display text alone is not a lossless identity |
| Scope | Shared by every device in the database; the table has no device suffix |
| Coverage | Every event produced by import resolves to a row; `callstackId` is `NULL` only in externally generated databases |

Current `pt-snap import` writes this v2 layout (`import_format_version = 2`).
Current standalone `pt-snap import` retains this v2 text layout and adds structured frame
tables (`import_format_version = 3`). Format 2 databases have the same text
layout but no original frame records.
Databases produced before callstack deduplication store inline `callstack` TEXT
on each `trace_entry_<device>` table and have no shared `callstack` table
(`import_format_version = 1` when metadata is present).

### Structured frames and ordered arrays (format 3)

`frame` stores each distinct original `(filename, line, name)` tuple once.
`callstack_frame` associates a stack with an ordered array of frame IDs:

```sql
CREATE TABLE frame (
id INTEGER PRIMARY KEY,
filename TEXT NOT NULL,
line INTEGER NOT NULL,
name TEXT NOT NULL
);
CREATE TABLE callstack_frame (
callstackId INTEGER NOT NULL,
position INTEGER NOT NULL,
frameId INTEGER NOT NULL
);
CREATE UNIQUE INDEX idx_callstack_frame_position
ON callstack_frame (callstackId, position);
```

These tables are shared across devices. An event refers to the array through
`trace_entry_<device>.callstackId`; read it in `position ASC` order to reproduce
the original snapshot array. Position 0 is the innermost frame. The frame tree
reverses traversal to start with the outermost caller. Recursive occurrences
retain distinct positions even when they share a frame ID. Empty arrays have
no association rows and a manifest frameCount of zero. Structured fields preserve colons, whitespace, Unicode
and embedded newlines without parsing display strings.

Use `event_frames` to read an event's ordered frames and
`active_memory_frame_tree_at_event` to obtain parent/child occupancy.
See [Frame memory breakdown](querying.md#frame-memory-breakdown).
Text-only databases remain readable by existing queries, but these new
queries require re-importing the original snapshot. Import detects the changed
format and rebuilds the cache; read-only analysis never migrates a database.

### Standalone frame capability boundary

Format 3 applies only to standalone imports. Native sharded imports remain format 2
and compatible exports remain format 1; neither claims the standalone frame-tree
capability. `callstack_frame_manifest(callstackId INTEGER PRIMARY KEY,
frameCount INTEGER NOT NULL)` records the expected length of every captured stack,
including zero. Frame fields must be text filename/name and an integer line.
`event_frames` returns no rows for absent, known-empty or corrupt arrays, never a
partial reconstruction. Trees keep known-empty captured stacks separate from
missing/corrupt evidence. Dropped tail/all links, noninteger positions and invalid
field types fail validation. Read-only queries never repair stored evidence.

### Callstack layout compatibility

`pt-snap` reads both layouts without rewriting the file:
Expand All @@ -351,7 +401,8 @@ on each `trace_entry_<device>` table and have no shared `callstack` table

`event`, `callstack_analysis`, and `active_memory_callstack_at_event` keep one
template name, parameter set, and output schema. The query engine selects v1 or
v2 SQL from the detected layout. Other templates do not depend on this split.
v2 SQL from the detected layout. Structured-frame queries additionally require
the format 3 tables; their frame evidence is not reconstructed from legacy text.

Detection is read-only: `focus`, `query`, reports, and the Python API all
open the database with SQLite `mode=ro`. Layout is not stored in
Expand Down
74 changes: 74 additions & 0 deletions docs/en/querying.md
Original file line number Diff line number Diff line change
Expand Up @@ -674,6 +674,80 @@ explanations; look up `semantics_version` and interpretation limits with
rows. `SnapshotAnalyzer.execute_query()` results also include `template`
and `semantics_version` so that lookup stays tied to the rows.

## Frame Memory Breakdown

After a standalone import with this version, inspect the original ordered frame array or
decompose live memory after an event:

```bash
pt-snap query --template-use event_frames --params '{"event_id":100}' --json
pt-snap query --template-use active_memory_frame_tree_at_event --params '{"event_id":100}' --json
pt-snap report memory-tree --event-id 100
pt-snap report memory-tree --event-id 100 --json
pt-snap report memory-tree --event-id 100 --format html > memory-tree.html
```

The HTML file is self-contained: open it locally, click a frame to zoom, reset
to show the whole tree, or search filenames/functions. Width measures live
block bytes, not CPU time or allocation traffic. No static server is needed.
`--device` selects the device; `--exclude-static` omits static/preexisting
memory; `--min-size` filters individual blocks. `--event-id` is required.
The report requests a complete tree without a ranked or row cap.

| Field | Meaning |
| --- | --- |
| `node_id`, `parent_id` | Caller-path identity and direct parent; `root` is the total |
| `depth`, `frame_id` | Caller-first level and structured frame identity |
| `size_bytes`, `requested_bytes`, `block_count` | Inclusive occupancy/count for this node and descendants |
| `self_bytes`, `self_requested_bytes`, `self_block_count` | Allocations whose recorded stack ends at this node |
| `percent_of_total` | 0–100 byte share of root occupancy after block/category filters, before output row caps |

For each node, inclusive occupancy equals self occupancy plus the inclusive
occupancy of its direct children. Sum `self_bytes` across all nodes, or sum
inclusive bytes at one complete cut through the tree; summing inclusive bytes
across levels counts allocations repeatedly. An internal node can have self
bytes when a recorded stack ends there. Recursion and identical frames under
different callers remain separate paths.

Only blocks with `allocEventId <= event_id` and a later or absent
`freeEventId` are attributed dynamically. `freeEventId` is `free_completed`,
so pending-free memory is still included. Static, preexisting, and missing or
damaged stack links remain explicit synthetic leaves. This is active-block
occupancy and does not decompose reserved bytes. Event IDs mark ordering, not
elapsed time. `min_size` and `include_static` change the root denominator.
If a direct query uses `-n`/`max_rows`, inspect its completeness flags: a
truncated set of node rows is not a complete tree. Old text-only databases
must be re-imported; the analyzer does not reconstruct frames from text.

The HTML chart keeps byte-proportional widths. Tiny and zero-byte frames are
still reachable through the direct-child list or the search results for the
entire tree, even when zoomed into another branch. Search matches filename,
function or node ID; each result shows its full caller path, inclusive bytes and
node ID to distinguish identical labels. Lists include every match and support
Tab then Enter/Space. Parent frame moves up one level; Reset zoom returns to the
root without clearing the search. Clear the search field to remove the results.

Frame-tree templates and `report memory-tree` support standalone format-3 databases
only. Dataset roots are rejected explicitly; the analyzer never selects a shard
implicitly. Direct native shards have no standalone frame tables: re-import the
original snapshot as a standalone database to use these templates. Existing
native `pt_snap_frame`/coverage readers remain unchanged. `--timeout` uses the
shared query deadline. Text and HTML renderers reject incomplete or nonconserving
trees. Known-empty stacks have an `empty_callstack` synthetic leaf;
missing/corrupt stacks retain the `missing` leaf. `event_frames` returns no rows
for missing events/stacks, known-empty stacks or invalid frame arrays.

The Python API uses the same template and contracts:

```python
from pathlib import Path
from pt_snap_cli import SnapshotAnalyzer

with SnapshotAnalyzer(db_path=Path("snapshot.pkl.db"), device_id=0) as analyzer:
frames = analyzer.execute_query("event_frames", params={"event_id": 100})
tree = analyzer.execute_query("active_memory_frame_tree_at_event", params={"event_id": 100})
```

## Template Architecture

Query templates are defined in YAML format with:
Expand Down
50 changes: 46 additions & 4 deletions docs/zh/database.md
Original file line number Diff line number Diff line change
Expand Up @@ -316,15 +316,57 @@ CREATE TABLE callstack (

| 属性 | 保证 |
|------|------|
| 唯一性 | 每条不同的调用栈文本只有一行,因此按 `callstackId` 分组等价于按文本分组 |
| 唯一性 | 每组不同的有序结构化帧数组只有一行;显示文本本身不能无损标识调用栈 |
| 作用域 | 数据库内所有设备共享,表名不带设备后缀 |
| 覆盖度 | 导入产生的每个事件都能解析到一行;只有外部生成的数据库才可能出现 `callstackId` 为 `NULL` |

当前 `pt-snap import` 写入的是这种 v2 布局(`import_format_version = 2`)。
当前 `pt-snap import` 保留这种 v2 文本布局,同时增加结构化帧表
(`import_format_version = 3`)。格式 2 数据库的文本布局相同,但没有原始帧记录。
调用栈去重之前生成的数据库在每张 `trace_entry_<device>` 表中内联存储
`callstack` TEXT,没有共享 `callstack` 表(有 metadata 时为
`import_format_version = 1`)。

### 结构化帧与有序数组(格式 3)

`frame` 对原始 `(filename, line, name)` 三元组去重;`callstack_frame`
把调用栈关联到有序的帧 ID 数组:

```sql
CREATE TABLE frame (
id INTEGER PRIMARY KEY,
filename TEXT NOT NULL,
line INTEGER NOT NULL,
name TEXT NOT NULL
);
CREATE TABLE callstack_frame (
callstackId INTEGER NOT NULL,
position INTEGER NOT NULL,
frameId INTEGER NOT NULL
);
CREATE UNIQUE INDEX idx_callstack_frame_position
ON callstack_frame (callstackId, position);
```

两张表在设备之间共享。事件通过 `trace_entry_<device>.callstackId` 关联数组,
按 `position ASC` 读取即可还原快照原始顺序;位置 0 是最内层帧。帧树分析反向
遍历,从最外层调用者开始。递归中的同一帧保留各自位置,即使帧 ID 相同。
空数组没有关联行。结构化字段直接保留冒号、空白、中文和内嵌换行,无需拆解显示文本。

使用 `event_frames` 读取事件有序帧,使用 `active_memory_frame_tree_at_event`
读取各父子节点的占用。详见[调用栈帧内存拆解](querying.md#调用栈帧内存拆解)。
旧文本数据库仍可用于原有查询,新查询需要从原始快照重新导入。导入器检测格式变化
后重建缓存;只读分析不会迁移数据库。

### 单库帧能力边界

格式 3 仅用于单库导入。原生分片导入保持格式 2,兼容导出保持格式 1;
它们不会声称支持单库帧树。新增 `callstack_frame_manifest(callstackId INTEGER PRIMARY KEY,
frameCount INTEGER NOT NULL)` 保存每个采集调用栈的预期长度,包括零。
filename/name 必须是文本,line 必须是整数。`event_frames` 对缺失、已知空数组
或损坏数组返回零行,不输出部分重建的数组。树将已知空栈与缺失/损坏证据分开。
删除末尾或全部关联、非整数位置以及无效字段类型均不能通过校验。
只读查询不会修复已存储的证据。

### 调用栈布局兼容

`pt-snap` 可以读取两种布局,且不会改写数据库文件:
Expand All @@ -335,8 +377,8 @@ CREATE TABLE callstack (
| v1(旧版) | `trace_entry_<device>.callstack` TEXT,无 `callstackId`,无共享表 | 直接读该列 |

`event`、`callstack_analysis` 和 `active_memory_callstack_at_event` 保持相同的
模板名、参数和输出契约。查询引擎按识别到的布局选择 v1 或 v2 SQL。其他模板不依赖
这次拆分。
模板名、参数和输出契约。查询引擎按识别到的布局选择 v1 或 v2 SQL。
结构化帧查询额外要求格式 3 的两张帧表,不会从旧显示文本猜测原始帧。

识别过程是只读的:`focus`、`query`、报告和 Python API 都以 SQLite
`mode=ro` 打开数据库。布局不会写入 `.pt-snap/focus.json`。列冲突、设备布局不一致、
Expand Down
Loading
Loading