Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@ See the [full quick start guide](docs/en/quickstart.md) for a walkthrough.
| `pt-snap overview [database.db]` | Read-only device list, per-device event-id bounds, and import-metadata status |
| `pt-snap query` | Run memory analysis queries |
| `pt-snap report` | Generate higher-level memory analysis reports |
| `pt-snap report memory-tree --event-id <id>` | Frame-level live-memory breakdown; `--format html` emits an interactive flamegraph |
| `pt-snap config` | Manage global configuration |
| `pt-snap skill` | List and install bundled agent skills |

Expand All @@ -73,6 +74,13 @@ Peak reports expose attribution completeness and same-event active-byte coverage
group percentages use included bytes after filtering and ranking. See the
[report guide](docs/en/querying.md#report-command) before interpreting a capped breakdown.

Imports preserve ordered structured callstack frames. Use `event_frames` to
inspect them, or `report memory-tree --event-id 100 --format html > memory-tree.html`
to explore event-time occupancy by caller path. See
[frame memory breakdown](docs/en/querying.md#frame-memory-breakdown); older
text-only databases require re-importing the original snapshot.
`report memory-tree` also accepts `--json`.

## Agent Skills

For long diagnostic stacks, opt into `stack_bytes` on `event` or
Expand Down
7 changes: 7 additions & 0 deletions README_zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,7 @@ pt-snap split snapshot.pkl --slices 4 --output snapshot-slices
| `pt-snap overview [database.db]` | 只读输出设备列表、各设备 event id 边界和导入 metadata 状态 |
| `pt-snap query` | 运行内存分析查询 |
| `pt-snap report` | 生成高层内存分析报告 |
| `pt-snap report memory-tree --event-id <id>` | 按帧分层拆解存活内存;`--format html` 输出交互式火焰图 |
| `pt-snap config` | 管理全局配置 |
| `pt-snap skill` | 列出并安装随包 agent skill |

Expand All @@ -68,6 +69,12 @@ pt-snap split snapshot.pkl --slices 4 --output snapshot-slices
included bytes 为分母。解读有上限的分组结果前,请参阅
[报告指南](docs/zh/querying.md#report-命令)。

导入时保留结构化调用栈帧及原始顺序。用 `event_frames` 查看帧,或执行
`report memory-tree --event-id 100 --format html > memory-tree.html`
按调用路径查看事件时刻的占用。详见[调用栈帧内存拆解](docs/zh/querying.md#调用栈帧内存拆解);
旧文本数据库需要从原始快照重新导入。
`report memory-tree` 同样支持 `--json`。

## Agent Skills

诊断长调用栈时,可在 `event` 或 `active_memory_callstack_at_event` 上显式设置
Expand Down
45 changes: 42 additions & 3 deletions docs/en/database.md
Original file line number Diff line number Diff line change
Expand Up @@ -327,15 +327,53 @@ CREATE TABLE callstack (

| Property | Guarantee |
|----------|-----------|
| Uniqueness | One row per distinct callstack text, so grouping by `callstackId` equals grouping by text |
| Uniqueness | One row per distinct ordered structured frame array; display text alone is not a lossless identity |
| Scope | Shared by every device in the database; the table has no device suffix |
| Coverage | Every event produced by import resolves to a row; `callstackId` is `NULL` only in externally generated databases |

Current `pt-snap import` writes this v2 layout (`import_format_version = 2`).
Current `pt-snap import` retains this v2 text layout and adds structured frame
tables (`import_format_version = 3`). Format 2 databases have the same text
layout but no original frame records.
Databases produced before callstack deduplication store inline `callstack` TEXT
on each `trace_entry_<device>` table and have no shared `callstack` table
(`import_format_version = 1` when metadata is present).

### Structured frames and ordered arrays (format 3)

`frame` stores each distinct original `(filename, line, name)` tuple once.
`callstack_frame` associates a stack with an ordered array of frame IDs:

```sql
CREATE TABLE frame (
id INTEGER PRIMARY KEY,
filename TEXT NOT NULL,
line INTEGER NOT NULL,
name TEXT NOT NULL
);
CREATE TABLE callstack_frame (
callstackId INTEGER NOT NULL,
position INTEGER NOT NULL,
frameId INTEGER NOT NULL
);
CREATE UNIQUE INDEX idx_callstack_frame_position
ON callstack_frame (callstackId, position);
```

Both tables are shared across devices. An event refers to the array through
`trace_entry_<device>.callstackId`; read it in `position ASC` order to reproduce
the original snapshot array. Position 0 is the innermost frame. The frame tree
reverses traversal to start with the outermost caller. Recursive occurrences
retain distinct positions even when they share a frame ID. Empty arrays have
no association rows. Structured fields preserve colons, whitespace, Unicode
and embedded newlines without parsing display strings.

Use `event_frames` to read an event's ordered frames and
`active_memory_frame_tree_at_event` to obtain parent/child occupancy.
See [Frame memory breakdown](querying.md#frame-memory-breakdown).
Text-only databases remain readable by existing queries, but these new
queries require re-importing the original snapshot. Import detects the changed
format and rebuilds the cache; read-only analysis never migrates a database.

### Callstack layout compatibility

`pt-snap` reads both layouts without rewriting the file:
Expand All @@ -347,7 +385,8 @@ on each `trace_entry_<device>` table and have no shared `callstack` table

`event`, `callstack_analysis`, and `active_memory_callstack_at_event` keep one
template name, parameter set, and output schema. The query engine selects v1 or
v2 SQL from the detected layout. Other templates do not depend on this split.
v2 SQL from the detected layout. Structured-frame queries additionally require
the format 3 tables; their frame evidence is not reconstructed from legacy text.

Detection is read-only: `focus`, `query`, reports, and the Python API all
open the database with SQLite `mode=ro`. Layout is not stored in
Expand Down
56 changes: 56 additions & 0 deletions docs/en/querying.md
Original file line number Diff line number Diff line change
Expand Up @@ -441,6 +441,62 @@ explanations; look up `semantics_version` and interpretation limits with
rows. `SnapshotAnalyzer.execute_query()` results also include `template`
and `semantics_version` so that lookup stays tied to the rows.

## Frame Memory Breakdown

After importing with this version, inspect the original ordered frame array or
decompose live memory after an event:

```bash
pt-snap query --template-use event_frames --params '{"event_id":100}' --json
pt-snap query --template-use active_memory_frame_tree_at_event --params '{"event_id":100}' --json
pt-snap report memory-tree --event-id 100
pt-snap report memory-tree --event-id 100 --json
pt-snap report memory-tree --event-id 100 --format html > memory-tree.html
```

The HTML file is self-contained: open it locally, click a frame to zoom, reset
to show the whole tree, or search filenames/functions. Width measures live
block bytes, not CPU time or allocation traffic. No static server is needed.
`--device` selects the device; `--exclude-static` omits static/preexisting
memory; `--min-size` filters individual blocks. `--event-id` is required.
The report requests a complete tree without a ranked or row cap.

| Field | Meaning |
| --- | --- |
| `node_id`, `parent_id` | Caller-path identity and direct parent; `root` is the total |
| `depth`, `frame_id` | Caller-first level and structured frame identity |
| `size_bytes`, `requested_bytes`, `block_count` | Inclusive occupancy/count for this node and descendants |
| `self_bytes`, `self_requested_bytes`, `self_block_count` | Allocations whose recorded stack ends at this node |
| `percent_of_total` | 0–100 byte share of root occupancy after block/category filters, before output row caps |

For each node, inclusive occupancy equals self occupancy plus the inclusive
occupancy of its direct children. Sum `self_bytes` across all nodes, or sum
inclusive bytes at one complete cut through the tree; summing inclusive bytes
across levels counts allocations repeatedly. An internal node can have self
bytes when a recorded stack ends there. Recursion and identical frames under
different callers remain separate paths.

Only blocks with `allocEventId <= event_id` and a later or absent
`freeEventId` are attributed dynamically. `freeEventId` is `free_completed`,
so pending-free memory is still included. Static, preexisting, and missing or
damaged stack links remain explicit synthetic leaves. This is active-block
occupancy and does not decompose reserved bytes. Event IDs mark ordering, not
elapsed time. `min_size` and `include_static` change the root denominator.
If a direct query uses `-n`/`max_rows`, inspect its completeness flags: a
truncated set of node rows is not a complete tree. Old text-only databases
must be re-imported; the analyzer does not reconstruct frames from text.

The Python API uses the same template and contracts:

```python
from pathlib import Path
from pt_snap_cli import SnapshotAnalyzer

with SnapshotAnalyzer(db_path=Path("snapshot.pkl.db"), device_id=0) as analyzer:
frames = analyzer.execute_query("event_frames", params={"event_id": 100})
tree = analyzer.execute_query("active_memory_frame_tree_at_event", params={"event_id": 100})
```

## Template Architecture

Query templates are defined in YAML format with:
Expand Down
40 changes: 36 additions & 4 deletions docs/zh/database.md
Original file line number Diff line number Diff line change
Expand Up @@ -312,15 +312,47 @@ CREATE TABLE callstack (

| 属性 | 保证 |
|------|------|
| 唯一性 | 每条不同的调用栈文本只有一行,因此按 `callstackId` 分组等价于按文本分组 |
| 唯一性 | 每组不同的有序结构化帧数组只有一行;显示文本本身不能无损标识调用栈 |
| 作用域 | 数据库内所有设备共享,表名不带设备后缀 |
| 覆盖度 | 导入产生的每个事件都能解析到一行;只有外部生成的数据库才可能出现 `callstackId` 为 `NULL` |

当前 `pt-snap import` 写入的是这种 v2 布局(`import_format_version = 2`)。
当前 `pt-snap import` 保留这种 v2 文本布局,同时增加结构化帧表
(`import_format_version = 3`)。格式 2 数据库的文本布局相同,但没有原始帧记录。
调用栈去重之前生成的数据库在每张 `trace_entry_<device>` 表中内联存储
`callstack` TEXT,没有共享 `callstack` 表(有 metadata 时为
`import_format_version = 1`)。

### 结构化帧与有序数组(格式 3)

`frame` 对原始 `(filename, line, name)` 三元组去重;`callstack_frame`
把调用栈关联到有序的帧 ID 数组:

```sql
CREATE TABLE frame (
id INTEGER PRIMARY KEY,
filename TEXT NOT NULL,
line INTEGER NOT NULL,
name TEXT NOT NULL
);
CREATE TABLE callstack_frame (
callstackId INTEGER NOT NULL,
position INTEGER NOT NULL,
frameId INTEGER NOT NULL
);
CREATE UNIQUE INDEX idx_callstack_frame_position
ON callstack_frame (callstackId, position);
```

两张表在设备之间共享。事件通过 `trace_entry_<device>.callstackId` 关联数组,
按 `position ASC` 读取即可还原快照原始顺序;位置 0 是最内层帧。帧树分析反向
遍历,从最外层调用者开始。递归中的同一帧保留各自位置,即使帧 ID 相同。
空数组没有关联行。结构化字段直接保留冒号、空白、中文和内嵌换行,无需拆解显示文本。

使用 `event_frames` 读取事件有序帧,使用 `active_memory_frame_tree_at_event`
读取各父子节点的占用。详见[调用栈帧内存拆解](querying.md#调用栈帧内存拆解)。
旧文本数据库仍可用于原有查询,新查询需要从原始快照重新导入。导入器检测格式变化
后重建缓存;只读分析不会迁移数据库。

### 调用栈布局兼容

`pt-snap` 可以读取两种布局,且不会改写数据库文件:
Expand All @@ -331,8 +363,8 @@ CREATE TABLE callstack (
| v1(旧版) | `trace_entry_<device>.callstack` TEXT,无 `callstackId`,无共享表 | 直接读该列 |

`event`、`callstack_analysis` 和 `active_memory_callstack_at_event` 保持相同的
模板名、参数和输出契约。查询引擎按识别到的布局选择 v1 或 v2 SQL。其他模板不依赖
这次拆分。
模板名、参数和输出契约。查询引擎按识别到的布局选择 v1 或 v2 SQL。
结构化帧查询额外要求格式 3 的两张帧表,不会从旧显示文本猜测原始帧。

识别过程是只读的:`focus`、`query`、报告和 Python API 都以 SQLite
`mode=ro` 打开数据库。布局不会写入 `.pt-snap/focus.json`。列冲突、设备布局不一致、
Expand Down
49 changes: 49 additions & 0 deletions docs/zh/querying.md
Original file line number Diff line number Diff line change
Expand Up @@ -396,6 +396,55 @@ CLI 和 Python API 的查询结果包含原始 SQLite 值。模板的 `output_sc
`SnapshotAnalyzer.execute_query()` 的结果还包含 `template` 与 `semantics_version`,
便于把行与契约对应起来。

## 调用栈帧内存拆解

使用当前版本导入后,可以读取原始有序帧数组,或拆解指定事件后的存活内存:

```bash
pt-snap query --template-use event_frames --params '{"event_id":100}' --json
pt-snap query --template-use active_memory_frame_tree_at_event --params '{"event_id":100}' --json
pt-snap report memory-tree --event-id 100
pt-snap report memory-tree --event-id 100 --json
pt-snap report memory-tree --event-id 100 --format html > memory-tree.html
```

HTML 文件可直接在本地打开:点击帧放大、重置查看整棵树、搜索文件或函数。
宽度表示存活内存块的字节数,不是 CPU 时间或累计分配量,无需启动服务器。
`--device` 选择设备,`--exclude-static` 排除静态和采集前已存在的内存,
`--min-size` 按单个块大小筛选;`--event-id` 必填。报告不做排名或行数截断。

| 字段 | 含义 |
| --- | --- |
| `node_id`、`parent_id` | 完整调用路径身份及直接父节点;`root` 为总占用 |
| `depth`、`frame_id` | 从调用者开始的层级和结构化帧身份 |
| `size_bytes`、`requested_bytes`、`block_count` | 当前节点及所有后代的占用和块数 |
| `self_bytes`、`self_requested_bytes`、`self_block_count` | 记录的分配栈终止于当前节点的自身占用和块数 |
| `percent_of_total` | 根节点总占用的 0–100 字节百分比;分母在块和类别筛选后、输出行截断前计算 |

每个节点满足“总占用 = 自身占用 + 各直接子节点总占用”。可以对所有节点的
`self_bytes` 求和,或对树的一组完整切面求和;不能跨层累加 `size_bytes`,
否则同一分配会重复计算。记录的调用栈可能止于中间节点,所以中间节点也可能有
自身占用。递归帧以及不同调用者下的相同帧保留为不同路径。

动态内存只统计 `allocEventId <= event_id` 且 `freeEventId` 晚于该事件或未记录
的块。`freeEventId` 对应 `free_completed`,因此待释放内存仍计入。
静态内存、采集前已存在的内存、缺失或损坏调用栈分别作为显式虚拟叶节点。
这表示 active 内存块占用,不是 reserved 内存拆解;事件 ID 是顺序标记,不是
耗时。`min_size` 和 `include_static` 会改变根节点分母。直接查询若设置
`-n`/`max_rows`,需检查完整性字段,截断结果不能当作完整树。
旧文本数据库需要从原始快照重新导入,分析器不会从文本猜测原始帧。

Python API 使用相同模板和契约:

```python
from pathlib import Path
from pt_snap_cli import SnapshotAnalyzer

with SnapshotAnalyzer(db_path=Path("snapshot.pkl.db"), device_id=0) as analyzer:
frames = analyzer.execute_query("event_frames", params={"event_id": 100})
tree = analyzer.execute_query("active_memory_frame_tree_at_event", params={"event_id": 100})
```

## 模板架构

查询模板使用 YAML 格式定义,包含:
Expand Down
45 changes: 45 additions & 0 deletions src/pt_snap_cli/cli_reports.py
Original file line number Diff line number Diff line change
Expand Up @@ -35,10 +35,55 @@
TemplateRenderError,
)
from pt_snap_cli.core.error_codes import DATABASE_NOT_FOUND, INVALID_PARAMETER
from pt_snap_cli.core.errors import PtSnapCoreError
from pt_snap_cli.memory_tree_render import render_memory_tree

report_app = typer.Typer(help="Generate memory analysis reports")


@report_app.command("memory-tree")
def report_memory_tree(
event_id: Annotated[
int, typer.Option("--event-id", help="Analyze live memory after this event")
],
db_path: Annotated[
Path | None, typer.Argument(help="Path to database (optional if configured)")
] = None,
device: Annotated[
int | None, typer.Option("--device", "-d", autocompletion=complete_device_ids)
] = None,
include_static: Annotated[bool, typer.Option("--include-static/--exclude-static")] = True,
min_size: Annotated[int, typer.Option(help="Minimum individual block size in bytes")] = 0,
output_format: Annotated[
Literal["text", "html"],
typer.Option("--format", help="Text breakdown or interactive HTML flamegraph on stdout"),
] = "text",
json_output: Annotated[bool, _json_flag()] = False,
) -> None:
"""Decompose event-time active memory by ordered allocation frames."""
service = ReportService()
try:
if json_output and output_format != "text":
raise ValueError("--json cannot be combined with --format html.")
result = service.memory_tree_report(
event_id=event_id,
db_path=db_path,
device_id=device,
include_static=include_static,
min_size=min_size,
)
if json_output:
typer.echo(json.dumps({"event_id": event_id, **asdict(result)}, indent=2))
else:
typer.echo(render_memory_tree(result, event_id, html=output_format == "html"))
except PtSnapCoreError as exc:
_error_from_exc(exc)
except ValueError as exc:
_error(str(exc), code=INVALID_PARAMETER)
finally:
service.close()


@report_app.command("peak-memory")
def report_peak_memory(
db_path: Annotated[
Expand Down
2 changes: 1 addition & 1 deletion src/pt_snap_cli/context.py
Original file line number Diff line number Diff line change
Expand Up @@ -169,7 +169,7 @@ def _detect_callstack_layout(
)

metadata_version = _import_format_version(cursor)
if metadata_version in (1, 2):
if metadata_version in (1, 2, 3):
expected: CallstackLayout = "v1" if metadata_version == 1 else "v2"
if layout is None:
return None, (
Expand Down
4 changes: 3 additions & 1 deletion src/pt_snap_cli/core/import_metadata.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,9 @@
METADATA_SCHEMA_VERSION = 1
# 2: trace_entry_<device> stores `callstackId` referencing the shared `callstack`
# table instead of an inlined `callstack` text column.
IMPORT_FORMAT_VERSION = 2
# 3: retains the v2 text layout and adds shared structured frame records plus
# original-order callstack_frame references. Old caches must be re-imported.
IMPORT_FORMAT_VERSION = 3
IMPORTER_NAME = "pt-snap-cli"
HASH_CHUNK_SIZE = 1024 * 1024

Expand Down
Loading
Loading