Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
61 changes: 54 additions & 7 deletions docs/en/database.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,8 +48,11 @@ pt-snap metadata snapshot.pkl.db --json
pt-snap import snapshot.pkl --force
```

Legacy or externally generated compatible databases without this table remain queryable. Their
metadata status is reported as unavailable, and a later import rebuilds them once before reuse.
Legacy or externally generated compatible databases without this table remain queryable for
templates that do not require the deduplicated callstack schema. Callstack templates require a
database produced by the current importer; otherwise the query reports that the snapshot must be
re-imported. Their metadata status is reported as unavailable, and a later import rebuilds them
once before reuse.

### Query workflow

Expand All @@ -75,11 +78,12 @@ the supported query surface. See [Querying](querying.md) for the complete workfl
| Table | Description | Records (example) |
|-------|-------------|-------------------|
| `dictionary` | Enum mapping dictionary | - |
| `callstack` | Deduplicated callstack text shared by all devices | 4,369 |
| `trace_entry_0` | Device 0 event trace table | 8,094 |
| `block_0` | Device 0 memory block table | - |
| `pt_snap_metadata` | First-party import provenance and cache metadata | 1 |

> **Naming convention**: For multi-device scenarios, table names are suffixed with the device ID, e.g., `trace_entry_1` and `block_1` for device 1.
> **Naming convention**: For multi-device scenarios, table names are suffixed with the device ID, e.g., `trace_entry_1` and `block_1` for device 1. The `callstack` table carries no device suffix because callstacks are shared across devices.

---

Expand All @@ -101,7 +105,7 @@ CREATE TABLE trace_entry_0 (
`allocated` INTEGER,
`active` INTEGER,
`reserved` INTEGER,
`callstack` TEXT
`callstackId` INTEGER
);
```

Expand All @@ -117,7 +121,7 @@ CREATE TABLE trace_entry_0 (
| allocated | INTEGER | — | Total allocated bytes | `136426496` |
| active | INTEGER | — | Total active bytes | `136426496` |
| reserved | INTEGER | — | Memory pool reserved bytes | `155189248` |
| callstack | TEXT | NULL | Callstack information (multi-line text) | See example |
| callstackId | INTEGER | NULL | Reference to `callstack.id`; join to read the text | `12` |

#### Action Type Codes

Expand All @@ -144,13 +148,24 @@ CREATE TABLE trace_entry_0 (
```
id=1, action=4, address=20697535234048, size=41943040, stream=1276474240
allocated=136426496, active=136426496, reserved=155189248
callstack:
callstackId=12 -> callstack.callstack:
/home/liuyekang/dev/projects/test/memory_leaks_demo.py:60 <module>
/home/liuyekang/dev/projects/test/memory_leaks_demo.py:34 main
/home/liuyekang/dev/projects/test/memory_leaks_demo.py:24 train
/home/liuyekang/dev/projects/test/memory_leaks_demo.py:12 train_one_step
```

#### Reading Callstack Text

Join `callstack` to resolve `callstackId`:

```sql
SELECT t.id, t.size, c.callstack
FROM trace_entry_0 t
LEFT JOIN callstack c ON c.id = t.callstackId
WHERE t.id = 1;
```

---

### 2. block_{device} — Memory Block Table
Expand Down Expand Up @@ -264,7 +279,39 @@ table=trace_entry_0, column=action, key=4, value=alloc
table=block_0, column=state, key=1, value=active_allocated
```

### 4. pt_snap_metadata — Import Metadata Table
---

### 4. callstack — Deduplicated Callstack Text

Snapshots repeat a small number of callstacks across a very large number of
events, so the text is stored once per distinct callstack and referenced by
`trace_entry_<device>.callstackId`.

#### Schema

```sql
CREATE TABLE callstack (
`id` INTEGER PRIMARY KEY,
`callstack` TEXT
);
```

#### Columns

| Column | Type | Description |
|--------|------|-------------|
| id | INTEGER | Callstack ID referenced by `trace_entry_<device>.callstackId` |
| callstack | TEXT | Callstack text, outermost frame first, one `file:line name` per line |

#### Properties

| Property | Guarantee |
|----------|-----------|
| Uniqueness | One row per distinct callstack text, so grouping by `callstackId` equals grouping by text |
| Scope | Shared by every device in the database; the table has no device suffix |
| Coverage | Every event produced by import resolves to a row; `callstackId` is `NULL` only in externally generated databases |

### 5. pt_snap_metadata — Import Metadata Table

First-party imports create one metadata row for provenance and cache
compatibility checks:
Expand Down
58 changes: 51 additions & 7 deletions docs/zh/database.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,8 +44,9 @@ pt-snap metadata snapshot.pkl.db --json
pt-snap import snapshot.pkl --force
```

旧版或外部生成且结构兼容的 DB 仍可正常查询。若没有 metadata,查询会返回 unavailable,
下一次执行对应 pickle 导入时会重建一次。
旧版或外部生成且结构兼容的 DB,仍可用于不依赖去重调用栈结构的查询。调用栈相关模板要求
数据库由当前导入器生成;否则会提示需要重新导入快照。若没有 metadata,查询会返回
unavailable,下一次执行对应 pickle 导入时会重建一次。

### 查询工作流

Expand All @@ -70,11 +71,12 @@ pt-snap query --template-use block --params '{"min_size": 1048576}'
| 表名 | 说明 | 记录数(示例) |
|------|------|---------------|
| `dictionary` | 字典映射表 | - |
| `callstack` | 去重后的调用栈文本,所有设备共享 | 4,369 |
| `trace_entry_0` | 设备 0 的事件跟踪表 | 8,094 |
| `block_0` | 设备 0 的内存块表 | - |
| `pt_snap_metadata` | 第一方导入来源与缓存 metadata | 1 |

> **命名规则**: 多设备场景下,表名后缀为设备 ID,如 `trace_entry_1`、`block_1` 表示设备 1 的数据。
> **命名规则**: 多设备场景下,表名后缀为设备 ID,如 `trace_entry_1`、`block_1` 表示设备 1 的数据。`callstack` 表没有设备后缀,因为调用栈在设备之间共享。

---

Expand All @@ -96,7 +98,7 @@ CREATE TABLE trace_entry_0 (
`allocated` INTEGER,
`active` INTEGER,
`reserved` INTEGER,
`callstack` TEXT
`callstackId` INTEGER
);
```

Expand All @@ -112,7 +114,7 @@ CREATE TABLE trace_entry_0 (
| allocated | INTEGER | — | 当前已分配总量 | `136426496` |
| active | INTEGER | — | 当前活跃总量 | `136426496` |
| reserved | INTEGER | — | 内存池保留总量 | `155189248` |
| callstack | TEXT | NULL | 调用栈,多行文本格式 | 见示例 |
| callstackId | INTEGER | NULL | 指向 `callstack.id`,需 join 才能取到文本 | `12` |

#### 动作类型编码(action 字段)

Expand All @@ -139,13 +141,24 @@ CREATE TABLE trace_entry_0 (
```
id=1, action=4, address=20697535234048, size=41943040, stream=1276474240
allocated=136426496, active=136426496, reserved=155189248
callstack:
callstackId=12 -> callstack.callstack:
/home/liuyekang/dev/projects/test/memory_leaks_demo.py:60 <module>
/home/liuyekang/dev/projects/test/memory_leaks_demo.py:34 main
/home/liuyekang/dev/projects/test/memory_leaks_demo.py:24 train
/home/liuyekang/dev/projects/test/memory_leaks_demo.py:12 train_one_step
```

#### 读取调用栈文本

join `callstack` 表解析 `callstackId`:

```sql
SELECT t.id, t.size, c.callstack
FROM trace_entry_0 t
LEFT JOIN callstack c ON c.id = t.callstackId
WHERE t.id = 1;
```

---

### 2. block_{device} — 内存块表
Expand Down Expand Up @@ -257,7 +270,38 @@ table=trace_entry_0, column=action, key=4, value=alloc
table=block_0, column=state, key=1, value=active_allocated
```

### 4. pt_snap_metadata — 导入 Metadata 表
---

### 4. callstack — 去重调用栈表

快照中少量调用栈会在极多事件上重复出现,因此文本按不同调用栈只存一份,由
`trace_entry_<device>.callstackId` 引用。

#### Schema

```sql
CREATE TABLE callstack (
`id` INTEGER PRIMARY KEY,
`callstack` TEXT
);
```

#### 字段说明

| 字段名 | 类型 | 说明 |
|--------|------|------|
| id | INTEGER | 调用栈 ID,被 `trace_entry_<device>.callstackId` 引用 |
| callstack | TEXT | 调用栈文本,最外层帧在前,每行一个 `文件:行号 函数名` |

#### 表属性

| 属性 | 保证 |
|------|------|
| 唯一性 | 每条不同的调用栈文本只有一行,因此按 `callstackId` 分组等价于按文本分组 |
| 作用域 | 数据库内所有设备共享,表名不带设备后缀 |
| 覆盖度 | 导入产生的每个事件都能解析到一行;只有外部生成的数据库才可能出现 `callstackId` 为 `NULL` |

### 5. pt_snap_metadata — 导入 Metadata 表

第一方导入会创建一条 metadata 记录,用于来源追踪和缓存兼容性检查:

Expand Down
4 changes: 3 additions & 1 deletion src/pt_snap_cli/core/import_metadata.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,9 @@

METADATA_TABLE = "pt_snap_metadata"
METADATA_SCHEMA_VERSION = 1
IMPORT_FORMAT_VERSION = 1
# 2: trace_entry_<device> stores `callstackId` referencing the shared `callstack`
# table instead of an inlined `callstack` text column.
IMPORT_FORMAT_VERSION = 2
IMPORTER_NAME = "pt-snap-cli"
HASH_CHUNK_SIZE = 1024 * 1024

Expand Down
6 changes: 4 additions & 2 deletions src/pt_snap_cli/core/split_service.py
Original file line number Diff line number Diff line change
Expand Up @@ -145,7 +145,9 @@ def _slice_device(
max_entries=max_entries if max_entries is not None else event_count,
dump_type="pkl",
)
_, replayed = replay_snapshot(representation, device, hooker=hooker)
# Slices serialize the original frame dictionaries, so eager Frame
# objects would be rebuilt only to be discarded.
_, replayed = replay_snapshot(representation, device, hooker=hooker, _raw_frames=True)

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[低] _raw_frames=True 使 torch-npu workspace 块在切片中丢失帧

simulate.py:124-131 的 _adapt_workspace_snapshot 构造 Block 时只传 frames=existed_seg.frames——raw 模式下它是 [](entities.py:241-242)——且未传播 _raw_frames,于是 Block.to_dict() 序列化为 "frames": [](entities.py:196-200)。PR 之前 eager 路径经 Frame._origin 能回传真实帧。该适配块经 slice_dump/hooker.py:99-101 深拷贝进入切片文件。

不影响回放有效性与 DB 导入(block2record 无帧列),也不崩溃,但 npu workspace 快照(首事件为 workspace_snapshot)的切片相对变更前静默丢数据。其余所有构造点均正确传播 _raw_frames(Block.build_from_event、Segment.build_from_event、snapshot_mutator.py:267-296),此处是唯一遗漏。

修复:Block 构造补传 _raw_frames=existed_seg._raw_frames;现有 test_workspace_marker_splits_and_replays 用的正是 "frames": [],无法捕获该问题,需补一个带非空 workspace segment 帧的 split 测试。

except Exception as exc:
self._fail("load/engine", source, f"device {device}: {exc}", exc)
if not replayed:
Expand Down Expand Up @@ -174,7 +176,7 @@ def _slice_device(
or not traces[device]
):
raise ValueError(f"generated slice has no trace entries for device {device}")
_, replayed = replay_snapshot(generated, device)
_, replayed = replay_snapshot(generated, device, _raw_frames=True)
if not replayed:
raise ValueError("replay returned false")
except Exception as exc:
Expand Down
17 changes: 17 additions & 0 deletions src/pt_snap_cli/query/executor.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
from __future__ import annotations

import re
import sqlite3
import warnings
from pathlib import Path
from typing import Any
Expand Down Expand Up @@ -101,6 +102,18 @@ def _append_limit(self, sql: str, limit: int) -> str:
trimmed = trimmed[:-1].rstrip()
return f"{trimmed} LIMIT {int(limit)}"

@staticmethod
def _execution_error(error: sqlite3.OperationalError) -> QueryExecutionError:
detail = str(error)
if "no such table: callstack" in detail or re.search(
r"no such column: (?:[\w]+\.)?callstackId", detail
):
return QueryExecutionError(
"Query execution failed: database uses the legacy inline-callstack layout; "
"re-import the snapshot to use callstack queries"
)
return QueryExecutionError(f"Query execution failed: {error}")

def _load_templates(self) -> None:
"""Load all YAML templates from template directory."""
if not self._template_dir:
Expand Down Expand Up @@ -199,6 +212,8 @@ def execute(
cursor.execute(sql)
columns = [desc[0] for desc in cursor.description]
return [dict(zip(columns, row, strict=True)) for row in cursor.fetchall()]
except sqlite3.OperationalError as e:
raise self._execution_error(e) from e
except Exception as e:
raise QueryExecutionError(f"Query execution failed: {e}") from e

Expand All @@ -212,6 +227,8 @@ def count(self, sql: str) -> int:
cursor.execute(count_sql)
row = cursor.fetchone()
return int(row[0]) if row is not None else 0
except sqlite3.OperationalError as e:
raise self._execution_error(e) from e
except Exception as e:
raise QueryExecutionError(f"Query execution failed: {e}") from e

Expand Down
15 changes: 14 additions & 1 deletion src/pt_snap_cli/query/templates/basic/event.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,20 @@ queries:
required: false
description: Number of rows to skip
query: |
SELECT * FROM {{ device_trace_table }}
SELECT
id,
action,
address,
size,
stream,
allocated,
active,
reserved,
(
SELECT callstack FROM callstack

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[低] 旧布局数据库上此模板硬失败:no such table: callstack

用真实管线(Context + QueryExecutor)在 v1 布局(内联 callstack TEXT)数据库上实测复现:QueryExecutionError: Query execution failed: no such table: callstack。此处此前是 SELECT *,与布局无关;callstack_analysis 与 active_memory_callstack_at_event 在 v1 库上同样失败(no such column: callstackId)。

问题链路:Context 仍只校验 dictionary 表(context.py:69-80),数据库能正常打开,错误推迟到查询时才以裸 SQLite 错误暴露、无重导入提示;而 docs/en/database.md:51-52(及 zh 镜像)仍声称遗留/外部数据库「remain queryable」。第一方缓存由 IMPORT_FORMAT_VERSION 1→2 触发重建保护,残留影响面是仅持有旧 .db(无源 pickle)的用户与外部/遗留数据库。目前没有任何测试对旧布局 fixture 执行涉及调用栈的模板(旧布局套件只用 leak_detection/required_query/size_query 一类)。

建议:渲染前探测缺列/缺表时给出友好的 schema 不匹配提示(引导重新导入)或做兼容回退;同步修订 docs 中该句;补一个锁定该行为的测试。

WHERE callstack.id = {{ device_trace_table }}.callstackId
) AS callstack
FROM {{ device_trace_table }}
WHERE 1=1
{% if id is not none %}AND id = {{ id }}{% endif %}
{% if min_id is not none %}AND id >= {{ min_id }}{% endif %}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ queries:
CASE
WHEN b.category = 'static' THEN '[static] allocEventId=-1, freeEventId=-1'
WHEN b.category = 'preexisting_live_at_event' THEN '[preexisting live] allocEventId=-1'
ELSE COALESCE(t.callstack, '[missing callstack]')
ELSE COALESCE(cs.callstack, '[missing callstack]')
END AS callstack,
b.category,
COUNT(*) AS block_count,
Expand All @@ -59,6 +59,8 @@ queries:
FROM active_blocks b
LEFT JOIN {{ device_trace_table }} t
ON b.allocEventId = t.id
LEFT JOIN callstack cs
ON t.callstackId = cs.id
GROUP BY callstack, b.category
),
ranked_dynamic AS (
Expand Down
29 changes: 19 additions & 10 deletions src/pt_snap_cli/query/templates/statistical/callstack_analysis.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -24,16 +24,25 @@ queries:
description: Maximum rows to return (-1 for unlimited). Mirrored from the caller-side ``max_rows`` for SQL pushdown.
query: |
SELECT
callstack,
COUNT(*) as alloc_count,
SUM(size) as total_size,
AVG(size) as avg_size,
MAX(size) as max_size
FROM {{ device_trace_table }}
WHERE callstack IS NOT NULL
GROUP BY callstack
HAVING alloc_count >= {{ min_count }}
AND total_size >= {{ min_size }}
COALESCE(c.callstack, '[missing callstack]') AS callstack,
grouped.alloc_count AS alloc_count,
grouped.total_size AS total_size,
grouped.avg_size AS avg_size,
grouped.max_size AS max_size
FROM (
SELECT
callstackId,
COUNT(*) as alloc_count,
SUM(size) as total_size,
AVG(size) as avg_size,
MAX(size) as max_size
FROM {{ device_trace_table }}
WHERE callstackId IS NOT NULL
GROUP BY callstackId
HAVING alloc_count >= {{ min_count }}
AND total_size >= {{ min_size }}
) AS grouped
LEFT JOIN callstack c ON c.id = grouped.callstackId

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[提示] 悬空分组经结果映射后显示为字符串 "None"

LEFT JOIN 修复后悬空 callstackId 分组以 c.callstack = NULL 呈现,但经 ResultMapper(mapper.py:69,str 转换器 str(None))后会显示为字符串 "None"。兄弟模板 active_memory_callstack_at_event 用 COALESCE(cs.callstack, '[missing callstack]')。第一方导入库不可达此场景(覆盖性有测试保证),仅为一贸性对齐建议:COALESCE(c.callstack, '[missing callstack]') AS callstack。

ORDER BY total_size DESC, callstack ASC
LIMIT {{ limit|int }}
output_schema:
Expand Down
Loading
Loading