From a9d7b18b933cefcaaf62ae6d690453f418151faa Mon Sep 17 00:00:00 2001 From: codeasier Date: Sat, 18 Apr 2026 17:13:52 +0800 Subject: [PATCH] docs: supplement SnapshotDB overview and block field constraints - Add replay explanation in overview section - Clarify action=7 as NPU-specific workspace snapshot - Add allocEventId and freeEventId constraint sections Closes #33 --- docs/en/database.md | 20 ++++++++++++++++++-- docs/zh/database.md | 19 ++++++++++++++++++- 2 files changed, 36 insertions(+), 3 deletions(-) diff --git a/docs/en/database.md b/docs/en/database.md index 02f2c43..ed4430d 100644 --- a/docs/en/database.md +++ b/docs/en/database.md @@ -4,7 +4,9 @@ ## Overview -SnapshotDB is the SQLite database format for persisting PyTorch memory profiling data. It supports multi-device (multi-GPU) snapshot storage and querying. +SnapshotDB is the SQLite database format for persisting PyTorch memory profiling data. It supports multi-device (multi-GPU) snapshot storage and querying. Notably, it does more than simply converting raw pickle snapshot data into SQLite — it performs a complete "replay" of the raw memory snapshot data and records additional information during the replay, such as: +1. The total memory pool size and total allocated block size after any event; +2. The complete lifecycle of all memory blocks during the collection period (which event allocated them, which event freed them). **Example database file**: `snapshot_expandable.pkl.db` @@ -71,7 +73,7 @@ CREATE TABLE trace_entry_0 ( | 4 | alloc | Memory allocation event | | 5 | free_requested | Free request | | 6 | free_completed | Free completed | -| 7 | workspace_snapshot | Workspace snapshot | +| 7 | workspace_snapshot | NPU-specific workspace memory pool snapshot | #### ID Constraints @@ -140,6 +142,20 @@ CREATE TABLE block_0 ( | `id >= 0` | `block.id` matches `allocEventId`, pointing to the same-ID allocation event in `trace_entry` | | `id < 0` | The block was already allocated before snapshot collection began. Only its existence and initial state are known from pickle data; allocation time is unknown. The negative value itself carries no semantic meaning beyond uniqueness | +#### allocEventId Constraints + +| Condition | Constraint | +|-----------|------------| +| `allocEventId >= 0` | `block.id` matches `allocEventId`, pointing to the same-ID allocation event in `trace_entry` | +| `allocEventId == -1` | The allocation event for this block was not captured during snapshot collection (the block was already allocated before collection started) | + +#### freeEventId Constraints + +| Condition | Constraint | +|-----------|------------| +| `freeEventId >= 0` | `block.id` matches `freeEventId`, pointing to the same-ID free event in `trace_entry` | +| `freeEventId == -1` | The free completion event for this block was not captured during snapshot collection (the block was not freed when collection ended) | + #### State Usage `block.state` is only meaningful when `block.id` is negative; otherwise it has no practical use. diff --git a/docs/zh/database.md b/docs/zh/database.md index e1691a0..9eb5656 100644 --- a/docs/zh/database.md +++ b/docs/zh/database.md @@ -5,6 +5,9 @@ ## 概述 SnapshotDB 是内存快照数据的 SQLite 数据库存储格式,用于持久化内存分配追踪信息。支持多设备(多 GPU)快照数据的存储和查询。 +需要说明的是,它不仅仅是将内存快照原始pickle数据转成了SQLite数据库,而是针对原始内存快照数据进行了一次完整的“回放”,并记录了回放过程中的额外信息,如 +1. 任一事件发生后的内存池总大小、总分配内存块大小; +2. 采集周期中,所有的内存块的完整生命周期(在哪个事件申请、哪个事件释放) **数据库示例文件**: `snapshot_expandable.pkl.db` @@ -71,7 +74,7 @@ CREATE TABLE trace_entry_0 ( | 4 | alloc | 内存分配事件 | | 5 | free_requested | 释放请求 | | 6 | free_completed | 释放完成 | -| 7 | workspace_snapshot | 工作区快照 | +| 7 | workspace_snapshot | NPU特有的workspace内存池快照 | #### id 字段约束 @@ -140,6 +143,20 @@ CREATE TABLE block_0 ( | `id >= 0` | `block.id` 与 `allocEventId` 一致,共同指向 `trace_entry` 中相同 ID 的分配事件 | | `id < 0` | 仅代表通过原始 pickle 数据中的 Segment 信息得知该内存块在采集开始时已分配,无从得知分配时间。负数值本身无实际含义,仅用于唯一标识 | +#### allocEventId约束 + +| 条件 | 约束 | +|------|------| +| `allocEventId >= 0` | `block.id` 与 `allocEventId` 一致,共同指向 `trace_entry` 中相同 ID 的分配事件 | +| `allocEventId == -1` | 代表在内存快照采集期间,未采集到该内存块的申请事件(在开始采集之前该内存块就已经申请分配完成) | + +#### freeEventId约束 + +| 条件 | 约束 | +|------|------| +| `freeEventId >= 0` | `block.id` 与 `allocEventId` 一致,共同指向 `trace_entry` 中相同 ID 的分配事件 | +| `freeEventId == -1` | 代表在内存快照采集期间,未采集到该内存块的释放完成事件(在结束采集时该内存块并未释放) | + #### state 约束 `block.state` 仅在 `block.id` 为负数时才有实际意义,否则无实际用途。